[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Übung / Exercise 8 Discrete-Time Convolution

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import MaxNLocator
```

## Task:

Calculate and plot the convolution result
\begin{equation}
y[k] = x[k] * h[k]
\end{equation}
of the two finite length, discrete-time signals
\begin{align}
&x[k] = [1,2,3,4] \quad &\mathrm{for} \quad &k = [2,3,4,5]\\
&h[k] = [8,0,7,6,5] \quad &\mathrm{for} \quad &k = [-2,-1,0,1,2]
\end{align}
depicted in the plots below. These signals shall be zero outside the visible range.

```python
plt.figure(figsize=(9, 3))

kx = (2, 3, 4, 5)
x = (1, 2, 3, 4)
ax = plt.subplot(1, 2, 1)
plt.stem(kx, x, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.grid(True)
plt.xlabel(r'$k$')
plt.ylabel(r'$x[k]$')
plt.ylim((-1, 9))
plt.yticks(np.arange(0, 9), ['0', '1', '2', '3', '4', '', '', '', ''])

kh = (-2, -1, 0, 1, 2)
h = (8, 0, 7, 6, 5)
ax = plt.subplot(1, 2, 2)
plt.stem(kh, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.grid(True)
plt.xlabel(r'$k$')
plt.ylabel(r'$h[k]$')
plt.ylim((-1, 9))
plt.yticks(np.arange(0, 9), ['0', '', '', '', '', '5', '6', '7', '8'])
```

## Solution

We elaborate three different ways of calculating the result and validate these with the result computed by the `numpy` implementation.

A highly recommended reference not only for this topic, but generally on digital signal processing basics, is

[Lyo11] Richard G. Lyons (2011): "*Understanding Digital Signal Processing*", Prentice Hall, Upper Saddle River, NJ (USA), 3rd ed.

### Numpy Result

To get an expectation on the result, we check the solution using a computer.
For that, numpy's `convolve` function can be used.
Normally, you would need to verify a used algorithm beforehand.
In this case it is fair to assume, that `convolve` is properly implemented and yields the correct convolution result.
This is depicted below.

```python
y = np.convolve(x, h)

plt.figure(figsize=(9, 3))
ax = plt.subplot(1, 1, 1)
plt.stem(y, use_line_collection=True, linefmt='C0:',
         markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.ylabel(r'$y$')
plt.title(['y = x * h =', y])
plt.grid(True)
```

`convolve` returns a sequence of numbers as the convolution result, but we do not know yet, how $y$ is aligned with respect to $k$.
The `convolve` function does not care and actually does not need to know about the time alignment.
The other way around: both sequences are treated as they would both start at $k=0$.
The physical link of the sequence towards a discrete-time index is something we made up.
Thus, it is our task to figure this out.
For the moment we must leave the x-axis in the plot unlabeled.

### Convolution

The discrete-time convolution is defined as
\begin{equation}
y[k] = \sum\limits_{\kappa = -\infty}^{+\infty} x[k -\kappa] \cdot h[\kappa] = \sum\limits_{\kappa = -\infty}^{+\infty} x[\kappa] \cdot h[k -\kappa]
\end{equation}
with $k, \kappa\in\mathbb{Z}$.
We might either take $x$ as the signal to be shifted and mirrored ($x[k -\kappa]$) or take $h$ as the signal to be shifted and mirrored ($h[k -\kappa]$).
The result is identical and the choice is a matter of taste when performing manual calculations.
For the given example, mirroring and shifting the signal $x$ seems to be a bit more conclusive.
In pratice, the first version is meaningful, when we consider the impulse response $h[k]$ given and fix, and input signal is 'coming in' to the system, which then needs to delayed by all possible $\kappa$.

Since we deal with finite length sequences (in fact: if we interpret $h[k]$ as the impulse response of an LTI system, this then constitutes a finite impulse response (FIR) filter), the summation is not required to be performed from $-\infty \leq \kappa \leq \infty$, but rather for a limited range that depends on the actual $k$.

This is elaborated in the following solution, where version I of III potential approaches is presented.

### Solution I: Analytical Method based on the Convolution Equation

This version might be the most straightforward, but potentially least enlightening.
See Ch. 5.9.1 in [Lyo11] for another example using the same approach.
We use the convolution equation
\begin{equation}
y[k] = \sum\limits_{\kappa = -\infty}^{+\infty} x[k -\kappa] \cdot h[\kappa]
\end{equation}
directly to solve for the two sequences in the picture below.
Note that as reference we now depict $x[\nu]$ and $h[\nu]$ with $\nu\in\mathbb{Z}$ to be able to grab values out $x$ for $\nu=k-\kappa$ and out of $h$ for $\nu=\kappa$.

```python
plt.figure(figsize=(9, 3))

ax = plt.subplot(1, 2, 1)
plt.stem(kx, x, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.grid(True)
plt.xlabel(r'$\nu$')
plt.ylabel(r'$x[\nu]$')
plt.ylim((-1, 9))
plt.yticks(np.arange(0, 9), ['0', '1', '2', '3', '4', '', '', '', ''])

ax = plt.subplot(1, 2, 2)
plt.stem(kh, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.grid(True)
plt.xlabel(r'$\nu$')
plt.ylabel(r'$h[\nu]$')
plt.ylim((-1, 9))
plt.yticks(np.arange(0, 9), ['0', '', '', '', '', '5', '6', '7', '8'])
```

The required summation range for $\kappa$ depends on the actual $k$.
Thus in the first instance, for a specific $k$ it is helpful to indicate the minimum $\kappa$ and the maximum $\kappa$, such that all non-zero entries of $x[\nu=k-\kappa]$ and $h[\nu=\kappa]$ are covered.

For $k=0$ this yields
\begin{align}
y[0] = \sum\limits_{\kappa = -5}^{+2} x[0 -\kappa] \cdot h[\kappa] =
&x[0 -(-5)] \cdot h[-5]\\
+&x[0 -(-4)] \cdot h[-4]\\
+&x[0 -(-3)] \cdot h[-3]\\
+&x[0 -(-2)] \cdot h[-2]\\
+&x[0 -(-1)] \cdot h[-1]\\
+&x[0 -(0)] \cdot h[0]\\
+&x[0 -(1)] \cdot h[1]\\
+&x[0 -(2)] \cdot h[2]
\end{align}
since $\kappa=-5$ and $k=0$ refers to $x[\nu=k-\kappa = 5]$ and $\kappa=+2$ refers to $h[\nu=\kappa=2]$ as the most 'right' non-zero entries.
Both choices also cover the most 'left' non-zero entries $x[2]$ and $h[-2]$.

In the second step, we see that actually only for $\kappa=-2$, both $x$ and $h$ exhibit non-zero contributions, thus the summation reduces to
\begin{align}
y[0] = \sum\limits_{\kappa = -2}^{-2} x[0 -\kappa] \cdot h[\kappa] = x[0 -(-2)] \cdot h[-2] = 1 \cdot 8 = 8.
\end{align}

For $k=1$ we take the same considerations.
In step one, the sum
\begin{align}
y[1] = \sum\limits_{\kappa = -4}^{+2} x[1 -\kappa] \cdot h[\kappa] =
&x[1 -(-4)] \cdot h[-4]\\
+&x[1 -(-3)] \cdot h[-3]\\
+&x[1 -(-2)] \cdot h[-2]\\
+&x[1 -(-1)] \cdot h[-1]\\
+&x[1 -(0)] \cdot h[0]\\
+&x[1 -(1)] \cdot h[1]\\
+&x[1 -(2)] \cdot h[2]
\end{align}
covers all potential non-zero entries in $x[\mu]$ and $h[\mu]$ at according positions $\mu$. Then, we can realize that actually only
\begin{align}
y[1] = \sum\limits_{\kappa = -2}^{-1} x[1 -\kappa] \cdot h[\kappa] =
&x[1 -(-2)] \cdot h[-2]\\
+&x[1 -(-1)] \cdot h[-1] = 2\cdot 8 + 1\cdot 0 = 16
\end{align}
contribute in the summation.
Note that actually $h[\nu=\kappa=-1]=0$ does not contribute, but we should treat it for clarity, because this zero is surrounded by other non-zero samples in the FIR $h$.

For $k=2$ we get
\begin{align}
y[2] = \sum\limits_{\kappa = -3}^{+2} x[2 -\kappa] \cdot h[\kappa] =
&x[2 -(-3)] \cdot h[-3]\\
+&x[2 -(-2)] \cdot h[-2]\\
+&x[2 -(-1)] \cdot h[-1]\\
+&x[2 -(0)] \cdot h[0]\\
+&x[2 -(1)] \cdot h[1]\\
+&x[2 -(2)] \cdot h[2],
\end{align}
where summation can be reduced to the non-zero contributions
\begin{align}
y[2] = \sum\limits_{\kappa = -2}^{0} x[2 -\kappa] \cdot h[\kappa] =
&x[2 -(-2)] \cdot h[-2]\\
+&x[2 -(-1)] \cdot h[-1]\\
+&x[2 -(0)] \cdot h[0] = 3\cdot 8 + 2\cdot 0 + 1\cdot 7 = 31.
\end{align}

For $k=3$ we get
\begin{align}
y[3] = \sum\limits_{\kappa = -2}^{+2} x[3 -\kappa] \cdot h[\kappa] =
&x[3 -(-2)] \cdot h[-2]\\
+&x[3 -(-1)] \cdot h[-1]\\
+&x[3 -(0)] \cdot h[0]\\
+&x[3 -(1)] \cdot h[1]\\
+&x[3 -(2)] \cdot h[2],
\end{align}
where summation can be reduced to the non-zero contributions
\begin{align}
y[3] = \sum\limits_{\kappa = -2}^{+1} x[3 -\kappa] \cdot h[\kappa] =
&x[3 -(-2)] \cdot h[-2]\\
+&x[3 -(-1)] \cdot h[-1]\\
+&x[3 -(0)] \cdot h[0]\\
+&x[3 -(1)] \cdot h[1] = 4\cdot 8 + 3\cdot 0 + 2\cdot 7 + 1\cdot 6 = 52.
\end{align}

For $k=4$ we get
\begin{align}
y[4] = \sum\limits_{\kappa = -2}^{+2} x[4 -\kappa] \cdot h[\kappa] =
&x[4 -(-2)] \cdot h[-2]\\
+&x[4 -(-1)] \cdot h[-1]\\
+&x[4 -(0)] \cdot h[0]\\
+&x[4 -(1)] \cdot h[1]\\
+&x[4 -(2)] \cdot h[2],
\end{align}
where summation can be reduced to the non-zero contributions
\begin{align}
y[4] = \sum\limits_{\kappa = -1}^{+2} x[4 -\kappa] \cdot h[\kappa] =
&x[4 -(-1)] \cdot h[-1]\\
+&x[4 -(0)] \cdot h[0]\\
+&x[4 -(1)] \cdot h[1]\\
+&x[4 -(2)] \cdot h[2] = 4\cdot 0 + 3\cdot 7 + 2\cdot 6 + 1\cdot 5 = 38.
\end{align}

For $k=5$ we get
\begin{align}
y[5] = \sum\limits_{\kappa = -2}^{+3} x[5 -\kappa] \cdot h[\kappa] =
&x[5 -(-2)] \cdot h[-2]\\
+&x[5 -(-1)] \cdot h[-1]\\
+&x[5 -(0)] \cdot h[0]\\
+&x[5 -(1)] \cdot h[1]\\
+&x[5 -(2)] \cdot h[2]\\
+&x[5 -(3)] \cdot h[3],
\end{align}
where summation can be reduced to the non-zero contributions
\begin{align}
y[5] = \sum\limits_{\kappa = 0}^{+2} x[5 -\kappa] \cdot h[\kappa] =
&x[5 -(0)] \cdot h[0]\\
+&x[5 -(1)] \cdot h[1]\\
+&x[5 -(2)] \cdot h[2] = 4\cdot 7 + 3\cdot 6 + 2\cdot 5 = 56.
\end{align}

For $k=6$ we get
\begin{align}
y[6] = \sum\limits_{\kappa = -2}^{+4} x[6 -\kappa] \cdot h[\kappa] =
&x[6 -(-2)] \cdot h[-2]\\
+&x[6 -(-1)] \cdot h[-1]\\
+&x[6 -(0)] \cdot h[0]\\
+&x[6 -(1)] \cdot h[1]\\
+&x[6 -(2)] \cdot h[2]\\
+&x[6 -(3)] \cdot h[3]\\
+&x[6 -(4)] \cdot h[4],
\end{align}
where summation can be reduced to the non-zero contributions
\begin{align}
y[6] = \sum\limits_{\kappa = +1}^{+2} x[6 -\kappa] \cdot h[\kappa] =
&x[6 -(1)] \cdot h[1]\\
+&x[6 -(2)] \cdot h[2] = 4\cdot 6 + 3\cdot 5 = 39.
\end{align}

For $k=7$ we get
\begin{align}
y[7] = \sum\limits_{\kappa = -2}^{+5} x[7 -\kappa] \cdot h[\kappa] =
&x[7 -(-2)] \cdot h[-2]\\
+&x[7 -(-1)] \cdot h[-1]\\
+&x[7 -(0)] \cdot h[0]\\
+&x[7 -(1)] \cdot h[1]\\
+&x[7 -(2)] \cdot h[2]\\
+&x[7 -(3)] \cdot h[3]\\
+&x[7 -(4)] \cdot h[4]\\
+&x[7 -(5)] \cdot h[5],
\end{align}
where summation can be reduced to the non-zero contribution
\begin{align}
y[7] = \sum\limits_{\kappa = +2}^{+2} x[7 -\kappa] \cdot h[\kappa] =
+&x[7 -(2)] \cdot h[2] = 4\cdot 5 = 20.
\end{align}

You might verify that for all other $k$ the convolution result $y[k]=0$.
Thus, we now know the correct time alignment $0\leq k \leq 7$ of the sequence that was computed by `numpy`, which we just verified by our manual calculus.
The plot of the convolution result with proper labeling is depicted below.
Note that starting the manual calculus at $k=0$ that just resulted in the first non-zero entry of $y$ is a coincidence.
For other $x$ and $h$ the time alignment must be properly checked with the handling discussed above.

```python
y = np.convolve(x, h)
N = len(x)+len(h)-1  # length of y
M = len(y)  # N == M!
k = np.arange(0, M)  # time alignment valid for the specific example
plt.figure(figsize=(10, 3))
ax = plt.subplot(1, 1, 1)
plt.stem(k, y, basefmt='None')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.ylabel(r'$y[k]$')
plt.xlabel(r'$k$')
plt.title(['y[k] = x[k]*h[k] =', y])
plt.grid(True)
```

### Solution II: ASCII-Graphics' Method to Visualise Mirroring and Shifting

Virtually every textbook on discrete-time signal processing treats the convolution of sequences in an illustrative manner.

See for example
Bernd Girod, Rudolf Rabenstein, Alexander Stenger (2001): "*Signals and Systems*." Wiley, Chichester (UK), Ch. 14.6.4

It is also treated in Ch. 5.9.1 of [Lyo11].

We already have dealt with the graphical representation for continuous-time signals, please see [convolution.ipynb](https://github.com/spatialaudio/signals-and-systems-lecture/blob/master/systems_time_domain/convolution.ipynb)
to recapitulate.

We use this concept now for the convolution of the two discrete-time signals $x$ and $h$ given above.
Instead of providing and explaining the approach with stem plots as usually, we treat this as a little Python programming task, where nicely formated `print` output clarifies the concept of mirroring, shifting and multiplying/accumulating.
Very similar this output could be manually performed in a written exam.

First, let us set up the signals $x$ and $h$ with the correct time alignment with respect to $\kappa$.
For that, inserting additional zeros is meaningful to bring both vectors to length $N = N_x + N_h -1 = 8$.
Note that instead of $\kappa$ we use the letter v in the little program and the print out for improved readability.
The print out shows $\kappa$ (which is treated as v), $x[\kappa]$ and $h[\kappa]$ properly aligned.

```python
x = np.array([0, 0, 0, 0, 1, 2, 3, 4])  # length Nx = 4 of x
h = np.array([8, 0, 7, 6, 5, 0, 0, 0])  # length Nh = 5 of h
v = np.arange(-2, 6)  # length N = Nx + Nh - 1 = 8 of resulting non-zero y
# print with integer sign
np.set_printoptions(formatter={'int': '{:+}'.format})
print('  v  = ' + str(v))
print('x[v] = ' + str(x))
print('h[v] = ' + str(h))
```

Second, we deploy a for loop to get the convolution result $y[k]$ for all required $0\leq k \leq 7$.

Within this loop, we mirror the array x (actually we reverse the order of the array's entries) and afterwards apply a circular shift by $k$.

The second and third print operation do the proper alignment between $x[k-\kappa]$ and $h[\kappa]$ for the print out.

All pairs of $x[k-\kappa]$ and $h[\kappa]$ that overlap in the print out need to be [multiplied and accumulated (MAC)](https://en.wikipedia.org/wiki/Multiply–accumulate_operation) to yield $y[k]$.
This is realized in the last print operation, which also prints the convolution result.

To keep it easily understandable, the little program contains some hard coded numbers specific to the chosen example.
You might think of a more general solution by introducing variables that handle the different array alignments for other examples.
After this has been done, one might get rid of the print outs and ends with a compact (but for large $N$ not very performant) code for calculation of the convolution in Python.

```python
np.set_printoptions()  # reset print out of integers
for k in range(0, 8):  # hard coded
    tmp = np.roll(np.flip(x), k)  # important: roll is a circular shift here
    print('circ shift x[-v] to right by k = ' + str(k))
    print('x[' + str(k) + '-v] = ' + str(tmp))
    print('h[v]   = ', 4*' ', h)  # hard coded
    print('y[k=' + str(k) + '] = ' +
          str(sum(tmp[3:]*h[0:5])) + '\n')  # hard coded
```

**Important Note:**

To make this code somewhat elegant in terms of shifting handling, we deployed a circular shift on $x$.
This means that an entry that is being shifted out at the one array bound is being reinserted at the other array bound.
Thus, instead of filling $x$ left with zeros, entries reappear at the beginning of x.
This can be observed for $k=5,6,7$ where 1,2,3 are successively reinserted.
However, this re-insertion has no impact on the convolution result, since there is no overlap of these values with $h$.
This is due to the initial, properly done zero padding to ensure (i) proper time alignment between $x$ and $h$ and (ii) same lengths $N=N_x+N_h-1$ of $x$, $h$ and $y$.

This handling is also used to perform a [fast linear convolution](https://github.com/spatialaudio/signals-and-systems-lecture/blob/master/discrete_fourier_transform/fast_convolution.ipynb) of two finite length sequences with the help of a cyclic convolution, which corresponds to a multiplication in the [DFT](https://github.com/spatialaudio/signals-and-systems-lecture/blob/master/discrete_fourier_transform/definition.ipynb)-domain.

When doing handwritten calculus, you might ignore this circular shift feature and fill with zeros as plain linear convolution would ask for.

### Solution III: FIR Filtering of Weighted Single Dirac Impulses and Summation of Partial Results

See Fig. 2.8 in Alan V. Oppenheim, Ronald Schafer (2010): "*Discrete-Time Signal Processing*", Pearson, Upper Saddle River, NJ (USA).

```python
x = np.array([1, 2, 3, 4])
h = np.array([8, 0, 7, 6, 5])
Nx = x.size
Nh = h.size
N = Nx+Nh-1  # length of y
k = np.arange(0, N)  # time alignment valid for the specific example
y = np.zeros(N)
# take one impulse of x and filter it with h (this works since)
y[0:Nh] += x[0]*h
y[1:Nh+1] += x[1]*h  # convolution of a weigthed Dirac with h yields weigthed h
y[2:Nh+2] += x[2]*h  # superposition of all partial results yields precisely
y[3:Nh+3] += x[3]*h  # the convolution result
# note that this implementation works for the specific example, where
# h starts at -2 and x at +2, thus compensating each other and aligning the
# partial signal x[0]*h at k=0
plt.figure(figsize=(9, 3))
ax = plt.subplot(1, 1, 1)
plt.stem(k, y, basefmt='None')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.ylabel('y[k]')
plt.xlabel('k')
plt.title(['y[k] = x[k]*h[k] =', y])
plt.grid(True)
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


Detección de cáncer de mama utilizando `sklearn`

Diego García Morate

diegogm at unlimiteck.com

Meetup Data Science with Python

Miércoles, 7 de Marzo de 2018 @ TheCubeMadrid

Los problemas de clasificación

Imaginate que tenemos un dataset con datos de distintos sensores de un coche (temperatura del motor, nivel de aceite...), y queremos saber a partir de esos sensores si el coche está funcionando bien o mal. ¿Cómo podríamos hacerlo?.

La manera tradicional consiste en crear una serie de reglas que relacionen los distintos sensores y que definan que rangos de funcionamiento son buenos o malos. De esta manera podríamos definir si el funcionamiento es `bueno` o `malo`.

Cada una de las reglas podría ser de este estilo:

si (temp_motor > 100ºC) y (nivel_aceite < 30) entonces estado malo
si (temp_motor >  30ºC) y (nivel_aceite > 30) entonces estado bueno

El problema de este enfoque es que nos obliga a conocer las relaciones que existen entre cada una de las variables y esto puede ser muy complicado.

La premisa de la que parte el aprendizaje automático es la siguiente: **¿Podríamos a partir de un conjunto de datos ya etiquetados generar automáticamente esas `reglas`?**

La respuesta es sí, y eso es una de los principales problemas a resolver en Machine Learning.

Antes de empezar:

* Llamaremos a esas `reglas` que queremos generar: **`modelo`**.
* Llamaremos al conjunto de datos previamente etiquetados: `conjunto de entrenamiento` o `dataset`.
* A cada uno de los casos individuales a evaluar lo llamaremos `instancia`.

Objetivo de este challenge

El objetivo de este challenge es entrenar un **modelo** capaz de determinar de forma automática si un tumor es `benigno` o `maligno`.

De paso, aprenderemos algunas técnicas de evaluación algoritmos de clasificación.

### Configuración del entorno

Antes de empezar vamos a cargar unos cuantos paquetes de python. Entre ellos usaremos `sklearn` como biblioteca de ML sobre la cual entrenar nuestro modelo.

```python
%matplotlib inline

import warnings
warnings.filterwarnings("ignore")

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sklearn
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_score, recall_score
```

Dataset

En este caso vamos a utilizar un dataset real de análisis de cáncer de mama del Breast Cancer Center de Wisconsin. Este dataset está disponible en `sklearn` así que cargarlo es tan fácil como:

```python
from sklearn import datasets
dataset = sklearn.datasets.load_breast_cancer()
```

Como siempre, una vez cargado debemos inspeccionar y comprender el dataset:

```python
print(dataset.keys())
```

    dict_keys(['data', 'target', 'target_names', 'DESCR', 'feature_names'])

```python
print(dataset.DESCR)
```

    Breast Cancer Wisconsin (Diagnostic) Database
    =============================================

    Notes
    -----
    Data Set Characteristics:
        :Number of Instances: 569

        :Number of Attributes: 30 numeric, predictive attributes and the class

        :Attribute Information:
            - radius (mean of distances from center to points on the perimeter)
            - texture (standard deviation of gray-scale values)
            - perimeter
            - area
            - smoothness (local variation in radius lengths)
            - compactness (perimeter^2 / area - 1.0)
            - concavity (severity of concave portions of the contour)
            - concave points (number of concave portions of the contour)
            - symmetry
            - fractal dimension ("coastline approximation" - 1)

            The mean, standard error, and "worst" or largest (mean of the three
            largest values) of these features were computed for each image,
            resulting in 30 features.  For instance, field 3 is Mean Radius, field
            13 is Radius SE, field 23 is Worst Radius.

            - class:
                    - WDBC-Malignant
                    - WDBC-Benign

        :Summary Statistics:

        ===================================== ====== ======
                                               Min    Max
        ===================================== ====== ======
        radius (mean):                        6.981  28.11
        texture (mean):                       9.71   39.28
        perimeter (mean):                     43.79  188.5
        area (mean):                          143.5  2501.0
        smoothness (mean):                    0.053  0.163
        compactness (mean):                   0.019  0.345
        concavity (mean):                     0.0    0.427
        concave points (mean):                0.0    0.201
        symmetry (mean):                      0.106  0.304
        fractal dimension (mean):             0.05   0.097
        radius (standard error):              0.112  2.873
        texture (standard error):             0.36   4.885
        perimeter (standard error):           0.757  21.98
        area (standard error):                6.802  542.2
        smoothness (standard error):          0.002  0.031
        compactness (standard error):         0.002  0.135
        concavity (standard error):           0.0    0.396
        concave points (standard error):      0.0    0.053
        symmetry (standard error):            0.008  0.079
        fractal dimension (standard error):   0.001  0.03
        radius (worst):                       7.93   36.04
        texture (worst):                      12.02  49.54
        perimeter (worst):                    50.41  251.2
        area (worst):                         185.2  4254.0
        smoothness (worst):                   0.071  0.223
        compactness (worst):                  0.027  1.058
        concavity (worst):                    0.0    1.252
        concave points (worst):               0.0    0.291
        symmetry (worst):                     0.156  0.664
        fractal dimension (worst):            0.055  0.208
        ===================================== ====== ======

        :Missing Attribute Values: None

        :Class Distribution: 212 - Malignant, 357 - Benign

        :Creator:  Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian

        :Donor: Nick Street

        :Date: November, 1995

    This is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.
    https://goo.gl/U2Uwz2

    Features are computed from a digitized image of a fine needle
    aspirate (FNA) of a breast mass.  They describe
    characteristics of the cell nuclei present in the image.

    Separating plane described above was obtained using
    Multisurface Method-Tree (MSM-T) [K. P. Bennett, "Decision Tree
    Construction Via Linear Programming." Proceedings of the 4th
    Midwest Artificial Intelligence and Cognitive Science Society,
    pp. 97-101, 1992], a classification method which uses linear
    programming to construct a decision tree.  Relevant features
    were selected using an exhaustive search in the space of 1-4
    features and 1-3 separating planes.

    The actual linear program used to obtain the separating plane
    in the 3-dimensional space is that described in:
    [K. P. Bennett and O. L. Mangasarian: "Robust Linear
    Programming Discrimination of Two Linearly Inseparable Sets",
    Optimization Methods and Software 1, 1992, 23-34].

    This database is also available through the UW CS ftp server:

    ftp ftp.cs.wisc.edu
    cd math-prog/cpo-dataset/machine-learn/WDBC/

    References
    ----------
       - W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction
         for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on
         Electronic Imaging: Science and Technology, volume 1905, pages 861-870,
         San Jose, CA, 1993.
       - O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and
         prognosis via linear programming. Operations Research, 43(4), pages 570-577,
         July-August 1995.
       - W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques
         to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994)
         163-171.

Como se indica en la descripción del dataset, éste consta de 569 instancias con 30 atributos numéricos cada uno.

```python
dataset_X = dataset.data
```

```python
dataset_X.shape
```

    (569, 30)

```python
dataset_X[0]
```

    array([  1.79900000e+01,   1.03800000e+01,   1.22800000e+02,
             1.00100000e+03,   1.18400000e-01,   2.77600000e-01,
             3.00100000e-01,   1.47100000e-01,   2.41900000e-01,
             7.87100000e-02,   1.09500000e+00,   9.05300000e-01,
             8.58900000e+00,   1.53400000e+02,   6.39900000e-03,
             4.90400000e-02,   5.37300000e-02,   1.58700000e-02,
             3.00300000e-02,   6.19300000e-03,   2.53800000e+01,
             1.73300000e+01,   1.84600000e+02,   2.01900000e+03,
             1.62200000e-01,   6.65600000e-01,   7.11900000e-01,
             2.65400000e-01,   4.60100000e-01,   1.18900000e-01])

Este dataset clasifica los casos en cáncer maligno ($0$) o benigno ($1$).

```python
print(dataset.target_names)
```

    ['malignant' 'benign']

```python
dataset_y = dataset.target
```

```python
dataset_y.shape
```

    (569,)

```python
print(dataset_y)
```

    [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
     1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0
     1 0 1 0 0 1 1 1 0 0 1 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 1 1
     1 1 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 1 1 1 1 0 1
     1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 0
     1 0 1 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1
     1 0 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0
     0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1
     1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 1
     1 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0
     0 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1
     1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1
     0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1
     1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0
     1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
     1 1 1 1 1 1 1 0 0 0 0 0 0 1]

### Preguntas sobre el dataset
#### ¿Está balanceado o desbalanceado?

Un dataset se denomina balanceado cuando el número de instancias de cada clase es similar.

```python
np.bincount(dataset.target)
```

    array([212, 357])

Como podemos ver hay 212 casos malignos y 357 benignos.

```python
np.bincount(dataset.target)[0] / dataset.target.shape[0]
```

    0.37258347978910367

El 37% de los casos son malignos y el 63% son benignos, el dataset está desbalanceado pero no mucho.

#### ¿Es suficientemente extenso para entrenar un modelo?

Tiene 569 instancias, con 30 atributos cada una.

* Si cada atributo fuera una variable binaria totalmente independiente necesitaríamos $2^{30}$ = 1.000 millones de instancias para recubrir todas las posibles combinaciones

No existe una respuesta fácil a esta pregunta, dependerá de la redundancia interna entre variables, la relación entre las mismas y la aportación de cada variable a la variable objetivo.

Métricas de evaluación

Ya tenemos cargado el dataset y, antes de entrenar un modelo, tenemos que definir una **función objetivo**. En este caso lo que tenemos que hacer es establecer una métrica de evaluación que generalmente tendremos que maximizar.

La métrica más sencilla es el porcentaje acierto:

$$\%_{acierto} = \frac{{Casos \space bien \space clasificados}}{{Casos \space totales}}$$

### ¿Cual sería el clasificador más sencillo con mejor % de acierto que se te ocurre? ¿es útil en la práctica?

#### a) Moneda al aire

Una moneda al aire acertaría el 50% de los casos positivos y 50% de los negativos. Es decir:

$$
\begin{align}
P(acertar) &= 50\% * P(Malignos) + 50\% * P(Benignos) \\
P(acertar) &= 50\% * 37\% + 50\% * 63\% \\
P(acertar) &= 18,5\% + 31,5\% \\
P(acertar) &= 50\%
\end{align}
$$

#### b) Quedarse siempre con la clase más frecuente

Clasificar todos los casos como benignos que es la clase más frecuente (63%)

```python
from sklearn.dummy import DummyClassifier

dc_most_frequent = DummyClassifier(strategy='most_frequent', random_state=None)
```

```python
dc_most_frequent.fit(dataset_X, dataset_y)
dc_most_frequent.score(dataset_X, dataset_y)
```

    0.62741652021089633

### ¿Estos clasificadores son útiles en la práctica?

No mucho, porque en el fondo el incremento de información que proporciona este clasificador sobre el problema es 0.

Aunque son útiles para derterminar el porcentaje de acierto mínimo que debería cumplir un clasificador. En este caso todo lo que esté por debajo de un 63% no resulta muy interesante.

Conjunto de entrenamiento y test

Algo fundamental cuando entrenamos un modelo es hacer una división del conjunto de entrenamiento y de test. No tendría sentido entrenar el modelo con los mismos datos que luego a vamos a utilizar en su evaluación..

Podemos partir el dataset en conjunto de entrenamiento y test con `sklearn` con la siguiente función:

```python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(dataset_X, dataset_y, test_size=.3, random_state=42)
```

```python
X_train.shape
```

    (398, 30)

```python
X_test.shape
```

    (171, 30)

Entrena un modelo

Una vez que tenemos el conjunto de entrenamiento y test vamos a entrenar el modelo. En este caso vamos a utilizar un algoritmo k-vecinos (`KNeighborsClassifier`) como clasificador.

Podemos instanciar este modelo con `sklearn` de la siguiente forma:

```python
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier()
```

Una vez instanciado, lo podemos entrenar con la función `fit` utilizando el conjunto de entrenamiento:

```python
knn.fit(X_train, y_train)
```

    KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
               metric_params=None, n_jobs=1, n_neighbors=5, p=2,
               weights='uniform')

Una vez entrenado lo podemos utilizar para predecir las instancias del conjunto de test.

```python
y_predicted = knn.predict(X_test)
```

```python
y_predicted
```

    array([1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0,
           1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1,
           1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1,
           1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1,
           1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1,
           0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1,
           1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1,
           1, 1, 1, 0, 1, 1, 1, 1, 0, 1])

Y podemos calcular el porcentaje de acierto de la siguiente manera:

```python
(knn.predict(X_test) == y_test).mean()
```

    0.95906432748538006

También existe la función `score` que nos devuelve automáticamente el porcentaje de acierto:

```python
knn.score(X_test, y_test)
```

    0.95906432748538006

El porcentaje de acierto es un 95%. **¿Es bueno o malo?**

Evaluación de resultados

##  Matriz de confusión

Aunque el porcentaje de acierto es sencillo de entender muestra una realidad muy parcial del problema.

Imaginate que en nuestro ejemplo del automóvil tuviéramos un 83% de acierto. Podría ocurrir que nuestro modelo clasificara bien todos los casos en los que el coche estuviera bien, pero ninguno en los que estuviera mal. Podemos tener un buen porcentaje de acierto, y que el clasificador funcione mal.

Un método un poco más complejo, pero muy efectivo es utilizar la `matriz de confusión` esta matriz nos indica por cada categoría a clasificar (lo que se denomina `clase`) cuantos casos han sido clasificados bien y cuantos mal.

```python
from sklearn.metrics import confusion_matrix
confusion_matrix(y_test, y_predicted)
```

    array([[ 57,   6],
           [  1, 107]])

Podemos dibujar la matriz de confusión con `sklearn` y el paquete `scikitplot` de la siguiente manera:

```python
import scikitplot as skplt
skplt.metrics.plot_confusion_matrix(y_test, y_predicted)
```

### Si tengo un paciente con un cáncer maligno (0), ¿qué probabilidad hay de que el modelo lo clasifique como cáncer maligno (0)?

La probabilidad de que lo clasifique bien es:

```python
57 / (57+6)
```

    0.9047619047619048

### Si tengo un paciente con un cáncer benigno (1), ¿qué probabilidad hay de que el modelo lo clasifique como cáncer benigno (1)?

```python
107 / (107 + 1)
```

    0.9907407407407407

En este caso esta medida se conoce como `recall`, también lo podemos calcular como:

```python
recall_score(y_test, y_predicted)
```

    0.9907407407407407

```python
skplt.metrics.plot_confusion_matrix(y_test, y_predicted)
```

### Si tengo un paciente con un cáncer maligno (0), ¿qué probabilidad hay de que el modelo lo clasifique como cáncer benigno (1)?

```python
6 / (57+6)
```

### Si tengo un paciente con un cáncer benigno (1), ¿qué probabilidad hay de que el modelo lo clasifique como cáncer maligno (0)?

```python
1 / (107+1)
```

Conclusiones

## ¿Qué podrías concluir del clasificador?

Está bien, pero sería mejor que aumentara el % de acierto de los cánceres malignos. Porque a esa gente es mejor identificarla y no fallar nunca en su identificación.

A los falsos positivos les puedo repetir la prueba, pero a los falsos negativos no.

## ¿Se te ocurre alguna manera de mejorarlo?

* Utilizar modelos más potentes
* Ampliar el dataset
* Penalizar al modelo los falsos negativos con una matriz de coste

## ¿Es útil este modelo en la práctica?

Habría que contrastar los resultados con los clasificadores actuales (médicos) y también habría que identificar y cuantificar el coste de la extracción y preprocesamiento de los datos.

2018-03-08 DataScienceWithPython Meetup/Meetup ML cancer.ipynb


# Programing exercise 3: Coupled oscillators

Due on Monday, 12.11.2018, 14.15h

### Total: 100 points

__This exercise sheet has been solved by:__ Mathieu Kaltschmidt and Carl von Randow

## The problem

Consider two coupled harmonic oscillators described by the Hamiltonian
$$
H=H_1 + H_2 + V =\frac{1}{2m}(p_1^2 + p_2^2) + \frac{1}{2}k(x_1^2 + x_2^2) + \frac{1}{2}\lambda(x_1 - x_2)^2
$$
We want to calculate the eigenvalues and eigenfunctions of this Hamiltonian and compare the dynamics of the corresponding classical problem. This is exactly solvable by transforming into center of mass and relatice coordinates. However, we also want to treat the coupling term lambda as a perturbation and solve the problem by expanding into eigenfunctions of the 2D harmonic oscillator and solve the resulting problem, both, perturbatively for small $\lambda$ and by exact diagonalization.

```python
# load standard libraries
import numpy as np
import numpy.linalg as LA
import matplotlib.pyplot as plt
%matplotlib inline

#for 3D plotting
from mpl_toolkits.mplot3d import Axes3D

#Plot settings
plt.style.use('seaborn-talk')
plt.rcParams["font.family"]='serif'
plt.rcParams["figure.figsize"][0] = 16
plt.rcParams["figure.figsize"][1] = 10
```

$$\newcommand{\braket}[3]{\left\langle{#1}\middle|{#2}\middle|{#3}\right\rangle}$$

### Exercise 1 (20 points)

Transform the problem into center of mass and relative coordinates using
$$
x_{CM}=(x_1+x_2)/2 \\
x_{rel}=(x_1-x_2)
$$
Write a function that returns the classical trajectory for initial conditions $\{x_1(t=0)=x_{10}, x_2(t=0)=x_{20}, \dot{x}_1(t=0)=v_{10}, \dot{x}_2(t=0)=v_{20}\}$.

Make a density plot of the potential and plot the classical trajectory for $m=k=1,\lambda=0.2$ and ${x_{10}=1,x_{20}=v_{10}=v_{20}=0}$ up to a time $t_f$ into it. Also, plot the positions of the two particles as a function of time. Observe the beating between center-of-mass and relative frequency.

Very instructive 2D plotting tips:
https://jakevdp.github.io/PythonDataScienceHandbook/04.04-density-and-contour-plots.html

### Analytical solution of the problem
To change our old coordinates to the new ones we have to use:

\begin{align}
x_{1}&=x_{CM}+\frac{1}{2}x_{rel} \\
x_{2}&=x_{CM}-\frac{1}{2}x_{rel}
\end{align}

This yields the following Hamiltonian:

\begin{align}
H=m(\dot{x}_{CM}^2 + \frac{1}{4}\dot{x}{rel}^2)+\frac{1}{2} k(\frac{1}{2}x_{rel}^2+x_{CM}^2)+\frac{1}{2} \lambda x_{rel}^2
\end{align}

$H$ can be split into two independent Hamiltonians $H_{rel}$ and $H_{CM}$:

\begin{align}
H_{CM}&=m\dot{x}_{CM}^2+\frac{1}{2} k x_{CM}^2 \\
H_{rel}&=m\frac{1}{4}\dot{x}_{rel}^2+x_{rel}^2(\frac{1}{4}k+\frac{1}{2} \lambda)
\end{align}

By solving the Euler-Lagrange equations we obtain:

\begin{align}
x_{rel}(t) &= S_{rel} sin(\omega_{rel}t) + C_{rel} cos(\omega_{rel}t)\\
x_{CM}(t) &= S_{CM} sin(\omega_{CM}t) + C_{CM} cos(\omega_{CM}t)\\
\end{align}

$\omega_{rel}$ and $\omega_{CM}$ are given by:

\begin{align}
\omega_{rel} &= \sqrt{\frac{k+2\lambda}{m}}\\
\omega_{CM} &= \sqrt{\frac{k}{2m}}
\end{align}

To transform this solution back to our old coordinates we use the first two equations in this exercise and get the following result:

\begin{align}
x_1(t) &= C_{CM} cos(\omega_{CM}t) + S_{CM} sin(\omega_{CM}t)+\frac{1}{2} \left[ C_{rel} cos(\omega_{rel}t) + S_{rel} sin(\omega_{rel}t) \right] \\
x_2(t) &= C_{CM} cos(\omega_{CM}t) + S_{CM} sin(\omega_{CM}t)-\frac{1}{2} \left[ C_{rel} cos(\omega_{rel}t) + S_{rel} sin(\omega_{rel}t) \right]
\end{align}

The initial conditions lead to the following coefficients:

\begin{align}
C_{rel} &= x_{10}-x_{20}\\
S_{rel} &= \frac{1}{\omega_{rel}} \left( v_{10}-v_{20} \right)\\
C_{CM} &= \frac{1}{2} \left( x_{10}+x_{20} \right) \\
S_{CM} &= \frac{1}{2 \omega_{CM} } \left( v_{10}+v_{20} \right)
\end{align}

```python
#Defining spatial grid structure
x1=np.linspace(-2,2,100)
x2=np.linspace(-2,2,100)

#some parameters
(m, k, hbar, omega, const)  = (1, 1, 1, 1, 0.2)

#initial conditions
(x10, v10) = (1, 0)
(x20, v20) = (0, 0)
```

```python
#Implementing the solution for the trajectories
def trajectories(x1,v1,x2,v2,t):
    x_rel = (x1 - x2)
    omega_rel = np.sqrt((k + 2*const)/(2*m))
    omega_cm = np.sqrt(k/(2*m))

    #defining the known prefactors
    C_rel = x_rel
    S_rel =(v1 - v2) / omega_rel

    C_CM = 0.5 * (x1 + x2)
    S_CM =(v1 + v2) / (2 * omega_cm)

    x1=C_CM*np.cos(omega_cm*t) + S_CM*np.sin(omega_cm*t) + ((C_rel*np.cos(omega_rel*t) + S_rel*np.sin(omega_rel*t)))/2
    x2=C_CM*np.cos(omega_cm*t) + S_CM*np.sin(omega_cm*t) - ((C_rel*np.cos(omega_rel*t) + S_rel*np.sin(omega_rel*t)))/2

    return x1, x2
```

```python
#Defining the harmonic oscillator potential
def oscillator(x1,x2):
    return 0.5*(x1**2 + x2**2)

#Defining the given coupling potential
def coupling(x1,x2,const):
    return 0.5*const*(x1-x2)**2

#Defining the total potential (oscillator + coupling)
def V_total(x1,x2,const):
    return oscillator(x1,x2) + coupling(x1,x2,const)

#Creating the grid structure + density plots
def density_plot(x1,x2,potential,const):
    x,y = np.meshgrid(x1,x2)
    z = potential(x,y,const)
    plt.contour(x,y,z,20,cmap = 'plasma')
    plt.colorbar()
```

```python
#Calcultating the trajectory
time = np.linspace(0,40,1000)
particle1, particle2 = trajectories(x10,v10,x20,v20,time)
```

```python
#Plotting the trajectories in the total potential
plt.plot(particle1, particle2, color='teal', linewidth = 4, label = 'Trajectory of the coupled system')
density_plot(x1,x2,V_total,const)
plt.title('Trajectory of a system of two particles in a coupled oscillator potential \n Initial conditions: $x_{1,0} = 1, \ x_{2,0} = v_{1,0} = v_{2,0} = 0$', size = 20)
plt.xlabel('Coordinate $x_1$', size = 18)
plt.ylabel('Coordinate $x_2$', size = 18)
plt.legend(frameon=True,fontsize='large', borderpad=1.5, borderaxespad=1.5,loc='lower right')
```

```python
#Plotting both trajectories as a function of time
plt.plot(time, particle1, label="Particle 1", color = 'darkred')
plt.plot(time ,particle2, label="Particle 2", color = 'darkblue')
plt.ylabel("Position", size = 22)
plt.xlabel("Time $t$", size = 22)
plt.title("Trajectories of both particles over time", size = 24)
plt.legend(frameon=True,fontsize='large', borderpad=1.5, borderaxespad=1.5,loc='best')
```

### Exercise 2 (10 points)

What are the exact quantum mechanical eigenenergies?
Calculate the ground state energy perturbatively to second order in $\lambda$. Use the decomposition into ladder operators
$$
x/x_{HO} = (a^\dagger + a)/\sqrt{2}
$$
Show that the same result is obtained by Taylor expanding the exact ground state energy $E_0 = \hbar\omega/2(1+\sqrt{1+2\lambda/k}) $ to second order in $\lambda/k$.

Plot the exact and perturbative ground state energy as a function of $\lambda/k$.

### Result from Taylor expanding around small $\frac{\lambda}{k}$

In the following calculation we substitute $\frac{\lambda}{k} =: \alpha$ and calculate the Taylor expansion around $\alpha_0 \approx 0$.

The ground state energy is defined as: $E_0 = \frac{\hbar\omega}{2}\left(1+\sqrt{\frac{2\lambda}{k}}\right)$.

We get the following result for the expansion to second order in $\alpha$:

\begin{align}
E_0\kern-0.3em\mid\kern-0.3em\small_{\alpha_0 = 0} &\approx \frac{\hbar\omega}{2} \left( 1 + \sqrt{2\cdot\alpha_0}\cdot\alpha^{0}
     + \frac{1}{1!}\cdot\frac{1}{2}\frac{2}{\sqrt{1+2\alpha_0}}\cdot \alpha^1 + \frac{1}{2!}\cdot\left(-\frac{1}{2}\right)\frac{2}{\sqrt{1+2\alpha_0}^3}\cdot \alpha^2 + \mathcal{O}\left(\alpha^3\right) \right) \\
     &=  \frac{\hbar\omega}{2} \left( 2 + \alpha - \frac{1}{2}\alpha^2 + \mathcal{O}\left(\alpha^3\right) \right) = \hbar\omega \left( 1 + \frac{\alpha}{2} - \frac{\alpha^2}{4}\right) + \mathcal{O}\left(\alpha^3\right)
\end{align}

The final result of the expansion, reads:

$$
E_0 \approx \hbar\omega \left( 1 + \frac{\lambda}{2k} - \frac{\lambda^2}{4k^2}\right) + \mathcal{O}\left(\frac{\lambda^3}{k^3}\right)
$$

### Result using perturbation theory

The idea of perturbation theory is to seperate the general Hamiltonian into the two unperturbed parts for both particles and a small perturbated part, e.g. :

$$
\mathcal{H} = \underbrace{\mathcal{H}_1 + \mathcal{H}_2}_{:= \ \mathcal{H}_0} + V = \mathcal{H}_0 + \lambda\cdot\mathcal{H}_1
$$
The potential $V=\frac{1}{2}\left(x_1-x_2\right)^2$ corresponds to the harmonic oscillator problem, in this case for a system of two particles.

#### Finding the correct eigeneenergies

One computes the eigenenergies $E_n$ by finding the correction terms from perturbation theory:

$$
E_n = E_n^0 + \lambda \ E_n^1 + \lambda^2 \ E_n^2 +\mathcal{O}\left(\lambda^3\right)
$$

#### Lowest order in unperturbed Hamiltonian

For this case we only have to consider the general solution of the harmonic oscillator problem and assume both particles occupying the groundstate $n=0$:

$$
E_0^0 = \hbar\omega\left( n_1 + n_2 + 1\right) = \hbar\omega
$$

#### First order perturbation theory

\begin{align}
E^1_0 &= \langle n_1,n_2| \ \mathcal{H}_1 \ |n_1,n_2\rangle = \langle n_1,n_2|\frac12 ( x_1-x_2)^2|n_1,n_2\rangle
      = \frac{\hbar}{4m\omega}\langle n_1,n_2| \left(a^\dagger_1+a_1-(a^\dagger_2+a_2)\right)^2|n_1,n_2\rangle \\
      &= \frac{\hbar}{4m\omega} \langle n_1,n_2|\left(a^\dagger_1+a_1-(a^\dagger_2+a_2)\right)\left(\sqrt{n_1+1}|n_1+1,n_2\rangle+\sqrt{n_1}|n_1-1,n_2\rangle-\sqrt{n_2+1}|n_1,n_2+1\rangle-\sqrt{n_2}|n_1,n_2-1\rangle\right)
\end{align}

For the ground states we know that $n_1=n_2=0$:

$$
E^1_0=
\frac{\hbar}{4m\omega} \langle 0,0|\left(a^\dagger_1+a_1-a^\dagger_2-a_2\right)\left(|1,0\rangle-|0,1\rangle\right)=
\frac{\hbar}{2m\omega}=
\frac{\hbar\omega}{2k}
$$

#### Second order perturbation theory:
\begin{align}
E^2_{0,0}&=\sum_{m_1,m_2\neq 0}\frac{\left|\langle m^0_1,m^0_2|\ \mathcal{H}_1 \ |0,0\rangle\right|^2}{E_{0,0}^0-E_{m_1,m_2}^0} \\
&=\sum_{m_1,m_2\neq 0}\frac{\left|\langle0,0|\frac{\hbar}{4m\omega}\left(a_1-a_2\right)|m_1, m_2\rangle\right|^2}{-\hbar\omega(m_1+m_2)} \\
&=\left(\frac{\hbar}{4m\omega}\right)^2\left[\frac{|\sqrt{2}|^2}{-2\hbar\omega}+\frac{|-1|^2}{-2\hbar\omega}+\frac{|-1|^2}{-2\hbar\omega}+\frac{|\sqrt{2}|^2}{-2\hbar\omega}\right] \\
&=\frac{-3\hbar}{16m^2\omega^3}=
-\frac{3\hbar\omega}{16k^2}
\end{align}

We approximate the last term and get:

$$
E_0 \approx \hbar\omega \left( 1 + \frac{\lambda}{2k} - \frac{\lambda^2}{4k^2}\right)
$$

#### Conclusion
Therefore we found (almost) the same result for both methods.

### Plotting the result

```python
testrange = np.linspace(0,2,100)

#Exact round state energy
def E0(const):
    return (hbar*omega)/2*(1+np.sqrt(1+2*const/k))
#Result from perturbation theory
def E0_PT(const):
    return hbar*omega*(1+const/(2*k)-const**2/(4*k**2))

plt.plot(testrange, E0(testrange), label="Exact ground state energy", color = 'darkred')
plt.plot(testrange, E0_PT(testrange), label="Ground state energy from perturbation theory", color = 'darkblue')
plt.title("Comparison of the exact ground state energy with the result from perturbation theory", size = 20)
plt.ylabel("Ground State Energy [$\hbar\omega$]", size = 22)
plt.xlabel("Order of perturbation $\ \lambda/k$", size = 22)

plt.grid(True)
plt.legend(frameon=True,fontsize='large', borderpad=1.5, borderaxespad=1.5,loc='best')
```

#### Conclusion
As one can see, the calculated approximation using perturbation theory holds for small $\frac{\lambda}{k}$.

### Exercise 3 (40 points)

Expand the Hamiltonian in the eigenbasis of the 2D harminic oscillators (use the same cutoff $N_1=N_2=N_{max}$ for both oscillator modes 1 and 2).
For this, one can expand the perturbation term $\frac{1}{2}\lambda(x_1 - x_2)^2$ into ladder operators and then apply it to the unperturbed basis states.
How many non-zero elements will there be at most in each row?

There are many ways of building the Hamiltonian matrix in the chosen basis. First, make sure you understand what the basis states are and how one would order them in a reasonable way.
A failsafe way to build up the Hamiltonian matrix is to iterate over all states of the bare basis and determine the matrix-elments it couples to. Another, arguably more elegant, way is to build the matrix representing the anihilation operator on the Hilbert space of one of the oscillators. The creation operator is then simply the transpose of that and the position operator can be built by adding the two. We have a 2D problem so we work in a Hilbert space that is a tensor product of the two Hilbert spaces of the oscillators. Thus, to represent the position operator $\hat{x}_1$ acting on oscillator 1 in the full Hilbert space  we have to use $\hat{x}_1 \otimes \mathbf{1}$, where $\mathbf{1}$ is the identity acting on the second subspace (see the numpy "eye" function). This can be achieved by using the Kronecker product. For multiplying or squaring operators you can use the numpy dot product function.
The second method might be less efficient if you want to exploit the sparsity of the problem but it is very handy since it requires only to build one operator explicitly and the rest is matrix algebra (or two if you use different basis sizes for the two oscillators).

Diagonalize the Hamiltonian and check your result by comparing the ground state energy to the exact result. How small can you make the basis such that the ground sate energy still converged? (You should find that $N_1=N_2=5$, i.e. 25 basis states, is sufficient to get the ground state with good precision up to $\lambda=2$. Note that using a simple spatial grid to represent the wave function we would need much larger basis sizes!) Also look at some excited states, at least the lowest 6, make sure that your diagonalization results agree with the exact one, and understand the structure of the spectrum.

Bonus (not graded): This is of course still a problem that can be solved analytically, and thus we are complicating it by doing the basis set expansion. However, one could now use this method to apply it to non-integrable cases like a quartic (non-linear) coupling or even the quantum version of the Henon-Heiles problem, which does not separate and is classically chaotoic!
If you want to go beyond, try other potentials!

### Solution:
#### Non-zero elements in each row
With this method one finds at most $3$ non-zero elements in each row. This can easily be seen by writing down a general matrix multiplication of the form of our problem. These non-zero entries are located on the main diagonal and the respective second next diagonals.

```python
#Implementation of x1- and x2-operators with N_cut = N
def x_ops(N):
    #Creating the identity Matrix with dim = N x N
    identity = np.eye(N)
    #Matrix representation of the ladder operators
    values = np.sqrt(np.linspace(1,N-1,N-1))
    a = np.diag(values,1)
    a_dagger = np.diag(values,-1)
    #Computing final result using the tensor product
    x1 = 1/np.sqrt(2)*(np.kron(a,identity)+np.kron(a_dagger,identity))
    x2 = 1/np.sqrt(2)*(np.kron(identity,a)+np.kron(identity,a_dagger))
    return x1, x2

def perturbation(N,const):
    x1, x2 = x_ops(N)
    H1 = (const/2) * (np.dot(x1,x1)-np.dot(x1,x2)-np.dot(x2,x1)+np.dot(x2,x2))
    return H1, x1, x2

#Now we want to compute the eigenenergies
def eigenenergies(N,const):
    H1, x1, x2 = perturbation(N,const)
    #Unperturbed Hamiltonian H_0
    H0 = np.diag(np.linspace((hbar*omega)/2,hbar*omega*(N-1/2),N))
    identity = np.eye(N)
    #Applying the tensor product again
    H0_tensor = np.kron(H0,identity)+np.kron(identity,H0)
    eigvals, eigvecs = LA.eig(H0_tensor + H1)
    return eigvals,eigvecs
```

```python
#Comparing some values for N
Nvals = np.linspace(2,10,9, dtype=int)
const = 2

#We directly compare the theoretical with the numerical values for all considered values for N
print("Energy for the ground state (theoretical):", E0(const))
print()
print('Numerical values for different N:')

for N in Nvals:
    vals, vecs = eigenenergies(N, const)
    print('For N =', N, ': ', sorted(vals)[0], ' , Difference to theoretical value = ', np.abs(sorted(vals)[0]-E0(const)) )
```

    Energy for the ground state (theoretical): 1.61803398875

    Numerical values for different N:
    For N = 2 :  1.58578643763  , Difference to theoretical value =  0.032247551123
    For N = 3 :  1.5748352925  , Difference to theoretical value =  0.0431986962489
    For N = 4 :  1.61723525671  , Difference to theoretical value =  0.000798732040295
    For N = 5 :  1.61663936756  , Difference to theoretical value =  0.0013946211869
    For N = 6 :  1.6180001152  , Difference to theoretical value =  3.38735513465e-05
    For N = 7 :  1.61798832975  , Difference to theoretical value =  4.5658998066e-05
    For N = 8 :  1.61803186541  , Difference to theoretical value =  2.12333610228e-06
    For N = 9 :  1.61803237262  , Difference to theoretical value =  1.61613160099e-06
    For N = 10 :  1.61803385578  , Difference to theoretical value =  1.32965591426e-07

#### Conclusion
We found out that we have to choose at least $N=4$ to get a sufficient precision for the ground state energy.

#### Excitation states

```python
#Testing some exitation states
excitation = np.linspace(1,6,6, dtype=int)
for state in excitation:
    print(state,'. excited state: E = ',sorted(vals)[state])
```

    1 . excited state: E =  2.61802723699
    2 . excited state: E =  3.61807359778
    3 . excited state: E =  3.85409571552
    4 . excited state: E =  4.61725673509
    5 . excited state: E =  4.85407373673
    6 . excited state: E =  5.62139155209

We chose the ordering of the excitation states depending on the value of their respective energy level.

### Exercise 4 (30 points)

Calculate the time evolution for different initial conditions: 1) oscillator 1 in the first excited state, oscillator two in its ground state $|0\rangle$; 2) oscillator 1 in $|\psi_0\rangle = (|0\rangle+|1\rangle)/\sqrt(2)$, oscillator two in state $|0\rangle$;  3) oscillator 1 in a coherent state (e.g. with $\langle n \rangle = |\alpha|^2 = 2$) and oscillator 2 in $|0\rangle$. Example parameters: $\lambda=0.2$, $N_1=N_2=10$, $t_f = 40$.

Monitor the dynamics by calculating $\langle x_1 \rangle$ and $\langle x_2 \rangle$, the energy expectation value of each oscillator, as well as the probabilities for each oscillator to be in state $|n\rangle$. Compare your observations to the classical expectation: propagate a classical particle with $x_{i0}=\langle x_i(t=0) \rangle$ and $p_{i0}=\langle p_i(t=0) \rangle$. Describe and interpret your observations.

Bonus (not graded): Represent the wave function on a 2D spatial grid using the exact harmonic oscillator eigenfunctions. Animate the time-dependence of the wave packet.

### Analytical solution

As usual the time evolutions are given by:

1)
\begin{align}
\left| \ \Psi_1(t) \ \right\rangle = e^{-i E_{1,0}t}(\left| 1 \right\rangle \otimes \left| 0 \right\rangle )\\
\end{align}

2)
\begin{align}
\left| \ \Psi_2(t) \ \right\rangle = \frac{1}{\sqrt{2}} \left(  e^{-i E_{0,0}t}(\left| 0 \right\rangle \otimes \left| 0 \right\rangle) + e^{-i E_{1,0}t}(\left| 1 \right\rangle \otimes \left| 0 \right\rangle)\right)
\end{align}

3)
The coherent state is described by:

\begin{align}
\left| \ \Psi_3(t=0) \ \right\rangle = e^{-\frac{1}{2}|z|^2}e^{za^{\dagger{}}}\left| 0 \right\rangle
\end{align}

Since we want $\left\langle n \right\rangle = 2 $, we get $z = \sqrt{2}$.

#### x expectation values:
For the first case we have no time dependency for $\left\langle x_i \right\rangle$, as $\left\langle \ \Psi(t) \ \middle| \ x\ \middle| \ \Psi(t) \ \right\rangle $ (with $x$ being either $\hat{x}_1 \otimes \mathbf{1}$ or $\mathbf{1} \otimes \hat{x}_2$) makes the complex rotation go away. Since we have an orthogonal basis, which means that the scalar products vanish after applying the $x$-Operators, we obtain an expectation value of 0 in both instances. For the second particle this is also true in all further systems.

This means for the first system:

\begin{align}
\left\langle x_1 \right\rangle =0
\end{align}

And in all systems:

\begin{align}
\left\langle x_2 \right\rangle =0
\end{align}

After applying $\hat{x}_1 \otimes \mathbf{1}$ we get the following for the time evolution of system two:

\begin{align}
x\left| \ \Psi_2(t) \ \right\rangle =  \frac{1}{2} \left( e^{-i E_{1,0}t}((\left| 0 \right\rangle + \sqrt{2} \left| 2 \right\rangle) \otimes \left| 0 \right\rangle) + e^{-i E_{0,0}t}(\left| 1 \right\rangle \otimes \left| 0 \right\rangle) \right)
\end{align}

This results in an expectation value of:

\begin{align}
\frac{1}{2\sqrt{2}} (e^{it(E_{1,0}-E_{0,0})}+e^{it(E_{0,0}-E_{1,0})}) =
\frac{1}{\sqrt{2}} cos((E_{1,0}-E_{0,0})t)
\end{align}

Also in the last system we have an oscillation of the expectation value with:

\begin{align}
\sqrt{2} \ Re \left\{ \sqrt{2}e^{-iEt} \right\}
\end{align}

#### Energy expectation values:

The first two systems are formulated in the energy eigenbasis. Therefore the expectation value for their respective energies is constant in time.

For the first system we find: $\langle E \rangle = E_{1,0}$

The second system is a bit more complicated: $\langle E \rangle =  \frac{1}{2} \cdot (E_{0,0} + E_{1,0})$

The coherent state also has a constant energy expectation value, e.g. $\langle E \rangle = E_{0,0} + \hbar\omega\mid\kern-0.2em\alpha\kern-0.2em\mid^2  = E_{0,0} + 2\hbar\omega$

__We were not sure, which parts of this exercise were supposed to be solved numerically and which parts were supposed to be solved analytically.__

### Dynamics

#### Second example

```python
#Calculating important values
N=5
vals2, vecs2 = eigenenergies(N,const)
vals2 = sorted(vals2)
E00 = vals2[0]
E10 = vals2[1]

def expval_x(t,E00,E10):
    return 1/np.sqrt(2)*np.cos((E10-E00)*t)

timesteps = np.linspace(0,20,100)

#Plotting the result
plt.plot(timesteps ,expval_x(timesteps,E00,E10), color = 'darkblue')
plt.title('Expectation value for $x$ in the second example system', size = 20)
plt.xlabel('Time  $t$', size = 18)
plt.ylabel(r'$ \langle x \rangle $', size = 18)
plt.xlim(0,20)
plt.grid(True)
```

#### Interpretation of the probabilities for states $|n\rangle$

The probability to detect particle two in state $|0\rangle$ is always 1.

In the first system, we detect $x_1$ with a probabilty of 1 ( = 100% ) in the first excitation state and in the second system with a probability of $\frac{1}{2}$ in either state of the superposition, e.g  $ |0\rangle$ and $ |1\rangle$.


```python
# uncomment these to filter Matplotlib deprecation warnings for sympy
import warnings
warnings.filterwarnings('ignore')
```

The unitstep function gives a unit impulse at a given value $t$.

For $x <= t$ unitstep(x) = 0. Otherwise, initstep(x) = 1.

```python
import numpy as np
import matplotlib.pyplot as plt
def unitstep(t):
  u = np.arange(t.shape[0])
  lcv = np.arange(t.shape[0])
  for place in lcv:
    if t[place] > 0 + c:
      u[place] = 1
    else:
      u[place] = 0
  return u

c = 2

t = np.linspace(c - 5,c + 5,200)
fig, ax = plt.subplots(dpi=96)
ax.plot(t, unitstep(t), label="unitstep(" + str(c) + ")")
ax.grid()
ax.legend();
```

The derivative of the unitstep function is the DiracDelta function. It has some interesting properties shown here.

```python
from sympy import *
# init_printing(use_unicode=False, wrap_line=False)
print('DiracDelta(x) for any value but 0 is 0:')
print(DiracDelta(-1))
print(DiracDelta(1))
print('\nDiracDelta(0) is undefined. In at least some versions of Sympy, it is unevaluated:')
print(DiracDelta(0),'\n')
x, d = symbols('x d')
h = 0.0001
print('Integrating DiracDelta(x) for bounds that include x in the interval produces 1:')
sol1 = integrate(DiracDelta(x - d), (x, d-h, d+h))
print(sol1)
sol2 = integrate(DiracDelta(x - d), (x, -oo, d+h))
print(sol2)
sol3 = integrate(DiracDelta(x - d), (x, d-h, oo))
print(sol3,'\n')
print('Integrating DiracDelta(x) using bounds that don\'t include x in the interval produces 0:')
sol4 = integrate(DiracDelta(x - d), (x, -oo, d-h))
print(sol4)
sol5 = integrate(DiracDelta(x - d), (x, d+h, oo))
print(sol5,'\n')
print('Integrating DiracDelta(x) when one of the bounds is 0 produces Heaviside(0), which is defined to be 1/2. \nBut some versions of Sympy don\'t evalute this:')
sol6 = integrate(DiracDelta(x - d), (x, -oo, d))
print(sol6)
sol7 = integrate(DiracDelta(x - d), (x, d, oo))
print(sol7,'\n')
print('Heaviside(x) = 1 for x > 0, -1 for x < 0, and 1/2 for x = 0. \nBut some versions of Sympy don\'t evaluate Heaviside(0):')
print(Heaviside(1))
print(Heaviside(-1))
print(Heaviside(0))
```

    DiracDelta(x) for any value but 0 is 0:
    0
    0

    DiracDelta(0) is undefined. In at least some versions of Sympy, it is unevaluated:
    DiracDelta(0)

    Integrating DiracDelta(x) for bounds that include x in the interval produces 1:
    1
    1
    1

    Integrating DiracDelta(x) using bounds that don't include x in the interval produces 0:
    0
    0

    Integrating DiracDelta(x) when one of the bounds is 0 produces Heaviside(0), which is defined to be 1/2.
    But some versions of Sympy don't evalute this:
    Heaviside(0)
    1 - Heaviside(0)

    Heaviside(x) = 1 for x > 0, -1 for x < 0, and 1/2 for x = 0.
    But some versions of Sympy don't evaluate Heaviside(0):
    1
    0
    Heaviside(0)

Here is an example forced differential equation involving DiracDelta with analytic and graphical solutions:

$\hspace{0.5in} y'(t) + 0.45y(t) = 8\mathrm{DiracDelta}(t-4)$ for $y(0) = 9$

```python
init_printing()
from sympy.plotting import plot
t = symbols('t')
y = Function('y')
y1 = Derivative(y(t), t)
eqdiff = y1 + 0.45*y(t) - 8*DiracDelta(t - 4)
sol1 = dsolve(eqdiff, y(t), ics={y(0): '9'})
sol1
```

```python
xtol = 1e-3
p1 = plot(sol1.rhs, show=False, xlim=[-0.5,10], ylim=[0,10], ylabel='y(t)')
p1.show()
```

The DiracDelta function provides an impulse of 8 units at $t = 4$.

In the following function, an impulse of $d$ units for integral values on the interval $1 <= t <= 9$.

$\hspace{0.5in} y'(t) + 0.04y(t) = \sum_{k=1}^{9}d\mathrm{DiracDelta}(t-k)$ for $y(0) = 1000$

```python
t, k = symbols('t k')
y = Function('y')
y1 = Derivative(y(t), t)
r = 0.04
yinit = 1000
d = 200
f = Sum(d*DiracDelta(t - k),(k,1,9)).doit()
# print(f)
eqdiff = y1 - r*y(t) - f
#print(eqdiff)
sol2 = dsolve(eqdiff, y(t), ics={y(0): yinit})
sol2
```

```python
xtol = 1e-3
p2 = plot(sol2.rhs, show=False, xlim=[0,10], axis_center=(0,900), ylabel='y(t)')
p2.show()
```

A comparison to a constant forcing function of 200:

```python
t = symbols('t')
y = Function('y')
y1 = Derivative(y(t), t)
eqdiff = y1 - 0.04*y(t) - 200
sol3 = dsolve(eqdiff, y(t), ics={y(0): '1000'})
xtol = 1e-3
p3 = plot(sol3.rhs, show=False, xlim=[0,10], ylim=[500,4000], axis_center=(0,500), ylabel='y(t)')
p3.show()
```

Written by Dan Liddell. October, 2021.

These sources were consulted in preparing this content and provided ideas, examples, and source code for this material:

https://personal.math.ubc.ca/~pwalls/math-python/

https://www.scipy.org/docs.html

https://stackexchange.com/

https://stackoverflow.com/

A generous amount of credit goes to the following:

    Davis, Bill and Jerry Uhl. Differential Equations&Mathematica [sic]. version 6.0.
    Math Everywhere, Inc., 2007. Published as a Mathematica notebook.

    Math 285 -- Introduction to Differential Equations course at University of Illinois at Urbana-Champaign.

```python

```