# Assignment 1
*Name (ID)*: Vinh Nguyen (1029531)
*Workshop*: Friday 04:15 PM

---

## Setup

```
import sys
from pathlib import Path

# Notebook Library
url = "https://github.com/Data-Science-and-Data-Analytics-Courses/Notebook-Library"
repo = Path("/nblib")
!git clone "{url}" "{repo}"
if repo.parent.as_posix() not in sys.path:
  sys.path.append(repo.parent.as_posix())
%run "{repo}/.Importable.ipynb"

from nblib.imports.Basic import *
from nblib import Git

# Remote
URL = "https://github.com/Data-Science-and-Data-Analytics-Courses/UniMelb-A-First-Course-In-Statistical-Learning-MAST90104_2019_SM2"
REPO = Git.clone(URL, dest="/content")
if REPO.as_posix() not in sys.path:
  sys.path.append(REPO.as_posix())

# Working directory
os.chdir(REPO)

from nblib.imports.RPy import *
from setup.Setup import *

clear_output()
```

## Problem 1
Suppose that A is a symmetric matrix with $A^k = A^{k+1}$ for some integer k ≥ 1. Show that A is
idempotent.

Suppose $A: (n,  n) $ matrix

Since A symmetric,
$$
\begin{align}
A &= P\Lambda P^T \\
\end{align}
$$
where:
$\Lambda$: (n, n) diagonal matrix, entries eigen values of A
$P$: (n, n) orthogonal matrix, columns eigen vectors of A (corresponding to eigen values)

Calculate $A^k, A^{k+1}$:
$$
\begin{align}
A^k &= \underbrace{\left(P\Lambda P^T\right)\left(P\Lambda P^T\right) \dots \left(P\Lambda P^T\right)}_\text{k terms} \\
&= P\Lambda (P^TP)\Lambda (P^TP) \dots (P^TP)\Lambda P^T & (P^TP=I) \\
&= P\Lambda^k P^T \\
A^{k+1} &= P\Lambda^{k+1} P^T & (\text{similarly}) \\
\end{align}
$$

Plugging in:
$$
\begin{align}
A^k &= A^{k+1} \\
P\Lambda^k P^T &= P\Lambda^{k+1} P^T \\
\Lambda^k &= \Lambda^{k+1} \quad (\text{left-multiply and right-multiply on both sides with } P^T, P) \\
\end{align}
$$

$N^{th}$ power of diagonal matrix:
$$
\begin{align}
\Lambda^k &= \begin{bmatrix}
    \lambda_1 & 0 & \dots & 0 \\
    0 & \lambda_2 & \dots & 0 \\
    \vdots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \lambda_n
\end{bmatrix}^k \\
&= \begin{bmatrix}
    \lambda_1^k & 0 & \dots & 0 \\
    0 & \lambda_2^k & \dots & 0 \\
    \vdots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \lambda_n^k
\end{bmatrix}
\end{align}
$$

Solve:
$$
\begin{align}
\Lambda^k - \Lambda^{k+1} &= 0 \\
\begin{bmatrix}
    \lambda_1^k - \lambda_1^{k+1} & 0 & \dots & 0 \\
    0 & \lambda_2^k - \lambda_2^{k+1} & \dots & 0 \\
    \vdots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \lambda_n^k - \lambda_n^{k+1}
\end{bmatrix} &= 0 \\
\lambda_i^k - \lambda_i^{k+1} &= 0 \\
\lambda_i^k (1 - \lambda_i) &= 0 \\
\lambda_i &\in \{0, 1\}  \\
\end{align}
$$
Which implies $\Lambda$ (diagonal matrix with entries 0's or 1's) idempotent.

Come back:
$$
\begin{align}
A &= P\Lambda P^T \\
A^{2} &= P\Lambda^{2} P^T = P\Lambda P^T \quad (\Lambda^{2} = \Lambda) \\
\implies A &= A^2 \\
\end{align}
$$

Thus, A idempotent.

## Problem 2
Let $A_1, A_2, . . . , A_m$ be a set of symmetric k × k matrices. Suppose that there exists an orthogonal
matrix P such that $P^T A_i P$ is diagonal for all i. Show that $A_i A_j = A_j A_i$ for every pair i, j =
1, 2, . . . , m.

Let $D_i = P^T A_i P$ (diagonal)
$$
\begin{align}
D_i D_j &= (P^T A_i P)(P^T A_j P) \\
&= P^T A_i (P P^T) A_j P \\
&= P^T A_i A_j P & (P P^T = I) \\
\Rightarrow A_i A_j &= P D_i D_j P^T \\
A_j A_i &= P D_j D_i P^T & \text{(similarly)} \\
D_i D_j &= D_j D_i = D & (\text{D diagonal, entries element-wise products of } D_i's, D_j's) \\
\Rightarrow A_i A_j &= A_j A_i \\
\end{align}
$$

## Problem 3
Show that for any random vector y and compatible matrix A, we have $\text{var } Ay = A(\text{var } y)A^T$.

$$
\begin{align}
\text{var } Ay &= E\left[ (Ay-E[Ay]) (Ay-E[Ay])^T\right] \\
&= E\left[ (Ay-AE[y]) (Ay-AE[y])^T\right] \\
&= E\left[ A(y-E[y]) (y-E[y])^TA^T\right] \\
&= AE\left[ (y-E[y]) (y-E[y])^T\right]A^T \\
&= A(\text{var }y)A^T \\
\end{align}
$$

## Problem 4
Let y be a 3-dimensional multivariate normal random vector with mean and variance
$$
µ = \begin{bmatrix}
3 \\
0 \\
−2 \\
\end{bmatrix} \quad
V = \begin{bmatrix}
2 & 0 & 0 \\
0 & 2 & 0 \\
0 & 0 & 1 \\
\end{bmatrix}
$$

$$
A = \frac{1}{10} \begin{bmatrix}
4 & −2 & 0 \\
−2 & 1 & 0 \\
0 & 0 & 10 \\
\end{bmatrix}
$$

(a) Describe the distribution of Ay.
Let $w = Ay \Rightarrow w \sim MVN(\mu_w=A\mu, V_w=AVA^T)$

```r
%%R

mu <- c(3, 0, -2)
v <- matrix(c(2, 0, 0, 0, 2, 0, 0, 0, 1), nrow=3, ncol=3, byrow=TRUE)
a <- 1/10 * matrix(c(4, -2, 0, -2, 1, 0, 0, 0, 10), nrow=3, ncol=3, byrow=TRUE)

print("Mean of Ay")
print(a %*% mu)

print("Variance of Ay")
print(a %*% v %*% t(a))
```

    [1] "Mean of Ay"
         [,1]
    [1,]  1.2
    [2,] -0.6
    [3,] -2.0
    [1] "Variance of Ay"
         [,1] [,2] [,3]
    [1,]  0.4 -0.2    0
    [2,] -0.2  0.1    0
    [3,]  0.0  0.0    1

(b) Find $E[y^T Ay]$.
$E[y^T Ay] = tr(AV ) + µ^T Aµ$

```r
%%R

print("Expected value of quadratic form")
sum(diag(a%*%v)) + t(mu) %*% a %*% mu
```

    [1] "Expected value of quadratic form"
         [,1]
    [1,]  9.6

(c) Describe the distribution of $y^T Ay$.
Calculate and compare $AV$, $(AV)^2$

```r
%%R

print("AV")
av <- a%*%v
print(av)

print("(AV) squared")
av_sq <- av%*%av
print(av_sq)

print("AV = (AV) squared?")
print(all.equal(av, av_sq))
```

    [1] "AV"
         [,1] [,2] [,3]
    [1,]  0.8 -0.4    0
    [2,] -0.4  0.2    0
    [3,]  0.0  0.0    1
    [1] "(AV) squared"
         [,1] [,2] [,3]
    [1,]  0.8 -0.4    0
    [2,] -0.4  0.2    0
    [3,]  0.0  0.0    1
    [1] "AV = (AV) squared?"
    [1] TRUE

Since $AV = (AV)^2$, $AV$ idempotent

```r
%%R

print("Rank of AV")
print(qr(av)$rank)
```

    [1] "Rank of AV"
    [1] 2

And, rank of AV = 2  (A symmetric)
Hence, $y^T Ay \sim \chi^2_{k=2, \lambda=\frac{1}{2}µ^T Aµ}$

```r
%%R

print("λ")
print(1/2 * t(mu) %*% a %*% mu)
```

    [1] "λ"
         [,1]
    [1,]  3.8

(d) Find all linear combinations of y elements which are independent of $y^T Ay$.
Let $b^T = [b_1, b_2, b_3]$ be multiplying factors for y elements.
Linear combinations of y elements: $b^Ty$
$y^T Ay$ and $b^Ty$ are independent if and only if $b^TV A = 0$ or $(V A)^Tb = 0$.

```r
%%R

print("Transpose of (VA)")
t(v%*%a)
```

    [1] "Transpose of (VA)"
         [,1] [,2] [,3]
    [1,]  0.8 -0.4    0
    [2,] -0.4  0.2    0
    [3,]  0.0  0.0    1

$$
\begin{align}
(VA)^Tb &= \begin{bmatrix}
0.8 & -0.4 & 0 \\
-0.4 & 0.2 & 0 \\
0.0 & 0.0 & 1 \\
\end{bmatrix}
\begin{bmatrix}
b_1 \\
b_2 \\
b_3 \\
\end{bmatrix} \\
&= \begin{bmatrix}
0.8b_1-0.4b_2 \\
-0.4b_1 + 0.2b_2\\
b_3 \\
\end{bmatrix} \\
&\stackrel{\text{set}}{=} 0 \\
\end{align}
$$

Solve:
$$
\begin{align}
\begin{bmatrix}
0.8b_1-0.4b_2 \\
-0.4b_1 + 0.2b_2\\
b_3 \\
\end{bmatrix} &= 0 \\
\begin{bmatrix}
2b_1-b_2 \\
b_3 \\
\end{bmatrix} &= 0 \\
\Rightarrow \left\{ \begin{array}{ll}
2b_1-b_2 &= 0 \\
b_3 &= 0 \end{array} \right. & \\
\Rightarrow \left\{ \begin{array}{ll}
b_1 &= a \\
b_2 &= 2a \\
b_3 &= 0 \end{array} \right. & & a \in \rm I\!R
\end{align}
$$

All linear combinations of y elements which are independent of $y^TAy$:
$$
\begin{align}
b^Ty &= \begin{bmatrix}
b_1 & b_2 & b_3 \\
\end{bmatrix} \begin{bmatrix}
y_1 \\
y_2 \\
y_3 \\
\end{bmatrix} \\
&= ay_1 + 2ay_2 \quad (a \in \rm I\!R)
\end{align}
$$

## Problem 5
The table below shows prices in US cents per pound received by fishermen and vessel owners for
various species of fish and shellfish in 1970 and 1980. (Taken from Moore & McCabe, Introduction
to the Practice of Statistics, 1989.)
Type of fish Price (1970) Price (1980)
Cod 13.1 27.3
Flounder 15.3 42.4
Haddock 25.8 38.7
Menhaden 1.8 4.5
Ocean perch 4.9 23.0
Salmon, chinook 55.4 166.3
Salmon, coho 39.3 109.7
Tuna, albacore 26.7 80.1
Clams, soft-shelled 47.5 150.7
Clams, blue hard-shelled 6.6 20.3
Lobsters, american 94.7 189.7
Oysters, eastern 61.1 131.3
Sea scallops 135.6 404.2
Shrimp 47.6 149.0

We will model this data using a linear model.

(a) The linear model is of the form y = Xβ + ε. Write down the matrices and vectors involved
in this equation.
$$
\begin{align}
y &= \begin{bmatrix}
    27.3 \\
    42.4 \\
    38.7 \\
    4.5 \\
    23.0 \\
    166.3 \\
    109.7 \\
    80.1 \\
    150.7 \\
    20.3 \\
    189.7 \\
    131.3 \\
    404.2 \\
    149.0 \\
\end{bmatrix} & (\text{Price (1980)}) \\
X &= \begin{bmatrix}
1 & 13.1 \\
1 & 15.3 \\
1 & 25.8 \\
1 & 1.8 \\
1 & 4.9 \\
1 & 55.4 \\
1 & 39.3 \\
1 & 26.7 \\
1 & 47.5 \\
1 & 6.6 \\
1 & 94.7 \\
1 & 61.1 \\
1 & 135.6 \\
1 & 47.6 \\
\end{bmatrix} & (\text{Price (1970)}) \\
\end{align}
$$

$$
\begin{align}
\beta &= \begin{bmatrix}
    \beta_0 \\
    \beta_1 \\
\end{bmatrix} \quad \epsilon = \begin{bmatrix}
    \epsilon_1 \\
    \epsilon_2 \\
    \vdots \\
    \epsilon_{14} \\
\end{bmatrix} \sim MVN(0, \lambda^2 I) \\
\end{align}
$$

(b) Using matrices, find the least squares estimators of the parameters.

```r
%%R

y <- c(27.3, 42.4, 38.7, 4.5, 23.0, 166.3, 109.7, 80.1, 150.7, 20.3, 189.7, 131.3, 404.2, 149.0)
x <- matrix(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       13.1, 15.3, 25.8, 1.8, 4.9, 55.4, 39.3, 26.7, 47.5, 6.6, 94.7, 61.1, 135.6, 47.6),
      nrow=14, ncol=2)
print(y)
print(x)

print("Least squares estimators of the parameters b")
b <- solve(t(x)%*%x) %*% t(x) %*% y
print(b)
```

     [1]  27.3  42.4  38.7   4.5  23.0 166.3 109.7  80.1 150.7  20.3 189.7 131.3
    [13] 404.2 149.0
          [,1]  [,2]
     [1,]    1  13.1
     [2,]    1  15.3
     [3,]    1  25.8
     [4,]    1   1.8
     [5,]    1   4.9
     [6,]    1  55.4
     [7,]    1  39.3
     [8,]    1  26.7
     [9,]    1  47.5
    [10,]    1   6.6
    [11,]    1  94.7
    [12,]    1  61.1
    [13,]    1 135.6
    [14,]    1  47.6
    [1] "Least squares estimators of the parameters b"
              [,1]
    [1,] -1.233836
    [2,]  2.701553

(c) Calculate the sample variance s
2
.

$$
s^2 = \frac{(y − Xb)^T (y − Xb)}{n − (k + 1)}
$$
where:
$(y − Xb)^T (y − Xb)$: sum of squares of the residuals
$n$: number of samples
$k$: number of features

```r
%%R

n <- length(y)
k <- ncol(x) - 1
print("Residuals")
res <- y - x%*%b
print(res)

print("Sample variance")
var_samp <- t(res) %*% res / (n-k-1)
print(var_samp)
```

    [1] "Residuals"
                 [,1]
     [1,]  -6.8565106
     [2,]   2.3000724
     [3,] -29.7662361
     [4,]   0.8710405
     [5,]  10.9962256
     [6,]  17.8677893
     [7,]   4.7627957
     [8,]   9.2023660
     [9,]  23.6100596
    [10,]   3.7035852
    [11,] -64.9032511
    [12,] -32.5310639
    [13,]  39.1032233
    [14,]  21.6399042
    [1] "Sample variance"
             [,1]
    [1,] 777.1528

(d) A fisherman sold ocean trout for 18c/pound in 1970. Predict the price for ocean trout in 1980.

```r
%%R

print("Predicted price for ocean trout in 1980")
print(c(1, 18) %*% b)
```

    [1] "Predicted price for ocean trout in 1980"
             [,1]
    [1,] 47.39412

(e) Calculate the standardised residual for sea scallops.

$$
z_i = \frac{e_i}{\sqrt{s^2(1 − H_{ii})}}
$$

```r
%%R

print("Hat matrix (first rows)")
H <- x %*% solve(t(x)%*%x) %*% t(x)
head(H)
```

    [1] "Hat matrix (first rows)"
               [,1]       [,2]       [,3]       [,4]       [,5]       [,6]
    [1,] 0.11396694 0.11062464 0.09467275 0.13113421 0.12642461 0.04970362
    [2,] 0.11062464 0.10754495 0.09284642 0.12644305 0.12210349 0.05141058
    [3,] 0.09467275 0.09284642 0.08412986 0.10405344 0.10147998 0.05955744
    [4,] 0.13113421 0.12644305 0.10405344 0.15522970 0.14861943 0.04093605
    [5,] 0.12642461 0.12210349 0.10147998 0.14861943 0.14253059 0.04334131
    [6,] 0.04970362 0.05141058 0.05955744 0.04093605 0.04334131 0.08252382
               [,7]       [,8]       [,9]      [,10]        [,11]      [,12]
    [1,] 0.07416318 0.09330545 0.06170552 0.12384192 -0.010002020 0.04104402
    [2,] 0.07394832 0.09158655 0.06246947 0.11972373 -0.003603902 0.04343138
    [3,] 0.07292284 0.08338272 0.06611562 0.10006872  0.026932570 0.05482559
    [4,] 0.07526679 0.10213433 0.05778157 0.14499445 -0.042865080 0.02878169
    [5,] 0.07496403 0.09971225 0.05885805 0.13919154 -0.033849550 0.03214569
    [6,] 0.07003197 0.06025574 0.07639427 0.04466033  0.113016338 0.08694639
                [,13]      [,14]
    [1,] -0.072138423 0.06155359
    [2,] -0.060858159 0.06232948
    [3,] -0.007020536 0.06603260
    [4,] -0.130077960 0.05756833
    [5,] -0.114183042 0.05866164
    [6,]  0.144750286 0.07647186

```r
%%R

ix <- 13 # index to sea scallops
print("Standardised residual for sea scallops")
z_sea <- res[ix] / sqrt(var_samp*(1 - H[ix, ix]))
print(z_sea)
```

    [1] "Standardised residual for sea scallops"
             [,1]
    [1,] 2.104999

(f) Calculate the Cook’s distance for sea scallops.

$$
D_i = \frac{1}{k + 1}z_i^2 \frac{H_{ii}}{1 − H_{ii}}
$$

where:
$z_i$: standardised residual

```r
%%R

print("Cook’s distance for sea scallops")
print(1/(k+1) * z_sea^2 * H[ix, ix]/(1-H[ix, ix]))
```

    [1] "Cook’s distance for sea scallops"
             [,1]
    [1,] 2.774008

(g) Does sea scallops fit the linear model? Justify your argument.

Since Cook’s distance for sea scallops > 1, it can be considered to be large. Thus, sea scallops do not well fit the linear model, as a general rule.
Visually on the graph below, point 13 (sea scallops) appears to be far away from the rest of the points.

```r
%%R

plot(x[, 2], y,
     main= "Price (1980) vs. Price (1970)",
     xlab= "Price (1970)",
     ylab= "Price (1980)",
     type= "n")
text(x[, 2], y, labels=1:length(y))
points(x[13, 2], y[13], pch=1, col="blue", cex=3, add=TRUE) # add circle
```

```
!apt-get update
!sudo apt-get install pandoc
!sudo apt-get install texlive-xetex --upgrade
```

    Get:1 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ InRelease [3,626 B]
    Ign:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease
    Ign:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease
    Get:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release [564 B]
    Get:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release [564 B]
    Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release.gpg [819 B]
    Get:7 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release.gpg [833 B]
    Get:8 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
    Get:9 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ Packages [65.9 kB]
    Get:10 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease [21.3 kB]
    Hit:11 http://archive.ubuntu.com/ubuntu bionic InRelease
    Get:12 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Packages [103 kB]
    Get:13 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
    Get:14 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Packages [12.3 kB]
    Get:15 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB]
    Get:16 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic/main amd64 Packages [29.0 kB]
    Get:17 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [6,222 B]
    Get:18 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4,173 B]
    Get:19 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [612 kB]
    Get:20 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,677 kB]
    Get:21 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [736 kB]
    Get:22 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
    Get:23 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [805 kB]
    Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,263 kB]
    Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [921 kB]
    Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [7,216 B]
    Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [16.8 kB]
    Fetched 6,553 kB in 38s (172 kB/s)
    Reading package lists... Done
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    pandoc is already the newest version (1.19.2.4~dfsg-1build4).
    pandoc set to manually installed.
    The following package was automatically installed and is no longer required:
      libnvidia-common-410
    Use 'sudo apt autoremove' to remove it.
    0 upgraded, 0 newly installed, 0 to remove and 66 not upgraded.
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following package was automatically installed and is no longer required:
      libnvidia-common-410
    Use 'sudo apt autoremove' to remove it.
    The following additional packages will be installed:
      fonts-droid-fallback fonts-lato fonts-lmodern fonts-noto-mono fonts-texgyre
      javascript-common libcupsfilters1 libcupsimage2 libgs9 libgs9-common
      libijs-0.35 libjbig2dec0 libjs-jquery libkpathsea6 libpotrace0 libptexenc1
      libruby2.5 libsynctex1 libtexlua52 libtexluajit2 libzzip-0-13 lmodern
      poppler-data preview-latex-style rake ruby ruby-did-you-mean ruby-minitest
      ruby-net-telnet ruby-power-assert ruby-test-unit ruby2.5
      rubygems-integration t1utils tex-common tex-gyre texlive-base
      texlive-binaries texlive-fonts-recommended texlive-latex-base
      texlive-latex-extra texlive-latex-recommended texlive-pictures
      texlive-plain-generic tipa
    Suggested packages:
      fonts-noto apache2 | lighttpd | httpd poppler-utils ghostscript
      fonts-japanese-mincho | fonts-ipafont-mincho fonts-japanese-gothic
      | fonts-ipafont-gothic fonts-arphic-ukai fonts-arphic-uming fonts-nanum ri
      ruby-dev bundler debhelper gv | postscript-viewer perl-tk xpdf-reader
      | pdf-viewer texlive-fonts-recommended-doc texlive-latex-base-doc
      python-pygments icc-profiles libfile-which-perl
      libspreadsheet-parseexcel-perl texlive-latex-extra-doc
      texlive-latex-recommended-doc texlive-pstricks dot2tex prerex ruby-tcltk
      | libtcltk-ruby texlive-pictures-doc vprerex
    The following NEW packages will be installed:
      fonts-droid-fallback fonts-lato fonts-lmodern fonts-noto-mono fonts-texgyre
      javascript-common libcupsfilters1 libcupsimage2 libgs9 libgs9-common
      libijs-0.35 libjbig2dec0 libjs-jquery libkpathsea6 libpotrace0 libptexenc1
      libruby2.5 libsynctex1 libtexlua52 libtexluajit2 libzzip-0-13 lmodern
      poppler-data preview-latex-style rake ruby ruby-did-you-mean ruby-minitest
      ruby-net-telnet ruby-power-assert ruby-test-unit ruby2.5
      rubygems-integration t1utils tex-common tex-gyre texlive-base
      texlive-binaries texlive-fonts-recommended texlive-latex-base
      texlive-latex-extra texlive-latex-recommended texlive-pictures
      texlive-plain-generic texlive-xetex tipa
    0 upgraded, 46 newly installed, 0 to remove and 66 not upgraded.
    Need to get 146 MB of archives.
    After this operation, 460 MB of additional disk space will be used.
    Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 fonts-droid-fallback all 1:6.0.1r16-1.1 [1,805 kB]
    Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 fonts-lato all 2.0-2 [2,698 kB]
    Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 poppler-data all 0.4.8-2 [1,479 kB]
    Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 tex-common all 6.09 [33.0 kB]
    Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 fonts-lmodern all 2.004.5-3 [4,551 kB]
    Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 fonts-noto-mono all 20171026-2 [75.5 kB]
    Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 fonts-texgyre all 20160520-1 [8,761 kB]
    Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 javascript-common all 11 [6,066 B]
    Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libcupsfilters1 amd64 1.20.2-0ubuntu3.1 [108 kB]
    Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libcupsimage2 amd64 2.2.7-1ubuntu2.6 [18.6 kB]
    Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libijs-0.35 amd64 0.35-13 [15.5 kB]
    Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjbig2dec0 amd64 0.13-6 [55.9 kB]
    Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgs9-common all 9.26~dfsg+0-0ubuntu0.18.04.10 [5,092 kB]
    Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgs9 amd64 9.26~dfsg+0-0ubuntu0.18.04.10 [2,263 kB]
    Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-jquery all 3.2.1-1 [152 kB]
    Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libkpathsea6 amd64 2017.20170613.44572-8ubuntu0.1 [54.9 kB]
    Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpotrace0 amd64 1.14-2 [17.4 kB]
    Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libptexenc1 amd64 2017.20170613.44572-8ubuntu0.1 [34.5 kB]
    Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 rubygems-integration all 1.11 [4,994 B]
    Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ruby2.5 amd64 2.5.1-1ubuntu1.4 [48.6 kB]
    Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby amd64 1:2.5.1 [5,712 B]
    Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 rake all 12.3.1-1 [45.1 kB]
    Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby-did-you-mean all 1.2.0-2 [9,700 B]
    Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby-minitest all 5.10.3-1 [38.6 kB]
    Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby-net-telnet all 0.1.1-2 [12.6 kB]
    Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby-power-assert all 0.3.0-1 [7,952 B]
    Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 ruby-test-unit all 3.2.5-1 [61.1 kB]
    Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libruby2.5 amd64 2.5.1-1ubuntu1.4 [3,068 kB]
    Get:29 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsynctex1 amd64 2017.20170613.44572-8ubuntu0.1 [41.4 kB]
    Get:30 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libtexlua52 amd64 2017.20170613.44572-8ubuntu0.1 [91.2 kB]
    Get:31 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libtexluajit2 amd64 2017.20170613.44572-8ubuntu0.1 [230 kB]
    Get:32 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzzip-0-13 amd64 0.13.62-3.1ubuntu0.18.04.1 [26.0 kB]
    Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 lmodern all 2.004.5-3 [9,631 kB]
    Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 preview-latex-style all 11.91-1ubuntu1 [185 kB]
    Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 t1utils amd64 1.41-2 [56.0 kB]
    Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 tex-gyre all 20160520-1 [4,998 kB]
    Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 texlive-binaries amd64 2017.20170613.44572-8ubuntu0.1 [8,179 kB]
    Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 texlive-base all 2017.20180305-1 [18.7 MB]
    Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 texlive-fonts-recommended all 2017.20180305-1 [5,262 kB]
    Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 texlive-latex-base all 2017.20180305-1 [951 kB]
    Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 texlive-latex-recommended all 2017.20180305-1 [14.9 MB]
    Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 texlive-pictures all 2017.20180305-1 [4,026 kB]
    Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 texlive-latex-extra all 2017.20180305-2 [10.6 MB]
    Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 texlive-plain-generic all 2017.20180305-2 [23.6 MB]
    Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 tipa all 2:1.3-20 [2,978 kB]
    Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 texlive-xetex all 2017.20180305-1 [10.7 MB]
    Fetched 146 MB in 8s (17.5 MB/s)
    debconf: unable to initialize frontend: Dialog
    debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 46.)
    debconf: falling back to frontend: Readline
    debconf: unable to initialize frontend: Readline
    debconf: (This frontend requires a controlling tty.)
    debconf: falling back to frontend: Teletype
    dpkg-preconfigure: unable to re-open stdin:
    Selecting previously unselected package fonts-droid-fallback.
    (Reading database ... 131289 files and directories currently installed.)
    Preparing to unpack .../00-fonts-droid-fallback_1%3a6.0.1r16-1.1_all.deb ...
    Unpacking fonts-droid-fallback (1:6.0.1r16-1.1) ...
    Selecting previously unselected package fonts-lato.
    Preparing to unpack .../01-fonts-lato_2.0-2_all.deb ...
    Unpacking fonts-lato (2.0-2) ...
    Selecting previously unselected package poppler-data.
    Preparing to unpack .../02-poppler-data_0.4.8-2_all.deb ...
    Unpacking poppler-data (0.4.8-2) ...
    Selecting previously unselected package tex-common.
    Preparing to unpack .../03-tex-common_6.09_all.deb ...
    Unpacking tex-common (6.09) ...
    Selecting previously unselected package fonts-lmodern.
    Preparing to unpack .../04-fonts-lmodern_2.004.5-3_all.deb ...
    Unpacking fonts-lmodern (2.004.5-3) ...
    Selecting previously unselected package fonts-noto-mono.
    Preparing to unpack .../05-fonts-noto-mono_20171026-2_all.deb ...
    Unpacking fonts-noto-mono (20171026-2) ...
    Selecting previously unselected package fonts-texgyre.
    Preparing to unpack .../06-fonts-texgyre_20160520-1_all.deb ...
    Unpacking fonts-texgyre (20160520-1) ...
    Selecting previously unselected package javascript-common.
    Preparing to unpack .../07-javascript-common_11_all.deb ...
    Unpacking javascript-common (11) ...
    Selecting previously unselected package libcupsfilters1:amd64.
    Preparing to unpack .../08-libcupsfilters1_1.20.2-0ubuntu3.1_amd64.deb ...
    Unpacking libcupsfilters1:amd64 (1.20.2-0ubuntu3.1) ...
    Selecting previously unselected package libcupsimage2:amd64.
    Preparing to unpack .../09-libcupsimage2_2.2.7-1ubuntu2.6_amd64.deb ...
    Unpacking libcupsimage2:amd64 (2.2.7-1ubuntu2.6) ...
    Selecting previously unselected package libijs-0.35:amd64.
    Preparing to unpack .../10-libijs-0.35_0.35-13_amd64.deb ...
    Unpacking libijs-0.35:amd64 (0.35-13) ...
    Selecting previously unselected package libjbig2dec0:amd64.
    Preparing to unpack .../11-libjbig2dec0_0.13-6_amd64.deb ...
    Unpacking libjbig2dec0:amd64 (0.13-6) ...
    Selecting previously unselected package libgs9-common.
    Preparing to unpack .../12-libgs9-common_9.26~dfsg+0-0ubuntu0.18.04.10_all.deb ...
    Unpacking libgs9-common (9.26~dfsg+0-0ubuntu0.18.04.10) ...
    Selecting previously unselected package libgs9:amd64.
    Preparing to unpack .../13-libgs9_9.26~dfsg+0-0ubuntu0.18.04.10_amd64.deb ...
    Unpacking libgs9:amd64 (9.26~dfsg+0-0ubuntu0.18.04.10) ...
    Selecting previously unselected package libjs-jquery.
    Preparing to unpack .../14-libjs-jquery_3.2.1-1_all.deb ...
    Unpacking libjs-jquery (3.2.1-1) ...
    Selecting previously unselected package libkpathsea6:amd64.
    Preparing to unpack .../15-libkpathsea6_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking libkpathsea6:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package libpotrace0.
    Preparing to unpack .../16-libpotrace0_1.14-2_amd64.deb ...
    Unpacking libpotrace0 (1.14-2) ...
    Selecting previously unselected package libptexenc1:amd64.
    Preparing to unpack .../17-libptexenc1_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking libptexenc1:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package rubygems-integration.
    Preparing to unpack .../18-rubygems-integration_1.11_all.deb ...
    Unpacking rubygems-integration (1.11) ...
    Selecting previously unselected package ruby2.5.
    Preparing to unpack .../19-ruby2.5_2.5.1-1ubuntu1.4_amd64.deb ...
    Unpacking ruby2.5 (2.5.1-1ubuntu1.4) ...
    Selecting previously unselected package ruby.
    Preparing to unpack .../20-ruby_1%3a2.5.1_amd64.deb ...
    Unpacking ruby (1:2.5.1) ...
    Selecting previously unselected package rake.
    Preparing to unpack .../21-rake_12.3.1-1_all.deb ...
    Unpacking rake (12.3.1-1) ...
    Selecting previously unselected package ruby-did-you-mean.
    Preparing to unpack .../22-ruby-did-you-mean_1.2.0-2_all.deb ...
    Unpacking ruby-did-you-mean (1.2.0-2) ...
    Selecting previously unselected package ruby-minitest.
    Preparing to unpack .../23-ruby-minitest_5.10.3-1_all.deb ...
    Unpacking ruby-minitest (5.10.3-1) ...
    Selecting previously unselected package ruby-net-telnet.
    Preparing to unpack .../24-ruby-net-telnet_0.1.1-2_all.deb ...
    Unpacking ruby-net-telnet (0.1.1-2) ...
    Selecting previously unselected package ruby-power-assert.
    Preparing to unpack .../25-ruby-power-assert_0.3.0-1_all.deb ...
    Unpacking ruby-power-assert (0.3.0-1) ...
    Selecting previously unselected package ruby-test-unit.
    Preparing to unpack .../26-ruby-test-unit_3.2.5-1_all.deb ...
    Unpacking ruby-test-unit (3.2.5-1) ...
    Selecting previously unselected package libruby2.5:amd64.
    Preparing to unpack .../27-libruby2.5_2.5.1-1ubuntu1.4_amd64.deb ...
    Unpacking libruby2.5:amd64 (2.5.1-1ubuntu1.4) ...
    Selecting previously unselected package libsynctex1:amd64.
    Preparing to unpack .../28-libsynctex1_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking libsynctex1:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package libtexlua52:amd64.
    Preparing to unpack .../29-libtexlua52_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking libtexlua52:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package libtexluajit2:amd64.
    Preparing to unpack .../30-libtexluajit2_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking libtexluajit2:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package libzzip-0-13:amd64.
    Preparing to unpack .../31-libzzip-0-13_0.13.62-3.1ubuntu0.18.04.1_amd64.deb ...
    Unpacking libzzip-0-13:amd64 (0.13.62-3.1ubuntu0.18.04.1) ...
    Selecting previously unselected package lmodern.
    Preparing to unpack .../32-lmodern_2.004.5-3_all.deb ...
    Unpacking lmodern (2.004.5-3) ...
    Selecting previously unselected package preview-latex-style.
    Preparing to unpack .../33-preview-latex-style_11.91-1ubuntu1_all.deb ...
    Unpacking preview-latex-style (11.91-1ubuntu1) ...
    Selecting previously unselected package t1utils.
    Preparing to unpack .../34-t1utils_1.41-2_amd64.deb ...
    Unpacking t1utils (1.41-2) ...
    Selecting previously unselected package tex-gyre.
    Preparing to unpack .../35-tex-gyre_20160520-1_all.deb ...
    Unpacking tex-gyre (20160520-1) ...
    Selecting previously unselected package texlive-binaries.
    Preparing to unpack .../36-texlive-binaries_2017.20170613.44572-8ubuntu0.1_amd64.deb ...
    Unpacking texlive-binaries (2017.20170613.44572-8ubuntu0.1) ...
    Selecting previously unselected package texlive-base.
    Preparing to unpack .../37-texlive-base_2017.20180305-1_all.deb ...
    Unpacking texlive-base (2017.20180305-1) ...
    Selecting previously unselected package texlive-fonts-recommended.
    Preparing to unpack .../38-texlive-fonts-recommended_2017.20180305-1_all.deb ...
    Unpacking texlive-fonts-recommended (2017.20180305-1) ...
    Selecting previously unselected package texlive-latex-base.
    Preparing to unpack .../39-texlive-latex-base_2017.20180305-1_all.deb ...
    Unpacking texlive-latex-base (2017.20180305-1) ...
    Selecting previously unselected package texlive-latex-recommended.
    Preparing to unpack .../40-texlive-latex-recommended_2017.20180305-1_all.deb ...
    Unpacking texlive-latex-recommended (2017.20180305-1) ...
    Selecting previously unselected package texlive-pictures.
    Preparing to unpack .../41-texlive-pictures_2017.20180305-1_all.deb ...
    Unpacking texlive-pictures (2017.20180305-1) ...
    Selecting previously unselected package texlive-latex-extra.
    Preparing to unpack .../42-texlive-latex-extra_2017.20180305-2_all.deb ...
    Unpacking texlive-latex-extra (2017.20180305-2) ...
    Selecting previously unselected package texlive-plain-generic.
    Preparing to unpack .../43-texlive-plain-generic_2017.20180305-2_all.deb ...
    Unpacking texlive-plain-generic (2017.20180305-2) ...
    Selecting previously unselected package tipa.
    Preparing to unpack .../44-tipa_2%3a1.3-20_all.deb ...
    Unpacking tipa (2:1.3-20) ...
    Selecting previously unselected package texlive-xetex.
    Preparing to unpack .../45-texlive-xetex_2017.20180305-1_all.deb ...
    Unpacking texlive-xetex (2017.20180305-1) ...
    Setting up libgs9-common (9.26~dfsg+0-0ubuntu0.18.04.10) ...
    Setting up libkpathsea6:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Setting up libjs-jquery (3.2.1-1) ...
    Setting up libtexlua52:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Setting up fonts-droid-fallback (1:6.0.1r16-1.1) ...
    Setting up libsynctex1:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Setting up libptexenc1:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Setting up tex-common (6.09) ...
    debconf: unable to initialize frontend: Dialog
    debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
    debconf: falling back to frontend: Readline
    update-language: texlive-base not installed and configured, doing nothing!
    Processing triggers for mime-support (3.60ubuntu1) ...
    Setting up poppler-data (0.4.8-2) ...
    Setting up tex-gyre (20160520-1) ...
    Setting up preview-latex-style (11.91-1ubuntu1) ...
    Setting up fonts-texgyre (20160520-1) ...
    Setting up fonts-noto-mono (20171026-2) ...
    Setting up fonts-lato (2.0-2) ...
    Setting up libcupsfilters1:amd64 (1.20.2-0ubuntu3.1) ...
    Setting up libcupsimage2:amd64 (2.2.7-1ubuntu2.6) ...
    Setting up libjbig2dec0:amd64 (0.13-6) ...
    Setting up ruby-did-you-mean (1.2.0-2) ...
    Processing triggers for libc-bin (2.27-3ubuntu1) ...
    Setting up t1utils (1.41-2) ...
    Setting up ruby-net-telnet (0.1.1-2) ...
    Setting up libijs-0.35:amd64 (0.35-13) ...
    Setting up rubygems-integration (1.11) ...
    Setting up libpotrace0 (1.14-2) ...
    Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
    Setting up javascript-common (11) ...
    Setting up ruby-minitest (5.10.3-1) ...
    Setting up libzzip-0-13:amd64 (0.13.62-3.1ubuntu0.18.04.1) ...
    Setting up libgs9:amd64 (9.26~dfsg+0-0ubuntu0.18.04.10) ...
    Setting up libtexluajit2:amd64 (2017.20170613.44572-8ubuntu0.1) ...
    Processing triggers for fontconfig (2.12.6-0ubuntu2) ...
    Setting up fonts-lmodern (2.004.5-3) ...
    Setting up ruby-power-assert (0.3.0-1) ...
    Setting up texlive-binaries (2017.20170613.44572-8ubuntu0.1) ...
    update-alternatives: using /usr/bin/xdvi-xaw to provide /usr/bin/xdvi.bin (xdvi.bin) in auto mode
    update-alternatives: using /usr/bin/bibtex.original to provide /usr/bin/bibtex (bibtex) in auto mode
    Setting up texlive-base (2017.20180305-1) ...
    mktexlsr: Updating /var/lib/texmf/ls-R-TEXLIVEDIST...
    mktexlsr: Updating /var/lib/texmf/ls-R-TEXMFMAIN...
    mktexlsr: Updating /var/lib/texmf/ls-R...
    mktexlsr: Done.
    tl-paper: setting paper size for dvips to a4: /var/lib/texmf/dvips/config/config-paper.ps
    tl-paper: setting paper size for dvipdfmx to a4: /var/lib/texmf/dvipdfmx/dvipdfmx-paper.cfg
    tl-paper: setting paper size for xdvi to a4: /var/lib/texmf/xdvi/XDvi-paper
    tl-paper: setting paper size for pdftex to a4: /var/lib/texmf/tex/generic/config/pdftexconfig.tex
    debconf: unable to initialize frontend: Dialog
    debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
    debconf: falling back to frontend: Readline
    Setting up texlive-fonts-recommended (2017.20180305-1) ...
    Setting up texlive-plain-generic (2017.20180305-2) ...
    Setting up texlive-latex-base (2017.20180305-1) ...
    Setting up lmodern (2.004.5-3) ...
    Setting up texlive-latex-recommended (2017.20180305-1) ...
    Setting up texlive-pictures (2017.20180305-1) ...
    Setting up tipa (2:1.3-20) ...
    Regenerating '/var/lib/texmf/fmtutil.cnf-DEBIAN'... done.
    Regenerating '/var/lib/texmf/fmtutil.cnf-TEXLIVEDIST'... done.
    update-fmtutil has updated the following file(s):
    	/var/lib/texmf/fmtutil.cnf-DEBIAN
    	/var/lib/texmf/fmtutil.cnf-TEXLIVEDIST
    If you want to activate the changes in the above file(s),
    you should run fmtutil-sys or fmtutil.
    Setting up texlive-latex-extra (2017.20180305-2) ...
    Setting up texlive-xetex (2017.20180305-1) ...
    Setting up ruby2.5 (2.5.1-1ubuntu1.4) ...
    Setting up ruby (1:2.5.1) ...
    Setting up ruby-test-unit (3.2.5-1) ...
    Setting up rake (12.3.1-1) ...
    Setting up libruby2.5:amd64 (2.5.1-1ubuntu1.4) ...
    Processing triggers for tex-common (6.09) ...
    debconf: unable to initialize frontend: Dialog
    debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
    debconf: falling back to frontend: Readline
    Running updmap-sys. This may take some time... done.
    Running mktexlsr /var/lib/texmf ... done.
    Building format(s) --all.
    	This may take some time... done.
    Processing triggers for libc-bin (2.27-3ubuntu1) ...
    [NbConvertApp] WARNING | pattern u'/content/Assignment1.ipynb' matched no files
    This application is used to convert notebook files (*.ipynb) to various other
    formats.

    WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.

    Options
    -------

    Arguments that take values are actually convenience aliases to full
    Configurables, whose aliases are listed on the help line. For more information
    on full configurables, see '--help-all'.

    --execute
        Execute the notebook prior to export.
    --allow-errors
        Continue notebook execution even if one of the cells throws an error and include the error message in the cell output (the default behaviour is to abort conversion). This flag is only relevant if '--execute' was specified, too.
    --no-input
        Exclude input cells and output prompts from converted document.
        This mode is ideal for generating code-free reports.
    --stdout
        Write notebook output to stdout instead of files.
    --stdin
        read a single notebook file from stdin. Write the resulting notebook with default basename 'notebook.*'
    --inplace
        Run nbconvert in place, overwriting the existing notebook (only
        relevant when converting to notebook format)
    -y
        Answer yes to any questions instead of prompting.
    --clear-output
        Clear output of current file and save in place,
        overwriting the existing notebook.
    --debug
        set log level to logging.DEBUG (maximize logging output)
    --no-prompt
        Exclude input and output prompts from converted document.
    --generate-config
        generate default config file
    --nbformat=<Enum> (NotebookExporter.nbformat_version)
        Default: 4
        Choices: [1, 2, 3, 4]
        The nbformat version to write. Use this to downgrade notebooks.
    --output-dir=<Unicode> (FilesWriter.build_directory)
        Default: ''
        Directory to write output(s) to. Defaults to output to the directory of each
        notebook. To recover previous default behaviour (outputting to the current
        working directory) use . as the flag value.
    --writer=<DottedObjectName> (NbConvertApp.writer_class)
        Default: 'FilesWriter'
        Writer class used to write the  results of the conversion
    --log-level=<Enum> (Application.log_level)
        Default: 30
        Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
        Set the log level by value or name.
    --reveal-prefix=<Unicode> (SlidesExporter.reveal_url_prefix)
        Default: u''
        The URL prefix for reveal.js (version 3.x). This defaults to the reveal CDN,
        but can be any url pointing to a copy  of reveal.js.
        For speaker notes to work, this must be a relative path to a local  copy of
        reveal.js: e.g., "reveal.js".
        If a relative path is given, it must be a subdirectory of the current
        directory (from which the server is run).
        See the usage documentation
        (https://nbconvert.readthedocs.io/en/latest/usage.html#reveal-js-html-
        slideshow) for more details.
    --to=<Unicode> (NbConvertApp.export_format)
        Default: 'html'
        The export format to be used, either one of the built-in formats
        ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf',
        'python', 'rst', 'script', 'slides'] or a dotted object name that represents
        the import path for an `Exporter` class
    --template=<Unicode> (TemplateExporter.template_file)
        Default: u''
        Name of the template file to use
    --output=<Unicode> (NbConvertApp.output_base)
        Default: ''
        overwrite base name use for output files. can only be used when converting
        one notebook at a time.
    --post=<DottedOrNone> (NbConvertApp.postprocessor_class)
        Default: u''
        PostProcessor class used to write the results of the conversion
    --config=<Unicode> (JupyterApp.config_file)
        Default: u''
        Full path of a config file.

    To see all available configurables, use `--help-all`

    Examples
    --------

        The simplest way to use nbconvert is

        > jupyter nbconvert mynotebook.ipynb

        which will convert mynotebook.ipynb to the default format (probably HTML).

        You can specify the export format with `--to`.
        Options include ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf', 'python', 'rst', 'script', 'slides'].

        > jupyter nbconvert --to latex mynotebook.ipynb

        Both HTML and LaTeX support multiple output templates. LaTeX includes
        'base', 'article' and 'report'.  HTML includes 'basic' and 'full'. You
        can specify the flavor of the format used.

        > jupyter nbconvert --to html --template basic mynotebook.ipynb

        You can also pipe the output to stdout, rather than a file

        > jupyter nbconvert mynotebook.ipynb --stdout

        PDF is generated via latex

        > jupyter nbconvert mynotebook.ipynb --to pdf

        You can get (and serve) a Reveal.js-powered slideshow

        > jupyter nbconvert myslides.ipynb --to slides --post serve

        Multiple notebooks can be given at the command line in a couple of
        different ways:

        > jupyter nbconvert notebook*.ipynb
        > jupyter nbconvert notebook1.ipynb notebook2.ipynb

        or you can specify the notebooks list in a config file, containing::

            c.NbConvertApp.notebooks = ["my_notebook.ipynb"]

        > jupyter nbconvert --config mycfg.py

```
!jupyter nbconvert --to html "/content/Assignment1.ipynb"
```

    [NbConvertApp] Converting notebook /content/Assignment1.ipynb to html
    [NbConvertApp] Writing 326351 bytes to /content/Assignment1.html


```python
from resources.workspace import *
```

$
% START OF MACRO DEF
% DO NOT EDIT IN INDIVIDUAL NOTEBOOKS, BUT IN macros.py
%
\newcommand{\Reals}{\mathbb{R}}
\newcommand{\Expect}[0]{\mathbb{E}}
\newcommand{\NormDist}{\mathcal{N}}
%
\newcommand{\DynMod}[0]{\mathscr{M}}
\newcommand{\ObsMod}[0]{\mathscr{H}}
%
\newcommand{\mat}[1]{{\mathbf{{#1}}}}
%\newcommand{\mat}[1]{{\pmb{\mathsf{#1}}}}
\newcommand{\bvec}[1]{{\mathbf{#1}}}
%
\newcommand{\trsign}{{\mathsf{T}}}
\newcommand{\tr}{^{\trsign}}
\newcommand{\tn}[1]{#1}
\newcommand{\ceq}[0]{\mathrel{≔}}
%
\newcommand{\I}[0]{\mat{I}}
\newcommand{\K}[0]{\mat{K}}
\newcommand{\bP}[0]{\mat{P}}
\newcommand{\bH}[0]{\mat{H}}
\newcommand{\bF}[0]{\mat{F}}
\newcommand{\R}[0]{\mat{R}}
\newcommand{\Q}[0]{\mat{Q}}
\newcommand{\B}[0]{\mat{B}}
\newcommand{\C}[0]{\mat{C}}
\newcommand{\Ri}[0]{\R^{-1}}
\newcommand{\Bi}[0]{\B^{-1}}
\newcommand{\X}[0]{\mat{X}}
\newcommand{\A}[0]{\mat{A}}
\newcommand{\Y}[0]{\mat{Y}}
\newcommand{\E}[0]{\mat{E}}
\newcommand{\U}[0]{\mat{U}}
\newcommand{\V}[0]{\mat{V}}
%
\newcommand{\x}[0]{\bvec{x}}
\newcommand{\y}[0]{\bvec{y}}
\newcommand{\z}[0]{\bvec{z}}
\newcommand{\q}[0]{\bvec{q}}
\newcommand{\br}[0]{\bvec{r}}
\newcommand{\bb}[0]{\bvec{b}}
%
\newcommand{\bx}[0]{\bvec{\bar{x}}}
\newcommand{\by}[0]{\bvec{\bar{y}}}
\newcommand{\barB}[0]{\mat{\bar{B}}}
\newcommand{\barP}[0]{\mat{\bar{P}}}
\newcommand{\barC}[0]{\mat{\bar{C}}}
\newcommand{\barK}[0]{\mat{\bar{K}}}
%
\newcommand{\D}[0]{\mat{D}}
\newcommand{\Dobs}[0]{\mat{D}_{\text{obs}}}
\newcommand{\Dmod}[0]{\mat{D}_{\text{obs}}}
%
\newcommand{\ones}[0]{\bvec{1}}
\newcommand{\AN}[0]{\big( \I_N - \ones \ones\tr / N \big)}
%
% END OF MACRO DEF
$
In this tutorial we're going to code an EnKF implementation using numpy.

# The EnKF algorithm

As with the KF, the EnKF consists of the recursive application of
a forecast step and an analysis step.
This presentation follows the traditional template, presenting the EnKF as the "the Monte Carlo version of the KF
where the state covariance is estimated by the ensemble covariance".
It is not obvious that this postulated method should work;
indeed, it is only justified upon inspection of its properties,
deferred to later.

NB:
Since we're going to focus on a single filtering cycle (at a time),
the subscript $k$ is dropped. Moreover,

The superscript $f$ indicates that $\{\x_n^f\}_{n=1..N}$ is the forecast (prior) ensemble.

The superscript $a$ indicates that $\{\x_n^a\}_{n=1..N}$ is the analysis (posterior) ensemble.

### The forecast step
Suppose $\{\x_n^a\}_{n=1..N}$ is an iid. sample from $p(\x_{k-1} \mid \y_1,\ldots, \y_{k-1})$, which may or may not be Gaussian.

The forecast step of the EnKF consists of a Monte Carlo simulation
of the forecast dynamics for each $\x_n^a$:
$$
	\forall n, \quad \x^f_n = \DynMod(\x_n^a) + \q_n  \, , \\
$$
where $\{\q_n\}_{n=1..N}$ are sampled iid. from $\NormDist(\bvec{0},\Q)$,
or whatever noise model is assumed,
and $\DynMod$ is the model dynamics.
The dynamics could consist of *any* function, i.e. the EnKF can be applied with nonlinear models.

The ensemble, $\{\x_n^f\}_{n=1..N}$, is then an iid. sample from the forecast pdf,
$p(\x_k \mid \y_1,\ldots,\y_{k-1})$. This follows from the definition of the latter, so it is a relatively trivial idea and way to obtain this pdf. However, before Monte-Carlo methods were computationally feasible, the computation of the forecast pdf required computing the [Chapman-Kolmogorov equation](https://en.wikipedia.org/wiki/Chapman%E2%80%93Kolmogorov_equation), which constituted a major hurdle for filtering methods.

### The analysis update step
of the ensemble is given by:
$$\begin{align}
	\forall n, \quad \x^\tn{a}_n &= \x_n^\tn{f} + \barK \left\{\y - \br_n - \ObsMod(\x_n^\tn{f}) \right\}
	\, , \\
	\text{or,}\quad
	\E^\tn{a} &=  \E^\tn{f}  + \barK \left\{\y\ones\tr - \Dobs - \ObsMod(\E^\tn{f})  \right\} \, ,
    \tag{4}
\end{align}
$$
where the "observation perturbations", $\br_n$, are sampled iid. from the observation noise model, e.g. $\NormDist(\bvec{0},\R)$,
and form the columns of $\Dobs$,
and the observation operator (again, any type of function) $\ObsMod$ is applied column-wise to $\E^\tn{f}$.

The gain $\barK$ is defined by inserting the ensemble estimates for
 * (i) $\B \bH\tr$: the cross-covariance between $\x^\tn{f}$ and $\ObsMod(\x^\tn{f})$, and
 * (ii) $\bH \B \bH\tr$: the covariance matrix of $\ObsMod(\x^\tn{f})$,

in the formula for $\K$, namely eqn. (K1) of [T4](T4%20-%20Multivariate%20Kalman.ipynb).
Using the estimators from [T7](T7%20-%20Ensemble%20representation.ipynb) yields
$$\begin{align}
	\barK &= \X \Y\tr ( \Y \Y\tr + (N{-}1) \R )^{-1} \, , \tag{5a}
\end{align}
$$
where $\Y \in \Reals^{P \times N}$
is the centered, *observed* ensemble
$\Y \ceq
\begin{bmatrix}
		\y_1 -\by, & \ldots & \y_n -\by, & \ldots & \y_N -\by
	\end{bmatrix} \, ,$ where $\y_n = \ObsMod(\x_n^\tn{f})$.

The EnKF is summarized in the animation below.

```python
EnKF_animation
```

#### Exc 2:
(a) Use the Woodbury identity (C2) of [T4](T4%20-%20Multivariate%20Kalman.ipynb) to show that eqn. (5) can also be written
$$\begin{align}
	\barK &= \X ( \Y\tr \Ri \Y + (N{-}1)\I_N  )^{-1} \Y\tr \Ri \, . \tag{5b}
\end{align}
$$
(b) What is the potential benefit?

#### Exc 4:
The above animation assumed that the observation operator is just the identity matrix, $\I$, rather than a general observation operator, $\ObsMod()$. Meanwhile, the Kalman gain used by the EnKF, eqn. (5a), is applicable for any $\ObsMod()$. On the other hand, the formula (5a) consists solely of linear algebra. Therefore it cannot perfectly represent any general (nonlinear) $\ObsMod()$. So how does it actually treat the observation operator? What meaning can we assign to the resulting updates?
*Hint*: consider the limit of $\R \rightarrow 0$.

#### Exc 6 (a):
Consider the ensemble averages,
 - $\bx^\tn{a} = \frac{1}{N}\sum_{n=1}^N \x^\tn{a}_n$, and
 - $\bx^\tn{f} = \frac{1}{N}\sum_{n=1}^N \x^\tn{f}_n$,

and recall that the analysis step, eqn. (4), defines $\x^\tn{a}_n$ from $\x^\tn{f}_n$.

(a) Show that, in case $\ObsMod$ is linear (the matrix $\bH$),
$$\begin{align}
	\Expect \bx^\tn{a} &=  \bx^\tn{f}  + \barK \left\{\y\ones\tr - \bH\bx^\tn{f}  \right\} \, , \tag{6}
\end{align}
$$
where the expectation, $\Expect$, is taken with respect to $\Dobs$ only (i.e. not the sampling of the forecast ensemble, $\E^\tn{f}$ itself).

What does this mean?

```python
#show_answer("EnKF_nobias_a")
```

#### Exc 6 (b)*:
Consider the ensemble covariance matrices:
$$\begin{align}
\barB &= \frac{1}{N-1} \X{\X}\tr \, , \tag{7a} \\\
\barP &= \frac{1}{N-1} \X^a{\X^a}\tr \, . \tag{7b}
\end{align}$$

Now, denote the centralized observation perturbations:
$$\begin{align}
\D &= \Dobs - \bar{\br}\ones\tr \\\
&= \Dobs\AN \, . \tag{8}
\end{align}$$
Note that $\D \ones = \bvec{0}$ and, with expectation over $\Dobs$,
$$
\begin{align}
	\label{eqn:R_sample_cov_of_D}
	\frac{1}{N-1}\D \D\tr = \R \, , \tag{9a} \\\
	\label{eqn:zero_AD_cov}
	\X \D\tr = \bvec{0} \, . \tag{9b}
\end{align}
$$
Assuming eqns (8) and (9) hold true, show that
$$\begin{align}
    \barP &= [\I_M - \barK \bH]\barB \, . \tag{10}
\end{align}$$

```python
#show_answer("EnKF_nobias_b")
```

#### Exc 6 (c)*:
Show that, if no observation perturbations are used in eqn. (4), then $\barP$ would be too small.

```python
#show_answer("EnKF_without_perturbations")
```

## Experimental setup

Before making the EnKF, we'll set up an experiment to test it with, so that you can check if you've implemented a working method or not.

To that end, we'll use the Lorenz-63 model, from [T6](T6%20-%20Dynamical%20systems,%20chaos,%20Lorenz.ipynb). The coupled ODEs are recalled here, but with some of the parameters fixed.

```python
M = 3 # ndim

def dxdt(x):
    sig  = 10.0
    rho  = 28.0
    beta = 8.0/3
    x,y,z = x
    d     = np.zeros(3)
    d[0]  = sig*(y - x)
    d[1]  = rho*x - y - x*z
    d[2]  = x*y - beta*z
    return d
```

Next, we make the forecast model $\DynMod$ out of $\frac{d \x}{dt}$ such that $\x(t+dt) = \DynMod(\x(t),t,dt)$. We'll make use of the "4th order Runge-Kutta" integrator `rk4`.

```python
def Dyn(E, t0, dt):

    def step(x0):
        return rk4(lambda t,x: dxdt(x), x0, t0, dt)

    if E.ndim == 1:
        # Truth (single state vector) case
        E = step(E)
    else:
        # Ensemble case
        for n in range(E.shape[1]):
            E[:,n] = step(E[:,n])

    return E

Q_chol = zeros((M,M))
Q      = Q_chol @ Q_chol.T
```

Notice the loop over each ensemble member. For better performance, this should be vectorized, if possible. Or, if the forecast model is computationally demanding (as is typically the case in real applications), the loop should be parallelized: i.e. the forecast simulations should be distributed to separate computers.

The following are the time settings that we will use

```python
dt    = 0.01           # integrational time step
dkObs = 25             # number of steps between observations
dtObs = dkObs*dt       # time between observations
KObs  = 60             # total number of observations
K     = dkObs*(KObs+1) # total number of time steps
```

Initial conditions

```python
mu0     = array([1.509, -1.531, 25.46])
P0_chol = eye(3)
P0      = P0_chol @ P0_chol.T
```

Observation model settings

```python
p = 3 # ndim obs
def Obs(E, t):
    if E.ndim == 1: return E[:p]
    else:           return E[:p,:]

R_chol = sqrt(2)*eye(p)
R      = R_chol @ R_chol.T
```

Generate synthetic truth (`xx`) and observations (`yy`)

```python
# Init
xx    = zeros((K+1   ,M))
yy    = zeros((KObs+1,p))
xx[0] = mu0 + P0_chol @ randn(M)

# Loop
for k in range(1,K+1):
    xx[k]  = Dyn(xx[k-1],(k-1)*dt,dt)
    xx[k] += Q_chol @ randn(M)
    if k%dkObs == 0:
        kObs = k//dkObs-1
        yy[kObs] = Obs(xx[k],nan) + R_chol @ randn(p)
```

## EnKF implementation

We will make use of `estimate_mean_and_cov` and `estimate_cross_cov` from the previous section. Paste them in below.

```python
# def estimate_mean_and_cov ...
```

**Exc 8:** Complete the code below

```python
xxhat = zeros((K+1,M))

# Useful linear algebra: compute B/A
def divide_1st_by_2nd(B,A):
    return nla.solve(A.T,B.T).T

def my_EnKF(N):
    # Init ensemble
    ...
    for k in range(1,K+1):
        # Forecast
        t   = k*dt
        # use model
        E   = ...
        # add noise
        E  += ...
        if k%dkObs == 0:
            # Analysis
            y        = yy[k//dkObs-1] # current observation
            Eo       = Obs(E,t)       # obsrved ensemble
            # Compute ensemble moments
            BH       = ...
            HBH      = ...
            # Compute Kalman Gain
            KG       = ...
            # Generate perturbations
            Perturb  = ...
            # Update ensemble with KG
            E       +=
        # Save statistics
        xxhat[k] = mean(E,axis=1)
```

Notice that we only store some stats (`xxhat`). This is because in large systems, keeping the entire ensemble in memory is probably too much.

```python
#show_answer('EnKF v1')
```

Now let's try out its capabilities

```python
# Run assimilation
my_EnKF(10)

# Plot
fig, axs = plt.subplots(3,1,True)
for m in range(3):
    axs[m].plot(dt*arange(K+1), xx   [:,m], 'k', label="Truth")
    axs[m].plot(dt*arange(K+1), xxhat[:,m], 'b', label="Estimate")
    if m<p:
        axs[m].plot(dtObs*arange(1,KObs+2),yy[:,m],'g*')
    axs[m].set_ylabel("Dim %d"%m)
axs[0].legend()
plt.xlabel("Time (t)")
```

**Exc 10:** The visuals of the plots are nice. But it would be good to have a summary statistic of the accuracy performance of the filter. Make a function `average_rmse(xx,xxhat)` that computes $ \frac{1}{K+1} \sum_{k=0}^K \sqrt{\frac{1}{M} \| \bx_k - \x_k \|_2^2} \, .$

```python
def average_rmse(xx,xxhat):
    ### INSERT ANSWER ###
    return average

# Test
average_rmse(xx,xxhat)
```

```python
#show_answer('rmse')
```

**Exc 12:**
 * (a). Repeat the above experiment, but now observing only the first (0th) component of the state.

```python
#show_answer('Repeat experiment a')
```

 * (b). Put a `seed()` command in the right place so as to be able to recreate exactly the same results from an experiment.

```python
#show_answer('Repeat experiment b')
```

 * (c). Use $N=5$, and repeat the experiments. This is quite a small ensemble size, and quite often it will yield divergence: the EnKF "definitely loses track" of the truth, typically because of strong nonlinearity in the forecast models, and underestimation (by $\barP)$ of the actual errors. Repeat the experiment with different seeds until you observe in the plots that divergence has happened.
 * (d). Implement "multiplicative inflation" to remedy the situation; this is a factor that should spread the ensemble further apart; a simple version is to inflate the perturbations. Implement it, and tune its value to try to avoid divergence.

```python
#show_answer('Repeat experiment cd')
```

### Next: [DAPPER example scripts](https://github.com/nansencenter/DAPPER/tree/master/examples)
We hope you have enjoyed these exercises. For your next steps, we suggest you try out the example notebooks that come with DAPPER. In particular, the first example contains exercises for the willing.

notebooks/T8 - Writing your own EnKF.ipynb


# Postulates of Quantum Mechanics

## 🥅 Learning Objectives
- List and Discuss the Postulates of Quantum Mechanics.
- Use Bra- Ket- notation.
- Learn how to manipulate quantum-mechanical operators.
- Understand the Heisenberg Uncertainty Principle.

## Postulates of Quantum Mechanics
### **Completeness**: All Observable Properties of a System can be Inferred from its Wavefunction
> $\Psi \rightarrow \text{everything}$
### **Schrodinger Postulate**: The wavefunction is found by solving the Schrödinger equation.
> $\hat{H}\Psi(x,t) = i \hbar \dot{\Psi}$ or $\hat{H}\Psi = E \Psi$.
>
> Relativitistic quantum mechanics (the Dirac equation) is needed when particles are moving close to the speed of light
### **Born Postulate**: $\left|\Psi(x,t) \right|^2$ is the probability of observing the system at position $x$ at time $t$.
> $\Psi(x,t)$ is the "square root of reality".
### **Correspondence Principle**: Every observable operator corresponds to a linear Hermitian operator.
> The analogue of a classical measurement is evaluating the expectation value of a suitable Hermitian operator. The only possible values that can be obtained from a measurement are the eigenvalues of the corresponding Hermitian operator.

We will now elaborate on each of these key postulates. Note that [some](http://vergil.chemistry.gatech.edu/notes/intro_estruc/node4.html) [people](http://web.mit.edu/8.05/handouts/jaffe1.pdf) list [more](https://en.wikipedia.org/wiki/Mathematical_formulation_of_quantum_mechanics) postulates. There is a some choice in how one groups the information in the postulates together, and I prefer to group the statements about Hermitian operators together.

## Completeness of $\Psi$ as a descriptor of a quantum system
The wavefunction is sufficient to determine all observable information about a quantum system. This does not mean that all information about a system can be observed. For example, while every classical observable corresponds to a Hermitian operator, it can be impossible to determine the values of multiple classical observables simultaneously.

As is often true, there are subtleties related to this postulate. In practice, we often use a density matrix, rather than a wavefunction, to describe a quantum system. A density matrix is just a mathematical way to capture a mixture of different quantum states, and can be written as,

$$
\Gamma(x,t;x',t') = \sum_{k=1}^{\infty} p_k \left(\Psi_k(x',t') \right)^*\Psi_k(x,t)
$$

The interpretation of a density matrix is that it is a mixture of quantum states, where the probability of observing the quantum state $k$ is $p_k$. It makes sense, therefore, to restrict $p_k$ to satisfy the constraints:

$$
1 = \sum_{k=1}^{\infty} p_k \qquad 0 \le p_k \le 1
$$

As with the wavefunction, the normalization of the density matrix is really only a convention. Any bounded (i.e. normalizable) positive semidefinite Hermitian operator can be interpreted as a density matrix. Bounded, positive semidefinite, non-Hermitian operators can be interpreted as transition density matrices, and are the mathematical representation for a change in a quantum state.

## Schrödinger Postulate
The wavefunction (or density matrix) is the central object in quantum mechanics. It is found by solving the Schrödinger equation in its time-independent or time-dependent form. For relativisitic particles, one needs to instead consider the [Dirac equation](https://en.wikipedia.org/wiki/Dirac_equation), and for relativisitic particles interacting with light, one needs to consider [quantum electrodynamics](https://en.wikipedia.org/wiki/Quantum_electrodynamics). If one needs to go beyond this (e.g., to include $\beta$ decay of nuclei), then one needs to extend even further, eventually all the way to the [standard model](https://en.wikipedia.org/wiki/Mathematical_formulation_of_the_Standard_Model).

For the purposes of this course, the Schrödinger equation is adequate. [Relativisitic effects](https://en.wikipedia.org/wiki/Relativistic_quantum_chemistry#Periodic-table_deviations) are [important](https://www.annualreviews.org/doi/10.1146/annurev-physchem-032511-143755) in [chemistry](https://aip.scitation.org/doi/10.1063/1.3702628): it is not uncommon to consider that corrections to the nonrelativistic Schrödinger equation become important for molecules containing elements heavier than Zinc (Z=30) or Krypton (Z=36). However, even actinide compounds can, [by mathematical trickery](https://en.wikipedia.org/wiki/Foldy%E2%80%93Wouthuysen_transformation), be treated by embellishing the Schrödinger equation in most cases. This means that for the purposes of understanding the structures of molecules and materials, their stability and thermodynamic properties, and the structural and chemical transformations between materials, the (relativistically-corrected) Schrödinger equation is used.

## Born Postulate
Colloquially, the Born postulate states that *the wavefunction is the square root of reality*. The wavefunction is a mathematical artifice, and is not directly observable. It's complex square is observable ($|\Psi(x,t)|^2$ indicates the probability of finding the quantum system at the location $x$ at time $t$) and the overlap (interference patterns) between two wavefunctions (e.g., the system's wavefunction and [light](https://www.nature.com/articles/ncomms5156) or another [electron](https://www.nature.com/articles/nature03183)) can also be observed.

## Correspondence Principle and Hermitian Operators
The first two postulate of quantum mechanics indicate that the wavefunction is the conveyor of all information about a quantum system and how the wavefunction is to be determined. The remaining postulate(s) all indicate that information about a quantum system is deduced from the action of Hermitian operators.

### Correspondence Principle
The [correspondence principle](https://en.wikipedia.org/wiki/Correspondence_principle) says that to every classical observable, there exists a Hermitian operator. This Hermitian operator provides the quantum manifestation of that classical observable. In the classical limit (where Planck's constant approaches zero, $h \rightarrow 0$), the quantum mechanical operator and the classical observable coincide. For example, we have used that the quantum mechanical operator for the momentum of a particle moving in one dimension is

$$
\hat{p} = -i \hbar \tfrac{d}{dx}
$$

There are some caveats associated with this. Some classical observables do not have unique representations in quantum mechanics. For example, in classical mechanics it is sensible to define, and measure, a *local kinetic energy*, that is, the kinetic energy at of a particle at a given point in space. Because one cannot specify the momentum (or its square) and position of a particle simultaneously, however, one cannot specify the kinetic energy at a point in space in quantum mechanics. [So the local kinetic energy is not well-defined in quantum mechanics](https://doi.org/10.1021/jp1029745).

In classical mechanics, all classical observables can be computed from the positions and momenta of the particles in the system. That is, classical observables of an $N$-particle system functions of particles' positions ,$\{\mathbf{r}_k\}$,  and momenta, $\{\mathbf{p}_k\}$:

$$
f(\mathbf{r}_1,\mathbf{r}_2,\ldots,\mathbf{r}_N;\mathbf{p}_1,\mathbf{p}_2,\dots,\mathbf{p}_N)
$$

In quantum mechanics, operators that depend *either* on the particle's positions or their momenta are clearly defined, but simultaneous observation of both is not. [There can be multiple mathematical representations for the quantum-mechanical analogue of a single classical observable](https://en.wikipedia.org/wiki/Quasiprobability_distribution). In practice, such ambiguities do not matter much since the most important observables are those that can be clearly defined. Sometimes, however, chemists like to discuss quantities (like the energy of a single atom or functional group in a molecule, or the interaction energy between two subsystems in a intermolecular complex) which are *not* defined in quantum mechanics. Such [discussions](https://doi.org/10.1016/j.comptc.2014.09.028) can be [useful](https://doi.org/10.1002/anie.201208206) in [practice]( https://doi.org/10.1002/jcc.26003), but they are contrary to the spirit of quantum mechanics (even though they are deeply entangled with the language of chemistry). Finding rigorous and systematic ways to capture chemical concepts from quantum-mechanical models is an [open problem](https://doi.org/10.1016/j.comptc.2018.09.006).

There are also entities that emerge in molecular quantum mechanics that have no classical analogue whatsoever: concepts like electronegativity, oxidation state, and bond order. One should not expect for such quantities, which are intrinsically tied to the properties of electrons and have no classical analogue, to correspond to quantum-mechanical observable or Hermitian operators.

In summary:
- every quantum-mechanical observable corresponds to a Hermitian operator.
- every quantum-mechanical observable reduces to a classical observable in the $h \rightarrow 0$ limit.
- some classical observables do not have a unique quantum-mechanical analogue.
- some inherently quantum-mechanical concepts do not have a classical analogue, nor do they have a unique mathematical representation within quantum mechanics.

### Hermitian Operators
Quantum-mechanical observables correspond to Hermitian operators. An operator, $\hat{Q}$ is Hermitian if for any wavefunctions $\psi(x)$ and $\phi(x)$

$$
\int \left( \psi(x) \right)^* \hat{Q} \phi(x) dx
= \int \left(\hat{Q} \psi(x) \right)^* \phi(x) dx
= \int  \left( \hat{Q} \right)^*\left( \psi(x) \right)^*  \phi(x) dx
$$

This basically means that a Hermitian operator can act either forwards or backwards. This is very useful in practice, since sometimes it is easier to apply  $\hat{Q}$ to $\psi(x)$ than to $\phi(x)$.

> **Note:** Quantum-mechanical observables do not need to be Hermitian; it is sufficient for them to be [essentially self-adjoint](https://en.wikipedia.org/wiki/Self-adjoint_operator), which is a somewhat weaker concept. While this is not quite a mere mathematical nuance (e.g., the molecular Hamiltonian is essentially self-adjoint), it does not affect the way we use quantum mechanics as a computational tool within chemistry.

#### 📝 Exercise: An alternative definition for an Hermitian operator is below. Show that this is equivalent to the preceding definition.

$$
\int \left( \Psi(x) \right)^* \hat{Q} \Psi(x) dx
= \int \left(\hat{Q} \Psi(x) \right)^* \Psi(x) dx
$$

> Hint: Choose $\Psi(x) = \psi(x) + i \phi(x)$.

#### 📝 Exercise: Show that the momentum operator is Hermitian.
> Hint: use integration by parts and assume the wavefunction and its derivatives are zero at $\pm \infty$.

### Observable Values

In an experimental measurement of a quantum-mechanical observable, the measured value is always one of the eigenvalues of the Hermitian operator.

Recall that Hermitian operators have real eigenvalues. This explains why Hermitian operators are so central to quantum mechanics: the result of an experimental measurement must be a real number. The "chunkiness" inherent in quantum mechanics emerges from the fact one cannot observe an arbitrary value for the observable, but only the discrete eigenvalues of the corresponding quantum-mechanical operator. Ergo, [Schrödinger's cat](https://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s_cat) can be alive or dead, but not half-dead.

#### 📝 Exercise: Show that the eigenvalues of Hermitian operators are always real.
> Hint: Use the Hermitian property of $\hat{Q}$ to evaluate $\int \left(\psi_k(x)\right)^* \hat{Q} \psi_k(x) dx$ in two different ways.

### Eigenvectors of Hermitian Operators form a Complete Basis
Denote the eigenvectors and eigenvalues of a Hermitian operator in the obvious way,

$$
\hat{Q} \psi_k(x) = q_k \psi_k(x)
$$

- As a consequence of the [spectral theorem](https://en.wikipedia.org/wiki/Spectral_theorem), the eigenvectors of a Hermitian operator form a complete basis. That is, any wavefunction can be exactly represented as a linear combination of the eigenfunctions of any Hermitian operator:

$$
\Psi(x) = \sum_{k=0}^{k=\infty} c_k \psi_k(x)
$$

- Moreover, the eigenvectors, $\psi_k(x)$, can always be chosen to be orthogonal and normalized,

$$
\int \left(\psi_k(x)\right)^* \psi_l(x) dx =
\begin{cases}
    1 & k = l\\
    0  & k \ne l
\end{cases}
$$

We call orthogonal and normalized functions *orthonormal*. A short-hand notation for orthonormality is:

$$
\int \left(\psi_k(x)\right)^* \psi_l(x) dx = \delta_{kl}
$$

where

$$\delta_{kl} =
\begin{cases}
    1 & k = l\\
    0  & k \ne l
\end{cases}
$$

denotes the [Kronecker delta](https://en.wikipedia.org/wiki/Kronecker_delta) symbol. The Kronecker delta symbol is the analogue of the identity matrix (extended to infinite dimensions).

#### 📝 Exercise: Show that the eigenvectors of a Hermitian operator with different eigenvalues are always orthogonal.
> Hint: Use the Hermitian property of $\hat{Q}$ to evaluate $\int \left(\psi_k(x)\right)^* \hat{Q} \psi_l(x) dx$ in two different ways.

### Expectation Values
When we expand a wavefunction in the eigenfunctions of a Hermitian operator, we say it is a superposition of the eigenstates of that operator.

$$
\Psi(x) = \sum_{k=0}^{k=\infty} c_k \psi_k(x)
$$

Measuring that operator always produces one of the eigenvalues, and the eigenvalues occur in proportion to the square magnitude of their coefficient in the eigenvector expansion,

$$
\text{probability of observing the eigenvalue }q_k = |c_k|^2 = c_k^* c_k
$$

The expectation value of the operator is thus:

$$
\left<Q \right> = \sum_{k=0}^{k=\infty} |c_k|^2 q_k = \int \left( \Psi(x) \right)^* \hat{Q} \Psi(x) dx
$$

This formula for the expectation value only holds for normalized wavefunctions. When the wavefunction is not normalized, instead one must use:

$$
\left<Q \right> = \frac{\sum_{k=0}^{k=\infty} |c_k|^2 q_k}{\sum_{k=0}^{k=\infty} |c_k|^2} = \frac{\int \left( \Psi(x) \right)^* \hat{Q} \Psi(x) dx}{\int \left( \Psi(x) \right)^* \Psi(x) dx}
$$

#### 📝 Exercise: Show that the equality in the last equation is true
> Hint: expand the wavefunction in the eigenbasis, use the eigenvalue relation and the orthonormality of the eigenvectors.

#### 📝 Exercise: Show that the Laplacian is a linear, Hermitian, negative-definite operator.
>It is OK (but not essential) to assume that you are interested in the Laplacian operator in 1 dimension, $\nabla^2 = \frac{d^2}{dx^2}$. When we say that $\nabla^2$ is negative definite, we mean that for *any* $\Psi$, it is *always* true that
>$$
\langle \Psi | \nabla^2 | \Psi \rangle < 0
>$$
>Note: You may assume that the wavefunction, and its first first and second derivatives, vanish at the ends of the region of integration. However, there are also ways to answer these questions without explicitly invoking those assumptions.
>[Solution.](../documents/LaplacianProperties.md)

### Wavefunction "Collapse"

Immediately after performing a measurement of $Q$ for the system defined by $\Psi(x)$, one knows definitively that the state of the system is described by $\Psi(x) = \psi_k(x)$, with eigenvalue $q_k$. This seems weird, as the wavefunction seems to have changed abruptly from $\Psi(x)$ to $\psi_k(x)$ *because* of the measurement. This would somehow imply that if the wavefunction for Schrödinger's cat were:

$$
\Psi_{cat} = \tfrac{1}{\sqrt{2}}|\text{alive} \rangle + \tfrac{1}{\sqrt{2}}|\text{dead} \rangle
$$

and you opened the box and observed that the cat was dead (so after you open the box, $\Psi_{cat} =  |\text{dead} \rangle$), then *you* killed Schrödinger's cat. To mildly exaggerate, some physicists would have you believe that every dead animal was slaughtered by the person who first observes its corpse. (To diminish culpability, it must be said that it the cat in this example was only technically half-dead, so the observer was merely a halfway-cat-assassin.) Most modern [interpretations of quantum mechanics](https://en.wikipedia.org/wiki/Interpretations_of_quantum_mechanics) tend to [deny such culpability](https://plato.stanford.edu/entries/qt-issues/). The physicists alibi is to assert that assert that while the system was described, mathematically, by $\Psi(x)$ prior to the measurement, this does not that the system existed in the state $\Psi(x)$. Similarly, after the measurement the system is in a state mathematically described by $\psi_k(x)$. While it would be weird for observing a system to be able to change its state, it is not weird for an observation to change our mathematical description of a system. For example, before you observe [Lake Wobegon](https://en.wikipedia.org/wiki/Lake_Wobegon), it is reasonable to assume that all the women are strong, all the men are good-looking, and all the children are above average. But were you to visit Lake Wobegon, then based on your observation you might have to change your model.

That said, you may find the aforementioned "Copenhagen interpretation" convenient. Before my mother visits my home, I always clean it thoroughly. Nonetheless, my thorough cleaning is not up to my mother's standards, and she's always scandalized to find dust-bunnies under the sofa. (Who moves the sofa to vacuum under it, just to put the sofa back the same place and obscure the now-clean carpet?) I always tell my mom that the dust-bunnies were not there until she observed them. Unfortunately, my mom taught quantum mechanics herself, and she tells me that the wavefunction was:

$$
\Psi_{dust-bunnies} = \sqrt{.0001}|\text{clean} \rangle + \sqrt{.9999}|\text{dirty} \rangle
$$

I guess my home is only .01% clean (up to my mother's standards, at least).

The sudden change in the wavefunction upon observation is called [wavefunction collapse](https://en.wikipedia.org/wiki/Wave_function_collapse). While the reality of wavefunction collapse in a physical sense is irresolvable, it is the mathematical description of what happens in a quantum system, and gives rise to strange quantum effects. If you find such things interesting, you may be interested to know that the aphorism that "a watched pot never boils" is [justifiable](https://en.wikipedia.org/wiki/Quantum_Zeno_effect), quantum-mechanically.

### The Born Postulate, revisited
While the Born postulate is usually presented separately, it is in fact a corollary of the fact that physical observables are represented by Hermitian operators. The Hermitian operator that represents a particle at position $x_0$ is $\delta(x-x_0)$, where the [Dirac delta function](https://en.wikipedia.org/wiki/Dirac_delta_function) is effectively defined by its so-called *sifting property*.
> **Sifting property of the Dirac Delta function:** Let $f(x)$ be a bounded function. Then
>$$
\int_{-\infty}^{+\infty} f(x) \delta(x-x_0) dx = f(x_0)
$$

Now according to the Hermitian postulate, the probability of observing the particle at position $x_0$ is given by

$$
\int_{-\infty}^{+\infty} \left(\Psi(x)\right)^* \delta(x-x_0) \Psi(x) dx = \left|\Psi(x_0)\right|^2
$$

which is the Born postulate.

> **Note**: The Kronecker delta function has a sifting property similar to the Dirac delta function,
>$$
\sum_j f_j \delta_{jk} = f_k
$$

This can be deduced directly from the definition, since $\delta_{jk}$ is one when $j=k$, and zero otherwise.

## Bra-Ket Notation and the Analogy to Linear Algebra
At this stage, your hand may be starting to hurt from writing integrals and wavefunctions. This is the motivation for [Dirac bra-ket notation](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation). A rather thorough explanation of this notation is available in my [pdf notes](https://github.com/QC-Edu/IntroQM2022/blob/master/documents/LinAlgAnalogy.pdf), so I am only presenting the elements here. The basic idea is that a wavefunction is written as a *ket*,

$$
| \Psi \rangle = \Psi(x)
$$

and its complex conjugate as a *bra*,

$$
\langle \Phi | = \left(\Phi(x)\right)^*
$$

The overlap between two wavefunctions, which as we shall see is very important when expanding a wavefunction as a linear combination of basis functions, is therefore:

$$
\langle \Phi | \Psi \rangle = \int \left(\Phi(x)\right)^* \Psi(x) dx
$$

Because a Hermitian operator could act either forward (towards the ket)

$$
\langle \Phi |\hat{Q} \Psi \rangle = \int \left(\Phi(x)\right)^* \hat{Q} \Psi(x) dx
$$

or backwards (towards the bra)

$$
\langle\hat{Q} \Phi | \Psi \rangle = \int \left(\hat{Q} \Phi(x)\right)^* \Psi(x) dx
$$

we often use a notation that makes it clear that the operator can act in either direction,

$$
\langle \Phi | \hat{Q} |\Psi \rangle = \int \left(\Phi(x)\right)^* \hat{Q} \Psi(x) dx
$$

## Application: Expansion in a Basis Set
A set of functions, $\{\phi_k(x) \}$ is said to be a complete basis set if any wavefunction can be written exactly as a linear combination of these basis functions,

$$
\Psi(x) = \sum_{k=0}^{\infty} c_k \phi_k(x)
$$

It is convenient and it is always possible, but it is not required, to choose the basis functions so that they are orthonormal,

$$
\int \left(\phi_j(x) \right)^* \phi_k(x) dx = \delta_{jk}
$$

where we have used the Kronecker-delta notation we introduced above.

In bra-ket notation, the preceding equations are, respectively,

$$
|{\Psi}\rangle = \sum_{k=0}^{\infty} c_k |\phi_k\rangle
$$

and

$$
\langle \phi_j | \phi_k \rangle = \delta_{jk}
$$

To obtain an equation for the expansion coefficient, multiply both sides of the first equation by $(\phi_j(x))^*$ and integrate over $x$. This gives:

$$
\int \left( \phi_j(x) \right)^* \Psi(x) dx = \int \left( \phi_j(x) \right)^*
 \sum_{k=0}^{\infty} c_k \phi_k(x) dx
$$

Because the integral of a sum is the sum of the integrals, and because the integral of a constant is a constant times the integral, this simplifies to:

$$
\begin{align}
\int \left( \phi_j(x) \right)^* \Psi(x) dx &=\sum_{k=0}^{\infty} c_k \left[ \int \left( \phi_j(x) \right)^*
  \phi_k(x) dx \right] \\
&=\sum_{k=0}^{\infty} c_k \delta_{jk}
\end{align}
$$

Then from the sifting property of the Dirac delta function,

$$
c_j = \int \left( \phi_j(x) \right)^* \Psi(x) dx
$$

*This is the equation for the expansion coefficient of a wavefunction in an orthonormal basis.*

#### 📝 Exercise: Show that the expansion coefficient for a wavefunction in a nonorthogonal basis can be determined by solving the system of linear equations:
$$
\sum_{k=0}^{\infty} c_k \int \left( \phi_j(x) \right)^* \phi_k(x) \, dx  = \int \left( \phi_j(x) \right)^* \Psi(x) dx
$$

#### 📝 Exercise: Show that the eigenvectors of a Hermitian operator with the same eigenvalue can always be chosen to be orthogonal, but do not have to be.
> Hint: Assume the eigenvectors associated with a given eigenvalue are orthogonal; show that you do not have to choose them to be orthogonal. Then, for the (more difficult) converse, assume the eigenvectors associated with a given eigenvalue are not orthogonal, and construct a linear transformation of them that is orthogonal.

## Application: Heisenberg Uncertainty Principle
We have already alluded to the [Heisenberg Uncertainty Principle](https://en.wikipedia.org/wiki/Uncertainty_principle), which states that some quantum-mechanical properties cannot be observed simultaneously. To provide a mathematical description of the Heisenberg Uncertainty Principle, we need to define what it means for operators to commute and anticommute.
> Two operators, $\hat{A}$ and $\hat{B}$, are said to *commute* if for any wavefunction $\Psi(x)$,

$$
0 = \left(\hat{A} \hat{B} - \hat{B}\hat{A}\right) \Psi(x) = \left[\hat{A}, \hat{B} \right] \Psi(x)
$$

Similarly, two operators are said to *anticommute* if

$$
0 = \left(\hat{A} \hat{B} + \hat{B}\hat{A}\right) \Psi(x) = \left\{\hat{A}, \hat{B} \right\} \Psi(x)
$$

In the rightmost equality, we have introduced the standard notation for commuting and anticommuting operators.

In classical mechanics, observables are simply functions of the momenta and positions of the system's particles, and the since the momenta and positions commute, observables commute. However, in quantum mechanics, the momentum operator, $\hat{p} = -i \hbar \tfrac{d}{dx}$ is a differential operator, and does not commute with the position operator $x$. Because of this, measuring a particle's position first, then its momentum is different from measuring its momentum first, then its position. Conceptually, then, it is not unreasonable that if you try to measure the position and the momentum simultaneously, the system is "confused" about how it should behave (as if momentum were measured first? or as if position were measured first?) and the answer is uncertain.

> If two operators, $\hat{A}$ and $\hat{B}$, commute, $\left[\hat{A}, \hat{B}, \right] = 0$. then one can simultaneously measure the corresponding properties $A$ and $B$. $A$ and $B$ are said to be *simultaneous observables*.

> If two operators, $\hat{A}$ and $\hat{B}$, do not commute, $\left[\hat{A}, \hat{B}, \right] \ne 0$. then one cannot simultaneously measure the corresponding properties $A$ and $B$. (The order of measurement matters.) A Heisenberg Uncertainty Relation then holds:

$$
\sigma_A^2 \sigma_B^2 \ge \tfrac{1}{4} \left| \langle \Psi |[\hat{A},\hat{B}]| \Psi \rangle \right|^2
$$

where the variance in the expectation value of the operator is defined as:

$$
\sigma_A^2 = \langle \Psi |\hat{A}^2| \Psi \rangle - \langle \Psi |\hat{A}| \Psi \rangle^2
$$

This expression can be made somewhat more precise by using the [Schrödinger Uncertainty Principle](https://en.wikipedia.org/wiki/Uncertainty_principle#Robertson%E2%80%93Schr%C3%B6dinger_uncertainty_relations)

$$
\sigma_A^2 \sigma_B^2 \ge  \left| \tfrac{1}{2}\langle \Psi |\{\hat{A},\hat{B}\}| \Psi \rangle
- \langle \Psi |\hat{A}| \Psi \rangle \langle \Psi |\hat{B}| \Psi \rangle \right|^2 + \tfrac{1}{4} \left|\langle \Psi |[\hat{A},\hat{B}]| \Psi \rangle \right|^2
$$

A detailed derivation of these uncertainty principles is provided as a [pdf](https://github.com/QC-Edu/IntroQM2022/blob/master/documents/Heisenberg.pdf).

## Application: Variational Principle
> [**Quantum-Mechanical Variational Principle**](https://en.wikipedia.org/wiki/Variational_method_(quantum_mechanics)): Given a wavefunction $\Psi(x)$ and a bounded Hermitian operator $\hat{Q}$, the expectation value of $Q$ is no less than the lowest eigenvalue of $\hat{Q}$ and no greater than the largest eigenvalue.

To understand where this principle comes from, let's focus on the lowest eigenvalue of $\hat{Q}$. We denote the eigenvalues and eigenvectors of $\hat{Q}$ as

$$
\hat{Q} | \psi_k \rangle = q_k | \psi_k \rangle
$$

and choose to list the eigenvalues are listed in increasing order, $q_0 < q_1 < \cdots $. Expand the wavefunction in the eigenvectors of $\hat{Q}$,

$$
| \Psi \rangle = \sum_{k=0}^{\infty} c_k | \psi_k \rangle
$$

The expectation value of $\Psi$, which we assume to be normalized, is:

$$
\begin{align}
\langle \Psi |\hat{Q}|\Psi \rangle &= \langle \sum_{j=0}^{\infty} c_j \psi_j  |\hat{Q}|\sum_{k=0}^{\infty} c_k \psi_k \rangle \\
&= \sum_{j=0}^{\infty} \sum_{k=0}^{\infty} c_j c_k \langle  \psi_j  |\hat{Q}|\psi_k \rangle \\
&= \sum_{j=0}^{\infty} \sum_{k=0}^{\infty} c_j c_k \langle  \psi_j  |\hat{Q}\psi_k \rangle \\
&= \sum_{j=0}^{\infty} \sum_{k=0}^{\infty} c_j c_k \langle  \psi_j  |q_k\psi_k \rangle \\
&= \sum_{j=0}^{\infty} \sum_{k=0}^{\infty} c_j c_k q_k \langle  \psi_j  |\psi_k \rangle \\
&= \sum_{j=0}^{\infty} \sum_{k=0}^{\infty} c_j c_k q_k \delta_{jk} \\
&= \sum_{j=0}^{\infty} |c_j|^2 q_j
\end{align}
$$

In the last line we used the sifting property of the Kronecker delta function. To complete the demonstration, recall that because the eigenvalues are listed in nondecreasing order, $q_j - q_0 \ge 0$ for all $j$. So:

$$
\begin{align}
\langle \Psi |\hat{Q}|\Psi \rangle &= \sum_{j=0}^{\infty} |c_j|^2 q_j \\
 &= \sum_{j=0}^{\infty} |c_j|^2 \left(q_0 + (q_j - q_0)\right) \\
 &= \sum_{j=0}^{\infty} |c_j|^2 q_0 + \sum_{j=0}^{\infty} |c_j|^2 (q_j - q_0) \\
 &= q_0 \sum_{j=0}^{\infty} |c_j|^2 + \sum_{j=0}^{\infty} |c_j|^2 (q_j - q_0) \\
 &= q_0  + \sum_{j=0}^{\infty} |c_j|^2 (q_j - q_0) \\
 &\ge q_0
\end{align}
$$

In the next to last line the normalization of the wavefunction, which implies that $\sum |c_j|^2 = 1$, was used. In the last line a nonnegative term was eliminated from the expression, resulting in the desired inequality.

This principle is especially useful for the energy, because it means that an approximate wavefunction will always have an energy greater than or equal to the true ground state energy. Therefore, if one is given two wavefunctions $\Psi(x)$ and $\Phi(x)$, the "better" ground-state wavefunction would be the one that has the lower energy. Similarly, given a normalized wavefunction that depends on a parameter, $\kappa$, the "best" wavefunction and the "best" energy can be obtained by minimizing the expectation value of the energy with respect to $\kappa$:

$$
\min_\kappa \langle \Psi(\kappa) |\hat{H} | \Psi(\kappa) \rangle
$$

The variational principle is one of the most important ways we approximate the energy and wavefunction of quantum systems.

### 📝 Exercise: Show why the expectation value of $Q$ is always a lower bound on the largest eigenvalue of $\hat{Q}$.

### 📝 Exercise: Demonstrate that the variational principle for $\hat{Q}$ holds even if the wavefunction is not normalized.

## Summary
Based on the preceding, the key postulates of quantum mechanics are
- the Schrödinger equation determines the wavefunction
- the wavefunction determines all observable properties of a quantum system
- observable properties of a quantum system, and their observable values, correspond to Hermitian operators
You should be able to apply, and expand upon, these principles at length.

## 🪞 Self-Reflection
- Try to come up with your own version of a Schrödinger's cat paradox. Have [fun with it](https://www.straightdope.com/21341296/the-story-of-schroedinger-s-cat-an-epic-poem).
- What is an example of a classical observable that is not a quantum-mechanical observable?
- Give an example of two quantum-mechanical operators that commute. What about two operators that don't commute?

## 🤔 Thought-Provoking Questions
- Can you imagine a case where the Hermitian nature of the Hamiltonian operator would be useful?
- Can you imagine a way to use the variational principle in practice?
- The variational principle allows one to choose a wavefunction that gives the lowest energy, which provides the best wavefunction in an energetic sense. This isn't the same as finding the closest wavefunction to the ground-state wavefunction (see below) but if your energy is close enough to the ground-state energy, you can be assured that the wavefunction is also very close to the ground-state wavefunction. Explain why. (Assume, for simplicity, that the ground state is nondegenerate.)

$$
\min_\Psi \langle \Psi - \psi_0 | \Psi - \psi_0 \rangle
$$

## ❓ Knowledge Tests
- Quantum-mechanical mathematical postulates, techniques, and notation. [GitHub Classroom Link](https://classroom.github.com/a/UeStyjxT)

## 🔁 Recapitulation
- What are the postulates of quantum mechanics?
- Why is it important that quantum-mechanical observables correspond to Hermitian operators?
- What is the Heisenberg Uncertainty principle?
- What is the variational principle?
- List all the Hermitian operators you know.
- What do the Dirac and Kronecker delta notations mean?

## 🔮 Next Up...
- Multielectron systems

## 📚 References
My favorite sources for this material are:
- [Randy's book](https://github.com/PaulWAyers/IntroQChem/blob/main/documents/DumontBook.pdf?raw=true)
- D. A. MacQuarrie, Quantum Chemistry (University Science Books, Mill Valley California, 1983)
- [Mathematical Features of Quantum Mechanics](https://github.com/PaulWAyers/IntroQChem/blob/main/documents/PinBox.pdf?raw=true) (my notes, starting page 6).
- [Davit Potoyan's](https://www.chem.iastate.edu/people/davit-potoyan) Jupyter-book course discusses the posultates in [Chapter 3.2](https://dpotoyan.github.io/Chem324/Lec3-2.html).

There are also some excellent wikipedia articles:
- [Postulates of Quantum Mechanics](https://en.wikipedia.org/wiki/Mathematical_formulation_of_quantum_mechanics#Postulates_of_quantum_mechanics)
- [Variational Principle](https://en.wikipedia.org/wiki/Spectral_theory#Spectral_theorem_and_Rayleigh_quotient)