# COMP 135 day06 Lab: Multivariate Gaussian distributions

## Outline

* **Part 1: Multivariate Gaussian Distribution Basics**
* * Learn to sample from a multivariate Gaussian
* * Learn to make plots of joint, marginal, and conditional distributions

* **Part 2: Computing covariances**
* * Learn to compute empirical covariances of observed data
* * Learn to do an ML estimate of the mean and covariance parameters

* **Part 3: Visualizing Gaussians in 2D**
* * Learn to make contour plots of the level sets of the pdf
* * See that the level sets are *elliptical*

## Skills

* To draw samples, use `multivariate_normal`
* * [`scipy.stats.multivariate_normal(mean_D, cov_DD).rvs(size=(nrows, ncols), random_state=SEED)`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.multivariate_normal.html)
* * OR [`np.random.multivariate_normal(mean_D, cov_DD)`](https://numpy.org/doc/stable/reference/random/generated/numpy.random.multivariate_normal.html)
* To evaluate the PDF:
* * Evaluate pdf with `scipy.stats.multivariate_normal(mean_D, cov_DD).pdf(values)`
* * Evaluate logpdf with `logpdf` instead of `pdf`

## Takeaways

* Visualization is fun!
* Using *vectorized* code (rather than for loops) is more efficient to compute covariances

```python
import numpy as np
import pandas as pd
import scipy.stats
```

```python
np.set_printoptions(precision=3, suppress=False)
```

```python
pd.options.display.float_format = '{:,.3g}'.format  # show 3 digits of precision
```

```python
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style("whitegrid")
sns.set_context("notebook", font_scale=1.25)
```

```python

```

# Part 1: Multivariate Gaussian basics

## Basic facts

#### Random Variable: Vector data $x \in \mathbb{R}^D$

This is a real-valued vector with $D$ entries: $x = [x_1, x_2, \ldots x_D]^T$

#### Parameters: Location $\mu$ and scale $\sigma$

* Location or "mean" vector: $\mu = [\mu_1, \mu_2, \ldots \mu_D]^T$, so $\mu \in \mathbb{R}^D$

We'll denote this in code as `mean_D` (the `_D` suffix reminds us the shape is (D,)

* Covariance matrix $\Sigma$, which must be a $D \times D$ matrix that is symmetric and positive definite

We'll denote this in code as `cov_DD` (the `_DD` suffix reminds us the shape is (D, D)

#### Formal definition of PDF:

\begin{align}
\text{MVNormalPDF}(x | \mu, \Sigma) = c(\mu, \Sigma) \cdot \exp \left( -\frac{1}{2} (x-\mu)^T \Sigma^{-1} (x-\mu) \right)
\end{align}

with normalizing "constant" (a term constant wrt our random variable $x$):

$$
c(\mu, \Sigma) = \frac{1}{(2\pi)^{D/2}} \frac{1}{\text{det}(\Sigma)^{1/2}}
$$

##  Sampling from the "standard" multivariate Gaussian with D = 2

We can use `np.random.multivariate_normal` to sample.

It takes two parameters: `mean_D` and `cov_DD`

Set our random state so things are reproducible.

```python
prng = np.random.RandomState(11)
```

Draw several samples from the *standard* multivariate Normal with D=2

```python
zero_mean_2 = np.zeros(2)
identity_cov_22 = np.eye(2) # identity matrix

prng.multivariate_normal(zero_mean_2, identity_cov_22)
```

    array([ 1.749, -0.286])

```python
prng.multivariate_normal(zero_mean_2, identity_cov_22)
```

    array([-0.485, -2.653])

```python
prng.multivariate_normal(zero_mean_2, identity_cov_22)
```

    array([-0.008, -0.32 ])

### Scatterplot of 10000 samples from a *standard* Gaussian

```python
N = 10000
x_N2 = prng.multivariate_normal(zero_mean_2, identity_cov_22, size=N)
plt.plot(x_N2[:,0], x_N2[:,1], 'b.', alpha=0.3)
plt.gca().set_aspect('equal');
plt.xlabel('$x_1$');
plt.ylabel('$x_2$');
```

## Pair plot visualization of standard D=2 Gaussian

```python
# Pack samples into a pandas dataframe to plot
x_df = pd.DataFrame(x_N2, columns=['$x_1$', '$x_2$'])

# Define consistent visual limits
ymax_hist = 100
xmax = 5.0
n_bins = 21

# Make the pair plot
h = sns.pairplot(
    x_df,
    aspect=1.0,
    corner=True, # only plot lower left half of square
    diag_kws={"color": 'b', "alpha": 0.3, "bins":np.linspace(-xmax, xmax, n_bins)},
    plot_kws={"color": 'b', "alpha": 0.3, 'marker': '.', 'edgecolor':'none'});
# ^- alpha here means transparency for the 'alpha' channel of the RGB-A scheme

# Set view limits of *diagonal plots* to a standardized window
for diag_tuple in [(0,0), (1,1)]:
    h.axes[diag_tuple[0], diag_tuple[1]].set_xlim([-xmax, xmax]);
    h.axes[diag_tuple[0], diag_tuple[1]].set_ylim([0, ymax_hist]);

# Set view limits of *off-diagonal* scatter plots to a standardized window
h.axes[1,0].set_xlim([-xmax, xmax]);
h.axes[1,0].set_ylim([-xmax, xmax]);

# Prettify the overall plot
plt.tight_layout();
plt.suptitle(
    "mean %s\ncov  %s" % (
        str(zero_mean_2),
        str(identity_cov_22).replace("\n","\n     ")),
    fontsize=14,
    horizontalalignment='left',
    font='monospace');
```

## Discussion 1a: What can you say about x1 and x2 in terms of *independence*?

```python
# TODO discuss
```

##  Sampling from a D=2 multivariate Gaussian with *negative correlations*

```python
zero_mean_2 = np.zeros(2)
neg_corr_cov_22 = np.asarray([
    [3.0, -1.7],
    [-1.7, 1.0],
])

print("Mean")
print("-----")
print(zero_mean_2)
print("\nCovar")
print("-----")
print(neg_corr_cov_22)
```

    Mean
    -----
    [0. 0.]

    Covar
    -----
    [[ 3.  -1.7]
     [-1.7  1. ]]

```python
prng.multivariate_normal(zero_mean_2, neg_corr_cov_22)
```

    array([-1.121,  0.61 ])

```python
prng.multivariate_normal(zero_mean_2, neg_corr_cov_22)
```

    array([ 1.699, -1.433])

```python
prng.multivariate_normal(zero_mean_2, neg_corr_cov_22)
```

    array([ 1.639, -1.237])

```python
prng.multivariate_normal(zero_mean_2, neg_corr_cov_22)
```

    array([ 1.33 , -1.073])

## Discussion 1b: What do you notice about the relationship between x1 and x2 here? Is there something about their signs that is a consistent pattern?

```python
# TODO discuss
```

## Draw many values from zero mean, negative correlation Normal

```python
N = 10000
x_N2 = prng.multivariate_normal(zero_mean_2, neg_corr_cov_22, size=N)
print(x_N2[:3])
print("...")
```

    [[-0.142 -0.092]
     [-0.168 -0.229]
     [-1.52   0.962]]
    ...

## Pair plot visualization with negative correlation

```python
# Pack samples into a pandas dataframe to plot
x_df = pd.DataFrame(x_N2, columns=['$x_1$', '$x_2$'])

# Define consistent visual limits
ymax_hist = 100
xmax = 6.5
n_bins = 21

h = sns.pairplot(
    x_df,
    aspect=1.0,
    corner=True, # only plot lower left half of square
    diag_kws={"color": 'b', "alpha": 0.3, "bins":np.linspace(-xmax, xmax, n_bins)},
    plot_kws={"color": 'b', "alpha": 0.3, 'marker': '.', 'edgecolor':'none'},
    );
# ^- alpha here means transparency for the 'alpha' channel of the RGB-A scheme

# Set view limits of *diagonal plots* to a standardized window
for diag_tuple in [(0,0), (1,1)]:
    h.axes[diag_tuple[0], diag_tuple[1]].set_xlim([-xmax, xmax]);
    h.axes[diag_tuple[0], diag_tuple[1]].set_ylim([0, ymax_hist]);

# Set view limits of *off-diagonal* scatter plots to a standardized window
h.axes[1,0].set_xlim([-xmax, xmax]);
h.axes[1,0].set_ylim([-xmax, xmax]);

# Prettify the overall plot
plt.tight_layout();
plt.suptitle(
    "mean %s\ncov  %s" % (
        str(zero_mean_2),
        str(neg_corr_cov_22).replace("\n","\n     ")),
    fontsize=14,
    horizontalalignment='left',
    font='monospace');
```

## Discussion 1b: Which of x1 or x2 has *smaller* marginal variance? Why?

How does what you see in the plots align with the definition of the covariance parameter here.

## Example with negative correlations AND non-zero mean

```python
nonzero_mean_2 = np.asarray([2.0, -0.5])
neg_corr_cov_22 = np.asarray([
    [3.0, -1.7],
    [-1.7, 1.0],
])

```

Draw many samples

```python
N = 10000
x_N2 = prng.multivariate_normal(nonzero_mean_2, neg_corr_cov_22, size=N)
print(x_N2[:3])
print("...")
```

    [[ 3.724 -1.948]
     [-0.31   0.768]
     [ 2.17  -0.52 ]]
    ...

```python
# Pack samples into a pandas dataframe to plot
x_df = pd.DataFrame(x_N2, columns=['$x_1$', '$x_2$'])

# Define consistent visual limits
ymax_hist = 100
xmax = 8
n_bins = 21

h = sns.pairplot(
    x_df,
    aspect=1.0,
    corner=True, # only plot lower left half of square
    diag_kws={"color": 'r', "alpha": 0.3, "bins":np.linspace(-xmax, xmax, n_bins)},
    plot_kws={"color": 'r', "alpha": 0.3, 'marker': '.', 'edgecolor':'none'},
    );
# ^- alpha here means transparency for the 'alpha' channel of the RGB-A scheme

# Set view limits of *diagonal plots* to a standardized window
for diag_tuple in [(0,0), (1,1)]:
    h.axes[diag_tuple[0], diag_tuple[1]].set_xlim([-xmax, xmax]);
    h.axes[diag_tuple[0], diag_tuple[1]].set_ylim([0, ymax_hist]);

# Set view limits of *off-diagonal* scatter plots to a standardized window
h.axes[1,0].set_xlim([-xmax, xmax]);
h.axes[1,0].set_ylim([-xmax, xmax]);

# Prettify the overall plot
plt.tight_layout();
plt.suptitle(
    "mean %s\ncov  %s" % (
        str(nonzero_mean_2),
        str(neg_corr_cov_22).replace("\n","\n     ")),
    fontsize=14,
    horizontalalignment='left',
    font='monospace');
```

## Part 2: Estimate covariances from data

Let's take the *data* $x$ we observed in our previous plot above, and see if we can estimate the empirical covariance.

Remember, the data is in array `x_N2`

```python
# Print out the array to remember what we're working with.
print("x_N2: shape = %s" % str(x_N2.shape))
print("----")
print(x_N2[:5])
print("...")
print(x_N2[-5:])
```

    x_N2: shape = (10000, 2)
    ----
    [[ 3.724 -1.948]
     [-0.31   0.768]
     [ 2.17  -0.52 ]
     [ 2.485 -0.895]
     [ 2.156 -0.222]]
    ...
    [[ 0.873 -0.055]
     [ 4.296 -2.234]
     [ 0.007  0.397]
     [ 0.955  0.053]
     [ 0.706  0.336]]

## Exercise 2a: Compute the empirical covariance for $x_1, x_1$

Remember, the general formula for the covariance of two random variables is:

$$
\text{Cov}(X_i, X_j) = \frac{1}{N} \sum_{n=1}^N (x_{ni} - \mu_i) (x_{nj} - \mu_j)
$$

Now, we want to compute for $i=1, j=1$

$$
\text{Cov}(X_1, X_1) = ?
$$

```python
mean_x1 = np.mean(x_N2[:,0])
cov_x1_x1 = -999.0 # TODO compute

print("Cov[x_1, x_1]")
print("-------------")
print(cov_x1_x1)
```

    Cov[x_1, x_1]
    -------------
    -999.0

## Exercise 2b: Compute the empirical covariance for $x_2, x_2$

```python
## TODO compute just like in 2a
```

## Exercise 2c: Compute the empirical covariance for $x_1, x_2$:

```python
## TODO compute just like 2a
```

## Exercise 2d: Compute the empirical covariance *matrix*

$$
\text{Cov}(X) = \frac{1}{N} \sum_{n=1}^N (x_{n} - \mu) (x_{n} - \mu)^T
$$

Inside the sum, that is an *outer product*

```python
## TODO compute covariance as a 2x2 matrix

# Please edit this starter code

mean_2 = np.mean(x_N2, axis=0) # compute emprical mean of each column of x as a 2-dim vector
cov_22 = np.zeros((2,2))
for n in range(N):
    # Add the outer product related to the n-th data vector
    cov_22 += np.outer(x_N2[n,:], x_N2[n,:])  # TODO fix me to use correct formula
cov_22 = cov_22 / N # divide by N

print(cov_22)
```

    [[ 6.945 -2.68 ]
     [-2.68   1.245]]

## Discussion 2e: Does your answers above match the "true" covariance that generated the data?

You should be able to look at the scalars in 2a, 2b, 2c, as well as the matrix in 2d, and see a good match.

Remember what the true cov was:

```python
print(neg_corr_cov_22)
```

    [[ 3.  -1.7]
     [-1.7  1. ]]

```python
# TODO discuss if your estimated empirical covariance is "close" to the true
```

## Demo 2f: A *short cut* to compute the empirical covariance

Use the built in function "numpy.cov".

See https://numpy.org/doc/stable/reference/generated/numpy.cov.html

```python
empirical_cov_22 = np.cov(x_N2.T, bias=True) # Make sure to pass data array as 2 x N
```

```python
print("empirical_cov_22")
print("----------------")
print(empirical_cov_22)
```

    empirical_cov_22
    ----------------
    [[ 3.014 -1.71 ]
     [-1.71   1.006]]

## Demo 2g: Computing empirical covariance using matrix-vector ops

$$
\text{Cov}(X) = \frac{1}{N} A^T A, \quad A = X - \mu
$$

where $A$ is an $N \times 2$ matrix with each row containing the *centered* data $x_n - \mu$

```python
mu_2 = np.mean(x_N2, axis=0)
x_diff_mu_N2 = x_N2 - mu_2[np.newaxis,:] # subtract mu from each row of x
empirical_cov_22 = 1 / N * np.dot(x_diff_mu_N2.T, x_diff_mu_N2) # dot does a
print("empirical_cov_22")
print("----------------")
print(empirical_cov_22)
```

    empirical_cov_22
    ----------------
    [[ 3.014 -1.71 ]
     [-1.71   1.006]]

```python

```

# Part 3: Visualization of the PDF as a *contour* plot

In lecture, we learned that the contours of 2D Gaussians are elliptical.

Below, we see a function that shows in NumPy how to plot the contours of a given Gaussian distribution.

Please skim thru the function to understand how it works.

```python
def plotGauss2DEllipticalContours(
        mean_D, cov_DD,
        color='b',
        radius_lengths=[0.333, 1.333, 3.0], # just chosen to look "reasonable"
        markersize=3.0,
        unit_circle_radian_step_size=0.03,
        ax_handle=None,
        ):
    ''' Plot elliptical contours for provided mean and covariance

    Args
    ----
    mu_D : 1D numpy array, shape (D,) = (2,)
        Must be a 2-d mean vector
    cov_DD : 2D numpy array, shape (D,D) = (2,2)
        Must be a 2x2 symmetric, positive definite covariance
    radius_lengths : list
        Each entry gives the "radius" of a desired contour.
        FUTURE: could be computed to contain certain fractions of all probability mass.

    Post Condition
    --------------
    Plot created on current axes (or the provided axis)
    '''
    # If user provided an axis, use that
    if ax_handle is not None:
        plt.sca(ax_handle)

    mu_D = np.squeeze(np.asarray(mean_D))
    D = mu_D.size
    cov_DD = np.asarray(cov_DD)
    assert mu_D.shape == (2,)
    assert cov_DD.shape == (2,2)

    # Decompose cov matrix into eigenvalues "lambda[d]" and eigenvectors "U[:,d]"
    lambda_D, U_DD = np.linalg.eig(cov_DD)

    # Verify orthonormal
    assert np.allclose(np.eye(D), np.dot(U_DD, U_DD.T))
    # View eigenvector matrix as a rotation transformation
    rot_DD = U_DD

    # Prep for plotting elliptical contours
    # by creating grid of G different (x,y) points along perfect circle
    # Recall that a perfect circle is swept by considering all radians between [-pi, +pi]
    t_G = np.arange(-np.pi, np.pi, unit_circle_radian_step_size)
    x_G = np.sin(t_G)
    y_G = np.cos(t_G)
    Zcirc_DG = np.vstack([x_G, y_G])

    # Warp circle into ellipse defined by Sigma's eigenvectors
    # Rescale according to eigenvalues
    Zellipse_DG = np.sqrt(lambda_D)[:,np.newaxis] * Zcirc_DG
    # Rotate according to eigenvectors
    Zrotellipse_DG = np.dot(rot_DD, Zellipse_DG)

    # Plot contour lines across several radius lengths
    for r in radius_lengths:
        Z_DG = r * Zrotellipse_DG + mu_D[:, np.newaxis]
        plt.plot(
            Z_DG[0], Z_DG[1], '.-',
            color=color,
            markersize=markersize,
            markerfacecolor=color,
            markeredgecolor=color)

```

## Discussion 3a: Why do we use the square root of the eigenvalues (lambda_D) to warp the circle into an ellipse?

```python
# TODO discuss
```

## Discussion 3b: What properties of the eigenvector matrix make it a valid rotation matrix?

```python
# TODO discuss
```

# Contours for a Standard Normal with D=2

```python
mu_D = np.asarray([0, 0])
cov_DD = np.asarray([[1, 0 ], [0, 1]])

# First, plot N samples
N = 5000
x_N2 = scipy.stats.multivariate_normal.rvs(mu_D, cov_DD, N)
plt.plot(x_N2[:,0], x_N2[:,1], 'r.')

# Second, plot the contours
plotGauss2DEllipticalContours(mu_D, cov_DD)

B = 8
plt.gca().set_aspect('equal', 'box');
plt.gca().set_xlim([-B, B]);
plt.gca().set_ylim([-B, B]);
```

## Demo: Contours for 2D Normal w/ 3-times-larger variance on x axis

```python
mu_D = np.asarray([0, 0])
cov_DD = np.asarray([[3, 0 ], [0, 1]])

# First, plot N samples
N = 5000
x_N2 = scipy.stats.multivariate_normal.rvs(mu_D, cov_DD, N)
plt.plot(x_N2[:,0], x_N2[:,1], 'r.')

# Second, plot the contours
plotGauss2DEllipticalContours(mu_D, cov_DD)

B = 8
plt.gca().set_aspect('equal', 'box');
plt.gca().set_xlim([-B, B]);
plt.gca().set_ylim([-B, B]);
```

## Exercise 3c: Contours for 2D Normal w/ strong negative correlation

```python
mu_D = np.asarray([0, 0])
cov_DD = np.asarray([[1, -1.5], [-1.5, 4]])

# TODO make plots as above.
```

```python

```

## Exercise 3d: Can you show the covariance matrix is positive definite?

You should try:

* Compute the eigenvalues and show they are all positive.
* * See https://numpy.org/doc/stable/reference/generated/numpy.linalg.eig.html
* Compute the determinant using `np.linalg.det` and show it is positive.
* * See https://numpy.org/doc/stable/reference/generated/numpy.linalg.det.html
* Show that a cholesky factorization exists
* * See https://numpy.org/doc/stable/reference/generated/numpy.linalg.cholesky.html

First, print out the covariance matrix

```python
cov_DD
```

    array([[ 1. , -1.5],
           [-1.5,  4. ]])

```python
# TODO compute eigenvalues and show they are all positive. See np.linalg.eig

```

```python
# TODO call np.linalg.det on cov_DD

```

```python

```

```python
L_DD = np.linalg.cholesky(cov_DD)
print(L_DD)
```

    [[ 1.     0.   ]
     [-1.5    1.323]]

```python
# TODO show that matrix_multiply of L_DD and L_DD.T reconstructs cov_DD
# use np.dot(A, B) to do matrix multiplication

```

```python

```


# Vibrational Polarizability: The Harmonic Oscillator and the Morse Oscillator

## Motivation: Nuclear Schrödinger Equation for a Diatomic Molecule
The nuclear Schrödinger equation for a diatomic molecule is

$$
\left(-\frac{\hbar^2}{2M_1}\nabla^2_1 -\frac{\hbar^2}{2M_2}\nabla^2_2 + E_{\text{electronic}}(\mathbf{R}_1,\mathbf{R}_2)\right) \chi(\mathbf{R}_1,\mathbf{R}_2) = E_{\text{total}}\chi(\mathbf{R}_1,\mathbf{R}_2)
$$

The potential energy surface on which the nuclei move, $E_{\text{electronic}}\left(\mathbf{R}_1,\mathbf{R}_2\right)$, depends only on the separation between the molecules,

$$
E_{\text{electronic}}\left(\mathbf{R}_1,\mathbf{R}_2\right) = V\left( \left|\mathbf{R}_1 - \mathbf{R}_2 \right| \right)
$$

This suggests changing coordinates using the [center of mass](https://en.wikipedia.org/wiki/Center_of_mass). Specifically, we define a coordinate that describes the position of the center of mass

$$
\mathbf{R} = \frac{M_1 \mathbf{R}_1 + M_2\mathbf{R}_2}{M_1 + M_2}
$$

and a coordinate that describes the internuclear position

$$
\mathbf{u} = \mathbf{R}_2 - \mathbf{R}_1
$$

To rewrite the Hamiltonian in this new coordinate system, we need to rewrite the kinetic energy in the new coordinates. To do this, notice that differentiation with respect to the old (Cartesian) coordinates of the nuclei can be re-expressed in the new coordinates as:

$$
\begin{align}
\frac{df}{dX_1} &= \frac{df}{du_x}\frac{du_x}{dX_1} + \frac{df}{dR_x}\frac{dR_x}{dX_1} \\
&= -\frac{df}{du_x} + \frac{M_1}{M_1+M_2}\frac{df}{dR_x} \\
\frac{df}{dX_2} &= \frac{df}{du_x}\frac{du_x}{dX_2} + \frac{df}{dR_x}\frac{dR_x}{dX_2} \\
&= \frac{df}{du_x} + \frac{M_2}{M_1+M_2}\frac{df}{dR_x}
\end{align}
$$

So the momentum operators for the nuclei can be written as:

$$
\hat{\mathbf{p}}_1 = -\hat{\mathbf{p}}_u + \frac{M_1}{M_1+M_2}\hat{\mathbf{p}}_R \\
\hat{\mathbf{p}}_2 = \hat{\mathbf{p}}_u + \frac{M_2}{M_1+M_2}\hat{\mathbf{p}}_R
$$

and the kinetic energy operators are

$$
\frac{\hat{\mathbf{p}}_1^2}{2M_1} = \frac{\hat{\mathbf{p}}_u^2}{2M_1} + \frac{M_1\hat{\mathbf{p}}_R^2}{2(M_1+M_2)^2} - \frac{1}{2(M_1+M_2)}\left(\hat{\mathbf{p}}_R \cdot \hat{\mathbf{p}}_u + \hat{\mathbf{p}}_u \cdot \hat{\mathbf{p}}_R\right)\\
\frac{\hat{\mathbf{p}}_2^2}{2M_1} = \frac{\hat{\mathbf{p}}_u^2}{2M_2} + \frac{M_2\hat{\mathbf{p}}_R^2}{2(M_1+M_2)^2} + \frac{1}{2(M_1+M_2)}\left(\hat{\mathbf{p}}_R \cdot \hat{\mathbf{p}}_u + \hat{\mathbf{p}}_u \cdot \hat{\mathbf{p}}_R\right)
$$

Adding together these expressions gives the Schrödinger equation in the new coordinate system,

$$
\left(-\frac{\hbar^2}{2} \left(\frac{1}{M_1} + \frac{1}{M_2} \right) \nabla^2_u
      -\frac{\hbar^2}{2} \left(\frac{1}{M_1 + M_2} \right)\nabla^2_R
      + V(u) \right) \chi(\mathbf{R},\mathbf{u}) = E_{\text{total}}\chi(\mathbf{R},\mathbf{u})
$$

or, introducing the reduced mass

$$
\mu = \frac{M_1 M_2}{M_1 + M_2} = \left(\frac{1}{M_1} + \frac{1}{M_2} \right)^{-1} \\
$$

and total mass

$$
M = M_1 + M_2
$$

$$
\left(-\frac{\hbar^2}{2 \mu}-\frac{\hbar^2}{2M}\nabla^2_R + V(u) \right) \chi(\mathbf{R},\mathbf{u}) = E_{\text{total}}\chi(\mathbf{R},\mathbf{u})
$$

This Schrödinger equation can be solved by separation of variables. The Schrödinger equation for the center of mass represents the translational motion of the molecule; in the absence of a potential confining the molecule, this is just the Schrödinger equation for a free particle,

$$
-\frac{\hbar^2}{2M}\nabla^2_R \eta(\mathbf{R}) = E \eta(\mathbf{R})
$$

This contributes nothing to the energy if we assume the molecule is not moving (so that its kinetic energy is zero). If the molecule were confined, that confining potential would be inserted here.

The Schrödinger equation for the internuclear coordinate represents the rotations and vibrations of the molecule:

$$
\left(-\frac{\hbar^2}{2 \mu}\nabla_u^2 + V(u) \right) \varphi(\mathbf{u}) = E_{\text{rovib}}\varphi(\mathbf{u})
$$

The Hamiltonian in this equation is typically called the rovibrational Hamiltonian; its energy is the rovibrational energy. Because the potential energy depends only on the internuclear distance, the system is spherically symmetric. Separating out the angular dependence in the usual way,

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}+ \frac{2}{u} \frac{d}{du}\right)
      + \frac{\hat{L}^2(\theta_u,\phi_u)}{2\mu u^2} + V(u) \right)
      \varphi_{k,J,M_J}(u,\theta_u,\phi_u)
= E_{k,J,M_J}\varphi_{k,J,M_J}(u,\theta_u,\phi_u)
$$

Recall that $\hat{L}^2$ is the angular momentum squared, and its eigenfunctions are the spherical harmonics, which we have chosen to denote with a different quantum number to avoid confusion with the electronic problem,

$$
\begin{align}
\hat{L}^2 Y_J^{M_J}(\theta_u,\phi_u) = \hbar^2 J(J+1) Y_J^{M_J}(\theta_u,\phi_u) \qquad \qquad J&=0,1,2,\ldots \\ M_J &= 0,\pm 1,\ldots\,\pm J
\end{align}
$$
The exact wavefunction is therefore the product of a radial wavefunction and a spherical harmonic,

$$
\varphi_{k,J,M_J}(u,\theta_u,\phi_u) = R_k(u)  Y_J^{M_J}(\theta_u,\phi_u)
$$

where the radial wavefunction and the rovibrational energy are obtained by solving

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}+ \frac{2}{u} \frac{d}{du}\right)
      + \frac{\hbar^2 J(J+1)}{2\mu u^2} + V(u) \right)
      R_{k}(u) = E_{k,J}R_{k}(u)
$$

### Quadratic Approximation to the Potential Energy Curve and the Rigid Rotor Approximation
The potential energy of a diatomic molecule goes to infinity at short internuclear separation ($u \rightarrow 0$) because of the nuclear-nuclear repulsion and approaches a constant (which we typically define to be zero) at infinite internuclear distance. In between these limits, there is typically a minimum at $r_e$, which represents the equilibrium bond length for the molecule (in the absence of quantum effects). If we assume that the atomic nuclei, which are much more massive than electrons, have relatively small De Broglie wavelength and do not deviate far from $u = r_e$, it is reasonable to expand the potential energy in a Taylor series about that point,

$$
V(u) = V(r_e) + V'(r_e)(u-r_e) + \tfrac{1}{2!}V''(r_e)(u - r_e)^2 + \cdots
$$

Because we are expanding around the minimum, $V'(r_e) = 0$. The term $V(r_e)$ only shifts the total energy of the system up or down by a constant, so we can *choose* to set the zero of the energy scale so that $V(r_e) = 0$. Neglecting higher-order terms in the expansion, which we hope will be small, we can then write

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}+ \frac{2}{u} \frac{d}{du}\right)
      + \frac{\hbar^2 J(J+1)}{2\mu u^2} + \tfrac{1}{2}V''(r_e)(u-r_e)^2 \right)
      R_{k}(u) = E_{k,J}R_{k}(u)
$$

The rotational contribution can also be simplified by taking a Taylor series:

$$
\frac{\hbar^2}{2\mu u^2} = \frac{\hbar^2}{2\mu r_e^2} - \frac{2\hbar^2}{2\mu r_e^3}(u - r_e) + \cdots
$$

The first and higher-order terms in the series represent centrifugal distortion, which indicates that a molecule rotates faster and faster, larger bond lengths are favored. Such effects are usually small for low-energy vibrational states of strong bonds, where it is reasonable to make a [*rigid rotor approximation*](https://en.wikipedia.org/wiki/Rigid_rotor#Quantum_mechanical_rigid_rotor) and truncate the Taylor series after the zeroth order term. The rotational and vibrational problems then decouple, with the rotational wavefunctions and energies being given by:

$$
\frac{\hat{L}^2}{2\mu r_e^2} Y_J^{M_J}(\theta_u,\phi_u) = \frac{\hbar^2 J(J+1)}{2 \mu r_e^2}      Y_J^{M_J}(\theta_u,\phi_u)
$$

and the vibrational Hamiltonian is then approximated as a [quantum mechanical harmonic oscillator](https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator):

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2} + \frac{2}{u} \frac{d}{du}\right) + \tfrac{1}{2}V''(r_e)(u-r_e)^2 \right)
      R_{k}(u) = E_{k}R_{k}(u)
$$

```python
# It's useful to import these libraries.
# You can import others or not even use these, though.
import numpy as np
import scipy
from scipy import constants
```

#### 🖩 Exercise: Isotope Effects in Rotational Spectroscopy
For a diatomic molecule that has a permanent dipole moment, allowed rotational transitions correspond to $\Delta J = \pm 1$.

- A measurement informs you that the transition associated with the $J=0$ to $J=1$ transition in ${}^1\text{H}{}^{35}\text{Cl}$ (note the isotope labels) is characterized by $\tilde{\nu} = \tfrac{1}{\lambda} = 20.9 \text{ cm}^{-1}$. **What is the equilibrium bond length of HCl in picometers?**
- Assume that $\text{T}{}^{37}\text{Cl}$ (i.e., ${}^3\text{H}{}^{37}\text{Cl}$) molecule has the same bond length. **What is the wavenumber of the $J=0$ to $J=1$ transition for $\text{T}{}^{37}\text{Cl}$ in $\text{ cm}^{-1}$?**

#### Answer:

Let's start with the second question because it is easier.

The excitation energy is given by the expression

$$
\Delta E = E(J=1) - E(J=0) = \frac{\hbar^2}{2\mu r_e^2}(2) - \frac{\hbar^2}{2\mu r_e^2}(0) = \frac{\hbar^2}{\mu r_e^2}
$$

We also know that the wavenumber of the transitions is related to the energy of the transition by

$$
\Delta E = h \nu = \frac{hc}{\lambda} = hc \tilde{\nu}
$$

Using the above expression for the change in energy, we see that:

$$
\frac{\Delta E_{{}^1\text{H}{}^{35}\text{Cl}}}{\Delta E_{{}^3\text{H}{}^{37}\text{Cl}}}
= \frac{\tilde{\nu}_{{}^1\text{H}{}^{35}\text{Cl}}}{\tilde{\nu}_{{}^3\text{H}{}^{37}\text{Cl}}}
= \dfrac{\dfrac{\hbar^2}{\mu_{{}^1\text{H}{}^{35}\text{Cl}} r_e^2}}{\dfrac{\hbar^2}{\mu_{{}^3\text{H}{}^{37}\text{Cl}} r_e^2}}
= \frac{\mu_{{}^3\text{H}{}^{37}\text{Cl}}}{\mu_{{}^1\text{H}{}^{35}\text{Cl}}}
$$

So:

$$
\tilde{\nu}_{{}^3\text{H}{}^{37}\text{Cl}} = \tilde{\nu}_{{}^1\text{H}{}^{35}\text{Cl}} \left(\frac{\mu_{{}^1\text{H}{}^{35}\text{Cl}}}{\mu_{{}^3\text{H}{}^{37}\text{Cl}}} \right)
$$

To a reasonable approximation, we can assume that the masses of these nuclides are integers. (This is true to the number of significant figures given as input.) So:

$$
\tilde{\nu}_{{}^3\text{H}{}^{37}\text{Cl}} = 20.9 \text{cm}^{-1} \left(\frac{\frac{1 \cdot 35}{36}}{\frac{3 \cdot 37}{3+37}}\right) =  20.9 \text{ cm}^{-1} \cdot 0.35035 = 7.32 \text{ cm}^{-1}
$$

For part A, we need to use the formula

$$
hc \tilde{\nu} = \Delta E = \frac{\hbar^2}{\mu r_e^2}
$$

Rearranging,

$$
r_e^2 = \frac{\hbar^2}{\mu h c \tilde{\nu}} = \frac{(2 \pi)^2 h}{\mu h c \tilde{\nu}}
$$

```python
# Report your answers in this cell. I have initialize the variables to None.
re_HCl = None
wavenumber_T37Cl = None

# We start by computing the reduced mass of the species of interest. Use
mu_H35Cl = (1.00782*34.9688)/(1.00782+34.9688) * scipy.constants.value("atomic mass constant")
mu_T37Cl = (3.01605*36.9659)/(3.01605+36.9659) * scipy.constants.value("atomic mass constant")

# We are given:
wavenumber_H35Cl = 20.9*100 #wavenumber in m^-1
#So (Part B)
wavenumber_T37Cl = wavenumber_H35Cl * mu_H35Cl/mu_T37Cl / 100 #Divide by 100 to get to cm-1

# For Part A, we have the transition energy in Joules
dE_H35Cl = constants.h * constants.c * wavenumber_H35Cl
# The square of the bond length is.
re_sq_HCl = constants.hbar**2 / (dE_H35Cl * mu_H35Cl)
# Multiply by 1e12 to get picometers from meters
re_HCl = np.sqrt(re_sq_HCl) * 1e12

print(f"The equilibrium bond length of HCl is {re_HCl:.1f} pm.")
print(f"The wavenumber of the J=0 -> 1 transition in TCl(37) {wavenumber_T37Cl:.3f} cm-1.")
```

    The equilibrium bond length of HCl is 128.3 pm.
    The wavenumber of the J=0 -> 1 transition in TCl(37) 7.342 cm-1.

## The Harmonic Oscillator Hamiltonian

It is convenient to change the coordinates in this problem, so that the dependence on $r_e$ is not explicit. Defining

$$
x = u - r_e \\
\kappa_e = V''(r_e)
$$

the harmonic-oscillator Hamiltonian becomes:

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{dx^2}
+ \frac{2}{u} \frac{d}{dx}\right) + \tfrac{1}{2}\kappa_e x^2 \right)
      R_{k}(x) = E_{k,J}R_{k}(x)
$$

In the picture of a diatomic molecule where the atoms are connected to each other with a harmonic spring $x$ is the deviation of the spring from its equilibrium length and $\kappa_e$ is the force constant for the spring. The angular frequency for the spring is

$$
\omega = \sqrt{\frac{\kappa_e}{\mu}}
$$

and the reduced mass is defined as previously,

$$
\mu = \frac{M_1 M_2}{M_1 + M_2}
$$

### Solutions to the Harmonic Oscillator Hamiltonian
We will not attempt to solve the harmonic oscillator Schrödinger equation. It suffices to note that the equation *is* exactly solvable using techniques similar to those we've used for other systems, and that the solution is written in terms of (yet another) type of special function, the [Hermite polynomials](https://en.wikipedia.org/wiki/Hermite_polynomials):

$$
\begin{align}
H_0(y) &= 1 \\
H_1(y) &= 2y \\
H_2(y) &= 4y^2 - 2 \\
H_3(y) &= 8y^3 - 12 \\
H_n(y) &= 2y H_{n-1}(y) - 2(n-1)H_{n-2}(y)
\end{align}
$$

As with the associated Laguerre polynomials, one must be careful because there are (at least) two different definitions of the Hermite polynomials that are prevalent in the literature. These are the-called [physicist's Hermite polynomials](https://en.wikipedia.org/wiki/Hermite_polynomials#Definition). Specifically, we have:

$$
R_k(x) = \sqrt{\frac{1}{2^k k!}}\left(\frac{\mu \omega}{\pi \hbar} \right)^{\frac{1}{4}}H_k\left(x\sqrt{\frac{\mu \omega}{\hbar}}\right) \exp \left[-\frac{1}{2} \left(x\sqrt{\frac{\mu \omega}{\hbar}}\right)^2  \right]
$$

and the corresponding eigenvalues are

$$
E_k = \hbar \omega \left( k+ \tfrac{1}{2} \right) = \hbar \sqrt{\frac{\kappa_e}{\mu}} \left( k+ \tfrac{1}{2} \right) \qquad \qquad k=0,1,2,\ldots
$$

### 🖩 Exercise: Isotope Effects in Vibrational Spectroscopy
For a diatomic molecule that is well-described by a harmonic oscillator, allowed vibrational transitions correspond to $\Delta k = \pm 1$.

- A measurement informs you that the fundamental vibrational transition associated with $k=0$ to $k=1$ in ${}^7\text{Li}{}^{7}\text{Li}$ (note the isotope labels) is characterized by $\tilde{\nu} = \tfrac{1}{\lambda} = 351.43 \text{ cm}^{-1}$. **What is the force constant, $\kappa_e$, for the Lithium dimer in Newton/meter?**
- Assume that ${}^6\text{Li}{}^{6}\text{Li}$ molecule has the same force constant. **What is the wavenumber of the $k=0$ to $k=1$ transition of ${}^6\text{Li}{}^{6}\text{Li}$ in $\text{cm}^{-1}$?**

#### Solution:
The energy of the transition is

$$
\Delta E = \hbar \omega = \hbar \sqrt{\frac{\kappa_e}{\mu}} = h c \tilde{\nu}
$$

So

$$
\tilde{\nu} = \sqrt{\frac{\kappa_e}{4c^2 \pi^2 \mu}}
$$

and

$$
\kappa_e = \frac{h^2 c^2 \tilde{\nu}^2 \mu}{\hbar^2} = 4 \pi^2 c^2 \tilde{\nu}^2 \mu
$$

For the second part, we take the ratio of the wavenumbers,

$$
\frac{\tilde{\nu}_{{}^6\text{Li}_2}}{\tilde{\nu}_{{}^7\text{Li}_2}}
= \frac{\sqrt{\frac{\kappa_e}{4c^2 \pi^2 \mu_{{}^6\text{Li}_2}}}}{\sqrt{\frac{\kappa_e}{4c^2 \pi^2 \mu_{{}^7\text{Li}_2}}}}
= \sqrt{\frac{\mu_{{}^7\text{Li}_2}}{\mu_{{}^6\text{Li}_2}}}=\sqrt{\frac{49\cdot 12 }{36 \cdot 14}}=1.0801
$$

So

$$
\tilde{\nu}_{{}^6\text{Li}_2} = 379.59 \text{ cm}^{-1}
$$

```python
# Report your answers in this cell. I have initialize the variables to None.
kappa_e = None                #Force constant in N/m
wavenumber_6Li_2 = None       #wavenumber in cm-1

# We start by computing the reduced mass of the species of interest. Use
mu_6Li_2 = (6.0151**2)/(2*6.0151) * scipy.constants.value("atomic mass constant")
mu_7Li_2 = (7.0160**2)/(2*7.0160) * scipy.constants.value("atomic mass constant")

# We are given:
wavenumber_7Li_2 = 351.43*100 #wavenumber in m^-1
#So (Part B)
wavenumber_6Li_2 = wavenumber_7Li_2 * np.sqrt(mu_7Li_2/mu_6Li_2) / 100 #Divide by 100 to get to cm-1

# For Part A, we have the transition energy in Joules
dE_7Li_2 = constants.h * constants.c * wavenumber_7Li_2
# Therefore
sqrt_kappa_e = dE_7Li_2 * np.sqrt(mu_7Li_2) / constants.hbar
# The force constant is
kappa_e = sqrt_kappa_e**2

print(f"The force constant at equilibrium bond length of the Lithium dimer is {kappa_e:.2f} N/m.")
print(f"The wavenumber of the k=0 -> 1 transition in the Li-6 dimer is {wavenumber_6Li_2:.1f} cm-1.")
```

    The force constant at equilibrium bond length of the Lithium dimer is 25.53 N/m.
    The wavenumber of the k=0 -> 1 transition in the Li-6 dimer is 379.5 cm-1.

## The Morse Oscillator

### Motivation
The harmonic oscillator is not very realistic for two reasons
- it predicts that negative bond lengths would be possible, since the potential energy curve does not diverge to infinity as $u \rightarrow 0$.
- it predicts that chemical bonds never break, since the potential energy curve does not approach an asymptotic constant as $u \rightarrow \infty$.

The last issue, which is the more severe one from the standpoint of qualitative chemical behavior, can be remedied by replacing the harmonic oscillator potential with the Morse potential,

$$
V_{\text{Morse}}(u) = D_e \left(1 - e^{-a(u-r_e)} \right)^2
$$

Here $D_e$ is the dissociation constant (the energy it takes to break the bond between the atoms) and $a$ is related to the force constant by

$$
a = \sqrt{\frac{\kappa_e}{2D_e}}
$$

Recall that

$$
\kappa_e = V''_{\text{Morse}}(r_e)
$$

### Solution
The Schrödinger equation for the Morse potential,

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}
+ \frac{2}{u} \frac{d}{du}\right) + V_{\text{Morse}}(u) \right)
      \phi_{k}(u) = E_{k}\phi_{k}(u)
$$

can also be solved exactly. The eigenfunctions are [quite complicated functions of the associated Laguerre polynomials](https://en.wikipedia.org/wiki/Morse_potential#Vibrational_states_and_energies) but the energies have a relatively simple expression,

$$
E_k = \hbar \omega_0 \left(k+\tfrac{1}{2}\right) - \frac{\left[ \hbar \omega_0 \left(k+\tfrac{1}{2}\right) \right]^2}{4 D_e}
\qquad \qquad k=0,1,2,\ldots,k_{\max}
$$

where, just as in the harmonic oscillator,

$$
\omega_0 = a \sqrt{\frac{2D_e}{\mu}} = \sqrt{\frac{\kappa_e}{\mu}}
$$

Unlike the harmonic oscillator, only a $k_{\max} < \infty$ states can be bound by the Morse potential, where

$$
k_{\max} = \left\lfloor \frac{2D_e}{\hbar \omega_0} - \frac{1}{2} \right\rfloor
$$

Here $\lfloor x \rfloor$ denotes the integer part of $x$. E.g., $\lfloor 20.3 \rfloor = 20$. The upshot of the equation for $k_{\max}$ is that the Morse oscillator has about twice as many states with energy less than $D_e$ as the harmonic oscillator does, because the states of the Morse oscillator become closer and closer together as the molecule gets closer to dissociation. The ability to have only a finite number of bound vibrational states is one way in which the Morse oscillator is more like a real vibrating molecule than the harmonic oscillator.

### 🖩 Exercise: Compare the Morse and Harmonic Oscillators
A measurement informs you that the fundamental vibrational transition associated with $k=0$ to $k=1$ in ${}^7\text{Li}{}^{7}\text{Li}$ (note the isotope labels) is characterized by $\omega_0 = 6.620\cdot 10^{13} \text{Hz}$. Moreover, the dissociation energy is $1.6918\cdot 10^{-19} \text{ Joules}$.
- How many bound states does ${}^7\text{Li}_2$ have in the Morse oscillator?
- How many bound states does ${}^6\text{Li}_2$ have in the Morse oscillator? Assume that the Born-Oppenheimer approximation holds.
- What is the zero-point energy of ${}^6\text{Li}_2$ in the harmonic oscillator in Joules?
- What is the zero-point energy of ${}^6\text{Li}_2$ in the Morse oscillator in Joules?

#### Answer:

This can be seen as a continuation of the previous problem (2.16). Moreover, we realize that the formula for $\omega_0$ in the Morse oscillator is *exactly* the same as the formula for $\omega$ in the harmonic oscillator.

The $\omega_0$ value for the Li-6 dimer is

$$
\frac{\omega_{0;{{}^6\text{Li}_2}}}{\omega_{0;{{}^7\text{Li}_2}}}
= \sqrt{\frac{\mu_{{}^7\text{Li}_2}}{\mu_{{}^6\text{Li}_2}}}=\sqrt{\frac{49\cdot 12 }{36 \cdot 14}}=1.0801
$$

The number of states can be computed relatively easily using the equation

$$
k_{\max} = \left\lfloor \frac{2D_e}{\hbar \omega_0} - 1 \right\rfloor
$$

The zero-point energy of the Morse oscillator can be written as a correction to the zero-point energy of the harmonic oscillator. Specifically, using

$$
E_0^{(\text{harmosc})} = \tfrac{1}{2} \hbar \omega_0
$$

one has

$$
E_0^{(\text{Morse})} = E_0^{(\text{harmosc})} - \frac{\left(E_0^{(\text{harmosc})} \right)^2}{4 D_e}
$$

```python
# Enter the answers for the variables below. I have initialized the variables to None.
n_boundst_7Li_2 = None       #integer; number of bound states in Li-7 dimer
n_boundst_6Li_2 = None       #integer; number of bound states in Li-6 dimer
zero_pt_6Li_2_Morse = None   #float. zero-point energy in Li-6 dimer in the Morse potential in Joules
zero_pt_6Li_2_harmosc = None #float. zero-point energy in Li-6 dimer in the harmonic oscillator in Joules

# We start by computing the reduced mass of the species of interest. Use
mu_6Li_2 = (6.0151**2)/(2*6.0151) * scipy.constants.value("atomic mass constant")
mu_7Li_2 = (7.0160**2)/(2*7.0160) * scipy.constants.value("atomic mass constant")

# We are given:
omega0_7Li_2 = 6.620e13 #omega_0 for Li-7 dimer in Hz.
De = 1.6918e-19          #Dissociation energy for Li dimer in Joules.

# Computed omega_0 for the Li-6 dimer in Hz
omega0_6Li_2 = omega0_7Li_2 * np.sqrt(mu_7Li_2/mu_6Li_2)

# Parts A and B
n_boundst_7Li_2 = int(2*De/(constants.hbar*omega0_7Li_2) - 1./2)
n_boundst_6Li_2 = int(2*De/(constants.hbar*omega0_6Li_2) - 1./2)

# Parts D and C
zero_pt_6Li_2_harmosc = constants.hbar*omega0_6Li_2/2
zero_pt_6Li_2_Morse = zero_pt_6Li_2_harmosc - zero_pt_6Li_2_harmosc**2/(4*De)
```

```python
assert isinstance(n_boundst_7Li_2,int), "Type error: The number of bound states should be an integer."
assert isinstance(n_boundst_6Li_2,int), "Type error: The number of bound states should be an integer."

print(f"The number of bound states in the Morse oscillator for the Li-7 dimer is {n_boundst_7Li_2}.")
print(f"The number of bound states in the Morse oscillator for the Li-6 dimer is {n_boundst_6Li_2}.")
print(f"The zero-point energy of the Morse oscillator for the Li-6 dimer is {zero_pt_6Li_2_Morse:.3e} Joules.")
print(f"The zero-point energy of the harmonic oscillator for the Li-6 dimer is {zero_pt_6Li_2_harmosc:.3e} Joules.")

### BEGIN HIDDEN TESTS
assert(n_boundst_7Li_2 == 47)
assert(n_boundst_6Li_2 == 44)
assert(np.isclose(zero_pt_6Li_2_Morse,3.749e-21,rtol=1e-3))
assert(np.isclose(zero_pt_6Li_2_harmosc,3.770e-21,rtol=1e-3))
### END HIDDEN TESTS
```

    The number of bound states in the Morse oscillator for the Li-7 dimer is 47.
    The number of bound states in the Morse oscillator for the Li-6 dimer is 44.
    The zero-point energy of the Morse oscillator for the Li-6 dimer is 3.749e-21 Joules.
    The zero-point energy of the harmonic oscillator for the Li-6 dimer is 3.770e-21 Joules.

## Vibrating Diatomic Molecules  in an External Field
### Motivation
For polar diatomic molecules, their dipole moment may be approximated as:

$$
\mathbf{p} = q \left(\mathbf{R}_2 - \mathbf{R}_1 \right)
$$

where $q$ is the magnitude of the charges on the atoms and $\mathbf{R}_2$ is the positively charged atom. The energy of a polar diatomic molecule interacting with an electric field is then

$$
V_{\text{dipole}} = -\mathbf{p} \cdot \mathbf{E}
$$

This energy is minimized when the dipole aligns in the field, giving an energy lowering that is proportional to the product of the magnitude of the dipole moment and the electric field strength,

$$
V_{\text{dipole}} = -|\mathbf{p}| |\mathbf{E}| = - q u F
$$

where $q$ is the atomic charge, $u$ is the internuclear distance, and F is the field strength. This is a quite simple model, and neglects the following
- atomic charges change when the bond length changes
- atoms are not simply point charges
- a molecule's electrons rearrange in response to the electric field. To a first approximation, this means that the atomic charges change in the presence of the field.

Nonetheless, this model does capture some key features, notably the fact that in the presence of a uniform external field, the bond in a polar diatomic molecule tends to elongate.

### Hamiltonian for a Polar Diatomic Molecule in an External Uniform Electric Field
The Hamiltonian for a vibrating molecule in an electric field can then be approximated as

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}
+ \frac{2}{u} \frac{d}{du}\right) + V(u) - q_e u F \right)
      R_{k}(u) = E_{k}R_{k}(u)
$$

where $q_e$ is magnitude of the atomic charge for $u = r_e$. If the equilibrium dipole moment of the molecule is known, then

$$
q_e = \frac{\left[\text{dipole moment in Debye}\right] \cdot 3.33564 \cdot 10^{-30} \text{ C/m}}{\text{bond length in meters}} \text{ Coulombs}
$$

or, in terms of the charge on the electron $e$,

$$
q_e = \frac{\left[\text{dipole moment in Debye}\right] \cdot 2.08919 \cdot 10^{-11} e/\text{m}}{\text{bond length in meters}} e
$$

The most interesting effects are not those that are associated with the classical interaction energy of the dipole with the field, we usually subtract that energy from the Hamiltonian, so that we only see the change in the molecule's energy that was *induced* by the field. I.e., we usually focus on the vibrational dipole polarization energy,

$$
E_k^{(\text{polarization})}(F) = E_k(F) - \left[ E_k(F=0) - q_e r_e F \right]
$$

The vibrational dipole polarization energy can be computed from the shifted Schrödinger equation,
where

$$
\begin{align}
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}+ \frac{2}{u} \frac{d}{du}\right) + V(u) - q_e (u-r_e) F \right)
      \psi_{k}(u) &= \left(E_k(F=0) + E_{k}^{(\text{polarization})}(F)\right)\psi_{k}(u) \\
      &= E_{k}^{(\text{shifted})}(F)\psi_{k}(u)
\end{align}
$$

where we have (re)defined the energy as

$$
E_k^{(\text{shifted})}(F) = E_k(0) + E_k^{(\text{polarization})}(F)
$$

The total energy of the vibrating diatomic molecule can then be expressed as:

$$
E_k(F) = E_k(0) - q_e r_e F + E_k^{(\text{polarization})}(F)
$$

If one expands the vibrational polarization energy in a Taylor series,

$$
E_k^{(\text{polarization})}(F) = -\tfrac{1}{2} \alpha_{\text{vib}} F^2 - \tfrac{1}{3!} \beta_{\text{vib}} F^3 - \tfrac{1}{4!} \gamma_{\text{vib}} F^3 + \cdots
$$

where $\alpha_{\text{vib}}$ is the vibrational contribution to the dipole polarizability (subject to the approximations mentioned above) and the higher-order coefficients are the first-, $\beta$, and higher-order vibrational dipole hyperpolarizabilities.

### The Harmonic Oscillator in a Uniform External Electric Field
To model a diatomic molecule vibrating in a uniform electric field, we need to choose a practical model for the vibrational potential energy function, $V(u)$. The simplest possible model is to use a Harmonic Oscillator, which leads to the Hamiltonian:

$$
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}
+ \frac{2}{u} \frac{d}{du}\right) + \tfrac{1}{2}\kappa_e(u-r_e)^2 - q_e (u-r_e) F \right)
      R_{k}(u) = E_{k}^{\text{shifted}}(F) R_{k}(u)
$$

#### Sketch of Perturbative Approach
One could treat this Hamiltonian with perturbation theory. By construction, $E_k^{(\text{shifted})}$ does not depend on the field to first order. Therefore, the leading-order term is the second-order contribution,

$$
\left[\frac{d^2 E_k}{dF^2}\right]_{F=0}
=\left[\frac{d^2E_k^{(\text{polarization})}}{dF^2}\right]_{F=0}=\left[\frac{d^2E_k^{(\text{shifted})}}{dF^2}\right]_{F=0} = 2q_e \sum_{j=0\\j \ne k}^{\infty}\frac{ \left|\langle R_j(0) | u-r_e |R_k(0) \rangle \right|^2}{E_k(F=0) - E_j(F=0)}
$$

Evaluating this expression is feasible because the integrals that one needs,

$$
V_{kl} = \int_{-\infty}^{\infty} R_k(u)(u-r_e) R_l(u) du
$$

have a simple explicit formulae. Specifically,

$$
V_{kl} =
\begin{cases}
\sqrt{\frac{k}{2}}   & k=l+1 \\
\sqrt{\frac{k+1}{2}} & k=l-1 \\
0  & \text{otherwise}
\end{cases}
$$

#### Exact Solution by Completing the Square
While a perturbative approach is possible, the harmonic oscillator in a uniform external electric field can be solved exactly. [Completing the square](https://en.wikipedia.org/wiki/Completing_the_square) on the potential,

$$
\tfrac{1}{2}\kappa_e(u-r_e)^2 - q_e (u-r_e) F = \tfrac{1}{2}\kappa_e\left(u - r_e - \frac{qF}{\kappa_e}\right)^2 - \frac{q^2F^2}{2 \kappa_e}
$$

The force constant in this new shifted harmonic oscillator is the still $\kappa_e$, but the bond length

$$
r_e(F) = r_e + \frac{qF}{\kappa_e}
$$

and the energies

$$
E_k^{\text{shifted}}(F) = \hbar \sqrt{\frac{\kappa_e}{\mu}}\left(k+\frac{1}{2}\right) - \frac{q^2F^2}{2 \kappa_e}
$$

have chagned. Referring to the definitions above, the vibrational polarization energy is:

$$
E_k^{\text{polarization}}(F) = - \frac{q^2}{2 \kappa_e}F^2
$$

the vibrational polarizability is

$$
\alpha_{\text{vib}} = \frac{q^2}{\kappa_e}
$$

and the total vibrational energy of the diatomic molecule in the field

$$
E_k(F) = \hbar \sqrt{\frac{\kappa_e}{\mu}}\left(k+\frac{1}{2}\right) - q r_e F- \frac{q^2}{2 \kappa_e}F^2
$$

### 🖩 Exercise: Model the HF molecule in a uniform electric field in the Harmonic Oscillator Approximation.
For the ${}^{1}\text{H}{}^{19}\text{F}$ molecule, $\mu = 6.361 \cdot 10^{-25} \text{ kg}$, $\kappa_e = 920 \text{ N/m} $, and $r_e = 91.68 \text{ pm}$. The dipole moment of HF is 1.91 Debye, which corresponds to a semi-reasonable charge of 0.44 $e$ on the hydrogen atom. Assume the electric field is 1,000,000 V/m.
- How much does the bond elongate in the presence of the external electric field? Report your answer in picometers.
- What value would you obtain if you used second-order perturbation theory to estimate the vibrational polarization energy, $E_k^{(\text{polarization})}$. Report your answer in Joules.

*Hint:* Results from the previous block of the notebook may be helpful.

#### Solution:
This is mostly a pretty straightforward "calculator" exercise. The only tricky part is the use of perturbation theory. However, note that the *exact* expression has only terms that are linear and quadratic in F. That means that second-order perturbation theory is *exact*. So you did not need to use perturbation theory. Even if you did, only *one* state would contribute, so the problem is pretty easy.....

```python
# Report your answers in the variables below, which I have initialized to None.
re_shift = None
e_polarization = None

charge_H = 1.91*3.33564e-30/91.68e-12   #Charge on the hydrogen in Coulombs
field = 1.0e6
kappa_e = 920
mu = 6.361e-25
re = 91.68e-12

re_shift = charge_H * field / kappa_e * 1e12           #multiply by 1e12 to get units of picometers.
e_polarization = -1 * charge_H**2 * field **2 / (2 * kappa_e)
e_shifted = constants.hbar * np.sqrt(kappa_e/mu) + e_polarization
e_true = e_shifted - charge_H * field * re

print(f"The bond length of HF elongates by {re_shift:.3e} pm in a field of 1,000,000 V/m.")
print(f"The zero-point energy HF changes by {e_polarization:.3e} J in a field of 1,000,000 V/m.")
```

    The bond length of HF elongates by 7.554e-05 pm in a field of 1,000,000 V/m.
    The zero-point energy HF changes by -2.625e-30 J in a field of 1,000,000 V/m.

### The Morse Oscillator in an External Electric Field
Instead of using a harmonic oscillator to describe the vibrational motion of a polar diatomic molecule, one could use the Morse Oscillator. In this case, the Schrödinger equation is:

$$
\begin{align}
\left(-\frac{\hbar^2}{2\mu} \left( \frac{d^2}{du^2}
+ \frac{2}{u} \frac{d}{du}\right) + V_{\text{Morse}}(u) - q_e (u-r_e) F \right)
      \phi_{k}(u) &= \left(E_k(F=0) + E_{k}^{(\text{polarization})}(F)\right)\phi_{k}(u) \\
      &= E_{k}^{(\text{shifted})}(F)\phi_{k}(u)
\end{align}
$$

Unlike the harmonic oscillator in an electric field, it is no longer possible to solve this system exactly. However, there are many possible ways to approximate the energy of a Morse oscillator in a uniform electric field, among them:
- Use the harmonic-oscillator in an external field as a zeroth-order Hamiltonian, and use perturbation theory to estimate the change in energy associated with using the Morse potential instead.
- Use the eigenfunctions of the harmonic oscillator in an electric field as a basis; expand the eigenfunctions of the Morse potential in an electric field in this basis.
- Use the Morse potential without an electric field as a zeroth-order Hamiltonian; use perturbation theory to estimate the effects of the electric field.
- Use the eigenfunctions of the Morse potential without an electric field as a basis; expand the eigenfunctions of the Morse potential in an electric field in this basis.

The first two approaches are arguably preferable mathematically, but the latter two are easier. This is because the matrix elements for the Morse potential in an electric field,

$$
V_{mn} = \int_{-\infty}^{\infty} \phi_m(u;F=0) (u - r_e) \phi_n(u;F=0) du
$$

are (relatively) easy to evaluate. Specifically we have:

$$
V_{m<n} = \frac{-1^{m-n+1}}{a(n-m)(2\lambda-n-m-1)}\left[\frac{n! \Gamma(2\lambda - n)(2\lambda - 2n - 1)(2\lambda - 2m - 1)}{m! \Gamma(2\lambda - m)} \right]^{\frac{1}{2}}
$$

In practice, this equation is numerically ill-conditioned when $m$, $n$, or $\lambda$ is large, because then some of the terms are gigantic. It is reasonable to cut off the expansion for large $m$ and large $n$; when $\lambda$ is large, one can use the Ramanujan expression for the $\Gamma$ function,

$$
\Gamma(y) = \sqrt{\pi} \left(\frac{y-1}{e} \right)^{y-1} \left(8(y-1)^3 + 4(y-1)^2 + (y-1) + \frac{1}{30} \right)^{\frac{1}{6}}
$$

to evaluate the square root of the ratio of the $\Gamma$ functions directly

$$
\frac{\Gamma(y)}{\Gamma(x)} = \left(\sqrt{\frac{y-1}{x-1}}\right)^{y-1} \left(\sqrt{\frac{e}{x-1}} \right)^{x-y}
\left(\frac{8(y-1)^3 + 4(y-1)^2 + (y-1) + \frac{1}{30}}{8(x-1)^3 + 4(x-1)^2 + (x-1) + \frac{1}{30}}\right)^{\frac{1}{12}}
$$

The code below actually uses the (more complicted) Stirling expansion, because in practice it seemed more accurate.

The diagonal elements are:

$$
V_{mm} \approx \frac{\frac{3}{2} + 3m + \frac{13/12 + (7/2)(m^2+m)}{2 \lambda} + \frac{(m+1)^4}{4\lambda^2}}{2\lambda a}
$$

In these expressions,

$$
\lambda = \frac{\sqrt{2 \mu D_e}}{a \hbar}
$$

Using the aforementioned definition of $a$,

$$
a = \sqrt{\frac{\kappa_e}{2D_e}}
$$

this simplifies to

$$
\lambda = \frac{2 D_e}{\hbar}\frac{\mu}{\kappa_e} = \frac{2 D_e}{\hbar \omega_0}
$$

> Note. Technically in both perturbation theory and in a variational approach one would need to consider the continuum states, corresponding to the states of the dissociated diatomic molecule. We are neglecting that tricky business here, but that is the main reason that an approach starting from the harmonic oscillator is mathematically preferable.

> Note. The choice of $u = |\mathbf{R}_1 - \mathbf{R}_2|$ or $u = |\mathbf{R}_2 - \mathbf{R}_1|$ is arbitrary. This means, in practice, that whether one chooses a field oriented in the $+u$ or $-u$ direction is arbitrary. One should compute both choices, and pick the one which stabilizes the molecule more. For a Morse oscillator, this corresponds to a field with a negative sign.

#### 👩🏽💻 Exercise: Model the HF molecule in a uniform electric field using the Morse potential.
For the ${}^{1}\text{H}{}^{19}\text{F}$ molecule, $\mu = 6.361 \cdot 10^{-25} \text{ kg}$, $\kappa_e = 920 \text{ N/m} $, $D_e = 9.402 \cdot 10^{-19} J$, and $r_e = 91.68 \text{ pm}$. The dipole moment of HF is 1.91 Debye, which corresponds to a semi-reasonable charge of 0.44 $e$ on the hydrogen atom. Assume the electric field is 1,000,000 V/m.

Using the integrals provided below, compute the vibrational polarization energy, $E_{k}^{(\text{polarization})}(F)$, for the Morse Oscillator in a field using perturbation theory and/or basis-set expansion.
1. Compute both first and second-order corrections to the polarization energy if you use perturbation theory. Note that in this case, the first-order contribution to the polarization energy is not exactly zero.
2. Summing over all bound states or using all bound states in the basis is not practical, but you should be sure to not use unbound states.

>Given how complicated this is, it is possible that *my* code below has some mistakes. I'm looking to test whether your code works well *assuming* that my matrix elements $V_{mn}$, are coded correctly.

```python
# I've initialized key variables to None. These quantities are printed in the next cell.
epol_pt = None    #Energy of polarization computed using first and second-order perturbation theory,
                  #    in Joules
epol_basis = None #Energy of polarization computed using basis-set expansion, in Joules

# The below code provides key quantities that are needed for this treatment.
# All inputs are assumed to be in SI units.

from scipy.linalg import eigh
from scipy.special import gamma,factorial
import math

def Ramanujan_ratio(x,m,n):
    """Computes (Gamma(2x-n)/Gamma(2x-m))^(1/2) using the Ramanujan approximation to the Gamma function"""
    ratio = np.sqrt((2*x-m-1)**(m+1))/np.sqrt((2*x-n-1)**(n+1)) \
            * (cubic_poly(2*x-n-1)/cubic_poly(2*x-m-1))**(1./12)  \
            * ((2*x-n-1)/(2*x-m-1))**x
    return ratio

def cubic_poly(y):
    """The cubic polynomial that shows up in the Ramanujan formula"""
    return 8*y**3 + 4*y**2 + 2*y + 1./30

def Stirling_ratio(x,y):
    """Computes (Gamma(y)/Gamma(x))^(1/2) using the Stirling approximation to the Gamma function
    This works best when x and y are approximately equal, and implicitly it is assumed that y < x.
    """
    ratio = (y/x)**(0.5*y-0.25) * np.sqrt(math.e/x)**(x-y) * np.sqrt(Stirling_poly(y)/Stirling_poly(x))
    return  ratio

def Stirling_poly(y):
    "Evaluates the polynomial part of the Stirling approximaton to the Gamma function"
    p = 0
    p = 1 + 1./(12*y) + 1./(288*y**2) - 139./(51840*y**3) - 571./(2488320*y**4) \
        + 163879/(209018880*y**5) + 5246819./(75246796800*y**6)
    return p

def compute_V(De, omega0, kappa_e, n_basis):
    """Compute the matrix <m|x|n> for a Morse oscillator

    Parameters
    ----------
    De : scalar
        The depth energy of the Morse potential in SI units (Joules)
    omega0 : scalar
        the fundamental vibrational frequency parameter in Hz; equal to (kappa_e/mu)^(1/2) where mu is the reduced mass.
    kappa_e : scalar
        the force constant in N/m; the curvature at the bottom of the potential well.
    n_basis : scalar, int
        the number of rows/columns in the V matrix to be computed

    Returns
    -------
    V : array-like (nbasis, nbasis)
         Matrix elements V_mn = <m | (u - r_e) | n> for the Morse Oscillator in units of meters

    Note: based on equations in
        Vasan & Cross; The Journal of Chemical Physics 78, 3869–3871 (1983).

    """

    # Compute key parameters
    lmbda = 2*De / (constants.hbar * omega0)  #lambda is a special word in Python so we can't use it. :-(
    a = np.sqrt(kappa_e/(2*De))

    # This program will crash and burn if the number of states is too large, because factorials of large numbers
    # are too big. So
    assert(n_basis < 150), "n_basis is too large, so we stop gracefully to avoid overflow"

    # initialize V to a zero matrix
    V = np.zeros((n_basis,n_basis))

    for n in range(n_basis):
        V[n,n] = (3/2 + 3*n + (13/12 + 7/12*(n**2 + n))/(2*lmbda) \
                  + (n+1)**4/(4*lmbda**2))/(2*lmbda*a)
        for m in range(n):
            prefactor = -1**(m-n+1)/a * 1/((n-m)*(2*lmbda -n - m -1))
            radicand = factorial(n)/factorial(m) * (2*lmbda - 2*n - 1) * (2*lmbda - 2*m - 1)
            if ((2*lmbda - min(m,n)) <= 50):
               #we can use the conventional expressions for the gamma function
               radicand *= gamma(2*lmbda - n)/gamma(2*lmbda - m)
               radical = np.sqrt(radicand)
            else:
               radical = np.sqrt(radicand)
               #we need to use an asymptotic formula to estimate the gamma-function ratio. Using the Stirling
               #based formula because it seems to work better.
               radical *= Stirling_ratio(2*lmbda-m,2*lmbda-n)
            V[m,n] = prefactor * radical
            V[n,m] = V[m,n]

    return V

def compute_E0(De, omega0, n_basis):
    """Compute the eigenenerties of the Morse oscillator in the absence of an electric field

    Parameters
    ----------
    De : scalar
        The depth energy of the Morse potential in SI units (Joules)
    omega0 : scalar
        the fundamental vibrational frequency parameter in Hz; equal to (kappa_e/mu)^(1/2) where mu is the reduced mass.
    n_basis : scalar, int
        the number of rows/columns in the V matrix to be computed

    Returns
    -------
    E0 : array-like (nbasis)
         eigenenergies of the Morse oscillator in units of Joules
               E0(m) =  hbar * omega0 (k + 1/2) - (hbar * omega0 (k + 1/2))^2 / 4 De

    """

    # initialize E0 to a zero vector
    E0 = np.zeros(n_basis)

    for m in range(n_basis):
        E0_harmosc = constants.hbar * omega0 * (m + 1/2)
        E0[m] = E0_harmosc - E0_harmosc**2/(4*De)

    return E0

def compute_H(De,omega0,kappa_e,charge,field,n_basis):
    """Compute the Hamiltonian matrix <m|H(field)|n> for a Morse oscillator of a molecular dipole in a field.

    Parameters
    ----------
    De : scalar
        The depth energy of the Morse potential in SI units (Joules)
    omega0 : scalar
        the fundamental vibrational frequency parameter in Hz; equal to (kappa_e/mu)^(1/2) where mu is the reduced mass.
    kappa_e : scalar
        the force constant in N/m; the curvature at the bottom of the potential well.
    charge : scalar
        the charge on the positively-charged atom in the dipole, in Coulombs
    field : scalar
        the applied electric field, in Volts/meter
    n_basis : scalar, int
        the number of rows/columns in the H matrix to be computed

    Returns
    -------
    H : array-like (nbasis, nbasis)
         Matrix elements H_mn = <m | H(F=0) + charge*field(u-r_e) | n> for the Morse Oscillator in units of Joules
    """

    V = compute_V(De,omega0,kappa_e,n_basis)
    E0 = compute_E0(De,omega0,n_basis)

    #Initialize H to zeros:
    H = np.zeros((n_basis,n_basis))

    #Fill diagonal of H with the energies at zero field and add appropriated scaled multiple of V.
    np.fill_diagonal(H,E0)

    H -= charge*field*V                      #Use negatively signed field.

    return H

def n_boundst(De,omega0):
    """The last bound state; n_basis should be no greater than this number plus one"""
    return int(2*De/(constants.hbar*omega0) - 0.5)

def egs_pol_pt(De,omega0,kappa_e,charge,field,n_basis):
    """Compute the polarization energy of the ground state using 1st- and 2nd-order perturbation theory"""

    V = compute_V(De,omega0,kappa_e,n_basis)
    E0 = compute_E0(De,omega0,n_basis)

    #At first order, the derivative is just the expectation value V_mm, scaled by q
    egs_pol_der1 = charge*V[0,0]

    #At second order, one needs to sum over the bound states
    egs_pol_der2 = 0.0
    for m in range(1,n_basis):
        egs_pol_der2 += 2*(charge*V[m,0])**2/(E0[0]-E0[m])

    return egs_pol_der1*(-1*field) + 0.5*egs_pol_der2*field**2

def egs_pol_basis(De,omega0,kappa_e,charge,field,n_basis):
    """Computes the polarization energy for the ground state of Morse Oscillator using basis-set expansion """
    H = compute_H(De,omega0,kappa_e,charge,field,n_basis)

    #compute energies by diagonalization
    energies = eigh(H, None, eigvals_only=True)

    #ground-state energy of polarization requires subtracting off the harmonic-oscillator term:
    return (energies[0] - 0.5* constants.hbar * omega0)

#Now we can compute what we need:
charge = 1.91*3.33564e-30/91.68e-12          #copied from the previous problem
mu = 6.361e-25                               #reduced mass in kg
kappa_e = 920                                #N/m given in this problem.
omega0 = np.sqrt(kappa_e/mu)                 #fundamental frequency from harmonic oscillator
De = 9.402e-19                               #Dissociation energy = depth of Morse potential well; in Joules; given.
n_basis = min(n_boundst(De,omega0) + 1,10)   #computed using function above; restrict to ensure unbound states not used
field = 1.0e6                                #field in V/m, given in this problem
epol_pt = egs_pol_pt(De,omega0,kappa_e,charge,field,n_basis)
epol_basis = egs_pol_basis(De,omega0,kappa_e,charge,field,n_basis)
Ezeropt_harmosc = constants.hbar * omega0/2
Ezeropt_Morse = Ezeropt_harmosc - Ezeropt_harmosc**2/(4*De)
Epol_harmosc = - charge**2 * field**2 / (2*kappa_e)

print(f"The zero-point energy for the harmonic oscillator model is {Ezeropt_harmosc:.3e} J.")
print(f"The zero-point energy for the Morse oscillator model is {Ezeropt_Morse:.3e} J.")
print(f"The polarization energy for the harmonic oscillator model is {Epol_harmosc:.3e} J.")
print(f"Using 1st- and 2nd-order perturbation theory, the polarization energy for HF in a field of 1e6 V/m is {epol_pt:.3e} J.")
print(f"Using basis-set expansion, the polarization energy for HF in a field of 1e6 V/m is {epol_basis:.3e} J.")
```

    The zero-point energy for the harmonic oscillator model is 2.005e-21 J.
    The zero-point energy for the Morse oscillator model is 2.004e-21 J.
    The polarization energy for the harmonic oscillator model is -2.625e-30 J.
    Using 1st- and 2nd-order perturbation theory, the polarization energy for HF in a field of 1e6 V/m is -5.032e-27 J.
    Using basis-set expansion, the polarization energy for HF in a field of 1e6 V/m is -1.074e-24 J.

## 📚 References
My favorite sources for this material are:
- [Randy's book](https://github.com/PaulWAyers/IntroQChem/blob/main/documents/DumontBook.pdf?raw=true) (See Chapter 3)
- [McQuarrie and Simon summary](https://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Map%3A_Physical_Chemistry_(McQuarrie_and_Simon)/05%3A_The_Harmonic_Oscillator_and_the_Rigid_Rotor)
- [Rogelio's notes](https://github.com/PaulWAyers/IntroQChem/blob/main/documents/HarmonicOscillator.pdf?raw=true)
- [Python Notebook for Morse Oscillator](https://scipython.com/blog/the-morse-oscillator/)
- [Vasan, V.S., Cross, R.J., 1983. Matrix elements for Morse oscillators. The Journal of Chemical Physics 78, 3869–3871.](https://aip.scitation.org/doi/10.1063/1.445164) Matrix elements for the Morse Oscillator in a field.
- [Emanuel F de Lima and José E M Hornos 2005 J. Phys. B: At. Mol. Opt. Phys. **38** 815](https://iopscience.iop.org/article/10.1088/0953-4075/38/7/004/meta) Matrix elements for the Morse Oscillator in a field including the continuum.

There are also some excellent wikipedia articles:
- [Harmonic Oscillator](https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator)
- [Morse Oscillator](https://en.wikipedia.org/wiki/Morse_potential)
- [Other exactly solvable models](https://en.wikipedia.org/wiki/List_of_quantum-mechanical_systems_with_analytical_solutions)


# **Least squares regression**

    Notebook version: 1.3 (Sep 14, 2017)

    Author: Jerónimo Arenas García (jarenas@tsc.uc3m.es)

    Changes: v.1.0 - First version
             v.1.1 - UTAD version
             v.1.2 - Minor corrections
             v.1.3 - Python 3 compatibility

    Pending changes: *

```python
# Import some libraries that will be necessary for working with data and displaying plots

# To visualize plots in the notebook
%matplotlib inline

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import scipy.io       # To read matlab files
import pylab

# For the student tests (only for python 2)
import sys
if sys.version_info.major==2:
    from test_helper import Test
```

This notebook covers the problem of fitting parametric regression models with a minimum least-squares criterion. The material presented here is based on the first lectures of this Machine Learning course. In particular, you can refer to the following presentation:  Probabilistic Regression.

## 1. A parametric approach to the regression problem

We have already presented the goal of regression. Given that we have access to a set of training points, $\{{\bf x}^{(l)}, s^{(l)}\}_{l=0}^{L-1}$, the goal is to learn a function $f({\bf x})$ that we can use to make good predictions for an arbitrary input vector.

The following plot illustrates a regression example for unidimensional input data. We have also generated three different regression curves corresponding to polynomia of degrees 1, 2, and 3 with random coefficients.

```python
n_points = 35
n_grid = 200
frec = 3
std_n = 0.3

# Location of the training points
X_tr = (3 * np.random.random((n_points,1)) - 0.5)

# Labels are obtained from a sinusoidal function, and contaminated by noise
S_tr = np.cos(frec*X_tr) + std_n * np.random.randn(n_points,1)

# Equally spaced points in the X-axis
X_grid = np.linspace(np.min(X_tr),np.max(X_tr),n_grid)

# Gererate random prediction curves
f1 = np.random.random() + np.random.random()*X_grid
f2 = np.random.random() + np.random.random()*X_grid + \
     np.random.random()*(X_grid**2)
f3 = np.random.random() + np.random.random()*X_grid + \
     np.random.random()*(X_grid**2) + np.random.random()*(X_grid**3)

plt.plot(X_tr,S_tr,'b.')
plt.plot(X_grid,f1.T,'g-',label='Arbitrary Linear function')
plt.plot(X_grid,f2.T,'r-',label='Arbitrary Quadratic function')
plt.plot(X_grid,f3.T,'m-',label='Arbitrary Cubic function')

plt.legend(loc='best')
plt.show()
```

### 1.1. Parametric model

Parametric regression models assume a parametric expression for the regression curve, adjusting the free parameters according to some criterion that measures the quality of the proposed model.

   - For a unidimensional case like the one in the previous figure, a convenient approach is to recur to polynomial expressions:

$${\hat s}(x) = f(x) = w_0 + w_1 x + w_2 x^2 + \dots + w_M x^M$$

   - For multidimensional regression, polynomial expressions can include cross-products of the variables. For instance, for a case with two input variables, the degree 2 polynomial would be given by

$${\hat s}({\bf x}) = f({\bf x}) = w_0 + w_1 x_1 + w_2 x_2 + w_3 x_1^2 + w_4 x_2^2 + w_5 x_1 x_2$$

   - A linear model for multidimensional regression can be expressed as

$${\hat s}({\bf x}) = f({\bf x}) = w_0 + {\bf w}^\top {\bf x}$$

When we postulate such models, the regression model is reduced to finding the most appropriate values of the parameters ${\bf w} = [w_i]$.

All the previous models have in common the fact that they are linear in the parameters, even though they can implement highly non-linear functions. All the derivations in this notebook are equally valid for other non-linear transformations of the input variables, as long as we keep linear-in-the-parameters models.

```python
## Next, we represent some random polynomial functions for degrees between 0 and 14

max_degree = 15
n_points = 200

#Values of X to evaluate the function
X_grid = np.linspace(-1.5, 1.5,n_points)

for idx in range(max_degree):
    x1 = plt.subplot(3,5, idx+1)
    x1.get_xaxis().set_ticks([])
    x1.get_yaxis().set_ticks([])

    for kk in range(5):
        #Random generation of coefficients for the model
        we = np.random.randn(idx+1, 1)

        #Evaluate the polynomial with previous coefficients at X_grid values
        fout = np.polyval(we, X_grid)
        x1.plot(X_grid,fout,'g-')
        x1.set_ylim([-5,5])
```

   - Should we choose a polynomial?

   - What degree should we use for the polynomial?

   - For a given degree, how do we choose the weights?

For now, we will find the single "best" polynomial. In a future session, we will see how we can design methods that take into account different polynomia simultaneously.

Next, we will explain how to choose optimal weights according to Least-Squares criterion.

## 2. Least squares regression

### 2.1. Problem definition

   - The goal is to learn a (possibly non-linear) regression model from a set of $L$ labeled points, $\{{\bf x}^{(l)},s{(l)}\}_{l=0}^{L-1}$.

   - We assume a parametric function of the form:

   $${\hat s}({\bf x}) = f({\bf x}) = w_0 z_0({\bf x}) + w_1 z_1({\bf x}) + \dots w_M z_M({\bf x})$$

   where $z_i({\bf x})$ are particular transformations of the input vector variables.

Some examples are:

   - If ${\bf z} = {\bf x}$, the model is just a linear combination of the input variables

   - If ${\bf z} = \left[\begin{array}{c}1\\{\bf x}\end{array}\right]$, we have again a linear combination with the inclusion of a constant term.

   - For unidimensional input $x$, ${\bf z} = [1, x, x^2, \dots,x^{M}]^\top$ would implement a polynomia of degree $M$.

   - Note that the variables of ${\bf z}$ could also be computed combining different variables of ${\bf x}$. E.g., if ${\bf x} = [x_1,x_2]^\top$, a degree-two polynomia would be implemented with
      $${\bf z} = \left[\begin{array}{c}1\\x_1\\x_2\\x_1^2\\x_2^2\\x_1 x_2\end{array}\right]$$

   - The above expression does not assume a polynomial model. For instance, we could consider ${\bf z} = [\log(x_1),\log(x_2)]$

Least squares (LS) regression finds the coefficients of the model with the aim of minimizing the square of the residuals. If we define ${\bf w} = [w_0,w_1,\dots,w_M]^\top$, the LS solution would be defined as

\begin{equation}{\bf w}_{LS} = \arg \min_{\bf w} \sum_{l=1}^L [e^{(l)}]^2 = \arg \min_{\bf w} \sum_{l=1}^L \left[s^{(l)} - {\hat s}^{(l)} \right]^2 \end{equation}

### 2.2. Vector Notation

In order to solve the LS problem it is convenient to define the following vectors and matrices:

   - We can group together all available target values to form the following vector

   $${\bf s} = \left[s^{(1)}, s^{(2)}, \dots, s^{(L)} \right]^\top$$

   - The estimation of the model for a single input vector ${\bf z}^{(l)}$ (which would be computed from ${\bf x}^{(l)}$), can be expressed as the following inner product

   $${\hat s}^{(l)} = {{\bf z}^{(l)}}^\top {\bf w}$$

   - If we now group all input vectors into a matrix ${\bf Z}$, so that each row of ${\bf Z}$ contains the transpose of the corresponding ${\bf z}^{(l)}$, we can express

   $$\hat{{\bf s}} = \left[{\hat s}^{1}, {\hat s}^{2}, \dots, {\hat s}^{(L)} \right]^\top = {\bf Z} {\bf w}, \;\;\;\; \text{with} \;\; {\bf Z} = \left[\begin{array}{cccc}z_0^{(1)}&z_1^{(1)}&\cdots&z_M^{(1)} \\ z_0^{(2)}&z_1^{(2)}&\cdots&z_M^{(2)} \\ \vdots & \vdots & \ddots & \vdots \\ z_0^{(L)}&z_1^{(L)}&\cdots&z_M^{(L)}\end{array}\right]$$

### 2.3. Least-squares solution

   - Using the previous notation, the cost minimized by the LS model can be expressed as

   $$C({\bf w}) = \sum_{l=1}^L \left[s^{(l)} - {\hat s}^{(l)} \right]^2 = \|{\bf s} - {\hat{\bf s}}\|^2 = \|{\bf s} - {\bf Z}{\bf w}\|^2$$

   - Since the above expression depends quadratically on ${\bf w}$ and is non-negative, we know that there is only one point where the derivative of $C({\bf w})$ becomes zero, and that point is necessarily a minimum of the cost

   $$\nabla_{\bf w} \|{\bf s} - {\bf Z}{\bf w}\|^2\Bigg|_{{\bf w} = {\bf w}_{LS}} = {\bf 0}$$

Exercise:
Solve the previous problem to show that
$${\bf w}_{LS} = \left( {\bf Z}^\top{\bf Z} \right)^{-1} {\bf Z}^\top{\bf s}$$

The next fragment of code adjusts polynomia of increasing order to randomly generated training data. To illustrate the composition of matrix ${\bf Z}$, we will avoid using functions $\mbox{np.polyfit}$ and $\mbox{np.polyval}$.

```python
n_points = 20
n_grid = 200
frec = 3
std_n = 0.2
max_degree = 20

colors = 'brgcmyk'

#Location of the training points
X_tr = (3 * np.random.random((n_points,1)) - 0.5)

#Labels are obtained from a sinusoidal function, and contaminated by noise
S_tr = np.cos(frec*X_tr) + std_n * np.random.randn(n_points,1)

#Equally spaced points in the X-axis
X_grid = np.linspace(np.min(X_tr),np.max(X_tr),n_grid)

#We start by building the Z matrix
Z = []
for el in X_tr.tolist():
    Z.append([el[0]**k for k in range(max_degree+1)])
Z = np.matrix(Z)

Z_grid = []
for el in X_grid.tolist():
    Z_grid.append([el**k for k in range(max_degree+1)])
Z_grid = np.matrix(Z_grid)

plt.plot(X_tr,S_tr,'b.')

for k in [1, 2, n_points]: # range(max_degree+1):
    Z_iter = Z[:,:k+1]

    # Least square solution
    #w_LS = (np.linalg.inv(Z_iter.T.dot(Z_iter))).dot(Z_iter.T).dot(S_tr)

    # Least squares solution, with leass numerical errors
    w_LS, resid, rank, s = np.linalg.lstsq(Z_iter, S_tr, rcond=None)
    #estimates at all grid points
    fout = Z_grid[:,:k+1].dot(w_LS)
    fout = np.array(fout).flatten()
    plt.plot(X_grid,fout,colors[k%len(colors)]+'-',label='Degree '+str(k))

plt.legend(loc='best')
plt.ylim(1.2*np.min(S_tr), 1.2*np.max(S_tr))
plt.show()
```

### 2.4. Overfitting the training data

It may seem that increasing the degree of the polynomia is always beneficial, as we can implement a more expressive function. A polynomia of degree $M$ would include all polynomia of lower degrees as particular cases. However, if we increase the number of parameters without control, the polynomia would eventually get expressive enough to adjust any given set of training points to arbitrary precision, what does not necessarily mean that the solution is obtaining a model that can be extrapolated to new data, as we show in the following example:

```python
n_points = 35
n_test = 200
n_grid = 200
frec = 3
std_n = 0.7
max_degree = 25

colors = 'brgcmyk'

#Location of the training points
X_tr = (3 * np.random.random((n_points,1)) - 0.5)
#Labels are obtained from a sinusoidal function, and contaminated by noise
S_tr = np.cos(frec*X_tr) + std_n * np.random.randn(n_points,1)
#Test points to validate the generalization of the solution
X_tst = (3 * np.random.random((n_test,1)) - 0.5)
S_tst = np.cos(frec*X_tst) + std_n * np.random.randn(n_test,1)

#Equally spaced points in the X-axis
X_grid = np.linspace(np.min(X_tr),np.max(X_tr),n_grid)

#We start by building the Z matrix
def extend_matrix(X,max_degree):
    Z = []
    X = X.reshape((X.shape[0],1))
    for el in X.tolist():
        Z.append([el[0]**k for k in range(max_degree+1)])
    return np.matrix(Z)

Z = extend_matrix(X_tr,max_degree)
Z_grid = extend_matrix(X_grid,max_degree)
Z_test = extend_matrix(X_tst,max_degree)
#Variables to store the train and test errors
tr_error = []
tst_error = []

for k in range(max_degree):
    Z_iter = Z[:,:k+1]
    #Least square solution
    #w_LS = (np.linalg.inv(Z_iter.T.dot(Z_iter))).dot(Z_iter.T).dot(S_tr)

    # Least squares solution, with leass numerical errors
    w_LS, resid, rank, s = np.linalg.lstsq(Z_iter, S_tr)

    #estimates at traint and test points
    f_tr = Z_iter.dot(w_LS)
    f_tst = Z_test[:,:k+1].dot(w_LS)
    tr_error.append(np.array((S_tr-f_tr).T.dot(S_tr-f_tr)/len(S_tr))[0,0])
    tst_error.append(np.array((S_tst-f_tst).T.dot(S_tst-f_tst)/len(S_tst))[0,0])

plt.stem(range(max_degree),tr_error,'b-',label='Train error')
plt.stem(range(max_degree),tst_error,'r-o',label='Test error')
plt.legend(loc='best')
plt.show()
```

#### 2.4.1 Limitations of the LS approach. The need for assumptions

Another way to visualize the effect of overfiting is to analyze the effect of variations o a single sample. Consider a training dataset consisting of 15 points which are given, and depict the regression curves that would be obtained if adding an additional point at a fixed location, depending on the target value of that point:

(You can run this code fragment several times, to check also the changes in the regression curves between executions, and depending also on the location of the training points)

```python
n_points = 15
n_grid = 200
frec = 3
std_n = 0.2
n_val_16 = 5
degree = 18

X_tr = 3 * np.random.random((n_points,1)) - 0.5
S_tr = - np.cos(frec*X_tr) + std_n * np.random.randn(n_points,1)
X_grid = np.linspace(-.5,2.5,n_grid)
S_grid = - np.cos(frec*X_grid) #Noise free for the true model

X_16 = .3 * np.ones((n_val_16,))
S_16 = np.linspace(np.min(S_tr),np.max(S_tr),n_val_16)

fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(X_tr,S_tr,'b.',markersize=10)
ax.plot(X_16,S_16,'ro',markersize=6)
ax.plot(X_grid,S_grid,'r-',label='True model')

for el in zip(X_16,S_16):
    #Add point to the training set
    X_tr_iter = np.append(X_tr,el[0])
    S_tr_iter = np.append(S_tr,el[1])

    #Obtain LS regression coefficients and evaluate it at X_grid
    w_LS = np.polyfit(X_tr_iter, S_tr_iter, degree)
    S_grid_iter = np.polyval(w_LS,X_grid)
    ax.plot(X_grid,S_grid_iter,'g-')

ax.set_xlim(-.5,2.5)
ax.set_ylim(S_16[0]-2,S_16[-1]+2)
ax.legend(loc='best')
plt.show()
```

### Exercise

Analyze the performance of LS regression on the `Advertising` dataset. You can analyze:

  - The performance of linear regression when using just one variable, or using all of them together
  - The performance of different non-linear methods (e.g., polynomial or logarithmic transformations)
  - Model selection using CV strategies


# 低次元化

```python
import sympy as sy
from sympy.printing.numpy import NumPyPrinter
from sympy import julia_code
from sympy.utilities.codegen import codegen
import tqdm
import os
from pathlib import Path

from kinematics import Local
```

```python
# アクチュエータベクトル
l1, l2, l3 = sy.symbols("l1, l2, l3")
q = sy.Matrix([[l1, l2, l3]]).T

# ディスク位置を表すスカラ変数
xi = sy.Symbol("xi")
```

```python
kinema = Local()

P = kinema.P(q, xi)  # 位置ベクトル
R = kinema.R(q, xi)  # 回転行列
```

# 2次まで減らす

```python
P[2,0]
```

$\displaystyle 60131984.3684659 \xi^{9} \left(l_{1} + l_{2} + l_{3} + 0.45\right) \left(l_{1}^{2} - l_{1} l_{2} - l_{1} l_{3} + l_{2}^{2} - l_{2} l_{3} + l_{3}^{2}\right)^{4} - 1522090.85432679 \xi^{7} \left(l_{1} + l_{2} + l_{3} + 0.45\right) \left(l_{1}^{2} - l_{1} l_{2} - l_{1} l_{3} + l_{2}^{2} - l_{2} l_{3} + l_{3}^{2}\right)^{3} + 22474.6227709191 \xi^{5} \left(l_{1} + l_{2} + l_{3} + 0.45\right) \left(l_{1}^{2} - l_{1} l_{2} - l_{1} l_{3} + l_{2}^{2} - l_{2} l_{3} + l_{3}^{2}\right)^{2} - 79.0123456790123 \xi^{3} \left(l_{1} + l_{2} + l_{3} + 0.45\right) \left(2 l_{1}^{2} - 2 l_{1} l_{2} - 2 l_{1} l_{3} + 2 l_{2}^{2} - 2 l_{2} l_{3} + 2 l_{3}^{2}\right) + \frac{\xi \left(l_{1} + l_{2} + l_{3} + 0.45\right)}{3}$

```python
f = P[2,0].expand()
f
```

$\displaystyle 60131984.3684659 l_{1}^{9} \xi^{9} - 180395953.105398 l_{1}^{8} l_{2} \xi^{9} - 180395953.105398 l_{1}^{8} l_{3} \xi^{9} + 27059392.9658097 l_{1}^{8} \xi^{9} + 360791906.210795 l_{1}^{7} l_{2}^{2} \xi^{9} - 108237571.863239 l_{1}^{7} l_{2} \xi^{9} + 360791906.210795 l_{1}^{7} l_{3}^{2} \xi^{9} - 108237571.863239 l_{1}^{7} l_{3} \xi^{9} - 1522090.85432679 l_{1}^{7} \xi^{7} - 360791906.210795 l_{1}^{6} l_{2}^{3} \xi^{9} + 360791906.210795 l_{1}^{6} l_{2}^{2} l_{3} \xi^{9} + 270593929.658097 l_{1}^{6} l_{2}^{2} \xi^{9} + 360791906.210795 l_{1}^{6} l_{2} l_{3}^{2} \xi^{9} + 216475143.726477 l_{1}^{6} l_{2} l_{3} \xi^{9} + 3044181.70865359 l_{1}^{6} l_{2} \xi^{7} - 360791906.210795 l_{1}^{6} l_{3}^{3} \xi^{9} + 270593929.658097 l_{1}^{6} l_{3}^{2} \xi^{9} + 3044181.70865359 l_{1}^{6} l_{3} \xi^{7} - 684940.884447057 l_{1}^{6} \xi^{7} + 180395953.105398 l_{1}^{5} l_{2}^{4} \xi^{9} - 1443167624.84318 l_{1}^{5} l_{2}^{3} l_{3} \xi^{9} - 432950287.452954 l_{1}^{5} l_{2}^{3} \xi^{9} - 324712715.589716 l_{1}^{5} l_{2}^{2} l_{3} \xi^{9} - 4566272.56298038 l_{1}^{5} l_{2}^{2} \xi^{7} - 1443167624.84318 l_{1}^{5} l_{2} l_{3}^{3} \xi^{9} - 324712715.589716 l_{1}^{5} l_{2} l_{3}^{2} \xi^{9} + 4566272.56298038 l_{1}^{5} l_{2} l_{3} \xi^{7} + 2054822.65334117 l_{1}^{5} l_{2} \xi^{7} + 180395953.105398 l_{1}^{5} l_{3}^{4} \xi^{9} - 432950287.452954 l_{1}^{5} l_{3}^{3} \xi^{9} - 4566272.56298038 l_{1}^{5} l_{3}^{2} \xi^{7} + 2054822.65334117 l_{1}^{5} l_{3} \xi^{7} + 22474.6227709191 l_{1}^{5} \xi^{5} + 180395953.105398 l_{1}^{4} l_{2}^{5} \xi^{9} + 1623563577.94858 l_{1}^{4} l_{2}^{4} l_{3} \xi^{9} + 514128466.350383 l_{1}^{4} l_{2}^{4} \xi^{9} + 721583812.421591 l_{1}^{4} l_{2}^{3} l_{3}^{2} \xi^{9} + 108237571.863239 l_{1}^{4} l_{2}^{3} l_{3} \xi^{9} + 1522090.85432679 l_{1}^{4} l_{2}^{3} \xi^{7} + 721583812.421591 l_{1}^{4} l_{2}^{2} l_{3}^{3} \xi^{9} + 649425431.179432 l_{1}^{4} l_{2}^{2} l_{3}^{2} \xi^{9} - 9132545.12596076 l_{1}^{4} l_{2}^{2} l_{3} \xi^{7} - 4109645.30668234 l_{1}^{4} l_{2}^{2} \xi^{7} + 1623563577.94858 l_{1}^{4} l_{2} l_{3}^{4} \xi^{9} + 108237571.863239 l_{1}^{4} l_{2} l_{3}^{3} \xi^{9} - 9132545.12596076 l_{1}^{4} l_{2} l_{3}^{2} \xi^{7} - 2054822.65334117 l_{1}^{4} l_{2} l_{3} \xi^{7} - 22474.6227709191 l_{1}^{4} l_{2} \xi^{5} + 180395953.105398 l_{1}^{4} l_{3}^{5} \xi^{9} + 514128466.350383 l_{1}^{4} l_{3}^{4} \xi^{9} + 1522090.85432679 l_{1}^{4} l_{3}^{3} \xi^{7} - 4109645.30668234 l_{1}^{4} l_{3}^{2} \xi^{7} - 22474.6227709191 l_{1}^{4} l_{3} \xi^{5} + 10113.5802469136 l_{1}^{4} \xi^{5} - 360791906.210795 l_{1}^{3} l_{2}^{6} \xi^{9} - 1443167624.84318 l_{1}^{3} l_{2}^{5} l_{3} \xi^{9} - 432950287.452954 l_{1}^{3} l_{2}^{5} \xi^{9} + 721583812.421591 l_{1}^{3} l_{2}^{4} l_{3}^{2} \xi^{9} + 108237571.863239 l_{1}^{3} l_{2}^{4} l_{3} \xi^{9} + 1522090.85432679 l_{1}^{3} l_{2}^{4} \xi^{7} - 2886335249.68636 l_{1}^{3} l_{2}^{3} l_{3}^{3} \xi^{9} - 432950287.452954 l_{1}^{3} l_{2}^{3} l_{3}^{2} \xi^{9} + 19787181.1062483 l_{1}^{3} l_{2}^{3} l_{3} \xi^{7} + 4794586.1911294 l_{1}^{3} l_{2}^{3} \xi^{7} + 721583812.421591 l_{1}^{3} l_{2}^{2} l_{3}^{4} \xi^{9} - 432950287.452954 l_{1}^{3} l_{2}^{2} l_{3}^{3} \xi^{9} - 4566272.56298038 l_{1}^{3} l_{2}^{2} l_{3}^{2} \xi^{7} + 2054822.65334117 l_{1}^{3} l_{2}^{2} l_{3} \xi^{7} + 22474.6227709191 l_{1}^{3} l_{2}^{2} \xi^{5} - 1443167624.84318 l_{1}^{3} l_{2} l_{3}^{5} \xi^{9} + 108237571.863239 l_{1}^{3} l_{2} l_{3}^{4} \xi^{9} + 19787181.1062483 l_{1}^{3} l_{2} l_{3}^{3} \xi^{7} + 2054822.65334117 l_{1}^{3} l_{2} l_{3}^{2} \xi^{7} - 89898.4910836763 l_{1}^{3} l_{2} l_{3} \xi^{5} - 20227.1604938272 l_{1}^{3} l_{2} \xi^{5} - 360791906.210795 l_{1}^{3} l_{3}^{6} \xi^{9} - 432950287.452954 l_{1}^{3} l_{3}^{5} \xi^{9} + 1522090.85432679 l_{1}^{3} l_{3}^{4} \xi^{7} + 4794586.1911294 l_{1}^{3} l_{3}^{3} \xi^{7} + 22474.6227709191 l_{1}^{3} l_{3}^{2} \xi^{5} - 20227.1604938272 l_{1}^{3} l_{3} \xi^{5} - 158.024691358025 l_{1}^{3} \xi^{3} + 360791906.210795 l_{1}^{2} l_{2}^{7} \xi^{9} + 360791906.210795 l_{1}^{2} l_{2}^{6} l_{3} \xi^{9} + 270593929.658097 l_{1}^{2} l_{2}^{6} \xi^{9} - 324712715.589716 l_{1}^{2} l_{2}^{5} l_{3} \xi^{9} - 4566272.56298038 l_{1}^{2} l_{2}^{5} \xi^{7} + 721583812.421591 l_{1}^{2} l_{2}^{4} l_{3}^{3} \xi^{9} + 649425431.179432 l_{1}^{2} l_{2}^{4} l_{3}^{2} \xi^{9} - 9132545.12596076 l_{1}^{2} l_{2}^{4} l_{3} \xi^{7} - 4109645.30668234 l_{1}^{2} l_{2}^{4} \xi^{7} + 721583812.421591 l_{1}^{2} l_{2}^{3} l_{3}^{4} \xi^{9} - 432950287.452954 l_{1}^{2} l_{2}^{3} l_{3}^{3} \xi^{9} - 4566272.56298038 l_{1}^{2} l_{2}^{3} l_{3}^{2} \xi^{7} + 2054822.65334117 l_{1}^{2} l_{2}^{3} l_{3} \xi^{7} + 22474.6227709191 l_{1}^{2} l_{2}^{3} \xi^{5} + 649425431.179432 l_{1}^{2} l_{2}^{2} l_{3}^{4} \xi^{9} - 4566272.56298038 l_{1}^{2} l_{2}^{2} l_{3}^{3} \xi^{7} - 6164467.96002351 l_{1}^{2} l_{2}^{2} l_{3}^{2} \xi^{7} + 67423.8683127572 l_{1}^{2} l_{2}^{2} l_{3} \xi^{5} + 30340.7407407407 l_{1}^{2} l_{2}^{2} \xi^{5} + 360791906.210795 l_{1}^{2} l_{2} l_{3}^{6} \xi^{9} - 324712715.589716 l_{1}^{2} l_{2} l_{3}^{5} \xi^{9} - 9132545.12596076 l_{1}^{2} l_{2} l_{3}^{4} \xi^{7} + 2054822.65334117 l_{1}^{2} l_{2} l_{3}^{3} \xi^{7} + 67423.8683127572 l_{1}^{2} l_{2} l_{3}^{2} \xi^{5} + 360791906.210795 l_{1}^{2} l_{3}^{7} \xi^{9} + 270593929.658097 l_{1}^{2} l_{3}^{6} \xi^{9} - 4566272.56298038 l_{1}^{2} l_{3}^{5} \xi^{7} - 4109645.30668234 l_{1}^{2} l_{3}^{4} \xi^{7} + 22474.6227709191 l_{1}^{2} l_{3}^{3} \xi^{5} + 30340.7407407407 l_{1}^{2} l_{3}^{2} \xi^{5} - 71.1111111111111 l_{1}^{2} \xi^{3} - 180395953.105398 l_{1} l_{2}^{8} \xi^{9} - 108237571.863239 l_{1} l_{2}^{7} \xi^{9} + 360791906.210795 l_{1} l_{2}^{6} l_{3}^{2} \xi^{9} + 216475143.726477 l_{1} l_{2}^{6} l_{3} \xi^{9} + 3044181.70865359 l_{1} l_{2}^{6} \xi^{7} - 1443167624.84318 l_{1} l_{2}^{5} l_{3}^{3} \xi^{9} - 324712715.589716 l_{1} l_{2}^{5} l_{3}^{2} \xi^{9} + 4566272.56298038 l_{1} l_{2}^{5} l_{3} \xi^{7} + 2054822.65334117 l_{1} l_{2}^{5} \xi^{7} + 1623563577.94858 l_{1} l_{2}^{4} l_{3}^{4} \xi^{9} + 108237571.863239 l_{1} l_{2}^{4} l_{3}^{3} \xi^{9} - 9132545.12596076 l_{1} l_{2}^{4} l_{3}^{2} \xi^{7} - 2054822.65334117 l_{1} l_{2}^{4} l_{3} \xi^{7} - 22474.6227709191 l_{1} l_{2}^{4} \xi^{5} - 1443167624.84318 l_{1} l_{2}^{3} l_{3}^{5} \xi^{9} + 108237571.863239 l_{1} l_{2}^{3} l_{3}^{4} \xi^{9} + 19787181.1062483 l_{1} l_{2}^{3} l_{3}^{3} \xi^{7} + 2054822.65334117 l_{1} l_{2}^{3} l_{3}^{2} \xi^{7} - 89898.4910836763 l_{1} l_{2}^{3} l_{3} \xi^{5} - 20227.1604938272 l_{1} l_{2}^{3} \xi^{5} + 360791906.210795 l_{1} l_{2}^{2} l_{3}^{6} \xi^{9} - 324712715.589716 l_{1} l_{2}^{2} l_{3}^{5} \xi^{9} - 9132545.12596076 l_{1} l_{2}^{2} l_{3}^{4} \xi^{7} + 2054822.65334117 l_{1} l_{2}^{2} l_{3}^{3} \xi^{7} + 67423.8683127572 l_{1} l_{2}^{2} l_{3}^{2} \xi^{5} + 216475143.726477 l_{1} l_{2} l_{3}^{6} \xi^{9} + 4566272.56298038 l_{1} l_{2} l_{3}^{5} \xi^{7} - 2054822.65334117 l_{1} l_{2} l_{3}^{4} \xi^{7} - 89898.4910836763 l_{1} l_{2} l_{3}^{3} \xi^{5} + 474.074074074074 l_{1} l_{2} l_{3} \xi^{3} + 71.1111111111111 l_{1} l_{2} \xi^{3} - 180395953.105398 l_{1} l_{3}^{8} \xi^{9} - 108237571.863239 l_{1} l_{3}^{7} \xi^{9} + 3044181.70865359 l_{1} l_{3}^{6} \xi^{7} + 2054822.65334117 l_{1} l_{3}^{5} \xi^{7} - 22474.6227709191 l_{1} l_{3}^{4} \xi^{5} - 20227.1604938272 l_{1} l_{3}^{3} \xi^{5} + 71.1111111111111 l_{1} l_{3} \xi^{3} + \frac{l_{1} \xi}{3} + 60131984.3684659 l_{2}^{9} \xi^{9} - 180395953.105398 l_{2}^{8} l_{3} \xi^{9} + 27059392.9658097 l_{2}^{8} \xi^{9} + 360791906.210795 l_{2}^{7} l_{3}^{2} \xi^{9} - 108237571.863239 l_{2}^{7} l_{3} \xi^{9} - 1522090.85432679 l_{2}^{7} \xi^{7} - 360791906.210795 l_{2}^{6} l_{3}^{3} \xi^{9} + 270593929.658097 l_{2}^{6} l_{3}^{2} \xi^{9} + 3044181.70865359 l_{2}^{6} l_{3} \xi^{7} - 684940.884447057 l_{2}^{6} \xi^{7} + 180395953.105398 l_{2}^{5} l_{3}^{4} \xi^{9} - 432950287.452954 l_{2}^{5} l_{3}^{3} \xi^{9} - 4566272.56298038 l_{2}^{5} l_{3}^{2} \xi^{7} + 2054822.65334117 l_{2}^{5} l_{3} \xi^{7} + 22474.6227709191 l_{2}^{5} \xi^{5} + 180395953.105398 l_{2}^{4} l_{3}^{5} \xi^{9} + 514128466.350383 l_{2}^{4} l_{3}^{4} \xi^{9} + 1522090.85432679 l_{2}^{4} l_{3}^{3} \xi^{7} - 4109645.30668234 l_{2}^{4} l_{3}^{2} \xi^{7} - 22474.6227709191 l_{2}^{4} l_{3} \xi^{5} + 10113.5802469136 l_{2}^{4} \xi^{5} - 360791906.210795 l_{2}^{3} l_{3}^{6} \xi^{9} - 432950287.452954 l_{2}^{3} l_{3}^{5} \xi^{9} + 1522090.85432679 l_{2}^{3} l_{3}^{4} \xi^{7} + 4794586.1911294 l_{2}^{3} l_{3}^{3} \xi^{7} + 22474.6227709191 l_{2}^{3} l_{3}^{2} \xi^{5} - 20227.1604938272 l_{2}^{3} l_{3} \xi^{5} - 158.024691358025 l_{2}^{3} \xi^{3} + 360791906.210795 l_{2}^{2} l_{3}^{7} \xi^{9} + 270593929.658097 l_{2}^{2} l_{3}^{6} \xi^{9} - 4566272.56298038 l_{2}^{2} l_{3}^{5} \xi^{7} - 4109645.30668234 l_{2}^{2} l_{3}^{4} \xi^{7} + 22474.6227709191 l_{2}^{2} l_{3}^{3} \xi^{5} + 30340.7407407407 l_{2}^{2} l_{3}^{2} \xi^{5} - 71.1111111111111 l_{2}^{2} \xi^{3} - 180395953.105398 l_{2} l_{3}^{8} \xi^{9} - 108237571.863239 l_{2} l_{3}^{7} \xi^{9} + 3044181.70865359 l_{2} l_{3}^{6} \xi^{7} + 2054822.65334117 l_{2} l_{3}^{5} \xi^{7} - 22474.6227709191 l_{2} l_{3}^{4} \xi^{5} - 20227.1604938272 l_{2} l_{3}^{3} \xi^{5} + 71.1111111111111 l_{2} l_{3} \xi^{3} + \frac{l_{2} \xi}{3} + 60131984.3684659 l_{3}^{9} \xi^{9} + 27059392.9658097 l_{3}^{8} \xi^{9} - 1522090.85432679 l_{3}^{7} \xi^{7} - 684940.884447057 l_{3}^{6} \xi^{7} + 22474.6227709191 l_{3}^{5} \xi^{5} + 10113.5802469136 l_{3}^{4} \xi^{5} - 158.024691358025 l_{3}^{3} \xi^{3} - 71.1111111111111 l_{3}^{2} \xi^{3} + \frac{l_{3} \xi}{3} + 0.15 \xi$

```python
f = f.coeff(l1, 1)
f = f.coeff(l2, 0)
f = f.coeff(l3, 0)
f
```

$\displaystyle \frac{\xi}{3}$