# Bundle adjustment

This notebook visualizes the optimization of 3D points and camera poses using bundle adjustment.

**Subjects covered**
1. **Definition of bundle adjustment.**
2. **The g2o optimization library.**
3. **Refining 3D point coordinates using bundle adjustment.**
4. **Refining camera poses using bundle adjustment.**
5. **Conclusion**
6. **Sources**

**Why are we interested in these subjects?**
Previous notebooks demonstrated algorithms for 3D point triangulation and camera pose optimization. These algorithms had **shortcomings**, as they were incapable of optimizing multiple points and poses simultaneously. The triangulation assumed perfect knowledge of poses. The PnP pose estimation assumed perfect knowledge of 3D points. Often we do not have perfect knowledge of either, we only have noisy data and initial guesses. Thus, we need an algorithm that can refine this noisy data to an optimal guess. Bundle adjustment achieves this.

**Further reading**
For a comprehensive treatment of bundle adjustment, see chapter eighteen of [Multiple view geometry in computer vision](https://www.robots.ox.ac.uk/~vgg/hzbook/) by Richard Hartley and Andrew Zisserman. For a less thorough but more merciful introduction, see chapter seven of [Computer vision: algorithms and applications](https://szeliski.org/Book/) by Richard Szeliski and the [lectures and shorts](https://www.youtube.com/watch?v=lmj2Jk5tl60&ab_channel=CyrillStachniss) by Cyrill Stachniss.

## Bundle adjustment definition

Bundle adjustment refers to the problem of refining 3D point coordinates and camera poses using noisy 2D images of those points and initial camera pose estimates. Optionally, the camera intrinsics can also be optimized. In other words, it is an algorithm that determines the 3D map of a scene and the cameras' parameters within that scene. The solution is a map and cameras that best explain the observations, namely the cameras' images.

Essentially, bundle adjustment is just an instance of a multi-parameter optimization problem. Similar to the problems of curve fitting and determining the weights of a neural network. As with those problems, bundle adjustment is concerned with minimizing an error metric. The error metric for bundle adjustment is the reprojection error.

The reprojection error measures the in-image distance between the pixel point in the provided image, and the pixel point from the projected 3D world point. The projected 3D world point is calculated using the estimated camera pose and estimated 3D world point coordinate.

Image from Pix4D article

This error can be expressed as a summation over all cameras and points

$$\begin{align}
E &= \sum_{\text{camera } \in \text{ cameras}} \hspace{0.2em} \sum_{<\text{3D point, 2D point}> \hspace{0.2em} \in \text{ points}} \text{distance}(\text{ 2D point}, \text{ project}(\text{ 3D point }, \text{ camera})) \\\\
E &= \sum_{c \hspace{0.1 em} \in \hspace{0.1 em} C} \hspace{0.2em} \sum_{p, \hspace{0.1 em} p' \hspace{0.1 em} \in \hspace{0.1 em} P} d(p', Q(p, c)))
\end{align}
$$

For a large scene, not all points will be observed by all cameras. If a point is not observed, there is no reprojection error to minimize. Thus, it is useful to add a binary variable $v_{c, \hspace{0.05 em} p}$ to ignore these non-interacting camera-point pairs.

$$ E = \sum_{c \hspace{0.1 em} \in \hspace{0.1 em} C} \hspace{0.2em} \sum_{p, \hspace{0.1 em} p' \hspace{0.1 em} \in \hspace{0.1 em} P} v_{c,\hspace{0.05 em} p}\hspace{0.2 em} d(p', Q(p, c))) $$

#### Multi-parameter optimization
Bundle adjustment is a multi-parameter optimization problem. There are many multi-parameter optimization schemes one can choose from, such as:

* iterative gradient descent
* iterative least-squares (Gauss-Newton)
* iterative gradient descent + least-squares (Levenberg-Marquardt)
* interior-point methods

The goal is to minimize the error by refining the camera poses and point coordinates. Assume we encode camera pose by a unit quaternion $q$ plus a translation vector $c$. Points are encoded as a 3-vector $p$. All these optimization algorithms would calculate the derivatives of $E$ w.r.t. the parameters: $\frac{\partial E}{\partial q_{a}}$, $\frac{\partial E}{\partial q_{b}}$, $\frac{\partial E}{\partial q_{c}}$, $\frac{\partial E}{\partial q_{d}}$, $\frac{\partial E}{\partial c_{x}}$, $\frac{\partial E}{\partial c_{y}}$, $\frac{\partial E}{\partial c_{z}}$, $\frac{\partial E}{\partial t_{x}}$, $\frac{\partial E}{\partial t_{y}}$, $\frac{\partial E}{\partial t_{z}}$. These derivatives would then inform an update rule that iteratively brings the parameters closer to a local optimum.

**Levenberg-Marquardt** is often chosen in the bundle adjustment literature (see implementation and notes in notebook 2). This is because:
* The dampening factor allows balancing between robust gradient descent and fast least-sqaures updates
* There are many sparse implementations available for problems with sparse Jacobians

Why would a sparse implementation be important? Well, applying Levenberg-Marquardt directly to large scenes with many points and cameras would be extremely computationally expensive. Take for example the derivative of the reprojection error w.r.t. a 3D point. Assume this derivative has to be calculated for $100$ thousand points. Now assume that there are $100$ cameras. That means $30$ million derivatives that need to be calculated at each optimization step. Most of these derivatives will be zero, as not all points will be observed by all cameras. Operating on these zeros and keeping them in memory is wasteful. The solution is a sparse implementation. The design of such a sparse algorithm is out of this notebook's scope. For a good summary of such a method, see the paper [SBA: A Software Package for Generic Sparse Bundle Adjustment](http://users.ics.forth.gr/~lourakis/sba/sba-toms.pdf) by Manolis Lourakis and Antonis Argyros with code available [here](https://github.com/balintfodor/sba).

## g2o optimization library

The [g2o optimization library](https://github.com/RainerKuemmerle/g2o) is a framework for optimizing graph-based nonlinear error functions. Here, a graph is a general way to specify a system of coupled variables that produce the error. Formally, each vertex of the graph represents a state variable to optimize, each edge between two variables represents a pairwise
observation of the two vertices it connects. For our bundle adjustment problem, the state variables are world points and camera poses. The edge that connects a point and a camera is the 2D observation of the point by the camera.

The g2o library defines types for common types of state variables. They a listed here to avoid inline code comments. For cameras, there is the `VertexSCam`, where S stands for 'sparse'. For points, there is `VertexSBAPointXYZ`, which stands for 'Vertex Sparse Bundle Adjustment Point XYZ'. `VertexSCam` defines the camera pose as an `Isometry3d` type. An isometry is a distance-preserving transformation between metric spaces, i.e. a change of coordinate system. This makes sense, as we treat camera poses as coordinate systems in notebook 1. The edge connecting cameras and points is of type `Edge_XYZ_VSC`, where VSC stands for 'Vertex Sparse Camera'. Any mention of `SE3` refers to the mathematical group of rigid body motions, e.g. camera poses.

We will use [g2opy](https://github.com/uoip/g2opy), a python binding of g2o. For more details on g2o, see the paper [g2o: A General Framework for Graph Optimization](http://ais.informatik.uni-freiburg.de/publications/papers/kuemmerle11icra.pdf). The code documentation is somewhat lacking. The best way to understand its structure is this [system description](https://github.com/uoip/g2opy/blob/master/doc/g2o.pdf) and reading the C++ [source code](https://github.com/RainerKuemmerle/g2o).

## Imports

```python
from notebook_functions import (camera_centers,
                                extrinsics,
                                images,
                                init_3d_plot,
                                get_bunny,
                                project_points_to_picture,
                                intrinsics,
                                plot_camera_wireframe,
                                plot_picture,
                                update_camera_wireframe,
                                update_picture,
                                distort_extrinsics,
                                camera_centers_from_extrinsics)
import ipyvolume as ipv
import numpy as np
import time
import g2o
```

## Set up a scene and distort bunny
In this example, we will have perfect knowledge of the camera poses, perfect knowledge of observations, and extremely noisy estimates of 3D points. The goal is to recover the shape of the bunny through optimization.

```python
vis_scale = 5
init_3d_plot()
ipv.zlim(-35, 25)
bunny_coords = get_bunny()
b_xs, b_ys, b_zs = bunny_coords[:3]
bunny_coords = np.vstack([bunny_coords, np.ones((1, bunny_coords.shape[1]))])
distort_bunny_coords = bunny_coords + np.random.random(bunny_coords.shape) * 5
b_xs, b_ys, b_zs, _ = distort_bunny_coords
scatter_plot = ipv.scatter(b_xs, b_ys, b_zs, size=0.5, marker="sphere", color='lime')
for cam_center, extrinsic, image in zip(camera_centers, extrinsics, images):
    image = np.zeros_like(image)
    image = project_points_to_picture(image, bunny_coords, intrinsics, extrinsic)
    inv_extrinsic = np.linalg.pinv(extrinsic)
    plot_camera_wireframe(cam_center, vis_scale, inv_extrinsic)
    plot_picture(image, inv_extrinsic, vis_scale)

ipv.show()
```

## Define cameras, points, and observations as a g2o graph
Here, we assume all points are visible in all cameras. As can be seen above, this is the case for this simple scene.

```python
# Sets optimizer
optimizer = g2o.SparseOptimizer()
solver = g2o.BlockSolverSE3(g2o.LinearSolverCSparseSE3())
solver = g2o.OptimizationAlgorithmLevenberg(solver)
optimizer.set_algorithm(solver)

# Set real points in space
true_points = bunny_coords[:3]
distorted_points = distort_bunny_coords[:3]

# Set camera intrinsics
focal_length = (intrinsics[0, 0], intrinsics[1, 1])
principal_point = ((intrinsics[0, 2], intrinsics[1, 2]))
baseline = 0.0 # Not relevant, not a stereo setup
K = np.array([*focal_length, *principal_point, baseline])
g2o.VertexSCam.set_cam(*K)

# Create vertices from cameras and fix them, as their pose is known.
for idx_extr, extrinsic in enumerate(extrinsics):
    pose = g2o.Isometry3d(np.linalg.inv(extrinsic))
    v_se3 = g2o.VertexSCam()
    v_se3.set_id(idx_extr)
    v_se3.set_estimate(pose)
    if idx_extr == 0:  # Fixing all cameras causes a segmentation fault, so unfix one
        v_se3.set_fixed(False)
    else:
        v_se3.set_fixed(True)
    v_se3.set_all() # Calculates matrices related to projection
    optimizer.add_vertex(v_se3)

# Each vertex must have a unique id.
# Ensure point and camera ids do not overlap.
last_pose_id = idx_extr
first_point_id = last_pose_id + 1

# Create vertices from all points and connect them to the cameras
for idx, (distorted_point, true_point) in enumerate(zip(distorted_points.T, true_points.T)):
    point_id = idx + first_point_id
    visible = []

    # Project the true points to the images, as observations are perfectly known
    for ext_id in range(len(extrinsics)):
        projected_point = optimizer.vertex(ext_id).map_point(true_point)
        visible.append((ext_id, projected_point))

    # Create a point vertex with distorted estimate
    vertex_point = g2o.VertexSBAPointXYZ()
    vertex_point.set_id(point_id)
    # see https://github.com/RainerKuemmerle/g2o/issues/109 on why we use this
    vertex_point.set_marginalized(True)
    vertex_point.set_estimate(distorted_point)
    optimizer.add_vertex(vertex_point)

    # Create an point-camera edge with each camera
    for ext_idx, projected_point in visible:
        edge = g2o.Edge_XYZ_VSC()
        edge.set_vertex(0, vertex_point)
        edge.set_vertex(1, optimizer.vertex(ext_idx))
        edge.set_measurement(projected_point)
        # The information matrix is the measurement
        # uncertainty. We set this equal for all measurements.
        edge.set_information(np.identity(3))
        edge.set_parameter_id(0, 0)
        optimizer.add_edge(edge)
    last_point_id = point_id

optimizer.initialize_optimization()
optimizer.set_verbose(False)
print("g2o graph has been created")
```

## Visualize point optimization
Visualize 50 steps of point refinement using sparse bundle adjustment optimization.

```python
def vertex_to_points(optimizer, first_point_id, last_point_id):
    """ Converts g2o point vertices to their current 3d point estimates. """
    vertices_dict = optimizer.vertices()
    estimated_points = list()
    for idx in range(first_point_id, last_point_id):
        estimated_points.append(vertices_dict[idx].estimate())
    estimated_points = np.array(estimated_points)
    xs, ys, zs = estimated_points.T
    return xs, ys, zs

ipv.show()
for i in range(50):
    optimizer.optimize(1)
    xs, ys, zs = vertex_to_points(optimizer, first_point_id, last_point_id)
    scatter_plot.x = xs
    scatter_plot.y = ys
    scatter_plot.z = zs
    time.sleep(0.25)
```

## Set up a scene and distort cameras
In this example, we will have perfect knowledge of the bunny points, perfect knowledge of observations, and noisy estimates of camera positions. The goal is to recover camera positions through optimization. Ground truth camera positions are shown in red. Distorted camera positions are shown in blue.

```python
def vertex_to_extrinsics(optimizer, last_pose_id):
    """ Converts g2o camera vertices to their current 3d camera pose estimates. """
    vertices_dict = optimizer.vertices()
    extrinsics = list()
    for idx in range(last_pose_id + 1):
        extrinsic = np.linalg.inv(vertices_dict[idx].estimate().matrix())
        extrinsics.append(extrinsic)
    return extrinsics

distorted_extrinsics = [distort_extrinsics(extrinsic, max_angle=30, max_trans=5) for extrinsic in extrinsics]
distorted_extrinsics[0] = extrinsics[0] # Do not distort the first camera
distorted_centers = camera_centers_from_extrinsics(distorted_extrinsics)
vis_scale = 5
init_3d_plot()
ipv.zlim(-35, 25)

bunny_coords = get_bunny()
b_xs, b_ys, b_zs = bunny_coords
bunny_coords = np.vstack([bunny_coords, np.ones((1, bunny_coords.shape[1]))])
scatter_plot = ipv.scatter(b_xs, b_ys, b_zs, size=0.5, marker="sphere", color='lime')

# Plot ground truth wireframes in red
for cam_center, extrinsic in zip(camera_centers, extrinsics):
    inv_extrinsic = np.linalg.pinv(extrinsic)
    plot_camera_wireframe(cam_center, vis_scale, inv_extrinsic, color='blue')

# Plot the distorted cameras  in blue
extrinsics_estimates = vertex_to_extrinsics(optimizer, last_pose_id)
old_plots = list()
for distort_center, distort_extrinsic, true_extrinsic in zip(distorted_centers, distorted_extrinsics, extrinsics):
    image = np.zeros_like(image)
    image = project_points_to_picture(image, bunny_coords, intrinsics, true_extrinsic)
    inv_extrinsic = np.linalg.pinv(distort_extrinsic)
    wireframe = plot_camera_wireframe(distort_center, vis_scale, inv_extrinsic, color='red')
    picture = plot_picture(image, inv_extrinsic, vis_scale)
    old_plots.append([wireframe, picture])

ipv.show()
```

## Define cameras, points, and observations as a g2o graph
As with the previous problem, we define a g2o graph.
Ideally, we would have been able to adapt the graph from the first problem and distort and unfix the cameras.
Unfortunately, this leads to a segmentation fault. G2opy probably permanently sets values upon `optimizer.initialize_optimization()`. Thus, at least for now, redefining all vertices seems to be the only option.

```python
optimizer = g2o.SparseOptimizer()
solver = g2o.BlockSolverSE3(g2o.LinearSolverCSparseSE3())
solver = g2o.OptimizationAlgorithmLevenberg(solver)
optimizer.set_algorithm(solver)

for idx_extr, extrinsic in enumerate(extrinsics):
    # Initialize with true intrisics so the
    # projection images will be correct.
    # Distorted extrinsic estimates are set at the end
    pose = g2o.Isometry3d(np.linalg.inv(extrinsic))
    v_se3 = g2o.VertexSCam()
    v_se3.set_id(idx_extr)
    v_se3.set_estimate(pose)
    if idx_extr:
        v_se3.set_fixed(False)
    else:
        # Fix the first camera
        v_se3.set_fixed(True)
    v_se3.set_all()
    optimizer.add_vertex(v_se3)

last_pose_id = idx_extr
first_point_id = last_pose_id + 1

for idx, true_point in enumerate(true_points.T):
    point_id = idx + first_point_id
    visible = []
    # Use true extrinsics to project points
    for ext_id in range(len(extrinsics)):
        projected_point = optimizer.vertex(ext_id).map_point(true_point)
        visible.append((ext_id, projected_point))
    vertex_point = g2o.VertexSBAPointXYZ()
    vertex_point.set_id(point_id)
    vertex_point.set_marginalized(True)
    vertex_point.set_estimate(true_point)
    optimizer.add_vertex(vertex_point)
    for ext_idx, projected_point in visible:
        edge = g2o.Edge_XYZ_VSC()
        edge.set_vertex(0, vertex_point)
        edge.set_vertex(1, optimizer.vertex(ext_idx))
        edge.set_measurement(projected_point)
        edge.set_information(np.identity(3))
        edge.set_parameter_id(0, 0)
        optimizer.add_edge(edge)
    last_point_id = point_id

# Set distorted extrinsics as pose estimate
for idx_extr, distorted_extrinsic in enumerate(distorted_extrinsics):
    if idx_extr:
        pose = g2o.Isometry3d(np.linalg.inv(distorted_extrinsic))
        optimizer.vertex(idx_extr).set_estimate(pose)
        optimizer.vertex(idx_extr).set_all()

optimizer.initialize_optimization()
optimizer.set_verbose(False)
print("g2o graph has been created")
```

## Visualize camera pose optimization
Visualize 400 steps of camera pose refinement using sparse bundle adjustment optimization.

```python
ipv.show()
time.sleep(2)
for _ in range(50):
    optimizer.optimize(2)
    extrinsics_estimates = vertex_to_extrinsics(optimizer, last_pose_id)
    new_plots = list()
    for idx, (extrinsic, image) in enumerate(zip(extrinsics_estimates, images)):
        wireframe, picture = old_plots[idx]
        cam_center = -extrinsic[:3, :3].T @ extrinsic[:3, 3]
        inv_extrinsic = np.linalg.pinv(extrinsic)
        wireframe = update_camera_wireframe(cam_center, vis_scale, inv_extrinsic, wireframe)
        new_picture = update_picture(image.copy(), inv_extrinsic.copy(), vis_scale, picture)
        new_plots.append([wireframe, new_picture])
    old_plots = new_plots
    time.sleep(2)
```

## Excercise: Distort bunny, camera poses, and point projections
In the previous cells, we refined the scene and camera locations in isolation.
As an exercise, create a graph where both camera estimates and scene point estimates are noisy.
One may even add noise to the coordinates of the projected points.
Judge the quality of the reconstruction: what is the mean squared error between estimated and true points?
Does the optimization require more iterations than when refining points or cameras in isolation?
How "correct" does the reconstruction look?

## Conclusion
In this notebook we:
* Learned about bundle adjustment
* Used bundle adjustment to refine scene points and camera locations

Yes, Mr. Frodo, it's over now.
This concludes the notebook series.
I hope that you have learned something new about computer vision!
The reason I started this series was to gain an understanding of the methods that are involved in solving the SLAM problem.
For the next project, I would like to work on a well-documented implementation of [ORB-SLAM](https://courses.cs.washington.edu/courses/csep576/21au/resources/ORB-SLAM_A_Versatile_and_Accurate_Monocular_SLAM_System.pdf) written in C++, so stay
tuned if that is of interest to you.

Looking back at this project, I am quite pleased with the notebook visualization format.
In the future, I would like to continue this series to visualize more classical computer vision subjects, such as:
* Stereo rectification to create depth maps
* Camera distortion parameters
* Models of more extreme lenses, such as fisheye and wide-angle  lenses
* Zhang's camera calibration algorithm
* [Alternative camera models](https://www.microsoft.com/en-us/research/uploads/prod/2020/06/Why-Having-10000-Parameters-in-Your-Camera-Model-is-Better-Than-Twelve.pdf) with more parameters for greater accuracy and expressivity
* Dense 3D reconstruction
* Automatic camera calibration using heuristics and visual cues such as angles and straight lines
* 3D point registration such as iterative closest point and derivatives
* Classical optical flow
* The bird's-eye view reprojection of a street as seen in some modern cars [[1]](https://img1.tongtool.com/r/ECEEHGKLCGEFNBEHCGIEMMILJLMJNGLILJGDX0Uk.jpg)[[2]](https://i.ytimg.com/vi/NQYxZ3DTToA/hqdefault.jpg)[[3]](https://img.alicdn.com/imgextra/i3/6000000005662/O1CN01Ph4xT31rhF17bkRxx_!!6000000005662-0-tbvideo.jpg)
* Binary image features, ORB, FAST, and BRIEF
* Modeling 3D lines using Plücker coordinates

Good luck and stay well!

## Sources
1. [g2o graph optimization framework](https://github.com/RainerKuemmerle/g2o)
2. [Zisserman, Richard Hartley Andrew. "Multiple view geometry in computer vision." 2004.](https://www.robots.ox.ac.uk/~vgg/hzbook/)
3. [Szeliski, Richard. "Computer vision: algorithms and applications." Springer Science & Business Media, 2010](https://szeliski.org/Book/)


#### 11.

Midterm Review #2:

1. Calculate the dipole moment of formamide (in units D), given the bond lengths and partial charges below:

$$\mu = \sum_{i}q_{i}\vec{r}_{i},$$

where $r=\sqrt{x^{2} + y^{2}+z^{2}}$ and so each component is considered in the resultant dipole vector $\mu = \sqrt{{\mu_{x}}^{2} + {\mu_{y}}^{2}+{\mu_{z}}^{2}}$.

Here, in this problem we are given a molecule and coordinates in 2-D space, therefore we will only have two components.

$$\mu_{x} = \sum_{i}q_{i}x_{i} \hspace{0.5cm} \text{and} \hspace{0.5cm} \mu_{y} = \sum_{i}q_{i}y_{i}  \tag{1}$$

$$\mu = \sqrt{{\mu_{x}}^{2} + {\mu_{y}}^{2}} \tag{2}$$

See Page 428 for more details.

Experimental value for dipole moment of formamide.

```python
import numpy as np
import pandas as pd
# Creating a dictionary for each of the atoms in the molecule
e = 1.602*10**(-19.) # C
atoms = {"H1":[0.270, (-1.3-0.5,0.86)], # [charge, (x,y)] in e and Å
        "O":[-0.571, (0.5,1.0)],
        "N":[-0.669, (-1.3,0.0)],
        "C":[0.650, (0.0,0.0)],         # deamed to be the origin
        "H2":[0.270, (-1.3-0.5,-0.86)],
        "H3":[0.050, (0.5,-0.86)]}
df = pd.DataFrame(atoms)
df
```

      H1
      O
      N
      C
      H2
      H3

      0
      0.27
      -0.571
      -0.669
      0.65
      0.27
      0.05

      1
      (-1.8, 0.86)
      (0.5, 1.0)
      (-1.3, 0.0)
      (0.0, 0.0)
      (-1.8, -0.86)
      (0.5, -0.86)

```python
print("Atoms: %s"%[atom for atom in atoms.keys()])
print("Charges: %s"%[charge[0] for charge in atoms.values()])
print("Positions: %s"%[pos[1] for pos in atoms.values()])
mu_x = sum([atom[1][0]*e*atom[1][1][0]*10**-10. for atom in atoms.items()]) # see eq 1 above
mu_y = sum([atom[1][0]*e*atom[1][1][1]*10**-10. for atom in atoms.items()]) # see eq 1 above
print("mu_x,mu_y = %0.4g,%0.4g C m"%(mu_x,mu_y))
mu = np.sqrt(mu_x**2 + mu_y**2) # see eq 2 above
print("mu = %0.4g C m"%(mu))
mu = mu/(3.335*10**(-30.))
print("mu = %0.4g D"%(mu))
print("Experimental mu = %s"%(3.73)) # Source given above
```

    Atoms: ['H1', 'O', 'N', 'C', 'H2', 'H3']
    Charges: [0.27, -0.571, -0.669, 0.65, 0.27, 0.05]
    Positions: [(-1.8, 0.86), (0.5, 1.0), (-1.3, 0.0), (0.0, 0.0), (-1.8, -0.86), (0.5, -0.86)]
    mu_x,mu_y = -5.812e-30,-9.836e-30 C m
    mu = 1.143e-29 C m
    mu = 3.426 D
    Experimental mu = 3.73

2. Two equal and opposite charges places 10 Å apart experience an attractive electrostatic force. Is this force stronger in water or in the gas phase? Why?

$$V(r) =  \frac{q_{1}q_{2}}{4 \pi \epsilon_{r}\epsilon_{0} r^{2}}, $$

where $q_{1}$ and $q_{2}$ are equal and opposite charges, the permittivity of free space $\epsilon_{0} = 8.854 × 10^{-12}$  $C^{2}$ $J^{-1}$ $m^{-1}$, $r=10 Å$ and the dielectric constant $\epsilon_{r}$ is different for both cases and depends on the medium. The dielectric constant is related to the effective electric field generated by interactions with the medium.

Therefore, the potential energy of the two charges separated by bulk water is reduced by nearly two orders of magnitude compared to the value it would have if the charges were separated by a vacuum. See Page 425 for more details.

3. True or False? The dielectric constant of water is 1.0.

False. The dielectric constant of water is 78. The dielectric constant of vacuum is 1.0

4. Are the dielectric properties of polar solvents due mainly to (a) atomic or (b) molecular polarization?

The larger the polarizability of the molecule, the greater is the distortion cause by a given strength of electric field.  If the molecule contains large atoms with electrons some distance from the nucleus, the nuclear control is less and the polarizability of the molecule is greater. The polarizability also depends on the orientation of the molecule with respect to the field unless geometrically symmetric ($CH_{4}$, $SF_{6}$, etc.).  See Pages 426-432 for more information.

5. Draw the Lennard-Jones potential, labeling the parts that are influenced by the r-12 term and the r-6 term.

##### Lennard-Jones Potential

$$V(r) = 4\epsilon[\overbrace{(\frac{\sigma}{r})^{12}}^{\text{Repulsion}}  - \overbrace{(\frac{\sigma}{r})^{6}}^{\text{Attraction}}]$$

 At short ranges, the main contribution from the potential energy function comes from $(\frac{1}{r})^{12}$, which corresponds to the repulsive interactions (Pauli repulsion due to overlap of electron orbitals).  At long ranges the main contribution from the potential energy function comes from $(\frac{1}{r})^{6}$,which corresponds to the attractive force (van der Waals force, or dispersion force).

Recall that the force is related to the potential energy by

$$F = - \frac{dV(r)}{dr} $$

Find the distance $r$, at which the potential energy reaches a minimum.

Set the first derivative of the potential energy function equal to zero and solve for $r$.

Find $\sigma$, the distance $r$ at which the potential energy is zero.

 Set $V(r) = 0$ and solve for $r$.

See Page 436 for more details.

6. True or False: The polymer scaling law Rg ~ Nν for a protein in a compact globule state has an exponent $v = \frac{1}{2}$.

**Flase.** The radius of gyration, $R_{g}$ can be approximated using Flory's mean field approach $N^{v}$, where $N$ is the the degree of polymerization (number of bond segments) and $v$ is known as the Flory exponent.  The Flory exponent is related to the quality of the solvent with respect to the polymer chain. For solvent that expands the chain into a globular shape $v = \frac{3}{5}$.
When the polymer is barely soluble/insoluble the chain becomes more compact (similar to a hard sphere) $v = \frac{1}{3}$. For an ideal linear polymer $v = \frac{1}{2}$ (simple random walk—ideal chain).

#### [Jump to table of contents.](#Table-of-Contents:)

Spectroscopy

7. The molar absorption coefficient for a dye is ϵ = 12000 L/mol/cm for light of wavelength λ = 590 nm. At a dye concentration of 100 mM, what is the cuvette path length necessary to decrease the intensity of light by a factor of 10?

$$A =  log(\frac{I_{0}}{I})  =  -log(T) = \epsilon b c$$

$$ b = -\frac{log(T)}{\epsilon c},$$

where the intensity of light transmitted has decreased by a factor of 10, so $T= \frac{I}{I_{0}}=\frac{I_{0}*0.1}{I_{0}} = 0.10$

8. What is an isobestic point? What does observing an isobestic point say about the system you are studying?

The isobestic point is a specific wavelength at which the absorbance intensity of a sample is invariant during a chemical reaction and comes about when there are at least two interrelated absorbing species in solution.

This says that if an isobestic point is to occur, then the two species involved are related linearly by stoichiometry, such that the absorbance is invariant for one particular wavelength. See Page 469 for more details.

Now, let's view an example. Consider the following reaction:

Figure 1. Overlaid absorbance spectra of solutions consisting of 22.5 mL buffer component (found in legend) that was added to 2.5 mL of ArOH (2-Naphthol) [(0.0592±0.0001) g of 99% ArOH  into 1L of DI Water]. The pH of each solution was determined to be 2.09, 11.82, 9.00, 8.70 and 8.59, from top of legend to bottom.

Note:

The isostilbic point is the wavelength at which the intensity of light emitted light is invariant during a chemical reaction.

Figure 2. Overlaid fluoresence spectra for the solutions in Figure 1.

9. What is a transition dipole moment? Given a ground-state wavefunction ψ0(x) and an excited-state ψ1(x), write down the formula for the dipole moment.

\begin{align}
\mu_{fi} &= < \psi_{f} | \vec{\mu} | \psi_{i} > \\
\mu_{10} &= < \psi_{1} | \vec{\mu} | \psi_{0} > = \int \psi_{1}(x) \space \vec{\mu} \space \psi_{0}(x) dx,
\end{align}

where $\vec{\mu}$ is the electric dipole moment operator, and $ \psi_{i}$ and $ \psi_{f}$ are the wavefunctions for the initial and final states, respectively.  See Page 469 for more details.

10. Is N2 vibrationally active? i.e. can its vibrational modes be studied by IR spectroscopy? Why or why not?

No. $N_{2}$ and $O_{2}$ are both infrared inactive since they do not possess vibrational modes that result in a change of dipole moment. See Pages 476-477 for more information.

11. How many vibrational modes are there for the linear molecule NO?

Linear molecule: $3N-5$

Non-Linear molecule: $3N-6$

12. UV light is used to optically excite a singlet S0 state to a S1 state, which decays to the triplet state T (which is “forbidden” from transition back to S0). What emission process does this describe? Over what range of time scales does this process occur?

Phosphorescent, seconds or fraction of seconds.  For more information, see the table of Rates of various processes.

### Rates of various processes

| $\text{Process}$ | $\text{Timescales (s)}$ | $\text{Radiative}$ | $\text{Transition}$ |
| :--: | :--: | :--: | :--: |
| IC         | $10^{-14}-10^{-11}$ | N  | $S_{n} \to S_{1}$ |
| Vib Relax  | $10^{-14}-10^{-11}$ | N  | ${S_{n}}^{*} \to S_{n}$  |
| Abs        | $10^{-15}$          | Y  | $S_{0} \to S_{n}$ |
| Fluor      | $10^{-9}-10^{-7}$   | Y  | $S_{1} \to S_{0}$  |
| ISC        | $10^{-8}-10^{-3}$   | N  | $S_{1} \to T_{1}$  |
| Phos       | $10^{-4}-10^{0}$   | Y  | $T_{1} \to S_{0}$  |

13. What kind of information can be obtained using FRET spectroscopy? What is the distance dependence of the FRET effect?

Förster resonance energy transfer (FRET) spectroscopy is useful for studying processes involving inter and intra-molecular energy transfer and can be used to measure distances (ranging from 1 to 9 nm) in biological systems.  Furthermore, conformational changes can be studied, and also good for studying bulk distances. Single molecule FRET —create histograms of binned FRET distances, ultimately revealing states.See Pages 500,501 for more information.

 Förster theory:

States that the efficiency of resonance energy transfer is related to the distance $R$ between donor-acceptor pairs by

$$\eta_{T} = \frac{{R_{0}}^{6}}{{R_{0}}^{6} + {R}^{6}}, $$

where $R_{0}$ is the distance at which $50 \%$ of the energy is transfered from donor to acceptor (known value and varified by experiment), and $R$ is the distance between donor and acceptor. This dependence is related by

$$\eta_{T} \propto {R}^{-6} $$

14. The fluorescence lifetime of tryptophan is τ0 = 2.6 ns. The ratio of fluorescence intensity with quencher (I) and without quencher (I0) was measured for a series of quencher concentrations [Q], as shown below. What is the quenching rate kQ for the quencher? Give correct units(!)

Because the fluorecence intensity and lifetime are both proportional to the fluorescence quantum yield, $I_{F,0}/I_{F}$ and $\tau_{0}/\tau$ against $[Q]$ should also be linear with the same slope and intercept. Therefore,

$$\frac{I_{F,0}}{I_{F}} = 1 + \tau_{0}k_{Q}[Q], $$

where the slope = $\tau_{0}k_{Q}$. Now, we pick two points and get the slope of the line.

$$ slope = m = \frac{(y_{2} - y_{1})}{(x_{2} - x_{1})} = \frac{(3 - 1)}{(15 - 0)} = 0.13333 mM^{-1}$$

$$k_{Q} = \frac{m}{\tau_{0}} = \frac{0.13333 mM^{-1}}{2.6 ns}  = 0.0513 mM^{-1}ns^{-1}$$

See Page 498 for more details.

Diffraction and crystallography

15. Draw two or more lines through the 2D lattice of points to show planes corresponding to Miller indices (2,1,0)

$$(2,1,0) = (k,h,l) \implies (\frac{a}{h},\frac{b}{k},0) = (\frac{a}{2},b,0) \\ \implies 2\times(\frac{a}{2},b,0) = (a,2b,0)$$

16. The crystal unit cell of MDM2 has dimensions a = 43.4 Å, b = 100.5 Å, c = 54.8 Å. What is the spacing d of diffraction planes with Miller indices (2,1,0)?

\begin{equation}
\frac{1}{d^{2}}=\frac{h^{2}}{a^{2}}+\frac{k^{2}}{b^{2}}+\frac{l^{2}}{c^{2}}
\end{equation}

See Page 418 for more information.

17. Describe the “phase problem” in X-ray crystallography, and at least one way the problem can be addressed (or at least circumvented to solve X-ray structures).

See Page 420 for phase problem, See Page 421 for the way the problem can be addressed.

Thermodynamics

18. What is the First Law of Thermodynamics?

The 1st Law is concerned with the internal energy. We postulate that it obeys two properties. The first assumption made is that **internal energy is extensive, which means it is additive.**

$$ U = U_{1} + U_{2} $$

The second assumption states that **energy is conserved**, states that energy can neither be created nor destroyed; energy can only be transfered or changed from one form to another e.g., $heat \rightarrow electrical$. Internal energy, $\Delta U$ of a system is the sum of the heat, $Q$ that flows across the system—surroundings barrier and the work, w done on the system by the surroundings:

$$ \boxed{\Delta U = Q + w} ,$$

where work is defined as organized motion that can transfer energy.

where $Q > 0$ is when heat is absorbed by the system, and $Q < 0$ is when heat is transfered from system to surroundings. The previous equation can be summarized in the table below:

| Internal Energy, U | Heat, Q | Work, w | Physical Description |
| :--: | :--: | :--: | :--: |
| + | + | + | Heat flows into the system & Work done by the surroundings. |
| +/- | + | - | Heat flows into the system & Work done by the system. |
| +/- | - | + | Heat flows into the surroundings & Work done by the surroundings. |
| - | - | - | Heat flows into the surroundings & Work done by the system. |

Work

The work term has the general form, $dW = \int Fdx $ where $F$ is the applied force, and x is a mechanical extensive variable.

We already know that $P = F/A$, so lets substitute this in for F, and say that area, $A = x^{2}$. Then,

\begin{align}
dW &= \int Fdx = \int P dx^{3}\\
dW &= \int_{V_{i}}^{V_{f}}P_{ext}dV = P(V_{f} - V_{i})
\end{align}

where $V$ is the volume of a bulk system, and $p_{ext}$ is the external pressure.

In the diagram above, one can see that $w>0$ is related to compression and $w<0$ corresponds to expansion.

19. A volume of ideal gas is compressed at constant temperature. Does the internal energy $U$ increase?

The internal energy of an ideal gas ONLY depends on its temperature.  $U = (3/2)nRT$

20. An ideal gas is slowly expanded from 1 L to 2 L at standard pressure and temperature.

(a) What is the work $w$ done on the system? (Make sure you have the correct sign!)

Expansion work @ Constant P,T

$$w = -P \Delta V = -0.987 atm (+1 \space L) = -0.987 \space L \space atm = - 0.100 \space kJ$$

(b) What is the heat change $Q$ of the system?

Since T is fixed, $\Delta U = w + q$ must be zero, which means $q = -w = +0.100$ $kJ$

21. What is the mechanical work done for free expansion of a gas (say, in a vacuum)?

There is no work done when a system expands freely (nothing to push against).

 Since $P_{external} = 0$, then $w = -PdV = 0$

22. Consider the following reaction performed at a constant pressure of 1 bar, at 25˚ C = 298 K, with all species in the gas phase:

$$2 {NH_{3}}_{(g)} \rightarrow {N_{2}}_{(g)} + 3 {H_{2}}_{(g)}$$

If we assume that all reactants and products behave like ideal gases, what is the expansion work done (in kJ/mol)? (Remember to carefully consider the sign of this value)

In this reaction, there is a net production of $\Delta n = +2 mol$ of gas per mole of reactant.  By the ideal gas law, there must be an expansion of volume: $P\Delta V = \Delta n RT$.  Therefore, the work done is $w = -p∆V = -(+2 mol)(8.314 J/K/mol)(298 K) = -4.95$ $kJ$.

23. Draw a picture of the enthalpy H of water as a function of temperature T, from -25 ˚C to 100 ˚C. Clearly label the diagram.

\begin{align}
\Delta H_{total} \space = \space & mC_{ice}\Delta T_{-25°C \rightarrow 0°C}  +  n\Delta H_{fusion} \\
                  &+ mC_{water} \Delta T_{0°C\rightarrow100°C} + n \Delta H_{vaporization}\\
\end{align}

Note that the temperature does not change during a phase change...

24. The heat capacity Cp of an unfolded protein is 8.0 kJ K-1 mol-1 at 350 K. Assuming Cp is relatively temperature-independent, calculate the change in enthalpy ∆H when the temperature is increased from 350 K to 375 K.

The following expression is derived in problem 25.

$$C_{P} = (\frac{\partial H}{\partial T})_{P} \implies dH = C_{P} dT$$

$$ \Delta H = C_{P} \Delta T$$

25. True or False? For an ideal gas, Cp + CV = nR

False. $nR =  C_{P} - C_{V}$


# Determining concentrations by absorption spectroscopy
#### I. Beer's law

The relation between the concentration of an absorbing analyte and the measured absorbance $A$ or transmittance $T$ of solutions contained in transparent cells that have a path length of $b$ centimeters is described by Beer's law:

$A = -\log{T} = \log{\frac{P_{0}}{P}} = \epsilon b c$

where $P_{0}$ is the radiant power in watts incident on the sample and $P$ is the radiant power transmitted by the sample, $\epsilon$ is the molar absorptivity, and $c$ is the concentration of the absorbing substance.

A mixture containing more than one absorbing substance, i.e. a multicomponent absorbing system, will exhibit a total absorbance,

    $\begin{align} A_{total} &= A_{1} + A_{2} + \dots + A_{n} \\
    &= \epsilon_{1}bc_1 + \epsilon_{2}bc_2 + \dots + \epsilon_{n}bc_n \end{align}$

#### II. Key assumptions and observations
1. Beer's law only applies to mixtures of non-interacting species.
2. Significant deviations from the direct (linear) proportionality expressed by the law are often observed under the following conditions:
    * At high analyte concentrations, usually above $0.01 M$, solute-solvent and solute-solute interactions, or hydrogen bonding can affect the analyte absorptivity.
    * At low analyte concentrations but high concentrations of another species, particularly electrolytes.
    * When changes in the analyte concentration significantly alters the refractive index $n$ of the solution.
    $A_{observed} = \frac{n}{(n^2+2)^2}\epsilon bc$

    * When the analyte associates, dissociates or reacts with the solvent (e.g., an unbuffered solution of a species affected by an acid-base equilibrium) resulting in a multicomponent absorbing system.
    $A_{observed} = A_{1} + A_{2} + \dots + A_{n}$

    * When polychromatic source radiation is employed to make absorbance measurements and the molar absorptivity of the analyte is not constant across the wavelengths considered.

    $A_{observed} =
    \log{
    \frac{P_{0\lambda_{1}} + P_{0\lambda_{2}} + \dots + P_{0\lambda_{n}}}
    {P_{\lambda_{1}} + P_{\lambda_{2}} + \dots + P_{\lambda_{n}}}}
    =
    \log{
    \frac{P_{0\lambda_{1}} + P_{0\lambda_{2}} + \dots + P_{0\lambda_{n}}}
    {P_{0\lambda_{1}}10^{-\epsilon_{\lambda_{1}}bc} + P_{0\lambda_{2}}10^{-\epsilon_{\lambda_{2}}bc} + \dots + P_{0\lambda_{n}}10^{-\epsilon_{\lambda_{n}}bc}}}
    $

    * When stray radiation is present inside the instrument used to measure absorbance.
    $A_{observed} = \log{\frac{P_{0} + P_{stray}}{P + P_{stray}}}$

    * When the cells holding the analyte and blank solutions are not of equal path length and equivalent optical characteristics.
    $A_{observed} = \epsilon b c + k$

3. For typical organic molecules, strong absorption bands have molar absorptivities ranging from $10^4$ to $10^5M^{-1}cm^{-1}$.

## Experimental methods
#### I. Determining concentrations of absorbing components in a mixture
In principle, it is possible to determine the concentrations $c_{1}, c_{2}, \dots, c_{n}$ of the individual components of a mixture even if their spectra overlap completely.

## Computation
#### Using principal component analysis on Fourier-transformed spectra
0. Import required modules/libraries

```python
import numpy
import math
```

1. Compute concentrations of standard mixtures. The number of standard mixtures must be equal to or greater than the number of absorbing components. The concentration of each component must be independently varied within the set of standard mixtures to avoid redundancy. To achieve better accuracy, the concentration range of each component across all standard mixtures should closely bracket the range expected in the unknowns. It is also important to maximize the number of components contained in each of the standard mixtures to compensate for the effects of molecular interactions on the measured absorbance.

2. Compute the Fourier transform of the absorption spectra divided by the path length (i.e. the optical densities) measured from each of the standard mixtures.

```python
def compFTOfSpectrum(absorbances, path_length = 1.0):
    '''Computes the Fourier transform of the absorbances measured divided the path length

    Parameters
    ----------
    absorbances : array of floats
        Array of absorbances in absorbance units from the spectrum measured.
    path_length : float
        Path length of cuvette holding the sample in centimeters.

    Returns
    -------
    array of floats
       Fourier transform of the optical densities.
    '''
    fourier_transform = numpy.fft.fft(numpy.divide(absorbances, path_length))
    return fourier_transform
```

3. Create a matrix $F$ with $t$ rows and $m$ columns by arranging a subset of consecutive terms $t$ of the computed Fourier transform in a column for each standard mixture $m$. Compute the variance-covariance matrix $A$ ($A = FF^T$) and diagonalize $A$ to obtain the set of $t$ eigenvalues and eigenvectors. Arrange the eigenvectors in order of decreasing eigenvalues.

```python
def compSortedEigenSet(f):
    '''Computes the set of eigenvalues and eigenvectors of the variance-covariance matrix computed from the input matrix
    and sorts them in order of decreasing eigenvalue magnitude.

    Parameters
    ----------
    f : 2D-array of floats
        The input matrix. For example, the Fourier terms of each standard mixture structured as
        // mix_1,   mix_2,  ...,  mix_m
        [[ f_1_1,   f_1_2,  ...,  f_1_m], // term_1
         [ f_2_1,   f_2_2,  ...,  f_2_m], // term_2
                            ...,
         [ f_t_1,   f_t_2,  ...,  f_t_m], // term_t

    Returns
    -------
    tuple of arrays
       The first element of the tuple is the list of eigenvalues.
       The second element of the tuple is the 2D numpy array of the corresponding eigenvectors.
    '''
    f = numpy.array(f)
    a = numpy.matmul(f, f.transpose())
    w, v = numpy.linalg.eigh(a)
    eigvals, eigvecs = zip(*sorted(zip(w, v.T), reverse=True))
    return list(eigvals), numpy.array(list(eigvecs)).T
```

4. Compute the value of the indicator function $ind(i)$ at each number $i$ of eigenvectors used. Find the minimum value of this function. The number of eigenvectors to select from the sorted set will be one less than the number of eigenvectors used at this minimum value. Arrange this subset of eigenvectors in a matrix $V$.

```python
def compNumEigenVectors(eigenvalues):
    '''Computes the number of eigenvectors to select from the sorted set of eigenvalues by finding
    the minimum value of the indicator function

    Parameters
    ----------
    eigenvalues : array of floats
        Array of eigenvalues sorted in order of decreasing magnitude.
    Returns
    -------
    int
       Number of eigenvectors to select from the sorted set.
    '''
    num_eigenvectors = numpy.argmin([eigenvalue/numpy.sum(eigenvalues[i:]) for i, eigenvalue in enumerate(eigenvalues)]) - 1
    return num_eigenvectors
```

5. Compute the matrix $Z$ of projections of each of the vector representations for the standard mixtures onto the orthogonal eigenvectors, $Z = V^TF$.

6. Arrange the concentrations of each one of the $n$ components for each of the $m$ standard mixtures in a $n$ by $m$ matrix $C$. Then, compute the proportionality matrix $P = CZ^T(ZZ^T)^{-1}$. Finally, compute the calibration matrix $M = PV^{T}$. The concentrations of each component in an unknown mixture $c_u$ can be obtained by the following relationship after taking the Fourier transform of the mixture's absorbance spectrum $f_u$.
$c_u = Mf_u$

7. Compute the expected error in the concentrations of the unknown samples, known as the standard error of estimation ($SEE$), by calculating the concentrations of the standard mixtures from their Fourier-transformed spectra using the calibration matrix $M$.

$SEE = \frac{\|C_{actual} - C_{estimated} \|_{F}}{\sqrt{n(m-k)}} = \sqrt{\frac{Tr(C^TMF)}{n(m-k)}}$,

    where $n$ is the number of absorbing components, $m$ the number of standard mixtures and $k$ the number of eigenvectors used.

```python
def compSEE(concentration_matrix, calibration_matrix, ft_spectra, num_eigenvectors):
    '''Computes the standard error of estimation

    Parameters
    ----------
    concentration_matrix : 2D-array of floats
        Concentrations of each one of the n components for each of the m standard mixtures arranged in a n by m matrix.
    calibration_matrix : 2D-array of floats
        Matrix obtained from the calibration experiments that relates the Fourier transform of a mixture's abosrbance
        spectrum to the concentrations of each component in an unknown mixture.
    ft_spectra : 2D-array of floats
        Obtained by arranging the subset of t Fourier terms computed from the absorption spectrum
        of each of the m standard mixtures as a t by m matrix.
    num_eigenvectors : int
        Number of eigenvectors used.
    Returns
    -------
    float
       Standard error of estimation (SEE)
    '''
    C = numpy.array(concentration_matrix)
    M = numpy.array(calibration_matrix)
    F = numpy.array(ft_spectra)
    n, m = C.shape
    try:
        SEE = math.sqrt(numpy.trace(numpy.matmul(C.T,numpy.matmul(M,F))).real/(n*(m-num_eigenvectors)))
        return SEE
    except ZeroDivisionError:
        print('''
        The number of eigenvectors is coincidentally the same as the number of standard mixtures.
        This causes a division by zero in the computation.
        SEE cannot be computed in this case.
        ''')
        return math.nan
```

# References
1. Skoog, D. A., Holler, F. J. and Crouch, S. R., Principles of Instrumental Analysis, 6th Edition, Thomson Brooks/Cole, Belmont, CA, 2007.
2. Weismüller, J. A., & Chanady, A. (1992). Quantitative multicomponent analysis of complex mixtures by means of Full Spectrum quantitation and principal component analysis. TrAC Trends in Analytical Chemistry, 11(3), 86-90.
3. Brown, C. W., Bump, E. A., & Obremski, R. J. (1986). Accounting for impurities in spectroscopic multicomponent analysis using Fourier vectors. Applied spectroscopy, 40(7), 1023-1031.
4. Brown, C. W., Obremski, R. J., & Anderson, P. (1986). Infrared quantitative analysis in the Fourier domain: processing vector representations. Applied spectroscopy, 40(6), 734-742.
5. Donahue, S. M., Brown, C. W., & Obremski, R. J. (1988). Multicomponent analysis using Fourier transform infrared and UV spectra. Applied spectroscopy, 42(2), 353-359.


```python
from sympy import symbols, init_printing, Function, Sum, Eq, Matrix,cos, sin, pi, I, exp

#Paper T.Lubin 2010b
#Machine with no load rotor, rotor slots, current sheet on stator side

init_printing()

R_1, R_2, R_3, beta, mu_0 = symbols('R_1, R_2, R_3, beta, mu_0', real = 'true', positive = 'true', nonzero ='true')

theta_i = symbols('theta_i')

#Declaration of the motor geometry
Z_r = symbols('Z_r', integer = 'true', positive = 'true', nonzero ='true') #Number of rotor, stator slots

#Declaration of the space variables
r, t = symbols('r t', real = 'true', positive = 'true')
theta = symbols('theta', real ='true')

#Declaration of the discretizing integers for stator and rotor slots
i = symbols('i', integer='true', positive = 'true', nonzero='true')

#Declaration of th magnetic potentials in the 5 areas
P = Function("P")
E = Function("E")
```

```python
##AREA I : AIR GAP
#Dummy variable(s) of summation
n, N, k, K  = symbols('n N k K', integer = 'true', positive = 'true', nonzero ='true')
#Integration constants
A_I0, A_In, B_In, C_In, D_In = symbols('A_I0, A_In, B_In, C_In, D_In', commutative=False)
#Expression of the potential
AzI_cst = A_I0
AzI_exp = A_In*R_2/n*P(n, r, R_3)/E(n, R_2, R_3) - B_In*R_3/n*P(n, r, R_2)/E(n, R_2, R_3)
expn = exp(I*(n*theta + k*t))
AzI = AzI_cst + Sum(Sum(AzI_exp*expn,(n,1,N)), (k,1,K))

#Expression of the field
#BrI_cst, BrI_cos, BrI_sin = compute_Br(AzI_cst, AzI_cos, AzI_sin, n, r, theta)
#BrI = BrI_cst + Sum(BrI_cos*cosn+BrI_sin*sinn,(n,1,N))

#BthetaI_cst, BthetaI_cos, BthetaI_sin = compute_Btheta(AzI_cst, AzI_cos, AzI_sin, r)
#BthetaI = BthetaI_cst + Sum(BthetaI_cos*cosn+BthetaI_sin*sinn,(n,1,N))

fAzI = Function('Az_I')(r,theta,t)
fBrI = Function('Br_I')(r,theta,t)
fBthetaI = Function('Btheta_I')(r,theta)

Eq(fAzI, AzI) #, Eq(fBrI, BrI), Eq(fBthetaI, BthetaI)
```

```python
##AREA i : ROTOR SLOT
#Dummy variable(s) of summation
k, K  = symbols('k, K', integer = 'true', nonzero = 'true')
#Integration constants
A_i0, A_ik = symbols('A_i0, A_ik', commutative=False)
#Expression of the potential
Azi_cst = A_i0
Azi_cos = A_ik*P(k*pi/beta, R_1, r)/P(k*pi/beta, R_1, R_2)
Azi_sin = 0
coski = cos(k*pi/beta*(theta-theta_i+beta/2))
sinki = sin(k*pi/beta*(theta-theta_i+beta/2))
Azi = Azi_cst + Sum(Azi_cos*coski,(k,1,K))

#Expression of the field
Bri_cst, Bri_cos, Bri_sin = compute_Br(Azi_cst, Azi_cos, Azi_sin, k*pi/beta, r, theta)
Bri = Bri_cst + Sum(Bri_cos*coski+Bri_sin*sinki,(k,1,K))

Bthetai_cst, Bthetai_cos, Bthetai_sin = compute_Btheta(Azi_cst, Azi_cos, Azi_sin, r)
Bthetai = Bthetai_cst + Sum(Bthetai_cos*coski+Bthetai_sin*sinki,(k,1,K))

fAzi = Function('Az_i')(r,theta)
fBri = Function('Br_i')(r,theta)
fBthetai = Function('Btheta_i')(r,theta)
```

```python
Potentials = Matrix([Eq(fAzI, AzI), Eq(fAzi, Azi)])
Fields = Matrix([Eq(fBrI, BrI), Eq(fBthetaI, BthetaI), Eq(fBri, Bri), Eq(fBthetai, Bthetai)])

#Current sheet
p, m, M  = symbols('p, m, M', integer = 'true', nonzero = 'true')
fK = Function('K')(theta)
K_m, alpha = symbols('K_m, alpha')
K_cos = K_m
cosm = cos(m*p(theta-alpha))
K = Sum(K_cos*cosm, (m,1,M))

## RESULTING EQUATIONS

Csts = Matrix([A_In, B_In, C_In, D_In, A_ik])
var = [n, n, n, n, (k, i)]

##General integrals to compute
fI_cosni, fI_sinni = symbols('I_cosni, I_sinni', commutative = False)
fI_cosksinni, fI_coskcosni  = symbols('I_cosksinni, I_coskcosni', commutative = False)

##CONDITION A.11 = A.9
A_11 = Eq(BthetaI_cos.subs(r, R_2), 1/pi*(Bthetai_cst.subs(r, R_2)*fI_cosni +Bthetai_cos.subs(r, R_2)*fI_coskcosni))

##CONDITION A.7
A_7 = Eq(B_In, mu_0*K_m*cos(m*p*alpha))

##CONDITION A.12 = A.10
A_12 = Eq(BthetaI_sin.subs(r, R_2), 1/pi*(Bthetai_cst.subs(r, R_2)*fI_sinni +Bthetai_cos.subs(r, R_2)*fI_cosksinni))

##CONDITION A.8
A_8 = Eq(D_In, mu_0*K_m*sin(m*p*alpha))

##CONDITION A.13
A_13 = Eq(A_ik, 2/beta*((A_In*R_2/n*P(n, R_2, R_3)/E(n, R_2, R_3) + B_In*R_3/n*2/E(n, R_3, R_2))*fI_coskcosni + (C_In*R_2/n*P(n, R_2, R_3)/E(n, R_2, R_3) + D_In*R_3/n*2/E(n, R_3, R_2))*fI_cosksinni))
A_13bis = Eq(Azi_cos.subs(r, R_2), 2/beta*(AzI_cos.subs(r, R_2)*fI_coskcosni + AzI_sin.subs(r, R_2)*fI_cosksinni))

SetEqs = Matrix([A_11, A_7, A_12, A_8, A_13])

Mat, Vect, Index = get_System(var, var, Csts, SetEqs)

#I_coskcosni = computeInt_coscos(k*pi/beta, -theta_i + beta/2, n, 0, theta_i - beta/2, theta_i + beta/2)
#I_cosksinni = computeInt_cossin(k*pi/beta, -theta_i + beta/2, n, 0, theta_i - beta/2, theta_i + beta/2)

#I_coskcosni = computeInt_coscos(k*pi/beta, -theta_i, n, 0, theta_i, theta_i + beta)
#I_cosksinni = computeInt_cossin(k*pi/beta, -theta_i, n, 0, theta_i, theta_i + beta)

#def P(n,x,y) :
#
#    return (x/y)**n + (y/x)**n
#
#def E(n,x,y) :
#
#    return (x/y)**n - (y/x)**n
#
#P_n_R2_R3 = P(n, R_2, R_3)
#E_n_R2_R3 = E(n, R_2, R_3)
#E_n_R3_R2 = E(n, R_3, R_2)
#E_k_R1_R2 = E(k*pi/beta, R_1, R_2)
#P_k_R1_R2 = P(k*pi/beta, R_1, R_2)

#Current sheet Fourier series expansion
#I1 = computeInt_coscos(m*p, -alpha, n, 0, 0,2*pi)
#I2 = computeInt_coscos(m*p, -alpha, m*p, 0, 0,2*pi)
```


# 第２章 確率空間

### 参考
+ [ギリシャ文字等の入力方法](https://qiita.com/alchemist/items/0ce850770d8cc3df0ab4)
+ [Pythonからはじめる数学入門](https://www.amazon.co.jp/dp/4873117682)
+ [確率論のσ加法族（ほんとは有限だけど）の自動生成を Sympy で書いてみた！ - Qiita](https://qiita.com/ktsysd/items/97f75330f9492e727799)
    + [2019-10-17] tehen_さんのコメントをご覧ください．めっちゃ短くて洗練された **2.2.3 σ代数** の実装を書いてくれました！

```python
from sympy import *
```

## 2.1 見本空間 $\Omega$

### 見本

注意点：sympy の仕様なのか，要素を文字にすると思うような動作が得られないので，以下，整数に置き換えて計算します．

```python
a=1
b=2
c=3
```

### 見本空間

```python
Ω = FiniteSet(a,b,c) #\Omega[Tab] で入力可能
Ω
```

    {1, 2, 3}

## 2.2 事象の代数 $\mathcal F$

## 2.2.1 事象

```python
A1 = FiniteSet(a,c)
A1
```

    {1, 3}

```python
A2 = FiniteSet(b,c)
A2
```

    {2, 3}

## 2.2.2 事象の代数

### 事象の族 F

```python
F = FiniteSet(A1,A2)
F
```

    {{1, 3}, {2, 3}}

### 和事象

```python
A1 + A2
```

    {1, 2, 3}

### 積事象

```python
A1 & A2
```

    {3}

### 余事象

```python
Ω - A1
```

    {2}

```python
Ω - A2
```

    {1}

### 式(2.5) --- 追加後の族

```python
F1 = F + FiniteSet( A1+A2, A1&A2, Ω-A1, Ω-A2 )
F1
```

    {{1}, {2}, {3}, {1, 3}, {2, 3}, {1, 2, 3}}

### 式(2.6) --- 不足チェック

```python
Ω-(A1+A2)
```

    EmptySet()

```python
Ω-(A1+A2) in F1
```

    False

### 式(2.7) --- 完成

```python
FF = F1 + FiniteSet(Ω-(A1+A2),Ω-(A1&A2))
FF
```

    {EmptySet(), {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}

## 2.2.3 $\sigma$代数

### 定義2.1 --- チェック用の関数はこんな感じ？

```python
def is_sigma_algebra(Ω,FF):
    is_ok = True

    # (σ1)のチェック
    is_ok = is_ok and (Ω in FF)

    # for文で全要素を巡るためのリスト化
    elm=list(FF) #リスト化
    n=len(elm)   #リストの要素数

    # (σ2)のチェック
    for i in range(0,n):
        is_ok = is_ok and (Ω-elm[i] in FF)

    # (σ3)の全数チェック
    for i in range(0,n):
        for j in range(i,n):
            is_ok = is_ok and (elm[i]+elm[j] in FF)
    return is_ok
```

```python
is_sigma_algebra(Ω,F1)
```

    False

```python
is_sigma_algebra(Ω,FF)
```

    True

### 生成用の関数ってこんな感じ？

#### 部品１： 補集合を追加する関数

```python
def append_complements(Ω,F):
    result=F
    for elm in list(F):
        celm = Ω - elm
        result = result + FiniteSet(celm)
    return result
```

```python
F
```

    {{1, 3}, {2, 3}}

```python
append_complements(Ω,F)
```

    {{1}, {2}, {1, 3}, {2, 3}}

#### 部品2： 全ての組み合わせの和集合を追加する関数

```python
def append_unions(F):
    result=F
    Flist=list(F)
    Fn=len(Flist)
    for i in range(0,Fn):
        for j in range(i,Fn):
            result = result + FiniteSet(Flist[i]+Flist[j])
    return result
```

```python
append_unions(F)
```

    {{1, 3}, {2, 3}, {1, 2, 3}}

#### $\sigma$代数（有限版）を生成する関数　（注）ここでの$\sigma$代数は要素数が有限なので，本当はブール代数という．

```python
def generate_sigma_algebra(Ω,F):
    cur_F = F + FiniteSet(Ω)
    prev_F = EmptySet()
    while(prev_F != cur_F):
        prev_F = cur_F
        cur_F = append_complements(Ω,cur_F)
        cur_F = append_unions(cur_F)
    return cur_F
```

```python
FFauto = generate_sigma_algebra(Ω,F)
FFauto
```

    {EmptySet(), {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}

```python
is_sigma_algebra(Ω,FFauto)
```

    True

### 例題2.1

```python
F_2_1=FiniteSet(FiniteSet(a,c),FiniteSet(b))
generate_sigma_algebra(Ω,F_2_1)
```

    {EmptySet(), {2}, {1, 3}, {1, 2, 3}}

### 演習2.2

```python
F_eg21=FiniteSet(FiniteSet(a,b),FiniteSet(b))
generate_sigma_algebra(Ω,F_eg21)
```

    {EmptySet(), {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}

### おまけ --- その他の場合

```python
generate_sigma_algebra(Ω,FiniteSet(FiniteSet(b)))
```

    {EmptySet(), {2}, {1, 3}, {1, 2, 3}}

### 補足： 最大の$\sigma$代数 = $\Omega$の冪集合

```python
Ω.powerset()
```

    {EmptySet(), {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}

この問題のFで生成した$\sigma$代数は，$\Omega$の冪集合にたまたま一致する．そうでないのもある．例題2.1参照．

```python
FF == Ω.powerset()
```

    True

## 2.3 確率 $P$

## 2.3.1 見本に基づく方法

### 式(2.9)

```python
def P(A):
    if A == FiniteSet(a):
        p = 1/3
    elif A == FiniteSet(b):
        p = 1/3
    elif A == FiniteSet(c):
        p = 1/3
    else:
        p = False
    return p
```

```python
P(FiniteSet(a))
```

    0.3333333333333333

### 確率を自動計算する関数

```python
def Pauto(A):
    lst = list(A)
    n = len(lst)
    if n == 1:
        if A == FiniteSet(a):
            p = 1/3
        elif A == FiniteSet(b):
            p = 1/3
        elif A == FiniteSet(c):
            p = 1/3
        else:
            p = False
    elif n > 1:
        p = 0;
        for x in lst:
            p += Pauto(FiniteSet(x))
    return p
```

### 演習2.4

```python
Pauto(A2)
```

    0.6666666666666666

```python
Pauto(Ω)
```

    1.0

### おまけ --- 式(2.10)の確率を使った場合

```python
def Pauto2(A):
    lst = list(A)
    n = len(lst)
    if n == 1:
        if A == FiniteSet(a):
            p = 1/4
        elif A == FiniteSet(b):
            p = 2/4
        elif A == FiniteSet(c):
            p = 1/4
        else:
            p = False
    elif n > 1:
        p = 0;
        for x in lst:
            p += Pauto2(FiniteSet(x))
    return p
```

```python
Pauto2(A2)
```

    0.75

```python
Pauto2(Ω)
```

    1.0

## 2.4 確率変数 $X(\omega)$

## 2.4.1 確率変数

### 式(2.16)

```python
def X(ω):
    if ω == a:
        val = 3.5
    if ω == b:
        val = 2.8
    if ω == c:
        val = 3.5
    return val
```

```python
def Y(ω):
    if ω == a:
        val = 22
    if ω == b:
        val = 22
    if ω == c:
        val = 25
    return val
```

```python
X(a), X(b), X(c), Y(a), Y(b), Y(c)
```

    (3.5, 2.8, 3.5, 22, 22, 25)

## 2.4.2 測定値の生起確率

### $X^{-1}$ を求める関数

```python
def FUNCinv(func,Ω,val):
    invset = EmptySet()
    for ω in Ω:
        if func(ω) == val:
            invset += FiniteSet(ω)
    return invset
```

```python
def Xinv(Ω,val):
    return FUNCinv(X,Ω,val)
```

```python
Xinv(Ω, 3.5)
```

    {1, 3}

```python
Xinv(Ω, 2.8)
```

    {2}

### 演習2.6 --- $Y^{-1}$ を求める関数

```python
def Yinv(Ω,val):
    return FUNCinv(Y,Ω,val)
```

```python
Yinv(Ω, 22)
```

    {1, 2}

```python
Yinv(Ω, 25)
```

    {3}

### 演習2.7 --- 確率変数の値の確率

```python
Pauto(Xinv(Ω, 3.5))
```

    0.6666666666666666

```python
Pauto(Xinv(Ω, 2.8))
```

    0.3333333333333333

```python
Pauto(Yinv(Ω, 22))
```

    0.6666666666666666

```python
Pauto(Yinv(Ω, 25))
```

    0.3333333333333333

### おまけ --- 式(2.10)の確率を使った場合

```python
Pauto2(Xinv(Ω, 3.5))
```

    0.5

```python
Pauto2(Xinv(Ω, 2.8))
```

    0.5

```python
Pauto2(Yinv(Ω, 22))
```

    0.75

```python
Pauto2(Yinv(Ω, 25))
```

    0.25

## 2.4.3 確率変数から生成される$\sigma$代数

### 式(2.24), (2.25) --- 確率変数$X(\omega)$から生成される$\sigma$代数

```python
generate_sigma_algebra(Ω, FiniteSet( Xinv(Ω, 2.8), Xinv(Ω, 3.5) ))
```

    {EmptySet(), {2}, {1, 3}, {1, 2, 3}}

### 演習2.8 --- 確率変数$Y(\omega)$から生成される$\sigma$代数

```python
generate_sigma_algebra(Ω, FiniteSet( Yinv(Ω, 22), Yinv(Ω, 25) ))
```

    {EmptySet(), {3}, {1, 2}, {1, 2, 3}}

### 演習2.9 --- 超つまんない例　．．．足の数　申し訳ございません <(_"_)>

```python
def Z(ω):
    if ω == a:
        val = 4
    if ω == b:
        val = 2
    if ω == c:
        val = 2
    return val
```

```python
def Zinv(Ω,val):
    invset = EmptySet()
    for ω in Ω:
        if Z(ω) == val:
            invset += FiniteSet(ω)
    return invset
```

```python
generate_sigma_algebra(Ω, FiniteSet( Zinv(Ω, 2), Zinv(Ω, 4) ))
```

    {EmptySet(), {1}, {2, 3}, {1, 2, 3}}

今度は {2,3} = {b,c} が含まれているので，確率変数$Z$を採用すれば，二足歩行{b,c}は可測となる．

感想： 確率空間の計算，python で全部書けてしまった．すごいな python．もっとはやく使えばよかった・・・


# Listen

In der Praxis sind Berechnungen häufig nicht nur für einen einzelnen Wert durchzuführen, sondern für mehrere gleichartige Werte. Als Beispiel kann eine Wohnung dienen, bei der der Abluftvolumenstrom für jeden einzelnen Abluftraum berechnet werden muss und verschiedene weitere Berechnungen davon ebenfalls betroffen sind.

Nehmen wir an, ein Bad ($45\frac{m^3}{h}$), zwei WCs je ($25\frac{m^3}{h}$), eine Abstellkammer ($25\frac{m^3}{h}$) und ein Saunaraum ($100\frac{m^3}{h}$) sollen mit Abluft versorgt werden.

Dann wären die erforderlichen Berechnungen für jeden einzelnen Raum durchzuführen. Dazu werden Listen benutzt:

`# Liste der Räume`

`raum = ['Bad','WC_1','WC_2','Abstellkammer', 'Saunaraum']`

`# Liste der Abluftvolumenströme je Raum`

`dV_ab = [45,25,25,25,100]`

Die Berechnungen können nun mit der Liste durchgeführt werden:

`dV_ges_ab = sum(dV_ab)`

# Beispiel

Berechnen Sie für die oben gegebenen Werte den Gesamtvolumenstrom `dV_ges_ab` und ermitteln Sie, wieviel Prozent der Abluft in jeden Raum geführt wird.

# Lösung

Zunächst werden die Listen angelegt.

Dann wird die Summe aller Listeneinträge gebildet und anschließend angezeigt:

```python
raum = ['Bad','WC_1','WC_2','Abstellkammer', 'Saunaraum']

dV_ab = [45,25,25,25,100]

dV_ges_ab = sum(dV_ab)
dV_ges_ab
```

    220

Für die Berechnung der Prozentzahlen muss eine neue Liste gebildet werden. Dazu wird oft eine `for`-Schleife verwendet.

Der Lösungsweg sieht ist prinzipiell:

```
# neue Liste initialisieren
neue_liste = []

# Schleife für die Berechnung der neue Werte
for wert in alte_liste:
    neuer_wert = ...
    neue_liste.append(neuer_wert)

# Ergebnis anzeigen
neue_liste
```

```python
# leere Liste anlegen (initialisieren)
dV_prozent = []

# Schleife über alle Elemente von dV_ab
for dV in dV_ab:
    # Berechnung der Prozentzahl für dies dV
    prozent = dV/dV_ges_ab * 100
    # Erweiterung der Liste dV_prozent
    # um den berechneten Wert
    dV_prozent.append(prozent)

# Ergebnis anzeigen
dV_prozent
```

    [20.454545454545457,
     11.363636363636363,
     11.363636363636363,
     11.363636363636363,
     45.45454545454545]

Häufig ist die Berechnungsvorschrift sehr einfach. Dann ist es einfacher, für die gleiche Berechnung eine *List Comprehension* zu verwenden:

`neue_liste = [Berechnungs_vorschrift(wert) for wert in alte_liste]`

Solche List Comprehensions sind oft leicht zu lesen. Sie können meist deutlich schneller berechnet werden, als eine `for`-Schleife.

```python
dV_prozent = [dV/dV_ges_ab * 100 for dV in dV_ab]
dV_prozent # prozentuale Verteilung
```

    [20.454545454545457,
     11.363636363636363,
     11.363636363636363,
     11.363636363636363,
     45.45454545454545]

Die Zahlenwerte der Liste `dV_prozent` müssen sich zu 100% aufsummieren:

```python
sum(dV_prozent) # Gesamter Abluftvolumenstrom in Prozent
```

    100.0

Zur Anzeige wird oft der sogenannte Reißverschluss (englisch zipper) verwendet. Dazu dient die Funktion

`zip(liste_1, liste_2, ...)`

```python
list(zip(raum,dV_prozent)) # prozentuale Verteilung der Abluft auf die Räume
```

    [('Bad', 20.454545454545457),
     ('WC_1', 11.363636363636363),
     ('WC_2', 11.363636363636363),
     ('Abstellkammer', 11.363636363636363),
     ('Saunaraum', 45.45454545454545)]

# Aufgabe

Berechnen Sie die Liste der erforderlichen Anschlussdurchmesser `d_erf # in mm` nach der Formel aus den gegebenen Abluftvolumenströmen in der Liste `dV_ab # in m**3/h`.

$$
  d = \sqrt{\dfrac{4\,\dot V}{\pi\,v}}
$$

für jeden Abluftraum. Gehen Sie dabei von einer zulässigen Strömungsgeschwindigkeit `v=2 # m/s` in den Lüftungsrohren aus.

Achtung: Vergessen Sie die Einheitenumrechnung von $1h = 3600\,s$ und $1m=1000\,mm$ nicht!

```python
import math
v = 2 # m/s
```

```python
# Ihre Lösung beginnt hier
d_erf = [math.sqrt(4*dV/(3600*math.pi*v))*1000 for dV in dV_ab]
d_erf # erforderliche Durchmesser in mm
```

    [89.20620580763855,
     66.49038006690546,
     66.49038006690546,
     66.49038006690546,
     132.98076013381092]

# Aufgabe

Sie können selbstgeschriebene Funktionen verwenden, um den Aufbau einer List Comprehension einfach zu halten.

Im folgenden habe ich im Modul `utilities.py` eine Funktion `ermittle_normdurchmesser(d)` angegeben, die zu einem vorgegebenen Durchmesser `d` in mm den nächstgrößeren Normdurchmesser ermittelt. Dabei ist die Liste der Normdurchmesser von einem Hersteller übernommen worden, siehe

https://www.msl-bauartikel.de/fileadmin/user_upload/produktbilder/Rohrsysteme/Lueftungsrohr/System_Lueftungsrohrde_eng_.pdf

Damit ergibt sich z.B. für `d=87 # mm`

`ermittle_normdurchmesser(d)`

`90`

Benutzen Sie die Durchmesser, die Sie in der vorigen Aufgabe berechnet haben, um die zugehörigen Normdurchmesser zu ermitteln. Verwenden Sie für die Berechnung eine List Comprehension.

```python
from utilities import ermittle_normdurchmesser

# Anwendungsbeispiel
ermittle_normdurchmesser(87)
```

    90

```python
# Ihre Lösung beginnt hier.
d_gew = [ermittle_normdurchmesser(d) for d in d_erf]
list(zip(raum,d_erf,d_gew))
```

    [('Bad', 89.20620580763855, 90),
     ('WC_1', 66.49038006690546, 80),
     ('WC_2', 66.49038006690546, 80),
     ('Abstellkammer', 66.49038006690546, 80),
     ('Saunaraum', 132.98076013381092, 140)]

# Beispiel

Die Liste der Normdurchmesser und der Wandstärken von Lüftungsrohren läßt sich folgendermaßen angeben:

```python
normdurchmesser = [
    80,90,100,112,125,140,
    150,160,180,200,224,250,
    280,300,315,355,400,450,
    500,560,600,630,710,800,
    900,1000,1120,1250,1400,
    1600,1800,2000
]

# Um hier nicht 32 Werte von Hand eintragen zu müssen:
wandstaerken=2*[0.4]+12*[0.6]+7*[0.8]+4*[1.0]+3*[1.2]+4*[1.5]
```

```python
# Das Ergebnis können Sie sich anzeigen lassen,
# wenn Sie vor der nächsten Zeile das Kommentarzeichen entfernen
list(zip(normdurchmesser,wandstaerken))
```

    [(80, 0.4),
     (90, 0.4),
     (100, 0.6),
     (112, 0.6),
     (125, 0.6),
     (140, 0.6),
     (150, 0.6),
     (160, 0.6),
     (180, 0.6),
     (200, 0.6),
     (224, 0.6),
     (250, 0.6),
     (280, 0.6),
     (300, 0.6),
     (315, 0.8),
     (355, 0.8),
     (400, 0.8),
     (450, 0.8),
     (500, 0.8),
     (560, 0.8),
     (600, 0.8),
     (630, 1.0),
     (710, 1.0),
     (800, 1.0),
     (900, 1.0),
     (1000, 1.2),
     (1120, 1.2),
     (1250, 1.2),
     (1400, 1.5),
     (1600, 1.5),
     (1800, 1.5),
     (2000, 1.5)]

Berechnen Sie das Gewicht je Meter Lüftungsrohr aus Wickelfalzrohr. Die Dichte von Stahl ist
$\varrho=7.85\,\frac{kg}{dm^3}$

Die Masse berechnet sich nach der Formel

$$
  m = \varrho\,V
$$

mit dem Volumen

\begin{align}
  V &= \dfrac{\pi \left(d_a^2 - d_i^2\right)}{4}\,\ell \\[2ex]
    &= \pi\,d_m\,s\,\ell
\end{align}

Verwenden Sie eine `for`-Schleife für die Berechnung.

```python
# Ihre Lösung beginnt hier
l = 1e1 # Länge l in dm
rho = 7.85 # kg/dm**3

m_je_meter = []

for d_a,s in zip(normdurchmesser,wandstaerken):
    d_a = d_a*1e-2 # Umrechnung von mm in dm
    s = s*1e-2 #
    d_i = d_a-2*s
    m = rho*math.pi*(d_a**2 - d_i**2)/4*l # Masse in kg
    m_je_meter.append(m)

# Ergebnisse
list(zip(normdurchmesser,wandstaerken,m_je_meter))
```

    [(80, 0.4, 0.7852222342088522),
     (90, 0.4, 0.8838682435315676),
     (100, 0.6, 1.4708119990017468),
     (112, 0.6, 1.6483748157826348),
     (125, 0.6, 1.840734533961952),
     (140, 0.6, 2.062688054938074),
     (150, 0.6, 2.2106570689221368),
     (160, 0.6, 2.3586260829062273),
     (180, 0.6, 2.6545641108743805),
     (200, 0.6, 2.950502138842534),
     (224, 0.6, 3.305627772404323),
     (250, 0.6, 3.6903472087629305),
     (280, 0.6, 4.134254250715174),
     (300, 0.6, 4.4301922786833545),
     (315, 0.8, 6.19891522583965),
     (355, 0.8, 6.988083300421538),
     (400, 0.8, 7.875897384325915),
     (450, 0.8, 8.862357477553221),
     (500, 0.8, 9.848817570780307),
     (560, 0.8, 11.03256968265303),
     (600, 0.8, 11.821737757234699),
     (630, 1.0, 15.51208496599741),
     (710, 1.0, 17.48500515245202),
     (800, 1.0, 19.704540362212857),
     (900, 1.0, 22.170690595280355),
     (1000, 1.2, 29.558290233458454),
     (1120, 1.2, 33.10954656907684),
     (1250, 1.2, 36.95674093266182),
     (1400, 1.5, 51.73366651418207),
     (1600, 1.5, 59.13211721338631),
     (1800, 1.5, 66.53056791259405),
     (2000, 1.5, 73.92901861179831)]


```
%pylab inline
from sympy.interactive import init_printing
init_printing()
from sympy import pi, var, S
from sympy.utilities.codegen import codegen
```

    Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.zmq.pylab.backend_inline].
    For more information, type 'help(pylab)'.

The following code is taken from the article:

Čertík, O., Winkler, P. (2013). Computation of screened two-electron matrix elements. International Journal of Quantum Chemistry. doi:10.1002/qua.24431

```
from sympy import var, legendre, Integral, \
    exp, latex
var("l R alpha t")
f = (2*l+1) / (2*t) * Integral(legendre(l, \
    (1-R**2+t**2) / (2*t)) * \
    exp(-alpha*R), (R, 1-t, 1+t))
for _l in range(3):
    expr = f.subs(l, _l).doit().simplify( ) \
        / exp(-alpha)
    expr = expr.series(alpha, 0, 14)
    print " s_%d =" % _l, latex(expr)
```

     s_0 = 1 + \frac{1}{6} \alpha^{2} t^{2} + \frac{1}{120} \alpha^{4} t^{4} + \frac{1}{5040} \alpha^{6} t^{6} + \frac{1}{362880} \alpha^{8} t^{8} + \frac{1}{39916800} \alpha^{10} t^{10} + \frac{1}{6227020800} \alpha^{12} t^{12} + \mathcal{O}\left(\alpha^{14}\right)
     s_1 = t + \alpha t + \frac{1}{10} \alpha^{2} t^{3} + \frac{1}{10} \alpha^{3} t^{3} + \frac{1}{280} \alpha^{4} t^{5} + \frac{1}{280} \alpha^{5} t^{5} + \frac{1}{15120} \alpha^{6} t^{7} + \frac{1}{15120} \alpha^{7} t^{7} + \frac{1}{1330560} \alpha^{8} t^{9} + \frac{1}{1330560} \alpha^{9} t^{9} + \frac{1}{172972800} \alpha^{10} t^{11} + \frac{1}{172972800} \alpha^{11} t^{11} + \frac{1}{31135104000} \alpha^{12} t^{13} + \frac{1}{31135104000} \alpha^{13} t^{13} + \mathcal{O}\left(\alpha^{14}\right)
     s_2 = t^{2} + \alpha t^{2} + \alpha^{2} \left(\frac{1}{14} t^{4} + \frac{1}{3} t^{2}\right) + \frac{1}{14} \alpha^{3} t^{4} + \alpha^{4} \left(\frac{1}{504} t^{6} + \frac{1}{42} t^{4}\right) + \frac{1}{504} \alpha^{5} t^{6} + \alpha^{6} \left(\frac{1}{33264} t^{8} + \frac{1}{1512} t^{6}\right) + \frac{1}{33264} \alpha^{7} t^{8} + \alpha^{8} \left(\frac{1}{3459456} t^{10} + \frac{1}{99792} t^{8}\right) + \frac{1}{3459456} \alpha^{9} t^{10} + \alpha^{10} \left(\frac{1}{518918400} t^{12} + \frac{1}{10378368} t^{10}\right) + \frac{1}{518918400} \alpha^{11} t^{12} + \alpha^{12} \left(\frac{1}{105859353600} t^{14} + \frac{1}{1556755200} t^{12}\right) + \frac{1}{105859353600} \alpha^{13} t^{14} + \mathcal{O}\left(\alpha^{14}\right)

```
f
```

```
expr
```

```
[(fname, fcode), (hfile, hcode)] = codegen(("f", expr), "F95", "test", header=False, empty=False)
```

```
fcode
```

                           REAL*8 function f(alpha, t)
                                  implicit none
                           REAL*8, intent(in) :: alpha
                             REAL*8, intent(in) :: t
                              REAL*8 :: O(alpha**14)
    f = t**2 + alpha*t**2 + alpha**2*(t**4/14 + t**2/3) + alpha**3*t**4/14 + &
          alpha**4*(t**6/504 + t**4/42) + alpha**5*t**6/504 + alpha**6*(t** &
            8/33264 + t**6/1512) + alpha**7*t**8/33264 + alpha**8*(t**10/ &
          3459456 + t**8/99792) + alpha**9*t**10/3459456 + alpha**10*(t**12 &
           /518918400 + t**10/10378368) + alpha**11*t**12/518918400 + alpha &
           **12*(t**14/105859353600 + t**12/1556755200) + alpha**13*t**14/ &
                              105859353600 + O(alpha**14)
                                   end function

```

```


# Selección óptima de portafolios II

Entonces, tenemos que:
- La LAC describe las posibles selecciones de riesgo-rendimiento entre un activo libre de riesgo y un activo riesgoso.
- Su pendiente es igual al radio de Sharpe del activo riesgoso.
- La asignación óptima de capital para cualquier inversionista es el punto tangente de la curva de indiferencia del inversionista con la LAC.

Para todo lo anterior, supusimos que ya teníamos el portafolio óptimo (activo riesgoso).

En la clase pasada aprendimos a hallar este portafolio óptimo si el conjunto de activos riesgosos estaba conformado únicamente por dos activos:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

- Sin embargo, la complejidad del problema crece considerablemente con el número de variables, y la solución analítica deja de ser viable cuando mencionamos que un portafolio bien diversificado consta aproximadamente de 50-60 activos.
- En esos casos, este problema se soluciona con rutinas numéricas que hagan la optimización por nosotros, porque son una solución viable y escalable a más variables.

**Objetivos:**
- ¿Cuál es el portafolio óptimo de activos riesgosos cuando tenemos más de dos activos?
 - ¿Cómo construir la frontera de mínima varianza cuando tenemos más de dos activos?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Maximizando el radio de Sharpe

### ¿Qué pasa si tenemos más de dos activos riesgosos?

En realidad es algo muy similar a lo que teníamos con dos activos.
- Para dos activos, construir la frontera de mínima varianza es trivial: todas las posibles combinaciones.
- Con más de dos activos, recordar la definición: la frontera de mínima varianza es el lugar geométrico de los portafolios que proveen el mínimo riesgo para un nivel de rendimiento dado.

 Ver en el tablero.

Analíticamente:
- $n$ activos,
- caracterizados por $(\sigma_i,E[r_i])$,
- cada uno con peso $w_i$, con $i=1,2,\dots,n$.

Entonces, buscamos los pesos tales que

\begin{align}
\min_{w_1,\dots,w_n} & \quad \sum_{i=1}^{n}w_i^2\sigma_i^2+\sum_{i=1}^{n}\sum_{j=1,j\neq i}^{n}w_iw_j\sigma_{ij}\\
\text{s.a.}          & \quad \sum_{i=1}^{n}w_i=1, w_i\geq0\\
                     & \quad \sum_{i=1}^{n}w_iE[r_i]=\bar{\mu},
\end{align}

donde $\bar{\mu}$ corresponde a un nivel de rendimiento objetivo.

**Obviamente, tendríamos que resolver este problema para muchos niveles de rendimiento objetivo.**

-  Explicar relación con gráfica.

-  Recordar clase 10.

Lo anterior se puede escribir vectorialmente como:

\begin{align}
\min_{\boldsymbol{w}} & \quad \boldsymbol{w}^T\Sigma\boldsymbol{w}\\
\text{s.a.}          & \quad \boldsymbol{1}^T\boldsymbol{w}=1, \boldsymbol{w}\geq0\\
                     & \quad E[\boldsymbol{r}^T]\boldsymbol{w}=\bar{\mu},
\end{align}

donde:
- $\boldsymbol{w}=\left[w_1,\dots,w_n\right]^T$ es el vector de pesos,
- $\boldsymbol{1}=\left[1,\dots,1\right]^T$ es un vector de unos,
- $E[\boldsymbol{r}]=\left[E[r_1],\dots,E[r_n]\right]^T$ es el vector de rendimientos esperados, y
- $\Sigma=\left[\begin{array}{cccc}\sigma_{1}^2 & \sigma_{12}  & \dots  & \sigma_{1n} \\
                                   \sigma_{21}  & \sigma_{2}^2 & \dots  & \sigma_{2n} \\
                                   \vdots       & \vdots       & \ddots & \vdots      \\
                                   \sigma_{n1}  & \sigma_{n2}  & \dots  & \sigma_{n}^2\end{array}\right]$ es la matriz de varianza-covarianza.

**Esta última forma es la que comúnmente usamos al programar, por ser eficiente y escalable a problemas de N variables.**

### Entonces, ¿para cuántos niveles de rendimiento objetivo tendríamos que resolver el anterior problema con el fin de graficar la frontera de mínima varianza?
- Observar que el problema puede volverse muy pesado a medida que incrementamos el número de activos en nuestro portafolio...
- Una tarea bastante compleja.

### Sucede que, en realidad, sólo necesitamos conocer dos portafolios que estén sobre la *frontera de mínima varianza*.
- Si logramos encontrar dos portafolios sobre la frontera, entonces podemos a la vez encontrar todas las posibles combinaciones de estos dos portafolios para trazar la frontera de mínima varianza.
- Ver el caso de dos activos.

### ¿Qué portafolios usar?
Hasta ahora, hemos estudiando profundamente como hallar dos portafolios muy importantes que de hecho yacen sobre la frontera de mínima varianza:
1. Portafolio de EMV: máximo SR.
2. Portafolio de mínima varianza: básicamente, el mismo problema anterior, sin la restricción de rendimiento objetivo.

Luego, tomar todas las posibles combinaciones de dichos portafolios usando las fórmulas para dos activos de medias y varianzas:
- w: peso para el portafolio EMV,
- 1-w: peso para le portafolio de mínima varianza.

## 2. Ejemplo ilustrativo.

Retomamos el ejemplo de mercados de acciones en los países integrantes del $G5$: EU, RU, Francia, Alemania y Japón.

```python
# Importamos pandas y numpy
import pandas as pd
import numpy as np
```

```python
# Resumen en base anual de rendimientos esperados y volatilidades
annual_ret_summ = pd.DataFrame(columns=['EU', 'RU', 'Francia', 'Alemania', 'Japon'], index=['Media', 'Volatilidad'])
annual_ret_summ.loc['Media'] = np.array([0.1355, 0.1589, 0.1519, 0.1435, 0.1497])
annual_ret_summ.loc['Volatilidad'] = np.array([0.1535, 0.2430, 0.2324, 0.2038, 0.2298])

annual_ret_summ.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      Media
      0.1355
      0.1589
      0.1519
      0.1435
      0.1497

      Volatilidad
      0.1535
      0.243
      0.2324
      0.2038
      0.2298

```python
# Matriz de correlación
corr = pd.DataFrame(data= np.array([[1.0000, 0.5003, 0.4398, 0.3681, 0.2663],
                                    [0.5003, 1.0000, 0.5420, 0.4265, 0.3581],
                                    [0.4398, 0.5420, 1.0000, 0.6032, 0.3923],
                                    [0.3681, 0.4265, 0.6032, 1.0000, 0.3663],
                                    [0.2663, 0.3581, 0.3923, 0.3663, 1.0000]]),
                    columns=annual_ret_summ.columns, index=annual_ret_summ.columns)
corr.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      EU
      1.0000
      0.5003
      0.4398
      0.3681
      0.2663

      RU
      0.5003
      1.0000
      0.5420
      0.4265
      0.3581

      Francia
      0.4398
      0.5420
      1.0000
      0.6032
      0.3923

      Alemania
      0.3681
      0.4265
      0.6032
      1.0000
      0.3663

      Japon
      0.2663
      0.3581
      0.3923
      0.3663
      1.0000

```python
# Tasa libre de riesgo
rf = 0.05
```

Esta vez, supondremos que tenemos disponibles todos los mercados de acciones y el activo libre de riesgo.

#### 1. Construir la frontera de mínima varianza

##### 1.1. Encontrar portafolio de mínima varianza

```python
# Importamos funcion minimize del modulo optimize de scipy
from scipy.optimize import minimize
```

```python
np.diag(annual_ret_summ.loc['Volatilidad'])
```

    array([[0.1535, 0, 0, 0, 0],
           [0, 0.243, 0, 0, 0],
           [0, 0, 0.2324, 0, 0],
           [0, 0, 0, 0.2038, 0],
           [0, 0, 0, 0, 0.2298]], dtype=object)

```python
## Construcción de parámetros
# 1. Sigma: matriz de varianza-covarianza Sigma = S.dot(corr).dot(S)
S = np.diag(annual_ret_summ.loc['Volatilidad'])
Sigma = S.dot(corr).dot(S)
# 2. Eind: rendimientos esperados activos individuales
Eind = annual_ret_summ.loc['Media']
```

```python
# Función objetivo
def varianza(w, Sigma):
    return w.T.dot(Sigma).dot(w)
```

```python
n = len(Eind)
# Dato inicial
# w0 = np.array([.5,0,.25,0,.25])
w0 = np.ones((n, )) / n
# Cotas de las variables
bnds = ((0,1), )*n
# Restricciones
cons = {'type': 'eq','fun':lambda w: w.sum()-1}
```

```python
# Portafolio de mínima varianza
min_var = minimize(fun=varianza,
                  x0 = w0,
                  args = (Sigma,),
                  bounds = bnds,
                  constraints = cons)
min_var
```

         fun: 0.01861776391061502
         jac: array([0.03718246, 0.03881475, 0.03859101, 0.03755156, 0.0370423 ])
     message: 'Optimization terminated successfully.'
        nfev: 49
         nit: 7
        njev: 7
      status: 0
     success: True
           x: array([6.17797049e-01, 3.46944695e-18, 0.00000000e+00, 2.09394358e-01,
           1.72808594e-01])

```python
# Pesos, rendimiento y riesgo del portafolio de mínima varianza
w_minvar = min_var.x
E_minvar = Eind.T.dot(w_minvar)
s_minvar = varianza(w_minvar,Sigma)**.5
RS_minvar = (E_minvar - rf) / s_minvar

w_minvar, E_minvar, s_minvar, RS_minvar
```

    (array([6.17797049e-01, 3.46944695e-18, 0.00000000e+00, 2.09394358e-01,
            1.72808594e-01]),
     0.13962903688859557,
     0.13644692708381168,
     0.6568783834431207)

##### 1.2. Encontrar portafolio EMV

```python
# Función objetivo
def m_RS(w, Sigma, rf, Eind):
    Ep = Eind.T.dot(w)
    sp = varianza(w,Sigma)**.5
    RS = (Ep - rf) / sp
    return -RS
```

```python
n = len(Eind)
# Dato inicial
# w0 = np.array([.5,0,.25,0,.25])
w0 = np.ones((n, )) / n
# Cotas de las variables
bnds = ((0,1), )*n
# Restricciones
cons = {'type': 'eq','fun':lambda w: w.sum()-1}
```

```python
# Portafolio EMV
EMV = minimize(fun=m_RS,
                  x0 = w0,
                  args = (Sigma,rf,Eind),
                  bounds = bnds,
                  constraints = cons)
EMV
```

         fun: -0.6644372965632436
         jac: array([-0.3608895 , -0.36076408, -0.36036385, -0.36108153, -0.36062376])
     message: 'Optimization terminated successfully.'
        nfev: 35
         nit: 5
        njev: 5
      status: 0
     success: True
           x: array([0.50714174, 0.07470888, 0.02471533, 0.18943972, 0.20399434])

```python
# Pesos, rendimiento y riesgo del portafolio EMV
w_EMV = EMV.x
E_EMV = Eind.T.dot(w_EMV)
s_EMV = varianza(w_EMV,Sigma)**.5
RS_EMV = (E_EMV - rf) / s_EMV

w_EMV, E_EMV, s_EMV, RS_EMV
```

    (array([0.50714174, 0.07470888, 0.02471533, 0.18943972, 0.20399434]),
     0.1420657564717856,
     0.13856199365687238,
     0.6644372965632436)

```python
RS_EMV, RS_minvar
```

    (0.6644372965632436, 0.6568783834431207)

```python
s_EMV, s_minvar
```

    (0.13856199365687238, 0.13644692708381168)

##### 1.3. Construir frontera de mínima varianza

También debemos encontrar la covarianza (o correlación) entre estos dos portafolios:

```python
# Covarianza entre los portafolios
cov_EMV_minvar = w_EMV.T.dot(Sigma).dot(w_minvar)
cov_EMV_minvar
```

    0.018690275386034134

```python
# Correlación entre los portafolios
corr_EMV_minvar = cov_EMV_minvar / (s_EMV * s_minvar)
corr_EMV_minvar
```

    0.9885708894197612

```python
# Vector de w
wp = np.linspace(0,10,100)
```

```python
# DataFrame de portafolios:
# 1. Índice: i
# 2. Columnas 1-2: w, 1-w
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
front_minvar = pd.DataFrame({'w(EMV)':wp,
                            'w(minvar)':1-wp,
                            'Media': E_EMV*wp + E_minvar*(1-wp),
                            'Vol': ((wp*s_EMV)**2 + ((1-wp)*s_minvar)**2 + 2*wp*(1-wp)*cov_EMV_minvar)**.5})
front_minvar['RS'] = (front_minvar['Media']-rf)/front_minvar['Vol']
front_minvar
```

      w(EMV)
      w(minvar)
      Media
      Vol
      RS

      0
      0.00000
      1.00000
      0.139629
      0.136447
      0.656878

      1
      0.10101
      0.89899
      0.139875
      0.136517
      0.658345

      2
      0.20202
      0.79798
      0.140121
      0.136619
      0.659652

      3
      0.30303
      0.69697
      0.140367
      0.136755
      0.660800

      4
      0.40404
      0.59596
      0.140614
      0.136922
      0.661790

      ...
      ...
      ...
      ...
      ...
      ...

      95
      9.59596
      -8.59596
      0.163012
      0.245390
      0.460539

      96
      9.69697
      -8.69697
      0.163258
      0.247147
      0.458261

      97
      9.79798
      -8.79798
      0.163504
      0.248910
      0.456004

      98
      9.89899
      -8.89899
      0.163750
      0.250678
      0.453769

      99
      10.00000
      -9.00000
      0.163996
      0.252452
      0.451556

100 rows × 5 columns

```python
# Importar librerías de gráficos
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, los activos individuales
# y los portafolios hallados
plt.figure(figsize=(12,8))
plt.scatter(front_minvar['Vol'],front_minvar['Media'],c=front_minvar['RS'],cmap='inferno', label='Front. Minvar')
for i in range(n):
    plt.plot(annual_ret_summ.iloc[1,i],annual_ret_summ.iloc[0,i],'o', ms=10, label=annual_ret_summ.columns[i])

plt.plot(s_minvar, E_minvar, '*g', ms=10, label = 'Port. Minvar')
plt.plot(s_EMV, E_EMV, '*r', ms=10, label = 'Port. EMV')
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento Esperado $E[r]$')
plt.colorbar()
plt.legend(loc='best')
plt.grid()
```

**A partir de lo anterior, solo restaría construir la LAC y elegir la distribución de capital de acuerdo a las preferencias (aversión al riesgo).**
___

## 3. Comentarios finales

### 3.1. Restricciones adicionales

Los inversionistas pueden tener restricciones adicionales:
1. Restricciones en posiciones cortas.
2. Pueden requerir un rendimiento mínimo.
3. Inversión socialmente responsable: prescinden de inversiones en negocios o paises considerados éticamente o políticamente indeseables.

Todo lo anterior se puede incluir como restricciones en el problema de optimización, y puede ser llevado a cabo a costa de un cociente de Sharpe menor.

### 3.2. Críticas a la optimización media varianza
1. Solo importan medias y varianzas: recordar que la varianza subestima el riesgo en algunos casos.
2. Preferencias media-varianza tratan las ganancias y pérdidas simétricamente: el sentimiento de insatisfacción de una perdida es mayor al sentimiento de satisfacción de una ganancia (aversión a pérdidas).
3. La aversión al riesgo es constante: la actitud frente al riesgo puede cambiar, por ejemplo con el estado de la economía.
4. Horizonte corto (un periodo).
5. Basura entra - basura sale: la optimización media varianza es supremamente sensible a las entradas: estimaciones de rendimientos esperados y varianzas.
___

# Anuncios parroquiales

## 1. Quiz la próxima clase (clases 12, 13, y 14).
## 2. Revisar archivo Tarea 6. Definir fecha.
## 3. Fin de módulo. Abrir Clase0 y revisar objetivos del módulo.
## 4. [Nota interesante](http://yetanothermathprogrammingconsultant.blogspot.com/2016/08/portfolio-optimization-maximize-sharpe.html)

Created with Jupyter by Esteban Jiménez Rodríguez.


# Density Fitting

Density fitting is an extremely useful tool to reduce the computational scaling of many quantum chemical methods.
Density fitting works by approximating the four-index electron repulsion integral (ERI) tensors from Hartree-Fock
theory, $g_{\mu\nu\lambda\sigma} = (\mu\nu|\lambda\sigma)$, by

$$(\mu\nu|\lambda\sigma) \approx \widetilde{(\mu\nu|P)}[J^{-1}]_{PQ}\widetilde{(Q|\lambda\sigma)}$$

where the Coulomb metric $[J]_{PQ}$ and the three-index integral $\widetilde{(Q|\lambda\sigma)}$ are defined as

\begin{align}
[J]_{PQ} &= \int P(r_1)\frac{1}{r_{12}}Q(r_2){\rm d}^3r_1{\rm d}^3r_2\\
\widetilde{(Q|\lambda\sigma)} &= \int Q(r_1)\frac{1}{r_{12}}\lambda(r_2)\sigma(r_2){\rm d}^3r_1{\rm d}^3r_2
\end{align}

To simplify the density fitting notation, the inverse Coulomb metric is typically folded into the three-index tensor:

\begin{align}
(P|\lambda\sigma) &= [J^{-\frac{1}{2}}]_{PQ}\widetilde{(Q|\lambda\sigma)}\\
g_{\mu\nu\lambda\sigma} &\approx (\mu\nu|P)(P|\lambda\sigma)
\end{align}

These transformed three-index tensors can then be used to compute various quantities, including the four-index ERIs,
as well as Coulomb (J) and exchange (K) matrices, and therefore the Fock matrix (F).  Before we go any further, let's
see how to generate these transformed tensors using  Psi4.

First, let's import  Psi4 and set up some global options, as well as
define a molecule and initial wavefunction:

```python
# ==> Psi4 & NumPy options, Geometry Definition <==
import numpy as np
import psi4

# Set numpy defaults
np.set_printoptions(precision=5, linewidth=200, suppress=True)

# Set Psi4 memory & output options
psi4.set_memory(int(2e9))
psi4.core.set_output_file('output.dat', False)

# Geometry specification
mol = psi4.geometry("""
O
H 1 0.96
H 1 0.96 2 104.5
symmetry c1
""")

# Psi4 options
psi4.set_options({'basis': 'aug-cc-pvdz',
                  'scf_type': 'df',
                  'e_convergence': 1e-10,
                  'd_convergence': 1e-10})

wfn = psi4.core.Wavefunction.build(mol, psi4.core.get_global_option('basis'))
```

## Building the Auxiliary Basis Set

One component of our density-fitted tensors $g_{\mu\nu\lambda\sigma} \approx (\mu\nu|P)(P|\lambda\sigma)$ which
is unique from their exact, canonical counterparts $(\mu\nu|\lambda\sigma)$ is the additional "auxiliary" index, $P$.
This index corresponds to inserting a resolution of the identity, which is expanded in an auxiliary basis set $\{P\}$.
In order to build our density-fitted integrals, we first need to generate this auxiliary basis set.  Fortunately,
we can do this with the `psi4.core.BasisSet` object:
~~~python
# Build auxiliary basis set
aux = psi4.core.BasisSet.build(mol, "DF_BASIS_SCF", "", "JKFIT", "aug-cc-pVDZ")
~~~

There are special fitting basis sets that are optimal for a given orbital basis. As we will be building J and K
objects we want the `JKFIT` basis associated with the orbital `aug-cc-pVDZ` basis. This basis is straightfowardly
named `aug-cc-pVDZ-jkfit`.

```python
# Build auxiliary basis set
aux = psi4.core.BasisSet.build(mol, "DF_BASIS_SCF", "", "JKFIT", "aug-cc-pVDZ")
```

## Building Density-Fitted ERIs
Now, we can use our orbital and auxiliary basis sets to construct the `Qpq` object with the inverted metric. As the
tensors are very similar to full ERI's we can use the same computer for both with the aid of a "zero basis". If we
think carefully about the $\widetilde{(Q|\lambda\sigma)}$ and $(\mu\nu|\lambda\sigma)$ we should note that on the
right and left hand sides the two gaussian basis functions are contracted to a single density.

Specifically, for $\widetilde{(Q|\lambda\sigma)}$ the right hand side is a single basis function without being
multiplied by another, so we can "trick" the MintsHelper object into computing these quanties if we have a "basis
set" which effectively does not act on another. This is, effectively, what a "zero basis" does.

The $[J^{-\frac{1}{2}}]_{PQ}$ object can be built in a similar way where we use the Psi4 Matrix's built in `power`
function to raise this to the $-\frac{1}{2}$ power. The call `Matrix.power(-0.5, 1.e-14)` will invert the Matrix to
the $-\frac{1}{2}$ while guarding against values smaller than 1.e-14. Recall that machine epsilon is ~1.e-16, when
these small values are taken to a negative fractional power they could become very large and dominate the resulting
matrix even though they are effectively noise before the inversion.

~~~python
orb = wfn.basisset()
zero_bas = psi4.core.BasisSet.zero_ao_basis_set()

# Build MintsHelper Instance
mints = psi4.core.MintsHelper(orb)

# Build (P|pq) raw 3-index ERIs, dimension (1, Naux, nbf, nbf)
Ppq = mints.ao_eri(aux, zero_bas, orb, orb)

# Build and invert the metric
metric = mints.ao_eri(aux, zero_bas, aux, zero_bas)
metric.power(-0.5, 1.e-14)

# Remove the excess dimensions of Ppq & metric
Ppq = np.squeeze(Ppq)
metric = np.squeeze(metric)

# Contract Ppq & Metric to build Qso
Qso = np.einsum('QP,Ppq->Qpq', metric, Ppq, optimize=True)
~~~

```python
# ==> Build Density-Fitted Integrals <==
# Get orbital basis & build zero basis
orb = wfn.basisset()
zero_bas = psi4.core.BasisSet.zero_ao_basis_set()

# Build instance of MintsHelper
mints = psi4.core.MintsHelper(orb)

# Build (P|pq) raw 3-index ERIs, dimension (1, Naux, nbf, nbf)
Ppq = mints.ao_eri(aux, zero_bas, orb, orb)

# Build & invert Coulomb metric, dimension (1, Naux, 1, Naux)
metric = mints.ao_eri(aux, zero_bas, aux, zero_bas)
metric.power(-0.5, 1.e-14)

# Remove excess dimensions of Ppq, & metric
Ppq = np.squeeze(Ppq)
metric = np.squeeze(metric)

# Build the Qso object
Qpq = np.einsum('QP,Ppq->Qpq', metric, Ppq, optimize=True)
```

## Example: Building a Density-Fitted Fock Matrix
Now that we've obtained our `Qpq` tensors, we may use them to build the Fock matrix.  To do so, since we aren't
implementing a fully density-fitted RHF program, we'll first need to get a density matrix and one-electron Hamiltonian
from somewhere. Let's get them from a converged HF wavefunction, so we can check our work later:

```python
# ==> Compute SCF Wavefunction, Density Matrix, & 1-electron H <==
scf_e, scf_wfn = psi4.energy('scf', return_wfn=True)
D = scf_wfn.Da()
H = scf_wfn.H()
```

Now that we have our density-fitted integrals and a density matrix, we can build a Fock matrix.  There are several
different algorithms which we can successfully use to do so; for now, we'll use a simple algorithm and `np.einsum()`
to illustrate how to perform contractions with these density fitted tensors and leave a detailed discussion of those
algorithms/different tensor contraction methods elsewhere.  Recall that the Fock matrix, $F$, is given by

$$F = H + 2J - K,$$

where $H$ is the one-electron Hamiltonian matrix, $J$ is the Coulomb matrix, and $K$ is the exchange matrix.  The
Coulomb and Exchange matrices have elements guven by

\begin{align}
J[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\nu|\lambda\sigma)D_{\lambda\sigma}\\
K[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\lambda|\nu\sigma)D_{\lambda\sigma}.
\end{align}

When employing conventional 4-index ERI tensors, computing both $J$ and $K$ involves contracting over four unique
indices, which involves four distinct loops -- one over each unique index in the contraction.  Therefore, the
scaling of this procedure is $\mathcal{O}(N^4)$, where $N$ is the number of iterations in each loop (one for each
basis function).  The above expressions can be coded using `np.einsum()` to handle the tensor contractions:

~~~python
J = np.einsum('pqrs,rs->pq', I_pqrs, D, optimize=True)
K = np.einsum('prqs,rs->pq', I_pqrs, D, optimize=True)
~~~

for exact ERIs `I_pqrs`.  If we employ density fitting, however, we can reduce this scaling by reducing the number
of unique indices involved in the contractions.  Substituting in the density-fitted $(P|\lambda\sigma)$ tensors into
the above expressions, we obtain the following:

\begin{align}
J[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\nu|P)(P|\lambda\sigma)D_{\lambda\sigma}\\
K[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\lambda|P)(P|\nu\sigma)D_{\lambda\sigma}.
\end{align}

Naively, this seems like we have actually *increased* the scaling of our algorithm, because we have added the $P$
index to the expression, bringing the total to five unique indices, meaning this would scale like .  We've actually
made our lives easier, however: with three different tensors to contract, we can perform one contraction at a time!

For $J$, this works out to the following two-step procedure:

\begin{align}
\chi_P &= (P|\lambda\sigma)D_{\lambda\sigma} \\
J[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\nu|P)\chi_P
\end{align}

In the cell below, using `np.einsum()` and our `Qpq` tensor, try to construct `J`:

```python
# Two-step build of J with Qpq and D
X_Q = np.einsum('Qpq,pq->Q', Qpq, D, optimize=True)
J = np.einsum('Qpq,Q->pq', Qpq, X_Q, optimize=True)
```

Each of the above contractions, first constructing the `X_Q` intermediate and finally the full Coulomb matrix `J`, only involve three unique indices.  Therefore, the Coulomb matrix build above scales as $\mathcal{O}(N_{\rm aux}N^2)$.  Notice that we have distinguished the number of auxiliary ($N_{\rm aux}$) and orbital ($N$) basis functions; this is because auxiliary basis sets are usually around double the size of their corresponding orbital counterparts.

We can play the same intermediate trick for building the Exchange matrix $K$:

\begin{align}
\zeta_{P\nu\lambda} &= (P|\nu\sigma)D_{\lambda\sigma} \\
K[D_{\lambda\sigma}]_{\mu\nu} &= (\mu\lambda|P)\zeta_{P\nu\lambda}
\end{align}

Just like with $J$, try building $K$ in the cell below:

```python
# Two-step build of K with Qpq and D
Z_Qqr = np.einsum('Qrs,sq->Qrq', Qpq, D, optimize=True)
K = np.einsum('Qpq,Qrq->pr', Qpq, Z_Qqr, optimize=True)
```

Unfortunately, our two-step $K$ build does not incur a reduction in the overall scaling of the algorithm, with each contraction above scaling as $\mathcal{O}(N^3N_{\rm aux})$. The major benefit of density fitting for $K$ builds comes in the form of the small storage overhead of the three-index `Qpq` tensors compared to the full four-index `I_pqrs` tensors.  Even when exploiting the full eight-fold symmetry of the $(\mu\nu|\lambda\sigma)$ integrals, storing `I_pqrs` for a system with 3000 AO basis functions will require 81 TB of space, compared to a mere 216 GB to store the full `Qpq` object when exploiting the twofold symmetry of $(P|\lambda\sigma)$.

Now that we've built density-fitted versions of the $J$ and $K$ matrices, let's check our work by comparing a Fock matrix built using our $J$ and $K$ with the fully converged Fock matrix from our original SCF/aug-cc-pVDZ computation.

Below, build F using the one-electron Hamiltonian from the converged SCF wavefuntion and our $J$ and $K$ matrices.  Then, get the converged $F$ from the SCF wavefunction:

```python
# Build F from SCF 1 e- Hamiltonian and our density-fitted J & K
F = H + 2 * J - K
# Get converged Fock matrix from converged SCF wavefunction
scf_F = scf_wfn.Fa()
```

Feeling lucky? Execute the next cell to see if you've computed $J$, $K$, and $F$ correctly:

```python
if np.allclose(F, scf_F):
    print("Nicely done!! Your density-fitted Fock matrix matches Psi4!")
else:
    print("Whoops...something went wrong.  Try again!")
```

    Nicely done!! Your density-fitted Fock matrix matches Psi4!

Finally, we can remember the identity of the $D$ matrix for SCF which will be $D_{\lambda\sigma} = C_{\lambda i} C_{\sigma i}$, where $i$ is the occupied index. We can factor our $K$ build once more:
\begin{align}
D_{\lambda\sigma} &= C_{\lambda i} C_{\sigma i} \\
\zeta_{P\nu i} &= (P|\nu\sigma)C_{\sigma i} \\
K[D_{\lambda\sigma}]_{\mu\nu} &= \zeta_{P\nu i}\zeta_{P\mu i}
\end{align}

Consider the ratio between the number of basis functions and the size of the occupied index. Why would the above be beneficial?

## References
1. F. Weigend, Phys. Chem. Chem. Phys. 4, 4285 (2002).
2. O. Vahtras, J. Alml ̈of, and M. W. Feyereisen, Chem. Phys. Lett. 213, 514 (1993).
3. B. I. Dunlap, J. W. D. Connolly, and J. R. Sabin, J. Chem. Phys. 71, 3396 (1979).
4. J. L. Whitten, J. Chem. Phys. 58, 4496 (1973).