# 02 - Reverse Time Migration

This notebook is the second in a series of tutorial highlighting various aspects of seismic inversion based on Devito operators. In this second example we aim to highlight the core ideas behind seismic inversion, where we create an image of the subsurface from field recorded data. This tutorial follows on the modelling tutorial and will reuse the modelling operator and velocity model.

## Imaging requirement

Seismic imaging relies on two known parameters:

- **Field data** - or also called **recorded data**. This is a shot record corresponding to the true velocity model. In practice this data is acquired as described in the first tutorial. In order to simplify this tutorial we will generate synthetic field data by modelling it with the **true velocity model**.

- **Background velocity model**. This is a velocity model that has been obtained by processing and inverting the field data. We will look at this methods in the following tutorial as it relies on the method we are describing here. This velocity model is usually a **smooth version** of the true velocity model.

## Imaging computational setup

In this tutorial, we will introduce the back-propagation operator. This operator simulates the adjoint wave-equation, that is a wave-equation solved in a reversed time order. This time reversal led to the naming of the method we present here, called Reverse Time Migration. The notion of adjoint in exploration geophysics is fundamental as most of the wave-equation based imaging and inversion methods rely on adjoint based optimization methods.

## Notes on the operators

As we have already described the creation of a forward modelling operator, we will use a thin wrapper function instead. This wrapper is provided by a utility class called `AcousticWaveSolver`, which provides all the necessary operators for seismic modeling, imaging and inversion. The `AcousticWaveSolver` provides a more concise API for common wave propagation operators and caches the Devito `Operator` objects to avoid unnecessary recompilation. Operators introduced for the first time in this tutorial will be properly described.

As before we initialize printing and import some utilities. We also raise the Devito log level to avoid excessive logging for repeated operator invocations.

```python
import numpy as np
%matplotlib inline

from devito import configuration
configuration['log-level'] = 'WARNING'
```

## Computational considerations

Seismic inversion algorithms are generally very computationally demanding and require a large amount of memory to store the forward wavefield. In order to keep this tutorial as lightweight as possible we are using a very simple
velocity model that requires low temporal and spatial resolution. For a more realistic model, a second set of preset parameters for a reduced version of the 2D Marmousi data set [1] is provided below in comments. This can be run to create some more realistic subsurface images. However, this second preset is more computationally demanding and requires a slightly more powerful workstation.

```python
# Configure model presets
from examples.seismic import demo_model

# Enable model presets here:
preset = 'twolayer-isotropic'  # A simple but cheap model (recommended)
# preset = 'marmousi2d-isotropic'  # A larger more realistic model

# Standard preset with a simple two-layer model
if preset == 'twolayer-isotropic':
    def create_model(grid=None):
        return demo_model('twolayer-isotropic', origin=(0., 0.), shape=(101, 101),
                          spacing=(10., 10.), nbl=20, grid=grid, ratio=2)
    filter_sigma = (1, 1)
    nshots = 21
    nreceivers = 101
    t0 = 0.
    tn = 1000.  # Simulation last 1 second (1000 ms)
    f0 = 0.010  # Source peak frequency is 10Hz (0.010 kHz)

# A more computationally demanding preset based on the 2D Marmousi model
if preset == 'marmousi2d-isotropic':
    def create_model(grid=None):
        return demo_model('marmousi2d-isotropic', data_path='../../../../data/',
                          grid=grid, nbl=20)
    filter_sigma = (6, 6)
    nshots = 301  # Need good covergae in shots, one every two grid points
    nreceivers = 601  # One recevier every grid point
    t0 = 0.
    tn = 3500.  # Simulation last 3.5 second (3500 ms)
    f0 = 0.025  # Source peak frequency is 25Hz (0.025 kHz)
```

# True and smooth velocity models

First, we create the model data for the "true" model from a given demonstration preset. This model represents the subsurface topology for the purposes of this example and we will later use it to generate our synthetic data readings. We also generate a second model and apply a smoothing filter to it, which represents our initial model for the imaging algorithm. The perturbation between these two models can be thought of as the image we are trying to recover.

```python
#NBVAL_IGNORE_OUTPUT
from examples.seismic import plot_velocity, plot_perturbation
from scipy import ndimage

# Create true model from a preset
model = create_model()

# Create initial model and smooth the boundaries
model0 = create_model(grid=model.grid)
model0.vp = ndimage.gaussian_filter(model0.vp.data, sigma=filter_sigma, order=0)

# Plot the true and initial model and the perturbation between them
plot_velocity(model)
plot_velocity(model0)
plot_perturbation(model0, model)
```

## Acquisition geometry

Next we define the positioning and the wave signal of our source, as well as the location of our receivers. To generate the wavelet for our source we require the discretized values of time that we are going to use to model a single "shot",
which again depends on the grid spacing used in our model. For consistency this initial setup will look exactly as in the previous modelling tutorial, although we will vary the position of our source later on during the actual imaging algorithm.

```python
#NBVAL_IGNORE_OUTPUT
# Define acquisition geometry: source
from examples.seismic import AcquisitionGeometry

# First, position source centrally in all dimensions, then set depth
src_coordinates = np.empty((1, 2))
src_coordinates[0, :] = np.array(model.domain_size) * .5
src_coordinates[0, -1] = 20.  # Depth is 20m

# Define acquisition geometry: receivers

# Initialize receivers for synthetic and imaging data
rec_coordinates = np.empty((nreceivers, 2))
rec_coordinates[:, 0] = np.linspace(0, model.domain_size[0], num=nreceivers)
rec_coordinates[:, 1] = 30.

# Geometry

geometry = AcquisitionGeometry(model, rec_coordinates, src_coordinates, t0, tn, f0=.010, src_type='Ricker')
# We can plot the time signature to see the wavelet
geometry.src.show()
```

# True and smooth data

We can now generate the shot record (receiver readings) corresponding to our true and initial models. The difference between these two records will be the basis of the imaging procedure.

For this purpose we will use the same forward modelling operator that was introduced in the previous tutorial, provided by the `AcousticWaveSolver` utility class. This object instantiates a set of pre-defined operators according to an initial definition of the acquisition geometry, consisting of source and receiver symbols. The solver objects caches the individual operators and provides a slightly more high-level API that allows us to invoke the modelling modelling operators from the initial tutorial in a single line. In the following cells we use this to generate shot data by only specifying the respective model symbol `m` to use, and the solver will create and return a new `Receiver` object the represents the readings at the previously defined receiver coordinates.

```python
# Compute synthetic data with forward operator
from examples.seismic.acoustic import AcousticWaveSolver

solver = AcousticWaveSolver(model, geometry, space_order=4)
true_d , _, _ = solver.forward(vp=model.vp)
```

```python
# Compute initial data with forward operator
smooth_d, _, _ = solver.forward(vp=model0.vp)
```

```python
#NBVAL_IGNORE_OUTPUT
# Plot shot record for true and smooth velocity model and the difference
from examples.seismic import plot_shotrecord

plot_shotrecord(true_d.data, model, t0, tn)
plot_shotrecord(smooth_d.data, model, t0, tn)
plot_shotrecord(smooth_d.data - true_d.data, model, t0, tn)
```

# Imaging with back-propagation

As explained in the introduction of this tutorial, this method is based on back-propagation.

## Adjoint wave equation

If we go back to the modelling part, we can rewrite the simulation as a linear system solve:

\begin{equation}
\mathbf{A}(\mathbf{m}) \mathbf{u} = \mathbf{q}
\end{equation}

where $\mathbf{m}$ is the discretized square slowness, $\mathbf{q}$ is the discretized source and $\mathbf{A}(\mathbf{m})$ is the discretized wave-equation. The discretized wave-equation matricial representation is a lower triangular matrix that can be solve with forward substitution. The pointwise writing or the forward substitution leads to the time-stepping stencil.

On a small problem one could form the matrix explicitly and transpose it to obtain the adjoint discrete wave-equation:

\begin{equation}
\mathbf{A}(\mathbf{m})^T \mathbf{v} = \delta \mathbf{d}
\end{equation}

where $\mathbf{v}$ is the discrete **adjoint wavefield** and  $\delta \mathbf{d}$ is the data residual defined as the difference between the field/observed data and the synthetic data $\mathbf{d}_s = \mathbf{P}_r \mathbf{u}$. In our case we derive the discrete adjoint wave-equation from the discrete forward wave-equation to get its stencil.

## Imaging

Wave-equation based imaging relies on one simple concept:

- If the background velocity model is cinematically correct, the forward wavefield $\mathbf{u}$ and the adjoint wavefield $\mathbf{v}$ meet at the reflectors position at zero time offset.

The sum over time of the zero time-offset correlation of these two fields then creates an image of the subsurface. Mathematically this leads to the simple imaging condition:

\begin{equation}
  \text{Image} = \sum_{t=1}^{n_t} \mathbf{u}[t] \mathbf{v}[t]
\end{equation}

In the following tutorials we will describe a more advanced imaging condition that produces shaper and more accurate results.

## Operator

We will now define the imaging operator that computes the adjoint wavefield $\mathbf{v}$ and correlates it with the forward wavefield $\mathbf{u}$. This operator essentially consists of three components:
* Stencil update of the adjoint wavefield `v`
* Injection of the data residual at the adjoint source (forward receiver) location
* Correlation of `u` and `v` to compute the image contribution at each timestep

```python
# Define gradient operator for imaging
from devito import TimeFunction, Operator, Eq, solve
from examples.seismic import PointSource

def ImagingOperator(model, image):
    # Define the wavefield with the size of the model and the time dimension
    v = TimeFunction(name='v', grid=model.grid, time_order=2, space_order=4)

    u = TimeFunction(name='u', grid=model.grid, time_order=2, space_order=4,
                     save=geometry.nt)

    # Define the wave equation, but with a negated damping term
    eqn = model.m * v.dt2 - v.laplace - model.damp * v.dt

    # Use `solve` to rearrange the equation into a stencil expression
    stencil = Eq(v.backward, solve(eqn, v.backward))

    # Define residual injection at the location of the forward receivers
    dt = model.critical_dt
    residual = PointSource(name='residual', grid=model.grid,
                           time_range=geometry.time_axis,
                           coordinates=geometry.rec_positions)
    res_term = residual.inject(field=v, expr=residual * dt**2 / model.m)

    # Correlate u and v for the current time step and add it to the image
    image_update = Eq(image, image - u * v)

    return Operator([stencil] + res_term + [image_update],
                    subs=model.spacing_map)
```

## Implementation of the imaging loop

As just explained, the forward wave-equation is solved forward in time while the adjoint wave-equation is solved in a reversed time order. Therefore, the correlation of these two fields over time requires to store one of the two fields. The computational procedure for imaging follows:

- Simulate the forward wave-equation with the background velocity model to get the synthetic data and save the full wavefield $\mathbf{u}$
- Compute the data residual
- Back-propagate the data residual and compute on the fly the image contribution at each time step.

This procedure is applied to multiple source positions (shots) and summed to obtain the full image of the subsurface. We can first visualize the varying locations of the sources that we will use.

```python
#NBVAL_IGNORE_OUTPUT

# Prepare the varying source locations
source_locations = np.empty((nshots, 2), dtype=np.float32)
source_locations[:, 0] = np.linspace(0., 1000, num=nshots)
source_locations[:, 1] = 30.

plot_velocity(model, source=source_locations)
```

```python
# Run imaging loop over shots
from devito import Function

# Create image symbol and instantiate the previously defined imaging operator
image = Function(name='image', grid=model.grid)
op_imaging = ImagingOperator(model, image)

for i in range(nshots):
    print('Imaging source %d out of %d' % (i+1, nshots))

    # Update source location
    geometry.src_positions[0, :] = source_locations[i, :]

    # Generate synthetic data from true model
    true_d, _, _ = solver.forward(vp=model.vp)

    # Compute smooth data and full forward wavefield u0
    smooth_d, u0, _ = solver.forward(vp=model0.vp, save=True)

    # Compute gradient from the data residual
    v = TimeFunction(name='v', grid=model.grid, time_order=2, space_order=4)
    residual = smooth_d.data - true_d.data
    op_imaging(u=u0, v=v, vp=model0.vp, dt=model0.critical_dt,
               residual=residual)
```

    Imaging source 1 out of 21
    Imaging source 2 out of 21
    Imaging source 3 out of 21
    Imaging source 4 out of 21
    Imaging source 5 out of 21
    Imaging source 6 out of 21
    Imaging source 7 out of 21
    Imaging source 8 out of 21
    Imaging source 9 out of 21
    Imaging source 10 out of 21
    Imaging source 11 out of 21
    Imaging source 12 out of 21
    Imaging source 13 out of 21
    Imaging source 14 out of 21
    Imaging source 15 out of 21
    Imaging source 16 out of 21
    Imaging source 17 out of 21
    Imaging source 18 out of 21
    Imaging source 19 out of 21
    Imaging source 20 out of 21
    Imaging source 21 out of 21

```python
#NBVAL_IGNORE_OUTPUT
from examples.seismic import plot_image

# Plot the inverted image
plot_image(np.diff(image.data, axis=1))
```

```python
assert np.isclose(np.linalg.norm(image.data), 1e6, rtol=1e1)
```

And we have an image of the subsurface with a strong reflector at the original location.

## References

[1] _Versteeg, R.J. & Grau, G. (eds.) (1991): The Marmousi experience. Proc. EAGE workshop on Practical Aspects of Seismic Data Inversion (Copenhagen, 1990), Eur. Assoc. Explor. Geophysicists, Zeist._


```python
"""
Appendix: Kinetic, Overlap, and Dipole Integrals
"""

__author__    = ["Justin M. Turney"]
__credit__    = ["Justin M. Turney"]

__copyright__ = "(c) 2014-2017, The Psi4NumPy Developers"
__license__   = "BSD-3-Clause"
__date__      = "2017-08-10"
```

## Recurrence formula for one-electron integrals over Gaussian functions
### Cartesian Gaussian functions
The computation of molecular integrals can be considerably simplified by using recursion. We write the unnormalized Cartesian Gaussian function with origin at $\mathbf R$ as
\begin{equation}
\phi(\mathbf r; \alpha, \mathbf n, \mathbf R) = (x - \mathbf R_x)^{n_x} (y - \mathbf R_y)^{n_y} (z - \mathbf R_z)^{n_z} \exp[-\alpha (\mathbf r - \mathbf R)^2
\end{equation}
where $\mathbf r = (x, y, z)$ represents the coordinates of the electron, $\alpha$ is the orbital exponent, and $\mathbf n$ denotes a set of nonnegative integers $n_x$, $n_y$, and $n_z$:
\begin{equation}
\mathbf n = (n_x, n_y, n_z) \ .
\end{equation}
The normalization constant of the function is then given by
\begin{equation}
N(\alpha, \mathbf n) = \left(\frac{2\alpha}{\pi}\right)^{3/4} (4\alpha)^{(n_x + n_y + n_z)/2} [(2n_x-1)!!\ (2n_y-1)!!\ (2n_z-1)!!]^{-1/2}
\end{equation}

Letting $\lambda$ be defined as the sum of $n_x$, $n_y$, and $n_z$, one notes that it is closely related to the total angular momentum quantum number. $\lambda$ and $\mathbf n$ will hereafter be termed the angular momentum and the angular momentum index, respectively. The functions with $\lambda$ equal to $0,1,2,\ldots$, are referred to as $s,p,d,\ldots$, respectively. A set of $(\lambda+1)(\lambda+2)/2$ functions at $\mathbf R$ associated with the same angular momentum $\lambda$ and orbital exponent $\alpha$ constitute a *shell*, and the functions in the shell are *components* of the shell. The single component of the *s* shell with the angular momentum index $\mathbf 0 = (0,0,0)$ is usually designated *s*. The components of the *p* shell have the angular momentum indices $\mathbf 1_i\ (i=x,y,z)$, where  $\mathbf 1_i$ is defined by
\begin{equation}
\mathbf 1_i = (\delta_{ix},\delta_{iy},\delta_{iz})
\end{equation}
with Kronecker's deltas. These components will be designated $p_x$, $p_y$, and $p_z$. The components of the  *d* shell with the angular momentum indices $\mathbf 1_i + \mathbf 1_j\ (i,j=x,y,z)$ will be designated $d_{x^2}$, $d_{y^2}$, $d_{z^2}$, $d_{xy}$, $d_{xz}$, and $d_{yz}$. In later discussions all the components  in a shell are assumed to be exhausted as the basic functions, and to be treated together in the computation of molecular integrals.

First we give the basic equations for Cartesian Gaussian functions that will be utilized in the following derivations of molecular integrals. The Cartesian Gaussian functions satisfy the differential relation
\begin{equation}
\frac{\partial}{\partial R_i} \phi(\mathbf r; \alpha, \mathbf n, \mathbf R) = 2\alpha \phi(\mathbf r; \alpha, \mathbf n + \mathbf 1_i, \mathbf R) - N_i(\mathbf n)\phi(\mathbf r; \alpha, \mathbf n - \mathbf 1_i, \mathbf R) \qquad (i=x,y,z)
\end{equation}
where $N_i(\mathbf n)$, standing for $n_i$, is meant to take the value of the *i* component of the angular momentum index $\mathbf n$, and thus  $N_i(\mathbf 1_j)$ plays the same role as the Kronecker delta $\delta_{ij}$.

### Two-center overlap integrals
Two-center overlap integrals over unnormalized Cartesian Gaussian functions are of the form:
\begin{equation}
(\mathbf a|\mathbf b) = \int d\mathbf r\ \phi(\mathbf r; \alpha_a, \mathbf a, \mathbf A)\phi(\mathbf r; \alpha_b, \mathbf b, \mathbf B)
\end{equation}
According to the differential relation above the integral $(\mathbf a+\mathbf 1_i|\mathbf b)$ can be decomposed as
\begin{equation}
(\mathbf a+\mathbf 1_i|\mathbf b) = \frac{1}{2\alpha_a} \frac{\partial}{\partial \mathbf A_i} (\mathbf a|\mathbf b) + \frac{1}{2\alpha_a} N_i(\mathbf a) (\mathbf a - \mathbf 1_i|\mathbf b)
\end{equation}
Here the integral $(\mathbf a | \mathbf b)$ can be factored as
\begin{equation}
(\mathbf a | \mathbf b) = \kappa_{ab}\ I_x(n_{ax},n_{bx},n_{cx})\ I_y(n_{ay},n_{by},n_{cy})\ I_z(n_{az},n_{bz},n_{cz})
\end{equation}
where
\begin{align}
\kappa_{ab} =& \exp[-\xi (\mathbf A - \mathbf B)^2] \\
\xi =& \frac{\alpha_a \alpha_b}{\alpha_a + \alpha_b} \\
\alpha =& \alpha_a + \alpha_b \\
\mathbf P =& \frac{\alpha_a \mathbf A + \alpha_b \mathbf B}{\alpha_a + \alpha_b}
\end{align}
and
\begin{align}
I_i(n_{ai}, n_{bi}) =& \left(\frac{\pi}{\alpha_a + \alpha_b}\right)^{\tfrac{1}{2}} \\
 &\times \sum_{k_{ai}=0}^{n_{ai}} \sum_{k_{bi}=0}^{n_{bi}} \binom{n_{ai}}{k_{ai}} \binom{n_{bi}}{k_{bi}} \\
 &\times (\mathbf P_i - \mathbf A_i)^{n_{ai}-k_{ai}} (\mathbf P_i - \mathbf B_i)^{n_{bi}-k_{bi}} \\
 &\times \frac{(k_{ai} + k_{bi} - 1)!!}{(2\alpha)^{k_{ai}+k_{bi}}}
\end{align}
Differentiating $\kappa_{ab}$ and $I_i(n_{ai}, n_{bi})$ with respect to $A_i$, we have
\begin{equation}
\frac{1}{2\alpha_a}\ \frac{\partial}{\partial A_i} \kappa_{ab} = (P_i - A_i) \kappa_{ab}
\end{equation}
and
\begin{equation}
\frac{1}{2\alpha_a}\ \frac{\partial}{\partial A_i} I_i(n_{ai}, n_{bi}) = n_{ai}\left\{\frac{1}{2\alpha} - \frac{1}{2\alpha_a}\right\} I_i(n_{ai} - 1, n_{bi}) + n_{bi} \frac{1}{2\alpha} I_i(n_{ai}, n_{bi} - 1)
\end{equation}
Substituting these two equations into the expression for $(\mathbf a+\mathbf 1_i|\mathbf b)$ yields
\begin{equation}
(\mathbf a+\mathbf 1_i|\mathbf b) = (P_i - A_i)(\mathbf a|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf a)(\mathbf a-\mathbf 1_i|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf b)(\mathbf a|\mathbf b-\mathbf 1_i)
\end{equation}
and similarily
\begin{equation}
(\mathbf a|\mathbf b+\mathbf 1_i) = (P_i - B_i)(\mathbf a|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf a)(\mathbf a-\mathbf 1_i|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf b)(\mathbf a|\mathbf b-\mathbf 1_i)
\end{equation}
The integral over $s$ functions is given by
\begin{equation}
(\mathbf 0_A | \mathbf 0_B) = \left(\frac{\pi}{\alpha}\right)^{3/2} \exp[-\xi(\mathbf A-\mathbf B)^2]
\end{equation}

Given the recursive equations above, it should be seen that the recursion only depends on $\mathbf P-\mathbf A$, $\mathbf P-\mathbf B$, $\alpha$, and the target angular momentum of $\mathbf a$ and $\mathbf b$. The starting point of the recursion $(\mathbf 0_A | \mathbf 0_B)$ can be accounted for after the recursion has been performed.

### Recurrence expressions for overlap integrals over $s$, $p$, and $d$  Cartesian Gaussian funcions
\begin{align}
(s|s) =& \left(\frac{\pi}{\alpha}\right)^{3/2} \exp\left\{-\xi(\mathbf A-\mathbf B)^2\right\} \\
(p_i|s) =& (P_i - A_i)(s|s) \\
(p_i|p_j) =& (P_j - B_j)(p_i|s) + \frac{\delta_{ij}}{2\alpha}(s|s) \\
(d_{ij}|s) =& (P_j-A_j)(p_i|s) + \frac{\delta_{ij}}{2\alpha}(s|s) \\
(d_{ij}|p_k) =& (P_k-B_k)(d_{ij}|s) + \frac{\delta_{ik}}{2\alpha}(p_j|s) + \frac{\delta_{jk}}{2\alpha}(p_i|s) \\
(d_{ij}|d_{kl}) =& (P_l-B_l)(d_{ij}|p_k) + \frac{\delta_{il}}{2\alpha}(p_j|p_k) + \frac{\delta_{jl}}{2\alpha}(p_i|p_k) + \frac{\delta_{kl}}{2\alpha}(d_{ij}|s) \\
\\
& (i,j,k,l = x,y,z)
\end{align}

### Cartesian function ordering that Psi4 uses
In writing your recursion, you will need to be aware of how Psi4 orders Cartesian functions with higher angular momentum:
\begin{align}
\mathbf s &= s \\
\mathbf p &= p_x, p_y, p_z \\
\mathbf d &= d_{x^2}, d_{xy}, d_{xz}, d_{y^2}, d_{yz}, d_{z^2} \\
\mathbf f &= f_{x^3}, f_{x^2y}, f_{x^2z}, f_{xy^2}, f_{xyz}, f_{xz^2}, f_{y^3}, f_{y^2z}, f_{yz^2}, f_{z^3}
\end{align}

## One electron integrals
Now it is time to compute the following one-electron integrals:
- overlap
- kinetic
- dipole moment

Due to the added complexity of the nuclear attraction integrals and the two-electron repulsion integrals, we will not compute them in this tutorial.

### Kinetic energy integrals
Before proceeding to an analysis of the kinetic energy integral, it will prove convenient to establish a short hard notation for integrals related to the overlap integral:
\begin{equation}
(0|0) = \int G(\alpha_1, \mathbf A, l_1, m_1, n_1) G(\alpha_2, \mathbf B, l_2, m_2, n_2)\ d\tau
\end{equation}
The symbol $<+\mathrm{n}|0>_{\mathrm{x}}$ will denote an integral of the form given by the above equation, except that the quantum number $l_1$ has been incremented by n. Similar notation applies to the $m_1$ and $n_1$ quantum numbers with the use of subscripts x and y.  We will also use the symbols $<0|+\mathrm{n}>_{\mathrm{x}}$, where we have incremented the quantum number $l_2$ by n. You should see the pattern.

The kinetic energy integral is defined as
\begin{equation}
T_{ij} = -\frac{1}{2} \int \phi_i(1) \nabla_1^2 \phi_j(1)\ d\tau_1
\end{equation}
and as
\begin{equation}
KE = - \frac{1}{2}  \int G(\alpha_1, \mathbf A, l_1, m_1, n_1) \left(\frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} + \frac{\partial^2}{\partial z^2}\right) G(\alpha_2, \mathbf B, l_2, m_2, n_2)\ d\tau
\end{equation}

Evaluate the kinetic energy integrals for $I_x$, $I_y$, and $I_z$ and code them up to obtain $T_{ij}$. The equations that you obtain for these integrals are "unsymmetric" because the quantum numbers of the Gaussian-type function centered on $\mathbf B$ are altered while those of the Gaussian-type function on $\mathbf A$ are not.

Here is the symmetric form of the kinetic energy integral $I_x$ component.
\begin{equation}
I_x = \frac{1}{2}\Big\{l_1 l_2 (-1|-1)_{\mathrm{x}} + 4\alpha_1 \alpha_2 (+1|+1)_{\mathrm{x}} - 2\alpha_1 l_2 (+1|-1)_{\mathrm{x}} - 2\alpha_2 l_1 (-1|+1)_{\mathrm{x}}\Big\}
\end{equation}

### Dipole moment integrals
We discuss these non-energy integrals here because they are frequently used and closely connected with the overlap and kinetic energy integrals. The dipole moment is defined with respect to a point in space $\mathbf C$, which is almost always taken to be the center of mass. Fortunately for our purposes, Psi4 automatically moves the molecule to be centered on the center of mass and thus $\mathbf C$ is $\mathbf 0$. The dipole moment integral is written for the $x$ direction as
\begin{equation}
d_{\mathrm{x}} =  \int G(\alpha_1, \mathbf A, l_1, m_1, n_1) G(\alpha_2, \mathbf B, l_2, m_2, n_2) \mathrm{x_c}\ d\tau
\end{equation}
and similarly for the operators $\mathrm{y_c}$ and $\mathrm{z_c}$. A convenient procedure is to redefine $\mathrm{x_c}$ in terms of  $\mathrm{x_A}$ or $\mathrm{x_B}$, and we will use $\mathrm{x_A}$. Thus
\begin{equation}
\mathrm{x_c} = \mathrm{x} - \mathrm{C_x} = (\mathrm{x - A_x}) + (\mathrm{A_x - C_x}) = \mathrm{x_A + \mathbf {AC}_x}
\end{equation}
and we find
\begin{equation}
d_\mathrm{x} = (+1|0)_{\mathrm{x}} + \mathbf{AC}_{\mathrm{x}} (0|0)
\end{equation}
Similarily for $d_{\mathrm{y}}$ and $d_{\mathrm{z}}$.

#### The x component
\begin{align}
\mu_x =& - \mu_{\mathrm{elec}} + \mu_{\mathrm{nuclear}}\\
\mu_x =& - \sum_{\mu \nu}^{\mathrm{AO}} D_{\mu \nu} d_{\mu \nu}^{x} + \sum_A^N Z_A X_A
\end{align}

You can obtain the nuclear contribution to the dipole $\mu_{\mathrm{nuclear}}$ from Psi4 using

    psi4.core.nuclear_dipole(molecule).to_array()

#### The total dipole moment
\begin{equation}
\mu = \sqrt{\mu_x^2 + \mu_y^2 + \mu_z^2}
\end{equation}


## 1-3. 複数量子ビットの記述

ここまでは１量子ビットの状態とその操作（演算）の記述について学んできた。この章の締めくくりとして、$n$個の量子ビットがある場合の状態の記述について学んでいこう。テンソル積がたくさん出てきてややこしいが、コードをいじりながら身につけていってほしい。

$n$個の**古典**ビットの状態は$n$個の$0,1$の数字によって表現され、そのパターンの総数は$2^n$個ある。
量子力学では、これらすべてのパターンの重ね合わせ状態が許されているので、$n$個の**量子**ビットの状態$|\psi \rangle$はどのビット列がどのような重みで重ね合わせになっているかという$2^n$個の複素確率振幅で記述される：

$$
\begin{eqnarray}
|\psi \rangle &= &
c_{00...0} |00...0\rangle +
c_{00...1} |00...1\rangle + \cdots +
c_{11...1} |11...1\rangle =
\left(
\begin{array}{c}
c_{00...0}
\\
c_{00...1}
\\
\vdots
\\
c_{11...1}
\end{array}
\right).
\end{eqnarray}
$$

ただし、
複素確率振幅は規格化
$\sum _{i_1,..., i_n} |c_{i_1...i_n}|^2=1$
されているものとする。
そして、この$n$量子ビットの量子状態を測定するとビット列$i_1 ... i_n$が確率

$$
\begin{eqnarray}
p_{i_1 ... i_n} &=&|c_{i_1 ... i_n}|^2
\label{eq02}
\end{eqnarray}
$$

でランダムに得られ、測定後の状態は$|i_1 \dotsc i_n\rangle$となる。

**このように**$n$**量子ビットの状態は、**$n$**に対して指数的に大きい**$2^n$**次元の複素ベクトルで記述する必要があり、ここに古典ビットと量子ビットの違いが顕著に現れる**。
そして、$n$量子ビット系に対する操作は$2^n \times 2^n$次元のユニタリ行列として表される。
言ってしまえば、量子コンピュータとは、量子ビット数に対して指数的なサイズの複素ベクトルを、物理法則に従ってユニタリ変換するコンピュータのことなのである。

※ここで、複数量子ビットの順番と表記の関係について注意しておく。状態をケットで記述する際に、「1番目」の量子ビット、「2番目」の量子ビット、……の状態に対応する0と1を左から順番に並べて表記した。例えば$|011\rangle$と書けば、1番目の量子ビットが0、2番目の量子ビットが1、3番目の量子ビットが1である状態を表す。一方、例えば011を2進数の表記と見た場合、上位ビットが左、下位ビットが右となることに注意しよう。すなわち、一番左の0は最上位ビットであって$2^2$の位に対応し、真ん中の1は$2^1$の位、一番右の1は最下位ビットであって$2^0=1$の位に対応する。つまり、「$i$番目」の量子ビットは、$n$桁の2進数表記の$n-i+1$桁目に対応している。このことは、SymPyなどのパッケージで複数量子ビットを扱う際に気を付ける必要がある（下記「SymPyを用いた演算子のテンソル積」も参照）。

(詳細は Nielsen-Chuang の `1.2.1 Multiple qbits` を参照)

### 例：2量子ビットの場合
2量子ビットの場合は、 00, 01, 10, 11 の４通りの状態の重ね合わせをとりうるので、その状態は一般的に

$$
c_{00} |00\rangle + c_{01} |01\rangle + c_{10}|10\rangle + c_{11} |11\rangle  =
\left(
\begin{array}{c}
c_{00}
\\
c_{01}
\\
c_{10}
\\
c_{11}
\end{array}
\right)
$$

とかける。

一方、2量子ビットに対する演算は$4 \times 4$行列で書け、4行4列の行列成分はそれぞれ$|00\rangle,|01\rangle,|10\rangle, |01\rangle$に対応する。
このような2量子ビットに作用する演算としてもっとも重要なのが**制御NOT演算（CNOT演算）**であり、
行列表示では

$$
\begin{eqnarray}
\Lambda(X) =
\left(
\begin{array}{cccc}
1 & 0 & 0& 0
\\
0 & 1 & 0& 0
\\
0 & 0 & 0 & 1
\\
0 & 0 & 1& 0
\end{array}
\right)
\end{eqnarray}
$$

となる。
CNOT演算が2つの量子ビットにどのように作用するか見てみよう。まず、1つ目の量子ビットが$|0\rangle$の場合、$c_{10} = c_{11} = 0$なので、

$$
\Lambda(X)
\left(
\begin{array}{c}
c_{00}\\
c_{01}\\
0\\
0
\end{array}
\right) =
\left(
\begin{array}{c}
c_{00}\\
c_{01}\\
0\\
0
\end{array}
\right)
$$

となり、状態は変化しない。一方、1つ目の量子ビットが$|1\rangle$の場合、$c_{00} = c_{01} = 0$なので、

$$
\Lambda(X)
\left(
\begin{array}{c}
0\\
0\\
c_{10}\\
c_{11}
\end{array}
\right) =
\left(
\begin{array}{c}
0\\
0\\
c_{11}\\
c_{10}
\end{array}
\right)
$$

となり、$|10\rangle$と$|11\rangle$の確率振幅が入れ替わる。すなわち、2つ目の量子ビットが反転している。

つまり、CNOT演算は１つ目の量子ビットをそのままに保ちつつ、

- １つ目の量子ビットが$|0\rangle$の場合は、２つ目の量子ビットにも何もしない（恒等演算$I$が作用）
- １つ目の量子ビットが$|1\rangle$の場合は、２つ目の量子ビットを反転させる（$X$が作用）

という効果を持つ。
そこで、１つ目の量子ビットを**制御量子ビット**、２つ目の量子ビットを**ターゲット量子ビット**と呼ぶ。

このCNOT演算の作用は、$\oplus$を mod 2の足し算、つまり古典計算における排他的論理和（XOR）とすると、

$$
\begin{eqnarray}
\Lambda(X) |ij \rangle = |i \;\; (i\oplus j)\rangle \:\:\: (i,j=0,1)
\end{eqnarray}
$$

とも書ける。よって、CNOT演算は古典計算でのXORを可逆にしたものとみなせる
（ユニタリー行列は定義$U^\dagger U = U U^\dagger = I$より可逆であることに注意）。
例えば、１つ目の量子ビットを$|0\rangle$と$|1\rangle$の
重ね合わせ状態にし、２つ目の量子ビットを$|0\rangle$として

$$
\begin{eqnarray}
\frac{1}{\sqrt{2}}(|0\rangle + |1\rangle )\otimes |0\rangle =
\frac{1}{\sqrt{2}}
\left(
\begin{array}{c}
1
\\
0
\\
1
\\
0
\end{array}
\right)
\end{eqnarray}
$$

にCNOTを作用させると、

$$
\begin{eqnarray}
\frac{1}{\sqrt{2}}( |00\rangle + |11\rangle ) =
\frac{1}{\sqrt{2}}
\left(
\begin{array}{c}
1
\\
0
\\
0
\\
1
\end{array}
\right)
\end{eqnarray}
$$

が得られ、2つ目の量子ビットがそのままである状態$|00\rangle$と反転された状態$|11\rangle$の重ね合わせになる。（記号$\otimes$については次節参照）

さらに、CNOT ゲートを組み合わせることで重要な2量子ビットゲートである**SWAP ゲート**を作ることができる。

$$\Lambda(X)_{i,j}$$

を$i$番目の量子ビットを制御、$j$番目の量子ビットをターゲットとするCNOT ゲートとして、

$$
\begin{align}
\mathrm{SWAP} &= \Lambda(X)_{1,2} \Lambda(X)_{2,1} \Lambda(X)_{1,2}\\
&=
\left(
\begin{array}{cccc}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1 \\
0 & 0 & 1 & 0
\end{array}
\right)
\left(
\begin{array}{cccc}
1 & 0 & 0 & 0 \\
0 & 0 & 0 & 1 \\
0 & 0 & 1 & 0 \\
0 & 1 & 0 & 0
\end{array}
\right)
\left(
\begin{array}{cccc}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1 \\
0 & 0 & 1 & 0
\end{array}
\right)\\
&=
\left(
\begin{array}{cccc}
1 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1
\end{array}
\right)
\end{align}
$$

のように書ける。これは1 番目の量子ビットと2 番目の量子ビットが交換するゲートであることが分かる。

このことは、上記のmod 2の足し算$\oplus$を使った表記で簡単に確かめることができる。3つのCNOTゲート$\Lambda(X)_{1,2} \Lambda(X)_{2,1} \Lambda(X)_{1,2}$の$|ij\rangle$への作用を1ステップずつ書くと、$i \oplus (i \oplus j) = (i \oplus i) \oplus j = 0 \oplus j = j$であることを使って、

$$
\begin{align}
|ij\rangle &\longrightarrow
|i \;\; (i\oplus j)\rangle\\
&\longrightarrow
|(i\oplus (i\oplus j)) \;\; (i\oplus j)\rangle =
|j \;\; (i\oplus j)\rangle\\
&\longrightarrow
|j \;\; (j\oplus (i\oplus j))\rangle =
|ji\rangle
\end{align}
$$

となり、2つの量子ビットが交換されていることが分かる。

(詳細は Nielsen-Chuang の `1.3.2 Multiple qbit gates` を参照)

### テンソル積の計算
手計算や解析計算で威力を発揮するのは、**テンソル積**（$\otimes$）である。
これは、複数の量子ビットがある場合に、それをどのようにして、上で見た大きな一つのベクトルへと変換するのか？という計算のルールを与えてくれる。

量子力学の世界では、２つの量子系があってそれぞれの状態が$|\psi \rangle$と$|\phi \rangle$のとき、

$$
|\psi \rangle \otimes |\phi\rangle
$$

とテンソル積 $\otimes$ を用いて書く。このような複数の量子系からなる系のことを**複合系**と呼ぶ。例えば２量子ビット系は複合系である。

基本的にはテンソル積は、**多項式と同じような計算ルール**で計算してよい。
例えば、

$$
(\alpha |0\rangle + \beta |1\rangle )\otimes (\gamma |0\rangle + \delta |1\rangle )
= \alpha \gamma |0\rangle |0\rangle + \alpha \delta |0\rangle |1\rangle + \beta \gamma |1 \rangle | 0\rangle + \beta \delta |1\rangle |1\rangle
$$

のように計算する。列ベクトル表示すると、$|00\rangle$, $|01\rangle$, $|10\rangle$, $|11\rangle$に対応する4次元ベクトル、

$$
\left(
\begin{array}{c}
\alpha
\\
\beta
\end{array}
\right)
\otimes
\left(
\begin{array}{c}
\gamma
\\
\delta
\end{array}
\right) =
\left(
\begin{array}{c}
\alpha \gamma
\\
\alpha \delta
\\
\beta \gamma
\\
\beta \delta
\end{array}
\right)
$$

を得る計算になっている。

### SymPyを用いたテンソル積の計算

```python
from IPython.display import Image, display_png
from sympy import *
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import Qubit,QubitBra
from sympy.physics.quantum.gate import X,Y,Z,H,S,T,CNOT,SWAP, CPHASE
init_printing() # ベクトルや行列を綺麗に表示するため
```

```python
# Google Colaboratory上でのみ実行してください
from IPython.display import HTML
def setup_mathjax():
    display(HTML('''

    '''))
get_ipython().events.register('pre_run_cell', setup_mathjax)
```

```python
a,b,c,d = symbols('alpha,beta,gamma,delta')
psi = a*Qubit('0')+b*Qubit('1')
phi = c*Qubit('0')+d*Qubit('1')
```

```python
TensorProduct(psi, phi) #テンソル積
```

```python
represent(TensorProduct(psi, phi))
```

さらに$|\psi\rangle$とのテンソル積をとると８次元のベクトルになる：

```python
represent(TensorProduct(psi,TensorProduct(psi, phi)))
```

### 演算子のテンソル積
演算子についても何番目の量子ビットに作用するのか、というのをテンソル積をもちいて表現することができる。たとえば、１つめの量子ビットには$A$という演算子、2つめの演算子には$B$を作用させるという場合には、

$$ A \otimes B$$

としてテンソル積演算子が与えられる。
$A$と$B$をそれぞれ、2×2の行列とすると、$A\otimes B$は4×4の行列として

$$
\left(
\begin{array}{cc}
a_{11} & a_{12}
\\
a_{21} & a_{22}
\end{array}
\right)
\otimes
\left(
\begin{array}{cc}
b_{11} & b_{12}
\\
b_{21} & b_{22}
\end{array}
\right) =
\left(
\begin{array}{cccc}
a_{11} b_{11} & a_{11} b_{12} & a_{12} b_{11} & a_{12} b_{12}
\\
a_{11} b_{21} & a_{11} b_{22} & a_{12} b_{21} & a_{12} b_{22}
\\
a_{21} b_{11} & a_{21} b_{12} & a_{22} b_{11} & a_{22} b_{12}
\\
a_{21} b_{21} & a_{21} b_{22} & a_{22} b_{21} & a_{22} b_{22}
\end{array}
\right)
$$

のように計算される。

テンソル積状態

$$|\psi \rangle \otimes | \phi \rangle $$

に対する作用は、

$$ (A|\psi \rangle ) \otimes (B |\phi \rangle )$$

となり、それぞれの部分系$|\psi \rangle$と$|\phi\rangle$に$A$と$B$が作用する。
足し算に対しては、多項式のように展開してそれぞれの項を作用させればよい。

$$
(A+C)\otimes (B+D) |\psi \rangle \otimes | \phi \rangle =
(A \otimes B +A \otimes D + C \otimes B + C \otimes D) |\psi \rangle \otimes | \phi \rangle\\ =
(A|\psi \rangle) \otimes (B| \phi \rangle)
+(A|\psi \rangle) \otimes (D| \phi \rangle)
+(C|\psi \rangle) \otimes (B| \phi \rangle)
+(C|\psi \rangle) \otimes (D| \phi \rangle)
$$

テンソル積やテンソル積演算子は左右横並びで書いているが、本当は

$$
\left(
\begin{array}{c}
A
\\
\otimes
\\
B
\end{array}
\right)
\begin{array}{c}
|\psi \rangle
\\
\otimes
\\
|\phi\rangle
\end{array}
$$

のように縦に並べた方がその作用の仕方わかりやすいのかもしれない。

例えば、CNOT演算を用いて作られるエンタングル状態は、

$$
\left(
\begin{array}{c}
|0\rangle \langle 0|
\\
\otimes
\\
I
\end{array}
+
\begin{array}{c}
|1\rangle \langle 1|
\\
\otimes
\\
X
\end{array}
\right)
\left(
\begin{array}{c}
\frac{1}{\sqrt{2}}(|0\rangle + |1\rangle)
\\
\otimes
\\
|0\rangle
\end{array}
\right) =
\frac{1}{\sqrt{2}}\left(
\begin{array}{c}
|0 \rangle
\\
\otimes
\\
|0\rangle
\end{array}
+
\begin{array}{c}
|1 \rangle
\\
\otimes
\\
|1\rangle
\end{array}
\right)
$$

のようになる。

### SymPyを用いた演算子のテンソル積
SymPyで演算子を使用する時は、何桁目の量子ビットに作用する演算子かを常に指定する。「何**番目**」ではなく2進数表記の「何**桁目**」であることに注意しよう。$n$量子ビットのうちの左から$i$番目の量子ビットを指定する場合、SymPyのコードでは`n-i`を指定する（0を基点とするインデックス）。

`H(0)` は、1量子ビット空間で表示すると

```python
represent(H(0),nqubits=1)
```

2量子ビット空間では$H \otimes I$に対応しており、その表示は

```python
represent(H(1),nqubits=2)
```

CNOT演算は、

```python
represent(CNOT(1,0),nqubits=2)
```

パウリ演算子のテンソル積$X\otimes Y \otimes Z$も、

```python
represent(X(2)*Y(1)*Z(0),nqubits=3)
```

このようにして、上記のテンソル積のルールを実際にたしかめてみることができる。

### 複数の量子ビットの一部分だけを測定した場合

複数の量子ビットを全て測定した場合の測定結果の確率については既に説明した。複数の量子ビットのうち、一部だけを測定することもできる。その場合、測定結果の確率は、測定結果に対応する（部分系の）基底で射影したベクトルの長さの2乗になり、測定後の状態は射影されたベクトルを規格化したものになる。

具体的に見ていこう。以下の$n$量子ビットの状態を考える。
\begin{align}
|\psi\rangle &=
c_{00...0} |00...0\rangle +
c_{00...1} |00...1\rangle + \cdots +
c_{11...1} |11...1\rangle\\
&= \sum_{i_1 \dotsc i_n} c_{i_1 \dotsc i_n} |i_1 \dotsc i_n\rangle =
\sum_{i_1 \dotsc i_n} c_{i_1 \dotsc i_n} |i_1\rangle \otimes \cdots \otimes |i_n\rangle
\end{align}
1番目の量子ビットを測定するとしよう。1つ目の量子ビットの状態空間の正規直交基底$|0\rangle$, $|1\rangle$に対する射影演算子はそれぞれ$|0\rangle\langle0|$, $|1\rangle\langle1|$と書ける。1番目の量子ビットを$|0\rangle$に射影し、他の量子ビットには何もしない演算子

$$
|0\rangle\langle0| \otimes I \otimes \cdots \otimes I
$$

を使って、測定値0が得られる確率は

$$
\bigl\Vert \bigl(|0\rangle\langle0| \otimes I \otimes \cdots \otimes I\bigr) |\psi\rangle \bigr\Vert^2 =
\langle \psi | \bigl(|0\rangle\langle0| \otimes I \otimes \cdots \otimes I\bigr) | \psi \rangle
$$

である。ここで

$$
\bigl(|0\rangle\langle0| \otimes I \otimes \cdots \otimes I\bigr) | \psi \rangle =
\sum_{i_2 \dotsc i_n} c_{0 i_2 \dotsc i_n} |0\rangle \otimes |i_2\rangle \otimes \cdots \otimes |i_n\rangle
$$

なので、求める確率は

$$
p_0 = \sum_{i_2 \dotsc i_n} |c_{0 i_2 \dotsc i_n}|^2
$$

となり、測定後の状態は

$$
\frac{1}{\sqrt{p_0}}\sum_{i_2 \dotsc i_n} c_{0 i_2 \dotsc i_n} |0\rangle \otimes |i_2\rangle \otimes \cdots \otimes |i_n\rangle
$$

となる。0と1を入れ替えれば、測定値1が得られる確率と測定後の状態が得られる。

ここで求めた$p_0$, $p_1$の表式は、測定値$i_1, \dotsc, i_n$が得られる同時確率分布$p_{i_1, \dotsc, i_n}$から計算される$i_1$の周辺確率分布と一致することに注意しよう。実際、

$$
\sum_{i_2, \dotsc, i_n} p_{i_1, \dotsc, i_n} = \sum_{i_2, \dotsc, i_n} |c_{i_1, \dotsc, i_n}|^2 = p_{i_1}
$$

である。

測定される量子ビットを増やし、最初の$k$個の量子ビットを測定する場合も同様に計算できる。測定結果$i_1, \dotsc, i_k$を得る確率は

$$
p_{i_1, \dotsc, i_k} = \sum_{i_{k+1}, \dotsc, i_n} |c_{i_1, \dotsc, i_n}|^2
$$

であり、測定後の状態は

$$
\frac{1}{\sqrt{p_{i_1, \dotsc, i_k}}}\sum_{i_{k+1} \dotsc i_n} c_{i_1 \dotsc i_n} |i_1 \rangle \otimes \cdots \otimes |i_n\rangle
$$

となる。（和をとるのは$i_{k+1},\cdots,i_n$だけであることに注意）

SymPyを使ってさらに具体的な例を見てみよう。H演算とCNOT演算を組み合わせて作られる次の状態を考える。
$$
|\psi\rangle = \Lambda(X) (H \otimes H) |0\rangle \otimes |0\rangle = \frac{|00\rangle + |10\rangle + |01\rangle + |11\rangle}{2}
$$

```python
psi = qapply(CNOT(1, 0)*H(1)*H(0)*Qubit('00'))
psi
```

この状態の1つ目の量子ビットを測定して0になる確率は

$$
p_0 = \langle \psi | \bigl( |0\rangle\langle0| \otimes I \bigr) | \psi \rangle =
\left(\frac{\langle 00 | + \langle 10 | + \langle 01 | + \langle 11 |}{2}\right)
\left(\frac{| 00 \rangle + | 01 \rangle}{2}\right) =
\frac{1}{2}
$$

で、測定後の状態は

$$
\frac{1}{\sqrt{p_0}} \bigl( |0\rangle\langle0| \otimes I \bigr) | \psi \rangle =
\frac{| 00 \rangle + | 01 \rangle}{\sqrt{2}}
$$

である。

この結果をSymPyでも計算してみよう。SymPyには測定用の関数が数種類用意されていて、一部の量子ビットを測定した場合の確率と測定後の状態を計算するには、`measure_partial`を用いればよい。測定する状態と、測定を行う量子ビットのインデックスを引数として渡すと、測定後の状態と測定の確率の組がリストとして出力される。１つめの量子ビットが0だった場合の量子状態と確率は`[0]`要素を参照すればよい。

```python
from sympy.physics.quantum.qubit import measure_all, measure_partial
measured_state_and_probability = measure_partial(psi, (1,))
```

```python
measured_state_and_probability[0]
```

上で手計算した結果と合っていることが分かる。測定結果が1だった場合も同様に計算できる。

```python
measured_state_and_probability[1]
```

---
## コラム：ユニバーサルゲートセットとは

古典計算機では、NANDゲート（論理積ANDの出力を反転したもの）さえあれば、これをいくつか組み合わせることで、任意の論理演算が実行できることが知られている。
それでは、量子計算における対応物、すなわち任意の量子計算を実行するために最低限必要な量子ゲートは何であろうか？
実は、本節で学んだ
$$\{H, T, {\rm CNOT} \}$$

の3種類のゲートがその役割を果たしている、いわゆる**ユニバーサルゲートセット**であることが知られている。
これらをうまく組み合わせることで、任意の量子計算を実行できる、すなわち「**万能量子計算**」が可能である。

### 【より詳しく知りたい人のための注】

以下では$\{H, T, {\rm CNOT} \}$の3種のゲートの組が如何にしてユニバーサルゲートセットを構成するかを、順を追って説明する。
流れとしては、一般の$n$量子ビットユニタリ演算からスタートし、これをより細かい部品にブレイクダウンしていくことで、最終的に上記3種のゲートに行き着くことを見る。

#### ◆ $n$量子ビットユニタリ演算の分解
まず、任意の$n$量子ビットユニタリ演算は、以下の手順を経て、いくつかの**1量子ビットユニタリ演算**と**CNOTゲート**に分解できる。

1. 任意の$n$量子ビットユニタリ演算は、いくつかの**2準位ユニタリ演算**の積に分解できる。ここで2準位ユニタリ演算とは、例として3量子ビットの場合、$2^3=8$次元空間のうち2つの基底(e.g., $\{|000\rangle, |111\rangle \}$)の張る2次元部分空間にのみ作用するユニタリ演算である
2. 任意の2準位ユニタリ演算は、**制御**$U$**ゲート**（CNOTゲートのNOT部分を任意の1量子ビットユニタリ演算$U$に置き換えたもの）と**Toffoliゲート**（CNOTゲートの制御量子ビットが2つになったもの）から構成できる
3. 制御$U$ゲートとToffoliゲートは、どちらも**1量子ビットユニタリ演算**と**CNOTゲート**から構成できる

#### ◆ 1量子ビットユニタリ演算の構成
さらに、任意の1量子ビットユニタリ演算は、$\{H, T\}$の2つで構成できる。

1. 任意の1量子ビットユニタリ演算は、オイラーの回転角の法則から、回転ゲート$\{R_X(\theta), R_Z(\theta)\}$で（厳密に）実現可能である
2. 実は、ブロッホ球上の任意の回転は、$\{H, T\}$のみを用いることで実現可能である(注1)。これはある軸に関する$\pi$の無理数倍の回転が$\{H, T\}$のみから実現できること（**Solovay-Kitaevアルゴリズム**）に起因する

(注1) ブロッホ球上の連続的な回転を、離散的な演算である$\{H, T\}$で実現できるか疑問に思われる読者もいるかもしれない。実際、厳密な意味で1量子ビットユニタリ演算を離散的なゲート操作で実現しようとすると、無限個のゲートが必要となる。しかし実際には厳密なユニタリ演算を実現する必要はなく、必要な計算精度$\epsilon$で任意のユニタリ演算を近似できれば十分である。ここでは、多項式個の$\{H, T\}$を用いることで、任意の1量子ビットユニタリ演算を**十分良い精度で近似的に構成できる**ことが、**Solovay-Kitaevの定理**により保証されている。

以上の議論により、3種のゲート$\{H, T, {\rm CNOT} \}$があれば、任意の$n$量子ビットユニタリ演算が実現できることがわかる。

ユニバーサルゲートセットや万能量子計算について、より詳しくは以下を参照されたい:
[1] Nielsen-Chuang の 4.5 Universal quantum gates
[2] 藤井 啓祐 「量子コンピュータの基礎と物理との接点」（第62回物性若手夏の学校　講義）DOI: 10.14989/229039 http://mercury.yukawa.kyoto-u.ac.jp/~bussei.kenkyu/archives/1274.html


# Block 1: Linear programming
# R Notebook
### Alfred Galichon (NYU)
## `math+econ+code' masterclass on matching models, optimal transport and applications
© 2018-2019 by Alfred Galichon. Support from NSF grant DMS-1716489 is acknowledged. James Nesbit contributed.

## Linear programming: duality

### Learning objectives

* Linear programming duality

* Economic interpretation of the dual

* Numerical computation

### References

* [OTME], App. B

* Stigler (1945), The cost of subsistence. *Journal of Farm Economics*.

* Dantzig (1990), The diet problem. *Interface*.

* Complements:

    * Gale (1960), *The theory of linear economic models*.

    * Vohra (2011), *Mechanism Design: A Linear Programming Approach*.

### The diet problem

During World War II, engineers in US Army were wondering how to feed their personnel at minimal cost, leading to what is now called the **optimal diet problem**.

* Nutritionists have identified a number of vital nutrients (calories, protein, calcium, iron, etc.) that matter for a person's health, and have determined the minimum daily intake of each nutrient

* For each basic food (pasta, butter, bread, etc), nutritionists have characterized the intake in each of the various nutrients

* Each food has a unit cost, and the problem is to find the optimal diet = combination of foods that meet the minimal intake in each of the nutrients and achieves minimal cost

The problem was taken on by G. Stigler, who published a paper about it in 1945, giving a first heuristic solution, exhibiting a diet that costs $\$39.93$ per year in $1939$ dollars. Later (in $1947$) it was one of the first
application of G.B. Dantzig's method (the simplex algorithm), which provided the exact solution ($\$39.67$). It then took $120$ man-day to perform this operation. However today the computer will perform it for us in a
fraction of second.

However, don't try this diet at home! Dantzig did so and almost died from it...

## Motivation

### A look at the Data

Our dataset was directly taken from Stigler's article. It is a .csv file called `StiglerData1939.txt':

```R
thepath = paste0(getwd(),"/../data_mec_optim/lp_stigler-diet")
filename = "/StiglerData1939.txt"
thedata = as.matrix(read.csv(paste0(thepath, filename), sep = "\t", header = T))
thedata[1:(dim(thedata)[1] - 2), ]  # cut off 2 redunant rows at the bottom
```

A matrix: 78 × 13 of type chr

Commodity	Unit	Price.Aug.15.1939.cents.	Edible.Weight.per..1.00..grams.	Calories..1000.	Protein.grams.	Calcium.grams.	Iron.mg..	Vitamin.A.1000.I.U.	Thiamine.mg..	Riboflavin.mg..	Niacin.mg..	Asorbic.Acid..mg..

1. Wheat Flour (Enriched) 	10 lb.  	36.0	12600	44.7	1411	 2.0	365	NA   	55.4	33.3	441	NA

2. Macaroni               	1 lb.   	14.1	 3217	11.6	 418	 0.7	 54	NA   	 3.2	 1.9	 68	NA

3. Wheat Cereal (Enriched)	28 oz.  	24.2	 3280	11.8	 377	14.4	175	NA   	14.4	 8.8	114	NA

4. Corn Flakes            	8 oz.   	 7.1	 3194	11.4	 252	 0.1	 56	NA   	13.5	 2.3	 68	NA

5. Corn Meal              	1 lb.   	 4.6	 9861	36.0	 897	 1.7	 99	 30.9	17.4	 7.9	106	NA

6. Hominy Grits           	24 oz.  	 8.5	 8005	28.6	 680	 0.8	 80	NA   	10.6	 1.6	110	NA

7. Rice                   	1 lb.   	 7.5	 6048	21.2	 460	 0.6	 41	NA   	 2.0	 4.8	 60	NA

8. Rolled Oats            	1 lb.   	 7.1	 6389	25.3	 907	 5.1	341	NA   	37.1	 8.9	 64	NA

9. White Bread (Enriched) 	1 lb.   	 7.9	 5742	15.0	 488	 2.5	115	NA   	13.8	 8.5	126	NA

10. Whole Wheat Bread     	1 lb.   	 9.1	 4985	12.2	 484	 2.7	125	NA   	13.9	 6.4	160	NA

11. Rye Bread             	1 lb.   	 9.2	 4930	12.4	 439	 1.1	 82	NA   	 9.9	 3.0	 66	NA

12. Pound Cake            	1 lb.   	24.8	 1829	 8.0	 130	 0.4	 31	 18.9	 2.8	 3.0	 17	NA

13. Soda Crackers         	1 lb.   	15.1	 3004	12.5	 288	 0.5	 50	NA   	NA  	NA  	NA 	NA

14. Milk                  	1 qt.   	11.0	 8867	 6.1	 310	10.5	 18	 16.8	 4.0	16.0	  7	 177

15. Evaporated Milk (can) 	14.5 oz.	 6.7	 6035	 8.4	 422	15.1	  9	 26.0	 3.0	23.5	 11	  60

16. Butters               	1 lb.   	30.8	 1473	10.8	   9	 0.2	  3	 44.2	NA  	 0.2	  2	NA

17. Oleomargarine         	1 lb.   	16.1	 2817	20.6	  17	 0.6	  6	 55.8	 0.2	NA  	NA 	NA

18. Eggs                  	1 doz.  	32.6	 1857	 2.9	 238	 1.0	 52	 18.6	 2.8	 6.5	  1	NA

19. Cheese (Cheddar)      	1 lb.   	24.2	 1874	 7.4	 448	16.4	 19	 28.1	 0.8	10.3	  4	NA

20. Cream                 	0.5 pt. 	14.1	 1689	 3.5	  49	 1.7	  3	 16.9	 0.6	 2.5	NA 	  17

21. Peanut Butter         	1 lb.   	17.9	 2534	15.7	 661	 1.0	 48	NA   	 9.6	 8.1	471	NA

22. Mayonnaise            	0.5 pt. 	16.7	 1198	 8.6	  18	 0.2	  8	  2.7	 0.4	 0.5	NA 	NA

23. Crisco                	1 lb.   	20.3	 2234	20.1	NA  	NA  	NA 	NA   	NA  	NA  	NA 	NA

24. Lard                  	1 lb.   	 9.8	 4628	41.7	NA  	NA  	NA 	  0.2	NA  	 0.5	  5	NA

25. Sirloin Steak         	1 lb.   	39.6	 1145	 2.9	 166	 0.1	 34	  0.2	 2.1	 2.9	 69	NA

26. Round Steak           	1 lb.   	36.4	 1246	 2.2	 214	 0.1	 32	  0.4	 2.5	 2.4	 87	NA

27. Rib Roast             	1 lb.   	29.2	 1553	 3.4	 213	 0.1	 33	NA   	NA  	 2.0	NA 	NA

28. Chuck Roast           	1 lb.   	22.6	 2007	 3.6	 309	 0.2	 46	  0.4	 1.0	 4.0	120	NA

29. Plate                 	1 lb.   	14.6	 3107	 8.5	 404	 0.2	 62	NA   	 0.9	NA  	NA 	NA

30. Liver (Beef)          	1 lb.   	26.8	 1692	 2.2	 333	 0.2	139	169.2	 6.4	50.8	316	 525

...	...	...	...	...	...	...	...	...	...	...	...	...

49. Lettuce              	1 head  	 8.2	 2247	 0.4	  27	 1.1	 22	112.4	 1.8	 3.4	 11	 449

50. Onions               	1 lb.   	 3.6	11844	 5.8	 166	 3.8	 59	 16.6	 4.7	 5.9	 21	1184

51. Potatoes             	15 lb.  	34.0	16810	14.3	 336	 1.8	118	  6.7	29.4	 7.1	198	2522

52. Spinach              	1 lb.   	 8.1	 4592	 1.1	 106	NA  	138	918.4	 5.7	13.8	 33	2755

53. Sweet Potatoes       	1 lb.   	 5.1	 7649	 9.6	 138	 2.7	 54	290.7	 8.4	 5.4	 83	1912

54. Peaches (can)        	No.2.5  	16.8	 4894	 3.7	  20	 0.4	 10	 21.5	 0.5	 1.0	 31	 196

55. Pears (can)          	No.2.5  	20.4	 4030	 3.0	   8	 0.3	  8	  0.8	 0.8	 0.8	  5	  81

56. Pineapple (can)      	No.2.5  	21.3	 3993	 2.4	  16	 0.4	  8	  2.0	 2.8	 0.8	  7	 399

57. Asparagus (can)      	No.2    	27.7	 1945	 0.4	  33	 0.3	 12	 16.3	 1.4	 2.1	 17	 272

58. Green Beans (can)    	No.2    	10.0	 5386	 1.0	  54	 2.0	 65	 53.9	 1.6	 4.3	 32	 431

59. Pork and Beans (can) 	16 oz.  	 7.1	 6389	 7.5	 364	 4.0	134	  3.5	 8.3	 7.7	 56	NA

60. Corn (can)           	No.2    	10.4	 5452	 5.2	 136	 0.2	 16	 12.0	 1.6	 2.7	 42	 218

61. Peas (can)           	No.2    	13.8	 4109	 2.3	 136	 0.6	 45	 34.9	 4.9	 2.5	 37	 370

62. Tomatoes (can)       	No.2    	 8.6	 6263	 1.3	  63	 0.7	 38	 53.2	 3.4	 2.5	 36	1253

63. Tomato Soup (can)    	10.5 oz.	 7.6	 3917	 1.6	  71	 0.6	 43	 57.9	 3.5	 2.4	 67	 862

64. Peaches Dried        	1 lb.   	15.7	 2889	 8.5	  87	 1.7	173	 86.8	 1.2	 4.3	 55	  57

65. Prunes Dried         	1 lb.   	 9.0	 4284	12.8	  99	 2.5	154	 85.7	 3.9	 4.3	 65	 257

66. Raisins Dried        	15 oz.  	 9.4	 4524	13.5	 104	 2.5	136	  4.5	 6.3	 1.4	 24	 136

67. Peas Dried           	1 lb.   	 7.9	 5742	20.0	1367	 4.2	345	  2.9	28.7	18.4	162	NA

68. Lima Beans Dried     	1 lb.   	 8.9	 5097	17.4	1055	 3.7	459	  5.1	26.9	38.2	 93	NA

69. Navy Beans Dried     	1 lb.   	 5.9	 7688	26.9	1691	11.4	792	NA   	38.4	24.6	217	NA

70. Coffee               	1 lb.   	22.4	 2025	NA  	NA  	NA  	NA 	NA   	 4.0	 5.1	 50	NA

71. Tea                  	0.25 lb.	17.4	  652	NA  	NA  	NA  	NA 	NA   	NA  	 2.3	 42	NA

72. Cocoa                	8 oz.   	 8.6	 2657	 8.7	 237	 3.0	 72	NA   	 2.0	11.9	 40	NA

73. Chocolate            	8 oz.   	16.2	 1400	 8.0	  77	 1.3	 39	NA   	 0.9	 3.4	 14	NA

74. Sugar                	10 lb.  	51.2	 8773	34.9	NA  	NA  	NA 	NA   	NA  	NA  	NA 	NA

75. Corn Sirup           	24 oz.  	13.7	 4966	14.7	NA  	 0.5	 74	NA   	NA  	NA  	  5	NA

76. Molasses             	18 oz.  	13.6	 3752	 9.0	NA  	10.3	244	NA   	 1.9	 7.5	146	NA

77. Strawberry Preserves 	1 lb.   	20.5	 2213	 6.4	  11	 0.4	  7	  0.2	 0.2	 0.4	  3	NA

Daily allowance          	        	NA  	NA   	 3.0	  70	 0.8	 12	  5.0	 1.8	 2.7	 18	  75

Our dataset has the nutritional content of $77$ commodities, and in the final row, the daily minimum requirement of each of these nutrients.

### The Diet problem

Problem setup:

* Assume there are nutrients $i\in\left\{  1,...,m\right\}  $ (calories, protein, calcium, iron, etc.) that matter for a person's health, in such way that the minimum daily intake of nutrient $i$ should be $d_{i}$.

* Nutrients do not come as standalone elements, but are combined into various foods. Each unit of food $j\in\left\{  1,...,n\right\}$ yields a quantity $N_{ij}$ of nutrient $i\in\left\{1,...,m\right\}$. The dollar cost of food $j$ is $c_{j}$.

The problem is to find the diet that achieves the minimal intake of each nutrient at a cheapest price. If $q\in\mathbb{R}^{n}$ is a vector such that $q_{j}\geq0$ is the quantity of food $i$ purchased, the quantity of nutrient $i$ ingested is $\sum_{j=1}^{n}N_{ij}q_{j}$, and the cost of the diet is $\sum_{j=1}^{n}q_{j}c_{j}$. The optimal diet is therefore given by

\begin{align*}
\min_{q\geq0}  &  ~c^{\top}q\\
s.t.~  &  Nq\geq d.\nonumber
\end{align*}

Before we tackle this problem, let's look into the linear programming problem in standard form.

## A Crash Course on Linear Programming

### Linear programming in standard form

Let $c\in\mathbb{R}^{n}$, $d\in\mathbb{R}^{m}$, $A$ be a $m\times n$ matrix, and consider the following problem

\begin{align}
V_{P} = \max_{x\in\mathbb{R}_{+}^{n}}  & \, c^{\top} x \\
s.t.~Ax  &  \leq d
\end{align}

This problem is a *linear programming problem*, as the objective function, namely $x\rightarrow c^{\top}x$ is linear, and as the constraint, namely $x\in\mathbb{R}_{+}^{n}$ and $Ax=d$ are also linear (or more accurately, affine). This problem is called the *primal program*, for reasons to be explained soon. The set of $x$'s that satisfy the constraint are called *feasible solutions*; the set of solutions of the primal problem are called *optimal solutions*.

### Remarks

* The previous diet problem can be reformulate into this problem - why?

* A problem does not necessarly have a feasible solution (e.g. if $A=0$ and $d\neq0$), in which case (by convention) $V_{P}=-\infty$.

* The whole space may be solution (e.g. if $A=0$ and $d=0$), in which case $V_{P}=+\infty$.

### Duality

There is a powerful tool called duality which provides much insight into the analysis of the primal problem. The idea is to rewrite the problem as

\begin{align*}
V_{P}=\max_{x\in\mathbb{R}_{+}^{n}}\left\{  c^{\top}x+L_{P}\left(
d-Ax\right)  \right\}
\end{align*}

where $L_{P}\left(z\right)$ is a penalty function whose value is zero if the constraint is met, that is if $z=0$, and $-\infty$ if it is not, namely if $z\neq0$. The simplest choice of such penalty function is given by $L_{P}\left(  z\right)  =\min_{y\in\mathbb{R}^{m}}\left\{  z^{\top}y\right\}$. One has

\begin{align*}
V_{P}=\max_{x\in\mathbb{R}_{+}^{n}}\min_{y\in\mathbb{R}^{m}}\left\{c^{\top}x+\left(  d-Ax\right)  ^{\top}y\right\}  .
\end{align*}

However, the minimax inequality $\max_{x}\min_{y}\leq\min_{y}\max_{x}$ always holds, thus

\begin{align*}
V_{P}  &  \leq\min_{y\in\mathbb{R}^{m}}\max_{x\in\mathbb{R}_{+}^{n}}\left\{
c^{\top}x+\left(  d-Ax\right)  ^{\top}y\right\}  =\min_{y\in\mathbb{R}^{m}
}\max_{x\in\mathbb{R}_{+}^{n}}\left\{  x^{\top}\left(  c-A^{\top}y\right)
+d^{\top}y\right\} \\
&  \leq\min_{y\in\mathbb{R}^{m}}\left\{  d^{\top}y+L_{D}\left(  c-A^{\top
}y\right)  \right\}  =:V_{D}
\end{align*}

where $L_{D}\left(z\right) = \max_{x\in\mathbb{R}_{+}^{n}}\left\{x^{\top}z\right\}$ is equal to $0$ if $z\in\mathbb{R}_{-}^{n}$, and to $+\infty$ if not. Therefore, the value $V_{D}$ is expressed by the *dual program*

\begin{align}
V_{D}=\min_{y\in\mathbb{R}^{m}}  & \,  d^{\top}y, \\
s.t.~A^{\top}y  &  \geq c
\end{align}

and the weak duality inequality $V_{P}\leq V_{D}$ holds. It turns out that as
soon as either the primal or dual program has an optimal solution, then both
programs have an optimal solution and the values of the two programs coincide,
so the weak duality becomes an equality $V_{P}=V_{D}$ called strong duality.
Further, if $x^{\ast}\in\mathbb{R}_{+}^{n}$ is an optimal primal solution, and
$y^{\ast}\in\mathbb{R}^{m}$ is an optimal dual solution, then complementary
slackness holds, that is $x_{i}^{\ast}>0$ implies $\left(  A^{\top}y^{\ast
}\right)  _{i}=c_{i}$.

### Duality theorem
We summarize these results into the following statement.

---
**Theorem.** In the setting described above:

1. The weak duality inequality holds:

\begin{align}
V_{P}\leq V_{D}.
\end{align}

2. As soon as the primal or the dual program have an optimal solution, then both programs have an optimal solution, and strong duality holds:

\begin{align}
V_{P}=V_{D}.
\end{align}

3. If $x^{\ast}\in\mathbb{R}_{+}^{n}$ is an optimal primal solution, and $y^{\ast}\in\mathbb{R}^{m}$ is an optimal dual solution, then complementary slackness holds:

\begin{align}
x_{i}^{\ast}>0\text{  implies  }\left(  A^{\top}y^{\ast}\right)  _{i}=c_{i}.
\end{align}

---

## The diet problem (revisited)

Recall the optimal diet problem

\begin{align*}
\min_{q\geq0}  &  \, c^{\top}q\\
s.t.~  &  Nq\geq d.
\end{align*}

which has minimax formulation $\min_{q\geq0}\max_{\pi\geq0}c^{\top}q+d^{\top}\pi-q^{\top}N^{\top}\pi$, so the dual is

\begin{align*}
\max_{\pi\geq0}  & \, d^{\top}\pi\\
s.t.~  &  N^{\top}\pi\leq c
\end{align*}

Interpretation: imagine that there is a new firm called Nutrient Shoppe, who sells raw nutrients. Let $\pi_{i}$ be the price of nutrient $i$. The cost of the diet is $d^{\top}\pi$. Consumer purchase raw nutrients and can generate
"synthetic foods". The cost of the synthetic version of food $j$ is $\sum_{i=1}^{m}N_{ij}\pi_{i}=\left(N^{\intercal}\pi\right)_{j}$. The constraint thus means that each "synthetic food" is more affordable than
its natural counterpart.

The duality means that it is possible to price the nutrients so that the
synthetic foods are cheaper than the natural ones, in such a way that the
price of the synthetic diet equals the price of the natural diet.

Complementary slackness yields:

* $q_{j}>0$ implies $\left(  N^{\intercal}\pi\right)  _{j}=c_{j}$; that
is, if natural food $j$ is actually purchased, then the prices of its
synthetic and natural versions coincide

* $\pi_{i}>0$ implies $\left(  Nq\right)  _{i}=d_{i}$; that is, if
nutrient $i$ has a positive price, then the natural diet has the
"just right" amount.

### Solving the diet problem

To solve the primal problem we need to construct the objects $c$, $N$ and $d$. $c$ is simply a vector of ones, the size of the number of commodities. $N$ is a matrix of amount of nutrients in each commodity. $d$ is the required daily allowance of each nutrient.

```R
nbCommodities = length(which(thedata[, 1] != "")) - 1
names = thedata[1:nbCommodities, 1]
themat = matrix(as.numeric(thedata[, 3:13]), ncol = 11)
themat[is.na(themat)] = 0
N = t(themat[1:nbCommodities, 3:11])
d = themat[(nbCommodities + 1), 3:11]
c = rep(1, nbCommodities)
```

We will be using [Gurobi](http://www.gurobi.com/) as our linear programming solver for the entirety of this course. Whilst Gurobi is not neccassary-as there are many available linear programming solvers-it is lightning quick and free for acedemics.

So let's first load up the Gurobi library.

```R
library(gurobi)
```

    Loading required package: slam
    Warning message:
    "package 'slam' was built under R version 3.6.2"

And look at the functions syntax.

```R
?gurobi
```

```R
?gurobi_model
```

```R
?gurobi_params
```

So mapping from gurobis notation to ours,
* `A` = $N$
* `obj` = $c$
* `sense` = '$>$'
* `rhs` = $d$
* `modelsense` = '$\min$'

```R
result = gurobi(list(A = N, obj = c, sense = ">", rhs = d, modelsense = "min"))  #, params = list(OutputFlag = 0))
```

    Gurobi Optimizer version 9.0.0 build v9.0.0rc2 (win64)
    Optimize a model with 9 rows, 77 columns and 570 nonzeros
    Model fingerprint: 0x7ae6b743
    Coefficient statistics:
      Matrix range     [1e-01, 5e+03]
      Objective range  [1e+00, 1e+00]
      Bounds range     [0e+00, 0e+00]
      RHS range        [8e-01, 8e+01]
    Presolve removed 0 rows and 47 columns
    Presolve time: 0.00s
    Presolved: 9 rows, 30 columns, 240 nonzeros

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
           0    0.0000000e+00   1.384688e+01   0.000000e+00      0s
           5    1.0866228e-01   0.000000e+00   0.000000e+00      0s

    Solved in 5 iterations and 0.00 seconds
    Optimal objective  1.086622782e-01

Let's see what is in the `result` object:

```R
str(result)
```

    List of 13
     $ status      : chr "OPTIMAL"
     $ runtime     : num 0.001
     $ itercount   : num 5
     $ baritercount: int 0
     $ nodecount   : num 0
     $ objval      : num 0.109
     $ x           : num [1:77] 0.0295 0 0 0 0 ...
     $ slack       : num [1:9] 0 -77.4 0 -48.5 0 ...
     $ rc          : num [1:77] 0 0.845 0.296 0.859 0.489 ...
     $ pi          : num [1:9] 0.00877 0 0.03174 0 0.0004 ...
     $ vbasis      : int [1:77] 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 ...
     $ cbasis      : int [1:9] -1 0 -1 0 -1 0 -1 0 -1
     $ objbound    : num 0.109

We are after the optimal solutions `x`, the dual solution `pi` and the value function `objval`

```R
q_yearly = result$x * 365  # convert into yearly cost
pi = result$pi
cost_daily = result$objval
```

Our optimal solution (including only foods which are non-zero)

```R
print("*** Optimal solution ***")
toKeep = which(q_yearly != 0)
foods = q_yearly[toKeep]
names(foods) = names[toKeep]
foods
print(paste0("Total cost (optimal)= ", sum(q_yearly * c)))
```

    [1] "*** Optimal solution ***"

1. Wheat Flour (Enriched)

10.7744575119182

30. Liver (Beef)

0.690783411107421

46. Cabbage

4.09326886484288

52. Spinach

1.8277960703547

69. Navy Beans Dried

22.275425687243

    [1] "Total cost (optimal)= 39.6617315454663"

As promised, we achieve the minimum cost bundle at $\$39.67$ per year in $1939$ dollars. We can compare this to Stigler's solutions which was:

|Food| Annual Quantities| Annual Cost|
| ---------- | ------------------ | ------------ |
| Wheat Flour | 	370 lb.|   \$13.33 |
| Evaporated Milk | 	57 cans |	  \$3.84 |
|Cabbage| 	111 lb. 	  |\$4.11|
|Spinach| 	23 lb. 	  |\$1.85|
|Dried Navy Beans| 	285 lb. |	\$16.80|
|Total Annual Cost| 	  	| \$39.93 |

## Recovering the solution with the glpk solver

```R
library(Rglpk)
resGlpk = Rglpk_solve_LP(obj = c, mat = N, dir = rep(">", length(d)), rhs = d, bounds = NULL, max = FALSE, control = list())
print(resGlpk$optimum * 365)
```

    Warning message:
    "package 'Rglpk' was built under R version 3.6.2"Using the GLPK callable library version 4.47

    [1] 39.66173

```R

```


Exercise material of the MSc-level course **Numerical Methods in Geotechnical Engineering**.
Held at Technische Universität Bergakademie Freiberg.

Comments to:

*Prof. Dr. Thomas Nagel
Chair of Soil Mechanics and Foundation Engineering
Geotechnical Institute
Technische Universität Bergakademie Freiberg.*

https://tu-freiberg.de/en/soilmechanics

```python
import numpy as np
import matplotlib.pyplot as plt

#Some plot settings
plt.style.use('seaborn-deep')
plt.rcParams['lines.linewidth']= 2.0
plt.rcParams['lines.color']= 'black'
plt.rcParams['legend.frameon']=True
plt.rcParams['font.family'] = 'serif'
plt.rcParams['legend.fontsize']=14
plt.rcParams['font.size'] = 14
plt.rcParams['axes.spines.right'] = False
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.left'] = True
plt.rcParams['axes.spines.bottom'] = True
plt.rcParams['axes.axisbelow'] = True
plt.rcParams['figure.figsize'] = (12, 6)
```

# Exercise 7 - 1D Consolidation, Terzaghi: Crank-Nicolson version

## Governing differential equation

Neglecting lateral displacements and fluid flow

$$
	\epsilon_{xx} = \epsilon_{yy} = 0 \quad \text{and} \quad q_x = q_y = 0
$$

the volume balance simplifies to

$$
	\dot{\epsilon}_{zz} = \frac{\partial q_z}{\partial z}
$$

Now we substitute the constitutive law (linear elasticity) of the solid and Darcy's law for the fluid motion into the volume balance and get

$$
    \dot{\sigma}_{zz}^\text{eff} = E_\text{S} \dot{\epsilon}_{zz} \quad \text{and} \quad \frac{\partial q_z}{\partial z} = - \frac{k}{\mu_\text{F}} \frac{\partial^2 p}{\partial z^2} \quad \text{ergibt} \quad
		 \frac{\dot{\sigma}_{zz}^\text{eff}}{E_\text{S}} = - \frac{k}{\mu_\text{F}} \frac{\partial^2 p}{\partial z^2}
$$

Assuming constant total stress (constant external loads) we find with the use of the effective stress principle

$$
	\sigma_{zz} = \sigma_{zz}^\text{eff} + p = \text{const} \quad \rightarrow \quad \dot{\sigma}_{zz} = 0 = \dot{\sigma}_{zz}^\text{eff} + \dot{p}
$$

Substitution yields the partial, linear and homogeneous differential equation

$$
	\dot{p} = \frac{E_\text{S} k}{\mu_\text{F}} p_{,zz} = \frac{E_\text{S} K_\text{D}}{\gamma_\text{F}} p_{,zz} = c_\text{v} p_{,zz} \quad \text{with } K_\text{D} = \frac{k\rho_\text{F} g}{\mu_\text{F}}
$$

$[k] = \text{m}^2$: intrinsic permeability

$[K_\text{D}] = \text{m s}^{-1}$: hydraulic conductivity

$[c_\text{v}] = \text{m}^2\text{s}^{-1}$: coefficient of consolidation

## Weak form

The pore pressure can have (the essential/Dirichlet) boundary conditions in the form:

$$
    p = \bar{p}\ \forall z \in \partial \Omega_\mathrm{D}
$$

We now introduce a test function $\eta$ which vanishes where the pore pressure is given

$$
    \eta = 0\ \forall z \in \partial \Omega_\mathrm{D}
$$

and construct the weak form (using integration by parts):

\begin{align}
   0 &= \int \limits_0^H \eta \left[\dot{p} - c_\text{v} p_{,zz} \right] \text{d}z
   \\
   &= \int \limits_0^H \left[\eta \dot{p} - \left( \eta c_\text{v} p_{,z} \right)_{,z} + \eta_{,z} c_\text{v} p_{,z} \right] \, \text{d}z
   \\
   &= \int \limits_0^H \left[\eta \dot{p} + \eta_{,z} c_\text{v} p_{,z} \right] \, \text{d}z + \left[ \eta E_\text{S} q_z \right]^H_0
\end{align}

where the natural/Neumann boundary conditions have appeared.

## Finite elements in 1D

We have a soil column of height $H$ on top of the bed rock at $z=0$.

We first create an element class. An element knows the number of nodes it has, their IDs in the global node vector, and the coordinates of its nodes. Linear elements have 2 nodes and 2 quadrature points, quadratic elements 3 nodes and 3 quadrature points. The natural coordinates of the element run from -1 to 1, and the quadrature points and weights are directly taken from Numpy.

```python
#element class
class line_element():#local coordinates go from -1 to 1
    #takes number of nodes, global nodal coordinates, global node ids
    def __init__(self, nnodes=2, ncoords=[0.,1.], nids=[0,1]):
        self.__nnodes = nnodes
        if (len(ncoords) != self.__nnodes):
            raise Exception("Number of coordinates does not match number \
                            of nodes of element (%i vs of %i)" %(self.__nnodes,len(ncoords)))
        else:
            self.__coords = np.array(ncoords)

        self.__natural_coords = (self.__coords-self.__coords[0])/(self.__coords[-1]-self.__coords[0])*2. - 1.

        if (len(nids) != self.__nnodes):
            raise Exception("Number of node IDs does not match number \
                            of nodes of element (%i vs of %i)" %(self.__nnodes,len(nids)))
        else:
            self.__global_ids = np.array(nids)
        self.__quad_degree = self.__nnodes
        self.__quad_points, self.__quad_weights = np.polynomial.legendre.leggauss(self.__quad_degree)

```

Next, we wish to generate a one-dimensional mesh by specifying the length of a line, the number of elements into which the mesh is to be split, and the number of nodes per element.

```python
def number_of_nodes(nelems,nodes_per_elem):
    return nelems*nodes_per_elem - (nelems - 1)

def generate_mesh(domain_length,nelems,nodes_per_elem):
    nn = number_of_nodes(nelems,nodes_per_elem)
    #coordinate vector of global nodes
    global_nodal_coordinates = np.linspace(0.,domain_length,nn)
    global_solution = np.array([0.]*nn)

    #generate elements
    element_vector = []
    for i in range(nelems):
        node_start = (nodes_per_elem-1)*i
        element_vector.append(
            line_element(nodes_per_elem,
                         global_nodal_coordinates[node_start:node_start+nodes_per_elem],
                         list(range(node_start,node_start+nodes_per_elem))))

    return global_nodal_coordinates, element_vector, global_solution
```

## Shape functions in 1D

As in exercise 06, we allow linear and higher-order shape functions.

```python
#N
def shape_function(element_order,xi):
    if (element_order == 2): #-1,1
            return np.array([(1.-xi)/2., (1.+xi)/2.])
    elif (element_order == 3): #-1, 0, 1
            return np.array([(xi - 1.)*xi/2., (1-xi)*(1+xi), (1+xi)*xi/2.])
#dN_dxi
def dshape_function_dxi(element_order,xi):
    if (element_order == 2): #-1,1
        return np.array([-0.5*xi/xi, 0.5*xi/xi])  #xi only later for plotting dimensions
    elif (element_order == 3):#-1,0,1
        return np.array([xi - 0.5,-2.*xi,xi + 0.5])

#dz_dxi
def element_jacobian(element,xi):
    element_order = element._line_element__nnodes
    Jacobian = 0.
    Jacobian += dshape_function_dxi(element_order,xi).dot(element._line_element__coords)
    return Jacobian

#dN_dz
def grad_shape_function(element,xi):
    element_order = element._line_element__nnodes
    Jac = element_jacobian(element,xi)
    return dshape_function_dxi(element_order,xi)/Jac

```

## Time and space discretization, Picard iterations

Using a forward Euler approach for simplicity we find the time-discrete weak form as ($p_{n+1} \equiv p$)

$$
    \int \limits_0^H \left[ N_i \frac{1}{\Delta t} N_k + \nabla N_i \frac{c_\text{v}}{2} \nabla N_k\right] \, \text{d}z\ \hat{p}_k = N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=H} \delta_{n_\text{n}i} - N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=0} \delta_{i0} +  \int \limits_0^H N_i \frac{1}{\Delta t} N_k \, \text{d}z\ \hat{p}_{n,k} - \int \limits_0^H \left[ \nabla N_i \frac{c_\text{v}}{2} \nabla N_k \right] \, \text{d}z\ \hat{p}_{n,k}
$$

which leaves us with $n_\text{n}$ equations for the $n_\text{n}$ unknown nodal pressures $\hat{p}_k$. If any coefficients in the above are taken as pressure-dependent, the system could be solved repeatedly usind Picard iterations. For strong non-linearities, a Newton linearization would typically be used.

What we require now is the local assembler to calculate the stiffness matrix and the local right-hand side. Local integration is performed by Gauss quadrature:

$$
    \int \limits_{-1}^1 f(\xi)\,\text{d}\xi \approx \sum \limits_{i=1}^{n_\text{gp}} f(\xi_i) w_i
$$

## Local assember

```python
def Stiffness(z):
    E0 = 5.e6 #Pa
    return E0

def Conductivity(z):
    kf = 1.e-9#m/s
    return kf

def SpecificWeight(z):
    g = 9.81 #m/s²
    rhow = 1000. #kg/m³
    return g*rhow

def ConsolidationCoeff(z):#m²/s
    return Stiffness(z)*Conductivity(z)/SpecificWeight(z)
```

```python
def local_assembler(elem,dt,prev_sol):
    element_order = elem._line_element__nnodes
    K_loc = np.zeros((element_order,element_order))
    b_loc = np.zeros(element_order)
    z_nodes = elem._line_element__coords
    for i in range(elem._line_element__quad_degree):
        #local integration point coordinate
        xi = elem._line_element__quad_points[i]
        #shape function
        N = shape_function(element_order,xi)
        #gradient of shape function
        dN_dX = grad_shape_function(elem,xi)
        #determinant of Jacobian
        detJ = np.abs(element_jacobian(elem,xi))
        #integration weight
        w = elem._line_element__quad_weights[i]

        #global integration point coordinate (for spatially varying properties)
        z_glob = np.dot(N,z_nodes)
        #evaluation of local material/structural properties
        E = Stiffness(z_glob)
        #evaluation of local body force
        CV = ConsolidationCoeff(z_glob)

        #assembly of local stiffness matrix
        K_loc += (np.outer(N,N) / dt + np.outer(dN_dX,dN_dX) * CV/2)* w * detJ

        #assembly of local RHS
        p_prev = np.dot(N,prev_sol)#pressure in integration point
        grad_p_prev = np.dot(dN_dX,prev_sol)
        b_loc += (N * p_prev/dt - dN_dX * CV/2 * grad_p_prev) * w * detJ
    return K_loc,b_loc
```

## Global assembly

Now we can construct the global matrix system $\mathbf{K}\mathbf{u} = \mathbf{f}$ or $\mathbf{A}\mathbf{x}=\mathbf{b}$ (see lecture script).

```python
def global_assembler(nodes,elements,solution,dt):
    K_glob = np.zeros((len(nodes),len(nodes)))
    b_glob = np.zeros(len(nodes))
    for i,elem in enumerate(elements):
        start_id = elem._line_element__global_ids[0]
        end_id = elem._line_element__global_ids[-1]
        K_i, b_i = local_assembler(elem,dt,solution[start_id:end_id+1])
        K_glob[start_id:end_id+1,start_id:end_id+1] += K_i
        b_glob[start_id:end_id+1] += b_i
    return K_glob, b_glob
```

## Application of boundary conditions

First we apply flux boundary conditions

```python
def apply_Neumann_bc(b_glob,node_id,value):
    b_glob[node_id] += value
    return b_glob
```

Then we apply Dirichlet bc

```python
def apply_Dirichlet_bc(K_glob,b_glob,node_id,value):
    K_glob[node_id,:] = 0.# = K_glob[:,node_id] = 0.
    K_glob[node_id,node_id] = 1.
    b_glob[node_id] = value
    return K_glob, b_glob
```

## Application of initial conditions

Since we're dealing with a time-dependent problem (rate problem) we require initial conditions for the pore pressure, i.e. $p_0 = p(t=0)\ \forall\ z$. Due to the very specific assumptions in deriving this consolidation equation, the initial pressure is given by the (suddenly) applied load:

$$
    p_0 = \sigma_{zz} = \text{const.}
$$

```python
def apply_initial_conditions(solution,sig_v):
    solution *= 0.
    solution += sig_v
    return
```

## Time loop and problem solution

We now establish the time loop and in each time step perform the global assembly, apply a vanishing traction on the top and constrain the displacement at the bottom to zero.

```python
def time_loop(dt,nodes,elements,solution):
    #Startwerte
    t_end = 100*24*60*60 #s
    absolute_tolerance = 1.e-6
    apply_initial_conditions(solution,200.e3)
    y = [solution] #create a list that will hold the solution vectors at all time points
    times = np.array([0.])
    #
    while times[-1]+dt < t_end: #repeat the loop as long as the final time step is below the end point
        times = np.append(times,times[-1]+dt) #here define the next time point as the previous time point plus the time increment dt
        K, f = global_assembler(nodes,elements,y[-1],dt)
        #f = apply_Neumann_bc(f,len(nodes)-1,0)
        K, f = apply_Dirichlet_bc(K, f, len(nodes)-1, 0.)#free draining top
        solution = np.linalg.solve(K,f)
        y.append(solution) #append the new found solution to the solution vector
    return times, y
```

```python
#spatial discretization
H = 10.
nel = 20
n_per_el = 3
nodes,elements,solution=generate_mesh(H,nel,n_per_el)
```

```python
times, sols = time_loop(24*60.*60.,nodes,elements,solution)
```

```python
plt.xlabel('$z$ / m')
plt.ylabel('$p$ / kPa')
plt.title('Finite element solution')
plt.plot(nodes, sols[0]/1.e3, marker='o', label='$t = %i$ h' %(times[0]/60/60))
plt.plot(nodes, sols[1]/1.e3, marker='o', label='$t = %i$ h' %(times[1]/60/60))
plt.plot(nodes, sols[10]/1.e3, marker='o', label='$t = %i$ h' %(times[10]/60/60))
plt.plot(nodes, sols[-1]/1.e3, marker='o', label='$t = %i$ d' %(times[-1]/60/60/24))
#plt.plot(nodes, sols[200]/1.e3, marker='o', label='$t = %i$ d' %(times[200]/60/60/24))
#plt.plot(nodes, sols[-1]/1.e3, marker='o', label='$t = %i$ d' %(times[-1]/60/60/24))
plt.legend();
```

### Convergence study

Let's do a simple convergence study.

```python
dts = [12*3600,24.*3600,48.*3600]
nels = [10,20,40]
H = 10.

fig, ax = plt.subplots(nrows=3,ncols=3,figsize=(18,18))

for i,dt in enumerate(dts):
    for j,nel in enumerate(nels):
        #print("Running (n,dt) combination ", dt, nel)
        number_of_elements = nel
        nodes_per_element = 2
        nodes,elements,solution=generate_mesh(H,number_of_elements,nodes_per_element)
        times, sols = time_loop(dt,nodes,elements,solution)
        ax[i][j].plot(nodes, sols[0]/1.e3, marker='o', label='$t = %i$ h' %(times[0]/60/60))
        ax[i][j].plot(nodes, sols[1]/1.e3, marker='o', label='$t = %i$ h' %(times[1]/60/60))
        ax[i][j].plot(nodes, sols[10]/1.e3, marker='o', label='$t = %i$ h' %(times[10]/60/60))
        ax[i][j].plot(nodes, sols[-1]/1.e3, marker='o', label='$t = %i$ d' %(times[10]/60/60/24))
        ax[i][j].set_xlabel('$z$ / m')
        ax[i][j].set_ylabel('$p$ / kPa')
        ax[i][j].set_title('dt = %i h, nel = %i' %(dt/3600,nel))
        ax[i][j].legend(fontsize=10)
fig.tight_layout()
```

We observe that time step size and discretization size act together, and that unsuitable choices can lead to unphysical oscillations.

## Tasks

* What happens if you switch the element order to 3 in the last example?
* Explicit solutions can be coded so as to avoid matrix inversion.

```python

```


# Table of Contents

1  Presentation

2  Computations

2.1  Conclusion

# Presentation

I want to reproduce the symbolic (algebraic) computations done in §5.A of [L.S.'s PhD thesis](http://www.cmap.polytechnique.fr/~sacchelli/).

I want to only use a free and open-source software, so I'm using [Python 3](https://docs.python.org/3) with the [Sympy](https://sympy.org) module.

```python
%load_ext watermark
%watermark -v -m -p sympy -g
```

    CPython 3.6.5
    IPython 6.4.0

    sympy 1.1.1

    compiler   : GCC 7.3.0
    system     : Linux
    release    : 4.15.0-23-generic
    machine    : x86_64
    processor  : x86_64
    CPU cores  : 4
    interpreter: 64bit
    Git hash   : 3f62e1141f480e65f685cabe81f7a79b17bcd06d

```python
from sympy import *
```

```python
init_printing(use_latex='mathjax')
```

Just a small introduction to SymPy: it works by using Python expressions on formal variables.
First, we define variables, and then we can solve linear equations for example:

```python
var("x y z")
```

$$\left ( x, \quad y, \quad z\right )$$

```python
solve(x + 2 + 19)
```

$$\left [ -21\right ]$$

```python
solve(x + 2*y + 19)
```

$$\left [ \left \{ x : - 2 y - 19\right \}\right ]$$

# Computations

We start by defining $b_1 > 0$ and the other variables.

```python
b1 = symbols("b1", positive=True)
```

We need a lot of variables.

```python
var("h1 h2 h3 h4 k131 k132 k141 k142 k231 k232 k241 k242 rest1 rest2 t s a b");
```

Then we can start to follow Ludovic's notebook and define the first expressions.

```python
J = Matrix([[0, -b1], [b1, 0]])
J
```

$$\left[\begin{matrix}0 & - b_{1}\\b_{1} & 0\end{matrix}\right]$$

```python
h0 = Matrix([h1, h2])
h0
```

$$\left[\begin{matrix}h_{1}\\h_{2}\end{matrix}\right]$$

$\hat{h}$ is defined as $t \mapsto \exp(t J) . h_0$ and $\hat{x}$ as $t \mapsto \int_{s=0}^{s=t} \hat{h}(s) \mathrm{d}s$.

```python
hhat = Lambda(t, (t * J).exp() * h0)
hhat
```

$$\left( t \mapsto \left[\begin{matrix}h_{1} \left(\frac{1}{2} e^{i b_{1} t} + \frac{1}{2} e^{- i b_{1} t}\right) + h_{2} \left(\frac{i}{2} e^{i b_{1} t} - \frac{i}{2} e^{- i b_{1} t}\right)\\h_{1} \left(- \frac{i}{2} e^{i b_{1} t} + \frac{i}{2} e^{- i b_{1} t}\right) + h_{2} \left(\frac{1}{2} e^{i b_{1} t} + \frac{1}{2} e^{- i b_{1} t}\right)\end{matrix}\right] \right)$$

```python
xhat = Lambda(t, integrate(hhat(s), (s, 0, t)))
xhat
```

$$\left( t \mapsto \left[\begin{matrix}- \frac{h_{2}}{b_{1}} + \frac{1}{4 b_{1}^{2}} \left(\left(- 2 i b_{1} h_{1} + 2 b_{1} h_{2}\right) e^{i b_{1} t} + \left(2 i b_{1} h_{1} + 2 b_{1} h_{2}\right) e^{- i b_{1} t}\right)\\\frac{h_{1}}{b_{1}} + \frac{1}{4 b_{1}^{2}} \left(\left(- 2 b_{1} h_{1} - 2 i b_{1} h_{2}\right) e^{i b_{1} t} + \left(- 2 b_{1} h_{1} + 2 i b_{1} h_{2}\right) e^{- i b_{1} t}\right)\end{matrix}\right] \right)$$

$\hat{z}$ is slightly more complex:

```python
integrand = simplify(b1 * (hhat(s)[0] * xhat(s)[1] - hhat(s)[1] * xhat(s)[0] )/2)
integrand
```

$$\frac{h_{1}^{2}}{4} e^{i b_{1} s} - \frac{h_{1}^{2}}{2} + \frac{h_{1}^{2}}{4} e^{- i b_{1} s} + \frac{h_{2}^{2}}{4} e^{i b_{1} s} - \frac{h_{2}^{2}}{2} + \frac{h_{2}^{2}}{4} e^{- i b_{1} s}$$

```python
zhat = Lambda(t, integrate(integrand, (s, 0, t)))
zhat
```

$$\left( t \mapsto t \left(- \frac{h_{1}^{2}}{2} - \frac{h_{2}^{2}}{2}\right) + \frac{1}{16 b_{1}^{2}} \left(\left(- 4 i b_{1} h_{1}^{2} - 4 i b_{1} h_{2}^{2}\right) e^{i b_{1} t} + \left(4 i b_{1} h_{1}^{2} + 4 i b_{1} h_{2}^{2}\right) e^{- i b_{1} t}\right) \right)$$

As we asked $b_1 > 0$, this won't get too complicated:

```python
factor(zhat(t))
```

$$- \frac{e^{- i b_{1} t}}{4 b_{1}} \left(h_{1}^{2} + h_{2}^{2}\right) \left(2 b_{1} t e^{i b_{1} t} + i e^{2 i b_{1} t} - i\right)$$

Two more expressions:

```python
exp21 = (3 * a * h1**2 + a * h2**2 + 2 * b * h1 * h2 + k131 * h1 * h3 + k141 * h1 * h4
         + k231 * h2 * h3 + k241 * h2 * h4 + rest1(h3, h4))
```

```python
exp22 = (b * h1**2 + 3 * b * h2**2 + 2 * a * h1 * h2 + k132 * h1 * h3 + k142 * h1 * h4
         + k232 * h2 * h3 + k242 * h2 * h4 + rest2(h3, h4))
```

Both terms `rest1` and `rest2` are not important, they only depend on $h_3$ and $h_4$ and will vanish as soon as we only differentiate with respect to $h_1$ and $h_2$.

So far so good. Next cell:

```python
C10 = 2 * (pi / b1) * Matrix([h1, h2])
C10
```

$$\left[\begin{matrix}\frac{2 \pi}{b_{1}} h_{1}\\\frac{2 \pi}{b_{1}} h_{2}\end{matrix}\right]$$

```python
A0 = simplify(Matrix([
    [diff(exp21, h1), diff(exp21, h2)],
    [diff(exp22, h1), diff(exp22, h2)],
]))
A0
```

$$\left[\begin{matrix}6 a h_{1} + 2 b h_{2} + h_{3} k_{131} + h_{4} k_{141} & 2 a h_{2} + 2 b h_{1} + h_{3} k_{231} + h_{4} k_{241}\\2 a h_{2} + 2 b h_{1} + h_{3} k_{132} + h_{4} k_{142} & 2 a h_{1} + 6 b h_{2} + h_{3} k_{232} + h_{4} k_{242}\end{matrix}\right]$$

`rest1` and `rest2` already vanished.

```python
jacobian = simplify(Matrix([
    [diff(exp21, h1), diff(exp21, h2), C10[0]],
    [diff(exp22, h1), diff(exp22, h2), C10[1]],
    [diff(zhat(2 * pi / b1), h1), diff(zhat(2 * pi / b1), h2), 0],
]))
jacobian
```

$$\left[\begin{matrix}6 a h_{1} + 2 b h_{2} + h_{3} k_{131} + h_{4} k_{141} & 2 a h_{2} + 2 b h_{1} + h_{3} k_{231} + h_{4} k_{241} & \frac{2 \pi}{b_{1}} h_{1}\\2 a h_{2} + 2 b h_{1} + h_{3} k_{132} + h_{4} k_{142} & 2 a h_{1} + 6 b h_{2} + h_{3} k_{232} + h_{4} k_{242} & \frac{2 \pi}{b_{1}} h_{2}\\- \frac{2 \pi}{b_{1}} h_{1} & - \frac{2 \pi}{b_{1}} h_{2} & 0\end{matrix}\right]$$

We need one more variable to solve an equation.

```python
var('dt')
```

$$dt$$

```python
tc = factor(solve(Equality((jacobian + Matrix([[dt,0,0], [0,dt,0], [0,0,0]])).det(), 0), dt)[0])
```

```python
tc
```

$$- \frac{1}{h_{1}^{2} + h_{2}^{2}} \left(2 a h_{1}^{3} + 2 a h_{1} h_{2}^{2} + 2 b h_{1}^{2} h_{2} + 2 b h_{2}^{3} + h_{1}^{2} h_{3} k_{232} + h_{1}^{2} h_{4} k_{242} - h_{1} h_{2} h_{3} k_{132} - h_{1} h_{2} h_{3} k_{231} - h_{1} h_{2} h_{4} k_{142} - h_{1} h_{2} h_{4} k_{241} + h_{2}^{2} h_{3} k_{131} + h_{2}^{2} h_{4} k_{141}\right)$$

I can compare by copying the result from the document:

```python
tc2 = (-1 / (h1**2 + h2**2)) * (
      2 * a * h1**3
    + 6 * a * h1**2 * h2
    - 4 * b * h1**2 * h2
    + 2 * a * h1 * h2**2
    + 2 * b * h2**3
    + h2**2 * h3 * k131
    - h1 * h2 * h3 * k132
    + h2**2 * h4 * k141
    - h1 * h2 * h4 * k142
    - h1 * h2 * h3 * k231
    + h1**2 * h3 * k232
    - h1 * h2 * h4 * k241
    + h1**2 * h4 * k242 )
```

Drat, they are not equal! We might have a mistake somewhere, even though I just CAN'T find it!

```python
simplify(tc - tc2)
```

$$\frac{6 h_{1}^{2} h_{2} \left(a - b\right)}{h_{1}^{2} + h_{2}^{2}}$$

Let's use the one from Ludovic's notebook.

```python
tc = tc2
```

Next cell.

```python
A12 = simplify(A0 + Matrix([[tc, 0], [0, tc]]))
```

```python
var("u1 u2 u5")
```

$$\left ( u_{1}, \quad u_{2}, \quad u_{5}\right )$$

```python
Psi = Lambda((u1, u2, u5), simplify(
    u5 * Matrix(A12.dot(Matrix([h1, h2]))).dot(Matrix([h2, -h1])) + 2 * pi / b1 * ( h1**2 + h2**2) * (h1 * u2 - h2 * u1)
))
```

```python
my_psi = factor(simplify(Psi(u1, u2, u5)))
my_psi
```

$$- \frac{1}{b_{1}} \left(- 2 a b_{1} h_{1}^{2} h_{2} u_{5} - 2 a b_{1} h_{2}^{3} u_{5} + 2 b b_{1} h_{1}^{3} u_{5} + 2 b b_{1} h_{1} h_{2}^{2} u_{5} + b_{1} h_{1}^{2} h_{3} k_{132} u_{5} + b_{1} h_{1}^{2} h_{4} k_{142} u_{5} - b_{1} h_{1} h_{2} h_{3} k_{131} u_{5} + b_{1} h_{1} h_{2} h_{3} k_{232} u_{5} - b_{1} h_{1} h_{2} h_{4} k_{141} u_{5} + b_{1} h_{1} h_{2} h_{4} k_{242} u_{5} - b_{1} h_{2}^{2} h_{3} k_{231} u_{5} - b_{1} h_{2}^{2} h_{4} k_{241} u_{5} - 2 \pi h_{1}^{3} u_{2} + 2 \pi h_{1}^{2} h_{2} u_{1} - 2 \pi h_{1} h_{2}^{2} u_{2} + 2 \pi h_{2}^{3} u_{1}\right)$$

Let's compare with the value from Ludovic's notebook:

```python
his_psi = (1 / b1) * (
    2 * h1**3 * (pi * u2 - b * b1 * u5 ) -
    h1**2 * (2 * h2 * pi * u1 - 2 * a * b1 * h2 * u5 + b1 * h3 * k132 * u5 +
             b1 * h4 * k142 * u5 ) +
    h2**2 * ( -2 * h2 * pi * u1 + 2 * a * b1 * h2 * u5 + b1 * h3 * k231 * u5 +
            b1 * h4 * k241 * u5 ) +
    h1 * h2 * (b1 * (h3 * k131 + h4 * k141 - h3 * k232 - h4 * k242) * u5 +
    2 * h2 * (pi * u2 - b * b1 * u5))
)
```

```python
simplify(my_psi - his_psi)
```

$$0$$

Ok, we have the same result so far! Great!

Next cell.

```python
var("nu0")
```

$$\nu_{0}$$

Here again, Sympy fails to solve the equation. It can solve on both lines separately, but cannot find a solution that satisfies both lines.

```python
s1 = solve(Eq((Matrix(A12.dot(Matrix([-h2, h1]))) + nu0 * C10)[0]), nu0)[0]
```

```python
s2 = solve(Eq((Matrix(A12.dot(Matrix([-h2, h1]))) + nu0 * C10)[1]), nu0)[0]
```

```python
solve(Eq(s1, s2))
```

$$\left [ \left \{ a : b\right \}, \quad \left \{ h_{1} : 0\right \}\right ]$$

It is not possible to impose these constraints. And Sympy fails to solve both equation simultaneously:

```python
solve(Eq(Matrix(A12.dot(Matrix([-h2, h1]))) + nu0 * C10, Matrix([0,0])), nu0)
```

$$\left [ \right ]$$

```python
nu = simplify(solve(Eq(Matrix(A12.dot(Matrix([-h2, h1]))) + nu0 * C10, Matrix([0,0])), nu0))
if nu == []:
    nu = var("nu")
```

We can continue by using a formal variable for $\nu$ (`nu`).

```python
v = Matrix([nu, -h2, h1])
```

```python
v
```

$$\left[\begin{matrix}\nu\\- h_{2}\\h_{1}\end{matrix}\right]$$

Let's finish.

```python
var("eta f F");
```

```python
d = [
    lambda F: -eta**2 * diff(F, eta) + eta * t * diff(F, t),
    lambda F: diff(F, h1),
    lambda F: diff(F, h2)
]
```

```python
d
```

    [<function __main__.<lambda>(F)>,
     <function __main__.<lambda>(F)>,
     <function __main__.<lambda>(F)>]

Next cell.

```python
var("i1 i2 g1 g2 dt1 dt2");
```

```python
g = eta * g1(t + eta * dt1, h1, h2) + eta**2 * g2(t, h1, h2)
g
```

$$\eta^{2} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \eta \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}$$

We have a sum to compute:

```python
res = 0
for i1 in range(0, 3):
    for i2 in range(0, 3):
        res += v[i1] * v[i2] * d[i1](d[i2](g))
```

```python
simplify(res)
```

$$\eta \left(\eta^{2} \nu^{2} \left(2 dt_{1} \eta \left. \frac{\partial}{\partial \xi_{1}} \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + \eta \left(dt_{1}^{2} \eta \left. \frac{\partial^{2}}{\partial \xi_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + 2 dt_{1} \left. \frac{\partial}{\partial \xi_{1}} \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + 2 \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) + 4 \eta \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} - t \left(\eta \frac{\partial}{\partial t} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \left. \frac{\partial}{\partial \xi_{1}} \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }}\right) + t \left(- dt_{1} \eta \left. \frac{\partial^{2}}{\partial \xi_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} - \eta \frac{\partial}{\partial t} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + t \left(\eta \frac{\partial^{2}}{\partial t^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \left. \frac{\partial^{2}}{\partial \xi_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }}\right)\right) - t \left(dt_{1} \eta \left. \frac{\partial^{2}}{\partial \xi_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + 2 \eta \frac{\partial}{\partial t} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \left. \frac{\partial}{\partial \xi_{1}} \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }}\right) + 2 \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right) - 2 \eta h_{1} \nu \left(dt_{1} \eta \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + 2 \eta \frac{\partial}{\partial h_{2}} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} - t \left(\eta \frac{\partial^{2}}{\partial h_{2}\partial t}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }}\right) + \frac{\partial}{\partial h_{2}} \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right) + 2 \eta h_{2} \nu \left(dt_{1} \eta \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{1}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }} + 2 \eta \frac{\partial}{\partial h_{1}} \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} - t \left(\eta \frac{\partial^{2}}{\partial h_{1}\partial t}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{1}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=dt_{1} \eta + t }}\right) + \frac{\partial}{\partial h_{1}} \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right) + h_{1}^{2} \left(\eta \frac{\partial^{2}}{\partial h_{2}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \frac{\partial^{2}}{\partial h_{2}^{2}}  \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right) - 2 h_{1} h_{2} \left(\eta \frac{\partial^{2}}{\partial h_{1}\partial h_{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \frac{\partial^{2}}{\partial h_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right) + h_{2}^{2} \left(\eta \frac{\partial^{2}}{\partial h_{1}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )} + \frac{\partial^{2}}{\partial h_{1}^{2}}  \operatorname{g_{1}}{\left (dt_{1} \eta + t,h_{1},h_{2} \right )}\right)\right)$$

Then a double derivative

```python
d2f = simplify((diff(res, eta, eta) / 2).subs(eta, 0))
```

Now, we should replace $g_1$ and $g_2$ with the following expression:

```python
g1 = Lambda((t, h1, h2), simplify(Matrix(list(xhat(t)) + [0])))
```

```python
g1(t, h1, h2)
```

$$\left[\begin{matrix}\frac{e^{- i b_{1} t}}{2 b_{1}} \left(i h_{1} - 2 h_{2} e^{i b_{1} t} + h_{2} + \left(- i h_{1} + h_{2}\right) e^{2 i b_{1} t}\right)\\\frac{e^{- i b_{1} t}}{2 b_{1}} \left(2 h_{1} e^{i b_{1} t} - h_{1} + i h_{2} - \left(h_{1} + i h_{2}\right) e^{2 i b_{1} t}\right)\\0\end{matrix}\right]$$

```python
g2 = Lambda((t, h1, h2), simplify(Matrix([exp21, exp22] + [zhat(t)])))
```

```python
g2(t, h1, h2)
```

$$\left[\begin{matrix}3 a h_{1}^{2} + a h_{2}^{2} + 2 b h_{1} h_{2} + h_{1} h_{3} k_{131} + h_{1} h_{4} k_{141} + h_{2} h_{3} k_{231} + h_{2} h_{4} k_{241} + \operatorname{rest_{1}}{\left (h_{3},h_{4} \right )}\\2 a h_{1} h_{2} + b h_{1}^{2} + 3 b h_{2}^{2} + h_{1} h_{3} k_{132} + h_{1} h_{4} k_{142} + h_{2} h_{3} k_{232} + h_{2} h_{4} k_{242} + \operatorname{rest_{2}}{\left (h_{3},h_{4} \right )}\\\frac{e^{- i b_{1} t}}{4 b_{1}} \left(- 2 b_{1} t \left(h_{1}^{2} + h_{2}^{2}\right) e^{i b_{1} t} + i h_{1}^{2} + i h_{2}^{2} - i \left(h_{1}^{2} + h_{2}^{2}\right) e^{2 i b_{1} t}\right)\end{matrix}\right]$$

Let's see if the replacement is possible:

```python
simplify(d2f)
```

$$h_{1}^{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{2}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{2}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) - 2 h_{1} h_{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{1}\partial h_{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) + 2 h_{1} \nu \left(t \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} - \frac{\partial}{\partial h_{2}} \operatorname{g_{1}}{\left (t,h_{1},h_{2} \right )}\right) + h_{2}^{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{1}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) - 2 h_{2} \nu \left(t \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{1}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} - \frac{\partial}{\partial h_{1}} \operatorname{g_{1}}{\left (t,h_{1},h_{2} \right )}\right)$$

```python
simplify(d2f.subs({
    g1: Lambda((t, h1, h2), simplify(Matrix(list(xhat(t)) + [0]))),
    g2: Lambda((t, h1, h2), simplify(Matrix([exp21, exp22] +[zhat(t)]))),
}))
```

$$h_{1}^{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{2}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{2}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) - 2 h_{1} h_{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{1}\partial h_{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) + 2 h_{1} \nu \left(t \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} - \frac{\partial}{\partial h_{2}} \operatorname{g_{1}}{\left (t,h_{1},h_{2} \right )}\right) + h_{2}^{2} \left(dt_{1} \left. \frac{\partial^{3}}{\partial \xi_{1}\partial h_{1}^{2}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} + \frac{\partial^{2}}{\partial h_{1}^{2}}  \operatorname{g_{2}}{\left (t,h_{1},h_{2} \right )}\right) - 2 h_{2} \nu \left(t \left. \frac{\partial^{2}}{\partial \xi_{1}\partial h_{1}}  \operatorname{g_{1}}{\left (\xi_{1},h_{1},h_{2} \right )} \right|_{\substack{ \xi_{1}=t }} - \frac{\partial}{\partial h_{1}} \operatorname{g_{1}}{\left (t,h_{1},h_{2} \right )}\right)$$

OK. This failed. But we can copy and paste this and the replacement of $g_1$ and $g_2$ will be effective:

```python
d2f = (
    h1**2*(dt1*diff(g1(t, h1, h2), t, h2, h2)
         + diff(g2(t, h1, h2), h2, h2))
    - 2*h1*h2*(dt1*diff(g1(t, h1, h2), t, h1, h2)
               + diff(g2(t, h1, h2), h1, h2))
    + 2*h1*nu*(t*diff(g1(t, h1, h2), t, h2)
               - diff(g1(t, h1, h2), h2))
    + h2**2*(dt1*diff(g1(t, h1, h2), t, h1, h1)
             + diff(g2(t, h1, h2), h1, h1))
    - 2*h2*nu*(t*diff(g1(t, h1, h2), t, h1)
               - diff(g1(t, h1, h2), h1))
)
```

And the same for $t$.

```python
t = 2 * pi / b1
```

```python
d2f
```

$$\left[\begin{matrix}2 a h_{1}^{2} + 6 a h_{2}^{2} - 4 b h_{1} h_{2} + 2 h_{1} \nu \left(i t \left(- \frac{1}{2} \left(e^{2 i b_{1} t} - 2 e^{i b_{1} t} + 1\right) e^{- i b_{1} t} + e^{i b_{1} t} - 1\right) - \frac{e^{- i b_{1} t}}{2 b_{1}} \left(e^{2 i b_{1} t} - 2 e^{i b_{1} t} + 1\right)\right) - 2 h_{2} \nu \left(t \left(- \frac{1}{2} \left(e^{2 i b_{1} t} - 1\right) e^{- i b_{1} t} + e^{i b_{1} t}\right) - \frac{e^{- i b_{1} t}}{2 b_{1}} \left(- i e^{2 i b_{1} t} + i\right)\right)\\- 4 a h_{1} h_{2} + 6 b h_{1}^{2} + 2 b h_{2}^{2} + 2 h_{1} \nu \left(t \left(- \frac{1}{2} \left(e^{2 i b_{1} t} - 1\right) e^{- i b_{1} t} + e^{i b_{1} t}\right) - \frac{e^{- i b_{1} t}}{2 b_{1}} \left(- i e^{2 i b_{1} t} + i\right)\right) - 2 h_{2} \nu \left(i t \left(\frac{1}{2} \left(e^{2 i b_{1} t} - 2 e^{i b_{1} t} + 1\right) e^{- i b_{1} t} - e^{i b_{1} t} + 1\right) - \frac{e^{- i b_{1} t}}{2 b_{1}} \left(- e^{2 i b_{1} t} + 2 e^{i b_{1} t} - 1\right)\right)\\- \frac{h_{1}^{2}}{2 b_{1}} \left(2 b_{1} t e^{i b_{1} t} + i e^{2 i b_{1} t} - i\right) e^{- i b_{1} t} - \frac{h_{2}^{2}}{2 b_{1}} \left(2 b_{1} t e^{i b_{1} t} + i e^{2 i b_{1} t} - i\right) e^{- i b_{1} t}\end{matrix}\right]$$

The replacement does not work apparently, so let's do it manually:

```python
d2f = Matrix([
[ 2*a*h1**2 + 6*a*h2**2 - 4*b*h1*h2 + 2*h1*nu*(I*t*(-(exp(2*I*b1*t) - 2*exp(I*b1*t) + 1)*exp(-I*b1*t)/2 + exp(I*b1*t) - 1) - (exp(2*I*b1*t) - 2*exp(I*b1*t) + 1)*exp(-I*b1*t)/(2*b1)) - 2*h2*nu*(t*(-(exp(2*I*b1*t) - 1)*exp(-I*b1*t)/2 + exp(I*b1*t)) - (-I*exp(2*I*b1*t) + I)*exp(-I*b1*t)/(2*b1))],
[-4*a*h1*h2 + 6*b*h1**2 + 2*b*h2**2 + 2*h1*nu*(t*(-(exp(2*I*b1*t) - 1)*exp(-I*b1*t)/2 + exp(I*b1*t)) - (-I*exp(2*I*b1*t) + I)*exp(-I*b1*t)/(2*b1)) - 2*h2*nu*(I*t*((exp(2*I*b1*t) - 2*exp(I*b1*t) + 1)*exp(-I*b1*t)/2 - exp(I*b1*t) + 1) - (-exp(2*I*b1*t) + 2*exp(I*b1*t) - 1)*exp(-I*b1*t)/(2*b1))],
[                                                                                -h1**2*(2*b1*t*exp(I*b1*t) + I*exp(2*I*b1*t) - I)*exp(-I*b1*t)/(2*b1) - h2**2*(2*b1*t*exp(I*b1*t) + I*exp(2*I*b1*t) - I)*exp(-I*b1*t)/(2*b1)]])
```

```python
d2Exp = simplify(d2f)
```

It still works well. And we removed the complex exponential, this result is purely real now!

```python
d2Exp
```

$$\left[\begin{matrix}2 a h_{1}^{2} + 6 a h_{2}^{2} - 4 b h_{1} h_{2} - \frac{4 \pi}{b_{1}} h_{2} \nu\\- 4 a h_{1} h_{2} + 6 b h_{1}^{2} + 2 b h_{2}^{2} + \frac{4 \pi}{b_{1}} h_{1} \nu\\- \frac{2 \pi}{b_{1}} \left(h_{1}^{2} + h_{2}^{2}\right)\end{matrix}\right]$$

```python
Psi
```

$$\left( \left ( u_{1}, \quad u_{2}, \quad u_{5}\right ) \mapsto 2 a h_{1}^{2} h_{2} u_{5} + 2 a h_{2}^{3} u_{5} - 2 b h_{1}^{3} u_{5} - 2 b h_{1} h_{2}^{2} u_{5} - h_{1}^{2} h_{3} k_{132} u_{5} - h_{1}^{2} h_{4} k_{142} u_{5} + h_{1} h_{2} h_{3} k_{131} u_{5} - h_{1} h_{2} h_{3} k_{232} u_{5} + h_{1} h_{2} h_{4} k_{141} u_{5} - h_{1} h_{2} h_{4} k_{242} u_{5} + h_{2}^{2} h_{3} k_{231} u_{5} + h_{2}^{2} h_{4} k_{241} u_{5} + \frac{2 \pi}{b_{1}} h_{1}^{3} u_{2} - \frac{2 \pi}{b_{1}} h_{1}^{2} h_{2} u_{1} + \frac{2 \pi}{b_{1}} h_{1} h_{2}^{2} u_{2} - \frac{2 \pi}{b_{1}} h_{2}^{3} u_{1} \right)$$

So now we can call $\Psi$ on the three components of this `d2Exp` :

```python
Psi(d2Exp[0], d2Exp[1], d2Exp[2])
```

$$- \frac{4 \pi}{b_{1}} a h_{1}^{2} h_{2} \left(h_{1}^{2} + h_{2}^{2}\right) - \frac{4 \pi}{b_{1}} a h_{2}^{3} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{4 \pi}{b_{1}} b h_{1}^{3} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{4 \pi}{b_{1}} b h_{1} h_{2}^{2} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{2 \pi}{b_{1}} h_{1}^{3} \left(- 4 a h_{1} h_{2} + 6 b h_{1}^{2} + 2 b h_{2}^{2} + \frac{4 \pi}{b_{1}} h_{1} \nu\right) - \frac{2 \pi}{b_{1}} h_{1}^{2} h_{2} \left(2 a h_{1}^{2} + 6 a h_{2}^{2} - 4 b h_{1} h_{2} - \frac{4 \pi}{b_{1}} h_{2} \nu\right) + \frac{2 \pi}{b_{1}} h_{1}^{2} h_{3} k_{132} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{2 \pi}{b_{1}} h_{1}^{2} h_{4} k_{142} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{2 \pi}{b_{1}} h_{1} h_{2}^{2} \left(- 4 a h_{1} h_{2} + 6 b h_{1}^{2} + 2 b h_{2}^{2} + \frac{4 \pi}{b_{1}} h_{1} \nu\right) - \frac{2 \pi}{b_{1}} h_{1} h_{2} h_{3} k_{131} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{2 \pi}{b_{1}} h_{1} h_{2} h_{3} k_{232} \left(h_{1}^{2} + h_{2}^{2}\right) - \frac{2 \pi}{b_{1}} h_{1} h_{2} h_{4} k_{141} \left(h_{1}^{2} + h_{2}^{2}\right) + \frac{2 \pi}{b_{1}} h_{1} h_{2} h_{4} k_{242} \left(h_{1}^{2} + h_{2}^{2}\right) - \frac{2 \pi}{b_{1}} h_{2}^{3} \left(2 a h_{1}^{2} + 6 a h_{2}^{2} - 4 b h_{1} h_{2} - \frac{4 \pi}{b_{1}} h_{2} \nu\right) - \frac{2 \pi}{b_{1}} h_{2}^{2} h_{3} k_{231} \left(h_{1}^{2} + h_{2}^{2}\right) - \frac{2 \pi}{b_{1}} h_{2}^{2} h_{4} k_{241} \left(h_{1}^{2} + h_{2}^{2}\right)$$

```python
simplify(expand(Psi(d2Exp[0], d2Exp[1], d2Exp[2]) / (1 / (1/b1 * 2 * (h1**2 + h2**2) * pi))))
```

$$\frac{4 \pi^{2}}{b_{1}^{3}} \left(b_{1} \left(- 8 a h_{1}^{6} h_{2} - 24 a h_{1}^{4} h_{2}^{3} - 24 a h_{1}^{2} h_{2}^{5} - 8 a h_{2}^{7} + 8 b h_{1}^{7} + 24 b h_{1}^{5} h_{2}^{2} + 24 b h_{1}^{3} h_{2}^{4} + 8 b h_{1} h_{2}^{6} + h_{1}^{6} h_{3} k_{132} + h_{1}^{6} h_{4} k_{142} - h_{1}^{5} h_{2} h_{3} k_{131} + h_{1}^{5} h_{2} h_{3} k_{232} - h_{1}^{5} h_{2} h_{4} k_{141} + h_{1}^{5} h_{2} h_{4} k_{242} + 2 h_{1}^{4} h_{2}^{2} h_{3} k_{132} - h_{1}^{4} h_{2}^{2} h_{3} k_{231} + 2 h_{1}^{4} h_{2}^{2} h_{4} k_{142} - h_{1}^{4} h_{2}^{2} h_{4} k_{241} - 2 h_{1}^{3} h_{2}^{3} h_{3} k_{131} + 2 h_{1}^{3} h_{2}^{3} h_{3} k_{232} - 2 h_{1}^{3} h_{2}^{3} h_{4} k_{141} + 2 h_{1}^{3} h_{2}^{3} h_{4} k_{242} + h_{1}^{2} h_{2}^{4} h_{3} k_{132} - 2 h_{1}^{2} h_{2}^{4} h_{3} k_{231} + h_{1}^{2} h_{2}^{4} h_{4} k_{142} - 2 h_{1}^{2} h_{2}^{4} h_{4} k_{241} - h_{1} h_{2}^{5} h_{3} k_{131} + h_{1} h_{2}^{5} h_{3} k_{232} - h_{1} h_{2}^{5} h_{4} k_{141} + h_{1} h_{2}^{5} h_{4} k_{242} - h_{2}^{6} h_{3} k_{231} - h_{2}^{6} h_{4} k_{241}\right) + 4 \pi \nu \left(h_{1}^{6} + 3 h_{1}^{4} h_{2}^{2} + 3 h_{1}^{2} h_{2}^{4} + h_{2}^{6}\right)\right)$$

We don't have the value for `nu` !

## Conclusion

We do NOT obtain the same result as the document. Everything failed at the end.

Too bad, but still, it was interesting. I guess?

> See [here](https://github.com/Naereen/notebooks) for other notebooks I wrote.