# Simulating Molecules using VQE

In this tutorial, we introduce the Variational Quantum Eigensolver (VQE), motivate its use, explain the necessary theory, and demonstrate its implementation in finding the ground state energy of molecules.

## Contents
1. [Introduction](#introduction)
2. [The Variational Method of Quantum Mechanics](#varmethod)
    1. [Mathematical Background](#backgroundmath)
    2. [Bounding the Ground State](#groundstate)
3. [The Variational Quantum Eigensolver](#vqe)
    1. [Variational Forms](#varforms)
    2. [Simple Variational Forms](#simplevarform)
    3. [Parameter Optimization](#optimization)
    4. [Example with a Single Qubit Variational Form](#example)
    5. [Structure of Common Variational Forms](#commonvarforms)
4. [VQE Implementation in Qiskit](#implementation)
    1. [Running VQE on a Statevector Simulator](#implementationstatevec)
    2. [Running VQE on a Noisy Simulator](#implementationnoisy)
5. [Problems](#problems)
6. [References](#references)

## Introduction
In many applications it is important to find the minimum eigenvalue of a matrix. For example, in chemistry, the minimum eigenvalue of a Hermitian matrix characterizing the molecule is the ground state energy of that system. In the future, the quantum phase estimation algorithm may be used to find the minimum eigenvalue. However, its implementation on useful problems requires circuit depths exceeding the limits of hardware available in the NISQ era. Thus, in 2014, Peruzzo *et al.* proposed VQE to estimate the ground state energy of a molecule using much shallower circuits [1].

Formally stated, given a Hermitian matrix $H$ with an unknown minimum eigenvalue $\lambda_{min}$, associated with the eigenstate $|\psi_{min}\rangle$, VQE provides an estimate $\lambda_{\theta}$ bounding $\lambda_{min}$:

\begin{align*}
    \lambda_{min} \le \lambda_{\theta} \equiv \langle \psi(\theta) |H|\psi(\theta) \rangle
\end{align*}

where $|\psi(\theta)\rangle$ is the eigenstate associated with $\lambda_{\theta}$. By applying a parameterized circuit, represented by $U(\theta)$, to some arbitrary starting state $|\psi\rangle$, the algorithm obtains an estimate $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle$ on $|\psi_{min}\rangle$. The estimate is iteratively optimized by a classical controller changing the parameter $\theta$ minimizing the expectation value of $\langle \psi(\theta) |H|\psi(\theta) \rangle$.

## The Variational Method of Quantum Mechanics
### Mathematical Background

VQE is an application of the variational method of quantum mechanics. To better understand the variational method, some preliminary mathematical background is provided. An eigenvector, $|\psi_i\rangle$, of a matrix $A$ is invariant under transformation by $A$ up to a scalar multiplicative constant (the eigenvalue $\lambda_i$). That is,

\begin{align*}
    A |\psi_i\rangle = \lambda_i |\psi_i\rangle
\end{align*}

Furthermore, a matrix $H$ is Hermitian when it is equal to its own conjugate transpose.

\begin{align*}
    H = H^{\dagger}
\end{align*}

The spectral theorem states that the eigenvalues of a Hermitian matrix must be real. Thus, any eigenvalue of $H$ has the property that $\lambda_i = \lambda_i^*$. As any measurable quantity must be real, Hermitian matrices are suitable for describing the Hamiltonians of quantum systems. Moreover, $H$ may be expressed as

\begin{align*}
    H = \sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |
\end{align*}

where each $\lambda_i$ is the eigenvalue corresponding to the eigenvector $|\psi_i\rangle$. Furthermore, the expectation value of the observable $H$ on an arbitrary quantum state $|\psi\rangle$ is given by

\begin{align}
    \langle H \rangle_{\psi} &\equiv \langle \psi | H | \psi \rangle
\end{align}

Substituting $H$ with its representation as a weighted sum of its eigenvectors,

\begin{align}
    \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle &= \langle \psi | \left(\sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |\right) |\psi\rangle\\
    &= \sum_{i = 1}^{N} \lambda_i \langle \psi | \psi_i\rangle \langle \psi_i | \psi\rangle \\
    &= \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The last equation demonstrates that the expectation value of an observable on any state can be expressed as a linear combination using the eigenvalues associated with $H$ as the weights. Moreover, each of the weights in the linear combination is greater than or equal to 0, as $| \langle \psi_i | \psi\rangle |^2 \ge 0$ and so it is clear that

\begin{align}
    \lambda_{min} \le \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle = \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The above equation is known as the **variational method** (in some texts it is also known as the variational principle) [2]. It is important to note that this implies that the expectation value of any wave function will always be at least the minimum eigenvalue associated with $H$. Moreover, the expectation value of state $|\psi_{min}\rangle$ is given by $\langle \psi_{min}|H|\psi_{min}\rangle = \langle \psi_{min}|\lambda_{min}|\psi_{min}\rangle = \lambda_{min}$. Thus, as expected, $\langle H \rangle_{\psi_{min}}=\lambda_{min}$.

### Bounding the Ground State
When the Hamiltonian of a system is described by the Hermitian matrix $H$ the ground state energy of that system, $E_{gs}$, is the smallest eigenvalue associated with $H$. By arbitrarily selecting a wave function $|\psi \rangle$ (called an *ansatz*) as an initial guess approximating $|\psi_{min}\rangle$, calculating its expectation value, $\langle H \rangle_{\psi}$, and iteratively updating the wave function, arbitrarily tight bounds on the ground state energy of a Hamiltonian may be obtained.

## The Variational Quantum Eigensolver
### Variational Forms
A systematic approach to varying the ansatz is required to implement the variational method on a quantum computer. VQE does so through the use of a parameterized circuit with a fixed form. Such a circuit is often called a *variational form*, and its action may be represented by the linear transformation $U(\theta)$. A variational form is applied to a starting state $|\psi\rangle$ (such as the vacuum state $|0\rangle$, or the Hartree Fock state) and generates an output state $U(\theta)|\psi\rangle\equiv |\psi(\theta)\rangle$. Iterative optimization over $|\psi(\theta)\rangle$ aims to yield an expectation value $\langle \psi(\theta)|H|\psi(\theta)\rangle \approx E_{gs} \equiv \lambda_{min}$. Ideally, $|\psi(\theta)\rangle$ will be close to $|\psi_{min}\rangle$ (where 'closeness' is characterized by either state fidelity, or Manhattan distance) although in practice, useful bounds on $E_{gs}$ can be obtained even if this is not the case.

Moreover, a fixed variational form with a polynomial number of parameters can only generate transformations to a polynomially sized subspace of all the states in an exponentially sized Hilbert space. Consequently, various variational forms exist. Some, such as Ry and RyRz are heuristically designed, without consideration of the target domain. Others, such as UCCSD, utilize domain specific knowledge to generate close approximations based on the problem's structure. The structure of common variational forms is discussed in greater depth later in this document.

### Simple Variational Forms
When constructing a variational form we must balance two opposing goals. Ideally, our $n$ qubit variational form would be able to generate any possible state $|\psi\rangle$ where $|\psi\rangle \in \mathbb{C}^N$ and $N=2^n$. However, we would like the variational form to use as few parameters as possible. Here, we aim to give intuition for the construction of variational forms satisfying our first goal, while disregarding the second goal for the sake of simplicity.

Consider the case where $n=1$. The U3 gate takes three parameters, $\theta, \phi$ and $\lambda$, and represents the following transformation:

\begin{align}
    U3(\theta, \phi, \lambda) = \begin{pmatrix}\cos(\frac{\theta}{2}) & -e^{i\lambda}\sin(\frac{\theta}{2}) \\ e^{i\phi}\sin(\frac{\theta}{2}) & e^{i\lambda + i\phi}\cos(\frac{\theta}{2}) \end{pmatrix}
\end{align}

Up to a global phase, any possible single qubit transformation may be implemented by appropriately setting these parameters. Consequently, for the single qubit case, a variational form capable of generating any possible state is given by the circuit:

Moreover, this universal 'variational form' only has 3 parameters and thus can be efficiently optimized. It is worth emphasising that the ability to generate an arbitrary state ensures that during the optimization process, the variational form does not limit the set of attainable states over which the expectation value of $H$ can be taken. Ideally, this ensures that the minimum expectation value is limited only by the capabilities of the classical optimizer.

A less trivial universal variational form may be derived for the 2 qubit case, where two body interactions, and thus entanglement, must be considered to achieve universality. Based on the work presented by *Shende et al.* [3] the following is an example of a universal parameterized 2 qubit circuit:

Allow the transformation performed by the above circuit to be represented by $U(\theta)$. When optimized variationally, the expectation value of $H$ is minimized when $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle \approx |\psi_{min}\rangle$. By formulation, $U(\theta)$ may produce a transformation to any possible state, and so this variational form may obtain an arbitrarily tight bound on two qubit ground state energies, only limited by the capabilities of the classical optimizer.

### Parameter Optimization
Once an efficiently parameterized variational form has been selected, in accordance with the variational method, its parameters must be optimized to minimize the expectation value of the target Hamiltonian. The parameter optimization process has various challenges. For example, quantum hardware has various types of noise and so objective function evaluation (energy calculation) may not necessarily reflect the true objective function. Additionally, some optimizers perform a number of objective function evaluations dependent on cardinality of the parameter set. An appropriate optimizer should be selected by considering the requirements of a application.

A popular optimization strategy is gradient decent where each parameter is updated in the direction yielding the largest local change in energy. Consequently, the number of evaluations performed depends on the number of optimization parameters present. This allows the algorithm to quickly find a local optimum in the search space. However, this optimization strategy often gets stuck at poor local optima, and is relatively expensive in terms of the number of circuit evaluations performed. While an intuitive optimization strategy, it is not recommended for use in VQE.

An appropriate optimizer for optimizing a noisy objective function is the *Simultaneous Perturbation Stochastic Approximation* optimizer (SPSA). SPSA approximates the gradient of the objective function with only two measurements. It does so by concurrently perturbing all of the parameters in a random fashion, in contrast to gradient decent where each parameter is perturbed independently. When utilizing VQE in either a noisy simulator or on real hardware, SPSA is a recommended as the classical optimizer.

When noise is not present in the cost function evaluation (such as when using VQE with a statevector simulator), a wide variety of classical optimizers may be useful. Two such optimizers supported by Qiskit Aqua are the *Sequential Least Squares Programming* optimizer (SLSQP) and the *Constrained Optimization by Linear Approximation* optimizer (COBYLA). It is worth noting that COBYLA only performs one objective function evaluation per optimization iteration (and thus the number of evaluations is independent of the parameter set's cardinality). Therefore, if the objective function is noise-free and minimizing the number of performed evaluations is desirable, it is recommended to try COBYLA.

### Example with a Single Qubit Variational Form

We will now use the simple single qubit variational form to solve a problem similar to ground state energy estimation. Specifically, we are given a random probability vector $\vec{x}$ and wish to determine a possible parameterization for our single qubit variational form such that it outputs a probability distribution that is close to $\vec{x}$ (where closeness is defined in terms of the Manhattan distance between the two probability vectors).

We first create the random probability vector in python:

```python
import numpy as np
np.random.seed(999999)
target_distr = np.random.rand(2)
# We now convert the random vector into a valid probability vector
target_distr /= sum(target_distr)
```

We subsequently create a function that takes the parameters of our single U3 variational form as arguments and returns the corresponding quantum circuit:

```python
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister
def get_var_form(params):
    qr = QuantumRegister(1, name="q")
    cr = ClassicalRegister(1, name='c')
    qc = QuantumCircuit(qr, cr)
    qc.u3(params[0], params[1], params[2], qr[0])
    qc.measure(qr, cr[0])
    return qc
```

Now we specify the objective function which takes as input a list of the variational form's parameters, and returns the cost associated with those parameters:

```python
from qiskit import Aer, execute
backend = Aer.get_backend("qasm_simulator")
NUM_SHOTS = 10000

def get_probability_distribution(counts):
    output_distr = [v / NUM_SHOTS for v in counts.values()]
    if len(output_distr) == 1:
        output_distr.append(0)
    return output_distr

def objective_function(params):
    # Obtain a quantum circuit instance from the paramters
    qc = get_var_form(params)
    # Execute the quantum circuit to obtain the probability distribution associated with the current parameters
    result = execute(qc, backend, shots=NUM_SHOTS).result()
    # Obtain the counts for each measured state, and convert those counts into a probability vector
    output_distr = get_probability_distribution(result.get_counts(qc))
    # Calculate the cost as the distance between the output distribution and the target distribution
    cost = sum([np.abs(output_distr[i] - target_distr[i]) for i in range(2)])
    return cost
```

Finally, we create an instance of the COBYLA optimizer, and run the algorithm. Note that the output varies from run to run. Moreover, while close, the obtained distribution might not be exactly the same as the target distribution, however, increasing the number of shots taken will increase the accuracy of the output.

```python
from qiskit.aqua.components.optimizers import COBYLA

# Initialize the COBYLA optimizer
optimizer = COBYLA(maxiter=500, tol=0.0001)

# Create the initial parameters (noting that our single qubit variational form has 3 parameters)
params = np.random.rand(3)
ret = optimizer.optimize(num_vars=3, objective_function=objective_function, initial_point=params)

# Obtain the output distribution using the final parameters
qc = get_var_form(ret[0])
counts = execute(qc, backend, shots=NUM_SHOTS).result().get_counts(qc)
output_distr = get_probability_distribution(counts)

print("Target Distribution:", target_distr)
print("Obtained Distribution:", output_distr)
print("Output Error (Manhattan Distance):", ret[1])
print("Parameters Found:", ret[0])

```

    Target Distribution: [0.51357006 0.48642994]
    Obtained Distribution: [0.5182, 0.4818]
    Output Error (Manhattan Distance): 0.0001401187388391789
    Parameters Found: [1.59966854 0.66273002 0.28432001]

### Structure of Common Variational Forms
As already discussed, it is not possible for a polynomially parameterized variational form to generate a transformation to any state. Variational forms can be grouped into two categories, depending on how they deal with this limitation. The first category of variational forms use domain or application specific knowledge to limit the set of possible output states. The second approach uses a heuristic circuit without prior domain or application specific knowledge.

The first category of variational forms exploit characteristics of the problem domain to restrict the set of transformations that may be required. For example, when calculating the ground state energy of a molecule, the number of particles in the system is known *a priori*. Therefore, if a starting state with the correct number of particles is used, by limiting the variational form to only producing particle preserving transformations, the number of parameters required to span the new transformation subspace can be greatly reduced. Indeed, by utilizing similar information from Coupled-Cluster theory, the variational form UCCSD can obtain very accurate results for molecular ground state energy estimation when starting from the Hartree Fock state. Another example illustrating the exploitation of domain-specific knowledge follows from considering the set of circuits realizable on real quantum hardware. Extant quantum computers, such as those based on super conducting qubits, have limited qubit connectivity. That is, it is not possible to implement 2-qubit gates on arbitrary qubit pairs (without inserting swap gates). Thus, variational forms have been constructed for specific quantum computer architectures where the circuits are specifically tuned to maximally exploit the natively available connectivity and gates of a given quantum device. Such a variational form was used in 2017 to successfully implement VQE for the estimation of the ground state energies of molecules as large as BeH$_2$ on an IBM quantum computer [4].

In the second approach, gates are layered such that good approximations on a wide range of states may be obtained. Qiskit Aqua supports three such variational forms: RyRz, Ry and SwapRz (we will only discuss the first two). All of these variational forms accept multiple user-specified configurations. Three essential configurations are the number of qubits in the system, the depth setting, and the entanglement setting. A single layer of a variational form specifies a certain pattern of single qubit rotations and CX gates. The depth setting says how many times the variational form should repeat this pattern. By increasing the depth setting, at the cost of increasing the number of parameters that must be optimized, the set of states the variational form can generate increases. Finally, the entanglement setting selects the configuration, and implicitly the number, of CX gates. For example, when the entanglement setting is linear, CX gates are applied to adjacent qubit pairs in order (and thus $n-1$ CX gates are added per layer). When the entanglement setting is full, a CX gate is applied to each qubit pair in each layer. The circuits for RyRz corresponding to `entanglement="full"` and `entanglement="linear"` can be seen by executing the following code snippet:

```python
from qiskit.aqua.components.variational_forms import RYRZ
entanglements = ["linear", "full"]
for entanglement in entanglements:
    form = RYRZ(num_qubits=4, depth=1, entanglement=entanglement)
    if entanglement == "linear":
        print("=============Linear Entanglement:=============")
    else:
        print("=============Full Entanglement:=============")
    # We initialize all parameters to 0 for this demonstration
    print(form.construct_circuit([0] * form.num_parameters).draw(line_length=100))
    print()
```

    =============Linear Entanglement:=============
            ┌───────────┐┌───────┐                      ┌───────────┐  ┌───────┐                »
    q_0: |0>┤ U3(0,0,0) ├┤ U1(0) ├──────────────────■───┤ U3(0,0,0) ├──┤ U1(0) ├────────────────»
            ├───────────┤├───────┤┌──────────────┐┌─┴─┐┌┴───────────┴─┐└───────┘ ┌───────────┐  »
    q_1: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├┤ X ├┤ U2(0,3.1416) ├────■─────┤ U3(0,0,0) ├──»
            ├───────────┤├───────┤├──────────────┤└───┘└──────────────┘  ┌─┴─┐  ┌┴───────────┴─┐»
    q_2: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├───────────────────────┤ X ├──┤ U2(0,3.1416) ├»
            ├───────────┤├───────┤├──────────────┤                       └───┘  └──────────────┘»
    q_3: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├──────────────────────────────────────────────»
            └───────────┘└───────┘└──────────────┘                                              »
    «                                                     ░
    «q_0: ────────────────────────────────────────────────░─
    «     ┌───────┐                                       ░
    «q_1: ┤ U1(0) ├───────────────────────────────────────░─
    «     └───────┘ ┌───────────┐    ┌───────┐            ░
    «q_2: ────■─────┤ U3(0,0,0) ├────┤ U1(0) ├────────────░─
    «       ┌─┴─┐  ┌┴───────────┴─┐┌─┴───────┴─┐┌───────┐ ░
    «q_3: ──┤ X ├──┤ U2(0,3.1416) ├┤ U3(0,0,0) ├┤ U1(0) ├─░─
    «       └───┘  └──────────────┘└───────────┘└───────┘ ░

    =============Full Entanglement:=============
            ┌───────────┐┌───────┐                                                               »
    q_0: |0>┤ U3(0,0,0) ├┤ U1(0) ├──────────────────■────■────────────────────■──────────────────»
            ├───────────┤├───────┤┌──────────────┐┌─┴─┐  │  ┌──────────────┐  │                  »
    q_1: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├┤ X ├──┼──┤ U2(0,3.1416) ├──┼──────────────────»
            ├───────────┤├───────┤├──────────────┤└───┘┌─┴─┐└──────────────┘  │  ┌──────────────┐»
    q_2: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├─────┤ X ├──────────────────┼──┤ U2(0,3.1416) ├»
            ├───────────┤├───────┤├──────────────┤     └───┘                ┌─┴─┐└──────────────┘»
    q_3: |0>┤ U3(0,0,0) ├┤ U1(0) ├┤ U2(0,3.1416) ├──────────────────────────┤ X ├────────────────»
            └───────────┘└───────┘└──────────────┘                          └───┘                »
    «      ┌───────────┐     ┌───────┐                                                              »
    «q_0: ─┤ U3(0,0,0) ├─────┤ U1(0) ├──────────────────────────────────────────────────────────────»
    «      └───────────┘     └───────┘                          ┌───────────┐     ┌───────┐         »
    «q_1: ───────────────────────■──────────■───────────────────┤ U3(0,0,0) ├─────┤ U1(0) ├─────────»
    «     ┌──────────────┐     ┌─┴─┐        │  ┌──────────────┐ └───────────┘     └───────┘         »
    «q_2: ┤ U2(0,3.1416) ├─────┤ X ├────────┼──┤ U2(0,3.1416) ├──────────────────────────────────■──»
    «     ├──────────────┤┌────┴───┴─────┐┌─┴─┐└──────────────┘┌──────────────┐┌──────────────┐┌─┴─┐»
    «q_3: ┤ U2(0,3.1416) ├┤ U2(0,3.1416) ├┤ X ├────────────────┤ U2(0,3.1416) ├┤ U2(0,3.1416) ├┤ X ├»
    «     └──────────────┘└──────────────┘└───┘                └──────────────┘└──────────────┘└───┘»
    «                                            ░
    «q_0: ───────────────────────────────────────░─
    «                                            ░
    «q_1: ───────────────────────────────────────░─
    «      ┌───────────┐    ┌───────┐            ░
    «q_2: ─┤ U3(0,0,0) ├────┤ U1(0) ├────────────░─
    «     ┌┴───────────┴─┐┌─┴───────┴─┐┌───────┐ ░
    «q_3: ┤ U2(0,3.1416) ├┤ U3(0,0,0) ├┤ U1(0) ├─░─
    «     └──────────────┘└───────────┘└───────┘ ░

Assume the depth setting is set to $d$. Then, RyRz has $n\times (d+1)\times 2$ parameters, Ry with linear entanglement has $2n\times(d + \frac{1}{2})$ parameters, and Ry with full entanglement has $d\times n\times \frac{(n + 1)}{2} + n$ parameters.

## VQE Implementation in Qiskit
This section illustrates an implementation of VQE using the programmatic approach. Qiskit Aqua also enables a declarative implementation, however, it reveals less information about the underlying algorithm. This code, specifically the preparation of qubit operators, is based on the code found in the Qiskit Tutorials repository (and as of July 2019, may be found at: https://github.com/Qiskit/qiskit-tutorials ).

The following libraries must first be imported.

```python
from qiskit.aqua.algorithms import VQE, ExactEigensolver
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
from qiskit.chemistry.aqua_extensions.components.variational_forms import UCCSD
from qiskit.aqua.components.variational_forms import RYRZ
from qiskit.chemistry.aqua_extensions.components.initial_states import HartreeFock
from qiskit.aqua.components.optimizers import COBYLA, SPSA, SLSQP
from qiskit import IBMQ, BasicAer, Aer
from qiskit.chemistry.drivers import PySCFDriver, UnitsType
from qiskit.chemistry import FermionicOperator
from qiskit import IBMQ
from qiskit.providers.aer import noise
from qiskit.aqua import QuantumInstance
from qiskit.ignis.mitigation.measurement import CompleteMeasFitter
```

### Running VQE on a Statevector Simulator
We demonstrate the calculation of the ground state energy for LiH at various interatomic distances. A driver for the molecule must be created at each such distance. Note that in this experiment, to reduce the number of qubits used, we freeze the core and remove two unoccupied orbitals. First, we define a function that takes an interatomic distance and returns the appropriate qubit operator, $H$, as well as some other information about the operator.

```python
def get_qubit_op(dist):
    driver = PySCFDriver(atom="Li .0 .0 .0; H .0 .0 " + str(dist), unit=UnitsType.ANGSTROM,
                         charge=0, spin=0, basis='sto3g')
    molecule = driver.run()
    freeze_list = [0]
    remove_list = [-3, -2]
    repulsion_energy = molecule.nuclear_repulsion_energy
    num_particles = molecule.num_alpha + molecule.num_beta
    num_spin_orbitals = molecule.num_orbitals * 2
    remove_list = [x % molecule.num_orbitals for x in remove_list]
    freeze_list = [x % molecule.num_orbitals for x in freeze_list]
    remove_list = [x - len(freeze_list) for x in remove_list]
    remove_list += [x + molecule.num_orbitals - len(freeze_list)  for x in remove_list]
    freeze_list += [x + molecule.num_orbitals for x in freeze_list]
    ferOp = FermionicOperator(h1=molecule.one_body_integrals, h2=molecule.two_body_integrals)
    ferOp, energy_shift = ferOp.fermion_mode_freezing(freeze_list)
    num_spin_orbitals -= len(freeze_list)
    num_particles -= len(freeze_list)
    ferOp = ferOp.fermion_mode_elimination(remove_list)
    num_spin_orbitals -= len(remove_list)
    qubitOp = ferOp.mapping(map_type='parity', threshold=0.00000001)
    qubitOp = qubitOp.two_qubit_reduced_operator(num_particles)
    shift = energy_shift + repulsion_energy
    return qubitOp, num_particles, num_spin_orbitals, shift
```

First, the exact ground state energy is calculated using the qubit operator and a classical exact eigensolver. Subsequently, the initial state $|\psi\rangle$ is created, which the VQE instance uses to produce the final ansatz $\min_{\theta}(|\psi(\theta)\rangle)$. The exact result and the VQE result at each interatomic distance is stored. Observe that the result given by `vqe.run(backend)['energy'] + shift` is equivalent the quantity $\min_{\theta}\left(\langle \psi(\theta)|H|\psi(\theta)\rangle\right)$, where the minimum is not necessarily the global minimum.

When initializing the VQE instance with `VQE(qubitOp, var_form, optimizer, 'matrix')` the expectation value of $H$ on $|\psi(\theta)\rangle$ is directly calculated through matrix multiplication. However, when using an actual quantum device, or a true simulator such as the `qasm_simulator` with `VQE(qubitOp, var_form, optimizer, 'paulis')` the calculation of the expectation value is more complicated. A Hamiltonian may be represented as a sum of a Pauli strings, with each Pauli term acting on a qubit as specified by the mapping being used. Each Pauli string has a corresponding circuit appended to the circuit corresponding to $|\psi(\theta)\rangle$. Subsequently, each of these circuits is executed, and all of the results are used to determine the expectation value of $H$ on $|\psi(\theta)\rangle$. In the following example, we initialize the VQE instance with `matrix` mode, and so the expectation value is directly calculated through matrix multiplication.

Note that the following code snippet may take a few minutes to run to completion.

```python
backend = BasicAer.get_backend("statevector_simulator")
distances = np.arange(0.5, 4.0, 0.1)
exact_energies = []
vqe_energies = []
optimizer = SLSQP(maxiter=5)
for dist in distances:
    qubitOp, num_particles, num_spin_orbitals, shift = get_qubit_op(dist)
    result = ExactEigensolver(qubitOp).run()
    exact_energies.append(result['energy'] + shift)
    initial_state = HartreeFock(
        qubitOp.num_qubits,
        num_spin_orbitals,
        num_particles,
        'parity'
    )
    var_form = UCCSD(
        qubitOp.num_qubits,
        depth=1,
        num_orbitals=num_spin_orbitals,
        num_particles=num_particles,
        initial_state=initial_state,
        qubit_mapping='parity'
    )
    vqe = VQE(qubitOp, var_form, optimizer, 'matrix')
    results = vqe.run(backend)['energy'] + shift
    vqe_energies.append(results)
    print("Interatomic Distance:", np.round(dist, 2), "VQE Result:", results, "Exact Energy:", exact_energies[-1])

print("All energies have been calculated")
```

    Interatomic Distance: 0.5 VQE Result: -7.039710219020506 Exact Energy: -7.039732521635202
    Interatomic Distance: 0.6 VQE Result: -7.313344302334236 Exact Energy: -7.313345828761008
    Interatomic Distance: 0.7 VQE Result: -7.500921095743192 Exact Energy: -7.500922090905936
    Interatomic Distance: 0.8 VQE Result: -7.630976914468914 Exact Energy: -7.630978249333209
    Interatomic Distance: 0.9 VQE Result: -7.7208107952020795 Exact Energy: -7.720812412134773
    Interatomic Distance: 1.0 VQE Result: -7.782240655298441 Exact Energy: -7.782242402637011
    Interatomic Distance: 1.1 VQE Result: -7.823597493320795 Exact Energy: -7.82359927636281
    Interatomic Distance: 1.2 VQE Result: -7.850696622934822 Exact Energy: -7.850698377596024
    Interatomic Distance: 1.3 VQE Result: -7.867561602181376 Exact Energy: -7.867563290110052
    Interatomic Distance: 1.4 VQE Result: -7.876999876757721 Exact Energy: -7.877001491818373
    Interatomic Distance: 1.5 VQE Result: -7.8810141736656405 Exact Energy: -7.881015715646992
    Interatomic Distance: 1.6 VQE Result: -7.881070662952161 Exact Energy: -7.88107204403092
    Interatomic Distance: 1.7 VQE Result: -7.878267162143656 Exact Energy: -7.878268167584993
    Interatomic Distance: 1.8 VQE Result: -7.873440112155302 Exact Energy: -7.873440293132828
    Interatomic Distance: 1.9 VQE Result: -7.86723366674701 Exact Energy: -7.8672339648160285
    Interatomic Distance: 2.0 VQE Result: -7.860152327529411 Exact Energy: -7.86015320737878
    Interatomic Distance: 2.1 VQE Result: -7.852595105536979 Exact Energy: -7.852595827876738
    Interatomic Distance: 2.2 VQE Result: -7.844878726366329 Exact Energy: -7.844879093009722
    Interatomic Distance: 2.3 VQE Result: -7.837257439448259 Exact Energy: -7.8372579676155025
    Interatomic Distance: 2.4 VQE Result: -7.829935045088515 Exact Energy: -7.829937002623394
    Interatomic Distance: 2.5 VQE Result: -7.823070191557451 Exact Energy: -7.82307664213409
    Interatomic Distance: 2.6 VQE Result: -7.816782591999657 Exact Energy: -7.816795150472929
    Interatomic Distance: 2.7 VQE Result: -7.8111534373726 Exact Energy: -7.811168284803366
    Interatomic Distance: 2.8 VQE Result: -7.806218299266321 Exact Energy: -7.806229560089845
    Interatomic Distance: 2.9 VQE Result: -7.801962397475152 Exact Energy: -7.8019736023325486
    Interatomic Distance: 3.0 VQE Result: -7.798352412318197 Exact Energy: -7.7983634309151295
    Interatomic Distance: 3.1 VQE Result: -7.795326815750017 Exact Energy: -7.795340451637537
    Interatomic Distance: 3.2 VQE Result: -7.792800698225245 Exact Energy: -7.792834806738612
    Interatomic Distance: 3.3 VQE Result: -7.790603799019874 Exact Energy: -7.790774009971014
    Interatomic Distance: 3.4 VQE Result: -7.788715354695274 Exact Energy: -7.789088897991478
    Interatomic Distance: 3.5 VQE Result: -7.787215781080283 Exact Energy: -7.787716973466144
    Interatomic Distance: 3.6 VQE Result: -7.786080393658009 Exact Energy: -7.786603763673838
    Interatomic Distance: 3.7 VQE Result: -7.785203497342158 Exact Energy: -7.785702912499886
    Interatomic Distance: 3.8 VQE Result: -7.7844795319924325 Exact Energy: -7.784975591698873
    Interatomic Distance: 3.9 VQE Result: -7.783853361693722 Exact Energy: -7.7843896116723315
    All energies have been calculated

```python
plt.plot(distances, exact_energies, label="Exact Energy")
plt.plot(distances, vqe_energies, label="VQE Energy")
plt.xlabel('Atomic distance (Angstrom)')
plt.ylabel('Energy')
plt.legend()
plt.show()

```

Note that the VQE results are very close to the exact results, and so the exact energy curve is hidden by the VQE curve.

### Running VQE on a Noisy Simulator

Here, we calculate the ground state energy for H$_2$ using a noisy simulator and error mitigation.

First, we prepare the qubit operator representing the molecule's Hamiltonian:

```python
driver = PySCFDriver(atom='H .0 .0 -0.3625; H .0 .0 0.3625', unit=UnitsType.ANGSTROM, charge=0, spin=0, basis='sto3g')
molecule = driver.run()
num_particles = molecule.num_alpha + molecule.num_beta
qubitOp = FermionicOperator(h1=molecule.one_body_integrals, h2=molecule.two_body_integrals).mapping(map_type='parity')
qubitOp = qubitOp.two_qubit_reduced_operator(num_particles)
```

Now, we load a device coupling map and noise model from the IBMQ provider and create a quantum instance, enabling error mitigation:

```python
provider = IBMQ.load_account()
backend = Aer.get_backend("qasm_simulator")
device = provider.get_backend("ibmqx4")
coupling_map = device.configuration().coupling_map
noise_model = noise.device.basic_device_noise_model(device.properties())
quantum_instance = QuantumInstance(backend=backend, shots=1000,
                                   noise_model=noise_model,
                                   coupling_map=coupling_map,
                                   measurement_error_mitigation_cls=CompleteMeasFitter,
                                   cals_matrix_refresh_period=30,)
```

Finally, we must configure the optimizer, the variational form, and the VQE instance. As the effects of noise increase as the number of two qubit gates circuit depth increase, we use a heuristic variational form (RYRZ) rather than UCCSD as RYRZ has a much shallower circuit than UCCSD and uses substantially fewer two qubit gates.

The following code may take a few minutes to run to completion.

```python
exact_solution = ExactEigensolver(qubitOp).run()
print("Exact Result:", exact_solution['energy'])
optimizer = SPSA(max_trials=100)
var_form = RYRZ(qubitOp.num_qubits, depth=1, entanglement="linear")
vqe = VQE(qubitOp, var_form, optimizer=optimizer, operator_mode="grouped_paulis")
ret = vqe.run(quantum_instance)
print("VQE Result:", ret['energy'])
```

    Exact Result: -1.86712097834127
    VQE Result: -1.8220854070067132

When noise mitigation is enabled, even though the result does not fall within chemical accuracy (defined as being within 0.0016 Hartree of the exact result), it is fairly close to the exact solution.

## Problems
1. You are given a Hamiltonian $H$ with the promise that its ground state is close to a maximally entangled $n$ qubit state. Explain which variational form (or forms) is likely to efficiently and accurately learn the the ground state energy of $H$. You may also answer by creating your own variational form, and explaining why it is appropriate for use with this Hamiltonian.
2. Calculate the number of circuit evaluations performed per optimization iteration, when using the COBYLA optimizer, the `qasm_simulator` with 1000 shots, and a Hamiltonian with 60 Pauli strings.
3. Use VQE to estimate the ground state energy of BeH$_2$ with an interatomic distance of $1.3$Å. You may re-use the function `get_qubit_op(dist)` by replacing `atom="Li .0 .0 .0; H .0 .0 " + str(dist)` with `atom="Be .0 .0 .0; H .0 .0 -" + str(dist) + "; H .0 .0 " + str(dist)` and invoking the function with `get_qubit_op(1.3)`. Note that removing the unoccupied orbitals does not preserve chemical precision for this molecule. However, to get the number of qubits  required down to 6 (and thereby allowing efficient simulation on most laptops), the loss of precision is acceptable. While beyond the scope of this exercise, the interested reader may use qubit tapering operations to reduce the number of required qubits to 7, without losing any chemical precision.

## References
1. Peruzzo, Alberto, et al. "A variational eigenvalue solver on a photonic quantum processor." *Nature communications* 5 (2014): 4213.
2. Griffiths, David J., and Darrell F. Schroeter. Introduction to quantum mechanics. *Cambridge University Press*, 2018.
3. Shende, Vivek V., Igor L. Markov, and Stephen S. Bullock. "Minimal universal two-qubit cnot-based circuits." arXiv preprint quant-ph/0308033 (2003).
4. Kandala, Abhinav, et al. "Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets." Nature 549.7671 (2017): 242.


# Linear algebra

## TOC:
* [Basics](#basics)
* [Sparse matrices](#sparse)
* [Broadcasting](#broadcasting)
* [Einstein summation](#einstein)
* [Eigenvalue decomposition](#EVD)
* [Singular value decomposition](#SVD)
* [Other decompositions](#decomp)
* [Matrix calculus](#calculus)
* [Linear systems of equations](#linear)

```python
# Standard Python libraries
from __future__ import absolute_import, division, print_function, unicode_literals

import os
import time
import numpy as np
np.set_printoptions(precision=3)
import glob
import matplotlib.pyplot as plt
import PIL
import imageio

from IPython import display

import sklearn

import seaborn as sns;
sns.set(style="ticks", color_codes=True)

import pandas as pd
pd.set_option('precision', 2) # 2 decimal places
pd.set_option('display.max_rows', 20)
pd.set_option('display.max_columns', 30)
pd.set_option('display.width', 100) # wide windows

```

```python
# https://github.com/google/jax
import jax
import jax.numpy as np
import numpy as onp # original numpy
from jax.scipy.special import logsumexp
from jax import grad, hessian, jacfwd, jacrev, jit, vmap
from jax.experimental import optimizers
print("jax version {}".format(jax.__version__))
```

    jax version 0.2.7

## Basics

```python
# Create 1d vector
v = np.array([0,1,2]) # 1d vector
print(v.ndim) ## 1
print(v.shape) ## (3,)

```

    1
    (3,)

```python
# Note that Python uses 0-indexing, not 1-indexing.
# Thus the elements are accessed as follows:
print(v[0], v[1], v[2]) ## 0 1 2
```

    0 1 2

```python
# Create 2d array
A = np.array([ [0,1,2], [3,4,5] ])
print(A)
## [[0, 1, 2],
##  [3, 4, 5]])
print(A.ndim) ## 2
print(A.shape) ## (2,3)
print(A.size) ## 6
print(A.T.shape) ## (3,2)
```

    [[0 1 2]
     [3 4 5]]
    2
    (2, 3)
    6
    (3, 2)

```python
# If we want to make a vector into a matrix with one row, we can use any of the following:

x = np.array([1,2]) # vector
X1 = np.array([x]) # matrix with one row
X2 = np.reshape(x, (1,-1))
X3 = x[None, :]
X4 = x[np.newaxis, :]
assert np.array_equal(X1, X2)
assert np.array_equal(X1, X3)
print(np.shape(X1)) ## (1,2)

```

    (1, 2)

```python
# If we want to make a vector into a matrix with one column, we can use any of the following:
x = np.array([1,2]) # vector
X1 = np.array([x]).T # matrix with one column
X2 = np.reshape(x, (-1,1))
X3 = x[:, None]
X4 = x[:, np.newaxis]
assert np.array_equal(X1, X2)
assert np.array_equal(X1, X3)
print(np.shape(X1)) ## (2,1)
```

    (2, 1)

```python
# Here is how to create a one-hot encoding of integers.

def one_hot(x, k, dtype=np.float32):
  return np.array(x[:, None] == np.arange(k), dtype)

# Example
x = np.array([1,2,0,2]);
X = one_hot(x, 3)
print(X)

```

    [[0. 1. 0.]
     [0. 0. 1.]
     [1. 0. 0.]
     [0. 0. 1.]]

```python

# We can construct arrays from a list of column vectors as follows:
A1 = np.array([ [0,1,2], [3,4,5] ])
col0 = A1[:,0]; col1 = A1[:,1]; col2=A1[:,2];
A2 = np.stack([col0,col1,col2],axis=1)
assert np.array_equal(A1, A2)

# We can construct arrays from a list of row vectors as follows:
row0=A1[0,:]; row1=A1[1,:];
A2 = np.stack([row0,row1],axis=0)
assert np.array_equal(A1, A2)
```

```python
# We can construct arrays from a list of arrays
# using the hstack or vstack functions,
# which stack horizontally or vertically,  as illustrated below.

M = np.array([[9,8,7],[6,5,4]])
C = np.array([[99], [99]])
A1 = np.concatenate([M, C], axis=1)
A2 = np.hstack([M, C])
#A3 = np.c_[M, C] # c_ does not work in jax
assert np.array_equal(A1, A2)
#assert np.array_equal(A1, A3)
print(A1)

```

    [[ 9  8  7 99]
     [ 6  5  4 99]]

```python

R = np.array([[1,2,3]])
A1 = np.concatenate([R, M], axis=0)
A2 = np.vstack([R, M])
assert np.array_equal(A1, A2)
print(A1)

```

    [[1 2 3]
     [9 8 7]
     [6 5 4]]

```python
# A very common idiom  is to add a column of 1s to a datamatrix.
# We can do this using horizontal stacking (along the columns) as follows.

X = np.array([[9,8,7],[6,5,4]])
N = np.shape(X)[0] # num. rows
X1 = np.hstack([np.ones((N,1)), X])
print(X1)
```

    [[1. 9. 8. 7.]
     [1. 6. 5. 4.]]

```python

# We can flatten a matrix to a vector (concatenating its rows, one by one) using ravel

A = np.reshape(np.arange(6),(2,3))
print(A.ravel()) ##  [0 1 2 3 4 5]

```

    [0 1 2 3 4 5]

In numpy,  arrays are layed out in memory
such that, if we iterate over neighboring elements,
the rightmost index changes the fastest.
This is called row-major order,
and is used by other languages such as C++, Eigen and PyTorch.
By contrast, other languages (such as Julia, Matlab, R and Fortran)
use column-major order.
See below for an illustration of the difference.

(Source: https://commons.wikimedia.org/wiki/File:Row_and_column_major_order.svg)

Thus in numpy, for speed reasons, we should always write loops like this:
```
A = np.reshape(np.arange(6),(2,3))
d1, d2 = np.shape(A)
for i in range(d1):
  for j in range(d2):
    # Do something with A[i,j]
 ```

For similar reasons, data matrices are usually stored
in the form $(N,D)$, where $N$ is the batchsize (number of examples),
so that we can efficiently extract minibatches by slicing blocks of consecutive memory.

```python
## We can create a tensor in numpy as in this example:

#T = np.ndarray([2,3,4]) # fill with random values # does not work with jax
T = np.zeros((2,3,4))
T = np.reshape(np.arange(24),(2,3,4)) # fill with 0..23
print(np.shape(T))
print(T)

```

    (2, 3, 4)
    [[[ 0  1  2  3]
      [ 4  5  6  7]
      [ 8  9 10 11]]

     [[12 13 14 15]
      [16 17 18 19]
      [20 21 22 23]]]

```python
#We can permute the order of the dimensions using np.transpose.

x = np.ones((1, 2, 3))
print(np.transpose(x, (1, 0, 2)).shape) ## (2, 1, 3)

#Note that this does not actually move the data in memory
#(which would be slow),
#it merely provides a different \keywordDef{view} of the same data,
#i.e., it changes the mapping from $n$-dimensional vectors of
#subscripts to 1d integers.
```

    (2, 1, 3)

```python
# matrix multiplication

A = onp.random.rand(2,3);
B = onp.random.rand(3,4);
C = np.dot(A,B)
assert np.shape(C) == (2,4)
print(C)
C2 = A.dot(B)
C3 = A @ B
assert np.allclose(C, C2)
assert np.allclose(C, C3)

#Note that we need to use np.dot(A,B)
#if we use A * B, Python tries to compute the elementwise product,
#which is invalid, since $A$ and $B$ have incompatible shapes.

```

    [[0.31  0.55  0.287 0.419]
     [0.418 0.621 0.307 0.52 ]]

```python
# Outer products

x = np.arange(1,3); y = np.arange(1,3);
A = np.outer(x,y);
print(A)

```

    [[1 2]
     [2 4]]

```python
# We can sum across the rows

X = np.reshape(np.arange(6), (2,3))
XS = np.dot(np.ones((1,2)), X)
print(XS)
XS2 = np.sum(X, axis=0)
assert np.allclose(XS, XS2)

```

    [[3. 5. 7.]]

```python
# We can sum across the columns

X = np.reshape(np.arange(6), (2,3))
XS = np.dot(X, np.ones((3,1)))
print(XS)
XS2 = np.sum(X, axis=1).reshape(-1, 1)
assert np.allclose(XS, XS2)

```

    [[ 3.]
     [12.]]

```python
# We can sum across all entries

X = np.reshape(np.arange(6), (2,3))
S1 = np.dot(np.ones((1,2)), np.dot(X, np.ones((3,1))))[0]
S2 = np.sum(X)
assert np.allclose(S1, S2)

```

```python
# Kronecker product

np.kron(np.eye(2), np.ones((2,2)))

```

    DeviceArray([[1., 1., 0., 0.],
                 [1., 1., 0., 0.],
                 [0., 0., 1., 1.],
                 [0., 0., 1., 1.]], dtype=float32)

```python
# Vector Norms
x = np.arange(6)
print(np.linalg.norm(x, 2) ** 2)
print(np.sum(np.power(x, 2)))
print(np.linalg.norm(x, np.inf))

# Matrix norms
A = onp.random.randn(4,4)
print(onp.linalg.norm(A, ord=2)) # not supported by jax
print(onp.linalg.norm(A, ord='nuc')) # not supported by jax
print(np.linalg.norm(A, ord='fro'))

```

    54.999996
    55
    5.0
    2.8985264155894463
    6.954729906402635
    4.0065484

```python
# Size of a matrix

print(np.trace(A))
print(onp.linalg.det(A)) # not supported by jax
print(onp.linalg.cond(A)) # not supported by jax
```

    0.61824524
    3.76606451160376
    5.924791970356876

## Sparse matrices

```python
from scipy.sparse import diags
A = diags([1,2,3])
print(A)
print(A.toarray())

```

      (0, 0)	1.0
      (1, 1)	2.0
      (2, 2)	3.0
    [[1. 0. 0.]
     [0. 2. 0.]
     [0. 0. 3.]]

```python
# Block diagonal

from scipy.linalg import block_diag
block_diag([2, 3], [[4, 5], [6, 7]])

```

    array([[2, 3, 0, 0],
           [0, 0, 4, 5],
           [0, 0, 6, 7]])

Band diagonal

See (https://pypi.org/project/bandmat)

## Broadcasting

In numpy, the command A * B computes the elementwise multiplication of arrays or tensors A and B.
If these arrays have different shapes,
they will be automatically converted to have compatible shapes by
implictly replicating  certain dimensions; this is called
**broadcasting**. The following conversion rules are applied
in order:

* If the two arrays differ in their number of dimensions, the
   shape of the one with fewer dimensions is padded with ones on the
   left side. For example, a scalar will be converted to a vector,
   and a vector to a matrix with one row.
* If the shape of the two arrays does not match in any dimension,
   the array with shape equal to 1 in that dimension is stretched to
   match the other shape, by replicating the corresponding contents.
* If in any dimension the sizes disagree and neither is equal to
   1, an error is raised.

Figure made by [broadcasting_fig.py](https://github.com/probml/pyprobml/blob/master/scripts/broadcasting_fig.py) by Jake VanderPlas.

```python
# Example: scaling each column
X = np.reshape(np.arange(6), (2,3))
s = np.array([1,2,3])
XS = X * s
print(XS)
XS2 = np.dot(X, np.diag(s)) # post-multiply by diagonal
assert np.allclose(XS, XS2)

```

    [[ 0  2  6]
     [ 3  8 15]]

```python
# Example: scaling each row
X = np.reshape(np.arange(6), (2,3))
s  = np.array([1,2])
XS = X *  np.reshape(s, (-1,1))
print(XS)
XS2 = np.dot(np.diag(s), X) # pre-multiply by diagonal
assert np.allclose(XS, XS2)
```

    [[ 0  1  2]
     [ 6  8 10]]

## Einstein summation

Einstein summation lets us write formula such as  inputs -> outputs, which name the dimensions
of the input tensor and output tensors; dimensions which are not named in the output are summed over - this is called **tensor contraction**.

```python
# Sample data
a = np.arange(3)
b = np.arange(3)
A = np.arange(6).reshape(2,3)
B = np.arange(15).reshape(3,5)
S = np.arange(9).reshape(3,3)
T = onp.random.randn(2,2,2,2)
```

Now consider einsum with  a single tensor.

```python

# Matrix transpose
assert np.allclose(A.T, np.einsum('ij->ji', A))

# Sum all elements
assert np.allclose(np.sum(A), np.einsum('ij->', A))

# Sum across rows
assert np.allclose(np.sum(A, axis=0), np.einsum('ij->j', A))

# Sum across columns
assert np.allclose(np.sum(A, axis=1), np.einsum('ij->i', A))

# Sum specific axis of tensor
assert np.allclose(np.sum(T, axis=1), np.einsum('ijkl->ikl', T))
assert np.allclose(np.sum(np.sum(T, axis=0), axis=0), np.einsum('ijkl->kl', T))

# repeated indices with one arg extracts diagonals
assert np.allclose(np.diag(S), np.einsum('ii->i', S))

# Trace
assert np.allclose(np.trace(S), np.einsum('ii->', S))

```

Now consider einsum with 2 tensors.

```python

# Matrix vector multiplication
assert np.allclose(np.dot(A, b), np.einsum('ik,k->i', A, b))

# Matrix matrix multiplication
assert np.allclose(np.dot(A, B), np.einsum('ik,kj->ij', A, B))
assert np.allclose(np.matmul(A, B), np.einsum('ik,kj->ij', A, B))

# Inner product
assert np.allclose(np.dot(a, b), np.einsum('i,i->', a, b))
assert np.allclose(np.inner(a, b), np.einsum('i,i->', a, b))

# Outer product
assert np.allclose(np.outer(a, b), np.einsum('i,j->ij', a, b))

# Elementwise product
assert np.allclose(a * a, np.einsum('i,i->i', a, a))
assert np.allclose(A * A, np.einsum('ij,ij->ij', A, A))
assert np.allclose(np.multiply(A, A), np.einsum('ij,ij->ij', A, A))

```

 As a more complex example,
 suppose we have a 3d tensor $S_{ntk}$ where $n$ indexes examples in the
 batch, $t$ indexes locations in the sequence, and $k$ indexes words
 in a one-hot representation.
 Let $W_{kd}$ be an embedding matrix that maps sparse one-hot vectors
 $R^k$  to dense vectors in $R^d$.
 We can convert the batch of sequences of one-hots
 to a batch of sequences of embeddings as follows:
$$
E_{ntd} = \sum_k S_{ntk} W_{kd}
$$
We can compute the sum of the embedding vectors for
each sequence (to get a global representation
of each bag of words) as follows:
$$
E_{nd} = \sum_k \sum_t S_{ntk} W_{kd}
$$
Finally we can pass each sequence's vector representation
through another linear transform $V_{dc}$ to map to the logits over a
classifier
with $c$ labels:
$$
L_{nc} = \sum_d E_{nd} V_{dc}
= \sum_d \sum_k \sum_t S_{ntk} W_{kd} V_{dc}
$$
In einsum notation, we have
$$
L_{nc} = S_{ntk} W_{kd} V_{dc}
$$
We sum  over $k$ and $d$  because those
indices occur twice on the RHS.
We sum over $t$  because that index does not occur
on the LHS.

```python
# sentence embedding example in code

N = 2; C = 3; D = 4; K = 5; T = 6;
S = onp.random.randn(N, T, K)
W = onp.random.randn(K, D)
V = onp.random.randn(D, C)
Lfast = np.einsum('ntk,kd,dc->nc', S, W, V)
# Compare to brute force way of computing L below.
# We can only do elementwise assignment to L in original numpy, not jax
L = onp.zeros((N,C))
for n in range(N):
    for c in range(C):
        s = 0
        for d in range(D):
            for k in range(K):
                for t in range(T):
                    s += S[n,t,k] * W[k,d] * V[d,c]
        L[n,c] = s # does not work in jax
assert np.allclose(L, Lfast)
```

```python
# Optimization

path = np.einsum_path('ntk,kd,dc->nc', S, W, V, optimize='optimal')[0]
assert np.allclose(L, np.einsum('ntk,kd,dc->nc', S, W, V, optimize=path))

```

## Eigenvalue decomposition (EVD)

```python
onp.random.seed(42)
M = onp.random.randn(4, 4)
A = M  + M.T # ensure symmetric
assert (A == A.T).all() # check symmetric
evals, evecs = np.linalg.eigh(A) # tell JAX matrix is symmetric
#evals, evecs = onp.linalg.eig(A)
print(evals)
print(evecs)

# Sort columns so one with largest evals (absolute value) are first
idx = np.argsort(np.abs(evals))[::-1] # largest first
evecs = evecs[:, idx] # sort columns

evals = evals[idx]
print(evals)
print(evecs)
```

    [-3.806 -2.09   1.016  3.353]
    [[ 0.277  0.202 -0.824  0.45 ]
     [ 0.246 -0.751 -0.369 -0.489]
     [-0.68   0.316 -0.427 -0.505]
     [-0.633 -0.543 -0.045  0.55 ]]
    [-3.806  3.353 -2.09   1.016]
    [[ 0.277  0.45   0.202 -0.824]
     [ 0.246 -0.489 -0.751 -0.369]
     [-0.68  -0.505  0.316 -0.427]
     [-0.633  0.55  -0.543 -0.045]]

### Example: Diagonalizing a rotation matrix

As an example, let us construct $A$
by combining
a rotation by 45 degrees about the $z$ axis,
a scaling by $\diag(1,2,3)$, followed by another rotation of -45 degrees.
These components can be recovered from $A$ using EVD, as we show below.

```python
a = (45/180) * np.pi
R = np.array(
        [[np.cos(a), -np.sin(a), 0],
          [np.sin(a), np.cos(a), 0],
          [0, 0, 1]])
print(R)

S = np.diag(np.array([1.0, 2.0, 3.0]))
A = np.dot(np.dot(R, S), R.T) # Rotate, scale, then unrotate

evals, evecs = np.linalg.eig(A)
idx = np.argsort(np.abs(evals)) # smallest first
U = evecs[:, idx] # sort columns

evals2 = evals[idx]
D = np.diag(evals2)

assert np.allclose(A, np.dot(U, np.dot(D, U.T))) # eigen decomposition
assert np.allclose(np.abs(R), np.abs(U)) # Recover rotation
assert np.allclose(D, S)  # Recover scale

```

    [[ 0.707 -0.707  0.   ]
     [ 0.707  0.707  0.   ]
     [ 0.     0.     1.   ]]

### Example: checking for positive definitness

A symmetric matrix is positive definite iff all its eigenvalues are positive.

```python
onp.random.seed(42)
M = onp.random.randn(3, 4)
A = np.dot(M, M.T) # ensure A is positive definite

def is_symmetric(A):
  return (A == A.T).all()

def isposdef(A):
  if not is_symmetric(A):
    return False
  evals, evecs = np.linalg.eigh(A)
  return np.all(evals > 0)

print(isposdef(A))
```

    True

### Power method

```python
from numpy.linalg import norm

onp.random.seed(0)

def power_method(A, max_iter=100, tol=1e-5):
    n = np.shape(A)[0]
    u = onp.random.rand(n)
    converged = False
    iter = 0
    while (not converged) and (iter < max_iter):
        old_u = u
        u = np.dot(A, u)
        u = u / norm(u)
        lam = np.dot(u, np.dot(A, u))
        converged = (norm(u - old_u) < tol)
        iter += 1
    return lam, u

X = onp.random.randn(10, 5)
A = np.dot(X.T, X) # psd matrix
lam, u = power_method(A)

evals, evecs = np.linalg.eigh(A)
idx = np.argsort(np.abs(evals))[::-1] # largest first
evals = evals[idx]
evecs = evecs[:,idx]

tol = 1e-3
assert np.allclose(evecs[:,0], u, tol)
```

## Singular value decomposition (SVD)

```python
onp.random.seed(0)

A = onp.random.randn(10, 5)

U, S, V = np.linalg.svd(A,full_matrices=False)
print("Full=False: shape of U {}, S {}, V {}".format(U.shape, S.shape, V.shape))

U, S, V = np.linalg.svd(A,full_matrices=True)
print("Full=True: shape of U {}, S {}, V {}".format(U.shape, S.shape, V.shape))

```

    Full=False: shape of U (10, 5), S (5,), V (5, 5)
    Full=True: shape of U (10, 10), S (5,), V (5, 5)

```python
onp.random.seed(0)

def make_random_low_rank(D, K):
  A = onp.zeros((D, D), dtype=np.float32) # we use onp so we can mutate A in place
  for i in range(K):
    x = onp.random.randn(D)
    A = A + np.outer(x, x)
  return A

A = make_random_low_rank(10, 3)
U, S, V = np.linalg.svd(A,full_matrices=False)
print(np.sum(S > 1e-5))
print(onp.linalg.matrix_rank(A))

```

    3
    3

## Low rank approximation to an image

```python
import matplotlib.image

def rgb2gray(rgb):
  #Y' = 0.2989 R + 0.5870 G + 0.1140 B
  return onp.dot(rgb[...,:3], [0.2989, 0.5870, 0.1140])

#url = 'https://github.com/probml/pyprobml/blob/master/data/clown.png'
#img = matplotlib.image.imread(url) # invalid png file, apparently...
#X = rgb2gray(img)

import scipy
racoon = scipy.misc.face().astype(onp.float)
X = rgb2gray(racoon)
plt.gray()
plt.imshow(X)
plt.show()

r = onp.linalg.matrix_rank(X)
print(r)

U, sigma, V = np.linalg.svd(X, full_matrices=True)
ranks = [1, 2, 5, 10, 20, r]
R = len(ranks)

fig, axes = plt.subplots(2, 3, figsize=[15, 10])
axes = axes.reshape(-1)
for i in range(R):
    k = ranks[i]
    x_hat = np.dot(np.dot(U[:, :k], np.diag(sigma[:k])), V[:k, :])
    ax = axes[i]
    #plt.subplot(2, 3, i+1)
    ax.imshow(x_hat, cmap='gray')
    ax.set_title("rank {}".format(k))
    #save_fig("svdImageDemoClown{}.pdf".format(k))
plt.show()

```

```python
# Plot singular value spectrum
k = 100
plt.figure()
plt.plot(np.log(sigma[:k]), 'r-', linewidth=4, label="Original")
plt.ylabel(r"$log(\sigma_i)$")
plt.xlabel("i")

# Compare this to a random shuffled version of the image
x2 = onp.random.permutation(X)
# so we convert to a 1d vector, permute, and convert back
x1d = X.ravel()
onp.random.shuffle(x1d) # inplace
x2 = x1d.reshape(X.shape)
U, sigma2, V = np.linalg.svd(x2, full_matrices = False)
plt.plot(np.log(sigma2[:k]), 'g:', linewidth=4, label="Randomized")
plt.legend()
#save_fig("svdImageDemoClownSigmaScrambled.pdf")
plt.show()
```

## Other matrix decompositions

In this section, we illustrate a few other matrix decompositions.

### LU decomposition

```python

onp.random.seed(42)
A = onp.random.randn(5,5)
L, U = scipy.linalg.lu(A, True)
print(L)
print(U)
```

    [[ 0.339 -0.04   0.733  1.     0.   ]
     [-0.16   1.     0.     0.     0.   ]
     [-0.316 -0.348  0.597 -0.423  1.   ]
     [-0.384 -0.712  1.     0.     0.   ]
     [ 1.     0.     0.     0.     0.   ]]
    [[ 1.466 -0.226  0.068 -1.425 -0.544]
     [ 0.     1.543  0.778 -0.697  0.456]
     [ 0.     0.     0.895 -1.951 -1.297]
     [ 0.     0.     0.     3.409  0.919]
     [ 0.     0.     0.     0.    -0.576]]

### QR decomposition

```python
# Economy vs full mode

onp.random.seed(42)
A = onp.random.randn(5,3)
Q, R = scipy.linalg.qr(A, mode='economic')
print("economy: Q shape {}, R shape {}".format(Q.shape, R.shape))
print(Q)
print(R)

Q, R = scipy.linalg.qr(A, mode='full')
print("full: Q shape {}, R shape {}".format(Q.shape, R.shape))
print(Q)
print(R)
assert np.allclose(np.eye(5), np.dot(Q, Q.T), atol=1e-3)
```

    economy: Q shape (5, 3), R shape (3, 3)
    [[-0.213 -0.068  0.765]
     [-0.655 -0.119  0.303]
     [-0.679  0.35  -0.49 ]
     [-0.233 -0.221  0.004]
     [-0.104 -0.9   -0.29 ]]
    [[-2.327 -0.031  0.622]
     [ 0.     2.13   1.474]
     [ 0.     0.     1.152]]
    full: Q shape (5, 5), R shape (5, 3)
    [[-0.213 -0.068  0.765 -0.018  0.604]
     [-0.655 -0.119  0.303 -0.248 -0.635]
     [-0.679  0.35  -0.49  -0.046  0.418]
     [-0.233 -0.221  0.004  0.943 -0.085]
     [-0.104 -0.9   -0.29  -0.215  0.222]]
    [[-2.327 -0.031  0.622]
     [ 0.     2.13   1.474]
     [ 0.     0.     1.152]
     [ 0.     0.     0.   ]
     [ 0.     0.     0.   ]]

### Cholesky decomposition

```python
# Sample from multivariate Gaussian

from scipy.stats import multivariate_normal as mvn

def sample_mvn(mu, Sigma, N):
    L = np.linalg.cholesky(Sigma)
    D = len(mu)
    Z = onp.random.randn(N, D)
    X = np.dot(Z, L.T) + np.reshape(mu, (-1,D))
    return X

D = 5
onp.random.seed(42)
mu = onp.random.randn(D)
A = onp.random.randn(D,D)
Sigma = np.dot(A, A.T)
N = 10000
X = sample_mvn(mu, Sigma, N)
mu_hat = np.mean(X)
C = onp.cov(X, rowvar=False) # not yet implemented by jax
print(C)
print(Sigma)
assert np.allclose(C, Sigma, 1e-0) # not that close, even after 10k samples...

dist = mvn(mu, Sigma)
X = dist.rvs(size=N)
C = onp.cov(X, rowvar=False)
assert np.allclose(C, Sigma, 1e-0)

```

    [[ 3.762 -0.445 -1.572 -0.246 -1.439]
     [-0.445  7.072  4.914  3.109  2.191]
     [-1.572  4.914  4.203  1.467  2.151]
     [-0.246  3.109  1.467  4.522  1.431]
     [-1.439  2.191  2.151  1.431  1.896]]
    [[ 3.652 -0.479 -1.567 -0.274 -1.432]
     [-0.479  7.126  4.982  3.107  2.228]
     [-1.567  4.982  4.26   1.488  2.179]
     [-0.274  3.107  1.488  4.53   1.462]
     [-1.432  2.228  2.179  1.462  1.924]]

## Matrix calculus

### Automatic differentiation in Jax

In this section, we show how to use Jax to compute gradients, Jacobians and Hessians
of some simple convex functions.

```python
from jax import grad, hessian, jacfwd, jacrev, vmap, jit
```

Linear function: multi-input, scalar output.

$$
\begin{align}
f(x; a) &= a^T x\\
\nabla_x f(x;a) &= a
\end{align}
$$

```python
# We construct a single output linear function.
# In this case, the Jacobian and gradient are the same.
Din = 3; Dout = 1;
onp.random.seed(42)
a = onp.random.randn(Dout, Din)
def fun1d(x):
    return np.dot(a, x)[0]
x = onp.random.randn(Din)
g = grad(fun1d)(x)
assert np.allclose(g, a)
J = jacrev(fun1d)(x)
assert np.allclose(J, g)

```

Linear function: multi-input, multi-output.

$$
\begin{align}
f(x;A) &= A x \\
\nabla_x f(x;A) &= A
\end{align}
$$

```python
# We construct a multi-output linear function.
# We check forward and reverse mode give same Jacobians.
Din = 3; Dout = 4;
A = onp.random.randn(Dout, Din)
def fun(x):
    return np.dot(A, x)
x = onp.random.randn(Din)
Jf = jacfwd(fun)(x)
Jr = jacrev(fun)(x)
assert np.allclose(Jf, Jr)
assert np.allclose(Jf, A)
```

Quadratic form.

$$
\begin{align}
f(x;A) &= x^T A x \\
\nabla_x f(x;A) &= (A+A^T) x \\
\nabla^2 x^2 f(x;A) &= A + A^T
\end{align}
$$

```python

D = 4
A = onp.random.randn(D, D)
x = onp.random.randn(D)
quadfun = lambda x: np.dot(x, np.dot(A, x))

J = jacfwd(quadfun)(x)
assert np.allclose(J, np.dot(A+A.T, x))

H1 = hessian(quadfun)(x)
assert np.allclose(H1, A+A.T)

def my_hessian(fun):
  return jacfwd(jacrev(fun))
H2 = my_hessian(quadfun)(x)
assert np.allclose(H1, H2)
```

## Solving linear systems of equations

### Square systems with unique solution

```python
A = np.array([[3,2,-1], [2, -2, 4], [-1, 0.5, -1]])
b = np.array([1, -2, 0])
x  = jax.scipy.linalg.solve(A,b)
print(x)
print(np.dot(A, x) - b)
```

    [ 1. -2. -2.]
    [ 0.000e+00 -4.768e-07  0.000e+00]

```python
# Now use LU decomposition and backsubstitution.

L, U = jax.scipy.linalg.lu(A, permute_l=True)
print(L)
print(U)
y = jax.scipy.linalg.solve_triangular(L, b, lower=True)
x = jax.scipy.linalg.solve_triangular(U, y, lower=False)
print(x)
```

    [[ 1.     0.     0.   ]
     [ 0.667  1.     0.   ]
     [-0.333 -0.35   1.   ]]
    [[ 3.     2.    -1.   ]
     [ 0.    -3.333  4.667]
     [ 0.     0.     0.3  ]]
    [ 1. -2. -2.]

### Underconstrained systems: least norm solution

We will compute the minimum norm solution.

```python
onp.random.seed(42)
m = 3
n = 4
A = onp.random.randn(m, n)
x = onp.random.randn(n)
b = np.dot(A, x)

#x_least_norm = jax.scipy.linalg.lstsq(A, b)[0]
x_least_norm = scipy.linalg.lstsq(A, b)[0]
print(x_least_norm)
print(np.dot(A, x_least_norm) - b)
print(np.linalg.norm(x_least_norm, 2))

```

    [ 1.162 -1.083 -1.26  -0.985]
    [ 0.000e+00  0.000e+00 -1.192e-07]
    2.2538505

If you look at the [source code for scipy.linalg.lstsq](https://github.com/scipy/scipy/blob/v0.19.0/scipy/linalg/basic.py#L892-L1058),
you will see that it just a Python wrapper
to some LAPACK code written in Fortran. LAPACK offers multiple methods for solving
linear systems, including `gelsd` (default), `gelss` , and `gelsy`. The first two methods use SVD, the latter uses QR decomposition.

A lot of numpy and scipy functions are just wrappers to legacy libraries,
  written in Fortran or C++, since Python itself is too slow for
  efficient numerical computing.
  Confusingly, sometimes numpy and scipy offer different wrappers to the same
  underlying LAPACK functions, but with different interfaces.
  For example, as of 2018, `np.linalg.lstsq` and
  `scipy.linalg.lstsq` have been modified to behave the same.
  However,
  `np.linalg.qr` and `scipy.linalg.qr`
  have slightly different optional arguments and may give different
  results.

Jax does not yet implement lstsq, but does implement some of the underlying methods
 [here](https://github.com/google/jax/blob/master/jax/scipy/linalg.py).
 Unlike the legacy code, this can run fast on GPUs and TPUs.

### Overconstrained systems: least squares solution

```python
def naive_solve(A, b):
    return jax.numpy.linalg.inv(A.T @ A) @ A.T @ b
    #return jnp.linalg.inv(A.T @ A) @ A.T @ b

def qr_solve(A, b):
    Q, R = np.linalg.qr(A)
    Qb = np.dot(Q.T,b)
    return jax.scipy.linalg.solve_triangular(R, Qb)

def lstsq_solve(A, b):
    return scipy.linalg.lstsq(A, b, rcond=None)[0]

def pinv_solve(A, b):
    return np.dot(np.linalg.pinv(A), b)

onp.random.seed(42)
m = 4
n = 3
A = onp.random.randn(m, n)
x = onp.random.randn(n)
b = np.dot(A, x)

methods = list()
solns = list()

methods.append('naive')
solns.append(naive_solve(A, b))

#methods.append('pinv')
#solns.append(pinv_solve(A,b)) # pinv not yet implemented by jax

#methods.append('lstsq')
#solns.append(lstsq_solve(A, b)) # lstsq not yet implemented by jax

methods.append('qr')
solns.append(qr_solve(A, b))

for (method, soln) in zip(methods, solns):
    residual = b -  np.dot(A, soln)
    print('method {}, norm {:0.5f}, residual {:0.5f}'.format(method, np.linalg.norm(soln), np.linalg.norm(residual)))
    print(soln.T)
    print('\n')
```

    method naive, norm 2.58738, residual 0.00000
    [ 0.242 -1.913 -1.725]

    method qr, norm 2.58738, residual 0.00000
    [ 0.242 -1.913 -1.725]

```python

```


# Tutoriel MID Générique
<u>Objectif</u> : Illustrer les propriétés de la MID.

Concrètement, nous considérons un système qui a un controle par une loi de commande retardée (expression mathématique) et l'on prend compte et tirons avantage de ce retard dans le calcul des gains de la loi de commande retardée pour stabiliser le système.

<u> Informations importantes </u> :

- Stabilité : Lorsque l'on a l'équation décrivant un système dans l'espace de Laplace (transformée de Laplace), la stabilité est traduite par l'ensemble des racines (complexes) de l'équation ayant leurs parties réelles négatives. On dit que les racines sont à gauche (de l'axe des ordonnées).
- Transformée de Laplace (juste pour info) : Dans notre cas, la partie la plus importante dans la transformée de Laplace est la dérivation, c'est ce qui nous permet de passer d'une équation différentielle à une équation polynomiale. L'idée très grossière est qu'une dérivée d'ordre $n$ dans l'espace "classique"/"temporel" correspond à une multiplication par un terme $s^n$ dans l'espace de Laplace (fréquentiel). Donc, par exemple, $4\times k \times y''(t)\rightarrow 4ks^2y(s)$

On traite une équation différentielle du type :
$$
\frac{d^n y(t)}{dt^n} + \sum\limits_{i=0}^{n-1}a_i\frac{d^i y(t)}{dt^i} + \sum\limits_{j=0}^m \alpha_j\frac{d^j y(t-\tau)}{dt^j}=0
$$

- $n, m\in\mathbb{N}, n>m$
- $a_i, \alpha_j \in \mathbb{R}$
- $s\in\mathbb{C}$,
- $\tau \in \mathbb{R}$

Par une transformée de Laplace :
$$
Q(s) = s^n+a_{n-1}s^{n-1}+...+a_0 + \left[\alpha_ms^m+...+\alpha_0\right]e^{-s\tau}
$$

Cette équation est polynomiale et non différentielle ce qui la rend beaucoup plus simple à traiter.

On souhaite réaliser ce que l'on appelle un placement de pôle. Cela signifie dans ce cas que l'on veut trouver les coefficients ($a_i$ et $\alpha_j$ parfois notés $b_j$) soient de telle sorte que $Q(s_0)=0$.

On connaît donc une valeur de racine à "forcer" pour le quasipolynôme en jouant sur les coefficients.

Le code suivant a été développé sur la base du pseudo-code de la MID Générique, décrit par Franck, puis adapté en version interactive.

Version : 1.3 (01/04/2021)

```python
from IPython.core.display import display, HTML
display(HTML("<style>div.output_scroll { height: 44em; }</style>"))
%matplotlib notebook
import numpy as np
import sympy as sp
import cxroots as cx
import matplotlib.pyplot as plt
import ipywidgets as widgets
from IPython.display import display

def MID_function(n,m,s0,value_tau):

    s = sp.symbols('s')  # define variable s for our problem to be solved
    tau = sp.symbols('tau')  # define variable tau : delay
    a = sp.symbols(["a{:d}".format(i) for i in range(n)], real=True)
    alpha = sp.symbols(["alpha{:d}".format(i) for i in range(m + 1)], real=True)
    Polynomial = s**n + np.array(a).dot([s**i for i in range(n)]) # Revient à faire s^n + a_{n-1}^{n-1}...
    Delayed = np.array(alpha).dot([s**i for i in range(m+1)])*sp.exp(-s*tau) # Revient à faire
    Q = Polynomial + Delayed
    SysDerivatif = [Q]
    for i in range(n+m+1):
        DerniereDerivee = SysDerivatif[-1]
        SysDerivatif.append(DerniereDerivee.diff(s)) # Dérivée par rapport à s
    sol = sp.linsolve(SysDerivatif[:-1], alpha + a).args[0] # Solveur selon les alpha et les a
    solNum = sol.subs({s : s0})
    solNum = solNum.subs({tau : value_tau})
    a_num = list(solNum[m + 1:])
    alpha_num = list(solNum[:m + 1])
    QNumerique = s**n + np.array(a_num).dot([s**i for i in range(n)])+\
    np.array(alpha_num).dot([s**i for i in range(m+1)])*sp.exp(-s*tau)
    QNumerique = QNumerique.subs(tau, value_tau)
    sysRootFinding = [QNumerique, QNumerique.diff(s)]
    sysFunc = [sp.lambdify(s, i) for i in sysRootFinding]
    rect = cx.Rectangle([-100, 10], [-100, 100])
    roots = rect.roots(sysFunc[0], sysFunc[1], rootErrTol=1e-5, absTol=1e-5, M = n + m + 1)
    xroot = np.real(roots[0])
    yroot = np.imag(roots[0])
    return xroot,yroot,QNumerique
```

```python
output_slider = widgets.Output()
output_equation = widgets.Output()

slider_n = widgets.IntSlider(min=0,max=3,step=1,description='n :',value=2)
slider_m = widgets.IntSlider(min=0,max=3,step=1,description='m :',value=1)
slider_s0 = widgets.IntSlider(min=-6,max=-4,step=1,description='s0 :',value=-5)
slider_tau = widgets.IntSlider(min=1,max=2,step=1,description='tau :',value=1)

def common_slider(n,m,s0,value_tau):
    output_slider.clear_output()
    output_equation.clear_output()

    with output_slider:
        fig, ax = plt.subplots(figsize=(6, 4))
        ax.set_title("Root spectrum")
        ax.set_xlabel(r"Re$(s)$")
        ax.set_ylabel(r"Im$(s)$")
        ax.axhline(linewidth=2, color='black', zorder = 2)
        ax.axvline(linewidth=2, color='black', zorder = 2)
        ax.axvspan(0, 1, alpha=0.5, color='red', zorder = 2)
        ax.grid()
        line, = ax.plot([], [], 'o', color='steelblue', zorder = 3)

        xr,yr,eq = MID_function(n,m,s0,value_tau)
        line.set_data(xr, yr)
        ax.relim()
        ax.autoscale_view()
        fig.canvas.draw()

    with output_equation :
        print ("Fp = ",eq)

def slider_n_eventhandler(change):
    common_slider(change.new, slider_m.value, slider_s0.value, slider_tau.value)
def slider_m_eventhandler(change):
    common_slider(slider_n.value, change.new, slider_s0.value, slider_tau.value)
def slider_s0_eventhandler(change):
    common_slider(slider_n.value,slider_m.value , change.new, slider_tau.value)
def slider_tau_eventhandler(change):
    common_slider(slider_n.value,slider_m.value , slider_s0.value, change.new)

slider_n.observe(slider_n_eventhandler, names='value')
slider_m.observe(slider_m_eventhandler, names='value')
slider_s0.observe(slider_s0_eventhandler, names='value')
slider_tau.observe(slider_tau_eventhandler, names='value')

input_widgets = widgets.HBox([slider_n, slider_m, slider_s0, slider_tau])

tab = widgets.Tab([output_slider, output_equation])
tab.set_title(0, 'Root plot ')
tab.set_title(1, 'Output equation')

dashboard = widgets.VBox([widgets.Label('Insert degree of polynomial n, degree of delay polynomial m, s0 and tau :'),input_widgets, tab])
```

```python
display(dashboard)
```