# Kernel Density Estimation

This notebook is an introduction into the practical usage of KDEs in zfit and explains the different parameters.
*A complete introduction to Kernel Density Estimations, explanations to all methods implemented in zfit and a throughout
comparison of the performance can be either found in
[Performance of univariate kernel density estimation methods in TensorFlow](https://astroviking.github.io/ba-thesis/)
by Marc Steiner from which parts here are taken or in the [documentation of zfit](https://zfit.readthedocs.io/en/latest/)*

Kernel Density Estimation is a non-parametric method to estimate the density of a population and offers a more accurate way than a
histogram.
In a kernel density estimation each data point is substituted with a kernel function
that specifies how much it influences its neighboring regions. This kernel functions can then be summed up to get an
estimate of the probability density distribution, quite similarly as summing up data points inside bins.

However, since
the kernel functions are centered on the data points directly, KDE circumvents the problem of arbitrary bin positioning.
KDE still depends on the kernel bandwidth (a measure of the spread of the kernel function), however, the total PDF
does depend less strongly on the kernel bandwidth than histograms do on bin width and it is much easier to specify
rules for an approximately optimal kernel bandwidth than it is to do so for bin width.

## Definition

Given a set of $n$ sample points $x_k$ ($k = 1,\cdots,n$), kernel density estimation $\widehat{f}_h(x)$ is defined as

\begin{equation}
\widehat{f}_h(x) = \frac{1}{nh} \sum_{k=1}^n K\Big(\frac{x-x_k}{h}\Big)
\end{equation}

where $K(x)$ is called the kernel function, $h$ is the bandwidth of the kernel and $x$ is the value for which the estimate is calculated. The kernel function defines the shape and size of influence of a single data point over the estimation, whereas the bandwidth defines the range of influence. Most typically a simple Gaussian distribution ($K(x) :=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}$) is used as kernel function.
The larger the bandwidth parameter $h$ the larger is the range of influence of a single data point on the estimated distribution.

### Weights

It is straightforward to add event weights in the above expression. zfit KDE fully support weights in data samples and will be taken into account automatically in the KDE.

## Computational complexity

This leads for a large n however to computational problems, as the computational complexity of the exact KDE above is given by $\mathcal{O}(nm)$ where $n$ is the number of sample points to estimate from and $m$ is the number of evaluation points (the points where you want to calculate the estimate).

To circumvent this problem, there exist several approximative methods to decrease this complexity and therefore decrease the runtime as well.

**Hint on running the notebook**

Feel free to rerun a cell a few times. This will change the sample drawn and gives an impression of _how the PDF based on this sample could also look like_.

```python
import matplotlib.pyplot as plt
import mplhep
import numpy as np
import zfit

np.random.seed(23)
```

## Exact KDE

Using the definition above of a KDE, this is implemented in the `KDE1DimExact`. We can start out with a simple Gaussian.

```python
obs_wide = zfit.Space('x', (-10, 10))
x = np.linspace(-10, 10, 200)
```

```python
gauss = zfit.pdf.Gauss(obs=obs_wide, mu=0, sigma=2)
sample = gauss.sample(60)
sample_np = zfit.run(sample.value())

kde = zfit.pdf.KDE1DimExact(sample,
                            # obs,
                            # kernel,
                            # padding,
                            # weights,
                            # name
                            )
plt.plot(x, kde.pdf(x), label='Default exact KDE')
plt.plot(x, gauss.pdf(x), label='True PDF')
plt.plot(sample_np, np.zeros_like(sample_np), 'b|', ms=12)
plt.legend()
```

This looks already reasonable and we see that the PDF is overestimated in the regions where we sampled, by chance, many events and underestimated in other regions. Since this was a simple example, /et's create a more complitated one (and let's use a bit more samples in order to be able to infer the shape).

```python
gauss1 = zfit.pdf.Gauss(obs=obs_wide, mu=0, sigma=2)
gauss2 = zfit.pdf.Gauss(obs=obs_wide, mu=3, sigma=0.5)
true_pdf = zfit.pdf.SumPDF([gauss1, gauss2], fracs=0.85)
sample = true_pdf.sample(1000)
sample_np = zfit.run(sample.value())

kde = zfit.pdf.KDE1DimExact(sample,
                            # obs,
                            # kernel,
                            # padding,
                            # weights,
                            # name
                            )
plt.plot(x, kde.pdf(x), label='Default exact KDE')
plt.plot(x, true_pdf.pdf(x), label='True PDF')
plt.legend()
```

This is more difficult, actually impossible for the current configuration, to approximate the actual PDF well, because we use by default a single bandwidth.

## Bandwidth

The bandwidth of a kernel defines it's width, corresponding to the `sigma` of a Gaussian distribution. There is a distinction between global and local bandwidth:

Global bandwidth

A is a single parameter that is shared amongst all kernels.
      While this is a fast and robust method,
      it is a rule of thumb approximation. Due to its global nature,
      it cannot take into account the different varying
      local densities.

Local bandwidth

A local bandwidth
      means that each kernel $i$ has a different bandwidth.
      In other words, given some data points with size $n$,
      we will need $n$ bandwidth parameters.
      This is often more accurate than a global bandwidth,
      as it allows to have larger bandwiths in areas of smaller density,
      where, due to the small local sample size, we have less certainty
      over the true density while having a smaller bandwidth in denser
      populated areas.

We can compare the effects of different global bandwidths

```python
plt.plot(x, true_pdf.pdf(x), label='True PDF')

for h in [0.1, 0.5, 2, 10]:
    kde = zfit.pdf.KDE1DimExact(sample, bandwidth=h)
    plt.plot(x, kde.pdf(x), '--', alpha=0.6, label=f'KDE h={h}')
plt.legend()
```

Making the bandwidth larger makes the KDE less dependent on the randomness of the sample and overfitting of it while it tends to smear out features.
0.1 is clearly too wigly while 2 already smears out the feature of having actually two Gaussian peaks.

There are a few methods to estimate the optimal global bandwidth, Silvermans (default) and Scotts rule of thumb, respectively. There are also adaptive methods implemented that create an initial density estimation with a rule of thumb and use it to define local bandwidth that are inversely proportional to the (squareroot of) the density.

```python
kdes_all = [("silverman", '--'), ("scott", '--'), ("adaptive_geom", ':'), ("adaptive_zfit", ':'), ("isj", '-.')]
kdes_some = [("silverman", '--'), ("adaptive_zfit", ':'), ("isj", '-.')]
for subplot in range(1, 4):
    plt.figure(subplot)
    if subplot != 1:
        # let's zoom in to better see the details
        plt.xlim(-2.5, 4)
        plt.ylim(0.1, 0.2)
    plt.plot(x, true_pdf.pdf(x), 'k', label='True PDF')
    kdes = kdes_some if subplot == 3 else kdes_all
    for h, fmt in kdes:
        kde = zfit.pdf.KDE1DimExact(sample, bandwidth=h)
        plt.plot(x, kde.pdf(x), fmt, alpha=0.8, label=f'KDE {h}')
    plt.legend()
```

As we can see, the adaptive method and "isj" (we will see more on that later on) better takes into account the nuances of the peaks. It is very well possible to use local bandwidths directly as an array parameter to bandwidth.

## Kernel

The kernel is the heart of the Kernel Density Estimation, which consists of the sum of
kernels around each sample point. Therefore, a kernel should represent
the distribution probability of a single data point as close as
possible.

The most widespread kernel is a Gaussian, or Normal, distribution as many real world example follow it.
However, there are many cases where this assumption is not per-se true. In
this cases an alternative kernel may offers a better choice.

Valid choices are callables that return a
[tensorflow_probability.distribution.Distribution](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions?version=nightly), such as all distributions
that belong to the [loc-scale family](https://en.wikipedia.org/wiki/Location%E2%80%93scale_family).

```python
import tensorflow_probability as tfp

tfd = tfp.distributions
```

```python
plt.plot(x, true_pdf.pdf(x), label='True PDF')

for kernel in [tfd.Normal, tfd.Cauchy, tfd.Moyal]:
    kde = zfit.pdf.KDE1DimExact(sample, kernel=kernel)
    plt.plot(x, kde.pdf(x), '--', alpha=0.6, label=f'KDE {kernel.__name__}')
plt.legend()
```

## Boundary bias

The boundaries of KDEs turn out to be a bit tricky: As a KDE approximates the sample density, the density outside of our sample goes to zero. This means that the KDE close to the boundaries will drop close to zero, as a KDE represents the accumulation of a "local average" density.

To demonstrate this, let's use a perfect uniform kernel.

```python
uniform_x = np.linspace(-5, 5, 70)
uniform_sample = zfit.Data.from_numpy(obs=obs_wide, array=uniform_x)
kde = zfit.pdf.KDE1DimExact(uniform_sample)

plt.plot(x, kde.pdf(x), label='Rectangular KDE')
plt.plot(uniform_x, np.zeros_like(uniform_x), 'b|', ms=12, label='Sample')
plt.legend()
```

```python
obs = zfit.Space('x', (-2, 0.5))  # will cut of data at -2, 0.5
data_narrow = gauss.sample(1000, limits=obs)

kde = zfit.pdf.KDE1DimExact(data_narrow)

x = np.linspace(-2, 0.5, 200)
plt.plot(x, kde.pdf(x), label='Biased KDE')
plt.plot(x, gauss.pdf(x, obs), label='True PDF')
plt.legend()
```

While this is maybe not a surprise, this means that if we're only interested in the region of the sample, the KDE goes to zero too early. To illustrate, let's zoom in into the region of our sample. Then the PDF looks like this:

```python
plt.plot(uniform_x, kde.pdf(uniform_x), label='Rectangular KDE')
plt.plot(uniform_x, np.ones_like(uniform_x) / 10, label='True shape')
plt.plot(uniform_x, np.zeros_like(uniform_x), 'b|', ms=12, label='Sample')
plt.legend()
```

This effect is only a problem if the density of the sample itself does not go to zero by itself within our range of interest.

### Wider data

If the dataset that we have goes further than our PDF is meant to go, the best strategy is to simply provide a larger datasample.

The PDF takes an `obs` argument, which tells the "default space" and, most importantly, the normalization range. This does not need to coincide with the range of the given sample set, only the name of the obs has to.

Let's define our PDF in a smaller range therefore.

```python
limits_narrow = (-3, 3)
obs_narrow = zfit.Space('x', limits_narrow)

kde = zfit.pdf.KDE1DimExact(uniform_sample, obs=obs_narrow)

x_narrow = np.linspace(-3, 3, 100)
plt.plot(x_narrow, kde.pdf(x_narrow), label='Rectangular KDE')
plt.plot(x_narrow, np.ones_like(x_narrow) / 4, label='True shape')
plt.plot(x_narrow, np.zeros_like(x_narrow), 'b|', ms=12, label='Sample')
plt.legend()
```

### Padding

As we don't always have simply more data points outside of our region, the outside of the PDF can be padded in a different way: by mirroring the boundaries. This is effectively achieved by mirroring the dataset and can also be done manually.

The `padding` argument takes either a single float or a `dict` with "lowermirror" and/or "uppermirror" matched to a float to control mirroring the data on the lower respectively on the upper boundary of the data using. The number determines the fraction (in terms of difference between upper and lower boundary) of data that will be reflected.

This is effectively the same as requireing derivatives of zero at the boundary. While this helps lower the bias at the boundary, it is not a perfect solution.

```python
gauss = zfit.pdf.Gauss(obs=obs_narrow, mu=1, sigma=2)
sample = gauss.sample(2000)
sample_np = zfit.run(sample.value())

padding = 0.2
kde = zfit.pdf.KDE1DimExact(sample)

for pad in [{'uppermirror': 0.2}, padding]:
    kde_padded = zfit.pdf.KDE1DimExact(sample, padding=pad)

    plt.figure()
    plt.title(f'KDE padded with {pad}')
    plt.plot(x_narrow, kde_padded.pdf(x_narrow), label='KDE padded')
    plt.plot(x_narrow, kde.pdf(x_narrow), label='KDE no padding')
    plt.plot(x_narrow, gauss.pdf(x_narrow), label='True PDF')
    plt.legend()
```

As the PDF is not truncated on the edges but the sample data is effectively modified, we can plot the PDF in a larger range, which makes the effects visible.

```python
x_medium = np.linspace(-5, 5, 200)

plt.plot(x_medium, kde_padded.pdf(x_medium), label='KDE padded')
plt.plot(x_medium, kde.pdf(x_medium), label='KDE no padding')
plt.plot(x_medium, gauss.pdf(x_medium), label='True PDF')
width = (limits_narrow[1] - limits_narrow[0]) * padding
plt.bar([-3 - width, 3], height=[0.08, 0.2], width=width, align='edge', alpha=0.3, color='red', label="Mirrored data")
plt.legend()
```

As can be seen, the mirrored data creates peaks which distorts our KDE. At the same time, it improves the shape in our region of interest.

### Conclusion

KDE suffers inherently from a boundary bias because the function *inside* our region of interest is affected by the kernels -- or the lack thereof -- *outside* of our region of interest.

- If the PDF (=approximate sample size density) goes to zero inside our region of interest, we're not affected.
- Otherwise having a data sample that is larger than our region of interest is the best approach. `obs` of a PDF can be different from the `data` argument.
- If no data is available beyond the borders, an ad-hoc method is to mirror the data on the boundary, resulting in an (approximate) zero gradient condition, preventing the PDF from going towards zero too fast. This however changes the shape of the KDE outside of our region of interest. However, if the density does not go to zero at the boundary, the region beyond the boundary should usually anyway not be used.

# Large sample KDE

The exact KDE works well for smaller (~hundreds to thousands) of points, but becomes increasingly computationally demanding and is unfeasible for larger datasets. This computational demand can be reduced by a reduced number of sample points that can be achieved by binning the sample first.

Other approaches bin the data directly and then use it do estimate the density.

## Grid KDE

This implementation is almost the same as the exakt KDE expept that it bins the data first. So any argument that the `KDE1DimExact` takes and effect that applies to it are also valid for the `KDE1DimExact`.

Plotting the two, they lay pretty precisely on top of each and we don't really loose by binning the PDF. This has two main reasons, the number of grid points and the binning method.

```python
sample = true_pdf.sample(1000)
sample_np = zfit.run(sample.value())

kde_exact = zfit.pdf.KDE1DimExact(sample)
kde = zfit.pdf.KDE1DimGrid(sample)
plt.plot(x, kde_exact.pdf(x), ':', label='Exact KDE')
plt.plot(x, kde.pdf(x), '--', label='Grid KDE', alpha=0.5)
plt.plot(x, true_pdf.pdf(x), label='True PDF')
plt.legend()
```

### Number of grid points

By default, the `num_grid_points` is set to a comparably large numbe around 256-1024. This creates a very fine binning of the space and renders the information loss due to binning nearly neglectable.

This number can be adjusted, mainly to reduce the computational cost which is directly proportional to the number of grid points.

```python
for num_grid_points in [5, 10, 20, 100, 1024]:
    kde = zfit.pdf.KDE1DimGrid(sample, num_grid_points=num_grid_points)

    plt.plot(x_medium, kde.pdf(x_medium), '--', label=f'n={num_grid_points}')
#     plt.plot(x, true_pdf.pdf(x), label='True PDF')
plt.plot(x_medium, kde_exact.pdf(x_medium), label='Exact KDE', alpha=0.3, linewidth=7)
plt.legend()
```

For a small number of bins, the approximation breaks down, but for a larger number, it is as good as the exact KDE.

### Binning method

Binning is the process of filling data points into bins or buckets. Simple binning that is used in classical histograms means that each bin receives plus one count for each event (or plus the event weight in case of weights) falling within the bin edges while the neighbouring bins do not get any count. This can lead to fluctuating bins, as an event close to the bin edges actually lies "in the middle of two bins".

To reduce this fluctuation, zfit implements additionally linear binning. This method assigns a fraction of the whole weight to both grid points (bins) on either side, proportional to the closeness of grid point and data point, e.g. the distance between grid points (bin width).

```python
plt.bar([-1], height=0.17, width=2, align='edge', alpha=0.3, label='zoomed in')
x_tiny = np.linspace(-1, 1, 200)
for title, xplot in [('',x_medium), ('Zoomed in', x_tiny)]:
    plt.title(title)
    for binning_method in ['simple', 'linear']:
        for num_grid_points in [32, 256]:

            kde = zfit.pdf.KDE1DimGrid(sample, num_grid_points=num_grid_points, binning_method=binning_method)

            plt.plot(xplot, kde.pdf(xplot), '--', label=f'binning={binning_method}, n={num_grid_points}, ')
    #     plt.plot(x, true_pdf.pdf(x), label='True PDF')
    plt.plot(xplot, kde_exact.pdf(xplot), label='Exact KDE', alpha=0.3, linewidth=7)
    plt.legend()
    plt.figure()

```

Linear binning is smoother than the simple binning and follows closer the exact PDF, although the effect is rather small in our example with the standard number of grid points.
For less grid points, the effect becomes significant and the improvement of linear binning allows an accurate KDE with a minimal number of grid points.

### Data size

The Grid KDE is able to handle large amounts of data with a similar evaluation time.

```python
def plot_large(kdetype, npower=6):
    sample_sizes = 10 ** np.linspace(1, npower, npower)
    for n in sample_sizes:
        n = int(n)
        for i, xplot in enumerate([x_medium, x_tiny]):
            plt.figure(i)
            sample = true_pdf.sample(n)
            kde = kdetype(sample)

            plt.plot(xplot, kde.pdf(xplot), label=f'data={n}')
            if n == sample_sizes[-1]:
                plt.title(f'Data size comparison {kde.name}')
                plt.plot(xplot, true_pdf.pdf(xplot), 'b', label='True PDF', alpha=0.3, linewidth=5)
                plt.legend()
                if i == 0:
                    plt.bar([-1], height=0.2, width=2, align='edge', alpha=0.3, label='zoomed in')
                else:
                    plt.gca().set_ylim((0.145, 0.18))
```

```python
plot_large(zfit.pdf.KDE1DimGrid)
```

## FFT KDE

Another technique to speed up the computation is to rewrite
the kernel density estimation as convolution operation between the kernel function and the grid counts (bin counts). Since this method relies on the discrete Fourier transform, the result is an interpolation.

By using the fact that a convolution is just a multiplication in Fourier space and only evaluating the KDE at grid points one can reduce the computational complexity down to $\mathcal{O}(\log{N} \cdot N)$.

```python
plot_large(zfit.pdf.KDE1DimFFT)
```

## Improved Sheather-Jones Algorithm

A different take on KDEs is the improved Sheather-Jones (ISJ) algorithm. It is an adaptive kernel density estimator based on linear diffusion processes, where the optimality is treated as solving the heat equation. This algorith also includes an estimation for the optimal bandwidth of a normal KDE (and can be used in the exact and grid KDE as `"isj"`).

This yields discrete points, so the result is an interpolation.

```python
plot_large(zfit.pdf.KDE1DimISJ)
```

The ISJ algorithm is rather wiggly for smaller datasets and tends to overfit while delivering great performance with larger datasets.

## KDE comparison for large data

```python
sample_size = 300_000
kde_grid = zfit.pdf.KDE1DimGrid
kde_fft = zfit.pdf.KDE1DimFFT
kde_isj = zfit.pdf.KDE1DimISJ
for i, xplot in enumerate([x_medium,x_tiny/5 + 3]):
    plt.figure(i)
    sample = true_pdf.sample(sample_size)
    for kdetype, linestyle in [
        (kde_grid, '-.'),
                    (kde_fft, ':'),
                    (kde_isj, '--')
                   ]:
        kde = kdetype(sample, num_grid_points=1024)
        plt.plot(xplot, kde.pdf(xplot),linestyle, label=f'{kde.name}')

    plt.title(f'Large data KDE comparison')
    plt.plot(xplot, true_pdf.pdf(xplot), 'b', label='True PDF', alpha=0.3, linewidth=5)
    plt.legend()
    if i == 0:
        width = 2 / 5
        plt.bar([3], height=0.185, width=width, alpha=0.3, label='zoomed in')
```

As we can see, the PDFs all perform nearly equally well for large datasets.

```python

```

_website/tutorials/components/13 - Kernel Density Estimation.ipynb


# Excercises Electric Machinery Fundamentals
## Chapter 6

## Animation: Determining the rotor-slip with  a compass

```python
from sympy import init_session
init_session()
%matplotlib inline
```

    IPython console for SymPy 1.0 (Python 3.5.2-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at http://docs.sympy.org/1.0/

Look at a superposition of two sinusoidal signals:

```python
f=sin(x)*sin(y)
f
```

This product can be rewritten as a sum using trigonometric equalities. SymPy has a special function for those:

```python
from sympy.simplify.fu import *
g=TR8(f)     # TR8 is a trigonometric expression function from Fu paper
Eq(f, g)
```

Now let us take an example with two different angular frequencies:

```python
s = 0.03        # slip
fs = 50         # stator frequency in Hz
fr = (1-s)*fs   # rotor frequency in Hz
fr
```

```python
alpha=2*pi*fs*t
beta=2*pi*fr*t
```

Now plot the product of both frequencies:

```python
# Create the plot of the stator rotation frequency in blue:
p1=plot(sin(alpha), (t, 0, 1), show=False, line_color='b', adaptive=False, nb_of_points=5000)
# Create the plot of the rotor rotation frequency in green:
p2=plot(0.5*sin(beta), (t, 0, 1), show=False, line_color='g', adaptive=False, nb_of_points=5000)
# Create the plot of the combined flux in red:
p3=plot(0.5*f.subs([(x, alpha), (y, beta)]), (t, 0, 1),
            show=False, line_color='r', adaptive=False, nb_of_points=5000)
# Make the second and third one a part of the first one.
p1.extend(p2)
p1.extend(p3)
# Display the modified plot object.
p1.show()
```


IBM Quantum Challenge Africa 2021

Introduction and the Crop-Yield Problem

Quantum Computing has the potential to revolutionize computing, as it can solve problems that are not possible to solve on a classical computer. This extra ability that quantum computers have is called quantum advantage. To achieve this goal, the world needs passionate and competent end-users: those who know how to apply the technology to their field.

In this challenge you will be exposed, at a high-level, to quantum computing through Qiskit. As current or future users of quantum computers, you need to know what problems are appropriate for quantum computation, how to structure the problem model/inputs so that they are compatible with your chosen algorithm, and how to execute a given algorithm and quantum solution to solve the problem.

This is the first notebook for the IBM Quantum Challenge Africa. Before starting here, ensure you have completed the Week 0 content in preparation for the following exercises.

## Initialization

To ensure the demonstrations and exercises have the required Python modules and libraries, run the following cell before continuing.

```python
# Import auxiliary libraries
import numpy as np

# Import Qiskit
from qiskit import IBMQ, Aer
from qiskit.algorithms import QAOA, VQE, NumPyMinimumEigensolver
from qiskit.algorithms.optimizers import COBYLA
from qiskit.utils import QuantumInstance, algorithm_globals
from qiskit.providers.aer.noise.noise_model import NoiseModel

from qiskit_optimization import QuadraticProgram
from qiskit_optimization.algorithms import MinimumEigenOptimizer
from qiskit_optimization.converters import QuadraticProgramToQubo

import qiskit.test.mock as Fake
```

## Table of Contents

The notebook is structured as follows:

1. Initialization
2. Table of Contents
3. Qiskit and its parts
4. Setting up the Qiskit Environment
5. Quadratic Problems
6. Crop-Yield Problem as a Quadratic Problem
7. Solving the Crop-Yield Problem using Quantum Computing
8. Simulating a Real Quantum Computer for the Crop-Yield Problem

## Qiskit and its parts

Qiskit is divided into multiple modules for different purposes, with Terra at the core of the Qiskit ecosystem. It helps to be familiar with what each module can do and whether you will need to utilize the software contained within them for your specific problem. There are four modules that deal with the application of quantum computing; with tools and algorithms built specifically for their fields.

Have a quick look at the [Qiskit documentation](https://qiskit.org/overview) and this IBM Research [blog post](https://www.ibm.com/blogs/research/2021/04/qiskit-application-modules/) to see what these modules are called. Once you have done so, you can complete the first exercise of the challenge: replace the ficticious module names, in the python cell below, with the correct module names. Though it would be fun to have a Qiskit Gaming module, it has not yet been developed. If you are interested in contributing to the open-source Qiskit community, have a look at the [contribution guide](https://qiskit.org/documentation/contributing_to_qiskit.html).

### Exercise 1a: Qiskit Applications Modules

```python
# Definitely real Qiskit module names
qiskit_module_names = [
    "Qiskit Nature",
    "Qiskit Optimization",
    "Qiskit Finance",
    "Qiskit Machine Learning",
]
```

Run the following python cell to check if you have the correct module names.

```python
from qc_grader import grade_ex1a

grade_ex1a(qiskit_module_names)
```

    Submitting your answer for ex1/partA. Please wait...
    Congratulations 🎉! Your answer is correct and has been submitted.

### Qiskit Applications Modules

In this notebook, we will use the `Qiskit Optimization` applications module for a specific problem to illustrate how

1. classical and mathematical problem definitions are represented in Qiskit,
2. some quantum algorithms are defined,
3. to execute a quantum algorithm for a given problem definition,
4. and algorithms and Qiskit problems are executed on real and simulated quantum computers.

We will also be using Qiskit Terra and Aer as they provide the foundation of Qiskit and high-performance quantum simulators respectively.

## Quadratic Problems

Some computational problems can be formulated into quadratic equations such that the minimum of the quadratic equations is the optimal solution, if any exist. These problems are encountered in finance, agriculture, operations and production management, and economics.

Quadratic programming is also used to identify an optimal financial portfolio with minimum risk and optimizing the layout of production components in a factory to minimize the travel distance of resources. This notebook focuses on agriculture as it is a relevant application of quantum computing to problems facing the African continent. However, all of these applications share two common characteristics: the system can be modelled as a quadratic equation and the system variables may be constrained, with their values limited to within a given range.

---

Quadratic problems take on the following structure. Given a vector of $n$ variables $x\in\mathbb{R}^n$, the quadratic function to minimize is as follows.

$$
\begin{align}
\text{minimize}\quad & f\left(x\right)=\frac{1}{2}x^\top{}\mathbf{Q}x + c^\top{}x &\\
\text{subject to}\quad & \mathbf{A}x\leq{}b&\\
& x^\top{}\mathbf{Q}_ix + c_{i}^\top{}x\leq{}r_i,\quad&\forall{}i\in[1,k_q]\\
& l_i\leq{}x_i\leq{}u_i,\quad&\forall{}i\in[1,k_l]\\
\end{align}
$$

$\mathbf{Q}$, $\mathbf{Q}_i$, and $\mathbf{A}$ are $n\times{}n$ symmetric matrices. $c$ and $c_i$ are $n\times{}1$ column vectors. $\mathbf{Q}_i$, $\mathbf{A}$, $c_i$, $l_i$, and $u_i$ define constraints on the variables in $x$. The quadratic equation at the core of the quadratic problem is found by multiplying out the matrices in the minimization function. Though '$\leq{}$' is used in the constraint equations above, any identity relationship may be used for any number of constraints: i.e. "$<$", "$=$", "$>$", "$\geq$", or "$\leq$".

A valid solution to the quadratic must satisfy all conditions for the problem. Examples of some constraints are given below. The first two are linear constraints whereas the third example is a quadratic constraint.

$$ x_1 + x_4 \leq{} 10$$

$$ x_2 - 3x_6 = 10$$

$$x_1x_2 - 4x_3x_4 + x_5 \leq{} 15 $$

Qiskit has Python code that allows you to implement a quadratic problem as a `QuadraticProgram` instance. Though our definition above used matrices to define the coefficients, `QuadraticProgram` allows you to define the objective (function overwhich to minimize) directly. To illustrate how to use `QuadraticProgram`, we will use the following quadratic problem definition, with three integer variables.

$$\begin{align}
\text{minimize}\quad{} & f(x)=(x_1)^2 + (x_2)^2 - x_1x_2 - 6x_3 \\
\text{subject to}\quad{} & x_1 + x_2 = 2             \\
                         & x_2x_3 \geq{} 1           \\
                         & -2 \leq{} x_2 \leq{} 2    \\
                         & -2 \leq{} x_3 \leq{} 4    \\
\end{align}$$

The figure below shows the constraints on $x_1$ and $x_3$, with some simplifcation. The shaded area denotes valid values for $x_1$ and $x_3$, within which $f(x)$ must be minimized.

In the following code, the above quadratic problem is defined as a `QuadraticProgram` instance. Have a look at the [Qiskit documentation for `QuadraticProgram`](https://qiskit.org/documentation/stubs/qiskit.optimization.QuadraticProgram.html), as it can be very useful in helping your understanding of its interface.

The quadratic to minimize, called an objective, is implemented using dictionaries. This allows you, the developer, to explicitly define coefficients for specific variables and terms. The keys in the dictionaries are the variable names identifying a term in $f(x)$. For example, `("x_1","x_2")` is for $x_1x_2$. The values for each item are the coefficients for said terms. Terms that are subtracted in $f(x)$ must have a negative coefficient.

```python
quadprog = QuadraticProgram(name="example 1")
quadprog.integer_var(name="x_1", lowerbound=0, upperbound=4)
quadprog.integer_var(name="x_2", lowerbound=-2, upperbound=2)
quadprog.integer_var(name="x_3", lowerbound=-2, upperbound=4)
quadprog.minimize(
    linear={"x_3": -6},
    quadratic={("x_1", "x_1"): 1, ("x_2", "x_2"): 1, ("x_1", "x_2"): -1},
)
quadprog.linear_constraint(linear={"x_1": 1, "x_2": 1}, sense="=", rhs=2)
quadprog.quadratic_constraint(quadratic={("x_2", "x_3"): 1}, sense=">=", rhs=1)
```

    <qiskit_optimization.problems.quadratic_constraint.QuadraticConstraint at 0x7f2d2409d4f0>

A `QuadraticProgram` can have three types of variables: binary, integer, and continuous. The Qiskit implementation of the algorithms we are going to use currently only support binary and integer variables. There are other algorithms that allow for the simulation of continuous variables, but they are not covered in this notebook. If you want to know more about them though, have a look at this Qiskit tutorial on [algorithms to solve mixed-variable quadratic problems](https://qiskit.org/documentation/tutorials/optimization/5_admm_optimizer.html).

We can visualize the `QuadraticProgram` as an LP string, a portable text-based format used representating the model as a **L**inear **P**rogramming problem.

```python
print(quadprog.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: example 1

    Minimize
     obj: - 6 x_3 + [ 2 x_1^2 - 2 x_1*x_2 + 2 x_2^2 ]/2
    Subject To
     c0: x_1 + x_2 = 2
     q0: [ x_2*x_3 ] >= 1

    Bounds
           x_1 <= 4
     -2 <= x_2 <= 2
     -2 <= x_3 <= 4

    Generals
     x_1 x_2 x_3
    End

Any optimization problem that can be represented as a single second-order equation, in that the greatest exponent of any term is 2, can be transformed into a quadratic problem or program of the form given above. The above example is arbitrary and does not necessarily represent a given real-world problem. The main problem of this notebook focuses on optimizing the yield of a farm, though only the problem definition need be changed to apply this technique to other quadratic problem applications.

## Crop-Yield Problem as a Quadratic Problem

To show how to solve your quadratic program using a quantum computer, we will use two algorithms to solve the crop-yield problem. It is a common need to optimize the crops and management of a farm to reduce risk while increasing profits. One of the big challenges facing Africa and the whole world is how to produce enough food for everyone. The problem here focuses not on profits but on the tonnage of crops harvested. Imagine you have a farm with three hectares of land suitable for farming. You need to choose which crops to plant from a selection of four. Furthermore, you also need to determine how many hectares of each you should plant. The four crops you can plant are wheat, soybeans, maize, and a push-pull crop. The fourth cannot be sold once harvested but it can help increase the yield of the other crops.

            Our beautiful three hectare farm

            Wheat

            Soybeans

            Maize

            Push-Pull

There are three types of farming methods we can use: monocropping, intercropping, and push-pull farming. These are shown below. Monocropping is where only one crop is farmed. This is can make the farm susceptible to disease and pests as the entire yield would be affected. In some instances, growing two different plants nearby each other will increase the yield of both, though sometimes it can decrease the yield. Intercropping is the process where different plants are chosen to _increase_ the yield. Push-Pull crops are pairs of plants that repel pests and attract pests respectively. Integrating them into a larger farm increases the yield of harvested food but with the cost of not necessarily being able to use the harvest of Push-Pull crops as part of the total yield. This is because the Push-Pull crop may not be usable or even edible.

            Monocropping

            Intercropping

            Push-Pull farming

---
Only in certain cases can quadratic programming problems be solved easily using classical problems. In their general sense, they are NP-Hard; a class of problems that is difficult to solve using classical computational methods. In fact, the best classical method to solve these problems involves heuristics, a technique that finds an approximate solution. Quantum Computers have been shown to provide significant speed-up and better scaling for some heuristic problems. The crop-yield problem is a combinatorial problem, in that the solution is a specific combination of input parameters. Though the problem shown here is small enough to solve classically, larger problems become intractable on a classical computer owing to the number of combinations of which to optimize.

Solving the above problem using quantum computing involves three components:

1. Defining the problem
2. Defining the algorithm
3. Executing the algorithm on a backend

Many problems in Qiskit follow this structure as the algorithm you use can typically be swapped for another without significantly redefining your problem. Execution on different backends is the easiest, as long as the device has sufficient resource. The first component is given below, with the second and third in sections 1.5 and 1.6.

### Define the Crop-Yield problem

The following problem is defined for you but the `QuadraticProgram` is not implemented. Your task at the end of this section is to implement the `QuadraticProgram` for the given crop-yield model.

Your farm has three hectares available, $3~ha$, with each crop taking up $0~ha$ or $1~ha$. We define the yield of the farm as a quadratic function where the influence of each crop on eachother is represented by the quadratic coefficients. The variables in this quadratic are the number of hectares of the crop to be planted and the objective function to maximize is the yield of usable crops in tons. Here is the mathematical model for the problem. In this scenario, all crops increase the yield of other crops. However, the problem to solve is which crops to use to achieve the maximum yield.

The farm yield, in tons, is modelled as a quadratic equation, given below, with constraints on the hectares used by each crop and the total hectares available. Each crop is shown using a different symbol, as shown above, representing the number of hectares to be planted of said plant. Note that we can only plant up to 1 hectare of each crop and that our farm is constrained to 3 hectares.

----
#### Non-graphical notation
Here is a non-graphical representation of the above model, if you are struggling to interpret the above graphic.

$$
\begin{align}
    \text{maximize} \quad & 2(\operatorname{Wheat}) + \operatorname{Soybeans} + 4(\operatorname{Maize}) \\
    & + 2.4(\operatorname{Wheat}\times\operatorname{Soybeans}) \\ & + 4(\operatorname{Wheat}\times\operatorname{Maize})\\
    &+ 4(\operatorname{Wheat}\times\operatorname{PushPull}) \\ & + 2(\operatorname{Soybeans}\times\operatorname{Maize}) \\
                          & + (\operatorname{Soybeans}\times\operatorname{PushPull}) \\ & + 5(\operatorname{Maize}\times\operatorname{PushPull})
\end{align}
$$

$$
\begin{align}
\text{subject to} \quad & \operatorname{Wheat} + \operatorname{Soybeans} + \operatorname{Maize} + \operatorname{PushPull} \leq{} 3\\
& 0\leq{}\operatorname{Wheat}\leq{}1\\
& 0\leq{}\operatorname{Soybeans}\leq{}1\\
& 0\leq{}\operatorname{Maize}\leq{}1\\
& 0\leq{}\operatorname{PushPull}\leq{}1
\end{align}
$$

### Exercise 1b: Create Quadratic Program from crop-yield variables

Your first exercise is to create a `QuadraticProgram` that represents the above model. Write your implementation in the `cropyield_quadratic_program` function below. Remember to use the example as a guide, and to look at the [QuadraticProgram documentation](https://qiskit.org/documentation/tutorials/optimization/1_quadratic_program.html?highlight=quadraticprogram) and [Qiskit reference](https://qiskit.org/documentation/stubs/qiskit.optimization.QuadraticProgram.html?highlight=quadraticprogram#qiskit.optimization.QuadraticProgram).

**Note:** Ensure your variables are named `Wheat`, `Soybeans`, `Maize,` and `PushPull`. This is necessary for the grader to work.

```python
def cropyield_quadratic_program():
    cropyield = QuadraticProgram(name="Crop Yield")
    ##############################
    # Put your implementation here

    cropyield.integer_var(name="Wheat", lowerbound=0, upperbound=1)
    cropyield.integer_var(name="Soybeans", lowerbound=0, upperbound=1)
    cropyield.integer_var(name="Maize", lowerbound=0, upperbound=1)
    cropyield.integer_var(name="PushPull", lowerbound=0, upperbound=1)
    cropyield.maximize(
        linear={"Wheat": 2, "Soybeans":1, "Maize":4},
        quadratic={("Wheat", "Soybeans"):2.4,
                   ("Wheat", "Maize"):4,
                   ("Wheat", "PushPull"):4,
                   ("Soybeans", "Maize"):2,
                   ("Soybeans", "PushPull"):1,
                   ("Maize", "PushPull"):5}
    )
    cropyield.linear_constraint(linear={"Wheat":1, "Soybeans":1, "Maize":1, "PushPull":1}, sense="<=", rhs=3)
    #cropyield.quadratic_constraint(quadratic={"Wheat":1, "Soybeans":1, "Maize":1, "PushPull":1}, sense="<=", rhs=3)

    #
    ##############################
    return cropyield

cropyield = cropyield_quadratic_program()
print(cropyield.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: Crop Yield

    Maximize
     obj: 2 Wheat + Soybeans + 4 Maize + [ 4.800000000000 Wheat*Soybeans
          + 8 Wheat*Maize + 8 Wheat*PushPull + 4 Soybeans*Maize
          + 2 Soybeans*PushPull + 10 Maize*PushPull ]/2
    Subject To
     c0: Wheat + Soybeans + Maize + PushPull <= 3

    Bounds
           Wheat <= 1
           Soybeans <= 1
           Maize <= 1
           PushPull <= 1

    Generals
     Wheat Soybeans Maize PushPull
    End

Once you feel your implementation is correct, you can grade your solution in the following cell.

```python
# Execute this cell to grade your solution
from qc_grader import grade_ex1b

cropyield = cropyield_quadratic_program()
grade_ex1b(cropyield)
```

    Submitting your answer for ex1/partB. Please wait...
    Congratulations 🎉! Your answer is correct and has been submitted.

### Converting QuadraticPrograms

If we want to estimate how many qubits this quadratic program requires, we can convert it to an Ising Model and print the `num_qubits` parameter. An [ising model](https://qiskit.org/documentation/apidoc/qiskit.optimization.applications.ising.html?highlight=ising) is a special system model type that is well suited for quantum computing. Though we will not be using an ising model explicitly, the algorithms and Qiskit classes we are using do this conversion internally.

```python
# Estimate the number of qubits required
ising_operations, _ = (
    QuadraticProgramToQubo()
    .convert(
        cropyield,
    )
    .to_ising()
)
print(f"Number of qubits required is {ising_operations.num_qubits}")
```

    Number of qubits required is 6

Even though quadratic programs are widely used in Qiskit, the algorithms we are going to use require binary variables. Qiskit provides an automated method for converting our integer variables into binary variables. The binary-only form is called a _Quadratic Unconstrained Binary Optimization_ problem, or `QUBO`. The conversion is done using `QuadraticProgramToQUBO` from the Qiskit optimization module. Every integer variable, and their associated constraints, are transformed into binary variables.

Run the following code to see how the QUBO version of the cropyield problem looks. Notice how the quadratic becomes longer and more variables are added. This is to account for the bits in each variable, including the constraints. When we run our quantum algorithm to solve this QuadraticProgram, it is converted to a QUBO instance within the Qiskit algorith implementation, implicitly.

```python
QuadraticProgramToQubo().convert(cropyield)
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: Crop Yield

    Minimize
     obj: - 160.400000000000 Wheat@0 - 159.400000000000 Soybeans@0
          - 162.400000000000 Maize@0 - 158.400000000000 PushPull@0
          - 158.400000000000 c0@int_slack@0 - 316.800000000000 c0@int_slack@1 + [
          52.800000000000 Wheat@0^2 + 100.800000000000 Wheat@0*Soybeans@0
          + 97.600000000000 Wheat@0*Maize@0 + 97.600000000000 Wheat@0*PushPull@0
          + 105.600000000000 Wheat@0*c0@int_slack@0
          + 211.200000000000 Wheat@0*c0@int_slack@1 + 52.800000000000 Soybeans@0^2
          + 101.600000000000 Soybeans@0*Maize@0
          + 103.600000000000 Soybeans@0*PushPull@0
          + 105.600000000000 Soybeans@0*c0@int_slack@0
          + 211.200000000000 Soybeans@0*c0@int_slack@1 + 52.800000000000 Maize@0^2
          + 95.600000000000 Maize@0*PushPull@0
          + 105.600000000000 Maize@0*c0@int_slack@0
          + 211.200000000000 Maize@0*c0@int_slack@1 + 52.800000000000 PushPull@0^2
          + 105.600000000000 PushPull@0*c0@int_slack@0
          + 211.200000000000 PushPull@0*c0@int_slack@1
          + 52.800000000000 c0@int_slack@0^2
          + 211.200000000000 c0@int_slack@0*c0@int_slack@1
          + 211.200000000000 c0@int_slack@1^2 ]/2 + 237.600000000000
    Subject To

    Bounds
     0 <= Wheat@0 <= 1
     0 <= Soybeans@0 <= 1
     0 <= Maize@0 <= 1
     0 <= PushPull@0 <= 1
     0 <= c0@int_slack@0 <= 1
     0 <= c0@int_slack@1 <= 1

    Binaries
     Wheat@0 Soybeans@0 Maize@0 PushPull@0 c0@int_slack@0 c0@int_slack@1
    End

## Solving the Crop-Yield Problem using a Quantum Computer

There are three ways to _run_ a quantum algorithm using Qiskit:
1. on a simulator locally on your own machine
2. on a simulator hosted in the cloud by IBM
3. on an actual quantum computer accessible through IBM Quantum.

All of these are called backends. In all cases, the _backend_ can easily be swapped for another as long as the simulator or device has appropriate resources (number of qubits etc.). In the code below, we show how to access different backends. We demonstrate this using the local Aer QASM simulator from Qiskit. The Aer QASM simulator models the physical properties of a real quantum computer so you, researchers, and developers can test their quantum computing code and algorithms before running on real devices.

```python
# We will use the Aer provided QASM simulator
backend = Aer.get_backend("qasm_simulator")

# Given we are using a simulator, we will fix the algorithm seed to ensure our results are reproducible
algorithm_globals.random_seed = 271828
```

We would like to compare our quantum solution to that obtained classically. Secondly, we also want to try different algorithms. The following three subsections show how these different methods for solving the Crop-Yield problem are implemented in Qiskit. The two algorithms used are the [_Quantum Approximate Optimization Algorithm_](https://qiskit.org/documentation/stubs/qiskit.algorithms.QAOA.html?highlight=qaoa#qiskit.algorithms.QAOA) `QAOA` and the [_Variational Quantum Eigensolver_](https://qiskit.org/documentation/stubs/qiskit.algorithms.VQE.html?highlight=vqe#qiskit.algorithms.VQE) `VQE`.

Both of these algorithms are hybrid, in that they use a classical _optimizer_ to alter parameters that affect the quantum computation. The VQE algorithm is used to find the lowest eigenvalue of a matrix, which can describe a system to optimize. The QAOA also finds the lowest eigenvalue, but achieves this is a different way to VQE. Both are very popular algorithms, with varying applications and strengths.

### Classical Solution

The classical solution to the crop-yield problem can easily be found using Numpy and Qiskit. The QUBO problem can be solved by finding the minimum eigenvalue of its underlying matrix representation. Fortunately, we don't have to know what this matrix looks like. We only need to pass it to a `MinimumEigensolver` and `MinimumEigenOptimizer`.

The optimizer translates the provided problem into a parameterised representation which is then passed to the solver. By optimizing the paramters, the solver will eventually give the minimum eigenvalue for the parameterized representation and thus the solution to the original problem. Here we use a classical solver from NumPy, the `NumPyMinimumEigensolver`.

```python
def get_classical_solution_for(quadprog: QuadraticProgram):
    # Create solver
    solver = NumPyMinimumEigensolver()

    # Create optimizer for solver
    optimizer = MinimumEigenOptimizer(solver)

    # Return result from optimizer
    return optimizer.solve(quadprog)
```

If we execute the classical method for our crop-yield problem, we get a valid solution that maximises the yield.

```python
# Get classical result
classical_result = get_classical_solution_for(cropyield)

# Format and print result
print("Solution found using the classical method:\n")
print(f"Maximum crop-yield is {classical_result.fval} tons")
print(f"Crops used are: ")

_crops = [v.name for v in cropyield.variables]
for cropIndex, cropHectares in enumerate(classical_result.x):
    print(f"\t{cropHectares} ha of {_crops[cropIndex]}")
```

    Solution found using the classical method:

    Maximum crop-yield is 19.0 tons
    Crops used are:
    	1.0 ha of Wheat
    	0.0 ha of Soybeans
    	1.0 ha of Maize
    	1.0 ha of PushPull

### QAOA Solution

To solve our problem using QAOA, we need only replace the classical_solver with a `QAOA` class instance. Now that we are running a quantum algorithm, we need to tell the solver where to execute the quantum component. We use a `QuantumInstance` to store the backend information. The QAOA is an iterative algorithm, and thus is run multiple times with different internal parameters. The parameters are tuned classically during the optimization step of the computation by `optimizer`. If we leave `optimizer` as `None`, our algorithms will use the default optimization algorithm. To determine how many iterations there are, we define a callback function that runs for each iteration and stores the number of evaluations thus far. At the end of our algorithm execution, we return the result and the number of iterations.

```python
def get_QAOA_solution_for(
    quadprog: QuadraticProgram, quantumInstance: QuantumInstance, optimizer=None,
):
    _eval_count = 0

    def callback(eval_count, parameters, mean, std):
        nonlocal _eval_count
        _eval_count = eval_count

    # Create solver
    solver = QAOA(
        optimizer=optimizer, quantum_instance=quantumInstance, callback=callback,
    )

    # Create optimizer for solver
    optimizer = MinimumEigenOptimizer(solver)

    # Get result from optimizer
    result = optimizer.solve(quadprog)
    return result, _eval_count
```

If we execute the QAOA method for our crop-yield problem, we get the same result as the classical method, showing that 1) the quantum solution is correct and 2) that you now know how to use a quantum algorithm! 🌟

```python
# Create a QuantumInstance
simulator_instance = QuantumInstance(
    backend=backend,
    seed_simulator=algorithm_globals.random_seed,
    seed_transpiler=algorithm_globals.random_seed,
)

# Get QAOA result
qaoa_result, qaoa_eval_count = get_QAOA_solution_for(cropyield, simulator_instance)

# Format and print result
print("Solution found using the QAOA method:\n")
print(f"Maximum crop-yield is {qaoa_result.fval} tons")
print(f"Crops used are: ")
for cropHectares, cropName in zip(qaoa_result.x, qaoa_result.variable_names):
    print(f"\t{cropHectares} ha of {cropName}")

print(f"\nThe solution was found within {qaoa_eval_count} evaluations of QAOA.")
```

    Solution found using the QAOA method:

    Maximum crop-yield is 19.0 tons
    Crops used are:
    	1.0 ha of Wheat
    	0.0 ha of Soybeans
    	1.0 ha of Maize
    	1.0 ha of PushPull

    The solution was found within 3 evaluations of QAOA.

### VQE Solution

The `VQE` algorithm works in a very similar way to the `QAOA`. Not only in a mathematical modelling and algorithmic perspective, but also programmatically. There is a quantum solver and a classical optimizer. The `VQE` instance is also iterative, and so we can measure how many iterations are needed to find a solution to the Crop-Yield problem.

```python
def get_VQE_solution_for(
    quadprog: QuadraticProgram, quantumInstance: QuantumInstance, optimizer=None,
):
    _eval_count = 0

    def callback(eval_count, parameters, mean, std):
        nonlocal _eval_count
        _eval_count = eval_count

    # Create solver and optimizer
    solver = VQE(
        optimizer=optimizer, quantum_instance=quantumInstance, callback=callback
    )

    # Create optimizer for solver
    optimizer = MinimumEigenOptimizer(solver)

    # Get result from optimizer
    result = optimizer.solve(quadprog)
    return result, _eval_count
```

And we should get the exact same answer as before.

```python
# Create a QuantumInstance
simulator_instance = QuantumInstance(
    backend=backend,
    seed_simulator=algorithm_globals.random_seed,
    seed_transpiler=algorithm_globals.random_seed,
)

# Get VQE result
vqe_result, vqe_eval_count = get_VQE_solution_for(cropyield, simulator_instance)

# Format and print result
print("Solution found using the VQE method:\n")
print(f"Maximum crop-yield is {vqe_result.fval} tons")
print(f"Crops used are: ")
for cropHectares, cropName in zip(vqe_result.x, vqe_result.variable_names):
    print(f"\t{cropHectares} ha of {cropName}")

print(f"\nThe solution was found within {vqe_eval_count} evaluations of VQE")
```

    Solution found using the VQE method:

    Maximum crop-yield is 19.0 tons
    Crops used are:
    	1.0 ha of Wheat
    	0.0 ha of Soybeans
    	1.0 ha of Maize
    	1.0 ha of PushPull

    The solution was found within 25 evaluations of VQE

### Exercise 1c: Classical and Quantum Computational Results

From the above computations you received six results, the maximum crop-yield and the number of evaluations for three different methods. The maximum yield values should be the same. If your yield values aren't all the same, rerun the algorithms. Sometimes the optimization process can miss the correct answer because of the randomness used to initialize the algorithm parameters.

Run the code cell below to see if the maximum yields you computed are correct.

```python
from qc_grader import grade_ex1c

max_yield_qaoa = qaoa_result.fval
max_yield_vqe = vqe_result.fval

grade_ex1c(tonnage_qaoa=max_yield_qaoa, tonnage_vqe=max_yield_vqe)
```

    Submitting your answer for ex1/partC. Please wait...
    Congratulations 🎉! Your answer is correct and has been submitted.

_You could always verify your result with the classical method, though this is only possible here given the size of the problem. Larger problems become more difficult to verify._

## Simulating a Real Quantum Computer for the Crop-Yield Problem

Sometimes one would want to _simulate_ a real quantum computer to see how the actual hardware may impact the performance of the algorithm. All quantum computers have an underlying architecture, different noise characeristics, and error rates. These three aspects impact how well the algorithm can perform on a given deivce. To test the impact a given quantum computer has on the QAOA instance, we can utilize a _fake_ instance of the device in Qiskit to tell our simulator what parameters to use. In this example we will be simulating `ibmq_johannesburg`, a device named after the city of Johannesburg in South Africa.

```python
fake_device = Fake.FakeJohannesburg()
```

We can inspect what this device _looks_ like using the Qiskit Jupyter tools, shown below. You do not need to know about this structure to execute quantum programs on a device, but it is useful to visualize the parameters.

```python
import qiskit.tools.jupyter

fake_device
```

    VBox(children=(HTML(value="<h1 style='color:#ffffff;background-color:#000000;padding-top: 1%;padding-bottom: 1…

    <FakeJohannesburg('fake_johannesburg')>

The three aforementioned components of a quantum computer are represented as a noise model, coupling map, and the basis gate set. The noise model is a representation of how the noise and errors in the computer behave. The coupling map and basis gate set are core to the architecture of the device. The coupling map represents how the physical qubits can interact whereas the basis gate set is analogous to the set of fundamental computatonal instructions we can use. You can see the coupling map in the above widget as the lines connecting each qubit in the architecture diagram.

To simulate `ibmq_johannesburg`, we must pass these three components to our Aer simulator.

```python
# Create the noise model, which contains the basis gate set
noise_model = NoiseModel.from_backend(fake_device)

# Get the coupling map
coupling_map = fake_device.configuration().coupling_map
```

Next we create a new `QuantumInstance` with these parameters

```python
fake_instance = QuantumInstance(
    backend=backend,
    basis_gates=noise_model.basis_gates,
    coupling_map=coupling_map,
    noise_model=noise_model,
    seed_simulator=algorithm_globals.random_seed,
    seed_transpiler=algorithm_globals.random_seed,
)
```

We can then execute the `QAOA` from before on this new _fake_ quantum device.

```python
# Get QAOA result
qaoa_result, qaoa_eval_count = get_QAOA_solution_for(cropyield, fake_instance)

# Format and print result
print("Solution found using the QAOA method:\n")
print(f"Maximum crop-yield is {qaoa_result.fval} tons")
print(f"Crops used are: ")
for cropHectares, cropName in zip(qaoa_result.x, qaoa_result.variable_names):
    print(f"\t{cropHectares} ha of {cropName}")

print(f"\nThe solution was found within {qaoa_eval_count} evaluations of QAOA.")
```

    Solution found using the QAOA method:

    Maximum crop-yield is 19.0 tons
    Crops used are:
    	1.0 ha of Wheat
    	0.0 ha of Soybeans
    	1.0 ha of Maize
    	1.0 ha of PushPull

    The solution was found within 3 evaluations of QAOA.

### Scaling of the Quantum Solution vs Classical

When we created our quadratic program for the crop-yield problem, we saw that the Ising Model required 6 qubits. We had constrained our problem such that we could only plant up to 1 hectare per crop. However, we could change the model so that we can plot 3 hectares per crop, upto our maximum available farm area of 3 hectares.

How many qubits would this ising model require?

---

Furthermore, what if we had more land to farm? We know that this problem is NP-Hard and thus classical solutions are mostly found using heuristics. This is the core reason why quantum computers are promising to solve these kinds of problems. But what are the resource requirements for the quantum solution, with a larger farm and crops that can be planted in more hectares?

To illustrate this, we've provided a function that returns the number of qubits required by the underlying Ising Model for the Crop-Yield Problem. We then see the estimated number of qubits needed for different problem parameters. Feel free to modify the variables being used to see how the qubit resource requirements change.

```python
# Function to estimate the number of qubits required
def estimate_number_of_qubits_required_for(max_hectares_per_crop, hectares_available):
    return int(
        4 * np.ceil(np.log2(max_hectares_per_crop + 1))
        + np.ceil(np.log2(hectares_available + 1))
    )

# Our new problem parameters
hectares_available = 10
max_hectares_per_crop = 10

# Retrieving the number of qubits required
number_of_qubits_required = estimate_number_of_qubits_required_for(
    max_hectares_per_crop=max_hectares_per_crop, hectares_available=hectares_available
)

print(
    f"Optimizing a {hectares_available} ha farm with each crop taking up to {max_hectares_per_crop} ha each,",
    f"the computation is estimated to require {number_of_qubits_required} qubits.",
)
```

    Optimizing a 10 ha farm with each crop taking up to 10 ha each, the computation is estimated to require 20 qubits.

The number of qubits required is related to the constraints in the quadratic program and how the integer variables are converted to binary variables. In fact, the scaling of the number of qubits, as a function of the hectares available, is logarithmic in nature; owing to this conversion.

## Running on real quantum hardware

To use the IBM Quantum platform is easy. First you need to load the account you enabled in the week 0 content. If you didn't complete this, follow this [quick guide](https://quantum-computing.ibm.com/lab/docs/iql/manage/account/ibmq) on connecting your IBM Quantum account with Qiskit in python and Jupyter.

```python
IBMQ.load_account()
```

    ibmqfactory.load_account:WARNING:2021-09-13 08:47:44,561: Credentials are already in use. The existing account in the session will be replaced.

    <AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>

IBM Quantum backends are accessed through a provider, which manages the devices to which you have access. For this challenge, you have access to the new `ibm_perth` quantum computer! Typically, you would find your provider details under your [IBM Quantum account details](https://quantum-computing.ibm.com/account). Under your account you can see the different hubs, groups, and projects you are a part of. Qiskit allows us to retrieve a provider using just the hub, group, and project as follows:

```python
provider = IBMQ.get_provider(hub="ibm-q", group="open", project="main")
```

However, because we have given you special access for this challenge, we are going to retrive the provider using a different method. Execute the code cell below to retrieve the correct provider.

```python
provider = None
for prov in IBMQ.providers():
    if (
        "iqc-africa-21" in prov.credentials.hub
        and "q-challenge" in prov.credentials.group
        and "ex1" in prov.credentials.project
    ):
        # Correct provider found
        provider = prov

if provider == None:
    print("ERROR: The expected provider was not found!")
else:
    print("Yay! The expected provider was found!")
```

    Yay! The expected provider was found!

If the above code cell returned an error, you may not yet have access to the real quantum computer. The list of participants is updated daily, so you may have to wait some time before the correct provider appears. If you need assistance, send a message to the challenge Slack channel [#challenge-africa-2021](https://qiskit.slack.com/archives/C02C8MKP153) and make sure to tag the admin team with [@africa_admin](#).

------

To retrieve a backend from the provider, one needs only request it by name. For example, we can request `ibm_perth` as follows.

```python
backend_real = provider.get_backend("ibm_perth")
```

We can also list all backends available through a given backend. In this example we use the _open_ provider as it has access to all open devices and simulators, instead of the limited few for the challenge.

```python
for _backend in IBMQ.get_provider(hub='ibm-q', group='open', project='main').backends():
    print(_backend.name())
```

    ibmq_qasm_simulator
    ibmq_armonk
    ibmq_santiago
    ibmq_bogota
    ibmq_lima
    ibmq_belem
    ibmq_quito
    simulator_statevector
    simulator_mps
    simulator_extended_stabilizer
    simulator_stabilizer
    ibmq_manila

Qiskit provides visual tools to view backend information in a jupyter notebook. To accomplish this, one needs to import the `jupyter` submodule and call the appropriate _magic comment_. With `qiskit_backend_overview` you can view all devices accessible by the current IBMQ account. Notice how it does not include simulators. Furthermore, you should see that all devices available through the _open group_ have at most 5 qubits. This is a problem for solving the crop-yield problem we created earlier, as we showed it requires 6 qubits.

To demonstrate how a real quantum device is used, a smaller `QuadraticProgram` is provided, requiring a maximum of 4 qubits.

```python
%qiskit_backend_overview
```

    VBox(children=(HTML(value="<h2 style ='color:#ffffff; background-color:#000000;padding-top: 1%; padding-bottom…

If you want access to larger and more sophisticated quantum computers through IBM, see if your university or company is part of the [IBM Quantum Network](https://www.ibm.com/quantum-computing/network/members/). Researchers are institutions that are part of the [African Research Universities Alliance (ARUA)](https://arua.org.za/) can also apply for access through the University of the Witwatersrand, in South Africa; which is a member of the IBM Quantum Network. If you are a researcher, you can also apply for access through the [IBM Quantum Researchers Program](https://www.ibm.com/quantum-computing/researchers-program/). If you're a student at a highschool or university, you can ask your teachers or lecturers to apply for access through the [IBM Quantum Educators Program](https://www.ibm.com/quantum-computing/educators-program/).

Given that we have imported `qiskit.tools.jupyter`, Jupyter will now display a helpful widget when an IBMQ backend is displayed. We do not have to use a _magic comment_ here as the jupyter submodule defines how some variables are displayed in a jupyter notebook, without requiring a _magic comment_.

```python
backend_real
```

    VBox(children=(HTML(value="<h1 style='color:#ffffff;background-color:#000000;padding-top: 1%;padding-bottom: 1…

    <IBMQBackend('ibm_perth') from IBMQ(hub='iqc-africa-21-8', group='q-challenge-2', project='ex1-recnPEbPGZpUOAAaH')>

We can create a new `QuantumInstance` object to contain our real quantum computer backend, similar to how we created one to manage our simulator. For real devices, there is an extra parameter we can set: `shots`. The output of a quantum computing algorithm is probabilistic. Therefore, we must execute the quantum computation multiple times, sampling the outputs to estimate their probabilities. The number of `shots` is the number of executions of the quantum computation. Here we set out `QuantumInstance` to use the least busy backend with 2048 shots.

```python
quantum_instance_real = QuantumInstance(backend_real, shots=2048)
```

The VQE algorithm and QAOA are iterative, meaning that they incorporate a classical-quantum loop which repeats certain computations, _hopefully_ converging to a valid solution. In each iteration, or evaluation, the quantum backend will execute the quantum operations 2048 times. Each shot is quite fast, so we do not have to worry about a significant increase in processing time by using more shots.

----
We now define our small crop-yield problem, which only requires 4 qubits. In this example, only Wheat and Maize are used. The model is altered to illustrate the impact of growing too much of a single crop, with the yield decreasing as the number of hectares of a single crop is increased. However, utilizing both wheat and maize increases yield, showing the benefits of intercropping.

**NB:** The maximum number of hectares available is 4, but given that the model would never exceed this limit, the linear constraint defining the maximum number of hectares is not included. This reduces the number of qubits required from 6 to 4.

```python
# Create a small crop-yield example quadratic program
cropyield_small = QuadraticProgram(name="Small Crop-Yield")

# Add two variables, indicating whether we grow 0, 1, or 2 hectares for two different crops
cropyield_small.integer_var(lowerbound=0, upperbound=2, name="Wheat")
cropyield_small.integer_var(lowerbound=0, upperbound=2, name="Maize")

# Add the objective function defining the yield in tonnes
cropyield_small.maximize(
    linear={"Wheat": 3, "Maize": 3},
    quadratic={("Maize", "Wheat"): 1, ("Maize", "Maize"): -2, ("Wheat", "Wheat"): -2},
)

# This linear constraint is not used as the model never reaches this. This is because the
# sum of the upperbounds on both variables is 4 already. If this constraint is applied, the
# model would require 6 qubits instead of 4.
# cropyield_small.linear_constraint(linear={"Wheat": 1, "Maize": 1}, sense="<=", rhs=4)

print(cropyield_small)
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: Small Crop-Yield

    Maximize
     obj: 3 Wheat + 3 Maize + [ - 4 Wheat^2 + 2 Wheat*Maize - 4 Maize^2 ]/2
    Subject To

    Bounds
           Wheat <= 2
           Maize <= 2

    Generals
     Wheat Maize
    End

Here we verify that our small crop-yield problem requires only 4 qubits.

```python
# Estimate the number of qubits required
ising_operations_small, _ = (
    QuadraticProgramToQubo()
    .convert(
        cropyield_small,
    )
    .to_ising()
)
print(f"Number of qubits required is {ising_operations_small.num_qubits}")
```

    Number of qubits required is 4

### Exercise 1d: Submitting a job to a real quantum computer

Now that we know the problem can be run on our chosen device, we can execute the VQE algorithm. In this case we will set the optimizer, with a maximum number of iterations of 1, so that we do not occupy the device for too long. Our answer will be incorrect, but we only want to see how to send a quantum program to a real quantum computer.

```python
# Create our optimizer
optimizer = COBYLA(maxiter=1)

## Get result from real device with VQE
vqe_result_real, vqe_eval_count_real = get_VQE_solution_for(
    cropyield_small, quantum_instance_real, optimizer=optimizer
)
```

    /opt/conda/lib/python3.8/site-packages/qiskit/utils/run_circuits.py:695: UserWarning: max_credits is not a recognized runtime option and may be ignored by the backend.
      return backend.run(circuits, **run_kwargs)
    FAILURE: Job id: 613f10ade3f915b111349621 is cancelled. Re-submit the circuits.
    FAILURE: Can not get job id, Resubmit the qobj to get job id.Error: Expecting value: line 1 column 1 (char 0)
    FAILURE: Job id: 613f307217e5886a6fba5ba3 is cancelled. Re-submit the circuits.

Qiskit uses `jobs` to track computations and their results on remote devices and simulators. We can query the backend object for the jobs it received, which would be those created by the VQE algorithm.

```python
# Retrieve the VQE job sent
job_real = backend_real.jobs()[0]

print(f"VQE job created at {job_real.creation_date()} and has a job id of {job_real.job_id()}")
```

Put the job id for your the above job into the cell below and execute the code cell.

```python
from qc_grader import grade_ex1d

job_id = '613f31526feb7e17126f42a2'

grade_ex1d(job_id)
```

    /opt/conda/lib/python3.8/site-packages/pyscf/lib/misc.py:47: H5pyDeprecationWarning: Using default_file_mode other than 'r' is deprecated. Pass the mode to h5py.File() instead.
      h5py.get_config().default_file_mode = 'a'

    Submitting your answer for ex1/partD. Please wait...
    Congratulations 🎉! Your answer is correct and has been submitted.

You have now completed the first lab of the IBM Quantum Challenge Africa 2021! Make sure that you are on the [Qiskit Slack channel](https://ibm.co/Africa_Slack) so you can ask questions and talk to other participants. There are two more labs left in the challenge, which are more difficult than this introductory lab, covering quantum computing for finance and HIV.

## References

[1] A. A. Nel, ‘Crop rotation in the summer rainfall area of South Africa’, South African Journal of Plant and Soil, vol. 22, no. 4, pp. 274–278, Jan. 2005, doi: 10.1080/02571862.2005.10634721.

[2] H. Ritchie and M. Roser, ‘Crop yields’, Our World in Data, 2013, [Online]. Available: https://ourworldindata.org/crop-yields.

[3] G. Brion, ‘Controlling Pests with Plants: The power of intercropping’, UVM Food Feed, Jan. 09, 2014. https://learn.uvm.edu/foodsystemsblog/2014/01/09/controlling-pests-with-plants-the-power-of-intercropping/ (accessed Feb. 15, 2021).

[4] N. O. Ogot, J. O. Pittchar, C. A. O. Midega, and Z. R. Khan, ‘Attributes of push-pull technology in enhancing food and nutrition security’, African Journal of Agriculture and Food Security, vol. 6, pp. 229–242, Mar. 2018.

```python
import qiskit.tools.jupyter

%qiskit_version_table
%qiskit_copyright
```

    /opt/conda/lib/python3.8/site-packages/qiskit/aqua/__init__.py:86: DeprecationWarning: The package qiskit.aqua is deprecated. It was moved/refactored to qiskit-terra For more information see <https://github.com/Qiskit/qiskit-aqua/blob/main/README.md#migration-guide>
      warn_package('aqua', 'qiskit-terra')

Version Information

Qiskit Software	Version

qiskit-terra	0.18.1

qiskit-aer	0.8.2

qiskit-ignis	0.6.0

qiskit-ibmq-provider	0.16.0

qiskit-aqua	0.9.4

qiskit	0.29.0

qiskit-nature	0.1.5

qiskit-finance	0.2.0

qiskit-optimization	0.2.1

qiskit-machine-learning	0.2.0

System information

Python	3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05)
[GCC 9.3.0]

OS	Linux

CPUs	8

Memory (Gb)	31.400043487548828

Mon Sep 13 15:04:27 2021 UTC

This code is a part of Qiskit

© Copyright IBM 2017, 2021.

This code is licensed under the Apache License, Version 2.0. You may
obtain a copy of this license in the LICENSE.txt file in the root directory
 of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
Any modifications or derivative works of this code must retain this
copyright notice, and modified files need to carry a notice indicating
that they have been altered from the originals.

```python

```


```python
%matplotlib notebook

import math
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

import seaborn as sns
sns.set()
```

# Finite-Difference Formula

## Taylor Series

The derivative of the function $f(x)$ with respect to $x$ is

\begin{equation}
\frac{ \textrm{d} f}{\textrm{d} x} = \lim_{\Delta x \to 0} \frac{ f(x + \Delta x) - f(x) }{\Delta x}.
\end{equation}

What if the function $f(x)$ is not continuous? For numerical calculations, $f(x)$ is approximated as a discrete function $f(x_i)$ where $x_i$ is only defined on *nodes* within a *mesh*. We can't take the limit of a function where $\Delta x$ does not go to zero. Instead we use the idea of a series expansion. Though our understanding of limits refer to functions of real variables, we can adapt them to sequences. If the limit as $x$ goes to infinity of $f(x) = L$ and $f(x) = a_n$ when $n$ is an integer, then the limit as $n$ goes to infinity of $a_n = L$. Can we use a series expansion to *represent* our discrete function? Yes!

A Taylor series is a polynomial series expansion of function about the point $x_0$. We write the Taylor series as

$$
f(x) = \sum_{n=0}^{\infty} \frac{f^{(n)}(x_0) }{n!} (x - x_0)^n
$$

where the $n$-th order partial derivative is

$$
f^{(n)} = \frac{\partial^n f}{\partial x^n}
$$

Consider the exponential function, $f(x) = e^x$.

```python
def exp_function(x, x0, N):
    """
    Taylor series appoximation of the exponential function.

    Args:
       x (array-like) - Independent variable
       x0 (float) - Point of exapnsion
       N (int) - Number of terms to include in the Taylor series.
    """
    f = 0
    dfdx = 1.0
    for n in range(N):
        f += dfdx / math.factorial(n) * (x - x0)**n
    return f
```

```python
x = np.linspace(0.0, 10.0, 100)
f = np.exp(x)

# Approximate the function using a Taylor series with the following number of terms
N = [1,2,4,5,6,7,8,9,10,15,20,25,30,35,40,45,50,55,60]

# Compute the L2 Normalization of the residual, i.e., the truncation error
error = []
for n in N:
    p_n = exp_function(x, 0.0, n)
    error.append(np.sum((p_n - f)**2))

fig = plt.figure()
ax = fig.add_subplot(111)
line = ax.plot(N, error, lw=1, c='k', marker='o', ms=2)

ax.xaxis.set_major_formatter(matplotlib.ticker.FormatStrFormatter('%i'))
plt.yscale('log')
ax.set_xlim((1,60))
ax.set_xlabel('Number of Terms')
ax.set_ylabel('L2 Normalization of Truncation Error')
```

    <IPython.core.display.Javascript object>

    Text(0, 0.5, 'L2 Normalization of Truncation Error')

## Taylor's Theorem and Finite-difference Formulae

Consider the function, $f(x)$, to be discretized on a uniform mesh, where $\Delta x$ is the mesh spacing and is constant. If the $x$-position of the node indices of the mesh are $x_i$ and $x_{i+1}$, then $\Delta x = x_{i+1} - x_i$. Consider a Taylor series expansion approximation of the function $f$ at $x_{i+1}$ expanded about the point $x_i$,

$$
f(x_i + \Delta x) = f(x_{i+1}) = \sum_{n=0}^{\infty} \frac{f^{(n)}(x_{i}) }{n!} (x_{i+1} - x_i)^n
$$

Recognizing that what we mean by $f(x_i)$ is just the value of the function at the mesh point $i$, which can be more accurately and succintly stated as $f_i$, and substituting $x_{i+1} - x_{i}$ for $\Delta x$,

$$
f_{i+1} = \sum_{n=0}^{\infty} \frac{f_i^{(n)} }{n!} (\Delta x)^n =  f_i + \Delta x \left[ \frac{\textrm{d} f}{\textrm{d} x} \right]_i + \frac{(\Delta x)^2}{2} \left[ \frac{\textrm{d}^2 f}{\textrm{d} x^2}\right]_i + \frac{(\Delta x)^3}{3!} \left[\frac{\textrm{d}^3 f}{\textrm{d} x^3}\right]_i + \dots
$$

where the derivatives are evaluated at the $i^{\textrm{th}}$ mesh point.

In practice, we *truncate* the series. [Taylor's theorem](https://en.wikipedia.org/wiki/Taylor%27s_theorem) provides a mathematical basis for evaulating the trunction error. Let $P_k(x)$ be the Taylor series expansion of the function $f(x)$ using $k$ terms. The remainder would then be defined as

$$
R_k(x) = f(x) - P_k(x)
$$

Consider a $k=2$ Taylor series of the discrete function $f(x_i)$,

$$
f_{i+1} =  f_i + \Delta x \left[ \frac{\textrm{d} f}{\textrm{d} x} \right]_i + R_2(x_{i}).
$$

Rearranging, the derivative of the function at $x_i$ is

$$
 \frac{\textrm{d} f}{\textrm{d} x} \bigg\rvert_i = \frac{f_{i+1} - f_i}{\Delta x} + \hat{R}_2(x_{i})
$$

Truncation error is therefore,
$$
\hat{R}_2(x_{i}) = \frac{R_2(x_{i})}{\Delta x} = \frac{1}{\Delta x} \left[\frac{(\Delta x)^2}{2} \left[ \frac{\textrm{d}^2 f}{\textrm{d} x^2}\right]_i + \frac{(\Delta x)^3}{3!} \left[\frac{\textrm{d}^3 f}{\textrm{d} x^3}\right]_i + \dots  \right]
$$

As $\Delta x$ changes (reduces), how does the truncation error $\hat{R}_2(x_{i})$ change (reduce)? What is the leading order? Using Big-$\mathcal{O}$ notation,

$$
\hat{R}_2(x_{i}) = \mathcal{O}(\Delta x), \,\, \textrm{as} \,\, \Delta x \to 0
$$

Using this terminology, the derivative of function can be approximated using a *first-order*, finite-difference

$$
\frac{\delta f}{\delta x} \bigg\rvert_i = \delta_x f_i = \frac{f_{i+1} - f_i}{\Delta x} + \mathcal{O}(\Delta x)
$$

where we have introduced the notation, $\delta_x$, to represent the *approximation* of the first derivative of the function, $f$ at the mesh point $x_i$. Furthermore, we can define a *forward*, *first-order*, finite-difference as

$$
\delta_x^+ f_i = \frac{f_{i+1} - f_i}{\Delta x} + \mathcal{O}(\Delta x)
$$

and a *backward*, *first-order*, finite-difference as

$$
\delta_x^- f_i = \frac{f_{i} - f_{i-1}}{\Delta x} + \mathcal{O}(\Delta x)
$$

where the superscript $+$ and $-$ give the direction of the finite-difference approximation. Notice we are describing these finite-difference formulae in terms of their *direction* (forward or backward) and *order* (first, second, third, etc.). Forward and backward difference formula involve different nodes. We refer to the nodes used by a finite-difference approximation as its *stencil*.

### Key Terminology

**Order** - How the truncation error for a finite-difference approximation changes with respect to the mesh size, $\Delta x$.

**Stencil** - What mesh points / nodes are used for a finite-difference approximation

## Derivations of finite-difference formulae

Consider the function $\phi(x,t)$, using the Taylor series expressions above, we can derive finite-difference approximations for the first-order, spatial derivatives. With some tedius algebraic manipulation, we can derive the forward, backward, and central difference approximations.

### Forward differencing

Rearranging the terms of

$$
\phi_{i+1} = \phi_i + \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

we get

$$
\phi_{i+1} - \phi_i = \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i  + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

Dividing by $\Delta x$,

$$
\frac{\phi_{i+1} - \phi_i} {\Delta x} = \frac{\partial \phi}{\partial x} \bigg\rvert_i  + \frac{\Delta x}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^2}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

and writing this with big-O notation,

$$
\frac{\phi_{i+1} - \phi_i}{ \Delta x} = \frac{\partial \phi}{\partial x} \bigg\rvert_i + \mathcal{O}(\Delta x)
$$

With some rearrangement we get the first-order accurate, forward difference

$$
\frac{\partial \phi}{\partial x} \bigg\rvert_i = \frac{\phi_{i+1} - \phi_i}{ \Delta x} + \mathcal{O}(\Delta x)
$$

### Backward differencing

Following a similar process, we can derive an expression for the backward difference approximation. Starting with

$$
\phi_{i-1} = \phi_i - \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i - \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i - \dots
$$

and then rearranging,

$$
\frac{\phi_{i} - \phi_{i-1}}{ \Delta x} = \frac{\partial \phi}{\partial x} \bigg\rvert_i - \frac{\Delta x}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^2}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i - \dots
$$

Using big-O notation, we can solve for the backward difference approximation of the first spatial derivative

$$
\frac{\partial \phi}{\partial x} \bigg\rvert_i = \frac{\phi_i - \phi_{i-1}}{ \Delta x} + \mathcal{O}(\Delta x)
$$

### Central differencing

Combining the forward and backward differences, we can derive a central difference formula. Using the a Taylor series for $i+1$

$$
\phi_{i+1} = \phi_i + \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

and one for $i-1$

$$
\phi_{i-1} = \phi_i - \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i - \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

we can combine the two equations to solve for the first-derviative. Using the backward difference approximation, solve for $\phi_i$,

$$
\phi_i = \phi_{i-1} + \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i - \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i - \dots
$$

and then substitue that expressions into the forward difference approximation

$$
\phi_{i+1} = \phi_{i-1} + \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i - \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i + \frac{(\Delta x)^2}{2} \frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_i + \frac{(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

Combing and canceling terms

$$
\phi_{i+1} = \phi_{i-1} + 2 \Delta x \frac{\partial \phi}{\partial x} \bigg\rvert_i  + \frac{2(\Delta x)^3}{3!} \frac{\partial^3 \phi}{\partial x^3}\bigg\rvert_i + \dots
$$

Dividing by $2 \Delta x$,

$$
\frac{\partial \phi}{\partial x} \bigg\rvert_i = \frac{\phi_{i+1} - \phi_{i-1}}{ 2\Delta x} + \mathcal{O}(\Delta x^2)
$$

## Finite-difference formulae of higher-order

Compute the coefficinets of a third-order, difference formula that uses the points $i-2$, $i-1$, $i$ , and $i+1$

```python
A = np.array([[1, 1, 1, 1], [-2, -1, 0, 1], [4, 1, 0, 1], [-8, -1, 0, 1]])
# print(A.shape)
# print(A)
b = np.array([0, 1, 0, 0])
x = np.linalg.solve(A,b)
print(x)
```

    [ 0.16666667 -1.          0.5         0.33333333]

## Observations

- The order of accuracy indicates how fast the truncation error diminishes as the mesh size, $\Delta x$ is reduced. The order does not say anything about the magnitude of the truncation error, only about its relative rate of convergence.

- The order of accuracy for a finite-difference approximation is defines the local truncation error, i.e., the truncation error of the difference approximation at a single point in the mesh. It is not a statement of the global order of convergence for the finite-difference scheme.

- There are many reasons why one may not want to use very high-order finite-difference approximations, but one reason is that as the order increases so do the number mesh points in the stencil.

- By inspection of the Taylor series we see that the magnitude of the truncation error is a not only a function of $\Delta x$, but also a function of the magnitude of the derivative. This informs us that we should use a smaller $\Delta x$ in regions where derivatives of the function are large.

## Pade Formula

### Lagrange Interpolation

Use Lagrange interpolation to approximation the function, $f(x) = x^3$ on the interval, $-1 \le x \le 1$. Use three control points, $x_0 = -0.5$, $x_1 = 0.0$, and $x_2 = 0.5$.

```python
N = 100
xmin = -1.0
xmax = 1.0
x = np.linspace(xmin, xmax, N)

# Define the control points
x0 = -0.5
x1 = 0.0
x2 = 0.5
x3 = 0.76
```

```python
# Evaluate the control points
u0 = x0**3
u1 = x1**3
u2 = x2**3
u3 =
```

```python
# Define the coefficients
a0 = (x - x1)*(x - x2) / (x0 - x1) / (x0 - x2)
a1 = (x - x0)*(x - x2) / (x1 - x0) / (x1 - x2)
a2 = (x - x0)*(x - x1) / (x2 - x0) / (x2 - x1)
```

```python
f = x**3
Lf = u0 * a0 + u1 * a1 + u2 * a2
```

```python
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(9, 6))

line = ax.plot(x, f, lw=2, color='b', label=r'$f(x)$')
line = ax.plot(x, Lf, lw=2, color='k', label=r'$L(x)$')

ax.set_title("Lagrangian Interpolation", fontsize=15)
ax.legend(loc=4, fontsize=15)
ax.set_xlabel(r'$x$', fontsize=15.0)
ax.set_ylabel(r'$y$', fontsize=15.0)
```

    <IPython.core.display.Javascript object>

    Text(0, 0.5, '$y$')

### Deriving coefficients for  Pade formula

Compute the coefficients for a Pade formula to approximate the first-order derivative?

```python
A = np.array([[-1, -1, -1, 0, 0], [1, 0, -1, 1, 1], [-1, 0, -1, -2, 2], [1, 0, -1, 3, 3], [-1, 0, -1, -4, 4]])
b = np.array([0, -1, 0, 0, 0])
x = np.linalg.solve(A,b)
print(x)
```

```python
A = np.array([[-1, -1, 0], [1, 0, -1], [-0.5, 0, -1]])
# print(A.shape)
# print(A)
b = np.array([0, 0, -1])
x = np.linalg.solve(A,b)
print(x)
```

    [ 0.66666667 -0.66666667  0.66666667]

```python

```


# Lecture 03 Multiplication and Inverse Matrices

Today's lecture includes:
1. Matrix Multiplication (4 ways)
2. Inverse of $A, AB, A^{T}$
3. Gauss-Jordan / Find $A^{-1}$

## Matrix Multiplication
### 1.1 First method

We first take a look on this matrix:
\begin{align}
\begin{matrix}-&-&-\\-&-&-&\end{matrix} × \begin{matrix}-&-\\-&-\\-&-\end{matrix} = \begin{matrix}-&-\\-&-\end{matrix}
\end{align}
* 行列内积：有$m\times n$矩阵$A$和$n\times p$矩阵$B$（$A$的总行数必须与$B$的总列数相等），两矩阵相乘有$AB=C$，$C$是一个$m\times p$矩阵，对于$C$矩阵中的第$i$行第$j$列元素$c_{ij}$，有：

    $$c_{ij}=row_i\cdot column_j=\sum_{k=i}^na_{ik}b_{kj}$$

    其中$a_{ik}$是$A$矩阵的第$i$行第$k$列元素，$b_{kj}$是$B$矩阵的第$k$行第$j$列元素。

    可以看出$c_{ij}$其实是$A$矩阵第$i$行点乘$B$矩阵第$j$列 $\begin{bmatrix}&\vdots&\\&row_i&\\&\vdots&\end{bmatrix}\begin{bmatrix}&&\\\cdots&column_j&\cdots\\&&\end{bmatrix}=\begin{bmatrix}&\vdots&\\\cdots&c_{ij}&\cdots\\&\vdots&\end{bmatrix}$

```python

```

Lecture 03 Multiplication and Inverse Matrices.ipynb


# Euler's integration

We are going to look at how Euler's integration performs at the integration of the 2nd order ODE governing the evolution of a planet on a circular orbit.
This problem can be simplified as two first order ODEs:

$$\frac{\mathrm{d} x}{\mathrm{d} t}	= u $$
$$\frac{\mathrm{d} u}{\mathrm{d} t}	= - \frac{\mu}{(x^2+y^2)^{3/2}} x,$$

where $x$ and $u$ are the position and velocity along the x-coordinates, while $y$ and $v$ are the position and velocity along the y-coordinates.

And Euler's integration for this system yields, using a discrete notation

$$x_{n+1} = x_n + u_n\Delta t $$
$$u_{n+1} = u_n - \frac{\mu}{(x_n^2+y_n^2)^{3/2}} x_n \Delta t.$$

Similarly,

$$y_{n+1} = y_n + v_n\Delta t $$
$$v_{n+1} = v_n - \frac{\mu}{(x_n^2+y_n^2)^{3/2}} y_n \Delta t.$$

```python
import numpy as np
import matplotlib.pyplot as plt
```

Let us define the function of the exact solution (which we know !)

Here $\omega = n = \mu/a^3$, where $a$ is the semi-major axis

```python
# Exact solution for the x coordinate of the position
def position_x(a, t, omega):
    return a * np.cos(omega * t)
def position_y(a, t, omega):
    return a * np.sin(omega * t)
# Exact solution for the x coordinate of the velocity
def velocity_x(a, t, omega):
    return - a * omega * np.sin(omega * t)
def velocity_y(a, t, omega):
    return a * omega * np.cos(omega * t)
```

Let us define the 4 derivatives

```python
# Velocities for the 2 components
def dx_dt(x, y, u, v):
    return u
def dy_dt(x, y, u, v):
    return v

# Accelerations for the 2 components
def du_dt(x, y, u, v, mu):
    omega_squared = mu/np.power(x*x + y*y, 3./2.)
    return - omega_squared * x
def dv_dt(x, y, u, v, mu):
    omega_squared = mu/np.power(x*x + y*y, 3./2.)
    return - omega_squared * y
```

Now let us define an evolver function, which will perform the Euler integration. We want to keep in memory the solution x(t), v(t) and the time t to later plot the result and compare it to the exact solution

```python
def evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps):
    xe = np.zeros((N_steps + 1))
    ue = np.zeros((N_steps + 1))
    ye = np.zeros((N_steps + 1))
    ve = np.zeros((N_steps + 1))
    te = np.zeros((N_steps + 1))

    xe[0], ye[0] = x0, y0
    ue[0], ve[0], te[0] = u0, v0, 0

    print("now is time to work")
    for i in range(N_steps):
        xe[i+1] = xe[i] + (dx_dt(xe[i], ye[i], ue[i], ve[i])*dt)
        ye[i+1] = ye[i] + (dy_dt(xe[i], ye[i], ue[i], ve[i])*dt)
        ue[i+1] = ue[i] + (du_dt(xe[i], ye[i], ue[i], ve[i], mu)*dt)
        ve[i+1] = ve[i] + (dv_dt(xe[i], ye[i], ue[i], ve[i], mu)*dt)
        te[i+1] = te[i] + dt
    return te, xe, ye, ue, ve
```

Now let's define the initial conditions.
And compute both an array with the exact solution and 2 arrays with the integration results.

```python
Time_max = 10

# Initial conditions for the integration
mu = 1.0
x0, y0 = 1.0, 0.0
u0, v0 = 0.0, 1.0
dt = 0.001

N_steps = int(Time_max/float(dt))

# Computation of the Euler's integration
te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

# Computation of the exact solution
omega = np.sqrt(mu/np.power(x0*x0+y0*y0, 3./2.))

exact_solution_x  = [position_x(x0, tt, omega) for tt in te]
exact_solution_y  = [position_y(x0, tt, omega) for tt in te]
exact_solution_u  = [velocity_x(x0, tt, omega) for tt in te]
exact_solution_v  = [velocity_y(x0, tt, omega) for tt in te]

```

    now is time to work

Now let's plot both the exact solution and the integration result

```python
fig = plt.figure(figsize=(16, 16))
ax = fig.add_subplot(2,1,1)

line, = ax.plot(te, xe, '--', label="x, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_x, '-', c=line.get_color(), label="x exact")
line, = ax.plot(te, ye, '--', label="y, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_y, '-', c=line.get_color(), label="y exact")

ax.set_ylim([-1.5,1.5])
ax.set_ylabel("x, y")
ax.legend(loc=0, prop={'size':10})

ax = fig.add_subplot(2,1,2, sharex=ax)

line, = ax.plot(te, ue, '--', label="u, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_u, '-', c=line.get_color(), label="vx exact")
line, = ax.plot(te, ve, '--', label="v, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_v, '-', c=line.get_color(), label="vy exact")

ax.set_ylim([-1.5,1.5])
ax.set_ylabel("u, v")
ax.set_xlabel("Time")
ax.legend(loc=0, prop={'size':10})
```

Let's evolve it a bit more

```python
Time_max = 100
N_steps = int(Time_max/float(dt))

# Computation of the Euler's integration
te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

# Computation of the exact solution
exact_solution_x  = [position_x(x0, tt, omega) for tt in te]
exact_solution_y  = [position_y(x0, tt, omega) for tt in te]
exact_solution_u  = [velocity_x(x0, tt, omega) for tt in te]
exact_solution_v  = [velocity_y(x0, tt, omega) for tt in te]
```

    now is time to work

And plot

```python
fig = plt.figure(figsize=(16, 16))
ax = fig.add_subplot(2,1,1)

line, = ax.plot(te, xe, '--', label="x, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_x, '-', c=line.get_color(), label="x exact")
line, = ax.plot(te, ye, '--', label="y, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_y, '-', c=line.get_color(), label="y exact")

ax.set_ylim([-1.5,1.5])
ax.set_ylabel("x, y")
ax.legend(loc=0, prop={'size':10})

ax = fig.add_subplot(2,1,2, sharex=ax)

line, = ax.plot(te, ue, '--', label="u, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_u, '-', c=line.get_color(), label="vx exact")
line, = ax.plot(te, ve, '--', label="v, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_v, '-', c=line.get_color(), label="vy exact")

ax.set_ylim([-1.5,1.5])
ax.set_ylabel("u, v")
ax.set_xlabel("Time")
ax.legend(loc=0, prop={'size':10})
```

This looks  bad... Let's integrate it even more

```python
Time_max = 1000
N_steps = int(Time_max/float(dt))

# Computation of the Euler's integration
te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

# Computation of the exact solution
exact_solution_x  = [position_x(x0, tt, omega) for tt in te]
exact_solution_y  = [position_y(x0, tt, omega) for tt in te]
exact_solution_u  = [velocity_x(x0, tt, omega) for tt in te]
exact_solution_v  = [velocity_y(x0, tt, omega) for tt in te]
```

    now is time to work

```python
fig = plt.figure(figsize=(16, 16))
ax = fig.add_subplot(2,1,1)

line, = ax.plot(te, xe, '-o', label="x, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_x, '-', c=line.get_color(), label="x exact")
line, = ax.plot(te, ye, '-o', label="y, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_y, '-', c=line.get_color(), label="y exact")

ax.set_ylabel("x, y")
ax.legend(loc=0, prop={'size':10})

ax = fig.add_subplot(2,1,2, sharex=ax)

line, = ax.plot(te, ue, '-o', label="u, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_u, '-', c=line.get_color(), label="vx exact")
line, = ax.plot(te, ve, '-o', label="v, Euler, dt = %.2f" %dt)
ax.plot(te, exact_solution_v, '-', c=line.get_color(), label="vy exact")

ax.set_ylabel("u, v")
ax.set_xlabel("Time")
ax.legend(loc=0, prop={'size':10})
```

Let's investigate the effect of the timestep

```python
Time_max = 500

fig = plt.figure(figsize=(12, 8))
ax = fig.add_subplot(1,1,1)

for dt in [0.2, 0.1, 0.001]:
    N_steps = int(Time_max/float(dt))
    # Computation of the Euler's integration
    te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)
    #te, xe, ve = evolve_Euler(x0 , v0, omega, dt, int(Time_max/float(dt)))
    ax.plot(te, xe, label="dt={}".format(dt))

ax.set_ylim([-15,7.5])
ax.set_ylabel("x")
ax.set_xlabel("Time")
ax.legend(loc=0, prop={'size':10})
```

## Phase space

To visualize this a bit differently, let us plot the phase space: $u = f(x)$ and $v = f(y)$

```python
Time_max = 10
dt = 0.01
N_steps = int(Time_max/float(dt))
te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(1,1,1)

print("now is time to work: plot phase space u vs x")
# plot calculated phase space
ax.plot(xe, ue, '--', label="Euler, dt={}".format(dt))
# plot analytical phase space
ax.plot(position_x(a=x0, t=te, omega=omega), velocity_x(a=x0, t=te, omega=omega), label="Exact solution")

ax.set_ylabel("u")
ax.set_xlabel("x")
ax.legend(loc=0, prop={'size':10})
ax.set_ylim([-1.5,1.5])
ax.set_xlim([-1.5,1.5])
plt.show()

fig = plt.figure(figsize=(8, 8))
ax1 = fig.add_subplot(1,1,1)

print("now is time to work: plot phase space v vs y")
# plot calculated phase space
ax1.plot(ye, ve, '--', label="Euler, dt={}".format(dt))
# plot analytical phase space
ax1.plot(position_y(a=x0, t=te, omega=omega), velocity_y(a=x0, t=te, omega=omega), label="Exact solution")

ax1.set_ylabel("v")
ax1.set_xlabel("y")
ax1.legend(loc=0, prop={'size':10})
ax1.set_ylim([-1.5,1.5])
ax1.set_xlim([-1.5,1.5])

```

There is something clearly very wrong.

## Energy of the system: is it conserved?

The energy of the system is given by
\begin{equation}
E = E_{\rm grav} + E_{\rm kin} = - \frac{\mu}{r} + \frac{1}{2} V^2 %= -\frac{C}{r}
\end{equation}
where $r = \sqrt{x^2+y^2}$ and $V^2 = u^2 + v^2$.

We will compute $\Delta E = E(t) - E(t=0)$. If there is conservation of the energy, this should be zero.

```python
fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(1,1,1)

Time_max = 1000

for dt in [0.1, 0.01, 0.001]:
    N_steps = int(Time_max/float(dt))
    # Computation of the Euler's integration
    te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

    print("now is time to work")
    # Compute the energy compared to the Energy at time 0 (called energy_0)
    delta_energy = np.zeros((N_steps+1))
    energy_0 = - mu/((xe[0]**2 + ye[0]**2)**0.5) + (ue[0]**2 + ve[0]**2)/2
    for i in range(N_steps+1):
        delta_energy[i] = - mu/((xe[i]**2 + ye[i]**2)**0.5) + (ue[i]**2 + ve[i]**2)/2 - energy_0

    ax.plot(te, delta_energy, label="dt={}".format(dt))
    ax.set_ylabel("$\Delta$ Energy")
    ax.set_xlabel("Time")
    ax.set_yscale('log')
    ax.set_xscale('log')
    ax.legend(loc=0, prop={'size':10})
```

The energy is not conserved...

## Angular momentum: is it conserved?

Another quantity which is supposed to be conserved is the total angular momentum.

The angular momentum in the heliocentric frame is given by
\begin{equation}
\mathbf{h} = \frac{m_0 m_1}{m_0 + m_1}\mathbf{r} \times \mathbf{v}
\end{equation}

For our example, the angular momentum (per unit mass) is given by
\begin{equation}
\begin{split}
\mathbf{h} &= \mathbf{r} \times \mathbf{v}\\
h_z &= x v - y u
\end{split}
\end{equation}

Let us see if this quantity is conserved. To do that let us compute $\Delta h_z = h_z(t) - h_z(t=0)$.

```python
fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(1,1,1)

Time_max = 1000

for dt in [0.1, 0.01, 0.001]:
    N_steps = int(Time_max/float(dt))
    # Computation of the Euler's integration
    te, xe, ye, ue, ve = evolve_Euler(x0, y0, u0, v0,  mu, dt, N_steps)

    print("now is time to work")
    # Compute the angular momentum compared to the angular momentum at time 0 (called angular_momentum_0)
    delta_angular_momentum = np.zeros((N_steps+1))
    angular_momentum_0 = (xe[0]*ve[0]) - (ye[0]*ue[0])
    for i in range(N_steps+1):
        delta_angular_momentum[i] = (xe[i]*ve[i]) - (ye[i]*ue[i]) - angular_momentum_0

    ax.plot(te, delta_angular_momentum, label="dt={}".format(dt))
    ax.set_ylabel("$\Delta$ angular momentum")
    ax.set_xlabel("Time")
    ax.set_yscale('log')
    ax.set_xscale('log')
    ax.legend(loc=0, prop={'size':10})
```

The angular momentum is not conserved either...