# Solving quantum mechanical problems with Machine Learning and Boltzmann machines

**Morten Hjorth-Jensen**, Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University and Department of Physics, University of Oslo, Norway

Date: **Jul 26, 2018**

Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license

## What is this talk about?
The main aim is to give you a short and pedestrian introduction to  how we can use Machine Learning methods
to solve quantum mechanical many-body problems. And why this could be of interest.

The hope is that after this talk you have gotten the basic ideas to get you started. Peeping into <https://github.com/mhjensenseminars/MachineLearningTalk>, you'll find a Jupyter notebook, slides, codes etc that will allow you to reproduce the simulations discussed here, and perhaps run your own very first calculations.

Furthermore, since we will also talk about Monte Carlo simulations, the exciting idea of Monte Carlo optimization and Machine Learning can be used to mesh strategies for reducing the curse of dimensionality.

## Why?

How can we avoid the dimensionality curse? Many possibilities
1. smarter basis functions

2. resummation of specific correlations

3. stochastic sampling of high-lying states (stochastic FCI, CC and soon SRG)

4. many more

Machine Learning and Quantum Computing hold also great promise in tackling the
ever increasing dimensionalities. Here we will focus on Machine Learning.

## Overview
* Short intro to Machine Learning

* Variational Monte Carlo (Markov Chain Monte Carlo, $\mathrm{MC}^2$) and two-electron quantum dots, solving quantum mechanical problems in a stochastic way. It  will serve as  our motivation for switching to Machine Learning.

* From Variational Monte Carlo to Boltzmann Machines and Machine Learning

## What are the Machine Learning calculations here based on?

This work is inspired by the idea of representing the wave function with
a restricted Boltzmann machine (RBM), presented recently by [G. Carleo and M. Troyer, Science **355**, Issue 6325, pp. 602-606 (2017)](http://science.sciencemag.org/content/355/6325/602). They
named such a wave function/network a *neural network quantum state* (NQS). In their article they apply it to the quantum mechanical
spin lattice systems of the Ising model and Heisenberg model, with
encouraging results.

Thanks to Vilde Flugsrud and Alfred Alocias Mariadason for many discussions and interpretations of the above paper.

## A new world
Machine learning (ML) is an extremely rich field, in spite of its young age. The
increases we have seen during the last three decades in computational
capabilities have been followed by developments of methods and
techniques for analyzing and handling large date sets, relying heavily
on statistics, computer science and mathematics.  The field is rather
new and developing rapidly.

Popular software packages written in Python for ML are

* [Scikit-learn](http://scikit-learn.org/stable/),

* [Tensorflow](https://www.tensorflow.org/),

* [PyTorch](http://pytorch.org/) and

* [Keras](https://keras.io/).

These are all freely available at their respective GitHub sites. They
encompass communities of developers in the thousands or more. And the number
of code developers and contributors keeps increasing.

## Lots of room for creativity
Not all the
algorithms and methods can be given a rigorous mathematical
justification, opening up thereby for experimenting
and trial and error and thereby exciting new developments.

A solid command of linear algebra, multivariate theory,
probability theory, statistical data analysis,
understanding errors and Monte Carlo methods is important in order to understand many of the
various algorithms and methods.

****Job market, a personal statement**.**

A familiarity with ML is almost becoming a prerequisite for many of the most exciting employment
opportunities. Add quantum computing and there you are.

## Some members of the ML family

1. Linear regression and its variants, in essence polynomial regression

2. Decision tree algorithms, from simpler to more complex ones like random forests

3. Nearest neighbors models

4. Bayesian statistics and regression

5. Support vector machines and finally various variants of

6. Artifical neural networks and deep learning

7. ....

## What are the basic ingredients?
Almost every problem in ML and data science starts with the same ingredients:
* The dataset $\mathbf{x}$ (could be some observable quantity of the system we are studying)

* A model which is a function of a set of parameters $\mathbf{\alpha}$ that relates to the dataset, say a likelihood  function $p(\mathbf{x}\vert \mathbf{\alpha})$ or just a simple model $f(\mathbf{\alpha})$

* A so-called **cost** function $\mathcal{C} (\mathbf{x}, f(\mathbf{\alpha}))$ which allows us to decide how well our model represents the dataset.

We seek to minimize the function $\mathcal{C} (\mathbf{x}, f(\mathbf{\alpha}))$ by finding the parameter values which minimize $\mathcal{C}$. This leads to  various minimization algorithms.

## What is Machine Learning?
Machine learning is the science of giving computers the ability to
learn without being explicitly programmed.  The idea is that there
exist generic algorithms which can be used to find patterns in a broad
class of data sets without having to write code specifically for each
problem. The algorithm will build its own logic based on the data.

Machine learning is a subfield of computer science, and is closely
related to computational statistics.  It evolved from the study of
pattern recognition in artificial intelligence (AI) research, and has
made contributions to AI tasks like computer vision, natural language
processing and speech recognition. It has also, especially in later
years, found applications in a wide variety of other areas, including
bioinformatics, economy, physics, finance and marketing.

You will notice however that many of the basic ideas discussed do come from Physics!

## Types of Machine Learning

The approaches to machine learning are many, but are often split into two main categories.
In *supervised learning* we know the answer to a problem,
and let the computer deduce the logic behind it. On the other hand, *unsupervised learning*
is a method for finding patterns and relationship in data sets without any prior knowledge of the system.
Some authours also operate with a third category, namely *reinforcement learning*. This is a paradigm
of learning inspired by behavioural psychology, where learning is achieved by trial-and-error,
solely from rewards and punishment.

Another way to categorize machine learning tasks is to consider the desired output of a system.
Some of the most common tasks are:

  * Classification: Outputs are divided into two or more classes. The goal is to   produce a model that assigns inputs into one of these classes. An example is to identify  digits based on pictures of hand-written ones. Classification is typically supervised learning.

  * Regression: Finding a functional relationship between an input data set and a reference data set.   The goal is to construct a function that maps input data to continuous output values.

  * Clustering: Data are divided into groups with certain common traits, without knowing the different groups beforehand.  It is thus a form of unsupervised learning.

## References
* An excellent reference, [Mehta *et al.*, arXiv:1803.08823 and Physics Reports in press (2018)](https://arxiv.org/abs/1803.08823)

* A cute paper by [Utama and Piekarewicz, Validating neural-network refinements of nuclear mass models, Phys. Rev. C 97, 014306 ](https://journals.aps.org/prc/abstract/10.1103/PhysRevC.97.014306)

* [Every issue of Physical Review Letters has now one or more articles on ML](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.120.156001)

* [Books and lectures notes](https://github.com/CompPhysics/MachineLearning)

* See also Metha and Schwab, arXiv.1410.3831, how to link Variational renormalization group theory with deep learning (recommended read)

## Another interesting article

Here we will use so-called **reduced Boltzmann Machines** to simulate quantum many-body problems. For Monte Carlo aficionados, there is a very close similarity with what are called **shadow wave functions**, see the work of [Pederiva and Kalos and collaborators, Phys Rev. E 90, 053304 (2014)](https://journals.aps.org/pre/abstract/10.1103/PhysRevE.90.053304).

## Just for fun illustration of scikit-learn: Hudson bay data
Most mammalian predators rely on a variety of prey, which complicates mathematical modeling; however, a few predators have become highly specialized and seek almost exclusively a single prey species. An example of this simplified predator-prey interaction is seen in Canadian northern forests, where the populations of the lynx and the snowshoe hare are intertwined in a life and death struggle.

One reason that this particular system has been so extensively studied is that the Hudson Bay company kept careful records of all furs from the early 1800s into the 1900s. The records for the furs collected by the Hudson Bay company showed distinct oscillations (approximately 12 year periods), suggesting that these species caused almost periodic fluctuations of each other's populations. The table here shows data from 1900 to 1920.

## Hares and lynx in Hudson bay from 1900 to 1920

## Linear regression in Python

```
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
from IPython.display import display
import sklearn
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor

data = np.loadtxt('src/Hudson_Bay.csv', delimiter=',', skiprows=1)
x = data[:,0]
y = data[:,1]
line = np.linspace(1900,1920,1000,endpoint=False).reshape(-1,1)
reg = DecisionTreeRegressor(min_samples_split=3).fit(x.reshape(-1,1),y.reshape(-1,1))
plt.plot(line, reg.predict(line), label="decision tree")
regline = LinearRegression().fit(x.reshape(-1,1),y.reshape(-1,1))
plt.plot(line, regline.predict(line), label= "Linear Regression")
plt.plot(x, y, label= "Linear Regression")
plt.show()
```

## Artificial neurons

The field of artificial neural networks has a long history of
development, and is closely connected with the advancement of computer
science and computers in general. A model of artificial neurons was
first developed by McCulloch and Pitts in 1943 to study signal
processing in the brain and has later been refined by others. The
general idea is to mimic neural networks in the human brain, which is
composed of billions of neurons that communicate with each other by
sending electrical signals.  Each neuron accumulates its incoming
signals, which must exceed an activation threshold to yield an
output. If the threshold is not overcome, the neuron remains inactive,
i.e. has zero output.

This behaviour has inspired a simple mathematical model for an artificial neuron.

$$
y = f\left(\sum_{i=1}^n w_ix_i\right) = f(u)
$$

Here, the output $y$ of the neuron is the value of its activation function, which have as input
a weighted sum of signals $x_i, \dots ,x_n$ received by $n$ other neurons.

## A simple perceptron model

## Neural network types
An artificial neural network (NN), is a computational model that consists of layers of connected neurons, or *nodes*.
It is supposed to mimic a biological nervous system by letting each neuron interact with other neurons
by sending signals in the form of mathematical functions between layers.
A wide variety of different NNs have
been developed, but most of them consist of an input layer, an output layer and eventual layers in-between, called
*hidden layers*. All layers can contain an arbitrary number of nodes, and each connection between two nodes
is associated with a weight variable.

## The system: two electrons in a harmonic oscillator trap in two dimensions

The Hamiltonian of the quantum dot is given by

$$
\hat{H} = \hat{H}_0 + \hat{V},
$$

where $\hat{H}_0$ is the many-body HO Hamiltonian, and $\hat{V}$ is the
inter-electron Coulomb interactions. In dimensionless units,

$$
\hat{V}= \sum_{i<j}^N \frac{1}{r_{ij}},
$$

with $r_{ij}=\sqrt{\mathbf{r}_i^2 - \mathbf{r}_j^2}$.

This leads to the  separable Hamiltonian, with the relative motion part given by ($r_{ij}=r$)

$$
\hat{H}_r=-\nabla^2_r + \frac{1}{4}\omega^2r^2+ \frac{1}{r},
$$

plus a standard Harmonic Oscillator problem  for the center-of-mass motion.
This system has analytical solutions in two and three dimensions ([M. Taut 1993 and 1994](https://journals.aps.org/pra/abstract/10.1103/PhysRevA.48.3561)).

## Quantum Monte Carlo Motivation
Given a hamiltonian $H$ and a trial wave function $\Psi_T$, the variational principle states that the expectation value of $\langle H \rangle$, defined through

$$
\langle E \rangle =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})H(\boldsymbol{R})\Psi_T(\boldsymbol{R})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})\Psi_T(\boldsymbol{R})},
$$

is an upper bound to the ground state energy $E_0$ of the hamiltonian $H$, that is

$$
E_0 \le \langle H \rangle .
$$

In general, the integrals involved in the calculation of various  expectation values  are multi-dimensional ones. Traditional integration methods such as the Gauss-Legendre will not be adequate for say the  computation of the energy of a many-body system.

## Quantum Monte Carlo Motivation
**Basic steps.**

Choose a trial wave function
$\psi_T(\boldsymbol{R})$.

$$
P(\boldsymbol{R},\boldsymbol{\alpha})= \frac{\left|\psi_T(\boldsymbol{R},\boldsymbol{\alpha})\right|^2}{\int \left|\psi_T(\boldsymbol{R},\boldsymbol{\alpha})\right|^2d\boldsymbol{R}}.
$$

This is our model, or likelihood/probability distribution function  (PDF). It depends on some variational parameters $\boldsymbol{\alpha}$.
The approximation to the expectation value of the Hamiltonian is now

$$
\langle E[\boldsymbol{\alpha}] \rangle =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R},\boldsymbol{\alpha})H(\boldsymbol{R})\Psi_T(\boldsymbol{R},\boldsymbol{\alpha})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R},\boldsymbol{\alpha})\Psi_T(\boldsymbol{R},\boldsymbol{\alpha})}.
$$

## Quantum Monte Carlo Motivation
**Define a new quantity.**

$$
E_L(\boldsymbol{R},\boldsymbol{\alpha})=\frac{1}{\psi_T(\boldsymbol{R},\boldsymbol{\alpha})}H\psi_T(\boldsymbol{R},\boldsymbol{\alpha}),
$$

called the local energy, which, together with our trial PDF yields

$$
E[\boldsymbol{\alpha}]=\int P(\boldsymbol{R})E_L(\boldsymbol{R},\boldsymbol{\alpha}) d\boldsymbol{R}\approx \frac{1}{N}\sum_{i=1}^NE_L(\boldsymbol{R_i},\boldsymbol{\alpha})
$$

with $N$ being the number of Monte Carlo samples.

## Quantum Monte Carlo
The Algorithm for performing a variational Monte Carlo calculations runs thus as this

   * Initialisation: Fix the number of Monte Carlo steps. Choose an initial $\boldsymbol{R}$ and variational parameters $\alpha$ and calculate $\left|\psi_T(\boldsymbol{R},\boldsymbol{\alpha})\right|^2$.

   * Initialise the energy and the variance and start the Monte Carlo calculation by looping over trials.

      * Calculate  a trial position  $\boldsymbol{R}_p=\boldsymbol{R}+r*step$ where $r$ is a random variable $r \in [0,1]$.

      * Metropolis algorithm to accept or reject this move  $w = P(\boldsymbol{R}_p,\boldsymbol{\alpha})/P(\boldsymbol{R},\boldsymbol{\alpha})$.

      * If the step is accepted, then we set $\boldsymbol{R}=\boldsymbol{R}_p$.

      * Update averages

   * Finish and compute final averages.

Observe that the jumping in space is governed by the variable *step*. This is often called brute-force sampling.
Need importance sampling to get more relevant sampling.

## The trial wave function
We want to perform  a Variational Monte Carlo calculation of the ground state of two electrons in a quantum dot well with different oscillator energies, assuming total spin $S=0$.
Our trial wave function has the following form

$$
\begin{equation}
   \psi_{T}(\boldsymbol{r}_1,\boldsymbol{r}_2) =
   C\exp{\left(-\alpha_1\omega(r_1^2+r_2^2)/2\right)}
   \exp{\left(\frac{r_{12}}{(1+\alpha_2 r_{12})}\right)},
\label{eq:trial} \tag{1}
\end{equation}
$$

where the $\alpha$s represent our variational parameters, two in this case.

Why does the trial function look like this? How did we get there? **This will be our main motivation** for switching to
Machine Learning.

## The correlation part of the wave function

To find an ansatz for the correlated part of the wave function, it is useful to rewrite the two-particle
local energy in terms of the relative and center-of-mass motion.
Let us denote the distance between the two electrons as
$r_{12}$. We omit the center-of-mass motion since we are only interested in the case when
$r_{12} \rightarrow 0$. The contribution from the center-of-mass (CoM) variable ${\bf R}_{\mathrm{CoM}}$
gives only a finite contribution.
We focus only on the terms that are relevant for $r_{12}$ and for three dimensions. The relevant local energy becomes then

$$
\lim_{r_{12} \rightarrow 0}E_L(R)=
    \frac{1}{{\cal R}_T(r_{12})}\left(2\frac{d^2}{dr_{ij}^2}+\frac{4}{r_{ij}}\frac{d}{dr_{ij}}+
\frac{2}{r_{ij}}-\frac{l(l+1)}{r_{ij}^2}+2E
\right){\cal R}_T(r_{12}) = 0.
$$

Set $l=0$ and we have the so-called **cusp** condition

$$
\frac{d {\cal R}_T(r_{12})}{dr_{12}} = -\frac{1}{2(l+1)}
{\cal R}_T(r_{12})\qquad r_{12}\to 0
$$

## Resulting ansatz
The above  results in

$$
{\cal R}_T  \propto \exp{(r_{ij}/2)},
$$

for anti-parallel spins and

$$
{\cal R}_T  \propto \exp{(r_{ij}/4)},
$$

for anti-parallel spins.
This is the so-called cusp condition for the relative motion, resulting in a minimal requirement
for the correlation part of the wave fuction.
For general systems containing more than say two electrons, we have this
condition for each electron pair $ij$.

## The VMC code

```
# Importing various packages
from math import exp, sqrt
from random import random, seed
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import sys

#Trial wave function for quantum dots in two dims
def WaveFunction(r,alpha,beta):
    r1 = r[0,0]**2 + r[0,1]**2
    r2 = r[1,0]**2 + r[1,1]**2
    r12 = sqrt((r[0,0]-r[1,0])**2 + (r[0,1]-r[1,1])**2)
    deno = r12/(1+beta*r12)
    return exp(-0.5*alpha*(r1+r2)+deno)

#Local energy  for quantum dots in two dims, using analytical local energy
def LocalEnergy(r,alpha,beta):

    r1 = (r[0,0]**2 + r[0,1]**2)
    r2 = (r[1,0]**2 + r[1,1]**2)
    r12 = sqrt((r[0,0]-r[1,0])**2 + (r[0,1]-r[1,1])**2)
    deno = 1.0/(1+beta*r12)
    deno2 = deno*deno
    return 0.5*(1-alpha*alpha)*(r1 + r2) +2.0*alpha + 1.0/r12+deno2*(alpha*r12-deno2+2*beta*deno-1.0/r12)

# The Monte Carlo sampling with the Metropolis algo
def MonteCarloSampling():

    NumberMCcycles= 100000
    StepSize = 1.0
    # positions
    PositionOld = np.zeros((NumberParticles,Dimension), np.double)
    PositionNew = np.zeros((NumberParticles,Dimension), np.double)
    # seed for rng generator
    seed()
    # start variational parameter
    alpha = 0.9
    for ia in range(MaxVariations):
        alpha += .025
        AlphaValues[ia] = alpha
        beta = 0.2
        for jb in range(MaxVariations):
            beta += .01
            BetaValues[jb] = beta
            energy = energy2 = 0.0
            DeltaE = 0.0
            #Initial position
            for i in range(NumberParticles):
                for j in range(Dimension):
                    PositionOld[i,j] = StepSize * (random() - .5)
            wfold = WaveFunction(PositionOld,alpha,beta)

            #Loop over MC MCcycles
            for MCcycle in range(NumberMCcycles):
                #Trial position
                for i in range(NumberParticles):
                    for j in range(Dimension):
                        PositionNew[i,j] = PositionOld[i,j] + StepSize * (random() - .5)
                wfnew = WaveFunction(PositionNew,alpha,beta)

                #Metropolis test to see whether we accept the move
                if random() < wfnew**2 / wfold**2:
                   PositionOld = PositionNew.copy()
                   wfold = wfnew
                   DeltaE = LocalEnergy(PositionOld,alpha,beta)
                energy += DeltaE
                energy2 += DeltaE**2

            #We calculate mean, variance and error ...
            energy /= NumberMCcycles
            energy2 /= NumberMCcycles
            variance = energy2 - energy**2
            error = sqrt(variance/NumberMCcycles)
            Energies[ia,jb] = energy
    return Energies, AlphaValues, BetaValues

#Here starts the main program with variable declarations
NumberParticles = 2
Dimension = 2
MaxVariations = 10
Energies = np.zeros((MaxVariations,MaxVariations))
AlphaValues = np.zeros(MaxVariations)
BetaValues = np.zeros(MaxVariations)
(Energies, AlphaValues, BetaValues) = MonteCarloSampling()

# Prepare for plots
fig = plt.figure()
ax = fig.gca(projection='3d')
# Plot the surface.
X, Y = np.meshgrid(AlphaValues, BetaValues)
surf = ax.plot_surface(X, Y, Energies,cmap=cm.coolwarm,linewidth=0, antialiased=False)
# Customize the z axis.
zmin = np.matrix(Energies).min()
zmax = np.matrix(Energies).max()
ax.set_zlim(zmin, zmax)
ax.set_xlabel(r'$\alpha$')
ax.set_ylabel(r'$\beta$')
ax.set_zlabel(r'$\langle E \rangle$')
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))
# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()
```

## Technical aspect, improvements and how to define the cost function

**The above procedure is not the smartest one**. Looping over all variational parameters becomes expensive.
Also, we don't use importance sampling and optimizations of the standard deviation (blocking, bootstrap, jackknife).
Such codes are included in the above Github address.

We can also be smarter and use minimization methods to find the **optimal** variational parameters with fewer Monte Carlo cycles and then
fire up our heavy artillery.

One way to achieve this is to minimize the energy as function of the variational parameters.

## Energy derivatives
To find the derivatives of the local energy expectation value as function of the variational parameters, we can use the chain rule and the hermiticity of the Hamiltonian.

Let us define

$$
\bar{E}_{\alpha_i}=\frac{d\langle  E_L\rangle}{d\alpha_i}.
$$

as the derivative of the energy with respect to the variational parameter $\alpha_i$
We define also the derivative of the trial function (skipping the subindex $T$) as

$$
\bar{\Psi}_{i}=\frac{d\Psi}{d\alpha_i}.
$$

## Derivatives of the local energy
The elements of the gradient of the local energy are then (using the chain rule and the hermiticity of the Hamiltonian)

$$
\bar{E}_{i}= 2\left( \langle \frac{\bar{\Psi}_{i}}{\Psi}E_L\rangle -\langle \frac{\bar{\Psi}_{i}}{\Psi}\rangle\langle E_L \rangle\right).
$$

From a computational point of view it means that you need to compute the expectation values of

$$
\langle \frac{\bar{\Psi}_{i}}{\Psi}E_L\rangle,
$$

and

$$
\langle \frac{\bar{\Psi}_{i}}{\Psi}\rangle\langle E_L\rangle
$$

These integrals are evaluted using MC intergration (with all its possible error sources).
We can then use methods like stochastic gradient or other minimization methods to find the optimal variational parameters (I don't discuss this topic here, but these methods are very important in ML).

## How do we define our cost function?
We have a model, our likelihood function.

How should we define the cost function?

## Meet the variance and its derivatives
**Why the variance?**

Suppose the trial function (our model) is the exact wave function. The action of the hamiltionan
on the wave function

$$
H\Psi = \mathrm{constant}\times \Psi,
$$

The integral which defines various
expectation values involving moments of the hamiltonian becomes then

$$
\langle E^n \rangle =   \langle H^n \rangle =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}(\boldsymbol{R})H^n(\boldsymbol{R})\Psi(\boldsymbol{R})}
        {\int d\boldsymbol{R}\Psi^{\ast}(\boldsymbol{R})\Psi(\boldsymbol{R})}=
\mathrm{constant}\times\frac{\int d\boldsymbol{R}\Psi^{\ast}(\boldsymbol{R})\Psi(\boldsymbol{R})}
        {\int d\boldsymbol{R}\Psi^{\ast}(\boldsymbol{R})\Psi(\boldsymbol{R})}=\mathrm{constant}.
$$

**This gives an important information: If I want the variance, the exact wave function leads to zero variance!**
The variance is defined as

$$
\sigma_E = \langle E^2\rangle - \langle E\rangle^2.
$$

Variation is then performed by minimizing both the energy and the variance.

## The variance defines the cost function
We can then take the derivatives of

$$
\sigma_E = \langle E^2\rangle - \langle E\rangle^2,
$$

with respect to the variational parameters. The derivatives of the variance can then be used to defined the
so-called Hessian matrix, which in turn allows us to use minimization methods like Newton's method or
standard gradient methods.

This leads to however a more complicated expression, with obvious errors when evaluating integrals by Monte Carlo integration. Less used, see however [Filippi and Umrigar](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.94.150201). The expression becomes complicated

$$
\bar{E}_{ij} = 2\left[ \langle (\frac{\bar{\Psi}_{ij}}{\Psi}+\frac{\bar{\Psi}_{j}}{\Psi}\frac{\bar{\Psi}_{i}}{\Psi})(E_L-\langle E\rangle)\rangle -\langle \frac{\bar{\Psi}_{i}}{\Psi}\rangle\bar{E}_j-\langle \frac{\bar{\Psi}_{j}}{\Psi}\rangle\bar{E}_i\right] +\langle \frac{\bar{\Psi}_{i}}{\Psi}E_L{_j}\rangle +\langle \frac{\bar{\Psi}_{j}}{\Psi}E_L{_i}\rangle -\langle \frac{\bar{\Psi}_{i}}{\Psi}\rangle\langle E_L{_j}\rangle \langle \frac{\bar{\Psi}_{j}}{\Psi}\rangle\langle E_L{_i}\rangle.
$$

Evaluating the cost function means having to evaluate the above second derivative of the energy.

## Boltzmann Machines

Why use a generative model rather than the more well known discriminative deep neural networks (DNN)?

* Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.

* A generative model can learn to represent and sample from a probability distribution. The core idea is to learn a parametric model of the probability distribution from which the training data was drawn. As an example

 a. A model for images could learn to draw new examples of cats and dogs, given a training dataset of images of cats and dogs.

 b. Generate a sample of an ordered or disordered Ising model phase, having been given samples of such phases.

 c. Model the trial wave function for Monte Carlo calculations

## Some similarities and differences from DNNs

1. Both use gradient-descent based learning procedures for minimizing cost functions

2. Energy based models don't use backpropagation and automatic differentiation for computing gradients, instead turning to Markov Chain Monte Carlo methods.

3. DNNs often have several hidden layers. A restricted Boltzmann machine has only one hidden layer, however several RBMs can be stacked to make up Deep Belief Networks, of which they constitute the building blocks.

History: The RBM was developed by amongst others Geoffrey Hinton, called by some the "Godfather of Deep Learning", working with the University of Toronto and Google.

## The structure of the RBM network

## The network

**The network layers**:
 1. A function $\mathbf{x}$ that represents the visible layer, a vector of $M$ elements (nodes). This layer represents both what the RBM might be given as training input, and what we want it to be able to reconstruct. This might for example be the pixels of an image, the spin values of the Ising model, or coefficients representing speech.

 2. The function $\mathbf{h}$ represents the hidden, or latent, layer. A vector of $N$ elements (nodes). Also called "feature detectors".

## Goals

The goal of the hidden layer is to increase the model's expressive power. We encode complex interactions between visible variables by introducing additional, hidden variables that interact with visible degrees of freedom in a simple manner, yet still reproduce the complex correlations between visible degrees in the data once marginalized over (integrated out).

Examples of this trick being employed in physics:
 1. The Hubbard-Stratonovich transformation

 2. The introduction of ghost fields in gauge theory

 3. Shadow wave functions in Quantum Monte Carlo simulations

**The network parameters, to be optimized/learned**:
 1. $\mathbf{a}$ represents the visible bias, a vector of same length as $\mathbf{x}$.

 2. $\mathbf{b}$ represents the hidden bias, a vector of same lenght as $\mathbf{h}$.

 3. $W$ represents the interaction weights, a matrix of size $M\times N$.

## Joint distribution and the Energy function
The restricted Boltzmann machine is described by a Bolztmann distribution

$$
\begin{equation}
	P_{rbm}(\mathbf{x},\mathbf{h}) = \frac{1}{Z} e^{-\frac{1}{T_0}E(\mathbf{x},\mathbf{h})},
\label{_auto1} \tag{2}
\end{equation}
$$

where $Z$ is the normalization constant or partition function, defined as

$$
\begin{equation}
	Z = \int \int e^{-\frac{1}{T_0}E(\mathbf{x},\mathbf{h})} d\mathbf{x} d\mathbf{h}.
\label{_auto2} \tag{3}
\end{equation}
$$

It is common to ignore $T_0$ by setting it to one.

## Network Elements

The function $E(\mathbf{x},\mathbf{h})$ gives the **energy** of a
configuration (pair of vectors) $(\mathbf{x}, \mathbf{h})$. The lower
the energy of a configuration, the higher the probability of it. This
function also depends on the parameters $\mathbf{a}$, $\mathbf{b}$ and
$W$. Thus, when we adjust them during the learning procedure, we are
adjusting the energy function to best fit our problem.

## Defining different types of RBMs
There are different variants of RBMs, and the differences lie in the types of visible and hidden units we choose as well as in the implementation of the energy function $E(\mathbf{x},\mathbf{h})$.

**Binary-Binary RBM:**

RBMs were first developed using binary units in both the visible and hidden layer. The corresponding energy function is defined as follows:

$$
\begin{equation}
	E(\mathbf{x}, \mathbf{h}) = - \sum_i^M x_i a_i- \sum_j^N b_j h_j - \sum_{i,j}^{M,N} x_i w_{ij} h_j,
\label{_auto3} \tag{4}
\end{equation}
$$

where the binary values taken on by the nodes are most commonly 0 and 1.

**Gaussian-Binary RBM:**

Another varient is the RBM where the visible units are Gaussian while the hidden units remain binary:

$$
\begin{equation}
	E(\mathbf{x}, \mathbf{h}) = \sum_i^M \frac{(x_i - a_i)^2}{2\sigma_i^2} - \sum_j^N b_j h_j - \sum_{i,j}^{M,N} \frac{x_i w_{ij} h_j}{\sigma_i^2}.
\label{_auto4} \tag{5}
\end{equation}
$$

## More about RBMs
1. Useful when we model continuous data (i.e., we wish $\mathbf{x}$ to be continuous)

2. Requires a smaller learning rate, since there's no upper bound to the value a component might take in the reconstruction

Other types of units include:
1. Softmax and multinomial units

2. Gaussian visible and hidden units

3. Binomial units

4. Rectified linear units

## Sampling: Metropolis sampling
In order to sample from the RBM probability distribution it is common to use Markov Chain Monte Carlo (MCMC) algorithms such as Metropolis-Hastings or Gibbs sampling.

Metropolis sampling starts by suggesting a new configuration $\boldsymbol{x}^{k+1}$. In the brute force method this is done by some random change of the visible units. The new configuration is then accepted with the acceptance probability

$$
\begin{equation}
	A(\boldsymbol{x}^k \rightarrow \boldsymbol{x}^{k+1}) = \text{min} (1, \frac{P(\boldsymbol{x}^{k+1})}{P(\boldsymbol{x}^k)}),
\label{_auto5} \tag{6}
\end{equation}
$$

where we need the marginalized probability

$$
\begin{equation}
	P(\boldsymbol{x})  = \sum_\mathbf{h} P_{rbm}(\mathbf{x}, \mathbf{h})
\label{_auto6} \tag{7}
\end{equation}
$$

$$
\begin{equation}
				= \frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}.
\label{_auto7} \tag{8}
\end{equation}
$$

## Sampling: Gibbs sampling

In this method we sample from the joint probability $P_{rbm} (\mathbf{x}, \mathbf{h})$ by way of a two step sampling process. We alternately update the visible and hidden units.
New samples are generated according to the conditional probabilities $P(x_i|\mathbf{h})$ and $P(h_j|\mathbf{x})$ respectively and accepted with the probability of $1$. While the the visible nodes are dependent on the hidden nodes and vice versa, the nodes are independent of other nodes within the same layer. This is due to there being no intra layer interactions in the restricted Boltzmann machine.

The conditional probabilities are often referred to as the activitation functions in the neural networks context due to their role in determining the node outputs. For the binary-binary RBM they are

$$
\begin{equation}
	P(h_j = 1 | \boldsymbol{x}) = \frac{1}{1 + e^{-b_j - \sum_i x_i w_{ij}}}
\label{_auto8} \tag{9}
\end{equation}
$$

$$
\begin{equation}
	P(x_i = 1 | \boldsymbol{h}) = \frac{1}{1 + e^{-a_j - \sum_j h_j w_{ij}}},
\label{_auto9} \tag{10}
\end{equation}
$$

where we recognize the logistic sigmoid function $\sigma (x) = 1/(1+exp(-x))$.

## Gaussian RBM
For the Gaussian-Binary RBM the conditional probabilities are

$$
\begin{equation}
	P(x_i|\mathbf{h}) = \mathcal{N}(x_i; a_i+ \sum_j h_j w_{ij}, \sigma^2)
\label{_auto10} \tag{11}
\end{equation}
$$

$$
\begin{equation}
	P(h_j=1|\mathbf{x}) =  \frac{1}{1+e^{-b_j-\frac{1}{\sigma^2} \sum_i x_i w_{ij}}},
\label{_auto11} \tag{12}
\end{equation}
$$

while the visible units now follow a normal distribution, we see the hidden units again follow the logistic sigmoid function.

## Cost function

When working with a training dataset, the most common training approach is maximizing the log-likelihood of the training data. The log likelihood characterizes the log-probability of generating the observed data using our generative model. Using this method our cost function is chosen as the negative log-likelihood. The learning then consists of trying to find parameters that maximize the probability of the dataset, and is known as Maximum Likelihood Estimation (MLE).
Denoting the parameters as $\boldsymbol{\theta} = a_1,...,a_M,b_1,...,b_N,w_{11},...,w_{MN}$, the log-likelihood is given by

$$
\begin{equation}
	\mathcal{L}(\{ \theta_i \}) = \langle \text{log} P_\theta(\boldsymbol{x}) \rangle_{data}
\label{_auto12} \tag{13}
\end{equation}
$$

$$
\begin{equation}
	= - \langle E(\boldsymbol{x}; \{ \theta_i\}) \rangle_{data} - \text{log} Z(\{ \theta_i\}),
\label{_auto13} \tag{14}
\end{equation}
$$

where we used that the normalization constant does not depend on the data, $\langle \text{log} Z(\{ \theta_i\}) \rangle = \text{log} Z(\{ \theta_i\})$
Our cost function is the negative log-likelihood, $\mathcal{C}(\{ \theta_i \}) = - \mathcal{L}(\{ \theta_i \})$

## Optimization / Training
The training procedure of choice often is Stochastic Gradient Descent (SGD). It consists of a series of iterations where we update the parameters according to the equation

$$
\begin{equation}
	\boldsymbol{\theta}_{k+1} = \boldsymbol{\theta}_k - \eta \nabla \mathcal{C} (\boldsymbol{\theta}_k)
\label{_auto14} \tag{15}
\end{equation}
$$

at each $k$-th iteration. There are a range of variants of the algorithm which aim at making the learning rate $\eta$ more adaptive so the method might be more efficient while remaining stable.

We now need the gradient of the cost function in order to minimize it. We find that

$$
\begin{equation}
	\frac{\partial \mathcal{C}(\{ \theta_i\})}{\partial \theta_i}
	= \langle \frac{\partial E(\boldsymbol{x}; \theta_i)}{\partial \theta_i} \rangle_{data}
	+ \frac{\partial \text{log} Z(\{ \theta_i\})}{\partial \theta_i}
\label{_auto15} \tag{16}
\end{equation}
$$

$$
\begin{equation}
	= \langle O_i(\boldsymbol{x}) \rangle_{data} - \langle O_i(\boldsymbol{x}) \rangle_{model},
\label{_auto16} \tag{17}
\end{equation}
$$

where in order to simplify notation we defined the "operator"

$$
\begin{equation}
	O_i(\boldsymbol{x}) = \frac{\partial E(\boldsymbol{x}; \theta_i)}{\partial \theta_i},
\label{_auto17} \tag{18}
\end{equation}
$$

and used the statistical mechanics relationship between expectation values and the log-partition function:

$$
\begin{equation}
	\langle O_i(\boldsymbol{x}) \rangle_{model} = \text{Tr} P_\theta(\boldsymbol{x})O_i(\boldsymbol{x}) = - \frac{\partial \text{log} Z(\{ \theta_i\})}{\partial \theta_i}.
\label{_auto18} \tag{19}
\end{equation}
$$

## More on RBMs

The data-dependent term in the gradient is known as the positive phase of the gradient, while the model-dependent term is known as the negative phase of the gradient. The aim of the training is to lower the energy of configurations that are near observed data points (increasing their probability), and raising the energy of configurations that are far from observed data points (decreasing their probability).

The gradient of the negative log-likelihood cost function of a Binary-Binary RBM is then

$$
\begin{equation}
	\frac{\partial \mathcal{C} (w_{ij}, a_i, b_j)}{\partial w_{ij}} = \langle x_i h_j \rangle_{data} - \langle x_i h_j \rangle_{model}
\label{_auto19} \tag{20}
\end{equation}
$$

$$
\begin{equation}
	\frac{\partial \mathcal{C} (w_{ij}, a_i, b_j)}{\partial a_{ij}} = \langle x_i \rangle_{data} - \langle x_i \rangle_{model}
\label{_auto20} \tag{21}
\end{equation}
$$

$$
\begin{equation}
	\frac{\partial \mathcal{C} (w_{ij}, a_i, b_j)}{\partial b_{ij}} = \langle h_i \rangle_{data} - \langle h_i \rangle_{model}.
\label{_auto21} \tag{22}
\end{equation}
$$

$$
\begin{equation}
\label{_auto22} \tag{23}
\end{equation}
$$

To get the expecation values with respect to the *data*, we set the visible units to each of the observed samples in the training data, then update the hidden units according to the conditional probability found before. We then average over all samples in the training data to calculate expectation values with respect to the data.

## Which sampling to use

To get the expectation values with respect to the *model*, we use Gibbs sampling. We can either initialize the $\boldsymbol{x}$ randomly or with a training sample. While we ideally want a large number of Gibbs iterations $n\rightarrow n$, one might decide to truncate it earlier for efficiency. Doing this while having intialized $\boldsymbol{x}$ with a training data vector is referred to as contrastive divergence (CD), because one is then closer to approximating the gradient of this function than the negative log-likelihood. The contrastive divergence function is the difference between two Kullback-Leibler divergences (also called relative entropy), which measure how one probability distribution diverges from a second, expected probability distribution (in this case the estimated one from the ground truth one).

## RBMs for the quantum many body problem

The idea of applying RBMs to quantum many body problems was presented by G. Carleo and M. Troyer, working with ETH Zurich and Microsoft Research.

Some of their motivation included

* "The wave function $\Psi$ is a monolithic mathematical quantity that contains all the information on a quantum state, be it a single particle or a complex molecule. In principle, an exponential amount of information is needed to fully encode a generic many-body quantum state."

* There are still interesting open problems, including fundamental questions ranging from the dynamical properties of high-dimensional systems to the exact ground-state properties of strongly interacting fermions.

* The difficulty lies in finding a general strategy to reduce the exponential complexity of the full many-body wave function down to its most essential features. That is

a. $\rightarrow$ Dimensional reduction

b. $\rightarrow$ Feature extraction

* Among the most successful techniques to attack these challenges, artifical neural networks play a prominent role.

* Want to understand whether an artifical neural network may adapt to describe a quantum system.

## Choose the right RBM

Carleo and Troyer applied the RBM to the quantum mechanical spin lattice systems of the Ising model and Heisenberg model, with encouraging results. Our goal is to test the method on systems of moving particles. For the spin lattice systems it was natural to use a binary-binary RBM, with the nodes taking values of 1 and -1. For moving particles, on the other hand, we want the visible nodes to be continuous, representing position coordinates. Thus, we start by choosing a Gaussian-binary RBM, where the visible nodes are continuous and hidden nodes take on values of 0 or 1. If eventually we would like the hidden nodes to be continuous as well the rectified linear units seem like the most relevant choice.

## Representing the wave function
The wavefunction should be a probability amplitude depending on $\boldsymbol{x}$. The RBM model is given by the joint distribution of $\boldsymbol{x}$ and $\boldsymbol{h}$

$$
\begin{equation}
	F_{rbm}(\mathbf{x},\mathbf{h}) = \frac{1}{Z} e^{-\frac{1}{T_0}E(\mathbf{x},\mathbf{h})}.
\label{_auto23} \tag{24}
\end{equation}
$$

To find the marginal distribution of $\boldsymbol{x}$ we set:

$$
\begin{equation}
	F_{rbm}(\mathbf{x}) = \sum_\mathbf{h} F_{rbm}(\mathbf{x}, \mathbf{h})
\label{_auto24} \tag{25}
\end{equation}
$$

$$
\begin{equation}
				= \frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}.
\label{_auto25} \tag{26}
\end{equation}
$$

Now this is what we use to represent the wave function, calling it a neural-network quantum state (NQS)

$$
\begin{equation}
	\Psi (\mathbf{X}) = F_{rbm}(\mathbf{x})
\label{_auto26} \tag{27}
\end{equation}
$$

$$
\begin{equation}
	= \frac{1}{Z}\sum_{\boldsymbol{h}} e^{-E(\mathbf{x}, \mathbf{h})}
\label{_auto27} \tag{28}
\end{equation}
$$

$$
\begin{equation}
	= \frac{1}{Z} \sum_{\{h_j\}} e^{-\sum_i^M \frac{(x_i - a_i)^2}{2\sigma^2} + \sum_j^N b_j h_j + \sum_{i,j}^{M,N} \frac{x_i w_{ij} h_j}{\sigma^2}}
\label{_auto28} \tag{29}
\end{equation}
$$

$$
\begin{equation}
	= \frac{1}{Z} e^{-\sum_i^M \frac{(x_i - a_i)^2}{2\sigma^2}} \prod_j^N (1 + e^{b_j + \sum_i^M \frac{x_i w_{ij}}{\sigma^2}}).
\label{_auto29} \tag{30}
\end{equation}
$$

$$
\begin{equation}
\label{_auto30} \tag{31}
\end{equation}
$$

## Choose the cost function
Now we don't necessarily have training data (unless we generate it by using some other method). However, what we do have is the variational principle which allows us to obtain the ground state wave function by minimizing the expectation value of the energy of a trial wavefunction (corresponding to the untrained NQS). Similarly to the traditional variational Monte Carlo method then, it is the local energy we wish to minimize. The gradient to use for the stochastic gradient descent procedure is

$$
\begin{equation}
	G_i = \frac{\partial \langle E_L \rangle}{\partial \theta_i}
	= 2(\langle E_L \frac{1}{\Psi}\frac{\partial \Psi}{\partial \theta_i} \rangle - \langle E_L \rangle \langle \frac{1}{\Psi}\frac{\partial \Psi}{\partial \theta_i} \rangle ),
\label{_auto31} \tag{32}
\end{equation}
$$

where the local energy is given by

$$
\begin{equation}
	E_L = \frac{1}{\Psi} \hat{\mathbf{H}} \Psi.
\label{_auto32} \tag{33}
\end{equation}
$$

## Running the codes
You can find the codes for the simple two-electron case at the Github repository <https://github.com/mhjensenseminars/MachineLearningTalk/tree/master/doc/Programs/MLcpp/src>. Python codes to come, only c++ as of now.

The trial wave function is based on the product of a Slater determinant with Gaussian orbitals, a simple Jastrow factor $\exp{(r_{ij})}$ and the reduced Boltzmann machines.

The Broyden-Fletcher-Goldfarb-Shanno algorithm was used to perform the minimization. We used $14$ hidden nodes in the calculations below.

## Energy as function of iterations, $N=2$ electrons

## Energy as function of iterations, $N=6$ electrons

## Conclusions and where do we stand
* A simple extension of the work of [G. Carleo and M. Troyer, Science **355**, Issue 6325, pp. 602-606 (2017)](http://science.sciencemag.org/content/355/6325/602) gives excellent results for two-electron systems as well as good agreement with standard VMC calculations for $N=6$ and $N=12$ electrons.

* Minimization problem can be tricky.

* Anti-symmetry dealt with multiplying the trail wave function with an optimized Slater determinant.

* To come: Analysis of wave function from ML and compare with diffusion and Variational Monte Carlo calculations as well as the analytical results of Taut for the two-electron case.

* Extend to more fermions. How do we deal with the antisymmetry of the multi-fermion wave function?

 a. Here we used standard Hartree-Fock theory to define an optimal Slater determinant. Takes care of the antisymmetry. What about constructing an anti-symmetrized network function?

 b. Use thereafter ML to determine the correlated part of the wafe function (including a standard Jastrow factor).

 c. Test this for multi-fermion systems and compare with other many-body methods.

* Can we use ML to find out which correlations are relevant and thereby diminish the dimensionality problem in say CC or SRG theories?


$\newcommand{\myvector}[2]{\left\{ \begin{array}{c}#1\\#2\end{array} \right\}}$
$\newcommand{\sM}[1]{s_{\mathrm{M#1}}}$
$\newcommand{\xM}[1]{x_{\mathrm{M#1}}}$
$\newcommand{\xpM}[1]{x'_{\mathrm{M#1}}}$

# APPENDIX C:  Bayesian inverse kinematics basics

This Appendix describes a simple Bayesian inverse kinematics (IK) model to introduce the computational details of the main manuscript's Bayesian IK model.

Consider the slider mechanism pictured below.  This mechanism consists of a single rigid slider which moves to a global position $x$, and to which two markers ("M1" and "M2") are rigidly fixed.  Let's assume that we know that the marker positions relative to the slider's left edge are: $\sM{1}$ = 35 mm and $\sM{2}$ = 45 mm.

Let's also assume that we have imperfectly measured the global marker positions ($\xpM{1}$ and $\xpM{2}$) as 46.0 and 55.0 mm, respectively. The IK problem is to find the slider position $x$ given the marker measurements $\xpM{1}$ and $\xpM{2}$.

### Approach 1: Least-squares

The true global marker positions can be expressed in terms of our unknown variable $x$ as follows:

\begin{align}
\xM{1} &= x + 35 \ \ \ \ \ \ \ \ \ \ \ \rm{(Equation \ 1)} \\
\xM{2} &= x + 45\\
\end{align}

The sum of the squared measurement errors is:

\begin{equation}
f(x) = (\xM{1} - 46)^2 + (\xM{2} - 55)^2
\end{equation}

which reduces to:

\begin{align}
f(x)   &= (x + 35 - 46)^2 + (x + 45 - 55)^2 \\
       &= (x - 11)^2 + (x - 10)^2 \\
       &= (x^2 -22x + 121) + (x^2 - 20x + 100)\\
       &= 2x^2 -42x + 221
\end{align}

The minimum value of this function is the least-squares (LS) solution, and is given when the derivative is zero:

\begin{align}
\frac{d}{dx}f(x) &= 4x - 42 = 0\\
x &= \frac{42}{4}
\end{align}

so our least-squares estimate is: $x$ = 10.5 mm, and the LS-minimized measurement error is 0.5 mm for each marker.

### Approach 2:  Bayesian

In contrast to the least-squares (LS) approach, which minimizes measurement error, **the Bayesian approach maximizes model parameters' mean posterior probabilities** given observed (noisy) data. A roughly equivalent interpretation is that the Bayesian approach maximizes the probability of observing the given data. This is done by optimizing the parameters which characterize a stochastic system.

To implement Bayesian IK we start with a model of our measurements as depicted below.

This is a diagram which illustrates what we know, what we don't know, and what we measure.

- We don't know the true slider position $x$, so $x$ is modeled as a **stochastic variable**. Like in Appendix A and Appendix B, our pre-measurement beliefs regarding the true slider position can be modeled using a prior distribution, as detailed below.

- We also don't know the true positions of the two markers ($\xM{1}$ and $\xM{2}$). However, these positions will become known once the slider moves to a certain position $x$.  In other words, setting the slider position determines the marker positions, so the marker positions are **deterministic variables**.

- We know the measured marker positions ($\boldsymbol{q}$). Bayesian inference should treat these positions as **measured variables**.

- We know that marker measurements generally contain error ($\boldsymbol{\epsilon}$), but we don't yet know exactly what the error is for each marker.  Like the slider position $x$, $\boldsymbol{\epsilon}$ can also generaly be treated as a stochastic variable. In some cases we know the measurement error, based on manufacturer specifications and/or based on experiments, so the error amplitude ($\boldsymbol{\epsilon}$) can instead be treated as a constant value. The diagram above depicts $\boldsymbol{\epsilon}$ as a stochastic variable simply because it is more general to model the data this way, and Bayesian inference can easily and simultaneously estimate the most likely values for both $x$ and $\boldsymbol{\epsilon}$.

Here the overall goal of Bayesian inference is to estimate the most likely values of both $x$ and $\boldsymbol{\epsilon}$, given the marker measurements and given our measurements model (Equation 1). More specifically, Bayesian inference will mutually maximize the maxima of the posterior distributions for $x$ and $\boldsymbol{\epsilon}$, and from these posteriors we can extract the most likely values for $x$ and $\boldsymbol{\epsilon}$.

Let's start to implement this Bayesian model in Python by defining known values:  the true local positions and the observed measurement values.

```python
%matplotlib notebook

import numpy as np
from matplotlib import pyplot

### local positions:
sM1 = 35.0
sM2 = 45.0
### measurements:
xpM1 = 46.0
xpM2 = 55.0
```

Let's next implement a function which will generate random datasets $\boldsymbol{q}$ according to our measurement model. Let's arbitrarily set the current slider position to $x$ = 8 mm, and let's assume that our measurement error $\boldsymbol{\epsilon}$ is Gaussian with a standard deviation of 0.5 mm.

```python
import random

noise_sd = 0.5  #mm

#define measurement model:
def random_measurement(x):
    xM1 = x + sM1 + random.gauss(0, noise_sd)
    xM2 = x + sM2 + random.gauss(0, noise_sd)
    return [xM1, xM2]

#generate random data (20 observations):
random.seed(0)
x = 8.0
X = np.array([random_measurement(x)  for i in range(20)])
print( X )
```

    [[ 43.4708577   52.30171095]
     [ 42.66014278  53.18525178]
     [ 42.49182555  52.96393999]
     [ 43.08959824  52.58445039]
     [ 42.34548132  53.09694387]
     [ 43.49662485  52.67650918]
     [ 42.83316601  53.82283588]
     [ 42.72055512  52.74292167]
     [ 44.20205967  52.2344587 ]
     [ 43.39823292  51.99817574]
     [ 42.70151863  53.75184044]
     [ 43.61071821  52.54943992]
     [ 42.77315063  53.04011652]
     [ 42.37094835  53.27611   ]
     [ 44.11378865  52.32237925]
     [ 42.00923346  53.14412187]
     [ 42.94043834  53.90216497]
     [ 42.91981891  52.97467014]
     [ 42.90456306  52.50469688]
     [ 43.33651499  52.33795877]]

This represents 20 different sets of measurements we could expect if our model is accurate. The results can be visualized as follows:

```python
def myplot(X):
    ax = pyplot.axes()
    ax.plot(xpM1, 0, 'bo', markersize=10, label='Measured M1')
    ax.plot(xpM2, 0, 'ro', markersize=10, label='Measured M2')
    ax.plot(X[:,0], [0]*X.shape[0], 'b.', markersize=4, label='Simulated M1')
    ax.plot(X[:,1], [0]*X.shape[0], 'r.', markersize=4, label='Simulated M2')
    ax.legend()
    ax.set_xlabel('Position', size=20)

pyplot.figure()
myplot(X)
```

    <IPython.core.display.Javascript object>

For the current position ($x$=8) and assumed measurement error ($\epsilon$=0.5) the simulated data do not overlap with the measured data, implying that the current value of $x$ is unlikely to have produced the observed (measured) data. Let's change $x$ to 10 and try again.

```python
X = np.array([random_measurement(10)  for i in range(20)])
pyplot.figure()
myplot(X)
```

    <IPython.core.display.Javascript object>

There is now much greater overlap between the simulated and measured data, implying that $x$=10 is more likely to have produced the observed data than $x$=8. To find the optimum value for $x$ using Bayesian inference, let's first implement our observations model in PyMC.

```python
import pymc

tau = 1 / 0.5**2    #measurement precision ( 1 / SD^2 )
x   = pymc.Uniform("x", -20, 20)  #prior distribution for x

@pymc.deterministic
def observations_model(x=x):
    xM1 = x + sM1  #global position of Marker 1
    xM2 = x + sM2  #global position of Marker 2
    return [xM1, xM2]
q = pymc.Normal("q", observations_model, tau)
```

In the code snippet above a variable "`tau`" is created to specify our assumed measurement precision, where precision ($\tau$) is defined as the inverse of variance ($\sigma^2$):

\begin{equation}
\tau \equiv \dfrac{1}{\sigma^2}
\end{equation}

Since we have assumed a standard deviation of 0.5 mm, our assumed precision is: $\tau$=4.

Next we specify a prior distribution for the slider position $x$. The prior distribution used above is uniform between $x=-20$ and $x=+20$, implying that the slider can be anywhere in that range with equal probability. Note that the variable "`x`" is a stochastic PyMC variable.  This means that we can generate random values for this variable based on its current distribution as follows:

```python
print( x.random() )
print( x.random() )
print( x.random() )
print( x.random() )
print( x.random() )
```

    -11.315754709950458
    7.105600574322246
    -11.94659527862421
    -11.516805732079959
    15.236101916711526

You can see that these random values span the range [-20, 20]. Every time we call `random()`, PyMC changes `x`'s  "`value`" attribute as follows.

```python
print( x.value )  #the final value from above
print( x.value )  #the value hasn't changed yet
x.random()
print( x.value )  #now it has
```

    15.236101916711526
    15.236101916711526
    4.087767389281446

The last step in the code snippet above is to create a PyMC observations model. First we create a function called `observations_model` which computes the expected marker positions based on the value of `x` and returns both values.  Next we tell PyMC that the result of `observations_model` should be regarded as stochastic, with a normal (Gaussian) distribution that has a mean of the true marker positions (given `x`) and a precision of `tau`.  The "`@pymc.deterministic`" statement is a function decorator which alerts PyMC to regard this function as a deterministic variable, and to use `x`'s value inside that function. Just like `x`, `q` is now a PyMC stochastic variable and we can generate random `q` values as follows.

```python
x.set_value(0)      #set the slider position
print( q.random() ) #random marker positions given x=0
print( q.random() )
print( q.random() )
print( q.random() )
print( q.random() )
```

    [ 34.85133707  45.18584781]
    [ 35.65154373  44.95182808]
    [ 34.3090463  44.7054686]
    [ 35.29373893  45.08080711]
    [ 34.2352072   45.66026864]

If we instead choose the least-squares solution $x=10.5$ we can generate random data which looks much more like our observed values of:  $\xpM{1}=46$ and $\xpM{2}=55$.

```python
x.set_value(10.5)
print( q.random() )
print( q.random() )
print( q.random() )
print( q.random() )
print( q.random() )
```

    [ 45.97886828  55.90633564]
    [ 45.0061945   55.79180418]
    [ 44.88987861  55.76826227]
    [ 45.10581923  55.49284068]
    [ 45.92758787  55.22932364]

Instead of directly altering $x$ in this manner, and then finding its optimal value in a numerical optimization sense, PyMC instead uses the observed data to alter $x$'s current distribution, and then examine $x$'s and $q$'s posterior distributions in attempts to maximize their maxima.  In other words, rather than searching for a single optimum value for $x$, PyMC recognizes that both $x$ and $q$ are probabilistic, so attempts to mutually maximize their posterior probabilities, given the observed data.  In order to achieve that we first have to tell PyMC what our observations were. This can be done by slightly altering our $q$ variable as follows:

```python
qobs = pymc.Normal("qobs", observations_model, tau, value=[46,55], observed=True)
```

Next, maximizing the posterior distributions can be achieved with a Markov Chain Monte Carlo (MCMC) simulation:

```python
mcmc = pymc.MCMC([qobs, x])
mcmc.sample(40000, 20000)
```

     [-----------------100%-----------------] 40000 of 40000 complete in 1.0 sec

The first line initializes a model for MCMC simulation using all stochastic variables. The second line iteratively samples from their posterior distributions, discarding the first 20,000 iterations, and retaining the final 20,000 iterations for the final posterior distributions.  Let's examine our posterior distribution for $x$.

```python
X = mcmc.trace('x')[:]
pyplot.figure()
pyplot.hist(X);
```

    <IPython.core.display.Javascript object>

This posterior distribution spans the least-squares solution ($x$=10.5), and the maximum a posteriori likelihood (MAP) is very close to the least-square solution:

```python
print( X.mean() )  #MAP
```

    10.5414669957

However, as depicted in the posterior distribution above there is also a fair bit of uncertainty in our estimate for $x$. Surely values of $x$=10 or $x$=11.0 aren't the best choices, but this result suggests that they are still plausible, based on our modeling assumptions.  So let's change the assumptions and examine the effects. Let's first try increasing the presumed measurement precision (from SD=0.5 mm to SD=0.1 mm).

```python
tau  = 1 / 0.1**2    #measurement precision ( 1 / SD^2 )
x    = pymc.Uniform("x", -20, 20)  #prior distribution for x
@pymc.deterministic
def observations_model(x=x):
    xM1 = x + sM1  #global position of Marker 1
    xM2 = x + sM2  #global position of Marker 2
    return [xM1, xM2]
qobs = pymc.Normal("qobs", observations_model, tau, value=[46,55], observed=True)

mcmc = pymc.MCMC([qobs, x])
mcmc.sample(40000, 20000)

pyplot.figure()
X = mcmc.trace('x')[:]
pyplot.hist(X, range=(10.2,10.8), bins=31);
```

     [-----------------100%-----------------] 40000 of 40000 complete in 0.9 sec

    <IPython.core.display.Javascript object>

Greater assumed measurement precision has narrowed the posterior. Let's next try a presumption of extremely precise measurements (SD=0.01 mm).

```python
tau  = 1 / 0.01**2    #measurement precision ( 1 / SD^2 )
x    = pymc.Uniform("x", -20, 20)  #prior distribution for x
@pymc.deterministic
def observations_model(x=x):
    xM1 = x + sM1  #global position of Marker 1
    xM2 = x + sM2  #global position of Marker 2
    return [xM1, xM2]
qobs = pymc.Normal("qobs", observations_model, tau, value=[46,55], observed=True)

mcmc = pymc.MCMC([qobs, x])
mcmc.sample(40000, 20000)

pyplot.figure()
X = mcmc.trace('x')[:]
pyplot.hist(X, range=(10.2,10.8), bins=31);
```

     [-----------------100%-----------------] 40000 of 40000 complete in 0.9 sec

    <IPython.core.display.Javascript object>

Clearly our assumptions regarding $\tau$ have affected our posterior distribution.  In fact, it has become clear that **the least-squares solution is equivalent to the Bayesian solution under an assumption of infinite measurement precision**.  An equivalent interpretation is that as the presumed measurement precision increases, the Bayesian solution converges to $x$=10.5, with no uncertainty, just like the least-squares solution.

This seems somewhat strange:  the least-squares solution has minimized measurement error, so how can that be equivalent to an assumption of no measurement error?  This apparent paradox is resolved by the opposite perspectives on measurements that the two approaches adopt.  The least-squares approach regards the measured data as hard, and minimizes an explicit function of those measurements.  Contrastingly, the Bayesian approach regards the measured data as soft -- just one manifestation of an infinite set of plausible measurements.  Thus setting infinite precision forces the Bayesian approach to regard the data as hard, and consequently to assume zero variability in its estimate of $x$.

This raises an important point:  what actual value of precision should we use?  Most measurement device manufacturers provide technical specifications regarding measurement accuracy values, so one option is to use those precision data. A second, more Bayesian option, is to regard precision itself as stochastic, and to then let Bayesian inference decide the most likely precision, based on the data.  This can be implemented as follows:

```python
x    = pymc.Uniform("x", -20, 20)  #prior distribution for x
tau  = pymc.Normal("tau", 4, 2, value=4)  #prior distribution for tau

@pymc.deterministic
def observations_model(x=x):
    xM1 = x + sM1  #global position of Marker 1
    xM2 = x + sM2  #global position of Marker 2
    return [xM1, xM2]
qobs = pymc.Normal("qobs", observations_model, tau, value=np.array([46,55]), observed=True)

mcmc = pymc.MCMC([qobs, x, tau])   #now tau also appears in the collection of stochastic variables
mcmc.sample(40000, 20000)

pyplot.figure(figsize=(8,6))
X   = mcmc.trace('x')[:]
TAU = mcmc.trace('tau')[:]
SD  = (1/TAU)**0.5
ax  = pyplot.subplot(221);  ax.hist(X);   ax.text(0.05, 0.85, r'$x$', size=16, transform=ax.transAxes)
ax  = pyplot.subplot(222);  ax.hist(TAU); ax.text(0.05, 0.85, r'$\tau$', size=16, transform=ax.transAxes)
ax  = pyplot.subplot(224);  ax.hist(SD);  ax.text(0.05, 0.85, r'$SD$', size=16, transform=ax.transAxes);  ax.set_xlim(0.3, 0.8)

print( 'Most likely value for x: %.5f' %X.mean() )
print( 'Most likely value for tau: %.5f' %TAU.mean() )
print()
print( 'Most likely value for sd [sd = sqrt(1/tau)]: %.5f' %SD.mean() )
```

     [-----------------100%-----------------] 40000 of 40000 complete in 1.4 sec

    <IPython.core.display.Javascript object>

    Most likely value for x: 10.49799
    Most likely value for tau: 3.94621

    Most likely value for sd [sd = sqrt(1/tau)]: 0.51012

Our new definition of "`tau`" implies that we are reasonably certain that the true measurement precision is 4 (sd=0.5), but it might also be slightly larger or slightly smaller.  This flexibility permits Bayesian inference to simultaneously find the most likely range of values for both $x$ and $\tau$. The final results depicted above suggest the following:

* The observed measurements tell us that the true value of $x$ is most likely very close to 10.5, but due to measurement errors we cannot be completely confident in this value.  The true value of $x$ almost certainly lies in the range [9.5, 11.5].

* The observed measurements suggest that our presumed measurement error model is probably fine, and that a value close to sd=0.5 is most likely the true error value. However, due to random measurement errors we cannot be certain. The true standard deviation of marker errors almost certainly lies in the range [0.4, 0.7].

### Summary

The mechanical slider model and marker measurements discussed in this Appendix are mechanically simple but highlight a variety of important differences between the least-squares (LS) and Bayesian approaches to inverse kinematics (IK).  While the LS approach minimizes an explicit function of the measured data, the Bayesian approach instead maximizes the maximim a posteriori probabilities (MAP) of all stochastic variables. Consequenty, the Bayesian approach propagates measurement uncertainty to the IK solution, but the LS solution does not.  The Bayesian and LS squares approaches yield equivalent IK solutions under an assumption of infinite measurement precision. Last, results above show that Bayesian IK can be sensitive to modeling assumptions, including especially to the prior distributions.


# Papoulis

    TODO

```python
import sys
sys.path.append('../lib')

import sympy
import sympy.functions.elementary.exponential as symExp
import numpy as np
import matplotlib.pyplot as plt
import visuals
import auxlib
```

```python
c, t, U = sympy.symbols('c, t, U')
```

```python
f = c*symExp.exp(-c*t)
f
```

$\displaystyle c e^{- c t}$

```python
paramsToSub = {c:2}
```

```python
auxlib.integrateFunc(f, t, bounds=[0, sympy.oo], paramsToSub=paramsToSub)
```

$\displaystyle 1$

```python
func_int = auxlib.integrateFunc(f, t, paramsToSub=paramsToSub, conds='piecewise')
func_int - auxlib.get_constant()
```

$\displaystyle - e^{- 2 t}$

sympy.limit(q, t, sympy.oo)

$\infty$

```python
from sympy.utilities.lambdify import lambdify
func = lambdify(t, func_int - auxlib.get_constant(), 'scipy') # returns a numpy-ready function1
func
```

    <function _lambdifygenerated(t)>

```python
interval = np.linspace(-1, 5, num=1000)
values = func(interval)
```

def pdfInfo(interval, values, rng, xlim=None):
    fig, ax = plt.subplots()
    ax.plot(interval, values, 'k')

    if not np.isscalar(rng) and len(rng)==1:
        rng = rng[0]

    if not np.isscalar(rng):
        x_1 = np.argmax(interval>rng[0])
        x_2 = np.argmax(interval>rng[1])
        ax.fill_between(np.linspace(rng[0], rng[1], num=len(values[x_1:x_2])), values[x_1:x_2], color='k')
    else:
        x = np.argmax(interval>rng)
        x_0 = interval[0]
        ax.fill_between(np.linspace(x_0, rng, num=len(values[0:x])), values[0:x], color='k')

    ax.axhline(0, c='k', alpha=.5)
    ax.axvline(0, c='k', alpha=.5)

    ax.yaxis.grid(b=True, which='minor', color='k', linestyle='--', alpha=.2, zorder=0)
    ax.yaxis.grid(b=True, which='major', color='k', linestyle='--', alpha=.5, zorder=0)

    ax.set_facecolor('xkcd:white')
    ax.spines["top"].set_visible(False)
    ax.spines["right"].set_visible(False)
    ax.spines["left"].set_visible(False)
    ax.spines["bottom"].set_visible(False)

    if xlim != None:
         ax.set_xlim(xlim)

```python
visuals.plotPdf(interval, values, [-.5, 3], xlim=[-2, 7])
```


```python
from scipy import signal
import numpy as np
from matplotlib import pyplot as plt
import control
import tikzmagic
import warnings
warnings.filterwarnings('ignore')
from jupyterthemes import jtplot
jtplot.style()

%matplotlib inline
```

```python
from __future__ import print_function
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
```

```javascript
%%javascript
IPython.OutputArea.prototype._should_scroll = function(lines) {
    return false;
}
```

    <IPython.core.display.Javascript object>

#### Initial modelling of the system

Euler-Lagrange Equations <br/>

\begin{align}
L & := T - V  \\
\end{align}

\begin{align}
\frac{d}{dt} \left(\frac{\partial L}{\partial \dot{q_i}} \right) - \frac{\partial L}{\partial q_i} = Q_i
\end{align}

For the pendulum we have $q$ as $x$ , this will give us the equations of movement for $x$. We also have $q$ as $\theta$, this will give us the equations of motion for $\theta$ , soon we'll need to solve both equations

\begin{align}
\frac{d}{dt} \left(\frac{\partial L}{\partial \dot{x}} \right) - \frac{\partial L}{\partial x} \\ \\
\frac{d}{dt} \left(\frac{\partial L}{\partial \dot{\theta}} \right) - \frac{\partial L}{\partial \theta}
\end{align}

Where T is the total kinetic energy of the system and V is the potential energy. Considering the cart as V = 0. For m,l,g,$\theta$ we have respectively the mass, length of the pendulum, gravity and the angle between the vertical and the pendulum

\begin{align}
V & = mgl cos \theta \\
\end{align}

Now we are interested in calculating the kinetic energy of the system. Taking M as the total mass of the cart, F as the force that pushes the cart, we have

\begin{align}
T & = \frac{1}{2}\left(M+m\right)\dot{x}^2 - m\dot{x}l\dot{\theta}cos\theta + \frac{1}{2}ml^2\dot{\theta}^2   \\
\end{align}

Therefore

\begin{align}
L & = \frac{1}{2}\left(M+m\right)\dot{x}^2 - m\dot{x}l\dot{\theta}cos\theta + \frac{1}{2}ml^2\dot{\theta}^2 -mglcos\theta  \\
\end{align}

Solving both equations for $x$ and $\theta$ we have

\begin{align}
(M+m)\ddot{x} + ml\dot{\theta}^2sen\theta - ml\ddot{\theta}cos\theta = F \\
\ddot{x}cos\theta + l\ddot{\theta} - gsen\theta = 0
\end{align}

This modelling is for a simple pendulum moved by a cartm for a physical pendulum, some adjustments are required. Every adjustment were taken from this thesis http://ethesis.nitrkl.ac.in/2213/1/Inverted_Pendulum.pdf

\begin{align}
(M+m)\ddot{x} + b\dot{x} - ml\dot{\theta}^2sen\theta + ml\ddot{\theta}cos\theta = F \\
ml\ddot{x}cos\theta + (I + ml^2)\ddot{\theta} - mglsen\theta = 0
\end{align}

##### Linearization about a point

\begin{align}
\theta = \Pi + \phi \\
\\
\end{align}
Being $\phi \approx 0$ we can make a small angle approximation
\begin{align}
\\
cos\phi = -1 \\
sen\phi = \phi \\
\dot{\theta} = 0 \\
\\
\end{align}
After the linearization we have
\begin{align}
(M+m)\ddot{x} + b\dot{x} -ml\dot{\phi} = u \\
(I + ml^2)\ddot{\phi} -mgl\phi - ml\ddot{x} = 0
\\
\end{align}
#### Laplace
\begin{align}
X(s)(M+m)s^2 + X(s)bs -\Phi(s)mls = U(s) \\
\Phi(s)(I + ml^2)s^2 -\Phi(s)mgls - X(s)mls^2 = 0
\\
\end{align}

#### Transfer functions

\begin{align}
\frac{X(s)}{U(s)} = \frac{(I + ml^2)s^2 - mgl}{ [(M+m)(I + ml^2) - ml^2]s^4 + b(I + mIl^2)s^3 - (M + m)mgls^2 -mglbs} \\
\\
\end{align}
\begin{align}
\frac{\Phi(s)}{U(s)} = \frac{\frac{ml}{q}s}{s^3 - \frac{b(I+ml^2)}{q}s^2 - \frac{(M+m)mgl}{q}s - \frac{bmgl}{q}}
\\
q := [(M+m)(I+ml^2) - ml^2]
\end{align}

Igoring the friction ( Oh my dear hearth ) and applying the parameters we have

\begin{align}
\frac{X(s)}{U(s)} = \frac{b_2}{s^2} \\
\\
\end{align}
\begin{align}
\frac{\Phi(s)}{U(s)} = \frac{b_1}{s^2 - a_1^2}
\end{align}

Where

\begin{align}
\therefore && b_1 = 0.2570 &&
a_1 = 6.6307 &&
b_2 = 0.3802 & \implies
\frac{X(s)}{U(s)} = \frac{0.3802}{s^2} && \frac{\Phi(s)}{U(s)} = \frac{0.2570}{s^2 - 6.6307}
\end{align}

#### Inverted pendulum Time Domain

Just the angle transfer function
\begin{align}
\\
\frac{\Phi(s)}{U(s)} = \frac{0.2570}{s^2 - 6.6307}
\end{align}

```python
num = [0.2570]
den = [1,0,-6.6370]
ft_angulo = signal.TransferFunction(num, den)
```

```python
polos = ft_angulo.poles
polos
```

    array([ 2.57623757, -2.57623757])

```python
fig = plt.figure(figsize=(15,5))
plt.grid(True,linewidth=.3)
[plt.scatter(polo,0,marker='x',c='b',s=100) for polo in polos]
plt.show()

```

### Step and Impulse Response

```python
fig = plt.figure(figsize=(15,5))
t,y = signal.impulse(ft_angulo)
t_degrau,y_degrau = signal.step(ft_angulo)
plt.plot(t,y)
plt.plot(t_degrau,y_degrau,'r')
plt.legend(['Resposta ao impulso','Resposta ao degrau'])
plt.title('Análise resposta no domínio do tempo')
plt.show()
```

### Control

```python
roots_real = []
roots_img = []
colors_vec = []
ks = np.linspace(0,50,30)
for k in ks:
    tf = [1,0,0.2570*k - 6.6370]
    poles = np.roots(tf)
    [roots_real.append(pole.real) for pole in poles]
    [roots_img.append(pole.imag) for pole in poles]
    colors_vec.append(0)
    colors_vec.append(1)

def update_rootlocus(kp):
    fig = plt.figure(figsize=(15,5))
    ax = fig.add_subplot(1, 1, 1)
    colors = ['b','r']
    ax.grid(True)
    ax.set_ylim(-3,3)
    ax.set_xlim(-3,3)
    plt.scatter(2.57,0,s=150,c=colors[0],marker='X')
    plt.scatter(-2.57,0,s=150,c=colors[1],marker='X')
    for i in range(len(roots_real)):
        plt.scatter(roots_real[i],roots_img[i],marker='x',s=100,c=colors[colors_vec[i]])

    tf_interact = [1,0,0.2570*kp - 6.6370]
    poles_interact = np.roots(tf_interact)
    is_real_interact = np.isreal(poles_interact[0])
    if is_real_interact:
        for i,pole in enumerate(poles_interact):
            plt.scatter(pole,0,s=150,marker='o',c=colors[i])
    else:
        for i,pole in enumerate(poles_interact):
            plt.scatter(pole.real,pole.imag,s=150,marker='o',c=colors[i])
    legend_1 = str(round(poles_interact[0].real,3)) + ' + ' + str(round(poles_interact[0].imag,3))  + 'j'
    legend_2 = str(round(poles_interact[1].real,3)) + ' + ' + str(round(poles_interact[1].imag,3))  + 'j'
    ax.legend([legend_1,legend_2])

    plt.show()

interact(update_rootlocus,kp=(-5,50,1))
```

    interactive(children=(IntSlider(value=22, description='kp', max=50, min=-5), Output()), _dom_classes=('widget-…

    <function __main__.update_rootlocus(kp)>

### PID Control

```python
from control import tf,feedback,step_response,impulse_response
def pzmap(transfer_function):
    # plot the poles and zeros of the transfer function
    poles = transfer_function.pole()
    zeros = transfer_function.zero()
    plt.xlim(20,-20)
    plt.ylim(20,-20)
    [plt.scatter(pole.real,pole.imag,marker='X') for pole in poles]
    [plt.scatter(zero.real,zero.imag,marker='o') for zero in zeros]
    plt.grid()
    plt.show()

def _step(transfer_function):
    # plot the step response for the transfer function
    T, yout = step_response(transfer_function)
    plt.plot(T,yout)
    plt.show()
```

```python
# PID ISA Kc (1 + 1/(Ti * s) + Td * s )

colors = ['rosybrown','olivedrab','darkgoldenrod','lightslategray']

def test_PID(K,Ti,Td,points,step_function,show_zeros_and_poles,ax_autoresize):
    roots_real = []
    roots_img = []
    color_vector = []
    Kcs = np.linspace(0,250,points)
    s = tf([1,0],1)
    G = (0.2570)/(s**2 - 6.6370)
    for Kc in Kcs:
        if(Ti == 0):
            Gc = Kc*(1 + s*Td)
        else:
            Gc = Kc*(1 + s*Td + 1/(Ti*s))
        closed_loop_tf = feedback(Gc*G,1)
        poles = closed_loop_tf.pole()
        how_many_poles = len(poles)
        counter = 0
        if(how_many_poles > 0):
            for i,pole in enumerate(poles):
                roots_real.append(poles[i].real)
                roots_img.append(poles[i].imag)
                color_vector.append(colors[counter])
                counter += 1
                if(counter == how_many_poles): counter = 0
    length = 5
    if(step_function): length = 10
    fig = plt.figure(figsize=(15,length))
    if(step_function):
        ax = fig.add_subplot(211)
        ax2 = fig.add_subplot(212)
        ax2.grid(True)
    else:
        ax = fig.add_subplot(111)
    ax.grid(True)

    for i in range(len(roots_real)):
        ax.scatter(roots_real[i],roots_img[i],marker='x',s=30,color=color_vector[i])

    if(K != 0):
        if(Ti == 0):
            Gc = K*(1 + s*Td)
        else:
            Gc = K*(1 + s*Td + 1/(Ti*s))
        closed_loop_tf = feedback(Gc*G,1)
        _poles = closed_loop_tf.pole()
        _zeros = closed_loop_tf.zero()
        [ax.scatter(pole.real,pole.imag,marker='X',s=100,color='black') for pole in _poles]
        [ax.scatter(zero.real,zero.imag,marker='o',s=100,color='darkgreen') for zero in _zeros]
        if(step_function):
            T, yout = step_response(closed_loop_tf)
            ax2.plot(T,yout)
        total_number_of_poles_and_zeros = len(_poles) + len(_zeros)
        legends = []
        [legends.append(str(round(pole.real,3)) + ' + ' + str(round(pole.imag,3))  + 'j') for pole in _poles]
        [legends.append(str(round(zero.real,3)) + ' + ' + str(round(zero.imag,3))  + 'j') for zero in _zeros]
        if(show_zeros_and_poles): ax.legend(legends)
        if(not ax_autoresize):
            ax.set_ylim(-5,5)
            ax.set_xlim(-5,5)

interact(test_PID,
         K=widgets.BoundedFloatText(
                            value=0,
                            min=-500,
                            max=500.0,
                            step=.2,
                            description='K:',
                            disabled=False
                        ),
         Ti=widgets.BoundedFloatText(
                            value=0,
                            min=-500,
                            max=500.0,
                            step=.01,
                            description='Ti:',
                            disabled=False
                        ),
         Td=widgets.BoundedFloatText(
                            value=0,
                            min=-500,
                            max=500.0,
                            step=.01,
                            description='Td:',
                            disabled=False
                        ),
         points=widgets.BoundedFloatText(
                            value=10,
                            min=10.0,
                            max=200.0,
                            step=1,
                            description='Points:',
                            disabled=False
                        ),
         step_function = widgets.Checkbox(
                                            value=False,
                                            description='Resposta ao Degrau',
                                            disabled=False
                                        ),
         show_zeros_and_poles = widgets.Checkbox(
                                            value=True,
                                            description='Mostrar polos e zeros',
                                            disabled=False
                                        ),
         ax_autoresize = widgets.Checkbox(
                                            value=True,
                                            description='Auto ajustar escala',
                                            disabled=False
                                        )
        )
#K=widgets.FloatSlider(min=-1000.0, max=1000.0, value=1,step=0.2, description='K')
```

    interactive(children=(BoundedFloatText(value=0.0, description='K:', max=500.0, min=-500.0, step=0.2), BoundedF…

    <function __main__.test_PID(K, Ti, Td, points, step_function, show_zeros_and_poles, ax_autoresize)>

### Even more functionality

```python
""" General parameters """
tab_titles   = ['Simulation','Process Transfer Function','ISA PID Parameters']

""" Tab Plant """
label_numerador = widgets.Label('Numerator :',layout = widgets.Layout(width = '12%'))
box_layout = widgets.Text(layout=widgets.Layout(width = "15%",height = '40px'))
numerador = widgets.Text(layout = box_layout.layout)
space = widgets.Label('',layout = widgets.Layout(width = '3%'))
tip = widgets.Label('e.g 1,0,-6.6370 --> s^2 - 6.6370',layout = widgets.Layout(width = '25%'))

container_numerador = widgets.HBox(children=[label_numerador,numerador,space,tip])

label_denominador = widgets.Label('Denominator :',layout = widgets.Layout(width = '12%'))
denominador = widgets.Text(layout = box_layout.layout)
container_denominador = widgets.HBox(children=[label_denominador,denominador])

container_plant = widgets.VBox(children=[container_numerador,container_denominador])

""" Tab ISA PID Parameters """

def k_box(description,tip,default_value):
    label_kp = widgets.Label(description + ' :',layout = widgets.Layout(width = '4%'))
    box_layout = widgets.Text(layout=widgets.Layout(width = "15%",height = '40px'))
    kp = widgets.BoundedFloatText(
                                value=default_value,
                                min=-1000.0,
                                max=1000.0,
                                step=.05,
                                disabled=False,
                                layout = box_layout.layout
                            )
    space = widgets.Label('',layout = widgets.Layout(width = '3%'))
    tip_kp = widgets.Label(tip,layout = widgets.Layout(width = '20%'))

    return widgets.HBox(children=[label_kp,kp,space,tip_kp])

container_kp = k_box('Kp','Proportional part',1)
container_Td = k_box('Td','Derivative part',0)
container_Ti = k_box('Ti','Integral part',0)

container_ISA_PID = widgets.VBox(children=[container_kp,container_Td,container_Ti])

""" Config parameters """
def config_box(description,tip,Min,Max,default_value,step):
    label_kp = widgets.Label(description + ' :',layout = widgets.Layout(width = '10%'))
    box_layout = widgets.Text(layout=widgets.Layout(width = "15%",height = '40px'))
    kp = widgets.BoundedFloatText(
                                value=default_value,
                                min=Min,
                                max=Max,
                                step=step,
                                disabled=False,
                                layout = box_layout.layout
                            )
    space = widgets.Label('',layout = widgets.Layout(width = '3%'))
    tip_kp = widgets.Label(tip,layout = widgets.Layout(width = '25%'))

    return widgets.HBox(children=[label_kp,kp,space,tip_kp])

container_kpoints = config_box("K point's",'Root locus range of points for k',10,1000,10,1)
container_linspace = config_box("Spacement",'Parameter for linspace of the K points',1,1000,10,1)
container_config = widgets.VBox(children=[container_kpoints,container_linspace])

""" Contents of the GUI """
tab_contents = [container_config,container_plant,container_ISA_PID]

""" Creating the tab element """
tab = widgets.Tab()
tab.children = tab_contents

""" Indexing titles for the tabs """
[tab.set_title(i,tab_titles[i]) for i in range(len(tab_titles))]

""" Displaying """
display(tab)

""" Update function for the animation """
def ISA_PID(GUI_container):
    fig = plt.figure(figsize=(15,10))
    #plt.close()
    ax = fig.add_subplot(211)
    ax2 = fig.add_subplot(212)
    ax2.grid(True)
    ax.grid(True)
    """ Simulation """
    k_range = tab.children[0].children[0].children[1].value
    spacement = tab.children[0].children[1].children[1].value
    # for test purposes
    tab.children[0].children[0].children[1].value = 500
    """ Plant """
    numerador_value = tab.children[1].children[0].children[1].value
    denominador_value = tab.children[1].children[1].children[1].value
    # for test purposes
    tab.children[1].children[0].children[1].value = '0.2570'
    tab.children[1].children[1].children[1].value = '1,0,-6.6370'
    """ ISA PID Parameters """
    K = tab.children[2].children[0].children[1].value
    Td = tab.children[2].children[1].children[1].value
    Ti = tab.children[2].children[2].children[1].value

    if(numerador_value != ''):
        num = np.array([np.float(numero) for numero in numerador_value.split(',')])
    else:
        num = 1
    if(denominador_value != ''):
        den = np.array([np.float(numero) for numero in denominador_value.split(',')])
    else:
        den = 1
    G = tf(num,den)
    print('Transfer Function : ', G)
    """ Simulation part """
    roots_real = []
    roots_img = []
    color_vector = []
    Kcs = np.linspace(0,k_range,spacement)
    s = tf([1,0],1)
    if( (type(np.ones(1)) == type(num)) or (type(np.ones(1)) == type(den)) ):
        for Kc in Kcs:
            if(Ti == 0):
                Gc = Kc*(1 + s*Td)
            else:
                Gc = Kc*(1 + s*Td + 1/(Ti*s))
            closed_loop_tf = feedback(Gc*G,1)
            poles = closed_loop_tf.pole()
            how_many_poles = len(poles)
            counter = 0
            if(how_many_poles > 0):
                for i,pole in enumerate(poles):
                    roots_real.append(poles[i].real)
                    roots_img.append(poles[i].imag)
                    color_vector.append(colors[counter])
                    counter += 1
                    if(counter == how_many_poles): counter = 0
        length = 5
        for i in range(len(roots_real)):
            ax.scatter(roots_real[i],roots_img[i],marker='x',s=30,color=color_vector[i])

        if(K != 0):
            if(Ti == 0):
                Gc = K*(1 + s*Td)
            else:
                Gc = K*(1 + s*Td + 1/(Ti*s))
            closed_loop_tf = feedback(Gc*G,1)
            _poles = closed_loop_tf.pole()
            _zeros = closed_loop_tf.zero()

            [ax.scatter(pole.real,pole.imag,marker='X',s=100,color='black') for pole in _poles]
            [ax.scatter(zero.real,zero.imag,marker='o',s=100,color='darkgreen') for zero in _zeros]
            T, yout = step_response(closed_loop_tf)
            ax2.plot(T,yout)
            total_number_of_poles_and_zeros = len(_poles) + len(_zeros)
            legends = []
            [legends.append(str(round(pole.real,3)) + ' + ' + str(round(pole.imag,3))  + 'j') for pole in _poles]
            [legends.append(str(round(zero.real,3)) + ' + ' + str(round(zero.imag,3))  + 'j') for zero in _zeros]
            ax.legend(legends)

interact(ISA_PID,GUI_container=widgets.ToggleButton(
            description='Simulate!',
            button_style='info', # 'success', 'info', 'warning', 'danger' or ''
            tooltip='Atualizar parêmetros',
            icon='check'
        )
)
```

    Tab(children=(VBox(children=(HBox(children=(Label(value="K point's :", layout=Layout(width='10%')), BoundedFlo…

    interactive(children=(ToggleButton(value=False, button_style='info', description='Simulate!', icon='check', to…

    <function __main__.ISA_PID(GUI_container)>


# Analisando o erro numérico

Seja $z$ a raiz de $f$ para a qual o método de Newton converge.
Temos $x_n$ e $x_{n+1}$, e queremos ver como o erro (ou seja, a distância até $z$) evolui com 1 passo.

## Relações que possuímos:

- $x_{n+1} = x_n - f(x_n)/f'(x_n)$
- Taylor: $f(x_n) = f(z) + f'(z)(x_n - z) + \frac{f''(\xi)}{2}(x_n - z)^2$

Poderíamos usar esta expansão de Taylor, mas então teríamos que analisar $f'(z)$ e $f'(x_n)$.
O que pode ser feito usando Taylor (de novo) para a primeira derivada, mas começam a aparecer muitos "pontos intermediários".

Vamos usar uma outra expansão de Taylor:
- $f(z) = f(x_n) + f'(x_n)(z - x_n) + \frac{f''(\xi)}{2}(z - x_n)^2$ (o $\xi$ desta fórmula é diferente do da anterior!)

### Comentário

Aqui vemos a grande utilidade da técnica "dar nome aos burros" ao resolver um problema.
Por mais que $z$ seja desconhecido, podemos incluí-lo em nossas fórmulas.
Além disso, por mais que seja desconhecido, podemos também usar alguma propriedade -
no nosso caso, $f(z) = 0$ - nas contas.

## Continuando as contas

Da fórmula de Taylor, temos:
$$\begin{align}
0 & = f(x_n) + f'(x_n)(z - x_n) + \frac{f''(\xi)}{2}(z - x_n)^2 \\
- f(x_n) & = f'(x_n)(z - x_n) +  \frac{f''(\xi)}{2}(z - x_n)^2 \\
- \frac{f(x_n)}{f'(x_n)} & = (z - x_n) + \frac{f''(\xi)}{2 f'(x_n)}(z - x_n)^2.
\end{align}$$
Portanto, $x_{n+1} = x_n + (z - x_n) + \frac{f''(\xi)}{2 f'(x_n)}(z - x_n)^2 = \frac{f''(\xi)}{2 f'(x_n)}(z - x_n)^2$.

Isso mostra que o erro $\varepsilon_{n+1} = x_{n+1} - z$ será, em função do erro anterior $\varepsilon_n$:
$$\varepsilon_{n+1} = \frac{f''(\xi)}{2 f'(x_n)} \varepsilon_n^2,$$
que é a famosa propriedade de "elevar o erro anterior ao quadrado" (a menos de um fator).

### Exercício

Suponha que o quociente das derivadas seja sempre menor do que 2,
e suponha que o erro inicial é 1/3.
Em quantas etapas ele será menor do que `1e-16`?

```python
def tempo_para_convergir(erro, max_quot, alvo):
    """Retorna o número de passos necessários para que o `erro` fique menor do que `alvo`,
    se o quociente das derivadas do método de Newton for sempre menor do que `max_quot`. """
    i = 0
    ### Resposta aqui

    return i
```

```python
tempo_para_convergir(1/3, 2, 1e-16)
```

    6

Como evolui o erro?

```python
def tempo_para_convergir_lista(erro, max_quot, alvo):
    """Retorna a lista com os erros intermediários produzidos pelo método de Newton, cf `tempo_para_convergir`. """
    ### Resposta aqui

```

```python
tempo_para_convergir_lista(1/3, 2, 1e-16)
```

    [0.3333333333333333,
     0.1111111111111111,
     0.012345679012345678,
     0.00015241579027587256,
     2.3230573125418767e-08,
     5.396595277354286e-16,
     2.9123240587562585e-31]

O que acontece se apenas garantimos que o fator é menor do que 4?
E se for 10?

```python
tempo_para_convergir_lista(1/3, 4, 1e-16)
```

    [0.3333333333333333,
     0.2222222222222222,
     0.09876543209876543,
     0.019509221155311687,
     0.0007612194201737221,
     1.1589100112992354e-06,
     2.686144828579188e-12,
     1.443074808020543e-23]

```python
tempo_para_convergir_lista(1/3, 10, 1e-16)
```

    []

O que fazer para ainda ter convergência caso só seja possível limitar o quociente das derivadas por 10??

## Mas e o $\xi$?

A nossa fórmula já explica uma boa parte da rapidez de convergência do método de Newton,
mas ela ainda contém vários elementos desconhecidos, como o $\xi$, $f'(x_n)$,
que dependem da função, do ponto onde estamos, etc.
Para transformar esta conta exata, mas com termos variáveis,
numa estimativa mais independente, o que vamos fazer é **supor** algo sobre a função $f$.

Por exemplo, podemos supor que a segunda derivada é _limitada_ perto de $z$.
Assim, quando estivermos "perto" da raiz, podemos supor (como fizemos acima) que uma parte do fator não é muito grande.
Novamente, podemos supor que a primeira derivada não é **pequena demais** perto de $z$.
Assim, o quociente estará certamente limitado, e, se estivermos "perto o suficiente" da raiz,
vamos convergir bem rápido.

### Exercício

Resolva a recorrência $x_{n+1} = Cx_n^2$, com $x_0 = \delta$.
Dica: escreva uns 5 termos, e veja o padrão em $\delta$, $C$ e $n$!


# Homework 1 Problem 2

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
```

```python
from scipy.constants import e, m_e, k, pi, epsilon_0, c

temperature = 273
concentration = 8.47 * 10 ** 28
```

```python
def plasma_frequency(concentration):
    return np.sqrt((concentration * e ** 2) / (m_e * epsilon_0))
```

```python
omega_p = plasma_frequency(concentration)
omega_p
```

    1.6418490871041346e+16

$\omega_p = 1.64185 \cdot 10^{16} s^{-1}$

```python
def tau(density, resistivity):
    return m_e / (e ** 2 * density * resistivity)

tau_ = tau(density = 8.47 * 10 ** (28), resistivity = 1.56 * 10 ** (-8))
```

```python
tau_
```

    2.685716701399944e-14

```python
tau_ * omega_p
```

    440.9541514413826

$\tau \cdot \omega_p \approx 500 >> 1$

In Drude model, if $\omega_p \tau >> 1$, dielectric constant is given by:

\begin{equation}
\epsilon(\omega) = 1 - \frac{\omega_p^2}{\omega^2}
\end{equation}

For frequencies $\omega >> \omega_p$, we have $\epsilon(\omega) \approx 1$, therefore metal is transparent.

Equivalently, for $\lambda << \lambda_p = \frac{2 \pi c}{\omega_p}$, metal is transparent

```python
2 * pi * c / omega_p
```

    1.1472744858854266e-07

Therefore, limiting values are:

$\omega_p = 1.64185 \cdot 10^{16} s^{-1}$

$\lambda_p = 1.14727 \cdot 10^{-7} m$

```python

```


# Euler Equation

## Conservation Form
The one dimensional version of the Euler equation for gas dynamics:
\begin{equation}
  \mathbf{u}_t + \mathbf{f}(\mathbf{u})_x = 0,\,\, t \geq 0, x \in \mathbb{R},
\end{equation}
where
\begin{equation}
  \mathbf{u} = \left(\begin{array}{c}\rho\\m\\E\end{array}\right),\,\, \mathbf{f}(\mathbf{u}) = \left(\begin{array}{c}m\\\rho v^2 + p\\(E+p)v\end{array}\right),
\end{equation}
with
\begin{equation*}
  m = \rho v, \,\, E = \frac{1}{2}\rho v^2 + \rho e,
\end{equation*}
where $\rho$ is the density, $v$ is the velocity, $m$ is the momentum, $E$ is the total energy, $e$ is the internal, and the pressure $p$ can be obtained from the equation of state.  For instance, for the perfect gas,
\begin{equation*}
  p(\mathbf{u}) = (\gamma -1)\rho e = (\gamma-1)\left(E - \frac{1}{2}\frac{m^2}{\rho}\right)
\end{equation*}
It is easy to calculate that
\begin{equation*}
    \frac{\partial p}{\partial \rho} = \frac{1}{2}(\gamma-1)v^2,\,\, \frac{\partial p}{\partial m} = -(\gamma-1)v,\,\, \frac{\partial p}{\partial E} = \gamma - 1,
\end{equation*}
and the Jacobian of $\mathbf{f}(\mathbf{u})$ is given by
\begin{equation*}
    \mathbf{f}'(\mathbf{u}) = \left(\begin{array}{ccc}0 & 1 & 0 \\ \frac{1}{2}(\gamma-3)v^2 & (3-\gamma)v & \gamma-1 \\ \frac{1}{2}(\gamma-1)v^3-v(E+p)/\rho & (E+p)/\rho - (\gamma-1)v^2 & \gamma v\end{array}\right)
\end{equation*}
To get the eignvalue of $\mathbf{f}'(\mathbf{u})$, we have
\begin{equation*}
    \det(\lambda I - \mathbf{f}') = \lambda^3 -3v\lambda^2 + (3v^2-c^2)\lambda + v(v^2 - c^2)= (\lambda - v)(\lambda - v-c)(\lambda-v+c),
\end{equation*}
where $c = \sqrt{\frac{\gamma p}{\rho}}$.

```python

```


# Predator-Prey Population Differential Equations

In this notebook we will model the interaction of Lynx and Hare populations [1].

```python
## Library
import numpy as np
import math
import matplotlib.pyplot as plt
```

## Predators and prey
The cyclical relationship of Predators and Prey in the wild  means that when there is an abundance of prey, the predators will eat more and their population will increase. On the other hand, if there is a shortage in the number of prey, the number of predators will decrease. One of the of the most famous examples of this was observed by the fur traders  Hudson's Bay Company of Canada. The company collected data about the number of lynx and hare pelts they purchased over a 90 year period. Their data showed the cylical nature of the predator prey relationship.
This kind of behaviour has been observed in many pairs of predators and prey from mammals to fish to beyond.

## Snow Hares and Lynxes
In a balanced eco system the relationship between snow hares (prey) and lynxes (predator) can be described mathematically using a Loketta-Volterra model [1-3]
\begin{equation}
\begin{array}{cl}
\frac{d H}{dt}=a_{Birth} H-b_{Con}LH,\\
\frac{d L}{dt}=-c_{Death}L+d_{Food} L H,\\
\end{array}
\end{equation}
this set of equations models the interaction between snow hares (H) and lynxes (L). The lynes are the predators who eat the snow hares to survive. Where there parameters
* $a_{Birth}$ is the birth rate of the snow hares;
* $b_{Con}$ is the consumation rate of snow hares by lynxes (interaction);
* $c_{Death}$ is the death rate of lynxes if there is no food source;
* $d_{Food}$ is the rate that lynxes eat the snow hares to survive (interaction).

```python
a_Birth=0.4
b_Con=0.1
c_Death=0.09
d_Food=0.01

```

## Discrete Domain
To numiercally approximate the solution we discretise the continuous domain using a step size, $h=1$,
which gives
\begin{equation}
t_i=0+i h,
\end{equation}
for $i=0,...10$.
The figure below illustrate the discrete domain.

```python
h=0.001
t=np.arange(1850,1925+h/2,h)
fig = plt.figure(1,figsize=(5,4))
ax = fig.add_subplot(1,1,1)
ax.plot(t,0*t,'o:',color='k')
ax.set_xlabel('Time (yrs)')
plt.title('Discrete Time Domain')
plt.show()
```

```python
N=len(t) # Number of Steps
H=np.zeros(N) # Numerical Solution Hares
H[0]=6 # Intial Hare population
L=np.zeros(N) # Numerical Solution Lynxes
L[0]=2# Initial Lynx population

```

We numerically approximate the differential equation using Eulers method to give the difference equation
\begin{equation}
H[i+1] = H[i]+ h*\big(a_{Birth}*H[i]-b_{Con}*H[i]*L[i] \big)
\end{equation}

\begin{equation}
L[i+1] = L[i]+ h*\big(-c_{Death}*H[i]+d_{Food}*H[i]*L[i] \big)
\end{equation}
where $h$ is the stepsize and  $H[i]$ and $L[i]$ is the numerical approximation of the hare $H(t_i)$ and Lynx $L(t_i)$ at time $t_i$, for $i=0,...,N$

```python
for i in range (0,N-1):
    H[i+1]=H[i]+h*(a_Birth*H[i]-b_Con*H[i]*L[i])# INSERT EQUATION HERE
    L[i+1]=L[i]+h*(-c_Death*L[i]+d_Food*H[i]*L[i])# INSERT EQUATION HERE

```

## Results
The figure below plots the numerical approximation of the Hare population (green) and the Lynx poulation (red) as a function of time in years between 1850 and 1925.
The figure illustrates the oscilatory nature of the Hare and Lynx populations.
The hare population fluctuates from close to 0 to 800 ten of thousands.
The lynx population fluctuates from close to 0 to 120 ten of thousands.
The peak hare population preceeds the lynx population by about half a year.
There is a gradual increase of peak populations over time.
The model is similar to the observed data by the Hudson Bay Company.

```python
## Plotting Figure
fig = plt.figure(1,figsize=(10,4))

ax = fig.add_subplot(1,1,1)

ax.plot(t,H,':',color='green',label='Hares')
ax.plot(t,L,':',color='red',label='Lynxes')

ax.legend()
ax.set_xlabel('Time (yrs)')
ax.set_ylabel('Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('The figure below plots the numerical approximation \n of the Hare population (green)  \nand the Lynx poulation (red) as a function of time in years between 1850 and 1925.')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.show()

```

## Phase Plane Plot
The phase plane plot below illustrates the population of Hares as a function of Lynxes. This removes time from out plot.
The rate of change of hares as function of the rate of lynxes is written as
\begin{equation}
\frac{d H}{dL}=\frac{a_{Birth} H-b_{Con}LH}{-c_{Death}L+d_{Food} L H}.
\end{equation}

The plot below shows a stable cyclical relationship between hares and lynxes.
It looks a bit like an orbit of a moon.

```python
## Plotting Figure
fig = plt.figure(1,figsize=(5,5))

ax = fig.add_subplot(1,1,1)

ax.plot(H,L,'-',color='black')

ax.set_ylabel('Lynx population (tens of thousands)')
ax.set_xlabel('Hare Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.ylim(0.9*np.min(L),1.1*np.max(L))
plt.xlim(0.9*np.min(H),1.1*np.max(H))

plt.show()
```

# References

[1] Stover, Christopher and Weisstein, Eric W. "Population Growth." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/PopulationGrowth.html

[2]  Vito Volterra.  Fluctuations in the abundance of a species considered mathematically. Nature,118:558–560,1926.

[3] Alfred J Lotka.   Analytical note on certain rhythmic  relations inorganic  systems.Proceedings of the National Academy of Sciences,6(7):410–415,1920.