# Week 4 January 25-29: Metropolis Algoritm and Markov Chains, Importance Sampling, Fokker-Planck and Langevin equations

**Morten Hjorth-Jensen  Email morten.hjorth-jensen@fys.uio.no**, Department of Physics and Center fo Computing in Science Education, University of Oslo, Oslo, Norway and Department of Physics and Astronomy and Facility for Rare Ion Beams, Michigan State University, East Lansing, Michigan, USA

Date: **Jan 28, 2021**

Copyright 1999-2021, Morten Hjorth-Jensen  Email morten.hjorth-jensen@fys.uio.no. Released under CC Attribution-NonCommercial 4.0 license

## Overview of week 4, January 25-29
**Topics.**

* Metropolis-Hastings sampling and Importance Sampling

* Fokker-Planck and Langevin equations

**Teaching Material, videos and written material.**

* Overview video on [Metropolis algoritm](https://www.youtube.com/watch?v=h1NOS_wxgGg&ab_channel=JeffPicton)

* Lecture notes here.

## Basics of the Metropolis Algorithm

The Metropolis et al.
algorithm was invented by Metropolis et. a
and is often simply called the Metropolis algorithm.
It is a method to sample a normalized probability
distribution by a stochastic process. We define ${\cal P}_i^{(n)}$ to
be the probability for finding the system in the state $i$ at step $n$.
The algorithm is then

## The basic of the Metropolis Algorithm

* Sample a possible new state $j$ with some probability $T_{i\rightarrow j}$.

* Accept the new state $j$ with probability $A_{i \rightarrow j}$ and use it as the next sample.

* With probability $1-A_{i\rightarrow j}$ the move is rejected and the original state $i$ is used again as a sample.

We wish to derive the required properties of $T$ and $A$ such that
${\cal P}_i^{(n\rightarrow \infty)} \rightarrow p_i$ so that starting
from any distribution, the method converges to the correct distribution.
Note that the description here is for a discrete probability distribution.
Replacing probabilities $p_i$ with expressions like $p(x_i)dx_i$ will
take all of these over to the corresponding continuum expressions.

## More on the Metropolis

The dynamical equation for ${\cal P}_i^{(n)}$ can be written directly from
the description above. The probability of being in the state $i$ at step $n$
is given by the probability of being in any state $j$ at the previous step,
and making an accepted transition to $i$ added to the probability of
being in the state $i$, making a transition to any state $j$ and
rejecting the move:

$$
\begin{equation}
\label{eq:eq1} \tag{1}
{\cal P}^{(n)}_i = \sum_j \left [
{\cal P}^{(n-1)}_jT_{j\rightarrow i} A_{j\rightarrow i}
+{\cal P}^{(n-1)}_iT_{i\rightarrow j}\left ( 1- A_{i\rightarrow j} \right)
\right ] \,.
\end{equation}
$$

## Metropolis algorithm, setting it up
Since the probability of making some transition must be 1,
$\sum_j T_{i\rightarrow j} = 1$, and Eq. ([1](#eq:eq1)) becomes

$$
\begin{equation}
{\cal P}^{(n)}_i = {\cal P}^{(n-1)}_i +
 \sum_j \left [
{\cal P}^{(n-1)}_jT_{j\rightarrow i} A_{j\rightarrow i}
-{\cal P}^{(n-1)}_iT_{i\rightarrow j}A_{i\rightarrow j}
\right ] \,.
\label{_auto1} \tag{2}
\end{equation}
$$

## Metropolis continues

For large $n$ we require that ${\cal P}^{(n\rightarrow \infty)}_i = p_i$,
the desired probability distribution. Taking this limit, gives the
balance requirement

$$
\begin{equation}
\sum_j \left [p_jT_{j\rightarrow i} A_{j\rightarrow i}-p_iT_{i\rightarrow j}A_{i\rightarrow j}
\right ] = 0,
\label{_auto2} \tag{3}
\end{equation}
$$

## Detailed Balance

The balance requirement is very weak. Typically the much stronger detailed
balance requirement is enforced, that is rather than the sum being
set to zero, we set each term separately to zero and use this
to determine the acceptance probabilities. Rearranging, the result is

$$
\begin{equation}
\frac{ A_{j\rightarrow i}}{A_{i\rightarrow j}}
= \frac{p_iT_{i\rightarrow j}}{ p_jT_{j\rightarrow i}} \,.
\label{_auto3} \tag{4}
\end{equation}
$$

## More on Detailed Balance

The Metropolis choice is to maximize the $A$ values, that is

$$
\begin{equation}
A_{j \rightarrow i} = \min \left ( 1,
\frac{p_iT_{i\rightarrow j}}{ p_jT_{j\rightarrow i}}\right ).
\label{_auto4} \tag{5}
\end{equation}
$$

Other choices are possible, but they all correspond to multilplying
$A_{i\rightarrow j}$ and $A_{j\rightarrow i}$ by the same constant
smaller than unity.\footnote{The penalty function method uses just such
a factor to compensate for $p_i$ that are evaluated stochastically
and are therefore noisy.}

Having chosen the acceptance probabilities, we have guaranteed that
if the  ${\cal P}_i^{(n)}$ has equilibrated, that is if it is equal to $p_i$,
it will remain equilibrated. Next we need to find the circumstances for
convergence to equilibrium.

## Dynamical Equation

The dynamical equation can be written as

$$
\begin{equation}
{\cal P}^{(n)}_i = \sum_j M_{ij}{\cal P}^{(n-1)}_j
\label{_auto5} \tag{6}
\end{equation}
$$

with the matrix $M$ given by

$$
\begin{equation}
M_{ij} = \delta_{ij}\left [ 1 -\sum_k T_{i\rightarrow k} A_{i \rightarrow k}
\right ] + T_{j\rightarrow i} A_{j\rightarrow i} \,.
\label{_auto6} \tag{7}
\end{equation}
$$

Summing over $i$ shows that $\sum_i M_{ij} = 1$, and since
$\sum_k T_{i\rightarrow k} = 1$, and $A_{i \rightarrow k} \leq 1$, the
elements of the matrix satisfy $M_{ij} \geq 0$. The matrix $M$ is therefore
a stochastic matrix.

## Interpreting the Metropolis Algorithm

The Metropolis method is simply the power method for computing the
right eigenvector of $M$ with the largest magnitude eigenvalue.
By construction, the correct probability distribution is a right eigenvector
with eigenvalue 1. Therefore, for the Metropolis method to converge
to this result, we must show that $M$ has only one eigenvalue with this
magnitude, and all other eigenvalues are smaller.

Even a defective matrix has at least one left and right eigenvector for
each eigenvalue. An example of a defective matrix is

$$
\begin{bmatrix}
0 & 1\\
0 & 0 \\
\end{bmatrix},
$$

with two zero eigenvalues, only one right eigenvector

$$
\begin{bmatrix}
1 \\
0\\
\end{bmatrix}
$$

and only one left eigenvector $(0\ 1)$.

## Gershgorin bounds and Metropolis

The Gershgorin bounds for the eigenvalues can be derived by multiplying on
the left with the eigenvector with the maximum and minimum eigenvalues,

$$
\sum_i \psi^{\rm max}_i M_{ij} = \lambda_{\rm max}  \psi^{\rm max}_j
\nonumber
$$

$$
\begin{equation}
\sum_i \psi^{\rm min}_i M_{ij} = \lambda_{\rm min}  \psi^{\rm min}_j
\label{_auto7} \tag{8}
\end{equation}
$$

## Normalizing the Eigenvectors

Next we choose the normalization of these eigenvectors so that the
largest element (or one of the equally largest elements)
has value 1. Let's call this element $k$, and
we can therefore bound the magnitude of the other elements to be less
than or equal to 1.
This leads to the inequalities, using the property that $M_{ij}\geq 0$,

$$
\begin{eqnarray}
\sum_i M_{ik} \leq \lambda_{\rm max}
\nonumber\\
M_{kk}-\sum_{i \neq k} M_{ik} \geq \lambda_{\rm min}
\end{eqnarray}
$$

where the equality from the maximum
will occur only if the eigenvector takes the value 1 for all values of
$i$ where $M_{ik} \neq 0$, and the equality for the minimum will
occur only if the eigenvector takes the value -1 for all values of $i\neq k$
where $M_{ik} \neq 0$.

## More Metropolis analysis

That the maximum eigenvalue is 1 follows immediately from the property
that $\sum_i M_{ik} = 1$. Similarly the minimum eigenvalue can be -1,
but only if $M_{kk} = 0$ and the magnitude of all the other elements
$\psi_i^{\rm min}$ of
the eigenvector that multiply nonzero elements $M_{ik}$ are -1.

Let's first see what the properties of $M$ must be
to eliminate any -1 eigenvalues.
To have a -1 eigenvalue, the left eigenvector must contain only $\pm 1$
and $0$ values. Taking in turn each $\pm 1$ value as the maximum, so that
it corresponds to the index $k$, the nonzero $M_{ik}$ values must
correspond to $i$ index values of the eigenvector which have opposite
sign elements. That is, the $M$ matrix must break up into sets of
states that always make transitions from set A to set B ... back to set A.
In particular, there can be no rejections of these moves in the cycle
since the -1 eigenvalue requires $M_{kk}=0$. To guarantee no eigenvalues
with eigenvalue -1, we simply have to make sure that there are no
cycles among states. Notice that this is generally trivial since such
cycles cannot have any rejections at any stage. An example of such
a cycle is sampling a noninteracting Ising spin. If the transition is
taken to flip the spin, and the energy difference is zero, the Boltzmann
factor will not change and the move will always be accepted. The system
will simply flip from up to down to up to down ad infinitum. Including
a rejection probability or using a heat bath algorithm
immediately fixes the problem.

## Final Considerations I

Next we need to make sure that there is only one left eigenvector
with eigenvalue 1. To get an eigenvalue 1, the left eigenvector must be
constructed from only ones and zeroes. It is straightforward to
see that a vector made up of
ones and zeroes can only be an eigenvector with eigenvalue 1 if the
matrix element $M_{ij} = 0$ for all cases where $\psi_i \neq \psi_j$.
That is we can choose an index $i$ and take $\psi_i = 1$.
We require all elements $\psi_j$ where $M_{ij} \neq 0$ to also have
the value $1$. Continuing we then require all elements $\psi_\ell$ $M_{j\ell}$
to have value $1$. Only if the matrix $M$ can be put into block diagonal
form can there be more than one choice for the left eigenvector with
eigenvalue 1. We therefore require that the transition matrix not
be in block diagonal form. This simply means that we must choose
the transition probability so that we can get from any allowed state
to any other in a series of transitions.

## Final Considerations II

Finally, we note that for a defective matrix, with more eigenvalues
than independent eigenvectors for eigenvalue 1,
the left and right
eigenvectors of eigenvalue 1 would be orthogonal.
Here the left eigenvector is all 1
except for states that can never be reached, and the right eigenvector
is $p_i > 0$ except for states that give zero probability. We already
require that we can reach
all states that contribute to $p_i$. Therefore the left and right
eigenvectors with eigenvalue 1 do not correspond to a defective sector
of the matrix and they are unique. The Metropolis algorithm therefore
converges exponentially to the desired distribution.

## Final Considerations III

The requirements for the transition $T_{i \rightarrow j}$ are
* A series of transitions must let us to get from any allowed state to any other by a finite series of transitions.

* The transitions cannot be grouped into sets of states, A, B, C ,... such that transitions from $A$ go to $B$, $B$ to $C$ etc and finally back to $A$. With condition (a) satisfied, this condition will always be satisfied if either $T_{i \rightarrow i} \neq 0$ or there are some rejected moves.

## Importance sampling
We need to replace the brute force
Metropolis algorithm with a walk in coordinate space biased by the trial wave function.
This approach is based on the Fokker-Planck equation and the Langevin equation for generating a trajectory in coordinate space.  The link between the Fokker-Planck equation and the Langevin equations are explained, only partly, in the slides below.
An excellent reference on topics like Brownian motion, Markov chains, the Fokker-Planck equation and the Langevin equation is the text by  [Van Kampen](http://www.elsevier.com/books/stochastic-processes-in-physics-and-chemistry/van-kampen/978-0-444-52965-7)
Here we will focus first on the implementation part first.

For a diffusion process characterized by a time-dependent probability density $P(x,t)$ in one dimension the Fokker-Planck
equation reads (for one particle /walker)

$$
\frac{\partial P}{\partial t} = D\frac{\partial }{\partial x}\left(\frac{\partial }{\partial x} -F\right)P(x,t),
$$

where $F$ is a drift term and $D$ is the diffusion coefficient.

## Importance sampling
The new positions in coordinate space are given as the solutions of the Langevin equation using Euler's method, namely,
we go from the Langevin equation

$$
\frac{\partial x(t)}{\partial t} = DF(x(t)) +\eta,
$$

with $\eta$ a random variable,
yielding a new position

$$
y = x+DF(x)\Delta t +\xi\sqrt{\Delta t},
$$

where $\xi$ is gaussian random variable and $\Delta t$ is a chosen time step.
The quantity $D$ is, in atomic units, equal to $1/2$ and comes from the factor $1/2$ in the kinetic energy operator. Note that $\Delta t$ is to be viewed as a parameter. Values of $\Delta t \in [0.001,0.01]$ yield in general rather stable values of the ground state energy.

## Importance sampling
The process of isotropic diffusion characterized by a time-dependent probability density $P(\mathbf{x},t)$ obeys (as an approximation) the so-called Fokker-Planck equation

$$
\frac{\partial P}{\partial t} = \sum_i D\frac{\partial }{\partial \mathbf{x_i}}\left(\frac{\partial }{\partial \mathbf{x_i}} -\mathbf{F_i}\right)P(\mathbf{x},t),
$$

where $\mathbf{F_i}$ is the $i^{th}$ component of the drift term (drift velocity) caused by an external potential, and $D$ is the diffusion coefficient. The convergence to a stationary probability density can be obtained by setting the left hand side to zero. The resulting equation will be satisfied if and only if all the terms of the sum are equal zero,

$$
\frac{\partial^2 P}{\partial {\mathbf{x_i}^2}} = P\frac{\partial}{\partial {\mathbf{x_i}}}\mathbf{F_i} + \mathbf{F_i}\frac{\partial}{\partial {\mathbf{x_i}}}P.
$$

## Importance sampling
The drift vector should be of the form $\mathbf{F} = g(\mathbf{x}) \frac{\partial P}{\partial \mathbf{x}}$. Then,

$$
\frac{\partial^2 P}{\partial {\mathbf{x_i}^2}} = P\frac{\partial g}{\partial P}\left( \frac{\partial P}{\partial {\mathbf{x}_i}}  \right)^2 + P g \frac{\partial ^2 P}{\partial {\mathbf{x}_i^2}}  + g \left( \frac{\partial P}{\partial {\mathbf{x}_i}}  \right)^2.
$$

The condition of stationary density means that the left hand side equals zero. In other words, the terms containing first and second derivatives have to cancel each other. It is possible only if $g = \frac{1}{P}$, which yields

$$
\mathbf{F} = 2\frac{1}{\Psi_T}\nabla\Psi_T,
$$

which is known as the so-called *quantum force*. This term is responsible for pushing the walker towards regions of configuration space where the trial wave function is large, increasing the efficiency of the simulation in contrast to the Metropolis algorithm where the walker has the same probability of moving in every direction.

## Importance sampling
The Fokker-Planck equation yields a (the solution to the equation) transition probability given by the Green's function

$$
G(y,x,\Delta t) = \frac{1}{(4\pi D\Delta t)^{3N/2}} \exp{\left(-(y-x-D\Delta t F(x))^2/4D\Delta t\right)}
$$

which in turn means that our brute force Metropolis algorithm

$$
A(y,x) = \mathrm{min}(1,q(y,x))),
$$

with $q(y,x) = |\Psi_T(y)|^2/|\Psi_T(x)|^2$ is now replaced by the [Metropolis-Hastings algorithm](http://scitation.aip.org/content/aip/journal/jcp/21/6/10.1063/1.1699114) as well as [Hasting's article](http://biomet.oxfordjournals.org/content/57/1/97.abstract),

$$
q(y,x) = \frac{G(x,y,\Delta t)|\Psi_T(y)|^2}{G(y,x,\Delta t)|\Psi_T(x)|^2}
$$

## Importance sampling, program elements
The full code is [this link](https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/pub/vmc/programs/c%2B%2B). Here we include only the parts pertaining to the computation of the quantum force and the Metropolis update. The program is a modfication of our previous c++ program discussed previously. Here we display only the part from the *vmcsolver.cpp*  file.  Note the usage of the function *GaussianDeviate*.

        void VMCSolver::runMonteCarloIntegration()
        {
          rOld = zeros<mat>(nParticles, nDimensions);
          rNew = zeros<mat>(nParticles, nDimensions);
          QForceOld = zeros<mat>(nParticles, nDimensions);
          QForceNew = zeros<mat>(nParticles, nDimensions);

          double waveFunctionOld = 0;
          double waveFunctionNew = 0;

          double energySum = 0;
          double energySquaredSum = 0;

          double deltaE;

          // initial trial positions
          for(int i = 0; i < nParticles; i++) {
            for(int j = 0; j < nDimensions; j++) {
              rOld(i,j) = GaussianDeviate(&idum)*sqrt(timestep);
            }
          }
          rNew = rOld;

## Importance sampling, program elements

          for(int cycle = 0; cycle < nCycles; cycle++) {

            // Store the current value of the wave function
            waveFunctionOld = waveFunction(rOld);
            QuantumForce(rOld, QForceOld); QForceOld = QForceOld*h/waveFunctionOld;
            // New position to test
            for(int i = 0; i < nParticles; i++) {
              for(int j = 0; j < nDimensions; j++) {
        	rNew(i,j) = rOld(i,j) + GaussianDeviate(&idum)*sqrt(timestep)+QForceOld(i,j)*timestep*D;
              }
              //  for the other particles we need to set the position to the old position since
              //  we move only one particle at the time
              for (int k = 0; k < nParticles; k++) {
        	if ( k != i) {
        	  for (int j=0; j < nDimensions; j++) {
        	    rNew(k,j) = rOld(k,j);
        	  }
        	}
              }

## Importance sampling, program elements

          // loop over Monte Carlo cycles
              // Recalculate the value of the wave function and the quantum force
              waveFunctionNew = waveFunction(rNew);
              QuantumForce(rNew,QForceNew) = QForceNew*h/waveFunctionNew;
              //  we compute the log of the ratio of the greens functions to be used in the
              //  Metropolis-Hastings algorithm
              GreensFunction = 0.0;
              for (int j=0; j < nDimensions; j++) {
        	GreensFunction += 0.5*(QForceOld(i,j)+QForceNew(i,j))*
        	  (D*timestep*0.5*(QForceOld(i,j)-QForceNew(i,j))-rNew(i,j)+rOld(i,j));
              }
              GreensFunction = exp(GreensFunction);

              // The Metropolis test is performed by moving one particle at the time
              if(ran2(&idum) <= GreensFunction*(waveFunctionNew*waveFunctionNew) / (waveFunctionOld*waveFunctionOld)) {
        	for(int j = 0; j < nDimensions; j++) {
        	  rOld(i,j) = rNew(i,j);
        	  QForceOld(i,j) = QForceNew(i,j);
        	  waveFunctionOld = waveFunctionNew;
        	}
              } else {
        	for(int j = 0; j < nDimensions; j++) {
        	  rNew(i,j) = rOld(i,j);
        	  QForceNew(i,j) = QForceOld(i,j);
        	}
              }

## Importance sampling, program elements
**Note numerical derivatives.**

        double VMCSolver::QuantumForce(const mat &r, mat &QForce)
        {
            mat rPlus = zeros<mat>(nParticles, nDimensions);
            mat rMinus = zeros<mat>(nParticles, nDimensions);
            rPlus = rMinus = r;
            double waveFunctionMinus = 0;
            double waveFunctionPlus = 0;
            double waveFunctionCurrent = waveFunction(r);

            // Kinetic energy

            double kineticEnergy = 0;
            for(int i = 0; i < nParticles; i++) {
                for(int j = 0; j < nDimensions; j++) {
                    rPlus(i,j) += h;
                    rMinus(i,j) -= h;
                    waveFunctionMinus = waveFunction(rMinus);
                    waveFunctionPlus = waveFunction(rPlus);
                    QForce(i,j) =  (waveFunctionPlus-waveFunctionMinus);
                    rPlus(i,j) = r(i,j);
                    rMinus(i,j) = r(i,j);
                }
            }
        }

## Importance sampling, program elements
The general derivative formula of the Jastrow factor is (the subscript $C$ stands for Correlation)

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{\partial g_{ik}}{\partial x_k}
+
\sum_{i=k+1}^{N}\frac{\partial g_{ki}}{\partial x_k}
$$

However,
with our written in way which can be reused later as

$$
\Psi_C=\prod_{i< j}g(r_{ij})= \exp{\left\{\sum_{i<j}f(r_{ij})\right\}},
$$

the gradient needed for the quantum force and local energy is easy to compute.
The function $f(r_{ij})$ will depends on the system under study. In the equations below we will keep this general form.

## Importance sampling, program elements
In the Metropolis/Hasting algorithm, the *acceptance ratio* determines the probability for a particle  to be accepted at a new position. The ratio of the trial wave functions evaluated at the new and current positions is given by ($OB$ for the onebody  part)

$$
R \equiv \frac{\Psi_{T}^{new}}{\Psi_{T}^{old}} =
\frac{\Psi_{OB}^{new}}{\Psi_{OB}^{old}}\frac{\Psi_{C}^{new}}{\Psi_{C}^{old}}
$$

Here $\Psi_{OB}$ is our onebody part (Slater determinant or product of boson single-particle states)  while $\Psi_{C}$ is our correlation function, or Jastrow factor.
We need to optimize the $\nabla \Psi_T / \Psi_T$ ratio and the second derivative as well, that is
the $\mathbf{\nabla}^2 \Psi_T/\Psi_T$ ratio. The first is needed when we compute the so-called quantum force in importance sampling.
The second is needed when we compute the kinetic energy term of the local energy.

$$
\frac{\mathbf{\mathbf{\nabla}}  \Psi}{\Psi}  = \frac{\mathbf{\nabla}  (\Psi_{OB} \, \Psi_{C})}{\Psi_{OB} \, \Psi_{C}}  =  \frac{ \Psi_C \mathbf{\nabla}  \Psi_{OB} + \Psi_{OB} \mathbf{\nabla}  \Psi_{C}}{\Psi_{OB} \Psi_{C}} = \frac{\mathbf{\nabla}  \Psi_{OB}}{\Psi_{OB}} + \frac{\mathbf{\nabla}   \Psi_C}{ \Psi_C}
$$

## Importance sampling
The expectation value of the kinetic energy expressed in atomic units for electron $i$ is

2
6

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

$$
\hat{K}_i = -\frac{1}{2}\frac{\mathbf{\nabla}_{i}^{2} \Psi}{\Psi}.
$$

## Importance sampling
The second derivative which enters the definition of the local energy is

$$
\frac{\mathbf{\nabla}^2 \Psi}{\Psi}=\frac{\mathbf{\nabla}^2 \Psi_{OB}}{\Psi_{OB}} + \frac{\mathbf{\nabla}^2  \Psi_C}{ \Psi_C} + 2 \frac{\mathbf{\nabla}  \Psi_{OB}}{\Psi_{OB}}\cdot\frac{\mathbf{\nabla}   \Psi_C}{ \Psi_C}
$$

We discuss here how to calculate these quantities in an optimal way,

## Importance sampling
We have defined the correlated function as

$$
\Psi_C=\prod_{i< j}g(r_{ij})=\prod_{i< j}^Ng(r_{ij})= \prod_{i=1}^N\prod_{j=i+1}^Ng(r_{ij}),
$$

with
$r_{ij}=|\mathbf{r}_i-\mathbf{r}_j|=\sqrt{(x_i-x_j)^2+(y_i-y_j)^2+(z_i-z_j)^2}$ in three dimensions or
$r_{ij}=|\mathbf{r}_i-\mathbf{r}_j|=\sqrt{(x_i-x_j)^2+(y_i-y_j)^2}$ if we work with two-dimensional systems.

In our particular case we have

$$
\Psi_C=\prod_{i< j}g(r_{ij})=\exp{\left\{\sum_{i<j}f(r_{ij})\right\}}.
$$

## Importance sampling
The total number of different relative distances $r_{ij}$ is $N(N-1)/2$. In a matrix storage format, the relative distances  form a strictly upper triangular matrix

$$
\mathbf{r} \equiv \begin{pmatrix}
  0 & r_{1,2} & r_{1,3} & \cdots & r_{1,N} \\
  \vdots & 0       & r_{2,3} & \cdots & r_{2,N} \\
  \vdots & \vdots  & 0  & \ddots & \vdots  \\
  \vdots & \vdots  & \vdots  & \ddots  & r_{N-1,N} \\
  0 & 0  & 0  & \cdots  & 0
 \end{pmatrix}.
$$

This applies to  $\mathbf{g} = \mathbf{g}(r_{ij})$ as well.

In our algorithm we will move one particle  at the time, say the $kth$-particle.  This sampling will be seen to be particularly efficient when we are going to compute a Slater determinant.

## Importance sampling
We have that the ratio between Jastrow factors $R_C$ is given by

$$
R_{C} = \frac{\Psi_{C}^\mathrm{new}}{\Psi_{C}^\mathrm{cur}} =
\prod_{i=1}^{k-1}\frac{g_{ik}^\mathrm{new}}{g_{ik}^\mathrm{cur}}
\prod_{i=k+1}^{N}\frac{ g_{ki}^\mathrm{new}} {g_{ki}^\mathrm{cur}}.
$$

For the Pade-Jastrow form

$$
R_{C} = \frac{\Psi_{C}^\mathrm{new}}{\Psi_{C}^\mathrm{cur}} =
\frac{\exp{U_{new}}}{\exp{U_{cur}}} = \exp{\Delta U},
$$

where

$$
\Delta U =
\sum_{i=1}^{k-1}\big(f_{ik}^\mathrm{new}-f_{ik}^\mathrm{cur}\big)
+
\sum_{i=k+1}^{N}\big(f_{ki}^\mathrm{new}-f_{ki}^\mathrm{cur}\big)
$$

## Importance sampling
One needs to develop a special algorithm
that runs only through the elements of the upper triangular
matrix $\mathbf{g}$ and have $k$ as an index.

The expression to be derived in the following is of interest when computing the quantum force and the kinetic energy. It has the form

$$
\frac{\mathbf{\nabla}_i\Psi_C}{\Psi_C} = \frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_i},
$$

for all dimensions and with $i$ running over all particles.

## Importance sampling
For the first derivative only $N-1$ terms survive the ratio because the $g$-terms that are not differentiated cancel with their corresponding ones in the denominator. Then,

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{1}{g_{ik}}\frac{\partial g_{ik}}{\partial x_k}
+
\sum_{i=k+1}^{N}\frac{1}{g_{ki}}\frac{\partial g_{ki}}{\partial x_k}.
$$

An equivalent equation is obtained for the exponential form after replacing $g_{ij}$ by $\exp(f_{ij})$, yielding:

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{\partial g_{ik}}{\partial x_k}
+
\sum_{i=k+1}^{N}\frac{\partial g_{ki}}{\partial x_k},
$$

with both expressions scaling as $\mathcal{O}(N)$.

## Importance sampling

Using the identity

$$
\frac{\partial}{\partial x_i}g_{ij} = -\frac{\partial}{\partial x_j}g_{ij},
$$

we get expressions where all the derivatives acting on the particle  are represented by the *second* index of $g$:

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{1}{g_{ik}}\frac{\partial g_{ik}}{\partial x_k}
-\sum_{i=k+1}^{N}\frac{1}{g_{ki}}\frac{\partial g_{ki}}{\partial x_i},
$$

and for the exponential case:

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{\partial g_{ik}}{\partial x_k}
-\sum_{i=k+1}^{N}\frac{\partial g_{ki}}{\partial x_i}.
$$

## Importance sampling
For correlation forms depending only on the scalar distances $r_{ij}$ we can use the chain rule. Noting that

$$
\frac{\partial g_{ij}}{\partial x_j} = \frac{\partial g_{ij}}{\partial r_{ij}} \frac{\partial r_{ij}}{\partial x_j} = \frac{x_j - x_i}{r_{ij}} \frac{\partial g_{ij}}{\partial r_{ij}},
$$

we arrive at

$$
\frac{1}{\Psi_C}\frac{\partial \Psi_C}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{1}{g_{ik}} \frac{\mathbf{r_{ik}}}{r_{ik}} \frac{\partial g_{ik}}{\partial r_{ik}}
-\sum_{i=k+1}^{N}\frac{1}{g_{ki}}\frac{\mathbf{r_{ki}}}{r_{ki}}\frac{\partial g_{ki}}{\partial r_{ki}}.
$$

## Importance sampling
Note that for the Pade-Jastrow form we can set $g_{ij} \equiv g(r_{ij}) = e^{f(r_{ij})} = e^{f_{ij}}$ and

$$
\frac{\partial g_{ij}}{\partial r_{ij}} = g_{ij} \frac{\partial f_{ij}}{\partial r_{ij}}.
$$

Therefore,

$$
\frac{1}{\Psi_{C}}\frac{\partial \Psi_{C}}{\partial x_k} =
\sum_{i=1}^{k-1}\frac{\mathbf{r_{ik}}}{r_{ik}}\frac{\partial f_{ik}}{\partial r_{ik}}
-\sum_{i=k+1}^{N}\frac{\mathbf{r_{ki}}}{r_{ki}}\frac{\partial f_{ki}}{\partial r_{ki}},
$$

where

$$
\mathbf{r}_{ij} = |\mathbf{r}_j - \mathbf{r}_i| = (x_j - x_i)\mathbf{e}_1 + (y_j - y_i)\mathbf{e}_2 + (z_j - z_i)\mathbf{e}_3
$$

is the relative distance.

## Importance sampling
The second derivative of the Jastrow factor divided by the Jastrow factor (the way it enters the kinetic energy) is

$$
\left[\frac{\mathbf{\nabla}^2 \Psi_C}{\Psi_C}\right]_x =\
2\sum_{k=1}^{N}
\sum_{i=1}^{k-1}\frac{\partial^2 g_{ik}}{\partial x_k^2}\ +\
\sum_{k=1}^N
\left(
\sum_{i=1}^{k-1}\frac{\partial g_{ik}}{\partial x_k} -
\sum_{i=k+1}^{N}\frac{\partial g_{ki}}{\partial x_i}
\right)^2
$$

## Importance sampling
But we have a simple form for the function, namely

$$
\Psi_{C}=\prod_{i< j}\exp{f(r_{ij})},
$$

and it is easy to see that for particle  $k$
we have

$$
\frac{\mathbf{\nabla}^2_k \Psi_C}{\Psi_C }=
\sum_{ij\ne k}\frac{(\mathbf{r}_k-\mathbf{r}_i)(\mathbf{r}_k-\mathbf{r}_j)}{r_{ki}r_{kj}}f'(r_{ki})f'(r_{kj})+
\sum_{j\ne k}\left( f''(r_{kj})+\frac{2}{r_{kj}}f'(r_{kj})\right)
$$

## Use the [C++ random class for random number generations](http://www.cplusplus.com/reference/random/)

         // Initialize the seed and call the Mersienne algo
          std::random_device rd;
          std::mt19937_64 gen(rd());
          // Set up the uniform distribution for x \in [[0, 1]
          std::uniform_real_distribution<double> UniformNumberGenerator(0.0,1.0);
          std::normal_distribution<double> Normaldistribution(0.0,1.0);

## Use the C++ random class for RNGs, the Mersenne twister class
**Finding the new position for importance sampling**

         for (int cycles = 1; cycles <= NumberMCsamples; cycles++){
            // new position
            for (int i = 0; i < NumberParticles; i++) {
              for (int j = 0; j < Dimension; j++) {
                // gaussian deviate to compute new positions using a given timestep
                NewPosition(i,j) = OldPosition(i,j) + Normaldistribution(gen)*sqrt(timestep)+OldQuantumForce(i,j)*timestep*D;

              }

## Use the C++ random class for RNGs, the Metropolis test
**Using the uniform distribution for the Metropolis test**

              //  Metropolis-Hastings algorithm
              double GreensFunction = 0.0;
              for (int j = 0; j < Dimension; j++) {
                GreensFunction += 0.5*(OldQuantumForce(i,j)+NewQuantumForce(i,j))*
                  (D*timestep*0.5*(OldQuantumForce(i,j)-NewQuantumForce(i,j))-NewPosition(i,j)+OldPosition(i,j));
              }
              GreensFunction = exp(GreensFunction);
              // The Metropolis test is performed by moving one particle at the time
              if(UniformNumberGenerator(gen) <= GreensFunction*NewWaveFunction*NewWaveFunction/OldWaveFunction/OldWaveFunction ) {
                for (int  j = 0; j < Dimension; j++) {
                  OldPosition(i,j) = NewPosition(i,j);
                  OldQuantumForce(i,j) = NewQuantumForce(i,j);
                }
                OldWaveFunction = NewWaveFunction;
              }

## Importance sampling, Fokker-Planck and Langevin equations
A stochastic process is simply a function of two variables, one is the time,
the other is a stochastic variable $X$, defined by specifying
* the set $\left\{x\right\}$ of possible values for $X$;

* the probability distribution, $w_X(x)$,  over this set, or briefly $w(x)$

The set of values $\left\{x\right\}$ for $X$
may be discrete, or continuous. If the set of
values is continuous, then $w_X (x)$ is a probability density so that
$w_X (x)dx$
is the probability that one finds the stochastic variable $X$ to have values
in the range $[x, x + dx]$ .

## Importance sampling, Fokker-Planck and Langevin equations
     An arbitrary number of other stochastic variables may be derived from
$X$. For example, any $Y$ given by a mapping of $X$, is also a stochastic
variable. The mapping may also be time-dependent, that is, the mapping
depends on an additional variable $t$

$$
Y_X (t) = f (X, t) .
$$

The quantity $Y_X (t)$ is called a random function, or, since $t$ often is time,
a stochastic process. A stochastic process is a function of two variables,
one is the time, the other is a stochastic variable $X$. Let $x$ be one of the
possible values of $X$ then

$$
y(t) = f (x, t),
$$

is a function of $t$, called a sample function or realization of the process.
In physics one considers the stochastic process to be an ensemble of such
sample functions.

## Importance sampling, Fokker-Planck and Langevin equations
     For many physical systems initial distributions of a stochastic
variable $y$ tend to equilibrium distributions: $w(y, t)\rightarrow w_0(y)$
as $t\rightarrow\infty$. In
equilibrium detailed balance constrains the transition rates

$$
W(y\rightarrow y')w(y ) = W(y'\rightarrow y)w_0 (y),
$$

where $W(y'\rightarrow y)$
is the probability, per unit time, that the system changes
from a state $|y\rangle$ , characterized by the value $y$
for the stochastic variable $Y$ , to a state $|y'\rangle$.

Note that for a system in equilibrium the transition rate
$W(y'\rightarrow y)$ and
the reverse $W(y\rightarrow y')$ may be very different.

## Importance sampling, Fokker-Planck and Langevin equations
Consider, for instance, a simple
system that has only two energy levels $\epsilon_0 = 0$ and
$\epsilon_1 = \Delta E$.

For a system governed by the Boltzmann distribution we find (the partition function has been taken out)

$$
W(0\rightarrow 1)\exp{-(\epsilon_0/kT)} = W(1\rightarrow 0)\exp{-(\epsilon_1/kT)}
$$

We get then

$$
\frac{W(1\rightarrow 0)}{W(0 \rightarrow 1)}=\exp{-(\Delta E/kT)},
$$

which goes to zero when $T$ tends to zero.

## Importance sampling, Fokker-Planck and Langevin equations
If we assume a discrete set of events,
our initial probability
distribution function can be  given by

$$
w_i(0) = \delta_{i,0},
$$

and its time-development after a given time step $\Delta t=\epsilon$ is

$$
w_i(t) = \sum_{j}W(j\rightarrow i)w_j(t=0).
$$

The continuous analog to $w_i(0)$ is

$$
w(\mathbf{x})\rightarrow \delta(\mathbf{x}),
$$

where we now have generalized the one-dimensional position $x$ to a generic-dimensional
vector $\mathbf{x}$. The Kroenecker $\delta$ function is replaced by the $\delta$ distribution
function $\delta(\mathbf{x})$ at  $t=0$.

## Importance sampling, Fokker-Planck and Langevin equations
The transition from a state $j$ to a state $i$ is now replaced by a transition
to a state with position $\mathbf{y}$ from a state with position $\mathbf{x}$.
The discrete sum of transition probabilities can then be replaced by an integral
and we obtain the new distribution at a time $t+\Delta t$ as

$$
w(\mathbf{y},t+\Delta t)= \int W(\mathbf{y},t+\Delta t| \mathbf{x},t)w(\mathbf{x},t)d\mathbf{x},
$$

and after $m$ time steps we have

$$
w(\mathbf{y},t+m\Delta t)= \int W(\mathbf{y},t+m\Delta t| \mathbf{x},t)w(\mathbf{x},t)d\mathbf{x}.
$$

When equilibrium is reached we have

$$
w(\mathbf{y})= \int W(\mathbf{y}|\mathbf{x}, t)w(\mathbf{x})d\mathbf{x},
$$

that is no time-dependence. Note our change of notation for $W$

## Importance sampling, Fokker-Planck and Langevin equations
We can solve the equation for $w(\mathbf{y},t)$ by making a Fourier transform to
momentum space.
The PDF $w(\mathbf{x},t)$ is related to its Fourier transform
$\tilde{w}(\mathbf{k},t)$ through

$$
w(\mathbf{x},t) = \int_{-\infty}^{\infty}d\mathbf{k} \exp{(i\mathbf{kx})}\tilde{w}(\mathbf{k},t),
$$

and using the definition of the
$\delta$-function

$$
\delta(\mathbf{x}) = \frac{1}{2\pi} \int_{-\infty}^{\infty}d\mathbf{k} \exp{(i\mathbf{kx})},
$$

we see that

$$
\tilde{w}(\mathbf{k},0)=1/2\pi.
$$

## Importance sampling, Fokker-Planck and Langevin equations
We can then use the Fourier-transformed diffusion equation

$$
\frac{\partial \tilde{w}(\mathbf{k},t)}{\partial t} = -D\mathbf{k}^2\tilde{w}(\mathbf{k},t),
$$

with the obvious solution

$$
\tilde{w}(\mathbf{k},t)=\tilde{w}(\mathbf{k},0)\exp{\left[-(D\mathbf{k}^2t)\right)}=
    \frac{1}{2\pi}\exp{\left[-(D\mathbf{k}^2t)\right]}.
$$

## Importance sampling, Fokker-Planck and Langevin equations
With the Fourier transform we obtain

$$
w(\mathbf{x},t)=\int_{-\infty}^{\infty}d\mathbf{k} \exp{\left[i\mathbf{kx}\right]}\frac{1}{2\pi}\exp{\left[-(D\mathbf{k}^2t)\right]}=
    \frac{1}{\sqrt{4\pi Dt}}\exp{\left[-(\mathbf{x}^2/4Dt)\right]},
$$

with the normalization condition

$$
\int_{-\infty}^{\infty}w(\mathbf{x},t)d\mathbf{x}=1.
$$

## Importance sampling, Fokker-Planck and Langevin equations
The solution represents the probability of finding
our random walker at position $\mathbf{x}$ at time $t$ if the initial distribution
was placed at $\mathbf{x}=0$ at $t=0$.

There is another interesting feature worth observing. The discrete transition probability $W$
itself is given by a binomial distribution.
The results from the central limit theorem state that
transition probability in the limit $n\rightarrow \infty$ converges to the normal
distribution. It is then possible to show that

$$
W(il-jl,n\epsilon)\rightarrow W(\mathbf{y},t+\Delta t|\mathbf{x},t)=
    \frac{1}{\sqrt{4\pi D\Delta t}}\exp{\left[-((\mathbf{y}-\mathbf{x})^2/4D\Delta t)\right]},
$$

and that it satisfies the normalization condition and is itself a solution
to the diffusion equation.

## Importance sampling, Fokker-Planck and Langevin equations
Let us now assume that we have three PDFs for times $t_0 < t' < t$, that is
$w(\mathbf{x}_0,t_0)$, $w(\mathbf{x}',t')$ and $w(\mathbf{x},t)$.
We have then

$$
w(\mathbf{x},t)= \int_{-\infty}^{\infty} W(\mathbf{x}.t|\mathbf{x}'.t')w(\mathbf{x}',t')d\mathbf{x}',
$$

and

$$
w(\mathbf{x},t)= \int_{-\infty}^{\infty} W(\mathbf{x}.t|\mathbf{x}_0.t_0)w(\mathbf{x}_0,t_0)d\mathbf{x}_0,
$$

and

$$
w(\mathbf{x}',t')= \int_{-\infty}^{\infty} W(\mathbf{x}'.t'|\mathbf{x}_0,t_0)w(\mathbf{x}_0,t_0)d\mathbf{x}_0.
$$

## Importance sampling, Fokker-Planck and Langevin equations
We can combine these equations and arrive at the famous Einstein-Smoluchenski-Kolmogorov-Chapman (ESKC) relation

$$
W(\mathbf{x}t|\mathbf{x}_0t_0)  = \int_{-\infty}^{\infty} W(\mathbf{x},t|\mathbf{x}',t')W(\mathbf{x}',t'|\mathbf{x}_0,t_0)d\mathbf{x}'.
$$

We can replace the spatial dependence with a dependence upon say the velocity
(or momentum), that is we have

$$
W(\mathbf{v},t|\mathbf{v}_0,t_0)  = \int_{-\infty}^{\infty} W(\mathbf{v},t|\mathbf{v}',t')W(\mathbf{v}',t'|\mathbf{v}_0,t_0)d\mathbf{x}'.
$$

## Importance sampling, Fokker-Planck and Langevin equations
We will now derive the Fokker-Planck equation.
We start from the ESKC equation

$$
W(\mathbf{x},t|\mathbf{x}_0,t_0)  = \int_{-\infty}^{\infty} W(\mathbf{x},t|\mathbf{x}',t')W(\mathbf{x}',t'|\mathbf{x}_0,t_0)d\mathbf{x}'.
$$

Define $s=t'-t_0$, $\tau=t-t'$ and $t-t_0=s+\tau$. We have then

$$
W(\mathbf{x},s+\tau|\mathbf{x}_0)  = \int_{-\infty}^{\infty} W(\mathbf{x},\tau|\mathbf{x}')W(\mathbf{x}',s|\mathbf{x}_0)d\mathbf{x}'.
$$

## Importance sampling, Fokker-Planck and Langevin equations
Assume now that $\tau$ is very small so that we can make an expansion in terms of a small step $xi$, with $\mathbf{x}'=\mathbf{x}-\xi$, that is

$$
W(\mathbf{x},s|\mathbf{x}_0)+\frac{\partial W}{\partial s}\tau +O(\tau^2) = \int_{-\infty}^{\infty} W(\mathbf{x},\tau|\mathbf{x}-\xi)W(\mathbf{x}-\xi,s|\mathbf{x}_0)d\mathbf{x}'.
$$

We assume that $W(\mathbf{x},\tau|\mathbf{x}-\xi)$ takes non-negligible values only when $\xi$ is small. This is just another way of stating the Master equation!!

## Importance sampling, Fokker-Planck and Langevin equations
We say thus that $\mathbf{x}$ changes only by a small amount in the time interval $\tau$.
This means that we can make a Taylor expansion in terms of $\xi$, that is we
expand

$$
W(\mathbf{x},\tau|\mathbf{x}-\xi)W(\mathbf{x}-\xi,s|\mathbf{x}_0) =
\sum_{n=0}^{\infty}\frac{(-\xi)^n}{n!}\frac{\partial^n}{\partial x^n}\left[W(\mathbf{x}+\xi,\tau|\mathbf{x})W(\mathbf{x},s|\mathbf{x}_0)
\right].
$$

## Importance sampling, Fokker-Planck and Langevin equations
We can then rewrite the ESKC equation as

$$
\frac{\partial W}{\partial s}\tau=-W(\mathbf{x},s|\mathbf{x}_0)+
\sum_{n=0}^{\infty}\frac{(-\xi)^n}{n!}\frac{\partial^n}{\partial x^n}
\left[W(\mathbf{x},s|\mathbf{x}_0)\int_{-\infty}^{\infty} \xi^nW(\mathbf{x}+\xi,\tau|\mathbf{x})d\xi\right].
$$

We have neglected higher powers of $\tau$ and have used that for $n=0$
we get simply $W(\mathbf{x},s|\mathbf{x}_0)$ due to normalization.

## Importance sampling, Fokker-Planck and Langevin equations
We say thus that $\mathbf{x}$ changes only by a small amount in the time interval $\tau$.
This means that we can make a Taylor expansion in terms of $\xi$, that is we
expand

$$
W(\mathbf{x},\tau|\mathbf{x}-\xi)W(\mathbf{x}-\xi,s|\mathbf{x}_0) =
\sum_{n=0}^{\infty}\frac{(-\xi)^n}{n!}\frac{\partial^n}{\partial x^n}\left[W(\mathbf{x}+\xi,\tau|\mathbf{x})W(\mathbf{x},s|\mathbf{x}_0)
\right].
$$

## Importance sampling, Fokker-Planck and Langevin equations
We can then rewrite the ESKC equation as

$$
\frac{\partial W(\mathbf{x},s|\mathbf{x}_0)}{\partial s}\tau=-W(\mathbf{x},s|\mathbf{x}_0)+
\sum_{n=0}^{\infty}\frac{(-\xi)^n}{n!}\frac{\partial^n}{\partial x^n}
\left[W(\mathbf{x},s|\mathbf{x}_0)\int_{-\infty}^{\infty} \xi^nW(\mathbf{x}+\xi,\tau|\mathbf{x})d\xi\right].
$$

We have neglected higher powers of $\tau$ and have used that for $n=0$
we get simply $W(\mathbf{x},s|\mathbf{x}_0)$ due to normalization.

## Importance sampling, Fokker-Planck and Langevin equations
We simplify the above by introducing the moments

$$
M_n=\frac{1}{\tau}\int_{-\infty}^{\infty} \xi^nW(\mathbf{x}+\xi,\tau|\mathbf{x})d\xi=
\frac{\langle [\Delta x(\tau)]^n\rangle}{\tau},
$$

resulting in

$$
\frac{\partial W(\mathbf{x},s|\mathbf{x}_0)}{\partial s}=
\sum_{n=1}^{\infty}\frac{(-\xi)^n}{n!}\frac{\partial^n}{\partial x^n}
\left[W(\mathbf{x},s|\mathbf{x}_0)M_n\right].
$$

## Importance sampling, Fokker-Planck and Langevin equations
When $\tau \rightarrow 0$ we assume that $\langle [\Delta x(\tau)]^n\rangle \rightarrow 0$ more rapidly than $\tau$ itself if $n > 2$.
When $\tau$ is much larger than the standard correlation time of
system then $M_n$ for $n > 2$ can normally be neglected.
This means that fluctuations become negligible at large time scales.

If we neglect such terms we can rewrite the ESKC equation as

$$
\frac{\partial W(\mathbf{x},s|\mathbf{x}_0)}{\partial s}=
-\frac{\partial M_1W(\mathbf{x},s|\mathbf{x}_0)}{\partial x}+
\frac{1}{2}\frac{\partial^2 M_2W(\mathbf{x},s|\mathbf{x}_0)}{\partial x^2}.
$$

## Importance sampling, Fokker-Planck and Langevin equations
In a more compact form we have

$$
\frac{\partial W}{\partial s}=
-\frac{\partial M_1W}{\partial x}+
\frac{1}{2}\frac{\partial^2 M_2W}{\partial x^2},
$$

which is the Fokker-Planck equation!  It is trivial to replace
position with velocity (momentum).

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

Consider a particle  suspended in a liquid. On its path through the liquid it will continuously collide with the liquid molecules. Because on average the particle  will collide more often on the front side than on the back side, it will experience a systematic force proportional with its velocity, and directed opposite to its velocity. Besides this systematic force the particle  will experience a stochastic force  $\mathbf{F}(t)$.
The equations of motion are
* $\frac{d\mathbf{r}}{dt}=\mathbf{v}$ and

* $\frac{d\mathbf{v}}{dt}=-\xi \mathbf{v}+\mathbf{F}$.

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

From hydrodynamics  we know that the friction constant  $\xi$ is given by

$$
\xi =6\pi \eta a/m
$$

where $\eta$ is the viscosity  of the solvent and a is the radius of the particle .

Solving the second equation in the previous slide we get

$$
\mathbf{v}(t)=\mathbf{v}_{0}e^{-\xi t}+\int_{0}^{t}d\tau e^{-\xi (t-\tau )}\mathbf{F }(\tau ).
$$

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

If we want to get some useful information out of this, we have to average over all possible realizations of
$\mathbf{F}(t)$, with the initial velocity as a condition. A useful quantity for example is

8
6

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

$$
+\int_{0}^{t}d\tau ^{\prime }\int_{0}^{t}d\tau e^{-\xi (2t-\tau -\tau ^{\prime })}
\langle \mathbf{F}(\tau )\cdot \mathbf{F}(\tau ^{\prime })\rangle_{ \mathbf{v}_{0}}.
$$

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

In order to continue we have to make some assumptions about the conditional averages of the stochastic forces.
In view of the chaotic character of the stochastic forces the following
assumptions seem to be appropriate

$$
\langle \mathbf{F}(t)\rangle=0,
$$

and

$$
\langle \mathbf{F}(t)\cdot \mathbf{F}(t^{\prime })\rangle_{\mathbf{v}_{0}}=  C_{\mathbf{v}_{0}}\delta (t-t^{\prime }).
$$

We omit the subscript $\mathbf{v}_{0}$, when the quantity of interest turns out to be independent of $\mathbf{v}_{0}$. Using the last three equations we get

$$
\langle \mathbf{v}(t)\cdot \mathbf{v}(t)\rangle_{\mathbf{v}_{0}}=v_{0}^{2}e^{-2\xi t}+\frac{C_{\mathbf{v}_{0}}}{2\xi }(1-e^{-2\xi t}).
$$

For large t this should be equal to 3kT/m, from which it follows that

$$
\langle \mathbf{F}(t)\cdot \mathbf{F}(t^{\prime })\rangle =6\frac{kT}{m}\xi \delta (t-t^{\prime }).
$$

This result is called the fluctuation-dissipation theorem .

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

Integrating

$$
\mathbf{v}(t)=\mathbf{v}_{0}e^{-\xi t}+\int_{0}^{t}d\tau e^{-\xi (t-\tau )}\mathbf{F }(\tau ),
$$

we get

$$
\mathbf{r}(t)=\mathbf{r}_{0}+\mathbf{v}_{0}\frac{1}{\xi }(1-e^{-\xi t})+
\int_0^td\tau \int_0^{\tau}\tau ^{\prime } e^{-\xi (\tau -\tau ^{\prime })}\mathbf{F}(\tau ^{\prime }),
$$

from which we calculate the mean square displacement

$$
\langle ( \mathbf{r}(t)-\mathbf{r}_{0})^{2}\rangle _{\mathbf{v}_{0}}=\frac{v_0^2}{\xi}(1-e^{-\xi t})^{2}+\frac{3kT}{m\xi ^{2}}(2\xi t-3+4e^{-\xi t}-e^{-2\xi t}).
$$

## Importance sampling, Fokker-Planck and Langevin equations
**Langevin equation.**

For very large $t$ this becomes

$$
\langle (\mathbf{r}(t)-\mathbf{r}_{0})^{2}\rangle =\frac{6kT}{m\xi }t
$$

from which we get the Einstein relation

$$
D= \frac{kT}{m\xi }
$$

where we have used $\langle (\mathbf{r}(t)-\mathbf{r}_{0})^{2}\rangle =6Dt$.


# BHPToolkit Spring 2020 Workshop: EMRISur1dq1e4 Proejct Tutorial

Some portions of this notebook are also found in the notebook
[EMRISur1dq1e4.ipynb](https://github.com/BlackHolePerturbationToolkit/EMRISurrogate/blob/master/EMRISur1dq1e4.ipynb). The waveform model is described in [arXiv:1910.10473](https://arxiv.org/abs/1910.10473).

EMRISur1dq1e4 is a surrogate gravitational-waveform model for non-spinning black hole binary systems with mass-ratios varying from 3 to $10^4$. This surrogate model is trained on waveform data generated by point-particle black hole perturbation theory (ppBHPT), with the total mass rescaling paramter tuned to NR simulations according to the paper's Eq. 4. Available modes are  $\{(2,2), (2,1), (3,3), (3,2), (3,1), (4,4), (4,3), (4,2), (5,5), (5,4), (5,3)\}$. The $m<0$ modes can be deduced from the m>0 modes due to symmetry of the system about the orbital plane.

**NOTE**: This notebook rquires the file emri_sur_load.py and EMRISur1dq1e4.h5 datafile in the same directory for waveform generation

 # Setup

 1. You should now have the programs git and Jupyter as well as the Python packages scipy, h5py, and hashlib
 2. clone the EMRISurrogate code and this tutorial

 ```shell
 >>> git clone https://github.com/BlackHolePerturbationToolkit/EMRISurrogate.git
 >>> cd EMRISurrogate    # move into the new directory
 >>> jupyter notebook BHPToolkit_Spring2020_Tutorial.ipynb   # launch this tutorial
 ```

(no-git fallback plan: download the zip file from https://github.com/BlackHolePerturbationToolkit/EMRISurrogate)

```python
# If your setup is correct, then this block of code should execute...

import matplotlib.pyplot as plt
import numpy as np
import hashlib
import h5py
```

# Lesson 1: The data

Surrogate models are numerical models. They require code *and* data

The EMRI surrogate model's data is [hosted on zenodo](https://zenodo.org/record/3612600#.XsoAP3VKg5k).

Our first task is to download the data:

```shell
>>> wget https://zenodo.org/record/3612600/files/EMRISur1dq1e4.h5
```

or click the "download" button and move the file to the same folder as this notebook.

## Make sure your data is at the latest version

The data file has a unique hash, which from zenodo is d145958484738e0c7292e084a66a96fa.

If the surrogate model is updated then EMRISur1dq1e4.h5 will be replaced with a newer file.

The [EMRI surrogate code](https://github.com/BlackHolePerturbationToolkit/EMRISurrogate/blob/master/emri_sur_load.py#L48) checks that your local h5 file's hash matches the most recent version.

```python
# data integrity: lets check that your file's hash is correct
# Current hash: d145958484738e0c7292e084a66a96fa

def md5(fname):
    hash_md5 = hashlib.md5()
    with open(fname, "rb") as f:
        for chunk in iter(lambda: f.read(4096), b""):
            hash_md5.update(chunk)
    return hash_md5.hexdigest()

print( md5('EMRISur1dq1e4.h5') )
```

    d145958484738e0c7292e084a66a96fa

```python
# now import the some functions to help with evaluating the model
# This step requires that you've cloaned the git
# project EMRISurrogate and have the data file listed above

import emri_sur_load as emriSur
```

## Data file's structure

Recall that the full surrogate model is

\begin{align}
h_{\tt S}(t,\theta,\phi;q) & = \sum_{\ell,m} h_{\tt S}^{\ell,m}(t;q) {}_{-2}Y_{\ell m} (\theta,\phi) \\
h_{\tt S}^{\ell,m}(t;q) & = A_{\tt S}^{\ell,m}(t;q) \exp(- \mathrm{i} \phi_{\tt S}^{\ell,m}(t;q)) \,,
\end{align}
and we build models for $A_{\tt S}^{\ell,m}$, $\phi_{\tt S}^{\ell,m}$. Each $A_{\tt S}^{\ell,m}$ and $\phi_{\tt S}^{\ell,m}$ is represented as an empirical interpolant (EI) with parametric fits that the time nodes.

Lets check that our data file matches this structure.

```python
# Each mode is a separate surrogate model

# open the file
fp = h5py.File("EMRISur1dq1e4.h5","r")

# inspect the data groups
print("EMRI surrogate data groups: ",fp.keys() )
print("\n\n" )

# grab the 22 mode data group
sur_22 = fp["l2_m2"]

# inspect (2,2)-mode's data groups
print("22 mode surrogate data groups: ",sur_22.keys() )
```

    EMRI surrogate data groups:  <KeysViewHDF5 ['l2_m1', 'l2_m2', 'l3_m1', 'l3_m2', 'l3_m3', 'l4_m2', 'l4_m3', 'l4_m4', 'l5_m3', 'l5_m4', 'l5_m5']>

    22 mode surrogate data groups:  <KeysViewHDF5 ['B', 'B_phase', 'degree', 'eim_indicies', 'eim_indicies_phase', 'fit_max', 'fit_min', 'fit_type_amp', 'fit_type_norm', 'fit_type_phase', 'fitparams_amp', 'fitparams_phase', 'n_spline_knots', 'parameterization', 'spline_knots', 'surrogate_mode_type', 'times']>

## The surrogate model for $A^{22}(t;q)$

Do to an unfortunate choice made a long time ago, the quantitites without any extra postfix denote ampitude date: B_phase is the basis for the phase, while B is the basis for the amplitude. So we have the following data for the 22 mode's amplitude:

### Information about the amplitude's parametric dependence
* 'fit_type_amp': model for the amplitude's dependence with mass ratio
* 'parameterization': how we parameterize the amplitude data
* 'fit_max' / 'fit_min' : smallest / largest values of parametric domain
* 'n_spline_knots': number of spline knots
* 'spline_knots':  location of spline knots
* 'fitparams_amp': spline parameters
* 'degree': degree of the spline used

### Information about the amplitude's temporal dependence
* 'B': basis
* 'eim_indicies': location of empirical interpolation nodes
* 'times': temporal domain on which the amplitude is modeled

The model is given by

\begin{align}
A_{\tt S}^{22}(t;q) = \sum_{i=0}^9 A(T_i^\mathrm{EIM};q) B_i(t) \,,
\end{align}

where $B_i(t)$ is the $i^{th}$ basis function and $T_i^\mathrm{EIM}$ is the $i^{th}$ empirical interpolation time node. We see that, $A(T_i^\mathrm{EIM};q)$ are the coefficients defining the amplitude's expansion in terms of the basis.

Data found in "Information about the amplitude's parametric dependence" is used to model each coefficient, $A(T_i^\mathrm{EIM};q)$, over the paramter space.

```python
# Lets inspect the basis for the 22 mode's amplitude
# Because each basis is a linear combination of amplitudes,
# the basis will look kinda like an waveform amplitude.

# The basis will not look like any standard function (polynomials, sine/cosine, etc...)

### plot the ith cardinal basis function  ###
i = 0
times = sur_22['times']
B     = sur_22['B']

plt.figure(1)
plt.plot(times,B[:,i])
plt.title('Cardinal basis %i for amplitude'%i)

plt.show()
```

```python
# Lets inspect the empirical interpolation time nodes for the
# 22 mode's amplitude.

# These nodes are used to define the interpolation problem: the
# surrogate interpolates the waveform training data at these times.

# The time nodes will automatically be placed in regions of higher
# activitiy. Unlike uniform, Chebyshev, or other common time nodes
# the EI nodes are adaptive to the problem

### plot the location of all emprical interpolation points  ###
plt.figure(2)
eim_indicies = sur_22['eim_indicies'][:]
plt.plot(times,B[:,0]) # 0^th basis function
plt.plot(times[np.sort(eim_indicies)],np.zeros_like(eim_indicies),'r*')
plt.title('EI time node locations')

plt.figure(3)
eim_indicies = sur_22['eim_indicies'][:]
plt.plot(times,B[:,0]) # 0^th basis function
plt.plot(times[np.sort(eim_indicies)],np.zeros_like(eim_indicies),'r*')
plt.xlim([-500,100])
plt.title('EI time node locations')
```

# Lesson 2: Evaluating the model

Input: mass ratio q, and (optionally) modes to generate = [(2,1),(2,2),(3,1),(3,2),(3,3),(4,2),(4,3),(4,4),(5,3),(5,4),(5,5)]

Output: dictionary of modes, $h_{\tt S}^{\ell,m}(t;q)$

\begin{align}
h_{\tt S}(t,\theta,\phi;q) & = \sum_{\ell,m} h_{\tt S}^{\ell,m}(t;q) {}_{-2}Y_{\ell m} (\theta,\phi)
\end{align}

Both h and t are in geometric units

```python
q = 10.0 # mass ratio
time, h = emriSur.generate_surrogate(q) # h is a dictionary with modes as its key

# h is a dictionary of modes
h_22=h[(2,2)]
h_21=h[(2,1)]

plt.figure(figsize=(14,4))
plt.plot(time,np.real(h_22),label='{2,2} mode')
plt.plot(time,np.real(h_21),label='{2,1} mode')
plt.xlabel('t',fontsize=12)
plt.ylabel('h(t)',fontsize=12)
plt.legend(fontsize=12)
#plt.savefig('emri_sur_q_%f.png'%q)
plt.show()
```

```python
# waveform in mks units (roughly what the detector would observe)

G=6.674*1e-11
MSUN_SI = 1.9885469549614615e+30
PC_SI = 3.085677581491367e+16
C_SI = 299792458.0

# values of M and dL
M=80.0*MSUN_SI
dL=100.0* PC_SI

# scaling of time and h(t)
time=time*(G*M/C_SI**3)
ht22=np.array(h[(2,2)])*(G*M/C_SI**3)/dL

plt.figure(figsize=(14,4))
plt.title('M=80, dL=100.0')
plt.plot(time,np.real(ht22),label='{2,2} mode')
plt.xlabel('t',fontsize=12)
plt.ylabel('h(t)',fontsize=12)
plt.legend(fontsize=12)
#plt.savefig('emri_sur_q_%f_physical.png'%q)
plt.show()
```

# Lesson 3: EMRI surrogate vs output from the Teukolsky solver

Our underlying model (denoted $h^{\ell,m}_{\tt S}$ below) is for the output of the Teukolsky solver for point-particle perturbation theory.

However, the EMRI surrogate model (denoted $h^{\ell,m}_{\tt S, \alpha}$ below) is calibrated to numerical relativity waveforms at comparable mass binaries with a single parameter $\alpha$:

\begin{align*}
h^{\ell,m}_{\tt S, \alpha}(t ; q)= {\alpha} h^{\ell,m}_{\tt S}\left( t \alpha;q \right) \,,
\end{align*}

Which is enacted in emri_sur_load.py file as shown below:

### To generate output from the Teukolsky solver (no NR calibration) simply set $\alpha = 1$ in lines 110 and 118

(Issue tracker: feature request to allow for this without needing to modify the code)

# Summary

1. To understand how to use the EMRI surrogate model, please consult the Jupyter noteobook found with the [EMRISur1dq1e4 proejct](https://github.com/BlackHolePerturbationToolkit/EMRISurrogate/blob/master/EMRISur1dq1e4.ipynb) in addition to this notebook.

2. Future EMRI surrogate models or improvements will be added here.

3. If you find any issues or have suggestions please open up an issue (or, better yet, a pull request!)

```python

```


```python
#remove cell visibility
from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)
```

Toggle cell visibility here.

## Jordanova oblika - kompleksne lastne vrednosti

Ta interaktivni primer prikazuje, kako lahko osnovno Jordanovo obliko, v primeru matrike $A$ s kompleksnimi lastnimi vrednostmi, spremenimo na način, da Jordanova matrika $J$ sestoji samo iz realnih elementov.

Kadar ima matrika $A$ kompleksne lastne vrednosti $\lambda$, ima tudi kompleksno konjugirane lastne vrednosti $\bar \lambda$. Možno je dokazati, da, če je lastni vektor $v$ povezan z lastno vrednostjo $\lambda$, je potem lastni vektor povezan z lastno vrednostjo $\bar \lambda$ kompleksno konjugirani par lastnega vektorja $v$; označimo ga z $\bar v$.

Ta pomembna lastnost omogoča, da spremenimo matriko $V$ na način, da zamenjamo stolpce $v$ in $\bar v$ z njuno linearno kombinacijo $(v+\bar v)$ in $(v-\bar v)$; pri tem dobimo drugačno obliko Jordanove matrike, v kateri kompleksno konjugiran par lastnih vrednosti $\lambda$ in $\bar \lambda$ generira element Jordanove matrike z naslednjo obliko:

$$
J = \begin{bmatrix}
\sigma & \omega \\
-\omega & \sigma \\
\end{bmatrix},
$$

kjer je $\lambda = \sigma + i \omega$.

### Kako upravljati s tem interaktivnim primerom?

- Določi elemente želene matrike in opazuj njeno Jordanovo obliko; eksperimentiraj tako z realnimi kot kompleksnimi lastnimi vrednostmi.
- Poizkusi sestaviti matriko s kompleksnimi lastnimi vrednostmi in večkratnostjo (ang. multiplicity) enako 2.
- Ustvari poljubno matriko $A$ ali uporabi prednastavljene, izbrane primere matrik.

[//]: # "This example shows how the basic Jordan Form can be modified when the matrix $A$ has complex eigenvalues in order for its Jordan form $J$ to be composed by real numbers only.

When matrix $A$ has a complex eigenvalue $\lambda$, it has also its complex conjugate $\bar \lambda$ as eigenvalue. It can be shown that if the eigenvector $v$ is associated with the eigenvalue $\lambda$, then the eigenvector associated with the eigenvalue $\bar \lambda$ is the complex conjugate of $v$: $\bar v$.

This important result allows us to change the $V$ matrix (namely by substituting the columns of $v$ and $\bar v$ with their linear combinations $(v+\bar v)$ and $(v-\bar v)$) obtaining a different form of the Jordan Matrix where the complex conjugate pair of eigenvalues $\lambda$ and $\bar \lambda$ generates a Jordan element of the form:

$$
J = \begin{bmatrix}
\sigma & \omega \\
-\omega & \sigma \\
\end{bmatrix},
$$

with $\lambda = \sigma + i \omega$.

### How to use this notebook?

- Define a matrix and watch its Jordan form; experiment with both real and complex eigenvalues.
- Try to build a matrix with complex eigenvalues and multiplicity equal to 2.
- Explore the effects of changing the values of the matrix $A$ or load example matrices."

```python
#Preparatory Cell

import control
import numpy
import sympy
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt

from sympy import Matrix

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#create a NxM matrix widget
def createMatrixWidget(n,m):
    M = widgets.GridBox(children=[widgets.FloatText(layout=widgets.Layout(width='100px', height='40px'),
    value=0.0, disabled=False, label=i) for i in range(n*m)],
    layout=widgets.Layout(
        #width='50%',
        grid_template_columns= ''.join(['100px ' for i in range(m)]),
        #grid_template_rows='80px 80px 80px',
        grid_row_gap='0px',
        track_size='0px')
    )
    return M

#extract matrix from widgets and convert to numpy matrix
def getNumpyMatFromWidget(M,n,m):
    #get W gridbox dims
    M_ = numpy.matrix(numpy.zeros((n,m)))
    for irow in range(0,n):
        for icol in range(0,m):
            M_[irow,icol] = M.children[irow*3+icol].value

#this is a simple derived class from FloatText used to experience with interact
class floatWidget(widgets.FloatText):
    def __init__(self,**kwargs):
        #self.n = n
        self.value = 30.0
        #self.M =
        widgets.FloatText.__init__(self, **kwargs)

#    def value(self):
#        return 0 #self.FloatText.value

from traitlets import Unicode
from ipywidgets import register

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    def dummychangecallback(self,change):
        pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(control.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        control.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass

```

```python
#define matrices
A = matrixWidget(4,4)

#this is the main callback and does all the computations and plots
def main_callback(matA,DW,sel):
    #check if a specific matrix is requested or is manual
    if sel=='ročna določitev sistema':
        pass
    elif sel == 'stabilni sistem brez para kompleksno konjugiranih polov':
        matA = numpy.zeros((4,4))
        matA[0,0] = -1
        matA[1,1] = -2
        matA[2,2] = -3
        matA[3,3] = -4
        A.setM(matA)
    elif sel == 'stabilni sistem s parom kompleksno konjugiranih polov':
        matA = numpy.zeros((4,4))
        matA[0,0] = -1
        matA[0,1] = 3
        matA[1,0] = -3
        matA[1,1] = -1
        matA[2,2] = -3
        matA[3,3] = -4
        A.setM(matA)
    elif sel == 'nestabilni sistem z nestabilnim realnim polom':
        matA = numpy.zeros((4,4))
        matA[0,0] = 1
        matA[1,1] = -2
        matA[2,2] = -3
        matA[3,3] = -4
        A.setM(matA)
    elif sel == 'nestabilni sistem z nestabilnim parom kompleksno konjugiranih polov':
        matA = numpy.zeros((4,4))
        matA[0,0] = 1
        matA[0,1] = 3
        matA[1,0] = -3
        matA[1,1] = 1
        matA[2,2] = -3
        matA[3,3] = -4
        A.setM(matA)
    else :
        matA = numpy.zeros((4,4))
        A.setM(matA)

    # Work with symbolic matrix
    matAs = sympy.Matrix(matA)
    dictEig = matAs.eigenvals()
    eigs = list(dictEig.keys())
    algMult = list(dictEig.values())

    # check dimension of jordan blocks
    dimJblock = []
    for i in range(len(eigs)):
        dimJblock.append(algMult[i]-len((matAs-eigs[i]*sympy.eye(4)).nullspace())+1)

    # jordan form with real numbers
    matAs_P, matAs_J = matAs.jordan_form(chop=True)
    zeroV = Matrix([0,0,0,0])
    cols = []
    for i in range(4):
        RE, IM = matAs_P.col(i).as_real_imag()
        if IM == zeroV:
            cols.append(RE)
        elif not any([matAs_P.col(i).conjugate() == matAs_P.col(j) for j in range(i+1)]):
            cols.append(RE)
            cols.append(IM)
    matAs_P = cols[0].row_join(cols[1]).row_join(cols[2]).row_join(cols[3])
    matAs_J = matAs_P.inv()*matAs*matAs_P

    timeVectors = []
    modeVectors = []
    # compute modes simulations and prepare modestring
    modestring = ''
    for i in range(len(eigs)):
        sim = []
        if sympy.re(eigs[i]) >= 0:
            # instable or integral like
            time = numpy.linspace(0,10,1000)
            for n in range(dimJblock[i]):
                if n==0:
                    if sympy.im(eigs[i]) != 0 and (sympy.conjugate(eigs[i]) not in eigs[0:i]):
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time)*numpy.cos(float(sympy.im(eigs[i]))*time))
                        modestring = modestring + "$e^{%s t} cos(%s t + \phi)$  " % (str(float(sympy.re(eigs[i]))), str(float(sympy.im(eigs[i]))))
                    elif sympy.im(eigs[i]) == 0:
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time))
                        modestring = modestring + "$e^{%s t}$  " % (str(float(sympy.re(eigs[i]))))
                else:
                    if sympy.im(eigs[i]) != 0 and (sympy.conjugate(eigs[i]) not in eigs[0:i]):
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time)*numpy.cos(float(sympy.im(eigs[i]))*time))
                        modestring = modestring + "$t^{%s}e^{%s t} cos(%s t + \phi)$  " % (str(n), str(float(sympy.re(eigs[i]))), str(float(sympy.im(eigs[i]))))
                    elif sympy.im(eigs[i]) == 0:
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time))
                        modestring = modestring + "$t^{%s}e^{%s t}$  " % (str(n), str(float(sympy.re(eigs[i]))))
        else:
            # stable mode
            time = numpy.linspace(0,10*(1/float(sympy.Abs(eigs[i]))),1000)
            for n in range(dimJblock[i]):
                if n==0:
                    if sympy.im(eigs[i]) != 0 and (sympy.conjugate(eigs[i]) not in eigs[0:i]):
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time)*numpy.cos(float(sympy.im(eigs[i]))*time))
                        modestring = modestring + "$e^{%s t} cos(%s t + \phi)$  " % (str(float(sympy.re(eigs[i]))), str(float(sympy.im(eigs[i]))))
                    elif sympy.im(eigs[i]) == 0:
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time))
                        modestring = modestring + "$e^{%s t}$  " % (str(float(sympy.re(eigs[i]))))
                else:
                    if sympy.im(eigs[i]) != 0 and (sympy.conjugate(eigs[i]) not in eigs[0:i]):
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time)*numpy.cos(float(sympy.im(eigs[i]))*time))
                        modestring = modestring + "$t^{%s}e^{%s t} cos(%s t + \phi)$  " % (str(n), str(float(sympy.re(eigs[i]))), str(float(sympy.im(eigs[i]))))
                    elif sympy.im(eigs[i]) == 0:
                        sim.append(time**n*numpy.exp(float(sympy.re(eigs[i]))*time))
                        modestring = modestring + "$t^{%s}e^{%s t}$  " % (str(n), str(float(sympy.re(eigs[i]))))
        if len(sim) != 0:
            timeVectors.append(time)
            modeVectors.append(sim)

    #print(dimJblock)
    #print(len(modeVectors))

    #create textual output
    display(Markdown('Matrika $%s$ ima lastne vrednosti $%s$' % (vmatrix(matA), vmatrix(numpy.array(numpy.linalg.eig(matA)[0])))))
    #for better visualization
    matJlist = []
    for i in range(4):
        temp = []
        for j in range(4):
            if sympy.im(matAs_J[i,j]) != 0:
                temp.append(numpy.complex(matAs_J[i,j]))
            else:
                temp.append(numpy.real(matAs_J[i,j]))
        matJlist.append(temp)
    matJ = numpy.matrix(matJlist)
    display(Markdown('in njena Jordanova oblika je enaka: $%s$' %str(vmatrix(matJ))))
    #for better visualization
    matPlist = []
    for i in range(4):
        temp = []
        for j in range(4):
            if sympy.im(matAs_P[i,j]) != 0:
                temp.append(numpy.complex(matAs_P[i,j]))
            else:
                temp.append(numpy.real(matAs_P[i,j]))
        matPlist.append(temp)
    matP = numpy.matrix(matPlist)
    display(Markdown('z generaliziranimi lastnimi vrednostmi $%s$.' %str(vmatrix(matP))))
    display(Markdown('MODES so: %s' % modestring))

    #compute total number of figures
    totfig=0
    for i in range(len(modeVectors)):
            totfig = totfig + len(modeVectors[i])

    #plot each single mode
    fig = plt.figure(figsize=(20, 4))
    idx = 1
    for i in range(len(timeVectors)):
        for j in range(len(modeVectors[i])):
            sf = fig.add_subplot(1,totfig,idx)
            idx = idx + 1
            sf.plot(timeVectors[i],modeVectors[i][j])
            sf.grid(True)
            plt.xlabel(r'$t$ [s]')
            plt.axvline(x=0,color='black',linewidth=0.8)
            plt.axhline(y=0,color='black',linewidth=0.8)

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

#define type of ipout
SELECT = widgets.Dropdown(
    options=['ročna določitev sistema', 'ponastavi', 'stabilni sistem brez para kompleksno konjugiranih polov',
             'stabilni sistem s parom kompleksno konjugiranih polov',
             'nestabilni sistem z nestabilnim realnim polom',
             'nestabilni sistem z nestabilnim parom kompleksno konjugiranih polov'],
    value='ročna določitev sistema',
    description='Primeri:',
    disabled=False,
)

#create a graphic structure to hold all widgets
alltogether =  widgets.VBox([SELECT, widgets.Label(''), widgets.HBox([widgets.Label('$\dot{x}(t) = $',border=3), A,widgets.Label('$x(t)$',border=3), START])] )

out = widgets.interactive_output(main_callback,{'matA': A, 'DW': DW, 'sel': SELECT})
out.layout.height = '620px'
display(alltogether,out)
```

    VBox(children=(Dropdown(description='Primeri:', options=('ročna določitev sistema', 'ponastavi', 'stabilni sis…

    Output(layout=Layout(height='620px'))

```python

```

```python

```


```python
import pandas as pd
import matplotlib.pyplot as plt
import os
```

```python
# traverse root directory, and list directories as dirs and files as files
for root, dirs, files in os.walk("."):
    path = root.split(os.sep)
    print((len(path) - 1) * '---', os.path.basename(root))
    for file in files:
        print(len(path) * '---', file)
```

     .
    --- Simple-Recomender-System.ipynb
    --- README.md
    --- datasets
    ------ movies_metadata.csv
    ------ keywords.csv
    ------ credits.csv
    --- .ipynb_checkpoints
    ------ Simple-Recomender-System-checkpoint.ipynb

```python
# kaggle datasets
metadt = pd.read_csv('./datasets/movies_metadata.csv', low_memory=False)
print("Raw data shape: ", metadt.shape)
# 5k data (if you only have low compute engine)
metadt = metadt[:5000]
print("Sample data shape: ", metadt.shape)
metadt.head(3)
```

    Raw data shape:  (45466, 24)
    Sample data shape:  (5000, 24)

      adult
      belongs_to_collection
      budget
      genres
      homepage
      id
      imdb_id
      original_language
      original_title
      overview
      ...
      release_date
      revenue
      runtime
      spoken_languages
      status
      tagline
      title
      video
      vote_average
      vote_count

      0
      False
      {'id': 10194, 'name': 'Toy Story Collection', ...
      30000000
      [{'id': 16, 'name': 'Animation'}, {'id': 35, '...
      http://toystory.disney.com/toy-story
      862
      tt0114709
      en
      Toy Story
      Led by Woody, Andy's toys live happily in his ...
      ...
      1995-10-30
      373554033.0
      81.0
      [{'iso_639_1': 'en', 'name': 'English'}]
      Released
      NaN
      Toy Story
      False
      7.7
      5415.0

      1
      False
      NaN
      65000000
      [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...
      NaN
      8844
      tt0113497
      en
      Jumanji
      When siblings Judy and Peter discover an encha...
      ...
      1995-12-15
      262797249.0
      104.0
      [{'iso_639_1': 'en', 'name': 'English'}, {'iso...
      Released
      Roll the dice and unleash the excitement!
      Jumanji
      False
      6.9
      2413.0

      2
      False
      {'id': 119050, 'name': 'Grumpy Old Men Collect...
      0
      [{'id': 10749, 'name': 'Romance'}, {'id': 35, ...
      NaN
      15602
      tt0113228
      en
      Grumpier Old Men
      A family wedding reignites the ancient feud be...
      ...
      1995-12-22
      0.0
      101.0
      [{'iso_639_1': 'en', 'name': 'English'}]
      Released
      Still Yelling. Still Fighting. Still Ready for...
      Grumpier Old Men
      False
      6.5
      92.0

3 rows × 24 columns

# Recommender system using weighted rating

Penggunaan rating untuk rekomendasi memiliki kekurangan:
* rating tidak memberikan gambaran popularitas produk. Produk A memilki rating 9.5 dari 20 voters sedangkan produk B memiliki rating 8.7 dari 1000 voters. Mana yang lebih baik? tentu saja produk B, rating yang diberikan oleh lebih banyak user lebih terpercaya dibandingkan rating tinggi dengan sedikit user

Oleh karena itu, perlu dilakukan pembobotan rating sebagaimana rumus dibawah ini:
\begin{equation}
\text Weighted Rating (\bf WR) = \left({{\bf v} \over {\bf v} + {\bf m}} \cdot R\right) + \left({{\bf m} \over {\bf v} + {\bf m}} \cdot C\right)
\end{equation}
Keterangan:
v = jumlah voters
m = minimum votes yang dibutuhkan untuk masuk dalam list
R = rata-rata rating
C = mean atau rata-rata vote secara keseluruhan

```python
# average rating in datasets
C = metadt.vote_average.mean()
print(C)

# calculate minimum number of vote (m)
# here i'm gonna using 90% percentile
m = metadt.vote_count.quantile(0.9)
print(m)
```

    6.069160000000003
    568.1000000000004

```python
# filter movie with vote_count more than m
top_movies = metadt.copy().loc[metadt['vote_count'] > m]
print('shape:', metadt.shape)
print('shape:', top_movies.shape)
```

    shape: (5000, 24)
    shape: (500, 24)

```python
def weighted_rating(data, m=m, C=C):
    v = data['vote_count']
    R = data['vote_average']
    return (v/(v+m) * R) + (m/(v+m) * C)
```

```python
top_movies['weighted_rating'] = top_movies.apply(weighted_rating, axis=1)
```

```python
top_movies = top_movies.sort_values('weighted_rating', ascending=False)
print('shape:', top_movies.shape)
top_movies[['title', 'vote_count', 'vote_average', 'weighted_rating']].head(10)
```

    shape: (500, 25)

      title
      vote_count
      vote_average
      weighted_rating

      314
      The Shawshank Redemption
      8358.0
      8.5
      8.345290

      834
      The Godfather
      6024.0
      8.5
      8.290513

      2843
      Fight Club
      9678.0
      8.3
      8.176310

      292
      Pulp Fiction
      8670.0
      8.3
      8.162814

      351
      Forrest Gump
      8147.0
      8.2
      8.061100

      522
      Schindler's List
      4436.0
      8.3
      8.046740

      1154
      The Empire Strikes Back
      5998.0
      8.2
      8.015639

      2211
      Life Is Beautiful
      3643.0
      8.3
      7.999048

      1178
      The Godfather: Part II
      3418.0
      8.3
      7.982060

      289
      Leon: The Professional
      4293.0
      8.2
      7.950976

# Content-based Recommender
Natural Language Processing (TF-IDF) using **overview** feature on dataset

```python
from sklearn.feature_extraction.text import TfidfVectorizer

# create TF-IDF object and remove all stop_word like 'and', 'or', 'the', etc.
tfidf = TfidfVectorizer(stop_words='english')

metadt['overview'] = metadt['overview'].fillna('')
tfidf_matrix = tfidf.fit_transform(metadt['overview'])

tfidf_matrix.shape
```

    (5000, 22304)

Selanjutnya hitung nilai similarity. Score similarity dapat ditentukan menggunakan manhattan distance, euclidean distance, pearson atau cosine similarity. Berikut merupakan rumus untuk menghitung cosine similarity:

kita akan menggunakan sklearn **linear_kernel()** karena lebih cepat dibandingkan **cosine_similarity()**

```python
from sklearn.metrics.pairwise import linear_kernel
cos_sim = linear_kernel(tfidf_matrix, tfidf_matrix)
print(cos_sim.shape)
cos_sim[1]
```

    (5000, 5000)

    array([0.0176179 , 1.        , 0.04691288, ..., 0.        , 0.        ,
           0.        ])

```python
# reverse mapping title and index for recommender function
indices = pd.Series(metadt.index, index=metadt['title']).drop_duplicates()
```

```python
def get_recommendation(title, cos_sim=cos_sim):
    ind = indices[title] # find index for title
    sim = list(enumerate(cos_sim[ind])) # enumerate all cosine similarity for the title
    sim = sorted(sim, key=lambda x: x[1], reverse=True) # sorted by second column (cosine similarity)
    sim = sim[1:10] # get top 10 highest similarity
    movie_indices = [ x[0] for x in sim ]
    return metadt['title'].loc[movie_indices]
```

```python
title_sample = metadt.iloc[0]['title']
print(title_sample)
get_recommendation(title_sample)
```

    Toy Story

    2997              Toy Story 2
    1071    Rebel Without a Cause
    3057          Man on the Moon
    485                    Malice
    1932                Condorman
    448         For Love or Money
    1032            The Sunchaser
    2157        Indecent Proposal
    3252          Bound for Glory
    Name: title, dtype: object

## add more feature recommendation

```python
credits = pd.read_csv('./datasets/credits.csv')
keywords = pd.read_csv('./datasets/keywords.csv')
```

```python
def checkInteger(data):
    try:
        int(data)
    except ValueError:
        return True

bad_id = [x for x in metadt.id if checkInteger(x)]
print(bad_id)
index_id = metadt.loc[metadt['id'].isin(bad_id)].index
metadt.drop(index_id, axis=0, inplace=True)
```

    []

```python
keywords['id'] = keywords['id'].astype('int')
credits['id'] = credits['id'].astype('int')
metadt['id'] = metadt['id'].astype('int')

#merge keywords and credits to metadt
metadt = metadt.merge(keywords, on='id')
metadt = metadt.merge(credits, on='id')
metadt.head(3)
```

      adult
      belongs_to_collection
      budget
      genres
      homepage
      id
      imdb_id
      original_language
      original_title
      overview
      ...
      spoken_languages
      status
      tagline
      title
      video
      vote_average
      vote_count
      keywords
      cast
      crew

      0
      False
      {'id': 10194, 'name': 'Toy Story Collection', ...
      30000000
      [{'id': 16, 'name': 'Animation'}, {'id': 35, '...
      http://toystory.disney.com/toy-story
      862
      tt0114709
      en
      Toy Story
      Led by Woody, Andy's toys live happily in his ...
      ...
      [{'iso_639_1': 'en', 'name': 'English'}]
      Released
      NaN
      Toy Story
      False
      7.7
      5415.0
      [{'id': 931, 'name': 'jealousy'}, {'id': 4290,...
      [{'cast_id': 14, 'character': 'Woody (voice)',...
      [{'credit_id': '52fe4284c3a36847f8024f49', 'de...

      1
      False
      NaN
      65000000
      [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...
      NaN
      8844
      tt0113497
      en
      Jumanji
      When siblings Judy and Peter discover an encha...
      ...
      [{'iso_639_1': 'en', 'name': 'English'}, {'iso...
      Released
      Roll the dice and unleash the excitement!
      Jumanji
      False
      6.9
      2413.0
      [{'id': 10090, 'name': 'board game'}, {'id': 1...
      [{'cast_id': 1, 'character': 'Alan Parrish', '...
      [{'credit_id': '52fe44bfc3a36847f80a7cd1', 'de...

      2
      False
      {'id': 119050, 'name': 'Grumpy Old Men Collect...
      0
      [{'id': 10749, 'name': 'Romance'}, {'id': 35, ...
      NaN
      15602
      tt0113228
      en
      Grumpier Old Men
      A family wedding reignites the ancient feud be...
      ...
      [{'iso_639_1': 'en', 'name': 'English'}]
      Released
      Still Yelling. Still Fighting. Still Ready for...
      Grumpier Old Men
      False
      6.5
      92.0
      [{'id': 1495, 'name': 'fishing'}, {'id': 12392...
      [{'cast_id': 2, 'character': 'Max Goldman', 'c...
      [{'credit_id': '52fe466a9251416c75077a89', 'de...

3 rows × 27 columns

```python
# stringified list
from ast import literal_eval

features = ['cast', 'crew', 'keywords', 'genres']
for feature in features:
    metadt[feature] = metadt[feature].apply(literal_eval)
```

```python
import numpy as np
```

```python
def get_director(x):
    for i in x:
        if i['job'] == 'Director':
            return i['name']
    return np.nan
```

```python
# get top 3 from instance (crews, keywords, genres)
def get_list(x):
    if isinstance(x, list):
        names = [i['name'] for i in x]
        if len(names) > 3:
            names = names[:3]
        return names
    return []
```

```python
# apply get_director and get_list
metadt['director'] = metadt['crew'].apply(get_director)

features = ['cast', 'keywords', 'genres']
for feature in features:
    metadt[feature] = metadt[feature].apply(get_list)

metadt[['title', 'cast', 'director', 'keywords', 'genres']].head(3)
```

      title
      cast
      director
      keywords
      genres

      0
      Toy Story
      [Tom Hanks, Tim Allen, Don Rickles]
      John Lasseter
      [jealousy, toy, boy]
      [Animation, Comedy, Family]

      1
      Jumanji
      [Robin Williams, Jonathan Hyde, Kirsten Dunst]
      Joe Johnston
      [board game, disappearance, based on children'...
      [Adventure, Fantasy, Family]

      2
      Grumpier Old Men
      [Walter Matthau, Jack Lemmon, Ann-Margret]
      Howard Deutch
      [fishing, best friend, duringcreditsstinger]
      [Romance, Comedy]

```python
# strip and lower case all string
def clean_data(x):
    if isinstance(x, list):
        return [str.lower(i.replace(' ', '')) for i in x]
    else:
        if isinstance(x, str):
            return str.lower(x.replace(' ', ''))
        else:
            return ''
```

```python
# apply clean data
features = ['cast', 'keywords', 'director', 'genres']

for feature in features:
    metadt[feature] = metadt[feature].apply(clean_data)
```

```python
# combine all data needed to string
def create_soup(x):
    return ' '.join(x['keywords']) + ' ' + ' '.join(x['cast']) + ' ' + x['director'] + ' ' + ' '.join(x['genres'])
```

```python
# apply soup
metadt['soup'] = metadt.apply(create_soup, axis=1)
metadt[['soup']].head(3)
```

      soup

      0
      jealousy toy boy tomhanks timallen donrickles ...

      1
      boardgame disappearance basedonchildren'sbook ...

      2
      fishing bestfriend duringcreditsstinger walter...

```python
# recommending section
# Director, genre, cast on relatively more movies doesn't affect their presence
# so we used CountVectorizer instead of TF-IDF
from sklearn.feature_extraction.text import CountVectorizer
count = CountVectorizer(stop_words='english')
count_matrix = count.fit_transform(metadt['soup'])
count_matrix.shape
```

    (5021, 12184)

```python
# measure distance using cosine_similarity
from sklearn.metrics.pairwise import cosine_similarity

cos_sim2 = cosine_similarity(count_matrix, count_matrix)
```

```python
# reset index and remapping like before
metadt = metadt.reset_index()
indices = pd.Series(metadt.index, index=metadt['title'])
```

```python
# get recommendation
title_sample = metadt.iloc[0]['title']
print(title_sample)
get_recommendation(title_sample, cos_sim2)
```

    Toy Story

    3012             Toy Story 2
    3324       Creature Comforts
    1730        Meet the Deedles
    1123      The Wrong Trousers
    1429         Jungle 2 Jungle
    734            A Close Shave
    405     Addams Family Values
    581                  Aladdin
    608           The Aristocats
    Name: title, dtype: object