<h1 align="center">
	Quantum phase estimation
</h1>

The quantum phase estimation (QPE, also known as PEA) algorithm is a quantum algorithm for estimating the phase of an eigenvector of a unitary operator [[1](https://en.wikipedia.org/wiki/Quantum_phase_estimation_algorithm)]. One of the main applications of the algorithm is to estimate eigenvalues (energies) of some molecule's $H$ Hamiltonian [[2](https://www.tandfonline.com/doi/abs/10.1080/00268976.2011.552441)]. Also, phase estimation is frequently used as a subroutine in other quantum algorithms, such as Shor's algorithm and the quantum algorithm for linear systems of equations (HHL algorithm) [[1](https://en.wikipedia.org/wiki/Quantum_phase_estimation_algorithm)].

The goal of the algorithm is to estimate the eigenvalue of a given Hamiltonian $\hat{H}$. We exponentiate it and obtain $U = e^{i\hat{H}t}$ unitary operator, where $t$ is some parameter (in our example $t = 1$). Then we apply quantum circuit that corresponds to $U = e^{i\hat{H}t}$ on one of the eigenvectors of the $\hat{H}$. For simplicity, we will take such $\hat{H}$, which eigenvectors will be the computational basis states $\left| 0 \right\rangle$ and $\left| 1 \right\rangle$. For that purpose $\hat{H}$ should be a diagonal matrix). In our example, we will try to find eigenvalue that corresponds to $\left| 1 \right\rangle$ eigenvector. More mathematically:

\begin{align}
\hat{H} = \begin{pmatrix}
E_1 & 0\\
0 & E_2
\end{pmatrix} \longrightarrow
e^{i\hat{H}t} = \begin{pmatrix}
e^{i E_1 t} & 0\\
0 & e^{i E_2 t}
\end{pmatrix} \longrightarrow e^{i H t} \left| 1 \right\rangle = e^{i E_2 t} \left| 1 \right\rangle = e^{i 2 \pi \varphi_2} \left| 1 \right\rangle,
\end{align}

where $E_1$ and $E_2$ are the eigenvalues of the $\hat{H}$, $\varphi_2 = \frac{E_2 t}{2 \pi}$ is the phase that the algorithm is capable to estimate. Note that $U = e^{i H t}$ is a unitary operator and in the code, we will name it as ```unitary```. For simplicity, we are considering here diagonal Hermitian operator $\hat{H}$. In general case, it is not diagonal.

The code and the algorithm is similar to the our covered [IQPE](https://github.com/DavitKhach/quantum-algorithms-tutorials/blob/master/iterative_quantum_phase_estimation.ipynb) tutorial. IQPE actually is a modification of QPE algorithm. So, that's why some parts/explanations are omitted here and more details are provided about the parts of QPE that are different from IQPE.

In the first code, all necessary packages are imported.

```python
import numpy as np
from random import random
from qiskit import *
from qiskit.aqua.utils.controlled_circuit import get_controlled_circuit
from qiskit.extensions.standard.barrier import Barrier
```

In the following code the quantum ```circuit``` is created with
* $q$ quantum state register, where we encode the $\left| 1 \right\rangle$ eigenvector by applying ```x``` gate.

* $a$ ancillary quantum register, where, in the course of the algorithm, the bits of the phase $\varphi_2$ will be stored.

* $c$ classical register, where the measurement results will be stored.

```python
num_bits_estimate = 10
# For 2x2 matrix one qubit is enough
q = QuantumRegister(1, name="q")
# In QPE we use n ancillas to estimate n bits from the phase
a = QuantumRegister(num_bits_estimate, name="a")
# For n ancillary qubit measurment we need n cllasical bits
c = ClassicalRegister(num_bits_estimate, name="c")

# Create a quantum circuit
circuit = QuantumCircuit(q, a, c)

# |1> eigenstate initialization
circuit.x(q[0])
```

    <qiskit.circuit.instructionset.InstructionSet at 0x7fb56c871780>

The creation of the circuit for $e^{iHt}$ unitary evolution for a random diagonal Hermitian operator $\hat{H}$ is presented in the code below:

```python
E_1, E_2 = (2 * np.pi * random(), 2 * np.pi * random())
print("We are going to estimate E_2 via QPE algorithm \nE_2 = {}".format(E_2))

# circuit for unitary operator exp(iHt)
t = 1
unitary = QuantumCircuit(q)

unitary.u1(E_2 * t, q[0]) # q[0] is the only qubit in q register
unitary.x(q[0])
unitary.u1(E_1 * t, q[0])
unitary.x(q[0])
```

    We are going to estimate E_2 via QPE algorithm
    E_2 = 0.7294990675852743

    <qiskit.circuit.instructionset.InstructionSet at 0x7fb56c871dd8>

Here we prepare the ancillary qubits by applying Hadamard gates to all of them:

```python
for ancillary in a:
    circuit.h(ancillary)
```

The quantum state of ancillary qubits before Hadamard gates (in the text we will examine 5 ancillary qubit case, but in the code we will use 10 ancillary qubits for better results):

\begin{align*}
\left| \psi_a \right\rangle = \left| 00000 \right\rangle = \left| 0 \right\rangle \left| 0 \right\rangle \left| 0 \right\rangle \left| 0 \right\rangle \left| 0 \right\rangle
\end{align*}

The quantum state of ancillary qubits after Hadamard gates:

\begin{align*}
\left| \psi_a \right\rangle &= \frac{1}{2^{5/2}}
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right) =
\\
&= \left| + \right\rangle \left| + \right\rangle \left| + \right\rangle \left| + \right\rangle \left| + \right\rangle
=\left| +++++ \right\rangle ,
\end{align*}

because $H \left| 0 \right\rangle = \frac{1}{\sqrt{2}}
\left(\left| 0 \right\rangle + \left| 1 \right\rangle \right) = \left| + \right\rangle$. Please don't confuse Hadamard gate $H$ with Hermitian operator $\hat{H}$, because we use the same letter for both of them.

For general case we will represent $\varphi$ in its fractional binary representation:

\begin{align*}
\varphi = 0.\varphi_0 \varphi_1 \varphi_2 \varphi_3... =  \frac{\varphi_0}{2} + \frac{\varphi_1}{2^2} + \frac{\varphi_2}{2^3} + \frac{\varphi_3}{2^4} + ...,
\end{align*}

where each $\varphi_k$ is a bit (it is equal to $0$ or $1$).

Now, for each $n$th ancillary qubit, we apply the controlled version of the circuit of $U = e^{i \hat{H} t}$ $2^{n}$ times ($U^{2^{n}}$) to $q$ quantum register controlled by $n$th ancillary qubit ($n$ starts from $0$ to $4$). Here is the circuit so far:

This will give us a quantum state for ancillary qubits:

\begin{align*}
\left| \psi_a \right\rangle = \frac{1}{2^{5/2}}
&\left(\left| 0 \right\rangle + e^{i 2 \pi \varphi} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 2\varphi} \left| 1 \right\rangle \right)
\\
&\left(\left| 0 \right\rangle + e^{i 2 \pi 4\varphi} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 8\varphi} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 16\varphi} \left| 1 \right\rangle \right).
\end{align*}

By taking into account the fractional binary representation of $\varphi$:

\begin{align*}
\left| \psi_a \right\rangle = \frac{1}{2^{5/2}}
&\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_0 \varphi_1 \varphi_2 \varphi_3 \varphi_4} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_1 \varphi_2 \varphi_3 \varphi_4} \left| 1 \right\rangle \right)
\\
&\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_2 \varphi_3 \varphi_4} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_3 \varphi_4} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_4} \left| 1 \right\rangle \right)
\end{align*}.

Here is the code for the controlled $U^{2^{n}}$:

```python
for n in range(a.size):
    for m in range(2**n):
        get_controlled_circuit(unitary, a[n], circuit)
```

At the next step, we will apply a Hadamard gate to the last $n = 5$ qubit and controlled $u1 \left( -\frac{2 \pi}{2^{(n - m + 1)}} \right)$ gates from $n$th qubit to the rest $m < n$ qubits. Here is the $u1$ gate:

$$
u1(\lambda) =
\begin{pmatrix}
1 & 0 \\
0 & e^{i\lambda}.
\end{pmatrix}
$$

This will give us a quantum state:

\begin{align*}
\left| \psi_a \right\rangle = \frac{1}{2^{5/2}}
&\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_0 \varphi_1 \varphi_2 \varphi_3} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_1 \varphi_2 \varphi_3} \left| 1 \right\rangle \right) \\
&\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_2 \varphi_3} \left| 1 \right\rangle \right)
\left(\left| 0 \right\rangle + e^{i 2 \pi 0.\varphi_3} \left| 1 \right\rangle \right)
\left| \varphi_4 \right\rangle.
\end{align*}

Here we take into account that when $\varphi_4 = 0$ the state of the $5$th qubit is equal to $\left| + \right\rangle$ state and when $\varphi_4 = 1$ it equal to $\left| - \right\rangle = \frac{1}{\sqrt{2}}\left( \left| 0 \right\rangle - \left| 1 \right\rangle \right) $ state. Also note that when we apply Hadamard gate on $5$th qubit, it will give us either

\begin{align*}
H \left| + \right\rangle= \left| 0 \right\rangle
\qquad \text{ or } \qquad
H \left| - \right\rangle = \left| 1 \right\rangle
\end{align*}

We repeat this procedure for all qubits and at the end, the quantum state will look like:

\begin{align*}
\left| \psi_a \right\rangle=
\left| \varphi_0 \right\rangle
\left| \varphi_1 \right\rangle
\left| \varphi_2 \right\rangle
\left| \varphi_3 \right\rangle
\left| \varphi_4 \right\rangle.
\end{align*}

So, by measuring each qubit we will obtain all bits of the phase. This part of the algorithm is the well-known inverse Quantum Fourier transform (inverse QFT) [[3](https://en.wikipedia.org/wiki/Quantum_Fourier_transform)] without SWAP gates. Here is the code that adds this part of the algorithm:

```python
# inverse QFT without SWAP gates
for n in reversed(range(a.size)):
    circuit.h(a[n])
    if n != 0:
        for m in reversed(range(n)):
            angle = -2*np.pi / (2**(n - m + 1))
            circuit.cu1(angle, a[n], a[m])

# measurements on the ancillary qubits stored in c classical register
for n in reversed(range(a.size)):
    circuit.measure(a[n],c[n])
```

The quantum circuit that corresponds to the whole algorithm looks like this (here we draw a circuit that has only 4 ancillary qubits):

At the final piece of code, we run the algorithm on the QC simulator and store measurement results (bits of phase) in $c$ classical register that gives us the estimated value for the phase. After it, by remembering that $E_2 = \frac{2 \pi \varphi_2}{t}$, we obtain desired $E_2$ eigenvalue of the $\hat{H}$ Hamiltonian:

```python
backend = BasicAer.get_backend('qasm_simulator')
shots = 1024  # how many time execute the algorithm
job = execute(circuit, backend, shots=shots)
result = job.result()
counts = result.get_counts()

phase_bits = max(counts, key=counts.get) # take the most often obtaned result

phase = 0
for index, bit in enumerate(reversed(phase_bits)):
    phase += int(bit) / 2**(index + 1)

estimated_E_2 = 2 * np.pi * phase / t

print("Eigenvalue of the Hamiltonian: {}".format(E_2))
print("Estimated eigenvalue of the Hamiltonian: {}".format(estimated_E_2))
```

    Eigenvalue of the Hamiltonian: 0.7294990675852743
    Estimated eigenvalue of the Hamiltonian: 0.7301748550335652

**Final notes**: Here we didn't implement classical textbook version of QPE algorithm. In the textbook [[4](https://www.cambridge.org/am/academic/subjects/physics/quantum-physics-quantum-information-and-quantum-computation/quantum-computation-and-quantum-information-10th-anniversary-edition?format=HB)] the algorithm is done via inverse QFT with SWAP gates. If we want to implement the algorithm with inverse QFT, the code should be changed a little bit. Firstly, the order of control ```unitary``` gates should be changed: $i$th ancillary should control $U^{2^{(n - i - 1)}}$ gate on $q$ register. Secondly, SWAP gates are added after controlled unitaries of the algorithm in order to obtain inverse QFT. Here is the textbook [[4](https://www.cambridge.org/am/academic/subjects/physics/quantum-physics-quantum-information-and-quantum-computation/quantum-computation-and-quantum-information-10th-anniversary-edition?format=HB)] implementation of the algorithm:

where the inverse QFT is the following part of the circuit:

But we did the algorithm's job without SWAP gates. Actually, the SWAP gates don't add anything to this algorithm and we can omit them. Not only they increase the gate number of the circuit, but also they add errors. So, that is why we are avoiding SWAP gates in our implementation.

They are some details about Trotterization, the $t$ parameter in the $U = e^{iHt}$, what should we do when the eigenvalue is negative that we don't cover here. If you interested in these topics you can check our [IQPE](https://github.com/DavitKhach/quantum-algorithms-tutorials/blob/master/iterative_quantum_phase_estimation.ipynb) tutorial, that is a modification of QPE algorithm. There are two main differences between QPE and IQPE algorithms. QPE demands $n$ ancillary qubits for $n$ bit phase estimation, IQPE demands only one ancillary qubit for any $n$ bit phase estimation. The second difference is that QPE applies inverse QFT, meanwhile, IQPE applies inverse QFT iteratively ($u1$ gates are applied iteratively).

[1] [Quantum phase estimation algorithm (wiki)](https://en.wikipedia.org/wiki/Quantum_phase_estimation_algorithm)

[J.D. Whitfield,  J. Biamonte  and  A. Aspuru-Guzik,
Molecular Physics, "Simulation of electronic structure Hamiltonians using
quantum computers" (2011)](https://www.tandfonline.com/doi/abs/10.1080/00268976.2011.552441)

[3] [Quantum Fourier transform (wiki)](https://en.wikipedia.org/wiki/Quantum_Fourier_transform)

[4] [M.A. Nielsen, I.L. Chuang, Cambridge University Press New York, "Quantum Computation and Quantum Information: 10th Anniversary Edition
10th" (2011)](https://www.cambridge.org/am/academic/subjects/physics/quantum-physics-quantum-information-and-quantum-computation/quantum-computation-and-quantum-information-10th-anniversary-edition?format=HB)


### An algorithm to find numerical solution to the final state of an Ising chain with Hamiltonian, ###
### $$ \begin{align} H = \sum_{i<j}^{n} \left [- \frac{1}{2 \left | i-j \right |}\sigma_{i}^{x}\sigma_{j}^{x} -\frac{1}{2}\sigma_{j}^{x} - \frac{1}{2}\sigma_{j}^{z}\right ] \end{align} $$
### where i<j and the interaction $J_{ij} = \frac{1}{\left | i-j \right |}$. ###

```python
from pyquil import Program, get_qc
from pyquil.gates import *
import numpy as np,os
import math
from scipy.linalg import expm
import matplotlib.pyplot as plt
%matplotlib inline
from tqdm import tqdm_notebook,tqdm
from itertools import permutations
from itertools import combinations
from pyquil.paulis import trotterize
from pyquil.paulis import ID, sX, sY, sZ,ID
from pyquil.gates import CNOT, Z, X, Y, H, RZ, RX
from pyquil.api import local_qvm
from pyquil.api import WavefunctionSimulator
#return exact matrix form for the Pauli sum
from pyquil.unitary_tools import lifted_pauli
from pyquil.paulis import PauliSum
output_dir = 'Ising Chain Simu'
if not os.path.exists(output_dir): os.makedirs(output_dir)
pauli_x = [sX(0),sX(1),sX(2),sX(3),sX(4),sX(5),sX(6),sX(7),sX(8)]
pauli_z = [sZ(0),sZ(1),sZ(2),sZ(3),sZ(4),sZ(5),sZ(6),sZ(7),sZ(8)]
```

```python
# Get current size
fig_size = plt.rcParams["figure.figsize"]
# Prints: [6.0, 4.0]
print("Current size:", fig_size)
# Set figure width to 11 and height to 8
fig_size[0] = 11
fig_size[1] = 8
plt.rcParams["figure.figsize"] = fig_size
```

    Current size: [6.0, 4.0]

```python
def convert_row(x,n):
    '''Definition of a function to convert the output
       vector from PyQuil to a ROW vector, this function
       shall be used for writing final state data.
       param x: an output vector.
       param n: number of qubits.'''
    vector = []
    for i in np.arange(2**n):
        key = x[i]
        vector.append(key)
    return vector
```

```python
def ini_state(n):
    '''Definition of a function to generate an initial state
       |00...00> for n qubits.
       param n: numble of qubits'''
    state = 1
    for i in np.arange(n):
        up_col = np.array([[1],[0]])
        state = np.kron(state,up_col)
    return state
```

```python
def convert_col(x,n):
    '''Definition of a function to convert the output
       vector from PyQuil to a COLUMN vector, this function
       shall be used for reading final state data.
       param x: an output vector.
       param n: number of qubits.'''
    vector = []
    for i in np.arange(2**n):
        key = x[i]
        vector.append([key])
    return vector
```

```python
def fidelity(x,y):
    '''Definition of fidelity calculation of vector x and y
       param x: input column vector 1
       param y: input co;lumn vector 2
       function returns a fidility value between 0 and 1'''
    #create some initial values for later iteration of addition
    norm_x = 0
    norm_y = 0
    #loops to compute normalisation constants for both vector x
    #and y respectively
    for i in np.arange(len(x)):
        norm_x = norm_x + abs(x[i,0])**2
    for i in np.arange(len(y)):
        norm_y = norm_y + abs(y[i,0])**2
    #now normalise the vectors
    x_norm = x/np.sqrt(norm_x)
    y_norm = y/np.sqrt(norm_y)
    #compute Hermitian conjugate of vector x
    x_norm_trans = np.matrix.conjugate(x_norm.transpose())
    #compute inner product of vector x and y
    inner_pro = x_norm_trans.dot(y_norm)
    #to obtain a value for the fidility
    fid = (abs(inner_pro)[0,0])**2
    return fid
```

```python
def intera_J(n):
    '''Definition of a function to compute the interaction
       between site i and j and return a list of value for
       1/|i-j|
       param n: number of qubits (system size).
       output: an array of interaction strength values for
       each coupling term.
       '''
    index = np.array(list(combinations(range(n),2)))
    no_of_comb = len(index)
    #collect values for interaction strength to form a list
    lis = []
    for i in np.arange(no_of_comb):
        diff = abs(index[i,0] - index[i,1])
        factor = 1.0/diff
        lis.append(factor)
    lis_factor = np.array(lis)
    return lis_factor
```

```python
def comb_xx(n):
    '''Definition of a function to find all combinations
       of the Pauli terms in the XX part of the Ising-like
       Hamiltonian for n qubits to be digitally simulated.
       Each of the terms is of the form sX(i)*sX(j).
       param n: number of qubits.
       output: an array of all Paulis terms.'''
    index = np.array(list(combinations(range(n),2)))
    no_of_comb = len(index)
    comb_xx = []
    for i in np.arange(no_of_comb):
        key1 = index[i][0]
        key2 = index[i][1]
        comb_xx.append(pauli_x[key1]*pauli_x[key2]*intera_J(n)[i])
    return comb_xx
```

```python
def comb_z(n):
    '''Definition of a function to find all combinations
       of the Pauli terms in the field interaction along
       z direction part of the Ising-like Hamiltonian for
       n qubits to be digitally simulated. Each of the
       terms is of the form sZ(j).
       param n: number of qubits.
       output: an array of all Pauli Z terms.'''
    index = np.array(list(combinations(range(n),2)))
    no_of_comb = len(index)
    comb_z = []
    for i in np.arange(no_of_comb):
        key = index[i][1]
        comb_z.append(pauli_z[key])
    return comb_z
```

```python
def comb_x(n):
    '''Definition of a function to find all combinations
       of the Pauli terms in the field interaction along
       x direction part of the Ising-like Hamiltonian for
       n qubits to be digitally simulated. Each of the
       terms is of the form sX(j).
       param n: number of qubits.
       output: an array of all Pauli Z terms.'''
    index = np.array(list(combinations(range(n),2)))
    no_of_comb = len(index)
    comb_x = []
    for i in np.arange(no_of_comb):
        key = index[i][1]
        comb_x.append(pauli_x[key])
    return comb_x
```

### ${\color{Blue}{\mathit{\text{Part I i):}}}}$ The classical analogue simulation.

```python
def theo_Hamil(n,t):
    '''Difinition of a function to compute and return theoretical
       Hamiltonian for n qubits with (choosing-2-out-of-n)
       coupling terms in matrix form for evolution time t.
       param n: number of qubits.
       param t: total evolution time'''
    factor = 0.5
    # number of combination for n qubits
    length = np.arange(len(comb_xx(n)))
    # compute the xx interaction part of the Hamiltonian
    part_xx = [comb_xx(n)[i]*t*factor for i in length]
    # compute the x field interaction part of the Hamiltonian
    part_x = [x*factor*t for x in comb_x(n)]
    # compute the z field interaction part of the Hamiltonian
    part_z = [x*factor*t for x in comb_z(n)]
    # sum up all Pauli terms in the part_xx
    pauli_sum_xx = PauliSum(part_xx)
    # sum up all Pauli terms in the part_x
    pauli_sum_x = PauliSum(part_x)
    # sum up all Pauli terms in the part_z
    pauli_sum_z = PauliSum(part_z)
    # acquire a list of qubit indices for the applied
    # Pauli operators
    list_qbt = PauliSum.get_qubits(pauli_sum_xx)
    # add up the final matrices of both the pauli_xx
    # pauli_z to obtain the Hamiltonian in its matrix
    # form
    Hamil = lifted_pauli(pauli_sum_xx,list_qbt) + lifted_pauli(pauli_sum_z,list_qbt) \
            + lifted_pauli(pauli_sum_x,list_qbt)
    return Hamil
```

```python
n = 2  # no of qubits
t = 0.1 # evolution time
theo_9qbt = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
```

### ${\color{Blue}{\mathit{\text{Part I: ii)}}}}$ The virtual digital simulation.

#### The first order Trotter approximant to $e^{t(A+B+C)}$  is given by, ###
### $e^{t(A+B+C)} = e^{t(A+B)}e^{tC} = e^{tA}e^{tB}e^{tC}$. ###
#### We let $A = \sigma^{x}\sigma^{x}, B = \sigma^{x}, C = \sigma^{z}$. Obviously, operators A and B are mutually commutative and therefore they are seperable.  ###

```python
def Ising_prog_1(n,t):
    '''Definition of a function to obtain a sequence of gates
       for an Ising chain with H = -0.5*(1/|i-j|)*sigma_x_i*sigma_x_j
       - 0.5*sigma_x_j - 0.5*sigma_z_j where index i j run from 0th
       to (n-1)th site and i is always smaller than j. Here 1st-order
       Trotter approximant is applied.
       param n: number of qubits (n>=2).
       param t: evolution time.'''
    from pyquil.paulis import exponentiate
    factor = -0.5
    prog = Program()
    # The coefficients of exponential operators
    for i in np.arange(len(comb_xx(n))):
        prog_ini = Program()
        prog_ini += trotterize(t*factor*comb_xx(n)[i],t*factor*comb_x(n)[i],trotter_order=1)
        prog_ini += exponentiate(t*factor*comb_z(n)[i])
        prog += prog_ini
    return prog
```

#### The second order Trotter approximant to $e^{t(A+B+C)}$  is given by, ###
### $e^{t(A+B+C)} = e^{\frac{t}{2}(A+B)}e^{tC}e^{\frac{t}{2}(A+B)} = e^{\frac{t}{2}A}e^{\frac{t}{2}B}e^{tC}e^{\frac{t}{2}A}e^{\frac{t}{2}B} $. ###
#### We let $A = \sigma^{x}\sigma^{x}, B = \sigma^{x}, C = \sigma^{z}$. Obviously, operators A and B are mutually commutative and therefore they are seperable.  ###

```python
def Ising_prog_2(n,t):
    '''Definition of a function to obtain a sequence of gates
       for an Ising chain with H = -0.5*(1/|i-j|)*sigma_x_i*sigma_x_j
       - 0.5*sigma_x_j - 0.5*sigma_z_j where index i j run from 0th
       to (n-1)th site and i is always smaller than j. Here 2nd-order
       Trotter approximant is applied.
       param n: number of qubits (n>=2).
       param t: evolution time.'''
    from pyquil.paulis import exponentiate
    factor = -0.5
    prog = Program()
    # The coefficients of exponential operators
    coef_lis = [1/2,1,1/2]
    for i in np.arange(len(comb_xx(n))):
        prog_ini = Program()
        prog_ini += trotterize(t*factor*coef_lis[0]*comb_xx(n)[i],t*factor*coef_lis[0]*comb_x(n)[i],trotter_order=2)
        prog_ini += exponentiate(t*factor*coef_lis[1]*comb_z(n)[i])
        prog_ini += trotterize(t*factor*coef_lis[2]*comb_xx(n)[i],t*factor*coef_lis[2]*comb_x(n)[i],trotter_order=2)
        prog += prog_ini
    return prog
```

#### The third order Trotter approximant to $e^{t(A+B+C)}$  is given by, ###
### $e^{t(A+B+C)} = e^{\frac{7}{24}t(A+B)}e^{\frac{2}{3}tC}e^{\frac{3}{4}t(A+B)}e^{-\frac{2}{3}tC}e^{-\frac{1}{24}t(A+B)}e^{tC} $. ###
#### We let $A = \sigma^{x}\sigma^{x}, B = \sigma^{x}, C = \sigma^{z}$. Obviously, operators A and B are mutually commutative and therefore they are seperable.  ###

```python
def Ising_prog_3(n,t):
    '''Definition of a function to obtain a sequence of gates
       for an Ising chain with H = -0.5*(1/|i-j|)*sigma_x_i*sigma_x_j
       - 0.5*sigma_x_j - 0.5*sigma_z_j where index i j run from 0th
       to (n-1)th site and i is always smaller than j. Here 3rd-order
       Trotter approximant is applied.
       param n: number of qubits (n>=2).
       param t: evolution time.'''
    from pyquil.paulis import exponentiate
    factor = -0.5
    prog = Program()
    # The coefficients of exponential operators
    coef_lis = [7/24, 2/3, 3/4, -2/3, -1/24, 1]
    for i in np.arange(len(comb_xx(n))):
        prog_ini = Program()
        prog_ini += trotterize(t*factor*coef_lis[0]*comb_xx(n)[i],t*factor*coef_lis[0]*comb_x(n)[i],trotter_order=3)
        prog_ini += exponentiate(t*factor*coef_lis[1]*comb_z(n)[i])
        prog_ini += trotterize(t*factor*coef_lis[2]*comb_xx(n)[i],t*factor*coef_lis[2]*comb_x(n)[i],trotter_order=3)
        prog_ini += exponentiate(t*factor*coef_lis[3]*comb_z(n)[i])
        prog_ini += trotterize(t*factor*coef_lis[4]*comb_xx(n)[i],t*factor*coef_lis[4]*comb_x(n)[i],trotter_order=3)
        prog_ini += exponentiate(t*factor*coef_lis[5]*comb_z(n)[i])
        prog += prog_ini
    return prog
```

#### The fourth order Trotter approximant to $e^{t(A+B+C)}$  is given by, ###
### $e^{t(A+B+C)} = e^{\frac{k_{2}}{2}t(A+B)}e^{k_{2}tC}e^{k_{2}t(A+B)}e^{k_{2}tC}e^{\frac{1-3k_{2}}{2}t(A+B)}e^{(1-4k_{2}tC)}e^{\frac{1-3k_{2}}{2}t(A+B)}e^{k_{2}tC}e^{k_{2}t(A+B)}e^{k_{2}tC}e^{\frac{k_{2}}{2}t(A+B)}$, ###
#### where $k_{2} = \frac{1}{4-\sqrt[4]{3}}$.  We let $A = \sigma^{x}\sigma^{x}, B = \sigma^{x}, C = \sigma^{z}$. Obviously, operators A and B are mutually commutative and therefore they are seperable. ###

```python
def Ising_prog_4(n,t):
    '''Definition of a function to obtain a sequence of gates
       for an Ising chain with H = -0.5*(1/|i-j|)*sigma_x_i*sigma_x_j
       - 0.5*sigma_x_j - 0.5*sigma_z_j where index i j run from 0th
       to (n-1)th site and i is always smaller than j. Here 4th-order
       Trotter approximant is applied.
       param n: number of qubits (n>=2).
       param t: evolution time.'''
    from pyquil.paulis import exponentiate
    factor = -0.5
    prog = Program()
    # The coefficient k2
    coef_k = 1/(4-4**(1/3))
    for i in np.arange(len(comb_xx(n))):
        prog_ini = Program()
        prog_ini += trotterize(t*factor*(coef_k/2)*comb_xx(n)[i],t*factor*(coef_k/2)*comb_x(n)[i],trotter_order=4)
        prog_ini += exponentiate(t*factor*coef_k*comb_z(n)[i])
        prog_ini += trotterize(t*factor*coef_k*comb_xx(n)[i],t*factor*coef_k*comb_x(n)[i],trotter_order=4)
        prog_ini += exponentiate(t*factor*coef_k*comb_z(n)[i])
        prog_ini += trotterize(t*factor*((1-3*coef_k)/2)*comb_xx(n)[i],t*factor*((1-3*coef_k)/2)*comb_x(n)[i],trotter_order=4)
        prog_ini += exponentiate(t*factor*(1-4*coef_k)*comb_z(n)[i])
        prog_ini += trotterize(t*factor*((1-3*coef_k)/2)*comb_xx(n)[i],t*factor*((1-3*coef_k)/2)*comb_x(n)[i],trotter_order=4)
        prog_ini += exponentiate(t*factor*coef_k*comb_z(n)[i])
        prog_ini += trotterize(t*factor*coef_k*comb_xx(n)[i],t*factor*coef_k*comb_x(n)[i],trotter_order=4)
        prog_ini += exponentiate(t*factor*coef_k*comb_z(n)[i])
        prog_ini += trotterize(t*factor*(coef_k/2)*comb_xx(n)[i],t*factor*(coef_k/2)*comb_x(n)[i],trotter_order=4)
        prog += prog_ini
    return prog
```

```python
wavefunction_simulator = WavefunctionSimulator()
wavefunc_var = wavefunction_simulator.wavefunction(Ising_prog_4(n,t))
simu_qbt = np.array(convert_col(wavefunc_var,n))
fidelity(theo_9qbt,simu_qbt)
#simu_qbt
```

    1.0

```python
fid_qbt_var_1 = []
for n in tqdm_notebook(np.arange(2,10)):
    t = 0.1
    theo_qbt_var = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
    wavefunction_simulator = WavefunctionSimulator()
    wavefunc_var_1 = wavefunction_simulator.wavefunction(Ising_prog_1(n,t))
    simu_qbt_1 = np.array(convert_col(wavefunc_var_1,n))
    fid_qbt_1 = fidelity(theo_qbt_var,simu_qbt_1)
    fid_qbt_var_1.append(fid_qbt_1)
```

    HBox(children=(IntProgress(value=0, max=8), HTML(value='')))

```python
fid_qbt_var_2 = []
for n in tqdm_notebook(np.arange(2,10)):
    t = 0.1
    theo_qbt_var = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
    wavefunction_simulator = WavefunctionSimulator()
    wavefunc_var_2 = wavefunction_simulator.wavefunction(Ising_prog_2(n,t))
    simu_qbt_2 = np.array(convert_col(wavefunc_var_2,n))
    fid_qbt_2 = fidelity(theo_qbt_var,simu_qbt_2)
    fid_qbt_var_2.append(fid_qbt_2)
```

    HBox(children=(IntProgress(value=0, max=8), HTML(value='')))

```python
fid_qbt_var_3 = []
for n in tqdm_notebook(np.arange(2,10)):
    t = 0.1
    theo_qbt_var = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
    wavefunction_simulator = WavefunctionSimulator()
    wavefunc_var_3 = wavefunction_simulator.wavefunction(Ising_prog_3(n,t))
    simu_qbt_3 = np.array(convert_col(wavefunc_var_3,n))
    fid_qbt_3 = fidelity(theo_qbt_var,simu_qbt_3)
    fid_qbt_var_3.append(fid_qbt_3)
```

    HBox(children=(IntProgress(value=0, max=8), HTML(value='')))

```python
fid_qbt_var_4 = []
for n in tqdm_notebook(np.arange(2,10)):
    t = 0.1
    theo_qbt_var = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
    wavefunction_simulator = WavefunctionSimulator()
    wavefunc_var_4 = wavefunction_simulator.wavefunction(Ising_prog_4(n,t))
    simu_qbt_4 = np.array(convert_col(wavefunc_var_4,n))
    fid_qbt_4 = fidelity(theo_qbt_var,simu_qbt_4)
    fid_qbt_var_4.append(fid_qbt_4)
```

    HBox(children=(IntProgress(value=0, max=8), HTML(value='')))

```python
plt.grid(True)
no_of_steps = np.arange(1,101)
plt.plot(np.arange(2,10),fid_qbt_var_1, '-', label='1st-order Trotter')
plt.plot(np.arange(2,10),fid_qbt_var_2, '-', label='2nd-order Trotter')
plt.plot(np.arange(2,10),fid_qbt_var_3, '-', label='3rd-order Trotter')
plt.plot(np.arange(2,10),fid_qbt_var_4, '-', label='4th-order Trotter')
plt.xlabel('System size')
plt.ylabel('Fidelity')
plt.legend(loc = "center left")
#plt.savefig(output_dir+'/fid_vs_qubits_and_time_plots/'
#            +'fid_vs_no_of_qbt_with_simu_t_01', dpi=100)
```

Explanation for seperation between the first-order and the higher-order Trotter approximants behaving parabolically: initially, when system size is relatively small, the seperation increases because weight of correction terms tends to be more significant than weight of commutators of non-commutative Pauli terms; however, when system size is relatively large, which contributes to a binomially increased number of non-commutative Pauli terms, the seperation then decreases because weight of corrrection terms tends to be overwhelmed by weight of the commutators. This means, in this very situation, that approximation delivered by the higher-order Trotter approximants is asymptotically as poor as that by the first-order.

#### Now we implement coefficient of correlation to investigate the closeness of the Trotter approximants over increased evolution time.

### ${\color{Blue}{\mathit{\text{Part II:}}}}$ Discretization of evolution time.

#### We first set total evolution time t = 1s.

```python
def discre_of_time(n,t,no_of_step):
    '''Definition of a function to implement discretization of time
       for an Ising chain with Hamiltonian H =
       -0.5*(1/|i-j|)*sigma_x_i*sigma_x_j - 0.5*sigma_x_j - 0.5*sigma_z_j
       where index i j run from 0th to (n-1)th site and i is always smaller
       than j. Here 2nd-order Trotter approximant is applied. Function returns
       a wavefunc after evolution time of 1s with no_of_step number of Trotter
       steps implemented.
       param n: number of qubits (n>=2).
       param t: total evolution time.
       param no_of_step: number of Trotter steps.'''
    wavefunction_simulator = WavefunctionSimulator()
    prog_ini = Program()
    prog = Ising_prog_2(n,t/no_of_step)
    for i in np.arange(no_of_step):
        prog_ini += prog
    wavefunc = wavefunction_simulator.wavefunction(prog_ini)
    return wavefunc
```

```python
interval = [  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
        14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,
        27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,
        40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,
        53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,  65,
        66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,  78,
        79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,
        92,  93,  94,  95,  96,  97,  98,  99, 100]
```

```python
n = 9 # system size
t = 1 # evolution time
fid_9qbt_demo_1 = []
theo_9qbt_demo_1 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_1,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_1 = np.append(fid_9qbt_demo_1,key)
```

    HBox(children=(IntProgress(value=0), HTML(value='')))

```python
n = 9 # system size
t = 2 # evolution time
fid_9qbt_demo_2 = []
theo_9qbt_demo_2 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_2,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_2 = np.append(fid_9qbt_demo_2,key)
```

    HBox(children=(IntProgress(value=0), HTML(value='')))

```python
n = 9 # system size
t = 3 # evolution time
fid_9qbt_demo_3 = []
theo_9qbt_demo_3 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_3,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_3 = np.append(fid_9qbt_demo_3,key)
```

    HBox(children=(IntProgress(value=0), HTML(value='')))

```python
n = 9 # system size
t = 4 # evolution time
fid_9qbt_demo_4 = []
theo_9qbt_demo_4 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_4,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_4 = np.append(fid_9qbt_demo_4,key)
```

    HBox(children=(IntProgress(value=0), HTML(value='')))

```python
n = 9 # system size
t = 5 # evolution time
fid_9qbt_demo_5 = []
theo_9qbt_demo_5 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_5,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_5 = np.append(fid_9qbt_demo_5,key)
```

```python
n = 9 # system size
t = 6 # evolution time
fid_9qbt_demo_6 = []
theo_9qbt_demo_6 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_6,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_6 = np.append(fid_9qbt_demo_6,key)
```

```python
n = 9 # system size
t = 7 # evolution time
fid_9qbt_demo_7 = []
theo_9qbt_demo_7 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_7,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_7 = np.append(fid_9qbt_demo_7,key)
```

```python
n = 9 # system size
t = 8 # evolution time
fid_9qbt_demo_8 = []
theo_9qbt_demo_8 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_8,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_8 = np.append(fid_9qbt_demo_8,key)
```

```python
n = 9 # system size
t = 9 # evolution time
fid_9qbt_demo_9 = []
theo_9qbt_demo_9 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_9,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_9 = np.append(fid_9qbt_demo_9,key)
```

```python
n = 9 # system size
t = 10 # evolution time
fid_9qbt_demo_10 = []
theo_9qbt_demo_10 = expm(1j*theo_Hamil(n,t)).dot(ini_state(n))
for i in tqdm_notebook(interval):
    key = fidelity(theo_9qbt_demo_10,np.array(convert_col(discre_of_time(n,t,i),n)))
    fid_9qbt_demo_10 = np.append(fid_9qbt_demo_10,key)
```

    HBox(children=(IntProgress(value=0), HTML(value='')))

```python
plt.grid(True)
no_of_steps = np.arange(1,101)
plt.plot(no_of_steps, fid_9qbt_demo_1, '-', label = 't = 1')
plt.plot(no_of_steps, fid_9qbt_demo_2, '-', label = 't = 2')
plt.plot(no_of_steps, fid_9qbt_demo_3, '-', label = 't = 3')
plt.plot(no_of_steps, fid_9qbt_demo_4, '-', label = 't = 4')
plt.plot(no_of_steps, fid_9qbt_demo_10, '-', label = 't = 10')
plt.xlabel('Number of Trotter steps')
plt.ylabel('Fidelity')
plt.legend(loc = "lower right")
#plt.savefig(output_dir+'/fid_demo_1_to_100_steps_plots/'
#            +'fid_vs_steps_1to100_simu_t_10', dpi=100)
```

```python

```


# Examples using the FuzzySet class

# Pre-configuration

```python
# Path
import sys
import os
qprc_path = os.path.abspath(os.path.join('..'))
if qprc_path not in sys.path:
    sys.path.append(qprc_path)
```

# Global imports

```python
import numpy as np
from kernelfuzzy.fuzzyset import FuzzySet
from kernelfuzzy.memberships import gaussmf
```

# Use case: without explicit membership function

In this block we have 3 examples of use of the FuzzySet class using pre-defined membership degrees, instead of using a membership function.

## Example 1

The first example consists of a set with 1-dimension elements. In this case, we are going to use strings.

```python
elements = ["apple", "banana", "cherry"]
membership_degrees = [.1, .5, .1]

X = FuzzySet(elements, md=membership_degrees)
X.show_class()

print('\n')
print("Fuzzy set: ", X.get_pair())
```

    (_elements)
     ['apple', 'banana', 'cherry']

    (_elements_type)
     <class 'str'>

    (_mf)
     None

    (_md)
     [0.1, 0.5, 0.1]

    (_params)
     None

    Fuzzy set:  [('apple', 0.1), ('banana', 0.5), ('cherry', 0.1)]

## Example 2

In this block, the elements are 2-dimensional NumPy arrays.

```python
elements = [np.array([2,3]), np.array([4,5]), np.array([3,7])]
membership_degrees = [.1, .5, .1]

X = FuzzySet(elements, md=membership_degrees)
X.show_class()

print('\n')
print("Fuzzy set: ", X.get_pair())
```

    (_elements)
     [array([2, 3]), array([4, 5]), array([3, 7])]

    (_elements_type)
     <class 'numpy.ndarray'>

    (_mf)
     None

    (_md)
     [0.1, 0.5, 0.1]

    (_params)
     None

    Fuzzy set:  [(array([2, 3]), 0.1), (array([4, 5]), 0.5), (array([3, 7]), 0.1)]

## Example 3

This example is the same as the example 2, except that now we are using the built-in lists from Python.

```python
elements = [[2,3], [4,5], [3,7]]
membership_degrees = [.1, .5, .1]

X = FuzzySet(elements, md=membership_degrees)
X.show_class()

print('\n')
print("Fuzzy set", X.get_pair())
```

    (_elements)
     [[2, 3], [4, 5], [3, 7]]

    (_elements_type)
     <class 'list'>

    (_mf)
     None

    (_md)
     [0.1, 0.5, 0.1]

    (_params)
     None

    Fuzzy set [([2, 3], 0.1), ([4, 5], 0.5), ([3, 7], 0.1)]

# Use case: explicit membership function

In this block we have a example of use of the FuzzySet class, using a membership function.

# Example 1

Among the various membership functions, we are going to use the Gaussian membership function, for multivariated sets, which is implemented as:

\begin{align}
    f(x)=e^{(-(x-\mu)^TA(x-\mu))}
\end{align}

Observe that __only the function signature is used__. Their parameters are separately passed for the _FuzzySet_ class.

```python
elements = np.random.uniform(0, 100, 3)
mf = gaussmf
params_mf = [np.mean(elements), np.std(elements)]

X = FuzzySet(elements, mf=mf, params=params_mf)
X.show_class()

print('\n')
print("Fuzzy set", X.get_pair())
```

    (_elements)
     [78.12341389 57.3660294  47.0315784 ]

    (_elements_type)
     <class 'numpy.float64'>

    (_mf)
     <function gaussmf at 0x7faf2c066158>

    (_md)
     [0.16745862 0.93033128 0.319574  ]

    (_params)
     [60.840340565790605, 12.928745417165153]

    Fuzzy set [(78.12341389378197, 0.16745861528767214), (57.3660293998024, 0.9303312799613191), (47.03157840378746, 0.319573997601598)]

# Use case: explicit membership function, with elements in $\mathbb{R}^2$

The _FuzzySet_ class supports the membership calculation of multi-dimensional sets. In this example, a 2-dimensional set is used.

```python
covar = np.array([[1, 0], [0, 1]])
mean = np.random.uniform(0, 100, 2)
elements = np.random.multivariate_normal(mean, covar, 10)

X = FuzzySet(elements, mf=gaussmf, params=[mean, covar])
X.show_class()

print('\n')
print("Fuzzy set", X.get_pair())
```

    (_elements)
     [[56.83957818 53.65880728]
     [55.52585483 52.91772526]
     [55.39087803 52.37236829]
     [56.14117182 53.11222581]
     [55.18414538 53.43017777]
     [54.3317501  52.71268461]
     [56.14912484 54.21089931]
     [54.79413517 54.61365357]
     [56.878515   52.54680145]
     [55.83611427 53.52920781]]

    (_elements_type)
     <class 'numpy.ndarray'>

    (_mf)
     <function gaussmf at 0x7faf2c066158>

    (_md)
     [0.24045758 0.74148556 0.29253202 0.7125924  0.79353762 0.09870101
     0.44103608 0.12005959 0.10197481 0.96447243]

    (_params)
     [array([55.66475079, 53.4466936 ]), array([[1, 0],
           [0, 1]])]

    Fuzzy set [(array([56.83957818, 53.65880728]), 0.24045757775306212), (array([55.52585483, 52.91772526]), 0.7414855552742433), (array([55.39087803, 52.37236829]), 0.29253201572182397), (array([56.14117182, 53.11222581]), 0.7125923955443285), (array([55.18414538, 53.43017777]), 0.7935376174047657), (array([54.3317501 , 52.71268461]), 0.09870101455965256), (array([56.14912484, 54.21089931]), 0.4410360802353732), (array([54.79413517, 54.61365357]), 0.12005958528158023), (array([56.878515  , 52.54680145]), 0.10197481136543422), (array([55.83611427, 53.52920781]), 0.9644724257183609)]


# Financial bubble prediction using the LPPL model

### Wolfgang Plaschg, wolfgang.plaschg@gmail.com

LPPL model

$$ \mathbb{E}[\ln(p(t))] = A + B(t_c - t)^m + C(t_c - t)^m\cos(\omega\ln(t_c - t) - \phi) $$

with

$$
\begin{align}
0.1 \leq{} m & \leq 0.9 \\
6 \leq{}  \omega & \leq 13 \\
 |C\,| & \le 1 \\
 B & \le 0
\end{align}
$$

A model example:

A data example:

We want to fit a model for a given data:

The original model has 7 parameters $A$, $B$, $C$, $m$, $\omega$, $\phi$ and $t_c$ but can be reduced to a non-linear optimization problem in 3 variables:

Example: Bitcoin bubble

```python
%pylab inline
import scipy
import pandas as pd

# disable warnings
np.seterr(divide='ignore', invalid='ignore', over='ignore')
```

    Populating the interactive namespace from numpy and matplotlib

    {'divide': 'warn', 'invalid': 'warn', 'over': 'warn', 'under': 'ignore'}

```python
DATA_SIZE = 400
NOISE_FACTOR = 0.5
CUTOFF = 0.8
```

## Generate a model and test data

```python
# LPPL 4 factor model

tc = 6
m = np.random.uniform(0.1, 0.9)     # 0.1 <= m <= 0.9
omega = np.random.uniform(6, 13)  # 6 <= omega <= 13

C = abs(np.random.normal())    # |C| < 1
B = np.random.uniform(-10, 0)      # B < 0
A = 200
phi = 10

t = np.linspace(0, tc, num=DATA_SIZE)
line_data = A + B * (tc - t) ** m + C * (tc - t) ** m * np.cos(omega * np.log(tc - t) - phi)
line_data_index = np.linspace(0, tc, len(line_data))
log_prices = [x + np.random.normal(0, NOISE_FACTOR) for x in line_data]
log_prices = log_prices[:int(DATA_SIZE * CUTOFF)]
t_cutoff = t[:int(DATA_SIZE * CUTOFF)]
factor = 1 / max(t_cutoff)
t_cutoff = t_cutoff * factor
line_data_index = line_data_index * factor
t = t * factor
tc = max(t)

simulated_data = pd.Series(data=log_prices, index=t_cutoff)

print("tc: %.2f" % tc)
plot(simulated_data, '.')
# plot(line_data_index, line_data, 'b-')
```

## Generate test data using geometric brownian motion

```python
# x0 = start value, mu = drift, sigma = volatility

def make_gbm_data(x0=200, mu=0.8, sigma=0.6, data_size=DATA_SIZE):
    n = int(data_size * CUTOFF)
    dt = 1/n
    x = pd.DataFrame()
    t = np.linspace(0, 1, n)
    step = np.exp((mu - sigma**2 / 2) * dt) * np.exp(sigma * np.random.normal(0, np.sqrt(dt), (1, n)))
    return pd.Series(data = x0 * step.cumprod(), index=t)

gbm_data = make_gbm_data()

plot(gbm_data, '.')
```

```python
# LPPL 3 factor model

C1 = C * np.cos(phi)
C2 = C * np.sin(phi)

line_data = A + B * (tc - t) ** m + C1 * (tc - t) ** m * np.cos(omega * np.log(tc - t)) + \
            C2 * (tc - t) ** m * np.sin(omega * np.log(tc - t))
line_data_index = np.linspace(0, tc, len(line_data))
log_prices = [x + np.random.normal(0, NOISE_FACTOR) for x in line_data]
log_prices = log_prices[:int(DATA_SIZE * CUTOFF)]
t_cutoff = t[:int(DATA_SIZE * CUTOFF)]
factor = 1 / max(t_cutoff)
t_cutoff = t_cutoff * factor
line_data_index = line_data_index * factor
t = t * factor
tc = max(t)

simulated_data = pd.Series(data=log_prices, index=t_cutoff)

print("tc: %.2f" % tc)
plot(simulated_data, '.')
plot(line_data_index, line_data, 'b-')
```

```python
# delete data points
import copy
cutout_data = copy.deepcopy(simulated_data)
del_ival = []
for i in range(len(cutout_data.index)):
    ival = cutout_data.index[i]
    if ival >= 0.4 and ival <= 0.7:
        del_ival.append(ival)
for ival in del_ival:
    del cutout_data[ival]

plot(cutout_data, '.')
plot(line_data_index, line_data, 'b-')
```

```python
def reduce_data(data, target_size):
    while len(data) > target_size:
        del_keys = np.random.choice(data.index, len(data.index) - target_size)
        removed = 0
        for k in del_keys:
            try:
                del data[k]
                removed += 1
            except:
                pass
reduced_data = gbm_data.copy()
reduce_data(reduced_data, 159)
plot(gbm_data, '.')
figure()
plot(reduced_data, '.')
```

```python
x = gbm_data.values

x1 = min(gbm_data.values)
x2 = max(gbm_data.values)
b = (x1 + x2) / (x1 - x2)
a = (-1 - b) / x1
scaled = np.array(gbm_data.values) * a + b

data = line_data[:int(DATA_SIZE * CUTOFF)] + scaled * 1.1
gbm_sim_data = pd.Series(data, index=t_cutoff)

plot(gbm_sim_data.values, '.')
plot(line_data, 'b-')
```

```python
def F1_get_linear_parameters(X, stock_data):
    tc, m, omega = X

    t = np.array(stock_data.index)
    y = np.array(stock_data.values)

    N = len(stock_data)
    f = (tc - t) ** m
    g = (tc - t) ** m * np.cos(omega * np.log(tc - t))
    h = (tc - t) ** m * np.sin(omega * np.log(tc - t))

    LHS = np.array([[N,      sum(f),    sum(g),    sum(h)   ],
                    [sum(f), sum(f**2), sum(f*g),  sum(f*h) ],
                    [sum(g), sum(f*g),  sum(g**2), sum(g*h) ],
                    [sum(h), sum(f*h),  sum(g*h),  sum(h**2)]])

    RHS = np.array([[sum(y)],
                    [sum(y*f)],
                    [sum(y*g)],
                    [sum(y*h)]])

    A, B, C1, C2 = np.linalg.solve(LHS, RHS)
    return A, B, C1, C2

def F1(X, stock_data):
    tc, m, omega = X
    t = np.array(stock_data.index)
    y = np.array(stock_data.values)
    A, B, C1, C2 = F1_get_linear_parameters(X, stock_data)
    error = y - A - B * (tc - t) ** m - C1 * (tc - t) ** m * np.cos(omega * np.log(tc - t)) - \
                C2 * (tc - t) ** m * np.sin(omega * np.log(tc - t))
    cost = sum(error ** 2)
    return cost

def F1_normalized(result, stock_data):
    x1 = min(stock_data.values)
    x2 = max(stock_data.values)
    b = (x1 + x2) / (x1 - x2)
    a = (-1 - b) / x1
    data = np.array(stock_data.values) * a + b
    stock_data_norm = pd.Series(data=data, index=stock_data.index)
    return F1(result, stock_data_norm)

from scipy import optimize

class Result:
    def __init__(self):
        self.success = None

        # model parameters
        self.tc = None
        self.m = None
        self.omega = None
        self.A = None
        self.B = None
        self.C1 = None
        self.C2 = None
        self.C = None
        self.pruned = None   # True if one of the parameters has been pruned to the
                             # valid range after fitting.
        self.price_tc = None  # Estimated price at tc
        self.price_chg = None  # Price difference between est. price at tc and last price in percent

        self.mse = None  # mean square error
        self.mse_hist = []  # history of mean square errors
        self.norm_mse = None  # normalized mean square error
        self.opt_rv = None  # Return object from optimize function
        self.tc_start = []
        self.m_start = []
        self.omega_start = []

def LPPL_fit(data, tries=20, min_distance=0.2):
    rv = Result()
    fitted_parameters = None
    mse_min = None
    fitted_pruned = False

    tc_min, tc_max = 1, 1.6  # Critical time
    m_min, m_max = 0.1, 0.5  # Convexity: smaller is more convex
    omega_min, omega_max = 6, 13  # Number of oscillations

    # Scaling parameters to scale tc, m and omega to range 0 .. 1

    tc_scale_b = tc_min / (tc_min - tc_max)
    tc_scale_a = -tc_scale_b / tc_min

    m_scale_b = m_min / (m_min - m_max)
    m_scale_a = -m_scale_b / m_min

    omega_scale_b = omega_min / (omega_min - omega_max)
    omega_scale_a = -omega_scale_b / omega_min

    for n in range(tries):

        found = False

        while not found:
            tc_start = numpy.random.uniform(low=tc_min, high=tc_max)
            m_start = numpy.random.uniform(low=m_min, high=m_max)
            omega_start = numpy.random.uniform(low=omega_min, high=omega_max)
            found = True

            for i in range(len(rv.tc_start)):
                # Scale values to range 0 .. 1
                # Calculate distance and reject starting point if too close to
                # already used starting point
                a = np.array([tc_start * tc_scale_a + tc_scale_b,
                              m_start * m_scale_a + m_scale_b,
                              omega_start * omega_scale_a + omega_scale_b])
                b = np.array([rv.tc_start[i] * tc_scale_a + tc_scale_b,
                              rv.m_start[i] * m_scale_a + m_scale_b,
                              rv.omega_start[i] * omega_scale_a + omega_scale_b])
                distance = numpy.linalg.norm(a - b)
                if distance < min_distance:
                    found = False
                    # print("Points to close together: ", a, b)
                    break

        rv.tc_start.append(tc_start)
        rv.m_start.append(m_start)
        rv.omega_start.append(omega_start)

        x0 = [tc_start, m_start, omega_start]

        try:
            opt_rv = optimize.minimize(F1, x0, args=(data,), method='Nelder-Mead')
            if opt_rv.success:

                tc_est, m_est, omega_est = opt_rv.x
                pruned = False

                if tc_est < tc_min:
                    tc_est = tc_min
                    pruned = True
                elif tc_est > tc_max:
                    tc_est = tc_max
                    pruned = True

                if m_est < m_min:
                    m_est = m_min
                    pruned = True
                elif m_est > m_max:
                    m_est = m_max
                    pruned = True

                if omega_est < omega_min:
                    omega_est = omega_min
                    pruned = True
                elif omega_est > omega_max:
                    omega_est = omega_max
                    pruned = True

                mse = F1([tc_est, m_est, omega_est], data)

                if mse_min is None or mse < mse_min:
                    fitted_parameters = [tc_est, m_est, omega_est]
                    fitted_pruned = pruned
                    mse_min = mse
                    rv.mse_hist.append(mse)
                else:
                    rv.mse_hist.append(mse_min)
        except LinAlgError as e:
            # print("Exception occurred: ", e)
            pass

    if fitted_parameters is not None:
        rv.tc, rv.m, rv.omega = fitted_parameters
        rv.A, rv.B, rv.C1, rv.C2 = F1_get_linear_parameters(fitted_parameters, data)
        rv.C = abs(rv.C1) + abs(rv.C2)
        rv.price_tc = rv.A + rv.B * (0.001) ** rv.m + \
                      rv.C1 * (0.001) ** rv.m * np.cos(rv.omega * np.log(0.001)) + \
                      rv.C2 * (0.001) ** rv.m * np.sin(rv.omega * np.log(0.001))
        rv.price_chg = (rv.price_tc - data.iat[-1]) / data.iat[-1] * 100
        rv.pruned = fitted_pruned
        rv.mse = mse_min
        rv.norm_mse = F1_normalized(fitted_parameters, data) / len(data) * 1000
        rv.success = True
    return rv
```

```python
data = gbm_data
data = simulated_data
# data = gbm_sim_data
data = reduced_data
data = cutout_data

rv = LPPL_fit(data)

if rv.success:
    line_points = len(data.values) * rv.tc
    t_ = np.linspace(0, rv.tc, num=line_points)
    est_line_data = rv.A + rv.B * (rv.tc - t_) ** rv.m + \
                    rv.C1 * (rv.tc - t_) ** rv.m * np.cos(rv.omega * np.log(rv.tc - t_)) + \
                    rv.C2 * (rv.tc - t_) ** rv.m * np.sin(rv.omega * np.log(rv.tc - t_))
    est_line_data_index = np.linspace(0, rv.tc, len(est_line_data))

    price_tc = rv.A + rv.B * (0.001) ** rv.m + \
               rv.C1 * (0.001) ** rv.m * np.cos(rv.omega * np.log(0.001)) + \
               rv.C2 * (0.001) ** rv.m * np.sin(rv.omega * np.log(0.001))

    print()
    print("== RESULTS ==")
    print("  price tc: %.2f (%.2f)" % (price_tc, est_line_data[-2]))
    print("   tc:   real value: % 8.2f    estimation: % 8.2f" % (tc, rv.tc))
    print("    m:   real value: % 8.2f    estimation: % 8.2f" % (m, rv.m))
    print("omega:   real value: % 8.2f    estimation: % 8.2f" % (omega, rv.omega))
    print("    A:   real value: % 8.2f    estimation: % 8.2f" % (A, rv.A))
    print("    B:   real value: % 8.2f    estimation: % 8.2f" % (B, rv.B))
    print("   C1:   real value: % 8.2f    estimation: % 8.2f" % (C1, rv.C1))
    print("   C2:   real value: % 8.2f    estimation: % 8.2f" % (C2, rv.C2))
    print()
    print("== ERROR STATISTICS ==")
    print("Mean square error: % 20.2f" % (rv.mse,))
    print(" MSE (normalized): % 20.2f" % (rv.norm_mse,))

    plot(data.index, data.values, '.')
    plot(est_line_data_index, est_line_data, 'b-')
    title("MSE: %d, NMSE: %.2f, tc: %.2f" % (rv.mse, rv.norm_mse, rv.tc))
```

# Cost function plots
Model contraints
$$
\begin{align}
1 \leq{} & t_c \leq 4 \\
0.1 \leq{} & m \leq 0.9 \\
6 \leq{}  & \omega \leq 13
\end{align}
$$

```python
# Calculate values

MESH_SIZE = 100
LEVELS = 50

tc_est, m_est, omega_est = rv.tc, rv.m, rv.omega
tc_ = np.linspace(1.0, 4.0, MESH_SIZE)
m_ = np.linspace(0.1, 0.9, MESH_SIZE)
omega_ = np.linspace(6, 13, MESH_SIZE)

X1, Y1 = np.meshgrid(tc_, m_)
Z1 = np.zeros((MESH_SIZE, MESH_SIZE))
for i in range(len(tc_)):
    for j in range(len(m_)):
        Z1[i, j] = F1([tc_[i], m_[j], omega_est], data)

X2, Y2 = np.meshgrid(tc_, omega_)
Z2 = np.zeros((MESH_SIZE, MESH_SIZE))
for i in range(len(tc_)):
    for j in range(len(omega_)):
        Z2[i, j] = F1([tc_[i], m_est, omega_[j]], data)

X3, Y3 = np.meshgrid(m_, omega_)
Z3 = np.zeros((MESH_SIZE, MESH_SIZE))
for i in range(len(m_)):
    for j in range(len(omega_)):
        Z3[i, j] = F1([tc_est, m_[i], omega_[j]], data)

```

```python
from mpl_toolkits.mplot3d.axes3d import get_test_data
# This import registers the 3D projection, but is otherwise unused.
from mpl_toolkits.mplot3d import Axes3D  # noqa: F401 unused import

# Plot graph

fig = plt.figure(figsize=(18., 30.))

# Plot tc vs m
ax = fig.add_subplot(4, 2, 1)
c = ax.contour(X1, Y1, Z1, LEVELS)
# ax.plot(tc, m, 'ro', markersize=10)
# ax.plot(tc_est, m_est, 'r*', markersize=15)
ax.set_xlabel(r"$t_c$", fontsize=18)
ax.set_ylabel(r"m", fontsize=18)
ax.plot(tc_est, m_est, 'r*', markersize=15)
plt.colorbar(c, ax=ax)

# 3D surface
ax = fig.add_subplot(4, 2, 5, projection='3d')
surf = ax.plot_surface(X1, Y1, Z1, cmap=cm.coolwarm, linewidth=0, antialiased=False)

# Plot omega vs tc
ax = fig.add_subplot(4, 2, 3)
c = ax.contour(X2, Y2, Z2, LEVELS)
# ax.plot(tc, m, 'ro', markersize=10)
# ax.plot(tc_est, m_est, 'r*', markersize=15)
ax.set_xlabel(r"$t_c$", fontsize=18)
ax.set_ylabel(r"$\omega$", fontsize=18)
ax.plot(tc_est, omega_est, 'r*', markersize=15)
plt.colorbar(c, ax=ax)

# 3D surface
ax = fig.add_subplot(4, 2, 7, projection='3d')
surf = ax.plot_surface(X2, Y2, Z2, cmap=cm.coolwarm, linewidth=0, antialiased=False)

# Plot omega vs m
ax = fig.add_subplot(4, 2, 4)
c = ax.contour(X3, Y3, Z3, LEVELS)
# ax.plot(tc, m, 'ro', markersize=10)
# ax.plot(tc_est, m_est, 'r*', markersize=15)
ax.set_xlabel(r"$m$", fontsize=18)
ax.set_ylabel(r"$\omega$", fontsize=18)
ax.plot(m_est, omega_est, 'r*', markersize=15)
plt.colorbar(c, ax=ax)

# 3D surface
ax = fig.add_subplot(4, 2, 8, projection='3d')
surf = ax.plot_surface(X3, Y3, Z3, cmap=cm.coolwarm, linewidth=0, antialiased=False)
```


# TALENT Course 11
## Learning from Data: Bayesian Methods and Machine Learning
### York, UK, June 10-28, 2019

# Bayesian neural networks
* The introduction part of this lecture, with accompanying lecture notes, are inspired by the chapter *"Learning as Inference"* in the excellent book [Information Theory, Inference, and Learning Algorithms](http://www.inference.org.uk/mackay/itila/) by David MacKay.
* The variational inference part of this lecture is adapted from the [blog entry](http://krasserm.github.io/2019/03/14/bayesian-neural-networks/) by Martin Krasser with some modifications. See also the following reference:
* [Weight Uncertainty in Neural Networks](https://arxiv.org/abs/1505.05424) (*Bayes by Backprop*).

Additional references for Bayesian Neural Networks:
* [Tensorflow Probability](https://www.tensorflow.org/probability)
* [Edward](http://edwardlib.org/)
* [Keras](https://keras.io/) (for the implementation that is used in the example).

You will need `tensorflow` and `keras` for running this lecture notebook and the accompanying exercise. In this directory, please run

    conda deactivate
	conda env create

which reads the environment.yml file and creates the virtual environment `talent-env-ml`. It can be activated:

    conda activate talent-env-ml

## Basic neural network

We will consider neurons with a vector of $I$ input signals $\mathbf{x} = \left\{ x_i \right\}_{i=1}^I$, and an output signal $y(a)$, which is a (often non-linear) function of the *activation*
$$
a = \sum_i w_i x_i,
$$
where $\mathbf{w} = \left\{ w_i \right\}_{i=1}^I$ are the weights of the neuron. The sum runs from either 1 to $I$, or from 0 to $I$ if there is also a bias ($b \equiv w_0$).

The training of the network implies feeding it with training data and finding the sets of weights and biases that minimizes a loss function that has been selected for that particular problem.

Consider, e.g., a classification problem where the single output $y$ of the final network layer is a real number $\in [0,1]$ that indicates the probability for input $\mathbf{x}$ belonging to either class $t=1$ or $t=0$:
$$
\begin{align}
p(t=1 | w,x) &= y \\
p(t=0 | w,x) &= 1-y,
\end{align}
$$
where we have simplified the notation, not using boldface anymore for the arrays $w$ and $x$.

A simple neural network classifier can be trained by minimizing the loss function
$$
M(w) = G(w) +  \alpha E_W(w),
$$
made up of an error function
$$
G(w) = -\sum_n \left[ t^{(n)} \log ( y(x^{(n)},w)) + (1 - t^{(n)}) \log (1 - y(x^{(n)},w)) \right],
$$
where $t^{(n)}$ is the training data, and the regularizer
$$
E_W(w) = \frac{1}{2} \sum_i w_i^2,
$$
that is designed to avoid overfitting.

The error function can be interpreted as minus the log likelihood
$$
p(D|w) = \exp\left[ - G(w) \right].
$$
Similarly the regularizer can be interpreted in terms of a log prior probability distribution over the parameters
$$
p(w | \alpha) = \frac{1}{Z_W(\alpha)} \exp \left[ -\alpha E_W \right].
$$
If $E_W$ is quadratic as given above, then the corresponding prior distribution is a Gaussian with variance $\sigma_W^2 = 1/\alpha$ and $1/Z_W = (\alpha/2\pi)^{K/2}$, where $K$ is the number of parameters in $w$.

The objective function $M(w)$ then corresponds to the inference of the parameters $w$ given the data
$$
p(w | D, \alpha) = \frac{p(D|w) p(w|\alpha)}{p(D|\alpha)} = \frac{1}{Z_M} \exp [ -M(w) ].
$$

The network parameters $w$ that are found by minimizing $M(w)$ can be interpreted as the (locally) most probable parameter vector $w^*$. We show the different probability distributions for a typical training sequence in the following figure (reproduced from [Information Theory, Inference, and Learning Algorithms](http://www.inference.org.uk/mackay/itila/) by David MacKay)

Instead, we will use the Bayesian approach and consider the information that is contained in the probability distribution. A Bayesian prediction of a new datum $t^{(n+1)}$ involves *marginalizing* over the parameters
$$
p(t^{(n+1)} | x^{(n+1)}, D, \alpha) = \int d^K w p(t^{(n+1)} | x^{(n+1)}, w, \alpha) p(w|D,\alpha).
$$
We show an example of such inference (with the point estimate $w^*$ and the Bayesian approach) in the follwoing figure (again from MacKay)

But how shall we compute the marginalization integral for real network with thousands of parameters?

In short, there are three different approaches:
1. "exact" methods by MCMC sampling (they are exact as the number of samples $\rightarrow \infty$);
2. deterministic approximate methods, for example using Gaussian approximations with the Laplace method;
3. variational methods.

The first two are discussed in MacKay's book, while we will focus on a variational method in the following regression problem.

## Variational inference for Bayesian neural networks

This article demonstrates how to implement and train a Bayesian neural network with Keras following the approach described in [Weight Uncertainty in Neural Networks](https://arxiv.org/abs/1505.05424) (*Bayes by Backprop*). The implementation is kept simple for illustration purposes and uses Keras 2.2.4 and Tensorflow 1.12.0. For more advanced implementations of Bayesian methods for neural networks consider using [Tensorflow Probability](https://www.tensorflow.org/probability), for example.

Bayesian neural networks differ from plain neural networks in that their weights are assigned a probability distribution instead of a single value or point estimate. These probability distributions describe the uncertainty in weights and can be used to estimate uncertainty in predictions. Training a Bayesian neural network via variational inference learns the parameters of these distributions instead of the weights directly.

### Probabilistic model

A neural network can be viewed as probabilistic model $p(y \lvert \mathbf{x},\mathbf{w})$. For classification, $y$ is a set of classes and $p(y \lvert \mathbf{x},\mathbf{w})$ is a categorical distribution. For regression, $y$ is a continuous variable and $p(y \lvert \mathbf{x},\mathbf{w})$ is a Gaussian distribution.

Given a training dataset $\mathcal{D} = \left\{\mathbf{x}^{(i)}, y^{(i)}\right\}$ we can construct the likelihood function $p(\mathcal{D} \lvert \mathbf{w}) = \prod_i p(y^{(i)} \lvert \mathbf{x}^{(i)}, \mathbf{w})$ which is a function of parameters $\mathbf{w}$. Maximizing the likelihood function gives the maximimum likelihood estimate (MLE) of $\mathbf{w}$. The usual optimization objective during training is the negative log likelihood. For a categorical distribution this is the *cross entropy* error function, for a Gaussian distribution this is proportional to the *sum of squares* error function. MLE can lead to severe overfitting though.

Multiplying the likelihood with a prior distribution $p(\mathbf{w})$ is, by Bayes theorem, proportional to the posterior distribution $p(\mathbf{w} \lvert \mathcal{D}) \propto p(\mathcal{D} \lvert \mathbf{w}) p(\mathbf{w})$. Maximizing $p(\mathcal{D} \lvert \mathbf{w}) p(\mathbf{w})$ gives the maximum a posteriori (MAP) estimate of $\mathbf{w}$. Computing the MAP estimate has a regularizing effect and can prevent overfitting. The optimization objectives here are the same as for MLE plus a regularization term coming from the log prior.

Both MLE and MAP give point estimates of parameters. If we instead had a full posterior distribution over parameters we could make predictions that take weight uncertainty into account. This is covered by the posterior predictive distribution $p(y \lvert \mathbf{x},\mathcal{D}) = \int p(y \lvert \mathbf{x}, \mathbf{w}) p(\mathbf{w} \lvert \mathcal{D}) d\mathbf{w}$ in which the parameters have been marginalized out. This is equivalent to averaging predictions from an ensemble of neural networks weighted by the posterior probabilities of their parameters $\mathbf{w}$.

### Variational inference

Unfortunately, an analytical solution for the posterior $p(\mathbf{w} \lvert \mathcal{D})$ in neural networks is intractable. We therefore have to approximate the true posterior with a variational distribution $q(\mathbf{w} \lvert \boldsymbol{\theta})$ of known functional form whose parameters we want to estimate. This can be done by minimizing the [Kullback-Leibler divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence) between $q(\mathbf{w} \lvert \boldsymbol{\theta})$ and the true posterior $p(\mathbf{w} \lvert \mathcal{D})$  w.r.t. $\boldsymbol{\theta}$.

The Kullback-Leibler (KL) Divergence is a numeric measure of the difference between two distributions. For two probability distributions $Q$ and $P$, the KL divergence in a continuous case,
$$
\mathrm{KL}(Q||P) = \int_{-\infty}^\infty dx q(x) \log \frac{q(x)}{p(x)} dx \equiv \mathbb{E}_{q(x)} \left[ \log q(x) - \log p(x) \right]
$$
As we can see, the KL divergence calculates the expected log differences in between two distributions with respect to distribution Q.

It can be shown that the corresponding optimization objective or cost function can be written as

$$
\mathcal{F}(\mathcal{D},\boldsymbol{\theta}) =
\mathrm{KL}(q(\mathbf{w} \lvert \boldsymbol{\theta}) \mid\mid p(\mathbf{w})) -
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log p(\mathcal{D} \lvert \mathbf{w})
\tag{1}
$$

This is known as the *variational free energy*. The first term is the Kullback-Leibler divergence between the variational distribution $q(\mathbf{w} \lvert \boldsymbol{\theta})$ and the prior $p(\mathbf{w})$ and is called the *complexity cost*. The second term is the expected value of the likelihood w.r.t. the variational distribution and is called the *likelihood cost*. By re-arranging the KL term, the cost function can also be written as

$$
\mathcal{F}(\mathcal{D},\boldsymbol{\theta}) =
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log q(\mathbf{w} \lvert \boldsymbol{\theta}) -
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log p(\mathbf{w}) -
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log p(\mathcal{D} \lvert \mathbf{w})
\tag{2}
$$

We realize that minimization of the variational free energy is obtained by finding the parameters $\theta$ that
* maximizes

$$
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log \big( p(\mathbf{w}) p(\mathcal{D} \lvert \mathbf{w}) \big) =
\mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log p(\mathbf{w, \mathcal{D}}),
$$

  which encourages weights that fit data well.
* and maximizes

$$
- \mathbb{E}_{q(\mathbf{w} \lvert \boldsymbol{\theta})} \log q(\mathbf{w} \lvert \boldsymbol{\theta}) = -\int d \mathbf{w} q(\mathbf{w} \lvert \boldsymbol{\theta}) \log q(\mathbf{w} \lvert \boldsymbol{\theta}),
$$

  which we recognize as the entropy.

We see that all three terms in equation (2) are expectations w.r.t. the variational distribution $q(\mathbf{w} \lvert \boldsymbol{\theta})$. The cost function can therefore be approximated by drawing [Monte Carlo](https://en.wikipedia.org/wiki/Monte_Carlo_method) samples $\mathbf{w}^{(i)}$ from $q(\mathbf{w} \lvert \boldsymbol{\theta})$.

$$
\mathcal{F}(\mathcal{D},\boldsymbol{\theta}) \approx {1 \over N} \sum_{i=1}^N \left[
\log q(\mathbf{w}^{(i)} \lvert \boldsymbol{\theta}) -
\log p(\mathbf{w}^{(i)}) -
\log p(\mathcal{D} \lvert \mathbf{w}^{(i)})\right]
\tag{3}
$$

In the following example, we'll use a Gaussian distribution for the variational posterior, parameterized by $\boldsymbol{\theta} = (\boldsymbol{\mu}, \boldsymbol{\sigma})$ where $\boldsymbol{\mu}$ is the mean vector of the distribution and $\boldsymbol{\sigma}$ the standard deviation vector. The elements of $\boldsymbol{\sigma}$ are the elements of a diagonal covariance matrix which means that weights are assumed to be uncorrelated. Instead of parameterizing the neural network with weights $\mathbf{w}$ directly we parameterize it with $\boldsymbol{\mu}$ and $\boldsymbol{\sigma}$ and therefore double the number of parameters compared to a plain neural network.

### Network training

A training iteration consists of a forward-pass and and backward-pass. During a forward pass a single sample is drawn from the variational posterior distribution. It is used to evaluate the approximate cost function defined by equation (3). The first two terms of the cost function are data-independent and can be evaluated layer-wise, the last term is data-dependent and is evaluated at the end of the forward-pass. During a backward-pass, gradients of $\boldsymbol{\mu}$ and $\boldsymbol{\sigma}$ are calculated via backpropagation so that their values can be updated by an optimizer.

Since a forward pass involves a stochastic sampling step we have to apply the so-called *re-parameterization trick* for backpropagation to work. The trick is to sample from a parameter-free distribution and then transform the sampled $\boldsymbol{\epsilon}$ with a deterministic function $t(\boldsymbol{\mu}, \boldsymbol{\sigma}, \boldsymbol{\epsilon})$ for which a gradient can be defined. Here, we choose $\boldsymbol{\epsilon}$ to be drawn from a standard normal distribution i.e. $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ and the function $t$ is taken to be $t(\boldsymbol{\mu}, \boldsymbol{\sigma}, \boldsymbol{\epsilon}) = \boldsymbol{\mu} + \boldsymbol{\sigma} \odot \boldsymbol{\epsilon}$, i.e., it shifts the sample by mean $\boldsymbol{\mu}$ and scales it with $\boldsymbol{\sigma}$ where $\odot$ is element-wise multiplication.

For numerical stability we will parameterize the network with $\boldsymbol{\rho}$ instead of $\boldsymbol{\sigma}$ directly and transform $\boldsymbol{\rho}$ with the softplus function to obtain $\boldsymbol{\sigma} = \log(1 + \exp(\boldsymbol{\rho}))$. This ensures that $\boldsymbol{\sigma}$ is always positive. As prior, a scale mixture of two Gaussians is used $p(\mathbf{w}) = \pi \mathcal{N}(\mathbf{w} \lvert 0,\sigma_1^2) + (1 - \pi) \mathcal{N}(\mathbf{w} \lvert 0,\sigma_2^2)$ where $\sigma_1$, $\sigma_2$ and $\pi$ are shared parameters. Their values are learned during training (which is in contrast to the paper where a fixed prior is used).

### Uncertainty characterization

Uncertainty in predictions that arise from the uncertainty in weights is called [epistemic uncertainty](https://en.wikipedia.org/wiki/Uncertainty_quantification). This kind of uncertainty can be reduced if we get more data. Consequently,  epistemic uncertainty is higher in regions of no or little training data and lower in regions of more training data. Epistemic uncertainty is covered by the variational posterior distribution. Uncertainty coming from the inherent noise in training data is an example of [aleatoric uncertainty](https://en.wikipedia.org/wiki/Uncertainty_quantification). It cannot be reduced if we get more data. Aleatoric uncertainty is covered by the probability distribution used to define the likelihood function.

### Implementation example

Variational inference of neural network parameters is now demonstrated on a simple regression problem. We therefore use a Gaussian distribution for $p(y \lvert \mathbf{x},\mathbf{w})$. The training dataset consists of a few noisy samples `x_train`, `y_train` drawn from the `true_function` function with Gaussian noise.

```python
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

import seaborn as sns;
sns.set('talk')

# make your loops show a smart progress meter - just wrap any iterable with tqdm(iterable)
import tqdm
```

```python
x_train_min = -3
x_train_max = 3

def true_function(x, k=1., slope=1.):
    """
    Linear plus cosine function with adjustable slope and wave number.
    """
    return slope * x + np.cos(k*x)

def build_toy_dataset(N=20, noise_std=.1, k=1., slope=1.):
    """
    Generate toy data arrays with N points and added Gaussian noise with
    standard deviation noise_std.
    """
    #np.random.seed(47)
    x = np.random.uniform(x_train_min, x_train_max, size=N)
    y = true_function(x, k, slope) \
        + np.random.normal(0, noise_std * abs(x), size=N)
    x = x.astype(np.float32).reshape((N, 1))
    y = y.astype(np.float32)
    return x, y
```

```python
train_size = 20       # number of data ponts
k = 1.0      # scale wavelength
slope = 0.2  # linear term: slope*x
noise = 0.1

x_train, y_train = build_toy_dataset(train_size, noise_std=noise, k=k, slope=slope)
```

```python
# Produce an array with true function evaluations
x_linspace = np.linspace(x_train_min, x_train_max, 100).reshape(-1, 1)
y_true = true_function(x_linspace, k=k, slope=slope)
```

The noise in training data gives rise to aleatoric uncertainty. To cover epistemic uncertainty we implement the variational inference logic in a custom `DenseVariational` Keras layer. The learnable parameters of the mixture prior, $\sigma_1$ $\sigma_2$ and $\pi$, are shared across layers. The complexity cost (`kl_loss`) is computed layer-wise and added to the total loss with the `add_loss` method. Implementations of `build` and `call` directly follow the equations defined above.

Note that many of these features are now available in the [Tensorflow Probability](https://www.tensorflow.org/probability) Python library. It might be instructive, however, to see how the DenseVariational Layer is constructed explicitly.

```python
from keras import backend as K
from keras import activations, initializers
from keras.layers import Layer

import tensorflow as tf

def mixture_prior_params(sigma_1, sigma_2, pi, return_sigma=False):
    params = K.variable([sigma_1, sigma_2, pi], name='mixture_prior_params')
    sigma = np.sqrt(pi * sigma_1 ** 2 + (1 - pi) * sigma_2 ** 2)
    return params, sigma

def log_mixture_prior_prob(w):
    comp_1_dist = tf.distributions.Normal(0.0, prior_params[0])
    comp_2_dist = tf.distributions.Normal(0.0, prior_params[1])
    comp_1_weight = prior_params[2]
    return K.log(comp_1_weight * comp_1_dist.prob(w) + (1 - comp_1_weight) * comp_2_dist.prob(w))

# Mixture prior parameters shared across DenseVariational layer instances
prior_params, prior_sigma = mixture_prior_params(sigma_1=1.0, sigma_2=0.1, pi=0.2)

class DenseVariational(Layer):
    def __init__(self, output_dim, kl_loss_weight, activation=None, **kwargs):
        self.output_dim = output_dim
        self.kl_loss_weight = kl_loss_weight
        self.activation = activations.get(activation)
        super().__init__(**kwargs)

    def build(self, input_shape):
        self._trainable_weights.append(prior_params)

        self.kernel_mu = self.add_weight(name='kernel_mu',
                                         shape=(input_shape[1], self.output_dim),
                                         initializer=initializers.normal(stddev=prior_sigma),
                                         trainable=True)
        self.bias_mu = self.add_weight(name='bias_mu',
                                       shape=(self.output_dim,),
                                       initializer=initializers.normal(stddev=prior_sigma),
                                       trainable=True)
        self.kernel_rho = self.add_weight(name='kernel_rho',
                                          shape=(input_shape[1], self.output_dim),
                                          initializer=initializers.constant(0.0),
                                          trainable=True)
        self.bias_rho = self.add_weight(name='bias_rho',
                                        shape=(self.output_dim,),
                                        initializer=initializers.constant(0.0),
                                        trainable=True)
        super().build(input_shape)

    def call(self, x):
        kernel_sigma = tf.math.softplus(self.kernel_rho)
        kernel = self.kernel_mu + kernel_sigma * tf.random.normal(self.kernel_mu.shape)

        bias_sigma = tf.math.softplus(self.bias_rho)
        bias = self.bias_mu + bias_sigma * tf.random.normal(self.bias_mu.shape)

        self.add_loss(self.kl_loss(kernel, self.kernel_mu, kernel_sigma) +
                      self.kl_loss(bias, self.bias_mu, bias_sigma))

        return self.activation(K.dot(x, kernel) + bias)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], self.output_dim)

    def kl_loss(self, w, mu, sigma):
        variational_dist = tf.distributions.Normal(mu, sigma)
        return kl_loss_weight * K.sum(variational_dist.log_prob(w) - log_mixture_prior_prob(w))
```

    Using TensorFlow backend.

```python
from keras.layers import Input
from keras.models import Model

batch_size = train_size
num_batches = train_size / batch_size
kl_loss_weight = 1.0 / num_batches

x_in = Input(shape=(1,))
x = DenseVariational(20, kl_loss_weight=kl_loss_weight, activation='relu')(x_in)
x = DenseVariational(20, kl_loss_weight=kl_loss_weight, activation='relu')(x)
x = DenseVariational(1, kl_loss_weight=kl_loss_weight)(x)

model = Model(x_in, x)
```

Our model is a neural network with two `DenseVariational` hidden layers, each having 20 units, and one `DenseVariational` output layer with one unit. Instead of modeling a full probability distribution $p(y \lvert \mathbf{x},\mathbf{w})$ as output the network simply outputs the mean of the corresponding Gaussian distribution. In other words, we do not model aleatoric uncertainty here and assume it is known. We only model epistemic uncertainty via the `DenseVariational` layers.

Since the training dataset has very few data points we train the network with all of them per epoch so that the number of batches per epoch is 1. For other configurations, the complexity cost (`kl_loss`) must be weighted by $1/M$ as described in section 3.4 of the [paper](https://arxiv.org/abs/1505.05424) where $M$ is the number of mini-batches per epoch.

The network can now be trained with a Gaussian negative log likelihood function (`neg_log_likelihood`) as loss function assuming a fixed standard deviation (`noise`). This corresponds to the *likelihood cost*, the last term in equation $3$.

```python
from keras import callbacks, optimizers

def neg_log_likelihood(y_true, y_pred, sigma=noise):
    dist = tf.distributions.Normal(loc=y_pred, scale=sigma)
    return K.sum(-dist.log_prob(y_true))

model.compile(loss=neg_log_likelihood, optimizer=optimizers.Adam(lr=0.03), metrics=['mse'])
```

```python
model.fit(x_train, y_train, batch_size=batch_size, epochs=5000, verbose=0);
```

    W0625 22:17:44.618273 4618782144 deprecation_wrapper.py:119] From /Applications/anaconda3/envs/talent-env-ml3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.

```python
X_test = np.linspace(-5, 5, 1000).reshape(-1, 1)
y_true_test = true_function(X_test, k=k, slope=slope)
y_pred_list = []

for i in tqdm.tqdm(range(500)):
    y_pred = model.predict(X_test)
    y_pred_list.append(y_pred)

y_preds = np.concatenate(y_pred_list, axis=1)
```

    100%|██████████| 500/500 [00:05<00:00, 97.92it/s]

When calling `model.predict` we draw a random sample from the variational posterior distribution and use it to compute the output value of the network. This is equivalent to obtaining the output from a single member of a hypothetical ensemble of neural networks. Drawing 500 samples means that we get predictions from 500 ensemble members. From these predictions we can compute statistics such as the mean and standard deviation. In our example, the standard deviation is a measure of epistemic uncertainty.

```python
y_mean = np.mean(y_preds, axis=1)
y_sigma = np.std(y_preds, axis=1)
```

```python
fig, ax = plt.subplots(1,1,figsize=(8,6))
ax.plot(X_test, y_mean, 'r-', label='Predictive mean');
ax.scatter(x_train, y_train, marker='+', label='Training data')
ax.plot(X_test, y_true_test, label='Truth')
ax.fill_between(X_test.ravel(),
                 y_mean + 2 * y_sigma,
                 y_mean - 2 * y_sigma,
                 alpha=0.5, label='Epistemic uncertainty')
ax.set(xlabel='x', ylabel = 'y', title='Prediction')
ax.legend();
```

For an example how to model both epistemic and aleatoric uncertainty I recommend reading [Regression with Probabilistic Layers in TensorFlow Probability](https://medium.com/tensorflow/regression-with-probabilistic-layers-in-tensorflow-probability-e46ff5d37baf) which uses probabilistic Keras layers from the upcoming Tensorflow Probability 0.7.0 release. Their approach to variational inference is similar to the approach described here but differs in some details. For example, they compute the complexity cost analytically instead of estimating it from Monte Carlo samples. There are other differences too.

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Discussion: Week 6

## Exercise: Two-Period Cake-Eating Problem

Recall the two-period cake-eating problem. A person has an initial quantity of cake $K_0>0$ and receives utility from consuming cake in period 0 and period 1:

\begin{align}
U(C_0,C_1) & = \log C_0 + \beta \log C_1.
\end{align}

The person chooses how much to consume in each period, $C_0$ and $C_1$, and a quantity of cake $K_1$ to carry forward from period 0 to maximize her utility subject to the following two budget constraints:

\begin{align}
C_0 + K_1 & = K_0\\
C_1 & = K_1,
\end{align}

where the boundary condition $K_2 = 0$ has been imposed. Use the budget constraints to substitute $C_0$ and $C_1$ out of the utility function. Then the maximization problem can be written concisely as a choice of just $K_1$:

\begin{align}
\max_{K_1} \; \log (K_0-K_1) + \beta \log K_1
\end{align}

The first-order condition with respect to $K_1$ is:

\begin{align}
-\frac{1}{K_0-K_1} + \frac{\beta}{K_1} & = 0,
\end{align}

Given $K_1$, values for $C_0$ and $C_1$ are implied by the budget constraints

The function in the next cell computes the solution for the two-period cake-eating problem and provides an example of how to use the function.

```python
# Define a function to compute the solution for the two-period cake-eating problem. CELL PROVIDED.
def two_period_cake(beta,cake0):

    '''Function for computing solution to two-period cake-eating problem

            Args:
            beta (float):   subjective discount factor
            cake0 (float):  initial quantity of cake

        Returns:
            numpy array, numpy array
    '''

    # Initialize arrays for cake and consumption
    cake = np.zeros(3)
    consumption = np.zeros(2)

    # Assign values to cake array elements
    cake[0] = cake0
    cake[1] = beta/(1+beta)*cake[0]

    # Assign values to consumption array elements as implied by the budget constraints
    consumption[0] = cake[0]-cake[1]
    consumption[1] = cake[1]

    return cake,consumption

# Example of function use: Plot cake path for beta=0.99 and cake0=100
beta = 0.99
cake0 = 100

cake,consumption = two_period_cake(beta=beta,cake0=cake0)

plt.plot(cake)
plt.title('Cake Quantity for $\\beta='+str(beta)+'$')
plt.grid()
```

Suppose that the initial quantity of cake is $K_0 = 100$. For $\beta = 0, 0.25, 0.5, 0.75, 0.99, 1$, use the function `two_period_cake()` to plot the optimal path for the quantity of cake for periods 0 through 2. Add labels to the plot command so you know which line corresponds to which value of $\beta$.

```python
# Create a variable called 'cake0' that stores the initial quantity of cake
cake0=100

# Plot optimal path of cake for the different values of beta.
for beta in [1, 0.99, 0.75, 0.5, 0.25, 0]:

    cake,consumption = two_period_cake(beta=beta,cake0=cake0)
    plt.plot(cake,'o-',lw=3,alpha=0.75,label='$\\beta=$'+str(beta))

plt.title('Cake Quantity for different $\\beta$ values')

# Put a legend to the right of the current axis. PROVIDED
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.grid()
```

**Questions**

1. How does increasing $\beta$ affect the optimal path of $K$?
2. For $\beta=0$, $K_1 = 0$. Why does this make sense?

**Answers**

1. As $\beta$ increases, the person is placing more weight on the utility flow from consumption in period 1 and so the optimal path entails more cake saved for period 1 consummption. The path of cake is more smooth.<!-- answer -->
2. For $\beta=0$, the person places no weight on utility flow in period 1 and so the person consumes all of the cake in period 0. This person is very impatient!<!-- answer -->

## Exercise: Three-Period Cake-Eating Problem

Now consider a the three-period cake-eating problem. A person has an initial quantity of cake $K_0>0$ and receives utility from consuming cake in period 0, period 1, and period 2:

\begin{align}
U(C_0,C_1,C_2) & = \log C_0 + \beta \log C_1+ \beta^2 \log C_2.
\end{align}

The person chooses how much to consume in each period, $C_0$, $C_1$, and $C_2$, and quantities of cake $K_1$ and $K_2$ to carry foward from period 0 and from period 1 to maximize her utility subject to the following two budget constraints:

\begin{align}
C_0 + K_1 & = K_0\\
C_1 + K_2 & = K_1\\
C_2 & = K_2,
\end{align}

where the boundary condition $K_3 = 0$ has been imposed. Use the budget constraints to substitute $C_0$, $C_1$, and $C_2$ out of the utility function. Then the maximiation problem can be written concisely as a choice of $K_1$ and $K_2$:

\begin{align}
\max_{K_1,K_2} \; \log (K_0-K_1) + \beta\log (K_1-K_2) + \beta^2 \log K_2
\end{align}

The first-order conditions with respect to $K_1$ and $K_2$ are:

\begin{align}
-\frac{1}{K_0-K_1} + \frac{\beta}{K_1-K_2} & = 0\\
-\frac{1}{K_1-K_2} + \frac{\beta}{K_2} & = 0,
\end{align}

### Part (a)

Use the first-order conditions for the three-period cake-eating problem to solve by hand for:

1. $K_2$ as a function of $K_1$
2. $K_1$ as a function of $K_0$

Use what you derive in this part to do **Part (b)** and **Part (c)**.

### Part (b)

Copy and paste the full definition of the function `two_period_cake()` into the cell below and do the following:

1. Rename the function `three_period_cake`
2. Edit the function so that it computes the optimal path for the three-period problem

```python
# Define a function to compute the solution for the three-period cake-eating problem.
def three_period_cake(beta,cake0):

    '''Function for computing solution to three-period cake-eating problem

            Args:
            beta (float):   subjective discount factor
            cake0 (float):  initial quantity of cake

        Returns:
            numpy array, numpy array
    '''

    # Initialize arrays for cake and consumption. NOT PROVIDED
    cake = np.zeros(4)
    consumption = np.zeros(3)

    # Assign values to cake array elements. NOT PROVIDED
    cake[0] = cake0
    cake[1] = (beta+ beta**2)/(1+beta+beta**2)*cake[0]
    cake[2] = beta/(1+beta)*cake[1]

    # Assign values to consumption array elements as implied by the budget constraints. NOT PROVIDED
    consumption[0] = cake[0]-cake[1]
    consumption[1] = cake[1]-cake[2]
    consumption[2] = cake[2]

    return cake,consumption
```

### Part (c)

Suppose that the initial quantity of cake is $K_0 = 100$. For $\beta = 0, 0.25, 0.5, 0.75, 0.99, 1$, plot the optimal path for the quantity of cake for periods 0 through 3. Add labels to the plot command so you know which line corresponds to which value of $\beta$.

```python
# Create a variable called 'cake0' that stores the initial quantity of cake
cake0=100

# Plot optimal path of cake for the different values of beta.
for beta in [1, 0.99, 0.75, 0.5, 0.25, 0]:

    cake,consumption = three_period_cake(beta=beta,cake0=cake0)
    plt.plot(cake,'o-',lw=3,alpha=0.75,label='$\\beta=$'+str(beta))

plt.title('Cake Quantity for different $\\beta$ values')

# Put a legend to the right of the current axis. PROVIDED
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.grid()
```


## Problem Sheet 3 Question 2b

The general form of the population growth differential equation
\begin{equation} y^{'}=y-t, \ \ (0 \leq t \leq 2) \end{equation}
with the initial condition
\begin{equation}y(0)=2\end{equation}
For N=4
with the analytic (exact) solution
\begin{equation} y= e^{t}+t+1.\end{equation}

# Runge Kutta Solution

The Runge Kutta difference equation is
\begin{equation}w_{i+1} = w_{i} + \frac{1}{6}(k_1+2k_2+2k_3+k_4) \end{equation}
where
\begin{equation}k_1=h(w_i-t_i)\end{equation}
\begin{equation}k_2=h((w_i+\frac{1}{2}k_1)-(t_i+\frac{h}{2}))\end{equation}
\begin{equation}k_3=h((w_i+\frac{1}{2}k_2)-(t_i+\frac{h}{2}))\end{equation}
\begin{equation}k_4=h((w_i+k_3)-(t_i+h))\end{equation}

```python
import numpy as np
import math
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import matplotlib.gridspec as gridspec # subplots
import pandas as pd

import warnings
#from ipywidgets import *

```

```python
def myfun_ty(t,y):
    return y-t#+3*y

#PLOTS
def RK4_Question2(N,IC):

    x_end=4
    x_start=0
    INTITIAL_CONDITION=IC
    h=x_end/(N)
    N=N+2;
    k_list=np.zeros(N)
    t=np.zeros(N)
    w=np.zeros(N)
    k_mat=np.zeros((4,N))
    Analytic_Solution=np.zeros(N)
    k=0
    w[0]=INTITIAL_CONDITION
    Analytic_Solution[0]=INTITIAL_CONDITION
    t[0]=x_start
    k_list[k]=k
    for k in range (0,N-1):
        k_mat[0,k]=myfun_ty(t[k],w[k])
        k_mat[1,k]=myfun_ty(t[k]+h/2.0,w[k]+h/2.0*k_mat[0,k])
        k_mat[2,k]=myfun_ty(t[k]+h/2.0,w[k]+h/2.0*k_mat[1,k])
        k_mat[3,k]=myfun_ty(t[k]+h,w[k]+h*k_mat[2,k])
        w[k+1]=w[k]+h/6.0*(k_mat[0,k]+2*k_mat[1,k]+2*k_mat[2,k]+k_mat[3,k])
        t[k+1]=t[k]+h
        k_list[k+1]=k+1
        Analytic_Solution[k+1]=math.exp(t[k+1])+t[k+1]+1

    fig = plt.figure(figsize=(10,4))
    # --- left hand plot
    ax = fig.add_subplot(1,3,1)
    plt.plot(t,w,color='k')
    #ax.legend(loc='best')
    plt.title('Numerical Solution h=%s'%(h))

    # --- right hand plot
    ax = fig.add_subplot(1,3,2)
    plt.plot(t,Analytic_Solution,'-.o',color='blue')
    plt.title('Analytic Solution')

    #ax.legend(loc='best')
    ax = fig.add_subplot(1,3,3)
    plt.plot(t,Analytic_Solution-w,':o',color='red')
    plt.title('Error')

    # --- title, explanatory text and save

    # --- title, explanatory text and save
    fig.suptitle(r"$y'=y-t$", fontsize=20)
    plt.tight_layout()
    plt.subplots_adjust(top=0.85)
```

```python
RK4_Question2(4,2)
```

```python

```

Chapter 03 - Runge Kutta/3_Problem_Sheet/302b_Problem_Sheet.ipynb