**Attention**: Please run this notebook in Google Colaboratory, else the interactive functionality might not work as intended

---

# Introduction

In this notebook you can interactively examine the stability of the underlying dynamical systems for different types neurons. We will show the equations for the underlying dynamical systems for the neuron models and implement them using the [brian2 toolbox](https://briansimulator.org) (Stimberg et al., 2019). Then, we will use our Python implementation of [pplane](https://www.mathworks.com/matlabcentral/fileexchange/61636-pplane) with [Sympy](https://www.sympy.org/en/index.html) to solve and examine the dynamical systems.

To take a look at the spiking and reset behavior of these neuron types and how we implemented them in brian2, please check out [this notebook](https://colab.research.google.com/github/DiGyt/neuropynamics/blob/master/notebooks/Single_neurons.ipynb) beforehand.

Before we dive in, we need to install some mandatory stuff by executing the cells below.

# Preparations

## Installation of required packages

```
# We use the brian2 toolbox for our models in the backend
!pip install brian2 -q
```

         |████████████████████████████████| 1.6MB 2.7MB/s
         |████████████████████████████████| 5.8MB 15.5MB/s
      Building wheel for brian2 (setup.py) ... done

## Imports

```
# Main packages
import numpy as np
from sympy import symbols, solve, nsolve, lambdify, sympify, dsolve, Eq, solveset, linear_eq_to_matrix, nonlinsolve, Matrix, diff, sqrt, exp

# Brian2 package
# Unit definitions
from brian2 import mV, ms, volt, second, umetre, ufarad, siemens, cm, msiemens, amp, uA, nA
# Other stuff
from brian2 import start_scope, NeuronGroup, StateMonitor, run

# Plotting stuff
import matplotlib.pyplot as plt
import seaborn as sns

# Interactive widgets
import ipywidgets as widgets
from ipywidgets import interact, interactive, fixed, HBox, VBox, Layout
```

    WARNING    /usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm
     [py.warnings]

## Configuration

```
# Allow for larger output cells
from IPython.display import Javascript
display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 5000})'''))

# Set button layout
button_layout = Layout(width='180px', height='30px')

# Set slider config
slider_total_width = '800px'
slider_desc_width = '200px'
```

    <IPython.core.display.Javascript object>

# 1D dynamical systems

## $I_{Na,p}$ model

To revise the stability of dynamical systems, we will take a look at the $I_{Na,p}$ model.

The voltage change is determined by the following function:

$C * \dot{v} = I - g_L * (V-E_L) - g_Na*\frac{1}{1+e^{(V_{1/2}-V)/k}} * (V-E_{Na}) $

Notice that the function slightly differs from that in the lecture. This is actually a mistake in the slides. The correct equation can be seen in chapter 3.1.2 in Izhikevich (2007).

Aside from the voltage $v$, all other parameters are actually constants in this model. Therefore, this is a onedimensional dynamical system.

Let's take a look at how we model this in Sympy. First, we have to define our constants. Then, the single variable in the expression has to be defined as a symbol so Python does not confuse this with an actual Python variable. Finally, we can define and display our expression.

```
# Define model parameters
C = 10/1000 # We found that scaling up the whole thing by factor 1000 made it easier to find all solutions
v_onehalf = 1.5
k = 16
g_Na = 74
E_Na = 60
g_L = 19
E_L = -67
I = 0
# Define symbols
v = symbols('v')
# Define expression
expr = (I - g_Na * (1 / (1 + exp((v_onehalf - v) / k))) * (v - E_Na) - g_L * (v - E_L)) / C
# Show expression
display(expr)
```

$\displaystyle - 1900.0 v - 127300.0 - \frac{7400.0 \left(v - 60\right)}{1 + 1.09828514030783 e^{- \frac{v}{16}}}$

As you can see, Sympy already changes the expression internally by resolving the constants and rearranging a lot of stuff.

If we now set this function to zero, we can find the attractors and equilibrim points of this system. To do just that, we can use Sympy. However, as this equation cannot easily be solved analytically with Sympy, we use `nsolve`, which is a numerical solver. As this one only finds one solution per run, we have to run it multiple times with different starting values to find multiple solutions. It is not guaranteed that we actually find all solutions.

```
# Function that tries to find multiple different solutions for the zero-crossings of an expression
def find_solutions(expr, target_variable):
  # Set up list of found solutions
  solutions = []
  # Iterate through different starting values and add new solutions
  for starting_point in [-1000,-100,-50,-10,-5,-1,0,1,5,10,50,100,1000]:
    # Use nsolve to try to find a solution with current starting point
    try:
      solution = nsolve(Eq(expr, target_variable), v, starting_point)
      # Check if solution is new
      if not solution in solutions:
        # Add to list of found solutions
        solutions.append(solution)
    except:
      pass
  # Return result
  return solutions

solutions = find_solutions(expr, v)
print(solutions)
```

    [-52.4025572050729, -40.3594420345325, 30.8585337204716]

To examine where the attractions domains and equilibrium points are, let's plot the function. To do this, we first have to convert our expression to an actual function that takes input values using the `lambdify` method. Then we just plot it using Matplotlib. Additionally, we plot a point for each of the solutions above.

```
# Convert to function
function = lambdify((v), expr)
# Generate range of x values
x_values = np.linspace(-100,50)
# Create plotting area
fig, ax = plt.subplots(figsize=(10,10))
# Calculate y values and plot the function
ax.plot(x_values, function(x_values) / 1000, c = 'mediumseagreen', label = 'F(V)')
# Add horizontal line
ax.axhline(0, c = 'gray', ls = '--')
# Plot the equilibrium point(s)
for s in solutions:
  ax.scatter(s, function(float(s)) / 1000)
# Add axis description
ax.set_xlabel('membrane potential (v)')
# Add legend
ax.legend()
# Show plot
fig.show()
```

As you can see, the orange point separates the two attraction domains. Therefore this is an unstable equilibrium. Now lets see what happens if we apply an input current.

```
# Create function that plots the stuff above dynamically for a variable input current
def plot_nap_model(I):
  # Redefine expression
  expr = (I - g_Na * (1 / (1 + exp((v_onehalf - v) / k))) * (v - E_Na) - g_L * (v - E_L)) / C
  # Set equation to zero
  solutions = find_solutions(expr, v)
  # Convert to function
  function = lambdify((v), expr)
  # Generate range of x values
  x_values = np.linspace(-100,50)
  # Create plotting area
  fig, ax = plt.subplots(figsize=(10,10))
  # Calculate y values and plot the function
  ax.plot(x_values, function(x_values) / 1000, c = 'mediumseagreen', label = 'F(V)')
  # Add horizontal line
  ax.axhline(0, c = 'gray', ls = '--')
  # Plot the equilibrium point(s)
  for s in solutions:
    ax.scatter(s, function(float(s)) / 1000)
  # Add axis description
  ax.set_xlabel('membrane potential (v)')
  # Add legend
  ax.legend()
  # Show plot
  fig.show()

# Set default parameters
I_def = 1.

# Create slider for input current
I_slider = widgets.FloatSlider(value = I_def, min = 0., max = 20., step = 0.01, description = 'Input current I:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})

# Make interactive widget for function above with the given sliders
main_widgets = interactive(plot_nap_model, I = I_slider)

# Display main widgets
display(main_widgets)

```

    interactive(children=(FloatSlider(value=1.0, continuous_update=False, description='Input current I:', layout=L…

If you have not done it, try amping up the current to values of 15.4 or higher.

As you can see, the attraction domain on the lefthand side gets "smaller" with rising current until it finally disappears and the dynamical system is left with a single attraction domain.

Our previously bistable model has now changed to a monostable solution. In terms of spiking behavior, our model now always enters an excited state, no matter the initial membrane potential.

# 2D dynamical systems

In the following, we will take a look at two neuron models that use a 2D dynamical system: The Izhikevich and FitzHugh-Nagumo model.

Using the same sliders from the [notebook about single neurons](https://colab.research.google.com/github/DiGyt/neuropynamics/blob/master/notebooks/Single_neurons.ipynb), you may adjust the model parameters and examine their impact on the model nullclines, stable points and the trajectory when provided with an input current.

To determine the exact stable point type, we also analyze the eigenvalues similar to the lecture. To recall the types and the corresponding eigenvalues, check out the image below:

## Main functions

Here we set up our main functions to examine a 2D dynamical system. The function `solve_dynamical_system` calculates the values for our quiver plot and then uses the previously defined functions `calculate_nullclines` and `examine_stable_points` to calculate the nullclines and determine the stable point type.

Finally, we create a single plot to show the results of the analysis as well as the simulated model trajectory when an external current is applied.

For more detail on these functions, check out our [Python implementation of pplane](https://colab.research.google.com/github/DiGyt/neuropynamics/blob/master/notebooks/pplane.ipynb).

```
def calculate_nullclines(eq, solvar, plotvar, inputrange):
  # Set our equations to zero and solve them
  eq = Eq(eq, 0)
  sol = solve(eq, solvar)
  nullclines = []
  # Calculate y-values for each of the solutions
  for s in sol:
      f = lambdify((plotvar), sol)
      nullclines.append([f(input) for input in inputrange])
  return nullclines

def examine_stable_points(expr1, expr2, x_var, y_var, solutions):
  # First, calculate the jacobian matrix for our equations
  equation_matrix = Matrix([expr1, expr2])
  var_mat = Matrix([x_var, y_var])
  jacobian = equation_matrix.jacobian(var_mat)
  # Set up list that contains all stable points
  stable_points = []
  # Calculate eigenvalues for each of the stablepoints
  for stable_point in solutions:
    # Eigenvalue calculation
    eqmat = jacobian.subs([(x_var, stable_point[0]), (y_var, stable_point[1])])
    eigenvalues = list(eqmat.eigenvals().keys())
    # Check the eigenvalues to determine type of stable point
    if eigenvalues[0].is_real:
        if eigenvalues[0] > 0 and eigenvalues[1] > 0:
            stable_point_type = 'Unstable Node'
        elif eigenvalues[0] < 0 and eigenvalues[1] < 0:
            stable_point_type = 'Stable Node'
        elif (eigenvalues[0] < 0 and eigenvalues[1] > 0) or (eigenvalues[0] > 0 and eigenvalues[1] < 0):
            stable_point_type = 'Saddle Point'
    else:
        if eigenvalues[0].args[0] > 0:
            stable_point_type = 'Unstable Focus'
        if eigenvalues[0].args[0] < 0:
            stable_point_type = 'Stable Focus'
    # Add tuple for each stable point to list
    stable_points.append((stable_point, stable_point_type))
  # Return list of stable points
  return stable_points

def solve_dynamical_system(expr1, expr2, x_range, y_range, x_var, y_var):
  # Convert our equations to functions
  f1 = lambdify((x_var, y_var), expr1)
  f2 = lambdify((x_var, y_var), expr2)
  # Define range that we will examine the system in
  start_x = x_range[0]
  end_x = x_range[1]
  start_y = y_range[0]
  end_y = y_range[1]
  # Set up list of x and y values inside this range
  x_range = np.linspace(start_x, end_x)
  y_range = np.linspace(start_y, end_y)
  total_range = np.linspace(min(start_x, start_y), max(end_x, end_y))
  # Compute quivers by calculating the expression for a combination of x and y values
  f1_val = [[f1(x_cur, y_cur) for x_cur in x_range] for y_cur in y_range];
  f2_val = [[f2(x_cur, y_cur) for x_cur in x_range] for y_cur in y_range];
  # Solve analytically using sympy
  solutions = solve((Eq(expr1, 0), Eq(expr2, 0)), x_var, y_var)
  # Calculate nullclines
  x_nullclines = calculate_nullclines(expr1, y_var, x_var, total_range)
  y_nullclines = calculate_nullclines(expr2, y_var, x_var, total_range)
  # Get stable points and check their type
  stable_points = examine_stable_points(expr1, expr2, x_var, y_var, solutions)
  # Return relevant results
  return x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points

def plot_dynamical_system(x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points, x_behavior, y_behavior, x_var_name, y_var_name):
  # Create plotting area and set axis limits
  fig, ax = plt.subplots(figsize=(10,10))
  ax.set_xlim([x_range[0], x_range[-1]])
  ax.set_ylim([y_range[0], y_range[-1]])
  # Set axis lables
  ax.set_xlabel(x_var_name)
  ax.set_ylabel(y_var_name)
  # Plot quivers
  ax.quiver(x_range, y_range, f1_val, f2_val, alpha=.5, headwidth = 3, headlength = 2, headaxislength = 2)
  # Plot nullclines
  [ax.plot(y_range, nullcline, c = 'b', alpha=.7, label= x_var_name + ' Nullcline') for nullcline in x_nullclines]
  [ax.plot(x_range, nullcline, c = 'r', alpha=.7, label= y_var_name + ' Nullcline') for nullcline in y_nullclines]
  # Plot stable points
  for stable_point in stable_points:
    # Extract information
    stable_point_x = stable_point[0][0]
    stable_point_y = stable_point[0][1]
    stable_point_type = stable_point[1]
    # Plot stable points with different color for each type
    try: # We have to use try to ignore errors that occur when trying to plot complex value
      if stable_point_type == 'Unstable Node':
        ax.scatter(stable_point_x, stable_point_y, marker = '.', label = stable_point_type, s = 150, c = 'coral')
      elif stable_point_type == 'Stable Node':
        ax.scatter(stable_point_x, stable_point_y, marker = '.', label = stable_point_type, s = 150, c = 'cyan')
      elif stable_point_type == 'Saddle Point':
        ax.scatter(stable_point_x, stable_point_y, marker = '.', label = stable_point_type, s = 150, c = 'violet')
      elif stable_point_type == 'Unstable Focus':
        ax.scatter(stable_point_x, stable_point_y, marker = '.', label = stable_point_type, s = 150, c = 'orange')
      elif stable_point_type == 'Stable Focus':
        ax.scatter(stable_point_x, stable_point_y, marker = '.', label = stable_point_type, s = 150, c = 'springgreen')
    except:
      # Do nothing for complex values
      pass
  # Plot the actual behavior of the variables over time
  ax.plot(x_behavior, y_behavior, c = 'mediumseagreen', label = 'Model trajectory')
  # Add legend
  ax.legend()
  # Show plot
  fig.show()
```

## Izhikevich model

The equations of the dynamical system of the Izhikevich model (Izhikevich, 2003) are as follows (remember that we deviate a bit from the equations given in the lecture as explained in the [notebook for single neurons](https://colab.research.google.com/github/DiGyt/neuropynamics/blob/master/notebooks/Single_neurons.ipynb):

$\dot{v} = I + 0.04v^2 + 5v + 140 - u$

$\dot{u} = a(bv - u)$

Notice that only the parameters $a$ and $b$ are actually contained in the second equation, whereas $c$ and $d$ only apply to the reset behavior of the neuron but do not affect the dynamical system.

```
# Brain 2 implementation of Izhikevich neuron
def create_izhikevich_neuron(v_max):
    """Creates a brian2 NeuronGroup that contains a single izhikevich neuron"""
    # Define differential equation for izhikevich neuron
    eqs = '''
        dv/dt = I + (0.04/ms/mV)*v**2 + (5/ms)*v + 140*mV/ms - u : volt

        du/dt = a*(b*v-u) : volt/second

        I : volt/second
        '''
    # Define reset function
    reset = '''
        v = c
        u = u + d
        '''
    # Define threshold
    threshold = 'v > {}*mV'.format(v_max)
    # Return NeuronGroup object
    return NeuronGroup(1, eqs, threshold = threshold, reset = reset, method = 'euler')

# Create function that creates a neuron and plots its behavior based on the given parameters
def plot_izhikevich_dynamical_sytem(I_ext, a_input, b_input, c_input, d_input, v_max):

  # Regular simulation of Izhikevich model using brian2
  # Start the scope to register all activity
  start_scope()
  # Define the neuron
  neuron = create_izhikevich_neuron(v_max)
  # Set neuron parameters
  a = a_input/ms
  b = b_input/ms
  c = c_input * mV
  d = d_input * volt/second
  # Start monitoring the neurons state
  statemon = StateMonitor(source = neuron, variables = ['v', 'u'], record = 0)
  # Run neuron simulation for 100ms without input
  run(100*ms)
  # Set input current to neuron
  neuron.I = I_ext * volt / second
  # Run 500ms with input
  run(500*ms)
  # Remove input current to neuron
  neuron.I = 0 * volt / second
  # Run neuron simulation for 100ms without input
  run(100*ms)

  # Define model for sympy and calculate nullclines
  # Define the symbols
  v, u = symbols('v u')
  # First expression
  expr1 = I_ext + 0.04 * v**2 + 5*v + 140 - u
  # Second expression
  expr2 = a_input * (b_input * v - u)

  # Solve dynamical system
  x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points = solve_dynamical_system(expr1, expr2, x_range = [-120, 40], y_range = [-120, 40], x_var = v, y_var = u)

  # Plot results
  plot_dynamical_system(x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points, x_behavior = statemon.v[0]/mV, y_behavior = statemon.u[0], x_var_name = 'v', y_var_name = 'w')

# Set default parameters
I_ext_def = 0.3
a_def = 0.02
b_def = 0.2
c_def = -65.
d_def = 8.
vmax_def = 10.

# Create sliders for neuron parameters
a_slider = widgets.FloatSlider(value = a_def, min = 0., max = 0.15, step = 0.01, description = 'Recovery Time Constant a:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})
b_slider = widgets.FloatSlider(value = b_def, min = 0., max = 0.35, step = 0.01, description = 'Dynamic Behavior b:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width}) # Somehow max = 0.3 does not work
c_slider = widgets.FloatSlider(value = c_def, min = -75., max = -40., step = 1, description = 'Voltage Reset Value c:', readout_format = '.1f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})
d_slider = widgets.FloatSlider(value = d_def, min = 0., max = 10., step = 1, description = 'Spike Activated Net Current d:', readout_format = '.1f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})

# Create slider for input current
I_slider = widgets.FloatSlider(value = I_ext_def, min = 0., max = 7., step = 0.01, description = 'Input Current I:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})

# Make interactive widget for function above with the given sliders
main_widgets = interactive(plot_izhikevich_dynamical_sytem, I_ext = I_slider, a_input = a_slider, b_input = b_slider, c_input = c_slider, d_input = d_slider, v_max = fixed(vmax_def))

# Create functions to set specific neuron configurations
def apply_config(I_ext, a, b, c, d):
  I_slider.value = I_ext
  a_slider.value = a
  b_slider.value = b
  c_slider.value = c
  d_slider.value = d

def reset_config(name):
  apply_config(I_ext = I_ext_def, a = a_def, b = b_def, c = c_def, d = d_def)

def apply_intrinsically_bursting_config(name):
  apply_config(I_ext = 12., a = a_def, b = b_def, c = -55, d = 4)

def apply_chattering_config(name):
  apply_config(I_ext = I_ext_def, a = a_def, b = b_def, c = -50, d = 2)

def apply_fast_spiking_config(name):
  apply_config(I_ext = I_ext_def, a = 0.1, b = b_def, c = c_def, d = d_def)

def apply_low_thresh_spiking_config(name):
  apply_config(I_ext = I_ext_def, a = a_def, b = 0.25, c = c_def, d = d_def)

def apply_resonator_config(name):
  apply_config(I_ext = I_ext_def, a = 0.1, b = 0.26, c = c_def, d = d_def)

# Create buttons for specific neuron configurations
# Reset button
regular_spiking_button = widgets.Button(description='Regular Spiking', layout = button_layout)
regular_spiking_button.on_click(reset_config)
# Intrinsically bursting
intrinsically_bursting_button = widgets.Button(description='Intrinsically Bursting', layout = button_layout)
intrinsically_bursting_button.on_click(apply_intrinsically_bursting_config)
# Chattering neuron
chattering_button = widgets.Button(description='Chattering', layout = button_layout)
chattering_button.on_click(apply_chattering_config)
# Fast spiking
fast_spiking_button = widgets.Button(description='Fast Spiking', layout = button_layout)
fast_spiking_button.on_click(apply_fast_spiking_config)
# Low-threshold spiking
low_thresh_spiking_button = widgets.Button(description='Low-threshold Spiking', layout = button_layout)
low_thresh_spiking_button.on_click(apply_low_thresh_spiking_config)
# Resonator
resonator_button = widgets.Button(description='Resonator', layout = button_layout)
resonator_button.on_click(apply_resonator_config)

# Place buttons into grid
button_description = widgets.Label(value='Select predefined neuron types:', layout = Layout(width='300px', height='30px'))
buttons = HBox(children=[regular_spiking_button, intrinsically_bursting_button, chattering_button, fast_spiking_button, low_thresh_spiking_button, resonator_button])

# Display main widget and buttons
parameter_description = widgets.Label(value='Set parameters yourself:', layout = Layout(width='300px', height='30px'))
display(VBox(children=[VBox(children=[button_description, buttons]), VBox(children=[parameter_description, main_widgets])]))
```

    VBox(children=(VBox(children=(Label(value='Select predefined neuron types:', layout=Layout(height='30px', widt…

As you can see, only the aptly named parameter $b$ (and the input current) actually influences the dynamical system. Even though $a$ appears in the equations, it does not influence the dynamical system as it is only a constant used for multiplication and therefore is irrelevant to the solutions of the dynamical system.

## FitzHugh-Nagumo model

For the FitzHugh-Nagumo model (FitzHugh, 1961; Nagumo et al., 1962), we deviate a bit from the equations for the dynamical system given in the lecture and instead use the original ones:

$\dot{v} = v - \frac{v^3}{3} - w + I$

$\dot{w} = \frac{v + a - bw}{\tau}$

```
# Brain2 implementation of the neuron model
def create_fhn_neuron():
    """Creates a brian2 NeuronGroup that contains a single FitzHugh Nagumo neuron"""
    # Define differential equation for FitzHugh-Nagumo neuron model
    eqs = '''
        dv/dt = 1/ms * (v - (v**3 / 3) / (mV**2) - w + I * Mohm) : volt

        dw/dt = (v + a - b*w) / tau : volt

        I : amp
        '''
    # Return NeuronGroup object without threshold
    return NeuronGroup(1, eqs, method='euler')

# Function for plotting the dynamical system
def plot_fhn_dynamical_sytem(I_ext, a_input, b_input, tau_input):

  # Regular simulation of FitzHugh-Nagumo model using brian2
  # Start the scope for the brian2 toolbox to register all neurons that are created
  start_scope()
  # Define the neuron
  neuron = create_fhn_neuron()
  # Set neuron parameters
  a = a_input*mV
  b = b_input
  tau = tau_input*ms
  # Start monitoring the neurons state
  statemon = StateMonitor(source = neuron, variables = ['v', 'w', 'I'], record = 0)
  # Run neuron simulation for 100ms without input
  run(100*ms)
  # Set input current to neuron
  neuron.I = I_ext * nA
  # Run 500ms with input
  run(500*ms)
  # Remove input current to neuron
  neuron.I = 0 * nA
  # Run neuron simulation for 100ms without input
  run(100*ms)

  # Define model for sympy and calculate nullclines
  # Define the symbols
  v, w = symbols('v w')
  # First expression
  expr1 = v - v**3 / 3 - w + I_ext
  # Second expression
  expr2 = (v + a_input - b_input * w) / tau_input

  # Solve dynamical system
  x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points = solve_dynamical_system(expr1, expr2, x_range = [-2.1, 2.1], y_range = [-2.1, 2.1], x_var = v, y_var = w)

  # Plot results
  plot_dynamical_system(x_range, y_range, f1_val, f2_val, x_nullclines, y_nullclines, stable_points, x_behavior = statemon.v[0]/mV, y_behavior = statemon.w[0]/mV, x_var_name = 'v', y_var_name = 'w')

# Set default parameters
I_ext_def = 0.5
a_def = 0.7
b_def = 0.8
tau_def = 12.5

# Create sliders for neuron parameters
a_slider = widgets.FloatSlider(value = a_def, min = 0., max = 1., step = 0.01, description = 'a:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})
b_slider = widgets.FloatSlider(value = b_def, min = 0.5, max = 1.5, step = 0.01, description = 'b:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})
tau_slider = widgets.FloatSlider(value = tau_def, min = 0., max = 20, step = 0.01, description = 'tau:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})

# Create slider for input current
I_ext_slider = widgets.FloatSlider(value = I_ext_def, min = 0., max = 1., step = 0.01, description = 'I:', readout_format = '.2f', continuous_update = False, layout = {'width': slider_total_width}, style = {'description_width': slider_desc_width})

# Make interactive widget for function above with the given sliders
main_widgets = interactive(plot_fhn_dynamical_sytem, I_ext = I_ext_slider, a_input = a_slider, b_input = b_slider, tau_input = tau_slider)

def reset_fhn(name):
  I_ext_slider.value = I_ext_def
  a_slider.value = a_def
  b_slider.value = b_def
  tau_slider.value = tau_def

# Reset button
reset_button = widgets.Button(description='Reset', layout = button_layout)
reset_button.on_click(reset_fhn)

# Display main widgets and reset button
display(VBox(children=[reset_button, main_widgets]))
```

    VBox(children=(Button(description='Reset', layout=Layout(height='30px', width='180px'), style=ButtonStyle()), …

In the [notebook for single neuron behavior] we already explained that the FitzHugh-Nagumo model is an example of a [relaxation oscillator](https://en.wikipedia.org/wiki/Relaxation_oscillator). As you can see in the plot above, the model jumps around the cubic nullcline of the membrane potential $v$ and therefore produces an oscillating effect.

# Final notes

We hope you enjoyed this examination of the stability of dynamical systems for some neuron models.

Please take a look at the other notebooks in our [repository](https://github.com/DiGyt/neuropynamics) to learn more about dendritic computation, bifurcation, multi-neuron networks and phase plots of dynamical systems.

# References

FitzHugh, R. (1961). Impulses and physiological states in theoretical models of nerve membrane. *Biophysical journal*, *1*(6), 445.

Izhikevich, E. M. (2003). Simple model of spiking neurons. *IEEE Transactions on neural networks*, *14*(6), 1569-1572.

Izhikevich, E. M. (2007). *Dynamical systems in neuroscience*. MIT press.

Nagumo, J., Arimoto, S., & Yoshizawa, S. (1962). An active pulse transmission line simulating nerve axon. *Proceedings of the IRE*, *50*(10), 2061-2070.

Stimberg, M., Brette, R., & Goodman, D. F. (2019). Brian 2, an intuitive and efficient neural simulator. *Elife*, 8, e47314.


# Parameter estimation example: fitting a straight line II
## Bayesian handling of nuisance parameters
$% Some LaTeX definitions we'll use.
\newcommand{\pr}{\textrm{p}}
$

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import seaborn; seaborn.set("talk") # for plot formatting
```

## The Data and the question

Let's start by defining some data that we will fit with a straight line.  The following data is measured velocities and distances for a set of galaxies. We will assume that there is a constant standard deviation of $\sigma = 200$ km/sec on the $y$ values and no error on $x$.

```python
# Data from student lab observations;
# d0 = Galaxy distances in MPc
# v0 = Galaxy velocity in km/sec
d0 = np.array([6.75, 25, 33.8, 9.36, 21.8, 5.58, 8.52, 15.1])
v0 = np.array([462, 2562, 2130, 750, 2228, 598, 224, 971])
# Assumed exp. uncertainty
err_v0 = 200
```

```python
x=d0; y=v0; dy=err_v0
fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(1,1,1)
ax.errorbar(x, y, dy, fmt='o')
ax.set_xlabel(r'distance [MPc]')
ax.set_ylabel(r'velocity [km/sec]')
fig.tight_layout()
```

The question that we will be asking is:
> What value would you infer for the Hubble constant given this data?

We will make the prior assumption that the data can be fitted with a straight line (see also the [parameter_estimation_fitting_straight_line_I.ipynb](parameter_estimation_fitting_straight_line_I.ipynb) notebook). But we note that we are actually not interested in the offset of the straight line, but just its slope.

We will try three different approaches:
* Maximum likelihood estimate
* Single-parameter inference
* Full Bayesian analysis

As a final part of this notebook, we will also explore how the posterior belief from this analysis can feed into a second data analysis.

## The Model

We follow the procedure outlined in [parameter_estimation_fitting_straight_line_I.ipynb](../bayesian-parameter-estimation/parameter_estimation_fitting_straight_line_I.ipynb).

Thus, we're fitting a straight line to data,

$$
y_M(x) = mx + b
$$

where our parameter vector will be

$$
\theta = [b, m].
$$

But this is only half the picture: what we mean by a "model" in a Bayesian sense is not only this expected value $y_M(x;\theta)$, but a **probability distribution** for our data.
That is, we need an expression to compute the likelihood $\pr(D\mid\theta)$ for our data as a function of the parameters $\theta$.

Here we are given data with simple error bars, which imply that the probability for any *single* data point is a normal distribution about the true value. That is,

$$
y_i \sim \mathcal{N}(y_M(x_i;\theta), \sigma)
$$

or, in other words,

$$
\pr(y_i\mid x_i,\theta) = \frac{1}{\sqrt{2\pi\varepsilon_i^2}} \exp\left(\frac{-\left[y_i - y_M(x_i;\theta)\right]^2}{2\varepsilon_i^2}\right)
$$

where $\varepsilon_i$ are the (known) measurement errors indicated by the error bars.

Assuming all the points are independent, we can find the full likelihood by multiplying the individual likelihoods together:

$$
\pr(D\mid\theta) = \prod_{i=1}^N \pr(x_i,y_i\mid\theta)
$$

For convenience (and also for numerical accuracy) this is often expressed in terms of the log-likelihood:

$$
\log \pr(D\mid\theta) = -\frac{1}{2}\sum_{i=1}^N\left(\log(2\pi\varepsilon_i^2) + \frac{\left[y_i - y_M(x_i;\theta)\right]^2}{\varepsilon_i^2}\right)
$$

## Step 1: Maximum likelihood estimate

```python
# Log likelihood
def log_likelihood(theta, x, y, dy):
    y_model = theta[0] + theta[1] * x
    return -0.5 * np.sum(np.log(2 * np.pi * dy ** 2) +
                         (y - y_model) ** 2 / dy ** 2)
```

Use tools in [``scipy.optimize``](http://docs.scipy.org/doc/scipy/reference/optimize.html) to maximize this likelihood (i.e. minimize the negative log-likelihood).

```python
from scipy import optimize

def minfunc(theta, x, y, dy):
    """
    Function to be minimized: minus the logarithm of the likelihood
    """
    return -log_likelihood(theta, x, y, dy)

result = optimize.minimize(minfunc, x0=[0, 0], args=(x, y, dy))
```

The output from 'scipy.optimize' contains the set of parameters, and also the inverse of the hessian matrix (that measures the second-order curvature of the optimum). The inverse hessian is related to the covariance matrix. Very often you see the square root of the diagonal elements of this matrix as uncertainty estimates. We will not discuss this measure here, but refer to the highly recommended review: [Error estimates of theoretical models: a guide](https://iopscience.iop.org/article/10.1088/0954-3899/41/7/074001).

```python
# Print the MLE and the square-root of the diagonal elements of the inverse hessian
print(f'Maximum Likelihood Estimate (MLE):')
ndim = len(result.x)
theta_MLE=result.x
err_theta_MLE = np.array([np.sqrt(result.hess_inv[i,i]) for i in range(ndim)])
for i in range(ndim):
    print(f'... theta[{i}] = {theta_MLE[i]:>5.1f} +/- {err_theta_MLE[i]:>5.1f}')
```

    Maximum Likelihood Estimate (MLE):
    ... theta[0] = -26.7 +/- 136.6
    ... theta[1] =  80.5 +/-   7.4

## Step 2: Single-parameter model

As we are not interested in the offset parameter, we might be tempted to fix its value to the most-likely estimate and then infer our knowledge about the slope from a single-parameter model. You have probably realized by now that this is not the Bayesian way of doing the analysis, but since this is a rather common way of handling nuisance parameters, we will still try it.

```python
offset = theta_MLE[0]
```

Let's define the log-likelihood for the case that the offset is fixed. It will be a function of a single free parameter: the slope.

```python
# Log likelihood
def log_likelihood_single(slope, x, y, dy, offset=0.):
    y_model = offset + slope * x
    return -0.5 * np.sum(np.log(2 * np.pi * dy ** 2) +
                         (y - y_model) ** 2 / dy ** 2)
```

Next we will plot the log-likelihood (left panel) and the likelihood (right panel) pdfs as a function of the slope. We normalize the peak of the likelihood to one

```python
slope_range = np.linspace(60, 100, num=1000)
log_P1 = [log_likelihood_single(slope, x, y, dy,offset=offset) for slope in slope_range]
log_P1_1 = log_P1 - np.max(log_P1)

fig,ax = plt.subplots(1, 2, figsize=(12,6),sharex=True)

ax[0].plot(slope_range,log_P1_1,'-k');
ax[1].plot(slope_range,np.exp(log_P1_1),'-k');
```

```python
def contour_levels(grid,sigma):
    _sorted = np.sort(grid.ravel())[::-1]
    pct = np.cumsum(_sorted) / np.sum(_sorted)
    cutoffs = np.searchsorted(pct, np.array(sigma) )
    return _sorted[cutoffs]

P1 = np.exp(log_P1 - np.max(log_P1))
sigma_contours = contour_levels(P1,0.68)

# Find the max likelihood and the 68% contours
slope_max = slope_range[P1==1.][0]
err_slope_min = np.min(slope_range[P1>sigma_contours])
err_slope_max = np.max(slope_range[P1>sigma_contours])

# The error will be symmetric around the max
err_slope = (err_slope_max - err_slope_min) / 2

fig, ax = plt.subplots(1, 1, figsize=(12, 8))
ax.plot(slope_range,P1,'-k')
ax.vlines([err_slope_min,slope_max,err_slope_max],0,[sigma_contours,1.,sigma_contours])
ax.fill_between(slope_range, P1, where=P1>=sigma_contours, interpolate=True, alpha=0.2);

print('Single parameter estimate')
print(f'... slope = {slope_max:>5.1f} +/- {err_slope:>5.1f}')
```

## Step 3: Full Bayesian approach

You might not be surprised to learn that we underestimate the uncertainty of the slope since we are making the assumption that we know the value of the offset (by fixing it to a specific estimate).

We will now repeat the data analysis, but with the full model and with marginalization on the posterior.

Let's use the (log) symmetric prior, which is the scale-invariant one.

```python
def log_prior(theta):
    # symmetric prior for the slope, and normal pdf (mean=0, variance=200) for the intercept
    return - 0.5* theta[0]**2 / dy**2 - 1.5 * np.log(1 + theta[1] ** 2)
```

With these defined, we now have what we need to compute the log posterior as a function of the model parameters.

```python
def log_posterior(theta, x, y, dy):
    return log_prior(theta) + log_likelihood(theta, x, y, dy)
```

Next we will plot the posterior probability as a function of the slope and intercept.

We will illustrate the use of MCMC sampling for obtaining the posterior pdf, which also offers a very convenient way of performing the marginalization

```python
import emcee
print('emcee sampling (version: )', emcee.__version__)

ndim = 2  # number of parameters in the model
nwalkers = 50  # number of MCMC walkers
nsteps = 2000 # steps per walker

print(f'{nwalkers} walkers: {nsteps} samples each')

# initialize walkers
starting_guesses = np.random.randn(nwalkers, ndim)

sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[x, y, dy])
%time sampler.run_mcmc(starting_guesses, nsteps)
print("done")
```

    emcee sampling (version: ) 2.2.1
    50 walkers: 2000 samples each
    CPU times: user 1.91 s, sys: 9.9 ms, total: 1.92 s
    Wall time: 1.91 s
    done

```python
# sampler.chain is of shape (nwalkers, nsteps, ndim)
# Let us reshape and all walker chains together
# Then make a scatter plot
emcee_trace = sampler.chain[:, :, :].reshape(-1, ndim).T
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(1,1,1)
ax.plot(emcee_trace[0], emcee_trace[1], ',k', alpha=0.1);
```

Our choice of starting points were not optimal. It takes some time for the MCMC chains to converge. Let us study the traces.

```python
fig, ax = plt.subplots(ndim, sharex=True,figsize=(10,6))
for i in range(ndim):
    ax[i].plot(sampler.chain[:, :, i].T, '-k', alpha=0.2);
```

```python
# We choose a warm-up time
nwarmup = 200 # warm up

# sampler.chain is of shape (nwalkers, nsteps, ndim)
# we'll throw-out the warmup points and reshape:
emcee_trace = sampler.chain[:, nwarmup:, :].reshape(-1, ndim).T
emcee_lnprob = sampler.lnprobability[:, nwarmup:].reshape(-1).T
```

Let us create some convenience tools for plotting, including a machinery to extract 1-, 2-, 3-sigma contour levels.

We will later use the 'corner' package to achieve such visualization.

```python
def compute_sigma_level(trace1, trace2, nbins=20):
    """From a set of traces, bin by number of standard deviations"""
    L, xbins, ybins = np.histogram2d(trace1, trace2, nbins)
    L[L == 0] = 1E-16
    logL = np.log(L)

    shape = L.shape
    L = L.ravel()

    # obtain the indices to sort and unsort the flattened array
    i_sort = np.argsort(L)[::-1]
    i_unsort = np.argsort(i_sort)

    L_cumsum = L[i_sort].cumsum()
    L_cumsum /= L_cumsum[-1]

    xbins = 0.5 * (xbins[1:] + xbins[:-1])
    ybins = 0.5 * (ybins[1:] + ybins[:-1])

    return xbins, ybins, L_cumsum[i_unsort].reshape(shape)

def plot_MCMC_trace(ax, xdata, ydata, trace, scatter=False, **kwargs):
    """Plot traces and contours"""
    xbins, ybins, sigma = compute_sigma_level(trace[0], trace[1])
    ax.contour(xbins, ybins, sigma.T, levels=[0.683, 0.955, 0.997], **kwargs)
    if scatter:
        ax.plot(trace[0], trace[1], ',k', alpha=0.1)
    ax.set_xlabel(r'$\theta_0$')
    ax.set_ylabel(r'$\theta_1$')

# Convenience function to extract the peak position of the mode
def max_of_mode(sampler_object):
    max_arg = np.argmax(sampler.flatlnprobability)
    return(sampler.flatchain[max_arg])
```

```python
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(1,1,1)
plot_MCMC_trace(ax, x, y, emcee_trace, scatter=True,colors='k');
max_mode_theta=max_of_mode(sampler)
with np.printoptions(precision=3):
    print(f'Max posterior is at: {max_mode_theta}')
```

#### Marginalization

Next we will perform the marginalization over the offset parameter using the samples from the posterior pdf.

Furthermore, the extraction of a 68% credible region (not to be confused with the frequentist _confidence interval_) is made simple since the posterior is well described by a single mode.

```python
# Sort the samples according to the log-probability.
# Note that we want to have the sorted by increasing -log(p)
sorted_lnprob = -np.sort(-emcee_lnprob)
# In this sorted list we then keep 1-sigma volume of the samples
# (note 1-sigma = 1-exp(-0.5) ~ 0.393 for 2-dim pdf). See
# https://corner.readthedocs.io/en/latest/pages/sigmas.html
# We then identify what log-prob this corresponds to
log_prob_max = sorted_lnprob[0]
level_1sigma = 1-np.exp(-0.5)
log_prob_cutoff = sorted_lnprob[np.int(level_1sigma*nwalkers*(nsteps-nwarmup))]
# From the list of samples that have log-prob larger than this cutoff,
# we then find the smallest and largest value for the slope parameter.
# Here we simply ignore the values for the offset parameter (this is marginalization when having MCMC samples).

slope_samples = emcee_trace[1,:]
# Mode
bayesian_slope_maxprob = slope_samples[emcee_lnprob==log_prob_max][0]
# Mean
bayesian_slope_mean = np.sort(slope_samples)[np.int(0.5*nwalkers*(nsteps-nwarmup))]
# 68% CR
bayesian_CR_slope_min = np.min(slope_samples[emcee_lnprob>log_prob_cutoff])
bayesian_CR_slope_max = np.max(slope_samples[emcee_lnprob>log_prob_cutoff])

```

```python
print('Bayesian slope parameter estimate')
print(f'... slope = {bayesian_slope_mean:>6.2f} ',\
      f'(-{bayesian_slope_mean-bayesian_CR_slope_min:>4.2f},',\
      f'+{bayesian_CR_slope_max-bayesian_slope_mean:>4.2f})')

```

    Bayesian slope parameter estimate
    ... slope =  78.23  (-6.27, +6.82)

```python
# Alternatively we can use corner
import corner
fig, ax = plt.subplots(2,2, figsize=(10,10))
corner.corner(emcee_trace.T,labels=[r"$\theta_0$", r"$\theta_1$"],
                       quantiles=[0.16, 0.5, 0.84],fig=fig,show_titles=True
                       );
```

We can use the parameter samples to create corrsponding samples of our model. Finally, we plot the mean and 1-sigma contours of these samples.

```python
def plot_MCMC_model(ax, xdata, ydata, trace, yerr=0):
    """Plot the linear model and 2sigma contours"""
    ax.errorbar(xdata, ydata, yerr, fmt='o')

    alpha, beta = trace[:2]
    xfit = np.linspace(0, 50, 5)
    yfit = alpha[:, None] + beta[:, None] * xfit
    mu = yfit.mean(0)
    sig = yfit.std(0)

    ax.plot(xfit, mu, '-k')
    ax.fill_between(xfit, mu - sig, mu + sig, color='lightgray')

    ax.set_xlabel('x')
    ax.set_ylabel('y')
```

```python
fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(1,1,1)
plot_MCMC_model(ax,x,y,emcee_trace,dy)
```

Caution:
  - Might we have underestimated the error bars on the experimental data?
  - And/or is some of the data "bad" (are there outliers)?

More on this later.

### Summary

```python
print('{0:>25s}: {1:>3.1f} +/- {2:>3.1f}'.format('Max Likelihood (1-sigma)', theta_MLE[1],err_theta_MLE[1]))
print('{0:>25s}: {1:>3.1f} +/- {2:>3.1f}'.format('Fixed offset (1-sigma)',slope_max, err_slope))
print('{0:>25s}: {1:>3.1f} (+{2:>3.1f};-{3:>3.1f})'.format('Full Bayesian (68% CR)',bayesian_slope_mean, bayesian_CR_slope_max-bayesian_slope_mean,bayesian_slope_mean-bayesian_CR_slope_min))

```

     Max Likelihood (1-sigma): 80.5 +/- 7.4
       Fixed offset (1-sigma): 80.5 +/- 3.8
       Full Bayesian (68% CR): 78.2 (+6.8;-6.3)

### Breakout session
In the Bayesian analysis we had to specify our prior assumption on the value for the offset.
* *Is this a feature or a deficiancy of the Bayesian approach? Discuss!*

What happens if we modify this prior assumption?
* *Redo the analysis with a very broad, uniform prior on the offset. How is the inference affected?*
* *Redo the analysis with a very narrow, normal prior on the offset. How is the inference affected?*

## Step 4: Error propagation

The Bayesian approach offers a straight-forward approach for dealing with (known) systematic uncertainties; namely by marginalization.

### Systematic error example
The Hubble constant acts as a galactic ruler as it is used to measure astronomical distances according to $v = H_0 x$. An error in this ruler will therefore correspond to a systematic uncertainty in such measurements.

Suppose that a particular galaxy has a measured recessional velocity $v_\mathrm{measured} = (100 \pm 5) \times 10^3$ km/sec. Also assume that the Hubble constant $H_0$ is known from the analysis performed above in Step 3. Determine the posterior pdf for the distance to the galaxy assuming:
1. A fixed value of $H_0$ corresponding to the mean of the previous analysis.
1. Using the sampled posterior pdf for $H_0$ from the above analysis.

```python
vm=100000
sig_vm=5000
```

We assume that we can write
$$
v_\mathrm{measured} = v_\mathrm{theory} + \delta v_\mathrm{exp},
$$
where $v_\mathrm{theory}$ is the recessional velocity according to our model, and $\delta v_\mathrm{exp}$ represents the noise component of the measurement. We know that $\delta v_\mathrm{exp}$ can be described by a Gaussian pdf with mean 0 and standard deviation $\sigma_v = 5 \times 10^3$ km/sec. Note that we have also assumed that our model is perfect, i.e. $\delta v_\mathrm{theory}$ is negligible.

In the following, we also assume that the error in the measurement in $v$ is uncorrelated with the uncertainty in $H_0$.

Through application of Bayes' rule we can readily evaluate the posterior pdf $p(x|D,I)$ for the distance $x$ to the galaxy.

#### Case 1: Fixed $H_0$

\begin{align}
p(x | D,I) & \propto p(D | x, I) p(x|I) \\
& = \frac{1}{\sqrt{2\pi}\sigma_v} \exp \left( - \frac{(v_\mathrm{measured} - v_\mathrm{theory})^2}{2\sigma_v^2} \right) p(x|I)\\
&= \left\{ \begin{array}{ll} \frac{1}{\sqrt{2\pi}\sigma_v} \exp \left( - \frac{(v_\mathrm{measured} - H_0 x)^2}{2\sigma_v^2} \right) & \text{with }x \in [x_\mathrm{min},x_\mathrm{max}] \\
0 & \text{otherwise},
\end{array} \right.
\end{align}

where $p(x|I)$ is the prior for the distance, which we have assumed to be uniform, i.e. $p(x|I) \propto 1$ in some (possibly large) region $[x_\mathrm{min},x_\mathrm{max}]$.

```python
def x_with_fixedH(x,H0,vmeasured=vm,vsigma=sig_vm,xmin=0,xmax=10000):
    # Not including the prior
    x_posterior = np.exp(-(vmeasured-H0*x)**2/(2*vsigma**2))
    return x_posterior
```

#### Case 2: Using the inferred pdf for $H_0$

Here we use marginalization to obtain the desired posterior pdf $p(x|D,I)$ from the joint distribution of $p(x,H_0|D,I)$
$$
p(x|D,I) = \int_{-\infty}^\infty dH_0 p(x,H_0|D,I).
$$

Using Bayes' rule, the product rule, and the fact that $H_0$ is independent of $x$ we find that
$$
p(x|D,I) \propto p(x|I) \int dH_0 p(H_0|I) p(D|x,H_0,I),
$$
which means that we have expressed the quantity that we want (the posterior for $x$) in terms of quantities that we know.

The pdf $p(H_0 | I)$ is known via its $N$ samples $\{H_{i}\}$ generated by the MCMC sampler.

This means that we can approximate
$$
p(x |D,I) = \int dH_0 p(H_0|I) p(D|x,H_0,I) \approx \frac{1}{N} \sum_{i=1}^N p(D | x, H_0^{(i)}, I)
$$
where we have used $p(x|I) \propto 1$ and where $H_0^{(i)}$ is drawn from $p(H_0|I)$.

```python
x_arr = np.linspace(800,2000,1200)

xposterior_fixedH = x_with_fixedH(x_arr,bayesian_slope_mean)
xposterior_fixedH /= np.sum(xposterior_fixedH)

xposterior_pdfH = np.zeros_like(x_arr)
for H0 in slope_samples:
    xposterior_pdfH += x_with_fixedH(x_arr,H0)
xposterior_pdfH /= np.sum(xposterior_pdfH)
```

```python
fig, ax = plt.subplots(1,1, figsize=(8,6))
ax.plot(x_arr,xposterior_fixedH);
ax.plot(x_arr,xposterior_pdfH,'--');

print("The mean and 68% DoB of the inferred distance is")
for ix, xposterior in enumerate([xposterior_fixedH,xposterior_pdfH]):
    # Mean
    x_mean = np.min(x_arr[np.cumsum(xposterior)>0.5])
    # 68% DoB
    x_min = np.min(x_arr[np.cumsum(xposterior)>0.16])
    x_max = np.min(x_arr[np.cumsum(xposterior)>0.84])
    print(f"... Case {ix+1}: x_mean = {x_mean:.0f}; 68% DoB [-{x_mean-x_min:.0f},+{x_max-x_mean:.0f}]")
```

```python

```


# 15 min Intro to Persistent Homology
### Benjamin Matthias Ruppik, Max-Planck Institute for Mathematics, Bonn

## Filtrations of topological spaces

### Motivating example: Sublevel filtrations

A lot of topological spaces $X$ come with a natural *filtration*,
an increasing union of subspaces $X_{i}$,
i.e. for $i < j$ in the index set we have $X_{i} \subseteq X_{j}$.

Here is an example of a topological space $X$
(with a Morse function $f \colon X \rightarrow \mathbb{R}$) ...

... and the *sublevel set filtration*
$X_{t} = f^{-1}((-\infty, t])$:

### Persistent homology

The $i$-dimensional homology group of a space measures $i$-dimensional holes (or cavities).

\begin{equation}
H_{i}(X) = i\textrm{-dim cycles modulo boundaries}
\end{equation}

Here are some examples of 1-dimensional homology classes:

persist = "to stick around for a long time"

Example where a first homology class persists from its appearance until the end in red.

Example where a first homology class appears but soon thereafter is canceled at a maximum in green.

**Slogan: Persistent homology measures non-trivial cycles which can be detected in significant parts of the filtrations.**

## Toy data and Simplex trees

### Defining a simplicial complex with filtration

```python
import pandas
import gudhi
import numpy as np
import networkx
from pylab import *
```

```python
# Toy example working with the simplex tree

toy_st = gudhi.SimplexTree()

# 0-simplices in filtration level 0.0
for i in range(0, 5):
    toy_st.insert([i], 0.0)

# 1-simplices in filtration level 1.0
toy_st.insert([0, 1], 1.0)
toy_st.insert([0, 2], 1.0)
toy_st.insert([1, 3], 1.0)
toy_st.insert([2, 3], 1.0)

# additional 1-simplex in filtration level 2.0
toy_st.insert([1, 2], 2.0)

# last 1-simplex and 2-simplex
toy_st.insert([3, 4], 3.0)
toy_st.insert([0, 1, 2], 3.0)

for simplex in toy_st.get_filtration():
    print(simplex)
```

    ([0], 0.0)
    ([1], 0.0)
    ([2], 0.0)
    ([3], 0.0)
    ([4], 0.0)
    ([0, 1], 1.0)
    ([0, 2], 1.0)
    ([1, 3], 1.0)
    ([2, 3], 1.0)
    ([1, 2], 2.0)
    ([0, 1, 2], 3.0)
    ([3, 4], 3.0)

```python
toy_bar = toy_st.persistence(min_persistence=0.0, persistence_dim_max=True)
print(toy_bar)

gudhi.plot_persistence_barcode(persistence=toy_bar, legend=True)
```

```python
gudhi.plot_persistence_diagram(persistence=toy_bar, legend=True)
```

## Persistent homology applied to "real data": Clique complex of a graph

Repository of various graphs: https://networkrepository.com/index.php

Let us take a graph representing a social network:

Quote: "Data collected about Facebook pages (November 2017).
These datasets represent blue verified Facebook page networks of different categories. Nodes represent the pages and edges are mutual likes among them."

```python
def simplex_graph_from_csv(path_to_csv, delimiter=','):
    """
    Input: Path to csv file with list of edges

    Output: 1-skeleton of simplex tree
    """
    # Create networkx graph from edge list
    G_nx = networkx.read_edgelist(path_to_csv, delimiter=',', nodetype=int)

    # Create simplex tree
    st = gudhi.SimplexTree()

    # Insert vertices into simplex tree
    for v in G_nx.nodes():
        st.insert([v], 0.0)
    # Insert edges into simplex tree
    for e in G_nx.edges():
        st.insert(e, 1.0)

    return st
```

```python
# Import the real data graph and build 1-complex

path_to_fb_pages_food_csv = './datasets/fb-pages-food/fb-pages-food.edges'

fb_pages_food_st = simplex_graph_from_csv(path_to_fb_pages_food_csv)

#for simplex in fb_pages_food_st.get_filtration():
#    print(simplex)

fb_pages_food_bar = fb_pages_food_st.persistence(min_persistence=-1.0, persistence_dim_max=True)
# print(fb_pages_food_bar)

gudhi.plot_persistence_barcode(fb_pages_food_bar, legend=True, max_intervals=0)
```

Clique complex of a graph:
We add a $k$-simplex for any clique of $(k+1)$-many vertices in the graph.

This is also known as the flag complex of a graph.

The clique complex of a graph $G$ can also be interpreted as a Vietoris–Rips complex for the metric space
where the points are vertices of $G$ and and the distances are the lengths of the shortest paths in $G$.

```python
def simplex_2_complex_from_csv(path_to_csv, delimiter=','):
    """
    Input: Path to csv file with list of edges

    Output: 2-skeleton of simplex tree for clique complex
    """
    # Create networkx graph from edge list
    G_nx = networkx.read_edgelist(path_to_csv, delimiter=',', nodetype=int)

    # Create simplex tree
    st = gudhi.SimplexTree()

    # Insert vertices into simplex tree
    for v in G_nx.nodes():
        st.insert([v], 0.0)
    # Insert edges into simplex tree
    for e in G_nx.edges():
        st.insert(e, 1.0)

    # Inserting cliques with 3 members
    for clq in networkx.clique.find_cliques(G_nx):
        if len(clq) == 3:
            print('Adding clique ', clq)
            st.insert(clq, 2.0)

    return st
```

```python
# Import the real data graph and build 2-complex

path_to_fb_pages_food_csv = './datasets/fb-pages-food/fb-pages-food.edges'

fb_pages_food_st = simplex_2_complex_from_csv(path_to_fb_pages_food_csv)

#for simplex in fb_pages_food_st.get_filtration():
#    print(simplex)
```

    Adding clique  [0, 603, 58]
    Adding clique  [3, 352, 40]
    Adding clique  [3, 65, 608]
    Adding clique  [3, 65, 185]
    Adding clique  [3, 65, 450]
    Adding clique  [3, 65, 299]
    Adding clique  [3, 484, 40]
    Adding clique  [3, 377, 608]
    Adding clique  [3, 377, 185]
    Adding clique  [3, 377, 450]
    Adding clique  [3, 377, 299]
    Adding clique  [8, 73, 369]
    Adding clique  [17, 65, 608]
    Adding clique  [17, 65, 185]
    Adding clique  [17, 65, 450]
    Adding clique  [17, 65, 299]
    Adding clique  [21, 329, 441]
    Adding clique  [21, 516, 334]
    Adding clique  [22, 187, 104]
    Adding clique  [23, 257, 157]
    Adding clique  [23, 434, 58]
    Adding clique  [23, 157, 288]
    Adding clique  [31, 449, 193]
    Adding clique  [31, 193, 41]
    Adding clique  [31, 193, 285]
    Adding clique  [31, 518, 327]
    Adding clique  [31, 518, 424]
    Adding clique  [31, 488, 524]
    Adding clique  [31, 491, 424]
    Adding clique  [31, 491, 142]
    Adding clique  [31, 507, 546]
    Adding clique  [39, 576, 603]
    Adding clique  [39, 340, 43]
    Adding clique  [39, 340, 603]
    Adding clique  [40, 240, 484]
    Adding clique  [46, 578, 417]
    Adding clique  [46, 578, 110]
    Adding clique  [46, 578, 86]
    Adding clique  [50, 67, 578]
    Adding clique  [50, 67, 389]
    Adding clique  [50, 67, 183]
    Adding clique  [50, 182, 289]
    Adding clique  [50, 182, 131]
    Adding clique  [50, 182, 389]
    Adding clique  [52, 113, 494]
    Adding clique  [63, 518, 139]
    Adding clique  [65, 240, 608]
    Adding clique  [65, 240, 185]
    Adding clique  [65, 240, 450]
    Adding clique  [65, 240, 299]
    Adding clique  [65, 333, 185]
    Adding clique  [69, 498, 385]
    Adding clique  [69, 498, 253]
    Adding clique  [72, 269, 70]
    Adding clique  [78, 331, 154]
    Adding clique  [79, 491, 357]
    Adding clique  [79, 518, 85]
    Adding clique  [79, 518, 357]
    Adding clique  [79, 518, 414]
    Adding clique  [79, 507, 546]
    Adding clique  [79, 507, 85]
    Adding clique  [81, 290, 563]
    Adding clique  [82, 451, 261]
    Adding clique  [82, 451, 389]
    Adding clique  [84, 329, 545]
    Adding clique  [84, 329, 374]
    Adding clique  [89, 128, 446]
    Adding clique  [89, 136, 155]
    Adding clique  [89, 136, 558]
    Adding clique  [89, 32, 300]
    Adding clique  [89, 300, 182]
    Adding clique  [89, 558, 254]
    Adding clique  [89, 558, 343]
    Adding clique  [89, 54, 275]
    Adding clique  [89, 245, 217]
    Adding clique  [91, 563, 253]
    Adding clique  [93, 146, 567]
    Adding clique  [93, 103, 94]
    Adding clique  [93, 103, 567]
    Adding clique  [101, 545, 124]
    Adding clique  [101, 374, 124]
    Adding clique  [102, 67, 43]
    Adding clique  [106, 305, 583]
    Adding clique  [109, 181, 320]
    Adding clique  [109, 181, 169]
    Adding clique  [115, 145, 485]
    Adding clique  [118, 518, 424]
    Adding clique  [118, 518, 357]
    Adding clique  [123, 179, 491]
    Adding clique  [124, 613, 545]
    Adding clique  [124, 613, 374]
    Adding clique  [130, 288, 340]
    Adding clique  [134, 232, 563]
    Adding clique  [135, 386, 458]
    Adding clique  [135, 446, 128]
    Adding clique  [141, 461, 574]
    Adding clique  [142, 164, 491]
    Adding clique  [154, 533, 214]
    Adding clique  [157, 289, 288]
    Adding clique  [157, 289, 90]
    Adding clique  [157, 289, 510]
    Adding clique  [157, 131, 288]
    Adding clique  [157, 131, 90]
    Adding clique  [157, 131, 510]
    Adding clique  [157, 603, 90]
    Adding clique  [162, 323, 498]
    Adding clique  [162, 323, 610]
    Adding clique  [164, 518, 327]
    Adding clique  [164, 488, 524]
    Adding clique  [164, 491, 424]
    Adding clique  [164, 507, 546]
    Adding clique  [170, 70, 340]
    Adding clique  [170, 70, 383]
    Adding clique  [174, 264, 318]
    Adding clique  [176, 478, 497]
    Adding clique  [176, 478, 362]
    Adding clique  [177, 389, 350]
    Adding clique  [178, 539, 495]
    Adding clique  [181, 476, 258]
    Adding clique  [185, 377, 240]
    Adding clique  [185, 377, 538]
    Adding clique  [185, 377, 356]
    Adding clique  [195, 90, 227]
    Adding clique  [199, 518, 316]
    Adding clique  [199, 518, 566]
    Adding clique  [202, 340, 288]
    Adding clique  [202, 340, 343]
    Adding clique  [203, 345, 435]
    Adding clique  [204, 540, 215]
    Adding clique  [206, 90, 9]
    Adding clique  [206, 90, 473]
    Adding clique  [208, 128, 446]
    Adding clique  [208, 455, 288]
    Adding clique  [208, 340, 288]
    Adding clique  [208, 340, 446]
    Adding clique  [208, 340, 343]
    Adding clique  [215, 405, 540]
    Adding clique  [222, 365, 522]
    Adding clique  [224, 248, 352]
    Adding clique  [224, 262, 470]
    Adding clique  [224, 470, 352]
    Adding clique  [235, 368, 363]
    Adding clique  [235, 472, 244]
    Adding clique  [240, 377, 608]
    Adding clique  [240, 377, 299]
    Adding clique  [240, 377, 450]
    Adding clique  [240, 377, 563]
    Adding clique  [260, 386, 432]
    Adding clique  [260, 386, 458]
    Adding clique  [262, 368, 470]
    Adding clique  [263, 613, 311]
    Adding clique  [265, 4, 254]
    Adding clique  [265, 7, 35]
    Adding clique  [265, 531, 345]
    Adding clique  [265, 531, 67]
    Adding clique  [265, 531, 436]
    Adding clique  [265, 549, 229]
    Adding clique  [265, 43, 360]
    Adding clique  [265, 43, 83]
    Adding clique  [265, 558, 136]
    Adding clique  [265, 558, 254]
    Adding clique  [265, 54, 275]
    Adding clique  [265, 59, 116]
    Adding clique  [265, 576, 128]
    Adding clique  [265, 578, 67]
    Adding clique  [265, 578, 588]
    Adding clique  [265, 68, 242]
    Adding clique  [265, 594, 41]
    Adding clique  [265, 599, 408]
    Adding clique  [265, 599, 346]
    Adding clique  [265, 599, 172]
    Adding clique  [265, 611, 1]
    Adding clique  [265, 611, 136]
    Adding clique  [265, 611, 426]
    Adding clique  [265, 611, 172]
    Adding clique  [265, 611, 432]
    Adding clique  [265, 611, 436]
    Adding clique  [265, 611, 458]
    Adding clique  [265, 611, 361]
    Adding clique  [265, 616, 352]
    Adding clique  [265, 616, 361]
    Adding clique  [265, 111, 229]
    Adding clique  [265, 120, 375]
    Adding clique  [265, 121, 433]
    Adding clique  [265, 121, 227]
    Adding clique  [265, 121, 41]
    Adding clique  [265, 143, 172]
    Adding clique  [265, 143, 397]
    Adding clique  [265, 143, 151]
    Adding clique  [265, 143, 408]
    Adding clique  [265, 159, 32]
    Adding clique  [265, 159, 360]
    Adding clique  [265, 159, 458]
    Adding clique  [265, 159, 397]
    Adding clique  [265, 183, 32]
    Adding clique  [265, 183, 67]
    Adding clique  [265, 183, 166]
    Adding clique  [265, 198, 432]
    Adding clique  [265, 227, 352]
    Adding clique  [265, 227, 436]
    Adding clique  [265, 229, 436]
    Adding clique  [265, 243, 35]
    Adding clique  [265, 243, 116]
    Adding clique  [265, 246, 182]
    Adding clique  [265, 261, 436]
    Adding clique  [265, 261, 397]
    Adding clique  [265, 302, 58]
    Adding clique  [265, 321, 41]
    Adding clique  [265, 321, 404]
    Adding clique  [265, 322, 245]
    Adding clique  [265, 360, 364]
    Adding clique  [265, 360, 404]
    Adding clique  [265, 360, 479]
    Adding clique  [265, 364, 32]
    Adding clique  [265, 364, 458]
    Adding clique  [265, 364, 397]
    Adding clique  [265, 364, 432]
    Adding clique  [265, 375, 404]
    Adding clique  [265, 386, 432]
    Adding clique  [265, 397, 41]
    Adding clique  [265, 397, 169]
    Adding clique  [265, 439, 136]
    Adding clique  [265, 465, 41]
    Adding clique  [265, 479, 432]
    Adding clique  [265, 479, 458]
    Adding clique  [269, 505, 410]
    Adding clique  [276, 555, 619]
    Adding clique  [282, 62, 392]
    Adding clique  [282, 62, 169]
    Adding clique  [289, 325, 238]
    Adding clique  [289, 56, 128]
    Adding clique  [289, 510, 217]
    Adding clique  [291, 67, 248]
    Adding clique  [291, 67, 107]
    Adding clique  [291, 67, 245]
    Adding clique  [292, 518, 328]
    Adding clique  [292, 518, 612]
    Adding clique  [292, 518, 421]
    Adding clique  [293, 570, 155]
    Adding clique  [297, 448, 518]
    Adding clique  [298, 360, 389]
    Adding clique  [299, 377, 538]
    Adding clique  [299, 377, 356]
    Adding clique  [313, 524, 488]
    Adding clique  [319, 522, 619]
    Adding clique  [323, 242, 434]
    Adding clique  [323, 242, 498]
    Adding clique  [325, 131, 238]
    Adding clique  [327, 480, 518]
    Adding clique  [328, 491, 508]
    Adding clique  [328, 518, 508]
    Adding clique  [329, 609, 545]
    Adding clique  [329, 609, 374]
    Adding clique  [335, 518, 414]
    Adding clique  [339, 444, 7]
    Adding clique  [342, 90, 597]
    Adding clique  [342, 343, 372]
    Adding clique  [342, 343, 340]
    Adding clique  [347, 524, 488]
    Adding clique  [348, 424, 518]
    Adding clique  [357, 518, 430]
    Adding clique  [357, 518, 532]
    Adding clique  [362, 429, 478]
    Adding clique  [368, 352, 470]
    Adding clique  [377, 538, 608]
    Adding clique  [377, 538, 450]
    Adding clique  [385, 498, 352]
    Adding clique  [389, 451, 554]
    Adding clique  [389, 451, 182]
    Adding clique  [389, 451, 111]
    Adding clique  [389, 229, 136]
    Adding clique  [389, 229, 554]
    Adding clique  [389, 229, 111]
    Adding clique  [389, 229, 446]
    Adding clique  [389, 454, 432]
    Adding clique  [389, 531, 67]
    Adding clique  [389, 350, 446]
    Adding clique  [403, 526, 517]
    Adding clique  [403, 526, 70]
    Adding clique  [410, 505, 67]
    Adding clique  [416, 67, 248]
    Adding clique  [416, 67, 107]
    Adding clique  [416, 67, 245]
    Adding clique  [421, 518, 524]
    Adding clique  [424, 480, 518]
    Adding clique  [434, 90, 242]
    Adding clique  [434, 90, 597]
    Adding clique  [445, 229, 576]
    Adding clique  [448, 518, 585]
    Adding clique  [448, 518, 487]
    Adding clique  [451, 345, 182]
    Adding clique  [457, 524, 518]
    Adding clique  [459, 340, 288]
    Adding clique  [459, 340, 343]
    Adding clique  [460, 88, 498]
    Adding clique  [463, 182, 350]
    Adding clique  [473, 90, 217]
    Adding clique  [473, 90, 182]
    Adding clique  [473, 245, 217]
    Adding clique  [473, 245, 340]
    Adding clique  [474, 88, 498]
    Adding clique  [474, 88, 550]
    Adding clique  [478, 497, 535]
    Adding clique  [488, 596, 524]
    Adding clique  [491, 612, 515]
    Adding clique  [498, 526, 352]
    Adding clique  [498, 526, 70]
    Adding clique  [498, 242, 248]
    Adding clique  [515, 612, 518]
    Adding clique  [518, 579, 524]
    Adding clique  [518, 566, 524]
    Adding clique  [522, 555, 619]
    Adding clique  [526, 536, 70]
    Adding clique  [526, 536, 558]
    Adding clique  [536, 288, 70]
    Adding clique  [536, 70, 116]
    Adding clique  [536, 558, 136]
    Adding clique  [536, 558, 116]
    Adding clique  [536, 558, 343]
    Adding clique  [550, 238, 90]
    Adding clique  [557, 90, 227]
    Adding clique  [557, 90, 182]
    Adding clique  [557, 90, 151]
    Adding clique  [577, 67, 43]
    Adding clique  [580, 151, 345]
    Adding clique  [580, 151, 143]
    Adding clique  [581, 32, 300]
    Adding clique  [581, 143, 408]
    Adding clique  [581, 143, 172]
    Adding clique  [581, 143, 397]
    Adding clique  [589, 601, 217]
    Adding clique  [593, 576, 288]
    Adding clique  [593, 558, 116]
    Adding clique  [593, 558, 343]
    Adding clique  [593, 340, 343]
    Adding clique  [603, 576, 361]
    Adding clique  [603, 361, 616]
    Adding clique  [603, 558, 198]
    Adding clique  [617, 217, 56]
    Adding clique  [617, 217, 317]

```python
fb_pages_food_bar = fb_pages_food_st.persistence(min_persistence=-1.0, persistence_dim_max=True)
#print(fb_pages_food_bar)

gudhi.plot_persistence_barcode(fb_pages_food_bar, legend=True, max_intervals=0)

gudhi.plot_persistence_diagram(fb_pages_food_bar, legend=True, max_intervals=0)
```

This is still a little boring, since homology classes only appear
at integral time steps.

## The weight rank clique filtration

Quoting from Nguyen, Aktas, Akbas: Bot Detection on Social Networks Using Persistent Homology:

"Instead of working the clique complex of the whole graph, we can
induce a family of simplicial complexes out of the clique complex of the subgraphs of G and track how
homology changes in this family"

"One of the popular filtration methods defined on undirected weighted networks is called
weight rank clique filtration. This filtration, first defined in [22], uses weights as threshold values
δ. The authors first rank the edge weights from wmin to wmax (it can also be done in other direction,
from wmax to wmin as well) and let the parameter increase from wmin to wmax. At each step δ, they just
consider the thresholded subgraph Gδ ⊆ G that is the subgraph with edges of weight larger than δ.
Then, they create the clique complex of Gδ
to obtain the filtration."

```python
# Import the real data graph and build weighted 1-complex

path_to_fb_pages_food_csv = './datasets/fb-pages-food/fb-pages-food.edges'

# Create networkx graph from edge list
G_nx = networkx.read_edgelist(path_to_fb_pages_food_csv, delimiter=',', nodetype=int)
print("Imported graph with", G_nx.number_of_nodes(), "nodes and", G_nx.number_of_edges(), "edges.")

# Create simplex tree
fb_pages_food_st = gudhi.SimplexTree()

# Insert vertices into simplex tree
for v in G_nx.nodes():
    if G_nx.degree(v) != 0:
        fb_pages_food_st.insert([v], 1/(G_nx.degree(v)))

fb_pages_food_bar = fb_pages_food_st.persistence(min_persistence=-1.0, persistence_dim_max=True)
# print(fb_pages_food_bar)

gudhi.plot_persistence_barcode(fb_pages_food_bar, legend=True, max_intervals=0)
gudhi.plot_persistence_diagram(fb_pages_food_bar, legend=True, max_intervals=0)
```

```python
#Insert edges into simplex tree with filtration value the maximum of its endpoints
for e in G_nx.edges():
    fb_pages_food_st.insert(e, max(fb_pages_food_st.filtration([e[0]]), fb_pages_food_st.filtration([e[1]])))

fb_pages_food_bar = fb_pages_food_st.persistence(min_persistence=-1.0, persistence_dim_max=True)

gudhi.plot_persistence_barcode(fb_pages_food_bar, legend=True, max_intervals=0)
gudhi.plot_persistence_diagram(fb_pages_food_bar, legend=True, max_intervals=0)
```

```python
# Expand simplex tree by constructing flag complex up to max_dim,
# the filtration of a simplex is the maximal filtration index of its vertices
fb_pages_food_st.expansion(4)

fb_pages_food_bar = fb_pages_food_st.persistence(min_persistence=-1.0, persistence_dim_max=True)

gudhi.plot_persistence_barcode(fb_pages_food_bar, legend=True, max_intervals=0)
gudhi.plot_persistence_diagram(fb_pages_food_bar, legend=True, max_intervals=0)
```

```python
def simplex_graph_with_weight_rank_clique_filtration_from_csv(path_to_csv, delimiter=',', max_dim=4):
    """
    Input: Path to csv file with list of edges

    Output: Build the clique complex of the graph
    where the vertices are weighted by the reciprocal of their degree
    """
    # Create networkx graph from edge list
    G_nx = networkx.read_edgelist(path_to_csv, delimiter=',', nodetype=int)
    print("Imported graph with", G_nx.number_of_nodes(), "nodes and", G_nx.number_of_edges(), "edges.")

    # Create simplex tree
    st = gudhi.SimplexTree()

    # Insert vertices into simplex tree
    for v in G_nx.nodes():
        if G_nx.degree(v) != 0:
            st.insert([v], 1/(G_nx.degree(v)))

    # Insert edges into simplex tree with
    # filtration value the maximum of its endpoints
    for e in G_nx.edges():
        st.insert(e, max(st.filtration([e[0]]), st.filtration([e[1]])))

    # Expand simplex tree by constructing flag complex up to max_dim,
    # the filtration of a simplex is the maximal filtration index of its vertices
    st.expansion(max_dim)

    return st
```

```python
def simplex_graph_with_normalized_weight_rank_clique_filtration_from_csv(path_to_csv,
                                                                         delimiter=',',
                                                                         max_dim=4):
    """
    Input: Path to csv file with list of edges

    Output: Build the clique complex of the graph
    where the vertices are weighted by the reciprocal of their degree
    and we have normalized to take the maximal degree into account
    """
    # Create networkx graph from edge list
    G_nx = networkx.read_edgelist(path_to_csv, delimiter=',', nodetype=int)
    print("Imported graph with", G_nx.number_of_nodes(), "nodes and", G_nx.number_of_edges(), "edges.")

    # Calculating maximal degree
    # (will be needed to normalize the filtration later)
    degrees = [val for (node, val) in G_nx.degree()]
    max_degree = max(degrees)
    print("Maximal degree in the graph is", max_degree)

    # Create simplex tree
    st = gudhi.SimplexTree()

    # Insert vertices into simplex tree
    # Filtration by degree of vertex, where the degree
    # is scaled linearly to lie in the interval [0, 1]
    for v in G_nx.nodes():
        if G_nx.degree(v) != 0:
            st.insert([v], 1 - G_nx.degree(v)/max_degree)

    # Insert edges into simplex tree with
    # filtration value the maximum of its endpoints
    for e in G_nx.edges():
        st.insert(e, max(st.filtration([e[0]]), st.filtration([e[1]])))

    # Expand simplex tree by constructing flag complex up to max_dim,
    # the filtration of a simplex is the maximal filtration index of its vertices
    st.expansion(max_dim)

    return st
```

```python
# Building normalized filtered clique complex for the food pages graph

path_to_fb_pages_food_csv = './datasets/fb-pages-food/fb-pages-food.edges'

fb_pages_food_normalized_st = simplex_graph_with_normalized_weight_rank_clique_filtration_from_csv(path_to_fb_pages_food_csv,
                                                                                delimiter=',',
                                                                                max_dim=4)

# for simplex in fb_pages_government_st.get_filtration():
#     print(simplex)

fb_pages_normalized_food_bar = fb_pages_food_normalized_st.persistence(min_persistence=-1.0,
                                                                       persistence_dim_max=True)

# print(len(fb_pages_government_bar))
# for bar in fb_pages_government_bar:
#     print(bar)

gudhi.plot_persistence_diagram(fb_pages_normalized_food_bar, legend=True, max_intervals=0)
```

## Conclusion

We could now analyze the persistence diagrams further
and for example compare the food pages graph with say a politics graph.

```python
# fb-pages_food.edges is already imported above

path_to_fb_pages_government_csv = './datasets/fb-pages-government/fb-pages-government.edges'

fb_pages_government_st = simplex_graph_with_weight_rank_clique_filtration_from_csv(path_to_fb_pages_government_csv,
                                                                                delimiter=',',
                                                                                max_dim=2)

# for simplex in fb_pages_government_st.get_filtration():
#     print(simplex)

fb_pages_government_bar = fb_pages_government_st.persistence(min_persistence=-1.0, persistence_dim_max=True)

# print(len(fb_pages_government_bar))
# for bar in fb_pages_government_bar:
#     print(bar)

# gudhi.plot_persistence_diagram(fb_pages_government_bar, legend=True, max_intervals=500)

##########

# Comparing different persistence diagrams using the bottleneck distance
# persistence_interval_dim_1_1 = fb_pages_food_st.persistence_intervals_in_dimension(1)
# persistence_interval_dim_1_2 = fb_pages_government_st.persistence_intervals_in_dimension(1)

# print(gudhi.bottleneck_distance(persistence_interval_dim_1_1,
#                                 persistence_interval_dim_1_2))

##########

path_to_fb_pages_politician_csv = './datasets/fb-pages-politician/fb-pages-politician.edges'
```

    Imported graph with 7057 nodes and 89455 edges.

```python
# Unfortunately, this graph as given is too large,
# so we would need to prune it before processing

path_to_fb_pages_sport_csv = './datasets/fb-pages-sport/fb-pages-sport.edges'

fb_pages_sport_st = simplex_graph_with_weight_rank_clique_filtration_from_csv(path_to_fb_pages_sport_csv,
                                                                              delimiter=',',
                                                                              max_dim=2)

# fb_pages_sport_bar = fb_pages_sport_st.persistence(min_persistence=-1.0, persistence_dim_max=True)

# gudhi.plot_persistence_diagram(fb_pages_sport_bar, legend=True, max_intervals=0)
```

    Imported graph with 13866 nodes and 86858 edges.

### Properties of the data we could detect with persistent homology

 * Qualitative large scale features in the data
 * Measures both the global and local topology of a dataset
 * We can use the persistence data as input for further machine learning applications

### Drawbacks of using persistent homology

 * Have to deal with noise in the data and stability
 * For example, points close to the diagonal in the persistence diagram only appear for a short time: these could come from random fluctuations in the data, but could also represent local properties
 * It can be challanging to interpret the meaning of the persistent homology groups for the application

## References

 * An introduction to Topological Data Analysis: fundamental and practical aspects for data scientists: https://arxiv.org/abs/1710.04019
 * Bot Detection on Social Networks Using Persistent Homology: https://www.semanticscholar.org/paper/Bot-Detection-on-Social-Networks-Using-Persistent-Nguyen-Aktas/e3944fac408415965b3d24d52d3ac7b7a0e9aa17
 * Persistent Homology of Collaboration Networks: https://www.hindawi.com/journals/mpe/2013/815035/

### Python libraries used

#### TDA tools

 * [GUDHI Python module](https://gudhi.inria.fr/)
 * https://github.com/GUDHI/TDA-tutorial

#### Graphs

 * https://networkx.org/

### Datasets

 @inproceedings{nr,
      title = {The Network Data Repository with Interactive Graph Analytics and Visualization},
      author={Ryan A. Rossi and Nesreen K. Ahmed},
      booktitle = {AAAI},
      url={https://networkrepository.com},
      year={2015}
 }

### Appendix: Vietoris-Rips complex from Distance matrix

We will consider the following graph with 5 vertices where pairwise distances are encoded in the following matrix:

```python
toy_matrix = [[0,1,1,2,inf],
              [1,0,2,1,inf],
              [1,2,0,1,inf],
              [2,1,1,0,inf],
              [inf,inf,inf,inf,0]]
```

Vietoris-Rips filtration for $r \in \mathbb{R}_{\ge 0}$:
Insert a simplex for any finite set of vertices that has diameter at most $r$.

```python
graph_distance_matrix = toy_matrix

# Build Vietoris-Rips complex

skeleton = gudhi.RipsComplex(
    distance_matrix = graph_distance_matrix,
    max_edge_length = 5.0
)

Rips_simplex_tree = skeleton.create_simplex_tree(max_dimension = 3)

for simplex in Rips_simplex_tree.get_filtration():
    print(simplex)
```

    ([0], 0.0)
    ([1], 0.0)
    ([2], 0.0)
    ([3], 0.0)
    ([4], 0.0)
    ([0, 1], 1.0)
    ([0, 2], 1.0)
    ([1, 3], 1.0)
    ([2, 3], 1.0)
    ([1, 2], 2.0)
    ([0, 1, 2], 2.0)
    ([0, 3], 2.0)
    ([0, 1, 3], 2.0)
    ([0, 2, 3], 2.0)
    ([1, 2, 3], 2.0)
    ([0, 1, 2, 3], 2.0)

```python
# Compute persistence of the simplex tree
BarCodes_Rips = Rips_simplex_tree.persistence(min_persistence=0.0, persistence_dim_max=True)

print(BarCodes_Rips)
```

    [(1, (1.0, 2.0)), (0, (0.0, inf)), (0, (0.0, inf)), (0, (0.0, 1.0)), (0, (0.0, 1.0)), (0, (0.0, 1.0))]

```python
gudhi.plot_persistence_diagram(persistence=BarCodes_Rips, legend=True)
```

```python
gudhi.plot_persistence_barcode(persistence=BarCodes_Rips, legend=True)
```


# Task One : Fuandamentals of Qunatum Computation

## Basic Arithmatic Operations & Complex Numbers

```python
5+9
```

    14

```python
12*4
```

    48

Complex numbers are always of the form

\begin{align}
\alpha = a + bi
\end{align}

```python
1j * 1j
```

    (-1+0j)

```python
z = 3 + 4j
w = 1 - 2j
```

```python
import numpy as np
```

```python
print('z=', z)
print('Real(z)=', np.real(z))
print('Imag(z)=', np.imag(z))
```

    z= (3+4j)
    Real(z)= 3.0
    Imag(z)= 4.0

```python
z+w
```

    (4+2j)

```python
z-w
```

    (2+6j)

```python
z/w
```

    (-1+2j)

## Complex conjugate

```python
z
```

    (3+4j)

```python
np.conj(z)
```

    (3-4j)

## Norms/Absolute Values

\begin{align} ||z|| &= \sqrt{zz^*} = \sqrt{|z|^2},\\ ||w|| &= \sqrt{ww^*} = \sqrt{|w|^2}, \end{align}

```python
print('||z||=', np.abs(z))
```

    ||z||= 5.0

```python
print('||w||=', np.abs(w))
```

    ||w||= 2.23606797749979

## Row Vectors, Column Vectors, and Bra-Ket Notation

\begin{align} \text{Column Vector:} \ \begin{pmatrix}
a_1 \\ a_2 \\ \vdots \\ a_n
\end{pmatrix}
\quad \quad \text{Row Vector:} \ \begin{pmatrix}
a_1, & a_2, & \cdots, & a_n
\end{pmatrix} \end{align}

```python
# Create a vector as a row
row_vector = np.array([2-1j, 7j, -3])

# Create a vector as a column
column_vector = np.array([[2+1j],
                          [-5],
                          [2j]])
```

```python
row_vector
```

    array([ 2.-1.j,  0.+7.j, -3.+0.j])

Row vectors in quantum mechanics are also called **bra-vectors**, and are denoted as follows:

\begin{align} \langle A| = \begin{pmatrix}
a_1, & a_2, \cdots, & a_n
\end{pmatrix} \end{align}

Column vectors are also called **ket-vectors** in quantum mechanics denoted as follows:

\begin{align} |B\rangle = \begin{pmatrix}
b_1 \\ b_2 \\ \vdots \\ b_n
\end{pmatrix} \end{align}

In general, if we have a column vector, i.e. a ket-vector:

\begin{align} |A\rangle = \begin{pmatrix}
a_1 \\ a_2 \\ \vdots \\ a_n
\end{pmatrix} \end{align}

the corresponding bra-vector:

\begin{align} \langle A| = \begin{pmatrix}
a_1^*, & a_2^*, & \cdots, & a_n^*
\end{pmatrix} \end{align}

```python
ket_A = np.array([[2-1j],
                  [5]])
ket_B = np.array([[7],
                  [-3j]])
```

```python
print(ket_A + ket_B)
```

    [[9.-1.j]
     [5.-3.j]]

## Inner Product

\begin{align} \langle A| = \begin{pmatrix}
a_1, & a_2, & \cdots, & a_n
\end{pmatrix}, \quad \quad
|B\rangle = \begin{pmatrix}
b_1 \\ b_2 \\ \vdots \\ b_n
\end{pmatrix} \end{align}

Taking the inner product of $\langle A|$ and $|B\rangle$ gives the following:

\begin{align} \langle A| B \rangle &= \begin{pmatrix}
a_1, & a_2, & \cdots, & a_n
\end{pmatrix}
\begin{pmatrix}
b_1 \\ b_2 \\ \vdots \\ b_n
\end{pmatrix}\\
&=
a_1b_1 + a_2b_2 + \cdots + a_nb_n\\
&= \sum_{i=1}^n a_ib_i
\end{align}

```python
# Define the 4x1 matrix version of a column vector (instead of using the np.array() version):
A = np.matrix([[1-1j],
               [3],
               [2j],
               [5+1j]])
A
```

    matrix([[1.-1.j],
            [3.+0.j],
            [0.+2.j],
            [5.+1.j]])

```python
# Define B as a 1x4 matrix
B = np.matrix([[-3j, 2+2j, -6j, -7]])

B
```

    matrix([[-0.-3.j,  2.+2.j, -0.-6.j, -7.+0.j]])

```python
# Compute <B|A>
np.dot(B,A)
```

    matrix([[-20.-4.j]])

## Matrices

\begin{align}
M = \begin{pmatrix}
2-i & -3 \\
-5i & 2
\end{pmatrix}
\end{align}

```python
M = np.array([[2-1j, -3],
              [-5j, 2]])

print(M)
```

    [[ 2.-1.j -3.+0.j]
     [-0.-5.j  2.+0.j]]

```python
M = np.matrix([[2-1j, -3],
               [-5j, 2]])

print(M)
```

    [[ 2.-1.j -3.+0.j]
     [-0.-5.j  2.+0.j]]

Hermitian conjugates are given by taking the conjugate transpose of the matrix

```python
print(M.H)
```

    [[ 2.+1.j -0.+5.j]
     [-3.-0.j  2.-0.j]]

## Tensor Products of Matrices

\begin{align}
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix} \otimes
\begin{pmatrix}
x & y \\
z & w
\end{pmatrix} =
\begin{pmatrix}
a \begin{pmatrix}
x & y \\
z & w
\end{pmatrix} & b \begin{pmatrix}
x & y \\
z & w
\end{pmatrix} \\
c \begin{pmatrix}
x & y \\
z & w
\end{pmatrix} & d \begin{pmatrix}
x & y \\
z & w
\end{pmatrix}
\end{pmatrix} =
\begin{pmatrix}
ax & ay & bx & by \\
az & aw & bz & bw \\
cx & cy & dx & dy \\
cz & cw & dz & dw
\end{pmatrix}
\end{align}

```python
A = np.matrix([[1j+8 , 2j-6], []])

```

# Task Two: Qubits, Bloch Sphere and Basis States

```python
import qiskit
qiskit.__qiskit_version__
```

    {'qiskit-terra': '0.14.2',
     'qiskit-aer': '0.5.2',
     'qiskit-ignis': '0.3.3',
     'qiskit-ibmq-provider': '0.7.2',
     'qiskit-aqua': '0.7.3',
     'qiskit': '0.19.6'}

\begin{align}
|\psi \rangle = \begin{pmatrix}
\alpha \\ \beta
\end{pmatrix}, \quad \text{where } \sqrt{\langle \psi | \psi \rangle} = 1.
\end{align}

Think of Qubit as an Electron:

\begin{align}
\text{spin-up}: \ |0\rangle &= \begin{pmatrix} 1\\0 \end{pmatrix} \\
\text{spin-down}: \ |1\rangle & = \begin{pmatrix} 0\\1 \end{pmatrix}
\end{align}

```python
import numpy as np
from qiskit import *
from qiskit.visualization import plot_bloch_vector
%matplotlib inline
```

Another representation is via Bloch Sphere:

```python
plot_bloch_vector([0,0,1], title="Spin-Up")
```

## Spin + / - :

\begin{align}
\text{spin +}: \ |+\rangle &= \begin{pmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{pmatrix} = \frac{1}{\sqrt{2}} \left(|0\rangle + |1\rangle\right) \\
\text{spin -}: \ |-\rangle & = \begin{pmatrix} 1/\sqrt{2} \\ -1/\sqrt{2} \end{pmatrix} = \frac{1}{\sqrt{2}} \left(|0\rangle - |1\rangle\right)
\end{align}

```python
plot_bloch_vector([1,0,0], title="spin-plus")
```

## Basis States

\begin{align}
 |0\rangle &= \begin{pmatrix} 1\\0 \end{pmatrix} \\
 |1\rangle & = \begin{pmatrix} 0\\1 \end{pmatrix}
\end{align}
Preapring other states from Basis States:
\begin{align}
|00 \rangle &= |0\rangle \otimes |0\rangle = \begin{pmatrix} 1\\0 \end{pmatrix} \otimes \begin{pmatrix} 1\\0 \end{pmatrix} = \begin{pmatrix} 1\\0\\0\\0 \end{pmatrix} \\
|01 \rangle &= |0\rangle \otimes |1\rangle = \begin{pmatrix} 1\\0 \end{pmatrix} \otimes \begin{pmatrix} 0\\1 \end{pmatrix} = \begin{pmatrix} 0\\1\\0\\0 \end{pmatrix} \\
|10 \rangle &= |1\rangle \otimes |0\rangle = \begin{pmatrix} 0\\1 \end{pmatrix} \otimes \begin{pmatrix} 1\\0 \end{pmatrix} = \begin{pmatrix} 0\\0\\1\\0 \end{pmatrix} \\
|11 \rangle &= |1\rangle \otimes |1\rangle = \begin{pmatrix} 0\\1 \end{pmatrix} \otimes \begin{pmatrix} 0\\1 \end{pmatrix} = \begin{pmatrix} 0\\0\\0\\1 \end{pmatrix}
\end{align}

```python
qubit1_state = np.matrix([[1],[0]])
qubit2_state = np.matrix([[0],[1]])

two_qubit_state =np.kron(qubit1_state, qubit2_state)
```

```python

```

# Task Three: Qunatum Gates and Circuits

```python
from qiskit import *
from qiskit.visualization import plot_bloch_multivector
```

## Pauli Matrices

\begin{align}
I = \begin{pmatrix} 1&0 \\ 0&1 \end{pmatrix}, \quad
X = \begin{pmatrix} 0&1 \\ 1&0 \end{pmatrix}, \quad
Y = \begin{pmatrix} 0&i \\ -i&0 \end{pmatrix}, \quad
Z = \begin{pmatrix} 1&0 \\ 0&-1 \end{pmatrix} \quad
\end{align}

## X-gate

The X-gate is represented by the Pauli-X matrix:

$$ X = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = |0\rangle\langle1| + |1\rangle\langle0| $$

Effect a gate has on a qubit:

$$ X|0\rangle = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} = |1\rangle$$

```python
# Let's do an X-gate on a |0> qubit
qc = QuantumCircuit(1)
qc.x(0)
qc.draw('mpl')
```

```python
# Let's see the result
backend = Aer.get_backend('statevector_simulator')
out = execute(qc,backend).result().get_statevector()
plot_bloch_multivector(out)
```

## Z & Y-Gate

$$ Y = \begin{bmatrix} 0 & -i \\ i & 0 \end{bmatrix} \quad\quad\quad\quad Z = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} $$

$$ Y = -i|0\rangle\langle1| + i|1\rangle\langle0| \quad\quad Z = |0\rangle\langle0| - |1\rangle\langle1| $$

```python
qc.y(0) # Do Y-gate on qubit 0
qc.z(0) # Do Z-gate on qubit 0
qc.draw('mpl')
```

## Hadamard Gate

$$ H = \tfrac{1}{\sqrt{2}}\begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} $$

We can see that this performs the transformations below:

$$ H|0\rangle = |+\rangle $$

$$ H|1\rangle = |-\rangle $$

```python
qc = QuantumCircuit(3)
# Apply H-gate to each qubit:
for qubit in range(3):
    qc.h(qubit)
# See the circuit:
qc.draw('mpl')
```

## Identity Gate

$$
I = \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}
$$

$$ I = XX $$

```python
qc = QuantumCircuit(1)
qc.i(0)
qc.draw('mpl')
```

 ** Other Gates: S-gate , T-gate, U-gate

## Multiple Qubits, Entanglement

## Multiple Qubits

The state of two qubits :

$$ |psi\rangle = a_{00}|00\rangle + a_{01}|01\rangle + a_{10}|10\rangle + a_{11}|11\rangle = \begin{bmatrix} a_{00} \\ a_{01} \\ a_{10} \\ a_{11} \end{bmatrix} $$

```python
from qiskit import *
```

```python
qc = QuantumCircuit(3)

for qubit in range(3):
    qc.h(qubit)

qc.draw('mpl')
```

Each qubit is in the state $|+\rangle$, so we should see the vector:

$$
|{+++}\rangle = \frac{1}{\sqrt{8}}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \\
                              1 \\ 1 \\ 1 \\ 1 \\
              \end{bmatrix}
$$

```python
backend = Aer.get_backend('statevector_simulator')
final_state= execute(qc, backend).result().get_statevector()
print(final_state)
```

    [0.35355339+0.j 0.35355339+0.j 0.35355339+0.j 0.35355339+0.j
     0.35355339+0.j 0.35355339+0.j 0.35355339+0.j 0.35355339+0.j]

```python
qc = QuantumCircuit(2)

qc.x(0)
qc.h(1)

qc.draw('mpl')
```

$$
X|q_1\rangle \otimes H|q_0\rangle = (X\otimes H)|q_1 q_0\rangle
$$

The operation looks like this:

$$
X\otimes H = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \otimes \tfrac{1}{\sqrt{2}}\begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} = \frac{1}{\sqrt{2}}
\begin{bmatrix} 0 & 0 & 1 & 1 \\
                0 & 0 & 1 & -1 \\
                1 & 1 & 0 & 0 \\
                1 & -1 & 0 & 0 \\
\end{bmatrix}
$$

Which we can then apply to our 4D statevector $|q_1 q_0\rangle$. You will often see the clearer notation:

$$
X\otimes H =
\begin{bmatrix} 0 & H \\
               H & 0\\
\end{bmatrix}
$$

## C-Not Gate

```python
qc = QuantumCircuit(2)

qc.cx(0, 1)

qc.draw('mpl')

```

Classical truth table of C-Not gate:

| Input (t,c) | Output (t,c) |
|:-----------:|:------------:|
| 00          | 00           |
| 01          | 11           |
| 10          | 10           |
| 11          | 01           |

## Entanglement

```python
qc = QuantumCircuit(2)
# Apply H-gate to the first:
qc.h(0)
qc.draw('mpl')
```

```python
# Let's see the result:
backend = Aer.get_backend('statevector_simulator')
final_state = execute(qc,backend).result().get_statevector()
print(final_state)
```

    [0.70710678+0.j 0.70710678+0.j 0.        +0.j 0.        +0.j]

Quantum System Sate is:

$$
|0{+}\rangle = \tfrac{1}{\sqrt{2}}(|00\rangle + |01\rangle)
$$

```python
qc = QuantumCircuit(2)
# Apply H-gate to the first:
qc.h(0)
# Apply a CNOT:
qc.cx(0,1)
qc.draw('mpl')
```

```python
# Let's see the result:
backend = Aer.get_backend('statevector_simulator')
final_state = execute(qc,backend).result().get_statevector()
print(final_state)
```

    [0.70710678+0.j 0.        +0.j 0.        +0.j 0.70710678+0.j]

We see we have this final state (Bell State):

$$
\text{CNOT}|0{+}\rangle = \tfrac{1}{\sqrt{2}}(|00\rangle + |11\rangle)
$$

Other Bell States:

## Bernstein-Vazirani Algorithm

A black-box function $f$, which takes as input a string of bits ($x$), and returns either $0$ or $1$, that is:

$$f(\{x_0,x_1,x_2,...\}) \rightarrow 0 \textrm{ or } 1 \textrm{ where } x_n \textrm{ is }0 \textrm{ or } 1  $$

The function is guaranteed to return the bitwise product of the input with some string, $s$.

In other words, given an input $x$, $f(x) = s \cdot x \, \text{(mod 2)} =\ x_0 * s_0+x_1*s_1+x_2*s_2+...\ $ mod 2

# Classical-solution:

|Input(x)|
|:-----:|
|100...0|
|010...0|
|001...0|
|000...1|

Where each query reveals a different bit of $s$ (the bit $s_i$)

# The quantum Bernstein-Vazirani Oracle:

1. Initialise the inputs qubits to the $|0\rangle^{\otimes n}$ state, and output qubit to $|{-}\rangle$.
2. Apply Hadamard gates to the input register
3. Query the oracle
4. Apply Hadamard gates to the input register
5. Measure

## Example Two Qubits:

<ol>
    <li> The register of two qubits is initialized to zero:

$$\lvert \psi_0 \rangle = \lvert 0 0 \rangle$$

   </li>

   <li> Apply a Hadamard gate to both qubits:

$$\lvert \psi_1 \rangle = \frac{1}{2} \left( \lvert 0 0 \rangle + \lvert 0 1 \rangle + \lvert 1 0 \rangle + \lvert 1 1 \rangle \right) $$

   </li>

   <li> For the string $s=11$, the quantum oracle performs the operation:
$$
|x \rangle \xrightarrow{f_s} (-1)^{x\cdot 11} |x \rangle.
$$

$$\lvert \psi_2 \rangle = \frac{1}{2} \left( (-1)^{00\cdot 11}|00\rangle + (-1)^{01\cdot 11}|01\rangle + (-1)^{10\cdot 11}|10\rangle + (-1)^{11\cdot 11}|11\rangle \right)$$

$$\lvert \psi_2 \rangle = \frac{1}{2} \left( \lvert 0 0 \rangle - \lvert 0 1 \rangle - \lvert 1 0 \rangle + \lvert 1 1 \rangle \right)$$

   </li>

   <li> Apply a Hadamard gate to both qubits:

$$\lvert \psi_3 \rangle = \lvert 1 1 \rangle$$

   </li>

   <li> Measure to find the secret string $s=11$
   </li>

</ol>

```python
from qiskit import *
%matplotlib inline
from qiskit.tools.visualization import plot_histogram
```

```python
secret_num = '101001'
```

```python
circuit = QuantumCircuit(6+1,6)
circuit.h([0,1,2,3,4,5])

circuit.x(6)
circuit.h(6)

circuit.barrier()

circuit.cx(5,6)
circuit.cx(3,6)
circuit.cx(0,6)

circuit.barrier()
circuit.h([0,1,2,3,4,5])

circuit.measure([0,1,2,3,4,5],[0,1,2,3,4,5])
```

    <qiskit.circuit.instructionset.InstructionSet at 0x277a2388>

```python
circuit.draw(output= 'mpl')
```

```python
simulator = Aer.get_backend('qasm_simulator')
result = execute(circuit, backend= simulator, shots = 1).result()
counts= result.get_counts()
print(counts)
```

    {'101001': 1}

```python

```