#  NMA Model Types Tutorial 3: "Why" models

In this tutorial we will explore models and techniques that can potentially explain *why* the spiking data we have observed is produced the way it is.

Tutorial objectives:

- Write code to compute formula for entropy, a measure of information
- Compute the entropy of a number of toy distributions
- Compute the entropy of spiking activity from the Steinmetz dataset

```python
#@title Video: “Why” models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='fxbBJu258oE', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video

```

    Video available at https://youtube.com/watch?v=fxbBJu258oE

## Setup

```python
#@title Imports
import io

import matplotlib.pyplot as plt
import numpy as np
import requests
import scipy.stats as stats

import ipywidgets as widgets

fig_w, fig_h = (6, 4)
plt.rcParams.update({'figure.figsize': (fig_w, fig_h)})

%matplotlib inline
```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins.

  TODO:
    * Passing the mean directly, but could estimate from the counts and bins.
  """
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, 0, ymax, colors='r', linestyles='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)

  if ax_args.get('yscale', 'linear') == 'log':
    ymax *= 1.5
  else:
    ymax *= 1.1
    if ymin is None:
      ymin = 0
  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=True, axis='x', tight=True)

def eventplot(event_times, colos='k', ax=None, ax_args=None, **kwargs):
  if ax is None:
    _, ax = plt.subplots()

  p = ax.eventplot(event_times, colors='k', **kwargs)

  if ax_args is None:
    ax_args = {}

  # `eventplot` behaves differently with a single series? just blank the y-axis for now
  if len(p) == 1:
    ax.yaxis.set_visible(False)

  ax.set(**ax_args)
  ax.autoscale(enable=True, axis='x', tight=True)
```

```python
#@title Download Data
r = requests.get('https://osf.io/sy5xt/download')
if r.status_code != 200:
    print('oops')
steinmetz_spikes = np.load(io.BytesIO(r.content), allow_pickle=True)['spike_times']
```

## Optimization and Information

Neurons can only fire so often in a fixed period of time, as the act of emitting a spike consumes energy that is depleted and must eventually be replenished. To communicate effectively for downstream computation, the neuron would need to make good use of its limited spiking capability. This becomes an optimization problem:

What is the optimal way for a neuron to fire in order to maximize its ability to communicate information?

In order to explore this question, we first need to have a quantifiable measure for information. Shannon intoduced the concept of entropy to do just that, and defined it as

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

where $H$ is entropy measured in units of base $b$ and $p(x)$ is the probability of observing the event $x$ from the set of all possible events in $X$. See the Appendix for a more detailed look at how this equation was derived.

The most common base of measuring entropy is $b=2$, so we often talk about *bits* of information, though other bases are used as well e.g. when $b=e$ we call the units *nats*.

First, let's explore how entropy changes between some simple discrete probability distributions. In the rest of this tutorial we will refer to these as probability mass functions (PMF), where $p(x_i)$ equals the $i^{th}$ value in an array, and mass refers to how much of the distribution is contained at that value.

For our first PMF, we will choose one where all of the probability mass is located in the middle of the distribution.

```python
n_points = 50  # points supporting the distribution
bins = np.linspace(0, 1, n_points + 1)

pmf = np.zeros(n_points)
pmf[len(pmf) // 2] = 1.0  # middle point has all the mass
# Note: the middle point of `p` maps to the middle bin of `bins` because of shared indexing

histogram(pmf, bins, ax_args={
  'xlabel': "x",
  'ylabel': "p(x)",
  'xlim': [0, None],
})
```

If we were to draw a sample from this distribution, we know exactly what we would get every time. Distributions where all the mass is concentrated on a single event are known as *deterministic*.

How much entropy is contained in a deterministic distribution?

### Exercise: Computing Entropy

Your first exercise it to implement a method that computes the entropy of a discrete probability mass function. Remember that we are interested in bits, so be sure to use the correct log function. Also recall that $log(0)$ is undefined and will produce a `nan` (Not a Number) value if evaluated.

```python
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  Returns:
    h (number): The entropy of the distribution in `pmf`.

  """
  ###############################################################
  ## TODO for students: compute the entropy of the provided PMF #
  ###############################################################
  raise NotImplementedError("Student excercise: implement the equation for entropy")

# Uncomment once the entropy function is complete
# print(f"{entropy(pmf):.2f} bits")
```

```python
# to_remove solution
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  """
  # remove non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]
  h = -np.sum(pmf * np.log2(pmf))
  # absolute value applied to avoid getting a -0 result
  return np.abs(h)

print(f"{entropy(pmf):.2f} bits")
```

    0.00 bits

We expect zero surprise from a deterministic distribution. If we had done this calculation by hand, it would simply be

$-1\log_2 1 = 0$

Note that changing the location of the peak (i.e. the point and bin on which all the mass rests) doesn't alter the entropy. The entropy is about how predictable a sample is with respect to a distribution. A single peak is deterministic regardless of which point it sits on.

```python
pmf = np.zeros(n_points)
pmf[2] = 1.0  # arbitrary point has all the mass

histogram(pmf, bins, ax_args={
  'title': f'Entropy = {entropy(pmf):.2f} bits',
  'xlabel': "x",
  'ylabel': "p(x)",
})
```

What about a distribution with mass split equally between two points?

```python
pmf = np.zeros(n_points)
pmf[len(pmf) // 3] = 0.5
pmf[2 * len(pmf) // 3] = 0.5

histogram(pmf, bins, ax_args={
  'title': f'Entropy = {entropy(pmf):.2f} bits',
  'xlabel': "x",
  'ylabel': "p(x)",
  'ylim': [0, 1],
})
```

Here, the entropy calculation is

$-(0.5 \log_2 0.5 + 0.5\log_2 0.5)=1$

There is 1 bit of entropy. This means that before we take a random sample, there is 1 bit of uncertainty about which point in the distribution the sample will fall on: it will either be the first peak or the second one.

Likewise, if we make one of the peaks taller (i.e. its point holds more of the probability mass) and the other one shorter, the entropy will decrease because of the increased certainty that the sample will fall on one point and not the other:

$-(0.2 \log_2 0.2 + 0.8\log_2 0.8)\approx 0.72$

Try changing the definition of the number and weighting of peaks, and see how the entropy varies.

If we split the probability mass among even more points, the entropy continues to increase. Let's derive the general form for $N$ points of equal mass, where $p_i=p=1/N$:

\begin{align}
 -\sum_i p_i \log_b p_i&= -\sum_i^N \frac{1}{N} \log_b \frac{1}{N}\\
                       &= -\log_b \frac{1}{N} \\
                       &= \log_b N
\end{align}
$$$$

If we have $N$ discrete points, the _uniform distribution_ (where points have equal mass) is the distribution with the highest entropy: $\log_b N$. This upper bound on entropy is useful when considering binning strategies, as any estimate of entropy over $N$ discrete points (or bins) must be in the interval $[0, \log_b N]$.

```python
pmf = np.ones(n_points) / n_points  # [1/N] * N

histogram(pmf, bins, ax_args={
  'title': f'Entropy = {entropy(pmf):.2f} bits',
  'xlabel': "x",
  'ylabel': "p(x)",
  'ylim': [0, 1],
})
```

Here, there are 50 bins and the entropy of the uniform distribution is $\log_2 50\approx 5.64$. If we construct _any_ discrete distribution $X$ over 50 bins and calculate an entropy of $H_2(X)>\log_2 50$, something must be wrong with our implementation of the discrete entropy computation.

## Information, neurons, and spikes

Recall the discussion of spike times and inter-spike intervals (ISIs) from Tutorial 1. What does the information content (or distributional entropy) of these measures say about our theory of nervous systems?

Let us consider three hypothetical neurons similar to those we explored in the Steinmetz data that all have the same mean ISIs but with different distributions:

1. Deterministic
2. Uniform
3. Exponential

How do we expect their entropies to differ?

```python
n_bins = 50
mean_isi = 0.025
bins = np.linspace(0, 0.25, n_bins + 1)
mean_idx = np.searchsorted(bins, mean_isi)

# 1. all mass concentrated on the ISI mean
pmf_single = np.zeros_like(bins[1:])
pmf_single[mean_idx] = 1.0

# 2. mass uniformly distributed about the ISI mean
pmf_uniform = np.zeros_like(bins[1:])
pmf_uniform[0:2*mean_idx] = 1 / (2 * mean_idx)

# 3. mass exponentially distributed about the ISI mean
pmf_exp = stats.expon.pdf(bins[1:], scale=mean_isi)
pmf_exp /= np.sum(pmf_exp)

fig, (ax1, ax2, ax3) = plt.subplots(ncols=3, figsize=(18, 5))
histogram(pmf_single, bins, ax=ax1, ax_args={
  'title': '(1) Deterministic',
  'xlabel': "Inter-spike interval (s)",
  'ylabel': "Probability mass",
  'ylim': [0, 1.0]
})
histogram(pmf_uniform, bins, ax=ax2, ax_args={
  'title': '(2) Uniform',
  'xlabel': "Inter-spike interval (s)",
  'ylabel': "Probability mass",
  'ylim': [0, 0.25]
})
histogram(pmf_exp, bins, ax=ax3, ax_args={
  'title': '(3) Exponential',
  'xlabel': "Inter-spike interval (s)",
  'ylabel': "Probability mass",
  'ylim': [0, 0.25]
})
```

```python
print(
  f"Deterministic: {entropy(pmf_single):.2f} bits",
  f"Uniform: {entropy(pmf_uniform):.2f} bits",
  f"Exponential: {entropy(pmf_exp):.2f} bits",
  sep="\n",
)
```

    Deterministic: 0.00 bits
    Uniform: 3.32 bits
    Exponential: 3.77 bits

In the previous example we created the PMFs by hand to illustrate idealized scenarios. How would we compute them from data recorded from actual neurons?

One way is to convert the ISI histograms we've previously computed and convert them into discrete probability distributions using the following equation:

\begin{align}
p(i) = \frac{n_i}{\sum\nolimits_{i}n_i}
\end{align}

where $p(i)$ is the probability of an ISI falling within a particular interval $i$ and $n_i$ is the count of how many ISIs were observed in that interval.

```python
#@title Video: Entropy of different distributions
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='hySy-J51vcI', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=hySy-J51vcI

### Exercise: Probabilty Mass Function

Your second exercise is to implement a method that will produce a probability mass function from an array of ISI bin counts.

To verify your solution, we will compute the probability distribution of ISIs from real neural data taken from the Steinmetz dataset.

```python
neuron_idx = 283
isi = np.diff(steinmetz_spikes[neuron_idx])
bins = np.linspace(0, 0.25, n_bins + 1)
counts, _ = np.histogram(isi, bins)
```

```python
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  ###########################################################################
  ## TODO for students: compute the probability mass function from ISI counts
  ###########################################################################
  raise NotImplementedError("Student excercise: compute the PMF from ISI counts")

# Uncomment once you complete the function
# pmf = pmf_from_counts(counts)
# histogram(pmf, bins, ax_args={
#   'title': f"Neuron {neuron_idx}",
#   'xlabel': "Inter-spike interval (s)",
#   'ylabel': "Probability mass",
# })
```

```python
# to_remove solution
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf
pmf = pmf_from_counts(counts)
with plt.xkcd():
  histogram(pmf, bins, ax_args={
    'title': f"Neuron {neuron_idx}",
    'xlabel': "Inter-spike interval (s)",
    'ylabel': "Probability mass",
  })
```

```python
#@title Video: Calculating entropy of a probability distribution
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='cyBu1pEGOh4', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=cyBu1pEGOh4

Now that we have the probability distribution for the actual neuron spiking activity, we can calculate its entropy.

```python
print(f"Entropy for Neuron {neuron_idx}: {entropy(pmf):.2f} bits")
```

    Entropy for Neuron 283: 3.36 bits

### Data Exploration

We can combine the above distribution plot and entropy calculation with an interactive widget to explore how the different neurons in the dataset vary in spiking activity and relative information.

```python
#@title Steinmetz Neuron Information Explorer

def _pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf

def _entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits."""
  # remove non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]
  h = -np.sum(pmf * np.log2(pmf))
  # absolute value applied to avoid getting a -0 result
  return np.abs(h)

@widgets.interact(neuron=widgets.IntSlider(0, min=0, max=(len(steinmetz_spikes)-1)))
def steinmetz_pmf(neuron):
  """ Given a neuron from the Steinmetz data, compute its PMF and entropy """
  isi = np.diff(steinmetz_spikes[neuron])
  bins = np.linspace(0, 0.25, n_bins + 1)
  counts, _ = np.histogram(isi, bins)
  pmf = _pmf_from_counts(counts)

  ymax = max(0.2, np.max(pmf))
  histogram(pmf, bins, ax_args={
    'title': f"Neuron {neuron}: H = {_entropy(pmf):.2f} bits",
    'xlabel': "Inter-spike interval (s)",
    'ylabel': "Probability mass",
    'ylim': [0, ymax]
  })
```

```python
#@title Video: Summary of “why” models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='-eGFd7E_smA', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=-eGFd7E_smA

## Appendix

### The foundations for Entropy

**Shannon** [ref] began with three criteria for a function $H$ defining the entropy of a discrete distribution of probability masses $p_i\in p(X)$ over the points $x_i\in X$:
1. $H$ should be continuous in the $p_i$.
  - That is, $H$ should change smoothly in response to smooth changes to the mass $p_i$ on each point $x_i$.
2. If all the points have equal shares of the probability mass, $p_i=1/N$, $H$ should be a non-decreasing function of $N$.
  - That is, if $X_N$ is the support with $N$ discrete points and $p(x\in X_N)$ assigns constant mass to each point, then $H(X_1) < H(X_2) < H(X_3) < \dots$
3. $H$ should be preserved by (invariant to) the equivalent (de)composition of distributions.
  - For example [from Shannon's paper] if we have a discrete distribution over three points with masses $(\frac{1}{2},\frac{1}{3},\frac{1}{6})$, then their entropy can be represented in terms of a direct choice between the three and calculated $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})$. However, it could also be represented in terms of a series of two choices:
    1. either we sample the point with mass $1/2$ or not (_not_ is the other $1/2$, whose subdivisions are not given in the first choice),
    2. if (with probability $1/2$) we _don't_ sample the first point, we sample one of the two remaining points, masses $1/3$ and $1/6$.

    Thus in this case we require that $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})=H(\frac{1}{2},\frac{1}{2}) + \frac{1}{2}H(\frac{1}{3}, \frac{1}{6})$

There is a unique function which satisfies these 3 requirements: [the function is unique up to scaling by a linear factor]

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

Where the base of the logarithm $b>1$ controls the units of entropy. The two most common cases are $b=2$ for units of _bits_, and $b=e$ for _nats_.

We can view this function as the expectation of the self-information over a distribution:

$$H_b(X) = \mathbb{E}_{x\in X} \left[I_b(x)\right]$$

$$I_b(x)=-\log_b p(x)$$

Self-information is just the negative logarithm of probability, and is a measure of how surprising an event sampled from the distribution would be. Events with $p(x)=1$ are certain to occur, and their self-information is zero [as is the entropy of the distribution they compose] meaning they are totally unsurprising. The smaller the probability of an event, the higher its self-information, and the more surprising the event would be to observe.


```python
from sympy import symbols,sqrt
from sympy.physics.mechanics import Point,ReferenceFrame,dynamicsymbols,Particle
from sympy import pprint
from sympy.physics.mechanics import inertia
from sympy import Matrix
from sympy import sin,cos
```

```python
## 4. Propiedades de masa e inercia
```

```python
## 4.1 Centro de masa
```

```python
# Exprese la ubicación del centro de masa de la parte móvil del robot scara
# en función de q1,q2,q3.

# Defina variables de movimiento y parámetros del modelo para el
q1,q2,q3=dynamicsymbols('q1,q2,q3')
l1,l2,l3=symbols('l1,l2,l3')

# Defina los marcos y puntos de interes en el robot
a=ReferenceFrame('A')
o=Point('O')
b=a.orientnew('B','Axis',(q1,a.z))
p=o.locatenew('P',l1*a.z)
c=b.orientnew('C','Axis',(q2,b.z))
q=p.locatenew('Q',l2*b.x)
r=p.locatenew('R',l3*c.x)
s=r.locatenew('Q',-q3*c.z)

# defina variables para la masa
m1,m2,m3=symbols('m1,m2,m3') #masa de brazo movil1, movil2, y ventosa (m3).
com1=l2*b.x/2
com2=q.pos_from(p)+l3*c.x/2
com3=s.pos_from(p)
com=((m1*com1+m2*com2+m3*com3)/(m1+m2+m3)).simplify()

# Puede expresar com explicitamente en qi al calcular respecto a vectores unitarios en A
com.express(a)
```

$\displaystyle (\frac{l_{2} \left(m_{1} + 2 m_{2}\right) \cos{\left(\operatorname{q_{1}}{\left(t \right)} \right)}}{2 \left(m_{1} + m_{2} + m_{3}\right)} + \frac{l_{3} \left(m_{2} + 2 m_{3}\right) \left(- \sin{\left(\operatorname{q_{1}}{\left(t \right)} \right)} \sin{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + \cos{\left(\operatorname{q_{1}}{\left(t \right)} \right)} \cos{\left(\operatorname{q_{2}}{\left(t \right)} \right)}\right)}{2 \left(m_{1} + m_{2} + m_{3}\right)})\mathbf{\hat{a}_x} + (\frac{l_{2} \left(m_{1} + 2 m_{2}\right) \sin{\left(\operatorname{q_{1}}{\left(t \right)} \right)}}{2 \left(m_{1} + m_{2} + m_{3}\right)} + \frac{l_{3} \left(m_{2} + 2 m_{3}\right) \left(\sin{\left(\operatorname{q_{1}}{\left(t \right)} \right)} \cos{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + \sin{\left(\operatorname{q_{2}}{\left(t \right)} \right)} \cos{\left(\operatorname{q_{1}}{\left(t \right)} \right)}\right)}{2 \left(m_{1} + m_{2} + m_{3}\right)})\mathbf{\hat{a}_y} -  \frac{m_{3} \operatorname{q_{3}}{\left(t \right)}}{m_{1} + m_{2} + m_{3}}\mathbf{\hat{a}_z}$

```python
com
```

$\displaystyle \frac{l_{2} \left(m_{1} + 2 m_{2}\right)}{2 \left(m_{1} + m_{2} + m_{3}\right)}\mathbf{\hat{b}_x} + \frac{l_{3} \left(m_{2} + 2 m_{3}\right)}{2 \left(m_{1} + m_{2} + m_{3}\right)}\mathbf{\hat{c}_x} -  \frac{m_{3} \operatorname{q_{3}}{\left(t \right)}}{m_{1} + m_{2} + m_{3}}\mathbf{\hat{c}_z}$

## 4.2 Propiedades de inercia

```python
# Ejemplo helice
# Considere una hélice con propiedades de inercia definidas respecto al centro de masa en
# el sistema de coordenadas definido por B.
b=ReferenceFrame('B')
# Defina las variables para los momentos de inercia (escalares)
Ix,Iy,Iz=symbols('Ix,Iy,Iz')

#Defina los vectores de inercia I1, I2, I3 (los productos de inercia son nulos en este ejemplo)
I1=Ix*b.x
I2=Iy*b.y
I3=Iz*b.z

#Defina los vectores unitarios
na=sqrt(2)*(b.y+b.z)/2
nb=sqrt(2)*(b.x+b.z)/2

#Calcule los vectores de inercia respecto a na y nb
Ia=na.dot(b.x)*I1+na.dot(b.y)*I2+na.dot(b.z)*I3
Ib=nb.dot(b.x)*I1+nb.dot(b.y)*I2+nb.dot(b.z)*I3

#Calcule los escalares de inercia
Iaa=Ia.dot(na)
Ibb=Ib.dot(nb)
Iab=Ia.dot(nb)
pprint(Iaa)
pprint(Ibb)
pprint(Iab)
```

    Iy   Iz
    ── + ──
    2    2
    Ix   Iz
    ── + ──
    2    2
    Iz
    ──
    2

## 4.2.3 Matrices y diádicas de inercia

```python
# Puede utilizar la clase inertia para definir diádicas de inercia
n=ReferenceFrame('N')
O=Point('O')
i11,i12,i13,i22,i23,i33=symbols('i11,i12,i13,i22,i23,i33')
I=inertia(n,i11,i22,i33,i12,i13,i23)
I
```

$\displaystyle i_{11}\mathbf{\hat{n}_x}\otimes \mathbf{\hat{n}_x} + i_{12}\mathbf{\hat{n}_x}\otimes \mathbf{\hat{n}_y} + i_{23}\mathbf{\hat{n}_x}\otimes \mathbf{\hat{n}_z} + i_{12}\mathbf{\hat{n}_y}\otimes \mathbf{\hat{n}_x} + i_{22}\mathbf{\hat{n}_y}\otimes \mathbf{\hat{n}_y} + i_{13}\mathbf{\hat{n}_y}\otimes \mathbf{\hat{n}_z} + i_{23}\mathbf{\hat{n}_z}\otimes \mathbf{\hat{n}_x} + i_{13}\mathbf{\hat{n}_z}\otimes \mathbf{\hat{n}_y} + i_{33}\mathbf{\hat{n}_z}\otimes \mathbf{\hat{n}_z}$

```python
# Puede usar la representación matricial con el método to_matrix
I.to_matrix(n)
```

$\displaystyle \left[\begin{matrix}i_{11} & i_{12} & i_{23}\\i_{12} & i_{22} & i_{13}\\i_{23} & i_{13} & i_{33}\end{matrix}\right]$

```python
r11,r12,r13,r22,r23,r33=symbols('r11,r12,r13,r22,r23,r33')
b=n.orientnew('B','DCM',Matrix([[r11,-r12,-r13],[r12,r22,-r23],[r13,r23,r33]]))
# Puede expresar la matriz respecto a otro marco de referencia
I.to_matrix(b)
```

$\displaystyle \left[\begin{matrix}r_{11} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) + r_{12} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{13} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right) & - r_{12} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) + r_{22} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{23} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right) & - r_{13} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) - r_{23} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{33} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right)\\r_{11} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) + r_{12} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{13} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right) & - r_{12} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) + r_{22} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{23} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right) & - r_{13} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) - r_{23} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{33} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right)\\r_{11} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) + r_{12} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{13} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right) & - r_{12} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) + r_{22} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{23} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right) & - r_{13} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) - r_{23} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{33} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right)\end{matrix}\right]$

```python
# Lo que es equivalente a la operación de cambio de base (eq. 4.16)
b.dcm(n)*I.to_matrix(n)*b.dcm(n).transpose()
```

$\displaystyle \left[\begin{matrix}r_{11} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) + r_{12} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{13} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right) & - r_{12} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) + r_{22} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{23} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right) & - r_{13} \left(i_{11} r_{11} + i_{12} r_{12} + i_{23} r_{13}\right) - r_{23} \left(i_{12} r_{11} + i_{13} r_{13} + i_{22} r_{12}\right) + r_{33} \left(i_{13} r_{12} + i_{23} r_{11} + i_{33} r_{13}\right)\\r_{11} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) + r_{12} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{13} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right) & - r_{12} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) + r_{22} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{23} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right) & - r_{13} \left(- i_{11} r_{12} + i_{12} r_{22} + i_{23} r_{23}\right) - r_{23} \left(- i_{12} r_{12} + i_{13} r_{23} + i_{22} r_{22}\right) + r_{33} \left(i_{13} r_{22} - i_{23} r_{12} + i_{33} r_{23}\right)\\r_{11} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) + r_{12} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{13} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right) & - r_{12} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) + r_{22} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{23} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right) & - r_{13} \left(- i_{11} r_{13} - i_{12} r_{23} + i_{23} r_{33}\right) - r_{23} \left(- i_{12} r_{13} + i_{13} r_{33} - i_{22} r_{23}\right) + r_{33} \left(- i_{13} r_{23} - i_{23} r_{13} + i_{33} r_{33}\right)\end{matrix}\right]$

```python
# Observe esto con el ejemplo de la hélice
b=ReferenceFrame('B')
O=Point('O')
ix,iy,iz=symbols('ix,iy,iz')
q=symbols('q')
I=inertia(b,ix,iy,iz)
a=b.orientnew('A','DCM',Matrix([[cos(q),sin(q),0],[-sin(q),cos(q),0],[0,0,1]]))
# b.dcm(a) #Observe la matriz de rotación entre A y B (recuerde que en sympy dcm es definido por filas)
I.to_matrix(a)
```

$\displaystyle \left[\begin{matrix}ix \cos^{2}{\left(q \right)} + iy \sin^{2}{\left(q \right)} & ix \sin{\left(q \right)} \cos{\left(q \right)} - iy \sin{\left(q \right)} \cos{\left(q \right)} & 0\\ix \sin{\left(q \right)} \cos{\left(q \right)} - iy \sin{\left(q \right)} \cos{\left(q \right)} & ix \sin^{2}{\left(q \right)} + iy \cos^{2}{\left(q \right)} & 0\\0 & 0 & iz\end{matrix}\right]$

```python
# Alternativamente puede utilizar algebra de diádicas para expresar la inercia en A
I.express(a)
```

$\displaystyle (ix \cos^{2}{\left(q \right)} + iy \sin^{2}{\left(q \right)})\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_x} + (ix \sin{\left(q \right)} \cos{\left(q \right)} - iy \sin{\left(q \right)} \cos{\left(q \right)})\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_y} + (ix \sin{\left(q \right)} \cos{\left(q \right)} - iy \sin{\left(q \right)} \cos{\left(q \right)})\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_x} + (ix \sin^{2}{\left(q \right)} + iy \cos^{2}{\left(q \right)})\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_y} + iz\mathbf{\hat{a}_z}\otimes \mathbf{\hat{a}_z}$

```python
## 4.2.4 Teoremas de ejes paralelos
```

```python
# Para el ejemplo del quadricóptero calcule la inercia respecto al centro de masa de la base A
a=ReferenceFrame('A')
O=Point('O')
# Defina variables de movimiento para cada hélice
q1,q2,q3,q4=dynamicsymbols('q1,q2,q3,q4')
# defina simbolos para los escalares
mb,jx,jy,jz,ix,iy,iz,r=symbols('mb,jx,jy,jz,ix,iy,iz,r')

# Defina el marco de referencia de cada hélice
qi=[q1,q2,q3,q4]
Bnames=['B{}'.format(i) for i in range(0,4)]
blist=[a.orientnew(Bnames[i],'Axis',(qi[i],a.z)) for i in range(0,4)]

# Defina la ubicacion de cada hélice en A
# utilice la clase Particle para definir una masa puntual en cada Bi
pnames=['P{}'.format(i) for i in range(0,4)]
loclist=[a.x*r,a.y*r,-a.x*r,-a.y*r]
plist=[O.locatenew(pnames[i],loclist[i]) for i in range(0,4)]
particleList=[Particle(pnames[i],plist[i],mb) for i in range(0,4)]

# Calcule la diádica de inercia de la base del cuadricóptero
iaa=inertia(a,jx,jy,jz)
# Calcule la diádica de inercia de la hélice en B
ibbList=[inertia(bi,ix,iy,iz) for bi in blist]
# Calcule la diádica de inercia de la masa puntual de cada hélice en A
ibaList=[pi.parallel_axis(O,a) for pi in particleList]
```

```python
# Observe cada término Ibi/A
ibaList[0]
#ibaList[1]
#ibaList[2]
#ibaList[3]
```

$\displaystyle mb r^{2}\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_y} + mb r^{2}\mathbf{\hat{a}_z}\otimes \mathbf{\hat{a}_z}$

```python
# Para simplificar el código, defina una función para sumar todas las diádicas en una lista
def sumlist(x):
    total=x[0]
    for xi in x[1:]:
        total=total+xi
    return total

Icuad=iaa+sumlist(ibbList)+sumlist(ibaList)
Icuad.express(a).simplify()
```

$\displaystyle (ix \cos^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + ix \cos^{2}{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + ix \cos^{2}{\left(\operatorname{q_{3}}{\left(t \right)} \right)} + ix \cos^{2}{\left(\operatorname{q_{4}}{\left(t \right)} \right)} + iy \sin^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + iy \sin^{2}{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + iy \sin^{2}{\left(\operatorname{q_{3}}{\left(t \right)} \right)} + iy \sin^{2}{\left(\operatorname{q_{4}}{\left(t \right)} \right)} + jx + 2 mb r^{2})\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_x} + (ix \sin^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + ix \sin^{2}{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + ix \sin^{2}{\left(\operatorname{q_{3}}{\left(t \right)} \right)} + ix \sin^{2}{\left(\operatorname{q_{4}}{\left(t \right)} \right)} + iy \cos^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + iy \cos^{2}{\left(\operatorname{q_{2}}{\left(t \right)} \right)} + iy \cos^{2}{\left(\operatorname{q_{3}}{\left(t \right)} \right)} + iy \cos^{2}{\left(\operatorname{q_{4}}{\left(t \right)} \right)} + jy + 2 mb r^{2})\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_y} + (4 iz + jz + 4 mb r^{2})\mathbf{\hat{a}_z}\otimes \mathbf{\hat{a}_z} + (\frac{ix \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{2}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{3}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{4}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{2}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{3}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{4}}{\left(t \right)} \right)}}{2})\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_y} + (\frac{ix \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{2}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{3}}{\left(t \right)} \right)}}{2} + \frac{ix \sin{\left(2 \operatorname{q_{4}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{2}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{3}}{\left(t \right)} \right)}}{2} - \frac{iy \sin{\left(2 \operatorname{q_{4}}{\left(t \right)} \right)}}{2})\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_x}$

```python
# Suponiendo que las hélices se mueven sincronizadamente (q1=q2=q3=q4)
Icuad.express(a).subs({q2:q1,q3:q1,q4:q1}).simplify()
```

$\displaystyle (4 ix \cos^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + 4 iy \sin^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + jx + 2 mb r^{2})\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_x} + (4 ix \sin^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + 4 iy \cos^{2}{\left(\operatorname{q_{1}}{\left(t \right)} \right)} + jy + 2 mb r^{2})\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_y} + (4 iz + jz + 4 mb r^{2})\mathbf{\hat{a}_z}\otimes \mathbf{\hat{a}_z} + 2 \left(ix - iy\right) \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}\mathbf{\hat{a}_x}\otimes \mathbf{\hat{a}_y} + 2 \left(ix - iy\right) \sin{\left(2 \operatorname{q_{1}}{\left(t \right)} \right)}\mathbf{\hat{a}_y}\otimes \mathbf{\hat{a}_x}$


```python
import numpy as np
import pandas as pd
from sympy import primerange
from sympy.utilities.iterables import multiset_permutations
from multiprocessing import Pool, cpu_count
import numba
from sympy import isprime
from math import sqrt
```

```python
cities = pd.read_csv('../data/raw/cities.csv', index_col=['CityId'], dtype={'X': np.float32, 'Y': np.float32})
```

```python
XY = np.stack((cities.X.astype(np.float32), cities.Y.astype(np.float32)), axis=1)
is_not_prime = np.array([not isprime(city_id) for city_id in cities.index], dtype=np.int32)

@numba.jit('f8(i8[:])', nopython=True, parallel=False)
def pure_score(path):
    '''Pure path score without penalties.'''
    dist = 0.0
    for i in numba.prange(path.shape[0] - 1):
        a, b = XY[path[i]], XY[path[i+1]]
        dx, dy = a[0] - b[0], a[1] - b[1]
        dist += sqrt(dx * dx + dy * dy)
    return dist

@numba.jit('f8(i4, i8[:])', nopython=True, parallel=False)
def chunk_score(start_offset, chunk):
    '''Score of path's chunk that starts at index 'start_offset'.'''
    dist = 0.0
    penalty = 0.0
    penalty_modulo = 9 - start_offset % 10
    for i in numba.prange(chunk.shape[0] - 1):
        id_a = chunk[i]
        a, b = XY[id_a], XY[chunk[i+1]]
        dx, dy = a[0] - b[0], a[1] - b[1]
        d = sqrt(dx * dx + dy * dy)
        dist += d
        if i % 10 == penalty_modulo and is_not_prime[id_a]:
            penalty += d
    return dist + 0.1 * penalty

@numba.jit('f8(i8[:])', nopython=True, parallel=False)
def path_score(path):
    return chunk_score(0, path)
```

```python
tour = pd.read_csv('submission.151557248.csv')['Path'].values
path_score(tour)
```

    1515572.4502716302

```python
def riffle2(batch, i, permutations):
    best = batch
    best_score = chunk_score(i, best)
    for p in permutations:
        perm = batch[p]
        perm_score = chunk_score(i, perm)
        if perm_score < best_score:
#             print(str(best_score) +" ---> "+str(perm_score))
            best = perm
            best_score = perm_score
    if (best != batch).any():
        return best
    else:
        return None
```

```python
def riffle(split_tour1):
    order, batch, step, nperms = split_tour1
    n = nperms + 2
    permutations = np.array([np.array([0]+i+[nperms+1]) for i in multiset_permutations(np.arange(1, nperms+1))])
    for i in range(0, batch.size-n):
        if (i + n >= batch.size): break
        r = riffle2(batch[i:i+n], order * step + i, permutations)
        if r is not None:
#             print(r,i)
#             print(batch[i:i+n])
#             print(r)
            score_old = chunk_score(order * step, batch)
            batch_new = np.concatenate((batch[:i], r, batch[i+n:]))
            score_new = chunk_score(order * step, batch_new)
            if (score_new < score_old):
                batch = batch_new
                print(str(score_old) + " -> " + str(score_new))
    return [order, batch]
```

```python
def multi_riffle(tour, ncores, step, nperms):
    p = Pool(ncores)
    t = [tour[i*step:(i+1)*step] for i in range(0,(tour.size//step)+1)]
    t = [(i,t,step,nperms) for i, t in enumerate(t)]
    ret = p.map(riffle, t)

    ret_d = {}
    for i in range(len(ret)):
        ret_d[ret[i][0]] = ret[i][1]
    ret2 = []
    for i in range(len(ret_d)):
        ret2 += list(ret_d[i])
    return ret2
```

```python
# %%time
step = 100
nperms=3
ncores=2
tour_new = multi_riffle(tour, ncores, step, nperms)
```

    CPU times: user 418 ms, sys: 103 ms, total: 521 ms
    Wall time: 22 s

```python
pd.DataFrame({'Path': list(tour_new)}).to_csv('submission.csv', index=False)
```

```python
path_score(np.array(tour_new, dtype=np.int64))
```

    1515572.4502716302

```python

```


# Lab 4 Intro: Simulating Molecules using VQE

## Alexandra Semposki

### Due date: 15 April 2022

## Contents
1. [Introduction](#introduction)
2. [The Variational Method of Quantum Mechanics](#varmethod)
    1. [Mathematical Background](#backgroundmath)
    2. [Bounding the Ground State](#groundstate)
3. [The Variational Quantum Eigensolver](#vqe)
    1. [Variational Forms](#varforms)
    2. [Simple Variational Forms](#simplevarform)
    3. [Parameter Optimization](#optimization)
    4. [Example with a Single Qubit Variational Form](#example)
    5. [Structure of Common Variational Forms](#commonvarforms)

# Introduction
In many applications it is important to find the minimum eigenvalue of a matrix. For example, in chemistry, the minimum eigenvalue of a Hermitian matrix characterizing the molecule is the ground state energy of that system. In the future, the quantum phase estimation algorithm may be used to find the minimum eigenvalue. However, its implementation on useful problems requires circuit depths exceeding the limits of hardware available in the NISQ era. Thus, in 2014, Peruzzo *et al.* proposed VQE to estimate the ground state energy of a molecule using much shallower circuits [1].

Formally stated, given a Hermitian matrix $H$ with an unknown minimum eigenvalue $\lambda_{min}$, associated with the eigenstate $|\psi_{min}\rangle$, VQE provides an estimate $\lambda_{\theta}$ bounding $\lambda_{min}$:

\begin{align*}
    \lambda_{min} \le \lambda_{\theta} \equiv \langle \psi(\theta) |H|\psi(\theta) \rangle
\end{align*}

where $|\psi(\theta)\rangle$ is the eigenstate associated with $\lambda_{\theta}$. By applying a parameterized circuit, represented by $U(\theta)$, to some arbitrary starting state $|\psi\rangle$, the algorithm obtains an estimate $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle$ on $|\psi_{min}\rangle$. The estimate is iteratively optimized by a classical controller changing the parameter $\theta$ minimizing the expectation value of $\langle \psi(\theta) |H|\psi(\theta) \rangle$.

# The Variational Method of Quantum Mechanics
## Mathematical Background

VQE is an application of the variational method of quantum mechanics. To better understand the variational method, some preliminary mathematical background is provided. An eigenvector, $|\psi_i\rangle$, of a matrix $A$ is invariant under transformation by $A$ up to a scalar multiplicative constant (the eigenvalue $\lambda_i$). That is,

\begin{align*}
    A |\psi_i\rangle = \lambda_i |\psi_i\rangle
\end{align*}

Furthermore, a matrix $H$ is Hermitian when it is equal to its own conjugate transpose.

\begin{align*}
    H = H^{\dagger}
\end{align*}

The spectral theorem states that the eigenvalues of a Hermitian matrix must be real. Thus, any eigenvalue of $H$ has the property that $\lambda_i = \lambda_i^*$. As any measurable quantity must be real, Hermitian matrices are suitable for describing the Hamiltonians of quantum systems. Moreover, $H$ may be expressed as

\begin{align*}
    H = \sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |
\end{align*}

where each $\lambda_i$ is the eigenvalue corresponding to the eigenvector $|\psi_i\rangle$. Furthermore, the expectation value of the observable $H$ on an arbitrary quantum state $|\psi\rangle$ is given by

\begin{align}
    \langle H \rangle_{\psi} &\equiv \langle \psi | H | \psi \rangle
\end{align}

Substituting $H$ with its representation as a weighted sum of its eigenvectors,

\begin{align}
    \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle &= \langle \psi | \left(\sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |\right) |\psi\rangle\\
    &= \sum_{i = 1}^{N} \lambda_i \langle \psi | \psi_i\rangle \langle \psi_i | \psi\rangle \\
    &= \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The last equation demonstrates that the expectation value of an observable on any state can be expressed as a linear combination using the eigenvalues associated with $H$ as the weights. Moreover, each of the weights in the linear combination is greater than or equal to 0, as $| \langle \psi_i | \psi\rangle |^2 \ge 0$ and so it is clear that

\begin{align}
    \lambda_{min} \le \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle = \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The above equation is known as the **variational method** (in some texts it is also known as the variational principle) [2]. It is important to note that this implies that the expectation value of any wave function will always be at least the minimum eigenvalue associated with $H$. Moreover, the expectation value of the eigenstate $|\psi_{min}\rangle$ is given by $\langle \psi_{min}|H|\psi_{min}\rangle = \langle \psi_{min}|\lambda_{min}|\psi_{min}\rangle = \lambda_{min}$. Thus, as expected, $\langle H \rangle_{\psi_{min}}=\lambda_{min}$.

### Bounding the Ground State
When the Hamiltonian of a system is described by the Hermitian matrix $H$ the ground state energy of that system, $E_{gs}$, is the smallest eigenvalue associated with $H$. By arbitrarily selecting a wave function $|\psi \rangle$ (called an *ansatz*) as an initial guess approximating $|\psi_{min}\rangle$, calculating its expectation value, $\langle H \rangle_{\psi}$, and iteratively updating the wave function, arbitrarily tight bounds on the ground state energy of a Hamiltonian may be obtained.

# The Variational Quantum Eigensolver
## Variational Forms
A systematic approach to varying the ansatz is required to implement the variational method on a quantum computer. VQE does so through the use of a parameterized circuit with a fixed form. Such a circuit is often called a *variational form*, and its action may be represented by the linear transformation $U(\theta)$. A variational form is applied to a starting state $|\psi\rangle$ (such as the vacuum state $|0\rangle$, or the Hartree Fock state) and generates an output state $U(\theta)|\psi\rangle\equiv |\psi(\theta)\rangle$. Iterative optimization over $|\psi(\theta)\rangle$ aims to yield an expectation value $\langle \psi(\theta)|H|\psi(\theta)\rangle \approx E_{gs} \equiv \lambda_{min}$. Ideally, $|\psi(\theta)\rangle$ will be close to $|\psi_{min}\rangle$ (where 'closeness' is characterized by either state fidelity, or Manhattan distance) although in practice, useful bounds on $E_{gs}$ can be obtained even if this is not the case.

Moreover, a fixed variational form with a polynomial number of parameters can only generate transformations to a polynomially sized subspace of all the states in an exponentially sized Hilbert space. Consequently, various variational forms exist. Some, such as Ry and RyRz are heuristically designed, without consideration of the target domain. Others, such as UCCSD, utilize domain specific knowledge to generate close approximations based on the problem's structure. The structure of common variational forms is discussed in greater depth later in this document.

## Simple Variational Forms
When constructing a variational form we must balance two opposing goals. Ideally, our $n$ qubit variational form would be able to generate any possible state $|\psi\rangle$ where $|\psi\rangle \in \mathbb{C}^N$ and $N=2^n$. However, we would like the variational form to use as few parameters as possible. Here, we aim to give intuition for the construction of variational forms satisfying our first goal, while disregarding the second goal for the sake of simplicity.

Consider the case where $n=1$. The U3 gate takes three parameters, $\theta, \phi$ and $\lambda$, and represents the following transformation:

$$
\begin{align}
    U3(\theta, \phi, \lambda) = \begin{pmatrix}\cos(\frac{\theta}{2}) & -e^{i\lambda}\sin(\frac{\theta}{2}) \\ e^{i\phi}\sin(\frac{\theta}{2}) & e^{i\lambda + i\phi}\cos(\frac{\theta}{2}) \end{pmatrix}
\end{align}
$$

Up to a global phase, any possible single qubit transformation may be implemented by appropriately setting these parameters. Consequently, for the single qubit case, a variational form capable of generating any possible state is given by the circuit:

Moreover, this universal 'variational form' only has 3 parameters and thus can be efficiently optimized. It is worth emphasizing that the ability to generate an arbitrary state ensures that during the optimization process, the variational form does not limit the set of attainable states over which the expectation value of $H$ can be taken. Ideally, this ensures that the minimum expectation value is limited only by the capabilities of the classical optimizer.

A less trivial universal variational form may be derived for the 2 qubit case, where two body interactions, and thus entanglement, must be considered to achieve universality. Based on the work presented by *Shende et al.* [3] the following is an example of a universal parameterized 2 qubit circuit:

Allow the transformation performed by the above circuit to be represented by $U(\theta)$. When optimized variationally, the expectation value of $H$ is minimized when $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle \approx |\psi_{min}\rangle$. By formulation, $U(\theta)$ may produce a transformation to any possible state, and so this variational form may obtain an arbitrarily tight bound on two qubit ground state energies, only limited by the capabilities of the classical optimizer.

## Parameter Optimization
Once an efficiently parameterized variational form has been selected, in accordance with the variational method, its parameters must be optimized to minimize the expectation value of the target Hamiltonian. The parameter optimization process has various challenges. For example, quantum hardware has various types of noise and so objective function evaluation (energy calculation) may not necessarily reflect the true objective function. Additionally, some optimizers perform a number of objective function evaluations dependent on cardinality of the parameter set. An appropriate optimizer should be selected by considering the requirements of an application.

A popular optimization strategy is gradient descent where each parameter is updated in the direction yielding the largest local change in energy. Consequently, the number of evaluations performed depends on the number of optimization parameters present. This allows the algorithm to quickly find a local optimum in the search space. However, this optimization strategy often gets stuck at poor local optima, and is relatively expensive in terms of the number of circuit evaluations performed. While an intuitive optimization strategy, it is not recommended for use in VQE.

An appropriate optimizer for optimizing a noisy objective function is the ***Simultaneous Perturbation Stochastic Approximation*** optimizer (SPSA). SPSA approximates the gradient of the objective function with only two measurements. It does so by concurrently perturbing all of the parameters in a random fashion, in contrast to gradient descent where each parameter is perturbed independently. When utilizing VQE in either a noisy simulator or on real hardware, SPSA is recommended as the classical optimizer.

When noise is not present in the cost function evaluation (such as when using VQE with a statevector simulator), a wide variety of classical optimizers may be useful. Two such optimizers supported by Qiskit are the ***Sequential Least Squares Programming*** optimizer (SLSQP) and the ***Constrained Optimization by Linear Approximation*** optimizer (COBYLA). It is worth noting that COBYLA only performs one objective function evaluation per optimization iteration (and thus the number of evaluations is independent of the parameter set's cardinality). Therefore, if the objective function is noise-free and minimizing the number of performed evaluations is desirable, it is recommended to try COBYLA.

## Example with a Single Qubit Variational Form

We will now use the simple single qubit variational form to solve a problem similar to ground state energy estimation. Specifically, we are given a random probability vector $\vec{x}$ and wish to determine a possible parameterization for our single qubit variational form such that it outputs a probability distribution that is close to $\vec{x}$ (where closeness is defined in terms of the Manhattan distance between the two probability vectors).

We first create the random probability vector in python:

```python
import numpy as np
np.random.seed(999999)
target_distr = np.random.rand(2)
# We now convert the random vector into a valid probability vector
target_distr /= sum(target_distr)
print("Target Probability Vector: ", target_distr)
```

    Target Probability Vector:  [0.51357006 0.48642994]

We subsequently create a function that takes the parameters of our single U3 variational form as arguments and returns the corresponding quantum circuit:

```python
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister
def get_var_form(params):
    qr = QuantumRegister(1, name="q")
    cr = ClassicalRegister(1, name='c')
    qc = QuantumCircuit(qr, cr)
    qc.u(params[0], params[1], params[2], qr[0])
    qc.measure(qr, cr[0])
    return qc
```

Now we specify the objective function which takes as input a list of the variational form's parameters, and returns the cost associated with those parameters:

```python
from qiskit import Aer, transpile, assemble
backend = Aer.get_backend("qasm_simulator")
NUM_SHOTS = 10000

def get_probability_distribution(counts):
    output_distr = [v / NUM_SHOTS for v in counts.values()]
    if len(output_distr) == 1:
        output_distr.append(1 - output_distr[0])
    return output_distr

def objective_function(params):
    # Obtain a quantum circuit instance from the paramters
    qc = get_var_form(params)
    # Execute the quantum circuit to obtain the probability distribution associated with the current parameters
    t_qc = transpile(qc, backend)
    qobj = assemble(t_qc, shots=NUM_SHOTS)
    result = backend.run(qobj).result()
    # Obtain the counts for each measured state, and convert those counts into a probability vector
    output_distr = get_probability_distribution(result.get_counts(qc))
    # Calculate the cost as the distance between the output distribution and the target distribution
    cost = sum([np.abs(output_distr[i] - target_distr[i]) for i in range(2)])
    return cost
```

Finally, we create an instance of the COBYLA optimizer which will be the method used to optimize over the parameters $\vec{\theta}$. Note that the output varies from run to run. Moreover, while close, the obtained distribution might not be exactly the same as the target distribution, however, increasing the number of shots taken will increase the accuracy of the output.

```python
from qiskit.algorithms.optimizers import COBYLA
# Initialize the COBYLA optimizer
optimizer = COBYLA(maxiter=500, tol=0.0001)

# Create the initial parameters (noting that our single qubit variational form has 3 parameters)
params = np.random.rand(3)
ret = optimizer.optimize(num_vars=3, objective_function=objective_function, initial_point=params)

# Obtain the output distribution using the final parameters
qc = get_var_form(ret[0])
t_qc = transpile(qc, backend)
qobj = assemble(t_qc, shots=NUM_SHOTS)
counts = backend.run(qobj).result().get_counts(qc)
output_distr = get_probability_distribution(counts)

print("Target Distribution:", target_distr)
print("Obtained Distribution:", output_distr)
print("Output Error (Manhattan Distance):", ret[1])
print("Parameters Found:", ret[0])
```

    Target Distribution: [0.51357006 0.48642994]
    Obtained Distribution: [0.5211, 0.4789]
    Output Error (Manhattan Distance): 0.014459881261160878
    Parameters Found: [1.61344107 1.21594491 0.69965842]

## Structure of Common Variational Forms
As already discussed, it is not possible for a polynomially parameterized variational form to generate a transformation to any state. Variational forms can be grouped into two categories, depending on how they deal with this limitation. The first category of variational forms use domain or application specific knowledge to limit the set of possible output states. The second approach uses a heuristic circuit without prior domain or application specific knowledge.

The first category of variational forms exploit characteristics of the problem domain to restrict the set of transformations that may be required. For example, when calculating the ground state energy of a molecule, the number of particles in the system is known *a priori*. Therefore, if a starting state with the correct number of particles is used, by limiting the variational form to only producing particle preserving transformations, the number of parameters required to span the new transformation subspace can be greatly reduced. Another example illustrating the exploitation of domain-specific knowledge follows from considering the set of circuits realizable on real quantum hardware. Extant quantum computers, such as those based on super conducting qubits, have limited qubit connectivity. That is, it is not possible to implement 2-qubit gates on arbitrary qubit pairs (without inserting swap gates). Thus, variational forms have been constructed for specific quantum computer architectures where the circuits are specifically tuned to maximally exploit the natively available connectivity and gates of a given quantum device. Such a variational form was used in 2017 to successfully implement VQE for the estimation of the ground state energies of molecules as large as BeH$_2$ on an IBM quantum computer [4].

In the second approach, gates are layered such that good approximations on a wide range of states may be obtained. Qiskit supports three such variational forms: **RyRz**, **Ry** and **SwapRz**. All of these variational forms accept multiple user-specified configurations. Three essential configurations are the number of qubits in the system, the depth setting, and the entanglement setting. A single layer of a variational form specifies a certain pattern of single qubit rotations and CX gates. The depth setting says how many times the variational form should repeat this pattern. By increasing the depth setting, at the cost of increasing the number of parameters that must be optimized, the set of states the variational form can generate increases. Finally, the entanglement setting selects the configuration, and implicitly the number, of CX gates. For example, when the entanglement setting is linear, CX gates are applied to adjacent qubit pairs in order (and thus $n-1$ CX gates are added per layer). When the entanglement setting is full, a CX gate is applied to each qubit pair in each layer. The circuits for RyRz corresponding to `entanglement="full"` and `entanglement="linear"` can be seen by executing the following code snippet:

```python
from qiskit.circuit.library import EfficientSU2
entanglements = ["linear", "full"]
for entanglement in entanglements:
    form = EfficientSU2(num_qubits=4, entanglement=entanglement)
    if entanglement == "linear":
        print("=============Linear Entanglement:=============")
    else:
        print("=============Full Entanglement:=============")
    # We initialize all parameters to 0 for this demonstration
    display(form.decompose().draw('mpl',fold=100))
    print()
```

Assume the depth setting is set to $d$. Then, RyRz has $2n\times (d+1)$ parameters and Ry (not shown) has $n\times (d+1)$ parameters.

```python

```

Lab-4 Variational Quantum Eigensolver/Lab-4_Intro-VariationalQuantumEigensolver.ipynb


# System Properties

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Classes of Systems

The spectral and temporal characteristics of a linear-time invariant (LTI) system are given by its impulse response $h(t)$ and transfer function $H(s) = \mathcal{L} \{ h(t) \}$, respectively. However it is useful to introduce some classes of LTI systems with specific higher-level properties. These are for instance useful for the design of systems with specific desired properties.

### Real-Valued System

A real-valued system is a system whose output signal $y(t) = \mathcal{H} \{ x(t) \}$ is real-valued $y(t) \in \mathbb{R}$ for a real-valued input signal $x(t) \in \mathbb{R}$. Since the output signal $y(t)$ of an LTI system is given by convolving the input signal $x(t)$ with its impulse response $h(t)$, the impulse response $h(t)$ of a real-valued system has to be real-valued

\begin{equation}
h(t) \in \mathbb{R}
\end{equation}

If existing, the transfer function $H(s) = \mathcal{L} \{ h(t) \}$ shows conjugate complex symmetry

\begin{equation}
H(s) = H^*(s^*)
\end{equation}

due to the [symmetry of the Laplace transform for real-valued signals](../laplace_transform/properties.ipynb#Symmetry-for-Real-Valued-Signals). It follows, that the poles and zeros of a rational transfer function $H(s)$ are either real-valued or complex conjugate pairs.

If existing, the transfer function $H(j \omega) = \mathcal{F} \{ h(t) \}$ in the Fourier domain shows also conjugate complex symmetry

\begin{equation}
H(j \omega) = H^*(-j \omega)
\end{equation}

due to the [symmetry of the Fourier transform for real-valued signals](../fourier_transform/properties.ipynb#Real-valued-signals). The magnitude spectrum $|H(j \omega)|$ shows even symmetry and the phase $\varphi(j \omega)$ odd symmetry

\begin{align}
|H(j \omega)| &= |H(-j \omega)| \\
\varphi(j \omega) &= - \varphi(-j \omega)
\end{align}

Due to these symmetries often only the positive part $\omega \geq 0$ of the magnitude and phase of the transfer function are plotted. Note that such a representation is ambiguous without the additional knowledge that the system is real-valued.

Since physically existing signals are real-valued, real-valued systems are of paramount importance in practical applications. The symmetry relations derived above have to be considered when designing real-valued systems in the spectral domain.

### Distortionless System

A distortionless system is a system whose output signal $y(t)$ is the attenuated and delayed version of the input signal $x(t)$

\begin{equation}
y(t) = H_0 \cdot x(t - \tau)
\end{equation}

where $H_0 \in \mathbb{R}$ denotes the attenuation and $\tau \in \mathbb{R}$ the delay. The impulse response of a distortionless system is consequently given as

\begin{equation}
h(t) = H_0 \cdot \delta(t - \tau)
\end{equation}

This can be concluded from the [sifting property of the Dirac pulse](../continuous_signals/standard_signals.ipynb#Dirac-Impulse). Its transfer function $H(j \omega) = \mathcal{F} \{ h(t) \}$ reads

\begin{equation}
H(j \omega) = H_0 \cdot e^{- j \omega \tau}
\end{equation}

It follows that the magnitude response of a distortonless system  is constant $| H(j \omega) | = H_0$, the phase response is linearly dependent on the frequency $\varphi_\text{H}(j \omega) = - \omega \tau$, and the [phase and group delay](../systems_spectral_domain/phase_group_delay.ipynb) are constant $t_p = t_g = \tau$.

The characteristics of a distortionless system are often the desired properties of sensors, actuators, amplifiers and transmission paths. In this context the distortionless system serves as an idealized model for such elements or is used as goal for the equalization of non-ideal elements.

### Linear-Phase System

Systems whose unwrapped phase $\varphi_\text{H}(j \omega) = \arg \{ H(j \omega) \}$ scales linearly with frequency

\begin{equation}
\varphi_\text{H}(j \omega) = - c \cdot \omega + d
\end{equation}

with $c, d \in \mathbb{R}$ are termed *linear-phase systems*. There are no constraints with respect to their magnitude response $|H(j \omega)|$. The transfer function $H(j \omega)$ is given as

\begin{equation}
H(j \omega) = |H(j \omega)| \; e^{j (- c \omega + d)}
\end{equation}

The group delay of a linear-phase system is constant

\begin{equation}
t_g(\omega) = - \frac{\partial (- c \cdot \omega + d)}{\partial \omega} = c
\end{equation}

Systems with a linear phase are often desired in practical applications, since all spectral components of a signal are delayed by the same amount when passing the system. The distortionless system introduced above can be seen as a special linear-phase system with $|H(j \omega)| = H_0$, $c = \tau$ and $d=0$.

The impulse response of a linear-phase system shows a specific symmetry. Let's first assume that $c=0$ and $d=0$. Splitting $|H(j \omega)| \in \mathbb{R}$ into its odd and even part, and considering the [symmetries of the Fourier transform](../fourier_transform/properties.ipynb#Symmetries) yields

\begin{equation}
h(t) = h^*(-t)
\end{equation}

This result can be generalized to the case $c \neq 0$ by considering the [temporal shift theorem](../fourier_transform/theorems.ipynb#Temporal-Shift-Theorem) when deriving the impulse response from the transfer function. Above symmetry generalizes to

\begin{equation}
h(t - c) = h^*(-t + c)
\end{equation}

The impulse response of a linear-phase system exhibits complex conjugate symmetry with respect to the time instant $t-c$. In the general case $d \neq 0$, a constant phase shift is added.

**Example**

A real-valued linear-phase system with the following transfer function

\begin{equation}
H(j \omega) = \Lambda(\omega) \cdot e^{-j c \omega}
\end{equation}

is considered. Its impulse response $h(t) = \mathcal{F}^{-1} \{ H(j \omega) \}$ is derived by applying the [duality principle](../fourier_transform/properties.ipynb#Duality) to the [Fourier transformation of the triangular signal](../fourier_transform/theorems.ipynb#Transformation-of-the-triangular-signal) and considering the [shift theorem](../fourier_transform/theorems.ipynb#Temporal-Shift-Theorem)

\begin{equation}
h(t) = \frac{1}{2 \pi} \text{sinc}^2 \left( \frac{t - c}{2} \right)
\end{equation}

It is straightforward to show that the impulse response fulfills the properties stated above. For illustration, the impulse response is plotted for $c=5$

```python
import sympy as sym
%matplotlib inline
sym.init_printing()

t = sym.symbols('t', real=True)
c = 5

h = 1/(2*sym.pi) * sym.sinc((t-c)/2)**2
sym.plot(h, (t, -5, 15), xlabel='$t$', ylabel='$h(t)$')
```

    <sympy.plotting.plot.Plot at 0x122e8d828>

**Exercise**

The linear-phase system with transfer function

\begin{equation}
H(j \omega) = \text{rect}(\omega - \frac{1}{2}) \cdot e^{-j c \omega}
\end{equation}

is investigated in this exercise.

* Is the system real-valued?
* Derive the impulse response $h(t)$ of the linear-phase system. Is the system causal? Is it stable in the BIBO sennse?
* Do the above derived symmetries hold for the impulse response?

### Minimum-Phase System

For a given magnitude response $|H(j \omega)|$ often the realization which has the minimum possible group delay $t_g(\omega)$ is desired. The group delay quantifies the frequency dependent delay a system introduces to a signal. For many applications this delay should be as small as possible. A [causal and stable system](causality_stability.ipynb) with rational transfer function $H(s)$ is minimum-phase iff it has no zeros and poles in the right $s$-half-plane. Such a system is termed [*minimum-phase system*](https://en.wikipedia.org/wiki/Minimum_phase). It has the minimum possible group delay for a given magnitude response.

In order to proof this, the dependence of the magnitude response $|H(j \omega)|$ and phase $\varphi_\text{H}(j \omega)$ on the locations of the poles and zeros of $H(s)$ is analyzed. It is assumed in the following that the poles of the system are located in the left $s$-half-plane, hence that the system is stable. The magnitude response of the system can then be written as

\begin{equation}
|H(j \omega)| = |K| \cdot \frac{\prod_{\mu=0}^{Q} | j \omega - s_{0 \mu} |}{\prod_{\nu=0}^{P} | j \omega - s_{\infty \nu}|}
= |K| \cdot \frac{\prod_{\mu=0}^{Q} \sqrt{\Re \{ s_{0 \mu} \}^2 + (\omega - \Im \{ s_{0 \mu} \} )^2 }}{\prod_{\nu=0}^{P} \sqrt{\Re \{ s_{\infty \nu} \}^2 + (\omega - \Im \{ s_{\infty \nu} \} )^2 }}
\end{equation}

where $s_{0 \mu}$ and $s_{\infty \nu}$ denote the $\mu$-th zero and $\nu$-th pole of $H(s)$, and $Q$ and $P$ the total number of zeros and poles, respectively. The magnitude response for a given frequency $\omega$ is determined by the relative distances of the poles and zeros to the point $s = j \omega$ on the imaginary axis. Mirroring the zeros at the imaginary axis does not change the magnitude response. This is due to the fact that $| j \omega - s_{0 \mu} | = | j \omega + s^*_{0 \mu} |$ holds since only the sign of the real part inside the magnitude changes.

However, the group delay is different for zeros in the left and right $s$-half-plane. In the [foundations of Bode plots](../systems_spectral_domain/bode_plot.ipynb#Bode-Plots) it has been shown that the phase $\varphi_\text{H}(j \omega)$ of a rational transfer function $H(s)$ can be written as

\begin{equation}
\varphi_H(j \omega) = \sum_{\mu=0}^{Q} \arg (j \omega - s_{0 \mu}) - \sum_{\nu=0}^{P} \arg (j \omega - s_{\infty \nu})
\end{equation}

Since the contributions of the individual zeros are superimposed, let's first look at the contribution of a single zero. The group delay for a single zero $s_0 = \Re \{ s_0 \} + j \Im \{ s_0 \}$ can be derived as

\begin{equation}
t_g(\omega) = - \frac{d}{d \omega} \arg (j \omega - s_0) =
\frac{\Re \{ s_0 \}}{(\omega - \Im \{ s_0 \})^2 + \Re \{s_0\}} =
\frac{\Re \{ s_0 \}}{\omega^2 - 2 \omega \Im \{ s_0 \} + |s_0|^2}
\end{equation}

For a given magnitude response, the denominator is invariant to a mirroring of the zero at the imaginary axis. However, the numerator $\Re \{ s_0 \} < 0$ for a zero in the left $s$-half-plane and $\Re \{ s_0 \} > 0$ for a zero in the right $s$-half-plane. Hence, the group delay for the zero in the left $s$-half-plane is always smaller than for its mirror-imaged zero in the right $s$-half-plane. Minimizing the group delay for each zero minimizes the overall group delay.

A system where zeros are located in the left and right $s$-half-plane is called *non-minimum-phase system* or *mixed-phase system*. A system where all zeros are located in the right $s$-half-plane is called *maximum-phase system*.

**Example**

A stable real-valued minimum-phase system (MPS) with the following transfer function is investigated

\begin{equation}
H_\text{MPS}(j \omega) = \frac{(j \omega - s_0)(j \omega - s_0^*)}{(j \omega - s_\infty)^2}
\end{equation}

The system has a real-valued pole $s_\infty$ of order two, and a pair of complex conjugate zeros $s_0$ and $s_0^*$. The corresponding non-minimum-phase system (NMPS) with equal magnitude response $|H_\text{MPS}(j \omega)| = |H_\text{NMPS}(j \omega)|$ is yielded by mirroring the zeros at the imaginary axis

\begin{equation}
H_\text{NMPS}(j \omega) = \frac{(j \omega + s_0^*)(j \omega + s_0)}{(j \omega - s_\infty)^2}
\end{equation}

The magnitude response and group delay for both systems are computed and plotted for $s_\infty = -2$ and $s_0 = - 1 + j$. First the transfer functions are defined and the group delays are computed

```python
w = sym.symbols('omega', real=True)
jw = sym.I * w
s_0 = - 1 + sym.I
s_inf = -2

H_MPS = (jw - s_0)*(jw - sym.conjugate(s_0))/(jw - s_inf)**2
H_NMPS = (jw + sym.conjugate(s_0))*(jw + s_0)/(jw - s_inf)**2

tg_MPS = - sym.diff(sym.arg(H_MPS), w)
tg_NMPS = - sym.diff(sym.arg(H_NMPS), w)
```

The magnitude response $|H_\text{MPS}(j \omega)|$ of the MPS is illustrated by the green line and the magnitude response $|H_\text{NMPS}(j \omega)|$ of the NMPS by the red line, respectively. Note that both overlap.

```python
p1 = sym.plot(sym.Abs(H_MPS), (w, -5, 5), line_color='g', xlabel='$\omega$', ylabel='$|H(j \omega)|$', show=False)
p2 = sym.plot(sym.Abs(H_NMPS), (w, -5, 5), line_color='r', show=False)
p1.extend(p2)
p1.show()
```

The same color scheme is used for the group delay of both systems

```python
p1 = sym.plot(tg_MPS, (w, -5, 5), line_color='g', xlabel='$\omega$', ylabel='$t_g(\omega)$', show=False)
p2 = sym.plot(tg_NMPS, (w, -5, 5), line_color='r', show=False)
p1.extend(p2)
p1.show()
```

**Exercise**

* Add a second pair of complex conjugate zeros and a pair of complex conjugate poles to above MPS.
* For your MPS system derive the transfer function $H(j \omega)$ of the corresponding maximum-phase and one mixed-phase system.
* Compute the group delays for all three systems.
* In what range is the group delay of the mixed-phase system located with respect to the minimum- and maximum-phase system?

### All-Pass

An [all-pass](https://en.wikipedia.org/wiki/All-pass_filter) is a system with constant magnitude response $|H(j \omega)| = H_0$ but frequency dependent phase $\varphi_\text{H}(j \omega)$. It allows to modify the phase of a signal without modifying its magnitude. The [distortionless system](#Distortionless-System) is a special case of an all-pass with $\varphi_\text{H}(j \omega) = - \omega \tau$. A stable system with rational transfer function $H(s)$ is an causal all-pass iff for each pole in the left $s$-half-plane there exists a zero in the right $s$-half-plane which is mirrored at the imaginary axis $\Re \{ s \} = 0$.

The magnitude response $|H(j \omega)|$ of a system $H(s)$ with respect to its poles and zeros has already been derived before as

\begin{equation}
|H(j \omega)| = |K| \cdot \frac{\prod_{\mu=0}^{Q} | j \omega - s_{0 \mu} |}{\prod_{\nu=0}^{P} | j \omega - s_{\infty \nu}|}
\end{equation}

For a pole $s_{\infty \mu}$ and its mirror-imaged zero $s_{0 \nu} = - s_{\infty \mu}^*$ the following holds

\begin{equation}
|j \omega - s_{\infty \mu}| = |j \omega - s_{0 \nu}|
\end{equation}

since only the sign of the real part of the mirror-imaged zero changes. Introducing this result into $|H(j \omega)|$ above for all pole/zero pairs yields that the magnitude response of an all-pass is constant.

**Example**

The properties of a stable real-valued 2nd-order all-pass with transfer function

\begin{equation}
H(j \omega) = \frac{(j \omega - s_0)(j \omega - s_0^*)}{(j \omega - s_\infty)(j \omega - s_\infty^*)}
\end{equation}

are investigated. The pole is chosen as $s_\infty = -1 + j$, and the zero due to the required symmetry as $s_0 = - s_\infty^*$. First the transfer function is defined

```python
s_inf = -1 + sym.I
s_0 = - sym.conjugate(s_inf)

H = (jw - s_0)*(jw - sym.conjugate(s_0)) / \
    ((jw - s_inf)*(jw - sym.conjugate(s_inf)))
H
```

The magnitude response $|H(j \omega)|$ of the all-pass is plotted

```python
sym.plot(sym.Abs(H), (w, -5, 5), xlabel='$\omega$', ylabel='$|H(j \omega)|$', ylim=(0, 1.2));
```

as well as its phase $\varphi_\text{H}(j \omega)$

```python
sym.plot(sym.arg(H), (w, -5, 5), xlabel='$\omega$', ylabel=r'$\varphi(j \omega)$');
```

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


# 1.1 Activation Maximization

## Introduction

**This section corresponds to Section 3.1 of the original paper.**

When we are asked to describe a certain concept, say, a chair, it is impossible to come up with a precise description because there are so many types of chairs. However, there are certain characteristics unique to chairs that let us recognize chairs if we see one, such as a platform that we can sit on or legs that support the platform. With such features of chair in mind, we may be able draw a picture of a typical chair. In this section, we ask a deep neural network (DNN) to give us a prototype image $x^{*}$ that describes characteristics common to the set of objects it was trained to recognize.

A deep neural network classifier mapping a set of data points or images $x$ to a set of classes $(\omega_c)_c$ models the conditional probability distribution $p(\omega_c | x)$. Therefore, a prototype $x^{*}$ can be found by optimizing the following objective:

\begin{equation}
\max_x \log p(\omega_c \, | \, x) - \lambda \lVert x\rVert^2
\end{equation}

This can be achieved by first training a deep neural network and then performing gradient ascent on images with randomly initialized pixel values. The rightmost term of the objective function ($\lambda \lVert x\rVert^2$) is an $l_2$-norm regularizer that prevents prototype images from deviating largely from the origin. As we will see later, activation maximization is crude compared to other interpretation techniques in that it is unable to produce natural-looking prototype images.

## Tensorflow Walkthrough

### 1. Import Dependencies

We import a pre-built DNN that we will train and then perform activation maximization on. You can check out `models_1_1.py` in the models directory for more network details.

```python
import os

from tensorflow.examples.tutorials.mnist import input_data
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np

from models.models_1_1 import MNIST_DNN

%matplotlib inline

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

logdir = './tf_logs/1_1_AM/'
ckptdir = logdir + 'model'

if not os.path.exists(logdir):
    os.mkdir(logdir)
```

    Extracting MNIST_data/train-images-idx3-ubyte.gz
    Extracting MNIST_data/train-labels-idx1-ubyte.gz
    Extracting MNIST_data/t10k-images-idx3-ubyte.gz
    Extracting MNIST_data/t10k-labels-idx1-ubyte.gz

### 2. Building Graph

In this step, we initialize a DNN classifier and attach necessary nodes for model training onto the computation graph.

```python
with tf.name_scope('Classifier'):

    # Initialize neural network
    DNN = MNIST_DNN('DNN')

    # Setup training process
    lmda = tf.placeholder_with_default(0.01, shape=[], name='lambda')
    X = tf.placeholder(tf.float32, [None, 784], name='X')
    Y = tf.placeholder(tf.float32, [None, 10], name='Y')

    tf.add_to_collection('placeholders', lmda)

    logits = DNN(X)

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y))

    optimizer = tf.train.AdamOptimizer().minimize(cost, var_list=DNN.vars)

    correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

cost_summary = tf.summary.scalar('Cost', cost)
accuray_summary = tf.summary.scalar('Accuracy', accuracy)
summary = tf.summary.merge_all()
```

### 3. Building Subgraph for Generating Prototypes

Before training the network, we attach a subgraph for generating prototypes onto the computation graph.

```python
with tf.name_scope('Prototype'):

    X_mean = tf.placeholder(tf.float32, [10, 784], name='X_mean')
    X_prototype = tf.get_variable('X_prototype', shape=[10, 784], initializer=tf.constant_initializer(0.))
    Y_prototype = tf.one_hot(tf.cast(tf.lin_space(0., 9., 10), tf.int32), depth=10)

    logits_prototype = DNN(X_prototype, reuse=True)

    # Objective function definition
    cost_prototype = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits_prototype, labels=Y_prototype)) \
                     + lmda * tf.nn.l2_loss(X_prototype - X_mean)

    optimizer_prototype = tf.train.AdamOptimizer().minimize(cost_prototype, var_list=[X_prototype])

# Add the subgraph nodes to a collection so that they can be used after training of the network
tf.add_to_collection('prototype', X_mean)
tf.add_to_collection('prototype', X_prototype)
tf.add_to_collection('prototype', Y_prototype)
tf.add_to_collection('prototype', logits_prototype)
tf.add_to_collection('prototype', cost_prototype)
tf.add_to_collection('prototype', optimizer_prototype)
```

Here's the general structure of the computation graph visualized using tensorboard.

### 4. Training Network

This is the step where the DNN is trained to classify the 10 digits of the MNIST images. Summaries are written into the logdir and you can visualize the statistics using tensorboard by typing this command: `tensorboard --lodir=./tf_logs`

```python
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())

saver = tf.train.Saver()
file_writer = tf.summary.FileWriter(logdir, tf.get_default_graph())

# Hyper parameters
training_epochs = 15
batch_size = 100

for epoch in range(training_epochs):
    total_batch = int(mnist.train.num_examples / batch_size)
    avg_cost = 0
    avg_acc = 0

    for i in range(total_batch):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size)
        _, c, a, summary_str = sess.run([optimizer, cost, accuracy, summary], feed_dict={X: batch_xs, Y: batch_ys})
        avg_cost += c / total_batch
        avg_acc += a / total_batch

        file_writer.add_summary(summary_str, epoch * total_batch + i)

    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost), 'accuracy =', '{:.9f}'.format(avg_acc))

    saver.save(sess, ckptdir)

print('Accuracy:', sess.run(accuracy, feed_dict={X: mnist.test.images, Y: mnist.test.labels}))

sess.close()
```

    Epoch: 0001 cost = 0.235915433 accuracy = 0.931200002
    Epoch: 0002 cost = 0.089025966 accuracy = 0.973109100
    Epoch: 0003 cost = 0.058212864 accuracy = 0.981927283
    Epoch: 0004 cost = 0.037602341 accuracy = 0.988036373
    Epoch: 0005 cost = 0.030371556 accuracy = 0.990381826
    Epoch: 0006 cost = 0.021637062 accuracy = 0.993490915
    Epoch: 0007 cost = 0.018915237 accuracy = 0.994000006
    Epoch: 0008 cost = 0.015711658 accuracy = 0.995127277
    Epoch: 0009 cost = 0.014997141 accuracy = 0.995127277
    Epoch: 0010 cost = 0.015114903 accuracy = 0.994981823
    Epoch: 0011 cost = 0.012736740 accuracy = 0.995909095
    Epoch: 0012 cost = 0.011209372 accuracy = 0.996545458
    Epoch: 0013 cost = 0.009346475 accuracy = 0.996727276
    Epoch: 0014 cost = 0.008826889 accuracy = 0.996781821
    Epoch: 0015 cost = 0.009214011 accuracy = 0.996618185
    Accuracy: 0.9813

### 5. Restoring Subgraph

Here we first rebuild the DNN graph from metagraph, restore the DNN parameters from the checkpoint and then gather the necessary nodes for prototype generation using the `tf.get_collection()` function (recall prototype subgraph nodes were added onto the 'prototype' collection at step 3).

```python
tf.reset_default_graph()

sess = tf.InteractiveSession()

new_saver = tf.train.import_meta_graph(ckptdir + '.meta')
new_saver.restore(sess, tf.train.latest_checkpoint(logdir))

# Get necessary placeholders
lmda = tf.get_collection('placeholders')[0]

# Get prototype nodes
prototype = tf.get_collection('prototype')
X_mean = prototype[0]
X_prototype = prototype[1]
cost_prototype = prototype[4]
optimizer_prototype = prototype[5]
```

    INFO:tensorflow:Restoring parameters from ./tf_logs/1_1_AM/model

### 6. Generating Prototype Images

Before performing gradient ascent, we calculate the image means $\overline{x}$ that will be used to regularize the prototype images. Then, we generate prototype images that maximize $\log p(\omega_c | x) - \lambda \lVert x - \overline{x}\rVert^2$. I used 0.1 for lambda (lmda), but fine tuning may produce better prototype images.

```python
images = mnist.train.images
labels = mnist.train.labels

img_means = []
for i in range(10):
    img_means.append(np.mean(images[np.argmax(labels, axis=1) == i], axis=0))

for epoch in range(5000):
    _, c = sess.run([optimizer_prototype, cost_prototype], feed_dict={lmda: 0.1, X_mean: img_means})

    if epoch % 500 == 0:
        print('Epoch: {:05d} Cost = {:.9f}'.format(epoch, c))

X_prototypes = sess.run(X_prototype)

sess.close()
```

    Epoch: 00000 Cost = 25.786497116
    Epoch: 00500 Cost = 2.708985567
    Epoch: 01000 Cost = 0.190258697
    Epoch: 01500 Cost = 0.012256578
    Epoch: 02000 Cost = 0.005008637
    Epoch: 02500 Cost = 0.004844339
    Epoch: 03000 Cost = 0.004842392
    Epoch: 03500 Cost = 0.004842398
    Epoch: 04000 Cost = 0.004842440
    Epoch: 04500 Cost = 0.004842412

### 7. Displaying Images

As I mentioned in the introduction, the resulting prototype images are rather blurry.

```python
plt.figure(figsize=(15,15))
for i in range(5):
    plt.subplot(5, 2, 2 * i + 1)
    plt.imshow(np.reshape(X_prototypes[2 * i], [28, 28]), cmap='gray', interpolation='none')
    plt.title('Digit: {}'.format(2 * i))
    plt.colorbar()

    plt.subplot(5, 2, 2 * i + 2)
    plt.imshow(np.reshape(X_prototypes[2 * i + 1], [28, 28]), cmap='gray', interpolation='none')
    plt.title('Digit: {}'.format(2 * i + 1))
    plt.colorbar()

plt.tight_layout()
```

```python

```


# Random Signals and LTI-Systems

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing.

## Auto-Correlation Function

The auto-correlation function (ACF) $\varphi_{yy}[\kappa]$ of the output signal of an LTI system $y[k] = \mathcal{H} \{ x[k] \}$ is derived. It is assumed that the input signal is a wide-sense stationary (WSS) real-valued random process and that the LTI system has a real-valued impulse repsonse $h[k] \in \mathbb{R}$.

Introducing the output relation $y[k] = h[k] * x[k]$ of an LTI system into the definition of the ACF and rearranging terms yields

\begin{align}
\varphi_{yy}[\kappa] &= E \{ y[k+\kappa] \cdot y[k] \} \\
&= E \left\{  \sum_{\mu = -\infty}^{\infty} h[\mu] \; x[k+\kappa-\mu] \cdot
\sum_{\nu = -\infty}^{\infty} h[\nu] \; x[k-\nu] \right\} \\
&= \underbrace{h[\kappa] * h[-\kappa]}_{\varphi_{hh}[\kappa]} * \varphi_{xx}[\kappa]
\end{align}

where the ACF $\varphi_{hh}[\kappa]$ of the deterministic impulse response $h[k]$ is commonly termed as *filter ACF*. This is related to the [link between ACF and convolution](../random_signals/correlation_functions.ipynb#Definition). The relation above is known as the *Wiener-Lee theorem*. It states that the ACF of the output $\varphi_{yy}[\kappa]$ of an LTI system is given by the convolution of the input signal's ACF $\varphi_{xx}[\kappa]$ with the filter ACF $\varphi_{hh}[\kappa]$. For a system which just attenuates the input signal $y[k] = A \cdot x[k]$ with $A \in \mathbb{R}$, the ACF at the output is given as $\varphi_{yy}[\kappa] = A^2 \cdot \varphi_{xx}[\kappa]$.

### Example - System Response to White Noise

Let's assume that the wide-sense ergodic input signal $x[k]$ of an LTI system with impulse response $h[k] = \text{rect}_N[k]$ is normal distributed white noise. Introducing $\varphi_{xx}[\kappa] = N_0\, \delta[\kappa]$ and $h[k]$ into the Wiener-Lee theorem yields

\begin{equation}
\varphi_{yy}[\kappa] = N_0 \cdot \varphi_{hh}[\kappa] = N_0 \cdot (\text{rect}_N[\kappa] * \text{rect}_N[-\kappa])
\end{equation}

The example is evaluated numerically for $N_0 = 1$ and $N=5$

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

L = 10000  # number of samples
K = 30  # limit for lags in ACF

# generate input signal (white Gaussian noise)
np.random.seed(2)
x = np.random.normal(size=L)
# compute system response
y = np.convolve(x, [1, 1, 1, 1, 1], mode='full')

# compute and truncate ACF
acf = 1/len(y) * np.correlate(y, y, mode='full')
acf = acf[len(y)-K-1:len(y)+K-1]
kappa = np.arange(-K, K)

# plot ACF
plt.figure(figsize = (10, 6))
plt.stem(kappa, acf)
plt.title('Estimated ACF of output signal $y[k]$')
plt.ylabel(r'$\hat{\varphi}_{yy}[\kappa]$')
plt.xlabel(r'$\kappa$')
plt.axis([-K, K, 1.2*min(acf), 1.1*max(acf)]);
plt.grid()
```

**Exercise**

* Derive the theoretic result for $\varphi_{yy}[\kappa]$ by calculating the filter-ACF $\varphi_{hh}[\kappa]$.
* Why is the estimated ACF $\hat{\varphi}_{yy}[\kappa]$ of the output signal not exactly equal to its theoretic result $\varphi_{yy}[\kappa]$?
* Change the number of samples `L` and rerun the example. What changes?

Solution: The filter-ACF is given by $\varphi_{hh}[\kappa] = \text{rect}_N[\kappa] * \text{rect}_N[-\kappa]$. The convolution of two rectangular signals $\text{rect}_N[\kappa]$ results in a triangular signal. Taking the time reversal into account yields

\begin{equation}
\varphi_{hh}[\kappa] = \begin{cases}
N - |\kappa| & \text{for } -N < \kappa \leq N \\
0 & \text{otherwise}
\end{cases}
\end{equation}

for even $N$. The estimated ACF $\hat{\varphi}_{yy}[\kappa]$ differs from its theoretic value due to the statistical uncertainties when using random signals of finite length. Increasing its length `L` lowers the statistical uncertainties.

## Cross-Correlation Function

The cross-correlation functions (CCFs) $\varphi_{xy}[\kappa]$ and $\varphi_{yx}[\kappa]$ between the in- and output signal of an LTI system $y[k] = \mathcal{H} \{ x[k] \}$ are derived. As for the ACF it is assumed that the input signal originates from a wide-sense stationary real-valued random process and that the LTI system's impulse response is real-valued, i.e. $h[k] \in \mathbb{R}$.

Introducing the convolution into the definition of the CCF and rearranging the terms yields

\begin{align}
\varphi_{xy}[\kappa] &= E \{ x[k+\kappa] \cdot y[k] \} \\
&= E \left\{ x[k+\kappa] \cdot \sum_{\mu = -\infty}^{\infty} h[\mu] \; x[k-\mu] \right\} \\
&= \sum_{\mu = -\infty}^{\infty} h[\mu] \cdot E \{ x[k+\kappa] \cdot x[k-\mu] \} \\
&= h[-\kappa] * \varphi_{xx}[\kappa]
\end{align}

The CCF $\varphi_{xy}[\kappa]$ between in- and output is given as the time-reversed impulse response of the system convolved with the ACF of the input signal.

The CCF between out- and input is yielded by taking the symmetry relations of the CCF and ACF into account

\begin{equation}
\varphi_{yx}[\kappa] = \varphi_{xy}[-\kappa] = h[\kappa] * \varphi_{xx}[\kappa]
\end{equation}

The CCF $\varphi_{yx}[\kappa]$ between out- and input is given as the impulse response of the system convolved with the ACF of the input signal.

For a system which just attenuates the input signal $y[k] = A \cdot x[k]$, the CCFs between input and output are given as $\varphi_{xy}[\kappa] = A \cdot \varphi_{xx}[\kappa]$ and $\varphi_{yx}[\kappa] = A \cdot \varphi_{xx}[\kappa]$.

## System Identification by Cross-Correlation

The process of determining the impulse response or transfer function of a system is referred to as *system identification*. The CCFs of an LTI system play an important role in the estimation of the impulse response $h[k]$ of an unknown system. This is illustrated in the following.

The basic idea is to use a specific measurement signal as input signal to the system. Let's assume that the unknown LTI system is excited by [white noise](../random_signals/white_noise.ipynb). The ACF of the wide-sense stationary input signal $x[k]$ is then given as $\varphi_{xx}[\kappa] = N_0 \cdot \delta[\kappa]$. According to the relation derived above, the CCF between out- and input for this special choice of the input signal becomes

\begin{equation}
\varphi_{yx}[\kappa] = h[\kappa] * N_0 \cdot \delta[\kappa] = N_0 \cdot h[\kappa]
\end{equation}

For white noise as input signal $x[k]$, the impulse response of an LTI system can be estimated by estimating the CCF between its out- and input signals. Using noise as measurement signal instead of a Dirac impulse is beneficial since its [crest factor](https://en.wikipedia.org/wiki/Crest_factor) is limited.

### Example

The application of the CCF to the identification of a system is demonstrated. The system is excited by wide-sense ergodic normal distributed white noise with $N_0 = 1$. The ACF of the in- and output, as well as the CCF between out- and input is estimated and plotted.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sig

N = 10000  # number of samples for input signal
K = 50  # limit for lags in ACF

# generate input signal
np.random.seed(5)
x = np.random.normal(size=N)  # normally distributed (zero-mean, unit-variance) white noise
# impulse response of the system
h = np.concatenate((np.zeros(10), sig.triang(10), np.zeros(10)))
# output signal by convolution
y = np.convolve(h, x, mode='full')

# compute correlation functions
acfx = 1/len(x) * np.correlate(x, x, mode='full')
acfy = 1/len(y) * np.correlate(y, y, mode='full')
ccfyx = 1/len(y) * np.correlate(y, x, mode='full')

def plot_correlation_function(cf):
    cf = cf[N-K-1:N+K-1]
    kappa = np.arange(-len(cf)//2,len(cf)//2)
    plt.stem(kappa, cf)
    plt.xlabel(r'$\kappa$')
    plt.axis([-K, K, -0.2, 1.1*max(cf)])

# plot ACFs and CCF
plt.rc('figure', figsize=(10, 3))
plt.figure()
plot_correlation_function(acfx)
plt.title('Estimated ACF of input signal')
plt.ylabel(r'$\hat{\varphi}_{xx}[\kappa]$')

plt.figure()
plot_correlation_function(acfy)
plt.title('Estimated ACF of output signal')
plt.ylabel(r'$\hat{\varphi}_{yy}[\kappa]$')

plt.figure()
plot_correlation_function(ccfyx)
plt.plot(np.arange(len(h)), h, 'g-')
plt.title('Estimated and true impulse response')
plt.ylabel(r'$\hat{h}[k]$, $h[k]$');
```

**Exercise**

* Why is the estimated CCF $\hat{\varphi}_{yx}[k]$ not exactly equal to the true impulse response $h[k]$ of the system?
* What changes if you change the number of samples `N` of the input signal?

Solution: The derived relations for system identification hold for the case of a wide-sense ergodic input signal of infinite duration. Since we can only numerically simulate signals of finite duration, the observed deviations are a result of the resulting statistical uncertainties. Increasing the length `N` of the input signal improves the estimate of the impulse response.


## Knowing the what, but not the where in Bayesian optimization. Vu Nguyen, Michael Osborne. ICML 2020

# Black-box optimization given the knowledge of the optimum value
\begin{align}
f^* = argmax_{x \in \mathcal{X}} f(x)
\end{align}

# Contact: Vu Nguyen, vu@ieee.org

```python
import sys
sys.path.insert(0,'..')
sys.path.insert(0,'../..')

from bayes_opt import BayesOpt_KnownOptimumValue,BayesOpt
import numpy as np
from bayes_opt import vis_ERM,functions
import warnings
warnings.filterwarnings("ignore")

```

# Transforming the surrogate function go below the optimum value $f*$

```python
myfunction=functions.fourier(sd=0)
# myfunction.func: contains the black-box function
# myfunction.bounds: contains the SearchSpace
# myfunction.fstar: contains the known optimum value

# four initial points
x0=[3.1,4.4,8,9]

init_X=np.reshape(x0,(len(x0),1))
init_Y=myfunction.func(init_X)

# create an empty object for BO using GP
acq_name='ei'
bo=BayesOpt(myfunction.func,myfunction.bounds,acq_name=acq_name,verbose=0)
bo.init_with_data(init_X=init_X,init_Y=init_Y)

# create an empty object for BO using transformed GP
acq_name='erm'
IsTGP=1 # using transformed GP
bo_tgp=BayesOpt_KnownOptimumValue(myfunction.func,myfunction.bounds,fstar=myfunction.fstar, \
                                  acq_name=acq_name,IsTGP=1)
bo_tgp.init_with_data(init_X=init_X,init_Y=init_Y)

vis_ERM.plot_1d_Fourier_GP_TGP(bo,bo_tgp,fstar=myfunction.fstar)
```

# Transforming the surrogate function closer to the optimum value $f*$

```python
myfunction=functions.forrester(sd=0)
# myfunction.func: contains the black-box function
# myfunction.bounds: contains the SearchSpace
# myfunction.fstar: contains the known optimum value

# initial 3 points
x0=[0.1,0.46,0.91]

init_X=np.reshape(x0,(len(x0),1))
init_Y=myfunction.func(init_X)

# create an empty object for BO using  vanilla GP
acq_name='ei'
bo=BayesOpt(myfunction.func,myfunction.bounds,acq_name=acq_name,verbose=0)
bo.init_with_data(init_X=init_X,init_Y=init_Y)

# create an empty object for BO using transformed GP
acq_name='erm'
IsTGP=1 # using TransformedGP
bo_tgp=BayesOpt_KnownOptimumValue(myfunction.func,myfunction.bounds,fstar=myfunction.fstar, \
                                  acq_name=acq_name,IsTGP=1,verbose=0)
bo_tgp.init_with_data(init_X=init_X,init_Y=init_Y)

vis_ERM.plot_1d_Forrester_GP_TGP(bo,bo_tgp,fstar=myfunction.fstar)

```

# demonstrating different acquisition functions given known optimum value

```python
myfunction=functions.fourier(sd=0)
# myfunction.func: contains the black-box function
# myfunction.bounds: contains the SearchSpace
# myfunction.fstar: contains the known optimum value

x0=[3.2,4.4,8,9]
init_X=np.reshape(x0,(len(x0),1))
init_Y=myfunction.func(init_X)

# create an empty object for BO using transformed GP
acq_name='erm'
IsTGP=1

bo_tgp=BayesOpt_KnownOptimumValue(myfunction.func,myfunction.bounds,fstar=myfunction.fstar, \
                                  acq_name=acq_name,IsTGP=IsTGP)
bo_tgp.init_with_data(init_X=init_X,init_Y=init_Y)

NN=1*myfunction.input_dim
for index in range(0,NN):
    vis_ERM.plot_acq_bo_1d_tgp(bo_tgp,fstar=myfunction.fstar)
```

# Running multiple iterations
# using vanilla GP

```python
myfunction=functions.forrester(sd=0)

x0=[0.1,0.46,0.91]
init_X=np.reshape(x0,(len(x0),1))
init_Y=myfunction.func(init_X)

# create an empty object for BO using vanilla GP
acq_name='ei'
bo=BayesOpt(myfunction.func,myfunction.bounds,acq_name=acq_name,verbose=1)
bo.init_with_data(init_X=init_X,init_Y=init_Y)

vis_ERM.plot_1d_Forrester_EI_ERM(bo,fstar=myfunction.fstar)

# number of recommended parameters
NN=5*myfunction.input_dim
for index in range(0,NN):
    xt=bo.select_next_point()
    vis_ERM.plot_1d_Forrester_EI_ERM(bo,fstar=myfunction.fstar)
```

# Running multiple iterations
# using Transformed GP and ERM

```python
myfunction=functions.forrester(sd=0)

x0=[0.1,0.46,0.91]
init_X=np.reshape(x0,(len(x0),1))
init_Y=myfunction.func(init_X)

# create an empty object for BO using transformed GP
acq_name='erm'
IsTGP=1 # using Transformed GP

bo_tgp=BayesOpt_KnownOptimumValue(myfunction.func,myfunction.bounds,fstar=myfunction.fstar, \
                              acq_name=acq_name,IsTGP=IsTGP,verbose=1)
bo_tgp.init_with_data(init_X=init_X,init_Y=init_Y)

vis_ERM.plot_1d_tgp_Forrester_EI_ERM(bo_tgp,fstar=myfunction.fstar)

NN=5*myfunction.input_dim
for index in range(0,NN):
    xt=bo_tgp.select_next_point()
    print(bo_tgp.X_ori[-1])

    vis_ERM.plot_1d_tgp_Forrester_EI_ERM(bo_tgp,fstar=myfunction.fstar)

```