# Bayesian Hierarchical Linear Regression
Author: [Carlos Souza](mailto:souza@gatech.edu)

Probabilistic Machine Learning models can not only make predictions about future data, but also **model uncertainty**. In areas such as **personalized medicine**, there might be a large amount of data, but there is still a relatively **small amount of data for each patient**. To customize predictions for each person it becomes necessary to **build a model for each person** — with its inherent **uncertainties** — and to couple these models together in a **hierarchy** so that information can be borrowed from other **similar people** [1].

The purpose of this tutorial is to demonstrate how to **implement a Bayesian Hierarchical Linear Regression model using NumPyro**. To motivate the tutorial, I will use [OSIC Pulmonary Fibrosis Progression](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression) competition, hosted at Kaggle.

## 1. Understanding the task
Pulmonary fibrosis is a disorder with no known cause and no known cure, created by scarring of the lungs. In this competition, we were asked to predict a patient’s severity of decline in lung function. Lung function is assessed based on output from a spirometer, which measures the forced vital capacity (FVC), i.e. the volume of air exhaled.

In medical applications, it is useful to **evaluate a model's confidence in its decisions**. Accordingly, the metric used to rank the teams was designed to reflect **both the accuracy and certainty of each prediction**. It's a modified version of the Laplace Log Likelihood (more details on that later).

Let's explore the data and see what's that all about:

```python
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
```

```python
train = pd.read_csv('https://gist.githubusercontent.com/ucals/'
                    '2cf9d101992cb1b78c2cdd6e3bac6a4b/raw/'
                    '43034c39052dcf97d4b894d2ec1bc3f90f3623d9/'
                    'osic_pulmonary_fibrosis.csv')
train.head()
```

      Patient
      Weeks
      FVC
      Percent
      Age
      Sex
      SmokingStatus

      0
      ID00007637202177411956430
      -4
      2315
      58.253649
      79
      Male
      Ex-smoker

      1
      ID00007637202177411956430
      5
      2214
      55.712129
      79
      Male
      Ex-smoker

      2
      ID00007637202177411956430
      7
      2061
      51.862104
      79
      Male
      Ex-smoker

      3
      ID00007637202177411956430
      9
      2144
      53.950679
      79
      Male
      Ex-smoker

      4
      ID00007637202177411956430
      11
      2069
      52.063412
      79
      Male
      Ex-smoker

In the dataset, we were provided with a baseline chest CT scan and associated clinical information for a set of patients. A patient has an image acquired at time Week = 0 and has numerous follow up visits over the course of approximately 1-2 years, at which time their FVC is measured. For this tutorial, I will use only the Patient ID, the weeks and the FVC measurements, discarding all the rest. Using only these columns enabled our team to achieve a competitive score, which shows the power of Bayesian hierarchical linear regression models especially when gauging uncertainty is an important part of the problem.

Since this is real medical data, the relative timing of FVC measurements varies widely, as shown in the 3 sample patients below:

```python
def chart(patient_id, ax):
    data = train[train['Patient'] == patient_id]
    x = data['Weeks']
    y = data['FVC']
    ax.set_title(patient_id)
    ax = sns.regplot(x, y, ax=ax, ci=None, line_kws={'color':'red'})

f, axes = plt.subplots(1, 3, figsize=(15, 5))
chart('ID00007637202177411956430', axes[0])
chart('ID00009637202177434476278', axes[1])
chart('ID00010637202177584971671', axes[2])
```

On average, each of the 176 provided patients made 9 visits, when FVC was measured. The visits happened in specific weeks in the [-12, 133] interval. The decline in lung capacity is very clear. We see, though, they are very different from patient to patient.

We were are asked to predict every patient's FVC measurement for every possible week in the [-12, 133] interval, and the confidence for each prediction. In other words: we were asked fill a matrix like the one below, and provide a confidence score for each prediction:

The task was perfect to apply Bayesian inference. However, the vast majority of solutions shared by Kaggle community used discriminative machine learning models, disconsidering the fact that most discriminative methods are very poor at providing realistic uncertainty estimates. Because they are typically trained in a manner that optimizes the parameters to minimize some loss criterion (e.g. the predictive error), they do not, in general, encode any uncertainty in either their parameters or the subsequent predictions. Though many methods can produce uncertainty estimates either as a by-product or from a post-processing step, these are typically heuristic based, rather than stemming naturally from a statistically principled estimate of the target uncertainty distribution [2].

## 2. Modelling: Bayesian Hierarchical Linear Regression with Partial Pooling
The simplest possible linear regression, not hierarchical, would assume all FVC decline curves have the same $\alpha$ and $\beta$. That's the **pooled model**. In the other extreme, we could assume a model where each patient has a personalized FVC decline curve, and **these curves are completely unrelated**. That's the **unpooled model**, where each patient has completely separate regressions.

Here, I'll use the middle ground: **Partial pooling**. Specifically, I'll assume that while $\alpha$'s and $\beta$'s are different for each patient as in the unpooled case, **the coefficients all share similarity**. We can model this by assuming that each individual coefficient comes from a common group distribution. The image below represents this model graphically:

Mathematically, the model is described by the following equations:

\begin{align}
\mu_{\alpha} &\sim \mathcal{N}(0, 100) \\
\sigma_{\alpha} &\sim |\mathcal{N}(0, 100)| \\
\mu_{\beta} &\sim \mathcal{N}(0, 100) \\
\sigma_{\beta} &\sim |\mathcal{N}(0, 100)| \\
\alpha_i &\sim \mathcal{N}(\mu_{\alpha}, \sigma_{\alpha}) \\
\beta_i &\sim \mathcal{N}(\mu_{\beta}, \sigma_{\beta}) \\
\sigma &\sim \mathcal{N}(0, 100) \\
FVC_{ij} &\sim \mathcal{N}(\alpha_i + t \beta_i, \sigma)
\end{align}

where *t* is the time in weeks. Those are very uninformative priors, but that's ok: our model will converge!

Implementing this model in NumPyro is pretty straightforward:

```python
import numpyro
from numpyro.infer import MCMC, NUTS, Predictive
import numpyro.distributions as dist
from jax import random
```

```python
def model(PatientID, Weeks, FVC_obs=None):
    μ_α = numpyro.sample("μ_α", dist.Normal(0., 100.))
    σ_α = numpyro.sample("σ_α", dist.HalfNormal(100.))
    μ_β = numpyro.sample("μ_β", dist.Normal(0., 100.))
    σ_β = numpyro.sample("σ_β", dist.HalfNormal(100.))

    unique_patient_IDs = np.unique(PatientID)
    n_patients = len(unique_patient_IDs)

    with numpyro.plate("plate_i", n_patients):
        α = numpyro.sample("α", dist.Normal(μ_α, σ_α))
        β = numpyro.sample("β", dist.Normal(μ_β, σ_β))

    σ = numpyro.sample("σ", dist.HalfNormal(100.))
    FVC_est = α[PatientID] + β[PatientID] * Weeks

    with numpyro.plate("data", len(PatientID)):
        numpyro.sample("obs", dist.Normal(FVC_est, σ), obs=FVC_obs)
```

That's all for modelling!

## 3. Fitting the model
A great achievement of Probabilistic Programming Languages such as NumPyro is to decouple model specification and inference. After specifying my generative model, with priors, condition statements and data likelihood, I can leave the hard work to NumPyro's inference engine.

Calling it requires just a few lines. Before we do it, let's add a numerical Patient ID for each patient code. That can be easily done with scikit-learn's LabelEncoder:

```python
from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
train['PatientID'] = le.fit_transform(train['Patient'].values)

FVC_obs = train['FVC'].values
Weeks = train['Weeks'].values
PatientID = train['PatientID'].values
```

Now, calling NumPyro's inference engine:

```python
nuts_kernel = NUTS(model)

mcmc = MCMC(nuts_kernel, num_samples=2000, num_warmup=2000)
rng_key = random.PRNGKey(0)
mcmc.run(rng_key, PatientID, Weeks, FVC_obs=FVC_obs)

posterior_samples = mcmc.get_samples()
```

    sample: 100%|██████████| 4000/4000 [00:20<00:00, 195.69it/s, 63 steps of size 1.06e-01. acc. prob=0.89]

## 4. Checking the model
### 4.1. Inspecting the learned parameters
First, let's inspect the parameters learned. To do that, I will use [ArviZ](https://arviz-devs.github.io/arviz/), which perfectly integrates with NumPyro:

```python
import arviz as az

data = az.from_numpyro(mcmc)
az.plot_trace(data, compact=True);
```

Looks like our model learned personalized alphas and betas for each patient!

### 4.2. Visualizing FVC decline curves for some patients
Now, let's visually inspect FVC decline curves predicted by our model. We will completely fill in the FVC table, predicting all missing values. The first step is to create a table to fill:

```python
pred_template = []
for i in range(train['Patient'].nunique()):
    df = pd.DataFrame(columns=['PatientID', 'Weeks'])
    df['Weeks'] = np.arange(-12, 134)
    df['PatientID'] = i
    pred_template.append(df)
pred_template = pd.concat(pred_template, ignore_index=True)
```

Predicting the missing values in the FVC table and confidence (sigma) for each value becomes really easy:

```python
PatientID = pred_template['PatientID'].values
Weeks = pred_template['Weeks'].values
predictive = Predictive(model, posterior_samples,
                        return_sites=['σ', 'obs'])
samples_predictive = predictive(random.PRNGKey(0),
                                PatientID, Weeks, None)
```

Let's now put the predictions together with the true values, to visualize them:

```python
df = pd.DataFrame(columns=['Patient', 'Weeks', 'FVC_pred', 'sigma'])
df['Patient'] = le.inverse_transform(pred_template['PatientID'])
df['Weeks'] = pred_template['Weeks']
df['FVC_pred'] = samples_predictive['obs'].T.mean(axis=1)
df['sigma'] = samples_predictive['obs'].T.std(axis=1)
df['FVC_inf'] = df['FVC_pred'] - df['sigma']
df['FVC_sup'] = df['FVC_pred'] + df['sigma']
df = pd.merge(df, train[['Patient', 'Weeks', 'FVC']],
              how='left', on=['Patient', 'Weeks'])
df = df.rename(columns={'FVC': 'FVC_true'})
df.head()
```

      Patient
      Weeks
      FVC_pred
      sigma
      FVC_inf
      FVC_sup
      FVC_true

      0
      ID00007637202177411956430
      -12
      2219.361084
      159.272430
      2060.088623
      2378.633545
      NaN

      1
      ID00007637202177411956430
      -11
      2209.278076
      157.698868
      2051.579102
      2366.977051
      NaN

      2
      ID00007637202177411956430
      -10
      2212.443115
      154.503906
      2057.939209
      2366.947021
      NaN

      3
      ID00007637202177411956430
      -9
      2208.173096
      153.068268
      2055.104736
      2361.241455
      NaN

      4
      ID00007637202177411956430
      -8
      2202.373047
      157.185608
      2045.187500
      2359.558594
      NaN

Finally, let's see our predictions for 3 patients:

```python
def chart(patient_id, ax):
    data = df[df['Patient'] == patient_id]
    x = data['Weeks']
    ax.set_title(patient_id)
    ax.plot(x, data['FVC_true'], 'o')
    ax.plot(x, data['FVC_pred'])
    ax = sns.regplot(x, data['FVC_true'], ax=ax, ci=None,
                     line_kws={'color':'red'})
    ax.fill_between(x, data["FVC_inf"], data["FVC_sup"],
                    alpha=0.5, color='#ffcd3c')
    ax.set_ylabel('FVC')

f, axes = plt.subplots(1, 3, figsize=(15, 5))
chart('ID00007637202177411956430', axes[0])
chart('ID00009637202177434476278', axes[1])
chart('ID00011637202177653955184', axes[2])
```

The results are exactly what we expected to see! Highlight observations:

- The model adequately learned Bayesian Linear Regressions! The orange line (learned predicted FVC mean) is very inline with the red line (deterministic linear regression). But most important: it learned to predict uncertainty, showed in the light orange region (one sigma above and below the mean FVC line)
- The model predicts a higher uncertainty where the data points are more disperse (1st and 3rd patients). Conversely, where the points are closely grouped together (2nd patient), the model predicts a higher confidence (narrower light orange region)
- Finally, in all patients, we can see that the uncertainty grows as the look more into the future: the light orange region widens as the # of weeks grow!

### 4.3. Computing the modified Laplace Log Likelihood and RMSE

As mentioned earlier, the competition was evaluated on a modified version of the Laplace Log Likelihood. In medical applications, it is useful to evaluate a model's confidence in its decisions. Accordingly, the metric is designed to reflect both the accuracy and certainty of each prediction.

For each true FVC measurement, we predicted both an FVC and a confidence measure (standard deviation $\sigma$). The metric was computed as:

\begin{align}
\sigma_{clipped} &= max(\sigma, 70) \\
\delta &= min(|FVC_{true} - FVC_{pred}|, 1000) \\
metric &= -\dfrac{\sqrt{2}\delta}{\sigma_{clipped}} - \ln(\sqrt{2} \sigma_{clipped})
\end{align}

The error was thresholded at 1000 ml to avoid large errors adversely penalizing results, while the confidence values were clipped at 70 ml to reflect the approximate measurement uncertainty in FVC. The final score was calculated by averaging the metric across all (Patient, Week) pairs. Note that metric values will be negative and higher is better.

Next, we calculate the metric and RMSE:

```python
y = df.dropna()
rmse = ((y['FVC_pred'] - y['FVC_true']) ** 2).mean() ** (1/2)
print(f'RMSE: {rmse:.1f} ml')

sigma_c = y['sigma'].values
sigma_c[sigma_c < 70] = 70
delta = (y['FVC_pred'] - y['FVC_true']).abs()
delta[delta > 1000] = 1000
lll = - np.sqrt(2) * delta / sigma_c - np.log(np.sqrt(2) * sigma_c)
print(f'Laplace Log Likelihood: {lll.mean():.4f}')
```

    RMSE: 122.1 ml
    Laplace Log Likelihood: -6.1376

What do these numbers mean? It means if you adopted this approach, you would **outperform most of the public solutions** in the competition. Curiously, the vast majority of public solutions adopt a standard deterministic Neural Network, modelling uncertainty through a quantile loss. **Most of the people still adopt a frequentist approach**.

**Uncertainty** for single predictions becomes more and more important in machine learning and is often a requirement. **Especially when the consequenses of a wrong prediction are high**, we need to know what the probability distribution of an individual prediction is. For perspective, Kaggle just launched a new competition sponsored by Lyft, to build motion prediction models for self-driving vehicles. "We ask that you predict a few trajectories for every agent **and provide a confidence score for each of them**."

Finally, I hope the great work done by Pyro/NumPyro developers help democratize Bayesian methods, empowering an ever growing community of researchers and practitioners to create models that can not only generate predictions, but also assess uncertainty in their predictions.

## References

1. Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015). https://doi.org/10.1038/nature14541

2. Rainforth, Thomas William Gamlen. Automating Inference, Learning, and Design Using Probabilistic Programming. University of Oxford, 2017.


# Simulating gate noise on the Rigetti Quantum Virtual Machine
© Copyright 2017, Rigetti Computing.
$$
\newcommand{ket}[1]{\left|{#1}\right\rangle}
\newcommand{bra}[1]{\left\langle {#1}\right|}
\newcommand{tr}{\mathrm{Tr}}
$$

## Pure states vs. mixed states

Errors in quantum computing can introduce classical uncertainty in what the underlying state is.
When this happens we sometimes need to consider not only wavefunctions but also probabilistic sums of
wavefunctions when we are uncertain as to which one we have. For example, if we think that an X gate
was accidentally applied to a qubit with a 50-50 chance then we would say that there is a 50% chance
we have the $\ket{0}$ state and a 50% chance that we have a $\ket{1}$ state.
This is called an "impure" or
"mixed"state in that it isn't just a wavefunction (which is pure) but instead a distribution over
wavefunctions. We describe this with something called a density matrix, which is generally an
operator. Pure states have very simple density matrices that we can write as an outer product of a
ket vector $\ket{\psi}$ with its own bra version $\bra{\psi}=\ket{\psi}^\dagger$.
For a pure state the density matrix is simply

$$
   \rho_\psi = \ket{\psi}\bra{\psi}.
$$
The expectation value of an operator for a mixed state is given by

$$
   \langle X \rangle_\rho = \tr{X \rho}
$$
where $\tr{A}$ is the trace of an operator, which is the sum of its diagonal elements
which is independent of choice of basis.
Pure state density matrices satisfy

$$
   \rho \text{ is pure } \Leftrightarrow \rho^2 = \rho
$$
which you can easily verify for $\rho_\psi$ assuming that the state is normalized.
If we want to describe a situation with classical uncertainty between states $\rho_1$ and
$\rho_2$, then we can take their weighted sum
$$
   \rho = p \rho_1 + (1-p) \rho_2
$$
where $p\in [0,1]$ gives the classical probability that the state is $\rho_1$.

Note that classical uncertainty in the wavefunction is markedly different from superpositions.
We can represent superpositions using wavefunctions, but use density matrices to describe
distributions over wavefunctions. You can read more about density matrices [here](https://en.wikipedia.org/wiki/Density_matrix).

# Quantum gate errors
## What are they?
For a quantum gate given by its unitary operator $U$, a "quantum gate error" describes the scenario in which the actually induces transformation deviates from $\ket{\psi} \mapsto U\ket{\psi}$.
There are two basic types of quantum gate errors:

1. **coherent errors** are those that preserve the purity of the input state, i.e., instead of the above mapping we carry out a perturbed, but unitary operation $\ket{\psi} \mapsto \tilde{U}\ket{\psi}$, where $\tilde{U} \neq U$.
2. **incoherent errors** are those that do not preserve the purity of the input state,
    in this case we must actually represent the evolution in terms of density matrices.
    The state $\rho := \ket{\psi}\bra{\psi}$ is then mapped as
    $$
    \rho \mapsto \sum_{j=1}^n K_j\rho K_j^\dagger,
    $$
    where the operators $\{K_1, K_2, \dots, K_m\}$ are called Kraus operators and must obey
    $\sum_{j=1}^m K_j^\dagger K_j = I$ to conserve the trace of $\rho$.
    Maps expressed in the above form are called Kraus maps. It can be shown that every physical map on a finite
    dimensional quantum system can be represented as a Kraus map, though this representation is not generally unique.
    [You can find more information about quantum operations here](https://en.wikipedia.org/wiki/Quantum_operation#Kraus_operators)

In a way, coherent errors are *in principle* amendable by more precisely calibrated control. Incoherent errors are more tricky.

## Why do incoherent errors happen?
When a quantum system (e.g., the qubits on a quantum processor) is not perfectly isolated from its environment it generally co-evolves with the degrees of freedom it couples to. The implication is that while the total time evolution of system and environment can be assumed to be unitary, restriction to the system state generally is not.

**Let's throw some math at this for clarity:**
Let our total Hilbert space be given by the tensor product of system and environment Hilbert spaces:
$\mathcal{H} = \mathcal{H}_S \otimes \mathcal{H}_E$.
Our system "not being perfectly isolated" must be translated to the statement that the global Hamiltonian contains a contribution that couples the system and environment:
$$
H = H_S \otimes I + I \otimes H_E + V
$$
where $V$ non-trivally acts on both the system and the environment.
Consequently, even if we started in an initial state that factorized over system and environment $\ket{\psi}_{S,0}\otimes \ket{\psi}_{E,0}$
if everything evolves by the Schrödinger equation
$$
\ket{\psi_t} = e^{-i \frac{Ht}{\hbar}} \left(\ket{\psi}_{S,0}\otimes \ket{\psi}_{E,0}\right)
$$
the final state will generally not admit such a factorization.

## A toy model
**In this (somewhat technical) section we show how environment interaction can corrupt an identity gate and derive its Kraus map.**
For simplicity, let us assume that we are in a reference frame in which both the system and environment Hamiltonian's vanish $H_S = 0, H_E = 0$ and where the cross-coupling is small even when multiplied by the duration of the time evolution $\|\frac{tV}{\hbar}\|^2 \sim \epsilon \ll 1$ (any operator norm $\|\cdot\|$ will do here).
Let us further assume that $V = \sqrt{\epsilon} V_S \otimes V_E$ (the more general case is given by a sum of such terms) and that
the initial environment state satisfies $\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0$. This turns out to be a very reasonable assumption in practice but a more thorough discussion exceeds our scope.

Then the joint system + environment state $\rho = \rho_{S,0} \otimes \rho_{E,0}$ (now written as a density matrix) evolves as
$$
\rho \mapsto \rho' :=  e^{-i \frac{Vt}{\hbar}} \rho e^{+i \frac{Vt}{\hbar}}
$$
Using the Baker-Campbell-Hausdorff theorem we can expand this to second order in $\epsilon$
$$
\rho' = \rho - \frac{it}{\hbar} [V, \rho] - \frac{t^2}{2\hbar^2} [V, [V, \rho]] + O(\epsilon^{3/2})
$$
We can insert the initially factorizable state $\rho = \rho_{S,0} \otimes \rho_{E,0}$ and trace over the environmental degrees of freedom to obtain
\begin{align}
\rho_S' := \tr_E \rho' & = \rho_{S,0}  \underbrace{\tr \rho_{E,0}}_{1} - \frac{i\sqrt{\epsilon} t}{\hbar} \underbrace{\left[ V_S \rho_{S,0} \underbrace{\tr V_E\rho_{E,0}}_{\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0} - \rho_{S,0}V_S  \underbrace{\tr \rho_{E,0}V_E}_{\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0} \right]}_0 \\
& \qquad - \frac{\epsilon t^2}{2\hbar^2} \left[ V_S^2\rho_{S,0}\tr V_E^2 \rho_{E,0} + \rho_{S,0} V_S^2 \tr \rho_{E,0}V_E^2 - 2 V_S\rho_{S,0}V_S\tr V_E \rho_{E,0}V_E\right] \\
& = \rho_{S,0} - \frac{\gamma}{2} \left[ V_S^2\rho_{S,0} + \rho_{S,0} V_S^2  - 2 V_S\rho_{S,0}V_S\right]
\end{align}
where the coefficient in front of the second part is by our initial assumption very small $\gamma := \frac{\epsilon t^2}{2\hbar^2}\tr V_E^2 \rho_{E,0} \ll 1$.
This evolution happens to be approximately equal to a Kraus map with operators $K_1 := I - \frac{\gamma}{2} V_S^2, K_2:= \sqrt{\gamma} V_S$:
\begin{align}
\rho_S \to \rho_S' &= K_1\rho K_1^\dagger + K_2\rho K_2^\dagger
 = \rho - \frac{\gamma}{2}\left[ V_S^2 \rho + \rho V_S^2\right] + \gamma V_S\rho_S V_S + O(\gamma^2)
\end{align}
This agrees to $O(\epsilon^{3/2})$ with the result of our derivation above. This type of derivation can be extended to many other cases with little complication and a very similar argument is used to derive the [Lindblad master equation](https://en.wikipedia.org/wiki/Lindblad_equation).

# Support for noisy gates on the Rigetti QVM

As of today, users of our Forest API can annotate their QUIL programs by certain pragma statements that inform the QVM that a particular gate on specific target qubits should be replaced by an imperfect realization given by a Kraus map.

## But the QVM propagates *pure states*: How does it simulate noisy gates?
It does so by yielding the correct outcomes **in the average over many executions of the QUIL program**:
When the noisy version of a gate should be applied the QVM makes a random choice which Kraus operator is applied to the current state with a probability that ensures that the average over many executions is equivalent to the Kraus map.
In particular, a particular Kraus operator $K_j$ is applied to $\ket{\psi}_S$
$$
\ket{\psi'}_S = \frac{1}{\sqrt{p_j}} K_j \ket{\psi}_S
$$
with probability $p_j:= \bra{\psi}_S K_j^\dagger K_j \ket{\psi}_S$.
In the average over many execution $N \gg 1$ we therefore find that
\begin{align}
\overline{\rho_S'} & = \frac{1}{N} \sum_{n=1}^N \ket{\psi'_n}_S\bra{\psi'_n}_S \\
& = \frac{1}{N} \sum_{n=1}^N p_{j_n}^{-1}K_{j_n}\ket{\psi'}_S \bra{\psi'}_SK_{j_n}^\dagger
\end{align}
where $j_n$ is the chosen Kraus operator label in the $n$-th trial.
This is clearly a Kraus map itself! And we can group identical terms and rewrite it as
\begin{align}
\overline{\rho_S'} & =
  \sum_{\ell=1}^n \frac{N_\ell}{N}  p_{\ell}^{-1}K_{\ell}\ket{\psi'}_S \bra{\psi'}_SK_{\ell}^\dagger
\end{align}
where $N_{\ell}$ is the number of times that Kraus operator label $\ell$ was selected.
For large enough $N$ we know that $N_{\ell} \approx N p_\ell$ and therefore
\begin{align}
\overline{\rho_S'} \approx \sum_{\ell=1}^n K_{\ell}\ket{\psi'}_S \bra{\psi'}_SK_{\ell}^\dagger
\end{align}
which proves our claim.
**The consequence is that noisy gate simulations must generally be repeated many times to obtain representative results**.

## How do I get started?

1. Come up with a good model for your noise. We will provide some examples below and may add more such
    examples to our public repositories over time. Alternatively, you can characterize the gate under
    consideration using [Quantum Process Tomography](https://arxiv.org/abs/1202.5344) or
    [Gate Set Tomography](http://www.pygsti.info/) and use the resulting process matrices to obtain a
    very accurate noise model for a particular QPU.
2. Define your Kraus operators as a list of numpy arrays `kraus_ops = [K1, K2, ..., Km]`.
3. For your QUIL program `p`, call:
    ```
    p.define_noisy_gate("MY_NOISY_GATE", [q1, q2], kraus_ops)
    ```
    where you should replace `MY_NOISY_GATE` with the gate of interest and `q1, q2` the indices of the qubits.

**Scroll down for some examples!**

```python
from __future__ import print_function, division
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom
import matplotlib.colors as colors
%matplotlib inline
```

```python
from pyquil.quil import Program, MEASURE
from pyquil.api.qvm import QVMConnection
from pyquil.job_results import wait_for_job
from pyquil.gates import CZ, H, I, X
from scipy.linalg import expm
```

```python
cxn = QVMConnection()
```

# Example 1: Amplitude damping

Amplitude damping channels are imperfect identity maps with Kraus operators
$$
K_1 = \begin{pmatrix}
1 & 0 \\
0 & \sqrt{1-p}
\end{pmatrix} \\
K_2 = \begin{pmatrix}
0 & \sqrt{p} \\
0 & 0
\end{pmatrix}
$$
where $p$ is the probability that a qubit in the $\ket{1}$ state decays to the $\ket{0}$ state.

```python
def damping_channel(damp_prob=.1):
    """
    Generate the Kraus operators corresponding to an amplitude damping
    noise channel.

    :params float damp_prob: The one-step damping probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    damping_op = np.sqrt(damp_prob) * np.array([[0, 1],
                                                [0, 0]])

    residual_kraus = np.diag([1, np.sqrt(1-damp_prob)])
    return [residual_kraus, damping_op]

def append_kraus_to_gate(kraus_ops, g):
    """
    Follow a gate `g` by a Kraus map described by `kraus_ops`.

    :param list kraus_ops: The Kraus operators.
    :param numpy.ndarray g: The unitary gate.
    :return: A list of transformed Kraus operators.
    """
    return [kj.dot(g) for kj in kraus_ops]

def append_damping_to_gate(gate, damp_prob=.1):
    """
    Generate the Kraus operators corresponding to a given unitary
    single qubit gate followed by an amplitude damping noise channel.

    :params np.ndarray|list gate: The 2x2 unitary gate matrix.
    :params float damp_prob: The one-step damping probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    return append_kraus_to_gate(damping_channel(damp_prob), gate)
```

```python
%%time

# single step damping probability
damping_per_I = 0.02

# number of program executions
trials = 200

results_damping = []
lengths = np.arange(0, 201, 10, dtype=int)
for jj, num_I in enumerate(lengths):

    print("{}/{}, ".format(jj, len(lengths)), end="")

    p = Program(X(0))
    # want increasing number of I-gates
    p.inst([I(0) for _ in range(num_I)])
    p.inst(MEASURE(0, [0]))

    # overload identity I on qc 0
    p.define_noisy_gate("I", [0], append_damping_to_gate(np.eye(2), damping_per_I))
    cxn.random_seed = int(num_I)
    res = cxn.run(p, [0], trials=trials)
    results_damping.append([np.mean(res), np.std(res) / np.sqrt(trials)])

results_damping = np.array(results_damping)
```

    0/21, 1/21, 2/21, 3/21, 4/21, 5/21, 6/21, 7/21, 8/21, 9/21, 10/21, 11/21, 12/21, 13/21, 14/21, 15/21, 16/21, 17/21, 18/21, 19/21, 20/21, CPU times: user 168 ms, sys: 36.3 ms, total: 204 ms
    Wall time: 3.29 s

```python
dense_lengths = np.arange(0, lengths.max()+1, .2)
survival_probs = (1-damping_per_I)**dense_lengths
logpmf = binom.logpmf(np.arange(trials+1)[np.newaxis, :], trials, survival_probs[:, np.newaxis])/np.log(10)
```

```python
DARK_TEAL = '#48737F'
FUSCHIA = "#D6619E"
BEIGE = '#EAE8C6'
cm = colors.LinearSegmentedColormap.from_list('anglemap', ["white", FUSCHIA, BEIGE], N=256, gamma=1.5)
```

```python
plt.figure(figsize=(14, 6))
plt.pcolor(dense_lengths, np.arange(trials+1)/trials, logpmf.T, cmap=cm, vmin=-4, vmax=logpmf.max())
plt.plot(dense_lengths, survival_probs, c=BEIGE, label="Expected mean")
plt.errorbar(lengths, results_damping[:,0], yerr=2*results_damping[:,1], c=DARK_TEAL,
             label=r"noisy qvm, errorbars $ = \pm 2\hat{\sigma}$", marker="o")
cb = plt.colorbar()
cb.set_label(r"$\log_{10} \mathrm{Pr}(n_1; n_{\rm trials}, p_{\rm survival}(t))$", size=20)

plt.title("Amplitude damping model of a single qubit", size=20)
plt.xlabel(r"Time $t$ [arb. units]", size=14)
plt.ylabel(r"$n_1/n_{\rm trials}$", size=14)
plt.legend(loc="best", fontsize=18)
plt.xlim(*lengths[[0, -1]])
plt.ylim(0, 1)
```

# Example 2: dephased CZ-gate

Dephasing is usually characterized through a qubit's $T_2$ time.
For a single qubit the dephasing Kraus operators are
$$
K_1(p) = \sqrt{1-p} I_2 \\
K_2(p) = \sqrt{p} \sigma_Z
$$
where $p = 1 - \exp(-T_2/T_{\rm gate})$ is the probability that the qubit is dephased over the time interval of interest, $I_2$ is the $2\times 2$-identity matrix and $\sigma_Z$ is the Pauli-Z operator.

For two qubits, we must construct a Kraus map that has *four* different outcomes:

1. No dephasing
2. Qubit 1 dephases
3. Qubit 2 dephases
4. Both dephase

The Kraus operators for this are given by
\begin{align}
K'_1(p,q) = K_1(p)\otimes K_1(q) \\
K'_2(p,q) = K_2(p)\otimes K_1(q) \\
K'_3(p,q) = K_1(p)\otimes K_2(q) \\
K'_4(p,q) = K_2(p)\otimes K_2(q)
\end{align}
where we assumed a dephasing probability $p$ for the first qubit and $q$ for the second.

Dephasing is a *diagonal* error channel and the CZ gate is also diagonal, therefore we can get the combined map of dephasing and the CZ gate simply by composing $U_{\rm CZ}$ the unitary representation of CZ with each Kraus operator
\begin{align}
K^{\rm CZ}_1(p,q) = K_1(p)\otimes K_1(q)U_{\rm CZ} \\
K^{\rm CZ}_2(p,q) = K_2(p)\otimes K_1(q)U_{\rm CZ} \\
K^{\rm CZ}_3(p,q) = K_1(p)\otimes K_2(q)U_{\rm CZ} \\
K^{\rm CZ}_4(p,q) = K_2(p)\otimes K_2(q)U_{\rm CZ}
\end{align}

**Note that this is not always accurate, because a CZ gate is often achieved through non-diagonal interaction Hamiltonians! However, for sufficiently small dephasing probabilities it should always provide a good starting point.**

```python
def dephasing_kraus_map(p=.1):
    """
    Generate the Kraus operators corresponding to a dephasing channel.

    :params float p: The one-step dephasing probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    return [np.sqrt(1-p)*np.eye(2), np.sqrt(p)*np.diag([1, -1])]

def tensor_kraus_maps(k1, k2):
    """
    Generate the Kraus map corresponding to the composition
    of two maps on different qubits.

    :param list k1: The Kraus operators for the first qubit.
    :param list k2: The Kraus operators for the second qubit.
    :return: A list of tensored Kraus operators.
    """
    return [np.kron(k1j, k2l) for k1j in k1 for k2l in k2]

```

```python
%%time
# single step damping probabilities
ps = np.linspace(.001, .5, 200)

# number of program executions
trials = 500

results = []

for jj, p in enumerate(ps):

    corrupted_CZ = append_kraus_to_gate(
    tensor_kraus_maps(
        dephasing_kraus_map(p),
        dephasing_kraus_map(p)
    ),
    np.diag([1, 1, 1, -1]))

    print("{}/{}, ".format(jj, len(ps)), end="")

    # make Bell-state
    p = Program(H(0), H(1), CZ(0,1), H(1))

    p.inst(MEASURE(0, [0]))
    p.inst(MEASURE(1, [1]))

    # overload identity I on qc 0
    p.define_noisy_gate("CZ", [0, 1], corrupted_CZ)
    cxn.random_seed = jj
    res = cxn.run(p, [0, 1], trials=trials)
    results.append(res)

results = np.array(results)
```

    0/200, 1/200, 2/200, 3/200, 4/200, 5/200, 6/200, 7/200, 8/200, 9/200, 10/200, 11/200, 12/200, 13/200, 14/200, 15/200, 16/200, 17/200, 18/200, 19/200, 20/200, 21/200, 22/200, 23/200, 24/200, 25/200, 26/200, 27/200, 28/200, 29/200, 30/200, 31/200, 32/200, 33/200, 34/200, 35/200, 36/200, 37/200, 38/200, 39/200, 40/200, 41/200, 42/200, 43/200, 44/200, 45/200, 46/200, 47/200, 48/200, 49/200, 50/200, 51/200, 52/200, 53/200, 54/200, 55/200, 56/200, 57/200, 58/200, 59/200, 60/200, 61/200, 62/200, 63/200, 64/200, 65/200, 66/200, 67/200, 68/200, 69/200, 70/200, 71/200, 72/200, 73/200, 74/200, 75/200, 76/200, 77/200, 78/200, 79/200, 80/200, 81/200, 82/200, 83/200, 84/200, 85/200, 86/200, 87/200, 88/200, 89/200, 90/200, 91/200, 92/200, 93/200, 94/200, 95/200, 96/200, 97/200, 98/200, 99/200, 100/200, 101/200, 102/200, 103/200, 104/200, 105/200, 106/200, 107/200, 108/200, 109/200, 110/200, 111/200, 112/200, 113/200, 114/200, 115/200, 116/200, 117/200, 118/200, 119/200, 120/200, 121/200, 122/200, 123/200, 124/200, 125/200, 126/200, 127/200, 128/200, 129/200, 130/200, 131/200, 132/200, 133/200, 134/200, 135/200, 136/200, 137/200, 138/200, 139/200, 140/200, 141/200, 142/200, 143/200, 144/200, 145/200, 146/200, 147/200, 148/200, 149/200, 150/200, 151/200, 152/200, 153/200, 154/200, 155/200, 156/200, 157/200, 158/200, 159/200, 160/200, 161/200, 162/200, 163/200, 164/200, 165/200, 166/200, 167/200, 168/200, 169/200, 170/200, 171/200, 172/200, 173/200, 174/200, 175/200, 176/200, 177/200, 178/200, 179/200, 180/200, 181/200, 182/200, 183/200, 184/200, 185/200, 186/200, 187/200, 188/200, 189/200, 190/200, 191/200, 192/200, 193/200, 194/200, 195/200, 196/200, 197/200, 198/200, 199/200, CPU times: user 1.37 s, sys: 187 ms, total: 1.55 s
    Wall time: 39.3 s

```python
Z1s = (2*results[:,:,0]-1.)
Z2s = (2*results[:,:,1]-1.)
Z1Z2s = Z1s * Z2s

Z1m = np.mean(Z1s, axis=1)
Z2m = np.mean(Z2s, axis=1)
Z1Z2m = np.mean(Z1Z2s, axis=1)
```

```python
plt.figure(figsize=(14, 6))
plt.axhline(y=1.0, color=FUSCHIA, alpha=.5, label="Bell state")

plt.plot(ps, Z1Z2m, "x", c=FUSCHIA, label=r"$\overline{Z_1 Z_2}$")
plt.plot(ps, 1-2*ps, "--", c=FUSCHIA, label=r"$\langle Z_1 Z_2\rangle_{\rm theory}$")

plt.plot(ps, Z1m, "o", c=DARK_TEAL, label=r"$\overline{Z}_1$")
plt.plot(ps, 0*ps, "--", c=DARK_TEAL, label=r"$\langle Z_1\rangle_{\rm theory}$")

plt.plot(ps, Z2m, "d", c="k", label=r"$\overline{Z}_2$")
plt.plot(ps, 0*ps, "--", c="k", label=r"$\langle Z_2\rangle_{\rm theory}$")

plt.xlabel(r"Dephasing probability $p$", size=18)
plt.ylabel(r"$Z$-moment", size=18)
plt.title(r"$Z$-moments for a Bell-state prepared with dephased CZ", size=18)
plt.xlim(0, .5)
plt.legend(fontsize=18)
```


# Variational Linear Systems via QAOA

Ryan LaRose, Yigit Subasi, Lukasz Cincio, Patrick Coles

### Abstract

In this notebook, we implement an approach to solving linear systems on a NISQ computer. The approach uses QAOA [1] with the "linear system Hamiltonian" given in [2]. Other approaches for the "quantum linear systems problem" [3-5] require resources beyond current hardware capabilities. This method can be implemented on NISQ processors. Other "near-term approaches" have recently emerged in the literature [6-7].

```python
"""Imports for the notebook.

Requires:
    Matplotlib
    Numpy
    Scipy
    Cirq v0.5.0
"""
from time import time

import numpy as np
import matplotlib.pyplot as plt
from scipy.linalg import expm
from scipy.optimize import minimize

import cirq
assert cirq.__version__ == "0.5.0", "Wrong version of Cirq! Code may or may not work."
```

```python
"""Flags for testing behavior."""
# Option to use identity matrix if true
DEBUG = False

# Option to scale gamma parameters by matrix condition number if true
SCALE = False
```

## Preliminaries

We seek to solve the "quantum linear systems problem" (QLSP) defined by the system

\begin{equation}
    A \mathbf{x} = \mathbf{b}
\end{equation}

where $A \in \mathbb{R}^{N \times N}$ and $N = 2^n$. QLSP is similar to the classical linear systems problem except we only want to output a "quantum description" of the solution

\begin{equation}
    | \mathbf{x} \rangle = |A^{-1} \mathbf{b} \rangle
\end{equation}

(because we don't have a good way around this).

#### Challenge: Find a (practical) problem where we only want some expectation of the solution $\langle \mathbf{x} | \hat{O} | \mathbf{x} \rangle$ for some Hermitian operator $\hat{O}$.

## Ansatz wavefunction

_Note: This presentation is based off the slides Yigit presented at SQUINT 2019._

We follow the method outlined in [2] to construct our ansatz wavefunction. First, define the projector

\begin{equation}
    P_\mathbf{b}^\perp := I - |\mathbf{b}\rangle \langle \mathbf{b}| .
\end{equation}

Since $P_\mathbf{b}^\perp |\mathbf{b} \rangle = \mathbf{0}$, we have

\begin{equation}
    P_\mathbf{b}^\perp A | \mathbf{x} \rangle = \mathbf{0} .
\end{equation}

Define $B := P_\mathbf{b}^\perp A$ and the operator

\begin{equation}
    H_f := B^\dagger B = A P_\mathbf{b}^\perp A .
\end{equation}

The subscript $f$ will become apparent shortly. This operator (Hamiltonian) is:

* Hermitian.
* positive-semidefinite $\implies$ $|\mathbf{x}\rangle$ is the ground state.

Note that $|\mathbf{x}\rangle$ is the _unique_ ground state since $P_\mathbf{b}^\perp$ is an $N - 1$ dimensional projector.

We now parameterize the linear system $A$ with the schedule

\begin{equation}
    A \mapsto A(t) := (1 - t) I + t A
\end{equation}

for $0 \le t \le 1$ and similarly the Hamiltonian

\begin{equation}
        H(t) := A(t) P_\mathbf{b}^\perp A(t)  .
\end{equation}

Note that $H(t=1) = H_f$.

This defines our "driver Hamiltonian" in QAOA, i.e. the unitary operator

\begin{equation}
    U_t(\gamma) = e^{ - i \gamma H(t) }
\end{equation}

#### Question: Normally, the "cost Hamiltonian" in QAOA is assumed to be diagonal in the computational basis. In general, our Hamiltonian is not. Do we need a separate "mixer Hamiltonian" in this case?

#### Note: QAOA typically assumes the driver Hamiltonian is time-independent (so far as I know), but our Hamiltonian is time-dependent.

We will, for now, take the standard mixer Hamiltonian

\begin{equation}
    V(\beta) = \prod_j e^{ - i \beta X_j} ,
\end{equation}

i.e., a rotation of each qubit about the $x$-axis by $2 \beta$.

The ansatz wavefunction is thus

\begin{equation}
    |\mathbf{\gamma}, \mathbf{\beta} \rangle := \left[ \prod_{1}^{p} U_{t_i} (\gamma_i) V(\beta_i) \right] H^{\otimes n} |0\rangle^{\otimes n}
\end{equation}

where $n := \log_2 N$.

# Code Implementation

We now turn to a implementation of this algorithm using Cirq.

## Example Problem

In the cells below we'll generate a linear system of equations.

```python
"""Seed the random number generator."""
SEED = 29876
np.random.seed(SEED)
```

```python
"""Generate a system of equations to solve."""
# Dimension of system. Assume square for now
# Note: Studying different systems is (unfortunately) not as
# simple as changing n, since we defined the driver Hamiltonian
# as a cirq.TwoQubitGate
# TODO: Can this be changed to cirq.Gate?
n = 2
N = 2**n

if DEBUG:
    A = np.identity(4)
else:
    # Generate a random matrix
    A = np.random.rand(N, N)

    # Make sure it's Hermitian
    A = A + A.conj().T

    # Normalize it WLOG. Note this defaults to the Froebenius norm for matrices.
    A /= np.linalg.norm(A)

    # Control the condition number
    k = 100
    A = np.diag([1, 1 / k, 1, 1 / k])

# Compute the condition number of the matrix to scale exponentials
if SCALE:
    kappa = np.linalg.cond(A)
else:
    kappa = 1.0

if DEBUG:
    b = np.array([1.0, 0.0, 0.0, 0.0], dtype=float)
else:
    # Generate the vector
    b = np.random.rand(N)

    # Normalize it WLOG. Note this defaults to the l2-norm for vectors.
    b /= np.linalg.norm(b)

    # Use the |+> state
    b = 1/2 * np.ones(4)

# Do the solution classically
start = time()
x = np.linalg.solve(A, b)
total = time() - start

# Display the system, solution, and time to solve
print("Classically solving the system Ax = b...")
print("A =\n", A)
print("b =\n", b)
print()
print("Solution found in", total, "seconds.")
print("x =\n", x)
print("Ax =\n", np.dot(A, x))
```

    Classically solving the system Ax = b...
    A =
     [[1.   0.   0.   0.  ]
     [0.   0.01 0.   0.  ]
     [0.   0.   1.   0.  ]
     [0.   0.   0.   0.01]]
    b =
     [0.5 0.5 0.5 0.5]

    Solution found in 0.00021839141845703125 seconds.
    x =
     [ 0.5 50.   0.5 50. ]
    Ax =
     [0.5 0.5 0.5 0.5]

## Building up the ansatz

Now we'll build up the ansatz. We'll "cheat" a little by

1. Allowing for matrix gates (to avoid compiling).
1. Classically exponentiating the system (to avoid Trotterization).

We'll use the "half-way point" Hamiltonian. That is, setting $t = 0.5$ in the Hamiltonian $H(t)$.

```python
"""Get the matrix for the driver Hamiltonian."""
# Set the time for the Hamiltonian
t = 0.5

# Identity matrix, for convenince
iden = np.identity(N)

# Get the "b^\perp projector"
Pb = iden - np.outer(b, b)

# Get the parameterized linear system
At = (1.0 - t) * iden + t * A

# Get the Hamiltonian
Ht = At @ Pb @ At

# Make sure it's Hermitian
assert np.allclose(Ht, Ht.conj().T)
```

This gives us a matrix representation of the $U_t(\gamma)$ unitary -- except without the $\gamma$ parameter. We'll take care of this when we construct a matrix gate in Cirq by raising the gate to the power $\gamma$. We now construct a gate from this unitary that we can use in a circuit.

```python
"""Create a gate for the driver Hamiltonian (unitary)."""
class Driver(cirq.TwoQubitGate):
    """Gate for the U_t(\gamma) driver Hamiltonian (for a particular t)."""

    def __init__(self, H, gamma):
        self.gamma = gamma
        self.matrix = expm(-1j * kappa * gamma * H)
        assert np.allclose(self.matrix @ self.matrix.conj().T, iden)

    def _unitary_(self):
        return self.matrix

    def _circuit_diagram_info_(self, args):
        return "Driver^{}".format(self.gamma), "Driver^{}".format(self.gamma)
```

Now we'll make sure we can implement this in a quantum circuit.

```python
"""Function for building the ansatz circuit."""
def circuit(gamma, beta):
    """Returns the state |gamma, beta> defined above."""
    # Get a qubit register
    qreg = cirq.LineQubit.range(n)

    # Get a quantum circuit
    circ = cirq.Circuit()

    # How about a round of Hadamards?
    circ.append(cirq.H.on_each(*qreg))

    # Do the driver Hamiltonian
    U = Driver(Ht, gamma)
    circ.append(U.on(*qreg))

    # Do the mixer Hamiltonian
    circ.append(cirq.Rx(kappa * beta).on_each(*qreg))

    return circ
```

```python
"""Make sure the code above is working properly."""
# Display the circuit
circ = circuit(0, 0)
print("Circuit:")
print(circ)

# Make sure it can be simulated
sim = cirq.Simulator()
res = sim.simulate(circ)

# Display the final state
print("\nFinal state:")
print("|psi> = ", res.dirac_notation())
```

    Circuit:
    0: ───H───Driver^0───Rx(0.0π)───
              │
    1: ───H───Driver^0───Rx(0.0π)───

    Final state:
    |psi> =  0.5|00⟩ + 0.5|01⟩ + 0.5|10⟩ + 0.5|11⟩

## Computing the Expectation Value

To compute the expectation value (cost), we use the final Hamiltonian $H_f$:

\begin{equation}
    C(\mathbf{\gamma}, \mathbf{\beta}) =
        \langle \mathbf{\gamma}, \mathbf{\beta} | H_f | \mathbf{\gamma}, \mathbf{\beta} \rangle
\end{equation}

It is this quantity we minimize during the optimization procedure.

```python
"""Function for computing the cost."""
def cost(gamma, beta, simulator=cirq.Simulator()):
    """Returns the cost C(\gamma, \beta) defined above."""
    # Build the circuit
    circ = circuit(gamma, beta)

    # Get the final state
    psi = sim.simulate(circ).final_state

    # Return the expectation <\gamma, \beta| Hf | \gamma, \beta>
    Hf = A @ Pb @ A
    return abs(np.dot(psi.conj().T, np.dot(Hf, psi)))
```

## Minimizing the cost

We now minimize the cost. For simplicity, we use built-in optimizers from Scipy. Alternatively, a dynamic grid search or analytic-gradient methods could be used.

```python
"""Function that returns the objective and inputs an array (for Scipy)."""
# Functional form for scipy.optimize.minimize
def obj(x):
    val = cost(x[0], x[1])
    print("Current cost:", val, end="\r")
    return val
```

```python
"""Do the minimization and time it."""
start = time()
out = minimize(obj, np.random.rand(2), method="Powell", tol=1e-10)
total = time() - start
```

    Current cost: 0.0097207232108968440.009998400078796635

```python
print(out["x"])
```

    [5.00683180e+00 9.00587972e-05]

## Doing a grid search over the landscape

Here we visualize the cost landscape by evaluating the cost on a grid.

```python
"""Do a grid search over all angles."""
# Define ranges of parameters to search over
gammas = np.linspace(-np.pi, np.pi, 100)
betas = np.linspace(-np.pi, np.pi, 100)

# Initialize an array to store the costs at each point
costs = np.zeros((100, 100))

# Evaluate the cost at each point and store it
for ii, gamma in enumerate(gammas):
    for jj, beta in enumerate(betas):
        costs[ii, jj] = obj([gamma, beta])
```

    Current cost: 0.4134547543234798440.176811040322603470.31177489845282325 0.173510514740655820.11327110891036668

```python
"""Remind us of the system and condition number."""
print("A = ", A)
print("b = ", b)
print("kappa = ", np.linalg.cond(A))
```

    A =  [[1.   0.   0.   0.  ]
     [0.   0.01 0.   0.  ]
     [0.   0.   1.   0.  ]
     [0.   0.   0.   0.01]]
    b =  [0.5 0.5 0.5 0.5]
    kappa =  100.0

We can now make a 2D plot of the landscape.

```python
"""Plot the cost landscape."""
%matplotlib inline
plt.imshow(costs, origin="lower")
plt.colorbar()
if SCALE:
    plt.title("Scaled First Order QAOA Cost Landscape")
else:
    plt.title("Unscaled First Order QAOA Cost Landscape")

#plt.savefig("kapp100-scaled", format="pdf")
```

```python
"""See the quantum solution."""
# Get the wavefunction at the best angles
gamma_opt, beta_opt = out["x"]
circ = circuit(gamma_opt, beta_opt)
xquantum = sim.simulate(circ).final_state

# Display the system, solution, and time to solve
print("Quantumly solving the system Ax = b...")
print("A =\n", A)
print("b =\n", b)
print()
print("Quantum solution found in", total, "seconds.")
print("xquantum =\n", xquantum)
print("xclassical =\n", x)
print("Ax =\n", np.dot(A, xquantum))
print("Overlap with classical solution:",
      abs(np.dot(xquantum, x))**2 / abs(np.dot(x, x)))
```

    Quantumly solving the system Ax = b...
    A =
     [[1.   0.   0.   0.  ]
     [0.   0.01 0.   0.  ]
     [0.   0.   1.   0.  ]
     [0.   0.   0.   0.01]]
    b =
     [0.5 0.5 0.5 0.5]

    Quantum solution found in 2.9893102645874023 seconds.
    xquantum =
     [0.10558552+1.44229925e-05j 0.69917923-6.18084669e-05j
     0.10558552+1.44229925e-05j 0.69917923-6.18084669e-05j]
    xclassical =
     [ 0.5 50.   0.5 50. ]
    Ax =
     [0.10558552+1.44229925e-05j 0.00699179-6.18084669e-07j
     0.10558552+1.44229925e-05j 0.00699179-6.18084669e-07j]
    Overlap with classical solution: 0.9805603065821096

## Higher order QAOA

We now go beyond one layer $p > 1$ in the QAOA ansatz. This is just a slight modification of the code above.

```python
"""Circuit for higher order QAOA."""
def qaoa(gammas, betas):
    """Returns the state |gamma, beta> defined above."""
    # Make sure we have the correct number of parameters
    if len(gammas) != len(betas):
        raise ValueError("gammas and betas must have the same length.")

    # Get a qubit register
    qreg = cirq.LineQubit.range(n)

    # Get a quantum circuit
    circ = cirq.Circuit()

    # How about a round of Hadamards?
    circ.append(cirq.H.on_each(*qreg))

    # Do each layer of QAOA
    for gamma, beta in zip(gammas, betas):
        # Do the driver Hamiltonian
        U = Driver(Ht, gamma)
        circ.append(U.on(*qreg))

        # Do the mixer Hamiltonian
        circ.append(cirq.Rx(beta).on_each(*qreg))

    return circ
```

Let's test this to make sure it works properly.

```python
"""Testing the higher order QAOA circuit."""
gammas = [0, 0, 0]
betas = [0.0, 0.0, 0.0]

circ = qaoa(gammas, betas)

print("Circuit:")
print(circ)

# Make sure it can be simulated
sim = cirq.Simulator()
res = sim.simulate(circ)

# Display the final state
print("\nFinal state:")
print("|psi> = ", *res.final_state)
```

    Circuit:
    0: ───H───Driver^0───Rx(0.0π)───Driver^0───Rx(0.0π)───Driver^0───Rx(0.0π)───
              │                     │                     │
    1: ───H───Driver^0───Rx(0.0π)───Driver^0───Rx(0.0π)───Driver^0───Rx(0.0π)───

    Final state:
    |psi> =  (0.49999997+0j) (0.49999997+0j) (0.49999997+0j) (0.49999997+0j)

Now we modify the cost function.

```python
"""Defining the cost function for higher order QAOA."""
def expectation(gammas, betas, simulator=cirq.Simulator()):
    """Returns the cost C(\gamma, \beta) defined above."""
    # Build the circuit
    circ = qaoa(gammas, betas)

    # Get the final state
    psi = sim.simulate(circ).final_state

    # Return the expectation <\gamma, \beta| Hf |\gamma, \beta>
    Hf = A.conj().T @ Pb @ A
    return abs(np.dot(psi.conj().T, np.dot(Hf, psi)))
```

And now test it.

```python
"""Testing the cost function for higher order QAOA."""
print(expectation([1, 2], [3, 4]))
```

    0.1416514988247451

And finally minimization for higher order QAOA.

```python
"""Defining the objective function for optimization with multiple layers."""
def objective(x):
    """Returns the objective function for higher order QAOA."""
    # Make sure the number of parameters is valid
    if len(x) % 2 != 0:
        raise ValueError("Invalid number of parameters. Must be even.")

    # Parse the parameters (arbitrary convention)
    gammas = x[:len(x) // 2]
    betas = x[len(x) // 2:]

    # Compute the expectation for these parameters
    cval = expectation(gammas, betas)

    # Print it out
    print("Current cost:", cval, end="\r")

    return cval
```

## Higher order QAOA

Now we look at performing the optimization.

```python
"""Set the number of layers p for QAOA."""
p = 2
```

```python
"""Optimizing higher order QAOA."""
# Do the minimization
start = time()
out = minimize(objective, np.random.rand(2 * p), method="Powell")
total = time() - start
```

    Current cost: 1.7099391798091803e-16

```python
"""See the quantum solution."""
# Get the wavefunction at the best angles
opt_params = out["x"]
gammas = opt_params[:len(opt_params) // 2]
betas = opt_params[len(opt_params) // 2:]
circ = qaoa(gammas, betas)
xquantum = sim.simulate(circ).final_state

# Display the system, solution, and time to solve
print("Quantumly solving the system Ax = b...")
print("A =\n", A)
print("b =\n", b)
print()
print("Solution found in", total, "seconds.")
print("xquantum =\n", xquantum)
print("xclassical =\n", x / np.linalg.norm(x))
print("Ax =\n", np.dot(A, xquantum))
print("Overlap with classical solution:",
      abs(np.dot(xquantum, x))**2 / abs(np.dot(x, x)))
```

    Quantumly solving the system Ax = b...
    A =
     [[1.   0.   0.   0.  ]
     [0.   0.01 0.   0.  ]
     [0.   0.   1.   0.  ]
     [0.   0.   0.   0.01]]
    b =
     [0.5 0.5 0.5 0.5]

    Solution found in 1.31510591506958 seconds.
    xquantum =
     [0.00473688+0.00524945j 0.4736894 +0.5249459j  0.00473689+0.00524945j
     0.47368944+0.524946j  ]
    xclassical =
     [0.00707071 0.70707143 0.00707071 0.70707143]
    Ax =
     [0.00473688+0.00524945j 0.00473689+0.00524946j 0.00473689+0.00524945j
     0.00473689+0.00524946j]
    Overlap with classical solution: 0.9999998160642839

# References

[1] Edward Farhi, Jeffrey Goldstone, Sam Gutmann, A quantum approximate optimization algorithm. https://arxiv.org/abs/1411.4028

[2] Yigit Subasi, Rolando D. Somma, Davide Orsucci, Quantum algorithms for systems of linear equations inspired by adiabatic quantum computing. https://arxiv.org/abs/1805.10549.

[3] A. W. Harrow, A. Hassidim, and S. Lloyd, “Quantum algorithm for solving linear systems of equations,” Physical Review Letters, vol. 103, no. 15, Oct. 2009.

[4] L. Wossnig, Z. Zhao, and A. Prakash, “A quantum linear system algorithm for dense matrices,” Phys. Rev. Lett., vol. 120, no. 5, p. 050502, Jan. 2018.

[5] D. Dervovic, M. Herbster, P. Mountney, S. Severini, N. Usher, and L. Wossnig, “Quantum linear systems algorithms: a primer,” arXiv:1802.08227 [quant-ph], Feb. 2018.

[6] NISQLSP I.

[7] NISQLSP II.


# Variance estimators

## Outline

Consider a particle estimate computed at time $t$ of a SMC algorithm:
$$\mathbb{Q}_t^N(\varphi) = \sum_{n=1}^N W_t^n \varphi(X_t^n).$$

A basic way to evaluate its variability is to run the algorithm many times, and report the empirical variance. Which is of course expensive. (Note however that function `multiSMC` in the `core` module makes it possible to run SMC algorithms in parallel on multiple-core machines).

Several recent papers (Chan and Lai, 2013; Lee and Whiteley, 2018; Olsson and Douc, 2019) have proposed estimators of the variance of a particle estimate that may be computed from a **single run**. These estimators require to track the genealogy of the particle system. In fact, the expression of these estimators is a variation of this:
\begin{equation}
\sum_{n=1}^N \left[\sum_{m:B_t^m=n} W_t^m \left\{\varphi(X_t^m) - \mathbb{Q}_t^N(\varphi)\right\} \right]^2
\end{equation}
where $B_t^n$ is the index of the ancestor of $X_t^n$ at time $0$ (the so-called **eve variables** in Lee and Whiteley, 2018). Note that this quantity **equals zero** as soon as all the particles have the same ancestor at time $0$. More generally, we expect these estimators to suffer from **path degeneracy** (i.e. number of distinct ancestors drops quicky.)

This notebook:

* explains how to compute such estimators, using module `variance_estimators`;
* showcases their performance in a toy example.

## Computing the variance estimators

We start with the usual imports.

```python
import itertools as it

from matplotlib import pyplot as plt
import numpy as np
from numpy import random
import pandas as pd
from scipy import stats
import seaborn as sb

import particles
from particles import state_space_models as ssms
from particles import kalman  # Linear Gaussian state space models are defined here
from particles import collectors as col  # standard collectors

plt.style.use('ggplot')
```

We consider a basic univariate linear Gaussian model:

\begin{align*}
X_t & = \rho X_{t-1} + \sigma_X U_t \\
Y_t & = X_t + \sigma_Y V_t
\end{align*}
with $\rho=0.9$, $\sigma_X=1$, $\sigma_Y=0.2$.

We simulate data ($T=50)$ from the model.

Let's run a single bootstrap filter, and have a look at the genealogical tree.

```python
T = 50
ssm = kalman.LinearGauss(rho=0.9, sigmaX=1., sigmaY=0.2)
true_x, data = ssm.simulate(T)
fk = ssms.Bootstrap(ssm=ssm, data=data)
N = 1000
alg = particles.SMC(fk=fk, N=N, store_history=True) # store_history: keeps the complete history
alg.run()

B = alg.hist.compute_trajectories()

plt.figure()
for n in range(N):
    plt.plot(B[:, n], 'k')
plt.xlabel(r'$t$')
plt.ylabel(r'$B_t$');
```

The plot above represents the index of the ancestors of the final particles $X_T^n$, at times $t=0, ...,T$. Alternatively, we can plot the number of distinct ancestors (of particles $X_T^n$) at time $t$; see below. Clearly, although this is a toy example, and the number of time steps is small, we already observe some strong path degeneracy (although the number of ancestors at time 0 does not collapse to one).

```python
plt.figure()
plt.plot([np.unique(B[t, :]).shape[0] for t in range(T)], 'k')
plt.xlabel(r'$t$')
plt.ylabel('nr ancestors')
plt.ylim(bottom=1)
plt.yscale('log')
```

The block below shows how one may compute various variance estimators. These estimators are implemented as collectors (see module `collectors`); i.e. as objects that compute and collect, at each time $t$, a certain variance estimator, and save the result in an an attribute of ``smc.summaries``, where ``smc`` is the considered `SMC` instance (the algorithm you are running).

More precisely, module `variance_estimators` includes the following collectors:

* ``Var(phi=None)``: computes the variance estimator defined above, for function ``phi`` (default = identity function).
* ``Var_logLt()``: computes the variance estimator for the estimates of the log-normalising constants, $\log L_t$ (Lee & Whiteley, 2018).
* ``Lag_based_Var(phi=None)``: computes the lag-based estimator of Olsson and Douc (2019) for function ``phi``. You   must then specify a `maximum lag`, through the extra command ``store_history=max_lag``. See module `smoothing` for more details on storing (part of) the history of a SMC algorithm.

```python
from particles import variance_estimators as var

def phi(x):
    return x

max_lag = 11
nruns = 10_000
runs = particles.multiSMC(fk=fk, N=N, resampling='multinomial', # see below
                          collect=[col.Moments(), var.Var_logLt(), var.Var(phi=phi), var.Lag_based_var(phi=phi)],
                          store_history=max_lag, nruns=nruns, nprocs=0)

# We could do var.Var() and var.Lag_based_var(), since identity function is the defaut test function anyway
```

**Note:** In principle, these variance estimators are valid only when multinomial resampling is used, which is why we chose that particular resampling scheme.

## Numerical results

We now use the results to assess the variability of the standard (not lag-based) variance estimator. To do so, we compare the empirical variance (over the $10^4$ runs) with the empirical distribution of the estimates (summarised by the median, the $10\%-$ and the $90\%-$ quantile).

We start with the normalising constants (i.e. the marginal likelihood of the data until time $t$).

```python
def plot_ci(time_range, arr):
    "Plot (empirical) confidence intervals."
    plt.plot(time_range, np.percentile(arr, 50, axis=0), 'k:',
             label='var est median')
    plt.fill_between(time_range, np.percentile(arr, 10, axis=0), np.percentile(arr, 90, axis=0),
                     alpha=0.1, color='black', label='var est 80% CI')

plt.plot([np.var([r['output'].summaries.logLts[t] for r in runs]) for t in range(T)],
         label='empirical var')
ll_var_ests = np.array([r['output'].summaries.var_logLt for r in runs])
plot_ci(np.arange(T), ll_var_ests)
plt.xlabel(r'$t$')
plt.ylabel('log-likelihood')
plt.legend(loc='upper left');
```

Not too bad, right? Note however that the error (in estimating the variance) seems to grow over time, and tends to be negative (under-estimation of the variance).

Let's now look at the performance of the variance estimators for function $\varphi(x)=x$; i.e. the variance of the particle estimates of the filtering expectations.

```python
plt.figure()
ests =  np.array([[r['output'].summaries.moments[t]['mean'] for r in runs]
                  for t in range(T)]).T
plt.plot(np.var(ests, axis=0), label='empirical var')
var_ests = np.array([r['output'].summaries.var for r in runs])
plot_ci(np.arange(T), var_ests)
plt.legend()
plt.xlabel(r'$t$')
plt.yscale('log')
```

Again, error seems to grow with time. Let's zoom on the first time steps:

```python
t0 = 20
plt.figure()
plt.plot(np.var(ests[:, :t0], axis=0))
plot_ci(np.arange(t0), var_ests[:, :t0])
plt.legend()
plt.xlabel(r'$t$')
plt.yscale('log')
```

At the first time steps, the variance estimates seem  more reliable, presumably because path degeneracy remains limited at times $\leq 20$.

## Performance over repeated runs

Variance estimates may get more robust if averaged over repeated runs. To see if this is true, we bootstrap the sample of estimates to assess the performance over 10 runs.

```python
# Bootstrap
nsample = 10
nboot = 10_000
boot_emp_var = np.zeros((nboot, T))
boot_var_ests = np.zeros((nboot, T))
for n in range(nboot):
    idx = random.choice(nruns, size=nsample, replace=False)
    boot_emp_var[n, :] = np.var(ests[idx, :], axis=0)
    boot_var_ests[n, :] = np.mean(var_ests[idx, :], axis=0)
```

The following plot compares the performance of:

* the empirical variance (of the considered estimate) over 10 runs;
* the empirical mean of the variance estimator over 10 runs.

Through averaging, the estimator becomes far more reliable, and seems to show comparable or better performance than the empirical variance. However, the level of improvement seems to decrease over time (again because of path degeneracy).

```python
import itertools
import pandas

d1 = [{'t': t, 'type': 'empirical', 'est': boot_emp_var[n, t]}
      for n, t in it.product(range(N), range(T))]
d2 = [{'t': t, 'type': 'var est', 'est': boot_var_ests[n, t]}
      for n, t in it.product(range(N), range(T))]
df = pandas.DataFrame(d1 + d2)
dft = df[df['t'] % 10 == 9 ]
sb.boxplot(x='t', y='est', hue='type', data=dft)
```

Thus, averaging does seem to help in making these estimators more robust (even at times where there is already strong path degeneracy).

## Lag-based estimators

Olsson and Douc (2019) proposed a variant of the above estimators based on a lag approximation. This introduces a bias, but this also should alleviate the path degeneracy.

We now look at the performance of lag-based variance estimators.

```python
def get_list(l, k):
    if k < len(l):
        return l[k]
    else:
        return 0.
for lag in range(0, 6):
    plt.figure()
    plt.title('lag=%i' % lag)
    plt.plot(np.var(ests, axis=0), label='empirical var')
    times = list(range(lag, T))
    lag_var_ests = np.array([[get_list(est, lag)
                              for est in r['output'].summaries.lag_based_var[lag:]]
                             for r in runs])
    plot_ci(times, lag_var_ests)
    plt.legend()
    plt.yscale('log')
```

These results are a tad surprising: little bias is observed even for lag=0. Increasing the lag seems only to increase the error.

## Bottom line

Based on these experiments (which are of course a bit limited), the following recommendations seem reasonable:

* Averaging over a small number of runs seems to make  variance estimators more robust.
* Even with averaging, the performance gain (relative to the empirical variance) may be modest as soon as the number of distinct eve variables is too small.
* When the number of distinct ever variables drop to one, these variance estimators do not work any more.
* In that case, it may be worth switching to the lag-based variant of Olsson and Douc (2019).
* However, choosing the value of the lag seems non-trivial, and deserves further investigation.

## References

* Chan, H.P. and Lai, T.L. (2013). A general theory of particle filters in
  hidden Markov models and some applications. Ann. Statist. 41, pp. 2877–2904.

* Lee, A and Whiteley, N (2018). Variance estimation in the particle filter.
  Biometrika 3, pp. 609-625.

* Olsson, J. and Douc, R. (2019). Numerically stable online estimation of
  variance in particle filters. Bernoulli 25.2, pp. 1504-1535.