# The Characteristic GRFFE speeds
## Author: Patrick Nelson

This notebook documents the function from the original `GiRaFFE` that calculates the speeds of the characteristic GRFFE hydrodynamics waves.

**Notebook Status:**  Validated

**Validation Notes:** This code has been validated to round-off level agreement with the corresponding code in the original `GiRaFFE` by means of its implementation in [Tutorial-GiRaFFE_NRPy-Stilde-flux](Tutorial-GiRaFFE_NRPy-Stilde-flux.ipynb) and [Tutorial-GiRaFFE_NRPy-Afield_flux_handwritten](Tutorial-GiRaFFE_NRPy-Afield_flux_handwritten.ipynb)

### NRPy+ Source Code for this module:
* [GiRaFFE_NRPy/GiRaFFE_NRPy_Characteristic_Speeds.py](../../edit/in_progress/GiRaFFE_NRPy/GiRaFFE_NRPy_Characteristic_Speeds.py)

## Introduction

Here, we will find the speeds at which the hydrodynamics waves propagate. We start from the speed of light (since FFE deals with very diffuse plasmas), which is $c=1.0$ in our chosen units. We then find the speeds $c_+$ and $c_-$ on each face with the function `find_cp_cm`; then, we find minimum and maximum speeds possible from among those. A complete derivation of these speeds is included [below](#derive_speed) as an appendix; here, for brevity, we simply start with the implementation of the code in the original `GiRaFFE`. That code itself is adapted from the more general `IllinoisGRMHD` code; we will implement several more simplifications to help improve performance.

Below is the source code for `find_cp_cm`, edited to work with the NRPy+ version of `GiRaFFE`. One edit we need to make in particular is to the term `psim4*gupii` in the definition of `c`; that was written assuming the use of the conformal metric $\tilde{g}^{ii}$. Since we are not using that here, and are instead using the ADM metric, we should not multiply by $\psi^{-4}$.

```c
static inline void find_cp_cm(REAL &cplus,REAL &cminus,const REAL v02,const REAL u0,
                              const REAL vi,const REAL lapse,const REAL shifti,
                              const REAL gammadet,const REAL gupii) {
  const REAL u0_SQUARED=u0*u0;
  const REAL ONE_OVER_LAPSE_SQUARED = 1.0/(lapse*lapse);
  // sqrtgamma = psi6 -> psim4 = gammadet^(-1.0/3.0)
  const REAL psim4 = pow(gammadet,-1.0/3.0);
  //Find cplus, cminus:
  const REAL a = u0_SQUARED * (1.0-v02) + v02*ONE_OVER_LAPSE_SQUARED;
  const REAL b = 2.0* ( shifti*ONE_OVER_LAPSE_SQUARED * v02 - u0_SQUARED * vi * (1.0-v02) );
  const REAL c = u0_SQUARED*vi*vi * (1.0-v02) - v02 * ( gupii -
                                                               shifti*shifti*ONE_OVER_LAPSE_SQUARED);
  REAL detm = b*b - 4.0*a*c;
  //ORIGINAL LINE OF CODE:
  //if(detm < 0.0) detm = 0.0;
  //New line of code (without the if() statement) has the same effect:
  detm = sqrt(0.5*(detm + fabs(detm))); /* Based on very nice suggestion from Roland Haas */

  cplus = 0.5*(detm-b)/a;
  cminus = -0.5*(detm+b)/a;
  if (cplus < cminus) {
    const REAL cp = cminus;
    cminus = cplus;
    cplus = cp;
  }
}
```
Comments documenting this have been excised for brevity, but are reproduced in $\LaTeX$ [below](#derive_speed).

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#prelim): Preliminaries
1. [Step 2](#simplifications): Apply GRFFE assumptions to simplify expressions
1. [Step 3](#char_speeds): Compute the characteristic speeds
    1. [Step 3.a](#cp_cm): Compute $c_+$ and $c_-$
    1. [Step 3.b](#cmax_cmin): Compute $c_\max$ and $c_\min$
1. [Step 4](#code_validation): Code Validation against `GiRaFFE_NRPy.GiRaFFE_NRPy_Characteristic_Speeds` NRPy+ Module
1. [Step 5](#derive_speed): Complete Derivation of the Wave Speeds
1. [Step 6](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Preliminaries \[Back to [top](#toc)\]
$$\label{prelim}$$

This first block of code imports the core NRPy+ functionality after first adding the main NRPy+ directory to the path.

```python
# Step 0: Add NRPy's directory to the path
# https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
import os,sys
nrpy_dir_path = os.path.join("..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
```

# Step 2: Apply GRFFE assumptions to simplify expressions \[Back to [top](#toc)\]
$$\label{simplifications}$$

We could use this code directly, but there's substantial improvement we can make by changing the code into a NRPyfied form. Note the `if` statement; NRPy+ does not know how to handle these, so we must eliminate it if we want to leverage NRPy+'s full power. (Calls to `fabs()` are also cheaper than `if` statements.) This can be done if we rewrite this, taking inspiration from the other eliminated `if` statement documented in the above code block:
```c
  cp = 0.5*(detm-b)/a;
  cm = -0.5*(detm+b)/a;
  cplus  = 0.5*(cp+cm+fabs(cp-cm));
  cminus = 0.5*(cp+cm-fabs(cp-cm));
```
This can be simplified further, by substituting `cp` and `cm` into the below equations and eliminating terms as appropriate. First note that `cp+cm = -b/a` and that `cp-cm = detm/a`. Thus,
```c
  cplus  = 0.5*(-b/a + fabs(detm/a));
  cminus = 0.5*(-b/a - fabs(detm/a));
```
This fulfills the original purpose of the `if` statement in the original code because we have guaranteed that $c_+ \geq c_-$.

This leaves us with an expression that can be much more easily NRPyfied. So, we will rewrite the following in NRPy+, making only minimal changes to be proper Python. However, it turns out that we can make this even simpler. In GRFFE, $v_0^2$ is guaranteed to be exactly one. In GRMHD, this speed was calculated as $$v_{0}^{2} = v_{\rm A}^{2} + c_{\rm s}^{2}\left(1-v_{\rm A}^{2}\right),$$ where the Alfvén speed $v_{\rm A}^{2}$ $$v_{\rm A}^{2} = \frac{b^{2}}{\rho_{b}h + b^{2}}.$$ So, we can see that when the density $\rho_b$ goes to zero, $v_{0}^{2} = v_{\rm A}^{2} = 1$. Then
\begin{align}
a &= (u^0)^2 (1-v_0^2) + v_0^2/\alpha^2 \\
&= 1/\alpha^2 \\
b &= 2 \left(\beta^i v_0^2 / \alpha^2 - (u^0)^2 v^i (1-v_0^2)\right) \\
&= 2 \beta^i / \alpha^2 \\
c &= (u^0)^2 (v^i)^2 (1-v_0^2) - v_0^2 \left(\gamma^{ii} - (\beta^i)^2/\alpha^2\right) \\
&= -\gamma^{ii} + (\beta^i)^2/\alpha^2,
\end{align}
are simplifications that should save us some time; we can see that $a \geq 0$ is guaranteed. Note that we also force `detm` to be positive. Thus, `detm/a` is guaranteed to be positive itself, rendering the calls to `nrpyAbs()` superfluous. Furthermore, we eliminate any dependence on the Valencia 3-velocity and the time compoenent of the four-velocity, $u^0$. This leaves us free to solve the quadratic in the familiar way: $$c_\pm = \frac{-b \pm \sqrt{b^2-4ac}}{2a}$$.

# Step 3: Compute the characteristic speeds \[Back to [top](#toc)\]
$$\label{char_speeds}$$

## Step 3.a: Compute $c_+$ and $c_-$ \[Back to [top](#toc)\]
$$\label{cp_cm}$$

We now have the expressions we need for $c_+$ and $c_-$:
$$c_\pm = \frac{-b \pm \sqrt{b^2-4ac}}{2a},$$
where
\begin{align}
a &= 1/\alpha^2 \\
b &= 2 \beta^i / \alpha^2 \\
c &= -\gamma^{ii} + (\beta^i)^2/\alpha^2.
\end{align}

```python
# We'll write this as a function so that we can calculate the expressions on-demand for any choice of i
def find_cp_cm(lapse,shifti,gammaUUii):
    # Inputs:  u0,vi,lapse,shift,gammadet,gupii
    # Outputs: cplus,cminus

    # a = 1/(alpha^2)
    a = sp.sympify(1)/(lapse*lapse)
    # b = 2 beta^i / alpha^2
    b = sp.sympify(2) * shifti /(lapse*lapse)
    # c = -g^{ii} + (beta^i)^2 / alpha^2
    c = - gammaUUii + shifti*shifti/(lapse*lapse)

    # Now, we are free to solve the quadratic equation as usual. We take care to avoid passing a
    # negative value to the sqrt function.
    detm = b*b - sp.sympify(4)*a*c

    import Min_Max_and_Piecewise_Expressions as noif
    detm = sp.sqrt(noif.max_noif(sp.sympify(0),detm))
    global cplus,cminus
    cplus  = sp.Rational(1,2)*(-b/a + detm/a)
    cminus = sp.Rational(1,2)*(-b/a - detm/a)
```

## Step 3.b: Compute $c_\max$ and $c_\min$ \[Back to [top](#toc)\]
$$\label{cmax_cmin}$$

In flat spacetime, where $\alpha=1$, $\beta^i=0$, and $\gamma^{ij} = \delta^{ij}$, $c_+ > 0$ and $c_- < 0$. For the HLLE solver, we will need both `cmax` and `cmin` to be positive; we also want to choose the speed that is larger in magnitude because overestimating the characteristic speeds will help damp unwanted oscillations. (However, in GRFFE, we only get one $c_+$ and one $c_-$, so we only need to fix the signs here.) Hence, the following function.

We will now write a function in NRPy+ similar to the one used in the old `GiRaFFE`, allowing us to generate the expressions with less need to copy-and-paste code; the key difference is that this one will be in Python, and generate optimized C code integrated into the rest of the operations. Notice that since we eliminated the dependence on velocities, none of the input quantities are different on either side of the face. So, this function won't really do much besides guarantee that `cmax` and `cmin` are positive, but we'll leave the machinery here since it is likely to be a useful guide to somebody who wants to something similar. We use the same technique as above to replace the `if` statements inherent to the `MAX()` and `MIN()` functions.

```python
# We'll write this as a function, and call it within HLLE_solver, below.
def find_cmax_cmin(flux_dirn,gamma_faceDD,beta_faceU,alpha_face):
    # Inputs:  flux direction flux_dirn, Inverse metric gamma_faceUU, shift beta_faceU,
    #          lapse alpha_face, metric determinant gammadet_face
    # Outputs: maximum and minimum characteristic speeds cmax and cmin
    # First, we need to find the characteristic speeds on each face
    gamma_faceUU,unusedgammaDET = ixp.generic_matrix_inverter3x3(gamma_faceDD)
    # Original needed for GRMHD
#     find_cp_cm(alpha_face,beta_faceU[field_comp],gamma_faceUU[field_comp][field_comp])
#     cpr = cplus
#     cmr = cminus
#     find_cp_cm(alpha_face,beta_faceU[field_comp],gamma_faceUU[field_comp][field_comp])
#     cpl = cplus
#     cml = cminus
    find_cp_cm(alpha_face,beta_faceU[flux_dirn],gamma_faceUU[flux_dirn][flux_dirn])
    cp = cplus
    cm = cminus

    # The following algorithms have been verified with random floats:

    global cmax,cmin
    # Now, we need to set cmax to the larger of cpr,cpl, and 0

    import Min_Max_and_Piecewise_Expressions as noif
    cmax = noif.max_noif(cp,sp.sympify(0))

    # And then, set cmin to the smaller of cmr,cml, and 0
    cmin = -noif.min_noif(cm,sp.sympify(0))
```

# Step 4:  Code Validation against `GiRaFFE_NRPy.GiRaFFE_NRPy_Characteristic_Speeds` NRPy+ Module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the `GiRaFFE` evolution equations and auxiliary quantities we intend to use between
1. this tutorial and
2. the NRPy+ [GiRaFFE_NRPy.GiRaFFE_NRPy_Characteristic_Speeds](../../edit/in_progress/GiRaFFE_NRPy/GiRaFFE_NRPy_Characteristic_Speeds.py) module.

This first validation directly compares the sympy expressions. This is generally quicker and more reliable, but might overlook some complexities in implementing the C code.

```python
all_passed=True
def comp_func(expr1,expr2,basename,prefixname2="Sf."):
    if str(expr1-expr2)!="0":
        print(basename+" - "+prefixname2+basename+" = "+ str(expr1-expr2))
        all_passed=False

def gfnm(basename,idx1,idx2=None,idx3=None):
    if idx2 is None:
        return basename+"["+str(idx1)+"]"
    if idx3 is None:
        return basename+"["+str(idx1)+"]["+str(idx2)+"]"
    return basename+"["+str(idx1)+"]["+str(idx2)+"]["+str(idx3)+"]"

# We'll first declare the inputs to the function.
gammaDD = ixp.declarerank2("gammaDD","sym01",DIM=3)
betaU = ixp.declarerank1("betaU",DIM=3)
alpha = sp.sympify("alpha")
import GiRaFFE_NRPy.GiRaFFE_NRPy_Characteristic_Speeds as chsp

for flux_dirn in range(3):
    expr_list = []
    exprcheck_list = []
    namecheck_list = []

    print("Checking the flux in direction "+str(flux_dirn))
    find_cmax_cmin(flux_dirn,gammaDD,betaU,alpha)
    chsp.find_cmax_cmin(flux_dirn,gammaDD,betaU,alpha)

    namecheck_list.extend(["cmax","cmin"])
    exprcheck_list.extend([chsp.cmax,chsp.cmin])
    expr_list.extend([cmax,cmin])

    for i in range(len(expr_list)):
        comp_func(expr_list[i],exprcheck_list[i],namecheck_list[i])

if all_passed:
    print("ALL TESTS PASSED!")
else:
    print("ERROR: AT LEAST ONE TEST DID NOT PASS")
    sys.exit(1)

```

    Checking the flux in direction 0
    Checking the flux in direction 1
    Checking the flux in direction 2
    ALL TESTS PASSED!

# Step 5: Complete Derivation of the Wave Speeds \[Back to [top](#toc)\]
$$\label{derive_speed}$$

This computes phase speeds in the direction given by `flux_dirn`. Note that we replace the full dispersion relation with a simpler one, which overestimates the maximum speeds by a factor of ~2. See full discussion around Eqs. 49 and 50 in [Duez, et al.](http://arxiv.org/pdf/astro-ph/0503420.pdf). In summary, we solve the dispersion relation (in, e.g., the $x$-direction) with a wave vector of $k_\mu = (-\omega,k_x,0,0)$. So, we solve the approximate dispersion relation $\omega_{\rm cm}^2 = [v_A^2 + c_s^2 (1-v_A^2)]k_{\rm cm}^2$ for the wave speed $\omega/k_x$, where the sound speed $c_s = \sqrt{\Gamma P/(h \rho_0)}$, the Alfvén speed $v_A = 1$ (in GRFFE), $\omega_{\rm cm} = -k_\mu k^\mu$ is the frequency in the comoving frame, $k_{\rm cm}^2 = K_\mu K^\mu$ is the wavenumber squared in the comoving frame, and $K_\mu = (g_{\mu\nu} + u_\mu u_\nu)k^\nu$ is the part of the wave vector normal to the four-velocity $u^\mu$. See below for a complete derivation.

What follows is a complete derivation of the quadratic we solve. We start from the following relations:
\begin{align}
w_{\rm cm} &= (-k_0 u^0 - k_x u^x) \\
k_{\rm cm}^2 &= K_{\mu} K^{\mu}, \\
K_{\mu} K^{\mu} &= (g_{\mu a} + u_{\mu} u_a) k^a  g^{\mu b} [ (g_{c b} + u_c u_b) k^c ] \\
\end{align}
The last term of the above can be written as follow:
$$
(g_{c b} + u_{c} u_{b}) k^c = (\delta^{\mu}_c + u_c u^{\mu} ) k^c
$$

Then,
\begin{align}
K_{\mu} K^{\mu} &= (g_{\mu a} + u_{\mu} u_a) k^a  g^{\mu b} [ (g_{c b} + u_c u_b) k^c ] \\
                 &= (g_{\mu a} + u_{\mu} u_a) k^a  (\delta^{\mu}_c + u_c u^{\mu} ) k^c \\
                 &=[(g_{\mu a} + u_{\mu} u_a) \delta^{\mu}_c + (g_{\mu a} + u_{\mu} u_a) u_c u^{\mu} ] k^c k^a \\
                 &=[(g_{c a} + u_c u_a) + (u_c u_a -  u_a u_c] k^c k^a \\
                 &=(g_{c a} + u_c u_a) k^c k^a \\
                 &= k_a k^a + u^c u^a k_c k_a \\
k^a = g^{\mu a} k_{\mu} &= g^{0 a} k_0 + g^{x a} k_x \\
k_a k^a &= k_0 g^{0 0} k_0 + k_x k_0 g^{0 x} + g^{x 0} k_0 k_x + g^{x x} k_x k_x \\
         &= g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2 \\
u^c u^a k_c k_a &= (u^0 k_0 + u^x k_x) (u^0 k_0 + u^x k_x) = (u^0 k_0)^2 + 2 u^x k_x u^0 k_0 + (u^x k_x)^2 \\
(k_0 u^0)^2  + 2 k_x u^x k_0 u^0 + (k_x u^x)^2 &= v_0^2 [ (u^0 k_0)^2 + 2 u^x k_x u^0 k_0 + (u^x k_x)^2 + g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2] \\
(1-v_0^2) (u^0 k_0 + u^x k_x)^2 &= v_0^2 (g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2) \\
(1-v_0^2) (u^0 k_0/k_x + u^x)^2 &= v_0^2 (g^{00} (k_0/k_x)^2 + 2 g^{x0} k_0/k_x + g^{xx}) \\
(1-v_0^2) (u^0 X + u^x)^2 &= v_0^2 (g^{00} X^2 + 2 g^{x0} X + g^{xx}) \\
(1-v_0^2) ((u^0)^2 X^2 + 2 u^x (u^0) X + (u^x)^2) &= v_0^2 (g^{00} X^2 + 2 g^{x0} X + g^{xx}) \\
0 &= X^2 ( (1-v_0^2) (u^0)^2 - v_0^2 g^{00}) + X (2 u^x u^0 (1-v_0^2) - 2 v_0^2 g^{x0}) + (1-v_0^2) (u^x)^2 - v_0^2 g^{xx} \\
a &= (1-v_0^2) (u^0)^2 - v_0^2 g^{00} = (1-v_0^2) (u^0)^2 + v_0^2/\alpha^2 \leftarrow {\rm VERIFIED} \\
b &= 2 u^x u^0 (1-v_0^2) - 2 v_0^2 \beta^x/\alpha^2 \leftarrow {\rm VERIFIED,\ } X\rightarrow -X, {\rm because\ } X = -w/k_1, {\rm \ and\ we\ are\ solving\ for} -X. \\
c &= (1-v_0^2) (u^x)^2 - v_0^2 (\gamma^{xx}\psi^{-4} - (\beta^x/\alpha)^2) \leftarrow {\rm VERIFIED} \\
v_0^2 &= v_A^2 + c_s^2 (1 - v_A^2) \\
\end{align}

# Step 6: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-GiRaFFE_NRPy-Characteristic_Speeds.pdf](Tutorial-GiRaFFE_NRPy-Characteristic_Speeds.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-GiRaFFE_NRPy-Characteristic_Speeds",location_of_template_file=os.path.join(".."))
```

    Created Tutorial-GiRaFFE_NRPy-Characteristic_Speeds.tex, and compiled LaTeX
        file to PDF file Tutorial-GiRaFFE_NRPy-Characteristic_Speeds.pdf


# Direct optimal control of a pendulum

We want to control an inverted pendulum and stabilize it in the upright position. The equations in Hamiltonian form describing an inverted pendulum with a torsional spring are as following:

$$\begin{equation}
    \begin{bmatrix} \dot{q}\\ \dot{p}\\ \end{bmatrix} =
    \begin{bmatrix}
    0& 1/m \\
    -k& -\beta/m\\
    \end{bmatrix}
    \begin{bmatrix} q\\ p\\ \end{bmatrix} -
    \begin{bmatrix}
    0\\
    mgl \sin{q}\\
    \end{bmatrix}+
    \begin{bmatrix}
    0\\
    1\\
    \end{bmatrix} u
\end{equation}$$

```python
import sys; sys.path.append(2*'../') # go n dirs back
import matplotlib.pyplot as plt

import torch
from torchdyn.numerics.odeint import odeint
from torchcontrol.systems.classic_control import Pendulum
from torchcontrol.cost import IntegralCost
from torchcontrol.controllers import *

%load_ext autoreload
%autoreload 2
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

```python
# Change device according to your configuration
device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu') # feel free to change :)
device = torch.device('cpu') # override
```

## Optimal control problem
In order to control the pendulum, we have to define a proper _integral cost function_ which will be our loss to be minimized during training. In a general form, it can be defined as:

$$\begin{equation}
        \min_{u_\theta} J = (x(t_f) - x^\star)^\top\mathbf{P} (x(t_f) - x^\star)) + \int_{t_0}^{t_f} \left[ (x(t) - x^\star)^\top \mathbf{Q} (x(t) - x^\star) + (u_\theta(t) - u^\star)^\top \mathbf{R} (u_\theta(t) - u^\star) \right] dt
\end{equation}$$

where $ x $ is the state and $u_\theta$ is the controller; $x^\star$ and $u^\star$ are the desired position and zero-cost controller; matrices $\mathbf{P},~\mathbf{Q}, ~ \mathbf{R}$ are weights for tweaking the performance.

```python
# Declaring the cost function
x_star = torch.Tensor([0, 0]).to(device)
u_star = 0.
cost = IntegralCost(x_star=x_star, u_star=u_star, P=0, Q=1, R=0)
```

## Initial conditions
Now we can see how the system behaves with no control input in time. Let's declare some initial variables:

```python
from math import pi as π

# Time span
dt = 0.05 # step size
t0, tf = 0, 3 # initial and final time
steps = int((tf - t0)/dt) + 1
t_span = torch.linspace(t0, tf, steps).to(device)

# Initial distribution
x_0 = π # limit of the state distribution (in rads and rads/second)
init_dist = torch.distributions.Uniform(torch.Tensor([-x_0, -x_0]), torch.Tensor([x_0, x_0]))
```

## Box-constrained controller
We want to give a limited control input. We consider the box-constrained neural controller (parameters $\theta$ of $u_\theta$ belong to a feed-forward neural network):

```python
?? BoxConstrainedController
```

    Init signature:
     BoxConstrainedController(
        in_dim,
        out_dim,
        h_dim=64,
        num_layers=2,
        zero_init=True,
        input_scaling=None,
        output_scaling=None,
        constrained=False,
    )
    Source:
    class BoxConstrainedController(nn.Module):
        """Simple controller  based on a Neural Network with
        bounded control inputs

        Args:
            in_dim: input dimension
            out_dim: output dimension
            hid_dim: hidden dimension
            zero_init: initialize last layer to zeros
        """
        def __init__(self,
                     in_dim,
                     out_dim,
                     h_dim=64,
                     num_layers=2,
                     zero_init=True,
                     input_scaling=None,
                     output_scaling=None,
                     constrained=False):

            super().__init__()
            # Create Neural Network
            layers = []
            layers.append(nn.Linear(in_dim, h_dim))
            for i in range(num_layers):
                if i < num_layers-1:
                    layers.append(nn.Softplus())
                else:
                    # last layer has tanh as activation function
                    # which acts as a regulator
                    layers.append(nn.Tanh())
                    break
                layers.append(nn.Linear(h_dim, h_dim))
            layers.append(nn.Linear(h_dim, out_dim))
            self.layers = nn.Sequential(*layers)

            # Initialize controller with zeros in the last layer
            if zero_init: self._init_zeros()
            self.zero_init = zero_init

            # Scaling
            if constrained is False and output_scaling is not None:
                warn("Output scaling has no effect without the `constrained` variable set to true")
            if input_scaling is None:
                input_scaling = torch.ones(in_dim)
            if output_scaling is None:
                # scaling[:, 0] -> min value
                # scaling[:, 1] -> max value
                output_scaling = torch.cat([-torch.ones(out_dim),
                                             torch.ones(out_dim)], -1)
            self.in_scaling = input_scaling
            self.out_scaling = output_scaling
            self.constrained = constrained

        def forward(self, t, x):
            x = self.layers(self.in_scaling.to(x)*x)
            if self.constrained:
                # We consider the constraints between -1 and 1
                # and then we rescale them
                x = tanh(x)
                # TODO: fix the tanh to clamp
    #             x = torch.clamp(x, -1, 1) # not working in some applications
                x = self._rescale(x)
            return x

        def _rescale(self, x):
            s = self.out_scaling.to(x)
            return 0.5*(x + 1)*(s[...,1]-s[...,0]) + s[...,0]

        def _reset(self):
            '''Reinitialize layers'''
            for p in self.layers.children():
                if hasattr(p, 'reset_parameters'):
                    p.reset_parameters()
            if self.zero_init: self._init_zeros()

        def _init_zeros(self):
            '''Reinitialize last layer with zeros'''
            for p in self.layers[-1].parameters():
                nn.init.zeros_(p)
    File:           ~/Desktop/torchcontrol/torchcontrol/controllers.py
    Type:           type
    Subclasses:

```python
# Controller
output_scaling = torch.Tensor([-5, 5]).to(device) # controller limits
u = BoxConstrainedController(2, 1, constrained=True, output_scaling=output_scaling).to(device)

# Initialize pendulum with given controller
pendulum = Pendulum(u, solver='euler')
```

## Optimization loop
Here we run the optimization: in particular, we use stochastic gradient descent with `Adam` to optimize the parameters

```python
from tqdm import trange

# Hyperparameters
lr = 1e-3
epochs = 300
bs = 1024
opt = torch.optim.Adam(u.parameters(), lr=lr)

# Training loop
losses=[]
with trange(0, epochs, desc="Epochs") as eps:
    for epoch in eps:
        x0 = init_dist.sample((bs,)).to(device)
        trajectory = pendulum(x0, t_span)
        loss = cost(trajectory); losses.append(loss.detach().cpu().item())
        loss.backward(); opt.step(); opt.zero_grad()
        eps.set_postfix(loss=(loss.detach().cpu().item()))
```

    Epochs: 100%|██████████| 300/300 [00:24<00:00, 12.34it/s, loss=1.26]

```python
fig, ax = plt.subplots(1, 1, figsize=(8,4))
ax.plot(losses)
ax.set_title('Losses')
ax.set_xlabel('Epochs')
ax.set_yscale('log')
```

## Plot results

```python
# Change the solver to 'dopri5' (adaptive step size, more accurate than Euler)
pendulum.solver = 'dopri5'

# Forward propagate some trajectories
x0 = init_dist.sample((100,)).to(device)*0.8

# Prolong time span
dt = 0.05 # step size
t0, tf = 0, 5 # initial and final time
steps = int((tf - t0)/dt) + 1
t_span = torch.linspace(t0, tf, steps).to(device)

traj = pendulum(x0, t_span)

def plot_pendulum_trajs():
    fig, axs = plt.subplots(1, 2, figsize=(12,4))
    for i in range(len(x0)):
        axs[0].plot(t_span.cpu(), traj[:,i,0].detach().cpu(), 'tab:red', alpha=.3)
        axs[1].plot(t_span.cpu(), traj[:,i,1].detach().cpu(), 'tab:blue', alpha=.3)
    axs[0].set_xlabel(r'Time [s]'); axs[1].set_xlabel(r'Time [s]')
    axs[0].set_ylabel(r'p'); axs[1].set_ylabel(r'q')
    axs[0].set_title(r'Positions'); axs[1].set_title(r'Momenta')

plot_pendulum_trajs()
```

```python
# Plot learned vector field and trajectories in phase space
n_grid = 50
graph_lim = π

def plot_phase_space():
    fig, ax = plt.subplots(1, 1, figsize=(6,6))

    x = torch.linspace(-graph_lim, graph_lim, n_grid).to(device)
    Q, P = torch.meshgrid(x, x) ; z = torch.cat([Q.reshape(-1, 1), P.reshape(-1, 1)], 1)
    f = pendulum.dynamics(0, z).detach().cpu()
    Fq, Fp = f[:,0].reshape(n_grid, n_grid), f[:,1].reshape(n_grid, n_grid)
    val = pendulum.u(0, z).detach().cpu()
    U = val.reshape(n_grid, n_grid)
    ax.streamplot(Q.T.detach().cpu().numpy(), P.T.detach().cpu().numpy(),
                    Fq.T.detach().cpu().numpy(), Fp.T.detach().cpu().numpy(), color='black', density=0.6, linewidth=0.5)

    ax.set_xlim([-graph_lim, graph_lim]) ; ax.set_ylim([-graph_lim, graph_lim])
    traj = pendulum(x0, t_span).detach().cpu()
    for j in range(traj.shape[1]):
        ax.plot(traj[:,j,0], traj[:,j,1], color='tab:purple', alpha=.4)
    ax.set_title('Phase Space')
    ax.set_xlabel(r'p')
    ax.set_ylabel(r'q')

plot_phase_space()
```

Nice! The controller manages to stabilize the pendulum in our desired $x^\star$ 🎉


## Devito CFD Tutorial series

The following series of notebook tutorials will demonstrate the use of Devito and it's SymPy-based API to solve a set of classic examples from Computational Fluid Dynamics (CFD). The tutorials are based on the excellent tutorial series _CFD Python: 12 steps to Navier-Stokes_ by Lorena Barba and focus on the implementation with Devito rather than pure CFD or finite difference theory. For a refresher on how to implement 2D finite difference solvers for CFD problems, please see the original tutorial series here: <br>
http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/

### Example 1: Linear convection in 2D

Lets start with a simple 2D convection example - step 5 in the original blog. This will already allow us to demonstrate a lot about the use of Devito's symbolic data objects and how to use them to build a simple operator directly from the symbolic notation of the equation. The governing equation we will implement in this tutorial is:

$$\frac{\partial u}{\partial t}+c\frac{\partial u}{\partial x} + c\frac{\partial u}{\partial y} = 0$$

In order to implement this equation we first discretize it using forward differences in time and backward differences in space. Just as the original tutorial, we will use $u_{i,j}^n$ to denote a finite difference stencil point with $i$ and $j$ denoting spatial indices and $n$ denoting the time index. So, after re-arranging the discretized equation for the forward stencil point in time we get

$$u_{i,j}^{n+1} = u_{i,j}^n-c \frac{\Delta t}{\Delta x}(u_{i,j}^n-u_{i-1,j}^n)-c \frac{\Delta t}{\Delta y}(u_{i,j}^n-u_{i,j-1}^n)$$

Using this, we can start deriving the computational stencil for this equation. Let's first look at the original _pure Python_ implementation of the linear convection flow - but first we import our tools and define some parameters:

```python
from examples.cfd import plot_field, init_hat
import numpy as np
%matplotlib inline

# Some variable declarations
nx = 81
ny = 81
nt = 100
c = 1.
dx = 2. / (nx - 1)
dy = 2. / (ny - 1)
print("dx %s, dy %s" % (dx, dy))
sigma = .2
dt = sigma * dx
```

    dx 0.025, dy 0.025

**A small note on style:** Throughout this tutorial series we will use utility functions to plot the various 2D functions and data sets we deal with. These are all taken from the original tutorial series, but have been slightly modified for our purposes. One of the differences readers might find is that the original series uses _(y, x)_ indexing for 2d data arrays, whereas many of the examples have been adapted to use _(x, y)_ notation in our tutorials.

So, let's start by creating a simple 2D function and initialising it with a "hat function". We will use that initialisation function a lot, so it comes from our utility scripts:

```python
#NBVAL_IGNORE_OUTPUT

# Create field and assign initial conditions
u = np.empty((nx, ny))
init_hat(field=u, dx=dx, dy=dy, value=2.)

# Plot initial condition
plot_field(u)
```

Now we can repeat the _pure NumPy_ solve from the original tutorial, where we use NumPy array operations to speed up the computation. Note that we skip the derivation of the stencil used to implement our convection equation, as we are going to walk though this process using the Devito API later in this tutorial.

```python
# Repeat initialisation, so we can re-run the cell
init_hat(field=u, dx=dx, dy=dy, value=2.)

for n in range(nt + 1):
    # Copy previous result into a new buffer
    un = u.copy()

    # Update the new result with a 3-point stencil
    u[1:, 1:] = (un[1:, 1:] - (c * dt / dy * (un[1:, 1:] - un[1:, :-1])) -
                              (c * dt / dx * (un[1:, 1:] - un[:-1, 1:])))

    # Apply boundary conditions.
    u[0, :] = 1.  # left
    u[-1, :] = 1. # right
    u[:, 0] = 1.  # bottom
    u[:, -1] = 1. # top
    # Note that in the above expressions the NumPy index -1 corresponds to the final point of the array along the indexed dimension,
    #    i.e. here u[-1, :] is equivalent to u[80, :].

```

```python
#NBVAL_IGNORE_OUTPUT

# A small sanity check for auto-testing
assert (u[45:55, 45:55] > 1.8).all()
u_ref = u.copy()

plot_field(u)
```

Hooray, the wave moved! Now, this little example is already very concise from a notational point of view and it teaches us quite a few key points about how to perform finite difference stencil computation via NumPy:
* Due to the backward differencing scheme in space (more later) we use only three stencil point in this example:  $u^{n}_{i, j}$, $u^{n}_{i-1, j}$ and $u^{n}_{i, j-1}$. These can be identified in the code through the array indices and correspond to `un[1:, 1:]`, `un[:-1, 1:]` and `un[1:, :-1]` respectively.
* Two buffers for array data are used throughout: `un[...]` is read from, while `u[...]` is updated, where the line `un = u.copy()` performs a deep copy of the field to switch buffers between timesteps. Note that in some other finite difference tutorials the cost of this copy operation is sometimes amortised by using two pre-allocated buffers and switching the indices of them explicitly.
* The final four lines within the loop code show us how to implement simple Dirichlet boundary condition by simply setting a value on the outermost rows and columns of our cartesian grid.

You may have noticed that the hat has not only moved to a different location, but has also changed its shape into a smooth bump. This is a little surprising, as the correct solution of the convection equation would be movement without shape change. The smooth shape is caused by numerical diffusion, a well-known limitation of low-order finite difference schemes. We will discuss this issue and some solutions later in this tutorial.

#### Devito implementation
Now we want to re-create the above example via a Devito operator. To do this, we can start by defining our computational grid and creating a function `u` as a symbolic `devito.TimeFunction`. The core thing to note here is that this is one of Devito's symbolic functions, which have a dual role in the creation of finite difference solvers:
* They behave symbolically like `sympy.Function` objects, so that we can construct derivatives and use them in symbolic expressions, thus inheriting all the power of automated symbolic manipulation that SymPy provides.
* They act as containers for user data by providing a `.data` property that wraps automatically allocated memory space in a neat NumPy array.

The particular `TimeFunction` type that we will declare our variable $u$ as in this case is aware of the fact that we will want to implement a timestepping algorithm with it. So the object `u` will declare two buffers of shape `(nx, ny)` for us, as defined by the `Grid` object, and present them as `u.data[0]` and `u.data[1]`. Let's fill the initial buffer with some data and look at it:

```python
#NBVAL_IGNORE_OUTPUT
from devito import Grid, TimeFunction

grid = Grid(shape=(nx, ny), extent=(2., 2.))
u = TimeFunction(name='u', grid=grid)
init_hat(field=u.data[0], dx=dx, dy=dy, value=2.)
# Note u.data[0] == u.data[0,:,:]
plot_field(u.data[0])
```

Nice. Now we can look at deriving our 3-point stencil using the symbolic capabilities given to our function $u$ by SymPy. For this we will first construct our derivative terms in space and time. For the forward derivative in time we can easily use Devito's shorthand notation `u.dt` to denote the first derivative in time and `u.dxl` and `u.dyl` to denote the space derivatives. Note that the `l` means were using the "left" or backward difference here to adhere to the discretization used in the original tutorials.

From the resulting terms we can then create a `sympy.Equation` object that contains the fully discretized equation, but from a neat high-level notation, as shown below.

```python
from devito import Eq

# Specify the `interior` flag so that the stencil is only
# applied to the interior of the domain.
eq = Eq(u.dt + c*u.dxl + c*u.dyl, subdomain=grid.interior)

print(eq)
```

    Eq(1.0*Derivative(u(t, x, y), x) + 1.0*Derivative(u(t, x, y), y) + Derivative(u(t, x, y), t), 0)

The above step resulted in a fully discretized version of our equation, which includes place-holder symbols for the spacing in time (`dt`) and space (`h`). These symbols are based on an internal convention and will later be replaced when we build an operator.

But before we can build an operator, we first need to change our discretized expression so that we are updating the forward stencil point in our timestepping scheme - Devito provides another short-hand notation for this: `u.forward`. For the actual symbolic reordering, SymPy comes to the rescue with the `solve` utility that we can use to re-organise our equation.

```python
from devito import solve
from sympy import nsimplify

stencil = solve(eq, u.forward)

# nsimplify: 1.0*x = x
print(nsimplify(stencil))
```

    dt*(-u(t, x, y)/h_y + u(t, x, y - h_y)/h_y - u(t, x, y)/h_x + u(t, x - h_x, y)/h_x + u(t, x, y)/dt)

The careful reader will note that this is equivalent to the symbolic expression for the RHS of the NumPy code we showed earlier - only with `dx` and `dy` denoted as `h_x` and `h_y` respectively, while `u(t, x, y)`, `u(t, x - h, y)` and `u(t, x, y - h)` denote the equivalent of $u^{n}_{i, j}$, $u^{n}_{i-1, j}$ and $u^{n}_{i, j-1}$ analytically and `u[1:, 1:]`, `u[:-1, 1:]` and `u[1:, :-1]` in the numpy implementation respectively.

We can now use this stencil expression to create an operator to apply to our data object:

```python
#NBVAL_IGNORE_OUTPUT
from devito import Operator

# Reset our initial condition in both buffers.
# This is required to avoid 0s propagating into
# our solution, which has a background value of 1.
init_hat(field=u.data[0], dx=dx, dy=dy, value=2.)
init_hat(field=u.data[1], dx=dx, dy=dy, value=2.)

# Create an operator that updates the forward stencil point
op = Operator(Eq(u.forward, stencil, subdomain=grid.interior))

# Apply the operator for a number of timesteps
op(time=nt, dt=dt)

plot_field(u.data[0, :, :])

# Some small sanity checks for the testing framework
assert (u.data[0, 45:55, 45:55] > 1.8).all()
assert np.allclose(u.data[0], u_ref, rtol=3.e-2)
```

Great, that looks to have done the same thing as the original NumPy example, so we seem to be doing something right, at least.

**A note on performance:** During the code generation phase of the previous operator Devito has introduced several performance optimisations. We will ignore this aspect for now, also because our example is tiny - but for large runs where performance matters, these optimisations will make a Devito kernel run very fast in comparison to raw Python/NumPy.

Now, despite getting a correct looking result, there is still one problem with the above operator: It doesn't set any boundary conditions as part of the time loop. We also note that the operator includes a time loop, but at this point Devito doesn't actually provide any language constructs to explicitly define different types of boundary conditions (Devito is probably still a kind of prototype at this point). Luckily though, Devito provides a backdoor for us to insert custom expression in the so-called "indexed" or "low-level" API that allow us to encode the Dirichlet boundary condition of the original example.

#### The "indexed" or low-level API
The `TimeFunction` field we created earlier behaves symbolically like a `sympy.Function` object with the appropriate indices, eg. `u(t, x, y)`. If we take a simple first-order derivative of that we have a term that includes the spacing variable `h`, which Devito uses as the default for encoding $dx$ or $dy$. For example, `u.dx` simply expands to `-u(t, x, y)/h + u(t, x + h, y)/h`.

Now, when the `Operator` creates explicit C-code from that expression, it at some point "lowers" that expression by resolving explicit data accesses (or indices) into our grid by transforming it into a `sympy.Indexed` object. During this process all occurrences of `h` in data accesses get replaced with integers, so that the expression now looks like `-u[t, x, y]/h + u[t, x + 1, y]/h`. This is the "indexed" notation and we can create custom expression of the same kind by explicitly writing `u[...]`, that is with indices in square-bracket notation. These custom expressions can then be injected into our operator like this:

```python
#NBVAL_IGNORE_OUTPUT

# Reset our data field and ICs in both buffers
init_hat(field=u.data[0], dx=dx, dy=dy, value=2.)
init_hat(field=u.data[1], dx=dx, dy=dy, value=2.)

# For defining BCs, we want to explicitly set rows/columns in our field
# We can use Devito's "indexed" notation to do this:
x, y = grid.dimensions
t = grid.stepping_dim
bc_left = Eq(u[t + 1, 0, y], 1.)
bc_right = Eq(u[t + 1, nx-1, y], 1.)
bc_top = Eq(u[t + 1, x, ny-1], 1.)
bc_bottom = Eq(u[t + 1, x, 0], 1.)

# Now combine the BC expressions with the stencil to form operator
expressions = [Eq(u.forward, stencil)]
expressions += [bc_left, bc_right, bc_top, bc_bottom]
op = Operator(expressions=expressions, opt=None, openmp=False)  # <-- Turn off performance optimisations
op(time=nt, dt=dt)

plot_field(u.data[0])

# Some small sanity checks for the testing framework
assert (u.data[0, 45:55, 45:55] > 1.8).all()
assert np.allclose(u.data[0], u_ref, rtol=3.e-2)
```

You might have noticed that we used the argument `opt=None` in the creation of the previous operator. This suppresses the various performance optimisation steps in the code-generation pipeline, which makes the auto-generated C code much easier to look at. So, for the brave, let's have a little peek under the hood...

```python
print(op.ccode)
```

    #define _POSIX_C_SOURCE 200809L
    #define START_TIMER(S) struct timeval start_ ## S , end_ ## S ; gettimeofday(&start_ ## S , NULL);
    #define STOP_TIMER(S,T) gettimeofday(&end_ ## S, NULL); T->S += (double)(end_ ## S .tv_sec-start_ ## S.tv_sec)+(double)(end_ ## S .tv_usec-start_ ## S .tv_usec)/1000000;

    #include "stdlib.h"
    #include "math.h"
    #include "sys/time.h"

    struct dataobj
    {
      void *restrict data;
      int * size;
      int * npsize;
      int * dsize;
      int * hsize;
      int * hofs;
      int * oofs;
    } ;

    struct profiler
    {
      double section0;
      double section1;
      double section2;
    } ;

    int Kernel(const float dt, const float h_x, const float h_y, struct dataobj *restrict u_vec, const int time_M, const int time_m, const int x_M, const int x_m, const int y_M, const int y_m, struct profiler * timers)
    {
      float (*restrict u)[u_vec->size[1]][u_vec->size[2]] __attribute__ ((aligned (64))) = (float (*)[u_vec->size[1]][u_vec->size[2]]) u_vec->data;

      for (int time = time_m, t0 = (time)%(2), t1 = (time + 1)%(2); time <= time_M; time += 1, t0 = (time)%(2), t1 = (time + 1)%(2))
      {
        /* Begin section0 */
        START_TIMER(section0)
        for (int x = x_m; x <= x_M; x += 1)
        {
          for (int y = y_m; y <= y_M; y += 1)
          {
            u[t1][x + 1][y + 1] = dt*(1.0F*u[t0][x + 1][y]/h_y - 1.0F*u[t0][x + 1][y + 1]/h_y + 1.0F*u[t0][x][y + 1]/h_x - 1.0F*u[t0][x + 1][y + 1]/h_x + u[t0][x + 1][y + 1]/dt);
          }
        }
        STOP_TIMER(section0,timers)
        /* End section0 */
        /* Begin section1 */
        START_TIMER(section1)
        for (int y = y_m; y <= y_M; y += 1)
        {
          u[t1][1][y + 1] = 1.00000000000000F;
          u[t1][81][y + 1] = 1.00000000000000F;
        }
        STOP_TIMER(section1,timers)
        /* End section1 */
        /* Begin section2 */
        START_TIMER(section2)
        for (int x = x_m; x <= x_M; x += 1)
        {
          u[t1][x + 1][81] = 1.00000000000000F;
          u[t1][x + 1][1] = 1.00000000000000F;
        }
        STOP_TIMER(section2,timers)
        /* End section2 */
      }

      return 0;
    }


# Simulating Readout Noise

## Theoretical Overview

Qubit-Readout can be corrupted in a variety of ways. The two most relevant error mechanisms on the Rigetti QPU right now are:

1. Transmission line noise that makes a 0-state look like a 1-state or vice versa. We call this **classical readout bit-flip error**. This type of readout noise can be reduced by tailoring optimal readout pulses and using superconducting, quantum limited amplifiers to amplify the readout signal before it is corrupted by classical noise at the higher temperature stages of our cryostats.
2. T1 qubit decay during readout (our readout operations can take more than a µsecond unless they have been specially optimized), which leads to readout signals that initially behave like 1-states but then collapse to something resembling a 0-state. We will call this **T1-readout error**. This type of readout error can be reduced by achieving shorter readout pulses relative to the T1 time, i.e., one can try to reduce the readout pulse length, or increase the T1 time or both.

## Qubit measurements

This section provides the necessary theoretical foundation for accurately modeling noisy quantum measurements on superconducting quantum processors. It relies on some of the abstractions (density matrices, Kraus maps) introduced in our notebook on [gate noise models](GateNoiseModels.ipynb).

The most general type of measurement performed on a single qubit at a single time can be characterized by some set $\mathcal{O}$ of measurement outcomes, e.g., in the simplest case $\mathcal{O} = \{0, 1\}$, and some unnormalized quantum channels (see notebook on gate noise models) that encapsulate
1. the probability of that outcome
2. how the qubit state is affected conditional on the measurement outcome.

Here the _outcome_ is understood as classical information that has been extracted from the quantum system.

### Projective, ideal measurement
The simplest case that is usually taught in introductory quantum mechanics and quantum information courses are Born's rule and the projection postulate which state that there exist a complete set of orthogonal projection operators
$$
P_{\mathcal{O}} := \{\Pi_x \text{ Projector }\mid x \in \mathcal{O}\},
$$
i.e., one for each measurement outcome. Any projection operator must satisfy $\Pi_x^\dagger = \Pi_x = \Pi_x^2$ and for an _orthogonal_ set of projectors any two members satisfy
$$
\Pi_x\Pi_y = \delta_{xy} \Pi_x = \begin{cases} 0 & \text{ if } x \ne y \\ \Pi_x & \text{ if } x = y \end{cases}
$$
and for a _complete_ set we additionally demand that $\sum_{x\in\mathcal{O}} \Pi_x = 1$.
Following our introduction to gate noise, we write quantum states as density matrices as this is more general and in closer correspondence with classical probability theory.

With these the probability of outcome $x$ is given by $p(x) = \tr{\Pi_x \rho \Pi_x} = \tr{\Pi_x^2 \rho} = \tr{\Pi_x \rho}$ and the post measurement state is
$$
\rho_x = \frac{1}{p(x)} \Pi_x \rho \Pi_x,
$$
which is the projection postulate applied to mixed states.

If we were a sloppy quantum programmer and accidentally erased the measurement outcome then our best guess for the post measurement state would be given by something that looks an awful lot like a Kraus map:
$$
\rho_{\text{post measurement}} = \sum_{x\in\mathcal{O}} p(x) \rho_x = \sum_{x\in\mathcal{O}} \Pi_x \rho \Pi_x.
$$
The completeness of the projector set ensures that the trace of the post measurement is still 1 and the Kraus map form of this expression ensures that $\rho_{\text{post measurement}}$ is a positive (semi-)definite operator.

### Classical readout bit-flip error

Consider now the ideal measurement as above, but where the outcome $x$ is transmitted across a noisy classical channel that produces a final outcome $x'\in \mathcal{O}' = \{0', 1'\}$ according to some conditional probabilities $p(x'|x)$ that can be recorded in the _assignment probability matrix_
$$
P_{x'|x} = \begin{pmatrix}
p(0 | 0) & p(0 | 1) \\
p(1 | 0) & p(1 | 1)
\end{pmatrix}
$$
Note that this matrix has only two independent parameters as each column must be a valid probability distribution, i.e. all elements are non-negative and each column sums to 1.

This matrix allows us to obtain the probabilities $\mathbf{p}' := (p(x'=0), p(x'=1))^T$ from the original outcome probabilities $\mathbf{p} := (p(x=0), p(x=1))^T$  via $\mathbf{p}' = P_{x'|x}\mathbf{p}$.
The difference relative to the ideal case above is that now an outcome $x' = 0$ does not necessarily imply that the post measurement state is truly $\Pi_{0} \rho \Pi_{0} / p(x=0)$. Instead, the post measurement state given a noisy outcome $x'$ must be
\begin{align}
\rho_{x'} & = \sum_{x\in \mathcal{O}} p(x|x') \rho_x \\
          & = \sum_{x\in \mathcal{O}} p(x'|x)\frac{p(x)}{p(x')} \rho_x \\
          & = \frac{1}{p(x')}\sum_{x\in \mathcal{O}} p(x'|x) \Pi_x \rho \Pi_x
\end{align}
where
\begin{align}
p(x') & = \sum_{x\in\mathcal{O}} p(x'|x) p(x)  \\
& = \tr{\sum_{x\in \mathcal{O}} p(x'|x) \Pi_x \rho \Pi_x} \\
& = \tr{\rho \sum_{x\in \mathcal{O}} p(x'|x)\Pi_x} \\
& = \tr{\rho E_x'}.
\end{align}
where we have exploited the cyclical property of the trace $\tr{ABC}=\tr{BCA}$ and the projection property $\Pi_x^2 = \Pi_x$. This has allowed us to derive the noisy outcome probabilities from a set of positive operators
$$
E_{x'} := \sum_{x\in \mathcal{O}} p(x'|x)\Pi_x \ge 0
$$
that must sum to 1:
$$
\sum_{x'\in\mathcal{O}'} E_x' = \sum_{x\in\mathcal{O}}\underbrace{\left[\sum_{x'\in\mathcal{O}'} p(x'|x)\right]}_{1}\Pi_x = \sum_{x\in\mathcal{O}}\Pi_x = 1.
$$
The above result is a type of generalized **Bayes' theorem** that is extremely useful for this type of (slightly) generalized measurement and the family of operators $\{E_{x'}| x' \in \mathcal{O}'\}$ whose expectations give the probabilities is called a **positive operator valued measure** (POVM). These operators are not generally orthogonal nor valid projection operators but they naturally arise in this scenario. This is not yet the most general type of measurement, but it will get us pretty far.

### How to model $T_1$ error
T1 type errors fall outside our framework so far as they involve a scenario in which the _quantum state itself_ is corrupted during the measurement process in a way that potentially erases the pre-measurement information as opposed to a loss of purely classical information. The most appropriate framework for describing this is given by that of measurement instruments, but for the practical purpose of arriving at a relatively simple description, we propose describing this by a T1 damping Kraus map followed by the noisy readout process as described above.

### Further reading
Chapter 3 of John Preskill's lecture notes http://www.theory.caltech.edu/people/preskill/ph229/notes/chap3.pdf

## How do I get started?

1. Come up with a good guess for your readout noise parameters $p(0|0)$ and $p(1|1)$, the off-diagonals then follow from the normalization of $P_{x'|x}$. If your assignment fidelity $F$ is given, and you assume that the classical bit flip noise is roughly symmetric, then a good approximation is to set $p(0|0)=p(1|1)=F$.
2. For your QUIL program `p`, and a qubit index `q` call:
    ```
    p.define_noisy_readout(q, p00, p11)
    ```
    where you should replace `p00` and `p11` with the assumed probabilities.

### Estimate $P_{x'|x}$ yourself!
You can also run some simple experiments to estimate the assignment probability matrix directly from a QPU.

**Scroll down for some examples!**

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from pyquil.quil import Program, MEASURE, Pragma
from pyquil.api import QVMConnection
from pyquil.gates import I, X, RX, H, CNOT
from pyquil.noise import (estimate_bitstring_probs, correct_bitstring_probs,
                          bitstring_probs_to_z_moments, estimate_assignment_probs)

DARK_TEAL = '#48737F'
FUSCHIA = '#D6619E'
BEIGE = '#EAE8C6'

cxn = QVMConnection()
```

## Example 1: Rabi sequence with noisy readout

```python
%%time

# number of angles
num_theta = 101

# number of program executions
trials = 200

thetas = np.linspace(0, 2*np.pi, num_theta)

p00s = [1., 0.95, 0.9, 0.8]

results_rabi = np.zeros((num_theta, len(p00s)))

for jj, theta in enumerate(thetas):
    for kk, p00 in enumerate(p00s):
        cxn.random_seed = hash((jj, kk))
        p = Program(RX(theta, 0))
        ro = p.declare("ro")
        # assume symmetric noise p11 = p00
        p.define_noisy_readout(0, p00=p00, p11=p00)
        p.measure(0, ro[0])
        res = cxn.run(p, [0], trials=trials)
        results_rabi[jj, kk] = np.sum(res)

```

    CPU times: user 875 ms, sys: 50.4 ms, total: 925 ms
    Wall time: 2.23 s

```python
plt.figure(figsize=(14, 6))
for jj, (p00, c) in enumerate(zip(p00s, [DARK_TEAL, FUSCHIA, "k", "gray"])):
    plt.plot(thetas, results_rabi[:, jj]/trials, c=c, label=r"$p(0|0)=p(1|1)={:g}$".format(p00))
plt.legend(loc="best")
plt.xlim(*thetas[[0,-1]])
plt.ylim(-.1, 1.1)
plt.grid(alpha=.5)
plt.xlabel(r"RX angle $\theta$ [radian]", size=16)
plt.ylabel(r"Excited state fraction $n_1/n_{\rm trials}$", size=16)
plt.title("Effect of classical readout noise on Rabi contrast.", size=18)
```

## Example 2: Estimate the assignment probabilities

### Estimate assignment probabilities for a perfect quantum computer

```python
estimate_assignment_probs(0, 1000, cxn, Program())
```

    /Users/peter/.pyenv/versions/3.6.3/envs/forest-tutorials/lib/python3.6/site-packages/pyquil/quil.py:979: UserWarning: Please DECLARE all memory. I'm adding a declaration for the `ro` register, but I won't do this for you in the future.
      "Please DECLARE all memory. I'm adding a declaration for the `ro` register, "

    array([[1., 0.],
           [0., 1.]])

### Re-Estimate assignment probabilities for an imperfect quantum computer

```python
cxn.seed = None
header0 = Program().define_noisy_readout(0, .85, .95)
header1 = Program().define_noisy_readout(1, .8, .9)
header2 = Program().define_noisy_readout(2, .9, .85)

ap0 = estimate_assignment_probs(0, 100000, cxn, header0)
ap1 = estimate_assignment_probs(1, 100000, cxn, header1)
ap2 = estimate_assignment_probs(2, 100000, cxn, header2)
```

```python
print(ap0, ap1, ap2, sep="\n")
```

    [[0.84949 0.05007]
     [0.15051 0.94993]]
    [[0.79957 0.10064]
     [0.20043 0.89936]]
    [[0.89964 0.15048]
     [0.10036 0.84952]]

## Example 3: Use `pyquil.noise.correct_bitstring_probs` to correct for noisy readout

### 3a) Correcting the Rabi signal from above

```python
ap_last = np.array([[p00s[-1], 1 - p00s[-1]],
                    [1 - p00s[-1], p00s[-1]]])
corrected_last_result = [correct_bitstring_probs([1-p, p], [ap_last])[1] for p in results_rabi[:, -1] / trials]
```

```python
plt.figure(figsize=(14, 6))
for jj, (p00, c) in enumerate(zip(p00s, [DARK_TEAL, FUSCHIA, "k", "gray"])):
    if jj not in [0, 3]:
        continue
    plt.plot(thetas, results_rabi[:, jj]/trials, c=c, label=r"$p(0|0)=p(1|1)={:g}$".format(p00), alpha=.3)
plt.plot(thetas, corrected_last_result, c="red", label=r"Corrected $p(0|0)=p(1|1)={:g}$".format(p00s[-1]))
plt.legend(loc="best")
plt.xlim(*thetas[[0,-1]])
plt.ylim(-.1, 1.1)
plt.grid(alpha=.5)
plt.xlabel(r"RX angle $\theta$ [radian]", size=16)
plt.ylabel(r"Excited state fraction $n_1/n_{\rm trials}$", size=16)
plt.title("Corrected contrast", size=18)
```

**We find that the corrected signal is fairly noisy (and sometimes exceeds the allowed interval $[0,1]$) due to the overall very small number of samples $n=200$.**

### 3b) In this example we will create a GHZ state $\frac{1}{\sqrt{2}}\left[\left|000\right\rangle + \left|111\right\rangle \right]$ and measure its outcome probabilities with and without the above noise model. We will then see how the Pauli-Z moments that indicate the qubit correlations are corrupted (and corrected) using our API.

```python
ghz_prog = Program()
ro = ghz_prog.declare("ro", "BIT", 3)
ghz_prog += H(0)
ghz_prog += CNOT(0, 1)
ghz_prog += CNOT(1, 2)
ghz_prog += MEASURE(0, ro[0])
ghz_prog += MEASURE(1, ro[1])
ghz_prog += MEASURE(2, ro[2])
print(ghz_prog)
results = cxn.run(ghz_prog, [0, 1, 2], trials=10000)
```

    DECLARE ro BIT[3]
    H 0
    CNOT 0 1
    CNOT 1 2
    MEASURE 0 ro[0]
    MEASURE 1 ro[1]
    MEASURE 2 ro[2]

```python
header = header0 + header1 + header2
noisy_ghz = header + ghz_prog
print(noisy_ghz)
noisy_results = cxn.run(noisy_ghz, [0, 1, 2], trials=10000)
```

    PRAGMA READOUT-POVM 0 "(0.85 0.050000000000000044 0.15000000000000002 0.95)"
    PRAGMA READOUT-POVM 1 "(0.8 0.09999999999999998 0.19999999999999996 0.9)"
    PRAGMA READOUT-POVM 2 "(0.9 0.15000000000000002 0.09999999999999998 0.85)"
    DECLARE ro BIT[3]
    H 0
    CNOT 0 1
    CNOT 1 2
    MEASURE 0 ro[0]
    MEASURE 1 ro[1]
    MEASURE 2 ro[2]

### Uncorrupted probability for $\left|000\right\rangle$ and $\left|111\right\rangle$

```python
probs = estimate_bitstring_probs(results)
probs[0, 0, 0], probs[1, 1, 1]
```

    (0.4959, 0.5041)

As expected the outcomes `000` and `111` each have roughly probability $1/2$.

### Corrupted probability for $\left|000\right\rangle$ and $\left|111\right\rangle$

```python
noisy_probs = estimate_bitstring_probs(noisy_results)
noisy_probs[0, 0, 0], noisy_probs[1, 1, 1]
```

    (0.3093, 0.3611)

The noise-corrupted outcome probabilities deviate significantly from their ideal values!

### Corrected probability for $\left|000\right\rangle$ and $\left|111\right\rangle$

```python
corrected_probs = correct_bitstring_probs(noisy_probs, [ap0, ap1, ap2])
corrected_probs[0, 0, 0], corrected_probs[1, 1, 1]
```

    (0.5054052808756202, 0.4957404969469219)

The corrected outcome probabilities are much closer to the ideal value.

### Estimate $\langle Z_0^{j} Z_1^{k} Z_2^{\ell}\rangle$ for $jkl=100, 010, 001$ from non-noisy data
*We expect these to all be very small*

```python
zmoments = bitstring_probs_to_z_moments(probs)
zmoments[1, 0, 0], zmoments[0, 1, 0], zmoments[0, 0, 1]
```

    (-0.008199999999999985, -0.008199999999999985, -0.008199999999999985)

### Estimate $\langle Z_0^{j} Z_1^{k} Z_2^{\ell}\rangle$ for $jkl=110, 011, 101$ from non-noisy data
*We expect these to all be close to 1.*

```python
zmoments[1, 1, 0], zmoments[0, 1, 1], zmoments[1, 0, 1]
```

    (1.0, 1.0, 1.0)

### Estimate $\langle Z_0^{j} Z_1^{k} Z_2^{\ell}\rangle$ for $jkl=100, 010, 001$ from noise-corrected data

```python
zmoments_corr = bitstring_probs_to_z_moments(corrected_probs)
zmoments_corr[1, 0, 0], zmoments_corr[0, 1, 0], zmoments_corr[0, 0, 1]
```

    (0.005804208050837423, 0.012290215042994768, 0.009450584654813854)

### Estimate $\langle Z_0^{j} Z_1^{k} Z_2^{\ell}\rangle$ for $jkl=110, 011, 101$ from noise-corrected data

```python
zmoments_corr[1, 1, 0], zmoments_corr[0, 1, 1], zmoments_corr[1, 0, 1]
```

    (1.0242766053748604, 0.9909609480429594, 0.9893455578723482)

##### Overall the correction can restore the contrast in our multi-qubit observables, though we also see that the correction can lead to slightly non-physical expectations. This effect is reduced the more samples we take.


```python
import os
import subprocess
import numpy as np
import pandas as pd
import matplotlib as plt
import sympy as sp
import hurwitz as hr

# Tworzymy plik do latexa, ale na poczatku sprawdzamy czy plik o takiej samej nazwie istnieje
# jak tak to go usuwamy

# Ustawiamy s oraz k w SymPy
s = sp.Symbol('s', rational=True)
k = sp.Symbol('k', rational=True, real=True)

# Nasze transmitancje i ich postacie w latexu
K_a = 1/((s+1)*(s+2))
K_b = k
K_c = (s+2)/((s+1)*(s+2))
K_d = 1/(s**2+4*s+5)

LATEX_K_LIST = [str(sp.latex(K_a)), str(sp.latex(K_b)),
                str(sp.latex(K_c)), str(sp.latex(K_d))]
K_LIST = [(K_a), (K_b),
          (K_c), (K_d)]
G_a = k
G_b = 1/((s+1)*(s+2))
G_c = 1/(s+3)
G_d = 1/(s+3)

G_LIST = [(G_a), (G_b),
          (G_c), (G_d)]
LATEX_G_LIST = [str(sp.latex(G_a)), str(sp.latex(G_b)),
                str(sp.latex(G_c)), str(sp.latex(G_d))]

```

```python
LATEX_K_LIST
```

    ['\\frac{1}{\\left(s + 1\\right) \\left(s + 2\\right)}',
     'k',
     '\\frac{1}{s + 1}',
     '\\frac{1}{s^{2} + 4 s + 5}']

```python
LATEX_G_LIST
```

    ['k',
     '\\frac{1}{\\left(s + 1\\right) \\left(s + 2\\right)}',
     '\\frac{1}{s + 3}',
     '\\frac{1}{s + 3}']


# Building Chemical Reaction Networks (CRNs) Directly with BioCRNpyler

**Overview:**  This tutorial shows how to use [BioCRNpyler](https://github.com/BuildACell/BioCRNPyler) to represent simple CRNs

## What is a CRN?
A CRN is a widely established model of chemistry and biochemistry.
* A set of species $S$
* A set of reactions $R$ interconvert species $I_r$ to $O_r$

\begin{align}
\\
I \xrightarrow[]{\rho(s)} O
\\
\end{align}

 * $I$ and $O$ are multisets of species $S$.
 * $\rho(s): S \to \mathbb{R}$ is a function that determines how fast the reaction occurs.

```python
#Import everything from biocrnpyler
from biocrnpyler import *

#Used for simulating and plotting
import numpy as np
import pylab as plt
```

## Combining Species and Reactions into a CRN

The following code defines a species called 'S' made out of material 'material'. Species can also have attributes to help identify them. Note that Species with the same name, but different materials or attributes are considered different species in terms of the reactions they participate in.

    S = Species('name', material_type = 'material', attributes = [])

The collowing code produces a reaction R

    R = Reaction(Inputs, Outputs, k)

here Inputs and Outputs must both be a list of Species. the parameter k is the rate constant of the reaction. By default, propensities in BioCRNpyler are massaction:

### $\rho(S) = k \Pi_{s} s^{I_s}$

Note: for stochastic models mass action propensities are $\rho(S) = k \Pi_{s} s!/(s - I_s)!$.

Massaction reactions can be made reversible with the k_rev keyword:

    R_reversible = Reaction(Inputs, Outputs, k, k_rev = krev)

is the same as two reactions:

    R = Reaction(Inputs, Outputs, k)
    Rrev = Reaction(Outputs, Inputs, krev)

Finally, a CRN can be made by combining species and reactions:

    CRN = ChemicalReactionNetwork(species = species, reactions = reactions)

```python
#Example: Model the CRN consisting of: A --> 2B, 2B <--> B + C where C has the same name as B but a new material
A = Species("A", material_type = "m1", attributes = ["attribute"])
B = Species("B", material_type = "m1")
C = Species("B", material_type = "m2")
D = Species("D")

print("Species can be printed to show their string representation:", A, B, C, D)

#Reaction Rates
k1 = 3.
k2 = 1.4
k2rev = 0.15

#Reaciton Objects
R1 = Reaction([A], [B, B], k1)
R2 = Reaction([B], [C, D], k2)

print("\nReactions can be printed as well:\n", R1,"\n", R2)

#Make a CRN
CRN = ChemicalReactionNetwork(species = [A, B, C, D], reactions = [R1, R2])

#CRNs can be printed in two different ways
print("\nDirectly printing a CRN shows the string representation of the species used in BioCRNpyler:")
print(CRN)

print("\nCRN.pretty_print(...) is a function that prints a more customizable version of the CRN, but doesn't show the proper string representation of species.")
print(CRN.pretty_print(show_materials = True, show_rates = True, show_attributes = True))
```

    Species can be printed to show their string representation: m1_A_attribute m1_B m2_B D

    Reactions can be printed as well:
     m1_A_attribute --> 2 m1_B        massaction: k_f(m1_A_attribute)=3.0*m1_A_attribute
     m1_B --> m2_B + D        massaction: k_f(m1_B)=1.4*m1_B

    Directly printing a CRN shows the string representation of the species used in BioCRNpyler:
    Species = m1_A_attribute, m1_B, m2_B, D
    Reactions = [
    	m1_A_attribute --> 2 m1_B        massaction: k_f(m1_A_attribute)=3.0*m1_A_attribute
    	m1_B --> m2_B + D        massaction: k_f(m1_B)=1.4*m1_B
    ]

    CRN.pretty_print(...) is a function that prints a more customizable version of the CRN, but doesn't show the proper string representation of species.
    Species (4) = {0. m1[A(attribute)], 1. m1[B], 2. m2[B], 3. D}
    Reactions (2) = [
    0. m1[A(attribute)] --> 2 m1[B]
            massaction: k_f(m1[A(attribute)])=3.0*m1[A(attribute)]
    1. m1[B] --> m2[B] + D
            massaction: k_f(m1[B])=1.4*m1[B]
    ]

## CRNs can be saved as SBML and simulated

To save a CRN as SBML:

    CRN.write_sbml_file("file_name.xml")

To simulate a CRN with biosrape:

    Results, Model = CRN_expression.simulate_with_bioscrape(timepoints, initial_condition_dict = x0)

Where x0 is a dictionary: x0 = {str(species_name):initial_value}

```python
#Saving and simulating a CRN
CRN.write_sbml_file("build_crns_directly.xml")

#Initial conditions can be set with a dictionary:
x0 = {str(A):120}
timepoints = np.linspace(0, 1, 100)#Timepoints to simulate over

R = CRN.simulate_with_bioscrape(timepoints = timepoints, initial_condition_dict = x0)
#Can also simulate and save at the same time
#R, M = CRN.simulate_with_bioscrape_via_sbml(timepoints = timepoints, initial_condition_dict = x0, file_name = "build_crns_directly.xml")

if R is not False and R is not None: #Check to ensure simulation worked
    #Results are in a Pandas Dictionary and can be accessed via string-names of species
    plt.plot(R['time'], R[str(A)], label = "A")
    plt.plot(R['time'], R[str(B)], label = "B")
    plt.plot(R['time'], R[str(C)], "--", label = "C")
    plt.plot(R['time'], R[str(D)],":", label = "D")
    plt.legend()
```

## ComplexSpecies and OrderedComplexSpecies

When Species bind together to form complexes, it is recommended to use the ComplexSpecies, Multimer, or OrderedComplexSpecies subclasses which contain information about the species inside of them. ComplexSpecies treats its internal species as an unordered multiset. Multimer contains $n$ compies of a sinlge Species. OrderedComplexSpecies treats its internal species as an ordered list.

_Note: These objects do not automatically generate binding reactions. To do that, use the Component wrappers ChemicalComplex and OrderedChemicalComplex._

```python
A = Species("A", material_type = "m1")
B = Species("B", material_type = "m1")
B2 = Species("B", material_type = "m2")

print("\nFor ComplexSpecies, the order of the elements does not matter:")
C1 = ComplexSpecies([A, B, B2, A])
C2 = ComplexSpecies([B, A, B2, A])
print("C1=ComplexSpecies([A, B, B2, A])=", C1)
print("C2=ComplexSpecies([B, A, B2, A])=", C2)
print("C1==C2 ==>", C1==C2)

print("\nMultimers are the same as ComplexSpecies:")
M1 = ComplexSpecies([A]*3)
M2 = Multimer(A, 3)
print("ComplexSpecies([A]*3)", M1)
print("Multimer(A, 3)", M2)
print("M1 == M2 ==>", M1 == M2)

print("\nFor OrderedComplexSpecies, the Order Does Matter:")
C3 = OrderedComplexSpecies([A, B, B2, A])
C4 = OrderedComplexSpecies([B, A, B2, A])
print("C3=OrderedComplexSpecies([A, B, B2, A])=", C3)
print("C4=OrderedComplexSpecies([B, A, B2, A])=", C4)
print("C3==C4 ==>", C3==C4)

print("\nComplexSpecies (and the OrderedComplexSpecies and Multimers) are Species and can be used in reactions:")
R = Reaction([A, B, B2, A], [C1], k = 10)
print("Reaction([A, B, B2, A], [C1], k = 10)=")
print(R)
```

## Non-massaction propensities in BioCRNpyler
By default, BioCRNpyler assumes that propensities are massaction with only one parameter, the rate constant $k$. However, non-massaction propensities are also supported. However, additional propensity types are also supported. When creating a reaction with non-massaction propensities, a propensity_params dictionary must be passed into the chemical_reaction_network.reaction constructor with the appropriate parameters for that particular propensity type. The supported propensity types and their relevant parameters are:

1. "massaction": $\rho(S) = k \Pi_{s} s^{I_s}$. Required parameters: "k" the rate constant. Note: for stochastic models mass action propensities are $\rho(S) = k \Pi_{s} s!/(s - I_s)!$.
2. "positivehill": $\rho(s) = k \frac{s^n}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".
3. "negativehill": $\rho(s) = k \frac{1}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".
4. "proportionalpositivehill": $\rho(s) = k d \frac{s^n}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", propritional species "d".
5. "proportionalnegativehill": $\rho(s) = k d \frac{1}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", propritional species "d".
6. "general": $\rho(s) = f(s)$ where $f$ can be any algebraic function typed as a string. Required parameters: "rate" an algebraic expression including species and model parameters written as a string.

The following notebook contains examples of the propensity types support by BioCRNpyler applied to a very simple model of genetic activation and repression.

```python
from biocrnpyler.chemical_reaction_network import Species, Reaction, ComplexSpecies, ChemicalReactionNetwork

#Parameter Values
kb = 100
ku = 10
kex = 1.
kd = .1

#Species
G = Species(name = "G", material_type = "dna") #DNA
A = Species(name = "A", material_type = "protein") #Activator
GA = ComplexSpecies([G, A, A]) #Activated Gene
X = Species(name = "X", material_type = "protein")

rxnd = Reaction([X], [], kd)

#Massaction Activation
species1 = [G, A, GA, X]
rxn0_1 = Reaction([G, A, A], [GA], k=kb, k_rev = ku)
rxn0_2 = Reaction([GA], [GA, X], k=kex)
CRN0 = ChemicalReactionNetwork(species1, [rxn0_1, rxn0_2, rxnd])
print("\nMassaction Activation", repr(CRN0))

#Massaction Repressed
rxn1_1 = Reaction([G, A, A], [GA], k=kb, k_rev = ku)
rxn1_2 = Reaction([G], [G, X], k=kex)
CRN1 = ChemicalReactionNetwork(species1, [rxn1_1, rxn1_2, rxnd])
print("\nMassaction Repression", repr(CRN1))

#hill positive
species2 = [G, A, X]
rxn2_1 = Reaction([G], [G, X], propensity_type = "hillpositive", propensity_params = {"k":kex, "n":2, "K":float(kb/ku), "s1":A})
CRN2 = ChemicalReactionNetwork(species2, [rxn2_1, rxnd])
print("\nHill Positive CRN", repr(CRN2))

#proportional hill positive
rxn3_1 = Reaction([G], [G, X], propensity_type = "proportionalhillpositive", propensity_params = {"k":kex, "n":2, "K":float(kb/ku), "s1":A, "d":G})
CRN3 = ChemicalReactionNetwork(species2, [rxn3_1, rxnd])
print("\nProportional Hill Positive CRN", repr(CRN3))

#hill Negative
rxn4_1 = Reaction([G], [G, X], propensity_type = "hillnegative", propensity_params = {"k":kex, "n":2, "K":float(kb/ku), "s1":A})
CRN4 = ChemicalReactionNetwork(species2, [rxn4_1, rxnd])
print("\nHill Negative CRN", repr(CRN4))

#proportional hill negative
rxn5_1 = Reaction([G], [G, X], propensity_type = "proportionalhillnegative", propensity_params = {"k":kex, "n":2, "K":float(kb/ku), "s1":A, "d":G})
CRN5 = ChemicalReactionNetwork(species2, [rxn5_1, rxnd])
print("\nProportional Hill Negative CRN", repr(CRN5))
```

examples/1. Building CRNs Directly.ipynb


Table of Contents

1  Kullback-Leibler Divergence

2  Reference

```python
# code for loading the format for the notebook
import os

# path : store the current path to convert back to it later
path = os.getcwd()
os.chdir(os.path.join('..', 'notebook_format'))

from formats import load_style
load_style(css_style='custom2.css', plot_style=False)
```

```python
os.chdir(path)

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom

# 1. magic for inline plot
# 2. magic to print version
# 3. magic so that the notebook will reload external python modules
# 4. magic to enable retina (high resolution) plots
# https://gist.github.com/minrk/3301035
%matplotlib inline
%load_ext watermark
%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format='retina'
%watermark -a 'Ethen' -d -t -v -p numpy,scipy,matplotlib
```

    Ethen 2018-10-13 21:20:42

    CPython 3.6.4
    IPython 6.4.0

    numpy 1.14.1
    scipy 1.1.0
    matplotlib 2.2.2

# Kullback-Leibler Divergence

In this post we're going to take a look at way of comparing two probability distributions called **Kullback-Leibler Divergence (a.k.a KL divergence)**. Very often in machine learning, we'll replace observed data or a complex distributions with a simpler, approximating distribution. KL Divergence helps us to measure just how much information we lose when we choose an approximation, thus we can even use it as our objective function to pick which approximation would work best for the problem at hand.

Let's look at an example: (The example here is borrowed from the following link. [Blog: Kullback-Leibler Divergence Explained](https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained)).

Suppose we're a group of scientists visiting space and we discovered some space worms. These space worms have varying number of teeth. After a decent amount of collecting, we have come to this empirical probability distribution of the number of teeth in each worm:

```python
# ensure the probability adds up to 1
true_data = np.array([0.02, 0.03, 0.05, 0.14, 0.16, 0.15, 0.12, 0.08, 0.1, 0.08, 0.07])
n = true_data.shape[0]
index = np.arange(n)
assert sum(true_data) == 1.0

# change default style figure and font size
plt.rcParams['figure.figsize'] = 8, 6
plt.rcParams['font.size'] = 12

plt.bar(index, true_data)
plt.xlabel('Teeth Number')
plt.title('Probability Distribution of Space Worm Teeth')
plt.ylabel('Probability')
plt.xticks(index)
plt.show()
```

Now we need to send this information back to earth. But the problem is that sending information from space to earth is expensive. So we wish to represent this information with a minimum amount of information, perhaps just one or two parameters. One option to represent the distribution of teeth in worms is a uniform distribution.

```python
uniform_data = np.full(n, 1.0 / n)

# we can plot our approximated distribution against the original distribution
width = 0.3
plt.bar(index, true_data, width=width, label='True')
plt.bar(index + width, uniform_data, width=width, label='Uniform')
plt.xlabel('Teeth Number')
plt.title('Probability Distribution of Space Worm Teeth')
plt.ylabel('Probability')
plt.xticks(index)
plt.legend()
plt.show()
```

Another option is to use a binomial distribution.

```python
# we estimate the parameter of the binomial distribution
p = true_data.dot(index) / n
print('p for binomial distribution:', p)
binom_data = binom.pmf(index, n, p)
binom_data
```

    p for binomial distribution: 0.49454545454545457

    array([0.00055018, 0.00592134, 0.0289677 , 0.08502751, 0.16638476,
           0.22791121, 0.22299226, 0.15584249, 0.07623949, 0.02486468,
           0.00486561])

```python
width = 0.3
plt.bar(index, true_data, width=width, label='True')
plt.bar(index + width, binom_data, width=width, label='Binomial')
plt.xlabel('Teeth Number')
plt.title('Probability Distribution of Space Worm Teeth')
plt.ylabel('Probability')
plt.xticks(np.arange(n))
plt.legend()
plt.show()
```

Comparing each of our models with our original data we can see that neither one is the perfect match, but the question now becomes, which one is better?

```python
plt.bar(index - width, true_data, width=width, label='True')
plt.bar(index, uniform_data, width=width, label='Uniform')
plt.bar(index + width, binom_data, width=width, label='Binomial')
plt.xlabel('Teeth Number')
plt.title('Probability Distribution of Space Worm Teeth Number')
plt.ylabel('Probability')
plt.xticks(index)
plt.legend()
plt.show()
```

Given these two distributions that we are using to approximate the original distribution, we need a quantitative way to measure which one does the job better. This is where **Kullback-Leibler (KL) Divergence** comes in.

KL Divergence has its origins in information theory. The primary goal of information theory is to quantify how much information is in our data. To recap, one of the most important metric in information theory is called Entropy, which we will denote as $H$. The entropy for a probability distribution is defined as:

\begin{align}
H = -\sum_{i=1}^N p(x_i) \cdot \log p(x_i)
\end{align}

If we use $log_2$ for our calculation we can interpret entropy as, using a distribution $p$, the minimum number of bits it would take us to encode events drawn from distribution $p$. Knowing we have a way to quantify how much information is in our data, we now extend it to quantify how much information is lost when we substitute our observed distribution for a parameterized approximation.

The formula for Kullback-Leibler Divergence is a slight modification of entropy. Rather than just having our probability distribution $p$ we add in our approximating distribution $q$, then we look at the difference of the log values for each:

\begin{align}
D_{KL}(p || q) = \sum_{i=1}^{N} p(x_i)\cdot (\log p(x_i) - \log q(x_i))
\end{align}

Essentially, what we're looking at with KL divergence is the expectation of the log difference between the probability of data in the original distribution with the approximating distribution. Because we're multiplying the difference between the two distribution with $p(x_i)$, this means that matching areas where the original distribution has a higher probability is more important than areas that has a lower probability. Again, if we think in terms of $\log_2$, we can interpret this as, how many extra bits of information we need to encode events drawn from true distribution $p$, if using an optimal code from distribution $q$ rather than $p$.

The more common way to see KL divergence written is as follows:

\begin{align}
D_{KL}(p || q) = \sum_{i=1}^N p(x_i) \cdot \log \frac{p(x_i)}{q(x_i)}
\end{align}

since $\text{log}a - \text{log}b = \text{log}\frac{a}{b}$.

If two distributions, $p$ and $q$ perfectly match, $D_{KL}(p || q) = 0$, otherwise the lower the KL divergence value, the better we have matched the true distribution with our approximation.

Side Note: If you're interested in having an understanding of the relationship between entropy, cross entropy and KL divergence, the following links are good places to start. Maybe they will clear up some of the hand-wavy explanation of these concepts ... [Youtube: A Short Introduction to Entropy, Cross-Entropy and KL-Divergence](https://www.youtube.com/watch?v=ErfnhcEV1O8) and [StackExchange: Why do we use Kullback-Leibler divergence rather than cross entropy in the t-SNE objective function?](https://stats.stackexchange.com/questions/265966/why-do-we-use-kullback-leibler-divergence-rather-than-cross-entropy-in-the-t-sne/265989)

Given these information, we can go ahead and calculate the KL divergence for our two approximating distributions.

```python
# both function are equivalent ways of computing KL-divergence
# one uses for loop and the other uses vectorization
def compute_kl_divergence(p_probs, q_probs):
    """"KL (p || q)"""
    kl_div = 0.0
    for p, q in zip(p_probs, q_probs):
        kl_div += p * np.log(p / q)

    return kl_div

def compute_kl_divergence(p_probs, q_probs):
    """"KL (p || q)"""
    kl_div = p_probs * np.log(p_probs / q_probs)
    return np.sum(kl_div)

print('KL(True||Uniform): ', compute_kl_divergence(true_data, uniform_data))
print('KL(True||Binomial): ', compute_kl_divergence(true_data, binom_data))
```

    KL(True||Uniform):  0.13667971094966938
    KL(True||Binomial):  0.32819435311402045

As we can see the information lost by using the Binomial approximation is greater than using the uniform approximation. If we have to choose one to represent our observations, we're better off sticking with the Uniform approximation.

To close this discussion, we used KL-divergence to calculate which our approximate distribution more closely reflects our true distribution. One caveat to note is that it may be tempting to think of KL-divergence as a way of measuring distance, however, whenever we talk about KL-divergence, we do not categorized it as a distance metric due to the fact that it is asymmetric. In other words, $D_{KL}(p || q) \neq D_{KL}(q || p)$.

# Reference

- [Blog: Kullback-Leibler Divergence Explained](https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained)
- [Youtube: A Short Introduction to Entropy, Cross-Entropy and KL-Divergence](https://www.youtube.com/watch?v=ErfnhcEV1O8)
- [StackExchange: Why do we use Kullback-Leibler divergence rather than cross entropy in the t-SNE objective function?](https://stats.stackexchange.com/questions/265966/why-do-we-use-kullback-leibler-divergence-rather-than-cross-entropy-in-the-t-sne/265989)


# TASI Lecture: Quantum teleportation in a quantum circuit

Author: J. Lykken

This notebook goes with Lecture One, June 23 2020, of
"Quantum Information for Particle Theorists"
at the TASI summer school.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

[Cirq](https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quantumlib_cirq&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=Gl74vKKu9i-SToN0SgQC_w&m=ED3KJ2syYM_OQ64L0CIWSMP_ROvlcoHv83uHUp3_ryg&s=9kHGNyGghHB2ir0kC36e0CFYGNmbEIOnUVnvOrAausM&e=) is a framework for writing quantum algorithms for noisy intermediate scale quantum (NISQ) devices.

---

### Connect to the Google Colab cloud GPU service

```python
#### GOOGLE COLAB SPECIFIC ##########################
# You will need a free Google Colab account or a paid Colab Pro account
# Put this notebook in your Google Drive, then open it using Colab
#
# CODE SNIPPET TO ACCESS THE FILES IN YOUR GOOGLE DRIVE
# IT WILL ASK YOU FOR AUTHORIZATION VIA YOUR GOOGLE ACCOUNT

# Load the Drive helper and mount
from google.colab import drive

# This will prompt for authorization.
# drive.mount('/content/drive')
drive.mount("/content/drive", force_remount=True)

# After executing the cell above, Drive
# files will be present in "/content/drive/My Drive".
# !ls "/content/drive/My Drive"

```

    Mounted at /content/drive

### Installing cirq

Executing the following cell will attempt to install cirq. On Google Colab it may then ask you to restart the runtime; do so.

```python
!pip install cirq --quiet
```

         |████████████████████████████████| 1.6MB 2.7MB/s

To verify that Cirq is installed in your environment, try to `import cirq` and print out a diagram of the 22-qubit Foxtail chip. It should produce a 2x11 grid of qubits.

```python
import cirq
print(cirq.google.Foxtail)
```

    (0, 0)───(0, 1)───(0, 2)───(0, 3)───(0, 4)───(0, 5)───(0, 6)───(0, 7)───(0, 8)───(0, 9)───(0, 10)
    │        │        │        │        │        │        │        │        │        │        │
    │        │        │        │        │        │        │        │        │        │        │
    (1, 0)───(1, 1)───(1, 2)───(1, 3)───(1, 4)───(1, 5)───(1, 6)───(1, 7)───(1, 8)───(1, 9)───(1, 10)

### Import things we might need and define path to our Google Drive directory

```python
# Define a googlepath to your googledrive to load/save files.
# Modify the example below as needed
googlepath = "drive/My Drive/Colab Notebooks/TASI"
from cirq.circuits import InsertStrategy

import tensorflow as tf
import numpy as np
from numpy import around, log2, isreal, all, trace, conj, outer
from math import factorial, sqrt, pi
from cmath import exp
from numpy import matrix, eye, count_nonzero, around, sum, save
import scipy, sympy
from scipy import interpolate
from sympy import Symbol
import matplotlib.pyplot as plt
from matplotlib import gridspec
import gc, random, timeit
from timeit import default_timer as timer
from IPython.display import Markdown, display
def printmd(string):
    display(Markdown(string))
```

### Experiment 1: Simplest single qubit circuit in Cirq
From Cirq tutorial and Jack Hidary's book Chapt 6

```python
# Get a qubit and a circuit
qbit = cirq.LineQubit(0)
circuit = cirq.Circuit()

# Add an X gate: acts like the Pauli Matrix sigma_x
circuit.append(cirq.X(qbit))

# Run a simple simulation that extracts the wavefunction of this state
sim = cirq.Simulator()
result = sim.simulate(circuit)
printmd("\n**Bloch Sphere of the qubit in the final state:**")
state = cirq.bloch_vector_from_state_vector(result.final_state_vector,0)
print("x: ", around(state[0], 4), " y: ", around(state[1], 4),
      " z: ", around(state[2], 4))

# Add a measurement at the end of the circuit:
circuit.append(cirq.measure(qbit, key="Final state"))

# Display the circuit:
printmd("\n**Cirq circuit:**")
print(circuit)

# Invoke the Cirq quantum simulator to execute the circuit:
simulator = cirq.Simulator()

# Simulate the circuit several times:
result = simulator.run(circuit, repetitions=10)

# Print the results:
printmd("\n**Results of 10 trials:**")
print(result)
```

**Bloch Sphere of the qubit in the final state:**

    x:  0.0  y:  0.0  z:  -1.0

**Cirq circuit:**

    0: ───X───M('Final state')───

**Results of 10 trials:**

    Final state=1111111111

### Experiment 2: Circuit to create a Bell state in Cirq
From Jack Hidary's book Chapt 6

```python
# Get two qubits and a circuit
qubit = [cirq.LineQubit(x) for x in range(2)]
circuit = cirq.Circuit()

# Add a Hadamard gate to qubit 0, then a CNOT gate from qubit 0 to qubit 1:
circuit.append([cirq.H(qubit[0]),
             cirq.CNOT(qubit[0], qubit[1])])

# Run a simple simulation that extracts the actual final states
sim = cirq.Simulator()
result = sim.simulate(circuit)
printmd("\n**Bloch Sphere of the qubit 0 in the final state:**")
state = cirq.bloch_vector_from_state_vector(result.final_state_vector,0)
print("x: ", around(state[0], 4), " y: ", around(state[1], 4),
      " z: ", around(state[2], 4))
printmd("\n**Bloch Sphere of the qubit 1 in the final state:**")
state = cirq.bloch_vector_from_state_vector(result.final_state_vector,1)
print("x: ", around(state[0], 4), " y: ", around(state[1], 4),
      " z: ", around(state[2], 4))

# Add a measurement at the end of the circuit:
circuit.append(cirq.measure(*qubit, key="Final state"))

# Display the circuit:
printmd("\n**Cirq circuit:**")
print(circuit)

# Invoke the Cirq quantum simulator to execute the circuit:
simulator = cirq.Simulator()

# Simulate the circuit several times:
result = simulator.run(circuit, repetitions=10)

# Print the results:
printmd("\n**Results:**")
print(result)
```

**Bloch Sphere of the qubit 0 in the final state:**

    x:  0.0  y:  0.0  z:  0.0

**Bloch Sphere of the qubit 1 in the final state:**

    x:  0.0  y:  0.0  z:  0.0

**Cirq circuit:**

    0: ───H───@───M('Final state')───
              │   │
    1: ───────X───M──────────────────

**Results:**

    Final state=1000111111, 1000111111

### Experiment 3: Circuit to SWAP two qubit states

```python
# Get two qubits and a circuit
qubit = [cirq.LineQubit(x) for x in range(2)]
circuit = cirq.Circuit()

# Add a Hadamard gate to make the initial state of qubit 0:
circuit.append([cirq.H(qubit[0])])

# Get a symbol
symbol = Symbol("t")
# Add a parameterized XPowGate to make the initial state of qubit 1:
circuit.append([cirq.XPowGate(exponent=symbol)(qubit[1])])

# Add three CNOT gates to make a SWAP gate:
circuit.append([cirq.CNOT(qubit[0], qubit[1]),
               cirq.CNOT(qubit[1], qubit[0]),
               cirq.CNOT(qubit[0], qubit[1])])

# Measure qubit 1 first, then measure qubit 0:
circuit.append(cirq.measure(qubit[1], key='q1'))
circuit.append(cirq.measure(qubit[0], key='q0'), strategy=InsertStrategy.NEW)

# Display the circuit:
printmd("\n**Cirq circuit:**")
print(circuit)

# Get a sweep over parameter values
sweep = cirq.Linspace(key=symbol.name, start=0.0, stop=1.0, length=3)

# Execute the circuit for all values in the sweep
sim = cirq.Simulator()
results = sim.run_sweep(circuit, sweep, repetitions=50)
printmd("\n**Results for t = 0:**")
print(results[0])
printmd("\n**Results for t = 1:**")
print(results[2])
printmd("\n**Results for t = 0.5:**")
print(results[1])

```

**Cirq circuit:**

    0: ───H─────@───X───@─────────────M('q0')───
                │   │   │
    1: ───X^t───X───@───X───M('q1')─────────────

**Results for t = 0:**

    q0=00000000000000000000000000000000000000000000000000
    q1=10110011100101111100100001001100101011100011110110

**Results for t = 1:**

    q0=11111111111111111111111111111111111111111111111111
    q1=00010001001010010011010001000011000010000001111001

**Results for t = 0.5:**

    q0=00111110100111110101100000110001100001000011001010
    q1=00010001110100101110111001111110001011100101000110

### Experiment 4: Quantum Teleportation
Quantum Teleportation is a process by which a quantum state can be transmitted
by sending only two classical bits of information. This is accomplished by
pre-sharing an entangled state between the sender (Alice) and the receiver
(Bob). This entangled state allows the receiver (Bob) of the two classical
bits of information to possess a qubit with the same state as the one held by
the sender (Alice).

In the following example output, qubit 0 (the Message) is set to a random state
by applying X and Y gates. By sending two classical bits of information after
a Bell State Measurement of qubit 0 (the Message) and qubit 1 (Alice's entangled qubit), the
final state of qubit 2 (Bob's entangled qubit) will be identical to the
original random state of qubit 0 (the Message). This is only possible given
that an entangled state is pre-shared between Alice and Bob.

Example adapted from the Cirq tutorials

```python
# Define three qubits: msg = qubit[0], qalice = qubit[1], qbob = qubit[2]
qubit=[0]*(3)
qubit[0] = cirq.NamedQubit('msg')
qubit[1] = cirq.NamedQubit('qalice')
qubit[2] = cirq.NamedQubit('qbob')

circuit = cirq.Circuit()
# Create a Bell state entangled pair to be shared between Alice and Bob.
circuit.append([cirq.H(qubit[1]), cirq.CNOT(qubit[1], qubit[2])])

# Creates a random state for the Message.
ranX = random.random()
ranY = random.random()
circuit.append([cirq.X(qubit[0])**ranX, cirq.Y(qubit[0])**ranY])

# Unitary operator rotating the two-qubit basis of the Message and Alice's entangled qubit;
# rotates the Bell state basis to the computational basis:
circuit.append([cirq.CNOT(qubit[0], qubit[1]), cirq.H(qubit[0])])
# Combining now with a measurment in the computational basis,
# we effectively have projected this two-qubit state onto one of the four states of
# the Bell state basis:
circuit.append(cirq.measure(qubit[0], qubit[1]))

# Use the two classical bits from the Bell measurement to recover the
# original quantum Message on Bob's entangled qubit.
circuit.append([cirq.CNOT(qubit[1], qubit[2]), cirq.CZ(qubit[0], qubit[2])])

printmd("\n**Cirq circuit:**")
print(circuit)

sim = cirq.Simulator()

# Run a simple simulation that applies the random X and Y gates that
# create our message.
q0 = cirq.LineQubit(0)
message = sim.simulate(cirq.Circuit([cirq.X(q0)**ranX, cirq.Y(q0)**ranY]))

printmd("\n**Bloch Sphere of the Message qubit in the initial state:**")
expected = cirq.bloch_vector_from_state_vector(message.final_state_vector,0)
print("x: ", around(expected[0], 4), " y: ", around(expected[1], 4),
      " z: ", around(expected[2], 4))

# Records the final state of the simulation.
final_results = sim.simulate(circuit)

printmd("\n**Bloch Sphere of Bob's qubit in the final state:**")
teleported = cirq.bloch_vector_from_state_vector(
    final_results.final_state_vector, 2)
print("x: ", around(teleported[0], 4), " y: ",
    around(teleported[1], 4), " z: ", around(teleported[2], 4))

printmd("\n**Bloch Sphere of the Message qubit in the final state:**")
message_final = cirq.bloch_vector_from_state_vector(
    final_results.final_state_vector, 0)
print("x: ", around(message_final[0], 4), " y: ",
    around(message_final[1], 4), " z: ", around(message_final[2], 4))
```

**Cirq circuit:**

    msg: ──────X^0.565───Y^0.176───@───H───M───────@───
                                   │       │       │
    qalice: ───H─────────@─────────X───────M───@───┼───
                         │                     │   │
    qbob: ───────────────X─────────────────────X───@───

**Bloch Sphere of the Message qubit in the initial state:**

    x:  -0.1066  y:  -0.9793  z:  -0.1721

**Bloch Sphere of Bob's qubit in the final state:**

    x:  -0.1066  y:  -0.9793  z:  -0.1721

**Bloch Sphere of the Message qubit in the final state:**

    x:  0.0  y:  0.0  z:  -1.0

### Experiment 5: Quantum Teleportation without measurement

```python
# Define three qubits: msg = qubit[0], qalice = qubit[1], qbob = qubit[2]
qubit=[0]*(3)
qubit[0] = cirq.NamedQubit('msg')
qubit[1] = cirq.NamedQubit('qalice')
qubit[2] = cirq.NamedQubit('qbob')

circuit = cirq.Circuit()
# Create a Bell state entangled pair to be shared between Alice and Bob.
circuit.append([cirq.H(qubit[1]), cirq.CNOT(qubit[1], qubit[2])])

# Creates a random state for the Message.
ranX = random.random()
ranY = random.random()
circuit.append([cirq.X(qubit[0])**ranX, cirq.Y(qubit[0])**ranY])

# Unitary operator rotating the two-qubit basis of the Message and Alice's entangled qubit;
# rotates the Bell state basis to the computational basis:
circuit.append([cirq.CNOT(qubit[0], qubit[1]), cirq.H(qubit[0])])
# But this time skip the measurement
# circuit.append(cirq.measure(qubit[0], qubit[1]))

# Use the same operations as before to recover the
# original quantum Message on Bob's entangled qubit.
circuit.append([cirq.CNOT(qubit[1], qubit[2]), cirq.CZ(qubit[0], qubit[2])])

printmd("**Cirq circuit:**")
print(circuit)

sim = cirq.Simulator()

# Run a simple simulation that applies the random X and Y gates that
# create our message.
q0 = cirq.LineQubit(0)
message = sim.simulate(cirq.Circuit([cirq.X(q0)**ranX, cirq.Y(q0)**ranY]))

printmd("\n**Bloch Sphere of the Message qubit in the initial state:**")
expected = cirq.bloch_vector_from_state_vector(message.final_state_vector,0)
print("x: ", around(expected[0], 4), " y: ", around(expected[1], 4),
      " z: ", around(expected[2], 4))

# Records the final state of the simulation.
final_results = sim.simulate(circuit)

printmd("\n**Bloch Sphere of Bob's qubit in the final state:**")
bob_final = cirq.bloch_vector_from_state_vector(
    final_results.final_state_vector, 2)
print("x: ", around(bob_final[0], 4), " y: ",
    around(bob_final[1], 4), " z: ", around(bob_final[2], 4))

printmd("\n**Bloch Sphere of the Message qubit in the final state:**")
message_final = cirq.bloch_vector_from_state_vector(
    final_results.final_state_vector, 0)
print("x: ", around(message_final[0], 4), " y: ",
    around(message_final[1], 4), " z: ", around(message_final[2], 4))

printmd("\n**Bloch Sphere of the Alice qubit in the final state:**")
alice_final = cirq.bloch_vector_from_state_vector(
    final_results.final_state_vector, 1)
print("x: ", around(alice_final[0], 4), " y: ",
    around(alice_final[1], 4), " z: ", around(alice_final[2], 4))
```

**Cirq circuit:**

    msg: ──────X^0.531───Y^0.277───@───H───@───
                                   │       │
    qalice: ───H─────────@─────────X───@───┼───
                         │             │   │
    qbob: ───────────────X─────────────X───@───

**Bloch Sphere of the Message qubit in the initial state:**

    x:  -0.0732  y:  -0.9954  z:  -0.0618

**Bloch Sphere of Bob's qubit in the final state:**

    x:  -0.0732  y:  -0.9954  z:  -0.0618

**Bloch Sphere of the Message qubit in the final state:**

    x:  1.0  y:  0.0  z:  0.0

**Bloch Sphere of the Alice qubit in the final state:**

    x:  1.0  y:  0.0  z:  0.0