## Interactive Variogram Calculation and Modeling Demonstration

### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### The Interactive Workflow

Here's an interactive workflow for calculating directional experimental variograms in 2D.

* setting the variogram calculation parameters for identifying spatial data pairs

This approach is essential for quantifying spatial continuity with sparsely sampled, irregular spatial data.

I have more comprehensive workflows for variogram calculation:

* [Experimental Variogram Calculation in Python with GeostatsPy](https://github.com/GeostatsGuy/PythonNumericalDemos/blob/master/GeostatsPy_variogram_calculation.ipynb)

* [Determination of Major and Minor Spatial Continuity Directions in Python with GeostatsPy](https://github.com/GeostatsGuy/PythonNumericalDemos/blob/master/GeostatsPy_spatial_continuity_directions.ipynb)

#### Spatial Continuity

**Spatial Continuity** is the correlation between values over distance.

* No spatial continuity – no correlation between values over distance, random values at each location in space regardless of separation distance.

* Homogenous phenomenon have perfect spatial continuity, since all values as the same (or very similar) they are correlated.

We need a statistic to quantify spatial continuity! A convenient method is the Semivariogram.

#### The Semivariogram

Function of difference over distance.

* The expected (average) squared difference between values separated by a lag distance vector (distance and direction), $h$:

\begin{equation}
\gamma(\bf{h}) = \frac{1}{2 N(\bf{h})} \sum^{N(\bf{h})}_{\alpha=1} (z(\bf{u}_\alpha) - z(\bf{u}_\alpha + \bf{h}))^2
\end{equation}

where $z(\bf{u}_\alpha)$ and $z(\bf{u}_\alpha + \bf{h})$ are the spatial sample values at tail and head locations of the lag vector respectively.

* Calculated over a suite of lag distances to obtain a continuous function.

* the $\frac{1}{2}$ term converts a variogram into a semivariogram, but in practice the term variogram is used instead of semivariogram.
* We prefer the semivariogram because it relates directly to the covariance function, $C_x(\bf{h})$ and univariate variance, $\sigma^2_x$:

\begin{equation}
C_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

Note the correlogram is related to the covariance function as:

\begin{equation}
\rho_x(\bf{h}) = \frac{C_x(\bf{h})}{\sigma^2_x}
\end{equation}

The correlogram provides of function of the $\bf{h}-\bf{h}$ scatter plot correlation vs. lag offset $\bf{h}$.

\begin{equation}
-1.0 \le \rho_x(\bf{h}) \le 1.0
\end{equation}

#### Variogram Observations

The following are common observations for variograms that should assist with their practical use.

##### Observation \#1 - As distance increases, variability increase (in general).

This is common since in general, over greater distance offsets, there is often more difference between the head and tail samples.

In some cases, such as with spatial cyclicity of the hole effect variogram model the variogram may have negative slope over somelag distance intervals

Negative slopes at lag distances greater than half the data extent are often caused by too few pairs for a reliable variogram calculation

##### Observation \#2 - Calculated with over all possible pairs separated by lag vector, $\bf{𝐡}$.

We scan through the entire data set, searching for all possible pair combinations with all other data.  We then calculate the variogram as one half the expectation of squared difference between all pairs.

More pairs results in a more reliable measure.

##### Observation \#3 - Need to plot the sill to know the degree of correlation.

**Sill** is the variance, $\sigma^2_x$

Given stationarity of the variance, $\sigma^2_x$, and variogram $\gamma(\bf{h})$:

we can define the covariance function:

\begin{equation}
C_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

The covariance measure is a measure of similarity over distance (the mirror image of the variogram as shown by the equation above).

Given a standardized distribution $\sigma^2_x = 1.0$, the covariance, $C_x(\bf{h})$, is equal to the correlogram, $\rho_x(\bf{h})$:

\begin{equation}
\rho_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

##### Observation \#4 - The lag distance at which the variogram reaches the sill is know as the range.

At the range, knowing the data value at the tail location provides no information about a value at the head location of the lag distance vector.

##### Observation \#5 - The nugget effect, a discontinuity at the origin

Sometimes there is a discontinuity in the variogram at distances less than the minimum data spacing.  This is known as **nugget effect**.

The ratio of nugget / sill, is known as relative nugget effect (%). Modeled as a discontinuity with no correlation structure that at lags, $h \gt \epsilon$, an infinitesimal lag distance, and perfect correlation at $\bf{h} = 0$.
Caution when including nuggect effect in the variogram model as measurement error, mixing populations cause apparent nugget effect

This exercise demonstrates the semivariogram calculation with GeostatsPy. The steps include:

1. generate a 2D model with sequential Gaussian simulation
2. sample from the simulation
3. calculate and visualize experimental semivariograms

#### Variogram Calculation Parameters

The variogram calculation parameters include:

* **azimuth** is the azimuth of the lag vector

* **azimuth tolerance** is the maximum allowable departure from the azimuth (isotropic variograms are calculated with an azimuth tolerance of to 90.0)

* **unit lag distance** the size of the bins in lag distance, usually set to the minimum data spacing

* **lag distance tolerance** - the allowable tolerance in lage distance, commonly set to 50% of unit lag distanceonal smoothing

* **number of lags** - set based on the spatial extent of the dataset, we can typically calculate reliable variograms up to 1/2 the extent of the dataset

* **bandwidth** is the maximum offset allowable from the lag vector

#### Variogram Modeling

Spatial continuity can be modeled with nested, positive definate variogram structures:

\begin{equation}
\Gamma_x(\bf{h}) = \sum_{i=1}^{nst} \gamma_i(\bf{h})
\end{equation}

where $\Gamma_x(\bf{h})$ is the nested variogram model resulting from the summation of $nst$ nested variograms  $\gamma_i(\bf{h})$.

The types of structure commonly applied include:

* spherical

* exponential

* gaussian

* nugget

Other less common models include:

* hole effect

* dampenned hole effect

* power law

these will not be covered here.

Each one of these variogram structures, $\gamma_i(\bf{h})$, is based on a geometric anisotropy model parameterized by the orientation and range in the major and minor directions.  In 2D this is simply an azimuth and ranges, $azi$, $a_{maj}$ and $a_{min}$. Note, the range in the minor direction (orthogonal to the major direction).

The geometric anisotropy model assumes that the range in all off-diagonal directions is based on an ellipse with the major and minor axes alligned with and set to the major and minor for the variogram.

\begin{equation}
\bf{h}_i = \sqrt{\left(\frac{r_{maj}}{a_{maj_i}}\right)^2 + \left(\frac{r_{maj}}{a_{maj_i}}\right)^2}
\end{equation}

Therefore, if we know the major direction, range in major and minor directions, we may completely describe each nested componnent of the complete spatial continuity of the variable of interest, $i = 1,\dots,nst$.

Some comments on modeling nested variograms:

* we can capture nugget, short and long range continuity structures

* we rely on the geometric anisotropy model, so all structures must inform the same level of contribution (porportion of the sill) in all directions.

* the geometric anisotropy model is based on azimuth of the major direction of continuity, range in the major direction and range in the minor direction (orthogonal to the major direction).  The range is interpolated between the major and minor azimuths with a ellipse model

* we can vary the type of variogram, direction or azimuth of the major direction, and major and minor ranges by structure

In this workflow we will explore methods to:

1. detect directionality from a spatial dataset
2. calculate the directional variograms in the major and minor directions
3. build a consistent 2D model fit to the major and minor directions

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data.csv at https://git.io/fh4gm.

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import geostatspy.GSLIB as GSLIB                       # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                 # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
%matplotlib inline
import os                                               # to set current working directory
import sys                                              # supress output to screen for interactive variogram modeling
import io
import numpy as np                                      # arrays and matrix math
import pandas as pd                                     # DataFrames
import matplotlib.pyplot as plt                         # plotting
from matplotlib.pyplot import cm                        # color maps
from ipywidgets import interactive                      # widgets and interactivity
from ipywidgets import widgets
from ipywidgets import Layout
from ipywidgets import Label
from ipywidgets import VBox, HBox
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see above) GSLIB executables in this directory or a location identified in the environmental variable *Path*.

```python
#os.chdir("c:/PGE383")                                   # set the working directory
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.

```python
data = 4

if data == 1:
    df = pd.read_csv("https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/sample_data_MV_biased.csv") # load from Prof. Pyrcz's GitHub repository
    df = df.rename(columns = {'Por':'Porosity'})            # rename feature(s)
    df['Porosity'] = df['Porosity']*100.0
    df = df.iloc[:,1:]                                      # remove first column
elif data == 2:
    df = pd.read_csv("https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/spatial_nonlinear_MV_facies_v3.csv") # load from Prof. Pyrcz's GitHub repository
    df = df.rename(columns = {'Por':'Porosity'})            # rename feature(s)
    df = df.iloc[:,1:]
elif data == 3:
    df = pd.read_csv("https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/12_sample_data.csv") # load from Prof. Pyrcz's GitHub repository
    df = df.rename(columns = {'Por':'Porosity'})            # rename feature(s)
    df['Porosity'] = df['Porosity']*100.0
    df = df.iloc[:,1:]
else:
    df = pd.read_csv("https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/spatial_nonlinear_MV_facies_v5_sand_only.csv") # load from Prof. Pyrcz's GitHub repository
    df = df.rename(columns = {'Por':'Porosity'})            # rename feature(s)
    df = df.iloc[:,1:]
df.head()                                               # we could also use this command for a table preview
```

      X
      Y
      Porosity
      Perm
      AI

      0
      150
      909
      8.947420
      379.859420
      4081.726625

      1
      170
      929
      8.650138
      343.878040
      4152.949017

      2
      825
      875
      6.577249
      934.495444
      4522.980121

      3
      580
      569
      16.480268
      201.505057
      3608.928335

      4
      775
      225
      14.964127
      674.097711
      3919.297374

The features:

* **X** - x coordinate in meters
* **Y** - y coordinate in meters
* **Porosity** - rock porosity averaged over a specific rock unit from a vertical well
* **Perm** - rock permeability averaged (scaled up) over a specific rock unit from a vertical well
* **AI** - acoustic impedance from a seismic cube assigned at a specific rock unit and at the location of a vertical well
* **facies** - facies, 0 - shale, 1 - sandstone

Concerning facies:

We will work with all facies pooled together. I wanted to simplify this workflow and focus more on spatial continuity direction detection. Finally, by not using facies we do have more samples to support our statistical inference. Most often facies are essential in the subsurface model. Don't worry we will check if this is reasonable in a bit.

You are welcome to repeat this workflow on a by-facies basis.  The following code could be used to build DataFrames ('df_sand' and 'df_shale') for each facies.

```p
df_sand = pd.DataFrame.copy(df[df['Facies'] == 1]).reset_index()  # copy only 'Facies' = sand records
df_shale = pd.DataFrame.copy(df[df['Facies'] == 0]).reset_index() # copy only 'Facies' = shale records
```

Let's look at summary statistics for all facies combined:

```python
df.describe().transpose()                               # summary table of sand only DataFrame statistics
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X
      450.0
      470.444444
      288.791974
      0.000000
      221.250000
      475.000000
      725.000000
      980.000000

      Y
      450.0
      516.160000
      283.500794
      9.000000
      275.000000
      525.000000
      756.500000
      999.000000

      Porosity
      450.0
      11.626604
      3.237444
      4.378792
      9.246387
      11.283306
      13.656570
      20.835838

      Perm
      450.0
      442.290250
      286.734825
      0.000000
      226.181214
      422.573712
      628.742290
      1437.594828

      AI
      450.0
      4172.222166
      290.509566
      3493.755740
      3973.941281
      4158.552677
      4370.203466
      5071.276705

#### Set the Model Parameters

See the the following model parameters:

* **xmin**, **xmax**, **ymin** and **ymax** - extents of the dataset for plotting
* **feature** and **feature_units** - feature of interest and associated units
* **vmin** and **vmax** - minimum and maximum of the feature of interest

```python
xmin = 0.0; xmax = 1000.0                                # spatial extents in x and y
ymin = 0.0; ymax = 1000.0
feature = 'Porosity'; feature_units = 'percentage'         # name and units of the feature of interest
vmin = 0.0; vmax = 25.0                                  # min and max of the feature of interest
cmap = plt.cm.inferno                                    # set the color map
```

Let's transform the feature to standard normal (mean = 0.0, standard deviation = 1.0, Gaussian shape). This is required for sequential Gaussian simulation (common target for our variogram models) and the Gaussian transform assists with outliers and provides more interpretable variograms.

Let's look at the inputs for the GeostatsPy nscore program.  Note the output include an ndarray with the transformed values (in the same order as the input data in Dataframe 'df' and column 'vcol'), and the transformation table in original values and also in normal score values.

```python
geostats.nscore                                         # see the input parameters required by the nscore function
```

    <function geostatspy.geostats.nscore(df, vcol, wcol=None, ismooth=False, dfsmooth=None, smcol=0, smwcol=0)>

The following command will transform the Porosity and Permeabilty to standard normal.

```python
#Transform to Gaussian by Facies
df['N' + feature], tvPor, tnsPor = geostats.nscore(df, feature) # nscore transform for all facies porosity
```

Let's look at the updated DataFrame to make sure that we now have the normal score porosity and permeability.

```python
df.head()                                               # preview sand DataFrame with nscore transforms
```

      X
      Y
      Porosity
      Perm
      AI
      NPorosity

      0
      150
      909
      8.947420
      379.859420
      4081.726625
      -0.783500

      1
      170
      929
      8.650138
      343.878040
      4152.949017
      -0.853588

      2
      825
      875
      6.577249
      934.495444
      4522.980121
      -1.737935

      3
      580
      569
      16.480268
      201.505057
      3608.928335
      1.354567

      4
      775
      225
      14.964127
      674.097711
      3919.297374
      0.971878

That looks good! One way to check is to see if the relative magnitudes of the normal score transformed values match the original values.  e.g. that the normal score transform of 0.10 porosity normal score is less than the normal score transform of 0.14 porsity.  Also, the normal score transform of values close to the mean value should be close to 0.0

Let's also check the original and transformed sand and shale porosity distributions.

```python
plt.subplot(121)                                        # plot original sand and shale porosity histograms
plt.hist(df[feature], facecolor='red',bins=np.linspace(vmin,vmax,1000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black')
plt.xlim([vmin,vmax]); plt.ylim([0,1.0])
plt.xlabel(feature + '(' + feature_units + ')'); plt.ylabel('Frequency'); plt.title('Porosity')
plt.grid(True)

plt.subplot(122)
plt.hist(df['N'+feature], facecolor='blue',bins=np.linspace(-3.0,3.0,1000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black')
plt.xlim([-3.0,3.0]); plt.ylim([0,1.0])
plt.xlabel('Gaussian Transformed ' + feature); plt.ylabel('Frequency'); plt.title('Guassian Transformed ' + feature)
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

We can see that the normal score transform has correctly transformed the feature to standard normal.

#### Inspection of Posted Data

Data visualization is very useful to detect patterns. Our brains are very good at pattern detection. I promote quantitative methods and recognize issues with cognitive bias, but it is important to recognize the value is expert intepretation based on data visualization.

* This data visualization will also be important to assist with parameter selection for the variogram calculation search template.

Let's plot the location maps of the original feature and the normal score transforms of the feature.

```python
plt.subplot(121)                                        # location map of normal score transform of porosity
GSLIB.locmap_st(df,'X','Y',feature,0,1000,0,1000,vmin,vmax,feature,'X (m)','Y (m)',feature,cmap)

plt.subplot(122)                                        # location map of normal score transform of porosity
GSLIB.locmap_st(df,'X','Y','N'+feature,0,1000,0,1000,-3,3,'Gaussian Transformed ' + feature,'X (m)','Y (m)','Gaussian Transformed ' + feature,cmap)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.0, wspace=0.5, hspace=0.3)
plt.show()
```

What do you see?  Here's my observations:

* there is a high degree of spatial agreement between porosity and permeability, this is supported by the high correlation evident in the cross plot.
* there are no discontinuities that could suggest that facies represent a distinct change, rather the porosity and permeability seem continuous and the assigned facies are a truncation of their continous behavoir, we doing 'ok' with no facies
* suspect a 045 azimuth major direction of continuity (up - right)
* there may be cycles in the 135 azimuth
* there will not likely be a nugget effect, but there is an hint of some short scale discontinuity?

**Do you agree?** If you have a different observations, drop me a line at mpyrcz@austin.utexas.edu and I'll add to this lesson with credit!

#### Experimental Variograms

We can use the location maps to help determine good variogram calculation parameters. For example:

```p
tmin = -9999.; tmax = 9999.;
lag_dist = 100.0; lag_tol = 50.0; nlag = 7; bandh = 9999.9; azi = azi; atol = 22.5; isill = 1
```
* **tmin**, **tmax** are trimming limits - set to have no impact, no need to filter the data
* **lag_dist**, **lag_tol** are the lag distance, lag tolerance - set based on the common data spacing (100m) and tolerance as 100% of lag distance for additonal smoothing
* **nlag** is number of lags - set to extend just past 50 of the data extent
* **bandh** is the horizontal band width - set to have no effect
* **azi** is the azimuth -  it has not effect since we set atol, the azimuth tolerance, to 90.0
* **isill** is a boolean to standardize the distribution to a variance of 1 - it has no effect since the previous nscore transform sets the variance to 1.0

#### Dashboard for Interactive Variogram Calculation and Modeling

Below we make a dashboard with the ipywidgets and matplotlib Python packages for calculating and modeling experimental variograms.

* allowing you to calculate and model the variogram of the normal score transformed variogram interactively while changing (and exploring) the search template parameters.

* first calculate the isotropic or directional variogram(s)

* then fit the same isotropic or directional variogram(s)

```python
# interactive calculation of the experimental variogram
l = widgets.Text(value='                              Variogram Calculation Interactive Demonstration, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))
lag = widgets.FloatSlider(min = 10, max = 500, value = 10, step = 10, description = 'lag',orientation='vertical',layout=Layout(width='90px', height='200px'))
lag.style.handle_color = 'gray'

lag_tol = widgets.FloatSlider(min = 5, max = 500, value = 5, step = 10, description = 'lag tolerance',orientation='vertical',layout=Layout(width='90px', height='200px'))
lag_tol.style.handle_color = 'gray'

nlag = widgets.IntSlider(min = 1, max = 100, value = 100, step = 1, description = 'number of lags',orientation='vertical',layout=Layout(width='90px', height='200px'))
nlag.style.handle_color = 'gray'

azi = widgets.FloatSlider(min = 0, max = 360, value = 0, step = 5, description = 'azimuth',orientation='vertical',layout=Layout(width='90px', height='200px'))
azi.style.handle_color = 'gray'

azi_tol = widgets.FloatSlider(min = 10, max = 90, value = 10, step = 5, description = 'azimuth tolerance',orientation='vertical',layout=Layout(width='120px', height='200px'))
azi_tol.style.handle_color = 'gray'

bandwidth = widgets.FloatSlider(min = 100, max = 2000, value = 2000, step = 100, description = 'bandwidth',orientation='vertical',layout=Layout(width='90px', height='200px'))
azi_tol.style.handle_color = 'gray'

ui1 = widgets.HBox([lag,lag_tol,nlag,azi,azi_tol,bandwidth],) # basic widget formatting
ui = widgets.VBox([l,ui1],)

def f_make(lag,lag_tol,nlag,azi,azi_tol,bandwidth):     # function to take parameters, calculate variogram and plot
#    text_trap = io.StringIO()
#    sys.stdout = text_trap
    global lags,gammas,npps,lags2,gammas2,npps2
    tmin = -9999.9; tmax = 9999.9
    lags, gammas, npps = geostats.gamv(df,"X","Y","N"+feature,tmin,tmax,lag,lag_tol,nlag,azi,azi_tol,bandwidth,isill=1.0)
    lags2, gammas2, npps2 = geostats.gamv(df,"X","Y","N"+feature,tmin,tmax,lag,lag_tol,nlag,azi+90.0,azi_tol,bandwidth,isill=1.0)

    plt.subplot(111)                                    # plot experimental variogram
    plt.scatter(lags,gammas,color = 'black',s = npps*0.03,label = 'Major Azimuth ' +str(azi), alpha = 0.8)
    plt.scatter(lags2,gammas2,color = 'red',s = npps*0.03,label = 'Minor Azimuth ' +str(azi+90.0), alpha = 0.8)
    plt.plot([0,2000],[1.0,1.0],color = 'black')
    plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
    plt.ylabel(r'$\gamma \bf(h)$')
    if azi_tol < 90.0:
        plt.title('Directional NSCORE ' + feature + ' Variogram - Azi. ' + str(azi) + ', Azi. Tol.' + str(azi_tol))
    else:
        plt.title('Omni Directional NSCORE ' + feature + ' Variogram ')
    plt.xlim([0,1000]); plt.ylim([0,1.8])
    plt.legend(loc="lower right")
    plt.grid(True)

    plt.subplots_adjust(left=0.0, bottom=0.0, right=1.5, top=1.0, wspace=0.3, hspace=0.3)
    plt.show()

    return

# connect the function to make the samples and plot to the widgets
interactive_plot = widgets.interactive_output(f_make, {'lag':lag,'lag_tol':lag_tol,'nlag':nlag,'azi':azi,'azi_tol':azi_tol,'bandwidth':bandwidth})
interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

### Interactive Variogram Calculation Demonstration

* calculate omnidirectional and direction experimental variograms

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

### The Problem

Calculate interpretable experimental variograms for sparse, irregularly-space spatial data. Note, size of the experimental point is scaled by the number of pairs.

* **azimuth** is the azimuth of the lag vector

* **azimuth tolerance** is the maximum allowable departure from the azimuth

* **unit lag distance** the size of the bins in lag distance

* **lag distance tolerance** - the allowable tolerance in lage distance

* **number of lags** - number of lags in the experimental variogram

* **bandwidth** - maximum departure from the lag vector

```python
display(ui, interactive_plot)                           # display the interactive plot
```

    VBox(children=(Text(value='                              Variogram Calculation Interactive Demonstration, Mich…

    Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '
', 'i…

```python
# interactive calculation of the sample set (control of source parametric distribution and number of samples)
l = widgets.Text(value='                              Variogram Modeling, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))
nug = widgets.FloatSlider(min = 0, max = 1.0, value = 0.0, step = 0.1, description = 'nug',orientation='vertical',layout=Layout(width='60px', height='200px'))
nug.style.handle_color = 'gray'
it1 = widgets.Dropdown(options=['Spherical', 'Exponential', 'Gaussian'],value='Spherical',
    description='Type1:',disabled=False,layout=Layout(width='200px', height='30px'))
c1 = widgets.FloatSlider(min=0.0, max = 1.0, value = 0.1, description = 'c1',orientation='vertical',layout=Layout(width='60px', height='200px'))
c1.style.handle_color = 'gray'
hmaj1 = widgets.FloatSlider(min=0.01, max = 10000.0, value = 0.01, step = 25.0, description = 'hmaj1',orientation='vertical',layout=Layout(width='60px', height='200px'))
hmaj1.style.handle_color = 'black'
hmin1 = widgets.FloatSlider(min = 0.01, max = 10000.0, value = 0.01, step = 25.0, description = 'hmin1',orientation='vertical',layout=Layout(width='60px', height='200px'))
hmin1.style.handle_color = 'red'

it2 = widgets.Dropdown(options=['Spherical', 'Exponential', 'Gaussian'],value='Spherical',
    description='Type2:',disabled=False,layout=Layout(width='200px', height='30px'))
c2 = widgets.FloatSlider(min=0.0, max = 1.0, value = 0.0, description = 'c2',orientation='vertical',layout=Layout(width='60px', height='200px'))
c2.style.handle_color = 'gray'
hmaj2 = widgets.FloatSlider(min=0.01, max = 10000.0, value = 0.01, step = 100.0, description = 'hmaj2',orientation='vertical',layout=Layout(width='60px', height='200px'))
hmaj2.style.handle_color = 'black'
hmin2 = widgets.FloatSlider(min = 0.01, max = 10000.0, value = 0.01, step = 100.0, description = 'hmin2',orientation='vertical',layout=Layout(width='60px', height='200px'))
hmin2.style.handle_color = 'red'

ui1 = widgets.HBox([nug,it1,c1,hmaj1,hmin1,it2,c2,hmaj2,hmin2],)                   # basic widget formatting
#ui2 = widgets.HBox([it2,c2,hmaj2,hmin2],)                   # basic widget formatting
ui = widgets.VBox([l,ui1],)

def convert_type(it):
    if it == 'Spherical':
        return 1
    elif it == 'Exponential':
        return 2
    else:
        return 3

def f_make(nug,it1,c1, hmaj1, hmin1, it2, c2, hmaj2, hmin2):                       # function to take parameters, make sample and plot
    text_trap = io.StringIO()
    sys.stdout = text_trap

    it1 = convert_type(it1); it2 = convert_type(it2)
    if c2 > 0.0:
        nst = 2
    else:
        nst = 1

    vario = GSLIB.make_variogram(nug,nst,it1,c1,0.0,hmaj1,hmin1,it2,c2,0.0,hmaj2,hmin2) # make model object
    nlag = 100; xlag = 10;
    index_maj,h_maj,gam_maj,cov_maj,ro_maj = geostats.vmodel(nlag,xlag,0.0,vario)   # project the model in the major azimuth                                                  # project the model in the 135 azimuth
    index_min,h_min,gam_min,cov_min,ro_min = geostats.vmodel(nlag,xlag,90.0,vario)

    plt.subplot(111)                                    # plot experimental variogram
    plt.scatter(lags,gammas,color = 'black',s = npps*0.03,label = 'Major Azimuth ' +str(azi.value), alpha = 0.8)
    plt.plot(h_maj,gam_maj,color = 'black')
    plt.scatter(lags2,gammas2,color = 'red',s = npps*0.03,label = 'Minor Azimuth ' +str(azi.value+90.0), alpha = 0.8)
    plt.plot(h_min,gam_min,color = 'red')
    plt.plot([0,2000],[1.0,1.0],color = 'black')
    plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
    plt.ylabel(r'$\gamma \bf(h)$')
    if azi_tol.value < 90.0:
        plt.title('Directional NSCORE ' + feature + ' Variogram - Azi. ' + str(azi.value) + ', Azi. Tol.' + str(azi_tol.value))
    else:
        plt.title('Omni Directional NSCORE ' + feature + ' Variogram ')
    plt.xlim([0,1000]); plt.ylim([0,1.8])
    plt.legend(loc="lower right")
    plt.grid(True)

    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.2, top=1.5, wspace=0.3, hspace=0.3)
    plt.show()

# connect the function to make the samples and plot to the widgets
interactive_plot = widgets.interactive_output(f_make, {'nug':nug, 'it1':it1,'c1':c1, 'hmaj1':hmaj1, 'hmin1':hmin1, 'it2':it2, 'c2':c2, 'hmaj2':hmaj2, 'hmin2':hmin2})
interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

### Interactive Nested Variogram Modeling Demostration

* select the nested structures and their types, contributions and major and minor ranges

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

### The Problem

Fit a positive definite variogram model based on the addition of multiple structures each describing spatial components of the feature variance

* **nug**: nugget effect

* **c1 / c2**: contributions of the sill

* **hmaj1 / hmaj2**: range in the major direction

* **hmin1 / hmin2**: range in the minor direction

```python
display(ui, interactive_plot)                           # display the interactive plot
```

    VBox(children=(Text(value='                              Variogram Modeling, Michael Pyrcz, Associate Professo…

    Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '
', 'i…

#### Comments

This was a basic demonstration / exercise of variogram calculation and modeling for spatial continuity analysis. Much more could be done, I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)


# Circuit Quantum Electrodynamics

## Contents

1. [Introduction](#intro)
2. [The Schrieffer-Wolff Transformation](#tswt)
3. [Block-diagonalization of the Jaynes-Cummings Hamiltonian](#bdotjch)
4. [Full Transmon](#full-transmon)
5. [Qubit Drive with cQED](#qdwcqed)
6. [The Cross Resonance Entangling Gate](#tcreg)

## 1. Introduction

By analogy with Cavity Quantum Electrodynamics (CQED), circuit QED (cQED) exploits the fact that a simple model can be used to both describe the interaction of an atom with an optical cavity and a qubit with a microwave resonator. This model includes the number of photons in the cavity/resonator, the state of the atom/qubit, and the electric dipole interaction between the atom/qubit and cavity/resonator. As we saw in the last section, transmons are actually multi-level systems, but restricting ourselves to the ground $|0\rangle = |g\rangle$ and first excited $|1\rangle = |e\rangle$ states is possible because of the anharmonicity of the transmon. Therefore we can describe the transmon as a qubit described by the Pauli spin matrices
$$
\sigma^x = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \qquad
\sigma^y = \begin{pmatrix} 0 & -i \\ i & 0 \end{pmatrix} \qquad
\sigma^z = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \qquad
$$
that generate rotations the respective axes around the Bloch sphere. In that case, the simplest model to describe this interaction is the Jaynes-Cummings Hamiltonian in the rotating wave approximation,
$$
H_{\rm JC}^{\rm (RWA)}/\hbar = \omega_r a^\dagger a + \frac{1}{2} \omega_q \sigma_z + g(a^\dagger \sigma^- + a \sigma^+).
$$
where $\omega_r$ and $\omega_r$ are the frequencies of the resonator and "qubit", respectively, $a$ ($a^\dagger$) is the resonator photon annihilation (creation) operator, and $g$ is the electric dipole coupling (half the vacuum Rabi splitting). Note that we are now omitting the hats from the operators. Here, the first term corresponds to the number of photons in the resonator, the second term corresponds to the state of the qubit, and the third is the electric dipole interaction, where $\sigma^\pm = (1/2)(\sigma^x \mp i\sigma^y)$ is the qubit raising/lowering operator. (Note that the signs are inverted from those of *spin* raising/lowering operators, as discussed in the previous chapter).

This Hamiltonian can be solved exactly, and the solutions are hybrid qubit/resonator states where an excitation (either a photon in the resonator or excited state of the qubit) swaps between the two at a rate $g$ when they are on-resonance ($\omega_r = \omega_q$). For example, the $a^\dagger \sigma^-$ in the third term creates a photon in the resonator and lowers the qubit from $|1\rangle$ to $|0\rangle$, while the $a\sigma^+$ term destroys a photon in the resonators and excites the qubit from $|0\rangle$ to $|1\rangle$. While interesting, for our quantum computer we want to deal with qubits, and not these hybrid states. This means we want to move to a regimes where the resonator acts as a perturbation to the qubit (and vice-versa), so that their properties merely become "dressed" by the presence of the other. Using a type of perturbation theory, called the Schrieffer-Wolff (S-W) transformation, we can calculate the properties of the qubit and resonator in the regime we wish to operate. Here it should be noted that treating the transmon as a qubit is illustrative for pedagogical reasons, but the same techniques apply when you consider all the levels of the transmon. The higher levels of the transmon have profound effects and must be considered when designing and simulating them.

## 2. The Schrieffer-Wolff Transformation

    Schrödinger's Equation (Click here to expand)
Problems in quantum mechanics are often that of diagonalizing a Hamiltonian eigenvalue equation
$$
H\psi_m = E_m \psi_m \qquad {\rm for} \quad 1 \le m \le n
$$
where the $\psi_m$ are the eigenstates with eigenvalue $E_m$. This consists of finding a unitary matrix $U$, such that $H' = U H U^\dagger$ is diagonal. Then the eigenvalue equation
$$
\hat{H} \psi_m = E_m \psi_m \Longrightarrow U H U^\dagger U \psi_m = E_m U \psi_m \Longrightarrow H' \psi_m' = E_m \psi_m'
$$
where $\psi_m' = U\psi_m$ are the transformed eigenstates and
$$
H' = \begin{pmatrix}
E_1 & 0 & \cdots & 0 \\
0 & E_2 & \cdots & 0 \\
\vdots & \vdots & \ddots & 0 \\
0 & 0 & \cdots & E_n \end{pmatrix}
$$
is the diagonalized Hamiltonian.

With the S-W transformation, instead of diagonalizing the Hamiltonian, we seek to *block-diagonalize* it. Suppose we have a Hamiltonian that can be broken up into a diagonal part and perturbation
$$
H \quad = \quad \underbrace{\begin{pmatrix}
\Box &  &  &  &  &  &  \\
 & \Box &  &  &  &  &  \\
 &  & \Box &  &  &  &  \\
 &  &  & \Box &  &  &  \\
 &  &  &  & \Box &  &  \\
 &  &  &  &  & \Box &  \\
 &  &  &  &  &  & \Box \end{pmatrix}}_\text{diagonal} \quad + \quad
\underbrace{\begin{pmatrix}
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \end{pmatrix}}_\text{perturbation}
$$
and then write the perturbation as $H_1 + H_2$ so that $H = H_0 + H_1 + H_2$, with $H_0$ diagonal, $H_1$ block-diagonal, and $H_2$ non-block diagonal, and we have
$$
H \quad = \quad \underbrace{\begin{pmatrix}
\Box &  &  &  &  &  &  \\
 & \Box &  &  &  &  &  \\
 &  & \Box &  &  &  &  \\
 &  &  & \Box &  &  &  \\
 &  &  &  & \Box &  &  \\
 &  &  &  &  & \Box &  \\
 &  &  &  &  &  & \Box \end{pmatrix}}_\text{diagonal}
\quad + \quad
\underbrace{\begin{pmatrix}
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
 &  &  &  & \times & \times & \times \\
 &  &  &  & \times & \times & \times \\
 &  &  &  & \times & \times & \times \end{pmatrix}}_\text{block diagonal}
\quad + \quad
\underbrace{\begin{pmatrix}
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
\cdot & \cdot & \cdot & \cdot &  &  &  \\
\cdot & \cdot & \cdot & \cdot &  &  &  \\
\cdot & \cdot & \cdot & \cdot &  &  &  \end{pmatrix}}_\text{block off-diagonal}
$$

Block-diagonalizing $H$ consists of finding an operator $S$ such that

$$
H_{\rm eff} = e^{iS} H e^{-iS} = \sum_{m=0}^\infty \frac{1}{m!} [H, S]^{(m)} = \sum_{m=0}^\infty \lambda^m H^{(m)},
$$

where $H^{(m)}$ are successive approximations to $H$ (with $H^{(0)} = H_0$) and the generalized commutator is defined recursively as

$$
[H,S]^{(m)} = [[H,S]^{(m-1)},S] \qquad {\rm with} \qquad [H,S]^{(0)} = H.
$$

Here we treat $S$ as a Taylor series with

$$ S = \sum_{m=1}^\infty \lambda^m S^{(m)} $$

to keep track of the order $\lambda$. Then expanding the effective Hamiltonian as a perturbation of $H_1+H_2$ to second order in $\lambda$,

$$
H_{\rm eff} = H_0 + \lambda (H_1+H_2) + \left[H_0 + \lambda(H_1+H_2), \lambda S^{(1)}\right]
  + \frac{1}{2} \left[ \left[ H_0 + \lambda(H_1+H_2), \lambda S^{(1)}\right], \lambda S^{(1)}\right]
  + \left[H_0 + \lambda(H_1+H_2), \lambda^2 S^{(2)}\right] + \ldots \\
  \approx H_0 + \lambda \left( H_1 + H_2 + \left[H_0, S^{(1)}\right] \right)
  + \lambda^2 \left( \left[H_1+H_2, S^{(1)}\right] + \frac{1}{2} \left[ \left[H_0, S^{(1)},\right] S^{(1)}\right] + \left[H_0, S^{(2)}\right]\right)
$$

Now we know $S$ must be block off-diagonal and anti-hermitian to force the block off-diagonal elements of $H_{\rm eff}$ to vanish, we must have that

$$
H_{\rm eff}^{\rm off-diag} = \sum_{m=0}^\infty \frac{1}{(2m+1)!} [\underbrace{H_0 + H_1}_\text{block diag}, S]^{(2m+1)} + \sum_{m=0}^\infty \frac{1}{(2m)!} [\underbrace{H_2}_\text{block off-diag}, S]^{(2m)} \equiv 0,
$$

noting that all the terms in the first series are block off-diagonal and all of those in the second series are block diagonal. This is because the commutator of a block diagonal and block off-diagonal matrix is block off-diagonal and the commutator of two block off-diagonal matrices is block diagonal. Expanding this to the generalized commutator, we can see that $[H^0 + H^1, S]^{(n)}$ with odd $n$ must always be block off-diagonal as well as $[H^2, S]^{(n)}$ with even $n$. Now expanding the off-diagonal part of the Hamiltonian to second order yields

$$
H_{\rm eff}^{\rm off-diag} = \left[ H_0 + \lambda H_1, \lambda S^{(1)} \right]+\lambda H_2 + \left[H_0 + \lambda H_1, \lambda^2 S^{(2)}\right]
  + \frac{1}{3!} \left[ H_0+\lambda H_1, \lambda S^{(1)}\right]^{(3)} + \frac{1}{2!} \left[ \lambda H_2, \lambda S^{(1)}\right]^{(2)} \\
  = \lambda \left( \left[ H_0, S^{(1)} \right] + H_2 \right) + \lambda^2 \left( \left[H_1, S^{(1)} \right] + \left[H_0, S^{(2)}\right]\right) + \ldots.
$$
Since each order of $\lambda$ must be identically zero, the following equations determine $S^{(m)}$,
$$
[H_0, S^{(1)}] = -H_2 \qquad
[H_0, S^{(2)}] = -[H_1, S^{(1)}] \qquad
[H_0, S^{(3)}] = -[H_1, S^{(2)}] - \frac{1}{3} [[H_2, S^{(1)}], S^{(1)}],
$$
where our ansatz that satisfied these equations is guaranteed unique by Winkler's work. Then our effective Hamiltonian becomes

$$
H_{\rm eff} = H_0+H_1+[H_2,S^{(1)}] + \frac{1}{2} [[H_0, S^{(1)}], S^{(1)}] + \ldots = H_0+H_1+\frac{1}{2}[H_2,S^{(1)}] + \ldots
$$

where the effective Hamiltonian is calculated here to second order and we have taken $\lambda \to 1$.

## 3. Block-diagonalization of the Jaynes-Cummings Hamiltonian

Using the S-W transformation consists of two problems: 1) finding the correct ansatz, and 2) performing the calculations. In most examples, an ansatz of similar form (i.e. anti-hermitian) to the off-diagonal parts is made and confirmed *a postori*. Recently, the manuscript [A Systematic Method for Schrieffer-Wolff Transformation and Its Generalizations](http://www.arxiv.org/abs/2004.06534) has appeared on the arXiv attesting to systematically providing the ansatz and applying it to numerous systems (including the Jaynes-Cumming Hamiltonian below).

As such, the *generator* $\eta$ is calculated as $\eta = [H_0, H_2]$. In keeping the scalar coefficients of $\eta$ undetermined, then $S^{(1)}$ can be calculated as the specific $\eta$ that satisfies $[H_0, \eta]=H_2$. Note the hermiticity of $H_0$ and $H_2$ guarantee the anti-hermiticity of $\eta$ and thus $S^{(1)}$.

For ease of tedious calculations, we will use the Python package [`sympy`](http://www.sympy.org) for symbolic mathematics.

```python
# import SymPy and define symbols
import sympy as sp
sp.init_printing(use_unicode=True)
wr = sp.Symbol('\omega_r') # resonator frequency
wq = sp.Symbol('\omega_q') # qubit frequency
g = sp.Symbol('g', real=True) # vacuum Rabi coupling
Delta = sp.Symbol('Delta', real=True) # wr - wq; defined later
```

```python
# import operator relations and define them
from sympy.physics.quantum.boson import BosonOp
a = BosonOp('a') # resonator photon annihilation operator
from sympy.physics.quantum import pauli, Dagger, Commutator
from sympy.physics.quantum.operatorordering import normal_ordered_form

# Pauli matrices
sx = pauli.SigmaX()
sy = pauli.SigmaY()
sz = pauli.SigmaZ()

# qubit raising and lowering operators
splus = pauli.SigmaPlus()
sminus = pauli.SigmaMinus()
```

```python
# define J-C Hamiltonian in terms of diagonal and non-block diagonal terms
H0 = wr*Dagger(a)*a - (1/2)*wq*sz;
H2 = g*(Dagger(a)*sminus + a*splus);
HJC = H0 + H2; HJC # print
```

```python
# using the above method for finding the ansatz
eta = Commutator(H0, H2); eta
```

As a note about `sympy`, we will need to used the methods `doit()`, `expand`, `normal_ordered_form`, and `qsimplify_pauli` to proceed with actually taking the commutator, expanding it into terms, normal ordering the bosonic modes (creation before annihilation), and simplify the Pauli algebra. Trying this with $\eta$ yields

```python
pauli.qsimplify_pauli(normal_ordered_form(eta.doit().expand()))
```

Now take $A$ and $B$ as the coefficients of $a^\dagger \sigma_-$ and $a\sigma_+$, respectively. Then the commutator

```python
A = sp.Symbol('A')
B = sp.Symbol('B')
eta = A * Dagger(a) * sminus - B * a * splus;
pauli.qsimplify_pauli(normal_ordered_form(Commutator(H0, eta).doit().expand()))
```

This expression should be equal to $H_2$

```python
H2
```

which implies $A = B = g/\Delta$ where $\Delta = \omega_r - \omega_q$ is the frequency detuning between the resonator and qubit. Therefore our $S^{(1)}$ is determined to be

```python
S1 = eta.subs(A, g/Delta)
S1 = S1.subs(B, g/Delta); S1.factor()
```

Then we can calculate the effective second order correction to $H_0$

```python
Heff = H0 + 0.5*pauli.qsimplify_pauli(normal_ordered_form(Commutator(H2, S1).doit().expand())).simplify(); Heff
```

This is typically written as
$$
H_{\rm eff} = \left(\omega_r + \frac{g^2}{\Delta}\sigma_z\right)a^\dagger a  - \frac{1}{2}\left(\omega_q -\frac{g^2}{\Delta}\right) \sigma_z
$$
which shows a state-dependent shift by $\chi \equiv g^2/\Delta$ of the resonator frequency called the *ac Stark shift* and a shift in qubit frequency due to quantum vacuum fluctuations called the *Lamb shift*.

## 4.  Full Transmon

Because we are using *transmons* instead of *qubits*, we need to be careful to take the higher-order energy terms into effect when designing and simulating devices. The full transmon Hamiltonian coupled to the readout resonators is

$$
H^{\rm tr} = \omega_r a^\dagger a + \sum_j \omega_j |j\rangle\langle j| + g\left(a^\dagger c + ac^\dagger \right),
$$

where now $c = \sum_j \sqrt{j+1}|j\rangle\langle j+1|$ is the transmon lowering operator. Similarly, taking the weakly interacting subsets $A$ as the even-numbered transmon modes and $B$ as the odd-numbered transmon modes. Using the ansatz

$$
S^{(1)} = \sum_j \alpha_j a^\dagger \sqrt{j+1}|j\rangle\langle j+1| - \alpha_j^* a \sqrt{j+1}|j+1\rangle\langle j|,
$$

one may proceed along a messier version of the Jaynes-Cummings Hamiltonian. With some effort one can show the second order effective Hamiltonian is

$$
H^{\rm tr}_{\rm eff} = \left( \omega_r + \sum_j \frac{g^2(\omega_r-\omega+\delta)}{(\omega_r-\omega-\delta j)(\omega_r - \omega - \delta(j-1))} |j\rangle\langle j| \right) a^\dagger a + \sum_j
\left[
j\omega + \frac{\delta}{2} (j-1)j + \frac{jg^2}{\omega-\omega_r+(j-1)\delta} \right]|j\rangle\langle j|.
$$

## 5. Qubit Drive with cQED

Following that of [Blais *et al* (2004)](https://arxiv.org/abs/cond-mat/0402216), we model the drive Hamiltonian as
$$
H^d(t) = \xi(t)\left( a^\dagger e^{-i\omega_d t} + ae^{i\omega_d t}\right).
$$
Following the treatment in the [Ph.D. dissertation of Lev Bishop](https://arxiv.org/abs/1007.3520), the drive acts on the qubit via the Glauber operator
$$
D(\alpha) = e^{\alpha(t) a^\dagger - \alpha^*(t) a}.
$$
Moving to the Jaynes-Cumming Hamiltonian rotating at the drive frequency,
$$
H = \Delta_r a^\dagger a - \frac{1}{2} \Delta_q \sigma^z + g(a^\dagger \sigma^- + a\sigma^+) + \xi(t)(a^\dagger + a)
$$
with $\Delta_r = \omega_r - \omega_d$ and $\Delta_q = \omega_q - \omega_d$. Applying Hadamard's Lemma to nested commutators,
$$
e^{A}BA^{-A} = B + [A,B] + \frac{1}{2!} [A,[A,B]] + \frac{1}{3!}[A,[A,[A,B]]] + \ldots
$$
we see that
$$
D^\dagger a^{(\dagger)} D = \exp\{-\alpha(t) a^\dagger + \alpha^*(t) a\} a^{(\dagger)}\exp\{\alpha(t) a^\dagger - \alpha^*(t) a\}
= a^{(\dagger)} + \left[-\alpha(t) a^\dagger + \alpha^*(t) a, a^{(\dagger)}\right] + \frac{1}{2!}\left[-\alpha(t) a^\dagger + \alpha^*(t) a, \left[-\alpha(t) a^\dagger + \alpha^*(t) a, a^{(\dagger)}\right]\right] + \ldots
 = a^{(\dagger)} + \alpha^{(*)}
$$
and

$$
D^\dagger a^\dagger a D = a^\dagger a + \left[-\alpha(t) a^\dagger + \alpha^*(t) a, a^\dagger a\right] + \frac{1}{2!}\left[-\alpha(t) a^\dagger + \alpha^*(t) a, \left[-\alpha(t) a^\dagger + \alpha^*(t) a, a^\dagger a\right]\right] + \ldots
= a^\dagger a + \alpha(t)a^\dagger + \alpha^*(t)a + |\alpha(t)|^2
$$

So that we can transform the Hamiltonian

$$
\tilde{H} = D^\dagger H D - iD^\dagger \dot{D} = \Delta_r\left(a^\dagger a + \alpha(t)a^\dagger + \alpha^*(t)a + |\alpha(t)|^2\right) - \frac{1}{2} \Delta_q \sigma^z \\ + g\left((a^\dagger + \alpha^*(t))\sigma^- + (a+\alpha(t))\sigma^+\right) + \xi(t)\left(a^\dagger + \alpha^*(t) + a + \alpha(t) \right) - i\left(\dot{\alpha}(t) a^\dagger - \dot{\alpha}^*(t) a\right) \\
= \Delta_r a^\dagger a - \frac{1}{2}\Delta_q \sigma^z + g\left((a^\dagger + \alpha^*(t))\sigma^- + (a+\alpha(t))\sigma^+\right) \\
+\xi(t)\left(a^\dagger  + a \right) + \Delta_r\left(\alpha(t)a^\dagger + \alpha^*(t)a\right)- i\left(\dot{\alpha}(t) a^\dagger - \dot{\alpha}^*(t) a\right)
$$

where the non-operator terms have been dropped. The last line can be set to zero if we choose

$$
-i\dot{\alpha}(t) + \Delta_r \alpha(t) + \xi(t) = 0,
$$

and finally introducing the Rabi frequency $\Omega(t) = 2g\alpha(t)$, we arrive at

$$
\tilde{H} = \Delta_r a^\dagger a - \frac{1}{2}\Delta_q \sigma^z + g\left(a^\dagger\sigma^- + a\sigma^+\right)
+\frac{1}{2} \left( \Omega^*(t)\sigma^- + \Omega(t) \sigma^+\right).
$$

Since the drive part of the Hamiltonian is block off-diagonal, we can perform a Schrieffer-Wolff transformation on it (for a real drive $\Omega^*(t) = \Omega(t)$) and add it to the effective Hamiltonian,

$$
[\tilde{H}^d, S^{(1)}] = -\frac{\Omega(t)}{2} \left[ (\sigma^- + \sigma^+),\frac{g}{\Delta}\left( a^\dagger \sigma^- - a\sigma^+\right)\right] = \frac{g\Omega(t)}{2\Delta}(a + a^\dagger)\sigma^z
$$

so the effective Hamiltonian becomes

$$
\tilde{H}_{\rm eff} = \left( \Delta_r + \frac{g^2}{\Delta}\sigma^z\right) a^\dagger a - \frac{1}{2}\left(\Delta_q - \frac{g^2}{\Delta}\right) \sigma^z + \frac{\Omega(t)}{2}\sigma^x
+ \frac{g\Omega(t)}{4\Delta}(a + a^\dagger)\sigma^z.
$$

Note here that to eliminate the $z$ rotations, one should drive at the Lamb-shifted qubit frequency. The additional $\sigma^z$ term is small because $\Delta \gg g$ in the dispersive regime.

## 6. The Cross Resonance Entangling Gate

Driving qubit one at the frequency of qubit two can be written as

$$
H^d(t) = \frac{\Omega(t)}{2} \left( \sigma_1^+ e^{-i\tilde{\omega}_2 t} + \sigma_1^- e^{i\tilde{\omega}_2 t}\right).
$$

Now, we need to apply Schrieffer-Wolff to the drive term to get the effective Hamiltonian, and then do the RWA at frequency $\tilde{\omega}_2$.

$$
[\tilde{H}^d, S^{(1)}] = -\frac{J\Omega(t)}{2\Delta_{12}} \left[ \sigma_1^+ e^{-i\tilde{\omega}_2 t} + \sigma_1^- e^{i\tilde{\omega}_2 t}, \sigma_1^+ \sigma_2^- - \sigma_2^+ \sigma_1^-\right]
=-\frac{J\Omega(t)}{2\Delta_{12}} \left(\sigma_1^z \sigma_2^+ e^{-i\tilde{\omega}_2 t}
+\sigma_1^z \sigma_2^- e^{i\tilde{\omega}_2 t} \right)
$$

Transforming back the rotating frame at $\omega_2$, we get the effective qubit cross resonance Hamiltonian

$$
\tilde{H}_{\rm eff}^{\rm CR} = - \frac{\tilde{\omega}_1-\tilde{\omega}_2}{2}\sigma_1^z
+ \frac{\Omega(t)}{2} \left(\sigma_2^x - \frac{J}{2\Delta_{12}} \sigma_1^z \sigma_2^x \right).
$$

The first two terms involve the $ZI$ interaction due to a Stark shift on qubit 1 and an unconditional $IX$ rotation on qubit 2, but the final term represents the $ZX$-interaction that produces entanglement. By putting qubit 1 into an equal superposition of $|0\rangle$ and $|1\rangle$ and applying the cross resonance gate for a duration corresponding to a $\pi/2$ rotation around the $x$-axis, a maximally entangled state is produced. Using Qiskit to characterize the two-qubit cross resonance Hamiltonian for transmons can be done with [this tutorial](https://github.com/Qiskit/qiskit-tutorials/blob/9405254b38312771f8d5c2dd6f451cec35307995/tutorials/noise/1_hamiltonian_and_gate_characterization.ipynb). Further reading on the cross resonance gate is found [here](https://arxiv.org/abs/1106.0553) and [here](https://arxiv.org/abs/1603.04821).

```python

```


#  L'Évolution Différentielle

**TODO**
* Livre metaheuristiques p.165
* [Scipy's implementation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.differential_evolution.html#scipy.optimize.differential_evolution)
* https://en.wikipedia.org/wiki/Differential_evolution
* http://www.scholarpedia.org/article/Metaheuristic_Optimization#Differential_Evolution
* [Practical advises](http://www1.icsi.berkeley.edu/~storn/code.html#prac)

L'Évolution Différentielle (*Differential Evolution* ou DE en anglais) est une méthode d'optimisation pour des problèmes continus développée par R. Storn et K. Price en 1995 [Storn95] [Storn97].

* **[Storn95]**: Storn, R., & Price, K. (1995). *Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces*. Technical Report TR95-012, International Computer Science Institute, Berkeley, California. [PDF document](ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-012.pdf).
* **[Storn97]**: Storn, R., & Price, K. (1997). *Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces*. Journal of global optimization, 11(4), 341-359.

## L'algorithme

### Notation simplifiée

À chaque génération

$\quad$ Pour tous les individus $i$ de la population

$\quad\quad$ 1. **MUTATION**:

$\quad\quad\quad$ 1.1 On choisi 3 individus différents au hasard $\boldsymbol{x}_{r1}$, $\boldsymbol{x}_{r2}$ et $\boldsymbol{x}_{r3}$

$\quad\quad\quad\quad$ $\boldsymbol{x}_{r2}$ et $\boldsymbol{x}_{r3}$ vont permettre de déterminer automatiquement la direction et l'amplitude le la mutation

$\quad\quad\quad\quad$ (i.e. la direction de recherche et son amplitude)

$\quad\quad\quad$ 1.2 On calcule un mutant $\boldsymbol{v}_{i} \leftarrow \boldsymbol{x}_{r1} + F . (\boldsymbol{x}_{r2} - \boldsymbol{x}_{r3})$

$\quad\quad$ 2. **CROISEMENT**:

$\quad\quad\quad$ On crée un individu "test" $\boldsymbol{u}_{i}$ dont les composantes sont aléatoirement prises soit dans $\boldsymbol{x}_i$ soit dans $\boldsymbol{v}_i$

$\quad\quad$ 3. **SELECTION**:

$\quad\quad\quad$ Si cet individu test $\boldsymbol{u}_{i}$ est meilleur que $\boldsymbol{x}_{i}$, il le remplace à la génération suivante.

### Notation plus formelle

Ci-dessous une écriture plus formelle de l'algorithme.

**Notations**:
* $D$ : the dimension of input vectors, $D \in \mathbb{N}^*_+$.
* $t$ : the current iteration index (or generation index), $t \in \mathbb{N}^*_+$.
* $T$ : the total number of iteration (or generation), $T \in \mathbb{N}^*_+$.
* $N$ : the size of the population, $N > 3$.
* $F$ : the *differential weight*. In principle, $F \in [0,2]$, but in practice, a scheme with $F \in [0,1]$ is more efficient and stable ([scholarpedia](http://www.scholarpedia.org/article/Metaheuristic_Optimization#Differential_Evolution)).
* $Cr$ : the crossover probability parameter, $Cr \in [0,1]$.

---

**Algorithm's parameters:** $N$, $Cr$ and $F$

---

**Input:**
* An initial population $(\boldsymbol{x}_{1,1}, \boldsymbol{x}_{2,1}, \dots, \boldsymbol{x}_{N,1})$ with $\boldsymbol{x}_{0,i} \in \mathbb{R}^D$

---

**pour chaque** generation $t = 1, \dots, T$

$\quad$ **pour chaque** individu $i = 1, \dots, N$

$\quad\quad$ **1. MUTATION:**

$\quad\quad\quad$ Choisi aléatoirement 3 individus différents $\boldsymbol{x}_{r1,t}$, $\boldsymbol{x}_{r2,t}$ et $\boldsymbol{x}_{r3,t}$ dans la population

$\quad\quad\quad$ Construit un individu mutant $\boldsymbol{v}_{i,t} \leftarrow \boldsymbol{x}_{r1,t} + F (\boldsymbol{x}_{r2,t} - \boldsymbol{x}_{r3,t})$

$\quad\quad\quad$ où $F$ est un "facteur d'amplification" de la mutation.

$\quad\quad\quad$ L'algorithme extrait des informations de direction et de distance à produire pour sa composante aléatoire à partir des deux individus $\boldsymbol{x}_{r2,t}$ et $\boldsymbol{x}_{r3,t}$.

$\quad\quad$ **2. CROISEMENT:** l'individu $\boldsymbol{x}_{i,t}$ est "mélangé" avec le mutant $\boldsymbol{v}_{i,t}$ pour former un individu test $\boldsymbol{u}_{i,t}$

$\quad\quad\quad$ Choisi aléatoirement une dimension $d_{r,i}$.

$\quad\quad\quad$ La $d^{ieme}$ composante $v_{d,i,t}$ de $\boldsymbol{v}_{i,t}$ sera toujours transmise à $\boldsymbol{u}_{i,t}$ afin de s'assurer qu'il obtienne au moins un élément de $\boldsymbol{v}_{i,t}$.

$\quad\quad\quad$ **pour chaque** dimension $d = 1, \dots, D$

$$
u_{d,i,t} \leftarrow
\left\{
\begin{align}
v_{d,i,t} \quad & \text{ si } \mathcal{B}(Cr) = 1 \text{ ou si } d = d_{r,i}\\
x_{d,i,t} \quad & \text{ sinon.}
\end{align}
\right.
$$

$\quad\quad\quad\quad$  où $\mathcal{B}(Cr)$ est une variable aléatoire suivant une loi de Bernoulli de paramètre $Cr$

$\quad\quad\quad\quad$ et ou $u_{d,i,t}$, $v_{d,i,t}$ et $x_{d,i,t}$ sont respectivement les $d^{ieme}$ composantes de $\boldsymbol{u}_{i,t}$, $\boldsymbol{v}_{i,t}$ et $\boldsymbol{x}_{i,t}$

$\quad\quad$ **3. SELECTION:**

$\quad\quad\quad$ **si** $f(\boldsymbol{u}_{i,t}) < f(\boldsymbol{x}_{i,t})$

$\quad\quad\quad\quad \boldsymbol{x}_{i,t+1} \leftarrow \boldsymbol{u}_{i,t}$

$\quad\quad\quad$ **sinon**

$\quad\quad\quad\quad \boldsymbol{x}_{i,t+1} \leftarrow \boldsymbol{x}_{i,t}$

$\quad$ **fin pour**

**fin pour**

```python
# import python packages here...
```


# MBQC 模型下求解多项式组合优化问题

<em> Copyright (c) 2021 Institute for Quantum Computing, Baidu Inc. All Rights Reserved. </em>

在[基于测量的量子近似优化算法](QAOA_CN.ipynb)中，我们介绍了多项式无约束布尔优化问题（polynomial unconstrained boolean optimization problem, PUBO），并提出了**基于测量的量子近似优化算法（MB-QAOA）** 对该类问题的求解方式，感兴趣的读者可以参阅前面的内容。这里我们给出两个示例作为 MB-QAOA 的实战演示：一个具体的 PUBO 问题和一个最大割问题。

## 示例：PUBO 问题

我们首先简单回顾一下什么是 PUBO 问题。给定一个变量为 $x = \{x_1,\cdots,x_n\}$ 的 $n$ 元多项式，

$$
C(x) = \sum_{\lambda \in \Lambda } \alpha_{\lambda} \prod_{j \in \lambda} x_j,\tag{1}
$$

其中每个变量 $x_i \in \{0,1\}$，$\prod_{j \in \lambda} x_j$ 为一个单项式，$\lambda \subseteq [n]:= \{1, 2, ..., n\}$ 为一个指标集，$\Lambda$ 为指标集的集合，$\alpha_\lambda$ 为每个单项式对应的实系数。在 PUBO 问题中，$C(x)$ 称为目标多项式。PUBO 问题就是寻找一组最优解 $x^* = \{x_1^*, x_2^*, ..., x_n^*\} $ 使得目标多项式的取值最大，即

$$
x^* =\underset{x}{\text{argmax}} \ C(x).\tag{2}
$$

代码实现上，我们规定输入多项式的标准形式为一个列表，列表中的第一个元素为多项式的最大变量数，列表的第二个元素为一个字典，字典的键为每个单项式涉及的变量（常数则为 ‘cons’），变量之间用逗号隔开，值为该单项式的系数。比如，我们希望输入一个三元多项式 $x_1 + x_2 - x_3 + x_1 x_2 - x_1 x_2 x_3 + 0.5$，应输入如下代码：

```python
# 变量数
var_num = 3
# 以字典格式存储多项式的各个单项式
poly_dict = {'x_1': 1, 'x_2': 1, 'x_3': -1, 'x_1,x_2': 1, 'x_1,x_2,x_3': -1, 'cons':0.5}
# 构造符合约定的多项式
polynomial = [var_num, poly_dict]
```

**注意：** 由于我们考虑的变量为布尔值，所以每个单项式中变量指数最大为 1，即输入的多项式字典的键中，每个变量最多出现一次。例如：{'x_1,x_1,x_2': 1} 为不符合规范的输入。我们同样约定自变量的下角标从 "1" 开始，与数学习惯相一致。并且为了统一输入规范，自变量需要连续编号，不能出现输入多项式为 $x_1x_2 + x_6$ 的情况，应该改为 $x_1x_2 + x_3$，否则我们会以报错的形式提醒输入不符合规范。

代码实现上，我们在 `pubo` 中定义了 `is_poly_valid` 函数，用于检查输入多项式的合法性。如果合法，将会打印 "The polynomial is valid." 字样，否则会报错。

```python
from paddle_quantum.mbqc.QAOA.pubo import is_poly_valid
```
我们同样定义了一个函数 `random_poly` 根据变量个数随机生成一个布尔型多项式。
```python
from paddle_quantum.mbqc.QAOA.pubo import random_poly
```
**注意：** 随机生成的多项式并不一定是合法的！因此我们需要在计算之前检查多项式的合法性。

### 求解 PUBO 问题的代码实现

```python
# 引入记时模块
from time import perf_counter
# 引入符号计算模块
from sympy import symbols
#引入 paddle 相关模块
from paddle import seed, optimizer
# 引入 pubo 模块
from paddle_quantum.mbqc.QAOA.pubo import dict_to_symbol, is_poly_valid, brute_force_search
# 引入 qaoa 模块
from paddle_quantum.mbqc.QAOA.qaoa import MBQC_QAOA_Net, get_solution_string
```

利用 MB-QAOA 算法，我们可以定义 PUBO 主函数 ``mbqc_pubo``，输入目标函数和 QAOA 算法深度，最终返回最优解对应的比特串。在主函数中，**最核心的是 MBQC_QAOA_Net 类**，其集成了 MB-QAOA 以及优化网络，感兴趣的读者可以返回到 [基于测量的量子近似优化算法](QAOA_CN.ipynb) 中查看，这里我们直接调用。

```python
# 定义 MBQC 模型求解 PUBO 问题的主函数
def mbqc_pubo(OBJ_POLY, DEPTH, SEED, LR, ITR, EPOCH, SHOTS=1024):

    # 将字典类型的目标多项式转化为 sympy 能处理的符号多项式
    obj_poly = dict_to_symbol(OBJ_POLY)
    var_num, poly_symbol = obj_poly

    # 打印当前 QAOA 算法电路深度
    print("QAOA 算法深度为：", DEPTH)

    # 开始计时器
    start_time = perf_counter()

    # 初始化一个 MB-QAOA 训练网络
    seed(SEED)
    mbqc_net = MBQC_QAOA_Net(DEPTH)

    # 选择 Adams 优化器
    opt = optimizer.Adam(learning_rate=LR, parameters=mbqc_net.parameters())

    # 开始训练
    for epoch in range(EPOCH):
        # 每次迭代后更新训练参数
        for itr in range(1, ITR + 1):
            # 用网络进行训练并返回损失函数值和当前输出量子态
            loss, vector_out = mbqc_net(poly=obj_poly)
            # 根据损失函数值优化参数
            loss.backward()
            opt.minimize(loss)
            opt.clear_grad()
            if itr % 10 == 0:
                print("iter:", itr, "  loss_MBQC:", "%.4f" % loss.numpy())

    # 停止计时器并打印训练用时
    end_time = perf_counter()
    print("MBQC 运行时间为： ", end_time - start_time)

    # 打印优化后的训练参数 gamma, beta
    print("最优参数 gamma 为： ", mbqc_net.gamma.numpy())
    print("最优参数 beta 为： ", mbqc_net.beta.numpy())

    # 解码原问题的答案
    solution_str = get_solution_string(vector_out, SHOTS)

    # 将最优解带回目标函数，求出对应的最优值
    relation = {symbols('x_' + str(j + 1)): int(solution_str[j]) for j in range(var_num)}
    value = poly_symbol.evalf(subs=relation)

    # 返回最优解和最优值
    opt = [solution_str, value]
    return opt
```

为了检验训练结果的正确性，我们在 `pubo` 中定义了 `brute_force_search` 函数，用遍历的方法在解空间里暴力搜索 PUBO 问题的解，方便与 MBQC 模型训练得到的解进行对比。

```python
from paddle_quantum.mbqc.QAOA.pubo import brute_force_search
```

### Main 函数

主函数定义后，我们就可以输入参数运行啦！

```python
# 定义主函数
def main():

    # 以上面的例子 x_1 + x_2 - x_3 + x_1*x_2 -x_1*x_2*x_3 + 0.5 为例，求解 PUBO 问题
    var_num = 3
    poly_dict = {'x_1': 1, 'x_2': 1, 'x_3': -1, 'x_1,x_2': 1, 'x_1,x_2,x_3': -1, 'cons':0.5}
    polynomial = [var_num, poly_dict]

    # 打印输入的多项式
    print("输入的多项式为：", polynomial)

    # 我们也可以随机生成一个目标多项式
    # polynomial = random_poly(var_num)

    # 检查输入多项式的合法性
    is_poly_valid(polynomial)

    # 进行训练并返回最优结果
    mbqc_result = mbqc_pubo(
        OBJ_POLY=polynomial,  # 目标多项式函数
        DEPTH=6,  # QAOA 算法电路深度
        SEED=1024,  # 随机种子
        LR=0.1,  # 学习率
        ITR=120,  # 训练迭代次数
        EPOCH=1  # 迭代周期
    )

    # 打印 MBQC 模型训练出来的最优结果
    print("MBQC 模型得到的最优解为：", mbqc_result[0])
    print("MBQC 模型得到的最优值为：", mbqc_result[1])

    # 通过暴力搜索计算 PUBO 问题的最优结果并打印
    brute_result = brute_force_search(polynomial)
    print("暴力搜索得到的最优解为：", brute_result[0])
    print("暴力搜索得到的最优值为：", brute_result[1])

    # 将两种结果进行比较
    print("MBQC 模型和暴力搜索的差值为：", mbqc_result[1] - brute_result[1])

if __name__ == '__main__':
    main()
```

## 示例：最大割问题

### 图与割

最大割问题（MaxCut Problem）是图论中常见的一个组合优化问题，在统计物理学和电路设计中都有重要应用。

在图论中，一个图由 $G = (V, E)$ 表示，其中 $V$ 为图的点集合，$E$ 为边集合。例如一个正方形就可以由 $G = (V,E)$ 其中 $V = [1,2,3,4]$， $E = [(1,2),(2,3),(3,4),(1,4)]$ 来表示。

代码上，我们可以用 `maxcut` 中的 `plot_graph` 画出给定的图。

```python
from paddle_quantum.mbqc.QAOA.maxcut import plot_graph
V = [1,2,3,4]
E = [(1,2),(2,3),(3,4),(1,4)]
G = [V, E]
plot_graph(G,"A square")
```

一个图的割是指将该图的顶点集 $V$ 分割成两个互不相交的子集合 $S_0$ 和 $S_1$ 的一种划分。如果图中边的两个顶点被划分到不同集合中，那么记录得一分，总得分则定义为这个割的大小。最大割问题就是要找到一个割使得该割的大小最大。

比如对应如上所述正方形图 $G$， 其中一个最大割就是将点 $1$ 和 $3$ 分为同一个集合 $S_0$ 中，点 $2$ 和 $4$ 分到另一个集合 $S_1$ 中。

### 转化为 PUBO 问题

最大割问题其实可以转化为一个 PUBO 问题。假设待分割的图 $G = (V, E)$ 有 $n=|V|$ 个顶点和 $m =|E|$ 条边，那么我们可以将最大割问题等价为拥有 $n$ 个自变量的组合优化问题。每个变量 $x_v$ 对应图 $G$ 中的一个顶点 $v \in V$，其取值 $x_v \in \{0,1\}$，分别对应于顶点属于集合 $S_0$ 或 $S_1$，因此 $x = \{x_1,\cdots,x_n\}$ 的每一种取值方式都对应一个割。由于最大割问题中有效计数的边为那些顶点属于不同集合的边，即顶点 $u$ 和 $v$ 对应的变量取值不同 $x_u \neq x_v$ 才会被记一分。所以对于给定的割 $x$，其对应的割的大小为

$$
C(x) = \sum_{(u,v) \in E} (x_u \oplus x_v),\tag{3}
$$

其中 $\oplus$ 为异或运算。那么最大割问题等价于求解 $ \underset{x}{\max} \ C(x)$。进一步，上述函数 $C(x)$ 可以写成多项式形式

$$
C(x) = \sum_{(u, v) \in E} (x_u + x_v - 2 x_u x_v),\tag{4}
$$

从而最大割问题等价于一个 $n$ 元 $2$ 次多项式的 PUBO 问题。我们希望找到最优解 $x^{*}$ 使得目标多项式最大，

$$
x^* = \underset{x}{\text{argmax}} \ \left( \sum_{(u, v) \in E} (x_u + x_v - 2 x_u x_v) \right).\tag{5}
$$

在 `maxcut` 当中，我们定义了 `graph_to_poly` 函数，可以输入待分割的图，返回等价的 PUBO 问题的目标多项式。

```python
# 引入 maxcut 模块
from paddle_quantum.mbqc.QAOA.maxcut import graph_to_poly

# 输入点和边的信息
V = [1,2,3,4]
E = [(1,2),(2,3),(3,4),(1,4)]
# 构造待分割的图
G = [V, E]

# 将图转化为对应 PUBO 问题的目标多项式
poly = graph_to_poly(G)
print("与图等价的目标多项式为：\n", poly)
```

### 求解最大割问题的代码实现

得到等价的目标多项式之后，我们便可以仿照 PUBO 的流程，把最大割问题作为 PUBO 问题来求解。具体代码如下：

```python
# 引入符号运算模块
from sympy import symbols
# 引入 paddle 模块
from paddle import seed, optimizer
# 引入 qaoa 模块
from paddle_quantum.mbqc.QAOA.qaoa import MBQC_QAOA_Net, get_solution_string
# 引入 maxcut 模块
from paddle_quantum.mbqc.QAOA. maxcut import plot_graph, graph_to_poly, plot_solution
```

利用 MB-QAOA 算法，我们可以定义 MaxCut 主函数，将图输入到 MB-QAOA 算法中，最终返回求得的解并画出割的方式。

```python
# 定义 MaxCut 主函数
def mbqc_maxcut(GRAPH, DEPTH, SEED, LR, ITR, EPOCH, SHOTS=1024):

    # 打印待分割的图
    plot_graph(graph=GRAPH, title="Graph to be cut")

    # 找到图对应的目标多项式
    polynomial = graph_to_poly(GRAPH)
    print("与图等价的目标多项式为：", polynomial[1])

    # 开始算法的记时
    start_time = perf_counter()

    # 实例化一个 MB-QAOA 训练网络
    seed(SEED)
    mbqc_net = MBQC_QAOA_Net(DEPTH)
    # 选择 Adams 优化器
    opt = optimizer.Adam(learning_rate=LR, parameters=mbqc_net.parameters())

    # 开始训练
    for epoch in range(EPOCH):
        # 每次迭代后更新训练参数
        for itr in range(1, ITR + 1):
            # 训练参数并返回损失函数值和当前输出量子态
            loss, state_out = mbqc_net(poly=polynomial)
            # 根据损失函数值优化训练参数
            loss.backward()
            opt.minimize(loss)
            opt.clear_grad()
            if itr % 10 == 0:
                print("iter:", itr, "  loss_MBQC:", "%.4f" % loss.numpy())

    # 停止算法的记时并打印用时
    end_time = perf_counter()
    print("MBQC 运行时间为：", end_time - start_time)

    # 打印训练得到的最优参数
    print("最优参数 gamma 为： ", mbqc_net.gamma.numpy())
    print("最优参数 beta 为：", mbqc_net.beta.numpy())

    # 从量子态中解码问题的答案
    mbqc_solution = get_solution_string(state_out, SHOTS)
    # 绘制割的方式
    plot_solution(GRAPH, mbqc_solution)

    # 将最优解带入目标函数，求得对应最优值，并返回
    var_num, poly_symbol = polynomial
    relation = {symbols('x_' + str(j + 1)): int(mbqc_solution[j]) for j in range(var_num)}

    mbqc_value = int(poly_symbol.evalf(subs=relation))
    mbqc_opt = [mbqc_solution, mbqc_value]

    # 返回最优解和对应的最优值
    return mbqc_opt
```

### Main 函数

主函数定义后，我们就可以输入参数运行啦！

```python
def main():
    # 以正方形的 MaxCut 问题为例，输入节点和边信息
    V = [1, 2, 3, 4]
    E = [(1, 2), (2, 3), (3, 4), (4, 1)]
    G = [V, E]

    # 进行训练并返回最优结果
    mbqc_result = mbqc_maxcut(
        GRAPH=G,  #  待分割的图
        DEPTH=6,  # QAOA 算法电路深度
        SEED=1024,  # 随机种子
        LR=0.1,  # 学习率
        ITR=120,  # 训练迭代次数
        EPOCH=1,  # 迭代周期
        SHOTS=1024  # 解码答案时的采样数
    )

    # 打印训练的最优结果
    print("最优解为：", mbqc_result[0])
    print("最优值为： ", mbqc_result[1])

if __name__ == '__main__':
    main()
```

至此，我们完成了本教程的两个完整示例。MB-QAOA 算法实现预示着 MBQC 模型在量子机器学习领域强大的潜力。很显然，MBQC 模型所能够处理的算法不仅有 QAOA，我们非常期待 MBQC 这种非同寻常的计算方式能在某些特殊情境下展现出无与伦比的优势！

---

## 参考文献

[1] Farhi, Edward, et al. "A quantum approximate optimization algorithm." [arXiv preprint arXiv:1411.4028 (2014).](https://arxiv.org/abs/1411.4028)


# Introduction

In this tutorial, we will use fast Fourier transform to develop a version of the semantic model that predicts reaction time distributions. As will be illustrated below, the model predicts multimodal distributions when a variable number of memory retrievals can lead to a response through category chaining. The structure of the present model is identical to the Markov process version presented earlier. Some details of the model will be presented below. Please review the first version of the [model](../Semantic/Semantic_Model.ipynb) if you need to refamiliarize yourself with the details.

## Task

In the task, participants must verify whether one category is a member of another category by responding "yes" or "no". For example, a person might recieve a question such as "Is a bird a canary?" and must respond "yes" or "no" with the appropriate keys.

# Semantic Model

According to the model, category relationships are represented as a semantic network in which nodes correspond to concepts and directed edges correspond to membership. The model can answer questions about category membership directly through a process called direct verification, or indirectly through a process called category chaining. For example, the model can answer the question "Is a bird an animal?" through direct verification by traversing from the bird node to the animal node using a single memory retrieval. By contrast, the question "is a canary an animal?" is answered indirectly through category chaining, which involves traversing from the canary node to the bird node and then from the bird node to the animal node using successive memory retrievals.

## Declarative Memory

Declarative memory $M$ consists of 20 chunks with the slots, (i.e., domain), $Q = \{\textrm{object, attribute, value}\}$. The object slot refers to the entity being evaluated (e.g. canary); the attribute slot refers to the particular property on which the object is being evaluated (e.g. category, flies); and the value slot refers to the particular value of the attribute (e.g. bird, true). For example, chunk 20 is $c_{20}(\rm{object}) = \textrm{bird}$, $c_{20}(\textrm{attribute}) =\textrm{category}$, $c_{20}(\textrm{value}) = \textrm{animal}$, which indicates that the object bird belongs to the category animal.

### Activation

Memory activation for chunk $m$ is defined as

\begin{equation}\label{eq:semantic_activation}
a_m = \textrm{blc} + \rho_m + \epsilon_m
\end{equation}

where $\textrm{blc}$ is the base-level constant and $\rho_m$ is partial matching. Note that $E[a_m] = \mu_m$ The base-level constant simply scales memory activation up or down. Partial matching allows chunks to be retrieved as a function of dissimilarity to the retrieval request values, $r$. For simplicity, partial matching is defined as a weighted count of mismatching slot-values:

\begin{equation}\label{eq:penalty_function}
\rho_m = -\delta  \sum_{q \in Q_r} I^c\left(r(q),  c_{m}(q)\right).
\end{equation}

where $Q_r = \{\textrm{object},\textrm{attribute}\}$ is the set of slots in the retrieval request, the mismatch penalty parameter $\delta$ controls the steepness of the dissimilarity gradient, and $I$ is an indicator function:

$$ I^c(x,y) =
  \begin{cases}
    1      & x \neq y\\
    0  & x = y
  \end{cases}
$$

Thus, chunks that are more similar to the retrieval request are more likely to be retrieved.

## Perceptual-Motor Time

In the semantic model, a series of production rules are used to encode the words (e.g. canary, bird), initialize memory retrievals, and execute a response. As with previous models, a normal approximation will be used to represent the sum of these independent and serial processes. The number of perceptual motor processes depends on the number of category chains required to give a response. Thus, the the general expression of $\mu$ and $\sigma$ for perceptual-motor processing time is defined as:
\begin{align}\mu_{pm,c} &= (9+c)\mu_{\text{cr}} + 2\mu_{a} + \mu_{\text{me}}\\\
\sigma_{pm,c} &= \sqrt{(9+c)\sigma_{\text{cr}}^2 + 2\sigma_{a}^2 + \sigma_{\text{me}}^2}
\end{align}
where $c$ represents the number of category chains, pm indicates the sum of perceptual-motor processes, cr denotes conflict resolution, me denotes motor execution and a denotes visual attention. To make the notation more compact, we will represent these parameters with a single variable, $\gamma_c = \{\mu_{pm,c}, \sigma_{pm,c}\}$.

## State Space

The state space for the Semantic model is

$S = \{s_{\rm ir}, s_{\rm cc1},   s_{\rm cc2},  s_{\rm yes},  s_{\rm no}\}$

where $s_{\rm ir}$ is the initial retrieval state, $s_{\rm cc1}$ is the first category chain retrieval state, $s_{\rm ccs}$ is the second category chain retrieval state, $s_{\rm yes}$ is the state in which the model responds "yes", and $s_{\rm no}$ is the state in which model responds "no".

## Mappings

The mapping between retrievals and transitions is the same as described in the first version of the [Semantic model](../Semantic/Semantic_Model.ipynb). See the section titled "Transition Matrix" for details.

## Likelihood Function

The likelihood function for the Semantic model is divided into components for "yes" and "no" responses. Each component is a convolution of normal random variables representing perceptual-motor processes and LNR random variables representing memory retrieval processes. The likelihood function for "yes" responses is divided into two functions based on the number of category chains required to reach the correct response.

### Yes Responses

The likelihood of a "yes" response requiring zero category chains is given by:
\begin{equation}
    \mathcal{L}_0(\Theta; \textrm{yes},rt) = \sum_{\mathbf{c}_m \in R_{\rm ir,yes}} \left(f_{{\rm normal}; \gamma_0}
    * f_{{\rm LNR}; \mathbf{c}_m, \Theta}\right)(\textrm{rt}).
\end{equation}

The likelihood function for correctly responding "yes" with one category chain includes the convolution with an additional LNR random variable:
\begin{equation}
    \mathcal{L}_1(\Theta; \rm yes,rt) = \sum_{\mathbf{c}_z \in R_{\rm cc1,yes}} \sum_{\mathbf{c}_m \in R_{\rm ir,cc1}}
    \left(f_{{\rm normal}; \gamma_1}  *
    f_{{\rm LNR}; \mathbf{c}_m, \Theta} * f_{{\rm LNR}; \mathbf{c}_z,  \Theta}\right)(\textrm{rt}).
\end{equation}

### No Responses

Unlike the likelihood function for a "yes" response, the number of category chains leading to a "no" response is unknown. For this reason, the likelihood function is a mixture based on the number of possible category chains leading to a "no" response, either 0, 1, or 2, depending on the stimulus. Each component of this mixture is a mixture itself, based on mismatching chunks and retrieval failures\footnote{We follow the convention that the sum of an empty set is zero.}.

In the first case, the likelihood of responding "no" with zero category chains is defined as:
\begin{align*}
\mathcal{L}_0(\textrm{no},rt) = \sum_{\mathbf{c}_m \in R_{\rm ir,no}} \left(f_{{\rm normal}; \gamma_0} *  f_{{\rm LNR};\mathbf{c}_m,\Theta} \right)(\textrm{rt}).
\end{align*}
The likelihood function for responding "no" using one category chain is defined as:
\begin{align*}
\mathcal{L}_1(\Theta; \textrm{no},rt)  =
 \sum_{c_k \in R_{\rm ir,cc1}} \sum_{c_z \in R_{\rm cc1,no}}  \left(f_{{\rm normal}; \gamma_1} * f_{{\rm LNR}; \mathbf{c}_k,\Theta} * f_{{\rm LNR};\mathbf{c}_z,\Theta}\right) (\textrm{rt}).
\end{align*}
The likelihood function for responding "no using two category chains is defined as:
\begin{align*}
\mathcal{L}_2(\Theta; \textrm{no},rt) =
  \sum_{c_j \in R_{\rm ir,cc1}} \sum_{c_w \in R_{\rm cc1,cc2}} \sum_{c_s \in R_{\rm cc2,no}}
  \left(
  f_{{\rm normal}; \gamma_2} *
  f_{{\rm LNR};\mathbf{c}_j, \Theta} *
  f_{{\rm LNR};\mathbf{c}_w, \Theta} *
  f_{{\rm LNR};\mathbf{c}_s, \Theta} \right)(\textrm{rt})
\end{align*}

```julia
# change current directory to file's directory
cd(@__DIR__)
# load package manager
using Pkg
# activate the package environment
Pkg.activate("../../")
# load packages and model functions
using Revise, StatsPlots, ACTRModels, Distributions, FFTDists, DifferentialEvolutionMCMC
using MCMCChains
include("Semantic_FFT_Model.jl")
include("model_functions.jl")
Random.seed!(37101);
```

      Activating environment at `~/.julia/dev/FundamentalToolsACTR/Project.toml`
    ┌ Info: Precompiling DifferentialEvolutionMCMC [607db5a9-722a-4af8-9a06-1810c0fe385b]
    └ @ Base loading.jl:1342

# Generate Data

The `simulate` function generates multiple trials of simulated data for a given stimulus and returns a `NamedTuple` containing the stimulus, the rts for "yes" responses and the rts for "no" responses. `simulate` accepts the following inputs:

- fixed_parms: a `NamedTuple` of fixed parameters
- stimulus: a `NamedTuple` of stimulus slot-value pairs
- n_reps: the number of trials or repetitions for a given stimulus
- blc: the base-level constant parameter.
- $\delta$: the mismatch penalty parameter

A model object and the stimulus is passed to `simulate_trial` where most of the heavy lifting is performed. `simulate_trial` begins by adding conflict resolution time and stimulus encoding time to the variable `rt`. The next stage of the stimulation is performed in a `while` loop, which continues to execute until a response is given. This allows the model to perform multiple category chains depending on the stimulus and which chunk is retrieved. Inside the `while` loop, there are four states or conditions:

1. a retrieval failure, which leads to a "no" response
2. direct verify, which leads to a "yes" response when the chunk matches the retrieval request
3. chain category, which prompts the model "chain the category" by modifying the retrieval request and performing an additional memory retrieval
4. the state occurs when the chunk does not match a category chain or the retrievial request, resulting in a "no" response.

```julia
function simulate(fixed_parms, stimulus, n_reps; blc, δ)
    # generate chunks
    chunks = populate_memory()
    # add chunks to declarative memory
    memory = Declarative(;memory=chunks)
    # add declarative memory and parameters to ACT-R object
    actr = ACTR(;declarative=memory, fixed_parms..., blc, δ)
    # rts for yes responses
    yes_rts = Float64[]
    # rates for no respones
    no_rts = Float64[]
    for rep in 1:n_reps
        # simulate a single trial
        resp,rt = simulate_trial(actr, stimulus)
        # save simulated data
        resp == :yes ? push!(yes_rts, rt) : push!(no_rts, rt)
    end
    return (stimulus=stimulus, yes_rts = yes_rts, no_rts = no_rts)
end

function simulate_trial(actr, stimulus)
    retrieving = true
    probe = stimulus
    response = :_
    # add conflict resolution times
    rt = mapreduce(_ -> process_time(.05), +, 1:7)
    # add stimulus encoding times
    rt += mapreduce(_ -> process_time(.085), +, 1:2)
    while retrieving
        # conflict resolution
        rt += process_time(.05)
        chunk = retrieve(actr; object=probe.object, attribute=:category)
        rt += compute_RT(actr, chunk)
        # retrieval failure, respond "no"
        if isempty(chunk)
            # add motor execution time
            rt += process_time(.05) + process_time(.21)
            retrieving = false
            response = :no
        # can respond "yes"
        elseif direct_verify(chunk[1], probe)
            # add motor execution time
            rt += process_time(.05) + process_time(.21)
            retrieving = false
            response = :yes
        # category chain
        elseif chain_category(chunk[1], probe)
            probe = delete(probe, :object)
            # update memory probe for category chain
            probe = (object = chunk[1].slots.value, probe...)
        else
            rt += process_time(.05) + process_time(.21)
            retrieving = false
            response = :no
        end
    end
    return response, rt
end
```

    simulate_trial (generic function with 1 method)

We will generate 10 repetitions for each of the four stimuli. The resulting data are formated as an array of `NamedTuples` where

- stimulus: a `NamedTuple` of stimulus properties

- yes_rts: a vector of reaction times for yes responses

- no_rts: a vector of reaction times for no responses

```julia
# true value for base level constant
blc = 1.5
# true value for mismatch penalty
δ = 1.0
# fixed parameters
fixed_parms = (noise = true, τ = 0.0, s = 0.2, mmp = true)
stimuli = get_stimuli()
# repeat simulation 10 times per stimulus
n_reps = 10
data = map(s -> simulate(fixed_parms, s, n_reps; blc, δ), stimuli);
first(data)
```

    (stimulus = (object = :canary, category = :bird, ans = :yes), yes_rts = [0.9278338694769119, 1.092477724935062, 1.0702123205340264, 1.0636195698245046, 1.120007016805608, 1.2502342903580215, 1.135248920407693, 1.086731898816198], no_rts = [1.1710571563457006, 1.1582610596588718])

## Define Log Likelihood Function

The functions that comprise the likelihood function are orginized in a similar way to the mathematical description of the likelihood function. The top-level function `loglike` iterates through the data and calls one of three functions that computes the likelihood based on the maximum number of category chains for a given stimulus. These functions are `zero_chains`, `one_chain`, and `two_chains`. Each of these functions call other functions based on whether the response is "yes" or "no" and the number of category chains up to the maximum.

`loglike` requires the following arguments

- blc: the base level constant parameter
- $\delta$: the mismatch penalty parameter
- fixed_parms: a `NamedTuple` of parameters that are fixed
- data: an array of data

`loglike` initializes the model and model components before looping through the data. Within the loop, the functions `zero_chains`, `one_chain`, or `two_chains` are called, depending on the stimulus.

```julia
function loglike(data, fixed_parms, blc, δ)
    # populate memory
    chunks = populate_memory()
    # add chunks to declarative memory object
    memory = Declarative(;memory=chunks)
    # add declarative memory and parameters to ACT-R object
    actr = ACTR(;declarative=memory, fixed_parms..., blc, δ, noise=false)
    LL = 0.0
    for d in data
        stimulus = d.stimulus
        # evaluate log likelihood based on maximum number of catory chains
        if (stimulus.object == :canary) && (stimulus.category == :bird)
            LL += zero_chains(actr, stimulus, d)
        elseif (stimulus.object == :canary) && (stimulus.category == :fish)
            LL += two_chains(actr, stimulus, d)
        else
            LL += one_chain(actr, stimulus, d)
        end
    end
    return LL
end
```

    loglike (generic function with 1 method)

The functions `zero_chains`, `one_chain`, and `two_chains` compute the likelihood of yes and no responses for stimuli that have a maximum of zero, one or two chains. Each function requires the following inputs:

- actr: an ACT-R model object
- stimulus: a `NamedTuple` with slot-value pars for object, attribute and value
- data: a `NamedTuple` containing a vector of reaction times for yes responses and a vector of reaction times for no responses

```julia
function zero_chains(actr, stimulus, data)
    # this function computes yes and no responses that can be answered directly
    # no category chaining
    # log likelihood for yes responses
    LL = zero_chain_yes(actr, stimulus, data.yes_rts)
    # log likelihood for no responses
    LL += zero_chain_no(actr, stimulus, data.no_rts)
    return LL
end

function one_chain(actr, stimulus, data)
    # handles yes and no responses for a single category chain
    LL = one_chain_yes(actr, stimulus, data.yes_rts)
    LL += one_chain_no(actr, stimulus, data.no_rts)
    return LL
end

function two_chains(actr, stimulus, data)
    # no is the only response for two category chains
    return two_chains_no(actr, stimulus, data.no_rts)
end
```

    two_chains (generic function with 1 method)

For ease of exposition, we will only detail low level function, `zero_chain_no`. Similar functions can be found in `Semantic_FFT_Model.jl` located in this folder. `zero_chain_no` corresponds to the following function introduced above:

\begin{align*}
\mathcal{L}_0(\textrm{no},rt) = \sum_{\mathbf{c}_m \in R_{\rm ir,no}} \left(f_{{\rm normal}; \gamma_0} *  f_{{\rm LNR};\mathbf{c}_m,\Theta} \right)(\textrm{rt}).
\end{align*}

A question such as "Is a bird a canary" requires zero category chains because there are no intermediate categories between canary and bird. Thus, the set of chunks leading to a "no" response is the compliment of the set of chunks leading to a "yes" response (this set has only one element). At a high level, `zero_chain_no` performs two basic operations. First, it precomputes the activation values for each chunk and convolves the the distribution for perceptual-motor time, which is the same for all possible mappings in $R_{\rm ir,no}$. Second, it marginalizes across all chunks whose index does *not* match the index of the chunk that maps to a yes response.

```julia
function zero_chain_no(actr, stimulus, rts)
    @unpack τ,s = actr.parms
    chunks = actr.declarative.memory
    σ = s * π / sqrt(3)
    # normal approximation for perceptual-motor time
    μpm,σpm = convolve_normal(motor=(μ = .21,N = 1), cr=(μ = .05,N = 9), visual=(μ = .085,N = 2))
    # compute the activation for all chunks
    compute_activation!(actr; object=get_object(stimulus), attribute=:category)
    # extract the mean activation values
    μ = map(x -> x.act, chunks)
    # add retrieval threshold to mean activation values
    push!(μ, τ)
    # find the chunk index corresponding to a "yes" response
    yes_idx = find_index(actr, object=get_object(stimulus), value=get_category(stimulus))
    # Initialize likelihood
    n_resp = length(rts)
    # initialize likelihoods for each no response
    likelihoods = fill(0.0, n_resp)
    Nc = length(chunks) + 1
    # Marginalize over all of the possible chunks that could have lead to the
    # observed response
    for i in 1:Nc
        # Exclude the chunk representing the stimulus because the response was "no"
        if i != yes_idx
            # create Lognormal race distribution for chunk i
            retrieval_dist = LNRC(;μ=-μ, σ=σ, ϕ=0.0, c=i)
            # sum the percptual-motor distribution and the retrieval distribution
            model = Normal(μpm, σpm) + retrieval_dist
            # convolve the distributions
            convolve!(model)
            # compute likelihood for each rt using "." broadcasting
            likelihoods .+= pdf.(model, rts)
        end
    end
    return sum(log.(likelihoods))
end
```

    zero_chain_no (generic function with 1 method)

# Plot Likelihood Function

In the following code blocks, the likelihood function is superimposed on simulated data to visually confirm that the likelihood is correctly specified. The plots are divided up by the number of category chains and whether a "yes" or "no" respose is given. In each case, there is a close correspondence between the simulated data and the likelihood function represented by the solid black line. The "no" distributions for 1 and 2 category chains are bimodal because there are multiple ways to respond "no".

```julia
blc = 1.0
δ = 1.0
fixed_parms = (noise = true, τ = 0.0, s = 0.2, mmp = true)
chunks = populate_memory()
memory = Declarative(;memory=chunks)
actr = ACTR(;declarative=memory, fixed_parms..., blc=blc, δ=δ, noise=false)
stimuli = get_stimuli()
sim_reps = 10_000
stimulus = stimuli[1]
sim_data = simulate(fixed_parms, stimulus, sim_reps; blc=blc, δ=δ)
p_correct = length(sim_data.yes_rts) / sim_reps
x = .8:.01:1.8
exact_density_yes = map(i -> zero_chain_yes(actr, stimulus, i) |> exp, x)
pyplot()
plot_yes = histogram(sim_data.yes_rts, norm=true, grid=false, xlabel="Reaction Time (seconds)", yaxis="Density",
    bins=75, leg=false, title="Yes, Zero Chains")
plot_yes[1][1][:y] .*= p_correct
plot!(x, exact_density_yes, linewidth=2, color=:black)

exact_density_no = map(i -> zero_chain_no(actr, stimulus, i) |> exp, x)
plot_no = histogram(sim_data.no_rts, norm=true, grid=false, xlabel="Reaction Time (seconds)", yaxis="Density",
    bins=75, leg=false, title="No, Zero Chains")
plot_no[1][1][:y] .*= (1-p_correct)
plot!(x, exact_density_no, linewidth=2, color=:black)
plot(plot_yes, plot_no, layout=(2,1), size=(800,400))
```

```julia
blc = 1.0
δ = 1.0
fixed_parms = (noise = true, τ = 0.0, s = 0.2, mmp = true)
chunks = populate_memory()
memory = Declarative(;memory=chunks)
actr = ACTR(;declarative=memory, fixed_parms..., blc=blc, δ=δ, noise=false)
stimuli = get_stimuli()
sim_reps = 10_000
stimulus = stimuli[2]
sim_data = simulate(fixed_parms, stimulus, sim_reps; blc=blc, δ=δ)
p_correct = length(sim_data.yes_rts) / sim_reps
x = .8:.01:2.25
exact_density_yes = map(i -> one_chain_yes(actr, stimulus, i) |> exp, x)
pyplot()
plot_yes = histogram(sim_data.yes_rts, norm=true, grid=false, xlabel="Reaction Time (seconds)", yaxis="Density",
    leg=false, bins=75, title="Yes, One Chain")
plot_yes[1][1][:y] .*= p_correct
plot!(x, exact_density_yes, linewidth=2, color=:black)

exact_density_no = map(i -> one_chain_no(actr, stimulus, i) |> exp, x)
plot_no = histogram(sim_data.no_rts, norm=true, grid=false, xlabel="Reaction Time (seconds)", yaxis="Density",
    leg=false, bins=75, title="No, One Chain")
plot_no[1][1][:y] .*= (1-p_correct)
plot!(x, exact_density_no, linewidth=2, color=:black)
plot(plot_yes, plot_no, layout=(2,1), size=(800,400))
```

```julia
blc = 1.0
δ = 1.0
fixed_parms = (noise = true, τ = 0.0, s = 0.2, mmp = true)
chunks = populate_memory()
memory = Declarative(;memory=chunks)
actr = ACTR(;declarative=memory, fixed_parms..., blc=blc, δ=δ, noise=false)
stimuli = get_stimuli()
sim_reps = 10_000
stimulus = stimuli[4]
sim_data = simulate(fixed_parms, stimulus, sim_reps; blc=blc, δ=δ)
x = .8:.01:3.25

exact_density_no = map(i -> two_chains_no(actr, stimulus, i) |> exp, x)
plot_no = histogram(sim_data.no_rts, norm=true, grid=false, xlabel="Reaction Time (seconds)", yaxis="Density",
    bins=100, leg=false, title="No, Two Chains")
plot!(x, exact_density_no, linewidth=2, color=:black,size=(800,400))
```

## Define Model

The prior distributions and model are summarized as follows:

\begin{align}
\rm blc \sim Normal(1.5,1)
\end{align}

\begin{align}
\rm \delta \sim Normal(1,.5)
\end{align}

\begin{align}
\textrm{rts}_{\textrm{yes}} \sim f_{\textrm{yes}}(\textrm{blc},\delta)
\end{align}

\begin{align}
\textrm{rts}_{\textrm{no}} \sim f_{\textrm{no}}(\textrm{blc},\delta)
\end{align}

In computer code, the model is specified as follows:

```julia
# define prior distribution of parameters
priors = (
    blc = (Normal(1.5, 1),),
    δ = (Truncated(Normal(1.0, .5), 0.0, Inf),)
  )

# lower and upper bounds for each parameter
bounds = ((-Inf,Inf),(eps(),Inf))
```

    ((-Inf, Inf), (2.220446049250313e-16, Inf))

## Estimate Parameters

Now that the priors, likelihood and Turing model have been specified, we can now estimate the parameters. In the following code, we will run four MCMC chains with the NUTS sample for 2,000 iterations and omit the first 1,000 warmup samples.

```julia
# create model object
model = DEModel(fixed_parms; priors, model=loglike, data)
# create DE sampler object
de = DE(;bounds, burnin=1000, priors, n_groups=2, Np=4)
# total iterations
n_iter = 2000
chains = sample(model, de, MCMCThreads(), n_iter, progress=true)
```

    Progress: 100%|█████████████████████████████████████████| Time: 0:15:41

    Chains MCMC chain (1000×4×8 Array{Float64, 3}):

    Iterations        = 1:1:1000
    Number of chains  = 8
    Samples per chain = 1000
    parameters        = blc, δ
    internals         = acceptance, lp

    Summary Statistics
      parameters      mean       std   naive_se      mcse        ess      rhat
          Symbol   Float64   Float64    Float64   Float64    Float64   Float64

             blc    1.4535    0.0607     0.0007    0.0025   494.7616    1.0177
               δ    0.9508    0.0823     0.0009    0.0035   418.4100    1.0184

    Quantiles
      parameters      2.5%     25.0%     50.0%     75.0%     97.5%
          Symbol   Float64   Float64   Float64   Float64   Float64

             blc    1.3300    1.4122    1.4546    1.4964    1.5735
               δ    0.7963    0.8929    0.9489    1.0083    1.1134

    ┌ Warning: timestamp of type Missing unknown
    └ @ MCMCChains /home/dfish/.julia/packages/MCMCChains/tfR90/src/chains.jl:364
    ┌ Warning: timestamp of type Missing unknown
    └ @ MCMCChains /home/dfish/.julia/packages/MCMCChains/tfR90/src/chains.jl:364

## Results

According to the summary above, $\hat{r} \approx 1$ for both parameters, which suggests that the chains converged.
The trace plot below shows acceptable mixing among the chains.  In the density plots, the posterior distributions are centered near the data generating parameters of blc = 1.5 and $\delta = 1.$

```julia
pyplot()
font_size = 12
ch = group(chains, :blc)
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcτ = plot(p1, p2, p3, layout=(3,1), size=(600,600))
```

```julia
ch = group(chains, :δ)
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcτ = plot(p1, p2, p3, layout=(3,1), size=(600,600))
```

We can get a cleaner plot of the posterior distributions by pooling samples across chains:

```julia
posteriors = plot(chains, seriestype=:pooleddensity, grid=false,
     xaxis=font(12), yaxis=font(12), color=:grey, size=(500,500), linewidth=2)
plot!(title = "", xlabel = "blc", subplot = 1)
plot!(title = "", xlabel = "δ", subplot = 2)
```

## Correlation of posterior distribution

Inspecting correlations in the posterior distribution can be helpful for identifying tradeoffs between parameters or non-identifiability. Moderate correlations indicate tradeoffs between parameters whereas correlations close to 1 or -1 indicate problems with identifiability. The correlation matrix below indicates a moderate tradeoff between parameters $\delta$ and blc.

```julia
cor(chains)
```

    Correlation
      parameters       blc         δ
          Symbol   Float64   Float64

             blc    1.0000    0.7645
               δ    0.7645    1.0000

### Posterior Predictive Distribution

The plot below shows the posterior predictive distribution of reaction times of "yes" and "no" responses for each of the four stimuli. Three patterns can be observed: (1) reaction times increase with the number of category chains required to answer the question (e.g. reaction times are longer for canary-animal compared to canary-bird), (2) the model cannot respond "yes" when the correct answer is "no", (3) the mixture of category chains is apparent on "no" responses for the questions canary-animal, bird-fish, canary-fish.

```julia
rt_preds(s) = posterior_predictive(x -> simulate(fixed_parms, s, n_reps; x...), chains, 1000)
temp_preds = map(s -> rt_preds(s), stimuli)
preds = merge.(temp_preds)
grid_plot(preds, stimuli)
```

# References

Weaver, R. (2008). Parameters, predictions, and evidence in computational modeling: A statistical view informed by ACT–R. Cognitive Science, 32(8), 1349-1375.