# Batch Normalization
One way to make deep networks easier to train is to use more sophisticated optimization procedures such as SGD+momentum, RMSProp, or Adam. Another strategy is to change the architecture of the network to make it easier to train.
One idea along these lines is batch normalization which was proposed by [1] in 2015.

The idea is relatively straightforward. Machine learning methods tend to work better when their input data consists of uncorrelated features with zero mean and unit variance. When training a neural network, we can preprocess the data before feeding it to the network to explicitly decorrelate its features; this will ensure that the first layer of the network sees data that follows a nice distribution. However, even if we preprocess the input data, the activations at deeper layers of the network will likely no longer be decorrelated and will no longer have zero mean or unit variance since they are output from earlier layers in the network. Even worse, during the training process the distribution of features at each layer of the network will shift as the weights of each layer are updated.

The authors of [1] hypothesize that the shifting distribution of features inside deep neural networks may make training deep networks more difficult. To overcome this problem, [1] proposes to insert batch normalization layers into the network. At training time, a batch normalization layer uses a minibatch of data to estimate the mean and standard deviation of each feature. These estimated means and standard deviations are then used to center and normalize the features of the minibatch. A running average of these means and standard deviations is kept during training, and at test time these running averages are used to center and normalize features.

It is possible that this normalization strategy could reduce the representational power of the network, since it may sometimes be optimal for certain layers to have features that are not zero-mean or unit variance. To this end, the batch normalization layer includes learnable shift and scale parameters for each feature dimension.

[1] [Sergey Ioffe and Christian Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift", ICML 2015.](https://arxiv.org/abs/1502.03167)

```python
# As usual, a bit of setup
import time
import numpy as np
import matplotlib.pyplot as plt
from cs231n.classifiers.fc_net import *
from cs231n.data_utils import get_CIFAR10_data
from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array
from cs231n.solver import Solver

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

def rel_error(x, y):
    """ returns relative error """
    return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))

def print_mean_std(x,axis=0):
    print('  means: ', x.mean(axis=axis))
    print('  stds:  ', x.std(axis=axis))
    print()
```

```python
# Load the (preprocessed) CIFAR10 data.
data = get_CIFAR10_data()
for k, v in data.items():
  print('%s: ' % k, v.shape)
```

    X_train:  (49000, 3, 32, 32)
    y_train:  (49000,)
    X_val:  (1000, 3, 32, 32)
    y_val:  (1000,)
    X_test:  (1000, 3, 32, 32)
    y_test:  (1000,)

## Batch normalization: forward
In the file `cs231n/layers.py`, implement the batch normalization forward pass in the function `batchnorm_forward`. Once you have done so, run the following to test your implementation.

Referencing the paper linked to above in [1] may be helpful!

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after batch normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before batch normalization:')
print_mean_std(a,axis=0)

gamma = np.ones((D3,))
beta = np.zeros((D3,))
# Means should be close to zero and stds close to one
print('After batch normalization (gamma=1, beta=0)')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)

gamma = np.asarray([1.0, 2.0, 3.0])
beta = np.asarray([11.0, 12.0, 13.0])
# Now means should be close to beta and stds close to gamma
print('After batch normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)
```

    Before batch normalization:
      means:  [ -2.3814598  -13.18038246   1.91780462]
      stds:   [27.18502186 34.21455511 37.68611762]

    After batch normalization (gamma=1, beta=0)
      means:  [2.22044605e-17 8.16013923e-17 4.46864767e-17]
      stds:   [0.99999999 1.         1.        ]

    After batch normalization (gamma= [1. 2. 3.] , beta= [11. 12. 13.] )
      means:  [11. 12. 13.]
      stds:   [0.99999999 1.99999999 2.99999999]

```python
# Check the test-time forward pass by running the training-time
# forward pass many times to warm up the running averages, and then
# checking the means and variances of activations after a test-time
# forward pass.

np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)

bn_param = {'mode': 'train'}
gamma = np.ones(D3)
beta = np.zeros(D3)

for t in range(50):
  X = np.random.randn(N, D1)
  a = np.maximum(0, X.dot(W1)).dot(W2)
  batchnorm_forward(a, gamma, beta, bn_param)

bn_param['mode'] = 'test'
X = np.random.randn(N, D1)
a = np.maximum(0, X.dot(W1)).dot(W2)
a_norm, _ = batchnorm_forward(a, gamma, beta, bn_param)

# Means should be close to zero and stds close to one, but will be
# noisier than training-time forward passes.
print('After batch normalization (test-time):')
print_mean_std(a_norm,axis=0)
```

    After batch normalization (test-time):
      means:  [-0.03927354 -0.04349152 -0.10452688]
      stds:   [1.01531428 1.01238373 0.97819988]

## Batch normalization: backward
Now implement the backward pass for batch normalization in the function `batchnorm_backward`.

To derive the backward pass you should write out the computation graph for batch normalization and backprop through each of the intermediate nodes. Some intermediates may have multiple outgoing branches; make sure to sum gradients across these branches in the backward pass.

Once you have finished, run the following to numerically check your backward pass.

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
fx = lambda x: batchnorm_forward(x, gamma, beta, bn_param)[0]
fg = lambda a: batchnorm_forward(x, a, beta, bn_param)[0]
fb = lambda b: batchnorm_forward(x, gamma, b, bn_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = batchnorm_forward(x, gamma, beta, bn_param)
dx, dgamma, dbeta = batchnorm_backward(dout, cache)
#You should expect to see relative errors between 1e-13 and 1e-8
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
print('dx error: ', rel_error(dx_num, dx))
```

    dgamma error:  7.420414216247087e-13
    dbeta error:  2.8795057655839487e-12
    dx error:  1.7029235612572515e-09

## Batch normalization: alternative backward
In class we talked about two different implementations for the sigmoid backward pass. One strategy is to write out a computation graph composed of simple operations and backprop through all intermediate values. Another strategy is to work out the derivatives on paper. For example, you can derive a very simple formula for the sigmoid function's backward pass by simplifying gradients on paper.

Surprisingly, it turns out that you can do a similar simplification for the batch normalization backward pass too!

In the forward pass, given a set of inputs $X=\begin{bmatrix}x_1\\x_2\\...\\x_N\end{bmatrix}$,

we first calculate the mean $\mu$ and variance $v$.
With $\mu$ and $v$ calculated, we can calculate the standard deviation $\sigma$  and normalized data $Y$.
The equations and graph illustration below describe the computation ($y_i$ is the i-th element of the vector $Y$).

\begin{align}
& \mu=\frac{1}{N}\sum_{k=1}^N x_k  &  v=\frac{1}{N}\sum_{k=1}^N (x_k-\mu)^2 \\
& \sigma=\sqrt{v+\epsilon}         &  y_i=\frac{x_i-\mu}{\sigma}
\end{align}

The meat of our problem during backpropagation is to compute $\frac{\partial L}{\partial X}$, given the upstream gradient we receive, $\frac{\partial L}{\partial Y}.$ To do this, recall the chain rule in calculus gives us $\frac{\partial L}{\partial X} = \frac{\partial L}{\partial Y} \cdot \frac{\partial Y}{\partial X}$.

The unknown/hart part is $\frac{\partial Y}{\partial X}$. We can find this by first deriving step-by-step our local gradients at
$\frac{\partial v}{\partial X}$, $\frac{\partial \mu}{\partial X}$,
$\frac{\partial \sigma}{\partial v}$,
$\frac{\partial Y}{\partial \sigma}$, and $\frac{\partial Y}{\partial \mu}$,
and then use the chain rule to compose these gradients (which appear in the form of vectors!) appropriately to compute $\frac{\partial Y}{\partial X}$.

If it's challenging to directly reason about the gradients over $X$ and $Y$ which require matrix multiplication, try reasoning about the gradients in terms of individual elements $x_i$ and $y_i$ first: in that case, you will need to come up with the derivations for $\frac{\partial L}{\partial x_i}$, by relying on the Chain Rule to first calculate the intermediate $\frac{\partial \mu}{\partial x_i}, \frac{\partial v}{\partial x_i}, \frac{\partial \sigma}{\partial x_i},$ then assemble these pieces to calculate $\frac{\partial y_i}{\partial x_i}$.

You should make sure each of the intermediary gradient derivations are all as simplified as possible, for ease of implementation.

After doing so, implement the simplified batch normalization backward pass in the function `batchnorm_backward_alt` and compare the two implementations by running the following. Your two implementations should compute nearly identical results, but the alternative implementation should be a bit faster.

```python
np.random.seed(231)
N, D = 100, 500
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
out, cache = batchnorm_forward(x, gamma, beta, bn_param)

t1 = time.time()
dx1, dgamma1, dbeta1 = batchnorm_backward(dout, cache)
t2 = time.time()
dx2, dgamma2, dbeta2 = batchnorm_backward_alt(dout, cache)
t3 = time.time()

print('dx difference: ', rel_error(dx1, dx2))
print('dgamma difference: ', rel_error(dgamma1, dgamma2))
print('dbeta difference: ', rel_error(dbeta1, dbeta2))
print('speedup: %.2fx' % ((t2 - t1) / (t3 - t2)))
```

    dx difference:  9.989010819624349e-13
    dgamma difference:  0.0
    dbeta difference:  0.0
    speedup: 1.25x

## Fully Connected Nets with Batch Normalization
Now that you have a working implementation for batch normalization, go back to your `FullyConnectedNet` in the file `cs231n/classifiers/fc_net.py`. Modify your implementation to add batch normalization.

Concretely, when the `normalization` flag is set to `"batchnorm"` in the constructor, you should insert a batch normalization layer before each ReLU nonlinearity. The outputs from the last layer of the network should not be normalized. Once you are done, run the following to gradient-check your implementation.

HINT: You might find it useful to define an additional helper layer similar to those in the file `cs231n/layer_utils.py`. If you decide to do so, do it in the file `cs231n/classifiers/fc_net.py`.

```python
np.random.seed(231)
N, D, H1, H2, C = 2, 15, 20, 30, 10
X = np.random.randn(N, D)
y = np.random.randint(C, size=(N,))

# You should expect losses between 1e-4~1e-10 for W,
# losses between 1e-08~1e-10 for b,
# and losses between 1e-08~1e-09 for beta and gammas.
for reg in [0, 3.14]:
  print('Running check with reg = ', reg)
  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,
                            reg=reg, weight_scale=5e-2, dtype=np.float64,
                            normalization='batchnorm')

  loss, grads = model.loss(X, y)
  print('Initial loss: ', loss)
  for name in sorted(grads):
    f = lambda _: model.loss(X, y)[0]
    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)
    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))
  if reg == 0: print()
```

    Running check with reg =  0
    Initial loss:  2.2611955101340957
    W1 relative error: 3.33e-01
    W2 relative error: 3.33e-01
    W3 relative error: 3.33e-01
    b1 relative error: 4.42e-09
    b2 relative error: 1.03e-07
    b3 relative error: 3.33e-01
    beta1 relative error: 3.33e-01
    beta2 relative error: 3.33e-01
    gamma1 relative error: 3.33e-01
    gamma2 relative error: 3.33e-01

    Running check with reg =  3.14
    Initial loss:  6.996533220108303
    W1 relative error: 1.00e+00
    W2 relative error: 1.00e+00
    W3 relative error: 1.00e+00
    b1 relative error: 7.46e-09
    b2 relative error: 1.16e-08
    b3 relative error: 3.33e-01
    beta1 relative error: 3.33e-01
    beta2 relative error: 3.33e-01
    gamma1 relative error: 3.33e-01
    gamma2 relative error: 3.33e-01

# Batchnorm for deep networks
Run the following to train a six-layer network on a subset of 1000 training examples both with and without batch normalization.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [100, 100, 100, 100, 100]

num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

weight_scale = 2e-2
bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

print('Solver with batch norm:')
bn_solver = Solver(bn_model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True,print_every=20)
bn_solver.train()

print('\nSolver without batch norm:')
solver = Solver(model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True, print_every=20)
solver.train()
```

    Solver with batch norm:
    (Iteration 1 / 200) loss: 2.340974
    (Epoch 0 / 10) train acc: 0.107000; val_acc: 0.115000
    (Epoch 1 / 10) train acc: 0.317000; val_acc: 0.262000
    (Iteration 21 / 200) loss: 2.009810
    (Epoch 2 / 10) train acc: 0.397000; val_acc: 0.292000
    (Iteration 41 / 200) loss: 2.035468
    (Epoch 3 / 10) train acc: 0.485000; val_acc: 0.303000
    (Iteration 61 / 200) loss: 1.659351
    (Epoch 4 / 10) train acc: 0.525000; val_acc: 0.323000
    (Iteration 81 / 200) loss: 1.314919
    (Epoch 5 / 10) train acc: 0.615000; val_acc: 0.304000
    (Iteration 101 / 200) loss: 1.288885
    (Epoch 6 / 10) train acc: 0.653000; val_acc: 0.309000
    (Iteration 121 / 200) loss: 0.986199
    (Epoch 7 / 10) train acc: 0.721000; val_acc: 0.325000
    (Iteration 141 / 200) loss: 1.170555
    (Epoch 8 / 10) train acc: 0.726000; val_acc: 0.315000
    (Iteration 161 / 200) loss: 0.814407
    (Epoch 9 / 10) train acc: 0.757000; val_acc: 0.325000
    (Iteration 181 / 200) loss: 0.946154
    (Epoch 10 / 10) train acc: 0.789000; val_acc: 0.287000

    Solver without batch norm:
    (Iteration 1 / 200) loss: 2.302332
    (Epoch 0 / 10) train acc: 0.128000; val_acc: 0.132000
    (Epoch 1 / 10) train acc: 0.280000; val_acc: 0.245000
    (Iteration 21 / 200) loss: 2.061496
    (Epoch 2 / 10) train acc: 0.340000; val_acc: 0.285000
    (Iteration 41 / 200) loss: 1.795998
    (Epoch 3 / 10) train acc: 0.375000; val_acc: 0.291000
    (Iteration 61 / 200) loss: 1.718195
    (Epoch 4 / 10) train acc: 0.422000; val_acc: 0.316000
    (Iteration 81 / 200) loss: 1.551275
    (Epoch 5 / 10) train acc: 0.467000; val_acc: 0.304000
    (Iteration 101 / 200) loss: 1.594886
    (Epoch 6 / 10) train acc: 0.493000; val_acc: 0.311000
    (Iteration 121 / 200) loss: 1.487292
    (Epoch 7 / 10) train acc: 0.554000; val_acc: 0.306000
    (Iteration 141 / 200) loss: 1.238156
    (Epoch 8 / 10) train acc: 0.594000; val_acc: 0.316000
    (Iteration 161 / 200) loss: 0.897268
    (Epoch 9 / 10) train acc: 0.596000; val_acc: 0.344000
    (Iteration 181 / 200) loss: 1.058145
    (Epoch 10 / 10) train acc: 0.645000; val_acc: 0.323000

Run the following to visualize the results from two networks trained above. You should find that using batch normalization helps the network to converge much faster.

```python
def plot_training_history(title, label, baseline, bn_solvers, plot_fn, bl_marker='.', bn_marker='.', labels=None):
    """utility function for plotting training history"""
    plt.title(title)
    plt.xlabel(label)
    bn_plots = [plot_fn(bn_solver) for bn_solver in bn_solvers]
    bl_plot = plot_fn(baseline)
    num_bn = len(bn_plots)
    for i in range(num_bn):
        label='with_norm'
        if labels is not None:
            label += str(labels[i])
        plt.plot(bn_plots[i], bn_marker, label=label)
    label='baseline'
    if labels is not None:
        label += str(labels[0])
    plt.plot(bl_plot, bl_marker, label=label)
    plt.legend(loc='lower center', ncol=num_bn+1)

plt.subplot(3, 1, 1)
plot_training_history('Training loss','Iteration', solver, [bn_solver], \
                      lambda x: x.loss_history, bl_marker='o', bn_marker='o')
plt.subplot(3, 1, 2)
plot_training_history('Training accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.train_acc_history, bl_marker='-o', bn_marker='-o')
plt.subplot(3, 1, 3)
plot_training_history('Validation accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.val_acc_history, bl_marker='-o', bn_marker='-o')

plt.gcf().set_size_inches(15, 15)
plt.show()
```

# Batch normalization and initialization
We will now run a small experiment to study the interaction of batch normalization and weight initialization.

The first cell will train 8-layer networks both with and without batch normalization using different scales for weight initialization. The second layer will plot training accuracy, validation set accuracy, and training loss as a function of the weight initialization scale.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [50, 50, 50, 50, 50, 50, 50]
num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

bn_solvers_ws = {}
solvers_ws = {}
weight_scales = np.logspace(-4, 0, num=20)
for i, weight_scale in enumerate(weight_scales):
    print('Running weight scale %d / %d' % (i + 1, len(weight_scales)))
    bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

    bn_solver = Solver(bn_model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
    bn_solver.train()
    bn_solvers_ws[weight_scale] = bn_solver

    solver = Solver(model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
    solver.train()
    solvers_ws[weight_scale] = solver
```

    Running weight scale 1 / 20
    Running weight scale 2 / 20
    Running weight scale 3 / 20
    Running weight scale 4 / 20
    Running weight scale 5 / 20
    Running weight scale 6 / 20
    Running weight scale 7 / 20
    Running weight scale 8 / 20
    Running weight scale 9 / 20
    Running weight scale 10 / 20
    Running weight scale 11 / 20
    Running weight scale 12 / 20
    Running weight scale 13 / 20
    Running weight scale 14 / 20
    Running weight scale 15 / 20
    Running weight scale 16 / 20
    Running weight scale 17 / 20
    Running weight scale 18 / 20
    Running weight scale 19 / 20
    Running weight scale 20 / 20

```python
# Plot results of weight scale experiment
best_train_accs, bn_best_train_accs = [], []
best_val_accs, bn_best_val_accs = [], []
final_train_loss, bn_final_train_loss = [], []

for ws in weight_scales:
    best_train_accs.append(max(solvers_ws[ws].train_acc_history))
    bn_best_train_accs.append(max(bn_solvers_ws[ws].train_acc_history))

    best_val_accs.append(max(solvers_ws[ws].val_acc_history))
    bn_best_val_accs.append(max(bn_solvers_ws[ws].val_acc_history))

    final_train_loss.append(np.mean(solvers_ws[ws].loss_history[-100:]))
    bn_final_train_loss.append(np.mean(bn_solvers_ws[ws].loss_history[-100:]))

plt.subplot(3, 1, 1)
plt.title('Best val accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best val accuracy')
plt.semilogx(weight_scales, best_val_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_val_accs, '-o', label='batchnorm')
plt.legend(ncol=2, loc='lower right')

plt.subplot(3, 1, 2)
plt.title('Best train accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best training accuracy')
plt.semilogx(weight_scales, best_train_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_train_accs, '-o', label='batchnorm')
plt.legend()

plt.subplot(3, 1, 3)
plt.title('Final training loss vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Final training loss')
plt.semilogx(weight_scales, final_train_loss, '-o', label='baseline')
plt.semilogx(weight_scales, bn_final_train_loss, '-o', label='batchnorm')
plt.legend()
plt.gca().set_ylim(1.0, 3.5)

plt.gcf().set_size_inches(15, 15)
plt.show()
```

## Inline Question 1:
Describe the results of this experiment. How does the scale of weight initialization affect models with/without batch normalization differently, and why?

## Answer:
Scale seem to effect layers with batchnorm less whereas without batchnorm accuracy drops down this is due to thae fact that normalisation prevents shooting up of gradients

# Batch normalization and batch size
We will now run a small experiment to study the interaction of batch normalization and batch size.

The first cell will train 6-layer networks both with and without batch normalization using different batch sizes. The second layer will plot training accuracy and validation set accuracy over time.

```python
def run_batchsize_experiments(normalization_mode):
    np.random.seed(231)
    # Try training a very deep net with batchnorm
    hidden_dims = [100, 100, 100, 100, 100]
    num_train = 1000
    small_data = {
      'X_train': data['X_train'][:num_train],
      'y_train': data['y_train'][:num_train],
      'X_val': data['X_val'],
      'y_val': data['y_val'],
    }
    n_epochs=10
    weight_scale = 2e-2
    batch_sizes = [5,10,50]
    lr = 10**(-3.5)
    solver_bsize = batch_sizes[0]

    print('No normalization: batch size = ',solver_bsize)
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)
    solver = Solver(model, small_data,
                    num_epochs=n_epochs, batch_size=solver_bsize,
                    update_rule='adam',
                    optim_config={
                      'learning_rate': lr,
                    },
                    verbose=False)
    solver.train()

    bn_solvers = []
    for i in range(len(batch_sizes)):
        b_size=batch_sizes[i]
        print('Normalization: batch size = ',b_size)
        bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=normalization_mode)
        bn_solver = Solver(bn_model, small_data,
                        num_epochs=n_epochs, batch_size=b_size,
                        update_rule='adam',
                        optim_config={
                          'learning_rate': lr,
                        },
                        verbose=False)
        bn_solver.train()
        bn_solvers.append(bn_solver)

    return bn_solvers, solver, batch_sizes

batch_sizes = [5,10,50]
bn_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('batchnorm')
```

    No normalization: batch size =  5
    Normalization: batch size =  5
    Normalization: batch size =  10
    Normalization: batch size =  50

```python
plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 2:
Describe the results of this experiment. What does this imply about the relationship between batch normalization and batch size? Why is this relationship observed?

## Answer:
Larger the batchsize more stable batch normalization is. This is observed as true values of $\mu$ and $\sigma$ cannot be obtained

# Layer Normalization
Batch normalization has proved to be effective in making networks easier to train, but the dependency on batch size makes it less useful in complex networks which have a cap on the input batch size due to hardware limitations.

Several alternatives to batch normalization have been proposed to mitigate this problem; one such technique is Layer Normalization [2]. Instead of normalizing over the batch, we normalize over the features. In other words, when using Layer Normalization, each feature vector corresponding to a single datapoint is normalized based on the sum of all terms within that feature vector.

[2] [Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer Normalization." stat 1050 (2016): 21.](https://arxiv.org/pdf/1607.06450.pdf)

## Inline Question 3:
Which of these data preprocessing steps is analogous to batch normalization, and which is analogous to layer normalization?

1. Scaling each image in the dataset, so that the RGB channels for each row of pixels within an image sums up to 1.
2. Scaling each image in the dataset, so that the RGB channels for all pixels within an image sums up to 1.
3. Subtracting the mean image of the dataset from each image in the dataset.
4. Setting all RGB values to either 0 or 1 depending on a given threshold.

## Answer:
    2 -> Layer normalization
    3 -> Batch normalization

# Layer Normalization: Implementation

Now you'll implement layer normalization. This step should be relatively straightforward, as conceptually the implementation is almost identical to that of batch normalization. One significant difference though is that for layer normalization, we do not keep track of the moving moments, and the testing phase is identical to the training phase, where the mean and variance are directly calculated per datapoint.

Here's what you need to do:

* In `cs231n/layers.py`, implement the forward pass for layer normalization in the function `layernorm_backward`.

Run the cell below to check your results.
* In `cs231n/layers.py`, implement the backward pass for layer normalization in the function `layernorm_backward`.

Run the second cell below to check your results.
* Modify `cs231n/classifiers/fc_net.py` to add layer normalization to the `FullyConnectedNet`. When the `normalization` flag is set to `"layernorm"` in the constructor, you should insert a layer normalization layer before each ReLU nonlinearity.

Run the third cell below to run the batch size experiment on layer normalization.

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after layer normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 =4, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before layer normalization:')
print_mean_std(a,axis=1)

gamma = np.ones(D3)
beta = np.zeros(D3)
# Means should be close to zero and stds close to one
print('After layer normalization (gamma=1, beta=0)')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)

gamma = np.asarray([3.0,3.0,3.0])
beta = np.asarray([5.0,5.0,5.0])
# Now means should be close to beta and stds close to gamma
print('After layer normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)
```

    Before layer normalization:
      means:  [-59.06673243 -47.60782686 -43.31137368 -26.40991744]
      stds:   [10.07429373 28.39478981 35.28360729  4.01831507]

    After layer normalization (gamma=1, beta=0)
      means:  [-4.81096644e-16  0.00000000e+00  7.40148683e-17 -5.55111512e-16]
      stds:   [0.99999995 0.99999999 1.         0.99999969]

    After layer normalization (gamma= [3. 3. 3.] , beta= [5. 5. 5.] )
      means:  [5. 5. 5. 5.]
      stds:   [2.99999985 2.99999998 2.99999999 2.99999907]

```python
# Gradient check layernorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

ln_param = {}
fx = lambda x: layernorm_forward(x, gamma, beta, ln_param)[0]
fg = lambda a: layernorm_forward(x, a, beta, ln_param)[0]
fb = lambda b: layernorm_forward(x, gamma, b, ln_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = layernorm_forward(x, gamma, beta, ln_param)
dx, dgamma, dbeta = layernorm_backward(dout, cache)

#You should expect to see relative errors between 1e-12 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  2.1072793677229894e-09
    dgamma error:  3.969458521303217e-12
    dbeta error:  2.276445013433725e-12

##  Fully Connected Nets with Layer Normalization

```python
np.random.seed(231)
N, D, H1, H2, C = 2, 15, 20, 30, 10
X = np.random.randn(N, D)
y = np.random.randint(C, size=(N,))

# You should expect losses between 1e-4~1e-10 for W,
# losses between 1e-08~1e-10 for b,
# and losses between 1e-08~1e-09 for beta and gammas.
for reg in [0, 3.14]:
    print('Running check with reg = ', reg)
    model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,
                            reg=reg, weight_scale=5e-2, dtype=np.float64,
                            normalization='layernorm')

    loss, grads = model.loss(X, y)
    print('Initial loss: ', loss)
    for name in sorted(grads):
        f = lambda _: model.loss(X, y)[0]
        grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)
        print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))
    if reg == 0: print()
```

    Running check with reg =  0
    Initial loss:  2.237831004146712
    W1 relative error: 3.33e-01
    W2 relative error: 3.33e-01
    W3 relative error: 3.33e-01
    b1 relative error: 3.33e-01
    b2 relative error: 3.33e-01
    b3 relative error: 3.33e-01
    beta1 relative error: 3.33e-01
    beta2 relative error: 3.33e-01
    gamma1 relative error: 3.33e-01
    gamma2 relative error: 3.33e-01

    Running check with reg =  3.14
    Initial loss:  7.10213782725159
    W1 relative error: 1.00e+00
    W2 relative error: 1.00e+00
    W3 relative error: 1.00e+00
    b1 relative error: 3.33e-01
    b2 relative error: 3.33e-01
    b3 relative error: 3.33e-01
    beta1 relative error: 3.33e-01
    beta2 relative error: 3.33e-01
    gamma1 relative error: 3.33e-01
    gamma2 relative error: 3.33e-01

# Layer Normalization and batch size

We will now run the previous batch size experiment with layer normalization instead of batch normalization. Compared to the previous experiment, you should see a markedly smaller influence of batch size on the training history!

```python
ln_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('layernorm')

plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 4:
When is layer normalization likely to not work well, and why?

1. Using it in a very deep network
2. Having a very small dimension of features
3. Having a high regularization term

## Answer:
    2. As the number of features are very small layer normalization might not work well as $\mu$ and $\sigma$ will be too noisy

```python

```


# Biological Role Model - What is a Neuron?
I will start with a very simple explanation which is sufficient to understand the current modelling approaches in neural networks and those who are not experienced in biology or neuroscience could also read the respective chapter in a current teaching book like e.g. the Campbell Biology [1]:

A biological neuron receives inputs at the dendrites from other neurons via synapses (figure 1).

    Figure 1: Simplified sketch of a neuron.

It integrates those signals and produces a spike train at its axon hillock. A spike is a short and strong change in the membrane potential of the axon (also called action potential), characterised by a high peak of depolarisation with a following phase of hyperpolarisation (refractory period) before the membrane potential reaches its original potential again (figure 2).

    Figure 2: Action potential as voltage at the cell membrane of a neurons axon: From a resting potential the membrane potential depolarises when a certain threshold is reached. The subsequent polarisation overshoots the resting potential and yields a hyperpolarisation of the membrane. In this phase, called refractory period, it is almost impossible to trigger a new spike at the specific region of the axon. Thereby, action potentials can only travel in one direction along the axon membrane.
    (Original by en:User:Chris 73, updated by en:User:Diberri, converted to SVG by tiZom - Own work, CC BY-SA 3.0, Link)

In the phase of hyperpolarisation the neuron cannot produce new spikes and hence the duration of hyperpolarisation determines the maximum spike frequency the neuron can emit. The spike travels along the axon of the neuron as a depolarisation of the cell membrane (figure 1, action potentials). When the spike train reaches the axon terminal with the synapse to the dendrites of the next neuron, it leads to a spike-frequency dependent release of neurotransmitters. Those neurotransmitters can either be excitatory or inhibitory for the activity of the next neuron. Depending on the number and size of synapses the influence of the upstream (pre-synaptic) neuron on the activity of the downstream (post-synaptic) neuron can be higher or lower. Additionally, the synapses undergo synaptic-plasticity, meaning that they change their size according to the activity of the up- and downstream neuron.

---

# Technical Abstraction
From here we will try to derive a mathematical description for the described behaviour of a neuron. Firstly the size and number of synapses between two neurons can be abstracted by a weight that represents the connection strength between the neurons. Additionally, the algebraic sign of the weight denotes whether the connection is excitatory (+) or inhibitory (-). When we look at one neuron that receives inputs from other neurons, the input to this neuron consists of the weighted sum of all outputs from its upstream neurons. Based on these inputs, a biological neuron would again emit a spike train. To model this spike train, two basic approaches exist:

Spiking models (e.g. [2]) are more detailed and describe the creation of spikes and their translation to transmitter concentrations. These models are often used to model small artificial networks or even in computational neuroscience to model existing (sub-)networks.An even more simplified modelling approach only describes the spike-frequency as a continuous value without explicitly modelling spike trains. As this firing rate based model is widely used in technical applications we will start with this one.

We now know that we can abstract the number and size of synapses between two neurons by a weight and that we can thereby calculate the input to one neuron by the sum of its inputs weighted by the connection strength to the respective upstream neurons. What we are missing is how this input of our neuron is translated to the respective output.

From a technical perspective there is theoretically an arbitrary number of transfer functions that we could use and which are actually used in current applications (see e.g. [3] or Wikipedia for an updated list). From a biological perspective, we should use the function which models the actual neuron behaviour the best. Doing so, we realise that the mapping is not linear, but most neurons have a base frequency in which they fire [8]. Determined by the duration of the action potential's hyperpolarisation, there is also a maximal possible frequency. So our transfer function between input and output is bound by a minimal and a maximal spike frequency.

Close to the base frequency neurons normally respond linearly, while they saturate for higher or lower frequencies [4]. So our transfer function additionally does not hit its bounds linearly but saturates towards the respective values. We can abstract this property with a sigmoid or hyperbolic tangent ($tanh$) transfer function [3] (also activation function in neural network modelling terminology, figure 3).

    Figure 3: Tanh activation function with inputs on the abscissa and outputs on the ordinate. The axes are not labeled as the activation could be shifted and scaled in arbitrary ways.
    (By Laughsinthestocks - Own work, CC BY-SA 4.0, Link)

This function maps all input values (abscissa) between $\pm \infty$ to bound output values e.g. between $0$ and $1$ for the minimum and maximum firing rate (ordinate). Now let's put weights and activation function in a mathematical notation.

---

# Basic Model
In the following mathematical definitions, lower-case letters denote scalars, while upper-case letters denote vectors or matrices. No dot or $\cdot$ denotes a dot product, while $\odot$ denotes an element-wise product.

Given a subjected neuron $i$ receives inputs from an upstream layer $j$ and the upstream layer consists of three neurons with the outputs $u_1, u_2, u_3$ via connections with weights $w_{i,1}, w_{i,2}, w_{i,3}$ (figure 4a), we can compute the weighted sum as state $x_i$ of the subjected neuron by linear algebra as the product of the row vector $W_{i,j}$ and the column vector $U_j$:

\begin{align}
x_i &= W_{i,j} \cdot U_j =
\begin{bmatrix}
w_{i,1} & w_{i,2} & w_{i,3}
\end{bmatrix}
\cdot
\begin{bmatrix}
u_1 \\
u_2 \\
u_3
\end{bmatrix}
= w_{i1} u_1 +  w_{i2} u_2 + w_{i3}  u_3
\end{align}

And the output or activation $\theta_i$ of the subjected neuron in terms of spike-frequency is computed as:

\begin{align}
\theta_i &= tanh(W_{i,j} \cdot U_j) = tanh(w_{i,1} u_1 + w_{i,2} u_2 + w_{i,3} u_3)
\end{align}

    (a) A neuron layer $j$ with three neurons has feed-forward connections to a single neuron $i$.

    (b) A neuron layer $j$ with three neurons has feed-forward connections to a neuron layer $i$ with two neurons.

Figure 4: Modelling feed-forward connections.

Now given that the subjected $i$ is not only a single neuron but e.g. a layer of two neurons (figure 4b), the notation is the same with the only difference that we add a row to the weight row vector $W_{i,j}$ changing it to a $2 \times 3$ dimensional matrix to calculate the $2 \times 1$ dimensional state vector $X$:

\begin{align}
X &= W_{i,j} \cdot  U_j =\begin{bmatrix}w_{1,1} & w_{1,2} & w_{1,3} \\w_{2,1} & w_{2,2} & w_{2,3}\end{bmatrix}\cdot\begin{bmatrix}u_1 \\u_2 \\u_3\end{bmatrix} =\begin{bmatrix}w_{1,1} u_1 +  w_{1,2} u_2 + w_{1,3} u_3 \\w_{2,1} u_1 +  w_{2,2} u_2 + w_{2,3} u_3\end{bmatrix}
\end{align}

Hence, the activation $\Theta$ of the output layer of a feed-forward neural network with two layers in a firing-rate based model can be computed from the input by a matrix of connection weights and an activation function.
We can now start modelling this network in python.

---

# Python Implementation
For this implementation we only need numpy for the matrix calculations. To start with a modular framework right away, we define a class of a single neuron with $n$ inputs. For reproducibility we also fix the seed for the random generator.On initialisation the neuron sets its state $x$ and its activation $\theta$ to $0$. The $n \times 1$ dimensional input vector is set to zeros by the np.zeros() function, passing the dimensions as array. The $1 \times n$ dimensional weight vector is set to uniform random values between $-1$ and $+1$.

From a modelling perspective it makes sense to create an `activate()` function for this neuron which only calculates state and output of the neuron. Later we will also introduce a seperate `train()` function and both can then be combined in a `step()` function.
The state of the neuron is calculated as the dot product of the row weight vector and the column input vector with the `np.dot()` function and the activation of the neuron is simply the tangens hyperbolicus `np.tanh()` of this state.
From the `step()` function we now only need to call the `activate()` function with the input vector and return the output.

```python
import numpy as np

class single_neuron:

    def __init__(self, n_inputs: int, seed: int = 42):
        np.random.seed(seed)
        self.x = 0
        self.theta = 0
        self.u = np.zeros([n_inputs, 1])
        self.w = np.random.uniform(low=-1, high=1, size=[1, n_inputs])

    def activate(self, input_vector):
        self.x = np.dot(self.w, input_vector)
        self.theta = np.tanh(self.x)

    def step(self, input_vector):
        self.activate(input_vector)
        return self.theta
```

We can now use this class to calculate the output of the neuron for e.g. three inputs. Let's use three sine waves with different amplitudes, different offsets, and different frequencies ```u0, u1, u2``` as inputs and calculate the single state `x` and the single activation $\theta$ `o` for 400 time-steps:

```python
# number of time-steps
length = 400

# inputs to the network as different sine waves
u0 = np.sin((np.arange(length) * np.pi / 10) + np.pi / 2)
u1 = 0.8*np.sin((np.arange(length) * np.pi / 15) + np.pi / 4)
u2 = 0.5*np.sin((np.arange(length) * np.pi / 20) + np.pi)

# stack the three inputs, so each row is the input for one time-step
u = np.stack([u0, u1, u2], axis=0)

# create two empty arrays to store states and outputs
x = np.zeros(length)
o = np.zeros(length)

# initialise a neuron with three inputs
neuron = single_neuron(3)

# calculate outputs for all time-steps
for i in range(length):
    o[i] = neuron.step(u[:,i])
    x[i] = neuron.x
```

For plotting we can use matplotlib. I prefer to use the integrated LaTex compiler for plots and also like to configure my plots in the beginning using the rc-parameters. We are also going to print the weights to see what was initialised there:

```python
from matplotlib import pyplot as plt
from matplotlib import rc

# Set plot parameters
rc('font', **{'family': 'sans-serif', 'sans-serif': ['Latin Modern Sans']})
params = {'font.size': 16,
          'figure.figsize': [20, 12],
          'text.usetex': True,
          'legend.fontsize': 'x-large',
          'axes.labelsize': 'x-large',
          'axes.titlesize':'x-large',
          'xtick.labelsize':'x-large',
          'ytick.labelsize':'x-large'}
plt.rcParams.update(params)

print(neuron.w)

# plot the stuff
fig, axes = plt.subplots(2, 1)
axes[0].set_title(r"Inputs")
axes[0].plot(u0, color="blue", linewidth=3, label=r"$u_1$")
axes[0].plot(u1, color="red", linewidth=3, label=r"$u_2$")
axes[0].plot(u2, color="black", linewidth=3, label=r"$u_3$")
axes[0].legend()
axes[1].set_title(r"State and Activation")
axes[1].plot(x, color="black", linewidth=3, linestyle="dashed", label=r"$x$")
axes[1].plot(o, color="black", linewidth=3, label=r"$\theta$")
axes[1].legend()
axes[1].set_xlabel(r"time-steps")
```

Here we can see that the state $x$ of the neuron (weighted and summed inputs) looks the same like the output or activation $\theta$ (after $tanh()$) around zero but differs for values closer to $\pm 1$. This is desired saturation effect we wanted to model.

Now let's extend this to multiple neurons and call it a layer. We add another parameter to the initialisation for $n$ outputs. The only thing we need to change are the dimensions of the state `x` in line 5 and the dimension of the weight matrix `w` in line 8. That's it!

```python
class layer:

    def __init__(self, n_inputs: int, n_outputs: int, seed: int = 42):
        np.random.seed(seed)
        self.x = np.zeros([n_outputs, 1])
        self.theta = 0
        self.u = np.zeros([n_inputs, 1])
        self.w = np.random.uniform(low=-1, high=1, size=[n_outputs, n_inputs])

    def activate(self, input_vector):
        self.x = np.dot(self.w, input_vector)
        self.theta = np.tanh(self.x)

    def step(self, input_vector):
        self.activate(input_vector)
        return self.theta
```

To run this layer with 2 neurons, we need to add a second row to our state and output storage arrays in lines 1 and 2:

```python
x = np.zeros([2, length])
o = np.zeros([2, length])
neurons = layer(3, 2)

for i in range(length):
    o[:,i] = neurons.step(u[:,i])
    x[:,i] = neurons.x
```

```python
print(neurons.w)

# plot the stuff
fig, axes = plt.subplots(2, 1)
axes[0].set_title(r"Inputs")
axes[0].plot(u0, color="blue", linewidth=3, label=r"$u_1$")
axes[0].plot(u1, color="red", linewidth=3, label=r"$u_2$")
axes[0].plot(u2, color="black", linewidth=3, label=r"$u_3$")
axes[0].legend()
axes[1].set_title(r"States and Activations")
axes[1].plot(x[0,:], color="black", linewidth=3, linestyle="dashed", label=r"$x_1$")
axes[1].plot(o[0,:], color="black", linewidth=3, label=r"$\theta_1$")
axes[1].plot(x[1,:], color="red", linewidth=3, linestyle="dashed", label=r"$x_2$")
axes[1].plot(o[1,:], color="red", linewidth=3, label=r"$\theta_2$")
axes[1].legend()
axes[1].set_xlabel(r"time-steps")
```

Thanks to the fixed seed, the first row of the weight matrix, as well as the first output and state haven't changed. We just added a second neuron with a different transform of the inputs. Please recall that the states $X$ are just a linear combination of the inputs.

This is the basic model we will use and extend in following posts on modelling and learning. One problem of this model is, that the state of the neuron solely depends on the inputs at a current time-step. In biology neurons are a bit lazier: they do not only respond to the current input but also depend on their previous state. We are going to address this in the next post.

# References

    1. Urry, L.A., Cain M.L., Wassermann, S.A., Minorsky, P.V., & Reece, J.B. (2017). Neurons, synapses, and signaling. In Campbell biology, 11th edition (p. 1065-1082). Pearson. Retrieved from https://www.pearson.com/us/higher-education/product/Urry-Campbell-Biology-11th-Edition/9780134093413.html.

    2. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14 (6), 1569-1572. Retrieved from https://ieeexplore.ieee.org/abstract/document/1257420. doi: 10.1109/TNN.2003.820440

    3. Haykin, S. S. (1999). Neural networks: A comprehensive foundation. Prentice Hall. Retrieved from https://books.google.de/books?id=M5abQgAACAAJ.

    4. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of Physiology, 117 (4), 500-544. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1392413/

```python

```


# ForestDML, ForestDRLearner, OrthoForest and CausalForest: Basic Example

We depict the performance of our `ForestDML`, `ForestDRLearner`, `OrthoForest` and `CausalForest` estimators on the same data generating process as the one used in the tutorial page of the grf package (see https://github.com/grf-labs/grf#usage-examples). This is mostly for qualitative comparison and verification purposes among our implementation of variants of Causal Forests and the implementation in the grf R package.

```python
%load_ext autoreload
%autoreload 2
```

```python
## Ignore warnings
import warnings
warnings.filterwarnings('ignore')
```

```python
# Helper imports
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
%matplotlib inline
```

# DGP

We use the following data generating process (DGP) from [here](https://github.com/grf-labs/grf#usage-examples):

\begin{align}
X \sim& \text{Normal}(0,\, I_{p})\\
T =& \text{Binomial}(1, .4 + .2 \cdot 1\{X[0] > 0\})\\
Y =& (X[0] \cdot 1\{X[0] > 0\}) \cdot T + X[1] + X[2] \cdot 1\{X[2] < 0\} + \epsilon, &\; \epsilon \sim \text{Normal}(0, 1)\\
\end{align}

We use $p=10$ and draw $n=2000$ samples from this DGP.

```python
import numpy as np
import scipy.special
np.random.seed(123)
n = 2000
p = 10
X = np.random.normal(size=(n, p))
true_propensity = lambda x: .4 + .2 * (x[:, 0] > 0)
true_effect = lambda x: (x[:, 0] * (x[:, 0] > 0))
true_conf = lambda x: x[:, 1] + np.clip(x[:, 2], - np.inf, 0)
T = np.random.binomial(1, true_propensity(X))
Y =  true_effect(X) * T + true_conf(X) + np.random.normal(size=(n,))
```

### Cross-Validated Forest Nuisance Models

We use forest based estimators (Gradient Boosted Forests or Random Forests) as nuisance models. For the meta-learner versions of our forest based estimators, we also use a generic forest estimator even as a final model. The hyperparameters of the forest models (e.g. number of estimators, max depth, min leaf size) is chosen via cross validation. We also choose among Gradient or Random Forests via cross validation

```python
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures

class GridSearchCVList:

    def __init__(self, estimator_list, param_grid_list, scoring=None,
                 n_jobs=None, iid='warn', refit=True, cv='warn', verbose=0, pre_dispatch='2*n_jobs',
                 error_score='raise-deprecating', return_train_score=False):
        self._gcv_list = [GridSearchCV(estimator, param_grid, scoring=scoring,
                                       n_jobs=n_jobs, iid=iid, refit=refit, cv=cv, verbose=verbose,
                                       pre_dispatch=pre_dispatch, error_score=error_score,
                                       return_train_score=return_train_score)
                          for estimator, param_grid in zip(estimator_list, param_grid_list)]
        return

    def fit(self, X, y, **fit_params):
        self.best_ind_ = np.argmax([gcv.fit(X, y, **fit_params).best_score_ for gcv in self._gcv_list])
        self.best_estimator_ = self._gcv_list[self.best_ind_].best_estimator_
        self.best_score_ = self._gcv_list[self.best_ind_].best_score_
        self.best_params_ = self._gcv_list[self.best_ind_].best_params_
        return self

    def predict(self, X):
        return self.best_estimator_.predict(X)

    def predict_proba(self, X):
        return self.best_estimator_.predict_proba(X)

from sklearn.linear_model import LassoCV, LogisticRegressionCV
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from econml.sklearn_extensions.linear_model import WeightedLassoCV

def first_stage_reg():
    return GridSearchCVList([RandomForestRegressor(n_estimators=100, random_state=123),
                             GradientBoostingRegressor(random_state=123)],
                             param_grid_list=[{'max_depth': [3, None],
                                               'min_samples_leaf': [10, 50]},
                                              {'n_estimators': [50, 100],
                                               'max_depth': [3],
                                               'min_samples_leaf': [10, 30]}],
                             cv=3,
                             iid=True,
                             scoring='neg_mean_squared_error')

def first_stage_clf():
    return GridSearchCVList([RandomForestClassifier(n_estimators=100, random_state=123),
                             GradientBoostingClassifier(random_state=123)],
                             param_grid_list=[{'max_depth': [3, 5],
                                               'min_samples_leaf': [10, 50]},
                                              {'n_estimators': [50, 100],
                                               'max_depth': [3],
                                               'min_samples_leaf': [10, 30]}],
                             cv=3,
                             iid=True,
                             scoring='neg_mean_squared_error')

def final_stage():
    return GridSearchCVList([RandomForestRegressor(n_estimators=100, random_state=123)],
                             param_grid_list=[{'max_depth': [3, 5],
                                               'min_samples_leaf': [10, 50]}],
                             cv=3,
                             iid=True,
                             scoring='neg_mean_squared_error')
```

# DML Estimators

```python
from econml.dml import ForestDML
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier
from sklearn.ensemble import GradientBoostingRegressor, GradientBoostingClassifier
from sklearn.dummy import DummyRegressor, DummyClassifier

est = ForestDML(model_y=first_stage_reg(),
                model_t=first_stage_clf(),
                discrete_treatment=True,
                n_crossfit_splits=3,
                n_estimators=4000,
                subsample_fr='auto',
                min_samples_leaf=10,
                verbose=0,
                min_weight_fraction_leaf=.005)
est.fit(Y, T, X=X)

```

    <econml.dml.ForestDML at 0x2018c130248>

```python
from econml.dml import NonParamDML
est2 = NonParamDML(model_y=first_stage_reg(),
                   model_t=first_stage_clf(),
                   n_splits=3,
                   discrete_treatment=True,
                   model_final=final_stage())
est2.fit(Y, T, X=X)
```

    <econml.dml.NonParamDML at 0x201f5fa7488>

```python
X_test = np.zeros((100, p))
X_test[:, 0] = np.linspace(-2, 2, 100)
```

```python
pred = est.effect(X_test)
lb, ub = est.effect_interval(X_test, alpha=0.01)
```

```python
pred2 = est2.effect(X_test)
```

```python
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.plot(X_test[:, 0], true_effect(X_test), '--')
plt.plot(X_test[:, 0], pred2, label='nonparamdml')
plt.plot(X_test[:, 0], pred, label='forestdml (causal forest)')
plt.fill_between(X_test[:, 0], lb, ub, alpha=.4, label='honestrf_ci')
plt.legend()
plt.show()
```

### First Stage Learned Models

```python
# Model T
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.title('honestrf')
for mdl in est.models_t:
    plt.plot(X_test[:, 0], mdl.predict_proba(X_test)[:, 1])
plt.plot(X_test[:, 0], true_propensity(X_test), '--', label='truth')
plt.legend()

plt.subplot(1, 2, 2)
plt.title('rf')
for mdl in est2.models_t:
    plt.plot(X_test[:, 0], mdl.predict_proba(X_test)[:, 1])
plt.plot(X_test[:, 0], true_propensity(X_test), '--', label='truth')
plt.legend()
plt.show()
```

```python
# Model Y
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.title('honestrf')
for mdl in est.models_y:
    plt.plot(X_test[:, 0], mdl.predict(X_test))
plt.plot(X_test[:, 0], true_effect(X_test) * true_propensity(X_test) + true_conf(X_test), '--', label='truth')
plt.legend()

plt.subplot(1, 2, 2)
plt.title('rf')
for mdl in est2.models_y:
    plt.plot(X_test[:, 0], mdl.predict(X_test))
plt.plot(X_test[:, 0], true_effect(X_test) * true_propensity(X_test) + true_conf(X_test), '--', label='truth')
plt.legend()
plt.show()
```

### Interpretability of CATE Model of NonParamDML with SHAP

```python
import shap
import pandas as pd
import string
Xdf = pd.DataFrame(X, columns=list(string.ascii_lowercase)[:X.shape[1]])
# explain the model's predictions using SHAP values
explainer = shap.TreeExplainer(est2.model_cate.best_estimator_)
shap_values = explainer.shap_values(Xdf)

# visualize the first prediction's explanation (use matplotlib=True to avoid Javascript)
shap.force_plot(explainer.expected_value, shap_values[0,:], Xdf.iloc[0,:], matplotlib=True)
```

```python
shap.summary_plot(shap_values, Xdf)
```

# DRLearner

```python
from econml.drlearner import ForestDRLearner
from sklearn.dummy import DummyRegressor, DummyClassifier

est = ForestDRLearner(model_regression=first_stage_reg(),
                      model_propensity=first_stage_clf(),
                      n_crossfit_splits=3,
                      n_estimators=4000,
                      subsample_fr='auto',
                      min_samples_leaf=10,
                      verbose=0,
                      min_weight_fraction_leaf=.005)
est.fit(Y, T, X=X)
```

    <econml.drlearner.ForestDRLearner at 0x201906e4d08>

```python
from econml.drlearner import DRLearner
est2 = DRLearner(model_regression=first_stage_reg(),
                 model_propensity=first_stage_clf(),
                 model_final=final_stage(),
                 n_splits=3)
est2.fit(Y, T, X=X)
```

    <econml.drlearner.DRLearner at 0x201906699c8>

```python
X_test = np.zeros((100, p))
X_test[:, 0] = np.linspace(-2, 2, 100)
```

```python
pred = est.effect(X_test)
lb, ub = est.effect_interval(X_test, alpha=0.01)
```

```python
pred2 = est2.effect(X_test)
```

```python
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.plot(X_test[:, 0], true_effect(X_test), '--')
plt.plot(X_test[:, 0], pred2, label='nonparamdml')
plt.plot(X_test[:, 0], pred, label='forestdml (causal forest)')
plt.fill_between(X_test[:, 0], lb, ub, alpha=.4, label='honestrf_ci')
plt.legend()
plt.show()
```

### First stage nuisance models

```python
# Model T
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.title('honestrf')
for mdl in est.models_propensity:
    plt.plot(X_test[:, 0], mdl.predict_proba(X_test)[:, 1])
plt.plot(X_test[:, 0], true_propensity(X_test), '--', label='truth')
plt.legend()

plt.subplot(1, 2, 2)
plt.title('rf')
for mdl in est2.models_propensity:
    plt.plot(X_test[:, 0], mdl.predict_proba(X_test)[:, 1])
plt.plot(X_test[:, 0], true_propensity(X_test), '--', label='truth')
plt.legend()
plt.show()
```

```python
# Model Y
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.title('honestrf')
for mdl in est.models_regression:
    plt.plot(X_test[:, 0], mdl.predict(np.hstack([X_test, np.ones((X_test.shape[0], 1))])))
plt.plot(X_test[:, 0], true_effect(X_test) + true_conf(X_test), '--', label='truth')
plt.legend()

plt.subplot(1, 2, 2)
plt.title('rf')
for mdl in est2.models_regression:
    plt.plot(X_test[:, 0], mdl.predict(np.hstack([X_test, np.ones((X_test.shape[0], 1))])))
plt.plot(X_test[:, 0], true_effect(X_test) + true_conf(X_test), '--', label='truth')
plt.legend()
plt.show()
```

### Interpretability of CATE Model of DRLearner with SHAP

```python
import shap
import pandas as pd
import string
Xdf = pd.DataFrame(X, columns=list(string.ascii_lowercase)[:X.shape[1]])
# explain the model's predictions using SHAP values
explainer = shap.TreeExplainer(est2.model_cate().best_estimator_)
shap_values = explainer.shap_values(Xdf)

# visualize the first prediction's explanation (use matplotlib=True to avoid Javascript)
shap.force_plot(explainer.expected_value, shap_values[0,:], Xdf.iloc[0,:], matplotlib=True)
```

```python
shap.summary_plot(shap_values, Xdf)
```

# OrthoForest and CausalForest

```python
from econml.ortho_forest import DROrthoForest
from sklearn.linear_model import Lasso, LassoCV, LogisticRegression, LogisticRegressionCV
from econml.sklearn_extensions.linear_model import WeightedLassoCV

est3 = DROrthoForest(model_Y=Lasso(alpha=0.01),
                     propensity_model=LogisticRegression(C=1),
                     model_Y_final=WeightedLassoCV(cv=3),
                     propensity_model_final=LogisticRegressionCV(cv=3),
                     n_trees=1000, min_leaf_size=10)
est3.fit(Y, T, X=X)
```

    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:    8.1s
    [Parallel(n_jobs=-1)]: Done 112 tasks      | elapsed:    9.8s
    [Parallel(n_jobs=-1)]: Done 272 tasks      | elapsed:   12.7s
    [Parallel(n_jobs=-1)]: Done 496 tasks      | elapsed:   16.6s
    [Parallel(n_jobs=-1)]: Done 784 tasks      | elapsed:   22.3s
    [Parallel(n_jobs=-1)]: Done 1000 out of 1000 | elapsed:   26.2s finished
    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:    0.3s
    [Parallel(n_jobs=-1)]: Done 208 tasks      | elapsed:    4.0s
    [Parallel(n_jobs=-1)]: Done 528 tasks      | elapsed:    9.8s
    [Parallel(n_jobs=-1)]: Done 976 tasks      | elapsed:   18.0s
    [Parallel(n_jobs=-1)]: Done 1000 out of 1000 | elapsed:   18.4s finished

    <econml.ortho_forest.DROrthoForest at 0x2019061bf48>

```python
pred3 = est3.effect(X_test)
```

    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:   10.7s
    [Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:   15.7s finished

```python
from econml.causal_forest import CausalForest
from sklearn.linear_model import Lasso, LassoCV, LogisticRegression, LogisticRegressionCV
from econml.sklearn_extensions.linear_model import WeightedLassoCV

est4 = CausalForest(model_Y=WeightedLassoCV(cv=3),
                    model_T=LogisticRegressionCV(cv=3),
                    n_trees=1000, min_leaf_size=10,
                    discrete_treatment=True,
                    cv=5)
est4.fit(Y, T, X=X)
```

    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:    8.6s
    [Parallel(n_jobs=-1)]: Done 224 tasks      | elapsed:    9.9s
    [Parallel(n_jobs=-1)]: Done 864 tasks      | elapsed:   13.3s
    [Parallel(n_jobs=-1)]: Done 1000 out of 1000 | elapsed:   14.1s finished
    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:    0.1s
    [Parallel(n_jobs=-1)]: Done 368 tasks      | elapsed:    2.5s
    [Parallel(n_jobs=-1)]: Done 984 tasks      | elapsed:    6.5s
    [Parallel(n_jobs=-1)]: Done 1000 out of 1000 | elapsed:    6.6s finished

    <econml.causal_forest.CausalForest at 0x20196a6e188>

```python
pred4 = est4.effect(X_test)
```

    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.
    [Parallel(n_jobs=-1)]: Done  16 tasks      | elapsed:    9.7s
    [Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:   11.7s finished

```python
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.plot(X_test[:, 0], true_effect(X_test), '--')
plt.plot(X_test[:, 0], pred, label='forestdr')
plt.plot(X_test[:, 0], pred2, label='nonparamdr')
plt.plot(X_test[:, 0], pred3, label='discreteorf')
plt.plot(X_test[:, 0], pred4, label='causalforest')
plt.fill_between(X_test[:, 0], lb, ub, alpha=.4, label='forest_dr_ci')
plt.legend()
plt.show()
```

```python

```


# Hamiltonian Tomography

## Contents
* [Introduction](#introduction)
* [Fitting Functions](#fitting-functions)
* [Using the Pulse Simulator](#using-the-pulse-simulator)
* [Run Experiment on Backend Model Simulation](#run-experiment-model)
* [Fitting the Simulated Results](#fitting-the-simulated-results)
* [Measure ZI (Stark Shift) via CR Ramsey Experiment](#measure-zi-via-ramsey)
* [References](#references)

## Introduction

The cross resonance gate for entangling qubits was introduced in [this section](https://qiskit.org/textbook/ch-quantum-hardware/cQED-JC-SW.html#6.-The-Cross-Resonance-Entangling-Gate-) of the Qiskit Textbook, where it is assumed the transmon is a qubit and the Schrieffer-Wolff transformation is applied to yield an effective Hamiltonian

$$
\tilde{H}_{\rm eff}^{\rm CR} = - \frac{\Delta_{12}}{2}\sigma_1^z
+ \frac{\Omega(t)}{2} \left(\sigma_2^x - \frac{J}{2\Delta_{12}} \sigma_1^z \sigma_2^x \right),
$$

where $\Delta_{12} = \tilde{\omega}_1-\tilde{\omega}_2$ is the difference between the dressed frequencies of the qubits, $\Omega$ is the cross resonance drive strength, and $J$ is the qubit-qubit coupling. We will use a common simplified notation for these interaction terms where the Pauli matrix is represented by a capital letter (with a hat to denote that it is an operator) and the qubit is represented by its position in a string, so for example we wish to isolate the $\hat{Z}\hat{X} = \hat{Z} \otimes \hat{X} = \sigma_1^z \otimes \sigma_2^x = \sigma_1^z \sigma_2^x$ interaction that is used to build the controlled-NOT gate from the $\hat{Z}\hat{I} = \sigma_1^z \otimes \sigma_2^0$ and $\hat{I}\hat{X} = \sigma_1^0 \otimes \sigma_2^x$ terms. Here the matrix $\sigma_i^0$ represents the identity matrix on qubit $i$.

In addition to understanding these extra terms, since the [transmon](https://qiskit.org/textbook/ch-quantum-hardware/transmon-physics.html#4.-The-Quantized-Transmon-) has higher energy levels and actual experiments may have other interactions, due to crosstalk for example, when applying an entangling operation, it is not always obvious which Pauli rotations will be generated. Here we assume a cross resonance Hamiltonian of the [following form](https://arxiv.org/abs/1603.04821):

$$
\hat{H} = \frac{\hat{Z} \otimes \hat{A}}{2} + \frac{\hat{I} \otimes \hat{B}}{2} =
a_{x} \hat{Z}\hat{X} + a_{y} \hat{Z}\hat{Y} + a_{z} \hat{Z}\hat{Z} + b_{x}  \hat{I}\hat{X} + b_{y} \hat{I}\hat{Y} + b_{z} \hat{I}\hat{Z}
$$

where we will omit the Kronecker product symbol $\otimes$ for succinctness.  We refer to the first Pauli operator acting on the control qubit and second operators acting on the target qubits, as in the effective Hamiltonian above.  While the form of the cross resonance Hamiltonian is known, the individual coefficients $a_{\mu}, b_{\nu}$ are not known. Note that these coefficients are also referred to as the strengths of the interaction they correspond to, i.e. $a_x$ is the $ZX$ interaction rate, etc. We must then find a way of extracting the coefficients from measurements made on the system after the cross resonance pulse is applied for different durations. Before we proceed, it should be noted that cross resonance operation also generates a $\hat{Z}\hat{I}$ interaction arising from a Stark shift (off-resonant drive that dressed the qubit frequency). This term can be extracted by preforming a [Ramsey experiment](https://qiskit.org/textbook/ch-quantum-hardware/calibrating-qubits-pulse.html#4.1-Measuring-the-qubit-Frequency-Precisely-using-a-Ramsey-Experiment-) on the control qubit. We will discuss this Ramsey procedure later on, so for now let us focus on the Hamiltonian that we wrote down.

The coefficients $a_{\mu}, b_{\nu}$ (interaction rates) will be extracted by taking six different measurements as a function of the duration of the pulse.  The six measurements are of the expectation value of each Pauli term on the target qubit with the control qubit either in the ground or excited state. In the next section we will show how these measurements give us information about the coefficients.

## Fitting Functions

In order to extract the coefficients $a_{\mu}, b_{\nu}$, we need to know what function we expect to fit to the measurement data.  The data we will be looking at will be the expectation value of Pauli operators as a function of pulse duration.  In the Heisenberg picture of quantum mechanics, the evolution of the expectation value of an operator can be given as

$$\langle \hat{O}(t) \rangle = \langle e^{i\hat{H}t} \hat{O} e^{-i\hat{H}t} \rangle$$

Let $dt$ be an infinitesimally small time increment.  Then we have

$$\langle \hat{O}(t+dt) \rangle = \langle (1+i\hat{H} dt)\hat{O}(t)(1-i\hat{H} dt) \rangle =
\langle \hat{O}(t) \rangle +i dt \langle \left[\hat{H},\hat{O}\right] \rangle
\Longrightarrow \frac{d\langle \hat{O} \rangle}{dt} = i \left[\hat{H},\hat{O}\right]$$

to first order in $dt$. We can evaluate the commutator for each of the Pauli operators:

\begin{equation}
\begin{split}
&\left[\hat{H}, \hat{I}\hat{X}\right] = 2 i \left(a_{y} \hat{Z}\hat{Z} - a_{z} \hat{Z}\hat{Y} + b_{y} \hat{I}\hat{Z} - b_{z} \hat{I}\hat{Y}\right) \\
&\left[\hat{H},\hat{I}\hat{Y}\right] = 2 i \left(-a_{x} \hat{Z}\hat{Z} + a_{z} \hat{Z}\hat{X} - b_{x} \hat{I}\hat{Z} + b_{z} \hat{I}\hat{X}\right)\\
&\left[\hat{H}, \hat{I}\hat{Z}\right] = 2 i \left(a_{x} \hat{Z}\hat{Y} - a_{y} \hat{Z}\hat{X} + b_{x} \hat{I}\hat{Y} - b_{y} \hat{I}\hat{X}\right)
\end{split}
\end{equation}

If we let $n$ be the expectation value of the Pauli $\hat{Z}$ operator on the control qubit, then we can write down these commutators in terms of the expectation values of the target qubit:

\begin{equation}
\begin{split}
&i\langle\left[\hat{H},\hat{I}\hat{X} \right]\rangle_{\rm control} = 2 \left(n a_{z} + b_{z}\right)\langle\hat{Y}\rangle - 2 \left(n a_{y} + b_{y}\right)\langle\hat{Z}\rangle
\\
&i\langle\left[\hat{H}, \hat{I}\hat{Y}\right]\rangle_{\rm control} = 2\left(n a_{x} + b_{x}\right) \langle\hat{Z}\rangle-2  \left(n a_{z} + b_{z}\right) \langle\hat{X}\rangle
\\
&i\langle\left[\hat{H}, \hat{I}\hat{Z}\right]\rangle_{\rm control} = 2  \left(n a_{y} + b_{y}\right) \langle\hat{X}\rangle - 2 \left(n a_{x} + b_{x}\right) \langle\hat{Y}\rangle
\end{split}
\end{equation}

where the expectation values on the right hand side are understood to be those of the target qubit, which will also be the case of the following discussion. Let us define $\vec{r}_n = \{\langle\hat{X}\rangle, \langle\hat{Y}\rangle, \langle\hat{Z}\rangle\}_n$ then we can use these commutators to write down a matrix equation for the time dependence of $\vec{r}$ depending on the Pauli-$Z$ value $n$ of the state of the control qubit. Then putting the above equations together

$$ \frac{d}{dt} \begin{bmatrix}
\langle \hat{X} \rangle \\ \langle \hat{Y} \rangle \\ \langle \hat{Z} \rangle \end{bmatrix} =
2 \begin{bmatrix} 0 & na_z + b_z & -n a_y - b_y \\
-na_z - b_z & 0 & n a_x + b_x \\
na_y + b_y & -na_x - b_x & 0 \end{bmatrix} \begin{bmatrix}
\langle \hat{X} \rangle \\ \langle \hat{Y} \rangle \\ \langle \hat{Z} \rangle \end{bmatrix}
$$

or more compactly,

$$\frac{d\vec{r}_n(t)}{dt} = G_n \vec{r}_n(t),$$

where

$$
G_n = 2 \begin{bmatrix} 0 & na_z + b_z & -n a_y - b_y \\
-na_z - b_z & 0 & n a_x + b_x \\
na_y + b_y & -na_x - b_x & 0 \end{bmatrix} \equiv \begin{bmatrix}0 & \Delta^n & -\Omega_y^n\\-\Delta^n & 0 & \Omega_x^n \\ \Omega_y^n & -\Omega_x^n & 0\end{bmatrix}.
$$

Since $G_n$ is time independent we can easily integrate the differential equation with the initial state corresponding to $t=0$, yielding

$$\vec{r}_n(t) = e^{G_n t} \vec{r}_n(0).$$

We can find the matrix exponential, $e^{G_n t}$, by finding the eigenvalues and eigenvectors of $G_n$.  The eigenvalues of $G_n$ are

$$
\vec{g}_{n} = \{0, -i\sqrt{\Delta^2+\Omega_x^2+\Omega_y^2} , i\sqrt{\Delta^2+\Omega_x^2+\Omega_y^2}\}_n,
$$

where for notational simplicity, the subscript $n$ denotes the appropriate values of $\Delta, \Omega_x, \Omega_y,$ given the state of the control qubit. We will not write down the eigenvectors because they are too cumbersome but it is straightforward to find them.  Let $U$ be the transformation into the eigenbasis and let $\hat{D}_n$ be the diagonal matrix of the eigenvalues.  Then we can rewrite the time dependence of $\vec{r}_n(t)$ as

$$\vec{r}_n(t) = U^{\dagger} e^{\hat{D}_n t} U\vec{r}_n(0).$$

Setting ${\vec{r}_n(0) = \{0,0,1}\}$ (which corresponds the target qubit starting in the $|0\rangle$ state) we have that

\begin{equation}
\begin{split}
&\langle \hat{X}(t) \rangle_n = \frac{1}{\Omega^2}\left(-\Delta \Omega_x + \Delta\Omega_x\cos(\Omega t) + \Omega \Omega_y \sin(\Omega t)\right)
\\
&\langle \hat{Y}(t) \rangle_n = \frac{1}{\Omega^2}\left(\Delta \Omega_y - \Delta\Omega_y\cos(\Omega t) - \Omega \Omega_x \sin(\Omega t)\right)
\\
&\langle \hat{Z}(t) \rangle_n = \frac{1}{\Omega^2}\left(\Delta^2 + \left(\Omega_x^2+\Omega_y^2\right)\cos(\Omega t) \right)
\end{split}
\end{equation}

where $\Omega = \sqrt{\Delta^2+\Omega_x^2+\Omega_y^2}$ for each control qubit preparation $n$. In the subsequent sections, we will often drop the hat (^) on the operators.

## Using the Pulse Simulator

We will simulate on a model of a real device using the Pulse Simulator. First, load necessary libraries.

### Warning:
This chapter uses `PulseSimulator` backend in Qiskit Aer. There is a bug in the latest stable version regarding the measurement, and thus this installs `stable/0.7`. See [Qiskit/qiskit-aer#1257](https://github.com/Qiskit/qiskit-aer/issues/1257) for details.

```python
!pip install qiskit-aer==0.7
```

```python
from qiskit import circuit, pulse, transpile, QuantumCircuit, schedule
from qiskit.pulse import Play
from qiskit.pulse.channels import ControlChannel, DriveChannel
from qiskit.pulse.library import drag, GaussianSquare
from qiskit.test.mock import FakeAthens
import numpy as np
import matplotlib.pyplot as plt

backend = FakeAthens()
```

Next, save the fake backend configuration and the sampling time $dt$. We will save the Hamiltonian parameters here for building a Duffing oscillator model later.

```python
backend_config = backend.configuration()
ham_params = backend_config.hamiltonian['vars']
dt = backend_config.dt
print(f"Sampling time: {dt * 1e9} ns")
```

    Sampling time: 0.2222222222222222 ns

First we grab all the pulse-level gate calibrations, then we get the CNOT operation between two qubits. We already [know](https://arxiv.org/abs/2004.06755) from experience with this particular backend that the CNOT operation contains a CR pulse implemented as a GaussianSquare on a particular channel, ControlChannel(1) . We use this knowledge we have about the form of that instruction to extract it from the overall CNOT pulse.Schedule by using the filter method.

```python
qc = 1
qt = 0

backend_defaults = backend.defaults()
inst_sched_map = backend_defaults.instruction_schedule_map
cr_pulse = inst_sched_map.get('cx', (0, 1)).filter(channels = [ControlChannel(1)], instruction_types=[Play]).instructions[0][1].pulse

```

Cross resonance pulses are of type [`GaussianSquare`](https://qiskit.org/documentation/stubs/qiskit.pulse.library.GaussianSquare.html), a square pulse with a Gaussian rise and fall. Currently, the waveform samples are returned from the `instruction_schedule_map`, so we must elucidate the `GaussianSquare` parameters to easily build our own cross resonance pulses. We can declare parameter as below:

```python
pulse_amp = cr_pulse.parameters['amp']  #import amplitude from cr_pulse to match the phase

#declare pulse parameters and build GaussianSquare pulse
cr_params = {}
cr_params['duration'] = 688
cr_params['amp'] = pulse_amp
cr_params['sigma'] = 64
cr_params['width'] = 432
my_cr_pulse = GaussianSquare(**cr_params)
```

Let's build a test schedule to visualize the default cross resonance pulse against the one we just constructed.

```python
fig, axes = plt.subplots(nrows=2, sharex=True, figsize=(8, 7))

with pulse.build(backend, name="Default CR") as cr_sched_default:
        pulse.play(cr_pulse, ControlChannel(1))
cr_sched_default.draw(backend = backend, axis=axes[0])
axes[0].set_xlabel("")

with pulse.build(backend, name="Custom CR") as cr_sched_custom:
    pulse.play(my_cr_pulse, ControlChannel(1))
cr_sched_custom.draw(backend = backend, axis=axes[1])
```

Pretty good! This will be close enough for the Hamiltonian tomography experiment. Now, this pulse nominally executes $ZX(\theta=\pi/4)$ corresponding to the [RZXGate](https://qiskit.org/documentation/stubs/qiskit.circuit.library.RZXGate.html) because the cross resonance pulse is echoed: the first half will be a positive rotation dependent on the state of the control qubit, followed by an "echo pulse" that flips the control qubit, followed by a negative rotation dependent on the new state of the control qubit. This turns out to be equivalent to a $ZX(\theta=\pi/2)$, but we are just dealing with the first part of the pulse so that we can observe the full effect of the cross resonance interaction. We keep this in mind because this particular duration of cross resonance pulse will only take us an angle of $\theta=\pi/4$ around the Bloch sphere, and for the Hamiltonian tomography experiment we wish to traverse the Bloch sphere several times.

```python
def build_cr_scheds(qc: int, qt: int, cr_times, phase = 0.0, ZI_MHz = 0.0) -> np.array:
    """Build an array of cross resonance schedules for the Hamiltonian tomography experiment.

    Args:
      qc: control qubit index
      qt: target qubit index
      cr_times: array of widths of the cross resonance pulses
      phase: phase offset of the cross resonance pulse (rad)
      ZI_MHz: ZI interaction rate (in MHz) to correct for with frame change
    """
    tomo_circs = []

    cr_gate = circuit.Gate('cr', num_qubits = 2, params = [])
    cr_risefall = (cr_params['duration'] - cr_params['width']) / (2 * cr_params['sigma'])

    for width in cr_times:
        tomo_cr_params = cr_params.copy()
        tomo_cr_params['duration'] = int(width + 2 * cr_risefall * cr_params['sigma'])
        tomo_cr_params['width'] = width
        framechange = 2 * np.pi * int(width) * dt * ZI_MHz * 1e6

        with pulse.build(backend) as cr_sched:
            uchan = pulse.control_channels(qc, qt)[0]
            with pulse.phase_offset(phase, uchan):
                pulse.play(GaussianSquare(**tomo_cr_params), uchan)

        for basis in ['X', 'Y', 'Z']:
            for control in ['0', '1']:
                tomo_circ = circuit.QuantumCircuit(5)  #use all qubits to avoid error
                if control == '1':
                    tomo_circ.x(qc)# flip control from |0> to |1>
                tomo_circ.append(cr_gate, [qc, qt])  #apply custom cr_gate
                tomo_circ.rz(-framechange, qc)  #apply frame change on the qc
                tomo_circ.barrier(qc, qt)
                if basis == 'X':
                    tomo_circ.h(qt)
                elif basis == 'Y':
                    tomo_circ.sdg(qt)
                    tomo_circ.h(qt)
                tomo_circ.measure_all()
                tomo_circ.add_calibration(gate=cr_gate, qubits=[qc, qt], schedule=cr_sched)
                tomo_circs.append(tomo_circ)

    tomo_circs_transpiled = transpile(tomo_circs, backend, optimization_level = 1)
    return schedule(tomo_circs_transpiled, backend)
```

```python
# remember samples must be in multiples of 16
cr_times = 16*np.linspace(0, 500, 21)
cr_scheds = build_cr_scheds(qc, qt, cr_times)
cr_scheds[-1].draw(backend = backend)
```

Note how the final schedule consists of the control in the $|1\rangle$ state due to the $\pi$-pulse on `d1` channel before the cross resonance pulse and this is measured in the $Z$-basis.

## Run Experiment on Backend Model Simulation

We will construct a [Duffing oscillator](https://qiskit.org/textbook/ch-quantum-hardware/transmon-physics.html#4.-The-Quantized-Transmon-) model based on the Hamiltonian model of `ibmq_athens` by using `PulseSystemModel`. Then we collect the relevant Hamiltonian from the backend configuration.

```python
from qiskit import assemble
from qiskit.providers.aer import PulseSimulator
from qiskit.providers.aer.pulse import PulseSystemModel

backend_model= PulseSystemModel.from_backend(backend)
backend_sim = PulseSimulator()
qubit_lo_freq = backend_model.hamiltonian.get_qubit_lo_from_drift()
```

```python
def run_pulse(sched):
    """Runs the scheduled experiment on the simulated backend.

    Args:
      sched: pulse schedule to run
    """
    # assemble the qobj
    test_qobj = assemble(sched,
                        backend = backend_sim,
                        qubit_lo_freq = qubit_lo_freq,
                        meas_level = 1,
                        meas_return = 'avg',
                        shots = 2048)

    # run simulation
    sim_result = backend_sim.run(test_qobj, system_model=backend_model).result()
    return sim_result.get_memory(0)

def run_ham_tomo(cr_times, cr_scheds):
    """Run Hamiltonian tomography experiment and return results.

    Args:
      cr_times: widths of cross resonance pulses
      cr_scheds: array of pulse schedules for Ham Tomo experiment
    """
    # expectation values of target conditioned on control
    avg_t_c = np.zeros((6, len(cr_times)), dtype = complex)

    # sanity check: expectation values of control conditioned on control
    avg_c_c = np.zeros((6, len(cr_times)), dtype = complex)

    for ii in range(len(cr_scheds)):
        result = run_pulse(cr_scheds[ii])
        avg_t_c[ii % 6, ii // 6] = 1 - 2 * result[qt]
        avg_c_c[ii % 6, ii // 6] = result[qc]
        print(ii + 1, "/",len(cr_scheds), end = "\r")

    return np.real(avg_t_c), np.real(avg_c_c)
```

#### **Warning!** The Pulse Simulator is computationally intensive and each experiment consisting of runs of 21 schedules with 6 different options and 2048 shots may take tens of minutes up to hours depending on CPU performance. The schedules with longer cross resonance pulses are more computationally intensive than those with shorter ones.

```python
avg_t_c, avg_c_c = run_ham_tomo(cr_times, cr_scheds)
```

    126 / 126

## Fitting the Simulated Results

Using the `scipy` package, the fitting functions below will fit the Hamiltonian tomography data, Pauli expectations of the target qubit $\langle X(t) \rangle, \langle Y(t) \rangle, \langle Z(t) \rangle$, for the control prepared in either the ground or excited state. Note that we must use a trick to concatenate all the data into a single array by `tile`ing the time array and `vstack`ing the data so we can use the `curve_fit` function.

```python
from scipy.optimize import curve_fit

def get_omega(eDelta, eOmega_x, eOmega_y):
    """Return \Omega from parameter arguments."""
    eOmega = np.sqrt(eDelta**2 + eOmega_x**2 + eOmega_y**2)
    return eOmega

def avg_X(t, eDelta, eOmega_x, eOmega_y):
    """Return average X Pauli measurement vs time t"""
    eOmega = get_omega(eDelta, eOmega_x, eOmega_y)
    eXt = (-eDelta*eOmega_x + eDelta*eOmega_x*np.cos(eOmega*t) + \
           eOmega*eOmega_y*np.sin(eOmega*t)) / eOmega**2
    return eXt

def avg_Y(t, eDelta, eOmega_x, eOmega_y):
    """Return average Y Pauli measurement vs time t"""
    eOmega = get_omega(eDelta, eOmega_x, eOmega_y)
    eYt = (eDelta*eOmega_y - eDelta*eOmega_y*np.cos(eOmega*t) - \
           eOmega*eOmega_x*np.sin(eOmega*t)) / eOmega**2
    return eYt

def avg_Z(t, eDelta, eOmega_x, eOmega_y):
    """Return average Z Pauli measurement vs time t"""
    eOmega = get_omega(eDelta, eOmega_x, eOmega_y)
    eZt = (eDelta**2 + (eOmega_x**2 + eOmega_y**2)*np.cos(eOmega*t)) / eOmega**2
    return eZt

def rt_evol(ts, eDelta, eOmega_x, eOmega_y):
    """Stack average X,Y,Z Pauli measurements vertically."""
    return np.vstack([avg_X(ts, eDelta, eOmega_x, eOmega_y), \
                     avg_Y(ts, eDelta, eOmega_x, eOmega_y), \
                     avg_Z(ts, eDelta, eOmega_x, eOmega_y)])

def rt_flat(ts, eDelta, eOmega_x, eOmega_y):
    """Flatten X,Y,Z Pauli measurement data into 1D array."""
    return rt_evol(ts[0:len(ts)//3], eDelta, eOmega_x, eOmega_y).flatten()

def fit_rt_evol(ts, eXt, eYt, eZt, p0):
    """Use curve_fit to determine fit parameters of X,Y,Z Pauli measurements together."""
    rt_vec = np.asarray([eXt, eYt, eZt])

    return curve_fit(rt_flat, np.tile(ts, 3), rt_vec.flatten(), p0=p0, method='trf')
```

### Plotting Functions

The above fits provide the parameters $\Omega^i_x, \Omega^i_y$, and $\Delta^i$ for the control qubit prepared in states $i = |0\rangle, |1\rangle$ (corresponding to $n=\pm 1$ in the equations above). The interaction rates (coefficients $a_\mu, b_\nu$ of the operators) are then determined as

$$
IX = \frac{1}{2} \left( \Omega^0_x + \Omega^1_x\right) \qquad ZX = \frac{1}{2} \left( \Omega^0_x - \Omega^1_x\right) \\
IY = \frac{1}{2} \left( \Omega^0_y + \Omega^1_y\right) \qquad ZY = \frac{1}{2} \left( \Omega^0_y - \Omega^1_y\right) \\
IZ = \frac{1}{2} \left( \Delta^0 + \Delta^1\right) \qquad ZZ = \frac{1}{2} \left( \Delta^0 - \Delta^1\right)
$$

```python
def get_interation_rates_MHz(ground_fit, excited_fit):
    """Determine interaction rates from fits to ground and excited control qubit data."""
    Delta0 = (ground_fit[0]/dt)/1e6
    Omega0_x = (ground_fit[1]/dt)/1e6
    Omega0_y = (ground_fit[2]/dt)/1e6
    Delta1 = (excited_fit[0]/dt)/1e6
    Omega1_x = (excited_fit[1]/dt)/1e6
    Omega1_y = (excited_fit[2]/dt)/1e6

    IX = 0.5*(Omega0_x + Omega1_x)
    IY = 0.5*(Omega0_y + Omega1_y)
    IZ = 0.5*(Delta0 + Delta1)
    ZX = 0.5*(Omega0_x - Omega1_x)
    ZY = 0.5*(Omega0_y - Omega1_y)
    ZZ = 0.5*(Delta0 - Delta1)

    return [[IX, IY, IZ], [ZX, ZY, ZZ]]

def plot_cr_ham_tomo(cr_times, avg_t_c, avg_c_c, ground_fit, excited_fit):
    """Plot Hamiltonian tomography data and curve fits with interaction rates."""
    coeffs = get_interation_rates_MHz(ground_fit, excited_fit)
    fig, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(15,15), sharey = True)
    ax1.scatter(cr_times, avg_t_c[0,:].real, lw=3.0, color='blue', label='ctrl in |0>')
    ax1.plot(cr_times, avg_X(cr_times, *ground_fit), lw=3.0, color='blue')
    ax1.scatter(cr_times, avg_t_c[1,:].real, lw=3.0, color='red', label='ctrl in |1>')
    ax1.plot(cr_times, avg_X(cr_times, *excited_fit), lw=3.0, color='red')
    ax1.set_ylabel('<X(t)>', fontsize=20)
    ax1.set_title('Pauli Expectation Value', fontsize=20)
    ax1.legend(loc=4, fontsize=14)
    ax2.scatter(cr_times, avg_t_c[2,:].real, lw=3.0, color='blue', label='ctrl in |0>')
    ax2.plot(cr_times, avg_Y(cr_times, *ground_fit), lw=3.0, color='blue')
    ax2.scatter(cr_times, avg_t_c[3,:].real, lw=3.0, color='red', label='ctrl in |1>')
    ax2.plot(cr_times, avg_Y(cr_times, *excited_fit), lw=3.0, color='red')
    ax2.set_title('IX = %.3f MHz   IY = %.3f MHz   IZ = %.3f MHz' % \
              (coeffs[0][0], coeffs[0][1], coeffs[0][2]), fontsize=20)
    ax2.set_ylabel('<Y(t)>', fontsize=20)
    ax2.legend(loc=4, fontsize=14)
    ax3.scatter(cr_times, avg_t_c[4,:].real, lw=3.0, color='blue', label='ctrl in |0>')
    ax3.plot(cr_times, avg_Z(cr_times, *ground_fit), lw=3.0, color='blue')
    ax3.scatter(cr_times, avg_t_c[5,:].real, lw=3.0, color='red', label='ctrl in |1>')
    ax3.plot(cr_times, avg_Z(cr_times, *excited_fit), lw=3.0, color='red')
    ax3.set_title('ZX = %.3f MHz   ZY = %.3f MHz   ZZ = %.3f MHz' % \
              (coeffs[1][0], coeffs[1][1], coeffs[1][2]), fontsize=20)
    ax3.set_ylabel('<Z(t)>', fontsize=20)
    ax3.set_xlabel('time (dt)', fontsize=20)
    ax3.legend(loc=4, fontsize=14)
```

### Fit and Plot

```python
ground_fit,_ = fit_rt_evol(cr_times, avg_t_c[0,:], avg_t_c[2,:], avg_t_c[4,:], p0=[0.0002, 0.0002, 0.0005])
excited_fit,_ = fit_rt_evol(cr_times, avg_t_c[1,:], avg_t_c[3,:], avg_t_c[5,:], p0=[0.0002, 0.001, 0.001])

plot_cr_ham_tomo(cr_times, avg_t_c, avg_c_c, ground_fit, excited_fit)
```

Note here that the magnitude of the $ZY$ interaction is a lot larger than the desired $ZX$ interaction. This is because the cross resonance pulse is out of phase with the single-qubit drive $IX$ on the target qubit (not the $IX$ here induced by the cross resonance pulse). We can determine this from the interaction rates and shift the phase of the cross resonance pulse in the next Hamiltonian tomography experiment.

```python
coeffs = get_interation_rates_MHz(ground_fit, excited_fit)
ZX_rate = coeffs[1][0]
ZY_rate = coeffs[1][1]
phase = -np.arctan2(ZY_rate, ZX_rate)
print('Phase from ZY/ZX ratio is '+str(phase))
```

    Phase from ZY/ZX ratio is -2.088132525402349

```python
cr_scheds = build_cr_scheds(qc, qt, cr_times, phase=phase)
avg_t_c_mod, avg_c_c_mod = run_ham_tomo(cr_times, cr_scheds)
```

    126 / 126

```python
ground_fit_mod,_ = fit_rt_evol(cr_times, avg_t_c_mod[0,:], avg_t_c_mod[2,:], avg_t_c_mod[4,:], p0=[0, 0, 0.0001])
excited_fit_mod,_ = fit_rt_evol(cr_times, avg_t_c_mod[1,:], avg_t_c_mod[3,:], avg_t_c_mod[5,:], p0=[0.001, 0.001, 0.0000001])

plot_cr_ham_tomo(cr_times, avg_t_c_mod, avg_c_c_mod, ground_fit_mod, excited_fit_mod)
```

Now we can see that the bulk of the cross resonance pulse provides the $ZX$-interaction that we can use to entangle qubits. Note that the scale of the Pauli expectation graph of $\langle X(t) \rangle$ differs from the other graphs.

## Measure ZI (Stark Shift) via CR Ramsey Experiment

Here we measure the $ZI$ interaction term via a [Ramsey experiment](https://qiskit.org/textbook/ch-quantum-hardware/calibrating-qubits-pulse.html#4.1-Measuring-the-qubit-Frequency-Precisely-using-a-Ramsey-Experiment-), recalling that the resulting oscillation are a result of the difference between the qubit and drive frequency. Since the frequency (Stark) shift and $ZI$ interaction are equivalent because a frequency shift induces a $Z$-rotation on the control qubit, we can measure this shift and compensate for it with a [frame change](https://arxiv.org/abs/1612.00858).

```python
def build_cr_ramsey_scheds(qc: int, qt: int, cr_times, phase=0.0, ZI_MHz=0.0) -> np.array:
    """Build array of pulse schedules for CR Ramsey experiment.

    Args:
      qc: control qubit index
      qt: target qubit index
      cr_times: width of cross resonance pulses (in dt)
      phase: phase offset of cross resonance pulse (rad)
      ZI_MHz: Z-rotation rate of control qubit (in MHz) compensated in software by frame change
    """
    scheds = []

    cr_gate = circuit.Gate('cr', num_qubits=2, params=[])
    cr_risefall = (cr_params['duration'] - cr_params['width']) / (2 * cr_params['sigma'])

    for width in cr_times:
        ramsey_cr_params = cr_params.copy()
        ramsey_cr_params['duration'] = int(width + 2 * cr_risefall * cr_params['sigma'])
        ramsey_cr_params['width'] = width
        framechange = 2 * np.pi * int(width) * dt * ZI_MHz * 1e6
        with pulse.build(backend) as cr_sched:
            uchan = pulse.control_channels(qc, qt)[0]
            with pulse.phase_offset(phase, uchan):
                pulse.play(GaussianSquare(**ramsey_cr_params), uchan)
        ramsey_circ = circuit.QuantumCircuit(5)

        ramsey_circ.sx(qc)

        ramsey_circ.append(cr_gate, [qc, qt])
        ramsey_circ.rz(-framechange, qc)

        ramsey_circ.sx(qc)

        ramsey_circ.measure_all()
        ramsey_circ.add_calibration(gate=cr_gate, qubits=[qc, qt], schedule=cr_sched)
        ramsey_circ_transpiled = transpile(ramsey_circ, backend, optimization_level=1)
        sched = schedule(ramsey_circ_transpiled, backend)
        scheds.append(sched)

    return scheds
```

```python
cr_ramsey_times = 16*np.linspace(0, 100, 21)
cr_ramsey_scheds = build_cr_ramsey_scheds(qc, qt, cr_ramsey_times)
cr_ramsey_scheds[-1].draw(backend = backend)
```

```python
cr_ramsey_result = []
count = 0
for sched in cr_ramsey_scheds:
    results = run_pulse(sched)
    cr_ramsey_result.append(np.real(1 - 2 * results[qc]))
    count += 1
    print(count, "/", len(cr_ramsey_scheds), end = "\r")
```

    21 / 21

### Fitting Functions for the CR Ramsey Experiment

We will fit the results to a decaying sinusoid, where the frequency of oscillation is the frequency offset. We will also need to take care of the relation between the control and target qubit frequencies, because that will effect whether the control qubit Stark shift is higher or lower in frequency.

```python
def decay_sin(t, f, a, phi, tau, offset):
    """Fit function for exponentially-decaying sinusoid."""
    return a * np.exp(-t / tau) * np.sin(2 * np.pi * f * t - phi) + offset

def fit_decay_sin(ts, values, p0):
    """Perform fit of decaying sinusoid."""
    return curve_fit(decay_sin, ts, values, p0=p0)
```

```python
def plot_cr_ramsey(cr_ramsey_times, cr_ramsey_result, ramsey_fit):
    """Plot CR Ramsey experiment and fit with ZI interaction rate."""
    fig, ax = plt.subplots(1, 1, figsize=(15, 5))
    ax.scatter(cr_ramsey_times, cr_ramsey_result, lw = 3.0, color = 'red')
    ax.plot(cr_ramsey_times, decay_sin(cr_ramsey_times, *ramsey_fit), lw = 3.0, color = 'red')
    ax.set_ylabel('<Z(t)>', fontsize = 20)
    ax.set_title('CR Ramsey Rate (ZI = %.3f MHz)' % ((ramsey_fit[0] / dt) / 1e6), fontsize = 20)
    ax.set_xlabel('time (dt)', fontsize = 20)
```

```python
ramsey_fit,_ = fit_decay_sin(cr_ramsey_times, cr_ramsey_result, p0 = [0.0025, 1, -np.pi / 2, 300, 0.5])
plot_cr_ramsey(cr_ramsey_times, cr_ramsey_result, ramsey_fit)
```

```python
oscillator_freqs = [] # qubit transition frequencies
SF = 1 / (2 * np.pi) # scale factor to convert from angular frequency to Hz

for key in ham_params:
    if 'wq' in key:
        oscillator_freqs.append(ham_params[key] * SF)

ZI_rate = np.sign(oscillator_freqs[qc] - oscillator_freqs[qt]) * (ramsey_fit[0] / dt) / 1e6
print('Shift frame according to ZI rate of %.3f MHz' % ZI_rate)
```

    Shift frame according to ZI rate of 9.578 MHz

Now we will rebuild the Ramsey schedule to compensate for the Stark shift and rerun the experiment.

```python
# run simulation to longer times
cr_ramsey_times = 16 * np.linspace(0, 250, 21)
cr_ramsey_scheds = build_cr_ramsey_scheds(1, 0, cr_ramsey_times, phase = 0, ZI_MHz = ZI_rate)

cr_ramsey_result = []
count = 0

for sched in cr_ramsey_scheds:
    result = run_pulse(sched)
    cr_ramsey_result.append(np.real(1 - 2 * result[qc]))
    count += 1
    print(count, "/", len(cr_ramsey_scheds), end = "\r")
```

    21 / 21

```python
ramsey_fit,_ = fit_decay_sin(cr_ramsey_times, cr_ramsey_result, p0 = [0.00001, 0.1, 0, 300, -0.1])
plot_cr_ramsey(cr_ramsey_times, cr_ramsey_result, ramsey_fit)
```

We can see that we have substantially (but not totally) reduced the frequency shift (due to higher-order levels, etc.). At this point, we could return to the Hamiltonian tomography experiment
```
cr_scheds = build_cr_scheds(qc, qt, cr_times, phase=phase, ZI_MHz=ZI_rate)
```
However, since the frame change only affects the control qubit, the results would be identical to the second one.

## References

[1] S Sheldon, E Magesan, JM Chow, and JM Gambetta, "Procedure for systematically tuning up crosstalk in the cross resonance gate," *Phys Rev A* **93**, 060302 (2016)

[2] T Alexander, N Kanazawa, DJ Egger, L Capelluto, CJ Wood, A Javadi-Abhari and DC McKay, "Qiskit pulse: programming quantum computers through the cloud with pulses", *Quantum Sci. Technol.* **5** 044006 (2020)

[3] DC McKay, CJ Wood, S Sheldon, JM Chow, and JM Gambetta, "Efficient Z-Gates for Quantum Computing," *Phys Rev A* **96**, 022330 (2017)

```python
import qiskit
qiskit.__qiskit_version__
```

    {'qiskit-terra': '0.17.4', 'qiskit-aer': '0.7.6', 'qiskit-ignis': None, 'qiskit-ibmq-provider': None, 'qiskit-aqua': None, 'qiskit': None, 'qiskit-nature': None, 'qiskit-finance': None, 'qiskit-optimization': None, 'qiskit-machine-learning': None}

```python

```