# Table of Contents

1  Intoduction to Machine Learning: Linear and Logistic Regression

1.1  Packages

2  Linear regression: problem overview

2.1  Define the loss

2.2  Define the gradient

2.3  Compute the loss and gradient together

2.4  Perform the optimization

2.5  Results

2.6  Optional Exercises

3  Logistic Regression: problem overview

3.1  Loading data

3.2  Sigmoid

3.3  Loss function

3.4  Gradients

3.5  Initialize the weights and check matrix multiplication

3.6  Forward and backward propagation

3.7  Optimization

4  Introduction to Neural Networks

4.1  Initialize the parameters

4.2  The loop

4.3  Loss

4.4  Backpropagation

4.5  Update parameters

4.6  Put it all together

4.7  Discussion

# Intoduction to Machine Learning: Linear and Logistic Regression

Welcome to the first tutorial. We will use Python packages to build up our intuition for how machine learning with neural networks for modern high energy physics. First, to get comfortable with Python packages, we will do some basic linear regression to fit lines and curves. From this, we will change the task into a classification problem, referred to as logistic regression. This is the natural building block of neural networks.

**Instructions:**
- Use the built in vectorized functions whenever possible. This means you should not use for/while loops unless asked to.

## Packages
First, lets load in the packages that we will use.
- [numpy](https://www.numpy.org/devdocs/user/index.html) is the package for numerical computations with Python (NUMerical PYthon)
- [scipy](https://docs.scipy.org/doc/scipy/reference/) contains some of the numerical proceedures, such as minimizations
- [matplotlib](https://matplotlib.org/3.1.0/tutorials/index.html) is a package for plotting

```python
# allows for plotting within the notebook
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
```

```python
# The commands make the plots look nice
plt.rcParams.update({'font.family': 'cmr10',
                     'font.size': 12,
                     'font.serif': 'cmr10',
                     'axes.unicode_minus': False,
                     'axes.labelsize': 12,
                     'figure.figsize': (3, 3),
                     'figure.dpi': 80,
                     'mathtext.fontset': 'cm',
                     'mathtext.rm': 'serif',
                     'xtick.direction': 'in',
                     'ytick.direction': 'in',
                     'xtick.top': True,
                     'ytick.right': True,
                     'axes.formatter.use_mathtext': True
                     })
```

# Linear regression: problem overview
Load in the data using np.load(). We want the files `../data/tutorial_1_data/linear_training.npy` and `../data/tutorial_1_data/linear_testing.npy`.

```python
xy_training = np.load('../data/tutorial_1_data/linear_training.npy')
xy_testing = np.load('../data/tutorial_1_data/linear_testing.npy')
```

```python
x = xy_training[:, 0]
y = xy_training[:, 1]
x_test = xy_testing[:, 0]
y_test = xy_testing[:, 1]
```

```python
plt.scatter(x, y, label='training data')
plt.scatter(x_test, y_test, label='testing data')
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.legend()
plt.show()
```

This is the data that we will try to fit a line to. We will want to see how close we get to the true values of the slope and $y$ intercept that we put in by hand.

## Define the loss
In order to get a good fit, we first have to define what constitutes a good fit. For a situation where we are trying to predict a number, we often use the Mean Squared Error. This is given by

$L^{\rm{MSE}} = \frac{1}{N}\sum_i^N \big(y_i - f(x_i) \big)^2$

where $y_i$ is the *true* value in the data and $f(x_i)$ is the predicted value.

```python
def MSE(y_true, y_pred):
    '''
    Calculate the mean squared error

    Arguments:
    y_true -- np.array of the observed values
    y_pred -- np.array of the predicted values

    Returns:
    mse -- scalar of the mean squared error
    '''
    N = y_true.shape[0]

    # compute the MSE
    # Do not use a loop
    ### Start code here ### (about 1 line of code)
    mse =
    ### End code here ###

    return mse
```

```python
y_true = np.array([1, 2, 3])
y_pred = np.array([1.1, 1.9, 3])
print('MSE = ' + str(MSE(y_true, y_pred)))
```

**Expected Output**:

    MSE
     0.006666666666666678

## Define the gradient
To find the minimum of the loss function (minimizing the error) we need to have the gradient of the loss with respect to the parameters of the model. As our data looks linear, let's use a simple form of $f(x_i) = m~x_i+b$. We write this out with the chain rule as a precursor for the deep neural networks.
- Partial derivative with respect to $m$
\begin{eqnarray}
\frac{\partial}{\partial m} L^{\rm{MSE}} &=& \frac{1}{N} \sum_i^N~-2\big(y_i - f(x_i)\big) ~\frac{\partial f(x_i)}{\partial m}\\
&=& \frac{1}{N} \sum_i^N~-2\big(y_i - f(x_i)\big) ~x_i
\end{eqnarray}
- Partial derivative with respect to $b$
\begin{eqnarray}
\frac{\partial}{\partial b} L^{\rm{MSE}} &=& \frac{1}{N} \sum_i^N~-2\big(y_i - f(x_i)\big) ~\frac{\partial f(x_i)}{\partial b}\\
&=& \frac{1}{N} \sum_i^N~-2\big(y_i - f(x_i)\big)
\end{eqnarray}

## Compute the loss and gradient together
This will be called the "forward propagation" step when we get to neural networks. We want a function that takes in the
- model parameters (m and b)
- input data
- true y values

and computes returns the loss and the gradient (for both m and b)

```python
def propagate(m, b, X, Y):
    '''
    Implement the loss function and its gradient

    Arguments:
    m -- scalar, slope of the line
    b -- scalar, bias
    X -- np.array
    Y -- true value

    Return:
    loss -- Mean squared error
    dm -- gradient of the loss with respect to m
    db -- gradient of the loss with respect to b
    '''
    ### Implement your code here ###
    y_pred =
    loss =
    ## End code ###

    ### Find gradient
    ### Implement your code here ###
    dm =
    db =
    ## End code ###

    assert(dm.dtype == float)
    assert(db.dtype == float)

    # this makes a look-up table for easy use later
    grads = {'dm': dm,
             'db': db
            }

    return grads, loss
```

Test the implementation

```python
m, b, X, Y = 1.0, 2.5, np.array([1, 2, 3, 4]), np.array([1.2, -2.3, 3.0, 4.5])
grads, loss = propagate(m, b, X, Y)
print('dm = ' + str(grads['dm']))
print('db = ' + str(grads['db']))
print('MSE = ' + str(loss))
```

**Expected Output**:

    dm
    15.7

    db
    6.8

    MSE
    15.444999999999999

## Perform the optimization
**Exercise:** Write down the optimization function. The goal is to learn $m$ and $b$ by minimizing the loss function. For a parameter $\theta$, the update rule is $\theta = \theta - \alpha~d\theta$, where $\alpha$ is the learning rate and $d\theta$ is the derivative.

```python
def optimize(m, b, X, Y, num_iterations, learning_rate, print_loss=False):
    '''
    The function finds the optimal values for m and b by running a gradient descent algorithm

    Arguments:
    m -- scalar, slope of the line
    b -- scalar, bias
    X -- np.array
    Y -- true value
    num_iterations -- number of iterations of the optimization loop
    learning_rate -- learning rate of the gradient descent update rule
    print_loss -- True to print the loss every 100 steps

    Returns:
    params -- dictionary containing the slope m and bias b
    grads -- dictionary containing the gradients of the slope and bias with respect to the loss function
    losss -- list of all the losss computed during the optimization, this will be used to plot the learning curve.
    '''

    losss = []
    slopes = []
    biases = []

    for i in range(num_iterations):
        slopes.append(m)
        biases.append(b)

        # loss and caculate gradient
        ### Start code here ### (about 1 line of code)
        grads, loss =
        ### End code here ###

        # Retrieve derivatives from grads
        ### Start code here ### (about 2 lines of code)
        dm =
        db =
        ### End code here ###

        # update rule
        ### Start code here ### (about 2 lines of code)
        m =
        b =
        ### End code here ###

        # Record the losss
        losss.append(loss)

        # Print the loss every 100 training iterations
        if print_loss and i % 100 == 0:
            print('loss after iteration %i: %f' %(i, loss))

    params = {'m': m,
              'b': b
             }
    grads = {'dm': dm,
             'db': db
            }

    return params, grads, losss, slopes, biases
```

**Exercise:** Now use the `optimize` function to find the best fit line, using an initial guess for $m,b=0$. For now, use a learning rate of 0.01 and at least 1000 iterations of training.

```python
params, grads, losss, slopes, biases =
```

Let's plot the results to see if the training worked. The left panel should show the MSE loss, which should decrease with every training step. The next two panels show the value of the slope and $y$-intercept for each training step. These should be asymptoting to values around -2 and 5, respectively. If you do not see this happening, you should debug the `optimize` code above.

```python
plt.figure(figsize=(9, 3))

plt.subplot(1, 3, 1)
plt.plot(losss)
plt.xlabel('Iteration')
plt.ylabel('MSE')
plt.yscale('log')
plt.minorticks_on()

plt.subplot(1, 3, 2)
plt.plot(slopes)
plt.xlabel('Iteration')
plt.ylabel('Slope')
plt.minorticks_on()

plt.subplot(1, 3, 3)
plt.plot(biases)
plt.xlabel('Iteration')
plt.ylabel('Bias')
plt.minorticks_on()

plt.tight_layout()
```

## Results
Now plot the final predictions compared to the test values.

```python
xrange = np.linspace(0, 10)
y_pred = xrange * params['m'] + params['b']
plt.plot(xrange, y_pred)
plt.scatter(x_test, y_test, color='C1', label='testing data')
plt.text(7., 1, r'$y=$' + '{0:.3f}'.format(params['m']) + r'$*x + $' + '{0:.3f}'.format(params['b']),
         ha='center', va='top')
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.legend()
plt.show()
```

We should also check the loss for the training and testing values, to make sure that they are similar.

```python
Loss_train = MSE(y_true=y, y_pred=x*params['m'] + params['b'])
Loss_test = MSE(y_true=y_test, y_pred=x_test*params['m'] + params['b'])
print('The losses are: training set={0:.3e}, test set={1:.3e}'.format(Loss_train, Loss_test))
```

Notice that the loss is smaller for the training set. This is because our model is trained to minimize the loss on that data. However, the losses are still similar, so it does not look like we have over-fit the data.

## Optional Exercises
1. What is the effect of the learning rate? Try varying the learning rate between $10^{-3}$ to $1$. What happens when the learning rate is large? Why?
2. How would the procedure change if there was a shape to the data? Load the files `../data/tutorial_1_data/linear_regression_curved_training.npy` and `../data/tutorial_1_data/linear_regression_curved_test.npy` and find a function to fit the curve.

As an extra thought experiment, this example and the curved example only have one piece of input information (namely $x$). How would we generalize the setup to more than 1 input variabeles? So now, each 'event' $x_i$ is actually a vector which has $j$ dimensions. I will now change notation and write the whole dataset as $X^i_j$, where the lower components represent the vector of information, and the upper component is the different 'events' or datapoints. We will develop this more in the next section, when we explore logistic regression.

# Logistic Regression: problem overview
In linear regression, we are trying to predict a continuous value. In the literature, it is still referred to as linear regression when the models are more complicated, involving higher powers than just linear. Logisitic regression is used when we want to classify data as belonging to one group or another. In high energy physics, this could come in the form of: was this event new physics or not? was this jet from QCD or a boosted Higgs? We will build up to working with a dataset from our parton showers to ask if the initiating parton of jet was a quark or a gluon.

Before we get to the physics cases with many inputs, we will work another simple example using a 2D dataset so that we can easily visualize how things are working. The datasets we will be working with for this example are:
- `../data/tutorial_1_data/logistic_regression_training.npy`
- `../data/tutorial_1_data/logistic_regression_validation.npy`
- `../data/tutorial_1_data/logistic_regression_testing.npy`

We have specifically added in a validation set, which will be used to check if the models we make are being overfit to the training set. It is important that the *test* data is not used in optimizing model selections.

## Loading data
Load the datasets. These numpy arrays have a shape of (number of events, 3). The first two columns of the 3 are $x_1$ and $x_2$. The last columns is a number 0 or 1, which indicates if the event was a background or signal, respectively.

**Exercise:** Load the data and seperate out the data columns from the label. (Hint: use the slicing feature of numpy)

```python
training = np.load(...)
X_Train = training[:, :2]
Y_Train = training[:, 2].reshape(-1, 1)  # makes it a vector

validation = np.load(...)
X_Val = validation[:, :2]
Y_Val = validation[:, 2].reshape(-1, 1)  # makes it a vector

testing = np.load(...)
X_Test = testing[:, :2]
Y_Test = testing[:, 2].reshape(-1, 1)  # makes it a vector
```

Plot the data to see what we are tryingto fit.

```python
plt.scatter(X_Train[np.ravel(Y_Train==1), 0], X_Train[np.ravel(Y_Train==1), 1], color='C0', s=9, label='Sig')
plt.scatter(X_Train[np.ravel(Y_Train==0), 0], X_Train[np.ravel(Y_Train==0), 1], color='C3', s=9, label='Back')
plt.xlim(-0.05, 5.05)
plt.ylim(-0.05, 5.05)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')
plt.legend(loc=(1.05, 0.45), frameon=False, fontsize=14)
plt.title('Training Data')

plt.show()
```

Our goal is now to find a model which will predict whether a point $(x_1, x_2)$ in parameter space comes from either the Signal set or the Background set. As a first approximation, the boundary between the two is approximately linear. We will use this our first model.

## Sigmoid
In order to interpret our results, we will first need to modify our model such that the output is always between 0 and 1. Such a function is usually called a sigmoid function. We will use the most common definition as
\begin{equation}
\sigma(z) \equiv \frac{1}{1+e^{-z}}
\end{equation}

**Exercise:** Program a function for the sigmoid definition. Then plot $\sigma(z)$ for $z\in [-5, 5]$. Hint, use [this](https://docs.scipy.org/doc/numpy/reference/generated/numpy.exp.html).

```python
def sigmoid(z):
    '''
    Computes the sigmoid of z

    Arguments:
    z -- A scalar or numpy array of any size

    Return:
    s -- sigmoid(z)
    '''

    ### Start code here ### (about 1 line of code)
    s =
    ### End code here ###

    return s
```

Test the implementation

```python
sigmoid(np.array([0, 1, 2]))
```

**Expected Output:**

      sigmoid([0,1,2])
    array([0.5       , 0.73105858, 0.88079708])

Plot the function to see what it looks like.

```python
z = np.linspace(-5, 5, 21)
plt.plot(z, sigmoid(z))
plt.minorticks_on()
plt.xlabel('$z$')
plt.ylabel(r'$\sigma(z)$')
plt.show()
```

With this simple modification, we can now use a very similar model to what we had before. The first part of our linear model will be to take
\begin{equation}
z^i = w_1~x_1^i + w_2~x_2^i + b~,
\end{equation}
where supsript $i$ refers to the specific data point and the subscript refers to the particular dimension of the data. This should look very similar to our linear model before, where $b$ is again the bias, but now we have two "slopes" which we mark as $w_j$ instead of $m$. From now on, these will be called "weights". In building up to large dimensional data, we don't want to write out all of this multiplication by hand. Instead we will make a vector $W = (w_1, w_2, \cdots)$ and use the dot product.

In our optimization, we will learn the values of $w_1$, $w_2$, and $b$ which minimizes the loss. Note that we want these to yield **large positive** values of $z$ for events that are signal-like (lower left corner) and **large negative** values for the background-like (upper right). Then, when we feed these through the sigmoid function, they will be mapped to 0 and 1.

Feeding the results through the sigmoid is called the *activation*. The sigmoid is not the only function that can be used for this. In the next tutorial we will explore the need for other *Activation Functions*.

## Loss function
It is possible to use the mean squared error for classification, however, minimizing this does not lead to the best classification. This can be seen from:
1. The largest possible 'error' is 1, so there is no sense of being very wrong in a prediction.
2. Large positive or negative values of $z$ asymptote to 0 or 1. With this, the there is little difference in the mean squared error if the model outputs $z=10$ or $z=20$. However, from the underlying definition of $z=W\cdot X + b$, there is a big difference. This relates to how 'confident' the model is. We want the loss to reward or punish predictions which are confident and right or wrong, respectively.

To accomplish this, we use the **Binary Cross Entropy**. For a very nice lecture on how this can be derived from a probability standpoint, see [Lecture 2 of Gilles Louppe's class](https://github.com/glouppe/info8010-deep-learning).

The binary cross entropy is defined as
\begin{equation}
L^{\rm{BCE}} = -\frac{1}{N} \sum_i^N y^i \log f(\mathbf{x}^i) + \big(1-y^i\big) \log\big(1-f(\mathbf{x}^i) \big)
\end{equation}

where $y^i$ is again the true label, and $f(\mathbf{x}^i)$ is the bold and the vector denotes that $\mathbf{x}$ has multiple dimensions.

Let's examine why BCE works and addresses the problems we had with MSE. If the event is from the class $y=1$, then the first term contributes and the second term is identically 0. If the overall prediction is close to 1, the loss asymptotes to 0. So the more confident the model is, the lower the loss. Conversely, the more negative $z$ is, the closer to 0 is the prediction. When we take the $\log$ of this, the loss blows up (note the overall negative sign in the front of the sum). The story is similar for events coming from $y=0$. Now the first term is identically 0, and the second term contributes. Predictions close to 0 now result in small losses while perdictions close to 1 yield very large losses. While this is not a proof, it at least gives some intuition for why this loss function is used for binary classification problems.

**Exercise:** Implement the Binary Cross Entropy loss function.

```python
def BCE(y_true, y_pred):
    '''
    Computes the binary cross entropy

    Arguments:
    y_true -- 1 dimensional numpy array with the true labels. Should all be either 0 or 1
    y_pred -- 1 dimensional numpy array containing predictions. Should be continuous between 0 and 1

    Return:
    bce -- scalar value
    '''

    # Compute the BCE and do not use a loop
    ### Start code here ### (about 1 line of code)
    bce =
    ### End code here ###

    return bce
```

Test the implementation

```python
BCE(np.array([0, 0, 1, 1]), np.array([0.01, 0.99, 0.01, 0.99]))
```

**Expected Output:**

     BCE([0, 0, 1, 1]), [0.01, 0.99, 0.01, 0.99])
     2.307610260920796

## Gradients
We are now almost ready to make our first classiciation model. The last things needed for the optimization are the gradients. You should check the following results. I use $z$ just to help with the chain rule.

\begin{eqnarray}
\frac{\partial}{\partial z} L^{\rm{BCE}}\big(y_{\rm{true}}, y_{\rm{pred}}(z) \big)
 & = & \frac{1}{N}\sum_i^N \bigg(\frac{1-y_{\rm{true}}}{1-y_{\rm{pred}}(z)} - \frac{y_{\rm{true}}}{y_{\rm{pred}}(z)} \bigg) \frac{\partial y_{\rm{pred}}(z)}{\partial z}
\end{eqnarray}

We use $y_{\rm{pred}}\big(z(x)\big) = \sigma\big(z(x)\big)$,
\begin{eqnarray}
\frac{\partial \sigma\big(z(x)\big)}{\partial x} &=& \frac{e^{-z(x)}}{\big(1+e^{-z(x)} \big)^2} \frac{\partial z(x)}{\partial x} \\
&=& \sigma\big(z(x) \big) \Big(1-\sigma\big(z(x) \big) \Big) \frac{\partial z(x)}{\partial x}
\end{eqnarray}
where $x$ is just another dummy variable, and *not* the data.

Finally, we use $z^i = W\cdot \mathbf{x}^i + b$, giving:
\begin{equation}
\frac{\partial z(w,b)}{\partial w_j} = x_j^i
\end{equation}
and
\begin{equation}
\frac{\partial z(w,b)}{\partial b} = 1
\end{equation}

It will be beneficial to program in a function for the derivative of the sigmoid.

**Exercise:** Implement the sigmoid_derivative to compute the derivative with respect to the input.

```python
def sigmoid_derivative(z):
    '''
    Compute the gradient of the sigmoid function with respect to its input z.
    It can be useful to store this into a variable later

    Arguments:
    z -- A scalar or numpy array

    Returns:
    ds -- The computed gradient
    '''

    ### Start code here ### (around a few lines of code)
    s =
    ds =
    ### End code here ####

    return ds
```

Test the implementation

```python
x = np.array([1, 2, 3])
print ("sigmoid_derivative(x) = " + str(sigmoid_derivative(x)))
```

**Expected Output**:

         sigmoid_derivative([1,2,3])
         [ 0.19661193  0.10499359  0.04517666]

## Initialize the weights and check matrix multiplication
In the linear regression section, we started with the slope and $y$-intercept set to 0. We will want to do the same sort of thing for our weights and bias. We will write this as a function of the number of dimensions so that when we go to larger datasets we don't have to do it by hand

**Exercise:** Implement a function to output a vector of zeros for the weights and a single 0 for the bias. Hint: look up [numpy's zeros function](https://docs.scipy.org/doc/numpy/reference/generated/numpy.zeros.html?highlight=zeros#numpy.zeros)

```python
def initialize_zeros(dim):
    '''
    This function creates a vecor of zeros with shape (dim, 1) for W.
    It also inializes b to 0

    Argument:
    dim -- interger, the size of the w vector

    Returns:
    w -- initialized vector of shape (dim, 1)
    b -- initialized scalar for the bias
    '''

    ### Start code here ###
    w =
    b =
    ### end code here

    assert(w.shape == (dim, 1))
    assert(isinstance(b, float))

    return w, b
```

Test the implementation

```python
dim = 2
w, b = initialize_zeros(dim)
print ("w = " + str(w))
print ("b = " + str(b))
```

**Expected Output**:

          w
         [[ 0.]
 [ 0.]]

          b
         0

With this setup, we now have that the shape of $W$ is (2, 1). Each data vector for $X$ is a shape (1, 2) as well. The overall shape of the data is (N, 2). We want to get out the dot product over each row, so the output should be of the shape (N, 1)

\begin{equation}
Z = X\cdot W + b
\end{equation}

In some cases (namely doing images) the input data may have yet and extra dimension in the shape (the RGB colors for each pixel). Care needs to be taken in doing the matrix multiplication correctly.

**Exercise:** Check that the matrix multiplication works as expected. Note the difference in numpy between `*` and `.dot()`.

```python
# Use *
```

```python
# Use np.dot()
```

## Forward and backward propagation
Now that the parameters have been initalized, we can do the forward and backward steps for learning the parameters.

**Exercise:** Implement a function `propagate()` that computes the loss function and gradient.

**Hints:**

Forward propagation:
* You get $X$
* Compute $A = \sigma\big(X\cdot W +b\big) = \big(a_{1}, a_{2},\cdots,a_{N}\big)$
* Calculate the loss function $L = -\frac{1}{N} \sum_{i=1}^N y_i \log a_i + \big(1-y_i\big) \log\big(1-a_i\big)$

It will be usefull to fill in the rest of the chain rule and simplify to get
\begin{equation}
\frac{\partial L}{\partial w^j} = \frac{1}{N} \sum_{i=1}^N (a_i - y_i) * x_i^j
\end{equation}
and
\begin{equation}
\frac{\partial L}{\partial b} = \frac{1}{N} \sum_{i=1}^N (a_i - y_i)~.
\end{equation}
Again, we want to not use any explicit for loops, so be careful with the shapes when multiplying arrays.

```python
def propagate(w, b, X, Y):
    '''
    Implement the loss function and its gradient for the propagation above

    Arguments:
    w -- weights, a numpy array of size (dim(x[0]), 1)
    b -- bias, a scalar
    X -- data with shape (number of datapoints, dimensionality of the data)
    Y -- true label, vector of 0 or 1

    Returns:
    loss -- BCE which is the negative log-likelihhood (loss and loss are used interchangeably)
    dw -- gradient of the loss with respect to w, needs to be the same shape as w
    db -- gradient of the loss with respect to b, needs to be the same shape as b

    Tips:
    Write the code step by step and use np.log() and np.dot().
    Print shapes if you need to get the dot products correct.
    '''
    N = X.shape[0]  # number of data points

    # Forward propagation (from X to loss)
    # Do not use a loop
    ### Start code here ### (around 2 lines of code)
    A =
    loss =
    ### End code here ###

    # Backward propogration (to find the gradient)
    ### Start code here ### (around 2 lines of code)
    # Do not use a loop
    dw =
    db =
    ### END CODE HERE ###

    assert(dw.shape == w.shape)
    assert(db.dtype == float)
    loss = np.squeeze(loss)
    assert(loss.shape == ())

    grads = {"dw": dw,
             "db": db}

    return grads, loss
```

Test the implementation

```python
w, b, X, Y = np.array([[1.],[2.]]), 2., np.array([[1.,2.,-1.],[3.,4.,-3.2]]).T, np.array([[1,0,1]])
grads, loss = propagate(w, b, X, Y.reshape(-1, 1))
print ("dw = " + str(grads["dw"]))
print ("db = " + str(grads["db"]))
print ("loss = " + str(loss))
```

**Expected Output**:

          ** dw **
       [[ 0.99845601]
     [ 2.39507239]]

          ** db **
         0.00145557813678

          ** loss **
         5.801545319394553

## Optimization
Now that we have initialized our parameters and can compute the loss and the gradient, we can find the minimum of the loss function using gradient descent.

**Exercise:** Write the optimization function. The goal is to learn $w$ and $b$ minimizing $L^{\rm{BCE}}$. As before, for a parameter $\theta$, the update rule is $\theta = \theta - \alpha~ d\theta$, where $\alpha$ is the learning rate.

```python
def optimize(w, b, X_train, Y_train, num_iterations, learning_rate, print_loss=False, X_val=None, Y_val=None):
    '''
    This function minimizes the BCE loss function by running gradient descent

    Arguments:
    w -- weights, a numpy array of size (dim(data), 1)
    b -- bias, a scalar
    X_train -- data of shape (number of samples, dim(data))
    Y_train -- true label of shape (number of samples, 1)
    num_iterations -- number of updates using the gradient, an integer
    learning_rate -- a float which affects the update size
    print_loss -- boolean to print the loss ever 100 steps. Default is False
    X_val -- data of shape (number of samples, dim(data)). Default is None
    Y_val -- true label of shape (number of samples, 1). Default is None

    If X_val and Y_val are not None, the loss is computed on the validation data

    Returns:
    params -- dictionary containing the weights w and bias b
    grads -- dictionary containing the gradients of the weights and bias with respect to the loss function
    losss -- list of all the losss computed during the optimization, this will be used to plot the learning curve.
    '''

    losss = {'train': [],
             'validation': []
            }

    for i in range(num_iterations):

        # loss and gradient calculation (around 1-4 lines of code)
        ### Start code here ###
        grads, loss =
        ### end code here

        # Retrieve derivatives from grads
        ### Start code here ###
        dw =
        db =
        ### End code here ###

        # Calculate the loss on the validation data
        if (X_val is not None) and (Y_val is not None):
            ### Star code here
            # Do not use a loop
            A_val =
            loss_val =
            ### End code here

            losss['validation'].append(loss_val)

        # update rule (≈ 2 lines of code)
        ### START CODE HERE ###
        w =
        b =
        ### END CODE HERE ###

        # Record the losss
        losss['train'].append(loss)

        # Print the loss every 100 training iterations
        if print_loss and i % 100 == 0:
            print ("loss after iteration %i: %f" %(i, loss))

    params = {"w": w,
              "b": b}

    grads = {"dw": dw,
             "db": db}

    return params, grads, losss

```

Test the implementation

```python
params, grads, losss = optimize(w, b, X, Y.reshape(-1, 1), num_iterations= 100, learning_rate = 0.009, print_loss = False)

print ("w = " + str(params["w"]))
print ("b = " + str(params["b"]))
print ("dw = " + str(grads["dw"]))
print ("db = " + str(grads["db"]))
```

**Expected Output**:

        w
       [[ 0.19033591]
 [ 0.12259159]]

        b
        1.92535983008

        dw
        [[ 0.67752042]
 [ 1.41625495]]

        db
        0.219194504541

If those worked, we are now ready to classify our example data.

**Exercise:** Train a logistic regression model on the example. Use around 10000 iterations and pick an appropriate learning rate. We want the performance to be saturating by the end.

```python
# Initialize w and b
w, b =

# Optimize
params, grads, losss =
```

Plot the loss as a function of epoch.

```python
plt.plot(losss['train'], label='Training')
plt.plot(losss['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.show()
```

Our training and validation losses are nearly identical, so we are not overfitting. This is expected because we are using such an easy model. Now lets plot the decision boundary found by the model. There will be a few lines of matplotlib and numpy in order to plot isocurves of constant model prediction.

```python
x_space = np.linspace(0, 5, 25)
y_space = np.linspace(0, 5, 25)
X, Y = np.meshgrid(x_space, y_space)
Z = sigmoid(X * params['w'][0] + Y * params['w'][1] + params['b'])  # the 1 - sigmoid just gets the numbering right for the color map below

plt.contourf(X, Y, 1-Z, 25, cmap='bwr')
CS = plt.contour(X, Y, Z, [0.1, 0.5, 0.9], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.05, 5.05)
plt.ylim(-0.05, 5.05)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')
plt.show()
```

Our linear model draws straight lines through the space. The nominal decision boundard is when the output it $\ge 0.5$, in which case it is more likely to be signal than background. How does this match up with out test data?

```python
plt.scatter(X_Test[Y_Test.ravel()==1, 0], X_Test[Y_Test.ravel()==1, 1], color='b', s=9, alpha=0.5)
plt.scatter(X_Test[Y_Test.ravel()==0, 0], X_Test[Y_Test.ravel()==0, 1], color='r', s=9, alpha=0.5)
CS = plt.contour(X, Y, Z, [0.1, 0.5, 0.9], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.05, 5.05)
plt.ylim(-0.05, 5.05)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')
plt.show()
```

If we look at the data with score to the left of the 0.9 line, we see that most of them are blue, likewise, most of the data to the right of the 0.1 line is red. The areas that the model is confident in do well. However, there are many misclassified points inbetween.

**Optional Exercise:**
With what we have, we can easily add in more parameters to get a better fit. For instance, we may think that there should be some curvature to the decision boundary. Using the [scikit-learn](https://scikit-learn.org/stable/index.html) package, we can add in all of the quadratic (or higher) terms. Specifically, look up the [PolynomialFeatures](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html#sklearn.preprocessing.PolynomialFeatures)

```python
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(2, include_bias=False)
```

```python
# Use poly from above
X_Quadratic_Train =
print(X_Quadratic_Train.shape)
```

```python
# Also do the transormation on the test and validation data
X_Quadratic_Test =
X_Quadratic_Validatoin =
```

Now fit the model. You can use the same parameters as before.

```python
# Initialize w and b
wquad, bquad =

# Optimize
paramsquad, gradsquad, losssquad =
```

Plot the decision boundar.

```python
Z = []
for x1 in x_space:
    zs = []
    for x2 in y_space:
        data = poly.transform([[x1, x2]])
        zs.append(float(sigmoid(np.dot(data, paramsquad['w']) + paramsquad['b']).flatten()[0]))
    Z.append(zs)
Z = np.array(Z)

plt.figure(figsize=(6, 3))

plt.subplot(1, 2, 1)
plt.contourf(X, Y, 1-Z, 25, cmap='bwr')
CS = plt.contour(X, Y, Z, [0.1, 0.5, 0.9], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.0, 5.0)
plt.ylim(-0.0, 5.0)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')

plt.subplot(1, 2, 2)
plt.scatter(X_Test[Y_Test.ravel()==1, 0], X_Test[Y_Test.ravel()==1, 1], color='b', s=9, alpha=0.5)
plt.scatter(X_Test[Y_Test.ravel()==0, 0], X_Test[Y_Test.ravel()==0, 1], color='r', s=9, alpha=0.5)
CS = plt.contour(X, Y, Z, [0.1, 0.5, 0.9], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.0, 5.0)
plt.ylim(-0.0, 5.0)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')

plt.tight_layout(w_pad=2)
```

Compare with the linear model.

```python
plt.figure(figsize=(6, 3))

ax0 = plt.subplot(1, 2, 1)
plt.plot(losss['train'], label='Training')
plt.plot(losss['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.title('Only Linear')

plt.subplot(1, 2, 2, sharey=ax0)
plt.plot(losssquad['train'], label='Training')
plt.plot(losssquad['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.title('All Quadratic')

plt.tight_layout(w_pad=2)
```

Note that the overall loss lower when the quadratic terms are added, and there is still no overfitting. As a side note, it also trains faster. How do we know when to stop adding in new features? Part of the revolultion in machine learning is that we let the machine choose which features it wants. This will be discussed in the next section, where we build our first neural network to fit the same data.

Also note, scikit-learn has [linear regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html) and [logistic regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) already programmed in.

# Introduction to Neural Networks

So far, out models have gone straight from the input to the output using matrix multiplication and an activation function. By adding an extra layer, we can use the same methodology to make a nerual network. The layer between the input and the output can be viewed as an extra vector, so an initial matrix goes from the inputs to the dimension of the new vector. Then a second matrix goes from the vector to the final answer. This layer is also called a **Hidden Layer**.

To add more notation, we will now use superscript brackets to denote the layer number. So the inputs are $X^{i[0]}_j$. The first weight vector, $W^{[1]}$ takes the input to the vector $z^{[1]}$. The explicit equations are:

\begin{eqnarray}
z^{i[1]} &=& X^i \cdot W^{[1]} + b^{[1]} \\
a^{i[1]} &=& \sigma\big(z^{[1]}_{i} \big) \\
z^{i[2]} &=& a^{i[1]} \cdot W^{[2]} + b^{[2]} \\
y^{i\rm{,pred}} = a^{i[2]} &=& \sigma \big( z^{[2]} \big)
\end{eqnarray}

Now we have to minimize the loss over $W^{[1]}, W^{[2]}, b^{[1]}, b^{[2]}$, where the loss is given by
\begin{equation}
L^{BCE} = -\frac{1}{N} \sum_{i=1}^{N} \bigg( y^i \log a^{i[2]} + \big(1-y^i \big) \log\big(1-a^{i[2]} \big) \bigg)
\end{equation}

Working out the individual gradients gives
\begin{eqnarray}
\frac{\partial z^{i[1]}}{\partial W^{[1]}} &=& X^i \\
\frac{\partial z^{i[1]}}{\partial b^{[1]}} &=& 1 \\
\frac{\partial a^{i[1]}}{\partial z^{i[1]}} &=& a^{i[1]} \big(1-a^{i[1]} \big) \\
& \rm{and} & \\
\frac{\partial z^{i[2]}}{\partial W^{[2]}} &=& a^{i[1]} \\
\frac{\partial z^{i[2]}}{\partial b^{[2]}} &=& 1 \\
\frac{\partial a^{i[2]}}{\partial z^{i[2]}} &=& a^{i[2]} \big(1-a^{i[2]} \big)
\end{eqnarray}

With these partial derivatives, we can use the chain rule to get
\begin{eqnarray}
\frac{\partial L}{\partial W^{[2]}} &=& \frac{\partial L}{\partial a^{[2]}_i}\frac{\partial a^{[2]}_i}{\partial z^{[2]}_i} \frac{\partial z^{[2]}_i}{\partial W^{[2]}} \\
&=& -\frac{1}{N} \sum_{i=1}^N \bigg(\frac{y_i}{a^{[2]}_i} - \frac{1-y_i}{1-a^{[2]}_i} \bigg) \bigg(a^{[2]}_i \big(1-a^{[2]}_i\big) \bigg) a^{[1]}_i \\
&=& -\frac{1}{N} \sum_{i=1}^N \big(y - a^{[2]}_i \big) a^{[1]}_i
\end{eqnarray}

and

\begin{eqnarray}
\frac{\partial L}{\partial b^{[2]}} &=&
\frac{\partial L}{\partial a^{[2]}_i}
\frac{\partial a^{[2]}_i}{\partial z^{[2]}_i}
\frac{\partial z^{[2]}_i}{\partial b^{[2]}} \\
&=&
-\frac{1}{N} \sum_{i=1}^N \bigg(\frac{y_i}{a^{[2]}_i} - \frac{1-y_i}{1-a^{[2]}_i} \bigg) \bigg(a^{[2]}_i \big(1-a^{[2]}_i\big) \bigg)  \\
&=& -\frac{1}{N} \sum_{i=1}^N \big(y - a^{[2]}_i \big)
\end{eqnarray}

Which are similar to what we had for the simple logistic regression. Now we step back to get the gradients for the $^{[1]}$ elements.

\begin{eqnarray}
\frac{\partial L}{\partial W^{[1]}} &=&
\frac{\partial L}{\partial a^{i[2]}}
\frac{\partial a^{i[2]}}{\partial z^{i[2]}}
\frac{\partial z^{i[2]}}{\partial a^{i[1]}}
\frac{\partial a^{i[1]}}{\partial z^{i[1]}}
\frac{\partial z^{i[1]}}{\partial W^{[1]}} \\
%
&=& -\frac{1}{N} \sum_{i=1}^N \bigg(\frac{y^i}{a^{i[2]}} - \frac{1-y^i}{1-a^{i[2]}} \bigg) \bigg(a^{i[2]} \big(1-a^{i[2]}\big) \bigg)
\bigg(W^{[2]} \bigg)
\bigg(a^{i[1]} \big(1-a^{i[1]}\big) \bigg)
X^i \\
%
&=& -\frac{1}{N} \sum_{i=1}^N \bigg(y^i - a^{i[2]} \bigg)
\bigg(W^{[2]} \bigg)
\bigg(a^{i[1]} \big(1-a^{i[1]}\big) \bigg)
X^i
\end{eqnarray}

Similarly,
\begin{eqnarray}
\frac{\partial L}{\partial b^{[1]}} &=&
\frac{\partial L}{\partial a^{i[2]}}
\frac{\partial a^{i[2]}}{\partial z^{i[2]}}
\frac{\partial z^{i[2]}}{\partial a^{i[1]}}
\frac{\partial a^{i[1]}}{\partial z^{i[1]}}
\frac{\partial z^{i[1]}}{\partial b^{[1]}} \\
%
&=& -\frac{1}{N} \sum_{i=1}^N \bigg(y^i - a^{i[2]} \bigg)
\bigg(W^{[2]} \bigg)
\bigg(a^{i[1]} \big(1-a^{i[1]}\big) \bigg)
\end{eqnarray}

While this was more detail than probably necessary, it is seeing the chain of partial derivatives is part of what has lead to the increased performance in machine learning. Keeping track of each partial derivative and the local values for the $a$ make these relatively easy. Note that we will have to be careful with our matrix multiplications, and may need to transpose from time to time.

## Initialize the parameters
Now that we know what paramters we want to use, we need to intialize them into memory. Earlier, we set all of the initial parameters to 0, this does not work with hidden layers, so instead we will randomly initialize.
- Use `np.random.randn(a,b) * 0.01` to randomly initialize a matrix of shape (a,b)
- Our input data will just be the two components.
- The biases can still be normalized to 0.

**Exercise:** Implement the initialize_random_parameters() function.

```python
def initialize_random_parameters(n_x, n_h):
    '''
    Arguments:
    n_x -- size of the input data
    n_h -- size of the hidden layer

    Returns:
    params -- python dictionary containg the parameters:
        W1 -- weight matrix of shape (n_x, n_h)
        b1 -- bias vector of shape (1, n_h)
        W2 -- weight matrix of shape (n_h, 1)
        b2 -- bias of shape (1, 1) or a scalar
    '''
    np.random.seed(42)  # For reproducible results

    ### Start code here ###
    W1 =
    b1 =
    W2 =
    b2 =
    ### End code here ###

    assert (W1.shape == (n_x, n_h))
    assert (b1.shape == (1, n_h))
    assert (W2.shape == (n_h, 1))
    assert (b2.shape == (1, 1))

    parameters = {"W1": W1,
                  "b1": b1,
                  "W2": W2,
                  "b2": b2}

    return parameters
```

Check the implementation

```python
parameters = initialize_random_parameters(2, 4)
print("W1 = " + str(parameters["W1"]))
print("b1 = " + str(parameters["b1"]))
print("W2 = " + str(parameters["W2"]))
print("b2 = " + str(parameters["b2"]))
```

**Expected output:**

W1 | [[ 0.00496714 -0.00138264  0.00647689  0.0152303 ][-0.00234153 -0.00234137  0.01579213  0.00767435]] |
------|------|
b1 |  [[0. 0. 0. 0.]]|
W2 | [[-0.00469474][ 0.0054256 ][-0.00463418][-0.0046573 ]]|
b2 | [[0.]]|

## The loop
**Excercise:** Implment the forward_propagation step.

**Hints:**
* Use the definitions from above
* Need to use the paramters from the output of the initialize funciton, i.e. parameter['..']
* Values need for the backpropagation are stored in 'cache'

```python
def forward_propagation(X, parameters):
    """
    Argument:
    X -- input data of size (n_samples, n_x)
    parameters -- python dictionary containing your parameters (output of initialization function)

    Returns:
    A2 -- The sigmoid output of the second activation
    cache -- a dictionary containing "Z1", "A1", "Z2" and "A2"
    """
    # Retrieve each parameter from the dictionary "parameters"
    ### START CODE HERE ### (≈ 4 lines of code)
    W1 = parameters['W1']
    b1 = parameters['b1']
    W2 = parameters['W2']
    b2 = parameters['b2']
    ### END CODE HERE ###

    # Implement Forward Propagation to calculate A2 (probabilities)
    # Do not use a loop
    ### START CODE HERE ### (≈ 4 lines of code)
    Z1 =
    A1 =
    Z2 =
    A2 =
    ### END CODE HERE ###

    assert(A2.shape == (X.shape[0], 1))

    cache = {"Z1": Z1,
             "A1": A1,
             "Z2": Z2,
             "A2": A2}

    return A2, cache
```

Test the implementation

```python
X_assess = np.array([[0.14092422, 0.80219698],
                     [0.07455064, 0.98688694],
                     [0.77224477, 0.19871568],
                     [0.00552212, 0.81546143],
                     [0.70685734, 0.72900717]])
A2, cache = forward_propagation(X_assess, parameters)
# print('A2: ', A2)
print(np.mean(cache['Z1']) ,np.mean(cache['A1']),np.mean(cache['Z2']),np.mean(cache['A2']))
```

**Expected Output**

[0.005467349042875904 0.5013668146565611 -0.004311049213484426 0.49892223936583624]

## Loss
Now that we have $a^{[2]}$ for all of the examples, we can compute the loss.

**Exercise:**
Implement the BCE loss function using the with inputs of `Y, A2, parameters`.

```python
def calculate_loss(Y, A2, parameters):
    '''
    Computes the binary cross entroy loss

    Arguments:
    Y -- Truth labels vector of size (n_Samples, 1)
    A2 -- The sigmoid output of the second activation, shape (n_Samples, 1)
    parameters -- python dictionary containing W1, b1, W2, b2

    Returns:
    loss -- BCE, scalar
    '''

    N = Y.shape[0]  # n_samples

    # Compute the cross entropy
    ### Start code here ###
    loss =
    ### End code here ###

    loss = np.squeeze(loss)

    assert(isinstance(loss, float))

    return loss
```

Test the implementation

```python
A2, cache = forward_propagation(X_assess, parameters)
Y_assess = np.array([[0], [1], [0], [1], [0]])
print("loss = " + str(calculate_loss(Y_assess, A2, parameters)))
```

**Expected Output**

loss  | 0.6927166002914666

## Backpropagation
Once the loss has been calculated, we are ready to do the backprop step, using the saved `cahce` to compute the gradients.

**Exercise:** Implement the backward_propagation to compute the gradients.

```python
def backward_propagation(parameters, cache, X, Y):
    '''
    Computes the gradients

    Arguments:
    parameters -- python dictionary containing W1,b1,W2,b2
    cache -- python dictionary containing Z1, A1, Z2, A2
    X -- input data of shape (n_samples, n_x)
    Y -- true labels of shape (n_samples, 1)

    Returns:
    grads -- python dictionary containing the gradients with respect to W1,b1,W2,b2
    '''

    N = X.shape[0]  # n_samples

    # First, pull values out of dictionary parameters
    ### Start code here ###
    W1 =
    W2 =
    ### End code here ###

    # Get the values of A1 and A2
    ### Start code here ###
    A1 =
    A2 =
    ### End code here ###

    # Compute gradients (hint use chain rule)
    ### Start code here
    dLdZ2 =
    dW2 =
    db2 =
    dZ1 =
    dW1 =
    db1 =
    ### END CODE HERE ###

    assert(dW2.shape == W2.shape)
    assert(dW1.shape == W1.shape)
    assert(db1.shape == parameters['b1'].shape)

    grads = {"dW1": dW1,
             "db1": db1,
             "dW2": dW2,
             "db2": db2}

    return grads
```

```python
parameters = initialize_random_parameters(2, 4)
X_assess = np.array([[0.14092422, 0.80219698],
                     [0.07455064, 0.98688694],
                     [0.77224477, 0.19871568],
                     [0.00552212, 0.81546143],
                     [0.70685734, 0.72900717]])
Y_assess = np.array([[0], [1], [0], [1], [0]])
A2, cache = forward_propagation(X_assess, parameters)
grads = backward_propagation(parameters, cache, X_assess, Y_assess)
```

## Update parameters
Now we need to update the parameters as $\theta = \theta - \alpha \frac{\partial L}{\partial\theta}$.

```python
def update_parameters(parameters, grads, learning_rate = 1.0):
    '''
    Updates the parameters using theta = theta - lr * dtheta

    Arguments:
    parameters -- python dictionary containing W1,b1,W2,b2
    grads -- python dictionary containing dW1,db1,dW2,db2
    learning_rate -- float, step size (default is 1.0)

    Outputs:
    parameters -- python dictionary containing W1,b1,W2,b2
    '''

    # Retrieve parameters
    ### Start code here ###
    W1 =
    b1 =
    W2 =
    b2 =
    ### End code here ###

    # Get gradients
    ### Start code here
    dW1 =
    db1 =
    dW2 =
    db2 =
    ### End code here ###

    # Update rule for each parameter
    ### START CODE HERE ### (≈ 4 lines of code)
    W1 =
    b1 =
    W2 =
    b2 =
    ### END CODE HERE ###

    parameters = {"W1": W1,
                  "b1": b1,
                  "W2": W2,
                  "b2": b2}

    return parameters
```

```python
parameters = update_parameters(parameters, grads)

print("W1 = " + str(parameters["W1"]))
print("b1 = " + str(parameters["b1"]))
print("W2 = " + str(parameters["W2"]))
print("b2 = " + str(parameters["b2"]))
```

**Expected output**

## Put it all together
**Exercise:**
Build a neural network model.

```python
def nn_model(X, Y, n_h, num_iterations=10000, validaion_data=None, learning_rate=1e-2, print_loss=False):
    """
    Arguments:
    X -- dataset of shape (number of examples, dim)
    Y -- labels of shape (number of examples, 1)
    n_h -- size of the hidden layer
    num_iterations -- Number of iterations in gradient descent loop

    Returns:
    parameters -- parameters learned by the model. They can then be used to predict.
    """

    losses = {'train': [],
              'validation': []
             }
    # initialize parameters
    n_x = X.shape[1]
    parameters = initialize_random_parameters(n_x, n_h)
    print(parameters)

    for i in range(num_iterations):
        # Forward propagation
        ### Start code here ###
        A2, cache =
        ### End code here ###

        # Compute loss
        ### Start code here ##
        loss =
        ### End code here ###
        losses['train'].append(loss)

        if validaion_data is not None:
            x_val = validaion_data[0]
            y_val = validaion_data[1]
            A2_val, _ = forward_propagation(x_val, parameters)
            loss_val = calculate_loss(y_val, A2_val, parameters)
            losses['validation'].append(loss_val)

        # Back propagation
        ### Start code here ###
        grads =
        ### End code here ###

        # Update parameters
        ### Start code here ###
        parameters =
        ### End code here ###

        if (print_loss) and (i % 1000 == 0):
            print('Loss after {0:04d} of {1:05d} iterations: {2:0.3e}'.format(i,
                                                                              num_iterations,
                                                                              loss)
                 )

    return parameters, losses
```

Ttrain the neural network. Use between 4 and 10 nodes on the hidden layer and a learnign rate of $10^{-3}$. Train for 100000 iterations.

```python
nn_params, nn_losses =
print(nn_params)
```

Plot the loss as a function of the epochs.

```python
plt.plot(nn_losses['train'])
plt.plot(nn_losses['validation'], ls=':')
```

Compare with other models.

```python
plt.figure(figsize=(9, 3))

ax0 = plt.subplot(1, 3, 1)
plt.plot(losss['train'], label='Training')
plt.plot(losss['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.title('Only Linear')

plt.subplot(1, 3, 2, sharey=ax0)
plt.plot(losssquad['train'], label='Training')
plt.plot(losssquad['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.title('All Quadratic')

plt.subplot(1, 3, 3, sharey=ax0)
plt.plot(nn_losses['train'], label='Training')
plt.plot(nn_losses['validation'], label='Validation', ls=':')
plt.xlabel('Epoch')
plt.ylabel('Loss (BCE)')
plt.legend(loc='upper right', frameon=False)
plt.title('NN (on linear input)')

plt.tight_layout(w_pad=2)
```

Plot the decision boundary

```python
x_space = np.linspace(0, 5, 25)
y_space = np.linspace(0, 5, 25)
X, Y = np.meshgrid(x_space, y_space)
Z_nn = []

for x in x_space:
    z_nn = []
    for y in y_space:
        A2, _ = forward_propagation(np.array([[x, y]]), nn_params)
        z_nn.append(A2.squeeze())
    Z_nn.append(z_nn)
Z_nn = np.array(Z_nn)

plt.figure(figsize=(6, 3))

plt.subplot(1, 2, 1)
plt.contourf(X, Y, 1-Z_nn, 25, cmap='bwr')
CS = plt.contour(X, Y, Z_nn, [0.1, 0.5, 0.9], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.05, 5.05)
plt.ylim(-0.05, 5.05)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')

plt.subplot(1, 2, 2)
plt.scatter(X_Test[Y_Test.ravel()==1, 0], X_Test[Y_Test.ravel()==1, 1], color='b', s=9, alpha=0.5)
plt.scatter(X_Test[Y_Test.ravel()==0, 0], X_Test[Y_Test.ravel()==0, 1], color='r', s=9, alpha=0.5)
CS = plt.contour(X, Y, Z_nn, [0.1, 0.5, 0.9,], colors='k')
plt.clabel(CS, inline=1, fontsize=12)
plt.xlim(-0.0, 5.0)
plt.ylim(-0.0, 5.0)
plt.minorticks_on()
plt.xlabel('$x_1$')
plt.ylabel('$x_2$')

plt.tight_layout(w_pad=2)

```

## Discussion
What happened here? We went through all of that effort and it seems like the neural network is not doing any better than the original logistic regression. It turns out that the choice of the activation function makes a big difference in how well the network performs. We still want to use the sigmoid on the last step to map the answers between 0 and 1. However, for the hidden layer, we can do better if we use a different function. The most common activation function for the middle layers are

Name | Equation | Derivative
--- | --- | ---
REctified Linear Unit (ReLU) | $a(z) = \left\{\begin{matrix}0 & z < 0\\ z & z >=0\end{matrix}\right.$ | $\frac{\partial a}{\partial z} = \left\{\begin{matrix}0 & z < 0\\ 1 & z >=0\end{matrix}\right.$
tanh | $a(z) = \tanh(z)$ | $\frac{\partial a}{\partial z} = 1-a^2$ |

Both of these are simple to implement.

**Exercise:**
Implement one of the new activation functions.
- Define the function (or use np.tanh()) above the `propagate` function cell. Then use this in the a1 = func(z1), but not for the a2 = func(z2).
- Add the derivative term to the dZ1 term in the `backward_propagation` cell.
- These should be the only changes needed, rerun using the same parameters. You should get out a decision boundary that looks more similar to a curve (but is an interpolation of straight lines).

**Extra Challenge:**

The real power of deep learning comes from making the networks deeper, rather than adding more units to the hidden layer. From the code we have here, generalize to network with 2 hidden layers, with the final output now being a3 instead of a2.

**Even more challenging:**

Adjust the code so that you only need to pick *how many layers* and it will add in the appropriate forward and backward propagation steps.

```python

```


This notebook is part of the *orix* documentation https://orix.readthedocs.io. Links to the documentation won’t work from the notebook.

# Crystal reference frame

This notebook describes the alignment of crystal reference frames and symmetry
operations in `orix`.

```python
%matplotlib inline

from diffpy.structure import Atom, Lattice, Structure
import matplotlib.pyplot as plt
import numpy as np

from orix.crystal_map import Phase
from orix.quaternion import Rotation
from orix.vector import Miller

plt.rcParams.update(
    {
        "figure.figsize": (10, 5),
        "font.size": 20,
        "axes.grid": True,
        "lines.markersize": 10,
        "lines.linewidth": 3,
    }
)
```

## Alignment and the structure matrix

The direct Bravais lattice is characterized by basis vectors ($\mathbf{a},
\mathbf{b}, \mathbf{c}$) with unit cell edge lengths ($a$, $b$, $c$) and angles
($\alpha$, $\beta$, $\gamma$). The reciprocal lattice has basis vectors given by

$$
\mathbf{a^*} = \frac{\mathbf{b} \times \mathbf{c}}{\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})},\:\:
\mathbf{b^*} = \frac{\mathbf{c} \times \mathbf{a}}{\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})},\:\:
\mathbf{c^*} = \frac{\mathbf{a} \times \mathbf{b}}{\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})},
$$

with reciprocal lattice parameters ($a^*$, $b^*$, $c^*$) and angles ($\alpha^*$,
$\beta^*$, $\gamma^*$).

Using these two crystallographic lattices, we can define a standard Cartesian
(orthonormal) reference frame by the unit vectors ($\mathbf{e_1}, \mathbf{e_2},
\mathbf{e_3}$). In principle, the direct lattice reference frame can be oriented
arbitrarily in the Cartesian reference frame. In `orix` we have chosen

$$
\mathbf{e_1} ||\: \frac{\mathbf{a}}{a},\:\:
\mathbf{e_2} ||\: \mathbf{e_3} \times \mathbf{e_1},\:\:
\mathbf{e_3} ||\: \frac{\mathbf{c^*}}{c^*}.
$$

This alignment is used for example in
Rowenhorst et al. (2015) and
De Graef (2003), the latter
which was the basis for the *EMsoft* Fortran suite of programs. Another common
option is $\mathbf{e_1} || \mathbf{a^*}/a^*$, $\mathbf{e_2} ||
\mathbf{e_3} \times \mathbf{e_1}$, $\mathbf{e_3} || \mathbf{c}/c$, which is used
for example in
Britton et al. (2016) and the
`diffpy.structure` Python package, which we'll come back to.

In calculations, it is useful to describe the transformation of the Cartesian
unit *row* vectors to the coordinates of the direct lattice vectors by the
structure *row* matrix $\mathbf{A}$ (also called the crystal *base*). Given the
chosen alignment of basis vectors with the Cartesian reference frame,
$\mathbf{A}$ is defined as

\begin{equation}
\mathbf{A} =
\begin{pmatrix}
a & 0 & 0\\
b\cos\gamma & b\sin\gamma & 0\\
c\cos\beta & -c\frac{(\cos\beta\cos\gamma - \cos\alpha)}{\sin\gamma} & \frac{\mathrm{V}}{ab\sin\gamma}
\end{pmatrix},
\end{equation}

where $V$ is the volume of the unit cell.

In `orix`, we use the
[Lattice](https://www.diffpy.org/diffpy.structure/mod_lattice.html#diffpy.structure.lattice.Lattice)
class in `diffpy.structure` to keep track of these properties internally. Let's
create a trigonal crystal with lattice parameters $(a, b, c)$ = (1.7, 1.7, 1.4)
nm and ($\alpha, \beta, \gamma$) = $(90^{\circ}, 90^{\circ}, 120^{\circ})$

```python
lattice = Lattice(1.7, 1.7, 1.4, 90, 90, 120)
lattice
```

`diffpy.structure` stores the structure matrix in the `Lattice.base` property

```python
lattice.base
```

and we see that `diffpy.structure` does not use the `orix` alignment mentioned
above, since $\mathbf{e1} \nparallel \mathbf{a} / a$. Instead, we see that
$\mathbf{e3} \parallel \mathbf{c} / c$, which is in line with the alternative
alignment mentioned above.

Thus, the alignment is updated internally whenever a
[Phase](reference.rst#orix.crystal_map.Phase) is created, a class which brings
together this crystal lattice and a point group
[Symmetry](reference.rst#orix.quaternion.Symmetry)

```python
phase = Phase(point_group="321", structure=Structure(lattice=lattice))
phase.structure.lattice.base
```

Warning

Using another alignment than the one described above has undefined behaviour in
orix. Therefore, it is important that the structure matrix of a `Phase` instance
is not changed.

Note

The lattice is included in a
[Structure](https://www.diffpy.org/diffpy.structure/package.html#diffpy.structure.structure.Structure)
because the latter class brings together a lattice and
[atoms](https://www.diffpy.org/diffpy.structure/mod_atom.html#diffpy.structure.atom.Atom),
which is useful when simulating diffraction.

We can visualize the alignment of the direct and reciprocal lattice basis
vectors with the Cartesian reference frame using the stereographic projection

```python
abc = Miller(uvw=[[1, 0, 0], [0, 1, 0], [0, 0, 1]], phase=phase)
abcr = Miller(hkl=[[1, 0, 0], [0, 1, 0], [0, 0, 1]], phase=phase)
```

```python
fig = abc.scatter(
    c=["r", "g", "b"],
    marker="o",
    return_figure=True,
    axes_labels=["e1", "e2"],
    hemisphere="both",
)
abcr.scatter(c=["r", "g", "b"], marker="x", s=300, figure=fig)
```

## Alignment of symmetry operations

To see where the crystallographic axes about which the point group symmetry
operations rotate, we can add symmetry operations to the figure, like is done
in the [Visualizing point groups](point_groups.ipynb) user guide for all point
groups supported in `orix`

```python
ori = Rotation.from_axes_angles((0, 1, 0), np.deg2rad(65))
phase.point_group.plot(figure=fig, orientation=ori, fc="none", ec="C0", s=150)
fig
```

## Converting crystal vectors

The reference frame of the stereographic projection above is the Cartesian
reference frame ($\mathbf{e_1}, \mathbf{e_2}, \mathbf{e_3}$). The Cartesian
coordinates $(\mathbf{x}, \mathbf{y}, \mathbf{z})$ of
$(\mathbf{a}, \mathbf{b}, \mathbf{c})$ and
$(\mathbf{a^*}, \mathbf{b^*}, \mathbf{c^*})$ were found using $\mathbf{A}$ in
the following conversions

\begin{align}
(x, y, z) &= [uvw] \cdot \mathbf{A},\\
(x, y, z) &= (hkl) \cdot (\mathbf{A}^{-1})^T.
\end{align}

Let's compute the internal conversions directly and check for equality

```python
np.allclose(abc.data, np.dot(abc.uvw, phase.structure.lattice.base))  # (x, y, z)
```

```python
np.allclose(
    abcr.data,  # (x, y, z)
    np.dot(abcr.hkl, np.linalg.inv(phase.structure.lattice.base).T),
)
```


# Neural Estimation via DV bound

$\def\abs#1{\left\lvert #1 \right\rvert}
\def\Set#1{\left\{ #1 \right\}}
\def\mc#1{\mathcal{#1}}
\def\M#1{\boldsymbol{#1}}
\def\R#1{\mathsf{#1}}
\def\RM#1{\boldsymbol{\mathsf{#1}}}
\def\op#1{\operatorname{#1}}
\def\E{\op{E}}
\def\d{\mathrm{\mathstrut d}}$

Estimating MI well neither requires nor implies the divergence/density to be estimated well. However,
- MI estimation is often not the end goal, but an objective to train a neural network to return the divergence/density.
- The features/representations learned by the neural network may be applicable to different downstream inference tasks.

## Neural estimation of KL divergence

To explain the idea of neural estimation, consider the following characterization of divergence:

---

**Proposition**
:label: DV1

$$
\begin{align}
D(P_{\R{Z}}\|P_{\R{Z}'}) & =  \sup_{Q\in \mc{P}(\mc{Z})} E \left[ \log \frac{dQ(\R{Z})}{dP_{\R{Z}'}(\R{Z})} \right]
\end{align}
$$ (D1)

where the unique optimal solution is $Q=P_{\R{Z}}$.

---

{eq}`D1` is {eq}`D` but with $P_{\R{Z}}$ replaced by a parameter $Q$.

- The proposition essentially gives a tight lower bound on KL divergence.
- The unknown distribution is recovered as the optimal solution.

---

**Proof**

To prove {eq}`D1`,

$$
\begin{align*}
D(P_{\R{Z}}\|P_{\R{Z}'})  &= D(P_{\R{Z}}\|P_{\R{Z}'}) - \inf_{Q\in \mc{P}(\mc{Z})} \underbrace{D(P_{\R{Z}}\|Q)}_{\geq 0 \text{ with equality iff } Q=P_{\R{Z}}\kern-3em} \\
&= \sup_{Q\in \mc{P}(\mc{Z})}  \underbrace{D(P_{\R{Z}}\|P_{\R{Z}'})}_{=E \left[\log  \frac{dP_{\R{Z}}(\R{Z})}{dP_{\R{Z}'}(\R{Z})}\right]} -  \underbrace{D(P_{\R{Z}}\|Q)}_{=E \left[\log \frac{dP_{\R{Z}}(\R{Z})}{dQ(\R{Z})}\right]}\\
&= \sup_{Q\in \mc{P}(\mc{Z})} E \left[\log \frac{dQ(\R{Z})}{dP_{\R{Z}'}(\R{Z})}\right]
\end{align*}
$$

---

The idea of neural estimation is to

- estimate the expectation in {eq}`D1` by the sample average

$$
\frac1n \sum_{i\in [n]} \log \underbrace{\frac{dQ(\R{Z}_i)}{dP_{\R{Z}'}(\R{Z}_i)}}_{\text{(*)}},
$$

- use a neural network to compute the density ratio (*), and train the network to maximizes the expectation, e.g., by gradient ascent on the above sample average.

Since $Q$ is arbitrary, the sample average above is a valid estimate.

**But how to compute the density ratio?**

We will first consider estimating the KL divergence $D(P_{\R{Z}}\|P_{\R{Z}'})$ when both $P_{\R{Z}}$ and $P_{\R{Z}'}$ are unknown.

## Donsker-Varadhan formula

If $P_{\R{Z}'}$ is unknown, we can apply a change of variable

$$
r(z) = \frac{dQ(z)}{dP_{\R{Z}'}(z)},
$$ (Q->r)

which absorbs the unknown reference into the parameter.

---

**Proposition**
:label: DV2

$$
\begin{align}
D(P_{\R{Z}}\|P_{\R{Z}'}) & =  \sup_{\substack{r:\mc{Z}\to \mathbb{R}_+\\ E[r(\R{Z}')]=1}} E \left[ \log r(\R{Z}) \right]
\end{align}
$$ (D2)

where the optimal $r$ satisfies
$
r(\R{Z}) = \frac{dP_{\R{Z}}(\R{Z})}{dP_{\R{Z}'}(\R{Z})}.
$

---

**Exercise**

Show using {eq}`Q->r` that the optimal solution satisfies the constraint stated in the supremum {eq}`D2`.

**Solution**

The constraint on $r$ is obtained from the constraint on $Q\in \mc{P}(\mc{Z})$, i.e., with $dQ(z)=r(z)dP_{\R{Z}'}(z)$,

$$
\begin{align*}
dQ(z) \geq 0 &\iff r(z)\geq 0\\
\int_{\mc{Z}}dQ(z)=1 &\iff E[r(\R{Z}')]=1.
\end{align*}
$$

The next step is to train a neural network that computes $r$. What about?

$$
\begin{align}
D(P_{\R{Z}}\|P_{\R{Z}'}) \approx \sup_{\substack{r:\mc{Z}\to \mathbb{R}_+\\ \frac1{n'}\sum_{i\in [n']} r(\R{Z}'_i)=1}} \frac1n \sum_{i\in [n]} \log r(\R{Z}_i)
\end{align}
$$ (avg-D1)

**How to impose the constraint on $r$ when training a neural network?**

We can apply a change of variable:

$$
\begin{align}
r(z)&=\frac{e^{t(z)}}{E[e^{t(\R{Z}')}]}.
\end{align}
$$ (r->t)

**Exercise**

Show that $r$ defined in {eq}`r->t` satisfies the constraint in {eq}`D1` for all real-valued function $t:\mc{Z}\to \mathbb{R}$.

**Solution**

$$
\begin{align}
E\left[ \frac{e^{t(\R{Z}')}}{E[e^{t(\R{Z}')}]} \right] =  \frac{E\left[ e^{t(\R{Z}')} \right]}{E[e^{t(\R{Z}')}]} = 1.
\end{align}
$$

Substituting {eq}`r->t` into {eq}`D1` gives the well-known *Donsker-Varadhan (DV)* formula {cite}`donsker1983asymptotic`:

---

**Corollary** Donsker-Varadhan
:label: DV3

$$
\begin{align}
D(P_{\R{Z}}\|P_{\R{Z}'}) =  \sup_{t: \mc{Z} \to \mathbb{R}} E[t(\R{Z})] - \log E[e^{t(\R{Z}')}]
\end{align}
$$ (DV)

where the optimal $t$ satisfies

$$
\begin{align}
t(\R{Z}) = \log \frac{dP_{\R{Z}}(\R{Z})}{dP_{\R{Z}'}(\R{Z})} + c
\end{align}
$$ (DV:sol)

almost surely for some constant $c$.

---

The divergence can be estimated as follows instead of {eq}`avg-D1`:

$$
\begin{align}
D(P_{\R{Z}}\|P_{\R{Z}'}) \approx \sup_{t: \mc{Z} \to \mathbb{R}} \frac1n \sum_{i\in [n]} t(\R{Z}_i) - \frac1{n'}\sum_{i\in [n']} e^{t(\R{Z}'_i)}
\end{align}
$$ (avg-DV)

In summary, the neural estimation of KL divergence is a sample average of {eq}`D` but

$$
D(P_{\R{Z}}\| P_{\R{Z}'}) = \underset{\stackrel{\uparrow}\sup_t}{} \overbrace{E}^{\op{avg}} \bigg[ \log \underbrace{\frac{P_{\R{Z}}(\R{Z})}{P_{\R{Z}'}(\R{Z})}}_{\frac{e^{t(\R{Z})}}{\underbrace{E}_{\op{avg}}[e^{t(\R{Z}')}]}} \bigg].
$$

but with the unknown density ratio replaced by {eq}`r->t` trained as a neural network.


# 2-level system with oscillating detuning

## Imports
Start by importing the necessary packages

```python
%load_ext autoreload
%autoreload 2

import joblib
import matplotlib.pyplot as plt
from matplotlib import animation
from mpl_toolkits.mplot3d import Axes3D
plt.style.use("ggplot")
import numpy as np
import qutip
from sympy import Symbol

from toy_systems.couplings import ToyCoupling, ToyEnergy
from toy_systems.decays import ToyDecay
from toy_systems.hamiltonian import Hamiltonian
from toy_systems.quantum_system import QuantumSystem
from toy_systems.states import Basis, BasisState, ToyQuantumNumbers
```

## Define states
We start by defining the two states of the system: the ground state $|g\rangle$ and the excited $|e\rangle$.

```python
g = BasisState(qn=ToyQuantumNumbers(label="g"))
e = BasisState(qn=ToyQuantumNumbers(label="e"))

# Print the states as a check
print(f"|g> = {g}")
print(f"|e> = {e}")

# Check that the states are orthogonal by taking inner product
print(f"<e|g> = {e @ g}")
```

    |g> = |g>
    |e> = |e>
    <e|g> = 0

## Define basis
We use the two orthogonal states to define a basis as shown below. The indexing of the matrix and vector representations of operators is determined by the order of the states when generating the state

```python
basis = Basis((g,e))
basis.print()
```

    |0> = |g>
    |1> = |e>

## Define energies
I'm going to define the system in the rotating frame so that the Hamiltonian doesn't have any quickly rotating terms of the form $e^{i\omega t}$. The detuning between the driving field and the natural frequency of the system is taken to be $ \Delta(t) = \Delta_0 + \frac{A}{2}\sin(\omega t) $.

```python
E_g = ToyEnergy([g], 1/2, time_dep = "Delta0/2 +  A/2 * sin(omega*t)", time_args = {"Delta0":0, "omega":5, "A":1})
E_e = ToyEnergy([e], 1/2,  time_dep = "-Delta0/2 - A/2 * sin(omega*t)", time_args = {"Delta0":0, "omega":5, "A":1})
```

### Define coupling
Take the coupling field between the two states to be constant

```python
Ω = Symbol('Omega')
coupling = ToyCoupling(g, e, Ω/2)
```

## Define Hamiltonian
Now that we have the states, energies and couplings, we can define the Hamiltonian

```python
H = Hamiltonian((E_g, E_e, coupling), basis)
display(H)
```

    H =
    [
    [0.5, 0.5],
    [0.5, 0.5]]

    in basis Basis: name = None

## Define a QuantumSystem
The QuantumSystem object combines the basis, decays, and Hamiltonian to make setting parameters for time evolution using QuTiP more convenient.

```python
# Define the system
system = QuantumSystem(
    basis=basis,
    couplings=[E_g, E_e, coupling],
    decays = []
)

# Get representations of the Hamiltonian and the decays that will be accepted by qutip
Hqobj, c_qobj = system.get_qobjs()
```

## Time-evolution using `QuTiP`
The `toy_systems` package is designed to take care of generating the Hamiltonian and collapse operators for the quantum system, and spit out a `qutip.QobjEVo`-object. The actual time-evolution is then taken care of by `qutip` (or something else if the user so desires).

```python
# Get a pointer to the time-evolution arguments
args = Hqobj.args
print("Keys for setting arguments:")
print(f"args = {args}")
```

    Keys for setting arguments:
    args = {'Delta0': 0, 'omega': 5, 'A': 1, 'Omega': 1}

```python
# Generate a Qobj representing the initial state
psi0 = (1*g).qobj(basis)

# Make operators for getting the probability of being in each state
P_g_op = qutip.Qobj((1*g).density_matrix(basis), type = "oper")
P_e_op = qutip.Qobj((1*e).density_matrix(basis), type = "oper")

# Pauli matrices for calculating components of Bloch vector
sx = qutip.sigmax()
sy = qutip.sigmay()
sz = qutip.sigmaz()

# Set the parameters for the system
args["Omega"] = 1 # The maximum value of the drive field Rabi rate (set to one so can use 1/Omega as unit of time)
args["Delta0"] = 0 # Constant part of detuning
args["A"] = 10 # Amplitude of detuning oscillations
args["omega"] = 5 # Frequency of detuning oscillations

# Times at which result is requested
times =  np.linspace(-10/np.abs(args["omega"]), 10/np.abs(args["omega"]), 3001) * 2* np.pi

# The solver used by qutip sometimes doesn't seem to notice that adiabatic passage is supposed to be happening, so reduce max step size to
# mitigate this
options = qutip.solver.Options(method = 'adams', nsteps=100000, max_step=1e0/np.abs(args["omega"]),)

# Setup a progress bar
pb = qutip.ui.progressbar.EnhancedTextProgressBar()

# Run the time-evolution
result = qutip.mesolve(Hqobj, psi0, times, c_ops = c_qobj, e_ops = [P_g_op, P_e_op, sx, sy, sz],
                      progress_bar=pb, options = options)
```

     Total run time:   1.68s*] Elapsed 1.68s / Remaining 00:00:00:00[******   25%           ] Elapsed 0.42s / Remaining 00:00:00:01[*********57%*          ] Elapsed 0.98s / Remaining 00:00:00:00[*********87%********   ] Elapsed 1.47s / Remaining 00:00:00:00

Plot the result:

```python
fig, (ax,ax2) = plt.subplots(2,1, figsize = (16,9))
ax.plot(times, result.expect[0], label = "P_g")
ax.plot(times, result.expect[1], label = "P_e")
ax.legend()
ax.set_xlabel("Time / (1/Ω)")
ax.set_ylabel("Population in each state")

# Plot the coupling strength and detuning also
ax2c = ax2.twinx()
ln1 = ax2.plot(times, E_e.eval(times, args) - E_g.eval(times, args) , label = "Δ")
ax2.set_ylabel('Detuning / Omega')
ln2 = coupling.plot_time_dep(times, args, ax = ax2c, label = 'Ω', c = 'tab:blue')
ax2c.grid(False)
ax2c.set_ylabel('')
ax2.legend(ln1+ln2, [l.get_label() for l in (ln1 + ln2)])
ax2c.set_ylabel('Rabi rate')
```

### Visualizing on Bloch sphere

```python
# Calculate components of Bloch vector
u = -result.expect[2]
v = result.expect[3]
w = result.expect[4]

# Calculate detuning and coupling strength
deltas = E_e.eval(times, time_args=args) - E_g.eval(times, time_args=args)
Omegas = coupling.eval(times, time_args=args)
W_length = np.sqrt(deltas**2 + Omegas**2)

# Set up the bloch sphere
fig = plt.figure(figsize = (16, 9))
ax0 = fig.add_subplot(221, projection = '3d')
ax0.set_title('Bloch sphere', fontsize = 18)
b = qutip.Bloch(fig = fig, axes = ax0)
b.frame_width = 1
b.figsize = (16,16)

# Add initial Bloch vector and Rabi vector
b.add_vectors([np.array([u[0],v[0],w[0]]), np.array([Omegas[0], 0, deltas[0]])])
b.make_sphere()

# Set up plot of detuning
ax1 = fig.add_subplot(222)
ax1.plot(times, deltas, label = "Δ")
ax1.set_ylabel('Detuning', fontsize = 16)
delta_marker = ax1.plot(times[0], deltas[0], marker = 'o', markersize=8, c = 'k', fillstyle='none')

# Set up plot of Rabi rate
ax2 = fig.add_subplot(224)
ax2.plot(times, Omegas, label = "Ω", c = 'C1')
ax2.set_ylabel('Rabi rate', fontsize = 16)
ax2.set_xlabel('Time')
rabi_marker = ax2.plot(times[0], Omegas[0], marker = 'o', markersize=8, c = 'k', fillstyle='none')

# Set up plot of populations
ax3 = fig.add_subplot(223)
ax3.plot(times, result.expect[0], label = r"P$_0$")
ax3.plot(times, result.expect[1], label = r"P$_1$")
ax3.set_ylabel('Population in each state', fontsize = 16)
vline = ax3.axvline(times[0], ls = '--', c = 'k')
ax3.set_xlabel('Time')
ax3.legend()

# Define function that generates frames for animation
def animate(i):
    b.clear()
    b.add_vectors([np.array([u[i],v[i],w[i]]), np.array([Omegas[i], 0, deltas[i]])/W_length[i]])
    i_start = np.max([0, i-100])
    b.add_points([u[i_start:i+1],v[i_start:i+1],w[i_start:i+1]])
    b.make_sphere()

    ax1.lines[1].remove()
    ax1.plot(times[i], deltas[i], marker = 'o', markersize=8, c = 'k', fillstyle='none')

    ax2.lines[1].remove()
    ax2.plot(times[i], Omegas[i], marker = 'o', markersize=8, c = 'k', fillstyle='none')

    ax3.lines[2].remove()
    ax3.axvline(times[i], ls = '--', c = 'k')

    return fig, (ax0, ax1, ax2, ax3)
```

```python
fig, _ = animate(200)
fig
```

```python
# %%timeit
ani = animation.FuncAnimation(fig, animate, range(0,len(times),3), blit=False, repeat=False)
ani.save(f"oscillating_detuning_Delta0={args['Delta0']:.1f}_A={args['A']:.1f}_omega={args['omega']:.1f}.mp4", fps = 60)
```

examples/Two-level system with oscillating detuning.ipynb


### by nick brønn

### Jaynes-Cummings Hamiltonian

For qubits, we can describe the dynamics of the qubit with the **Pauli Matrices**

$$
\sigma^x = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \qquad
\sigma^y = \begin{pmatrix} 0 & -i \\ i & 0 \end{pmatrix} \qquad
\sigma^z = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \qquad
$$

The Pauli matrices let us define raising and lowering operators

$$ \sigma^+ = \frac{1}{2}( \sigma^x - i\sigma^y) \qquad {\rm and}
\qquad \sigma^- = \frac{1}{2}( \sigma^x + i\sigma^y)$$

They raise and lower qubit states

$$ \sigma^+ |0\rangle = |1\rangle \qquad \sigma^+ |1\rangle = 0
\qquad {\rm and} \qquad
\sigma^-|1\rangle = |0\rangle \qquad \sigma^-|0\rangle = 0$$

The interaction between qubit and resonator is descibed by the **Jaynes-Cummings Hamiltonian**

$$
H_{\rm JC}^{\rm (RWA)}/\hbar = \underbrace{\omega_r a^\dagger a}_\text{resonator} - \overbrace{\frac{1}{2} \omega_q \sigma_z}^\text{qubit} + \underbrace{g(a^\dagger \sigma^- + a \sigma^+)}_\text{dipole interaction}.
$$

In order to understand it, we must *block-diagonalize* it using the *Schrieffer-Wolff transformation*.

## The Schrieffer-Wolff Transformation

You have a diagonalized Hamiltonian plus a perturbation
$$
H \quad = \quad \underbrace{\begin{pmatrix}
\Box &  &  &  &  &  &  \\
 & \Box &  &  &  &  &  \\
 &  & \Box &  &  &  &  \\
 &  &  & \Box &  &  &  \\
 &  &  &  & \Box &  &  \\
 &  &  &  &  & \Box &  \\
 &  &  &  &  &  & \Box \end{pmatrix}}_\text{diagonal} \quad + \quad
\underbrace{\begin{pmatrix}
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\times & \times & \times & \times & \cdot & \cdot & \cdot \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \\
\cdot & \cdot & \cdot & \cdot & \times & \times & \times \end{pmatrix}}_\text{perturbation}
$$

Split the perturbation into *block diagonal* and *block off-diagonal* parts so that $H = H_0 + H_1 + H_2$
$$
H \quad = \quad \underbrace{\begin{pmatrix}
\Box &  &  &  &  &  &  \\
 & \Box &  &  &  &  &  \\
 &  & \Box &  &  &  &  \\
 &  &  & \Box &  &  &  \\
 &  &  &  & \Box &  &  \\
 &  &  &  &  & \Box &  \\
 &  &  &  &  &  & \Box \end{pmatrix}}_\text{diagonal}
\quad + \quad
\underbrace{\begin{pmatrix}
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
\times & \times & \times & \times &  &  &  \\
 &  &  &  & \times & \times & \times \\
 &  &  &  & \times & \times & \times \\
 &  &  &  & \times & \times & \times \end{pmatrix}}_\text{block diagonal}
\quad + \quad
\underbrace{\begin{pmatrix}
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
 &  &  &  & \cdot & \cdot & \cdot \\
\cdot & \cdot & \cdot & \cdot &  &  &  \\
\cdot & \cdot & \cdot & \cdot &  &  &  \\
\cdot & \cdot & \cdot & \cdot &  &  &  \end{pmatrix}}_\text{block non-diagonal}
$$

## The Schrieffer-Wolff Transformation

We seek an anti-hermitian, block off-diagonal operator $S$ such that

$$
H_{\rm eff} = e^{iS} H e^{-iS} = \sum_{m=0}^\infty \frac{1}{m!} \left[H, S\right]^{(m)} \qquad {\rm with} \qquad \left[H,S\right]^{(m)} = \left[\left[H,S\right]^{(m-1)},S\right] \quad {\rm and} \quad \left[H,S\right]^{(0)} = H
$$

is the generalized commutator

Taking $H_1+H_2$ as a perturbation on the diagonal $H_0$ and $S$ as a Taylor expansion

$$ H = H_0 + \lambda(H_1 + H_2) \qquad S = \sum_{m=1}^\infty \lambda^m S^{(m)} $$

to keep track of the order $\lambda$

Then

$$
H_{\rm eff} = H_0 + \lambda (H_1+H_2) + \left[H_0 + \lambda(H_1+H_2), \lambda S^{(1)}\right]
  + \frac{1}{2} \left[ \left[ H_0 + \lambda(H_1+H_2), \lambda S^{(1)}\right], \lambda S^{(1)}\right]
  + \left[H_0 + \lambda(H_1+H_2), \lambda^2 S^{(2)}\right] + \ldots \\
  \approx H_0 + \lambda \left( H_1 + H_2 + \left[H_0, S^{(1)}\right] \right)
  + \lambda^2 \left( \left[H_1+H_2, S^{(1)}\right] + \frac{1}{2} \left[ \left[H_0, S^{(1)}\right], S^{(1)}\right] + \left[H_0, S^{(2)}\right]\right)
$$

up to second order in $\lambda$

## The Schrieffer-Wolff Transformation

The transformation eliminates the block off-diagonal part of $H_{\rm eff}$,

$$
H_{\rm eff}^{\rm off-diag} = \sum_{m=0}^\infty \frac{1}{(2m+1)!} [H_0 + H_1, S]^{(2m+1)} + \sum_{m=0}^\infty \frac{1}{(2m)!} [H_2, S]^{(2m)} \equiv 0,
$$

to each order of $\lambda$

Expanding

$$
H_{\rm eff}^{\rm off-diag} = \left[ H_0 + \lambda H_1, \lambda S^{(1)} \right]+\lambda H_2 + \left[H_0 + \lambda H_1, \lambda^2 S^{(2)}\right]
  + \frac{1}{3!} \left[ H_0+\lambda H_1, \lambda S^{(1)}\right]^{(3)} + \frac{1}{2!} \left[ \lambda H_2, \lambda S^{(1)}\right]^{(2)} \\
  = \lambda \left( \left[ H_0, S^{(1)} \right] + H_2 \right) + \lambda^2 \left( \left[H_1, S^{(1)} \right] + \left[H_0, S^{(2)}\right]\right) + \ldots
$$

Going to second order yields

$$
\left[ H_0, S^{(1)} \right] = -H_2 \qquad \left[ H_0, S^{(2)} \right] = -\left[H_1, S^{(1)}\right]
$$

## The Schrieffer-Wolff Transformation
$$
H_{\rm eff} = H_0 + \lambda \left( H_1 + H_2 + \left[H_0, S^{(1)}\right] \right)
  + \lambda^2 \left( \left[H_1+H_2, S^{(1)}\right] + \frac{1}{2} \left[ \left[H_0, S^{(1)},\right] S^{(1)}\right] + \left[H_0, S^{(2)}\right]\right)
$$

Substituting the equations needed to cancel the block off-diagonal terms and letting $\lambda \to 1$

$$
H_{\rm eff} = H_0 + H_1 + \frac{1}{2} \left[ H_2, S^{(1)} \right]
$$

Now we only need to find $S^{(1)}$ for a second-order correction!

The recent manuscript [A Systematic Method for Schrieffer-Wolff Transformation and Its Generalizations](http://www.arxiv.org/abs/2004.06534) systematically provides the ansatz by calculating

$$\eta = [H_0, H_2] $$

and set the coefficients such that

$$ [H_0, \eta] = H_2$$

which means $\eta$ is the same as $S^{(1)}$ by uniqueness

## S-W Tranformation of the J-C Hamiltonain
For ease of tedious calculations, we will use the Python package [`sympy`](http://www.sympy.org) for symbolic mathematics.

```python
# import SymPy and define symbols
import sympy as sp
sp.init_printing(use_unicode=True)
wr = sp.Symbol('\omega_r') # resonator frequency
wq = sp.Symbol('\omega_q') # qubit frequency
g = sp.Symbol('g', real=True) # vacuum Rabi coupling
Delta = sp.Symbol('Delta', real=True) # wr - wq; defined later
```

```python
# import operator relations and define them
from sympy.physics.quantum.boson import BosonOp
a = BosonOp('a') # resonator photon annihilation operator
from sympy.physics.quantum import pauli, Dagger, Commutator
from sympy.physics.quantum.operatorordering import normal_ordered_form
```

## S-W Tranformation of the J-C Hamiltonian

```python
# Pauli matrices
sx = pauli.SigmaX()
sy = pauli.SigmaY()
sz = pauli.SigmaZ()

# qubit raising and lowering operators
splus = pauli.SigmaPlus()
sminus = pauli.SigmaMinus()
```

```python
# define J-C Hamiltonian in terms of diagonal and non-block diagonal terms
H0 = wr*Dagger(a)*a - (1/2)*wq*sz;
H1 = 0
H2 = g*(Dagger(a)*sminus + a*splus);
HJC = H0 + H1 + H2; HJC # print
```

## S-W Tranformation of the J-C Hamiltonian

```python
# using the above method for finding the ansatz
eta = Commutator(H0, H2); eta
```

We will need to used the methods `doit()`, `expand`, `normal_ordered_form`, and `qsimplify_pauli`

```python
pauli.qsimplify_pauli(normal_ordered_form(eta.doit().expand()))
```

## S-W Tranformation of the J-C Hamiltonian

Now take $A$ and $B$ as the coefficients of $a^\dagger \sigma_-$ and $a\sigma_+$, respectively. Then the commutator

```python
A = sp.Symbol('A')
B = sp.Symbol('B')
eta = A * Dagger(a) * sminus - B * a * splus;
pauli.qsimplify_pauli(normal_ordered_form(Commutator(H0, eta).doit().expand()))
```

This expression should be equal to $H_2$

```python
H2
```

## S-W Tranformation of the J-C Hamiltonian

Therefore set $A = B = g/\Delta$ where $\Delta = \omega_r - \omega_q$ is the frequency detuning between the resonator and qubit. Then $S^{(1)}$ is determined as

```python
S1 = eta.subs(A, g/Delta)
S1 = S1.subs(B, g/Delta); S1.factor()
```

Then we can calculate the effective second order correction to $H_0$

```python
Heff = H0 + H1 + 0.5*pauli.qsimplify_pauli(normal_ordered_form(Commutator(H2, S1).doit().expand())).simplify(); Heff
```

This is typically written as
$$
H_{\rm eff} = \left(\omega_r + \frac{g^2}{\Delta}\sigma_z\right)a^\dagger a  - \frac{1}{2}\left(\omega_q -\frac{g^2}{\Delta}\right) \sigma_z
$$
which shows a state-dependent shift by $\chi \equiv g^2/\Delta$ of the resonator frequency called the *ac Stark shift* and a shift in qubit frequency due to quantum vacuum fluctuations called the *Lamb shift*.

## Qiskit Pulse - Spectroscopy

### Import Necessary Libraries

```python
from qiskit.tools.jupyter import *
from qiskit import IBMQ
IBMQ.load_account()
provider = IBMQ.get_provider(hub='ibm-q', group='open', project='main')
backend = provider.get_backend('ibmq_armonk')
```

### Verify Backend is Pulse-enabled

```python
backend_config = backend.configuration()
assert backend_config.open_pulse, "Backend doesn't support Pulse"

dt = backend_config.dt
print(f"Sampling time: {dt*1e9} ns")
backend_defaults = backend.defaults()
```

    Sampling time: 0.2222222222222222 ns

## Qiskit Pulse - Spectroscopy

```python
import numpy as np

# unit conversion factors -> all backend properties returned in SI (Hz, sec, etc)
GHz = 1.0e9 # Gigahertz
MHz = 1.0e6 # Megahertz
kHz = 1.0e3
us = 1.0e-6 # Microseconds
ns = 1.0e-9 # Nanoseconds

# We will find the qubit frequency for the following qubit.
qubit = 0

# The sweep will be centered around the estimated qubit frequency.
center_frequency_Hz = backend_defaults.qubit_freq_est[qubit]        # The default frequency is given in Hz
                                                                    # warning: this will change in a future release
print(f"Qubit {qubit} has an estimated frequency of {center_frequency_Hz / GHz} GHz.")
```

    Qubit 0 has an estimated frequency of 4.974454425671347 GHz.

## Qiskit Pulse - Spectroscopy

```python
# scale factor to remove factors of 10 from the data
scale_factor = 1e-14
# We will sweep 40 MHz around the estimated frequency
frequency_span_Hz = 40 * MHz
# in steps of 1 MHz.
frequency_step_Hz = 1 * MHz
# We will sweep 20 MHz above and 20 MHz below the estimated frequency
frequency_min = center_frequency_Hz - frequency_span_Hz / 2
frequency_max = center_frequency_Hz + frequency_span_Hz / 2
# Construct an np array of the frequencies for our experiment
frequencies_GHz = np.arange(frequency_min / GHz,
                            frequency_max / GHz,
                            frequency_step_Hz / GHz)

print(f"The sweep will go from {frequency_min / GHz} GHz to {frequency_max / GHz} GHz \
in steps of {frequency_step_Hz / MHz} MHz.")
```

    The sweep will go from 4.954454425671346 GHz to 4.994454425671346 GHz in steps of 1.0 MHz.

## Qiskit Pulse - Spectroscopy

```python
from qiskit import pulse            # This is where we access all of our Pulse features!

inst_sched_map = backend_defaults.instruction_schedule_map
measure = inst_sched_map.get('measure', qubits=[qubit])
x_pulse = inst_sched_map.get('x', qubits=[qubit])

### Collect the necessary channels
drive_chan = pulse.DriveChannel(qubit)
meas_chan = pulse.MeasureChannel(qubit)
acq_chan = pulse.AcquireChannel(qubit)
```

```python
# Create the base schedule
# Start with drive pulse acting on the drive channel
schedule = pulse.Schedule(name='Frequency sweep')
schedule += x_pulse
# The left shift `<<` is special syntax meaning to shift the start time of the schedule by some duration
schedule += measure << schedule.duration

# Create the frequency settings for the sweep (MUST BE IN HZ)
frequencies_Hz = frequencies_GHz*GHz
schedule_frequencies = [{drive_chan: freq} for freq in frequencies_Hz]
```

## Qiskit Pulse - Spectroscopy

```python
schedule.draw(label=True, scaling=0.8)
```

## Qiskit Pulse - Spectroscopy

```python
from qiskit import assemble
frequency_sweep_program = assemble(schedule,
                                   backend=backend,
                                   meas_level=1,
                                   meas_return='avg',
                                   shots=1024,
                                   schedule_los=schedule_frequencies)
```

```python
# RUN the job on a real device
#job = backend.run(rabi_experiment_program)
#print(job.job_id())
#from qiskit.tools.monitor import job_monitor
#job_monitor(job)

# OR retreive result from previous run
job = backend.retrieve_job('5ef3b081fbc24b001275b03b')
```

## Qiskit Pulse - Spectroscopy

```python
frequency_sweep_results = job.result()
```

```python
import matplotlib.pyplot as plt
plt.style.use('dark_background')

sweep_values = []
for i in range(len(frequency_sweep_results.results)):
    # Get the results from the ith experiment
    res = frequency_sweep_results.get_memory(i)*scale_factor
    # Get the results for `qubit` from this experiment
    sweep_values.append(res[qubit])

plt.scatter(frequencies_GHz, np.real(sweep_values), color='white') # plot real part of sweep values
plt.xlim([min(frequencies_GHz), max(frequencies_GHz)])
plt.xlabel("Frequency [GHz]")
plt.ylabel("Measured signal [a.u.]")
plt.show()
```

## Qiskit Pulse - Spectroscopy

```python
from scipy.optimize import curve_fit

def fit_function(x_values, y_values, function, init_params):
    fitparams, conv = curve_fit(function, x_values, y_values, init_params)
    y_fit = function(x_values, *fitparams)

    return fitparams, y_fit
```

```python
fit_params, y_fit = fit_function(frequencies_GHz,
                                 np.real(sweep_values),
                                 lambda x, A, q_freq, B, C: (A / np.pi) * (B / ((x - q_freq)**2 + B**2)) + C,
                                 [5, 4.975, 1, 3] # initial parameters for curve_fit
                                )
```

## Qiskit Pulse - Spectroscopy

$$
H_{\rm eff} = \left(\omega_r + \frac{g^2}{\Delta}\sigma_z\right)a^\dagger a  - \frac{1}{2}\left(\omega_q -\frac{g^2}{\Delta}\right) \sigma_z
$$

```python
plt.scatter(frequencies_GHz, np.real(sweep_values), color='white')
plt.plot(frequencies_GHz, y_fit, color='red')
plt.xlim([min(frequencies_GHz), max(frequencies_GHz)])

plt.xlabel("Frequency [GHz]")
plt.ylabel("Measured Signal [a.u.]")
plt.show()
```

## Qiskit Pulse - Dispersive Shift

```python
 # Create the schedules for 0 and 1
schedule_0 = pulse.Schedule(name='0')
schedule_0 += measure

schedule_1 = pulse.Schedule(name='1')
schedule_1 += x_pulse
schedule_1 += measure << schedule_1.duration
```

## Qiskit Pulse - Dispersive Shift

```python
schedule_0.draw()
```

## Qiskit Pulse - Dispersive Shift

```python
schedule_1.draw()
```

## Qiskit Pulse - Dispersive Shift

```python
frequency_span_Hz = 320 * kHz
frequency_step_Hz = 8 * kHz

center_frequency_Hz = backend_defaults.meas_freq_est[qubit]
print(f"Qubit {qubit} has an estimated readout frequency of {center_frequency_Hz / GHz} GHz.")
frequency_min = center_frequency_Hz - frequency_span_Hz / 2
frequency_max = center_frequency_Hz + frequency_span_Hz / 2
frequencies_GHz = np.arange(frequency_min / GHz,
                            frequency_max / GHz,
                            frequency_step_Hz / GHz)

print(f"The sweep will go from {frequency_min / GHz} GHz to {frequency_max / GHz} GHz\
      in steps of {frequency_step_Hz / MHz} MHz.")
```

## Qiskit Pulse - Dispersive Shift

```python
num_shots_per_frequency = 2048

frequencies_Hz = frequencies_GHz*GHz
schedule_los = [{meas_chan: freq} for freq in frequencies_Hz]

cavity_sweep_0 = assemble(schedule_0,
                               backend=backend,
                               meas_level=1,
                               meas_return='avg',
                               shots=num_shots_per_frequency,
                               schedule_los=schedule_los)

cavity_sweep_1 = assemble(schedule_1,
                               backend=backend,
                               meas_level=1,
                               meas_return='avg',
                               shots=num_shots_per_frequency,
                               schedule_los=schedule_los)
```

```python
# RUN the job on a real device
#job_0 = backend.run(cavity_sweep_0)
#job_monitor(job_0)
#job_0.error_message()

#job_1 = backend.run(cavity_sweep_1)
#job_monitor(job_1)
#job_1.error_message()

# OR retreive result from previous run
job_0 = backend.retrieve_job('5efa5b447c0d6800137fff1c')
job_1 = backend.retrieve_job('5efa6b2720eee10013be46b4')

cavity_sweep_0_results = job_0.result()
cavity_sweep_1_results = job_1.result()
```

## Qiskit Pulse - Dispersive Shift

```python
scale_factor = 1e-14

sweep_values_0 = []
for i in range(len(cavity_sweep_0_results.results)):
    res_0 = cavity_sweep_0_results.get_memory(i)*scale_factor
    sweep_values_0.append(res_0[qubit])

sweep_values_1 = []
for i in range(len(cavity_sweep_1_results.results)):
    res_1 = cavity_sweep_1_results.get_memory(i)*scale_factor
    sweep_values_1.append(res_1[qubit])

plotx = frequencies_Hz/kHz
ploty_0 = np.abs(sweep_values_0)
ploty_1 = np.abs(sweep_values_1)
```

## Qiskit Pulse - Dispersive Shift

$$
H_{\rm eff} = \left(\omega_r + \frac{g^2}{\Delta}\sigma_z\right)a^\dagger a  - \frac{1}{2}\left(\omega_q -\frac{g^2}{\Delta}\right) \sigma_z
$$

```python
plt.plot(plotx, ploty_0, color='blue', marker='.') # plot real part of sweep values
plt.plot(plotx, ploty_1, color='red', marker='.') # plot real part of sweep values
plt.legend([r'$\vert0\rangle$', r'$\vert1\rangle$'])

plt.grid()
plt.xlabel("Frequency [kHz]")
plt.ylabel("Measured signal [a.u.]")
plt.yscale('log')
plt.show()
```

# <center> On to Exercise 1... </center>


# べき乗測変換

\begin{equation}
  s = T(r) = (L-1)\left(\frac{r}{L-1}\right)^\gamma
\end{equation}

## 画像ファイルの準備
以下では、画像ファイルを手元からアップロードする方法とURLを指定してアクセスする方法のいずれかを利用してください。
### アップロードする方法

```
#from google.colab import files
#uploaded_file = files.upload()
```

```
#file_name = next(iter(uploaded_file))
#print(file_name)
```

### URLを指定してアクセスする方法
- 参照サイト：　http://homepages.cae.wisc.edu/~ece533/images/

```
import urllib.request
import io

url = 'http://homepages.cae.wisc.edu/~ece533/images/lena.png'
file_name = io.BytesIO(urllib.request.urlopen(url).read())
```

## 画像の読み込み
画像処理ライブラリ Scikit-image を利用

```
from skimage import io
img = io.imread(file_name)
img.dtype
```

    dtype('uint8')

## 画像の表示
可視化ライブラリMatplotlibを利用

```
from matplotlib import pyplot as plt

plt.imshow(img)
plt.axis('off')
```

## グレースケール画像に変換

```
from skimage.color import rgb2gray

gray_img = rgb2gray(img)
display(gray_img.dtype)
gray_img.max()
```

    dtype('float64')

    0.9654356862745097

## グレースケール画像の表示
PIL画像をNumPy配列(float32)に変換し，[0,1]の範囲に正規化して表示

```
import numpy as np

plt.imshow(gray_img, cmap='gray')
plt.axis('off')
```

## べき乗測変換のグラフプロット
パラメータ$\gamma$を設定して関数をプロット

```
gamma = 0.5

r = np.linspace(0,1,100)
s = np.power(r,gamma)
plt.plot(r,s)
plt.axis('square')
plt.xlabel('r')
plt.ylabel('s')
plt.show()
```

## 画像のべき乗測変換
NumPyライブラリの power 関数を利用

```
res_img = np.power(gray_img,gamma)
```

## 処理結果の表示

```
plt.imshow(res_img,cmap='gray')
plt.axis('off')
```

# 符号なし整数8ビットに変換

```
res_img_uint8 = np.uint8(255.0*res_img)
display(res_img_uint8.dtype)
res_img_uint8.max()
```

    dtype('uint8')

    250


```python
from sympy import *
from sympy.abc import *
from sympy.galgebra.ga import *
import numpy as np
from numpy import linalg as LA
from __future__ import print_function
init_printing()
```

# Operational intensity of  differential operation

We consder  differential operation on a vector $u$ at a given point in 3D with a 1D stencil size $k$ (number of points in the stencil) for every order, the subindex $i$ represent the dimension number $1$ for z, $2$ for $x$ and 3 for $y$,

First order :
$
\frac{d u}{dx_i}
$

Second order :
$
\frac{d^2 u}{dx_i^2}
$

Second order cross derivative
$
\frac{d^2 u}{dx_i dx_j}
$

```python
# Arithmetic operations
k = symbols('k')
s = symbols('s')
# 1D stencil
#                multiplication         addition
AI_dxi   =              k            +    k - 1
AI_dxxi  =           k + 1           +    k - 1
AI_dxxij =    (k+1)**2 - 2*k + 1 + 1 +   (k+1)**2 - 2*k + 1 -1
```

```python
# I/O operations
#                    load
IO_dxi   =           k
IO_dxxi  =           k
IO_dxxij =    (k+1)**2 - 2*k + 1
```

```python
# Operational intensity in sing;e precision

print(AI_dxi/(4*IO_dxi))
print(AI_dxxi/(4*IO_dxxi))
print(AI_dxxij/(4*IO_dxxij))

OI_dxi   = lambdify(k,AI_dxi/(4*IO_dxi))
OI_dxxi  = lambdify(k,AI_dxxi/(4*IO_dxxi))
OI_dxxij = lambdify(k,AI_dxxij/(4*IO_dxxij))
```

    (2*k - 1)/(4*k)
    1/2
    (-4*k + 2*(k + 1)**2 + 2)/(-8*k + 4*(k + 1)**2 + 4)

# Operational intensity of  wave equations

We now consider geophysical wave equations to obtain the theoretical expression of the operational intensity. We write directly the expression of a single time step as a function of differential operators. An operation on a wavefield is counted only once as we consider the minimum of arithmetic operations required.

## Acoustic isotropic

$ u(x,y,z,t+dt) = dt^2 v^2(x,y,z) ( 2 u(x,y,z,t) + u(x,y,z,t-dt) + \nabla^2 u(x,y,z,t) +q )  $

## VTI
$ p(x,y,z,t+dt) = dt^2 v^2(x,y,z) \left( 2 p(x,y,z,t) + p(x,y,z,t-dt) +(1+2\epsilon)(\frac{d^2 p(x,t)}{dx^2}+ \frac{d^2 p(x,t)}{dy^2}) + \sqrt{(1+2\delta)} \frac{d^2 r(x,t)}{dz^2} + q \right)  $
$ r(x,y,z,t+dt) = dt^2 v^2(x,y,z) \left( 2 r(x,y,z,t) + r(x,y,z,t-dt) +\sqrt{(1+2\delta)}(\frac{d^2 p(x,t)}{dx^2}+ \frac{d^2 p(x,t)}{dy^2}) +  \frac{d^2 r(x,t)}{dz^2} + q \right)   $

## TTI
$  p(x,y,z,t+dt) = dt^2 v^2(x,y,z) \left( 2 p(x,y,z,t) + p(x,y,z,t-dt) + (1+2\epsilon) (G_{\bar{x}\bar{x}} + G_{\bar{y}\bar{y}}) p(x,y,z,t) + \sqrt{(1+2\delta)} G_{\bar{z}\bar{z}} r(x,y,z,t) + q \right)  $
$  r(x,y,z,t+dt) = dt^2 v^2(x,y,z) \left( 2 r(x,y,z,t) + r(x,y,z,t-dt) + \sqrt{(1+2\delta)}(G_{\bar{x}\bar{x}} + G_{\bar{y}\bar{y}}) p(x,y,z,t) + G_{\bar{z}\bar{z}} r(x,y,z) +q  \right)  $

where
$
\begin{cases}
G_{\bar{x}\bar{x}} & = cos(\phi)^2 cos(\theta)^2 \frac{d^2}{dx^2} +sin(\phi)^2 cos(\theta)^2 \frac{d^2}{dy^2}+ sin(\theta)^2 \frac{d^2}{dz^2} + sin(2\phi) cos(\theta)^2 \frac{d^2}{dx dy} - sin(\phi) sin(2\theta) \frac{d^2}{dy dz} -cos(\phi) sin(2\theta) \frac{d^2}{dx dz} \\
G_{\bar{y}\bar{y}} & =  sin(\phi)^2 \frac{d^2}{dx^2} +cos(\phi)^2  \frac{d^2}{dy^2} - sin(2\phi)^2 \frac{d^2}{dx dy}\\
G_{\bar{z}\bar{z}} & = cos(\phi)^2 sin(\theta)^2 \frac{d^2}{dx^2} +sin(\phi)^2 sin(\theta)^2 \frac{d^2}{dy^2}+ cos(\theta)^2 \frac{d^2}{dz^2} + sin(2\phi) sin(\theta)^2 \frac{d^2}{dx dy} + sin(\phi) sin(2\theta) \frac{d^2}{dy dz} +cos(\phi) sin(2\theta) \frac{d^2}{dx dz} \\
\end{cases}
$

```python
# Arithmetic
#                    dxi             dxxi             dxxij        multiplications     additions   duplicates
AI_acou =        0*AI_dxi    +    3*AI_dxxi   +    0*AI_dxxij    +        3          +     5     -  2 * 2
AI_vti  = 2 * (  0*AI_dxi    +    3*AI_dxxi   +    0*AI_dxxij    +        5          +     5     -  2    )
AI_tti  = 2 * (  0*AI_dxi    +    3*AI_dxxi   +    3*AI_dxxij    +        44         +     17    -  0    )
```

```python
# I/O operations (we load a point once only)
#                     dxi              dxxi             dxxij         duplicate   other load/write
IO_acou   =         0*IO_dxi    +    3*IO_dxxi   +    0*IO_dxxij  -     2       +     3
IO_acoum  =         0*IO_dxi    +    3*IO_dxxi   +    0*IO_dxxij  -     2       +     3    +  1 / s
IO_vti    =  2 * (  0*IO_dxi    +    3*IO_dxxi   +    0*IO_dxxij  -     2       +     2      ) + 3
IO_tti    =  2 * (  0*IO_dxi    +    3*IO_dxxi   +    3*IO_dxxij  -  3*k +2     +     4     )  + 7
```

```python
print(simplify(AI_acou/(4*IO_acou)))
print(simplify(AI_vti/(4*IO_vti)))
print(simplify(AI_tti/(4*IO_tti)))
print(simplify(AI_acou/(4*IO_acoum)))

OI_acou  = lambdify(k,AI_acou/(4*IO_acou))
OI_acoum = lambdify((k,s),AI_acou/(4*IO_acoum))
OI_vti   = lambdify(k,AI_vti/(4*IO_vti))
OI_tti   = lambdify(k,AI_tti/(4*IO_tti))
```

    (3*k + 2)/(2*(3*k + 1))
    (3*k + 4)/(3*(2*k + 1))
    (6*k**2 + 6*k + 73)/(2*(6*k**2 + 31))
    s*(3*k + 2)/(2*(s*(3*k + 1) + 1))

```python
print(limit(OI_acou(k),k,oo))
print(limit(OI_vti(k),k,oo))
print(limit(OI_tti(k),k,oo))

```

    1/2
    1/2
    1/2

```python
k=[2,4,6,8,10,12,14,16,18,20,22,24,26,28,30]
OI_wave=np.zeros((15,3))
OI=np.zeros((15,3))
for i in range(0,15):
    OI_wave[i,0]=OI_acou(k[i])
    OI_wave[i,1]=OI_vti(k[i])
    OI_wave[i,2]=OI_tti(k[i])
    OI[i,0]=OI_dxi(k[i])
    OI[i,1]=OI_dxxi(k[i])
    OI[i,2]=OI_dxxij(k[i])
```

```python
k=[2,4,6,8,10,12,14,16,18,20,22,24,26,28,30]
s=[2,4,8,16,32,64,128,256]
OI_wavem=np.zeros((15,8))
for i in range(0,15):
    for j in range(0,8):
        OI_wavem[i,j]=OI_acoum(k[i],s[j])
```

```python
import matplotlib.pyplot as plt
```

```python
fig = plt.figure()
plt.hold("off")
intensity = plt.plot(OI)   # this is how you'd plot a single line...
```

```python
fig = plt.figure()
plt.hold("off")
acou = plt.plot(OI_wave[:,0],label='acou')   # this is how you'd plot a single line...
acou = plt.plot(OI_wavem[:,5],label='acoum')   # this is how you'd plot a single line...
vti  = plt.plot(OI_wave[:,1],label='vti')   # this is how you'd plot a single line...
tti  = plt.plot(OI_wave[:,2],label='tti')   # this is how you'd plot a single line...
```

```python
fig = plt.figure()
plt.hold("off")
intensity = plt.imshow(OI_wavem)   # this is how you'd plot a single line...
plt.show()
```

```python

```


```python
# https://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Map%3A_Physical_Chemistry_for_the_Biosciences_(Chang)/09%3A_Chemical_Kinetics/9.11%3A_Oscillating_Reactions

import scipy
```

The following cell is used to define the ODEs for the Belousov-Zhabotinsky
reactions.

The essential reactions taking place are (note: unbalanced and only
including species of interest):

\begin{align}
    BrO_3^- + Br^- &\xrightarrow{k_1} HBrO_2 + HOBr \\
    HBrO_2 + Br^- &\xrightarrow{k_2} 2 HOBr \\
    BrO_3^- + HBrO_2 &\xrightarrow{k_3} 2HBrO_2 + 2Ce^{4+} \\
    2HBrO_2 &\xrightarrow{k_4} BrO_3^- + HOBr \\
    Ce^{4+} &\xrightarrow{k_5} fBr
\end{align}

Set some variables to be used for calculations:

$x$ = [HBrO$_2$], $y$ = [Br$^-$], z = [Ce$^{4+}$]

Assume [BrO$_3^-$] constant $a$

\begin{align}
    \frac{dx}{dt} &= k_1ay - k_2xy + k_3ax - k_4x^2 \\
    \frac{dy}{dt} &= -k_1ay - k_2xy + fK-5z \\
    \frac{dz}{dt} &= 2k_3ax - k_5z \\
\end{align}

```python
def odes(t, var, a, f, k):
    dxdt = k[0]*a*var[1] - k[1]*var[0]*var[1] + k[2]*a*var[0] - k[3]*var[0]**2
    dydt = -k[0]*a*var[1] - k[1]*var[0]*var[1] + f*k[4]*var[2]
    dzdt = 2*k[2]*a*var[0] - k[4]*var[2]
    return [dxdt, dydt, dzdt]
```

Now it's time to solve the ODEs!

```python
a =
f =
k = []
scipy.integrate.solve_ivp(odes, [,], [,], args=(a, f, k))

```