L'Hospital's Rule allows us to simplify the evaluation of limits that involve indeterminate forms. An indeterminate form is defined as a limit that does not give enough information to determine the original limit. The most common indeterminate forms that occur in calculus and other areas of mathematics include:

$$ \frac{0}{0}, \qquad \frac{\infty}{\infty}, \qquad 0 \times \infty, \qquad 1^\infty, \qquad \infty - \infty, \qquad 0^0, \qquad \infty^0 $$

L'Hospital's Rule can be stated as the following:

Suppose $f$ and $g$ are both differentiable functions that approach $0$ (or $\pm \infty$) as $x \rightarrow a$. Then $\frac{f(x)}{g(x)}$ approaches the same limit as $\frac{f^\prime(x)}{g^\prime(x)}$.

The derivatives on the numerator and denominator are both taken separately.

## Examples

```python
from sympy import symbols, limit, diff, sin, cos, log, tan, sqrt, init_printing, plot, oo
from mpmath import ln, e, pi

init_printing()
x = symbols('x')
y = symbols('y')
```

### Example 1: Find the limit of $\lim_{x \rightarrow 1} \frac{x^2 - 1}{x^2 - x}$

We see the limit becomes the indeterminate form $\frac{0}{0}$ as $x \rightarrow 1$. Applying L'Hospital's Rule, we differentiate the numerator and denominator separately:

$$ \frac{\frac{d}{dx} x^2-1}{\frac{d}{dx} x^2-x} $$

Then carrying the derivation:

$$ \frac{\frac{d}{dx} x^2-1}{\frac{d}{dx} x^2-x} = \frac{2x}{2x-1} = \frac{2(1)}{2(1)-1)} = \frac{2}{1} = 2 $$

We can confirm our answer using Sympy and the `limit()` function.

```python
limit((x ** 2 - 1) / (x ** 2 - x), x, 1)
```

### Example 2: Find the limit of $\lim_{x \rightarrow 1} \frac{x^9-1}{x^5-1}$

As before, start by differentiating both the numerator and denominator:

$$ \frac{\frac{d}{dx} x^9-1}{\frac{d}{dx} x^5-1} = \frac{9x^8}{5x^4} = \frac{9(1)^8}{5(1)^4} = \frac{9}{5} $$

```python
limit((x ** 9 - 1) / (x ** 5 - 1), x, 1) # Check our above result
```

### Example 3: Determine the limit of $\lim_{x \rightarrow (\frac{\pi}{2})^+} \frac{\cos{x}}{1-\sin{x}}$

The limit we are evaluating is on the right-hand side, for now we can proceed to calculate the limit as before:

$$ \frac{\frac{d}{dx} \cos{x}}{\frac{d}{dx} 1-\sin{x}} = \frac{-\sin{x}}{\cos{x}} = -\tan{x} $$

The limit we are evaluating is therefore equivalent to $\lim_{x \rightarrow \frac{\pi}{2}^+} -\tan{x}$, which approaches $-\infty$ as it is coming from the right-hand side. We can visualize this to confirm by taking advantage of Sympy's `plot()` function.

```python
plot(cos(x) / (1 - sin(x)), xlim=(1, 2))
```

Although it's a little hard to see as we're doing a very rough plot, we see that the function shoots up to infinity near $\frac{\pi}{2}$ coming from the left-hand side and negative infinity from the right. We can verify our answer again with Sympy. The third argument `'+'` is added to the `limit()` function to instruct Sympy to only consider the limit on the right-hand side.

```python
limit(cos(x) / (1 - sin(x)), x, pi / 2, '+')
```

### Example 4: Find the limit of $\lim_{x \rightarrow 0} \frac{e^x-1}{x^3}$

Proceeding as before by evaluating the derivatives of the numerator and denominator of the limit:

$$ \frac{\frac{d}{dx} e^x - 1}{\frac{d}{dx} x^3} = \frac{e^x}{3x^2} = \frac{e^{(0)}}{3(0)^2} $$

Although we arrive at $\frac{1}{0}$ if we were to continue solving, that is not the correct limit. As $x \rightarrow 0$, the function approaches infinity. We can see this behavior by plotting a rough graph of the function.

```python
plot(e ** x / (3 * x ** 2), xlim=(0, 10))
```

From the graph, we can see that the function shoots up drastically as $x \rightarrow 0$ due to the denominator approaching $0$ quicker than the numerator. We can confirm the actual limit is $\infty$ by checking against SymPy.

```python
limit((e ** x - 1) / x ** 3, x, 0)
```

## Historical Note

Although known as L'Hospital's Rule, the theorem was developed by Johann Bernoulli, the Swiss mathematician, in 1694 and likely introduced to L'Hospital. The publication of L'Hospital's textbook on differential calculus in 1696, the first of its kind, included the theorem and thus L'Hospital's name became attached to the rule.

## References

Indeterminate form. (2017, December 23). In Wikipedia, The Free Encyclopedia. From https://en.wikipedia.org/w/index.php?title=Indeterminate_form&oldid=816695609

[Stewart, J. (2007). Essential calculus: Early transcendentals. Belmont, CA: Thomson Higher Education.](https://amzn.to/38dnRV0)

Weisstein, Eric W. "L'Hospital's Rule." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/LHospitalsRule.html

content/posts/L'Hospital's Rule for Calculating Limits and Indeterminate Forms.ipynb


```python
from IPython.core.display import HTML
def css_styling():
    styles = open("./styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

### BEFORE YOU DO ANYTHING...
In the terminal:
1. Navigate to __inside__ your ILAS_Python repository.
2. __COMMIT__ any un-commited work on your personal computer.
3. __PULL__ any changes *you* have made using another computer.
4. __PULL__ textbook updates (including homework answers).

1. __Open Jupyter notebook:__   Start >> Programs (すべてのプログラム) >> Programming >> Anaconda3 >> JupyterNotebook
1. __Navigate to the ILAS_Python folder__.
1. __Open Seminar 7___  by clicking on 7_Numerical_computation_with_Numpy.

Numerical Computation with Numpy

Lesson Goal

Compose programs to solve simple mathematical problems using the Python Numpy package.

## Objectives
- Represent data using the `array` data structure for numerical computation.
- Use 1D and 2D arrays to represent vectors and matrices.
- Manipulate arrays (indexing, slicing, vectorising etc)
- Perform familiar numerical operations using Python.
- Compare efficiency of vectorised and non-vectorised functions.

## Why are we studying this?
Numerical computation is central to almost all scientific and engineering problems.

There are programming languages specifically designed for numerical computation:
- Fortran
- MATLAB

There are libaries dedicated to efficient numerical computations:
- Numpy
- Scipy
- Sympy ...

NumPy (http://www.numpy.org/)
 - The most widely used Python library for numerical computations.
 - Large, extensive library of data structures and functions for numerical computation.
 - Useful for perfoming operation you will learn on mathematics-based courses.

Scipy (https://www.scipy.org/)
- Builds on Numpy, additional functionality
- More specialised data structures and functions over NumPy.

If you are familiar with MATLAB, NumPy and SciPy provide similar functionality.

Last week we covered an introduction to some basic functions of Numpy.

NumPy is a very extensisve library.

This seminar will:
- Introduce some useful functions
- Briefly discuss how to search for additional functions you may need.

your best resources are search engines, such as http://stackoverflow.com/.

## Importing the NumPy module

To make NumPy functions and variables available to use in our program in our programs, we need to __import__ it using.

`import numpy`

We typically import all modules at the start of a program or notebook.

The shortened name `np` isoften used for numpy.

```python
import numpy as np
```

## Data Structure: The Numpy `array`

### Why do we need another data structure?

Python lists hold 'arrays' of data.

Lists are very flexible. e.g. holding mixed data type.

There is a trade off between flexibility and performance e.g. speed.

Science engineering and mathematics problems often involve large amounts of data and numerous operations.

We therefore use specialised functions and data structures for numerical computation.

## Numpy array

A numpy array is a grid of values, *all of the same type*.

To create an array we use the Numpy `array` function.

It takes a list as an argument.

```python
a = np.array([1, 2, 3])

print(type(a))

print(a.dtype)
```

    <class 'numpy.ndarray'>
    int64

```python
b = [4.0, 5, 6.0]

c = np.array(b)

print(type(b))
print(type(c))
print(c.dtype)
```

    <class 'list'>
    <class 'numpy.ndarray'>
    float64

## Multi-dimensional arrays.

Unlike the data types we have studied so far, arrays can have multiple dimensions.

__`rank`:__ the number of *dimensons* of the array.

__`shape`:__ a *tuple* of *integers* giving the *size* of the array along each *dimension*.

```python
# 1-dimensional array
a = np.array([1, 2, 3])

# 2-dimensional array
b = np.array([[1, 2, 3], [4, 5, 6]])

b = np.array([[1, 2, 3],
              [4, 5, 6]])

print(a.shape)
print(b.shape)
```

    (3,)
    (2, 3)

```python
# 3-dimensional array

c = np.array(
    [[[1, 1],
      [1, 1]],

     [[1, 1],
      [1, 1]]])

print(c.shape)

c = np.array(
    [[[1, 1],
      [1, 1]],

     [[1, 1],
      [1, 1]],

     [[1, 1],
      [1, 1]]])

print(c.shape)
```

    (2, 2, 2)
    (3, 2, 2)

```python
# 3-dimensional array

c = np.array(
    [[[1, 1],
      [1, 1]],

     [[1, 1],
      [1, 1]]])

# 4-dimensional array
d = np.array(
    [[[[1, 1],
       [1, 1]],

      [[1, 1],
       [1, 1]]],

      [[[1, 1],
       [1, 1]],

      [[1, 1],
       [1, 1]]]])

print(c.shape)
print(d.shape)
```

    (2, 2, 2)
    (2, 2, 2, 2)

## Creating a numpy array.

There are several other ways we can create an array

```python
# Create an array of all zeros
# The zeros() function argument is the shape.
# Shape: tuple of integers giving the size along each dimension.

a = np.zeros(5)
print(a)

print()

a = np.zeros((2,2))
print(a)
```

    [ 0.  0.  0.  0.  0.]

    [[ 0.  0.]
     [ 0.  0.]]

```python
# Create an array of all ones

b = np.ones(5)
print(b)

print()

b = np.ones((1,2))
print(b)
```

    [ 1.  1.  1.  1.  1.]

    [[ 1.  1.]]

```python
# Create a constant array
# The second function argument is the constant value

c = np.full(6, 8)
print(c)

print()

c = np.full((2,2,2), 7)
print(c)

```

    [8 8 8 8 8 8]

    [[[7 7]
      [7 7]]

     [[7 7]
      [7 7]]]

```python
# Create an array filled with random values

e = np.random.rand(6)
print(e)

print()

e = np.random.random((2,2))
print(e)

print()

e = np.random.randint(16, size=(4,4))
print(e)

print()

e = np.random.randint(8, size=(2, 2, 2))
print(e)
```

    [ 0.85172471  0.8399638   0.78430877  0.22117159  0.59024844  0.52306961]

    [[ 0.77130148  0.72136067]
     [ 0.96344053  0.67477549]]

    [[ 1  0 14  4]
     [ 5  0 10 10]
     [ 0  0 15  2]
     [ 9 14  6  7]]

    [[[4 7]
      [6 0]]

     [[3 6]
      [5 0]]]

## Indexing.

We can index into an array exactly the same way as the other data structures we have studied.

```python
x = np.array([1, 2, 3, 4, 5])

print(x[4])

print(x[2:])
```

    5
    [3, 4, 5]

For an n-dimensional matrix we need n index values to address an element or range of elements.

Note the order in which dimensions are addressed.

```python
# 2 dimensional array

y = np.array([[1, 2, 3],
              [4, 5, 6]])

print(y[1,2])

print(y[1:, 0:2])
```

    6
    [[4 5]]

```python
# 3-dimensional array

c = np.array(
    [[[2, 1, 4],
      [2, 6, 8]],

     [[0, 1, 5],
      [7, 8, 9]]])

print(c[0, 1, 2])

```

    8

Where we want to select all elements in one dimension we can use :

__Exception__: If it is the last element , we can omit it.

```python
print(c[0, 1])

print(c[0, :, 1])
```

    [2 6 8]
    [1 6]

## Iterating over multi-dimensional arrays.
We can iterate over a 1D array in the same way as the data structures we have previously studied.

```python
A = np.array([1, 2, 3, 4, 5])
```

```python
for a in A:
    print(a)
```

    1
    2
    3
    4
    5

To loop through individual elements of a multi-dimensional array, we use a nested loop for each dimension of the array.

```python
B = np.array([[1, 2, 3],
              [4, 5, 6]])

for row in B:
    print("-----")
    for col in row:
        print(col)
```

    -----
    1
    2
    3
    -----
    4
    5
    6

## Manipulating arrays
We can use many of the same operations to manipulate arrays as we use for lists.

```python
# Length of an array

a = np.array([1, 3, 4, 17, 3, 21, 2, 12])

b = ([1, 3, 4, 17],
     [3, 21, 2, 12])

print(len(a))
print(len(b))

```

    8
    2

Note the length is the length of the first dimension (e.g. indexing).

```python
# Sort an array

a = np.array([1, 3, 4, 17, 3, 21, 2, 12])

# The method sort() and function sorted() give the same result
a.sort()

a = sorted(a)

print(a)
```

    [1, 2, 3, 3, 4, 12, 17, 21]

Arrays are *immutable* (unchangeable).

Technically you cannot delete an item from it.

However, you can make a new array without the values you don't want:

```python
# Remove items from an array

z = np.array([1, 3, 4, 17, 3, 21, 2, 12])

z = np.delete(z, 3)
print(z)

z = np.delete(z, [0, 1, 2])
print(z)

```

    [ 1  3  4  3 21  2 12]
    [ 3 21  2 12]

```python
# Add items to an array

a = ([1, 2, 3])
a = np.insert(a, 1, 4)
print(a)
```

    [1 4 2 3]

```python
# Add items to an array

b = np.array([[1, 1],
              [2, 2],
              [3, 3]])

b = np.insert(b, 1, 4)
print(b)
```

    [1 4 1 2 2 3 3]

```python
# Add items to an array

b = np.array([[1, 1],
              [2, 2],
              [3, 3]])

b = np.insert(b, 1, 4, axis=1)
print(b)
```

    [[1 4 1]
     [2 4 2]
     [3 4 3]]

```python
# Change items in an array

c = np.array([1, 2, 3])
c[1] = 4
print(c)
```

    [1 4 3]

### Boolean array indexing

Boolean indexing selects elements that satisfy a condition:

```python
a = np.array([[1,2],
              [3, 4],
              [5, 6]])

bool = a > 2

print(bool)

# The varaible bool can now be used as the index to select all elements
# greater than 2.
print(a[bool_idx])

# To do this in a single statement:
print(a[a > 2])
```

    [[False False]
     [ True  True]
     [ True  True]]
    [3 4 5 6]
    [3 4 5 6]

## Mathematics with arrays.

Unlike lists, NumPy arrays support common arithmetic operations, such as addition of two arrays.

```python
# To add the elements of two lists we need the Numpy function: add
x = [1.0, 0.2, 1.2]
y = [2.0, 0.1, 2.1]
print(x)
print(y)

# Sum x and y
z = x + y
print(z)

# Sum x and y
z = np.add(x, y)
print(z)
```

    [1.0, 0.2, 1.2]
    [2.0, 0.1, 2.1]
    [1.0, 0.2, 1.2, 2.0, 0.1, 2.1]
    [ 3.   0.3  3.3]

```python
# To add the elemnts of two arrays we can just use regular arithmetic operators
x = np.array([1.0, 0.2, 1.2])
y = np.array([2.0, 0.1, 2.1])
print(x)
print(y)

# Sum x and y
z = x + y
print(z)

# We can also use the Numpy function, add
z = np.add(x, y)
print(z)
```

    [ 1.   0.2  1.2]
    [ 2.   0.1  2.1]
    [ 3.   0.3  3.3]
    [ 3.   0.3  3.3]

```python
print(x - y)
print(np.subtract(x, y))

print()

print(x * y)
print(np.multiply(x, y))

print()

print(x / y)
print(np.divide(x, y))

print()

print(np.sqrt(x))
print(x ** (1/2))

print()

print(x ** 2)
```

    [-1.   0.1 -0.9]
    [-1.   0.1 -0.9]

    [ 2.    0.02  2.52]
    [ 2.    0.02  2.52]

    [ 0.5         2.          0.57142857]
    [ 0.5         2.          0.57142857]

    [ 1.          0.4472136   1.09544512]
    [ 1.          0.4472136   1.09544512]

    [ 1.    0.04  1.44]

Numpy functions are appled elementwise to an array.

This means the function is applied individually to each element in the list.

For example, we can apply trigonometric functions, elementwise, to arrays, lists and tuples.

```python
x = np.array([0.0, np.pi/2, np.pi, 3*np.pi/2])
y = [0.0, np.pi/2, np.pi, 3*np.pi/2]
z = (0.0, np.pi/2, np.pi, 3*np.pi/2)

print(np.sin(x))
print(np.cos(y))
print(np.tan(z))

```

    [  0.00000000e+00   1.00000000e+00   1.22464680e-16  -1.00000000e+00]
    [  1.00000000e+00   6.12323400e-17  -1.00000000e+00  -1.83697020e-16]
    [  0.00000000e+00   1.63312394e+16  -1.22464680e-16   5.44374645e+15]

## Vectorising Functions
Because the outcome of this is that a 1D array behave like a vector, we call this *vectorisation*.

Vectorisation can make a significant difference in the time taken to compute a value.

## Mathematics with Vectors (1D arrays)
Let's look at a  previous example for computing the dot product of two vectors.

The dot product of two $n$-length-vectors:

 $ \mathbf{A} = [A_1, A_2, ... A_n]$

 $ \mathbf{B} = [B_1, B_2, ... B_n]$

\begin{align}
\mathbf{A} \cdot \mathbf{B} = \sum_{i=1}^n A_i B_i.
\end{align}

We learnt to solve this very easily using a Python `for` loop.

With each iteration of the loop we increase the value of `dot_product` (initial value = 0.0) by the product of `a` and `b`.

```python
A = [1.0, 3.0, -5.0]
B = [4.0, -2.0, -1.0]

# Create a variable called dot_product with value, 0.
dot_product = 0.0

for a, b in zip(A, B):
    dot_product += a * b

print(dot_product)
```

Using Numpy arrays we can solve the dot product using the Numpy function `dot`.

```python
A = np.array([1.0, 3.0, -5.0])
B = np.array([4.0, -2.0, -1.0])

print(np.dot(A,B))
```

    3.0

__Try it yourself__

Recap the Seminar 5: Functions; in the cell below write a function that takes two lists and returns the dot product using the code from Seminar 4: Data Structures (shown above).

Use the magic function `%timeit` to compare the speed of the for loop with the Numpy `dot()` function for solving the dot product.

```python
# Write a function for the dot product of two vectors expressed as lists
# Compare the speed of your function to the Numpy function
```

## Mathematics with Matrices (2D arrays)
If you have previously studied matrices, the operations in this section will be familiar.

If you have not yet studied matrices, you may want to refer back to this section once matrices have been covered in your mathematics classes.

Material from this section will not be included in the exam.

2D arrays are a convenient way to represents matrices.

For example, to create the matrix

$$
A =
\begin{bmatrix}
3 & 5 & 7\\
2 & 4 & 6
\end{bmatrix}
$$

```python
A = np.array([[3, 5, 7],
              [2, 4, 6]])
print(A)
```

    [[3 5 7]
     [2 4 6]]

Recall, the method `shape()` tells us the dimensions of an array.

This gives us the number of rows and the number of columns, expressed as a tuple.

By describe the dimensions of a matrix as as "rows" by "columns" matrix.

```python
print(A.shape)
print(f"Number of rows is {A.shape[0]}, number of columns is {A.shape[1]}")
print(f"A is an {A.shape[0]} by {A.shape[1]} matrix")
```

    (2, 3)
    Number of rows is 2, number of columns is 3

### Matrix multiplication

If the number of __columns in A__

is the same as number of __rows in B__,

we can find the matrix product of $\mathbf{A}$ and $\mathbf{B}$.

\begin{align}
\mathbf{A} \mathbf{B} = \mathbf{C}
\end{align}

We multiply each __row__ in \mathbf{A} by each __column__ in \mathbf{B}

\begin{equation*}
\underbrace{
\begin{bmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\
a_{31} & a_{32} & a_{33} \\
\end{bmatrix}
}_{\mathbf{A} \text{ 3 rows} \text{ 3 columns}}
\times
\underbrace{
\begin{bmatrix}
b_{11} \\
b_{21} \\
b_{31} \\
\end{bmatrix}
}_{\mathbf{B} \text{  3 rows} \text{  1 column}}
=\underbrace{
\begin{bmatrix}
a_{11}b_{11} + a_{12}b_{21} + a_{13}b_{31} \\
a_{21}b_{11} + a_{22}b_{21} + a_{23}b_{31} \\
a_{31}b_{11} + a_{32}b_{21} + a_{33}b_{31} \\
\end{bmatrix}
}_{\mathbf{C} \text{  3 rows} \text{  1 column}}
\end{equation*}

In matrix $\mathbf{C}$, the element in

__row $i$__,

__column $j$__

is equal to the dot product of

__$i$th row__ of $\mathbf{A}$,

__$j$th column__ of $\mathbf{B}$.

Matrix $\mathbf{C}$ therefore has

the same number of __rows as A__,

the same number of __columns as B__.

#### Example 1: THIS EXAMPLE IS WRONG MAKE A GOOD ONE!!!

#### Example 2:
\begin{equation*}
\underbrace{
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9 \\
\end{bmatrix}
}_{\mathbf{A} \text{ 3 rows} \text{ 3 columns}}
\times
\underbrace{
\begin{bmatrix}
10 & 11 \\
12 & 13 \\
14 & 15 \\
\end{bmatrix}
}_{\mathbf{B} \text{  3 rows} \text{  2 columns}}
=\underbrace{
\begin{bmatrix}
(1 \cdot 10 + 2 \cdot 12 + 3 \cdot 14) \quad
(1 \cdot 11 + 2 \cdot 13 + 3 \cdot 15) \\
(4 \cdot 10 + 5 \cdot 12 + 6 \cdot 14) \quad
(4 \cdot 11 + 5 \cdot 13 + 6 \cdot 15) \\
(7 \cdot 10 + 8 \cdot 12 + 9 \cdot 14) \quad
(7 \cdot 11 + 8 \cdot 13 + 9 \cdot 15) \\
\end{bmatrix}
}_{\mathbf{C} \text{  3 rows} \text{  2 columns}}
=\underbrace{
\begin{bmatrix}
76  & 82 \\
184 & 199 \\
292 & 316 \\
\end{bmatrix}
}_{\mathbf{C} \text{  3 rows} \text{  2 columns}}
\end{equation*}

```python
#Example 1
A = np.array([[1, 1, 2],
              [2, 1, 3],
              [1, 4, 2]])

B = np.array([[3],
              [1],
              [2]])

C = A.dot(B)
print(C)

print()

C = np.dot(A,B)
print(C)
```

    [[ 8]
     [13]
     [11]]

    [[ 8]
     [13]
     [11]]

```python
#Example 2
A = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]])

B = np.array([[10, 11],
              [12, 13],
              [14, 15]])

C = A.dot(B)
print(C)

print()

C = np.dot(A,B)
print(C)
```

    [[ 76  82]
     [184 199]
     [292 316]]

    [[ 76  82]
     [184 199]
     [292 316]]

We can find the inverse $\mathbf{A}^{-1}$, of a sqaure matrix, $\mathbf{A}$.

```python
A = np.array([[1,2],
              [3, 4]])

Ainv = np.linalg.inv(A)

print(f"A = \n {A}")
print(f"Inverse of A = \n {Ainv}")
```

    A =
     [[1 2]
     [3 4]]
    Inverse of A =
     [[-2.   1. ]
     [ 1.5 -0.5]]

We can find the determinant, $\textrm{det}(\mathbf{A})$, of a sqaure matrix, $\mathbf{A}$.

```python
A = np.array([[1,2],
              [3, 4]])

Adet = np.linalg.det(A)

print(f"A = \n {A}")
print(f"Determinant of A = {round(Adet, 2)}")
```

    A =
     [[1 2]
     [3 4]]
    Determinant of A = -2.0

We can generate an *identity matrix*.

```python
I = np.eye(2)
print(I)

print()

I = np.eye(4)
print(I)
```

    [[ 1.  0.]
     [ 0.  1.]]

    [[ 1.  0.  0.  0.]
     [ 0.  1.  0.  0.]
     [ 0.  0.  1.  0.]
     [ 0.  0.  0.  1.]]

```python

```


NEGI-Course Andøya 2018

This cell is hidden as we generate the title using metadata (see below)

# General structure

- Project title
- Name, email, course title, date, group assistant
- Abstract (1/2 page max)
- Introduction (1 page)
- Method
    - Packages used
    - Datasets (models and observations)
    - Analysis method
    - …
- Results
- Discussion and outlook (1 page)
- Conclusions (1/2 page)
- References
- Acknowledgments

# How to generate pdf report

## Markdown

- Select Markdown cell instead of code cell
- [Markdown cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet)

## Add a report title

- Edit --> Edit Notebook Metadata

```bash
 "ipub": {
    "bibliography": "/mnt/data/teachers/annefou/test_report.bib",
    "titlepage": {
      "author": "Anne Fouilloux",
      "email": "authors@email.com",
      "institution": [
        "Institution1",
        "Institution2"
      ],
      "logo": "/mnt/data/teachers/annefou/NEGI2018.png",
      "subtitle": "Example for writing your report",
      "supervisors": [
        "First Supervisor",
        "Second Supervisor"
      ],
      "tagline": "A tagline for the report.",
      "title": "Climate science at high latitudes: Modeling and model evaluation"
    }
  },
```

## Hide a cell output

View --> Cell Toolbar --> Edit Metadata

And add:

```bash
  "ipub": {
    "ignore": true
  }
```

## Add Code block

In the Cell metadata:

```bash
"ipub": {
  "code": {
    "format" : {},
    "asfloat": true,
    "caption": "",
    "label": "code:example_code",
    "widefigure": false,
    "placement": "H"
    }
  }
```

## Add a latex reference

### Figures

There are two ways to reference plots:

1. Use latex syntax:

-  Save your figure in a code cell:

```python
fig.savefig('fig_example.png') ;
```
where fig is a matplotlib figure (fig = plt.figure ( figsize =[12 ,5])).

- Reference your plot in a markdown cell:

"ipub": {
    "figure": {
      "caption": "Simple time serie over Oslo",
      "label": "fig:oslo_timeserie"
    }
  }

In both cases, you can reference your plots using latex syntax `\ref`

### cite a paper

For all the paper in your bib file (added in the notebook metadata), you can use the following syntax to cite your paper:

- you can add a citation using latex citation `\cite`.

## Generate pdf with nbpublish

To convert your jupyter notebook into pdf, you need to use nbpublish. See last cell of this jupyter notebook.

```bash
!nbpublish -f latex_ipypublish_all -pdf report_example.ipynb
```

# Import python packages

```python
import xarray as xr
import dask . array as da
import matplotlib . pyplot as plt
import cartopy.crs as ccrs
import numpy as np
import pandas as pd
%matplotlib inline

```

# Data and Methods
## Read Data

```python
dset = xr.open_dataset('/mnt/data/students/evelien/Observations/air.mon.mean.seasonal.arctic.nc')
```

```python
dset . time
```

    <xarray.DataArray 'time' (time: 461)>
    array(['1900-01-16T12:00:00.000000000', '1900-04-01T00:00:00.000000000',
           '1900-07-01T00:00:00.000000000', ..., '2014-07-01T00:00:00.000000000',
           '2014-10-01T00:00:00.000000000', '2014-12-01T00:00:00.000000000'],
          dtype='datetime64[ns]')
    Coordinates:
      * time     (time) datetime64[ns] 1900-01-16T12:00:00 1900-04-01 ... 2014-12-01
    Attributes:
        standard_name:  time
        long_name:      Time
        bounds:         time_bnds
        axis:           T

### Select the nearest date

```python
x = dset["air"].sel( time ="1989-01-01T00:00:00.000000000", method = 'nearest')

```

# Results

```python
fig = plt . figure ( figsize =[12 ,5])
ax1 = fig . add_subplot (211 , projection = ccrs . PlateCarree () )
ax1 . coastlines ( resolution ="50m")

m1 = dset["air"].sel (time ="1989-01-01", method = 'nearest').plot( ax = ax1 , transform = ccrs.PlateCarree() , vmin =-50 , vmax =50 , add_colorbar =False )

ax1 . set_title ( 'Air temperature DJF 1989 ', fontsize =20)

ax2 = fig . add_subplot (212 , projection = ccrs . PlateCarree () )
ax2 . coastlines ( resolution ="50m")
m2 = dset ["air"]. sel ( time ="1989-07-01 T00:00:00.000000000", method = 'nearest') . plot ( ax = ax2 , transform = ccrs.PlateCarree () , vmin =-50 ,vmax =50 , add_colorbar = False )
ax2 . set_title ( 'Air temperature JJA 1989', fontsize =20)

fig . subplots_adjust ( bottom =0.1 , top =0.9 , left =0.1 , right =0.8 ,wspace =0.02 , hspace =0.02)
cb_ax = fig.add_axes ([0.83 , 0.1 , 0.02 , 0.8])
cbar = fig.colorbar ( m2 , cax = cb_ax)
cbar . ax . set_yticklabels ( cbar.ax.get_yticklabels() , fontsize =20)

# You can save your plots with savefig and for your pdf include your graphics using latex (see next cell)
# - To avoid having two plots in your pdf report, edit the cell metadata and add:
# ,
#  "ipub": {
#    "ignore": true
#  }
# Make sure you remove # in your metadata cell
#
fig.savefig('test_report.png') ;
```

\begin{figure}
  \includegraphics[width=\linewidth]{/mnt/data/teachers/annefou/test_report.png}
  \caption{A Nice plot.}
  \label{fig:plot_example}
\end{figure}

```python
# Select one location and group by year to plot
# to get your plot in your report (with a label you can reference):
# - edit cell metadata
# - Add the following (customize label for your own plot)
#
#
#"ipub": {
#    "figure": {
#      "caption": "Simple time serie over Oslo",
#      "label": "fig:oslo_timeserie"
#    }
#  }
#

# Add a semi-column at the end to suppress the output of the function
dset["air"].sel(lat=59.91,lon=10.74, method="nearest").groupby("time.year").mean(dim="time").plot();

```

And then you can reference your previous timeserie \ref{fig:oslo_timeserie}

```python
# Add a semi-column at the end to suppress the output of the function
# Andoya coordinates
dset["air"].sel(lat=60.0,lon=15.71, method="nearest").groupby("time.year").mean(dim="time").plot();
```

Timeserie over Andoya. See \ref{fig:andoya_timeserie}. The plot has been generated with \ref{code:example_code}.

# Discussion
Within a markdown cell, you can add a citation using latex citation \cite{allan:2000}.

You can also add equations:
\begin{equation}
    \qquad
    \frac{\partial p}{\partial z}=-\rho g\label{prim3}\end{equation}

The heat equation

\begin{equation}
    \frac{dQ}{dt}\equiv F_{T}=c_{p}\frac{dT}{dt}-\frac{\alpha T}{\rho}\frac{dp}{dt}\label{prim4}
    \end{equation}

The differential form of the heat equation (\ref{prim4})
without heating can be combined with the hydrostatic equation (\ref{prim3}) to give the temperature
equation for an adiabatic ascent of a parcel.

\begin{equation}
    \frac{dT}{dz}=-\Gamma\equiv\frac{g\alpha T}{c_{p}}\end{equation}

And you can make a reference to your plots. See Figure \ref{fig:plot_example} shows a nice plot.

# Acknowledgments

- model and data owners/providers

And make sure you acknowledge Sigma2:

- *This study was performed using jupyterhub deployed on resources provided by UNINETT Sigma2 - the National Infrastructure for High Performance Computing and Data Storage in Norway as part of NS1000K project. In particular, we would like to thank Thierry Toutain and Gurvinder Singh.*

```python
#This cell needs to be executed twice (for references)
!nbpublish -f latex_ipypublish_all -pdf report_example.ipynb
!nbpublish -f latex_ipypublish_all -pdf report_example.ipynb
```

    INFO:main:started ipypublish v0.6.7 at Tue Nov  6 20:43:55 2018
    INFO:main:logging to: /mnt/data/teachers/annefou/converted/report_example.nbpub.log
    INFO:main:running for ipynb(s) at: report_example.ipynb
    INFO:main:with conversion: latex_ipypublish_all
    INFO:nbmerge:Reading notebook
    INFO:main:getting output format from exporter plugin
    INFO:nbexport:running nbconvert
    INFO:latex_doc_defaults:adding ipub defaults to notebook
    INFO:split_outputs:splitting outputs into separate cells
    INFO:latex_doc_links:resolving external file paths in ipub metadata to: report_example.ipynb
    INFO:latex_doc_captions:extracting caption cells
    INFO:main:outputting converted file to: /mnt/data/teachers/annefou/converted/report_example.tex
    INFO:main:dumping external files to: /mnt/data/teachers/annefou/converted/report_example_files
    INFO:main:running pdf conversion
    INFO:pdfexport:running: latexmk -xelatex -bibtex -pdf --interaction=batchmode <outpath>
    INFO:pdfexport:latexmk: Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
    INFO:pdfexport:latexmk: Latexmk: All targets (report_example.pdf) are up-to-date
    INFO:pdfexport:pdf conversion complete
    INFO:main:process finished successfully
    INFO:main:started ipypublish v0.6.7 at Tue Nov  6 20:43:56 2018
    INFO:main:logging to: /mnt/data/teachers/annefou/converted/report_example.nbpub.log
    INFO:main:running for ipynb(s) at: report_example.ipynb
    INFO:main:with conversion: latex_ipypublish_all
    INFO:nbmerge:Reading notebook
    INFO:main:getting output format from exporter plugin
    INFO:nbexport:running nbconvert
    INFO:latex_doc_defaults:adding ipub defaults to notebook
    INFO:split_outputs:splitting outputs into separate cells
    INFO:latex_doc_links:resolving external file paths in ipub metadata to: report_example.ipynb
    INFO:latex_doc_captions:extracting caption cells
    INFO:main:outputting converted file to: /mnt/data/teachers/annefou/converted/report_example.tex
    INFO:main:dumping external files to: /mnt/data/teachers/annefou/converted/report_example_files
    INFO:main:running pdf conversion
    INFO:pdfexport:running: latexmk -xelatex -bibtex -pdf --interaction=batchmode <outpath>
    INFO:pdfexport:latexmk: Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
    INFO:pdfexport:latexmk: Latexmk: All targets (report_example.pdf) are up-to-date
    INFO:pdfexport:pdf conversion complete
    INFO:main:process finished successfully

```python

```


```python
### Magic functinos
%load_ext autoreload
%autoreload 2
%load_ext tensorboard
%matplotlib inline
```

```python
### imports
import warnings
warnings.simplefilter('ignore')
import itertools
import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
from sklearn.model_selection import train_test_split
from mliv.dgps import get_data, get_tau_fn, fn_dict
from mliv.neuralnet.deepiv_fit import deep_iv_fit
from mliv.neuralnet.utilities import log_metrics, plot_results
from mliv.neuralnet.rbflayer import gaussian, inverse_multiquadric
from mliv.neuralnet import AGMM, KernelLayerMMDGMM, CentroidMMDGMM, KernelLossAGMM, MMDGMM
```

# DGPs

```python
n = 100
n_z = 3
n_t = 3
iv_strength = .6
fname = 'abs'
dgp_num = 5
Z, T, Y, true_fn = get_data(n, n_z, iv_strength, get_tau_fn(fn_dict[fname]), dgp_num)
```

```python
ind = 0
x_grid = np.linspace(np.quantile(T[:, ind], .01), np.quantile(T[:, ind], .99), 100)
T_test = np.zeros((100, T.shape[1])) + np.median(T, axis=0, keepdims=True)
T_test[:, ind] = x_grid
```

```python
plt.figure(figsize=(10,3))
plt.subplot(1, 2, 1)
plt.scatter(Z[:, 0], Y)
plt.subplot(1, 2, 2)
plt.scatter(T[:, 0], Y)
plt.plot(T[np.argsort(T[:, 0]), 0], true_fn(T[np.argsort(T[:, 0])]))
plt.show()
```

# DeepIV

```python
deep_iv = deep_iv_fit(Z[:,[0]], T[:,[0]], Y, x=T[:,1:],
                      epochs=100, hidden=[100])
```

```python
y_pred = deep_iv.predict([T_test[:, 1:], T_test[:, [0]]])
plt.scatter(true_fn(T_test).flatten(), y_pred)
plt.title(1 - np.mean((true_fn(T_test).flatten()-y_pred.flatten())**2) / np.var(true_fn(T_test).flatten()))
plt.show()
```

# AGMM Variants

## Learner Network

```python
p = 0.1 # dropout prob of dropout layers throughout notebook
n_hidden = 100 # width of hidden layers throughout notebook

learner = nn.Sequential(nn.Dropout(p=p), nn.Linear(n_t, n_hidden), nn.LeakyReLU(),
                        #nn.Dropout(p=p), nn.Linear(n_hidden, n_hidden), nn.ReLU(),
                        nn.Dropout(p=p), nn.Linear(n_hidden, 1))
```

## Adversary Networks

```python
# For any method that use a projection of z into features g(z)
g_features = 100
adversary_g = nn.Sequential(nn.Dropout(p=p), nn.Linear(n_z, n_hidden), nn.LeakyReLU(),
                            nn.Dropout(p=p), nn.Linear(n_hidden, g_features), nn.ReLU())
# The kernel function
kernel_fn = gaussian
# kernel_fn = inverse_multiquadric

# For any method that uses an unstructured adversary test function f(z)
adversary_fn = nn.Sequential(nn.Dropout(p=p), nn.Linear(n_z, n_hidden), nn.LeakyReLU(),
                             #nn.Dropout(p=p), nn.Linear(n_hidden, n_hidden), nn.ReLU(),
                             nn.Dropout(p=p), nn.Linear(n_hidden, 1))
```

# HyperParameters

```python
learner_lr = 1e-4
adversary_lr = 1e-4
learner_l2 = 1e-3
adversary_l2 = 1e-4
adversary_norm_reg = 1e-3
n_epochs = 300
bs = 100
sigma = 2.0/g_features
n_centers = 100
device = torch.cuda.current_device() if torch.cuda.is_available() else None
```

```python
%tensorboard --logdir=runs
```

# Train-Val Split

```python
if torch.cuda.is_available():
    torch.cuda.empty_cache()
Z_train, Z_val, T_train, T_val, Y_train, Y_val = train_test_split(Z, T, Y, test_size=.1, shuffle=True)
Z_train, T_train, Y_train = map(lambda x: torch.Tensor(x), (Z_train, T_train, Y_train))
Z_val, T_val, Y_val = map(lambda x: torch.Tensor(x).to(device), (Z_val, T_val, Y_val))
G_train = true_fn(T_train)
G_val = true_fn(T_val)
G_train, G_val = map(lambda x: x.to(device), (G_train,G_val))
T_test_tens = torch.Tensor(T_test).to(device)
G_test_tens = true_fn(T_test_tens).to(device)
```

## $\ell_2$-Regularized AGMM with Neural Net Test Function

We solve the problem:
\begin{equation}
\min_{\theta} \max_{w} \frac{1}{n} \sum_i (y_i - h_{\theta}(x_i)) f_w(z_i) - f_w(z_i)^2
\end{equation}
where $h_{\theta}$ and $f_w$ are two neural nets.

```python
def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary[-1].weight, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val)

np.random.seed(12356)
agmm = AGMM(learner, adversary_fn).fit(Z_train, T_train, Y_train, learner_lr=learner_lr, adversary_lr=adversary_lr,
                                       learner_l2=learner_l2, adversary_l2=adversary_l2,
                                       n_epochs=n_epochs, bs=bs, logger=logger,
                                       model_dir='agmm_model', device=device)
```

```python
plot_results(agmm, T_test_tens, G_test_tens, ind=0)
```

## $\ell_2$-regularized MMD-GMM with Fixed Kernel Points and No Kernel Learning

We choose a fixed set of centers and approximate the kernel function with the kernel distance to these centers.

Here we learn only the $\beta$ and fix $c_i, \sigma_i$, i.e.:
\begin{align}
f(z) = \sum_{i=1}^s \beta_i K_{\sigma_i}(c_i, z_i)
\end{align}
and both $c_i$ is a grid and $\sigma$ is a constant. We are also approximating the RKHS norm with:
\begin{align}
\|f\|_{K}^2 = \sum_{i, j \in [s]} \beta_i \beta_j \frac{K_{\sigma_i}(c_i, c_j) + K_{\sigma_j}(c_j, c_i)}{2}
\end{align}

```python
def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary.beta.weight, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val)

# Fixed centers c_1, ..., c_s and kernel precision sigma_1, ..., sigma_s
centers = np.tile(np.linspace(-4, 4, n_centers).reshape(-1, 1), (1, n_z))
sigmas = np.ones((n_centers,)) * 2/n_z

mmdgmm_fixed = KernelLayerMMDGMM(learner, lambda x: x, n_z, n_centers, kernel_fn,
                      centers=centers, sigmas=sigmas, trainable=False)
mmdgmm_fixed.fit(Z_train, T_train, Y_train, learner_l2=learner_l2, adversary_l2=adversary_l2,
                 adversary_norm_reg=adversary_norm_reg,
                 learner_lr=learner_lr, adversary_lr=adversary_lr, n_epochs=n_epochs, bs=bs, logger=logger,
                 model_dir='mmd_fixed_model', device=device)
```

```python
plot_results(mmdgmm_fixed, T_test_tens, G_test_tens)
```

## $\ell_2$-regularized MMD-GMM with Kernel Learning and Approximation via Kernel Layer

Here we learn both the G function and the centers and sigmas, i.e.:
\begin{align}
f(z) = \sum_{i=1}^s \beta_i K_{\sigma_i}(c_i, g(z_i))
\end{align}
and both $c_i$ and $\sigma_i$ are also trained. $c_i$ are $n_{\text{features}}$-dimensional vectors, initialized to some inital set of grid values. Moreover, $g(z)$ is a fully connected network with RELU gates. We are also approximating the RKHS norm with:
\begin{align}
\|f\|_{K}^2 = \sum_{i, j \in [s]} \beta_i \beta_j \frac{K_{\sigma_i}(c_i, c_j) + K_{\sigma_j}(c_j, c_i)}{2}
\end{align}

```python
def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary.beta.weight, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val)

# Trainable centers c_1, ..., c_s and precisions sigma_1, ..., sigma_s initialization
#centers = np.tile(np.linspace(-4, 4, n_centers).reshape(-1, 1), (1, g_features))
centers = np.random.uniform(-4, 4, size=(n_centers, g_features))
sigmas = np.ones((n_centers,)) * sigma

klayermmdgmm = KernelLayerMMDGMM(learner, adversary_g, g_features,
                                 n_centers, kernel_fn, centers=centers, sigmas=sigmas)
klayermmdgmm.fit(Z_train, T_train, Y_train, learner_l2=learner_l2, adversary_l2=adversary_l2,
                 adversary_norm_reg=adversary_norm_reg,
                 learner_lr=learner_lr, adversary_lr = adversary_lr,  n_epochs=n_epochs, bs=bs, logger=logger,
                 model_dir='klayer_model', device=device)
```

```python
plot_results(klayermmdgmm, T_test_tens, G_test_tens, ind=0)
```

## $\ell_2$-regularized MMD-GMM with Kernel Learning and Approximation via KMeans Centroids

Here we do exactly what is described in the paper for low rank approximation. We choose centers of a kmeans clusters in the $z$-space, $c_1, \ldots, c_m$ and then we test function of the form:
\begin{equation}
f(z) = \sum_{i=1}^m \beta_i K_{\sigma_i}(g_w(c_i), g_w(z))
\end{equation}
and penalizes the approximate RKHS norm:
\begin{align}
\|f\|_{K}^2 = \sum_{i, j \in [s]} \beta_i \beta_j \frac{K_{\sigma_i}(g_w(z_i), g_w(z_j)) + K_{\sigma_j}(g_w(z_j), g_w(z_i))}{2}
\end{align}

```python
def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary.beta.weight, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val)

# Kmeans based centers z_1, ..., z_s
from sklearn.cluster import KMeans
centers = KMeans(n_clusters=n_centers).fit(Z).cluster_centers_
centroid_mmd = CentroidMMDGMM(learner, adversary_g, kernel_fn, centers, np.ones(n_centers)*sigma)
centroid_mmd.fit(Z_train, T_train, Y_train, learner_l2=learner_l2, adversary_l2=adversary_l2,
                 adversary_norm_reg=adversary_norm_reg,
                 learner_lr=learner_lr, adversary_lr=adversary_lr, n_epochs=n_epochs, bs=bs, logger=logger,
                 model_dir='centroid_model', device=device)
```

```python
plot_results(centroid_mmd, T_test_tens, G_test_tens, ind=0)
```

## Un-Regularized MMD-GMM with Kernel Loss and Learned Kernel

We are minimizing the objective:
\begin{equation}
\min_{\theta} \max_{w} \frac{1}{n^2} \sum_{i,j} (y_i - h_{\theta}(x_i)) K_{\sigma}(g_w(z_i), g_w(z_j)) (y_j - h_{\theta}(x_j)
\end{equation}

```python
def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary.sigma, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val, loss='kernel')

kernelgmm = KernelLossAGMM(learner, adversary_g, kernel_fn, sigma)
kernelgmm.fit(Z_train, T_train, Y_train, learner_l2=learner_l2**2, adversary_l2=adversary_l2,
              learner_lr=learner_lr, adversary_lr=adversary_lr, n_epochs=n_epochs,
              bs=bs, logger=logger, model_dir='kernel_model', device=device)
```

```python
plot_results(kernelgmm, T_test_tens, G_test_tens)
```

## $\ell_2$-Regularized MMD-GMM without Kernel Approximation

Here we test for function of the form:
\begin{equation}
f(z) = \sum_{j=1}^n \beta_j K(g_w(z_j), g_w(z))
\end{equation}
where $i$ ranges over all the training samples. Since the function itself depends on all the data, we need to create unbiased stochastic estimates of the test function at each time step, as well as unbiased stochastic estimtes of its RKHS and $\ell_{2,n}$ norms.

We sample a subset of the indices $S$ and test for:
\begin{equation}
\hat{f}(z) = \frac{n}{|S|}\sum_{j\in S} \beta_j K(g_w(z_i), g_w(z)).
\end{equation}
Moreover, in order to create an unbiased estimate of the second moment of the test function on the training data, we also draw another set of indices $T$ and a minibatch of data $B$ and approximate it via:
\begin{equation}
\widehat{\|f\|_{2,n}^2} = \frac{1}{|B|} \frac{n}{|S|} \frac{n}{|T|}\sum_{i\in B, j\in S, k\in T} \beta_j \beta_k K(g_w(z_j), g_w(z_i)) K(g_w(z_k), g_w(z_i)).
\end{equation}
Moreover, an unbiased estimate of the RKHS norm of the function is created as:
\begin{equation}
\widehat{\|f\|_{K}^2} = \frac{n}{|S|} \frac{n}{|T|}\sum_{j \in S, k\in T} \beta_j \beta_k K(g_w(z_j), g_w(z_k))
\end{equation}

This might require some more fine tuning as the current version is too slow to learn. The gradients here have huge variance. We most probably need to sample the indices $j$ should be sampled proportional to $|\beta_j|$.

```python
np.random.seed(123)

def logger(learner, adversary, epoch, writer):
    writer.add_histogram('learner', learner[-1].weight, epoch)
    writer.add_histogram('adversary', adversary.sigma, epoch)
    log_metrics(Z_val, T_val, Y_val, Z_val, T_val, Y_val, T_test_tens,
                learner, adversary, epoch, writer, true_of_T=G_val, loss=None)

mmdgmm = MMDGMM(learner, adversary_g, Y_train.shape[0], kernel_fn, sigma*np.ones(Y_train.shape[0]))
mmdgmm.fit(Z_train, T_train, Y_train, learner_l2=learner_l2, adversary_l2=adversary_l2,
           adversary_norm_reg=adversary_norm_reg,
           learner_lr=learner_lr, adversary_lr=adversary_lr, n_epochs=n_epochs, bs1=bs, bs2=300, bs3=100,
           logger=logger, model_dir='mmd_model', device=device)
```

```python
plot_results(mmdgmm, T_test_tens, G_test_tens)
```

```python

```


            prepared by Özlem Salehi (QTurkey)

This cell contains some macros. If there is a problem with displaying mathematical formulas, please run this cell to load these macros.

$ \newcommand{\bra}[1]{\langle #1|} $
$ \newcommand{\ket}[1]{|#1\rangle} $
$ \newcommand{\braket}[2]{\langle #1|#2\rangle} $
$ \newcommand{\dot}[2]{ #1 \cdot #2} $
$ \newcommand{\biginner}[2]{\left\langle #1,#2\right\rangle} $
$ \newcommand{\mymatrix}[2]{\left( \begin{array}{#1} #2\end{array} \right)} $
$ \newcommand{\myvector}[1]{\mymatrix{c}{#1}} $
$ \newcommand{\myrvector}[1]{\mymatrix{r}{#1}} $
$ \newcommand{\mypar}[1]{\left( #1 \right)} $
$ \newcommand{\mybigpar}[1]{ \Big( #1 \Big)} $
$ \newcommand{\sqrttwo}{\frac{1}{\sqrt{2}}} $
$ \newcommand{\dsqrttwo}{\dfrac{1}{\sqrt{2}}} $
$ \newcommand{\onehalf}{\frac{1}{2}} $
$ \newcommand{\donehalf}{\dfrac{1}{2}} $
$ \newcommand{\hadamard}{ \mymatrix{rr}{ \sqrttwo & \sqrttwo \\ \sqrttwo & -\sqrttwo }} $
$ \newcommand{\vzero}{\myvector{1\\0}} $
$ \newcommand{\vone}{\myvector{0\\1}} $
$ \newcommand{\stateplus}{\myvector{ \sqrttwo \\  \sqrttwo } } $
$ \newcommand{\stateminus}{ \myrvector{ \sqrttwo \\ -\sqrttwo } } $
$ \newcommand{\myarray}[2]{ \begin{array}{#1}#2\end{array}} $
$ \newcommand{\X}{ \mymatrix{cc}{0 & 1 \\ 1 & 0}  } $
$ \newcommand{\Z}{ \mymatrix{rr}{1 & 0 \\ 0 & -1}  } $
$ \newcommand{\Htwo}{ \mymatrix{rrrr}{ \frac{1}{2} & \frac{1}{2} & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & \frac{1}{2} & -\frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & -\frac{1}{2} & \frac{1}{2} } } $
$ \newcommand{\CNOT}{ \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0} } $
$ \newcommand{\norm}[1]{ \left\lVert #1 \right\rVert } $
$ \newcommand{\pstate}[1]{ \lceil \mspace{-1mu} #1 \mspace{-1.5mu} \rfloor } $

# Quantum Fourier Transform

What makes Fourier Transform special is that it can be computed faster on a quantum computer than on a classical computer. In this notebook, we will learn about the quantum analogue of Discrete Fourier Transform.

Let's recall the definition for $DFT$. $DFT$ of the vector $x=\myvector{x_0~x_1\dots~x_{N-1}}^T$ is the complex vector $y=\myvector{y_0~y_1\dots y_{N-1}}^T$ where

$$
y_k=\frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j.
$$

Now suppose that we have an $N=2^n$-dimensional quantum state vector $x=\myvector{x_0~x_1\dots~x_{N-1}}^T$ representing the state
$$
\ket{\psi} = \sum_{j=0}^{N-1} x_j \ket{j}.
$$

Quantum Fourier Transform $QFT$ of state $\ket{\psi}$ is given by
$$
\ket{\phi}=\sum_{k=0}^{N-1}y_k \ket{k} = \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j \ket{k},
$$
where $y_k$ is defined as above for $k=0,\dots,N-1$.

Hence for a basis state $\ket{j}$ represented by the vector $\myvector{x_0~x_1\dots~x_{N-1}}^T$ where $x_j=1$ and the rest of the entries are 0, its $QFT$ is given by

$$
 \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}e^{\frac{2\pi i j k }{N}} \ket{k}.
$$

$e^{2\pi i/N}$ is often denoted by $\omega$. The above expression can be equivalently expressed as
$$
 \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}{\omega^{ j k} } \ket{k}.
$$

Task 1 (on paper)

Apply $QFT$ to the basis state $\ket{10}$ and find the new quantum state.

click for our solution

Task 2 (on paper)

Apply $QFT$ to the state $ \ket \psi=\frac{1}{\sqrt{2}} \ket {01}+\frac{1}{\sqrt{2}} \ket {10} $.

click for our solution

Task 3 (on paper)

Apply $QFT$ to the quantum state $\ket{\psi}=\alpha \ket{0} + \beta \ket{1}$ and find the new quantum state. Conclude that applying 1 qubit $QFT$ is equivalent to applying Hadamard gate.

click for our solution

Remark: Note that two vectors that differ by a relative phase can not be distinguished when measured. For instance the vectors $\frac{1}{2} \myvector{1 \\ -i \\ -1 \\ i}$ and $\frac{1}{2} \myvector{1 \\ -1 \\ 1 \\ -1}$ are not distinguishable. But when $QFT$ is applied, then we obtain the vectors $ \myvector{0 \\ 1 \\ 0 \\ 0}$ and $ \myvector{ 0\\ 0 \\ 1 \\ 0}$ respectively and now the two vectors are distinguishable.

QFT Operator

Let's try to find out the matrix corresponding to $QFT$. Given $x$, the entries of the vector $y$ corresponding to $QFT$ of the vector $x$ is given by

$$
y_k=\frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j = \frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}\omega^{j k}x_j .
$$

Hence, we can equivalently write:

\begin{align*}
y_0 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1 + x_2 + \cdots + x_{N-1}  \right ) \\
y_1 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega  + x_2 \omega^2+ \cdots + x_{N-1} \omega^{N-1}  \right )\\
y_2 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega^2  + x_2 \omega^4+ \cdots + x_{N-1} \omega^{2N-2}  \right )\\
\vdots \\
y_{N-1} &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega^{N-1}  + x_2 \omega^{2N-2}+ \cdots + x_{N-1} \omega^{(N-1)(N-1)}  \right ).
\end{align*}

From above, the matrix representation can be given by the following matrix.

$$ \frac{1}{\sqrt{N}} \mymatrix{rrrrrr}{1 & 1 & 1 &1  & \cdots & 1\\
                   1 & \omega & \omega^2 & \omega^3 & \cdots& \omega^{N-1} \\
                   1 & \omega^2 &\omega^4 & \omega^6  & \cdots& \omega^{2N-2} \\
                  1 & \omega^3 & \omega^6 & \omega^9 & \cdots& \omega^{3N-3} \\
                  \vdots &  \vdots &  \vdots &  \vdots &  \vdots &  \vdots \\
                    1 & \omega^{N-1} & \omega^{2N-2} & \omega^{3N-3} & \cdots& \omega^{(N-1)(N-1)} \\
                    }  $$

 Observation

Let $\ket{j}$ be a basis state where $j=0,\dots N-1$. Then after applying $QFT$, the vector that is obtained is exactly the $j+1$'st column of the $QFT$ opreator.

Task 4 (on paper)

Apply $QFT$ to the basis state $\ket{10}$ using the matrix representation and find the new quantum state.

click for our solution

Properties of QFT (Optional)

 $QFT$ is unitary: You can check this from the matrix corresponding to $QFT$.

 Linear shift: Linear shift of a state vector causes relative phase shift of its $QFT$. The converse is also true.

  If  $QFT$ $\myvector{\alpha_0 \\ \alpha_1 \\ \vdots \\ \alpha_{N-1} } = \myvector{\beta_0 \\ \beta_1 \\ \vdots \\ \beta_{N-1}}$, then $QFT$ $\myvector{\alpha_{N-1} \\ \alpha_0 \\ \vdots \\ \alpha_{N-2} } = \myvector{\beta_0 \\ \omega \beta_1 \\ \vdots \\ \omega^{N-1}\beta_{N-1}}$ and
      $QFT$  $\myvector{\alpha_0 \\ \omega  \alpha_1 \\ \vdots \\ \omega^{N-1}\alpha_{N-1} } = \myvector{\beta_1 \\ \beta_2 \\ \vdots \\ \beta_{0}}$.

Task 5 (on paper)

What is the quantum state obtained after applying $QFT$ to the state $\ket{11}$? Find using Task 4 and the linear shift property.

click for our solution

 Circuit implementation for $QFT$

Recall that after applying $QFT$ to the state $\ket{\psi}=\alpha\ket{0} + \beta \ket{1}$, we get  $\frac{\alpha+\beta}{\sqrt{2}} \ket{0} + \frac{\alpha-\beta}{\sqrt{2}} \ket{1}$, which is the state obtained after applying Hadamard to the state $\ket{\psi}$. Hence, the effect of applying $QFT$ to a single qubit is equivalent to applying Hadamard.

Now let us look at the effect of applying $QFT$ to a $n$-qubit system. Let $N=2^n$.

Let $\ket{j}$ be a basis state where $j=0,\dots,N-1$. We will use the binary representation for $j$, that is $j=j_1j_2\cdots j_n$ for $j_i \in \{0,1\}$. More formally

\begin{equation} \label{eq: binary}
j=j_12^{n-1} + j_2 2^{n-2} + \cdots + j_n2^0 = 2^n \sum_{l=1}^n 2^{-l} j_l~~~~~~~~(1)
\end{equation}

We will stick to the expression above but equivalently you may encounter the following expression for $j$ in various places.

\begin{align*}
j&=2^n(j_1 2^{-1} + j_2 2^{-2} + \cdots j_n 2^{-n}) \\
&= 2^n (0.j_1j_2\cdots j_n)
\end{align*}

Now let's use this representation to express the state obtained after applying $QFT$ to $\ket{j}$.

\begin{align*}
\ket{j}=\ket{j_1j_2\cdots j_n} &\rightarrow  \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}e^{\frac{2\pi i j k }{N}} \ket{k}\\
&=  \frac{1}{2^{n/2}} \sum_{k_1=0}^{1} \cdots \sum_{k_n=0}^{1}e^{2\pi i j (\sum_{o=1}^n k_o2^{-o})} \ket{k_1k_2\cdots k_n} \mbox{   by replacing $k$ using Equation (1)} \\
&=\frac{1}{2^{n / 2}} \sum_{k_{1}=0}^{1}  \sum_{k_2=0}^{1}\cdots \sum_{k_{n}=0}^{1} \prod_{o=1}^{n} e^{2 \pi i j k_o 2^{-o}}|k_{1} \ldots k_{n}\rangle\\
&=  \frac{1}{2^{n/2}}\otimes_{o=1}^n  \sum_{k_o=0}^{1} e^{2\pi i jk_o 2^{-o} } \ket{k_o} \mbox{   by writing the quantum state as the tensor product of $n$ qubits} \\
&=  \frac{1}{2^{n/2}} \otimes_{o=1}^n \biggl (\ket{0} + e^{2\pi i j 2^{-o} } \ket{1} \biggr) \\
&= \frac{1}{2^{n/2}} \biggl ( \bigl(\ket{0} + e^{2\pi i j 2^{-1}} \ket{1} \bigr) \otimes  \bigl(\ket{0} + e^{2\pi i j 2^{-2}}\ket{1} \bigr) \otimes \cdots \otimes  \bigl(\ket{0} + e^{2\pi i j 2^{-n}}\ket{1} \bigr ) \biggr )~~~~~~ (2)
\end{align*}

Let's find the correct gates for applying $QFT$. We are going to use two gates: Hadamard and controlled rotation gate. Check the following unitary operator:

$$ CR_k = \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & e^{\frac{2\pi i}{2^{k}}} }  $$

$CR_k$ operator puts a relative phase of $e^{\frac{2\pi i}{2^k}}$ in front of the quantum state if both the controlled and the target qubits are in state $\ket{1}$. It corresponds to a rotation around $z$-axis.

\begin{align*}
CR_k \ket{00} &\rightarrow \ket{00} \\
CR_k \ket{01} &\rightarrow \ket{01} \\
CR_k \ket{10} &\rightarrow \ket{10} \\
CR_k \ket{11} &\rightarrow  e^{\frac{2\pi i}{{2^k}}}\ket{11} \\
\end{align*}

Let $\ket{j_1j_2\cdots j_n}$ be the input state. Applying Hadamard to the first qubit, we get

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i j_1 2^{-1}} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

since $e^{\pi i j_1}=1$ if $j_1=0$ and $e^{\pi i j_1} = -1$ if $j_1=1$.

Now let's apply the opreator $CR_2$ where second qubit is the control and the first qubit is the target. The resulting state is

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i  (j_1 2^{-1}+{j_2}{2^{-2}}  )} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

After applying $CR_i$ where qubit $i$ is the controller and the first qubit is the target consecutively for $i=3\dots n$, we obtain the state

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i  (j_1 2^{-1}+{j_2}{2^{-2}} +{j_3}{2^{-3}} \dots {j_n}{2^{-n}}   )} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

Noting that $j=2^n(j_1 2^{-1} + j_2 2^{-2} + \cdots j_n 2^{-n})$, we get

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

Task 6 (on paper)

Continue with the same procedure this time using the second qubit as the target and obtain the following state:

$$
\frac{1}{2^{1/{4}}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \bigl (\ket{0} + e^{2\pi i j 2^{-n+1}  } \ket{1} \bigr ) \ket{j_3\cdots j_n}
$$

- First apply Hadamard to second qubit

- Then apply $CR_i$ where qubit $i+1$ is the control and the second qubit is the target for $i= 2 \dots n-1$.

click for our solution

The above procedure is repeated for each qubit and finally we get the state

$$
\frac{1}{2^{n/2}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \otimes  (\ket{0} + e^{2\pi i j 2^{-n+1}} \ket{1} \bigr ) \otimes  \cdots \otimes  (\ket{0} + e^{2\pi i j 2^{-1}} \ket{1} \bigr  )
$$

Note that this is exactly Equation (2) except that the qubits are in the reverse order. This also proves that $QFT$ is unitary, as the gates we used are unitary gates. Overall, the $QFT$ circuit is given below:

Now we are ready to implement QFT circuits in Qiskit. To implement $CR_k$, we will use the `CPhase` gate form Qiskit's circuit library. Let's start by importing necessay Qiskit objects.

```python
from qiskit import QuantumCircuit, execute, Aer
from numpy import pi

# For pretty printing arrays
from IPython.display import display
from qiskit.visualization import array_to_latex

# Get Unitary and State Vector Simluators
statevector_simulator = Aer.get_backend('statevector_simulator')
unitary_simulator = Aer.get_backend('unitary_simulator')
```

### `CPhaseGate`

In Qiskit, `CPhaseGate` is given by the following unitary matrix: $$ CPhase(\theta) = \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & e^{i \theta} } $$

To apply `CPhaseGate`, you can use `QuantumCircuit.cp` method with the parameter `theta`.
```python
circuit = QuantumCircuit(2)
circuit.cp(
    theta = λ,
    control_qubit=0,
    target_qubit=1,
)
```

Drawing this circuit with `circuit.draw()` will result in:
```
q_0: ─■──
      │λ
q_1: ─■──
```

Task 7

Implement the circuit for applying $QFT$ to a 2 qubit system. Try it using $\ket{01}$ as your input state. Simulate your circuit and compare it with the solution of Task 1.

```python
#Create a QuantumCircuit with 2 qubits
circuit = QuantumCircuit(2)

# Initialize the qubit to 01

# Apply Hadamard to first qubit

# Apply CPhase gate with θ = 2πi / 2^2 to the circuit where
# the first qubit is target and the second qubit is control

# Apply Hadamard to second qubit

# Swap the qubits

#Simulate the circuit and print the results
```

click for our solution

### Task 8

Implement the circuit which applies $QFT$ to any $n$-qubit system. Define a function named `qft(n, circuit)` which takes the number of qubits, and the circuit as its parameters. Simulate the circuit for $n=3$ on all possible input states. Verify that your circuit works.

```python
def qft(n,circuit):

    #Your code here
    pass
```

```python
from itertools import product
from numpy import angle
QUBIT_NUMBER = 3

for digits in product((0, 1), repeat=3):

    # Create a QuantumCircuit
    circuit = QuantumCircuit(QUBIT_NUMBER)

    # Initialization

    # Call qft function

    # Execute the circuit
```

click for our solution

Inverse Quantum Fourier Transform

Inverse Quantum Fourier Transform ($QFT^{\dagger}$) is the transformation which satisfies $QFT\cdot QFT^{\dagger}=I$. It is defined exactly the same with the exponents having negative sign.

$$
 QFT^{\dagger} \ket{k}=\frac{1}{\sqrt{N}} \sum_{l=0}^{N-1}e^{-\frac{2\pi i j k }{N}}\ket{l}.
$$

To implement $QFT^{\dagger}$, one should apply all the operations in reverse order to undo the circuit.

### Task 9

Implement the circuit which applies $QFT^{\dagger}$ to any $n$-qubit system. Define a function named `iqft` which takes the circuit as its parameter. Simulate the circuit for different $n$ values. Apply both $QFT$ and $QFT^{\dagger}$ and check whether the corresponding unitary matrix is the identity matrix.

```python
def iqft(circuit):

    #Your code here
    pass
```

```python
QUBIT_NUMBER = 3

#Create circuit
circuit = QuantumCircuit(QUBIT_NUMBER)

# Create n qubits

# Call qft function

# Call iqft function

# Printing circuit

# Simulating circuit

# Unitary matrix representation

```

click for our solution

 Discussion on the number of gates

Gate numbers for the main body of the QFT circuit can be listed in the following fashion.
- First qubit:
  - $1$ Hadamard
  - $n-1$ controlled $R_k$
- Second qubits:
  - $1$ Hadamars
  - $n-2$ controlled $R_k$
- $\cdots$
- $n$th qubit:
  - $1$ Hadamard
  - $n-n=0$ controlled $R_k$

So overall, the gate numbers for the main body are are
\begin{align}
(1+ n-1 ) + (1 + n-2) + \dots + (1+1)
&= n + (n-1) + \dots + 1\\
& = \frac{n\,(n+1)}{2}.
\end{align}

Furthermore, ${n}\,/\,{2}$ $SWAP$ gates are required, each of which can be implemented using three $CNOT$ gates. To sum up, $\theta(n^2)$ gates are required for applying $QFT$ to an $n$-qubit system represented by a vector of size $N=2^n$.

The best known classical algorithm for computing the Discrete Fourier Transform of $2^n$ entries, such as Fast Fourier Transform (FFT) requires $\Theta(n2^n)$, equivalently $\Theta(N \log N)$ gates, which means that the classical algorithm requires exponentially many more operations to compute $DFT$ (Note that the obvious algorithm for computing $DFT$ requires ${2}^{2n}$ gates). An improved algorithm presented [here](https://ieeexplore.ieee.org/document/892139) provides a circuit for computing $QFT$ with $O(n \log n)$ gates.

Nevertheless, this does not mean that we can use $QFT$ directly to accelerate the classical computation process. There are two reasons for this: The first reason is that the amplitudes can not be accessed directly after applying $QFT$. The second reason is that we may not know how to efficiently prepare the input state to $QFT$.

Next, we will see some applications of $QFT$.

```python

```


```python
%load_ext autoreload
%autoreload 2
%config IPCompleter.greedy = True
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

# pyEPR Calculating Disipative Participation Ratios

Zlatko Minev

**Summary:** Following Appendix E of the energy-participation-ratio (EPR) paper, here we demonstrate how to calcualte the dielectric EPR of a chip substrate in a qubit eigen mode.

We use the following definitions for the RMS energy stored in a volume $V$,
\begin{align}
\mathcal{E}_{\mathrm{elec}}&=&\frac{1}{4}\mathrm{Re}\int_{V}\mathrm{d}v\vec{E}_{\text{max}}^{*}\overleftrightarrow{\epsilon}\vec{E}_{\text{max}}\;,\\\mathcal{E}_{\mathrm{mag}}&=&\frac{1}{4}\mathrm{Re}\int_{V}\mathrm{d}v\vec{H}_{\text{max}}^{*}\overleftrightarrow{\mu}\vec{H}_{\text{max}}\;,
\end{align}

## The simple way

Following the first tutorial, let's loads the `pyEPR` package under the shorthand name `epr`.

```python
import pyEPR as epr
```

#### Load Ansys HFSS tutorial file

As we did in the previous tutorial, let us first determine where the example file is stored.
For tutorial, let us get the path to the tutorial folder.

```python
# Load Path temporarily just to find where the tutorial folder is
# return path_to_project
from pathlib import Path
path_to_project = Path(epr.__file__).parent.parent / '_example_files'
print(f'We will the example project located in\n {path_to_project}')
```

    We will the example project located in
     C:\zkm-code\pyEPR\_example_files

Now, we will open Ansys Desktop and connect to a specific project and we will create the analsys eprh

```python
pinfo = epr.ProjectInfo(project_path = path_to_project,
                         project_name = 'pyEPR_tutorial1',
                         design_name  = '1. single_transmon')
eprh = epr.DistributedAnalysis(pinfo)
```

    INFO 02:35AM [connect]: Connecting to Ansys Desktop API...
    INFO 02:35AM [load_ansys_project]: 	File path to HFSS project found.
    INFO 02:35AM [load_ansys_project]: 	Opened Ansys App
    INFO 02:35AM [load_ansys_project]: 	Opened Ansys Desktop v2016.0.0
    INFO 02:35AM [load_ansys_project]: 	Opened Ansys Project
    	Folder:    C:/zkm-code/pyEPR/_example_files/
    	Project:   pyEPR_tutorial1
    INFO 02:35AM [connect]: 	Opened active design
    	Design:    1. single_transmon [Solution type: Eigenmode]
    INFO 02:35AM [get_setup]: 	Opened setup `Setup1`  (<class 'pyEPR.ansys.HfssEMSetup'>)
    INFO 02:35AM [connect]: 	Connection to Ansys established successfully. 😀

    Design "1. single_transmon" info:
    	# eigenmodes    2
    	# variations    8

# Calculate participation of the substrate for mode 1

First, select which eigenmode to work on. Here the fundamental mode, mode 0, is the qubit.
```python
eprh.set_mode(0)
```

Let us now calculate the dielectric energy-participatio ratio
of the substrate  relative to the dielectric energy of all objects, using the function
```python
eprh.calc_p_electric_volume
```
Note that when all objects are specified, this does not include any energy
that might be stored in any lumped elements or lumped capacitors.

Returns:
---------
    ℰ_object/ℰ_total, (ℰ_object, _total)

```python
eprh.set_mode(0)

# Calculate the EPR p_dielectic
p_dielectic, (ℰ_substr, ℰ_total) = eprh.calc_p_electric_volume('substrate', 'AllObjects')

print(f'Energy in silicon substrate  = {100*p_dielectic:.1f}%')
```

    Energy in silicon substrate  = 87.7%

Now, compute the electric energy stored in the vacuum
Use the calculated total energy in all objects
so that we don't have to recompute it, since we
computed it  above

```python
# Here we will pass in the precomputed E_total=ℰ_total
p_vac, (ℰ_vac, ℰ_total) = eprh.calc_p_electric_volume('cavity_enclosure', E_total=ℰ_total)

print(f'''Energy in vacuum = {100*p_vac:.1f}%

Since there are no other volumes,
the two energies should sum to one: {p_dielectic + p_vac}''')
```

    Energy in vacuum = 12.3%

    Since there are no other volumes,
    the two energies should sum to one: 0.9999999999999989

Let's find outmore about the functuion signature

```python
? eprh.calc_p_electric_volume
```

    Signature:
     eprh.calc_p_electric_volume(
        name_dielectric3D,
        relative_to='AllObjects',
        E_total=None,
    )
    Docstring:
    Calculate the dielectric energy-participatio ratio
    of a 3D object (one that has volume) relative to the dielectric energy of
    a list of object objects.

    This is as a function relative to another object or all objects.

    When all objects are specified, this does not include any energy
    that might be stored in any lumped elements or lumped capacitors.

    Returns:
    ---------
        ℰ_object/ℰ_total, (ℰ_object, _total)
    File:      c:\zkm-code\pyepr\pyepr\core.py
    Type:      method

## Calculating the energies directly

Using lower level functions

```python
ℰ_total  = eprh.calc_energy_electric(volume='AllObjects')
ℰ_substr = eprh.calc_energy_electric(volume='substrate')
print(f'Energy in substrate = {100*ℰ_substr/ℰ_total:.1f}%')
```

    Energy in substrate = 87.7%

```python
?eprh.calc_energy_electric
```

    Signature:
    eprh.calc_energy_electric(
        variation=None,
        volume='AllObjects',
        smooth=False,
    )
    Docstring:
    Calculates two times the peak electric energy, or 4 times the RMS, :math:`4*\mathcal{E}_{\mathrm{elec}}`
    (since we do not divide by 2 and use the peak phasors).

    .. math::
        \mathcal{E}_{\mathrm{elec}}=\frac{1}{4}\mathrm{Re}\int_{V}\mathrm{d}v\vec{E}_{\text{max}}^{*}\overleftrightarrow{\epsilon}\vec{E}_{\text{max}}

    volume : string | 'AllObjects'
    smooth : bool | False
        Smooth the electric field or not when performing calculation

    Example use to calcualte the energy participation of a substrate

    .. code-block python
        ℰ_total  = epr_hfss.calc_energy_electric(volume='AllObjects')
        ℰ_substr = epr_hfss.calc_energy_electric(volume='Box1')
        print(f'Energy in substrate = {100*ℰ_substr/ℰ_total:.1f}%')
    File:      c:\zkm-code\pyepr\pyepr\core.py
    Type:      method

# Using the Fields calculator in HFSS directly

We will do the same calculation again, but now using the internals of `eprh.calc_energy_electric` to demonstrate how the fields calcualtor object can be used for custom integrals and how the internals work.

#### Using the HFSS Fields Calculator
The Fields calculator enables you to perform computations using basic field quantities. The calculator will compute derived quantities from the general electric field solution; write field quantities to files, locate maximum and minimum field values, and perform other operations on the field solution.

The calculator does not perform the computations until a value is needed or is forced for a result. This makes it more efficient, saving computing resources and time; you can do all the calculations without regard to data storage of all the calculated points of the field. It is generally easier to do all the calculations first, then plot the results.

#### Direct calculation of

\begin{align}
\mathcal{E}_{\mathrm{elec}}&=&\mathrm{Re}\int_{V}\mathrm{d}v\vec{E}_{\text{max}}^{*}\overleftrightarrow{\epsilon}\vec{E}_{\text{max}}\;.
\end{align}

```python
from pyEPR.core import *
from pyEPR.core import CalcObject

self, volume = eprh, 'AllObjects'

calcobject = CalcObject([], self.setup)
vecE = calcobject.getQty("E").smooth()
A = vecE.times_eps()
B = vecE.conj()
A = A.dot(B)
A = A.real()
A = A.integrate_vol(name=volume)

E_total = A.evaluate(lv=self._get_lv())
# This command numerically evaluates and displays the
# results of calculator operations
E_total
```

    3.77403611623721

```python
from pyEPR.core import *

self, volume = eprh, 'substrate'

calcobject = CalcObject([], self.setup)
vecE = calcobject.getQty("E").smooth()
A = vecE.times_eps()
B = vecE.conj()
A = A.dot(B)
A = A.real()
A = A.integrate_vol(name=volume)

E_subs = A.evaluate(lv=self._get_lv())
# This command numerically evaluates and displays the
# results of calculator operations
E_subs
```

    3.30908711470017

```python
print(f'Energy in substrate: {100*E_subs/E_total:.1f}%')
```

    Energy in substrate: 87.7%

# Summary

We showed three levels of how to obtain the same result, using either the
1. `pyeprh.calc_p_electric_volume`
2. `pyeprh.calc_energy_electric`
3. Or the fields calcualtor

**NEXT:** Please see *Part III* of the tutorial to continue. You will find it in the tutorial folder as a jupyter notebook.

Updated pyEPR Files/pyEPR/_tutorial_notebooks/Tutorial 2.  Field calculations - dielectric energy participation ratios (EPRs).ipynb