# Superposition Kata Workbook

**What is this workbook?**
A workbook is a collection of problems, accompanied by solutions to them.
The explanations focus on the logical steps required to solve a problem; they illustrate the concepts that need to be applied to come up with a solution to the problem, explaining the mathematical steps required.

Note that a workbook should not be the primary source of knowledge on the subject matter; it assumes that you've already read a tutorial or a textbook and that you are now seeking to improve your problem-solving skills. You should attempt solving the tasks of the respective kata first, and turn to the workbook only if stuck. While a textbook emphasizes knowledge acquisition, a workbook emphasizes skill acquisition.

This workbook describes the solutions to the problems offered in the [Superposition kata](./Superposition.ipynb).
Since the tasks are offered as programming problems, the explanations also cover some elements of Q# that might be non-obvious for a first-time user.

**What you should know for this workbook**

You should be familiar with the following concepts before tackling the Superposition kata (and this workbook):
1. Basic linear algebra
2. The concept of qubit and multi-qubit systems
3. Single-qubit and multi-qubit quantum gates

To begin, first prepare this notebook for execution (if you skip this step, you'll get "Syntax does not match any known patterns" error when you try to execute Q# code in the next cells):

```qsharp
%package Microsoft.Quantum.Katas::0.9.1909.3002
```

> The package versions in the output of the cell above should always match. If you are running the Notebooks locally and the versions do not match, please install the IQ# version that matches the version of the `Microsoft.Quantum.Katas` package.
>
> How to install the right IQ# version
> For example, if the version of `Microsoft.Quantum.Katas` package above is 0.1.2.3, the installation steps are as follows:
>
> 1. Stop the kernel.
> 2. Uninstall the existing version of IQ#:
>        dotnet tool uninstall microsoft.quantum.iqsharp -g
> 3. Install the matching version:
>        dotnet tool install microsoft.quantum.iqsharp -g --version 0.1.2.3
> 4. Reinstall the kernel:
>        dotnet iqsharp install
> 5. Restart the Notebook.
>

##  Task 1. Plus state.

**Input:** A qubit in the $|0\rangle$ state.

**Goal:**  Change the state of the qubit to $|+\rangle = \frac{1}{\sqrt{2}} \big(|0\rangle + |1\rangle\big)$.

### Solution

Look up any list of quantum gates, for example, [Quantum logic gate @ Wikipedia](https://en.wikipedia.org/wiki/Quantum_gate). Typically one of the first gates described will be the [Hadamard gate](https://en.wikipedia.org/wiki/Quantum_logic_gate#Hadamard_(H)_gate):

$$H = \frac{1}{\sqrt2} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}$$

This gate converts $|0\rangle$ into $|+\rangle = \frac{1}{\sqrt{2}} \big(|0\rangle + |1\rangle\big)$ and $|1\rangle$ into $|−\rangle = \frac{1}{\sqrt{2}} \big(|0\rangle - |1\rangle\big)$.  The first of these transformations is exactly the one we're looking for!

To implement this operation in Q#, look at the list of [intrinsic gates](https://docs.microsoft.com/qsharp/api/qsharp/microsoft.quantum.intrinsic) available in Q#.
[Hadamard gate](https://docs.microsoft.com/en-us/qsharp/api/qsharp/microsoft.quantum.intrinsic.h) is one of them.
`Microsoft.Quantum.Intrinsic` namespace is open in Notebooks by default, so you can use the gates in the code right away.

```qsharp
%kata T01_PlusState_Test

operation PlusState (q : Qubit) : Unit {
    H(q);
}
```

[Return to task 1 of the Superposition kata.](./Superposition.ipynb#plus-state)

##  Task 2. Minus State.

**Input:** A qubit in the $|0\rangle$ state.

**Goal:**  Change the state of the qubit to $|-\rangle = \frac{1}{\sqrt{2}} \big(|0\rangle - |1\rangle\big)$.

### Solution

As we've seen in the previous task, the Hadamard gate maps the basis state $|0\rangle$ to $\frac{1}{\sqrt2}\big(|0\rangle + |1\rangle\big)$ and $|1\rangle$ to $\frac{1}{\sqrt2}\big(|0\rangle - |1\rangle\big)$.
If our qubit was already in the $|1\rangle$ state, we would simply apply the Hadamard gate to prepare the required $|-\rangle$ state.
However, there is another operation we can use to change the state $|0\rangle$ to $|1\rangle$, namely the [X gate](https://en.wikipedia.org/wiki/Quantum_logic_gate#Pauli-X_gate):

$$X = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$$

This gate transforms $|0\rangle \longmapsto |1\rangle$ and $|1\rangle \longmapsto |0\rangle$.

Here is the sequence of the steps to arrive to the solution:

    <col width=400>
    <col width=300>
    <col width=300>

        Description of operation
        Notation
        Circuit

        Apply the X gate to $|0\rangle$ to get $|1\rangle$
        $X|0\rangle = |1\rangle$

        Apply the Hadamard gate to $|1\rangle$ to get $\frac{1}{\sqrt2}\big(|0\rangle - |1\rangle\big)$
        $H|1\rangle = \frac{1}{\sqrt2}\big(|0\rangle - |1\rangle\big)$

In Q#, each gate is applied to the qubit sequentially, transforming its internal state. [X gate](https://docs.microsoft.com/en-us/qsharp/api/qsharp/microsoft.quantum.intrinsic.x) is another gate in the `Microsoft.Quantum.Intrinsic` namespace.

```qsharp
%kata T02_MinusState_Test

operation MinusState (q : Qubit) : Unit {
    X(q);
    H(q);
}
```

[Return to task 2 of the Superposition kata.](./Superposition.ipynb#minus-state)

##   Task 3. Unequal superposition.

**Input:**

1. A qubit in the $|0\rangle$ state.
2. Angle $\alpha$, in radians, represented as `Double`.

**Goal:**  Change the state of the qubit to $\cos\alpha|0\rangle$ + $\sin\alpha|1\rangle$.

### Solution

    <col width=130>
    <col width=300>
    <col width=130>
    <col width=230>
        We want to move from the starting state

        To the desired final state

In other words, we are looking for some kind of a rotation operation. There are three special gates that implement rotations around various axis of the Bloch Sphere:

    <col width=300>
    <col width=300>
    <col width=250>

        $R_x(\theta) = \begin{bmatrix} \cos\frac{\theta}{2} & -i\sin\frac{\theta}{2} \\ -i\sin\frac{\theta}{2} & \cos\frac{\theta}{2} \end{bmatrix}$
        $R_y(\theta) = \begin{bmatrix} \cos\frac{\theta}{2} & -\sin\frac{\theta}{2} \\ \sin\frac{\theta}{2} & \cos\frac{\theta}{2} \end{bmatrix}$
        $R_z(\theta) = \begin{bmatrix} e^{-i\theta/2} & 0 \\ 0 & e^{i\theta/2} \end{bmatrix}$

If we were to apply the $R_x$ gate to a qubit in the $|0\rangle$ state, we would introduce complex coefficients to the amplitudes, which is clearly not what we're looking for. Similarly, the $R_z$ gate introduces only a global phase when applied to $|0\rangle$ state, so we can rule it out as well. This leaves only the $R_y$ as a starting point for the solution.

Applying the $R_y$ gate to the $|0\rangle$ state, we get:
$$R_y(\theta) |0\rangle = \begin{bmatrix} \cos\frac{\theta}{2} & -\sin\frac{\theta}{2} \\ \sin\frac{\theta}{2} & \cos\frac{\theta}{2} \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} \cos\frac{\theta}{2} \\ \sin\frac{\theta}{2} \end{bmatrix} = \cos\frac{\theta}{2}|0\rangle + \sin\frac{\theta}{2}|1\rangle$$

Therefore, applying the $R_y(2\alpha$) gate to $|0\rangle$ is the solution to our problem.

>

    <col width=600>
    <col width=300>

Let us have a look at the Bloch sphere to obtain a geometrical interpretation of the operation.

The X-Z plane has no complex numbers, i.e. for qubit states that fall on it the amplitudes for both $|0\rangle$ and $|1\rangle$ basis states will always be real. We can see that the $R_y$ gate, which rotates the qubit state around the Y axis, keeps it within the X-Z plane.

Looking at the other two gates, $R_x$ gate rotates the qubit state around the X axis and thus takes the qubit state out of the X-Z plane, introducing complex coefficients. $R_z$ gate rotates the qubit state around the Z axis, so it doesn't modify the $|0\rangle$ state which lies on the Z axis.

In case you are surprised to see the $|0\rangle$ and $|1\rangle$ vectors 180 degrees apart from each other, the Bloch sphere represents orthogonal vectors as opposite points on the sphere.
You can learn more about the bloch sphere from this presentation

In Q#, $R_y$ gate is represented as [Ry](https://docs.microsoft.com/en-us/qsharp/api/qsharp/microsoft.quantum.intrinsic.ry) operation. Note that you have to apply $R_y(2\alpha)$, not $R_y(\alpha)$; Q# does not have implicit type casing from `Int` to `Double`, so you have to calculate the angle parameter of the operation as `2.0 * alpha`, using a double constant 2.0 instead of an integer 2.

```qsharp
%kata T03_UnequalSuperposition_Test

operation UnequalSuperposition (q : Qubit, alpha : Double) : Unit {
    Ry(2.0 * alpha, q);
}
```

[Return to task 3 of the Superposition kata.](./Superposition.ipynb#unequal-superposition)

## Task 4. Superposition of all basis vectors on two qubits.

**Input:** Two qubits in the $|00\rangle$ state (stored in an array of length 2).

**Goal:** Change the state of the qubits to $|+\rangle \otimes |+\rangle = \frac{1}{2} \big(|00\rangle + |01\rangle + |10\rangle + |11\rangle\big)$.

### Solution

We know that the Hadamard gate maps the basis state $|0\rangle$ to $\frac{1}{\sqrt2}(|0\rangle + |1\rangle)$, so it is a logical starting point for solving this problem.

Next, we see that the final state has a $\frac{1}{2}$ term hinting that we might be applying two operations involving a $\frac{1}{\sqrt{2}}$ term.

Now, how do we get the $|00\rangle + |01\rangle + |10\rangle + |11\rangle$ expression? Let's see what does multiplying the expression $|0\rangle + |1\rangle$ by itself look like:

$$\big(|0\rangle + |1\rangle\big) \otimes \big(|0\rangle + |1\rangle\big) = |0\rangle|0\rangle + |0\rangle|1\rangle + |1\rangle|0\rangle + |1\rangle|1\rangle = \\
= |00\rangle + |01\rangle + |10\rangle + |11\rangle$$

Thus, applying the Hadamard gate to each qubit in isolation will deliver the desired final result:

$$H|0\rangle \otimes H|0\rangle = \frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big) \otimes \frac{1}{\sqrt2}\big(|0\rangle + |1\rangle\big) = \\
= \frac{1}{2} (|00\rangle + |01\rangle + |10\rangle + |11\rangle)$$

Q# arrays are similar to arrays in other languages: you can access the $i$-th element of the array `qs` as `qs[i]` (indices are 0-based).

```qsharp
%kata T04_AllBasisVectors_TwoQubits_Test

operation AllBasisVectors_TwoQubits (qs : Qubit[]) : Unit {
    H(qs[0]);
    H(qs[1]);
}
```

[Return to task 4 of the Superposition kata.](./Superposition.ipynb#superposition-of-all-basis-vectors-on-two-qubits)

## Task 5. Superposition of basis vectors with phases.

**Input:** Two qubits in the $|00\rangle$ state (stored in an array of length 2).

**Goal:** Change the state of the qubits to $\frac{1}{2} \big(|00\rangle + i|01\rangle - |10\rangle - i|11\rangle\big)$.

### Solution

We will start approaching the problem from the desired end result. Let’s see if we can factor any expressions out of $\big(|00\rangle + i|01\rangle - |10\rangle - i|11\rangle\big)$:

\begin{equation}
|00\rangle + i|01\rangle - |10\rangle - i|11\rangle
= |00\rangle + \big(|0\rangle - |1\rangle\big) i|1\rangle - |10\rangle = \\
= \big(|0\rangle - |1\rangle\big) |0\rangle + \big(|0\rangle - |1\rangle\big) i|1\rangle
= \big(|0\rangle - |1\rangle\big) \otimes \big(|0\rangle + i|1\rangle\big)
\label{5.1} \tag{5.1}
\end{equation}

The fact that we were able to factor out the state into a tensor product of two terms means the state is separable.

This is looking promising.  Now let’s try to approach the problem from the other end, i.e. from the starting state of $|00\rangle$.
As we've seen in the previous task, applying a Hadamard operation to each $|0\rangle$ gets us closer to the factored-out expression:

\begin{equation}
H|0\rangle \otimes H|0\rangle = \frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big) \otimes \frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big)
=\frac{1}{2} \big(|0\rangle + |1\rangle\big) \otimes \big(|0\rangle + |1\rangle\big)
\label{5.2} \tag{5.2}
\end{equation}

If we compare equations 5.1 and 5.2 (while ignoring the $\frac{1}{2}$ term in equation 5.2), we end up with the following transformations that we need to perform on the individual qubits:

\begin{equation}
|0\rangle + |1\rangle \overset{???}\rightarrow |0\rangle - |1\rangle
\label{5.3} \tag{5.3}
\end{equation}

\begin{equation}
|0\rangle + |1\rangle \overset{???}\rightarrow |0\rangle + i|1\rangle
\label{5.4} \tag{5.4}
\end{equation}

Next lets take a look at our basic gates, in particular the Pauli Z gate:

$$Z = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$$

If it is applied to the state $\frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big)$, it will leave the basis state $|0\rangle$ unchanged and will map $|1\rangle$ to $-|1\rangle$. Thus,

$$Z\frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big) = \frac{1}{\sqrt2} \big(|0\rangle - |1\rangle\big)$$

So the Z gate is the answers to the question of how to do the conversion 5.3.

Looking for another gate to address the conversion 5.4, we find the S gate:

$$S = \begin{bmatrix} 1 & 0 \\ 0 & i \end{bmatrix}$$

If it is applied to the state $\frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big)$, it will leave the basis state $|0\rangle$ unchanged and will map $|1\rangle$ to $i|1\rangle$. Thus,

$$S\frac{1}{\sqrt2} \big(|0\rangle + |1\rangle\big) = \frac{1}{\sqrt2} \big(|0\rangle + i|1\rangle\big)$$

So the S gate now answers the question of how to do the conversion 5.4.

To summarize, the state we need to prepare can be represented as follows:
$$ZH|0\rangle \otimes SH|0\rangle$$

Remember that in Q# the gates have to be applied in reverse order compared to the mathematical notation - the gate closest to the ket symbol is applied first.

```qsharp
%kata T05_AllBasisVectorsWithPhases_TwoQubits_Test

operation AllBasisVectorsWithPhases_TwoQubits (qs : Qubit[]) : Unit {
    H(qs[0]);
    Z(qs[0]);

    H(qs[1]);
    S(qs[1]);
}
```

[Return to task 5 of the Superposition kata.](./Superposition.ipynb#superposition-of-basis-vectors-with-phases)

*Solutions to the rest of the tasks coming up soon...*


# Optimización de funciones escalares diferenciables con `SymPy`

> - Mediante optimización se obtienen soluciones elegantes tanto en teoría como en ciertas aplicaciones.
> - La teoría de optimización usa elementos comenzando con cálculo elemental y álgebra lineal básica, y luego se extiende con análisis funcional y convexo.
> - Las aplicaciones en optimización involucran ciencia, ingeniería, economía, finanzas e industria.
> - El amplio y creciente uso de la optimización lo hace escencial para estudiantes y profesionales de cualquier rama de la ciencia y la tecnología.

**Referencia:**
- http://www.math.uwaterloo.ca/~hwolkowi//henry/reports/talks.d/t06talks.d/06msribirs.d/optimportance.shtml

Algunas aplicaciones son:

1. Ingeniería
 - Encontrar la composición de equilibrio de una mezcla de diferentes átomos.
 - Planeación de ruta para un robot (o vehículo aéreo no tripulado).
 - Planeación de la mano de obra óptima en una construcción o planta de producción.
2. Distribución óptima de recursos.
 - Distribución de rutas de vuelo.
 - Encontrar una dieta óptima.
 - Planeación de ruta óptima.
3. Optimización financiera
 - Administración de riesgos.
 - Portafolios de inversión.

En esta clase veremos aspectos básicos de optimización. En específico, veremos cómo obtener máximos y mínimos de una función escalar de una variable (como en cálculo diferencial).

___
## 0. Librerías que usaremos

Como ya dijimos en la primer clase `python` es el lenguaje de programación (el cual es de alto nivel). Sin embargo, `python` solo tiene unos pocos comandos primitivos y para hacer más fácil su uso en nuestras actividades de simulación en ingeniería, otras personas ya han escrito ciertas librerías por nosotros.

### 0.1 `NumPy`

`NumPy` (Numerical Python) es la librería fundamental para computación científica (numérica) con `Python`. Contiene, entre otras cosas:
- un objeto tipo arreglo N-dimensional muy poderoso
- funciones sofisticadas
- funciones de álgebra lineal, transformada de Fourier y números aleatorios.

Por lo anterior, `NumPy` es de amplio uso entre la comunidad científica e ingenieril (por su manejo de cantidades vectoriales). De la misma manera, se usa para guardar datos. Para nuestros propósitos, se puede usar libremente.

**Referencia:**
- http://www.numpy.org/

`NumPy` ya viene incluido en la instalación estándar de Anaconda por defecto. Para comenzar a usarlo, solo debemos de importarlo:

```python
# importar la librería numpy
import numpy as np
```

### 0.2 `SymPy`

`SymPy` (Symbolic Python) es una librería de `Python` para matemáticas simbólicas. Su objetivo es convertirse en un sistema de álgebra computacional con las mejores características, manteniendo el código lo más simple posible para que sea comprensible.

**Referencia:**
- http://www.sympy.org/en/index.html

`SymPy` ya viene incluido en la instalación estándar de Anaconda por defecto. Para comenzar a usarlo, solo debemos de importarlo:

```python
# importar la librería sympy
import sympy as sym
# imprimir en formato latex
sym.init_printing(use_latex='mathjax')
```

La funcionalidad de imprimir en formato LaTex que nos da `SymPy` mediante el proyecto `mathjax` hace de `SymPy` una herramienta muy atractiva...

Notar que en `SymPy` y en `NumPy` existen funciones con el mismo nombre, pero reciben tipos de datos diferentes...

Explicar el uso de la sintaxis `from numpy import *` y sus peligros (no recomendable).

```python
# diferencias de funciones de sympy y numpy
sym.var('x')
sym.sin(x)*np.sin(3)
```

$$0.141120008059867 \sin{\left (x \right )}$$

```python
# importar con * y ver que pasa
# from numpy import *
# from sympy import *
# No recomendado
```

### 0.3 `PyPlot` de `matplotlib`

El módulo `PyPlot` de la librería `matplotlib` contiene funciones que nos permite generar una gran cantidad de gráficas rápidamente. Las funciones de este módulo están escritas con el mismo nombre que las funciones para graficar en `Matlab`.

**Referencia:**
- https://matplotlib.org/api/pyplot_summary.html

```python
# importar matplotlib.pyplot
import matplotlib.pyplot as plt
# comando para que las gráficas salgan en la misma ventana
%matplotlib inline
```

Ya que revisamos todas las librerías que usaremos, empecemos con la clase como tal...

___
Basamos todos los resultados en los siguientes teoremas:

## 1. Teorema de Fermat (análisis)

Si una función $f(x)$ alcanza un máximo o mínimo local en $x=c$, y si la derivada $f'(c)$ existe en el punto $c$, entonces $f'(c) = 0$.

### Ejemplo

Sabemos que la función $f(x)=x^2$ tiene un mínimo global en $x=0$, pues

$$f(x)=x^2\geq0,\qquad\text{y}\qquad f(x)=x^2=0 \qquad\text{si y solo si}\qquad x=0.$$

```python
# declarar la variable real x
#t = sym.Symbol('x')
#t
sym.var('x', real = True)
```

$$x$$

```python
# declarar ahora f=x^2 y mostrar
f = x**2 + sym.sin(7*x)
f
```

$$x^{2} + \sin{\left (7 x \right )}$$

```python
# derivar f respecto a x y mostrar
df = sym.diff(f, x)
df
```

$$2 x$$

```python
# resolver f'(x)=0 y mostrar soluciones
xc = sym.solve(df, x)
xc
```

$$\left [ 0\right ]$$

Veamos la gráfica...

```python
# convertir f e una función que se pueda evaluar numéricamente (función lambdify de la librería sympy)
f_num = sym.lambdify([x], f, 'numpy')
x_vec = np.linspace(-5, 5, 100)

# graficar
plt.plot(x_vec, f_num(x_vec))
plt.xlabel('$x$')
plt.ylabel('$x^2$')
plt.show()
```

Ver diferencias entre f y f_num

```python
# intentar evaluar f y f_num

```

**Otra manera de hacer lo anterior**

Concepto de función...

```python
def f(x):
    return x**2
```

```python
f(x)
```

$$x^{2}$$

```python
f(np.array([5.124, 2.5436]))
```

    array([26.255376  ,  6.46990096])

```python
df = sym.diff(f(x), x)
df
```

$$2 x$$

```python
x_c = sym.solve(df, x)
x_c[0]
```

$$0$$

```python
# graficar
plt.plot(x_vec, f(x_vec))
plt.xlabel('$x$')
plt.ylabel('$x^2$')
plt.show()
```

El converso del teorema anterior no es cierto.

### Actividad
Considere $g(x)=x^3$.
- Usando `sympy`, muestre que $g'(0)=0$.
- Sin embargo, descartar que $x=0$ es un extremo de $g(x)$ viendo su **gráfica**.

```python
def g(x):
    return x**3
```

```python
dg = sym.diff(g(x), x)
dg
```

$$3 x^{2}$$

```python
puntos_criticos = sym.solve(dg, x)
puntos_criticos
```

$$\left [ 0\right ]$$

```python
dg_eval0 = dg.subs(x, puntos_criticos[0])
dg_eval0
```

$$0$$

```python
# graficar
plt.plot(x_vec, g(x_vec))
plt.xlabel('$x$')
plt.ylabel('$x^3$')
plt.show()
```

## 2. Criterio de la segunda derivada

Sea $f(x)$ una función tal que $f’(c)=0$ y cuya segunda derivada existe en un intervalo abierto que contiene a $c$.
- Si $f’’(c)>0$, entonces $f(c)$ es un mínimo relativo.
- Si $f’’(c)<0$, entonces $f(c)$ es un máximo relativo.
- Si $f’’(c)=0$, entonces el criterio no decide.

### Ejemplo

Mostrar, usando `sympy`, que la función $f(x)=x^2$ tiene un mínimo relativo en $x=0$.

Ya vimos que $f'(0)=0$. Notemos que:

```python
f = x**2
#d2f = sym.diff(f, x, x)
d2f = sym.diff(f, x, 2)
d2f
```

$$2$$

```python
d2f>0
```

$$\mathrm{True}$$

Por tanto, por el criterio de la segunda derivada, $f(0)=0$ es un mínimo relativo (en efecto, el mínimo global).

### Ejemplo
¿Qué pasa con $g(x)=x^3$ al intentar utilizar el criterio de la segunda derivada? (usar `sympy`).

```python
g(x)
```

$$x^{3}$$

```python
d2g = sym.diff(g(x), x, x)
d2g
```

$$6 x$$

```python
d2g.subs(x, 0)
```

$$0$$

### Actividad

¿Qué pasa con $h(x)=x^4$ al intentar utilizar el criterio de la segunda derivada?.

```python
def h(x):
    return x**4
```

```python
dh = sym.diff(h(x), x, 1)
dh
```

$$4 x^{3}$$

```python
p_c_h = sym.solve(dh, x)
p_c_h
```

$$\left [ 0\right ]$$

```python
d2h = sym.diff(h(x), x, 2)
d2h
```

$$12 x^{2}$$

```python
d2h.subs(x, p_c_h[0])
```

$$0$$

Como la segunda derivada en el punto crítico es cero, el criterio de la segunda derivada no decide.

## 3. Método para determinar extremos absolutos de una función continua y=f(x) en [a,b]

- Determinar todos los valores críticos $c_1, c_2, c_3, \dots, c_n$ en $(a,b)$.
- Evaluar $f$ en todos los valores críticos y en los extremos $x=a$ y $x=b$.
- El más grande y el más pequeño de los valores de la lista $f(a), f(b), f(c_1), f(c_2), \dots, f(c_n)$ son el máximo absoluto y el mínimo absoluto, respectivamente, de f en el intervalo [a,b].

### Ejemplo

Determinar los extremos absolutos de $f(x)=x^2-6x$ en $\left[0,5\right]$.

Obtenemos los puntos críticos de $f$ en $\left[0,5\right]$:

```python
f = x**2-6*x
f
```

$$x^{2} - 6 x$$

```python
df = sym.diff(f, x)
df
```

$$2 x - 6$$

```python
x_c = sym.solve(df, x)
x_c
```

$$\left [ 3\right ]$$

Evaluamos $f$ en los extremos y en los puntos críticos:

```python
f.subs(x, 0), f.subs(x, 5), f.subs(x, x_c[0])
```

$$\left ( 0, \quad -5, \quad -9\right )$$

Concluimos que el máximo absoluto de $f$ en $\left[0,5\right]$ es $0$ y se alcanza en $x=0$, y que el mínimo absoluto es $-9$ y se alcanza en $x=3$.

```python
f_num = sym.lambdify([x], f, 'numpy')
x_vec = np.linspace(0, 5, 100)

plt.figure(figsize=(8,6))
plt.plot(x_vec, f_num(x_vec), 'k--', label = '$y=f(x)$')
plt.plot([0], [0], '*r', label = '$(0,0=\max_{0\leq x\leq 5} f(x))$')
plt.plot([3], [-9], '*b', label = '$(3,-9=\min_{0\leq x\leq 5} f(x))$')
plt.legend(loc='best')
plt.xlabel('x')
plt.show()
```

### Actividad
Determinar los valores extremos absolutos de $h(x)=x^3-3x$ en $\left[-2.2,1.8\right]$, usando `sympy`. Mostrar en una gráfica.

```python
sym.var('x', real = True)
```

$$x$$

```python
def h(x):
    return x**3-3*x
```

```python
dh = sym.diff(h(x), x)
dh
```

$$3 x^{2} - 3$$

```python
p_c = sym.solve(dh, x)
p_c
```

$$\left [ -1, \quad 1\right ]$$

```python
h(-1), h(1), h(-2.2), h(1.8)
```

$$\left ( 2, \quad -2, \quad -4.048000000000003, \quad 0.4320000000000004\right )$$

```python
x_vec = np.linspace(-2.2, 1.8, 50)

plt.figure(figsize=(7,5))
plt.plot(x_vec, h(x_vec), 'b:', label = '$y=x^3-3x$')
plt.plot([-2.2], [h(-2.2)], 'mo', label = '(-2.2,-4.048)')
plt.plot([-1], [h(-1)], 'yo', label = '(-1,2)')
plt.legend(loc='best')
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.grid()
plt.show()
```

### En varias variables...

El procedimiento es análogo.

Si una función $f:\mathbb{R}^n\to\mathbb{R}$ alcanza un máximo o mínimo local en $\boldsymbol{x}=\boldsymbol{c}\in\mathbb{R}^n$, y $f$ es diferenciable en el punto $\boldsymbol{x}=\boldsymbol{c}$, entonces $\left.\frac{\partial f}{\partial \boldsymbol{x}}\right|_{\boldsymbol{x}=\boldsymbol{c}}=\boldsymbol{0}$ (todas las derivadas parciales en el punto $\boldsymbol{x}=\boldsymbol{c}$ son cero).

**Criterio de la segunda derivada:** para ver si es máximo o mínimo, se toma la segunda derivada (matriz jacobiana) y se verifica definición negativa o positiva, respectivamente.

Teorema Sea $f$ un campo escalar con segundas derivadas continuas $D_{ij}$ en una bola $B(a,\varepsilon)$ entonces:

a) Si todos los valores propios de $H(a)$ son positivos, $f$ tiene un mínimo relativo en $a$.

b) Si todos los valores propios de $H(a)$ son negativos, $f$ tiene un máximo relativo en $a$.

c) Si $H(a)$ tiene valores propios positivos y negativos, $f$ tiene un punto de ensilladura en $a$.

La demostración la pueden encontrar en (CALCULUS volumen 2. Tom M.Apostol, pág. 311)

Si se restringe a cierta región, hay ciertas técnicas. La más general, pero también la más compleja es la de **multiplicadores de Lagrange**.

```python
sym.var('x y')
x, y
```

$$\left ( x, \quad y\right )$$

```python
def f(x, y):
    return x**2 + y**2
```

```python
dfx = sym.diff(f(x,y), x)
dfy = sym.diff(f(x,y), y)
dfx, dfy
```

$$\left ( 2 x, \quad 2 y\right )$$

```python
xy_c = sym.solve([dfx, dfy], [x, y])
xy_c
```

$$\left \{ x : 0, \quad y : 0\right \}$$

```python
x_c, y_c = xy_c[x], xy_c[y]
x_c, y_c
```

$$\left ( 0, \quad 0\right )$$

```python
d2fx = sym.diff(f(x,y), x, 2)
d2fy = sym.diff(f(x,y), y, 2)
dfxy = sym.diff(f(x,y), x, y)

Jf = sym.Matrix([[d2fx, dfxy], [dfxy, d2fy]])
Jf.eigenvals()
```

$$\left \{ 2 : 2\right \}$$

```python
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
```

```python
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x = np.linspace(-2, 2, 100)
y = x
X, Y = np.meshgrid(x, y)

ax.plot_surface(X, Y, f(X, Y))
ax.plot([x_c], [y_c], [f(x_c,y_c)], '*r')
```

Created with Jupyter by Xiuhcoatl.


# Part 0: Hello Qiskit

While skip talking about how a quantum computer is important and hot in recent days, let's jump into the quantum circuit directly by using Qiskit.

Qiskit is an open-source SDK for working with quantum computers at the level of pulses, circuits, and algorithms. Qiskit supports many quantum backends - IBM Quantum, IonQ, AQT, Honeywell and etc and quantum cloud - IBM Quantum Experience, Microsoft Azure Quantum. Qiskit's great contribution to the quantum computing ecosystem is its huge and active developer community and we will talk about this at the 29th Oct lecture.

With this 1st lecture, we will demonstrate:

- Part 1: Qiskit Basic (10min)
    - Basic Quantum Gates and Operations
    - Exercise: Quantum State manipulating with the gates - Bell State
- Part 2: Tutorial with GHZ Circuit (30 min)
    - Compose 3-qubits GHZ quantum circuits
    - simulating them on various backends, and visualizing the results
- Part 3: Introduction of the Quantum Information Theory (30min)
    - Phase Kickback
    - the Deutsch Jozsa algorithm
- Exercise (20min):
    - explore easy physics model by using qiskit with assignment1

# Part1 : Qiskit Basic - Basic Quantum Gates and Operations

## Single Qubit Quantum states

A single qubit quantum state can be written as

$$\left|\psi\right\rangle = \alpha\left|0\right\rangle + \beta \left|1\right\rangle$$

where $\alpha$ and $\beta$ are complex numbers. In a measurement the probability of the bit being in $\left|0\right\rangle$ is $|\alpha|^2$ and $\left|1\right\rangle$ is $|\beta|^2$. As a vector this is

$$
\left|\psi\right\rangle =
\begin{pmatrix}
\alpha \\
\beta
\end{pmatrix}.
$$

Note, due to the conservation of probability $|\alpha|^2+ |\beta|^2 = 1$ and since global phase is undetectable $\left|\psi\right\rangle := e^{i\delta} \left|\psi\right\rangle$ we only require two real numbers to describe a single qubit quantum state.

A convenient representation is

$$\left|\psi\right\rangle = \cos(\theta/2)\left|0\right\rangle + \sin(\theta/2)e^{i\phi}\left|1\right\rangle$$

where $0\leq \phi < 2\pi$, and $0\leq \theta \leq \pi$.  From this, it is clear that there is a one-to-one correspondence between qubit states ($\mathbb{C}^2$) and the points on the surface of a unit sphere ($\mathbb{R}^3$). This is called the Bloch sphere representation of a qubit state.

Quantum gates/operations are usually represented as matrices. A gate which acts on a qubit is represented by a $2\times 2$ unitary matrix $U$. The action of the quantum gate is found by multiplying the matrix representing the gate with the vector which represents the quantum state.

$$\left|\psi'\right\rangle = U\left|\psi\right\rangle$$

A general unitary must be able to take the $\left|0\right\rangle$ to the above state. That is

$$
U = \begin{pmatrix}
\cos(\theta/2) & a \\
e^{i\phi}\sin(\theta/2) & b
\end{pmatrix}
$$

where $a$ and $b$ are complex numbers constrained such that $U^\dagger U = I$ for all $0\leq\theta\leq\pi$ and $0\leq \phi<2\pi$. This gives 3 constraints and as such $a\rightarrow -e^{i\lambda}\sin(\theta/2)$ and $b\rightarrow e^{i\lambda+i\phi}\cos(\theta/2)$ where $0\leq \lambda<2\pi$ giving

$$
U = \begin{pmatrix}
\cos(\theta/2) & -e^{i\lambda}\sin(\theta/2) \\
e^{i\phi}\sin(\theta/2) & e^{i\lambda+i\phi}\cos(\theta/2)
\end{pmatrix}.
$$

This is the most general form of a single qubit unitary.

Qiskit support many single-qubi gates. You can check the whole list of them at here:
https://qiskit.org/textbook/ch-states/single-qubit-gates.html

The most frequent used  and famouse gate and it's matrix forms are examine below:

```python
from qiskit import QuantumCircuit, execute, Aer, IBMQ
from qiskit.quantum_info import state_fidelity
from qiskit.visualization import *
import numpy as np

#ignore warnings
import warnings
warnings.filterwarnings('ignore')

# Magic function to render plots
%matplotlib inline
```

#### U3 gates

In Qiskit we give you access to the general unitary using the $u3$ gate

$$
u3(\theta, \phi, \lambda) = U(\theta, \phi, \lambda)
$$

```python
u_gate = QuantumCircuit(1)
u_gate.u(np.pi/2,np.pi/2,np.pi/2,0)
u_gate.draw('mpl')
```

### Pauli gates

#### $X$: bit-flip gate

The bit-flip gate $X$ is defined as:

$$
X   =
\begin{pmatrix}
0 & 1\\
1 & 0
\end{pmatrix}= u3(\pi,0,\pi)
$$

```python
x_gate=QuantumCircuit(1) # Create a quantum circuit with 1 qubit
x_gate.x(0)
x_gate.draw(output='mpl')
```

#### $Y$: bit- and phase-flip gate

The $Y$ gate is defined as:

$$
Y  =
\begin{pmatrix}
0 & -i\\
i & 0
\end{pmatrix}=u3(\pi,\pi/2,\pi/2)
$$

```python
y_gate = QuantumCircuit(1)
y_gate.y(0)
y_gate.draw(output='mpl')
```

#### $Z$: phase-flip gate

The phase-flip gate $Z$ is defined as:

$$
Z =
\begin{pmatrix}
1 & 0\\
0 & -1
\end{pmatrix}=u1(\pi)
$$

```python
z_gate = QuantumCircuit(1)
z_gate.z(0)
z_gate.draw(output='mpl')
```

### Clifford gates
the Clifford gates are the elements of the Clifford group, a set of mathematical transformations which effect permutations of the Pauli operators.

#### Hadamard gate

A Hadamard gate represents a rotation of $\pi$ about the axis that is in the middle of the X-axis and Z-axis.
It maps the basis state $|0\rangle$ to $\frac{|0\rangle + |1\rangle}{\sqrt{2}}$, which means that a measurement will have equal probabilities of being `1` or `0`, creating a 'superposition' of states. This state is also written as $|+\rangle$. What the Hadamard does is to transform between the $|0\rangle$ $|1\rangle$ and the $|+\rangle$ $|-\rangle$ base.

$$
H =
\frac{1}{\sqrt{2}}
\begin{pmatrix}
1 & 1\\
1 & -1
\end{pmatrix}= u2(0,\pi)
$$

```python
# Let's do an H-gate on a |0> qubit
h_gate = QuantumCircuit(1)
h_gate.h(0)
h_gate.draw(output='mpl')
```

#### $S$ (or, $\sqrt{Z}$ phase) gate

$$
S =
\begin{pmatrix}
1 & 0\\
0 & i
\end{pmatrix}= u1(\pi/2)
$$

```python
s_gate = QuantumCircuit(1)
s_gate.s(0)
s_gate.draw(output='mpl')
```

#### $S^{\dagger}$ (or, conjugate of $\sqrt{Z}$ phase) gate

$$
S^{\dagger} =
\begin{pmatrix}
1 & 0\\
0 & -i
\end{pmatrix}= u1(-\pi/2)
$$

```python
sdg_gate = QuantumCircuit(1)
sdg_gate.sdg(0)
sdg_gate.draw(output='mpl')
```

### Standard Rotations

The standard rotation gates are those that define rotations around the Paulis $P=\{X,Y,Z\}$. They are defined as

$$ R_P(\theta) = \exp(-i \theta P/2) = \cos(\theta/2)I -i \sin(\theta/2)P$$

#### Rotation around X-axis

$$
R_x(\theta) =
\begin{pmatrix}
\cos(\theta/2) & -i\sin(\theta/2)\\
-i\sin(\theta/2) & \cos(\theta/2)
\end{pmatrix} = u3(\theta, -\pi/2,\pi/2)
$$

```python
rx_gate = QuantumCircuit(1)
rx_gate.rx(2,0)
rx_gate.draw(output='mpl')
```

#### Rotation around Y-axis

$$
R_y(\theta) =
\begin{pmatrix}
\cos(\theta/2) & - \sin(\theta/2)\\
\sin(\theta/2) & \cos(\theta/2).
\end{pmatrix} =u3(\theta,0,0)
$$

```python
ry_gate = QuantumCircuit(1)
ry_gate.ry(1,0)
ry_gate.draw(output='mpl')
```

#### Rotation around Z-axis

$$
R_z(\phi) =
\begin{pmatrix}
e^{-i \phi/2} & 0 \\
0 & e^{i \phi/2}
\end{pmatrix}\equiv u1(\phi)
$$

Note that here we have used an equivalent as it is different to u1 by a global phase $e^{-i \phi/2}$.

```python
rz_gate = QuantumCircuit(1)
rz_gate.rz(np.pi/2,0)
rz_gate.draw(output='mpl')
```

## Multi-Qubit Gates

### Mathematical Preliminaries

The space of a quantum computer grows exponentially with the number of qubits. For $n$ qubits the complex vector space has dimension $d=2^n$. To describe states of a multi-qubit system, the tensor product is used to "glue together" operators and basis vectors.

Let's start by considering a 2-qubit system. Given two operators $A$ and $B$ that each act on one qubit, the joint operator $A \otimes B$ acting on two qubits is

$$\begin{equation}
	A\otimes B =
	\begin{pmatrix}
		A_{00} \begin{pmatrix}
			B_{00} & B_{01} \\
			B_{10} & B_{11}
		\end{pmatrix} & A_{01} 	\begin{pmatrix}
				B_{00} & B_{01} \\
				B_{10} & B_{11}
			\end{pmatrix} \\
		A_{10} 	\begin{pmatrix}
					B_{00} & B_{01} \\
					B_{10} & B_{11}
				\end{pmatrix} & A_{11} 	\begin{pmatrix}
							B_{00} & B_{01} \\
							B_{10} & B_{11}
						\end{pmatrix}
	\end{pmatrix},
\end{equation}$$

where $A_{jk}$ and $B_{lm}$ are the matrix elements of $A$ and $B$, respectively.

Analogously, the basis vectors for the 2-qubit system are formed using the tensor product of basis vectors for a single qubit:
$$\begin{equation}\begin{split}
	\left|{00}\right\rangle &= \begin{pmatrix}
		1 \begin{pmatrix}
			1  \\
			0
		\end{pmatrix} \\
		0 \begin{pmatrix}
			1  \\
			0
		\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 0 \\0 \end{pmatrix}~~~\left|{01}\right\rangle = \begin{pmatrix}
	1 \begin{pmatrix}
	0 \\
	1
	\end{pmatrix} \\
	0 \begin{pmatrix}
	0  \\
	1
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix}0 \\ 1 \\ 0 \\ 0 \end{pmatrix}\end{split}
\end{equation}$$

$$\begin{equation}\begin{split}\left|{10}\right\rangle = \begin{pmatrix}
	0\begin{pmatrix}
	1  \\
	0
	\end{pmatrix} \\
	1\begin{pmatrix}
	1 \\
	0
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix}~~~ 	\left|{11}\right\rangle = \begin{pmatrix}
	0 \begin{pmatrix}
	0  \\
	1
	\end{pmatrix} \\
	1\begin{pmatrix}
	0  \\
	1
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\1 \end{pmatrix}\end{split}
\end{equation}.$$

Note we've introduced a shorthand for the tensor product of basis vectors, wherein $\left|0\right\rangle \otimes \left|0\right\rangle$ is written as $\left|00\right\rangle$. The state of an $n$-qubit system can be described using the $n$-fold tensor product of single-qubit basis vectors. Notice that the basis vectors for a 2-qubit system are 4-dimensional; in general, the basis vectors of an $n$-qubit sytsem are $2^{n}$-dimensional, as noted earlier.

### Basis vector ordering in Qiskit

Within the physics community, the qubits of a multi-qubit systems are typically ordered with the first qubit on the left-most side of the tensor product and the last qubit on the right-most side. For instance, if the first qubit is in state $\left|0\right\rangle$ and second is in state $\left|1\right\rangle$, their joint state would be $\left|01\right\rangle$. Qiskit uses a slightly different ordering of the qubits, in which the qubits are represented from the most significant bit (MSB) on the left to the least significant bit (LSB) on the right (big-endian). This is similar to bitstring representation on classical computers, and enables easy conversion from bitstrings to integers after measurements are performed. For the example just given, the joint state would be represented as $\left|10\right\rangle$. Importantly, *this change in the representation of multi-qubit states affects the way multi-qubit gates are represented in Qiskit*, as discussed below.

The representation used in Qiskit enumerates the basis vectors in increasing order of the integers they represent. For instance, the basis vectors for a 2-qubit system would be ordered as $\left|00\right\rangle$, $\left|01\right\rangle$, $\left|10\right\rangle$, and $\left|11\right\rangle$. Thinking of the basis vectors as bit strings, they encode the integers 0,1,2 and 3, respectively.

### Controlled operations on qubits

A common multi-qubit gate involves the application of a gate to one qubit, conditioned on the state of another qubit. For instance, we might want to flip the state of the second qubit when the first qubit is in $\left|0\right\rangle$. Such gates are known as _controlled gates_. The standard multi-qubit gates consist of two-qubit gates and three-qubit gates. The two-qubit gates are:
- controlled Pauli gates
- controlled Hadamard gate
- controlled rotation gates
- controlled phase gate
- controlled u3 gate
- swap gate

The three-qubit gates are:
- Toffoli gate
- Fredkin gate

## Two-qubit gates

Most of the two-qubit gates are of the controlled type (the SWAP gate being the exception). In general, a controlled two-qubit gate $C_{U}$ acts to apply the single-qubit unitary $U$ to the second qubit when the state of the first qubit is in $\left|1\right\rangle$. Suppose $U$ has a matrix representation

$$U = \begin{pmatrix} u_{00} & u_{01} \\ u_{10} & u_{11}\end{pmatrix}.$$

We can work out the action of $C_{U}$ as follows. Recall that the basis vectors for a two-qubit system are ordered as $\left|00\right\rangle, \left|01\right\rangle, \left|10\right\rangle, \left|11\right\rangle$. Suppose the **control qubit** is **qubit 0** (which, according to Qiskit's convention, is one the _right-hand_ side of the tensor product). If the control qubit is in $\left|1\right\rangle$, $U$ should be applied to the **target** (qubit 1, on the _left-hand_ side of the tensor product). Therefore, under the action of $C_{U}$, the basis vectors are transformed according to

$$\begin{align*}
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{U\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{U\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
\end{align*}.$$

In matrix form, the action of $C_{U}$ is

$$\begin{equation}
	C_U = \begin{pmatrix}
	1 & 0 & 0 & 0 \\
	0 & u_{00} & 0 & u_{01} \\
	0 & 0 & 1 & 0 \\
	0 & u_{10} &0 & u_{11}
		\end{pmatrix}.
\end{equation}$$

To work out these matrix elements, let

$$C_{(jk), (lm)} = \left(\underset{\text{qubit}~1}{\left\langle j \right|} \otimes \underset{\text{qubit}~0}{\left\langle k \right|}\right) C_{U} \left(\underset{\text{qubit}~1}{\left| l \right\rangle} \otimes \underset{\text{qubit}~0}{\left| k \right\rangle}\right),$$

compute the action of $C_{U}$ (given above), and compute the inner products.

As shown in the examples below, this operation is implemented in Qiskit as `cU(q[0],q[1])`.

If **qubit 1 is the control and qubit 0 is the target**, then the basis vectors are transformed according to
$$\begin{align*}
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{U\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{U\left|1\right\rangle}\\
\end{align*},$$

which implies the matrix form of $C_{U}$ is
$$\begin{equation}
	C_U = \begin{pmatrix}
	1 & 0 & 0  & 0 \\
	0 & 1 & 0 & 0 \\
	0 & 0 & u_{00} & u_{01} \\
	0 & 0 & u_{10} & u_{11}
		\end{pmatrix}.
\end{equation}$$

### Controlled Pauli Gates

#### Controlled-X (or, controlled-NOT) gate
The controlled-not gate flips the `target` qubit when the control qubit is in the state $\left|1\right\rangle$. If we take the MSB as the control qubit (e.g. `cx(q[1],q[0])`), then the matrix would look like

$$
C_X =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0
\end{pmatrix}.
$$

However, when the LSB is the control qubit, (e.g. `cx(q[0],q[1])`), this gate is equivalent to the following matrix:

$$
C_X =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0\\
0 & 1 & 0 & 0
\end{pmatrix}.
$$

```python
cx_gate = QuantumCircuit(2)
cx_gate.cx(0,1)
cx_gate.draw(output='mpl')
```

#### Controlled $Y$ gate

Apply the $Y$ gate to the target qubit if the control qubit is the MSB

$$
C_Y =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & -i\\
0 & 0 & i & 0
\end{pmatrix},
$$

or when the LSB is the control

$$
C_Y =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 0 & -i\\
0 & 0 & 1 & 0\\
0 & i & 0 & 0
\end{pmatrix}.
$$

```python
cy_gate = QuantumCircuit(2)
cy_gate.cy(0,1)
cy_gate.draw(output='mpl')
```

#### Controlled $Z$ (or, controlled Phase) gate

Similarly, the controlled Z gate flips the phase of the target qubit if the control qubit is $\left|1\right\rangle$. The matrix looks the same regardless of whether the MSB or LSB is the control qubit:

$$
C_Z =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & -1
\end{pmatrix}
$$

```python
cz_gate = QuantumCircuit(2)
cz_gate.cz(0,1)
cz_gate.draw(output='mpl')
```

### Controlled $u3$ rotation

Perform controlled-$u3$ rotation on the target qubit if the control qubit (here LSB) is $\left|1\right\rangle$.

$$
C_{u3}(\theta, \phi, \lambda) \equiv
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & e^{-i(\phi+\lambda)/2}\cos(\theta/2) & 0 & -e^{-i(\phi-\lambda)/2}\sin(\theta/2)\\
0 & 0 & 1 & 0\\
0 & e^{i(\phi-\lambda)/2}\sin(\theta/2) & 0 & e^{i(\phi+\lambda)/2}\cos(\theta/2)
\end{pmatrix}.
$$

```python
cu_gate = QuantumCircuit(2)
cu_gate.cu3(0.1, 1, 2, 0, 1)
cu_gate.draw(output='mpl')
```

### SWAP gate

The SWAP gate exchanges the two qubits. It transforms the basis vectors as

$$\left|00\right\rangle \rightarrow \left|00\right\rangle~,~\left|01\right\rangle \rightarrow \left|10\right\rangle~,~\left|10\right\rangle \rightarrow \left|01\right\rangle~,~\left|11\right\rangle \rightarrow \left|11\right\rangle,$$

which gives a matrix representation of the form

$$
\mathrm{SWAP} =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & 1
\end{pmatrix}.
$$

```python
swap_gate = QuantumCircuit(2)
swap_gate.swap(0, 1)
swap_gate.draw(output='mpl')
```

## Non-unitary operations

Now that we have gone through all the unitary operations in quantum circuits, we also have access to non-unitary operations. These include measurements, reset of qubits, and classical conditional operations.

### Measurements

We don't have access to all the information when we make a measurement in a quantum computer. The quantum state is projected onto the standard basis. Below are two examples showing a circuit that is prepared in a basis state and the quantum computer prepared in a superposition state.

```python
qc = QuantumCircuit(2)
qc.measure_all()
qc.draw(output='mpl')
```

### Reset
It is also possible to `reset` qubits to the $\left|0\right\rangle$ state in the middle of computation. Note that `reset` is not a Gate operation, since it is irreversible.

```python
qc = QuantumCircuit(1)
qc.x(0)
qc.reset(0)
qc.measure_all()
qc.draw(output='mpl')
```

### Conditional operations
It is also possible to do operations conditioned on the state of the classical register

```python
from qiskit import ClassicalRegister, QuantumRegister

q = QuantumRegister(1)
c = ClassicalRegister(1)
qc = QuantumCircuit(q,c)
qc.x(q[0]).c_if(c, 0)
qc.measure(0,0)
qc.draw(output='mpl')
```

## Arbitrary initialization
What if we want to initialize a qubit register to an arbitrary state? An arbitrary state for $n$ qubits may be specified by a vector of $2^n$ amplitudes, where the sum of amplitude-norms-squared equals 1. For example, the following three-qubit state can be prepared:

$$\left|\psi\right\rangle = \frac{i}{4}\left|000\right\rangle + \frac{1}{\sqrt{8}}\left|001\right\rangle + \frac{1+i}{4}\left|010\right\rangle + \frac{1+2i}{\sqrt{8}}\left|101\right\rangle + \frac{1}{4}\left|110\right\rangle$$

```python
# Initializing a three-qubit quantum state
import math
desired_vector = [
    1 / math.sqrt(16) * complex(0, 1),
    1 / math.sqrt(8) * complex(1, 0),
    1 / math.sqrt(16) * complex(1, 1),
    0,
    0,
    1 / math.sqrt(8) * complex(1, 2),
    1 / math.sqrt(16) * complex(1, 0),
    0]

init_qc = QuantumCircuit(3)
init_qc.initialize(desired_vector, [0,1,2])
init_qc.draw(output='mpl')
```

# Part1 Exercise: Quantum State manipulating with the gates - Bell State

There are four Bell states given by

$$ |\Phi^+\rangle = \frac{1}{\sqrt{2}}\left(|0\rangle\otimes |0\rangle + |1\rangle\otimes|1\rangle\right)$$
$$ |\Phi^-\rangle = \frac{1}{\sqrt{2}}\left(|0\rangle\otimes |0\rangle - |1\rangle\otimes|1\rangle\right)$$
$$ |\Psi^+\rangle = \frac{1}{\sqrt{2}} \left(|0\rangle\otimes |1\rangle + |1\rangle\otimes|0\rangle\right)$$
$$ |\Psi^-\rangle = \frac{1}{\sqrt{2}} \left(|0\rangle\otimes |1\rangle - |1\rangle\otimes|0\rangle\right)$$

Can you find circuits preparing all these four Bell states? Convince yourself that the Bell states are indeed orthogonal using the state_vector backend of Aer. We will see more details on running quantum circuit at Part2 so at here, just try to use below function to run the circuit.

```python
# We prepare a similar function for running on the state vector simulator
def circuit_run(quantum_circuit, decimals=6):
    """Takes a circuit, and runs it on the state vector simulator backend."""
    statevector_simulator = Aer.get_backend('statevector_simulator')
    job = execute(quantum_circuit, backend=statevector_simulator)
    result = job.result()
    statevector = result.get_statevector(quantum_circuit, decimals=decimals)
    return statevector
```

### The state $|\Phi^+\rangle$

|$\Phi^+$> = [0.707107+0.j 0.      +0.j 0.      +0.j 0.707107+0.j]

```python
qc_phi_plus = QuantumCircuit(2)

qc_phi_plus.h(0)
qc_phi_plus.cx(0,1)

qc_phi_plus.draw(output='mpl')
```

```python

phi_plus_state = circuit_run(qc_phi_plus)
print('|Phi^+> =', phi_plus_state)
```

    |Phi^+> = [0.707107+0.j 0.      +0.j 0.      +0.j 0.707107+0.j]

### The state $|\Phi^-\rangle$
|$\Phi^-$> = [ 0.707107+0.j  0.      +0.j  0.      +0.j -0.707107-0.j]

```python
# Let us first prepare the Phi^- state
qc_phi_minus = QuantumCircuit(2)

qc_phi_minus.x(0)
qc_phi_minus.h(0)
qc_phi_minus.cx(0,1)

qc_phi_minus.draw(output='mpl')
```

```python
phi_minus_state = circuit_run(qc_phi_minus)
print('|Phi^-> =', phi_minus_state)
```

    |Phi^-> = [ 0.707107+0.j  0.      +0.j  0.      +0.j -0.707107-0.j]

### The state $|\Psi^+\rangle$
|$\Psi^+$> = [0.      +0.j 0.707107+0.j 0.707107+0.j 0.      +0.j]

```python
#The Psi^+ state
qc_psi_plus = QuantumCircuit(2)

qc_psi_plus.x(1)
qc_psi_plus.h(0)
qc_psi_plus.cx(0,1)

qc_psi_plus.draw(output='mpl')
```

```python
psi_plus_state = circuit_run(qc_psi_plus)
print('|Psi^+> =', psi_plus_state)
```

    |Psi^+> = [0.      +0.j 0.707107+0.j 0.707107+0.j 0.      +0.j]

### The state $|\Psi^-\rangle$
|$\Psi^-$> = [ 0.      +0.j -0.707107-0.j  0.707107+0.j  0.      +0.j]

```python
# Let us first prepare the Psi^- state
qc_psi_minus = QuantumCircuit(2)

qc_psi_minus.x(0)
qc_psi_minus.x(1)
qc_psi_minus.h(0)
qc_psi_minus.cx(0,1)

qc_psi_minus.draw(output='mpl')
```

```python
psi_minus_state = circuit_run(qc_psi_minus)
print('|Psi^-> =', psi_minus_state)
```

    |Psi^-> = [ 0.      +0.j -0.707107-0.j  0.707107+0.j  0.      +0.j]

```python
### Check the orthogonality of the Bell states
print('|<Phi^+|Phi^+>|^2 =', state_fidelity(phi_plus_state, phi_plus_state))
print('|<Phi^+|Phi^->|^2 =', state_fidelity(phi_plus_state, phi_minus_state))
print('|<Phi^+|Psi^+>|^2 =', state_fidelity(phi_plus_state, psi_plus_state))
print('|<Phi^+|Psi^->|^2 =', state_fidelity(phi_plus_state, psi_minus_state))

print('|<Psi^+|Phi^+>|^2 =', state_fidelity(psi_plus_state, phi_plus_state))
print('|<Psi^+|Phi^->|^2 =', state_fidelity(psi_plus_state, phi_minus_state))
print('|<Psi^+|Psi^+>|^2 =', state_fidelity(psi_plus_state, psi_plus_state))
print('|<Psi^+|Psi^->|^2 =', state_fidelity(psi_plus_state, psi_minus_state))

```

    |<Phi^+|Phi^+>|^2 = 1.0000012377963834
    |<Phi^+|Phi^->|^2 = 0.0
    |<Phi^+|Psi^+>|^2 = 0.0
    |<Phi^+|Psi^->|^2 = 0.0
    |<Psi^+|Phi^+>|^2 = 0.0
    |<Psi^+|Phi^->|^2 = 0.0
    |<Psi^+|Psi^+>|^2 = 1.0000012377963834
    |<Psi^+|Psi^->|^2 = 0.0

```python
import qiskit.tools.jupyter
%qiskit_version_table
%qiskit_copyright
```

Version Information

Qiskit Software	Version

qiskit-terra	0.18.3

qiskit-aer	0.9.1

qiskit-ignis	0.6.0

qiskit-ibmq-provider	0.17.0

qiskit-aqua	0.9.5

qiskit	0.31.0

qiskit-nature	0.2.2

qiskit-finance	0.2.1

qiskit-optimization	0.2.3

qiskit-machine-learning	0.2.1

System information

Python	3.9.7 (default, Sep 16 2021, 08:50:36)
[Clang 10.0.0 ]

OS	Darwin

CPUs	4

Memory (Gb)	32.0

Thu Oct 21 23:13:20 2021 JST

This code is a part of Qiskit

© Copyright IBM 2017, 2021.

This code is licensed under the Apache License, Version 2.0. You may
obtain a copy of this license in the LICENSE.txt file in the root directory
 of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
Any modifications or derivative works of this code must retain this
copyright notice, and modified files need to carry a notice indicating
that they have been altered from the originals.

```python

```

Lecture1/sol/Lecture 1 - Intro to QC for the physicist Part0 Part1_with_sol.ipynb


# Neuromatch Academy: Week 1, Day 3, Tutorial 1
# Model Fitting: Linear regression with MSE

#Tutorial Objectives

This is Tutorial 1 of a series on fitting models to data. We start with simple linear regression, using least squares optimization (Tutorial 1) and Maximum Likelihood Estimation (Tutorial 2). We will use bootstrapping to build confidence intervals around the inferred linear model parameters (Tutorial 3). We'll finish our exploration of linear models by  generalizing to multiple linear regression (Tutorial 4). We then move on to polynomial regression (Tutorial 5). We end by learning how to choose between these various models. We discuss the bias-variance trade-off (Tutorial 6) and two common methods for model selection, AIC and Cross Validation (Tutorial 7).

In this tutorial, we will learn how to fit simple linear models to data.
- Learn how to calculate the mean-squared error (MSE)
- Explore how model parameters (slope) influence the MSE
- Learn how to find the optimal model parameter using least-squares optimization

---

**acknowledgements:**
- we thank Eero Simoncelli, much of this tutorial is inspired by exercises asigned in his mathtools class.

# Setup

```python
# @title  Imports
import numpy as np
import matplotlib.pyplot as plt

import ipywidgets as widgets

```

```python
#@title Figure Settings
%matplotlib inline
fig_w, fig_h = (8, 6)
plt.rcParams.update({'figure.figsize': (fig_w, fig_h)})
%config InlineBackend.figure_format = 'retina'
```

# Mean Squared Error (MSE)

```python
#@title Video: Linear Regression & Mean Squared Error
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="8e9Hvc8zaUQ", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video

```

    Video available at https://youtube.com/watch?v=8e9Hvc8zaUQ

Linear least squares regression is an old but gold  optimization procedure that we are going to use for data fitting. Least squares (LS) optimization problems are those in which the objective function is a quadratic function of the
parameter(s) being optimized.

Suppose you have a set of measurements, $y_{n}$ (the "dependent" variable) obtained for different input values, $x_{n}$ (the "independent" or "explanatory" variable). Suppose we believe the measurements are proportional to the input values, but are corrupted by some (random) measurement errors, $\epsilon_{n}$, that is:

$$y_{n}= \theta x_{n}+\epsilon_{n}$$

for some unknown slope parameter $\theta.$ The least squares regression problem uses mean squared error (MSE) as its objective function, it aims to find the value of the parameter $\theta$ by minimizing the average of squared errors:

\begin{align}
\min _{\theta} \frac{1}{N}\sum_{n=1}^{N}\left(y_{n}-\theta x_{n}\right)^{2}
\end{align}

We will now explore how MSE is used in fitting a linear regression model to data. For illustrative purposes, we will create a simple synthetic dataset where we know the true underlying model. This will allow us to see how our estimation efforts compare in uncovering the real model (though in practice we rarely have this luxury).

First we will generate some noisy samples $x$ from [0, 10) along the line $y = 1.2x$ as our dataset we wish to fit a model to.

```python
# setting a fixed seed to our random number generator ensures we will always
# get the same psuedorandom number sequence
np.random.seed(121)
theta = 1.2
n_samples = 30
x = 10*np.random.rand(n_samples) # sample from a uniform distribution over [0,10)
noise = np.random.randn(n_samples) # sample from a standard normal distribution
y = theta*x + noise

fig, ax = plt.subplots()
ax.scatter(x, y)  # produces a scatter plot
ax.set(xlabel='x', ylabel='y');
```

Now that we have our suitably noisy dataset, we can start trying to estimate the underlying model that produced it. We use MSE to evaluate how successful a particular slope estimate $\hat{\theta}$ is for explaining the data, with the closer to 0 the MSE is, the better our estimate fits the data.

### Exercise: Compute MSE

In this exercise you will implement a method to compute the mean squared error for a set of inputs $x$, measurements $y$, and slope estimate $\hat{\theta}$.

```python
def mse(x, y, theta_hat):
  """Compute the mean squared error

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta_hat (float): An estimate of the slope parameter.

  Returns:
    float: The mean squared error of the data with the estimated parameter.
  """
  ####################################################
  ## TODO for students: compute the mean squared error
  ####################################################
  # comment this out when you've filled
  raise NotImplementedError("Student excercise: compute the mean squared error")
  return mse
```

```python
# to_remove solution
def mse(x, y, theta_hat):
  """Compute the mean squared error

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta_hat (float): An estimate of the slope parameter.

  Returns:
    float: The mean squared error of the data with the estimated parameter.
  """
  y_hat = theta_hat * x
  mse = np.mean((y - y_hat)**2)
  return mse
```

Now that we have our MSE method, let's see what errors we observe for different values of $\hat{\theta}$.

```python
theta_hats = [0.75, 1.0, 1.5]
for theta_hat in theta_hats:
  print(f"theta_hat of {theta_hat} has an MSE of {mse(x, y, theta_hat):.2f}")
```

    theta_hat of 0.75 has an MSE of 9.08
    theta_hat of 1.0 has an MSE of 3.01
    theta_hat of 1.5 has an MSE of 4.52

We should see that $\hat{\theta} = 1.0$ is our best estimate from the three we tried. Looking just at the raw numbers, however, isn't always satisfying, so let's visualize what our estimated model looks like over the data.

```python
fig, axes = plt.subplots(ncols=3, figsize=(18,4))
for theta_hat, ax in zip(theta_hats, axes):
  ax.scatter(x, y)  # our data scatter plot
  y_hat = theta_hat * x
  ax.plot(x, y_hat, color='C1')  # our estimated model
  ax.set(
    title=f"theta_hat = {theta_hat}, MSE = {mse(x, y, theta_hat):.2f}",
    xlabel='x',
    ylabel='y'
  );
```

### Data Exploration

Using an interactive widget, we can easily see how changing our slope estimate changes our model fit. Note that we also display the residuals as segments between observed responses (the data point), and predicted responses (on the slope).

```python
#@title MSE Explorer
@widgets.interact(theta_hat=widgets.FloatSlider(1.0, min=0.0, max=2.0))
def plot_data_estimate(theta_hat):
  fig, ax = plt.subplots()
  ax.scatter(x, y)  # our data scatter plot
  y_hat = theta_hat * x
  ax.plot(x, y_hat, color='C1', label='fit')  # our estimated model
  # plot residuals
  ymin = np.minimum(y, y_hat)
  ymax = np.maximum(y, y_hat)
  ax.vlines(x, ymin, ymax, 'g', alpha=0.5, label='residuals')
  ax.set(
    title=f"theta_hat = {theta_hat}, MSE = {mse(x, y, theta_hat):.2f}",
    xlabel='x',
    ylabel='y'
  )
  ax.legend()
```

While exploring single estimates by hand can be instructive, it's not the most efficient for finding the best estimate to fit our data. Another technique we can use is to evaluate the MSE across a reasonable range of values and plot the resulting error surface.

```python
theta_hats = np.linspace(-2.0, 4.0)
errors = np.zeros(len(theta_hats))
for i, theta_hat in enumerate(theta_hats):
  errors[i] = mse(x, y, theta_hat)

fig, ax = plt.subplots()
ax.plot(theta_hats, errors, label='mse', c='C1')
ax.axvline(theta, color='r', ls='--', label="theta")
best_error = np.min(errors)
best_fit = theta_hats[np.argmin(errors)]
ax.set(
  title=f"Best fit: theta_hat = {best_fit:.2f}, MSE = {best_error:.2f}",
  xlabel='theta_hat',
  ylabel='MSE')
ax.legend();
```

We can see that our best fit is $\hat{\theta}=1.18$ with an MSE of 1.45. This is quite close to the original true value $\theta=1.2$!

While this approach quickly got us to a good estimate, it still relied on evaluating the MSE value across a grid of hand-specified values. If we didn't pick a good range to begin with, or with enough granularity, we might end up in a local minima and not find the best possible estimator. Let's go one step further, and instead of finding the minimum MSE from a set of candidate estimates, let's solve for it analytically.

We can do this by minimizing the cost function. Mean squared error is a convex objective function, therefore we can compute its minimum using calculus.

We set the derivative of the error expression with respect to $\theta$ equal to zero,

\begin{align}
\frac{d}{d\theta}\frac{1}{N}\sum_{i=1}^N(\hat{y}_i-y_i)^2 = 0
\end{align}

solving for $\theta$, we obtain an optimal value of:

\begin{align}
\hat\theta = \sum_{i=1}^N \frac{x_i y_i}{x_i^2}
\end{align}

This is known as solving the *normal equations*. For different ways of obtaining the solution, see the notes on [Least Squares Optimization](https://www.cns.nyu.edu/~eero/NOTES/leastSquares.pdf) by Eero Simoncelli.

### Exercise: Solve for the Optimal Estimator

In this exercise, you will write a function that finds the optimal $\hat{\theta}$ value using the least squares optimization approach (the previous equation above) to solving MSE minimization. It shoud take as arguments $x$ and $y$, and return the solution $\hat{\theta}$.

```python
def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.

    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.

  Returns:
    float: the value for theta_hat arrived from minimizing MSE
  """
  ######################################################################
  ## TODO for students: solve for the best parameter using least squares
  ######################################################################
  # comment this out when you've filled
  raise NotImplementedError("Student excercise: solve for theta_hat using least squares")
  return theta_hat
```

```python
# to_remove solution
def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.

    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.

  Returns:
    float: the value for theta_hat arrived from minimizing MSE
  """
  theta_hat = (x.T @ y) / (x.T @ x)
  return theta_hat
```

Use your function to compute an estimate $\hat{\theta}$ and plot the resulting prediction on top of the data.

```python
theta_hat = solve_normal_eqn(x, y)
y_hat = theta_hat * x

fig, ax = plt.subplots()
ax.scatter(x, y)  # our data scatter plot
ax.plot(x, y_hat, color='C1', label='fit')  # our estimated model
# plot residuals
ymin = np.minimum(y, y_hat)
ymax = np.maximum(y, y_hat)
ax.vlines(x, ymin, ymax, 'g', alpha=0.5, label='residuals')
ax.set(
  title=f"theta_hat = {theta_hat:0.2f}, MSE = {mse(x, y, theta_hat):.2f}",
  xlabel='x',
  ylabel='y'
)
ax.legend();
```

We see that the analytic solution produces an even better result than our grid search from before, producing $\hat{\theta} = 1.21$ with MSE = 1.43!

## Summary

- Linear least squares regression is an optimisation procedure that can be used for data fitting:
    - Task: predict a value for $y$ given $x$
    - Performance measure: $\textrm{MSE}$
    - Procedure: minimize $\textrm{MSE}$ by solving the normal equations
- **Key point**: We fit the model by defining an *objective function* and minimizing it.
- **Note**: In this case, there is an *analytical* solution to the minimization problem and in practice, this solution can be computed using *linear algebra*. This is *extremely* powerful and forms the basis for much of numerical computation throughout the sciences.


# Getting started with TensorFlow (Eager Mode)

**Learning Objectives**
  - Understand difference between Tensorflow's two modes: Eager Execution and Graph Execution
  - Practice defining and performing basic operations on constant Tensors
  - Use Tensorflow's automatic differentiation capability

## Introduction

**Eager Execution**

Eager mode evaluates operations immediatley and return concrete values immediately. To enable eager mode simply place `tf.enable_eager_execution()` at the top of your code. We recommend using eager execution when prototyping as it is intuitive, easier to debug, and requires less boilerplate code.

**Graph Execution**

Graph mode is TensorFlow's default execution mode (although it will change to eager with TF 2.0). In graph mode operations only produce a symbolic graph which doesn't get executed until run within the context of a tf.Session(). This style of coding is less inutitive and has more boilerplate, however it can lead to performance optimizations and is particularly suited for distributing training across multiple devices. We recommend using delayed execution for performance sensitive production code.

```python
import tensorflow as tf
print(tf.__version__)
```

## Eager Execution

```python
tf.enable_eager_execution()
```

### Adding Two Tensors

The value of the tensor, as well as its shape and data type are printed

```python
a = tf.constant(value = [5, 3, 8], dtype = tf.int32)
b = tf.constant(value = [3, -1, 2], dtype = tf.int32)
c = tf.add(x = a, y = b)
print(c)
```

#### Overloaded Operators
We can also perform a `tf.add()` using the `+` operator. The `/,-,*` and `**` operators are similarly overloaded with the appropriate tensorflow operation.

```python
c = a + b # this is equivalent to tf.add(a,b)
print(c)
```

### NumPy Interoperability

In addition to native TF tensors, tensorflow operations can take native python types and NumPy arrays as operands.

```python
import numpy as np

a_py = [1,2] # native python list
b_py = [3,4] # native python list

a_np = np.array(object = [1,2]) # numpy array
b_np = np.array(object = [3,4]) # numpy array

a_tf = tf.constant(value = [1,2], dtype = tf.int32) # native TF tensor
b_tf = tf.constant(value = [3,4], dtype = tf.int32) # native TF tensor

for result in [tf.add(x = a_py, y = b_py), tf.add(x = a_np, y = b_np), tf.add(x = a_tf, y = b_tf)]:
    print("Type: {}, Value: {}".format(type(result), result))
```

You can convert a native TF tensor to a NumPy array using .numpy()

```python
a_tf.numpy()
```

### Linear Regression

Now let's use low level tensorflow operations to implement linear regression.

Later in the course you'll see abstracted ways to do this using high level TensorFlow.

#### Toy Dataset

We'll model the following function:

\begin{equation}
y= 2x + 10
\end{equation}

```python
X = tf.constant(value = [1,2,3,4,5,6,7,8,9,10], dtype = tf.float32)
Y = 2 * X + 10
print("X:{}".format(X))
print("Y:{}".format(Y))
```

#### Loss Function

Using mean squared error, our loss function is:
\begin{equation}
MSE = \frac{1}{m}\sum_{i=1}^{m}(\hat{Y}_i-Y_i)^2
\end{equation}

$\hat{Y}$ represents the vector containing our model's predictions:
\begin{equation}
\hat{Y} = w_0X + w_1
\end{equation}

```python
def loss_mse(X, Y, w0, w1):
    Y_hat = w0 * X + w1
    return tf.reduce_mean(input_tensor = (Y_hat - Y)**2)
```

#### Gradient Function

To use gradient descent we need to take the partial derivative of the loss function with respect to each of the weights. We could manually compute the derivatives, but with Tensorflow's automatic differentiation capabilities we don't have to!

During gradient descent we think of the loss as a function of the parameters $w_0$ and $w_1$. Thus, we want to compute the partial derivative with respect to these variables. The `params=[2,3]` argument tells TensorFlow to only compute derivatives with respect to the 2nd and 3rd arguments to the loss function (counting from 0, so really the 3rd and 4th).

```python
# Counting from 0, the 2nd and 3rd parameter to the loss function are our weights
grad_f = tf.contrib.eager.gradients_function(f = loss_mse, params=[2,3])
```

#### Training Loop

Here we have a very simple training loop that converges. Note we are ignoring best practices like batching, creating a separate test set, and random weight initialization for the sake of simplicity.

```python
STEPS = 1000
LEARNING_RATE = .02

# Initialize weights
w0 = tf.constant(value = 0.0, dtype = tf.float32)
w1 = tf.constant(value = 0.0, dtype = tf.float32)

for step in range(STEPS):
    #1. Calculate gradients
    d_w0, d_w1 = grad_f(X, Y, w0, w1)

    #2. Update weights
    w0 = w0 - d_w0 * LEARNING_RATE
    w1 = w1 - d_w1 * LEARNING_RATE

    #3. Periodically print MSE
    if step % 100 == 0:
        print("STEP: {} MSE: {}".format(step, loss_mse(X, Y, w0, w1)))

# Print final MSE and weights
print("STEP: {} MSE: {}".format(STEPS,loss_mse(X, Y, w0, w1)))
print("w0:{}".format(round(float(w0), 4)))
print("w1:{}".format(round(float(w1), 4)))
```

## Bonus

Try modelling a non-linear function such as: $y=xe^{-x^2}$

```python
X = tf.constant(value = np.linspace(0,2,1000), dtype = tf.float32)
Y = X*np.exp(-X**2) * X

from matplotlib import pyplot as plt
%matplotlib inline
plt.plot(X, Y)
```

```python
def make_features(X):
    features = [X]
    features.append(tf.ones_like(X))  # Bias.
    features.append(tf.square(X))
    features.append(tf.sqrt(X))
    features.append(tf.exp(X))
    return tf.stack(features, axis=1)

def make_weights(n_weights):
    W = [tf.constant(value = 0.0, dtype = tf.float32) for _ in range(n_weights)]
    return tf.expand_dims(tf.stack(W),-1)

def predict(X, W):
    Y_hat = tf.matmul(X, W)
    return tf.squeeze(Y_hat, axis=-1)

def loss_mse(X, Y, W):
    Y_hat = predict(X, W)
    return tf.reduce_mean(input_tensor = (Y_hat - Y)**2)

X = tf.constant(value = np.linspace(0,2,1000), dtype = tf.float32)
Y = np.exp(-X**2) * X

grad_f = tf.contrib.eager.gradients_function(f = loss_mse, params=[2])
```

```python
STEPS = 2000
LEARNING_RATE = .02

# Weights/features.
Xf = make_features(X)
# Xf = Xf[:,0:2]  # Linear features only.
W = make_weights(Xf.get_shape()[1].value)

# For plotting
steps = []
losses = []

plt.figure()
for step in range(STEPS):
    #1. Calculate gradients
    dW = grad_f(Xf, Y, W)[0]
    #2. Update weights
    W -= dW * LEARNING_RATE
    #3. Periodically print MSE
    if step % 100 == 0:
        loss = loss_mse(Xf, Y, W)
        steps.append(step)
        losses.append(loss)
        plt.clf()
        plt.plot(steps, losses)
# Print final MSE and weights
print("STEP: {} MSE: {}".format(STEPS,loss_mse(Xf, Y, W)))

# Plot results
plt.figure()
plt.plot(X, Y, label='actual')
plt.plot(X, predict(Xf, W), label='predicted')
plt.legend()
```

Copyright 2019 Google Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License


# Transvectant computations

```python
from sympy import Function, Symbol, symbols, init_printing, expand
from IPython.display import Math, display

init_printing()
from transvectants import *

def disp(expr):
    display(Math(my_latex(expr)))
```

```python
x, y = symbols('x y')
f = Function('f')(x, y)
```

```python
disp(expand(partial_transvectant((f, f), [[0, 1]])))
```

$\displaystyle 0$

```python
disp(expand(partial_transvectant((f, f), [[0, 1], [0, 1], [0, 1], [0, 1]])))
```

$\displaystyle 2 f_{xxxx} f_{yyyy} - 8 f_{xyyy} f_{xxxy} + 6 \left(f_{xxyy}\right)^{2}$

```python
disp(expand(partial_transvectant((f, f, f, f, f, f), [[0, 1], [1, 2], [3, 4], [4, 5]])))
```

$\displaystyle \left(f_{x}\right)^{4} \left(f_{yy}\right)^{2} - 4 \left(f_{x}\right)^{3} f_{y} f_{yy} f_{xy} + 2 \left(f_{x}\right)^{2} f_{xx} \left(f_{y}\right)^{2} f_{yy} + 4 \left(f_{x}\right)^{2} \left(f_{y}\right)^{2} \left(f_{xy}\right)^{2} - 4 f_{x} f_{xx} \left(f_{y}\right)^{3} f_{xy} + \left(f_{xx}\right)^{2} \left(f_{y}\right)^{4}$

```python
disp(expand(partial_transvectant((f, f, f), [[0, 1], [0, 1], [0, 2], [0, 2]])))
```

$\displaystyle \left(f_{xx}\right)^{2} f_{yyyy} + 2 f_{xx} f_{yy} f_{xxyy} - 4 f_{xx} f_{xy} f_{xyyy} + f_{xxxx} \left(f_{yy}\right)^{2} - 4 f_{yy} f_{xy} f_{xxxy} + 4 \left(f_{xy}\right)^{2} f_{xxyy}$

```python
disp(expand(partial_transvectant((f, f, f, f, f), [[0, 1], [0, 1], [2, 3], [2, 3], [2, 4]]) ) -2*(expand(partial_transvectant((f, f, f, f, f), [[0, 1], [1, 2], [2, 3], [3, 0], [0, 4]]) )))
```

$\displaystyle 0$

```python
disp(expand(partial_transvectant((f, f, f), [[0, 1], [0, 1], [0, 1], [0, 2]])))
```

$\displaystyle f_{x} f_{xxx} f_{yyyy} - f_{x} f_{yyy} f_{xxxy} + 3 f_{x} f_{xyy} f_{xxyy} - 3 f_{x} f_{xyyy} f_{xxy} - f_{xxx} f_{y} f_{xyyy} + f_{xxxx} f_{y} f_{yyy} - 3 f_{y} f_{xyy} f_{xxxy} + 3 f_{y} f_{xxy} f_{xxyy}$

```python
disp(expand(partial_transvectant((f, f), [[0, 1], [0, 1], [0, 1]])))
```

$\displaystyle 0$

```python
#C = transvectant(f, f, 2)
#D = -partial_transvectant((f, f, f), [[0, 1], [1, 2]])
# We are going to build these by weight, not degree.
# Hence order does not match dispaper
# Weight 4 (2 of 'em)

I4_1 = partial_transvectant((f,f),[[0,1],[0,1]]) # = C
I4_2 = partial_transvectant((f, f, f), [[0, 1], [1, 2]]) # = -D

# Weight 6 (2 of 'em)
print('weight 3:')
I6_1 = partial_transvectant((f,f,f),[[0,1],[0,1],[0,2]]) # = transvectant(f, C, 1)
I6_2 = partial_transvectant((f,f,f,f),[[0,1],[0,2],[0,3]])

# Weight 8 (7 of 'em??)
print('weight 4:')
I8_1 = expand(partial_transvectant((f,f,f),[[0,1],[0,1],[1,2],[0,2]]))
I8_2 = expand(partial_transvectant((f,f,f,f),[[0,1],[0,1],[1,2],[2,3]]))
I8_3 = expand(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[3,0]]))
I8_4 = expand(partial_transvectant((f,f,f,f),[[0,1],[1,2],[1,2],[2,3]]))

I8_5 = expand(partial_transvectant((f,f,f,f,f),[[0,1],[1,2],[2,3],[3,4]]))
I8_6 = expand(partial_transvectant((f,f,f,f,f),[[0,1],[0,2],[0,3],[3,4]]))
I8_7 = expand(partial_transvectant((f,f,f,f,f),[[0,1],[0,2],[0,3],[0,4]]))

print('weight 2')
disp(I4_1)
disp(I4_2)

print('weight 3')
disp(I6_1)
disp(expand(I6_2))

print('weight 4')
disp(I8_1)
print('')
disp(I8_2)
print('')
disp(I8_3)
print('')
disp(I8_4)
print('')
disp(I8_5)
print('')
disp(I8_6)
print('')
disp(I8_7)
```

    weight 3:
    weight 4:
    weight 2

$\displaystyle 2 f_{xx} f_{yy} - 2 \left(f_{xy}\right)^{2}$

$\displaystyle - \left(f_{x}\right)^{2} f_{yy} + 2 f_{x} f_{y} f_{xy} - f_{xx} \left(f_{y}\right)^{2}$

    weight 3

$\displaystyle - \left(f_{xx} f_{yyy} + f_{yy} f_{xxy} - 2 f_{xy} f_{xyy}\right) f_{x} + \left(f_{xx} f_{xyy} + f_{xxx} f_{yy} - 2 f_{xy} f_{xxy}\right) f_{y}$

$\displaystyle - \left(f_{x}\right)^{3} f_{yyy} + 3 \left(f_{x}\right)^{2} f_{y} f_{xyy} - 3 f_{x} \left(f_{y}\right)^{2} f_{xxy} + f_{xxx} \left(f_{y}\right)^{3}$

    weight 4

$\displaystyle 2 f_{xx} f_{yyy} f_{xxy} - 2 f_{xx} \left(f_{xyy}\right)^{2} + 2 f_{xxx} f_{yy} f_{xyy} - 2 f_{xxx} f_{yyy} f_{xy} - 2 f_{yy} \left(f_{xxy}\right)^{2} + 2 f_{xy} f_{xyy} f_{xxy}$

$\displaystyle - f_{x} f_{xx} f_{yy} f_{xyy} + f_{x} f_{xx} f_{yyy} f_{xy} - f_{x} f_{xxx} \left(f_{yy}\right)^{2} + 3 f_{x} f_{yy} f_{xy} f_{xxy} - 2 f_{x} \left(f_{xy}\right)^{2} f_{xyy} - \left(f_{xx}\right)^{2} f_{y} f_{yyy} - f_{xx} f_{y} f_{yy} f_{xxy} + 3 f_{xx} f_{y} f_{xy} f_{xyy} + f_{xxx} f_{y} f_{yy} f_{xy} - 2 f_{y} \left(f_{xy}\right)^{2} f_{xxy}$

$\displaystyle 2 \left(f_{xx}\right)^{2} \left(f_{yy}\right)^{2} - 4 f_{xx} f_{yy} \left(f_{xy}\right)^{2} + 2 \left(f_{xy}\right)^{4}$

$\displaystyle - 2 \left(f_{x}\right)^{2} f_{yyy} f_{xxy} + 2 \left(f_{x}\right)^{2} \left(f_{xyy}\right)^{2} + 2 f_{x} f_{xxx} f_{y} f_{yyy} - 2 f_{x} f_{y} f_{xyy} f_{xxy} - 2 f_{xxx} \left(f_{y}\right)^{2} f_{xyy} + 2 \left(f_{y}\right)^{2} \left(f_{xxy}\right)^{2}$

$\displaystyle \left(f_{x}\right)^{2} f_{xx} \left(f_{yy}\right)^{2} - \left(f_{x}\right)^{2} f_{yy} \left(f_{xy}\right)^{2} - 2 f_{x} f_{xx} f_{y} f_{yy} f_{xy} + 2 f_{x} f_{y} \left(f_{xy}\right)^{3} + \left(f_{xx}\right)^{2} \left(f_{y}\right)^{2} f_{yy} - f_{xx} \left(f_{y}\right)^{2} \left(f_{xy}\right)^{2}$

$\displaystyle - \left(f_{x}\right)^{3} f_{yy} f_{xyy} + \left(f_{x}\right)^{3} f_{yyy} f_{xy} - \left(f_{x}\right)^{2} f_{xx} f_{y} f_{yyy} + 2 \left(f_{x}\right)^{2} f_{y} f_{yy} f_{xxy} - \left(f_{x}\right)^{2} f_{y} f_{xy} f_{xyy} + 2 f_{x} f_{xx} \left(f_{y}\right)^{2} f_{xyy} - f_{x} f_{xxx} \left(f_{y}\right)^{2} f_{yy} - f_{x} \left(f_{y}\right)^{2} f_{xy} f_{xxy} - f_{xx} \left(f_{y}\right)^{3} f_{xxy} + f_{xxx} \left(f_{y}\right)^{3} f_{xy}$

$\displaystyle \left(f_{x}\right)^{4} f_{yyyy} - 4 \left(f_{x}\right)^{3} f_{y} f_{xyyy} + 6 \left(f_{x}\right)^{2} \left(f_{y}\right)^{2} f_{xxyy} - 4 f_{x} \left(f_{y}\right)^{3} f_{xxxy} + f_{xxxx} \left(f_{y}\right)^{4}$

```python
# Only 'weight 4' affine invariant
disp(I4_2/I4_1)

# Only 'weight 6' affine invariant
disp(I6_2/I6_1)

```

$$\frac{- \left(f_{x}\right)^{2} f_{yy} + 2 f_{x} f_{y} f_{xy} - \left(f_{y}\right)^{2} f_{xx}}{2 f_{xx} f_{yy} - 2 \left(f_{xy}\right)^{2}}$$

$$\frac{- \left(f_{x}\right)^{3} f_{yyy} + 3 \left(f_{x}\right)^{2} f_{y} f_{xyy} - 3 f_{x} \left(f_{y}\right)^{2} f_{xxy} + \left(f_{y}\right)^{3} f_{xxx}}{\left(f_{xx} f_{xyy} - 2 f_{xy} f_{xxy} + f_{yy} f_{xxx}\right) f_{y} - \left(f_{xx} f_{yyy} - 2 f_{xy} f_{xyy} + f_{yy} f_{xxy}\right) f_{x}}$$

```python
disp(partial_transvectant((f,f,f,f,f),[[0,2],[1,2],[2,3],[3,4]]))
```

$$- \left(f_{x}\right)^{3} f_{xy} f_{yyy} + \left(f_{x}\right)^{3} f_{yy} f_{xyy} + \left(f_{x}\right)^{2} f_{y} f_{xx} f_{yyy} + \left(f_{x}\right)^{2} f_{y} f_{xy} f_{xyy} - 2 \left(f_{x}\right)^{2} f_{y} f_{yy} f_{xxy} - 2 f_{x} \left(f_{y}\right)^{2} f_{xx} f_{xyy} + f_{x} \left(f_{y}\right)^{2} f_{xy} f_{xxy} + f_{x} \left(f_{y}\right)^{2} f_{yy} f_{xxx} + \left(f_{y}\right)^{3} f_{xx} f_{xxy} - \left(f_{y}\right)^{3} f_{xy} f_{xxx}$$

```python
disp(partial_transvectant((f,f,C),[[0,1],[1,2]]))
```

$$4 \left(\left(f_{x} f_{xy} - f_{y} f_{xx}\right) \left(f_{xx} f_{yyy} - 2 f_{xy} f_{xyy} + f_{yy} f_{xxy}\right) - \left(f_{x} f_{yy} - f_{y} f_{xy}\right) \left(f_{xx} f_{xyy} - 2 f_{xy} f_{xxy} + f_{yy} f_{xxx}\right)\right) f{\left (x,y \right )}$$

```python
#disp(transvectant(C, C, 2))
funcs = (C, f**2)
pairs = [[0, 1]]
disp(partial_transvectant(funcs, pairs))
```

$$4 \left(\left(f_{xx} f_{xyy} - 2 f_{xy} f_{xxy} + f_{yy} f_{xxx}\right) f_{y} - \left(f_{xx} f_{yyy} - 2 f_{xy} f_{xyy} + f_{yy} f_{xxy}\right) f_{x}\right) f{\left (x,y \right )}$$

```python
# Construct linear, quadratic, cubic forms
fx, fy, fxx, fxy, fyy, fxxx, fxxy, fxyy, fyyy = symbols('f_x, f_y, f_{xx}, f_{xy}, f_{yy}, f_{xxx}, f_{xxy}, f_{xyy}, f_{yyy}')
l = fx*x + fy*y
q = fxx*x*x + 2*fxy*x*y + fyy*y*y
c = fxxx*x*x*x + 3*fxxy*x*x*y + 3*fxyy*x*y*y + fyyy*y*y*y
```

```python
# I3 as a form (Robert's method to annoy us...)
disp(-expand(transvectant(q,transvectant(c,c,2),2)/288))
# I5
disp(expand(transvectant(transvectant(c,c,2),transvectant(c,c,2),2)/10368))
# I6
disp(transvectant(c,l**3,3)/36)
```

$$- f_{xxx} f_{xyy} f_{yy} + f_{xxx} f_{xy} f_{yyy} + f_{xxy}^{2} f_{yy} - f_{xxy} f_{xx} f_{yyy} - f_{xxy} f_{xyy} f_{xy} + f_{xx} f_{xyy}^{2}$$

```python

```

```python

```

$$- 2 f_{xx} f_{xxy} f_{yyy} + 2 f_{xx} \left(f_{xyy}\right)^{2} + 2 f_{xy} f_{xxx} f_{yyy} - 2 f_{xy} f_{xxy} f_{xyy} - 2 f_{yy} f_{xxx} f_{xyy} + 2 f_{yy} \left(f_{xxy}\right)^{2}$$

```python

```

$$2 f_{xx} f_{xxy} f_{yyy} - 2 f_{xx} \left(f_{xyy}\right)^{2} - 2 f_{xy} f_{xxx} f_{yyy} + 2 f_{xy} f_{xxy} f_{xyy} + 2 f_{yy} f_{xxx} f_{xyy} - 2 f_{yy} \left(f_{xxy}\right)^{2}$$

```python

```

$$0$$

```python
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[0,1]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[0,2]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[0,1],[0,2]])))

disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[3,0]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[3,0],[0,1]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[3,0],[0,2]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[0,1],[1,2],[1,2],[2,3]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[0,1],[1,2],[1,2],[2,3],[2,3]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[1,2],[2,3],[3,0],[0,1],[1,2]])))
```

$$0$$

$$f_{x} f_{xx} f_{xy} f_{yyy} - f_{x} f_{xx} f_{yy} f_{xyy} - 2 f_{x} \left(f_{xy}\right)^{2} f_{xyy} + 3 f_{x} f_{xy} f_{yy} f_{xxy} - f_{x} \left(f_{yy}\right)^{2} f_{xxx} - f_{y} \left(f_{xx}\right)^{2} f_{yyy} + 3 f_{y} f_{xx} f_{xy} f_{xyy} - f_{y} f_{xx} f_{yy} f_{xxy} - 2 f_{y} \left(f_{xy}\right)^{2} f_{xxy} + f_{y} f_{xy} f_{yy} f_{xxx}$$

$$0$$

$$0$$

$$2 \left(f_{xx}\right)^{2} \left(f_{yy}\right)^{2} - 4 f_{xx} \left(f_{xy}\right)^{2} f_{yy} + 2 \left(f_{xy}\right)^{4}$$

$$0$$

$$0$$

$$\left(\left(f_{xx} f_{xxyy} - 2 f_{xy} f_{xxxy} + f_{yy} f_{xxxx}\right) f_{xyy} - 2 \left(f_{xx} f_{xyyy} - 2 f_{xy} f_{xxyy} + f_{yy} f_{xxxy}\right) f_{xxy} + \left(f_{xx} f_{yyyy} - 2 f_{xy} f_{xyyy} + f_{yy} f_{xxyy}\right) f_{xxx}\right) f_{y} - \left(\left(f_{xx} f_{xxyy} - 2 f_{xy} f_{xxxy} + f_{yy} f_{xxxx}\right) f_{yyy} - 2 \left(f_{xx} f_{xyyy} - 2 f_{xy} f_{xxyy} + f_{yy} f_{xxxy}\right) f_{xyy} + \left(f_{xx} f_{yyyy} - 2 f_{xy} f_{xyyy} + f_{yy} f_{xxyy}\right) f_{xxy}\right) f_{x}$$

$$2 \left(f_{xx}\right)^{2} f_{xxyy} f_{yyyy} - 2 \left(f_{xx}\right)^{2} \left(f_{xyyy}\right)^{2} - 4 f_{xx} f_{xy} f_{xxxy} f_{yyyy} + 4 f_{xx} f_{xy} f_{xxyy} f_{xyyy} + 2 f_{xx} f_{yy} f_{xxxx} f_{yyyy} - 4 f_{xx} f_{yy} f_{xxxy} f_{xyyy} + 2 f_{xx} f_{yy} \left(f_{xxyy}\right)^{2} + 8 \left(f_{xy}\right)^{2} f_{xxxy} f_{xyyy} - 8 \left(f_{xy}\right)^{2} \left(f_{xxyy}\right)^{2} - 4 f_{xy} f_{yy} f_{xxxx} f_{xyyy} + 4 f_{xy} f_{yy} f_{xxxy} f_{xxyy} + 2 \left(f_{yy}\right)^{2} f_{xxxx} f_{xxyy} - 2 \left(f_{yy}\right)^{2} \left(f_{xxxy}\right)^{2}$$

$$- f_{xx} \left(f_{xxy}\right)^{2} f_{yyyy} + 4 f_{xx} f_{xxy} f_{xyy} f_{xyyy} - 2 f_{xx} f_{xxy} f_{yyy} f_{xxyy} - 4 f_{xx} \left(f_{xyy}\right)^{2} f_{xxyy} + 4 f_{xx} f_{xyy} f_{yyy} f_{xxxy} - f_{xx} \left(f_{yyy}\right)^{2} f_{xxxx} + 2 f_{xy} f_{xxx} f_{xxy} f_{yyyy} - 4 f_{xy} f_{xxx} f_{xyy} f_{xyyy} + 2 f_{xy} f_{xxx} f_{yyy} f_{xxyy} - 4 f_{xy} \left(f_{xxy}\right)^{2} f_{xyyy} + 10 f_{xy} f_{xxy} f_{xyy} f_{xxyy} - 4 f_{xy} f_{xxy} f_{yyy} f_{xxxy} - 4 f_{xy} \left(f_{xyy}\right)^{2} f_{xxxy} + 2 f_{xy} f_{xyy} f_{yyy} f_{xxxx} - f_{yy} \left(f_{xxx}\right)^{2} f_{yyyy} + 4 f_{yy} f_{xxx} f_{xxy} f_{xyyy} - 2 f_{yy} f_{xxx} f_{xyy} f_{xxyy} - 4 f_{yy} \left(f_{xxy}\right)^{2} f_{xxyy} + 4 f_{yy} f_{xxy} f_{xyy} f_{xxxy} - f_{yy} \left(f_{xyy}\right)^{2} f_{xxxx}$$

```python
disp(simplify(partial_transvectant((f,f,f),[[0,1],[1,2],[2,0]])))
disp(simplify(partial_transvectant((f,f,f),[[0,1],[1,2],[0,1]])))
disp(simplify(partial_transvectant((f,f,f),[[0,1],[1,2],[2,0],[0,1]])))
```

```python
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[0,1],[1,2],[1,2],[2,3]])))
disp(simplify(partial_transvectant((f,f,f,f),[[0,1],[0,1],[1,2],[1,2],[2,3],[2,3]])))
```

$$\left(\left(f_{xx} f_{xxyy} - 2 f_{xy} f_{xxxy} + f_{yy} f_{xxxx}\right) f_{xyy} - 2 \left(f_{xx} f_{xyyy} - 2 f_{xy} f_{xxyy} + f_{yy} f_{xxxy}\right) f_{xxy} + \left(f_{xx} f_{yyyy} - 2 f_{xy} f_{xyyy} + f_{yy} f_{xxyy}\right) f_{xxx}\right) f_{y} - \left(\left(f_{xx} f_{xxyy} - 2 f_{xy} f_{xxxy} + f_{yy} f_{xxxx}\right) f_{yyy} - 2 \left(f_{xx} f_{xyyy} - 2 f_{xy} f_{xxyy} + f_{yy} f_{xxxy}\right) f_{xyy} + \left(f_{xx} f_{yyyy} - 2 f_{xy} f_{xyyy} + f_{yy} f_{xxyy}\right) f_{xxy}\right) f_{x}$$

$$2 \left(f_{xx}\right)^{2} f_{xxyy} f_{yyyy} - 2 \left(f_{xx}\right)^{2} \left(f_{xyyy}\right)^{2} - 4 f_{xx} f_{xy} f_{xxxy} f_{yyyy} + 4 f_{xx} f_{xy} f_{xxyy} f_{xyyy} + 2 f_{xx} f_{yy} f_{xxxx} f_{yyyy} - 4 f_{xx} f_{yy} f_{xxxy} f_{xyyy} + 2 f_{xx} f_{yy} \left(f_{xxyy}\right)^{2} + 8 \left(f_{xy}\right)^{2} f_{xxxy} f_{xyyy} - 8 \left(f_{xy}\right)^{2} \left(f_{xxyy}\right)^{2} - 4 f_{xy} f_{yy} f_{xxxx} f_{xyyy} + 4 f_{xy} f_{yy} f_{xxxy} f_{xxyy} + 2 \left(f_{yy}\right)^{2} f_{xxxx} f_{xxyy} - 2 \left(f_{yy}\right)^{2} \left(f_{xxxy}\right)^{2}$$

```python
disp(expand(partial_transvectant((f,f,f,f,f),[[0,1],[1,2],[2,3],[3,4]])))
```

$$\left(f_{x}\right)^{2} f_{xx} \left(f_{yy}\right)^{2} - \left(f_{x}\right)^{2} \left(f_{xy}\right)^{2} f_{yy} - 2 f_{x} f_{y} f_{xx} f_{xy} f_{yy} + 2 f_{x} f_{y} \left(f_{xy}\right)^{3} + \left(f_{y}\right)^{2} \left(f_{xx}\right)^{2} f_{yy} - \left(f_{y}\right)^{2} f_{xx} \left(f_{xy}\right)^{2}$$

```python
disp(expand(partial_transvectant((f,f,f,f,f),[[0,1],[1,2],[2,3],[3,4]])))
disp(expand(partial_transvectant((f,f,f,f,f),[[0,1],[0,2],[0,3],[3,4]])))
disp(expand(partial_transvectant((f,f,f,f,f),[[0,1],[0,2],[0,3],[0,4]])))
```

$$\left(f_{x}\right)^{2} f_{xx} \left(f_{yy}\right)^{2} - \left(f_{x}\right)^{2} \left(f_{xy}\right)^{2} f_{yy} - 2 f_{x} f_{y} f_{xx} f_{xy} f_{yy} + 2 f_{x} f_{y} \left(f_{xy}\right)^{3} + \left(f_{y}\right)^{2} \left(f_{xx}\right)^{2} f_{yy} - \left(f_{y}\right)^{2} f_{xx} \left(f_{xy}\right)^{2}$$

$$- \left(f_{x}\right)^{2} f_{xx} \left(f_{yy}\right)^{2} + \left(f_{x}\right)^{2} \left(f_{xy}\right)^{2} f_{yy} + 2 f_{x} f_{y} f_{xx} f_{xy} f_{yy} - 2 f_{x} f_{y} \left(f_{xy}\right)^{3} - \left(f_{y}\right)^{2} \left(f_{xx}\right)^{2} f_{yy} + \left(f_{y}\right)^{2} f_{xx} \left(f_{xy}\right)^{2}$$

$$\left(f_{x}\right)^{3} f_{xy} f_{yyy} - \left(f_{x}\right)^{3} f_{yy} f_{xyy} - \left(f_{x}\right)^{2} f_{y} f_{xx} f_{yyy} - \left(f_{x}\right)^{2} f_{y} f_{xy} f_{xyy} + 2 \left(f_{x}\right)^{2} f_{y} f_{yy} f_{xxy} + 2 f_{x} \left(f_{y}\right)^{2} f_{xx} f_{xyy} - f_{x} \left(f_{y}\right)^{2} f_{xy} f_{xxy} - f_{x} \left(f_{y}\right)^{2} f_{yy} f_{xxx} - \left(f_{y}\right)^{3} f_{xx} f_{xxy} + \left(f_{y}\right)^{3} f_{xy} f_{xxx}$$

$$\left(f_{x}\right)^{4} f_{yyyy} - 4 \left(f_{x}\right)^{3} f_{y} f_{xyyy} + 6 \left(f_{x}\right)^{2} \left(f_{y}\right)^{2} f_{xxyy} - 4 f_{x} \left(f_{y}\right)^{3} f_{xxxy} + \left(f_{y}\right)^{4} f_{xxxx}$$

```python

```

Transvectants:

(1) We can use transvectants to create lots of invariants. More than Robert can.
(2) We understand the role of weight and degree, and have a nice graph-based picture for all the invariants up to weight 8
(3) There is no issue with SA(2), but for A(2) the weight 4 invariants have non-removable singularities. There are three possible solutions: (i) claim you only do this on parts of images, (ii) look at the weight 8 invariants instead, particularly I8_1 and I8_3 which do not have f_x in at all, or I4^2/I8, (iii) consider projection as Robert did. Our preference is (ii) but we haven't done it yet.

Moving frame:

(1) We understand it, but it is still ugly
(2) So we just write it as is

Hence:

We can build all the invariants we need now.
We still have a projection issue. And hence a high weight invariant issue. And hence a serious noise problem.

But we probably now have enough for a paper. So we should write it.


# PRAKTIKUM 11
`Integral Numerik 2`

# Tinjau Ulang Aturan Trapesium dan Kompositnya

#### Apakah ada hubungannya?

## Aturan Rekursif Trapesium
Dimulai dengan $ T(0)=\frac{h}{2}(f(a)+f(b)) $, kemudian suatu barisam dari aturan trapesium $ \{ T(J) \} $ dapat dihasilkan oleh formula rekursif
\begin{equation}\label{eq:11 rekursif 2}
T(J)=\dfrac{T(J-1)}{2}+h\sum_{k=1}^{M}f(x_{2k-1}) \ \ \ \ \ \ \text{ untuk } J=1,2,\dots
\end{equation}
dengan $ h=(b-a)/2^J $ dan $ \{x_k=a+kh\} $.

```julia
function rekursif(f,a,b,n)
  M = 1;
  h = b-a;
  T = Array{Float64}(undef,n+1,1)
  T[1] = h/2*(f(a)+f(b))
  for j = 1:n
    h = h/2;
    s = 0;
    for k = 1:M
      x = a+h*(2*k-1);
      s = s+f(x);
    end
    M = 2*M;
    T[j+1] = T[j]/2+h*s;
  end
  return T
end
```

### Contoh 1
Berikut merupakan langkah-langkah untuk menghitung nilai hampiran $ T(J) $ dengan $ J=0,1,2,...,5 $ dari $$  \int_{1}^{5}\,\dfrac{dx}{x}  $$ serta nilai galat dari masing-masing hampiran, jika diketahui nilai solusi eksak integral adalah $ 1.609437912 $ menggunakan aturan rekursif trapesium.

**Langkah 1**: Penghitungan nilai hampiran $ T(0) $ hingga $ T(5) $.

```julia
f(x) = 1/x;
a = 1;
b = 5;
T = rekursif(f,a,b,5)
```

**Langkah 2**: Hitung nilai galat masing-masing hampiran

```julia
eksak = 1.609437912;
galat = T.-eksak
```

## Aturan Rekursif Simpson dan Boole
Aturan Rekursif Simpson: $S(J)=\dfrac{4T(J)-T(J-1)}{3}$ untuk $J=1,2,\dots$

Aturan Rekursif Boole  : $B(J)=\dfrac{16S(J)-S(J-1)}{15}$ untuk $J=2,3,\dots$

### Contoh 2
Dengan aturan rekursif Simpson dan Boole, hitung nilai hampiran integral menggunakan rekursif Simpson yaitu $S(1)$ hingga $S(5)$ serta rekursif Boole yaitu $B(2)$ hingga $B(5)$ berdasarkan hasil aturan rekursif trapesium pada **Contoh 1**.

**Langkah 1**: Hitung nilai hampiran $S(1)$ hingga $S(5)$

```julia
S = (4*T[2:end]-T[1:end-1])./3
```

**Langkah 2**: Hitung nilai hampiran $B(2)$ hingga $B(5)$

```julia
B = (16*S[2:end]-S[1:end-1])./15
```

# Integral Romberg
Diberikan 2 hampiran $ R(2h,K-1) $ dan $ R(h,K-1) $ untuk nilai $ Q $ yang memenuhi persamaan
\begin{equation}\label{eq:11 romberg 1}
Q = R(h,K-1) + O(h^{2K})
\end{equation}
dan
\begin{equation}\label{eq:11 romberg 2}
Q = R(2h,K-1) + O(h^{2K})
\end{equation}
Dengan demikian, pengembangan hampiran tersebut adalah
\begin{equation}\label{eq:11 romberg 3}
Q = \dfrac{4^KR(h,K-1)-R(2h,K-1)}{4^K-1} + O(h^{2K+2})
\end{equation}

atau dalam bentuk lain dapat dituliskan sebagai berikut

$$ R(J,K) = R(J,K-1)+\dfrac{R(J,K-1)-R(J-1,K-1)
}{4^K-1} $$
dengan $ R(J,0) $ merupakan aturan rekursif trapesium.

\begin{align*}
&R(J,0)=T(J)&& \text{ untuk } J\ge 0 \text{, adalah barisan aturan rekursif trapesium}\\
&R(J,1)=S(J)&& \text{ untuk } J\ge 1 \text{, adalah barisan aturan rekursif Simpson}\\
&R(J,2)=B(J)&& \text{ untuk } J\ge 2 \text{, adalah barisan aturan rekursif Boole}
\end{align*}

```julia
# flag 0 artinya toleransi terpenuhi, flag 1 artinya toleransi tidak terpenuhi dan maksimum iterasi tercapai
function romberg(f, a, b; maxi = 10, delta = 10^-9)
  flag = 1;
  M = 1;
  h = b-a;
  err = 1;
  R = h/2*(f(a)+f(b));
  for J = 1:maxi
    # Rekursif Trapesium
    h = h/2;
    M = 2*M;
    s = 0;
    for p = 1:M/2
      x = a+h*(2*p-1);
      s = s+f(x);
    end
    R = [R zeros(size(R,1),1) ; R[J,1]/2 + h*s zeros(1,size(R,1))]
    # Aturan Romberg
    for K=1:J
      R[J+1,K+1]=R[J+1,K]+(R[J+1,K]-R[J,K])/(4^K-1);
    end
    err = abs.(R[J,J]-R[J+1,J+1]);
    if err<delta; flag=0; break; end
  end
  sol = R[end,end];
  return sol, flag, err, R
end
```

### Contoh 3
Diberikan fungsi

$$  f(x) = (x^2+x+1)\cos(x)  $$

Berikut merupakan cara untuk menghitung nilai hampiran dari $ \displaystyle\int_{0}^{\pi/2}{f(x)}\,dx $ beserta lama waktu komputasi menggunakan metode Romberg (gunakan `BenchmarkTools` untuk menghitung waktu komputasi dan nilai $\delta=10^{-9}$ sebagai toleransi).

```julia
using BenchmarkTools
```

```julia
f(x) = (x^2+x+1)*cos(x)
a = 0;
b = pi/2;
@btime sol, flag, err, R = romberg(f, a, b)
sol, flag, err, R = romberg(f, a, b)
@show (sol,flag,err)
R
```

Berdasarkan hasil di atas, _flag_ bernilai 0, artinya solusi telah memenuhi nilai toleransi galat yang diberikan. Proses komputasi integral numerik dari $ \displaystyle\int_{0}^{\pi/2}{f(x)}\,dx $ menghasilkan solusi  2.038197427 dengan galat $ 1.213\times10^{-10} $ dalam waktu sekitar 150 mikrodetik. Waktu komputasi mungkin berbeda pada satu komputer dengan komputer lain, akan tetapi perbedaan waktu komputasi tidak akan berbeda signifikan.

# Aturan Kuadratur Adaptif
Aturan romberg memiliki waktu komputasi yang relatif lama untuk menghasilkan akurasi yang baik pada banyak kasus. Salah satu aturan yang dapat digunakan untuk mengatasinya adalah aturan kuadratur adaptif. Metode ini akan menyesuaikan
ukuran langkah $ h $ sesuai bentuk fungsi, sehingga menghemat waktu komputasi.

```julia
function adaptif(f, a, b; delta=10^-9)
  iterating = 0;
  done = 1;
  SRvec = srule(f,a,b,delta);
  SRmat = SRvec;
  m = 1;
  state = iterating;
  while(state==iterating)
    n = m;
    for j = n:-1:1
      p = j;
      SR0vec = SRmat[p,:];
      err = SR0vec[5];
      delta = SR0vec[6];
      if (delta<=err)
        state=done;
        SR1vec=SR0vec;
        SR2vec=SR0vec;
        a = SR0vec[1];
        b = SR0vec[2];
        c =(a+b)/2;
        err=SR0vec[5];
        delta=SR0vec[6];
        delta2=delta/2;
        SR1vec=srule(f,a,c,delta2);
        SR2vec=srule(f,c,b,delta2);
        err = abs(SR0vec[3]-SR1vec[3]-SR2vec[3])/10;
        if err<delta
          SRmat[p,:]=SR0vec;
          SRmat[p,4]=SR1vec[3]+SR2vec[3];
          SRmat[p,5]=err;
        else
          SRmat = [SRmat; zeros(1,6)]
          SRmat[p+1:m+1,:] = SRmat[p:m,:];
          m=m+1;
          SRmat[p,:]=SR1vec;
          SRmat[p+1,:]=SR2vec;
          state=iterating;
        end
      end
    end
  end
  sol = sum(SRmat[:,4]);
  err = sum(abs.(SRmat[:,5]));
  SRmat = SRmat[1:m,[1,2,4,5,6]];
  return sol, err, SRmat
end

function srule(f,a0,b0,delta0)
  h = (b0-a0)/2;
  C = zeros(1,3)
  C = f.([a0 (a0+b0)/2 b0]);
  S = h*(C[1]+4*C[2]+C[3])/3;
  S2= S;
  delta1=delta0;
  err=delta0;
  Z = [a0 b0 S S2 err delta1];
  return Z
end
```

### Contoh 4
Diberikan integral tentu

$$ \int_{0}^{4}{13(x-x^2)e^{-3x/2}}\,dx  $$

Berikut merupakan langkah-langkah untuk menghitung integral di atas menggunakan aturan kuadratur adaptif dengan toleransi awal $ \epsilon=0.00001 $ dan menggambarkan pembagian selang $ [a_k,b_k] $.

**Langkah 1**: Hitung nilai hampiran integral dengan metode kuadratur adaptif.

```julia
f(x) = 13*(x-x.^2).*exp(-3*x/2);
a = 0;
b = 4;
@btime sol,err,SRmat = adaptif(f,a,b,delta=0.00001)
sol,err,SRmat = adaptif(f,a,b,delta=0.00001)
@show sol
@show err
SRmat
```

**Langkah 2**: Gambarkan pembagian selang $ [a_k,b_k] $ metode kuadratur adaptif untuk fungsi $ f(x)=13(x-x^2)e^{-3x/2} $.

```julia
using Plots
```

```julia
x = [SRmat[:,1],b]
plt = plot(x,f,legend=:false)
scatter!(x,f)
```

# Soal Latihan
Kerjakan soal berikut pada saat kegiatan praktikum berlangsung.

`Nama: ________`

`NIM: ________`

### Soal 1
Ulangi langkah-langkah pada **Contoh 1** untuk menghitung nilai $ T(J) $ untuk $ J=0,1,2,3,4,5 $ dari $$ \int_{0}^{\pi/2} (x^2+x+1)\cos(x) \,dx $$
Hitung pula nilai galat $ T(J) $, jika diketahui nilai solusi eksak integral adalah $ 2.038197427 $.

```julia

```

### Soal 2
Ulangi langkah-langkah pada **Contoh 3** untuk menghitung nilai dari

$$  \int_{0}^{2}{\sqrt{4x-x^2}}\,dx  $$

menggunakan integral Romberg hingga mendapatkan galat $ \epsilon = 10^{-10} $. Hitunglah waktu komputasi yang diperlukan menggunakan macro `@btime` pada `BenchmarkTools`.

**Catatan**. Hal ini akan memungkinkan untuk mengubah nilai `maxi` dan `delta` pada fungsi `romberg`. Perhatikan pula nilai galat dan _flag_ yang dihasilkan. Pastikan bahwa _flag_ bernilai 0. Jika _flag_ bernilai 1, kemungkinan besar dikarenakan nilai maksimum iterasi terlalu kecil, sehingga sesuaikan nilai `maxi` agar `flag` bernilai 0.

```julia
@btime sol1,flag,err1,R = romberg(f,a,b,maxi=100,delta=tol)
sol1,flag,err1,R = romberg(f,a,b,maxi=100,delta=tol)
@show (sol1,flag,err1)
```

```julia

```

### Soal 3
Diberikan fungsi

$$ f(x)=\sqrt{4x-x^2} $$

Gunakan metode kuadratur adaptif dengan toleransi awal $ \epsilon=10^{-10} $ untuk menghitung solusi integral $ \int_{0}^{2}{f(x)}\,dx $, kemudian gambarkan pembagian selang $ [a_k,b_k] $ metode tersebut seperti pada **Contoh 4**.

Hitunglah waktu komputasi yang diperlukan dengan paket `BenchmarkTools`. Bandingkan lama waktu komputasi yang diperlukan dengan soal latihan pada aturan Romberg.

```jl
@btime sol2,err2,SRmat = adaptif(f,a,b,delta=tol)
sol2,err2,SRmat = adaptif(f,a,b,delta=tol)
@show (sol2,err2)
```

```julia

```

### Soal 4
Ulangi soal 3 dan soal 4 dengan mengatur toleransi menjadi $\epsilon=10^{-13}$ dan $\epsilon=10^{-16}$.

```julia

```