# Quantum Programming: Single Qubits

This code is meant to be used with the book.  The chapter should be in the same directory.  You will need the learn_quantum.py library and Qiskit.

This material assumes you know the basics of python programming and Qiskit.  Qiskit is used here because it has an easy interface to access the transformation matrix.

You can read the documentation on how to install and use [Qiskit](https://qiskit.org/documentation/terra/summary_of_quantum_operations.html#u-gates).

This goal of this material is to cover concepts used in quantum programming -- not to be a lesson in linear alegbra nor to provide a tutorial on Qiskit or any other library.  You can review basic information on the linear algebra of quantum computing from [Microsoft](https://docs.microsoft.com/en-us/quantum/concepts/vectors-and-matrices?view=qsharp-preview).

```python
from qiskit.visualization import plot_bloch_multivector
from qiskit.visualization.bloch import Bloch
from qiskit import QuantumCircuit, QuantumRegister, ClassicalRegister
import learn_quantum as lq
from learn_quantum import ket, bra, np_array_to_latex as a_to_l, show_array, show_me_the_matrix
from IPython.display import Latex
import numpy as np
from math import pi as π, pi, sqrt

```

## Single Qubits and the Standard Basis

The examples present the information in Latex using various support functions in the 'learn_quantum' library. By default, rounded brackets are used. If you prefer square brackets, set lq.default_bracket_type = 'b'.

Other common paramaters are factor_out and label which are demonstrated in these exercises.

Above, np_array_to_latex is setup with an alias of a_to_l.  "llf" is for Latex Line Feed and is used to put multiple expressions in a single output.

```python
Latex(a_to_l(ket('0')) + lq.llf + lq.np_array_to_latex(ket('1')))
```

\begin{equation*}\begin{pmatrix}1 \\
 0  \\
\end{pmatrix}\end{equation*}\begin{equation} \\ \end{equation}\begin{equation*}\begin{pmatrix} 0  \\
1 \\
\end{pmatrix}\end{equation*}

```python
lq.default_bracket_type = 'b'
Latex(a_to_l(ket('+')) + lq.llf + a_to_l(ket('-'), factor_out=False))
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{bmatrix}1 \\
1 \\
\end{bmatrix}\end{equation*}\begin{equation} \\ \end{equation}\begin{equation*}\begin{bmatrix}\frac{1}{\sqrt{2}} \\
-\frac{1}{\sqrt{2}} \\
\end{bmatrix}\end{equation*}

```python
lq.default_bracket_type = 'p'
```

____
## Single Qubit Transformations in Bra-Ket

All quantum programming amounts to matrix manipulation of an initial state by a transformation matrix.  This is represented as $\vert \psi'\rangle = U\vert \psi\rangle$. Where $U$ is a [unitary matrix](https://en.wikipedia.org/wiki/Unitary_matrix).

Quantum gates are basic unitary matricies that map one state onto another.  One way to view these gates is using the bra-ket notation to "project" an initial state onto a new state.

### X Gate

The X Gate can be constructed using ket and bra "projections."  The gate transforms 0 to 1 and 1 to 0.  Also known as the Not Gate.  But please see the discussion in the text about how a "Not" gate does not always perform a "Not" function.  In learning quantum programming, it is important to note that names given to various gates serve primmarily to identify the gate by its action on a non-quantum (classical) state.  This is suffcient for a basic understanding in this first chapter, but when acting on quantum states, the gate will not act in a classical manner.

\begin{equation*} X=\vert 0\rangle \langle 1 \vert   +   \vert 1\rangle \langle 0 \vert =
\end{equation*}

\begin{equation*}
\begin{pmatrix}1&0\end{pmatrix}\begin{pmatrix}0\\1\end{pmatrix} +
\begin{pmatrix}0&1\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix} =
\end{equation*}

\begin{equation*}
\begin{pmatrix}0&1\\0&0\end{pmatrix} + \begin{pmatrix}0&0\\1&0\end{pmatrix} = \begin{pmatrix}0&1\\1&0\end{pmatrix}
\end{equation*}

Note that above, the Latex was being formated to put multiple arrays in a single cell.  Use show_array to display a single array.

```python
Ux = ket('0')@bra('1') + ket('1')@bra('0')
show_array(Ux, label='X')
```

\begin{equation*}X = \begin{pmatrix} 0  & 1 \\
1 &  0  \\
\end{pmatrix}\end{equation*}

```python
# Multiply the X-Gate by an intial state of 0 give a new state of 1.
show_array(Ux@ket('0'))
```

\begin{equation*}\begin{pmatrix} 0  \\
1 \\
\end{pmatrix}\end{equation*}

```python
# Multiply the X-Gate by an intial state of 1 give a new state of 0.
show_array(Ux@ket('1'))
```

\begin{equation*}\begin{pmatrix}1 \\
 0  \\
\end{pmatrix}\end{equation*}

_____
### Z Gate

The Z gate maps 0 onto 0 and 1 onto -1.  It is known as the "phase flip" gate.

\begin{equation*}
Z = 1\vert 0\rangle \langle 0 \vert   - 1 \vert 1\rangle \langle 1 \vert =
\end{equation*}

\begin{equation*}
\begin{pmatrix}1&0\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix} -
\begin{pmatrix}0&1\end{pmatrix}\begin{pmatrix}0\\1\end{pmatrix} =
\end{equation*}

\begin{equation*}
\begin{pmatrix}1&0\\0&0\end{pmatrix} - \begin{pmatrix}0&0\\0&1\end{pmatrix} = \begin{pmatrix}1&0\\0&-1\end{pmatrix}
\end{equation*}

```python
U = ket('0')@bra('0') - ket('1')@bra('1')
show_array(U, label='Z')
```

\begin{equation*}Z = \begin{pmatrix}1 &  0  \\
 0  & -1 \\
\end{pmatrix}\end{equation*}

### Y Gate

The Y gate is represented in some texts as $Y = \begin{pmatrix}0&-1\\1&0\end{pmatrix}$ and is known as the bit and phase flip.  It is a combination of the X and Z gates. However, it is also frequently represented with an imaginary component. Either gate is the same gates up to the global phase -- that is, there is no computational difference between the gates and their entries differ by a common global factor.  See the text for detailed discussion of the global phase and how it is used to apply asthetics to certain matricies.  That is, you want a matrix to look a certain way to present it in literature, so a global phase is applied.  For computation purposes the matricies with 1/-1 and i/-i are identical.

The Y gate transforms $\vert 0\rangle$ onto $\vert 1\rangle$ with an amplitude of $\sqrt{1} = -i$ and $\vert 1\rangle$ onto $\vert 0\rangle$ with amplitude $\sqrt{-1} = i$.  In bra-ket notation, and factoring out the coefficient to the front of the term, this is:

\begin{equation*}
Y = -i\vert 0\rangle \langle 1 \vert   + i \vert 1\rangle \langle 0 \vert =
\end{equation*}

\begin{equation*}
\begin{pmatrix}0&-i\\0&0\end{pmatrix} + \begin{pmatrix}0&0\\i&0\end{pmatrix} =
\begin{pmatrix}0&-i\\i&0\end{pmatrix}
\end{equation*}

```python
U = -lq.imag*ket('0')@bra('1') + lq.imag*ket('1')@bra('0')
show_array(U, label='Y')
```

\begin{equation*}Y = \begin{pmatrix} 0  & -i \\
i &  0  \\
\end{pmatrix}\end{equation*}

```python
## Applly a global phase if you want the other form.  See text for discussion of global phase
show_array(-lq.imag * U, label='Y')
```

\begin{equation*}Y = \begin{pmatrix} 0  & -1 \\
1 &  0  \\
\end{pmatrix}\end{equation*}

```python
## Which is the same as X@Z
show_array(lq.pauliX@lq.pauliZ)
```

\begin{equation*}\begin{pmatrix} 0  & -1 \\
1 &  0  \\
\end{pmatrix}\end{equation*}

```python
## And a pi rotation on Y is the same
qr = QuantumRegister(1)
circuit = QuantumCircuit(qr)
circuit.ry(pi, qr)
show_me_the_matrix(circuit, label='Y')
```

\begin{equation*}Y = \begin{pmatrix} 0  & -1 \\
1 &  0  \\
\end{pmatrix}\end{equation*}

### Hadamard Gate

To transform an input of $\vert0\rangle$ onto either $\vert0\rangle$ or $\vert1\rangle$ with 50/50 probabilities, the values of a and b would be $\frac{1}{\sqrt{2}}$, as the sum of squares of these probabilities must equal one.  And to transform an input value of $\vert1\rangle$ on $\vert0\rangle$ or $\vert1\rangle$ with 50/50 probabilities, the values of c and d would also be $\frac{1}{\sqrt{2}}$.  Here, we will apply a "relative phase" of -1 to d to keep the matrix unitary, with an explanation of what they means coming later. The transformation is:
\begin{equation*}
 \frac{1}{\sqrt{2}}\vert 0 \rangle \langle 0 \vert + \frac{1}{\sqrt{2}}\vert 0 \rangle \langle 1 \vert + \frac{1}{\sqrt{2}}\vert 0 \rangle \langle 0 \vert - \frac{1}{\sqrt{2}} \vert 0 \rangle\langle 1 \vert =
 \end{equation*}
\begin{equation*}
 \begin{pmatrix}\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}\end{pmatrix} =
 \frac{1}{\sqrt{2}}\begin{pmatrix}1 & 1 \\
1 & -1 \\
\end{pmatrix}
\end{equation*}

```python
show_array(
    1/sqrt(2) * ket('0')@bra('0') + 1/sqrt(2) * ket('0')@bra('1')
+ 1/sqrt(2) * ket('1')@bra('0') - 1/sqrt(2) * ket('1')@bra('1'))
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}1 & 1 \\
1 & -1 \\
\end{pmatrix}\end{equation*}

___
## Visual Representation of a Qubit

```python
bloch1 = QuantumCircuit(2)
bloch1.x(1)
plot_bloch_multivector(lq.execute_state_vector(bloch1))
```

You can also step through a rotation to see how the rotation is performed.

```python
qr = QuantumRegister(1)
qc = QuantumCircuit(qr)

vectors = []
vectors.append(lq.get_bloch_vectors(qc)[0])
for k in range(8):
    qc.rx(π/8, qr[0])
    vectors.append(lq.get_bloch_vectors(qc)[0])

sphere = Bloch()
sphere.vector_color = ['#dc267f']
sphere.add_vectors(vectors)
sphere.render()
```

____
## Single qubits in superposition

```python
circuit = QuantumCircuit(1)
circuit.rx(π/2, 0)
lq.show_state_vector(circuit)
```

\begin{equation*} \vert \psi\rangle=\frac{1}{\sqrt{2}}\big( \vert0\rangle-i \vert1\rangle\big)\end{equation*}

```python
plot_bloch_multivector(lq.execute_state_vector(circuit))
```

```python
lq.show_state_vector(circuit, show_zeros=True)
```

\begin{equation*} \vert \psi\rangle=\frac{1}{\sqrt{2}}\big( \vert0\rangle-i \vert1\rangle\big)\end{equation*}

```python
lq.show_bloch_angles(circuit)
```

\begin{equation*} \vert \psi_0\rangle=cos\left(\frac{\pi}{2}\right)\vert 0 \rangle + sin\left(\frac{\pi}{2}\right)\vert 1 \rangle\end{equation*}\begin{equation} \\ \end{equation}

## Eigenvectors

Eigenvectors are discussed in the Appendix on eignevectors for discrete systems, but this is not to understate their importance.  Eigenvectors are the same thing as "quantum states," with an Eigenvector being the mathematical representation for an Eignstate or quantum state.  That is, any quantum system can be measured in a finite set of states.  Those states are the Eigenstates for the system.  A state can be in a superposition of Eigenstates, but when measured can only a be in single Eigenstate.  This is commonly referred to as "collapsing" -- a word that confuses the issue more than it explains or describes.

Here, Eigenvectors as just introduced without much explanation.  We will return to various Eigenvectors, Eigenvalues, Eignephases, and various other "eigen" properties of quantum systems throughout the book. All of quantum programming is manipulating a system to transition from one Eigenstate to another until it finally reaches the Eigenstate you want with a high degree of probability.

Qiskit only measures in the computational basis, which means that any final output will be Eigenvectors of the standard basis. Typically, this is handled by rotating the state into the basis you wish to measure in and the conducting a measurement in the standard basis.

This entire book is about how to manipulate Eigenstates within a quantum computer. The formal mathematics will be used where approriate, but if you really wish to understand quantum programming, a thorough understanding of Eigenvectors is essential.

```python
circuit = QuantumCircuit(1)
lq.show_eigens(circuit)
```

\begin{equation*}\lambda_0=1,\; v_0 = \begin{pmatrix}1 \\
 0  \\
\end{pmatrix}\quad\lambda_1=1,\; v_1 = \begin{pmatrix} 0  \\
1 \\
\end{pmatrix}\quad\end{equation*}

## Measurement

You can measure in other basis by performing a rotation into that basis and conducting a measumrent.  The X basis is measured by performing a Hadamared transformation and the Y basis is measured by performing a $S^\dagger$ and Hadamard.

Because the standard (Z) bases is definite, the other two are completely random.

Note that the "measureZ" variable does not do anything.  It is not necessary for this example to work, but is put in to illustrate the three different measruements.

```python
qr = QuantumRegister(1)
cr = ClassicalRegister(1)

## Put the qubit in the 1 state in the standard basis (Z),
# then measure it in the X, Y, Z
base = QuantumCircuit(qr, cr)
base.x(qr[0])

measureZ = QuantumCircuit(qr, cr)

measureX = QuantumCircuit(qr, cr)
measureX.h(qr[0])

measureY = QuantumCircuit(qr, cr)
measureY.sdg(qr[0])
measureY.h(qr[0])

m = QuantumCircuit(qr, cr)
m.measure(qr, cr)

shots = 1000
answers = lq.execute_simulated(base+measureX + m ,shots)
lq.print_reverse_results(answers, label='X')

answers = lq.execute_simulated(base + measureY + m ,shots)
lq.print_reverse_results(answers, label='Y')

answers = lq.execute_simulated(base + measureZ + m ,shots)
lq.print_reverse_results(answers, label='Z')
```

    Reversed:X: [('0', 499), ('1', 501)]
    Reversed:Y: [('0', 509), ('1', 491)]
    Reversed:Z: [('1', 1000)]

### Measurement support functions

Measurement across various axes is such a common tool for understanding small scale quantum systems that the library has some tools to make the task simple.  The following is identical to above code block:

```python
qr = QuantumRegister(1)
cr = ClassicalRegister(1)

## Put the qubit in the 1 state in the standard basis (Z),
# then measure it in the X, Y, Z
base = QuantumCircuit(qr, cr)
base.x(qr[0])

lq.print_measurements('XYZ', base, qr, cr, shots=1000)
```

    Reversed:X: [('0', 509), ('1', 491)]
    Reversed:Y: [('0', 506), ('1', 494)]
    Reversed:Z: [('1', 1000)]

## Basic superposition

The following sets up a basic superposition.  The factor=False prevents the $\frac{1}{\sqrt{2}}$ from being factored out.  Depending on which aspects of the system you are looking at, sometimes it is easier to see relationships when the terms are factored out.  Other times, it is easier without the factoring.

```python
circuit = QuantumCircuit(1,1)
circuit.rx(π/2, 0)
lq.show_state_vector(circuit, factor_out=False, label='\psi_{Rx}')
```

\begin{equation*} \vert \psi_{Rx}\rangle=\frac{1}{\sqrt{2}} \vert0\rangle-\frac{1}{\sqrt{2}}i \vert1\rangle\end{equation*}

And you can "colapse" the state vector.

```python
circuit.rx(-π/2, 0)
lq.show_state_vector(circuit, label='\psi_{Rx-Rx}')
```

\begin{equation*} \vert \psi_{Rx-Rx}\rangle=1 \vert0\rangle\end{equation*}

____
When you want to perform a measruement on a circuit but also wish to evaluate the transformation matrix or some sub-part of it, split the circuit into multiple parts.  Here, the measurement is split off so that we can evalute the circuit in its state prior to the measruement.

Note that if we are not passing the registers to any functions, you can leave off the QuantumRegister and ClassicalRegister declarations and just reference their indicies.

```python
circuit = QuantumCircuit(1,1)
m = QuantumCircuit(1,1)
circuit.rx(π/2, 0)
m.measure(0,0)
lq.show_state_vector(circuit + m, label='\psi_{Meas}')
```

\begin{equation*} \vert \psi_{Meas}\rangle=1 \vert0\rangle\end{equation*}

```python
lq.show_eigens(circuit)
```

\begin{equation*}\lambda_0=\frac{1}{\sqrt{2}}+\frac{1}{\sqrt{2}}i,\; v_0 = \frac{1}{\sqrt{2}}\begin{pmatrix}1 \\
-1 \\
\end{pmatrix}\quad\lambda_1=\frac{1}{\sqrt{2}}-\frac{1}{\sqrt{2}}i,\; v_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1 \\
1 \\
\end{pmatrix}\quad\end{equation*}

____
## Gate Composition

Gates are matricies and circuits, which are sequences of gates, a just matricies multiplied together.  However, the prior state vector is on the right, so the gates are applied in "reverse" order -- that is, from right-to-left.  The following is the general formula for the decomposition of any unitary matrix:

\begin{equation*}
    e^{i\alpha}
    \begin{pmatrix} e^{-i\beta/2} & 0 \\
                0 & e^{i\beta/2}\end{pmatrix}
    \begin{pmatrix} \cos{\frac{\gamma}{2}} & -\sin{\frac{\gamma}{2}} \\
    \sin{\frac{\gamma}{2}} & \cos{\frac{\gamma}{2}} \end{pmatrix}
    \begin{pmatrix}e^{-i\delta/2} & 0 \\
    0 & e^{i\delta/2} \end{pmatrix}
\end{equation*}

When working on constructing matricies, you can perform the multiplaction directly using numpy arrays or you can let Qiskit do it for you. Note that matricies are applied from right-to-left:

```python
show_array(lq.pauliX@lq.pauliY@lq.hadamard)
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}i & i \\
-i & i \\
\end{pmatrix}\end{equation*}

```python
circuit = QuantumCircuit(1)
circuit.h(0)
circuit.y(0)
circuit.x(0)
show_me_the_matrix(circuit)
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}i & i \\
-i & i \\
\end{pmatrix}\end{equation*}

```python
lq.show_state_vector(circuit, factor_out=False)
```

\begin{equation*} \vert \psi\rangle=\frac{1}{\sqrt{2}}i \vert0\rangle-\frac{1}{\sqrt{2}}i \vert1\rangle\end{equation*}

```python
show_array(lq.what_is_the_state_vector(circuit), factor_out=False)
```

\begin{equation*}\begin{pmatrix}\frac{1}{\sqrt{2}}i \\
-\frac{1}{\sqrt{2}}i \\
\end{pmatrix}\end{equation*}

___________
The state vector is just the transformation matrix multiplied by the existing state vector.  For a newly intialized qubit, that is just ket('0')

```python
show_array(lq.pauliX@lq.pauliY@lq.hadamard@ket('0'), factor_out=False)
```

\begin{equation*}\begin{pmatrix}\frac{1}{\sqrt{2}}i \\
-\frac{1}{\sqrt{2}}i \\
\end{pmatrix}\end{equation*}

# Exponential view

When composing gates, it can be helpful to view the matrices as exponents.

```python
show_array(lq.pauliX@lq.pauliY@lq.hadamard, display_exp=True)
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}e^{\frac{\pi}{2} i} & e^{\frac{\pi}{2} i} \\
e^{\frac{3\pi}{2} i} & e^{\frac{\pi}{2} i} \\
\end{pmatrix}\end{equation*}

```python
circuit = QuantumCircuit(1)
circuit.s(0)
show_me_the_matrix(circuit, display_exp=True)
```

\begin{equation*}\begin{pmatrix}1 &  0  \\
 0  & e^{\frac{\pi}{2} i} \\
\end{pmatrix}\end{equation*}

```python
circuit = QuantumCircuit(1)
circuit.t(0)
show_me_the_matrix(circuit, display_exp=True)
```

\begin{equation*}\begin{pmatrix}1 &  0  \\
 0  & e^{\frac{\pi}{4} i} \\
\end{pmatrix}\end{equation*}

# Hadamard as Fourier Transformation

The [Quantum Fourier Transformation](https://en.wikipedia.org/wiki/Quantum_Fourier_transform) will be used extensively and it is useful to think of the Hadamard gate as a 2x2 Fourier Tranformation.  The properties and uses of the Quantum Fourier Transforation will be covered in detail in the coming chapters, but it is useful to identify it as a simple case of the QFT rather than as the "superposition" gate.  Viewing it as an exponential makes this concept clear.

```python
circuit = QuantumCircuit(1)
circuit.h(0)
show_me_the_matrix(circuit, display_exp=True)
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}1 & 1 \\
1 & e^{\pi i} \\
\end{pmatrix}\end{equation*}

You should also look at the eigenvectors of the Pauli-X matrix -- which are the columns of the Hadamard transformation.  This is not a conincidence and will be explored in the section on the Quantum Fourier Transformation.

```python
lq.show_eigens(lq.pauliX)
```

\begin{equation*}\lambda_0=1,\; v_0 = \frac{1}{\sqrt{2}}\begin{pmatrix}1 \\
1 \\
\end{pmatrix}\quad\lambda_1=-1,\; v_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1 \\
-1 \\
\end{pmatrix}\quad\end{equation*}

```python
show_array(lq.hadamard)
```

\begin{equation*}\frac{1}{\sqrt{2}}\begin{pmatrix}1 & 1 \\
1 & -1 \\
\end{pmatrix}\end{equation*}

## Mechanical Composition

```python
qr = QuantumRegister(1,'r1') ## Apply a label to identify in the circuit diagram
custom = QuantumCircuit(qr)

alpha = π/4
beta = π/2
gamma = π/2
delta = -π/2

custom.append(lq.rrz_gate(delta), [qr[0]])
custom.ry(gamma, qr[0])
custom.append(lq.rrz_gate(beta), [qr[0]])
custom.append(lq.global_gate(alpha), [qr[0]])

custom.draw(output='mpl')
```

## Relative Phase

```python
qc = QuantumCircuit(1)
qc.s(0)
lq.show_me_the_matrix(qc, display_exp=True)
```

\begin{equation*}\begin{pmatrix}1 &  0  \\
 0  & e^{\frac{\pi}{2} i} \\
\end{pmatrix}\end{equation*}

```python
qc = QuantumCircuit(1)
qc.t(0)
lq.show_me_the_matrix(qc, display_exp=True)
```

\begin{equation*}\begin{pmatrix}1 &  0  \\
 0  & e^{\frac{\pi}{4} i} \\
\end{pmatrix}\end{equation*}

```python
qc = QuantumCircuit(1)
qc.u1(π/8, 0)
lq.show_me_the_matrix(qc, display_exp=True)
```

\begin{equation*}\begin{pmatrix}1 &  0  \\
 0  & e^{\frac{\pi}{8} i} \\
\end{pmatrix}\end{equation*}

## Exercise 1

```python
u = sqrt(3/4)*ket('0')@bra('0') \
    - sqrt(1/4)*ket('0')@bra('1') \
    + sqrt(1/4)*ket('1')@bra('0') \
    + sqrt(3/4)*ket('1')@bra('1')

lq.decompose_single(u)

```

    alpha= 0, beta= 0, gamma= pi/3, delta=0

```python
qr = QuantumRegister(1)
cr = ClassicalRegister(1)
circuit = QuantumCircuit(qr, cr)
circuit.ry(π/3, qr[0])
m = QuantumCircuit(qr, cr)
m.measure(qr, cr)
answers = lq.execute_simulated(circuit+m ,1000,
                              seed_simulator=12345)
lq.print_reverse_results(answers)
```

    Reversed: [('0', 746), ('1', 254)]

## Exercise 2
In this exercies, the negative sign is moved to the last term.  You can verify this still gives the same probability distribution when measured in the standard basis.  But it gives a very different probability distributions from the Ry rotation when they are both measured in the Hadamard basis.  This is because the "relative phase" of the systems is different -- in this case opposite.  When the system is rotated into a basis that reflects that phase, the results  will be different (in this case opposite).

Here, the system in Exercise 1 was under specified.  In our initial setup, the only requirment was a single axis measurement split 75/25.  This gave several options for how to handle the phase --- none of which affected the outcome of the simple exercies.  For most quantum programs, the phase is a major component of the computation and it needs to be properly defined.

This exerise introduces lq.decompose_single_qiskit(u) -- which decomposes a single qubit matrix into a single U3 gate.  The decompose_single function would work as well, but would requipe multiple gates.  To keep the example as simple as possible, it is easier to use the qiskit single gate decomposition.

```python
u = sqrt(3/4)*ket('0')@bra('0')  \
    + sqrt(1/4)*ket('0')@bra('1') \
    + sqrt(1/4)*ket('1')@bra('0') \
    - sqrt(3/4)*ket('1')@bra('1')

lq.decompose_single_qiskit(u)
```

    theta= pi/3, phi= 0, lambda= pi, phase=0

```python
qc = QuantumCircuit(1)
qc.u3(π/3,0, π, 0)
show_me_the_matrix(qc, label='U3(\pi/3,0,\pi)')
```

\begin{equation*}U3(\pi/3,0,\pi) = \begin{pmatrix}\frac{\sqrt{3}}{2} & \frac{1}{2} \\
\frac{1}{2} & -\frac{\sqrt{3}}{2} \\
\end{pmatrix}\end{equation*}

```python
qc = QuantumCircuit(1)
qc.ry(π/3, 0)
show_me_the_matrix(qc, label='R_y')
```

\begin{equation*}R_y = \begin{pmatrix}\frac{\sqrt{3}}{2} & -\frac{1}{2} \\
\frac{1}{2} & \frac{\sqrt{3}}{2} \\
\end{pmatrix}\end{equation*}

```python
qr = QuantumRegister(1)
cr = ClassicalRegister(1)
rel_phase = QuantumCircuit(qr, cr)
rel_phase.u3(-pi/3,0, pi, qr[0])
rel_phase.h(qr[0])

m_rel = QuantumCircuit(qr , cr)
m_rel.measure(qr[0], cr[0])

answers = lq.execute_simulated(rel_phase + m_rel ,1000,
                              seed_simulator=12345)
lq.print_reverse_results(answers)
```

    Reversed: [('0', 66), ('1', 934)]

```python
qr = QuantumRegister(1)
cr = ClassicalRegister(1)
initial_ry = QuantumCircuit(qr, cr)
initial_ry.ry(π/3, qr[0])
initial_ry.h(qr[0])

m_ry = QuantumCircuit(qr , cr)
m_ry.measure(qr[0], cr[0])

answers = lq.execute_simulated(initial_ry + m_ry ,1000,
                              seed_simulator=12345)
lq.print_reverse_results(answers)
```

    Reversed: [('0', 945), ('1', 55)]

```python
lq.show_state_vector(rel_phase, label='\psi_{u3}')
```

\begin{equation*} \vert \psi_{u3}\rangle=0.2588 \vert0\rangle+0.9659 \vert1\rangle\end{equation*}

```python
lq.show_state_vector(initial_ry, label='\psi_{Ry}')
```

\begin{equation*} \vert \psi_{Ry}\rangle=0.9659 \vert0\rangle+0.2588 \vert1\rangle\end{equation*}

## Exercise 3
See text, no code.

## Exercise 4

```python
ket('0')@bra('+')  + ket('1')@bra('-')
```

    array([[ 0.70710678,  0.70710678],
           [ 0.70710678, -0.70710678]])

## Exercise 5

```python
lq.decompose_single(lq.hadamard)
```

    alpha= pi/2, beta= 0, gamma= pi/2, delta=pi

```python
circuit = QuantumCircuit(1)

circuit.rz(pi, 0)
circuit.ry(pi/2, 0)

show_me_the_matrix(circuit, label='Hadamard')
```

\begin{equation*}Hadamard = \frac{1}{\sqrt{2}}\begin{pmatrix}1 & 1 \\
1 & -1 \\
\end{pmatrix}\end{equation*}

```python
circuit = QuantumCircuit(1)
circuit.h(0)
circuit.draw()
```

<pre style="word-wrap: normal;white-space: pre;background: #fff0;line-height: 1.1;font-family: "Courier New",Courier,monospace">        ┌───┐
q_0: |0>┤ H ├
        └───┘</pre>

```python
circuit = QuantumCircuit(1)
circuit.h(0)
circuit.decompose().draw()
```

<pre style="word-wrap: normal;white-space: pre;background: #fff0;line-height: 1.1;font-family: "Courier New",Courier,monospace">        ┌──────────┐
q_0: |0>┤ U2(0,pi) ├
        └──────────┘</pre>

## Exercise 6

```python
qr = QuantumRegister(1)
circuit = QuantumCircuit(qr)
circuit.append(lq.global_gate(π/4), [qr[0]])
circuit.h(qr[0])
show_me_the_matrix(circuit, label='Hadamard')
```

\begin{equation*}Hadamard = \frac{1}{2}\begin{pmatrix}1+i & 1+i \\
1+i & -1-i \\
\end{pmatrix}\end{equation*}

## Exercise 7

```python
srn = np.array([[0.5+0.5j, 0.5-0.5j],
       [0.5-0.5j, 0.5+0.5j]])
lq.decompose_single(srn)
```

    alpha= pi/4, beta= 3/2, gamma= pi/2, delta=pi/2

```python
qr = QuantumRegister(1)
circuit = QuantumCircuit(qr)

alpha = π/4
beta = -π/2
gamma = π/2
delta = π/2

circuit.append(lq.rrz_gate(delta), [qr[0]])
circuit.ry(gamma, qr)
circuit.append(lq.rrz_gate(beta), [qr[0]])
circuit.append(lq.global_gate(π/4), [qr[0]])

show_me_the_matrix(circuit, label='\sqrt{NOT}')
```

\begin{equation*}\sqrt{NOT} = \frac{1}{2}\begin{pmatrix}1+i & 1-i \\
1-i & 1+i \\
\end{pmatrix}\end{equation*}

```python
qc_srn = QuantumCircuit(1)
qc_srn.h(0)
qc_srn.s(0)
qc_srn.h(0)
show_me_the_matrix(qc_srn, label='\sqrt{NOT}')
```

\begin{equation*}\sqrt{NOT} = \frac{1}{2}\begin{pmatrix}1+i & 1-i \\
1-i & 1+i \\
\end{pmatrix}\end{equation*}

```python
lq.decompose_single_qiskit(srn)
```

    theta= pi/2, phi= 3/2, lambda= pi/2, phase=pi/4

```python
from qiskit.quantum_info.operators import Operator
```

```python
srn_gate = Operator(srn)
qr = QuantumRegister(1)
circ = QuantumCircuit(qr)
circ.append(srn_gate, [qr[0]])
print(circ.decompose().draw())

```

              ┌─────────────────────┐
    q12_0: |0>┤ U3(pi/2,-pi/2,pi/2) ├
              └─────────────────────┘

```python
qc_no_global = QuantumCircuit(1)
qc_no_global.u3(π/2, -π/2, π/2, 0)
show_me_the_matrix(qc_no_global, label='\sqrt{NOT}')

```

\begin{equation*}\sqrt{NOT} = \frac{1}{\sqrt{2}}\begin{pmatrix}1 & -i \\
-i & 1 \\
\end{pmatrix}\end{equation*}

```python
# Compare the two matrices with ignore_phase
mat = lq.what_is_the_matrix(qc_srn)
mat2 = lq.what_is_the_matrix(qc_no_global)
from qiskit.quantum_info.operators.predicates import matrix_equal
matrix_equal(mat, mat2, ignore_phase=True)
```

    True

```python

```


# Autoparametric resonance

System of equations of motion for the system:
\begin{aligned}
\ddot{R}&=R\dot{\theta}^2-\frac{k}{m}(R-l)+g\cos\theta+a\cos\Omega t\\
\ddot{\theta}&=-\frac{2}{R}\dot{R}\dot{\theta}-\frac{g}{m}\sin\theta
\end{aligned}

Find stationary point for linearization $\theta=0$, all derivatives are zero:
$$-\frac{k}{m}(R-l)+g=0\quad\implies\quad R=l+\frac{mg}{k}$$
linearize equations around stable point
$$R=R_0+r,\quad R_0=l+mg/k,\quad\theta_0=0$$
where $r/R_0\ll0$ and $\theta\ll0$. Scale time to get rid of frequencies $t'=\omega_1t$ assuming $\omega_1=2\omega_2$.
\begin{aligned}
\ddot{r}+\frac{1}{Q_1}\dot{r}+r&=l\dot{\theta}^2+a\cos\Omega t\\
\ddot{\theta}+\frac{1}{Q_2}\dot{\theta}+\frac{1}{4}\theta&=-2\dot{r}\dot{\theta}
\end{aligned}

Following to the averaging method:
\begin{aligned}
r &= u_1\cos\Omega t+v_1\sin\Omega t\\
\theta &= u_2\cos\frac{\Omega}{2}t+v_2\sin\frac{\Omega}{2}t\\
\dot{r} &= \Omega (-u_1\sin\Omega t+v_1\cos\Omega t)\\
\dot{\theta} &= \frac{\Omega}{2} (-u_2\sin\frac{\Omega}{2}t+v_2\cos\frac{\Omega}{2}t)
\end{aligned}
with additional constraints
\begin{aligned}
\dot{u}_1\cos\Omega t+\dot{v}_1\sin\Omega t &= 0\\
\dot{u}_2\cos\frac{\Omega}{2}t+\dot{v}_2\sin\frac{\Omega}{2}t &= 0
\end{aligned}

Introduce detuning parameters
$$\xi_1=1-\Omega^2,\quad\xi_2=1-\Omega^2/4$$

```python
import sympy as sp
```

```python
t, l, a, Q1, Q2 = sp.symbols('t, l, a, Q_1, Q_2')
u1, u2, v1, v2 = sp.symbols('u_1, u_2, v_1, v_2')
xi1, xi2 = sp.symbols('\\xi_1, \\xi_2')
W = sp.Symbol('\Omega', positive=True)
```

```python
r  =       u1 * sp.cos(W*t) + v1 * sp.sin(W*t)
dr = W * (-u1 * sp.sin(W*t) + v1 * sp.cos(W*t))
```

```python
th  =         u2 * sp.cos(W/2*t) + v2 * sp.sin(W/2*t)
dth = W/2 * (-u2 * sp.sin(W/2*t) + v2 * sp.cos(W/2*t))
```

```python
F1 = -dr/Q1/W  - xi1/W * r + l/W * dth**2 + a/W *sp.cos(W*t)
F2 = -dth/Q2/W - xi2/W * th - 2/l/(W/2) * dr * dth
```

```python
T = 4*sp.pi/W
```

```python
du1 = (sp.integrate(-sp.sin(W*t)*F1, (t, 0, T))/T).expand()
dv1 = (sp.integrate( sp.cos(W*t)*F1, (t, 0, T))/T).expand()
```

```python
du2 = (sp.integrate(-sp.sin(W/2*t)*F2, (t, 0, T))/T).expand()
dv2 = (sp.integrate( sp.cos(W/2*t)*F2, (t, 0, T))/T).expand()
```

Get equations for the steady-state solution.

```python
eq1=du1*2
eq1
```

$\displaystyle \frac{\Omega l u_{2} v_{2}}{4} + \frac{\xi_{1} v_{1}}{\Omega} - \frac{u_{1}}{Q_{1}}$

```python
eq2=dv1*2
eq2
```

$\displaystyle - \frac{\Omega l u_{2}^{2}}{8} + \frac{\Omega l v_{2}^{2}}{8} - \frac{\xi_{1} u_{1}}{\Omega} + \frac{a}{\Omega} - \frac{v_{1}}{Q_{1}}$

```python
eq3 = du2*2
eq3
```

$\displaystyle - \frac{\Omega u_{1} v_{2}}{l} + \frac{\Omega u_{2} v_{1}}{l} + \frac{\xi_{2} v_{2}}{\Omega} - \frac{u_{2}}{2 Q_{2}}$

```python
eq4=dv2*2
eq4
```

$\displaystyle - \frac{\Omega u_{1} u_{2}}{l} - \frac{\Omega v_{1} v_{2}}{l} - \frac{\xi_{2} u_{2}}{\Omega} - \frac{v_{2}}{2 Q_{2}}$

Replace real $u$, $v$ with complex analogs to easy account for damping
$$z_1=u_1+iv_1,\quad z_2=u_1-iv_1,\quad z_3=u_2+iv_2,\quad z_4=u_2-iv_2$$

```python
z1, z2, z3, z4=sp.symbols('z_1, z_2, z_3, z_4')
```

```python
sbs=[(u1, z1/2+z2/2), (v1, -sp.I*z1/2+sp.I*z2/2),
     (u2, z3/2+z4/2), (v2, -sp.I*z3/2+sp.I*z4/2)]
```

```python
eq5=eq1.subs(sbs).expand()
eq6=eq2.subs(sbs).expand()
eq7=eq3.subs(sbs).expand()
eq8=eq4.subs(sbs).expand()
```

```python
eq9  = (eq5 + sp.I * eq6).expand()
eq10 = (eq5 - sp.I * eq6).expand()
eq11 = (eq7 + sp.I * eq8).expand()
eq12 = (eq7 - sp.I * eq8).expand()
```

```python
eq9
```

$\displaystyle - \frac{i \Omega l z_{3}^{2}}{8} - \frac{i \xi_{1} z_{1}}{\Omega} + \frac{i a}{\Omega} - \frac{z_{1}}{Q_{1}}$

```python
eq10
```

$\displaystyle \frac{i \Omega l z_{4}^{2}}{8} + \frac{i \xi_{1} z_{2}}{\Omega} - \frac{i a}{\Omega} - \frac{z_{2}}{Q_{1}}$

```python
eq11
```

$\displaystyle - \frac{i \Omega z_{1} z_{4}}{l} - \frac{i \xi_{2} z_{3}}{\Omega} - \frac{z_{3}}{2 Q_{2}}$

```python
eq12
```

$\displaystyle \frac{i \Omega z_{2} z_{3}}{l} + \frac{i \xi_{2} z_{4}}{\Omega} - \frac{z_{4}}{2 Q_{2}}$

```python
sol1 = sp.solve((eq9,eq10),[z1,z2])
Z1 = sol1[z1].expand()
Z2 = sol1[z2].expand()
```

```python
Z1
```

$\displaystyle \frac{i Q_{1} \Omega^{2} l z_{3}^{2}}{- 8 i Q_{1} \xi_{1} - 8 \Omega} - \frac{8 i Q_{1} a}{- 8 i Q_{1} \xi_{1} - 8 \Omega}$

```python
Z2
```

$\displaystyle - \frac{i Q_{1} \Omega^{2} l z_{4}^{2}}{8 i Q_{1} \xi_{1} - 8 \Omega} + \frac{8 i Q_{1} a}{8 i Q_{1} \xi_{1} - 8 \Omega}$

Substitute them into the second couple of equations

```python
eq13 = eq11.subs([(z1,Z1),(z2,Z2)]).expand()
eq13
```

$\displaystyle \frac{Q_{1} \Omega^{3} z_{3}^{2} z_{4}}{- 8 i Q_{1} \xi_{1} - 8 \Omega} - \frac{8 Q_{1} \Omega a z_{4}}{- 8 i Q_{1} \xi_{1} l - 8 \Omega l} - \frac{i \xi_{2} z_{3}}{\Omega} - \frac{z_{3}}{2 Q_{2}}$

```python
eq14 = eq12.subs([(z1,Z1),(z2,Z2)]).expand()
eq14
```

$\displaystyle \frac{Q_{1} \Omega^{3} z_{3} z_{4}^{2}}{8 i Q_{1} \xi_{1} - 8 \Omega} - \frac{8 Q_{1} \Omega a z_{3}}{8 i Q_{1} \xi_{1} l - 8 \Omega l} + \frac{i \xi_{2} z_{4}}{\Omega} - \frac{z_{4}}{2 Q_{2}}$

Let the amplitudes of modes are
$$A_1^2 = z_1z_2 = u_1^2+v_1^2,\quad A_2^2 = z_3z_4 = u_2^2+v_2^2$$
The trivial solution to the system is $z_3=z_4=0$, $A_2=0$ and
\begin{equation}
z_{1,2}=\frac{a}{\pm\xi_1-\Omega/{iQ_1}},\quad A_1=\frac{a}{\sqrt{(1-\Omega^2)^2+\Omega^2/Q_1^2}}
\end{equation}

The system has nontrivial value

```python
A1, A2 = sp.symbols('A1, A2')
```

```python
eq15 = Q1*W**3*z3*A2**2/(-8*sp.I*Q1*xi1 - 8*W) - 8*Q1*W*a*z4/(-8*sp.I*Q1*xi1*l - 8*W*l) - sp.I*xi2*z3/W - z3/(2*Q2)
eq16 = Q1*W**3*z4*A2**2/(8*sp.I*Q1*xi1 - 8*W) - 8*Q1*W*a*z3/(8*sp.I*Q1*xi1*l - 8*W*l) + sp.I*xi2*z4/W - z4/(2*Q2)
```

```python
eq17 = eq15.collect([z3,z4])
eq18 = eq16.collect([z3,z4])
```

The linear system of two equations for $z_3$ and $z_4$ has a nontrivial solution when determinant is zero what yields $A_2^2$ value. Since A_2 is amplitude, nontrivial solution will be stable when $A_2$ has a real positive value, otherwise the solution is trivial.

```python
eq17
```

$\displaystyle - \frac{8 Q_{1} \Omega a z_{4}}{- 8 i Q_{1} \xi_{1} l - 8 \Omega l} + z_{3} \left(\frac{A_{2}^{2} Q_{1} \Omega^{3}}{- 8 i Q_{1} \xi_{1} - 8 \Omega} - \frac{i \xi_{2}}{\Omega} - \frac{1}{2 Q_{2}}\right)$

```python
eq18
```

$\displaystyle - \frac{8 Q_{1} \Omega a z_{3}}{8 i Q_{1} \xi_{1} l - 8 \Omega l} + z_{4} \left(\frac{A_{2}^{2} Q_{1} \Omega^{3}}{8 i Q_{1} \xi_{1} - 8 \Omega} + \frac{i \xi_{2}}{\Omega} - \frac{1}{2 Q_{2}}\right)$

```python

```


## Graphical Models

```python
import graphviz as gz
import numpy as np
```

GMs are depictions of independence/dependence relationships for distributions. All GMs have their strengths and weaknesses. Belief networks is one type of a GM, they are useful to represent ancestral conditional independence; however, they cannot always represent dependence relationships.

In this chapter, we will mention Markov networks, chain graphs, and factor graphs.

The framework used for graphical models can be summarized into two parts:
1. **Modelling**
  1. Identify variables in the problem environment.
  2. Describe how these variables interact. This is achieved by making structural assumptions about the joint probability distribution. **Each class of GM corresponds to a factorization property of the joint distribution**.
2. **Inference**
  1. Use algorithms to make inference on GMs.

### Markov Networks
\begin{definition}
A potential $\phi(x)$ is a non-negative function of $x$. A distribution is a special type of potential satisfying $\sum_x \phi(x) = 1$.
\end{definition}

For a set of variables $X = \{x_1, \dots, x_n\}$, a **Markov Network** is defined as a product of potentials on subsets of the variables $X_c \subseteq X$:
$$
p(x_1, \dots, x_n) = \frac{1}{Z}\prod_{c = 1}^C \phi_c(X_c)
$$
where $Z$ is a constant which ensures normalisation, called the partition function. Graphically, this is represented by an undirected graph G with $X_c, c = 1, \dots, C$ being the **maximal cliques** of G. If clique potentials are all strictly positive, then the network is called a Gibbs distribution.

* A **pairwise Markov network** corresponds to an undirected graph $G$ having maximal cliques of size $2$ only.

#### Examples

##### $p(A, B, C, D) = \frac{1}{Z}\phi(A, B)\phi(B, C)\phi(C, D)\phi(A, D)$

```python
G = gz.Graph()
G.engine = 'circo'
G.edges(['AB', 'BC', 'CD', 'AD'])
G
```

##### $p(A, B, C, D, E, F) = \frac{1}{Z}\phi(A, B, C)\phi(B, C, D)\phi(D, E)\phi(D, F)$

```python
G.clear(keep_attrs=True)
G.edges(['AB', 'BC', 'AC', 'CD', 'BD', 'DE', 'DF'])
G
```

#### Properties
\begin{definition}
A subset $S$ **separates** a subset $A$ from a subset $B$ (for disjoint $A$ and $B$) if every path from $A$ to $B$ passes through $S$. If there is no path from $A$ to $B$, then $A$ and $B$ are separated even if $S = \varnothing$.
\end{definition}

##### Marginalizing and Conditioning
Let $p(A, B, C) = \phi(A, C)\phi(B, C)/Z$.

* Initially $A$ and $B$ are not independent because of the path from $A$ to $B$.
1. **Marginalizing** over $C$ means we don't have any information on $C$. Not knowing $C$ doesn't get rid of the fact that $A$ and $B$ are not independent. Hence, marginalizing $C$ makes $A$ and $B$ graphically dependent, and in general $p(A, B) \neq p(A)p(B)$.
2. **Conditioning** on $C$ means we know $C$. In this case, $A$ can be explained without knowing $B$ due to the Markov property, and vice versa. Hence, $A \ci B \mid C$.
  * **Markov Property**: If we know the parents of a variable, then the ancestors of the parents doesn't tell us anything new about that variable.

##### Global Markov Property
For disjoint sets of variables $(A, B, S)$ where $S$ separates $A$ and $B$ in G, $A \ci B \mid S$.

##### Algorithm for Independence
Due to the separation property, $A \ci B \mid S$ can be easily checked in Markov networs with the following procedure
1. Remove all links that neighbour the set of variables $S$
2. If there is no path between $A$ and $B$, then $A \ci B \mid S$.

##### Local Markov Property
* **Valid for positive potentials**

When conditioned on its neighbours, $x$ is independent of the remaining variables in the graph, i.e. $p(x \mid X \setminus x) = p(x \mid ne(x))$ where $ne(x)$ is the set of neighbours of $x$.

##### Pairwise Markov Property
For any **non-adjacent** vertices $x$ and $y$, $x \ci y \mid X \setminus \{x, y\}$

#### Markov Random Fields
As MRF is defined by a set of distributions $p(x_i \mid ne(x_i))$ where $i$ indexes the distributions. A distribution is an MRF with respect to an undirected graph $G$ if $p(x_i \mid x_{\backslash i}) = p(x_i \mid ne(x_i))$ where $x_{\backslash i} = X \setminus x_i$.

#### Hammersley-Clifford Theorem
Let $G$ be an undirected graph and $F$ be the factorisation as the product of clique potentials defined on $G$.

1. ($G \implies F$): Local Markov property implies that if we know $ne(x)$, we don't need the other variables. If we do this for each variable, then we naturally get a product of maximal clique potentials.
2. ($F \implies G$): Given a factorisation $F$, local Markov properties on $G$ are implied.

See BRML 4.2.3 for a detailed discussion.

#### Conditional Independence in Belief and Markov Networks
For sets of variables $X, Y, Z$, determine whether $X \ci Y \mid Z$.

The following procedure applies to both belief (Bayes) and Markov networks. For Markov networks, only the final separation step is applied.

1. **Ancestral Graph**
  1. Identify the ancestors $A$ of $(X \cup Y \cup Z)$.
  2. For all nodes $n$, if $n \notin (A \cup X \cup Y \cup Z)$, remove $n$ and all edges in/out of $n$.
2. **Moralisation**
  1. For all immoralities $(A, B, C)$ where $C$ is child of both $A$ and $B$,
    1. add a link between $A$ and $B$,
    2. remove link between $A$ and $C$,
    3. remove link between $B$ and $C$.
3. **Separation**
  1. Remove links neighbouring $Z$.
  2. Look for a path from $X$ to $Y$. If there is no such path, then $X \ci Y \mid Z$.

#### Lattice Model for Image Cleaning
An example Lattice model is:

```python
def lattice_graph(n):
    """
    Construct an nxn lattice graph as graphviz.Graph object.

    Parameters
    ----------
    n : int
        Number of nodes per row and column.

    Returns
    -------
    graph : graphviz.Graph
        Corresponding lattice graph with nodes indexed from 0 to n-1.
    """
    G = gz.Graph()
    for i in range(n):
        with G.subgraph() as S:
            S.attr(rank='same')
            for j in range(n):
                S.node(str(n*i + j))

    edge_set = set()
    for index in range(n*n):
        i, j = divmod(index, n)
        if i > 0:
            edge_set.add(frozenset((index, n*(i - 1) + j)))
        if i < n - 1:
            edge_set.add(frozenset((index, n*(i + 1) + j)))
        if j > 0:
            edge_set.add(frozenset((index, n*i + j - 1)))
        if j < n - 1:
            edge_set.add(frozenset((index, n*i + j + 1)))

    sorted_edges = (sorted(pair) for pair in edge_set)
    str_edges = ((str(first), str(second)) for (first, second) in sorted_edges)
    G.edges(str_edges)
    return G

lattice_graph(4)
```

and the corresponding Markov network model corresponds to the following joint distribution
$$
p(x_1, \dots, x_9) = \frac{1}{Z}\prod_{i \sim j} \phi_{ij}(x_i, x_j)
$$
where $i \sim j$ if $i$ and $j$ are neighbours.

This model can be thought as representing a locality principle where if we know the neighbours of a node, we know everything we need to know about that node. We can use this idea for images since pixels tend to be similar to their neighbours.

### Chain Graphical Models
Chain Graphs are a mixture of directed models (Bayes Nets) and undirected models (Markov Networks).

#### Example

```python
G = gz.Digraph()
with G.subgraph() as S:
    S.attr(rank='same')
    S.node('a')
    S.node('b')
G.edges(['ac', 'bd'])
with G.subgraph() as S:
    S.attr(rank='same')
    S.edge('c', 'd', dir='none')
G
```

corresponds to $p(a, b, c, d) = p(c, d \mid a, b)p(a)p(b)$ with
$$
p(c, d \mid a, b) = \frac{\phi(c, d)p(c \mid a)p(d \mid b)}{\sum_{c, d} \phi(c, d)p(c \mid a)p(d \mid b)}
$$

A chain graph is a DAG over the chain components. Hence, chain components can be thought of as Markov networks whereas the whole DAG is a belief network over these separate Markov networks.

\begin{definition}
The chain components of a graph $G$ are obtained by
1. Forming a graph $G\prime$ with directed edges removed from $G$
2. Each connected component in $G\prime$ constitutes a chain component
\end{definition}

* A Bayesian Network is a Chain Graph where each connected component is a singleton.
* A Markov Network is a Chain Graph with a single connected component.

---

* Chain graphs can represent independence statements that no Markov or Bayes network can represent alone.
  1. Markov networks can represent cyclic dependencies while Bayes networks cannot.
  2. Bayes networks can represent certain marginal independencies which Markov networks cannot (3 node network with a collider).
  3. A chain graph can represent both of these independence assumptions in a single model.

#### Chain Graph Distribution
The distribution associated with a chain graph $G$ is found by first identifying the chain components, $\tau$ and their associated variables $X_{\tau}$. Then
$$
p(x) = \prod_{\tau} p(X_{\tau} \mid pa(X_{\tau}))
$$
where
$$
p(X_{\tau} \mid pa(X_{\tau})) \propto \prod_{d \in D_{\tau}} p(x_d \mid pa(x_d))\prod_{c \in C_{\tau}} \phi(X_c)
$$
where $C_{\tau}$ denotes the union of the cliques in the component $\tau$, $D_{\tau}$ is the set of variables in component $\tau$ that correspond to directed terms $p(x_d \mid pa(x_d))$. The proportionality factor is determined implicitly by the constraint that the distribution sums to $1$.

##### Example CG Distribution

```python
G = gz.Digraph()
for level, rank in [('bgc', 'min'), ('ade', 'same'), ('hf', 'max')]:
    with G.subgraph() as S:
        S.attr(rank=rank)
        for node in level:
            S.node(node)

directed_edges = ['ah', 'ab', 'cd', 'cg']
undirected_edges = ['df', 'ad', 'bg', 'eh', 'ef', 'ae']
G.edges(directed_edges)
for edge in undirected_edges:
    G.edge(edge[0], edge[1], dir='none')

G
```

When the directed edges are removed, the connected components are $b, cg, aedhf$. Hence, this CG corresponds to the following Bayes Net

```python
G.clear()
G.edges([('aedhf', 'bg'), ('c', 'bg'), ('c', 'aedhf')])
G
```

where each node of this Bayes Net is itself a Markov Network. Hence, the overall distribution of the CG can be written as
$$
p(\cdot) \propto p(b \mid a)p(g \mid c)\phi(b, g) \cdot p(d \mid c)\phi(a, e)\phi(e, h)\phi(e, f)\phi(d, f)\phi(a, d) \cdot p(c)
$$

### Factor Graphs
Given a function $f(x_1, \dots, x_n) = \prod_i \psi_i(X_i)$, the factor graph has a node (a square) for each factor $\psi_i$ , and a variable node (circle) for each variable $x_j$. For each $x_j \in X_i$ an undirected link is made between factor $\psi_i$ and variable $x_j$.

When used to represent a distribution, we need to normalize:
$$
p(x_1, \dots, x_n) = \frac{1}{Z}\prod_i \psi_i(X_i)
$$
where $Z = \sum_X \prod_i \psi_i(X_i)$ is the normalisation constant.

If a factor $\psi_i(X_i)$ is a conditional distribution in the form $p(x_i \mid pa(x_i))$, then we can use directed edges from the parents of the factor to the factor, and from the factor to the children of the factor. Directed edges allows us to preserve the information that the factor is a probability distribution.

#### Example
$$
p(a, b, c) = p(a \mid b, c)p(b)p(c).
$$

##### Corresponding Bayes Net

```python
G = gz.Digraph()
G.edges(['ba', 'ca'])
G
```

##### Corresponding Factor Graph
$$
p(a, b, c) \propto \psi_1(a, b, c)\psi_2(b)\psi_2(c)
$$

```python
G = gz.Digraph()
G.node('fa', shape='square', style='filled', color='black', width='0.1')
G.node('fb', shape='square', style='filled', color='black', width='0.1')
G.node('fc', shape='square', style='filled', color='black', width='0.1')
G.edges([('fc', 'c'), ('fb', 'b'), ('fa', 'a'), ('b', 'fa'), ('c', 'fa')])
G
```

#### Factor Graph Conditional Independence
Two disjoint sets of variables $A$ and $B$ are conditionally independent given a third set $C$ if all paths from $A$ to $B$ are blocked. A path is blocked if at least one of the following is satisfied:
1. One of the variables in the path is in the conditioning set
2. One of the variables **or factors** in the path is a collider, and neither the variable/factor nor any of its descendants are in the conditioning set.

### Expressiveness of Graphical Models

#### Bayes Network to Markov Network
Each directed distribution can be represented as an undirected distribution by associating a potential for each factor of the directed distribution. However, transformation from a bayes net to a markov net **may cause information loss!**

\begin{remark}
Markov network associated with a Bayes network is the **moralised** version of the Bayes network. One can moralise a Bayes network by drawing an undirected edge between each pair of unconnected nodes which have a common child.
\end{remark}

##### Moralisation Causes Loss of Independence Information
Consider

```python
G = gz.Digraph()
G.edges(['ac', 'bc'])
G
```

In this Bayes net, $a \ci b$. However, if we write the corresponding Markov network

```python
G = gz.Graph()
G.engine = 'neato'
G.edges(['ab', 'bc', 'ac'])
G
```

we cannot infer from the graph that $a \ci b$. During the transformation process, $a \ci b$ information is lost.

#### Markov Network to Bayes Network
Representing cyclic dependencies is not possible with a Bayes Network

#### Distribution Maps
\begin{definition}
A graph $G$ is an **independence map** of a distribution $P$ if every conditional independence statement that one can derive from $G$ is true in $P$.
  * By contraposition, every dependency statement one can derive from $P$ is true in $G$.

Conversely, a graph $G$ is a **dependence map** of a distribution $P$ if every conditional independence statement that one can derive from $P$ is true in $G$.
  * By contraposition, every dependency statement one can derive from $G$ is true in $P$.

A graph $G$ which is both an I-map and a D-map of distribution $P$ is a **perfect map** of $P$.
\end{definition}

Let $L_P$ be the set of all conditional independence statements one can derive from $P$. Define $L_G$ similarly. Then, according to these definitions,
$
\begin{align}
\text{G is a  D-map} &\implies L_P \subseteq L_G, \\
\text{G is an I-map} &\implies L_G \subseteq L_P, \\
\text{G is a perfect map} &\implies L_G = L_P.
\end{align}
$

##### Maps of Distribution Classes
In this case, all numerical instances of the distribution class must obey the requirements. To do this, $L_P = \cup_i L_{P_i}$ where $P_i$ is a numerical instance of distribution $P$.


# RST design for a DC-motor
From HW3 2017 spring semester

## Plant model
  Zero-order-hold sampling of the DC-motor with transfer function
  $$ G(s) = \frac{1}{s(2s+1)} $$
  gives a discrete-time system with pulse transfer function
  \begin{equation}
  G_d(z) = \frac{B(z)}{A(z)}= \frac{2\big(\frac{h}{2}-1+\mathrm{e}^{-\frac{h}{2}}\big)z + 2\big(1-\mathrm{e}^{-\frac{h}{2}}-\frac{h}{2}\mathrm{e}^{-\frac{h}{2}}\big)}{(z-1)\big(z-\mathrm{e}^{-\frac{h}{2}}\big)}.
  \end{equation}

```python
import numpy as np
import sympy as sy
from sympy.utilities.codegen import codegen
import control.matlab as cm
import re
import matplotlib.pyplot as plt
from scipy import signal
```

```python
z = sy.symbols('z', real=False)
r1,s0,s1 = sy.symbols('r1,s0,s1', real=True)
hh = sy.symbols('h', real=True, positive=True)

Bp = 2*(0.5*hh - 1 + sy.exp(-0.5*hh))*z + 2*(1-sy.exp(-0.5*hh) - 0.5*hh*sy.exp(-0.5*hh))
Ap = (z-sy.exp(-0.5*hh))*(z-1)
print(Bp)
print(Ap)
```

    -1.0*h*exp(-0.5*h) + z*(h + 2*exp(-0.5*h) - 2) - 2*exp(-0.5*h) + 2
    (z - 1)*(z - exp(-0.5*h))

## Determine sampling period and desired closed loop poles
   In a continuous-time description of the desired closed-loop system we want the system to have two dominating poles at
   $$ -5 \pm i5. $$
   In addition to the two dominating poles, we want a third pole at
   $$ a=-20 $$
   to be able to control the response to disturbances. Determine a suitable sampling period $h$, and determine the poles (and characteristic polynomial) of the desired discrete-time closed-loop system.

### Solution
   Since we have dominating complex-conjugated poles, the sampling period should be such that
   $$ h\omega_n = 0.1 -- 0.6. $$
   Choose $$h = \frac{0.3}{\omega_n} = \frac{0.3}{\sqrt{5^2+5^2}} = \frac{0.3}{5\sqrt{2}} \approx 0.04. $$
   We get the discrete-time poles
   \begin{align*}
   p_{d_1} &= \exp{0.04(-5+i5)} = \exp(-0.2) \exp(i0.2) = 0.81\big(\cos(0.21) + i\sin(0.21)\big) \approx 0.80 + i0.16\\
   p_{d_2} &= p_{d_1}^* = 0.80 - i0.16\\
   p_{d_3} &= \exp(ah) = \exp(-20\cdot 0.04) = 0.45.
   \end{align*}

```python
Ap = sy.simplify(Ap.subs(hh, 0.04))
Bp = sy.simplify(Bp.subs(hh, 0.04))
print(Bp)
print(Ap)
```

    0.000397346613510541*z + 0.000394706454219282
    (z - 1)*(z - 0.980198673306755)

## Design a 2-DoF controller
   Assume a structure of the controller as given in figure \ref{fig:2dof}. The controller is given by
   $$ R(q)u = -S(q)y + T(q)u_c. $$
   With the plant-model
   $$ A(q)y = B(q)u$$
   we get the following difference equation for the closed-loop system
   $$ \big( A(q)R(q) + B(q)S(q) \big) y = B(q)T(q) u_c. $$
   Assume a suitable order (as low as possible) of the controller polynomials $R(q)$ and $S(q)$ and solve the diophantine equation
   $$ A(q)R(q) + B(q)S(q)  = Ac(q) $$
   for $R$ and $S$.

   Solve the equations for arbitrary $a$: Use a symbol $a$ in your calculations so that you can easily recalculate your controller for a different value of $a$.

```python
z = sy.symbols('z', real=False)
r1,s0,s1,aa = sy.symbols('r1,s0,s1,aa', real=True)

Ac = z**2 - z*(0.8*2) + (0.8**2 + 0.16**2)
Ao = z-aa
Acl = Ac*Ao

Rp = z+r1
Sp = s0*z + s1
RHS = sy.simplify(sy.expand(Acl))
diopheq =sy.poly(Ap*Rp+Bp*Sp-Acl, z)
dioph = diopheq.all_coeffs()
print(sy.latex(sy.poly(RHS, z)))
print(dioph)
print(Acl)
print(Ap*Rp)
print(Ac)
print(Ap*Rp)
print(sy.poly(sy.simplify(sy.expand(Ap*Rp + Bp*Sp)), z))
```

    \operatorname{Poly}{\left( - 0.6656 aa + 1.0 z^{3} + z^{2} \left(- 1.0 aa - 1.6\right) + z \left(1.6 aa + 0.6656\right), z, domain=\mathbb{R}\left[aa\right] \right)}
    [1.0*aa + 1.0*r1 + 0.000397346613510541*s0 - 0.380198673306755, -1.6*aa - 1.98019867330676*r1 + 0.000394706454219282*s0 + 0.000397346613510541*s1 + 0.314598673306755, 0.6656*aa + 0.980198673306755*r1 + 0.000394706454219282*s1]
    (-aa + z)*(z**2 - 1.6*z + 0.6656)
    (r1 + z)*(z - 1)*(z - 0.980198673306755)
    z**2 - 1.6*z + 0.6656
    (r1 + z)*(z - 1)*(z - 0.980198673306755)
    Poly(1.0*z**3 + (1.0*r1 + 0.000397346613510541*s0 - 1.98019867330676)*z**2 + (-1.98019867330676*r1 + 0.000394706454219282*s0 + 0.000397346613510541*s1 + 0.980198673306755)*z + 0.980198673306755*r1 + 0.000394706454219282*s1, z, domain='RR[r1,s0,s1]')

```python
sol = sy.solve(dioph, (r1,s0,s1))
```

```python
print (-np.exp(-20*0.04) + 0.38)
print (- 0.98 + 0.6656)
print (1.6*np.exp(-20*0.04) - 0.98 + 0.6656)
print (-0.6656*np.exp(-20*0.04))

```

    -0.0693289641172
    -0.3144
    0.404526342588
    -0.299073358516

```python
sol[r1].subs(aa, np.exp(-20*0.04))
```

    -0.195457575341184

```python
print('r_1 = %f' % sol[r1].subs(aa, np.exp(-20*0.04)))
print('s_0 = %f' % sol[s0].subs(aa, np.exp(-20*0.04)))
print('s_1 = %f' % sol[s1].subs(aa, np.exp(-20*0.04)))

t0 = Ac.evalf(subs={z:1})/Bp.evalf(subs={z:1,})
print('t_0 = %f' % t0)

R = Rp.subs(sol)
S = Sp.subs(sol)
T = t0*Ao

Hc = T*Bp/(Ac*Ao)
Hcc = t0*0.8/Ac
sy.pretty_print(sy.expand(Hc))
sy.pretty_print(sy.expand(Hcc))
sy.pretty_print(Hc.evalf(subs={z:1}))

sy.pretty_print(sy.simplify(Ap*R + Bp*S))
```

    r_1 = -0.195458
    s_0 = 317.927170
    s_1 = -272.319090
    t_0 = 82.822733

                        - -0.0329093326044447⋅aa⋅z
    - ──────────────────────────────────────────────────────────────── - ─────────
            2                              3              2                    2
      - aa⋅z  + 1.6⋅aa⋅z - - -0.6656⋅aa + z  - -1.6⋅(-1)⋅z  + 0.6656⋅z   - aa⋅z  +

              - -0.0326906673955553⋅aa
    ─────────────────────────────────────────────────────── + ────────────────────
                                3              2                    2
     1.6⋅aa⋅z - - -0.6656⋅aa + z  - -1.6⋅(-1)⋅z  + 0.6656⋅z   - aa⋅z  + 1.6⋅aa⋅z -

                         2
     0.0329093326044447⋅z                                                0.0326906
    ──────────────────────────────────────────── + ───────────────────────────────
                     3              2                    2
     - -0.6656⋅aa + z  - -1.6⋅(-1)⋅z  + 0.6656⋅z   - aa⋅z  + 1.6⋅aa⋅z - - -0.6656⋅

    673955553⋅z
    ─────────────────────────────────
          3              2
    aa + z  - -1.6⋅(-1)⋅z  + 0.6656⋅z
         66.2581866520861
    ─────────────────────────
     2
    z  - -1.6⋅(-1)⋅z + 0.6656
    (--82.8227333151076⋅(-1)⋅aa + 82.8227333151076⋅z)⋅(0.000397346613510541⋅z + 0.
    ──────────────────────────────────────────────────────────────────────────────
                                            ⎛ 2                       ⎞
                                  (-aa + z)⋅⎝z  - -1.6⋅(-1)⋅z + 0.6656⎠

    000394706454219282)
    ───────────────────

                 2                                  3              2
    - - -1.0⋅aa⋅z  + 1.6⋅aa⋅z - - -0.6656⋅aa + 1.0⋅z  - -1.6⋅(-1)⋅z  + 0.665600000

    000001⋅z

```python
0 + 2*1j
```

    2j

```python
sy.Poly((Ac*Ao).subs(aa, -0.2), z).all_coeffs()
```

    [1.00000000000000, -1.40000000000000, 0.345600000000000, 0.133120000000000]

```python
def my_bode(num, den, h=1.0):
    n = len(den)-1
    m = len(num)-1
    w = np.linspace(0.01, np.pi, 400);
    z = np.exp(1j*w);
    dd = den[-1]
    nd = num[-1]
    for i in range(n):
        dd += den[i]*np.power(z, n-i)
    for i in range(m):
        nd += num[i]*np.power(z, m-i)
    return nd/dd,w
def bodeplots(a, aa, Ap, R, Ac, Ao):
    numS = sy.list2numpy(sy.Poly((Ap*R).subs(aa, a), z).all_coeffs(),
                        dtype=np.float64)
    numT = sy.list2numpy(sy.Poly((Bp*S).subs(aa, a), z).all_coeffs(),
                        dtype=np.float64)
    den = sy.list2numpy(sy.Poly((Ac*Ao).subs(aa, a), z).all_coeffs(),
                        dtype=np.float64)

    qS, wS = my_bode(numS, den)
    qT, wT = my_bode(numT, den)
    plt.figure()
    plt.loglog(wS, np.abs(qS))
    plt.loglog(wT, np.abs(qT))
    plt.ylim((0.001, 10))
    plt.legend(('Ss', 'Tt'))
    #cm.bode([Ss, Tt])
bodeplots(-0.009, aa, Ap, R, Ac, Ao)

```

```python
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
from functools import partial
```

```python
interact(bodeplots, a=(0.0, 1.0), aa=fixed(aa), Ap=fixed(Ap), R=fixed(R), Ac=fixed(Ac),
        Ao=fixed(Ao))
```

    interactive(children=(FloatSlider(value=0.5, description='a', max=1.0), Output()), _dom_classes=('widget-inter…

    <function __main__.bodeplots(a, aa, Ap, R, Ac, Ao)>

```python
# Reorganize solution expression for matlab code generation
sol_expr = ('RST_DC_lab', [Bp.all_coeffs()[0], Bp.all_coeffs()[1],
                           Ap.all_coeffs()[1], Ap.all_coeffs()[2],
                           sol[r1], sol[s0], sol[s1], A2p.subs(z, 1)/Bp.subs(z,1), h,np.exp(h*po1) ])
```

```python
# Export to matlab code
[(m_name, m_code)] = codegen(sol_expr, 'octave')
```

```python
m_code = m_code.replace("out1", "b0").replace("out2", "b1").replace("out3", "a1").replace("out4", "a2")
m_code = m_code.replace("out5", "r1").replace("out6", "s0").replace("out7", "s1").replace("out8", "t0")
m_code = m_code.replace("out9", "h").replace("out10", "obsPole")
m_code = m_code.replace("function ", "% function ")
m_code = m_code.replace("end", "")
print m_code
with open("/home/kjartan/Dropbox/undervisning/tec/MR2007/labs/dc_rst_design.m", "w") as text_file:
    text_file.write(m_code)
```

```python
cm.step?
```

```python

G = Km * cm.tf([1], [tau, 1, 0])
Gd = Km * cm.tf([tau*(hpt-1+np.exp(-hpt)), tau*(1-(1+hpt)*np.exp(-hpt))], [1, -(1+np.exp(-hpt)), np.exp(-hpt)], h)
Gd2 = cm.c2d(G, h)
print Gd
print Gd2
```

```python
print A2p
print A2p.evalf(subs={z:1})
print Bp
print Bp.evalf(subs={z:1})

```

```python
0.3/(5*np.sqrt(2))
```

```python
np.exp(-0.21)*np.sin(0.21)
```

```python
np.exp(0.03*(-14))
```

```python
 0.746*41.8

```

```python

```


# BANTS theory documentation

### Dynamic Bayesian network structure

A dynamic Bayesian network is denoted ${\rm DBN}(G,p)$ where $G$ denotes the graph structure and $p$ denotes the joint probability distribution over vertices. The latter, for the observation vector of an $N$-dimensional time series process $x_{t}$ (with $i$ indexing an element of the vector $x$, and $t$ denoting a discrete observation timepoint index) is factorisable with respect to the graph structure in the following manner

$$
\begin{equation}
p(x_{t},x_{t-1},\dots ,x_{1}) = \prod_{n=0}^{t-1}p[x_{t-n}\vert {\rm pa}(x_{t-n})]\,,
\end{equation}
$$

where ${\rm pa}(x_{t})$ denotes the parent nodes of $x_{t}$ which are defined by $G$. Choosing a set of parameters $\Theta$ (defined within their prior domain $\Omega_\Theta$), graph $G$, and an observation of $x_t$, Bayes' rule yields the following for the posterior distribution over $\Theta$ at timepoint $t$, given the data $x_{t}, x_{t-1}, \dots $

$$
\begin{equation}
{\cal P}_G(\Theta \vert x_{t}, x_{t-1}, \dots ) = \frac{\prod_{n=0}^{t-1}P_G(x_{t-n},\Theta )}{\prod_{n=0}^{t-1}\int_{\Theta \in \Omega_\Theta} {\rm d}\Theta P_G(x_{t-n},\Theta )} = \frac{\prod_{n=0}^{t-1}{\cal L}_G(x_{t-n} \vert \Theta )\pi_G (\Theta )}{\prod_{n=0}^{t-1}{\cal E}_G(x_{t-n})}\,,
\end{equation}
$$

where we have assumed that each data point $x_t$ is an independent observation from its predecessor $x_{t-1}$ and, as usual, ${\cal L}_G$ is the likelihood, $\pi_G$ is the prior and ${\cal E}_G$ is the evidence.

In [empirical Bayes](https://en.wikipedia.org/wiki/Empirical_Bayes_method), one may define an auxiliary time-dependent vector variable $m_t$ (which $\pi_G$ and ${\cal E}_G$ are typically conditioned on) to marginalise over in this posterior definition like so

$$
\begin{equation}
{\cal P}_G(\Theta \vert x_{t}, x_{t-1}, \dots ) = \prod_{n=0}^{t-1}\int_{m \in \Omega_{m_{t-n}}} {\rm d}m \, P_G(m\vert x_{t-n}, x_{t-n-1}, \dots )\frac{{\cal L}_G(x_{t-n}\vert \Theta )\pi_G (\Theta \vert m)}{{\cal E}_G(x_{t-n}\vert m)} \,.
\end{equation}
$$

The vector variable $m_t$, which will form an intermediate layer in our network, acts as a compression of $x_{t}, x_{t-1}, \dots $ into the mean vector of a Gaussian process at time $t$ which we will importantly assume to be _Markovian_ (note that this assumption will clearly not always hold well for the best compressions). For some references on Gaussian processes, see, e.g., [Friedman & Nachman (2000)](https://arxiv.org/abs/1301.3857), [Titsias & Lawrence (2010)](http://proceedings.mlr.press/v9/titsias10a/titsias10a.pdf), [Damianou & Lawrence (2013)](http://proceedings.mlr.press/v31/damianou13a.pdf) or [Frigola-Alcalde (2015)](http://mlg.eng.cam.ac.uk/pub/pdf/Fri15.pdf).

Up until this point, the technique used for compressing $x_{t}, x_{t-1}, \dots  \rightarrow m_t$ is still undetermined.

### `'AR-GP'`: a simple Gaussian process network with autoregressive kernel convolution compression

A simple nonparametric way to obtain $m_t$ is through autoregressive techniques with convolution kernels. This network models the whole vector process as

$$
\begin{align}
P(x_{t}\vert m_{t}, f_t, {\sf V}) &= {\cal N}( x_{t}; m_{t} + f_t, {\sf V}) \\
P[m_{t}\vert {\rm pa}(x_{t}), h, \Sigma] &= {\cal N}\big\{ m_{t};M_t[{\rm pa}(x_{t}) \vert h] ,\Sigma \big\} \\
M^i_t[{\rm pa}(x^i_{t}) \vert h^i] &= \sum_{n=1}^{t-1}\frac{x^i_{t-n}}{H^i}\exp \left[ -\frac{A^i(n)}{(h^i)^2}\right] \,,
\end{align}
$$

$$
\begin{equation}
H^i = \sum_{n=1}^{t-1}\exp \left[ -\frac{A^i(n)}{(h^i)^2}\right]\,, \quad A^i(n) = \begin{cases} \frac{n^2}{2} \,\, ({\rm and}\,\, f^i_t=0) &  {\rm Squared}\,\,{\rm exponential}\\ 2\sin^2\big( \big\vert \frac{\pi n}{n^i}\big\vert \big) \,\, ({\rm and}\,\, f^i_t=\sin (\frac{\pi t}{n^i} + \pi s^i) ) & {\rm Periodic}\end{cases}\,,
\end{equation}
$$

where ${\cal N}(\mu , \Sigma )$ is a multivariate normal distribution with mean vector $\mu$ and covariance matrix $\Sigma$. The likelihood of data point $x_t$ is therefore very simply

$$
\begin{equation}
{\cal L}_G(x_{t}\vert \Theta ) = {\cal N}\big[ x_{t};\tilde{M}_t(f_t,h),\tilde{\Sigma} \big] \,, \quad \tilde{M}_t(f_t,h) \equiv f_t+M_t[{\rm pa}(x_{t}) \vert h]\,, \quad  \tilde{\Sigma} \equiv {\sf V}+\Sigma\,.
\end{equation}
$$

The graph displayed below illustrates the structure of this network, where shaded nodes are observed at time $t$.

It is clear that investigating the data for evidence of seasonality (by, e.g., examining the autocorrelation functions) with be an important first step before deciding on the convolution kernels connecting the input layer to the hidden layer.

Not all of the graph edges should be strongly weighted by the data so we can (and should) select graph structures based on their combined Bayesian evidence over all of the past observations of the process $Z_t=\prod_{n=0}^{t-1}{\cal E}_G(x_{t-n})$. In order to convert the evaluation of $Z_t$ into an optimisation problem, we can choose an appropriate prior over $\tilde{\Sigma}$ that parameterises the family of posterior distributions. For a multivariate normal with fixed mean (assuming that the priors over $h$ and $f_t$ are independent) and unknown covariance, the conjugate prior is just the inverse-Wishart distribution ${\cal W}^{-1}$ so from the definition of the posterior, we have simply

$$
\begin{equation}
P_G(x_{t}, \Theta ) \propto {\cal N}\big[ x_{t};\tilde{M}_t(f_t,h),\tilde{\Sigma} \big]{\cal W}^{-1}(\tilde{\Sigma};\Psi , \nu) \quad \Longleftrightarrow \quad Z_t=\prod_{n=0}^{t-1}{\cal E}_G(x_{t-n}) \propto \prod_{n=0}^{t-1}{\sf t}_{\nu - N +1}\bigg[ x_{t-n};\tilde{M}_{t-n}(f_{t-n},h),\frac{\Psi}{\nu - N + 1} \bigg] \,,
\end{equation}
$$

where ${\sf t}_{\nu}(\mu , \Sigma)$ is a multivariate t-distribution and the latter expression is obtained by marginalisation over the degrees of freedom in $\tilde{\Sigma}$. It is preferable at this point to define the priors over $h$ and $f_t$ as simply Dirac delta distributions centered on single parameter values (or $n^i$ and $s^i$ in the case of the $f_t^i$ functions) so that all of the epistemic uncertainty is propagated to the hidden-to-output layer weights. Using this prior one may replace the proportionalities above with exact equalities, which correspond to the method that is actually implemented within the `bants` class. Note also that one may choose a non-informative prior over the covariance matrix by setting the degrees of freedom $\nu = N$.

In case a gradient descent algorithm is used to optimise $\ln Z_t$, it first derivatives (and other relevant quantities) are

$$
\begin{align}
\frac{\partial \ln H^i}{\partial (h^i)^2} &= \frac{1}{H^i}\sum_{n=1}^{t-1}\frac{A^i(n)}{(h^i)^4}\exp \left[ -\frac{A^i(n)}{(h^i)^2}\right] \\
\frac{\partial}{\partial (h^i)^2}M^i_t[{\rm pa}(x^i_{t}) \vert h^i] &= \sum_{n=1}^{t-1} \bigg[ \frac{A^i(n)}{(h^i)^4} - \frac{\partial \ln H^i}{\partial (h^i)^2}\bigg] \frac{x^i_{t-n}}{H^i} \exp \left[ -\frac{A^i(n)}{(h^i)^2}\right] \\
&= \sum_{n=1}^{t-1} \frac{A^i(n)}{(h^i)^4} \frac{x^i_{t-n}}{H^i} \exp \left[ -\frac{A^i(n)}{(h^i)^2}\right] - \frac{\partial \ln H^i}{\partial (h^i)^2}M^i_t[{\rm pa}(x^i_{t}) \vert h^i] \\
\ln Z_t &= \ln \Gamma \bigg(\frac{\nu + 1}{2}\bigg) - \ln\Gamma \bigg(\frac{\nu - N + 1}{2}\bigg) - \frac{N}{2}\ln ( \pi ) - \frac{1}{2}\ln {\rm det} ( \Psi ) \\
&- \frac{\nu + 1}{2}\sum_{n=0}^{t-1}\ln \bigg\{ 1+\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big]^{\rm T} \Psi^{-1}\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big] \bigg\} \\
\frac{\partial \ln Z_t}{\partial (h^i)^2} &= - (\nu + 1)\sum_{n=0}^{t-1}\bigg\{ 1+\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big]^{\rm T} \Psi^{-1}\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big] \bigg\}^{-1} \\
&\qquad \qquad \qquad \times \frac{\partial}{\partial (h^i)^2}M^i_{t-n}[{\rm pa}(x^i_{t-n}) \vert h^i] \sum_{j=1}^N\big( \Psi^{-1}\big)^i_j\big[ x^j_{t-n}-\tilde{M}^j_{t-n}(f^j_{t-n},h^j) \big] \\
\frac{\partial \ln Z_t}{\partial \Psi^i_j} &= - \frac{1}{2}\delta^i_j + \frac{\nu + 1}{2}\sum_{n=0}^{t-1} \bigg\{ 1+\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big]^{\rm T} \Psi^{-1}\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},h) \big] \bigg\}^{-1}\\
&\qquad \qquad \qquad \qquad \times \big[ x^i_{t-n}-\tilde{M}^i_{t-n}(f^i_{t-n},h^i) \big] \big( \Psi^{-2}\big)^i_j\big[ x^j_{t-n}-\tilde{M}^j_{t-n}(f^j_{t-n},h^j) \big]\,.
\end{align}
$$

### `'KM-GP'`: a simple Gaussian process network with $k$-means clustering compression

Let us now consider an alternative method to perform the compression $x_t, x_{t-1}, \dots \rightarrow m_t$. In particular, the method we shall choose for the compression in this case is [$k$-means clustering with dynamic time warping](https://tslearn.readthedocs.io/en/stable/user_guide/clustering.html#k-means-and-dynamic-time-warping). The graph below illustrates the structure of this alternative network, where, once again, the shaded nodes are observed at time $t$.

In this network, we will assume a linear transformation for $\tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u)$ such that the vector elements are

$$
\begin{equation}
\tilde{M}^i_{t}(f^i_{t},m_{t},{\sf U},u^i) = \sum_{j=1}^N{\sf U}^i_jm_t^j + u^i + f^i_t \,.
\end{equation}
$$

The empirical Bayes formula we discussed at the beginning of these notes can be used to evaluate the graph posterior, and the overall evidence in this case can be obtained through marginalisation like so

$$
\begin{align}
Z_t &=\prod_{n=0}^{t-1}{\cal E}_G(x_{t-n}) \\
&= \prod_{n=0}^{t-1}\int_{m \in \Omega_{m_{t-n}}} {\rm d}m \, P_G(m\vert x_{t-n}, x_{t-n-1}, \dots ,\Phi ) \, {\cal E}_G(x_{t-n} \vert m) \\
&= \prod_{n=0}^{t-1} \int_{m \in \Omega_{m_{t-n}}} {\rm d}m \, P_G(m\vert x_{t-n}, x_{t-n-1}, \dots ,\Phi )\, {\sf t}_{\nu - N +1}\bigg[ x_{t-n}; \tilde{M}_{t-n}(f_{t-n},m,{\sf U},u),\frac{\Psi}{\nu - N + 1} \bigg] \,,
\end{align}
$$

where $\Phi$ denotes the hyperparameter set used by the $k$-means algorithm for the compression. Note that this algorithm renders $P_G(m\vert x_{t-n}, x_{t-n-1}, \dots ,\Phi )$ to a (deterministic) Dirac delta $\delta (m-m_{t-n})$ for each timepoint $t$, and so we obtain simply

$$
\begin{equation}
Z_t = \prod_{n=0}^{t-1} {\sf t}_{\nu - N +1}\bigg[ x_{t-n}; \tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u),\frac{\Psi}{\nu - N + 1} \bigg] \,.
\end{equation}
$$

For each choice of $\Phi$, a gradient descent algorithm can be used to optimise the remaining parameters in $\ln Z_t$, where the first derivatives which need to be added for this particular network are

$$
\begin{align}
\frac{\partial \ln Z_t}{\partial u^i} &= (\nu + 1)\sum_{n=0}^{t-1}\bigg\{ 1+\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u) \big]^{\rm T} \Psi^{-1}\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u) \big] \bigg\}^{-1} \\
&\qquad \qquad \qquad \times \sum_{j=1}^N\big( \Psi^{-1}\big)^i_j\big[ x^j_{t-n}-\tilde{M}^j_{t-n}(f^j_{t-n},m_{t-n},{\sf U},u^j) \big] \\
\frac{\partial \ln Z_t}{\partial {\sf U}^i_j} &= (\nu + 1)\sum_{n=0}^{t-1}\bigg\{ 1+\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u) \big]^{\rm T} \Psi^{-1}\big[ x_{t-n}-\tilde{M}_{t-n}(f_{t-n},m_{t-n},{\sf U},u) \big] \bigg\}^{-1} \\
&\qquad \qquad \qquad \times m^j_{t-n} \sum_{j'=1}^N\big( \Psi^{-1}\big)^i_{j'}\big[ x^{j'}_{t-n}-\tilde{M}^{j'}_{t-n}(f^{j'}_{t-n},m_{t-n},{\sf U},u^{j'}) \big] \,.
\end{align}
$$

```python

```


# Fundamentos de Lógica

## Lógica Proposicional

É a matemática de $2$ (dois) valores Booleanos, `VERDADEIRO` (`TRUE`, `T`, `1`) e `FALSO` (`FALSE`, `F`, `0`),
e $5$ (cinco) operadores.

### Operadores da Lógica Proposicional

|   | operador   | nome         | programação |
|---|:----------:|:------------:|:-----------:|
| 1 | $\land$    | conjunção    | `AND`, `&&` |
| 2 | $\lor$     | disjunção    | `OR`, `\|\|`  |
| 3 | $\lnot$    | negação      | `NOT`, `!`  |
| 4 | $\implies$ | implicação   |             |
| 5 | $\equiv$   | equivalência |  `==`       |

**Lógica Proposicional: Exemplos**

1. $x \land y$ é `VERDADEIRO` se $x$ e $y$ forem `VERDADEIRO`;
2. $x \lor y$ é `VERDADEIRO` se $x$ ou $y$ forem `VERDADEIRO` (ou ambos);
3. $\lnot x$ é `VERDADEIRO` se $x$ for `FALSO`;
4. $x \implies y$ é `VERDADEIRO` se $x$ for `FALSO` e $y$ for `VERDADEIRO` (ou ambos);
5. $x\equiv y$ é `VERDADEIRO` se $x$ e $y$ forem `VERDADEIRO` ou ambos forem `FALSO`.

**Lógica Proposicional: Tabela-Verdade**

  $x$   |  $y$  |    |    | $\land$ | $\lor$  |
 ------ | ----- | -- | -- | ------- | ------  |
 *`F`*  |  *`F`*  |    |    | `F`     |  `F`    |
 *`F`*  |  *`T`*  |    |    | `F`     |  `T`    |
 *`T`*  |  *`F`*  |    |    | `F`     |  `T`    |
 *`T`*  |  *`T`*  |    |    | `T`     |  `T`    |

|  $x$   |  $y$  |    |    | $\Rightarrow$   | $\equiv$ |
| ------ | ----- | -- | -- | ------------ | -------- |
| *`F`*  |  *`F`*  |    |    |  `T`         |    `T`   |
| *`F`*  |  *`T`*  |    |    |  `T`         |    `F`   |
| *`T`*  |  *`F`*  |    |    |  `F`         |    `F`   |
| *`T`*  |  *`T`*  |    |    |  `T`         |    `T`   |

$x$   | $\lnot x$
:---: | :---:
`F`   |  `T`
`T`   |  `F`

**Python3**

```python
from sympy import *
x = True
y = True
x and y
```

    True

```python
x or y
```

    True

```python
not x
```

    False

```python
Implies(x, y)
```

$\displaystyle \text{True}$

```python
x == y
```

    True

## Lógica de Predicados (lógica de primeira ordem)

A Lógica de Predicados estende a Lógica Proposicional com a adição de operadores
que trabalham com conjuntos de elementos.

**Operadores**

| operador |  operação |
|:--:|:--:|
| $\forall$ | quantificação universal (para todo) |
| $\exists$ | quantificação existencial (existe) |

**Exemplos**

$(\forall\ x \in R : S)$  - para todo $x$ em $R$, a fórmula $S$ é `VERDADEIRA`;

$(\exists \ x \in R : S)$ - existe $x$ em R, em que a fórmula $S$ é `VERDADEIRA`.

$\forall\ x \in \{2, 3, 4, 5\} : x > 1$

$\exists\ x \in \{2, 3, 4, 5\} : x > 2$

$\forall\ x \in N : x\ mod\ 2 = 0$ (é par?) `F`

$\exists\ x \in N : x\ mod\ 2 = 0$ (é par?) `V`

$\exists\ x \in N : \sqrt{x} = 2$ (é par?) `V`

## Exercícios

1. Resolva as expressões a seguir para $(x,y)=(T,T)$, $(x,y)=(T,F)$, $(x,y)=(F,T)$ e $(x,y)=(F,F)$:

- As expressões são sempre avaliadas da esquerda para a direita com exceção da existência de parênteses, por exemplo para $x \lor (x \land y)$, primeiro avalia-se $(x \land y)$ e depois a disjunção $\lor$;
- $(x,y)=(T,F)$ significa que $x=T$ e $y=F$;
- Se preferir, monte uma tabela-verdade para cada expressão.

a. $x\land y \lor x$

b. $\neg (x\lor y \land x)$

c. $x \land y \implies \neg x$

d. $(x \implies y) \equiv (x \lor y)$

e. $(x \land y) \equiv (x \lor y)$

f. $(\neg x \land y) \equiv (x \lor \neg y)$

g. $\neg(x \implies y) \equiv (x \land y)$

h. $x \lor (x \lor y) \land \neg(x \land \neg y)$

2. Para o conjunto dos números inteiros $Z$ ($\ldots, -3, -2, -1, 0, 1, 2, 3, \ldots$). Avalie as
 seguintes expressões:

a. $\forall x \in Z : x \text{ é par} \land\ x \text{ é impar}$

b. $\exists x \in Z : x \text{ é par} \land\ x  \text{ é impar}$

c. $\forall x \in Z : x \text{ é par} \lor\ x \text{ é impar}$

d. $\exists x \in Z : x \text{ é par} \lor\ x \text{ é impar}$

e. $\forall x \in Z : x \text{ é primo}$

f. $\exists x \in Z : x \text{ é primo}$

g. $\neg(\forall x \in Z : x \text{ é primo} )$

h. $\forall x \in \{2,4,6,8\}, \forall y \in \{1,3,5,7\} : x + y = 11$

i. $\exists x \in \{2,4,6,8\}, \forall y \in \{1,3,5,7\} : x + y = 11$

j. $\exists x \in \{2,4,6,8\}, \exists y \in \{1,3,5,7\} : x + y = 11$

l. $\forall x \in \{0\}, \forall y \in \{1,3,5,7\} : xy = 0$

3. [Scheinerman] Assinale como verdadeira (T) ou false (F) cada uma das expressões sobre inteiros a seguir:

a. $\forall x, \forall y : x + y = 0$

b. $\forall x, \exists y : x + y = 0$

c. $\exists x, \forall y : x + y = 0$

d. $\exists x, \exists y : x + y = 0$

e. $\forall x, \forall y : xy = 0$

f. $\forall x, \exists y : xy = 0$

g. $\exists x, \forall y : xy = 0$

h. $\exists x, \exists y : xy = 0$


# CNTK 207: Sampled Softmax

For classification and prediction problems a typical criterion function is cross-entropy with softmax. If the number of output classes is high the computation of this criterion and the corresponding gradients could be quite costly. Sampled Softmax is a heuristic to speed up training in these cases. (see: [Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model](http://www.iro.umontreal.ca/~lisa/pointeurs/importance_samplingIEEEtnn.pdf), [Exploring the Limits of Language Modeling](https://arxiv.org/pdf/1602.02410v1.pdf), [What is Candidate Sampling](https://www.tensorflow.org/extras/candidate_sampling.pdf))

**Select the notebook runtime environment devices / settings**

Before we dive into the details we run some setup that is required for automated testing of this notebook.

```python
from __future__ import print_function # Use a function definition from future version (say 3.x from 2.7 interpreter)
from __future__ import division

import os
import cntk as C
import cntk.tests.test_utils
cntk.tests.test_utils.set_device_from_pytest_env() # (only needed for our build system)
C.cntk_py.set_fixed_random_seed(1) # fix a random seed for CNTK components
```

## Basic concept

The softmax function is used in neural networks if we want to interpret the network output as a probability distribution over a set of classes $C$ with $|C|=N_C$.

Softmax maps an $N_C$-dimensional vector $z$, which has unrestricted values, to an $N_C$ dimensional vector $p$ with non-negative values that sum up to 1 so that they can be interpreted as probabilities. More precisely:

$$
\begin{align}
p_i &= softmax(z, i)\\
    &= \frac{exp(z_i)}{\sum_{k\in C} exp(z_k)}\\
\end{align}
$$

In what follows we assume that the input $z$ to the softmax is computed from some hidden vector $h$ of dimension $N_h$  in a specific way, namely:

$$ z = W h + b $$

where $W$ is a learnable weight matrix of dimension $(N_c, N_h)$ and $b$ is a learnable bias vector.
We restrict ourselves to this specific choice of $z$ because it helps in implementing an efficient sampled softmax.

In a typical use-case like for example a recurrent language model, the hidden vector $h$ would be the output of the recurrent layers and $C$ would be the set of words to predict.

As a training criterion, we use cross-entropy which is a function of the expected (true) class $t\in C$ and the probability predicted for it:

$$cross\_entropy := -log(p_t)$$

## Sampled Softmax

For the normal softmax the CNTK Python-api provides the function [cross_entropy_with_softmax](https://cntk.ai/pythondocs/cntk.ops.html?highlight=softmax#cntk.ops.cross_entropy_with_softmax). This takes as input the $N_C$-dimensional vector $z$. As mentioned for our sampled softmax implementation we assume that this z is computed by $ z = W h + b $. In sampled softmax this has to be part of the whole implementation of the criterion.

Below we show the code for `cross_entropy_with_sampled_softmax_and_embedding`. Let’s look at the signature first.

One fundamental difference to the corresponding function in the Python-api (`cross_entropy_with_softmax`) is that in the Python api function the input corresponds to $z$ and must have the same dimension as the target vector, while in cross_entropy_with_full_softmax the input corresponds to our hidden vector $h$ can have any dimension (hidden_dim).
Actually, hidden_dim will be typically much lower than the dimension of the target vector.

We also have some additional parameters `num_samples, sampling_weights, allow_duplicates` that control the random sampling.
Another difference to the api function is that we return a triple (z, cross_entropy_on_samples, error_on_samples).

We will come back to the details of the implementation below.

```python
# Creates a subgraph computing cross-entropy with sampled softmax.
def cross_entropy_with_sampled_softmax_and_embedding(
    hidden_vector,            # Node providing hidden input
    target_vector,            # Node providing the expected labels (as sparse vectors)
    num_classes,              # Number of classes
    hidden_dim,               # Dimension of the hidden vector
    num_samples,              # Number of samples to use for sampled softmax
    sampling_weights,         # Node providing weights to be used for the weighted sampling
    allow_duplicates = True,  # Boolean flag to control whether to use sampling with replacemement
                              # (allow_duplicates == True) or without replacement.
    ):
    # define the parameters learnable parameters
    b = C.Parameter(shape = (num_classes, 1), init = 0)
    W = C.Parameter(shape = (num_classes, hidden_dim), init = C.glorot_uniform())

    # Define the node that generates a set of random samples per minibatch
    # Sparse matrix (num_samples * num_classes)
    sample_selector = C.random_sample(sampling_weights, num_samples, allow_duplicates)

    # For each of the samples we also need the probablity that it in the sampled set.
    inclusion_probs = C.random_sample_inclusion_frequency(sampling_weights, num_samples, allow_duplicates) # dense row [1 * vocab_size]
    log_prior = C.log(inclusion_probs) # dense row [1 * num_classes]

    # Create a submatrix wS of 'weights
    W_sampled = C.times(sample_selector, W) # [num_samples * hidden_dim]
    z_sampled = C.times_transpose(W_sampled, hidden_vector) + C.times(sample_selector, b) - C.times_transpose (sample_selector, log_prior)# [num_samples]

    # Getting the weight vector for the true label. Dimension hidden_dim
    W_target = C.times(target_vector, W) # [1 * hidden_dim]
    z_target = C.times_transpose(W_target, hidden_vector) + C.times(target_vector, b) - C.times_transpose(target_vector, log_prior) # [1]

    z_reduced = C.reduce_log_sum_exp(z_sampled)

    # Compute the cross entropy that is used for training.
    # We don't check whether any of the classes in the random samples conincides with the true label, so it might
    # happen that the true class is counted
    # twice in the normalising demnominator of sampled softmax.
    cross_entropy_on_samples = C.log_add_exp(z_target, z_reduced) - z_target

    # For applying the model we also output a node providing the input for the full softmax
    z = C.times_transpose(W, hidden_vector) + b
    z = C.reshape(z, shape = (num_classes))

    zSMax = C.reduce_max(z_sampled)
    error_on_samples = C.less(z_target, zSMax)
    return (z, cross_entropy_on_samples, error_on_samples)
```

To give a better idea of what the inputs and outputs are and how this all differs from the normal softmax we give below a corresponding function using normal softmax:

```python
# Creates subgraph computing cross-entropy with (full) softmax.
def cross_entropy_with_softmax_and_embedding(
    hidden_vector,  # Node providing hidden input
    target_vector,  # Node providing the expected labels (as sparse vectors)
    num_classes,    # Number of classes
    hidden_dim      # Dimension of the hidden vector
    ):
    # Setup bias and weights
    b = C.Parameter(shape = (num_classes, 1), init = 0)
    W = C.Parameter(shape = (num_classes, hidden_dim), init = C.glorot_uniform())

    z = C.reshape( C.times_transpose(W, hidden_vector) + b, (1, num_classes))

    # Use cross_entropy_with_softmax
    cross_entropy = C.cross_entropy_with_softmax(z, target_vector)

    zMax = C.reduce_max(z)
    zT = C.times_transpose(z, target_vector)
    error_on_samples = C.less(zT, zMax)

    return (z, cross_entropy, error_on_samples)
```

As you can see the main differences to the api function `cross_entropy_with_softmax` are:
* We include the mapping $ z = W h + b $ into the function.
* We return a triple (z, cross_entropy, error_on_samples) instead of just returning the cross entropy.

## A toy example

To explain how to integrate sampled softmax let us look at a toy example. In this toy example we first transform one-hot input vectors via some random projection into a lower dimensional vector $h$. The modeling task is to reverse this mapping using (sampled) softmax. Well, as already said this is a toy example.

```python
import numpy as np
from math import log, exp, sqrt
from cntk.logging import ProgressPrinter
import timeit

# A class with all parameters
class Param:
    # Learning parameters
    learning_rate = 0.03
    minibatch_size = 100
    num_minbatches = 100
    test_set_size = 1000
    momentum = 0.8187307530779818
    reporting_interval = 10
    allow_duplicates = False

    # Parameters for sampled softmax
    use_sampled_softmax = True
    use_sparse = True
    softmax_sample_size = 10

    # Details of data and model
    num_classes = 50
    hidden_dim = 10

data_sampling_distribution = lambda: np.repeat(1.0 / Param.num_classes, Param.num_classes)

softmax_sampling_weights = lambda: np.repeat(1.0 / Param.num_classes, Param.num_classes)

# Creates random one-hot vectors of dimension 'num_classes'.
# Returns a tuple with a list of one-hot vectors, and list with the indices they encode.
def get_random_one_hot_data(num_vectors):
    indices = np.random.choice(
        range(Param.num_classes),
        size=num_vectors,
        p = data_sampling_distribution()).reshape((num_vectors, 1))
    list_of_vectors = C.Value.one_hot(indices, Param.num_classes)
    return (list_of_vectors, indices.flatten())

# Create a network that:
# * Transforms the input one hot-vectors with a constant random embedding
# * Applies a linear decoding with parameters we want to learn
def create_model(labels):
    # random projection matrix
    random_data = np.random.normal(scale = sqrt(1.0/Param.hidden_dim), size=(Param.num_classes, Param.hidden_dim)).astype(np.float32)
    random_matrix = C.constant(shape = (Param.num_classes, Param.hidden_dim), value = random_data)

    h = C.times(labels, random_matrix)

    # Connect the latent output to (sampled/full) softmax.
    if Param.use_sampled_softmax:
        sampling_weights = np.asarray(softmax_sampling_weights(), dtype=np.float32)
        sampling_weights.reshape((1, Param.num_classes))
        softmax_input, ce, errs = cross_entropy_with_sampled_softmax_and_embedding(
            h,
            labels,
            Param.num_classes,
            Param.hidden_dim,
            Param.softmax_sample_size,
            softmax_sampling_weights(),
            Param.allow_duplicates)
    else:
        softmax_input, ce, errs = cross_entropy_with_softmax_and_embedding(
            h,
            labels,
            Param.num_classes,
            Param.hidden_dim)

    return softmax_input, ce, errs

def train(do_print_progress):
    labels = C.input_variable(shape = Param.num_classes, is_sparse = Param.use_sparse)
    z, cross_entropy, errs = create_model(labels)

    # Setup the trainer
    learning_rate_schedule = C.learning_parameter_schedule_per_sample(Param.learning_rate)
    momentum_schedule = C.momentum_schedule(Param.momentum, minibatch_size = Param.minibatch_size)
    learner = C.momentum_sgd(z.parameters, learning_rate_schedule, momentum_schedule, True)
    progress_writers = None
    if do_print_progress:
        progress_writers = [ProgressPrinter(freq=Param.reporting_interval, tag='Training')]
    trainer = C.Trainer(z, (cross_entropy, errs), learner, progress_writers)

    minbatch = 0
    average_cross_entropy = compute_average_cross_entropy(z)
    minbatch_data = [0] # store minibatch values
    cross_entropy_data = [average_cross_entropy] # store cross_entropy values

    # Run training
    t_total= 0

    # Run training
    for minbatch in range(1,Param.num_minbatches):
        # Specify the mapping of input variables in the model to actual minibatch data to be trained with
        label_data, indices = get_random_one_hot_data(Param.minibatch_size)
        arguments = ({labels : label_data})

        # If do_print_progress is True, this will automatically print the progress using ProgressPrinter
        # The printed loss numbers are computed using the sampled softmax criterion
        t_start = timeit.default_timer()
        trainer.train_minibatch(arguments)
        t_end = timeit.default_timer()

        t_delta = t_end - t_start
        samples_per_second = Param.minibatch_size / t_delta

        # We ignore the time measurements of the first two minibatches
        if minbatch > 2:
            t_total += t_delta

        # For comparison also print result using the full criterion
        if minbatch % Param.reporting_interval == int(Param.reporting_interval/2):
            # memorize the progress data for plotting
            average_cross_entropy = compute_average_cross_entropy(z)
            minbatch_data.append(minbatch)
            cross_entropy_data.append(average_cross_entropy)

            if do_print_progress:
                print("\nMinbatch=%d Cross-entropy from full softmax = %.3f perplexity = %.3f samples/s = %.1f"
                    % (minbatch, average_cross_entropy, exp(average_cross_entropy), samples_per_second))

    # Number of samples we measured. First two minbatches were ignored
    samples_measured = Param.minibatch_size * (Param.num_minbatches - 2)
    overall_samples_per_second = samples_measured / t_total
    return (minbatch_data, cross_entropy_data, overall_samples_per_second)

def compute_average_cross_entropy(softmax_input):
    vectors, indices = get_random_one_hot_data(Param.test_set_size)
    total_cross_entropy = 0.0
    arguments = (vectors)
    z = softmax_input.eval(arguments).reshape(Param.test_set_size, Param.num_classes)

    for i in range(len(indices)):
        log_p = log_softmax(z[i], indices[i])
        total_cross_entropy -= log_p

    return total_cross_entropy / len(indices)

# Computes log(softmax(z,index)) for a one-dimensional numpy array z in an numerically stable way.
def log_softmax(z,    # numpy array
                index # index into the array
            ):
    max_z = np.max(z)
    return z[index] - max_z - log(np.sum(np.exp(z - max_z)))

np.random.seed(1)

print("start...")
train(do_print_progress = True)
print("done.")
```

    start...
    Learning rate per 1 samples: 0.03
    Momentum per 100 samples: 0.8187307530779818

    Minbatch=5 Cross-entropy from full softmax = 3.844 perplexity = 46.705 samples/s = 9723.4
     Minibatch[   1-  10]: loss = 2.332016 * 1000, metric = 78.10% * 1000;

    Minbatch=15 Cross-entropy from full softmax = 3.467 perplexity = 32.042 samples/s = 9497.0
     Minibatch[  11-  20]: loss = 2.038090 * 1000, metric = 58.90% * 1000;

    Minbatch=25 Cross-entropy from full softmax = 3.067 perplexity = 21.484 samples/s = 8569.1
     Minibatch[  21-  30]: loss = 1.704383 * 1000, metric = 34.30% * 1000;

    Minbatch=35 Cross-entropy from full softmax = 2.777 perplexity = 16.074 samples/s = 10858.4
     Minibatch[  31-  40]: loss = 1.471806 * 1000, metric = 28.40% * 1000;

    Minbatch=45 Cross-entropy from full softmax = 2.459 perplexity = 11.688 samples/s = 10747.2
     Minibatch[  41-  50]: loss = 1.231299 * 1000, metric = 10.50% * 1000;

    Minbatch=55 Cross-entropy from full softmax = 2.269 perplexity = 9.671 samples/s = 10193.6
     Minibatch[  51-  60]: loss = 1.045713 * 1000, metric = 6.70% * 1000;

    Minbatch=65 Cross-entropy from full softmax = 2.104 perplexity = 8.203 samples/s = 10609.4
     Minibatch[  61-  70]: loss = 0.974900 * 1000, metric = 7.00% * 1000;

    Minbatch=75 Cross-entropy from full softmax = 1.937 perplexity = 6.941 samples/s = 10206.4
     Minibatch[  71-  80]: loss = 0.901665 * 1000, metric = 5.10% * 1000;

    Minbatch=85 Cross-entropy from full softmax = 1.798 perplexity = 6.036 samples/s = 10735.0
     Minibatch[  81-  90]: loss = 0.781453 * 1000, metric = 1.60% * 1000;

    Minbatch=95 Cross-entropy from full softmax = 1.707 perplexity = 5.514 samples/s = 11769.0
    done.

In the above code we use two different methods to report training progress:
1. Using a function that computes the average cross entropy on full softmax.
2. Using the built-in ProgressPrinter

ProgressPrinter reports how the value of the training criterion changes over time.
In our case the training criterion is cross-entropy from **sampled** softmax.
The same is true for the error rate computed by progress printer, this is computed only for true-class vs sampled-classes and will therefore underestimate the true error rate.

Therefore while ProgressPrinter already gives us some idea how training goes on, if we want to compare the behavior for different sampling strategies (sample size, sampling weights, ...) we should not rely on numbers that are computed only using the sampled subset of classes.

## Importance sampling

Often the we don't have uniform distribution for the classes on the output side. The typical example is when we have words as output classes. A typical example are words where e.g. 'the' will be much more frequent than most others.

In such cases one often uses a non uniform distribution for drawing the samples in sampled softmax but instead increases the sampling weight for the frequent classes. This is also called importane sampling.
In our example the sampling distribution is controlled by the weight array `softmax_sampling_weights`.

As an example let's look at the case where the classes are distrubted according to zipf-distrubtion like:
$$
p[i] \propto \frac{1}{i+5},
$$
actually we use this distribution already in our example.

How does training behavior change if we switch uniform sampling to sampling with the zipfian distribution in sampled softmax?

```python
# We want to lot the data
import matplotlib.pyplot as plt
%matplotlib inline

# Define weights of zipfian distributuion
def zipf(index):
    return 1.0 / (index + 5)

# Use zipifian distribution for the classes
def zipf_sampling_weights():
    return np.asarray([ zipf(i) for i in range(Param.num_classes)], dtype=np.float32)

data_sampling_distribution = lambda: zipf_sampling_weights() / np.sum(zipf_sampling_weights())

print("start...")

# Train using uniform sampling (like before)
np.random.seed(1)
softmax_sampling_weights = lambda: np.repeat(1.0/Param.num_classes, Param.num_classes)
minibatch_data, cross_entropy_data, _ = train(do_print_progress = False)

# Train using importance sampling
np.random.seed(1)
softmax_sampling_weights = zipf_sampling_weights
minibatch_data2, cross_entropy_data2, _ = train(do_print_progress = False)

plt.plot(minibatch_data, cross_entropy_data, 'r--',minibatch_data, cross_entropy_data2, 'b--')
plt.xlabel('number of mini-batches')
plt.ylabel('cross entropy')
plt.show()
```

In the example above we compare uniform sampling (red) vs sampling with the same distribution the classes have (blue).
You will need to experiment to find the best settings for all the softmax parameters.

## What speedups to expect?

The speed difference between full softmax and sampled softmax in terms of training instances depends strongly on the concrete settings, namely
* Number of classes. Typically the speed-up will increase the more output classes you have.
* Number of samples used in sampled softmax
* Dimension of hiddlen layer input
* Minibatch size
* Hardware

Also you need to test how much you can reduce sample size without degradation of the result.

```python
print("start...")

# Reset parameters
class Param:
    # Learning parameters
    learning_rate = 0.03
    minibatch_size = 8
    num_minbatches = 100
    test_set_size = 1 # we are only interrested in speed
    momentum = 0.8187307530779818
    reporting_interval = 1000000 # Switch off reporting to speed up
    allow_duplicates = False

    # Parameters for sampled softmax
    use_sampled_softmax = True
    use_sparse = True
    softmax_sample_size = 10

    # Details of data and model
    num_classes = 50000
    hidden_dim = 10

data_sampling_distribution = lambda: np.repeat(1.0 / Param.num_classes, Param.num_classes)
softmax_sampling_weights = lambda: np.repeat(1.0 / Param.num_classes, Param.num_classes)

sample_sizes = [5, 10, 100, 1000]
speed_with_sampled_softmax = []

# Get the speed with sampled softmax for different sizes
for sample_size in sample_sizes:
    print("Measuring speed of sampled softmax for sample size %d ..." % (sample_size))
    Param.use_sampled_softmax = True
    Param.softmax_sample_size = sample_size
    _, _,  samples_per_second = train(do_print_progress = False)
    speed_with_sampled_softmax.append(samples_per_second)

# Get the speed with full softmax
Param.use_sampled_softmax = False
print("Measuring speed of full softmax ...")
_, _,  samples_per_second = train(do_print_progress = False)
speed_without_sampled_softmax = np.repeat(samples_per_second, len(sample_sizes))

# Plot the speed of sampled softmax (blue) as a function of sample sizes
# and compare it to the speed with full softmax (red).
plt.plot(sample_sizes, speed_without_sampled_softmax, 'r--',sample_sizes, speed_with_sampled_softmax, 'b--')
plt.xlabel('softmax sample size')
plt.ylabel('speed: instances / second')
plt.title("Speed 'sampled softmax' (blue) vs. 'full softmax' (red)")
plt.ylim(ymin=0)
plt.show()
```

```python

```