```python
import numpy as np
import matplotlib.pyplot as plt

from collections import defaultdict
from scipy.optimize import minimize

import networkx as nx
from networkx.generators.random_graphs import erdos_renyi_graph

from IPython.display import Image
```

```python
from qiskit import QuantumCircuit, execute, Aer
from qiskit.tools.visualization import circuit_drawer, plot_histogram
```

```python
from quantuminspire.credentials import get_authentication
from quantuminspire.api import QuantumInspireAPI
from quantuminspire.qiskit import QI

QI_URL = 'https://api.quantum-inspire.com/'
```

In this notebook you will apply what you have just learned about cqasm and Quantum Inspire. We will consider a simple quantum algorithm, the quantum approximate optimization algorithm (QAOA), for which you will code the circuit in cqasm and send some jobs to real quantum hardware on the Quantum Inspire platform.

## 1. Recap: QAOA and MAXCUT

### Introduction to the Quantum Approximate Optimization Algorithm

$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[2]{\left\langle{#1}\middle|{#2}\right\rangle}$$

Consider some combinatorial optimization problem with objective function $C:x\rightarrow \mathbb{R}$ acting on $n$-bit strings $x\in \{0,1\}^n$, domain $\mathcal{D} \subseteq \{0,1\}^n$, and objective

\begin{align}
     \max_{x \in \mathcal{D}} C(x).
\end{align}

In maximization, an approximate optimization algorithm aims to find a string $x'$ that achieves a desired approximation ratio $\alpha$, i.e.

\begin{equation}
    \frac{C(x')}{C^*}\geq \alpha,
\end{equation}

where $C^* = \max_{x \in \mathcal{D}} C(x)$.
In QAOA, such combinatorial optimization problems are encoded into a cost Hamiltonian $H_C$, a mixing Hamiltonian $H_M$ and some initial quantum state $\ket{\psi_0}$. The cost Hamiltonian is diagonal in the computational basis by design, and represents $C$ if its eigenvalues satisfy

\begin{align}
    H_C \ket{x} = C(x) \ket{x} \text{ for all } x \in  \{0,1\}^n.
\end{align}

The mixing Hamiltonian $H_M$ depends on $\mathcal{D}$ and its structure, and is in the unconstrained case (i.e. when $\mathcal{D}=\{0,1\}^n$) usually taken to be the transverse field Hamiltonian $H_M = \sum_{j} X_j$. Constraints (i.e. when $\mathcal{D}\subset \{0,1\}^n$) can be incorporated directly into the mixing Hamiltonian or are added as a penalty function in the cost Hamiltonian. The initial quantum state $\ket{\psi_0}$ is usually taken as the uniform superposition over all possible states in the domain. $\text{QAOA}_p$, parametrized in $\gamma=(\gamma_0,\gamma_1,\dots,\gamma_{p-1}),\beta=(\beta_0,\beta_1,\dots,\beta_{p-1})$, refers to a level-$p$ QAOA circuit that applies $p$ steps of alternating time evolutions of the cost and mixing Hamiltonians on the initial state. At step $k$, the unitaries of the time evolutions are given by

\begin{align}
    U_C(\gamma_k) = e^{-i \gamma_k H_C }, \label{eq:UC} \\
    U_M(\beta_k) = e^{-i \beta_k H_M }. \label{eq:UM}
\end{align}

So the final state $\ket{\gamma,\beta}$ of $\text{QAOA}_p$ is given by

\begin{align}
    \ket{\gamma,\beta} = \prod_{k=0}^{p-1} U_M(\beta_k) U_C(\gamma_k) \ket{\psi_0}.
\end{align}

The expectation value $ F_p(\gamma,\beta)$ of the cost Hamiltonian for state $\ket{\gamma,\beta}$ is given by

\begin{align}
    F_p(\gamma,\beta) =
    \bra{\gamma,\beta}H_C\ket{\gamma,\beta},
    \label{eq:Fp}
\end{align}

and can be statistically estimated by taking samples of $\ket{\gamma,\beta}$. The achieved approximation ratio (in expectation) of $\text{QAOA}_p$ is then

\begin{equation}
    \alpha = \frac{F_p(\gamma,\beta)}{C^*}.
\end{equation}

The parameter combinations of $\gamma,\beta$ are usually found through a classical optimization procedure that uses $F_p(\gamma,\beta)$ as a black-box function to be maximized.

### Example application: MAXCUT

MaxCut is an NP-hard optimisation problem that looks for an optimal 'cut' for a graph $G(V,E)$, in the sense that the cut generates a subset of nodes $S \subset V$ that shares the largest amount of edges with its complement $ V\setminus S$. In slightly modified form (omitting the constant), it has the following objective function

\begin{align}
\max_{s} \frac{1}{2} \sum_{
\langle i,j \rangle \in E} 1-s_i s_j,
\end{align}

where the $s_i\in\{-1,1\}$ are the variables and $i,j$ are the edge indices. This function can be easily converted into an Ising cost Hamiltonian, which takes the form

\begin{align}
H_C = \frac{1}{2}\sum_{\langle i,j\rangle \in E} I-Z_i Z_j.
\end{align}

We use the standard mixing Hamiltonian that sums over all nodes:

\begin{align}
H_M = \sum_{v \in V} X_v.
\end{align}

As the initial state $\ket{\Psi_0}$ we take the uniform superposition, given by

\begin{align}
\ket{\psi_0} = \frac{1}{\sqrt{2^{|V|}}}\sum_{x=0}^{2^{|V|}-1} \ket{x}
\end{align}

The goal of this workshop is to guide you through an implemented code that simulates a small quantum computer running the QAOA algorithm applied to the MAXCUT problem. We will use qiskit as well as cqasm as SDK's. For the sake of run time, you will always run the classical optimization part using the qiskit simulator: it would take too long for our purposes to do the actual function evualtions in the classical optimization step on the hardware.

## 2. Some useful functions and intializations

We first define some useful functions to be used later throughout the code.

```python
# Just some function to draw graphs
def draw_cc_graph(G,node_color='b',fig_size=4):
    plt.figure(figsize=(fig_size,fig_size))
    nx.draw(G, G.pos,
        node_color= node_color,
        with_labels=True,
        node_size=1000,font_size=14)
    plt.show()
```

```python
# Define the objective function
def maxcut_obj(x,G):
    cut = 0
    for i, j in G.edges():
        if x[i] != x[j]:
            # the edge is cut, negative value in agreement with the optimizer (which is a minimizer)
            cut -= 1
    return cut

# Brute force method
def brute_force(G):
    n = len(G.nodes)
    costs = np.zeros(0)
    costs=[]
    for i in range(2**n):
        calc_costs = -1*maxcut_obj(bin(i)[2:].zfill(n),G)
        costs.append(calc_costs)
    max_costs_bf = max(costs)
    index_max = costs.index(max(costs))
    max_sol_bf = bin(index_max)[2:].zfill(n)
    return max_costs_bf, max_sol_bf,costs

```

```python
# Generating the distribution resulting from random guessing the solution
def random_guessing_dist(G):
    dictio= dict()
    n = len(G.nodes())
    for i in range(2**n):
        key = bin(i)[2:].zfill(n)
        dictio[key] = maxcut_obj(bin(i)[2:].zfill(n),G)
    RG_energies_dist = defaultdict(int)
    for x in dictio:
        RG_energies_dist[maxcut_obj(x,G)] += 1
    return RG_energies_dist

# Visualize multiple distributions
def plot_E_distributions(E_dists,p,labels):
    plt.figure()
    x_min = 1000
    x_max = - 1000
    width = 0.25/len(E_dists)
    for index,E_dist in enumerate(E_dists):
        pos = width*index-width*len(E_dists)/4
        label = labels[index]
        X_list,Y_list = zip(*E_dist.items())
        X = -np.asarray(X_list)
        Y = np.asarray(Y_list)
        plt.bar(X + pos, Y/np.sum(Y), color = 'C'+str(index), width = width,label= label+', $p=$'+str(p))
        if np.min(X)<x_min:
            x_min = np.min(X)
        if np.max(X)>x_max:
            x_max = np.max(X)
    plt.xticks(np.arange(x_min,x_max+1))
    plt.legend()
    plt.xlabel('Objective function value')
    plt.ylabel('Probability')
    plt.show()

# Determinet the expected objective function value from the random guessing distribution
def energy_random_guessing(RG_energies_dist):
    energy_random_guessing = 0
    total_count = 0
    for energy in RG_energies_dist.keys():
        count = RG_energies_dist[energy]
        energy_random_guessing += energy*count
        total_count += count
    energy_random_guessing = energy_random_guessing/total_count
    return energy_random_guessing
```

### Test instances

```python
w2  = np.matrix([
 [0, 1],
 [1, 0]])
G2 = nx.from_numpy_matrix(w2)
positions = nx.circular_layout(G2)
G2.pos=positions
print('G2:')
draw_cc_graph(G2)

w3  = np.matrix([
 [0, 1, 1],
 [1, 0, 1],
 [1, 1, 0]])
G3 = nx.from_numpy_matrix(w3)
positions = nx.circular_layout(G3)
G3.pos=positions
print('G3:')
draw_cc_graph(G3)
```

## 3. Circuit generators

We provide you with an example written in qiskit. You have to write the one for cqasm yourself.

### Qiskit generators

```python
class Qiskit(object):
    # Cost operator:
    def get_cost_operator_circuit(G, gamma):
        N = G.number_of_nodes()
        qc = QuantumCircuit(N,N)
        for i, j in G.edges():
            qc.cx(i,j)
            qc.rz(2*gamma, j)
            qc.cx(i,j)
        return qc

    # Mixing operator
    def get_mixer_operator_circuit(G, beta):
        N = G.number_of_nodes()
        qc = QuantumCircuit(N,N)
        for n in G.nodes():
            qc.rx(2*beta, n)
        return qc

    # Build the circuit:
    def get_qaoa_circuit(G, beta, gamma):
        assert(len(beta) == len(gamma))
        p = len(beta) # number of unitary operations
        N = G.number_of_nodes()
        qc = QuantumCircuit(N,N)
        # first step: apply Hadamards to obtain uniform superposition
        qc.h(range(N))
        # second step: apply p alternating operators
        for i in range(p):
            qc.compose(Qiskit.get_cost_operator_circuit(G,gamma[i]),inplace=True)
            qc.compose(Qiskit.get_mixer_operator_circuit(G,beta[i]),inplace=True)
        # final step: measure the result
        qc.barrier(range(N))
        qc.measure(range(N), range(N))
        return qc

```

```python
# Show the circuit for the G3 (triangle) graph
p = 1
beta = np.random.rand(p)*2*np.pi
gamma = np.random.rand(p)*2*np.pi
qc = Qiskit.get_qaoa_circuit(G3,beta, gamma)
qc.draw(output='mpl')

```

### cqasm generators

Now it is up to you to apply what we have learned about cqasm to write the script for the cost and mixing operators:

```python
class Cqasm(object):

    ### We give them this part
    def get_qasm_header(N_qubits):
        """
        Create cQASM header for `N_qubits` qubits and prepare all in |0>-state.
        """
        header = f"""
version 1.0
qubits {N_qubits}
prep_z q[0:{N_qubits-1}]
"""
        return header

    def get_cost_operator(graph, gamma, p=1):
        """
        Create cost operator for given angle `gamma`.
        """
        layer_list = graph.number_of_edges()*[None]
        for n, (i,j) in enumerate(graph.edges()):
            layer_list[n] = '\n'.join([f"CNOT q[{i}], q[{j}]",
                                       f"Rz q[{j}], {2*gamma}",
                                       f"CNOT q[{i}], q[{j}]"])

        return f".U_gamma_{p}\n" + '\n'.join(layer_list) + '\n'

    def get_mixing_operator(graph, beta, p=1):
        """
        Create mixing operator for given angle `beta`.
        Use parallel application of single qubit gates.
        """
        U_beta = "{" + ' | '.join([f"Rx q[{i}], {2*beta}" for i in graph.nodes()]) + "}"
        return f".U_beta_{p}\n" + U_beta + '\n'

    def get_qaoa_circuit(graph, beta, gamma):
        """
        Create full QAOA circuit for given `graph` and angles `beta` and `gamma`.
        """
        assert len(beta) == len(gamma)
        p = len(beta) # number of layers
        N_qubits = graph.number_of_nodes()
        circuit_str = Cqasm.get_qasm_header(5) #N_qubits)

        # first step: apply Hadamards to obtain uniform superposition
        circuit_str += "{" + ' | '.join([f"H q[{i}]" for i in graph.nodes()]) + "}\n\n"
        # second step: apply p alternating operators
        circuit_str += '\n'.join([Cqasm.get_cost_operator(graph, gamma[i], i+1)
                                  + Cqasm.get_mixing_operator(graph, beta[i], i+1) for i in range(p)])
        # final step: measure the result
        circuit_str += "\n"
        circuit_str += "measure_all"

        return circuit_str
```

## 4. Hybrid-quantum classical optimization

Since QAOA is usually adopted as a hybrid quantum-classical algorithm, we need to construct an outer loop which optimizes the estimated $\bra{\gamma,\beta}H\ket{\gamma,\beta}$.

```python
# Black-box function that describes the energy output of the QAOA quantum circuit
def get_black_box_objective(G, p, SDK = 'qiskit', backend = None, shots=2**10):
    if SDK == 'cqasm':
        if not backend:
            backend = 'QX single-node simulator'
            backend_type = qi.get_backend_type_by_name(backend)
            def f(theta):
                # first half is betas, second half is gammas
                beta = theta[:p]
                gamma = theta[p:]
                qc = Cqasm.get_qaoa_circuit(G, beta, gamma)
                result = qi.execute_qasm(qc, backend_type=backend_type, number_of_shots=shots)
                counts = result['histogram']
                # return the energy
                return compute_maxcut_energy(counts, G)

    if SDK == 'qiskit':
        if not backend:
            backend = 'qasm_simulator'
        backend = Aer.get_backend(backend)
        def f(theta):
            # first half is betas, second half is gammas
            beta = theta[:p]
            gamma = theta[p:]
            qc = Qiskit.get_qaoa_circuit(G,beta, gamma)
            counts = execute(qc, backend,shots=shots).result().get_counts()
            # return the energy
            return compute_maxcut_energy(counts, G)
    else:
        return 'error: SDK not found'
    return f

# Estimate the expectation value based on the circuit output
def compute_maxcut_energy(counts, G):
    energy = 0
    total_counts = 0
    for meas, meas_count in counts.items():
        obj_for_meas = maxcut_obj(meas, G)
        energy += obj_for_meas * meas_count
        total_counts += meas_count
    return energy / total_counts
```

## 5. A simple instance on the quantum inspire platform: 2-qubit case

Let us first consider the most simple MAXCUT instance. We have just two nodes, and an optimal cut with objective value 1 would be to place both nodes in its own set.

```python
G=G2
max_costs_bf, max_sol_bf,costs = brute_force(G)
print("brute force method best cut: ",max_costs_bf)
print("best string brute force method:",max_sol_bf)

colors = ['red' if x == '0' else 'b' for x in max_sol_bf]
draw_cc_graph(G,node_color = colors)
```

Using qiskit, the circuit would look the following:

```python
# Test and show circuit for some beta,gamma
p = 1
beta = np.random.rand(p)*np.pi
gamma = np.random.rand(p)*2*np.pi
qc = Qiskit.get_qaoa_circuit(G,beta, gamma)
qc.draw(output='mpl')
```

Now let's run our hybrid-quantum algorithm simulation using qiskit:

```python
# Parameters that can be changed:
p = 1
lb = np.zeros(2*p)
ub = np.hstack([np.full(p, np.pi), np.full(p, 2*np.pi)])
init_point = np.random.uniform(lb, ub, 2*p)
shots = 2**10
optimiser = 'COBYLA'
max_iter = 100

# Training of the parameters beta and gamma
obj = get_black_box_objective(G,p,SDK='qiskit',shots=shots)
# Lower and upper bounds: beta \in {0, pi}, gamma \in {0, 2*pi}
bounds = [lb,ub]

# Maximum number of iterations: 100
res = minimize(obj, init_point, method=optimiser, bounds = bounds, options={'maxiter':max_iter, 'disp': True})
print(res)
```

    /home/redwombat/miniconda3/envs/qi-py38/lib/python3.8/site-packages/scipy/optimize/_minimize.py:544: RuntimeWarning: Method COBYLA cannot handle bounds.
      warn('Method %s cannot handle bounds.' % method,

         fun: -1.0
       maxcv: 0.0
     message: 'Optimization terminated successfully.'
        nfev: 26
      status: 1
     success: True
           x: array([-0.39933054,  0.79140574])

```python
#Determine the approximation ratio:
print('Approximation ratio is',-res['fun']/max_costs_bf)
```

    Approximation ratio is 1.0

```python
# Extract the optimal values for beta and gamma and run a new circuit with these parameters
optimal_theta = res['x']
qc = Qiskit.get_qaoa_circuit(G, optimal_theta[:p], optimal_theta[p:])
counts = execute(qc,backend = Aer.get_backend('qasm_simulator'),shots=shots).result().get_counts()
```

```python
plt.bar(counts.keys(), counts.values())
plt.xlabel('String')
plt.ylabel('Count')
plt.show()
```

```python
RG_dist = random_guessing_dist(G)
```

```python
# Measurement distribution
E_dist = defaultdict(int)
for k, v in counts.items():
    E_dist[maxcut_obj(k,G)] += v

plot_E_distributions([E_dist,RG_dist],p,['Qiskit','random guessing'])

E_random_guessing = energy_random_guessing(RG_dist)
print('Energy from random guessing is', E_random_guessing)
```

```python
X_list,Y_list = zip(*E_dist.items())
X = -np.asarray(X_list)
Y = np.asarray(Y_list)
print('Probability of measuring the optimal solution is',Y[np.argmax(X)]/shots)
```

    Probability of measuring the optimal solution is 1.0

Now that we have obtained some good values for $\beta$ and $\gamma$ through classical simulation, let's see what Starmon-5 would give us.

The figure below shows the topology of Starmon-5. Since q0 is not connected to q1, we have to relabel the nodes. Networkx as such an option, by using 'nx.relabel_nodes(G,{1:2}' we can relabel node 1 as node 2. Since q0 is connected to q2, this does allow us to run our cqasm code on Starmon-5. For qiskit, this step is irrelevant as we have all-to-all connectivity in the simulation.

```python
Image(filename='Starmon5.png')
```

```python
qc_Cqasm = Cqasm.get_qaoa_circuit(nx.relabel_nodes(G, {1: 2}), optimal_theta[:p], optimal_theta[p:])
print(qc_Cqasm)
```

    version 1.0
    qubits 5
    prep_z q[0:4]
    {H q[0] | H q[2]}

    .U_gamma_1
    CNOT q[0], q[2]
    Rz q[2], 1.5828114789648722
    CNOT q[0], q[2]
    .U_beta_1
    {Rx q[0], -0.798661078152829 | Rx q[2], -0.798661078152829}

    measure_all

Now we run the Cqasm-circuit on the Starmon-5 Hardware.

```python
authentication = get_authentication()
QI.set_authentication(authentication, QI_URL)
```

```python
qiapi = QuantumInspireAPI(QI_URL, authentication)
result = qiapi.execute_qasm(qc_Cqasm, backend_type=qiapi.get_backend_type('Starmon-5'), number_of_shots=2**10)
counts_QI = result['histogram']
```

Inspecting 'counts_QI', we see that it returns the integer corresponding to the bit string result of the measurement

```python
counts_QI
```

    OrderedDict([('0', 0.056640625),
                 ('1', 0.4208984375),
                 ('2', 0.005859375),
                 ('3', 0.021484375),
                 ('4', 0.3994140625),
                 ('5', 0.0283203125),
                 ('6', 0.0234375),
                 ('7', 0.001953125),
                 ('8', 0.001953125),
                 ('9', 0.0068359375),
                 ('12', 0.015625),
                 ('13', 0.001953125),
                 ('17', 0.0009765625),
                 ('20', 0.0078125),
                 ('22', 0.001953125),
                 ('28', 0.0048828125)])

Note that we measure more than just the two relevant qubits, since we had the 'measure all' command in the the cqasm code. The distribution over the strings looks the following:

```python
counts_bin = {}
for k,v in counts_QI.items():
    counts_bin[f'{int(k):05b}'] = v
print(counts_bin)
plt.bar(counts_bin.keys(), counts_bin.values())
plt.xlabel('State')
plt.ylabel('Measurement probability')
plt.xticks(rotation='vertical')
plt.show()
```

Let's create another counts dictionary with only the relevant qubits, which are q0 and q2:

```python
counts_bin_red = defaultdict(float)
for string in counts_bin:
    q0 = string[-1]
    q1 = string[-3]
    counts_bin_red[(q0+q1)]+=counts_bin[string]
```

```python
counts_bin_red
```

    defaultdict(float,
                {'00': 0.064453125,
                 '10': 0.4501953125,
                 '01': 0.453125,
                 '11': 0.0322265625})

We now plot all distributions (qiskit, Starmon-5, and random guessing) in a single plot.

```python
#Determine the approximation ratio:
print('Approximation ratio on the hardware is',-compute_maxcut_energy(counts_bin_red,G)/max_costs_bf)

# Random guessing distribution
RG_dist = random_guessing_dist(G)

# Measurement distribution
E_dist_S5 = defaultdict(int)
for k, v in counts_bin_red.items():
    E_dist_S5[maxcut_obj(k,G)] += v

plot_E_distributions([E_dist,E_dist_S5,RG_dist],p,['Qiskit','Starmon-5','random guessing'])

X_list,Y_list = zip(*E_dist_S5.items())
X = -np.asarray(X_list)
Y = np.asarray(Y_list)
print('Probability of measuring the optimal solution is',Y[np.argmax(X)])

E_random_guessing = energy_random_guessing(RG_dist)
print('Expected approximation ratio random guessing is', -E_random_guessing/max_costs_bf)
```

## 6. Compilation issues: the triangle graph

For the graph with just two nodes we already had some minor compilation issues, but this was easily fixed by relabeling the nodes. We will now consider an example for which relabeling is simply not good enough to get it mapped to the Starmon-5 toplogy.

```python
G=G3
max_costs_bf, max_sol_bf,costs = brute_force(G)
print("brute force method best cut: ",max_costs_bf)
print("best string brute force method:",max_sol_bf)

colors = ['red' if x == '0' else 'b' for x in max_sol_bf]
draw_cc_graph(G,node_color = colors)
```

Due to the topology of Starmon-5 this graph cannot be executed without any SWAPS. Therefore, we ask you to write a new circuit generator that uses SWAPS in order to make the algorithm work with the Starmon-5 topology. Let's also swap back to the original graph configuration, so that we can in the end measure only the qubits that correspond to a node in the graph (this is already written for you)

```python
def QAOA_triangle_circuit_cqasm(graph, beta, gamma):
    circuit_str = Cqasm.get_qasm_header(5)
    circuit_str += "{" + ' | '.join([f"H q[{i}]" for i in graph.nodes()]) + "}\n\n"

    def get_triangle_cost_operator(graph, gamma, p):
        layer_list = graph.number_of_edges() * [None]
        for n, edge in enumerate(graph.edges()):
            if 0 in edge and 1 in edge:
                layer_list[n] = '\n'.join([f"SWAP q[{edge[0]}], q[2]",
                                            f"CNOT q[2], q[{edge[1]}]",
                                            f"Rz q[{edge[1]}], {2*gamma}",
                                            f"CNOT q[2], q[{edge[1]}]",
                                            f"SWAP q[{edge[0]}], q[2]" ])
            else:
                layer_list[n] = '\n'.join([f"CNOT q[{edge[0]}], q[{edge[1]}]",
                                           f"Rz q[{edge[1]}], {2*gamma}",
                                           f"CNOT q[{edge[0]}], q[{edge[1]}]"])

        return f".U_gamma_{p}\n" + '\n'.join(layer_list) + '\n'

    circuit_str += '\n'.join([get_triangle_cost_operator(graph, gamma[i], i+1)
                              + Cqasm.get_mixing_operator(graph, beta[i], i+1) for i in range(p)])
    circuit_str += "\n"
    circuit_str += "{" + ' | '.join([f"measure q[{i}]" for i in graph.nodes()]) + "}\n"
    return circuit_str
```

We now run the same procedure as before to obtain good parameter values

```python
# Parameters that can be changed:
p = 1
lb = np.zeros(2*p)
ub = np.hstack([np.full(p, np.pi), np.full(p, 2*np.pi)])
init_point = np.random.uniform(lb, ub, 2*p)
shots = 2**10
optimiser = 'COBYLA'
max_iter = 100

# Training of the parameters beta and gamma
obj = get_black_box_objective(G,p,SDK='qiskit',shots=shots)
# Lower and upper bounds: beta \in {0, pi}, gamma \in {0, 2*pi}
bounds = [lb,ub]

# Maximum number of iterations: 100
res = minimize(obj, init_point, method=optimiser, bounds = bounds,options={'maxiter':max_iter, 'disp': True})
print(res)
```

    /home/redwombat/miniconda3/envs/qi-py38/lib/python3.8/site-packages/scipy/optimize/_minimize.py:544: RuntimeWarning: Method COBYLA cannot handle bounds.
      warn('Method %s cannot handle bounds.' % method,

         fun: -2.0
       maxcv: 0.0
     message: 'Optimization terminated successfully.'
        nfev: 32
      status: 1
     success: True
           x: array([1.89052483, 4.39449308])

```python
#Determine the approximation ratio:
print('Approximation ratio is',-res['fun']/max_costs_bf)

# Extract the optimal values for beta and gamma and run a new circuit with these parameters
optimal_theta = res['x']
qc = Qiskit.get_qaoa_circuit(G, optimal_theta[:p], optimal_theta[p:])
counts = execute(qc,backend = Aer.get_backend('qasm_simulator'),shots=shots).result().get_counts()

# Random guessing distribution
RG_dist = random_guessing_dist(G)

# Measurement distribution
E_dist = defaultdict(int)
for k, v in counts.items():
    E_dist[maxcut_obj(k,G)] += v

X_list,Y_list = zip(*E_dist.items())
X = -np.asarray(X_list)
Y = np.asarray(Y_list)
print('Probability of measuring the optimal solution is',Y[np.argmax(X)]/shots)

E_random_guessing = energy_random_guessing(RG_dist)
print('Expected approximation ratio random guessing is', -E_random_guessing/max_costs_bf)
```

    Approximation ratio is 1.0
    Probability of measuring the optimal solution is 1.0
    Expected approximation ratio random guessing is 0.75

```python
plt.bar(counts.keys(), counts.values())
plt.xlabel('String')
plt.ylabel('Count')
plt.show()
```

Let's run it on Starmon-5 again!

```python
# Extract the optimal values for beta and gamma and run a new circuit with these parameters
optimal_theta = res['x']
qasm_circuit = QAOA_triangle_circuit_cqasm(G, optimal_theta[:p], optimal_theta[p:])
qiapi = QuantumInspireAPI(QI_URL, authentication)
result = qiapi.execute_qasm(qasm_circuit, backend_type=qiapi.get_backend_type('Starmon-5'), number_of_shots=shots)
counts = result['histogram']

print(qasm_circuit)
print(result)
```

    version 1.0
    qubits 5
    prep_z q[0:4]
    {H q[0] | H q[1] | H q[2]}

    .U_gamma_1
    SWAP q[0], q[2]
    CNOT q[2], q[1]
    Rz q[1], 8.788986167232244
    CNOT q[2], q[1]
    SWAP q[0], q[2]
    CNOT q[0], q[2]
    Rz q[2], 8.788986167232244
    CNOT q[0], q[2]
    CNOT q[1], q[2]
    Rz q[2], 8.788986167232244
    CNOT q[1], q[2]
    .U_beta_1
    {Rx q[0], 3.781049654543756 | Rx q[1], 3.781049654543756 | Rx q[2], 3.781049654543756}

    {measure q[0] | measure q[1] | measure q[2]}

    {'id': 7069985, 'url': 'https://api.quantum-inspire.com/results/7069985/', 'job': 'https://api.quantum-inspire.com/jobs/7078006/', 'created_at': '2021-11-26T12:22:32.339692Z', 'number_of_qubits': 5, 'execution_time_in_seconds': 0.155648, 'raw_text': '', 'raw_data_url': 'https://api.quantum-inspire.com/results/7069985/raw-data/c34440ffc174351f859a320acde97bf1f5596d01f3bbd9acacd93a0771764fe5/', 'histogram': OrderedDict([('0', 0.0390625), ('1', 0.1611328125), ('2', 0.08203125), ('3', 0.240234375), ('4', 0.1025390625), ('5', 0.15625), ('6', 0.15625), ('7', 0.0625)]), 'histogram_url': 'https://api.quantum-inspire.com/results/7069985/histogram/c34440ffc174351f859a320acde97bf1f5596d01f3bbd9acacd93a0771764fe5/', 'measurement_mask': 7, 'quantum_states_url': 'https://api.quantum-inspire.com/results/7069985/quantum-states/c34440ffc174351f859a320acde97bf1f5596d01f3bbd9acacd93a0771764fe5/', 'measurement_register_url': 'https://api.quantum-inspire.com/results/7069985/measurement-register/c34440ffc174351f859a320acde97bf1f5596d01f3bbd9acacd93a0771764fe5/', 'calibration': 'https://api.quantum-inspire.com/calibration/109027/'}

```python
counts
```

    OrderedDict([('0', 0.0390625),
                 ('1', 0.1611328125),
                 ('2', 0.08203125),
                 ('3', 0.240234375),
                 ('4', 0.1025390625),
                 ('5', 0.15625),
                 ('6', 0.15625),
                 ('7', 0.0625)])

```python
counts_bin = {}
for k,v in counts.items():
    counts_bin[f'{int(k):03b}'] = v
print(counts_bin)
plt.bar(counts_bin.keys(), counts_bin.values())
plt.xlabel('String')
plt.ylabel('Probability')
plt.show()
```

```python
#Determine the approximation ratio:
print('Approximation ratio on the hardware is',-compute_maxcut_energy(counts_bin,G)/max_costs_bf)

# Random guessing distribution
RG_dist = random_guessing_dist(G)

# Measurement distribution
E_dist_S5 = defaultdict(int)
for k, v in counts_bin.items():
    E_dist_S5[maxcut_obj(k,G)] += v

plot_E_distributions([E_dist,E_dist_S5,RG_dist],p,['Qiskit','Starmon-5','random guessing'])

X_list,Y_list = zip(*E_dist_S5.items())
X = -np.asarray(X_list)
Y = np.asarray(Y_list)
print('Probability of measuring the optimal solution is',Y[np.argmax(X)])

E_random_guessing = energy_random_guessing(RG_dist)
print('Expected approximation ratio random guessing is', -E_random_guessing/max_costs_bf)
```

## 7. More advanced questions

Some questions you could look at:

- What is the performance on other graph instances?
- How scalable is this hardware for larger problem sizes?
- How much can the circuit be optimized for certain graph instances?
- Are the errors perfectly random or is there some correlation?
- Are there tricks to find good parameters?

```python

```

```python

```


```python
import pandapower as pp
```

```python
from sympy import symbols
```

```python
net = pp.create_empty_network()
b1 = pp.create_bus(net, vn_kv=13.2)
b2 = pp.create_bus(net, vn_kv=13.2)
b3 = pp.create_bus(net, vn_kv=13.2)

pp.create_line(net, from_bus=b1, to_bus=b2, length_km=0.8, std_type="NAYY 4x50 SE")
pp.create_line(net, from_bus=b2, to_bus=b3, length_km=1.2, std_type="NAYY 4x50 SE")

pp.create_ext_grid(net, bus=b1)

pp.create_load(net, bus=b3, p_mw=0.350)
```

    0

```python
 pp.runpp(net)
```

```python

```

```python
print(net.res_bus.vm_pu)
print(net.res_line)
```

    0    1.000000
    1    0.998973
    2    0.997425
    Name: vm_pu, dtype: float64
       p_from_mw  q_from_mvar   p_to_mw     q_to_mvar     pl_mw   ql_mvar  \
    0   0.350909    -0.022814 -0.350545  1.367420e-02  0.000364 -0.009140
    1   0.350545    -0.013674 -0.350000 -9.562987e-10  0.000545 -0.013674

       i_from_ka   i_to_ka      i_ka  vm_from_pu  va_from_degree  vm_to_pu  \
    0   0.015381  0.015360  0.015381    1.000000        0.000000  0.998973
    1   0.015360  0.015348  0.015360    0.998973       -0.010749  0.997425

       va_to_degree  loading_percent
    0     -0.010749        10.831460
    1     -0.023998        10.816756

```python
pp.create_sgen(net, b2, p_mw=0.05, q_mvar=0.025, max_p_mw=0.2, max_q_mvar=0.2)
```

    0

```python
pp.runpp(net)
```

```python
print(net.res_bus.vm_pu)
print(net.res_line)
```

    0    1.000000
    1    0.999130
    2    0.997582
    Name: vm_pu, dtype: float64
       p_from_mw  q_from_mvar   p_to_mw     q_to_mvar     pl_mw   ql_mvar  \
    0   0.300817    -0.047832 -0.300544  3.867856e-02  0.000272 -0.009153
    1   0.350544    -0.013679 -0.350000 -7.725773e-10  0.000544 -0.013679

       i_from_ka   i_to_ka      i_ka  vm_from_pu  va_from_degree  vm_to_pu  \
    0   0.013323  0.013265  0.013323     1.00000        0.000000  0.999130
    1   0.015357  0.015346  0.015357     0.99913       -0.013882  0.997582

       va_to_degree  loading_percent
    0     -0.013882         9.382118
    1     -0.027127        10.815054

```python
print(net)
```

    This pandapower network includes the following parameter tables:
       - bus (3 elements)
       - load (1 element)
       - sgen (1 element)
       - ext_grid (1 element)
       - line (2 elements)
     and the following results tables:
       - res_bus (3 elements)
       - res_line (2 elements)
       - res_ext_grid (1 element)
       - res_load (1 element)
       - res_sgen (1 element)

```python
def net_add_optfw(net):
    net.ext_grid['initial_cost'] = 0.0
    net.ext_grid['operation_cost'] = 0.0
    net.ext_grid['ap_mw'] = 100.0
    net.ext_grid['op_mw'] = 0.0
    net.ext_grid['constraints'] = None

    net.load['initial_cost'] = 0.0
    net.load['operation_cost'] = 0.0
    net.load['ap_mw'] = 100.0
    net.load['op_mw'] = 0.0
    net.load['constraints'] = None

    net.sgen['initial_cost'] = 0.0
    net.sgen['operation_cost'] = 0.0
    net.sgen['ap_mw'] = 100.0
    net.sgen['op_mw'] = 0.0
    net.sgen['constraints'] = None
```

```python
net_add_optfw(net)
```

```python
net.ext_grid
```

      name
      bus
      vm_pu
      va_degree
      in_service
      initial_cost
      operation_cost
      ap_mw
      op_mw
      constraints

      0
      None
      0
      1.0
      0.0
      True
      0.0
      0.0
      100.0
      0.0
      None

Variables que define el modelo:

t para el tiempo, (en horas)

```python

```

```python
net.ext_grid
```

      name
      bus
      vm_pu
      va_degree
      in_service

      0
      None
      0
      1.0
      0.0
      True

```python
net.ext_grid.__class__
```

```python
net.load.__class__
```

```python
net.load.columns
```

```python
net.line
```

```python
net.load
```

      name
      bus
      p_mw
      q_mvar
      const_z_percent
      const_i_percent
      sn_mva
      scaling
      in_service
      type

      0
      None
      2
      0.35
      0.0
      0.0
      0.0
      NaN
      1.0
      True
      wye

```python
net.load['bus'][0]
```

    2

```python
net.bus.loc[2]['vn_kv']+5
```

    18.2

```python
net.load['ap_mw']=net.load['p_mw']
```

```python
net.load
```

```python
net.load['name'][0]='Carga'
```

```python
net.load
```

```python
net.ext_grid
```

```python
net.switch
```

```python

```


## Lecture 5 - Convolutional neural networks

> A deep dive into convolutional neural networks for jet tagging

## Learning objectives

* Know how to implement and train a convolutional neural network in PyTorch
* Learn how to debug unstable training runs with activation statistics
* Understand what the 1-cycle training policy is
* Understand what batch normalization is and how to incorporate it into a CNN

## References

* Chapter 13 of [_Deep Learning for Coders with fastai & PyTorch_](https://github.com/fastai/fastbook) by Jeremy Howard and Sylvain Gugger.
* Chapter 11 of [_Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow_](https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1492032646/ref=sr_1_1?crid=29Z1GSCOWEPDV&keywords=hands+on+machine+learning+with+scikit-learn+and+tensorflow+2&qid=1653288575&sprefix=hands+on+ma%2Caps%2C160&sr=8-1) by Aurélien Geron

## Setup

```python
# Uncomment and run this cell if using Colab, Kaggle etc
# %pip install fastai==2.6.0 datasets
```

## Imports

```python
from datasets import load_dataset
from fastai.callback.hook import *
from fastai.vision.all import *
from sklearn.metrics import accuracy_score
from torch.utils.data import TensorDataset
from torchvision.transforms import ToTensor
```

```python
import datasets

# Suppress logs to keep things tidy
datasets.logging.set_verbosity_error()
```

## Loading the data

Last lecture we fine-tuned a pretrained CNN on the top tagging dataset, where each jet was represented as a 2D image of "energy pixels". Today, we'll take a look at training our own CNN from scratch, and explore some of the techniques that fastai utilizes to stabilise the training of these neural nets. To get started, let's download the same dataset of jet images from last lecture:

```python
images_ds = load_dataset("dl4phys/top_tagging_images")

# Peek at one example
plt.imshow(images_ds["train"][0]["image"]);
```

As we saw last lecture, we need to convert these PIL images into PyTorch tensors, so let's reuse the same helper function to create a training and validation set to experiment with:

```python
def get_dataset(dataset, num_examples=None):
    if num_examples is not None:
        dataset = dataset.shuffle(seed=42).select(range(num_examples))

    x = torch.cat([ToTensor()(img) for img in dataset["image"]]).unsqueeze(1)
    y = torch.cat([torch.tensor(l).unsqueeze(0) for l in dataset["label"]])

    return TensorDataset(x, y)
```

With this function, we can now generate a sample of jets as follows:

```python
# Lower the training size if Colab RAM explodes 💣
train_ds = get_dataset(images_ds["train"], num_examples=350_000)
valid_ds = get_dataset(images_ds["validation"], num_examples=35_000)
```

Since we'll be experimenting a bit with the batch size in this lecture, we'll also implement a helper function to return the dataloaders associated with these datasets:

```python
def get_dls(bs=128):
    train_dl = DataLoader(train_ds, bs=bs, shuffle=True)
    valid_dl = DataLoader(valid_ds, bs=bs)
    return DataLoaders(train_dl, valid_dl)
```

Now that we have this function, we can get the dataloaders and grab a batch of data to test with:

```python
dls = get_dls()

xb, yb = first(dls.train)
xb.shape, yb.shape
```

    (torch.Size([128, 1, 40, 40]), torch.Size([128]))

## Creating a CNN

Recall in lecture 3, that we created a neural network for $N$-subjettiness features. We used the `nn.Sequential()` class to create a network that involved stacking fully-connected layers and ReLU activation functions:

```python
model = nn.Sequential(
    nn.Linear(20, 200),
    nn.ReLU(),
    nn.Linear(200, 2),
)

model
```

    Sequential(
      (0): Linear(in_features=20, out_features=200, bias=True)
      (1): ReLU()
      (2): Linear(in_features=200, out_features=2, bias=True)
    )

In this lecture, we'll take a similar approach, but this time using _convolutional layers_ instead of linear ones. In PyTorch, convolutional layers are created by using the `nn.Conv2d()` module. Let's try replacing the `nn.Linear()` layers in our previous architecture with `nn.Conv2d()` ones instead:

```python
broken_cnn = nn.Sequential(
    nn.Conv2d(1, 30, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.Conv2d(30, 1, kernel_size=3, padding=1),
)
```

Note that unlike linear layers, convolutional layers don't require us to specify the number of features in the input. That's because convolutions are applied across each pixel automatically, so the weights only depend on the number of channels and the kernel size.

Let's now see what happens if we pass a minibatch of data through this model:

```python
# Feed a minibatch to the model
outputs = broken_cnn(xb)
outputs.shape
```

    torch.Size([128, 1, 40, 40])

Hmm, this output isn't quite what we want for classification: we need a tensor of shape `(batch_size, num_classes)` but instead have a $40\times 40$ map of activations. The standard way to handle this is to apply a sequence of _stride-2 convolutions_, which decrease the size of the outputs so that the final layer size is 1. To see why this is the case, recall that for an image of height $n_H$ and width $n_W$, the dimension of the output activation map is given by:

$$\left( \left\lfloor\frac{n_H + 2p - k}{s} + 1 \right\rfloor ,  \left\lfloor\frac{n_W + 2p - k}{s} + 1 \right\rfloor \right) \,,$$

where $p$ is the padding, $k$ the kernel of size $k\times k$, and $s$ the stride. For fixed $p,f$ and $s>1$, we can see that the dimension of the activation map is shrunk with each convolutional layer.

With this in mind, let's create a stack of stride-2 convolutional layers with $3\times 3$ kernels. We'll intersperse each convolutional layer with a ReLU activation, so let's write a helper function that returns the two together:

```python
def conv(ni, nf, ks=3, act=True):
    layer = nn.Conv2d(ni, nf, stride=2, kernel_size=ks, padding=ks // 2)
    if act:
        layer = nn.Sequential(layer, nn.ReLU())
    return layer
```

As discussed in the fastai book, it's good practice to increase the number of output features `nf` with each convolutional layer. That's because, stride-2 convolutions decrease the size of the activation map, so we increase the number of output features to avoid compressing the capacity of our model. We also set the kernel size `ks` to the typical value of $3\times 3$, and set the padding to $ks//2$ to ensure the output activation map is the same shape as the input image.

We can now build a CNN by stacking the `conv()` function until we reach a $1\times 1$ activation map:

```python
simple_cnn = nn.Sequential(
    conv(1, 4),  # 20x20
    conv(4, 8),  # 10x10
    conv(8, 16),  # 5x5
    conv(16, 32),  # 3x3
    conv(32, 16),  # 2x2
    conv(16, 2, act=False),  # 1x1
    Flatten(),
)

learn = Learner(
    dls, simple_cnn, metrics=[accuracy, RocAucBinary()], loss_func=F.cross_entropy
)
learn.summary()
```

    Sequential (Input shape: 128 x 1 x 40 x 40)
    ============================================================================
    Layer (type)         Output Shape         Param #    Trainable
    ============================================================================
                         128 x 4 x 20 x 20
    Conv2d                                    40         True
    ReLU
    ____________________________________________________________________________
                         128 x 8 x 10 x 10
    Conv2d                                    296        True
    ReLU
    ____________________________________________________________________________
                         128 x 16 x 5 x 5
    Conv2d                                    1168       True
    ReLU
    ____________________________________________________________________________
                         128 x 32 x 3 x 3
    Conv2d                                    4640       True
    ReLU
    ____________________________________________________________________________
                         128 x 16 x 2 x 2
    Conv2d                                    4624       True
    ReLU
    ____________________________________________________________________________
                         128 x 2 x 1 x 1
    Conv2d                                    290        True
    ____________________________________________________________________________
                         128 x 2
    Flatten
    ____________________________________________________________________________

    Total params: 11,058
    Total trainable params: 11,058
    Total non-trainable params: 0

    Optimizer used: <function Adam at 0x7fea02760430>
    Loss function: <function cross_entropy at 0x7fea509cf8b0>

    Callbacks:
      - TrainEvalCallback
      - Recorder
      - ProgressCallback

Since the final convolutional layer will be a tensor of shape $128\times2\times1\times1$, we've stripped off those final $1\times 1$ axes via the `Flatten()` module. We can now verify that the output of this model is in a form suitable for classification:

```python
simple_cnn(xb).shape
```

    torch.Size([128, 2])

Okay, this looks good, so let's find a good learning rate and train for a few epochs:

```python
learn.lr_find()
```

```python
learn.fit(3, 3e-3)
```

      epoch
      train_loss
      valid_loss
      accuracy
      roc_auc_score
      time

      0
      0.251922
      0.257991
      0.892286
      0.957853
      00:12

      1
      0.248150
      0.243493
      0.899114
      0.963003
      00:12

      2
      0.231296
      0.237253
      0.900543
      0.964258
      00:12

This model is much worse than the pretrained ResNet34 model that we fine-tuned in the previous lecture. We can also see that we've started overfitting quite strongly after the first epoch (because the training and validation losses are diverging). Let's now see if we can make the model more accurate and train it even faster by using some of the techniques that the factory learners (like `vision_learner()`) provide under the hood.

## Debugging training with activation statistics

One way to train a more accurate CNN is simply double the number of filters in each stride-2 layer. However, increasing the number of filters also requires us to increase the kernel size. For example, if our first layer increases the number of filters from 4 to 8, then a $3 \times 3$ kernel will not force the CNN to learn any useful features. To handle that, we'll increase the kernel size to $5\times 5$ and adjust the CNN architecture as follows:

```python
def simple_cnn():
    return nn.Sequential(
        conv(1, 8, ks=5),  # 20x20
        conv(8, 16),  # 10x10
        conv(16, 32),  # 5x5
        conv(32, 64),  # 3x3
        conv(64, 128),  # 2x2
        conv(128, 2, act=False),  # 1x1
        Flatten(),
    ).to("cuda")
```

Since we'll be training this network in several ways, let's wrap the creation of the `Learner` and training in a `fit()` function:

```python
def fit(epochs=1, lr=0.06):
    learn = Learner(
        dls,
        simple_cnn(),
        metrics=[accuracy, RocAucBinary()],
        loss_func=F.cross_entropy,
        cbs=ActivationStats(with_hist=True),
    )
    learn.fit(epochs, lr)
    return learn
```

To give a sense for what can go wrong when training CNNs, we've used a much larger learning rate than before. We've also use the `ActivationStats()` callback, which records the statistics of the activations in each layer - this will be handy for debugging. Let's see how well this model performs after 1 epoch:

```python
learn = fit()
```

    /home/lewis/miniconda3/envs/dl4phys/lib/python3.9/site-packages/fastai/callback/core.py:67: UserWarning: You are shadowing an attribute (modules) that exists in the learner. Use `self.learn.modules` to avoid this
      warn(f"You are shadowing an attribute ({name}) that exists in the learner. Use `self.learn.{name}` to avoid this")

      epoch
      train_loss
      valid_loss
      accuracy
      roc_auc_score
      time

      0
      0.694601
      0.693960
      0.500343
      0.500000
      00:19

Okay, it seems the large learning rate has given us a model than doesn't perform better than random chance. To diagnose the problem, we can use the `plot_layer_stats()` method that the `ActivationStats` callback has tracked. For example, here are the mean and standard deviation of the activations in the first layer:

```python
learn.activation_stats.plot_layer_stats(0)
```

Here we can see that the mean and standard deviation have a somewhat dramatic spike / drop in the early stages of training. A bigger problem is that the fraction of activations that are near zero is almost 1 after a few iterations. This is a problem because activations that are zero in one layer, will propagate zeros to the next layer, and so on. As a result, these parts of the network are effectively turned off, which makes the learning process suboptimal.

Let's now have a look at the last convolutional layer in the CNN:

```python
learn.activation_stats.plot_layer_stats(-2)
```

Here we see a similar pattern, with the fraction of near zero activations reaching almost 1 in fewer iterations. Let's see if we can imprve the situation by increasing the batch size.

### Increasing the batch size

If you're GPU can handle it, increasing the batch size is one technique that can sometimes stabilise training. This is because a larger batch size implies more accurate gradients, so SGD can potentially train more efficiently. Let's try increasing our batch size by a factor of 4, training the model and investigating the activation statistics:

```python
dls = get_dls(512)
learn = fit()
learn.activation_stats.plot_layer_stats(-2)
```

Okay, increasing the batch size hasn't helped much. Clearly the problem is that our learning rate is too large, which is causing training to diverge. One way to handle this is to use a _dynamic_ learning rate that is adjusted from low to high, and back to low values during training itself! Let's take a look at 1-cycle training and finally understand what's going one when we call `Learner.fit_one_cycle()`.

### 1-cycle training

The basic idea behind 1-cycle training is to split training into two phases:

* **Warmup:** grow the learning rate from some minimum to maximum value. By starting with a small learning rate, we can avoid hopping out of a local minimum in the loss and diverging.
* **Annealing:** decrease the learning rate from the maximum value to the minimum. By gradually lowering the learning rate, we avoid skipping a good local minimum towards the end of training.

This technique is one of the main aspects that allows fastai to train fast and accurate models. As we've done in previous lectures, we can use the `fit_one_cycle()` method, so let's adjust our `fit()` function and train again:

```python
def fit(epochs=1, lr=0.06):
    learn = Learner(
        dls,
        simple_cnn(),
        metrics=[accuracy, RocAucBinary()],
        loss_func=F.cross_entropy,
        cbs=ActivationStats(with_hist=True),
    )
    learn.fit_one_cycle(epochs, lr)
    return learn

learn = fit()
```

    /home/lewis/miniconda3/envs/dl4phys/lib/python3.9/site-packages/fastai/callback/core.py:67: UserWarning: You are shadowing an attribute (modules) that exists in the learner. Use `self.learn.modules` to avoid this
      warn(f"You are shadowing an attribute ({name}) that exists in the learner. Use `self.learn.{name}` to avoid this")

      epoch
      train_loss
      valid_loss
      accuracy
      roc_auc_score
      time

      0
      0.255070
      0.257523
      0.892029
      0.957344
      00:12

Nice, we've finally got a model that performs better than random! All fastai `Learner`s come with a `Recorder()` callback, which tracks various aspect of training. Here we can use it to inspect the evolution of the learning rate:

```python
learn.recorder.plot_sched()
```

Here we can see the learning rate increases linearly until the maximum value, before being annealed with a cosine function. The second plot refers to a hyperparameter called _momentum_, which takes values between $[0,1]$ and is often denoted by $\beta$. This hyperparameter belongs to a technique called _momentum optimization_, which extends SGD to speed up training. Recall that in SGD, we update our weights and biases according to the following update rule:

$$ \theta \to \theta' = \theta - \eta \nabla_\theta L(\theta) .$$

One problem with this rule, is that if the gradients are small in some region, then the subsequent updates will also be small and training can be slow. To deal with this, momentum optimization uses an _exponentially weighted average_ to modify the update rule:

\begin{align}
\mathbf{m}'&= \beta\mathbf{m} + (1-\beta)\nabla_\theta L(\theta) \\
\theta' &= \theta - \eta\mathbf{m}'
\end{align}

Intuitively, the momentum vector $\mathbf{m}$ is storing a running average of past gradient values, and this enables the update step to pick directions based on these averages. The result is that SGD with momentum allows us to traverse the loss landscape much faster (especially through plateaus).

Let's now have a look at the activation statistics in the final `conv()` layer:

```python
learn.activation_stats.plot_layer_stats(-2)
```

Okay, we've getting somewhat better, but we still have quite a few activations getting close to zero towards the end of training. Let's look at one last technique that can help us deal with this.

### Batch normalization

A very effective technique to deal with vanishing activations in training is to use _batch normalization_ (or batchnorm for short). This techniuqe tries to maintain a good distribution of activations during training by applying a normalization operation just before or after each hidden layer. The way this works is to zero-center and normalize the activations in each layer by introducing two new parameters $\gamma$ and $\beta$ (not the same $\beta$ from momentum!). The parameters are used to learn the optimal scale of the inputs across each layer, and the batchnorm operation can be summarised in the following equations:

\begin{align}
\mathbf{\mu}_B &= \frac{1}{m_B} \sum_i \mathbf{x}^{(i)} \\
\sigma_B^2 &= \frac{1}{m_B} \sum_i \left(\mathbf{x}^{(i)} - \mathbf{\mu}_B\right)^2  \\
\hat{\mathbf{x}}^{(i)}  &= \frac{\mathbf{x}^{(i)} - \mathbf{\mu}_B}{\sqrt{\mathbf{\sigma}_B^2 + \epsilon}} \\
\mathbf{z}^{(i)} &= \mathbf{\gamma} \odot \hat{\mathbf{x}}^{(i)} + \mathbf{\beta}
\end{align}

These statistics make it easier to train models, since we don't have to enforce a global normalization on the data (as we did with the `MinMaxScaler()` in previous lectures).

Implementing batchnorm in PyTorch is rather simple: we just add a `nn.BatchNorm2d()` layer after each convolution:

```python
def conv(ni, nf, ks=3, act=True):
    layers = [nn.Conv2d(ni, nf, stride=2, kernel_size=ks, padding=ks // 2)]
    layers.append(nn.BatchNorm2d(nf))
    if act:
        layers.append(nn.ReLU())
    return nn.Sequential(*layers)
```

Let's now train the model again with 1-cycle policy training:

```python
learn = fit()
```

    /home/lewis/miniconda3/envs/dl4phys/lib/python3.9/site-packages/fastai/callback/core.py:67: UserWarning: You are shadowing an attribute (modules) that exists in the learner. Use `self.learn.modules` to avoid this
      warn(f"You are shadowing an attribute ({name}) that exists in the learner. Use `self.learn.{name}` to avoid this")

      epoch
      train_loss
      valid_loss
      accuracy
      roc_auc_score
      time

      0
      0.216597
      0.214471
      0.909714
      0.970490
      00:22

Nice! This is pretty great result after just one epoch of training, and gets is quite close to the results quoted for CNNs in the top taggin review paper. Let's also inspect the activation statistics:

```python
learn.activation_stats.plot_layer_stats(-2)
```

This is looking much better: the activations evolve smoothly and we've managed to prevent half of the activations from vanishing. Since batchnorm claims to work with large learning rates, let's ramp this up to 0.1 and see what we get:

```python
learn = fit(10, lr=0.1)
```

    /home/lewis/miniconda3/envs/dl4phys/lib/python3.9/site-packages/fastai/callback/core.py:67: UserWarning: You are shadowing an attribute (modules) that exists in the learner. Use `self.learn.modules` to avoid this
      warn(f"You are shadowing an attribute ({name}) that exists in the learner. Use `self.learn.{name}` to avoid this")

      epoch
      train_loss
      valid_loss
      accuracy
      roc_auc_score
      time

      0
      0.241621
      0.324524
      0.879571
      0.951736
      00:22

      1
      0.232091
      1.657844
      0.655914
      0.913255
      00:21

      2
      0.222661
      0.819799
      0.612086
      0.906052
      00:21

      3
      0.220339
      0.276002
      0.885429
      0.957145
      00:21

      4
      0.217905
      0.410375
      0.813257
      0.920187
      00:21

      5
      0.209055
      0.433610
      0.802971
      0.920204
      00:21

      6
      0.204782
      0.363477
      0.832943
      0.945951
      00:21

      7
      0.198509
      0.206748
      0.913429
      0.972911
      00:21

      8
      0.191656
      0.202564
      0.916000
      0.974434
      00:21

      9
      0.182151
      0.196292
      0.918886
      0.975237
      00:21

Great, this has given us a small boost and only took a few minutes to train a CNN form scratch!

## Exercises

* Implement the same CNN architecture from the review. Can you get close to their results?
* Read the [1-cycle policy training paper](https://arxiv.org/abs/1708.07120)

```python

```


## Example 1: `Hello world`

```python
# Print the character string `Hello world!`
print('Hello world!')
```

    Hello world!

## Example 2: Make a plot of the US unemployment rate

Download unemployment rate data from FRED (https://fred.stlouisfed.org/series/UNRATE/) and make a well-labeled plot.

```python
# Import the pandas library as pd
import pandas as pd

# Import the plotting library matplotlib.pyplot as plt
import matplotlib.pyplot as plt

# Download the unemployment rate data (PROVIDED)
unemployment = pd.read_csv('https://fred.stlouisfed.org/data/UNRATE.txt',skiprows=24,sep='\s+',index_col=0,parse_dates = True)

# Print first 5 rows of unemployment variable
unemployment.head()
```

      VALUE

      DATE

      1948-01-01
      3.4

      1948-02-01
      3.8

      1948-03-01
      4.0

      1948-04-01
      3.9

      1948-05-01
      3.5

```python
# Create a well-labeled plot of the US unemployment rate
unemployment.plot(legend=False,lw=3,alpha=0.75)
plt.ylabel('Percent')
plt.xlabel('Date')
plt.title('US Unemployment Rate')

# Save the figure to the current working directory at 120 dots per inch resolution
plt.savefig('unemployment.png',dpi=120)
```

## Example 2: Date and time

Use the `datetime` module to print the current date and time

```python
# Import the datetime module
import datetime

# Create a variable called `today` that stores the current date and time
today = datetime.datetime.today()

# Print the value of `today`
print(today)
```

    2021-01-05 12:02:08.861515

```python
# Print today's date formatted as: month day, year
print(today.strftime('%B %d, %Y'))
```

    January 05, 2021

```python
# Print the time component of `today` as: HH:MM:SS
print(today.strftime('%I:%M:%S'))
```

    12:02:08

## Example 3: Approximate a geometric series

Consider the infinite geometric series:

\begin{align}
\sum_{k=0}^{\infty} r^k & = r + r + r^2 + r^3 + \cdots
\end{align}

If $|r|<1$, then the series converges to:

\begin{align}
\frac{1}{1+r}
\end{align}

We can verify this fact numerically by computing the *truncated* series, i.e., compute the series to a finite number of terms. We will do this for $r=0.2$. The idea is to create a variable $s$ that is initially equal to zero. Then we let $k$ increment from 0, 1, up to $N$ and each time add $r^k$ to $s$.

```python
# Set the value of r
r = 0.2

# Create a variable `N` that stores the desired number of terms to compute
N = 25

# Initialize a variable that will store the value of the summation. Set equal to tzero
s = 0

# Iterate over values of k fron 0 to N. Add the new term to the sum and print the current value of the sum
for k in range(N+1):
    s+=r**k
    print(s)
```

    1.0
    1.2
    1.24
    1.248
    1.2496
    1.2499200000000001
    1.2499840000000002
    1.2499968000000001
    1.2499993600000001
    1.249999872
    1.2499999744
    1.24999999488
    1.249999998976
    1.2499999997952
    1.24999999995904
    1.249999999991808
    1.2499999999983618
    1.2499999999996725
    1.2499999999999347
    1.2499999999999871
    1.2499999999999976
    1.2499999999999996
    1.25
    1.25
    1.25
    1.25

## Example 4: Draw random values from a normal probability distribution

The `numpy` library has a bunch of powerful tools for working with numbers. Here we'll use `numpy` to draw a random sample from the normal probability distribution. Specifically, 100 values from $\mathcal{N}(2,0.2^2)$.

```python
# Import the numpy numerical tools library as np
import numpy as np

# Set the seed or state of the numpy random number generator
np.random.seed(126)

# Create a variable called 'x' that contains 100 draws from the normal(2,0.2^2) didistribution
x = np.random.normal(loc=2,scale=0.2,size=100)

plt.plot(x)
```

```python
# Print the mean of the values in the variable 'x'
print(np.mean(x))
```

    1.983109892630882

```python
# Print the standard deviation of the values in the variable 'x'
print(np.std(x))
```

    0.21843109555638324