# Qcamp - Terra
## IBMQ
### Donny Greenberg, Kevin Krsulich and Thomas Alexander

# Gameplan

* Basics
  * What is Terra?
  * Teleportation
  * QPE a few ways
* Browsing device info
* Tips and tricks
* Learning More, Resources

# What is Terra?

Terra’s core service is the compilation and execution of Quantum circuits for arbitrary backends, and shipping jobs to backends
  * It includes operations for circuit construction, including loading QASM
  * Terra can take the same circuit object and compile and run it on any Quantum hardware or simulator
  * Local simulators are included in Terra and Aer
  * Terra has IBM Q API connections built in - it will send your job to your desired backend and collect the results

Keep in mind:

  * Terra is not a language per se, but more of a large piece of infrastructure.
  * Qiskit is very much a work in progress. It is changing rapidly to converge toward the needs of its users. We welcome development suggestions and help!
  * See our Github (https://github.com/Qiskit/qiskit-terra).

In the future, Terra will include:

  * OpenPulse, pulse level control of IBM Quantum Hardware (find Thomas and ask him about it!)
  * More sophisticated circuit builder interface for constructing and composing large circuits (find Kevin and ask him about it!)

But first, install Terra:

```python
!pip install qiskit
```

    Requirement already satisfied: qiskit in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (0.7.0)
    Requirement already satisfied: qiskit-terra<0.8,>=0.7 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit) (0.7.0)
    Requirement already satisfied: qiskit-aer<0.2,>=0.1 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit) (0.1.0)
    Requirement already satisfied: networkx>=2.2 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (2.2)
    Requirement already satisfied: psutil>=5 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (5.4.8)
    Requirement already satisfied: requests-ntlm>=1.1.0 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (1.1.0)
    Requirement already satisfied: scipy!=0.19.1,>=0.19 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (1.1.0)
    Requirement already satisfied: pillow>=4.2.1 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (5.3.0)
    Requirement already satisfied: numpy>=1.13 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (1.15.4)
    Requirement already satisfied: sympy>=1.3 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (1.3)
    Requirement already satisfied: ply>=3.10 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (3.11)
    Requirement already satisfied: jsonschema<2.7,>=2.6 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (2.6.0)
    Requirement already satisfied: marshmallow<3,>=2.16.3 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (2.17.0)
    Requirement already satisfied: marshmallow-polyfield<4,>=3.2 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (3.2)
    Requirement already satisfied: requests>=2.19 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from qiskit-terra<0.8,>=0.7->qiskit) (2.20.1)
    Requirement already satisfied: decorator>=4.3.0 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from networkx>=2.2->qiskit-terra<0.8,>=0.7->qiskit) (4.3.0)
    Requirement already satisfied: ntlm-auth>=1.0.2 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (1.2.0)
    Requirement already satisfied: cryptography>=1.3 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (2.4.1)
    Requirement already satisfied: mpmath>=0.19 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from sympy>=1.3->qiskit-terra<0.8,>=0.7->qiskit) (1.0.0)
    Requirement already satisfied: idna<2.8,>=2.5 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests>=2.19->qiskit-terra<0.8,>=0.7->qiskit) (2.7)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests>=2.19->qiskit-terra<0.8,>=0.7->qiskit) (3.0.4)
    Requirement already satisfied: urllib3<1.25,>=1.21.1 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests>=2.19->qiskit-terra<0.8,>=0.7->qiskit) (1.24.1)
    Requirement already satisfied: certifi>=2017.4.17 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from requests>=2.19->qiskit-terra<0.8,>=0.7->qiskit) (2018.11.29)
    Requirement already satisfied: cffi!=1.11.3,>=1.7 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (1.11.5)
    Requirement already satisfied: asn1crypto>=0.21.0 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (0.24.0)
    Requirement already satisfied: six>=1.4.1 in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (1.11.0)
    Requirement already satisfied: pycparser in /Users/talexander/anaconda3/envs/qiskit/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.7->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-terra<0.8,>=0.7->qiskit) (2.19)

# Structural Elements

Let's start building circuits and get acquainted with Terra.

```python
# Housekeeping: uncomment this to suppress deprecation warnings
import warnings
warnings.filterwarnings('ignore')
```

```python
from qiskit import QuantumRegister, ClassicalRegister, QuantumCircuit
import numpy as np
```

```python
# Create a Quantum Register with 3 qubits
qr = QuantumRegister(3)

# Create a Classical Register with 3 bits
# Only necessary if you want to do measurement!
cr = ClassicalRegister(3)

# Create a Quantum Circuit acting on the qr and cr register
circuit = QuantumCircuit(qr, cr)
```

# QuantumCircuits are the primary unit of computation in Terra
* QuantumCircuits are backend agnostic
* They contain:
  * name - for referencing the circuit later (e.g. in the results object)
  * data - a list of gates in the circuit
  * regs - the QuantumRegisters and ClassicalRegisters in the gates of the circuit

# Gates! There are many.

Qiskit supports many gates. They are located in the `qiskit/extensions/standard` directory, but are loaded behind the scenes so you don’t need to import them one by one.

Gates are technically objects, but in practice you're likely to use them in the form of static functions on the circuit object. More info on gates [here](https://github.com/Qiskit/qiskit-tutorial/blob/master/qiskit/terra/summary_of_quantum_operations.ipynb).

The basis gateset of the IBM Q devices is `{id, u1, u2, u3, cx}`.

After we add some gates, we can print our circuit's Qasm:

```python
# Hadamard gate on qubit 0
circuit.h(qr[0])

# CNOT (Controlled-NOT) gate from qubit 0 to qubit 1
circuit.cx(qr[0], qr[1])

print(circuit.qasm())
```

    OPENQASM 2.0;
    include "qelib1.inc";
    qreg q2[3];
    creg c2[3];
    h q2[0];
    cx q2[0],q2[1];

We can also use the CircuitDrawer to visualize the circuit:

```python
circuit.draw(output='latex')
```

Now, we have enough to run the circuit. Let's import a backend, in this case a simulator, and run the circuit.

```python
from qiskit import Aer, execute
qasm_backend = Aer.get_backend('qasm_simulator')

job = execute(circuit, qasm_backend)

result = job.result()
result.get_counts(circuit)
```

    {'000': 1024}

Whoops! We forgot to measure. Let's do that.

```python
circuit.measure(qr, cr)
circuit.draw()
```

            ┌───┐        ┌─┐
q2_0: |0>───┤ H ├──■─────┤M├
            └───┘┌─┴─┐┌─┐└╥┘
q2_1: |0>────────┤ X ├┤M├─╫─
         ┌─┐     └───┘└╥┘ ║
q2_2: |0>┤M├───────────╫──╫─
         └╥┘           ║  ║
 c2_0: 0 ═╬════════════╬══╩═
          ║            ║
 c2_1: 0 ═╬════════════╩════
          ║
 c2_2: 0 ═╩═════════════════

```python
job = execute(circuit, qasm_backend)

result = job.result()
result.get_counts(circuit)
```

    {'000': 483, '011': 541}

Note that from version 0.6 onwards, Terra treats the rightmost qubit as qubit 0.

# Qobjs and DAGs

We actually skipped a few steps that happen under the hood during `execute`, but you'll often ignore these for algorithm development.

Execute calls `compile` to convert the circuit into a `Qobj`, which is a backend-specific object. While doing so, `compile` also calls the `transpiler`, which converts the circuit into a `Directed Acyclic Graph`(`DAG`) of gates, and optimizes the `DAG` for the target execution backend (`DAGs` are much easier to optimize than ciruits). We're going to breeze through these for now.

# Compilation Settings - a good picture of Terra’s robustness

```
def compile(circuits, backend,
            config=None, basis_gates=None,
            coupling_map=None, initial_layout=None,
            shots=1024, pass_manager=None, memory=False):
```

* circuits (QuantumCircuit or list[QuantumCircuit]): circuits to compile
* backend (BaseBackend or str): a backend for which to compile
* config (dict): dictionary of parameters (e.g. noise) used by runner - more info [here](https://github.com/Qiskit/qiskit-terra/blob/9149076d16dd98552077e389b21ed2f953d96b2e/src/qasm-simulator-cpp/README.md#config-settings)
* basis_gates (str): comma-separated basis gate set to compile to
* coupling_map (list): coupling map (perhaps custom) representing physical qubit connectivity
* initial_layout (list): user-specified mapping of logical to physical qubits
* shots (int): number of repetitions of each circuit, for sampling
* pass_manager (PassManager): a pass manger for the transpiler pipeline
* memory (bool): if True, per-shot measurement bitstrings are returned as well

# Computational Flow

Let's review our running count of Terra's core objects:
* QuantumRegister, ClassicalRegister
* QuantumCircuit
* Gate
* Backend
* DAG
* Qobj
* Job, result

And the computational flow of Terra is:
* Gates are added to QuantumCircuits
* QuantumCircuits are transpiled into DAGs, DAGs are compiled in Qobjs
* Qobjs are sent to backends
* Backends return results

So far we've run a very vanilla Bell state. Let's do some more interesting things.

```python
qr = QuantumRegister(3)
cr = ClassicalRegister(3)
circuit = QuantumCircuit(qr, cr)
circuit.ry(np.pi/2, qr[0])
circuit.h(qr[1])
circuit.cx(qr[1], qr[2])
circuit.barrier()

circuit.cx(qr[0], qr[1])
circuit.h(qr[0])
circuit.barrier()

circuit.measure(qr[0], cr[0])
circuit.measure(qr[1], cr[1])

circuit.draw()
```

                   ┌────────────┐ ░      ┌───┐ ░    ┌─┐
q3_0: |0>──────────┤ Ry(1.5708) ├─░───■──┤ H ├─░────┤M├
         ┌───┐     └────────────┘ ░ ┌─┴─┐└───┘ ░ ┌─┐└╥┘
q3_1: |0>┤ H ├──■─────────────────░─┤ X ├──────░─┤M├─╫─
         └───┘┌─┴─┐               ░ └───┘      ░ └╥┘ ║
q3_2: |0>─────┤ X ├───────────────░────────────░──╫──╫─
              └───┘               ░            ░  ║  ║
 c3_0: 0 ═════════════════════════════════════════╬══╩═
                                                  ║
 c3_1: 0 ═════════════════════════════════════════╩════

 c3_2: 0 ══════════════════════════════════════════════

Qiskit allows conditional gates in simulation, but not on the real quantum hardware (yet).

```python
circuit.z(qr[2]).c_if(cr, 1)
circuit.x(qr[2]).c_if(cr, 2)
circuit.y(qr[2]).c_if(cr, 3) # Note that ZX =iY
circuit.measure(qr[2], cr[2])
circuit.draw()
```

                   ┌────────────┐ ░      ┌───┐ ░    ┌─┐
q3_0: |0>──────────┤ Ry(1.5708) ├─░───■──┤ H ├─░────┤M├────────────────────────
         ┌───┐     └────────────┘ ░ ┌─┴─┐└───┘ ░ ┌─┐└╥┘
q3_1: |0>┤ H ├──■─────────────────░─┤ X ├──────░─┤M├─╫─────────────────────────
         └───┘┌─┴─┐               ░ └───┘      ░ └╥┘ ║ ┌─────┐┌─────┐┌─────┐┌─┐
q3_2: |0>─────┤ X ├───────────────░────────────░──╫──╫─┤  Z  ├┤  X  ├┤  Y  ├┤M├
              └───┘               ░            ░  ║  ║ ├──┴──┤├──┴──┤├──┴──┤└╥┘
 c3_0: 0 ═════════════════════════════════════════╬══╩═╡     ╞╡     ╞╡     ╞═╬═
                                                  ║    │     ││     ││     │ ║
 c3_1: 0 ═════════════════════════════════════════╩════╡ = 1 ╞╡ = 2 ╞╡ = 3 ╞═╬═
                                                       │     ││     ││     │ ║
 c3_2: 0 ══════════════════════════════════════════════╡     ╞╡     ╞╡     ╞═╩═
                                                       └─────┘└─────┘└─────┘

You can find a more in depth guide to teleportation in Anna Phan's notebook on the topic, [here](https://github.com/Qiskit/qiskit-tutorial/blob/master/community/terra/qis_intro/teleportation_superdensecoding.ipynb).

Let's see what pops out.

```python
job = execute(circuit, qasm_backend)

result = job.result()
result.get_counts(circuit)
```

    {'001': 128,
     '101': 124,
     '110': 132,
     '111': 137,
     '010': 119,
     '000': 138,
     '100': 115,
     '011': 131}

Maybe this isn't accurate enough to tell what we encoded on qubit 1. Let's increase the number of shots.

```python
job = execute(circuit, qasm_backend, shots = 10000)

result = job.result()
result.get_counts(circuit)
```

    {'001': 1205,
     '101': 1224,
     '110': 1289,
     '111': 1219,
     '010': 1315,
     '000': 1245,
     '100': 1208,
     '011': 1295}

Now let's visualize those results as a histogram

```python
from qiskit.tools.visualization import plot_histogram
```

```python
plot_histogram(result.get_counts(circuit))
```

And now, calculating the percentage of shots with |0> measured on qubit 2 (but qubit 0 in our results).

```python
counts = result.get_counts(circuit)
qubit3_p0 = sum([v for k, v in counts.items() if k[0]=='0'])/10000
qubit3_p0
```

    0.506

Our probability of finding 0 is 50%, which is correct, because Ry(pi/2) should put qubit 0 in the |+> state.

But how do we know that we're not in the |-> state, or any other state along the equator of the Bloch sphere? Let's use a Hadamard to see whether our phase is correct. We'll need to delete the final measurement and add the Hadamard to do this.

```python
del(circuit.data[-1])
circuit.h(qr[2])
circuit.measure(qr[2], cr[2])
circuit.draw(line_length=200)
```

                   ┌────────────┐ ░      ┌───┐ ░    ┌─┐
q3_0: |0>──────────┤ Ry(1.5708) ├─░───■──┤ H ├─░────┤M├─────────────────────────────
         ┌───┐     └────────────┘ ░ ┌─┴─┐└───┘ ░ ┌─┐└╥┘
q3_1: |0>┤ H ├──■─────────────────░─┤ X ├──────░─┤M├─╫──────────────────────────────
         └───┘┌─┴─┐               ░ └───┘      ░ └╥┘ ║ ┌─────┐┌─────┐┌─────┐┌───┐┌─┐
q3_2: |0>─────┤ X ├───────────────░────────────░──╫──╫─┤  Z  ├┤  X  ├┤  Y  ├┤ H ├┤M├
              └───┘               ░            ░  ║  ║ ├──┴──┤├──┴──┤├──┴──┤└───┘└╥┘
 c3_0: 0 ═════════════════════════════════════════╬══╩═╡     ╞╡     ╞╡     ╞══════╬═
                                                  ║    │     ││     ││     │      ║
 c3_1: 0 ═════════════════════════════════════════╩════╡ = 1 ╞╡ = 2 ╞╡ = 3 ╞══════╬═
                                                       │     ││     ││     │      ║
 c3_2: 0 ══════════════════════════════════════════════╡     ╞╡     ╞╡     ╞══════╩═
                                                       └─────┘└─────┘└─────┘

```python
shots = 100000
job = execute(circuit, qasm_backend, shots=shots)

result = job.result()
result.get_counts(circuit)
counts = result.get_counts(circuit)
qubit3_p0 = sum([v for k, v in counts.items() if k[0]=='0'])/100000
qubit3_p0
```

    1.0

Looks like our final state is indeed |+>, because our P(|0>) = 100%.

# Example: Phase Estimation

Let's see if we can work out a small phase estimation function, and sanity check it on simulators before trying it on the quantum hardware.

- We'll start with a QFT, which comes directly from Terra.
- We're going to use the Pauli X as our unitary, which simplifies things a lot
- I'll then define a function to give me my circuit

```python
def qft(circ, q, n):
    """n-qubit QFT on q in circ."""
    for j in range(n):
        for k in range(j):
            circ.cu1(np.pi / float(2**(j - k)), q[j], q[k])
        circ.h(q[j])
```

```python
#Takes in a circuit with an operator on qubit n and appends the qpe circuit
def x_qpe(circ, q, n):
    for i in range(n-1):
        circ.h(q[i])
    for j in range(0, n-1, 2): # Only place a CX^n on every other qubit, because CX^n = I for n even
        circ.cx(q[j], q[n-1])
    circ.barrier()
    qft(circ, q, n-1)
```

Play around with the ancilla number, the operator, the initial state, etc., see what happens!

```python
# n-1 is the number of ancilla
n = 4
qr = QuantumRegister(n)
cr = ClassicalRegister(n)
circuit = QuantumCircuit(qr, cr)
circuit.rx(np.pi/2, qr[n-1])
circuit.barrier()
x_qpe(circuit, qr, n)
```

```python
circuit.draw(line_length=200)
```

                        ░           ┌───┐           ░ ┌───┐
q4_0: |0>───────────────░───────────┤ H ├──■────────░─┤ H ├─■────────■──────────────────────────
                        ░      ┌───┐└───┘  │        ░ └───┘ │1.5708  │       ┌───┐
q4_1: |0>───────────────░──────┤ H ├───────┼────────░───────■────────┼───────┤ H ├─■────────────
                        ░ ┌───┐└───┘       │        ░                │0.7854 └───┘ │1.5708 ┌───┐
q4_2: |0>───────────────░─┤ H ├────────────┼────■───░────────────────■─────────────■───────┤ H ├
         ┌────────────┐ ░ └───┘          ┌─┴─┐┌─┴─┐ ░                                      └───┘
q4_3: |0>┤ Rx(1.5708) ├─░────────────────┤ X ├┤ X ├─░───────────────────────────────────────────
         └────────────┘ ░                └───┘└───┘ ░
 c4_0: 0 ═══════════════════════════════════════════════════════════════════════════════════════

 c4_1: 0 ═══════════════════════════════════════════════════════════════════════════════════════

 c4_2: 0 ═══════════════════════════════════════════════════════════════════════════════════════

 c4_3: 0 ═══════════════════════════════════════════════════════════════════════════════════════

Now that we have our basic algorithm, let's start trying to test and validate it in **quantum execution environments**.

# Interlude: Backends

Qiskit offers connectors into execution `providers`, each with several `backends`:

* BasicAer: Terra's built-in suite of pure-python simulators
* Aer: Qiskit's suite of high-performance simulators
* IBMQ: IBM's Quantum devices, and an HPC simulator

```python
from qiskit import IBMQ, Aer, BasicAer
```

# Simuators

* qasm_simulator - a shot-based simulator
  * Input: a Qobj and execution config
  * Output: a results object containing a dictionary with basis states and shots per state
    * {‘00’: 425, ‘01’: 267, ‘11’: 90}
  * You can specify a random seed so the probabilistic measurement and noise stays the same

# Simulators
* statevector_simulator - This is the qasm_simulator with a snapshot at the end
  * Returns result object containing a dictionary of basis states with complex amplitudes for each
* unitary_simulator - Returns a matrix of your circuit!
* ibmq_qasm_simulator - a public simulator on an HPC machine run by IBM (Note, this is under the IBMQ `provider`)

# Aer vs BasicAer
- Aer is fast
- BasicAer is slow

* `aer.noise` - includes sophisticated noise models, which you can find more info about [here](https://qiskit.org/documentation/aer/device_noise_simulation.html)
* `pip install qiskit` comes with binaries for many platforms so you shouldn’t need to compile cpp (but if you do, check out the [Terra contributing file on github](https://github.com/Qiskit/qiskit-terra/blob/master/.github/CONTRIBUTING.rst) for make instructions.)

```python
backend = Aer.get_backend("qasm_simulator")
print(backend)
```

    qasm_simulator

## Ok - Back to Phase Estimation:

```python
qasm_backend = Aer.get_backend('qasm_simulator')
```

Don't forget to measure! Recall that we don't measure our |u> qubit.

```python
circuit.barrier()
for i in range(n-1):
    circuit.measure(qr[i], cr[i])
```

```python
shots = 10000
job = execute(circuit, qasm_backend, shots = shots)

result = job.result()
counts = result.get_counts(circuit)
```

```python
plot_histogram(counts)
```

We seem to be getting an ok answer, so let's try running on the **Quantum hardware**.

# The IBMQ Provider: Executing on Quantum Hardware

To do this, you'll either get your Q Network API token and URL from the [console](https://q-console.mybluemix.net/) (if you are are a member of the Q Network), or you'll need to get an IBM Q Experience API token from the [Q Experience accounts page](https://quantumexperience.ng.bluemix.net/qx/account/advanced).

```python
# IBMQ.enable_account('<key>')
# uncomment this ^^^ and insert your API key. Add a 'url' argument for Q Network users

# Or you can use:
IBMQ.load_accounts()

print("Available backends:")
IBMQ.backends(filters= lambda b: b.hub is None)
```

    Available backends:

    [<IBMQBackend('ibmqx2') from IBMQ()>,
     <IBMQBackend('ibmqx4') from IBMQ()>,
     <IBMQBackend('ibmq_16_melbourne') from IBMQ()>,
     <IBMQBackend('ibmq_4_atlantis') from IBMQ()>,
     <IBMQBackend('ibmq_local_test_mock') from IBMQ()>,
     <IBMQBackend('ibmq_qasm_simulator') from IBMQ()>]

```python
q_backend = IBMQ.get_backend('ibmqx4')
```

## Back again to our circuit:

```python
shots = 8192        # Number of shots to run the program (experiment); maximum is 8192 shots.
job_exp = execute(circuit, q_backend, shots = shots)
```

```python
# Check the job status
job_exp.status()
```

    <JobStatus.QUEUED: 'job is queued'>

FYI, you can also retrieve an old job by its job_id.

```python
jobID = job_exp.job_id()

print('JOB ID: {}'.format(jobID))

job_get=q_backend.retrieve_job(jobID)
job_get.result().get_counts(circuit)
```

    JOB ID: 5c72d443c426dc0062a3525c

    {'0110': 343,
     '0000': 2415,
     '0010': 447,
     '0101': 1604,
     '0100': 1792,
     '0011': 566,
     '0111': 412,
     '0001': 613}

Note that I increase the timeout and wait time considerably - this is often necessary. The defaults can be too short.

```python
# We recommend increasing the timeout to 30 minutes to avoid timeout errors when the queue is long.
result_real = job_exp.result(timeout=3600, wait=5)
counts = result_real.get_counts(circuit)
plot_histogram(counts)
```

# Visualizing Devices, and Pulling Device Info

- Terra has some neat built-in Jupyter magics for browsing device information, such as:
    - qubit error
    - job queues for public devices
    - coupling maps

- More info [here](https://github.com/Qiskit/qiskit-tutorials/blob/master/qiskit/jupyter/jupyter_backend_tools.ipynb).

You can view the raw properties data for any backend like this:

```python
q_backend.properties()
```

    BackendProperties(backend_name='ibmq_16_melbourne', backend_version='1.0.0', gates=[Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[0]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.001657744694001706)], qubits=[0]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.003315489388003412)], qubits=[0]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[1]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.007657438332263289)], qubits=[1]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.015314876664526578)], qubits=[1]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[2]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.003534263772314583)], qubits=[2]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.007068527544629166)], qubits=[2]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[3]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0011586761277807)], qubits=[3]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0023173522555614)], qubits=[3]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[4]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.001929259572122588)], qubits=[4]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.003858519144245176)], qubits=[4]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[5]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.002771458801925586)], qubits=[5]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.005542917603851172)], qubits=[5]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[6]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0011505196550350427)], qubits=[6]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0023010393100700854)], qubits=[6]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[7]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.001926840787433881)], qubits=[7]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.003853681574867762)], qubits=[7]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[8]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0021222280974397822)], qubits=[8]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0042444561948795645)], qubits=[8]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[9]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0035423748381594455)], qubits=[9]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.007084749676318891)], qubits=[9]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[10]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.001913741446806283)], qubits=[10]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.003827482893612566)], qubits=[10]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[11]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.00253643788613922)], qubits=[11]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.00507287577227844)], qubits=[11]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[12]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0033615212439866426)], qubits=[12]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.006723042487973285)], qubits=[12]), Gate(gate='u1', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.0)], qubits=[13]), Gate(gate='u2', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.005804811579962099)], qubits=[13]), Gate(gate='u3', parameters=[Nduv(date=datetime.datetime(2019, 2, 25, 7, 31, 1, tzinfo=tzutc()), name='gate_error', unit='', value=0.011609623159924198)], qubits=[13]), Gate(gate='cx', name='CX1_0', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 26, 7, tzinfo=tzutc()), name='gate_error', unit='', value=0.05827277445210416)], qubits=[1, 0]), Gate(gate='cx', name='CX1_2', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 29, 20, tzinfo=tzutc()), name='gate_error', unit='', value=0.03846091469791432)], qubits=[1, 2]), Gate(gate='cx', name='CX2_3', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 32, 52, tzinfo=tzutc()), name='gate_error', unit='', value=0.04739205412562961)], qubits=[2, 3]), Gate(gate='cx', name='CX4_3', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 36, 11, tzinfo=tzutc()), name='gate_error', unit='', value=0.02860996641163832)], qubits=[4, 3]), Gate(gate='cx', name='CX4_10', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 39, 28, tzinfo=tzutc()), name='gate_error', unit='', value=0.031374925181012675)], qubits=[4, 10]), Gate(gate='cx', name='CX5_4', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 42, 44, tzinfo=tzutc()), name='gate_error', unit='', value=0.051453686904960494)], qubits=[5, 4]), Gate(gate='cx', name='CX5_6', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 46, 7, tzinfo=tzutc()), name='gate_error', unit='', value=0.056018111963786504)], qubits=[5, 6]), Gate(gate='cx', name='CX5_9', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 50, 2, tzinfo=tzutc()), name='gate_error', unit='', value=0.1661321750914002)], qubits=[5, 9]), Gate(gate='cx', name='CX6_8', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 53, 25, tzinfo=tzutc()), name='gate_error', unit='', value=0.03190285369377577)], qubits=[6, 8]), Gate(gate='cx', name='CX7_8', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 8, 56, 52, tzinfo=tzutc()), name='gate_error', unit='', value=0.030998414283986614)], qubits=[7, 8]), Gate(gate='cx', name='CX9_8', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 9, 0, 7, tzinfo=tzutc()), name='gate_error', unit='', value=0.054620229393257336)], qubits=[9, 8]), Gate(gate='cx', name='CX9_10', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 9, 4, 10, tzinfo=tzutc()), name='gate_error', unit='', value=0.09468962960456251)], qubits=[9, 10]), Gate(gate='cx', name='CX11_3', parameters=[Nduv(date=datetime.datetime(2019, 2, 17, 9, 34, 13, tzinfo=tzutc()), name='gate_error', unit='', value=0.08470997965114974)], qubits=[11, 3]), Gate(gate='cx', name='CX11_10', parameters=[Nduv(date=datetime.datetime(2019, 2, 17, 9, 27, 43, tzinfo=tzutc()), name='gate_error', unit='', value=0.035970559672938496)], qubits=[11, 10]), Gate(gate='cx', name='CX11_12', parameters=[Nduv(date=datetime.datetime(2019, 2, 17, 9, 30, 59, tzinfo=tzutc()), name='gate_error', unit='', value=0.03696622318009893)], qubits=[11, 12]), Gate(gate='cx', name='CX12_2', parameters=[Nduv(date=datetime.datetime(2019, 2, 23, 9, 22, 46, tzinfo=tzutc()), name='gate_error', unit='', value=0.08406631659128154)], qubits=[12, 2]), Gate(gate='cx', name='CX13_1', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 9, 21, 56, tzinfo=tzutc()), name='gate_error', unit='', value=0.24332932218355438)], qubits=[13, 1]), Gate(gate='cx', name='CX13_12', parameters=[Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='gate_error', unit='', value=0.07046310537371045)], qubits=[13, 12])], general=[], last_update_date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), qubits=[[Nduv(date=datetime.datetime(2019, 2, 23, 7, 31, 25, tzinfo=tzutc()), name='T1', unit='µs', value=58.03116760943428), Nduv(date=datetime.datetime(2019, 2, 24, 7, 29, 49, tzinfo=tzutc()), name='T2', unit='µs', value=14.724030341325745), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.10007609747442), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.06919999999999993)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=59.83234286514375), Nduv(date=datetime.datetime(2019, 2, 24, 7, 30, 51, tzinfo=tzutc()), name='T2', unit='µs', value=107.86811061878761), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.23868683687039), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.14939999999999998)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=78.83230634480451), Nduv(date=datetime.datetime(2019, 2, 24, 7, 31, 51, tzinfo=tzutc()), name='T2', unit='µs', value=102.16011660048183), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.032936880275877), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.04349999999999998)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=80.71074437501859), Nduv(date=datetime.datetime(2019, 2, 24, 7, 32, 51, tzinfo=tzutc()), name='T2', unit='µs', value=93.47298373503907), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.896169943144527), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.2086)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=44.873050065955645), Nduv(date=datetime.datetime(2019, 2, 24, 7, 29, 49, tzinfo=tzutc()), name='T2', unit='µs', value=31.237964501113378), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.027220338176452), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.023900000000000032)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=22.34664027378627), Nduv(date=datetime.datetime(2019, 2, 24, 7, 30, 51, tzinfo=tzutc()), name='T2', unit='µs', value=40.959721679585265), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.067154635274023), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.09240000000000004)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=67.06584621261001), Nduv(date=datetime.datetime(2019, 2, 24, 7, 31, 51, tzinfo=tzutc()), name='T2', unit='µs', value=57.56186597322083), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.923808354943549), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.03509999999999991)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=37.321197970352195), Nduv(date=datetime.datetime(2019, 2, 24, 7, 32, 51, tzinfo=tzutc()), name='T2', unit='µs', value=35.41966924608856), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.974518925104361), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.04590000000000005)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=52.68491800085742), Nduv(date=datetime.datetime(2019, 2, 24, 7, 29, 49, tzinfo=tzutc()), name='T2', unit='µs', value=68.95434334639211), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.739784526128049), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.04410000000000003)], [Nduv(date=datetime.datetime(2019, 2, 20, 7, 23, 13, tzinfo=tzutc()), name='T1', unit='µs', value=33.78727044438043), Nduv(date=datetime.datetime(2019, 2, 24, 7, 31, 51, tzinfo=tzutc()), name='T2', unit='µs', value=29.148849628536166), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.963352994843123), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.1129)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=56.6604397300763), Nduv(date=datetime.datetime(2019, 2, 24, 7, 30, 51, tzinfo=tzutc()), name='T2', unit='µs', value=71.82073067161548), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.945087549654646), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.04590000000000005)], [Nduv(date=datetime.datetime(2019, 2, 22, 7, 35, 6, tzinfo=tzutc()), name='T1', unit='µs', value=5.62745350040824), Nduv(date=datetime.datetime(2019, 2, 22, 7, 38, 20, tzinfo=tzutc()), name='T2', unit='µs', value=21.6457911319178), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=5.005975975723903), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.38339999999999996)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=66.84149299149782), Nduv(date=datetime.datetime(2019, 2, 24, 7, 30, 51, tzinfo=tzutc()), name='T2', unit='µs', value=86.94911909117845), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.760146191276317), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.03410000000000002)], [Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 35, tzinfo=tzutc()), name='T1', unit='µs', value=25.15747019319897), Nduv(date=datetime.datetime(2019, 2, 24, 7, 29, 49, tzinfo=tzutc()), name='T2', unit='µs', value=30.64437222118539), Nduv(date=datetime.datetime(2019, 2, 24, 9, 25, 54, tzinfo=tzutc()), name='frequency', unit='GHz', value=4.968480671260165), Nduv(date=datetime.datetime(2019, 2, 24, 7, 28, 11, tzinfo=tzutc()), name='readout_error', unit='', value=0.1149)]])

# A Prettier Device Overview

```python
from qiskit.tools.jupyter import *
```

# Diving into a Specific Backend

```python
%qiskit_backend_monitor q_backend
```

    VBox(children=(HTML(value="<h1 style='color:#ffffff;background-color:#000000;padding-top: 1%;padding-bottom: 1…

# Noise Modelling
- The above contains enough information to simulate the approximate noise properties of the device. Chris will talk more about this later.

# Tips and tricks

* Put many circuits into a single execution!
  * Simulators will (generally) execute these in parallel
  * Quantum Hardware does a lot of calibration for each new job, so if you send 100 jobs it will generally take 100x as long as one job with 100 circuits, even if the circuits are completely different!
* Increase your timeout when waiting for results! Default is 30 seconds, better to set to 1800 (30 mins)
  * See notebook above (ctrl-f for 'timeout')
* Use an IDE!! For example Pycharm, VisualStudio, etc. Being able to step through the code is critical!

* Look at the debug log messages. There is a ton of important info in there. See notebook here
  * Even better, save them to a file.

```python
import logging
logging.getLogger('qiskit').setLevel(logging.DEBUG)
```

Redirecting logs to a file:

```
# Redirecting debug logs to a file (can't be done in colab):
    loggerc = logging.getLogger('qiskit_aqua_chemistry')
    loggerc.setLevel(logging.DEBUG)
    loggera = logging.getLogger('qiskit_aqua')
    loggera.setLevel(logging.DEBUG)
    loggerq = logging.getLogger('qiskit')
    loggerq.setLevel(logging.DEBUG)
    formatter = logging.Formatter(fmt='%(asctime)s %(levelname)-8s %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
    hdlr = logging.FileHandler(outdir + log_file_name, mode='w')
    hdlr.setFormatter(formatter)
    loggerc.addHandler(hdlr)
    loggera.addHandler(hdlr)
    loggerq.addHandler(hdlr)
    print('\nlog file: {}'.format(outdir + log_file_name))
# <build, execute, etc.>
# close up handlers
    loggerc.removeHandler(hdlr)
    loggera.removeHandler(hdlr)
    loggerq.removeHandler(hdlr)
    hdlr.close()
```

## Learning more

The [qiskit-tutorial](https://github.com/Qiskit/qiskit-tutorial) repo on Github has dozens of thoughtful and sophisticated tutorials.
- We highly recommend going through both the “[qiskit/](https://github.com/Qiskit/qiskit-tutorial/tree/master/qiskit)” directory and the “[community/](https://github.com/Qiskit/qiskit-tutorial/tree/master/community)” directory.
- We learn new things every time I look through them, and reference them regularly.
- If you have any questions come find us or one of the other IBM Q members!

# Review - Quantum Algorithm Building Blocks

Four major building blocks of quantum algorithms:

* Quantum Fourier Transform
   * Period-finding and phase↔norm swapping
   * Speedup from $O(2^n)$ to $O(n^2)$
   * E.g. Shor’s algorithm, Quantum Phase Estimation
* Hamiltonian Evolution
   * Applying a Hamiltonian to an initial state over an arbitrary time period
   * Exponential speedup (mostly, with complicated factors)
   * E.g. HHL, QAOA, QPE
* Unstructured Search (Grover’s)
   * Search for a state (string) exhibiting a binary condition (e.g. satisfy my 3SAT problem…)
   * Speedup of O(√n)
* Variational Optimization
   * Prepare a quantum state using a parameterized short circuit, use a classical optimizer to optimize parameters toward some desired quality evaluated on the QC (e.g. binary classification)
   * Speedups vary, usually no guaranteed speedup, but good for NISQ machines
   * E.g. VQE, VSVM, QAOA

# Quantum Fourier Transform

We've used it above and it is straightforward to implement, but it is not very intuitive as a building block, and I recommend the [tutorial dedicated to it](https://github.com/Qiskit/qiskit-tutorial/blob/master/community/terra/qis_adv/fourier_transform.ipynb) by Anna Phan. I also highly recommmend 3Blue1Brown's video on the [continuous fourier transform](https://www.youtube.com/watch?v=spUNpyF58BY).

# Hamiltonian Evolution

This is trickier, we're working on it. For now, the best way to learn about this in Qiskit is in the [Aqua operator class](https://github.com/Qiskit/aqua/blob/master/qiskit_aqua/operator.py#L1119), which includes lots of evolution logic.

# Grover’s Algorithm

Pretty straightforward in Terra. See [this notebook](https://github.com/Qiskit/qiskit-tutorial/blob/master/community/algorithms/grover_algorithm.ipynb) by Giacomo Nannicini and Rudy Raymond.

# Variational Optimization

This also doesn't have a standalone tutorial, but the [Aqua VQE](https://github.com/Qiskit/aqua/blob/master/qiskit_aqua/algorithms/adaptive/vqe/vqe.py) is a straightforward, well engineered example of variational optimization. The [Aqua Variational SVM](https://github.com/Qiskit/aqua/blob/master/qiskit_aqua/algorithms/adaptive/qsvm/qsvm_variational.py) is also a good example.

# Learning More - A Longer Course

[This doc](https://docs.google.com/document/d/1WoUQky2NXdbrdGkxaUA28VE7W3fryTQG6ezn8Fw-l4E/edit) details a longer course to fluency in Quantum Programming.

# Time Permitting: Transpilation and the DAGCircuit

The transpiler is the workhorse of Terra. It’s how we keep circuits backend agnostic and compilable for arbitrary quantum hardware. The transpiler in Terra 0.6 was not transparent or extensible enough for increasingly sophisticated transpilation methods, so we tore it down and rewrote it to be much more robust.

The transpiler now transpiles circuits into circuits, rather than into DAGCircuits. This is much more transparent, and allows the end user to view and understand what individual transpiler passes are doing to their circuit. Here's a sample circuit that won't fit nicely on IBM's hardware (our QPE circuit had nearest neighbor connections, so these qubit remappers won't do much):

```python
from qiskit.transpiler import PassManager
from qiskit.transpiler.passes import BasicSwap, CXCancellation, LookaheadSwap, StochasticSwap
from qiskit.transpiler import transpile
from qiskit.mapper import CouplingMap
```

```python
qr = QuantumRegister(7, 'q')
qr = QuantumRegister(7, 'q')
tpl_circuit = QuantumCircuit(qr)
tpl_circuit.h(qr[3])
tpl_circuit.cx(qr[0], qr[6])
tpl_circuit.cx(qr[6], qr[0])
tpl_circuit.cx(qr[0], qr[1])
tpl_circuit.cx(qr[3], qr[1])
tpl_circuit.cx(qr[3], qr[0])
tpl_circuit.draw()
```

# Swap mapping
The most naive thing we can do is simply move qubits around greedily with swaps. Let’s see how the BasicSwap pass does here:

```python
coupling = [[0, 1], [1, 2], [2, 3], [3, 4], [4, 5], [5, 6]]

simulator = BasicAer.get_backend('qasm_simulator')
coupling_map = CouplingMap(couplinglist=coupling)
pass_manager = PassManager()
pass_manager.append([BasicSwap(coupling_map=coupling_map)])
basic_circ = transpile(tpl_circuit, simulator, pass_manager=pass_manager)
basic_circ.draw()
```

Not great. Let’s try Sven Jandura's LookaheadSwap, submitted for the 2018 QISKit
Developer Challenge. Sven’s swap pass was merged into Terra, and we will have two more passess from other winners of the Qiskit Developer Challenge soon! We’re constructing a diverse set of passes, many user contributed, to meet the wide-ranging needs and mapping scenarios of circuits in the wild.

```python
pass_manager = PassManager()
pass_manager.append([LookaheadSwap(coupling_map=coupling_map)])
lookahead_circ = transpile(tpl_circuit, simulator, pass_manager=pass_manager)
lookahead_circ.draw()
```

Better! One more try with the StochasticSwap:

```python
pass_manager = PassManager()
pass_manager.append([StochasticSwap(coupling_map=coupling_map)])
stoch_circ = transpile(tpl_circuit, simulator, pass_manager=pass_manager)
stoch_circ.draw()
```

Even better, but still more room to go. Right now this all happens behind the scenes for many users, but we hope that these tools make digging into transpilation much more accessible to those attempting to squeeze as much performance as possible out of their experiments on hardware.

# Transpiling for Real Hardware

Finally, let's see what the default transpiler does to our circuit to be able to run on a real backend. Note that this will include unrolling into the {U, CX} basis, including the swaps.

```python
tok_circ = transpile(tpl_circuit, backend=q_backend)
tok_circ.draw(line_length=200)
```

# Modelling Noise in Aer Based on a Device's Properties

Now that you have these properties, you might want to create a noise model for the qasm_simulator which closely resembles this device. A new feature in Aer allows you to do just that. Much of the content below is drawn from [this notebook](https://github.com/Qiskit/qiskit-tutorials/blob/master/qiskit/aer/device_noise_simulation.ipynb).

First, let's pick a backend:

```python
IBMQ.backends(filters= lambda b: b.hub is None)
```

    [<IBMQBackend('ibmqx2') from IBMQ()>,
     <IBMQBackend('ibmqx4') from IBMQ()>,
     <IBMQBackend('ibmq_16_melbourne') from IBMQ()>,
     <IBMQBackend('ibmq_4_atlantis') from IBMQ()>,
     <IBMQBackend('ibmq_local_test_mock') from IBMQ()>,
     <IBMQBackend('ibmq_qasm_simulator') from IBMQ()>]

Now, we need to pull the device information:

```python
device = IBMQ.get_backend('ibmq_16_melbourne')
properties = device.properties()
coupling_map = device.configuration().coupling_map
```

Now, let's construct the device noise model.

Note: The devices don't currently provide gate times, so we will manually provide them for the gates we are interested in using the optional gate_times argument for basic_device_noise_model.

```python
from qiskit.providers.aer import noise
```

```python
# List of gate times for ibmq_14_melbourne device
# Note that the None parameter for u1, u2, u3 is because gate
# times are the same for all qubits
gate_times = [
    ('u1', None, 0), ('u2', None, 50), ('u3', None, 100),
    ('cx', [1, 0], 678), # I can add gate times for specific couplings, or all couplings
    ('cx', [], 600)
]

# Construct the noise model from backend properties
# and custom gate times
noise_model = noise.device.\
              basic_device_noise_model(properties,
                                       gate_times=gate_times)
print(noise_model)
```

    NoiseModel:
      Instructions with noise: ['measure', 'u2', 'cx', 'u3']
      Specific qubit errors: [('u2', [0]), ('u2', [1]), ('u2', [2]), ('u2', [3]), ('u2', [4]), ('u2', [5]), ('u2', [6]), ('u2', [7]), ('u2', [8]), ('u2', [9]), ('u2', [10]), ('u2', [11]), ('u2', [12]), ('u2', [13]), ('u3', [0]), ('u3', [1]), ('u3', [2]), ('u3', [3]), ('u3', [4]), ('u3', [5]), ('u3', [6]), ('u3', [7]), ('u3', [8]), ('u3', [9]), ('u3', [10]), ('u3', [11]), ('u3', [12]), ('u3', [13]), ('cx', [1, 0]), ('cx', [1, 2]), ('cx', [2, 3]), ('cx', [4, 3]), ('cx', [4, 10]), ('cx', [5, 4]), ('cx', [5, 6]), ('cx', [5, 9]), ('cx', [6, 8]), ('cx', [7, 8]), ('cx', [9, 8]), ('cx', [9, 10]), ('cx', [11, 3]), ('cx', [11, 10]), ('cx', [11, 12]), ('cx', [12, 2]), ('cx', [13, 1]), ('cx', [13, 12]), ('measure', [0]), ('measure', [1]), ('measure', [2]), ('measure', [3]), ('measure', [4]), ('measure', [5]), ('measure', [6]), ('measure', [7]), ('measure', [8]), ('measure', [9]), ('measure', [10]), ('measure', [11]), ('measure', [12]), ('measure', [13])]

Now, let's use this model to simulate our QPE circuit. Note, this can take a few minutes to run.

```python
shots = 1000
basis_gates = noise_model.basis_gates

# Select the QasmSimulator from the Aer provider
simulator = Aer.get_backend('qasm_simulator')

# Execute noisy simulation and get counts
result_noise = execute(circuit, simulator,
                       shots=shots,
                       noise_model=noise_model,
                       coupling_map=coupling_map,
                       basis_gates=basis_gates).result()
counts_noise = result_noise.get_counts(circuit)
plot_histogram(counts_noise, title="Counts for QPE circuit with depolarizing noise model")
```

```python
# And now our phase estimate:
angles = np.array([v*int(k, 2) for k, v in counts_noise.items()]) / shots / 2**(n-1)
res = 2*sum(angles)
np.around(res, decimals=5)
```

This is actually worse than we get from the device! More tuning to do here.


# **CS224W - Colab 3**

In Colab 2 we constructed GNN models by using PyTorch Geometric's built in GCN layer, `GCNConv`. In this Colab we will go a step deeper and implement the **GraphSAGE** ([Hamilton et al. (2017)](https://arxiv.org/abs/1706.02216)) layer directly. Then we will run our models on the CORA dataset, which is a standard citation network benchmark dataset.

**Note**: Make sure to **sequentially run all the cells in each section** so that the intermediate variables / packages will carry over to the next cell

Have fun and good luck on Colab 3 :)

# Device
We recommend using a GPU for this Colab.

Please click `Runtime` and then `Change runtime type`. Then set the `hardware accelerator` to **GPU**.

## Installation

```python
# Install torch geometric
import os
if 'IS_GRADESCOPE_ENV' not in os.environ:
  !pip install torch-scatter -f https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html
  !pip install torch-sparse -f https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html
  !pip install torch-geometric
  !pip install -q git+https://github.com/snap-stanford/deepsnap.git
```

    Looking in links: https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html
    Collecting torch-scatter
      Downloading https://data.pyg.org/whl/torch-1.9.0%2Bcu111/torch_scatter-2.0.8-cp37-cp37m-linux_x86_64.whl (10.4 MB)
         |████████████████████████████████| 10.4 MB 12.5 MB/s
    Installing collected packages: torch-scatter
    Successfully installed torch-scatter-2.0.8
    Looking in links: https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html
    Collecting torch-sparse
      Downloading https://data.pyg.org/whl/torch-1.9.0%2Bcu111/torch_sparse-0.6.12-cp37-cp37m-linux_x86_64.whl (3.7 MB)
         |████████████████████████████████| 3.7 MB 12.2 MB/s
    Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from torch-sparse) (1.4.1)
    Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.7/dist-packages (from scipy->torch-sparse) (1.19.5)
    Installing collected packages: torch-sparse
    Successfully installed torch-sparse-0.6.12
    Collecting torch-geometric
      Downloading torch_geometric-2.0.1.tar.gz (308 kB)
         |████████████████████████████████| 308 kB 14.4 MB/s
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.19.5)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (4.62.3)
    Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.4.1)
    Requirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.6.3)
    Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (0.22.2.post1)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.23.0)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.1.5)
    Collecting rdflib
      Downloading rdflib-6.0.2-py3-none-any.whl (407 kB)
         |████████████████████████████████| 407 kB 63.7 MB/s
    Requirement already satisfied: googledrivedownloader in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (0.4)
    Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.11.3)
    Requirement already satisfied: pyparsing in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.4.7)
    Collecting yacs
      Downloading yacs-0.1.8-py3-none-any.whl (14 kB)
    Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (3.13)
    Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->torch-geometric) (2.0.1)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->torch-geometric) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->torch-geometric) (2.8.2)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->torch-geometric) (1.15.0)
    Collecting isodate
      Downloading isodate-0.6.0-py2.py3-none-any.whl (45 kB)
         |████████████████████████████████| 45 kB 3.5 MB/s
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from rdflib->torch-geometric) (57.4.0)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (3.0.4)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (2021.5.30)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->torch-geometric) (1.0.1)
    Building wheels for collected packages: torch-geometric
      Building wheel for torch-geometric (setup.py) ... done
      Created wheel for torch-geometric: filename=torch_geometric-2.0.1-py3-none-any.whl size=513822 sha256=0508097be7556c0ef204e9968e6883b06a3417e5d4502507cd4808897763bf9b
      Stored in directory: /root/.cache/pip/wheels/78/3d/42/20589db73c66b5109fb93a0c5743edfd6ab5ca820a52afacfc
    Successfully built torch-geometric
    Installing collected packages: isodate, yacs, rdflib, torch-geometric
    Successfully installed isodate-0.6.0 rdflib-6.0.2 torch-geometric-2.0.1 yacs-0.1.8
      Building wheel for deepsnap (setup.py) ... done

```python
import torch_geometric
torch_geometric.__version__
```

    '2.0.1'

# 1) GNN Layers

## Implementing Layer Modules

In Colab 2, we implemented a GCN model for node and graph classification tasks. However, for that notebook we took advantage of PyG's built in GCN module. For Colab 3, we provide a build upon a general Graph Neural Network Stack, into which we will be able to plugin our own module implementations: GraphSAGE and GAT.

We will then use our layer implemenations to complete node classification on the CORA dataset, a standard citation network benchmark. In this dataset, nodes correspond to documents and edges correspond to undirected citations. Each node or document in the graph is assigned a class label and features based on the documents binarized bag-of-words representation. Specifically, the Cora graph has 2708 nodes, 5429 edges, 7 prediction classes, and 1433 features per node.

## GNN Stack Module

Below is the implementation of a general GNN stack, where we can plugin any GNN layer, such as **GraphSage**, **GAT**, etc. This module is provided for you. Your implementations of the **GraphSage** and **GAT** (Colab 4) layers will function as components in the GNNStack Module.

```python
import torch
import torch_scatter
import torch.nn as nn
import torch.nn.functional as F

import torch_geometric.nn as pyg_nn
import torch_geometric.utils as pyg_utils

from torch import Tensor
from typing import Union, Tuple, Optional
from torch_geometric.typing import (OptPairTensor, Adj, Size, NoneType,
                                    OptTensor)

from torch.nn import Parameter, Linear
from torch_sparse import SparseTensor, set_diag
from torch_geometric.nn.conv import MessagePassing
from torch_geometric.utils import remove_self_loops, add_self_loops, softmax

class GNNStack(torch.nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim, args, emb=False):
        super(GNNStack, self).__init__()
        conv_model = self.build_conv_model(args.model_type)
        self.convs = nn.ModuleList()
        self.convs.append(conv_model(input_dim, hidden_dim))
        assert (args.num_layers >= 1), 'Number of layers is not >=1'
        for l in range(args.num_layers-1):
            self.convs.append(conv_model(args.heads * hidden_dim, hidden_dim))

        # post-message-passing
        self.post_mp = nn.Sequential(
            nn.Linear(args.heads * hidden_dim, hidden_dim), nn.Dropout(args.dropout),
            nn.Linear(hidden_dim, output_dim))

        self.dropout = args.dropout
        self.num_layers = args.num_layers

        self.emb = emb

    def build_conv_model(self, model_type):
        if model_type == 'GraphSage':
            return GraphSage
        elif model_type == 'GAT':
            # When applying GAT with num heads > 1, you need to modify the
            # input and output dimension of the conv layers (self.convs),
            # to ensure that the input dim of the next layer is num heads
            # multiplied by the output dim of the previous layer.
            # HINT: In case you want to play with multiheads, you need to change the for-loop that builds up self.convs to be
            # self.convs.append(conv_model(hidden_dim * num_heads, hidden_dim)),
            # and also the first nn.Linear(hidden_dim * num_heads, hidden_dim) in post-message-passing.
            return GAT

    def forward(self, data):
        x, edge_index, batch = data.x, data.edge_index, data.batch

        for i in range(self.num_layers):
            x = self.convs[i](x, edge_index)
            x = F.relu(x)
            x = F.dropout(x, p=self.dropout,training=self.training)

        x = self.post_mp(x)

        if self.emb == True:
            return x

        return F.log_softmax(x, dim=1)

    def loss(self, pred, label):
        return F.nll_loss(pred, label)
```

## Creating Our Own Message Passing Layer

Now let's start implementing our own message passing layers! Working through this part will help us become acutely familiar with the behind the scenes work of implementing Pytorch Message Passing Layers, allowing us to build our own GNN models. To do so, we will work with and implement 3 critcal functions needed to define a PyG Message Passing Layer: `forward`, `message`, and `aggregate`.

Before diving head first into the coding details, let us quickly review the key components of the message passing process. To do so, we will focus on a single round of messsage passing with respect to a single central node $x$. Before message passing, $x$ is associated with a feature vector $x^{l-1}$, and the goal of message passing is to update this feature vector as $x^l$. To do so, we implement the following steps: 1) each neighboring node $v$ passes its current message $v^{l-1}$ across the edge $(x, v)$ - 2) for the node $x$, we aggregate all of the messages of the neighboring nodes (for example through a sum or mean) - and 3) we transform the aggregated information by for example applying linear and non-linear transformations. Altogether, the message passing process is applied such that every node $u$ in our graph updates its embedding by acting as the central node $x$ in step 1-3 described above.

Now, we extending this process to that of a single message passing layer, the job of a message passing layer is to update the current feature representation or embedding of each node in a graph by propagating and transforming information within the graph. Overall, the general paradigm of a message passing layers is: 1) pre-processing -> 2) **message passing** / propagation -> 3) post-processing.

The `forward` fuction that we will implement for our message passing layer captures this execution logic. Namely, the `forward` function handles the pre and post-processing of node features / embeddings, as well as initiates message passing by calling the `propagate` function.

The `propagate` function encapsulates the message passing process! It does so by calling three important functions: 1) `message`, 2) `aggregate`, and 3) `update`. Our implementation will vary slightly from this, as we will not explicitly implement `update`, but instead place the logic for updating node embeddings after message passing and within the `forward` function. To be more specific, after information is propagated (message passing), we can further transform the node embeddings outputed by `propagate`. Therefore, the output of `forward` is exactly the node embeddings after one GNN layer.

Lastly, before starting to implement our own layer, let us dig a bit deeper into each of the functions described above:

1.

```
def propagate(edge_index, x=(x_i, x_j), extra=(extra_i, extra_j), size=size):
```
Calling `propagate` initiates the message passing process. Looking at the function parameters, we highlight a couple of key parameters.

  - `edge_index` is passed to the forward function and captures the edge structure of the graph.
  - `x=(x_i, x_j)` represents the node features that will be used in message passing. In order to explain why we pass the tuple `(x_i, x_j)`, we first look at how our edges are represented. For every edge $(i, j) \in \mathcal{E}$, we can differentiate $i$ as the source or central node ($x_{central}$) and j as the neighboring node ($x_{neighbor}$).

    Taking the example of message passing above, for a central node $u$ we will aggregate and transform all of the messages associated with the nodes $v$ s.t. $(u, v) \in \mathcal{E}$ (i.e. $v \in \mathcal{N}_{u}$). Thus we see, the subscripts `_i` and `_j` allow us to specifcally differenciate features associated with central nodes (i.e. nodes  recieving message information) and neighboring nodes (i.e. nodes passing messages).

    This is definitely a somewhat confusing concept; however, one key thing to remember / wrap your head around is that depending on the perspective, a node $x$ acts as a central node or a neighboring node. In fact, in undirected graphs we store both edge directions (i.e. $(i, j)$ and $(j, i)$). From the central node perspective, `x_i`, x is collecting neighboring information to update its embedding. From a neighboring node perspective, `x_j`, x is passing its message information along the edge connecting it to a different central node.

  - `extra=(extra_i, extra_j)` represents additional information that we can associate with each node beyond its current feature embedding. In fact, we can include as many additional parameters of the form `param=(param_i, param_j)` as we would like. Again, we highlight that indexing with `_i` and `_j` allows us to differentiate central and neighboring nodes.

  The output of the `propagate` function is a matrix of node embeddings after the message passing process and has shape $[N, d]$.

2.
```
def message(x_j, ...):
```
The `message` function is called by propagate and constructs the messages from
neighboring nodes $j$ to central nodes $i$ for each edge $(i, j)$ in *edge_index*. This function can take any argument that was initially passed to `propagate`. Furthermore, we can again differentiate central nodes and neighboring nodes by appending `_i` or `_j` to the variable name, .e.g. `x_i` and `x_j`. Looking more specifically at the variables, we have:

  - `x_j` represents a matrix of feature embeddings for all neighboring nodes passing their messages along their respective edge (i.e. all nodes $j$ for edges $(i, j) \in \mathcal{E}$). Thus, its shape is $[|\mathcal{E}|, d]$!
  - In implementing GAT we will see how to access additional variables passed to propagate

  Critically, we see that the output of the `message` function is a matrix of neighboring node embeddings ready to be aggregated, having shape $[|\mathcal{E}|, d]$.

3.
```
def aggregate(self, inputs, index, dim_size = None):
```
Lastly, the `aggregate` function is used to aggregate the messages from neighboring nodes. Looking at the parameters we highlight:

  - `inputs` represents a matrix of the messages passed from neighboring nodes (i.e. the output of the `message` function).
  - `index` has the same shape as `inputs` and tells us the central node that corresponding to each of the rows / messages $j$ in the `inputs` matrix. Thus, `index` tells us which rows / messages to aggregate for each central node.

  The output of `aggregate` is of shape $[N, d]$.

For additional resources refer to the PyG documentation for implementing custom message passing layers: https://pytorch-geometric.readthedocs.io/en/latest/notes/create_gnn.html

## GraphSage Implementation

For our first GNN layer, we will implement the well known GraphSage ([Hamilton et al. (2017)](https://arxiv.org/abs/1706.02216)) layer!

For a given *central* node $v$ with current embedding $h_v^{l-1}$, the message passing update rule to tranform $h_v^{l-1} \rightarrow h_v^l$ is as follows:

\begin{equation}
h_v^{(l)} = W_l\cdot h_v^{(l-1)} + W_r \cdot AGG(\{h_u^{(l-1)}, \forall u \in N(v) \})
\end{equation}

where $W_1$ and $W_2$ are learanble weight matrices and the nodes $u$ are *neighboring* nodes. Additionally, we use mean aggregation for simplicity:

\begin{equation}
AGG(\{h_u^{(l-1)}, \forall u \in N(v) \}) = \frac{1}{|N(v)|} \sum_{u\in N(v)} h_u^{(l-1)}
\end{equation}

One thing to note is that we're adding a **skip connection** to our GraphSage implementation through the term $W_l\cdot h_v^{(l-1)}$.

Before implementing this update rule, we encourage you to think about how different parts of the formulas above correspond with the functions outlined earlier: 1) `forward`, 2) `message`, and 3) `aggregate`. As a hint, we are given what the aggregation function is (i.e. mean aggregation)! Now the question remains, what are the messages passed by each neighbor nodes and when do we call the `propagate` function?

Note: in this case the message function or messages are actually quite simple. Additionally, remember that the `propagate` function encapsulates the operations of / the outputs of the combined `message` and `aggregate` functions.

Lastly, $\ell$-2 normalization of the node embeddings is applied after each iteration.

For the following questions, DON'T refer to any existing implementations online.

```python
class GraphSage(MessagePassing):

    def __init__(self, in_channels, out_channels, normalize = True,
                 bias = False, **kwargs):
        super(GraphSage, self).__init__(**kwargs)

        self.in_channels = in_channels
        self.out_channels = out_channels
        self.normalize = normalize

        self.lin_l = None
        self.lin_r = None

        ############################################################################
        # TODO: Your code here!
        # Define the layers needed for the message and update functions below.
        # self.lin_l is the linear transformation that you apply to embedding
        #            for central node.
        # self.lin_r is the linear transformation that you apply to aggregated
        #            message from neighbors.
        # Don't forget the bias!
        # Our implementation is ~2 lines, but don't worry if you deviate from this.

        ############################################################################

        self.reset_parameters()

    def reset_parameters(self):
        self.lin_l.reset_parameters()
        self.lin_r.reset_parameters()

    def forward(self, x, edge_index, size = None):
        """"""

        out = None

        ############################################################################
        # TODO: Your code here!
        # Implement message passing, as well as any post-processing (our update rule).
        # 1. Call the propagate function to conduct the message passing.
        #    1.1 See the description of propagate above or the following link for more information:
        #        https://pytorch-geometric.readthedocs.io/en/latest/notes/create_gnn.html
        #    1.2 We will only use the representation for neighbor nodes (x_j), so by default
        #        we pass the same representation for central and neighbor nodes as x=(x, x).
        # 2. Update our node embedding with skip connection from the previous layer.
        # 3. If normalize is set, do L-2 normalization (defined in
        #    torch.nn.functional)
        #
        # Our implementation is ~5 lines, but don't worry if you deviate from this.

        ############################################################################

        return out

    def message(self, x_j):

        out = None

        ############################################################################
        # TODO: Your code here!
        # Implement your message function here.
        # Hint: Look at the formulation of the mean aggregation function, focusing on
        # what message each neighboring node passes.
        #
        # Our implementation is ~1 lines, but don't worry if you deviate from this.

        ############################################################################

        return out

    def aggregate(self, inputs, index, dim_size = None):

        out = None

        # The axis along which to index number of nodes.
        node_dim = self.node_dim

        ############################################################################
        # TODO: Your code here!
        # Implement your aggregate function here.
        # See here as how to use torch_scatter.scatter:
        # https://pytorch-scatter.readthedocs.io/en/latest/functions/scatter.html#torch_scatter.scatter
        #
        # Our implementation is ~1 lines, but don't worry if you deviate from this.

        ############################################################################

        return out

```

## Building Optimizers

This function has been implemented for you. **For grading purposes please use the default Adam optimizer**, but feel free to play with other types of optimizers on your own.

```python
import torch.optim as optim

def build_optimizer(args, params):
    weight_decay = args.weight_decay
    filter_fn = filter(lambda p : p.requires_grad, params)
    if args.opt == 'adam':
        optimizer = optim.Adam(filter_fn, lr=args.lr, weight_decay=weight_decay)
    elif args.opt == 'sgd':
        optimizer = optim.SGD(filter_fn, lr=args.lr, momentum=0.95, weight_decay=weight_decay)
    elif args.opt == 'rmsprop':
        optimizer = optim.RMSprop(filter_fn, lr=args.lr, weight_decay=weight_decay)
    elif args.opt == 'adagrad':
        optimizer = optim.Adagrad(filter_fn, lr=args.lr, weight_decay=weight_decay)
    if args.opt_scheduler == 'none':
        return None, optimizer
    elif args.opt_scheduler == 'step':
        scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=args.opt_decay_step, gamma=args.opt_decay_rate)
    elif args.opt_scheduler == 'cos':
        scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=args.opt_restart)
    return scheduler, optimizer
```

## Training and Testing

Here we provide you with the functions to train and test. **Please do not modify this part for grading purposes.**

```python
import time

import networkx as nx
import numpy as np
import torch
import torch.optim as optim
from tqdm import trange
import pandas as pd
import copy

from torch_geometric.datasets import TUDataset
from torch_geometric.datasets import Planetoid
from torch_geometric.data import DataLoader

import torch_geometric.nn as pyg_nn

import matplotlib.pyplot as plt

def train(dataset, args):

    print("Node task. test set size:", np.sum(dataset[0]['test_mask'].numpy()))
    print()
    test_loader = loader = DataLoader(dataset, batch_size=args.batch_size, shuffle=False)

    # build model
    model = GNNStack(dataset.num_node_features, args.hidden_dim, dataset.num_classes,
                            args)
    scheduler, opt = build_optimizer(args, model.parameters())

    # train
    losses = []
    test_accs = []
    best_acc = 0
    best_model = None
    for epoch in trange(args.epochs, desc="Training", unit="Epochs"):
        total_loss = 0
        model.train()
        for batch in loader:
            opt.zero_grad()
            pred = model(batch)
            label = batch.y
            pred = pred[batch.train_mask]
            label = label[batch.train_mask]
            loss = model.loss(pred, label)
            loss.backward()
            opt.step()
            total_loss += loss.item() * batch.num_graphs
        total_loss /= len(loader.dataset)
        losses.append(total_loss)

        if epoch % 10 == 0:
          test_acc = test(test_loader, model)
          test_accs.append(test_acc)
          if test_acc > best_acc:
            best_acc = test_acc
            best_model = copy.deepcopy(model)
        else:
          test_accs.append(test_accs[-1])

    return test_accs, losses, best_model, best_acc, test_loader

def test(loader, test_model, is_validation=False, save_model_preds=False, model_type=None):
    test_model.eval()

    correct = 0
    # Note that Cora is only one graph!
    for data in loader:
        with torch.no_grad():
            # max(dim=1) returns values, indices tuple; only need indices
            pred = test_model(data).max(dim=1)[1]
            label = data.y

        mask = data.val_mask if is_validation else data.test_mask
        # node classification: only evaluate on nodes in test set
        pred = pred[mask]
        label = label[mask]

        if save_model_preds:
          print ("Saving Model Predictions for Model Type", model_type)

          data = {}
          data['pred'] = pred.view(-1).cpu().detach().numpy()
          data['label'] = label.view(-1).cpu().detach().numpy()

          df = pd.DataFrame(data=data)
          # Save locally as csv
          df.to_csv('CORA-Node-' + model_type + '.csv', sep=',', index=False)

        correct += pred.eq(label).sum().item()

    total = 0
    for data in loader.dataset:
        total += torch.sum(data.val_mask if is_validation else data.test_mask).item()

    return correct / total

class objectview(object):
    def __init__(self, d):
        self.__dict__ = d

```

## Let's Start the Training!

We will be working on the CORA dataset on node-level classification.

This part is implemented for you. **For grading purposes, please do not modify the default parameters.** However, feel free to play with different configurations just for fun!

**Submit your best accuracy and loss on Gradescope.**

```python
if 'IS_GRADESCOPE_ENV' not in os.environ:
    for args in [
        {'model_type': 'GraphSage', 'dataset': 'cora', 'num_layers': 2, 'heads': 1, 'batch_size': 32, 'hidden_dim': 32, 'dropout': 0.5, 'epochs': 500, 'opt': 'adam', 'opt_scheduler': 'none', 'opt_restart': 0, 'weight_decay': 5e-3, 'lr': 0.01},
    ]:
        args = objectview(args)
        for model in ['GraphSage']:
            args.model_type = model

            # Match the dimension.
            if model == 'GAT':
              args.heads = 2
            else:
              args.heads = 1

            if args.dataset == 'cora':
                dataset = Planetoid(root='/tmp/cora', name='Cora')
            else:
                raise NotImplementedError("Unknown dataset")
            test_accs, losses, best_model, best_acc, test_loader = train(dataset, args)

            print("Maximum test set accuracy: {0}".format(max(test_accs)))
            print("Minimum loss: {0}".format(min(losses)))

            # Run test for our best model to save the predictions!
            test(test_loader, best_model, is_validation=False, save_model_preds=True, model_type=model)
            print()

            plt.title(dataset.name)
            plt.plot(losses, label="training loss" + " - " + args.model_type)
            plt.plot(test_accs, label="test accuracy" + " - " + args.model_type)
        plt.legend()
        plt.show()

```

## Question 1.1: What is the maximum accuracy obtained on the test set for GraphSage? (10 points)

Running the cell above will show the results of your best model and save your best model's predictions to a file named *CORA-Node-GraphSage.csv*.

As we have seen before you can view this file by clicking on the *Folder* icon on the left side pannel. When you sumbit your assignment, you will have to download this file and attatch it to your submission.