## Tensorflow로 간단한 linear regression을 구현 (new)

### Hypothesis and Cost function
**Hypothesis**
+ 주어진 `x`값에 대하여 어떻게 값을 **예측**할 것인가

\begin{align}
H(x) = Wx + b \\
\end{align}

**Cost function**
+ 값을 얼마나 잘 **예측**했는지 나타나는 것
+ `Cost function`을 **가장 작게** 만드는 것이 **학습**

\begin{align}
cost(W,b) = \frac{1}{m}\sum_{i=1}^m(H(x^{(i)}) - y^{(i)})^2 \\
\end{align}

### 1. Build graph using TF operations

\begin{align}
H(x) = Wx + b \\
\end{align}

```python
import tensorflow as tf

# X and Y data
x_train = [1, 2, 3]
y_train = [1, 2, 3]

W = tf.Variable(tf.random_normal([1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

# Our hypothesis XW + b
hypothesis = x_train * W + b
```

    /anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters

\begin{align}
cost(W,b) = \frac{1}{m}\sum_{i=1}^m(H(x^{(i)}) - y^{(i)})^2 \\
\end{align}

```python
# Cost / Loss function
cost = tf.reduce_mean(tf.square(hypothesis - y_train))
```

**tf.reduce_mean()**
+ 주어진 값의 **평균**을 반환

**GradientDescent**

```python
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)
```

### Run / Update graph and get results
`tensorflow`의 `variable`을 사용하기 전에는 **무조건**

`tf.global_variables_initializer()`함수를 사용해 **초기화**

```python
# Launch the graph in a session.
sess = tf.Session()
# Init global var in the graph.
sess.run(tf.global_variables_initializer())

# Fit the line
for step in range(2001):
    sess.run(train)

    if step % 20 == 0:
        print(step, sess.run(cost), sess.run(W), sess.run(b))
```

    0 0.34209618 [0.997927] [-0.58074147]
    20 0.033545386 [1.1831212] [-0.47202426]
    40 0.027953567 [1.19214] [-0.44208765]
    60 0.025365116 [1.1847885] [-0.420573]
    80 0.023036808 [1.1762639] [-0.40073743]
    100 0.020922398 [1.1679953] [-0.38189754]
    120 0.019002063 [1.1601015] [-0.36394912]
    140 0.017257968 [1.1525775] [-0.34684482]
    160 0.01567395 [1.1454071] [-0.33054435]
    180 0.014235348 [1.1385735] [-0.31501007]
    200 0.0129288 [1.1320611] [-0.30020583]
    220 0.011742125 [1.1258547] [-0.28609723]
    240 0.01066438 [1.1199398] [-0.27265167]
    260 0.009685551 [1.1143031] [-0.25983799]
    280 0.00879657 [1.1089313] [-0.24762647]
    300 0.0079891905 [1.103812] [-0.2359889]
    320 0.007255899 [1.098933] [-0.22489826]
    340 0.0065899272 [1.0942836] [-0.21432881]
    360 0.00598507 [1.0898526] [-0.20425609]
    380 0.0054357345 [1.0856298] [-0.19465671]
    400 0.0049368194 [1.0816054] [-0.18550855]
    420 0.0044836923 [1.0777702] [-0.17679021]
    440 0.004072161 [1.0741154] [-0.16848172]
    460 0.0036984005 [1.0706321] [-0.16056366]
    480 0.0033589455 [1.0673127] [-0.15301773]
    500 0.0030506484 [1.0641494] [-0.1458265]
    520 0.0027706444 [1.0611343] [-0.13897318]
    540 0.0025163395 [1.0582613] [-0.13244183]
    560 0.0022853736 [1.0555233] [-0.12621751]
    580 0.0020756186 [1.0529138] [-0.1202857]
    600 0.001885115 [1.0504271] [-0.1146327]
    620 0.0017120908 [1.0480573] [-0.10924539]
    640 0.0015549489 [1.0457988] [-0.10411128]
    660 0.0014122262 [1.0436463] [-0.09921843]
    680 0.0012826066 [1.0415952] [-0.09455557]
    700 0.0011648793 [1.0396402] [-0.0901117]
    720 0.00105796 [1.0377773] [-0.0858767]
    740 0.00096085883 [1.0360019] [-0.08184079]
    760 0.0008726657 [1.0343099] [-0.07799457]
    780 0.000792567 [1.0326974] [-0.07432906]
    800 0.00071982417 [1.0311608] [-0.07083588]
    820 0.00065375585 [1.0296963] [-0.06750689]
    840 0.0005937492 [1.0283008] [-0.06433428]
    860 0.0005392534 [1.0269707] [-0.06131082]
    880 0.0004897586 [1.0257032] [-0.05842944]
    900 0.00044481046 [1.0244952] [-0.05568347]
    920 0.00040398026 [1.023344] [-0.05306651]
    940 0.00036690236 [1.022247] [-0.05057259]
    960 0.00033322643 [1.0212014] [-0.04819589]
    980 0.0003026446 [1.0202051] [-0.04593093]
    1000 0.0002748634 [1.0192553] [-0.04377222]
    1020 0.00024963482 [1.0183504] [-0.041715]
    1040 0.00022672194 [1.017488] [-0.0397545]
    1060 0.00020591177 [1.016666] [-0.03788612]
    1080 0.00018700992 [1.0158828] [-0.03610559]
    1100 0.00016984683 [1.0151366] [-0.03440881]
    1120 0.00015425656 [1.0144252] [-0.03279176]
    1140 0.00014009811 [1.0137471] [-0.03125063]
    1160 0.00012724116 [1.0131011] [-0.02978192]
    1180 0.000115560404 [1.0124854] [-0.02838225]
    1200 0.0001049542 [1.0118985] [-0.02704835]
    1220 9.532145e-05 [1.0113394] [-0.02577715]
    1240 8.65722e-05 [1.0108066] [-0.02456572]
    1260 7.862677e-05 [1.0102987] [-0.0234113]
    1280 7.141052e-05 [1.0098147] [-0.02231106]
    1300 6.485634e-05 [1.0093534] [-0.02126252]
    1320 5.8903035e-05 [1.0089138] [-0.02026324]
    1340 5.3496104e-05 [1.0084949] [-0.01931092]
    1360 4.8587113e-05 [1.0080957] [-0.01840341]
    1380 4.412682e-05 [1.0077152] [-0.01753851]
    1400 4.007656e-05 [1.0073526] [-0.01671425]
    1420 3.6398487e-05 [1.0070071] [-0.01592876]
    1440 3.3058153e-05 [1.0066777] [-0.01518017]
    1460 3.0023424e-05 [1.0063639] [-0.01446675]
    1480 2.7268057e-05 [1.0060649] [-0.01378687]
    1500 2.4765388e-05 [1.0057799] [-0.01313897]
    1520 2.2492006e-05 [1.0055083] [-0.0125215]
    1540 2.042811e-05 [1.0052495] [-0.01193309]
    1560 1.8553206e-05 [1.0050027] [-0.01137231]
    1580 1.685074e-05 [1.0047677] [-0.0108379]
    1600 1.5303938e-05 [1.0045435] [-0.01032858]
    1620 1.3899029e-05 [1.00433] [-0.00984315]
    1640 1.2624049e-05 [1.0041267] [-0.00938063]
    1660 1.1465578e-05 [1.0039326] [-0.00893985]
    1680 1.0412673e-05 [1.0037478] [-0.00851968]
    1700 9.457023e-06 [1.0035717] [-0.00811931]
    1720 8.58898e-06 [1.0034039] [-0.00773777]
    1740 7.80083e-06 [1.0032438] [-0.00737414]
    1760 7.0849633e-06 [1.0030915] [-0.00702758]
    1780 6.4345245e-06 [1.0029461] [-0.00669735]
    1800 5.844049e-06 [1.0028077] [-0.00638261]
    1820 5.307595e-06 [1.0026759] [-0.00608269]
    1840 4.82041e-06 [1.00255] [-0.00579687]
    1860 4.3781924e-06 [1.0024303] [-0.00552446]
    1880 3.9767715e-06 [1.0023161] [-0.00526487]
    1900 3.611425e-06 [1.0022072] [-0.0050175]
    1920 3.2802097e-06 [1.0021034] [-0.00478174]
    1940 2.9790438e-06 [1.0020046] [-0.00455706]
    1960 2.7056e-06 [1.0019104] [-0.00434294]
    1980 2.4575631e-06 [1.0018208] [-0.00413892]
    2000 2.2321947e-06 [1.0017353] [-0.00394444]

### Placeholders
사전에 `train data`를 설정하지 않고 `Session`을 실행시킬 때

`feed_dict`를 사용해 `train data` 지정이 가능하다.

```python
W = tf.Variable(tf.random_normal([1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

# Now we can use X and Y in place of x_data and y_data
# placeholders for a tensor tha will be always fed using feed_dict
X = tf.placeholder(tf.float32, shape=[None])
Y = tf.placeholder(tf.float32, shape=[None])

hypothesis = X * W + b
cost = tf.reduce_mean(tf.square(hypothesis - Y))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

# Fit the line
for step in range(2001):
    cost_val, W_val, b_val, _ = \
        sess.run([cost, W, b, train],
                feed_dict={X: [1, 2, 3, 4, 5],
                           Y: [2.1, 3.1, 4.1, 5.1, 6.1]})

    if step % 20 == 0:
        print(step, cost_val, W_val, b_val)
```

    0 11.289088 [0.35358387] [0.6417344]
    20 0.010992345 [1.0638802] [0.85676134]
    40 0.009398816 [1.0627136] [0.8735269]
    60 0.008208036 [1.0586201] [0.88836247]
    80 0.007168132 [1.0547811] [0.9022229]
    100 0.0062599713 [1.0511932] [0.91517556]
    120 0.005466898 [1.0478407] [0.92727983]
    140 0.004774262 [1.0447075] [0.9385914]
    160 0.004169411 [1.0417798] [0.94916224]
    180 0.0036411807 [1.0390434] [0.95904064]
    200 0.00317987 [1.0364865] [0.96827215]
    220 0.002777006 [1.034097] [0.97689915]
    240 0.0024251782 [1.0318639] [0.9849611]
    260 0.0021179225 [1.029777] [0.9924951]
    280 0.0018495988 [1.027827] [0.9995356]
    300 0.0016152605 [1.0260046] [1.0061152]
    320 0.0014106216 [1.0243014] [1.0122639]
    340 0.0012319067 [1.0227101] [1.0180097]
    360 0.0010758385 [1.0212228] [1.0233792]
    380 0.00093953824 [1.0198328] [1.0283973]
    400 0.0008204981 [1.018534] [1.0330868]
    420 0.00071654504 [1.0173199] [1.0374693]
    440 0.0006257567 [1.0161856] [1.0415646]
    460 0.00054647616 [1.0151256] [1.0453917]
    480 0.0004772433 [1.014135] [1.0489678]
    500 0.00041678114 [1.0132093] [1.0523099]
    520 0.0003639822 [1.0123442] [1.0554332]
    540 0.0003178666 [1.0115359] [1.0583516]
    560 0.0002776011 [1.0107805] [1.0610791]
    580 0.00024242947 [1.0100744] [1.0636281]
    600 0.00021171689 [1.0094146] [1.0660101]
    620 0.00018489076 [1.008798] [1.0682361]
    640 0.0001614657 [1.0082217] [1.0703166]
    660 0.00014100892 [1.0076833] [1.0722605]
    680 0.00012314346 [1.0071802] [1.0740772]
    700 0.00010754245 [1.0067099] [1.0757749]
    720 9.392006e-05 [1.0062705] [1.0773613]
    740 8.202072e-05 [1.0058599] [1.0788438]
    760 7.163043e-05 [1.0054762] [1.0802293]
    780 6.255442e-05 [1.0051175] [1.0815241]
    800 5.463013e-05 [1.0047823] [1.082734]
    820 4.7708098e-05 [1.0044692] [1.0838648]
    840 4.1665273e-05 [1.0041765] [1.0849214]
    860 3.6386897e-05 [1.003903] [1.0859088]
    880 3.1776268e-05 [1.0036474] [1.0868317]
    900 2.7750764e-05 [1.0034086] [1.0876939]
    920 2.42352e-05 [1.0031853] [1.0884999]
    940 2.1165635e-05 [1.0029767] [1.0892531]
    960 1.8480354e-05 [1.0027816] [1.0899576]
    980 1.6139435e-05 [1.0025995] [1.0906154]
    1000 1.4094636e-05 [1.0024291] [1.0912299]
    1020 1.2308749e-05 [1.00227] [1.0918043]
    1040 1.074942e-05 [1.0021214] [1.092341]
    1060 9.388066e-06 [1.0019825] [1.0928426]
    1080 8.198024e-06 [1.0018526] [1.0933114]
    1100 7.159786e-06 [1.0017313] [1.0937494]
    1120 6.253105e-06 [1.001618] [1.0941586]
    1140 5.4606608e-06 [1.001512] [1.0945411]
    1160 4.7687245e-06 [1.001413] [1.0948987]
    1180 4.1645444e-06 [1.0013204] [1.0952328]
    1200 3.6373112e-06 [1.001234] [1.0955449]
    1220 3.1763925e-06 [1.0011532] [1.0958365]
    1240 2.7744234e-06 [1.0010777] [1.096109]
    1260 2.4228366e-06 [1.0010071] [1.0963638]
    1280 2.1162505e-06 [1.0009413] [1.0966018]
    1300 1.8480727e-06 [1.0008798] [1.0968243]
    1320 1.6139018e-06 [1.0008221] [1.0970322]
    1340 1.4095725e-06 [1.0007683] [1.0972265]
    1360 1.2309991e-06 [1.0007179] [1.097408]
    1380 1.0752067e-06 [1.000671] [1.0975777]
    1400 9.3903253e-07 [1.000627] [1.0977364]
    1420 8.199281e-07 [1.0005859] [1.0978845]
    1440 7.1618706e-07 [1.0005475] [1.098023]
    1460 6.255058e-07 [1.0005118] [1.0981524]
    1480 5.462191e-07 [1.0004783] [1.0982732]
    1500 4.770899e-07 [1.0004469] [1.0983862]
    1520 4.1674247e-07 [1.0004178] [1.0984918]
    1540 3.6408133e-07 [1.0003905] [1.0985904]
    1560 3.1793948e-07 [1.0003649] [1.0986826]
    1580 2.777669e-07 [1.0003409] [1.0987687]
    1600 2.4259543e-07 [1.0003189] [1.0988493]
    1620 2.1188694e-07 [1.0002979] [1.0989246]
    1640 1.8513697e-07 [1.0002785] [1.098995]
    1660 1.6166857e-07 [1.0002602] [1.0990607]
    1680 1.4129925e-07 [1.0002433] [1.0991219]
    1700 1.2334017e-07 [1.0002272] [1.0991794]
    1720 1.0773766e-07 [1.0002126] [1.099233]
    1740 9.4123074e-08 [1.0001984] [1.0992832]
    1760 8.22517e-08 [1.0001857] [1.09933]
    1780 7.18329e-08 [1.0001736] [1.0993737]
    1800 6.275693e-08 [1.0001621] [1.0994147]
    1820 5.483298e-08 [1.0001516] [1.0994531]
    1840 4.7832373e-08 [1.0001417] [1.0994887]
    1860 4.1793328e-08 [1.0001323] [1.0995221]
    1880 3.6520042e-08 [1.0001237] [1.0995533]
    1900 3.1939056e-08 [1.0001158] [1.0995824]
    1920 2.794269e-08 [1.0001082] [1.0996096]
    1940 2.4392374e-08 [1.0001011] [1.099635]
    1960 2.1316282e-08 [1.0000944] [1.0996588]
    1980 1.8629601e-08 [1.0000883] [1.099681]
    2000 1.624063e-08 [1.0000825] [1.0997021]

### Testing our model

```python
# Testing our model
print(sess.run(hypothesis, feed_dict={X: [5]}))
print(sess.run(hypothesis, feed_dict={X: [2.5]}))
print(sess.run(hypothesis, feed_dict={X: [1.5, 3.5]}))
```

    [6.100115]
    [3.5999084]
    [2.5998259 4.599991 ]


```python
from collections import defaultdict
import numpy as np
import scipy
import scipy.sparse as sps
import math
import matplotlib.pyplot as plt
import time
from sklearn.datasets import load_svmlight_file
import random
%matplotlib inline
```

# Support Vector Machines
## Classification Using SVM
Load dataset. We will use w1a dataset from LibSVM datasets https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/

The original optimization problem for the Support Vector Machine (SVM) is given by
\begin{equation}\label{eq:primal}
  \min_{w \in R^d} \  \sum_{i=1}^n \ell(y_i A_i^\top w) + \frac\lambda2 \|w\|^2
\end{equation}
where $\ell : R\rightarrow R$, $\ell(z) := \max\{0,1-z\}$ is the hinge loss function.
Here for any $i$, $1\le i\le n$, the vector $A_i\in R^d$ is the $i$-th data example, and $y_i\in\{\pm1\}$ is the corresponding label.

The dual optimization problem for the SVM is given by
\begin{equation}\label{eq:dual}
 \max_{\boldsymbol{\alpha} \in R^n } \  \alpha^\top\boldsymbol{1} - \tfrac1{2\lambda} \alpha^\top Y A A^\top Y\alpha
 \text{    such that    $0\le \alpha_i \le 1  \ \forall i$}
\end{equation}
where $Y := \mathop{diag}(y)$, and $A\in R^{n \times d}$ again collects all $n$ data examples as its columns.

Note that $w$ can be derived from $\alpha$ as
\begin{equation}
    w(\alpha) = \frac{1}{\lambda} A^\top Y \alpha.
\end{equation}

```python
DATA_TRAIN_PATH = 'data/w1a'

A, y = load_svmlight_file(DATA_TRAIN_PATH)
A = A.toarray()
print(y.shape, A.shape)
```

## Prepare cost and prediction functions

```python
def calculate_primal_objective(y, A, w, lambda_):
    """
    Compute the full cost (the primal objective), that is loss plus regularizer.
    y: +1 or -1 labels, shape = (num_examples)
    A: Dataset matrix, shape = (num_examples, num_features)
    w: Model weights, shape = (num_features)
    return: scalar value
    """
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    raise NotImplementedError
```

```python
def calculate_accuracy(y, A, w):
    """
    Compute the training accuracy on the training set (can be called for test set as well).
    y: +1 or -1 labels, shape = (num_examples)
    A: Dataset matrix, shape = (num_examples, num_features)
    w: Model weights, shape = (num_features)
    return: scalar value
    """
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    raise NotImplementedError
```

## Coordinate Descent (Ascent) for SVM

Compute the closed-form update for the i-th variable alpha, in the dual optimization problem, given alpha and the current corresponding w.

Hints:
- Differentiate the dual objective with respect to one `alpha[i]`.
- Set the derivative to zero to compute a new `alpha[i]`.
- Make sure the values of alpha stay inside a `[0, 1]` box.
- You can formulate the update as `alpha[i] = projection(alpha[i] + lambda_ * (some update))`.
- You can test the correctness of your implementation by checking if the difference between the dual objective and primal objective goes to zero. This difference, the duality gap, should get smaller than 10 in 700000 iterations.

```python
def calculate_coordinate_update(y, A, lambda_, alpha, w, i):
    """
    Compute a coordinate update (closed form) for coordinate i.
    y: +1 or -1 labels, shape = (num_examples)
    A: Dataset matrix, shape = (num_examples, num_features)
    lambda_: Regularization parameter, scalar
    alpha: Dual variables, shape = (num_examples)
    w: Model weights, shape = (num_examples)
    i: Index of the entry of the dual variable 'alpha' that is to be updated
    return: New weights w (shape (num_features)), New dual variables alpha (shape (num_examples))
    """
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    # calculate the update of coordinate at index=n.
    a_i, y_i = A[i], y[i]
    old_alpha_i = np.copy(alpha[i])

    raise NotImplementedError
    return w, alpha
```

```python
def calculate_dual_objective(y, A, w, alpha, lambda_):
    """
    Calculate the objective for the dual problem.
    Follow the formula given above.
    y: +1 or -1 labels, shape = (num_examples)
    A: Dataset matrix, shape = (num_examples, num_features)
    alpha: Dual variables, shape = (num_examples)
    lambda_: Regularization parameter, scalar
    return: Scalar value
    """
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    raise NotImplementedError
```

```python
def coordinate_descent_for_svm_demo(y, A, trace=False):
    max_iter = 1000000
    lambda_ = 0.01
    history = defaultdict(list) if trace else None

    num_examples, num_features = A.shape
    w = np.zeros(num_features)
    alpha = np.zeros(num_examples)

    for it in range(max_iter):
        # i = sample one data point uniformly at random from the columns of A
        i = random.randint(0,num_examples-1)

        w, alpha = calculate_coordinate_update(y, A, lambda_, alpha, w, i)

        if it % 100000 == 0:
            # primal objective
            primal_value = calculate_primal_objective(y, A, w, lambda_)
            # dual objective
            dual_value = calculate_dual_objective(y, A, w, alpha, lambda_)
            # primal dual gap
            duality_gap = primal_value - dual_value

            print('iteration=%i, primal:%.5f, dual:%.5f, gap:%.5f'%(
                    it, primal_value, dual_value, duality_gap))
        if it % 1000 == 0:
            primal_value = calculate_primal_objective(y, A, w, lambda_)
            if trace:
                history["objective_function"] += [primal_value]
                history['iter'].append(it)

    print("training accuracy = {l}".format(l=calculate_accuracy(y, A, w)))
    return history

history_cd = coordinate_descent_for_svm_demo(y, A, trace=True)
```

# Stochastic gradient descent for SVM

Let's now compare it with SGD on original problem for the SVM. In this part, you will implement stochastic gradient descent on the primal SVM objective. The stochasticity comes from sampling data points.

```python
def compute_stoch_gradient_svm(A_sample, b_sample, lambda_, w_t, num_data_points):
    """
    Calculate stochastic gradient over A_batch, b_batch.
    A_sample: A data sample, shape=(num_features)
    b_sample: Corresponding +1 or -1 label, scalar
    w_t: Model weights, shape=(num_features)
    num_data_points: Total size of the dataset, scalar integer
    """
    # ***************************************************
    # INSERT YOUR CODE HERE
    # TODO
    # ***************************************************
    raise NotImplementedError
```

```python
def stochastic_gradient_descent_svm_demo(A, b, gamma, batch_size=1, trace=False):
    history = defaultdict(list) if trace else None
    num_data_points, num_features = np.shape(A)
    max_iter = 1000000
    lambda_ = 0.01

    w_t = np.zeros(num_features)

    current_iter = 0
    while (current_iter < max_iter):
        i = random.randint(0,num_data_points - 1)
        b_batch, A_batch = b[i], A[i]
        gradient = compute_stoch_gradient_svm(A_batch, b_batch, lambda_, w_t, num_data_points)
        w_t = w_t - gamma * gradient
        if current_iter % 100000 == 0:
            primal_value = calculate_primal_objective(y, A, w_t, lambda_)
            print('iteration=%i, primal:%.5f'%(
                    current_iter, primal_value))
        if current_iter % 1000 == 0:
            primal_value = calculate_primal_objective(y, A, w_t, lambda_)
            if trace:
                history['objective_function'].append(primal_value)
                history['iter'].append(current_iter)
        current_iter += 1
    print("training accuracy = {l}".format(l=calculate_accuracy(y, A, w_t)))
    return history

```

Try different stepsized and find the best one

```python
# ***************************************************
# INSERT YOUR CODE HERE
# TODO
# ***************************************************
```

Plot learning curves

```python
# ***************************************************
# INSERT YOUR CODE HERE
# TODO
# ***************************************************
```

## Compare SGD with Coordinate Descent

Compare two algorithms in terms of convergence, time complexities per iteration. Which one is easier to use?


###### Material desenvolvido para o minicurso: Introdução à solução numérica de EDP's, ministrado no ERMAC/2018 de 5 a 6 de abril de 2018 na Universidade Federal de Lavras, Lavras/MG, Brasil. Autor: [Jonas Laerte Ansoni](http://jonasansoni.blogspot.com.br/).

#  Minicurso: Introdução à solução numérica de EDP's

Neste _IPython notebook_ iniciaremos os estudos da solução numérica das EDP's apresentando um modelo simplificado da dispersão de poluentes no ar. Através deste exemplo será possível fazer a introdução ao método das diferenças finitas para resolvermos numericamente a equação de transporte (equação de convecção-difusão).

## 2. Modelo para a dispersão de poluentes

  Exemplo adaptado do material: Nachbin, A. ; TABAK, E. . Introdução à Modelagem Matemática e Computação Científica II, SBMAC. XX CNMAC/SBMAC, 1997. 114p - https://www.youtube.com/watch?v=iRzUUmEOUO8

A chaminé de uma fábrica solta fumaça com um produto tóxico, com uma concentração inicial $\alpha$. Uma pessoa interessada em comprar a casa a uma distância $d$ da fábrica nos consulta sobre a condição do ar na vizinhança.

####  Figura 1. Diagrama ilustrando o problema de dispersão de poluentes.

Considerando a pior situação possível, na qual o vento sopra na direção da casa com velocidade máxima $u$. O mecanismo de __transporte__, no qual a nuvem é carregada (transportada) pelo vento sem mudar de forma, é chamado de __advecção__. Também devemos levar em consideração o mecanismos de __difusão__, no qual a nuvem vai se espalhando ao mesmo tempo que a concentração do poluente vai baixando.

Acabamos de escolher o modelo físico, ou seja, decidimos que tipos de mecanismos levaremos em conta no nosso estudo. Agora precisamos fazer a representação matemática dos mecanismos de transporte e difusão.

 __Objetivo__: Calcular o nível final do poluente (ou seja, a concentração $c=c(x,t)$) nos arredores da casa.

### 2.1. Modelo simplificado: Equação de transporte

\begin{equation}
\frac{\partial c}{\partial t} = -u \frac{\partial c}{\partial x}+ k \frac{\partial^2 c}{\partial x^2}
\tag{1}
\end{equation}

 Termo advectivo (convectivo) + Temo difusivo: Equação de transporte

Vamos desprezar inicialmente o termo difusivo: $k \frac{\partial^2 c}{\partial x^2}$. Temos então a equação de advecção (convecção), em que $u>0$ é constante:

\begin{equation}
\frac{\partial c}{\partial t} + u \frac{\partial c}{\partial x} = 0 \Longrightarrow \frac{\partial c}{\partial t} = -u \frac{\partial c}{\partial x}
\tag{2}
\end{equation}
onde o termo $-u \frac{\partial c}{\partial x}$ é o termo convectivo, inercial ou advectivo.

A equação representa uma *onda* propagando com velocidade $u$ na direção $x$. Com condição inicial $c(x,0)=c_0(x)$ conforme a figura abaixo, a equação tem solução exata dada por:

\begin{equation}c(x,t)=c_0(x-ut)\tag{3}\end{equation}

####  Figura 2. Modelagem de uma nuvem de poluente.

 Verifique isso!!!

##### (Pode-se utilizar por exemplo o método das características. Uma exelente explicação pode ser vista nas aulas do prof. André Nachbin (IMPA) [Link](https://youtu.be/2ugsEkBko-0))

##### *Livro J. C. Strickwerda, 'Finite Difference Schemes and Partial Differential Equations' , Chapman & Hall, 1989

####  Figura 3. Curvas características para uma onda positiva.

 _Vamos ver agora como resolver numericamente essa equação da onda, e verificar quais as conseguencias das nossas escolhas de qual metodologia numérica utilizada._

###  2.2. Método das diferenças finitas

Para que a solução seja calculada precisamos realizar a discretização da equação em relação ao tempo e ao espaço.

Vamos considerar o domínio discreto em relação ao *espaço-tempo*, onde as coordenadas na direção horizontal representam o avanço no tempo de $t^n$ à $t^{n+1}$ - e as coordenadas na direção vertical em relação ao movimento no espaço: pontos consecutivos são $x_{i-1}$, $x_i$, e $x_{i+1}$. Isso cria uma malha onde um ponto tem ambos os índices, temporal e espacial. A seguir temos uma representação gráfica da malha tempo-espaço:

\begin{matrix}
t^{n+1} & \rightarrow & \bullet  && \bullet  && \bullet  \\
t^n & \rightarrow & \bullet  && \bullet  && \bullet  \\
& &  x_{i-1} && x_i && x_{i+1}
\end{matrix}

Para a solução numérica de $c(x,t)$, utilizaremos os índices subscritos para denotar a posição espacial, como $c_i$, e sobrescrito denota o instante temporal, como $c^n$. Temos portanto a representação discreta como $c^{n}_{i}$.

\begin{matrix}
& &\bullet & & \bullet & &  \bullet \\
& &c^{n+1}_{i-1} & & c^{n+1}_i & & c^{n+1}_{i+1} \\
& &\bullet & & \bullet & &  \bullet \\
& &c^n_{i-1} & & c^n_i & & c^n_{i+1} \\
& &\bullet & & \bullet & &  \bullet \\
& &c^{n-1}_{i-1} & & c^{n-1}_i & & c^{n-1}_{i+1} \\
\end{matrix}

Outra maneira de obter a nossa discretização é dizer que ela é obtida por passos constantes no tempo e espaço, $\Delta t$ e $\Delta x$, como a seguir:

\begin{eqnarray}
x_i &=& i\, \Delta x \quad \text{and} \quad t^n= n\, \Delta t \nonumber \\
c_i^n &=& c(i\, \Delta x, n\, \Delta t)
\end{eqnarray}

###  Desenvolvimento _"The Flash"_ dos métodos de diferença finitas na lousa.

 Graficamente:

####  Figura 4. Pontos utilizados aproximações derivada primeira ordem.

Seja $f$ uma função contínua

* Aproximação por diferença atrasada (_backward_):
$\frac{\partial f}{\partial x} \approx \frac{f_i - f_{i-1}}{\Delta x}$

* Aproximação por diferença adiantada (_forward_):
$\frac{\partial f}{\partial x} \approx \frac{f_{i+1} - f_{i}}{\Delta x}$

* Aproximação por diferença central:
$\frac{\partial f}{\partial x} \approx \frac{f_{i+1} - f_{i-1}}{2\Delta x}$

#### Expansão em série de Taylor

Supondo uma função $f$ qualquer contínua no intervalo $[a, b]$ de interesse e que possua derivada até ordem $N$ contínuas nesse intervalo, o Teorema de Taylor permite escrever, para todo $x \in [a,b]$,

\begin{equation}
f(x)=f(x_i)+(\Delta x) \frac{\partial f}{\partial x}\big|_i + \frac{(\Delta x)^2}{2!} \frac{\partial ^2 f}{\partial x^2}\big|_i + \frac{(\Delta x)^3}{3!} \frac{\partial ^3 f}{\partial x^3}\big|_i + ...
\tag{4}
\end{equation}

em que $\Delta x=x-x_i$.

Queremos determinar a primeira derivada de uma função $f$ no ponto $x_i$. Expandindo $f(x_i + \Delta x)$ em série de Taylor em torno do ponto $x_i$

\begin{equation}
f(x_i+\Delta x)=f(x_i)+(\Delta x) \frac{\partial f}{\partial x}\big|_i + \frac{(\Delta x)^2}{2!} \frac{\partial ^2 f}{\partial x^2}\big|_i + \frac{(\Delta x)^3}{3!} \frac{\partial ^3 f}{\partial x^3}\big|_i + ...
\tag{5}
\end{equation}

Isolando a primeira derivada $\frac{\partial c}{\partial x}\big|_i$, podemos escrever:

\begin{equation}
\frac{\partial f}{\partial x}\big|_i = \frac{f(x_i+\Delta x) - f(x_i)}{\Delta x}+ \left[-\frac{(\Delta x)}{2!} \frac{\partial ^2 f}{\partial x^2}\big|_i - \frac{(\Delta x)^2}{3!} \frac{\partial ^3 f}{\partial x^3}\big|_i - ...\right]
\tag{6}
\end{equation}

A expressão 6 indica que a primeira derivada (__diferenças progressivas__) é:

\begin{equation}
\frac{\partial f}{\partial x}\big|_i = \frac{f(x_i+\Delta x) - f(x_i)}{\Delta x} + ELT
\tag{7}
\end{equation}

onde

\begin{equation}
ELT = \left[-\frac{(\Delta x)}{2!} \frac{\partial ^2 f}{\partial x^2}\big|_i - \frac{(\Delta x)^2}{3!} \frac{\partial ^3 f}{\partial x^3}\big|_i - ...\right]
\end{equation}

é o _erro local de truncamento_. Este erro aparece porque utilizamos um número finito de termos na série de Taylor. O ELT fornece uma medida da diferença entre o valor exato da derivada e sua aproximação numérica, indicando também como essa diferença varia com a redução do $\Delta x$.

Como é considerado $0<\Delta x<1$, supõem-se que o termo dominante do ELT será o primeiro, isto é, aquele com a menor potência de $\Delta x$.

Os termos do ELT (Eq. 7) são então representados por ${\mathcal O}(\Delta x)$ ("ordem $\Delta x$. (Obs. A expressão  do tipo ${\mathcal O}(\Delta x)$ somente indica como o erro local de truncamento varia com o refinamento da malha, e não o valor do erro.

Realizando o mesmo processo a partir da expressão $f(x_i - \Delta x)$ em série de Taylor em torno do ponto $x_i$:

\begin{equation}
f(x_i-\Delta x)=f(x_i)-(\Delta x) \frac{\partial f}{\partial x}\big|_i + \frac{(\Delta x)^2}{2!} \frac{\partial^2 f}{\partial x^2}\big|_i + {\mathcal O}(\Delta x)^3
\tag{8}
\end{equation}

Isolando a primeira derivada temos:

\begin{equation}
\frac{\partial f}{\partial x}\big|_i = \frac{f_{i}-f_{i-1}}{\Delta x}+{\mathcal O}(\Delta x)
\tag{9}
\end{equation}

que é outra aproximação para a derivada primeira. (__diferenças atrasadas__)

Realizando a subtração das expressões 5 e 8 ($f(x_i+\Delta x)-f(x_i-\Delta x)$), de modo a eliminar o termo contendo a segunda derivada de $f$, obtemos:

\begin{equation}
f(x_i+\Delta x)=f(x_i)+(\Delta x) \frac{\partial f}{\partial x}\big|_i + \frac{(\Delta x)^2}{2!} \frac{\partial^2 f}{\partial x^2}\big|_i + {\mathcal O}(\Delta x)^3
\end{equation}

\begin{equation}
f(x_i-\Delta x)=f(x_i)-(\Delta x) \frac{\partial f}{\partial x}\big|_i + \frac{(\Delta x)^2}{2!} \frac{\partial^2 f}{\partial x^2}\big|_i + {\mathcal O}(\Delta x)^3
\end{equation}

Obtemos

\begin{equation}
\frac{\partial f}{\partial x}\big|_i = \frac{f_{i+i}-f_{i-1}}{2 \Delta x}+{\mathcal O}(\Delta x)^2
\tag{10}
\end{equation}

Observe que a expressão anterior utiliza os pontos $x_{i-1}$ e $x_{i+1}$ para o cálculo da primeira derivada de $f$ no ponto central $x_i$. Assim esta expressão é denominada aproximação por __diferenças centrais__. Ela também é uma aproximação de ordem 2.

Realizando a soma das expressões 5 e 8 ($f(x_i+\Delta x)+f(x_i-\Delta x)$) e realizando as operações de isolar o termo derivativo de menor ordem, obtemos a seguinte expressão:

\begin{equation}
\frac{\partial^2 f}{\partial x^2}\big|_i = \frac{f_{i+i}-2f_i+f_{i-1}}{(\Delta x)^2}+{\mathcal O}(\Delta x)^2
\tag{11}
\end{equation}

o que é uma aproximação para derivadas de segunda ordem.

  Para ver uma explicação passo-a-passo assista a aula
  (https://www.youtube.com/watch?v=iz22_37mMkk&list=PL30F4C5ABCE62CB61&index=5)

Em resumo temos até agora:

Diferenças progressivas:

\begin{equation}
       \frac{\partial f}{\partial x}\big|_i = \frac{f_{i+i}-f_i}{(\Delta x)}+{\mathcal O}(\Delta x)
    \end{equation}

Diferenças atrasadas:

\begin{equation}
       \frac{\partial f}{\partial x}\big|_i = \frac{f_{i}-f_{i-1}}{(\Delta x)}+{\mathcal O}(\Delta x)
    \end{equation}

Diferenças centrais:

\begin{equation}
       \frac{\partial f}{\partial x}\big|_i = \frac{f_{i+i}-f_{i-1}}{2 \Delta x}+{\mathcal O}(\Delta x)^2
    \end{equation}

Diferenças para derivadas de segunda ordem:

\begin{equation}
       \frac{\partial^2 f}{\partial x^2}\big|_i = \frac{f_{i+i}-2f_i+f_{i-1}}{(\Delta x)^2}+{\mathcal O}(\Delta x)^2
    \end{equation}

### 2.3. Discretizando nossa equação modelo: Equação hiperbólica

Vamos ver como discretizar a equação de convecção linear 1D tanto no espaço como no tempo. Por definição, a derivada parcial em relação ao tempo muda apenas com o tempo e não com o espaço; sua forma discretizada altera apenas os indices $n$. Da mesma forma, a derivada parcial em relação a $x$ muda com o espaço e não o tempo, e apenas os indices $i$ são afetados.

Vamos discretizar a coordenada espacial $x$ em pontos indexados de $i=0$ à $N$, e depois realizar os passos no tempo discreto em intervalos de tamanho $\Delta t$.

Considerando a derivada em relação ao tempo e discretizando com diferenças progressivas:

\begin{equation}\frac{\partial c}{\partial t}(i\Delta x, n\Delta t)\approx \frac{c^{n+1}_i-c^n_i}{\Delta t}\end{equation}

Aproximando a derivada para o espaço com diferenças atrasadas:

\begin{equation}
\frac{\partial c}{\partial x}(i\Delta x, n\Delta t)\approx \frac{c^{n}_i-c^n_{i-1}}{\Delta x}\end{equation}

Esta discretização é chamada de  **upwind,**  em que a aproximação da derivada espacial depende da direção de propagação das caracteristicas (na direção de onde vem o escoamento).

Para a equação abordada $u>0$ e aplicando a discretização *upwind* acima representada, temos:

\begin{equation} \frac{c^{n+1}_i-c^n_i}{\Delta t}+u\frac{c^{n}_i-c^n_{i-1}}{\Delta x}=0 \end{equation}
ou
\begin{equation} c^{n+1}_i=c^n_i-u\frac{\Delta t}{\Delta x}(c^{n}_i-c^n_{i-1}) \end{equation}

####  Figura 4. Molêcula computacional para o esquema "forward-time/backward-space".

### _Pergunta:_  Pergunta: O método apresentado é explícito ou implícito??

### Vamos calcular!!!

A seguir vamos carregar as bibliotecas que vamos utilizar para ler vetores e plotar gráficos. *Let's get a little Python on the road.*

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

Como o primeira exercício, vamos resolver a equação linear unidimencional com a condição inicial sendo uma *onda quadrada* como descrito a seguir:

\begin{equation}
c(x,0)=\begin{cases}1 & \text{se } 0,5\leq x \leq 1,\\
0 & \text{em qualquer outro lugar em } (0, 2)
\end{cases}
\end{equation}

Com a condição de contorno em $c=0$ em $x=0$. O domínio espacial para a solução numérica sendo o intervalo $x\in (0, 2)$.

####  Figura 5. Onda "quadrada" condição inicial.

Agora vamos definir algumas variáveis;
Now let's define a few variables; queremos fazer uma malha de pontos uniformemente espaçada dentro do domínio espacial. No código abaixo, definimos uma variável chamada`nx` que será o número de pontos de malha espacial, e a variável `dx` que será a distância entre qualquer par de pontos adjacentes. Também definimos o passo de tempo, `dt`, sendo o número total de passos no tempo, `nt`. A velocidade da onda (velocidade do vento) é definida como $u=1$.

```python
nx = 41
dx = 2/(nx-1)
nt = 25 #25
dt = .025
u = 1      #assume wavespeed of u = 1
x = numpy.linspace(0,2,nx)
```

Precisamos configurar a condição inicial do problema. Aqui utilizamos a função NumPy `zeros()`, definindo um vetor com `nx` elementos todos com valores igual a $0$. (Existe também a função `ones()`). Agora podemos *alterar um pedaço* do vetor em que $c=1$, para obtermos a onda e em seguida imprimir o vetor inicial, somente para verificarmos.

Mas quais valores podemos alterar?

O problema consiste em verificar quais os indices de `c` a onda quadrada inicia em $x = 0.5$ e termina em $x = 1$.
Para isto, utilizamos a função `numpy.where` para retornar uma lista dos indices onde o vetor $x$ encontra (ou não encontra) alguma condição.

```python
c = numpy.zeros(nx)      #numpy function ones()
lbound = numpy.where(x >= 0.5)
ubound = numpy.where(x <= 1)

print(lbound)
print(ubound)
```

```python
c = numpy.zeros(nx)      #numpy function ones()
lbound = numpy.where(x >= 0.5)
ubound = numpy.where(x <= 1)

print(lbound)
print(ubound)
```

    (array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
           27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40]),)
    (array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
           17, 18, 19, 20]),)

Com isto obtemos dois vetores. `lbound`, que indica os indices para $x \geq .5$ e 'ubound', que indica os indices para $x \leq 1$.  Para combinar ambos os vetores, utilizamos uma intersecção entre os vetores com a função `numpy.intersect1d`.

```python
bounds = numpy.intersect1d(lbound, ubound)
c[bounds]=1  #setting c = 1 between 0.5 and 1 as per our I.C.s
print(c)
```

    [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0.
     0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

Em Python podemos combinar comandos. Podemos escrever
```Python
c[numpy.intersect1d(numpy.where(x >= 0.5), numpy.where(x <= 1))] = 1
```
mas isto dificulta um pouco a leitura do código.

Agora vamos verificar as condições iniciais do problema se está tudo OK.

```python
pyplot.plot(x, c, color='#003366', ls='--', lw=3)
pyplot.ylim(0,1.5);
```

###  Parece muito perto do que esperávamos. Mas parece que os lados da onda quadrada não são perfeitamente verticais. Isso esta certo? Pense um pouco!!!

### Forma discreta da equação de convecção linear
Agora é hora de escrever algum código para a forma discreta da equação de convecção usando nosso esquema de diferenças finitas escolhido.

Para todo elemento da nossa matriz `c`, precisamos realizar a operação:

$$c_i^{n+1} = c_i^n - u \frac{\Delta t}{\Delta x}(c_i^n-c_{i-1}^n)$$

Vamos armazenar o resultado em uma nova matriz (temporária) `cn`, a qual será a solução $c$ para o próximo passo. Vamos repetir esta operação por tantos passos de tempo como especificamos e então podemos ver o quão longe a onda já percorreu.

Então, podemos pensar que temos duas operações iterativas: uma no espaço e uma em tempo (aprenderemos de forma diferente mais tarde), então podemos começar aninhando um loop espacial dentro do ciclo de tempo, como mostrado abaixo. Você vê que o código do esquema de diferenças finitas é uma expressão direta da equação discreta:

```python
for n in range(1,nt):
    cn = c.copy()
    for i in range(1,nx):
        c[i] = cn[i]-u*dt/dx*(cn[i]-cn[i-1])
```

**Observação 1:** Destacamos no código acima que o problema necessita de condição de contorno em $x=0$. Aqui não foi necessário impor esta condição para toda iteração porque a discretização não altera o valor de c[0].

**Observação 2:** Aprenderemos mais tarde que o código como escrito acima é bastante ineficiente, e há melhores maneiras de escrever isso, Python-style. Mas vamos continuar.

Agora vamos inspecionar nossa matriz de soluções e depois avançar no tempo com um gráfico de linha.

```python
pyplot.plot(x, c, color='#003366', ls='--', lw=3)
pyplot.ylim(0,1.5);
```

###   Nossa onda quadrada definitivamente se moveu para a direita, mas já não está na forma de um chapéu alto. **O que está acontecendo?**

```python
import io
import base64
from IPython.display import HTML

video = io.open('./figuras/wave1.mp4', 'r+b').read()
encoded = base64.b64encode(video)
HTML(data=''''''.format(encoded.decode('ascii')))
```

### Olhar mais profundo

A solução difere da onda quadrada esperada porque a equação discretizada é uma aproximação da equação diferencial contínua que queremos resolver. Há erros: nós sabíamos disso. Mas a forma modificada da onda inicial é algo curioso. Talvez possa ser melhorado, tornando o espaçamento da malha mais fino. Por que você não tenta? Isso ajuda?

### 2.4 Erro de truncamento espacial

Lembrando que a aproximação por diferença finitas para a derivada espacial:

\begin{equation}\frac{\partial c}{\partial x}\approx \frac{c(x+\Delta x)-c(x)}{\Delta x}\end{equation}

Obtemos usando a definição da derivada em um ponto, e simplesmente removendo o limite, assumindo que $\Delta x$ é muito pequeno. Mas já aprendemos com o método de Euler que isso apresenta um erro, chamado *erro de truncamento*.

Usando uma expansão da série Taylor para os termos espaciais agora, vemos que o esquema de diferença inversa produz um método de primeira ordem, no espaço.

\begin{equation}
\frac{\partial c}{\partial x}(x_i) = \frac{c(x_i)-c(x_{i-1})}{\Delta x} + \frac{\Delta x}{2} \frac{\partial^2 c}{\partial x^2}(x_i) - \frac{\Delta x^2}{6} \frac{\partial^3 c}{\partial x^3}(x_i)+ \cdots
\end{equation}

O termo dominante que é negligenciado na aproximação das diferenças finitas é de $\mathcal{O}(\Delta x)$. Também vemos que a aproximação *converge* para a derivada exata quando $\Delta x \rightarrow 0$.

Em resumo, o esquema de diferença de *"forward-time/backward space"* escolhido é de primeira ordem tanto no espaço quanto no tempo: os erros de truncamento são $\mathcal{O}(\Delta t, \Delta x)$.

__Nós vamos voltar para a falar disso!__

### 2.5. Convecção não-linear

Vamos considerar a equação de convecção não-linear, usando os mesmos métodos de antes. A equação de convecção 1-D é:

\begin{equation}\frac{\partial c}{\partial t} + c \frac{\partial c}{\partial x} = 0\end{equation}

A única diferença com o caso linear é que substituímos a velocidade da onda constante $u$ pela velocidade variável $c$. A equação é não-linear porque agora temos um produto da solução e uma de suas derivadas: o produto $c\partial c / \partial x$. Isso muda tudo!

Vamos utilizar a mesma discretização que aprendemos para o caso de convecção linear: diferenças adiantadas no tempo e atrasadas no espaço. Aqui temos a equação discretizada:

\begin{equation}\frac{c_i^{n+1}-c_i^n}{\Delta t} + c_i^n \frac{c_i^n-c_{i-1}^n}{\Delta x} = 0\end{equation}

Resolvendo para o termos $u_i^{n+1}$, temos a seguinte expressão utilizada para avançar no tempo:

\begin{equation}c_i^{n+1} = c_i^n - c_i^n \frac{\Delta t}{\Delta x} (c_i^n - c_{i-1}^n)\end{equation}

Há muito pouco que precisa mudar do código escrito até agora. Na verdade, até usaremos a mesma condição inicial de onda quadrada. Mas vamos reinicializar a variável `c` com os valores iniciais e redigitar os parâmetros numéricos aqui, por conveniência (já não precisamos mais de $u$).

```python
##problem parameters
nx = 41
dx = 2/(nx-1)
nt = 10
dt = .02

##initial conditions
u = numpy.ones(nx)
u[numpy.intersect1d(lbound, ubound)]=2
```

How does it look?

```python
pyplot.plot(x, u, color='#003366', ls='--', lw=3)
pyplot.ylim(0,2.5);
```

Alterando apenas uma linha de código na solução de convecção linear, podemos agora obter a solução não-linear: a linha que corresponde à equação discreta agora tem `cn[i]` no lugar onde antes tínhamos apenas `u`. Então você poderia escrever algo como:
```Python
for n in range(1,nt):
  un = u.copy()
  for i in range(1,nx):
    u[i] = un[i]-un[i]*dt/dx*(un[i]-un[i-1])
```
Nós vamos ser mais espertos do que isso e usar o NumPy para atualizar todos os valores da grade espacial de uma só vez. Nós realmente não precisamos escrever uma linha de código que seja executada *para cada* valor de $c$ na grade espacial. O Python pode atualizá-los todos de uma vez! Estude o código abaixo e compare-o com o código acima. Aqui está um esboço útil, para ilustrar a operação da matriz - também chamada de operação "vetorizada" - por $c_i-c_{i-1} $.

#### Figura 2: Esboço para explicar a operação de estêncil vetorizado..  Adapitado de ["Indices point between elements"](https://blog.nelhage.com/2015/08/indices-point-between-elements/) by Nelson Elhage.

```python
for n in range(1, nt):
    un = u.copy()
    u[1:] = un[1:]-un[1:]*dt/dx*(un[1:]-un[0:-1])
```

```python
pyplot.plot(x, u, color='#003366', ls='--', lw=3)
pyplot.ylim(0,2.5);
```

Hmm. Isso é bem interessante: como no caso linear, vemos que perdemos os lados afiados da nossa onda quadrada inicial, mas há mais. Agora, a onda também perdeu simetria! Parece estar ficando para trás, enquanto a frente da onda está se inclinando. Essa é outra forma de erro numérico, você pode se perguntar? Não! É física!

####  Pense um pouco!!!

Pense no efeito de ter substituído a velocidade de onda constante $u$ pela velocidade variável dada pela solução $c$. Isso significa que diferentes partes da onda se movem em velocidades diferentes. Faça um esboço de uma onda inicial e pense em onde a velocidade é mais alta e onde ela é mais baixa ...

## Exercício:

Com os parâmetros de solução que inicialmente sugerimos, a malha espacial teve 41 pontos e o *timestep* (passo de tempo) foi de 0,025. Agora, vamos alterar o número de pontos na malha. Escreva o código corresponde ao caso de convecção linear como uma função para que possamos examinar facilmente o que acontece quando ajustamos apenas uma variável: o tamanho da malha. Teste com  *nx=*41; 61; 71; e 85. Veja o que acontece.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```python
def linearconv(nx):
    """Solve the linear convection equation.

    Solves the equation d_t c + u d_x c = 0 where
    * the wavespeed u is set to 1
    * the domain is x \in [0, 5]
    * 20 timesteps are taken, with \Delta t = 0.025
    * the initial data is the hat function

    Produces a plot of the results

    Parameters
    ----------

    nx : integer
        number of internal grid points

    Returns
    -------

    None : none
    """
    dx = 2/(nx-1)
    nt = 25
    dt = .025
    u = 1

    x = numpy.linspace(0,2,nx)

    c = numpy.zeros(nx)
    lbound = numpy.where(x >= 0.5)
    ubound = numpy.where(x <= 1)
    c[numpy.intersect1d(lbound, ubound)]=1

    cn = numpy.zeros(nx)

    for n in range(nt):
        cn = c.copy()
        c[1:] = cn[1:] -u*dt/dx*(cn[1:] -cn[0:-1])
        c[0] = 0.0

    pyplot.plot(x, c, color='#003366', ls='--', lw=3)
    pyplot.ylim(-0.5,1.5);
```

### Vamos calcular!!

```python
linearconv(41) #convection using 60 grid points
```

```python
linearconv(61) #convection using 75 grid points
```

```python
linearconv(71) #convection using 80 grid points
```

```python
linearconv(85) #convection using 81 grid points
```

### O que aconteceu??

###  Vamos ver em mais detalhes!

```python
from IPython.core.display import HTML
css_file = '../styles/custom.css'
HTML(open(css_file, "r").read())
```


# Determine acceleration with Kalman filter for all tests

# Purpose
Yaw rate, Yaw acceleration and Yaw jerk are missing from the model test. These signals can be estimated with Kalman filters. This will be done for all runs in this project.

# Methodology
Loop through all tests and determine the "hidden states" with Kalman filters.

# Setup

```python
# %load imports.py
%matplotlib inline
%load_ext autoreload
%autoreload 2

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp
import seaborn as sns

## Local packages:
from src.data import mdl

```

```python
from src.data import kalman_filter
```

```python
id=22606
df, units, meta_data = mdl.load(id=id)
```

```python
df.head()
```

```python
df1 = kalman_filter.yaw(df=df, observation_covariance=0.5)
df2 = kalman_filter.yaw(df=df, observation_covariance=1000)
df3 = kalman_filter.yaw(df=df, observation_covariance=1000000)

```

```python
epsilon1 = df1['psi']-df1['psi_filtered']
epsilon2 = df2['psi']-df2['psi_filtered']
epsilon3 = df3['psi']-df3['psi_filtered']

fig,ax=plt.subplots()
ax.plot(df.index,  epsilon1  ,label=r'$\epsilon$', alpha=0.5, zorder=3)
ax.plot(df.index,  epsilon2 ,label=r'$\epsilon2$', alpha=0.5, zorder=2)
ax.plot(df.index,  epsilon3 ,label=r'$\epsilon3$', alpha=0.5, zorder=1)
ax.legend()

sns.displot(epsilon1)
sns.displot(epsilon2)
sns.displot(epsilon3)
```

```python
import scipy.stats as stats
fig,ax=plt.subplots()
stats.probplot(epsilon1, dist="norm", plot=ax);

fig,ax=plt.subplots()
stats.probplot(epsilon2, dist="norm", plot=ax);

fig,ax=plt.subplots()
stats.probplot(epsilon3, dist="norm", plot=ax);

```

```python
df = kalman_filter.filter_and_transform(df=df)
```

```python
fig,ax=plt.subplots()
df.plot(y='u', ax=ax)
df.plot(y='u_gradient', alpha=0.5, ax=ax)

fig,ax=plt.subplots()
df.plot(y='u1d', ax=ax)
df.plot(y='u1d_gradient', alpha=0.5, ax=ax)

fig,ax=plt.subplots()
df.plot(y='u1d', alpha=0.5, ax=ax)
```

```python
df_runs = pd.read_csv('../data/raw/runs.csv', index_col=0)
```

```python
save_dir = '../data/processed/kalman'
if not os.path.exists(save_dir):
    os.mkdir(save_dir)

for id, row in df_runs.iterrows():

    print(f'working on:{id}')
    df = mdl.load_run(id=id)
    df = kalman_filter.filter_and_transform(df=df)

    df.index = df.index.total_seconds()
    save_name = f'{id}.csv'
    save_path = os.path.join(save_dir,save_name)
    df.to_csv(save_path)

```

```python
mask = ((~df_runs['sailing']) &
        (
            (df_runs['test_type'] == 'reference speed') |
            (df_runs['test_type'] == 'rodergrundvinkel') |
            ((df_runs['series_number'] == 5) & (df_runs['test_number'] == 1)  & (df_runs['run_number'] == 3)) |
            ((df_runs['series_number'] == 5) & (df_runs['test_number'] == 2)  & (df_runs['run_number'] == 6)) |
            ((df_runs['series_number'] == 5) & (df_runs['test_number'] == 3)  & (df_runs['run_number'] == 2)) |
            ((df_runs['series_number'] == 5) & (df_runs['test_number'] == 4)  & (df_runs['run_number'] == 1)) |
            ((df_runs['series_number'] == 5) & (df_runs['test_number'] == 5)  & (df_runs['run_number'] == 1))

       ))
df_runs_selected = df_runs.loc[mask].copy()
```

```python

```


```python
import numpy as np
```

```python
from sympy import *
```

```python
x1 = Symbol('x1')
x2 = Symbol('x2')
x3 = Symbol('x3')

x1_ = Symbol('x1_')
x2_ = Symbol('x2_')
x3_ = Symbol('x3_')

u1 = Symbol('u1')
u2 = Symbol('u2')

u1_ = Symbol('u1_')
u2_ = Symbol('u2_')

t = Symbol('t')
```

### 1. calculate the expression $\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{d\mathbf{x}}{dt}$ where:
$$f(\mathbf{x}) = f(x_1, x_2) =x_1^2x_2^2 + x_1x_2$$
$$\mathbf{x} = (x_1, x_2)$$
$$x_1(t)  =1-t^2$$
$$x_2(t)  =1+t^2$$

```python
f = x1_**2 * x2_**2 + x1_*x2_
x1 = 1 - t**2
x2 = 1 + t**2
```

```python
np.array([diff(f, x1_), diff(f, x2_)])
```

    array([2*x1_*x2_**2 + x2_, 2*x1_**2*x2_ + x1_], dtype=object)

```python
np.array([diff(x1, t), diff(x2, t)])
```

    array([-2*t, 2*t], dtype=object)

$$\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{d\mathbf{x}}{dt} = [2x_1x_2^2 + x_2, 2x_1^2x_2 + x_1] \begin{bmatrix} -2t \\ 2t \end{bmatrix}$$

```python
df =  np.array([
    diff(f, x1_).subs({x1_:x1, x2_:x2}),
    diff(f, x2_).subs({x1_:x1, x2_:x2})])
dx = np.array([diff(x1, t), diff(x2, t)])

res = np.dot(df,dx)
simplify(res)
```

    t**3*(8*t**4 - 12)

### 2. For the following functions, calculate the expression $\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{d\mathbf{x}}{dt}$ in matrix form, where x = (x1, x2)
$$f(\mathbf{x}) = f(x_1, x_2, x_3) =x_1^3cos(x_2)e^{x_3}$$
$$x_1(t) = 2t$$
$$x_2(t) = 1-t^2$$
$$x_3(t) = e^t$$

```python
f = x1_**3 * cos(x2_) * exp(x3_)
x1 = 2*t
x2 = 1 - t**2
x3 = exp(t)
```

```python
np.array([diff(f, x1_), diff(f, x2_), diff(f, x3_)])
```

    array([3*x1_**2*exp(x3_)*cos(x2_), -x1_**3*exp(x3_)*sin(x2_),
           x1_**3*exp(x3_)*cos(x2_)], dtype=object)

```python
np.array([diff(x1, t), diff(x2, t), diff(x3, t)])
```

    array([2, -2*t, exp(t)], dtype=object)

$$\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{d\mathbf{x}}{dt} = [3x_1^2cos(x_2)e^{x_3}, -x_1^3sin(x_2)e^{x_3}, x_1^3cos(x_2)e^{x_3}] \begin{bmatrix} 2 \\ -2t \\ e^t \end{bmatrix}$$

```python
df = np.array([
        diff(f, x1_).subs({x1_:x1, x2_:x2, x3_:x3}),
        diff(f, x2_).subs({x1_:x1, x2_:x2, x3_:x3}),
        diff(f, x3_).subs({x1_:x1, x2_:x2, x3_:x3})])
dx = np.array([diff(x1, t), diff(x2, t), diff(x3, t)])

res = np.dot(df, dx)
simplify(res)
```

    8*t**2*(-2*t**2*sin(t**2 - 1) + t*exp(t)*cos(t**2 - 1) + 3*cos(t**2 - 1))*exp(exp(t))

### 3. For the following functions, calculate the expression $\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{u}} \frac{d\mathbf{u}}{dt}$ in matrix form, where x=(x1,x2) and u=(u1,u2)
$$f(\mathbf{x}) = f(x_1, x_2) = x_1^2 - x_2^2$$
$$x_1(u_1, u_2) = 2u_1 + 3u_2$$
$$x_2(u_1, u_2) =2u_1 -3u_2$$
$$u_1(t) = cos(t/2)$$
$$u_2(t) = sin(2t)$$

```python
f = x1_**2 - x2_**2
x1 = 2*u1_ + 3*u2_
x2 = 2*u1_ - 3*u2_
u1 = cos(t/2)
u2 = sin(2*t)
```

```python
np.array([diff(f, x1_), diff(f, x2_)])
```

    array([2*x1_, -2*x2_], dtype=object)

```python
np.array([
    [diff(x1, u1_), diff(x1, u2_)],
    [diff(x2, u1_), diff(x2, u2_)]])
```

    array([[2, 3],
           [2, -3]], dtype=object)

```python
np.array([diff(u1, t), diff(u2, t)])
```

    array([-sin(t/2)/2, 2*cos(2*t)], dtype=object)

$$\frac{df}{dt} =\frac{\partial f}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{u}} \frac{d\mathbf{u}}{dt} = [2x_1, -2x_2] \begin{bmatrix} 2 & 3 \\ 2 & -3  \end{bmatrix} \begin{bmatrix} -sin(t/2)/2 \\ 2cos(2t) \end{bmatrix}$$

```python
df = np.array([
        diff(f, x1_).subs({x1_:x1, x2_:x2}).subs({u1_:u1, u2_:u2}),
        diff(f, x2_).subs({x1_:x1, x2_:x2}).subs({u1_:u1, u2_:u2})])
dx = np.array([
    [diff(x1, u1_).subs({u1_:u1, u2_:u2}), diff(x1, u2_).subs({u1_:u1, u2_:u2})],
    [diff(x2, u1_).subs({u1_:u1, u2_:u2}), diff(x2, u2_).subs({u1_:u1, u2_:u2})]])
du = np.array([
    diff(u1, t),
    diff(u2, t)])

res = np.dot(np.dot(df, dx), du)
simplify(res)
```

    18*cos(3*t/2) + 30*cos(5*t/2)

### 4. For the following functions, calculate the expression $\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{u}} \frac{d\mathbf{u}}{dt}$ in matrix form, where x=(x1,x2) and u=(u1,u2)
$$f(\mathbf{x}) = f(x_1, x_2) = cos(x_1)sin(x_2)$$
$$x_1(u_1, u_2) = 2u_1^2 + 3u_2^2 -u_2$$
$$x_2(u_1, u_2) =2u_1 -5u_2^3$$
$$u_1(t) = e^{t/2}$$
$$u_2(t) = e^{-2t}$$

```python
f = cos(x1_) * sin(x2_)
x1 = 2*u1_**2 + 3*u2_**2 - u2_
x2 = 2*u1_ - 5*u2_**3
u1 = exp(t/2)
u2 = exp(-2*t)
```

```python
np.array([diff(f, x1_), diff(f, x2_)])
```

    array([-sin(x1_)*sin(x2_), cos(x1_)*cos(x2_)], dtype=object)

```python
np.array([
    [diff(x1, u1_), diff(x1, u2_)],
    [diff(x2, u1_), diff(x2, u2_)]])
```

    array([[4*u1_, 6*u2_ - 1],
           [2, -15*u2_**2]], dtype=object)

```python
np.array([diff(u1, t), diff(u2, t)])
```

    array([exp(t/2)/2, -2*exp(-2*t)], dtype=object)

$$\frac{df}{dt} =\frac{\partial f}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{u}} \frac{d\mathbf{u}}{dt} = [-sin(x_1)sin(x_2), cos(x_1)cos(x_2)] \begin{bmatrix} 4u_1 & 6u_2 - 1 \\ 2 & -15u_2^2 \end{bmatrix} \begin{bmatrix} e^{t/2}/2 \\ -2e^{-2t} \end{bmatrix}$$

```python
df = np.array([
        diff(f, x1_).subs({x1_:x1, x2_:x2}).subs({u1_:u1, u2_:u2}),
        diff(f, x2_).subs({x1_:x1, x2_:x2}).subs({u1_:u1, u2_:u2})])
dx = np.array([
    [diff(x1, u1_).subs({u1_:u1, u2_:u2}), diff(x1, u2_).subs({u1_:u1, u2_:u2})],
    [diff(x2, u1_).subs({u1_:u1, u2_:u2}), diff(x2, u2_).subs({u1_:u1, u2_:u2})]])
du = np.array([
    diff(u1, t),
    diff(u2, t)])

res = np.dot(np.dot(df, dx), du)
simplify(res)
```

    ((-2*exp(t/2)*sin(2*exp(t/2) - 5*exp(-6*t))*sin(2*exp(t) - exp(-2*t) + 3*exp(-4*t)) + cos(2*exp(t/2) - 5*exp(-6*t))*cos(2*exp(t) - exp(-2*t) + 3*exp(-4*t)))*exp(17*t/2) - 2*(exp(2*t) - 6)*exp(4*t)*sin(2*exp(t/2) - 5*exp(-6*t))*sin(2*exp(t) - exp(-2*t) + 3*exp(-4*t)) + 30*exp(2*t)*cos(2*exp(t/2) - 5*exp(-6*t))*cos(2*exp(t) - exp(-2*t) + 3*exp(-4*t)))*exp(-8*t)

### 5. For the following functions, calculate the expression $\frac{df}{dt} = \frac{\partial f}{\partial \mathbf{x}} \frac{\partial \mathbf{x}}{\partial \mathbf{u}} \frac{d\mathbf{u}}{dt}$ in matrix form, where x=(x1,x2) and u=(u1,u2).
$$f(\mathbf{x}) = f(x_1, x_2, x_3) = sin(x_1)cos(x_2)e^{x_3}$$
$$x_1(u_1, u_2) = sin(u_1) + cos(u_2)$$
$$x_2(u_1, u_2) = cos(u_1) - sin(u_2)$$
$$x_3(u_1, u_2) = e^{u_1 + u_2}$$
$$u_1(t) = 1 + t/2$$
$$u_2(t) = 1 - t/2$$

```python
f = sin(x1_)*cos(x2_)*exp(x3_)
x1 = sin(u1_) + cos(u2_)
x2 = cos(u1_) - sin(u2_)
x3 = exp(u1_ + u2_)
u1 = 1 + t/2
u2 = 1 - t/2
```

```python
np.array([diff(f, x1_), diff(f, x2_), diff(f, x3_)])
```

    array([exp(x3_)*cos(x1_)*cos(x2_), -exp(x3_)*sin(x1_)*sin(x2_),
           exp(x3_)*sin(x1_)*cos(x2_)], dtype=object)

```python
np.array([
    [diff(x1, u1_), diff(x1, u2_)],
    [diff(x2, u1_), diff(x2, u2_)],
    [diff(x3, u1_), diff(x3, u2_)]])
```

    array([[cos(u1_), -sin(u2_)],
           [-sin(u1_), -cos(u2_)],
           [exp(u1_ + u2_), exp(u1_ + u2_)]], dtype=object)

```python
np.array([diff(u1, t), diff(u2, t)])
```

    array([1/2, -1/2], dtype=object)

$$[cos(x_1)cos(x_2)e^{x_3}, -sin(x_1)sin(x_2)e^{x_3}, sin(x_1)cos(x_2)e^{x_3}] \begin{bmatrix} cos(u_1) & -sin(u_2) \\ -sin(u_1) & -cos(u_2) \\ e^{u_1 + u_2} & e^{u_1 + u_2} \end{bmatrix} \begin{bmatrix} 1/2 \\ -1/2 \end{bmatrix}$$

```python
df = np.array([
        diff(f, x1_).subs({x1_:x1, x2_:x2, x3_:x3}).subs({u1_:u1, u2_:u2}),
        diff(f, x2_).subs({x1_:x1, x2_:x2, x3_:x3}).subs({u1_:u1, u2_:u2}),
        diff(f, x3_).subs({x1_:x1, x2_:x2, x3_:x3}).subs({u1_:u1, u2_:u2})])
dx = np.array([
    [diff(x1, u1_).subs({u1_:u1, u2_:u2}), diff(x1, u2_).subs({u1_:u1, u2_:u2})],
    [diff(x2, u1_).subs({u1_:u1, u2_:u2}), diff(x2, u2_).subs({u1_:u1, u2_:u2})],
    [diff(x3, u1_).subs({u1_:u1, u2_:u2}), diff(x3, u2_).subs({u1_:u1, u2_:u2})]])
du = np.array([
    diff(u1, t),
    diff(u2, t)])

res = np.dot(np.dot(df, dx), du)
simplify(res)
```

    (-sin(t/2 - 1)*cos(sin(t/2 - 1) + cos(t/2 + 1))*cos(sin(t/2 + 1) + cos(t/2 - 1)) + sin(t/2 + 1)*sin(sin(t/2 - 1) + cos(t/2 + 1))*sin(sin(t/2 + 1) + cos(t/2 - 1)) - sin(sin(t/2 - 1) + cos(t/2 + 1))*sin(sin(t/2 + 1) + cos(t/2 - 1))*cos(t/2 - 1) + cos(t/2 + 1)*cos(sin(t/2 - 1) + cos(t/2 + 1))*cos(sin(t/2 + 1) + cos(t/2 - 1)))*exp(exp(2))/2

Coursera_Mathematics_for_Machine Learning/2_Mathematics_for_Machine_Learning_Multivariate_Calculus/Week 3 - Multivariate Chain Rule - Neural Network/Multivariate_Chain_Rule.ipynb


Trusted Notebook" align="middle">

# Summary of Quantum Operations

 In this section we will go into the different operations that are available in Qiskit Terra. These are:
- Single-qubit quantum gates
- Multi-qubit quantum gates
- Measurements
- Reset
- Conditionals
- State initialization

We will also show you how to use the three different simulators:
- unitary_simulator
- qasm_simulator
- statevector_simulator

```python
# Useful additional packages
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
from math import pi
```

```python
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister, execute
from qiskit.tools.visualization import circuit_drawer
from qiskit.quantum_info import state_fidelity
from qiskit import BasicAer

backend = BasicAer.get_backend('unitary_simulator')
```

## Single Qubit Quantum states

A single qubit quantum state can be written as

$$\left|\psi\right\rangle = \alpha\left|0\right\rangle + \beta \left|1\right\rangle$$

where $\alpha$ and $\beta$ are complex numbers. In a measurement the probability of the bit being in $\left|0\right\rangle$ is $|\alpha|^2$ and $\left|1\right\rangle$ is $|\beta|^2$. As a vector this is

$$
\left|\psi\right\rangle =
\begin{pmatrix}
\alpha \\
\beta
\end{pmatrix}.
$$

Note due to conservation probability $|\alpha|^2+ |\beta|^2 = 1$ and since global phase is undetectable $\left|\psi\right\rangle := e^{i\delta} \left|\psi\right\rangle$ we only requires two real numbers to describe a single qubit quantum state.

A convenient representation is

$$\left|\psi\right\rangle = \cos(\theta/2)\left|0\right\rangle + \sin(\theta/2)e^{i\phi}\left|1\right\rangle$$

where $0\leq \phi < 2\pi$, and $0\leq \theta \leq \pi$.  From this it is clear that there is a one-to-one correspondence between qubit states ($\mathbb{C}^2$) and the points on the surface of a unit sphere ($\mathbb{R}^3$). This is called the Bloch sphere representation of a qubit state.

Quantum gates/operations are usually represented as matrices. A gate which acts on a qubit is represented by a $2\times 2$ unitary matrix $U$. The action of the quantum gate is found by multiplying the matrix representing the gate with the vector which represents the quantum state.

$$\left|\psi'\right\rangle = U\left|\psi\right\rangle$$

A general unitary must be able to take the $\left|0\right\rangle$ to the above state. That is

$$
U = \begin{pmatrix}
\cos(\theta/2) & a \\
e^{i\phi}\sin(\theta/2) & b
\end{pmatrix}
$$

where $a$ and $b$ are complex numbers constrained such that $U^\dagger U = I$ for all $0\leq\theta\leq\pi$ and $0\leq \phi<2\pi$. This gives 3 constraints and as such $a\rightarrow -e^{i\lambda}\sin(\theta/2)$ and $b\rightarrow e^{i\lambda+i\phi}\cos(\theta/2)$ where $0\leq \lambda<2\pi$ giving

$$
U = \begin{pmatrix}
\cos(\theta/2) & -e^{i\lambda}\sin(\theta/2) \\
e^{i\phi}\sin(\theta/2) & e^{i\lambda+i\phi}\cos(\theta/2)
\end{pmatrix}.
$$

This is the most general form of a single qubit unitary.

## Single-Qubit Gates

The single-qubit gates available are:
- u gates
- Identity gate
- Pauli gates
- Clifford gates
- $C3$ gates
- Standard rotation gates

We have provided a backend: `unitary_simulator` to allow you to calculate the unitary matrices.

```python
q = QuantumRegister(1)
```

### u gates

In Qiskit we give you access to the general unitary using the $u3$ gate

$$
u3(\theta, \phi, \lambda) = U(\theta, \phi, \lambda)
$$

```python
qc = QuantumCircuit(q)
qc.u3(pi/2,pi/2,pi/2,q)
qc.draw()
```

         ┌──────────────────────────┐
q0_0: |0>┤ U3(1.5708,1.5708,1.5708) ├
         └──────────────────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 0.707+0.j   ,  0.   -0.707j],
           [ 0.   +0.707j, -0.707+0.j   ]])

The $u2(\phi, \lambda) =u3(\pi/2, \phi, \lambda)$ has the matrix form

$$
u2(\phi, \lambda) =
\frac{1}{\sqrt{2}} \begin{pmatrix}
1 & -e^{i\lambda} \\
e^{i\phi} & e^{i(\phi + \lambda)}
\end{pmatrix}.
$$

This is a useful gate as it allows us to create superpositions

```python
qc = QuantumCircuit(q)
qc.u2(pi/2,pi/2,q)
qc.draw()
```

         ┌───────────────────┐
q0_0: |0>┤ U2(1.5708,1.5708) ├
         └───────────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 0.707+0.j   ,  0.   -0.707j],
           [ 0.   +0.707j, -0.707+0.j   ]])

The $u1(\lambda)= u3(0, 0, \lambda)$ gate has the matrix form

$$
u1(\lambda) =
\begin{pmatrix}
1 & 0 \\
0 & e^{i \lambda}
\end{pmatrix},
$$

which is a useful as it allows us to apply a quantum phase.

```python
qc = QuantumCircuit(q)
qc.u1(pi/2,q)
qc.draw()
```

         ┌────────────┐
q0_0: |0>┤ U1(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 0.+1.j]])

The $u0(\delta)= u3(0, 0, 0)$ gate is the identity matrix. It has the matrix form

$$
u0(\delta) =
\begin{pmatrix}
1 & 0 \\
0 & 1
\end{pmatrix}.
$$

The identity gate does nothing (but can add noise in the real device for a period of time equal to fractions of the single qubit gate time)

```python
qc = QuantumCircuit(q)
qc.u0(pi/2,q)
qc.draw()
```

         ┌────────────┐
q0_0: |0>┤ U0(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j]])

### Identity gate

The identity gate is $Id = u0(1)$.

```python
qc = QuantumCircuit(q)
qc.iden(q)
qc.draw()
```

         ┌────┐
q0_0: |0>┤ Id ├
         └────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j]])

### Pauli gates

#### $X$: bit-flip gate

The bit-flip gate $X$ is defined as:

$$
X   =
\begin{pmatrix}
0 & 1\\
1 & 0
\end{pmatrix}= u3(\pi,0,\pi)
$$

```python
qc = QuantumCircuit(q)
qc.x(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ X ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[0.+0.j, 1.+0.j],
           [1.+0.j, 0.+0.j]])

#### $Y$: bit- and phase-flip gate

The $Y$ gate is defined as:

$$
Y  =
\begin{pmatrix}
0 & -i\\
i & 0
\end{pmatrix}=u3(\pi,\pi/2,\pi/2)
$$

```python
qc = QuantumCircuit(q)
qc.y(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ Y ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[0.+0.j, 0.-1.j],
           [0.+1.j, 0.+0.j]])

#### $Z$: phase-flip gate

The phase flip gate $Z$ is defined as:

$$
Z =
\begin{pmatrix}
1 & 0\\
0 & -1
\end{pmatrix}=u1(\pi)
$$

```python
qc = QuantumCircuit(q)
qc.z(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ Z ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 1.+0.j,  0.+0.j],
           [ 0.+0.j, -1.+0.j]])

### Clifford gates

#### Hadamard gate

$$
H =
\frac{1}{\sqrt{2}}
\begin{pmatrix}
1 & 1\\
1 & -1
\end{pmatrix}= u2(0,\pi)
$$

```python
qc = QuantumCircuit(q)
qc.h(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ H ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 0.707+0.j,  0.707+0.j],
           [ 0.707+0.j, -0.707+0.j]])

#### $S$ (or, $\sqrt{Z}$ phase) gate

$$
S =
\begin{pmatrix}
1 & 0\\
0 & i
\end{pmatrix}= u1(\pi/2)
$$

```python
qc = QuantumCircuit(q)
qc.s(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ S ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 0.+1.j]])

#### $S^{\dagger}$ (or, conjugate of $\sqrt{Z}$ phase) gate

$$
S^{\dagger} =
\begin{pmatrix}
1 & 0\\
0 & -i
\end{pmatrix}= u1(-\pi/2)
$$

```python
qc = QuantumCircuit(q)
qc.sdg(q)
qc.draw()
```

         ┌─────┐
q0_0: |0>┤ Sdg ├
         └─────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 0.-1.j]])

### $C3$ gates
#### $T$ (or, $\sqrt{S}$ phase) gate

$$
T =
\begin{pmatrix}
1 & 0\\
0 & e^{i \pi/4}
\end{pmatrix}= u1(\pi/4)
$$

```python
qc = QuantumCircuit(q)
qc.t(q)
qc.draw()
```

         ┌───┐
q0_0: |0>┤ T ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.   +0.j   , 0.   +0.j   ],
           [0.   +0.j   , 0.707+0.707j]])

#### $T^{\dagger}$ (or, conjugate of $\sqrt{S}$ phase) gate

$$
T^{\dagger} =
\begin{pmatrix}
1 & 0\\
0 & e^{-i \pi/4}
\end{pmatrix}= u1(-pi/4)
$$

They can be added as below.

```python
qc = QuantumCircuit(q)
qc.tdg(q)
qc.draw()
```

         ┌─────┐
q0_0: |0>┤ Tdg ├
         └─────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.   +0.j   , 0.   +0.j   ],
           [0.   +0.j   , 0.707-0.707j]])

### Standard Rotations

The standard rotation gates are those that define rotations around the Paulis $P=\{X,Y,Z\}$. They are defined as

$$ R_P(\theta) = \exp(-i \theta P/2) = \cos(\theta/2)I -i \sin(\theta/2)P$$

#### Rotation around X-axis

$$
R_x(\theta) =
\begin{pmatrix}
\cos(\theta/2) & -i\sin(\theta/2)\\
-i\sin(\theta/2) & \cos(\theta/2)
\end{pmatrix} = u3(\theta, -\pi/2,\pi/2)
$$

```python
qc = QuantumCircuit(q)
qc.rx(pi/2,q)
qc.draw()
```

         ┌────────────┐
q0_0: |0>┤ Rx(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[0.707+0.j   , 0.   -0.707j],
           [0.   -0.707j, 0.707+0.j   ]])

#### Rotation around Y-axis

$$
R_y(\theta) =
\begin{pmatrix}
\cos(\theta/2) & - \sin(\theta/2)\\
\sin(\theta/2) & \cos(\theta/2).
\end{pmatrix} =u3(\theta,0,0)
$$

```python
qc = QuantumCircuit(q)
qc.ry(pi/2,q)
qc.draw()
```

         ┌────────────┐
q0_0: |0>┤ Ry(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 0.707+0.j, -0.707+0.j],
           [ 0.707+0.j,  0.707+0.j]])

#### Rotation around Z-axis

$$
R_z(\phi) =
\begin{pmatrix}
e^{-i \phi/2} & 0 \\
0 & e^{i \phi/2}
\end{pmatrix}\equiv u1(\phi)
$$

Note here we have used an equivalent as is different to u1 by global phase $e^{-i \phi/2}$.

```python
qc = QuantumCircuit(q)
qc.rz(pi/2,q)
qc.draw()
```

         ┌────────────┐
q0_0: |0>┤ Rz(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j],
           [0.+0.j, 0.+1.j]])

Note this is different due only to a global phase

## Multi-Qubit Gates

### Mathematical Preliminaries

The space of quantum computer grows exponential with the number of qubits. For $n$ qubits the complex vector space has dimensions $d=2^n$. To describe states of a multi-qubit system, the tensor product is used to "glue together" operators and basis vectors.

Let's start by considering a 2-qubit system. Given two operators $A$ and $B$ that each act on one qubit, the joint operator $A \otimes B$ acting on two qubits is

$$\begin{equation}
	A\otimes B =
	\begin{pmatrix}
		A_{00} \begin{pmatrix}
			B_{00} & B_{01} \\
			B_{10} & B_{11}
		\end{pmatrix} & A_{01} 	\begin{pmatrix}
				B_{00} & B_{01} \\
				B_{10} & B_{11}
			\end{pmatrix} \\
		A_{10} 	\begin{pmatrix}
					B_{00} & B_{01} \\
					B_{10} & B_{11}
				\end{pmatrix} & A_{11} 	\begin{pmatrix}
							B_{00} & B_{01} \\
							B_{10} & B_{11}
						\end{pmatrix}
	\end{pmatrix},
\end{equation}$$

where $A_{jk}$ and $B_{lm}$ are the matrix elements of $A$ and $B$, respectively.

Analogously, the basis vectors for the 2-qubit system are formed using the tensor product of basis vectors for a single qubit:
$$\begin{equation}\begin{split}
	\left|{00}\right\rangle &= \begin{pmatrix}
		1 \begin{pmatrix}
			1  \\
			0
		\end{pmatrix} \\
		0 \begin{pmatrix}
			1  \\
			0
		\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 0 \\0 \end{pmatrix}~~~\left|{01}\right\rangle = \begin{pmatrix}
	1 \begin{pmatrix}
	0 \\
	1
	\end{pmatrix} \\
	0 \begin{pmatrix}
	0  \\
	1
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix}0 \\ 1 \\ 0 \\ 0 \end{pmatrix}\end{split}
\end{equation}$$

$$\begin{equation}\begin{split}\left|{10}\right\rangle = \begin{pmatrix}
	0\begin{pmatrix}
	1  \\
	0
	\end{pmatrix} \\
	1\begin{pmatrix}
	1 \\
	0
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix}~~~ 	\left|{11}\right\rangle = \begin{pmatrix}
	0 \begin{pmatrix}
	0  \\
	1
	\end{pmatrix} \\
	1\begin{pmatrix}
	0  \\
	1
	\end{pmatrix}
	\end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\1 \end{pmatrix}\end{split}
\end{equation}.$$

Note we've introduced a shorthand for the tensor product of basis vectors, wherein $\left|0\right\rangle \otimes \left|0\right\rangle$ is written as $\left|00\right\rangle$. The state of an $n$-qubit system can described using the $n$-fold tensor product of single-qubit basis vectors. Notice that the basis vectors for a 2-qubit system are 4-dimensional; in general, the basis vectors of an $n$-qubit sytsem are $2^{n}$-dimensional, as noted earlier.

### Basis vector ordering in Qiskit

Within the physics community, the qubits of a multi-qubit systems are typically ordered with the first qubit on the left-most side of the tensor product and the last qubit on the right-most side. For instance, if the first qubit is in state $\left|0\right\rangle$ and second is in state $\left|1\right\rangle$, their joint state would be $\left|01\right\rangle$. Qiskit uses a slightly different ordering of the qubits, in which the qubits are represented from the most significant bit (MSB) on the left to the least significant bit (LSB) on the right (big-endian). This is similar to bitstring representation on classical computers, and enables easy conversion from bitstrings to integers after measurements are performed. For the example just given, the joint state would be represented as $\left|10\right\rangle$. Importantly, _this change in the representation of multi-qubit states affects the way multi-qubit gates are represented in Qiskit_, as discussed below.

The representation used in Qiskit enumerates the basis vectors in increasing order of the integers they represent. For instance, the basis vectors for a 2-qubit system would be ordered as $\left|00\right\rangle$, $\left|01\right\rangle$, $\left|10\right\rangle$, and $\left|11\right\rangle$. Thinking of the basis vectors as bit strings, they encode the integers 0,1,2 and 3, respectively.

### Controlled operations on qubits

A common multi-qubit gate involves the application of a gate to one qubit, conditioned on the state of another qubit. For instance, we might want to flip the state of the second qubit when the first qubit is in $\left|0\right\rangle$. Such gates are known as _controlled gates_. The standard multi-qubit gates consist of two-qubit gates and three-qubit gates. The two-qubit gates are:
- controlled Pauli gates
- controlled Hadamard gate
- controlled rotation gates
- controlled phase gate
- controlled u3 gate
- swap gate

The three-qubit gates are:
- Toffoli gate
- Fredkin gate

## Two-qubit gates

Most of the two-gates are of the controlled type (the SWAP gate being the exception). In general, a controlled two-qubit gate $C_{U}$ acts to apply the single-qubit unitary $U$ to the second qubit when the state of the first qubit is in $\left|1\right\rangle$. Suppose $U$ has a matrix representation

$$U = \begin{pmatrix} u_{00} & u_{01} \\ u_{10} & u_{11}\end{pmatrix}.$$

We can work out the action of $C_{U}$ as follows. Recall that the basis vectors for a two-qubit system are ordered as $\left|00\right\rangle, \left|01\right\rangle, \left|10\right\rangle, \left|11\right\rangle$. Suppose the **control qubit** is **qubit 0** (which, according to Qiskit's convention, is one the _right-hand_ side of the tensor product). If the control qubit is in $\left|1\right\rangle$, $U$ should be applied to the **target** (qubit 1, on the _left-hand_ side of the tensor product). Therefore, under the action of $C_{U}$, the basis vectors are transformed according to

$$\begin{align*}
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{U\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{U\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
\end{align*}.$$

In matrix form, the action of $C_{U}$ is

$$\begin{equation}
	C_U = \begin{pmatrix}
	1 & 0 & 0 & 0 \\
	0 & u_{00} & 0 & u_{01} \\
	0 & 0 & 1 & 0 \\
	0 & u_{10} &0 & u_{11}
		\end{pmatrix}.
\end{equation}$$

To work out these matrix elements, let

$$C_{(jk), (lm)} = \left(\underset{\text{qubit}~1}{\left\langle j \right|} \otimes \underset{\text{qubit}~0}{\left\langle k \right|}\right) C_{U} \left(\underset{\text{qubit}~1}{\left| l \right\rangle} \otimes \underset{\text{qubit}~0}{\left| k \right\rangle}\right),$$

compute the action of $C_{U}$ (given above), and compute the inner products.

As shown in the examples below, this operation is implemented in Qiskit as `cU(q[0],q[1])`.

If **qubit 1 is the control and qubit 0 is the target**, then the basis vectors are transformed according to
$$\begin{align*}
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|0\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|0\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{U\left|0\right\rangle}\\
C_{U}: \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{\left|1\right\rangle} &\rightarrow \underset{\text{qubit}~1}{\left|1\right\rangle}\otimes \underset{\text{qubit}~0}{U\left|1\right\rangle}\\
\end{align*},$$

which implies the matrix form of $C_{U}$ is
$$\begin{equation}
	C_U = \begin{pmatrix}
	1 & 0 & 0  & 0 \\
	0 & 1 & 0 & 0 \\
	0 & 0 & u_{00} & u_{01} \\
	0 & 0 & u_{10} & u_{11}
		\end{pmatrix}.
\end{equation}$$

```python
q = QuantumRegister(2)
```

### Controlled Pauli Gates

#### Controlled-X (or, controlled-NOT) gate
The controlled-not gate flips the `target` qubit when the control qubit is in the state $\left|1\right\rangle$. If we take the MSB as the control qubit (e.g. `cx(q[1],q[0])`), then the matrix would look like

$$
C_X =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0
\end{pmatrix}.
$$

However, when the LSB is the control qubit, (e.g. `cx(q[0],q[1])`), this gate is equivalent to the following matrix:

$$
C_X =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0\\
0 & 1 & 0 & 0
\end{pmatrix}.
$$

```python
qc = QuantumCircuit(q)
qc.cx(q[0],q[1])
qc.draw()
```

q1_0: |0>──■──
         ┌─┴─┐
q1_1: |0>┤ X ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j]])

#### Controlled $Y$ gate

Apply the $Y$ gate to the target qubit if the control qubit is the MSB

$$
C_Y =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & -i\\
0 & 0 & i & 0
\end{pmatrix},
$$

or when the LSB is the control

$$
C_Y =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 0 & -i\\
0 & 0 & 1 & 0\\
0 & i & 0 & 0
\end{pmatrix}.
$$

```python
qc = QuantumCircuit(q)
qc.cy(q[0],q[1])
qc.draw()
```

q1_0: |0>──■──
         ┌─┴─┐
q1_1: |0>┤ Y ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.-1.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 0.+1.j, 0.+0.j, 0.+0.j]])

#### Controlled $Z$ (or, controlled Phase) gate

Similarly, the controlled Z gate flips the phase of the target qubit if the control qubit is $\left|1\right\rangle$. The matrix looks the same regardless of whether the MSB or LSB is the control qubit:

$$
C_Z =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & -1
\end{pmatrix}
$$

```python
qc = QuantumCircuit(q)
qc.cz(q[0],q[1])
qc.draw()
```

q1_0: |0>─■─
          │
q1_1: |0>─■─

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 1.+0.j,  0.+0.j,  0.+0.j,  0.+0.j],
           [ 0.+0.j,  1.+0.j,  0.+0.j,  0.+0.j],
           [ 0.+0.j,  0.+0.j,  1.+0.j,  0.+0.j],
           [ 0.+0.j,  0.+0.j,  0.+0.j, -1.+0.j]])

### Controlled Hadamard gate

Apply $H$ gate to the target qubit if the control qubit is $\left|1\right\rangle$. Below is the case where the control is the LSB qubit.

$$
C_H =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}}\\
0 & 0 & 1 & 0\\
0 & \frac{1}{\sqrt{2}}  & 0& -\frac{1}{\sqrt{2}}
\end{pmatrix}
$$

```python
qc = QuantumCircuit(q)
qc.ch(q[0],q[1])
qc.draw()
```

q1_0: |0>──■──
         ┌─┴─┐
q1_1: |0>┤ H ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 0.707+0.707j,  0.   +0.j   ,  0.   +0.j   ,  0.   +0.j   ],
           [ 0.   +0.j   ,  0.5  +0.5j  ,  0.   +0.j   ,  0.5  +0.5j  ],
           [ 0.   +0.j   ,  0.   +0.j   ,  0.707+0.707j,  0.   +0.j   ],
           [ 0.   +0.j   ,  0.5  +0.5j  ,  0.   +0.j   , -0.5  -0.5j  ]])

### Controlled rotation gates

#### Controlled rotation around Z-axis

Perform rotation around Z-axis on the target qubit if the control qubit (here LSB) is $\left|1\right\rangle$.

$$
C_{Rz}(\lambda) =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & e^{-i\lambda/2} & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & e^{i\lambda/2}
\end{pmatrix}
$$

```python
qc = QuantumCircuit(q)
qc.crz(pi/2,q[0],q[1])
qc.draw()
```

q1_0: |0>──────■───────
         ┌─────┴──────┐
q1_1: |0>┤ Rz(1.5708) ├
         └────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.   +0.j   , 0.   +0.j   , 0.   +0.j   , 0.   +0.j   ],
           [0.   +0.j   , 0.707-0.707j, 0.   +0.j   , 0.   +0.j   ],
           [0.   +0.j   , 0.   +0.j   , 1.   +0.j   , 0.   +0.j   ],
           [0.   +0.j   , 0.   +0.j   , 0.   +0.j   , 0.707+0.707j]])

### Controlled phase rotation

Perform a phase rotation if both qubits are in the $\left|11\right\rangle$ state. The matrix looks the same regardless of whether the MSB or LSB is the control qubit.

$$
C_{u1}(\lambda) =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & e^{i\lambda}
\end{pmatrix}
$$

```python
qc = QuantumCircuit(q)
qc.cu1(pi/2,q[0], q[1])
qc.draw()
```

q1_0: |0>─■───────
          │1.5708
q1_1: |0>─■───────

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+1.j]])

### Controlled $u3$ rotation

Perform controlled-$u3$ rotation on the target qubit if the control qubit (here LSB) is $\left|1\right\rangle$.

$$
C_{u3}(\theta, \phi, \lambda) \equiv
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & e^{-i(\phi+\lambda)/2}\cos(\theta/2) & 0 & -e^{-i(\phi-\lambda)/2}\sin(\theta/2)\\
0 & 0 & 1 & 0\\
0 & e^{i(\phi-\lambda)/2}\sin(\theta/2) & 0 & e^{i(\phi+\lambda)/2}\cos(\theta/2)
\end{pmatrix}.
$$

```python
qc = QuantumCircuit(q)
qc.cu3(pi/2, pi/2, pi/2, q[0], q[1])
qc.draw()
```

q1_0: |0>─────────────■──────────────
         ┌────────────┴─────────────┐
q1_1: |0>┤ U3(1.5708,1.5708,1.5708) ├
         └──────────────────────────┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[ 1.   +0.j   ,  0.   +0.j   ,  0.   +0.j   ,  0.   +0.j   ],
           [ 0.   +0.j   ,  0.   -0.707j,  0.   +0.j   , -0.707+0.j   ],
           [ 0.   +0.j   ,  0.   +0.j   ,  1.   +0.j   ,  0.   +0.j   ],
           [ 0.   +0.j   ,  0.707+0.j   ,  0.   +0.j   ,  0.   +0.707j]])

### SWAP gate

The SWAP gate exchanges the two qubits. It transforms the basis vectors as

$$\left|00\right\rangle \rightarrow \left|00\right\rangle~,~\left|01\right\rangle \rightarrow \left|10\right\rangle~,~\left|10\right\rangle \rightarrow \left|01\right\rangle~,~\left|11\right\rangle \rightarrow \left|11\right\rangle,$$

which gives a matrix representation of the form

$$
\mathrm{SWAP} =
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 0 & 1
\end{pmatrix}.
$$

```python
qc = QuantumCircuit(q)
qc.swap(q[0], q[1])
qc.draw()
```

q1_0: |0>─X─
          │
q1_1: |0>─X─

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j]])

## Three-qubit gates

There are two commonly-used three-qubit gates. For three qubits, the basis vectors are ordered as

$$\left|000\right\rangle, \left|001\right\rangle, \left|010\right\rangle, \left|011\right\rangle, \left|100\right\rangle, \left|101\right\rangle, \left|110\right\rangle, \left|111\right\rangle,$$

which, as bitstrings, represent the integers $0,1,2,\cdots, 7$. Again, Qiskit uses a representation in which the first qubit is on the right-most side of the tensor product and the third qubit is on the left-most side:

$$\left|abc\right\rangle : \underset{\text{qubit 2}}{\left|a\right\rangle}\otimes \underset{\text{qubit 1}}{\left|b\right\rangle}\otimes \underset{\text{qubit 0}}{\left|c\right\rangle}.$$

### Toffoli gate ($ccx$ gate)

The [Toffoli gate](https://en.wikipedia.org/wiki/Quantum_logic_gate#Toffoli_(CCNOT)_gate) flips the third qubit if the first two qubits (LSB) are both $\left|1\right\rangle$:

$$\left|abc\right\rangle \rightarrow \left|bc\oplus a\right\rangle \otimes \left|b\right\rangle \otimes \left|c\right\rangle.$$

In matrix form, the Toffoli gate is
$$
C_{CX} =
\begin{pmatrix}
1 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\
0 & 1 & 0 & 0 & 0 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\
0 & 0 & 0 & 0 & 1 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 1 & 0 & 0 & 0 & 0
\end{pmatrix}.
$$

```python
q = QuantumRegister(3)
```

```python
qc = QuantumCircuit(q)
qc.ccx(q[0], q[1], q[2])
qc.draw()
```

q2_0: |0>──■──
           │
q2_1: |0>──■──
         ┌─┴─┐
q2_2: |0>┤ X ├
         └───┘

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j]])

### Controlled swap gate (Fredkin Gate)

The [Fredkin gate](https://en.wikipedia.org/wiki/Quantum_logic_gate#Fredkin_(CSWAP)_gate), or the _controlled swap gate_, exchanges the second and third qubits if the first qubit (LSB) is $\left|1\right\rangle$:

$$ \left|abc\right\rangle \rightarrow \begin{cases} \left|bac\right\rangle~~\text{if}~c=1 \cr \left|abc\right\rangle~~\text{if}~c=0 \end{cases}.$$

In matrix form, the Fredkin gate is

$$
C_{\mathrm{SWAP}} =
\begin{pmatrix}
1 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\
0 & 1 & 0 & 0 & 0 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 0 & 1 & 0 & 0 & 0\\
0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1
\end{pmatrix}.
$$

```python
qc = QuantumCircuit(q)
qc.cswap(q[0], q[1], q[2])
qc.draw()
```

q2_0: |0>─■─
          │
q2_1: |0>─X─
          │
q2_2: |0>─X─

```python
job = execute(qc, backend)
job.result().get_unitary(qc, decimals=3)
```

    array([[1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j, 0.+0.j],
           [0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 1.+0.j]])

## Non unitary operations

Now we have gone through all the unitary operations in quantum circuits we also have access to non-unitary operations. These include measurements, reset of qubits, and classical conditional operations.

```python
q = QuantumRegister(1)
c = ClassicalRegister(1)
```

### Measurements

We don't have access to all the information when we make a measurement in a quantum computer. The quantum state is projected onto the standard basis. Below are two examples showing a circuit that is prepared in a basis state and the quantum computer prepared in a superposition state.

```python
qc = QuantumCircuit(q, c)
qc.measure(q, c)
qc.draw()
```

         ┌─┐
q3_0: |0>┤M├
         └╥┘
 c0_0: 0 ═╩═

```python
backend = BasicAer.get_backend('qasm_simulator')
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'0': 1024}

 The simulator predicts that 100 percent of the time the classical register returns 0.

```python
qc = QuantumCircuit(q, c)
qc.h(q)
qc.measure(q, c)
qc.draw()
```

         ┌───┐┌─┐
q3_0: |0>┤ H ├┤M├
         └───┘└╥┘
 c0_0: 0 ══════╩═

```python
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'0': 513, '1': 511}

 The simulator predicts that 50 percent of the time the classical register returns 0 or 1.

### Reset
It is also possible to `reset` qubits to the $\left|0\right\rangle$ state in the middle of computation. Note that `reset` is not a Gate operation, since it is irreversible.

```python
qc = QuantumCircuit(q, c)
qc.reset(q[0])
qc.measure(q, c)
qc.draw()
```

              ┌─┐
q3_0: |0>─|0>─┤M├
              └╥┘
 c0_0: 0 ══════╩═

```python
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'0': 1024}

```python
qc = QuantumCircuit(q, c)
qc.h(q)
qc.reset(q[0])
qc.measure(q, c)
qc.draw()
```

         ┌───┐     ┌─┐
q3_0: |0>┤ H ├─|0>─┤M├
         └───┘     └╥┘
 c0_0: 0 ═══════════╩═

```python
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'0': 1024}

Here we see that for both of these circuits the simulator always predicts that the output is 100 percent in the 0 state.

### Conditional operations
It is also possible to do operations conditioned on the state of the classical register

```python
qc = QuantumCircuit(q, c)
qc.x(q[0]).c_if(c, 0)
qc.measure(q,c)
qc.draw()
```

         ┌─────┐┌─┐
q3_0: |0>┤  X  ├┤M├
         ├──┴──┤└╥┘
 c0_0: 0 ╡ = 0 ╞═╩═
         └─────┘

Here the classical bit always takes the value 0 so the qubit state is always flipped.

```python
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'1': 1024}

```python
qc = QuantumCircuit(q, c)
qc.h(q)
qc.measure(q,c)
qc.x(q[0]).c_if(c, 0)
qc.measure(q,c)
qc.draw()
```

         ┌───┐┌─┐┌─────┐┌─┐
q3_0: |0>┤ H ├┤M├┤  X  ├┤M├
         └───┘└╥┘├──┴──┤└╥┘
 c0_0: 0 ══════╩═╡ = 0 ╞═╩═
                 └─────┘

```python
job = execute(qc, backend, shots=1024)
job.result().get_counts(qc)
```

    {'1': 1024}

Here the classical bit by the first measurement is random but the conditional operation results in the qubit being deterministically put into $\left|1\right\rangle$.

### Arbitrary initialization
What if we want to initialize a qubit register to an arbitrary state? An arbitrary state for $n$ qubits may be specified by a vector of $2^n$ amplitudes, where the sum of amplitude-norms-squared equals 1. For example, the following three-qubit state can be prepared:

$$\left|\psi\right\rangle = \frac{i}{4}\left|000\right\rangle + \frac{1}{\sqrt{8}}\left|001\right\rangle + \frac{1+i}{4}\left|010\right\rangle + \frac{1+2i}{\sqrt{8}}\left|101\right\rangle + \frac{1}{4}\left|110\right\rangle$$

```python
# Initializing a three-qubit quantum state
import math
desired_vector = [
    1 / math.sqrt(16) * complex(0, 1),
    1 / math.sqrt(8) * complex(1, 0),
    1 / math.sqrt(16) * complex(1, 1),
    0,
    0,
    1 / math.sqrt(8) * complex(1, 2),
    1 / math.sqrt(16) * complex(1, 0),
    0]

q = QuantumRegister(3)

qc = QuantumCircuit(q)

qc.initialize(desired_vector, [q[0],q[1],q[2]])
qc.draw(output='latex')
```

```python
backend = BasicAer.get_backend('statevector_simulator')
job = execute(qc, backend)
qc_state = job.result().get_statevector(qc)
qc_state
```

    array([0.25      +0.j        , 0.        -0.35355339j,
           0.25      -0.25j      , 0.        +0.j        ,
           0.        +0.j        , 0.70710678-0.35355339j,
           0.        -0.25j      , 0.        +0.j        ])

[Fidelity](https://en.wikipedia.org/wiki/Fidelity_of_quantum_states) is useful to check whether two states are same or not.
For quantum (pure) states $\left|\psi_1\right\rangle$ and $\left|\psi_2\right\rangle$, the fidelity is

$$
F\left(\left|\psi_1\right\rangle,\left|\psi_2\right\rangle\right) = \left|\left\langle\psi_1\middle|\psi_2\right\rangle\right|^2.
$$

The fidelity is equal to $1$ if and only if two states are same.

```python
state_fidelity(desired_vector,qc_state)
```

    1.0

#### Further details:

How does the desired state get generated behind the scenes? There are multiple methods for doing this. Qiskit uses a [method proposed by Shende et al](https://arxiv.org/abs/quant-ph/0406176). Here, the idea is to assume the quantum register to have started from our desired state, and construct a circuit that takes it to the $\left|00..0\right\rangle$ state. The initialization circuit is then the reverse of such circuit.

To take an arbitrary quantum state to the zero state in the computational basis, we perform an iterative procedure that disentangles qubits from the register one-by-one. We know that any arbitrary single-qubit state $\left|\rho\right\rangle$ can be taken to the $\left|0\right\rangle$ state using a $\phi$-degree rotation about the Z axis followed by a $\theta$-degree rotation about the Y axis:

$$R_y(-\theta)R_z(-\phi)\left|\rho\right\rangle = re^{it}\left|0\right\rangle$$

Since now we are dealing with $n$ qubits instead of just 1, we must factorize the state vector to separate the Least Significant Bit (LSB):

$$\begin{align*}
 \left|\psi\right\rangle =& \alpha_{0_0}\left|00..00\right\rangle + \alpha_{0_1}\left|00..01\right\rangle + \alpha_{1_0}\left|00..10\right\rangle + \alpha_{1_1}\left|00..11\right\rangle + ... \\&+ \alpha_{(2^{n-1}-1)_0}\left|11..10\right\rangle + \alpha_{(2^{n-1}-1)_1}\left|11..11\right\rangle \\
=& \left|00..0\right\rangle (\alpha_{0_0}\left|0\right\rangle + \alpha_{0_1}\left|1\right\rangle) + \left|00..1\right\rangle (\alpha_{1_0}\left|0\right\rangle + \alpha_{1_1}\left|1\right\rangle) + ... \\&+ \left|11..1\right\rangle (\alpha_{(2^{n-1}-1)_0}(\left|0\right\rangle + \alpha_{(2^{n-1}-1)_1}\left|1\right\rangle) \\
=& \left|00..0\right\rangle\left|\rho_0\right\rangle + \left|00..1\right\rangle\left|\rho_1\right\rangle + ... + \left|11..1\right\rangle\left|\rho_{2^{n-1}-1}\right\rangle
\end{align*}$$

Now each of the single-qubit states $\left|\rho_0\right\rangle, ..., \left|\rho_{2^{n-1}-1}\right\rangle$ can be taken to $\left|0\right\rangle$ by finding appropriate $\phi$ and $\theta$ angles per the equation above. Doing this simultaneously on all states amounts to the following unitary, which disentangles the LSB:

$$U = \begin{pmatrix}
R_{y}(-\theta_0)R_{z}(-\phi_0) & & & &\\
& R_{y}(-\theta_1)R_{z}(-\phi_1) & & &\\
& . & & &\\
& & . & &\\
& & & & R_y(-\theta_{2^{n-1}-1})R_z(-\phi_{2^{n-1}-1})
\end{pmatrix} $$

Hence,

$$U\left|\psi\right\rangle = \begin{pmatrix} r_0e^{it_0}\\ r_1e^{it_1}\\ . \\ . \\ r_{2^{n-1}-1}e^{it_{2^{n-1}-1}} \end{pmatrix}\otimes\left|0\right\rangle$$

U can be implemented as a "quantum multiplexor" gate, since it is a block diagonal matrix. In the quantum multiplexor formalism, a block diagonal matrix of size $2^n \times 2^n$, and consisting of $2^s$ blocks, is equivalent to a multiplexor with $s$ select qubits and $n-s$ data qubits. Depending on the state of the select qubits, the corresponding blocks are applied to the data qubits. A multiplexor of this kind can be implemented after recursive decomposition to primitive gates of cx, rz and ry.