CTC（ Connectionist Temporal Classification，连接时序分类）是一种用于序列建模的工具，其核心是定义了特殊的**目标函数/优化准则**[1]。

> jupyter notebook 版见 [repo](https://github.com/DingKe/ml-tutorial/tree/master/ctc).

# 1. 算法
这里大体根据 Alex Grave 的开山之作[1]，讨论 CTC 的算法原理，并基于 numpy 从零实现 CTC 的推理及训练算法。

## 1.1 序列问题形式化。
序列问题可以形式化为如下函数：

$$N_w: (R^m)^T \rightarrow (R^n)^T$$
其中，序列目标为字符串（词表大小为 $n$），即 $N_w$ 输出为 $n$ 维多项概率分布（e.g. 经过 softmax 处理）。

网络输出为：$y = N_w(x)$，其中，$y_k^t$ $t$ 表示时刻第 $k$ 项的概率。

**图1. 序列建模【[src](https://distill.pub/2017/ctc/)】**

虽然并没为限定 $N_w$ 具体形式，下面为假设其了某种神经网络（e.g. RNN）。
下面代码示例 toy $N_w$：

```python
import numpy as np

np.random.seed(1111)

T, V = 12, 5
m, n = 6, V

x = np.random.random([T, m])  # T x m
w = np.random.random([m, n])  # weights, m x n

def softmax(logits):
    max_value = np.max(logits, axis=1, keepdims=True)
    exp = np.exp(logits - max_value)
    exp_sum = np.sum(exp, axis=1, keepdims=True)
    dist = exp / exp_sum
    return dist

def toy_nw(x):
    y = np.matmul(x, w)  # T x n
    y = softmax(y)
    return y

y = toy_nw(x)
print(y)
print(y.sum(1, keepdims=True))
```

    [[ 0.24654511  0.18837589  0.16937668  0.16757465  0.22812766]
     [ 0.25443629  0.14992236  0.22945293  0.17240658  0.19378184]
     [ 0.24134404  0.17179604  0.23572466  0.12994237  0.22119288]
     [ 0.27216255  0.13054313  0.2679252   0.14184499  0.18752413]
     [ 0.32558002  0.13485564  0.25228604  0.09743785  0.18984045]
     [ 0.23855586  0.14800386  0.23100255  0.17158135  0.21085638]
     [ 0.38534786  0.11524603  0.18220093  0.14617864  0.17102655]
     [ 0.21867406  0.18511892  0.21305488  0.16472572  0.21842642]
     [ 0.29856607  0.13646801  0.27196606  0.11562552  0.17737434]
     [ 0.242347    0.14102063  0.21716951  0.2355229   0.16393996]
     [ 0.26597326  0.10009752  0.23362892  0.24560198  0.15469832]
     [ 0.23337289  0.11918746  0.28540761  0.20197928  0.16005275]]
    [[ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]
     [ 1.]]

## 1.2 align-free 变长映射
上面的形式是输入和输出的一对一的映射。序列学习任务一般而言是多对多的映射关系（如语音识别中，上百帧输出可能仅对应若干音节或字符，并且每个输入和输出之间，也没有清楚的对应关系）。CTC 通过引入一个特殊的 blank 字符（用 % 表示），解决多对一映射问题。

扩展原始词表 $L$ 为 $L^\prime = L \cup \{\text{blank}\}$。对输出字符串，定义操作 $B$：1）合并连续的相同符号；2）去掉 blank 字符。

例如，对于 “aa%bb%%cc”，应用 $B$，则实际上代表的是字符串 "abc"。同理“%a%b%cc%” 也同样代表 "abc"。
$$
B(aa\%bb\%\%cc) = B(\%a\%b\%cc\%) = abc
$$

通过引入blank 及 $B$，可以实现了变长的映射。
$$
L^{\prime T} \rightarrow L^{\le T}
$$

	因为这个原因，CTC 只能建模输出长度小于输入长度的序列问题。

## 1.3 似然计算
和大多数有监督学习一样，CTC 使用最大似然标准进行训练。

给定输入 $x$，输出 $l$ 的条件概率为：
$$
p(l|x) =  \sum_{\pi \in B^{-1}(l)} p(\pi|x)
$$

其中，$B^{-1}(l)$ 表示了长度为 $T$ 且示经过 $B$ 结果为 $l$ 字符串的集合。

**CTC 假设输出的概率是（相对于输入）条件独立的**，因此有：
$$p(\pi|x) = \prod y^t_{\pi_t}, \forall \pi \in L^{\prime T}$$

然而，直接按上式我们没有办理有效的计算似然值。下面用动态规划解决似然的计算及梯度计算, 涉及前向算法和后向算法。

## 1.4 前向算法

在前向及后向计算中，CTC 需要将输出字符串进行扩展。具体的，$(a_1,\cdots,a_m)$ 每个字符之间及首尾分别插入 blank，即扩展为 $(\%, a_1,\%,a_2, \%,\cdots,\%, a_m,\%)$。下面的 $l$ 为原始字符串，$l^\prime$ 指为扩展后的字符串。

定义
$$
\alpha_t(s) \stackrel{def}{=} \sum_{\pi \in N^T: B(\pi_{1:t}) = l_{1:s}} \prod_{t^\prime=1}^t y^t_{\pi^\prime}
$$

显然有，

$$
\begin{align}
\alpha_1(1) = y_b^1,\\
\alpha_1(2) = y_{l_1}^1,\\
\alpha_1(s) = 0, \forall s > 2
\end{align}
$$
根据 $\alpha$ 的定义，有如下递归关系：
$$
\alpha_t(s) = \{  \begin{array}{l}
(\alpha_{t-1}(s)+\alpha_{t-1}(s-1)) y^t_{l^\prime_s},\  \  \    if\  l^\prime_s = b \ or\  l_{s-2}^\prime = l_s^{\prime}  \\
(\alpha_{t-1}(s)+\alpha_{t-1}(s-1) + \alpha_{t-1}(s-2)) y^t_{l^\prime_s} \ \ otherwise
\end{array}
$$

### 1.4.1 Case 2
递归公式中 case 2 是一般的情形。如图所示，$t$ 时刻字符为 $s$ 为 blank 时，它可能由于两种情况扩展而来：1）重复上一字符，即上个字符也是 a，2）字符发生转换，即上个字符是非 a 的字符。第二种情况又分为两种情形，2.1）上一字符是 blank；2.2）a 由非 blank 字符直接跳转而来（$B$） 操作中， blank 最终会被去掉，因此 blank 并不是必须的）。

**图2. 前向算法 Case 2 示例【[src](https://distill.pub/2017/ctc/)】**

### 1.4.2 Case 1
递归公式 case 1 是特殊的情形。
如图所示，$t$ 时刻字符为 $s$ 为 blank 时，它只能由于两种情况扩展而来：1）重复上一字符，即上个字符也是 blank，2）字符发生转换，即上个字符是非 blank 字符。$t$ 时刻字符为 $s$ 为非 blank 时，类似于 case 2，但是这时两个相同字符之间的 blank 不能省略（否则无法区分"aa"和"a"），因此，也只有两种跳转情况。

**图3. 前向算法 Case 1 【[src](https://distill.pub/2017/ctc/)】**

我们可以利用动态规划计算所有 $\alpha$ 的值，算法时间和空间复杂度为 $O(T * L)$。

似然的计算只涉及乘加运算，因此，CTC 的似然是可导的，可以尝试 tensorflow 或 pytorch 等具有自动求导功能的工具自动进行梯度计算。下面介绍如何手动高效的计算梯度。

```python
def forward(y, labels):
    T, V = y.shape
    L = len(labels)
    alpha = np.zeros([T, L])

    # init
    alpha[0, 0] = y[0, labels[0]]
    alpha[0, 1] = y[0, labels[1]]

    for t in range(1, T):
        for i in range(L):
            s = labels[i]

            a = alpha[t - 1, i]
            if i - 1 >= 0:
                a += alpha[t - 1, i - 1]
            if i - 2 >= 0 and s != 0 and s != labels[i - 2]:
                a += alpha[t - 1, i - 2]

            alpha[t, i] = a * y[t, s]

    return alpha

labels = [0, 3, 0, 3, 0, 4, 0]  # 0 for blank
alpha = forward(y, labels)
print(alpha)
```

    [[  2.46545113e-01   1.67574654e-01   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  6.27300235e-02   7.13969720e-02   4.26370730e-02   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  1.51395174e-02   1.74287803e-02   2.75214373e-02   5.54036251e-03
        0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  4.12040964e-03   4.61964998e-03   1.22337658e-02   4.68965079e-03
        1.50787918e-03   1.03895167e-03   0.00000000e+00]
     [  1.34152305e-03   8.51612635e-04   5.48713543e-03   1.64898136e-03
        2.01779193e-03   1.37377693e-03   3.38261905e-04]
     [  3.20028190e-04   3.76301179e-04   1.51214552e-03   1.22442454e-03
        8.74730268e-04   1.06283215e-03   4.08416903e-04]
     [  1.23322177e-04   1.01788478e-04   7.27708889e-04   4.00028082e-04
        8.08904808e-04   5.40783712e-04   5.66942671e-04]
     [  2.69673617e-05   3.70815141e-05   1.81389560e-04   1.85767281e-04
        2.64362267e-04   3.82184328e-04   2.42231029e-04]
     [  8.05153930e-06   7.40568461e-06   6.52280509e-05   4.24527009e-05
        1.34393412e-04   1.47631121e-04   1.86429242e-04]
     [  1.95126637e-06   3.64053019e-06   1.76025677e-05   2.53612828e-05
        4.28581244e-05   5.31947855e-05   8.09585256e-05]
     [  5.18984675e-07   1.37335633e-06   5.65009596e-06   1.05520069e-05
        1.81445380e-05   1.87825719e-05   3.56811933e-05]
     [  1.21116956e-07   3.82213679e-07   1.63908339e-06   3.27248912e-06
        6.69699576e-06   7.59916314e-06   1.27103665e-05]]

最后可以得到似然 $p(l|x) = \alpha_T(|l^\prime|) + \alpha_T(|l^\prime|-1)$。

```python
p = alpha[-1, labels[-1]] + alpha[-1, labels[-2]]
print(p)
```

    6.81811271177e-06

## 1.5 后向计算
类似于前向计算，我们定义后向计算。
首先定义
$$
\beta_t(s)   \stackrel{def}{=} \sum_{\pi \in N^T: B(\pi_{t:T}) = l_{s:|l|}} \prod_{t^\prime=t}^T y^t_{\pi^\prime}
$$

显然，
$$
\begin{align}
\beta_T(|l^\prime|) = y_b^T,\\
\beta_T(|l^\prime|-1) = y_{l_{|l|}}^T,\\
\beta_T(s) = 0, \forall s < |l^\prime| - 1
\end{align}
$$

易得如下递归关系：
$$
\beta_t(s) = \{  \begin{array}{l}
(\beta_{t+1}(s)+\beta_{t+1}(s+1)) y^t_{l^\prime_s},\  \  \    if\  l^\prime_s = b \ or\  l_{s+2}^\prime = l_s^{\prime}  \\
(\beta_{t+1}(s)+\beta_{t+1}(s+1) + \beta_{t+1}(s+2)) y^t_{l^\prime_s}
\end{array}
$$

```python
def backward(y, labels):
    T, V = y.shape
    L = len(labels)
    beta = np.zeros([T, L])

    # init
    beta[-1, -1] = y[-1, labels[-1]]
    beta[-1, -2] = y[-1, labels[-2]]

    for t in range(T - 2, -1, -1):
        for i in range(L):
            s = labels[i]

            a = beta[t + 1, i]
            if i + 1 < L:
                a += beta[t + 1, i + 1]
            if i + 2 < L and s != 0 and s != labels[i + 2]:
                a += beta[t + 1, i + 2]

            beta[t, i] = a * y[t, s]

    return beta

beta = backward(y, labels)
print(beta)
```

    [[  1.25636660e-05   7.74586366e-06   8.69559539e-06   3.30990037e-06
        2.41325357e-06   4.30516936e-07   1.21116956e-07]
     [  3.00418145e-05   2.09170784e-05   2.53062822e-05   9.96351200e-06
        8.39236521e-06   1.39591874e-06   4.91256769e-07]
     [  7.14014755e-05   4.66705755e-05   7.46535563e-05   2.48066359e-05
        2.77113594e-05   5.27279259e-06   1.93076535e-06]
     [  1.69926001e-04   1.25923340e-04   2.33240296e-04   7.60839197e-05
        9.89830489e-05   1.58379311e-05   8.00005392e-06]
     [  4.20893778e-04   2.03461048e-04   6.84292101e-04   1.72696845e-04
        3.08627225e-04   5.50636993e-05   2.93943967e-05]
     [  4.81953899e-04   8.10796738e-04   1.27731424e-03   8.24448952e-04
        7.48161143e-04   1.99769340e-04   9.02831714e-05]
     [  9.80428697e-04   1.03986915e-03   3.68556718e-03   1.66879393e-03
        2.56724754e-03   5.68961868e-04   3.78457146e-04]
     [  2.40870506e-04   2.30339872e-03   4.81028886e-03   4.75397134e-03
        4.31752827e-03   2.34462771e-03   9.82118206e-04]
     [  0.00000000e+00   1.10150469e-03   1.28817322e-02   9.11579592e-03
        1.35011919e-02   6.24293419e-03   4.49124231e-03]
     [  0.00000000e+00   0.00000000e+00   9.52648414e-03   3.36188472e-02
        2.50664437e-02   2.01536701e-02   1.50427081e-02]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   3.93092725e-02
        4.25697510e-02   6.08622868e-02   6.20709492e-02]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   1.60052748e-01   2.33372894e-01]]

## 1.6 梯度计算
下面，我们利用前向、后者计算的 $\alpha$ 和 $\beta$ 来计算梯度。

根据 $\alpha$、$\beta$ 的定义，我们有：
$$\alpha_t(s)\beta_t(s) = \sum_{\pi \in  B^{-1}(l):\pi_t=l_s^\prime} y^t_{l_s^\prime} \prod_{t=1}^T y^t_{\pi_t} = y^t_{l_s^\prime} \cdot \sum_{\pi \in  B^{-1}(l):\pi_t=l_s^\prime} \prod_{t=1}^T y^t_{\pi_t}$$
则
$$
\frac{\alpha_t(s)\beta_t(s)}{ y^t_{l_s^\prime}} = \sum_{\pi \in  B^{-1}(l):\pi_t=l_s^\prime} \prod_{t=1}^T y^t_{\pi_t}  = \sum_{\pi \in  B^{-1}(l):\pi_t=l_s^\prime} p(\pi|x)
$$
于是，可得似然
$$
p(l|x) = \sum_{s=1}^{|l^\prime|} \sum_{\pi \in  B^{-1}(l):\pi_t=l_s^\prime} p(\pi|x) =  \sum_{s=1}^{|l^\prime|}  \frac{\alpha_t(s)\beta_t(s)}{ y^t_{l_s^\prime}}
$$

为计算 $\frac{\partial p(l|x)}{\partial y^t_k}$，观察上式右端求各项，仅有 $s=k$ 的项包含 $y^t_k$，因此，其他项的偏导都为零，不用考虑。于是有：

$$
\frac{\partial p(l|x)}{\partial y^t_k} = \frac{\partial \frac{\alpha_t(k)\beta_t(k)}{ y^t_{k}} }{\partial y^t_k}
$$

利用除法的求导准则有：
$$
\frac{\partial p(l|x)}{\partial y^t_k} =  \frac{\frac{2 \cdot \alpha_t(k)\beta_t(k)}{ y^t_{k}} \cdot y^t_{k} -  \alpha_t(k)\beta_t(k) \cdot 1}{{y^t_k}^2} = \frac{\alpha_t(k)\beta_t(k)}{{y^t_k}^2}
$$

> 求导中，分子第一项是因为 $\alpha(k)\beta(k)$ 中包含为两个 $y^t_k$ 乘积项（即 ${y^t_k}^2$），其他均为与 $y^t_k$ 无关的常数。

$l$ 中可能包含多个 $k$ 字符，它们计算的梯度要进行累加，因此，最后的梯度计算结果为：
$$
\frac{\partial p(l|x)}{\partial y^t_k} = \frac{1}{{y^t_k}^2} \sum_{s \in lab(l, k)} \alpha_t(s)\beta_t(s)
$$
其中，$lab(s)=\{s: l_s^\prime = k\}$。

一般我们优化似然函数的对数，因此，梯度计算如下：
$$
\frac{\partial \ln(p(l|x))}{\partial y^t_k} =\frac{1}{p(l|x)} \frac{\partial p(l|x)}{\partial y^t_k}
$$
其中，似然值在前向计算中已经求得： $p(l|x) = \alpha_T(|l^\prime|) + \alpha_T(|l^\prime|-1)$。

对于给定训练集 $D$，待优化的目标函数为：
$$
O(D, N_w) = -\sum_{(x,z)\in D} \ln(p(z|x))
$$

得到梯度后，我们可以利用任意优化方法（e.g. SGD， Adam）进行训练。

```python
def gradient(y, labels):
    T, V = y.shape
    L = len(labels)

    alpha = forward(y, labels)
    beta = backward(y, labels)
    p = alpha[-1, -1] + alpha[-1, -2]

    grad = np.zeros([T, V])
    for t in range(T):
        for s in range(V):
            lab = [i for i, c in enumerate(labels) if c == s]
            for i in lab:
                grad[t, s] += alpha[t, i] * beta[t, i]
            grad[t, s] /= y[t, s] ** 2

    grad /= p
    return grad

grad = gradient(y, labels)
print(grad)
```

    [[ 2.50911241  0.          0.          2.27594441  0.        ]
     [ 2.25397118  0.          0.          2.47384957  0.        ]
     [ 2.65058465  0.          0.          2.77274592  0.        ]
     [ 2.46136916  0.          0.          2.29678159  0.02303985]
     [ 2.300259    0.          0.          2.37548238  0.10334851]
     [ 2.40271071  0.          0.          2.19860276  0.23513657]
     [ 1.68914157  0.          0.          1.78214377  0.51794046]
     [ 2.32536762  0.          0.          1.75750877  0.92477606]
     [ 1.92883907  0.          0.          1.45529832  1.44239844]
     [ 2.06219335  0.          0.          0.7568118   1.96405515]
     [ 2.07914466  0.          0.          0.33858403  2.35197258]
     [ 2.6816852   0.          0.          0.          2.3377753 ]]

将基于前向-后向算法得到梯度与基于数值的梯度比较，以验证实现的正确性。

```python
def check_grad(y, labels, w=-1, v=-1, toleration=1e-3):
    grad_1 = gradient(y, labels)[w, v]

    delta = 1e-10
    original = y[w, v]

    y[w, v] = original + delta
    alpha = forward(y, labels)
    log_p1 = np.log(alpha[-1, -1] + alpha[-1, -2])

    y[w, v] = original - delta
    alpha = forward(y, labels)
    log_p2 = np.log(alpha[-1, -1] + alpha[-1, -2])

    y[w, v] = original

    grad_2 = (log_p1 - log_p2) / (2 * delta)
    if np.abs(grad_1 - grad_2) > toleration:
        print('[%d, %d]：%.2e' % (w, v, np.abs(grad_1 - grad_2)))

for toleration in [1e-5, 1e-6]:
    print('%.e' % toleration)
    for w in range(y.shape[0]):
        for v in range(y.shape[1]):
            check_grad(y, labels, w, v, toleration)

```

    1e-05
    1e-06
    [0, 3]：3.91e-06
    [1, 0]：3.61e-06
    [1, 3]：2.66e-06
    [2, 0]：2.67e-06
    [2, 3]：3.88e-06
    [3, 0]：4.71e-06
    [3, 3]：3.39e-06
    [4, 0]：1.24e-06
    [4, 3]：4.79e-06
    [5, 0]：1.57e-06
    [5, 3]：2.98e-06
    [6, 0]：5.03e-06
    [6, 3]：4.89e-06
    [7, 0]：1.05e-06
    [7, 4]：4.19e-06
    [8, 4]：5.57e-06
    [9, 0]：5.95e-06
    [9, 3]：3.85e-06
    [10, 0]：1.09e-06
    [10, 3]：1.53e-06
    [10, 4]：3.82e-06

可以看到，前向-后向及数值梯度两种方法计算的梯度差异都在 1e-5 以下，误差最多在 1e-6 的量级。这初步验证了前向-后向梯度计算方法原理和实现的正确性。

## 1.7 logits 梯度
在实际训练中，为了计算方便，可以将 CTC 和 softmax 的梯度计算合并，公式如下：
$$
\frac{\partial \ln(p(l|x))}{\partial y^t_k} = y^t_k - \frac{1}{y^t_k \cdot p(l|x)} \sum_{s \in lab(l, k)} \alpha_t(s)\beta_t(s)
$$

这是因为，softmax 的梯度反传公式为：

$$
\frac{\partial \ln(p(l|x))}{\partial u^t_k} = y^t_k (\frac{\partial \ln(p(l|x))}{\partial y^t_k}  - \sum_{j=1}^{V} \frac{\partial \ln(p(l|x))}{\partial y^t_j} y^t_j)
$$

接合上面两式，有:
$$
\frac{\partial \ln(p(l|x))}{\partial u^t_k} = \frac{1}{y^t_k p(l|x)} \sum_{s \in lab(l, k)} \alpha_t(s)\beta_t(s) - y^t_k
$$

```python
def gradient_logits_naive(y, labels):
    '''
    gradient by back propagation
    '''
    y_grad = gradient(y, labels)

    sum_y_grad = np.sum(y_grad * y, axis=1, keepdims=True)
    u_grad = y * (y_grad - sum_y_grad)

    return u_grad

def gradient_logits(y, labels):
    '''
    '''
    T, V = y.shape
    L = len(labels)

    alpha = forward(y, labels)
    beta = backward(y, labels)
    p = alpha[-1, -1] + alpha[-1, -2]

    u_grad = np.zeros([T, V])
    for t in range(T):
        for s in range(V):
            lab = [i for i, c in enumerate(labels) if c == s]
            for i in lab:
                u_grad[t, s] += alpha[t, i] * beta[t, i]
            u_grad[t, s] /= y[t, s] * p

    u_grad -= y
    return u_grad

grad_l = gradient_logits_naive(y, labels)
grad_2 = gradient_logits(y, labels)

print(np.sum(np.abs(grad_l - grad_2)))
```

    1.59941504485e-15

同上，我们利用数值梯度来初步检验梯度计算的正确性：

```python
def check_grad_logits(x, labels, w=-1, v=-1, toleration=1e-3):
    grad_1 = gradient_logits(softmax(x), labels)[w, v]

    delta = 1e-10
    original = x[w, v]

    x[w, v] = original + delta
    y = softmax(x)
    alpha = forward(y, labels)
    log_p1 = np.log(alpha[-1, -1] + alpha[-1, -2])

    x[w, v] = original - delta
    y = softmax(x)
    alpha = forward(y, labels)
    log_p2 = np.log(alpha[-1, -1] + alpha[-1, -2])

    x[w, v] = original

    grad_2 = (log_p1 - log_p2) / (2 * delta)
    if np.abs(grad_1 - grad_2) > toleration:
        print('[%d, %d]：%.2e, %.2e, %.2e' % (w, v, grad_1, grad_2, np.abs(grad_1 - grad_2)))

np.random.seed(1111)
x = np.random.random([10, 10])
for toleration in [1e-5, 1e-6]:
    print('%.e' % toleration)
    for w in range(x.shape[0]):
        for v in range(x.shape[1]):
            check_grad_logits(x, labels, w, v, toleration)
```

    1e-05
    [0, 6]：-8.00e-02, -8.00e-02, 1.03e-05
    [4, 0]：4.29e-01, 4.29e-01, 1.10e-05
    [5, 6]：-7.59e-02, -7.59e-02, 1.22e-05
    [6, 2]：-1.38e-01, -1.38e-01, 1.23e-05
    [6, 3]：3.33e-01, 3.33e-01, 1.02e-05
    1e-06
    [0, 0]：3.88e-01, 3.88e-01, 7.03e-06
    [0, 1]：-1.59e-01, -1.59e-01, 2.78e-06
    [0, 2]：-8.89e-02, -8.89e-02, 3.47e-06
    [0, 3]：4.57e-01, 4.57e-01, 1.64e-06
    [0, 4]：-6.32e-02, -6.32e-02, 7.19e-06
    [0, 5]：-7.98e-02, -7.98e-02, 5.46e-06
    [0, 6]：-8.00e-02, -8.00e-02, 1.03e-05
    [0, 7]：-1.32e-01, -1.32e-01, 2.21e-06
    [0, 8]：-1.04e-01, -1.04e-01, 7.75e-06
    [0, 9]：-1.38e-01, -1.38e-01, 5.95e-06
    [1, 0]：3.41e-01, 3.41e-01, 2.79e-06
    [1, 1]：-1.18e-01, -1.18e-01, 6.08e-06
    [1, 3]：5.04e-01, 5.04e-01, 4.06e-06
    [1, 4]：-9.96e-02, -9.96e-02, 5.77e-06
    [1, 5]：-8.22e-02, -8.22e-02, 4.03e-06
    [1, 6]：-9.46e-02, -9.46e-02, 4.49e-06
    [1, 7]：-1.49e-01, -1.49e-01, 3.96e-06
    [1, 8]：-1.24e-01, -1.24e-01, 4.96e-06
    [1, 9]：-7.48e-02, -7.47e-02, 5.94e-06
    [2, 0]：3.29e-01, 3.29e-01, 3.47e-06
    [2, 1]：-9.42e-02, -9.42e-02, 1.63e-06
    [2, 2]：-9.17e-02, -9.17e-02, 4.47e-06
    [2, 3]：4.50e-01, 4.50e-01, 2.14e-06
    [2, 5]：-1.07e-01, -1.07e-01, 6.33e-06
    [2, 6]：-5.42e-02, -5.42e-02, 1.71e-06
    [2, 7]：-9.68e-02, -9.68e-02, 7.69e-06
    [2, 9]：-1.21e-01, -1.21e-01, 9.06e-06
    [3, 0]：4.42e-01, 4.42e-01, 9.21e-06
    [3, 1]：-6.71e-02, -6.71e-02, 5.75e-06
    [3, 2]：-1.16e-01, -1.16e-01, 5.26e-06
    [3, 3]：4.03e-01, 4.03e-01, 6.39e-06
    [3, 4]：-1.07e-01, -1.07e-01, 2.42e-06
    [3, 5]：-1.25e-01, -1.25e-01, 8.90e-06
    [3, 6]：-1.17e-01, -1.17e-01, 2.08e-06
    [3, 7]：-1.32e-01, -1.32e-01, 2.21e-06
    [3, 8]：-6.90e-02, -6.90e-02, 1.72e-06
    [3, 9]：-1.13e-01, -1.13e-01, 6.68e-06
    [4, 0]：4.29e-01, 4.29e-01, 1.10e-05
    [4, 1]：-7.17e-02, -7.17e-02, 7.76e-06
    [4, 3]：3.25e-01, 3.25e-01, 2.93e-06
    [4, 4]：-5.91e-02, -5.91e-02, 1.88e-06
    [4, 5]：-7.78e-02, -7.78e-02, 5.82e-06
    [4, 6]：-9.08e-02, -9.08e-02, 8.04e-06
    [4, 7]：-1.12e-01, -1.12e-01, 2.11e-06
    [4, 8]：-1.26e-01, -1.26e-01, 5.92e-06
    [4, 9]：-1.40e-01, -1.40e-01, 4.89e-06
    [5, 0]：1.86e-01, 1.86e-01, 6.60e-06
    [5, 2]：-9.52e-02, -9.52e-02, 5.01e-06
    [5, 3]：4.97e-01, 4.97e-01, 6.92e-06
    [5, 4]：2.50e-02, 2.50e-02, 5.20e-06
    [5, 5]：-7.93e-02, -7.93e-02, 2.78e-06
    [5, 6]：-7.59e-02, -7.59e-02, 1.22e-05
    [5, 7]：-1.05e-01, -1.05e-01, 9.53e-06
    [5, 8]：-1.25e-01, -1.25e-01, 9.07e-06
    [5, 9]：-6.74e-02, -6.74e-02, 3.08e-06
    [6, 0]：1.49e-01, 1.49e-01, 5.17e-06
    [6, 1]：-6.67e-02, -6.66e-02, 4.40e-06
    [6, 2]：-1.38e-01, -1.38e-01, 1.23e-05
    [6, 3]：3.33e-01, 3.33e-01, 1.02e-05
    [6, 4]：1.72e-01, 1.72e-01, 7.79e-06
    [6, 6]：-6.73e-02, -6.73e-02, 7.13e-06
    [6, 7]：-6.08e-02, -6.08e-02, 8.58e-06
    [6, 8]：-9.98e-02, -9.98e-02, 6.14e-06
    [6, 9]：-1.31e-01, -1.31e-01, 8.89e-06
    [7, 0]：2.39e-01, 2.39e-01, 2.53e-06
    [7, 2]：-1.61e-01, -1.61e-01, 1.78e-06
    [7, 3]：1.02e-01, 1.02e-01, 4.73e-06
    [7, 4]：3.63e-01, 3.63e-01, 7.51e-06
    [7, 5]：-6.02e-02, -6.02e-02, 4.63e-06
    [7, 6]：-1.05e-01, -1.05e-01, 3.29e-06
    [7, 8]：-8.83e-02, -8.83e-02, 4.16e-06
    [7, 9]：-6.05e-02, -6.05e-02, 2.89e-06
    [8, 0]：2.92e-01, 2.92e-01, 1.38e-06
    [8, 1]：-9.70e-02, -9.70e-02, 2.27e-06
    [8, 2]：-9.87e-02, -9.87e-02, 4.36e-06
    [8, 4]：5.07e-01, 5.07e-01, 2.30e-06
    [8, 5]：-1.28e-01, -1.28e-01, 7.41e-06
    [8, 6]：-1.09e-01, -1.09e-01, 6.32e-06
    [8, 7]：-1.20e-01, -1.20e-01, 6.01e-06
    [8, 8]：-7.00e-02, -7.00e-02, 1.44e-06
    [8, 9]：-1.55e-01, -1.55e-01, 2.47e-06
    [9, 0]：4.90e-01, 4.90e-01, 2.04e-06
    [9, 1]：-7.69e-02, -7.69e-02, 1.54e-06
    [9, 2]：-9.72e-02, -9.72e-02, 9.59e-06
    [9, 4]：2.61e-01, 2.61e-01, 2.16e-06
    [9, 5]：-9.27e-02, -9.27e-02, 5.07e-06
    [9, 6]：-7.70e-02, -7.70e-02, 1.03e-06
    [9, 7]：-8.30e-02, -8.30e-02, 5.42e-06
    [9, 8]：-1.17e-01, -1.17e-01, 4.26e-06
    [9, 9]：-1.39e-01, -1.39e-01, 3.53e-06

# 2. 数值稳定性

CTC 的训练过程面临数值下溢的风险，特别是序列较大的情况下。下面介绍两种数值上稳定的工程优化方法：1）log 域（许多 CRF 实现的常用方法）；2）scale 技巧（原始论文 [1] 使用的方法）。

## 2.1 log 域计算

log 计算涉及 logsumexp 操作。
[经验表明](https://github.com/baidu-research/warp-ctc)，在 log 域计算，即使使用单精度，也表现出良好的数值稳定性，可以有效避免下溢的风险。稳定性的代价是增加了运算的复杂性——原始实现只涉及乘加运算，log 域实现则需要对数和指数运算。

```python
ninf = -np.float('inf')

def _logsumexp(a, b):
    '''
    np.log(np.exp(a) + np.exp(b))

    '''

    if a < b:
        a, b = b, a

    if b == ninf:
        return a
    else:
        return a + np.log(1 + np.exp(b - a))

def logsumexp(*args):
    '''
    from scipy.special import logsumexp
    logsumexp(args)
    '''
    res = args[0]
    for e in args[1:]:
        res = _logsumexp(res, e)
    return res
```

### 2.1.1 log 域前向算法
基于 log 的前向算法实现如下：

```python
def forward_log(log_y, labels):
    T, V = log_y.shape
    L = len(labels)
    log_alpha = np.ones([T, L]) * ninf

    # init
    log_alpha[0, 0] = log_y[0, labels[0]]
    log_alpha[0, 1] = log_y[0, labels[1]]

    for t in range(1, T):
        for i in range(L):
            s = labels[i]

            a = log_alpha[t - 1, i]
            if i - 1 >= 0:
                a = logsumexp(a, log_alpha[t - 1, i - 1])
            if i - 2 >= 0 and s != 0 and s != labels[i - 2]:
                a = logsumexp(a, log_alpha[t - 1, i - 2])

            log_alpha[t, i] = a + log_y[t, s]

    return log_alpha

log_alpha = forward_log(np.log(y), labels)
alpha = forward(y, labels)
print(np.sum(np.abs(np.exp(log_alpha) - alpha)))
```

    8.60881935942e-17

### 2.1.2 log 域后向算法
基于 log 的后向算法实现如下：

```python
def backward_log(log_y, labels):
    T, V = log_y.shape
    L = len(labels)
    log_beta = np.ones([T, L]) * ninf

    # init
    log_beta[-1, -1] = log_y[-1, labels[-1]]
    log_beta[-1, -2] = log_y[-1, labels[-2]]

    for t in range(T - 2, -1, -1):
        for i in range(L):
            s = labels[i]

            a = log_beta[t + 1, i]
            if i + 1 < L:
                a = logsumexp(a, log_beta[t + 1, i + 1])
            if i + 2 < L and s != 0 and s != labels[i + 2]:
                a = logsumexp(a, log_beta[t + 1, i + 2])

            log_beta[t, i] = a + log_y[t, s]

    return log_beta

log_beta = backward_log(np.log(y), labels)
beta = backward(y, labels)
print(np.sum(np.abs(np.exp(log_beta) - beta)))
```

    1.10399945005e-16

### 2.1.3 log 域梯度计算
在前向、后向基础上，也可以在 log 域上计算梯度。

```python
def gradient_log(log_y, labels):
    T, V = log_y.shape
    L = len(labels)

    log_alpha = forward_log(log_y, labels)
    log_beta = backward_log(log_y, labels)
    log_p = logsumexp(log_alpha[-1, -1], log_alpha[-1, -2])

    log_grad = np.ones([T, V]) * ninf
    for t in range(T):
        for s in range(V):
            lab = [i for i, c in enumerate(labels) if c == s]
            for i in lab:
                log_grad[t, s] = logsumexp(log_grad[t, s], log_alpha[t, i] + log_beta[t, i])
            log_grad[t, s] -= 2 * log_y[t, s]

    log_grad -= log_p
    return log_grad

log_grad = gradient_log(np.log(y), labels)
grad = gradient(y, labels)
#print(log_grad)
#print(grad)
print(np.sum(np.abs(np.exp(log_grad) - grad)))
```

    4.97588081849e-14

## 2.2 scale

### 2.2.1 前向算法

为了下溢，在前向算法的每个时刻，都对计算出的 $\alpha$ 的范围进行缩放：
$$
C_t \stackrel{def}{=} \sum_s\alpha_t(s)
$$

$$
\hat{\alpha}_t = \frac{\alpha_t(s)}{C_t}
$$

缩放后的 $\alpha$，不会随着时刻的积累变得太小。$\hat{\alpha}$ 替代 $\alpha$，进行下一时刻的迭代。

```python
def forward_scale(y, labels):
    T, V = y.shape
    L = len(labels)
    alpha_scale = np.zeros([T, L])

    # init
    alpha_scale[0, 0] = y[0, labels[0]]
    alpha_scale[0, 1] = y[0, labels[1]]
    Cs = []

    C = np.sum(alpha_scale[0])
    alpha_scale[0] /= C
    Cs.append(C)

    for t in range(1, T):
        for i in range(L):
            s = labels[i]

            a = alpha_scale[t - 1, i]
            if i - 1 >= 0:
                a += alpha_scale[t - 1, i - 1]
            if i - 2 >= 0 and s != 0 and s != labels[i - 2]:
                a += alpha_scale[t - 1, i - 2]

            alpha_scale[t, i] = a * y[t, s]

        C = np.sum(alpha_scale[t])
        alpha_scale[t] /= C
        Cs.append(C)

    return alpha_scale, Cs
```

由于进行了缩放，最后计算概率时要时行补偿：
$$
p(l|x) = \alpha_T(|l^\prime|) + \alpha_T(|l^\prime|-1) = (\hat\alpha_T(|l^\prime|) + \hat\alpha_T(|l^\prime|-1) * \prod_{t=1}^T C_t
$$

$$
\ln(p(l|x)) = \sum_t^T\ln(C_t) + \ln(\hat\alpha_T(|l^\prime|) + \hat\alpha_T(|l^\prime|-1))
$$

```python
labels = [0, 1, 2, 0]  # 0 for blank

alpha_scale, Cs = forward_scale(y, labels)
log_p = np.sum(np.log(Cs)) + np.log(alpha_scale[-1][labels[-1]] + alpha_scale[-1][labels[-2]])

alpha = forward(y, labels)
p = alpha[-1, labels[-1]] + alpha[-1, labels[-2]]

print(np.log(p), log_p, np.log(p) - log_p)
```

    (-13.202925982240107, -13.202925982240107, 0.0)

### 2.2.2 后向算法
后向算法缩放类似于前向算法，公式如下：

$$
D_t \stackrel{def}{=} \sum_s\beta_t(s)
$$

$$
\hat{\beta}_t = \frac{\beta_t(s)}{D_t}
$$

```python
def backward_scale(y, labels):
    T, V = y.shape
    L = len(labels)
    beta_scale = np.zeros([T, L])

    # init
    beta_scale[-1, -1] = y[-1, labels[-1]]
    beta_scale[-1, -2] = y[-1, labels[-2]]

    Ds = []

    D = np.sum(beta_scale[-1,:])
    beta_scale[-1] /= D
    Ds.append(D)

    for t in range(T - 2, -1, -1):
        for i in range(L):
            s = labels[i]

            a = beta_scale[t + 1, i]
            if i + 1 < L:
                a += beta_scale[t + 1, i + 1]
            if i + 2 < L and s != 0 and s != labels[i + 2]:
                a += beta_scale[t + 1, i + 2]

            beta_scale[t, i] = a * y[t, s]

        D = np.sum(beta_scale[t])
        beta_scale[t] /= D
        Ds.append(D)

    return beta_scale, Ds[::-1]

beta_scale, Ds = backward_scale(y, labels)
print(beta_scale)
```

    [[ 0.71362347  0.18910147  0.07964328  0.01763178]
     [ 0.70165268  0.15859852  0.11849423  0.02125457]
     [ 0.67689676  0.165374    0.13221504  0.02551419]
     [ 0.71398181  0.11936432  0.13524265  0.03141122]
     [ 0.70769657  0.13093688  0.12447135  0.0368952 ]
     [ 0.63594568  0.1790638   0.14250065  0.04248987]
     [ 0.63144322  0.1806382   0.13366043  0.05425815]
     [ 0.33926289  0.35149591  0.24988622  0.05935497]
     [ 0.30303623  0.26644554  0.33088584  0.0996324 ]
     [ 0.12510056  0.3297143   0.3956509   0.14953425]
     [ 0.          0.22078343  0.5153114   0.26390517]
     [ 0.          0.          0.550151    0.449849  ]]

### 2.2.3 梯度计算

$$
\frac{\partial \ln(p(l|x))}{\partial y^t_k} = \frac{1}{p(l|x)} \frac{\partial p(l|x)}{\partial y^t_k} = \frac{1}{p(l|x)} \frac{1}{{y^t_k}^2} \sum_{s \in lab(l, k)} \alpha_t(s)\beta_t(s)
$$

考虑到
$$
p(l|x) =  \sum_{s=1}^{|l^\prime|}  \frac{\alpha_t(s)\beta_t(s)}{ y^t_{l_s^\prime}}
$$
以及
$$
\alpha_t(s) = \hat\alpha_t(s) \cdot \prod_{k=1}^t C_k
$$

$$
\beta_t(s) = \hat\beta_t(s) \cdot \prod_{k=t}^T D_k
$$

$$
\frac{\partial \ln(p(l|x))}{\partial y^t_k} = \frac{1}{\sum_{s=1}^{|l^\prime|} \frac{\hat\alpha_t(s)  \hat\beta_t(s)}{y^t_{l^\prime_s}}} \frac{1}{{y^t_k}^2} \sum_{s \in lab(l, k)} \hat\alpha_t(s)  \hat\beta_t(s)
$$

式中最右项中的各个部分我们都已经求得。梯度计算实现如下：

```python
def gradient_scale(y, labels):
    T, V = y.shape
    L = len(labels)

    alpha_scale, _ = forward_scale(y, labels)
    beta_scale, _ = backward_scale(y, labels)

    grad = np.zeros([T, V])
    for t in range(T):
        for s in range(V):
            lab = [i for i, c in enumerate(labels) if c == s]
            for i in lab:
                grad[t, s] += alpha_scale[t, i] * beta_scale[t, i]
            grad[t, s] /= y[t, s] ** 2

        # normalize factor
        z = 0
        for i in range(L):
            z += alpha_scale[t, i] * beta_scale[t, i] / y[t, labels[i]]
        grad[t] /= z

    return grad

labels = [0, 3, 0, 3, 0, 4, 0]  # 0 for blank
grad_1 = gradient_scale(y, labels)
grad_2 = gradient(y, labels)
print(np.sum(np.abs(grad_1 - grad_2)))
```

    6.86256607096e-15

### 2.2.4 logits 梯度
类似于 y 梯度的推导，logits 梯度计算公式如下：
$$
\frac{\partial \ln(p(l|x))}{\partial u^t_k} = \frac{1}{y^t_k Z_t} \sum_{s \in lab(l, k)} \hat\alpha_t(s)\hat\beta_t(s) - y^t_k
$$
其中，
$$
Z_t \stackrel{def}{=} \sum_{s=1}^{|l^\prime|} \frac{\hat\alpha_t(s)\hat\beta_t(s)}{y^t_{l^\prime_s}}
$$

# 3. 解码
训练和的 $N_w$ 可以用来预测新的样本输入对应的输出字符串，这涉及到解码。
按照最大似然准则，最优的解码结果为：
$$
h(x) = \underset{l \in L^{\le T}}{\mathrm{argmax}}\ p(l|x)
$$

然而，上式不存在已知的高效解法。下面介绍几种实用的近似破解码方法。

## 3.1 贪心搜索 （greedy search）
虽然 $p(l|x)$ 难以有效的计算，但是由于 CTC 的独立性假设，对于某个具体的字符串 $\pi$（去 blank 前），确容易计算：
$$
p(\pi|x) = \prod_{k=1}^T p(\pi_k|x)
$$

因此，我们放弃寻找使 $p(l|x)$ 最大的字符串，退而寻找一个使 $p(\pi|x)$ 最大的字符串，即：

$$
h(x) \approx B(\pi^\star)
$$
其中，
$$
\pi^\star = \underset{\pi \in N^T}{\mathrm{argmax}}\ p(\pi|x)
$$

简化后，解码过程（构造 $\pi^\star$）变得非常简单（基于独立性假设）： 在每个时刻输出概率最大的字符:
$$
\pi^\star = cat_{t=1}^T(\underset{s \in L^\prime}{\mathrm{argmax}}\ y^t_s)
$$

```python
def remove_blank(labels, blank=0):
    new_labels = []

    # combine duplicate
    previous = None
    for l in labels:
        if l != previous:
            new_labels.append(l)
            previous = l

    # remove blank
    new_labels = [l for l in new_labels if l != blank]

    return new_labels

def insert_blank(labels, blank=0):
    new_labels = [blank]
    for l in labels:
        new_labels += [l, blank]
    return new_labels

def greedy_decode(y, blank=0):
    raw_rs = np.argmax(y, axis=1)
    rs = remove_blank(raw_rs, blank)
    return raw_rs, rs

np.random.seed(1111)
y = softmax(np.random.random([20, 6]))
rr, rs = greedy_decode(y)
print(rr)
print(rs)
```

    [1 3 5 5 5 5 1 5 3 4 4 3 0 4 5 0 3 1 3 3]
    [1, 3, 5, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3]

## 3.2 束搜索（Beam Search）
显然，贪心搜索的性能非常受限。例如，它不能给出除最优路径之外的其他其优路径。很多时候，如果我们能拿到 nbest 的路径，后续可以利用其他信息来进一步优化搜索的结果。束搜索能近似找出 top 最优的若干条路径。

```python
def beam_decode(y, beam_size=10):
    T, V = y.shape
    log_y = np.log(y)

    beam = [([], 0)]
    for t in range(T):  # for every timestep
        new_beam = []
        for prefix, score in beam:
            for i in range(V):  # for every state
                new_prefix = prefix + [i]
                new_score = score + log_y[t, i]

                new_beam.append((new_prefix, new_score))

        # top beam_size
        new_beam.sort(key=lambda x: x[1], reverse=True)
        beam = new_beam[:beam_size]

    return beam

np.random.seed(1111)
y = softmax(np.random.random([20, 6]))
beam = beam_decode(y, beam_size=100)
for string, score in beam[:20]:
    print(remove_blank(string), score)
```

    ([1, 3, 5, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.261797539205567)
    ([1, 3, 5, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.279020152518033)
    ([1, 3, 5, 1, 5, 3, 4, 2, 3, 4, 5, 3, 1, 3], -29.300726142201842)
    ([1, 5, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.310307014773972)
    ([1, 3, 5, 1, 5, 3, 4, 2, 3, 3, 5, 3, 1, 3], -29.317948755514308)
    ([1, 5, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.327529628086438)
    ([1, 3, 5, 1, 5, 4, 3, 4, 5, 3, 1, 3], -29.331572723457334)
    ([1, 3, 5, 5, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.332631809924511)
    ([1, 3, 5, 4, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.334649090836038)
    ([1, 3, 5, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.33969505198154)
    ([1, 3, 5, 2, 1, 5, 3, 4, 3, 4, 5, 3, 1, 3], -29.339823066915415)
    ([1, 3, 5, 1, 5, 4, 3, 3, 5, 3, 1, 3], -29.3487953367698)
    ([1, 5, 1, 5, 3, 4, 2, 3, 4, 5, 3, 1, 3], -29.349235617770248)
    ([1, 3, 5, 5, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.349854423236977)
    ([1, 3, 5, 1, 5, 3, 4, 3, 4, 5, 3, 3], -29.350803198551016)
    ([1, 3, 5, 4, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.351871704148504)
    ([1, 3, 5, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.356917665294006)
    ([1, 3, 5, 2, 1, 5, 3, 4, 3, 3, 5, 3, 1, 3], -29.357045680227881)
    ([1, 3, 5, 1, 5, 3, 4, 5, 4, 5, 3, 1, 3], -29.363802591012263)
    ([1, 5, 1, 5, 3, 4, 2, 3, 3, 5, 3, 1, 3], -29.366458231082714)

## 3.3 前缀束搜索（Prefix Beam Search）
直接的束搜索的一个问题是，在保存的 top N 条路径中，可能存在多条实际上是同一结果（经过去重复、去 blank 操作）。这减少了搜索结果的多样性。下面介绍的前缀搜索方法，在搜索过程中不断的合并相同的前缀[2]。参考 [gist](https://gist.github.com/awni/56369a90d03953e370f3964c826ed4b0)，前缀束搜索实现如下：

```python
from collections import defaultdict

def prefix_beam_decode(y, beam_size=10, blank=0):
    T, V = y.shape
    log_y = np.log(y)

    beam = [(tuple(), (0, ninf))]  # blank, non-blank
    for t in range(T):  # for every timestep
        new_beam = defaultdict(lambda : (ninf, ninf))

        for prefix, (p_b, p_nb) in beam:
            for i in range(V):  # for every state
                p = log_y[t, i]

                if i == blank:  # propose a blank
                    new_p_b, new_p_nb = new_beam[prefix]
                    new_p_b = logsumexp(new_p_b, p_b + p, p_nb + p)
                    new_beam[prefix] = (new_p_b, new_p_nb)
                    continue
                else:  # extend with non-blank
                    end_t = prefix[-1] if prefix else None

                    # exntend current prefix
                    new_prefix = prefix + (i,)
                    new_p_b, new_p_nb = new_beam[new_prefix]
                    if i != end_t:
                        new_p_nb = logsumexp(new_p_nb, p_b + p, p_nb + p)
                    else:
                        new_p_nb = logsumexp(new_p_nb, p_b + p)
                    new_beam[new_prefix] = (new_p_b, new_p_nb)

                    # keep current prefix
                    if i == end_t:
                        new_p_b, new_p_nb = new_beam[prefix]
                        new_p_nb = logsumexp(new_p_nb, p_nb + p)
                        new_beam[prefix] = (new_p_b, new_p_nb)

        # top beam_size
        beam = sorted(new_beam.items(), key=lambda x : logsumexp(*x[1]), reverse=True)
        beam = beam[:beam_size]

    return beam

np.random.seed(1111)
y = softmax(np.random.random([20, 6]))
beam = prefix_beam_decode(y, beam_size=100)
for string, score in beam[:20]:
    print(remove_blank(string), score)
```

    ([1, 5, 4, 1, 3, 4, 5, 2, 3], (-18.189863809114193, -17.613677981426175))
    ([1, 5, 4, 5, 3, 4, 5, 2, 3], (-18.19636512622969, -17.621013424585406))
    ([1, 5, 4, 1, 3, 4, 5, 1, 3], (-18.317018960331531, -17.666629973270073))
    ([1, 5, 4, 5, 3, 4, 5, 1, 3], (-18.323388267369936, -17.674125139073176))
    ([1, 5, 4, 1, 3, 4, 3, 2, 3], (-18.415808498759556, -17.862744326248826))
    ([1, 5, 4, 1, 3, 4, 3, 5, 3], (-18.366422766638632, -17.898463479112884))
    ([1, 5, 4, 5, 3, 4, 3, 2, 3], (-18.42224294936932, -17.870025672291458))
    ([1, 5, 4, 5, 3, 4, 3, 5, 3], (-18.372199113900191, -17.905130493229173))
    ([1, 5, 4, 1, 3, 4, 5, 4, 3], (-18.457066311773847, -17.880630315602037))
    ([1, 5, 4, 5, 3, 4, 5, 4, 3], (-18.462614293487096, -17.88759583852546))
    ([1, 5, 4, 1, 3, 4, 5, 3, 2], (-18.458941701567706, -17.951422824358747))
    ([1, 5, 4, 5, 3, 4, 5, 3, 2], (-18.464527031120184, -17.958629487208658))
    ([1, 5, 4, 1, 3, 4, 3, 1, 3], (-18.540857550725587, -17.920589910093689))
    ([1, 5, 4, 5, 3, 4, 3, 1, 3], (-18.547146092248852, -17.928030266681613))
    ([1, 5, 4, 1, 3, 4, 5, 3, 2, 3], (-19.325467801462263, -17.689203224408899))
    ([1, 5, 4, 5, 3, 4, 5, 3, 2, 3], (-19.328748799764973, -17.694105969982637))
    ([1, 5, 4, 1, 3, 4, 5, 3, 4], (-18.79699026165903, -17.945090229238392))
    ([1, 5, 4, 5, 3, 4, 5, 3, 4], (-18.803585534273239, -17.95258394264377))
    ([1, 5, 4, 3, 4, 3, 5, 2, 3], (-19.181531846082809, -17.859420073785095))
    ([1, 5, 4, 1, 3, 4, 5, 2, 3, 2], (-19.439349296385199, -17.884502168470895))

## 3.4 其他解码方法
上述介绍了基本解码方法。实际中，搜索过程可以接合额外的信息，提高搜索的准确度（例如在语音识别任务中，加入语言模型得分信息, [前缀搜索+语言模型](https://github.com/PaddlePaddle/DeepSpeech/blob/develop/decoders/decoders_deprecated.py
)）。

本质上，CTC 只是一个训练准则。训练完成后，$N_w$ 输出一系列概率分布，这点和常规基于交叉熵准则训练的模型完全一致。因此，特定应用领域常规的解码也可以经过一定修改用来 CTC 的解码。例如在语音识别任务中，利用 CTC 训练的声学模型可以无缝融入原来的 WFST 的解码器中[5]（e.g. 参见 [EESEN](https://github.com/srvk/eesen)）。

此外，[1] 给出了一种利用 CTC 顶峰特点的启发式搜索方法。

# 4. 工具

[warp-ctc](https://github.com/baidu-research/warp-ctc) 是百度开源的基于 CPU 和 GPU 的高效并行实现。warp-ctc 自身提供 C 语言接口，对于流利的机器学习工具（[torch](https://github.com/baidu-research/warp-ctc/tree/master/torch_binding)、 [pytorch](https://github.com/SeanNaren/deepspeech.pytorch) 和 [tensorflow](https://github.com/baidu-research/warp-ctc/tree/master/tensorflow_binding)、[chainer](https://github.com/jheymann85/chainer_ctc)）都有相应的接口绑定。

[cudnn 7](https://developer.nvidia.com/cudnn) 以后开始提供 CTC 支持。

Tensorflow 也原生支持 [CTC loss](https://www.tensorflow.org/api_docs/python/tf/nn/ctc_loss)，及 greedy 和 beam search 解码器。

# 小结
1. CTC 可以建模无对齐信息的多对多序列问题（输入长度不小于输出），如语音识别、连续字符识别 [3,4]。
2. CTC 不需要输入与输出的对齐信息，可以实现端到端的训练。
3. CTC 在 loss 的计算上，利用了整个 labels 序列的全局信息，某种意义上相对逐帧计算损失的方法，"更加区分性"。

# References
1. Graves et al. [Connectionist Temporal Classification : Labelling Unsegmented Sequence Data with Recurrent Neural Networks](ftp://ftp.idsia.ch/pub/juergen/icml2006.pdf).
2. Hannun et al. [First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs](https://arxiv.org/abs/1408.2873).
3. Graves et al. [Towards End-To-End Speech Recognition with Recurrent Neural Networks](http://jmlr.org/proceedings/papers/v32/graves14.pdf).
4. Liwicki et al. [A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks](https://www.cs.toronto.edu/~graves/icdar_2007.pdf).
5. Zenkel et al. [Comparison of Decoding Strategies for CTC Acoustic Models](https://arxiv.org/abs/1708.004469).
5. Huannun. [Sequence Modeling with CTC](https://distill.pub/2017/ctc/).


# 3. Performance

Typically Python is slower than compiled languages due to **dynamic-typing**.

For example, let's say you wanted to add two integers together, in C this might look like:

    int a = 1;
    int b = 2;
    int c = a + b;

the C compiler knows from the start that a and b are integers; they cannot be anything else, hence it calls the appropriate instruction sets from assembly, returning another integer value, which might look something like:

    1. Assign 1 to a
    2. Assign 2 to b
    3. add<int,int>(a, b)
    4. Assign result to c

Compared to Python, which is:

    a = 1
    b = 2
    c = a + b

here the interpreter has no idea what type a, b and c are - only that they are *Python objects*. The interpreter must inspect the PyObject_HEAD for each variable to find the type information, then call the appropriate summation routine for the two types:

    Assign 1 to a
        Set a->PyObject_HEAD->typecode to integer
        Set a->val = 1
    Assign 2 to b
        Set b->PyObject_HEAD->typecode to integer
        Set b->val = 2
    call binary_add(a, b)
        find typecode in a->PyObject_HEAD
        a is an integer; value is a->val
        find typecode in b->PyObject_HEAD
        b is an integer; value is b->val
        call binary_add<int, int>(a->val, b->val)
        result of this is result, and is an integer.
    Create a Python object c
        set c->PyObject_HEAD->typecode to integer
        set c->val to result

### Python is Interpreted, not compiled

A compiler can look ahead and optimize for repeated or unneeded operations, which can result in significant speedups, interpeters on the other hand do not have this luxury.

### Python's object model leads to inefficient memory access

When it comes to applying batch operations to multiple integers (in an array for instance), C is much more efficient as there is significantly *less overhead* in creating arrays, where as *Python Lists* are a contiguous buffer of *pointers* which could potential point to random areas in memory, whereas small C arrays are likely to be cached. NumPy arrays get around this by wrapping a C array with a *single Python object*.

```python
import time
import numpy as np
import pandas as pd
import sympy

%load_ext Cython

from numba import jit
```

## Time

Python has a *time* module that simply returns the time in seconds from the Epoch (01/01/1970).

```python
time.time()
```

Naive profiling can take place by differencing the times before and after running some code of interest:

```python
t0 = time.time()
# some function
np.dot(np.random.randn(1000,3),np.ones((3,2))*10)

print(time.time() - t0)
```

Some of the most powerful tools used in Performance analysis are part of **iPython Magicks**: %timeit, %run and %prun.

Let's illustrate using the trapezoidal rule:

### Trapezoidal Rule

This is a method from numerical integration for approximating a definite integral:

$$
\int_a^b f(x)dx \approx (b-a)\frac{f(b)-f(a)}{2}
$$

Rather than using a single interval for this estimate, we break the interval down into $n$ subintervals, to obtain a more accurate approximation:

$$
\int_a^b f(x)dx \approx \sum_{k=1}^N \frac{f(x_{k-1})+f(x_k)}{2}\Delta x_k
$$

which for a uniform grid of equally-spaced panels becomes:

$$
I = \frac{\Delta x}{2}(f(x_0) + 2f(x_1) + 2f(x_2) + \dots + 2f(x_{N-1}) + f(x_N))
$$

```python
def f(x):
    return 2*x*x + 3*x + 1

def trapz(f, a, b, N):
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f(x)
    sum_y += .5 * (f(a) + f(b))
    return sum_y*h
```

```python
trapz(f, 1, 5, 10000)
```

Confirm this using *sympy*:

```python
xs = sympy.symbols("xs")
fx = 2*xs*xs + 3*xs + 1
ifx = sympy.integrate(fx, (xs, 1, 5))
ifx.evalf()
```

```python
%timeit trapz(f, 1, 5, 10000)
```

```python
%prun trapz(f, 1, 5, 100000)
```

## Speeding up Python/Pandas

When you have cratered under the weight of slow code, and you have profiled your code to find the bottleneck, there are a number of easy tools to speed up your code;

One of these methods is making effective use of **list comprehensions**.

```python
def calculate_y(x):
    return 3*x**3 + 4*x**2 + 10*x - x**4
```

```python
%%timeit
calcs = []
for x in range(10000):
    calcs.append(calculate_y(x))
```

```python
%timeit calcs = [calculate_y(x) for x in range(10000)]
```

A moderate speed up for very little work, and it's slightly easier to read and write

## Eval

In addition, Pandas provides access to *fast array expression evaluation* with eval():

```python
import pandas as pd
x1, x2, x3 = [pd.DataFrame(np.random.normal(3.0, 2.0, size=(100000,500))) for i in range(3)]
x1.head(1)
```

```python
%timeit x1 + x2 + x3
```

```python
%timeit pd.eval("x1 + x2 + x3")
```

```python
%timeit (x1 < x2) & (x2 > x3)
```

```python
%timeit pd.eval("(x1 < x2) & (x2 > x3)")
```

```python
dfx = pd.DataFrame(np.random.randn(1000000,2), columns=['x','y'])
```

```python
%timeit dfx.x*1.5 + .25*dfx.x**2 - 3.4*dfx.y + .75*dfx.y**2 - 10
```

```python
%timeit dfx.eval("1.5*x + 0.25*x**2 - 3.4*y + 0.75*y**2 - 10")
```

The calculations for this can easily be *assigned* using inplace:

```python
dfx.eval("z = x<0.5", inplace=True)
dfx.head()
```

Local variables can be assigned using *const @* identifier:

```python
cs = 0.007
dfx.eval("x * @cs").head()
```

## Cython

Python developers typically solve performance constraints by building Python extensions by wrapping code written in other compiled languages (such as C/C++). However the C/Python API is hideous complex for all but the most veteran developers.

Cython is a language that allows programmers to write fast code without having to write C/C++/Fortran directly. It looks like Python code but with type declarations. Cython code is translated to C, which is then compiled to create a Python extension that we can import and use.

Cython often achieves several orders of magnitude increase, often faster than hand-coded C, but can take a long time to get right.

Recall our trapz() function:

```python
# Pure Python

def trapz(f, a, b, N):
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f(x)
    sum_y += .5 * (f(a) + f(b))
    return sum_y*h
```

```python
df = pd.DataFrame({'a': np.random.randn(10000),
                   'b': np.random.randn(10000),
                   'N': np.random.randint(100, 1000, (10000)),
                   'x': 'x'})
```

```python
%timeit df.apply(lambda x: trapz(f, x.a, x.b, x.N), axis=1)
```

Let's profile to see why it's slow:

```python
%prun -l 4 df.apply(lambda x: trapz(f, x.a, x.b, x.N), axis=1)
```

Most of the time is spent in our functions, so we should convert them to Cython.

```python
%load_ext Cython
```

The easiest thing we can do is simply use the **iPython Magic** to convert a Jupyter notebook block into Cython for us:

```cython
%%cython

def f2(x):
    return 2*x*x + 3*x + 1

def trapz2(f, a, b, N):
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f(x)
    sum_y += .5 * (f(a) + f(b))
    return sum_y*h
```

```python
%timeit df.apply(lambda x: trapz2(f2, x.a, x.b, x.N), axis=1)
```

A fair speed-up just by using Cython's import, now let's try and improve things by using the --annotate flag to the declaration:

```cython
%%cython --annotate

def f2(x):
    return 2*x*x + 3*x + 1

def trapz2(f, a, b, N):
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f(x)
    sum_y += .5 * (f(a) + f(b))
    return sum_y*h
```

In the above, the colour indicates the 'typedness' of the extension, where yellower lines are closer to Python, and therefore require more calls to the Python C API, while whiter lines indicate code that is closer to pure C, hence requiring few, if any, Python API calls.

Clicking on a line reveals the C code underneath the call to Cython.

The goal in speeding up code with Cython is to turn as many lines to white as possible. The easiest way to do this is with type declarations:

```cython
%%cython --annotate

def f3(double x):
    return 2*x*x + 3*x + 1

def trapz3(f, double a, double b, int N):
    # declare types
    cdef double h, x, sum_y
    cdef int i
    # continue
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f(x)
    sum_y += .5 * (f(a) + f(b))
    return sum_y*h
```

```python
%timeit df.apply(lambda x: trapz3(f3, x.a, x.b, x.N), axis=1)
```

As we can see, another halving of the speed, just by using type declarations. The next thing we could do is inline the polynomial function. What this means is we ask the compiler to paste the function wherever it is called rather than making an expensive function call - this is particularly useful when we call $f(x)$ many, many times in the calculations of this integral, this involves:
* changing *Python* def to cdef
* add a return type
* add the *inline* keyword

```cython
%%cython --annotate

import cython

cdef inline double f4(double x):
    return 2*x*x + 3*x + 1

@cython.cdivision(True)
cpdef double trapz4(double a, double b, int N):
    # declare types
    cdef double h, x, sum_y
    cdef int i
    # continue
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f4(x)
    sum_y += .5 * (f4(a) + f4(b))
    return sum_y*h
```

```python
%timeit df.apply(lambda x: trapz4(x.a, x.b, x.N), axis=1)
```

The cdef keyword declares a C object. Everything that follows it is therefore specified in terms of C; we are essentially writing C, but using a subset of Python's syntax rules. So when we create function cdef f4, it is a C function, and not available to you in Python. This is worth considering to ensure it is not called in Python by accident.

cpdef keyword is however a hybrid declaration that creates both a C interface and a Python interface to the function.

### Using Numpy Arrays

If we profile the function now, we see that our functions are not longer near the top:

```python
%prun -l 4 df.apply(lambda x: trapz4(x.a, x.b, x.N), axis=1)
```

However *series* is being called a lot. This is because each row is being turned into a *series*.

```cython
%%cython --annotate

cimport numpy as np
import numpy as np

import cython

cdef inline double f4(double x):
    return 2*x*x + 3*x + 1

@cython.cdivision(True)
cpdef double trapz4(double a, double b, int N):
    # declare types
    cdef double h, x, sum_y
    cdef int i
    # continue
    h = (b-a)/float(N)
    sum_y = 0
    x = a
    for i in range(N):
        x += h
        sum_y += f4(x)
    sum_y += .5 * (f4(a) + f4(b))
    return sum_y*h

cpdef np.ndarray[double] apply_trapz(np.ndarray col_a, np.ndarray col_b, np.ndarray col_n):
    assert(col_a.dtype == np.float and col_b.dtype == np.float and col_n.dtype == np.int)

    cdef Py_ssize_t i, n = len(col_n)
    assert(len(col_a) == len(col_b) == n)
    cdef np.ndarray[double] res = np.empty(n)

    for i in range(len(col_a)):
        res[i] = trapz4(col_a[i], col_b[i], col_n[i])
    return res
```

```python
%timeit apply_trapz(df.a.values, df.b.values, df.N.values)
```

Our work appears to be finished in terms of optimizations here.

### Compiler Directives

For example, calculating the euclidean distance between 2 arrays:

```python
def euclidean(x, y):
    return np.sqrt(((x - y)**2).sum())
```

```python
%timeit euclidean(np.random.randn(1000), np.random.randn(1000))
```

In order to get a speed up under Cython, we need to iterate over the elements manually to aggregate them:

```cython
%%cython --annotate

import cython
cimport numpy as np
from libc.math cimport sqrt

@cython.boundscheck(False)
@cython.wraparound(False)
cpdef double euclidean2(np.ndarray[np.float64_t, ndim=1] x, np.ndarray[np.float64_t, ndim=1] y):
    cdef:
        double diff
        int i
    diff = 0
    for i in range(x.shape[0]):
        diff += (x[i] - y[i])**2
    return sqrt(diff)
```

```python
%timeit euclidean2(np.random.randn(1000), np.random.randn(1000))
```

Setting *boundscheck* to False removes boundary checking for indexing operations, forcing us to ensure that we do not try to index arrays using index values that are out of bounds. When we set *wraparound* to False, Cython will not support negative indexes, as is the case with Python. Using directives is powerful, but dangerous; if we do not index properly or make some other error, it can cause segmentation faults and/or corruption.

## Numba

Where Cython pre-compiles parts of Python code before running. Another approach is **Just-in-Time (JIT)** compilation. Numba is a compiler that runs Python code through an LLVM compiler to produce optimized bytecode for fast execution. Numba doesn't need a C/C++ compiler on your machine.

The *@jit* decorator runs the decorated function through bytecode analysis using a type inference engine.

```python
def pairwise_py(X):
    M, N = X.shape
    D = np.empty((M,M), dtype=np.float64)
    for i in range(M):
        for j in range(M):
            d = 0.0
            for k in range(N):
                tmp = X[i,k] - X[j,k]
                d += tmp*tmp
            D[i,j] = np.sqrt(d)
    return D
```

```python
%timeit pairwise_py(np.random.rand(1000, 3))
```

```python
@jit
def jit_pairwise_py(X):
    M, N = X.shape
    D = np.empty((M,M), dtype=np.float64)
    for i in range(M):
        for j in range(M):
            d = 0.0
            for k in range(N):
                tmp = X[i,k] - X[j,k]
                d += tmp*tmp
            D[i,j] = np.sqrt(d)
    return D
```

```python
%timeit jit_pairwise_py(np.random.rand(1000, 3))
```

As you can see for specific functions, jit makes a huge improvement in performance. One performance caveat is that it will only speed up code that uses NumPy arrays. When your codes includes things like lists, strings or dictionaries, it will revert to *Object* mode and not provide an appreciable speedup to your code.

# Tasks

## Task 1

Gradient descent is the method of taking steps to reduce the minimized objective function with regards to the optimum weights for a linear-regression problem. The algorithm works as:
1. Initialise $\bf w$ at uniform random, $k = 0$
1. While $k$ < maximum iterations:
    1. Calculate $\nabla_w \mathbf{e}$, as:

    $$
    \frac{\partial \mathbf{e}}{\partial w}=2X^T(y-Xw)
    $$

    2. Update $w^{(k+1)}$, as:

    $$
    w^{(k+1)}=w^{(k)} - \gamma  \frac{\partial \textbf{e}}{\partial w}
    $$
1. Until convergence

where $\gamma$ is the learning rate, capital letters refer to matrices, and hence multiplications refer to matrix-vector or matrix-matrix dot products.

Run the code below in normal Python to see how fast it is with %timeit, then try to Cythonize it and see who gets the best performance.

```python
def gradient_descent(X, y, gamma = 1.0, n_iter = 1000):
    n, P = X.shape
    nX = np.column_stack(((np.ones(n,)), X))
    w = np.random.rand(P+1)
    for i in range(1,n_iter):
        dE = np.dot((2*nX.T),(np.dot(nX,w) - y))
        w -= gamma*dE
    return w
```

```python
X = np.random.normal(3.0, 1.0, size=(10000,500))
y = np.random.randn(10000)
%timeit gradient_descent(X, y, n_iter=300)
```

```python
# your codes here
```

## Task 2

Implement the function `monte_carlo_pi(n)`, which accepts one parameter and runs as follows:

1. Set variable $acc=0$
2. For loop: variable $i$ to $n$:
    1. Set variable $x \in U[0,1]$
    2. Set variable $y \in U[0,1]$
    3. If $x^2+y^2 \lt 1$:
        1. Increment $acc=acc+1$
3. Return $\frac{4acc}{n}$

where $U[0,1]$ draws from the random uniform distribution.

Run this program using %timeit **with** and **without** jit, using $n=10^6$. Use the random function: `from random import random`. Try using the `(nopython=True)` parameter with jit also.

```python
# your codes here
```


```python
import tensorflow as tf
import tensorflow_quantum as tfq

import numpy as np
import sympy
import cirq

%matplotlib inline
from matplotlib import style
style.use('bmh')
style.use('dark_background')

import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

```python
qubit = cirq.GridQubit(0, 0)

# Define some circuits
circuit1 = cirq.Circuit(cirq.X(qubit))
circuit2 = cirq.Circuit(cirq.H(qubit))

# Convert to a tensor
input_circuit_tensor = tfq.convert_to_tensor([circuit1, circuit2])

# Define a circuit that we want to append
y_circuit = cirq.Circuit(cirq.Y(qubit))

# Instantiate our layer
y_appender = tfq.layers.AddCircuit()

# Run our circuit tensor through the layer and save the output.
output_circuit_tensor = y_appender(input_circuit_tensor, append=y_circuit)
```

```python
print(tfq.from_tensor(input_circuit_tensor))
```

    [cirq.Circuit([
        cirq.Moment(operations=[
            cirq.X.on(cirq.GridQubit(0, 0)),
        ]),
    ])
     cirq.Circuit([
        cirq.Moment(operations=[
            cirq.H.on(cirq.GridQubit(0, 0)),
        ]),
    ])]

```python
print(tfq.from_tensor(output_circuit_tensor))
```

    [cirq.Circuit([
        cirq.Moment(operations=[
            cirq.X.on(cirq.GridQubit(0, 0)),
        ]),
        cirq.Moment(operations=[
            cirq.Y.on(cirq.GridQubit(0, 0)),
        ]),
    ])
     cirq.Circuit([
        cirq.Moment(operations=[
            cirq.H.on(cirq.GridQubit(0, 0)),
        ]),
        cirq.Moment(operations=[
            cirq.Y.on(cirq.GridQubit(0, 0)),
        ]),
    ])]

### 1.4 Data

```python
def generate_data(qubits):
    """Generate training and testing data."""
    n_rounds = 20 # Produces n_rounds * n_qubits datapoints
    excitations = []
    labels = []
    for n in range(n_rounds):
        for bit in qubits:
            rng = np.random.uniform(-np.pi, np.pi)
            excitations.append(cirq.Circuit(cirq.rx(rng).on(bit)))
            labels.append(1 if (-np.pi/2) <= rng <= (np.pi/2) else -1)

    split_ind = int(len(excitations) * 0.7)
    train_excitations = excitations[:split_ind]
    test_excitations = excitations[split_ind:]

    train_labels = labels[:split_ind]
    test_labels = labels[split_ind:]

    return tfq.convert_to_tensor(train_excitations), np.array(train_labels), \
                tfq.convert_to_tensor(test_excitations), np.array(test_labels)
```

```python
sample_points, sample_labels, _, __ = generate_data(cirq.GridQubit.rect(1, 4))
print('Input:', tfq.from_tensor(sample_points)[0], 'Output:', sample_labels[0])
print('Input:', tfq.from_tensor(sample_points)[1], 'Output:', sample_labels[1])
```

    Input: (0, 0): ───Rx(0.482π)─── Output: 1
    Input: (0, 1): ───Rx(0.738π)─── Output: -1

### 1.5 Define layers

#### 1.5.1 Cluster state

```python
# The cluster state circuit
def cluster_state_circuit(bits):
    """Return a cluster state on the qubits in `bits`."""
    circuit = cirq.Circuit()
    circuit.append(cirq.H.on_each(bits))
    for this_bit, next_bit in zip(bits, bits[1:] + [bits[0]]):
        circuit.append(cirq.CZ(this_bit, next_bit))
    return circuit
```

```python
SVGCircuit(cluster_state_circuit(cirq.GridQubit.rect(1, 4)))
```

#### 1.5.2 QCNN layers

```python
def one_qubit_unitary(bit, symbols):
    """Make a Cirq circuit enacting a rotation of the bloch sphere about the X,
    Y and Z axis, that depends on the values in `symbols`.
    """
    return cirq.Circuit(
        cirq.X(bit)**symbols[0],
        cirq.Y(bit)**symbols[1],
        cirq.Z(bit)**symbols[2])

def two_qubit_unitary(bits, symbols):
    circuit = cirq.Circuit()
    circuit += one_qubit_unitary(bits[0], symbols[0:3])
    circuit += one_qubit_unitary(bits[1], symbols[3:6])
    circuit += [cirq.ZZ(*bits)**symbols[7]]
    circuit += [cirq.YY(*bits)**symbols[8]]
    circuit += [cirq.XX(*bits)**symbols[9]]
    circuit += one_qubit_unitary(bits[0], symbols[9:12])
    circuit += one_qubit_unitary(bits[1], symbols[12:])
    return circuit

def two_qubit_pool(src_qubit, sink_qubit, symbols):
    pool_circuit = cirq.Circuit()
    sink_basis_selector = one_qubit_unitary(sink_qubit, symbols[0:3])
    src_basis_selector = one_qubit_unitary(src_qubit, symbols[3:6])
    pool_circuit.append(sink_basis_selector)
    pool_circuit.append(src_basis_selector)
    pool_circuit.append(cirq.CNOT(src_qubit, sink_qubit))
    pool_circuit.append(sink_basis_selector**-1)
    return pool_circuit
```

```python
SVGCircuit(one_qubit_unitary(cirq.GridQubit(0, 0), sympy.symbols('x0:3')))
```

```python
SVGCircuit(two_qubit_unitary(cirq.GridQubit.rect(1, 2), sympy.symbols('x0:15')))
```

```python
SVGCircuit(two_qubit_pool(*cirq.GridQubit.rect(1, 2), sympy.symbols('x0:6')))
```

#### 1.5.2.1 Quantum Convolution

```python
def quantum_conv_circuit(bits, symbols):
    """Quantum Convolution Layer following the above diagram.
    Return a Cirq circuit with the cascade of `two_qubit_unitary` applied
    to all pairs of qubits in `bits` as in the diagram above.
    """
    circuit = cirq.Circuit()
    for first, second in zip(bits[0::2], bits[1::2]):
        circuit += two_qubit_unitary([first, second], symbols)
    for first, second in zip(bits[1::2], bits[2::2] + [bits[0]]):
        circuit += two_qubit_unitary([first, second], symbols)
    return circuit
```

```python
SVGCircuit(quantum_conv_circuit(cirq.GridQubit.rect(1, 8), sympy.symbols('x0:15')))
```

#### 1.5.2.2 Quantum Pooling

```python
def quantum_pool_circuit(src_bits, sink_bits, symbols):
    """A layer that specifies a quantum pooling operation.
    A Quantum pool tries to learn to pool the relevant information from two
    qubits onto 1.
    """
    circuit = cirq.Circuit()
    for src, sink in zip(src_bits, sink_bits):
        circuit += two_qubit_pool(src, sink, symbols)

    return circuit
```

```python
test_bits = cirq.GridQubit.rect(1, 8)

SVGCircuit(quantum_pool_circuit(test_bits[:4], test_bits[4:], sympy.symbols('x0:6')))
```

### 1.6 Model definition

```python
def create_model_circuit(qubits):
    """Create sequence of alternating convolution and pooling operators
    which gradually shrink over time."""
    model_circuit = cirq.Circuit()
    symbols = sympy.symbols('qconv0:63')
    # Cirq uses sympy.Symbols to map learnable variables. TensorFlow Quantum
    # scans incoming circuits and replaces these with TensorFlow variables.

    model_circuit += quantum_conv_circuit(qubits, symbols[0:15])
    model_circuit += quantum_pool_circuit(qubits[:4], qubits[4:], symbols[15:21])

    model_circuit += quantum_conv_circuit(qubits[4:], symbols[21:36])
    model_circuit += quantum_pool_circuit(qubits[4:6], qubits[6:], symbols[36:42])

    model_circuit += quantum_conv_circuit(qubits[6:], symbols[42:57])
    model_circuit += quantum_pool_circuit([qubits[6]], [qubits[7]], symbols[57:63])

    return model_circuit

# Create our qubits and readout operations in Cirq
cluster_state_bits = cirq.GridQubit.rect(1, 8)
readout_operators = cirq.Z(cluster_state_bits[-1])

# Build a sequential model enacting the logic in 1.3 of this notebook.
# Here you are making the static cluster state prep as a part of the AddCircuit and the
# "quantum datapoints" are coming in the form of excitation
excitation_input = tf.keras.Input(shape=(), dtype=tf.dtypes.string)
cluster_state = tfq.layers.AddCircuit()(
        excitation_input, prepend=cluster_state_circuit(cluster_state_bits))

quantum_model = tfq.layers.PQC(create_model_circuit(cluster_state_bits), readout_operators)(cluster_state)

qcnn_model = tf.keras.Model(inputs=[excitation_input], outputs=[quantum_model])

# To show the keras plot of the model
tf.keras.utils.plot_model(qcnn_model,
                          show_shapes=True,
                          show_layer_names=False,
                          dpi=70)
```

    Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

### 1.7 Train the model

```python
# Generate some training data
train_excitations, train_labels, test_excitations, test_labels = generate_data(cluster_state_bits)

# Custom accuracy metric
@tf.function
def custom_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true)
    y_pred = tf.map_fn(lambda x: 1.0 if x >= 0 else -1.0, y_pred)
    return tf.keras.backend.mean(tf.keras.backend.equal(y_true, y_pred))

qcnn_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.02),
                  loss=tf.losses.mse,
                  metrics=[custom_accuracy])

history = qcnn_model.fit(x=train_excitations,
                        y=train_labels,
                        batch_size=16,
                        epochs=25,
                        verbose=1,
                        validation_data=(test_excitations, test_labels))
```

    Train on 112 samples, validate on 48 samples
    Epoch 1/25

```python
plt.plot(history.history['loss'][1:], label='Training')
plt.plot(history.history['val_loss'][1:], label='Validation')
plt.title('Training a Quantum CNN to Detect Excited Cluster States')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
```

## Hybrid Models

```python

```


# Coaddition and Subtraction

Version 0.1

By Yusra AlSayyad (Princeton University)

This notebook provides some problems for ground-based coaddition and subtraction.

We add and subtract images for different purposes: addition to get the benefits of longer exposures and subtraction to reveal what has changed.  However, in order to perform either operation, the following  image characteristics need to be normalized:
* Astrometric calibration (i.e. WCS)
* Photometric calibration (i.e. Zeropoint)
* Background level
* Point Spread Function (PSF) (optional for coaddition)

For these problems, we'll assume that we have images that have already been normalized for WCS, zeropoint, and background variations.

### Problem 1) Expected depth of a coadd

In order of magnitude calculations, you may hear people throw around the statement that the depth of a coadd increases by a factor of $\sqrt{N}$  for N single-epoch images. Where does this come from? Under what conditions is this a good approximation?

We are going to use our same star + noise 1-D simulation from  `IntroductionToBasicStellarPhotometry.ipynb` and `FindingSources.ipynb`:

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import MultipleLocator
from scipy.stats import norm
```

```python
def pixel_plot(pix, counts, fig=None, ax=None):
    '''Make a pixelated 1D plot'''
    if fig is None and ax is None:
        fig, ax = plt.subplots()

    ax.step(pix, counts,
            where='post')

    ax.set_xlabel('pixel number')
    ax.set_ylabel('relative counts')
    ax.xaxis.set_minor_locator(MultipleLocator(1))
    ax.xaxis.set_major_locator(MultipleLocator(5))
    fig.tight_layout()
    return fig, ax

# It is sufficient to copy and paste from
# your introductionToBasicStellarPhotometry noteboook

def phi(x, mu, fwhm):
    """Evalute the 1d PSF N(mu, sigma^2) along x

    Parameters
    ----------
    x : array-like of shape (n_pixels,)
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector

    Returns
    -------
    flux : array-like of shape (n_pixels,)
        Flux in each pixel of the input array
    """
    sigmaPerFwhm = 2*np.sqrt(2*np.log(2))
    sigma = fwhm/sigmaPerFwhm
    flux =  norm.pdf(x, mu, sigma)
    return flux

# Define your image simulation function to
# It is sufficient to copy and paste from
# your introductionToBasicStellarPhotometry noteboook
# Note that the background S should now be supplied as
# an array of length (x) or a constant.

def simulate(x, mu, fwhm, S, F):
    """simulate a noisy stellar signal

    Parameters
    ----------
    x : array-like
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector
    S : float or array-like of len(x)
        Sky background for each pixel
    F : float
        Total stellar flux

    Returns
    -------
    noisy_counts : array-like (same shape as x)
        the (noisy) number of counts in each pixel
    """
    # complete
    signal = F * phi(x=x, mu=mu, fwhm=fwhm) + S
    noise = np.random.normal(loc=0, scale=np.sqrt(signal))
    noisy_counts = signal + noise

    return noisy_counts
```

### Problem 1.1) Make a simple mean coadd

Simulate N observations of a star, and coadd them by taking the mean of the N observations. (We can only do this because they are already astrometrically and photometrically aligned and have the same background value.)

```python
MU = 35
S = 100
F = 100
FWHM = 5

x = np.arange(100)

# simulate a single observation of the star and plot:
y = simulate(x=x, mu=MU, F=F, fwhm=FWHM, S=S)
pixel_plot(x, y)
```

```python
# Write a simulateN function that returns an array of size (N, x)
# representing N realizations of your simulated star
# This will stand in as a stack of multiple observations of one star

def simulateN(x, mu, fwhm, S, F, N):
    """simulate a noisy stellar signal

    Parameters
    ----------
    x : array-like
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector
    S : float or array-like of len(x)
        Sky background for each pixel
    F : float
        Total stellar flux
    N: int
        Number of images to simulate

    Returns
    -------
    noisy_counts : array-like of shape (N, x)
        the (noisy) number of counts in each pixel
    """
    signal = F * phi(x=x, mu=mu, fwhm=fwhm) + S
    noise = np.random.normal(loc=0, scale=np.sqrt(signal), size=(N, len(x)))
    noisy_counts = signal + noise
    return noisy_counts
```

```python
# simulate N=50 images with the same star
x = np.arange(100)
y = simulateN(x=x, mu=35, F=100, fwhm=5, S=100, N=50)

# where stack is an array of size (50, 100) representing a pile of 50 images with 100 pixels

# coadd by taking the mean and plot the result
coadd = np.mean(y, axis=0)
pixel_plot(x, coadd)

# Try a few different N to see how it affects the S/N of your result

```

```python
# Plot the coadds of N=[1, 10, and 100] on the same plot:

fig = None
ax = None
x = np.arange(100)
for N in [1, 10, 100]:
    y = simulateN(x=x, mu=35, F=100, fwhm=5, S=100, N=N)

    # coadd by taking the mean and plot the result
    coadd = np.mean(y, axis=0)
    fig, ax = pixel_plot(x, coadd, fig=fig, ax=ax)
```

### Problem 1.2)  SNR vs N

Now compute the observed SNR of the simulated star on each coadd and compare to the expected SNR in the idealized case. The often repeated mnemonic for SNR inscrease as a function of number of images, $N$, is that noise decreses like $\sqrt{N}$. This is of course idealized case where the noise in each observation is identical.

Using your `simulateN` function, simulate a series of mean coadds with increasing N.

* First, plot the empirical noise/uncertainty/stdev as a function of N. and overplot the expected uncertainty given the input sky level. You can use an area you know isn't touched by the star.
* Next, plot the empirical SNR of the star (measured flux/fluxErr) as a function of N. Overplot the expected SNR. You can assume you know the sky level.

Your expected scaling with N should roughly track your empirical estimate.

```python
# complete

MU = 35
S = 100
F = 100
FWHM = 5

# hint. One way to start this
std = []
flux = []
fluxErrExpect = []
fluxErrObs = []
Ns = np.arange(1, 1000, 1)
for N in Ns:
    y = simulateN(x=x, mu=MU, F=F, fwhm=FWHM, S=S, N=N)
    # now coadd them:
    coadd = np.mean(y, axis=0)
    errEst = np.std(coadd[50:100])
    std.append(errEst)
    flux.append(np.sum(phi(x=x, mu=MU, fwhm=FWHM)*(coadd-100))/np.sum(phi(x=x, mu=MU, fwhm=FWHM)**2))
    fluxErrExpect.append(np.sum(phi(x=x, mu=MU, fwhm=FWHM)*(10/np.sqrt(N)))/np.sum(phi(x=x, mu=MU, fwhm=FWHM)**2))
    fluxErrObs.append(np.sum(phi(x=x, mu=MU, fwhm=FWHM)*(errEst))/np.sum(phi(x=x, mu=MU, fwhm=FWHM)**2))

```

```python
plt.title('Flux estimate in coadd of N images')
plt.plot(Ns, flux, '.', label="coadd")
#plt.plot(Ns, 100, label="expected")
plt.xlabel('N')
plt.ylabel('Coadd PSF Flux')
```

```python
plt.plot(Ns, np.array(std), '.', label="coadd", alpha=0.5)
plt.plot(Ns, np.sqrt(S)/np.sqrt(Ns), label="expected")
plt.xlabel('N')
plt.ylabel('pixel noise')
plt.legend()
```

```python
plt.plot(Ns, np.array(flux)/np.array(fluxErrObs), '.', label="coadd", alpha=0.5)
plt.plot(Ns, F/np.array(fluxErrExpect), label="expected")
plt.xlabel('N')
plt.ylabel('PSF Flux SNR')
plt.legend()
```

## Problem 2) PSFs and Image weights in coadds

Problem (1) pretends that the input images are identical in quality, however this is never the case in practice. In practice, adding another image does not necessarily increase the SNR. For example, imagine you have two exposures, but in one the dome light was accidentally left on. A coadd with these two images weighted equally will have a worse SNR than the first image alone. Therefore the images should be aggregated with a weighted mean, so that images of poor quality don't degrade the quality of the coadd. What weights to we pick?

Weights can be chosen to either minimize the variance on the coadd or maximize the SNR of point sources on the coadd.

#### Some background:

Assuming that all noise sources are independent, the SNR of the measurement of flux from a star is:
\begin{equation}
SNR \propto {{N_{\rm photons}}\over{ \sigma_{\rm sky}} \sqrt{A} },
\end{equation}
where $N_{\rm photons}$ is the number of photons detected from the star,
$A$ is the area in pixels covered by the star.
The per-pixel sky noise  $\sigma_{sky}$ includes all sources of noise: dark current, read noise and sky-background, and it coded in the variance plane of the image. For the epoch $i$,  $\sigma^2_{i, {\rm sky}}$ is the average of the variance plane.

The $N_{\rm photons}$ is proportional to transparency $T$, and the area that the stellar photons cover is determined by the seeing: $A \propto {\rm FWHM}^2$.

Therefore, a **coadd optimized for point-source detection** would, to weight each image by the SNR$^2$, use the following as weights which prefers good-seeing epochs taken when the sky is transparent and dark.
\begin{align}
w_i & = {\rm SNR}^2 \propto {{T_i^2}\over{{\rm FWHM_i}^2 \sigma_i^2}}.
\end{align}

The usual inverse-variance weighting which produces the **minimum-variance co-add**, is given by,
\begin{align}
w_i & =T_i^2/\sigma_i^2
\end{align}

In practice, the factor of $T_i^2$ is incorporated into the variance when flux-scaling the single-epoch images to a common zeropoint. This step multiplies the image by a scale-factor, which increases the variance of the image by the square of the scale factor. The scale factor is inversely proportional to the transparency, so that
$\sigma_{scaled}= \sigma/T$.

For this problem assume the images are all on the same zeropoint (like problem 1) i.e. T=1.

### Problem 2.1  Weighting images in Variable Sky

Now simulate 50 observations of stars with Sky `S` ranging from 100 to 1000. Remember to subtract this background off before stacking this time! Plot the plain (unweighted) mean coadd vs. the minimum variance coadd. Weights should add up to 1. What's the empirical noise estimate of the coadd?

```python
N = 50
skyArr = np.linspace(1, 1000, N)
stack = np.zeros((N, 100))
stackErr = np.zeros((N, 100))
inputErr = np.zeros((N, 100))

for i, sky in enumerate(skyArr):
    y = simulate(x=x, mu=MU, F=F, fwhm=FWHM, S=sky)
    stack[i] = y - np.repeat(sky, 100)
    stackErr[i] = np.sqrt(y)
    inputErr[i] = np.repeat(np.sqrt(sky), 100)
```

    /Users/yusra/.local/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in sqrt
      # Remove the CWD from sys.path while we load stuff.

```python
plt.plot(np.average(stack, weights=1/inputErr**2, axis=0), 'b-.', label='Minimum Variance')
plt.plot(np.average(stack, axis=0), 'r-.', label='Mean')
plt.legend()
```

### Problem 2.2  Weighting images in Variable Seeing

Simulate 50 observations with FWHM's ranging from 2-10 pixels. Keep the flux amplitude `F`, and sky noise `S` both fixed.
Generate two coadds, (1) with the weights that minimize variance and (2) with the weights that maximize point source SNR. Weights should add up to 1. Plot both coadds.

```python
N = 100
S = 100
FWHMs = np.linspace(2, 20, N)
stack = np.zeros((N, 100))
stackErr = np.zeros((N,100))
fwhmArr = np.zeros((N, 100))

for i, fwhm in enumerate(FWHMs):
    y = simulate(x=x, mu=MU, F=F, fwhm=FWHM, S=S)
    stack[i] = y - S
    stackErr[i] = np.sqrt(y)
    fwhmArr[i] = np.repeat(fwhm, 100)
```

```python
# complete
plt.plot(np.average(stack, weights=1/fwhmArr**2, axis=0), 'b--', label='1/fwhm**2')
plt.plot(np.average(stack, axis=0), 'r--', label='Mean')
plt.legend()
```

### Problem 2.3 Image variance vs per pixel variance (Challenge Problem)

Why do we use per image variances instead of per pixel variances? Let's see! Start tracking the per-pixel variance when you simulate the star. Make a coadd of 200 observations with FWHM's ranging from 2 to 20 pixels.  Make a coadd weighted by the per-pixel inverse variance. How does the profile of the star look in this coadd compared to an unweighted coadd and compared to the coadd with the $w_i  = \frac{1}{{\rm FWHM_i}^2 \sigma_i^2}$? (You may have to plot the difference to see the change).

```python
N = 5000
S = 100
FWHMs = np.linspace(2, 20, N)
stack = np.zeros((N, 100))
stackErr = np.zeros((N,100))
fwhmArr = np.zeros((N, 100))
inputArr = np.zeros((N, 100))

for i, fwhm in enumerate(FWHMs):
    y = simulate(x=x, mu=MU, F=F, fwhm=fwhm, S=S)
    stack[i] = y - S
    stackErr[i] = np.sqrt(y)
    fwhmArr[i] = np.repeat(fwhm, 100)

plt.plot(np.average(stack, weights=1/stackErr**2, axis=0), 'r-.', label='per-pixel')
plt.plot(np.average(stack, weights=1/FWHMs**2, axis=0), 'b--', label='per-image')
plt.plot(np.average(stack, axis=0), 'g--', label='unweighted')
plt.legend()

```

## Problem 3) Dipoles in Image Subtraction

In the lesson, we said that just because you see a dipole in a difference image, does not mean that the astrometric registration is terrible. For this problem, we'll forgo the pixelated simulated star from the previous problems and operate with Gaussian profiles.

```python
# Create two Gaussian 1-D profiles with

ASTROM_OFFSET = 0.1  # units of e.g. pixels
FLUX_SCALE = 1. # units of e.g nanojansky
PSF = 1  # rms pixels

x = np.linspace(-5, 5)
y1 = FLUX_SCALE * norm.pdf(x, ASTROM_OFFSET, PSF)
y2 = FLUX_SCALE * norm.pdf(x, -ASTROM_OFFSET, PSF)
plt.plot(x, y1, label='profile 1')
plt.plot(x, y2, label='profile 2')
plt.xlabel('x (pixel)')
plt.ylabel('y (flux)')
plt.legend()

```

### Problem 3.1) Plot the difference of these two profiles:

```python
# complete
```

### Problem 3.2)  What if we have amazing astrometric registration
and shrink the astrometric offset by a factor of a thousand. Is there a star sufficiently bright to produce the same dipole? What is its `FLUX SCALE`?

```python
# Plot both dipoles (for the offset=0.1 and the offset=0.0001 in the same figure.
# Same or different subplots up to you.

OFFSET = 0.1
FLUX = 1.

fig = plt.figure(figsize=(12,6))
plt.subplot(121)
x = np.linspace(-5, 5)
y1 = FLUX * norm.pdf(x, OFFSET, 1)
y2 = FLUX * norm.pdf(x, -OFFSET, 1)

plt.plot(x, y1-y2)
plt.ylim(-0.1, +0.1)
plt.ylabel('counts')
plt.xlabel('x')
plt.subplot(122)

OFFSET = 0.0001
FLUX = 1000.
x = np.linspace(-5, 5)
y1 = FLUX * norm.pdf(x, OFFSET, 1)
y2 = FLUX * norm.pdf(x, -OFFSET, 1)
plt.plot(x, y1-y2)
plt.xlabel('x')
plt.ylim(-0.1, +0.1)
plt.tight_layout()
```

### Problem 3.3) Distance between peaks.

Does the distance between the dipole's positive and negative peaks depend on the astrometric offset? If not, what *does* it depend on?  You can answer this by vizualizing the dipoles vs offsets. But for a **challenge, measure the distance between peaks** and plot them as a function of astrometric offset or another factor.

```python
# complete

# Adapted from Day 3 PSF Photometry notebook
def find_peaks(image, threshold):
    # We are going to append the peaks we find to these two lists
    peak_loc = []
    skip_edge = 3
    for i, pixel in enumerate(image):
        if i < skip_edge or (i > len(image) - skip_edge):
            continue
        if pixel > threshold and pixel >= np.max(image[i-2:i+2]):
            peak_loc.append(i)

    return np.array(peak_loc)

def distance(x, y1, y2):
    high = x[find_peaks(y2-y1, 0)[0]]
    low =  x[find_peaks(y1-y2, 0)[0]]
    return low - high

```

```python
# Hold FWHM fixed:
offsetArr =  np.logspace(-4, 0.2, 50)
PSF = 1
x = np.linspace(-5, 5, 1000)
peakDist = []
FLUX = 1.

for offset in offsetArr:
    y1 = (FLUX) * norm.pdf(x, offset, PSF)
    y2 = (FLUX) * norm.pdf(x, -offset, PSF)
    peakDist.append(distance(x, y1, y2))
```

```python
plt.plot(offsetArr, peakDist)
plt.axvline(PSF, color='grey') # line for offset = PSF
plt.ylabel('peak dist (pixels)')
plt.xlabel('offset (pixels)')
plt.title('PSF rms = 1 "pixel"')
plt.xscale('log')
```

```python
# Hold OFFSET fixed:
offset = 0.1
psfArr = np.logspace(-1,1, 50)
x = np.linspace(-20, 20, 1000)
peakDist = []
FLUX = 1.

for psf in psfArr:
    y1 = (FLUX) * norm.pdf(x, offset, psf)
    y2 = (FLUX) * norm.pdf(x, -offset, psf)
    peakDist.append(distance(x, y1, y2))
```

```python
plt.plot(psfArr, peakDist)
plt.axvline(0.1, color='grey') # line for offset = PSF
plt.ylabel('peak dist (pixels)')
plt.xlabel('PSF width (sigma)')
plt.title('offset = 0.1 "pixel"')
plt.xscale('log')
```

*For an astrometric offset that is << the width of the PSF, the distance between the peaks is determined by the PSF width. In other words, the distance between the peaks in a dipole tells you nothing about the astrometric offset (or effect of DCR).*

### Problem  3.4)
In the problem setup we assumed that the astrometric error was < the PSF width. Is this assumption likely to hold in an image you'd get from a wide-field imager like Rubin's?

*Yes, For Rubin data you can assume that the astrometric error is much smaller than the PSF width. See table of angular scales in the Astrometry lecture on Day 2 for reference.*

```python

```