# 数字認識タスク
Notebook の利用が初めての場合 - ようこそ！Notebook はコードの近くに結果が表示されるのを見ながら実行できる素晴らしいツールです。
1. セルを実行するには、セルをクリックして SHIFT-Enter を押します。
2. 何かが実行されると、変数はメモリに保存されます - それらを調べてください。

# 訳注 Jupyter Notebook の使い方:
このワークショップでは、Azure Notebooks Service によってホストされている Jupyter ノートブックを使用します。あなたが Jupyter に精通しているならば、このセルを飛ばしてください。そうでない場合は、ここでいくつかのヒントがあります。

* Python 3.6 のカーネルを選択してください。
* Jupyter には「セル」があります。ノートブックのどこかをクリックすると、ポインタを含むセルが選択されます。
* セル内のコードを実行または実行するには、トップメニューで [セル]-[セルの実行] を選択します。または、Shift + Enterキーを使用することもできます。
* セルを編集するには - セルをダブルクリックします。
* セルを実行すると、セルの左側の外側にあるポインタが [ ] から [ \* ] に変わります。セルの実行が終了すると、[ \* ] が数字に変わります。
* ノートブックまたはセルがハング(フリーズ)した場合は、トップメニューからノートブック全体の [Kernel]-[Restart Kernel and Clear All Outputs] を選択するか、そのセルだけを選択して [Kernel]-[Interrupt kernel] を選択します。再起動には数秒かかり、その後ですべてのセルを再実行する必要があります。

Jupyterでは、順番に実行する必要はありません。すでに実行された前のセルに戻って再度実行することができます。特定のセルのコードが、前後のセルと依存関係を持たない限り、問題ありません。

## 始めましょう
最初のセルは必要なライブラリをインポートします。ここから始めましょう。

```python
import torch
import numpy as np
from PIL import Image
import torch.nn as nn
import torch.onnx as onnx
import torch.nn.functional as F
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
```

    /home/nbuser/anaconda3_501/lib/python3.6/site-packages/matplotlib/font_manager.py:229: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
      'Matplotlib is building the font cache using fc-list. '

## データ!
データがなければ、機械学習では何もできません。今回は「**28x28のベクトルで数字を予測できるか？**」という課題を与えられています。

用意できたら、データを見てみる必要があります。次のセルは数字を視覚化するヘルパー関数です（健全性チェック）。

```python
def draw_digits(digits):
    fig, axes = plt.subplots(6, 20, figsize=(18, 7),
                            subplot_kw={'xticks':[], 'yticks':[]},
                            gridspec_kw=dict(hspace=0.1, wspace=0.1))
    for i, ax in enumerate(axes.flat):
        X, y = digits[i]
        ax.imshow(255 - X.reshape(28,28) * 255, cmap='gray')
        ax.set_title('{:.0f}'.format(torch.argmax(y).item()))
```

次のセルは、標準的な数値データセット（MNISTと呼ばれる）をダウンロードします。この呼び出しの transform 部分と target_transform 部分は、これから行うモデルにより適したデータにするための変換手順がいくつか入っています。

```python
digits = datasets.MNIST('data', train=True, download=True,
                        transform=transforms.Compose([
                            transforms.ToTensor(),
                            transforms.Lambda(lambda x: x.reshape(28*28))
                        ]),
                        target_transform=transforms.Compose([
                            transforms.Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, y, 1))
                        ])
                     )
```

ここでデータの健全性をチェックします。

```python
draw_digits(digits)
```

xやyを表示して、ざっとデータを見てください。これらは本当に単なる数字です！それらを見てみるといくつか奇妙に思えるかもしれません：

1.「画像は 0～255 の 784個のベクトルではないのですか？」と思いましたか。データは見た通りです！新しい範囲が 0～1 になるように255で割ってベクトルを正規化しただけです（数値がより正確になるようにしました）
2.「y が数値の答えだったのではないですか？代わりに 10次元のベクトルになっています」と思いましたか。その通りです！これは答えの One-Hot 表現と呼ばれます。正解のインデクスの位置に "1" が入っています。繰り返しになりますが、これにより、これから作成するモデルで数値演算がよりよく機能するようになります。

```python
x, y = digits[0]
print(x)
print(y)
```

    tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0118,
            0.0706, 0.0706, 0.0706, 0.4941, 0.5333, 0.6863, 0.1020, 0.6510, 1.0000,
            0.9686, 0.4980, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1176, 0.1412, 0.3686, 0.6039,
            0.6667, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.8824, 0.6745, 0.9922,
            0.9490, 0.7647, 0.2510, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1922, 0.9333, 0.9922, 0.9922,
            0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9843, 0.3647, 0.3216,
            0.3216, 0.2196, 0.1529, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.8588, 0.9922,
            0.9922, 0.9922, 0.9922, 0.9922, 0.7765, 0.7137, 0.9686, 0.9451, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3137,
            0.6118, 0.4196, 0.9922, 0.9922, 0.8039, 0.0431, 0.0000, 0.1686, 0.6039,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0549, 0.0039, 0.6039, 0.9922, 0.3529, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.5451, 0.9922, 0.7451, 0.0078, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0431, 0.7451, 0.9922, 0.2745,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1373, 0.9451,
            0.8824, 0.6275, 0.4235, 0.0039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.3176, 0.9412, 0.9922, 0.9922, 0.4667, 0.0980, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.1765, 0.7294, 0.9922, 0.9922, 0.5882, 0.1059, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0627, 0.3647, 0.9882, 0.9922, 0.7333,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.9765, 0.9922,
            0.9765, 0.2510, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1804, 0.5098, 0.7176, 0.9922,
            0.9922, 0.8118, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.1529, 0.5804, 0.8980, 0.9922, 0.9922,
            0.9922, 0.9804, 0.7137, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0941, 0.4471, 0.8667, 0.9922, 0.9922, 0.9922,
            0.9922, 0.7882, 0.3059, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0902, 0.2588, 0.8353, 0.9922, 0.9922, 0.9922, 0.9922,
            0.7765, 0.3176, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0706, 0.6706, 0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.7647,
            0.3137, 0.0353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.2157, 0.6745, 0.8863, 0.9922, 0.9922, 0.9922, 0.9922, 0.9569, 0.5216,
            0.0431, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.5333, 0.9922, 0.9922, 0.9922, 0.8314, 0.5294, 0.5176, 0.0627,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000])
    tensor([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.])

# モデルを選択する
いくつかのデータが得られたので、今度はうまくいくと思われるモデルの選択を開始します。これがデータサイエンスの科学的な部分が出てくるところです：私たちは推測し、それから仮定が正しいかどうかチェックします。 784個の蛇口から10本の異なるホースに水を分配する必要がある水道管のようなモデルを想像してみてください。これらの784個の蛇口は数字の個々のピクセルを表し、最後の10個のホースは実際の数（または少なくとも水が最も多く出ているもののインデックス）を表します。私たちの仕事は今その間の配管を選ぶことです。

次の3つのセルは、順に難しくなる 3つの別の構造を表しています。

1. 最初のモデルは単純な線形モデルです
2. 2番目のモデルは3層ニューラルネットワークです
3. 最後のモデルは全てが畳み込み層で構成されるニューラルネットワークです

それらがどのように働くかを完全に説明することはこのチュートリアルの範囲外ですが、正しいインデックスから最も多くの水を押し出せるように、最後に正しい水圧を生み出すように調整されなければならない内部ノブで配管していると想像してください 。下のセルほど、配管とそれに対応する内部ノブがより複雑になります。

```python
class SimpleLinear(nn.Module):
    def __init__(self):
        super(SimpleLinear, self).__init__()
        self.layer1 = nn.Linear(28*28, 10)

    def forward(self, x):
        x = self.layer1(x)
        return F.softmax(x, dim=1)
```

```python
class NeuralNework(nn.Module):
    def __init__(self):
        super(NeuralNework, self).__init__()
        self.layer1 = nn.Linear(28*28, 512)
        self.layer2 = nn.Linear(512, 512)
        self.output = nn.Linear(512, 10)

    def forward(self, x):
        x = F.relu(self.layer1(x))
        x = F.relu(self.layer2(x))
        x = self.output(x)
        return F.softmax(x, dim=1)
```

```python
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = x.view(-1, 1, 28, 28)
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.softmax(x, dim=1)
```

# モデルパラメータの最適化
いくつかのモデルができたので、数字を認識するのに良い仕事ができるかどうか確認するために内部パラメータを最適化する時が来ました！訓練する方法を調整するために最適化アルゴリズムを与える際に使用する、いくつかのパラメータです - これらはハイパーパラメータと呼ばれます。次に示す2つの変数です。

```python
learning_rate = 1e-3
batch_size = 64
epochs = 5
```

'learning_rate' は基本的に、アルゴリズムがモデルパラメータを学習する速度を指定します。あなたが「それを5000万に設定するのはどうかな？」と考えたとします。それが悪い考えである理由の最も良い例えはゴルフです。私はゴルフがうまくないので、本当によくわかりませんが、ショットを沈めようとしているのに毎回同じ距離ボールを打つようなものです。簡単ですね？今いる場所から穴までの正確な距離を打てばいいのです！それで完了です！さて、あなたは穴がどこにあるのかわからないが、おおざっぱな方向は知っているとします。あなたが選ぶ距離は実際に重要です。距離が長すぎると穴を逃すことになり、それを叩くと再び行きすぎます。距離が短すぎると、穴にたどり着くまでに時間がかかりますが、最終的には穴に到達するはずです。基本的には、1ショットあたりの正しい距離を推測してから試してみる必要があります。それが、穴に入れるための正しいパラメーターを見つけられるように、基本的に学習率がすることです。（さて、ゴルフの話は終わりです）

以下は、完全に動作するのに必要な 3つのものです:
1. **モデル** - 私たちが作ろうとしている関数であり、正解の数値を返すべきものです
2. **コスト関数** (損失関数と呼ばれることもあります) - ゴルフの話は終わったと約束したことを覚えていますが、嘘をつきました。おかしなゴルフゲームであなたが穴の大体の方向を知っていると言ったのを覚えていますか？ コスト関数は穴までの距離を教えてくれます。ゼロになると穴に入ったことになるのです！実際の科学的な用語では、コスト関数は、モデルが正しい答えを得るのにどれほど悪いかを示すものです。ゴルフショットと同じように、コスト関数が減少するのも見るべきです。減少しなければ、何か間違っています。そうならば、ショットの距離（または学習率）をもっと小さい値に変更して、もう一度試してみます。それでもうまくいかない場合は、モデルを変更してください。
3. **オプティマイザー** - この部分は、実際にモデルパラメーターを変更する部分です。それは我々がボールを打つべき方向の意味を持ち、正しい数字を予測する最良の内部の蛇口を見つけるために、モデル内部のすべての値を更新します。ここでは、バイナリークロスエントロピーコスト関数を使用します。これがうまく機能することがわかっているからです。さまざまなシナリオに合わせて、さまざまなコスト関数を選択できます。

```python
# 実行する場所
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print('Using {} device'.format(device))

# モデルの選択
model = SimpleLinear().to(device)
#model = NeuralNework().to(device)
#model = CNN().to(device)
print(model)

# 最良のパラメーターを決定するのに使用するコスト関数
cost = torch.nn.BCELoss()

# 最適なパラメーターを作成するために使用
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
```

    Using cpu device
    SimpleLinear(
      (layer1): Linear(in_features=784, out_features=10, bias=True)
    )

'CUDA' について言及するのをすっかり忘れていました。基本的に for ループの中はすべての大きな行列の乗算問題であるため、GPUはこのプロセス全体をスピードアップします。PyTorchは基本的にどこで実行するか（CPU または CUDA - CUDA は計算を GPU に移行するためのプラットフォームです）をモデルに伝えるだけでいいのです。素晴らしい。

それでは、学習に進みます！データローダーの仕事は、データセット全体（ここでは 60,000個の例とそれに対応するラベル）を反復処理することですが、処理するデータを batch_sizeの サイズごとに取得します。batch_size は選択する必要があるもう1つのハイパーパラメータです。'epoch' は、データセット全体をループする回数です（これもまた、実験の進行状況に基づいて選択したものです）。

次のセルの実行には時間がかかります。

```python
# loader
dataloader = DataLoader(digits, batch_size=batch_size, num_workers=0, pin_memory=True)

# golfing time!
for t in range(epochs):
    print('Epoch {}'.format(t))
    print('-------------------------------')
    # go through the entire dataset once
    for batch, (X, Y) in enumerate(dataloader):
        X, Y = X.to(device), Y.to(device)
        # zero out gradient
        optimizer.zero_grad()
        # make a prediction on this batch!
        pred = model(X)
        # how bad is it?
        loss = cost(pred, Y)
        # compute gradients
        loss.backward()
        # update parameters
        optimizer.step()

        if batch % 100 == 0:
            print('loss: {:>10f}  [{:>5d}/{:>5d}]'.format(loss.item(), batch * len(X), len(dataloader.dataset)))

    print('loss: {:>10f}  [{:>5d}/{:>5d}]\n'.format(loss.item(), len(dataloader.dataset), len(dataloader.dataset)))

print('Done!')
```

    Epoch 0
    -------------------------------
    loss:   0.327997  [    0/60000]
    loss:   0.117502  [ 6400/60000]
    loss:   0.095166  [12800/60000]
    loss:   0.082182  [19200/60000]
    loss:   0.064765  [25600/60000]
    loss:   0.073638  [32000/60000]
    loss:   0.050911  [38400/60000]
    loss:   0.082337  [44800/60000]
    loss:   0.066851  [51200/60000]
    loss:   0.073611  [57600/60000]
    loss:   0.028481  [60000/60000]

    Epoch 1
    -------------------------------
    loss:   0.052711  [    0/60000]
    loss:   0.051933  [ 6400/60000]
    loss:   0.046167  [12800/60000]
    loss:   0.063981  [19200/60000]
    loss:   0.041912  [25600/60000]
    loss:   0.065289  [32000/60000]
    loss:   0.036312  [38400/60000]
    loss:   0.073197  [44800/60000]
    loss:   0.056139  [51200/60000]
    loss:   0.073723  [57600/60000]
    loss:   0.020512  [60000/60000]

    Epoch 2
    -------------------------------
    loss:   0.039570  [    0/60000]
    loss:   0.047737  [ 6400/60000]
    loss:   0.037469  [12800/60000]
    loss:   0.057078  [19200/60000]
    loss:   0.038286  [25600/60000]
    loss:   0.062519  [32000/60000]
    loss:   0.032140  [38400/60000]
    loss:   0.070401  [44800/60000]
    loss:   0.052664  [51200/60000]
    loss:   0.074159  [57600/60000]
    loss:   0.017821  [60000/60000]

    Epoch 3
    -------------------------------
    loss:   0.033085  [    0/60000]
    loss:   0.046338  [ 6400/60000]
    loss:   0.033637  [12800/60000]
    loss:   0.052403  [19200/60000]
    loss:   0.037789  [25600/60000]
    loss:   0.060779  [32000/60000]
    loss:   0.030151  [38400/60000]
    loss:   0.068874  [44800/60000]
    loss:   0.050695  [51200/60000]
    loss:   0.074084  [57600/60000]
    loss:   0.016465  [60000/60000]

    Epoch 4
    -------------------------------
    loss:   0.029473  [    0/60000]
    loss:   0.045452  [ 6400/60000]
    loss:   0.031541  [12800/60000]
    loss:   0.049232  [19200/60000]
    loss:   0.038242  [25600/60000]
    loss:   0.059505  [32000/60000]
    loss:   0.029063  [38400/60000]
    loss:   0.067829  [44800/60000]
    loss:   0.049341  [51200/60000]
    loss:   0.073767  [57600/60000]
    loss:   0.015651  [60000/60000]

    Done!

さて、ここでは実際には何をしたのでしょうか。素晴らしい質問です。最初の実行した SimpleLinear モデルから説明します。基本的に1つの行列と1つのベクトルを学習しています。それだけです。実際にはこういうことです:

\begin{align}
prediction = W \cdot x + b
\end{align}

\begin{align}
\begin{bmatrix}
\hat{y}_1 \\ \vdots \\ \hat{y}_{10}
\end{bmatrix}_{10 \times 1} =
\begin{bmatrix}
w_{1,1} &  \ldots &  w_{1,784} \\
\vdots    & \ddots &  \\
w_{10,1}  & \ldots & w_{10, 784} \\
\end{bmatrix}_{10 \times 784}
\cdot
\begin{bmatrix}x_1 \\ x_2 \\ \vdots \\ x_{783} \\ x_{784}\end{bmatrix}_{784 \times 1}
 +
\begin{bmatrix}
b_1 \\ \vdots \\ b_{10}
\end{bmatrix}_{10 \times 1}
\end{align}

行列 W とベクトル b は、SimpleLinear モデルが学習するものです。出力は 10×1 行列で、最大値を持っているものが予測したい数値のインデックスです。アルゴリズムが 2つの変数（およびサイズ）について学習した内容を見てみましょう:

```python
for p in model.parameters():
    print(p.shape)
    print(p)
```

    torch.Size([10, 784])
    Parameter containing:
    tensor([[ 0.0062,  0.0153,  0.0184,  ..., -0.0116,  0.0088, -0.0087],
            [-0.0174,  0.0058,  0.0123,  ..., -0.0268, -0.0128,  0.0324],
            [ 0.0220, -0.0080, -0.0083,  ..., -0.0000, -0.0168,  0.0139],
            ...,
            [ 0.0100,  0.0282, -0.0219,  ..., -0.0071, -0.0025, -0.0157],
            [-0.0154,  0.0260,  0.0105,  ..., -0.0116, -0.0063,  0.0103],
            [-0.0010,  0.0121, -0.0262,  ...,  0.0111,  0.0119, -0.0262]],
           requires_grad=True)
    torch.Size([10])
    Parameter containing:
    tensor([-0.2752,  0.3982,  0.0538, -0.2269,  0.1070,  0.6785, -0.0616,  0.3549,
            -0.7796, -0.1151], requires_grad=True)

# 動作しているか？
正常に機能しているかどうかを判断する最良の方法は、学習プロセスで使用されていないデータでモデルをテストすることです。幸いなことに、そのようなデータセットがあります（これは既に使用したデータとは分けておいたものです）。以前と同じ方法でそれをすべてロードし、前のデータとは異なることを示すために表示してみます。

```python
test_digits = datasets.MNIST('data', train=False, download=True,
                        transform=transforms.Compose([
                            transforms.ToTensor(),
                            transforms.Lambda(lambda x: x.reshape(28*28))
                        ]),
                        target_transform=transforms.Compose([
                            transforms.Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, y, 1))
                        ])
                     )
draw_digits(test_digits)
```

最初の数値でテストしましょう（7 です）。

```python
x, y = test_digits[0]
x = x.to(device).view(1, 28*28)
```

評価に使用するモデル（推論とも呼ばれます）を指定して、これまでに使ったことのない数値を渡します。

```python
model.eval()
with torch.no_grad():
    pred = model(x)
    pred = pred.to('cpu').detach()[0]

print(pred)
```

    tensor([8.6886e-06, 1.3510e-10, 1.3847e-05, 1.6423e-03, 4.5304e-07, 1.5702e-05,
            2.6928e-09, 9.9798e-01, 1.6462e-05, 3.1789e-04])

予測された数字が実際の数字と一致するかどうか見てみましょう:

```python
pred.argmax(0), y.argmax(0)
```

    (tensor(7), tensor(7))

**すべて** のテストデータに対してどれだけうまくいっているか見てみましょう！

```python
# 推論用の数値のデータローダー
test_dataloader = DataLoader(test_digits, batch_size=64, num_workers=0, pin_memory=True)

# モデルを評価モードに設定
model.eval()
test_loss = 0
correct = 0

# ループ
with torch.no_grad():
    for batch, (X, Y) in enumerate(dataloader):
        X, Y = X.to(device), Y.to(device)
        pred = model(X)

        test_loss += cost(pred, Y).item()
        correct += (pred.argmax(1) == Y.argmax(1)).type(torch.float).sum().item()

test_loss /= len(dataloader.dataset)
correct /= len(dataloader.dataset)
print('\nTest Error:')
print('acc: {:>0.1f}%, avg loss: {:>8f}'.format(100*correct, test_loss))
```

    Test Error:
    acc: 92.6%, avg loss: 0.000701

# モデルの保存
今回は PyTorch を使っていますが、すべてのフレームワークは異なります。モデルを (PyTorch の）内部フォーマットと ONNXフォーマットの両方で保存します（これらは大きな行列Wとベクトルbであることを覚えています）。モデルは数字を認識する必要があるたびに、実行されるプログラムにアセットとしてロードすることができます。

```python
# グラフ全体を通すためのダミーの変数を作成
x = torch.randint(255, (1, 28*28), dtype=torch.float).to(device) / 255
onnx.export(model, x, 'model.onnx')
print('Saved onnx model to model.onnx')

# PyTorch モデルを保存
torch.save(model.state_dict(), 'model.pth')
print('Saved PyTorch Model to model.pth')
```

    Saved onnx model to model.onnx
    Saved PyTorch Model to model.pth

# 試してみましょう！
これでプロセス全体が完了したので、上に戻って試してみてください。次のように変更してみましょう:
1. 他のモデル（SimpleLinear を NeuralNetwork, CNN）を選択。他のモデルは、画像が最終的な答えを得るために通過する追加の行列（Wとb）を学習することを除けば、呼び出し方はほとんど同じです。
2. learning_rate、batch_size、epochなどのハイパーパラメーター。 変更で結果が良くなったり悪くなったりしますか？ 92％は問題ないですが、他のモデルとハイパーパラメータの組み合わせのほうがいいでしょうか。

## 最後に
フィードバックを歓迎します！これは役に立ちましたか？ 分かりにくい部分はありますか？簡単にでもいいので教えてください。


```python
from web3 import Web3
import pprint as pp
```

```python
w3 = Web3(Web3.HTTPProvider('http://gluteus.owodunni.home:8545'))
w3.isConnected()
```

    True

```python
from Arbie.Contracts.contract import Address, ContractFactory, Network
from Arbie.Contracts.uniswap import Factory, Pair

# https://uniswap.info/pair/0xbb2b8038a1640196fbe3e38816f3e67cba72d940
# WBTC-ETH Pair
pair_address = Address("0xBb2b8038a1640196FbE3e38816F3e67Cba72D940")

# https://uniswap.info/pair/0x94cdd18f53a8f3ec9a3ec0cbe897aed5ea009c43
# DAI-yDAI Pair
#pair_address = Address("0x94cDd18F53a8f3EC9A3Ec0CBE897aED5ea009c43")

#uni_factory = ContractFactory(w3, Factory).load_contract(
#    Address(""), network=Network.mainnet)

#pair_address = uni_factory.get_pair_address(3)
```

```python
pair = ContractFactory(w3, Pair).load_contract(Address(""), address=pair_address)

tokens = pair.get_tokens()
wbtc = tokens[0].create_token()
weth = tokens[1].create_token()

pair_amm = pair.create_amm()
pair_amm.spot_price(wbtc, weth)
```

```python
from Arbie.Contracts.balancer import Pool

pool_address = Address("0xB93F696E5C46b6772FC2ff65E459cf9685FAc782")

pool = ContractFactory(w3, Pool).load_contract(Address(""), address=pool_address)

pool_tokens = pool.get_tokens()

pool_amm = pool.create_amm()
pool_amm.spot_price(wbtc, weth)
```

```python
print(f"Pool:\n {pool_amm}")
print(f"Pair:\n {pair_amm}")
```

    Pool:
     Amm(
        Tokens: [Token(Name: Dai Stablecoin, Address: 0x6B175474E89094C44Da98b954EedeAC495271d0F), Token(Name: USD Coin, Address: 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48), Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), Token(Name: Synthetix Network Token, Address: 0xC011a73ee8576Fb46F5E1c5751cA3B9Fe0af2a6F), Token(Name: ChainLink Token, Address: 0x514910771AF9Ca656af840dff83E8264EcF986CA), Token(Name: Kyber Network Crystal, Address: 0xdd974D5C2e2928deA5F71b9825b8b646686BD200), Token(Name: EthLend Token, Address: 0x80fB784B7eD66730e8b1DBd9820aFD29931aab03)],
        Balances: [Variable(Token: Token(Name: Dai Stablecoin, Address: 0x6B175474E89094C44Da98b954EedeAC495271d0F), Value: 196.5368232523624), Variable(Token: Token(Name: USD Coin, Address: 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48), Value: 186.83305), Variable(Token: Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Value: 0.00866811), Variable(Token: Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), Value: 0.28568478018433124), Variable(Token: Token(Name: Synthetix Network Token, Address: 0xC011a73ee8576Fb46F5E1c5751cA3B9Fe0af2a6F), Value: 55.51107618156416), Variable(Token: Token(Name: ChainLink Token, Address: 0x514910771AF9Ca656af840dff83E8264EcF986CA), Value: 41.130332138141746), Variable(Token: Token(Name: Kyber Network Crystal, Address: 0xdd974D5C2e2928deA5F71b9825b8b646686BD200), Value: 1358.7032639964873), Variable(Token: Token(Name: EthLend Token, Address: 0x80fB784B7eD66730e8b1DBd9820aFD29931aab03), Value: 1266.8985699649068)],
        Weights: [Variable(Token: Token(Name: Dai Stablecoin, Address: 0x6B175474E89094C44Da98b954EedeAC495271d0F), Value: 0.06), Variable(Token: Token(Name: USD Coin, Address: 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48), Value: 0.06), Variable(Token: Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Value: 0.03), Variable(Token: Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), Value: 0.03), Variable(Token: Token(Name: Synthetix Network Token, Address: 0xC011a73ee8576Fb46F5E1c5751cA3B9Fe0af2a6F), Value: 0.08), Variable(Token: Token(Name: ChainLink Token, Address: 0x514910771AF9Ca656af840dff83E8264EcF986CA), Value: 0.12), Variable(Token: Token(Name: Kyber Network Crystal, Address: 0xdd974D5C2e2928deA5F71b9825b8b646686BD200), Value: 0.4), Variable(Token: Token(Name: EthLend Token, Address: 0x80fB784B7eD66730e8b1DBd9820aFD29931aab03), Value: 0.22)],
        Fee: 0.0097)
    Pair:
     Amm(
        Tokens: [Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2)],
        Balances: [Variable(Token: Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Value: 24855.04777308), Variable(Token: Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), Value: 755814.5673692599)],
        Weights: [Variable(Token: Token(Name: Wrapped BTC, Address: 0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599), Value: 0.5), Variable(Token: Token(Name: Wrapped Ether, Address: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), Value: 0.5)],
        Fee: 0.003)

```python
from sympy import *
from sympy.plotting import plot
init_printing()

# We can plot our return function
x = symbols('x')
expr = pool_amm.out_given_in_expr(wbtc, weth)
plot(expr, (x, 0, 200))
expr = pair_amm.out_given_in_expr(wbtc, weth)
plot(expr, (x, 0, 50000))
```

```python
# How can we found if there is an arbitrage opertunity between ?
from Arbie.Actions.arbitrage import arbitrage_expr, arbitrage_diff_expr, TradeOpertunity

trade = TradeOpertunity([pool_amm, pair_amm], wbtc, weth)
arb_expr = arbitrage_expr(trade)
arb_expr
```

```python
# If we plot we can clearly see that there seams to be some profit to be made!
plot(arb_expr, (x, 0, 0.0005))
```

```python
from Arbie.Actions.arbitrage import find_arbitrage
find_arbitrage(trade)
```

```python

```


```python
%matplotlib inline
```

Sequence Models and Long-Short Term Memory Networks
===================================================

At this point, we have seen various feed-forward networks. That is,
there is no state maintained by the network at all. This might not be
the behavior we want. Sequence models are central to NLP: they are
models where there is some sort of dependence through time between your
inputs. The classical example of a sequence model is the Hidden Markov
Model for part-of-speech tagging. Another example is the conditional
random field.

A recurrent neural network is a network that maintains some kind of
state. For example, its output could be used as part of the next input,
so that information can propogate along as the network passes over the
sequence. In the case of an LSTM, for each element in the sequence,
there is a corresponding *hidden state* $h_t$, which in principle
can contain information from arbitrary points earlier in the sequence.
We can use the hidden state to predict words in a language model,
part-of-speech tags, and a myriad of other things.

LSTM's in Pytorch
~~~~~~~~~~~~~~~~~

Before getting to the example, note a few things. Pytorch's LSTM expects
all of its inputs to be 3D tensors. The semantics of the axes of these
tensors is important. The first axis is the sequence itself, the second
indexes instances in the mini-batch, and the third indexes elements of
the input. We haven't discussed mini-batching, so lets just ignore that
and assume we will always have just 1 dimension on the second axis. If
we want to run the sequence model over the sentence "The cow jumped",
our input should look like

\begin{align}\begin{bmatrix}
   \overbrace{q_\text{The}}^\text{row vector} \\
   q_\text{cow} \\
   q_\text{jumped}
   \end{bmatrix}\end{align}

Except remember there is an additional 2nd dimension with size 1.

In addition, you could go through the sequence one at a time, in which
case the 1st axis will have size 1 also.

Let's see a quick example.

```python
# Author: Robert Guthrie

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

import config

torch.manual_seed(1)
```

    <torch._C.Generator at 0x7f12970c2db0>

```python
lstm = nn.LSTM(3, 3)  # Input dim is 3, output dim is 3
inputs = [torch.randn(1, 3) for _ in range(5)]  # make a sequence of length 5

print(inputs)

# initialize the hidden state.
hidden = (torch.randn(1, 1, 3),
          torch.randn(1, 1, 3))
print('init hidden:', hidden)

hidden2 = hidden

print('===========feed one by one')
for i in inputs:
    # Step through the sequence one element at a time.
    # after each step, hidden contains the hidden state.
    out, hidden = lstm(i.view(1, 1, -1), hidden)
    print('input:', i)
    print('hidden:', hidden)
    print('output:' , out)

print('===========feed all')
# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument  to the lstm at a later time
# Add the extra 2nd dimension
inputs2 = torch.cat(inputs).view(len(inputs), 1, -1)
print('inputs2:', inputs2)
# hidden = (torch.randn(1, 1, 3), torch.randn(1, 1, 3))  # clean out hidden state
print('init hidden2:', hidden2)
out2, hidden2 = lstm(inputs2, hidden2)
print('out2:', out2)
print('hidden2:', hidden2)
```

    [tensor([[-0.3280,  1.2700, -0.3375]]), tensor([[-1.0383,  1.1400, -0.4601]]), tensor([[-0.2561, -0.0529,  0.9539]]), tensor([[-2.0218,  0.0329, -1.4392]]), tensor([[ 0.0629,  0.4398, -0.1711]])]
    init hidden: (tensor([[[0.1788, 0.0076, 0.9767]]]), tensor([[[-0.9539, -1.5047,  0.6871]]]))
    ===========feed one by one
    input: tensor([[-0.3280,  1.2700, -0.3375]])
    hidden: (tensor([[[-0.0153, -0.4988, -0.1350]]], grad_fn=<ViewBackward>), tensor([[[-0.0283, -1.2673, -0.4140]]], grad_fn=<ViewBackward>))
    output: tensor([[[-0.0153, -0.4988, -0.1350]]], grad_fn=<CatBackward>)
    input: tensor([[-1.0383,  1.1400, -0.4601]])
    hidden: (tensor([[[ 0.2090, -0.4120, -0.2800]]], grad_fn=<ViewBackward>), tensor([[[ 0.4934, -1.3638, -0.6186]]], grad_fn=<ViewBackward>))
    output: tensor([[[ 0.2090, -0.4120, -0.2800]]], grad_fn=<CatBackward>)
    input: tensor([[-0.2561, -0.0529,  0.9539]])
    hidden: (tensor([[[ 0.1996, -0.2438, -0.2389]]], grad_fn=<ViewBackward>), tensor([[[ 0.6782, -0.6125, -0.3330]]], grad_fn=<ViewBackward>))
    output: tensor([[[ 0.1996, -0.2438, -0.2389]]], grad_fn=<CatBackward>)
    input: tensor([[-2.0218,  0.0329, -1.4392]])
    hidden: (tensor([[[ 0.2111, -0.5262, -0.1426]]], grad_fn=<ViewBackward>), tensor([[[ 0.3793, -1.0703, -0.2457]]], grad_fn=<ViewBackward>))
    output: tensor([[[ 0.2111, -0.5262, -0.1426]]], grad_fn=<CatBackward>)
    input: tensor([[ 0.0629,  0.4398, -0.1711]])
    hidden: (tensor([[[ 0.1680, -0.3853, -0.2647]]], grad_fn=<ViewBackward>), tensor([[[ 0.3685, -0.8883, -0.4577]]], grad_fn=<ViewBackward>))
    output: tensor([[[ 0.1680, -0.3853, -0.2647]]], grad_fn=<CatBackward>)
    ===========feed all
    inputs2: tensor([[[-0.3280,  1.2700, -0.3375]],

            [[-1.0383,  1.1400, -0.4601]],

            [[-0.2561, -0.0529,  0.9539]],

            [[-2.0218,  0.0329, -1.4392]],

            [[ 0.0629,  0.4398, -0.1711]]])
    init hidden2: (tensor([[[0.1788, 0.0076, 0.9767]]]), tensor([[[-0.9539, -1.5047,  0.6871]]]))
    out2: tensor([[[-0.0153, -0.4988, -0.1350]],

            [[ 0.2090, -0.4120, -0.2800]],

            [[ 0.1996, -0.2438, -0.2389]],

            [[ 0.2111, -0.5262, -0.1426]],

            [[ 0.1680, -0.3853, -0.2647]]], grad_fn=<CatBackward>)
    hidden2: (tensor([[[ 0.1680, -0.3853, -0.2647]]], grad_fn=<ViewBackward>), tensor([[[ 0.3685, -0.8883, -0.4577]]], grad_fn=<ViewBackward>))

Example: An LSTM for Part-of-Speech Tagging
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In this section, we will use an LSTM to get part of speech tags. We will
not use Viterbi or Forward-Backward or anything like that, but as a
(challenging) exercise to the reader, think about how Viterbi could be
used after you have seen what is going on.

The model is as follows: let our input sentence be
$w_1, \dots, w_M$, where $w_i \in V$, our vocab. Also, let
$T$ be our tag set, and $y_i$ the tag of word $w_i$.
Denote our prediction of the tag of word $w_i$ by
$\hat{y}_i$.

This is a structure prediction, model, where our output is a sequence
$\hat{y}_1, \dots, \hat{y}_M$, where $\hat{y}_i \in T$.

To do the prediction, pass an LSTM over the sentence. Denote the hidden
state at timestep $i$ as $h_i$. Also, assign each tag a
unique index (like how we had word\_to\_ix in the word embeddings
section). Then our prediction rule for $\hat{y}_i$ is

\begin{align}\hat{y}_i = \text{argmax}_j \  (\log \text{Softmax}(Ah_i + b))_j\end{align}

That is, take the log softmax of the affine map of the hidden state,
and the predicted tag is the tag that has the maximum value in this
vector. Note this implies immediately that the dimensionality of the
target space of $A$ is $|T|$.

Prepare data:

```python
def prepare_sequence(seq, to_ix):
    idxs = [to_ix[w] for w in seq]
    return torch.tensor(idxs, dtype=torch.long)

training_data = [
    ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
    ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
for sent, tags in training_data:
    for word in sent:
        if word not in word_to_ix:
            word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}

# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
EMBEDDING_DIM = 6
HIDDEN_DIM = 6
```

    {'The': 0, 'dog': 1, 'ate': 2, 'the': 3, 'apple': 4, 'Everybody': 5, 'read': 6, 'that': 7, 'book': 8}

Create the model:

```python
class LSTMTagger(nn.Module):

    def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
        super(LSTMTagger, self).__init__()
        self.hidden_dim = hidden_dim

        self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)

        # The LSTM takes word embeddings as inputs, and outputs hidden states
        # with dimensionality hidden_dim.
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)

        # The linear layer that maps from hidden state space to tag space
        self.hidden2tag = nn.Linear(hidden_dim, tagset_size)
        self.hidden = self.init_hidden()

    def init_hidden(self):
        # Before we've done anything, we dont have any hidden state.
        # Refer to the Pytorch documentation to see exactly
        # why they have this dimensionality.
        # The axes semantics are (num_layers, minibatch_size, hidden_dim)
        return (torch.zeros(1, 1, self.hidden_dim),
                torch.zeros(1, 1, self.hidden_dim))

    def forward(self, sentence):
        embeds = self.word_embeddings(sentence)
        lstm_out, self.hidden = self.lstm(
            embeds.view(len(sentence), 1, -1), self.hidden)
        tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
        tag_scores = F.log_softmax(tag_space, dim=1)
        return tag_scores
```

Train the model:

```python
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
# Here we don't need to train, so the code is wrapped in torch.no_grad()
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)
    print(tag_scores)

```

    tensor([[-1.4856, -0.6402, -1.4005],
            [-1.3564, -0.7680, -1.2784],
            [-1.3736, -0.6914, -1.4027],
            [-1.3263, -0.7143, -1.4065],
            [-1.3803, -0.6737, -1.4327]])

```python
for epoch in range(300):  # again, normally you would NOT do 300 epochs, it is toy data
    for sentence, tags in training_data:
        # Step 1. Remember that Pytorch accumulates gradients.
        # We need to clear them out before each instance
        model.zero_grad()

        # Also, we need to clear out the hidden state of the LSTM,
        # detaching it from its history on the last instance.
        model.hidden = model.init_hidden()

        # Step 2. Get our inputs ready for the network, that is, turn them into
        # Tensors of word indices.
        sentence_in = prepare_sequence(sentence, word_to_ix)
        targets = prepare_sequence(tags, tag_to_ix)

        # Step 3. Run our forward pass.
        tag_scores = model(sentence_in)

        # Step 4. Compute the loss, gradients, and update the parameters by
        #  calling optimizer.step()
        loss = loss_function(tag_scores, targets)
        loss.backward()
        optimizer.step()

```

```python
# See what the scores are after training
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)

    # The sentence is "the dog ate the apple".  i,j corresponds to score for tag j
    # for word i. The predicted tag is the maximum scoring tag.
    # Here, we can see the predicted sequence below is 0 1 2 0 1
    # since 0 is index of the maximum value of row 1,
    # 1 is the index of maximum value of row 2, etc.
    # Which is DET NOUN VERB DET NOUN, the correct sequence!
    print(tag_scores)
```

    tensor([[-0.0949, -3.3769, -2.8751],
            [-4.3791, -0.0155, -5.8517],
            [-2.8604, -5.5493, -0.0631],
            [-0.1404, -2.5039, -3.0111],
            [-3.0904, -0.0493, -5.9488]])

Exercise: Augmenting the LSTM part-of-speech tagger with character-level features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the example above, each word had an embedding, which served as the
inputs to our sequence model. Let's augment the word embeddings with a
representation derived from the characters of the word. We expect that
this should help significantly, since character-level information like
affixes have a large bearing on part-of-speech. For example, words with
the affix *-ly* are almost always tagged as adverbs in English.

To do this, let $c_w$ be the character-level representation of
word $w$. Let $x_w$ be the word embedding as before. Then
the input to our sequence model is the concatenation of $x_w$ and
$c_w$. So if $x_w$ has dimension 5, and $c_w$
dimension 3, then our LSTM should accept an input of dimension 8.

To get the character level representation, do an LSTM over the
characters of a word, and let $c_w$ be the final hidden state of
this LSTM. Hints:

* There are going to be two LSTM's in your new model.
  The original one that outputs POS tag scores, and the new one that
  outputs a character-level representation of each word.
* To do a sequence model over characters, you will have to embed characters.
  The character embeddings will be the input to the character LSTM.

```python

```


[Table of Contents](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb)
\appendix
# Installation, Python, NumPy, and FilterPy

```python
#format the book
%matplotlib inline
from __future__ import division, print_function
import matplotlib.pyplot as plt
import book_format
book_format.load_style()
```

<style>
@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');
@import url('http://fonts.googleapis.com/css?family=Vollkorn');
@import url('http://fonts.googleapis.com/css?family=Arimo');

    div.cell{
        width: 850px;
        margin-left: 0% !important;
        margin-right: auto;
    }
    div.text_cell code {
        background: transparent;
        color: #000000;
        font-weight: 600;
        font-size: 11pt;
        font-style: bold;
        font-family:  'Source Code Pro', Consolas, monocco, monospace;
   }
    h1 {
        font-family: 'Open sans',verdana,arial,sans-serif;
	}

    div.input_area {
        background: #F6F6F9;
        border: 1px solid #586e75;
    }

    .text_cell_render h1 {
        font-weight: 200;
        font-size: 30pt;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1em;
        display: block;
        white-space: wrap;
    }
    h2 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h2 {
        font-weight: 200;
        font-size: 20pt;
        font-style: italic;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1.5em;
        display: block;
        white-space: nowrap;
    }
    h3 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h3 {
        font-weight: 300;
        font-size: 18pt;
        line-height: 100%;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 2em;
        display: block;
        white-space: nowrap;
    }
    h4 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h4 {
        font-weight: 300;
        font-size: 16pt;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
        white-space: nowrap;
    }
    h5 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-style: normal;
        color: #1d3b84;
        font-size: 16pt;
        margin-bottom: 0em;
        margin-top: 1.5em;
        display: block;
        white-space: nowrap;
    }
    div.text_cell_render{
        font-family: 'Arimo',verdana,arial,sans-serif;
        line-height: 135%;
        font-size: 125%;
        width:750px;
        margin-left:auto;
        margin-right:auto;
        text-align:justify;
        text-justify:inter-word;
    }
    div.output_subarea.output_text.output_pyout {
        overflow-x: auto;
        overflow-y: scroll;
        max-height: 50000px;
    }
    div.output_subarea.output_stream.output_stdout.output_text {
        overflow-x: auto;
        overflow-y: scroll;
        max-height: 50000px;
    }
    code{
      font-size: 70%;
    }
    .rendered_html code{
    background-color: transparent;
    }
    ul{
        margin: 2em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li li{
        padding-left: 0.2em;
        margin-bottom: 0.2em;
        margin-top: 0.2em;
    }
    ol{
        margin: 2em;
    }
    ol li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.2em;
    }
    a:link{
       font-weight: bold;
       color:#447adb;
    }
    a:visited{
       font-weight: bold;
       color: #1d3b84;
    }
    a:hover{
       font-weight: bold;
       color: #1d3b84;
    }
    a:focus{
       font-weight: bold;
       color:#447adb;
    }
    a:active{
       font-weight: bold;
       color:#447adb;
    }
    .rendered_html :link {
       text-decoration: underline;
    }
    .rendered_html :hover {
       text-decoration: none;
    }
    .rendered_html :visited {
      text-decoration: none;
    }
    .rendered_html :focus {
      text-decoration: none;
    }
    .rendered_html :active {
      text-decoration: none;
    }
    .warning{
        color: rgb( 240, 20, 20 )
    }
    hr {
      color: #f3f3f3;
      background-color: #f3f3f3;
      height: 1px;
    }
    blockquote{
      display:block;
      background: #fcfcfc;
      border-left: 5px solid #c76c0c;
      font-family: 'Open sans',verdana,arial,sans-serif;
      width:680px;
      padding: 10px 10px 10px 10px;
      text-align:justify;
      text-justify:inter-word;
      }
      blockquote p {
        margin-bottom: 0;
        line-height: 125%;
        font-size: 100%;
      }
</style>

This book is written in IPython Notebook, a browser based interactive Python environment that mixes Python, text, and math. I choose it because of the interactive features - I found Kalman filtering nearly impossible to learn until I started working in an interactive environment. It is difficult to form an intuition of the effect of many of the parameters that you can tune until you can change them rapidly and immediately see the output. An interactive environment also allows you to play 'what if' scenarios out. "What if I set $\mathbf{Q}$ to zero?" It is trivial to find out with IPython Notebook.

Another reason I choose it is because I find that a typical textbook leaves many things opaque. For example, there might be a beautiful plot next to some pseudocode. That plot was produced by software, but software that is not available to me as a reader. I want everything that went into producing this book to be available to the reader. How do you plot a covariance ellipse? You won't know if you read most books. With IPython Notebook all you have to do is look at the source code.

Even if you choose to read the book online you will want Python and the SciPy stack installed so that you can write your own Kalman filters. There are many different ways to install these libraries, and I cannot cover them all, but I will cover a few typical scenarios.

## Installing the SciPy Stack

This book requires IPython, NumPy, SciPy, SymPy, and Matplotlib. The SciPy stack depends on third party Fortran and C code, and is *not* trivial to install from source code - even the SciPy website strongly urges using a pre-built installation, and I concur with this advice.

I use the Anaconda distribution from Continuum Analytics. This is an excellent package that combines all of the packages listed above, plus many others in one distribution. Installation is very straightforward, and it can be done alongside other Python installation you might already have on your machine. It is free to use, and Continuum Analytics will always ensure that the latest releases of all it's packages are available and built correctly for your OS. You may download it from here: http://continuum.io/downloads

I strongly recommend using the latest Python 3 version that they provide; I am developing in Python 3.x, and only sporadically test that everything works in Python 2.7. However, it is my long term goal to support 2.7, and if you wish to either not be bothered with 3.x and are willing to be occasionally frustrated with breaking check ins you may stick with Python 2.7.

I am still writing the book, so I do not know exactly what versions of each package is required. I do strongly urge you to use IPython 2.0 or later (this version number has nothing to do with Python 2 vs 3, by the way), as it provides many useful features which I will explain later.

There are other choices for installing the SciPy stack. The SciPy stack provides instructions here: http://scipy.org/install.html

## Installing FilterPy

FilterPy is a Python library that implements all of the filters used in this book, and quite a few not used by this book. Installation is easy using Pip. Issue the following command from the command prompt.

     pip install filterpy

FilterPy is written by me, and the latest development version is always available at github.com/rlabbe/filterpy.

### Manual Install of the SciPy stack

This really isn't easy, and again I advice you to follow the advice of the SciPy website and to use a prepackaged solution. Still, I will give you one example of an install from a fresh operating system. You will have to adapt the instructions to fit your OS and OS version. I will use xubuntu, a linux distribution, because that is what I am most familiar with. I know there are pre-built binaries for Windows (link provided on the SciPy installation web page), and I have no experience with Mac and Python.

I started by doing a fresh install of xubuntu 14.04 on a virtual machine. At that point there is a version of Python 2.7.6 and 3.4 pre-installed. As discussed above you may use either version; I will give the instructions for version 3.4 both because I prefer version 3 and because it can be slightly more tricky because 2.7 is the default Python that runs in xubuntu. Basically the difference is that you have to append a '3' at the end of commands. `python3` instead of `python`, `pip3` instead of `pip`, and so on.

First we will install pip with the following command:

    sudo apt-get install python3-pip

Now you will need to install the various packages from the Ubuntu repositories. Unfortunately they usually do not contain the latest version, so we will also install the development tools necessary to perform an upgrade, which requires compiling various modules.

    sudo apt-get install python3-numpy python3-scipy python3-matplotlib
    sudo apt-get install libblas-dev liblapack-dev gfortran python3-dev

Now you can upgrade the packages. This will take a long time as everything needs to be compiled from source. If you get an error I do not know how to help you!

    sudo pip3 install numpy --upgrade
    sudo pip3 install scipy --upgrade

Now you get to install SymPy. You can download it from github (replace version number with the most recent version):

    wget https://github.com/sympy/sympy/releases/download/sympy-0.7.6/sympy-0.7.6.tar.gz
    tar -zxvf sympy-0.7.6.tar.gz

Now go into the directory you just extracted and run setup.

    sudo python3 setup.py install

If all of this went without a hitch you should have a good install. Try the following. From the command line type `ipython3` to launch ipython, then issue the following commands. Each should run with no exceptions.

    import numpy as np
    import scipy as sp
    import sympy as sym
    np.__version__
    sp.__version__
    sum.__version__

Now let's make sure plotting works correctly.

    import matplotlib.pyplot as plt
    plt.plot([1, 4, 3])
    plt.show()

Now you get to fix IPython so ipython notebook will run. First, I had to uninstall IPython with

    sudo pip3 uninstall ipython

Then, I reinstalled it with the `[all]` option so that all the required dependencies are installed.

    sudo pip3 install "ipython3[all]"

Now test the installation by typing

    ipython notebook

If successful, it should launch a browser showing you the IPython home page.

That was not fun. It actually goes somewhat smoother in Windows, where you can download pre-built binaries for these packages; however, you are dependent on this being done for you in a timely manner. So, please follow the SciPy advice and use a pre-built distribution! I will not be supporting this manual install process.

## Installing/downloading and running the book

Okay, so now you have the SciPy stack installed, how do you download the book? This is easy as the book is in a github repository. From the command line type the following:

    git clone https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python.git

Alternatively, browse to https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python and download it via your browser.

Now, from the command prompt change to the directory that was just created, and then run ipython notebook:

    cd Kalman-and-Bayesian-Filters-in-Python
    ipython notebook

A browser window should launch showing you all of the chapters in the book. Browse to the first chapter by clicking on it, then open the notebook in that subdirectory by clicking on the link.

## Using IPython Notebook

A complete tutorial on IPython Notebook is beyond the scope of this book. Many are available online. In short, Python code is placed in cells. These are prefaced with text like `In [1]:`, and the code itself is in a boxed area. If you press CTRL-ENTER while focus is inside the box the code will run and the results will be displayed below the box. Like this:

```python
print(3+7.2)
```

    10.2

If you have this open in ipython notebook now, go ahead and modify that code by changing the expression inside the print statement and pressing CTRL+ENTER. The output should be changed to reflect what you typed in the code cell.

## SymPy

SymPy is a Python package for performing symbolic mathematics. The full scope of its abilities are beyond this book, but it can perform algebra, integrate and differentiate equations, find solutions to differential equations, and much more. For example, we use use to to compute the Jacobian of matrices and the expected value integral computations.

First, a simple example. We will import SymPy, initialize its pretty print functionality (which will print equations using LaTeX). We will then declare a symbol for Numpy to use.

```python
import sympy
sympy.init_printing(use_latex='mathjax')

phi, x = sympy.symbols('\phi, x')
phi
```

$$\phi$$

Notice how we use a latex expression for the symbol `phi`. This is not necessary, but if you do it will render as LaTeX when output. Now let's do some math. What is the derivative of $\sqrt{\phi}$?

```python
sympy.diff('sqrt(phi)')
```

$$\frac{1}{2 \sqrt{\phi}}$$

We can factor equations

```python
sympy.factor(phi**3 -phi**2 + phi - 1)
```

$$\left(\phi - 1\right) \left(\phi^{2} + 1\right)$$

and we can expand them.

```python
((phi+1)*(phi-4)).expand()
```

$$\phi^{2} - 3 \phi - 4$$

You can find the value of an equation by substituting in a value for a variable

```python
w =x**2 -3*x +4
w.subs(x,4)
```

$$8$$

You can also use strings for equations that use symbols that you have not defined

```python
x = sympy.expand('(t+1)*2')
x
```

$$2 t + 2$$

Now let's use SymPy to compute the Jacobian of a matrix. Let us have a function

$$h=\sqrt{(x^2 + z^2)}$$

for which we want to find the Jacobian with respect to x, y, and z.

```python
x, y, z = sympy.symbols('x y z')

H = sympy.Matrix([sympy.sqrt(x**2 + z**2)])

state = sympy.Matrix([x, y, z])
H.jacobian(state)
```

$$\left[\begin{matrix}\frac{x}{\sqrt{x^{2} + z^{2}}} & 0 & \frac{z}{\sqrt{x^{2} + z^{2}}}\end{matrix}\right]$$

Now let's compute the discrete process noise matrix $\mathbf{Q}_k$ given the continuous process noise matrix
$$\mathbf{Q} = \Phi_s \begin{bmatrix}0&0&0\\0&0&0\\0&0&1\end{bmatrix}$$

and the equation

$$\mathbf{Q} = \int_0^{\Delta t} \Phi(t)\mathbf{Q}\Phi^T(t) dt$$

where
$$\Phi(t) = \begin{bmatrix}1 & \Delta t & {\Delta t}^2/2 \\ 0 & 1 & \Delta t\\ 0& 0& 1\end{bmatrix}$$

```python
dt = sympy.symbols('\Delta{t}')
F_k = sympy.Matrix([[1, dt, dt**2/2],
                    [0,  1,      dt],
                    [0,  0,       1]])
Q = sympy.Matrix([[0,0,0],
                  [0,0,0],
                  [0,0,1]])

sympy.integrate(F_k*Q*F_k.T,(dt, 0, dt))
```

$$\left[\begin{matrix}\frac{\Delta{t}^{5}}{20} & \frac{\Delta{t}^{4}}{8} & \frac{\Delta{t}^{3}}{6}\\\frac{\Delta{t}^{4}}{8} & \frac{\Delta{t}^{3}}{3} & \frac{\Delta{t}^{2}}{2}\\\frac{\Delta{t}^{3}}{6} & \frac{\Delta{t}^{2}}{2} & \Delta{t}\end{matrix}\right]$$