# Extended Kalman Filter

EKF(Extended Kalman Filter、拡張カルマンフィルタ）はガウシアンフィルタの一つです。

位置推定を行う際にも用いられます。

本ページでは、2次元の平面上でロボットがランドマークを観測しながら拡張カルマンフィルタで自己位置を推定します。

注がある場合を除き、本ページに掲載されているコードは[MIT](https://tiryoh.mit-license.org/)ライセンスに、文章は[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.ja)ライセンスに基づいて公開します。

## Requirements

Python 3を必要とします。

## References

S. Thrun, W. Burgard and D. Fox (著), 上田 (訳): 確率ロボティクス, 毎日コミュニケーションズ, 2007

https://github.com/ryuichiueda/probrobo_practice/blob/master/monte_carlo_localization/1.monte_calro_localization.ipynb

このJupyter Notebookではロボットの移動をシミュレーションしながら、位置推定を行っていきます。
移動のシミュレーションにはJupyter Notebookでグラフを描画するためのコードが読み書きできる必要があります。
Jupyter Notebookを使ってグラフ描画機能でアニメーションを表示する手法についてまとめたJupyter Notebookは[こちら](https://github.com/Tiryoh/probrobo_note/blob/master/introduction/animate_robot_motion.ipynb)です。

# カルマンフィルタ

カルマンフィルタはガウシアンフィルタの一つです。
ガウシアンフィルタはノイズがガウス分布に従うと仮定して、
事後信念をガウス分布として計算できるのが特徴です。

状態空間モデルにおいて、
観測値、観測方程式、状態方程式を利用して、
観測することのできない状態値を推定するための計算手法です。
予測ステップと修正ステップに分かれています。

* 予測ステップ
  * 一つ前の時刻の事後推定値から、システムのモデルを元に事前推定値を算出します。
* 修正ステップ
  * 予測ステップで得られた事前推定値を、観測情報を元に修正を加え、事後推定値を算出します。

それぞれの記号を次のように定義します。

* 状態ベクトル(状態遷移確率) : ${x}_t$
* 観測ベクトル(計測確率) : ${z}_t$
* 制御入力ベクトル : ${u}_t$
* システム雑音ベクトル : ${\epsilon}_t$
  * 平均ベクトル : $0$
  * 共分散行列 : ${R}_t$
* 観測雑音ベクトル : ${\delta}_t$
  * 平均ベクトル : $0$
  * 共分散行列 : ${Q}_t$

このとき、線形カルマンフィルタで扱うことのできる線形モデルの場合、
${A}$を状態遷移行列、
${B}$を制御行列、
${C}$を観測行列とすると、
状態空間モデルは

\begin{align}
    {x}_t &=& {A}_t {x}_{t-1} +
    {B}_t {u}_t +  {\epsilon}_t \\
    {z}_t &=& {C}_t {x}_t + {\delta}_t
\end{align}

と表すことができます。

実際にはロボットの状態は線形ではなく、
非線形なモデルとなります。
ロボットの状態遷移確率は$g$に、計測確率は$h$に（$g,h$はともに非線形関数）従うと仮定します。
このとき、状態空間モデルは

\begin{align}
    {x}_t &=& g({u}_t, {x}_{t-1})+{\epsilon}_t \\
    {z}_t &=& h({x}_t ) + {\delta}_t
\end{align}

と表すことができます。

非線形関数は直接共分散行列を計算することができませんが、
関数を線形近似することで、カルマンフィルタの原理を適用することは可能になります。
具体的には、線形カルマンフィルタにおいて
${A}_t, {B}_t$
と計算していた行列をEKFではヤコビ行列の
${G}_t$
として、
${C}_t$
と計算していた行列をEKFではヤコビ行列の
${H}_t$
として計算します。
ガウス分布では事後信念の平均値$\mu_{t-1}$が最尤な値となるので、
ヤコビ行列は

\begin{align}
    {G}_t &:=& g'({u}_t,{\mu}_{t-1}) &:=& \frac{\partial g({u}_t,{\mu}_{t-1})}{\partial {x}_{t-1}} \\
    {H}_t &:=& h'({\mu}_t) &:=& \frac{\partial h({\mu}_t)}{\partial {x}_{t}}
\end{align}

として定義します。
状態空間モデルを元に、予測ステップと修正ステップを行います。

$g,h$ともに非線形であり、共分散行列を直接求めることができません。
テイラー展開によって線形近似することで、線形カルマンフィルタと同様に予測ステップと修正ステップを繰り返し、
システムを推定・制御することが可能になります。

# モジュールのインポート

Jupyter Notebookに必要な各種モジュールをインポートします。

```python
# 数値計算及び描画に必要なモジュール
import numpy as np
import math, random
from copy import copy
from matplotlib import pyplot as plt
from matplotlib.patches import Ellipse

# アニメーション作成・表示に必要なモジュール
from matplotlib import animation
from IPython.display import HTML
```

# ランドマークのクラス

ロボットにとって目印となるランドマークを宣言します。
ロボットはこのランドマークの座標をあらかじめ知っています。

```python
class Landmarks:
    """
    landmarks = Landmarks([[x1, y1], [x2, y2], ... , [x, y]])
    引数にランドマークのX座標とY座標を配列で指定
    """
    def __init__(self, array):
        """
        ランドマークの座標を配列で取得
        あとで計算処理しやすくするため、numpyのarrayに変換
        """
        self.positions = np.array(array)

    def draw(self):
        """
        取得したすべてのランドマークの座標をプロット
        """
        xs = [ e[0] for e in self.positions]
        ys = [ e[1] for e in self.positions]
        plt.scatter(xs, ys, s=300, marker='*', label='landmarks', color='orange')
```

# ロボットのクラス

ロボットを操作するためのクラスと、ロボットの描画のためのクラスに分かれています。
今回は説明用のグラフ描画のためのメソッド（`draw_graph`）もこれらのクラスで取り扱っています。

状態空間モデルの状態遷移確率 : $ {x}_t = g({u}_t, {x}_{t-1})+{\epsilon}_t $

## ロボットを操作するためのクラス
実際のロボットは自身の絶対座標を知るすべはありません。そのため、ロボットを操作するためのクラスではロボットの座標を把握していません。

ロボットを操作するためのクラスに与えることができるのは制御入力ベクトルの${u}_t$です。`move`メソッドに`distance`を引数として与えます。

## ロボットの描画のためのクラス
ロボットの描画のためのクラスでは、実際の位置を描画するためにロボットの座標や実際の移動距離を把握しています。

ロボットの描画のためのクラスでは実際の移動をシミュレーションします。`motion`メソッドが $x_t$ そのものです。

```python
class Robot:
    """
    robot = Robot(0,0,0)
    ロボットの動作を決めるためのクラス
    ロボットは現在の自身の座標を把握できません。
    そのため、描画のための座標は別のクラスで管理します。
    """
    def __init__(self, x = 0, y = 0, rad = 0):
        self.motion = RobotMotion(x, y, rad)
        self.ekf = EKF(np.array([x, y, rad]))

    def move(self, distance, direction):
        self.motion.update_pos(distance, direction)
        # 制御入力(distance)に基づき、EKFの予測ステップを適用
        self.ekf.prediction_step(distance, direction)

    def observation(self, landmakrs):
        # 全てのランドマークについてロボットの現在地から観測できるか確認
        obss = self.motion.observation(landmarks)
        # 観測結果を使いながらEKFの修正ステップを適用
        self.ekf.update_step(obss, landmarks)
        return obss

    def draw(self, subplot, obss):
        self.motion.draw(subplot, obss)
        self.ekf.draw(subplot)

class RobotMotion:
    """
    ロボットの描画のためのクラス
    ロボットが実際にどのように動くかはこのクラスの中で決まっています。
    ロボットを操作するのはこのクラスではありません。
    """
    def __init__(self, x, y, theta):
        """
        ロボットの座標、角度を配列で取得、保存
        あとで計算処理しやすくするため、numpyのarrayに変換
        """
        self.actual_poses = [np.array([x, y, theta])]

    def draw(self, subplot, obss, color = 'red',label = 'actual robot motion'):
        """
        ロボットの座標、角度情報を元にベクトルを描画
        """
        xs = [e[0] for e in self.actual_poses]
        ys = [e[1] for e in self.actual_poses]
        vxs = [math.cos(e[2]) for e in self.actual_poses]
        vys = [math.sin(e[2]) for e in self.actual_poses]
        sub_color = color
        sub_label = label
        subplot.quiver(xs, ys, vxs, vys, color = sub_color, label = sub_label)

        for obs in obss:
            obs[0].draw(subplot, self.actual_poses[-1])

    def motion(self, robot_pos, distance, direction):
        """
        ロボットを移動
        進む距離、進む角度を元に移動する先の座標をnumpyのarray形式で返す
        """
        # 移動距離に10%の標準偏差
        actual_distance = random.gauss(distance, distance * 0.1)
        # 進む角度に10%の標準偏差
        actual_direction = random.gauss(direction, direction * 0.1)
        # 移動方向に対して3[deg]の標準偏差
        dir_error = random.gauss(0.0, np.radians(3))
        px, py, pth = robot_pos
        x = px + actual_distance * math.cos(pth)
        y = py + actual_distance * math.sin(pth)
        th = pth + actual_direction + dir_error
        return np.array([x, y, th])

    def update_pos(self, distance, direction):
        """
        ロボットを移動し、位置情報を配列に追加
        進む距離、進む角度を元に位置情報を更新
        """
        self.actual_poses.append(self.motion(self.actual_poses[-1], distance, direction))

    def observation(self, landmarks):
        obss = []
        # すべてのランドマークについて、観測できるか確認
        for i,landmark in enumerate(landmarks.positions):
            # ランドマークの観測結果を保存
            obs = Observation(landmark)
            obs.measure(self.actual_poses[-1])
            # センサの範囲内だったランドマークをリストに追加
            if obs.distance != None:
                obss.append([obs, i])

        # 観測結果をリストで返す
        return obss
```

# 観測のためのクラス

ランドマークを観測するためのクラスです。
ロボットの現在地からセンサの観測範囲内にランドマークがあるかどうかをチェックします。

状態空間モデルの計測確率: $ {z}_t = h({x}_t ) + {\delta}_t $

`measure`メソッドの中の`actual_distance`が $z_t$ に、`distance`が $h({x}_t )$ に該当します。

```python
class Observation:
    """
    ロボットの観測モデル
    """
    def __init__(self, landmark_pos, max_range = 1.0, min_range = 0.1, max_angle = (math.pi / 2), min_angle = (-math.pi / 2)):
        # ランドマークの位置を保存
        self.landmark_x, self.landmark_y = landmark_pos
        # センサの計測できる範囲を設定
        self.sensor_max_range = 1.5
        self.sensor_min_range = 0.05
        self.sensor_max_angle = np.deg2rad(90)
        self.sensor_min_angle = np.deg2rad(-90)

        # 観測できない場合のセンサ値を決めておく（ここではNone）
        # 観測できた場合はこの値を上書きする
        self.distance, self.direction, self.error_ellipse = None, None, None

        # センサの値を取得してセンサの観測範囲内に対象物があるかチェックする
        # 対象物があれば、&IDを保存
        #self.distance, self.direction, self.error_ellipse = self.measure(robot_pos, landmark)
        #self.lid = lid if (self.distance != None) else None

    def measure(self, robot_pos):
        """
        センサの値を取得してセンサの観測範囲内に対象物があるかチェックする
        """
        # ロボットの位置(ロボット自身は観測不可能)を一時保存
        robot_x, robot_y, robot_th = robot_pos

        # センサの計測距離範囲内かどうか確認する
        self.distance_x = self.landmark_x - robot_x
        self.distance_y = self.landmark_y - robot_y
        distance = math.sqrt(self.distance_x ** 2 + self.distance_y ** 2)
        if not distance < self.sensor_max_range and distance > self.sensor_min_range:
            #print(str(distance) + " out of range(distance)")
            return# 計測範囲外だった場合はNoneのまま終了

        # センサの計測角度範囲内かどうか確認する
        direction = math.atan2((self.landmark_y - robot_y),(self.landmark_x - robot_x)) - robot_th
        if direction > math.pi:
            tmp_direction = direction - math.pi * 2
        elif direction < -math.pi:
            tmp_direction = direction + math.pi * 2
        else:
            tmp_direction = direction
        if tmp_direction > self.sensor_max_angle or tmp_direction < self.sensor_min_angle:
            #print(str(np.rad2deg(direction)) + " out of range(angle)")
            return # 計測範囲外だった場合はNoneのまま終了

        # 真値に混入する雑音の大きさを設定
        sigma_distance = distance * 0.1 # 標準偏差 : 距離に対して10%
        sigma_direction = np.radians(3) # 標準偏差 : ランドマークの方向に対して3deg

        # 雑音を混ぜてセンサの値とする
        actual_distance = random.gauss(distance, sigma_distance)
        actual_direction = random.gauss(direction, sigma_direction)

        # ロボット座標系での共分散行列を作っておく。あとで尤度を計算するときに使用
        # x方向が奥行きで、sigma_distanceを標準偏差に設定。y方向がロボットから見て横方向の誤差で、距離*sin(3[deg])となる。
        error_ellipse = Gaussian2D(sigma_x = sigma_distance, sigma_y = actual_distance * math.sin(sigma_direction) , cov_xy = 0.0)

        #return actual_distance, actual_direction, error_ellipse
        self.distance, self.direction, self.error_ellipse = actual_distance, actual_direction, error_ellipse

    def draw(self,subplot,robot_pos):
        # ロボットの姿勢と、このインスタンスに保存されているセンサの値から、ランドマークの位置を求める
        rx, ry, rt = robot_pos[0], robot_pos[1], robot_pos[2]
        proposed_lx = rx + self.distance * math.cos(rt + self.direction)
        proposed_ly = ry + self.distance * math.sin(rt + self.direction)

        # ロボットからランドマークまでの直線を描画
        subplot.plot([rx, proposed_lx], [ry, proposed_ly], color="pink")

        # 以下、「観測したランドマークがどの位置に見えているか」を示す誤差楕円を描画するための処理

        # このインスタンスに保存されている共分散行列を、計算されたランドマークの位置に移動
        # 移動後、ロボットの向きに合わせて共分散行列を回転
        e = copy(self.error_ellipse)
        e.shift(np.array([proposed_lx, proposed_ly]).T, rt + self.direction)

        eig_vals, eig_vec= np.linalg.eig(e.cov)

        v1 = eig_vals[0] * eig_vec[:,0]
        v2 = eig_vals[1] * eig_vec[:,1]
        v1_direction = math.atan2(v1[1],v1[0])

        elli = Ellipse([proposed_lx, proposed_ly],width=math.sqrt(np.linalg.norm(v1)),height=math.sqrt(np.linalg.norm(v2)),angle=np.rad2deg(v1_direction))
        elli.set_alpha(0.2)

        # 誤差楕円を描画
        subplot.add_artist(elli)
```

# 2次元ガウス分布のためのクラス

```python
class Gaussian2D:
    """
    参照 : https://github.com/ryuichiueda/probrobo2017/blob/master/13.ipynb
    描画の際の計算や尤度の計算に使用する2次元ガウス分布
    mu_x : 中心のX座標（以下X）
    mu_y : 中心のY座標（以下Y）
    sigma_x : Xの分散
    sigma_y : Yの分散
    cov_xy : XYの共分散
    """
    def __init__(self, sigma_x = 1.0, sigma_y = 1.0, cov_xy = 0.0, mu_x = 0.0, mu_y = 0.0):
        """
        共分散行列と中心の座標を定義
        """
        self.cov = np.array([[sigma_x ** 2, cov_xy],
                             [cov_xy, sigma_y ** 2]])
        self.mean = np.array([mu_x, mu_y])

    def shift(self, delta, angle):
        """
        ガウス分布を移動するためのメソッド
        """
        # 2次元回転行列（反時計回りが正回転）
        rot2d = np.array([[math.cos(angle), -math.sin(angle)],
                          [math.sin(angle), math.cos(angle)]])
        # 共分散行列は正方行列でベクトルではないので注意
        self.cov = rot2d.dot(self.cov).dot(rot2d.T)
        self.mean = self.mean + delta

    def value(self, pos):
        """
        密度の算出
        """
        delta = pos - self.mean
        numerator = math.exp(-0.5 * (delta.T).dot(np.linalg.inv(self.cov)).dot(delta))
        denominator = 2 * math.pi * math.sqrt(np.linalg.det(self.cov))
        return numerator / denominator
```

# カルマンフィルタのクラス

本ページでは直線上を移動するロボットの自己位置推定を行っています。
直線上の移動の場合、非線形関数を線形近似する段階がありません。一般的なカルマンフィルタと同じ計算をしています。

## 予測ステップ
予測ステップでは、1つ前の時間($t-1$)の状態を元に、
事前推定値と事前誤差共分散を求めます。
1つ前の時間($t-1$)の
事後推定値を${{\mu}}_{t-1}$
、
事後誤差共分散を${{\Sigma}_{{\bar{x}}_{t-1}}}$
と表します。
時刻$t$における
事前推定値を$\bar{{\mu}}_t$、
事前誤差共分散を${{\Sigma}_{\bar{{x}}_t}}$
とすると、

カルマンフィルタの場合は、

\begin{align}
    \bar{{\mu}}_t
    &=& {A}_t {{\mu}}_{t-1} + {B}_t {u}_{t} \\
    {{\Sigma}_{\bar{{x}}_t}}
    &=& {A}_t {{\Sigma}_{{{x}}_{t-1}}} {{A}_t}^T+ {R}_t
\end{align}

と表すことができます。

拡張カルマンフィルタの場合は、

\begin{align}
    \bar{{\mu}}_t
    &=& g({u}_t, {\mu}_{t-1}) \\
    {{\Sigma}_{\bar{{x}}_t}}
    &=& {G}_t {{\Sigma}_{{{x}}_{t-1}}} {{G}_t}^T+ {R}_t
\end{align}

と表すことができます。
${G}_t$は、非線形関数$g$を線形近似して状態遷移確率${x}_t$を求める際に用いるヤコビ行列です。

## 修正ステップ

修正ステップでは、予測ステップを元に、カルマンゲイン、事後推定値、事後誤差共分散を求めます。
カルマンゲインを${K}_t$、
事後推定値を${\mu}_t$、
事後誤差共分散を${\Sigma}_{{x}_t}$
とすると、

カルマンフィルタの場合、

\begin{align}
    {K}_t
    &=& {{\Sigma}_{\bar{{x}}_t}}{{C}_t}^T({C}_t {{\Sigma}_{\bar{{x}}_t}} {{C}_t}^T + {{Q}_t})^{-1} \\
    {\mu}_t
    &=& \bar{{\mu}}_t + {K}_t ({z}_t - {{C}_t} \bar{{\mu}}_t) \\
    {{\Sigma}_{{{x}}_t}}
    &=& (1 - {K}_t {C}_t) {{\Sigma}_{\bar{{x}}_t}}
\end{align}

と表すことができます。

拡張カルマンフィルタの場合、

\begin{align}
    {K}_t
    &=& {{\Sigma}_{\bar{{x}}_t}}{{H}_t}^T
    ({H}_t {{\Sigma}_{\bar{{x}}_t}} {{H}_t}^T
    + {{Q}_t})^{-1} \\
    {\mu}_t
    &=& \bar{{\mu}}_t + {K}_t ({z}_t - h( \bar{{\mu}}_t)) \\
    {{\Sigma}_{{{x}}_t}}
    &=& (1 - {K}_t {H}_t)
    {{\Sigma}_{{\bar{x}}_t}}
\end{align}

と表すことができます。
${H}_t$ は、ロボットの計測確率 ${z}_t$ を求める非線形関数$h$のヤコビ行列です。

ここで求められた
事後推定値${\mu}_t$
と
事後誤差共分散${{\Sigma}_{{{x}}_t}}$
が、時刻$t$におけるカルマンフィルタ適用後の
事後信念です。

$K$と`kalman_gain` 、$\mu_t$ と `posteriori_bel_x` 、${\Sigma}_{{{x}}_t}$ と`posteriori_cov_x`がそれぞれ対応しています。

カルマンゲインの
${{H}_t {{\Sigma}_{{\bar{x}}_t}} {{H}_t}^T + {{Q}_t}}$
は計測予測の不確かさを表しています。
計測がより確かであれば、不確かさは小さくなり、
カルマンゲインは大きくます。
今回の環境においては、
ロボットがランドマークを観測できると、
計測の不確かさが小さくます。
EKFの修正ステップにおける修正幅が大きくなり、
事後推定値がより真のロボットの位置姿勢に近づいていきます。

```python
class EKF:
    """
    EKFのクラス
    """
    def __init__(self, robot_pos):
        # 推定値の初期値
        self.bel_pos = robot_pos # x, y, rad
        # 誤差共分散
        self.cov_t = np.eye(3)
        # 更新間隔（t-1からtまでの間隔のこと）
        # ここでは計算を簡略化するために1.0秒とした
        self.delta_t = 1.0
        # 誤差パラメータ
        # 今回は全て10[%]と仮定（センサ固有のもの）
        self.alpha = np.array([0.1, 0.1, 0.1, 0.1])

        # debug = TrueにするとEKFの計算の途中過程が表示される
        self.debug = False
        if self.debug : print("initial robot_pos", self.bel_pos)

    def prediction_step(self, distance, direction):
        """
        予測ステップ
        """
        if self.debug : print("bel_pos(t-1)", self.bel_pos)

        # 移動速度
        vel_t = distance / self.delta_t
        # 移動角速度
        omega_t = direction / self.delta_t

        # 状態遷移確率を求める際に用いるヤコビ行列（事前誤差共分散の計算に使用する）
        G_t = np.array([[1, 0, distance/direction*(-math.cos(self.bel_pos[2])+math.cos(self.bel_pos[2]+direction))],
                        [0, 1, distance/direction*(-math.sin(self.bel_pos[2])+math.sin(self.bel_pos[2]+direction))],
                        [0, 0, 1]])
        if self.debug : print("G_t",G_t)
        # 制御空間における雑音を状態空間へ変換する際、線形近似する。線形近似する際に用いるヤコビ行列（事前誤差共分散の計算に使用する）
        V_t = np.array([[(-math.sin(self.bel_pos[2])+math.sin(self.bel_pos[2]+direction))/omega_t, vel_t*(math.sin(self.bel_pos[2])-math.sin(self.bel_pos[2]+direction))/omega_t**2 + vel_t*math.cos(self.bel_pos[2]+direction)*self.delta_t/omega_t],
                        [(math.cos(self.bel_pos[2])-math.cos(self.bel_pos[2]+direction))/omega_t, -vel_t*(math.cos(self.bel_pos[2])-math.cos(self.bel_pos[2]+direction))/omega_t**2 + vel_t*math.sin(self.bel_pos[2]+direction)*self.delta_t/omega_t],
                        [0, self.delta_t]])
        if self.debug : print("V_t", V_t)
        # 制御空間における雑音の共分散（事前誤差共分散の計算に使用する）
        M_t = np.array([[self.alpha[0]*vel_t**2 + self.alpha[1]*omega_t**2, 0],
                        [0, self.alpha[2]*vel_t**2 + self.alpha[3]*omega_t**2]])
        if self.debug : print("M_t", M_t)

        # 事前推定値
        self.bel_pos = self.bel_pos + np.array([-math.sin(self.bel_pos[2])+math.sin(self.bel_pos[2]+direction),
                                                math.cos(self.bel_pos[2])-math.cos(self.bel_pos[2]+direction),
                                                direction])
        if self.debug : print("priori bel_pos(t)", self.bel_pos)
        # 事前誤差共分散
        self.cov_t = G_t.dot(self.cov_t).dot(G_t.T) + V_t.dot(M_t).dot(V_t.T)
        if self.debug : print("cov(t)", self.cov_t)

    def update_step(self, obss, landmarks):
        """
        修正ステップ
        """

        if self.debug : print("landmarks", landmarks)

        # 計測雑音の共分散行列
        Q_t = np.array([[0.01, 0, 0],
                        [0, 0.01, 0],
                        [0, 0, 0.01]])
        if self.debug : print("Q_t", Q_t)

        # 全ての観測結果を使用して推定値を更新
        for obs in obss:

            if self.debug : print("landmarks.positions[obs[1]]",landmarks.positions[obs[1]])

            # 事前推定値からランドマークまでの距離の二乗（観測値を予測する際に使用する）
            q = (landmarks.positions[obs[1]][0] - self.bel_pos[0]) ** 2 + (landmarks.positions[obs[1]][1] - self.bel_pos[1]) ** 2
            if self.debug : print("q",q)

            # 予測される観測値
            z_hat_t = np.array([math.sqrt(q),
                                math.atan2((landmarks.positions[obs[1]][1] - self.bel_pos[1]), (landmarks.positions[obs[1]][0] - self.bel_pos[0])) - self.bel_pos[2],
                                obs[1]])
            if self.debug : print("z_hat_t", z_hat_t)

            # 実際の観測値
            z_t = np.array([obs[0].distance,
                            obs[0].direction,
                            obs[1]])
            if self.debug : print("z_t", z_t)

            # ロボットの位置に対するhのヤコビ行列（事後誤差共分散の計算に使用する）
            H_t = np.array([[-(landmarks.positions[obs[1]][0] - self.bel_pos[0])/math.sqrt(q), -(landmarks.positions[obs[1]][1] - self.bel_pos[1])/math.sqrt(q), 0],
                            [(landmarks.positions[obs[1]][1] - self.bel_pos[1])/math.sqrt(q), -(landmarks.positions[obs[1]][0] - self.bel_pos[0])/math.sqrt(q), -1],
                            [0, 0, 0]])
            if self.debug : print("H_t", H_t)

            # カルマンゲインの計算に使用
            S_t = H_t.dot(self.cov_t).dot(H_t.T) + Q_t
            if self.debug : print("S_t", S_t)

            # カルマンゲイン
            K_t = self.cov_t.dot(H_t.T).dot(np.linalg.inv(S_t))
            if self.debug : print("K_t", K_t)

            # 事後推定値
            self.bel_pos = self.bel_pos + K_t.dot(z_t - z_hat_t)
            if self.debug : print("posteriori bel_pos(t)", self.bel_pos)

            # 事後誤差共分散
            self.cov_t = (np.eye(3) - K_t.dot(H_t)).dot(self.cov_t)
            if self.debug : print("cov_t", self.cov_t)

    def draw(self, subplot, color='blue', label='belief'):
        """
        推定される位置を描画するためのメソッド
        """
        xs = self.bel_pos[0]
        ys = self.bel_pos[1]
        vxs = math.cos(self.bel_pos[2])
        vys = math.sin(self.bel_pos[2])
        sub_color = color
        sub_label = label
        subplot.quiver(xs, ys, vxs, vys, color = sub_color, label = sub_label)
```

```python
# 新規ウィンドウと座標軸用subplotを用意
# タイトルは個別につけるので今は指定しない
fig, world = plt.subplots(figsize=(8, 8))

# ランドマークの座標を指定
landmarks = Landmarks([[-1.0, 0.5], [1.0, 0.5], [0.0, 1.5]])
#landmarks = Landmarks([[-0.5, 0.0], [0.5, 0.0], [0.0, 0.5]])

# ロボットの座標を指定
robot = Robot(0.0, 0.0, 0.0)

# アニメーション作成の初期処理
# 初期処理を指定しないと、更新処理が1回分多く呼び出される
def init():
    pass

# アニメーション作成の更新処理
def update(i):
    # 座標軸用subplotの描画内容を一度クリア
    plt.cla()
    # 座標軸の範囲を指定
    world.set_xlim(-2.0,2.0)
    world.set_ylim(-1.5,2.5)
    # ランドマークを描画
    landmarks.draw()
    # ロボットを移動
    robot.move(0.2, np.radians(20))
    # 移動後の位置でランドマークを観測
    obss = robot.observation(landmarks)
    # ロボットを描画
    robot.draw(world, obss)
    # タイトルを追加
    plt.title("Robot Motion : frame{0:04d}".format(i))
    # 左上に凡例を追加
    world.legend(loc='upper left')

# アニメーションの作成
anim = animation.FuncAnimation(fig, update, init_func=init, interval=500, frames=20)
```

```python
HTML(anim.to_html5_video())
```

```python
# figureの初期化
plt.close()

# ランドマークの座標を指定
landmarks = Landmarks([[-1.0, 0.5], [1.0, 0.5], [0.0, 1.5]])
#landmarks = Landmarks([[-0.5, 0.0], [0.5, 0.0], [0.0, 0.5]])

# ロボットの座標を指定
robot = Robot(0.0, 0.0, 0.0)

# アニメーション作成の更新処理
def update(i):
    # 新規ウィンドウと座標軸用subplotを用意
    # タイトルは個別につけるので今は指定しない
    fig, world = plt.subplots(figsize=(12, 12))
    # 座標軸の範囲を指定
    world.set_xlim(-2.0,2.0)
    world.set_ylim(-1.5,2.5)
    # ランドマークを描画
    landmarks.draw()
    # ロボットを移動
    robot.move(0.2, np.radians(20))
    # 移動後の位置でランドマークを観測
    obss = robot.observation(landmarks)
    # ロボットを描画
    robot.draw(world, obss)
    # タイトルを追加
    plt.title("Robot Motion : frame{0:04d}".format(i))
    # 左上に凡例を追加
    world.legend(loc='upper left')

# アニメーションの作成
for i in range(0, 20):
    update(i)
    plt.show()
```


# Week 4. Training Issues
In this part, we will formally set up a simple but powerful classification network, to recogize 0-9 nubmers in MNIST dataset.

Yep, we will build a classification network and train from scratch.

We would introduce some techniques to improve your train model performance.

This part is designed and completed by **Jiaxin Zhuang( zhuangjx5@mail2.sysu.edu.cn )** and Feifei Xue(xueff@mail2.sysu.edu.cn), if you have some questions about this part and you think there are still some things to do, dont't hesitate to email us or add our wechat.

# Outline

1. Outline
    1. Required modules ( If you use your own computer, Just pip install it ! )
    2. Common Setup
2. classificatioon network
    1. short introdution of MNIST
    2. Define a Feedforward Neural network
3. Training
    0. Including that define a model, loss function, metric, data-augmentation for training data
    1. Pre-set hyper-parameters
    2. Initialize model parameters
    3. repeat over certain number of epochs
        1. Shuffle whole training data
        2. For each mini-batch data
            1. load mini-batch data
            2. compute gradient of loss over parameters
            3. update parameters with gradient descent
    4. save model
4. Training advanced
    1. l2\_norm
    2. dropout
    3. batch_normalization
    4. data augmentation
5. Visualizatio of training and validation phase
    1. add tensorboardX to writer summary into tensorboard
    2. download your file in local
    3. run tensorboard in pc and open http://localhost:6666 to browse the tensorboard
6. Gradient
    1. Gradient vanishing
    2. Gradient exploding

```python
%load_ext autoreload
%autoreload 2
```

# 1.1 Required Module

[numpy](https://docs.scipy.org/doc/numpy-1.13.0/user/whatisnumpy.html): NumPy is the fundamental package for scientific computing in Python.

[pytorch](https://pytorch.org/docs/stable/index.html): End-to-end deep learning platform.

[torchvision](https://pytorch.org/docs/stable/torchvision/index.html): This package consists of popular datasets, model architectures, and common image transformations for computer vision.

[tensorflow](https://www.tensorflow.org/guide): An open source machine learning framework.

[tensorboard](https://www.tensorflow.org/guide/summaries_and_tensorboard): A suite of visualization tools to make training easier to understand, debug, and optimize TensorFlow programs.

[tensorboardX](https://tensorboardx.readthedocs.io/en/latest/tensorboard.html): Tensorboard for Pytorch.

[matplotlib](https://matplotlib.org/): It is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms.

# 1.2 Common Setup

```python
# Load all necessary modules here, for clearness
import torch
import numpy as np
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
# from torchvision.datasets import MNIST
import torchvision
from torchvision import transforms
from torch.optim import lr_scheduler
from tensorboardX import SummaryWriter
from collections import OrderedDict
import matplotlib.pyplot as plt
from tqdm import tqdm
```

```python
# Whether to put data in GPU according to GPU is available or not
# cuda = torch.cuda.is_available()
#  In case the default gpu does not have enough space, you can choose which device to use
# torch.cuda.set_device(device) # device: id

# Since gpu in lab is not enough for your guys, we prefer to cpu computation
cuda = torch.device('cpu')
```

# 2. Classfication Model
Ww would define a simple Convolutional Neural Network to classify MNIST

## 2.1 Short indroduction of MNIST
The MNIST database (Modified National Institute of Standards and Technology database) is a large database of handwritten digits that is commonly used for training various image processing systems.

The MNIST database contains 60,000 training images and 10,000 testing images. Each class has 5000 traning images and 1000 test images.

Each image is 28x28.

And they look like images below.

# 2.2 Define A FeedForward Neural Network
We would define a FeedForward Neural Network with **3 hidden layers**.

Each layer is followed a activation function, we would try **sigmoid** and **relu** respectively.

For simplicity, each hidden layer has the equal neurons.

In reality, however, we would apply different amount of neurons in different hidden layers.

## 2.2.1 Activation Function
There are many useful activation function and you can choose one of them to use. Usually we use **relu** as our network function.

#### 2.2.1.1 ReLU
Applies the rectified linear unit function element-wise

\begin{equation}
    ReLU(x) = max(0, x)
\end{equation}

#### 2.2.1.2 Sigmoid
Applies the element-wise function:

\begin{equation}
Sigmoid(x)=\frac{1}{1+e^{-x}}
\end{equation}

## 2.2.2 Network's Input and output
Inputs: For every batch
>\[batchSize, channels, height, width\] -> [B,C,H,W]

Outputs: prediction scores of each images, eg. \[0.001, 0.0034 ..., 0.3\]
> \[batchSize, classes\]

Network Strutrue

```
    Inputs                Linear/Function        Output
    [128, 1, 28, 28]   -> Linear(28*28, 100) -> [128, 100]  # first hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 100)   -> [128, 100]  # second hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 100)   -> [128, 100]  # third hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 10)    -> [128, 10]   # Classification Layer
```

```python
class FeedForwardNeuralNetwork(nn.Module):
    """
    Inputs                Linear/Function        Output
    [128, 1, 28, 28]   -> Linear(28*28, 100) -> [128, 100]  # first hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 100)   -> [128, 100]  # second hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 100)   -> [128, 100]  # third hidden lyaer
                       -> ReLU               -> [128, 100]  # relu activation function, may sigmoid
                       -> Linear(100, 10)    -> [128, 10]   # Classification Layer
   """
    def __init__(self, input_size, hidden_size, output_size, activation_function='RELU'):
        super(FeedForwardNeuralNetwork, self).__init__()
        self.use_dropout = False
        self.use_bn = False
        self.hidden1 = nn.Linear(input_size, hidden_size)  # Linear function 1: 784 --> 100
        self.hidden2 = nn.Linear(hidden_size, hidden_size) # Linear function 2: 100 --> 100
        self.hidden3 = nn.Linear(hidden_size, hidden_size) # Linear function 3: 100 --> 100
        # Linear function 4 (readout): 100 --> 10
        self.classification_layer = nn.Linear(hidden_size, output_size)
        self.dropout = nn.Dropout(p=0.5) # Drop out with prob = 0.5
        self.hidden1_bn = nn.BatchNorm1d(hidden_size) # Batch Normalization
        self.hidden2_bn = nn.BatchNorm1d(hidden_size)
        self.hidden3_bn = nn.BatchNorm1d(hidden_size)

        # Non-linearity
        if activation_function == 'SIGMOID':
            self.activation_function1 = nn.Sigmoid()
            self.activation_function2 = nn.Sigmoid()
            self.activation_function3 = nn.Sigmoid()
        elif activation_function == 'RELU':
            self.activation_function1 = nn.ReLU()
            self.activation_function2 = nn.ReLU()
            self.activation_function3 = nn.ReLU()

    def forward(self, x):
        """Defines the computation performed at every call.
           Should be overridden by all subclasses.
        Args:
            x: [batch_size, channel, height, width], input for network
        Returns:
            out: [batch_size, n_classes], output from network
        """

        x = x.view(x.size(0), -1) # flatten x in [128, 784]
        out = self.hidden1(x)
        out = self.activation_function1(out) # Non-linearity 1
        if self.use_bn == True:
            out = self.hidden1_bn(out)
        out = self.hidden2(out)
        out = self.activation_function2(out)
        if self.use_bn == True:
            out = self.hidden2_bn(out)
        out = self.hidden3(out)
        if self.use_bn == True:
            out = self.hidden3_bn(out)
        out = self.activation_function3(out)
        if self.use_dropout == True:
            out = self.dropout(out)
        out = self.classification_layer(out)
        return out

    def set_use_dropout(self, use_dropout):
        """Whether to use dropout. Auxiliary function for our exp, not necessary.
        Args:
            use_dropout: True, False
        """
        self.use_dropout = use_dropout

    def set_use_bn(self, use_bn):
        """Whether to use batch normalization. Auxiliary function for our exp, not necessary.
        Args:
            use_bn: True, False
        """
        self.use_bn = use_bn

    def get_grad(self):
        """Return average grad for hidden2, hidden3. Auxiliary function for our exp, not necessary.
        """
        hidden2_average_grad = np.mean(np.sqrt(np.square(self.hidden2.weight.grad.detach().numpy())))
        hidden3_average_grad = np.mean(np.sqrt(np.square(self.hidden3.weight.grad.detach().numpy())))
        return hidden2_average_grad, hidden3_average_grad
```

# 3. Training
We would define training function here. Additionally, hyper-parameters, loss function, metric would be included here too.

## 3.1 Pre-set hyper-parameters
setting hyperparameters like below

hyper paprameters include following part

* learning rate: usually we start from a quite bigger lr like 1e-1, 1e-2, 1e-3, and slow lr as epoch moves.
* n_epochs: training epoch must set large so model has enough time to converge. Usually, we will set a quite big epoch at the first training time.
* batch_size: usually, bigger batch size mean's better usage of GPU and model would need less epoches to converge. And the exponent of 2 is used, eg. 2, 4, 8, 16, 32, 64, 128. 256.

```python
### Hyper parameters

batch_size = 128 # batch size is 128
n_epochs = 5 # train for 5 epochs
learning_rate = 0.01 # learning rate is 0.01
input_size = 28*28 # input image has size 28x28
hidden_size = 100 # hidden neurons is 100 for each layer
output_size = 10 # classes of prediction
l2_norm = 0 # not to use l2 penalty
dropout = False # not to use
get_grad = False # not to obtain grad
```

```python
# create a model object
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

## 3.2 Initialize model parameters
Pytorch provide default initialization (**uniform intialization**) for linear layer. But there is still some useful intialization method.

Read more about initialization from this [link](https://pytorch.org/docs/stable/_modules/torch/nn/init.html)

```
    torch.nn.init.normal_
    torch.nn.init.uniform_
    torch.nn.init.constant_
    torch.nn.init.eye_
    torch.nn.init.xavier_uniform_
    torch.nn.init.xavier_normal_
    torch.nn.init.kaiming_uniform_
```

```python
def show_weight_bias(model):
    """Show some weights and bias distribution every layers in model.
       !!YOU CAN READ THIS CODE LATER!!
    """
    # Create a figure and a set of subplots
    fig, axs = plt.subplots(2,3, sharey=False, tight_layout=True)

    # weight and bias for every hidden layer
    h1_w = model.hidden1.weight.detach().numpy().flatten()
    h1_b = model.hidden1.bias.detach().numpy().flatten()
    h2_w = model.hidden2.weight.detach().numpy().flatten()
    h2_b = model.hidden2.bias.detach().numpy().flatten()
    h3_w = model.hidden3.weight.detach().numpy().flatten()
    h3_b = model.hidden3.bias.detach().numpy().flatten()

    axs[0,0].hist(h1_w)
    axs[0,1].hist(h2_w)
    axs[0,2].hist(h3_w)
    axs[1,0].hist(h1_b)
    axs[1,1].hist(h2_b)
    axs[1,2].hist(h3_b)

    # set title for every sub plots
    axs[0,0].set_title('hidden1_weight')
    axs[0,1].set_title('hidden2_weight')
    axs[0,2].set_title('hidden3_weight')
    axs[1,0].set_title('hidden1_bias')
    axs[1,1].set_title('hidden2_bias')
    axs[1,2].set_title('hidden3_bias')
```

```python
# Show default initialization for every hidden layer by pytorch
# it's uniform distribution
show_weight_bias(model)
```

```python
# If you want to use other intialization method, you can use code below
# and define your initialization below

def weight_bias_reset(model):
    """Custom initialization, you can use your favorable initialization method.
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # initialize linear layer with mean and std
            mean, std = 0, 0.1

            # Initialization method
            torch.nn.init.normal_(m.weight, mean, std)
            torch.nn.init.normal_(m.bias, mean, std)

#             Another way to initialize
#             m.weight.data.normal_(mean, std)
#             m.bias.data.normal_(mean, std)
```

```python
weight_bias_reset(model) # reset parameters for each hidden layer
show_weight_bias(model) # show weight and bias distribution, normal distribution now.
```

## 作业1
使用 torch.nn.init.constant, torch.nn.init.xavier_uniform_, torch.nn.init_xavier_normal_去重写初始化函数，使用对应函数初始化模型，并且使用show_weight_bias显示模型隐藏层的参数分布。此处应该有6个cell作答。!!不必初始化bias！！

```python
# TODO

def weight_bias_reset_constant(model):
    """Constant initalization
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # remove pass and code here
            pass
```

```python
# TODO

# Reset parameters and show their distribution
```

```python
# TODO

def weight_bias_reset_xavier_uniform(model):
    """xaveir_uniform, gain=1
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # remove pass and code here
            pass
```

```python
# TODO

# Reset parameters and show their distribution
```

```python
# TODO

def weight_bias_reset_kaiming_uniform(model):
    """kaiming_uniform, a=0,model='fan_in', non_linearity='relu'
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # remove pass and code here
            pass
```

```python
# TODO

# Reset parameters and show their distribution
```

## 3.3 Repeat over certain numbers of epoch

* Shuffle whole training data

```shuffle
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True, **kwargs)
```
* For each mini-batch data

    * load mini-batch data

    ```
    for batch_idx, (data, target) in enumerate(train_loader): \
        ...
    ```

    * compute gradient of loss over parameters

    ```
     output = net(data) # make prediction
     loss = loss_fn(output, target)  # compute loss
     loss.backward() # compute gradient of loss over parameters
    ```

    * update parameters with gradient descent

    ```
    optimzer.step() # update parameters with gradient descent
    ```

### 3.3.1 Shuffle whole traning data

#### 3.3.1.1 Data Loading
Please pay attention to data augmentation.

Read more data augmentation method from this [link](https://pytorch.org/docs/stable/torchvision/transforms.html).

```
torchvision.transforms.RandomVerticalFlip
torchvision.transforms.RandomHorizontalFlip
...
```

```python
# define method of preprocessing data for evaluating

train_transform = transforms.Compose([
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean 0.1307 and standard deviation 0.3081
    transforms.Normalize((0.1307,), (0.3081,))
])

test_transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])
```

```python
# use MNIST provided by torchvision

# torchvision.datasets provide MNIST dataset for classification

train_dataset = torchvision.datasets.MNIST(root='./data',
                            train=True,
                            transform=train_transform,
                            download=True)

test_dataset = torchvision.datasets.MNIST(root='./data',
                           train=False,
                           transform=test_transform,
                           download=False)
```

```python
# pay attention to this, train_dataset doesn't load any data
# It just defined some method and store some message to preprocess data
train_dataset
```

    Dataset MNIST
        Number of datapoints: 60000
        Split: train
        Root Location: ./data
        Transforms (if any): Compose(
                                 ToTensor()
                                 Normalize(mean=(0.1307,), std=(0.3081,))
                             )
        Target Transforms (if any): None

```python
# Data loader.

# Combines a dataset and a sampler,
# and provides single- or multi-process iterators over the dataset.

train_loader = torch.utils.data.DataLoader(dataset=train_dataset,
                                           batch_size=batch_size,
                                           shuffle=False)

test_loader = torch.utils.data.DataLoader(dataset=test_dataset,
                                          batch_size=batch_size,
                                          shuffle=False)
```

```python
# functions to show an image

def imshow(img):
    """show some imgs in datasets
        !!YOU CAN READ THIS CODE LATER!! """

    npimg = img.numpy() # convert tensor to numpy
    plt.imshow(np.transpose(npimg, (1, 2, 0))) # [channel, height, width] -> [height, width, channel]
    plt.show()
```

```python
# get some random training images by batch

dataiter = iter(train_loader)
images, labels = dataiter.next() # get a batch of images

# show images
imshow(torchvision.utils.make_grid(images))
```

### 3.3.2 & 3.3.3 compute gradient of loss over parameters & update parameters with gradient descent

```python
def train(train_loader, model, loss_fn, optimizer, get_grad=False):
    """train model using loss_fn and optimizer. When thid function is called, model trains for one epoch.
    Args:
        train_loader: train data
        model: prediction model
        loss_fn: loss function to judge the distance between target and outputs
        optimizer: optimize the loss function
        get_grad: True, False
    Returns:
        total_loss: loss
        average_grad2: average grad for hidden 2 in this epoch
        average_grad3: average grad for hidden 3 in this epoch
    """

    # set the module in training model, affecting module e.g., Dropout, BatchNorm, etc.
    model.train()

    total_loss = 0
    grad_2 = 0.0 # store sum(grad) for hidden 3 layer
    grad_3 = 0.0 # store sum(grad) for hidden 3 layer

    for batch_idx, (data, target) in enumerate(train_loader):
        optimizer.zero_grad() # clear gradients of all optimized torch.Tensors'
        outputs = model(data) # make predictions
        loss = loss_fn(outputs, target) # compute loss
        total_loss += loss.item() # accumulate every batch loss in a epoch
        loss.backward() # compute gradient of loss over parameters

        if get_grad == True:
            g2, g3 = model.get_grad() # get grad for hiddern 2 and 3 layer in this batch
            grad_2 += g2 # accumulate grad for hidden 2
            grad_3 += g3 # accumulate grad for hidden 2

        optimizer.step() # update parameters with gradient descent

    average_loss = total_loss / batch_idx # average loss in this epoch
    average_grad2 = grad_2 / batch_idx # average grad for hidden 2 in this epoch
    average_grad3 = grad_3 / batch_idx # average grad for hidden 3 in this epoch

    return average_loss, average_grad2, average_grad3
```

```python
def evaluate(loader, model, loss_fn):
    """test model's prediction performance on loader.
    When thid function is called, model is evaluated.
    Args:
        loader: data for evaluation
        model: prediction model
        loss_fn: loss function to judge the distance between target and outputs
    Returns:
        total_loss
        accuracy
    """

    # context-manager that disabled gradient computation
    with torch.no_grad():

        # set the module in evaluation mode
        model.eval()

        correct = 0.0 # account correct amount of data
        total_loss = 0  # account loss

        for batch_idx, (data, target) in enumerate(loader):
            outputs = model(data) # make predictions
            # return the maximum value of each row of the input tensor in the
            # given dimension dim, the second return vale is the index location
            # of each maxium value found(argmax)
            _, predicted = torch.max(outputs, 1)
            # Detach: Returns a new Tensor, detached from the current graph.
            #The result will never require gradient.
            correct += (predicted == target).sum().detach().numpy()
            loss = loss_fn(outputs, target)  # compute loss
            total_loss += loss.item() # accumulate every batch loss in a epoch

        accuracy = correct*100.0 / len(loader.dataset) # accuracy in a epoch

    return total_loss, accuracy
```

Define function fit and use train_epoch and test_epoch

```python
def fit(train_loader, val_loader, model, loss_fn, optimizer, n_epochs, get_grad=False):
    """train and val model here, we use train_epoch to train model and
    val_epoch to val model prediction performance
    Args:
        train_loader: train data
        val_loader: validation data
        model: prediction model
        loss_fn: loss function to judge the distance between target and outputs
        optimizer: optimize the loss function
        n_epochs: training epochs
        get_grad: Whether to get grad of hidden2 layer and hidden3 layer
    Returns:
        train_accs: accuracy of train n_epochs, a list
        train_losses: loss of n_epochs, a list
    """

    grad_2 = [] # save grad for hidden 2 every epoch
    grad_3 = [] # save grad for hidden 3 every epoch

    train_accs = [] # save train accuracy every epoch
    train_losses = [] # save train loss every epoch

    for epoch in range(n_epochs): # train for n_epochs
        # train model on training datasets, optimize loss function and update model parameters
        train_loss, average_grad2, average_grad3 = train(train_loader, model, loss_fn, optimizer, get_grad)

        # evaluate model performance on train dataset
        _, train_accuracy = evaluate(train_loader, model, loss_fn)
        message = 'Epoch: {}/{}. Train set: Average loss: {:.4f}, Accuracy: {:.4f}'.format(epoch+1, \
                                                                n_epochs, train_loss, train_accuracy)
        print(message)

        # save loss, accuracy, grad
        train_accs.append(train_accuracy)
        train_losses.append(train_loss)
        grad_2.append(average_grad2)
        grad_3.append(average_grad3)

        # evaluate model performance on val dataset
        val_loss, val_accuracy = evaluate(val_loader, model, loss_fn)
        message = 'Epoch: {}/{}. Validation set: Average loss: {:.4f}, Accuracy: {:.4f}'.format(epoch+1, \
                                                                n_epochs, val_loss, val_accuracy)
        print(message)

    # Whether to get grad for showing
    if get_grad == True:
        fig, ax = plt.subplots() # add a set of subplots to this figure
        ax.plot(grad_2, label='Gradient for Hidden 2 Layer') # plot grad 2
        ax.plot(grad_3, label='Gradient for Hidden 3 Layer') # plot grad 3
        plt.ylim(top=0.004)
        # place a legend on axes
        legend = ax.legend(loc='best', shadow=True, fontsize='x-large')

    return train_accs, train_losses
```

```python
def show_curve(ys, title):
    """plot curlve for Loss and Accuacy

    !!YOU CAN READ THIS LATER, if you are interested

    Args:
        ys: loss or acc list
        title: Loss or Accuracy
    """
    x = np.array(range(len(ys)))
    y = np.array(ys)
    plt.plot(x, y, c='b')
    plt.axis()
    plt.title('{} Curve:'.format(title))
    plt.xlabel('Epoch')
    plt.ylabel('{} Value'.format(title))
    plt.show()
```

## 作业 2
1. 运行一下fit函数，根据结束时候训练集的accuracy，回答：模型是否训练到过拟合。
2. 使用提供的show_curve函数，画出训练的时候loss和accuracy的变化

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。

```python
### Hyper parameters

batch_size = 128 # batch size is 128
n_epochs = 5 # train for 5 epochs
learning_rate = 0.01 # learning rate is 0.01
input_size = 28*28 # input image has size 28x28
hidden_size = 100 # hidden neurons is 100 for each layer
output_size = 10 # classes of prediction
l2_norm = 0 # not to use l2 penalty
dropout = False # not to use
get_grad = False # not to obtain grad

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

```python
# TODO

# show curve
```

## 作业 3
1. 将n_epochs设为10，观察模型是否能在训练集上达到过拟合， 使用show_curve作图。
2. 当希望模型在5个epoch内在训练集上达到过拟合，可以通过适当调整learning rate来实现。选择一个合适的learing rate，训练模型，并且使用show_curve作图， 验证你的learning rate

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。

```python
### Hyper parameters

batch_size = 128 # batch size is 128
n_epochs = 5 # train for 5 epochs
learning_rate = 0.01 # learning rate is 0.01
input_size = 28*28 # input image has size 28x28
hidden_size = 100 # hidden neurons is 100 for each layer
output_size = 10 # classes of prediction
l2_norm = 0 # not to use l2 penalty
dropout = False # not to use
get_grad = False # not to obtain grad

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
# TODO

# 3.1 Train
```

```python
# TODO

# 3.1 show_curve
```

```python
# TODO

# 3.2 Train
```

```python
# TODO

# 3.2 show_curve
```

### 3.4 save model
Pytorch provide two kinds of method to save model. We recommmend the method which only saves parameters. Because it's more feasible and dont' rely on fixed model.

When saving parameters, we not only save **learnable parameters in model**, but also **learnable parameters in optimizer**.

A common PyTorch convention is to save models using either a .pt or .pth file extension.

Read more abount save load from this [link](https://pytorch.org/tutorials/beginner/saving_loading_models.html)

```python
# show parameters in model

# Print model's state_dict
print("Model's state_dict:")
for param_tensor in model.state_dict():
    print(param_tensor, "\t", model.state_dict()[param_tensor].size())

# Print optimizer's state_dict
print("\nOptimizer's state_dict:")
for var_name in optimizer.state_dict():
    print(var_name, "\t", optimizer.state_dict()[var_name])
```

    Model's state_dict:
    hidden1.weight 	 torch.Size([100, 784])
    hidden1.bias 	 torch.Size([100])
    hidden2.weight 	 torch.Size([100, 100])
    hidden2.bias 	 torch.Size([100])
    hidden3.weight 	 torch.Size([100, 100])
    hidden3.bias 	 torch.Size([100])
    classification_layer.weight 	 torch.Size([10, 100])
    classification_layer.bias 	 torch.Size([10])
    hidden1_bn.weight 	 torch.Size([100])
    hidden1_bn.bias 	 torch.Size([100])
    hidden1_bn.running_mean 	 torch.Size([100])
    hidden1_bn.running_var 	 torch.Size([100])
    hidden1_bn.num_batches_tracked 	 torch.Size([])
    hidden2_bn.weight 	 torch.Size([100])
    hidden2_bn.bias 	 torch.Size([100])
    hidden2_bn.running_mean 	 torch.Size([100])
    hidden2_bn.running_var 	 torch.Size([100])
    hidden2_bn.num_batches_tracked 	 torch.Size([])
    hidden3_bn.weight 	 torch.Size([100])
    hidden3_bn.bias 	 torch.Size([100])
    hidden3_bn.running_mean 	 torch.Size([100])
    hidden3_bn.running_var 	 torch.Size([100])
    hidden3_bn.num_batches_tracked 	 torch.Size([])

    Optimizer's state_dict:
    state 	 {}
    param_groups 	 [{'lr': 0.01, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, 'nesterov': False, 'params': [140033366951904, 140033366951544, 140033366952768, 140033366953560, 140033366952552, 140033366952480, 140033304926824, 140033304924736, 140033304925384, 140033304926968, 140033304926248, 140033304926176, 140033304926680, 140033304927760]}]

```python
# save model

save_path = './model.pt'
torch.save(model.state_dict(), save_path)
```

```python
# load parameters from files
saved_parametes = torch.load(save_path)
print(saved_parametes)
```

    OrderedDict([('hidden1.weight', tensor([[-0.0056,  0.0096,  0.0256,  ...,  0.0314, -0.0123, -0.0245],
            [ 0.0174, -0.0271,  0.0291,  ...,  0.0249,  0.0207,  0.0309],
            [-0.0281, -0.0064, -0.0113,  ...,  0.0044, -0.0126,  0.0320],
            ...,
            [ 0.0276, -0.0181, -0.0149,  ..., -0.0175, -0.0306, -0.0275],
            [-0.0207,  0.0350, -0.0084,  ...,  0.0275, -0.0060, -0.0181],
            [-0.0036,  0.0163, -0.0011,  ..., -0.0013, -0.0349, -0.0043]])), ('hidden1.bias', tensor([-1.1949e-02,  3.9362e-03,  9.4911e-03,  8.6605e-03, -1.0223e-02,
            -2.2284e-02, -3.4768e-02,  2.3229e-02, -8.8324e-03,  2.2501e-02,
            -3.2246e-02,  5.1050e-03,  1.0391e-02,  2.7352e-02,  1.1459e-02,
            -1.1725e-02, -3.0930e-02,  2.0169e-02,  2.9972e-03,  2.5611e-02,
             3.2411e-02, -3.5713e-02, -3.0771e-02, -1.5290e-02, -2.1979e-03,
             1.5742e-02,  9.2924e-03,  1.2081e-02,  5.5597e-03,  3.0818e-02,
            -1.2302e-02,  2.2055e-02, -4.0043e-04, -1.4790e-02,  1.0790e-02,
             1.1346e-02,  3.2512e-03, -1.1293e-02,  3.2924e-02, -1.5291e-02,
            -7.9001e-03, -2.3216e-02, -6.0322e-03,  1.5571e-02, -3.5401e-02,
             4.2528e-03,  3.1193e-02, -1.0147e-02, -2.0139e-03, -1.2018e-02,
            -3.2551e-03, -2.2296e-02, -1.8701e-02, -1.1076e-02,  1.3640e-02,
             2.1068e-02, -3.2070e-02, -1.6288e-02,  3.0494e-02,  1.0417e-03,
             2.6660e-02, -2.6842e-02,  9.0875e-03,  1.0330e-02,  1.3220e-02,
            -1.4060e-02,  3.5329e-02, -1.6400e-02,  2.7394e-02,  1.0365e-02,
             2.1799e-02, -9.8232e-03,  2.3748e-02, -2.1644e-03, -2.2229e-02,
            -1.9397e-02, -1.8717e-02, -6.9130e-05, -1.3481e-02, -2.3714e-02,
             2.1718e-02,  2.9248e-03, -8.3387e-04, -3.1779e-02, -1.5879e-02,
            -3.0405e-02, -2.1847e-02,  1.0549e-02,  2.5400e-02,  1.2406e-02,
            -2.1816e-02, -2.9822e-02,  1.3116e-02, -1.1967e-02,  2.8561e-02,
            -2.1895e-02, -2.1483e-02,  2.4633e-02,  3.4562e-02, -3.3654e-02])), ('hidden2.weight', tensor([[ 0.0649, -0.0519,  0.0146,  ..., -0.0631,  0.0116, -0.0864],
            [-0.0470, -0.0516, -0.0868,  ..., -0.0634, -0.0820,  0.0274],
            [ 0.0007, -0.0025, -0.0272,  ...,  0.0437, -0.0126, -0.0186],
            ...,
            [ 0.0482,  0.0474, -0.0640,  ...,  0.0428,  0.0875,  0.0842],
            [-0.0976, -0.0642, -0.0021,  ..., -0.0706,  0.0800,  0.0147],
            [ 0.0588, -0.0735,  0.0110,  ...,  0.0324, -0.0083, -0.0898]])), ('hidden2.bias', tensor([-0.0144, -0.0361, -0.0283, -0.0489,  0.0079,  0.0916, -0.0738,  0.0614,
            -0.0209,  0.0889, -0.0243,  0.0847,  0.0437,  0.0450,  0.0265, -0.0497,
             0.0850,  0.0315,  0.0431, -0.0312,  0.0113, -0.0903, -0.0952, -0.0910,
            -0.0679,  0.0321,  0.0612, -0.0019, -0.0444,  0.0302,  0.0355,  0.0317,
             0.0654,  0.0464,  0.0652,  0.0465, -0.0680,  0.0164, -0.0721, -0.0306,
             0.0007, -0.0142,  0.0183,  0.0354, -0.0466, -0.0208, -0.0910, -0.0069,
             0.0673, -0.0196,  0.0389, -0.0908, -0.0654, -0.0812,  0.0753, -0.0985,
             0.0061,  0.0300,  0.0763,  0.0513, -0.0316,  0.0259, -0.0836, -0.0454,
            -0.0266, -0.0053, -0.0438,  0.0279, -0.0181, -0.0499,  0.0293, -0.0053,
            -0.0637,  0.0553, -0.0622, -0.0756,  0.0062,  0.0331, -0.0716, -0.0437,
             0.0918, -0.0075, -0.0988, -0.0708,  0.0840, -0.0479, -0.0055, -0.0654,
            -0.0481,  0.0077,  0.0885, -0.0813,  0.0214, -0.0405,  0.0435,  0.0887,
             0.0957, -0.0758, -0.0664, -0.0945])), ('hidden3.weight', tensor([[-0.0654, -0.0952, -0.0538,  ..., -0.0442,  0.0721, -0.0422],
            [ 0.0928, -0.0530,  0.0196,  ...,  0.0611,  0.0505,  0.0238],
            [ 0.0390,  0.0025,  0.0027,  ...,  0.0778,  0.0148, -0.0749],
            ...,
            [ 0.0738, -0.0315, -0.0227,  ..., -0.0263, -0.0458, -0.0501],
            [ 0.0056,  0.0554, -0.0756,  ..., -0.0179,  0.0821,  0.0949],
            [ 0.0437, -0.0884, -0.0438,  ...,  0.0247, -0.0304,  0.0375]])), ('hidden3.bias', tensor([-0.0969, -0.0380, -0.0387, -0.0677,  0.0018, -0.0203,  0.0170,  0.0932,
             0.0049, -0.0671, -0.0996,  0.0424,  0.0831,  0.0844, -0.0162, -0.0407,
            -0.0583, -0.0721,  0.0526,  0.0030, -0.0787, -0.0264, -0.0056, -0.0624,
             0.0447, -0.0642,  0.0267,  0.0743, -0.0642,  0.0662,  0.0309, -0.0805,
             0.0945, -0.0376,  0.0609,  0.0771,  0.0777,  0.0649, -0.0735, -0.0407,
             0.0939,  0.0473, -0.0336, -0.0521,  0.0866, -0.0966,  0.0594, -0.0247,
             0.0195, -0.0466, -0.0815,  0.0717, -0.0017, -0.0482, -0.0636, -0.0906,
            -0.0512, -0.0676,  0.0662, -0.0913, -0.0707,  0.0690,  0.0346,  0.0164,
             0.0992, -0.0535, -0.0031,  0.0451,  0.0032, -0.0806, -0.0416, -0.0028,
            -0.0206, -0.0118, -0.0101,  0.0054,  0.0747, -0.0436,  0.0233,  0.0769,
             0.0009,  0.0481, -0.0146, -0.0732, -0.0245, -0.0059, -0.0221,  0.0369,
             0.0181, -0.0196, -0.0123,  0.0310,  0.0489, -0.0764,  0.0204,  0.0968,
             0.0471,  0.0450, -0.0374,  0.0066])), ('classification_layer.weight', tensor([[-0.0947, -0.0296,  0.0547,  0.0275,  0.0324, -0.0106,  0.0525,  0.0910,
             -0.0291, -0.0878, -0.0730, -0.0210, -0.0944,  0.0910, -0.0143,  0.0826,
              0.0060, -0.0581, -0.0078,  0.0852,  0.0970,  0.0238, -0.0738, -0.0322,
             -0.0762, -0.0103, -0.0805,  0.0280,  0.0066,  0.0583,  0.0112,  0.0859,
             -0.0311,  0.0254,  0.0421,  0.0716,  0.0604, -0.0575,  0.0196,  0.0720,
             -0.0653, -0.0766, -0.0840,  0.0042,  0.0919,  0.0165,  0.0524,  0.0608,
              0.0584,  0.0879,  0.0500,  0.0138, -0.0440, -0.0189,  0.0981,  0.0064,
             -0.0861, -0.0770,  0.0372, -0.0983, -0.0405,  0.0369,  0.0041, -0.0889,
             -0.0664,  0.0457, -0.0861, -0.0901, -0.0905,  0.0317,  0.0856, -0.0376,
              0.0467, -0.0812, -0.0667,  0.0240, -0.0752, -0.0958,  0.0100,  0.0337,
              0.0060, -0.0181,  0.0887,  0.0060, -0.0835,  0.0054,  0.0644, -0.0451,
              0.0974, -0.0632, -0.0172,  0.0953, -0.0782,  0.0881,  0.0041, -0.0394,
             -0.0921, -0.0176,  0.0322, -0.0305],
            [ 0.0930,  0.0268, -0.0289, -0.0409,  0.0739, -0.0384,  0.0306, -0.0782,
              0.0095, -0.0651, -0.0745,  0.0054,  0.0929, -0.0283, -0.0974, -0.0762,
              0.0683, -0.0871,  0.0824,  0.0384, -0.0625, -0.0946, -0.0960, -0.0205,
              0.0057,  0.0937, -0.0340,  0.0576, -0.0128, -0.0115, -0.0686, -0.0532,
              0.0421,  0.0452,  0.0994,  0.0287, -0.0260,  0.0369, -0.0862, -0.0185,
              0.0464, -0.0456, -0.0083, -0.0605,  0.0149,  0.0577,  0.0180,  0.0061,
             -0.0762,  0.0285,  0.0883, -0.0095,  0.0019,  0.0131, -0.0955,  0.0169,
             -0.0065, -0.0789,  0.0333, -0.0681, -0.0992, -0.0118,  0.0704, -0.0332,
              0.0602, -0.0441, -0.0574,  0.0546, -0.0472,  0.0266,  0.0823,  0.0734,
              0.0832,  0.0558, -0.0361,  0.0113, -0.0030, -0.0814,  0.0805, -0.0716,
              0.0263,  0.0872,  0.0516, -0.0693, -0.0538,  0.0151,  0.0556, -0.0448,
             -0.0030, -0.0627,  0.0145,  0.0602, -0.0139, -0.0631,  0.0139,  0.0819,
              0.0417, -0.0734,  0.0735,  0.0062],
            [ 0.0963,  0.0663, -0.0192,  0.0440,  0.0083, -0.0632,  0.0942,  0.0317,
              0.0396, -0.0195, -0.0632,  0.0785, -0.0902,  0.0560, -0.0099, -0.0173,
              0.0399, -0.0048,  0.0264, -0.0266, -0.0064,  0.0673, -0.0921,  0.0538,
              0.0897, -0.0720, -0.0081,  0.0813, -0.0003, -0.0451,  0.0355, -0.0125,
             -0.0757,  0.0775, -0.0061, -0.0008,  0.0246, -0.0940, -0.0759, -0.0841,
              0.0783, -0.0949, -0.0270, -0.0531, -0.0888, -0.0781, -0.0684, -0.0798,
             -0.0503,  0.0236, -0.0876,  0.0238, -0.0450,  0.0714,  0.0677, -0.0623,
              0.0522,  0.0473,  0.0416, -0.0206, -0.0477,  0.0693, -0.0964, -0.0330,
              0.0869, -0.0205,  0.0734,  0.0544, -0.0125, -0.0289, -0.0420, -0.0270,
              0.0475,  0.0537, -0.0822,  0.0700, -0.0392, -0.0808, -0.0178,  0.0813,
             -0.0964, -0.0211,  0.0681, -0.0793,  0.0596, -0.0997, -0.0780, -0.0490,
             -0.0997,  0.0741, -0.0557,  0.0320,  0.0609,  0.0353, -0.0437, -0.0104,
             -0.0174, -0.0215, -0.0454, -0.0495],
            [ 0.0972,  0.0933, -0.0555,  0.0119,  0.0512, -0.0152,  0.0041, -0.0527,
              0.0571,  0.0185, -0.0364, -0.0396,  0.0417,  0.0214,  0.0657, -0.0947,
             -0.0724, -0.0099,  0.0211,  0.0290, -0.0964, -0.0568,  0.0499,  0.0351,
              0.0566,  0.0678, -0.0521,  0.0628, -0.0615,  0.0270, -0.0987,  0.0372,
             -0.0808,  0.0183, -0.0261, -0.0561,  0.0623,  0.0503, -0.0188,  0.0567,
              0.0961, -0.0032, -0.0293,  0.0711,  0.0457,  0.0987, -0.0714,  0.0281,
             -0.0077, -0.0596,  0.0835,  0.0241, -0.0054, -0.0814, -0.0568,  0.0041,
             -0.0203, -0.0093,  0.0208, -0.0836,  0.0844, -0.0945, -0.0677, -0.0165,
              0.0285, -0.0291, -0.0725, -0.0683, -0.0475, -0.0608,  0.0062, -0.0227,
              0.0714,  0.0923, -0.0394,  0.0032,  0.0015,  0.0500,  0.0351, -0.0502,
              0.0644,  0.0267, -0.0621,  0.0472, -0.0687,  0.0452,  0.0962, -0.0710,
             -0.0503,  0.0205, -0.0692,  0.0776,  0.0231, -0.0033, -0.0520,  0.0563,
              0.0535,  0.0969, -0.0309,  0.0308],
            [ 0.0101, -0.0181,  0.0973,  0.0282, -0.0560,  0.0585,  0.0920, -0.0356,
             -0.0247,  0.0798,  0.0433,  0.0718, -0.0035, -0.0760, -0.0725, -0.0992,
              0.0444,  0.0406, -0.0544, -0.0056, -0.0208, -0.0954, -0.0584,  0.0960,
             -0.0243,  0.0413,  0.0973,  0.0087, -0.0568, -0.0886, -0.0296,  0.0127,
              0.0702,  0.0040,  0.0784, -0.0509,  0.0680, -0.0258,  0.0314, -0.0849,
              0.0221, -0.0584,  0.0745,  0.0608,  0.0695, -0.0266,  0.0051, -0.0599,
             -0.0639,  0.0307,  0.0472,  0.0483,  0.0774,  0.0952, -0.0295, -0.0014,
             -0.0700, -0.0099, -0.0991, -0.0664,  0.0657,  0.0247,  0.0876, -0.0917,
              0.0666,  0.0468,  0.0920,  0.0977, -0.0539,  0.0721, -0.0511, -0.0506,
             -0.0292,  0.0928, -0.0901, -0.0014,  0.0300, -0.0257,  0.0685, -0.0039,
             -0.0398,  0.0622,  0.0579,  0.0440,  0.0849, -0.0172, -0.0149, -0.0162,
             -0.0498, -0.0463, -0.0621,  0.0949,  0.0223,  0.0283, -0.0764,  0.0305,
              0.0139,  0.0762, -0.0097,  0.0347],
            [-0.0219, -0.0119,  0.0605,  0.0616,  0.0125, -0.0453,  0.0712,  0.0317,
             -0.0263, -0.0428,  0.0146,  0.0547,  0.0151,  0.0450,  0.0955, -0.0983,
              0.0091, -0.0964, -0.0668, -0.0438, -0.0047, -0.0014,  0.0347,  0.0121,
              0.0987, -0.0331, -0.0830, -0.0518, -0.0840,  0.0232, -0.0939, -0.0273,
              0.0952, -0.0281, -0.0494,  0.0023,  0.0366, -0.0445, -0.0314, -0.0334,
             -0.0908, -0.0058,  0.0893,  0.0006, -0.0914,  0.0177,  0.0429, -0.0670,
             -0.0658,  0.0661, -0.0795,  0.0095,  0.0757, -0.0478,  0.0707,  0.0782,
             -0.0895, -0.0065,  0.0791, -0.0637, -0.0514,  0.0035,  0.0113,  0.0223,
              0.0574,  0.0588,  0.0242, -0.0692,  0.0460, -0.0265, -0.0401,  0.0061,
             -0.0761,  0.0507,  0.0076, -0.0223, -0.0848,  0.0713,  0.0012,  0.0335,
              0.0469, -0.0862, -0.0344, -0.0462,  0.0405,  0.0003,  0.0400,  0.0509,
             -0.0958,  0.0309,  0.0454,  0.0955,  0.0994,  0.0263, -0.0561,  0.0608,
              0.0262, -0.0942,  0.0964, -0.0474],
            [-0.0994,  0.0172,  0.0555,  0.0566,  0.0710, -0.0119, -0.0140,  0.0583,
              0.0053, -0.0094,  0.0705, -0.0376,  0.0641, -0.0147,  0.0286, -0.0051,
             -0.0320, -0.0493, -0.0675,  0.0327, -0.0137, -0.0216, -0.0328,  0.0781,
              0.0053, -0.0028,  0.0395,  0.0025, -0.0167, -0.0576,  0.0933,  0.0491,
             -0.0698, -0.0958,  0.0096,  0.0810,  0.0474, -0.0046, -0.0615,  0.0218,
             -0.0019, -0.0972, -0.0919, -0.0783, -0.0556, -0.0606, -0.0150,  0.0777,
              0.0423, -0.0608, -0.0989, -0.0941,  0.0247,  0.0268, -0.0916,  0.0792,
             -0.0930,  0.0261, -0.0354, -0.0578,  0.0612,  0.0838, -0.0704,  0.0441,
              0.0019, -0.0633, -0.0302,  0.0370,  0.0491, -0.0618,  0.0146, -0.0403,
             -0.0127,  0.0306,  0.0747, -0.0165, -0.0634, -0.0502, -0.0665,  0.0318,
              0.0460, -0.0738,  0.0343, -0.0614,  0.0634,  0.0869, -0.0585,  0.0068,
              0.0165, -0.0526,  0.0471,  0.0023,  0.0282, -0.0710, -0.0987, -0.0873,
             -0.0237, -0.0286,  0.0169, -0.0311],
            [-0.0465,  0.0555,  0.0380, -0.0614, -0.0425, -0.0710,  0.0767, -0.0357,
             -0.0903, -0.0528,  0.0200, -0.0697, -0.0016, -0.0994,  0.0088, -0.0180,
              0.0790,  0.0566,  0.0725, -0.0158,  0.0996, -0.0162, -0.0947,  0.0624,
             -0.0252, -0.0529,  0.0243,  0.0683, -0.0833,  0.0586, -0.0722,  0.0215,
              0.0829,  0.0655, -0.0293, -0.0516, -0.0144, -0.0061,  0.0082, -0.0438,
             -0.0699, -0.0466, -0.0149, -0.0204,  0.0088,  0.0276, -0.0269,  0.0544,
             -0.0782,  0.0921, -0.0512, -0.0765, -0.0907, -0.0571, -0.0464, -0.0224,
              0.0298, -0.0861, -0.0933,  0.0484,  0.0101, -0.0977, -0.0766,  0.0294,
              0.0847, -0.0305,  0.0364,  0.0886,  0.0610, -0.0966, -0.0012,  0.0750,
              0.0272,  0.0226,  0.0497, -0.0781,  0.0828, -0.0595,  0.0631, -0.0456,
              0.0118, -0.0900, -0.0539, -0.0114, -0.0666,  0.0370, -0.0017,  0.0599,
              0.0157,  0.0314, -0.0895,  0.0141, -0.0848,  0.0176,  0.0045, -0.0730,
              0.0453,  0.0096, -0.0119, -0.0657],
            [-0.0080, -0.0633, -0.0267, -0.0506,  0.0475, -0.0934, -0.0716, -0.0372,
             -0.0666,  0.0093, -0.0563, -0.0398, -0.0012,  0.0570,  0.0689, -0.0458,
              0.0294,  0.0301, -0.0182, -0.0112, -0.0118, -0.0164, -0.0055,  0.0959,
             -0.0294,  0.0959,  0.0219, -0.0104,  0.0294,  0.0729,  0.0102, -0.0696,
              0.0923, -0.0276, -0.0717, -0.0677,  0.0771, -0.0384,  0.0373,  0.0630,
             -0.0980,  0.0888,  0.0050,  0.0391, -0.0481, -0.0475, -0.0968, -0.0959,
             -0.0041,  0.0891, -0.0441, -0.0018, -0.0381,  0.0839,  0.0094, -0.0007,
              0.0087, -0.0944,  0.0427, -0.0027,  0.0400,  0.0887,  0.0932, -0.0718,
             -0.0310,  0.0406, -0.0443,  0.0092, -0.0731,  0.0990, -0.0593, -0.0779,
             -0.0131, -0.0957,  0.0309,  0.0120,  0.0635, -0.0156,  0.0138, -0.0623,
              0.0374, -0.0853, -0.0190, -0.0553,  0.0446, -0.0207,  0.0467,  0.0951,
             -0.0805, -0.0974,  0.0105,  0.0428,  0.0192,  0.0235, -0.0746,  0.0212,
             -0.0675,  0.0490, -0.0324, -0.0965],
            [-0.0278,  0.0832,  0.0290,  0.0513,  0.0037,  0.0654,  0.0255,  0.0501,
              0.0464, -0.0271,  0.0028,  0.0485, -0.0307, -0.0568,  0.0459,  0.0288,
              0.0446, -0.0991, -0.0120, -0.0194, -0.0199,  0.0507,  0.0141,  0.0769,
              0.0406,  0.0892,  0.0965, -0.0600, -0.0763, -0.0007,  0.0395,  0.0648,
              0.0315, -0.0756,  0.0224,  0.0840,  0.0128, -0.0479, -0.0298,  0.0469,
             -0.0394, -0.0007, -0.0811, -0.0001,  0.0171, -0.0917, -0.0712,  0.0015,
             -0.0722, -0.0999, -0.0026, -0.0763,  0.0237, -0.0425, -0.0388,  0.0907,
              0.0652,  0.0520, -0.0358,  0.0128, -0.0782, -0.0524,  0.0241,  0.0131,
             -0.0822, -0.0841, -0.0869,  0.0376,  0.0882, -0.0602,  0.0032,  0.0956,
             -0.0062,  0.0242,  0.0762,  0.0259,  0.0156,  0.0999,  0.0397,  0.0613,
              0.0085,  0.0415,  0.0318, -0.0359,  0.0814, -0.0048,  0.0883,  0.0148,
             -0.0377, -0.0708, -0.0435, -0.0307,  0.0248,  0.0799, -0.0005,  0.0474,
              0.0618,  0.0964,  0.0108,  0.0173]])), ('classification_layer.bias', tensor([ 0.0938, -0.0915,  0.0312,  0.0080,  0.0923, -0.0631,  0.0445, -0.0516,
             0.0802,  0.0539])), ('hidden1_bn.weight', tensor([0.2641, 0.2875, 0.2584, 0.8671, 0.7530, 0.8996, 0.1242, 0.2656, 0.8032,
            0.5599, 0.8794, 0.8440, 0.6962, 0.6451, 0.7139, 0.2835, 0.7829, 0.5907,
            0.7485, 0.7074, 0.4070, 0.4516, 0.5127, 0.6463, 0.2160, 0.2042, 0.8773,
            0.1200, 0.5566, 0.2667, 0.1846, 0.0938, 0.4583, 0.8859, 0.7345, 0.9188,
            0.3178, 0.0675, 0.2951, 0.3370, 0.7652, 0.8962, 0.7824, 0.8657, 0.5546,
            0.1125, 0.6883, 0.2024, 0.7088, 0.9314, 0.2489, 0.9793, 0.5461, 0.0610,
            0.3534, 0.3852, 0.6393, 0.3760, 0.3415, 0.4445, 0.1629, 0.7077, 0.4490,
            0.5131, 0.4164, 0.0485, 0.7431, 0.3195, 0.4365, 0.1668, 0.2620, 0.2585,
            0.7262, 0.8578, 0.6468, 0.7684, 0.1772, 0.7548, 0.4943, 0.6163, 0.7765,
            0.5626, 0.3520, 0.3227, 0.8284, 0.0965, 0.5794, 0.0522, 0.1627, 0.9507,
            0.8736, 0.9385, 0.0475, 0.5115, 0.6523, 0.4838, 0.9199, 0.8586, 0.9675,
            0.4444])), ('hidden1_bn.bias', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden1_bn.running_mean', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden1_bn.running_var', tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])), ('hidden1_bn.num_batches_tracked', tensor(0)), ('hidden2_bn.weight', tensor([0.0080, 0.7764, 0.8470, 0.8384, 0.1727, 0.5827, 0.3746, 0.9699, 0.1763,
            0.4227, 0.2199, 0.0308, 0.1622, 0.1386, 0.0721, 0.1250, 0.8746, 0.5801,
            0.5857, 0.3904, 0.4019, 0.9434, 0.2623, 0.0735, 0.4327, 0.2965, 0.7626,
            0.8738, 0.0899, 0.0639, 0.7353, 0.5767, 0.5398, 0.1324, 0.8093, 0.0025,
            0.6160, 0.2558, 0.9963, 0.1607, 0.3707, 0.1134, 0.2655, 0.9932, 0.1528,
            0.6505, 0.1393, 0.4991, 0.6387, 0.9305, 0.9338, 0.0969, 0.1521, 0.7443,
            0.5617, 0.5793, 0.7303, 0.3408, 0.1844, 0.3070, 0.8134, 0.2563, 0.7673,
            0.9219, 0.5718, 0.9630, 0.1569, 0.9759, 0.4024, 0.3746, 0.9691, 0.8241,
            0.0731, 0.2630, 0.7671, 0.0594, 0.7128, 0.5354, 0.1626, 0.4043, 0.4170,
            0.8843, 0.6048, 0.5487, 0.1190, 0.9286, 0.5068, 0.9690, 0.6766, 0.1146,
            0.5533, 0.6867, 0.2475, 0.3321, 0.3633, 0.2163, 0.5001, 0.8818, 0.5112,
            0.7430])), ('hidden2_bn.bias', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden2_bn.running_mean', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden2_bn.running_var', tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])), ('hidden2_bn.num_batches_tracked', tensor(0)), ('hidden3_bn.weight', tensor([0.1546, 0.0437, 0.1645, 0.9504, 0.3686, 0.8011, 0.8887, 0.4545, 0.1227,
            0.6376, 0.0423, 0.4627, 0.6473, 0.6717, 0.2636, 0.2921, 0.4598, 0.0062,
            0.0241, 0.8096, 0.2212, 0.2601, 0.5115, 0.9143, 0.6449, 0.3428, 0.9887,
            0.1188, 0.0889, 0.8317, 0.5211, 0.8097, 0.5319, 0.8839, 0.3562, 0.7626,
            0.8409, 0.6521, 0.4797, 0.5439, 0.2615, 0.9632, 0.8309, 0.9491, 0.0033,
            0.3386, 0.5573, 0.6740, 0.8810, 0.7728, 0.3341, 0.7041, 0.4257, 0.6194,
            0.6305, 0.2669, 0.3417, 0.1234, 0.0237, 0.7384, 0.5803, 0.6837, 0.1358,
            0.1136, 0.2143, 0.7967, 0.0320, 0.4477, 0.0941, 0.8139, 0.3432, 0.7913,
            0.1458, 0.9725, 0.1054, 0.2943, 0.0988, 0.6605, 0.1120, 0.4030, 0.7504,
            0.8279, 0.9817, 0.8320, 0.3695, 0.9013, 0.6641, 0.9816, 0.2163, 0.0487,
            0.0324, 0.1352, 0.7339, 0.8152, 0.3193, 0.9740, 0.0833, 0.0827, 0.5218,
            0.0251])), ('hidden3_bn.bias', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden3_bn.running_mean', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0.])), ('hidden3_bn.running_var', tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])), ('hidden3_bn.num_batches_tracked', tensor(0))])

```python
# initailze model by saved parameters
new_model = FeedForwardNeuralNetwork(input_size, hidden_size, output_size)
new_model.load_state_dict(saved_parametes)
```

## 作业 4
使用 test_epoch 函数，预测new_model在test_loader上的accuracy和loss

```python
# TODO

# test your model prediction performance

# new_test_loss, new_test_accuracy = test_epoch(test_loader, new_model, loss_fn)
# message = 'Average loss: {:.4f}, Accuracy: {:.4f}'.format(new_test_loss, new_test_accuracy)
# print(message)
```

## 4. Training Advanced

### 4.1 l2_norm
we could minimize the regularization term below by use $weight\_decay$ in **SGD optimizer**
\begin{equation}
    L\_norm =  {\sum_{i=1}^{m}{\theta_{i}^{2}}}
\end{equation}

set l2_norm=0.01, let's train and see

```python
### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0.01 # use l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

    Epoch: 1/5. Train set: Average loss: 1.9009, Accuracy: 74.3017
    Epoch: 1/5. Validation set: Average loss: 74.4788, Accuracy: 74.6000
    Epoch: 2/5. Train set: Average loss: 0.6102, Accuracy: 87.1717
    Epoch: 2/5. Validation set: Average loss: 34.3180, Accuracy: 87.7700
    Epoch: 3/5. Train set: Average loss: 0.3998, Accuracy: 89.2317
    Epoch: 3/5. Validation set: Average loss: 28.3469, Accuracy: 89.4900
    Epoch: 4/5. Train set: Average loss: 0.3501, Accuracy: 90.2517
    Epoch: 4/5. Validation set: Average loss: 25.7806, Accuracy: 90.4200
    Epoch: 5/5. Train set: Average loss: 0.3222, Accuracy: 90.9783
    Epoch: 5/5. Validation set: Average loss: 23.9897, Accuracy: 91.0600

# 作业 5
思考正则项在loss中占比的影响。使用 l2_norm = 1, 训练模型

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。

```python
# TODO

### Hyper parameters

# declare a model

# l2_norm can be done in SGD
```

```python
# TODO

# Train
```

### 4.2 dropout
During training, randomly zeroes some of the elements of the input tensor with probability p using samples from a Bernoulli distribution.

Each channel will be zeroed out independently on every forward call.

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。

```python
### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # without using l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
# Set dropout to True and probability = 0.5
model.set_use_dropout(True)
```

```python
train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

    Epoch: 1/5. Train set: Average loss: 1.8824, Accuracy: 80.4967
    Epoch: 1/5. Validation set: Average loss: 69.8040, Accuracy: 81.0000
    Epoch: 2/5. Train set: Average loss: 0.7348, Accuracy: 87.4400
    Epoch: 2/5. Validation set: Average loss: 33.8798, Accuracy: 87.6500
    Epoch: 3/5. Train set: Average loss: 0.5086, Accuracy: 89.5917
    Epoch: 3/5. Validation set: Average loss: 27.1355, Accuracy: 89.5500
    Epoch: 4/5. Train set: Average loss: 0.4285, Accuracy: 90.8233
    Epoch: 4/5. Validation set: Average loss: 23.7725, Accuracy: 90.8500
    Epoch: 5/5. Train set: Average loss: 0.3815, Accuracy: 91.8033
    Epoch: 5/5. Validation set: Average loss: 21.2870, Accuracy: 91.8900

### 4.3 batch_normalization
Batch normalization is a technique for improving the performance and stability of artificial neural networks

\begin{equation}
    y=\frac{x-E[x]}{\sqrt{Var[x]+\epsilon}} * \gamma + \beta,
\end{equation}

$\gamma$ and $\beta$ are learnable parameters

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。

```python
### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # without using l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
model.set_use_bn(True)
```

```python
model.use_bn
```

    True

```python
train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

    Epoch: 1/5. Train set: Average loss: 1.0694, Accuracy: 91.1317
    Epoch: 1/5. Validation set: Average loss: 35.9896, Accuracy: 91.4500
    Epoch: 2/5. Train set: Average loss: 0.3393, Accuracy: 94.3667
    Epoch: 2/5. Validation set: Average loss: 19.5889, Accuracy: 94.1300
    Epoch: 3/5. Train set: Average loss: 0.2161, Accuracy: 95.6933
    Epoch: 3/5. Validation set: Average loss: 14.6173, Accuracy: 95.3200
    Epoch: 4/5. Train set: Average loss: 0.1645, Accuracy: 96.5617
    Epoch: 4/5. Validation set: Average loss: 12.0746, Accuracy: 95.8500
    Epoch: 5/5. Train set: Average loss: 0.1339, Accuracy: 97.1417
    Epoch: 5/5. Validation set: Average loss: 10.6426, Accuracy: 96.2500

### 4.4 data augmentation
data augmentation can be more complicated to gain a better generalization on test dataset

```python
# only add random horizontal flip
train_transform_1 = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])

# only add random crop
train_transform_2 = transforms.Compose([
    transforms.RandomCrop(size=[28,28], padding=4),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])

# add random horizontal flip and random crop
train_transform_3 = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.RandomCrop(size=[28,28], padding=4),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])
```

```python
# reload train_loader using trans

train_dataset_1 = torchvision.datasets.MNIST(root='./data',
                            train=True,
                            transform=train_transform_1,
                            download=False)

train_loader_1 = torch.utils.data.DataLoader(dataset=train_dataset_1,
                                           batch_size=batch_size,
                                           shuffle=True)
```

```python
print(train_dataset_1)
```

    Dataset MNIST
        Number of datapoints: 60000
        Split: train
        Root Location: ./data
        Transforms (if any): Compose(
                                 RandomHorizontalFlip(p=0.5)
                                 ToTensor()
                                 Normalize(mean=(0.1307,), std=(0.3081,))
                             )
        Target Transforms (if any): None

```python
### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # without using l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
train_accs, train_losses = fit(train_loader_1, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

    Epoch: 1/5. Train set: Average loss: 2.0063, Accuracy: 68.9483
    Epoch: 1/5. Validation set: Average loss: 91.0255, Accuracy: 69.8400
    Epoch: 2/5. Train set: Average loss: 0.7965, Accuracy: 79.6317
    Epoch: 2/5. Validation set: Average loss: 49.2182, Accuracy: 80.1000
    Epoch: 3/5. Train set: Average loss: 0.5844, Accuracy: 82.9250
    Epoch: 3/5. Validation set: Average loss: 40.5368, Accuracy: 83.4900
    Epoch: 4/5. Train set: Average loss: 0.5082, Accuracy: 85.2550
    Epoch: 4/5. Validation set: Average loss: 35.7995, Accuracy: 85.8400
    Epoch: 5/5. Train set: Average loss: 0.4541, Accuracy: 86.4417
    Epoch: 5/5. Validation set: Average loss: 32.2371, Accuracy: 87.0600

## 作业 6
使用提供的train_transform_2, train_transform_3，重新加载train_loader，并且使用fit进行训练

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。注意到此处用的是默认初始化。

```python
# TODO

# train_transform_2
```

```python
# TODO

# train_transform_3
```

## 5. Visualizatio of training and validation phase
We could use tensorboard to visualize our training and test phase.
You could find example [here](https://github.com/lanpa/tensorboardX)

## 6. Gradient explosion and vanishing
We have embedded code which shows grad for hidden2 and hidden3 layer. By observing their grad changes, we can
see whether gradient is normal or not.

For plot grad changes, you need to **set get_grad=True** in **fit function**

```python
### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)
```

### 6.1.1 Gradient Vanishing

Set learning=e-10

```python
### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 1e-20
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad=get_grad)
```

# 6.1.2 Gradient Explosion

#### 6.1.2.1 learning rate
set learning rate = 10

```python
### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 1.0168
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # not to use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad=True)
```

#### 6.1.2.2 normalization for input data

#### 6.1.2.3 unsuitable weight initialization

```python
### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 1
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # not to use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)
```

```python
# reset parameters as 10
def wrong_weight_bias_reset(model):
    """Using normalization with mean=0, std=1 to initialize model's parameter
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # initialize linear layer with mean and std
            mean, std = 0, 1

            # Initialization method
            torch.nn.init.normal_(m.weight, mean, std)
            torch.nn.init.normal_(m.bias, mean, std)
```

```python
wrong_weight_bias_reset(model)
show_weight_bias(model)
```

```python
fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad=True)
```

## References
1. [Training a Classifier](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)
2. [Save Model and Load Model](https://pytorch.org/tutorials/beginner/saving_loading_models.html)
3. [Visualize your training phase](https://github.com/lanpa/tensorboardX)
4. [Exploding and Vanishing Gradients](http://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/readings/L15%20Exploding%20and%20Vanishing%20Gradients.pdf)
5. [Gradient disappearance and gradient explosion in neural network training](https://bzdww.com/article/19659/)
6. [tensorboardX](https://github.com/lanpa/tensorboardX)