# Characterization of Discrete Systems in the Time Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Impulse Response

The concept of the so-called impulse response of discrete linear time-invariant (LTI) system and its connection to linear difference equations is introduced in the following.

### Output Signal

The response $y[k] = \mathcal{H} \{ x[k] \}$ of an LTI system to an arbitrary input signal $x[k]$ is derived. By applying the [sifting-property of the discrete Dirac impulse](../discrete_signals/standard_signals.ipynb#Dirac-Impulse), the input signal can be represented as

\begin{equation}
x[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \delta[k-\kappa]
\end{equation}

The output signal of the system is given by

\begin{equation}
y[k] = \mathcal{H} \left\{ \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \delta[k-\kappa] \right\}
\end{equation}

The summation and system response operator $\mathcal{H}$ can be exchanged under the assumption that the system is linear

\begin{equation}
y[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \mathcal{H} \left\{ \delta[k-\kappa] \right\}
\end{equation}

where $\mathcal{H} \{\cdot\}$ was only applied to the Dirac impulse, since $x[\kappa]$ can be regarded as constant factor with respect to the index $k$.

The response of a system to a Dirac impulse as input is termed [*impulse response*](https://en.wikipedia.org/wiki/Impulse_response). It is defined as

\begin{equation}
h[k] = \mathcal{H} \left\{ \delta[k] \right\}
\end{equation}

If the system is time-invariant, the response to a shifted Dirac impulse is $\mathcal{H} \left\{ \delta[k-\kappa]) \right\} = h[k-\kappa]$. Hence, for a discrete LTI system we finally get

\begin{equation}
y[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot h[k-\kappa] = y[k] = x[k] * h[k]
\end{equation}

This operation is termed as linear [*convolution*](https://en.wikipedia.org/wiki/Convolution) and commonly abbreviated by $*$. The properties of an LTI system are entirely characterized by its impulse response. The response $y[k]$ of a system to an arbitrary input signal $x[k]$ is given by the convolution of the input signal $x[k]$ with its impulse response $h[k]$.

### Relation to Difference Equation

The impulse response $h[k] = \mathcal{H} \{ \delta[k] \}$ is the response of an LTI system to an Dirac impulse at the input. It can be derived from the coefficients of a [linear differential equation representing the LTI system](difference_equation.ipynb) by computing the output signal for the input signal $x[k] = \delta[k]$. Introducing this into the [solution of the difference equation](difference_equation.ipynb#Computation-of-the-Output-Signal) yields

\begin{equation}
h[k] = \frac{1}{a_0} \left( \sum_{m=0}^{M} b_m \; \delta[k-m] - \sum_{n=1}^{N} a_n \; h[k-n] \right)
\end{equation}

### Finite Impulse Response

Let's consider the case of a [non-recursive system](difference_equation.ipynb#Recursive-and-Non-Recursive-Systems) with $a_n = 0$ for $n > 0$. Without loss of generality it can be assumed that $a_0 = 1$, since $\frac{1}{a_0}$ can be incorporated into the other coefficients by dividing them through $a_0$. The impulse response is given as

\begin{equation}
h[k] = \sum_{m=0}^{M} b_m \; \delta[k-m] = \begin{cases} b_k & \text{for } 0 \leq k < M \\ 0 & \text{otherwise} \end{cases}
\end{equation}

Note that the summation in above formula constitutes a convolution between the signal given by the samples $b_m$ and the Dirac impulse $\delta[k]$. The impulse response of a non-recursive system is of finite length $M$. Its values are given by the coefficients $b_m$ of the linear difference equation characterizing the system. An impulse response of finite length is commonly termed [finite impulse response (FIR)](https://en.wikipedia.org/wiki/Finite_impulse_response). The term FIR (system) is used synonymously to non-recursive system. The former relates to the length of the impulse response and the latter to the structure of the system.

**Example - Moving Average**

According to above findings, the impulse response of the [moving average filter](difference_equation.ipynb#Moving-Average) is given as

\begin{equation}
h[k] = \frac{1}{N} \cdot \text{rect}_N[k]
\end{equation}

As alternative to the [solution of the difference equation illustrated before](difference_equation.ipynb#Moving-Average), the output signal $y[k] = \mathcal{H} \{ y[k] \}$ is computed by convolution of the (same) input signal.

```python
import numpy as np
from scipy import signal
import matplotlib.pyplot as plt
%matplotlib inline

def rect(k, N):
    return np.where((k >= 0) & (k < N), 1.0, 0.0)

np.random.seed(seed=0)

N = 10

k = np.arange(0, 60)
h = 1/N * rect(k, N)
x = np.cos(2*np.pi/30 * k) + .2 * np.random.normal(size=(len(k)))
y = np.convolve(x, h)

plt.figure(figsize=(10, 4))
plt.subplot(121)
plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel(r'$x[k]$')
plt.ylim([-1.5, 1.5])

plt.subplot(122)
plt.stem(k, y[0:len(x)])
plt.xlabel('$k$')
plt.ylabel('$y[k]$')
plt.ylim([-1.5, 1.5])
plt.tight_layout()
```

**Exercise**

* Compare above output signal $y[k]$ derived by convolution with the output signal derived by [solution of the difference equation](difference_equation.ipynb#Moving-Average).

### Infinite Impulse Response

Now the general case of a [recursive system](difference_equation.ipynb#Recursive-and-Non-Recursive-Systems) is regarded. By inspection of above formula for the computation of the impulse response from the coefficients of a linear difference equation it becomes clear that the impulse response $h[k]$ for time-instant $k$ depends on the impulse response of past time instants $k-1, \dots, k-N+1$. This feedback generally results in an impulse response of infinite length. Such an impulse response is commonly termed as [infinite impulse response (IIR)](https://en.wikipedia.org/wiki/Infinite_impulse_response). The term IIR (system) is used synonymously to recursive system. The former relates to the length of the impulse response and the latter to the structure of the system.

For a finite-length input signal $x[k]$ the output signal of a recursive system cannot be computed by a linear convolution $y[k] = x[k] * h[k]$ in practice. This is due to the infinite length of its impulse response. As practical solution, the impulse response is often truncated to a finite length after its has decayed reasonably.

**Example**

The impulse response $h[k]$ of the previously introduced [second-order recursive LTI system](difference_equation.ipynb#Second-Order-System) with the difference equation

\begin{equation}
y[k] - y[k-1] + \frac{1}{2} y[k-2] = x[k]
\end{equation}

is derived by solution of the difference equation for a Dirac impulse as input signal. As can be deduced from its difference equation, the second-order recursive system has an IIR. Its first 256 samples are computed and plotted in the following. Note, less samples have been shown for ease of illustration.

```python
def dirac(k):
    return np.where(k == 0, 1.0, 0.0)

a = [1.0, -1.0, 1/2]
b = [1.0]

k = np.arange(256)
x = dirac(k)
h = signal.lfilter(b, a, x)

plt.figure(figsize=(6, 3))
plt.stem(k, h)
plt.xlabel('$k$')
plt.ylabel(r'$h[k]$')
plt.axis([0, 25, -.5, 1.2])
```

    [0, 25, -0.5, 1.2]

In order to illustrate the amplitude decay of the impulse response over a wider range, its magnitude $A[k]$ is plotted in [decibel (dB)](https://en.wikipedia.org/wiki/Decibel)

\begin{equation}
A[k] = 20 \cdot \log_{10} ( |h[k]| ) \quad \text{in dB}
\end{equation}

```python
import warnings
warnings.filterwarnings('ignore', 'divide by zero encountered in log10')

plt.figure(figsize=(10, 4))
plt.stem(k, 20*np.log10(np.abs(h)))
plt.xlabel('$k$')
plt.ylabel(r'$|h[k]|$ in dB')
plt.axis([0, k[-1], -800, 0])
plt.grid()
```

Its obvious that the magnitude of the impulse response has decayed to quite small values after 256 samples. The truncated impulse response can be used to calculate the output signal $y[k]$ by convolution with the input signal $x[k] = \text{rect}_{20}[k]$. The resulting output signal is plotted for illustration.

```python
def rect(k, N):
    return np.where((k >= 0) & (k < N), 1.0, 0.0)

x = rect(k, 20)
y_ir = np.convolve(h, x)

plt.figure(figsize=(6, 3))
plt.stem(k, y_ir[:len(k)])
plt.xlabel('$k$')
plt.ylabel(r'$y[k]$')
plt.axis([0, 40, -.7, 2.6])
```

    [0, 40, -0.7, 2.6]

**Exercise**

* Compare the output signal computed by convolution with the one [computed by solution of the difference equation](difference_equation.ipynb#Second-Order-System).

* Investigate the effect of truncating an IIR on the output signal computed by convolution

    * Split the IIR $h[k]$ into two parts: a first part holding the samples of the truncated impulse response and second part holding the remaining samples
    * Split the output signal into to parts by splitting the convolution using the splitted impulse response

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


```python
from IPython.display import display
from sympy import *
```

# 前置知识

理解这份笔记的内容需，读者需要具备基础的python知识并且对函数，微积分和矩阵有一定的基础。

# 辅助函数

由于后面的笔记中， 我们会大量将一个Sympy object和应用某个函数之后，调用某个方法之后， 或是和执行计算之后的结果比较。 为了减少代码的重复，我编写了helper.py帮忙做这事。

```python
from helper import comparator_factory

func_comparator = comparator_factory('应用函数{}前：','使用后:')
```

```comparator_factory```返回的comparator是一个函数, 他否则使用的语法是```comparator(target, func, *args, *kwargs)```。 ```target```是待处理的Sympy对象, ```func```是希望应用的Sympy函数,```args```,```kwargs```是传给```func```的额外位置参数和关键字参数。

```python
from helper import comparator_method_factory
method_comparator = comparator_method_factory('Before calling {}：','After:')
```

```comparator_method_factory```返回的使用的语法是```comparator(target, method_name, *args, *kwargs)```。 ```target```是待处理的Sympy对象, ```method_name```是希望调用的方法的名字,```args```,```kwargs```是传给调用方法的额外位置参数和关键字参数。

```python
from helper import comparator_eval_factory

eval_comparator = comparator_eval_factory('计算前：','计算后')
```

```comparator_eval_factory```返回的```comparator```使用的语法是```comparator(uneval)```。 ```uneval```是未执行计算的Sympy object。

# 符号计算是什么？
Sympy能以符号形式进行数学计算。数学表达式中未经计算的变量可以以符号的形式存在。我们看下面的例子。首先，我们用python的内置函数计算开方，我们可能会这么做。

```python
from math import sqrt

sqrt(8)
```

    2.8284271247461903

这个结果并不能精确的表达$\sqrt{8}$而且我们也很难从这一大串float数值推出原来的表达式是什么。这就是为什么我们需要符号计算。在像Sympy这样的符号计算系统中,不能准确表达的开发运算会保留未经计算的符号形态。

```python
from sympy import sqrt

sqrt(8)
```

    2*sqrt(2)

# 更好的显示效果

上面的例子中，结果很棒。但是在Jupyter中的显示效果看起来并不怎么样。如果我们要更好的显示效果，可以调用```sympy.init_printing()```方法

```python
from sympy import init_printing
init_printing()
```

```python
sqrt(8)
```

看上去棒极了!!

# 对变量进行符号计算

Sympy能够对包含符号变量的表达式进行计算。下面是个例子。

```python
from sympy import symbols
x, y = symbols('x y')

expr = x + 2*y

expr
```

Sympy能自动应用一些明显的化简。 所以下面的例子里，我们得到的结果是$y$而不是$x+2y-x-y$

```python
expr-x-y
```

如果没有像Sympy这样的符号计算系统的帮助，我们是实现不了这样的效果的。因为大部分情况下，编程语言都没法去处理一个没有赋上具体值的变量。

# Sympy的效果演示

为了满足你的好奇心，下面挑了一小部分例子，演示Sympy在符号计算的威力。 先创建一写符号变量。

```python
x, t, z, nu = symbols('x t z nu')
```

## 求微分

计算$\sin{(x)}e^x$的微分

```python
s = Derivative(sin(x)*exp(x),x)

eval_comparator(s)
```

## 求积分

计算$\int(e^x\sin{(x)} + e^x\cos{(x)})\,dx$

```python
s = Integral(exp(x)*sin(x) + exp(x)*cos(x))

eval_comparator(s)
```

计算$\int_{-\infty}^\infty \sin{(x^2)}\,dx$

```python
s = Integral(sin(x**2),(x,-oo,oo))

eval_comparator(s)
```

计算$\lim_{x\to 0}\frac{\sin{(x)}}{x}$

## 求极限

```python
s = Limit(sin(x)/x, x, 0)

eval_comparator(s)
```

求解 $x^2 - 2 = 0$

## 求解等式

```python
s = Eq(x**2 - 2, 0)

func_comparator(s, solve, x)
```

## 求微分方程

计算微分方程$y'' - y = e^t$

```python
y = Function('y')

s = Eq(y(t).diff(t, t) - y(t), exp(t))

func_comparator(s, dsolve, y(t))
```

## 矩阵计算

计算$\left[\begin{smallmatrix}1 & 2\\2 &
2\end{smallmatrix}\right]$的engenvalue

```python
Matrix([[1, 2], [2, 2]]).eigenvals()
```

用spherical Bessel function jν(z)改写Bessel function Jν(z)

## 作图

画函数图像

```python
expr_1 = x**2
range_1 = (x,-2,2)

expr_2 = x
range_2 = (x,-1,1)

p = plot(
    (expr_1,range_1),
    (expr_2,range_2),
    show = False,
    legend = True
);

p[0].line_color = 'r'
p[1].line_color = 'b'

p[0].label = 'Line 1'
p[1].label = 'Line 2'

p.show()
```

画3D平面

```python
from sympy.plotting import plot3d

x,y = symbols('x y')

plot3d(
    (x**2 + y**2, (x, -5, 5), (y, -5, 5)),
    (x*y, (x, -3, 3), (y, -3, 3))
);
```

## 定义符号

## 定义变量符号

可以用symbols()去一次性定义多个符号变量。将想要的符号用空格隔开，以字符串的方式传入函数。

```python
from sympy import symbols
```

```python
x,y,z = symbols('x y z')

expr = (x+y)**z/(x+1-y/5)

expr
```

除了用字母，也可以用单词作为符号名称。

```python
speed,time = symbols('speed time')
display(speed, time)
```

有些字符串是保留字，用于定义诸如$\lambda, \nu$等特殊的符号。

```python
lamda, n = symbols('lamda nu')

display(lamda, n)
```

Python变量名不一定要和符号名保持一致。

```python
y, x = symbols('x y')

display(x, y)
```

但是为了避免一些不必要的混乱，建议还是让Python变量名和符号名保持一致比较好。

## 定义函数符号

传入```cls = Function```定义函数符号。

```python
f, g = symbols('f g', cls=Function)

display(f(x))

display(g(x,y))
```

## 定义数字

可以使用Integer, Float, Rational去定义Sympy中的整数，浮点数和有理数。

```python
from sympy import Integer, Float, Rational
```

```python
i = Integer(1)

i
```

```python
f = Float(2.31)

f
```

```python
r = Rational(2,7)

r
```

# 定义表达式
## 基本表达式

基本的数学表达式用符号变量和python的运算符就够构造。

```python
x,y,z = symbols('x y z')

expr = (x+y)**z/(x+1-y/5)

expr
```

当Python的objects和Sympy的objects相遇的时候，Python objects会被自动转化成Sympy objects.所以大部分使用我们直接使用python内置的数。

但是,遇到两个python数值相除的时候，Python会先完成除法运算，将有理数转变成浮点数。

```python
expr = x+1/2

expr
```

所以如果我们需要使用有理数，需要显示的去进行定义。

```python
expr = x+Rational(1,2)

expr
```

更复杂的表达式例如微积分需要借助Sympy的函数来实现。这部分内容会在后面的教程中介绍。

## 定义等式
可以用```Eq```来定义等式。

```python
x,y = symbols('x,y')

eq = Eq(x**2-x,0)

print('等式n:')
display(eq)
```

# 处理表达式

## 多项/有理函数

### simplify()
Sympy提供了多种函数用于表达式的化简。simplify()是一个通用的函数，它尝试“智能”的应用所有这些函数，让表达式达到一个“最简化”的状态。

下面是一些例子。

```python
expr = sin(x)**2 + cos(x)**2
func_comparator(expr, simplify)
```

```python
expr = (x**3 + x**2 - x - 1)/(x**2 + 2*x + 1)

func_comparator(expr, simplify)
```

```python
expr = gamma(x)/gamma(x - 2)

func_comparator(expr, simplify)
```

由于很难定义什么是"最简化"，所以你可能得不到你想要的结果。

```python
expr = x**2 + 2*x + 1

func_comparator(expr, simplify)
```

在上面的例子里。你可能觉得$(x+1)^2$是最简化的结果但是simplify并不同意。 这种时候，我们就要使用更加具体的化简函数去更好的控制结果。这些函数后面会介绍。

除此之外，由于simplify()需要把各种各样的化简都尝试一下才能决定哪种方案最好，处理速度会慢。所以如果你已经知道自己想要哪种类型的化简，直接使用特定的函数就好。

### expand()
如果传入一个多项式， expand()会把它处理成由单项式的和构成的标准型。

```python
from sympy import expand

expr = (x + 1)**2

func_comparator(expr,expand)
```

```python
expr = (x + 1)*(x - 2) - (x - 1)*x
func_comparator(expr,expand)
```

### factor()
factor()将表达式在有理数范围内分解成不可约的因子项。

```python
from sympy import factor

expr = (x**2)*z + 4*x*y*z + 4*y**2*z

func_comparator(expr, factor)
```

### factor_list()
factor_list()做和factor()一样的工作,但是返回的结果不可约因子项组成的list。

```python
from sympy import factor_list

expr = x**2*z + 4*x*y*z + 4*y**2*z

func_comparator(expr, factor_list)
```

### collect()
collect()对表达式进行同类项合并。

```python
from sympy import collect

expr = x*y + x - 3 + 2*x**2 - z*x**2 + x**3

func_comparator(expr, collect, x)
```

### cancel()
cancel()接受有理函数，然后处理成$p/q$的标准型。做到

* $p$和$q$是展开的多项式，没有未合并的同类项。
* $p$和$q$的第一个系数不包含分母。

```python
from sympy import cancel

expr = (x*y**2 - 2*x*y*z + x*z**2 + y**2 - 2*y*z + z**2)/(x**2 - 1)

func_comparator(expr, cancel)
```

### apart
apart()对有理函数进行部分分式分解。它将原表达式表示成若干多项式和若干分母较简单的分式的和。

```python
from sympy import apart

expr = (4*x**3 + 21*x**2 + 10*x + 12)/(x**4 + 5*x**3 + 5*x**2 + 4*x)

func_comparator(expr, apart)
```

## 三角函数

### trigsimp
要根据三角恒等式对三角函数进行化简的话，可以用trigsimp()。和simplify()很像，trigsimp()尝试使用各种三角恒等式去处理接受的表达式，然后根据“直觉”找到最好的选择。

```python
from sympy import trigsimp

expr = sin(x)**4 - 2*cos(x)**2*sin(x)**2 + cos(x)**4

func_comparator(expr, trigsimp)
```

trigsimp()也能用在双曲函数上。

```python
expr = cosh(x)**2 + sinh(x)**2

func_comparator(expr, trigsimp)
```

### expand_trig
如果想展开三角函数，例如，想利用和角公式和倍角公式的话，可以用expand_trig()。

```python
from sympy import expand_trig

expr = sin(x + y)

func_comparator(expr,expand_trig)
```

## 幂函数

### 假设

介绍针对指数函数的化简函数之前，得先讨论一下和指数有关的几个等式。

我们有三个等式。

1. $x^ax^b = x^{a + b}$
2. $x^ay^a = (xy)^a$
3. $(x^a)^b = x^{ab}$

等式1总是成立。

等式2不总是成立。我们可以举一个针对等式2的反例。

如果$x=y=−1$ and $a=1/2$, 那么$x^ay^a = \sqrt{-1}\sqrt{-1} = i\cdot i = -1$, 可是$x^ay^a = \sqrt{-1}\sqrt{-1} = i\cdot i = -1$.

等式3也不是一直成立。例如， 如果$x=−1$, $a=2$, and $b=1/2$, 那么$(x^a)^b = {\left ((-1)^2\right )}^{1/2} = \sqrt{1} = 1$且有$x^{ab} = (-1)^{2\cdot1/2} = (-1)^1 = -1$

记得这些很重要，因为默认情况下，Sympy并不会利用并不总是成立的等式用于化简操作。

但是我们可以添加额外的假设条件，让等式2和等式3在这些假设条件下做到衡成立。

一套让等式2满足的条件是，$x, y \geq 0$ and $a \in \mathbb{R};一套让等式3满足的条件是$b \in \mathbb{Z}$

为了让Sympy利用这些只有在特定假设下才成立的等式进行化简，我们需要给符号添加假设(默认假设是它们都是复数)。

我们后面会对假设系统进行更细致的探讨。下面先举一个简单的用法的例子。这个例子里，我们假设$x,y$值为正且$a,b$是实数。

```python
x, y = symbols('x y', positive=True)
a, b = symbols('a b', real=True)
```

另一个强制进行化简，无视假设的方法是传入```force = True```。这个用法我们后面会遇到。

### powsimp
powsimp()会从左到右应用等式1和2.

```python
from sympy import powsimp
expr = x**a*x**b
func_comparator(expr, powsimp)
```

```python
from sympy import powsimp
expr = x**a*y**a
func_comparator(expr, powsimp)
```

如果没有相应的假设让等式2成立，化简不会发生。

```python
x, y = symbols('x y')
a, b = symbols('a b')

from sympy import powsimp

expr = x**a*y**a
func_comparator(expr, powsimp)
```

如果你确信希望应用化简，无论假设条件如何，可以传入```force=True```

```python
x, y = symbols('x y')
a, b = symbols('a b')

from sympy import powsimp
expr = x**a*y**a
func_comparator(expr, powsimp,force=True)
```

### expand_power_exp
expand_power_exp()从右往左应用等式1。

```python
from sympy import expand_power_exp
expr = x**(a + b)
func_comparator(expr, expand_power_exp)
```

### expand_power_base
expand_power_base()从左到右应用等式2.

```python
from sympy import expand_power_base
expr = (x*y)**a
func_comparator(expr, expand_power_base)
```

### powdenest
powdenest()从左往右应用等式3。

```python
from sympy import powdenest
expr = (x**a)**b
func_comparator(expr, powdenest,force=True)
```

## 指数函数和对数函数

对数函数有个主要的等式。

1. $\log{(xy)} = \log{(x)} + \log{(y)}$
2. $\log{(x^n)} = n\log{(x)}$

它们有和幂函数一样的问题。为了让化简时能利用上这些等式，我们需要传入```force = True```或者添加额外的假设。

一套充分条件是

```python
x, y = symbols('x y', positive=True)
n = symbols('n', real=True)
```

### expand_log
expand_log()从左往右应用等式1和2

```python
from sympy import expand_log

expr = log(x*y)

func_comparator(expr,expand_log)
```

```python
from sympy import expand_log

expr = log(x**n)

func_comparator(expr,expand_log)
```

```python
from sympy import expand_log

expr = log(x/y)

func_comparator(expr,expand_log)
```

### logcombine
expand_log()从右往左应用等式1和2

```python
from sympy import logcombine

expr = log(x) + log(y)

func_comparator(expr, logcombine)
```

```python
from sympy import logcombine

expr = n*log(x)

func_comparator(expr, logcombine)
```

## 组合函数

## combsimp
要化简组合函数的话，可以用combsimp()

```python
from sympy import combsimp

expr = factorial(n)/factorial(n - 3)

func_comparator(expr, combsimp)
```

```python
from sympy import combsimp, binomial

n,k = symbols('n k')

expr = binomial(n+1, k+1)/binomial(n, k)

func_comparator(expr, combsimp)
```

## 参考资料

(Sympy官方文档)[http://docs.sympy.org/latest/index.html]

# 相关文章
* [Sympy笔记I]({filename}0026_sympy_intro_1_ch.ipynb)
* [Sympy笔记II]({filename}0027_sympy_intro_2_ch.ipynb)
* [Sympy笔记III]({filename}0028_sympy_intro_3_ch.ipynb)
* [Sympy笔记IV]({filename}0029_sympy_intro_4_ch.ipynb)


# 2D Backends

In this tutorial we are going to examine the 2D capabilities of this plotting module, whose major objective is to integrate modern interactive plotting libraries with SymPy. We are also going to illustrate a couple of important capabilities:
1. Correct detection and plotting of singularities.
2. Correct visualization of `Piecewise` expressions.

## Comparison between 2D backends

Let's start by comparing the 2D backends. First, we will show the results with Matplotlib, and explain the limitations. Then we will show what a different backend can do.

```python
%matplotlib widget
from sympy import *
from spb import *
from spb.backends.matplotlib import MB
from spb.backends.plotly import PB
from spb.backends.bokeh import BB
var("u, v, x, y")
```

In the above code cell we first imported every plot function from our custom plotting module. Specifically, we imported `plot, plot_parametric, plot3d, plot3d_parametric_line, plot3d_parametric_surface, plot_contour, plot_implicit`. Remember: while the plot functions from the current module are identical to the ones from `sympy`, they are not compatible when using a different backend!

We also imported three backends. The following backends are available from the following submodules:

|       Sub-module      | Backend             | Alias |
|:---------------------:|:-------------------:|:-----:|
| backends.bokeh      | BokehBackend      |  BB |
| backends.matplotlib | MatplotlibBackend |  MB |
| backends.plotly     | PlotlyBackend     |  PB |
| backends.k3d        | K3DBackend        |  KB |

Only `MatplotlibBackend`, `BokehBackend` and `PlotlyBackend` support 2D plots.

```python
plot(sin(x), cos(x), log(x), backend=MB)
```

Once we plot multiple expression simultaneously, the legend will automatically show up. We can disable it by setting `legend=False`.

Note that:
* In order to interact with it we have to use the buttons on the toolbar.
* If we move the cursor over the plot, we can see its coordinates. With this behaviour we can only get approximate coordinates if we move the cursor over a line.

Let's visualize the same plot with `PlotlyBackend`:

```python
plot(sin(x), cos(x), log(x), backend=PB)
```

We can also interact with this plot using the top toolbar, however there are more natural ways:
* Click and drag to zoom into a rectangular selection.
* Move the cursor in the middle of the horizontal axis, click and drag to pan horizontally.
* Move the cursor in the middle of the vertical axis, click and drag to pan vertically.
* Move the cursor near the ends of the horizontal/vertical axis: click and drag to resize.
* Move the cursor over a line: a tooltip will show the coordinate of that point in the data series. Note that there is no interpolation between two consecutive points.
* Click over a label in the legend to hide/show that data series.

Let's now visualize the same plot with BokehBackend:

```python
plot(sin(x), cos(x), log(x), backend=BB)
```

Here, we can:
* Click and drag to pan the plot around. **Once we are done panning, the plot automatically updates all the data series according to the new range**. This is a wonderful feature of Bokeh, which allows us to type less and explore more.
* Click and drag the axis to pan the plot only on one direction.
* Click the legend entries to hide/show the data series.
* Move the cursor over a line: a tooltip will show the coordinate of that point in the data series.
* Use the toolbar to change the tool, for example we can select the _Box Zoom_ to zoom into a rectangular region.

Is some occasion, it might be helpful to assign a custom label to a specific expression. We can do that in the following way:

```python
p = plot((cos(x), "$f_{1}$"), (sin(x), "$f_{2}$"), (log(x), "$f_{3}$"),
         backend=MB, legend=True, title="Latex Support",
         xlabel=r"$\xi$", ylabel=r"$\eta$", detect_poles=False)
```

At the time of writing this, there might be problems when using [Plotly with Latex on Firefox](https://github.com/plotly/plotly.js/issues/5374).

Also, note that, differently from Matplotlib and Plotly, Bokeh doesn't support Latex!

## Singularity Detection

Another difference in comparison with the SymPy plotting module is that we can chose to correctly plot singularities (or at least try to). By default, this detection is turned off, for example:

```python
plot(tan(x), (x, -10, 10), backend=PB)
```

As we can see, the plot is hardly readable:
* There is a pretty huge data range covered by the y-axis, thus "flattening" the non-linearities.
* There is a continous line that connects the points even through singularities.

We can turn-on the singularity detection by setting `detect_poles=True`:

```python
plot(tan(x), (x, -10, 10), backend=PB, detect_poles=True)
```

Here:
* There is a clear separation where the singularities are located.
* A better y-axis range has been automatically chosen. **Warning**: the algorithm is still at its infancy, so it might be possible that a very small y-range is chosen. In such cases we can re-execute the plot command or we can manually set the `ylim` keyword argument.
* Thanks to the interactive nature of Plotly, we can easily scroll the y-axis in order to explore the data.

It is important to realize that the finer the discretization of the domain, the better the detection. Therefore, it might be necessary to either increase the `depth` of the adaptive algorithm (default value to 9), or set `adaptive=False` and `n=2000` (some arbitrary large number). Also, as a last resort, one might also change the value of `eps` (default to 0.1).

Note that there is a bias in the y-axis: it is not perfectly centered. That's perfectly fine with this initial implementation of the singularity detection. We can easily fix it by scrolling the y-axis or by setting the `ylim` keyword:

```python
plot(tan(x), (x, -10, 10), backend=PB, detect_poles=True, ylim=(-7, 7))
```

## Plotting Piecewise functions

We can also plot `Piecewise` expressions, eventually showing the discontinuities. For example:

```python
pf = Piecewise(
    (sin(x), x < -5),
    (2, Eq(x, 0)),
    (3, Eq(x, 2)),
    (cos(x), (x > 0) & (x < 2)),
    (x / 2, True)
)
display(pf)
plot(pf, backend=MB)
```

As a design choice, the algorithm is going to extract the different pieces and plot them separately. Note that:
* the points are visible too!
* the number on labels refers to the piece being plotted. So, ``P5`` appears two times because that piece covers two different parts of the range.

Let's try to reduce the plotted range:

```python
plot(pf, (x, -3, 1.5), backend=MB)
```

Note that ``P1`` and ``P3`` are not visible because they are outside of the provided range.

When using ``BokehBackend`` to plot piecewise functions, the user should avoid to drag the plot around, as this will recompute the series and override the original ranges, thus leading to an incorrect plot:

```python
plot(pf, (x, -3, 1.5), backend=BB)
```

Now, let's consider a case where this standard behaviour with ``Plotwise`` is going to fail:

```python
f = real_root((log(x / (x - 2))), 3)
display(f)
plot(f)
```

As we can see from the expression, the conditions are not trivial, hence the current implementation is not able to split the sub-expressions. In these occasions we can set ``process_piecewise=False``. This forces the algorithm to consider the piecewise as a single function:

```python
plot(f, process_piecewise=False)
```

## Combining Plots

Let's consider two different plots:

```python
p1 = plot(sin(x), cos(x), backend=MB)
p2 = plot(log(x), exp(-x / 5) * sin(5 * x), (x, 1e-05, 10),
          backend=MB, line_kw=dict(linestyle="--"))
```

In this case, both `p1` and `p2` contains two data series. Note that we used the `line_kw` dictionary to provide custom options that have been passed to Matplotlib. Type `help(MB)` (or any other backend) to discover more customization options.

In case we need to access these data series we can use the index notation:

```python
s1 = p1[0]
type(s1)
```

The data series are responsible to generate the numerical data. We can also extract the numerical data with:

```python
data = s1.get_data()
```

We can combine 2 plot objects in 3 ways:

1. by calling `p1.extend(p2)`: this method copy all the data series from `p2` into `p1`.
2. by calling the `p1.append(p2[idx])`: copy the data series ad index `idx` of `p2` and append it to `p1`.
3. by summing plot objects `p1 + p2`: create a new plot copying all the data series from `p1` and `p2` and also merge the keyword arguments of the two plots.

Let's try the first way. Note that the keyword arguments of `p2` are lost:

```python
p1.extend(p2)
p1.show()
```

Let's append a data series from `p2` into `p1`. In this case too the keyword arguments of `p2` are disregarded:

```python
p1 = plot(sin(x), cos(x), backend=MB, show=False)
p2 = plot(log(x), exp(-x / 5) * sin(5 * x), (x, 1e-05, 10),
          backend=MB, line_kw=dict(linestyle="--"), show=False)
p1.append(p2[1])
p1.show()
```

Finally, let's add the plot objects. In this case, the keyword arguments will get merged:

```python
p1 = plot(sin(x), cos(x), backend=MB, show=False)
p2 = plot(log(x), exp(-x / 5) * sin(5 * x), (x, 1e-05, 10),
          backend=MB, line_kw=dict(linestyle="--"), show=False)
p3 = (p1 + p2)
p3.show()
```

As you can see, the keyword arguments have been merged, thus every data series is affected by them!

## Saving Plots

Generally, there are two ways to save a plot:

1. Manually, by clicking the save button in the toolbar. For Matplotlib, this only works if the magic line `%matplotlib widget` has been executed.
2. Programmatically, by calling the `save` method of a plot object. This method is just a wrapper to the `save` method exposed by the actual plotting library, therefore we can save jpg, png, pdf, svg or html files if the library supports these functionality.

Note that some backends requires additional dependencies to be installed in order to export pictures. Run the following command and follow the links in the Reference section to learn more.

```python
help(MB.save)
```

```python
help(PB.save)
```

```python
help(BB.save)
```

```python

```


# Programming Exercise 4:  Neural Networks Learning

## Introduction

In this exercise, you will implement the backpropagation algorithm for neural networks and apply it to the task of hand-written digit recognition. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.

All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).

Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments).

```python
# used for manipulating directory paths
import os

# Scientific and vector computation for python
import numpy as np

# Plotting library
from matplotlib import pyplot

# Optimization module in scipy
from scipy import optimize

# will be used to load MATLAB mat datafile format
from scipy.io import loadmat

# library written for this exercise providing additional functions for assignment submission, and others
import utils

# define the submission/grader object for this exercise
grader = utils.Grader()

# tells matplotlib to embed plots within the notebook
%matplotlib inline
```

## Submission and Grading

After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.

| Section | Part                                             | Submission function | Points
| :-      |:-                                                | :-                  | :-:
| 1       | [Feedforward and Cost Function](#section1)                    | [`nnCostFunction`](#nnCostFunction)   | 30
| 2       | [Regularized Cost Function](#section2)                        | [`nnCostFunction`](#nnCostFunction)   | 15
| 3       | [Sigmoid Gradient](#section3)                                 | [`sigmoidGradient`](#sigmoidGradient) | 5
| 4       | [Neural Net Gradient Function (Backpropagation)](#section4)   | [`nnCostFunction`](#nnCostFunction)   | 40
| 5       | [Regularized Gradient](#section5)                             | [`nnCostFunction`](#nnCostFunction)   |10
|         | Total Points                                     |    | 100

You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.

At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.

## Neural Networks

In the previous exercise, you implemented feedforward propagation for neural networks and used it to predict handwritten digits with the weights we provided. In this exercise, you will implement the backpropagation algorithm to learn the parameters for the neural network.

We start the exercise by first loading the dataset.

```python
#  training data stored in arrays X, y
data = loadmat(os.path.join('data',ex4data1.mat'))
X, y = data['X'], data['y'].ravel()

# set the zero digit to 0, rather than its mapped 10 in this dataset
# This is an artifact due to the fact that this dataset was used in
# MATLAB where there is no index 0
y[y == 10] = 0

# Number of training examples
m = y.size
print(X.shape)
print(y.shape)
```

    (5000, 400)
    (5000,)

### 1.1 Visualizing the data

You will begin by visualizing a subset of the training set, using the function `displayData`, which is the same function we used in Exercise 3. It is provided in the `utils.py` file for this assignment as well. The dataset is also the same one you used in the previous exercise.

There are 5000 training examples in `ex4data1.mat`, where each training example is a 20 pixel by 20 pixel grayscale image of the digit. Each pixel is represented by a floating point number indicating the grayscale intensity at that location. The 20 by 20 grid of pixels is “unrolled” into a 400-dimensional vector. Each
of these training examples becomes a single row in our data matrix $X$. This gives us a 5000 by 400 matrix $X$ where every row is a training example for a handwritten digit image.

$$ X = \begin{bmatrix} - \left(x^{(1)} \right)^T - \\
- \left(x^{(2)} \right)^T - \\
\vdots \\
- \left(x^{(m)} \right)^T - \\
\end{bmatrix}
$$

The second part of the training set is a 5000-dimensional vector `y` that contains labels for the training set.
The following cell randomly selects 100 images from the dataset and plots them.

```python
# Randomly select 100 data points to display
rand_indices = np.random.choice(m, 100, replace=False)
sel = X[rand_indices, :]

utils.displayData(sel)
```

### 1.2 Model representation

Our neural network is shown in the following figure.

It has 3 layers - an input layer, a hidden layer and an output layer. Recall that our inputs are pixel values
of digit images. Since the images are of size $20 \times 20$, this gives us 400 input layer units (not counting the extra bias unit which always outputs +1). The training data was loaded into the variables `X` and `y` above.

You have been provided with a set of network parameters ($\Theta^{(1)}, \Theta^{(2)}$) already trained by us. These are stored in `ex4weights.mat` and will be loaded in the next cell of this notebook into `Theta1` and `Theta2`. The parameters have dimensions that are sized for a neural network with 25 units in the second layer and 10 output units (corresponding to the 10 digit classes).

```python
# Setup the parameters you will use for this exercise
input_layer_size  = 400  # 20x20 Input Images of Digits
hidden_layer_size = 25   # 25 hidden units
num_labels = 10          # 10 labels, from 0 to 9

# Load the weights into variables Theta1 and Theta2
weights = loadmat(os.path.join('data',ex4weights.mat'))

# Theta1 has size 25 x 401
# Theta2 has size 10 x 26
Theta1, Theta2 = weights['Theta1'], weights['Theta2']

# swap first and last columns of Theta2, due to legacy from MATLAB indexing,
# since the weight file ex3weights.mat was saved based on MATLAB indexing
Theta2 = np.roll(Theta2, 1, axis=0)

# Unroll parameters
print(Theta1.ravel().shape)
print(Theta2.ravel().shape)
nn_params = np.concatenate([Theta1.ravel(), Theta2.ravel()])
print(nn_params.shape)
```

    (10025,)
    (260,)
    (10285,)

### 1.3 Feedforward and cost function

Now you will implement the cost function and gradient for the neural network. First, complete the code for the function `nnCostFunction` in the next cell to return the cost.

Recall that the cost function for the neural network (without regularization) is:

$$ J(\theta) = \frac{1}{m} \sum_{i=1}^{m}\sum_{k=1}^{K} \left[ - y_k^{(i)} \log \left( \left( h_\theta \left( x^{(i)} \right) \right)_k \right) - \left( 1 - y_k^{(i)} \right) \log \left( 1 - \left( h_\theta \left( x^{(i)} \right) \right)_k \right) \right]$$

where $h_\theta \left( x^{(i)} \right)$ is computed as shown in the neural network figure above, and K = 10 is the total number of possible labels. Note that $h_\theta(x^{(i)})_k = a_k^{(3)}$ is the activation (output
value) of the $k^{th}$ output unit. Also, recall that whereas the original labels (in the variable y) were 0, 1, ..., 9, for the purpose of training a neural network, we need to encode the labels as vectors containing only values 0 or 1, so that

$$ y =
\begin{bmatrix} 1 \\ 0 \\ 0 \\\vdots \\ 0 \end{bmatrix}, \quad
\begin{bmatrix} 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix}, \quad \cdots  \quad \text{or} \qquad
\begin{bmatrix} 0 \\ 0 \\ 0 \\ \vdots \\ 1 \end{bmatrix}.
$$

For example, if $x^{(i)}$ is an image of the digit 5, then the corresponding $y^{(i)}$ (that you should use with the cost function) should be a 10-dimensional vector with $y_5 = 1$, and the other elements equal to 0.

You should implement the feedforward computation that computes $h_\theta(x^{(i)})$ for every example $i$ and sum the cost over all examples. **Your code should also work for a dataset of any size, with any number of labels** (you can assume that there are always at least $K \ge 3$ labels).

**Implementation Note:** The matrix $X$ contains the examples in rows (i.e., X[i,:] is the i-th training example $x^{(i)}$, expressed as a $n \times 1$ vector.) When you complete the code in `nnCostFunction`, you will need to add the column of 1’s to the X matrix. The parameters for each unit in the neural network is represented in Theta1 and Theta2 as one row. Specifically, the first row of Theta1 corresponds to the first hidden unit in the second layer. You can use a for-loop over the examples to compute the cost.

```python
def nnCostFunction(nn_params,
                   input_layer_size,
                   hidden_layer_size,
                   num_labels,
                   X, y, lambda_=0.0):
    """
    Implements the neural network cost function and gradient for a two layer neural
    network which performs classification.

    Parameters
    ----------
    nn_params : array_like
        The parameters for the neural network which are "unrolled" into
        a vector. This needs to be converted back into the weight matrices Theta1
        and Theta2.

    input_layer_size : int
        Number of features for the input layer.

    hidden_layer_size : int
        Number of hidden units in the second layer.

    num_labels : int
        Total number of labels, or equivalently number of units in output layer.

    X : array_like
        Input dataset. A matrix of shape (m x input_layer_size).

    y : array_like
        Dataset labels. A vector of shape (m,).

    lambda_ : float, optional
        Regularization parameter.

    Returns
    -------
    J : float
        The computed value for the cost function at the current weight values.

    grad : array_like
        An "unrolled" vector of the partial derivatives of the concatenatation of
        neural network weights Theta1 and Theta2.

    Instructions
    ------------
    You should complete the code by working through the following parts.

    - Part 1: Feedforward the neural network and return the cost in the
              variable J. After implementing Part 1, you can verify that your
              cost function computation is correct by verifying the cost
              computed in the following cell.

    - Part 2: Implement the backpropagation algorithm to compute the gradients
              Theta1_grad and Theta2_grad. You should return the partial derivatives of
              the cost function with respect to Theta1 and Theta2 in Theta1_grad and
              Theta2_grad, respectively. After implementing Part 2, you can check
              that your implementation is correct by running checkNNGradients provided
              in the utils.py module.

              Note: The vector y passed into the function is a vector of labels
                    containing values from 0..K-1. You need to map this vector into a
                    binary vector of 1's and 0's to be used with the neural network
                    cost function.

              Hint: We recommend implementing backpropagation using a for-loop
                    over the training examples if you are implementing it for the
                    first time.

    - Part 3: Implement regularization with the cost function and gradients.

              Hint: You can implement this around the code for
                    backpropagation. That is, you can compute the gradients for
                    the regularization separately and then add them to Theta1_grad
                    and Theta2_grad from Part 2.

    Note
    ----
    We have provided an implementation for the sigmoid function in the file
    `utils.py` accompanying this assignment.
    """
    # Reshape nn_params back into the parameters Theta1 and Theta2, the weight matrices
    # for our 2 layer neural network
    Theta1 = np.reshape(nn_params[:hidden_layer_size * (input_layer_size + 1)],
                        (hidden_layer_size, (input_layer_size + 1)))

    Theta2 = np.reshape(nn_params[(hidden_layer_size * (input_layer_size + 1)):],
                        (num_labels, (hidden_layer_size + 1)))

    # Setup some useful variables
    m = y.size

    # You need to return the following variables correctly
    J = 0
    Theta1_grad = np.zeros(Theta1.shape)
    Theta2_grad = np.zeros(Theta2.shape)

    # ====================== YOUR CODE HERE ======================

    cost = 0

    for i in range(m):
      #Here we can assume that the nn is 3 layered, but need to generalise it for K(=10 here)
      a_1 = np.hstack((np.ones((1)), X[i,:])).reshape(input_layer_size+1)  #bias unit added
      z_2 = np.dot(Theta1, a_1)
      a_2 = np.hstack((np.ones((1,)),utils.sigmoid(z_2))).reshape(hidden_layer_size+1)
      z_3 = np.dot(Theta2, a_2)
      a_3 = utils.sigmoid(z_3)

      k = num_labels
      y_i = np.zeros((k))
      y_i[y[i]]=1

      cost += np.sum(-y_i*np.log(a_3)-(1-y_i)*np.log(1-a_3))

      delta_3 = a_3 - y_i
      delta_2 = np.dot(Theta2[:,1:].T,delta_3)*sigmoidGradient(z_2)  #removed 1st coloumn of the Theta2
      #There are basically two ways to do it
      # 1. If I do element wise multiplication with a_2*(1-a_2), I should have removed the first element of delta_2
      # 2. If I do element wise multiplication with g'(z_2), I should remove the first column of Theta2

      Theta1_grad = Theta1_grad + np.dot(delta_2.reshape(delta_2.shape[0],1), a_1.reshape(a_1.shape[0],1).T)
      Theta2_grad = Theta2_grad + np.dot(delta_3.reshape(delta_3.shape[0],1), a_2.reshape(a_2.shape[0],1).T)

    Theta1_grad = np.hstack(((1/m)*Theta1_grad[:,0:1] , (1/m)*(Theta1_grad[:,1:] + lambda_*Theta1[:,1:])))

    Theta2_grad = np.hstack(((1/m)*Theta2_grad[:,0:1] , (1/m)*(Theta2_grad[:,1:] + lambda_*Theta2[:,1:])))

    J = cost/m + (lambda_/(2*m))*(np.sum(Theta1[0:,1:]**2) + np.sum(Theta2[0:,1:]**2))

    # ================================================================
    # Unroll gradients
    grad = np.concatenate([Theta1_grad.ravel(), Theta2_grad.ravel()])
    grad = np.concatenate([Theta1_grad.ravel(), Theta2_grad.ravel()])

    return J, grad
```

Use the following links to go back to the different parts of this exercise that require to modify the function `nnCostFunction`.

Back to:
- [Feedforward and cost function](#section1)
- [Regularized cost](#section2)
- [Neural Network Gradient (Backpropagation)](#section4)
- [Regularized Gradient](#section5)

Once you are done, call your `nnCostFunction` using the loaded set of parameters for `Theta1` and `Theta2`. You should see that the cost is about 0.287629.

```python
lambda_ = 0
J, _ = nnCostFunction(nn_params, input_layer_size, hidden_layer_size,
                   num_labels, X, y, lambda_)
print('Cost at parameters (loaded from ex4weights): %.6f ' % J)
print('The cost should be about                   : 0.287629.')
```

    Cost at parameters (loaded from ex4weights): 0.287629
    The cost should be about                   : 0.287629.

*You should now submit your solutions.*

```python
grader = utils.Grader()
grader[1] = nnCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise neural-network-learning

    Use token from last successful submission (sachinkumar04428@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
                  Feedforward and Cost Function |  30 /  30 | Nice work!
                      Regularized Cost Function |   0 /  15 |
                               Sigmoid Gradient |   0 /   5 |
      Neural Network Gradient (Backpropagation) |   0 /  40 |
                           Regularized Gradient |   0 /  10 |
                                      --------------------------------
                                                |  30 / 100 |

### 1.4 Regularized cost function

The cost function for neural networks with regularization is given by:

$$ J(\theta) = \frac{1}{m} \sum_{i=1}^{m}\sum_{k=1}^{K} \left[ - y_k^{(i)} \log \left( \left( h_\theta \left( x^{(i)} \right) \right)_k \right) - \left( 1 - y_k^{(i)} \right) \log \left( 1 - \left( h_\theta \left( x^{(i)} \right) \right)_k \right) \right] + \frac{\lambda}{2 m} \left[ \sum_{j=1}^{25} \sum_{k=1}^{400} \left( \Theta_{j,k}^{(1)} \right)^2 + \sum_{j=1}^{10} \sum_{k=1}^{25} \left( \Theta_{j,k}^{(2)} \right)^2 \right] $$

You can assume that the neural network will only have 3 layers - an input layer, a hidden layer and an output layer. However, your code should work for any number of input units, hidden units and outputs units. While we
have explicitly listed the indices above for $\Theta^{(1)}$ and $\Theta^{(2)}$ for clarity, do note that your code should in general work with $\Theta^{(1)}$ and $\Theta^{(2)}$ of any size. Note that you should not be regularizing the terms that correspond to the bias. For the matrices `Theta1` and `Theta2`, this corresponds to the first column of each matrix. You should now add regularization to your cost function. Notice that you can first compute the unregularized cost function $J$ using your existing `nnCostFunction` and then later add the cost for the regularization terms.

[Click here to go back to `nnCostFunction` for editing.](#nnCostFunction)

Once you are done, the next cell will call your `nnCostFunction` using the loaded set of parameters for `Theta1` and `Theta2`, and $\lambda = 1$. You should see that the cost is about 0.383770.

```python
# Weight regularization parameter (we set this to 1 here).
lambda_ = 1
J, _ = nnCostFunction(nn_params, input_layer_size, hidden_layer_size,
                      num_labels, X, y, lambda_)

print('Cost at parameters (loaded from ex4weights): %.6f' % J)
print('This value should be about                 : 0.383770.')
```

*You should now submit your solutions.*

```python
grader[2] = nnCostFunction
grader.grade()
```

## 2 Backpropagation

In this part of the exercise, you will implement the backpropagation algorithm to compute the gradient for the neural network cost function. You will need to update the function `nnCostFunction` so that it returns an appropriate value for `grad`. Once you have computed the gradient, you will be able to train the neural network by minimizing the cost function $J(\theta)$ using an advanced optimizer such as `scipy`'s `optimize.minimize`.
You will first implement the backpropagation algorithm to compute the gradients for the parameters for the (unregularized) neural network. After you have verified that your gradient computation for the unregularized case is correct, you will implement the gradient for the regularized neural network.

### 2.1 Sigmoid Gradient

To help you get started with this part of the exercise, you will first implement
the sigmoid gradient function. The gradient for the sigmoid function can be
computed as

$$ g'(z) = \frac{d}{dz} g(z) = g(z)\left(1-g(z)\right) $$

where

$$ \text{sigmoid}(z) = g(z) = \frac{1}{1 + e^{-z}} $$

Now complete the implementation of `sigmoidGradient` in the next cell.

```python
def sigmoidGradient(z):
    """
    Computes the gradient of the sigmoid function evaluated at z.
    This should work regardless if z is a matrix or a vector.
    In particular, if z is a vector or matrix, you should return
    the gradient for each element.

    Parameters
    ----------
    z : array_like
        A vector or matrix as input to the sigmoid function.

    Returns
    --------
    g : array_like
        Gradient of the sigmoid function. Has the same shape as z.

    Instructions
    ------------
    Compute the gradient of the sigmoid function evaluated at
    each value of z (z can be a matrix, vector or scalar).

    Note
    ----
    We have provided an implementation of the sigmoid function
    in `utils.py` file accompanying this assignment.
    """

    g = np.zeros(z.shape)

    # ====================== YOUR CODE HERE ======================

    g = utils.sigmoid(z)*(1-utils.sigmoid(z))

    # =============================================================
    return g
```

When you are done, the following cell call `sigmoidGradient` on a given vector `z`. Try testing a few values by calling `sigmoidGradient(z)`. For large values (both positive and negative) of z, the gradient should be close to 0. When $z = 0$, the gradient should be exactly 0.25. Your code should also work with vectors and matrices. For a matrix, your function should perform the sigmoid gradient function on every element.

```python
z = np.array([-1, -0.5, 0, 0.5, 1])
g = sigmoidGradient(z)
print('Sigmoid gradient evaluated at [-1 -0.5 0 0.5 1]:\n  ')
print(g)
```

    Sigmoid gradient evaluated at [-1 -0.5 0 0.5 1]:

    [0.19661193 0.23500371 0.25       0.23500371 0.19661193]

*You should now submit your solutions.*

```python
grader[3] = sigmoidGradient
grader.grade()
```

    Submitting Solutions | Programming Exercise neural-network-learning

    Use token from last successful submission (sachinkumar04428@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
                  Feedforward and Cost Function |   0 /  30 |
                      Regularized Cost Function |   0 /  15 |
                               Sigmoid Gradient |   5 /   5 | Nice work!
      Neural Network Gradient (Backpropagation) |   0 /  40 |
                           Regularized Gradient |   0 /  10 |
                                      --------------------------------
                                                |   5 / 100 |

## 2.2 Random Initialization

When training neural networks, it is important to randomly initialize the parameters for symmetry breaking. One effective strategy for random initialization is to randomly select values for $\Theta^{(l)}$ uniformly in the range $[-\epsilon_{init}, \epsilon_{init}]$. You should use $\epsilon_{init} = 0.12$. This range of values ensures that the parameters are kept small and makes the learning more efficient.

One effective strategy for choosing $\epsilon_{init}$ is to base it on the number of units in the network. A good choice of $\epsilon_{init}$ is $\epsilon_{init} = \frac{\sqrt{6}}{\sqrt{L_{in} + L_{out}}}$ where $L_{in} = s_l$ and $L_{out} = s_{l+1}$ are the number of units in the layers adjacent to $\Theta^{l}$.

Your job is to complete the function `randInitializeWeights` to initialize the weights for $\Theta$. Modify the function by filling in the following code:

```python
# Randomly initialize the weights to small values
W = np.random.rand(L_out, 1 + L_in) * 2 * epsilon_init - epsilon_init
```
Note that we give the function an argument for $\epsilon$ with default value `epsilon_init = 0.12`.

```python
def randInitializeWeights(L_in, L_out, epsilon_init=0.12):
    """
    Randomly initialize the weights of a layer in a neural network.

    Parameters
    ----------
    L_in : int
        Number of incomming connections.

    L_out : int
        Number of outgoing connections.

    epsilon_init : float, optional
        Range of values which the weight can take from a uniform
        distribution.

    Returns
    -------
    W : array_like
        The weight initialiatized to random values.  Note that W should
        be set to a matrix of size(L_out, 1 + L_in) as
        the first column of W handles the "bias" terms.

    Instructions
    ------------
    Initialize W randomly so that we break the symmetry while training
    the neural network. Note that the first column of W corresponds
    to the parameters for the bias unit.
    """

    # You need to return the following variables correctly
    W = np.zeros((L_out, 1 + L_in))

    # ====================== YOUR CODE HERE ======================

    W = np.random.rand(L_out, 1+L_in)*2*epsilon_init - epsilon_init

    # ============================================================
    return W
```

*You do not need to submit any code for this part of the exercise.*

Execute the following cell to initialize the weights for the 2 layers in the neural network using the `randInitializeWeights` function.

```python
print('Initializing Neural Network Parameters ...')

initial_Theta1 = randInitializeWeights(input_layer_size, hidden_layer_size)
initial_Theta2 = randInitializeWeights(hidden_layer_size, num_labels)

# Unroll parameters
initial_nn_params = np.concatenate([initial_Theta1.ravel(), initial_Theta2.ravel()], axis=0)
```

    Initializing Neural Network Parameters ...

### 2.4 Backpropagation

Now, you will implement the backpropagation algorithm. Recall that the intuition behind the backpropagation algorithm is as follows. Given a training example $(x^{(t)}, y^{(t)})$, we will first run a “forward pass” to compute all the activations throughout the network, including the output value of the hypothesis $h_\theta(x)$. Then, for each node $j$ in layer $l$, we would like to compute an “error term” $\delta_j^{(l)}$ that measures how much that node was “responsible” for any errors in our output.

For an output node, we can directly measure the difference between the network’s activation and the true target value, and use that to define $\delta_j^{(3)}$ (since layer 3 is the output layer). For the hidden units, you will compute $\delta_j^{(l)}$ based on a weighted average of the error terms of the nodes in layer $(l+1)$. In detail, here is the backpropagation algorithm (also depicted in the figure above). You should implement steps 1 to 4 in a loop that processes one example at a time. Concretely, you should implement a for-loop `for t in range(m)` and place steps 1-4 below inside the for-loop, with the $t^{th}$ iteration performing the calculation on the $t^{th}$ training example $(x^{(t)}, y^{(t)})$. Step 5 will divide the accumulated gradients by $m$ to obtain the gradients for the neural network cost function.

1. Set the input layer’s values $(a^{(1)})$ to the $t^{th }$training example $x^{(t)}$. Perform a feedforward pass, computing the activations $(z^{(2)}, a^{(2)}, z^{(3)}, a^{(3)})$ for layers 2 and 3. Note that you need to add a `+1` term to ensure that the vectors of activations for layers $a^{(1)}$ and $a^{(2)}$ also include the bias unit. In `numpy`, if a 1 is a column matrix, adding one corresponds to `a_1 = np.concatenate([np.ones((m, 1)), a_1], axis=1)`.

1. For each output unit $k$ in layer 3 (the output layer), set
$$\delta_k^{(3)} = \left(a_k^{(3)} - y_k \right)$$
where $y_k \in \{0, 1\}$ indicates whether the current training example belongs to class $k$ $(y_k = 1)$, or if it belongs to a different class $(y_k = 0)$. You may find logical arrays helpful for this task (explained in the previous programming exercise).

1. For the hidden layer $l = 2$, set
$$ \delta^{(2)} = \left( \Theta^{(2)} \right)^T \delta^{(3)} * g'\left(z^{(2)} \right)$$
Note that the symbol $*$ performs element wise multiplication in `numpy`.

1. Accumulate the gradient from this example using the following formula. Note that you should skip or remove $\delta_0^{(2)}$. In `numpy`, removing $\delta_0^{(2)}$ corresponds to `delta_2 = delta_2[1:]`.

1. Obtain the (unregularized) gradient for the neural network cost function by dividing the accumulated gradients by $\frac{1}{m}$:
$$ \frac{\partial}{\partial \Theta_{ij}^{(l)}} J(\Theta) = D_{ij}^{(l)} = \frac{1}{m} \Delta_{ij}^{(l)}$$

**Python/Numpy tip**: You should implement the backpropagation algorithm only after you have successfully completed the feedforward and cost functions. While implementing the backpropagation alogrithm, it is often useful to use the `shape` function to print out the shapes of the variables you are working with if you run into dimension mismatch errors.

[Click here to go back and update the function `nnCostFunction` with the backpropagation algorithm](#nnCostFunction).

After you have implemented the backpropagation algorithm, we will proceed to run gradient checking on your implementation. The gradient check will allow you to increase your confidence that your code is
computing the gradients correctly.

### 2.4  Gradient checking

In your neural network, you are minimizing the cost function $J(\Theta)$. To perform gradient checking on your parameters, you can imagine “unrolling” the parameters $\Theta^{(1)}$, $\Theta^{(2)}$ into a long vector $\theta$. By doing so, you can think of the cost function being $J(\Theta)$ instead and use the following gradient checking procedure.

Suppose you have a function $f_i(\theta)$ that purportedly computes $\frac{\partial}{\partial \theta_i} J(\theta)$; you’d like to check if $f_i$ is outputting correct derivative values.

$$
\text{Let } \theta^{(i+)} = \theta + \begin{bmatrix} 0 \\ 0 \\ \vdots \\ \epsilon \\ \vdots \\ 0 \end{bmatrix}
\quad \text{and} \quad \theta^{(i-)} = \theta - \begin{bmatrix} 0 \\ 0 \\ \vdots \\ \epsilon \\ \vdots \\ 0 \end{bmatrix}
$$

So, $\theta^{(i+)}$ is the same as $\theta$, except its $i^{th}$ element has been incremented by $\epsilon$. Similarly, $\theta^{(i−)}$ is the corresponding vector with the $i^{th}$ element decreased by $\epsilon$. You can now numerically verify $f_i(\theta)$’s correctness by checking, for each $i$, that:

$$ f_i\left( \theta \right) \approx \frac{J\left( \theta^{(i+)}\right) - J\left( \theta^{(i-)} \right)}{2\epsilon} $$

The degree to which these two values should approximate each other will depend on the details of $J$. But assuming $\epsilon = 10^{-4}$, you’ll usually find that the left- and right-hand sides of the above will agree to at least 4 significant digits (and often many more).

We have implemented the function to compute the numerical gradient for you in `computeNumericalGradient` (within the file `utils.py`). While you are not required to modify the file, we highly encourage you to take a look at the code to understand how it works.

In the next cell we will run the provided function `checkNNGradients` which will create a small neural network and dataset that will be used for checking your gradients. If your backpropagation implementation is correct,
you should see a relative difference that is less than 1e-9.

**Practical Tip**: When performing gradient checking, it is much more efficient to use a small neural network with a relatively small number of input units and hidden units, thus having a relatively small number
of parameters. Each dimension of $\theta$ requires two evaluations of the cost function and this can be expensive. In the function `checkNNGradients`, our code creates a small random model and dataset which is used with `computeNumericalGradient` for gradient checking. Furthermore, after you are confident that your gradient computations are correct, you should turn off gradient checking before running your learning algorithm.

**Practical Tip:** Gradient checking works for any function where you are computing the cost and the gradient. Concretely, you can use the same `computeNumericalGradient` function to check if your gradient implementations for the other exercises are correct too (e.g., logistic regression’s cost function).

```python
utils.checkNNGradients(nnCostFunction)
```

    [[-9.27825235e-03 -9.27825236e-03]
     [-3.04978709e-06 -3.04978914e-06]
     [-1.75060082e-04 -1.75060082e-04]
     [-9.62660618e-05 -9.62660620e-05]
     [ 8.89911959e-03  8.89911960e-03]
     [ 1.42869427e-05  1.42869443e-05]
     [ 2.33146358e-04  2.33146357e-04]
     [ 1.17982666e-04  1.17982666e-04]
     [-8.36010761e-03 -8.36010762e-03]
     [-2.59383071e-05 -2.59383100e-05]
     [-2.87468729e-04 -2.87468729e-04]
     [-1.37149709e-04 -1.37149706e-04]
     [ 7.62813551e-03  7.62813551e-03]
     [ 3.69883213e-05  3.69883234e-05]
     [ 3.35320349e-04  3.35320347e-04]
     [ 1.53247077e-04  1.53247082e-04]
     [-6.74798369e-03 -6.74798370e-03]
     [-4.68759764e-05 -4.68759769e-05]
     [-3.76215585e-04 -3.76215587e-04]
     [-1.66560294e-04 -1.66560294e-04]
     [ 3.14544970e-01  3.14544970e-01]
     [ 1.64090819e-01  1.64090819e-01]
     [ 1.64567932e-01  1.64567932e-01]
     [ 1.58339334e-01  1.58339334e-01]
     [ 1.51127527e-01  1.51127527e-01]
     [ 1.49568335e-01  1.49568335e-01]
     [ 1.11056588e-01  1.11056588e-01]
     [ 5.75736493e-02  5.75736493e-02]
     [ 5.77867378e-02  5.77867378e-02]
     [ 5.59235296e-02  5.59235296e-02]
     [ 5.36967009e-02  5.36967009e-02]
     [ 5.31542052e-02  5.31542052e-02]
     [ 9.74006970e-02  9.74006970e-02]
     [ 5.04575855e-02  5.04575855e-02]
     [ 5.07530173e-02  5.07530173e-02]
     [ 4.91620841e-02  4.91620841e-02]
     [ 4.71456249e-02  4.71456249e-02]
     [ 4.65597186e-02  4.65597186e-02]]
    The above two columns you get should be very similar.
    (Left-Your Numerical Gradient, Right-Analytical Gradient)

    If your backpropagation implementation is correct, then
    the relative difference will be small (less than 1e-9).
    Relative Difference: 2.48422e-11

*Once your cost function passes the gradient check for the (unregularized) neural network cost function, you should submit the neural network gradient function (backpropagation).*

```python
grader[4] = nnCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise neural-network-learning

    Use token from last successful submission (sachinkumar04428@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
                  Feedforward and Cost Function |  30 /  30 | Nice work!
                      Regularized Cost Function |   0 /  15 |
                               Sigmoid Gradient |   0 /   5 |
      Neural Network Gradient (Backpropagation) |  40 /  40 | Nice work!
                           Regularized Gradient |   0 /  10 |
                                      --------------------------------
                                                |  70 / 100 |

### 2.5 Regularized Neural Network

After you have successfully implemented the backpropagation algorithm, you will add regularization to the gradient. To account for regularization, it turns out that you can add this as an additional term *after* computing the gradients using backpropagation.

Specifically, after you have computed $\Delta_{ij}^{(l)}$ using backpropagation, you should add regularization using

$$ \begin{align}
& \frac{\partial}{\partial \Theta_{ij}^{(l)}} J(\Theta) = D_{ij}^{(l)} = \frac{1}{m} \Delta_{ij}^{(l)} & \qquad \text{for } j = 0 \\
& \frac{\partial}{\partial \Theta_{ij}^{(l)}} J(\Theta) = D_{ij}^{(l)} = \frac{1}{m} \Delta_{ij}^{(l)} + \frac{\lambda}{m} \Theta_{ij}^{(l)} & \qquad \text{for } j \ge 1
\end{align}
$$

Note that you should *not* be regularizing the first column of $\Theta^{(l)}$ which is used for the bias term. Furthermore, in the parameters $\Theta_{ij}^{(l)}$, $i$ is indexed starting from 1, and $j$ is indexed starting from 0. Thus,

$$
\Theta^{(l)} = \begin{bmatrix}
\Theta_{1,0}^{(i)} & \Theta_{1,1}^{(l)} & \cdots \\
\Theta_{2,0}^{(i)} & \Theta_{2,1}^{(l)} & \cdots \\
\vdots &  ~ & \ddots
\end{bmatrix}
$$

[Now modify your code that computes grad in `nnCostFunction` to account for regularization.](#nnCostFunction)

After you are done, the following cell runs gradient checking on your implementation. If your code is correct, you should expect to see a relative difference that is less than 1e-9.

```python
#  Check gradients by running checkNNGradients
lambda_ = 3
utils.checkNNGradients(nnCostFunction, lambda_)

# Also output the costFunction debugging values
debug_J, _  = nnCostFunction(nn_params, input_layer_size,
                          hidden_layer_size, num_labels, X, y, lambda_)

print('\n\nCost at (fixed) debugging parameters (w/ lambda = %f): %f ' % (lambda_, debug_J))
print('(for lambda = 3, this value should be about 0.576051)')
```

    [[-9.27825235e-03 -9.27825236e-03]
     [-1.67679797e-02 -1.67679797e-02]
     [-6.01744725e-02 -6.01744725e-02]
     [-1.73704651e-02 -1.73704651e-02]
     [ 8.89911959e-03  8.89911960e-03]
     [ 3.94334829e-02  3.94334829e-02]
     [-3.19612287e-02 -3.19612287e-02]
     [-5.75658668e-02 -5.75658668e-02]
     [-8.36010761e-03 -8.36010762e-03]
     [ 5.93355565e-02  5.93355565e-02]
     [ 2.49225535e-02  2.49225535e-02]
     [-4.51963845e-02 -4.51963845e-02]
     [ 7.62813551e-03  7.62813551e-03]
     [ 2.47640974e-02  2.47640974e-02]
     [ 5.97717617e-02  5.97717617e-02]
     [ 9.14587966e-03  9.14587966e-03]
     [-6.74798369e-03 -6.74798370e-03]
     [-3.26881426e-02 -3.26881426e-02]
     [ 3.86410548e-02  3.86410548e-02]
     [ 5.46101547e-02  5.46101547e-02]
     [ 3.14544970e-01  3.14544970e-01]
     [ 1.18682669e-01  1.18682669e-01]
     [ 2.03987128e-01  2.03987128e-01]
     [ 1.25698067e-01  1.25698067e-01]
     [ 1.76337550e-01  1.76337550e-01]
     [ 1.32294136e-01  1.32294136e-01]
     [ 1.11056588e-01  1.11056588e-01]
     [ 3.81928666e-05  3.81928696e-05]
     [ 1.17148233e-01  1.17148233e-01]
     [-4.07588279e-03 -4.07588279e-03]
     [ 1.13133142e-01  1.13133142e-01]
     [-4.52964427e-03 -4.52964427e-03]
     [ 9.74006970e-02  9.74006970e-02]
     [ 3.36926556e-02  3.36926556e-02]
     [ 7.54801264e-02  7.54801264e-02]
     [ 1.69677090e-02  1.69677090e-02]
     [ 8.61628953e-02  8.61628953e-02]
     [ 1.50048382e-03  1.50048382e-03]]
    The above two columns you get should be very similar.
    (Left-Your Numerical Gradient, Right-Analytical Gradient)

    If your backpropagation implementation is correct, then
    the relative difference will be small (less than 1e-9).
    Relative Difference: 2.4094e-11

    Cost at (fixed) debugging parameters (w/ lambda = 3.000000): 0.576051
    (for lambda = 3, this value should be about 0.576051)

```python
grader[5] = nnCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise neural-network-learning

    Use token from last successful submission (sachinkumar04428@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
                  Feedforward and Cost Function |  30 /  30 | Nice work!
                      Regularized Cost Function |   0 /  15 |
                               Sigmoid Gradient |   0 /   5 |
      Neural Network Gradient (Backpropagation) |  40 /  40 | Nice work!
                           Regularized Gradient |  10 /  10 | Nice work!
                                      --------------------------------
                                                |  80 / 100 |

### 2.6 Learning parameters using `scipy.optimize.minimize`

After you have successfully implemented the neural network cost function
and gradient computation, the next step we will use `scipy`'s minimization to learn a good set parameters.

```python
#  After you have completed the assignment, change the maxiter to a larger
#  value to see how more training helps.
options= {'maxiter': 100}

#  You should also try different values of lambda
lambda_ = 1

# Create "short hand" for the cost function to be minimized
costFunction = lambda p: nnCostFunction(p, input_layer_size,
                                        hidden_layer_size,
                                        num_labels, X, y, lambda_)

# Now, costFunction is a function that takes in only one argument
# (the neural network parameters)
res = optimize.minimize(costFunction,
                        initial_nn_params,
                        jac=True,
                        method='TNC',
                        options=options)

# get the solution of the optimization
nn_params = res.x

# Obtain Theta1 and Theta2 back from nn_params
Theta1 = np.reshape(nn_params[:hidden_layer_size * (input_layer_size + 1)],
                    (hidden_layer_size, (input_layer_size + 1)))

Theta2 = np.reshape(nn_params[(hidden_layer_size * (input_layer_size + 1)):],
                    (num_labels, (hidden_layer_size + 1)))
```

After the training completes, we will proceed to report the training accuracy of your classifier by computing the percentage of examples it got correct. If your implementation is correct, you should see a reported
training accuracy of about 95.3% (this may vary by about 1% due to the random initialization). It is possible to get higher training accuracies by training the neural network for more iterations. We encourage you to try
training the neural network for more iterations (e.g., set `maxiter` to 400) and also vary the regularization parameter $\lambda$. With the right learning settings, it is possible to get the neural network to perfectly fit the training set.

```python
pred = utils.predict(Theta1, Theta2, X)
print('Training Set Accuracy: %f' % (np.mean(pred == y) * 100))
```

    Training Set Accuracy: 95.940000

## 3 Visualizing the Hidden Layer

One way to understand what your neural network is learning is to visualize what the representations captured by the hidden units. Informally, given a particular hidden unit, one way to visualize what it computes is to find an input $x$ that will cause it to activate (that is, to have an activation value
($a_i^{(l)}$) close to 1). For the neural network you trained, notice that the $i^{th}$ row of $\Theta^{(1)}$ is a 401-dimensional vector that represents the parameter for the $i^{th}$ hidden unit. If we discard the bias term, we get a 400 dimensional vector that represents the weights from each input pixel to the hidden unit.

Thus, one way to visualize the “representation” captured by the hidden unit is to reshape this 400 dimensional vector into a 20 × 20 image and display it (It turns out that this is equivalent to finding the input that gives the highest activation for the hidden unit, given a “norm” constraint on the input (i.e., $||x||_2 \le 1$)).

The next cell does this by using the `displayData` function and it will show you an image with 25 units,
each corresponding to one hidden unit in the network. In your trained network, you should find that the hidden units corresponds roughly to detectors that look for strokes and other patterns in the input.

```python
utils.displayData(Theta1[:, 1:])
```

### 3.1 Optional (ungraded) exercise

In this part of the exercise, you will get to try out different learning settings for the neural network to see how the performance of the neural network varies with the regularization parameter $\lambda$ and number of training steps (the `maxiter` option when using `scipy.optimize.minimize`). Neural networks are very powerful models that can form highly complex decision boundaries. Without regularization, it is possible for a neural network to “overfit” a training set so that it obtains close to 100% accuracy on the training set but does not as well on new examples that it has not seen before. You can set the regularization $\lambda$ to a smaller value and the `maxiter` parameter to a higher number of iterations to see this for youself.


# Discrete Choice

Discrete choice models are models that model a single (mutually exclusive) choice, in contrast to the standard models where a quantity is estimated.

In this notebook we will try to get you familiarized with discrete choice, the difference between logit and probit, and how to implement them (and more advanced models), using the module [statsmodels](https://www.statsmodels.org/stable/index.html).

### Linear Regression
As a lazy student we want to study as little as possible, but still pass the final test. Let's pretend we have a dataset of last year's students, with the hours they studied for the test, and whether or not they passed. From this dataset we can make an estimation how many hours we have to study ourselves to pass. If we would try a linear expression approach we would try to fit the function:

\begin{equation}
 Y = \beta_0 + \beta_1 X_1
\end{equation}

where $Y$ equals the chance of passing, $\beta_0$ the base chance of passing, $\beta_1$ the increase of chance of passing per hour we study, and $X_1$ the hours a student studied.

First: We install statsmodels and test if we can import it.

```python
from IPython.display import clear_output
!pip install statsmodels
import statsmodels
clear_output()
print("Everything A-Okay!")
```

    Everything A-Okay!

```python
%matplotlib inline
import matplotlib.pyplot as plt
import statsmodels.api as sm
import numpy as np

# the hours each student studied for the test, and whether they passed or failed
students = {'hours': [0, 2, 3, 4, 4.5, 4.5, 5, 6, 6.5, 7, 8, 8, 9, 9.5, 10, 10.5, 12, 13.5],
            'passed': [False, False, False, False, False, False, False, False, False, True, True, True, False, True, True, False, True, True]}

# use ordinary least squares (OLS) to fit our function Y
intercept, slope = sm.OLS(students['passed'],
                          sm.add_constant(students['hours'])).fit().params

# plot the results of the students
plt.scatter(students['hours'], students['passed'])
plt.xlabel('hours studied'); plt.ylabel('pass/fail')

# plot the results of the fit
x_vals = np.array(plt.gca().get_xlim())
y_vals = intercept + slope * x_vals
plt.plot(x_vals, y_vals, '--')

# set proper axes
plt.xlim([-1, 14]); plt.ylim([-0.1, 1.1])

plt.show()
```

We fitted our $Y$ function with a simple linear square approach, by using the method [sm.OLS](https://www.statsmodels.org/dev/generated/statsmodels.regression.linear_model.OLS.html) from statsmodels. Its first argument is the $Y$ value we try to fit, and the second argument are the $\beta$ values we try to fit. Note that we have to add a constant value (`sm.add_constant`) if we want a $\beta_0$ value

The obvious problem with the linear regression approach is that we try to model the chance of pass (or failure), but our model can give values outside of the range (0, 1). If a student did not study the model gives a chance lower than 0% of succes, and after more than 14 hours of study the chance of passing is higher than a 100%! To solve this problem we need discrete models.

### Binary Discrete Choice
Discrete models are similar to our previous approach, except the $Y$ value is not modelled on a continuous scale, but is limited between discrete alternatives. To solve these models we need a **utility** function, which closely resembles the function we tried to fit using linear regression, but with some added noise $\epsilon$:
\begin{equation}
 U = \beta_0 + \beta_1 X_1 + \epsilon \\
 \begin{cases}
 pass & U > 0\\
 fail & else\\
 \end{cases}
\end{equation}

This utility function represents the preference for an outcome. In our case if the utility is a number above zero, it means the student passes, otherwise the student fails. To get a probability from our utility we need a function $F$ which maps the utility to a probability between the range (0, 1).
\begin{equation}
 P_{n1} = F(U)
\end{equation}

Here we will discuss two of the most common $F$ funcitons, **Logit** & **Probit**.

### Logit (Logistic regression)
When using the Logit approach we assume that the log-odds of pass/failure can be expressed as a linear function of our input (the utility), and our unobserved fraction of the utility ($\epsilon$) follows a logistic distribution:
\begin{equation}
 log (\frac{P_{n1}}{1 - P_{n1}}) = U \\
 \frac{P_{n1}}{1 - P_{n1}} = e^U
\end{equation}

which we can rewrite to:
\begin{equation}
 P_{n1} = \frac{e^U}{1 + e^U}
\end{equation}

In the Logit case our function $F$ is just the sigmoid/logistic function!

So what did we gain from this approach? Our values are now limited between the range (0, 1), but more importantly, we can interpret out coefficients as odds! If for instance after fitting our $\beta_1$ has a value of $1.1$, it means that for each hour of study the chance of passing  would be $e^{1.1} \approx 3$ times as likely to happen!

### Probit
The probit model assumes that the unobserved fraction of the utility ($\epsilon$) follows a standard normal distribution:
\begin{equation}
P_{n1} = \Phi(\beta_0 + \beta_1 X_1)
\end{equation}
where $\Phi$ is the cumulative distribution function of the (standard) normal distribution.

### Difference
So what is the difference between a normal distribution and a logit distribution? Let's plot them both:

```python
from scipy.stats import norm, logistic
import math

# standard normal distribution
mu = 0; std = 1
x = np.linspace(-4, 4, 100)

# plot the normal pdf & cdf
normal = norm.pdf(x, loc=mu, scale=std)
plt.plot(x, normal, label='normal distribution')
plt.plot(x, np.cumsum(normal) / sum(normal), label='cumulative normal distribution')

# plot the logistic pdf & cdf
logist = logistic.pdf(x, loc=mu, scale=std * math.sqrt(3) / math.pi)
plt.plot(x, logist, label='logistic distribution')
plt.plot(x, np.cumsum(logist) / sum(logist), label='cumulative logistic distribution')

plt.ylabel('probability')
plt.legend()
plt.show()
```

They are very similar! Note that the logit distribution has fatter tails, so it will produce more extreme values than the normal distribution. Now let's see how they differ in performance of the fit:

```python
# plot the results of the students
plt.scatter(students['hours'], students['passed'])
plt.xlabel('hours studied'); plt.ylabel('pass/fail')

# set proper axes
plt.xlim([-1, 14]); plt.ylim([-0.1, 1.1])
x_vals = sm.add_constant(np.linspace(-1, 14, 100))

# use probit to fit our function
probit = sm.Probit(students['passed'], sm.add_constant(students['hours']))
pr_model = probit.fit(disp=0)  # disp=0 to silence the verbose function
pseudo_r_p = pr_model.prsquared

# plot the results of probit
y_vals = pr_model.predict(x_vals)
plt.plot(x_vals[:, 1], y_vals, '--', label='probit')

# use logit to fit our function
logit = sm.Logit(students['passed'], sm.add_constant(students['hours']))
lo_model = logit.fit(disp=0)  # disp=0 to silence the verbose function
pseudo_r_l = lo_model.prsquared

# plot the results of logit
y_vals = lo_model.predict(x_vals)
plt.plot(x_vals[:, 1], y_vals, '--', label='logit')

plt.legend()
plt.show()

# show summary of both models
print(pr_model.summary())
print(lo_model.summary())
```

```python
# so what is the probability of passing the course if you study 9 hours for the test?
# your answer has to be correct for at least two significant digits
chance = None
def calc_utility(model, hours):
    """
    Determines the utility of a certain regression model
    and input of hours studied
    """
    constant, slope = model.params
    return constant + slope * hours

def prob_Logit(utility):
    """
    Determines the probability of passing the exam
    given a Logistic regression and utility
    """
    return 1 / (1 + math.exp(-utility))

def prob_Probit(utility):
    """
    Determines the probability of passing the exam
    given a Probit regression and utility
    """
    return norm.cdf(utility, loc=mu, scale=std)

utility = calc_utility(lo_model, 9)
chance = prob_Logit(utility)
chance_lo = lo_model.predict([1, 9])[0]
assert round(chance, 2) == round(chance_lo, 2), "Probability is not significant for at least two digits"

utility = calc_utility(pr_model, 9)
chance = prob_Probit(utility)
chance_pr = pr_model.predict([1, 9])[0]
assert round(chance, 2) == round(chance_pr, 2), "Probability is not significant for at least two digits"
```

```python
assert 0 <= chance <= 1
```

Even though the fitted parameters of both models are quite different, the actual fits are extremely close, and differ little in their predictions, pseudo R squares, or looks.

### Multinomial logit
When dealing with multiple discrete alternatives, we have to make use of multinomial discrete choice. We rewrite our original utility function into one utility function per choice, where the chance of choice $i$ is defined as such:
\begin{equation}
    P_i = Prob(U_i > U_j \quad \forall j \neq i)
\end{equation}

We generated a dataset of 250 students, which contains the students' income, distance to university, how lazy they are, and what transport (either bike, car, bus 40 or bus 240) they use to get to university.

Implement multinomial logit yourself, take a look at [MNLogit](https://www.statsmodels.org/dev/generated/statsmodels.discrete.discrete_model.MNLogit.html). Remember to add a constant (`sm.add_constant`) to our observed variables. Also note that you should use numeric labels, and not the text-label.

```python
import pandas as pd

# load our dataset
df = pd.read_csv('transport.csv', sep=';', usecols=['income', 'distance', 'lazy', 'transport', 'transport_id'])

# print the 'head' of the dataframe to get a feel for the data
print(df.head())

# implement multinomial logit
df_exogeneous = df.drop(columns=["transport", "transport_id"])
df_endogeneous = df["transport_id"]
x_vals = sm.add_constant(df_exogeneous)
MNL = sm.MNLogit(df_endogeneous, x_vals)
model = MNL.fit(disp=0)

# let's see how it predicts on our own dataset (you should get at least 200 out of 250 predictions correct!)
# the predict function returns a dataframe shape (250, 4), where each column is the chance of that choice.
# Assume that the option with the highest chance is chosen
MNL_predict = model.predict(x_vals)
MNL_predict["choice"] = MNL_predict.idxmax(axis=1)
correct_predictions = np.where(MNL_predict["choice"] == df["transport_id"], 1, 0)
correct_predictions = correct_predictions.sum()
assert correct_predictions > 200, "Number of correct predictions should at least be 200"
```

       income  distance  lazy transport  transport_id
    0     0.0      11.0   7.0   bus 240             3
    1   433.0       6.0   7.0    bus 40             2
    2   450.0       8.0   7.0   bus 240             3
    3   662.0       9.0   5.0       car             1
    4   168.0       5.0   1.0      bike             0

```python
assert type(model).__name__ == 'MultinomialResultsWrapper'
```

### Logit limitations
- **Taste variation**: every students shares the same $\beta$ values, while this not necessarily has to be true. Some students might've done earlier courses which resemble a lot of the subject matter of the course, so they have a higher $\beta_0$ value, and some student might just be more efficient while learning, resulting in a higher $\beta_1$ value. Logit does not allow different $\beta$ values for its choice makers.
- **Independece of Irrelevant Alternatives (IIA)**: If we make people choose between two options (e.g. bulbasaur and squirtle), adding a third option (charmander) should not change peoples original order of the two options. For example: if someone prefers a squirtle over a bulbasaur, by also giving them the choice of a charmander, they should not suddenly like bulbasaur more. Multinomial logit does not allow independence of irrelevant alternatives.
- **Repeated choice**: Logit assumes no correlation in repeated choices. If a person takes the bike to work one day, it might influence him/her to take the bike the next day. Maybe he/she got lost, so won't take the bike again. Or the person gets to know the road better, so biking the next day becomes faster.

### Nested logit
When we look closer at the data we see that bus 240 and bus 40 are similar choices, and after a quick questionnaire we realize that if bus 40 does not go all student's will use bus 240 and vice versa. Multinomial logit violates this bus-dependency (independence of irrelevant alternatives). However if we would implement nested logit we would be guaranteed of this dependency:

Your task now is to finish the NestedLogit class, which incorporates this logic. It should fit the choices bike, car, and bus using `sm.MNLogit` and the two different buses by `sm.Logit`.

```python
class NestedLogit():
    def __init__(self, labels, variables):
        self.labels = labels
        self.variables = variables

        # Makes a binary choice problem for the different busses
        self.bus = self.labels[self.labels.isin([2, 3])]
        self.bus.where(self.bus == 2, 0, inplace = True)
        self.bus.where(self.bus == 0, 1, inplace = True)
        self.variables_bus = self.variables[self.variables.index.isin(self.bus.index)]

    def fit(self):
        """
        Method that fits the predictions of the NestedLogit.
        """
        # use logit to fit our function for the different busses
        logit_bus = sm.Logit(self.bus, sm.add_constant(self.variables_bus))
        self.model_bus = logit_bus.fit(disp=0)  # disp=0 to silence the verbose function

        # use Multinomial Logit to fit our function to choose the transport
        labels_others = self.labels.where(self.labels != 3, 2)
        MNL_others = sm.MNLogit(labels_others, sm.add_constant(self.variables))
        self.model_others = MNL_others.fit(disp=0)

    def predict(self, variables):
        """
        Method that returns the predictions of the NestedLogit, based on the fit, shape (N, 4)
        """
        predict_others = self.model_others.predict(variables)
        predict_busses = self.model_bus.predict(variables)
        prob_bus = predict_others[2]
        prob_bus40 = prob_bus.multiply(predict_busses)
        prob_bus240 = prob_bus.multiply(1 - predict_busses)
        predict_others[2] = prob_bus40
        predict_others[3] = prob_bus240

        return predict_others

# Calls to NestedLogit
nlogit = NestedLogit(df['transport_id'], sm.add_constant(df[['income', 'distance', 'lazy']]))
nlogit.fit()
y_vals = nlogit.predict(sm.add_constant(df[['income', 'distance', 'lazy']]))

# How does nested logit compare to multinomial logit? You should get at least 170 predictions correct!
y_vals = model.predict(x_vals)
y_vals["choice"] = y_vals.idxmax(axis=1)
correct_predictions = np.where(y_vals["choice"] == df["transport_id"], 1, 0)
correct_predictions = correct_predictions.sum()
assert correct_predictions > 170, "Number of correct predictions should at least be 170"
```

```python
nlogit = NestedLogit(df['transport_id'], sm.add_constant(df[['income', 'distance', 'lazy']]))
nlogit.fit()
y_vals = nlogit.predict(sm.add_constant(df[['income', 'distance', 'lazy']]))
assert y_vals.shape == (250, 4)
```

In the file generate_data.py is the data generated. Can you design a dataset where NestedLogit outperforms MultiNomialLogit? Why does nested logit not outperform multinomial logit?

### Advanced models
For more complex logit models, such as mixed logit which allows for taste variation. Take a look at [PyLogit](https://github.com/timothyb0912/pylogit)!


# Explain Kaiming Initialization
> drafting

- toc: true
- badges: true
- comments: true
- categories: [self-learning]
- image: images/bone.jpeg
- hide: true

- **nn.init.kaiming_uniform_**
    - this funciton implements the initialization recommendation from the sound paper [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852)
    - https://towardsdatascience.com/understand-kaiming-initialization-and-implementation-detail-in-pytorch-f7aa967e9138

Firstly, we take a brief look on notation of a standard neural network. The image below shows notation of 4 layer neural network.

The initialialization method proposed in this paper was to tackle the problem of hard convergence with randomly initialized weight drawn from Gaussian distribution. Earlier, there was another paper ,`Xavier initialization`, which also tackled this problem but they only considered linear layer and did not consider non-linear layer.

I will go along part 2.2 in the original paper,`Delving Deep in Rectifiers: Surpassing Human-Level Performance on ImageNet Classification`, with some explanations in detail either in math or code.

The central idea is to investigate the impact of initialization in the variance of responses in each layer.

### Forward Propagation Case

For each layer, the response is:
\begin{equation}
\mathbf{y}_{l}=\mathrm{W}_{l} \mathbf{x}_{l}+\mathbf{b}_{l}
\label{eq1}
\tag{1}
\end{equation}

\begin{equation}
\mathbf{x}_{l}=f\left(\mathbf{y}_{l-1}\right)
\label{eq2}
\tag{2}
\end{equation}

        To be consistent with the paper, `x` is used instead of `a` - response after activation fucntion.

A few assumption about the initialization:
- Initialized elements of $\mathrm{W}_{l}$ and $\mathrm{x}_{l}$ are mutually independent and share the same distribution.
- $\mathrm{W}_{l}$ and $\mathrm{x}_{l}$ are independent each other.

Now, let do some transformation from the function \ref{eq1}, \ref{eq2}

Given $\mathrm{y'}_{l}$, $\mathrm{x'}_{l}$, $\mathrm{w'}_{l}$ is the random variables of each element in $\mathrm{y}_{l}$, $\mathrm{x}_{l}$ and $\mathrm{W}_{l}$ respectively. Then we have:

\begin{equation}
\begin{aligned}
\operatorname{Var}\left[y'_{l}\right] &=\operatorname{Var} \sum_{1}^{n_{l}} \left(w'_{l} x'_{l}\right) \\
&= \sum_{1}^{n_{l}} \operatorname{Var}\left[w'_{l} x'_{l}\right] \\
&= n_{l} \operatorname{Var}\left[w'_{l} x'_{l}\right]
\end{aligned}
\label{eq3}
\tag{3}
\end{equation}

Intuitively explaination:
- for each node, such as node 1 in layer 2, its value will be sum of product of x and w. Therefore, the variance of y will be variance of sum of those products. Because all the w and x follow the same distribution (respectively), I am using a common notation $\mathrm{w'}_{l}$$\mathrm{x'}_{l}$ which represent those products.
- $\mathrm{n}_{l-1}$ represent the number of product between $\mathrm{w'}_{l}$ and $\mathrm{x'}_{l}$
- Bias term $\mathrm{b}$ is ignore because it usually is initialized with a constant value, so it variance is 0.
- Because w and x are independent, so equation 3 can be transformed to equation 4.

\begin{equation}
\begin{aligned}
\operatorname{Var}\left[y'_{l}\right] &= n_{l} \operatorname{Var}\left[w'_{l} x'_{l}\right] \\
&= n_{l}(\underbrace{\mathbb{E}\left[{w'}_{l}^{2}\right]}_{=\operatorname{Var}\left[w'_{l}\right]} \mathbb{E}\left[{x'}_{l}^{2}\right]-\underbrace{\mathbb{E}\left[w'_{l}\right]^{2}}_{=0} \mathbb{E}\left[x'_{l}\right]^{2}) \\
&=n_{l} \operatorname{Var}\left[w'_{l}\right] \mathbb{E}\left[{x'}_{l}^{2}\right]
\end{aligned}
\label{eq4}
\tag{4}
\end{equation}

Intuitively explaination:
- assuming 2 random variables are independent, we can derive line 1 into line 2 by applying formular [wiki formular variance ](https://en.wikipedia.org/wiki/Variance#Product_of_independent_variables).
- By assuming random variable $\mathrm{w}_{l}$ has zero mean, we have:
\begin{equation}
\begin{aligned}
\operatorname{Var}\left[w'_{l}\right] &=\mathbb{E}\left[w'^{2}\right]-\mathbb{E}[w']^{2} \\
&=\mathbb{E}\left[w'^{2}\right]
\end{aligned}
\label{eq5}
\tag{5}
\end{equation}

But $\mathbb{E}[x']^{2} \neq \operatorname{Var}\left[x'\right]$ because $\mathrm{E}[x']$ does not have zero mean, it is the result of ReLU function, $x_{l}=\max \left(0, y_{l-1}\right)$, from previous layer.

\begin{equation}
\begin{aligned}
\mathbb{E}\left[{x'}_{l}^{2}\right] &=\mathbb{E}\left[\max \left(0, y'_{l-1}\right)^{2}\right] \\
&=\frac{1}{2} \mathbb{E}\left[{y'}_{l-1}^{2}\right] \\
&=\frac{1}{2} \operatorname{Var}\left[y'_{l-1}\right]
\end{aligned}
\label{eq6}
\tag{6}
\end{equation}

Intuitively explaination:
- Assuming $\mathrm{w}_{l-1}$ has a symmetric distribution around 0 and $\mathrm{b}_{l-1}$ = 0 then $\mathrm{y}_{l-1}$ has zero mean and symmetric distribution around 0 => that's why we can derive equation \ref{eq6}.

\begin{equation}
\begin{aligned}
\mathbb{E}\left(y_{l-1}\right) &=\mathbb{E}\left(w_{l-1} x_{l-1}\right) \\
&=\mathbb{E}\left(w_{l-1}\right) \mathbb{E}\left(x_{l-1}\right) \\
&=0
\end{aligned}
\label{eq7}
\tag{7}
\end{equation}

\begin{equation}
\begin{aligned}
\mathbb{P}\left(y_{l-1}>0\right) &=\mathbb{P}\left(w_{l-1} x_{l-1}>0\right) \\
&=\mathbb{P}\left(\left(w_{l-1}>0 \text { and } x_{l-1}>0\right) \text { or }\left(w_{l-1}<0 \text { and } x_{l-1}<0\right)\right) \\
&=\mathbb{P}\left(w_{l-1}>0\right) \mathbb{P}\left(x_{l-1}>0\right)+\mathbb{P}\left(w_{l-1}<0\right) \mathbb{P}\left(x_{l-1}<0\right) \\
&=\frac{1}{2} \mathbb{P}\left(x_{l-1}>0\right)+\frac{1}{2} \mathbb{P}\left(x_{l-1}<0\right) \\
&=\frac{1}{2}
\end{aligned}
\label{eq8}
\tag{8}
\end{equation}

Plugging back to equation \ref{eq4} we have:
\begin{equation}
\operatorname{Var}\left[y_{l}\right]=\frac{1}{2} n_{l} \operatorname{Var}\left[w_{l}\right] \operatorname{Var}\left[y_{l-1}\right]
\label{eq9}
\tag{9}
\end{equation}

With L layers put together, we have:
\begin{equation}
\operatorname{Var}\left[y_{L}\right]=\operatorname{Var}\left[y_{1}\right]\left(\prod_{l=2}^{L} \frac{1}{2} n_{l} \operatorname{Var}\left[w_{l}\right]\right)
\label{eq10}
\tag{10}
\end{equation}

From here, the layer part on paper is quite clear, equation \ref{eq10} is the key to the initialization design. A proper initialization method should avoid reducing or magnifying the magnitudes of input signals exponentially. So we expect the above product to take a proper scalar, eg 1. A sufficient condition is:
\begin{equation}
\frac{1}{2} n_{l} \operatorname{Var}\left[w_{l}\right]=1, \quad \forall l
\label{eq11}
\tag{11}
\end{equation}

\begin{equation}
\begin{aligned}
\Rightarrow \operatorname{Var}\left[w_{l}\right] = \frac{2}{n_{l}} \\
\Rightarrow \mathbb{E}\left[w_{l}\right] = \sqrt{\frac{2}{n_{l}}}
\end{aligned}
\label{eq12}
\tag{12}
\end{equation}

Equation \ref{12} is the He. initialilzation, together with b=0.

Note: from equation \ref{eq6} we can see that if previous layer is not ReLU-kind, such as the first layer, we will have $\mathbb{E}\left[{x'}_{l}^{2}\right] =\operatorname{Var}\left[y'_{l-1}\right]$ then $\mathbb{E}\left[w_{l}\right] = \sqrt{\frac{1}{n_{l}}}$. But the factor 1/2 here does not matter if it just exists on one layer. So we adopt equation \ref

```python

```

### Backward Propagation Case

```python

```

Code implementation

```python

```

```python

```

https://pouannes.github.io/blog/initialization/#mjx-eqn-eqfwd
https://prateekvjoshi.com/2016/03/29/understanding-xavier-initialization-in-deep-neural-networks/
https://medium.com/a-paper-a-day-will-have-you-screaming-hurray/day-8-delving-deep-into-rectifiers-surpassing-human-level-performance-on-imagenet-classification-f449a886e604

- **trace of matrix**: deep learning page 44
- **expectation, variance and covariance**: deep learning page 58

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```