# Models and Pricing of Financial Derivatives HW_04

**11510691 程远星$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\bbspace}{\;\;\;\;\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\given}[1]{\left. #1 \right|}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N}\left( #1 \right)}
\newcommand{\Exp}[1]{\mathrm{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Avar}[1]{\mathrm{Avar}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left( #1 \right)}
\newcommand{\Corr}[1]{\mathrm{Corr}\left( #1 \right)}
\newcommand{\ExpH}{\mathrm{E}}
\newcommand{\VarH}{\mathrm{Var}}
\newcommand{\AVarH}{\mathrm{Avar}}
\newcommand{\CovH}{\mathrm{Cov}}
\newcommand{\CorrH}{\mathrm{Corr}}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\wp}{\text{with probability }}
\newcommand{\FSD}{\text{FSD}}
\newcommand{\SSD}{\text{SSD}}
\void^\dagger$**

## Question 1

A stock price is currently $50$ bucks. Assume that the expected return from the stock is $18\%$ and its volatility is $30\%$. What is the probability distribution for the stock price in $2$ years? Calculate the mean and standard deviation of the distribution.

$\bspace Answer$

>$S_T$ is lognormally distributed. At $T=2$, we have
>
>$$\begin{align}
S_T &= S_0 \cdot \exp\CB{\P{\mu - \ffrac{\sigma^2}{2}}T +\sigma W_T}\\
&= 50 \exp\CB{2\P{0.18 - \ffrac{0.3^2}{2}} + 0.3 W_2}\\
\Rightarrow \log S_T &\sim \N{\log 50 + 0.27, 2\times0.3^2}\sim \N{4.1820,0.1800}
\end{align}$$
>
>$$
\begin{align}\Exp{S_T} &= e^{\mu-\sigma^2/2}=e^{\log 50 + 0.27 + 0.3^2} = 71.6665\\
\sqrt{\Var{S_T}} &= \sqrt{\P{\exp\CB{\sigma ^{2}}-1}\exp\CB{2\mu +\sigma ^{2}}} \\
&= \sqrt{ \P{e^{0.18}-1} \exp\CB{2\log 50+0.54+0.18}} = 31.8265\end{align}$$

## Question 2

A stock price follows geometric Brownian motion with an expected return of $16\%$ and a volatility of $35\%$. The current price is $\$38$.

$\P{\text a}$ What is the probability that a European call option on the stock with an exercise price of $\$40$ and a maturity date in $6$ months will be exercised?

$\bspace Answer$

>We first calculate the distribution of the future stock price. Similarly,
>
>$$S_T = 38\exp\CB{0.5\P{0.16 - \ffrac{0.35^2}{2}} + 0.35 W_{0.5}}\Rightarrow \log S_T \sim\N{3.6870,0.06125}$$
>
>Since $\log$ function is monotonic, we have the probability
>
>$$\begin{align}
P\CB{S_T > 40} &= P\CB{\log S_T > \log 40} \\
&= 1- \N{\ffrac{\log 40 - 3.6870}{\sqrt{0.06125}}}\\
&= 1 - 0.5031 = 0.4969
\end{align}$$

$\P{\text b}$ What is the probability that a European put option on the stock with the same exercise price and maturity will be exercised?

$\bspace Answer$

>As for put option, the probablity to exercise is $P\CB{S_T < 40} = 0.5031$

## Question 3

A financial institution plans to offer a security that pays off a dollar amount equal to $S_T^2$ at time $T$, where $S_T$ is the price at time $T$ of a stock that pays *no dividends*.

$\P{\text a}$ Use *risk-neutral valuation* to calculate the price of the security at time $t$ in terms of the stock price $S_t$.

$\bspace Answer$

>We first find the risk neutral measure $\QQ$ under which the discounted stock price $\CB{ \tilde S_t}_{t\geq 0}$ is a martingale. Then $\given{\ffrac{\dd\QQ}{\dd\PP}}_{\FcF_t} = \exp\CB{-\theta W_t - \ffrac{1}{2}\theta^2 t}$, where $\theta = \ffrac{\mu - r}{\sigma}$, under which $\dd S_t = rS_t\dd t + \sigma S_t \dd \widetilde W_t$. Using the risk-neutral valuation method we have
>
>$$\begin{align}
V_t &= \ExpH^\QQ_t\SB{e^{-r\P{T-t}} S_T^2}\\
&= \ExpH^\QQ\SB{e^{-r\P{T-t}} S_T^2 \mid S_t}\\
&= e^{-r\P{T-t}} \ExpH^\QQ\SB{S_t^2\P{\exp\CB{\P{r-\ffrac{\sigma^2}{2}}\P{T-t} + \sigma\P{\widetilde W_T-\widetilde W_t}}}^2 \mid S_t}\\
&= e^{-r\P{T-t}}\ExpH^{\QQ}_t\SB{S_t^2} \cdot \exp\CB{\P{2r - \sigma^2}\P{T-t}} \cdot \ExpH_t^\QQ \SB{\exp\CB{2\sigma \P{\widetilde W_T-\widetilde W_t}}}\\
&= S_t^2\exp\CB{\P{-r + 2r - \sigma^2 + \ffrac{4\sigma^2}{2}}\P{T-t}}\\
&= S_t^2\exp\CB{\P{r+\sigma^2}\P{T-t}} \equiv f
\end{align}$$

$\P{\text b}$ Confirm that your price satisfies the differential equation

$$\ffrac{\partial f}{\partial t} + \ffrac{1}{2} \sigma^2 S_t^2 \ffrac{\partial^2 f}{\partial S_t^2} + rS_t \ffrac{\partial f}{\partial S_t} = rf$$

$\bspace Answer$

>The result is obvious using the Feynman-Kac theorem. Or applying Itô's formula term by term, we have
>
>$$\begin{align}
& \ffrac{\partial f}{\partial t} + \ffrac{1}{2} \sigma^2 S_t^2 \ffrac{\partial^2 f}{\partial S_t^2} + rS_t \ffrac{\partial f}{\partial S_t} - rf\\
=\; & - \P{r+\sigma^2} f + \ffrac{1}{2} \sigma^2 S_t^2 \cdot \ffrac{2f}{S_t^2} + rS_t \cdot\ffrac{2f}{S_t} - rf = 0
\end{align}$$

## Question 4

Consider an option on a *non-dividend-paying* stock when the stock price is $\$30$, the exercise price is $\$29$, the risk-free interest rate is $5\%$, the volatility is $25\%$ per annum, and the time to maturity is $4$ months.

$\P{\text a}$ What is the price of the option if it is a European call?

$\bspace Answer$

>$$\begin{align}
d_1 &= \ffrac{1}{\sigma\sqrt{T-0}} \P{\log\ffrac{S_0}{K} + \P{r+\ffrac{1}{2}\sigma^2}\P{T-0}} = \ffrac{1}{0.25\sqrt{1/3} }\P{\log\ffrac{30}{29} + \ffrac{1}{3}\P{0.05+\ffrac{1}{2}0.25^2}} = 0.4225\\
d_2 &= \ffrac{1}{\sigma\sqrt{T-0}} \P{\log\ffrac{S_0}{K} + \P{r-\ffrac{1}{2}\sigma^2}\P{T-0}} = \ffrac{1}{0.25\sqrt{1/3} }\P{\log\ffrac{30}{29} + \ffrac{1}{3}\P{0.05-\ffrac{1}{2}0.25^2}} = 0.2782\\
c &= S_t\N{d_1} - Ke^{-r\P{T-t}} \N{d_2} = 30 \times 0.6637 - 29 e^{-0.05 /3 } \times 0.6096 = 2.5251
\end{align}$$

$\P{\text b}$ What is the price of the option if it is an American call?

$\bspace Answer$

>The American Call option has the same price with the European Call option and thus, $2.5251$.

$\P{\text c}$ What is the price of the option if it is a European put?

$\bspace Answer$

>$p = Ke^{-r\P{T-t}} \N{-d_2} - S_t\N{-d_1} = 29 e^{-0.05 /3 } \times \P{1-0.6096} - 30\times \P{1-0.6637} = 1.0458$

$\P{\text d}$ Verify that putCcall parity holds.

$\bspace Answer$

>$c + Ke^{-rT} -p-S_0 = 2.5251 - 1.0458 + 29 e^{-0.05 /3 } - 30 \approx 0 $, and thus the parity holds

## Question 5

Consider a derivative that pays off $S_T^n$ at time $T$, where $S_T$ is the stock price at that time. When the stock price follows *geometric Brownian motion*, it can be shown that its price at time $t\leq T$, has the form

$$h\P{t,T}S^n$$

where $S$ is the stock price at time $t$ and $h$ is a function only of $t$ and $T$.

$\P{\text a}$ By substituting into the Black-Scholes-Merton partial differential equation, derive an ordinary differential equation satisfied by $h(t, T)$.

$\bspace Answer$

>The BS Formula for european call option is
>
>$$\ffrac{\partial c}{\partial t} + \ffrac{1}{2} \sigma^2 S_t^2 \ffrac{\partial^2 c}{\partial S_t^2} + rS_t \ffrac{\partial c}{\partial S_t} - rc = 0$$
>
>so similarly, letting $V_t = h\P{t,T} S^n$ we write
>
>$$\ffrac{\partial h\P{t,T}}{\partial t} + \ffrac{h\P{t,T}}{2} \sigma^2 \cdot n\P{n-1} + rh\P{t,T} \cdot n-rh\P{t,T} = 0$$

$\P{\text b}$ What is the boundary condition for the differential equation for $h(t, T)$

$\bspace Answer$

>At the boundary time, $t = T$, the payoff is $S_T^n = h\P{T,T}S_T^n$ and thus $h\P{T,T} = 1$.

$\P{\text c}$ Show that $h\P{t,T} = \exp\CB{\P{\ffrac{n\P{n-1}}{2}\sigma^2+ \P{n-1}r}\P{T-t}}$ where $r$ is the *risk-free interest rate* and $\sigma$ is the stock price *volatility*.

$\bspace Answer$

>Using the risk-neutral valuation we have
>
>$$\begin{align}V_t &= h\P{t,T} S_t^n \\
&= \ExpH^\QQ_t\SB{e^{-r\P{T-t}} S_T^n} \\
&= e^{-r\P{T-t}}\ExpH^\QQ_t\SB{ S_t^n \exp\CB{n\P{r-\ffrac{\sigma^2}{2}}\P{T-t} + n\sigma\P{\widetilde W_T - \widetilde W_t}}} \\
&= S_t^n e^{\P{n-1}r\P{T-t}} \cdot \exp\CB{-\ffrac{\sigma^2}{2}n\P{T-t}} e^{n^2\sigma^2 \P{T-t}/2}\\
&= S_t^n \exp\CB{\P{\ffrac{n\P{n-1}}{2}\sigma^2+ \P{n-1}r}\P{T-t}}\\
\Rightarrow h\P{t,T} &= \exp\CB{\P{\ffrac{n\P{n-1}}{2}\sigma^2+ \P{n-1}r}\P{T-t}}
\end{align}$$

$Remark$

>You can also rewrite the ODE, trying to solve that. More details can be seen in file `Review.pdf`

## Question 6

Show that the Black-Scholes formulas for call and put options satisfy put-call parity.

$\bspace Answer$

>$$\begin{align}
&c\P{t,S_t} + Ke^{-r\P{T-t}} - p\P{t,S_t} - S_t\\
=\;& S_t\N{d_1} - Ke^{-r\P{T-t}} \N{d_2} + Ke^{-r\P{T-t}} - \P{Ke^{-r\P{T-t}} \N{-d_2} - S_t\N{-d_1}} - S_t\\
=\;& S_t\P{\N{d_1} + \N{-d_1} - 1} - Ke^{-r\P{T-t}} \N{d_2} \P{\N{d_2} + \N{-d_2} - 1 }\\
=\;& S_t \cdot 0 - Ke^{-r\P{T-t}} \cdot 0 = 0
\end{align}$$

## Question 7

Let $S\P0 = \$100.00$, $K_1 = \$92.00$, $K_2 = \$125.00$, and $r = 5\%$. Find the Black-Scholes formula
for the option paying $\$10.00$ in $3$ months if $S\P T \leq K_1$ or if $S\P T \geq K_2$, and zero otherwise, in the Black-Scholes continuous-time model.

$\bspace Answer$

>The equation for the option keeps unchanged, only the boundary condition alted.
>
>$$\ffrac{\partial V}{\partial t} + \ffrac{1}{2} \sigma^2 S_t^2 \ffrac{\partial^2 V}{\partial S_t^2} + rS_t \ffrac{\partial V}{\partial S_t} - rV = 0\\
V\P{T,S_T} = 10 - 10 \cdot \mathbf{1}_{K_1 < S_T < K_2}$$
>
>We solve this using the Feynman-Kac theorem

>$$\begin{align}
V\P{t,S_t} &= \ExpH^\QQ\SB{e^{-r\P{T-t}} \P{10 - 10 \cdot \mathbf{1}_{K_1 < S_T < K_2}} \mid S_t}\\
&= 10 e^{-r\P{T-t}}\P{1 - \ExpH^\QQ_t\SB{ \mathbf{1}_{K_1 < S_T < K_2} }}\\
&= 10 e^{-r\P{T-t}}\P{1 - \P{\ExpH^\QQ_t\SB{ \mathbf{1}_{S_T>K_1}} - \ExpH^\QQ_t\SB{ \mathbf{1}_{S_T>K_2}}}}\\
&= 10 e^{-r\P{T-t}}\P{\N{-k_1} + \N{k_2}}\\
k_1 &= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\ffrac{S_t}{K_1} + \P{r-\ffrac{1}{2}\sigma^2}\P{T-t}}\\
k_2 &= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\ffrac{S_t}{K_2} + \P{r-\ffrac{1}{2}\sigma^2}\P{T-t}}
\end{align}$$
>
>When $t=0$, we have
>
>$$\begin{align}
V &= 10 e^{-0.05\times 3/12}\P{\N{-k_1} + \N{k_2}}\\
&=9.8758\P{\N{-k_1} + \N{k_2}} \\
k_1 &= \ffrac{1}{\sigma\sqrt{T}} \P{\log\ffrac{S_t}{K_1} + \P{r-\ffrac{1}{2}\sigma^2}T}\\
k_2 &= \ffrac{1}{\sigma\sqrt{T}} \P{\log\ffrac{S_t}{K_2} + \P{r-\ffrac{1}{2}\sigma^2}T}
\end{align}$$

## Question 8

A portfolio manger manages portfolio whose current value is $2$ *million*. She assumes the *portfolio value* satisfies the Black-Scholes model, with $\sigma = 0.2$. The interest rate is $r = 0.3$. For simplicity, we assume *zero dividend* rate. The manager wants to protect the portfolio by *replicating a put option* on its value, with *strike price* equal to $1.9$ *million* and maturity one year. What amount does she have to *sell* from the portfolio *today*, or add to the portfolio, in order to start replicating the put option?

$\bspace Answer$

>$$-\Delta_p = -\ffrac{\pi_0}{S_0} = -\ffrac{\partial p}{\partial S_t} = 1 - \N{d_1} = \N{-d_1} = 0.03169$$
>
>Thus she need to sell $S_0 \cdot 0.03169 = 0.06338$ million amount to start replicating.

***

```python
import math as m
import scipy.stats as scs
```

```python
mu = m.log(50)+0.27
mu
```

    4.182023005428146

```python
m.exp(mu+0.3**2)
```

    71.666470728017

```python
m.sqrt((m.exp(0.18)-1)*m.exp(2*mu+0.18))
```

    31.826478776054607

***

```python
mu = m.log(38)+0.08-0.35**2/4
mu
```

    3.6869611597263856

```python
sigma = m.pow(0.35,2)*0.5
sigma
```

    0.06124999999999999

```python
scs.norm.cdf((m.log(40)-mu)/m.sqrt(sigma))
```

    0.50309220249891906

***

```python
d1 = 4*m.sqrt(3)*(m.log(30/29)+(0.05+0.25*0.25/2)/3)
d2 = 4*m.sqrt(3)*(m.log(30/29)+(0.05-0.25*0.25/2)/3)
print(d1)
print(d2)
```

    0.42251567731743644
    0.27817811002003

```python
30*scs.norm.cdf(d1)-29*m.exp(-0.05/3)*scs.norm.cdf(d2)
```

    2.525146966700003

```python
29*m.exp(-0.05/3)*scs.norm.cdf(-d2) - 30*scs.norm.cdf(-d1)
```

    1.045819127526908

***

```python
10*m.exp(-0.05*3/12)
```

    9.875778004938814

***

```python
d_1 = 5*(m.log(2/1.9)+0.3+0.2*0.1)
scs.norm.cdf(-d_1)
```

    0.031693548850187665

FinMath/Models and Pricing of Financial Derivatives/HW/HW_04_11510691.ipynb


```python
%load_ext autoreload
%autoreload 2

from utils import build_transf, full_homo_transf, prop_velo, prop_force_torque, comp_jacobian
from sympy import sqrt
import sympy as sy
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

# Exercise 3 Problem 1

Denavit-Hartenberg Parameters are from problem 1 of exercise 3.

```python
dh_params = [[0, 0, 0, "theta_1"],
             [0, 1, 0, "theta_2"],
             [45, 0, sqrt(2), "theta_3"],
             [0, sqrt(2), 0, "theta_4"]]
```

## Homogeneous transforms

```python
transforms = build_transf(dh_params, verbose=True)
```

$\displaystyle {}^0_1T = \left[\begin{matrix}\cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} & 0 & 0\\\sin{\left(\theta_{1} \right)} & \cos{\left(\theta_{1} \right)} & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^1_2T = \left[\begin{matrix}\cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{2} \right)} & 0 & 1\\\sin{\left(\theta_{2} \right)} & \cos{\left(\theta_{2} \right)} & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^2_3T = \left[\begin{matrix}\cos{\left(\theta_{3} \right)} & - \sin{\left(\theta_{3} \right)} & 0 & 0\\\frac{\sqrt{2} \sin{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2} \cos{\left(\theta_{3} \right)}}{2} & - \frac{\sqrt{2}}{2} & -1\\\frac{\sqrt{2} \sin{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2} \cos{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2}}{2} & 1\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^3_4T = \left[\begin{matrix}\cos{\left(\theta_{4} \right)} & - \sin{\left(\theta_{4} \right)} & 0 & \sqrt{2}\\\sin{\left(\theta_{4} \right)} & \cos{\left(\theta_{4} \right)} & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
full_transform = full_homo_transf(transforms)
```

$\displaystyle {}^0_2T = \left[\begin{matrix}\cos{\left(\theta_{1} + \theta_{2} \right)} & - \sin{\left(\theta_{1} + \theta_{2} \right)} & 0 & \cos{\left(\theta_{1} \right)}\\\sin{\left(\theta_{1} + \theta_{2} \right)} & \cos{\left(\theta_{1} + \theta_{2} \right)} & 0 & \sin{\left(\theta_{1} \right)}\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^0_3T = \left[\begin{matrix}- \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{1} + \theta_{2} \right)}}{2} + \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)} & - \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)} - \frac{\sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)}}{2} & \sin{\left(\theta_{1} + \theta_{2} \right)} + \cos{\left(\theta_{1} \right)}\\\frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)}}{2} + \sin{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} \right)} & - \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{1} + \theta_{2} \right)} + \frac{\sqrt{2} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)}}{2} & - \frac{\sqrt{2} \cos{\left(\theta_{1} + \theta_{2} \right)}}{2} & \sin{\left(\theta_{1} \right)} - \cos{\left(\theta_{1} + \theta_{2} \right)}\\\frac{\sqrt{2} \sin{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2} \cos{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2}}{2} & 1\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^0_4T = \left[\begin{matrix}- \frac{\sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)} \sin{\left(\theta_{3} + \theta_{4} \right)}}{2} + \cos{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} & - \frac{\sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)}}{2} - \sin{\left(\theta_{3} + \theta_{4} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)} & \frac{\sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)}}{2} & - \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{1} + \theta_{2} \right)} + \sin{\left(\theta_{1} + \theta_{2} \right)} + \cos{\left(\theta_{1} \right)} + \sqrt{2} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)}\\\sin{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} + \frac{\sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)}}{2} & - \sin{\left(\theta_{1} + \theta_{2} \right)} \sin{\left(\theta_{3} + \theta_{4} \right)} + \frac{\sqrt{2} \cos{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)}}{2} & - \frac{\sqrt{2} \cos{\left(\theta_{1} + \theta_{2} \right)}}{2} & \sin{\left(\theta_{1} \right)} + \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{1} + \theta_{2} \right)} + \sqrt{2} \sin{\left(\theta_{1} + \theta_{2} \right)} \cos{\left(\theta_{3} \right)} - \cos{\left(\theta_{1} + \theta_{2} \right)}\\\frac{\sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)}}{2} & \frac{\sqrt{2} \cos{\left(\theta_{3} + \theta_{4} \right)}}{2} & \frac{\sqrt{2}}{2} & \sin{\left(\theta_{3} \right)} + 1\\0 & 0 & 0 & 1\end{matrix}\right]$

## Propagate linear-angular velocities

```python
joint_points = [sy.Matrix([0, 0, 0]),
                sy.Matrix([1, 0, 0]),
                sy.Matrix([0, -1, 1]),
                sy.Matrix([sqrt(2), 0, 0])]
v, omega, joint_params = prop_velo(dh_params, joint_points)
```

$\displaystyle {}^1v_1 = \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

$\displaystyle {}^1\omega_1 = \left[\begin{matrix}0\\0\\\dot\theta_{1}\end{matrix}\right]$

$\displaystyle {}^2v_2 = \left[\begin{matrix}\dot\theta_{1} \sin{\left(\theta_{2} \right)}\\\dot\theta_{1} \cos{\left(\theta_{2} \right)}\\0\end{matrix}\right]$

$\displaystyle {}^2\omega_2 = \left[\begin{matrix}0\\0\\\dot\theta_{1} + \dot\theta_{2}\end{matrix}\right]$

$\displaystyle {}^3v_3 = \left[\begin{matrix}\frac{\sqrt{2} \dot\theta_{1} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{2} \right)}}{2} + \left(\dot\theta_{1} \sin{\left(\theta_{2} \right)} + \dot\theta_{1} + \dot\theta_{2}\right) \cos{\left(\theta_{3} \right)}\\\frac{\sqrt{2} \dot\theta_{1} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} \right)}}{2} - \left(\dot\theta_{1} \sin{\left(\theta_{2} \right)} + \dot\theta_{1} + \dot\theta_{2}\right) \sin{\left(\theta_{3} \right)}\\- \frac{\sqrt{2} \dot\theta_{1} \cos{\left(\theta_{2} \right)}}{2}\end{matrix}\right]$

$\displaystyle {}^3\omega_3 = \left[\begin{matrix}\frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right) \sin{\left(\theta_{3} \right)}}{2}\\\frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right) \cos{\left(\theta_{3} \right)}}{2}\\\dot\theta_{3} + \frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right)}{2}\end{matrix}\right]$

$\displaystyle {}^4v_4 = \left[\begin{matrix}\dot\theta_{1} \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} + \dot\theta_{1} \sin{\left(\theta_{4} \right)} + \frac{\sqrt{2} \dot\theta_{1} \sin{\left(\theta_{3} + \theta_{4} \right)} \cos{\left(\theta_{2} \right)}}{2} + \dot\theta_{1} \cos{\left(\theta_{3} + \theta_{4} \right)} + \dot\theta_{2} \sin{\left(\theta_{4} \right)} + \dot\theta_{2} \cos{\left(\theta_{3} + \theta_{4} \right)} + \sqrt{2} \dot\theta_{3} \sin{\left(\theta_{4} \right)}\\- \dot\theta_{1} \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{3} + \theta_{4} \right)} - \dot\theta_{1} \sin{\left(\theta_{3} + \theta_{4} \right)} + \frac{\sqrt{2} \dot\theta_{1} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)}}{2} + \dot\theta_{1} \cos{\left(\theta_{4} \right)} - \dot\theta_{2} \sin{\left(\theta_{3} + \theta_{4} \right)} + \dot\theta_{2} \cos{\left(\theta_{4} \right)} + \sqrt{2} \dot\theta_{3} \cos{\left(\theta_{4} \right)}\\- \frac{\sqrt{2} \dot\theta_{1} \cos{\left(\theta_{2} \right)}}{2} - \left(\dot\theta_{1} + \dot\theta_{2}\right) \cos{\left(\theta_{3} \right)}\end{matrix}\right]$

$\displaystyle {}^4\omega_4 = \left[\begin{matrix}\frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right) \sin{\left(\theta_{3} + \theta_{4} \right)}}{2}\\\frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right) \cos{\left(\theta_{3} + \theta_{4} \right)}}{2}\\\dot\theta_{3} + \dot\theta_{4} + \frac{\sqrt{2} \left(\dot\theta_{1} + \dot\theta_{2}\right)}{2}\end{matrix}\right]$

We can read off the Jacobian from the linear-angular velocities.

Or use the next function to calculate it explicitly:

```python
J = comp_jacobian(dh_params, joint_points, verbose=False)
```

$\displaystyle {}^4J = \left[\begin{matrix}- \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)} + \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} - \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)} + \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{4} \right)}}{2} + \frac{\sqrt{2} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} \right)}}{2} + \sin{\left(\theta_{4} \right)} + \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} & - \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)} + \sin{\left(\theta_{4} \right)} + \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} & \sqrt{2} \sin{\left(\theta_{4} \right)} & 0\\- \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} - \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{3} \right)} - \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{2} \right)}}{2} - \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} - \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{3} \right)} + \frac{\sqrt{2} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)}}{2} + \cos{\left(\theta_{4} \right)} & - \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)} - \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{3} \right)} + \cos{\left(\theta_{4} \right)} & \sqrt{2} \cos{\left(\theta_{4} \right)} & 0\\- \frac{\sqrt{2} \cos{\left(\theta_{2} \right)}}{2} - \cos{\left(\theta_{3} \right)} & - \cos{\left(\theta_{3} \right)} & 0 & 0\\\frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)}}{2} + \frac{\sqrt{2} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{3} \right)}}{2} & \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)}}{2} + \frac{\sqrt{2} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{3} \right)}}{2} & 0 & 0\\- \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)}}{2} + \frac{\sqrt{2} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)}}{2} & - \frac{\sqrt{2} \sin{\left(\theta_{3} \right)} \sin{\left(\theta_{4} \right)}}{2} + \frac{\sqrt{2} \cos{\left(\theta_{3} \right)} \cos{\left(\theta_{4} \right)}}{2} & 0 & 0\\\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} & 1 & 1\end{matrix}\right]$

```python
config = {
    sy.Symbol("theta_1"): 0,
    sy.Symbol("theta_2"): 90/180 * sy.pi,
    sy.Symbol("theta_3"): -90/180 * sy.pi,
    sy.Symbol("theta_4"): 0,
}

J_config = sy.simplify(J.subs(config))
J_config
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0\\3 & 2 & \sqrt{2} & 0\\0 & 0 & 0 & 0\\- \frac{\sqrt{2}}{2} & - \frac{\sqrt{2}}{2} & 0 & 0\\0 & 0 & 0 & 0\\\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} & 1 & 1\end{matrix}\right]$

Calculate the joint torques needed to support the external force-torque vector.

```python
sy.N(J_config.T @ sy.Matrix([0, 6, 0, 7, 0, 8]))
```

$\displaystyle \left[\begin{matrix}18.7071067811865\\12.7071067811865\\16.4852813742386\\8.0\end{matrix}\right]$

## Propagate force-torque vector

The force-torque vector at the end-effector is from the exercise.

```python
prop_force_torque(dh_params, joint_points, sympy.Matrix([0, 6, 0, 7, 0, 8]))
```

$\displaystyle {}^4f_4 = \left[\begin{matrix}0\\6\\0\end{matrix}\right]$

$\displaystyle {}^4n_4 = \left[\begin{matrix}7\\0\\8\end{matrix}\right]$

$\displaystyle {}^3f_3 = \left[\begin{matrix}- 6 \sin{\left(\theta_{4} \right)}\\6 \cos{\left(\theta_{4} \right)}\\0\end{matrix}\right]$

$\displaystyle {}^3n_3 = \left[\begin{matrix}7 \cos{\left(\theta_{4} \right)}\\7 \sin{\left(\theta_{4} \right)}\\6 \sqrt{2} \cos{\left(\theta_{4} \right)} + 8\end{matrix}\right]$

$\displaystyle {}^2f_2 = \left[\begin{matrix}- 6 \sin{\left(\theta_{3} + \theta_{4} \right)}\\3 \sqrt{2} \cos{\left(\theta_{3} + \theta_{4} \right)}\\3 \sqrt{2} \cos{\left(\theta_{3} + \theta_{4} \right)}\end{matrix}\right]$

$\displaystyle {}^2n_2 = \left[\begin{matrix}\left(7 - 6 \sqrt{2}\right) \cos{\left(\theta_{3} + \theta_{4} \right)}\\- 6 \sin{\left(\theta_{3} + \theta_{4} \right)} + \frac{7 \sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)}}{2} - 6 \cos{\left(\theta_{4} \right)} - 4 \sqrt{2}\\- 6 \sin{\left(\theta_{3} + \theta_{4} \right)} + \frac{7 \sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)}}{2} + 6 \cos{\left(\theta_{4} \right)} + 4 \sqrt{2}\end{matrix}\right]$

$\displaystyle {}^1f_1 = \left[\begin{matrix}- 3 \sqrt{2} \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} - 6 \sin{\left(\theta_{3} + \theta_{4} \right)} \cos{\left(\theta_{2} \right)}\\- 6 \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{3} + \theta_{4} \right)} + 3 \sqrt{2} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)}\\3 \sqrt{2} \cos{\left(\theta_{3} + \theta_{4} \right)}\end{matrix}\right]$

$\displaystyle {}^1n_1 = \left[\begin{matrix}\frac{\left(- 7 \sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)} + 12 \sin{\left(\theta_{3} + \theta_{4} \right)} + 12 \cos{\left(\theta_{4} \right)} + 8 \sqrt{2}\right) \sin{\left(\theta_{2} \right)}}{2} + \left(7 - 6 \sqrt{2}\right) \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)}\\- \frac{\left(- 7 \sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)} + 12 \sin{\left(\theta_{3} + \theta_{4} \right)} + 12 \cos{\left(\theta_{4} \right)} + 8 \sqrt{2}\right) \cos{\left(\theta_{2} \right)}}{2} + \left(7 - 6 \sqrt{2}\right) \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} - 3 \sqrt{2} \cos{\left(\theta_{3} + \theta_{4} \right)}\\- 6 \sin{\left(\theta_{2} \right)} \sin{\left(\theta_{3} + \theta_{4} \right)} - 6 \sin{\left(\theta_{3} + \theta_{4} \right)} + \frac{7 \sqrt{2} \sin{\left(\theta_{3} + \theta_{4} \right)}}{2} + 3 \sqrt{2} \cos{\left(\theta_{2} \right)} \cos{\left(\theta_{3} + \theta_{4} \right)} + 6 \cos{\left(\theta_{4} \right)} + 4 \sqrt{2}\end{matrix}\right]$

We can now read off the joint torques acting on the revolute joints from the Z-components of the torque vectors at each link.


```python
from sympy import *
from sympy.physics.quantum import TensorProduct
from sympy.physics.matrices import msigma
```

```python
def U(theta, phi, gamma):
    return Matrix(
        [
            [cos(theta / 2), -exp(1j * gamma) * sin(theta / 2)],
            [exp(1j * phi) * sin(theta / 2), exp(1j * (phi + gamma)) * cos(theta / 2)],
        ]
    )

def controlled_gate(U, switch=False):
    if not switch:
        return Matrix(BlockDiagMatrix(eye(2), U))
    else:
        return Matrix([[1, 0, 0, 0], [0, U[0, 0], 0, U[0, 1]], [0, 0, 1, 0], [0, U[1, 0], 0, U[1, 1]]])

CX = Matrix(BlockDiagMatrix(eye(2), Matrix([[0, 1], [1, 0]])))
CXT2 = Matrix([[1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0]])

alpha, beta = symbols("alpha, beta")
theta1, phi1, gamma1 = symbols("theta1, phi1, gamma1")
theta2, phi2, gamma2 = symbols("theta2, phi2, gamma2")
theta3, phi3, gamma3 = symbols("theta3, phi3, gamma3")
```

```python
U(theta1, phi1, gamma1)
```

$\displaystyle \left[\begin{matrix}\cos{\left(\frac{\theta_{1}}{2} \right)} & - e^{1.0 i \gamma_{1}} \sin{\left(\frac{\theta_{1}}{2} \right)}\\e^{1.0 i \phi_{1}} \sin{\left(\frac{\theta_{1}}{2} \right)} & e^{1.0 i \left(\gamma_{1} + \phi_{1}\right)} \cos{\left(\frac{\theta_{1}}{2} \right)}\end{matrix}\right]$

```python
D, e, t = symbols('Delta, varepsilon, t')

T = exp(1j * D * msigma(3) * t / 4) * exp(1j * e * msigma(2) * t / 2) * exp(1j * D * msigma(3) * t / 4)#

sol = solve(T - U(theta1, phi1, gamma1) * exp(1j * phi2), theta1, phi1, gamma1, phi2, dict=True)
```

```python
sol[0]
```

    {phi2: I*(0.346573590279973 - log((exp(I*t*varepsilon) + 1.0)*sqrt(1/(cos(t*varepsilon) + 1.0))*exp(0.5*I*t*(Delta - varepsilon)))),
     gamma1: -I*log(-exp(-0.5*I*Delta*t)),
     phi1: -I*log(-exp(-0.5*I*Delta*t)),
     theta1: 2.0*atan(tan(0.5*t*varepsilon))}

```python
simplify(U(theta1, phi1, gamma1).subs(sol[0]))
```

$\displaystyle \left[\begin{matrix}\cos{\left(1.0 \operatorname{atan}{\left(\tan{\left(0.5 t \varepsilon \right)} \right)} \right)} & - \left(- e^{- 0.5 i \Delta t}\right)^{1.0} \sin{\left(1.0 \operatorname{atan}{\left(\tan{\left(0.5 t \varepsilon \right)} \right)} \right)}\\\left(- e^{- 0.5 i \Delta t}\right)^{1.0} \sin{\left(1.0 \operatorname{atan}{\left(\tan{\left(0.5 t \varepsilon \right)} \right)} \right)} & \left(- e^{- 0.5 i \Delta t}\right)^{2.0} \cos{\left(1.0 \operatorname{atan}{\left(\tan{\left(0.5 t \varepsilon \right)} \right)} \right)}\end{matrix}\right]$

```python
init = TensorProduct(Matrix([alpha, beta]), Matrix([1, 0]))
init
```

$\displaystyle \left[\begin{matrix}\alpha\\0\\\beta\\0\end{matrix}\right]$

```python
# Loss
CU = Matrix(BlockDiagMatrix(eye(2), U(theta1, 0, 0)))
CUT2 = controlled_gate(U(theta1, phi1, gamma1), True)

circuit1 = CXT2 * CU
circuit1

```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & 0 & \sin{\left(\frac{\theta_{1}}{2} \right)} & \cos{\left(\frac{\theta_{1}}{2} \right)}\\0 & 0 & \cos{\left(\frac{\theta_{1}}{2} \right)} & - \sin{\left(\frac{\theta_{1}}{2} \right)}\\0 & 1 & 0 & 0\end{matrix}\right]$

```python
circuit1 * init
```

$\displaystyle \left[\begin{matrix}\alpha\\\beta \sin{\left(\frac{\theta_{1}}{2} \right)}\\\beta \cos{\left(\frac{\theta_{1}}{2} \right)}\\0\end{matrix}\right]$

```python
## Loss with fewer two qubit gates

L = (
    TensorProduct(U(0, 0, -pi / 2), U(0, 0, -pi / 2))
    * CXT2
    * TensorProduct(U(pi / 2, -pi / 2, 0), U(-theta1 / 2, pi, pi / 2))
    * CXT2
    * TensorProduct(U(pi / 2, -pi, 0), U(-theta1 / 2, -pi / 2, pi))
)
L = simplify(L)
L

```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & 0 & \sin{\left(\frac{\theta_{1}}{2} \right)} & \cos{\left(\frac{\theta_{1}}{2} \right)}\\0 & 0 & \cos{\left(\frac{\theta_{1}}{2} \right)} & - \sin{\left(\frac{\theta_{1}}{2} \right)}\\0 & 1 & 0 & 0\end{matrix}\right]$

```python
L * init
```

$\displaystyle \left[\begin{matrix}\alpha\\\beta \sin{\left(\frac{\theta_{1}}{2} \right)}\\\beta \cos{\left(\frac{\theta_{1}}{2} \right)}\\0\end{matrix}\right]$

```python
# Gain

U1 = TensorProduct(U(-pi, 0, 0), eye(2))
CU1 = controlled_gate(U(theta2, 0, 0), True)
U2 = TensorProduct(U(pi, 0, 0), eye(2))

# circuit = U2 * CX * CU1 * CX * U1
circuit2 = U2 * CX * CU1 * CX * U1
circuit2
```

$\displaystyle \left[\begin{matrix}\cos{\left(\frac{\theta_{2}}{2} \right)} & 0 & 0 & - \sin{\left(\frac{\theta_{2}}{2} \right)}\\0 & 1 & 0 & 0\\0 & 0 & 1 & 0\\\sin{\left(\frac{\theta_{2}}{2} \right)} & 0 & 0 & \cos{\left(\frac{\theta_{2}}{2} \right)}\end{matrix}\right]$

```python
circuit2 * init
```

$\displaystyle \left[\begin{matrix}\alpha \cos{\left(\frac{\theta_{2}}{2} \right)}\\0\\\beta\\\alpha \sin{\left(\frac{\theta_{2}}{2} \right)}\end{matrix}\right]$


# Frequentist Inference Case Study - Part A

## 1. Learning objectives

Welcome to part A of the Frequentist inference case study! The purpose of this case study is to help you apply the concepts associated with Frequentist inference in Python. Frequentist inference is the process of deriving conclusions about an underlying distribution via the observation of data. In particular, you'll practice writing Python code to apply the following statistical concepts:
* the _z_-statistic
* the _t_-statistic
* the difference and relationship between the two
* the Central Limit Theorem, including its assumptions and consequences
* how to estimate the population mean and standard deviation from a sample
* the concept of a sampling distribution of a test statistic, particularly for the mean
* how to combine these concepts to calculate a confidence interval

## Prerequisites

To be able to complete this notebook, you are expected to have a basic understanding of:
* what a random variable is (p.400 of Professor Spiegelhalter's *The Art of Statistics, hereinafter AoS*)
* what a population, and a population distribution, are (p. 397 of *AoS*)
* a high-level sense of what the normal distribution is (p. 394 of *AoS*)
* what the t-statistic is (p. 275 of *AoS*)

Happily, these should all be concepts with which you are reasonably familiar after having read ten chapters of Professor Spiegelhalter's book, *The Art of Statistics*.

We'll try to relate the concepts in this case study back to page numbers in *The Art of Statistics* so that you can focus on the Python aspects of this case study. The second part (part B) of this case study will involve another, more real-world application of these tools.

For this notebook, we will use data sampled from a known normal distribution. This allows us to compare our results with theoretical expectations.

## 2. An introduction to sampling from the normal distribution

First, let's explore the ways we can generate the normal distribution. While there's a fair amount of interest in [sklearn](https://scikit-learn.org/stable/) within the machine learning community, you're likely to have heard of [scipy](https://docs.scipy.org/doc/scipy-0.15.1/reference/index.html) if you're coming from the sciences. For this assignment, you'll use [scipy.stats](https://docs.scipy.org/doc/scipy-0.15.1/reference/tutorial/stats.html) to complete your work.

This assignment will require some digging around and getting your hands dirty (your learning is maximized that way)! You should have the research skills and the tenacity to do these tasks independently, but if you struggle, reach out to your immediate community and your mentor for help.

```python
from scipy.stats import norm
from scipy.stats import t
import numpy as np
import pandas as pd
from numpy.random import seed
import matplotlib.pyplot as plt
```

__Q1:__ Call up the documentation for the `norm` function imported above. (Hint: that documentation is [here](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.norm.html)). What is the second listed method?

```python
help(norm)

```

    Help on norm_gen in module scipy.stats._continuous_distns object:

    class norm_gen(scipy.stats._distn_infrastructure.rv_continuous)
     |  norm_gen(momtype=1, a=None, b=None, xtol=1e-14, badvalue=None, name=None, longname=None, shapes=None, extradoc=None, seed=None)
     |
     |  A normal continuous random variable.
     |
     |  The location (``loc``) keyword specifies the mean.
     |  The scale (``scale``) keyword specifies the standard deviation.
     |
     |  %(before_notes)s
     |
     |  Notes
     |  -----
     |  The probability density function for `norm` is:
     |
     |  .. math::
     |
     |      f(x) = \frac{\exp(-x^2/2)}{\sqrt{2\pi}}
     |
     |  for a real number :math:`x`.
     |
     |  %(after_notes)s
     |
     |  %(example)s
     |
     |  Method resolution order:
     |      norm_gen
     |      scipy.stats._distn_infrastructure.rv_continuous
     |      scipy.stats._distn_infrastructure.rv_generic
     |      builtins.object
     |
     |  Methods defined here:
     |
     |  fit(self, data, **kwds)
     |      Return MLEs for shape (if applicable), location, and scale
     |      parameters from data.
     |
     |      MLE stands for Maximum Likelihood Estimate.  Starting estimates for
     |      the fit are given by input arguments; for any arguments not provided
     |      with starting estimates, ``self._fitstart(data)`` is called to generate
     |      such.
     |
     |      One can hold some parameters fixed to specific values by passing in
     |      keyword arguments ``f0``, ``f1``, ..., ``fn`` (for shape parameters)
     |      and ``floc`` and ``fscale`` (for location and scale parameters,
     |      respectively).
     |
     |      Parameters
     |      ----------
     |      data : array_like
     |          Data to use in calculating the MLEs.
     |      arg1, arg2, arg3,... : floats, optional
     |          Starting value(s) for any shape-characterizing arguments (those not
     |          provided will be determined by a call to ``_fitstart(data)``).
     |          No default value.
     |      kwds : floats, optional
     |          - `loc`: initial guess of the distribution's location parameter.
     |          - `scale`: initial guess of the distribution's scale parameter.
     |
     |          Special keyword arguments are recognized as holding certain
     |          parameters fixed:
     |
     |          - f0...fn : hold respective shape parameters fixed.
     |            Alternatively, shape parameters to fix can be specified by name.
     |            For example, if ``self.shapes == "a, b"``, ``fa`` and ``fix_a``
     |            are equivalent to ``f0``, and ``fb`` and ``fix_b`` are
     |            equivalent to ``f1``.
     |
     |          - floc : hold location parameter fixed to specified value.
     |
     |          - fscale : hold scale parameter fixed to specified value.
     |
     |          - optimizer : The optimizer to use.  The optimizer must take ``func``,
     |            and starting position as the first two arguments,
     |            plus ``args`` (for extra arguments to pass to the
     |            function to be optimized) and ``disp=0`` to suppress
     |            output as keyword arguments.
     |
     |      Returns
     |      -------
     |      mle_tuple : tuple of floats
     |          MLEs for any shape parameters (if applicable), followed by those
     |          for location and scale. For most random variables, shape statistics
     |          will be returned, but there are exceptions (e.g. ``norm``).
     |
     |      Notes
     |      -----
     |      This function uses explicit formulas for the maximum likelihood
     |      estimation of the normal distribution parameters, so the
     |      `optimizer` argument is ignored.
     |
     |      Examples
     |      --------
     |
     |      Generate some data to fit: draw random variates from the `beta`
     |      distribution
     |
     |      >>> from scipy.stats import beta
     |      >>> a, b = 1., 2.
     |      >>> x = beta.rvs(a, b, size=1000)
     |
     |      Now we can fit all four parameters (``a``, ``b``, ``loc`` and ``scale``):
     |
     |      >>> a1, b1, loc1, scale1 = beta.fit(x)
     |
     |      We can also use some prior knowledge about the dataset: let's keep
     |      ``loc`` and ``scale`` fixed:
     |
     |      >>> a1, b1, loc1, scale1 = beta.fit(x, floc=0, fscale=1)
     |      >>> loc1, scale1
     |      (0, 1)
     |
     |      We can also keep shape parameters fixed by using ``f``-keywords. To
     |      keep the zero-th shape parameter ``a`` equal 1, use ``f0=1`` or,
     |      equivalently, ``fa=1``:
     |
     |      >>> a1, b1, loc1, scale1 = beta.fit(x, fa=1, floc=0, fscale=1)
     |      >>> a1
     |      1
     |
     |      Not all distributions return estimates for the shape parameters.
     |      ``norm`` for example just returns estimates for location and scale:
     |
     |      >>> from scipy.stats import norm
     |      >>> x = norm.rvs(a, b, size=1000, random_state=123)
     |      >>> loc1, scale1 = norm.fit(x)
     |      >>> loc1, scale1
     |      (0.92087172783841631, 2.0015750750324668)
     |
     |  ----------------------------------------------------------------------
     |  Methods inherited from scipy.stats._distn_infrastructure.rv_continuous:
     |
     |  __getstate__(self)
     |
     |  __init__(self, momtype=1, a=None, b=None, xtol=1e-14, badvalue=None, name=None, longname=None, shapes=None, extradoc=None, seed=None)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |
     |  cdf(self, x, *args, **kwds)
     |      Cumulative distribution function of the given RV.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      cdf : ndarray
     |          Cumulative distribution function evaluated at `x`
     |
     |  expect(self, func=None, args=(), loc=0, scale=1, lb=None, ub=None, conditional=False, **kwds)
     |      Calculate expected value of a function with respect to the
     |      distribution by numerical integration.
     |
     |      The expected value of a function ``f(x)`` with respect to a
     |      distribution ``dist`` is defined as::
     |
     |                  ub
     |          E[f(x)] = Integral(f(x) * dist.pdf(x)),
     |                  lb
     |
     |      where ``ub`` and ``lb`` are arguments and ``x`` has the ``dist.pdf(x)``
     |      distribution. If the bounds ``lb`` and ``ub`` correspond to the
     |      support of the distribution, e.g. ``[-inf, inf]`` in the default
     |      case, then the integral is the unrestricted expectation of ``f(x)``.
     |      Also, the function ``f(x)`` may be defined such that ``f(x)`` is ``0``
     |      outside a finite interval in which case the expectation is
     |      calculated within the finite range ``[lb, ub]``.
     |
     |      Parameters
     |      ----------
     |      func : callable, optional
     |          Function for which integral is calculated. Takes only one argument.
     |          The default is the identity mapping f(x) = x.
     |      args : tuple, optional
     |          Shape parameters of the distribution.
     |      loc : float, optional
     |          Location parameter (default=0).
     |      scale : float, optional
     |          Scale parameter (default=1).
     |      lb, ub : scalar, optional
     |          Lower and upper bound for integration. Default is set to the
     |          support of the distribution.
     |      conditional : bool, optional
     |          If True, the integral is corrected by the conditional probability
     |          of the integration interval.  The return value is the expectation
     |          of the function, conditional on being in the given interval.
     |          Default is False.
     |
     |      Additional keyword arguments are passed to the integration routine.
     |
     |      Returns
     |      -------
     |      expect : float
     |          The calculated expected value.
     |
     |      Notes
     |      -----
     |      The integration behavior of this function is inherited from
     |      `scipy.integrate.quad`. Neither this function nor
     |      `scipy.integrate.quad` can verify whether the integral exists or is
     |      finite. For example ``cauchy(0).mean()`` returns ``np.nan`` and
     |      ``cauchy(0).expect()`` returns ``0.0``.
     |
     |      The function is not vectorized.
     |
     |      Examples
     |      --------
     |
     |      To understand the effect of the bounds of integration consider
     |
     |      >>> from scipy.stats import expon
     |      >>> expon(1).expect(lambda x: 1, lb=0.0, ub=2.0)
     |      0.6321205588285578
     |
     |      This is close to
     |
     |      >>> expon(1).cdf(2.0) - expon(1).cdf(0.0)
     |      0.6321205588285577
     |
     |      If ``conditional=True``
     |
     |      >>> expon(1).expect(lambda x: 1, lb=0.0, ub=2.0, conditional=True)
     |      1.0000000000000002
     |
     |      The slight deviation from 1 is due to numerical integration.
     |
     |  fit_loc_scale(self, data, *args)
     |      Estimate loc and scale parameters from data using 1st and 2nd moments.
     |
     |      Parameters
     |      ----------
     |      data : array_like
     |          Data to fit.
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |
     |      Returns
     |      -------
     |      Lhat : float
     |          Estimated location parameter for the data.
     |      Shat : float
     |          Estimated scale parameter for the data.
     |
     |  isf(self, q, *args, **kwds)
     |      Inverse survival function (inverse of `sf`) at q of the given RV.
     |
     |      Parameters
     |      ----------
     |      q : array_like
     |          upper tail probability
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      x : ndarray or scalar
     |          Quantile corresponding to the upper tail probability q.
     |
     |  logcdf(self, x, *args, **kwds)
     |      Log of the cumulative distribution function at x of the given RV.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      logcdf : array_like
     |          Log of the cumulative distribution function evaluated at x
     |
     |  logpdf(self, x, *args, **kwds)
     |      Log of the probability density function at x of the given RV.
     |
     |      This uses a more numerically accurate calculation if available.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      logpdf : array_like
     |          Log of the probability density function evaluated at x
     |
     |  logsf(self, x, *args, **kwds)
     |      Log of the survival function of the given RV.
     |
     |      Returns the log of the "survival function," defined as (1 - `cdf`),
     |      evaluated at `x`.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      logsf : ndarray
     |          Log of the survival function evaluated at `x`.
     |
     |  nnlf(self, theta, x)
     |      Return negative loglikelihood function.
     |
     |      Notes
     |      -----
     |      This is ``-sum(log pdf(x, theta), axis=0)`` where `theta` are the
     |      parameters (including loc and scale).
     |
     |  pdf(self, x, *args, **kwds)
     |      Probability density function at x of the given RV.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      pdf : ndarray
     |          Probability density function evaluated at x
     |
     |  ppf(self, q, *args, **kwds)
     |      Percent point function (inverse of `cdf`) at q of the given RV.
     |
     |      Parameters
     |      ----------
     |      q : array_like
     |          lower tail probability
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      x : array_like
     |          quantile corresponding to the lower tail probability q.
     |
     |  sf(self, x, *args, **kwds)
     |      Survival function (1 - `cdf`) at x of the given RV.
     |
     |      Parameters
     |      ----------
     |      x : array_like
     |          quantiles
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      sf : array_like
     |          Survival function evaluated at x
     |
     |  ----------------------------------------------------------------------
     |  Methods inherited from scipy.stats._distn_infrastructure.rv_generic:
     |
     |  __call__(self, *args, **kwds)
     |      Freeze the distribution for the given arguments.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution.  Should include all
     |          the non-optional arguments, may include ``loc`` and ``scale``.
     |
     |      Returns
     |      -------
     |      rv_frozen : rv_frozen instance
     |          The frozen distribution.
     |
     |  __setstate__(self, state)
     |
     |  entropy(self, *args, **kwds)
     |      Differential entropy of the RV.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |      loc : array_like, optional
     |          Location parameter (default=0).
     |      scale : array_like, optional  (continuous distributions only).
     |          Scale parameter (default=1).
     |
     |      Notes
     |      -----
     |      Entropy is defined base `e`:
     |
     |      >>> drv = rv_discrete(values=((0, 1), (0.5, 0.5)))
     |      >>> np.allclose(drv.entropy(), np.log(2.0))
     |      True
     |
     |  freeze(self, *args, **kwds)
     |      Freeze the distribution for the given arguments.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution.  Should include all
     |          the non-optional arguments, may include ``loc`` and ``scale``.
     |
     |      Returns
     |      -------
     |      rv_frozen : rv_frozen instance
     |          The frozen distribution.
     |
     |  interval(self, alpha, *args, **kwds)
     |      Confidence interval with equal areas around the median.
     |
     |      Parameters
     |      ----------
     |      alpha : array_like of float
     |          Probability that an rv will be drawn from the returned range.
     |          Each value should be in the range [0, 1].
     |      arg1, arg2, ... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |      loc : array_like, optional
     |          location parameter, Default is 0.
     |      scale : array_like, optional
     |          scale parameter, Default is 1.
     |
     |      Returns
     |      -------
     |      a, b : ndarray of float
     |          end-points of range that contain ``100 * alpha %`` of the rv's
     |          possible values.
     |
     |  mean(self, *args, **kwds)
     |      Mean of the distribution.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      mean : float
     |          the mean of the distribution
     |
     |  median(self, *args, **kwds)
     |      Median of the distribution.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          Location parameter, Default is 0.
     |      scale : array_like, optional
     |          Scale parameter, Default is 1.
     |
     |      Returns
     |      -------
     |      median : float
     |          The median of the distribution.
     |
     |      See Also
     |      --------
     |      rv_discrete.ppf
     |          Inverse of the CDF
     |
     |  moment(self, n, *args, **kwds)
     |      n-th order non-central moment of distribution.
     |
     |      Parameters
     |      ----------
     |      n : int, n >= 1
     |          Order of moment.
     |      arg1, arg2, arg3,... : float
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |  rvs(self, *args, **kwds)
     |      Random variates of given type.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |      loc : array_like, optional
     |          Location parameter (default=0).
     |      scale : array_like, optional
     |          Scale parameter (default=1).
     |      size : int or tuple of ints, optional
     |          Defining number of random variates (default is 1).
     |      random_state : {None, int, `~np.random.RandomState`, `~np.random.Generator`}, optional
     |          If `seed` is `None` the `~np.random.RandomState` singleton is used.
     |          If `seed` is an int, a new ``RandomState`` instance is used, seeded
     |          with seed.
     |          If `seed` is already a ``RandomState`` or ``Generator`` instance,
     |          then that object is used.
     |          Default is None.
     |
     |      Returns
     |      -------
     |      rvs : ndarray or scalar
     |          Random variates of given `size`.
     |
     |  stats(self, *args, **kwds)
     |      Some statistics of the given RV.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional (continuous RVs only)
     |          scale parameter (default=1)
     |      moments : str, optional
     |          composed of letters ['mvsk'] defining which moments to compute:
     |          'm' = mean,
     |          'v' = variance,
     |          's' = (Fisher's) skew,
     |          'k' = (Fisher's) kurtosis.
     |          (default is 'mv')
     |
     |      Returns
     |      -------
     |      stats : sequence
     |          of requested moments.
     |
     |  std(self, *args, **kwds)
     |      Standard deviation of the distribution.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      std : float
     |          standard deviation of the distribution
     |
     |  support(self, *args, **kwargs)
     |      Return the support of the distribution.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, ... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information).
     |      loc : array_like, optional
     |          location parameter, Default is 0.
     |      scale : array_like, optional
     |          scale parameter, Default is 1.
     |      Returns
     |      -------
     |      a, b : float
     |          end-points of the distribution's support.
     |
     |  var(self, *args, **kwds)
     |      Variance of the distribution.
     |
     |      Parameters
     |      ----------
     |      arg1, arg2, arg3,... : array_like
     |          The shape parameter(s) for the distribution (see docstring of the
     |          instance object for more information)
     |      loc : array_like, optional
     |          location parameter (default=0)
     |      scale : array_like, optional
     |          scale parameter (default=1)
     |
     |      Returns
     |      -------
     |      var : float
     |          the variance of the distribution
     |
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from scipy.stats._distn_infrastructure.rv_generic:
     |
     |  __dict__
     |      dictionary for instance variables (if defined)
     |
     |  __weakref__
     |      list of weak references to the object (if defined)
     |
     |  random_state
     |      Get or set the RandomState object for generating random variates.
     |
     |      This can be either None, int, a RandomState instance, or a
     |      np.random.Generator instance.
     |
     |      If None (or np.random), use the RandomState singleton used by np.random.
     |      If already a RandomState or Generator instance, use it.
     |      If an int, use a new RandomState instance seeded with seed.

__A:__ In the linked documentation: pdf(x, loc=0, scale=1)
 As returned from the help call: cdf(self, x, *args, **kwds)

__Q2:__ Use the method that generates random variates to draw five samples from the standard normal distribution.

__A:__

```python
seed(47)
# draw five samples here
A=np.random.normal(loc=0, scale=1,size=5)
print(A)
```

    [-0.84800948  1.30590636  0.92420797  0.6404118  -1.05473698]

__Q3:__ What is the mean of this sample? Is it exactly equal to the value you expected? Hint: the sample was drawn from the standard normal distribution. If you want a reminder of the properties of this distribution, check out p. 85 of *AoS*.

__A:__

```python
# Calculate and print the mean here, hint: use np.mean()
B=np.mean(A)
print(B)
```

    0.19355593334131074

__Q4:__ What is the standard deviation of these numbers? Calculate this manually here as $\sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n}}$ (This is just the definition of **standard deviation** given by Professor Spiegelhalter on p.403 of *AoS*). Hint: np.sqrt() and np.sum() will be useful here and remember that numPy supports [broadcasting](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html).

__A:__

```python
print(np.sqrt(np.sum((A-B)**2)/len(A)))
```

    0.9606195639478641

Here we have calculated the actual standard deviation of a small data set (of size 5). But in this case, this small data set is actually a sample from our larger (infinite) population. In this case, the population is infinite because we could keep drawing our normal random variates until our computers die!

In general, the sample mean we calculate will not be equal to the population mean (as we saw above). A consequence of this is that the sum of squares of the deviations from the _population_ mean will be bigger than the sum of squares of the deviations from the _sample_ mean. In other words, the sum of squares of the deviations from the _sample_ mean is too small to give an unbiased estimate of the _population_ variance. An example of this effect is given [here](https://en.wikipedia.org/wiki/Bessel%27s_correction#Source_of_bias). Scaling our estimate of the variance by the factor $n/(n-1)$ gives an unbiased estimator of the population variance. This factor is known as [Bessel's correction](https://en.wikipedia.org/wiki/Bessel%27s_correction). The consequence of this is that the $n$ in the denominator is replaced by $n-1$.

You can see Bessel's correction reflected in Professor Spiegelhalter's definition of **variance** on p. 405 of *AoS*.

__Q5:__ If all we had to go on was our five samples, what would be our best estimate of the population standard deviation? Use Bessel's correction ($n-1$ in the denominator), thus $\sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n-1}}$.

__A:__

```python
print(np.sqrt(np.sum((A-B)**2)/(len(A)-1)))
```

    1.0740053227518152

__Q6:__ Now use numpy's std function to calculate the standard deviation of our random samples. Which of the above standard deviations did it return?

__A:__

```python
np.std(A)
```

    0.9606195639478641

__Q7:__ Consult the documentation for np.std() to see how to apply the correction for estimating the population parameter and verify this produces the expected result.

__A:__

```python
help(np.std)
```

    Help on function std in module numpy:

    std(a, axis=None, dtype=None, out=None, ddof=0, keepdims=<no value>, *, where=<no value>)
        Compute the standard deviation along the specified axis.

        Returns the standard deviation, a measure of the spread of a distribution,
        of the array elements. The standard deviation is computed for the
        flattened array by default, otherwise over the specified axis.

        Parameters
        ----------
        a : array_like
            Calculate the standard deviation of these values.
        axis : None or int or tuple of ints, optional
            Axis or axes along which the standard deviation is computed. The
            default is to compute the standard deviation of the flattened array.

            .. versionadded:: 1.7.0

            If this is a tuple of ints, a standard deviation is performed over
            multiple axes, instead of a single axis or all the axes as before.
        dtype : dtype, optional
            Type to use in computing the standard deviation. For arrays of
            integer type the default is float64, for arrays of float types it is
            the same as the array type.
        out : ndarray, optional
            Alternative output array in which to place the result. It must have
            the same shape as the expected output but the type (of the calculated
            values) will be cast if necessary.
        ddof : int, optional
            Means Delta Degrees of Freedom.  The divisor used in calculations
            is ``N - ddof``, where ``N`` represents the number of elements.
            By default `ddof` is zero.
        keepdims : bool, optional
            If this is set to True, the axes which are reduced are left
            in the result as dimensions with size one. With this option,
            the result will broadcast correctly against the input array.

            If the default value is passed, then `keepdims` will not be
            passed through to the `std` method of sub-classes of
            `ndarray`, however any non-default value will be.  If the
            sub-class' method does not implement `keepdims` any
            exceptions will be raised.

        where : array_like of bool, optional
            Elements to include in the standard deviation.
            See `~numpy.ufunc.reduce` for details.

            .. versionadded:: 1.20.0

        Returns
        -------
        standard_deviation : ndarray, see dtype parameter above.
            If `out` is None, return a new array containing the standard deviation,
            otherwise return a reference to the output array.

        See Also
        --------
        var, mean, nanmean, nanstd, nanvar
        :ref:`ufuncs-output-type`

        Notes
        -----
        The standard deviation is the square root of the average of the squared
        deviations from the mean, i.e., ``std = sqrt(mean(x))``, where
        ``x = abs(a - a.mean())**2``.

        The average squared deviation is typically calculated as ``x.sum() / N``,
        where ``N = len(x)``. If, however, `ddof` is specified, the divisor
        ``N - ddof`` is used instead. In standard statistical practice, ``ddof=1``
        provides an unbiased estimator of the variance of the infinite population.
        ``ddof=0`` provides a maximum likelihood estimate of the variance for
        normally distributed variables. The standard deviation computed in this
        function is the square root of the estimated variance, so even with
        ``ddof=1``, it will not be an unbiased estimate of the standard deviation
        per se.

        Note that, for complex numbers, `std` takes the absolute
        value before squaring, so that the result is always real and nonnegative.

        For floating-point input, the *std* is computed using the same
        precision the input has. Depending on the input data, this can cause
        the results to be inaccurate, especially for float32 (see example below).
        Specifying a higher-accuracy accumulator using the `dtype` keyword can
        alleviate this issue.

        Examples
        --------
        >>> a = np.array([[1, 2], [3, 4]])
        >>> np.std(a)
        1.1180339887498949 # may vary
        >>> np.std(a, axis=0)
        array([1.,  1.])
        >>> np.std(a, axis=1)
        array([0.5,  0.5])

        In single precision, std() can be inaccurate:

        >>> a = np.zeros((2, 512*512), dtype=np.float32)
        >>> a[0, :] = 1.0
        >>> a[1, :] = 0.1
        >>> np.std(a)
        0.45000005

        Computing the standard deviation in float64 is more accurate:

        >>> np.std(a, dtype=np.float64)
        0.44999999925494177 # may vary

        Specifying a where argument:

        >>> a = np.array([[14, 8, 11, 10], [7, 9, 10, 11], [10, 15, 5, 10]])
        >>> np.std(a)
        2.614064523559687 # may vary
        >>> np.std(a, where=[[True], [True], [False]])
        2.0

```python
print(np.std(A,ddof=1))
```

    1.0740053227518152

### Summary of section

In this section, you've been introduced to the scipy.stats package and used it to draw a small sample from the standard normal distribution. You've calculated the average (the mean) of this sample and seen that this is not exactly equal to the expected population parameter (which we know because we're generating the random variates from a specific, known distribution). You've been introduced to two ways of calculating the standard deviation; one uses $n$ in the denominator and the other uses $n-1$ (Bessel's correction). You've also seen which of these calculations np.std() performs by default and how to get it to generate the other.

You use $n$ as the denominator if you want to calculate the standard deviation of a sequence of numbers. You use $n-1$ if you are using this sequence of numbers to estimate the population parameter. This brings us to some terminology that can be a little confusing.

The population parameter is traditionally written as $\sigma$ and the sample statistic as $s$. Rather unhelpfully, $s$ is also called the sample standard deviation (using $n-1$) whereas the standard deviation of the sample uses $n$. That's right, we have the sample standard deviation and the standard deviation of the sample and they're not the same thing!

The sample standard deviation
\begin{equation}
s = \sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n-1}} \approx \sigma,
\end{equation}
is our best (unbiased) estimate of the population parameter ($\sigma$).

If your dataset _is_ your entire population, you simply want to calculate the population parameter, $\sigma$, via
\begin{equation}
\sigma = \sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n}}
\end{equation}
as you have complete, full knowledge of your population. In other words, your sample _is_ your population. It's worth noting that we're dealing with what Professor Spiegehalter describes on p. 92 of *AoS* as a **metaphorical population**: we have all the data, and we act as if the data-point is taken from a population at random. We can think of this population as an imaginary space of possibilities.

If, however, you have sampled _from_ your population, you only have partial knowledge of the state of your population. In this case, the standard deviation of your sample is not an unbiased estimate of the standard deviation of the population, in which case you seek to estimate that population parameter via the sample standard deviation, which uses the $n-1$ denominator.

Great work so far! Now let's dive deeper.

## 3. Sampling distributions

So far we've been dealing with the concept of taking a sample from a population to infer the population parameters. One statistic we calculated for a sample was the mean. As our samples will be expected to vary from one draw to another, so will our sample statistics. If we were to perform repeat draws of size $n$ and calculate the mean of each, we would expect to obtain a distribution of values. This is the sampling distribution of the mean. **The Central Limit Theorem (CLT)** tells us that such a distribution will approach a normal distribution as $n$ increases (the intuitions behind the CLT are covered in full on p. 236 of *AoS*). For the sampling distribution of the mean, the standard deviation of this distribution is given by

\begin{equation}
\sigma_{mean} = \frac{\sigma}{\sqrt n}
\end{equation}

where $\sigma_{mean}$ is the standard deviation of the sampling distribution of the mean and $\sigma$ is the standard deviation of the population (the population parameter).

This is important because typically we are dealing with samples from populations and all we know about the population is what we see in the sample. From this sample, we want to make inferences about the population. We may do this, for example, by looking at the histogram of the values and by calculating the mean and standard deviation (as estimates of the population parameters), and so we are intrinsically interested in how these quantities vary across samples.

In other words, now that we've taken one sample of size $n$ and made some claims about the general population, what if we were to take another sample of size $n$? Would we get the same result? Would we make the same claims about the general population? This brings us to a fundamental question: _when we make some inference about a population based on our sample, how confident can we be that we've got it 'right'?_

We need to think about **estimates and confidence intervals**: those concepts covered in Chapter 7, p. 189, of *AoS*.

Now, the standard normal distribution (with its variance equal to its standard deviation of one) would not be a great illustration of a key point. Instead, let's imagine we live in a town of 50,000 people and we know the height of everyone in this town. We will have 50,000 numbers that tell us everything about our population. We'll simulate these numbers now and put ourselves in one particular town, called 'town 47', where the population mean height is 172 cm and population standard deviation is 5 cm.

```python
seed(47)
pop_heights = norm.rvs(172, 5, size=50000)
```

```python
_ = plt.hist(pop_heights, bins=30)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of heights in entire town population')
_ = plt.axvline(172, color='r')
_ = plt.axvline(172+5, color='r', linestyle='--')
_ = plt.axvline(172-5, color='r', linestyle='--')
_ = plt.axvline(172+10, color='r', linestyle='-.')
_ = plt.axvline(172-10, color='r', linestyle='-.')
```

Now, 50,000 people is rather a lot to chase after with a tape measure. If all you want to know is the average height of the townsfolk, then can you just go out and measure a sample to get a pretty good estimate of the average height?

```python
def townsfolk_sampler(n):
    return np.random.choice(pop_heights, n)
```

Let's say you go out one day and randomly sample 10 people to measure.

```python
seed(47)
daily_sample1 = townsfolk_sampler(10)
```

```python
_ = plt.hist(daily_sample1, bins=10)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of heights in sample size 10')
```

The sample distribution doesn't resemble what we take the population distribution to be. What do we get for the mean?

```python
np.mean(daily_sample1)
```

    173.47911444163503

And if we went out and repeated this experiment?

```python
daily_sample2 = townsfolk_sampler(10)
```

```python
np.mean(daily_sample2)
```

    173.7317666636263

__Q8:__ Simulate performing this random trial every day for a year, calculating the mean of each daily sample of 10, and plot the resultant sampling distribution of the mean.

__A:__

```python
daily_mean=[]
days_of_year=len(range(365))
i=0
```

```python
seed(47)
# take your samples here
year_of_samples = [np.mean(townsfolk_sampler(10)) for i in range(365)]

```

```python
_ = plt.hist(year_of_samples, bins=10)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of the mean heights in sample size 10')
```

The above is the distribution of the means of samples of size 10 taken from our population. The Central Limit Theorem tells us the expected mean of this distribution will be equal to the population mean, and standard deviation will be $\sigma / \sqrt n$, which, in this case, should be approximately 1.58.

__Q9:__ Verify the above results from the CLT.

__A:__

```python
mean_y_o_s=np.mean(year_of_samples)
print(mean_y_o_s)
std_y_o_s=np.std(year_of_samples,ddof=0)#/np.sqrt(len(year_of_samples))
print(std_y_o_s)
```

    171.8660049358649
    1.5756704135286475

```python

```

Remember, in this instance, we knew our population parameters, that the average height really is 172 cm and the standard deviation is 5 cm, and we see some of our daily estimates of the population mean were as low as around 168 and some as high as 176.

__Q10:__ Repeat the above year's worth of samples but for a sample size of 50 (perhaps you had a bigger budget for conducting surveys that year)! Would you expect your distribution of sample means to be wider (more variable) or narrower (more consistent)? Compare your resultant summary statistics to those predicted by the CLT.

__A:__

```python
seed(47)
# calculate daily means from the larger sample size here
year_of_samples_50 = [np.mean(townsfolk_sampler(50)) for i in range(365)]

```

```python
_ = plt.hist(year_of_samples_50, bins=10)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of the mean heights in sample size 50')
```

What we've seen so far, then, is that we can estimate population parameters from a sample from the population, and that samples have their own distributions. Furthermore, the larger the sample size, the narrower are those sampling distributions.

### Normally testing time!

All of the above is well and good. We've been sampling from a population we know is normally distributed, we've come to understand when to use $n$ and when to use $n-1$ in the denominator to calculate the spread of a distribution, and we've  seen the Central Limit Theorem in action for a sampling distribution. All seems very well behaved in Frequentist land. But, well, why should we really care?

Remember, we rarely (if ever) actually know our population parameters but we still have to estimate them somehow. If we want to make inferences to conclusions like "this observation is unusual" or "my population mean has changed" then we need to have some idea of what the underlying distribution is so we can calculate relevant probabilities. In frequentist inference, we use the formulae above to deduce these population parameters. Take a moment in the next part of this assignment to refresh your understanding of how these probabilities work.

Recall some basic properties of the standard normal distribution, such as that about 68% of observations are within plus or minus 1 standard deviation of the mean. Check out the precise definition of a normal distribution on p. 394 of *AoS*.

__Q11:__ Using this fact, calculate the probability of observing the value 1 or less in a single observation from the standard normal distribution. Hint: you may find it helpful to sketch the standard normal distribution (the familiar bell shape) and mark the number of standard deviations from the mean on the x-axis and shade the regions of the curve that contain certain percentages of the population.

__A:__ 84%

Calculating this probability involved calculating the area under the curve from the value of 1 and below. To put it in mathematical terms, we need to *integrate* the probability density function. We could just add together the known areas of chunks (from -Inf to 0 and then 0 to $+\sigma$ in the example above). One way to do this is to look up tables (literally). Fortunately, scipy has this functionality built in with the cdf() function.

__Q12:__ Use the cdf() function to answer the question above again and verify you get the same answer.

__A:__

```python

print(norm.cdf(1,0,1))
```

    0.8413447460685429

__Q13:__ Using our knowledge of the population parameters for our townsfolks' heights, what is the probability of selecting one person at random and their height being 177 cm or less? Calculate this using both of the approaches given above.

__A:__

```python
print(norm.cdf(177,172,5))
```

    0.8413447460685429

__Q14:__ Turning this question around — suppose we randomly pick one person and measure their height and find they are 2.00 m tall. How surprised should we be at this result, given what we know about the population distribution? In other words, how likely would it be to obtain a value at least as extreme as this? Express this as a probability.

__A:__

```python
print("1-" + str(norm.cdf(200,172,5)) +"="+ str(1-norm.cdf(200,172,5)))
```

    1-0.9999999892824097=1.0717590259723409e-08

What we've just done is calculate the ***p-value*** of the observation of someone 2.00m tall (review *p*-values if you need to on p. 399 of *AoS*). We could calculate this probability by virtue of knowing the population parameters. We were then able to use the known properties of the relevant normal distribution to calculate the probability of observing a value at least as extreme as our test value.

We're about to come to a pinch, though. We've said a couple of times that we rarely, if ever, know the true population parameters; we have to estimate them from our sample and we cannot even begin to estimate the standard deviation from a single observation.

This is very true and usually we have sample sizes larger than one. This means we can calculate the mean of the sample as our best estimate of the population mean and the standard deviation as our best estimate of the population standard deviation.

In other words, we are now coming to deal with the sampling distributions we mentioned above as we are generally concerned with the properties of the sample means we obtain.

Above, we highlighted one result from the CLT, whereby the sampling distribution (of the mean) becomes narrower and narrower with the square root of the sample size. We remind ourselves that another result from the CLT is that _even if the underlying population distribution is not normal, the sampling distribution will tend to become normal with sufficiently large sample size_. (**Check out p. 199 of AoS if you need to revise this**). This is the key driver for us 'requiring' a certain sample size, for example you may frequently see a minimum sample size of 30 stated in many places. In reality this is simply a rule of thumb; if the underlying distribution is approximately normal then your sampling distribution will already be pretty normal, but if the underlying distribution is heavily skewed then you'd want to increase your sample size.

__Q15:__ Let's now start from the position of knowing nothing about the heights of people in our town.
* Use the random seed of 47, to randomly sample the heights of 50 townsfolk
* Estimate the population mean using np.mean
* Estimate the population standard deviation using np.std (remember which denominator to use!)
* Calculate the (95%) [margin of error](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/hypothesis-testing/margin-of-error/#WhatMofE) (use the exact critial z value to 2 decimal places - [look this up](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/find-critical-values/) or use norm.ppf()) Recall that the ***margin of error*** is mentioned on p. 189 of the *AoS* and discussed in depth in that chapter).
* Calculate the 95% Confidence Interval of the mean (***confidence intervals*** are defined on p. 385 of *AoS*)
* Does this interval include the true population mean?

__A:__

```python
seed(47)
# take your sample now
daily_sample3 = townsfolk_sampler(50)
print(daily_sample3)
```

    [179.04949004 168.54189384 181.71974048 176.56433733 165.91968052
     170.65629646 179.37664509 166.40103988 177.15105034 169.41097044
     169.96353026 175.15960886 177.08203296 175.57499516 174.54290109
     172.47967099 168.61822612 174.73274884 170.88926244 178.27468993
     171.52874629 173.48970606 172.0752456  172.29707499 171.58202648
     169.84480352 178.67104246 174.28346436 174.29645483 167.42082214
     168.57939161 169.54392578 167.58621293 176.8657094  164.25755366
     177.38929575 171.97964488 164.94136054 177.90675289 175.2758845
     176.64336688 169.68013675 175.41851966 171.9408919  179.69843965
     169.93651872 169.3320981  172.17010876 171.18783138 171.14370134]

```python
pop_mean3=np.mean(daily_sample3)
print("Population mean: " + str(pop_mean3))
```

    Population mean: 172.7815108576788

```python
pop_std3= np.std(daily_sample3,ddof=1)
print("Population standard deviation: " + str(pop_std3))
```

    Population standard deviation: 4.195424364433547

```python
# using a 95%  2 tail zscore
z_score=norm.ppf(.975).round(2)
print("two tail z score factor:" + str(z_score))
margin_of_error= z_score*(pop_std3/np.sqrt(50))
print("Margin of error:" + str(margin_of_error))
```

    two tail z score factor:1.96
    Margin of error:1.162912303074119

```python
upper_limit= pop_mean3+margin_of_error
lower_limit= pop_mean3-margin_of_error

print( "Lower control limit: " + str(lower_limit) , "Upper control limit: " + str(upper_limit), sep='\n')
```

    Lower control limit: 171.6185985546047
    Upper control limit: 173.9444231607529

there is a 95% chance that the population mean lies between the lower control limit and the upper control limit

__Q16:__ Above, we calculated the confidence interval using the critical z value. What is the problem with this? What requirement, or requirements, are we (strictly) failing?

__A:__ z score assumes statisics are representative of the population.

__Q17:__ Calculate the 95% confidence interval for the mean using the _t_ distribution. Is this wider or narrower than that based on the normal distribution above? If you're unsure, you may find this [resource](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/confidence-interval/) useful. For calculating the critical value, remember how you could calculate this for the normal distribution using norm.ppf().

__A:__

```python
t_score= t.ppf(q=(1-.05)/2,df=49)
print("two tail t score factor:" + str(t_score))
t_margin_of_error= t_score*(pop_std3/np.sqrt(50))
print("t margin of error:" + str(t_margin_of_error))
t_upper_limit= pop_mean3+t_margin_of_error
t_lower_limit= pop_mean3-t_margin_of_error
print( "t Lower control limit: " + str(t_lower_limit) , "t Upper control limit: " + str(t_upper_limit), sep='\n')
```

    two tail t score factor:-0.06302878095516458
    t margin of error:-0.03739640041863501
    t Lower control limit: 172.81890725809743
    t Upper control limit: 172.74411445726017

confidence interval is wider than the confidence interval using the z score.

```python

```

This is slightly wider than the previous confidence interval. This reflects the greater uncertainty given that we are estimating population parameters from a sample.

## 4. Learning outcomes

Having completed this project notebook, you now have hands-on experience:
* sampling and calculating probabilities from a normal distribution
* identifying the correct way to estimate the standard deviation of a population (the population parameter) from a sample
* with sampling distribution and now know how the Central Limit Theorem applies
* with how to calculate critical values and confidence intervals

```python

```

notebooks/Frequentist Inference Case Study - Part A (3).ipynb