# Bayesian analysis of neutrality probability of mutations

Method based on that detailed at http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/, modifying code from the challenger disaster example.

First, import required modules

```python
import pymc3 as pm
import pandas as pd
import numpy as np
import theano.tensor as tt
import matplotlib.pyplot as plt
```

Next import the curated mutation data table, including the impact statistics from mutfunc. Then process the data to prepare it for analysis.

```python
impact = pd.read_table('../data/jelier.impact')
## Add bits of panda processing (e.g. set mutation type)
impact.head()
```

      id
      mut_id
      ref_codon
      alt_codon
      pos_aa
      ref_aa
      alt_aa
      gene
      type
      acc
      ...
      foldx_evidence
      foldx_int_ddG
      foldx_int_ddG_sd
      foldx_int_evidence
      foldx_int_interactor
      pho_prob
      ptm_modification
      tf_score_diff
      tf_perc_diff
      effect

      0
      YDR123C R237Q
      NaN
      NaN
      NaN
      237
      R
      Q
      YDR123C
      NaN
      P26798
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1

      1
      YDR123C R273K
      NaN
      NaN
      NaN
      273
      R
      K
      YDR123C
      NaN
      P26798
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1

      2
      YDR123C P275L
      NaN
      NaN
      NaN
      275
      P
      L
      YDR123C
      NaN
      P26798
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1

      3
      YNR003C D171H
      NaN
      NaN
      NaN
      171
      D
      H
      YNR003C
      NaN
      P32910
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1

      4
      YNR003C K135A
      NaN
      NaN
      NaN
      135
      K
      A
      YNR003C
      NaN
      P32910
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1

5 rows × 25 columns

## Relationship between SIFT score and Probability of neutrality

```python
sift_data = impact[~np.isnan(impact.sift_score)]
sift_data = sift_data.loc[:,["sift_score","effect"]]
sift_data['neutral'] = pd.Series(abs(sift_data.effect - 1), index=sift_data.index)

plt.scatter(sift_data.sift_score, sift_data.effect)
plt.yticks([0, 1])
plt.ylabel("Mutation Impact")
plt.xlabel("SIFT Score")
plt.title("Incidence of functional mutations with SIFT score");
```

We use the effset logistic distribution on the logarithm of the sift score to model the probability:

\begin{equation}
p = \dfrac{1}{1 + e^{\alpha \log{x} + \beta}}
\end{equation}

This gives two parameters to estimate, which can be done using MCMC.

```python
sift_data2 = sift_data.sample(1000)
sift_data2['sift_log'] = np.log(sift_data2.sift_score)

with pm.Model() as model:
    beta = pm.Normal("beta", mu=0, tau=0.0001, testval=0)
    alpha = pm.Normal("alpha", mu=0, tau=0.0001, testval=0)
    p = pm.Deterministic("p", 1.0/(1. + beta * sift_data2['sift_score'].values ** alpha ))

    observed = pm.Bernoulli("bernoulli_obs", p, observed=sift_data2.neutral)
    start = pm.find_MAP()
    #step = pm.Metropolis()
    trace = pm.sample(1000, step=step, start=start, njobs=1)
    burned_trace = trace[500::2]
```

    logp = nan, ||grad|| = 0: 100%|██████████| 42/42 [00:00<00:00, 2172.12it/s]
    /Users/ally/.local/share/virtualenvs/hog-TxCN4_y4/lib/python3.6/site-packages/pymc3/model.py:384: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      if not np.issubdtype(var.dtype, float):

This gives us samples from the posterior of $\alpha$ and $\beta$

```python
alpha_samples = burned_trace["alpha"][:, None]
beta_samples = burned_trace["beta"][:, None]

#histogram of the samples:
plt.subplot(211)
plt.title(r"Posterior distributions of the variables $\alpha, \beta$")
plt.hist(beta_samples, histtype='stepfilled', bins=35, alpha=0.85,
         label=r"posterior of $\beta$", color="#7A68A6", density=True)
plt.legend()

plt.subplot(212)
plt.hist(alpha_samples, histtype='stepfilled', bins=35, alpha=0.85,
         label=r"posterior of $\alpha$", color="#A60628", density=True)
plt.legend();
```

```python
s = np.linspace(0, 1, 100)[:, None]
p_s = 1.0 / (1 + np.exp(np.dot(alpha_samples, s.T) + beta_samples))

mean_prob_s = p_s.mean(axis=0)
```

```python
plt.plot(s, mean_prob_s)

from scipy.stats.mstats import mquantiles

qs = mquantiles(p_s, [0.025, 0.975], axis=0)
plt.fill_between(s[:, 0], *qs, alpha=0.7,
                 color="#7A68A6")

plt.scatter(sift_data.sift_score, sift_data.neutral)

plt.ylim(-0.1, 1.1)
plt.xlim(-0.1, 1.1)
plt.ylabel("Probability of Neutrality")
plt.xlabel("SIFT Score")
plt.title("Bayesian Neutral Probability");
```

```python

```


Demo - Poisson equation 2D
=======================

Solve Poisson's equation in 2D with homogeneous Dirichlet bcs in one direction and periodicity in the other.

$$
\begin{align}
\nabla^2 u(x, y) &= f(x, y), \quad \forall \, (x, y) \in [-1, 1] \times [0, 2\pi]\\
u(\pm 1, y) &= 0 \\
u(x, 2\pi) &= u(x, 0)
\end{align}
$$

where $u(x, y)$ is the solution and $f(x, y)$ is some right hand side function.

Use either Chebyshev basis $P=\{T_k(x)\}_{k=0}^{N_0-1}$ or Legendre $P=\{L_k(x)\}_{k=0}^{N_0-1}$ and define Shen's composite Dirichlet basis as

$$
V^{N_0}(x) = \{P_k(x) - P_{k+2}(x)\, | \, k=0, 1, \ldots, N_0-3\}.
$$

For the periodic direction use Fourier exponentials

$$
V^{N_1}(y) = \{\exp(i l y)\, | \, l=-N_1/2, -N_1/2+1, \ldots, N_1/2-1\}.
$$

And then define tensor product space as an outer product of these spaces

$$
V^N(x, y) = V^{N_0}(x) \times V^{N_1}(y).
$$

We get the test function

$$
\phi_{kl}(x, y) = (P_k(x) - P_{k+2}(x))\exp(i l y),
$$

and define for simplicity

$$
\begin{align}
v(x, y) &= \phi_{kl}(x, y), \\
u(x, y) &= \sum_{k=0}^{N_0-3}\sum_{l=-N_1/2}^{N_1/2-1} \hat{u}_{kl} \phi_{kl}(x, y),
\end{align}
$$

where $u(x, y)$ is the trial function.

The weighted inner product is defined almost exactly like in 1D, however, we now have to take into account that the solution is complex valued. The inner product is now

$$
(u, v)_w = \int_{-1}^{1}\int_{0}^{2\pi} u v^* w dxdy,
$$

where $v^*$ is the complex conjugate of $v$. Furthermore, we use the constant weight $w(x, y)=1/(2\pi)$ for Legendre/Fourier and get

Find $u \in V^N$ such that

$$     (\nabla u, \nabla v)_w = -(f, v)_w, \quad \forall \, v \in V^N.$$

For Chebyshev the weight is $1/\sqrt{1-x^2}/(2\pi)$ and we do not perform integration by parts:

Find $u \in V^N$ such that

$$     (\nabla^2 u, v)_w = (f, v)_w, \quad \forall \, v \in V^N.$$

## Implementation using shenfun

```python
from shenfun import *
import matplotlib.pyplot as plt

N = (16, 12)
BX = FunctionSpace(N[0], 'L', bc=(0, 0))
BY = FunctionSpace(N[1], 'F')
V = TensorProductSpace(comm, (BX, BY))
```

```python
v = TestFunction(V)
u = TrialFunction(V)
A = inner(grad(u), grad(v))
```

```python
print(A)
```

`TPMatrix` is a tensor product matrix. It is the outer product of two smaller matrices. Consider the inner product:

$$
\begin{align}
(\nabla u, \nabla v) &= \frac{1}{2\pi}\int_{-1}^{1}\int_{0}^{2\pi} \left(\frac{\partial u}{\partial x}, \frac{\partial u}{\partial y}\right) \cdot \left(\frac{\partial v^*}{\partial x}, \frac{\partial v^*}{\partial y}\right) {dxdy} \\
(\nabla u, \nabla v) &= \frac{1}{2\pi} \int_{-1}^1 \int_{0}^{2\pi} \left( \frac{\partial u}{\partial x}\frac{\partial v^*}{\partial x} + \frac{\partial u}{\partial y}\frac{\partial v^*}{\partial y} \right) {dxdy} \\
(\nabla u, \nabla v) &= \frac{1}{2\pi}\int_{-1}^1 \int_{0}^{2\pi} \frac{\partial u}{\partial x}\frac{\partial v^*}{\partial x} {dxdy} + \int_{-1}^1 \int_{0}^{2\pi} \frac{\partial u}{\partial y}\frac{\partial v^*}{\partial y} {dxdy}
\end{align}
$$

which is also a sum of two terms. These two terms are the two `TPMatrix`es returned by `inner` above.

Now each one of these two terms can be written as the outer product of two smaller matrices. Consider the first:

$$
\begin{align}
\frac{1}{2\pi}\int_{-1}^1 \int_{0}^{2\pi} \frac{\partial u}{\partial x}\frac{\partial v^*}{\partial x} {dxdy} &= \frac{1}{2\pi}\int_{-1}^1 \int_{0}^{2\pi} \frac{\partial \sum_{m}\sum_{n} \hat{u}_{mn} \phi_{mn}}{\partial x}\frac{\partial \phi_{kl}^*}{\partial x }{dxdy} \\
   &= \sum_{m}\sum_{n} \hat{u}_{mn} \frac{1}{2\pi} \int_{-1}^1 \int_{0}^{2\pi} \frac{\partial (P_m(x)-P_{m+2}(x))\exp(iny)}{\partial x}\frac{\partial (P_k(x)-P_{k+2}(x))\exp(-ily)}{\partial x} {dxdy} \\
   &= \sum_{m}\sum_{n} \hat{u}_{mn} \frac{1}{2\pi} \int_{-1}^1 \int_{0}^{2\pi} \frac{\partial (P_m(x)-P_{m+2}(x))}{\partial x}\frac{\partial (P_k(x)-P_{k+2}(x))}{\partial x} \exp(iny) \exp(-ily) {dxdy} \\
   &= \sum_{m}\sum_{n} \hat{u}_{mn} \underbrace{\int_{-1}^1 \frac{\partial (P_m(x)-P_{m+2}(x))}{\partial x}\frac{\partial (P_k(x)-P_{k+2}(x))}{\partial x} {dx}}_{a_{km}} \underbrace{\frac{1}{2\pi}\int_{0}^{2\pi}  \exp(iny) \exp(-ily) {dy}}_{\delta_{ln}} \\
   &= a_{km} \delta_{ln} \hat{u}_{mn} \\
   &= a_{km} \hat{u}_{ml}
\end{align}
$$

```python
print(A[0].mats)
```

The first item of the `A[0].mats` list is the $a_{km}$ matrix and the second is the identity matrix.

Now create a manufactured solution to test the implementation.

```python
import sympy as sp
x, y = sp.symbols('x,y')
ue = (sp.cos(4*x) + sp.sin(2*y))*(1 - x**2)
fe = ue.diff(x, 2) + ue.diff(y, 2)
fl = sp.lambdify((x, y), fe, 'numpy')
fj = Array(V, buffer=fl(*V.mesh()))
```

Assemble right hand side

```python
f_tilde = Function(V)
f_tilde = inner(v, -fj, output_array=f_tilde)
```

Solve system of equations by fetching an efficient Helmholtz solver

```python
u_hat = Function(V)
solver = legendre.la.Helmholtz(*A)
u_hat = solver(u_hat, f_tilde)
```

```python
X = V.local_mesh(True)
plt.contourf(X[0], X[1], u_hat.backward(), 100)
plt.colorbar()
```


### 必要なライブラリをインポートする

```python
from sympy import *
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import Qubit,QubitBra
from sympy.physics.quantum.qubit import measure_all, measure_all_oneshot
from sympy.physics.quantum.gate import X,Y,Z,H,S,T,CNOT,SWAP, CGateS
from sympy.physics.quantum.gate import IdentityGate as _I
```

↓SymPy で良い感じに表示するためのおまじない

```python
# !pip install graphviz
from sympy.printing.dot import dotprint
from graphviz import Source
init_printing()
```

```python
%matplotlib inline
import matplotlib.pyplot as plt
from sympy.physics.quantum.circuitplot import CircuitPlot,labeller, Mz,CreateOneQubitGate
```

## 重ね合わせ状態の中の、ある状態のみマーキングする

重ね合わせ状態の中の、ある状態 $ s_t $ のみに、−１掛ける操作（ここでは、マーキングを呼びます）を考えます。

たとえば、２量子ビットの重ね合わせ状態 $\displaystyle \frac{1}{2} \left(\ |\ 00\ \rangle\ +\ |\ 01\ \rangle\ +\ |\ 10\ \rangle\ +\ |\ 11\ \rangle\ \right) $ を考えるとき、
状態 $\;|\ 10\ \rangle\;$ をマーキングするとします。

マーキング後の状態は、$\displaystyle  \frac{1}{2} \left(\ |\ 00\ \rangle\ +\ |\ 01\ \rangle\ -\ |\ 10\ \rangle\ +\ |\ 11\ \rangle\ \right) $ となることを目指します。

### ３量子ビットのマーキング操作を試してみましょう

```python
def hadamard(s,n):
    h = H(s)
    for i in range(s+1,n+s):
       h = H(i)*h
    return h
def CCX(c1,c2,t):  return CGateS((c1,c2),X(t))
```

```python
h_3 = hadamard(0,3)
target_state_3 = qapply(h_3*Qubit('000'))          #３量子ビットの重ね合わせ状態を準備します。
def CCZ(c1,c2,t): return (H(t)*CCX(c1,c2,t)*H(t)) # ＣＣＺ演算子を定義します。
```

```python
mark_7 = CCZ(1,2,0)
qapply(mark_7*target_state_3)
```

```python
mark_6 = X(0)*CCZ(1,2,0)*X(0)
qapply(mark_6*target_state_3)
```

```python
mark_5 = X(1)*CCZ(1,2,0)*X(1)
qapply(mark_5*target_state_3)
```

```python
mark_4 = X(1)*X(0)*CCZ(1,2,0)*X(1)*X(0)
qapply(mark_4*target_state_3)
```

```python
mark_3 = X(2)*CCZ(1,2,0)*X(2)
qapply(mark_3*target_state_3)
```

```python
mark_2 = X(2)*X(0)*CCZ(1,2,0)*X(2)*X(0)
qapply(mark_2*target_state_3)
```

```python
mark_1 = X(2)*X(1)*CCZ(1,2,0)*X(2)*X(1)
qapply(mark_1*target_state_3)
```

```python
mark_0 = X(2)*X(1)*X(0)*CCZ(1,2,0)*X(2)*X(1)*X(0)
qapply(mark_0*h_3*Qubit('000'))
```

## 重ね合わせ状態の中に、マーキングした状態があるかを見る

著名な操作「Grover のアルゴリズム」を試してみましょう。

（ヒント１）平均値周りの反転操作：$\displaystyle D_{n} = H_{n} \cdot \Big( 2\ |\ 0\ \rangle\langle\ 0\ |_{n}\ -\ I_{n} \Big) \cdot H_{n} $ を使います。

（ヒント２）試行回数は、$\displaystyle \mathcal{O}(\sqrt{n}) $

```python
# d_3 = h_3 * X(0)*X(1)*X(2) * H(0)*CCX(1,2,0)*H(0) * X(0)*X(1)*X(2) * h_3    # グローバル位相（絶対位相）の差に注意
# d_3 = h_3 * X(0)*X(1)*X(2) * CGateS((1,2), Z(0)) * X(0)*X(1)*X(2) * h_3
def DOp(n): return (Qubit('0'*n)*QubitBra('0'*n)*2-_I(0))     # ゲート操作で計算するには、上記コメントのような演算になります。
d_3 = h_3 * DOp(3) * h_3    # 平均値周りの反転操作
represent(d_3,nqubits=3)
```

### ３量子ビットで状態｜７＞を探す

```python
ret1=qapply(d_3*mark_7*target_state_3)
measure_all(ret1)
```

```python
ret2=qapply(d_3*mark_7*ret1)  # シミュレーター計算ですから、上で一度観測した ret1 を使っても観測の影響なく次の計算に利用可能。
measure_all(ret2)
```

```python
over_1=qapply(d_3*mark_7*ret2)  # 試行回数が多いとどうなるでしょう。
measure_all(over_1)
```

```python
over_2=qapply(d_3*mark_7*over_1)  # 試行回数が多いとどうなるでしょう。
measure_all(over_2)
```

通しで計算すると、回路が長くなってきています。少し時間かかります。

```python
from sympy.physics.quantum.gate import gate_simp
search_7_in_3qubit = gate_simp(d_3*mark_7*d_3*mark_7)
ret3=qapply(search_7_in_3qubit*target_state_3)
ret3
```

```python
print(measure_all(ret3))
for i in range(10):
    pprint(measure_all_oneshot(ret3))
```

    [(|000>, 1/128), (|001>, 1/128), (|010>, 1/128), (|011>, 1/128), (|100>, 1/128), (|101>, 1/128), (|110>, 1/128), (|111>, 121/128)]
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩

```python
d_3_gate = h_3 * X(0)*X(1)*X(2) * H(0)*CCX(1,2,0)*H(0) * X(0)*X(1)*X(2) * h_3
CircuitPlot(gate_simp(d_3_gate*mark_7*d_3_gate*mark_7*h_3),nqubits=3,labels=labeller(3)[::-1])
```

## 【課題１】グローバーのアルゴリズム
<strong>
問１）

１. 次の「問１の初期状態」 quest_state を入力として、この量子状態に $\lvert 111 \rangle $ が含まれるか
　グローバーのアルゴリズムを使って調べてください。

２. 上の条件で、この量子状態に $\lvert 101 \rangle $ が含まれるかをグローバーのアルゴリズムを
　使って調べる考察をします。（うまくいかない例を見ます）

　・プログラムを作り、実際は、$\lvert 101 \rangle $ が高確率で検出されることを調べてください。
　・なぜ、初期状態に含まれていない状態が検出されるか理由を考えましょう。（解答は口頭でよい）

問２）
１. 下の「問２の初期状態」quest2_state を入力として、問１と同様、
　$\lvert 111 \rangle $ と $\lvert 101 \rangle $ の状態にの検知について  グローバーのアルゴリズムを適用して、
　その状況を考察してください。

</strong>

**以降、【課題１】問１−１）の回答欄：**

```python
# 問１の初期状態
quest_state = CNOT(1,0)*CNOT(2,1)*H(2)*H(0)*Qubit('000')
CircuitPlot(quest_state,nqubits=3)
```

```python
# 計算した初期状態を init_state とする
init_state = qapply(quest_state)
init_state
```

```python
# | 111 > の検索する量子回路を作成する。

mark_7 = CCZ(1,2,0)
grover_7 = gate_simp(d_3*mark_7*d_3*mark_7)
state1_7 = qapply(d_3*mark_7*init_state)
qapply(d_3*mark_7*state1_7)
```

```python
# 上で作った量子回路を初期状態と作用させて measure_all_oneshot() で何回か試行して、結果をみる。

for i in range(10):
    pprint(measure_all_oneshot(qapply(grover_7*init_state)))
```

    ❘011⟩
    ❘000⟩
    ❘111⟩
    ❘101⟩
    ❘011⟩
    ❘100⟩
    ❘010⟩
    ❘111⟩
    ❘111⟩
    ❘111⟩

**以降、【課題１】問１−２）の回答欄：**

```python
# | 101 > の検索する量子回路を作成する。

mark_5 = X(1)*CCZ(1,2,0)*X(1)
grover_5 = gate_simp(d_3*mark_5*d_3*mark_5)
state1_5 = qapply(d_3*mark_5*init_state)
qapply(d_3*mark_5*state1_5)
```

```python
# 上で作った量子回路を初期状態と作用させて measure_all() でかく状態の確率をみて、考察する。

measure_all(qapply(grover_5*init_state))
```

**以降、【課題１】問２−１）の回答欄：**

```python
# 問２の初期状態
quest2_state = CNOT(2,1)*H(2)*X(2)*CNOT(2,1)*CNOT(2,0)*H(2)*X(2)*Qubit('000')
CircuitPlot(quest2_state,nqubits=3)
```

```python
# 問２の回答欄（１）

init2_state = qapply(quest2_state)
init2_state
```

```python
# 問２の回答欄（２）

for i in range(10):
    pprint(measure_all_oneshot(qapply(grover_7*init2_state)))
```

    ❘000⟩
    ❘101⟩
    ❘101⟩
    ❘011⟩
    ❘101⟩
    ❘000⟩
    ❘000⟩
    ❘101⟩
    ❘011⟩
    ❘111⟩

```python
# 問２の回答欄（３）

measure_all(qapply(grover_5*init2_state))
```

```python

```


# Lecture 18 - Intro to data science (https://bit.ly/intro_python_18)

Today we're going to look at doing simple machine learning with Python, as an intro to very basic data science.

The idea is not to give you a full knowledge of any single package or technique, rather to give you a sense for what is possible.

To keep things simple, we're going to start by looking at **one variable linear regression**. This is the simplest form of machine learning we can think of.

For data, we're going to look at an archival database of breast tumor data to try to find variables that predict malignancy.

#### ORIGINAL DATASET http://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/

**As a bonus: At the end of this lecture are some notes on three core Python data science libraries that are widely used:  Numpy, Matplotlib and Pandas. These will not be covered in class due to time, but please play with them!**

# Wisconsin Breast Cancer Database

1. Number of Instances: 699 (as of 15 July 1992 (yeah, it's old))

2. Number of Attributes: 10 plus the class attribute

3. Attribute Information: (class attribute has been moved to last column)
   Attribute                     Domain
   1. Sample code number            id number
   2. Clump Thickness               1 - 10
   3. Uniformity of Cell Size       1 - 10
   4. Uniformity of Cell Shape      1 - 10
   5. Marginal Adhesion             1 - 10
   6. Single Epithelial Cell Size   1 - 10
   7. Bare Nuclei                   1 - 10
   8. Bland Chromatin               1 - 10
   9. Normal Nucleoli               1 - 10
  10. Mitoses                       1 - 10
  11. Class:                        (2 for benign, 4 for malignant)

4. Missing attribute values: 16

   There are 16 instances in Groups 1 to 6 that contain a single missing
   (i.e., unavailable) attribute value, now denoted by "?".

5. Class distribution:

   Benign: 458 (65.5%)
   Malignant: 241 (34.5%)

# Load the data

First copy the data from Google Drive to a local file.

The data is in a comma separated value (csv) file:

> **Year,Make,Model,Description,Price

> 1997,Ford,E350,"ac, abs, moon",3000.00

> 1999,Chevy,Venture Extended Edition,"",4900.00

> 1999,Chevy,"Venture Extended Edition, Very Large",5000.00
**

* In a CSV, Each line gives a comma separated sequence of N text strings

* CSV files (and tab seperated value (TSV) files) are formats for 2d tables

* they can also generally be loaded by spreadsheets

```python
# Copy the file from the internet
import urllib.request

url = "https://raw.githubusercontent.com/benedictpaten/intro_python/main/lecture_notebooks/data/breast-cancer-wisconsin.data.csv"
cancer_data_file = 'cancer_data.csv'

urllib.request.urlretrieve(url, cancer_data_file) # This function copies the thing the url points at into
# a local file copy
```

    ('cancer_data.csv', <http.client.HTTPMessage at 0x166d20c40>)

**Import the Pandas and Numpy modules**

* Numpy is an "array" and "matrix" library, used to represent large collections of data efficiently in Python

* Pandas builds on Numpy to provide a "spreadsheet like" package for manipulating tables of data and performing statistical analyses.

* For more details see the appendices at the end of the notebook and the linked tutorials.

```python
# These are two key libraries we need for representing the data in Python
import numpy as np
import pandas as pd
```

Next, load the data into Python using Pandas..

```python
# Load the data as a Pandas dataframe

# Read input
df = pd.read_csv(cancer_data_file)
df.head(5) # Head just shows us the first 5 rows
```

      id
      clump-thickness
      uniformity-of-cell-size
      uniformity-of-cell-shape
      marginal-adhesion
      single-epithelial-cell-size
      bare-nuclei
      bland-chromatin
      normal-nucleoli
      mitoses
      class

      0
      1000025
      5
      1
      1
      1
      2
      1
      3
      1
      1
      2

      1
      1002945
      5
      4
      4
      5
      7
      10
      3
      2
      1
      2

      2
      1015425
      3
      1
      1
      1
      2
      2
      3
      1
      1
      2

      3
      1016277
      6
      8
      8
      1
      3
      4
      3
      7
      1
      2

      4
      1017023
      4
      1
      1
      3
      2
      1
      3
      1
      1
      2

This converts process reads the csv table data into a Pandas "data frame", abbreviated "df".

With df.head(5) we print the first five rows to illustrate the nature of the data.

# Preprocess The Data

In data science and machine learning frequently much of the challenge is in preprocessing the data into a format that is ameniable to the algorithms used.

Here we don't need to do too much, we just normalize the data.

We use scikit-learn, a popular Python machine learning package to do the preprocessing. See: https://scikit-learn.org/stable/modules/preprocessing.html

```python
# Preprocess the data so we can use it for regression. Especially the class values.
# Access function/method docstrings in jupyter via '?'. Ex: preprocessing.LabelEncoder?
from sklearn import preprocessing

encoder = preprocessing.LabelEncoder()

for col in df.columns: # For each column in the data frame
    df[col] = encoder.fit_transform(df[col]) # Transform the series so is zero based
    # and dense

df.head(5)
```

      id
      clump-thickness
      uniformity-of-cell-size
      uniformity-of-cell-shape
      marginal-adhesion
      single-epithelial-cell-size
      bare-nuclei
      bland-chromatin
      normal-nucleoli
      mitoses
      class

      0
      172
      4
      0
      0
      0
      1
      0
      2
      0
      0
      0

      1
      175
      4
      3
      3
      4
      6
      1
      2
      1
      0
      0

      2
      176
      2
      0
      0
      0
      1
      2
      2
      0
      0
      0

      3
      177
      5
      7
      7
      0
      2
      4
      2
      6
      0
      0

      4
      179
      3
      0
      0
      2
      1
      0
      2
      0
      0
      0

# Linear Regression w/One Variable

To start, we suppose we have a set of pairs:

\begin{equation}
(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)
\end{equation}

The task is to try to predict $y_i$ given $x_i$. Call our prediction $y'_i$,  in linear regression we use the following simple linear equation for our prediction:

\begin{equation}
y'_i = w*x_i+b
\end{equation}

In this picture the pairs:

\begin{equation}
(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)
\end{equation}

are the blue dots and our prediction:

\begin{equation}
y'_i = w*x_i+b
\end{equation}

is the red line

In Python we encode the prediction as:

```python
def predict(x, w, b):
    return x*w + b
```

The task of machine learning here is to learn the parameters of the model, that is $w$ and $b$. To do this we need some way of deciding the value of our parameter choices - a "cost function"

# Cost function / loss function / risk function

The mean squared error (MSE) is a measure of the quality of an estimator—it is always non-negative, and values closer to zero are better.

To judge our parameters we use MSE:
\begin{equation}
MSE = \frac{1}{n} \sum_{i=1}^n (y_i - (wx_i+b))^2
\end{equation}

We visualize this error calculation as shown.

The error is proportional to the sum of the squared length of the green lines:

In Python:

```python
def cost_function(x, y, w, b):
    n = len(x)
    total_error = 0.0
    for i in range(n):
        total_error += (y[i] - (w*x[i] + b))**2
    return total_error / n
```

# Gradient descent
Having defined a model and a cost function, the next thing to do is define a learning method which can find good parameters for the model.

For the case of simple linear regression we could calculate the optimum parameters analytically, but here we choose to use gradient descent. An iterative method that can be used to optimize a large class of problems, many of which can not be solved exactly.

In Gradient Descent the idea is to use the derivative (rate of change) of the cost function to iteratively search for the point where the parameters minimize the MSE:

(In the picture J(w) is the cost function and we traverse the cost function for the w parameter by iteratively moving toward the global minimum)

There are two parameters (coefficients) in our cost function we can control: weight $w$ and bias $b$. Since we need to consider the impact each one has on the final prediction, we use partial derivatives.
Recall the MSE cost function (here called $f$):
\begin{equation}
f(w,b)= \frac{1}{n} \sum_{i=1}^{n} (y_i - (wx_i+b))^2
\end{equation}
First lets find the partial derivative for $w$:
\begin{equation}
\frac{\partial f}{\partial w} = \frac{1}{n} \sum_{i=1}^n -2x_i(y_i - (wx_i+b))
\end{equation}
And the partial derivative for $b$:
\begin{equation}
\frac{\partial f}{\partial b} = \frac{1}{n} \sum_{i=1}^n -2(y_i - (wx_i+b))
\end{equation}

Suppose we have estimates of $w$ and $b$, the gradient descent method estimates  new estimates, $w'$ and $b'$, as follows:

\begin{equation}
w' = w - l * \frac{\partial f}{\partial w}
\end{equation}
And the partial derivative for $b$:
\begin{equation}
b' = b - l * \frac{\partial f}{\partial b}
\end{equation}

Where $l$ is the "learning rate" hyperparameter (a parameter of the learning algorithm) that dictates the speed at which the model learns - moves along the slope.

Given these update equations, we can express them in Python:

```python
def update_weights(x, y, w, b, learning_rate):
    weight_deriv = 0
    bias_deriv = 0
    n = len(x)

    for i in range(n):
        # Calculate partial derivatives
        # -2x(y - (mx + b))
        weight_deriv += -2*x[i] * (y[i] - (w*x[i] + b))

        # -2(y - (mx + b))
        bias_deriv += -2*(y[i] - (w*x[i] + b))

    # We subtract because the derivatives point in direction of steepest ascent
    w -= (weight_deriv / n) * learning_rate
    b -= (bias_deriv / n) * learning_rate

    return w, b
```

Note: making the learning rate too large will stop the model converging, because the size of the jumps between parameter estimates will hop around the optimal values.

# Train loop

We now have a simple model, cost function and associated learning algorithm. We can put these together to train the model iteratively:

```python
def train_model(x, y, w, b, learning_rate, epochs):
    cost_history = []

    for i in range(epochs):
        w,b = update_weights(x, y, w, b, learning_rate)

        #Calculate cost for auditing purposes
        cost = cost_function(x, y, w, b)
        cost_history.append(cost)

        # Log Progress
        if (i+1) % 20 == 0:
            print("Epochs: ", str(i+1), " cost: ",  str(cost))

    return w, b, cost_history
```

An epoch is a cycle of learning from the data, the learning rate and number of epochs are the hyperparameters of this algorithm.

# Run the training

Let's test this all out:

```python
x = df['uniformity-of-cell-size']
y = df['class'] # Benign / malignant
w, b, cost_history = train_model(x, y, 0, 0, 0.02, 500) # Start with w and b as 0, 0
print(w, b)
```

    Epochs:  20  cost:  0.07538027961136255
    Epochs:  40  cost:  0.07498926619585158
    Epochs:  60  cost:  0.07485402644136209
    Epochs:  80  cost:  0.07480725108668539
    Epochs:  100  cost:  0.07479107290140467
    Epochs:  120  cost:  0.07478547735475996
    Epochs:  140  cost:  0.07478354202383664
    Epochs:  160  cost:  0.0747828726512064
    Epochs:  180  cost:  0.07478264113538957
    Epochs:  200  cost:  0.07478256106104562
    Epochs:  220  cost:  0.07478253336574431
    Epochs:  240  cost:  0.07478252378677425
    Epochs:  260  cost:  0.07478252047369835
    Epochs:  280  cost:  0.07478251932780343
    Epochs:  300  cost:  0.07478251893147433
    Epochs:  320  cost:  0.07478251879439674
    Epochs:  340  cost:  0.0747825187469841
    Epochs:  360  cost:  0.07478251873058646
    Epochs:  380  cost:  0.07478251872491373
    Epochs:  400  cost:  0.07478251872295176
    Epochs:  420  cost:  0.0747825187222749
    Epochs:  440  cost:  0.07478251872203892
    Epochs:  460  cost:  0.07478251872195751
    Epochs:  480  cost:  0.07478251872193077
    Epochs:  500  cost:  0.07478251872191918
    0.12748809923704618 0.07265767658062813

We can see that the model converges towards better parameters!

How do we judge how useful the model is in practice?

# Predictive separation index (PSI) as score
We can use Predictive Separation Index (PSI), to use as the strength of a predictor. The equation is:

\begin{equation}
PSI ( x ) = [ \textrm{mean }  y'_i  \textrm{ when } y_i = 1 ] - [ \textrm{mean }  y'_i  \textrm{ when } y_i = 0 ] \, .
\end{equation}

We want PSI(x) very close to 1 as the first term should be close to 1 and the second term should be close to 0.

In Python:

```python
def get_score(x, y, w, b):
    preds_0 = []
    preds_1 = []
    for i in range(len(x)):
        p = predict(x[i], w, b)
        if y[i] == 0:
            preds_0.append(p)
        else:
            preds_1.append(p)
    if len(preds_0) != 0:
        score = (sum(preds_1) / len(preds_1) - sum(preds_0) / len(preds_0))
    else:
        score = (sum(preds_1) /  len(preds_1) - 0)
    return preds_0, preds_1, score
```

```python
preds_0, preds_1, score = get_score(x, y, w, b)
print("PSI: ", score)
```

    PSI:  0.6689665943993244

# The prediction distribution plot

A better way to look at our prediction is to look at the distribution of $y'_i$ we get for each of the classes. For $y_i = 0$ the $y'_i$ should be 0. Conversely,  $y'_i$ should be close to 1 when $y_i = 1$. This is a visual way of seeing the strength of a predictor.

```python
%matplotlib inline
# The above is required to display matplotlib in jupyter
import matplotlib.pyplot as plt

# The y=0 class
n, bins, patches = plt.hist(preds_0, bins=100, density=1, cumulative=0)
plt.title('Predictive distribution for class y=0')
plt.show()
```

```python
# The y=1 class
n, bins, patches = plt.hist(preds_1, bins=100, density=1, cumulative=0)
plt.title('Predictive distribution for class y=1')
plt.show()
```

## Use a library to do the same thing! - SciKit Learn

We can use SciKit Learn (sklearn) to do the same thing with a couple lines of code!:

```python
x = df[['uniformity-of-cell-size']]
y = df['class']

from sklearn import datasets, linear_model

lm_model = linear_model.LinearRegression(fit_intercept=True, copy_X=True, n_jobs=1)
lm_model.fit(x, y)

print("sklearn, w, b, score: ", round(lm_model.coef_[0], 6), round(lm_model.intercept_, 6), round(lm_model.score(x,y), 6))
print("manualr, w, b, score: ", round(w, 6) , round(b, 6), round(score, 6))
# So, we have learned the same co-efficents (or, almost the same) through a library.
```

    sklearn, w, b, score:  0.127488 0.072658 0.668967
    manualr, w, b, score:  0.127488 0.072658 0.668967

# Accuracy and Classification

We have create a "regression model", which predicts a continious value. However, what we actually want for this task is a classifier - a model which predicts either "true" (malignant) or "benign" (false). We can make a classifier from our regression model in the simplest way possible: by picking a threshold.

From the above plots it looks like 0.2 is a good cutoff? Let's pick that and look at the accuracy of the model, i.e. how accurate is the model when we say y'_i > 0.2 is "true" but otherwise "false":

```python
y_pred = lm_model.predict(x)
y_pred = [1 if p > 0.2 else 0 for p in y_pred]
from sklearn.metrics import accuracy_score
accuracy_score(y, y_pred)
```

    0.882689556509299

Not bad! However, there is a lot more to this - we haven't considered:

* Integrating multiple variables
* Switching models - e.g. logisitic rather than linear regression is generally used for classification rather than regression tasks
* Splitting out test data from out training data.

Take a look at the SciKit Learn tutorial if you'd like to dig in more.

# Summary / Lectures Wrap

Okay, we've covered a lot of concepts and shown how we can code from first principles a machine learning algorithm to predict malignancy!

Obviously, we've barely scratched the surface, but you should know that you're now much closer to doing real data science yourself than you were 10 weeks ago.

We've now completed all the lectures - well done and thanks for listening, I appreciate it :)

Next lecture we'll reserve for a revision session.

# Numpy, Pandas, Sklearn and Matplotlib Reading (this won't be on the exam!)

* Browse through the NumPy tutorial here: https://docs.scipy.org/doc/numpy/user/quickstart.html
* Browse through the Pyplot tutorial here:
https://matplotlib.org/tutorials/introductory/pyplot.html#sphx-glr-tutorials-introductory-pyplot-py
* Browse the Pandas 10 min tutorial: https://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html#min
* Browse through the SciKit Learn tutorial here: https://scikit-learn.org/stable/tutorial/basic/tutorial.html#learning-and-predicting

# Homework

* Zybooks Reading 18

# Appendix - Exploring Numpy, Matplotlib and Pandas

# NumPy Arrays

* NumPy is a Python library for efficiently representing multi-dimensional arrays (e.g. matrices, tensors).

* Pretty much all ML and visualization in Python relies on these array types, or things derived from them.

So far we've seen multi-dimensional arrays using Python lists, e.g.:

```python
# Consider a 2D matrix represented using Python lists

a = [[ 0,  1,  2,  3,  4],
     [ 5,  6,  7,  8,  9],
     [10, 11, 12, 13, 14]]

a[1][0]
```

    5

* The problem is that Python lists are general purpose and flexible and as a result both memory inefficient and really slow.

* This is not a big deal with small data, but when dealing with large amounts of data this becomes of key concern.

* NumPy provides fast, memory efficient array backed multi-dimensional arrays.

We can directly convert to a NumPy 2D array as follows:

```python
import numpy as np # Import the numpy library using the
# common abbreviation np

a = np.array(a) # Replace the list-of-lists with a numpy 2D array

a
```

    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14]])

```python
type(a)
```

    numpy.ndarray

```python
# We can address this matrix two ways:

print("The first way", a[1][0]) # This looks familiar

print("The second way", a[1,0]) # This does not look familiar but is shorter and more efficient
```

    The first way 5
    The second way 5

So what is going on behind the scenes?

* A NumPy array is represented as contiguous block of memory in which each value is stored in place (not as a reference to separate object), with the elements arranged into a linear ordering, e.g.:

* As a consequence, arrays have a "data type" corresponding to the way memory is represented for each value stored:

```python
# We can can find out the data type of a NumPy array using the dtype
# variable:

a.dtype # NumPy stores each number as an integer, each using 64 bits
```

    dtype('int64')

If we create an array using floats then the data type will be different:

```python
a = np.array([ 1.0, 2, 4 ])

print(a) # All numbers in the array will be represented as a float

print(a.dtype) # The type is float, again each with 64 bits.
```

    [1. 2. 4.]
    float64

The dimensions of a numpy array are stored using the shape and ndim attributes:

```python
print(a.shape) # The shape of the array (here it has one dimension)
```

    (3,)

```python
print(a.ndim) # The number of dimensions in the array
```

    1

```python
# If we make the array multi-dimensional

a = np.array([[ 0,  1,  2,  3,  4],
              [ 5,  6,  7,  8,  9],
              [10, 11, 12, 13, 14]])

print("The dimensions of the array:", a.shape)
print("The number of dimensions", a.ndim)
```

    The dimensions of the array: (3, 5)
    The number of dimensions 2

The size of a numpy array is the size attribute:

```python
print("To get the size of the first dimension use len()", len(a))
print("The total size of the array is the .size attribute", a.size)
```

    To get the size of the first dimension use len() 3
    The total size of the array is the .size attribute 15

Building numpy arrays from lists defeats the purpose of trying to save memory.

To make large numeric arrays use zeros(), ones(), empty() or arange()

```python
a = np.zeros((3, 5, 6))

print("Shape:", a.shape)
print("Type", a.dtype) # The default type is 64bit float
print("The array\n", a)
```

    Shape: (3, 5, 6)
    Type float64
    The array
     [[[0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]]

     [[0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]]

     [[0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]
      [0. 0. 0. 0. 0. 0.]]]

```python
a = np.ones((2, 5)) # To get an array containing ones

a
```

    array([[1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1.]])

```python
a = np.empty((3, 2, 5)) # To get an array with unitialized memory (faster)

a # Note the values have no defined value in this case
```

    array([[[0.00000000e+000, 9.88131292e-324, 0.00000000e+000,
             0.00000000e+000, 4.83424335e-277],
            [1.16095484e-028, 1.06398206e+248, 3.03772881e-067,
             1.07548461e+272, 5.03032220e+180]],

           [[9.30860811e+199, 4.63461826e+228, 2.26249033e+137,
             9.00495205e+130, 8.01760059e-096],
            [8.95393586e-096, 1.81148490e-152, 9.89069227e-096,
             1.08266064e-095, 1.19683021e+141]],

           [[6.01347002e-154, 8.04868985e-096, 8.98502512e-096,
             1.76539239e+137, 7.79145813e+140],
            [6.48224637e+170, 3.67145870e+228, 1.13908484e-071,
             1.95714674e-305, 0.00000000e+000]]])

```python
a = np.arange(15)

a
```

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])

```python
a = np.arange(0.0, 3.0, 0.2) # You can do ranges in floating point!

a
```

    array([0. , 0.2, 0.4, 0.6, 0.8, 1. , 1.2, 1.4, 1.6, 1.8, 2. , 2.2, 2.4,
           2.6, 2.8])

A neat feature of a numpy array is that you can reshape it's dimensions with reshape():

```python
a = a.reshape(3, 5)

print(a)
```

    [[0.  0.2 0.4 0.6 0.8]
     [1.  1.2 1.4 1.6 1.8]
     [2.  2.2 2.4 2.6 2.8]]

# NumPy Slices

NumPy slicing on multi-dimensional arrays lets you quickly get slices of your data:

```python
a # All of a
```

    array([[0. , 0.2, 0.4, 0.6, 0.8],
           [1. , 1.2, 1.4, 1.6, 1.8],
           [2. , 2.2, 2.4, 2.6, 2.8]])

```python
a[:,2] # Just the third elements of the 2nd dimension of the array.
```

    array([0.4, 1.4, 2.4])

```python
a[1,2:] # Just the 3rd and subsequent elements of the 2nd element
#of the 1st dimension of the array.
```

    array([1.4, 1.6, 1.8])

# NumPy Iterators

```python
for i in a: # By default NumPy iterates over rows
  print(i)
```

    [0.  0.2 0.4 0.6 0.8]
    [1.  1.2 1.4 1.6 1.8]
    [2.  2.2 2.4 2.6 2.8]

```python
for i in a.flat: # But you can iterate over all the elements using flat
  print(i)
```

    0.0
    0.2
    0.4
    0.6000000000000001
    0.8
    1.0
    1.2000000000000002
    1.4000000000000001
    1.6
    1.8
    2.0
    2.2
    2.4000000000000004
    2.6
    2.8000000000000003

# NumPy Builtin Functions

NumPy lets you do math with vectors.

```python
a = np.arange(10)

print(a)
```

    [0 1 2 3 4 5 6 7 8 9]

```python
np.exp(a) # Calculate e**x for each entry x in the array
```

    array([1.00000000e+00, 2.71828183e+00, 7.38905610e+00, 2.00855369e+01,
           5.45981500e+01, 1.48413159e+02, 4.03428793e+02, 1.09663316e+03,
           2.98095799e+03, 8.10308393e+03])

```python
np.sqrt(a) # Calculate sqrt(x) for each entry x in the array
```

    array([0.        , 1.        , 1.41421356, 1.73205081, 2.        ,
           2.23606798, 2.44948974, 2.64575131, 2.82842712, 3.        ])

NumPy has lots of builtin math functions!

# NumPy Operator Overloading

```python
a = np.arange(10)

a + 10 # Add ten to each value
```

    array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19])

```python
a**2 # Make square of each number
```

    array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])

etc... each of these operators makes a new array.

To edit the array in place use the increment operators:

```python
a + 10

print(a) # a is not changed
```

    [0 1 2 3 4 5 6 7 8 9]

```python
a += 10 # a is now changed in place.

print(a)
```

    [10 11 12 13 14 15 16 17 18 19]

```python
a **= 2 #ditto

print(a)
```

    [100 121 144 169 196 225 256 289 324 361]

# Linear Algebra Functions

```python
a = np.array([ [1, 2, 3], [4, 5, 6], [ 7, 8, 9]])

print(a)

```

    [[1 2 3]
     [4 5 6]
     [7 8 9]]

```python
a.transpose() # Compute the transpose (rows become columns and vice versa)
```

    array([[1, 4, 7],
           [2, 5, 8],
           [3, 6, 9]])

```python
a @ a # This is matrix multiplication
```

    array([[ 30,  36,  42],
           [ 66,  81,  96],
           [102, 126, 150]])

# Plotting with MatPlotLib

MatPlotLib is a Python plotting library that works with NumPy to make super nice plots.

It is extremely customizable and can create very fancy plots.

It's pyplot function provides a MATLAB-like interface to the library for making plots.

We can use NumPy and MatPlotLib to create complex data visualization. Here we'll just look at histograms and graphing functions (like on a graphical calculator):

# Histograms

```python
# Build a vector of 10000 samples from a normal distribution with variance 0.5^2 and mean 2
mean, variance_sqrt = 2, 0.5
v = np.random.normal(mean, variance_sqrt, 100000) # Random module allows you to create
# arrays of random numbers drawn from a given distribution.

print(v[:100]) # First 100 members of array
```

    [3.05860056 2.47094643 2.52816327 1.06051309 1.51347561 2.07549409
     2.67532159 2.11362191 2.27810538 1.60530092 2.79430517 2.55384148
     2.71733437 1.66242764 1.93892845 1.39116753 2.56617408 1.76097175
     1.76275852 2.09832228 2.6808698  1.37928547 2.07241814 1.25749651
     1.59688044 2.07152009 1.44225236 2.68596062 2.47293694 1.96695516
     2.10813339 1.63712879 2.1365555  1.26180979 1.83714402 1.54215922
     1.89239566 2.3143319  2.42369037 2.19719057 1.79300188 1.64746831
     1.76278447 2.75239863 2.21301337 2.3131599  2.48666909 2.00507997
     1.61474314 1.36147258 2.26238744 1.84012176 0.86990009 1.69078918
     2.15750101 2.27658333 2.17649476 1.30026459 1.62300265 1.38821459
     2.150771   1.46014625 2.55485899 3.05612462 1.50334031 1.21879721
     2.34316225 1.4618033  1.32275568 0.54912742 1.35766123 2.29520379
     2.06355796 2.21548564 1.38581955 2.3951399  1.88086719 1.68092591
     2.38628586 1.52334477 1.11916155 2.89121608 2.80126514 2.46808187
     2.1362822  2.3105049  1.21183951 2.2169369  1.79903039 2.44571742
     2.45988852 2.11475738 2.31968988 1.530576   2.39368022 2.05532901
     2.50659493 2.02117424 1.56076555 1.87058332]

```python
import matplotlib.pyplot as plt # Import pyplot

# Plot a histogram with 50 bins
plt.hist(v, bins=50)       # pyplot call to create a histogram with 50 bins
plt.show() # This call is what causes the plot to create the graphic
```

```python
# Plot a normalized histogram with 50 bins
plt.hist(v, bins=50, density=1)  # Density argument normalizes histogram
plt.show()
```

```python
# Plot a normalized histogram with 50 bins and label axes, title, etc.
plt.hist(v, bins=50, density=1)  # Density argument normalizes histogram
plt.ylabel('density')
plt.xlabel('x')
plt.title('A Histogram')
plt.show()
```

# Graphing Functions

This allows us to introduce the basic plot command:

```python
# If you just give it one array (list or numpy array, it assumes that these are the y values)
plt.plot([1, 2, 3, 4], "ro") # "ro" makes pyplot create red points
plt.show()
```

```python
plt.plot([5, 10, 15, 20], [2, 4, 8, 16], 'ro') # If you give it two arrays, it treats
# each pair of elements of the two lists, e.g. (5, 2), (10, 4), etc. as points on the 2D plane
plt.show()
```

We can use this to graph functions really easily:

```python
# evenly sampled points from 0 to 5 and 0.2 intervals
t = np.arange(0., 5., 0.2)

# red dashes, blue squares and green triangles
plt.plot(t, t**2, 'bs')
plt.show()
```

If we give it multiple pairs of arrays we can show multiple lines on the same plot:

```python
# evenly sampled time at 200ms intervals
t = np.arange(0., 5., 0.2)

# red dashes, blue squares and green triangles
plt.plot(t, t, 'r--', t, t**2, 'bs', t, t**3, 'g^')
plt.show()
```

# Pandas Series

The basic array object in Pandas is the series object, it is very NumPy array like and most of the stuff we showed for NumPy arrays works with series:

```python
import numpy as np
import pandas as pd # Standard pandas abbreviation to pd

s = pd.Series(np.arange(5))

s
```

    0    0
    1    1
    2    2
    3    3
    4    4
    dtype: int64

Under the hood it is organized like NumPy, and is mostly compatible and functional like a NumPy array.

However, it is not a multi-dimensional object:

```python
a = np.array([ [ 3, 4, 5], [ 6, 7, 8 ]])

print(a)

s = pd.Series(a) # This doesn't work, because Pandas series are one dimensional

print(s)

```

# Data Frames

* Panda's Data Frames are 2D "tables".

* Each column is a series

* Think of them like a spreadsheet, with different columns having different types, etc.:

```python
a = np.random.randn(6, 4) # Create a matrix of random numbers with 6 rows and 4 columns

print("The numpy array", a)

df = pd.DataFrame(a)

print("The corresponding data frame\n", df)
```

```python
df.columns=list('ABCD') # We can name the columns

df # IF you don't use print, it makes this nice table format in the notebook:
```

```python
# We can also name the rows (here we use dates as an example)

dates = pd.date_range('20190501', periods=6) # Make a list of dates as a Pandas series

print("The list of dates\n", dates) #

df.index=dates # Now give the rows of the matrix these dates

print("\nThe data frame with dates naming the rows:")
df
```

```python
# We could have done this all in one line:

df = pd.DataFrame(a, index=dates, columns=list("ABCD"))

df
```

# Data Frames w/Heterogenous Data

Data frames can also contain heterogenous types, for example by construction with a dictionary:

```python
# Here the series are put in a dictionary and the colum indices are the keys

df2 = pd.DataFrame({'A': 1.,
                    'B': pd.Timestamp('20130102'),
                    'C': pd.Series(1, index=list(range(4)), dtype='float32'),
                    'D': np.array([3] * 4, dtype='int32'),
                    'E': pd.Categorical(["test", "train", "test", "train"]),
                    'F': 'foo'})

df2
```

```python
df2.dtypes # The types of the different columns
```

# Data Frame Selection, Slicing and Filter

Data frames are easy to subselect by row or column, allowing you to get subsets of the data easily.

To select by column name use the square bracket notation:

```python
df
```

```python
df['A'] # Select the 'A' column
```

To select by row uses square brackets and slice notation:

```python
df[0:2] # First two rows
```

To select rows and columns using indexes use the iloc index:

```python
df.iloc[:, 1:3] # Selects all the rows and the first two columns
```

To select rows by row and column names use the loc index:

```python
df.loc['20190502':'20190505', ['A', 'D']]
```

To filter the rows by a column attribute:

```python
df[df.A > 0]
```

Finally, to convert back to a NumPy array use .to_numpy(), then allowing you to slice using NumPy array rules:

```python
df.to_numpy() # Convert to a numpy array
```

```python
df2.to_numpy()
```

# Data Frame Sorting

It's easy to sort data frames:

```python
# Sort the column ordering

df.sort_index(axis=1, ascending=False)
```

```python
# Sort the rows by value, keying on "B"

df.sort_values(by='B')
```

# Data Frame Stats

```python
df.mean() # get the median of each column
```

```python
df.median() # get the median of each column
```

```python
df.describe() # Calculate summary stats
```

```python
# You can even devise your own functions and apply them to each column

def total_range(x):
  return x.max() - x.min()

df.apply(total_range) # This is an example of passing around functions as pointers
```

# Grouping

If you have multiple rows with the same value, it's easy to group by them:

```python
df2 # df2 has a test/train split, described by column "E"
```

```python
df2.groupby("E").sum() # This groups by the values of column E and then
# adds up the values which can be summed (dates and strings can't be summed)
```

# Data Frames and CSVs

It is easy to convert comma separated files into a data frame, and to write to other popular formats (JSON, HDF5, etc.): (see https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#csv-text-files)

```python
df2
```

```python
df2.to_csv('out.csv') # Phew, that was easy - write the df to the file out.csv in CSV format
```

```python
# as a reminder, df2 looks like this:

df2
```

```python
# Let's just inspect the file real quick:

with open("out.csv") as f:
   for i, l in enumerate(f):
      print("Line:", i, "is: ", l, end="")
```

```python
df3 = pd.read_csv('out.csv') # As simple as

df3 # But note that the row index is all messed up..
```

```python
# Fix the row index column by telling it to use column 0 as the row index:

df3 = pd.read_csv('out.csv', index_col=0) # Read the csv, telling it that the first row is the
# index of the column names, tell it to respect the dates

df3 # Looks good
```

```python
# The one thing to bear in mind, the types of the elements in the table are not necessarily
# preserved:

print("Before writing\n", df2.dtypes)

print("After writing\n", df3.dtypes)
```

# Data Frames Plotting

```python
df
```

```python
df.plot() # By default makes a line graph of each column
```

```python

df2.plot() # Doesn't work - too many different series types
```

```python
df2
```

```python
df2['A'].plot()
```

lecture_notebooks/L18 Data Science .ipynb


>### <font color ='red'> Ejemplo 3  **Tarea**
>El tiempo en el cual un movimiento browniano se mantiene sobre su punto máximo en el intervalo [0,1] tiene una distribución
>$$F(x)=\frac{2}{\pi}\sin^{-1}(\sqrt x),\quad 0\leq x\leq 1$$ </font>
Genere muestres aleatorias que distribuyan según la función dada usando el método de la transformada inversa y grafique en una gráfica el historias 100 muestras generadas y comparela con el función F(x) dada, esto con el fín de validar que el procedimiento fue realizado de manera correcta

```python
import numpy as np
import scipy.stats as ss
from scipy.stats import rayleigh
import statsmodels as st
import matplotlib.pyplot as plt
from sympy import Derivative, diff, simplify
import matplotlib.pyplot as plt
```

```python
u=(2/np.pi)*np.arcos(root(x))

```

```python
N=10**6
def d_fun(N):
    U= np.random(N)
    return np.sin(np.pi*U/2)**2
f= lambda x: 1/(np.pi*np.sqrt(x)*np.sqrt(1-x))
#dominio 0<=x<=1
x=np.arange(0.1,.99,.01)
plt.plot(x,f(x))

```

```python

d= d_fun(N)
plt.hist(d, bins= 100, density=True)
plt.show()
```

### Ejemplo 4
Distribución de Rayleigh
$$F(x)=1-e^{-2x(x-b)},\quad x\geq b $$


```python
import numpy as np
```

```python
x = np.array([12, 34, 2, 15])
y = np.array([22, 11, 54.5, 12])
```

```python
x + y
```

    array([ 34. ,  45. ,  56.5,  27. ])

```python
x[0]
```

    12

```python
for element in x:
    print(element, end=" ")
```

    12 34 2 15

```python
x
```

    array([12, 34,  2, 15])

```python
z = x[1:3]
```

```python
z
```

    array([34,  2])

```python
z[0] = 1024
```

```python
z
```

    array([1024,    2])

```python
x
```

    array([  12, 1024,    2,   15])

```python
w = x.copy()
```

```python
w
```

    array([  12, 1024,    2,   15])

```python
z = w[1:3]
```

```python
z[0] = -10
```

```python
z
```

    array([-10,   2])

```python
w
```

    array([ 12, -10,   2,  15])

```python
x
```

    array([  12, 1024,    2,   15])

```python
x[[1, 2, 0]]
```

    array([1024,    2,   12])

```python
x
```

    array([  12, 1024,    2,   15])

```python
small_elements_of_x = x[x < 100]
```

```python
small_elements_of_x
```

    array([12,  2, 15])

```python
x < 100
```

    array([ True, False,  True,  True], dtype=bool)

```python
x[np.array([True, False, True, True])]
```

    array([12,  2, 15])

```python
x[np.array([False, True, False, True])]
```

    array([1024,   15])

```python
x
```

    array([  12, 1024,    2,   15])

```python
y
```

    array([ 22. ,  11. ,  54.5,  12. ])

```python
x * y
```

    array([   264.,  11264.,    109.,    180.])

```python
x @ y
```

    11817.0

```python
x = np.array([1, 2, 0, -15, 20, 35, 100, 1024])
```

```python
x[(x > 0) & (x < 1000)]
```

    array([  1,   2,  20,  35, 100])

```python
(x < 1000)
```

    array([ True,  True,  True,  True,  True,  True,  True, False], dtype=bool)

```python
(x > 0) & (x < 1000)
```

    array([ True,  True, False, False,  True,  True,  True, False], dtype=bool)

```python
list_x = [1, 2, 3]
list_y = [1, 2, 3]
list_x == list_y
```

    True

```python
x = np.array([1, 2, 3])
y = np.array([1, 2, 3])
```

```python
x == y
```

    array([ True,  True,  True], dtype=bool)

```python
(x == y).all()
```

    True

```python
if (x == y).all():
    print("arrays are equal")
```

    arrays are equal

```python
x = np.array([1, 2, 3])
y = np.array([10, 2, 15])
```

```python
(x == y).all()
```

    False

```python
(x == y).any()
```

    True

```python
(np.array([1]) == np.array([])).all()
```

    True

```python
np.array([1]) == np.array([])
```

    array([], dtype=bool)

```python
np.array_equal(np.array([1]), np.array([]))
```

    False

```python
import pandas as pd
```

```python
[[3, 4, 5], [4, 3, 5]]
```

```python
df = pd.DataFrame([[3, 4, 5], [4, 3, 5]],
                 columns=['hw1', 'hw2', 'hw3'],
                 index=['Alice', 'Bob'])
```

```python
column1 = df['hw1']
```

```python
column1['Alice']
```

    3

```python
df
```

      hw1
      hw2
      hw3

      Alice
      3
      4
      5

      Bob
      4
      3
      5

```python
df.loc['Alice']
```

    hw1    3
    hw2    4
    hw3    5
    Name: Alice, dtype: int64

```python
df.loc['Alice', 'hw2']
```

    4

```python
df.loc['Alice', 'hw2'] = 5
```

```python
df
```

      hw1
      hw2
      hw3

      Alice
      3
      5
      5

      Bob
      4
      3
      5

```python
df.mean()
```

    hw1    3.5
    hw2    4.0
    hw3    5.0
    dtype: float64

```python
df.mean(axis=1)
```

    Alice    4.333333
    Bob      4.000000
    dtype: float64

```python
df['Average'] = df.mean(axis=1)
```

```python
df
```

      hw1
      hw2
      hw3
      Average

      Alice
      3
      5
      5
      4.333333

      Bob
      4
      3
      5
      4.000000

```python
df.loc['Average'] = df.mean()
```

```python
df
```

      hw1
      hw2
      hw3
      Average

      Alice
      3.0
      5.0
      5.0
      4.333333

      Bob
      4.0
      3.0
      5.0
      4.000000

      Average
      3.5
      4.0
      5.0
      4.166667

```python
df.drop('Average', axis=1, inplace=True)
```

```python
df
```

      hw1
      hw2
      hw3

      Alice
      3.0
      5.0
      5.0

      Bob
      4.0
      3.0
      5.0

      Average
      3.5
      4.0
      5.0

```python
df = pd.DataFrame(np.random.randint(-10, 10, size=(10, 10)))
```

```python
df
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9

      0
      -10
      -5
      -2
      9
      -4
      -7
      0
      -10
      -8
      5

      1
      -5
      -10
      -9
      0
      -8
      -8
      -7
      -4
      1
      -7

      2
      2
      9
      -7
      -7
      -1
      -5
      6
      -3
      -6
      -1

      3
      -9
      6
      3
      6
      -10
      -9
      9
      0
      -9
      -6

      4
      -3
      4
      4
      2
      9
      -9
      1
      8
      9
      9

      5
      4
      -9
      7
      6
      4
      -1
      3
      7
      -3
      -10

      6
      7
      3
      -4
      3
      -4
      -5
      2
      5
      -9
      -8

      7
      1
      -6
      -2
      4
      9
      5
      7
      0
      -3
      0

      8
      7
      -7
      9
      -3
      0
      -8
      7
      -9
      -6
      3

      9
      0
      -4
      -9
      -5
      -2
      -3
      -6
      -10
      0
      0

```python
df[1]
```

    0    -5
    1   -10
    2     9
    3     6
    4     4
    5    -9
    6     3
    7    -6
    8    -7
    9    -4
    Name: 1, dtype: int32

```python
my_list = [10, 20, 30, 40, 50, 60, 70, 80]
my_list[3:7]
```

    [40, 50, 60, 70]

```python
df[3:7]
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9

      3
      -9
      6
      3
      6
      -10
      -9
      9
      0
      -9
      -6

      4
      -3
      4
      4
      2
      9
      -9
      1
      8
      9
      9

      5
      4
      -9
      7
      6
      4
      -1
      3
      7
      -3
      -10

      6
      7
      3
      -4
      3
      -4
      -5
      2
      5
      -9
      -8

```python
df.loc[3:7]
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9

      3
      -9
      6
      3
      6
      -10
      -9
      9
      0
      -9
      -6

      4
      -3
      4
      4
      2
      9
      -9
      1
      8
      9
      9

      5
      4
      -9
      7
      6
      4
      -1
      3
      7
      -3
      -10

      6
      7
      3
      -4
      3
      -4
      -5
      2
      5
      -9
      -8

      7
      1
      -6
      -2
      4
      9
      5
      7
      0
      -3
      0

```python
df.loc[:, 3:7]
```

      3
      4
      5
      6
      7

      0
      9
      -4
      -7
      0
      -10

      1
      0
      -8
      -8
      -7
      -4

      2
      -7
      -1
      -5
      6
      -3

      3
      6
      -10
      -9
      9
      0

      4
      2
      9
      -9
      1
      8

      5
      6
      4
      -1
      3
      7

      6
      3
      -4
      -5
      2
      5

      7
      4
      9
      5
      7
      0

      8
      -3
      0
      -8
      7
      -9

      9
      -5
      -2
      -3
      -6
      -10

```python
df = pd.DataFrame(np.random.randint(-10, 10, size=(4, 4)),
                 columns=['a', 'b', 'c', 'd'],
                 index=['q', 'w', 'e', 'r'])
```

```python
df
```

      a
      b
      c
      d

      q
      -1
      1
      -2
      4

      w
      -3
      9
      4
      9

      e
      9
      -8
      -3
      2

      r
      -6
      -8
      7
      4

```python
df['q':'e']
```

      a
      b
      c
      d

      q
      -1
      1
      -2
      4

      w
      -3
      9
      4
      9

      e
      9
      -8
      -3
      2

```python
df[1:3]
```

      a
      b
      c
      d

      w
      -3
      9
      4
      9

      e
      9
      -8
      -3
      2

```python
df.loc[:, 'a':'c']
```

      a
      b
      c

      q
      -1
      1
      -2

      w
      -3
      9
      4

      e
      9
      -8
      -3

      r
      -6
      -8
      7

```python
df.iloc[:, 1:3]
```

      b
      c

      q
      1
      -2

      w
      9
      4

      e
      -8
      -3

      r
      -8
      7

```python
df = pd.DataFrame(np.random.randint(-10, 10, size=(4, 4)),
                 columns=[1, 2, 10, 4],
                 index=[3, 4, 5, 6])
```

```python
df
```

      1
      2
      10
      4

      3
      -8
      -8
      -3
      8

      4
      4
      -1
      -1
      9

      5
      0
      1
      8
      -6

      6
      -4
      -7
      1
      -7

```python
df.loc[3]
```

    1    -8
    2    -8
    10   -3
    4     8
    Name: 3, dtype: int32

```python
df.iloc[3]
```

    1    -4
    2    -7
    10    1
    4    -7
    Name: 6, dtype: int32

```python
df.columns
```

    Int64Index([1, 2, 10, 4], dtype='int64')

```python
df.index
```

    Int64Index([3, 4, 5, 6], dtype='int64')

```python
df.rename(columns={1: 15}, inplace=True)
```

```python
df = pd.read_csv("http://www.ats.ucla.edu/stat/data/hsb2.csv")
```

```python
df[['read', 'write', 'math', 'science', 'socst']].mean()
```

    read       52.230
    write      52.775
    math       52.645
    science    51.850
    socst      52.405
    dtype: float64

```python
%matplotlib inline
```

```python
df.plot.scatter(x='read', y='write')
```

```python
courses = ['read', 'write', 'math', 'science', 'socst']
df.groupby('ses').mean()[courses]
```

      read
      write
      math
      science
      socst

      ses

      1
      48.276596
      50.617021
      49.170213
      47.702128
      47.319149

      2
      51.578947
      51.926316
      52.210526
      51.705263
      52.031579

      3
      56.500000
      55.913793
      56.172414
      55.448276
      57.137931

```python
# df[(df['schtyp'] == 1) & (df['prog'] == 3)]
df.query('schtyp == 1 and prog == 3').hist('read')
```

```python
df['gender'] = 'male'
df.loc[df['female'] == 1, 'gender'] = 'female'
```

```python
df['average'] = df[courses].mean(axis=1)
```

```python
courses
```

    ['read', 'write', 'math', 'science', 'socst']

```python
weights = pd.Series({'read': 0.2, 'write': 0.1, 'math': 0.3, 'socst': 0.1,
                     'science': 0.3})
```

```python
weights
```

    math       0.3
    read       0.2
    science    0.3
    socst      0.1
    write      0.1
    dtype: float64

```python
df['weighted_average'] = df[courses].dot(weights)
```

```python
df.sort_values('id', inplace=True)
```

```python

```

```python
df.sort_values(['female', 'prog'])
```

      id
      female
      race
      ses
      schtyp
      prog
      read
      write
      math
      science
      socst
      gender
      average
      weighted_average

      52
      21
      0
      1
      2
      1
      1
      44
      44
      61
      50
      46
      male
      49.0
      51.1

      29
      29
      0
      2
      1
      1
      1
      52
      44
      49
      55
      41
      male
      48.2
      50.1

      43
      40
      0
      3
      1
      1
      1
      42
      41
      43
      50
      41
      male
      43.4
      44.5

      6
      50
      0
      3
      2
      1
      1
      50
      59
      42
      53
      61
      male
      53.0
      50.5

      44
      62
      0
      4
      3
      1
      1
      65
      65
      48
      63
      66
      male
      61.4
      59.4

      0
      70
      0
      4
      1
      1
      1
      57
      52
      41
      47
      57
      male
      50.8
      48.7

      8
      84
      0
      4
      2
      1
      1
      63
      57
      54
      58
      51
      male
      56.6
      57.0

      19
      85
      0
      4
      2
      1
      1
      55
      39
      57
      53
      46
      male
      50.0
      52.5

      2
      86
      0
      4
      3
      1
      1
      44
      33
      54
      58
      31
      male
      44.0
      48.8

      63
      108
      0
      4
      2
      1
      1
      34
      33
      41
      36
      36
      male
      36.0
      36.8

      15
      115
      0
      4
      1
      1
      1
      42
      49
      43
      50
      56
      male
      48.0
      46.8

      77
      123
      0
      4
      3
      1
      1
      68
      59
      56
      63
      66
      male
      62.4
      61.8

      30
      126
      0
      4
      2
      1
      1
      42
      31
      57
      47
      51
      male
      45.6
      47.8

      62
      134
      0
      4
      1
      1
      1
      44
      44
      39
      34
      46
      male
      41.4
      39.7

      35
      144
      0
      4
      3
      1
      1
      60
      65
      58
      61
      66
      male
      62.0
      60.8

      91
      149
      0
      4
      1
      1
      1
      63
      49
      49
      66
      46
      male
      54.6
      56.6

      70
      155
      0
      4
      2
      1
      1
      44
      44
      46
      39
      51
      male
      44.8
      43.8

      73
      157
      0
      4
      2
      1
      1
      68
      59
      58
      74
      66
      male
      65.0
      65.7

      20
      167
      0
      4
      2
      1
      1
      63
      49
      35
      66
      41
      male
      50.8
      51.9

      45
      169
      0
      4
      1
      1
      1
      55
      59
      63
      69
      46
      male
      58.4
      61.1

      17
      195
      0
      4
      2
      2
      1
      57
      57
      60
      58
      56
      male
      57.6
      58.1

      83
      3
      0
      1
      1
      1
      2
      63
      65
      48
      63
      56
      male
      59.0
      58.0

      75
      5
      0
      1
      1
      1
      2
      47
      40
      43
      45
      31
      male
      41.2
      42.9

      49
      7
      0
      1
      2
      1
      2
      57
      54
      59
      47
      51
      male
      53.6
      53.7

      7
      11
      0
      1
      2
      1
      2
      34
      46
      45
      39
      36
      male
      40.0
      40.2

      79
      14
      0
      1
      3
      1
      2
      47
      41
      54
      42
      56
      male
      48.0
      47.9

      23
      20
      0
      1
      3
      1
      2
      60
      52
      57
      61
      61
      male
      58.2
      58.7

      90
      24
      0
      2
      2
      1
      2
      52
      62
      66
      47
      46
      male
      54.6
      55.1

      50
      27
      0
      2
      2
      1
      2
      53
      61
      61
      57
      56
      male
      57.6
      57.7

      14
      38
      0
      3
      1
      1
      2
      45
      57
      50
      31
      56
      male
      47.8
      44.6

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      181
      193
      1
      4
      2
      2
      2
      44
      49
      48
      39
      51
      female
      46.2
      44.9

      99
      194
      1
      4
      3
      2
      2
      63
      63
      69
      61
      61
      female
      63.4
      64.0

      162
      198
      1
      4
      3
      2
      2
      47
      61
      51
      63
      31
      female
      50.6
      52.8

      98
      1
      1
      1
      1
      1
      3
      34
      44
      40
      39
      41
      female
      39.6
      39.0

      138
      2
      1
      1
      2
      1
      3
      39
      41
      33
      42
      41
      female
      39.2
      38.5

      171
      13
      1
      1
      2
      1
      3
      47
      46
      39
      47
      61
      female
      48.0
      45.9

      184
      32
      1
      2
      3
      1
      3
      50
      67
      66
      66
      56
      female
      61.0
      61.9

      119
      37
      1
      3
      1
      1
      3
      41
      47
      40
      39
      51
      female
      43.6
      41.7

      137
      42
      1
      3
      2
      1
      3
      46
      52
      55
      44
      56
      female
      50.6
      49.7

      124
      44
      1
      3
      1
      1
      3
      47
      62
      45
      34
      46
      female
      46.8
      43.9

      129
      45
      1
      3
      1
      1
      3
      34
      35
      41
      29
      26
      female
      33.0
      33.9

      178
      64
      1
      4
      3
      1
      3
      50
      52
      45
      58
      36
      female
      48.2
      49.7

      132
      66
      1
      4
      2
      1
      3
      68
      62
      56
      50
      51
      female
      57.4
      56.7

      165
      69
      1
      4
      1
      1
      3
      44
      44
      40
      40
      31
      female
      39.8
      40.3

      133
      72
      1
      4
      2
      1
      3
      42
      54
      47
      47
      46
      female
      47.2
      46.6

      146
      83
      1
      4
      2
      1
      3
      50
      62
      41
      55
      31
      female
      47.8
      48.1

      107
      89
      1
      4
      1
      1
      3
      35
      35
      40
      51
      33
      female
      38.8
      41.1

      128
      91
      1
      4
      3
      1
      3
      50
      49
      56
      47
      46
      female
      49.6
      50.4

      169
      98
      1
      4
      1
      1
      3
      57
      60
      51
      53
      37
      female
      51.6
      52.3

      115
      106
      1
      4
      2
      1
      3
      36
      44
      37
      42
      41
      female
      40.0
      39.4

      156
      110
      1
      4
      2
      1
      3
      52
      55
      50
      54
      61
      female
      54.4
      53.2

      1
      121
      1
      4
      2
      1
      3
      68
      59
      53
      63
      61
      female
      60.8
      60.4

      190
      124
      1
      4
      1
      1
      3
      42
      54
      41
      42
      41
      female
      44.0
      42.8

      152
      138
      1
      4
      2
      1
      3
      43
      57
      40
      50
      51
      female
      48.2
      46.4

      142
      142
      1
      4
      2
      1
      3
      47
      42
      52
      39
      51
      female
      46.2
      46.0

      196
      145
      1
      4
      2
      1
      3
      42
      46
      38
      36
      46
      female
      41.6
      39.8

      157
      148
      1
      4
      2
      1
      3
      42
      57
      51
      47
      61
      female
      51.6
      49.6

      123
      151
      1
      4
      2
      1
      3
      47
      46
      52
      48
      46
      female
      47.8
      48.6

      110
      162
      1
      4
      2
      1
      3
      57
      52
      40
      61
      56
      female
      53.2
      52.5

      192
      184
      1
      4
      2
      2
      3
      50
      52
      53
      55
      56
      female
      53.2
      53.2

200 rows × 14 columns

```python
df.to_
```

      id
      female
      race
      ses
      schtyp
      prog
      read
      write
      math
      science
      socst
      gender
      average
      weighted_average

      98
      1
      1
      1
      1
      1
      3
      34
      44
      40
      39
      41
      female
      39.6
      39.0

      138
      2
      1
      1
      2
      1
      3
      39
      41
      33
      42
      41
      female
      39.2
      38.5

      83
      3
      0
      1
      1
      1
      2
      63
      65
      48
      63
      56
      male
      59.0
      58.0

      111
      4
      1
      1
      1
      1
      2
      44
      50
      41
      39
      51
      female
      45.0
      42.9

      75
      5
      0
      1
      1
      1
      2
      47
      40
      43
      45
      31
      male
      41.2
      42.9

      148
      6
      1
      1
      1
      1
      2
      47
      41
      46
      40
      41
      female
      43.0
      43.4

      49
      7
      0
      1
      2
      1
      2
      57
      54
      59
      47
      51
      male
      53.6
      53.7

      93
      8
      1
      1
      1
      1
      2
      39
      44
      52
      44
      48
      female
      45.4
      45.8

      59
      9
      0
      1
      2
      1
      3
      48
      49
      52
      44
      51
      male
      48.8
      48.4

      153
      10
      1
      1
      2
      1
      1
      47
      54
      49
      53
      61
      female
      52.8
      51.5

      7
      11
      0
      1
      2
      1
      2
      34
      46
      45
      39
      36
      male
      40.0
      40.2

      24
      12
      0
      1
      2
      1
      3
      37
      44
      45
      39
      46
      male
      42.2
      41.6

      171
      13
      1
      1
      2
      1
      3
      47
      46
      39
      47
      61
      female
      48.0
      45.9

      79
      14
      0
      1
      3
      1
      2
      47
      41
      54
      42
      56
      male
      48.0
      47.9

      55
      15
      0
      1
      3
      1
      3
      39
      39
      44
      26
      42
      male
      38.0
      36.9

      38
      16
      0
      1
      1
      1
      3
      47
      31
      44
      36
      36
      male
      38.8
      40.1

      143
      17
      1
      1
      2
      1
      2
      47
      57
      48
      44
      41
      female
      47.4
      46.8

      69
      18
      0
      1
      2
      1
      3
      50
      33
      49
      44
      36
      male
      42.4
      44.8

      140
      19
      1
      1
      1
      1
      1
      28
      46
      43
      44
      51
      female
      42.4
      41.4

      23
      20
      0
      1
      3
      1
      2
      60
      52
      57
      61
      61
      male
      58.2
      58.7

      52
      21
      0
      1
      2
      1
      1
      44
      44
      61
      50
      46
      male
      49.0
      51.1

      57
      22
      0
      1
      2
      1
      3
      42
      39
      39
      56
      46
      male
      44.4
      45.4

      185
      23
      1
      2
      1
      1
      2
      65
      65
      64
      58
      71
      female
      64.6
      63.2

      90
      24
      0
      2
      2
      1
      2
      52
      62
      66
      47
      46
      male
      54.6
      55.1

      187
      25
      1
      2
      2
      1
      1
      47
      44
      42
      42
      36
      female
      42.2
      42.6

      173
      26
      1
      2
      3
      1
      2
      60
      59
      62
      61
      51
      female
      58.6
      59.9

      50
      27
      0
      2
      2
      1
      2
      53
      61
      61
      57
      56
      male
      57.6
      57.7

      127
      28
      1
      2
      2
      1
      1
      39
      53
      54
      50
      41
      female
      47.4
      48.4

      29
      29
      0
      2
      1
      1
      1
      52
      44
      49
      55
      41
      male
      48.2
      50.1

      193
      30
      1
      2
      3
      1
      2
      41
      59
      42
      34
      51
      female
      45.4
      42.0

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      66
      171
      0
      4
      2
      1
      2
      60
      54
      60
      55
      66
      male
      59.0
      58.5

      4
      172
      0
      4
      2
      1
      2
      47
      52
      57
      53
      61
      male
      54.0
      53.7

      95
      173
      1
      4
      1
      1
      1
      50
      62
      61
      63
      51
      female
      57.4
      58.5

      82
      174
      0
      4
      2
      2
      2
      68
      59
      71
      66
      56
      male
      64.0
      66.2

      191
      175
      1
      4
      3
      2
      1
      36
      57
      42
      50
      41
      female
      45.2
      44.6

      40
      176
      0
      4
      2
      2
      2
      47
      47
      41
      42
      51
      male
      45.6
      44.1

      41
      177
      0
      4
      2
      2
      2
      55
      59
      62
      58
      51
      male
      57.0
      58.0

      27
      178
      0
      4
      2
      2
      3
      47
      57
      57
      58
      46
      male
      53.0
      54.2

      194
      179
      1
      4
      2
      2
      2
      47
      65
      60
      50
      56
      female
      55.6
      54.5

      109
      180
      1
      4
      3
      2
      2
      71
      65
      69
      58
      71
      female
      66.8
      65.9

      60
      181
      0
      4
      2
      2
      2
      50
      46
      45
      58
      61
      male
      52.0
      51.6

      147
      182
      1
      4
      2
      2
      2
      44
      52
      43
      44
      51
      female
      46.8
      45.2

      53
      183
      0
      4
      2
      2
      2
      63
      59
      49
      55
      71
      male
      59.4
      56.8

      192
      184
      1
      4
      2
      2
      3
      50
      52
      53
      55
      56
      female
      53.2
      53.2

      58
      185
      0
      4
      2
      2
      2
      63
      57
      55
      58
      41
      male
      54.8
      56.3

      168
      186
      1
      4
      2
      2
      2
      57
      62
      63
      55
      41
      female
      55.6
      57.1

      197
      187
      1
      4
      2
      2
      1
      57
      41
      57
      55
      52
      female
      52.4
      54.3

      188
      188
      1
      4
      3
      2
      2
      63
      62
      56
      55
      61
      female
      59.4
      58.2

      48
      189
      0
      4
      2
      2
      2
      47
      59
      63
      53
      46
      male
      53.6
      54.7

      136
      190
      1
      4
      2
      2
      2
      47
      59
      54
      58
      46
      female
      52.8
      53.5

      145
      191
      1
      4
      3
      2
      2
      47
      52
      43
      48
      61
      female
      50.2
      48.0

      32
      192
      0
      4
      3
      2
      2
      65
      67
      63
      66
      71
      male
      66.4
      65.5

      181
      193
      1
      4
      2
      2
      2
      44
      49
      48
      39
      51
      female
      46.2
      44.9

      99
      194
      1
      4
      3
      2
      2
      63
      63
      69
      61
      61
      female
      63.4
      64.0

      17
      195
      0
      4
      2
      2
      1
      57
      57
      60
      58
      56
      male
      57.6
      58.1

      28
      196
      0
      4
      3
      2
      2
      44
      38
      49
      39
      46
      male
      43.2
      43.6

      64
      197
      0
      4
      3
      2
      2
      50
      42
      50
      36
      61
      male
      47.8
      46.1

      162
      198
      1
      4
      3
      2
      2
      47
      61
      51
      63
      31
      female
      50.6
      52.8

      34
      199
      0
      4
      3
      2
      2
      52
      59
      50
      61
      61
      male
      56.6
      55.7

      36
      200
      0
      4
      2
      2
      2
      68
      54
      75
      66
      66
      male
      65.8
      67.9

200 rows × 14 columns

```python
df.to_excel("some_file.xls")
```

```python
import os
```

```python
os.getcwd()
```

    'C:\\Users\\student\\Documents'

```python
from statsmodels.formula.api import ols
```

$$ science = \beta_0 + \beta_{read}\cdot read + \beta_{write}\cdot write + \beta_{math}\cdot math + \varepsilon$$

```python
fit1 = ols('science ~ read + write + math', data=df ).fit()
```

```python
fit1.summary()
```

OLS Regression Results

  Dep. Variable:	         science	       R-squared:         	    0.500

  Model:	                   OLS	         Adj. R-squared:    	    0.492

  Method:	             Least Squares	    F-statistic:       	    65.32

  Date:	             Tue, 21 Feb 2017	   Prob (F-statistic):	 2.54e-29

  Time:	                 18:48:18	       Log-Likelihood:    	  -672.51

  No. Observations:	         200	        AIC:               	    1353.

  Df Residuals:	             196	        BIC:               	    1366.

  Df Model:	                   3

  Covariance Type:	      nonrobust

      	         coef	     std err	      t	      P>|t|	 [95.0% Conf. Int.]

  Intercept	     8.4074	     3.193	     2.633	  0.009	     2.111    14.704

  read	          0.3015	     0.069	     4.390	  0.000	     0.166     0.437

  write	         0.2065	     0.071	     2.918	  0.004	     0.067     0.346

  math	          0.3190	     0.077	     4.161	  0.000	     0.168     0.470

  Omnibus:	        1.718	   Durbin-Watson:     	    1.694

  Prob(Omnibus):	  0.424	   Jarque-Bera (JB):  	    1.354

  Skew:	           0.128	   Prob(JB):          	    0.508

  Kurtosis:	       3.312	   Cond. No.          	     590.

```python
fit1 = ols('science ~ read + write + math + C(prog)', data=df ).fit()
```

```python
fit1.summary()
```

OLS Regression Results

  Dep. Variable:	         science	       R-squared:         	    0.527

  Model:	                   OLS	         Adj. R-squared:    	    0.515

  Method:	             Least Squares	    F-statistic:       	    43.24

  Date:	             Tue, 21 Feb 2017	   Prob (F-statistic):	 8.14e-30

  Time:	                 18:49:48	       Log-Likelihood:    	  -666.93

  No. Observations:	         200	        AIC:               	    1346.

  Df Residuals:	             194	        BIC:               	    1366.

  Df Model:	                   5

  Covariance Type:	      nonrobust

        	          coef	     std err	      t	      P>|t|	 [95.0% Conf. Int.]

  Intercept	        6.6091	     3.588	     1.842	  0.067	    -0.467    13.685

  C(prog)[T.2]	    -4.2742	     1.299	    -3.290	  0.001	    -6.837    -1.712

  C(prog)[T.3]	    -1.7394	     1.444	    -1.205	  0.230	    -4.587     1.108

  read	             0.3230	     0.068	     4.782	  0.000	     0.190     0.456

  write	            0.2204	     0.070	     3.149	  0.002	     0.082     0.358

  math	             0.3688	     0.077	     4.786	  0.000	     0.217     0.521

  Omnibus:	        2.391	   Durbin-Watson:     	    1.705

  Prob(Omnibus):	  0.302	   Jarque-Bera (JB):  	    2.023

  Skew:	           0.172	   Prob(JB):          	    0.364

  Kurtosis:	       3.353	   Cond. No.          	     688.

```python
import seaborn
```

```python
import matplotlib.pyplot as plt
```

```python
# pip install seaborn
```

```python
import seaborn as sns
```

```python
df.boxplot(column='read', by='gender')
```

```python
sns.boxplot(x='gender', y='read', hue='ses', data=df, palette='PRGn')
```

```python
sns.kdeplot(df[['read', 'write']])
```

```python
sns.jointplot(df['read'], df['write'], kind='scatter', ratio=5)
```

```python
import matplotlib.pyplot as plt
sns.lmplot(x='read', y='write', ci=None, hue='ses', data=df)
```

```python
M = np.array([[1, 2, 3], [2, 3, 4]])
N = np.array([[3, 2], [4, 5]])
```

```python
N @ M
```

    array([[ 7, 12, 17],
           [14, 23, 32]])

```python
M
```

    array([[1, 2, 3],
           [2, 3, 4]])

```python
N
```

    array([[3, 2],
           [4, 5]])

```python
from scipy.linalg import eigvals
```

```python
eigvals(N)
```

    array([ 1.+0.j,  7.+0.j])

```python
eigvals(np.array([[2, 3], [1, 4]]))
```

    array([ 1.+0.j,  5.+0.j])

```python
np.array([[2, 3], [1, 4]])
```

    array([[2, 3],
           [1, 4]])

```python
np.linalg.solve(np.array([[1, 2], [4, 5]]), np.array([6, 15]))
```

    array([ 0.,  3.])

```python
import sympy
```

```python
x, y = sympy.symbols('x y')
```

```python
sympy.init_printing(use_latex=True)
```

```python
f = sympy.sin(x) * sympy.exp(x ** 2)
```

```python
f.diff(x)
```

```python
sympy.latex(sympy.Matrix([[x, y], [y, x]]))
```

    '\\left[\\begin{matrix}x & y\\\\y & x\\end{matrix}\\right]'

```python
sympy.latex(f.series(n=10))
```

    'x + \\frac{5 x^{3}}{6} + \\frac{41 x^{5}}{120} + \\frac{461 x^{7}}{5040} + \\frac{6481 x^{9}}{362880} + \\mathcal{O}\\left(x^{10}\\right)'

```python
sympy.latex(sympy.expand((x+y)**100))
```

    'x^{100} + 100 x^{99} y + 4950 x^{98} y^{2} + 161700 x^{97} y^{3} + 3921225 x^{96} y^{4} + 75287520 x^{95} y^{5} + 1192052400 x^{94} y^{6} + 16007560800 x^{93} y^{7} + 186087894300 x^{92} y^{8} + 1902231808400 x^{91} y^{9} + 17310309456440 x^{90} y^{10} + 141629804643600 x^{89} y^{11} + 1050421051106700 x^{88} y^{12} + 7110542499799200 x^{87} y^{13} + 44186942677323600 x^{86} y^{14} + 253338471349988640 x^{85} y^{15} + 1345860629046814650 x^{84} y^{16} + 6650134872937201800 x^{83} y^{17} + 30664510802988208300 x^{82} y^{18} + 132341572939212267400 x^{81} y^{19} + 535983370403809682970 x^{80} y^{20} + 2041841411062132125600 x^{79} y^{21} + 7332066885177656269200 x^{78} y^{22} + 24865270306254660391200 x^{77} y^{23} + 79776075565900368755100 x^{76} y^{24} + 242519269720337121015504 x^{75} y^{25} + 699574816500972464467800 x^{74} y^{26} + 1917353200780443050763600 x^{73} y^{27} + 4998813702034726525205100 x^{72} y^{28} + 12410847811948286545336800 x^{71} y^{29} + 29372339821610944823963760 x^{70} y^{30} + 66324638306863423796047200 x^{69} y^{31} + 143012501349174257560226775 x^{68} y^{32} + 294692427022540894366527900 x^{67} y^{33} + 580717429720889409486981450 x^{66} y^{34} + 1095067153187962886461165020 x^{65} y^{35} + 1977204582144932989443770175 x^{64} y^{36} + 3420029547493938143902737600 x^{63} y^{37} + 5670048986634686922786117600 x^{62} y^{38} + 9013924030034630492634340800 x^{61} y^{39} + 13746234145802811501267369720 x^{60} y^{40} + 20116440213369968050635175200 x^{59} y^{41} + 28258808871162574166368460400 x^{58} y^{42} + 38116532895986727945334202400 x^{57} y^{43} + 49378235797073715747364762200 x^{56} y^{44} + 61448471214136179596720592960 x^{55} y^{45} + 73470998190814997343905056800 x^{54} y^{46} + 84413487283064039501507937600 x^{53} y^{47} + 93206558875049876949581681100 x^{52} y^{48} + 98913082887808032681188722800 x^{51} y^{49} + 100891344545564193334812497256 x^{50} y^{50} + 98913082887808032681188722800 x^{49} y^{51} + 93206558875049876949581681100 x^{48} y^{52} + 84413487283064039501507937600 x^{47} y^{53} + 73470998190814997343905056800 x^{46} y^{54} + 61448471214136179596720592960 x^{45} y^{55} + 49378235797073715747364762200 x^{44} y^{56} + 38116532895986727945334202400 x^{43} y^{57} + 28258808871162574166368460400 x^{42} y^{58} + 20116440213369968050635175200 x^{41} y^{59} + 13746234145802811501267369720 x^{40} y^{60} + 9013924030034630492634340800 x^{39} y^{61} + 5670048986634686922786117600 x^{38} y^{62} + 3420029547493938143902737600 x^{37} y^{63} + 1977204582144932989443770175 x^{36} y^{64} + 1095067153187962886461165020 x^{35} y^{65} + 580717429720889409486981450 x^{34} y^{66} + 294692427022540894366527900 x^{33} y^{67} + 143012501349174257560226775 x^{32} y^{68} + 66324638306863423796047200 x^{31} y^{69} + 29372339821610944823963760 x^{30} y^{70} + 12410847811948286545336800 x^{29} y^{71} + 4998813702034726525205100 x^{28} y^{72} + 1917353200780443050763600 x^{27} y^{73} + 699574816500972464467800 x^{26} y^{74} + 242519269720337121015504 x^{25} y^{75} + 79776075565900368755100 x^{24} y^{76} + 24865270306254660391200 x^{23} y^{77} + 7332066885177656269200 x^{22} y^{78} + 2041841411062132125600 x^{21} y^{79} + 535983370403809682970 x^{20} y^{80} + 132341572939212267400 x^{19} y^{81} + 30664510802988208300 x^{18} y^{82} + 6650134872937201800 x^{17} y^{83} + 1345860629046814650 x^{16} y^{84} + 253338471349988640 x^{15} y^{85} + 44186942677323600 x^{14} y^{86} + 7110542499799200 x^{13} y^{87} + 1050421051106700 x^{12} y^{88} + 141629804643600 x^{11} y^{89} + 17310309456440 x^{10} y^{90} + 1902231808400 x^{9} y^{91} + 186087894300 x^{8} y^{92} + 16007560800 x^{7} y^{93} + 1192052400 x^{6} y^{94} + 75287520 x^{5} y^{95} + 3921225 x^{4} y^{96} + 161700 x^{3} y^{97} + 4950 x^{2} y^{98} + 100 x y^{99} + y^{100}'

```python

```


# Pendulum Dynamics

If you read my article on Dynamic Systems you'll know by now that I think the concepts and tools of Dynamic Systems Theory are useful when trying to explain brain phenomena. That article also explained, in a rather abstract way, what Dynamic Systems are without actually giving a detailed example of one and how the theory of Dynamic Systems are put into practice.

With all that said, lets talk about pendulums!

[image here]

The diagram above shows a pendulum that's been lifted from it's resting place and let go to swing freely. I've made a few assumptions here, the first is that the pendulum's cord has a mass much less than that of it's bob, so small in fact that we can safely discount it from our calculations. The second is that the cord is completely rigid, it has no flex in it whatever. Lastly, I'm assuming that the pendulum is constrained to swing around a single axis meaining that it will swing in two-dimensions only.

Before we get any further into this article it's important to remind ourselves what we're trying to acheive.

When considering Dynamic Systems what we are asking oursleves is how the system evolves over time, in this case how the position of the bob changes over time. We're not interested in knowing where the bob is at any given time, that may not even be possible to know (take the famous three-body problem for example). Instead we're interested in the long term behaviour of a system, we ask questions like: "will this system eventually come to rest, and if so, where?", "does this system exhibit cyclical behaviour" and "are there any periods of sensitivity in the system where small changes lead to vastly different results?"

This article is a general exploration of the dynamics of pendula, we're not going to answer any question in particular, so starting with some basic expressions of moving bodies, such as Newton's 2nd law of motion, seems as good a place as any to begin. According to Newton.

$$
F = ma
$$

where $F$ is *force*, $m$ is the *mass* of the object in question and $a$ is it's *acceleration*. You often see this formulation in classical linear system like a ball rolling down a slope, but our pendulum is not a ball rolling down a slope, it's a ball rotating around a fixed point, so I'm going to introduce a similar and related equation for rotating objects.

$$
\tau = I \alpha
$$

In this equation $\tau$ is *torque*, $I$ is the *moment of inertia* and $\alpha$ is *angular acceleration*. If we take a closer look at what these quantities are you'll see the similarity to Newton's second law. Torque is an angular force or turning force, the angular acceleration is exactly what it sounds like - acceleration about a rotation point and the *moment of inertia* is the systems resistance to change, it's how much force is required to reach a particular angular acceleration in exactly the same way that mass defines the force required to reach a particular acceleration in a linear system.

$$
I = ml^2
$$

As I've already mentioned this article is an exploration of how pendula move over time. We're interested in how the bob's position changes over time, so it's the acceleration that's most important to us.

Let's begin then by deriving angular acceleration ($\alpha$) from Newton's second law.

## Deriving $\alpha$

Once the pendulum is let go it will swing freely due to gravity. For the moment we will ignore air resistance, friction and so forth so that it is only gravity acting upon the pendulum. On Earth gravity accelerates objects toward the ground at approximately ___$9.81 ms^{-2}$___ and we label this constant $g$.

If our pendulum were hanging in it's resting place it would feel gravity pulling it directly down toward the ground, but our pendulum has been deflected away from this vector by angle $\theta$ so, prevented from falling toward the Earth by it's cord our pendulum will instead be accelerated to it's resting place by portion or component $g$.

The diagram shows acceleration due to gravity $g$ and the component of this, labelled $a$ for acceleration, which is causing the pendulum to swing. As you can see, $g$ and $a$ form two sides of a right-angle triangle so we can use $\theta$ and a little trigonometry to calculate the length of $a$ and therefore the magnitude of this acceleration.

$$
a = -g \cdot sin(\theta)
$$

You might be wondering where that minus came from? The pendulum will feel an acceleration toward it's resting place so it will always be opposite to the side of the diagram that the bob is on. For example, if the bob is to the right of the resting place $a$ will point to the left and vice-versa.

So far we've derived an equation which can be used to calculate the acceleration of a pendulum. The only problem is that the acceleration is linear. You can see from the diagram that the acceleration vector begins at the centre of the bob and points perpendicular to the cord towards the resting position. But we know that the pendulum cannot follow this path because it is constrained by it cord. Instead it will follow the green dotted curve which is a section of a circle centred at the pendulums axis of rotation with a radius of $l$.

As long as $\theta$ is measured in radians it is simple to convert linear acceleration (the length of $a$) into angular acceleration (or arc-length which we shall label $\alpha$).

$$
\begin{align}
s & = l \theta \\
v & = l \frac{\delta \theta}{\delta t} & = l \dot \theta \\
a & = l \frac{\delta^{2} \theta}{\delta t^{2}} & = l \ddot \theta \\
\end{align}
$$

Where $s$ is arc length, $v$ is arc velocity (the first derivative of $s$) and $a$ is arc acceleration (the second derivate of $s$).

$$
\alpha = l \ddot \theta = -g \cdot sin(\theta)
$$

We've now derived $\alpha$ from Newton's second law so we can substitute it back in to the equation for angular force or torque

$$
\tau = I \alpha = I(-g \cdot sin(\theta))
$$

We can clean this up a little with some basic algebra and further substitutions. I apollogies in advance for the following block of maths, but I want to show as clearly as possible each step required to reach the final form.

$$
\begin{align}
I \alpha &= I(-g \cdot sin(\theta)) \Rightarrow \\
Il \ddot \theta &= I(-g \cdot sin(\theta)) \Rightarrow \\
I \ddot \theta &= I(-\frac{g}{l} \cdot sin(\theta)) \Rightarrow \\
I \ddot \theta &= ml^2(-\frac{g}{l} \cdot sin(\theta)) \Rightarrow \\
I \ddot \theta &= -mlg \cdot sin(\theta) \Rightarrow
\end{align}
$$

We can already see from the last equation above that a pendulum is a dynamic system because the change in $\theta$ is dependent upon $\theta$, ot to put it another way $\theta_{t + 1}$ depends on $\theta_t$.

## Damping & Control

So far I haven't taken into account any other forces that might act on the pendulum. Apart from the initial force to turn the pendulum by $\theta$ the only force acting on the pendulum is gravity. But we know in the real world there are many forces acting on the pendulum. We can break them into two groups; those forces that slow the pendulum down (damping) and those that speed it up (control).

Damping forces are generated from things like friction and air resistence and although we could model them with complex non-linear functions we want to keep things simple, besides depending on the system a simple linear model may be more appropriate.

$$
Q = -cv
$$

Where $c$ is a damping coefficient which models the strength of the combined damping force and $v$ is the velocity, which in our case is the change in $\theta$ over time which we write as $\dot \theta$. The minus sign is there because the damping force always acts to slow the pendulum down.

```python

```