Questo notebook è fortemente ispirato all'articolo *A Kalman Filtering Tutorial for Undergraduate Students*.

### Riferimenti bibliografici:

* Rhudy, M., B.; Salguero, R., A. & Holappa, K. (2017), [A Kalman Filtering Tutorial for Undergraduate Students](http://aircconline.com/ijcses/V8N1/8117ijcses01.pdf).

# Modello lineare dinamico e filtro di Kalman con NumPy

## Indice

1. [Modello lineare dinamico](#modello_lineare_dinamico)

2. [Filtro di Kalman](#kalman)

3. [Esempio](#esempio)

    3.1 [Oggetto in caduta libera senza resistenza aerea](#oggetto_caduta)

    3.2 [Termini del problema](#termini)

    3.3 [Generazione delle serie degli stati e degli output](#generazione)

    3.4 [Simulazione di misurazioni tramite l'aggiunta di rumore a y](#misurazioni)

    3.5 [Filtraggio di Kalman](#filtraggio)

    3.6 [Grafico degli stati veri e stimati](#grafico)

```python
import matplotlib.pyplot as plt
import numpy as np
import inspect

%load_ext autoreload
%autoreload 2
```

# 1. Modello lineare dinamico

$$
\begin{align}
    \textbf{x}_k &= \textbf{F}_{k-1}\textbf{x}_{k-1} + \textbf{G}_{k-1}\textbf{u}_{k-1} + \textbf{w}_k \tag{1} \\
    \textbf{y}_k &= \textbf{H}_k\textbf{x}_k  + \textbf{v}_k \tag{2}
\end{align}
$$

### Dimensioni delle variabili del sistema a tempo discreto

| Variabile  | Descrizione                          | Dimensione     |
|------------|--------------------------------------|----------------|
|$\textbf{x}$| Vettore di stato                     |$n_x \times 1$  |
|$\textbf{y}$| Vettore di output                    |$n_y \times 1$  |
|$\textbf{u}$| Vettore di input                     |$n_u \times 1$  |
|$\textbf{w}$| Vettore del rumore del processo      |$n_x \times 1$  |
|$\textbf{v}$| Vettore del rumore di misurazione    |$n_y \times 1$  |
|$\textbf{F}$| Matrice del sistema - stato          |$n_x \times n_x$|
|$\textbf{G}$| Matrice del sistema - input          |$n_x \times n_u$|
|$\textbf{H}$| Matrice delle osservazioni           |$n_y \times n_x$|
|$\textbf{P}$| Matrice di covarianza di $\textbf{x}$|$n_x \times n_x$|
|$\textbf{Q}$| Matrice di covarianza di $\textbf{w}$|$n_x \times n_x$|
|$\textbf{R}$| Matrice di covarianza di $\textbf{v}$|$n_y \times n_y$|
|$\textbf{z}$| Misurazione di $\textbf{y}$          |$n_y \times 1$  |

# 2. Filtro di Kalman

**Algoritmo** filtro di Kalman

$\hat{\textbf{x}}_0$: stima iniziale del vettore di stato.

$\hat{\textbf{P}}_0$: stima iniziale della matrice di covarianza dell'errore di stato.

1. $\hat{\textbf{x}}_{k|k-1} = \textbf{F}_{k-1}\hat{\textbf{x}}_{k-1} + \textbf{G}_{k-1}\textbf{u}_{k-1},\quad \textit{vettore di stato previsto}$;
2. $\hat{\textbf{P}}_{k|k-1} = \textbf{F}_{k-1}\hat{\textbf{P}}_{k-1}\textbf{F}^\top_{k-1} + \textbf{Q}_{k-1},\quad \textit{matrice di covarianza dell'errore di stato prevista}$;
3. $\textbf{K}_k = \hat{\textbf{P}}_{k|k-1}\textbf{H}^\top_k(\textbf{H}_k\hat{\textbf{P}}_{k|k-1}\textbf{H}^\top_k + \textbf{R}_k)^{-1},\quad \textit{matrice del guadagno di Kalman}$;
4. $\hat{\textbf{x}}_{k} = \hat{\textbf{x}}_{k|k-1} + \textbf{K}_{k}(\textbf{z}_k - \textbf{H}_k\hat{\textbf{x}}_{k|k-1}),\quad \textit{correzione del vettore di stato previsto}$;
5. $\hat{\textbf{P}}_{k} = (\textbf{I} - \textbf{K}_k\textbf{H}_k)\hat{\textbf{P}}_{k|k-1},\, \textbf{I}\, \text{matrice identità},\quad \textit{correzione della matrice di covarianza dell'errore di stato prevista}$.

# 3. Esempio

## 3.1 Oggetto in caduta libera senza resistenza aerea

**Obiettivo**: determinare l'altezza $h$ di un oggetto in caduta libera senza resistenza aerea, in base all'informazione incerta riguardo la sua altezza iniziale e a delle misure della sua posizione a tempi successivi fornite da un telemetro laser.

Sia $g$ l'accelerazione dovuta alla gravità ($g = 9.80665\, m/s^2$), usando la cinematica delle particelle abbiamo che

$\ddot{h}(t) = -g,\quad \textit{accelerazione}\, a(t) = a \label{1}\tag{1}$

Dall'equazione che relaziona velocità e accelerazione $v(t) = \int_{t_0}^t a(t)\, dt$, quando $a(t)$ è costante come nel nostro caso, abbiamo che $v(t) = v(t_0) + a\Delta t$ con $\Delta t = t - t_0$ (e quindi $t_0 = t - \Delta t$), otteniamo

$\dot{h}(t) =\dot{h}(t_0) - g\Delta t,\quad \textit{velocità}\, v(t) \label{2}\tag{2}$

Integrando di nuovo otteniamo l'equazione del *moto uniformemente accelerato* $s(t) = \int_{t_0}^t a\Delta t + v(t_0)\, dt = s(t_0) + v(t_0)\Delta t + \frac{1}{2}a\Delta t^2$. Nel nostro caso

$h(t) = h(t_0) + \dot{h}(t_0)\Delta t -\frac{1}{2}g(\Delta t)^2,\quad \textit{posizione}\, s(t) \label{3}\tag{3}$

Passando ad un indice di tempo discreto $t = k \Delta t$ e ponendo $h_k = h(k \Delta t)$, possiamo riscrivere $\eqref{2}$ e $\eqref{3}$ come

$\dot{h}_k =\dot{h}_{k-1} - g\Delta t,\quad \textit{velocità}\, v_k \label{4}\tag{4}$

$h_k = h_{k-1} + \dot{h}_{k-1}\Delta t -\frac{1}{2}g(\Delta t)^2,\quad \textit{posizione}\, s_k \label{5}\tag{5}$

Siamo interessato alla stima della posizione dell'oggetto, sappiamo quindi che la posizione deve essere inclusa nel vettore di stato. Tuttavia, dato che anche la velocità appare nell'equazione della posizione, è necessario ottenere anche questa informazione. Un modo per averla è includere la velocità nel vettore di stato. Definiamo quindi il vettore di stato del filtro di Kalman come

$\textbf{x}_k = \begin{pmatrix} h_k \\ \dot{h}_k \end{pmatrix}\label{6}\tag{6}$

che risulta nella seguente espressione

$\textbf{x}_k = \begin{pmatrix} h_{k-1} + \dot{h}_{k-1}\Delta t -\frac{1}{2}g(\Delta t)^2 \\ \dot{h}_{k-1} - g\Delta t \end{pmatrix}\label{7}\tag{7}$

Con questa definizione, possiamo riscrivere queste equazioni in termini del vettore di stato, $\textbf{x}$, come in

$\textbf{x}_k = \begin{pmatrix} 1 & \Delta t \\ 0 & 1 \end{pmatrix}\textbf{x}_{k-1} + \begin{pmatrix} -\frac{1}{2}(\Delta t)^2 \\ -\Delta t \end{pmatrix}g \label{8}\tag{8}$

Abbiamo quindi il problema nella forma necessaria per applicare il filtro di Kalman

$\textbf{x}_k = \textbf{F}_{k-1}\textbf{x}_{k-1} + \textbf{G}_{k-1}\textbf{u}_{k-1} + \textbf{w}_k \label{9}\tag{9}$

$\textbf{F}_{k-1} = \begin{pmatrix} 1 & \Delta t \\ 0 & 1 \end{pmatrix},\, \textbf{G}_{k-1} = \begin{pmatrix} -\frac{1}{2}(\Delta t)^2 \\ -\Delta t \end{pmatrix},\, \textbf{u}_{k-1} = g, \textbf{w}_k = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \label{10}\tag{10}$

Ponendo $\textbf{w}_k$ uguale a zero stiamo assumendo che non ci sia errore nell'equazione stessa. Per questo problema, questa è un'assunzione, dato che ci potrebbe essere un disturbo dovuto alla resistenza dell'aria o ad altre fonti. Se il caso generale con rumore gaussiano è $\textbf{w}_k \sim \mathcal{N}(0, \textbf{Q}_k)$, stiamo quindi ponendo

$\textbf{Q}_k = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix} \label{11}\tag{11}$

Consideriamo ora la parte relativa alle misurazioni del sistema.

Immaginiamo uno scenario dove la posizione dell'oggetto può essere misurata con un telemetro laser con una deviazione standard dell'errore $\textbf{v}$ pari a 2 $m$. La matrice di covarianza di $\textbf{v}$, $\textbf{R}$, è quindi pari a $(2\,m)^2 = 4\,m^2$. Notare che sia $\textbf{v}$ che $\textbf{R}$ sono in questo caso degli scalari essendoci solo un termine nel vettore di output $\textbf{y}$ dato dalla relazione

$\textbf{y}_k = h_k  + \textbf{v}_k \label{12}\tag{12}$

che possiamo riscrivere come

$\textbf{y}_k = \begin{pmatrix} 1 & 0 \end{pmatrix}\textbf{x}_k + \textbf{v}_k \label{13}\tag{13}$

Rispetto all'equazione generica

$\textbf{y}_k = \textbf{H}_k\textbf{x}_k  + \textbf{v}_k \label{14}\tag{14}$

nel nostro caso abbiamo che

$\textbf{H}_k = \begin{pmatrix} 1 & 0 \end{pmatrix} \label{15}\tag{15}$

Sia la vera altezza inziale $h_0 = 100\, m$ e sia la nostra stima di essa $\hat{h}_0 = \hat{\textbf{x}}_0[0]  = 105\, m$. Dato che abbiamo stimato a occhio la posizione inziale, cosideriamo una varianza relativamente alta dell'errore commesso, $\hat{\textbf{P}}_0[0, 0] = 10\, m^2$. Assumiamo che l'oggetto parta da fermo $\hat{\dot{h}}_0 = \hat{\textbf{x}}_0[1] = 0\, m/s$. Quest'assunzione è piuttosto ragionevole, possiamo quindi considerare una piccola incertezza, $\hat{\textbf{P}}_0[1, 1] = 0.01\, m^2/s^2$. Infine immaginiamo di prendere $N = 1000$ misurazioni e che $\Delta t$ sia pari a $0.001\, s$.

## 3.2 Termini del problema

```python
u = np.array([[9.80665]]) # vettore (scalare in questo caso) di input
dt = 0.001 # tempo di campionamento (s)
F = np.array( # matrice del sistema - stato
    [[1, dt],
     [0, 1]]
)
G = np.array( # matrice (vettore in questo caso) del sistema - input
    [[-1 / 2 * dt ** 2],
     [-dt]]
)
H = np.array([[1, 0]]) # matrice (vettore in questo caso) delle osservazioni
Q = np.zeros((2, 2)) # matrice di covarianza dell'errore del processo w
R = np.array([[4]]) # matrice di covarianza (scalare in questo caso) dell'errore di output v
x0 = np.array( # vettore del vero stato inziale
    [[100],
     [0]]
)
x0_hat = np.array( # vettore di stato inziale ipotizzato
    [[105],
     [0]]
)
P0_hat = np.array([ # matrice di covarianza inziale del vettore di stato ipotizzata
    [10, 0],
    [0, 0.01]]
)
N = 1000 # numero di passi temporali
t = dt * np.arange(0, N + 1) # vettore del tempo (s)
```

```python
from msbd.modello_lineare import ModelloLineareDinamico

print(inspect.getsource(ModelloLineareDinamico))
```

### Esercizio

1. Completare il metodo `eq1` della classe `ModelloLineareDinamico` definita in `msbd/modello_lineare/modello_lineare_dinamico.py`;
2. Completare il metodo `eq2` della classe `ModelloLineareDinamico` definita in `msbd/modello_lineare/modello_lineare_dinamico.py`;
2. Scrivere dei test per verificare che i metodi implementati siano corretti.

## 3.3 Generazione delle serie degli stati e degli output

```python
x = x0
y = np.array([[]])

for k in range(1, N + 1):
    x_k = ModelloLineareDinamico.eq1(F, x[:, [k - 1]], G, u)
    x = np.hstack([x, x_k])

    y_k = ModelloLineareDinamico.eq2(H, x[:, [k]])
    y = np.hstack([y, y_k])
```

## 3.4 Simulazione di misurazioni tramite l'aggiunta di rumore a y

```python
np.random.seed(42)

v = np.random.multivariate_normal(np.zeros(1), cov=R, size=N).reshape(y.shape) # nota: in questo caso bastava np.random.normal

z = y + v
```

### Esercizio

1. Completare il metodo `partial_fit` della classe `ModelloLineareDinamico` definita in `msbd/modello_lineare/modello_lineare_dinamico.py`;
2. Scrivere dei test e verificare che il metodo implementato sia corretto.

```python

```

## 3.5 Filtraggio di Kalman

```python
fk = ModelloLineareDinamico(x0_hat, P0_hat)

for k in range(0, N):
    fk.partial_fit(F, G, u, H, Q, R, z[:, [k]])

h_hat = np.array(fk.x_hat_)[:, 0, 0] # array delle altezze
v_hat = np.array(fk.x_hat_)[:, 1, 0] # array delle velocità
```

## 3.6 Grafico degli stati veri e stimati

```python
plt.figure(figsize=(8, 6))

plt.suptitle("Esempio di filtraggio di Kalman, stati veri e stimati")

plt.subplot(211)
plt.plot(t[1:], z[0, :], color="lime", label="Misurata")
plt.plot(t, h_hat, ls='--', color="blue", lw=3, label="Stimata")
plt.plot(t, x[0, :], ls=':', color="red", lw=2, label="Vera")
plt.ylabel("$x_0 = altezza\, (m)$")
plt.grid(ls='--')
plt.legend()

plt.subplot(212)
plt.plot(t, v_hat, ls='--', color="blue", lw=3, label="Stimata")
plt.plot(t, x[1, :], ls=':', color="red", lw=2, label="Vera")
plt.xlabel("secondi ($s$)")
plt.ylabel("$x_1 = velocità \, (m/s)$")
plt.grid(ls='--')
plt.legend()

plt.show()
```

```python

```


```python
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))
```

# Atlanta Police Department

The Atlanta Police Department provides Part 1 crime data at http://www.atlantapd.org/crimedatadownloads.aspx

A recent copy of the data file is stored in the cluster. Please, do not copy this data file into your home directory!

```python
### Load libraries
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
```

```python
help(plt.legend)
```

Load data (don't change this if you're running the notebook on the cluster)

We have two files
- `/home/data/APD/COBRA083016_2015.xlsx` for 2015
- `/home/data/APD/COBRA083016.xlsx` from 2009 to current date

```python
%%time
df = pd.read_excel('/home/data/APD/COBRA083016_2015.xlsx', sheetname='Query')
```

    CPU times: user 12 s, sys: 21.5 ms, total: 12 s
    Wall time: 12.1 s

```python
df.shape
```

    (30011, 23)

```python
for c in df.columns:
    print(c)
```

```python
df[0:5]
```

```python
df.describe()
```

```python
df.offense_id.min(), df.offense_id.max()
```

```python
df.groupby(['UC2 Literal']).offense_id.count()
```

## Exploring Dates

```python
df[['offense_id', 'occur_date', 'occur_time', 'rpt_date']][1:10]
```

Convert into date-time type

```python
df['occur_ts'] = pd.to_datetime(df.occur_date+' '+df.occur_time)
```

```python
#df[['offense_id', 'occur_date', 'occur_time', 'occur_ts', 'rpt_date']][1:10]
```

```python
df['occur_ts'] = pd.to_datetime(df.occur_date+' '+df.occur_time)
df['occur_month'] = df['occur_ts'].map(lambda x: x.month)
df['occur_woy'] = df.occur_ts.dt.weekofyear
```

```python
df.describe()
```

```python
resdf = df.groupby(['UC2 Literal', 'occur_month']).offense_id.count()
resdf
```

```python
resdf['BURGLARY-RESIDENCE'].as_matrix()
```

```python
resdf['BURGLARY-RESIDENCE'].iloc(0)
```

```python
%matplotlib inline
fig = plt.figure(figsize=(10,6)) # 10inx10in
#plt.plot(resdf['BURGLARY-RESIDENCE'].index, resdf['BURGLARY-RESIDENCE'])
plt.scatter(resdf['BURGLARY-RESIDENCE'].index, resdf['BURGLARY-RESIDENCE'], marker='x')
plt.scatter(resdf['BURGLARY-NONRES'].index, resdf['BURGLARY-NONRES'], marker='o')

plt.ylim(0, 500)
plt.title('BURGLARY-RESIDENCE')
plt.xticks(range(13), ['', 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
fig.savefig('BurglaryResidence_over_month.svg')
x = 1
```

```python
df = pd.read_excel('/home/data/APD/COBRA083016_2015.xlsx', sheetname='Query')
df['occur_ts'] = pd.to_datetime(df.occur_date+' '+df.occur_time)
df['occur_month'] = df['occur_ts'].map(lambda x: x.month)
df['occur_woy'] = df.occur_ts.dt.weekofyear
```

```python
%matplotlib inline
resdf = df.groupby(['UC2 Literal', 'occur_month']).offense_id.count()
fig = plt.figure(figsize=(10,6))
plt.scatter(resdf['BURGLARY-RESIDENCE'].index, resdf['BURGLARY-RESIDENCE'], marker='x')
plt.ylim(0, 500)
plt.title('BURGLARY-RESIDENCE')
plt.xticks(range(13), ['', 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
plt.savefig('quiz3-burglary-residence.png')

''
```

```python
plt.savefig('quiz3-burglary-residence.png')

```

# Part 1 - Observations from the data

```python

```

```python

```

# Part 2 - Seasonal Model

```python
## load complete dataset
dff = pd.read_excel('/home/data/APD/COBRA083016.xlsx', sheetname='Query')
```

```python
dff.shape
```

```python
for evt in ['occur', 'poss']:
    dff['%s_ts'%evt] = pd.to_datetime(dff['%s_date'%evt]+' '+dff['%s_time'%evt])
dff['rpt_ts'] = pd.to_datetime(dff.rpt_date)
```

```python
', '.join(dff.columns)
```

```python
dff['occur_year'] = dff.occur_ts.dt.year
dff['occur_month'] = dff.occur_ts.dt.month
dff['occur_dayweek'] = dff.occur_ts.dt.dayofweek
```

### Crime per year
Let's look at the

```python
crime_year = dff[dff.occur_year.between(2009, 2015)].groupby(by=['UC2 Literal', 'occur_year']).offense_id.count()
```

```python
%matplotlib inline
fig = plt.figure(figsize=(40,30))
crime_types = crime_year.index.levels[0]
years = crime_year.index.levels[1]
for c in range(len(crime_types)):
    y_max = max(crime_year.loc[crime_types[c]])

    plt.subplot(4,3,c+1)
    plt.hlines(crime_year.loc[crime_types[c]].iloc[-1]*100/y_max, years[0], years[-1], linestyles="dashed", color="r")
    plt.bar(crime_year.loc[crime_types[c]].index, crime_year.loc[crime_types[c]]*100/y_max, label=crime_types[c], alpha=0.5)
    ##plt.legend()
    plt.ylim(0, 100)
    plt.xticks(years+0.4, [str(int(y)) for y in years], rotation=0, fontsize=24)
    plt.yticks([0,20,40,60,80,100], ['0%','20%','40%','60%','80%','100%'], fontsize=24)
    plt.title(crime_types[c], fontsize=30)
    None
```

Let's look at residential burglary.

```python
c = 3
crime_types[c]
```

```python
crime_year_month = dff[dff.occur_year.between(2009, 2015)].groupby(by=['UC2 Literal', 'occur_year', 'occur_month']).offense_id.count()
```

```python
c = 3 ## 'BURGLARY-RESIDENCE'
resburglaries = crime_year_month.loc[crime_types[c]]
fig = plt.figure(figsize=(20,10))
for y in years:
    plt.plot(resburglaries.loc[y].index, resburglaries.loc[y], label=("%4.0f"%y))
plt.legend()
plt.title("Seasonal Trends - %s"%crime_types[c], fontsize=20)
plt.xticks(range(13), ['', 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
plt.xlim(0,13)
None
```

Normalized over the annual average

```python
c = 3 ## 'BURGLARY-RESIDENCE'
fig = plt.figure(figsize=(20,10))
for y in years:
    avg = resburglaries.loc[y].mean()
    plt.hlines(avg, 1, 13, linestyle='dashed')
    plt.plot(resburglaries.loc[y].index, resburglaries.loc[y], label=("%4.0f"%y))
plt.legend()
plt.title("Seasonal Trends - %s (with annuale averages)"%crime_types[c], fontsize=20)
plt.xticks(list(range(1,13)), ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
plt.xlim(0,13)
None
```

```python
c = 3 ## 'BURGLARY-RESIDENCE'
fig = plt.figure(figsize=(20,10))
for y in years:
    avg = resburglaries.loc[y].mean()
    std = resburglaries.loc[y].std()
    ##plt.hlines(avg, 1, 13, linestyle='dashed')
    plt.plot(resburglaries.loc[y].index, (resburglaries.loc[y]-avg)/std, label=("%4.0f"%y))
plt.legend()
plt.title("Seasonal Trends - %s (normalized)"%crime_types[c], fontsize=20)
plt.xticks(list(range(1,13)), ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
plt.xlim(0,13)
plt.ylabel("Standard deviations $\sigma_y$")
None
```

```python

```

```python
seasonal_adjust = resburglaries.reset_index().groupby(by=['occur_month']).offense_id.agg('mean')
```

```python

```

```python

```

```python

```

```python

```

```python

```

### Fitting the regression line
Suppose there are $n$ data points {{math|{(''xi'', ''yi''), ''i'' {{=}} 1, ..., ''n''}.}} The function that describes x and y is:

$$y_i = \alpha + \beta x_i + \varepsilon_i.$$

The goal is to find the equation of the straight line

$$y = \alpha + \beta x,$$

which would provide a "best" fit for the data points. Here the "best" will be understood as in the [[Ordinary least squares|least-squares]] approach: a line that minimizes the sum of squared residuals of the linear regression model. In other words, {{mvar|α}} (the {{mvar|y}}-intercept) and {{mvar|β}} (the slope) solve the following minimization problem:

$$\text{Find }\min_{\alpha,\,\beta} Q(\alpha, \beta), \qquad \text{for } Q(\alpha, \beta) = \sum_{i=1}^n\varepsilon_i^{\,2} = \sum_{i=1}^n (y_i - \alpha - \beta x_i)^2\ $$

By using either [[calculus]], the geometry of [[inner product space]]s, or simply expanding to get a quadratic expression in {{mvar|α}} and {{mvar|β}}, it can be shown that the values of {{mvar|α}} and {{mvar|β}} that minimize the objective function {{mvar|Q}}<ref>Kenney, J. F. and Keeping, E. S. (1962) "Linear Regression and Correlation." Ch. 15 in ''Mathematics of Statistics'', Pt. 1, 3rd ed. Princeton, NJ: Van Nostrand, pp. 252–285</ref> are

: <math>\begin{align}
  \hat\beta &= \frac{ \sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y}) }{ \sum_{i=1}^n (x_i - \bar{x})^2 } \\[6pt]
            &= \frac{ \sum_{i=1}^{n} (x_i y_i - x_i \bar{y} - \bar{x} y_i + \bar{x} \bar{y})} { \sum_{i=1}^n (x_i^2 - 2 x_i \bar{x} + \bar{x}^2) } \\[6pt]
            &= \frac{ \sum_{i=1}^{n} (x_i y_i) - \bar{y} \sum_{i=1}^{n}  x_i  - \bar{x} \sum_{i=1}^{n} y_i + n  \bar{x} \bar{y}} { \sum_{i=1}^n (x_i^2) - 2 \bar{x}  \sum_{i=1}^n  x_i  + n \bar{x}^2 } \\[6pt]
            &= \frac{ \frac{1}{n} \sum_{i=1}^{n} x_i y_i - \bar{x} \bar{y} }{ \frac{1}{n}\sum_{i=1}^n {x_i^2} -   \overline{x}^2 } \\[6pt]
            &= \frac{ \overline{xy} - \bar{x}\bar{y} }{ \overline{x^2} - \bar{x}^2 } = \frac{ \operatorname{Cov}[x, y] }{ \operatorname{Var}[x] } \\
            &= r_{xy} \frac{s_y}{s_x}, \\[6pt]
 \hat\alpha & = \bar{y} - \hat\beta\,\bar{x},
\end{align}</math>

where {{math|''rxy''}} is the [[Correlation#Pearson's product-moment coefficient|sample correlation coefficient]] between {{mvar|x}} and {{mvar|y}}; and {{math|''sx''}} and {{math|''sy''}} are the [[sample standard deviation]] of {{mvar|x}} and {{mvar|y}}. A horizontal bar over a quantity indicates the average value of that quantity. For example:

:<math>\overline{xy} = \frac{1}{n} \sum_{i=1}^n x_i y_i.</math>

Substituting the above expressions for <math>\hat{\alpha}</math> and <math>\hat{\beta}</math> into

: <math>f = \hat{\alpha} + \hat{\beta} x,</math>

yields

: <math>\frac{ f - \bar{y}}{s_y} = r_{xy} \frac{ x - \bar{x}}{s_x}  </math>

This shows that {{math|''rxy''}} is the slope of the regression line of the [[Standard score|standardized]] data points (and that this line passes through the origin).

It is sometimes useful to calculate {{math|''rxy''}} from the data independently using this equation:

:<math>r_{xy} = \frac{ \overline{xy} - \bar{x}\bar{y} }{ \sqrt{ \left(\overline{x^2} - \bar{x}^2\right)\left(\overline{y^2} - \bar{y}^2\right)} } </math>

The [[coefficient of determination]] (R squared) is equal to <math>r_{xy}^2</math> when the model is linear with a single independent variable. See [[Correlation#Pearson's product-moment coefficient|sample correlation coefficient]] for additional details.

===Linear regression without the intercept term===
Sometimes it is appropriate to force the regression line to pass through the origin, because {{mvar|x}} and {{mvar|y}} are assumed to be proportional. For the model without the intercept term, {{math|''y'' {{=}} ''βx''}}, the OLS estimator for {{mvar|β}} simplifies to

: <math>\hat{\beta} = \frac{ \sum_{i=1}^n x_i y_i }{ \sum_{i=1}^n x_i^2 } = \frac{\overline{x y}}{\overline{x^2}} </math>

Substituting {{math|(''x'' − ''h'', ''y'' − ''k'')}} in place of {{math|(''x'', ''y'')}} gives the regression through {{math|(''h'', ''k'')}}:

: <math>\begin{align}
  \hat\beta &= \frac{\overline{(x - h) (y - k)}}{\overline{(x - h)^2}} \\[6pt]
            &= \frac{\overline{x y} + k \bar{x} - h \bar{y} - h k }{\overline{x^2} - 2 h \bar{x} + h^2} \\[6pt]
            &= \frac{\overline{x y} - \bar{x} \bar{y} + (\bar{x} - h)(\bar{y} - k)}{\overline{x^2} - \bar{x}^2 + (\bar{x} - h)^2} \\[6pt]
            &= \frac{\operatorname{Cov}[x,y] + (\bar{x} - h)(\bar{y}-k)}{\operatorname{Var}[x] + (\bar{x} - h)^2}
\end{align}</math>

The last form above demonstrates how moving the line away from the center of mass of the data points affects the slope.

```python
### in case we want to save a DataFrame
#writer = pd.ExcelWriter('myresults.xlsx')
#df.to_excel(writer,'Results')
#writer.save()
```

```python
resdf
```

```python

```

```python

```

```python

```

```python

```


# 5. Gyakorlat - 1 DoF csillapított lengő kar
2021.03.08

## Feladat:

```python
from IPython.display import Image
Image(filename='gyak_5_1.png',width=500)
```

A mellékelt ábrán egy lengőkar látható, ami két különböző tömegű és hosszúságú rúdból és a hozzá csatlakozó $R$ sugarú korongból áll. A két rúd két $k_1$, illetve $k_2$ merevségű rugón és egy $c_1$ csillapítási tényezőjű csillapító elemen keresztül csatlakozik a környezetehéz. A lengőkar csak az $A$ csukló körül tud elfordulni. A mozgást leíró általános koordináta a $\varphi$ szög, melyet a vízszintes egyenestől mérünk. A lengőkar a Föld gravitációs terében helyezkedik el, egyensúlyi helyezete a $\varphi=0$ pozícióban van, ahol a $k_2$ merevségű rugó feszítetlen.

### Adatok:
|||
|-------------------|------------------|
| $l$ = 0,2 m       | $k_1$ = 300 N/m  |
| $R$ = 0,1 m       | $k_2$ = 10 N/m   |
| $m$ = 0,12 kg     | $c_1$ = 2 Ns/m   |

### Részfeladatok:

1. Írja fel a mozgásegyenletet és számítsa ki a csillapítatlan ($\omega_n$) illetve csillapított sajátfrekvenciákat ($\omega_d$), és a realtív csillapítási tényezőt ($\zeta$)!
2. Számítsa ki a kritikus csillapítási tényezőt ($c_{cr}$)!
3. Számítsa ki a $k_1$ merevségű rugóhoz tartozó maximális rugóerő értékét a következő kezdeti feltételek esetén: ($\varphi(t=0)=\varphi_0=0,01$[rad];$\dot{\varphi}(t=0)=0$[rad/s])!

## Megoldás:

```python
from IPython.display import Image
Image(filename='gyak_5_2.png',width=500)
```

### 1. Feladat:
A fenti ábrán a lengőkar szabadtest ábrája látható az egyensúgyi helyzetből kitérített helyezetében. Egy korábbi példában (3. gyakorlat) már be lett mutatva, hogy milyen egyszerűsítéséskkel lehet élni annak érdekében, hogy linearizáljunk egy ilyen lengőrendszert. Röviden összefoglalva az egyszerűsítéseket:
 - a vízszintes rúdelemre ható gravitációs erőnek nincs hatása a sajátkörfrekvenciára (viszont hatása lehet a maximális rugóerőkre);
 - a rugók deformációja az egyensúlyi helyzetből mért ívhosszakkal jól közelíthetők.

A mozgásegyenlet Newton II. alapján:

\begin{equation}
    \dot{\mathbf{I}}=\mathbf{F}.
\end{equation}

Ebből az SZTA alapján (és cos$\varphi\approx 1$, illetve a sin$\varphi\approx \varphi$ közelítéseket alkalmazva) adódik:

\begin{equation}
    \Theta_A\ddot{\varphi}=-F_{r,1}3l-F_{r,2}l-F_{cs,1}3l-3mg\frac{3}{2}l-2mgl\varphi-mg(2l+R)\varphi,
\end{equation}

ahol a csillapításból adódó erő és a rugerők

\begin{equation}
    F_{cs,1}\cong c_13l\dot{\varphi}, \hspace{10pt} F_{r,1}\cong F_{r,1st}+k_13l\varphi, \hspace{10pt} F_{r,2}\cong k_2l\varphi
\end{equation}

(megjegyzés: a mozgásegyenletben szerplő $-3mg\frac{3}{2}l$ tag és a $k_1$ merevségű rugó statikus deformációjáből adódó erő ($F_{r,1st}$) kiegyenlítik egymást, így kiesenek az egyenletből). Tehát a mozgásegyenlet:

\begin{equation}
    \Theta_A\ddot{\varphi}=-9l^2k_1\varphi-l^2k_2\varphi-9l^2c_1\dot{\varphi}-2mgl\varphi-mg(2l+R)\varphi.
\end{equation}

A szerkezet tehetetlenségi nyomatékát a Steiner tétel segítségével lehet az $A$ pontra meghatározni:

\begin{equation}
    \Theta_A=\frac{1}{3}3ml(3l)^2+\frac{1}{3}2ml(2l)^2+\frac{1}{2}mR^2+m(2l+R)^2.
\end{equation}

```python
import sympy as sp
from IPython.display import display, Math

sp.init_printing()
```

```python
l, R, m, k1, k2, c1, Θ_A, g = sp.symbols("l, R, m, k1, k2, c1, Θ_A, g", real=True)

# Készítsünk behelyettesítési listát az adatok alapján, SI-ben
adatok = [(l, 0.2), (R, 0.1), (m, 0.12), (k1, 300), (k2, 10), (c1, 2), (g, 9.81)]

# Az általános koordináta definiálása az idő függvényeként
t = sp.symbols("t",real=True, positive=True)
φ_t = sp.Function('φ')(t)

# A z tengelyre számított perdület derivált az A pontban
dΠ_Az = Θ_A*φ_t.diff(t,2)

# A z tengelyre számított nyomaték az A pontban
M_Az = -9*l**2*k1*φ_t-l**2*k2*φ_t-9*l**2*c1*φ_t.diff(t)-2*m*g*l*φ_t-m*g*(2*l+R)*φ_t

# A dinamika alapegyenlete
# (nullára rendezve)

mozgegy = dΠ_Az-M_Az
mozgegy
```

```python
# Osszunk le a főegyütthatóval:
foegyutthato = mozgegy.coeff(sp.diff(φ_t,t,2))

mozgegy = (mozgegy/foegyutthato).expand().apart(φ_t)

mozgegy
```

```python
# Írjuk be a tehetetlenségi nyomatékot
# a rúdhosszakkal és tömegekkel kifejezve
mozgegy = mozgegy.subs(Θ_A, 1/3*3*m*(3*l)**2+1/3*2*m*(2*l)**2+1/2*m*R**2+m*(2*l+R)**2)
```

```python
# A mozgásegyenletnek ebben az alakjában
# d/dt(φ(t)) együtthatója 2ζω_n -nel
# a φ(t) együtthatója pedig ω_n^2-tel egyezik meg,
# tehát mind a három kérdéses paraméter megadható.

ω_n_num = sp.sqrt((mozgegy.coeff(φ_t)).subs(adatok)).evalf(6)

ζ_num = ((mozgegy.coeff(φ_t.diff(t))).subs(adatok)/(2*ω_n_num)).evalf(4)

ω_d_num = (ω_n_num*sp.sqrt(1-ζ_num**2)).evalf(6)

display(Math('\omega_n = {}'.format(sp.latex(ω_n_num))),
        Math('\zeta = {}'.format(sp.latex(ζ_num))),
        Math('\omega_d = {}'.format(sp.latex(ω_d_num))))
# [rad/s]
# [1]
# [rad/s]
```

$\displaystyle \omega_n = 35.5523$

$\displaystyle \zeta = 0.1169$

$\displaystyle \omega_d = 35.3085$

```python
# Később még szükség lesz a csillapított frekvenciára és periódusidőre is:
T_d_num = (2*sp.pi/ω_d_num).evalf(4)

f_d_num = (ω_d_num/(2*sp.pi)).evalf(5)
display(Math('T_d = {}'.format(sp.latex(T_d_num))),
        Math('f_d = {}'.format(sp.latex(f_d_num))))

# [s]
# [1/s]
```

$\displaystyle T_d = 0.178$

$\displaystyle f_d = 5.6195$

## 2. Feladat
Kritikus csillapításról akkor beszélünk, amikor a relatív csillapítási tényező éppen 1.

```python
# A mozgásegyenletnek d/dt(φ(t)) együtthatóját kell vizsgálni
mozgegy.coeff(φ_t.diff(t)).evalf(5)
```

```python
# Ez az együttható pont 2ζω_n -nel egyenlő.
# Az így adódó egyenlet megoldásásval kapjuk
# a kritikus csillapítshoz tartozó c1 értéket:
ζ_cr = 1

# itt még nem helyettesítünk be, hanem csak kifejezzük a kritikus csillapítási együtthatót
c1_cr = sp.solve(mozgegy.coeff(φ_t.diff(t))-2*ζ_cr*ω_n_num,c1)[0]

# most már be lehet helyettesíteni
c1_cr_num = c1_cr.subs(adatok).evalf(5)
display(Math('c_{{1,cr}} = {}'.format(sp.latex(c1_cr_num))))

# [1]
```

$\displaystyle c_{1,cr} = 17.105$

## 3. Feladat
A $k_1$ merevségű rugóban az ébredő erő az alábbi alakban adható meg:

\begin{equation}
    F_{r,1}(t) = F_{r,1st}+k_13l\varphi(t).
\end{equation}

A rugó az egyensúlyi helyzetben feszített állapotban van. A statikus deformáció tehát az egyensúlyi egyenletből határozható meg:

\begin{equation}
    \sum M_A=0:\hspace{20pt}     -F_{r,1st}3l-3mg\frac{3}{2}l=0.
\end{equation}

```python
Fr_1st = sp.symbols("Fr_1st")
Fr_1st_num = (sp.solve(-Fr_1st*3*l-3*m*g*3/2*l,Fr_1st)[0]).subs(adatok)

display(Math('F_{{r,1st}} = {}'.format(sp.latex(Fr_1st_num))))
# [N]
```

$\displaystyle F_{r,1st} = -1.7658$

A dinamikus rugóerőnek ott lesz a maximuma ahol a legnagyobb a kitérés. Első körben tehát a mozgástörvényt kell meghatározni.

```python
kezdeti_ert = {φ_t.subs(t,0): 0.01, φ_t.diff(t).subs(t,0): 0}
display(kezdeti_ert)

mozg_torv = (sp.dsolve(mozgegy.subs(adatok),φ_t,ics=kezdeti_ert)).evalf(6)
mozg_torv
```

Keressük meg a kitérés maximumát numerikus módszerek segítéségvel. Ehhez először célszerű kirajzoltatni a függvényünket. (Analatikus megoldáshoz lásd 4. gyakorlat hasoló példa megoldása!)

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
# A matplotlib plottere az általunk megadott pontokat fogja egyenes vonalakkal összekötni.
# Elegendően kis lépésközt választva az így kapott görbe simának fog tűnni.
# Állítsuk elő az (x,y) koordinátákat!

t_val = np.linspace(0,0.5,1001) # lista létrehozása a [0 ; 0,5] intervallum 1001 részre való bontásával
φ_val = np.zeros(len(t_val)) # nulla lista létrehozása (ugyanannyi elemszámmal)
# for ciklus segítségével írjuk felül a nulla listában szerplő elemelet az adott x értékhez tartozó y értékekkel
for i in range(len(t_val)):
    φ_val[i] = mozg_torv.rhs.subs(t,t_val[i])  # Ezt lehetne list comprehension-nel is.

# rajzterület létrehozása
plt.figure(figsize=(40/2.54,30/2.54))

# függvény kirajzolása az x és y kordináta értékeket tartalmazó listák megadásásval
plt.plot(t_val,φ_val,color='b',label=r'num_sim')

# tengelyek
axes = plt.gca()
axes.set_xlim([0,t_val[-1]])
axes.set_ylim([-0.01, 0.01])

# rácsozás
plt.grid()

# tengely feliratozás
plt.xlabel(r'$ t [s] $',fontsize=30)
plt.ylabel(r'$ \varphi(t) [rad] $',fontsize=30)

plt.show()
```

A statikus rugóerő negatív előjelű, ezért két szélsőérték helyet is meg kell vizsgálni: az első lokális maximumot és a minimumot. Ezeket az értékekeket könnyen ki tudjuk szedni a korábban meghatározott listából.

```python
lok_max = max(φ_val)
lok_min = min(φ_val)

# Rugóerők meghatározása
Fr_11 = (Fr_1st_num+k1*3*l*φ_t).subs(adatok).subs(φ_t,lok_max).evalf(5)
Fr_12 = (Fr_1st_num+k1*3*l*φ_t).subs(adatok).subs(φ_t,lok_min).evalf(5)

display(Math('F_{{r,1}} = {}'.format(sp.latex(Fr_11))),Math('F_{{r,2}} = {}'.format(sp.latex(Fr_12))))

# [N]
```

$\displaystyle F_{r,1} = 0.0342$

$\displaystyle F_{r,2} = -3.0093$

A 1-es rugóban ébredő maximális erő tehát 3,009 N.

Készítette:

            Juhos-Kiss Álmos (Alkalmazott Mechanika Szakosztály)
            Takács Dénes (BME MM) ábrái és Vörös Illés (BME MM) kidolgozása alapján.

            Hibák, javaslatok:
            amsz.bme@gmail.com
            csuzdi02@gmail.com
            almosjuhoskiss@gmail.com

            2021.03.08


[<-Back to the algorithms_in_ipython_notebooks repository](https://github.com/rasbt/algorithms_in_ipython_notebooks).

```
%load_ext watermark
```

```
watermark -d -v -a "Sebastian Raschka"
```

    Sebastian Raschka 09/09/2014

    CPython 3.4.1
    IPython 2.2.0

[More information](https://github.com/rasbt/watermark) about the `watermark` magic command extension.

I would be happy to hear your comments and suggestions.
Please feel free to drop me a note via
[twitter](https://twitter.com/rasbt), [email](mailto:bluewoodtree@gmail.com), or [google+](https://plus.google.com/+SebastianRaschka).

# Rejection Sampling

While I was listening to the recent episode of the [Programming Throwdown](http://www.programmingthrowdown.com/2014/09/episode-36-swift.html) podcast on my way home, I heard about this interesting concept of rejection sampling. This is one of those simple ideas combined with the powerful principles of statistics that I find quite fascinating.

At its core, rejection sampling is similar to the popular Monte Carlo sampling with the difference of an additional bound.
The goal of rejection sampling is to simplify the task of drawing random samples from a complex probability distribution by using a uniform distribution instead; random samples drawn from the uniform distribution that lie outside certain boundary criteria are rejected, and all samples within the boundary are accepted, respectively.

A mathematical proof can be found [here](http://www.programmingthrowdown.com/2014/09/episode-36-swift.html).

Let's use a simple example to illustrate this concept: Our task is to draw random samples from a geometrically-bounded distribution in form a circle in a cartesian coordinate system with a radius of 2 centered at the coordinates x=4 and y=4.

```
%matplotlib inline
import matplotlib.pyplot as plt
```

```
center = (4,4)
radius = 2

def plot_circle(center, radius):
    """ Function to plot a circle. """
    fig = plt.figure(figsize=(6,6))
    circle = plt.Circle(center, radius, fill=False, color='b')
    plt.ylim([0,8])
    plt.xlim([0,8])
    fgca = fig.gca()
    fgca.add_artist(circle)
    return fgca

plot_circle(center, radius)
plt.show()
```

Now, we can draw a simple square around the circle that will represent our uniform distribution.

```
from matplotlib.patches import Rectangle

def plot_square(center, radius):
    """ Function to plot a square. """
    fgca = plot_circle(center, radius)
    fgca.add_patch(Rectangle((center[0] - radius, center[1] - radius),
                             2*radius, 2*radius, alpha=0.1))
    return fgca

plot_square(center, radius)
plt.show()
```

Next, we will define a function that generates pseudo-random X and Y coordinates that fall inside the square.

```
import random
random.seed(567)
```

```
def gen_points(n, center, radius):
    """
    Function that generates
    n x,y coordinates in a square of length radius*2.

    """
    x_coords = []
    y_coords = []
    for i in range(n):
        x_coords.append(random.random()*center[0]+radius)
        y_coords.append(random.random()*center[1]+radius)
    return x_coords, y_coords
```

```
x, y = gen_points(1, center, radius)
fgca = plot_square(center, radius)
fgca.plot(x, y, linestyle="", marker="x", color="red")
plt.show()
```

Let us generate 1000 random points and check if our function works correctly.

```
x, y = gen_points(1000, center, radius)
fgca = plot_square(center, radius)
fgca.plot(x, y, linestyle="", marker="x", color="red")
plt.show()
```

The plot above looks fine. In the last step, we only need to reject those points lie outside the circle, which is pretty straight forward using a Euclidean distance measure, that is

\begin{equation} \sqrt{(x-y)^2} = |x-y|. \end{equation}

```
def reject(radius, center, x_coords, y_coords):
    """ Returns those coordinates that fall within the circle. """
    x_clean = []
    y_clean = []
    for x,y in zip(x_coords, y_coords):
        if ((x - center[0])**2 + (y-center[1])**2)**0.5 <= radius:
            x_clean.append(x)
            y_clean.append(y)
    return x_clean, y_clean
```

Again, let us do a quick visual check if we correctly removed all points that didn't satisfy the condition $|x-y| \le \text{radius}$.

```
x_clean, y_clean = reject(radius, center, x, y)
fgca = plot_square(center, radius)
fgca.plot(x_clean, y_clean, linestyle="", marker="x", color="red")
plt.show()
```

Okay, it seems that we successfully wrote some code that can generate pseudo-random samples that fall inside a geometric circle. If this isn't exciting enough, let us use this concept to estimate the area of this circle pretending that we don't know π (pi).

\begin{equation} A_{\text{rectangle}} = (2 \times R)^2 \end{equation}

\begin{equation} \hat{A}_{\text{est_circle}} = A_{\text{rectangle}} \times \frac{\text{# points inside circle}}{\text{# all points}} \end{equation}

```
def estimate_circle_area(n, center, radius):
    """ Returns the estimated circle area via rejection sampling. """
    rect_area = (2*radius)**2
    x, y = gen_points(n, center, radius)
    x_clean, y_clean = reject(radius, center, x, y)
    est_circle_area = rect_area * (len(x_clean)/len(x))

    return est_circle_area
```

```
print('Estimated circle area: %s' %estimate_circle_area(100000, center, radius))
```

    Estimated circle area: 12.56224

Now, let's double-check how close we got using the more accurate equation:

\begin{equation} A_{\text{circle}} = \pi \times R^2 \end{equation}

```
from math import pi

print('Circle area using pi: %s' %(pi*radius**2))
```

    Circle area using pi: 12.566370614359172

Who would have guessed what comes next: Let us use our rejection sampling method to estimate pi itself:

\begin{equation} \hat{\pi} = \frac{\hat{A}_{\text{est_circle}}}{R^2}  \end{equation}

```
def approximate_pi(n, center, radius):
    """ Returns an approximation of pi via rejection sampling. """
    circ_area = estimate_circle_area(n, center, radius)
    return circ_area/radius**2

for i in (10, 10**2, 10**4, 10**7):
    pi_est = approximate_pi(i, center, radius)
    print('Pi estimate: %s (n=%s)' %(pi_est, i))
```

    Pi estimate: 1.6 (n=10)
    Pi estimate: 3.28 (n=100)
    Pi estimate: 3.1496 (n=10000)
    Pi estimate: 3.1412652 (n=10000000)

Again, this estimate is pretty accurate up to the 4th digit after the decimal point.