## Notebook Synopsis:

Here I develop a set of models similar to that of Step-1-, here substituting the single trend component for a piecewise trend sub-model. Specifically I:

* Load the training data generated and saved in previous NB.
* Develop and combine piecewise trend, seasonal, and residual noise submodels similar to  previous NB.
* Compare models using WAIC or PSIS-LOOCV.
* Retain and save models predicted to perform better.

```python
import pickle
import pathlib
from platform import python_version as pyver

import pandas as pd
import numpy as np
import pymc3 as pm
import theano.tensor as tt

from sklearn.preprocessing import MinMaxScaler

import arviz as ar
import matplotlib.pyplot as pl
import matplotlib.dates as mdates
from matplotlib import rcParams
```

```python
def print_ver(pkg, name=None):
    try:
        print(f'{pkg.__name__}: {pkg.__version__}')
    except AttributeError:
        print(f'{name}: {pkg}')
```

```python
print_ver(pyver(), 'python')
for pi in [np, pd, pm, ar]:
    print_ver(pi)
```

    python: 3.7.3
    numpy: 1.16.4
    pandas: 0.24.2
    pymc3: 3.7
    arviz: 0.4.1

```python
%matplotlib inline

years = mdates.YearLocator(day=1)
months = mdates.MonthLocator(bymonthday=1)
rcParams['xtick.major.size'] = 8
rcParams['xtick.minor.size'] = 4
rcParams['xtick.minor.visible'] = True
rcParams['xtick.labelsize'] = 16
rcParams['ytick.labelsize'] = 16
rcParams['axes.labelsize'] = 16
rcParams['axes.titlesize'] = 18
rcParams['axes.formatter.limits'] = (-3, 2)
```

```python
with open('../../pickleJar/datadict.pkl', 'rb') as fb:
    datadict = pickle.load(fb)
df = pd.DataFrame(datadict['frame'],)
df['aostz_scaled'] = datadict['y_s']
minmax_t = MinMaxScaler()
df['t_scaled'] = minmax_t.fit_transform(datadict['x'][:,None]).squeeze()
del datadict
```

```python
df.head()
```

      aostz
      aostz_scaled
      t_scaled

      date

      1997-09-01
      0.35295
      1.402588
      0.000000

      1997-10-01
      0.33579
      1.007622
      0.004049

      1997-11-01
      0.31870
      0.614268
      0.008097

      1997-12-01
      0.27345
      -0.427236
      0.012146

      1998-01-01
      0.25799
      -0.783073
      0.016194

### Modeling a Piecewise Trend:

Within the context of Generalized Additive Models(GAMs), which arise from the simple additive combination of submodels, I develop here a set of models following $$y(t) = g(t) + s(t) + ar1(t)$$
where \\(y(t)\\) is the modeled signal (chlorophyll in the AOSTZ sector), \\(g(t) \\) is the trend (i.e *rate of change*) sub-model, \\(s(t)\\) is the seasonal sub-model, \\(ar1(t)\\) is the AR1 residual.

The piecewise model is implemented by inserting a fixed number of changepoints such that $$g(t) = (k + a(t)^T\delta)t + (m + a(t)^T\gamma)$$
where \\(k\\) is the base trend, modified by preset changpoints stored in a vector \\(s\\). At each unique changepoint \\(s_j\\) the trend is adjusted by \\(\delta_j\\), stored in a vector \\(\delta\\), everytime \\(t\\) surpasses a changepoint \\(s_j\\). Used for this purpose, \\(a(t)\\) is basically a vectorized switchboard that turns on for a given switchpoint such that
\begin{equation}
a(t) =
    \begin{cases} 1 , & \text{if $t \geq s_j$} \\
0 , & \text{otherwise}
\end{cases}
\end{equation}
The second part, \\( m + a(t)^T\gamma\\) ensures the segments defined by the switchpoints are connected. Here, \\(m\\) is an offset parameter, and \\(\gamma_j\\) is set to \\(-s_j\delta_j\\).

The issue though is to find the right number of preset changepoint that will capture actual changepoints while not bogging down the inference. Moreover, for the sake of practicality, these will need to be regularly spaced. Here I try several setups including, one change point at the beginning of the year, and one for every season (4pts/year), one every two months (6pts/year), and one for every month (as many changepoints as data points). The idea is then to put a rather restrictive  Laplace prior on \\(\delta\\) to rule out unlikely changepoints, effectively setting the corresponding \\(\delta_j\\) to 0.

First is to define some [helper functions as in the previous notebook](https://tinyurl.com/y3nubuqu#helpers):

```python
def fourier_series(t, p=12, n=1):
    """
    input:
    ------
    t [numpy array]: vector of time index
    p [int]: period
    n [int]: number of fourier components

    output:
    -------
    sinusoids [numpy array]: 2D array of cosines and sines
    """
    p = p / t.size
    wls = 2 * π * np.arange(1, n+1) / p
    x_ = wls * t[:, None]
    sinusoids = np.concatenate((np.cos(x_), np.sin(x_)), axis=1)
    return sinusoids

def seasonality(mdl, n_fourier, t):
    """
    m [pymc3 Model class]: model object
    n_fourier [int]: number of fourier components
    t [numpy array]: vector of time index
    """
    with mdl:
        σ = pm.Exponential('σ', 1)
        f_coefs = pm.Normal('fourier_coefs', 0, sd=1, shape=(n_fourier*2))
        season = pm.Deterministic('season',
                                  tt.dot(fourier_series(t, n=n_fourier), f_coefs)
                                 )
    return season

def piecewise_trend(mdl, s, t, a_t, obs,
                    k_prior_scale=5, δ_prior_scale=0.05, m_prior_scale=5):
    """
    input:
    ------

    mdl [pymc3 Model class]: model object
    s [numpy array]: changepoint vector
    t [numpy array]: time vector
    obs [numpy array]: vector of observations
    a_t [numpy int array]: 2D (t*s) adjustment indicator array
    k_prior_scale [float]: base trend normal prior scale parameter (default=5)
    δ_prior_scale [float]: trend adjustment laplace prior scale param. (default=0.05)
    m_prior_scale [float]: base offset normal prior scale param. (default=5)
    """

    with mdl:
        # Priors:
        k = pm.Normal('k', 0, k_prior_scale) # base trend prior
        if δ_prior_scale is None:
            δ_prior_scale = pm.Exponential('τ', 1.5)
        δ = pm.Laplace('δ', 0, δ_prior_scale, shape=s.size) # rate of change prior
        m  = pm.Normal('m', 0, m_prior_scale) # offset prior
        γ = -s * δ
        trend = pm.Deterministic('trend',
                                 (k + tt.dot(a_t, δ)) * t + (m + tt.dot(a_t, γ)))
    return trend

def ar1_residual(mdl, n_obs):
    with mdl:
        k_ = pm.Uniform('k', -1.1, 1.1)
        tau_ = pm.Gamma('tau', 10, 3)
        ar1 = pm.AR1('ar1', k=k_, tau_e=tau_, shape=n_obs)
    return ar1

def changepoint_setup(t, n_changepoints, s_start=None, s=None, changepoint_range=1):
    """
    input:
    ------
    t [numpy array]: time vector
    n_changepoints [int]: number of changepoints to consider
    s [numpy array]: user-specified changepoint vector (default=None)
    s_start [int]: changepoint start index (default=0)
    changepoint_range[int]: adjustable time proportion (default=1)

    output:
    -------
    s [numpy array]: changepoint vector
    a_t [numpy int array]: 2D (t*s) adjustment indicator array
    """
    if s is None:
        if s_start is None:
            s = np.linspace(start=0, stop=changepoint_range*t.max(),
                            num=n_changepoints+1)[1:]
        else:
            s = np.linspace(start=s_start, stop=changepoint_range*t.max(),
                            num=n_changepoints)
    a_t = (t[:,None] > s) * 1
    return a_t, s

def model_runner(t_, obs_s, add_trend=False, add_season=False, add_AR1=False,
                **payload):
    mdl = pm.Model()
    a_t, s = None, None
    with mdl:
        y_ = 0
        σ = pm.HalfCauchy('σ', 2.5)
        if add_trend:
            n_switches = payload.pop('n_switches', t_.size)
            s_start = payload.pop('s_start', None)
            s = payload.pop('s', None)
            chg_pt_rng = payload.pop('changepoint_range', 1)
            k_prior_scale = payload.pop('k_prior_scale', 5)
            δ_prior_scale = payload.pop('δ_prior_scale', 0.05)
            m_prior_scale = payload.pop('m_prior_scale', 5)
            a_t, s = changepoint_setup(t_, n_switches, s_start=s_start, s=s,
                                       changepoint_range=chg_pt_rng)
            trend_ = piecewise_trend(mdl, s, t_, a_t, obs_s,
                                     k_prior_scale, δ_prior_scale, m_prior_scale)
            y_ += trend_
        if add_season:
            n_fourier = payload.pop('n_fourier', 4)
            season = seasonality(mdl, n_fourier=n_fourier, t=t_)
            y_ += season

        if add_AR1:
            ar1 = ar1_residual(mdl, obs_s.size)
            y_ += ar1
        pm.Normal('obs', mu=y_, sd=σ, observed=obs_s)
    return mdl, a_t, s

def sanity_check(m, df):
    """
    :param m: (pm.Model)
    :param df: (pd.DataFrame)
    """
    # Sample from the prior and check of the model is well defined.
    y = pm.sample_prior_predictive(model=m, vars=['obs'])['obs']
    pl.figure(figsize=(16, 6))
    pl.plot(y.mean(0), label='mean prior')
    pl.fill_between(np.arange(y.shape[1]), -y.std(0), y.std(0), alpha=0.25, label='standard deviation')
    pl.plot(df['y_scaled'], label='true value')
    pl.legend()

def plot_component(axi, x, y, hpd_=None, obs=None, line_label=None, y_axis_label=None,
                  ax_title=None):

    if isinstance(obs, np.ndarray):
        axi.plot(x, obs, color='k', label='observations')
    axi.plot(x, y, color='darkblue', label=line_label)
    if isinstance(hpd_, np.ndarray):
        axi.fill_between(x, hpd_[:, 0], hpd_[:, 1], color='steelblue',
                         alpha=0.5, label='95% CI')
    if y_axis_label:
        axi.set_ylabel(y_axis_label)
    axi.legend()
    if ax_title:
        axi.set_title(ax_title)
    axi.xaxis_date()
    axi.xaxis.set_major_locator(years)
    axi.xaxis.set_minor_locator(months)
    axi.tick_params(axis='x', labelrotation=30)
    axi.grid()
```

```python
mdl_trend_only, A, s_pts = model_runner(df.t_scaled, df.aostz_scaled,
                                        add_trend=True, n_switches=df.shape[0],)
```

```python
render = pm.model_to_graphviz(mdl_trend_only)
render.render('piecewise_trend_only', directory='../../figjar/', format='png')
```

    '../../figjar/piecewise_trend_only.png'

```python

```

```python
y = pm.sample_prior_predictive(model=mdl_trend_only, vars=['obs'])['obs']
pl.figure(figsize=(16, 6))
pl.plot(y.mean(0), label='mean prior')
pl.fill_between(np.arange(y.shape[1]), -y.std(0), y.std(0), alpha=0.25,
                label='standard deviation')
pl.plot(df.aostz_scaled.values, marker='.', label='true value', )
pl.hlines(0, 0,  df.shape[0], linestyles='--', label='expected trend')
pl.legend();
```

```python
with mdl_trend_only:
    trace_trend_only = pm.sample(2000, tune=2000)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [m, δ, k, σ]
    Sampling 4 chains: 100%|██████████| 16000/16000 [21:50<00:00, 12.21draws/s]

```python
trend_only_inference = ar.from_pymc3(trace=trace_trend_only,
                                     prior=pm.sample_prior_predictive(model=mdl_trend_only),
                                     posterior_predictive=pm.sample_posterior_predictive(trace_trend_only,
                                                                                model=mdl_trend_only
                                                                                )
                                    )
```

    100%|██████████| 8000/8000 [00:10<00:00, 774.34it/s]

```python
trend_only_inference.to_netcdf('../../pickleJar/model_results_nc/piecewise_trend_only_inference.nc')
```

    '../../pickleJar/model_results_nc/piecewise_trend_only_inference.nc'

```python
ppc_obs = pm.sample_posterior_predictive(trace_trend_only, model=mdl_trend_only)['obs']
```

    100%|██████████| 8000/8000 [00:10<00:00, 788.73it/s]

```python
f, ax = pl.subplots(nrows=1, sharex=True, figsize=(12, 4))

ylbl = 'standardized chl'
plot_component(ax, df.index, obs.mean(axis=0), hpd_= pm.hpd(obs),
               obs=df.aostz_scaled.values, line_label='model_mean', y_axis_label=ylbl,
               ax_title='All Components')
#plot_component(ax[1], d_aostz.index, ts_m4_f4_trend_mu, hpd_=ts_m4_f4_trend_hpd,
#               line_label='mean', y_axis_label=ylbl, ax_title='Trend')
#plot_component(ax[2], d_aostz.index, ts_m4_f4_season_mu, hpd_=ts_m4_f4_season_hpd,
#               line_label='mean', y_axis_label=ylbl,
#              ax_title='Season')
#plot_component(ax[3], d_aostz.index, ts_m4_f4_ar1_mu, hpd_=ts_m4_f4_ar1_hpd,
#              line_label='mean', y_axis_label=ylbl, ax_title='AR1')
ax.axhline(label='trend=0', color='r', ls=':');
ax.legend();
```

```python

```


# 1. Gyakorlat - Ütközések
2021.03.31

## Feladat:

A mellékelt ábrán egy 1 szabadságfokú rezgő rendszer látható, ami az $m_1$ tömegű golyóból és az $m_{\textrm{2}}$ tömegű rúdból áll. A rúd az A pontban található csuklópont körül el tud fordulni. A rendszer gravitációs erőtérben van, a nehézségi gyorsulás vektora függőlegesen lefelé mutat. A torziós rugó előfeszítése olyan, hogy a rúd egyensúlyi helyzete a vízszintes pozíció. A nyugalomban lévő rúd rezgését az $m_1$ tömegű golyóval való ütközés idézi elő. Az ütközés előtt a golyó $h$ magasságból szabadon esik nulla kezdeti sebesség mellett.

### Adatok:

|                       |                       |
|:-----------------------|:-----------------------|
| $m_1$ = 6 kg | $m_2$ = 6 kg         |
| $a$ = 0,3 m | $\beta$ = 60 ° |
| $h$ = 0,115 m | $e$ = 1|

### Részfeladatok:
1. Határozza meg a golyó sebességét és a rúd szögsebességét az ütközés utáni pillanatban!

## Megoldás:

### 1. Feladat

```python
import sympy as sp
import numpy as np
from IPython.display import Math #szükséges könyvtárak importálása

sp.init_printing() #szép kiiratás

h, m_1, m_2, e, β, a, g = sp.symbols("h, m_1, m_2, e, β, a, g") #használt szimbólumok definiálása

# Készítsünk behelyettesítési listát az adatok alapján, SI-ben
adatok = [(h,0.115), (m_1, 6), (m_2,6), (e,1), (β, sp.pi/3), (a,0.3), (g,9.81)]
```

A leejtett $m_1$ tömegű test sebességének meghatározása az ütközés pillanata előtt, a munkatétel szerint:

$$T_1 - T_0 = W_{01},$$

ahol $T_0$ a kezdeti kinetikus energia, $T_1$ pedig az ütközés pillanatában a kinetikus energia, $W_{01}$ a mechanikai munka, amit a gravitációs erőtér végez. Mivel az $m_1$ tömegű testet 0 kezdősebességgel ejtettük le: $T_0 = 0.$

$$\dfrac{1}{2} m_1 c_1^2 = m_1 g h \longrightarrow c_1 = \sqrt{2 g h}.$$

```python
c_1 = sp.sqrt(2*g*h)                               #1
display(Math('c_1 = {}'.format(sp.latex(c_1))))    #2

c_1_num = c_1.subs(adatok).evalf(2)                #3 `.subs(adatok)`: behelyettesítés, `.evalf(2)`: kiértékelés 2 tizedesjegyre
display(Math('c_1 = {}\ \\text{{m/s}}'.format(c_1_num)))  #4

c_1 = sp.symbols("c_1")                            #5
adatok.append((c_1,c_1_num))                       #6

# Megjegyzés:
# Az első kódsorban található a szimbólikus számítás, a másodikban a szimbólikus megoldás kiiratása történik
# A harmadik kódsorban a numerikus megoldás kiszámítása, a negyedikben a numerikus megoldás kiiratása történik
# Az ötödik kódsorban definiálunk egy szimbólumot a kiszámolt értékhez, a hatodik kódsorban a szimbólumot,
# és annak numerikus értékét hozzáadjuk a behelyettesítési listához, hogy a késöbbiekben tudjunk vele számolni.

# Ezen túl minden cellában hasonló metódus szerint történik a számítás
```

$\displaystyle c_1 = \sqrt{2} \sqrt{g h}$

$\displaystyle c_1 = 1.5\ \text{m/s}$

Az ütközés $m_1$ szempontjából centrikus ütközés, $m_2$ szempontjából álló tengely körül elforduló test ütközése, ezért meg kell keresni az ütközési talppontot, melyet a következőképpen tehetünk meg:
1. Kijelöljük az ütközési normálist.
2. "$A$" pontból (elforduló tengely) merőlegest állítunk $\textbf{n}$ ütközési normálisra

Szükség van a talppont és az álló tengely közötti távolságra, amely:

$$l = \overline{AT} = a\ \textrm{cos}(\beta).$$

```python
l = a*sp.cos(β)
display(Math('l = {}'.format(sp.latex(l))))

l_num = l.subs(adatok).evalf(2) #m-ben
display(Math('l = {}\ \\text{{m}}'.format(l_num)))

l = sp.symbols("l")
adatok.append((l,l_num))
```

$\displaystyle l = a \cos{\left(β \right)}$

$\displaystyle l = 0.15\ \text{m}$

```python

```

A redukált tömeg kiszámításához szükség van a tehetetlenségi nyomatékra, amely az $A$ pontra számolva:

$$\theta_{\textrm{A}} = \theta_{\textrm{S}} + \theta_{\textrm{AS}} = \dfrac{1}{12} m_2 (3a)^2 + m_2 \left(\dfrac{3}{2} a - a\right)^2.$$

```python
θ_A = sp.Rational(1,12)*m_2*(3*a)**2 + m_2*(sp.Rational(3,2)*a - a)**2
display(Math('\\theta_A = {}'.format(sp.latex(θ_A)))) # \theta_A nem elég, kettő \\ kell, mivel '\t'-t a Python külön értelmezi
                                                      # + egyből egyszerűsíti a fent látható kifejezést

θ_A_num = θ_A.subs(adatok).evalf(2)
display(Math('\\theta_A = {}\ \\text{{kgm$^2$}}'.format(θ_A_num)))

θ_A = sp.symbols("θ_A")
adatok.append((θ_A,θ_A_num))
```

$\displaystyle \theta_A = a^{2} m_{2}$

$\displaystyle \theta_A = 0.54\ \text{kgm$^2$}$

Így a redukált tömeg számítható:

$$m_{\textrm{T}} = \dfrac{\theta_{\textrm{A}}}{l^2}.$$

```python
m_T = θ_A / (l**2)
display(Math('m_T = {}'.format(sp.latex(m_T))))

m_T_num = m_T.subs(adatok).evalf(2)
display(Math('m_T = {}\ \\text{{kg}}'.format(m_T_num)))

m_T = sp.symbols("m_T")
adatok.append((m_T,m_T_num))
```

$\displaystyle m_T = \frac{θ_{A}}{l^{2}}$

$\displaystyle m_T = 24\ \text{kg}$

A golyó mozgásállapota az ütközés előtt: $[\mathbf{\Omega}_1; \mathbf{c}_1] = [\mathbf{0};-1,5\mathbf{j}\ $[m/s]$ ]$, ahol $\mathbf{j}$ jelöli az y irányú egységvektort.

A rúd mozgásállapota az ütközés előtt: $[\mathbf{\Omega}_2; \mathbf{c}_{\textrm{Tn}}] = [\mathbf{0};\mathbf{0}]$, mivel a rúd az ütközés pillanata előtt nyugalomban van.

```python
c_Tn = sp.symbols("c_Tn") # a rúd az ütközés előtt nyugalomban van, ezért c_Tn = 0
adatok.append((c_Tn,0))
```

A $\mathbf{c}_1$ sebesség vektor $\textbf{n}$ ütközési normális irányú, és az arra merőleges tangenciális irányú komponense a következőképpen számolható az ábra alapján:

$$c_{1\textrm{n}} = c_1\ \textrm{cos}(\beta),$$

$$c_{1\textrm{t}} = c_1\ \textrm{sin}(\beta).$$

```python
c_1n = c_1 * sp.cos(β)
display(Math('c_{{1n}} = {}'.format(sp.latex(c_1n))))

c_1n_num = c_1n.subs(adatok).evalf(2)
display(Math('c_{{1n}} = {} \ \\text{{m/s}}'.format(c_1n_num)))

c_1n = sp.symbols("c_1n")
adatok.append((c_1n,c_1n_num))
```

$\displaystyle c_{1n} = c_{1} \cos{\left(β \right)}$

$\displaystyle c_{1n} = 0.75 \ \text{m/s}$

```python
c_1t = c_1 * sp.sin(β)
display(Math('c_{{1t}} = {}'.format(sp.latex(c_1t))))

c_1t_num = c_1t.subs(adatok).evalf(2)
display(Math('c_{{1t}} = {}\ \\text{{m/s}}'.format(c_1t_num)))

c_1t = sp.symbols("c_1t")
adatok.append((c_1t,c_1t_num))
```

$\displaystyle c_{1t} = c_{1} \sin{\left(β \right)}$

$\displaystyle c_{1t} = 1.3\ \text{m/s}$

Ezzel a feladat az $m_1$ és $m_{\textrm{T}}$ tömegű testek centrikus ütközéseként modellezhető, melyeknek ütközés előtti sebességeinek normális komponensei $c_{1\textrm{n}}$ és $c_{\textrm{Tn}}$. Ez a probléma Maxwell diagram alkalmazásával grafikusan is megoldható.

Ehhez először kiszámítjuk a közös súlypont sebességét, ami az ütközés során változatlan marad.

```python
c_Sn = (m_1*c_1n + m_T*c_Tn)/(m_1 + m_T)
display(Math('c_{{Sn}} = {}'.format(sp.latex(c_Sn))))

c_Sn_num = c_Sn.subs(adatok).evalf(2)
display(Math('c_{{Sn}} = {}\ \\text{{m/s}}'.format(sp.latex(c_Sn_num))))

c_Sn = sp.symbols("c_Sn")
adatok.append((c_Sn,c_Sn_num))
```

$\displaystyle c_{Sn} = \frac{c_{1n} m_{1} + c_{Tn} m_{T}}{m_{1} + m_{T}}$

$\displaystyle c_{Sn} = 0.15\ \text{m/s}$

A Maxwell diagram megszerkesztésének folyamatát mutatja a fenti ábra. Először (a) az $\textrm{O}_1$ és $\textrm{O}_{\textrm{T}}$ pontokban párhuzamos egyeneseket húzunk az ütközési normálissal, valamint berajzoljuk a testek normális irányú, ütközés előtti sebességeit. Mivel a rúd az ütközés előtt nyugalomban volt, így annak ütközés előtti sebessége zérus. Ezt követően összekötjük a sebességvektorok végpontjait, majd megállapítjuk a közös súlypont helyét a golyó tömegének és a rúd redukált tömegének segítségével az $\frac{\overline{\textrm{SM}_{\textrm{T}}}}{\overline{\textrm{SM}_{\textrm{1}}}} = \frac{m_1}{m_{\textrm{T}}}$ arány felhasználásával. A közös súlypontnak a normális irányú egyenesekre eső merőleges vetületei alapján megállapíthatjuk a $\Delta_1$ és $\Delta_{\textrm{T}}$ 'távolságokat' (b), amik ahhoz a sebességváltozáshoz tartoznak, amikor az ütközésben részt vevő testek elérik a közös súlypont sebességét. Az $e$ ütközési tényező felhasználásával megállapíthatjuk az ütközés utáni ütközési normális irányú sebességeket, mégpedig $e\Delta_1$ és $e\Delta_{\textrm{T}}$-t mérve a $\textrm{P}_1\textrm{P}_{\textrm{T}}$ egyenes túloldalára (c). A golyó ütközés utáni normális irányú sebességét az $\overline{\textrm{O}_1\textrm{Q}_1}$, a rúdét az $\overline{\textrm{O}_{\textrm{T}}\textrm{Q}_{\textrm{T}}}$ adja.

Ezek alapján az ütközés utáni normális irányú sebességkomponensek

$$v_{1\textrm{n}} = c_{\textrm{Sn}} - e\Delta_1 = c_{\textrm{Sn}} - e(c_{\textrm{1n}} - c_{\textrm{Sn}}),$$

$$v_{\textrm{Tn}} = c_{\textrm{Sn}} - e\Delta_{\textrm{T}} = (1+e)c_{\textrm{Sn}}, $$

```python
v_1n = c_Sn - e*(c_1n - c_Sn)
display(Math('v_{{1n}} = {}'.format(sp.latex(v_1n))))

v_1n_num = v_1n.subs(adatok).evalf(2)
display(Math('v_{{1n}} = {}\ \\text{{m/s}}'.format(sp.latex(v_1n_num))))

v_1n = sp.symbols("v_1n")
adatok.append((v_1n,v_1n_num))

v_Tn = (1 + e)*c_Sn
display(Math('v_{{Tn}} = {}'.format(sp.latex(v_Tn))))

v_Tn_num = v_Tn.subs(adatok).evalf(2)
display(Math('v_{{Tn}} = {}\ \\text{{m/s}}'.format(sp.latex(v_Tn_num))))

v_Tn = sp.symbols("v_Tn")
adatok.append((v_Tn,v_Tn_num))
```

$\displaystyle v_{1n} = c_{Sn} - e \left(c_{1n} - c_{Sn}\right)$

$\displaystyle v_{1n} = -0.45\ \text{m/s}$

$\displaystyle v_{Tn} = c_{Sn} \left(e + 1\right)$

$\displaystyle v_{Tn} = 0.3\ \text{m/s}$

Így, a golyó ütközés utáni normális irányú sebessége

$$v_{1\textrm{n}} = -0,45\ \frac{\textrm{m}}{\textrm{s}},$$

a tangenciális irányú sebessége változatlan, ugyanis az ütközés súrlódásmentes, azaz

$$v_{1\textrm{t}} = c_{1\textrm{t}} = 1,3\ \frac{\textrm{m}}{\textrm{s}}.$$

A rúd az A pont körül végez forgómozgást, ezért T pontjának sebességének segítségével az ütközés utáni szögsebessége számítható:

$$\omega_2 = \frac{v_{\textrm{Tn}}}{l}.$$

```python
ω_2 = v_Tn / l
display(Math('ω_{{2}} = {}'.format(sp.latex(ω_2))))

ω_2_num = ω_2.subs(adatok).evalf(2)
display(Math('ω_{{2}} = {}\ \\text{{rad/s}}'.format(sp.latex(ω_2_num))))
```

$\displaystyle ω_{2} = \frac{v_{Tn}}{l}$

$\displaystyle ω_{2} = 2.0\ \text{rad/s}$

Így a rúd ütközés utáni szögsebessége

$$\omega_2 = 2\ \frac{\textrm{rad}}{\textrm{s}}.$$

Készítette:

            Hertelendy Krisztián és Piri Barnabás (Alkalmazott Mechanika Szakosztály)
            Takács Dénes (BME MM) kidolgozása és ábrái alapján.

            Hibák, javaslatok:
            amsz.bme@gmail.com

            2021.03.31.


# Propagate uncertainties with the `errors` add-on for CO2SYS-Matlab

James Orr

LSCE/IPSL, CEA-CNRS-UVSQ, Gif-sur-Yvette, France

27 February 2018

updated: 29 June 2020

**Abstract**: This notebook shows you how to use the 'errors' add-on for CO2SYS-Matlab to propagate uncertainties. It uses CO2SYS-Matlab and the add-on routine `errors.m` (which itself calls another add-on routine `derivnum.m`) in `octave`, GNU's clone of Matlab. You can either inspect the HTML version of this file or execute its commands interactively in your browser. For the latter, you'll need to install `jupyter notebook`, `octave`, and `oct2py`, which includes the python-octave interface called `octavemagic`. Fortunately, that installation is very easy (see below).

#### Table of Contents:

    1. Basics (install & load octave)
    2. Propagate uncertainties: use `errors` add-on (with ALK-DIC input pair)
       * total uncertainty
       * uncertainty from the constants (standard input uncertainties)
       * uncertainty from the boron-to-salinity ratio (standard input uncertainty)
       * percent relative uncertainty
       * convert uncertainty in computed [H+] to uncertainty in computed pH

## 1. Basics

#### Run interactively

If you are visualizing this after clicking on the link to this file on github, you are seeing the HTML version of a jupyter notebook. Alternatively, you may run cells interactively and modify them if you have `jupyter notebook` installed on your machine.  To install that software, just download the anaconda open software installer for your computing platform (Windows, OS X, or Linux) from https://www.anaconda.com/ and then follow the easy install instructions at

https://docs.anaconda.com/anaconda/install/

Then just download this `jupyter notebook` file as well as the 3 routines in the src directory (`CO2SYS.m`, `errors.m`, and `derivnum.m`).  Afterwards, you'll only need to install `octave` and `oct2py` using the 1-line command in the following section.

#### Install `octavemagic`

#### To install the octavemagic funtionality, we must install `oct2py`, with the following command at the Unix prompt:

  `conda install -c conda-forge octave oct2py`

That command also installs `octave`. Then launch the notebook as usual with the following command:

  `jupyter notebook`

A new window or tab should then appear in your browser, showing the files in the directory from where you launched the above command. Then just click on one of the `.ipynb` files, such as this one.

Once the notebook file appears in your browser, move to any of its cells with your mouse. Run a cell by clicking on it and hitting Ctrl-Return.  Alternatively, type Shift-Return to run a cell and then move to the next one. More information on all the interactive commands is available in the Jupyter Notebook Quick Start Guide: http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/execute.html

At the top of the notebook, you'll see a number of Tabs (File, Edit, View, Insert, ...). Those tabs provide commands that will allow you to do whatever you like. Under the Help tab you'll find keyboard shortcuts for commands. Alternatively, a cheat sheet for short cuts to commands within `jupyter notebook` is available at https://zenodo.org/record/44973/files/ipynb-cheat-sheet.pdf . Or use the command palette after typing Ctrl-Shift-P.

#### Documentation for octavemagic

Details on using octavemagic are here: https://ipython.org/ipython-doc/2/config/extensions/octavemagic.html

#### Load octave magic function

Because octavemagic is now in conda's oct2py module, it is loaded with a slight modification to what is given on the above web page, i.e., now with the command below

```python
%load_ext oct2py.ipython
```

#### Specify the directory where you have put the Matlab routines `CO2SYS.m`, `errors.m`, and `derivnum.m`.

```octave
%%octave
addpath ("~/Software/MATLAB/CO2SYS-MATLAB/src")
```

## 2. Propagate uncertainties with new `errors` add-on for CO2SYS-Matlab

```octave
%%octave
help errors
```

    'errors' is a function from the file /homel/orr/Software/MATLAB/CO2SYS-MATLAB/src/errors.m

     errors()

     This subroutine propagates uncertainties for the marine carbonate chemistry calculations

     from errors (or uncertainties) on six input

      - pair of carbonate system variables

      - nutrients (silicate and phosphate concentrations)

      - temperature and salinity

     plus errors in dissociation constants pK0, pK1, pK2, pKb, pKw, pKspa, and pKspc as well as total boron

     It calls derivnum, which computes numerical derivatives, and then

     it applies error propagation using the method of moments.

     The latter is a general technique to estimate the 2nd moment of a variable z

     (variance or standard deviation) based on a 1st-order approximation to z.

    **************************************************************************

      **** SYNTAX:

      [err, headers, units] = errors(PAR1,PAR2,PAR1TYPE,PAR2TYPE,..  .

                                         SAL,TEMPIN,TEMPOUT,PRESIN,PRESOUT,SI,PO4,...

                                         ePAR1,ePAR2,eSAL,eTEMP,eSI,ePO4,epK,eBt,r,...

                                         pHSCALEIN,K1K2CONSTANTS,KSO4CONSTANTS)

      **** SYNTAX EXAMPLES:

      [Result]                = errors(2400,2200,1,2,35,10,10,0,0,15,1,2,2,0.01,0.01,0,0,0,0,0,1,4,1)

      [Result,Headers]        = errors(2400,   8,1,3,35,25,5,0,3000,15,1,2,0.001,0,0,0,0,0,0,0,1,4,1)

      [Result,Headers,Units]  = errors(500,    8,5,3,35,25,5,0,4000,15,1,2,0.001,0,0,0,0,'','',0,1,4,1)

      [A]                     = errors(2400,2000:10:2400,1,2,35,10,10,0,0,15,2,2,0,0,0,0,'','',0,1,1,4,1)

      [A]                     = errors(2400,2200,1,2,0:1:35,0,25,4200,0,15,1,2,2,0,0,0,0,'','',0,1,4,1)

      epK = [0.002, 0.0075, 0.015, 0.01, 0.01, 0.02, 0.02];

      eBt = 0.02;

      [A, hdr, units]   = errors(2400,2200,1,2,35,0,25,0:100:4200,0,15,1,2,2,0,0,0,0,epK,eBt,0,1,4,1)

    **************************************************************************

     INPUT:

       - ePAR1, ePAR2   :  uncertainty of PAR1 and PAR2 of input pair of CO2 system variables (same units as PAR1 & PAR2)

       - eS, eT         :  uncertainty of Salinity and Temperature (same units as S and T)

       - ePO4, eSI      :  uncertainty of Phosphate and Silicate total concentrations (same units as PO4 and SI [umol/kg])

       - epK            :  uncertainty of all seven dissociation constants (a vector) [pK units]

       - eBt            :  uncertainty of total boron, given as fractional relative error (eBt=0.02 is a 2% error)

       - r              :  correlation coefficient between PAR1 AND PAR2 (typicaly 0)

       - others         :  same as input for subroutine  CO2SYS() : scalar or vectors

     All parameters may be scalars or vectors except epK and eBt.

       * epK must be vector of 7 values : errors of [pK0, pK1, pK2, pKb, pKw, pKspa, pKspc].

         These errors are assumed to be the same for all rows of data.

         These 7 values are in pK units

         if epK is empty (= ''), this routine specifies default values.

         These default standard errors are :

            pK0   :  0.002

            pK1   :  0.0075

            pK2   :  0.015

            pKb   :  0.01    boric acid

            pKw   :  0.01    water dissociation

            pKspa :  0.02    solubility product of Aragonite

            pKspc :  0.02    solubility product of Calcite

       * eBt is a scalar real number, fractional relative error (between 0.00 and 1.00)

         for TB, where the default is eBt=0.02. It is assumed to be the same

         for all rows of data.

     In constrast, ePAR1, ePAR2, eS, eT, ePO4 and eSI,

       - if vectors, are errors associated with each data point

       - if scalars, are one error value associated to all data points

     The same for parameter "r".

     If 'r' is nonzero with a value between -1.0 and 1.0, it indicates the correlation

     between uncertainties of the input pair of carbonate system variables.

     By default, 'r' is zero. However, for some pairs the user may want to specify a

     different value. For example, measurements of pCO2 and pH are often anti-correlated.

     The same goes for two other pairs: 'CO2 and CO3' and 'pCO2 and

     CO3'. But even for these cases, care is needed when using non-zero values of 'r'.

     When the user propagates errors for an individual

     measurement, 'r' should ALWAYS be zero if each member of the input pair is

     measured independently. In this case, we are interested in the

     correlation between the uncertainties in those measurements, not in

     the correlation between the measurments themselves. Uncertainties from

     those measurements are probably not correlated if they come from

     different instruments. Conversely, if users are interested in the

     error in the mean of a distribution of measurements (i.e., if they are

     propagating standard errors instead of standard deviations), one

     should then also account for the correlation between the measurements of

     the two variables of the input pair.

     For input pairs where one member is pH, this 'errors' routine automatically

     inverses the sign of 'r'.

     That inversion is done because the associated derivatives are computed in terms of

     the hydrogen ion concentration H+, not pH. Therefore for each of these 6

     flags, if the user wants to compute 'r' that should be done (1) using

     the H+ concentration instead of pH, and (2) the sign of that computed 'r'

     should be inversed when passing it as an argument to this routine.

     To express perfect anticorrelation with pH, the user should

     use 'r=-1.0'.

    **************************************************************************

     OUTPUT: * an array containing uncertainty for the following variables

               (one row per sample):

             *  a cell-array containing crudely formatted headers

        POS  PARAMETER        UNIT

        01 - TAlk                 (umol/kg)

        02 - TCO2                 (umol/kg)

        03   fCO2in               (uatm)

        04 - HCO3in               (umol/kg)

        05 - CO3in                (umol/kg)

        06 - CO2in                (umol/kg)

        07 - OmegaCAin            ()

        08 - OmegaARin            ()

        09 - xCO2in               (ppm)

        10 - Hout                 (nmol/kg)

        11 - pCO2out              (uatm)

        12 - fCO2out              (uatm)

        13 - HCO3out              (umol/kg)

        14 - CO3out               (umol/kg)

        15 - CO2out               (umol/kg)

        16 - OmegaCAout           ()

        17 - OmegaARout           ()

        18 - xCO2out              (ppm)

     NOTE: Only uncertainties for the output variables are provided.

           The 1st 2 columns will change as a function of PAR1TYPE and PAR2TYPE.

           In the case above, the input pair is pH-pCO2 (PAR1TYPE=3, PAR2TYPE=4).

           That is why the results in the first 2 columns are for the other 2

           possible input variables (TAlk and TCO2)

     EXAMPLE: a nice way to see the headers & units along with the results

     [e, ehead, eunits] = errors (PAR1,PAR2,PAR1TYPE,PAR2TYPE,SAL,TEMPIN,TEMPOUT,PRESIN,PRESOUT,SI,PO4,...

                                  ePAR1,ePAR2,eSAL,eTEMP,eSI,ePO4,epK,eBt,r,pHSCALEIN,K1K2CONSTANTS,KSO4CONSTANTS);

      printf("%s   %s %s %s %s  %s %s %s %s \n", ehead{1:9});

      printf("%s   %s  %s    %s %s %s    %s      %s          %s \n", eunits{1:9});

      printf("%f  %f  %f  %f  %f %f  %f     %f     %f \n", e(1:9));

    Additional help for built-in functions and operators is

    available in the online version of the manual.  Use the command

    'doc <topic>' to search the manual index.

    Help and information about Octave is also available on the WWW

    at http://www.octave.org and via the help@octave.org

    mailing list.

### 2.1 Specify input variables and choices

```octave
%%octave

# Standard input for CO2SYS:
# --------------------------
    # Input Variables:
    PAR1 = 2300;        % ALK
    PAR2 = 2000;        % DIC
    PAR1TYPE = 1;       % 1=ALK, 2=DIC, 3=pH, 4=pCO2, 5=fCO2
    PAR2TYPE = 2;       % Same 5 choices as PAR1TYPE
    SAL = 35;           % Salinity
    TEMPIN = 18;        % Temperature (input)
    TEMPOUT = 25;       % Temperature (output)
    PRESIN = 0;         % Pressure (input)
    PRESOUT = PRESIN;   % Pressure (output)
    SI = 60;            % Total dissolved inorganic silicon (Sit)
    PO4 = 2;            % Total dissoloved inorganic Phosphorus (Pt)

    # Input Parameters:
    pHSCALEIN = 2;      % pH scale (1=total, 2=seawater, 3=NBS, 4=Free)
    K1K2CONSTANTS = 15; % set for K1 & K2: (a) 10=Lueker et al. (2000); (b) 14=Millero (2010)
    KSO4CONSTANTS = 1;  % KSO4 of Dickson (1990a) & Total dissolved boron (Bt) from Uppstrom (1974)

# Input for CO2SYS:
# --------------------------
    # Input variables for error propagation:
    r = 0.0;            % Correlation between **uncertainties** in PAR1 and PAR2 (-1 < r < 1)
    ePAR1 = 2;          % uncertainty in PAR1 (same units as PAR1)
    ePAR2 = 2;          % uncertainty in PAR2 (same units as PAR2)
    eSAL = 0;           % uncertainty in Salinity
    eTEMP = 0;          % uncertainty in Temperature
    eSI = 4;            % uncertainty in Sit
    ePO4 = 0.1;         % uncertainty in Pt
    #Default uncertainties (pK units): epK0 , epK1, epK2, epKb, epKw, epKspA, epKspC
    epK =                [0.004, 0.015, 0.03, 0.01, 0.01,   0.02,  0.02];
    #Default uncertainty for Total boron (0.01, i.e., a 1% relative uncertainty)
    eBt = 0.01;

```

### 2.2 Propagate uncertainties neglecting uncertainties from equilibrium constants & total boron

```octave
%%octave
% With no errors from Ks
    epK = 0.0;
    eBt = 0.0;
    [e, ehead, enice] = errors (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4,...
                              ePAR1, ePAR2, eSAL, eTEMP, eSI, ePO4, epK, eBt, r, ...,
                              pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);
#   Print results
#    e
#    ehead
#    enice

# Print (nicely formatted):
  printf("%s   %s %s %s %s %s %s %s %s \n", ehead{1:9});
  printf("%f %f  %f  %f  %f %f %f     %f     %f \n", e(1:9));
```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    0.077469 3.805834  3.792600  3.408018  1.848113 0.130039 0.044126     0.028534     3.883373

### 2.3 Propagate uncertainties sequentially accounting for uncertainties from constants & total boron

#### Default uncertainties in equilibrium constants, but assume no uncertainty in total boron $B_\text{T}$

```octave
%%octave
% With std errors from constants except for Bt
    epK = [0.002, 0.0075, 0.015, 0.01, 0.01, 0.02, 0.02];
    eBt = 0.0;
    [ek, ekhead, eknice] = errors (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4, ...
                             ePAR1, ePAR2, eSAL, eTEMP, eSI, ePO4, epK, eBt, r, ...
                             pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);

  printf("%s   %s %s %s %s %s %s %s %s \n", ekhead{1:9});
  printf("%f %f %f %f %f %f  %f     %f     %f \n", ek(1:9));
```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    0.189647 9.461716 9.428816 4.261992 3.008619 0.319733  0.238120     0.153982     9.654489

#### Default uncertainties in equilibrium constants & default uncertainty in total $B_\text{T}$ (1%)

```octave
%%octave
% With std uncertainties from constants & Bt
    epK = [0.002, 0.0075, 0.015, 0.01, 0.01, 0.02, 0.02];
    eBt = 0.02;
    [ekb, ekbhead] = errors (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4, ...
                             ePAR1, ePAR2, eSAL, eTEMP, eSI, ePO4, epK, eBt, r, ...
                             pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);

  printf("%s   %s %s %s %s %s %s %s %s \n", ekbhead{1:9});
  printf("%f %f %f %f %f %f  %f     %f     %f \n", ekb(1:9));
```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    0.195943 9.731243 9.697405 4.425336 3.265272 0.329042  0.240039     0.155223     9.929506

Same calculation, but with a simpler way to specify the defaults for epK and eBt

```octave
%%octave
% With std uncertainties from constants & Bt
    epK = '';
    eBt = '';
    [ekb, ekbhead] = errors (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4, ...
                             ePAR1, ePAR2, eSAL, eTEMP, eSI, ePO4, epK, eBt, r, ...
                             pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);

  printf("%s   %s %s %s %s %s %s %s %s \n", ekbhead{1:9});
  printf("%f %f %f %f %f %f  %f     %f     %f \n", ekb(1:9));
```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    0.195943 9.731243 9.697405 4.425336 3.265272 0.329042  0.240039     0.155223     9.929506

### 2.4 Summarize effects of accouning for uncertainties in equil. constants & total boron

#### Fractional increase in propagated uncertainty due to accounting for uncertainties from equilibrium constants (default values)

```octave
%%octave
  printf("%s   %s %s %s %s %s %s %s %s \n", ekbhead{1:9});
 %printf("%f %f %f %f %f %f  %f     %f     %f \n", e(1:9));
 %printf("%f %f %f %f %f %f  %f     %f     %f \n", ek(1:9));
  printf("%f %f %f %f %f %f  %f     %f     %f \n", ek(1:9) ./ e(1:9));

```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    2.448047 2.486109 2.486109 1.250578 1.627941 2.458746  5.396363     5.396363     2.486109

Conclusion: Accounting for uncertainties from the equilibrium constants increases propagated uncertainties by 1.3 to 5.4 times for the At-Ct pair with default uncertainties.

#### Fractional increase in propagated uncertainty due to accounting for uncertainty in total boron (default value)

```octave
%%octave
  printf("%s   %s %s %s %s %s %s %s %s \n", ekbhead{1:9});
 %printf("%f %f %f %f %f %f  %f     %f     %f \n", ek(1:9));
 %printf("%f %f %f %f %f %f  %f     %f     %f \n", ekb(1:9));
  printf("%f %f %f %f %f %f  %f     %f     %f \n", ekb(1:9) ./ ek(1:9));

```

    u(Hin)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in) u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    1.033198 1.028486 1.028486 1.038326 1.085306 1.029115  1.008062     1.008062     1.028486

Conclusion: Accounting for uncertainty in total boron increases propagated absolute uncertainties by about 3% for most computed variables (except 9% for $[\mathrm{CO}_3^{2-}]$) when using the default uncertainty (eBt = 0.02) but by less than 1% when using the eBt=0.01, i.e., with the At-Ct pair.

### 2.5 Compute percent relative uncertainties

#### Compute CO2SYS variables (the reference)

```octave
%%octave
% With std uncertainties from Ks
    [d, dhead, dnice] = CO2SYS (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4, ...
                             pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);
```

#### Reorder CO2SYS output in new array having same order as output array from `errors` (above, section 2.3) for later division (below)

```octave
%%octave
ALK     = d(:,1);    %01 - TAlk                 (umol/kgSW)
DIC     = d(:,2);    %02 - TCO2                 (umol/kgSW)
pH      = d(:,3);    %03 - pHin                 ()
pCO2    = d(:,4);    %04 - pCO2 input           (uatm)
fCO2    = d(:,5);    %05 - fCO2 input           (uatm)
HCO3    = d(:,6);    %06 - HCO3 input           (umol/kgSW)
CO3     = d(:,7);    %07 - CO3 input            (umol/kgSW)
CO2     = d(:,8);    %08 - CO2 input            (umol/kgSW)
Hfree   = d(:,13);   %13 - Hfree input          (umol/kgSW)
OmegaCa = d(:,15);   %15 - OmegaCa input        ()
OmegaAr = d(:,16);   %16 - OmegaAr input        ()
xCO2    = d(:,17);   %17 - xCO2 input           (ppm)

H = 10**(-pH) * 1e9; # Convert pH to H+ and then from mol/kg to nmol/kg
dar = [H, pCO2, fCO2, HCO3, CO3, CO2, OmegaCa, OmegaAr, xCO2];
%dar
```

#### Compute percent error

```octave
%%octave
  perr = 100* ekb(1:9) ./ dar;

# Show uncertainties in percent of base value computed with CO2SYS
  printf("%s  %s   %s   %s  %s    %s   %s %s %s \n", ehead{1:9});
  printf("%4.2f  %6.2f     %7.2f    %8.2f    %7.2f     %7.2f    %7.2f      %7.2f      %7.2f \n", perr(1:9));
```

    u(Hin)  u(pCO2in)   u(fCO2in)   u(HCO3in)  u(CO3in)    u(CO2in)   u(OmegaCAin) u(OmegaARin) u(xCO2in)

    2.71    3.20        3.20        0.25       1.58        3.17       4.87         4.87         3.20

#### Compute absolute error of pH from relative error in H

An absolute change in pH is equivalent to a relative change in H+.  That is, it can be shown that for **small** changes in H+ (dH):

\begin{equation}
d\text{pH} = -\frac{1}{\text{ln}(10)}  \frac{d\text{H}}{\text{H}}
\end{equation}

To get to this result,

1) start with the basic definition: $\text{pH} = - \text{log}_{10} \, [\text{H}^+]$,

2) convert the logarithm base 10 to a natural log with $\text{log}_{10}(x) = \text{ln}(x) / \text{ln}(10)$,

3) take the derivative of each side, and

4) plug in the definition of the derivative of a natural log, i.e.,  $d \text{ln}(x) = dx/x$.

##### Do the actual calculation

```octave
%%octave
# We drop the minus sign below because uncertainties are positive by definition (1 sigma)
# Note that "(0.01 * perr)" is the fractional error in hydrogen ion concentration (i.e., dH/H)
  dpH = (0.01 * perr(1))/log(10);
  dpH
```

    dpH =  0.011768

### 2.6 Example of using `errors.m` routine with the pH-At pair

```octave
%%octave

# Standard input for CO2SYS:
# --------------------------
    # Input Variables:
    PAR1 = 2300;        % ALK
    PAR2 = 8.1;         % pH
    PAR1TYPE = 1;       % 1=ALK, 2=DIC, 3=pH, 4=pCO2, 5=fCO2
    PAR2TYPE = 3;       % Same 5 choices as PAR1TYPE
    SAL = 35;           % Salinity
    TEMPIN = 18;        % Temperature (input)
    TEMPOUT = 18;       % Temperature (output)
    PRESIN = 0;         % Pressure (input)
    PRESOUT = PRESIN;   % Pressure (output)
    SI = 60;            % Total dissolved inorganic silicon (Sit)
    PO4 = 2;            % Total dissoloved inorganic Phosphorus (Pt)

# Input Parameters:
    pHSCALEIN = 1;      %% pH scale (1=total, 2=seawater, 3=NBS, 4=Free)
    K1K2CONSTANTS = 10; %% set for K1 & K2: (a) 10=Lueker et al. (2000); (b) 14=Millero (2010)
    KSO4CONSTANTS = 1;  %% KSO4 of Dickson (1990a) & Total dissolved boron (Bt) from Uppstrom (1974)

# Input for CO2SYS:
# --------------------------
    # Input variables for error propagation:
    r = 0.0;            % Correlation between **uncertainties** in PAR1 and PAR2 (-1 < r < 1)
    ePAR1 = 2;          % uncertainty in PAR1 (same units as PAR1)
    ePAR2 = 0.01;          % uncertainty in PAR2 (same units as PAR2)
    eSAL = 0;           % uncertainty in Salinity
    eTEMP = 0;          % uncertainty in Temperature
    eSI = 4;            % uncertainty in Sit
    ePO4 = 0.1;         % uncertainty in Pt
    #Default uncertainties: epK0 , epK1, epK2, epKb, epKw, epKspA, epKspC
    epK = [0.002, 0.0075, 0.015, 0.01, 0.01, 0.02, 0.02];
    #Default uncertainty for Total Boron: 0.01 (i.e., a 1% relative error)
    eBt = 0.02;
```

```octave
%%octave
% With std uncertainties from constants & Bt
    epK = '';
    eBt = '';
    [e, ehead, eunits] = errors (PAR1, PAR2, PAR1TYPE, PAR2TYPE, SAL, TEMPIN, TEMPOUT, PRESIN, PRESOUT, SI, PO4, ...
                             ePAR1, ePAR2, eSAL, eTEMP, eSI, ePO4, epK, eBt, r, ...
                             pHSCALEIN, K1K2CONSTANTS, KSO4CONSTANTS);

  printf("%s   %s %s %s %s  %s %s %s %s \n", ehead{1:9});
  printf("%s   %s  %s    %s %s %s    %s      %s          %s \n", eunits{1:9});
  printf("%f  %f  %f  %f  %f %f  %f     %f     %f \n", e(1:9));
```

    u(TCO2)   u(pCO2in) u(fCO2in) u(HCO3in) u(CO3in)  u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    (umol/kg)   (uatm)  (uatm)    (umol/kg) (umol/kg) (umol/kg)    ( )      ( )          (ppm)

    8.327959  11.585334  11.545050  14.191636  6.483093 0.392120  0.261038     0.168802     11.821373

### 2.7 Example of using `errors.m` routine with the pH-pCO2 pair

```octave
%%octave

PAR1 = 7.91;    % pH_T
PAR2 = 500;    % pCO2
PAR1TYPE = 3;
PAR2TYPE = 4;
SAL = 35;
TEMPIN = 18;
TEMPOUT = 18;
PRESIN = 0;
PRESOUT = PRESIN;
SI = 0;
PO4 = 0;
pHSCALEIN = 1;   % total scale
K1K2CONSTANTS = 10; % Lueker 2000
KSO4CONSTANTS = 1;  % KSO4 of Dickson & TB of Uppstrom 1979
ePAR1 = 0.02;    % pH
ePAR2 = 5;    % pCO2
eSAL = 0;
eTEMP = 0;
eSI = 0;
ePO4 = 0;

% Set input uncertainties for constants and Bt/S to zero
epK=0;
eBt=0;

r=0;

[e, ehead, eunits] = errors (PAR1,PAR2,PAR1TYPE,PAR2TYPE,SAL,TEMPIN,TEMPOUT,PRESIN,PRESOUT,SI,PO4,...
ePAR1,ePAR2,eSAL,eTEMP,eSI,ePO4,epK,eBt,r,pHSCALEIN,K1K2CONSTANTS,KSO4CONSTANTS);

  printf("%s   %s %s %s %s  %s %s %s %s \n", ehead{1:9});
  printf("%s   %s  %s    %s %s %s    %s      %s          %s \n", eunits{1:9});
  printf("%f  %f  %f  %f  %f %f  %f     %f     %f \n", e(1:9));
```

    u(TAlk)   u(TCO2) u(fCO2in) u(HCO3in) u(CO3in)  u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    (umol/kg)   (umol/kg)  (uatm)    (umol/kg) (umol/kg) (umol/kg)    ( )      ( )          (ppm)

    104.239312  91.153989  4.982614  80.460830  10.711577 0.170842  0.255752     0.165384     5.101870

```octave
%%octave

# Use default uncertainties for constants instead of setting them to zero (case in previous cell)
epK = '';
eBt = '';

[e, ehead, eunits] = errors (PAR1,PAR2,PAR1TYPE,PAR2TYPE,SAL,TEMPIN,TEMPOUT,PRESIN,PRESOUT,SI,PO4,...
ePAR1,ePAR2,eSAL,eTEMP,eSI,ePO4,epK,eBt,r,pHSCALEIN,K1K2CONSTANTS,KSO4CONSTANTS);

  printf("%s   %s %s %s %s  %s %s %s %s \n", ehead{1:9});
  printf("%s   %s  %s    %s %s %s    %s      %s          %s \n", eunits{1:9});
  printf("%f  %f  %f  %f  %f %f  %f     %f     %f \n", e(1:9));
```

    u(TAlk)   u(TCO2) u(fCO2in) u(HCO3in) u(CO3in)  u(CO2in) u(OmegaCAin) u(OmegaARin) u(xCO2in)

    (umol/kg)   (umol/kg)  (uatm)    (umol/kg) (umol/kg) (umol/kg)    ( )      ( )          (ppm)

    110.150090  96.891395  4.982614  86.053315  11.617020 0.188087  0.305116     0.197306     5.101870

```octave
%%octave
ehead, eunits
```

    ehead =

    {

      [1,1] = u(TAlk)

      [2,1] = u(TCO2)

      [3,1] = u(fCO2in)

      [4,1] = u(HCO3in)

      [5,1] = u(CO3in)

      [6,1] = u(CO2in)

      [7,1] = u(OmegaCAin)

      [8,1] = u(OmegaARin)

      [9,1] = u(xCO2in)

      [10,1] = u(Hout)

      [11,1] = u(pCO2out)

      [12,1] = u(fCO2out)

      [13,1] = u(HCO3out)

      [14,1] = u(CO3out)

      [15,1] = u(CO2out)

      [16,1] = u(OmegaCAout)

      [17,1] = u(OmegaARout)

      [18,1] = u(xCO2out)

    }

    eunits =

    {

      [1,1] = (umol/kg)

      [2,1] = (umol/kg)

      [3,1] = (uatm)

      [4,1] = (umol/kg)

      [5,1] = (umol/kg)

      [6,1] = (umol/kg)

      [7,1] = ( )

      [8,1] = ( )

      [9,1] = (ppm)

      [10,1] = (nmol/kg)

      [11,1] = (uatm)

      [12,1] = (uatm)

      [13,1] = (umol/kg)

      [14,1] = (umol/kg)

      [15,1] = (umol/kg)

      [16,1] = ( )

      [17,1] = ( )

      [18,1] = (ppm)

    }


# Spatially selective excitation
This module will explore slice selection in which we aim to excite a slice (2D imaging) or slab (3D imaging).

First we will need to run some code to import libraries and define a Bloch solver. This time the Bloch solver has some code to make it faster but slightly less accurate.

```
# This is comment, Python will ignore this line

# Import libraries (load libraries which provide some functions)
%matplotlib inline
import numpy as np # array library
import math
import cmath
from scipy import interpolate
import numba

# For interactive plotting
from ipywidgets import interact, interactive, FloatSlider, ToggleButton
from IPython.display import clear_output, display, HTML

# for plotting modified style for better visualization
import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.rcParams['lines.linewidth'] = 4
mpl.rcParams['axes.titlesize'] = 24
mpl.rcParams['axes.labelsize'] = 20
mpl.rcParams['xtick.labelsize'] = 16
mpl.rcParams['ytick.labelsize'] = 16
mpl.rcParams['legend.fontsize'] = 16

@numba.jit(nopython=True)
def bloch_solver( B, time, freq=0, T1=2000, T2=2000, GAM=42.58e6*2*math.pi):
  # This is simple Rk4 solution to the Bloch Equations.
  #
  # Inputs:
  #   B(array)    -- Magentic Field [N x 3] (T)
  #   time(array) -- Time of each point in waveforms (s)
  #   freq        -- Frequency [Hz]
  #   T1          -- Longitudinal relaxation times (s)
  #   T2          -- Transverse relaxation times (s)
  #   M0          -- Initial state of magnetization (not equilibrium magnetization)
  # Outputs:
  #   MOutput -- Magnetization for each position in time

  # Convert frequency to rads/s
  act_freq = 2*math.pi*freq

  #Convert to rotion rates (gamma*B)
  Bx =  GAM*B[:,0]
  By =  GAM*B[:,1]
  Bz =  GAM*B[:,2] + act_freq

  # Double the resolution using linear interpolation (this is faster than splines)
  Bx2 = np.zeros( 2*len(Bz)+2)
  By2 = np.zeros( 2*len(Bz)+2)
  Bz2 = np.zeros( 2*len(Bz)+2)
  Bx2[:-4:2] = Bx[:-1]
  By2[:-4:2] = By[:-1]
  Bz2[:-4:2] = Bz[:-1]

  # Temp
  Bx2[1:-3:2] = 0.5*Bx[:-1] + 0.5*Bx[1:]
  By2[1:-3:2] = 0.5*By[:-1] + 0.5*By[1:]
  Bz2[1:-3:2] = 0.5*Bz[:-1] + 0.5*Bz[1:]

  #Initialize
  Mag = np.array([[0.0],[0.0],[1.0]])

  # Output storage
  MOutput = np.zeros_like(B)
  MOutput = np.expand_dims(MOutput,-1)

  #Runge-Kutta PDE Solution
  dt = time[2] - time[1]
  for count, t1 in enumerate(time):

    m1 = Mag

    bx = Bx2[count*2]
    by = By2[count*2]
    bz = Bz2[count*2]
    rhs = np.array([[  -1/T2,   bz,  -by],
                  [  -bz   ,-1/T2, bx],
                  [   by   , -bx, -1/T1]])

    k1 = np.dot(rhs, m1) + np.array([[0.0],[0.0],[1.0/T1]])

    t2 = t1 + dt/2
    bx = Bx2[count*2+1]
    by = By2[count*2+1]
    bz = Bz2[count*2+1]
    m2 = Mag + k1*dt/2
    k2 = np.dot(np.array([[  -1/T2,   bz,  -by],
                  [  -bz   ,-1/T2, bx],
                  [   by   ,  -bx, -1/T1]]), m2) + np.array([[0.0],[0.0],[1.0/T1]])

    t3 = t1 + dt/2
    bx = Bx2[count*2+1]
    by = By2[count*2+1]
    bz = Bz2[count*2+1]
    m3 = Mag + k2*dt/2
    k3 = np.dot(np.array([[  -1/T2,   bz,  -by],
                  [  -bz   ,-1/T2, bx],
                  [   by   ,  -bx, -1/T1]]), m3) + np.array([[0.0],[0.0],[1.0/T1]])

    t4 = t1 + dt
    bx = Bx2[count*2+2]
    by = By2[count*2+2]
    bz = Bz2[count*2+2]
    m4 = Mag + k3*dt
    k4 = np.dot(np.array([[  -1/T2,   bz,  -by],
                  [  -bz   ,-1/T2, bx],
                  [   by   ,  -bx, -1/T1]]), m4) + np.array([[0.0],[0.0],[1.0/T1]])

    # Runge-Kutta averages the above terms
    Mag = Mag + dt/6*(k1 + 2*k2 + 2*k3 + k4);

    # Save to an array
    MOutput[count,:]= Mag

  return MOutput
```

# Sinc Pulses

Not all pulses in MRI are Sinc pulses but we will consider pulses that are. Our pulses will have several paramaters:

* **TBW** [unitless]: The time bandwidth product. This is effectively how many of the sinc lobes we include. More lobes means higher selectivity
* **T** [s]: The time length of the RF pulse, this will control the total time the RF pulse takes. It will set *BW* [Hz], the bandwidth of the pulse in Hz
* **Window function** : This is a function that rolls off the ends of the sinc so that there is a smoother transition when cutting off lobes. For this exercise I am using a hamming window but other options exist.

The below code generated the pulse envelope. In this code, $B_1$ will be aligned in $x$ such that it rotates the magnetization into the $y$ direction. The sinc can also be modulated by a frequency to excite at a different center frequency.

```
def generate_sinc(T, TBW=4, window=True, dt=4e-6, GAM=42.58e6, flip=10, freq=0):

  # Number of points in waveform
  Nt = int(T/dt)

  # Time normalized to the time bandwidth product
  t = np.linspace(-TBW,TBW, Nt)

  # Get the pulse shape
  B1 = np.sinc(t)

  # To deal with the truncation we can apply a window function to taper the RF profile
  if window:
    B1 *= np.hamming(Nt)

  # Normalize to the flip angle
  B1 = B1 * (flip/360) / (GAM*np.sum(B1*dt))

  # Get actual time
  time = dt*np.arange(Nt)

  # Convert to complex with frequency
  B1 = B1*np.exp(2j*math.pi*time*freq)

  return time, B1

def simulate_rf(time, B1):
  B = np.zeros( (len(B1),3))
  B[:,0] = np.real(B1)
  B[:,1] = np.imag(B1)

  Mout = bloch_solver( B, time, T1=2000, T2=2000, GAM=42.58e6*2*math.pi)

  return Mout

def plot_rf(T, TBW, flip, freq, window):
  # Create Sinc
  time, B1 = generate_sinc(T/1e3, TBW, flip=flip, window=window, freq=freq)

  # Simulate
  Mout = simulate_rf( time, B1)

  plt.figure(figsize=(12,6))
  plt.subplot(121)
  plt.plot(1e3*time,np.real(B1),label='$B_x$')
  plt.plot(1e3*time,np.imag(B1),label='$B_y$')
  plt.xlabel('Time [ms]')
  plt.ylabel('$B_1$ [T]')
  plt.legend()

  plt.subplot(122)
  plt.plot(1e3*time,Mout[:,2], label=r'$M_z$')
  plt.plot(1e3*time,Mout[:,0], label=r'$M_x$')
  plt.plot(1e3*time,Mout[:,1], label=r'$M_y$')
  plt.xlabel('Time [ms]')
  plt.ylabel('Magnetization [a.u.]')

  plt.legend()
  plt.show()
```

# Sinc scaling parameters without gradients

Below is a simulation using a standard Bloch simulator. The paramaters are set to maintain a constant flip angle for on-resonant spins. Try the following purtibations, first thinking what the affect might be on the peak $B_1$ which is often limited on systems.
* Change the flip angle
* Change the the TBW and T
* Sweep the frequency, does the flip angle change? Is this different for a short and long pulse?

```
w = interactive(plot_rf,
                TBW=FloatSlider(min=1, max=12, step=1, value=2, description='TBW '),
                T=FloatSlider(min=0.5, max=10, step=0.5, value=2, description='T [ms]'),
                flip=FloatSlider(min=1, max=90, step=1, value=20, description='Flip [deg.]'),
                freq=FloatSlider(min=-2000, max=2000, step=100, value=0, description='RF Freq [Hz]'),
                window=ToggleButton(value=True,description='Toggle Window'))
display(w)
```

    interactive(children=(FloatSlider(value=2.0, description='T [ms]', max=10.0, min=0.5, step=0.5), FloatSlider(v…

# Adding a slice select gradient
 Now we will add a slice select gradient. In this code, the time of the pulse scales with the $TBW$ by:
\begin{equation}
T = TBW*0.25 \times 10^{-3}
\end{equation}

This means that the bandwidth of the pulse ($BW$) is fixed to:
\begin{equation}
BW=\frac{TBW}{T} = \frac{TBW}{TBW*0.25 \times 10^{-3} [s]}= 4000 [Hz]
\end{equation}

This makes seeing many of the effects much easier. Some questions to consider:

* What is the effect of changing the center frequency? Does it depend on the gradient strength?
* How does changing the gradient amplitude affect the slice thickness?
* Does toggling the window function alter the slice profile?
* What might be the practical need for the rephasing gradient?
* Does the profile look the same for a 90 degree flip angle as a 15 degree? [the small tip angle aproximation will be violated and the responsse will not be a Fourier transform]

```

def simulate_rf_g(time, G, z, B1, T1, T2):
  B = np.zeros( (len(B1),3))
  B[:,0] = np.real(B1)
  B[:,1] = np.imag(B1)
  B[:,2] = G*z

  Mout = bloch_solver( B, time)

  return Mout

def generate_sinc_and_grad( Gsel=1e-3, TBW=4, flip=10, window=True, freq=0, rephase=True):

  T = 0.25e-3*TBW

  # Generate RF
  time, B1 = generate_sinc( T, TBW, flip=flip, window=window, freq=freq)

  # Get delta time
  dt = time[1] - time[0]

  # Gradient for slice select (T/m)
  gselect = Gsel*np.ones_like(time)

  if rephase:
    # Generate rephaser
    Grephase = 20e-3 # amplitude of rephaser
    T_rephase = (Gsel * T * 0.5) / Grephase # Area / Gradient strength
    Nrephase = int(np.ceil( T_rephase / dt )) # number of points
    grephase = -Grephase*np.ones((Nrephase,)) #actual
    grephase = -grephase*0.5*np.sum(gselect)/np.sum(grephase)

    # Pad with zeros
    pad = np.zeros((20,))
    G = np.concatenate( (pad, gselect, grephase, pad))
    B1 = np.concatenate( (pad, B1, 0*grephase, pad))
    time = dt*np.arange(len(B1))
  else:
    # Pad with zeros
    pad = np.zeros((20,))
    G = np.concatenate( (pad, gselect, pad))
    B1 = np.concatenate( (pad, B1, pad))
    time = dt*np.arange(len(B1))

  return time, B1, G

def plot_rf_g(TBW, flip, Gsel, freq, window, rephase):

  # Essentially ignore T1/T2
  T1=1000
  T2=1000

  # Convert to si units
  Gsel=Gsel/1e3 # mT/m to T/m

  # Create Sinc
  time, B1, G = generate_sinc_and_grad( Gsel=Gsel, TBW=TBW, flip=flip, window=window, freq=freq, rephase=rephase)

  ## Simulate
  zsim = np.linspace(-0.05,0.05,501) # Z values to simulate
  Mall = []
  for z in zsim:
    Mout = simulate_rf_g(time, G, z, B1, T1, T2)
    Mall.append(Mout)
  Mall = np.stack(Mall,axis=0)

  # Plots
  fig=plt.figure(figsize=(12,8))

  # Plot gradients
  plt.subplot(221)
  plt.plot(1e3*time, 1e3*G, color='b')
  plt.ylim([-25, 25])
  plt.xlabel('Time [ms]')
  plt.ylabel('G [mT/m]', color='b')

  # Plot B1
  plt.subplot(223)
  plt.plot(1e3*time,np.real(B1),label='$B_x$')
  plt.plot(1e3*time,np.imag(B1),label='$B_y$')
  plt.xlabel('Time [ms]')
  plt.ylabel('$B_1$ [T]')
  plt.legend()
  plt.ylim([-1.2*np.max(np.abs(B1)), 1.2*np.max(np.abs(B1))])
  plt.xlabel('Time [ms]')

  # Plot of Mz
  plt.subplot(222)
  plt.plot(zsim*1e3,Mall[:,-1,2],label='$M_z$')
  plt.xlabel('Position [mm]')
  plt.ylabel('M [a.u.]')

  # Plot of Mxy
  plt.subplot(224)
  plt.plot(zsim*1e3,Mall[:,-1,1],label='$M_y$')
  plt.plot(zsim*1e3,Mall[:,-1,0],label='$M_x$')
  plt.xlabel('Position [mm]')
  plt.ylabel('M [a.u.]')
  plt.legend()

  plt.tight_layout(pad=0.4, w_pad=4.0, h_pad=1.0)
  plt.show()

```

```

w = interactive(plot_rf_g,
                TBW=FloatSlider(min=1, max=12, step=1, value=6, description='TBW',continuous_update=False),
                flip=FloatSlider(min=1, max=90, step=1, value=10, description='Flip [deg.]',continuous_update=False),
                freq=FloatSlider(min=-5000, max=5000, step=100, value=0, description='Freq [Hz]',continuous_update=False),
                Gsel=FloatSlider(min=3, max=20, step=1, value=10,description='Gsel [mT/m]', continuous_update=False),
                window=ToggleButton(value=True,description='Toggle Window',continuous_update=False),
                rephase=ToggleButton(value=True,description='Toggle Rephaser',continuous_update=False),
                )
display(w)
```

    interactive(children=(FloatSlider(value=6.0, continuous_update=False, description='TBW', max=12.0, min=1.0, st…


# Tutorial 1: Linear regression with MSE
**Week 1, Day 3: Model Fitting**

**By Neuromatch Academy**

**Content creators**: Pierre-Étienne Fiquet, Anqi Wu, Alex Hyafil with help from Byron Galbraith

**Content reviewers**: Lina Teichmann, Saeed Salehi, Patrick Mineault,  Ella Batty, Michael Waskom

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

___
# Tutorial Objectives

*Estimated timing of tutorial: 30 minutes*

This is Tutorial 1 of a series on fitting models to data. We start with simple linear regression, using least squares optimization (Tutorial 1) and Maximum Likelihood Estimation (Tutorial 2). We will use bootstrapping to build confidence intervals around the inferred linear model parameters (Tutorial 3). We'll finish our exploration of regression models by generalizing to multiple linear regression and polynomial regression (Tutorial 4). We end by learning how to choose between these various models. We discuss the bias-variance trade-off (Tutorial 5) and Cross Validation for model selection (Tutorial 6).

In this tutorial, we will learn how to fit simple linear models to data.
- Learn how to calculate the mean-squared error (MSE)
- Explore how model parameters (slope) influence the MSE
- Learn how to find the optimal model parameter using least-squares optimization

---

**acknowledgements:**
- we thank Eero Simoncelli, much of today's tutorials are inspired by exercises asigned in his mathtools class.

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/2mkq4/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

---
# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
#@title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
#@title Plotting Functions

def plot_observed_vs_predicted(x, y, y_hat, theta_hat):
  """ Plot observed vs predicted data

  Args:
    x (ndarray): observed x values
    y (ndarray): observed y values
    y_hat (ndarray): predicted y values
    theta_hat (ndarray):
  """
  fig, ax = plt.subplots()
  ax.scatter(x, y, label='Observed')  # our data scatter plot
  ax.plot(x, y_hat, color='r', label='Fit')  # our estimated model
  # plot residuals
  ymin = np.minimum(y, y_hat)
  ymax = np.maximum(y, y_hat)
  ax.vlines(x, ymin, ymax, 'g', alpha=0.5, label='Residuals')
  ax.set(
      title=fr"$\hat{{\theta}}$ = {theta_hat:0.2f}, MSE = {mse(x, y, theta_hat):.2f}",
      xlabel='x',
      ylabel='y'
  )
  ax.legend()
```

---
# Section 1: Mean Squared Error (MSE)

```python
# @title Video 1: Linear Regression & Mean Squared Error
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1tA411e7NW", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="HumajfjJ37E", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video covers a 1D linear regression and mean squared error.

 Click here for text recap of video

**Linear least squares regression** is an old but gold  optimization procedure that we are going to use for data fitting. Least squares (LS) optimization problems are those in which the objective function is a quadratic function of the
parameter(s) being optimized.

Suppose you have a set of measurements: for each data point or measurement, you have $y_{i}$ (the "dependent" variable) obtained for a different input value, $x_{i}$ (the "independent" or "explanatory" variable).  Suppose we believe the measurements are proportional to the input values, but are corrupted by some (random) measurement errors, $\epsilon_{i}$, that is:

\begin{align}
y_{i}= \theta x_{i}+\epsilon_{i}
\end{align}

for some unknown slope parameter $\theta.$ The least squares regression problem uses **mean squared error (MSE)** as its objective function, it aims to find the value of the parameter $\theta$ by minimizing the average of squared errors:

\begin{align}
\min _{\theta} \frac{1}{N}\sum_{i=1}^{N}\left(y_{i}-\theta x_{i}\right)^{2}
\end{align}

We will now explore how MSE is used in fitting a linear regression model to data. For illustrative purposes, we will create a simple synthetic dataset where we know the true underlying model. This will allow us to see how our estimation efforts compare in uncovering the real model (though in practice we rarely have this luxury).

First we will generate some noisy samples $x$ from [0, 10) along the line $y = 1.2x$ as our dataset we wish to fit a model to.

```python
# @title

# @markdown Execute this cell to generate some simulated data

# setting a fixed seed to our random number generator ensures we will always
# get the same psuedorandom number sequence
np.random.seed(121)

# Let's set some parameters
theta = 1.2
n_samples = 30

# Draw x and then calculate y
x = 10 * np.random.rand(n_samples)  # sample from a uniform distribution over [0,10)
noise = np.random.randn(n_samples)  # sample from a standard normal distribution
y = theta * x + noise

# Plot the results
fig, ax = plt.subplots()
ax.scatter(x, y)  # produces a scatter plot
ax.set(xlabel='x', ylabel='y');
```

Now that we have our suitably noisy dataset, we can start trying to estimate the underlying model that produced it. We use MSE to evaluate how successful a particular slope estimate $\hat{\theta}$ is for explaining the data, with the closer to 0 the MSE is, the better our estimate fits the data.

## Coding Exercise 1: Compute MSE

In this exercise you will implement a method to compute the mean squared error for a set of inputs $\mathbf{x}$, measurements $\mathbf{y}$, and slope estimate $\hat{\theta}$. Here, $\mathbf{x}$ and $\mathbf{y}$ are vectors of data points. We will then compute and print the mean squared error for 3 different choices of theta.

As a reminder, the equation for computing the estimated y for a single data point is:

$$\hat{y}_{i}= \theta x_{i}$$

and for mean squared error is:

\begin{align}
\min _{\theta} \frac{1}{N}\sum_{i=1}^{N}\left(y_{i}-\hat{y}_i\right)^{2}
\end{align}

```python
def mse(x, y, theta_hat):
  """Compute the mean squared error

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta_hat (float): An estimate of the slope parameter

  Returns:
    float: The mean squared error of the data with the estimated parameter.
  """
  ####################################################
  ## TODO for students: compute the mean squared error
  # Fill out function and remove
  # raise NotImplementedError("Student exercise: compute the mean squared error")
  ####################################################

  # Compute the estimated y
  y_hat = theta_hat*x

  # Compute mean squared error
  mse = np.mean((y-y_hat)**2)

  return mse

theta_hats = [0.75, 1.0, 1.5]
for theta_hat in theta_hats:
  print(f"theta_hat of {theta_hat} has an MSE of {mse(x, y, theta_hat):.2f}")
```

    theta_hat of 0.75 has an MSE of 9.08
    theta_hat of 1.0 has an MSE of 3.01
    theta_hat of 1.5 has an MSE of 4.52

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D3_ModelFitting/solutions/W1D3_Tutorial1_Solution_12a57de0.py)

The result should be:

theta_hat of 0.75 has an MSE of 9.08\
theta_hat of 1.0 has an MSE of 3.0\
theta_hat of 1.5 has an MSE of 4.52

We see that $\hat{\theta} = 1.0$ is our best estimate from the three we tried. Looking just at the raw numbers, however, isn't always satisfying, so let's visualize what our estimated model looks like over the data.

```python
#@title

#@markdown Execute this cell to visualize estimated models

fig, axes = plt.subplots(ncols=3, figsize=(18, 4))
for theta_hat, ax in zip(theta_hats, axes):

  # True data
  ax.scatter(x, y, label='Observed')  # our data scatter plot

  # Compute and plot predictions
  y_hat = theta_hat * x
  ax.plot(x, y_hat, color='r', label='Fit')  # our estimated model

  ax.set(
      title= fr'$\hat{{\theta}}$= {theta_hat}, MSE = {mse(x, y, theta_hat):.2f}',
      xlabel='x',
      ylabel='y'
  );

axes[0].legend()
```

## Interactive Demo 1: MSE Explorer

Using an interactive widget, we can easily see how changing our slope estimate changes our model fit. We display the **residuals**, the differences between observed and predicted data, as line segments between the data point (observed response) and the corresponding predicted response on the model fit line.

- What value of $\hat{\theta}$ results in the lowest MSE?
- Is this a good way of estimating $\theta$?

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

@widgets.interact(theta_hat=widgets.FloatSlider(1.0, min=0.0, max=2.0))
def plot_data_estimate(theta_hat):
  y_hat = theta_hat * x
  plot_observed_vs_predicted(x, y, y_hat, theta_hat)
```

    interactive(children=(FloatSlider(value=1.0, description='theta_hat', max=2.0), Output()), _dom_classes=('widg…

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D3_ModelFitting/solutions/W1D3_Tutorial1_Solution_d5c4912a.py)

While visually exploring several estimates can be instructive, it's not the most efficient for finding the best estimate to fit our data. Another technique we can use is choose a reasonable range of parameter values and compute the MSE at several values in that interval. This allows us to plot the error against the parameter value (this is also called an **error landscape**, especially when we deal with more than one parameter). We can select the final $\hat{\theta}$  ($\hat{\theta}_\textrm{MSE}$) as the one which results in the lowest error.

```python
# @title

# @markdown Execute this cell to loop over theta_hats, compute MSE, and plot results

# Loop over different thetas, compute MSE for each
theta_hat_grid = np.linspace(-2.0, 4.0)
errors = np.zeros(len(theta_hat_grid))
for i, theta_hat in enumerate(theta_hat_grid):
  errors[i] = mse(x, y, theta_hat)

# Find theta that results in lowest error
best_error = np.min(errors)
theta_hat = theta_hat_grid[np.argmin(errors)]

# Plot results
fig, ax = plt.subplots()
ax.plot(theta_hat_grid, errors, '-o', label='MSE', c='C1')
ax.axvline(theta, color='g', ls='--', label=r"$\theta_{True}$")
ax.axvline(theta_hat, color='r', ls='-', label=r"$\hat{{\theta}}_{MSE}$")
ax.set(
  title=fr"Best fit: $\hat{{\theta}}$ = {theta_hat:.2f}, MSE = {best_error:.2f}",
  xlabel=r"$\hat{{\theta}}$",
  ylabel='MSE')
ax.legend();
```

We can see that our best fit is $\hat{\theta}=1.18$ with an MSE of 1.45. This is quite close to the original true value $\theta=1.2$!

---
# Section 2: Least-squares optimization

*Estimated timing to here from start of tutorial: 20 min*

While the approach detailed above (computing MSE at various values of $\hat\theta$) quickly got us to a good estimate, it still relied on evaluating the MSE value across a grid of hand-specified values. If we didn't pick a good range to begin with, or with enough granularity, we might miss the best possible estimator. Let's go one step further, and instead of finding the minimum MSE from a set of candidate estimates, let's solve for it analytically.

We can do this by minimizing the cost function. Mean squared error is a convex objective function, therefore we can compute its minimum using calculus. Please see video or Bonus Section 1 for this derivation! After computing the minimum, we find that:

\begin{align}
\hat\theta = \frac{\mathbf{x}^\top \mathbf{y}}{\mathbf{x}^\top \mathbf{x}}
\end{align}
where $\mathbf{x}$ and $\mathbf{y}$ are vectors of data points.

This is known as solving the normal equations. For different ways of obtaining the solution, see the notes on [Least Squares Optimization](https://www.cns.nyu.edu/~eero/NOTES/leastSquares.pdf) by Eero Simoncelli.

## Coding Exercise 2: Solve for the Optimal Estimator

In this exercise, you will write a function that finds the optimal $\hat{\theta}$ value using the least squares optimization approach (the equation above) to solve MSE minimization. It shoud take arguments $x$ and $y$ and return the solution $\hat{\theta}$.

We will then use your function to compute $\hat{\theta}$ and plot the resulting prediction on top of the data.

```python
def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.

    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.

  Returns:
    float: the value for theta_hat arrived from minimizing MSE
  """

  ################################################################################
  ## TODO for students: solve for the best parameter using least squares
  # Fill out function and remove
  # raise NotImplementedError("Student exercise: solve for theta_hat using least squares")
  ################################################################################

  # Compute theta_hat analytically
  theta_hat = (np.dot(x,y))/(np.dot(x,x))

  return theta_hat

theta_hat = solve_normal_eqn(x, y)
y_hat = theta_hat * x
plot_observed_vs_predicted(x, y, y_hat, theta_hat)
```

```python
(x.T)@x
(np.dot(x,y))/(np.dot(x,x))

#@ es multiplicacion de matrices
#Cuando multiplico con * lo entiende como un array, multiplica elemento a elemento
```

    1.2084421298265953

```python
#x
x*np.zeros(x.shape)
#(np.transpose(x))*y
```

    array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
           0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D3_ModelFitting/solutions/W1D3_Tutorial1_Solution_7a89ba24.py)

*Example output:*

We see that the analytic solution produces an even better result than our grid search from before, producing $\hat{\theta} = 1.21$ with MSE = 1.43!

---
# Summary

*Estimated timing of tutorial: 30 minutes*

Linear least squares regression is an optimization procedure that can be used for data fitting:

  - Task: predict a value for $y_i$ given $x_i$
  - Performance measure: $\textrm{MSE}$
  - Procedure: minimize $\textrm{MSE}$ by solving the normal equations

**Key point**: We fit the model by defining an *objective function* and minimizing it.

**Note**: In this case, there is an *analytical* solution to the minimization problem and in practice, this solution can be computed using *linear algebra*. This is *extremely* powerful and forms the basis for much of numerical computation throughout the sciences.

# Notation
\begin{align}
x_{i} &\quad \text{input, independent variable}\\
y_{i} &\quad \text{measurement, dependent variable}\\
\mathbf{x} &\quad \text{vector of input values}\\
\mathbf{y} &\quad \text{vector of measurements}\\
\hat{y}_{i} &\quad \text{estimate of dependent variable}\\
\epsilon_{i} &\quad \text{measurement error}\\
\theta &\quad \text{slope parameter}\\
\hat{\theta} &\quad \text{estimated slope parameter}\\
\hat{\theta}_\text{MSE} &\quad \text{slope parameter estimated via the mean squared error}\\
\textrm{MSE} &\quad \text{mean squared error}\\
\end{align}

---
# Bonus

---
## Bonus Section 1: Least Squares Optimization Derivation

We will outline here the derivation of the least squares solution.

We first set the derivative of the error expression with respect to $\theta$ equal to zero,

\begin{align}
\frac{d}{d\theta}\frac{1}{N}\sum_{i=1}^N(y_i - \theta x_i)^2 = 0 \\
\frac{1}{N}\sum_{i=1}^N-2x_i(y_i - \theta x_i) = 0
\end{align}

where we used the chain rule. Now solving for $\theta$, we obtain an optimal value of:

\begin{align}
\hat\theta = \frac{\sum_{i=1}^N x_i y_i}{\sum_{i=1}^N x_i^2}
\end{align}

Which we can write in vector notation as:

\begin{align}
\hat\theta = \frac{\mathbf{x}^\top \mathbf{y}}{\mathbf{x}^\top \mathbf{x}}
\end{align}

This is known as solving the *normal equations*. For different ways of obtaining the solution, see the notes on [Least Squares Optimization](https://www.cns.nyu.edu/~eero/NOTES/leastSquares.pdf) by Eero Simoncelli.