# Jupyter Support

This page is built by the Sphinx's `nbsphinx` extension. The raw file is a Python-kernel Jupyter Notebook file `JupyterSupport.ipynb` under the `subpage` folder of the Simrofy project.

For more details about Jupyter Notebook & Jupyter Lab, you can visit [Jupyter's official website](https://jupyter.org/).

```python
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
```

Jupyter Notebook is famous for its plotting & data visualization along its Markdown-syntax typesetting. Let us have a quick look on these features.

## Plotting

There are some configurations for plotting, which are mentioned by `nbsphinx`.

1. **Figure format**: Since Sphinx will builds all Jupyter notebook into HTML, we probably prefer SVG over PNG figures.
2. **Figure dpi**: May increase the dpi (default is 72) for high resolution screen, especially when the format is PNG.
2. **Random SVG**: Note that the figure output would be stored for the website, so we also need to avoid randomness in SVG generating -- both its content and its filename. Jupyter notebook has a `svg.hashsalt` option to control the "random seed".

We can create a `matplotrc` file in the same folder with our Jupyter notebook file, and write:

```
figure.dpi: 96
figure.figsize: 8, 6
font.size: 14.0
svg.hashsalt: mplsalt
```

The figure format is an option of IPython, so we can't configure it through matplotlib. We need to run it inside our Jupyter notebook:

```python
# The PDF figure format is used when the noteboook is converted by LaTeX
%config InlineBackend.figure_formats = {'svg', 'pdf'}
```

Now we are pleased to draw a simple plot after finishing all these configurations.

```python
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)

# Overwrite the rcParam figsize
fig, ax = plt.subplots(figsize=(8, 5))
ax.plot(x, y, 'b-')
plt.show()
```

## Data frame visualization

```python
# Generate an array data and show its number of rows and columns
mat = np.array([x, y]).T
mat.shape
```

    (100, 2)

```python
df = pd.DataFrame(mat, columns=('x', 'sin(x)'))
df.head()
```

      x
      sin(x)

      0
      0.000000
      0.000000

      1
      0.063467
      0.063424

      2
      0.126933
      0.126592

      3
      0.190400
      0.189251

      4
      0.253866
      0.251148

We can control its display format by:

```python
pd.options.display.float_format = '{:.4f}'.format
```

```python
df.head()
```

      x
      sin(x)

      0
      0.0000
      0.0000

      1
      0.0635
      0.0634

      2
      0.1269
      0.1266

      3
      0.1904
      0.1893

      4
      0.2539
      0.2511

## Markdown syntax

All paragraphs in this Jupyter Notebook are written in Markdown syntax. For example, you can make:

- Italic *italic*,
- Bold **bold**, and
- Literal `literal` font styles.

Also, it's worthnoting that `nbsphinx` provides support for some admonitions.

Warning

This is an *experimental feature*!

Its usage will probably change in the future or it might be removed completely!

Please **strictly follow this format** (including blank lines around div labels and the title "Warning") when you use this admonition feature.

```html
<div class="alert alert-warning">

Warning

This is an *experimental feature*!
Its usage will probably change in the future or it might be removed completely!

</div>
```

Up to current `nbsphinx` version (see below), it only supports "warning" and "info" (i.e. alert-info) admonitions.

```python
import nbsphinx
print(nbsphinx.__version__)
```

    0.7.1

## LaTeX equations and cross-references

The LaTeX equation is natively supported by Jupyter. When the Sphinx builds the Jupyter file into a HTML webpage, it uses the MathJax to render equations. For more details of equation usage in `nbsphinx`, you can visit [nbsphinx: Equations](https://nbsphinx.readthedocs.io/en/0.8.6/markdown-cells.html#Equations).

Please remember to **avoid indentation inside the equation**, otherwise the HTML builds may fail.

Here is an example equation:

\begin{equation}
\int_0^\pi \sin^2 x \,dx = \frac{\pi}{2}
\label{eq:example}
\end{equation}

The text behind it is:

```
\begin{equation}
\int_0^\pi \sin^2 x \,dx = \frac{\pi}{2}
\label{eq:example}
\end{equation}
```

You can also use the asterisk-variant environment to produce equations without auto-numbering on the right.

### Auto-numbering

You can enable auto-numbering for the HTML output by adding following options to your `conf.py`:

```python
# For Sphinx >= 4.0
mathjax3_config = {
    'tex': {'tags': 'ams', 'useLabelIds': True},
}

# For older Sphinx
mathjax_config = {
    'TeX': {'equationNumbers': {'autoNumber': 'AMS', 'useLabelIds': True}},
}
```

### Cross-reference

You can refer the equation if you have given it a `\label{...}` field.

For the equation above, we can refer it by `\eqref{eq:example}` (which gives \eqref{eq:example}, a number in brackets) or `\ref{eq:example}` (\ref{eq:example}, a number without brackets).


```python
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>

```python
from IPython.display import HTML

HTML('''
<a href="{{ site.links.github }}/raw/nist-pages/hackathons/hackathon2/problem2.ipynb"
   download>
<button type="submit">Download Notebook</button>
</a>
''')
```

<a href="{{ site.links.github }}/raw/nist-pages/hackathons/hackathon2/problem2.ipynb"
   download>
<button type="submit">Download Notebook</button>
</a>

```python
from IPython.display import HTML

HTML('''{% include jupyter_benchmark_table.html num="[4]" revision=0 %}''')
```

{% include jupyter_benchmark_table.html num="[4]" revision=0 %}

# Benchmark Problem 4: Linear Elasticity in 3D

The linear elastic energy of a body is

\begin{equation}
E_{\rm elastic}=\frac{1}{2}\int \sigma_{ij}\epsilon_{ij}\,dV=\frac{1}{2}\int C_{ijkl}\epsilon_{ij}\epsilon_{kl}\,dV,
\end{equation}

where $\sigma_{ij}=C_{ijkl}\epsilon_{kl}$ is the stress, $C_{ijkl}$ is the elastic tensor, and $\epsilon_{ij}$ is the strain,
\begin{equation}
\epsilon_{ij}=\frac{1}{2}\left[\frac{\partial u_i}{\partial x_j}+\frac{\partial u_j}{\partial x_i}\right],
\end{equation}
with $u_i$ the displacement field. The indices $i,j,k,l$ run from 1 to 3, and $x_i,\,i=1,2,3$, are Cartesian coordinates; we are using Einstein summation convention so repeated indices are summed over.

The elastic tensor obeys symmetries $C_{ijkl}=C_{jikl}=C_{ijlk}=C_{jilk}$ and $C_{ijkl}=C_{klij}$. These symmetries imply that there are only 21 independent entries in the elastic tensor. Usually Voigt notation is used, in which the four indices $ijkl$ are replaced by two indices $IJ$. The mapping for each pair $ij$ (or $kl$) is $11\to1$, $22\to2$, $33\to3$, $23\to4$ (and $32\to4$), $13\to5$ (and $31\to5$), and $12\to6$ (and $21\to6$). The crystal symmetry may further reduce the number of independent entries. In an orthorombic crystal, there are only nine independent entries, and they are (in Voigt notation) $C_{11}, C_{22}, C_{33}, C_{44}, C_{55}, C_{66}, C_{12}, C_{13}$, and $C_{23}$. The tensor $C_{IJ}$ thus has the form

\begin{equation}
\left(
\begin{matrix}
C_{11} & C_{12} & C_{13} & 0 & 0 & 0\\
C_{12} & C_{22} & C_{23} & 0 & 0 & 0 \\
C_{13} & C_{23} & C_{33} & 0 & 0 & 0 \\
0 & 0 & 0 &              C_{44} & 0 & 0\\
0 & 0 & 0 & 0 & C_{55} & 0\\
0 & 0 & 0 & 0 & 0 & C_{66}
\end{matrix}
\right).
\end{equation}

For tetragonal symmetry, there are six independent entries, $C_{11}, C_{33}, C_{44}, C_{66}, C_{12}$, and $C_{13}$.

Aluminum silicate, Al$_2$SiO$_5$ is a crystal with orthorombic symmetry and unit cell parameters $a=7.738\;\unicode{x212B}$ , $b=7.857\;\unicode{x212B}$, and $c=5.534\;\unicode{x212B}$. The elastic tensor is given by $C_{11}=233.4$, $C_{22}=289.0$, $C_{33}=380.1$, $C_{44}=99.5$, $C_{55}=87.8$, $C_{66}=112.3$; $C_{11}+C_{22}-2C_{12}=233.4$, $C_{11}+C_{33}-2C_{13}=380.9$, and $C_{22}+C_{33}-2C_{23}=506.3$, all in units of GPa.

(a) (Potato in space) What is the equilibrium shape of a 0.0042~$\mu$m$^3$ volume of Al$_2$SiO$_5$ in free space (stress-free boundaries)? Take the surface energy, $\gamma$, to be equal to 200 mJ/m$^2$. The crystalline axes $a$, $b$, and $c$ are aligned with the $x$, $y$, and $z$-axes of a Cartesian lab coordinate system.

This problem can be cast as a phase field problem, where the phase field $\varphi\in[0,1]$ takes the value of 1 in one phase (the "potato"), and a value of 0 in the other (the surrounding vacuum). Thus, the total free energy can be written
\begin{equation}
{\mathcal F}=\int \left[f_{\rm elastic}+\frac{\kappa}{2}|\nabla\varphi|^2+h_0f(\varphi)\right]\,dV,
\end{equation}
with the integral extended over all space,
where
$f(\varphi)$ is a (dimensionless) double-well function
\begin{equation}
f(\varphi)=\varphi^2\left[\varphi-1\right]^2,
\end{equation}
and $h_0$ has the dimension of energy per unit volume. The interface width $W$ between approximately $\varphi=0.1$ and $\varphi=0.9$ in this model is given by $W=2\sqrt{2\kappa/h_0}$, while $\gamma=\sqrt{\kappa h_0/18}$, $\kappa=1.5\gamma W$, and $h_0=12\gamma/W$ (ignoring modification of the phase field order parameter $\varphi$ by the elastic interactions through the interface). Use $\kappa=3\times10^{-9}$~J/m, and
$h_0=2.4\times10^{8}$~J/m$^3$.

We use a simple interpolation of the elastic constants,

\begin{equation}
C_{ijkl}=h(\varphi)C_{ijkl}^{\rm potato},%+\left[1-h(\varphi)\right]C_{ijkl}^{\rm matrix},
\end{equation}

where $h(\varphi)$ is a smooth interpolation function,
\begin{equation}
h(\varphi)=\varphi^3\left[6\varphi^2-15\varphi+10\right],
\end{equation}
that interpolates between $h(\varphi=0)=0$ and $h(\varphi=1)=1$.

Hint: find time-evolution equations for $\varphi$ that monotonically drive the total energy to a minimum while preserving the volume. One way to do this is to set up a Cahn-Hilliard equation for $\varphi$.

(b) (Compressed potato in space) What is the equilibrium shape of a 0.0042 $\mu$m$^3$ volume Al$_2$SiO$_5$ with uniaxial compressive stress of 500~MPa applied in the $z$-direction?  Note that the strain must go to zero far from the "potato."

(c) (Potato in a stew) A volume of Al$_2$SiO$_5$ is embedded coherently in a matrix of tetragonal symmetry with unit cell parameters $a_m=b_m=7.6918\;\unicode{x212B}$ and $c_m=5.5674\;\unicode{x212B}$ such that $a$ and $a_m$, $b$ and $b_m$ and $c$ and $c_m$ are pairwise aligned. The elastic tensor of the matrix is given by $C_{11}=C_{22}=269.0$, $C_{12}=177.0$, $C_{13}=146.0$, $C_{33}=480.0$, $C_{44}=124.0$, and $C_{66}=192.0$, all in units of GPa.

This problem can also be cast as a phase field problem, where now the phase field $\varphi\in[0,1]$ takes the value of 1 in one phase (the inclusion), and a value of 0 in the other (the matrix). The total free energy is again
\begin{equation}
{\mathcal F}=\int \left[f_{\rm elastic}+\frac{\kappa}{2}|\nabla\varphi|^2+h_0f(\varphi)\right]\,dV,
\end{equation}
where
$f(\varphi)$ is given by above.

The interpolation of the elastic constants is now,

\begin{equation}
C_{ijkl}=h(\varphi)C_{ijkl}^{\rm inclusion}+\left[1-h(\varphi)\right]C_{ijkl}^{\rm matrix},
\end{equation}

where again $h(\varphi)$ is a smooth interpolation function given above.

In the elastic energy, the relevant strain is the total strain $\epsilon_{ij}$ minus the local misfit strain $\epsilon^0_{ij}$, so

\begin{equation}
f_{\rm elastic}=\frac{1}{2}C_{ijkl}\left(\epsilon_{ij}-\epsilon^0_{ij}\right)\left(\epsilon_{kl}-\epsilon^0_{kl}\right).
\end{equation}

We will use Vegard's law to determine the local misfit strain, for which we just interpolate the crystallographic misfit strain tensor, $\epsilon^T_{ij}$:

\begin{equation}
\epsilon^0_{ij}=h(\varphi)\epsilon^T_{ij},
\end{equation}

where $h(\varphi)$ is the interpolation function, and $\epsilon^T_{ij}$ is
\begin{equation}
\epsilon^T_{ij}=\left(
\begin{matrix}
\frac{a-a_m}{a_m} & 0 & 0\\
0 & \frac{b-b_m}{b_m} & 0\\
0& 0 & \frac{c-c_c}{c_m}
\end{matrix}
\right).
\end{equation}

Find the equilibrium shape of an isolated inclusion with a volume of 0.0042 $\mu$m$^3$ for $\kappa=3\times10^{-9}$ J/m, and $h_0=2.4\times10^{8}$ J/m$^3$. Use as an initial condition a prolate ellipsoid with $a=c=0.155$ $\mu$m and $b=0.042$ $\mu$m. Because of elastic strain energy and interfacial energy within the system, you will need to increase the intial values of the phase field variable in the matrix and the precipitate (e.g., $\varphi$ in the precipitate $\approx$ 1.05; $\varphi$ in the matrix $\approx 0.05$).  Note that the strain must go to zero far from the "potato".

For parts (a) - (c), present a rendering of the final shape as well as the lengths of the "potato" along the principal axes, and also track the total free energy as function of "time" and plot it.


# Notebook contents:

This notebook contains a lecture. The code for generating plots are found at the of the notebook. Links below.

- [presentation](#Session-1b:)
- [code for plots](#Code-for-plots)

# Session 11:
## Machine learning introduction

*Andreas Bjerre-Nielsen*

## Taking stock

*What have we learned until now?*
- Fundamental data types, functions, containes, loops
- Pandas: DataFrame, Series - these contain basic datatypes
    - A lot of powerful tools in methods/functions
    - E.g. groupby, join/merge
- Scraping: API, HTML, and a lot more

## Some coding advice

- *How do I extract an object from my function?*
    - Print or return?
- Solving complex problems:
    - Solve one thing at a time, start with the most essential that you can do now
- Is joining datasets difficult?
    - Check out [the pandas documentation on merging](https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html)
    - Or [the guide available from Jake Van der Plas](https://jakevdp.github.io/PythonDataScienceHandbook/03.07-merge-and-join.html)

```python
def my_fct(a):
    b = a + 1
    return (b)

c = my_fct(2)
print(c)
```

    3

## Agenda

1. [Math and stats review](#Math-review)
1. [Why machine learing](#Why-machine-learning)
1. [What is machine learning](#Machine-learning-overview)
1. Classification models
    1. [the perceptron](#The-perceptron-model)
    1. [beyond the perceptron](#Beyond-the-perceptron)

## Math review

Vector: 1-d dimensional array of numbers
\begin{align}\boldsymbol{x}=[x_0,x_1,x_2,..]\end{align}

Matrix: 2-d dimensional array of numbers
\begin{eqnarray}\boldsymbol{X}=[
[&x_{00}&,x_{01}&,x_{02}&,..&],\\
[&x_{10}&,x_{11}&,x_{12}&,..&],\\
[&x_{20}&,x_{21}&,x_{22}&,..&],\\
[&...   &,...   &,...   &,..&]]
\end{eqnarray}

## Function fitting
*What does (supervised) machine learning do?*

Suppose we have some data $y$ we want to model/predict from input $x$.

The aim is to find a function $f$ such that the distance between actual values $y$ and predicted values $f(x)$ are minimized.

*What are some examples?*

- Linear form: $y=x\beta$.
- Logistic form: $y=g(x\beta)$

where $x^T\beta=\beta_0+x_1\beta_1+x_2\beta_2+...+x_n\beta_n$ (vector dot product)

# Why machine learning

## Value of modelling
*Why are models useful?*

Models are pursued with differens aims. Suppose we have a linear model, $y=x\beta+\epsilon$.

- Social science:
    - They teach us something about the world.
    - We want to estimate $\hat{\beta}$ and distribution
- Data science:
    - To make optimal future decisions and precise predictions, i.e. $\hat{y}$.

## Model fragility (1)
*What is a polynomial regression?*

- Fitting a curve with an *n-dimenstional polynomial*
- Can fit any "regular" curve ~ Taylor Series Approximation.

## Model fragility (2)
*Suppose we build models of the size of the Danish population, how do polynomial fits perform?*
- We estimate model with data from 1769-1975.

```python
f_pop1
```

## Model fragility (3)
*Which model performs best when we extend the forecasting period from 1975 to now?*

```python
f_pop2
```

## Model fragility (4)
*What happens if we extend the prediction period until 2050? See the fifth order.*

```python
f_pop3
```

## Model fragility (5)
*What trade off do we face in modelling?*

- Making a model that is too simple and does not capture enough of data (`underfitting`)
- Making a model with great fit on estimation data, but poor out-of-sample prediction (`overfitting`)

The goal of machine learning is to find models that minimize these two problems simultaneously.

## Learning ML

- During lectures copy code for see what it does - ***listen*** to me. Write own notes.
- After lecture > understand code details
- Learn with your group - VERY IMPORTANT!

# Machine learning overview

## Machine learning outline for this course

ML: short for machine learning

- Problems: ***supervised*** vs unsupervised
- Linear supervised ML models
    - classification and regression
    - regularization
    - **getting hands dirty with implementing solver**
- Fundamental concepts of ML
    - overfitting, underfitting, model validation
    - model selection and hyperparameters
- Emphasize differences and synergies between ML and statistics
- Brief intro of non-linear models

## What is machine learning
*Can you define machine learning, i.e. ML?*

- Supervised learning
  - Models designed to infer a relationship between input and **labeled** data.
  - We define the `target` as labels in data we wish to model.
      - Example: population as a function of year.
- Unsupervised learning
  - Find patterns and relationships from **unlabeled** data.
  - This may involve clustering, dimensionality reduction and more.
  - Not part of the course.

## Why machine learning
*How might this be useful for social scientists?*

Supervised machine learning is important (elaborated in Lecture 14):
- Improve estimation by validating models (not only theory)
- We can generate new data (impute missing)
- Better predictive models
- Use in hybrid models that leverage machine learning for causal estimation
    - (e.g. causal forest, neural instrumentation)

## Supervised ML problems
*How can we categorize a supervised ML model?*

Suppose we have model $y=g(X\beta)$

We distinguish by type of the `target` variable `y`:
- **regression**: predict a numeric value
- **classification**: distinguish between target categories (non-numeric data)

## Supervised ML problems (2)
*Which one is classification, which one is regression?*

```python
f_identify_question
```

## Supervised ML problems (3)

```python
f_identify_answer
```

## Regression models
*What are examples of regressions models?*

- Example of targets: income, life expectancy, education length (years)

*What is the underlying data of the target, $y$?*

- target is `continuous`

## Classification models
*What are examples of classification models?*

Target, `y`, are categories
- Examples of target: kind of education (linguistics, math), mode of transportation
    - sometimes known as `factor` in statistics
- (work for `str`, `bool`,  `int`, `float` which are then interpreted as categories)

## Example of supervised ML
*Classification or regression?*

We load the titanic data. We select variables and make dummy variables from categorical. We split into target and features.

Target is: ...?

```python
import numpy as np
import pandas as pd
import seaborn as sns

titanic = sns.load_dataset('titanic')
cols = ['survived','class', 'sex', 'sibsp', 'age', 'alone']
titanic_sub = pd.get_dummies(titanic[cols].dropna(), drop_first=True).astype(np.int64)

X = titanic_sub.drop('survived', axis=1)
y = titanic_sub.survived
```

## Definitions

ML lingo and econometric equivalents (in italic)

- `feature` vector, $\textbf{x}_i$, i.e a row of input variables
  - = explanatory *variables* in econometrics
- `weight` vector, $\textbf{w}$, i.e model parameters
  - = *coefficients* in econometrics where denoted $\beta$
- `bias` term, $w_0$, i.e. the model intercept
  - = the *constant* variable in denoted $\beta_0$

# The perceptron model

## The articifial neuron

A real neuron maps stimulus (input) to output.

[Research estimates](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5063692) there are 55–70 billion neurons in the brain.

## The articifial neuron (2)
We are interested in making a decision rule that takes arbitrary input and outputs either positive or negative.

Mathematically we define this map as $\phi: \mathbb{R}^p\rightarrow\{-1, 1\}$.

\begin{align}
\phi(z_i)=\begin{cases}
\hfill1, & z_i>0\\
-1, & z_i\le0
\end{cases}
\end{align}

- `net-input`, $z_i = \underset{~vector\,form}{\underbrace{\boldsymbol{w}^{T}\boldsymbol{x}_i}} = \underset{~expanded\,form}{\underbrace{1\cdot w_0+w_1x_{i,1}+...+w_kx_{i,k}}}$

- `unit step function`, $\phi$, checks if value exceeds threshold

## The articifial neuron (3)
Quiz: what are the input dimensions of the neuron, what is the output dimension?

- Input is the p-dimensional space, $\mathbb{R}^p$.
- Output is binary, either $-1$ or $1$.

## The articifial neuron (4)
*The unit step function (left) and the decision boundary (right)*

## The articifial neuron (5)
*When does the articial neuron work?*

If the two target types are linearly separable:

## The perceptron learning rule (1)
*How do we estimate the model parameters?*

1. initialize the weight with small random number
1. for each training observation, i=1,..,n
  1. compute predicted target, $\hat{y}_i$
  1. update weights $\hat{w}$

## The perceptron learning rule (2)
*How do we compute the predicted target $\hat{y}$?*

We apply a transformation on the net-input :
- single observation, expanded notation:
\begin{align*}
\hat{y}_i= \phi(z_i),\quad z_i=w_0+w_1x_{i,1}+...+w_kx_{i,k}
\end{align*}

- single observation, vector notation:
\begin{align*}
\hat{y}_i= \phi(z_i),\quad z_i=\boldsymbol{w}^{T}\boldsymbol{x}_i
\end{align*}

- multiple observations, matrix notation:
\begin{align*}
\hat{\boldsymbol{y}}= & \phi(\boldsymbol{z}),\quad\boldsymbol{z}=\boldsymbol{X}\boldsymbol{w}
\end{align*}

## The perceptron learning rule (3)
*How do we update weights?*

Weights are updated as follows:
\begin{align*}
w&=w+\Delta w\\
\Delta w&=\eta\cdot(y_i-\phi(z_i))\cdot \textbf{x}_{i}\end{align*}

where $\eta$ is the learning rate, and the first order derivative is:

$$\frac{\partial SSE}{\partial w}=- \textbf{X}^T\textbf{e}$$

## The perceptron learning rule (4)

The computation process

## Implementation in Python (1)
*Let's set some values of input and output*

```python
X = np.random.normal(size=(3, 2)) # feature matrix
y = np.array([1, -1, 1]) # target vector
w = np.random.normal(size=(3)) # weight vector
print('X:\n',X)
print('y:',y)
print('w:',w)
```

    X:
     [[ 1.72431863 -2.03686152]
     [ 1.28450109 -0.40514301]
     [-0.61794548  0.80683709]]
    y: [ 1 -1  1]
    w: [1.26991801 0.53081782 1.16030887]

## Implementation in Python (2)
*How do we compute the errors vectorized?*

```python
# compute net-input
z = w[0] + X.dot(w[1:]) # (w[0]: bias, w[1:]: other weights, X: features)

# unit step-function
positive = z>0 # compute prediction (boolean)
y_hat = np.where(positive, 1, -1)  # convert prediction

# compute errors
e = y - y_hat # compute errors
SSE = e.T.dot(e)
```

## Implementation in Python (3)
*How do we compute the updated weights?*

```python
# learning rate
eta = 0.001

# update weights
update_vars = eta*X.T.dot(e)
update_bias = eta*e.sum()/2
```

## Working with the perceptron (1)
We load the iris data.

```python
iris = sns.load_dataset('iris').iloc[:100] # drop virginica

X = iris.iloc[:, [0, 2]].values # keep petal_length and sepal_length
y = np.where(iris.species=='setosa', 1, -1) # convert to 1, -1

sns.scatterplot(iris.sepal_length, iris.petal_length, hue=iris.species)
```

## Working with the perceptron (2)
*How do we fit the perceptron model?* [perceptron definition](#Code-from-Raschka-2017)

```python
# initialize the perceptron
clf = Perceptron(n_iter=10)
# clf: short for classifier (classification model),
# n_iter: number of times to run through all observations

# fit the perceptron (estimate the model)
# runs 10 iterations of updating the model
clf.fit(X,y)
```

    <ch02.Perceptron at 0x241692d76a0>

## Working with the perceptron (3)
*How can we evaluate the model??*

```python
print('Number of errors: %i' % sum(clf.predict(X)!=y))

# we plot the decisions
plot_decision_regions(X,y,clf)
```

## Working with the perceptron (4)
*How does the model performance change??*

```python
f,ax = plt.subplots(figsize=(12, 4))
ax.set_xticks(range(11))
ax.plot(range(1, len(clf.errors_) + 1), clf.errors_, marker='o')
ax.set_xlabel('Number of iterations')
ax.set_ylabel('Number of errors')
```

# Model validation

## Model validation
*How can we see how our model generalizes?*

We can simulate out-of-sample prediction. How?

- Idea: Use some of our sample for model evaluation.
- Implementation - divide data randomly into two subsets:
    - `training data` for estimation;
    - `test data` for evaluation.
- Note: does not work for time series.

## Model validation (2)
We revert to titanic, `y`: survived, `X`: everything else

```python
print(titanic_sub.head(3))
```

       survived  sibsp  age  alone  class_Second  class_Third  sex_male
    0         0      1   22      0             0            1         1
    1         1      1   38      0             0            0         0
    2         1      0   26      1             0            1         0

We split the data into test and training samples

```python
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.5, random_state=0)
```

# Beyond the perceptron

## Motivation
*What might we change about the perceptron?*

1. Change from updating errors that are binary to continuous
2. Use more than one observation a time for updating

## The activation function (1)
*What else might we use to update errors?*

- The most simple is **no transformation** of the net-input, i.e. $\phi(z_i)=z_i$.

- When we change this from perceptron we call it Adaptive Linear Neuron (**Adaline**).

## The activation function (2)
*How is this different from the Perceptron?*

## The activation function (3)
*Which activation functions can be used?*

- Linear
- Logistic (Sigmoid)
- Unit step, sign

See page 450 in Python for Machine Learning.

## The activation function (4)
*How do Adaline and Logistic regression differ?*

## A new objective (1)
*The update rule in perceptron seems ad hoc, is there a more general way?*

- Yes, we minimize the sum of squared errors (SSE). The SSE for Adaline is:
\begin{align}SSE&=\boldsymbol{e}^{T}\boldsymbol{e}=e_1^2+..+e_n^2\\\boldsymbol{e}&=\textbf{y}-\textbf{X}\textbf{w}\end{align}

*Doesn't the above look strangely familiar?*

- Yes, it is the same objective as OLS. The difference:
    - OLS computes the exact solution with system of equations from first order conditions.
    - We make an approximate solution.

## A new objective (2)
*So how the hell do we make the approximate solution?*

- Two general classes:
  -  We approximate the first order derivative ~ gradient descent (GD)
  -  We approximate both first and second order derivative ~ quasi Newton

- We take gradient descent - much simpler (often faster)

## A new objective (3)
*How does a gradient descent look?*

An algorithm that finds the direction where expected differences are largest. Attempt of satisfying first order condition (FOC).

## A new objective (4)
*What is the first order derivative of SSE wrt. weights in Adaline?*

\begin{align}\frac{\partial SSE}{\partial w}=\textbf{X}^T\textbf{e},\end{align}

*How do we update with GD in Adaline?*

  - Idea: take small steps to approximate the solution.

  - $\Delta w=\eta\textbf{X}^T\textbf{e}=\eta\cdot\textbf{X}^T(\textbf{y}-\hat{\textbf{y}})$

## A new objective (5)
The gradient descent algorithm we just learned uses the whole data.

- Often known as batch gradient descent.

*What might be a smart way of changing (batch) gradient descent?*

- We only use a subset of the data. Two variants:
    - *stochastic gradient descent* (SGD): uses random subset of observations
    - *mini batch*: uses deterministic subset of observations (loop whole dataset)

- Idea: we converge faster by computing update for subset of data
    - Note: we may need a million repetitions.

## Applying logistic regression
*How difficult is it to use `LogisticRegression`?*

Very easy:

```python
from sklearn.linear_model import LogisticRegression

# estimate model on train data, evaluate on test data
clf = LogisticRegression() # note try default values
# solver='lbfgs'
clf.fit(X_train, y_train) # model training
y_hat = clf.predict(X_test)
accuracy = (y_hat==y_test).mean() # model testing
print('Model accuracy is:', np.round(accuracy,3))
```

    Model accuracy is: 1.0

# The end
[Return to agenda](#Agenda)

# Code for plots

```python
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import requests
import seaborn as sns

plt.style.use('ggplot')
%matplotlib inline

SMALL_SIZE = 16
MEDIUM_SIZE = 18
BIGGER_SIZE = 20

plt.rc('font', size=SMALL_SIZE)          # controls default text sizes
plt.rc('axes', titlesize=SMALL_SIZE)     # fontsize of the axes title
plt.rc('axes', labelsize=MEDIUM_SIZE)    # fontsize of the x and y labels
plt.rc('xtick', labelsize=SMALL_SIZE)    # fontsize of the tick labels
plt.rc('ytick', labelsize=SMALL_SIZE)    # fontsize of the tick labels
plt.rc('legend', fontsize=SMALL_SIZE)    # legend fontsize
plt.rc('figure', titlesize=BIGGER_SIZE)  # fontsize of the figure title

plt.rcParams['figure.figsize'] = 10, 4 # set default size of plots
```

### Population plots

```python
%run pop_plots.ipynb
```

### Plots of ML types

```python
%run ../ML_plots.ipynb
```

### Plots from book

```python
import requests
import os
base_url = 'https://raw.githubusercontent.com/rasbt/python-machine-learning-book-2nd-edition/master/code/ch02/'

for filename in ('ch02.py', 'iris.data', 'iris.names.txt'):
    if not os.path.exists(filename):
        response = requests.get(base_url+filename)
        with open(filename,'wb') as f:
            f.write(response.text.encode('utf-8'))

from ch02 import Perceptron, AdalineGD, AdalineSGD, plot_decision_regions
```


Avtor: **Ime in priimek, vpisna številka**

Datum: 1. oktober 2018

*Potrjujem, da sem avtor projektne naloge in da sem vso vsebino pripravil sam. V primeru, da se ugotovi plagiatorstvo se zavedam, da ne bom izpolnjeval pogojev za pristop k izpitu.*

---

# *Navodila za uporabo te predloge*

[Pri oddaji navodila (do kazala) izbrišite]

Predloga je namenjena pripravi poročila individualnega projekta pri predmetih SE (Strojni elementi - 1 ali 2) ter NM (Numerične metode). Poročilo je razdeljeno na vsebinska poglavja, določena pri predmetu SE. Vsako vsebinsko poglavje lahko vsebuje več podpoglavij, celic, namenjenih kodi ali tekstu.

---

Čeprav je cilj medpredmetnega sodelovanja priprava skupnega poročila za oba predmeta, upoštevajte, da *se pri predmetu Strojni elementi vsebine programiranje in numeričnih metod ne preverjajo.* To pomeni, da:

* Mora biti  **postopek reševanja problema jasno razviden iz oddanega dokumenta, brez potrebe po pregledovanju oddane kode.** Celicam s kodo preračuna, ki se nanašajo na reševanje naloge pri SE, naj zato sledi **izpis matematičnega izraza, iz katerega so jasno razvidni ključni deli postopka ter končni rezultat preračuna, v naslednji obliki** (glejte tudi primere na dnu tega dokumenta):

$$\text{iskana veličina = simbolni izraz = izraz z vstavljenimi vrednostmi podatkov = rezultat z enotami}$$

* Je povsem razumljivo, da numerične vsebine in komentarji numeričnih metod v poročilu za SE še niso v svoji končni obliki. Numerične metode uporabljajte kot orodje za boljše reševanje naloge, definirane pri Strojnih elementih, pred oddajo končnega poročila pri NM pa numerični del po potrebi dopolnite in razširite.

---

Za potrebe predmeta NM morate **vsaj enkrat uporabiti vsako od zahtevanih vsebin:**

* *simbolno reševanje*,
* *sistemi linearnih enačb*,
* *interpolacija ali aproksimacija*,
* *iskanje ničel*,
* *integriranje ali odvajanje*,
* *reševanje diferencialnih enačb*.

Ko vsebino iz NM uporabite, to **nakažite v imenu podpoglavja** (glejte primer spodaj).

Poleg ustrezno implementirane in oblikovane kode pri Numeričnih metodah pričakujemo tudi smiselne **komentarje na izbiro, uporabo in rezultate numeričnih postopkov**.

### Kriteriji ocenjevanja pri NM

Ocena individualnega projekta pri predmetu Numerične metode je sestavljena iz:
* *numerična pravilnost* (30%),
* *struktura, urejenost, uporaba modulov, stil kode (docstringi)* (30%),
* *lasten odnos / kreativni dodatek* (30%),
* *pripravljeni testi kode in/ali uporabniški vmesnik* (10%).

### Oddaja pri NM
V obliki zip datoteke z imenom: **`Ime in priimek, vpisna številka.zip`** pošljete na email naslov svojega asistenta.

---

Individualni seminar mora vsebovati **kazalo vsebine** z delujočimi povezavami na poglavja. Najlažje ga prpravite z uporabo razširitve [toc2](https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tree/master/src/jupyter_contrib_nbextensions/nbextensions/toc2). Za namestitev poženite spodnjo celico in nato ponovno poženite `jupyter notebook`:

```python
# Namestitev toc2
# %sx conda install -c conda-forge jupyter_contrib_nbextensions -y
# %sx jupyter contrib nbextension install --user
# %sx jupyter nbextension enable toc2/main
```

Po tem, ko ste zgorni dodatek namestili, ga vključite v poročilo (**korak 1.**), odprete nastavitve kazala (**korak 2.**) in v oknu, ki se odpre, izberete možnost "Add notebook ToC cell" (**korak 3.**)

Kazalo

1  Vsebinsko poglavje 1

1.1  Uvod

1.2  Preračun zobnikov [simbolno reševanje]

2  Vsebinsko poglavje 2

2.1  Uvod

2.2  Analiza laboratorijskih meritev [aproksimacija, interpolacija]

3  Primeri izpisa matematičnih izrazov

3.1  Primer z uporabo SymPy

---

# Vsebinsko poglavje 1

## Uvod

Definicija naloge naj vsebuje kratek opis problema in sliko!

Slika 3.22 iz vira Slavič 2014:

## Preračun zobnikov [simbolno reševanje]

...

---

# Vsebinsko poglavje 2

## Uvod

...

## Analiza laboratorijskih meritev [aproksimacija, interpolacija]

...

---

# Primeri izpisa matematičnih izrazov

Za prikaz postopka reševanj naloge v pročilu SE lahko uporabite LaTeX. Pripravite nize z željeno LaTeX sintakso, pri tem pa upoštevajte:

* Povsod, kjer je v LaTeX sintaksi potrebna poševnica ``\``, to zapišite podvojeno (``\\``). Nekateri znaki z enojno poševnico (``\n``, ``\t`` itd.) imajo namreč poseben pomen.

* Pripravljen niz z LaTeX kodo izpišite z uporabo `Math` in `display` iz modula `IPython.display` (glejte primere spodaj).

Kadar uporabljate f-nize za formatiran izpis, upoštevajte tudi:

* Kjer želite v LaTeX sintaksi znak ``{`` ali `}`, uporabite podvojen znak  (na primer ``}}``), saj so enojni zaviti oklepaji namenjeni formatiranemu izpisu nizov (za vstavljanje vrednosti v formatiran niz pa, kot ponavadi, uporabite enojne oklepaje, na primer ``{F:.2f}``).

```python
from IPython.display import Math, display
import numpy as np
```

```python
a = 2.0001
primer_1 = f'$ \\sqrt{{ {a:.0f} }} = {np.sqrt(a):.10f} $'
display(Math(primer_1))
```

$$ \sqrt{ 2 } = 1.4142489173 $$

```python
b = 10.5
primer_2 = f'$ \\int_{{ a }}^{{ {b} }}{{ f(x) \\, \\text{{ d }}x }} $'
display(Math(primer_2))
```

$$ \int_{ a }^{ 10.5 }{ f(x) \, \text{ d }x } $$

```python
F = 1000 # N
A = 100 # mm^2
rezultat = F/A

primer_3 = f'$ \\sigma = \\frac{{ F }}{{ A }} = \\frac{{ {F} }}{{ {A} }} = {rezultat} \\, \\text{{ MPa }} $'
display(Math(primer_3))
```

$$ \sigma = \frac{ F }{ A } = \frac{ 1000 }{ 100 } = 10.0 \, \text{ MPa } $$

## Primer z uporabo `SymPy`

*Če delate simbolne preračune lahko izpise vmesnih rezultatov v postopku generirate tudi s Sympy, končni izraz pa mora vedno biti v obliki, predpisani zgoraj.*

```python
import sympy as sym
from fractions import Fraction
sym.init_printing()
```

```python
sigma, F, A = sym.symbols('sigma F A')
podatki = {F:1000, A:100}
```

```python
izraz =  F/A
izraz
```

```python
rezultat = izraz.subs(podatki)
```

```python
korak_1 = sym.Eq(izraz, rezultat)
korak_1
```

```python
korak_2 = sym.Eq(sigma, korak_1)
korak_2
```

Končni rezultat:

```python
display(Math(primer_3))
```

$$ \sigma = \frac{ F }{ A } = \frac{ 1000 }{ 100 } = 10.0 \, \text{ MPa } $$

---

pypinm-master/projekt/Individualni projekt - SE in NM.ipynb


```python
#remove cell visibility
from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)
```

Toggle cell visibility here.

## Krmiljenje horizontalnega pomika lunarne sonde

Ta primer prikazuje načrtovanje regulatorja za lateralno pozicijo lunarne sonde, izhajajoč iz dinamičnih enačb sistema.

Sistem je predstavljen na zgornji sliki; navpični spust sistema je upočasnjen z vertikalnim potiskom, ki deluje na sistem s konstantno silo $F$. Horizontalno gibanje dosežemo z rahlim nagibom sonde za kot $\theta$; nagib povzroči lateralno silo, ki je približno enaka $F\theta$. Nagib dosežemo z navorom $T$, ki ga povzročajo vodljive pogonske rakete (maksimalni navor znaša 500 Nm). Nagibni kot more biti znotraj intervala vrednosti $\pm15$, da se prepreči nevarno povečanje hitrosti navpičnega spusta. Merjeni veličini sta lateralni pomik in hitrost, zračni upor zanemarimo, ostali sistemski parametri pa so prikazani v spodnji tabeli:

| Parameter |                       Vrednost |
|-----------|-------------------------------:|
|$m$        |                        1000 kg |
|$J$        |           1000 kg$\text{m}^2$ |
|$F$        |                         1500 N |

Cilj krmilnega sistema je regulacija horizontalnega pomika $z$ ob izpolnjevanju naslednjih zahtev:
1. Maksimalni prenihaj znaša 30%.
2. Čas ustalitve krajši od 15 s (dosežena vrednost izhoda naj se razlikuje od tiste v stacionarnem stanju za 5%).
3. Kot $\theta$ naj bo ves čas znotraj omenjenega intervala vrednosti, ki zagotavlja maksimalno lateralne spremembo v velikost 10 m.
4. Brez odstopka v stacionarnem stanju v odzivu na koračno funkcijo.

Enačbi sistema sta:

\begin{cases}
J\ddot{\theta}=T \\
m\ddot{z}=F\theta
\end{cases}
in ob upoštevanju vektorja stanj $\textbf{x}=[x_1,x_2,x_3,x_4]^T=[z,\dot{z},\theta,\dot{\theta}]^T$ ter vhoda $u=T$, ju lahko preoblikujemo v obliko prostora stanj:

\begin{cases}
\dot{\textbf{x}}=\underbrace{\begin{bmatrix}0&1&0&0 \\ 0&0&F/m&0 \\ 0&0&0&1 \\ 0&0&0&0\end{bmatrix}}_{A}\textbf{x}+\underbrace{\begin{bmatrix}0\\0\\0\\1/J\end{bmatrix}}_{B}u \\ \\
\textbf{y}=\underbrace{\begin{bmatrix}1&0&0&0 \\ 0&1&0&0\end{bmatrix}}_{C}\textbf{x}.
\end{cases}

### Načrtovanje krmilnika
Za dosego ničelnega odstopka v stacionarnem stanju, sistem razširimo z novo spremenljivko stanj $\dot{x_5}=y_1-y_d$, v kateri $y_1$ predstavlja izmerjeno lateralen pomik, $y_d$ pa zahtevan pomik. Razširjen sistem lahko torej zapišemo kot:

\begin{cases}
\dot{\textbf{x}_a}=\underbrace{\begin{bmatrix}0&1&0&0&0 \\ 0&0&F/m&0&0 \\ 0&0&0&1&0 \\ 0&0&0&0&0 \\ 1&0&0&0&0 \end{bmatrix}}_{A_a}\textbf{x}_a+\underbrace{\begin{bmatrix} 0&0\\0&0\\0&0\\1/J&0\\0&-1 \end{bmatrix}}_{B_a}\underbrace{\begin{bmatrix} u\\y_d \end{bmatrix}}_{u_a} \\ \\
\textbf{y}_a=\underbrace{\begin{bmatrix}1&0&0&0&0\\0&1&0&0&0\\0&0&0&0&1\end{bmatrix}}_{C_a}\textbf{x}_a
\end{cases}

Sistem je vodljiv glede na prvi stolpec matrike $B_a$, zato lahko uporabimo metodo razporejanja polov. Z namenom ohranitve spoznavnosti sistema, je v matriki $C$ dodana vrstica, ker je novo stanje $x_5$ znano.

Matrika ojačanj $K_a$, ki ustreza vsem zahtevam je:
$$
K_a=\begin{bmatrix}2225.0&6244.0&13861.0&5275.0&316.0\end{bmatrix}
$$
Poli matrike $(A_a-B_aK_a)$ so tako razporejeni v $-0.28$, $-2.24+2.23i$, $-2.24-2.23i$, $-0.26+0.32i$ in $-0.26-0.32i$.

### Načrtovanje spoznavalnika
Sistem je spoznaven, in ker merimo tri spremenljivke stanj sistema, lahko načrtujemo poenostavljen spoznavalnik stanj (za $\theta$ in $\dot{\theta}$), ki ima naslednjo strukturo:
$$
\dot{\hat{\textbf{v}}}=(A_{11}+L_aA_{21})\hat{\textbf{v}}+(A_{12}+L_aA_{22}-A_{11}L_a-L_aA_{21}L_a)\textbf{y}_a+(B_1+L_aB_2)u_a,
$$
pri čemer velja
$$
T^{-1}A_aT=\begin{bmatrix}A_{11}&A_{12} \\ A_{21}&A_{22}\end{bmatrix},
\quad T^{-1}B_a=\begin{bmatrix}B_1 \\ B_2\end{bmatrix},
\quad \overline{\textbf{x}_a}=T^{-1}\textbf{x}_a=\begin{bmatrix}V \\ C\end{bmatrix}\textbf{x}_a,
\quad V=\begin{bmatrix}0&0&1&0&0 \\ 0&0&0&1&0\end{bmatrix},
\quad \hat{\textbf{x}_a}=\begin{bmatrix}\hat{\textbf{v}}-L_a\textbf{y}_a \\ \textbf{y}_a\end{bmatrix}.
$$

Izbiro lastnih vrednosti spoznavalnika izvedemo tako, da dinamika napake ocene stanj konvergira hitreje kot pa dinamika sistema, določena z zgornjimi zahtevami. Lastni vrednosti matrike $A_{11}+L_aA_{21}$ sta $\lambda_i=-10$rad/s, $i=1,2$ z $$ L_a=\begin{bmatrix}0&-\frac{40}{3}&0 \\ 0&-\frac{200}{3}&0\end{bmatrix} $$

### Kako upravljati s tem interaktivnim primerom?
Opazuj delovanje sistema z načrtovanim regulatorjem in direktno spreminjaj krmilnik in spoznavalnik. Simulacija se vedno začne z začetno napako spoznavalnika.

```python
#Preparatory Cell

%matplotlib notebook
import control as ctrl
import numpy
import sympy as sym
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.patches as patches
import matplotlib.transforms as transforms
import matplotlib.lines as lines

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    def dummychangecallback(self,change):
        pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(ctrl.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        ctrl.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass
```

```python
# Define matrixes

A = numpy.matrix('0 1 0 0; 0 0 1.5 0; 0 0 0 1; 0 0 0 0')
B = numpy.matrix('0;0;0;0.001')
C = numpy.matrix('1 0 0 0; 0 1 0 0')
Aa = numpy.matrix('0 1 0 0 0; 0 0 1.5 0 0; 0 0 0 1 0; 0 0 0 0 0; 1 0 0 0 0')
Ba = numpy.matrix('0 0;0 0;0 0;0.001 0;0 -1')
Ca = numpy.matrix('1 0 0 0 0; 0 1 0 0 0; 0 0 0 0 1')
Ka1 = numpy.matrix('[2225.0, 6244.0, 13861.0, 5275.0, 316.0') #318.9333 835 2012.5 2000 59.6
Ta = (numpy.matrix('0 0 1 0 0; 0 0 0 1 0; 1 0 0 0 0;0 1 0 0 0; 0 0 0 0 1'))**(-1)
Ar = Ta**(-1)*Aa*Ta
Br = Ta**(-1)*Ba
A11 = Ar[0:2,0:2]
A12 = Ar[0:2,2:5]
A21 = Ar[2:5,0:2]
A22 = Ar[2:5,2:5]
B1 = Br[0:2,:]
B2 = Br[2:5,:]
La1 = numpy.matrix([[0, -4*10/3, 0],[0, -3/8*(-4*10/3)**2, 0]])
X0a = numpy.matrix('0;0;0;0;0;0;0;0;0;0;0.002;0.002;0;0;0;0;0;0.002;0.002;0;0;0;0;0')
# X0a = numpy.matrix('0;0;0;0;0')
# V0 = numpy.matrix('0;0')
```

```python
# Define matrixes widget
Kaw = matrixWidget(1,5)
Law = matrixWidget(2,3)
eig1 = matrixWidget(1,1)
eig2 = matrixWidget(2,1)
eig3 = matrixWidget(2,1)
eig4 = matrixWidget(1,1)
eig5 = matrixWidget(1,1)
eig1o = matrixWidget(1,1)
eig2o = matrixWidget(2,1)

Ydw = widgets.FloatSlider(
                         value=10,
                         min=0,
                         max=10.0,
                         step=0.1,
                         description='$y_d$:',
                         disabled=False,
                         continuous_update=False,
                         orientation='horizontal',
                         readout=True,
                         readout_format='.1f',
                        )

# Init matrix widgets
Kaw.setM(Ka1)
Law.setM(La1)
#[-0.6,-0.5-0.35j,-0.5+0.35j,-0.2-0.6j,-0.2+0.6j]
eig1.setM(numpy.matrix([-0.28]))
eig2.setM(numpy.matrix([[-2.24],[-2.23]]))
eig3.setM(numpy.matrix([[-0.26],[-0.32]]))
eig4.setM(numpy.matrix([-1]))
eig5.setM(numpy.matrix([-1]))
eig1o.setM(numpy.matrix([-10]))
eig2o.setM(numpy.matrix([[-10],[0]]))
```

```python
# Support functions
# Simulation function
def simulation(Aa, Baa, Ca, A11, A12, A21, A22, B1, B2, La, Ka, Ta):
    Aa, Baa, Ca = sym.Matrix(Aa), sym.Matrix(Baa), sym.Matrix(Ca)
    A11, A12, A21, A22 = sym.Matrix(A11), sym.Matrix(A12), sym.Matrix(A21), sym.Matrix(A22)
    B1, B2 = sym.Matrix(B1), sym.Matrix(B2)
    La, Ka = sym.Matrix(La), sym.Matrix(Ka)
    Ta = sym.Matrix(Ta)
    sysS = sss(Aa, Baa, Ca, sym.zeros(3,2))
    sysX = sss(Aa, Baa, sym.eye(5), sym.zeros(5,2))
    sysO1 = sss((A11+La*A21), (B1+La*B2).row_join(A12+La*A22-A11*La-La*A21*La), sym.eye(2), sym.zeros(2,5))
    sysO2 = ctrl.append(sysO1, sysS)
    sysO3 = ctrl.connect(sysO2, [[3, 3], [4, 4], [5, 5]], [1, 2, 6, 7], [1, 2, 3, 4, 5])
    sysO = sss(sysO3.A,
               sysO3.B*sym.eye(2).col_join(sym.eye(2)),
               Ta*(sym.eye(2).row_join(-La)).col_join(sym.zeros(3, 2).row_join(sym.eye(3)))*sysO3.C,
               sym.zeros(5,2))
    sysU = sss(sysO.A, sysO.B, -Ka*sysO.C, sym.zeros(1,2))
    sysT = ctrl.append(sysS, sysX, sysO, sysU)
    sysT1 = ctrl.connect(sysT, [[1, 14], [3, 14], [5, 14], [7, 14]], [2, 4, 6, 8], [i for i in range(1, 15)])
    sys = sss(sysT1.A, sysT1.B*sym.Matrix([1, 1, 1, 1]), sysT1.C, sym.zeros(14, 1))
    return sys

# check functions
def eigen_choice(selc,selo):
    if selc == 'brez kompleksnih lastnih vrednosti':
        eig2.children[1].children[0].disabled = True
        eig3.children[1].children[0].disabled = True
        eig3.children[0].children[0].disabled = False
        eig4.children[0].children[0].disabled = False
        eig5.children[0].children[0].disabled = False
        eigc = 0
    if selc == 'dve kompleksni lastni vrednosti':
        eig2.children[1].children[0].disabled = False
        eig3.children[1].children[0].disabled = True
        eig3.children[0].children[0].disabled = True
        eig4.children[0].children[0].disabled = False
        eig5.children[0].children[0].disabled = False
        eigc = 2
    if selc == 'štiri kompleksne lastne vrednosti':
        eig2.children[1].children[0].disabled = False
        eig3.children[1].children[0].disabled = False
        eig3.children[0].children[0].disabled = False
        eig4.children[0].children[0].disabled = True
        eig5.children[0].children[0].disabled = True
        eigc = 4
    if selo == 'brez kompleksnih lastnih vrednosti':
        eig1o.children[0].children[0].disabled = False
        eig2o.children[1].children[0].disabled = True
        eigo = 0
    if selo == 'dve kompleksni lastni vrednosti':
        eig1o.children[0].children[0].disabled = True
        eig2o.children[1].children[0].disabled = False
        eigo = 2
    return (eigc, eigo)

def method_choice(selm):
    if selm == 'Nastavi Ka in La':
        method = 1
        selc.disabled = True
        selo.disabled = True
    if selm == 'Nastavi lastne vrednosti':
        method = 2
        selc.disabled = False
        selo.disabled = False
    return method

# Animation functions
def fun_animation(index):
    global Ydw, yout, T
    yd = Ydw.value
    frame = 1

    linez.set_data(T[0:index*frame],yd*yout[0][0:index*frame])
    linezv.set_data(T[0:index*frame],yd*yout[1][0:index*frame])
    lined.set_data(T,[yd for i in range(0,len(T))])
    lineu.set_data(T[0:index*frame],yd*yout[13][0:index*frame])
    linelimu1.set_data(T,[500 for j in range(0,len(T))])
    linelimu2.set_data(T,[-500 for j in range(0,len(T))])
    linethetaest.set_data(T[0:index*frame],yd*yout[10][0:index*frame]*180/numpy.pi)
    linetheta.set_data(T[0:index*frame],yd*yout[5][0:index*frame]*180/numpy.pi)

    rotation_transform.clear().translate(yd*yout[0][index*frame]*numpy.cos(float(yd*yout[6][index*frame])), yd*yout[0][index*frame]*numpy.sin(float(yd*yout[6][index*frame]))).rotate(float(-yd*yout[6][index*frame]))

    return (linez,linezv,lined,lineu,linelimu1,linelimu2,linethetaest,linetheta)

def anim_init():
    linez.set_data([], [])
    linezv.set_data([], [])
    lined.set_data([], [])
    lineu.set_data([], [])
    linelimu1.set_data([], [])
    linelimu2.set_data([], [])
    linethetaest.set_data([], [])
    linetheta.set_data([], [])
    return (linez,linezv,lined,lineu,linelimu1,linelimu2,linethetaest,linetheta)

```

```python
# Main cell
# Data
global yd, T, yout
yd = 10.
T = []
yout = []

# Figures
fig = plt.figure(num='Simulacija krmiljenja horizontalnega pomika lunarne sonde')
fig.set_size_inches((9.8, 6))
fig.set_tight_layout(True)

ax0 = fig.add_subplot(221)
ax0.set_title('Lunarna sonda')
ax0.set_xlim(-12,12)
ax0.set_ylim(-4,4)
ax0.grid()
# ax0.axis('off')

ax1 = fig.add_subplot(222)
linez = ax1.plot([],[])[0]
linezv = ax1.plot([],[])[0]
lined = ax1.plot([],[])[0]
ax1.set_title('Lateralni pomik in hitrost')
ax1.set_xlabel('$t$ [s]')
ax1.set_ylabel('y [m], $\dot y$ [m/s]')
ax1.set_xlim([0,17])
ax1.axvline(x=0,color='black',linewidth=0.8)
ax1.axhline(y=0,color='black',linewidth=0.8)
ax1.grid()
ax1.legend(['Lateralni pomik','Lateralna hitrost','Zahtevana vrednost'])

ax2 = fig.add_subplot(223)
lineu = ax2.plot([],[])[0]
linelimu1 = ax2.plot([],[],'r')[0]
linelimu2 = ax2.plot([],[],'r')[0]
ax2.set_title('Vhodni navor T')
ax2.set_xlabel('$t$ [s]')
ax2.set_ylabel('$T$ [Nm]')
ax2.set_xlim([0,17])
ax2.axvline(x=0,color='black',linewidth=0.8)
ax2.axhline(y=0,color='black',linewidth=0.8)
ax2.grid()
ax2.legend(['T','Limit'])

ax3 = fig.add_subplot(224)
linethetaest = ax3.plot([],[])[0]
linetheta = ax3.plot([],[])[0]
ax3.set_title(r'$\theta_{est}$ vs $\theta$')
ax3.set_xlabel('$t$ [s]')
ax3.set_ylabel(r'$\theta$ [deg]')
ax3.axvline(x=0,color='black',linewidth=0.8)
ax3.axhline(y=0,color='black',linewidth=0.8)
ax3.set_xlim([0,17])
ax3.grid()

# Patches
rotation_transform = transforms.Affine2D()
circle = patches.Circle((0, 0.6), fill=True, radius=0.5, ec='black', fc='gray', lw=1, zorder=20,
                        transform=rotation_transform + ax0.transData)
rect = patches.Rectangle((-1, -0.4), 2, 0.5, fill=True, ec='black', fc='gray', lw=1, zorder=20,
                         transform=rotation_transform + ax0.transData)
poly = patches.Polygon(numpy.stack(([-0.25, -0.15, 0.15, 0.25], [-0.8, -0.4, -0.4, -0.8])).T,
                       closed=True, fill=True, ec='black', fc='black', lw=1, zorder=20,
                       transform=rotation_transform + ax0.transData)
lleg = patches.Rectangle((-1, -1.2), 0.05, 1, angle=-15, fill=True, ec='black', fc='black', lw=1, zorder=10,
                         transform=rotation_transform + ax0.transData)
rleg = patches.Rectangle((1, -1.2), 0.05, 1, angle=15, fill=True, ec='black', fc='black', lw=1, zorder=10,
                         transform=rotation_transform + ax0.transData)
lfoot = patches.Rectangle((-1.1, -1.2), 0.2, 0.05, fill=True, ec='black', fc='black', lw=1, zorder=20,
                         transform=rotation_transform + ax0.transData)
rfoot = patches.Rectangle((0.9, -1.2), 0.2, 0.05, fill=True, ec='black', fc='black', lw=1, zorder=20,
                         transform=rotation_transform + ax0.transData)
ax0.add_patch(circle)
ax0.add_patch(rect)
ax0.add_patch(poly)
ax0.add_patch(lleg)
ax0.add_patch(rleg)
ax0.add_patch(lfoot)
ax0.add_patch(rfoot)
plt.show()

# Functions
def main_function(Ka,La,Ydw,eig1,eig2,eig3,eig4,eig5,eig1o,eig2o,selm,selc,selo,DW):
    global T, yout, yd, Aa, Ba, A11, A21
    method = method_choice(selm)
    eigc, eigo = eigen_choice(selc,selo)
    yd = Ydw
    ax1.set_ylim([-0.1*yd,yd*1.5])
    ax2.set_ylim([-51*yd,51*yd])
    ax3.set_ylim([-15,15])

    if method == 1: #Setted matrix gain
        sol = numpy.linalg.eig((Aa-Ba[:,0]*Ka))
        print('Lastne vrednosti Aa so: '+str(round(sol[0][0],3))+', '+str(round(sol[0][1],3))+', '+str(round(sol[0][2],3))+', '+str(round(sol[0][3],3))+' in '+str(round(sol[0][4],3)))
        sol = numpy.linalg.eig(A11+La*A21)
        print('Lastni vrednosti A11+La*A21 sta: '+str(round(sol[0][0],3))+' in '+str(round(sol[0][1],3)))
        sys = simulation(Aa, Ba, Ca, A11, A12, A21, A22, B1, B2, La, Ka, Ta)
        T = numpy.linspace(0, 17, 100)
        T, yout = ctrl.step_response(sys, T, X0a)
    if method == 2: #Setted eigenvalues
        if eigc == 0:
            Ka = ctrl.acker(Aa, Ba[:,0], [eig1[0,0], eig2[0,0], eig3[0,0], eig4[0,0], eig5[0,0]])
            Kaw.setM(Ka)
        if eigc == 2:
            Ka = ctrl.acker(Aa, Ba[:,0], [eig1[0,0], numpy.complex(eig2[0,0],eig2[1,0]), numpy.complex(eig2[0,0],-eig2[1,0]), eig4[0,0], eig5[0,0]])
            Kaw.setM(Ka)
        if eigc == 4:
            Ka = ctrl.acker(Aa, Ba[:,0], [eig1[0,0], numpy.complex(eig2[0,0],eig2[1,0]), numpy.complex(eig2[0,0],-eig2[1,0]), numpy.complex(eig3[0,0],eig3[1,0]), numpy.complex(eig3[0,0],-eig3[1,0])])
            Kaw.setM(Ka)
        if eigo == 0:
            La = numpy.matrix([[0, 2*eig1o[0,0]/3 + 2*eig2o[0,0]/3, 0], [0, -2*eig1o[0,0]*eig2o[0,0]/3, 0]])
            Law.setM(La)
        if eigo == 2:
            La = numpy.matrix([[0, 2*numpy.complex(eig2o[0,0],eig2o[1,0])/3 + 2*numpy.complex(eig2o[0,0],-eig2o[1,0])/3, 0], [0, -2*numpy.complex(eig2o[0,0],eig2o[1,0])*numpy.complex(eig2o[0,0],-eig2o[1,0])/3, 0]])
            Law.setM(La)
        sol = numpy.linalg.eig((Aa-Ba[:,0]*Ka))
        print('Lastne vrednosti Aa so: '+str(round(sol[0][0],3))+', '+str(round(sol[0][1],3))+', '+str(round(sol[0][2],3))+', '+str(round(sol[0][3],3))+' in '+str(round(sol[0][4],3)))
        sol = numpy.linalg.eig(A11+La*A21)
        print('Lastni vrednosti A11+La*A21 sta: '+str(round(sol[0][0],3))+' in '+str(round(sol[0][1],3)))
        sys = simulation(Aa, Ba, Ca, A11, A12, A21, A22, B1, B2, La, Ka, Ta)
        T = numpy.linspace(0, 17, 100)
        T, yout = ctrl.step_response(sys, T, X0a)

ani = animation.FuncAnimation(fig, fun_animation, init_func=anim_init, frames=100, repeat=True, interval=170, blit=True)

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

# Define type of method
selm = widgets.Dropdown(
    options= ['Nastavi Ka in La', 'Nastavi lastne vrednosti'],
    value= 'Nastavi Ka in La',
    description='',
    disabled=False
)

# Define the number of complex eigenvalues for the controller
selc = widgets.Dropdown(
    options= ['brez kompleksnih lastnih vrednosti', 'dve kompleksni lastni vrednosti', 'štiri kompleksne lastne vrednosti'],
    value= 'štiri kompleksne lastne vrednosti',
    description='Aa:',
    disabled=False
)

# Define the number of complex eigenvalues for the observer
selo = widgets.Dropdown(
    options= ['brez kompleksnih lastnih vrednosti', 'dve kompleksni lastni vrednosti'],
    value= 'brez kompleksnih lastnih vrednosti',
    description='Aobs:',
    disabled=False
)

alltogether = widgets.VBox([
    widgets.HBox([
        selm,
        selc,
        selo
    ]),
    widgets.Label('',border=3),
    widgets.HBox([
        widgets.Label('Ka:',border=3),
        Kaw,
        widgets.Label('',border=3),
        widgets.Label('',border=3),
        widgets.Label('La:',border=3),
        Law
    ]),
    widgets.Label('',border=3),
    widgets.HBox([
        widgets.Label('Aa\'s eigs:',border=3),
        eig1, eig2, eig3, eig4, eig5,
        widgets.Label('',border=3),
        widgets.Label('',border=3),
        widgets.Label('Aobs\'s eigs:',border=3),
        eig1o, eig2o
    ]),
    widgets.Label('',border=3),
    widgets.HBox([
        Ydw,
        widgets.Label('',border=3),
        widgets.Label('',border=3),
        widgets.Label('',border=3),
        START
    ])
])

out = widgets.interactive_output(main_function,{'Ka':Kaw, 'La':Law, 'Ydw':Ydw, 'eig1':eig1, 'eig2':eig2,
                                                'eig3':eig3, 'eig4':eig4, 'eig5':eig5,
                                                'eig1o':eig1o, 'eig2o':eig2o,
                                                'selm':selm, 'selc':selc, 'selo':selo, 'DW':DW})
display(out, alltogether)
```

    <IPython.core.display.Javascript object>

    Output()

    VBox(children=(HBox(children=(Dropdown(options=('Nastavi Ka in La', 'Nastavi lastne vrednosti'), value='Nastav…

```python

```


# Models from FNPF

# Purpose
Create roll decay models from the simulation results from FNPF (with/without viscous damping). save these models into pkl files.

# Methodology
* Load
* Cut
* Fit
* save

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

import sys
sys.path.append("../../")

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
import copy
from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import src.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
import shipflowmotionshelpers.shipflowmotionshelpers as helpers
import src.visualization.visualize as visualize
import scipy
from copy import deepcopy
import joblib
```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 463 ('figure.dpi       : 100        ## figure dots per inch')

```python
from rolldecayestimators import measure
from src.helpers import get_ikeda, calculate_ikeda, get_estimator_variation, get_data_variation , get_variation, hatify

from notebook_helpers import load_time_series_fnpf
```

## Load data from FNPF:

```python
df_parameters = pd.read_csv('../../data/processed/roll decay KVLCC2/fnpf_parameters.csv', index_col=0)
df_parameters.head()
```

      B
      IXY
      IXZ
      IYY
      IYZ
      IZZ
      KXX
      KXY
      KYY
      KZZ
      LPP
      S
      V
      XCG
      YCG
      ZCG
      accx
      acto
      b1c
      b1cr
      b1l
      b1q
      b2c
      b2cr
      b2l
      b2q
      b3c
      b3cr
      b3l
      b3q
      b4c
      b4cr
      b4l
      b4q
      b5c
      b5cr
      b5l
      b5q
      b6c
      b6cr
      b6l
      b6q
      bdens
      body
      clev
      clevel
      curv
      dens
      densi
      dofa
      dofactor
      dopa
      dopadding
      dopo
      dopower
      down
      downs
      downst
      dt
      file
      file_path_ts
      fn
      form
      free
      gravi
      heave
      heel
      hull
      inter
      k1nd
      k2nd
      k3nd
      k4nd
      k5nd
      k6nd
      kxx
      kyy
      leve
      level
      lpp
      m
      maxti
      maxtime
      mesh
      name
      npxp
      npyp
      nstep
      nthr
      pitch
      powe
      power
      refle
      reflen
      rn
      roll
      rud1
      rud2
      rud3
      rud4
      rud5
      rud6
      rudt
      side
      slim
      stre
      strength
      surge
      sway
      ta
      tf
      tfnhi
      tfnhigh
      tfnlo
      tfnlow
      titl
      title
      trim
      upst
      upstr
      vm_s
      wlin
      wlme
      yaw
      ymax
      ymin
      zcg
      conv
      encounters
      id

      kvlcc2_rolldecay_0kn
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.050
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.00000
      0.00000
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      50.0
      50.0
      2.0
      2.0
      2.0
      2.0
      1.00
      -5.0
      1.00
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.472020e-07
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.000000e+00
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      180.0
      180.0
      0.000000e+00
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.957280e+00
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.00
      30.0
      0.50
      0.50
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.00
      5.0
      0.000001
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21338.0

      kvlcc2_rolldecay_15-5kn_const_large2
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.00000
      0.00000
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      1.0
      1.0
      2.0
      2.0
      1.0
      1.0
      0.70
      -5.0
      0.70
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.000000e+00
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.000000e+00
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.70
      30.0
      0.05
      0.05
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.70
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

      kvlcc2_rolldecay_15-5kn_ikeda_dev
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.050
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      6.07217
      2.74371
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00004
      1000.0
      1000.0
      0.0
      0.0
      2.0
      2.0
      1.0
      1.0
      0.25
      -2.0
      0.25
      NaN
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.20
      0.3
      9.80665
      0.0
      0.0
      0.002
      1.000000e-07
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      600.0
      600.0
      1.000000e-07
      TRAN
      24.0
      24.0
      30.0
      6.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.25
      30.0
      1.00
      1.00
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.25
      2.0
      0.966976
      0.000001
      0.000001
      0.0
      2.0
      -2.0
      0.2735
      0.0001
      0.0
      21340.0

```python
mask_visc = (df_parameters[['b4l','b4q']] > 0).any(axis=1)
df_parameters.loc[mask_visc]
```

      B
      IXY
      IXZ
      IYY
      IYZ
      IZZ
      KXX
      KXY
      KYY
      KZZ
      LPP
      S
      V
      XCG
      YCG
      ZCG
      accx
      acto
      b1c
      b1cr
      b1l
      b1q
      b2c
      b2cr
      b2l
      b2q
      b3c
      b3cr
      b3l
      b3q
      b4c
      b4cr
      b4l
      b4q
      b5c
      b5cr
      b5l
      b5q
      b6c
      b6cr
      b6l
      b6q
      bdens
      body
      clev
      clevel
      curv
      dens
      densi
      dofa
      dofactor
      dopa
      dopadding
      dopo
      dopower
      down
      downs
      downst
      dt
      file
      file_path_ts
      fn
      form
      free
      gravi
      heave
      heel
      hull
      inter
      k1nd
      k2nd
      k3nd
      k4nd
      k5nd
      k6nd
      kxx
      kyy
      leve
      level
      lpp
      m
      maxti
      maxtime
      mesh
      name
      npxp
      npyp
      nstep
      nthr
      pitch
      powe
      power
      refle
      reflen
      rn
      roll
      rud1
      rud2
      rud3
      rud4
      rud5
      rud6
      rudt
      side
      slim
      stre
      strength
      surge
      sway
      ta
      tf
      tfnhi
      tfnhigh
      tfnlo
      tfnlow
      titl
      title
      trim
      upst
      upstr
      vm_s
      wlin
      wlme
      yaw
      ymax
      ymin
      zcg
      conv
      encounters
      id

      kvlcc2_rolldecay_15-5kn_ikeda_dev
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.05
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      6.07217
      2.74371
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00004
      1000.0
      1000.0
      0.0
      0.0
      2.0
      2.0
      1.0
      1.0
      0.25
      -2.0
      0.25
      NaN
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      0.142341
      0.2
      0.3
      9.80665
      0.0
      0.0
      0.002
      1.000000e-07
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      600.0
      600.0
      1.000000e-07
      TRAN
      24.0
      24.0
      30.0
      6.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3826600.0
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.25
      30.0
      1.0
      1.0
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.25
      2.0
      0.966976
      0.000001
      0.000001
      0.0
      2.0
      -2.0
      0.2735
      0.0001
      0.0
      21340.0

```python
df_parameters.loc[~mask_visc]
```

      B
      IXY
      IXZ
      IYY
      IYZ
      IZZ
      KXX
      KXY
      KYY
      KZZ
      LPP
      S
      V
      XCG
      YCG
      ZCG
      accx
      acto
      b1c
      b1cr
      b1l
      b1q
      b2c
      b2cr
      b2l
      b2q
      b3c
      b3cr
      b3l
      b3q
      b4c
      b4cr
      b4l
      b4q
      b5c
      b5cr
      b5l
      b5q
      b6c
      b6cr
      b6l
      b6q
      bdens
      body
      clev
      clevel
      curv
      dens
      densi
      dofa
      dofactor
      dopa
      dopadding
      dopo
      dopower
      down
      downs
      downst
      dt
      file
      file_path_ts
      fn
      form
      free
      gravi
      heave
      heel
      hull
      inter
      k1nd
      k2nd
      k3nd
      k4nd
      k5nd
      k6nd
      kxx
      kyy
      leve
      level
      lpp
      m
      maxti
      maxtime
      mesh
      name
      npxp
      npyp
      nstep
      nthr
      pitch
      powe
      power
      refle
      reflen
      rn
      roll
      rud1
      rud2
      rud3
      rud4
      rud5
      rud6
      rudt
      side
      slim
      stre
      strength
      surge
      sway
      ta
      tf
      tfnhi
      tfnhigh
      tfnlo
      tfnlow
      titl
      title
      trim
      upst
      upstr
      vm_s
      wlin
      wlme
      yaw
      ymax
      ymin
      zcg
      conv
      encounters
      id

      kvlcc2_rolldecay_0kn
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.050
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      50.0
      50.0
      2.0
      2.0
      2.0
      2.0
      1.0
      -5.0
      1.0
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.472020e-07
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      180.0
      180.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.957280e+00
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      30.0
      0.50
      0.50
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.0
      5.0
      0.000001
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21338.0

      kvlcc2_rolldecay_15-5kn_const_large2
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      1.0
      1.0
      2.0
      2.0
      1.0
      1.0
      0.7
      -5.0
      0.7
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.7
      30.0
      0.05
      0.05
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.7
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

## Also load meta data from corresponding model tests

```python
df_rolldecays = pd.read_csv('../../data/processed/roll decay KVLCC2/model_test_parameters.csv', index_col=0)
df_rolldecays.head()
```

      project_number
      series_number
      run_number
      test_number
      model_number
      ship_name
      loading_condition_id
      ascii_name
      ship_speed
      comment
      file_path_ascii
      file_path_ascii_temp
      file_path_log
      file_path_hdf5
      date
      test_type
      facility
      angle1
      angle2
      Körfallstyp
      name
      lcg
      kg
      gm
      CW
      TF
      TA
      BWL
      KXX
      KZZ
      BTT1
      CP
      Volume
      A0
      RH
      scale_factor
      lpp
      beam
      ABULB
      BKX
      TWIN
      DCLR
      VDES
      RHBL
      ASKEG
      PD
      ARH
      CFP
      AIX
      PDTDES
      RTYPE
      SFP
      BKL
      BKB
      PROT
      D
      LSKEG
      RR
      XSKEG
      NDES
      AR
      BR
      BRA
      IRUD
      PTYPE
      XRUD
      AI
      HSKEG
      RSKEG
      LOA
      ship_type_id
      rho
      g
      paper_name

      id

      21337
      40178362
      1
      94
      1
      M5057-01-A
      M5057-01-A
      166
      94.0
      0.0
      Roll decay, 0 kn
      NaN
      NaN
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      2018-04-03
      roll decay
      MDL
      NaN
      NaN
      NaN
      20.8
      11.2672
      18.6
      5.73
      NaN
      20.8
      20.8
      NaN
      23.2
      80.0
      NaN
      NaN
      312653.0
      0.99538
      NaN
      68.0
      320.0
      58.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1000
      9.81
      1

      21338
      40178362
      1
      95
      1
      M5057-01-A
      M5057-01-A
      166
      95.0
      0.0
      Roll decay, 0 kn
      NaN
      NaN
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      2018-04-03
      roll decay
      MDL
      NaN
      NaN
      NaN
      20.8
      11.2672
      18.6
      5.73
      NaN
      20.8
      20.8
      NaN
      23.2
      80.0
      NaN
      NaN
      312653.0
      0.99538
      NaN
      68.0
      320.0
      58.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1000
      9.81
      2

      21340
      40178362
      1
      97
      1
      M5057-01-A
      M5057-01-A
      166
      97.0
      15.5
      Roll decay, 15.5 kn
      NaN
      NaN
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      \\sspa.local\gbg\LABmeasuredataMDL\40178362\00...
      2018-04-04
      roll decay
      MDL
      NaN
      NaN
      NaN
      20.8
      11.2672
      18.6
      5.73
      NaN
      20.8
      20.8
      NaN
      23.2
      80.0
      NaN
      NaN
      312653.0
      0.99538
      NaN
      68.0
      320.0
      58.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      1000
      9.81
      3

## Load time series from FNPF

```python
time_series = load_time_series_fnpf(names=df_parameters.index)
```

```python
def find_t_max(X, phi_max):

    Xs = X.copy()
    Xs.index = pd.TimedeltaIndex(X.index, unit='s')
    X_ = Xs.abs().resample('5S').max()
    X_.index = X_.index.total_seconds()
    mask = X_['phi'] < phi_max

    X_small = X_.loc[mask]
    if len(X_small) > 0:
        t_max = X_small.iloc[0].name
    else:
        t_max = X.index[-1]

    return t_max
```

```python
time_series_raw = deepcopy(time_series)

phi_max = np.deg2rad(3.5)

for key,df in time_series.items():

    if df.mean().abs()['V1'] > 0.01:

        phi1d_limit = 10**-2
        index0 = (df['phi1d'].abs() > phi1d_limit).argmax()
        X = df.iloc[index0:].copy()
        t_max = find_t_max(X=X, phi_max=phi_max)
        mask = X.index <= t_max
        X = X.loc[mask]

        time_series[key] = X
```

## Cutting

```python
for key, df in time_series.items():

    fig,ax = plt.subplots()
    fig.set_size_inches(20,4)
    time_series_raw[key].plot(y='phi', ax=ax)
    df.plot(y='phi', ax=ax)
    ax.set_title(key)
```

## Fitting

```python
models = OrderedDict()

for key,X in time_series.items():

    pre_model= EstimatorQuadraticB(fit_method='derivation')
    pre_model.fit(X)

    model_motions = EstimatorQuadraticB(p0=pre_model.parameters)

    try:
        model_motions.fit(X=X)
    except scipy.linalg.LinAlgError:
        model_motions.fit(X=X)  # Retry

    if pre_model.score() > model_motions.score():
        model_motions = pre_model

    models[key] = model_motions
```

## Genereate results

```python
for key, model in models.items():

    parameters = df_parameters.loc[key]

    row = df_rolldecays.loc[parameters.id]

    scale_factor = row.scale_factor
    meta_data = {
        'Volume' : row.Volume/(scale_factor**3),
        'rho' : row.rho,
        'g' : row.g,
        'GM' : row.gm/scale_factor,
        }

    model.result_for_database(meta_data=meta_data)  # Results are stored in object

    if not 'B_3' in model.results:
        model.results['B_3'] = 0

```

## Saving models

```python
for key,model in models.items():

    # Saving the model as a pipline:
    pipeline = Pipeline(steps=[('estimator',model)])

    joblib.dump(pipeline, '../../models/%s.pkl' % key)

```

```python
for key,model in models.items():

    model.plot_damping()

```


# Template
The first heading is moved to the title in PDF export.

```
# numpy
import numpy as np

# import sympy and configure latex output
import sympy
sympy.init_printing(use_unicode=True)
from IPython.display import Math

# import and configure matplotlib
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
## use Scalable Vector Graphics (SVG) format for figures
## remember to employ rasterized option when plotting large
## datasets to keep figure size down
%config InlineBackend.figure_format = "svg"
## load matplotlib inline integration for base16 ipython notebook
## themes
## https://github.com/benjaminaschultz/base16-ipython-matplotlibrc
try:
    %load_ext base16_mplrc
    %base16_mplrc
except:
    pass
mpl.rcParams.update({
    'figure.dpi'        : 300,
    'savefig.dpi'       : 300,
    'grid.color'        : mpl.rcParams['xtick.color'],
})

# load ipython cython magic
%load_ext cythonmagic
```

                         Could not detect base-16 ipython notebook theme. Download base16 theme notebook theme
                         from https://github.com/nsonnad/base16-ipython-notebook . Using 'default' theme.

                         Could not detect base-16 ipython notebook theme shade. Download base16 theme notebook themes
                         from https://github.com/nsonnad/base16-ipython-notebook . Using 'default' theme.
    Setting plotting theme to default-light. Palette available in b16_colors

> *[This work](http://notebooks.asorge.de) is licensed under a [Creative
Commons Attribution 4.0 International
License](http://creativecommons.org/licenses/by/4.0/).*