```python
!pip install -q lxml
```

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
```

```python
import requests
import pandas as pd
```

```python
 %pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

# Motivation

I have downloaded prices of flats in Wrocław (the web-scraping code is in the notebook `03-scrape-house-prices`).

The scatter plot of the `price` vs the `area` of the flat reveals that there is a strong dependency between price and area. Linear regression finds a *linear* relation between them.

```python
house_prices = pd.read_csv(
    'https://raw.githubusercontent.com/janchorowski/nn_assignments/nn18/lectures/03-house-prices.csv',
    index_col=0)
house_prices.head()
```

      rooms
      area
      price

      0
      3
      66.80
      367.801

      1
      3
      55.03
      289.458

      2
      4
      70.80
      389.000

      3
      1
      29.02
      195.695

      4
      2
      45.76
      265.754

House area and price seem to be correlated:

```python
# house_prices = house_prices[:100]

_ = scatter(house_prices.area, house_prices.price, alpha=0.3)
_ = xlabel('area m2')
_ = ylabel('price kPLN')
ylim(0, 5000)
```

## Linear regression analytical solution

We will now find the linear relation between house area and price.

Our data forms a table:

```python
house_prices.head()
```

      rooms
      area
      price

      0
      3
      66.80
      367.801

      1
      3
      55.03
      289.458

      2
      4
      70.80
      389.000

      3
      1
      29.02
      195.695

      4
      2
      45.76
      265.754

## Overview of learning

We have gathered a dataset consisting of pairs $\{(x^{(i)}, y^{(i)}), i=1\ldots N \}$. We want to find the relation between $x$, the *independent* variable, area in our example and $y$, the *dependent* variable, price in the example.

We will define a family of *hypotheses* (functions) parameterized by a vector of parameters $\Theta$.

$$ y \approx h(x; \Theta). $$

To *learn* the relation between areas and prices, or in general between $x$ and $y$, we will search the space of hypotheses (by changing the parameters $\Theta$) to find the one whose predictions best match the true targets.

We will need a *cost function* to measure the goodness of fit between hypotheses and the data.

Finally, we will need a training (learning) algorithm that will perform the search for us.

## The hypothesis

We will start with a simple linear family of hypotheses:
$$ h(x;\Theta) = \Theta_0 + \Theta_1 x. $$

## The cost function

We will measure the discrepancy between predictions of a hypothesis and the true $y$ on the dataset by computing the sum of squares of errors:

$$ J(\Theta) = \sum_{i=1}^{N} (y^{(i)}-h(x^{(i)};\Theta))^2 $$

## The training algorithm

We will first find the optimal $\Theta$ by deriving the analytical solution. However we will first generalize the problem a little, by assuming that $x$ can be a vector of values, $x=[x_1,\ldots,x_m]$. We will also assume, that $x_0=1$. Therefore:

$$ h(x;\Theta) = \Theta{}_0 + \sum_{j=1}^m \Theta_j x_j = \sum_{j=0}^m \Theta_j x_j = \Theta^T x $$

The cost function then becomes

$$ J(\Theta) = \sum_{i=1}^N (y^{(i)} - \Theta^T x^{(i)})^2 $$

We will make even greater use of matrix calculus. If each data sample $x$ is a vector, we can stack them into a large matrix $X \in \mathbb{R}^{m\times N}$, such that the $i$-th column of $X$ is the sample $x^{(i)}$ likewise we will concatenate all targets $y^{(i)}$ into a matrix $Y\in\mathbb{R}^{1 \times N}$. Armed with this notation, we can compute the predictions on all data points as $\Theta^T X$ and compute the matrix of errors $E\in \mathbb{R}^{1 \times N}$ as:

$$ E = \Theta^T X - Y. $$

Finally, please note that the cost function becomes simply:

$$ J(\Theta) = E E^T = (\Theta^T X - Y)(\Theta^T X - Y)^T $$

We will compute the *gradient* of $J$ with respect to $\Theta$. By definition, we will assume that the gradient is also a vector of the same shape as $\Theta$. To quickly compute the gradient, we will use the matrix calculus relations described here: http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/calculus.html and here: http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf:

$$ \frac{\partial J(\Theta)}{\partial\Theta} = \frac{\partial}{\partial\Theta} (\Theta^T X - Y)(\Theta^T X - Y)^T =
\frac{\partial}{\partial\Theta} [\Theta^T X X^T \Theta - YX^T\Theta - \Theta^T X Y^T + YY^T] =
2 X X^T \Theta -2 X Y^T  = 2X(X^T\Theta - Y^T) = 2X E^T$$

Solving for the zero-crossing of the derivative we obtain:

$$
\begin{align}
2 X X^T \Theta &= 2 X Y^T \\
\Theta &= (X X^T) ^{-1} X Y^T
\end{align}
$$

Please note: this is also called the Moore-Penrose pseudo-inverse of matrix $X$.

## Example of operation

```python
X = np.vstack((np.ones_like(house_prices.area), house_prices.area))
X[:,:8]
```

    array([[ 1.  ,  1.  ,  1.  ,  1.  ,  1.  ,  1.  ,  1.  ,  1.  ],
           [66.8 , 55.03, 70.8 , 29.02, 45.76, 38.99, 55.7 , 76.42]])

```python
Y = np.asarray(house_prices.price)[None,:]
Y
```

    array([[367.801, 289.458, 389.   , ..., 266.   , 267.1  , 269.   ]])

Remember, $\Theta = (X X^T)^{-1} X Y^T$.

```python
Theta = np.linalg.inv(X.dot(X.T)).dot(X).dot(Y.T)
Theta
```

    array([[-14.09901832],
           [  6.54132271]])

```python
figure(figsize=(6,4), dpi=100)
scatter(house_prices.area, house_prices.price, alpha=0.3, label='data')

test_areas = np.linspace(xlim()[0], xlim()[1], 512)
test_prices = Theta[0] + Theta[1]*test_areas

plot(test_areas, test_prices,'g', label='lin. regression')
xlim(test_areas[0], test_areas[-1])
xlabel('area m2')
ylabel('price kPLN')
text(150, 3000, "$y=%.2fx+%.2f$" % (Theta[1], Theta[0]), fontsize=14)
legend()
ylim(0, 4000)
title('House prices in 2018')
grid(True)
```

## Interpretation of the least squares criterion

We will now work out a *probabilistic interpretation* of our model. We will assume that the data is generated according to the following process:
1. $x$ is sampled
2. $y$ is computed as $y=\Theta^T x + \epsilon$, where $\epsilon \sim \mathcal{N}(\mu=0,\sigma)$

We assume that the dependent variable $y$ is produced by linearly transforming $x$ and adding a little bit of Gaussian noise. The intuition is that there is the main linear relation that we model, and all other phenomena add up as noise, which by the central limit theorem we assume to be normal.

We assume that for each sample in our dataset the noise was sampled independently from the same noise distribution (IID). This allows us to write the following conditional distribution of $y$ given $x$:

$$ p(y|x;\Theta) = \mathcal{N}(y; \mu=\Theta^T x, \sigma) $$

Due to the IID assumption for the noise variables $\epsilon^{(i)}$, we can also assume that the targets $y$ in our data aree conditionally independent given the $x$s. This allows us to easily compute the probability of observing the collected data under our data-generating hypothesis as:

$$
\begin{split}
p(Data; \Theta) &= \prod_{i=1}^N p(y^{(i)}|x^{(i)};\Theta) \\
&= \prod_{i=1}^N \mathcal{N}(y^{(i)}; \mu =\Theta^T x^{(i)}, \sigma) \\
&=  \prod_{i=1}^N \frac{1}{\sqrt{2\pi}\sigma}\exp\left(-\frac{(y^{(i)}-\Theta^Tx^{(i)})^2}{2\sigma^2}\right)
\end{split}
$$

We will use the maximum likelihood principle to choose $\Theta$. More specifically, this time we will minimize the *negative log-likelihood*, written in short nll:

$$
\begin{align}
nll(\Theta) &= - \log p(Data; \Theta) \\
&= -\sum_{i=1}^{N}\log \frac{1}{\sqrt{2\pi}\sigma}\exp\left(-\frac{(y^{(i)}-\Theta^Tx^{(i)})^2}{2\sigma^2}\right)  \\
&= -\sum_{i=1}^{N}\log \frac{1}{\sqrt{2\pi}\sigma} - \frac{(y^{(i)}-\Theta^Tx^{(i)})^2}{2\sigma^2} \\
&= C + \frac{1}{2\sigma^2}\sum_{i=1}^{N}(y^{(i)}-\Theta^Tx^{(i)})^2
\end{align}
$$

Note that minimization of $nll$ leads to maximization of the likelihood. Therefore the Gaussian noise assumption leads to the least squares cost function!

## Validity of the Gaussian noise assumption

Statisticians like to check if the residuals (the errors made by our model on the train set) follow indeed the assumed noise distribution. In the house prices example we can see that the residuals do not strictly follow the Gaussian distribution. Maybe we should take into account other house features, such as neighborhood, age, whether it is a new or used one... Think how you would incorporate such new information. Is it still possible to use linear regression to fit a model dependent on many variables?

In the following example the Gaussian noise assumption will be satisfied. We will download data of people's height vs weight.

```python
url = 'http://socr.ucla.edu/docs/resources/SOCR_Data/SOCR_Data_Dinov_020108_HeightsWeights.html'
height_weight = pd.read_html(url)[0]
```

```python
height_weight.head()
```

      0
      1
      2

      0
      Index
      Height(Inches)
      Weight(Pounds)

      1
      1
      65.78331
      112.9925

      2
      2
      71.51521
      136.4873

      3
      3
      69.39874
      153.0269

      4
      4
      68.2166
      142.3354

```python
inch_to_meter = 0.0254
height = np.asarray(height_weight.iloc[1:,1].astype(float))[None,:] * inch_to_meter #convert to meters
pound_to_kg = 0.453592
weight = np.asarray(height_weight.iloc[1:,2].astype(float))[None,:] * pound_to_kg # convert to kg
```

```python
scatter(height, weight, alpha=0.2, linewidths=0)
xlabel('height [m]')
ylabel('weight [kg]')
```

```python
X = np.vstack((np.ones_like(height), height))
X
```

    array([[1.        , 1.        , 1.        , ..., 1.        , 1.        ,
            1.        ],
           [1.67089607, 1.81648633, 1.762728  , ..., 1.64334317, 1.71524117,
            1.74949129]])

```python
Y = weight
Y
```

    array([[51.25249406, 61.90954738, 69.41177762, ..., 53.64428468,
            59.99579737, 56.64193813]])

```python
Theta = np.linalg.inv(X.dot(X.T)).dot(X).dot(Y.T)
Theta
```

    array([[-37.45569645],
           [ 55.06457669]])

```python
scatter(height, weight, alpha=0.1)
test_heights = np.linspace(xlim()[0], xlim()[1], 512)
test_weights = Theta[0] + Theta[1]*test_heights

plot(test_heights, test_weights,'g')
xlim(test_heights[0], test_heights[-1])
xlabel('height [m]')
ylabel('weight [kg]')
```

```python
E = Theta.T.dot(X) - Y
E
```

    array([[ 3.29899449,  0.65880721, -9.80360316, ..., -0.60998512,
            -3.00246477,  2.23736295]])

```python
scatter (X[1,:], E, alpha=0.1)
title("Scatter plot of residuals.")
```

```python
counts, bins, patches = hist(E.ravel(),100, normed=True)

mu = E.mean()
std = E.std()
e_range = np.linspace(bins[0], bins[-1], 100)
plot(e_range, 1/np.sqrt(2*np.pi)/std*np.exp(-e_range**2/2/std**2), 'r', linewidth=2)
title("Histogram of residuals\nNote the nice gaussian fit")
```

```python
# Analyze the histogram of errors along a few heights

slice_height = 1.85
selected = np.abs(X[1,:]-slice_height) < 0.05

selected_E = E[0,selected]
counts, bins, patches = hist(selected_E, 10, normed=True)

selected_mu = selected_E.mean()
selected_std = selected_E.std()
selected_e_range = np.linspace(bins[0], bins[-1], 100)
plot(selected_e_range, 1/np.sqrt(2*np.pi)/selected_std*np.exp(-(selected_mu-selected_e_range)**2/2/selected_std**2), 'r', linewidth=2)

title('Histogram of residuals for heights=%f +-1cm'
      'Slice residual mean: %f stdev: %f'% (slice_height,selected_mu, selected_std))
```

```python
# Are the errors homoscedastic?

err_stats = []

slice_width = 0.02

for slice_height in np.arange(height.min(), height.max(), slice_width):
    selected = np.abs(X[1,:]-slice_height) < slice_width/2.0

    selected_E = E[0,selected]

    selected_mu = selected_E.mean()
    selected_std = selected_E.std()
    err_stats.append((slice_height, selected_mu, selected_std))

err_stats = np.asarray(err_stats)

plot(err_stats[:,0], err_stats[:,1], label='mean of residuals in slice')
plot(err_stats[:,0], err_stats[:,2], label='stdev of residuals in slice')
legend(loc='lower right')
title("mean and standard deviation of residuals vs height")
xlabel("height")
```

```python

```


# Manoeuvring simulations
Many simulation model for ship manoeuvring have been developed in the field of ship hydrodynamics such as: the Abkowitz model {cite:p}`abkowitz_ship_1964` or the Norrbin model {cite:p}`norrbin_study_1960`.
This chapter will develop a general simulation model for ship manoeuvring, that can be further specified to become either the Abkowitz or Norbin model. Expressing the models on a general form is important in this research where many different models will be tested and compared.

```python
import matplotlib
print(matplotlib.get_configdir())
```

```python
# %load imports.py
## Local packages:

%matplotlib inline
%load_ext autoreload
%autoreload 2
%config Completer.use_jedi = False  ## (To fix autocomplete)

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex, Markdown
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

from myst_nb import glue
from src.symbols import *

import src.symbols as symbols
from src import prime_system
from src.models.diff_eq_to_matrix import DiffEqToMatrix
from src.visualization.regression import show_pred
from src.visualization.plot import track_plot, plot

## Load models:
# (Uncomment these for faster loading):
import src.models.vmm_nonlinear_EOM  as vmm

import matplotlib.pyplot as plt
if os.name == 'nt':
    plt.style.use('book.mplstyle')  # Windows

## Examples
from example_1 import ship_parameters, df_parameters
from myst_nb import glue

p = df_parameters["symbol"]
```

```python
symbols.glue_equations(symbols)
```

```python
glue('eq_6DOF',Math(vlatex(eq_6DOF)))
```

```python
glue('eq_eta',Math(vlatex(eq_eta)))
```

```python
glue('eq_nu',Math(vlatex(eq_nu)))
```

The models can be expressed in a very general way as shown in {numref}`eq_6DOF` ( {cite:p}`fossen_handbook_2021`).

```{glue:math} eq_6DOF
:label: eq_6DOF
```

Where $\eta$ describes the position and $\nu$ is the velocities:

```{glue:math} eq_eta
:label: eq_eta
```

```{glue:math} eq_nu
:label: eq_nu
```

The model matrixes:
* $M$ : inertia
* $C$ : corriolis/centrepetal
* $D$ : damping
* $g(\eta)$ is a vector of generalized gravitational an buoyance forces.
* $g_0$ is static restoring forces due to ballast systems.

The inertia as well as the corriolis matrix can be split into actual mass/intertia and added mass/inertia:

```{glue:math} eq_M
:label: eq_M
```

```{glue:math} eq_C
:label: eq_C
```

So that the model equation can be written as:
```{glue:math} eq_6DOF_expanded
:label: eq_6DOF_expanded
```

The velocity of the current can be described as (assumed to be irrotational):
```{glue:math} eq_nu_r
:label: eq_nu_c
```

So that $\nu_r$ are the relative velocities (including current):
```{glue:math} eq_nu_r
:label: eq_nu_r
```

If the current is also assumed to be constant, acceleration of the relative velocities is the same:
```{glue:math} eq_nu_steady
:label: eq_nu_steady
```
So that the entire equation can be expressed in terms of relative velocities:

```{glue:math} eq_6DOF_expanded_r
:label: eq_6DOF_expanded_r
```

## Ship parameters
The Ocean Bird Wind Powered Car Carrier will be used as the reference ship of this book:

```python
ps = prime_system.PrimeSystem(**ship_parameters)  # model
```

A model scale version of this ship has the following main dimensions:

```python
ship_parameters
```

## 3 DOF
A simulation model with only three degrees of fredom (DOF) : *surge*, *sway* and *yaw* is often sufficient to describe the ship's manoeuvring performance. The equation of motion can be expressed as (Triantafyllou & Hover, 2003):

```python
display(vmm.X_eom)
display(vmm.Y_eom)
display(vmm.N_eom)

```

Where $X_{force}$, $Y_{force}$ and $N_{force}$ are the hydrodynamic forces from the ship. So these equations holds for most of the different models for manoeuvring, except linear models where the nonlinear $r\cdot u$ term have been linearized. The difference in the methods is rather in how these hydrodynamic forces are calculated.

The hydrodynamic forces can be split into added masses (that depend on the accelerations: $\dot{u}$, $\dot{v}$, $\dot{r}$) and dampings (that depend on the velocities: $u$, $v$, $r$):

```python
display(Math(vlatex(vmm.fx_eq)))
display(Math(vlatex(vmm.fy_eq)))
display(Math(vlatex(vmm.mz_eq)))
```

$X_{\dot{u}}$, $X_{\dot{v}}$, $X_{\dot{r}}$ are the added masses: when the ship is accelerating in water, not only the mass of the ship is accelerating, but also some mass in the water needs to be accelerated. This is referred to as added masses. Sometimes added  masses such as $X_{\dot{v}}$ is neglected, as in the case above.

The specialization of the various simulation models can now be further categorized to the functions to calculate the damping forces: $X_{D}$, $Y_{D}$, $N_{D}$.

```python
ship_parameters_prime = ps.prime(ship_parameters)
```

The general equation for all of the simulation models in this research can be written as:

```python
display(vmm.X_eq)
display(vmm.Y_eq)
display(vmm.N_eq)
```

The classic simulation methods express the damping forces as a truncated taylor series. Here is a rather simple model, that has been created by the author to be used in the followin examples.

```python
display(vmm.X_qs_eq)
display(vmm.Y_qs_eq)
display(vmm.N_qs_eq)
```

## Similitude

### Prime system
The prime system uses ship length $L$, water density $\rho$ and total velocity $U^2=u^2+v^2$ to express physical quantities in nondimensional form. The quantities have a prime symbol (') attached to them when the prime system is used. The quantities are made nondimensional with the prime system by dividing with a denominator according to the table below:

```python
prime_system.df_prime.loc[['denominator']].transpose()
```

The manoeuvring models are often expressed in the prime system so that a nondimensional force $F'$ for instance can be expressed as a function of some coefficients $C$ and nondimensional velocity $v$:

```python
F_,v_, C_=sp.symbols("F' v' C")
F=sp.symbols("F")

eq = sp.Eq(F_,
      C_*v_)
eq
```

This may look as a model where the force depend on the transverse velocity $v$ only, but this is actually not true. If this equation is converted converted back to SI units it is instead written:

```python
eq_F_prime = sp.Eq(F,prime_system.df_prime.loc['denominator','force']*F_)
eq_v_prime = sp.Eq(v,prime_system.df_prime.loc['denominator','linear_velocity']*v_)

eqs = [eq, eq_F_prime, eq_v_prime]

sp.Eq(F,sp.solve(eqs,F,F_,v_,dict=True)[0][F])
```

So it is now clear that the example force model above, also depends on the total velocity $U$.

## Brix parameters
The hydrodynamic derivatives can be estimated with semi empirical formulas according to (Brix, 1993).

```python
mask = df_parameters['prime'].notnull()
index = df_parameters.loc[mask,'prime'].index
coefficients=vmm.simulator.get_all_coefficients(sympy_symbols=False)
missing_coefficients = set(coefficients) - set(index)
missing_coefficients
```

```python
mask = df_parameters['prime'].notnull()
for index, parameter in df_parameters.loc[mask].iterrows():
    display(sp.Eq(parameter['symbol'],parameter['prime']))
```

```python
U_ = 2
glue("U_",U_);
```

```python
parameters=df_parameters['prime'].copy()

N_=30
df_captive_template = pd.DataFrame(index=np.arange(N_))
df_captive_template['U'] = U_
df_captive_template['u'] = df_captive_template['U']
df_captive_template['v'] = 0
df_captive_template['r'] = 0
df_captive_template['delta'] = 0

df_r = df_captive_template.copy()
df_r['r'] = np.linspace(-0.3,0,N_)
df_r['vary'] = 'r'

df_v = df_captive_template.copy()
df_v['v'] = np.linspace(0,0.5,N_)
df_v['u'] = np.sqrt(df_v['U']**2 - df_v['v']**2)
df_v['vary'] = 'v'

df_delta = df_captive_template.copy()
df_delta['delta'] = np.linspace(0.,0.4,N_)
df_delta['vary'] = 'delta'

df_captive = pd.concat([df_delta,df_v,df_r])

df_captive['X'] = run(vmm.simulator.X_qs_lambda, **parameters, **df_captive)
df_captive['Y'] = run(vmm.simulator.Y_qs_lambda, **parameters, **df_captive)
df_captive['N'] = run(vmm.simulator.N_qs_lambda, **parameters, **df_captive)
df_captive = ps.prime(df_captive, U=df_captive['U'])  # Convert to prime system

ncols = len(df_captive['vary'].unique())
fig,axes=plt.subplots(ncols=ncols, nrows=3)

vary_labels={
    'delta' : r'\delta',
}
for col,(vary, df_) in enumerate(df_captive.groupby(by='vary')):

    for row,dof in enumerate(['X','Y','N']):
        ax = axes[row,col]
        df_.plot(x=vary, y=dof, ax=ax)
        ax.get_legend().set_visible(False)
        ax.set_xlabel('')
        ax.grid(True)
        axes[row,0].set_ylabel(f"${dof}'$")

        ylims = [df_captive[dof].min(),df_captive[dof].max()]
        if ylims[0]!=ylims[1]:
            ax.set_ylim(ylims)

    axes[-1,col].set_xlabel(f"${vary_labels.get(vary,vary)}'$")
```

The figure above shows the quasi static forces for the present ship with the hydrodynamic derivatives during a variation of rudder angle $\delta$,
         yaw rate $r$ and tranverse velocity $v$ and total velocity $U$={glue:}`U_` m/s

## Simulation

### Decoupling
There is a coupling between the sway and yaw equation thruogh the added masses. These equations need to be decoupled to be usable in a simulation model. This is done by first expressing the equation in matrix form:

```python
X_eq = vmm.X_eq
Y_eq = vmm.Y_eq
N_eq = vmm.N_eq

A, b = sp.linear_eq_to_matrix([X_eq, Y_eq, N_eq], [u1d, v1d, r1d])

acceleration = sp.matrices.MutableDenseMatrix([u1d,v1d,r1d])
eq_simulator = sp.Eq(sp.UnevaluatedExpr(A)*sp.UnevaluatedExpr(acceleration),sp.UnevaluatedExpr(b))
eq_simulator
```

The decoupled equations are obtained by inverting the intertia matrix:

This equation can be written in a cleaner way if the $S$ term is introduced:

```python
A_inv = A.inv()
S = sp.symbols('S')
eq_S=sp.Eq(S,-sp.fraction(A_inv[1,1])[1])

A_inv_S = A_inv.subs(eq_S.rhs,S)
eq_acceleration_matrix_clean = sp.Eq(sp.UnevaluatedExpr(acceleration),sp.UnevaluatedExpr(A_inv_S)*sp.UnevaluatedExpr(b))
Math(vlatex(eq_acceleration_matrix_clean))
```

Where $S$ is a helper variable defined as:

```python
eq_S
```

### prime and SI
This concise expression to calculate the accelerations is expressed in the prime system. Conducting a simulation by integrating this expression will give a simulation that is conducted in the prime system, with prime system time etc., which is not very convenient. If a simulation should instead be conducted with a standard unit system (SI), the states (velocities as well as the resulting accelerations) and the ship parameters (length, masses and inertia) need to be converted.

The input velocities in SI should be converted to prime and the output accelerations in prime needs to be converted back to SI:
1. Input: velocities : SI -> prime
2. Output: accelerations : prime -> SI

This can be done for each time step in the simulations, by implementing this logic into the code. Another alternative is to rewrite the expression into SI units, which will be done below. The damping functions: $X_D(u,v,r,\delta)$, $Y_D(u,v,r,\delta)$, $N_D(u,v,r,\delta)$ is also substituted with their polynomials to derive the full expression.

```python
u1d_function = sp.Function(r'\dot{u}')(u,v,r,delta)
v1d_function = sp.Function(r'\dot{v}')(u,v,r,delta)
r_function = sp.Function(r'\dot{r}')(u,v,r,delta)

subs_prime = [

    (m,m/prime_system.df_prime.mass.denominator),
    (I_z,I_z/prime_system.df_prime.inertia_moment.denominator),
    (x_G,x_G/prime_system.df_prime.length.denominator),

    (u, u/sp.sqrt(u**2+v**2)),
    (v, v/sp.sqrt(u**2+v**2)),
    (r, r/(sp.sqrt(u**2+v**2)/L)),

]

subs = [
(X_D, vmm.X_qs_eq.rhs),
(Y_D, vmm.Y_qs_eq.rhs),
(N_D, vmm.N_qs_eq.rhs),
]

subs = subs + subs_prime

A_SI = A.subs(subs)
b_SI = b.subs(subs)

x_dot = sympy.matrices.dense.matrix_multiply_elementwise(A_SI.inv()*b_SI,
                                                 sp.Matrix([(u**2+v**2)/L,(u**2+v**2)/L,(u**2+v**2)/(L**2)]))

sp.Eq(sp.UnevaluatedExpr(sp.Matrix([
    u1d_function,
    v1d_function,
    r_function,
])),sp.UnevaluatedExpr(x_dot))

```

...so the expanded expression in SI units is a lot more complicated.

## Simulate data

```python
x, x1d = sp.symbols(r'\vec{x} \dot{\vec{x}}')  # State vector
h = sp.symbols('h')
u_input = sp.symbols(r'u_{input}')  # input vector
w_noise = sp.symbols(r'w_{noise}')  # input vector

f = sp.Function('f')(x,u_input,w_noise)
eq_system = sp.Eq(x1d, f)
eq_system
```

Where the state vector $\vec{x}$:

```python
eq_x = sp.Eq(x, sp.UnevaluatedExpr(sp.Matrix([x_0, y_0, psi, u, v, r])))
eq_x
```

```python
eq_x0_1d = sp.Eq(x_01d,u*sp.cos(psi)-v*sp.sin(psi))
eq_y0_1d = sp.Eq(y_01d,u*sp.sin(psi)+v*sp.cos(psi))
eq_psi_1d = sp.Eq(psi1d,r)

eq_f =sp.Eq(f,
sp.UnevaluatedExpr(
sp.Matrix([
    eq_x0_1d.rhs,
    eq_y0_1d.rhs,
    eq_psi_1d.rhs,
    u1d_function,
    v1d_function,
    r_function,

]
))
)
display(eq_f)
```

```python
x_ = sp.Matrix([u*sp.cos(psi)-v*sp.sin(psi),
                                u*sp.sin(psi)+v*sp.cos(psi),
                                r])

f_ = sp.Matrix.vstack(x_, x_dot)

subs = {value: key for key, value in p.items()}
subs[psi] = sp.symbols('psi')
lambda_f = lambdify(f_.subs(subs))
```

The actual simulation can be carried out with a simple Euler forward integration:

```python
def time_step(x_,u_):
    psi=x_[2]
    u=x_[3]
    v=x_[4]
    r=x_[5]
    delta = u_
    x_dot = run(lambda_f, **parameters, **ship_parameters, psi=psi, u=u, v=v, r=r, delta=delta).flatten()
    return x_dot

def simulate(x0,E, ws, t, us):

    simdata = np.zeros((6,len(t)))
    x_=x0

    for i,(u_,w_) in enumerate(zip(us,ws)):

        x_dot = time_step(x_,u_)

        x_=x_ + h_*x_dot

        simdata[:,i] = x_

    df = pd.DataFrame(simdata.T, columns=["x0","y0","psi","u","v","r"], index=t)
    df.index.name = 'time'
    df['delta'] = us

    return df
```

```python
parameters=df_parameters['prime'].copy()
N_ = 1000
t_ = np.linspace(0,50,N_)
h_ = float(t_[1]-t_[0])
ws = np.zeros(N_)
us = delta_ = np.deg2rad(20)*np.ones(N_)
x0_ = np.array([0,0,0,2,0,0])

E=np.array([0,0,0,1,1,1])
df = simulate(x0=x0_, E=E, ws=ws, t=t_, us=us)
df['beta'] = -np.arctan2(df['v'],df['u'])
df['U'] = np.sqrt(df['u']**2 + df['v']**2)
df.to_csv('first_simulation.csv', index=True)
```

```python
track_plot(
            df=df,
            lpp=ship_parameters["L"],
            beam=ship_parameters["B"],
            color="green",
        );

plot(df_result=df);
```

### Simulate parameter contributions

```python
df_result_prime = ps.prime(df, U=df['U'])
```

```python
X_ = sp.symbols('X_')
diff_eq_X = DiffEqToMatrix(ode=vmm.X_qs_eq.subs(X_D,X_),
                                      label=X_, base_features=[delta,u,v,r])
X = diff_eq_X.calculate_features(data=df_result_prime)
X_parameters = df_parameters.groupby(by='dof').get_group('X')['prime'].dropna()
X_forces = X*X_parameters
X_forces.index = df_result_prime.index

Y_ = sp.symbols('Y_')
diff_eq_Y = DiffEqToMatrix(ode=vmm.Y_qs_eq.subs(Y_D,Y_),
                                      label=Y_, base_features=[delta,u,v,r])
X = diff_eq_Y.calculate_features(data=df_result_prime)
Y_parameters = df_parameters.groupby(by='dof').get_group('Y')['prime'].dropna()
Y_forces = X*Y_parameters
Y_forces.index = df_result_prime.index

N_ = sp.symbols('N_')
diff_eq_N = DiffEqToMatrix(ode=vmm.N_qs_eq.subs(N_D,N_),
                                      label=N_, base_features=[delta,u,v,r])
X = diff_eq_N.calculate_features(data=df_result_prime)
N_parameters = df_parameters.groupby(by='dof').get_group('N')['prime'].dropna()
N_forces = X*N_parameters
N_forces.index = df_result_prime.index
```

Here is an interavtive graph showing how the various hydrodynamic derivatives contribute to the forces:

```python
display(px.line(X_forces, y=X_forces.columns, width=800, height=350, title='X'))
display(px.line(Y_forces, y=Y_forces.columns, width=800, height=350, title='Y'))
display(px.line(N_forces, y=N_forces.columns, width=800, height=350, title='N'))
```


```python
import sympy as sym
%matplotlib inline
sym.init_printing()

import matplotlib.pyplot as plt
from sympy.plotting.plot import MatplotlibBackend

def logx_plot(Hw, wrange, **kwargs):
    'create plot with logarithmic x-axis'
    p = sym.plot(Hw, wrange, show=False, **kwargs)
    backend = MatplotlibBackend(p)
    backend.process_series()
    backend.ax[0].spines['left'].set_position(('axes', 0))
    backend.ax[0].spines['bottom'].set_position(('axes', 0))
    plt.xscale('log')
    plt.grid(which='both')

def db(x):
    'compute dB value'
    return 20 * sym.log(sym.Abs(x), 10)
```

# Caracterización de Sistemas en Dominio de la Frecuencia

## Función de Transferencia

Hay distintas representaciones para modelar sistemas LTI. Algunas de estas son:
- Los coeficientes constantes de las Ecuaciones Diferenciales(ODEs).
- La [respuesta impulsional](../systems_time_domain/impulse_response.ipynb).
- La [respuesta al escalón](../systems_time_domain/step_response.ipynb).

La [Función de transferencia](../systems_time_domain/eigenfunctions.ipynb#Transfer-Function) usa la forma $e^{s t}$ como eigenfunción para establecer combinaciones de $e^{s t}$ al ser excitado un sistema LTI. Se usa la transformada de Laplace para esta representación.

### Respuesta al Impulso y Señales de Entrada y Salida

La operación de convolución en el dominio temporal tiene equivalencia en el dominio de la frecuencia.

En el dominio temporal, la salida $y(t)$ de un sistema LTI está dada por la convolución entre la entrada $x(t)$ y la [respuesta al impulso](../systems_time_domain/impulse_response.ipynb) $h(t)$.

\begin{equation}
y(t) = x(t) * h(t)
\end{equation}

En el dominio transformado se tiene

\begin{equation}
Y(s) = X(s) \cdot H(s)
\end{equation}

donde $X(s) = \mathcal{L} \{ x(t) \}$ y $Y(s) = \mathcal{L} \{ y(t) \}$ son las transformadas de Laplace de las señales de entrada y salida.

La función de transferencia $H(s)$ está dada por la transformada de Laplace de la respuesta al impulso $h(t)$

\begin{equation}
H(s) = \mathcal{L} \{ h(t) \}
\end{equation}

Así, un sistema LTI queda caracterizado mediante la respuesta impulsional en el dominio temporal y en el dominio frecuencial.

Note también que puede encontrarse la función de transferencia como el cociente entre las señales de salida $Y(s)$ y  $X(s)$ en el dominio de Laplace.

\begin{equation}
H(s) = \frac{Y(s)}{X(s)}
\end{equation}

### Función de Transferencia y Ecuaciones Diferenciales

Suponga una ecuación diferencial con coeficientes constantes $\alpha_n$ y $\beta_m$.

\begin{equation}
\alpha_N \frac{d^N}{d t^N} y(t) + \alpha_{N-1} \frac{d^{N-1}}{d t^{N-1}} y(t) + ... + \alpha_{2} \frac{d^{2}}{d t^{2}} y(t) + \alpha_{1} \frac{d}{d t} y(t) + \alpha_{0} y(t) = \beta_M \frac{d^M}{d t^M} x(t) + \beta_{M-1} \frac{d^{M-1}}{d t^{M-1}} x(t) + ... + \beta_{2} \frac{d^{2}}{d t^{2}} x(t) + \beta_{1} \frac{d}{d t} x(t) + \beta_{0} x(t)
\end{equation}

\begin{equation}
\sum_{n=0}^{N} \alpha_n \frac{d^n}{d t^n} y(t) = \sum_{m=0}^{M} \beta_m \frac{d^m}{d t^m} x(t)
\end{equation}

donde $\alpha_N \neq 0$ y $N$ es el orden del modelo.

Suponga también que los valores iniciales son cero.

Al aplicar la Transformada de Laplace a la ecuación se tiene:

\begin{equation}
Y(s) \sum_{n=0}^{N} \alpha_n s^n = X(s) \sum_{m=0}^{M} \beta_m s^m
\end{equation}

De donde se obtiene la función de transferencia como:

\begin{equation}
H(s) = \frac{Y(s)}{X(s)} = \frac{\sum_{m=0}^{M} \beta_m s^m}{\sum_{n=0}^{N} \alpha_n s^n}
\end{equation}

**Example**

Los coeficientes del filtro RLC analizado antes se pueden obtener de Ecuación Diferencial como:

\begin{equation}
C L \frac{d^2 y(t)}{dt^2} + C R \frac{d y(t)}{dt} + y(t) = x(t)
\end{equation}

\begin{align}
\beta_0 &= 1 & \alpha_0 &= 1 \\
& & \alpha_1 &= C R \\
& & \alpha_2 &= C L
\end{align}

A partir de estos coeficientes se define la función de transferencia $H(s)$ en `SymPy`.

```python

s = sym.symbols('s', complex=True)

R = 1
L = 0.5
C = 0.4

beta = 1
alpha = L*C, R*C, 1

Num_s = sym.Poly(beta, s)
Den_s = sym.Poly(alpha, s)
Den_s
```

```python
H = Num_s / Den_s
H
```

Varios programas definen la función de transferencia únicamente a partir de los coeficientes $\alpha$ y $\beta$.Este es el caso del Toolbox [`scipy.signal`](http://docs.scipy.org/doc/scipy/reference/signal.html) para python donde se puede evaluar numéricamente $H(j \omega)$ en un conjunto de valores de frecuencia angular $\omega_i$ a partir de los coeficientes del modelo.

```python
from scipy import signal

R = 1
L = 0.5
C = 0.4
beta = 1
alpha = L*C, R*C, 1

w, H = signal.freqresp((beta, alpha))

import matplotlib.pyplot as plt

plt.plot(w, abs(H))
plt.xlabel('$\omega$ in $s^{-1}$')
plt.ylabel('$|H(j \omega)|$')
plt.grid();
```

## Gráficas de Bode

La función de transferencia en el dominio de Fourier caracteriza la transmisión de un sistema LTI con respecto a una señal armónica $e^{j \omega t}$ con frecuencia angular $\omega = 2 \pi f$.

A menudo se trabajan las funciones magnitud $| H(j \omega) |$ y fase $\varphi_H(j \omega)$ de forma separada.

\begin{align}
| Y(j \omega) | &= | X(j \omega) | \cdot | H(j \omega) | \\
\varphi_Y(j \omega) &= \varphi_X(j \omega) + \varphi_H(j \omega)
\end{align}

donde $X(j \omega)$ es la señal de entrada y $| X(j \omega) |$ and $\varphi_X(j \omega)$ su magnitud y fase, respectivamente.

Observe que $| H(j \omega) |$ muestra la atenuación o amplificación en función de la frecuencia $| X(j \omega) |$ de la señal de entrada. Por otra parte, $\varphi_H(j \omega)$ muestra la fase que se introduce en el sistema.

Las [gráficas de Bode](https://en.wikipedia.org/wiki/Bode_plot) se usan para representar gráficamente las propiedades del sistema en términos de magnitud y fase en función de la frecuencia. Se trata de una gráfica logarítmica de la magnitud de la función de transferencia en [decibeles](https://en.wikipedia.org/wiki/Decibel)  $20 \log_{10} | H(j \omega) |$ contra el logaritmo de la frecuencia $\omega$ o $f$.

La fase $\varphi_H(j \omega)$ se grafica linealmente contra el logaritmo de la frecuencia.

Como $H(s)$ es racional respecto a $s$, se puede representar en términos de polos y ceros.

\begin{equation}
H(s) = K \cdot \frac{\prod_{\mu=1}^{M} (s - s_{0 \mu})}{\prod_{\nu=1}^{N} (s - s_{\infty \nu})}
\end{equation}

considerando que $s=j \omega$
\begin{equation}
H(j \omega) = K \cdot \frac{\prod_{\mu=1}^{M} (j \omega - s_{0 \mu})}{\prod_{\nu=1}^{N} (j \omega - s_{\infty \nu})}
\end{equation}

donde $s_{0 \mu}$ y $s_{\infty \nu}$ son el $\mu$-ecimo cero and $\nu$-ecimo polo de $H(s)$. El logaritmo de la magnitud y la fase son:

\begin{align}
\log_{10} | H(j \omega) | &= \sum_{\mu=0}^{Q} \log_{10} |j \omega - s_{0 \mu}| - \sum_{\nu=0}^{P} \log_{10} |j \omega - s_{\infty \nu}| + \log_{10} |K|   \\
\varphi_H(j \omega) &= \sum_{\mu=0}^{Q} \arg (j \omega - s_{0 \mu}) - \sum_{\nu=0}^{P} \arg (j \omega - s_{\infty \nu})
\end{align}

donde $\arg(\cdot)$ es el [argumento](https://en.wikipedia.org/wiki/Argument_%28complex_analysis%29) (fase) de una función compleja.

Los resultados anteriores permiten aproximar las gráficas de Bode a mano. Se recomienda revisar la [literatura](https://en.wikipedia.org/wiki/Bode_plot#Rules_for_handmade_Bode_plot).

**Ejemplo**

La gráfica de Bode del filtro anterior

La función de transferencia $H(j \omega)$ se obtiene a partir de $H(s)$ como

\begin{equation}
H(j \omega) = \frac{1}{C L s^2 + C R s + 1} \bigg\rvert_{s = j \omega}
\end{equation}

```python

s = sym.symbols('s')
w = sym.symbols('omega', real=True)

R = 1
L = 0.5
C = 0.4

H = 1/(C*L*s**2 + C*R*s + 1)
H = H.subs(s, sym.I * w)
H

```

```python
plt.rcParams['figure.figsize'] = 6, 3

logx_plot(db(H), (w, 0.1, 10),
          xlabel='$\omega$', ylabel='$20 \log_{10} | H(j \omega) |$ in dB')
logx_plot(sym.arg(H), (w, 0.1, 10),
          xlabel='$\omega$', ylabel=r'$\varphi(j \omega)$')
```

## Próxima sesión

Álgebra de bloques - Combinación de sistemas LTI


[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Task 10.2 Systemdiskussion für FIR-Filter

64BE42BAEF

```python
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
from scipy.signal import tf2zpk, dlti
from scipy.signal import TransferFunction, ZerosPolesGain, bilinear
from scipy.signal import dimpulse, dstep, dfreqresp, dbode
from scipy.fftpack import fft
import sys
import os
```

```python
cur_fol = os.getcwd()
print(cur_fol)
sys.path.append(cur_fol + '/../')  # access to root folder
```

```python
from sig_sys_tools import plot_zplane  # sig_sys_tools.py in the root folder
```

```python
Nf = 2**10  # values along frequency axis
Om = np.arange(Nf) * (2*np.pi/Nf)  # get frequency vector for bode, fft
```

# Define FIR System

```python
# FIR...finite impulse response, i.e. we actually refer to a non-recursive system
# linear-phase type I, i.e. odd length of b, axial symmetry
b = np.array([1, 0, 1, 0, 17/4, 0, 17/4, 0, 1, 0, 1])
a = np.zeros_like(b)  # length(a)==length(b)!
a[0] = 1
Nt = 2**4  # values along time axis
fir_flag = True
```

# System Descriptions

```python
z, p, k = tf2zpk(b, a)

# possible descriptions from scipy.signal package
sys1 = dlti(b, a)
sys2 = TransferFunction(b, a, dt=True)
sys3 = dlti(z, p, k)
sys4 = ZerosPolesGain(z, p, k, dt=True)

sys = sys1  # use one of the above possibilities
```

# Impulse Response

```python
h = dimpulse(sys, n=Nt)
plt.stem(np.squeeze(h[0]), np.squeeze(h[1]),
         use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
plt.xlabel(r'$k$')
plt.ylabel(r'$h[k]$')
plt.grid(True)
```

# Step Response

```python
he = dstep(sys, n=Nt)
plt.stem(np.squeeze(he[0]), np.squeeze(he[1]),
         use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
plt.xlabel(r'$k$')
plt.ylabel(r'$h_\epsilon[k]$')
plt.grid(True)
```

# Pole / Zero Map

```python
plot_zplane(z, p, k)
```

# Bode Plot

```python
_, level, phase = dbode(sys, Om)
_, H = dfreqresp(sys, Om)

Hmax = 30
Hmin = Hmax - 50

if fir_flag:
    # for FIR filters we can also utilize the FFT of the b-coefficients as
    Hfft = fft(b, Nf)  # zeropadded FFT is interpolation towards DTFT

plt.figure(figsize=(8, 6))
plt.subplot(2, 1, 1)
plt.plot(Om, level, 'C2')
plt.plot(Om, 20*np.log10(np.abs(H)), 'C1')
if fir_flag:
    plt.plot(Om, 20*np.log10(np.abs(Hfft)), 'C0')
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
plt.ylim(Hmin, Hmax)
#plt.xlabel(r'$\Omega$ / rad')
plt.ylabel(r'level / dB')
plt.grid(True)

plt.subplot(2, 1, 2)
plt.plot(Om, phase, 'C2')
plt.plot(Om, np.unwrap(np.angle(H))*180/np.pi, 'C1')
if fir_flag:
    plt.plot(Om, np.unwrap(np.angle(Hfft))*180/np.pi, 'C0')
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
plt.yticks(np.arange(-180*10, 0+180, 180))
plt.xlabel(r'$\Omega$ / rad')
plt.ylabel(r'phase / deg')
plt.grid(True)
plt.savefig('fir_filter_64BE42BAEF_bode.pdf')
```

# Impulse Response, Magnitude/Phase of Zerophase System

\begin{align}
&H_0(z) = z^5 + z^3 + \frac{17}{4}\cdot z^{+1} + \frac{17}{4}\cdot z^{-1} + z^{-3} + z^{-5}
&\\\bullet-\circ\\
&h_0[k] = \delta[k+5] + \delta[k+3] + \frac{17}{4}\cdot\delta[k+1] + \frac{17}{4}\cdot\delta[k-1] + \delta[k-3] + \delta[k-5]
\end{align}

```python
h = dimpulse(sys, n=Nt)
plt.stem(np.squeeze(h[0])-5, np.squeeze(h[1]),
         use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
plt.xlabel(r'$k$')
plt.ylabel(r'$h_0[k]$')
plt.grid(True)
```

```python
H01 = 2*np.cos(5*Om)
H02 = 2*np.cos(3*Om)
H03 = 17/2*np.cos(1*Om)
H0 = H01+H02+H03

Hphase = np.zeros_like(H0)
Hphase[H0 <= 0] = 180
Hphase[np.logical_and(H0 >= 0, Om > np.pi)] = 360

plt.figure(figsize=(8, 6))
plt.subplot(2, 1, 1)
plt.plot(Om, H01, 'C0', label=r'$2\cos(5\Omega)$')
plt.plot(Om, H02, 'C1', label=r'$2\cos(3\Omega)$')
plt.plot(Om, H03, 'C2', label=r'$\frac{17}{2}\cos(\Omega)$')
plt.plot(Om, H0, 'C6',
         label=r'$H_0(\Omega)$')
plt.plot(Om, np.abs(H0), 'C3',
         label=r'$|H_0(\Omega)|$', lw=2)
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
#plt.xlabel(r'$\Omega$ / rad')
plt.legend(loc='lower right')
plt.grid(True)

plt.subplot(2, 1, 2)
plt.plot(Om, Hphase, color='C3', lw=2, label=r'$\angle H_0$')
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
plt.yticks([0, 180, 360], [r'$0$', r'$180$', r'$360$'])
plt.xlabel(r'$\Omega$ / rad')
plt.ylabel('degree')
plt.legend(loc='lower right')
plt.grid(True)
plt.savefig('fir_filter_64BE42BAEF_zerophase.pdf')
```

```python
# check polarity inversion
N = 16
kx = np.arange(N)
phi = np.pi
x = np.cos(phi*kx)
xshift = np.cos(phi*(kx+5))  # simulate zero phase LTI system!
y = np.convolve(np.squeeze(h[1]), xshift)
ky = np.arange(y.size)
plt.plot(kx, x, label=r'$x[k]=\cos(\pi k)$')
plt.plot(ky, y/12.5, label=r'$y[k]=x[k] \, *_k \, h_0[k]/12.5$')
plt.legend()
plt.xlabel(r'$x[k]$')
plt.title('polarity inversion between in and out as expected')
plt.grid(True)
```

# Impulse Response, Magnitude/Phase of Linear Phase System = Pure Delay

\begin{align}
&H_d(z) = z^{-5}\bullet-\circ h_d[k] = \delta[k-5]
\end{align}

```python
bd = [0, 0, 0, 0, 0, 1]
ad = [1, 0, 0, 0, 0, 0]
sysd = dlti(bd, ad)
```

```python
hd = dimpulse(sysd, n=Nt)
plt.stem(np.squeeze(hd[0]), np.squeeze(hd[1]),
         use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
plt.xlabel(r'$k$')
plt.ylabel(r'$h_d[k]$')
plt.grid(True)
```

```python
_, level, phase = dbode(sysd, Om)
_, H = dfreqresp(sysd, Om)

Hmax = 10
Hmin = Hmax - 20

if fir_flag:
    # for FIR filters we can also utilize the FFT of the b-coefficients as
    Hfft = fft(bd, Nf)  # zeropadded FFT is interpolation towards DTFT

plt.figure(figsize=(8, 6))
plt.subplot(2, 1, 1)
plt.plot(Om, level, 'C2')
plt.plot(Om, 20*np.log10(np.abs(H)), 'C1')
if fir_flag:
    plt.plot(Om, 20*np.log10(np.abs(Hfft)), 'C0')
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
plt.ylim(Hmin, Hmax)
#plt.xlabel(r'$\Omega$ / rad')
plt.ylabel(r'level / dB')
plt.grid(True)

plt.subplot(2, 1, 2)
plt.plot(Om, phase, 'C2')
plt.plot(Om, np.unwrap(np.angle(H))*180/np.pi, 'C1')
if fir_flag:
    plt.plot(Om, np.unwrap(np.angle(Hfft))*180/np.pi, 'C0')
plt.xlim(0, 2*np.pi)
plt.xticks(np.arange(9)*2*np.pi/8,
           [r'$0$', r'$\frac{\pi}{4}$', r'$\frac{\pi}{2}$', r'$\frac{3\pi}{4}$',
            r'$\pi$', r'$\frac{5\pi}{4}$', r'$\frac{3\pi}{2}$',
            r'$\frac{7\pi}{4}$', r'$2\pi$'])
plt.yticks(np.arange(-180*10, 0+180, 180))
plt.xlabel(r'$\Omega$ / rad')
plt.ylabel(r'phase / deg')
plt.grid(True)
plt.savefig('fir_filter_64BE42BAEF_bode_delay.pdf')
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


## Notes on the Ammonia model ##

Exact equation from the [source code](https://github.com/pyspeckit/pyspeckit/blob/master/pyspeckit/spectrum/models/ammonia.py#L210):

            population_upperstate = lin_ntot * orthoparafrac * partition/(Z.sum())

            tau_dict[linename] = (population_upperstate /
                                  (1. + np.exp(-h*frq/(kb*tkin) ))*ccms**2 /
                                  (8*np.pi*frq**2) * aval *
                                  (1-np.exp(-h*frq/(kb*tex))) /
                                  (width/ckms*frq*np.sqrt(2*np.pi)) )

\begin{equation}
    \tau = N_{tot} g_{opr} Z_{upper} \frac{A_{ij} c^2}{8\pi\nu^2}
    \left(1-\exp{ \frac{-h \nu}{k_B T_{ex}} } \right)
    \left(1+\exp{\frac{-h \nu}{k_B T_K}}\right)
    \left((2\pi)^{1/2} \nu \sigma_\nu / c\right)^{-1}
\end{equation}

Equation 16 from Rosolowsky et al 2008:

$$N(1,1) = \frac{8 \pi k \nu_0^2}{h c^3} \frac{1}{A_{1,1}} \sqrt{2\pi}\sigma_\nu (T_{ex}-T_{bg})\tau$$

Rearranges to:

$$\tau = N(1,1) \frac{h c^3}{8\pi k \nu_0^2} A_{1,1} \frac{1}{\sqrt{2 \pi} \sigma_\nu} \left(T_{ex}-T_{bg}\right)^{-1}$$

Equation A4 of Friesen et al 2009:

$$N(1,1) = \frac{8\pi\nu^2}{c^2} \frac{g_1}{g_2} \frac{1}{A_{1,1}} \frac{1+\exp\left(-h\nu_0/k_B T_{ex}\right)}{1-\exp\left(-h \nu_0/k_B T_{ex}\right)} \int \tau(\nu) d\nu$$

Equation 98 of Mangum & Shirley 2015:

$$N_{tot} = \frac{3 h}{8 \pi \mu^2 R_i} \frac{J_u(J_u+1)}{K^2}
\frac{Q_{rot}}{g_J g_K g_I} \frac{\exp{E_u/k_B T_{ex}}}{\exp{h \nu/k_B T_{ex}} - 1}
\left[\frac{\int T_R dv}{f\left(J_\nu(T_{ex})-J_\nu{T_B}\right) }\right]$$

## From Scratch ##

$$\tau_\nu = \int \alpha_\nu ds$$
$$\alpha_\nu = \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left(1-\frac{g_l n_u}{g_u n_l}\right) \phi_\nu$$

Excitation temperature:
$$T_{ex} \equiv \frac{h\nu_0/k_b}{\ln \frac{n_l g_u}{n_u g_l} } $$

$\nu_0$ = rest frequency of the line

Rearranges to:
$$ \frac{n_l g_u}{n_u g_l} = \exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)$$

Boltzman  distribution:
$$ \frac{n_u}{n_l} = \frac{g_u}{g_l} \exp\left(\frac{-h \nu_0}{k_B T}\right)$$
where T is a thermal equilibrium temperature

Rearranges to:
$$ 1-\frac{n_u g_l}{n_l g_u} = 1-\exp\left(\frac{-h \nu_0}{k_B T}\right)$$

Column Density
$$N_u \equiv \int n_u ds$$
$$N_l \equiv \int n_l ds$$

Starting to substitute previous equations into each other:

$$\tau_\nu d\nu= \alpha_\nu d\nu = \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left(1-\frac{g_l n_u}{g_u n_l}\right) \phi_\nu d\nu$$

$$\frac{g_u}{g_l}N_l = N_u\exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)$$

First substitution is the Boltzmann distribution, with $T_{ex}$ for T
$$\int \tau_\nu d\nu = \int \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right] \phi_\nu d\nu $$

Second is the $N_l$ - $N_u$ relation:
$$\int \tau_\nu d\nu = \frac{c^2}{8\pi\nu_0^2}  A_{ul} N_u\left[\exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)\right] \left[  1-\exp\left(\frac{-h \nu_0}{k_B T}\right) \right] \int \phi_\nu d\nu $$

Then some simplification:
$$\int \tau_\nu d\nu = \frac{c^2}{8\pi\nu_0^2}  A_{ul} N_u \left[  \exp\left(\frac{h \nu_0}{k_B T}\right) - 1 \right] \int \phi_\nu d\nu $$

$$A_{ul} = \frac{64\pi^4\nu_0^3}{3 h c^3} \left|\mu_{lu}\right|^2$$

Becomes, via some manipulation, equation 29 of Mangum & Shirley 2015:
$$N_u = \frac{3 h c}{8\pi^3 \nu \left|\mu_{lu}\right|^2} \left[\exp\left(\frac{h\nu}{k_B T_{ex}}\right) -1\right]^{-1} \int \tau_\nu d\nu$$
where I have used $T_{ex}$ instead of $T$ here because that is one of the substitutions invoked (quietly) in their derivation.  There is some sleight-of-hand regarding assuming $N_l = n_l$ that essentially assumes $T_{ex}$ is constant along the line of sight, but that is fine.

(Equation 30 is the same as this one, but with $dv$ instead of $d\nu$ units)

Solve for tau again (because that's what's implemented in the code):

$$\mathrm{"tau"} = \int \tau_\nu d\nu = N_u \frac{
c^2 A_{ul}}{8\pi\nu_0^2} \left[\exp\left(\frac{h\nu}{k_B T_{ex}}\right) -1\right] $$

The key difference from Erik's derivation is that this is $N_u$, but he has defined $N_{(1,1)}= N_u + N_l$.

So, we get $N_l$ the same way as above:
$$N_l = \frac{8\pi\nu_0^2}{c^2} \frac{g_l}{g_u} A_{ul}^{-1} \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right]^{-1} \int \tau d\nu$$

$$N_l = \frac{3 h c}{8 \pi^3 \nu \left|\mu_{lu}\right|^2} \frac{g_l}{g_u}  \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right]^{-1} \int \tau d\nu$$

Added together:
$$N_u + N_l = \frac{3 h c}{8 \pi^3 \nu \left|\mu_{lu}\right|^2} \frac{\frac{g_l}{g_u}  +\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)}{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} \int \tau d\nu$$

We can solve that back for tau, which is what Erik has done:

$$\int \tau d\nu = (N_u + N_l) \frac{8 \pi^3 \nu \left|\mu_{lu}\right|^2}{3 h c} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {\frac{g_l}{g_u}  +\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

$$=(N_u + N_l) \frac{g_u}{g_l}\frac{8 \pi^3 \nu \left|\mu_{lu}\right|^2}{3 h c} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

$$=(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

now identical to Erik's equation.

This is actually a problem, because $N_u$ is related to $N_{tot}$ via the partition function, but there is some double-counting going on if we try to relate $N_{(1,1)}$ to $N_{tot}$ with the same equation.

So, to reformulate the equations in pyspeckit using the appropriate values, we want to use both the partition function (calculated using $T_{kin}$) and $N_u$.

Eqn 31:

$$N_u = N_{tot} \frac{g_u}{Q_{rot}} \exp\left(\frac{-E_u}{k_B T_{kin}}\right)$$

is implemented correctly in pyspeckit:

    population_upperstate = lin_ntot * orthoparafrac * partition/(Z.sum())

where ``partition`` is

$$Z_i(\mathrm{para}) = (2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{kin}}\right]$$

$$Z_i(\mathrm{ortho}) = 2(2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{kin}}\right]$$

...so I'm assuming (haven't checked) that $E_u = h (B_0 J (J+1) + (C_0-B_0)J^2)$

Note that the leading "2" above cancels out in the Z/sum(Z), so it doesn't matter if it's right or not.  I suspect, though, that the 2 belongs in front of both the para and ortho states, but it should be excluded for the J=0 case.

### An aside by Erik Rosolowsky ###
(Note May 16, 2018: I believe this was incorporated into the above analysis)

EWR: The above equation is problematic because it relates the total column density to the $(J,J)$ state which is the equivalent of the $N_{(1,1)}$ term.  In the notation above $N_{(1,1)} = N_u + N_l$, so to get this right, you need to consider the inversion transition splitting on top of the total energy of the state so that
$$ E_u = h (B_0 J (J+1) + (C_0-B_0)J^2) + \Delta E_{\mathrm{inv}}, g_u = 1 $$
and
$$ E_l = h (B_0 J (J+1) + (C_0-B_0)J^2) - \Delta E_{\mathrm{inv}}, g_l = 1 $$
or, since the splitting is small compared to the rotational energy (1 K compared to > 20 K), then

$$Z_J \approx 2 (2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{\mathrm{rot}}}\right]$$

where the leading 2 accounts for the internal inversion states.  Since this 2 appears in all the terms, it cancels out in the sum.  Note that I have also changed the $T_{\mathrm{kin}}$ to $T_{\mathrm{rot}}$ since these two aren't the same and it is the latter which establishes the level populations.

Returning to the above, I would then suggest

$$N_{(J,J)} = N_{tot} \frac{Z_J}{\sum_j Z_j} $$

## Is the treatment of optical depth correct? ##
May 16, 2018: https://github.com/pyspeckit/pyspeckit/blob/725746f517e9bdcc22b83f4f9d6c9b8666e0a99e/pyspeckit/spectrum/models/ammonia.py

In this version, we [compute the optical depth](https://github.com/pyspeckit/pyspeckit/blob/725746f517e9bdcc22b83f4f9d6c9b8666e0a99e/pyspeckit/spectrum/models/ammonia.py#L318) with the code:

```
        for kk,nuo in enumerate(nuoff):
            tauprof_ = (tau_dict[linename] * tau_wts[kk] *
                        np.exp(-(xarr.value+nuo-lines[kk])**2 /
                               (2.0*nuwidth[kk]**2)))
            if return_components:
                components.append(tauprof_)
            tauprof += tauprof_

```

The total tau is normalized such that $\Sigma(\tau_{hf})_{hf} = \tau_{tot}$ for each line, i.e., the hyperfine $\tau$s sum to the tau value specified for the line.

The question Nico raised is, should we be computing the synthetic spectrum as $1-e^{\Sigma(\tau_{hf,\nu})}$ or $\Sigma(1-e^{\tau_{hf,\nu}})$?

The former is correct: we only have one optical depth per frequency bin.  It doesn't matter what line the optical depth comes from.

```python
# This is a test to show what happens if you add lines vs. computing a single optical depth per channel
from pyspeckit.spectrum.models.ammonia_constants import (line_names, freq_dict, aval_dict, ortho_dict,
                                voff_lines_dict, tau_wts_dict)
from astropy import constants
from astropy import units as u
import pylab as pl

linename = 'oneone'

xarr_v = (np.linspace(-25,25,1000)*u.km/u.s)
xarr = xarr_v.to(u.GHz, u.doppler_radio(freq_dict['oneone']*u.Hz))
tauprof = np.zeros(xarr.size)
true_prof = np.zeros(xarr.size)
width = 0.1
xoff_v = 0
ckms = constants.c.to(u.km/u.s).value

pl.figure(figsize=(12,12))
pl.clf()
for ii,tau_tot in enumerate((0.001, 0.1, 1, 10,)):
    tau_dict = {'oneone':tau_tot}

    voff_lines = np.array(voff_lines_dict[linename])
    tau_wts = np.array(tau_wts_dict[linename])

    lines = (1-voff_lines/ckms)*freq_dict[linename]/1e9
    tau_wts = tau_wts / (tau_wts).sum()
    nuwidth = np.abs(width/ckms*lines)
    nuoff = xoff_v/ckms*lines

    # tau array
    tauprof = np.zeros(len(xarr))
    for kk,nuo in enumerate(nuoff):
        tauprof_ = (tau_dict[linename] * tau_wts[kk] *
                    np.exp(-(xarr.value+nuo-lines[kk])**2 /
                           (2.0*nuwidth[kk]**2)))

        tauprof += tauprof_
        true_prof += (1-np.exp(-tauprof_))

    ax = pl.subplot(4,1,ii+1)
    ax.plot(xarr_v, 1 - np.exp(-tauprof), label=str(tau_tot), zorder=20, linewidth=1)
    ax.plot(xarr_v, true_prof, label=str(tau_tot), alpha=0.7, linewidth=2)
    ax.plot(xarr_v, true_prof-(1-np.exp(-tauprof)) - tau_tot/20, linewidth=1)
    pl.title(str(tau_tot))
```

# Below are numerical checks for accuracy #
Some numerical checks: How bad was the use of Tkin instead of Tex in the $\tau$ equation?

$$(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

```python
from astropy import units as u
from astropy import constants
freq = 23*u.GHz
def tau_wrong(tkin, tex):
    return (1-np.exp(-constants.h * freq/(constants.k_B*tkin)))/(1+np.exp(-constants.h * freq/(constants.k_B*tex)))
def tau_right(tex):
    return (1-np.exp(-constants.h * freq/(constants.k_B*tex)))/(1+np.exp(-constants.h * freq/(constants.k_B*tex)))
```

```python
tkin = np.linspace(5,40,101)*u.K
tex = np.linspace(5,40,100)*u.K
```

```python
grid = np.array([[tau_wrong(tk,tx)/tau_right(tx) for tx in tex] for tk in tkin])
```

```python
%matplotlib inline
import pylab as pl
```

```python
pl.imshow(grid, cmap='hot', extent=[5,40,5,40])
pl.xlabel("Tex")
pl.ylabel("Tkin")
pl.colorbar()
pl.contour(tex, tkin, grid, levels=[0.75,1,1/0.75], colors=['w','w','k'])
```

So the error could be 50%-700% over a somewhat reasonable range.  That's bad, and it affects the temperature estimates.  However, the effect on temperature estimates should be pretty small, since each line will be affected in the same way.  The biggest effect will be on the column density.

But, is this error at all balanced by the double-counting problem?

Because we were using the partition function directly, it's not obvious.  I was assuming that we were using the equation with $N_u$ as the leader, but we were using $N_u+N_l$.  i.e., I was using this equation:

$$\int \tau d\nu =(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

but with $N_u$ in place of $N_u + N_l$.

The magnitude of the error can therefore be estimated by computing $(N_u+N_l)/N_u = 1 + \frac{N_l}{N_u}$.

We can use the Boltzmann distribution to compute this error, then:
$$ \frac{n_u}{n_l} = \frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T}\right)$$

```python
def nunlnu_error(Tkin):
    return 1+np.exp(-constants.h * freq / (constants.k_B * Tkin))
```

```python
pl.plot(tkin.value, nunlnu_error(tkin))
```

So we were always off by a factor very close to 2.  The *relative* values of $\tau$ should never have been affected by this issue.

It will be more work to determine exactly how much the T_K and column estimates were affected.

# New work in May 2016: T_{rot} #

Comparing Trot and Tkin.  If we start with the equation that governs level populations,

$$N_u = N_{tot} \frac{g_u}{Q_{rot}} \exp\left(\frac{-E_u}{k_B T_{kin}}\right)$$

we get

$$N_u / N_l = \frac{g_u}{g_l} \exp\left(\frac{-E_u}{k_B T_{kin}} + \frac{E_l}{k_B T_{kin}}\right)$$

where we really mean $T_{rot}$ instead of $T_{kin}$ here as long as we're talking about just two levels.  This gives us a definition

$$T_{rot} = \left(\frac{E_l-E_u}{k_B}\right)\left[\ln\left(\frac{N_u g_l}{N_l g_u}\right)\right]^{-1}$$

which is the rotational temperature for a two-level system... which is just a $T_{ex}$, but governing non-radiatively-coupled levels.

So, for example, if we want to know $T_{rot}$ for the 2-2 and 1-1 lines at $n=10^4$ and $T_{kin}=20$ K:

```python
from pyradex import Radex
from astropy import constants, units as u
R = Radex(species='p-nh3', column=1e13, collider_densities={'pH2':1e4}, temperature=20)
tbl = R(collider_densities={'ph2': 1e4}, temperature=20, column=1e13)
```

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.

```python
tbl[8:10]
```

Table length=2

Tex	tau	frequency	upperstateenergy	upperlevel	lowerlevel	upperlevelpop	lowerlevelpop	brightness	T_B

K		GHz	K					erg / (cm2 Hz s sr)	K

float64	float64	float64	float64	bytes6	bytes6	float64	float64	float64	float64

6.789360524825584	0.09294143095281081	23.6944955	1.14	01_01_	01_01_	0.393279209876068	0.4649878353939112	6.176294459844954e-17	0.358064436431466

6.533403488783305	0.02126727244093209	23.7226333	42.32	02_02_	02_02_	0.0646415825850439	0.07694706934948849	1.3749154382566439e-17	0.07952035764858055

```python
# we're comparing the upper states since these are the ones that are emitting photons
trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
        np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
               (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
        )
```

```python
trot
```

$17.776914 \; \mathrm{K}$

```python
tbl['Tex'][8:10].mean()
```

    6.661382006804445

## Pause here ##

$T_{rot} = 60$ K for $T_{kin}=25$ K?  That doesn't seem right.  Is it possible RADEX is doing something funny with level populations?

# ERIK I SOLVED IT #
I had left out the $^{-1}$ in the code.  Oops!

```python
dT_oneone = -(constants.h * u.Quantity(tbl['frequency'][8], u.GHz)/constants.k_B).to(u.K)
print("delta-T for 1-1_upper - 1-1_lower: {0}".format(dT_oneone))
tex = (dT_oneone *
        np.log((tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[8]) /
               (tbl['lowerlevelpop'][8] * R.upperlevel_statisticalweight[8]))**-1
        )
print("Excitation temperature computed is {0} and should be {1}".format(tex.to(u.K), tbl['Tex'][8]))
```

    delta-T for 1-1_upper - 1-1_lower: -1.1371564340528206 K
    Excitation temperature computed is 6.789341109004981 K and should be 6.789360524825584

## Moving on: comparison to Swift et al 2005 ##

Swift et al 2005 eqn A6

$$T_R = T_K \left[ 1 + \frac{T_K}{T_0} \ln \left[1+0.6\exp\left( -15.7/T_K \right)\right] \right]^{-1}$$

where $T_0=41.18$ K

```python
T0=tbl['upperstateenergy'][9]-tbl['upperstateenergy'][8]
T0
```

    41.18

```python
def tr_swift(tk, T0=T0):
    return tk*(1+tk/T0 * np.log(1+0.6*np.exp(-15.7/tk)))**-1
```

Note that the approximation "works" - gets something near 20 - for positive or negative values of T0 (but see below)

```python
tr_swift(20, T0=-41.18)
```

    22.662533151044173

```python
tr_swift(20, T0=41.18)
```

    17.897313974001626

```python
tr_swift(20, T0=41.5)
```

    17.91183463497009

```python
def trot_radex(column=1e13, density=1e4, tkin=20):
    tbl = R(collider_densities={'ph2': density}, temperature=tkin, column=column)
    trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
        np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
               (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
        )
    return trot
```

RADEX suggests that the *positive* T0 value is the correct one (the negative one appeared correct when incorrectly indexed statistical weights were being used)

```python
trot_radex(tkin=20)
```

$17.776914 \; \mathrm{K}$

```python
def tex_radex(column=1e13, density=1e4, tkin=20, lineno=8):
    """ used in tests below """
    tbl = R(collider_densities={'ph2': density}, temperature=tkin, column=column)
    return tbl[lineno]['Tex']
```

```python
%matplotlib inline
import pylab as pl
```

```python
cols = np.logspace(12,15)
trots = [trot_radex(column=c).to(u.K).value for c in cols]
pl.semilogx(cols, trots)
pl.hlines(tr_swift(20), cols.min(), cols.max(), color='k')
pl.xlabel("Column")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

```python
densities = np.logspace(3,9)
trots = [trot_radex(density=n).to(u.K).value for n in densities]
pl.semilogx(densities, trots)
pl.hlines(tr_swift(20), densities.min(), densities.max(), color='k')
pl.xlabel("Volume Density")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

This is the plot that really convinces me that the negative (black curve) value of T0 is the appropriate value to use for this approximation

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
pl.plot(temperatures, trots)
# wrong pl.plot(temperatures, tr_swift(temperatures, T0=-41.18), color='k')
pl.plot(temperatures, tr_swift(temperatures, T0=41.18), color='r')
pl.xlabel("Temperatures")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

```python
temperatures = np.linspace(5,40,50)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
pl.plot(temperatures, np.abs(trots-tr_swift(temperatures, T0=41.18))/trots)
pl.xlabel("Temperatures")
pl.ylabel("$(T_{rot}(\mathrm{RADEX}) - T_{rot}(\mathrm{Swift}))/T_{rot}(\mathrm{RADEX})$")
```

## Tests of cold_ammonia reproducing pyspeckit ammonia spectra ##

```python
from pyspeckit.spectrum.models.tests import test_ammonia
from pyspeckit.spectrum.models import ammonia
```

### Test 1: Use a constant excitatino temperature for all lines

```python
tkin = 20*u.K
trot = trot_radex(tkin=tkin)
print(trot)
```

    17.776914063182385 K

```python
spc = test_ammonia.make_synthspec(lte=False, tkin=None, tex=6.66, trot=trot.value, lines=['oneone','twotwo'])
spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])

print("For Tkin={1} -> Trot={2}, pyspeckit's cold_ammonia fitter got:\n{0}".format(spc.specfit.parinfo, tkin, trot))
```

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    For Tkin=20.0 K -> Trot=17.776914063182385 K, pyspeckit's cold_ammonia fitter got:
    Param #0        tkin0 =      19.8365 +/-         107.868   Range:[2.7315,inf)
    Param #1         tex0 =      6.65999 +/-         2277.77   Range:[2.7315,inf)
    Param #2        ntot0 =           13 +/-         107.384   Range:    [5,25]
    Param #3       width0 =          0.5 +/-          2.6139   Range:   [0,inf)
    Param #4      xoff_v0 = -3.60464e-05 +/-               0
    Param #5      fortho0 =            0 (fixed)  Range:     [0,1]

```python
spc.specfit(fittype='cold_ammonia', guesses=[22.80, 6.6, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])
bestfit_coldammonia_temperature = spc.specfit.parinfo[0]
print("The best fit cold ammonia temperature is {0} for an input T_rot={1}".format(bestfit_coldammonia_temperature, trot))
```

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]
    The best fit cold ammonia temperature is Param #0        tkin0 =      19.8365 +/-     8.39537e-05   Range:[2.7315,inf) for an input T_rot=17.776914063182385 K

### Test 2: Use a different (& appropriate) tex for each level in the input model spectrum

If we use the exact tex for each line in the input model, in principle, the resulting fitted temperature should be more accurate.  However, at present, it looks dramatically incorrect

```python
tex11 = tex_radex(tkin=tkin, lineno=8)
tex22 = tex_radex(tkin=tkin, lineno=9)
print("tex11={0}, tex22={1} for tkin={2}, trot={3}".format(tex11,tex22,tkin,trot))
```

    tex11=6.789360524825584, tex22=6.533403488783305 for tkin=20.0 K, trot=17.776914063182385 K

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.

```python
spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                  tex={'oneone':tex11, 'twotwo':tex22},
                                  trot=trot.value,
                                  lines=['oneone','twotwo'])
spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])

print("For Tkin={1} -> Trot={2}, pyspeckit's cold_ammonia fitter got:\n{0}"
      .format(spc.specfit.parinfo, tkin, trot))
print("The best fit cold ammonia temperature is {0} for an input T_rot={1}"
      .format(bestfit_coldammonia_temperature, trot))
```

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    For Tkin=20.0 K -> Trot=17.776914063182385 K, pyspeckit's cold_ammonia fitter got:
    Param #0        tkin0 =      19.5548 +/-         105.814   Range:[2.7315,inf)
    Param #1         tex0 =      6.79164 +/-         2369.71   Range:[2.7315,inf)
    Param #2        ntot0 =      12.9982 +/-         106.007   Range:    [5,25]
    Param #3       width0 =     0.500008 +/-           2.586   Range:   [0,inf)
    Param #4      xoff_v0 =  1.59279e-05 +/-               0
    Param #5      fortho0 =            0 (fixed)  Range:     [0,1]
    The best fit cold ammonia temperature is Param #0        tkin0 =      19.8365 +/-     8.39537e-05   Range:[2.7315,inf) for an input T_rot=17.776914063182385 K

### Test 3: compare cold_ammonia to "normal" ammonia model to see why they differ

In a previous iteration of the ammonia model, there was a big (and incorrect) difference between the synthetic spectra from ammonia and cold_ammonia.  This is now something of a regression test for that error, which turned out to be from yet another incorrect indexing of the degeneracy.

```python
tkin = 20*u.K
trot = trot_radex(tkin=tkin)
dT0=41.18
print(tkin * (1 + (tkin.value/dT0)*np.log(1 + 0.6*np.exp(-15.7/tkin.value)))**-1)
print("tkin={0} trot={1} tex11={2} tex22={3}".format(tkin, trot, tex11, tex22))

spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                  tex={'oneone':tex11, 'twotwo':tex22},
                                  trot=trot.value,
                                  lines=['oneone','twotwo'])
spc_666 = test_ammonia.make_synthspec(lte=False, tkin=None,
                                      tex=6.66,
                                      trot=trot.value,
                                      lines=['oneone','twotwo'])
# this one is guaranteed different because tex = trot
spc_cold = test_ammonia.make_synthspec_cold(tkin=tkin.value,
                                            lines=['oneone','twotwo'])
```

    17.897313974001626 K
    tkin=20.0 K trot=17.776914063182385 K tex11=6.789360524825584 tex22=6.533403488783305
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: Assuming tex=trot [pyspeckit.spectrum.models.ammonia]
    WARNING: No header given.  Creating an empty one.
    WARNING: Assuming tex=trot [pyspeckit.spectrum.models.ammonia]
    WARNING: No header given.  Creating an empty one.

```python
spc[0].plotter(linewidth=3, alpha=0.5)
spc_666[0].plotter(axis=spc[0].plotter.axis, clear=False, color='r', linewidth=1, alpha=0.7)
spc_cold[0].plotter(axis=spc[0].plotter.axis, clear=False, color='b', linewidth=1, alpha=0.7)
```

The red and black look too different to me; they should differ only by a factor of (tex11-6.66)/6.66 or so.  Instead, they differ by a factor of 5-6.

```python
spc[0].data.max(), spc_666[0].data.max()
```

    (0.14522377315518076, 0.14324530551193299)

```python
spc[1].plotter()
spc_666[1].plotter(axis=spc[1].plotter.axis, clear=False, color='r')
spc_cold[1].plotter(axis=spc[1].plotter.axis, clear=False, color='b')
```

## RADEX analysis: T_rot vs T_kin vs T_ex ##

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
tex11s = np.array([tex_radex(tkin=t, lineno=8) for t in temperatures])
tex22s = np.array([tex_radex(tkin=t, lineno=9) for t in temperatures])
pl.plot(trots, tex11s)
pl.plot(trots, tex22s)
#pl.plot(tr_swift(temperatures), color='k')
pl.ylabel("$T_{ex}$")
pl.xlabel("$T_{rot} (2-2)/(1-1)$")
```

Apparently there are some discreteness problems but the ratio changes very little.

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
tex11s = np.array([tex_radex(tkin=t, lineno=8) for t in temperatures])
tex22s = np.array([tex_radex(tkin=t, lineno=9) for t in temperatures])
pl.plot(trots, tex11s/tex22s)
#pl.plot(tr_swift(temperatures), color='k')
pl.ylabel("$T_{ex} (2-2)/(1-1)$")
pl.xlabel("$T_{rot} (2-2)/(1-1)$")
```

## run pyspeckit tests

```python
from pyspeckit.spectrum.models.tests import test_ammonia
test_ammonia.test_ammonia_parlimits()
test_ammonia.test_ammonia_parlimits_fails()
test_ammonia.test_cold_ammonia()
test_ammonia.test_self_fit()
```

    WARNING: No header given.  Creating an empty one.

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,50 [pyspeckit.spectrum.interactive]

    WARNING: gnorm=0.   wa2=[0. 0. 0. 0. 0. 0.] [pyspeckit.mpfit.mpfit]
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,50 [pyspeckit.spectrum.interactive]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,753 [pyspeckit.spectrum.interactive]

    tex > trot in the ammonia model.  This is unphysical and suggests that you may need to constrain tex.  See ammonia_model_restricted_tex.

## More extensive (& expensive) tests: recovered Tkin

### 1. Check the recovered temperature as a function of input temperature using RADEX to simulate "real" data

```python
temperatures = np.array((10,15,20,25,30,35,40))
recovered_tkin = {}
recovered_column = {}
for tkin in temperatures:

    tbl = R(collider_densities={'ph2': 1e4}, temperature=tkin, column=1e13)
    tex11 = tbl['Tex'][8]
    tex22 = tbl['Tex'][9]
    trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
            np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
                   (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
            )

    spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                      tex={'oneone':tex11, 'twotwo':tex22},
                                      trot=trot.value,
                                      lines=['oneone','twotwo'])
    spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
    spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
                fixed=[False,False,False,False,False,True])

    recovered_tkin[tkin] = spc.specfit.parinfo['tkin0'].value
    recovered_column[tkin] = spc.specfit.parinfo['ntot0'].value
```

    invalid value encountered in true_divide
    overflow encountered in exp
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    The inputs to `brightness_temperature` have changed. Frequency is now the first input, and angular area is the second, optional input.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

```python
pl.xlabel("$T_K$")
pl.ylabel("Fitted $T_K$ from cold_ammonia")
pl.plot(recovered_tkin.keys(), recovered_tkin.values(), 'o')
pl.plot(temperatures, temperatures)
```

```python
pl.xlabel("$T_K$")
pl.ylabel("$|T_K-T_{fit}|/T_K$")
inp = np.array(list(recovered_tkin.keys()), dtype='float')
rslt = np.array(list(recovered_tkin.values()), dtype='float')
pl.plot(inp, np.abs(rslt-inp)/rslt, 'o')
```

### 2. Check the recovery as a function of column density

```python
pl.xlabel("$N(NH_3)$")
pl.ylabel("Fitted $N(NH_3)$ from cold_ammonia")
pl.plot(recovered_column.keys(), recovered_column.values(), 'o')
pl.plot(temperatures, temperatures*0+13)
```

```python

```


# Assignment 2 problem 2

```python
from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

from IPython.display import display, Math, Latex
import sympy as sp
from sympy.stats import P, E, variance, Die, Normal
```

## 7  Fatigue reliability analysis

```python
beta, alpha, T, T_0, D, e, e_2 = sp.symbols('beta alpha T T_0 D e e_2')
```

```python
beta_equation = sp.Eq(beta,E(sp.log(alpha)))
beta_equation
```

$\displaystyle \beta = \log{\left(\alpha \right)}$

```python
alpha = sp.stats.Normal('alpha',mean=0,std=1)
```

```python
a = sp.stats.rv.RandomSymbol(alpha)
```

```python

```