```python
#!/usr/bin/env python3
# coding: utf-8

# Load dependencies
import numpy as np
import matplotlib.pylab as plt
import pandas as pd

# Parameter estimation - model fitting
from lmfit import minimize
from lmfit import Parameters
from lmfit import fit_report

# For more information about lmfit: https://lmfit.github.io/lmfit-py/
# Highly suggest you read about emcee too: https://lmfit.github.io/lmfit-py/fitting.html#lmfit.minimizer.Minimizer.emcee
```

# Load the data from the Varennes NetZEB Library
Column descriptions:

    # Temperatures
    'T_ext': exterior temp, C
    'T_int': average interior temp, C
    'T_slab: slab temperature, 2 inch from surface, C
    'T_room_air_over_slab': average room temperature over the slab where there is a radiant system, C
    'T_room_air_no_slab': same but where there isn't, C

    # BIPV/T System
    'BIPVT_outletT': temperature at BIPV/T outlet, C
    'BIPVT_dT': ^ minus T_ext, K
    'BIPVT_heat: m*cp*dT = fanstate*rated_flow*density*cp*dT, kW

    # Heating/Cooling
    'geo_heat': m*cp*dT at the geothermal well inlet: how much heat are we rejecting/extracting into the ground, kW
    'heating_total': m*cp*dT at the heat pump condensor/hot side, kW
    'cooling_total': evaporator/cold side, kW
    'Q_slab': heat going into the radiant slab, kW
    'Q_rest_heating': heat going into the air, kW
    'Q_rest': heat - cooling going into the air, kW
    'heating_slab_ratio': heating going into slab over total, rest goes into the air
    'cooling_slab_ratio': cooling going into slab over total, rest goes into the air

    # Windows
    'window_state: 0: closed, 1: open

    # Solar
    'DNI': direct normal irradiance, kW/m^2
    'GHI': global horizontal irradiance, kW/m^2
    'DHI': diffuse horizontal irradiance, kW/m^2
    'solar_projected_PV': unitized solar radiation incident on PV system, kW/m^2

    # Electrical
    'cons': total electrical consumption power, kW
    'prod: total electrical production power, kW
    'cons_noHVAC': approximate electrical consumption without the HVAC (cons - heating_total/4 - cooling_total/3.5), kW

```python
# Load data from csv file, assign index and assure it's a datetime object
df = pd.read_csv("data_varennes_subset.csv.zip", index_col=[0], parse_dates=[0])
print(df.head())
```

                           T_ext    T_int  BIPVT_outletT  BIPVT_dT  BIPVT_heat  \
    Datetime
    2018-01-01 00:00:00 -21.8566  18.4578       -18.6184    3.2382         0.0
    2018-01-01 00:15:00 -21.8434  18.4086       -18.6614    3.1820         0.0
    2018-01-01 00:30:00 -21.9561  18.3142       -18.6459    3.3102         0.0
    2018-01-01 00:45:00 -22.0408  18.2528       -18.7473    3.2935         0.0
    2018-01-01 01:00:00 -22.1299  18.1697       -18.6353    3.4946         0.0

                          geo_heat  heating_total  cooling_total  window_state  \
    Datetime
    2018-01-01 00:00:00 -54.118034      38.025668            0.0      0.335996
    2018-01-01 00:15:00 -50.619191      26.322083            0.0      0.336057
    2018-01-01 00:30:00  -9.066210      13.017722            0.0      0.336118
    2018-01-01 00:45:00 -37.896533      10.618306            0.0      0.336180
    2018-01-01 01:00:00 -34.623417      10.781639            0.0      0.336241

                         heating_slab_ratio  ...  prod  cons_noHVAC  \
    Datetime                                 ...
    2018-01-01 00:00:00            0.199966  ...   0.0     1.740303
    2018-01-01 00:15:00            0.282259  ...   0.0     6.494799
    2018-01-01 00:30:00            0.293446  ...   0.0    13.072209
    2018-01-01 00:45:00            0.281554  ...   0.0    13.261424
    2018-01-01 01:00:00            0.268706  ...   0.0    14.546990

                         Q_slab_heating    Q_slab  Q_rest_heating     Q_rest  DNI  \
    Datetime
    2018-01-01 00:00:00     7603.853490  7.603853       30.421814  30.421814  0.0
    2018-01-01 00:15:00     7429.656592  7.429657       18.892426  18.892426  0.0
    2018-01-01 00:30:00     3819.993391  3.819993        9.197729   9.197729  0.0
    2018-01-01 00:45:00     2989.624193  2.989624        7.628682   7.628682  0.0
    2018-01-01 01:00:00     2897.089583  2.897090        7.884549   7.884549  0.0

                         GHI  DHI  solar_projected_PV
    Datetime
    2018-01-01 00:00:00  0.0  0.0                 0.0
    2018-01-01 00:15:00  0.0  0.0                 0.0
    2018-01-01 00:30:00  0.0  0.0                 0.0
    2018-01-01 00:45:00  0.0  0.0                 0.0
    2018-01-01 01:00:00  0.0  0.0                 0.0

    [5 rows x 25 columns]

# Approach
Just to make things a bit simple, we will train the model on one day of data and test it on the following day; both days will be during the week to assume a consistent occupancy schedule and setpoint schedule.

We will try different models to see whether we're improving.

We will represent models 3, 4 and 5 in a simplified discrete time-invariant state-space formulation; it's much cleaner, I promise!

### Model 1: Naive
$$ T_{int}(t+1) = \alpha T_{int}(t) + \phi T_{ext}(t) $$

### Model 2: Naive Difference with exterior
$$ T_{int}(t+1) = \alpha [T_{ext}(t) - T_{int}(t)] $$

### Model 3: Delta temperature
$$
\begin{align}
T_{int}(t+1) - T_{int}(t) &= \alpha [T_{ext}(t) - T_{int}(t)] \\
T_{int}(t+1) &= T_{int}(t) + \alpha [T_{ext}(t) - T_{int}(t)] \\
T_{int}(t+1) &= (1-\alpha) T_{int}(t) + \alpha T_{ext}(t) \\
\end{align}
$$

### Model 4: Delta temperature
$$ T_{int}(t+1) - T_{int}(t) = \alpha [T_{ext}(t) - T_{int}(t)] + \beta \dot{Q}_{\textrm{cons-noHVAC}} $$

### Model 5: Delta temperature
$$ T_{int}(t+1) - T_{int}(t) = \alpha [T_{ext}(t) - T_{int}(t)] + \beta \dot{Q}_{\textrm{HVAC}} + \gamma \textrm{GHI} $$

### Fitting the model using mean squared error (MSE)
$$ \min_{\alpha, \beta, \gamma}{\sum_{i=t}^{data}{(T_{int}(i) - \hat{T}_{int}(i))^2}} $$

---
### Model 1: Naive
$$ T_{int}(t+1) = \alpha T_{int}(t) + \phi T_{ext}(t) $$

```python
# 2018-01-16: Tuesday
df_train = df.loc["2018-01-16"].copy()
df_test = df.loc["2018-01-17"].copy()

# TRAIN
# input
Tint_t_train = df_train["T_int"].iloc[:-1].values
Text_t_train = df_train["T_ext"].iloc[:-1].values
# output
Tint_tp1_train = df_train["T_int"].iloc[1:].values

# TEST
# input
Tint_t_test = df_test["T_int"].iloc[:-1].values
Text_t_test = df_test["T_ext"].iloc[:-1].values
# output
Tint_tp1_test = df_test["T_int"].iloc[1:].values
```

```python
# Optimization function
def opti_fun(
    params: Parameters,
    Tint_t_train: np.ndarray,
    Text_t_train: np.ndarray,
    Tint_tp1_train: np.ndarray,
) -> np.ndarray:

    Tint_tp1_train_pred = params["alpha"] * Tint_t_train + params["phi"] * Text_t_train

    error = Tint_tp1_train - Tint_tp1_train_pred
    return error  # least squares solved will square and sum for us :)

# Parameters we are trying to fits
params = Parameters()
params.add("alpha", value = 0.5)
params.add("phi", value = 0.5)

# Additional information needed to describe the problem
opti_kwargs = {
    "Tint_t_train": Tint_t_train,
    "Text_t_train": Text_t_train,
    "Tint_tp1_train": Tint_tp1_train,
}

# Run!
results = minimize(fcn=opti_fun, params=params, method='leastsq', kws=opti_kwargs)

# Print results
print(fit_report(results))
```

    [[Fit Statistics]]
        # fitting method   = leastsq
        # function evals   = 7
        # data points      = 95
        # variables        = 2
        chi-square         = 0.31744548
        reduced chi-square = 0.00341339
        Akaike info crit   = -537.625977
        Bayesian info crit = -532.518224
    [[Variables]]
        alpha:  1.00260422 +/- 0.00133209 (0.13%) (init = 0.5)
        phi:    0.00411429 +/- 0.00213328 (51.85%) (init = 0.5)
    [[Correlations]] (unreported correlations are < 0.100)
        C(alpha, phi) = 0.974

```python

```

```python
# Get optimal values
alpha = results.params["alpha"].value
phi = results.params["phi"].value

# Let's see how it performs
Tint_tp1_test_pred = alpha * Tint_t_test + phi * Text_t_test

# Place in correct location (remember, this is for t+1!)
# error = true - predicted
error = Tint_tp1_test - Tint_tp1_test_pred

plt.plot(Tint_tp1_test, label = "True")
plt.plot(Tint_tp1_test_pred, label = "Predicted")
plt.title("Model 1")
plt.legend()
plt.show()

plt.plot(error)
plt.title("Error = True - Pred")
plt.show()

plt.hist(error, bins=10)
plt.title("Error Histogram")
plt.show()
```

---
### Model 1: Naive (attempt 2!)
$$
\min_{\alpha, \beta, \gamma}{\sum_{i=t}^{nt}{(T_{int}(i) - \hat{T}_{int}(i))^2}} \\
\begin{align}
\textrm{s.t.} \\
& \hat{T}_{int}(1) = \alpha T_{int}(0) + \phi T_{ext}(0) \\
& \hat{T}_{int}(t+1) = \alpha \hat{T}_{int}(t) + \phi T_{ext}(t) ~~ | ~ t = 1, ~..., ~nt-1
\end{align}
$$

```python
# 2018-01-16: Tuesday
df_train = df.loc["2018-01-16"].copy()
df_test = df.loc["2018-01-17"].copy()

# TRAIN
# input
Tint_t_train = df_train["T_int"].iloc[:-1].values
Text_t_train = df_train["T_ext"].iloc[:-1].values
# output
Tint_tp1_train = df_train["T_int"].iloc[1:].values

# TEST
# input
Tint_t_test = df_test["T_int"].iloc[:-1].values
Text_t_test = df_test["T_ext"].iloc[:-1].values
# output
Tint_tp1_test = df_test["T_int"].iloc[1:].values
```

```python
# Optimization function
def opti_fun(
    params: Parameters,
    Tint_t_train: np.ndarray,
    Text_t_train: np.ndarray,
    Tint_tp1_train: np.ndarray,
) -> np.ndarray:

    nt = len(Tint_tp1_train)
    Tint_tp1_train_pred = np.zeros_like(Tint_tp1_train)

    # NOTE CAREFUL: first entry in Tint_tp1_train_pred (idx=0) is for t=1!
    for t in range(nt):
        if t == 0:
            Tint_tp1_train_pred[t] = params["alpha"] * Tint_t_train[t] + params["phi"] * Text_t_train[t]
        else:
            Tint_tp1_train_pred[t] = params["alpha"] * Tint_tp1_train_pred[t-1] + params["phi"] * Text_t_train[t]

    error = Tint_tp1_train - Tint_tp1_train_pred
    return error  # least squares solved will square and sum for us :)

# Parameters we are trying to fits
params = Parameters()
params.add("alpha", value = 0.5)
params.add("phi", value = 0.5)

# Additional information needed to describe the problem
opti_kwargs = {
    "Tint_t_train": Tint_t_train,
    "Text_t_train": Text_t_train,
    "Tint_tp1_train": Tint_tp1_train,
}

# Run!
results = minimize(fcn=opti_fun, params=params, method='leastsq', kws=opti_kwargs)

# Print results
print(fit_report(results))
```

    [[Fit Statistics]]
        # fitting method   = leastsq
        # function evals   = 49
        # data points      = 95
        # variables        = 2
        chi-square         = 28.9915804
        reduced chi-square = 0.31173742
        Akaike info crit   = -108.752786
        Bayesian info crit = -103.645033
    [[Variables]]
        alpha:  1.00308742 +/- 7.0862e-04 (0.07%) (init = 0.5)
        phi:    0.00398727 +/- 0.00109073 (27.36%) (init = 0.5)
    [[Correlations]] (unreported correlations are < 0.100)
        C(alpha, phi) = 0.998

```python
# Get optimal values
alpha = results.params["alpha"].value
phi = results.params["phi"].value

# Let's see how it performs
nt = len(Tint_tp1_test)
Tint_tp1_test_pred = np.zeros_like(Tint_tp1_test)

# NOTE CAREFUL: first entry in Tint_tp1_test_pred (idx=0) is for t=1!
for t in range(nt):
    if t == 0:
        Tint_tp1_test_pred[t] = alpha * Tint_t_test[t] + phi * Text_t_test[t]
    else:
        Tint_tp1_test_pred[t] = alpha * Tint_tp1_test_pred[t-1] + phi * Text_t_test[t]

# Place in correct location (remember, this is for t+1!)
# error = true - predicted
error = Tint_tp1_test - Tint_tp1_test_pred

plt.plot(Tint_tp1_test, label = "True")
plt.plot(Tint_tp1_test_pred, label = "Predicted")
plt.title("Model 1: Done Properly")
plt.legend()
plt.show()

plt.plot(error)
plt.title("Error = True - Pred")
plt.show()

plt.hist(error, bins=10)
plt.title("Error Histogram")
plt.show()
```

---
### Model 2
Do this as an execise.

Do you see an improvement over Model 1? What does the fit look like? Is the model able to follow the peaks or does it draw a line through the middle?

---
### Model 3
$$
\min_{\alpha, \beta, \gamma}{\sum_{i=t}^{nt}{(T_{int}(i) - \hat{T}_{int}(i))^2}} \\
\begin{align}
\textrm{s.t.} \\
& A = 1 - \alpha \\
& C = \alpha \\
& \hat{T}_{int}(1) = A T_{int}(0) + C T_{ext}(0) \\
& \hat{T}_{int}(t+1) = A \hat{T}_{int}(t) + C T_{ext}(t) ~~ | ~ t = 1, ~..., ~nt-1
\end{align}
$$

```python
def pred_recursive(
    A,
    X_initial,
    B = None,
    U = None,
    C = None,
    W = None,
):
    """
    Simplified approach where X is the estimated state:
    X(0) = X_initial
    X(t+1) = A*X(t) + B*U(t) + C*W(t)
    """
    # NOTE CAREFUL: first entry in predicted X (idx=0) is for t=1!
    # Only exogenous inputs
    if (B is None) and (C is not None):
        X = np.zeros((len(W), A.shape[1]))
        X[0,] = A @ X_initial + C @ W[0,]
        for t in range(1, len(W)):
            X[t,] = A @ X[t-1,] + C @ W[t,]
        return X

    # Only control inputs
    if (B is not None) and (C is None):
        X = np.zeros((len(U), A.shape[1]))
        X[0,] = A @ X_initial + B @ U[0,]
        for t in range(1, len(U)):
            X[t,] = A @ X[t-1,] + B @ U[t,]
        return X

    # Both control and exogenous inputs
    if (B is not None) and (C is not None):
        X = np.zeros((len(U), A.shape[1]))
        X[0,] = A @ X_initial + B @ U[0,] + C @ W[0,]
        for t in range(1, len(U)):
            X[t,] = A @ X[t-1,] + B @ U[t,] + C @ W[t,]
        return X

def state_space_pred_recursive_return_error(
    params: Parameters,
    X_true: np.ndarray,
    U: np.ndarray = None,
    W: np.ndarray = None,
    build_ABC = None,
) -> np.ndarray:
    """ State-space model formulation recursive prediction returning error."""

    # Build ABC
    A, B, C = build_ABC(params)

    # Run predictions
    X_initial = X_train[0, np.newaxis]  # keep it as a matrix
    X_pred = pred_recursive(A, X_initial, B, U, C, W)

    # Get error and ship
    error = X_true[1:] - X_pred
    return error  # least squares solved will square and sum for us :)
```

```python
# 2018-01-16: Tuesday
df_train = df.loc["2018-01-16"].copy()
df_test = df.loc["2018-01-17"].copy()

# Define our dependent/independent variables
X_cols = ["T_int"]  # state_columns
W_cols = ["T_ext"]  # exogenous_columns

# TRAIN
X_train = df_train[X_cols].values
W_train = df_train[W_cols].iloc[:-1].values

# TEST
X_test = df_test[X_cols].values
W_test = df_test[W_cols].iloc[:-1].values

```

```python
def build_ABC_model_3(params):
    # Place parameters in A, B, C matrices
    A = np.array([[1 - params["alpha"]]])
    B = None
    C = np.array([[params["alpha"]]])
    return A, B, C

# Parameters we are trying to fits
params = Parameters()
params.add("alpha", value = 0.5)

# Additional information needed to describe the problem
opti_kwargs = {
    "X_true": X_train,
    "W": W_train,
    "build_ABC": build_ABC_model_3,  # pass the ABC generator
}

# Run!
results = minimize(
    fcn = state_space_pred_recursive_return_error,
    params = params,
    method = 'leastsq',
    kws = opti_kwargs
)

# Print results
print(fit_report(results))
```

    [[Fit Statistics]]
        # fitting method   = leastsq
        # function evals   = 20
        # data points      = 95
        # variables        = 1
        chi-square         = 34.2808527
        reduced chi-square = 0.36468992
        Akaike info crit   = -94.8325427
        Bayesian info crit = -92.2786658
    [[Variables]]
        alpha: -3.1383e-04 +/- 3.3729e-05 (10.75%) (init = 0.5)

```python
# Assign optimized results.params to params
params = results.params

# build_ABC_model_3
A, B, C = build_ABC_model_3(params)
print(f"A: {A}")
print(f"B: {B}")
print(f"C: {C}")

# Run predictions
X_initial = X_test[0, np.newaxis]  # keep it as a matrix
X_pred = pred_recursive(A, X_initial, C, W_test)

# Get error and ship
error = X_test[1:] - X_pred

# Plot results
plt.plot(X_test, label = "True")
plt.plot(X_pred, label = "Predicted")
plt.title("Model 3")
plt.legend()
plt.show()

plt.plot(error)
plt.title("Error = True - Pred")
plt.show()

plt.hist(error, bins=10)
plt.title("Error Histogram")
plt.show()
```

---
### Model 4
Do this as an execise.

Do you see an improvement over Model 3?
What does the fit look like?
Is the model able to follow the peaks or does it draw a line through the middle?

Think about the degrees of freedom.

---
### Model 5: Delta temperature
$$ T_{int}(t+1) - T_{int}(t) = \alpha [T_{ext}(t) - T_{int}(t)] + \beta \dot{Q}_{\textrm{HVAC}} + \gamma \textrm{GHI} $$

$$
\min_{\alpha, \beta, \gamma}{\sum_{i=t}^{nt}{(T_{int}(i) - \hat{T}_{int}(i))^2}} \\
\begin{align}
\textrm{s.t.} \\
& A = [[1 - \alpha]] \\
& B = [[\beta]] \\
& C = [[\alpha, \gamma]] \\
& \hat{T}_{int}(1) = A T_{int}(0) + B \dot{Q}_{\textrm{HVAC}}(0) + C [T_{ext}(0), \textrm{GHI}] \\
& \hat{T}_{int}(t+1) = A \hat{T}_{int}(t) + B \dot{Q}_{\textrm{HVAC}}(t) + C [T_{ext}(t), \textrm{GHI}] ~~ | ~ t = 1, ~..., ~nt-1 \\
\end{align}
$$

Matrices $A$, $B$, and $C$ are matrices with number of rows matching the number of states and columns matching features.

```python
# 2018-01-16: Tuesday
df_train = df.loc["2018-01-16"].copy()
df_test = df.loc["2018-01-17"].copy()

# Define our dependent/independent variables
X_cols = ["T_int"]  # state_columns
U_cols = ["heating_total"]  # control_columns
W_cols = ["T_ext", "GHI"]  # exogenous_columns

# TRAIN
X_train = df_train[X_cols].values
U_train = df_train[U_cols].iloc[:-1].values
W_train = df_train[W_cols].iloc[:-1].values

# TEST
X_test = df_test[X_cols].values
U_test = df_test[U_cols].iloc[:-1].values
W_test = df_test[W_cols].iloc[:-1].values
```

```python
def build_ABC_model_5(params):
    # Place parameters in A, B, C matrices
    A = np.array([[1 - params["alpha"]]])
    B = np.array([[params["beta"]]])
    C = np.array([[params["alpha"], params["gamma"]]])
    return A, B, C

# Parameters we are trying to fits
params = Parameters()
params.add("alpha", value = 0.5)
params.add("beta", value = 0.5)
params.add("gamma", value = 0.5)

# Additional information needed to describe the problem
opti_kwargs = {
    "X_true": X_train,
    "U": U_train,
    "W": W_train,
    "build_ABC": build_ABC_model_5,  # pass the ABC generator
}

# Run!
results = minimize(
    fcn = state_space_pred_recursive_return_error,
    params = params,
    method = 'leastsq',
    kws = opti_kwargs
)

# Print results
print(fit_report(results))
```

    [[Fit Statistics]]
        # fitting method   = leastsq
        # function evals   = 52
        # data points      = 95
        # variables        = 3
        chi-square         = 12.5116919
        reduced chi-square = 0.13599665
        Akaike info crit   = -186.585267
        Bayesian info crit = -178.923636
    [[Variables]]
        alpha:  5.3594e-04 +/- 1.1918e-04 (22.24%) (init = 0.5)
        beta:   1.2115e-04 +/- 1.2698e-04 (104.81%) (init = 0.5)
        gamma:  0.30313001 +/- 0.05008927 (16.52%) (init = 0.5)
    [[Correlations]] (unreported correlations are < 0.100)
        C(alpha, beta)  = 0.845
        C(beta, gamma)  = -0.841
        C(alpha, gamma) = -0.438

```python
# Assign optimized results.params to params
params = results.params

# build_ABC_model_3
A, B, C = build_ABC_model_5(params)
print(f"A: {A}")
print(f"B: {B}")
print(f"C: {C}")

# Run predictions
X_initial = X_test[0, np.newaxis]  # keep it as a matrix
X_pred = pred_recursive(A, X_initial, B, U_test, C, W_test)

# Get error and ship
error = X_test[1:] - X_pred

# Plot results
plt.plot(X_test, label = "True")
plt.plot(X_pred, label = "Predicted")
plt.title("Model 5")
plt.legend()
plt.show()

plt.plot(error)
plt.title("Error = True - Pred")
plt.show()

plt.hist(error, bins=10)
plt.title("Error Histogram")
plt.show()
```

## Conclusion
+ What's going on as we add terms?
+ Can we put the features arbitrarily?
+ How are we injecting our domain expertise into the problem? Comment on (a) the feature engineering, (b) the model structure and (c) the loss function.
+ What would a neural network have done?
+ What would happen if we used a lot more data?
+ If we had 2 years of data, how would you reformulate the fit problem?
+ If there's no more information left in the residual (error), what would it look like?
+ What's the interpretation of the A, B and C matrices?
+ In the general state-space representation, there is the concept of hidden states, what are we assuming here?
+ If we wanted to add hidden states, what would they represent?


#### Procesamiento Digital de Señales

# Trabajo Práctico Nº0
#### Nombre y Apellido

# Introducción
Jupyter Notebook es una herramienta para la confección de reportes técnicos, dado que permite la interacción en el mismo ambiente de:
1. un procesador de texto elemental (formato Markdown) que permite resaltar texto, en forma de *itálica* o **negrita** de manera muy legible (haciendo doble click en este texto podrás ver el código fuente estilo Markdown). Cuenta con estilos predefinidos:

# Título 1
## Título 2
### Título 3

y también la capacidad de incluir enlaces a otras páginas, como por ejemplo [esta página](https://medium.com/ibm-data-science-experience/markdown-for-jupyter-notebooks-cheatsheet-386c05aeebed) donde encontrarás más funcionalidades del lenguaje **Markdown**

2. capacidad para incluir lenguaje matemático estilo LaTex, tanto de forma presentada

\begin{equation}
T(z) = \frac{Y(z)}{X(z)} = \frac{ b_2 \, z^{-2} + b_1 \,  z^{-1} + b_0 }
{a_2 \, z^{-2} + a_1 \,  z^{-1} + a_0}
\end{equation}

como *inline* en el propio párrafo $y[k] = \frac{1}{a_0} \left( \sum_{m=0}^{M} b_m \; x[k-m] - \sum_{n=1}^{N} a_n \; y[k-n] \right) $

3. La posibilidad de incluir scripts en Python, como los que usaremos para las simulaciones en los TPs de la materia. En este caso usaremos el *testbench0.py* como ejemplo. Una vez que lo probamos y estamos seguros que funciona de forma esperada en *Spyder*, podemos incluir los resultados de la simulación de manera casi transparente. Solo tenemos que agregar una celda de código donde incluimos el código, y los resultados directamente quedan incluidos en este documento.

```python
# Módulos para Jupyter
import warnings
warnings.filterwarnings('ignore')

import numpy as np
import matplotlib as mpl
#%%  Inicialización de librerías
# Setup inline graphics: Esto lo hacemos para que el tamaño de la salida,
# sea un poco más adecuada al tamaño del documento
mpl.rcParams['figure.figsize'] = (10,10)

import matplotlib.pyplot as plt
import pdsmodulos as pds

#%% Esto tiene que ver con cuestiones de presentación de los gráficos,
# NO ES IMPORTANTE
fig_sz_x = 14
fig_sz_y = 13
fig_dpi = 80 # dpi

fig_font_family = 'Ubuntu'
fig_font_size = 16

plt.rcParams.update({'font.size':fig_font_size})
plt.rcParams.update({'font.family':fig_font_family})

##############################################
#%% A partir de aquí comienza lo IMPORTANTE #
#############################################

def my_testbench( sig_type ):

    # Datos generales de la simulación
    fs = 1000.0 # frecuencia de muestreo (Hz)
    N = 1000   # cantidad de muestras

    ts = 1/fs # tiempo de muestreo
    df = fs/N # resolución espectral

    # grilla de sampleo temporal
    tt = np.linspace(0, (N-1)*ts, N).flatten()

    # grilla de sampleo frecuencial
    ff = np.linspace(0, (N-1)*df, N).flatten()

    # Concatenación de matrices:
    # guardaremos las señales creadas al ir poblando la siguiente matriz vacía
    x = np.array([], dtype=np.float).reshape(N,0)
    ii = 0

    # estructuras de control de flujo
    if sig_type['tipo'] == 'senoidal':

        # calculo cada senoidal de acuerdo a sus parámetros
        for this_freq in sig_type['frecuencia']:
            # prestar atención que las tuplas dentro de los diccionarios también pueden direccionarse mediante "ii"
            aux = sig_type['amplitud'][ii] * np.sin( 2*np.pi*this_freq*tt + sig_type['fase'][ii] )
            # para concatenar horizontalmente es necesario cuidar que tengan iguales FILAS
            x = np.hstack([x, aux.reshape(N,1)] )
            ii += 1

    elif sig_type['tipo'] == 'ruido':

        # calculo cada señal de ruido incorrelado (blanco), Gausiano de acuerdo a sus parámetros
        # de varianza
        for this_var in sig_type['varianza']:
            aux = np.sqrt(this_var) * np.random.randn(N,1)
            # para concatenar horizontalmente es necesario cuidar que tengan iguales FILAS
            x = np.hstack([x, aux] )

        # Podemos agregar algún dato extra a la descripción de forma programática
        # {0:.3f} significa 0: primer argunmento de format
        # .3f formato flotante, con 3 decimales
        # $ ... $ indicamos que incluiremos sintaxis LaTex: $\hat{{\sigma}}^2$
        sig_props['descripcion'] = [ sig_props['descripcion'][ii] + ' - $\hat{{\sigma}}^2$ :{0:.3f}'.format( np.var(x[:,ii]))  for ii in range(0,len(sig_props['descripcion'])) ]

    else:

        print("Tipo de señal no implementado.")
        return

    #%% Presentación gráfica de los resultados

    plt.figure(1)
    line_hdls = plt.plot(tt, x)
    plt.title('Señal: ' + sig_type['tipo'] )
    plt.xlabel('tiempo [segundos]')
    plt.ylabel('Amplitud [V]')
    #    plt.grid(which='both', axis='both')

    # presentar una leyenda para cada tipo de señal
    axes_hdl = plt.gca()

    # este tipo de sintaxis es *MUY* de Python
    axes_hdl.legend(line_hdls, sig_type['descripcion'], loc='upper right'  )

    plt.show()

```

Dado que nuestro *testbench* ha sido desarrollado de manera funcional, llamando a la función *my_testbench()* con diferentes parámetros, podemos lograr funcionalidades diferentes, como mostramos a continuación primero con una senoidal:

```python
sig_props = { 'tipo': 'senoidal',
              'frecuencia': (3, 100, 20), # Uso de tuplas para las frecuencias
              'amplitud':   (1, 10,  1),
              'fase':       (0, 0,  0)
             }
# Como también puedo agregar un campo descripción de manera programática
# este tipo de sintaxis es *MUY* de Python
sig_props['descripcion'] = [ str(a_freq) + ' Hz' for a_freq in sig_props['frecuencia'] ]

# Invocamos a nuestro testbench exclusivamente:
my_testbench( sig_props )
```

Y ahora con una señal aleatoria, en este caso ruido blanco Gaussiano incorrelado de varianza $\sigma^2$:

```python
# Usar CTRL+1 para comentar o descomentar el bloque de abajo.
sig_props = { 'tipo': 'ruido',
              'varianza': (1, 1, 1) # Uso de tuplas para las frecuencias
             }
sig_props['descripcion'] = [ '$\sigma^2$ = ' + str(a_var) for a_var in sig_props['varianza'] ]

# Invocamos a nuestro testbench exclusivamente:
my_testbench( sig_props )

```

Como puede verse en la figura anterior, al samplear una distribución estadística de media nula y varianza $\sigma^2=1$, obtenemos realizaciones cuyo parámetro $\sigma^2$ estimado, es decir $\hat\sigma^2$, tienen una desviación respecto al verdadero valor (sesgo). Nos ocuparemos de estudiar el sesgo y la varianza de algunos estimadores cuando veamos **Estimación Espectral**.

# Una vez terminado ...
Una vez que hayas termiando con la confección del documento, podemos utilizar una ventaja muy importante de este tipo de documentos que es la posibilidad de compartirlos *online* mediante la [página de nbviewer](http://nbviewer.jupyter.org/). Para ello es necesario que tu notebook y todos los recursos asociados estén alojados en un repositorio de [Github](https://github.com/). Como ejemplo, podés ver este mismo documento disponible [online](http://nbviewer.jupyter.org/github/marianux/pdstestbench/blob/master/notebook0.ipynb).

# Hola Mundo..
Este es un texto

```python

```


```python
from IPython.core.display import display_html, HTML
from urllib.request import urlopen

cssurl = 'http://j.mp/1DnuN9M'
display_html(urlopen(cssurl).read(), raw=True)
```

<link href='http://fonts.googleapis.com/css?family=EB+Garamond' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Lato:100,100italic' rel='stylesheet' type='text/css'>

<style>
	@font-face{
   		font-family: "Computer Modern";
        	src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');}

        div.cell{
	        max-width:    800px;
	        margin-left:  auto;
	        margin-right: auto;}

	h1 {
        	font-family: 'Lato', sans-serif;}

	h2 {
        	font-family: 'Lato', serif;}

	h3{
		font-family:   'Lato', serif;
        	margin-top:    12px;
		margin-bottom: 3px;}

        h4{
		font-family: 'Lato', serif;}

	h5 {
        	font-family: 'Lato', serif;}

	div.text_cell_render{
        	font-family:  'EB Garamond', 'Computer Modern', 'Helvetica Neue', Arial, Helvetica, Geneva, sans-serif;
	        line-height:  140%;
	        font-size:    14pt;
		max-width:    650px;
		margin-left:  auto;
        	margin-right: auto;}

	.CodeMirror{
        	font-family: 'Consolas';}

	.text_cell_render h1 {
	        font-weight:   100;
	        font-size:     48pt;
		line-height:   100%;
	        color:         #CD2305;
	        margin-bottom: 0.5em;
	        margin-top:    0.5em;
	        display:       block;}

	.text_cell_render h2 {
        	font-weight:   100;
        	font-size:     36pt;}

	.text_cell_render h3 {
        	font-weight:   100;
        	font-size:     21pt;}

	.text_cell_render h4 {
        	font-weight:   100;
        	font-size:     18pt;}

	.text_cell_render h5 {
	        font-weight:   100;
	        font-size:     16pt;
	        color:         #CD2305;
	        font-style:    italic;
	        margin-bottom: 0.5em;
	        margin-top:    0.5em;
	        display:       block;}

	.text_cell_render h6 {
	        font-weight:   100;
	        font-size:     14pt;
	        color:         #CD2305;
	        font-style:    italic;
	        margin-bottom: 0.5em;
	        margin-top:    0.5em;
	        display:       block;}

	.warning{
	    	background-color: #FFFFE0;
	    	border-color: #FFCC99;
	    	border-left: 5px solid #FFCC99;
	    	padding: 0.5em;}

	.error{
	    	background-color: #fcf2f2;
	    	border-color: #dFb5b4;
	    	border-left: 5px solid #dfb5b4;
	    	padding: 0.5em;}

</style>

# Examen Final

## Modelado de pendubot

### Variables de estado, posiciones y velocidades

$$
q =
\begin{pmatrix}
q_1 \\
q_2
\end{pmatrix} \implies
\dot{q} =
\begin{pmatrix}
\dot{q}_1 \\
\dot{q}_2
\end{pmatrix}
$$

Las posiciones de las juntas estan dadas por:

$$
\begin{align}
\begin{pmatrix}
x_1 & y_1
\end{pmatrix} &=
\begin{pmatrix}
l_1 \cos{(q_1)} & l_1 \sin{(q_1)}
\end{pmatrix} \\
\begin{pmatrix}
x_2 & y_2
\end{pmatrix} &=
\begin{pmatrix}
l_1 \cos{(q_1)} + l_2 \cos{(q_1 + q_2)} & l_1 \sin{(q_1)} + l_2 \sin{(q_1 + q_2)}
\end{pmatrix}
\end{align}
$$

y por lo tanto, las velocidades estarán dadas por:

$$
\begin{align}
\begin{pmatrix}
\dot{x}_1 & \dot{y}_1
\end{pmatrix} &=
\begin{pmatrix}
-l_1 \sin{(q_1)} \dot{q}_1 & l_1 \cos{(q_1)} \dot{q}_1
\end{pmatrix} \\
\begin{pmatrix}
\dot{x}_2 & \dot{y}_2
\end{pmatrix} &=
\begin{pmatrix}
-l_1 \sin{(q_1)} \dot{q}_1 - l_2 \sin{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2) & l_1 \cos{(q_1)} \dot{q}_1 + l_2 \cos{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2)
\end{pmatrix}
\end{align}
$$

### Energía cinética

Considerando que la velocidad $v_i$ es la norma de estas componentes, $v_i = \sqrt{\dot{x}_i^2 + \dot{y}_i^2}$, la energía cinética del sistema esta dada dada por:

$$
\begin{align}
K &= K_1 + K_2 \\
&= \frac{1}{2} m_1 v_1^2 + \frac{1}{2} m_2 v_2^2
\end{align}
$$

con $v_1^2$ dada por:

$$
\begin{align}
v_1^2 &= \dot{x}_1^2 + \dot{y}_1^2 \\
&= l_1^2 \sin^2{(q_1)} \dot{q}_1^2 + l_1^2 \cos^2{(q_1)} \dot{q}_1^2 \\
&= l_1^2 \dot{q}_1^2
\end{align}
$$

y $v_2^2$:

$$
\begin{align}
v_2^2 &= \dot{x}_2^2 + \dot{y}_2^2 \\
&= \left( -l_1 \sin{(q_1)} \dot{q}_1 - l_2 \sin{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2) \right)^2 + \left( l_1 \cos{(q_1)} \dot{q}_1 + l_2 \cos{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2) \right)^2 \\
&= l_1^2 \dot{q}_1^2 + l_2^2 (\dot{q}_1 + \dot{q}_2)^2 + 2\left[ l_1 \sin{(q_1)} \dot{q}_1 \cdot l_2 \sin{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2)
+ l_1 \cos{(q_1)} \dot{q}_1 \cdot l_2 \cos{(q_1 + q_2)} (\dot{q}_1 + \dot{q}_2) \right] \\
&= l_1^2 \dot{q}_1^2 + l_2^2 (\dot{q}_1 + \dot{q}_2)^2 + 2 l_1 l_2 \left[ \sin{(q_1)} \sin{(q_1 + q_2)} + \cos{(q_1)} \cos{(q_1 + q_2)} \right] \dot{q}_1 (\dot{q}_1 + \dot{q}_2) \\
&= l_1^2 \dot{q}_1^2 + l_2^2 (\dot{q}_1 + \dot{q}_2)^2 + 2 l_1 l_2 \cos{(q_2)} \dot{q}_1 (\dot{q}_1 + \dot{q}_2)
\end{align}
$$

por lo que las energías cinéticas de cada junta quedan:

$$
\begin{align}
K_1 &= \frac{1}{2} m_1 v_1^2 = \frac{1}{2} m_1 l_1^2 \dot{q}_1^2 \\
K_2 &= \frac{1}{2} m_2 v_2^2 = \frac{1}{2} m_2 \left[ l_1^2 \dot{q}_1^2 + l_2^2 (\dot{q}_1 + \dot{q}_2)^2 + 2 l_1 l_2 \cos{(q_2)} \dot{q}_1 (\dot{q}_1 + \dot{q}_2) \right] \\
&= \frac{1}{2} m_2 \left[ l_1^2 \dot{q}_1^2 + l_2^2 \dot{q}_1^2 + l_2^2 \dot{q}_2^2 + 2 l_2^2 \dot{q}_1 \dot{q}_2 + 2 l_1 l_2 \cos{(q_2)} \dot{q}_1^2 + 2 l_1 l_2 \cos{(q_2)} \dot{q}_1 \dot{q}_2 \right] \\
&= \frac{1}{2} m_2 \left[ \left( l_1^2 + l_2^2 + 2 l_1 l_2 \cos{(q_2)} \right) \dot{q}_1^2 + l_2^2 \dot{q}_2^2 + 2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2 \right]
\end{align}
$$

por lo que la energía cinética total, será:

$$
\begin{align}
K &= K_1 + K_2 \\
&= \frac{1}{2} m_1 l_1^2 \dot{q}_1^2 + \frac{1}{2} m_2 \left[ \left( l_1^2 + l_2^2 + 2 l_1 l_2 \cos{(q_2)} \right) \dot{q}_1^2 + l_2^2 \dot{q}_2^2 + 2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2 \right] \\
&= \frac{1}{2} \left\{ m_1 l_1^2 \dot{q}_1^2 + m_2 \left[ \left( l_1^2 + l_2^2 + 2 l_1 l_2 \cos{(q_2)} \right) \dot{q}_1^2 + l_2^2 \dot{q}_2^2 + 2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2 \right] \right\} \\
&= \frac{1}{2} \left\{ \left[ m_1 l_1^2 + m_2 \left( l_1^2 + l_2^2 + 2 l_1 l_2 \cos{(q_2)} \right) \right] \dot{q}_1^2 + m_2 l_2^2 \dot{q}_2^2 + 2 m_2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2\right\} \\
&= \frac{1}{2} \left\{ \left[ (m_1 + m_2) l_1^2 + m_2 l_2^2 + 2 m_2 l_1 l_2 \cos{(q_2)} \right] \dot{q}_1^2 + m_2 l_2^2 \dot{q}_2^2 + 2 m_2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2\right\} \\
\end{align}
$$

La cual, al ver las relaciones con las variables de posición generalizadas del pendulo, podemos escribir como:

$$
K = \frac{1}{2} \dot{q}^T M(q) \dot{q}
$$

en donde la matriz de inercia $M(q)$ queda de la forma:

$$
M(q) =
\begin{pmatrix}
(m_1 + m_2) l_1^2 + m_2 l_2^2 + 2 m_2 l_1 l_2 \cos{(q_2)} & m_2 l_2^2 + m_2 l_1 l_2 \cos{(q_2)} \\
m_2 l_2^2 + m_2 l_1 l_2 \cos{(q_2)} & m_2 l_2^2
\end{pmatrix}
$$

en donde podemos introducir las constantes $\mu_i$ para simplificar la notación:

$$
\begin{align}
\mu_1 &= (m_1 + m_2) l_1^2 \\
\mu_2 &= m_2 l_2^2 \\
\mu_3 &= m_2 l_1 l_2
\end{align}
$$

con lo que la expresión para la energía cinética queda:

$$
K = \frac{1}{2} \dot{q}^T
\begin{pmatrix}
\mu_1 + \mu_2 + 2\mu_3 \cos{(q_2)} & \mu_2 + \mu_3 \cos{(q_2)} \\
\mu_2 + \mu_3 \cos{(q_2)} & \mu_2
\end{pmatrix} \dot{q}
$$

### Energía potencial

La energía potencial del sistema esta dada por:

$$
\begin{align}
U &= U_1 + U_2 \\
&= m_1 g h_1 + m_2 g h_2 \\
&= m_1 g y_1 + m_2 g y_2 \\
&= m_1 g l_1 \sin{(q_1)} + m_2 g \left[ l_1 \sin{(q_1)} + l_2 \sin{(q_1 + q_2)} \right] \\
&= \left( m_1 + m_2 \right) g l_1 \sin{(q_1)} + m_2 g l_2 \sin{(q_1 + q_2)} \\
&= g \left[ \left( m_1 + m_2 \right) l_1 \sin{(q_1)} + m_2 l_2 \sin{(q_1 + q_2)} \right] \\
\end{align}
$$

### Lagrangiano

El Lagrangiano del sistema, esta dado por la expresión:

$$
L = K - U
$$

por lo que solo queda sumar las dos expresiones y obtener las condiciones de optimalidad para la energía del sistema por medio de la ecuación de Euler-Lagrange

### Euler-Lagrange

$$
\frac{d}{dt} \left( \frac{\partial L}{\partial \dot{q}} \right) - \frac{\partial L}{\partial q} = 0
$$

Si empezamos derivando con respecto a $\dot{q}$, tendremos:

$$
\frac{\partial L}{\partial \dot{q}} = \frac{\partial K}{\partial \dot{q}} - \frac{\partial U}{\partial \dot{q}}
$$

y obteniendo cada uno de estos tenemos que:

$$
\begin{align}
\frac{\partial K}{\partial \dot{q}} &= K_{\dot{q}} = \frac{\partial}{\partial \dot{q}} \left\{ \frac{1}{2} \dot{q}^T M(q) \dot{q} \right\} = M(q) \dot{q} \\
\frac{\partial U}{\partial \dot{q}} &= U_{\dot{q}} = 0
\end{align}
$$

y la derivada con respecto al tiempo de estos terminos será:

$$
\begin{align}
\frac{d K_{\dot{q}}}{dt} &= M(q) \ddot{q} + \dot{M}(q, \dot{q}) \dot{q} \\
\frac{d U_{\dot{q}}}{dt} &= 0
\end{align}
$$

en donde:

$$
\begin{align}
\dot{M}(q, \dot{q}) &= \frac{d}{dt}
\begin{pmatrix}
\mu_1 + \mu_2 + 2\mu_3 \cos{(q_2)} & \mu_2 + \mu_3 \cos{(q_2)} \\
\mu_2 + \mu_3 \cos{(q_2)} & \mu_2
\end{pmatrix} \\
&=
\begin{pmatrix}
-2 \mu_3 \sin{(q_2)} \dot{q}_2 & -\mu_3 \sin{(q_2)} \dot{q}_2 \\
-\mu_3 \sin{(q_2)} \dot{q}_2 & 0
\end{pmatrix} \\
&= -\mu_3 \sin{(q_2)}
\begin{pmatrix}
2 \dot{q}_2 & \dot{q}_2 \\
\dot{q}_2 & 0
\end{pmatrix}
\end{align}
$$

Ahora, derivando con respecto a $q$, tenemos:

$$
\frac{\partial L}{\partial q} = \frac{\partial K}{\partial q} - \frac{\partial U}{\partial q}
$$

en donde:

$$
\begin{align}
\frac{\partial K}{\partial q} = K_{q} &=
\begin{pmatrix}
\frac{\partial K}{\partial q_1} \\
\frac{\partial K}{\partial q_2}
\end{pmatrix} \\
&= \frac{1}{2}
\begin{pmatrix}
\frac{\partial}{\partial q_1} \left\{ \left[ (m_1 + m_2) l_1^2 + m_2 l_2^2 + 2 m_2 l_1 l_2 \cos{(q_2)} \right] \dot{q}_1^2 + m_2 l_2^2 \dot{q}_2^2 + 2 m_2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2 \right\} \\
\frac{\partial}{\partial q_2} \left\{ \left[ (m_1 + m_2) l_1^2 + m_2 l_2^2 + 2 m_2 l_1 l_2 \cos{(q_2)} \right] \dot{q}_1^2 + m_2 l_2^2 \dot{q}_2^2 + 2 m_2 \left( l_2^2 + l_1 l_2 \cos{(q_2)} \right) \dot{q}_1 \dot{q}_2 \right\} \\
\end{pmatrix} \\
&= \frac{1}{2}
\begin{pmatrix}
0 \\
-2 m_2 l_1 l_2 \sin{(q_2)} \dot{q}_1^2 - 2 m_2 l_1 l_2 \sin{(q_2)} \dot{q}_1 \dot{q}_2 \\
\end{pmatrix} \\
&= - \mu_3 \sin{(q_2)}
\begin{pmatrix}
0 \\
\dot{q}_1^2 + \dot{q}_1 \dot{q}_2 \\
\end{pmatrix} = -\mu_3 \sin{(q_2)}
\begin{pmatrix}
\dot{q}_2 & -\dot{q}_1 \\
\dot{q}_1 + \dot{q}_2 & 0
\end{pmatrix} \dot{q}
\end{align}
$$

y para la energía potencial:

$$
\begin{align}
\frac{\partial U}{\partial q} = U_q &=
\begin{pmatrix}
\frac{\partial U}{\partial q_1} \\
\frac{\partial U}{\partial q_2}
\end{pmatrix} \\
&= g
\begin{pmatrix}
\frac{\partial}{\partial q_1} \left[ \left( m_1 + m_2 \right) l_1 \sin{(q_1)} + m_2 l_2 \sin{(q_1 + q_2)} \right] \\
\frac{\partial}{\partial q_2} \left[ \left( m_1 + m_2 \right) l_1 \sin{(q_1)} + m_2 l_2 \sin{(q_1 + q_2)} \right]
\end{pmatrix} \\
&= g
\begin{pmatrix}
\left( m_1 + m_2 \right) l_1 \cos{(q_1)} + m_2 l_2 \cos{(q_1 + q_2)} \\
m_2 l_2 \cos{(q_1 + q_2)}
\end{pmatrix}
\end{align}
$$

o bien:

$$
U_q = g
\begin{pmatrix}
\mu_4 \cos{(q_1)} + \mu_5 \cos{(q_1 + q_2)} \\
\mu_5 \cos{(q_1 + q_2)}
\end{pmatrix}
$$

con $\mu_4 = \left( m_1 + m_2 \right) l_1$ y $\mu_5 = m_2 l_2$, por lo que la ecuación de Euler-Lagrange queda:

$$
\begin{align}
\frac{d}{dt} \left( \frac{\partial L}{\partial \dot{q}} \right) &- \frac{\partial L}{\partial q} = 0 \\
\frac{d K_{\dot{q}}}{dt} &- \left( K_q - U_q \right) = 0 \\
M(q) \ddot{q} + \dot{M}(q, \dot{q}) \dot{q} &- K_q + U_q = 0
\end{align}
$$

en donde recordando que $K_q$ tiene la forma $F(q, \dot{q}) \dot{q}$, podemos juntar esta matriz con $\dot{M}(q, \dot{q})$ y obtener una expresión de la forma:

$$
M(q) \ddot{q} + C(q, \dot{q}) \dot{q} + U_q = 0
$$

en donde:

$$
\begin{align}
M(q) &=
\begin{pmatrix}
\mu_1 + \mu_2 + 2\mu_3 \cos{(q_2)} & \mu_2 + \mu_3 \cos{(q_2)} \\
\mu_2 + \mu_3 \cos{(q_2)} & \mu_2
\end{pmatrix} \\
C(q, \dot{q}) &= \dot{M}(q, \dot{q}) - F(q, \dot{q}) =
-\mu_3 \sin{(q_2)}
\begin{pmatrix}
\dot{q}_2 & \dot{q}_1 + \dot{q}_2 \\
-\dot{q}_1 & 0
\end{pmatrix} \\
U_q &= g
\begin{pmatrix}
\mu_4 \cos{(q_1)} + \mu_5 \cos{(q_1 + q_2)} \\
\mu_5 \cos{(q_1 + q_2)}
\end{pmatrix}
\end{align}
$$

con

$$
\begin{align}
\mu_1 &= (m_1 + m_2) l_1^2 \\
\mu_2 &= m_2 l_2^2 \\
\mu_3 &= m_2 l_1 l_2 \\
\mu_4 &= \left( m_1 + m_2 \right) l_1 \\
\mu_5 &= m_2 l_2
\end{align}
$$

sin embargo este modelo no considera las señales de control, por lo que procederemos a analizar el sistema.

### Señales de control

Nuestra señal de control escalar se deberá enlazar al actuador del pendubot, el cual esta asociado al primer grado de libertad $q_1$, por lo que finalmente tenemos la ecuación:

$$
M(q) \ddot{q} + C(q, \dot{q}) \dot{q} + U_q = G u
$$

con:

$$
G =
\begin{pmatrix}
1 \\
0
\end{pmatrix}
$$

```python
def f(estado, tiempo, K):
    '''Modelo matematico pendubot
    Esta función f es tal que ẋ = f(x, t), es decir, dado el estado del sistema "x"
    y el tiempo "t", devuelve la derivada en el tiempo "t". Para el caso del pendubot,
    es invariante en el tiempo, por lo que es equivalente a ẋ = f(x).

    Ejemplo
    -------
    >>> x = [τ/4, 0.1, 0, 0]
    >>> t = 0
    >>> f(x, t)
    array([  0.        ,   0.        ,  -4.73091048,  17.41001448])
    '''

    from numpy import matrix, zeros, cos, sin, pi

    # Constantes fisicas del sistema
    m1 = 0.2 # kg
    m2 = 0.6 # kg
    l1 = 0.6 # m
    l2 = 0.3 # m
    g = 9.81 # m/s^2
    τ = 2*pi

    # Variables auxiliares del sistema
    μ1 = (m1 + m2)*l1**2
    μ2 = m2*l2**2
    μ3 = m2*l1*l2
    μ4 = (m1 + m2)*l1
    μ5 = m2*l2

    # Estado agregado
    q1, q2, q̇1, q̇2 = estado

    # Estado del sistema y derivada
    q = matrix([[q1], [q2]])
    q̇ = matrix([[q̇1], [q̇2]])

    # Matrix de inercia generalizada
    M = matrix([[μ1 + μ2 + 2*μ3*cos(q2), μ2 + μ3*cos(q2)],
                [μ2 + μ3*cos(q2), μ2]])

    # Matriz de Coriolis
    C = -μ3*sin(q2)*matrix([[ q̇2, q̇1 + q̇2],
                             [-q̇1, 0]])

    # Vector de gravedad
    U = g*matrix([[μ4*cos(q1) + μ5*cos(q1 + q2)],
                  [μ5*cos(q1 + q2)]])

    # Vector de señales de control
    G = matrix([[1], [0]])

    ξ = matrix(estado).T - matrix([[τ/4], [0], [0], [0]])

    # Ley de control
    u = -(matrix(K)*ξ).tolist()[0][0]

    # Aceleracion del sistema
    q̈ = M.I*(G*u - C*q̇ - U)

    # Incializamos la derivada del estado agregado
    ẏ = zeros(4)

    # Se asignan los valores a la derivada del estado agregado
    ẏ[0] = q̇1
    ẏ[1] = q̇2
    ẏ[2] = q̈[0]
    ẏ[3] = q̈[1]

    # Se devuelve la derivada
    return ẏ
```

```python
from matplotlib.pyplot import style, figure, legend, plot
style.use("ggplot")
```

```python
from scipy.integrate import odeint
from numpy import pi, linspace, sin, cos, array
τ = 2*pi
```

```python
ts = linspace(0, 5, 500)
estado_inicial = [τ/4, 0.1, 0, 0]

f_libre = lambda x, t: f(x, t, [0, 0, 0, 0])

x = odeint(f_libre, estado_inicial, ts)
```

```python
def graficatemporal(archivo, estados, tiempo, etiquetas):
    from matplotlib.pyplot import plot, figure, legend
    from IPython.core.display import HTML

    rosa = "#FB5377"
    azul = "#6BBBE2"

    q1 = estados[:, 0]
    q2 = estados[:, 1]

    fig = figure(figsize=(14, 8))

    p1, = plot(tiempo, q1, color=azul)
    p2, = plot(tiempo, q2, color=rosa)

    ax = fig.gca()
    ax.set_xlabel(r"$t$", fontsize=20)
    ax.set_xlim(min(tiempo), max(tiempo))

    legend([p1, p2], etiquetas, loc=2, fontsize=16)
    fig.savefig(archivo)

    return HTML('')
```

```python
graficatemporal("./imagenes/trayectoriaspendubotlibre.png", x, ts, [r'$q_1(t)$', r'$q_2(t)$'])
```

```python
def graficaxy(archivo, estado, etiquetas):
    from matplotlib.pyplot import figure, plot, legend
    from IPython.core.display import HTML

    l1 = 0.6 # m
    l2 = 0.3 # m

    rosa = "#FB5377"
    azul = "#6BBBE2"

    q1 = estado[:, 0]
    q2 = estado[:, 1]

    fig = figure(figsize=(8, 8))

    x1 = l1*cos(q1)
    y1 = l1*sin(q1)
    x2 = x1 + l2*cos(q1 + q2)
    y2 = y1 + l2*sin(q1 + q2)

    p1, = plot(x1, y1, color=azul)
    p2, = plot(x2, y2, color=rosa)

    ax = fig.gca()
    ax.set_ylabel(r"$y$", fontsize=20)
    ax.set_xlabel(r"$x$", fontsize=20)
    ax.set_xlim(-1, 1)
    ax.set_ylim(-1, 1)

    legend([p1, p2], etiquetas, loc=2, fontsize=16)
    fig.savefig(archivo)

    return HTML('')
```

```python
graficaxy("./imagenes/xypendubotlibre.png", x, [r'$q_1(t)$', r'$q_2(t)$'])
```

```python
def animacion_pendubot(archivo, estado, K):
    from matplotlib.patches import Circle
    from matplotlib.gridspec import GridSpec
    from matplotlib.pyplot import subplot, tight_layout, figure
    from matplotlib import animation
    from numpy import matrix
    from IPython.core.display import HTML

    l1 = 0.6 # m
    l2 = 0.3 # m
    m1 = 0.2 # kg
    m2 = 0.6 # kg

    rosa = "#FB5377"
    azul = "#6BBBE2"
    amarillo = "#F0D664"
    verde = "#8B9C67"

    q1 = estado[:, 0]
    q2 = estado[:, 1]

    x1 = l1*cos(q1)
    y1 = l1*sin(q1)
    x2 = x1 + l2*cos(q1 + q2)
    y2 = y1 + l2*sin(q1 + q2)

    u = (K*matrix(-estado).T).tolist()[0]

    fig = figure(figsize=(4.0902, 3.5))
    fig.set_tight_layout(True)

    gs = GridSpec(1, 2, width_ratios=[7, 1])

    ax0 = subplot(gs[0])
    ax1 = subplot(gs[1])

    ax0.set_xticklabels([])
    ax0.set_yticklabels([])
    ax0.set_xlim([-1, 1])
    ax0.set_ylim([-1, 1])

    ax1.axes.get_xaxis().set_visible(False)
    ax1.axes.get_yaxis().set_visible(False)
    ax1.set_ylim([min(u) - 0.1, max(u) + 0.1])
    ax1.set_xlim([0, 1])

    eslabones, = ax0.plot([], [], lw=2.5, color=verde)
    centro = Circle((10, 10), 0.075, lw=1.5, fc=amarillo)
    junta1 = Circle((10, 10), 0.1*m1/m2, lw=1.5, fc=azul)
    junta2 = Circle((10, 10), 0.1, lw=1.5, fc=rosa)

    barra, = ax1.bar([0], [0], width=1, color=amarillo)

    def init():
        eslabones.set_data([], [])

        centro.center = (0, 0)
        junta1.center = (x1[0], y1[0])
        junta2.center = (x2[0], y2[0])

        ax0.add_patch(centro)
        ax0.add_patch(junta1)
        ax0.add_patch(junta2)

        barra.set_height(u[0])
        return eslabones, junta1, junta2, barra

    def animate(i):
        eslabones.set_data([0, x1[i], x2[i]], [0, y1[i], y2[i]])

        junta1.center = (x1[i], y1[i])
        junta2.center = (x2[i], y2[i])

        barra.set_height(u[i])
        return eslabones, junta1, junta2, barra

    ani = animation.FuncAnimation(fig, animate, frames=len(x1), interval=25, blit=True, init_func=init)

    ani.save(archivo, writer='imagemagick');

    return HTML('')
```

```python
animacion_pendubot("./imagenes/doublependulum.gif", x, [0, 0, 0, 0])
```

## Linealización del modelo del pendubot

### Estado agregado

Dado el modelo de este sistema no lineal que hemos obtenido, podemos linealizar alrededor de los puntos de equilibrio, pero primero tenemos que obtener las ecuaciones del sistema agregado, de tal manera que:

$$
\dot{x} = f(x, u)
$$

por lo que primero convertimos el modelo representado por una ecuación diferencial de segundo orden, por uno representado por una ecuación diferencial ordinaria de primer orden. Recordemos el modelo obtenido:

$$
M(q) \ddot{q} + C(q, \dot{q}) \dot{q} + U_q = G u \implies \ddot{q} = M^{-1}(q) \left[ G u - C(q, \dot{q}) \dot{q} - U_q \right]
$$

Si definimos el estado $x$ como:

$$
x =
\begin{pmatrix}
x_1 \\
x_2 \\
x_3 \\
x_4
\end{pmatrix} =
\begin{pmatrix}
q_1 \\
q_2 \\
\dot{q}_1 \\
\dot{q}_2
\end{pmatrix}
$$

significa que tenemos que encontrar $f(x, u)$ tal que:

$$
\dot{x} =
\begin{pmatrix}
\dot{x}_1 \\
\dot{x}_2 \\
\dot{x}_3 \\
\dot{x}_4
\end{pmatrix} =
\begin{pmatrix}
\dot{q}_1 \\
\dot{q}_2 \\
\ddot{q}_1 \\
\ddot{q}_2
\end{pmatrix} =
\begin{pmatrix}
f_1(x, u) \\
f_2(x, u) \\
f_3(x, u) \\
f_4(x, u)
\end{pmatrix} = f(x, u)
$$

en donde $f_1$ y $f_2$, trivialmente son:

$$
\begin{align}
f_1(x, u) &= x_3 = \dot{q}_1 \\
f_2(x, u) &= x_4 = \dot{q}_2
\end{align}
$$

y $f_3$ y $f_4$ estan dadas por el modelo obtenido:

$$
\begin{pmatrix}
\ddot{q}_1 \\
\ddot{q}_2
\end{pmatrix} = \ddot{q} = M^{-1}(q) \left[ G u - C(q, \dot{q}) \dot{q} - U_q \right] =
\begin{pmatrix}
f_3(x, u) \\
f_4(x, u)
\end{pmatrix}
$$

por lo que ahora debemos obtener expresiones escalares para la dinámica del pendubot obtenida.

### Ecuaciones escalares de la dinámica del pendubot

<div style="display:none">
  $\DeclareMathOperator{\adjunta}{adj}$
</div>

Empezamos calculando $M^{-1}(q)$:

$$
\begin{align}
M^{-1}(q) &= \frac{1}{\det{\left( M(q) \right)}} \adjunta{\left( M(q) \right)} \\
&= \frac{1}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 & -\mu_2 - \mu_3 \cos{(q_2)} \\
-\mu_2 - \mu_3 \cos{(q_2)} & \mu_1 + \mu_2 + 2 \mu_3 \cos{(q_2)}
\end{pmatrix}
\end{align}
$$

y las ecuaciones que queremos obtener:

$$
\begin{pmatrix}
f_3(x, u) \\
f_4(x, u)
\end{pmatrix} = M^{-1}(q) G u - M^{-1}(q) C(q, \dot{q}) \dot{q} - M^{-1}(q) U_q
$$

tienen terminos:

$$
\begin{align}
M^{-1}(q) G u &= \frac{1}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 & -\mu_2 - \mu_3 \cos{(q_2)} \\
-\mu_2 - \mu_3 \cos{(q_2)} & \mu_1 + \mu_2 + 2 \mu_3 \cos{(q_2)}
\end{pmatrix}
\begin{pmatrix}
1 \\
0
\end{pmatrix} u \\
&= \frac{1}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 \\
-\mu_2 - \mu_3 \cos{(q_2)}
\end{pmatrix} u
\end{align}
$$

$$
\begin{align}
- M^{-1}(q) C(q, \dot{q}) \dot{q} &= \frac{\mu_3 \sin{(q_2)}}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 & -\mu_2 - \mu_3 \cos{(q_2)} \\
-\mu_2 - \mu_3 \cos{(q_2)} & \mu_1 + \mu_2 + 2 \mu_3 \cos{(q_2)}
\end{pmatrix}
\begin{pmatrix}
\dot{q}_2 & \dot{q}_1 + \dot{q}_2 \\
-\dot{q}_1 & 0
\end{pmatrix} \dot{q} \\
&= \frac{\mu_3 \sin{(q_2)}}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 \left( \dot{q}_1 + \dot{q}_2 \right)^2 + \mu_3 \cos{(q_2)} \dot{q}_1^2 \\
-\mu_1 \dot{q}_1^2 - \mu_2 \left( \dot{q}_1 + \dot{q}_2 \right)^2 - \mu_3 \cos{(q_2)} \left[ \left( \dot{q}_1 + \dot{q}_2 \right)^2 + \dot{q}_1^2 \right]
\end{pmatrix}
\end{align}
$$

$$
\begin{align}
-M^{-1}(q) U_q &= - \frac{g}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 & -\mu_2 - \mu_3 \cos{(q_2)} \\
-\mu_2 - \mu_3 \cos{(q_2)} & \mu_1 + \mu_2 + 2 \mu_3 \cos{(q_2)}
\end{pmatrix}
\begin{pmatrix}
\mu_4 \cos{(q_1)} + \mu_5 \cos{(q_1 + q_2)} \\
\mu_5 \cos{(q_1 + q_2)}
\end{pmatrix} \\
&= - \frac{g}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\begin{pmatrix}
\mu_2 \mu_4 \cos{(q_1)} - \mu_3 \mu_5 \cos{(q_2)} \cos{(q_1 + q_2)} \\
-\mu_2 \mu_4 \cos{(q_1)} - \mu_3 \mu_4 \cos{(q_1)} \cos{(q_2)} + \mu_1 \mu_5 \cos{(q_1 + q_2)} + \mu_3 \mu_5 \cos{(q_2)} \cos{(q_1 + q_2)}
\end{pmatrix}
\end{align}
$$

Por lo que al juntar todos los terminos, y hacer un poco de algebra, tendremos que:

$$
f_3(x, u) = \frac{\mu_2 u + \mu_3 \mu_5 \sin{(q_2)} \left[ l_2 \left( \dot{q}_1 + \dot{q}_2 \right)^2 + l_1 \cos{(q_2)} \dot{q}_1^2 \right] -g \mu_3 \left[ \mu_4 \cos{(q_1)} + \mu_5 \cos{(q_2)} \cos{(q_1 + q_2)} \right]}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
$$

$$
\begin{align}
f_4(x, u) &= \frac{-\mu_5 \left( l_2 + l_1 \cos{(q_2)} \right)u - \mu_3 \sin{(q_2)} \left\{ m_1 l_1^2 \dot{q}_1^2 + \mu_5 \left[ \left( l_2 + l_1 \cos{(q_2)} \right) \left( \dot{q}_1 + \dot{q}_2 \right)^2 + l_1 \cos{(q_2)} \dot{q}_1^2 \right] \right\}}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}} \\
&+ \frac{g \mu_3 \left[ \left( m_1 + m_2 \right)\left( l_2 \cos{(q_1)} + l_1 \sin{(q_1)} \sin{(q_2)} \right) - \mu_5 \cos{(q_2)} \cos{(q_1 + q_2)} \right]}{\mu_1 \mu_2 - \mu_3^2 \cos^2{(q_2)}}
\end{align}
$$

### Linealización del sistema no lineal

Dado este sistema no lineal, podemos obtener una linealización alrededor de algun punto de equilibrio al considerar la expansión de Taylor de la dinámica del sistema

$$
f(x, u) = f(\bar{x}, \bar{u}) + \left. \frac{\partial f}{\partial x} \right|_{x=\bar{x}, u=\bar{u}} (x - \bar{x}) + \left. \frac{\partial f}{\partial u} \right|_{x=\bar{x}, u=\bar{u}} (u - \bar{u}) + o(x, u)
$$

en donde $f(\bar{x}, \bar{u}) = 0$ y $o(x, u)$ son terminos de orden superior, por lo que al tener en cuenta variaciones pequeñas de nuestra diámica, podemos despreciarlas y obtener:

$$
\dot{\xi} = A \xi + B u
$$

en donde $\xi = x - \bar{x}$ es el estado de nuestra linealización, y $A$ y $B$ estan dados por:

$$
\begin{align}
A &= \left. \frac{\partial f}{\partial x} \right|_{x=\bar{x}, u=\bar{u}} \\
B &= \left. \frac{\partial f}{\partial u} \right|_{x=\bar{x}, u=\bar{u}}
\end{align}
$$

Calculando estas derivadas, tenemos que nuestra linealización queda:

$$
A =
\begin{pmatrix}
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 \\
\frac{g}{l_1} & -\frac{g m_2}{l_1 m_1} & 0 & 0 \\
-\frac{g}{l_1} & \frac{g}{m_1} \left( \frac{m_1}{l_2} + \frac{m_2}{l_2} + \frac{m_2}{l_1} \right) & 0 & 0
\end{pmatrix}
$$

$$
B =
\begin{pmatrix}
0 \\
0 \\
\frac{1}{m_1 l_1^2} \\
- \frac{l_1 + l_2}{m_1 l_1^2 l_2}
\end{pmatrix}
$$

```python
def f_lineal(x, t, K):
    '''Modelo matematico lineal pendubot
    Esta función f_lineal es tal que ẋ = f(x, t), es decir, dado el estado del sistema "x"
    y el tiempo "t", devuelve la derivada en el tiempo "t". Para el caso del pendubot,
    es invariante en el tiempo, por lo que es equivalente a ẋ = f(x).

    Ejemplo
    -------
    >>> x = [τ/4, 0.1, 0, 0]
    >>> t = 0
    >>> f(x, t)
    [0.0, 0.0, -4.905000000000001, 17.985]
    '''

    from numpy import matrix, zeros, cos, sin

    # Constantes fisicas del sistema
    m1 = 0.2 # kg
    m2 = 0.6 # kg
    l1 = 0.6 # m
    l2 = 0.3 # m
    g = 9.81 # m/s^2

    # Matriz sistema linealizado
    A = matrix([[0, 0, 1, 0],
                [0, 0, 0, 1],
                [g/l1, -g*m2/(l1*m1), 0, 0],
                [-g/l1, g/m1*(m1/l2 + m2/l2 + m2/l1), 0, 0]])

    # Vector de entrada
    B = matrix([[0],[0],[1/(m1*l1**2)],[-(l1 + l2)/(m1*l1**2*l2)]])

    # Ley de control lineal
    u = -matrix(K)*matrix(x).T

    # Dinámica del sistema linealizado
    ẋ = A*matrix(x).T + B*u

    # Se devuelve la derivada
    return (ẋ.T).tolist()[0]
```

```python
ts = linspace(0, 0.4, 100)
estado_inicial_lineal = [0.0, 0.1, 0.0, 0.0]

f_lineal_libre = lambda x, t: f_lineal(x, t, [0, 0, 0, 0])

ξs = odeint(f_lineal_libre, estado_inicial_lineal, ts)
xl = array([ξ + array([τ/4, 0, 0, 0]) for ξ in ξs])
```

```python
graficatemporal("./imagenes/trayectoriaspendubotlineallibre.png", xl, ts, [r'$\xi_1(t) + \frac{\tau}{4}$', r'$\xi_2(t)$'])
```

```python
graficaxy("./imagenes/xypendubotlineallibre.png", xl, [r'$\xi_1(t) + \frac{\tau}{4}$', r'$\xi_2(t)$'])
```

```python
x = odeint(f_libre, estado_inicial, ts)
q1, q2 = x[:, 0], x[:, 1]
q1_l, q2_l = xl[:, 0], xl[:, 1]
```

```python
fig = figure(figsize=(8, 8))

l1 = 0.6 # m
l2 = 0.3 # m

rosa = "#FB5377"
azul = "#6BBBE2"

x1 = l1*cos(q1)
y1 = l1*sin(q1)
x2 = x1 + l2*cos(q1 + q2)
y2 = y1 + l2*sin(q1 + q2)

x1_l = l1*cos(q1_l)
y1_l = l1*sin(q1_l)
x2_l = x1_l + l2*cos(q1_l + q2_l)
y2_l = y1_l + l2*sin(q1_l + q2_l)

p3, = plot(x1_l, y1_l, "x", color=azul)
p4, = plot(x2_l, y2_l, "x", color=rosa)

p1, = plot(x1, y1, ".", color=azul)
p2, = plot(x2, y2, ".", color=rosa)

ax = fig.gca()
ax.set_ylabel(r"$y$", fontsize=20)
ax.set_xlabel(r"$x$", fontsize=20)
ax.set_xlim(-0.4, 0.4)
ax.set_ylim(0.2, 1)

legend([p1, p2, p3, p4], [r'$q_1(t)$', r'$q_2(t)$', r'$\xi_1(t) + \frac{\tau}{4}$', r'$\xi_2(t)$'], loc=2, fontsize=16)
fig.savefig("./imagenes/xypendubotlinealynolineallibres.png")
HTML('')
```

Aqui podemos ver como el sistema linealizado deverge rapidamente del sistema original, por lo que podemos asumer que este modelo linealizado es valido para variaciones muy pequeñas de $q_1$ y $q_2$.

## Diseño de un control óptimo

Para el diseño de la ley de control óptima que minimice la funcional:

$$
J(u) = \int_0^{\infty} \left( x^T(t) Q x(t) + u^2(t) \right) dt
$$

con $Q$ de la forma:

$$
Q =
\begin{pmatrix}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 10 & 0 \\
0 & 0 & 0 & 10
\end{pmatrix}
$$

tal que la ley de control óptima sea:

$$
u^*(t) = -K \xi^*(t)
$$

Para calcular la ganancia $K$ asociada a esta función de costo, utilizaremos el sistema lineal encontrado.

```python
from control import lqr
from numpy import matrix
```

```python
m1 = 0.2 # kg
m2 = 0.6 # kg
l1 = 0.6 # m
l2 = 0.3 # m
g = 9.81 # m/s^2

A = matrix([[0, 0, 1, 0],
            [0, 0, 0, 1],
            [g/l1, -g*m2/(l1*m1), 0, 0],
            [-g/l1, g/m1*(m1/l2 + m2/l2 + m2/l1), 0, 0]])

B = matrix([[0],
            [0],
            [1/(m1*l1**2)],
            [-(l1+l2)/(m1*l1**2*l2)]])

Q = matrix([[1, 0, 0, 0],
            [0, 1, 0, 0],
            [0, 0, 10, 0],
            [0, 0, 0, 10]])

R = matrix([[1]])
```

```python
K, S, E = lqr(A, B, Q, R)
K
```

    array([[-51.600978  , -51.39319408, -16.05940057,  -8.92417942]])

Una vez que hemos obtenido las ganancias de realimentacion, tan solo tenemos que incluirlas en la simulación del sistema para ver su desempeño.

```python
ts = linspace(0, 5, 500)
estado_inicial_lineal = [0.0, 0.1, 0.0, 0.0]

f_lineal_optima = lambda x, t: f_lineal(x, t, K)

ξs = odeint(f_lineal_optima, estado_inicial_lineal, ts)
xl = array([ξ + array([τ/4, 0, 0, 0]) for ξ in ξs])
```

```python
graficatemporal("./imagenes/trayectoriaspendubotlinealoptimo.png", xl, ts, [r'$\xi_1(t) + \frac{\tau}{4}$', r'$\xi_2(t)$'])
```

```python
graficaxy("./imagenes/xypendubotlinealoptimo.png", xl, [r'$\xi_1(t) + \frac{\tau}{4}$', r'$\xi_2(t)$'])
```

Ahora sabemos que este control estabiliza al sistema linealizado, pero en realidad queremos saber como se desempeña con el sistema no lineal.

Podemos realimentar el sistema original para ver su desempeño.

```python
ts = linspace(0, 5, 500)
estado_inicial = [τ/4, 0.1, 0, 0]

f_optima = lambda x, t: f(x, t, K)

x = odeint(f_optima, estado_inicial, ts)
```

```python
graficatemporal("./imagenes/trayectoriaspendubotoptimo.png", x, ts, [r'$q_1(t)$', r'$q_2(t)$'])
```

```python
graficaxy("./imagenes/xypendubotoptimo.png", x, [r'$q_1(t)$', r'$q_2(t)$'])
```

```python
animacion_pendubot("./imagenes/pendubotoptimo.gif", x, K)
```

Y tan solo para divertirnos un poco, que pasa cuando le damos condiciones iniciales extremas:

```python
ts = linspace(0, 5, 500)
estado_inicial = [τ/4 + 0.35, 0.35, 0, 0]

x = odeint(f_optima, estado_inicial, ts)
```

```python
animacion_pendubot("./imagenes/pendubotoptimolimite.gif", x, K)
```

Si deseas compartir este Notebook de IPython utiliza la siguiente dirección:

http://bit.ly/1H1bR6Q

o bien el siguiente código QR:

```python
# Codigo para generar codigo :)
from qrcode import make
img = make("http://bit.ly/1H1bR6Q")
img.save("codigos/pendubot.jpg")
```


```c++
#pragma cling add_include_path("../../include")
#pragma cling add_include_path("../feltor/inc") // Feltor path
#pragma cling add_include_path("/usr/include/jsoncpp")
#pragma cling add_include_path("/usr/local/include")
#pragma cling load("netcdf")
#pragma cling load("jsoncpp")
#define THRUST_DEVICE_SYSTEM THRUST_DEVICE_SYSTEM_CPP
#include <iostream>
#include "dg/algorithm.h"
#include "dg/file/file.h"
```

    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:8:
    ../feltor/inc/dg/backend/config.h:20:9: warning: NOTE: Fast std::fma(a,b,c) not activated! Using a*b+c instead!
          [-W#pragma-messages]
    #pragma message( "NOTE: Fast std::fma(a,b,c) not activated! Using a*b+c ...
            ^
    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:11:
    In file included from ../feltor/inc/dg/topology/split_and_join.h:4:
    In file included from ../feltor/inc/dg/backend/blas1_dispatch_shared.h:12:
    In file included from ../feltor/inc/dg/backend/blas1_serial.h:6:
    In file included from ../feltor/inc/dg/backend/exblas/exdot_serial.h:25:
    In file included from ../feltor/inc/dg/backend/exblas/accumulate.h:19:
    ../feltor/inc/dg/backend/exblas/config.h:31:9: warning: WARNING: Instruction set below SSE4.1! Deactivating vectorization!
          [-W#pragma-messages]
    #pragma message("WARNING: Instruction set below SSE4.1! Deactivating vec...
            ^
    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:11:
    In file included from ../feltor/inc/dg/topology/split_and_join.h:4:
    In file included from ../feltor/inc/dg/backend/blas1_dispatch_shared.h:12:
    In file included from ../feltor/inc/dg/backend/blas1_serial.h:6:
    In file included from ../feltor/inc/dg/backend/exblas/exdot_serial.h:25:
    ../feltor/inc/dg/backend/exblas/accumulate.h:93:43: warning: shifting a negative signed value is undefined [-Wshift-negative-value]
            carrybit = (s ? 1ll << KRX : -1ll << KRX);
                                         ~~~~ ^
    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:11:
    In file included from ../feltor/inc/dg/topology/split_and_join.h:4:
    In file included from ../feltor/inc/dg/backend/blas1_dispatch_shared.h:12:
    In file included from ../feltor/inc/dg/backend/blas1_serial.h:6:
    In file included from ../feltor/inc/dg/backend/exblas/exdot_serial.h:26:
    ../feltor/inc/dg/backend/exblas/ExSUM.FPE.hpp:143:46: warning: unknown attribute 'optimize' ignored [-Wunknown-attributes]
    template<typename T, int N, typename TRAITS> UNROLL_ATTRIBUTE
                                                 ^
    ../feltor/inc/dg/backend/exblas/config.h:42:41: note: expanded from macro 'UNROLL_ATTRIBUTE'
    #define UNROLL_ATTRIBUTE __attribute__((optimize("unroll-loops")))
                                            ^
    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:11:
    In file included from ../feltor/inc/dg/topology/split_and_join.h:4:
    In file included from ../feltor/inc/dg/backend/blas1_dispatch_shared.h:12:
    In file included from ../feltor/inc/dg/backend/blas1_serial.h:6:
    In file included from ../feltor/inc/dg/backend/exblas/exdot_serial.h:26:
    ../feltor/inc/dg/backend/exblas/ExSUM.FPE.hpp:189:46: warning: unknown attribute 'optimize' ignored [-Wunknown-attributes]
    template<typename T, int N, typename TRAITS> UNROLL_ATTRIBUTE
                                                 ^
    ../feltor/inc/dg/backend/exblas/config.h:42:41: note: expanded from macro 'UNROLL_ATTRIBUTE'
    #define UNROLL_ATTRIBUTE __attribute__((optimize("unroll-loops")))
                                            ^
    In file included from input_line_8:2:
    In file included from ../feltor/inc/dg/algorithm.h:11:
    In file included from ../feltor/inc/dg/topology/split_and_join.h:4:
    In file included from ../feltor/inc/dg/backend/blas1_dispatch_shared.h:12:
    In file included from ../feltor/inc/dg/backend/blas1_serial.h:6:
    In file included from ../feltor/inc/dg/backend/exblas/exdot_serial.h:26:
    ../feltor/inc/dg/backend/exblas/ExSUM.FPE.hpp:221:46: warning: unknown attribute 'optimize' ignored [-Wunknown-attributes]
    template<typename T, int N, typename TRAITS> UNROLL_ATTRIBUTE
                                                 ^
    ../feltor/inc/dg/backend/exblas/config.h:42:41: note: expanded from macro 'UNROLL_ATTRIBUTE'
    #define UNROLL_ATTRIBUTE __attribute__((optimize("unroll-loops")))
                                            ^

(sec:project)=
# Create your own project

In this chapter we are going to learn how to write a code project implementing a set of equations from beginning to end including file I/O and diagnostics.
```{note}
In principle you can write your own codes however you want. Feltor does not force any specific structure onto you or force you to use more of it than you want to. This chapter is a reflection of best practices that we currently use for our own projects and that have proven themselves in scientific analysis. It is for example specifically designed to interoperate well with python. This design has evolved through the last 10 years and, for example when new C++ features become available, will likely continue to evolve in the future.
```

We use the simple advection diffusion equation as a model equation
\begin{align}
    \frac{\partial \omega}{\partial t} &= -v\cdot \nabla\omega + D \Delta \omega \\
     -\Delta \phi &= \omega \\
     v_x &:= -\partial_y \phi \\
     v_y &:= \partial_x \phi
\end{align}

## Basic file structure

- `init.h` Contains a list of initial conditions that can be chosen in the inputfile
- `diag.h` Contains a list of diagnostic functions that should be executed periodically in the main timeloop. This will determine the content of the output file
- `Makefile` How to compile the main program
- `parameters.h` [optional] Reads and stores parameters from the input file. The intention here is to provide quick (in terms of code length) access to often used parameters.
- `equations.h` Contains the main functor that is used in the timestepper. Implements the equations.
- `documentation.tex` Contains the documentation of the implemented equations, the numerical methods used, the initial conditions and the diagnostics. In particular it describes the parameters of the input file and what the output file contains.
- `input.json` The input file to a simulation
- `myprogram.cpp` Here is the main function. It reads in the input file, calls the relevant initial condition, constructs the Equations functor(s) and the Timestepper. Then, it implements the main time loop, the diagnostics and output strategy.

```{admonition} I/O file formats
We use and highly recommend [json](https://en.wikipedia.org/wiki/JSON) for the input file and [NetCDF-4](https://www.unidata.ucar.edu/software/netcdf/) for the output file.
The reason for this is that both of these formats interoperate well with many other programming languages, in particular python, which we use for data analysis.
```

## The init file
The goal of the init file is to provide various initial conditions that can be selected in the input file
```{admonition} Data layout
The return type of the initial conditions depends on what data layout you choose when implementing [The equations file](sec:equations). In this case it is `dg::x::DVec` because there is only one variable, but often used types are `std::vector<dg::x::DVec>`, `std::map<std::string, dg::x::DVec>` or `std::array<dg::x::DVec, 2>`
```

```cpp
// init.h
#pragma once

namespace myproject
{

dg::x::DVec initial_conditions(
    const dg::x::CartesianGrid2d& grid,
    dg::file::WrappedJsonValue init)
{
    dg::x::HVec omega;
    std::string initial = init.get("type", "lamb").asString();
    if( initial == "zero")
    {
        omega = dg::evaluate( dg::zero, grid);
    }
    else if( initial == "lamb")
    {
        double posX = init.get("posX", 0.5).asDouble();
        double posY = init.get("posY", 0.8).asDouble();
        double R =    init.get("sigma", 0.1).asDouble();
        double U =    init.get("velocity", 1).asDouble();
        dg::Lamb lamb( posX*grid.lx(), posY*grid.ly(), R, U);
        omega = dg::evaluate ( lamb, grid);
    }
    // ... implement more initial conditions here
    else
        throw dg::Error( dg::Message() << "Initial condition "
                        <<initial<<" not recognized!");
    return dg::construct<dg::x::DVec>(omega);
}

} //namespace myproject
```

The above example would require the following entry in the input file
```json
{
    "init" :
    {
        "type" : "lamb",
        "posX" : 0.5,
        "posY" : 0.8,
        "sigma" : 0.1,
        "velocity" : 1
    }
}
```
If the zero initial condition is chosen, the entry has to look like
```json
{
    "init" :
    {
        "type" : "zero"
    }
}
```
As you can see only the necessary parameters need to be present for each type of initial condition.
```{note}
You will have to document each initial condition and what parameters can be chosen in the documentation.tex file Here the "minted" package is highly recommended to document json parameters.
```

## The diag file

The desired list of variables in the output file typically grows (or sometimes shrinks) often during the lifetime of a project for example when new theoretical results are revealed or a new data-analysis needs to be performed.
The goal of our design here is therefore to provide a flexible and importantly an **extensible** approach to generating file output.

The purpose of the diag file is to provide one or more list of functions that generate output variables in the netcdf output file.
Each item in the list is a record consisting of a name, a description and a function that generates the data points of the variable. The functions' parameters are bundled in a struct `Variables`.
It is important to note that the list of records (the diagnostics\*\_list ) can be easily extended without side-effects of the existing records. The Variables can be extended, but the Variable construction line in the main file also needs to extend then (see [The main file](sec:main_file_construction) )

```cpp
// diag.h
#pragma once
#include "equations.h"

namespace myproject
{

struct Variables
{
    Equations<dg::x::CartesianGrid2d, dg::x::DMatrix, dg::x::DVec>& rhs;
    const dg::x::CartesianGrid2d& grid;
    const dg::x::DVec& omega;
};

struct Record
{
    std::string name; // variable name in the output file
    std::string long_name; // longer description as an attribute
    std::function<void(dg::x::DVec&, Variables&)> function;
    // function that generates the data points for the variable
};

// time - independent output (only called once)
std::vector<Record> diagnostics2d_static_list = {
    { "xc", "x-coordinate in Cartesian coordinate system",
        []( dg::x::DVec& result, Variables& v ) {
            result = dg::evaluate( dg::cooX2d, v.grid);
        }
    },
    { "yc", "y-coordinate in Cartesian coordinate system",
        []( dg::x::DVec& result, Variables& v ) {
            result = dg::evaluate( dg::cooY2d, v.grid);
        }
    },
    { "weights", "Gaussian integration weights",
        []( dg::x::DVec& result, Variables& v ) {
            result = dg::create::weights( v.grid);
        }
    },
    // ... extend here
};

// time - dependent output (called periodically)
std::vector<Record> diagnostics2d_list = {
    {"vorticity", "Vorticity in 2d",
        []( dg::x::DVec& result, Variables& v ) {
             dg::blas1::copy(v.omega, result);
        }
    },
    {"potential", "stream function",
        []( dg::x::DVec& result, Variables& v ) {
             dg::blas1::copy(v.rhs.potential(), result);
        }
    },
```
```{note}
You can write anything you want in the lambda, really.
At least anything you want to happen periodically in the main time loop.
```
```cpp
    {"enstrophy", "Squared vorticity",
        []( dg::x::DVec& result, Variables& v ) {
             // more complicated algorithms can be written here
             dg::blas1::pointwiseDot( v.omega, v.omega, result);
             dg::blas1::scal( result, 1./2.);
        }
    }
    // ... extend here
};

// ... write more lists, for example 1d diagnostics etc
// each list requires a loop in the main program

} //namespace myproject
```

(sec:equations)=
## The equations file

Our timesteppers require functors that implement the right hand side of the differential equation.
The purpose of the equations file is to provide such functor(s) for the specific equations at hand.

```cpp
// equations.h

#pragma once

#include "dg/algorithm.h"

namespace myproject
{

template<class Geometry, class Matrix, class Container>
struct Equations
{
    Equations( const Geometry& grid, dg::file::WrappedJsonValue& js):
        m_phi ( dg::evaluate(dg::zero, grid)),
        m_old_phi( 2, m_phi),
        m_adv( grid), // use grid's boundary conditions
        m_lapM( grid)
    {
        m_v = {m_phi, m_phi};
        m_pcg.construct(m_phi, grid.size());
        m_eps_pol = js["elliptic"].get("eps_pol", 1e-6).asDouble();
        m_nu = js["physical"].get( "nu", 1e-7).asDouble();
        m_centered[0] = dg::create::dx( grid, grid.bcx(), dg::centered);
        m_centered[1] = dg::create::dy( grid, grid.bcy(), dg::centered);
    }
    // accessors for diag.h
    const Container& potential() const {return m_phi;}
```
\begin{align}
    \frac{\partial \omega}{\partial t} &= -v\cdot \nabla\omega + D \Delta \omega \\
     -\Delta \phi &= \omega \\
     v_x &:= -\partial_y \phi \\
     v_y &:= \partial_x \phi
\end{align}
```{admonition} Data layout
Here, in the parameters for `operator()` you have to choose a suitable data-layout for the vector that is integrated in time. We chose a simple `Container` because there is only one dynamic variable in the equations. However, if there were three equations say then `std::array<Container,3>` could be a better type or even `std::vector<Container>` if you want to choose at runtime or the recently added `std::map<std::string, Container>` for a verbose access
```

```cpp
    // We implement advection diffusion equations:
    void operator()(double t, const Container& omega, Container& omegaDot)
    {

        // Solve potential equation
        m_old_phi.extrapolate( t, m_phi);
        // For demonstration we here use the simple unpreconditioned PCG
        // solver. In real code it is highly recommended to use nested_iterations instead
        // See "Solvers" chapter of this guide
        m_pcg.solve( m_lapM, m_phi, omega, 1., m_lapM.weights(), m_eps_pol);
        m_old_phi.update( t, m_phi);

        // add advection term
        dg::blas2::symv( -1., m_centered[1], m_phi, 0., m_v[0]);
        dg::blas2::symv( +1., m_centered[0], m_phi, 0., m_v[1]);
        m_adv.upwind( -1., m_v[0], m_v[1], omega, 0., omegaDot);

        // add diffusion
        dg::blas2::symv( -m_nu, m_lapM, omega, 1., omegaDot);
    }

    private:
    Container m_phi;
    dg::Extrapolation<Container> m_old_phi;
    dg::Advection<Geometry, Matrix, Container> m_adv;
    dg::Elliptic< Geometry, Matrix, Container> m_lapM;
    dg::PCG<Container> m_pcg;
    std::array<Matrix,2> m_centered;
    std::array<Container,2> m_v;
    double m_eps_pol;
    double m_nu;
};

}//namespace myproject
```
This necessitates two more parameters in the input file
```json
{
    "physical":
    {
        "nu" : 1e-7
    },
    "elliptic":
    {
        "eps_pol" : 1e-6
    }

}
```

We here showcase the simplest case where we implement everything explicitly (hence only one functor) and use a simple unpreconditioned PCG solver for the elliptic equation. This is just to show the basic structure. You can extend this however you want and make it arbitrarily complicated.

## The main file

The main function acts as a driver where everything comes together and has to be called in the right order.

### Front matter
We here decide to write a program that works for both shared and distributed memory. In order for this to work we need to initialize MPI, but only in case we actually need it. Typically, we would do the following:

```cpp
// myprogram.cpp
#include <iostream>
#include <iomanip>

#ifdef WITH_MPI
#include <mpi.h>
#endif //WITH_MPI

#include "dg/algorithm.h"
#include "dg/file/file.h"

#include "equations.h"
#include "init.h"
#include "diag.h"

int main( int argc, char* argv[])
{
#ifdef WITH_MPI
    dg::mpi_init( argc, argv);
    MPI_Comm comm;
    dg::mpi_init2d( dg::DIR, dg::PER, comm, std::cin, true);
    int rank;
    MPI_Comm_rank( MPI_COMM_WORLD, &rank);
#endif //WITH_MPI
```
Here, we use a Macro `WITH_MPI` that is intended to be defined at compile-time.
```{note}
We therefore add `-DWITH_MPI` to the mpi compiler flags in [the Makefile](sec:Makefile)
```
Then in the main program we need to initialize MPI. The `dg::mpi_init2d` function generates a Cartesian Communicator that we will need to generate a grid and also will generate a prompt on `std::cin` asking the user to provide the partition of MPI processes.
```{admonition} MPI partitioning
We have not found a good way yet to automate the process of partitioning MPI processes amoung the dimensions. Therefore the user will be responsible for providing this information. For example if we have 8 processes overall we can provide `npx = 2`, `npy= 4` or `npx = 1`, `npy = 8` and so on.
```
Note that we here also need to distinguish between periodic and non-periodic boundary conditions in each direction (This influences the way communication needs to be set up).

### Read in the input file
```cpp
dg::file::WrappedJsonValue js( dg::file::error::is_throw);
if( argc != 3 )
{
    DG_RANK0 std::cerr << "ERROR: Wrong number of arguments!\nUsage: "
            << argv[0]<<" [input.json] [output.nc]\n \n"<<std::endl;
    dg::abort_program();
}
try{
    dg::file::file2Json( argv[1], js.asJson(),
            dg::file::comments::are_discarded, dg::file::error::is_throw);
} catch( std::exception& e) {
    DG_RANK0 std::cerr << "ERROR in input file "<<argv[1]<<std::endl;
    DG_RANK0 std::cerr << e.what()<<std::endl;
    dg::abort_program();
}
DG_RANK0 std::cout << js.asJson() << std::endl;
```
Here, we notice the `DG_RANK0` macro, which expands to `if(rank==0)` when compiled with MPI and else stays empty. Note that all processes read in the input file.

(sec:main_file_construction)=
### Construction phase
Now, we have to construct the essential objects for time integration: Grid, vector with initial condition, Equations, Timestepper
```cpp
// Construct grid
unsigned n, Nx, Ny;
double x0, x1, y0, y1;
dg::bc bcx, bcy;
try{
    dg::file::WrappedJsonValue grid = js["grid"];
    n = grid.get( "n", 3).asUInt();
    Nx = grid.get( "Nx", 48).asUInt();
    Ny = grid.get( "Ny", 48).asUInt();
    x0 = grid["x"].get( 0u, 0.).asDouble();
    x1 = grid["x"].get( 1u, 1.).asDouble();
    y0 = grid["y"].get( 0u, 0.).asDouble();
    y1 = grid["y"].get( 1u, 1.).asDouble();
    bcx = dg::str2bc ( grid["bc"].get( 0u, "DIR").asString());
    bcy = dg::str2bc ( grid["bc"].get( 1u, "PER").asString());
}catch ( std::exception& error){
    DG_RANK0 std::cerr << "Error in input file " << argv[1]<< std::endl;
    DG_RANK0 std::cerr << error.what() << std::endl;
    dg::abort_program();
}
dg::x::CartesianGrid2d grid( x0, x1, y0, y1 , n, Nx, Ny, bcx, bcy
    // The MPI version of CartesianGrid2d needs a communicator:
    #ifdef WITH_MPI
    , comm
    #endif //WITH_MPI
    );

// Construct Equations
myproject::Equations<dg::x::CartesianGrid2d, dg::x::DMatrix,
    dg::x::DVec> rhs( grid, js);

// Construct initial condition
dg::x::DVec omega;
try{

    omega = myproject::initial_conditions(grid, js["init"] );
}catch ( std::exception& error){
    DG_RANK0 std::cerr << "Error in input file " << argv[1]<< std::endl;
    DG_RANK0 std::cerr << error.what() << std::endl;
    dg::abort_program();
}

// Construct timestepper
std::string tableau;
double rtol, atol, time = 0.;
try{
    rtol = js["timestepper"].get("rtol", 1e-5).asDouble();
    atol = js["timestepper"].get("atol", 1e-5).asDouble();
    tableau = js[ "timestepper"].get( "tableau", "Bogacki-Shampine-4-2-3").asString();
}catch ( std::exception& error){
    DG_RANK0 std::cerr << "Error in input file " << argv[1]<< std::endl;
    DG_RANK0 std::cerr << error.what() << std::endl;
    dg::abort_program();
}
dg::Adaptive< dg::ERKStep< dg::x::DVec>> adapt(tableau, omega);
dg::AdaptiveTimeloop<dg::x::DVec> timeloop( adapt, rhs,
                    dg::pid_control, dg::l2norm, rtol, atol);

```

In this part we notice that we put more requirements on the input file.
Specifically
```json
{
    "grid":
    {
        "n" : 3,
        "Nx" : 20,
        "Ny" : 20,
        "x" : [0,1],
        "y" : [0,1],
        "bc" : ["DIR", "PER"]
    },
    "timestepper":
    {
        "tableau" : "Bogacki-Shampine-4-2-3",
        "rtol" : 1e-5,
        "atol" : 1e-6
    }
}
```
Finally, we need to initialize the diag Variables
```cpp
myproject::Variables var = {rhs, grid, omega};
// trigger first computation of potential
{
    dg::x::DVec temp = omega;
    rhs( 0., omega, temp);
}

```

(sec:main_file_output)=
### Initialize the file output
```cpp
// Create netcdf file
dg::file::NC_Error_Handle err;
int ncid=-1;
try{
    DG_RANK0 err = nc_create( argv[2], NC_NETCDF4|NC_CLOBBER, &ncid);
}catch( std::exception& e)
{
    DG_RANK0 std::cerr << "ERROR creating file "<<argv[1]<<std::endl;
    DG_RANK0 std::cerr << e.what() << std::endl;
    dg::abort_program();
}
```
```{admonition} Meta-data
In order to fulfill the CF conventions there is a set of standard fields that should go into any netcdf file that you produce:
```
```cpp
std::map<std::string, std::string> att;
att["title"] = "Output file of myproject/myprogram.cpp";
att["Conventions"] = "CF-1.8";
///Get local time and begin file history
auto ttt = std::time(nullptr);

std::ostringstream oss;
///time string  + program-name + args
oss << std::put_time(std::localtime(&ttt), "%F %T %Z");
for( int i=0; i<argc; i++) oss << " "<<argv[i];
att["history"] = oss.str();
att["comment"] = "Find more info in myproject/documentation.tex";
att["source"] = "FELTOR";
att["git-hash"] = GIT_HASH;
att["git-branch"] = GIT_BRANCH;
att["compile-time"] = COMPILE_TIME;
att["references"] = "https://github.com/myname/myproject";
// Here we put the inputfile as a string without comments so that it can be read later by another parser
att["inputfile"] = js.asJson().toStyledString();
for( auto pair : att)
    DG_RANK0 err = nc_put_att_text( ncid, NC_GLOBAL,
        pair.first.data(), pair.second.size(), pair.second.data());
```
```{admonition} Compile time Macros
The relevant macros `GIT_HASH`, `GIT_BRANCH` and `COMPILE_TIME` need to be defined at compile time. We provide the Makefile `feltor/config/version.mk`. Simply include it in your own Makefile and add `$(VERSION_FLAGS)` to your compilation command!
```

### File output
For two-dimensional and three-dimensional programs it is often necessary to save on storage space when writing files. For this reason not every timestep is written to file but only every 100th say. Of course, the compression in time can be combined with a compression in space as well. In order to do so you would typically interpolate the simulation results onto a lower resolution grid.
```{note}
In this example we use the serial netcdf approach where all mpi threads send their data to the master thread which will write it to file. Pay special attention to which functions are called for all threads and which ones are only called by the master thread.
```
Let us set this up via some input parameters:
```cpp

unsigned n_out     = js[ "output"]["n"].asUInt( 3);
unsigned Nx_out    = js[ "output"]["Nx"].asUInt( 48);
unsigned Ny_out    = js[ "output"]["Ny"].asUInt( 48);

dg::x::CartesianGrid2d grid_out( x0, x1, y0, y1,
            n_out, Nx_out, Ny_out, bcx, bcy
            #ifdef WITH_MPI
            , comm
            #endif //WITH_MPI
            );
dg::x::IHMatrix projection = dg::create::interpolation( grid_out, grid);
```
#### Defining dimensions and variables
```cpp
int dim_ids[3], tvarID;
// the dimensions are the ones of grid_out!
err = dg::file::define_dimensions( ncid, dim_ids, &tvarID, grid_out,
                {"time", "y", "x"});

std::map<std::string, int> id1d, id3d;
for( auto& record : myproject::diagnostics2d_list)
{
    std::string name = record.name;
    std::string long_name = record.long_name;
    id3d[name] = 0;
    DG_RANK0 err = nc_def_var( ncid, name.data(), NC_DOUBLE, 3, dim_ids,
            &id3d.at(name));
    DG_RANK0 err = nc_put_att_text( ncid, id3d.at(name), "long_name",
            long_name.size(), long_name.data());
    // and the 1d fields (our idea is to just volume integrate the 2d fields
    name = name + "_1d";
    long_name = long_name + " (Volume integrated)";
    id1d[name] = 0;
    DG_RANK0 err = nc_def_var( ncid, name.data(), NC_DOUBLE, 1, &dim_ids[0],
        &id1d.at(name));
    DG_RANK0 err = nc_put_att_text( ncid, id1d.at(name), "long_name",
            long_name.size(), long_name.data());
}
```
#### Output the static list

For the static files we do not need to store the variable ids, we can directly write the output.
```cpp
dg::x::HVec resultH = dg::evaluate( dg::zero, grid);
dg::x::HVec transferH = dg::evaluate( dg::zero, grid_out);
dg::x::DVec resultD = transferH; // transfer to device
for( auto& record : myproject::diagnostics2d_static_list)
{
    std::string name = record.name;
    std::string long_name = record.long_name;
    int staticID = 0;
    DG_RANK0 err = nc_def_var( ncid, name.data(), NC_DOUBLE, 2, &dim_ids[1],
        &staticID);
    DG_RANK0 err = nc_put_att_text( ncid, staticID, "long_name",
                                   long_name.size(), long_name.data());
    record.function( resultD, var);
    dg::assign( resultD, resultH);
    dg::blas2::gemv( projection, resultH, transferH);
    dg::file::put_var_double( ncid, staticID, grid_out, transferH);
}
```
#### First file output
```cpp
dg::x::DVec volume = dg::create::volume( grid);
size_t start = {0};
size_t count = {1};
for( auto& record : myproject::diagnostics2d_list)
{
    record.function( resultD, var);
    double result = dg::blas1::dot( volume, resultD);
    dg::assign( resultD, resultH);
    dg::blas2::gemv( projection, resultH, transferH);
    // note that all processes call this function (for MPI)
    dg::file::put_vara_double( ncid, id3d.at(record.name), start,
            grid_out, transferH);
    // For the 1d output only the master thread needs to call the function
    DG_RANK0 err = nc_put_vara_double( ncid, id1d.at(record.name+"_1d"),
            &start, &count, &result);
}
DG_RANK0 err = nc_put_vara_double( ncid, tvarID, &start, &count, &time);
DG_RANK0 err = nc_close( ncid);
```

### The timeloop
We are finally ready to construct the main timeloop.
Normally, a file and/or dignostic output is wanted only at a fixed interval, determined by two of the three parameters
\begin{align}
    \Delta T_{output} = \frac{T_{end}}{N_{output}}
\end{align}
where $T_{end}$ is the simulation end time, $N_{output}$ is the number
of outputs in the output file and $\Delta T_{output}$ is the time
interval between outputs. Care must be taken that $N_{output}$ is integer.
```{note}
Usually, the adaptive time-output
given by simply letting an adaptive time-stepper run for a fixed number
of steps is not useful as it makes comparing simulations (with different spatial resolution say) difficult and
takes away control from the user.
```
Optionally, there is also a second diagnostic frequency, where for example higher time-resolved (low spatial) diagnostics need to be computed.
This requires a second output frequency
\begin{align}
    \delta T_{diag} = \frac{\Delta T_{output}}{N_{diag}}
\end{align}
```{note}
Especially for this higher frequency it is usually not necessary to output at _exactly_  the intermediary timesteps, just _close to those timesteps_. This is relevant because it is better to let the timestepper chose its own timestep over forcing it to land on a given point in time. This is shown in the following loop.
```
```cpp
double Tend = js["output"].get("tend", 1.0).asDouble();
unsigned maxout = js["output"].get("maxout", 10).asUInt();
double deltaT = Tend/(double)maxout;
bool abort = false;
for( unsigned u=1; u<=maxout; u++)
{
    try{
        // the documentation of dg::aTimeloop holds more details about how this construct works ...
        timeloop.integrate( time, omega, u*deltaT, omega,
                          u < maxout ? dg::to::at_least : dg::to::exact);
    }catch ( std::exception& fail)
    {
        DG_RANK0 std::cerr << "ERROR in Timestepper\n";
        DG_RANK0 std::cerr << fail.what() << std::endl;
        DG_RANK0 std::cerr << "Writing last output and exit ..."<<std::endl;
        abort = true;
    }
    start = u;
    DG_RANK0 err = nc_open(argv[2], NC_WRITE, &ncid);
    // First write the time variable
    DG_RANK0 err = nc_put_vara_double( ncid, tvarID, &start, &count, &time);
    for( auto& record : myproject::diagnostics2d_list)
    {
        record.function( resultD, var);
        double result = dg::blas1::dot( volume, resultD);
        dg::assign( resultD, resultH);
        dg::blas2::gemv( projection, resultH, transferH);
        dg::file::put_vara_double( ncid, id3d.at(record.name),
                                  start, grid_out, transferH);
        DG_RANK0 err = nc_put_vara_double( ncid, id1d.at(record.name+"_1d"),
                                          &start, &count, &result);
    }
    DG_RANK0 err = nc_close( ncid);
    if( abort) break;
}
```
```{admonition} Error handling mechanism
Here, a graceful exit strategy is
suggested, where an output is still written in case of failure in order to
analyse the reasons for failure.
```
```json
{
    "output":
    {
        "n" : 3,
        "Nx" : 20,
        "Ny" : 20,
        "tend" : 1.0,
        "maxout" : 10
    }
}
```

### Closing
```cpp
#ifdef WITH_MPI
    MPI_Finalize();
#endif // WITH_MPI
    return 0;
} //end of main
```

(sec:Makefile)=
## The Makefile
The easiest way to configure the Makefile variables is to simply include Feltor's configuration in your own Makefile
```make
# Makefile
device=omp
FELTOR_PATH=../feltor

#configure machine
include $(FELTOR_PATH)/config/default.mk
include $(FELTOR_PATH)/config/version.mk
include $(FELTOR_PATH)/config/*.mk
include $(FELTOR_PATH)/config/devices/devices.mk

INCLUDE+=-I$(FELTOR_PATH)/inc/

all: myprogram_hpc myprogram_mpi

# only necessary if you use the draw library
#myprogram: myprogram.cpp equations.h init.h diag.h
#    $(CC) $(OPT) $(CFLAGS) $< -o $@ $(INCLUDE) $(GLFLAGS) $(JSONLIB) -g

myprogram_hpc: myprogram.cpp equations.h init.h diag.h
    $(CC) $(OPT) $(CFLAGS) $< -o $@ $(INCLUDE) $(LIBS) $(JSONLIB) $(VERSION_FLAGS) -g

myprogram_mpi: myprogram.cpp equations.h init.h diag.h
    $(MPICC) $(OPT) $(MPICFLAGS) $< -o $@ $(INCLUDE) $(LIBS) $(JSONLIB) -DWITH_MPI $(VERSION_FLAGS) -g

.PHONY: clean

clean:
    rm -rf myprogram myprogram_hpc myprogram_mpi
```

```{admonition} git
If you change the `FELTOR_PATH` locally and you want git to ignore this line in the Makefile when you commit, you can follow this [stackoverflow question](https://stackoverflow.com/questions/6557467/can-git-ignore-a-specific-line)
```

Now you can compile your program like any feltor program. For example
```bash
make myprogram_hpc device=gpu # Compile for shared memory GPU platform
# or
make myprogram_mpi device=cpu # Compile pure mpi version
```
Then call the program with
```bash
./myprogram_hpc input.json output.nc
# or
echo 2 2 | mpirun -n 4 ./myprogram_mpi input.json output.nc
```

```c++

```


# Getting Started

## Preparing a radiative transfer object

After installation has finished, let's try to calculate a first spectrum!
You can simply load petitRADTRANS by typing:

```python
import numpy as np
from petitRADTRANS import Radtrans
```

An instance of a radiative transfer object can then be created like this:

```python
atmosphere = Radtrans(line_species = ['H2O', 'CO_all_iso', 'CH4', 'CO2', 'Na', 'K'], \
      rayleigh_species = ['H2', 'He'], \
      continuum_opacities = ['H2-H2', 'H2-He'], \
      wlen_bords_micron = [0.3, 15])
```

      Read CIA opacities for H2-H2...
      Read CIA opacities for H2-He...
     Done.

This will create an object ready to calculate spectra once the atmospheric parameters (temperature, abundances, mean molecular weight etc.) are set. Here we asked for the following molecular or atomic line absorbers to be inlcuded: $\rm H_2O$, $\rm CO$,  $\rm CH_4$, $\rm CO_2$, $\rm Na$, and $\rm K$. Additionally, the Rayleigh scattering cross-sections for $\rm H_2$ and $\rm He$ were loaded, as well as the collision induced absorption (CIA) cross-sections for the $\rm H_2$-$\rm H_2$ and $\rm H_2$-$\rm He$ pairs. The wavelength range that was loaded for the opacities here extends from 0.3 to 15 microns.

Now, let's define the pressures of the atmospheric layers.
Note that the pressures **must** always be sorted in increasing order, and be equidistant in log-space. The pressure is in units of **bar**, although all other units in *petitRADTRANS* are in cgs. Typically include of the order of 100 layers in your computations:

```python
pressures = np.logspace(-6, 2, 100)
```

**Units in petitRADTRANS:** all units in *petitRADTRANS* are in cgs, **except for pressure**, which is in bars, **and the mean molecular weight (MMW)**, which is in units of atomic mass units.

```python
atmosphere.setup_opa_structure(pressures)
```

And we are done! These steps only need to be carried out once, after this we are free to play with this radiative transfer object, for example change the temperature, abundances, etc. Only if additional opacity species need to be loaded is it necessary to create a new Radtrans instance.

## Calculating a transmission spectrum

Let's calculate a first spectrum for a hot Jupiter, first in transmission. Let's assume an isothermal temperature profile, for now. We start by specifing the temperature array, and abundance dictionery, containing the mass fractions of the atmospheric material, at every layer, and the atmospheric mean molecular weight, at every layer, in units of amu:

```python
temperature = 1200. * np.ones_like(pressures)

mass_fractions = {}
mass_fractions['H2'] = 0.74 * np.ones_like(temperature)
mass_fractions['He'] = 0.24 * np.ones_like(temperature)
mass_fractions['H2O'] = 0.001 * np.ones_like(temperature)
mass_fractions['CO_all_iso'] = 0.01 * np.ones_like(temperature)
mass_fractions['CO2'] = 0.00001 * np.ones_like(temperature)
mass_fractions['CH4'] = 0.000001 * np.ones_like(temperature)
mass_fractions['Na'] = 0.00001 * np.ones_like(temperature)
mass_fractions['K'] = 0.000001 * np.ones_like(temperature)

MMW = 2.33 * np.ones_like(temperature)
```

**Note that these are freely made up values, however. The P-T profile of a real planet may well be different, and not isothermal. Also the abundances are just made up for this simple example here** (see ["Interpolating chemical equilibrium abundances"](poor_man.html) for how to improve this.)

**Abundances in petitRADTRANS:** abundances in pRT are in units of **mass fractions**, not number fractions (aka volume mixing ratio, VMR). You can convert between mass fractions and VMRs by using
\begin{equation}
X_i = \frac{\mu_i}{\mu}n_i,
\end{equation}
where $X_i$ is the mass fraction of species $i$, $\mu_i$ the mass of a single molecule/atom/ion/... of species $i$, $\mu$ is the atmospheric mean molecular weight, and $n_i$ is the VMR of species $i$.

Next, let's assume a planetary radius and gravity at a given pressure $P_0=0.01$ bar in the atmosphere, and we are all set!

```python
from petitRADTRANS import nat_cst as nc
R_pl = 1.838*nc.r_jup_mean
gravity = 1e1**2.45
P0 = 0.01
```

Now we simply generate a transmission spectrum by typing:

```python
atmosphere.calc_transm(temperature, mass_fractions, gravity, MMW, R_pl=R_pl, P0_bar=P0)
```

Let's plot the transit radius!

```python
import pylab as plt
plt.rcParams['figure.figsize'] = (10, 6)

plt.plot(nc.c/atmosphere.freq/1e-4, atmosphere.transm_rad/nc.r_jup_mean)

plt.xscale('log')
plt.xlabel('Wavelength (microns)')
plt.ylabel(r'Transit radius ($\rm R_{Jup}$)')
plt.show()
plt.clf()
```

## Calculating an emission spectrum

**Scattering and petitRADTRANS:** scattering is included for emission spectra in petitRADTRANS only if requested specifically when generating the Radtrans object, as it increases the runtime (see ["Scattering for Emission Spectra"](emis_scat.html) for an example how to do this). We neglect the scattering here.

Let's calculate an emission spectrum. Using an isothermal temperature structure as above would simply result in a black body spectrum:

```python
atmosphere.calc_flux(temperature, mass_fractions, gravity, MMW)

plt.plot(nc.c/atmosphere.freq/1e-4, atmosphere.flux/1e-6)

plt.xscale('log')
plt.xlabel('Wavelength (microns)')
plt.ylabel(r'Planet flux $F_\nu$ (10$^{-6}$ erg cm$^{-2}$ s$^{-1}$ Hz$^{-1}$)')
plt.show()
plt.clf()
```

So let's try something different instead, namely the often-used analytical atmospheric P-T profile from [Guillot (2010)](http://adsabs.harvard.edu/abs/2010A%26A...520A..27G), his Equation 29:

\begin{equation}
T^4 = \frac{3T_{\rm int}^4}{4}\left(\frac{2}{3}+\tau\right) + \frac{3T_{\rm equ}^4}{4}\left[\frac{2}{3}+\frac{1}{\gamma\sqrt{3}}+\left(\frac{\gamma}{\sqrt{3}}-\frac{1}{\gamma\sqrt{3}}\right)e^{-\gamma\tau\sqrt{3}}\right]
\end{equation}

with $\tau = P\kappa_{\rm IR}/g$. Here, $\tau$ is the optical depth, $P$ the pressure, $\kappa_{\rm IR}$ is the atmospheric opacity in the IR wavelengths (i.e. the cross-section per unit mass), $\gamma$ is the ratio between the optical and IR opacity, $T_{\rm equ}$ the atmospheric equilibrium temperature, and $T_{\rm int}$ is the planetary internal temperature.

```python
kappa_IR = 0.01
gamma = 0.4
T_int = 200.
T_equ = 1500.

temperature = nc.guillot_global(pressures, kappa_IR, gamma, gravity, T_int, T_equ)
```

Let's plot the P-T profile:

```python
plt.plot(temperature, pressures)
plt.yscale('log')
plt.ylim([1e2, 1e-6])
plt.xlabel('T (K)')
plt.ylabel('P (bar)')
plt.show()
plt.clf()
```

An recalculate the spectrum:

```python
atmosphere.calc_flux(temperature, mass_fractions, gravity, MMW)

plt.plot(nc.c/atmosphere.freq/1e-4, atmosphere.flux/1e-6)

plt.xscale('log')
plt.xlabel('Wavelength (microns)')
plt.ylabel(r'Planet flux $F_\nu$ (10$^{-6}$ erg cm$^{-2}$ s$^{-1}$ Hz$^{-1}$)')
plt.show()
plt.clf()
```

```python

```