```python
class Dice:                           ### fig:dice1 ###
    def __init__(self):
        self.numbers = {1,2,3,4,5,6}

dice = Dice()
print(dice.numbers)
```

    {1, 2, 3, 4, 5, 6}

```python
1 in dice.numbers   ### fig:1_in_dice ####
```

    True

```python
7 not in dice.numbers    ### fig:1_not_in_dice ####
```

    True

```python
import random   # モジュールのインポート                        ### fig:dice2 ###

class Dice:
    def __init__(self):  self.numbers = {1,2,3,4,5,6}
    def roll(self):      return random.choice(list(self.numbers))         # 追加
    def rolls(self,num): return random.choices(list(self.numbers), k=num) # 追加

dice = Dice()
### roll, rollsを使ってみましょう ###
print(dice.roll())
print(dice.rolls(10))
```

    3
    [1, 5, 3, 5, 6, 4, 3, 3, 6, 4]

```python
S_bag = {"R","R","R","W","W","W","W"}       ### fig:wrong_set ###
print(S_bag)
```

    {'R', 'W'}

```python
S_bag = {"R_1","R_2","R_3","W_1","W_2","W_3","W_4"}       ### fig:ball_set ###
print(S_bag)
```

    {'R_1', 'R_3', 'R_2', 'W_1', 'W_3', 'W_4', 'W_2'}

```python
class Bag:                                                  ### fig:bag_and_ball ###
    def __init__(self):
        self.contents = set()  # 空の集合をひとつ作っておく

    def add(self,ball):
        self.contents.add(ball) # ボールを袋に入れる操作

class Ball:
    def __init__(self, color):
        self.color = color     # ボールは色を示す変数（文字列の予定）だけ持つ

bag = Bag()                    # 袋を一つ作る

for i in range(3):             # 赤い玉を3つ加える
    bag.add(Ball("red"))
for i in range(4):             # 白い玉を4つ加える
    bag.add(Ball("white"))

print(bag.contents)
```

    {<__main__.Ball object at 0x1089ee048>, <__main__.Ball object at 0x1089ee470>, <__main__.Ball object at 0x1089ee080>, <__main__.Ball object at 0x1089eeb00>, <__main__.Ball object at 0x1089ee978>, <__main__.Ball object at 0x1089ee5c0>, <__main__.Ball object at 0x1089ee9e8>}

```python
class Bag:                                                  ### fig:bag_and_ball2 ###
    def __init__(self):              # @@@省略@@@
        self.contents = set()        # @@@省略@@@

    def add(self,ball):              # @@@省略@@@
        self.contents.add(ball)      # @@@省略@@@

    def pop(self):                              # このメソッドを追加
        if not self.contents:
            return None                        #　空ならNoneを返す

        b = random.choice(list(self.contents))
        self.contents.remove(b)                 # setから選んだボールを消す
        return b

class Ball:                       # @@@省略@@@
    def __init__(self, color):   # @@@省略@@@
        self.color = color       # @@@省略@@@

bag = Bag()
for i in range(3):
    bag.add(Ball("red"))
for i in range(4):
    bag.add(Ball("white"))

result = []                 # 選んだボールの色を記録するためのリスト
b = bag.pop()
while b:
    result.append(b.color)
    b = bag.pop()

print(result)
```

    ['red', 'red', 'red', 'white', 'white', 'white', 'white']

```python
class Agent:                      ### fig:dice_probability ###
    def P_dice(self, dice, event):
        return len(event)/len(dice.numbers)

agent = Agent()
print(agent.P_dice(dice, {2,4,6}))         # 偶数
print(agent.P_dice(dice, {3,4,5,6}))       # 3以上
print(agent.P_dice(dice, {}))              # 要素が空
print(agent.P_dice(dice, {1,2,3,4,5,6}))   # 全要素
```

    0.5
    0.6666666666666666
    0.0
    1.0

```python
print(agent.P_dice(dice, 3))        # 集合にせずに出目を与える      ### fig:only_give_element ###
```

```python
import sympy                           ### fig:powerset ###
s = sympy.FiniteSet(*dice.numbers)
print(s.powerset())
```

    {EmptySet(), {1}, {2}, ..., {1, 3, 4, 5, 6}, {2, 3, 4, 5, 6}, {1, 2, 3, 4, 5, 6}}

```python
s = sympy.FiniteSet(*dice.numbers)

print(s**3)

twice = set(e for e in s**3)
print(twice)
print(len(twice))
```

    {1, 2, 3, 4, 5, 6} x {1, 2, 3, 4, 5, 6} x {1, 2, 3, 4, 5, 6}
    {(4, 2, 2), (1, 4, 4), (2, 2, 4), (5, 5, 1), (5, 2, 1), (1, 4, 2), (5, 5, 3), (3, 1, 6), (5, 2, 3), (5, 5, 5), (3, 1, 4), (2, 6, 5), (3, 2, 2), (4, 1, 5), (3, 1, 2), (2, 6, 3), (6, 5, 5), (2, 5, 3), (4, 4, 2), (1, 2, 2), (6, 6, 3), (2, 6, 1), (6, 5, 3), (3, 2, 6), (2, 5, 1), (4, 6, 1), (4, 1, 1), (1, 2, 4), (6, 6, 1), (5, 3, 2), (1, 5, 5), (6, 5, 1), (3, 2, 4), (4, 6, 3), (4, 1, 3), (1, 2, 6), (2, 5, 5), (4, 6, 5), (1, 3, 5), (6, 3, 1), (4, 3, 6), (6, 6, 5), (5, 3, 6), (1, 5, 1), (3, 4, 5), (2, 3, 4), (1, 3, 3), (6, 3, 3), (4, 3, 4), (5, 6, 2), (5, 3, 4), (1, 5, 3), (2, 3, 6), (1, 3, 1), (6, 3, 5), (4, 3, 2), (5, 6, 4), (6, 4, 4), (3, 3, 1), (5, 1, 5), (3, 4, 1), (6, 2, 6), (5, 6, 6), (6, 4, 6), (3, 3, 3), (3, 4, 3), (2, 3, 2), (6, 2, 4), (6, 1, 2), (3, 3, 5), (2, 4, 2), (5, 1, 1), (3, 6, 6), (6, 2, 2), (5, 4, 3), (1, 6, 4), (6, 4, 2), (5, 1, 3), (2, 4, 4), (3, 6, 4), (4, 5, 2), (6, 1, 6), (5, 4, 1), (1, 6, 6), (2, 4, 6), (3, 6, 2), (2, 1, 5), (4, 5, 4), (4, 2, 5), (6, 1, 4), (3, 5, 2), (2, 2, 3), (4, 5, 6), (2, 1, 3), (5, 4, 5), (1, 6, 2), (2, 2, 1), (4, 4, 5), (2, 1, 1), (4, 2, 1), (1, 1, 2), (5, 2, 4), (3, 5, 6), (4, 4, 3), (4, 2, 3), (1, 1, 4), (5, 2, 6), (3, 5, 4), (2, 2, 5), (4, 4, 1), (1, 1, 6), (1, 4, 5), (5, 2, 2), (1, 4, 3), (5, 5, 2), (3, 1, 5), (2, 6, 6), (6, 5, 6), (1, 4, 1), (4, 1, 4), (5, 5, 4), (3, 1, 3), (2, 6, 4), (6, 5, 4), (3, 2, 3), (2, 5, 2), (4, 1, 6), (1, 2, 1), (5, 5, 6), (3, 1, 1), (2, 6, 2), (6, 5, 2), (3, 2, 1), (1, 2, 3), (6, 6, 2), (5, 3, 3), (2, 5, 6), (4, 1, 2), (1, 2, 5), (5, 3, 1), (1, 5, 4), (3, 2, 5), (2, 5, 4), (4, 6, 2), (1, 3, 6), (6, 6, 6), (1, 5, 6), (3, 4, 4), (4, 6, 4), (1, 3, 4), (6, 3, 2), (5, 6, 1), (6, 6, 4), (5, 3, 5), (3, 4, 6), (2, 3, 5), (4, 6, 6), (1, 3, 2), (6, 3, 4), (4, 3, 5), (5, 6, 3), (6, 4, 5), (1, 5, 2), (6, 3, 6), (4, 3, 3), (5, 6, 5), (5, 1, 4), (2, 4, 1), (3, 4, 2), (2, 3, 1), (6, 2, 5), (4, 3, 1), (6, 4, 1), (3, 3, 2), (2, 4, 3), (5, 1, 6), (2, 3, 3), (6, 2, 3), (6, 1, 3), (5, 4, 2), (6, 4, 3), (3, 3, 4), (2, 4, 5), (4, 5, 1), (2, 1, 6), (6, 2, 1), (6, 1, 1), (1, 6, 5), (3, 3, 6), (5, 1, 2), (3, 6, 5), (2, 1, 4), (4, 5, 3), (5, 4, 6), (3, 5, 3), (3, 6, 3), (2, 1, 2), (4, 5, 5), (4, 2, 4), (1, 1, 1), (6, 1, 5), (5, 4, 4), (1, 6, 1), (3, 5, 1), (2, 2, 2), (4, 4, 6), (3, 6, 1), (4, 2, 6), (1, 1, 3), (1, 6, 3), (4, 4, 4), (1, 1, 5), (5, 2, 5), (1, 4, 6), (3, 5, 5), (2, 2, 6)}
    216

```python
6**3
```

    216

```python

```


```python
import numpy as np
import scipy as sc
import random as rand
from sklearn import preprocessing, linear_model
import matplotlib.pyplot as plt

from core.controllers import PDController
from core.dynamics import LinearSystemDynamics, ConfigurationDynamics

from koopman_core.controllers import OpenLoopController, MPCController,BilinearFBLinController, PerturbedController, LinearLiftedController
from koopman_core.dynamics import LinearLiftedDynamics, BilinearLiftedDynamics
from koopman_core.learning import Edmd, BilinearEdmd
from koopman_core.basis_functions import PlanarQuadBasis
from koopman_core.learning.utils import differentiate_vec
from koopman_core.systems import PlanarQuadrotorForceInput

class QuadrotorPdOutput(ConfigurationDynamics):
    def __init__(self, dynamics, xd, t_d, n, m):
        ConfigurationDynamics.__init__(self, dynamics, 1)
        self.xd = xd
        self.t_d = t_d
        self.xd_dot = differentiate_vec(self.xd, self.t_d)
        self.n = n
        self.m = m

    def proportional(self, x, t):
        q, q_dot = x[:int(n/2)], x[int(n/2):]
        return self.y(q) - self.y_d(t)

    def derivative(self, x, t):
        q, q_dot = x[:int(n/2)], x[int(n/2):]
        return self.dydq(q)@q_dot - self.y_d_dot(t)

    def y(self, q):
        return q

    def dydq(self, q):
        return np.eye(int(self.n/2))

    def d2ydq2(self, q):
        return np.zeros((int(self.n/2), int(self.n/2), int(self.n/2)))

    def y_d(self, t):
        return self.desired_state_(t)[:int(self.n/2)]

    def y_d_dot(self, t):
        return self.desired_state_(t)[int(self.n/2):]

    def y_d_ddot(self, t):
        return self.desired_state_dot_(t)[int(self.n/2):]

    def desired_state_(self, t):
        return [np.interp(t, self.t_d.flatten(),self.xd[:,ii].flatten()) for ii in range(self.xd.shape[1])]

    def desired_state_dot_(self, t):
        return [np.interp(t, self.t_d.flatten(),self.xd_dot[:,ii].flatten()) for ii in range(self.xd_dot.shape[1])]

class PlanarQuadrotorForceInputDiscrete(PlanarQuadrotorForceInput):
    def __init__(self, mass, inertia, prop_arm, g=9.81, dt=1e-2):
        PlanarQuadrotorForceInput.__init__(self, mass, inertia, prop_arm, g=g)
        self.dt=dt

    def eval_dot(self, x, u, t):
        return x + self.dt*self.drift(x, t) + self.dt*np.dot(self.act(x, t),u)

    def get_linearization(self, x0, x1, u0, t):
        m, J, b, g = self.params
        A_lin = np.eye(self.n) + self.dt*np.array([[0, 0, 0, 1, 0, 0],
                                                   [0, 0, 0, 0, 1, 0],
                                                   [0, 0, 0, 0, 0, 1],
                                                   [0, 0, -(1/m)*np.cos(x0[2])*u0[0] -(1/m)*np.cos(x0[2])*u0[1], 0, 0, 0],
                                                   [0, 0, -(1/m)*np.sin(x0[2])*u0[0] -(1/m)*np.sin(x0[2])*u0[1], 0, 0, 0],
                                                   [0, 0, 0, 0, 0, 0],])

        B_lin = self.dt*np.array([[0, 0],
                                  [0, 0],
                                  [0, 0],
                                  [-(1/m)*np.sin(x0[2]), -(1/m)*np.sin(x0[2])],
                                  [(1/m)*np.cos(x0[2]), (1/m)*np.cos(x0[2])],
                                  [-b/J, b/J]])

        if x1 is None:
            x1 = A_lin@x0 + B_lin@u0

        f_d = self.eval_dot(x0,u0,t)
        r_lin = f_d - x1

        return A_lin, B_lin, r_lin

```

    /Users/carlaxelfolkestad/opt/anaconda3/envs/crazyswarm/lib/python3.8/site-packages/ray/autoscaler/_private/cli_logger.py:57: FutureWarning: Not all Ray CLI dependencies were found. In Ray 1.4+, the Ray CLI, autoscaler, and dashboard will only be usable via `pip install 'ray[default]'`. Please update your install command.
      warnings.warn(

## Planar Quadrotor Example

Consider a planar quadrotor with states $\mathbf{x} = [y \, z \, \theta \, \dot{y} \, \dot{z} \, \dot{\theta}]^T$ and continuous-time dynamics

\begin{equation}
    \begin{bmatrix} \ddot{y} \\ \ddot{z} \\ \ddot{\theta} \end{bmatrix}
    = \begin{bmatrix}
    0\\-g\\0
    \end{bmatrix} +
    \begin{bmatrix}
    -\frac{1}{m}\text{sin}\theta & -\frac{1}{m}\text{sin}\theta\\
    \frac{1}{m}\text{cos}\theta & \frac{1}{m}\text{cos}\theta\\
    -\frac{l_{arm}}{I_{xx}} & \frac{l_{arm}}{I_{xx}}
    \end{bmatrix}
    \begin{bmatrix}
    T_1 \\ T_2
    \end{bmatrix}
\end{equation}

where $y,z$ describe the position of the vehicle in a fixed reference frame, $\theta$ is the orientation of the vehicle,
$T_1, T_2$ are the thrust from each of the propellers, $g$ is the gravitational acceleration, $m$ is the vehicle mass,
$l_{arm}$ is the distance from the vehicle's center of mass to the center of the propeller, and $I_{xx}$ is the inertia
around the x-axis.

```python
# Cart pole system parameters
mass = 2.
inertia = 1.
prop_arm = 0.2
gravity = 9.81
quadrotor = PlanarQuadrotorForceInput(mass, inertia, prop_arm, g=gravity)

# Linearized system specification:
n, m = 6, 2                                                         # Number of states, number of control inputs
A_nom = np.array([[0., 0., 0., 1., 0., 0.],                         # Linearization of the true system around the origin
                  [0., 0., 0., 0., 1., 0.],
                  [0., 0., 0., 0., 0., 1.],
                  [0., 0., -gravity, 0., 0., 0.],
                  [0., 0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0., 0.]])
B_nom = np.array([[0., 0.],                                         # Linearization of the true system around the origin
                  [0., 0.],
                  [0., 0.],
                  [0., 0.],
                  [1./mass, 1./mass],
                  [-prop_arm/inertia, prop_arm/inertia]])

hover_thrust = mass*gravity/m
```

### Collect data for learning

To collect data, a nominal controller is designed with LQR on the dynamics's linearization around hover. However, any
controller can be used and the method does not require the knowledge of model's linearization. In addition, a
exploratory white noise is added to the controller to ensure that the data is sufficiently excited. Note that the system
is underactuated and that trajectory optimization is necessary to control the position of the vehicle. We use a
simplified trajectory generator based on a model predictive controller for the linearized dynamics. More careful design
of the desired trajectory may be necessary for more demanding applications and this is readily compatible with our method.

```python
q_dc, r_dc = 5e2, 1                                                 # State and actuation penalty values, data collection
Q_dc = q_dc * np.identity(n)                                        # State penalty matrix, data collection
R_dc = r_dc*np.identity(m)                                          # Actuation penalty matrix, data collection
P_dc = sc.linalg.solve_continuous_are(A_nom, B_nom, Q_dc, R_dc)     # Algebraic Ricatti equation solution, data collection
K_dc = np.linalg.inv(R_dc)@B_nom.T@P_dc                             # LQR feedback gain matrix, data collection
K_dc_p = K_dc[:,:int(n/2)]                                          # Proportional control gains, data collection
K_dc_d = K_dc[:,int(n/2):]                                          # Derivative control gains, data collection
nominal_sys = LinearLiftedDynamics(A_nom, B_nom, np.eye(n), lambda x: x)

# Data collection parameters:
dt = 1.0e-2                                                         # Time step length
traj_length_dc = 2.                                                 # Trajectory length, data collection
n_pred_dc = int(traj_length_dc/dt)                                  # Number of time steps, data collection
t_eval = dt * np.arange(n_pred_dc + 1)                              # Simulation time points
n_traj_dc = 100                                                     # Number of trajectories to execute, data collection
noise_var = 5.                                                      # Exploration noise to perturb controller, data collection

xmax = np.array([2, 2, np.pi/3, 2.,2.,2.])                          # State constraints, trajectory generation
xmin = -xmax
umax = np.array([2*hover_thrust, 2*hover_thrust]) - hover_thrust    # Actuation constraint, trajectory generation
umin = np.array([0., 0.]) - hover_thrust
x0_max = np.array([xmax[0], xmax[1], xmax[2], 1., 1., 1.])          # Initial value limits
Q_trajgen = sc.sparse.diags([0,0,0,0,0,0])                          # State penalty matrix, trajectory generation
QN_trajgen = sc.sparse.diags([5e1,5e1,5e1,1e1,1e1,1e1])             # Final state penalty matrix, trajectory generation
R_trajgen = sc.sparse.eye(m)                                        # Actuation penalty matrix, trajectory generation
sub_sample_rate = 1                                                 # Rate to subsample data for training
model_fname = 'examples/planar_quad_models'                         # Path to save learned models
n_cols = 10                                                         # Number of columns in training data plot
save_figures = False
dropbox_folder = '/Users/carlaxelfolkestad/Dropbox/Apps/Overleaf/Koopman NMPC (ICRA21)/'
```

```python
xd = np.empty((n_traj_dc, n_pred_dc + 1, n))
xs = np.empty((n_traj_dc, n_pred_dc + 1, n))
us = np.empty((n_traj_dc, n_pred_dc, m))

plt.figure(figsize=(12, 12 * n_traj_dc / (n_cols ** 2)))
for ii in range(n_traj_dc):
    x0 = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
    set_pt_dc = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
    mpc_trajgen = MPCController(nominal_sys, n_pred_dc, dt, umin, umax, xmin, xmax, QN_trajgen, R_trajgen,
                                QN_trajgen, set_pt_dc)
    mpc_trajgen.eval(x0, 0)
    xd[ii, :, :] = mpc_trajgen.parse_result().T
    while np.linalg.norm(x0[:3] - set_pt_dc[:3]) < 2 or np.any(np.isnan(xd[ii, :, :])):
        x0 = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
        set_pt_dc = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
        mpc_trajgen = MPCController(nominal_sys, n_pred_dc, dt, umin, umax, xmin, xmax, QN_trajgen, R_trajgen,
                                    QN_trajgen, set_pt_dc)
        mpc_trajgen.eval(x0, 0)
        xd[ii, :, :] = mpc_trajgen.parse_result().T

    output = QuadrotorPdOutput(quadrotor, xd[ii, :, :], t_eval, n, m)
    pd_controller = PDController(output, K_dc_p, K_dc_d)
    perturbed_pd_controller = PerturbedController(quadrotor, pd_controller, noise_var, const_offset=hover_thrust)
    xs[ii, :, :], us[ii, :, :] = quadrotor.simulate(x0, perturbed_pd_controller, t_eval)

    plt.subplot(int(np.ceil(n_traj_dc / n_cols)), n_cols, ii + 1)
    plt.plot(t_eval, xs[ii, :, 0], 'b', label='$y$')
    plt.plot(t_eval, xs[ii, :, 1], 'g', label='$z$')
    plt.plot(t_eval, xs[ii, :, 2], 'r', label='$\\theta$')
    plt.plot(t_eval, xd[ii, :, 0], '--b', label='$y_d$')
    plt.plot(t_eval, xd[ii, :, 1], '--g', label='$z_d$')
    plt.plot(t_eval, xd[ii, :, 2], '--r', label='$\\theta_d$')

plt.suptitle(
    'Training data \nx-axis: time (sec), y-axis: state value, $x$ - blue, $xd$ - dotted blue, $\\theta$ - red, $\\theta_d$ - dotted red',
    y=0.94)
plt.show()
```

### Learn a linear model with dynamic mode decomposition (DMD)

To compare our method with existing techniques, we first learn a linear state space model from data. This is dubbed
dynamic mode decomposition. I.e. we use linear regression with LASSO regularization to learn an approximate linear model
with model structure

\begin{equation}
    \mathbf{\dot{x}} = A_{dmd}\mathbf{x} + B_{dmd}\mathbf{u}
\end{equation}

```python
#DMD parameters:
alpha_dmd = 9.8e-5                                                  # Regularization strength (LASSO) DMD
tune_mdl_dmd = False
```

```python
basis = lambda x: x
C_dmd = np.eye(n)

optimizer_dmd = linear_model.MultiTaskLasso(alpha=alpha_dmd, fit_intercept=False, selection='random')
cv_dmd = linear_model.MultiTaskLassoCV(fit_intercept=False, n_jobs=-1, cv=3, selection='random')
standardizer_dmd = preprocessing.StandardScaler(with_mean=False)

model_dmd = Edmd(n, m, basis, n, n_traj_dc, optimizer_dmd, cv=cv_dmd, standardizer=standardizer_dmd, C=C_dmd, first_obs_const=False, continuous_mdl=False, dt=dt)
xdmd, y_dmd = model_dmd.process(xs, us-hover_thrust, np.tile(t_eval,(n_traj_dc,1)), downsample_rate=sub_sample_rate)
model_dmd.fit(xdmd, y_dmd, cv=tune_mdl_dmd, override_kinematics=True)
sys_dmd = LinearLiftedDynamics(model_dmd.A, model_dmd.B, model_dmd.C, model_dmd.basis, continuous_mdl=False, dt=dt)
if tune_mdl_dmd:
    print('$\\alpha$ DMD: ',model_dmd.cv.alpha_)
```

### Learn a lifted linear model with extended dynamic mode decomposition (EDMD)

In addition, we compare our method with the current state of the art of Koopman based learning, the extended dynamic mode
decomposition. We use a dictionary of nonlinear functions $\boldsymbol{\phi(x)}$ to lift the state variables and learn a lifted state space model
of the dynamics. I.e. we first lift and then use linear regression with LASSO regularization to learn an approximate
lifted linear model with model structure

\begin{equation}
    \mathbf{\dot{z}} = A_{edmd}\mathbf{z} + B_{edmd}\mathbf{u}, \qquad \mathbf{z} = \boldsymbol{\phi(x)}
\end{equation}

```python
#EDMD parameters:
alpha_edmd = 2.22e-4                                                 # Regularization strength (LASSO) EDMD
tune_mdl_edmd = False
```

```python
basis = PlanarQuadBasis(n, poly_deg=3)
basis.construct_basis()
planar_quad_features = preprocessing.FunctionTransformer(basis.basis)
planar_quad_features.fit(np.zeros((1,n)))
n_lift_edmd = planar_quad_features.transform((np.zeros((1,n)))).shape[1]
C_edmd = np.zeros((n,n_lift_edmd))
C_edmd[:,1:n+1] = np.eye(n)

optimizer_edmd = linear_model.MultiTaskLasso(alpha=alpha_edmd, fit_intercept=False, selection='random', max_iter=2000)
cv_edmd = linear_model.MultiTaskLassoCV(fit_intercept=False, n_jobs=-1, cv=3, selection='random', max_iter=2000)
standardizer_edmd = preprocessing.StandardScaler(with_mean=False)

model_edmd = Edmd(n, m, basis.basis, n_lift_edmd, n_traj_dc, optimizer_edmd, cv=cv_edmd, standardizer=standardizer_edmd, C=C_edmd, continuous_mdl=False, dt=dt)
X_edmd, y_edmd = model_edmd.process(xs, us-hover_thrust, np.tile(t_eval,(n_traj_dc,1)), downsample_rate=sub_sample_rate)
model_edmd.fit(X_edmd, y_edmd, cv=tune_mdl_edmd, override_kinematics=True)
sys_edmd = LinearLiftedDynamics(model_edmd.A, model_edmd.B, model_edmd.C, model_edmd.basis, continuous_mdl=False, dt=dt)
if tune_mdl_edmd:
    print('$\\alpha$ EDMD: ',model_edmd.cv.alpha_)

```

### Learn a lifted bilinear model with bilinear extended mode decomposition (bEDMD)

Finally, we use the method developed in the paper to learn a lifted bilinear model of the dynamics, dubbed bilinear
extended mode decomposition (bEDMD). I.e. we first lift and then use linear regression with LASSO regularization to learn an approximate
lifted linear model with model structure

\begin{equation}
    \mathbf{\dot{z}}=F\mathbf{z}+\sum_{i=1}^m G_i\mathbf{z}\mathbf{u}_i, \qquad \mathbf{z} = \boldsymbol{\phi(x)}
\end{equation}

```python
#Bilinear EDMD parameters:
alpha_bedmd = 6.9e-5  # Regularization strength (LASSO) bEDMD
tune_mdl_bedmd = False
```

```python
n_lift_bedmd = n_lift_edmd
C_bedmd = np.zeros((n,n_lift_bedmd))
C_bedmd[:,1:n+1] = np.eye(n)

basis_bedmd = lambda x: planar_quad_features.transform(x)
optimizer_bedmd = linear_model.MultiTaskLasso(alpha=alpha_bedmd, fit_intercept=False, selection='random', max_iter=1e4)
cv_bedmd = linear_model.MultiTaskLassoCV(fit_intercept=False, n_jobs=-1, cv=3, selection='random')
standardizer_bedmd = preprocessing.StandardScaler(with_mean=False)

model_bedmd = BilinearEdmd(n, m, basis_bedmd, n_lift_bedmd, n_traj_dc, optimizer_bedmd, cv=cv_bedmd, standardizer=standardizer_bedmd, C=C_bedmd, continuous_mdl=False, dt=dt)
X_bedmd, y_bedmd = model_bedmd.process(xs, us-hover_thrust, np.tile(t_eval,(n_traj_dc,1)), downsample_rate=sub_sample_rate)
model_bedmd.fit(X_bedmd, y_bedmd, cv=tune_mdl_bedmd, override_kinematics=True)
sys_bedmd = BilinearLiftedDynamics(model_bedmd.n_lift, m, model_bedmd.A, model_bedmd.B, model_bedmd.C, model_bedmd.basis, continuous_mdl=False, dt=dt)
if tune_mdl_bedmd:
    print('$\\alpha$ bilinear EDMD: ', model_bedmd.cv.alpha_)

```

### Evaluate open loop prediction performance

We first evaluate the open loop prediction performance of the proposed method.
This is done by generating a new data set in the same way as the training set, predicting the evolution of the system
with the control sequence of each trajectory executed in the data set with each of the models, and finally comparing
the mean and standard deviation of the error between the true and predicted evolution over the trajectories. The
experimental results support what is to be expected from the theory as the error in the $y$ and $z$ terms are
significantly lower for the bEDMD method than both DMD and EDMD. The reason for this
improvement is that the bEDMD method can capture the nonlinearities present in the actuation matrix of the
$(y,z)$-dynamics.

```python
# Prediction performance evaluation parameters:
folder_plots = 'working_files/figures/'                                  # Path to save plots
n_traj_ol = 100                                                     # Number of trajectories to execute, open loop
```

```python
from tabulate import tabulate

xs_ol = np.empty((n_traj_ol, t_eval.shape[0], n))
xs_dmd_ol = np.empty((n_traj_ol, t_eval.shape[0]-1, n))
xs_edmd_ol = np.empty((n_traj_ol, t_eval.shape[0]-1, n))
xs_bedmd_ol = np.empty((n_traj_ol, t_eval.shape[0]-1, n))
us_test = np.empty((n_traj_ol, t_eval.shape[0]-1, m))

for ii in range(n_traj_ol):
    x0 = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
    set_pt_dc = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
    mpc_trajgen = MPCController(nominal_sys, n_pred_dc, dt, umin, umax, xmin, xmax, QN_trajgen, R_trajgen,
                                QN_trajgen, set_pt_dc)
    mpc_trajgen.eval(x0, 0)
    xd = mpc_trajgen.parse_result().T

    while xd[0,0] is None:
        x0 = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
        set_pt_dc = np.asarray([rand.uniform(l, u) for l, u in zip(-x0_max, x0_max)])
        mpc_trajgen = MPCController(nominal_sys, n_pred_dc, dt, umin, umax, xmin, xmax, QN_trajgen, R_trajgen,
                                    QN_trajgen, set_pt_dc)
        mpc_trajgen.eval(x0, 0)
        xd = mpc_trajgen.parse_result().T

    output = QuadrotorPdOutput(quadrotor, xd, t_eval, n, m)
    pd_controller = PDController(output, K_dc_p, K_dc_d)
    perturbed_pd_controller = PerturbedController(quadrotor, pd_controller, noise_var, const_offset=hover_thrust)

    xs_ol[ii,:,:], us_test[ii,:,:] = quadrotor.simulate(x0, perturbed_pd_controller, t_eval)
    ol_controller_nom = OpenLoopController(sys_bedmd, us_test[ii,:,:]-hover_thrust, t_eval[:-1])

    xs_dmd_ol[ii,:,:], _ = sys_dmd.simulate(x0, ol_controller_nom, t_eval[:-1])

    z_0_edmd = sys_edmd.basis(np.atleast_2d(x0)).squeeze()
    zs_edmd_tmp, _ = sys_edmd.simulate(z_0_edmd, ol_controller_nom, t_eval[:-1])
    xs_edmd_ol[ii,:,:] = np.dot(model_edmd.C, zs_edmd_tmp.T).T

    z_0_bedmd = sys_bedmd.basis(np.atleast_2d(x0)).squeeze()
    zs_bedmd_tmp, _ = sys_bedmd.simulate(z_0_bedmd, ol_controller_nom, t_eval[:-1])
    xs_bedmd_ol[ii,:,:] = np.dot(model_bedmd.C, zs_bedmd_tmp.T).T

error_dmd = xs_ol[:,:-1,:] - xs_dmd_ol
error_dmd_mean = np.mean(error_dmd, axis=0).T
error_dmd_std = np.std(error_dmd, axis=0).T
mse_dmd = np.mean(np.square(error_dmd))
std_dmd = np.std(error_dmd)

error_edmd = xs_ol[:,:-1,:] - xs_edmd_ol
error_edmd_mean = np.mean(error_edmd, axis=0).T
error_edmd_std = np.std(error_edmd, axis=0).T
mse_edmd = np.mean(np.square(error_edmd))
std_edmd = np.std(error_edmd)

error_bedmd = xs_ol[:,:-1,:] - xs_bedmd_ol
error_bedmd_mean = np.mean(error_bedmd, axis=0).T
error_bedmd_std = np.std(error_bedmd, axis=0).T
mse_bedmd = np.mean(np.square(error_bedmd))
std_bedmd = np.std(error_bedmd)

print('\nOpen loop performance statistics:\n')
print(tabulate([['DMD', "{:.5f}".format(mse_dmd), '-', '-', "{:.5f}".format(std_dmd), '-', '-'],
               ['EDMD', "{:.5f}".format(mse_edmd), "{:.2f}".format((1 - mse_edmd / mse_dmd) * 100)+' %', '-', "{:.5f}".format(std_edmd), "{:.2f}".format((1 - std_edmd / std_dmd) * 100)+' %', '-'],
               ['bEDMD', "{:.5f}".format(mse_bedmd), "{:.2f}".format((1 - mse_bedmd / mse_dmd) * 100)+' %', "{:.2f}".format((1 - mse_bedmd / mse_edmd) * 100)+' %', "{:.5f}".format(std_bedmd), "{:.2f}".format((1 - std_bedmd / std_dmd) * 100)+' %', "{:.2f}".format((1 - std_bedmd / std_edmd) * 100)+' %']],
               headers=['MSE', 'MSE improvement\nover DMD', 'MSE improvement\nover EDMD', 'Standard\ndeviation', 'std improvement\nover DMD', 'std improvement\nover EDMD']))
```

    Open loop performance statistics:

               MSE  MSE improvement    MSE improvement       Standard  std improvement    std improvement
                    over DMD           over EDMD            deviation  over DMD           over EDMD
    -----  -------  -----------------  -----------------  -----------  -----------------  -----------------
    DMD    0.0944   -                  -                      0.29415  -                  -
    EDMD   0.06139  34.97 %            -                      0.24754  15.85 %            -
    bEDMD  0.00241  97.45 %            96.08 %                0.04893  83.37 %            80.23 %

```python
import matplotlib.pyplot as plt
import matplotlib

figwidth = 12
lw = 2
fs = 16
y_lim_gain = 1.2

#Plot open loop results:
ylabels = ['$e_{y}$', '$e_z$', '$e_{\\theta}$']
plt.figure(figsize=(figwidth,3))
for ii in range(3):
    plt.subplot(1,3,ii+1)
    plt.plot(t_eval[:-1], error_dmd_mean[ii,:], linewidth=lw, label='DMD')
    plt.fill_between(t_eval[:-1], error_dmd_mean[ii,:] - error_dmd_std[ii,:], error_dmd_mean[ii,:] + error_dmd_std[ii,:], alpha=0.2)
    plt.plot(t_eval[:-1], error_edmd_mean[ii, :], linewidth=lw, label='EDMD')
    plt.fill_between(t_eval[:-1], error_edmd_mean[ii, :] - error_edmd_std[ii, :],error_edmd_mean[ii, :] + error_edmd_std[ii, :], alpha=0.2)
    plt.plot(t_eval[:-1], error_bedmd_mean[ii, :], linewidth=lw, label='bEDMD')
    plt.fill_between(t_eval[:-1], error_bedmd_mean[ii, :] - error_bedmd_std[ii, :],error_bedmd_mean[ii, :] + error_bedmd_std[ii, :], alpha=0.2)
    ylim = max(max(np.abs(error_bedmd_mean[ii, :] - error_bedmd_std[ii, :])), max(np.abs(error_bedmd_mean[ii, :] + error_bedmd_std[ii, :])))
    plt.ylim([-ylim * y_lim_gain, ylim * y_lim_gain])
    plt.xlabel('$t$ (sec)', fontsize=fs)
    plt.ylabel(ylabels[ii], fontsize=fs)

plt.legend(loc='upper left', fontsize=fs-4)
suptitle = plt.suptitle('Open loop prediction error of DMD, EDMD and bilinear EDMD models', y=1.05, fontsize=18)
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
plt.tight_layout()
plt.savefig(folder_plots + 'planar_quad_prediction.pdf', format='pdf', dpi=2400, bbox_extra_artists=(suptitle,), bbox_inches="tight")
plt.show()

```

# Design trajectories based on learned models

We now study the closed loop performance of the control design.

```python
solver_settings = {}
solver_settings['gen_embedded_ctrl'] = False
solver_settings['warm_start'] = True
solver_settings['polish'] = True
solver_settings['polish_refine_iter'] = 3
solver_settings['scaling'] = True
solver_settings['adaptive_rho'] = False
solver_settings['check_termination'] = 25
solver_settings['max_iter'] = 4000
solver_settings['eps_abs'] = 1e-6
solver_settings['eps_rel'] = 1e-6
solver_settings['eps_prim_inf'] = 1e-4
solver_settings['eps_dual_inf'] = 1e-4
solver_settings['linsys_solver'] = 'qdldl'
```

```python
#Closed loop performance evaluation parameters:
traj_length=250
t_eval = dt * np.arange(traj_length+1)                       # Simulation time points, closed loop
Q_mpc = sc.sparse.diags([0,0,0,0,0,0])                       # State penalty matrix, trajectory generation
QN_mpc = sc.sparse.diags([1e5,1e5,1e5,1e5,1e5,1e5])         # Final state penalty matrix, trajectory generation
R_mpc = sc.sparse.eye(m)                                     # Actuation penalty matrix, trajectory generation

ctrl_offset = np.array([[hover_thrust], [hover_thrust]])

# Design trajectory:
x0_cl = np.array([-0.8, 0.1, 0.1, -0.3, -0.2, 0.15])                   # Initial value, closed loop trajectory
set_pt_cl = np.array([1.9, 1.2, 0., 0., 0., 0.])              # Desired final value, closed loop trajectory
xmax = np.array([2, 2, np.pi/3, 2.,2.,2.])                          # State constraints, trajectory generation
xmin = -xmax
term_constraint=False

# Define initial solution for SQP algorithm:
x_init = np.linspace(x0_cl, set_pt_cl, int(traj_length)+1)
u_init = np.zeros((m,traj_length)).T
```

#### Design controllers for learned DMD, EDMD, and bEDMD models

```python
from koopman_core.controllers import MPCController, NonlinearMPCControllerNb, BilinearMPCControllerNb

# Define DMD-based controller:
controller_dmd = MPCController(sys_dmd, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt_cl, terminal_constraint=term_constraint, const_offset=ctrl_offset.squeeze())

# Define EDMD-based controller:
controller_edmd = MPCController(sys_edmd, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt_cl, terminal_constraint=term_constraint, const_offset=ctrl_offset.squeeze())

# Define bEDMD-based controller:
controller_bedmd = BilinearMPCControllerNb(sys_bedmd, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt_cl, solver_settings, terminal_constraint=term_constraint, const_offset=ctrl_offset)
z0_cl = sys_bedmd.basis(x0_cl.reshape((1,-1))).squeeze()
z_init = sys_bedmd.basis(x_init)
controller_bedmd.construct_controller(z_init, u_init)
```

#### Design controller using full knowledge of nonlinear controller

```python
quadrotor_d = PlanarQuadrotorForceInputDiscrete(mass, inertia, prop_arm, g=gravity, dt=dt)
controller_nmpc = NonlinearMPCControllerNb(quadrotor_d, traj_length, dt, umin+hover_thrust, umax+hover_thrust, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt_cl, solver_settings, terminal_constraint=term_constraint)
controller_nmpc.construct_controller(x_init, u_init+hover_thrust)
```

#### Design trajectories with the contructed MPCs

```python
max_iter = 100

controller_dmd.eval(x0_cl, 0)
xr_dmd = controller_dmd.parse_result()
ur_dmd = controller_dmd.get_control_prediction() + hover_thrust

controller_edmd.eval(x0_cl, 0)
xr_edmd = sys_edmd.C@controller_edmd.parse_result()
ur_edmd = controller_edmd.get_control_prediction() + hover_thrust

controller_bedmd.solve_to_convergence(z0_cl, 0., z_init, u_init, max_iter=max_iter)
xr_bedmd = sys_bedmd.C@controller_bedmd.get_state_prediction().T
ur_bedmd = controller_bedmd.get_control_prediction().T + hover_thrust
```

    /Users/carlaxelfolkestad/Coding_projects/koopman_learning_and_control/koopman_core/controllers/bilinear_mpc_controller_numba.py:14: NumbaPerformanceWarning: np.dot() is faster on contiguous arrays, called on (array(float64, 1d, C), array(float64, 2d, A))
      r_vec[i, :] = np.dot(z_init[i, :], A_reshaped[i*nx:(i+1)*nx, :])
    /Users/carlaxelfolkestad/Coding_projects/koopman_learning_and_control/koopman_core/controllers/bilinear_mpc_controller_numba.py:9: NumbaPerformanceWarning: np.dot() is faster on contiguous arrays, called on (array(float64, 2d, A), array(float64, 2d, F))
      B_lst_flat = (np.dot(z_init[:-1, :], B_arr)).flatten()
    /Users/carlaxelfolkestad/Coding_projects/koopman_learning_and_control/koopman_core/controllers/nonlinear_mpc_controller_numba.py:225: RuntimeWarning: divide by zero encountered in double_scalars
      while (iter == 0 or np.linalg.norm(u_prev - self.cur_u) / np.linalg.norm(u_prev) > eps) and iter < max_iter:

```python
controller_nmpc.solve_to_convergence(x0_cl, 0., x_init, u_init + ctrl_offset.reshape(1,-1), max_iter=max_iter)
xr_nmpc = controller_nmpc.get_state_prediction().T
ur_nmpc = controller_nmpc.get_control_prediction().T
```

#### Simulate designed trajectories open loop

```python
ol_controller_dmd = OpenLoopController(quadrotor, ur_dmd.T, t_eval[:-1])
xs_dmd, us_dmd = quadrotor.simulate(x0_cl, ol_controller_dmd, t_eval)
xs_dmd, us_dmd = xs_dmd.T, us_dmd.T

ol_controller_edmd = OpenLoopController(quadrotor, ur_edmd.T, t_eval[:-1])
xs_edmd, us_edmd = quadrotor.simulate(x0_cl, ol_controller_edmd, t_eval)
xs_edmd, us_edmd = xs_edmd.T, us_edmd.T

ol_controller_bedmd = OpenLoopController(quadrotor, ur_bedmd.T, t_eval[:-1])
xs_bedmd, us_bedmd = quadrotor.simulate(x0_cl, ol_controller_bedmd, t_eval)
xs_bedmd, us_bedmd = xs_bedmd.T, us_bedmd.T

ol_controller_nmpc = OpenLoopController(quadrotor, ur_nmpc.T, t_eval[:-1])
xs_nmpc, us_nmpc = quadrotor.simulate(x0_cl, ol_controller_nmpc, t_eval)
xs_nmpc, us_nmpc = xs_nmpc.T, us_nmpc.T
```

##### Compare performance

```python
import matplotlib.pyplot as plt
from matplotlib.ticker import MaxNLocator

plot_inds = [0, 1, 2, 3, 4, 5, 0, 1]
subplot_inds = [1, 2, 3, 5, 6, 7, 4, 8]
labels = ['$y$ (m)', '$z$ (m)', '$\\theta$ (rad)', '$\\dot{y}$ (m/s)','$\\dot{z}$ (m/s)', '$\\dot{\\theta}$', '$T_1$ (N)','$T_2$ (N)']
titles = ['y-coordinates', 'z-coordinates', '$\\theta$-coordinates', 'Control inputs']
colors = ['tab:blue', 'tab:orange', 'tab:brown', 'tab:cyan']

plt.figure(figsize=(12,4))
#plt.suptitle('Trajectory designed with model predictive controllers\nsolid lines - designed trajectory | dashed lines - open loop simulated trajectory | black dotted lines - state/actuation bounds')
for ii in range(8):
    ind = plot_inds[ii]
    if ii < 6:
        ax = plt.subplot(2,4,subplot_inds[ii])
        plt.plot(t_eval, xr_dmd[ind,:], colors[0], label='DMD MPC')
        plt.plot(t_eval, xr_edmd[ind, :], colors[1], label='EDMD MPC')
        plt.plot(t_eval, xr_bedmd[ind, :], colors[2], label='K-MPC')
        plt.plot(t_eval, xr_nmpc[ind,:], colors[3], label='NMPC')

        plt.plot(t_eval, xs_dmd[ind,:], '--', color=colors[0], linewidth=1)
        plt.plot(t_eval, xs_edmd[ind, :], '--', color=colors[1], linewidth=1)
        plt.plot(t_eval, xs_bedmd[ind, :], '--', color=colors[2], linewidth=1)
        plt.plot(t_eval, xs_nmpc[ind,:], '--', color=colors[3], linewidth=1)

        plt.scatter(t_eval[0], x0_cl[ind], color='g')
        plt.scatter(t_eval[-1], set_pt_cl[ind], color='r')
        plt.ylabel(labels[ind])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))
        if ii >= 3:
            plt.plot([0, t_eval[-1]], [xmax[ind], xmax[ind]], ':k')
            plt.plot([0, t_eval[-1]], [xmin[ind], xmin[ind]], ':k')
            #plt.ylim(xmin[ind]-0.1,xmax[ind]+0.1)
        if subplot_inds[ii]==1:
            plt.legend(loc='upper left', frameon=False)
    elif ii < 8:
        ax = plt.subplot(2,4,subplot_inds[ii])
        plt.plot(t_eval[:-1],ur_dmd[ind,:], color=colors[0], label='DMD MPC')
        plt.plot(t_eval[:-1], ur_edmd[ind, :], color=colors[1], label='EDMD MPC')
        plt.plot(t_eval[:-1], ur_bedmd[ind, :], color=colors[2], label='K-NMPC')
        plt.plot(t_eval[:-1],ur_nmpc[ind,:], color=colors[3], label='NMPC')
        plt.plot([0, t_eval[-1]], [umax[ind]+hover_thrust, umax[ind]+hover_thrust], ':k')
        plt.plot([0, t_eval[-1]], [umin[ind]+hover_thrust, umin[ind]+hover_thrust], ':k')
        plt.ylabel(labels[ii])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))

    if subplot_inds[ii] > 4:
        plt.xlabel('Time (sec)')
    else:
        plt.title(titles[subplot_inds[ii]-1])

if save_figures:
    matplotlib.rcParams['pdf.fonttype'] = 42
    matplotlib.rcParams['ps.fonttype'] = 42
    plt.tight_layout()
    plt.savefig(dropbox_folder + 'planar_quad_trajectory.pdf', format='pdf', dpi=2400)
plt.show()

cost_ref_dmd = (xr_dmd[:,-1]-set_pt_cl).T@QN_mpc@(xr_dmd[:,-1]-set_pt_cl) + np.sum(np.diag(ur_dmd.T@R_mpc@ur_dmd))
cost_ref_edmd = (xr_edmd[:,-1]-set_pt_cl).T@QN_mpc@(xr_edmd[:,-1]-set_pt_cl) + np.sum(np.diag(ur_edmd.T@R_mpc@ur_edmd))
cost_ref_bedmd = (xr_bedmd[:,-1]-set_pt_cl).T@QN_mpc@(xr_bedmd[:,-1]-set_pt_cl) + np.sum(np.diag(ur_bedmd.T@R_mpc@ur_bedmd))
cost_ref_nmpc = (xr_nmpc[:,-1]-set_pt_cl).T@QN_mpc@(xr_nmpc[:,-1]-set_pt_cl) + np.sum(np.diag(ur_nmpc.T@R_mpc@ur_nmpc))

dist_ol_dmd = np.linalg.norm(xs_dmd[:,-1] - set_pt_cl)
dist_ol_edmd = np.linalg.norm(xs_edmd[:,-1] - set_pt_cl)
dist_ol_bedmd = np.linalg.norm(xs_bedmd[:,-1] - set_pt_cl)
dist_ol_nmpc = np.linalg.norm(xs_nmpc[:,-1] - set_pt_cl)

print('Solution statistics:\n')
print(tabulate([['DMD MPC', "{:.4f}".format(cost_ref_dmd/cost_ref_nmpc), "{:.4f}".format(dist_ol_dmd), '-','-',sum(controller_dmd.comp_time)],
                ['EDMD MPC', "{:.4f}".format(cost_ref_edmd/cost_ref_nmpc), "{:.4f}".format(dist_ol_edmd),'-','-',sum(controller_edmd.comp_time)],
                ['bEDMD MPC', "{:.4f}".format(cost_ref_bedmd/cost_ref_nmpc), "{:.4f}".format(dist_ol_bedmd), len(controller_bedmd.x_iter), "{:.4f}".format(np.mean(controller_bedmd.comp_time)), sum(controller_bedmd.comp_time)],
                ['NMPC (benchmark)', 1, "{:.4f}".format(dist_ol_nmpc), len(controller_nmpc.x_iter), "{:.4f}".format(np.mean(controller_nmpc.comp_time)), sum(controller_nmpc.comp_time)]],
               headers=['Normalized cost,\ndesigned trajectory', 'Realized terminal,\nerror', '# of SQP\niterations','Mean comp. time\nper iteration (secs)', 'Total comp.\ntime (secs)']))
```

#### Study evolution of the solution after each iteration of the SQP-algorithm

```python
n_iter = min(len(controller_nmpc.x_iter),len(controller_bedmd.x_iter))

# Calculate cost after each iteration:
iter_cost_bedmd, iter_cost_nmpc = [], []

ol_controller_init = OpenLoopController(quadrotor, u_init, t_eval[:-1])
xs_init, _ = quadrotor.simulate(x0_cl, ol_controller_init, t_eval)
xs_init, us_init = xs_init.T, u_init.T+hover_thrust
init_cost = (xs_init[:,-1]-set_pt_cl).T@QN_mpc@(xs_init[:,-1]-set_pt_cl) + np.sum(np.diag(us_init.T@R_mpc@us_init))
iter_cost_bedmd = [init_cost]
iter_cost_nmpc = [init_cost]
iter_norm_dist_bedmd = [np.linalg.norm(xs_init[:,-1]-set_pt_cl)]
iter_norm_dist_nmpc = [np.linalg.norm(xs_init[:,-1]-set_pt_cl)]

for ii in range(len(controller_bedmd.x_iter)):
    ur_bedmd_iter = controller_bedmd.u_iter[ii].T+hover_thrust
    ol_controller_bedmd_iter = OpenLoopController(quadrotor, ur_bedmd_iter, t_eval[:-1])
    xs_bedmd_iter, _ = quadrotor.simulate(x0_cl, ol_controller_bedmd_iter, t_eval)
    xs_bedmd_iter, us_bedmd_iter = xs_bedmd_iter.T, ur_bedmd_iter.T
    iter_cost_bedmd.append((xs_bedmd_iter[:,-1]-set_pt_cl).T@QN_mpc@(xs_bedmd_iter[:,-1]-set_pt_cl) + np.sum(np.diag(us_bedmd_iter.T@R_mpc@us_bedmd_iter)))
    iter_norm_dist_bedmd.append(np.linalg.norm(xs_bedmd_iter[:,-1]-set_pt_cl))

for ii in range(len(controller_nmpc.x_iter)):
    ur_nmpc_iter = controller_nmpc.u_iter[ii].T
    ol_controller_nmpc_iter = OpenLoopController(quadrotor, ur_nmpc_iter, t_eval[:-1])
    xs_nmpc_iter, _ = quadrotor.simulate(x0_cl, ol_controller_nmpc_iter, t_eval)
    xs_nmpc_iter, us_nmpc_iter = xs_nmpc_iter.T, ur_nmpc_iter.T
    iter_cost_nmpc.append((xs_nmpc_iter[:,-1]-set_pt_cl).T@QN_mpc@(xs_nmpc_iter[:,-1]-set_pt_cl) + np.sum(np.diag(us_nmpc_iter.T@R_mpc@us_nmpc_iter)))
    iter_norm_dist_nmpc.append(np.linalg.norm(xs_nmpc_iter[:,-1]-set_pt_cl))
```

```python
plt.figure(figsize=(6,4))
plt.suptitle('Control solution after each iteration of the SQP-algorithm for NMPC and K-NMPC')
plt.subplot(2,1,1)
plt.plot(np.arange(n_iter), iter_cost_bedmd[:n_iter]/iter_cost_nmpc[-1], color=colors[2], label='K-NMPC')
plt.plot(np.arange(n_iter), iter_cost_nmpc[:n_iter]/iter_cost_nmpc[-1], color=colors[3], label='NMPC')
plt.ylim(0,5)
plt.title('Control effort')
plt.ylabel('$||u||$')
plt.legend(loc='upper right', frameon=False)
plt.xlabel('SQP iteration')

plt.subplot(2,1,2)
plt.plot(np.arange(n_iter), iter_norm_dist_bedmd[:n_iter], color=colors[2], label=labels[2])
plt.plot(np.arange(n_iter), iter_norm_dist_nmpc[:n_iter], color=colors[3], label=labels[3])
plt.ylim(0,5)
plt.title('Realized terminal distance from setpoint')
plt.ylabel('$||x_N - x_d||$')
plt.xlabel('SQP iteration')

if save_figures:
    matplotlib.rcParams['pdf.fonttype'] = 42
    matplotlib.rcParams['ps.fonttype'] = 42
    plt.tight_layout()
    plt.savefig(dropbox_folder + 'planar_quad_sqp_iterations.pdf', format='pdf', dpi=2400)

plt.show()

print('Solution statistics\n')
print(tabulate([['Nonlinear MPC', len(controller_nmpc.x_iter), np.mean(controller_nmpc.comp_time), np.std(controller_nmpc.comp_time), sum(controller_nmpc.comp_time)],
                ['Koopman bilinear MPC', len(controller_bedmd.x_iter), np.mean(controller_bedmd.comp_time), np.std(controller_bedmd.comp_time), sum(controller_bedmd.comp_time)]],
               headers=['Number of SQP\niterations','Mean comp. time per\niteration (secs)', 'Std comp. time per\niteration (secs)', 'Total comp.\ntime (secs)']))
```

# Evaluate performance of controllers for closed-loop control

#### Design finite horizon controllers

```python
from koopman_core.controllers import PerturbedController

Q_mpc_cl = sc.sparse.diags([1e3, 1e3, 1e3, 1e2, 1e2, 1e2])
QN_mpc_cl = Q_mpc_cl
R_mpc_cl = sc.sparse.eye(m)
traj_duration = 0.5
N_cl = int(traj_duration/dt)
t_eval_cl=np.arange(250)*dt

solver_settings_cl = solver_settings
```

```python
controller_dmd_cl = MPCController(sys_dmd, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt_cl, add_slack=True, const_offset=ctrl_offset.squeeze())
controller_dmd_cl = PerturbedController(sys_dmd,controller_dmd_cl,0.,const_offset=hover_thrust, umin=umin, umax=umax)

controller_edmd_cl = MPCController(sys_edmd, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt_cl, add_slack=True, const_offset=ctrl_offset.squeeze())
controller_edmd_cl = PerturbedController(sys_edmd,controller_edmd_cl,0.,const_offset=hover_thrust, umin=umin, umax=umax)

controller_bedmd_cl = BilinearMPCControllerNb(sys_bedmd, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt_cl, solver_settings_cl, add_slack=True, const_offset=ctrl_offset)
controller_bedmd_cl.construct_controller(controller_bedmd.cur_z[:N_cl+1,:], controller_bedmd.cur_u[:N_cl,:])
controller_bedmd_cl.solve_to_convergence(z0_cl, 0., controller_bedmd.cur_z[:N_cl+1,:], controller_bedmd.cur_u[:N_cl,:], max_iter=max_iter)
_ = controller_bedmd_cl.eval(x0_cl, 0.)
controller_bedmd_cl = PerturbedController(sys_bedmd,controller_bedmd_cl,0.,const_offset=hover_thrust, umin=umin, umax=umax)

controller_nmpc_cl = NonlinearMPCControllerNb(quadrotor_d, N_cl, dt, umin+hover_thrust, umax+hover_thrust, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt_cl, solver_settings_cl, add_slack=True)
controller_nmpc_cl.construct_controller(controller_nmpc.cur_z[:N_cl+1,:], controller_nmpc.cur_u[:N_cl,:])
controller_nmpc_cl.solve_to_convergence(x0_cl, 0., controller_nmpc.cur_z[:N_cl+1,:], controller_nmpc.cur_u[:N_cl,:], max_iter=max_iter)
_ = controller_nmpc_cl.eval(x0_cl, 0.)
```

```python
controller_bedmd_cl.nom_controller.comp_time, controller_bedmd_cl.nom_controller.prep_time, controller_bedmd_cl.nom_controller.qp_time,  = [], [], []
controller_nmpc_cl.comp_time, controller_nmpc_cl.prep_time, controller_nmpc_cl.qp_time,  = [], [], []
```

```python
solver_settings_cl['polish'] = False
solver_settings_cl['check_termination'] = 10
solver_settings_cl['max_iter'] = 10
solver_settings_cl['eps_abs'] = 1e-2
solver_settings_cl['eps_rel'] = 1e-2
solver_settings['eps_prim_inf'] = 1e-3
solver_settings['eps_dual_inf'] = 1e-3

controller_nmpc_cl.update_solver_settings(solver_settings_cl)
controller_bedmd_cl.nom_controller.update_solver_settings(solver_settings_cl)
```

#### Simulate designed trajectories closed-loop

```python
xs_dmd_cl, us_dmd_cl = quadrotor.simulate(x0_cl, controller_dmd_cl, t_eval_cl)
xs_dmd_cl, us_dmd_cl = xs_dmd_cl.T, us_dmd_cl.T

xs_edmd_cl, us_edmd_cl = quadrotor.simulate(x0_cl, controller_edmd_cl, t_eval_cl)
xs_edmd_cl, us_edmd_cl = xs_edmd_cl.T, us_edmd_cl.T

controller_bedmd_cl.comp_time = []
xs_bedmd_cl, us_bedmd_cl = quadrotor.simulate(x0_cl, controller_bedmd_cl, t_eval_cl)
xs_bedmd_cl, us_bedmd_cl = xs_bedmd_cl.T, us_bedmd_cl.T

controller_nmpc_cl.comp_time = []
xs_nmpc_cl, us_nmpc_cl = quadrotor.simulate(x0_cl, controller_nmpc_cl, t_eval_cl)
xs_nmpc_cl, us_nmpc_cl = xs_nmpc_cl.T, us_nmpc_cl.T
```

#### Plot/analyze the results

```python
plot_inds = [0, 1, 2, 0, 1]
subplot_inds = [1, 2, 3, 4, 8]

plt.figure(figsize=(12,2.5))
for ii in range(5):
    ind = plot_inds[ii]
    if ii < 3:
        ax = plt.subplot(1,4,subplot_inds[ii])
        plt.plot(t_eval_cl, xs_dmd_cl[ind,:], colors[0], label='DMD MPC')
        plt.plot(t_eval_cl, xs_edmd_cl[ind, :], colors[1], label='EDMD MPC')
        plt.plot(t_eval_cl, xs_bedmd_cl[ind, :], colors[2], label='K-NMPC')
        plt.plot(t_eval_cl, xs_nmpc_cl[ind,:], colors[3], label='NMPC')

        plt.scatter(t_eval_cl[0], x0_cl[ind], color='g')
        plt.scatter(t_eval_cl[-1], set_pt_cl[ind], color='r')
        plt.ylabel(labels[ind])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))
        plt.title(titles[subplot_inds[ii]-1])
        plt.xlabel('Time (sec)')
        if subplot_inds[ii]==1:
            plt.legend(loc='upper left', frameon=False)
    else:
        bx = plt.subplot(2,4,subplot_inds[ii])
        plt.plot(t_eval_cl[:-1],us_dmd_cl[ind,:], color=colors[0], label='DMD MPC')
        plt.plot(t_eval_cl[:-1], us_edmd_cl[ind, :], color=colors[1], label='EDMD MPC')
        plt.plot(t_eval_cl[:-1], us_bedmd_cl[ind, :], color=colors[2], label='K-NMPC')
        plt.plot(t_eval_cl[:-1],us_nmpc_cl[ind,:], color=colors[3], label='NMPC')
        plt.plot([0, t_eval_cl[-1]], [umax[ind]+hover_thrust, umax[ind]+hover_thrust], ':k')
        plt.plot([0, t_eval_cl[-1]], [umin[ind]+hover_thrust, umin[ind]+hover_thrust], ':k')
        plt.ylabel(labels[ii+3])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))
        if subplot_inds[ii] == 4:
            plt.title('Control inputs')
        else:
            plt.xlabel('Time (sec)')
if save_figures:
    matplotlib.rcParams['pdf.fonttype'] = 42
    matplotlib.rcParams['ps.fonttype'] = 42
    plt.tight_layout()
    plt.savefig(dropbox_folder + 'planar_quad_closed_loop.pdf', format='pdf', dpi=2400)
plt.show()

from matplotlib.offsetbox import AnnotationBbox, OffsetImage
from scipy import ndimage

draw_inds = np.arange(0,t_eval_cl.size)[::50]

plt.figure(figsize=(12,2))
ax = plt.subplot(1,1,1, frameon=False)
plt.plot(xs_bedmd_cl[0,:], xs_bedmd_cl[1,:], color=colors[2], label='Koopman NMPC closed loop trajectory with quadrotor orientation sampled at 2 hz')
plt.xlabel('y (m)')
plt.ylabel('z (m)')
plt.legend(loc='upper left',frameon=False)
for ii in draw_inds:
    im_quad = plt.imread('working_files/figures/quad_figure_rb.png')
    im_quad = ndimage.rotate(im_quad, xs_bedmd_cl[2,ii]*180)
    imagebox_quad = OffsetImage(im_quad, zoom=.11)
    ab = AnnotationBbox(imagebox_quad, [xs_bedmd_cl[0,ii], xs_bedmd_cl[1,ii]], frameon=False)
    ax.add_artist(ab)

if save_figures:
    matplotlib.rcParams['pdf.fonttype'] = 42
    matplotlib.rcParams['ps.fonttype'] = 42
    plt.tight_layout()
    plt.savefig(dropbox_folder + 'planar_quad_closed_loop_2.pdf', format='pdf', dpi=2400)
plt.show()

cost_cl_dmd = np.sum(np.diag((xs_dmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)).T@Q_mpc_cl@(xs_dmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)))) + (xs_dmd_cl[:,-1]-set_pt_cl).T@QN_mpc_cl@(xs_dmd_cl[:,-1]-set_pt_cl) + np.sum(np.diag(us_dmd_cl.T@R_mpc_cl@us_dmd_cl))
cost_cl_edmd = np.sum(np.diag((xs_edmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)).T@Q_mpc_cl@(xs_edmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)))) + (xs_edmd_cl[:,-1]-set_pt_cl).T@QN_mpc_cl@(xs_edmd_cl[:,-1]-set_pt_cl) + np.sum(np.diag(us_edmd_cl.T@R_mpc_cl@us_edmd_cl))
cost_cl_bedmd = np.sum(np.diag((xs_bedmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)).T@Q_mpc_cl@(xs_bedmd_cl[:,:-1]-set_pt_cl.reshape(-1,1)))) + (xs_bedmd_cl[:,-1]-set_pt_cl).T@QN_mpc_cl@(xs_bedmd_cl[:,-1]-set_pt_cl) + np.sum(np.diag(us_bedmd_cl.T@R_mpc_cl@us_bedmd_cl))
cost_cl_nmpc = np.sum(np.diag((xs_nmpc_cl[:,:-1]-set_pt_cl.reshape(-1,1)).T@Q_mpc_cl@(xs_nmpc_cl[:,:-1]-set_pt_cl.reshape(-1,1)))) + (xs_nmpc_cl[:,-1]-set_pt_cl).T@QN_mpc_cl@(xs_nmpc_cl[:,-1]-set_pt_cl) + np.sum(np.diag(us_nmpc_cl.T@R_mpc_cl@us_nmpc_cl))

print('Solution statistics:\n')
print(tabulate([['DMD MPC', "{:.4f}".format(cost_cl_dmd/cost_cl_nmpc), np.mean(controller_dmd_cl.nom_controller.comp_time), np.std(controller_dmd_cl.nom_controller.comp_time)],
                ['EDMD MPC', "{:.4f}".format(cost_cl_edmd/cost_cl_nmpc),np.mean(controller_edmd_cl.nom_controller.comp_time), np.std(controller_edmd_cl.nom_controller.comp_time)],
                ['bEDMD MPC', "{:.4f}".format(cost_cl_bedmd/cost_cl_nmpc), np.mean(controller_bedmd_cl.nom_controller.comp_time), np.std(controller_bedmd_cl.nom_controller.comp_time)],
                ['NMPC (benchmark, known model)',1, np.mean(controller_nmpc_cl.comp_time), np.std(controller_nmpc_cl.comp_time)]],
               headers=['Normalized cost,\nrealized trajectory', 'Mean comp. time (secs)', 'std comp. time (secs)']))
```

```python
print('\nSolution time profiling:\n')
print(tabulate([['NMPC', np.mean(controller_nmpc_cl.comp_time), np.mean(controller_nmpc_cl.prep_time), np.mean(controller_nmpc_cl.qp_time)],
                ['Koopman bilinear MPC', np.mean(controller_bedmd_cl.nom_controller.comp_time), np.mean(controller_bedmd_cl.nom_controller.prep_time), np.mean(controller_bedmd_cl.nom_controller.qp_time)]],
               headers=['Total comp time', 'setup time', 'qp solve time' ]))
```

    Solution time profiling:

                            Total comp time    setup time    qp solve time
    --------------------  -----------------  ------------  ---------------
    NMPC                         0.00412956    0.00358561      0.000543945
    Koopman bilinear MPC         0.00706125    0.00222308      0.00483817

```python

```


# MADE Demo

Autoregressive density estimation for MNIST using masked FFNN (MADE).

```python
import sys
import argparse
import pprint
import pathlib
import numpy as np
import torch
import json
from tqdm import tqdm
from collections import OrderedDict, defaultdict

from torch.distributions import Bernoulli

import dgm
from dgm.conditional import MADEConditioner
from dgm.likelihood import FullyFactorizedLikelihood, AutoregressiveLikelihood
from dgm.opt_utils import get_optimizer, ReduceLROnPlateau
from utils import load_mnist, Batcher

```

Organise some hyperparameters

```python
from collections import namedtuple

Config = namedtuple(
    "Config",
    ['seed', 'device', 'batch_size', 'data_dir',
     'height', 'width', 'input_dropout',
    'hidden_sizes', 'num_masks', 'resample_mask_every', 'll_samples',
    'epochs', 'opt', 'lr', 'momentum', 'l2_weight', 'patience', 'early_stopping'])

args = Config(
    seed=42,
    device='cuda:0',
    batch_size=128,
    data_dir='./data',
    height=28,
    width=28,
    input_dropout=0.,
    hidden_sizes=[8000, 8000],
    num_masks=1,
    resample_mask_every=20,
    ll_samples=10,
    epochs=200,
    opt='adam',
    lr=1e-4,
    momentum=0.,
    l2_weight=1e-4,
    patience=10,
    early_stopping=10,
)
```

We like reproducibility

```python
np.random.seed(args.seed)
torch.manual_seed(args.seed)
torch.cuda.manual_seed_all(args.seed)
```

Load data

```python
train_loader, valid_loader, test_loader = load_mnist(
    args.batch_size,
    save_to='{}/std/{}x{}'.format(args.data_dir, args.height, args.width),
    height=args.height,
    width=args.width)
```

Build a model

\begin{align}
    P(x) &= \prod_{i=1}^{|x|} \text{Bern}(x_i|\underbrace{f(x_{<i})}_{\text{MADE}})
\end{align}

For that we need a *conditioner*
* the part that maps prefixes into Bernoulli parameters

and an *autoregressive likelihood*
* the part that combines the Bernoulli factors

```python
x_size = args.width * args.height
device = torch.device(args.device)
made = MADEConditioner(
    input_size=x_size,  # our only input to the MADE layer is the observation
    output_size=x_size * 1,  # number of parameters to predict
    context_size=0,  # we do not have any additional inputs
    hidden_sizes=args.hidden_sizes,
    num_masks=args.num_masks
)
model = AutoregressiveLikelihood(
    event_size=x_size,  # size of observation
    dist_type=Bernoulli,
    conditioner=made
).to(device)

print("\n# Architecture")
print(model)
```

    # Architecture
    AutoregressiveLikelihood(
      (conditioner): MADEConditioner(
        (_made): MADE(
          (hidden_activation): ReLU()
          (net): Sequential(
            (0): MaskedLinear(in_features=784, out_features=8000, bias=True)
            (1): ReLU()
            (2): MaskedLinear(in_features=8000, out_features=8000, bias=True)
            (3): ReLU()
            (4): MaskedLinear(in_features=8000, out_features=784, bias=True)
          )
        )
      )
    )

Let's configure the optimiser

```python
print("\n# Optimizer")
opt = get_optimizer(args.opt, model.parameters(), args.lr, args.l2_weight, args.momentum)
scheduler = ReduceLROnPlateau(
    opt,
    factor=0.5,
    patience=args.patience,
    early_stopping=args.early_stopping,
    mode='min', threshold_mode='abs')
print(opt)
```

    # Optimizer
    Adam (
    Parameter Group 0
        amsgrad: False
        betas: (0.9, 0.999)
        eps: 1e-08
        lr: 0.0001
        weight_decay: 0.0001
    )

Some helper code for batching MNIST digits

```python
def get_batcher(data_loader):
    batcher = Batcher(
        data_loader,
        height=args.height,
        width=args.width,
        device=torch.device(args.device),
        binarize=True,
        num_classes=10,
        onehot=True
    )
    return batcher
```

Helper code for validating a model

```python
def validate(batcher, args, model,
    optimizer=None, scheduler=None, writer=None, name='dev'):
    """
    :return: stop flag, dict
        NLL can be found in the dict
    """

    if args.num_masks == 1:
        resample_mask = False
        num_samples = 1
    else:  # ensemble
        resample_mask = True
        num_samples = args.ll_samples

    with torch.no_grad():
        model.eval()
        print_ = defaultdict(list)
        nb_instances = 0.
        for x_mb, y_mb in batcher:
            # [B, H*W]
            x_mb = x_mb.reshape(-1, args.height * args.width)
            # [B, 10]
            made_inputs = x_mb
            # [B, H*W]
            p_x = model(
                inputs=None,
                history=x_mb,
                num_samples=num_samples, resample_mask=resample_mask
            )
            # [B]
            nll = -p_x.log_prob(x_mb).sum(-1)
            # accumulate metrics
            print_['NLL'].append(nll.sum().item())
            nb_instances += x_mb.size(0)

        return_dict = {k: np.sum(v) / nb_instances for k, v in print_.items()}
        if writer:
            writer.add_scalar('%s/NLL' % name, return_dict['NLL'])

        stop = False
        if scheduler is not None:
            stop = scheduler.step(return_dict['NLL'])

        return stop, return_dict
```

Main training loop

```python
print("\n# Training")

#from tensorboardX import SummaryWriter
#writer = SummaryWriter(args.logdir)
writer = None
step = 1

for epoch in range(args.epochs):

    iterator = tqdm(get_batcher(train_loader))

    for x_mb, y_mb in iterator:
        # [B, H*W]
        x_mb = x_mb.reshape(-1, args.height * args.width)
        # [B, 10]
        context = None
        model.train()
        opt.zero_grad()

        if args.num_masks == 1:
            resample_mask = False
        else:  # training with variable masks
            resample_mask = args.resample_mask_every > 0 and step % args.resample_mask_every == 0

        # [B, H*W]
        noisy_x = torch.where(
            torch.rand_like(x_mb) > args.input_dropout, x_mb, torch.zeros_like(x_mb)
        )
        p_x = model(
            inputs=context,
            history=noisy_x,
            resample_mask=resample_mask
        )
        # [B, H*W]
        ll_mb = p_x.log_prob(x_mb)
        # [B]
        ll = ll_mb.sum(-1)

        loss = -(ll).mean()
        loss.backward()
        opt.step()

        display = OrderedDict()
        display['0s'] = '{:.2f}'.format((x_mb == 0).float().mean().item())
        display['1s'] = '{:.2f}'.format((x_mb == 1).float().mean().item())
        display['NLL'] =  '{:.2f}'.format(-ll.mean().item())

        if writer:
            writer.add_scalar('training/LL', ll)
            #writer.add_image('training/posterior/sample', z.mean(0).reshape(1,1,-1) * 255)

        iterator.set_postfix(display, refresh=False)
        step += 1

    stop, dict_valid = validate(get_batcher(valid_loader), args, model, opt, scheduler,
        writer=writer, name="dev")

    if stop:
        print('Early stopping at epoch {:3}/{}'.format(epoch + 1, args.epochs))
        break

    print('Epoch {:3}/{} -- '.format(epoch + 1, args.epochs) + \
          ', '.join(['{}: {:4.2f}'.format(k, v) for k, v in sorted(dict_valid.items())]))
```

      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=540.60]

    # Training

    100%|██████████| 430/430 [00:27<00:00, 16.01it/s, 0s=0.87, 1s=0.13, NLL=183.00]
      0%|          | 2/430 [00:00<00:27, 15.56it/s, 0s=0.86, 1s=0.14, NLL=186.59]

    Epoch   1/200 -- NLL: 178.83

    100%|██████████| 430/430 [00:27<00:00, 15.48it/s, 0s=0.86, 1s=0.14, NLL=150.84]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=146.11]

    Epoch   2/200 -- NLL: 146.80

    100%|██████████| 430/430 [00:28<00:00, 15.45it/s, 0s=0.87, 1s=0.13, NLL=127.02]
      0%|          | 2/430 [00:00<00:28, 15.00it/s, 0s=0.87, 1s=0.13, NLL=129.89]

    Epoch   3/200 -- NLL: 129.88

    100%|██████████| 430/430 [00:28<00:00, 15.39it/s, 0s=0.86, 1s=0.14, NLL=130.06]
      0%|          | 2/430 [00:00<00:28, 15.01it/s, 0s=0.87, 1s=0.13, NLL=122.98]

    Epoch   4/200 -- NLL: 121.00

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.86, 1s=0.14, NLL=120.82]
      0%|          | 2/430 [00:00<00:28, 14.96it/s, 0s=0.87, 1s=0.13, NLL=111.05]

    Epoch   5/200 -- NLL: 115.91

    100%|██████████| 430/430 [00:28<00:00, 15.38it/s, 0s=0.87, 1s=0.13, NLL=116.86]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.88, 1s=0.12, NLL=105.97]

    Epoch   6/200 -- NLL: 112.54

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=104.61]
      0%|          | 2/430 [00:00<00:28, 14.97it/s, 0s=0.87, 1s=0.13, NLL=108.41]

    Epoch   7/200 -- NLL: 110.12

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=110.64]
      0%|          | 2/430 [00:00<00:29, 14.51it/s, 0s=0.87, 1s=0.13, NLL=110.41]

    Epoch   8/200 -- NLL: 108.26

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.87, 1s=0.13, NLL=103.89]
      0%|          | 2/430 [00:00<00:28, 14.93it/s, 0s=0.86, 1s=0.14, NLL=111.29]

    Epoch   9/200 -- NLL: 106.60

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=102.59]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.86, 1s=0.14, NLL=109.08]

    Epoch  10/200 -- NLL: 105.41

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.86, 1s=0.14, NLL=103.54]
      0%|          | 2/430 [00:00<00:28, 14.94it/s, 0s=0.87, 1s=0.13, NLL=100.54]

    Epoch  11/200 -- NLL: 104.03

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=102.78]
      0%|          | 2/430 [00:00<00:28, 14.95it/s, 0s=0.87, 1s=0.13, NLL=101.37]

    Epoch  12/200 -- NLL: 102.83

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=98.70]
      0%|          | 2/430 [00:00<00:28, 14.99it/s, 0s=0.86, 1s=0.14, NLL=100.94]

    Epoch  13/200 -- NLL: 101.76

    100%|██████████| 430/430 [00:28<00:00, 15.38it/s, 0s=0.87, 1s=0.13, NLL=98.34]
      0%|          | 2/430 [00:00<00:28, 15.03it/s, 0s=0.87, 1s=0.13, NLL=102.14]

    Epoch  14/200 -- NLL: 100.93

    100%|██████████| 430/430 [00:28<00:00, 15.36it/s, 0s=0.88, 1s=0.12, NLL=99.11]
      0%|          | 2/430 [00:00<00:28, 14.95it/s, 0s=0.87, 1s=0.13, NLL=96.83]

    Epoch  15/200 -- NLL: 100.19

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=98.58]
      0%|          | 2/430 [00:00<00:28, 15.01it/s, 0s=0.86, 1s=0.14, NLL=100.46]

    Epoch  16/200 -- NLL: 99.53

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=100.28]
      0%|          | 2/430 [00:00<00:28, 15.01it/s, 0s=0.87, 1s=0.13, NLL=98.57]

    Epoch  17/200 -- NLL: 98.87

    100%|██████████| 430/430 [00:28<00:00, 15.38it/s, 0s=0.87, 1s=0.13, NLL=99.80]
      0%|          | 2/430 [00:00<00:28, 14.91it/s, 0s=0.87, 1s=0.13, NLL=95.93]

    Epoch  18/200 -- NLL: 98.36

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.86, 1s=0.14, NLL=100.65]
      0%|          | 2/430 [00:00<00:28, 15.00it/s, 0s=0.88, 1s=0.12, NLL=94.52]

    Epoch  19/200 -- NLL: 97.65

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.86, 1s=0.14, NLL=96.53]
      0%|          | 2/430 [00:00<00:28, 14.98it/s, 0s=0.88, 1s=0.12, NLL=95.14]

    Epoch  20/200 -- NLL: 97.15

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=95.29]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.87, 1s=0.13, NLL=95.32]

    Epoch  21/200 -- NLL: 96.70

    100%|██████████| 430/430 [00:28<00:00, 15.38it/s, 0s=0.87, 1s=0.13, NLL=93.82]
      0%|          | 2/430 [00:00<00:28, 15.01it/s, 0s=0.86, 1s=0.14, NLL=98.83]

    Epoch  22/200 -- NLL: 96.50

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.86, 1s=0.14, NLL=95.41]
      0%|          | 2/430 [00:00<00:28, 15.05it/s, 0s=0.87, 1s=0.13, NLL=95.56]

    Epoch  23/200 -- NLL: 95.68

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=91.23]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=93.88]

    Epoch  24/200 -- NLL: 95.46

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.87, 1s=0.13, NLL=92.03]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=94.70]

    Epoch  25/200 -- NLL: 94.99

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=91.26]
      0%|          | 2/430 [00:00<00:28, 15.05it/s, 0s=0.87, 1s=0.13, NLL=94.32]

    Epoch  26/200 -- NLL: 94.78

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.87, 1s=0.13, NLL=94.88]
      0%|          | 2/430 [00:00<00:28, 15.23it/s, 0s=0.87, 1s=0.13, NLL=92.48]

    Epoch  27/200 -- NLL: 94.41

    100%|██████████| 430/430 [00:28<00:00, 15.38it/s, 0s=0.86, 1s=0.14, NLL=97.64]
      0%|          | 2/430 [00:00<00:28, 14.95it/s, 0s=0.88, 1s=0.12, NLL=90.83]

    Epoch  28/200 -- NLL: 94.17

    100%|██████████| 430/430 [00:28<00:00, 15.44it/s, 0s=0.87, 1s=0.13, NLL=92.35]
      0%|          | 2/430 [00:00<00:27, 15.30it/s, 0s=0.87, 1s=0.13, NLL=90.41]

    Epoch  29/200 -- NLL: 93.88

    100%|██████████| 430/430 [00:28<00:00, 15.44it/s, 0s=0.87, 1s=0.13, NLL=90.95]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.88, 1s=0.12, NLL=89.75]

    Epoch  30/200 -- NLL: 93.69

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.86, 1s=0.14, NLL=94.78]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=92.13]

    Epoch  31/200 -- NLL: 93.05

    100%|██████████| 430/430 [00:28<00:00, 15.55it/s, 0s=0.87, 1s=0.13, NLL=89.58]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.88, 1s=0.12, NLL=90.83]

    Epoch  32/200 -- NLL: 93.11

    100%|██████████| 430/430 [00:28<00:00, 15.37it/s, 0s=0.87, 1s=0.13, NLL=87.04]
      0%|          | 2/430 [00:00<00:28, 15.09it/s, 0s=0.87, 1s=0.13, NLL=88.42]

    Epoch  33/200 -- NLL: 92.71

    100%|██████████| 430/430 [00:28<00:00, 15.49it/s, 0s=0.86, 1s=0.14, NLL=89.68]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=87.11]

    Epoch  34/200 -- NLL: 92.52

    100%|██████████| 430/430 [00:28<00:00, 15.36it/s, 0s=0.87, 1s=0.13, NLL=90.26]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=88.90]

    Epoch  35/200 -- NLL: 92.36

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=86.81]
      0%|          | 2/430 [00:00<00:28, 15.14it/s, 0s=0.87, 1s=0.13, NLL=88.92]

    Epoch  36/200 -- NLL: 92.06

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.86, 1s=0.14, NLL=89.17]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.86, 1s=0.14, NLL=90.22]

    Epoch  37/200 -- NLL: 91.83

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=88.98]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=89.95]

    Epoch  38/200 -- NLL: 91.65

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.86, 1s=0.14, NLL=92.90]
      0%|          | 2/430 [00:00<00:28, 15.14it/s, 0s=0.87, 1s=0.13, NLL=86.14]

    Epoch  39/200 -- NLL: 91.48

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=88.64]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=88.80]

    Epoch  40/200 -- NLL: 91.10

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=86.71]
      0%|          | 2/430 [00:00<00:28, 15.14it/s, 0s=0.87, 1s=0.13, NLL=85.25]

    Epoch  41/200 -- NLL: 91.02

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=88.53]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=88.62]

    Epoch  42/200 -- NLL: 91.01

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.86, 1s=0.14, NLL=90.27]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=88.74]

    Epoch  43/200 -- NLL: 90.90

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=87.56]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.86, 1s=0.14, NLL=90.56]

    Epoch  44/200 -- NLL: 90.64

    100%|██████████| 430/430 [00:28<00:00, 15.45it/s, 0s=0.87, 1s=0.13, NLL=92.64]
      0%|          | 2/430 [00:00<00:28, 15.16it/s, 0s=0.87, 1s=0.13, NLL=89.84]

    Epoch  45/200 -- NLL: 90.45

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=88.97]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.88, 1s=0.12, NLL=85.55]

    Epoch  46/200 -- NLL: 90.15

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=93.56]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=86.29]

    Epoch  47/200 -- NLL: 90.18

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=90.79]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=88.41]

    Epoch  48/200 -- NLL: 89.92

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=87.27]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=89.30]

    Epoch  49/200 -- NLL: 89.88

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=90.28]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=86.43]

    Epoch  50/200 -- NLL: 89.77

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=90.08]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=85.85]

    Epoch  51/200 -- NLL: 89.75

    100%|██████████| 430/430 [00:28<00:00, 15.44it/s, 0s=0.87, 1s=0.13, NLL=88.34]
      0%|          | 2/430 [00:00<00:28, 15.03it/s, 0s=0.87, 1s=0.13, NLL=88.33]

    Epoch  52/200 -- NLL: 89.43

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.86, 1s=0.14, NLL=90.48]
      0%|          | 2/430 [00:00<00:28, 15.17it/s, 0s=0.87, 1s=0.13, NLL=87.43]

    Epoch  53/200 -- NLL: 89.21

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.86, 1s=0.14, NLL=87.72]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=86.13]

    Epoch  54/200 -- NLL: 89.27

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=88.06]
      0%|          | 2/430 [00:00<00:28, 15.05it/s, 0s=0.87, 1s=0.13, NLL=87.58]

    Epoch  55/200 -- NLL: 88.99

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.86, 1s=0.14, NLL=85.87]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=87.08]

    Epoch  56/200 -- NLL: 88.96

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=85.27]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=89.78]

    Epoch  57/200 -- NLL: 88.86

    100%|██████████| 430/430 [00:28<00:00, 15.50it/s, 0s=0.87, 1s=0.13, NLL=87.83]
      0%|          | 2/430 [00:00<00:28, 15.29it/s, 0s=0.88, 1s=0.12, NLL=82.58]

    Epoch  58/200 -- NLL: 88.77

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=86.85]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=85.00]

    Epoch  59/200 -- NLL: 88.69

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=85.80]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=88.11]

    Epoch  60/200 -- NLL: 88.38

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.86, 1s=0.14, NLL=88.80]
      0%|          | 2/430 [00:00<00:27, 15.30it/s, 0s=0.87, 1s=0.13, NLL=83.50]

    Epoch  61/200 -- NLL: 88.54

    100%|██████████| 430/430 [00:28<00:00, 15.15it/s, 0s=0.88, 1s=0.12, NLL=83.24]
      0%|          | 2/430 [00:00<00:28, 15.05it/s, 0s=0.87, 1s=0.13, NLL=88.49]

    Epoch  62/200 -- NLL: 88.37

    100%|██████████| 430/430 [00:28<00:00, 15.45it/s, 0s=0.87, 1s=0.13, NLL=86.54]
      0%|          | 2/430 [00:00<00:28, 15.17it/s, 0s=0.88, 1s=0.12, NLL=81.83]

    Epoch  63/200 -- NLL: 88.52

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.88, 1s=0.12, NLL=83.34]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=85.14]

    Epoch  64/200 -- NLL: 88.17

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=83.71]
      0%|          | 2/430 [00:00<00:28, 15.21it/s, 0s=0.87, 1s=0.13, NLL=81.86]

    Epoch  65/200 -- NLL: 88.04

    100%|██████████| 430/430 [00:28<00:00, 15.44it/s, 0s=0.87, 1s=0.13, NLL=84.48]
      0%|          | 2/430 [00:00<00:28, 15.20it/s, 0s=0.87, 1s=0.13, NLL=87.96]

    Epoch  66/200 -- NLL: 88.16

    100%|██████████| 430/430 [00:28<00:00, 15.48it/s, 0s=0.87, 1s=0.13, NLL=85.14]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=85.12]

    Epoch  67/200 -- NLL: 88.18

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.86, 1s=0.14, NLL=86.76]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=86.37]

    Epoch  68/200 -- NLL: 87.83

    100%|██████████| 430/430 [00:28<00:00, 15.47it/s, 0s=0.87, 1s=0.13, NLL=82.42]
      0%|          | 2/430 [00:00<00:28, 15.24it/s, 0s=0.87, 1s=0.13, NLL=85.36]

    Epoch  69/200 -- NLL: 87.82

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=86.63]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.86, 1s=0.14, NLL=88.78]

    Epoch  70/200 -- NLL: 87.87

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=86.15]
      0%|          | 2/430 [00:00<00:28, 15.20it/s, 0s=0.87, 1s=0.13, NLL=84.53]

    Epoch  71/200 -- NLL: 87.63

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.88, 1s=0.12, NLL=83.52]
      0%|          | 2/430 [00:00<00:28, 15.04it/s, 0s=0.87, 1s=0.13, NLL=83.23]

    Epoch  72/200 -- NLL: 87.63

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=86.17]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=81.23]

    Epoch  73/200 -- NLL: 87.57

    100%|██████████| 430/430 [00:28<00:00, 14.99it/s, 0s=0.87, 1s=0.13, NLL=84.90]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=82.50]

    Epoch  74/200 -- NLL: 87.42

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=84.88]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.87, 1s=0.13, NLL=81.42]

    Epoch  75/200 -- NLL: 87.41

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=82.15]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.87, 1s=0.13, NLL=82.34]

    Epoch  76/200 -- NLL: 87.37

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=86.65]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=84.66]

    Epoch  77/200 -- NLL: 87.20

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=85.89]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=85.22]

    Epoch  78/200 -- NLL: 87.29

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=81.57]
      0%|          | 2/430 [00:00<00:28, 15.01it/s, 0s=0.88, 1s=0.12, NLL=82.69]

    Epoch  79/200 -- NLL: 87.04

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=83.04]
      0%|          | 2/430 [00:00<00:28, 15.03it/s, 0s=0.87, 1s=0.13, NLL=86.07]

    Epoch  80/200 -- NLL: 87.28

    100%|██████████| 430/430 [00:28<00:00, 15.51it/s, 0s=0.87, 1s=0.13, NLL=83.22]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=84.26]

    Epoch  81/200 -- NLL: 87.18

    100%|██████████| 430/430 [00:28<00:00, 15.35it/s, 0s=0.88, 1s=0.12, NLL=80.37]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=82.31]

    Epoch  82/200 -- NLL: 87.01

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.86, 1s=0.14, NLL=84.00]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=84.94]

    Epoch  83/200 -- NLL: 86.98

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.88, 1s=0.12, NLL=86.64]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.87, 1s=0.13, NLL=83.31]

    Epoch  84/200 -- NLL: 86.97

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=83.95]
      0%|          | 2/430 [00:00<00:28, 15.03it/s, 0s=0.86, 1s=0.14, NLL=83.34]

    Epoch  85/200 -- NLL: 86.99

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=86.84]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=85.20]

    Epoch  86/200 -- NLL: 86.84

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=85.53]
      0%|          | 2/430 [00:00<00:28, 15.05it/s, 0s=0.87, 1s=0.13, NLL=81.82]

    Epoch  87/200 -- NLL: 86.77

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=84.83]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.87, 1s=0.13, NLL=83.47]

    Epoch  88/200 -- NLL: 86.65

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.86, 1s=0.14, NLL=83.04]
      0%|          | 2/430 [00:00<00:28, 15.00it/s, 0s=0.87, 1s=0.13, NLL=80.21]

    Epoch  89/200 -- NLL: 86.75

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=82.52]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.86, 1s=0.14, NLL=86.51]

    Epoch  90/200 -- NLL: 86.63

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.88, 1s=0.12, NLL=85.19]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.86, 1s=0.14, NLL=84.78]

    Epoch  91/200 -- NLL: 86.57

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=83.03]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.87, 1s=0.13, NLL=84.04]

    Epoch  92/200 -- NLL: 86.69

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.88, 1s=0.12, NLL=79.43]
      0%|          | 2/430 [00:00<00:28, 15.23it/s, 0s=0.86, 1s=0.14, NLL=86.96]

    Epoch  93/200 -- NLL: 86.53

    100%|██████████| 430/430 [00:28<00:00, 15.46it/s, 0s=0.87, 1s=0.13, NLL=79.47]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.87, 1s=0.13, NLL=80.32]

    Epoch  94/200 -- NLL: 86.59

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=86.14]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=84.00]

    Epoch  95/200 -- NLL: 86.43

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=83.54]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.86, 1s=0.14, NLL=85.80]

    Epoch  96/200 -- NLL: 86.21

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=80.04]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.86, 1s=0.14, NLL=84.77]

    Epoch  97/200 -- NLL: 86.36

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=79.34]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=82.33]

    Epoch  98/200 -- NLL: 86.32

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.87, 1s=0.13, NLL=80.63]
      0%|          | 2/430 [00:00<00:28, 15.25it/s, 0s=0.86, 1s=0.14, NLL=84.73]

    Epoch  99/200 -- NLL: 86.29

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=83.12]
      0%|          | 2/430 [00:00<00:28, 15.00it/s, 0s=0.87, 1s=0.13, NLL=81.92]

    Epoch 100/200 -- NLL: 86.23

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.86, 1s=0.14, NLL=87.93]
      0%|          | 2/430 [00:00<00:28, 15.17it/s, 0s=0.88, 1s=0.12, NLL=82.57]

    Epoch 101/200 -- NLL: 86.28

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=85.76]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=81.11]

    Epoch 102/200 -- NLL: 86.36

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=81.03]
      0%|          | 2/430 [00:00<00:28, 15.02it/s, 0s=0.87, 1s=0.13, NLL=82.56]

    Epoch 103/200 -- NLL: 86.35

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=79.41]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.88, 1s=0.12, NLL=81.03]

    Epoch 104/200 -- NLL: 86.24

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=83.91]
      0%|          | 2/430 [00:00<00:28, 15.16it/s, 0s=0.87, 1s=0.13, NLL=84.81]

    Epoch 105/200 -- NLL: 86.06

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.86, 1s=0.14, NLL=85.12]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.87, 1s=0.13, NLL=83.63]

    Epoch 106/200 -- NLL: 86.04

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=80.11]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=80.59]

    Epoch 107/200 -- NLL: 85.99

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=81.93]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=83.43]

    Epoch 108/200 -- NLL: 86.03

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=85.82]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.87, 1s=0.13, NLL=80.85]

    Epoch 109/200 -- NLL: 85.89

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=83.12]
      0%|          | 2/430 [00:00<00:28, 15.09it/s, 0s=0.86, 1s=0.14, NLL=80.67]

    Epoch 110/200 -- NLL: 85.90

    100%|██████████| 430/430 [00:28<00:00, 15.36it/s, 0s=0.87, 1s=0.13, NLL=83.61]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=82.41]

    Epoch 111/200 -- NLL: 85.75

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=85.15]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=81.26]

    Epoch 112/200 -- NLL: 85.85

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.88, 1s=0.12, NLL=79.55]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=79.91]

    Epoch 113/200 -- NLL: 85.83

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=80.87]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.86, 1s=0.14, NLL=83.63]

    Epoch 114/200 -- NLL: 85.73

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=82.48]
      0%|          | 2/430 [00:00<00:28, 15.18it/s, 0s=0.87, 1s=0.13, NLL=83.82]

    Epoch 115/200 -- NLL: 85.81

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=83.58]
      0%|          | 2/430 [00:00<00:28, 15.13it/s, 0s=0.87, 1s=0.13, NLL=83.58]

    Epoch 116/200 -- NLL: 85.76

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.88, 1s=0.12, NLL=84.03]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.86, 1s=0.14, NLL=84.93]

    Epoch 117/200 -- NLL: 85.81

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=80.20]
      0%|          | 2/430 [00:00<00:28, 15.16it/s, 0s=0.86, 1s=0.14, NLL=82.02]

    Epoch 118/200 -- NLL: 85.59

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=79.74]
      0%|          | 2/430 [00:00<00:28, 15.08it/s, 0s=0.86, 1s=0.14, NLL=80.44]

    Epoch 119/200 -- NLL: 85.61

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=82.00]
      0%|          | 2/430 [00:00<00:28, 14.96it/s, 0s=0.87, 1s=0.13, NLL=82.88]

    Epoch 120/200 -- NLL: 85.71

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=81.11]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=81.17]

    Epoch 121/200 -- NLL: 85.68

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=80.78]
      0%|          | 2/430 [00:00<00:28, 15.10it/s, 0s=0.87, 1s=0.13, NLL=85.49]

    Epoch 122/200 -- NLL: 85.61

    100%|██████████| 430/430 [00:28<00:00, 15.39it/s, 0s=0.87, 1s=0.13, NLL=81.06]
      0%|          | 2/430 [00:00<00:28, 15.18it/s, 0s=0.87, 1s=0.13, NLL=83.62]

    Epoch 123/200 -- NLL: 85.61

    100%|██████████| 430/430 [00:28<00:00, 15.33it/s, 0s=0.87, 1s=0.13, NLL=83.31]
      0%|          | 2/430 [00:00<00:28, 15.15it/s, 0s=0.87, 1s=0.13, NLL=80.56]

    Epoch 124/200 -- NLL: 85.57

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.88, 1s=0.12, NLL=81.85]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=81.25]

    Epoch 125/200 -- NLL: 85.65

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=83.00]
      0%|          | 2/430 [00:00<00:28, 15.17it/s, 0s=0.87, 1s=0.13, NLL=82.55]

    Epoch 126/200 -- NLL: 85.60

    100%|██████████| 430/430 [00:28<00:00, 15.39it/s, 0s=0.86, 1s=0.14, NLL=82.55]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.86, 1s=0.14, NLL=82.18]

    Epoch 127/200 -- NLL: 85.47

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=83.97]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=80.45]

    Epoch 128/200 -- NLL: 85.69

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=78.86]
      0%|          | 2/430 [00:00<00:28, 15.19it/s, 0s=0.87, 1s=0.13, NLL=82.06]

    Epoch 129/200 -- NLL: 85.70

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=83.12]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.86, 1s=0.14, NLL=82.71]

    Epoch 130/200 -- NLL: 85.51

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=85.25]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=80.89]

    Epoch 131/200 -- NLL: 85.29

    100%|██████████| 430/430 [00:28<00:00, 15.39it/s, 0s=0.87, 1s=0.13, NLL=79.68]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=77.44]

    Epoch 132/200 -- NLL: 85.40

    100%|██████████| 430/430 [00:28<00:00, 15.43it/s, 0s=0.86, 1s=0.14, NLL=86.57]
      0%|          | 2/430 [00:00<00:28, 15.07it/s, 0s=0.87, 1s=0.13, NLL=81.14]

    Epoch 133/200 -- NLL: 85.57

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=79.65]
      0%|          | 2/430 [00:00<00:28, 15.12it/s, 0s=0.87, 1s=0.13, NLL=79.15]

    Epoch 134/200 -- NLL: 85.38

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=79.83]
      0%|          | 2/430 [00:00<00:28, 15.04it/s, 0s=0.88, 1s=0.12, NLL=80.38]

    Epoch 135/200 -- NLL: 85.30

    100%|██████████| 430/430 [00:28<00:00, 15.40it/s, 0s=0.87, 1s=0.13, NLL=79.43]
      0%|          | 2/430 [00:00<00:28, 15.14it/s, 0s=0.87, 1s=0.13, NLL=79.21]

    Epoch 136/200 -- NLL: 85.32

    100%|██████████| 430/430 [00:28<00:00, 15.41it/s, 0s=0.87, 1s=0.13, NLL=80.84]
      0%|          | 2/430 [00:00<00:28, 15.16it/s, 0s=0.87, 1s=0.13, NLL=81.10]

    Epoch 137/200 -- NLL: 85.53

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=83.15]
      0%|          | 2/430 [00:00<00:28, 15.17it/s, 0s=0.87, 1s=0.13, NLL=81.54]

    Epoch 138/200 -- NLL: 85.32

    100%|██████████| 430/430 [00:28<00:00, 15.39it/s, 0s=0.88, 1s=0.12, NLL=75.25]
      0%|          | 2/430 [00:00<00:28, 15.06it/s, 0s=0.87, 1s=0.13, NLL=78.96]

    Epoch 139/200 -- NLL: 85.56

    100%|██████████| 430/430 [00:28<00:00, 15.45it/s, 0s=0.87, 1s=0.13, NLL=78.74]
      0%|          | 2/430 [00:00<00:28, 15.11it/s, 0s=0.87, 1s=0.13, NLL=81.66]

    Epoch 140/200 -- NLL: 85.54

    100%|██████████| 430/430 [00:28<00:00, 15.42it/s, 0s=0.87, 1s=0.13, NLL=80.93]

    Early stopping at epoch 141/200

Let's visualise samples

```python
from matplotlib import pyplot as plt

def visualize_made(batcher: Batcher, args, model, N=4, writer=None, name='dev'):

    assert N <= args.batch_size, "N should be no bigger than a batch"
    with torch.no_grad():
        model.eval()

        plt.figure(figsize=(2*N, 2*N))
        plt.subplots_adjust(wspace=0.5, hspace=0.5)

        # Some visualisations
        for x_mb, y_mb in batcher:

            # [B, H*W]
            x_mb = x_mb.reshape(-1, args.height * args.width)
            x_mb = x_mb[:N]
            # [B, 10]
            context = None

            p_x = model(inputs=None, history=x_mb, resample_mask=False)
            # [B, H*W]
            ll = p_x.log_prob(x_mb)
            prob = torch.exp(ll)

            # reconstruct bottom half of N instances
            x_rec = model.sample(inputs=None, history=x_mb, start_from=args.height * args.width // 2)
            # sample N instances
            x_sample = model.sample(
                inputs=None,
                history=torch.zeros(
                    [N, args.height * args.width],
                    device=torch.device(args.device),
                    dtype=torch.float32
                ),
                start_from=0
            )
            for i in range(N):
                plt.subplot(4, N, 0*N + i + 1)
                plt.imshow(x_mb[i].reshape(args.height, args.width).cpu(), cmap='Greys')
                plt.title("x%d" % (i + 1))

                plt.subplot(4, N, 1*N + i + 1)
                plt.imshow(prob[i].reshape(args.height, args.width).cpu(), cmap='Greys')
                plt.title("prob%d" % (i + 1))

                plt.subplot(4, N, 2*N + i + 1)
                plt.axhline(y=args.height//2, c='red', linewidth=1, ls='--')
                plt.imshow(x_rec[i].reshape(args.height, args.width).cpu(), cmap='Greys')
                plt.title("rec%d" % (i + 1))

                plt.subplot(4, N, 3*N + i + 1)
                plt.imshow(x_sample[i].reshape(args.height, args.width).cpu(), cmap='Greys')
                plt.title("sample%d" % (i + 1))

            break

    plt.show()
```

A few reconstructions for the validation set as well as samples from the autoregressive likelihood

```python
visualize_made(get_batcher(valid_loader), args, model)
```

```python

```


```python
import numpy as np
import math as m
import matplotlib.pyplot as plt
import sympy

def cel_to_kel(D):
    K = D + 273.15
    return K

def kel_to_cel(K):
    D = K - 273.15
    return D
```

# Useful constants

```python
sigma = 5.67*10**(-8)
```

# Exercício 3.1
- Página 69

### Considerações:
1. Condições de regime estacionário.

2. Transferência de calor unidimensional por condução por meio das camadas de pele/gordura e de isolante.

3. Resistência de contato é desprezível.

4. Condutividades térmicas são uniformes.

5. Troca radiante entre a superfície dos casacos e a vizinhança pode ser vista como entre uma pequena superfície e uma grande vizinhança na temperatura do ar.

6. Água líquida é opaca para a radiação térmica.

7. Radiação solar é desprezível.

8. O corpo encontra-se completamente imerso na água na parte 2.

```python
# Dados

T_i = cel_to_kel(35)
T_inf = T_viz = kel_to_cel(10)
k_pg = 0.3
L_pg = 3e-3
epsilon = 0.95
k_iso = 0.014
h1 = 2
h2 = 200
hr = 5.9
q = 100
A = 1.8

# Analise

R_tot = (T_i-T_inf)/q

print('R_tot = {:.6f} K/W'.format(R_tot))

# Para o ar

L_iso = k_iso*(A*R_tot-L_pg/k_pg-1/(h1+hr))

print('Para o ar: L_iso = {:.6e} m'.format(L_iso))

# Para o agua

L_iso = k_iso*(A*R_tot-L_pg/k_pg-1/(h2))

print('Para a agua: L_iso = {:.6e} m'.format(L_iso))

T_p = T_i - q*L_pg/(k_pg*A)

print('Temperatura da pele: T_p = {:.6f} K = {:.6f}ºC '.format(T_p,kel_to_cel(T_p)))
```

    R_tot = 5.713000 K/W
    Para o ar: L_iso = 1.420554e-01 m
    Para a agua: L_iso = 1.437576e-01 m
    Temperatura da pele: T_p = 307.594444 K = 34.444444ºC

# Exercício 3.2
- Página 71

### Considerações:
1. Condições de regime estacionário.

2. Condução unidimensional (transferência de calor desprezível nas laterais do conjunto).

3. Resistência térmica no chip desprezível (chip isotérmico).

4. Propriedades constantes.

5. Troca radiante com a vizinhança desprezível.

### Balanço de energia:

$ \Large \dot{E}_{acu}=\dot{E}_{ent}+\dot{E}_{g}-\dot{E}_{sai} \therefore 0=\dot{E}_{g}-\dot{E}_{sai} \therefore \dot{E}_{g}=\dot{E}_{sai} \therefore {q_c}''={q_1}''+{q_2}''\therefore \\ \Large  {q_c}'' = \frac{T_c-T_infty}{1/h} + \frac{T_c-T_\infty}{{R}''_{t,c} + (L/k_{al}) + (L/h)}$

```python
L = 8e-3
W = 10e-3
T_inf = cel_to_kel(25)
h = 100
flux_q_c = 10**4
k_al = 239
R_tc = 0.9e-4

T_c = T_inf + flux_q_c*(h+1/(R_tc +(L/k_al)+(1/h)))**(-1)

print('Temperatura do chip: {:.6f} K = {:.6f} ºC'.format(T_c,kel_to_cel(T_c)))
```

    Temperatura do chip: 348.456788 K = 75.306788 ºC

# Exercício 3.3
- Página 73

### Considerações:
1. Condições de regime estacionário.

2. Transferência de calor unidimensional.

3. Propriedades constantes.

4. Resistências térmicas de contato desprezíveis.

5. Diferenças de temperaturas no interior da camada de silício desprezíveis.

```python
k_v = 1.4
L_v = 3e-3
L_a = 0.1e-3
k_a = 145
L_s = 0.1e-3
L_n = 2e-3
a = 0.553
b = 0.001
G_s = 700
ref_v = 7/100
abs_v = 10/100
abs_s = 83/100
epsilon_v = 0.9
C = 1
W = 0.1
h = 35
T_inf = T_viz = cel_to_kel(20)

K = L_a/k_a + L_v/k_v

P, Tsi, Tvtop = sympy.symbols("P Tsi Tvtop", real=True)

eq1 = sympy.Eq(P/(0.83 * G_s * C * W) + b * Tsi, a)
eq2 = sympy.Eq(Tsi*(1/K-0.83*G_s*b)-Tvtop*(1/K),0.83*(1-a))
eq3 = sympy.Eq(h*(Tvtop-T_inf)+epsilon_v*sigma*(Tvtop**4-T_viz**4)-0.83*G_s*b*Tsi, 0.83*G_s*(1-a)+0.1*G_s)

sympy.solve([eq1, eq2,eq3])
```

    [{P: 14.3466715857189, Tsi: 306.069335874030, Tvtop: 305.687361634752},
     {P: 88.2376344520889, Tsi: -965.720042204628, Tvtop: -964.518129075350}]

```python
import sympy

x, y = sympy.symbols("x y", real=True)

eq1 = sympy.Eq((6.3205 - x)**2 + (-0.0347 - y)**2, 1.4869**2)
eq2 = sympy.Eq((8.3769 - x)**2 + (-0.6242 - y)**2, 0.8459**2)

sympy.solve([eq1, eq2])
```

    [{x: 7.56236430542394, y: -0.852406950510969},
     {x: 7.80697412192104, y: 0.000885037011715321}]

Solve the system of equations x0 + 2 * x1 = 1 and 3 * x0 + 5 * x1 = 2:

# Exercício 3.4

> # Não entendi
- Página 74

### Considerações:
1. Condições de regime estacionário.

2. Transferência de calor unidimensional.

3. As ilhas aquecida e sensora são isotérmicas.

4. Troca radiante entre as superfícies e a vizinhança é desprezível.

5. Perdas convectivas são desprezíveis.

6. Aquecimento ôhmico nas linhas de platina é desprezível.

7. Propriedades constantes.

8. Resistência de contato entre o nanotubo e as ilhas é desprezível.

# Exercício 3.5
- Página 77

### Considerações:
1. Condições de regime estacionário.

2. Condução unidimensional na direção x.

3. Não há geração de calor no interior do cone.

4. Propriedades constantes.

```python
a = 0.25
x1 = 50e-3
x2 = 250e-3
T1 = 400
T2 = 600
k = 3.46

qx = np.pi*a**2*k*(T1 - T2)/(4*((1/x1)-(1/x2)))

x = np.linspace(x1,x2,1000)

T = T1-(4*qx/(np.pi*a**2*k))*((1/x1)-(1/x))

print('A taxa de transferência de calor é {:.4f} W'.format(qx))
```

    A taxa de transferência de calor é -2.1230 W

```python
fig = plt.figure(figsize=[16,9])
ax = fig.subplots(1)

# Plot

ax.set_ylabel('$T(K)$',fontsize=16)
ax.set_xlabel('$x$(m)',fontsize=16)
ax.set_title('Gráfico da Temperatura (T) em função da distância (x)',fontsize=20)

ax.plot(x, T,'b', linewidth=4)

ax.grid()

plt.show()
```

# Exercício 3.6
> # Conceitual
- Página 80

### Considerações:
1. Condições de regime estacionário.

2. Transferência de calor unidimensional na direção radial (cilíndrica).

3. Resistência térmica na parede do tubo desprezível.

4. Propriedades constantes do isolante.

5. Troca térmica por radiação entre a superfície externa do isolante e a vizinhança desprezível.

# Exercício 3.7
- Página 84

### Considerações:
1. Condições de regime estacionário.

2. Condução unidimensional na direção x.

3. Resistência de contato entre as paredes desprezível.

4. Superfície interna de A adiabática.

5. Propriedades dos materiais A e B constantes.

```python
# Dados

dot_q = 1.5e6
ka = 75
La = 50e-3
kb = 150
Lb = 20e-3
T_inf = cel_to_kel(30)
h = 1e3

T2 = T_inf + dot_q*La/h

print('A Temperatura 2 é {:.4f} K = {:.4f} C '.format(T2,kel_to_cel(T2)))

flux_q = h*(T2-T_inf)

T1 = T_inf + (Lb/kb+1/h)*flux_q

print('A Temperatura 1 é {:.4f} K = {:.4f} C '.format(T1,kel_to_cel(T1)))

T0 = dot_q*La**2/(2*ka) + T1

print('A Temperatura 0 é {:.4f} K = {:.4f} C '.format(T0,kel_to_cel(T0)))
```

    A Temperatura 2 é 378.1500 K = 105.0000 C
    A Temperatura 1 é 388.1500 K = 115.0000 C
    A Temperatura 0 é 413.1500 K = 140.0000 C

# Exercício 3.8
> # Conceitual
- Página 86

### Considerações:
1. Condições de regime estacionário.

2. Condução radial unidimensional.

3. Propriedades constantes.

4. Geração de calor volumétrica uniforme.

5. Superfície externa adiabática.

# Exercício 3.9
- Página 92

### Considerações:
1. Condições de regime estacionário.

2. Temperatura uniforme na seção transversal do bastão.

3. Propriedades constantes.

4. Troca radiante com a vizinhança desprezível.

5. Coeficiente de transferência de calor uniforme.

6. Bastão com comprimento infinito.

```python
D = 5e-3
Tb = 100
Tinf = 25
h = 100
k_cu = 398
k_al = 180
k_aco = 14

P = np.pi * D
Atr = np.pi * D**2 / 4

theta_b = Tb - Tinf

Mcu = theta_b * (4*h/(k_cu*D))**0.5
Mal = theta_b * (4*h/(k_al*D))**0.5
Maco = theta_b * (4*h/(k_aco*D))**0.5

x = np.linspace(0,300e-3,1000)

Tcu = Tinf + (Tb-Tinf) * np.exp((-Mcu)*x)
Tal = Tinf + (Tb-Tinf) * np.exp((-Mal)*x)
Taco = Tinf + (Tb-Tinf) * np.exp((-Maco)*x)

fig = plt.figure(figsize=[16,9])
ax = fig.subplots(1)

# Plot

ax.set_ylabel('$T(K)$',fontsize=16)
ax.set_xlabel('$x$(m)',fontsize=16)
ax.set_title('Gráfico da Temperatura (T) em função da distância (x)',fontsize=20)

ax.plot(x, Tcu,'b', linewidth=4)
ax.plot(x, Tal,'r', linewidth=4)
ax.plot(x, Taco,'g', linewidth=4)

ax.grid()

plt.show()
```

# Exercício 3.10
- Página 98

### Considerações:
1. Condições de regime estacionário.

2. Temperatura uniforme através da espessura da aleta.

3. Propriedades constantes.

4. Troca radiante com a vizinhança desprezível.

5. Coeficiente convectivo uniforme sobre a superfície externa (com ou sem aletas).

```python
H = 100e-3
D = 50e-3
r1 = D/2
q_t = 2e3
Tinf = 300
h = 75
n = 10
t = 4e-3
L = 20e-3
r2 = r1 + L
k = 186

```

# Tabelas de resolução
> ### TABELA 3.3
Soluções unidimensionais, em regime estacionário, da equação do calor sem geração

> ### TABELA 3.4
Distribuição de temperaturas e taxas de perda de calor em aletas de seção transversal uniforme

> ### TABELA 3.5
Eficiência de perfis de aletas comuns

> ### TABELA C.1:
Soluções da Equação do Calor Unidimensionais e em Regime Estacionário para Paredes Planas, Cilíndricas e Esféricas com Geração Uniforme e Condições nas Superfícies Assimétricas

> ### TABELA C.2
Condições Superficiais Alternativas e Balanços de Energia para Soluções Unidimensionais em Regime Estacionário da Equação do Calor para Paredes Planas, Cilíndricas e Esféricas com Geração Uniforme

> ### TABELA C.3
Soluções Unidimensionais em Regime Estacionário da Equação do Calor com Geração Uniforme em uma Parede Plana com uma Superfície Adiabática, em um Cilindro Sólido e em uma Esfera Sólida

> ### TABELA C.4
Condições Superficiais Alternativas e Balanços de Energia para Soluções Unidimensionais em Regime Estacionário da Equação do Calor com Geração Uniforme em uma Parede Plana com uma Superfície Adiabática, em um Cilindro Sólido e em uma Esfera Sólida

```python

```