# Problema de selección de portafolio con preferencias media-varianza

En la clase pasada hablamos acerca de:
- preferencias,
- funciones de utilidad,
- la actitud de los inversionistas de cara al riesgo,
- la aversión al riesgo, entre otros.

Todas ellas son piezas que necesitamos para responder la pregunta de ¿cómo un inversionista toma la decisión óptima de selección de portafolio?

En esta clase al fin estamos listos para ensamblar estos conceptos y escribir el problema de selección de portafolios.

En el camino aprenderemos acerca del concepto de **utilidad esperada**, que nos permite trabajar con incertidumbre en el modelado económico (una de las ideas más importantes en economía). Esta idea tiene más de 60 años, y básicamente dice que los individuos, cuando están de cara a incertidumbre, maximizan el valor esperado de su utilidad (solo cierto si somos homo economicus).

Además del concepto de utilidad esperada, aprenderemos acerca de **preferencias media-varianza**. Es decir, supondremos que los inversionistas toman decisiones basados en un tipo particular de preferencias.

Con lo anterior, estableceremos el problema de selección de portafolios.

**Objetivos:**

- ¿Qué es utilidad esperada?
- ¿Qué son las preferencias media-varianza?
- Funciones de utilidad media-varianza.
- Enunciado y solución del problema básico de selección de portafolio.

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Utilidad esperada
- Básicamente combina las probabilidades de los resultados con cómo los inversionistas se sienten con dichos resultados.
- En otras palabras, la utilidad esperada multiplica la probabilidad de suceso de un evento con la utilidad que genera dicho evento.

Recordemos que las *funciones de utilidad* permiten a los inversionistas expresar cómo se sienten con los resultados, especialmente en los malos ratos.

Entonces la *utilidad esperada* es una herramienta que nos permite cuantificar cómo nos sentimos en nuestros malos momentos económicos, capturando el riesgo con la probabilidad de ocurrencia de dichos malos momentos.

Dado este marco de trabajo, cualquier decisión se puede escribir como la maximización de la utilidad esperada:
\begin{align}
\max_{\theta} & \quad E[U(W)], \\
\end{align}
mediante la escogencia de cierta variable $\theta$ (gastos, planes de ahorro, compra de activos, planes de producción, etc.).

Para nuestros propósitos, la variable de decisión serán los pesos o ponderaciones del portafolio.

Adicionalmente, en el contexto de la decisión de distribución de la riqueza entre activos, el problema de maximización tendrá comúnmente las siguientes restricciones:
- universo de inversión,
- posición en los activos dados.

**Ejemplo.**

Supongamos que un inversionista debe determinar la composición óptima de su portafolio, que contiene activos y bonos. Supongamos que son los únicos instrumentos disponibles.

Sean:
- $w_s$: peso o ponderación de activos en el portafolio,
- $w_b$: peso o ponderación de bonos en el portafolio,
- $r_s$: rendimiento de los activos, y
- $r_b$: rendimiento de los bonos.

De manera que podemos escribir el problema de selección de portafolios como la maximización de la utilidad esperade de nuestra riqueza futura, la cual dependerá de nuestros rendimientos:

\begin{align}
\max_{w_s,w_b} &\quad E[U(W)]\\
\text{s. a.}   &\quad W=W_0(1+w_sr_s+w_br_b)\\
               &\quad w_s+w_b=1\\
               &\quad w_s,w_b\geq0
\end{align}

Preguntas:
- ¿Qué significan las restricciones?
- Ya que tenemos planteado este problema básico, ¿qué haría falta para empezar a resolverlo?
___

## 2. Preferencias media-varianza

### 2.1. Utilidad media-varianza

Entonces, ¿qué funciones de utilidad deberíamos de usar en este problema de selección de portafolios?

- La respuesta es: **preferencias media-varianza**.
- Éstas serán representadas en términos de funciones de utilidad como: **utilidad media-varianza**.

Usamos la *utilidad media-varianza* en el problema de selección de portafolios dado que ésta decribe el "trade-off" entre riesgo y rendimiento que enfrentan los inversionistas. La *utilidad media-varianza* está dada por la siguiente expresión:

$$U=E[r_p]-\frac{1}{2}\gamma\sigma_p^2,$$

donde
- $E[r_p]$ es el rendimiento esperado del portafolio,
- $\sigma_p^2$ es la varianza del portafolio, y
- $\gamma$ es el coeficiente de aversión al riesgo.

#### Intuición acerca de la función de utilidad media-varianza:
- Sólo se preocupa por medias :) y varianzas :(.
- Incrementa con: rendimiento esperado del portafolio.
- Decrece con: varianza del portafolio.
- Malos tiempos: rendimientos son bajos y las volatilidades son altas.
- Conecta bastante bien con la teoría moderna de portafolios, la cual caracteriza los rendimientos con medias y varianzas únicamente.
- Criticada por su limitación: supone que los inversionistas sólo se preocupan por medias y varianzas.

### 2.2. Curvas de indiferencia

*¿Recuerdan las curvas de nivel que se ven en cálculo de varias variables?*
- Bien, acá nos servirán para representar la utilidad media-varianza gráficamente.
- En el contexto de utilidad media-varianza, las curvas de nivel se llaman **curvas de indiferencia**.

Dados ciertos niveles de utilidad $U_1>U_2>U_3$, las curvas de indiferencia relativas a estos niveles de utilidad, son los lugares geométricos en el espacio de rendimiento esperado vs. volatilidad representados por las siguientes expresiones

$$U_1=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_1,$$

$$U_2=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_2,$$

$$U_3=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_3.$$

**Gráficamente**

```python
# Importar numpy y pyplot
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
```

```python
# Coeficiente de aversión al riesgo (entre 1 y 10 comúnmente)
g = 5
# Niveles de utilidad
U1, U2, U3 = 0.05, 0.12, 0.19
# Vector de volatilidades (sugerido 1%-60%)
sp = np.linspace(0.01, 0.6)
# Curvas de indiferencia
Erp1 = U1 + 0.5 * g * sp**2
Erp2 = U2 + 0.5 * g * sp**2
Erp3 = U3 + 0.5 * g * sp**2
```

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(sp, Erp1, label=f'$U_1$={U1}')
plt.plot(sp, Erp2, label=f'$U_2$={U2}')
plt.plot(sp, Erp3, label=f'$U_3$={U3}')
plt.grid()
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento esperado $E[r_p]$')
plt.legend(loc='best')
```

Bueno, ¿y porqué se llaman curvas de indiferencia?, ¿qué representa una cuva de indiferencia?

- Porque sobre una misma curva el nivel de utilidad es el mismo (es indiferente).
- Son todas las combinaciones de riesgo y rendimiento que producen un mismo nivel de utilidad.

Volviendo al problema de selección de portafolios, queremos la utilidad más alta.
- ¿Cuál de las anteriores curvas de indiferencia corresponde a la utilidad más alta?
- Intuitivamente, ¿porqué?
- Curvas de indiferencia para niveles de utilidad más altos, estarán...

Notamos además que las anteriores curvas de indiferencia son *paralelas* una con otra. Claro, las dibujamos con el mismo coeficiente de aversión al riesgo.

$$\frac{d E[r_p]}{d\sigma_p}=\frac{d }{d\sigma_p}\left[\frac{1}{2}\gamma\sigma_p^2+U\right]=\gamma\sigma_p.$$

¿Cómo cambian estas curvas para coeficientes de aversión al riesgo más altos?

```python
# Coeficientes de aversión al riesgo (entre 1 y 10 comúnmente)
g1, g2, g3 = 3, 5, 7
# Nivel de utilidad
U = 0.15
# Vector de volatilidades (sugerido 1%-60%)
sp = np.linspace(0.01, 0.6)
# Curvas de indiferencia
Erp1 = U + 0.5 * g1 * sp**2
Erp2 = U + 0.5 * g2 * sp**2
Erp3 = U + 0.5 * g3 * sp**2
```

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(sp, Erp1, label=f'$\gamma_1$={g1}')
plt.plot(sp, Erp2, label=f'$\gamma_2$={g2}')
plt.plot(sp, Erp3, label=f'$\gamma_3$={g3}')
plt.grid()
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento esperado $E[r_p]$')
plt.legend(loc='best')
```

¿Cómo interpretamos las anteriores gráficas?, ¿qué pasa con las personas más aversas al riesgo?
- Se puede ver de dos maneras: para un mismo nivel de rendimiento esperado, una persona más aversa al riesgo soporta un nivel menor de riesgo; equivalentemente, para un mismo nivel de riesgo, una persona más aversa al riesgo requerirá un nivel de rendimiento esperado más alto.

Con todo lo anterior, el problema de selección de portafolios se puede plantear como *encontrar la curva de indeferencia más alta dado el conjunto de oportunidades de inversión y restricciones*.

## 3. Problema de selección de portafolios: una ilustración

Ahora ilustraremos el problema de selección de portafolios con algunos datos.
- Por ahora solo queremos ilustrar gráficamente cómo se resuelve este problema. Trabajar en la intuición.
- En las siguientes dos clases nos enfocaremos en cómo resolverlo analíticamente.

Acá tenemos el rendimiento medio anual y la volatilidad para dos instrumentos usando datos de EU: instrumentos de deuda (bonos) y acciones. Supondremos que el inversionista solo puede invertir en estas dos clases de instrumentos.

```python
# Importamos pandas
import pandas as pd
```

```python
# Datos
data = pd.DataFrame(index=['Stocks','Bonds', 'CorrSB'], columns=['Mean', 'Std'])
data['Mean'] = [0.119, 0.0591, 0.113]
data['Std'] = [0.1915, 0.0833, None]
data
```

      Mean
      Std

      Stocks
      0.1190
      0.1915

      Bonds
      0.0591
      0.0833

      CorrSB
      0.1130
      NaN

Entonces, ¿cuál es la distribución de riqueza óptima?, o más bien, ¿cuál es la composición óptima del portafolio para un inversionista dado su nivel de aversión al riesgo?

**Primero.** Recordamos que, para dos activos, podemos trazar la frontera de mínima varianza tomando todas las posibles combinaciones de los dos activos.

De nuevo, sean:
- $w_s=w$: peso o ponderación de activos en el portafolio,
- $w_b=1-w$: peso o ponderación de bonos en el portafolio,
- $r_s$: rendimiento de los activos, y
- $r_b$: rendimiento de los bonos.

Entonces

$$E[r_p]=wE[r_{s}]+(1-w)E[r_b]$$

$$\sigma_p^2=w^2\sigma_{s}^2+(1-w)^2\sigma_b^2+2w(1-w)\rho_{s,b}\sigma_s\sigma_b$$

```python
# Vector de w variando entre 0 y 1 con n pasos
w = np.linspace(0, 1, 100)
# Rendimientos esperados individuales
Es = data.loc['Stocks', 'Mean']
Eb = data.loc['Bonds', 'Mean']
# Volatilidades individuales
ss = data.loc['Stocks', 'Std']
sb = data.loc['Bonds', 'Std']
# Correlacion
rsb = data.loc['CorrSB', 'Mean']
```

```python
# Crear un DataFrame cuyas columnas sean rendimiento
# y volatilidad del portafolio para cada una de las w
# generadas
portafolios = pd.DataFrame(data={'w': w,
                                 'Media': w * Es + (1 - w) * Eb,
                                 'Vol': ((w * ss)**2 + ((1 - w) * sb)**2 + 2 * w *(1 - w)* ss * sb * rsb)**0.5})
portafolios.head()
```

      w
      Media
      Vol

      0
      0.000000
      0.059100
      0.083300

      1
      0.010101
      0.059705
      0.082700

      2
      0.020202
      0.060310
      0.082144

      3
      0.030303
      0.060915
      0.081635

      4
      0.040404
      0.061520
      0.081174

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Portafolios')
plt.grid()
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento esperado $E[r_p]$')
plt.legend(loc='best')
```

**Segundo.** Graficamos en la misma ventana, curvas de indiferencia.

```python
# Niveles de utilidad
U1, U2, U3 = 0.04, 0.06, 0.07
# Coeficiente de aversión al riesgo
g = 5
# Curvas de indiferencia
sp = np.linspace(0.08, 0.18)
Erp1 = U1 + 0.5 * g * sp**2
Erp2 = U2 + 0.5 * g * sp**2
Erp3 = U3 + 0.5 * g * sp**2
```

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Portafolios')
plt.plot(sp, Erp1, label=f'$U_1$={U1}')
plt.plot(sp, Erp2, label=f'$U_2$={U2}')
plt.plot(sp, Erp3, label=f'$U_3$={U3}')
plt.grid()
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento esperado $E[r_p]$')
plt.legend(loc='best')
```

**Tercero.** La elección óptima está dada por la curva de indiferencia para el nivel de utilidad más alto que es tangente a la frontera media-varianza.
- Claramente, esta selección depende del coeficiente de aversión al riesgo.

```python
# Gráfica con zoom
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Portafolios')
plt.plot(sp, Erp1, label=f'$U_1$={U1}')
plt.plot(sp, Erp2, label=f'$U_2$={U2}')
plt.plot(sp, Erp3, label=f'$U_3$={U3}')
plt.grid()
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento esperado $E[r_p]$')
plt.legend(loc='best')
plt.axis([0.09, 0.11, 0.08, 0.09])
```

# Anuncios parroquiales

## 1. Quiz la siguiente clase.

Created with Jupyter by Esteban Jiménez Rodríguez.


# Introduction to Reinforcement Learning (RL) in PyTorch

### Recap of Supervised Learning

So far, we've primarily concerned ourselves with supervised learning problems (mostly classification).
In supervised learning, we are given some sort of training data consisting of input/output pairs, with the goal being to be able to predict the output given some new inputs after learning the model.
For example, we previously looked at a Convolutional Neural Network (CNN) classification model for MNIST; given a training set of 60000 digit images and corresponding digit labels (e.g. '5'), we learned a model that was capable of predicting the digit label of new MNIST images.
In order words, something like (but not exactly) this:

What if we want to learn how to perform more complex behaviors, where data collection can be expensive?
How do you teach a robot to walk?
Self-driving cars?
How do you defeat human champions in the game of Go?

### Reinforcement Learning
Enter Reinforcement Learning.
In Reinforcement Learning, our model (commonly referred to as an *agent* in this context) interacts with an *environment* by taking *actions* $a$ and receives some sort of feedback from the environment in the form of a *reward* $r$.
In this sense, reinforcement learning algorithms learn by experience.
We call the trajectory of going from start to finish of a task an *episode*, and often our agent will learn by undergoing many episodes.

Many reinforcement learning algorithms are modeled as Markov Decision Processes (MDPs).
In these settings, we have a concept of a *state* $s$, which encapsulates the situation of the agent (e.g. location, velocity).
From each state $s_t$, the agent takes an action $a_t$, which results in a transition from one state $s_t$ to another $s_{t+1}$.
In many settings, there is stochasticity in this transition, meaning that there's is a distribution over $s_{t+1}$ conditioned on $s_t$ and $a_t$.
Often, several of these states are considered episode ending, after which the agent can no longer make any transitions or collect any more reward.
These correspond to states such as reaching the final goal, a game concluding, or falling of a cliff.
In the end, our goal is to learn a *policy* $\pi$, or a mapping from states to actions.

In an MDP, we assume that we can always tell which state $s_t$ our agents is in.
However, this isn't always the case.
Sometimes, all we have access to are observations $o_t$ that provide information the state $s_t$, but enough to precisely pinpoint the exact one.
We call such settings Partially Observable Markov Decision Processes (POMDPs).
Imagine for example a [Roomba](https://en.wikipedia.org/wiki/Roomba) being trained to navigate a living room with RL.
From its infrared and mechanical "bump" sensors, it receives partial information ($o_t$) as to where it might be, but not a definitive location ($s_t$).
Operating as a POMDP adds a whole layer of complexity to RL algorithms.
For the rest of day though, we'll focus on MDPs, as their much simpler and easier to use to teach basic concepts.

#### A simple MDP example

In the above example, we can see the 3 possible states for the agent as $s_0$, $s_1$, and $s_2$, with 2 actions $a_0$ and $a_1$ available from each state.
We can see that the each action doesn't lead to a determinstic transition to the next state, as shown by multiple paths from each action.
Note that each of the outcomes of an action are labeled with a small black number between 0 and 1.
This denotes the probability of that outcome (which state we end up at) given the the action; as these are probabilites, the sum of the probabilities of arriving at each of the next states $s_{t+1}$ given a previous state $s_t$ and selected action $a_t$ is 1.

#### Objective

The goal of the agent is to maximize the total reward $R$ it can receive over some number of steps.
It is important to ensure the reward actually captures the true goal we want the agent to achieve.
The agent will dutifully attempt maximize the objective it is given, without any considerations to any implicit objectives that a human may desire.
There are more than a few (amusing) anecdotes of RL agents learning undesirable behaviors by exploiting some aspect of the reward function.
As such, defining this reward requires special care.

One countermeasure commonly deployed by RL researchers is the concept of *discounted* rewards.
This is done with a multiplicative term $\gamma$: a reward $T$ steps in the future is discounted as $\gamma^T r_T$.
Using discounting encourages the agent to finish the task sooner rather than later, a common implicit criterion.
With discounting then, the RL agent's goal is to maximize:
\begin{align*}
\mathbb{E}[\sum_{t=0}^{\infty} \gamma^t r_t]
\end{align*}
This is far from the complete solution to making our rewards accurately capture our desired objectives, but achieving higher rewards sooner rather than later is an almost universal preference, so we almost always add it.
Designing a good reward function can be an art is highly dependent on the task.

#### Reinforcement Learning as Supervised Learning?
At first, this doesn't seem too different from the supervised methods we've looked at before, and some natural questions might arise:
- Why can't we just treat RL as a supervised task?
Why can't we use the reward (or rather, the negative of the reward) as our supervised loss?

Unlike in supervised learning, in reinforcement learning, we often don't have a pre-apportioned dataset to learn from.
In some problems set-ups, we may have examples of other agents (oftentimes humans) performing the desired task, but these aren't necessarily optimal examples of how to maximize the reward, which is what we want to learn.
In most RL settings, we don't have any examples of state-action trajectories beyond what our agent experiences through trial-and-error, which are even more suboptimal.

### Open AI Gym
Before we dive any deeper into implementing reinforcement learning models, first we need an environment.
Remember, the goal is to learn an agent that can interact with an environment in the way we want, so we need something that our agent can interact with and receive rewards from.
In robotics, this is often the real world (or some set-up in the real world).
However, it is oftentimes cheaper and quicker to first test our algorithms in simulated settings.
There are a number of tasks that are popular benchmarks for the reinforcement learning community, such as [cart pole](https://en.wikipedia.org/wiki/Inverted_pendulum), [mountain car](https://en.wikipedia.org/wiki/Mountain_car_problem), or [Atari 2600 games](https://gym.openai.com/envs/#atari).
In the spirit of accelerating progress and promoting openness in the research community, Open AI has very nicely coded up [Open AI Gym](https://gym.openai.com/), which has implementations of many of these environments for public use.
We will be using these environments, as it allows us to focus on the algorithms themselves, instead of worrying about implementing each problem setting ourselves.

To use it, we first need to download and install it. Make sure you're in your PyTorch environment first!

```Shell
# If you environment isn't currently active, activate it:
# conda activate pytorch

pip install gym
```

Once it's installed, we can import it like any other Python module:

```python
import gym
```

### FrozenLake (a Grid World)

Let's start with a simple environment: FrozenLake.
Here's the official description from OpenAI gym:

> *Winter is here. You and your friends were tossing around a frisbee at the park when you made a wild throw that left the frisbee out in the middle of the lake. The water is mostly frozen, but there are a few holes where the ice has melted. If you step into one of those holes, you'll fall into the freezing water. At this time, there's an international frisbee shortage, so it's absolutely imperative that you navigate across the lake and retrieve the disc. However, the ice is slippery, so you won't always move in the direction you intend.*

A visualization of FrozenLake as a grid world:

At the start of an episode, we begin in the upper left corner (S).
Our goal is to move ourself to the lower right corner (G), avoiding falling into the holes (H).
Icy water is cold.

In reinforcement learning terms, each of the 16 locations on the grid are a state, and an action is attempting to move in one of four directions (left, down, right, up).
Each move will result in the agent's state changing from $s_t$ to $s_{t+1}$ as it changes location, unless it attempts to move in the direction of a wall, which results in the agent's state not changing (the agent doesn't move).
We receive a positive reward of "+1" for reaching the goal (G), discounted according to how long it took.
While there is not a negative reward for falling into a hole (H), the agent still pays a penalty in the sense that falling into the hole is episode-ending and therefore prevents it from receiving any reward.
We want to learn a policy $\pi$ that takes us from our starting location (S) to the goal (G) in as few steps as possible.

To really establish what we are trying to accomplish here, it's worth debunking a few common initial misconceptions:

- **Knowledge of the states and transition probabilties:** From the top-down view, your first thought might be to plot out a path from the start to the finish, just as you would with a maze.
However, this view is provided to us the algorithm designers so we can visualize the problem at hand.
The agent learning the task does *not* get this prior knowledge; all we are about to tell it is that there are going to be 16 states and 4 possible actions from each state.
A more proper analogy would be if I blindfolded you and dropped you in the middle of a frozen lake, and told you your state (location) every time you decided to take a step in one of four directions, then set off fireworks when you stepped on the frisbee.

- **Knowledge of the goal (reward):** In OpenAI's official description of the environment, you (the agent) know what you're hoping to accomplish: You want to retrieve the frisbee, while avoiding falling through the ice.
The agent does *not* know this.
Rather, it learns the goal by experiencing rewards (or penalties), and the algorithm updates its policy such that it will be more (or less) likely to do those actions again.
Note that this means that if an agent never experiences certain rewards, it won't know they exist.

- **Prior knowledge of pathfinding, physics, etc.:** As a human, even if you haven't solved this task before, you still bring a tremondous amount of prior knowledge to this problem.
For example, you know the shortest path to a destination is a line.
You know that North, South, East, and West, are directions, and that going North and then South brings you back to where you already were.
You know ice is slippery.
You know icy water is cold.
You know being in icy cold water is bad.
It's important to keep in mind that our agent will begin knowing none of these things; it's initial policy is essentially picking actions completely at random.
By the end of the training, it still won't know what abstract concepts like "North/South," "cold," or "slippery" mean, but it will have (hopefully) learned a good policy that allows it to complete the goal.

#### Interacting with FrozenLake
This example is simple enough that we could code the environment and its interface ourselves fairly easily, but OpenAI has already done it, and we'd like to focus on the algorithm of solving it as much as possible.
We can create an instantiation of FrozenLake in a single line of code:

```python
env = gym.make('FrozenLake-v0')
```

Open AI Gym environments provide a mechanism to observe the state of the environment, and since FrozenLake is an MDP (as opposed to POMDP), the observation is the state itself.
For FrozenLake, there are 16 grid locations on the map, meaning we have 16 states.
We can confirm this by looking at the size of the `observation_space` attribute for the environment we just created.

```python
env.observation_space
```

Our agent will interact with this environment causing its state to change.
For FrozenLake, we have 4 options, each corresponding to attempting to step in a particular direction: `[Left, Down, Right, Up]`.
We can confirm this by looking at the size of the `action_space` of our environment.

```python
env.action_space
```

Before interacting with the environment, we have to first reset it to initialize it.
Resetting also returns an observation of the first state after resetting.
In FrozenLake, we always start in the upper left corner, which corresponds to state 0.
As such, we see the `reset()` command returning `0`.

```python
env.reset()
```

We can visualize the FrozenLake environment by calling `render()`.
In more complex tasks this will actually add frames to a video showing our agent's progress, but for FrozenLake, it just prints out a text representation, with the highlighted character showing our agent's current location.
We can see that we started in the upper-left corner, on the "S," as promised.

```python
env.render()
```

Now, let's try moving.
On thing to keep in mind is that the original FrozenLake environment is "slippery."
Because of the ice, if you try to go in one direction, you end up with a 1/3 chance of going in the direction you meant and the two adjacent directions each.
For example, if we try going right, we have equal probabilities of slipping and going up and down instead.
This makes things a little more complicated, so for now, let's first turn off the stochasticity and make this a deterministic transition instead.
We do this by registering a new type of environment, and then instantiating a copy of said environment, making sure to reset it first.

```python
# Non-slippery version

from gym.envs.registration import register
register(
    id='FrozenLakeNotSlippery-v0',
    entry_point='gym.envs.toy_text:FrozenLakeEnv',
    kwargs={'map_name' : '4x4', 'is_slippery': False},
)
env = gym.make('FrozenLakeNotSlippery-v0')
env.reset()
```

We advance time in an OpenAI environment with the `step()` method, which takes as argument an `action`.
Let's trying moving right, which corresponds to action `2`.
Notice that the output is a tuple of four elements: the next observation (`object`), the reward (`float`), whether or not the episode is done (`boolean`), and a dictionary of information (`dict`) that may be useful for debugging (this dict shouldn't be used in the final algorithm itself).

```python
env.step(2)
```

Next, let's `render()` to visualize what happened.
Observe that this particular environment prints out the action we took in parentheses up top, in this case "(Right)", and then shows the result of that action.
Notice that while most of the time, we succeed in going in the direction we want to, occasionally we slip on the ice and go in a direction we didn't intend.

```python
env.render()
```

We can keep doing this as many times as we want.
Since we're in Jupyter, we can just keep running the same cell (making small edits to change our action).

Notice that once we fall into a hole, the episode is over, and we can no longer do anything.
The same is true after reaching the goal.

```python
env.step(0)
env.render()
```

Before we get into any RL, let's see how random actions perform in this environment:

```python
env.reset()
done = False

while not done:
    env.render()
    action = env.action_space.sample()
    _, _, done, _ = env.step(action)
```

Hm.
Not great.
Alright, so clearly picking random steps isn't very likely to take us to the goal.
It's apparent just from looking at the map that there're much better policies that we can learn.
How are we going to do so?

#### Q-learning
There are many algorithms that we can use, but let's choose Q-learning, which we covered earlier today.
Remember, in Q-learning (and SARSA, it turns out), we're trying learn the Q values for the states in our system.

The Q value for a policy $\pi$ is a function of the state $s$ and action $a$ and is defined as the following:
\begin{equation}
Q_\pi (s,a) = \mathbb{E}\big[\sum_{t=0}^{\infty} \gamma^t r_t \big|\pi, s_0 = s, a_0 = a\big]
\end{equation}
Intuitively, the Q value is the total reward (including discounting) that the agent will gain if it takes action $a$ from state $s$ and then follows policy $\pi$ for the rest of the episode.
As one might expect, if Q is known exactly, the agent will attain the highest reward from $s$ if the policy $\pi$ is to pick the $a$ with the highest Q value.

Okay, so if we know the Q values for the system, then we can trivially find the optimal policy.
So what are the Q values of the system?
Well, at the beginning, we don't know, but we can try to learn them through experience.
This is where Q-learning comes in.
Q-learning iteratively updates the Q values in the following way:
\begin{equation}
Q_\pi (s_t, a_t) \leftarrow (1 - \alpha) \cdot Q_\pi(s_t, a_t) + \alpha \cdot \big(r_t + \gamma \max_a Q_\pi(s_{t+1}, a)\big)
\end{equation}
Notice that Q-learning is an *off-policy* method, in the sense that you don't actually learn from the trajectory you actually took (otherwise it'd be SARSA).
Instead, we learn from the *greedy* transition, i.e. the best action we know how to take.

And that's it!
We run our agent through many episodes, experiencing many $s_t \rightarrow a_t \rightarrow s_{t+1}$ transitions and rewards, and just like that, we eventually learn a good Q function (and thus a good policy).
Now of course, there are a bunch of small details and tweaks to make this work in practice, but we'll get to those later.

#### Q-learning in FrozenLake
FrozenLake is a very simple setting, one that we would call a toy problem.
With only 16 states and 4 actions, there are only 64 state-action pairs possible (16x4=64), less if we account for the goal and the holes being episode ending (for simplicity though, we won't).
With this few state-action pairs, we can actually solve this problem tabularly.
Let's set up a Q table, and initialize the Q-values for all state-action pairs to zeros.
Note that while we could, we're actually not going to need PyTorch in this example; PyTorch's autograd and neural network libraries are unnecessary here, as we're only going to be modifying a table of numbers.
Instead, we'll use a Numpy array to store the Q table.

```python
import numpy as np

#Initialize table with all zeros to be uniform
Q = np.zeros([env.observation_space.n, env.action_space.n])
```

A few hyperparameters we're going to set:
- `alpha`: learning rate for the Q function
- `gamma`: discount rate for future rewards
- `num_episodes`: number of episodes (trajectories from start to goal/hole) our agent will learn from

We're also going to store our rewards in an array called `rs`.

```python
# Learning parameters
alpha = 0.1
gamma = 0.95
num_episodes = 2000

# array of reward for each episode
rs = np.zeros([num_episodes])
```

Now for the bulk of algorithm itself.
Notice that we're going to loop through the process `num_episodes` times, resetting the environment each time.
At each step, we take the action with the highest Q value for our current state, with some randomness added in (especially at the beginning) to encourage exploration.
After each action, we update our Q table greedily based on the reward experienced and the next best action.
We also make sure to update our state, rinse, and repeat.
We continue taking actions in an episode until it is `done`, storing the final total reward for the episode.

```python
for i in range(num_episodes):
    # Set total reward and time to zero, done to False
    r_sum_i = 0
    t = 0
    done = False

    #Reset environment and get first new observation
    s = env.reset()

    while not done:
        # Choose an action by greedily (with noise) from Q table
        a = np.argmax(Q[s,:] + np.random.randn(1, env.action_space.n)*(1./(i/10+1)))

        # Get new state and reward from environment
        s1, r, done, _ = env.step(a)

        # Update Q-Table with new knowledge
        Q[s,a] = (1 - alpha)*Q[s,a] + alpha*(r + gamma*np.max(Q[s1,:]))

        # Add reward to episode total
        r_sum_i += r*gamma**t

        # Update state and time
        s = s1
        t += 1
    rs[i] = r_sum_i
```

How did we do?
Let's take a look at the rewards that we saved.
We can plot the reward versus the episode number, and hopefully we'll see some sort of an increase over time.
RL performance can be extremely noisy, so let's instead plot a moving average.

```python
## Plot reward vs episodes
import matplotlib.pyplot as plt

# Sliding window average
r_cumsum = np.cumsum(np.insert(rs, 0, 0))
r_cumsum = (r_cumsum[50:] - r_cumsum[:-50]) / 50

# Plot
plt.plot(r_cumsum)
plt.show()
```

Pretty good.
We might also be interested in how often our agent actually reached the goal.
This won't account for how quickly the agent got there (which might also of interest), but let's ignore that for now.
To prevent us from being overwhelmed by data points, let's bucket the values into 10 stages, printing out how many episodes of each stage resulted in finding the goal.

```python
# Print number of times the goal was reached
N = len(rs)//10
num_Gs = np.zeros(10)

for i in range(10):
    num_Gs[i] = np.sum(rs[i*N:(i+1)*N] > 0)

print("Rewards: {0}".format(num_Gs))
```

Our RL agent does a really good job at navigating the FrozenLake when its moves are deterministic, but after all, this is
supposed to be *Frozen*Lake, so where's the fun without the slipperiness?
Let's go back to the original environment and see how the agent does.

```python
env = gym.make('FrozenLake-v0')

#Initialize table with all zeros to be uniform
Q = np.zeros([env.observation_space.n, env.action_space.n])

# Learning parameters
alpha = 0.1
gamma = 0.95
num_episodes = 2000

# array of reward for each episode
rs = np.zeros([num_episodes])

for i in range(num_episodes):
    # Set total reward and time to zero, done to False
    r_sum_i = 0
    t = 0
    done = False

    #Reset environment and get first new observation
    s = env.reset()

    while not done:
        # Choose an action by greedily (with noise) from Q table
        a = np.argmax(Q[s,:] + np.random.randn(1, env.action_space.n)*(1./(i/10+1)))

        # Get new state and reward from environment
        s1, r, done, _ = env.step(a)

        # Update Q-Table with new knowledge
        Q[s,a] = (1 - alpha)*Q[s,a] + alpha*(r + gamma*np.max(Q[s1,:]))

        # Add reward to episode total
        r_sum_i += r*gamma**t

        # Update state and time
        s = s1
        t += 1
    rs[i] = r_sum_i

## Plot reward vs episodes
# Sliding window average
r_cumsum = np.cumsum(np.insert(rs, 0, 0))
r_cumsum = (r_cumsum[50:] - r_cumsum[:-50]) / 50

# Plot
plt.plot(r_cumsum)
plt.show()

# Print number of times the goal was reached
N = len(rs)//10
num_Gs = np.zeros(10)

for i in range(10):
    num_Gs[i] = np.sum(rs[i*N:(i+1)*N] > 0)

print("Rewards: {0}".format(num_Gs))
```

Much harder.
However, we can see that the model does eventually learn something.

### PyTorch in RL
Hey, not bad.
However, while the previous example was fun and simple, it was noticeably lacking any hint of PyTorch.

We could have used a PyTorch `Tensor` to store the Q table, but that's not any better than using a NumPy array.
PyTorch's true utility comes from building neural networks and calculating/applying gradients automatically, which learning the Q table didn't need.

#### Continuous domains
In our previous example, we mentioned that with only 16 discrete states and 4 actions/state, the Q table only needed to hold 64 values, which is very manageable.
However, what if the state or action space is continuous?
You could discretize it, but then you have to pick a resolution, and your state-action space could explode exponentially.
Treating these binned states or actions as completely different states is also ignoring that two consecutive bins are likely very similar in the needed policy.
You can learn these relationships, but doing so is horriby sample inefficient.

Instead of learning a Q table then, perhaps a Q function would be more appropriate.
This function would take in a state and action as an input and return a Q value as an output.
The Q function may be very complex, but as we've learned over the past few days, neural networks are very flexible and good for approximating arbitrary functions.
[Deep Q Networks](https://deepmind.com/research/dqn/) take such an approach.

### Cart Pole

Let's look at the cart pole problem next.
In this setting, we have a pole attached to a hinge on a cart, with the goal being to keep the pole vertical as long as possible, without traveling too far along the rail.
Because of gravity, the pole will fall unless the cart is exactly beneath the the pole's center of gravity.
To prevent the pole from falling, the agent can apply a force of +1 or -1 to the cart to move it left and right along a track.
The agent receives a reward of +1 for every timestamp the pole remains vertical; the game ends when the pole fall past 15 degrees from vertical or the cart moves more than 2.4 units away from the center.
We're going to somewhat arbitrarily call "success" achieving a reward of +200; alternatively, the agent needs to avoid the aforementioned failure conditions for 200 ticks.

First, let's create an instance of the cart pole environment:

```python
env = gym.make('CartPole-v0')
```

Again, we can look at the `observation_space` for this environment.
Also similar to FrozenLake, since this version of cart pole is an MDP (as opposed to POMDP), the observation is the state itself.
We can see that the states for cart pole have 4 dimensions, which correspond to `[cart position, cart velocity, pole angle, pole angular velocity]`.
Importantly, notice these states are *continuous* values.

```python
env.observation_space
```

We can look at the `action_space` again too.
In cart pole, there are two actions available to the agent: `[apply force left, apply force right]`.
We can see this by examining the `action_space` attribute:

```python
env.action_space
```

Resetting the environment returns our first observations, which we can see has 4 values corresponding to the 4 previously mentioned state variables.

```python
env.reset()
```

Before we get into any reinforcement learning, let's see how we perform actions within the environment.

```python
done = False

while not done:
    env.render()
    action = env.action_space.sample()
    _, _, done, _ = env.step(action)
```

Okay, so clearly choosing a random action at every time step doesn't really achieve our goal of keeping the pole vertical.
We're going to need something smarter.

Let's close that rendering window.
We do this with `close()`.
Note that `gym` renderings can be a little finicky, especially on Windows; either `close()` or restarting your Jupyter kernel may be necessary to close the rendered window.

```python
env.close()
```

Cart pole is actually a fairly simple problem (it's very low dimensional), and so there are simpler ways to do this, but since we've been having so much fun with deep learning, let's use a neural network.
Specifically, let's build a DQN that uses Q-learning to learn how to balance the pole.
We're going to give our DQN agent 1000 episodes to try and reach the goal of 200 ticks.

There are a lot of small details that go into making these models work well, so instead of going through it piece by piece, the full code:

```python
# Based on: https://gym.openai.com/evaluations/eval_EIcM1ZBnQW2LBaFN6FY65g/

from collections import deque
import random
import math

import gym
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

class DQN(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 24)
        self.fc2 = nn.Linear(24, 48)
        self.fc3 = nn.Linear(48, 2)

    def forward(self, x):
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        x = F.relu(x)
        x = self.fc3(x)
        return x

class DQNCartPoleSolver:
    def __init__(self, n_episodes=1000, n_win_ticks=195, max_env_steps=None, gamma=1.0, epsilon=1.0, epsilon_min=0.01, epsilon_log_decay=0.995, alpha=0.01, alpha_decay=0.01, batch_size=64, monitor=False, quiet=False):
        self.memory = deque(maxlen=100000)
        self.env = gym.make('CartPole-v0')
        if monitor: self.env = gym.wrappers.Monitor(self.env, '../data/cartpole-1', force=True)
        self.gamma = gamma
        self.epsilon = epsilon
        self.epsilon_min = epsilon_min
        self.epsilon_decay = epsilon_log_decay
        self.alpha = alpha
        self.alpha_decay = alpha_decay
        self.n_episodes = n_episodes
        self.n_win_ticks = n_win_ticks
        self.batch_size = batch_size
        self.quiet = quiet
        if max_env_steps is not None: self.env._max_episode_steps = max_env_steps

        # Init model
        self.dqn = DQN()
        self.criterion = torch.nn.MSELoss()
        self.opt = torch.optim.Adam(self.dqn.parameters(), lr=0.01)

    def get_epsilon(self, t):
        return max(self.epsilon_min, min(self.epsilon, 1.0 - math.log10((t + 1) * self.epsilon_decay)))

    def preprocess_state(self, state):
        return torch.tensor(np.reshape(state, [1, 4]), dtype=torch.float32)

    def choose_action(self, state, epsilon):
        if (np.random.random() <= epsilon):
            return self.env.action_space.sample()
        else:
            with torch.no_grad():
                return torch.argmax(self.dqn(state)).numpy()

    def remember(self, state, action, reward, next_state, done):
        reward = torch.tensor(reward)
        self.memory.append((state, action, reward, next_state, done))

    def replay(self, batch_size):
        y_batch, y_target_batch = [], []
        minibatch = random.sample(self.memory, min(len(self.memory), batch_size))
        for state, action, reward, next_state, done in minibatch:
            y = self.dqn(state)
            y_target = y.clone().detach()
            with torch.no_grad():
                y_target[0][action] = reward if done else reward + self.gamma * torch.max(self.dqn(next_state)[0])
            y_batch.append(y[0])
            y_target_batch.append(y_target[0])

        y_batch = torch.cat(y_batch)
        y_target_batch = torch.cat(y_target_batch)

        self.opt.zero_grad()
        loss = self.criterion(y_batch, y_target_batch)
        loss.backward()
        self.opt.step()

        if self.epsilon > self.epsilon_min:
            self.epsilon *= self.epsilon_decay

    def run(self):
        scores = deque(maxlen=100)

        for e in range(self.n_episodes):
            state = self.preprocess_state(self.env.reset())
            done = False
            i = 0
            while not done:
                if e % 100 == 0 and not self.quiet:
                    self.env.render()
                action = self.choose_action(state, self.get_epsilon(e))
                next_state, reward, done, _ = self.env.step(action)
                next_state = self.preprocess_state(next_state)
                self.remember(state, action, reward, next_state, done)
                state = next_state
                i += 1

            scores.append(i)
            mean_score = np.mean(scores)
            if mean_score >= self.n_win_ticks and e >= 100:
                if not self.quiet: print('Ran {} episodes. Solved after {} trials ✔'.format(e, e - 100))
                return e - 100
            if e % 100 == 0 and not self.quiet:
                print('[Episode {}] - Mean survival time over last 100 episodes was {} ticks.'.format(e, mean_score))

            self.replay(self.batch_size)

        if not self.quiet: print('Did not solve after {} episodes 😞'.format(e))
        return e

if __name__ == '__main__':
    agent = DQNCartPoleSolver()
    agent.run()
    agent.env.close()
```

Reinforcement learning can be kind of noisy.
In some sense, it depends on your agent "lucking" into the right behavior so that it can learn from it, and occasionally one can get stuck in a bad rut.
Even if your agent fails to "solve" the problem (i.e. reach 200 ticks), you should still see the mean survival time mostly climbing as the agent experiences more episodes.
You may need to re-run learning a couple times for the agent to reach 200 ticks.

Want to take this farther and connect this with the convolutional neural networks we've previously learned about?
See [here](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html) for a version of Cartpole where instead of directly measuring the translation/rotational position/velocity, the agent only sees the rendered screen.
This makes the problem much harder, as the agent needs to learn to infer translation/rotational position/velocity from images.

### Other materials:
Reinforcement Learning can easily be a full course (or multiple) on its own at most universities, with or without deep learning.
We only touch on a couple points here.
If you're interested in *exploring* this field further, a few recommendations to get you started:

- [The definitive textbook on Reinforcement Learning](http://incompleteideas.net/book/the-book-2nd.html)
- [Video lectures by David Silver](https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ)
- [Deep Q-Networks (DQN) for Atari 2600 games](https://deepmind.com/research/dqn/)
- [Popular blog post on using policy gradients to learn to play Pong from pixels](http://karpathy.github.io/2016/05/31/rl/)


# Mid section coefficient $\sigma$ statistics

# Purpose
We have found that the section coefficients cannot exceed 0.99 in ikedas eddy damping prediction. This notebook investigates how commonly the mid section area coefficient exceeds 0.99.

# Methodology
Load mid section coefficients from the MDL DB

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import rolldecayestimators.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables
import shipflowmotionshelpers.shipflowmotionshelpers as helpers

```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 461 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.dpi       : 100        ## figure dots per inch')

```python
sql = """
select * from loading_conditions
	inner join models on loading_conditions.model_number=models.model_number
		inner join ships on models.ship_name=ships.name
        	inner join association on association.ship_name=ships.name
            	inner join projects on projects.project_number=association.project_number

"""
```

```python
db = database.get_db()
loading_conditions = pd.read_sql(sql=sql, con=db.engine, index_col='id')
```

```python
loading_conditions.head()
```

      name
      model_number
      lcg
      kg
      gm
      CW
      TF
      TA
      BWL
      KXX
      KZZ
      BTT1
      CP
      Volume
      A0
      RH
      model_number
      ship_name
      scale_factor
      name
      lpp
      beam
      ABULB
      BKX
      TWIN
      DCLR
      VDES
      RHBL
      ASKEG
      PD
      ARH
      CFP
      AIX
      PDTDES
      RTYPE
      SFP
      BKL
      BKB
      PROT
      D
      LSKEG
      RR
      XSKEG
      NDES
      AR
      BR
      BRA
      IRUD
      PTYPE
      XRUD
      AI
      HSKEG
      RSKEG
      LOA
      ship_type_id
      project_number
      ship_name
      project_number
      name
      project_path

      id

      1
      8.57
      M3522-A
      -1.63
      10.16
      1.25
      0.833
      8.57
      8.57
      25.9
      9.01
      41.04
      2.547
      0.7253
      26622.0
      0.9862
      5.8
      M3522-A
      M3522-A
      55.00
      M3522-A
      167.68
      25.9
      35.0
      0.0
      1.0
      5.74
      20.70
      6.05
      144.0
      1.2000
      0.0
      None
      None
      13500.0
      3.0
      NaN
      58.70
      1.05
      3.0
      5.4
      46.0
      2.32
      -57.0
      103.0
      18.4
      0.0
      0.0
      2.0
      1.0
      -83.84
      None
      6.2
      4.5
      181.45
      8.0
      30126505
      M3522-A
      30126505
      30126505 BMT Naval Supply vessel
      S:\2012\30126505 BMT Naval Supply vessel

      2
      6.94
      M3522-A
      -0.04
      9.67
      2.09
      0.786
      6.94
      6.94
      25.9
      10.09
      44.22
      1.968
      0.7005
      20752.0
      0.9830
      5.8
      M3522-A
      M3522-A
      55.00
      M3522-A
      167.68
      25.9
      35.0
      0.0
      1.0
      5.74
      20.70
      6.05
      144.0
      1.2000
      0.0
      None
      None
      13500.0
      3.0
      NaN
      58.70
      1.05
      3.0
      5.4
      46.0
      2.32
      -57.0
      103.0
      18.4
      0.0
      0.0
      2.0
      1.0
      -83.84
      None
      6.2
      4.5
      181.45
      8.0
      30126505
      M3522-A
      30126505
      30126505 BMT Naval Supply vessel
      S:\2012\30126505 BMT Naval Supply vessel

      3
      12.0
      3161-A
      -2.45
      18.84
      4.82
      0.858
      12.00
      12.00
      50.0
      16.50
      71.30
      3.180
      0.7714
      139566.0
      0.9960
      10.2
      3161-A
      3161-A
      53.74
      3161-A
      302.00
      50.0
      49.5
      -4.1
      1.0
      9.80
      20.35
      10.45
      218.0
      0.9918
      12.8
      None
      None
      28034.0
      1.0
      NaN
      80.94
      0.40
      3.0
      7.9
      49.0
      2.32
      -125.0
      79.5
      68.6
      0.0
      0.0
      2.0
      1.0
      -151.00
      None
      9.3
      36.0
      314.00
      4.0
      20053838
      3161-A
      20053838
      20053838
      \\sspa.local\gbg\ProjektArkiv\2005\20053838

      4
      9.6
      3161-A
      -1.02
      14.93
      11.09
      0.828
      9.60
      9.60
      50.0
      20.00
      77.70
      2.800
      0.7532
      108940.0
      0.9950
      10.2
      3161-A
      3161-A
      53.74
      3161-A
      302.00
      50.0
      49.5
      -4.1
      1.0
      9.80
      20.35
      10.45
      218.0
      0.9918
      12.8
      None
      None
      28034.0
      1.0
      NaN
      80.94
      0.40
      3.0
      7.9
      49.0
      2.32
      -125.0
      79.5
      68.6
      0.0
      0.0
      2.0
      1.0
      -151.00
      None
      9.3
      36.0
      314.00
      4.0
      20053838
      3161-A
      20053838
      20053838
      \\sspa.local\gbg\ProjektArkiv\2005\20053838

      5
      7.0
      3166
      -3.85
      10.00
      11.10
      NaN
      7.00
      7.00
      NaN
      NaN
      NaN
      NaN
      NaN
      12412.0
      NaN
      NaN
      3166
      3166
      27.93
      3166
      94.30
      40.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      None
      None
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      None
      NaN
      NaN
      NaN
      NaN
      20054025
      3166
      20054025
      20054025 RAMFORM100
      \\sspa.local\gbg\ProjektArkiv\2005\20054025 RA...

```python
loading_conditions.describe()
```

      lcg
      kg
      gm
      CW
      TF
      TA
      BWL
      KXX
      KZZ
      BTT1
      CP
      Volume
      A0
      RH
      scale_factor
      lpp
      beam
      ABULB
      BKX
      TWIN
      DCLR
      VDES
      RHBL
      ASKEG
      PD
      ARH
      PDTDES
      RTYPE
      SFP
      BKL
      BKB
      PROT
      D
      LSKEG
      RR
      XSKEG
      NDES
      AR
      BR
      BRA
      IRUD
      PTYPE
      XRUD
      HSKEG
      RSKEG
      LOA
      ship_type_id
      project_number
      project_number

      count
      166.000000
      166.000000
      166.000000
      99.000000
      166.000000
      166.000000
      104.000000
      105.000000
      105.000000
      76.000000
      99.000000
      166.000000
      103.000000
      97.000000
      154.000000
      158.000000
      166.000000
      83.000000
      106.000000
      110.000000
      83.000000
      96.000000
      83.000000
      83.000000
      98.000000
      81.000000
      87.000000
      107.000000
      2.0
      113.000000
      108.000000
      81.000000
      105.000000
      83.000000
      83.000000
      83.000000
      83.000000
      110.000000
      83.0
      83.0
      108.000000
      106.000000
      83.000000
      83.000000
      83.000000
      110.000000
      70.000000
      1.660000e+02
      1.660000e+02

      mean
      3.287681
      11.234771
      6.441651
      0.853576
      9.139735
      9.383952
      42.642212
      15.486857
      59.486476
      2.344487
      1.516948
      82391.674699
      0.981872
      9.598660
      48.280990
      218.084633
      37.184078
      38.703614
      3.491368
      0.427273
      4.407590
      17.728646
      10.476145
      132.187952
      0.856557
      11.250741
      16964.885057
      1.532710
      49.5
      66.332655
      0.357398
      2.012346
      7.350667
      28.448193
      2.518867
      -112.077108
      90.789157
      60.755273
      0.0
      0.0
      1.518519
      1.066038
      -129.569759
      7.921205
      15.925904
      253.604818
      4.157143
      2.313056e+07
      2.313056e+07

      std
      16.494796
      5.951431
      10.133628
      0.057927
      5.463688
      5.428838
      12.361371
      4.521961
      16.517887
      0.918263
      7.460971
      84059.329612
      0.048530
      3.348669
      13.737312
      89.336724
      16.537831
      27.484867
      11.239162
      0.496946
      4.449252
      2.717388
      3.109978
      55.694542
      0.128879
      9.831539
      7489.787480
      1.290881
      0.0
      30.997376
      0.171081
      1.006154
      1.728519
      7.249178
      0.905185
      25.442423
      34.791469
      38.307828
      0.0
      0.0
      0.587748
      0.483418
      29.589814
      1.558740
      13.412493
      69.839454
      2.470907
      1.023319e+07
      1.023319e+07

      min
      -12.570000
      0.000000
      0.000000
      0.560000
      0.000000
      0.000000
      9.690000
      4.360000
      10.900000
      0.340000
      0.475100
      0.000000
      0.618000
      0.000000
      10.000000
      0.000000
      0.000000
      0.000000
      -29.300000
      0.000000
      0.000000
      10.000000
      0.000000
      22.000000
      0.671000
      0.000000
      1900.000000
      0.000000
      49.5
      0.000000
      0.000000
      1.000000
      2.200000
      13.500000
      2.320000
      -151.000000
      65.100000
      0.000000
      0.0
      0.0
      0.000000
      1.000000
      -174.000000
      2.900000
      0.000000
      50.200000
      1.000000
      2.005384e+07
      2.005384e+07

      25%
      -1.611000
      8.191250
      1.400000
      0.833000
      5.995000
      6.314500
      37.250000
      13.980000
      52.300000
      1.937500
      0.746150
      10399.000000
      0.987250
      7.700000
      43.575000
      169.260000
      30.550000
      19.000000
      -0.090000
      0.000000
      0.000000
      15.600000
      9.200000
      108.000000
      0.720000
      0.000000
      10348.000000
      1.000000
      49.5
      54.570000
      0.300000
      1.000000
      6.600000
      24.000000
      2.320000
      -125.000000
      73.000000
      37.820000
      0.0
      0.0
      1.000000
      1.000000
      -144.000000
      6.800000
      7.450000
      221.250000
      3.250000
      2.009537e+07
      2.009537e+07

      50%
      0.000000
      12.140000
      3.755000
      0.851000
      9.400000
      9.400000
      45.600000
      15.500000
      64.100000
      2.349500
      0.770000
      70747.500000
      0.993000
      9.600000
      50.340000
      242.000000
      43.200000
      43.100000
      0.150000
      0.000000
      5.400000
      17.600000
      10.000000
      130.000000
      0.870000
      10.300000
      18820.000000
      1.000000
      49.5
      78.660000
      0.400000
      3.000000
      7.600000
      28.000000
      2.320000
      -122.000000
      79.000000
      51.200000
      0.0
      0.0
      2.000000
      1.000000
      -139.500000
      7.800000
      11.100000
      288.250000
      4.000000
      2.012643e+07
      2.012643e+07

      75%
      4.765000
      16.382500
      7.581250
      0.898000
      11.900000
      11.900000
      47.000000
      18.200000
      70.500000
      2.835000
      0.809400
      117188.250000
      0.996000
      10.800000
      55.000000
      280.400000
      46.950000
      49.000000
      9.412500
      1.000000
      8.900000
      20.000000
      11.250000
      150.000000
      0.928000
      15.900000
      22680.000000
      1.000000
      49.5
      88.000000
      0.430000
      3.000000
      8.300000
      31.500000
      2.320000
      -99.000000
      97.100000
      74.275000
      0.0
      0.0
      2.000000
      1.000000
      -112.600000
      8.580000
      22.000000
      295.000000
      4.000000
      2.015766e+07
      2.015766e+07

      max
      140.030000
      22.620000
      77.420000
      0.950000
      23.000000
      23.000000
      70.000000
      24.050000
      87.000000
      4.920000
      75.000000
      425350.000000
      0.999000
      16.000000
      72.860000
      348.000000
      70.000000
      113.000000
      36.000000
      1.000000
      10.250000
      25.000000
      16.400000
      290.000000
      1.200000
      46.800000
      28034.000000
      6.000000
      49.5
      117.600000
      1.050000
      3.000000
      10.200000
      49.000000
      10.000000
      -26.500000
      286.800000
      158.330000
      0.0
      0.0
      3.000000
      5.000000
      0.000000
      11.200000
      58.000000
      363.500000
      11.000000
      9.003216e+07
      9.003216e+07

```python
loading_conditions.hist('A0', bins=100)
```

```python
fig,ax=plt.subplots()
loading_conditions.hist('A0', bins=300, ax=ax)
ax.set_xlim((0.98,1))
ax.annotate('limit', xy=(0.99,0), xytext=(0.99,1), arrowprops=dict(facecolor='black', shrink=0.05))
ax.set_title('Mid section coefficient')
```

```python

```