# Exercise Sheet 01: Concept Learning

## Introduction

This is the first exercise sheet. The homework sheets will usually be available at the beginning of the week and are supposed to be solved in groups of three. They have to be handed in before Monday morning of the following week. The exercises are then presented to your tutor in a small feedback session. To acquire the admission for the final exam, you will have to pass $N-2$ of the weekly provided exercise sheets.

Sign up for a group on Stud.IP (See `Participants` -> `Functions/Groups`). The times mentioned there are the times for the feedback session of your group. If none of them fits, send any of the tutors an e-mail so we can try to arrange something.

Your group will have a group folder in Stud.IP under `Documents`. Upload your solutions there to hand them in.

All exercise sheets will use [Jupyter Notebooks](http://jupyter-notebook.readthedocs.org/en/latest/notebook.html). To be able to run these on your system, you will need to install Python and a few packages. We suggest the newest version of Python 3 and installing the conda environment as explained in the practice session.

￼This week's sheet should be solved and handed in before the end of **Sunday, April 15, 2018**.
Please upload your results to your group's Stud.IP folder. In case you cannot do this first sheet (due to technical or organizational problems) please upload a description of your problem instead. Your tutor will help you to solve the problems in the first feedback session and you may hand in this sheet together with the second sheet one week later.

## Assignment 1: Candidate Elimination (by Hand) [6 Points]

Candidate Elimination is a learning algorithm that, in each step, tries to generate a description which is consistent with all previously observed examples in a training set. That description could hypothetically then be used to classify examples outside the training set.

Consider the following situation:

Earl and Fran have made it their mission to visit as many amusement parks as possible in the coming summer term. However, to maximize their enjoyment and not have any unnecessary arguments break out, they make a list of previous park visits and if they would go there again, to have a few criteria to decide if a park is worth their time.

This is the set of attributes along with their possible values Earl and Fran came up with:

| Attribute           | driving distance | ticket price      | rollercoasters | dinosaurs |
|---------------------|------------------|-------------------|----------------|-----------|
| **Possible Values** | short / far      | cheap / expensive | many / none    | yes / no  |

This is Earl and Fran's accumulated data from previous visits. The list will allow you to come to a learning decision which properties have to be fulfilled such that the two will enjoy a visit to an amusement park.

| Sample No. | driving distance | ticket price | rollercoasters | dinosaurs | go again? |
|------------|------------------|--------------|----------------|-----------|-----------|
| 1          | far              | cheap        | many           | no        | yes       |
| 2          | short            | expensive    | many           | no        | yes       |
| 3          | far              | expensive    | none           | yes       | no        |
| 4          | short            | cheap        | none           | yes       | no        |
| 5          | short            | cheap        | many           | yes       | yes       |

**a)** Apply Candidate Elimination to the samples 1-5 below and provide the version space boundaries $S_n$ and $G_n$ after each new training sample.

\begin{align}
S_0 =& ~(<0,0,0,0>) \\
G_0 =& ~(<?,?,?,?>) \\[3mm]
S_1 =& ~(<far,cheap,many,no>)\\
G_1 =& ~(<?,?,?,?>) \\[3mm]
S_2 =& ~(<?,?,many,no>) \\
G_2 =& ~(<?,?,?,?>) \\[3mm]
S_3 =& ~(<?,?,many,no>) \\
G_3 =& ~(<?,?,many,?>,<?,?,?,no>) \\[3mm]
S_4 =& ~(<?,?,many,no>) \\
G_4 =& ~(<?,?,many,?>,<?,?,?,no>) \\[3mm]
S_5 =& ~(<?,?,many,?>) \\
G_5 =& ~(<?,?,many,?>) \\
\end{align}

** b) ** Provide the complete version space bounded by $S_2$ and $G_2$.

$$S = (<?,?,many,no>)$$

$$(<?,?,many,?> \quad <?,?,?,no>)$$

$$G = (<?,?,?,?>)$$

** c) ** To what kind of amusement park should Earl and Fran go?

They should go to an amusement park with many rollercoasters or no dinosours (not exclusive or) $\Rightarrow (\neg \text{dinosours} \lor \text{many rollercoasters})$

## Assignment 2: Candidate Elimination (in Python) [10 Points]

Now let's get to real fun part: Programming our first machine learning algorithm!

But first some general remarks on coding style:

In general, try to write code that's consistent with [PEP8](https://www.python.org/dev/peps/pep-0008/), the offical Python Style Guide. Have a look a the [Google Style Guide](https://google.github.io/styleguide/pyguide.html) as well. It's based on PEP8 and has some nice examples. As a bonus always document your function and classes with docstrings. It is best practice to stick to some docstring convention for example [Google's](http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html). This has the big advantages that it allows for automated generation of documentation. We will follow that convention in the code we provide you, so you know what kind of objects to expect and what to return.

If, during the programming tasks, you run into a `NameError`, make sure that you executed all prior code cells beforehand. Later cells might rely on variables and function from prior cells. To see all currently defined variables you can make use of the `%whos` [magic function](https://github.com/lmmx/devnotes/wiki/IPython-'magic'-function-documentation#whos) anywhere in code cells. Additionally, it is sometimes handy to run all cells from the beginning by opeining the command palette with `Ctrl + p` and typing `run all cells`. Moreover, using `b` to add new cells below and `a` for adding new cells above your current cell will make your life often easier. Finally, using `l` to show line numbers is helpful for locating errors from error messages.

In the following Python code we have provided the building blocks for the `CANDIDATE-ELIMINATION` algorithm and put them together in a single function. Now you have to fill those building blocks with actual code. There are places marked with

```python
# YOUR CODE HERE
```

where you have to add some code to make the code work. Finish the code to automate the decision making for Earl and Fran!

```python
def is_consistent(h, datum):
    """
    Checks if a general hypothesis is consistent with a datum.

    Args:
        h (tuple): Hypothesis.
        datum (dict): Datum with values and target.

    Returns:
        bool: Whether the hypothesis correctly predicts the target value.
    """
    if 'values' not in datum or 'target' not in datum or len(h) != len(datum['values']):
        raise Exception("Hypothesis or datum is invalid")

    return datum['target'] != any(x != y and x != '?' for x, y in zip(h,datum['values']))

assert is_consistent(('far', 'cheap', '?', 'no'), {'values': ('far', 'cheap', 'many', 'no' ), 'target': True })
assert is_consistent(('far', 'cheap', '?', 'no'), {'values': ('far', 'expensive', '?', '?'), 'target': False})
```

```python
def is_more_general_or_equal(h1, h2):
    """
    Checks whether hypothesis h1 is more general than hypothesis h2 or equally general.

    Args:
        h1 (tuple): Hypothesis 1.
        h1 (tuple): Hypothesis 2.

    Returns:
        bool: True if the predicate is satisfied.
    """
    if len(h1) != len(h2):
        raise Exception("Hypothesis 1 and 2 have differing lengths")

    return not any(x != y and x != '?' for x, y in zip(h1,h2))

assert is_more_general_or_equal(('?', '?', '?', '?'), ('far', 'cheap', 'many', 'no'))
```

```python
def is_more_specific_or_equal(h1, h2):
    """
    Checks whether hypothesis h1 is more specific than hypothesis h2 or equally specific.

    Args:
        h1 (tuple): Hypothesis 1.
        h1 (tuple): Hypothesis 2.

    Returns:
        bool: True if the predicate is satisfied.
    """
    if len(h1) != len(h2):
        raise Exception("Hypothesis 1 and 2 have differing lengths")

    return not any(x != y and (x == '?' or y == '0') for x, y in zip(h1,h2))

assert is_more_specific_or_equal(('0', '0', '0', '0'), ('?', '?', 'many', '?'))
assert is_more_specific_or_equal(('?', '?', 'many', 'no'), ('?', '?', 'many', '?'))
```

```python
def generalize_minimally(h, datum):
    """Generalize hypothesis h so it becomes consistent with the datum.

    Args:
        h (tuple): The hypothesis to be generalized.
        datum (tuple): Attribute values of a datum. The datum is assumed to have a positive target value.

    Returns:
        tuple: The generalized hypothesis.
    """
    if len(h) != len(datum):
        raise Exception("Hypothesis and datum have differing lengths")

    h2 = list(h)

    for index in range(len(h)):
        # if h[i] is not more general than datum[i]
        if datum[index] != h[index] and h[index] != '?':
            if h[index] == '0':
                h2[index] = datum[index]
            else:
                h2[index] = '?'

    return tuple(h2)

assert generalize_minimally(('?', '?', 'many', 'no'), ('short', 'cheap', 'many', 'yes')) == ('?', '?', 'many', '?')
```

```python
def specialize_minimally(h, datum, attr_values):
    """
    Generate all consistent minimal specialization of hypothesis h.

    Args:
        h (tuple): The hypothesis to be specialized.
        datum (tuple): Attribute values of a datum. The datum is assumed to have a negative target value.
        attr_values (tuple of tuples): All possible attribute values for each attribute.

    Returns:
        tuple of tuples: Tuple of the specialized hypotheses.
    """
    specials = []

    for index in range(len(datum)):
        if datum[index] != "?" and h[index] == "?" or \
           h[index] == '0' and datum[index] != '0':
            for tuple_index in range(len(attr_values[index])):
                if attr_values[index][tuple_index] != datum[index]:
                    specials.append(list(h))
                    specials[-1][index] = attr_values[index][tuple_index]

    # return as tuple of tuples
    return tuple(tuple(x) for x in specials)

attr_values = (('short', 'far'), ('cheap', 'expensive'), ('many', 'none'), ('yes', 'no'))
assert specialize_minimally(('?', '?', 'many', 'no'), ('short', 'cheap', 'many', 'no'), attr_values) == \
    (('far', '?', 'many', 'no'), ('?', 'expensive', 'many', 'no'))
```

Now check that the algorithm works in the intended way by excecuting the cells below.

```python
def eliminate_candidates(data, attr_values):
    """
    Candidate elimination algorithm printing its progress at each step.

    Args:
        data (list of dicts): The dataset.
        attr_values (tuple of tuples): All possible attribute values for each attribute in the data.

    Returns:
        tuple: The general and the specific boundary of the version space.
    """
    # Initialize general and specific boundary.

    # Maximally general hypothesis.
    general_boundary = [('?', '?', '?', '?')]
    # Maximally specific hypothesis.
    specific_boundary = [('0', '0', '0', '0')]

    # Fit the version space to the data.
    for datum in data:
        if datum['target']:
            # If the sample is classified as positive...

            # Remove all inconsistent hypotheses from G.
            general_boundary = [g for g in general_boundary if is_consistent(g, datum)]

            for s in specific_boundary: # Copy set to allow changing it while iterating.
                # Remove all inconsistent hypothesis from S.
                if not is_consistent(s, datum):
                    specific_boundary.remove(s)

                    # Add to S all minimal generalizations s. In this case only one.
                    s_generalized = generalize_minimally(s, datum['values'])
                    # Add the new hypothesis to the specific boundary, if it is not more general
                    # than the general boundary. We do not need to check for consistency again
                    # as the hypothesis was constructed in such a way that it must be consisten.
                    if any([is_more_general_or_equal(g, s_generalized) for g in general_boundary]):
                        specific_boundary.append(s_generalized)

            # Remove from S any hypothesis that is more general than another hypothesis in S.
            for s in specific_boundary:
                if any([is_more_general_or_equal(s, s2) and not s == s2 for s2  in specific_boundary]):
                    specific_boundary.remove(s)

        else:
            # If the sample is classified as negative...

            # Remove all inconsistent hypotheses from S.
            specific_boundary = [s for s in specific_boundary if is_consistent(s, datum)]
            for g in general_boundary:
                # Remove all inconsistent hypotheses from G.
                if not is_consistent(g, datum):
                    general_boundary.remove(g)

                    # Add to G all minimal specializations of g.
                    for specialized_g in specialize_minimally(g, datum['values'], attr_values):
                        # Add the new specialized hypothesis to the general boundary, if it is not more
                        # specific than the specific boundary.
                        # We do not need to check for consistency again
                        # as the hypothesis was constructed in such a way that it must be consistent.
                        if (any([is_more_specific_or_equal(s, specialized_g) for s in specific_boundary])):
                            general_boundary.append(specialized_g)

                # Remove from G any hypothesis that is less general than another hypothesis in G.
                for g in general_boundary:
                    if any([is_more_specific_or_equal(g, g2) and not g == g2 for g2 in general_boundary]):
                        general_boundary.remove(g)

        # Print progress of algorithm at each iteration.
        print('Sample: {} {}\nG: {}\nS: {}\n'.format('+' if datum['target'] else '-', datum['values'],
                                                     general_boundary, specific_boundary))

    return general_boundary, specific_boundary
```

```python
attr_values = (('short', 'far'), ('cheap', 'expensive'), ('many', 'none'), ('yes', 'no'))

# samples
data = [
    {'values': ('far',   'cheap',     'many', 'no' ), 'target': True },
    {'values': ('short', 'cheap',     'many', 'no' ), 'target': True },
    {'values': ('far',   'expensive', 'none', 'yes'), 'target': False},
    {'values': ('short', 'cheap',     'none', 'yes'), 'target': False},
    {'values': ('short', 'cheap',     'many', 'yes'), 'target': True }
]

eliminate_candidates(data, attr_values)
```

    Sample: + ('far', 'cheap', 'many', 'no')
    G: [('?', '?', '?', '?')]
    S: [('far', 'cheap', 'many', 'no')]

    Sample: + ('short', 'cheap', 'many', 'no')
    G: [('?', '?', '?', '?')]
    S: [('?', 'cheap', 'many', 'no')]

    Sample: - ('far', 'expensive', 'none', 'yes')
    G: [('?', 'cheap', '?', '?'), ('?', '?', 'many', '?'), ('?', '?', '?', 'no')]
    S: [('?', 'cheap', 'many', 'no')]

    Sample: - ('short', 'cheap', 'none', 'yes')
    G: [('?', '?', 'many', '?'), ('?', '?', '?', 'no')]
    S: [('?', 'cheap', 'many', 'no')]

    Sample: + ('short', 'cheap', 'many', 'yes')
    G: [('?', '?', 'many', '?')]
    S: [('?', 'cheap', 'many', '?')]

    ([('?', '?', 'many', '?')], [('?', 'cheap', 'many', '?')])

## Assignment 3: Inductive Bias [4 Points]

**a)** What is an inductive bias? Describe the concept in your own words! Why do we need it?

Inductive bias is the set of assumptions an algorithm uses to predict to learn from given training data and use that learned concept to predict the outcome of previously unseen data. An unbiased learner makes no a priori assumptions, therefore it can't draw any inductive conclusions for unseen data from the previously seen training examples.

Consider this example:
I want to learn the concept of what coffee looks like. After seeing one black coffee, I think that all coffee is black. I can only infer that because of my underlying assumption that all coffee has the same color. Without this bias, there is nothing I could say about the color of coffee in general, only that there is some black coffee.

**b)** Describe and compare the inductive bias for the learning algorithms you heard about in the lecture (Candidate Elimination and Find-S).

The Find-S algorithm learns by looking at all available positive training examples and constructs a hypothesis that is consistent with all positive training examples. Here, the inductive bias means assuming that the concept is contained in the hypothesis space and that all instances are negative unless otherwise indicated by the training data.

In contrast, Candidate Elimination only assumes that the target concept is contained in the hypothesis space. It learns from both positive and negative training examples and finds the complete version space that fits the training data.


# Selección óptima de portafolios I

En la clase pasada vimos que:
- La LAC describe las posibles selecciones de riesgo-rendimiento entre un activo libre de riesgo y un activo riesgoso.
- Su pendiente es igual al radio de Sharpe del activo riesgoso.
- La asignación óptima de capital para cualquier inversionista es el punto tangente de la curva de indiferencia del inversionista con la LAC (depende de las preferencias particulares - aversión al riesgo).

Para todo lo anterior, supusimos que ya teníamos el portafolio óptimo (activo riesgoso).

En el siguiente análisis:

**Objetivos:**
- ¿Cuál es el portafolio óptimo de activos riesgosos?
 - ¿Cuál es el mejor portafolio de activos riesgosos?
 - Es un portafolio eficiente en media-varianza.
- Problema: dado un conjunto de activos riesgosos, ¿cómo construimos la mejor combinación?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Maximizando el radio de Sharpe

### ¿Qué pasa si tenemos dos activos riesgosos?

Cuando tenemos dos activos riesgosos, tenemos disponibles diferentes LAC. ¿Qué significan sus pendientes?

 Ver en el tablero.

Pregunta:
- ¿Qué es lo que se quiere?

**Conclusión:**
- el *portafolio tangente* de la LAC más pendiente;
- por tanto, maximiza el radio de Sharpe para todos los posibles portafolios de activos riesgosos;
- este portafolio se conoce como el portafolio eficiente en media-varianza (EMV).

**Idea principal: el portafolio óptimo de activos riesgosos es independiente de las preferencias del inversionista.**
- El portafolio EMV determina el portafolio óptimo de activos riesgosos.
- Todos tendremos el mismo portafolio de activos riesgosos (EMV), y lo combinaremos con el activo libre de reisgo de acuerdo con las preferencias de cada uno de nosotros (aversión al riesgo).
- La LAC combinando el activo libre de riesgo y el portafolio EMV, se vuelve el conjunto de portafolios eficientes.

Entonces, se deben seguir los siguientes pasos:
1. Crear la frontera media-varianza.
2. Encontrar el portafolio que maximize el radio de Sharpe (portafolio EMV).
3. Construir la frontera eficiente (LAC) del punto $(0,r_f)$ al punto $(E[r_s],\sigma_s)$ del portafolio EMV.
4. Combinar de acuerdo a sus preferencias.
___

## 2. Solución analítica del portafolio EMV: caso con dos activos.

Queremos solucionar el siguiente problema:

\begin{align}
\max_{w_1,w_2} &\quad \frac{E[r_p]-r_f}{\sigma_p}\\
\text{s.a.}    &\quad E[r_p]=w_1E[r_1]+w_2E[r_2]\\
               &\quad \sigma_p=\sqrt{w_1^2\sigma_1^2+w_2^2\sigma_2^2+2w_1w_2\rho_{12}\sigma_1\sigma_2}\\
               &\quad w_1+w_2=1, \quad w_1,w_2\geq0
\end{align}

el cual es equivalente a

\begin{align}
\max_{w_1} &\quad \frac{w_1E[r_1]+(1-w_1)E[r_2]-r_f}{\sqrt{w_1^2\sigma_1^2+(1-w_1)^2\sigma_2^2+2w_1(1-w_1)\rho_{12}\sigma_1\sigma_2}}\\
\text{s.a.}    &\quad 0\leq w_1\leq1
\end{align}

**Actividad.**
El anterior es un problema de maximizar una función de una variable en un dominio cerrado. No debaría representar dificultad.

Encontrar la solución analítica a este problema.

Quien primero lo haga, y salga a explicarlo al tablero, le subo alguna tarea, quiz o el primer examen a 100.

Deben llegar a:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

Si nadie lo ha hecho en 30 min., procederé a hacerlo yo.

**Nota:**
- así como obtuvimos una expresión para el peso del portafolio de mínima varianza con dos activos, obtenemos una expresión para el peso del portafolio Eficiente en Media-Varianza.
- Estas actividades son sin duda un buen ejercicio, y se pueden replicar usando técnicas de varias variables (multiplicadores de Lagrange) cuando se tengan más de dos activos.
- Sin embargo, la complejidad del problema crece considerablemente con el número de variables, y la solución analítica deja de ser viable cuando mencionamos que un portafolio bien diversificado consta aproximadamente de 50-60 activos.
- En esos casos, este problema se soluciona con rutinas numéricas que hagan la optimización por nosotros.
- Por eso, les enseño cómo resolver este problema con optimizadores numéricos, porque son una solución viable y escalable a más variables.

## 3. Ejemplo ilustrativo.

Retomamos el ejemplo de mercados de acciones en los países integrantes del $G5$: EU, RU, Francia, Alemania y Japón.

```python
# Importamos pandas y numpy
import pandas as pd
import numpy as np
```

```python
# Resumen en base anual de rendimientos esperados y volatilidades
annual_ret_summ = pd.DataFrame(columns=['EU', 'RU', 'Francia', 'Alemania', 'Japon'], index=['Media', 'Volatilidad'])
annual_ret_summ.loc['Media'] = np.array([0.1355, 0.1589, 0.1519, 0.1435, 0.1497])
annual_ret_summ.loc['Volatilidad'] = np.array([0.1535, 0.2430, 0.2324, 0.2038, 0.2298])

annual_ret_summ.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      Media
      0.1355
      0.1589
      0.1519
      0.1435
      0.1497

      Volatilidad
      0.1535
      0.243
      0.2324
      0.2038
      0.2298

```python
# Matriz de correlación
corr = pd.DataFrame(data= np.array([[1.0000, 0.5003, 0.4398, 0.3681, 0.2663],
                                    [0.5003, 1.0000, 0.5420, 0.4265, 0.3581],
                                    [0.4398, 0.5420, 1.0000, 0.6032, 0.3923],
                                    [0.3681, 0.4265, 0.6032, 1.0000, 0.3663],
                                    [0.2663, 0.3581, 0.3923, 0.3663, 1.0000]]),
                    columns=annual_ret_summ.columns, index=annual_ret_summ.columns)
corr.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      EU
      1.0000
      0.5003
      0.4398
      0.3681
      0.2663

      RU
      0.5003
      1.0000
      0.5420
      0.4265
      0.3581

      Francia
      0.4398
      0.5420
      1.0000
      0.6032
      0.3923

      Alemania
      0.3681
      0.4265
      0.6032
      1.0000
      0.3663

      Japon
      0.2663
      0.3581
      0.3923
      0.3663
      1.0000

Supondremos, además, que la tasa libre de riesgo es $r_f=5\%$.

```python
# Tasa libre de riesgo
rf = 0.05
```

Entonces, supondremos que tenemos disponibles los activos correspondientes a los mercados de acciones de EU y Japón, y en adición el activo libre de riesgo.

#### 1. Construir la frontera de mínima varianza

```python
# Vector de w variando entre 0 y 1 con n pasos
n = 101
w = np.linspace(0, 1, n)
# Rendimientos esperados individuales
# Activo1: EU, Activo2:Japon
E1 = annual_ret_summ['EU']['Media']
E2 = annual_ret_summ['Japon']['Media']
# Volatilidades individuales
s1 = annual_ret_summ['EU']['Volatilidad']
s2 = annual_ret_summ['Japon']['Volatilidad']
# Correlacion
r12 = corr['EU']['Japon']
```

```python
# DataFrame de portafolios:
# 1. Índice: i
# 2. Columnas 1-2: w, 1-w
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
portafolios = pd.DataFrame(index=np.arange(n),
                           columns=['w','1-w','E[r]','sigma','SR'])
portafolios['w']=w
portafolios['1-w']=1-w
portafolios['E[r]']=w*E1+(1-w)*E2
portafolios['sigma']=np.sqrt((w*s1)**2+((1-w)*s2)**2+2*w*(1-w)*r12*s1*s2)
portafolios['SR']=(portafolios['E[r]']-rf)/portafolios['sigma']

portafolios.round(4)
```

      w
      1-w
      E[r]
      sigma
      SR

      0
      0.00
      1.00
      0.1497
      0.2298
      0.4339

      1
      0.01
      0.99
      0.1496
      0.2279
      0.4368

      2
      0.02
      0.98
      0.1494
      0.2260
      0.4398

      3
      0.03
      0.97
      0.1493
      0.2242
      0.4428

      4
      0.04
      0.96
      0.1491
      0.2223
      0.4459

      5
      0.05
      0.95
      0.1490
      0.2205
      0.4490

      6
      0.06
      0.94
      0.1488
      0.2186
      0.4521

      7
      0.07
      0.93
      0.1487
      0.2168
      0.4552

      8
      0.08
      0.92
      0.1486
      0.2150
      0.4584

      9
      0.09
      0.91
      0.1484
      0.2132
      0.4616

      10
      0.10
      0.90
      0.1483
      0.2114
      0.4648

      11
      0.11
      0.89
      0.1481
      0.2097
      0.4681

      12
      0.12
      0.88
      0.1480
      0.2079
      0.4714

      13
      0.13
      0.87
      0.1479
      0.2061
      0.4747

      14
      0.14
      0.86
      0.1477
      0.2044
      0.4780

      15
      0.15
      0.85
      0.1476
      0.2027
      0.4814

      16
      0.16
      0.84
      0.1474
      0.2010
      0.4848

      17
      0.17
      0.83
      0.1473
      0.1993
      0.4882

      18
      0.18
      0.82
      0.1471
      0.1976
      0.4916

      19
      0.19
      0.81
      0.1470
      0.1959
      0.4951

      20
      0.20
      0.80
      0.1469
      0.1943
      0.4986

      21
      0.21
      0.79
      0.1467
      0.1926
      0.5020

      22
      0.22
      0.78
      0.1466
      0.1910
      0.5056

      23
      0.23
      0.77
      0.1464
      0.1894
      0.5091

      24
      0.24
      0.76
      0.1463
      0.1878
      0.5126

      25
      0.25
      0.75
      0.1462
      0.1863
      0.5162

      26
      0.26
      0.74
      0.1460
      0.1847
      0.5197

      27
      0.27
      0.73
      0.1459
      0.1832
      0.5233

      28
      0.28
      0.72
      0.1457
      0.1817
      0.5269

      29
      0.29
      0.71
      0.1456
      0.1802
      0.5304

      ...
      ...
      ...
      ...
      ...
      ...

      71
      0.71
      0.29
      0.1396
      0.1421
      0.6308

      72
      0.72
      0.28
      0.1395
      0.1419
      0.6305

      73
      0.73
      0.27
      0.1393
      0.1418
      0.6300

      74
      0.74
      0.26
      0.1392
      0.1417
      0.6293

      75
      0.75
      0.25
      0.1390
      0.1417
      0.6285

      76
      0.76
      0.24
      0.1389
      0.1417
      0.6275

      77
      0.77
      0.23
      0.1388
      0.1417
      0.6263

      78
      0.78
      0.22
      0.1386
      0.1418
      0.6249

      79
      0.79
      0.21
      0.1385
      0.1420
      0.6233

      80
      0.80
      0.20
      0.1383
      0.1421
      0.6216

      81
      0.81
      0.19
      0.1382
      0.1423
      0.6197

      82
      0.82
      0.18
      0.1381
      0.1426
      0.6176

      83
      0.83
      0.17
      0.1379
      0.1429
      0.6154

      84
      0.84
      0.16
      0.1378
      0.1432
      0.6130

      85
      0.85
      0.15
      0.1376
      0.1436
      0.6104

      86
      0.86
      0.14
      0.1375
      0.1440
      0.6077

      87
      0.87
      0.13
      0.1373
      0.1444
      0.6049

      88
      0.88
      0.12
      0.1372
      0.1449
      0.6019

      89
      0.89
      0.11
      0.1371
      0.1454
      0.5988

      90
      0.90
      0.10
      0.1369
      0.1460
      0.5955

      91
      0.91
      0.09
      0.1368
      0.1466
      0.5921

      92
      0.92
      0.08
      0.1366
      0.1472
      0.5886

      93
      0.93
      0.07
      0.1365
      0.1479
      0.5850

      94
      0.94
      0.06
      0.1364
      0.1486
      0.5813

      95
      0.95
      0.05
      0.1362
      0.1493
      0.5774

      96
      0.96
      0.04
      0.1361
      0.1501
      0.5735

      97
      0.97
      0.03
      0.1359
      0.1509
      0.5695

      98
      0.98
      0.02
      0.1358
      0.1517
      0.5654

      99
      0.99
      0.01
      0.1356
      0.1526
      0.5612

      100
      1.00
      0.00
      0.1355
      0.1535
      0.5570

101 rows × 5 columns

```python
# Importar librerías de gráficos
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR
plt.figure(figsize=(7,5))
plt.scatter(portafolios['sigma'],portafolios['E[r]'],
            c=portafolios['SR'],cmap='RdYlBu')
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')

plt.colorbar()
```

#### 2. Encontrar el portafolio que maximiza el radio de Sharpe (EMV)

Primero, encontramos este portafolio con la fórmula que obtuvimos:

```python
# Fórmula que obtuvimos
wEMV = ((E1-rf)*s2**2-(E2-rf)*r12*s1*s2)/((E2-rf)*s1**2+(E1-rf)*s2**2-((E1-rf)+(E2-rf))*r12*s1*s2)
wEMV
```

    0.6983139170512034

Ahora sí, con la función scipy.optimize.minimize

```python
# Importar el módulo optimize
from scipy.optimize import minimize
```

```python
## Construcción de parámetros
## Activo 1: EU, Activo 2: Japon
# 1. Sigma: matriz de varianza-covarianza
Sigma = np.array([[s1**2,     r12*s1*s2],
                  [r12*s1*s2, s2**2    ]])
# 2. Eind: rendimientos esperados activos individuales
Eind = np.array([E1, E2])
```

```python
# Función objetivo (-SR)
def obj_func(w, Eind, Sigma, rf):
    Erp = Eind.dot(w)
    sp = np.sqrt(w.dot(Sigma).dot(w))
    Sharpe_r = (Erp-rf)/sp
    return -Sharpe_r
```

```python
# Dato inicial
w0 = np.ones(2)/2
# Cotas de las variables
bnds = ((0,1),)*2
#bnds = ((0,1),(0,1))#
# Restricciones
cons = ({'type': 'eq', 'fun': lambda w: np.sum(w)-1},)
```

```python
# Optimización numérica
portEMV = minimize(obj_func, w0, args=(Eind,Sigma,rf), bounds=bnds,
                  constraints=cons)
# Resultado
portEMV
```

         fun: -0.6308725303332082
         jac: array([-0.35171176, -0.3502962 ])
     message: 'Optimization terminated successfully.'
        nfev: 17
         nit: 4
        njev: 4
      status: 0
     success: True
           x: array([0.6983075, 0.3016925])

Con lo anterior, podemos obtener datos de rendimiento esperado y volatilidad del portafolio EMV

```python
# Rendimiento esperado y volatilidad del portafolio EMV
wEMV = portEMV.x
EpEMV = wEMV.dot(Eind)
spEMV = np.sqrt(wEMV.dot(Sigma).dot(wEMV))
EpEMV, spEMV
```

    (0.13978403352815136, 0.14231723400720916)

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, y portafolio EMV
plt.figure(figsize=(7,5))
plt.scatter(portafolios['sigma'],portafolios['E[r]'],
            c=portafolios['SR'],cmap='RdYlBu')
plt.plot(spEMV, EpEMV, '*g', ms=20)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')

plt.colorbar()
```

#### 3. Construir LAC

Ahora, dibujamos la LAC, combinando el portafolio EMV con el activo libre de riesgo:

```python
# Vector de wp variando entre 0 y 1.5 con n pasos
wp = np.linspace(0, 1.5, n)
```

```python
# DataFrame de CAL:
# 1. Índice: i
# 2. Columnas 1-2: wp, wrf
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
LAC = pd.DataFrame(index=np.arange(n), columns=['wp','wrf','E[r]','sigma','SR'])
LAC['wp'] = wp
LAC['wrf']= 1-wp
LAC['E[r]']= wp*EpEMV+(1-wp)*rf
LAC['sigma']= wp*spEMV
LAC['SR'] = (LAC['E[r]']-rf)/LAC['sigma']

LAC.round(4)
```

      wp
      wrf
      E[r]
      sigma
      SR

      0
      0.000
      1.000
      0.0500
      0.0000
      NaN

      1
      0.015
      0.985
      0.0513
      0.0021
      0.6309

      2
      0.030
      0.970
      0.0527
      0.0043
      0.6309

      3
      0.045
      0.955
      0.0540
      0.0064
      0.6309

      4
      0.060
      0.940
      0.0554
      0.0085
      0.6309

      5
      0.075
      0.925
      0.0567
      0.0107
      0.6309

      6
      0.090
      0.910
      0.0581
      0.0128
      0.6309

      7
      0.105
      0.895
      0.0594
      0.0149
      0.6309

      8
      0.120
      0.880
      0.0608
      0.0171
      0.6309

      9
      0.135
      0.865
      0.0621
      0.0192
      0.6309

      10
      0.150
      0.850
      0.0635
      0.0213
      0.6309

      11
      0.165
      0.835
      0.0648
      0.0235
      0.6309

      12
      0.180
      0.820
      0.0662
      0.0256
      0.6309

      13
      0.195
      0.805
      0.0675
      0.0278
      0.6309

      14
      0.210
      0.790
      0.0689
      0.0299
      0.6309

      15
      0.225
      0.775
      0.0702
      0.0320
      0.6309

      16
      0.240
      0.760
      0.0715
      0.0342
      0.6309

      17
      0.255
      0.745
      0.0729
      0.0363
      0.6309

      18
      0.270
      0.730
      0.0742
      0.0384
      0.6309

      19
      0.285
      0.715
      0.0756
      0.0406
      0.6309

      20
      0.300
      0.700
      0.0769
      0.0427
      0.6309

      21
      0.315
      0.685
      0.0783
      0.0448
      0.6309

      22
      0.330
      0.670
      0.0796
      0.0470
      0.6309

      23
      0.345
      0.655
      0.0810
      0.0491
      0.6309

      24
      0.360
      0.640
      0.0823
      0.0512
      0.6309

      25
      0.375
      0.625
      0.0837
      0.0534
      0.6309

      26
      0.390
      0.610
      0.0850
      0.0555
      0.6309

      27
      0.405
      0.595
      0.0864
      0.0576
      0.6309

      28
      0.420
      0.580
      0.0877
      0.0598
      0.6309

      29
      0.435
      0.565
      0.0891
      0.0619
      0.6309

      ...
      ...
      ...
      ...
      ...
      ...

      71
      1.065
      -0.065
      0.1456
      0.1516
      0.6309

      72
      1.080
      -0.080
      0.1470
      0.1537
      0.6309

      73
      1.095
      -0.095
      0.1483
      0.1558
      0.6309

      74
      1.110
      -0.110
      0.1497
      0.1580
      0.6309

      75
      1.125
      -0.125
      0.1510
      0.1601
      0.6309

      76
      1.140
      -0.140
      0.1524
      0.1622
      0.6309

      77
      1.155
      -0.155
      0.1537
      0.1644
      0.6309

      78
      1.170
      -0.170
      0.1550
      0.1665
      0.6309

      79
      1.185
      -0.185
      0.1564
      0.1686
      0.6309

      80
      1.200
      -0.200
      0.1577
      0.1708
      0.6309

      81
      1.215
      -0.215
      0.1591
      0.1729
      0.6309

      82
      1.230
      -0.230
      0.1604
      0.1751
      0.6309

      83
      1.245
      -0.245
      0.1618
      0.1772
      0.6309

      84
      1.260
      -0.260
      0.1631
      0.1793
      0.6309

      85
      1.275
      -0.275
      0.1645
      0.1815
      0.6309

      86
      1.290
      -0.290
      0.1658
      0.1836
      0.6309

      87
      1.305
      -0.305
      0.1672
      0.1857
      0.6309

      88
      1.320
      -0.320
      0.1685
      0.1879
      0.6309

      89
      1.335
      -0.335
      0.1699
      0.1900
      0.6309

      90
      1.350
      -0.350
      0.1712
      0.1921
      0.6309

      91
      1.365
      -0.365
      0.1726
      0.1943
      0.6309

      92
      1.380
      -0.380
      0.1739
      0.1964
      0.6309

      93
      1.395
      -0.395
      0.1752
      0.1985
      0.6309

      94
      1.410
      -0.410
      0.1766
      0.2007
      0.6309

      95
      1.425
      -0.425
      0.1779
      0.2028
      0.6309

      96
      1.440
      -0.440
      0.1793
      0.2049
      0.6309

      97
      1.455
      -0.455
      0.1806
      0.2071
      0.6309

      98
      1.470
      -0.470
      0.1820
      0.2092
      0.6309

      99
      1.485
      -0.485
      0.1833
      0.2113
      0.6309

      100
      1.500
      -0.500
      0.1847
      0.2135
      0.6309

101 rows × 5 columns

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, portafolio EMV y LAC
plt.figure(figsize=(7,5))
plt.scatter(portafolios['sigma'],portafolios['E[r]'],
            c=portafolios['SR'],cmap='RdYlBu')
plt.plot(spEMV, EpEMV, '*g', ms=20)
plt.plot(LAC['sigma'],LAC['E[r]'],'k--',lw=3)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.xlim([0.13,0.16])
plt.ylim([0.13,0.15])

plt.colorbar()
```

#### 4. Combinación óptima de acuerdo a preferencias

Con los datos anteriores, y la caracterización de aversión al riesgo, se escoge la combinación óptima entre el portafolio EMV y el activo libre de riesgo de acuerdo a:

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}.$$

```python
# Para gamma=7

```

Created with Jupyter by Esteban Jiménez Rodríguez.


This notebook is based off of the [SGD mnist](https://github.com/fastai/fastai/blob/master/courses/ml1/lesson4-mnist_sgd.ipynb) lesson from fastai

In this notebook we will start with a pytorch neural network implementation of logitistic regression and then program it ourselves

# Imports and Paths

```python
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
from data_sci.imports import *
from data_sci.utilities import *
from data_sci.fastai import *
from data_sci.fastai.dataset import *
from data_sci.fastai.metrics import *
from data_sci.fastai.torch_imports import *
from data_sci.fastai.model import *

import torch.nn as nn
```

Path to download the data

```python
PATH = '/data/msnow/data_science/mnist/'
```

# MNIST Data

Let's download, unzip, and format the data.

```python
URL='http://deeplearning.net/data/mnist/'
FILENAME='mnist.pkl.gz'

def load_mnist(filename):
    return pickle.load(gzip.open(filename, 'rb'), encoding='latin-1')
```

```python
get_data(os.path.join(URL,FILENAME), os.path.join(PATH,FILENAME))
((x, y), (x_valid, y_valid), _) = load_mnist(os.path.join(PATH,FILENAME))
```

```python
type(x), x.shape, type(y), y.shape
```

    (numpy.ndarray, (50000, 784), numpy.ndarray, (50000,))

```python
y_valid.shape
```

    (10000,)

### Normalize

Many machine learning algorithms behave better when the data is *normalized*, that is when the mean is 0 and the standard deviation is 1. We will subtract off the mean and standard deviation from our training set in order to normalize the data:

```python
mean = x.mean()
std = x.std()

x=(x-mean)/std
mean, std, x.mean(), x.std()
```

    (0.13044983, 0.30728981, -3.1638146e-07, 0.99999934)

Note that for consistency (with the parameters we learn when training), we subtract the mean and standard deviation of our training set from our validation set.

```python
x_valid = (x_valid-mean)/std
x_valid.mean(), x_valid.std()
```

    (-0.0058509219, 0.99243325)

### Look at the data

```python
def plots(ims, figsize=(12,6), rows=2, titles=None):
    f = plt.figure(figsize=figsize)
    cols = len(ims)//rows
    for i in range(len(ims)):
        sp = f.add_subplot(rows, cols, i+1)
        sp.axis('Off')
        if titles is not None: sp.set_title(titles[i], fontsize=16)
        plt.imshow(ims[i], cmap='gray')
```

```python
x_valid.shape
```

    (10000, 784)

```python
y_valid.shape
```

    (10000,)

The data has been reduced from a rank 3 tensor of image number by image width by image height to a rank 2 tensor of number of image number by pixel. Before visualizing the image we need to convert it back into a rank 3 tensor using numpy's `reshape`.

`y_valid` is the list of integers which correspond to the images in `x_valid`

```python
x_imgs = np.reshape(x_valid, (-1,28,28)); x_imgs.shape
```

    (10000, 28, 28)

```python
plt.imshow(x_imgs[0],cmap='gray')
plt.title(y_valid[0]);
```

```python
plots(x_imgs[:8], titles=y_valid[:8])
```

# Neural Networks

A **function** takes inputs and returns outputs.  The classic example is the equation for a line

$$ f(x) = ax + b $$

where $x$ is the input, $a$ and $b$ are the **parameters** and $f(x)$ is the output.  A *neural network* is just a specific type of function (an *infinitely flexible function* to be exact), consisting of *layers*, which are made up of parameters.  A *layer* is a linear function such as matrix multiplication followed by a non-linear function (the *activation*).  For example, an equation for a simple neural network can be represented with the equation:

$$ f(\mathbf{x}) = a_{nl}\left(a_l \mathbf{x}\right) $$

where $\mathbf{x}$ is the input (the bold font just means that it is not just a scalar, i.e., it is a tensor of rank greater than 0), $a_l$ is the linear layer parameter, and $a_{nl}$ is the non linear layer parameter.

However, most neural networks are not nearly this simple.  They often have thousands, or even hundreds of thousands of parameters.  However the core idea is the same.  The neural network is a function, and we will learn the best parameters for modeling our data.

As an aside there always needs to be a non-linear function after each linear function, as multiple linear functions in a row can always be represented by a single linear function.  This is part of the definition of linear transformations.  For example, multiplying anything by 5 and then dividing the result by 2 is equivalent to multiplying the input by 5/2.

# PyTorch

PyTorch has two overlapping, yet distinct, purposes.  As described in the [PyTorch documentation](http://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html):

The neural network functionality of PyTorch is built on top of the Numpy-like functionality for fast matrix computations on a GPU. Although the neural network purpose receives way more attention, both are very useful.  We'll implement a neural net from scratch today using PyTorch.

**Further learning**: If you are curious to learn what *dynamic* neural networks are, you may want to watch [this talk](https://www.youtube.com/watch?v=Z15cBAuY7Sc) by Soumith Chintala, Facebook AI researcher and core PyTorch contributor.

If you want to learn more PyTorch, you can try this [introductory tutorial](http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) or this [tutorial to learn by examples](http://pytorch.org/tutorials/beginner/pytorch_with_examples.html).

# Use the built in Neural Net for Logistic Regression in PyTorch

We will begin with the highest level abstraction: using a neural net defined by PyTorch's Sequential class.  In PyTorch to switch between cpu and CUDA enabled gpu you just need to add `.cuda()` to the end of any nn architecture:

- GPU enabled
```python
net = nn.Sequential(
    nn.Linear(28*28, 10),
    nn.LogSoftmax()
).cuda()
```

- GPU disabled (using cpu)
```python
net = nn.Sequential(
    nn.Linear(28*28, 10),
    nn.LogSoftmax()
)
```

As we said above, each layer of the neural network is just the data modified first by a linear function and then by a non-linear function.  So if we wanted to build a neural network version of multinomial logistic regression, termed softmax in neural networks. Just as a reminder, here is the equation for softmax to predict the $j$-th class given a sample vector $\mathbf{x}$ and a weighting vector $\mathbf{w}$:

$$ Pr(y=j \mid  \mathbf{x}) = \dfrac{\exp\left(\mathbf{x}^T \mathbf{w}_j\right)}{\sum\limits_{k=1}^K\exp\left(\mathbf{x}^T\mathbf{w}_k\right)}$$

Each input is a vector of size `28*28` pixels and our output is of size `10` (since there are 10 digits: 0, 1, ..., 9).

We use the output of the final layer to generate our predictions.  Often for classification problems (like MNIST digit classification), the final layer has the same number of outputs as there are classes.  In that case, this is 10: one for each digit from 0 to 9.  These can be converted to comparative probabilities.  For instance, it may be determined that a particular hand-written image is 80% likely to be a 4, 18% likely to be a 9, and 2% likely to be a 3.

So putting everything together, we want a layer composed of a linear component which converts our `28*28` rank 1 tensor (aka vector) inputs into an output rank 1 tensor of size `10` and then we want to apply a softmax classification function.

```python
net = nn.Sequential(
    nn.Linear(28*28, 10),
    nn.LogSoftmax()
)
```

```python
md = ImageClassifierData.from_arrays(PATH, (x,y), (x_valid, y_valid))
```

`ImageClassifierData` is from the [Fast AI library](https://github.com/fastai/fastai/blob/master/fastai/dataset.py) and calls a PyTorch data loader, which grabs a few images, sticks them into a mini-batch and makes them available, this is equivalent to a python generator.  You can call each of the different data loaders

- training data loader = md.trn_dl
- validation data loader = md.val_dl
- test data loader = md.test_dl
- augmented data loader = md.aug_dl
- test augemented data loader = md.test_aug_dl

## Aside about loss functions and metrics

In machine learning the **loss** function or cost function is representing the price paid for inaccuracy of predictions.  To understand where this loss function comes from let's take a little detour into probability theory.

Given a fair coin what is the probability of the following outcomes:
 - Tails, Heads
 - Heads, Heads
 - Tails, Tails
You don't need any equations to know that the probability of each of those outcomes is 0.25, as there are four possible outcomes when you flip a coin twice, and if it is a fair coin the probability of each outcome is 0.25.  We can formalize this through the probability mass function (pmf) of the Bernoulli distribution:

$$
f(y;p) = p^y\left(1-p\right)^{1-y} \texttt{ for }k \in \{0,1\}
$$

where $y$ is the number of events occuring, $p$ is the probability of that event occuring and $f(y;p)$ is the probability of $y$ events occuring given $p$.  One way to determine the loss of this function is to take the negative log-likelihood (or cross-entropy) of our function.

\begin{align}
\ell\left(f\right) & = \log \left(p^y\left(1-p\right)^{1-y}\right)\\
&= \log \left(p^y\right) + \log\left(\left(1-p\right)^{1-y}\right)\\
&= y\log p + (1-y) \log (1-p)
\end{align}

In machine learning terms the equations are the same, but the terms means slightly different things.  For a given observation, $y$ is the true label and $p$ is the probability given by the model's prediction.

```python
def binary_loss(y, p):
    return np.mean(-(y * np.log(p) + (1-y)*np.log(1-p)))
```

```python
acts = np.array([1, 0, 0, 1])
preds = np.array([0.9, 0.1, 0.2, 0.8])
binary_loss(acts, preds)
```

    0.164252033486018

Note that in our toy example above our accuracy is 100% and our loss is 0.16.

Why not just maximize accuracy? The binary classification loss is an easier function to optimize.

For multi-class classification, we use *negative log liklihood* (also known as *categorical cross entropy*) which is exactly the same thing, but summed up over all classes.  For example, let's say we were trying to train a model to recognize pictures of animals, and the options were horse, dog, cat, lion, tiger, and bear.  We can represent each of the different options as a one hot encoded vectors:

|    |   horse |   dog |   cat |   lion |   tiger |   bear |
|---:|--------:|------:|------:|-------:|--------:|-------:|
|  0 |       1 |     0 |     0 |      0 |       0 |      0 |
|  1 |       0 |     1 |     0 |      0 |       0 |      0 |
|  2 |       0 |     0 |     1 |      0 |       0 |      0 |
|  3 |       0 |     0 |     0 |      1 |       0 |      0 |
|  4 |       0 |     0 |     0 |      0 |       1 |      0 |
|  5 |       0 |     0 |     0 |      0 |       0 |      1 |

let's say the model predicted the following output for a bear

|    |   bear |   predicted_bear |
|---:|-------:|-----------------:|
|  0 |      0 |             0.05 |
|  1 |      0 |             0.05 |
|  2 |      0 |             0.05 |
|  3 |      0 |             0.05 |
|  4 |      0 |             0.05 |
|  5 |      1 |             0.75 |

to calculate the loss for the prediction we look at the loss for each observation in the vector.

|    |   bear |   predicted_bear | loss_equation                     |   observation_loss |
|---:|-------:|-----------------:|:----------------------------------|-------------------:|
|  0 |      0 |             0.05 | 0\*log(0.05) + (1-0)\*log(1-0.05) |          0.0512933 |
|  1 |      0 |             0.05 | 0\*log(0.05) + (1-0)\*log(1-0.05) |          0.0512933 |
|  2 |      0 |             0.05 | 0\*log(0.05) + (1-0)\*log(1-0.05) |          0.0512933 |
|  3 |      0 |             0.05 | 0\*log(0.05) + (1-0)\*log(1-0.05) |          0.0512933 |
|  4 |      0 |             0.05 | 0\*log(0.05) + (1-0)\*log(1-0.05) |          0.0512933 |
|  5 |      1 |             0.75 | 1\*log(0.75) + (1-1)\*log(1-0.75) |          0.287682  |

The total loss for that prediction is the sum of the individual observation losses which comes out to 0.544

To understand this a little more, let's calculate the observation losses for a few more predictions

|    |   bear |   predict_1 |    loss_1 |   predict_2 |    loss_2 |   predict_3 |   loss_3 |
|---:|-------:|------------:|----------:|------------:|----------:|------------:|---------:|
|  0 |      0 |        0.05 | 0.0512933 |        0.01 | 0.0100503 |         0.1 | 0.105361 |
|  1 |      0 |        0.05 | 0.0512933 |        0.01 | 0.0100503 |         0.1 | 0.105361 |
|  2 |      0 |        0.05 | 0.0512933 |        0.01 | 0.0100503 |         0.1 | 0.105361 |
|  3 |      0 |        0.05 | 0.0512933 |        0.01 | 0.0100503 |         0.1 | 0.105361 |
|  4 |      0 |        0.05 | 0.0512933 |        0.01 | 0.0100503 |         0.1 | 0.105361 |
|  5 |      1 |        0.75 | 0.287682  |        0.95 | 0.0512933 |         0.5 | 0.693147 |

You can see that as the predictions get more accurate the individual observation losses decrease and vice versa as the predictions get less accurate.

This one-hot encoding for multiclass classification is why when  creating our nerual network above we built the layer to be (28,28,10).  The last value of the tuple is 10 to represent the 1 by 10 one-hot encoded vectors of the digits

## Now back to the model

As explained in the above aside, we want to use a negative log likelihood loss function for our model, `nn.NLLLoss()` in pytorch.  When evaluating our model we can use the accuracy.  For the optimization of our model we are going to use stochastic gradient descent, telling it to use the layers in `net` as the parameters, and setting other parameters which will be explained in other notebooks

```python
loss=nn.NLLLoss()
metrics=[accuracy]
opt=optim.SGD(net.parameters(), 1e-1, momentum=0.9)
```

## Fitting the model

*Fitting* is the process by which the neural net learns the best parameters for the dataset.

```python
fit(net, md, n_epochs=1, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=1), HTML(value='')))

    epoch      trn_loss   val_loss   accuracy
        0      1.301665   1.140233   0.8873

    [1.140232567659393, 0.88729999999999998]

GPUs are great at handling lots of data at once.  We break the data up into **batches**, and that specifies how many samples from our dataset we want to send to the processor (GPU or CPU) at a time.  The fastai library defaults to a batch size of 64.  On each iteration of the training loop, the error on 1 batch of data will be calculated, and the optimizer will update the parameters based on that.

An **epoch** is completed once each data sample has been used once in the training loop.

Now that we have the parameters for our model, we can make predictions on our validation set.

```python
preds = predict(net, md.val_dl)
```

```python
preds.shape
```

    (10000, 10)

Our output is 10,000 by 10 because we have 10,000 observations in our validation set and 10 predictions per observation, i.e., prediction that it's a zero, prediction that it's a one, ...

To see how accurate our predictions are, we want to know how often the digit highest predicted matched the actual digit.  For now, we don't care about the actual value of the prediction, just the location of the highest prediction.

```python
pd.DataFrame(preds[:3,:])
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9

      0
      -7.747974
      -3.077208
      -2.283786
      -0.378413
      -9.398509
      -3.487280
      -6.933934
      -12.972561
      -2.002663
      -10.997322

      1
      -11.389916
      -7.895541
      -5.798741
      -4.611126
      -4.631413
      -3.517211
      -11.018111
      -9.716209
      -0.054559
      -8.327415

      2
      -3.095443
      -11.162576
      -3.108638
      -6.810033
      -3.949405
      -2.309221
      -0.239779
      -6.866230
      -6.556308
      -6.815598

Argmax gives the location of the highest value in the array, which in this case since all the values are negative is actually the smallest value.  Since we want to know the location of the largest value across the columns, we have to set the axis to 1.

```python
preds[:3,:].argmax(axis=1)
```

    array([3, 8, 6])

```python
preds_argmax = preds.argmax(1)
```

Now let's see what percentage of predictions we got right

```python
np.mean(preds_argmax == y_valid)
```

    0.91549999999999998

If you notice this is the same accruacy that is the output of `fit`.  Accuracy is the metric being used because that is what we set it to, much higher up. And the `loss` function we set to `NLLLoss` which is the pytorch abbreviation for negative log likelihood loss

```python
plots(x_imgs[:8], titles=preds_argmax[:8])
```

This is a one layer neural net, i.e., a logistic regression, and we can actually recreate the same output using scikit learn's logistic regression function.  **TODO**

### DELETE ME Integrate the original code from the notebook into my cells

```python
%%time
fit(net, md, n_epochs=5, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=5), HTML(value='')))

      1%|          | 7/782 [00:00<00:21, 35.46it/s, loss=1.23]

    epoch      trn_loss   val_loss   accuracy
        0      1.213766   1.438505   0.8368

      1%|          | 7/782 [00:00<00:21, 36.57it/s, loss=1.27]

        1      1.257663   1.21375    0.8687

      1%|          | 6/782 [00:00<00:24, 32.17it/s, loss=1.3]

        2      1.304803   1.252223   0.8562

      1%|          | 6/782 [00:00<00:23, 32.76it/s, loss=1.22]

        3      1.223197   0.916236   0.8883

        4      1.208414   1.096059   0.8778

    CPU times: user 45.4 s, sys: 30.7 s, total: 1min 16s
    Wall time: 41.4 s

    [1.0960593797355891, 0.87780000000000002]

```python
set_lrs(opt, 1e-2)
```

```python
fit(net, md, n_epochs=3, crit=loss, opt=opt, metrics=metrics)
```

    A Jupyter Widget

    [ 0.       0.07224  0.13539  0.97024]
    [ 1.       0.05924  0.13015  0.97114]
    [ 2.       0.04723  0.1316   0.97124]

```python
fit(net, md, n_epochs=5, crit=loss, opt=opt, metrics=metrics)
```

    A Jupyter Widget

    [ 0.       0.27253  0.21465  0.93939]
    [ 1.       0.21963  0.23439  0.93481]
    [ 2.       0.23288  0.18333  0.94705]
    [ 3.       0.19822  0.1902   0.94636]
    [ 4.       0.205    0.27594  0.92227]

```python
set_lrs(opt, 1e-2)
```

```python
fit(net, md, n_epochs=3, crit=loss, opt=opt, metrics=metrics)
```

    A Jupyter Widget

    [ 0.       0.07269  0.09255  0.97373]
    [ 1.       0.06316  0.08361  0.97572]
    [ 2.       0.04525  0.08077  0.97681]

```python
t = [o.numel() for o in net.parameters()]
t, sum(t)
```

    ([78400, 100, 10000, 100, 1000, 10], 89610)

# Building our own Neural Network for Logistic Regression in PyTorch

Above, we used pytorch's `nn.Linear` to create a linear layer, defined as a matrix multiplication followed by an addition (these are also called `affine transformations`).  Let's try defining this ourselves, by building our own PyTorch class.  A PyTorch module is either a neural net or a layer in a nueral net (since neural nets are modular and a neural net itself can be a "layer" in a larger neural net)

Just as Numpy has `np.matmul` for matrix multiplication (in Python 3, this is equivalent to the `@` operator), PyTorch has `torch.matmul`.  In other words `torch.matmul(x,y) == x@y`

Our PyTorch class needs two things: constructor (says what the parameters are) and a forward method (how to calculate a prediction using those parameters)  The method `forward` describes how the neural net converts inputs to outputs.

In PyTorch, the optimizer knows to try to optimize any attribute of type **Parameter**.  This is why above when we called the SGD optimizer (`optim.sgd`) we had to call the `.parameter()` function on our layers

## Aside about sub classes in python

When creating a class, we can just add functionality to a pre-existing class by calling that class when creating your new class.  Our new class is a subclass of `nn.module` and it is inheriting the properties of the nn.module class.  There is one rule when creating a subclass, you need to include the following line in your init

```python
super().__init__()
```

This tells python to initialize the superclass (in this case the torch module) before initiating the current subclass (in this case LogReg)

## Back to the model

When initializing the random weight in the `get_weights` function, in order to prevent gradient shrinking or gradient explosion we divide the weight matrix by the size of the tensor.

Since we don't know how many dimensions our data will have, we can use `*args`, which when used as a parameter in a function allows us to send a variable-length argument list.  Note that it is the use of the asterisk, \*, that matters, the word `args` is just convention.

In torch `view` is the equivalent of `reshape`, and setting a dimension to -1 tells it to infer the size of that dimension from the other dimensions

```python
>>> x = torch.randn(4, 4)
>>> z = x.view(-1, 8)
>>> z.size()
    torch.Size([2, 8])
```

```python
def get_weights(*dims): return nn.Parameter(torch.randn(dims)/dims[0])
def softmax(x): return torch.exp(x)/(torch.exp(x).sum(dim=1)[:,None])

class LogReg(nn.Module):
    def __init__(self):
        super().__init__()
        self.l1_w = get_weights(28*28, 10)  # Layer 1 weights
        self.l1_b = get_weights(10)         # Layer 1 bias

    def forward(self, x):
        x = x.view(x.size(0), -1)  # equivalent to reshape in numpy
        x = (x @ self.l1_w) + self.l1_b  # Linear Layer
        x = torch.log(softmax(x)) # Non-linear (LogSoftmax) Layer
        return x
```

We create our neural net and the optimizer.  (We will use the same loss and metrics from above).

```python
# net2 = LogReg().cuda()
net2 = LogReg()
# opt=optim.Adam(net2.parameters())
opt=optim.SGD(net2.parameters(), 1e-1, momentum=0.9, weight_decay=1e-3)
```

```python
fit(net2, md, n_epochs=1, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=1), HTML(value='')))

    epoch      trn_loss   val_loss   accuracy
        0      0.361127   0.305531   0.9149

    [0.30553094034194944, 0.91490000000000005]

```python
preds2 = predict(net2, md.val_dl)
```

```python
preds2_argmax = preds2.argmax(axis=1)
```

```python
plots(x_imgs[:8], titles=preds2_argmax[:8])
```

```python
np.mean(preds2_argmax == y_valid)
```

    0.91490000000000005

## Not sure where this code is appropriate

```python
dl = iter(md.trn_dl)
```

```python
xmb,ymb = next(dl)
```

```python
print(xmb.shape)
xmb
```

    torch.Size([64, 784])

    tensor([[-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            ...,
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245]])

Wrapping a tensor in `Variable` is how you let PyTorch know to keep track of the differential for this parameter as it will need to perform SGD on it later.

```python
# vxmb = Variable(xmb.cuda())
vxmb = Variable(xmb)
vxmb
```

    tensor([[-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            ...,
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245],
            [-0.4245, -0.4245, -0.4245,  ..., -0.4245, -0.4245, -0.4245]])

```python
preds = net2(vxmb).exp(); preds[:3]
```

    tensor([[ 6.3626e-06,  1.2105e-06,  2.8107e-06,  7.3512e-06,  3.9612e-01,
              1.7406e-04,  2.1466e-04,  3.4038e-03,  1.5325e-02,  5.8474e-01],
            [ 2.2152e-04,  5.5954e-02,  5.2104e-04,  1.0306e-02,  6.7856e-05,
              6.0536e-01,  4.6721e-04,  7.0908e-02,  5.2787e-02,  2.0341e-01],
            [ 2.0823e-02,  2.3426e-05,  1.9553e-04,  5.9046e-01,  1.0586e-06,
              1.9001e-03,  5.4244e-06,  3.8488e-01,  8.8515e-06,  1.7022e-03]])

```python
preds.shape
```

    torch.Size([64, 10])

```python
preds.data.max(1)
```

    (tensor([ 0.5847,  0.6054,  0.5905,  0.9803,  0.9996,  0.9835,  0.9921,
              0.4827,  0.9459,  0.8103,  0.6352,  0.9750,  0.9640,  0.9042,
              0.9222,  0.9806,  0.8631,  0.9998,  0.9300,  0.9923,  0.9897,
              0.9733,  0.9717,  0.9304,  0.8534,  0.9967,  0.9979,  0.9072,
              0.9595,  0.9741,  0.9957,  0.9997,  0.5420,  0.9958,  0.7623,
              0.9162,  0.9322,  0.9749,  0.9908,  0.6102,  0.9590,  0.8889,
              0.9988,  0.9977,  0.6241,  0.8398,  0.9533,  0.5453,  0.9243,
              0.9640,  0.9956,  0.9500,  0.8289,  0.9947,  0.8073,  0.7368,
              0.9902,  0.9932,  0.9977,  0.7467,  0.9724,  0.9951,  0.9824,
              0.9456]),
     tensor([ 9,  5,  3,  3,  3,  1,  3,  4,  8,  8,  8,  9,  1,  8,
              1,  7,  4,  0,  0,  7,  7,  3,  5,  3,  6,  6,  6,  8,
              1,  1,  1,  0,  4,  5,  7,  9,  1,  3,  1,  2,  0,  2,
              2,  2,  4,  3,  6,  9,  9,  8,  2,  8,  2,  6,  5,  4,
              7,  7,  3,  8,  6,  6,  8,  7]))

```python
preds.max(1)
```

    (tensor([ 0.5847,  0.6054,  0.5905,  0.9803,  0.9996,  0.9835,  0.9921,
              0.4827,  0.9459,  0.8103,  0.6352,  0.9750,  0.9640,  0.9042,
              0.9222,  0.9806,  0.8631,  0.9998,  0.9300,  0.9923,  0.9897,
              0.9733,  0.9717,  0.9304,  0.8534,  0.9967,  0.9979,  0.9072,
              0.9595,  0.9741,  0.9957,  0.9997,  0.5420,  0.9958,  0.7623,
              0.9162,  0.9322,  0.9749,  0.9908,  0.6102,  0.9590,  0.8889,
              0.9988,  0.9977,  0.6241,  0.8398,  0.9533,  0.5453,  0.9243,
              0.9640,  0.9956,  0.9500,  0.8289,  0.9947,  0.8073,  0.7368,
              0.9902,  0.9932,  0.9977,  0.7467,  0.9724,  0.9951,  0.9824,
              0.9456]),
     tensor([ 9,  5,  3,  3,  3,  1,  3,  4,  8,  8,  8,  9,  1,  8,
              1,  7,  4,  0,  0,  7,  7,  3,  5,  3,  6,  6,  6,  8,
              1,  1,  1,  0,  4,  5,  7,  9,  1,  3,  1,  2,  0,  2,
              2,  2,  4,  3,  6,  9,  9,  8,  2,  8,  2,  6,  5,  4,
              7,  7,  3,  8,  6,  6,  8,  7]))

```python
preds = preds.data.max(1)[1]; preds
```

     5
     2
     4
     1
     8
     8
     3
     4
     2
     7
     1
     1
     7
     9
     3
     4
     3
     0
     1
     4
     3
     3
     6
     8
     9
     0
     9
     5
     2
     4
     7
     9
     0
     2
     5
     2
     9
     9
     6
     9
     4
     2
     2
     7
     1
     5
     5
     9
     0
     7
     4
     0
     9
     1
     1
     4
     7
     0
     9
     0
     4
     4
     0
     7
    [torch.cuda.LongTensor of size 64 (GPU 0)]

# Writing Our Own Training Loop

As a reminder, this is what we did above to write our own logistic regression class (as a pytorch neural net):

```python
# Our code from above
class LogReg(nn.Module):
    def __init__(self):
        super().__init__()
        self.l1_w = get_weights(28*28, 10)  # Layer 1 weights
        self.l1_b = get_weights(10)         # Layer 1 bias

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = x @ self.l1_w + self.l1_b
        return torch.log(softmax(x))

# net2 = LogReg().cuda()
net2 = LogReg()
# opt=optim.Adam(net2.parameters())
opt=optim.SGD(net2.parameters(), 1e-1, momentum=0.9, weight_decay=1e-3)

fit(net2, md, n_epochs=1, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=1), HTML(value='')))

    epoch      trn_loss   val_loss   accuracy
        0      0.37501    0.335794   0.9129

    [0.33579445953369141, 0.91290000000000004]

Above, we are using the fastai method `fit` to train our model.  Now we will try writing the training loop ourselves.

We will use the LogReg class we created, as well as the same loss function, learning rate, and optimizer as before:

```python
# net2 = LogReg().cuda()
net2 = LogReg()
loss=nn.NLLLoss()
# learning_rate = 1e-3
opt=optim.SGD(net2.parameters(), 1e-1, momentum=0.9, weight_decay=1e-3)
# optimizer=optim.Adam(net2.parameters(), lr=learning_rate)
```

md is the ImageClassifierData object we created above.  We want an iterable version of our training data

```python
dl = iter(md.trn_dl) # Data loader
```

First, we will do a **forward pass**, which means computing the predicted `y` by passing `x` to the model.

```python
xt, yt = next(dl)
# y_pred = net2(Variable(xt).cuda())
y_pred = net2(Variable(xt))
```

We can check the loss:

```python
# l = loss(y_pred, Variable(yt).cuda())
l = loss(y_pred, Variable(yt))
print(l)
```

    tensor(2.2973)

We may also be interested in the accuracy.  We don't expect our first predictions to be very good, because the weights of our network were initialized to random values.  Our goal is to see the loss decrease (and the accuracy increase) as we train the network:

```python
np.mean(to_np(y_pred).argmax(axis=1) == to_np(yt))
```

    0.125

Now we will use the optimizer to calculate which direction to step in.  That is, how should we update our weights to try to decrease the loss?

Pytorch has an automatic differentiation package ([autograd](http://pytorch.org/docs/master/autograd.html)) that takes derivatives for us, so we don't have to calculate the derivative ourselves!  We just call `.backward()` on our loss to calculate the direction of steepest descent (the direction to lower the loss the most).

```python
# Before the backward pass, use the optimizer object to zero all of the
# gradients for the variables it will update (which are the learnable weights
# of the model)
opt.zero_grad()

# Backward pass: compute gradient of the loss with respect to model parameters
l.backward()

# Calling the step function on an Optimizer makes an update to its parameters
opt.step()
```

Now, let's make another set of predictions and check if our loss is lower:

```python
xt, yt = next(dl)
# y_pred = net2(Variable(xt).cuda())
y_pred = net2(Variable(xt))
```

```python
# l = loss(y_pred, Variable(yt).cuda())
l = loss(y_pred, Variable(yt))
print(l)
```

    tensor(2.1683)

Note that we are using **stochastic** gradient descent, so the loss is not guaranteed to be strictly better each time.  The stochasticity comes from the fact that we are using **mini-batches**; we are just using 64 images to calculate our prediction and update the weights, not the whole dataset.

```python
np.mean(to_np(y_pred).argmax(axis=1) == to_np(yt))
```

    0.390625

If we run several iterations in a loop, we should see the loss decrease and the accuracy increase with time.

```python
for t in range(100):
    xt, yt = next(dl)
#     y_pred = net2(Variable(xt).cuda())
    y_pred = net2(Variable(xt))
#     l = loss(y_pred, Variable(yt).cuda())
    l = loss(y_pred, Variable(yt))

    if t % 10 == 0:
        accuracy = np.mean(to_np(y_pred).argmax(axis=1) == to_np(yt))
        print("loss: ", l.data[0], "\t accuracy: ", accuracy)

    opt.zero_grad()
    l.backward()
    opt.step()
```

    loss:  tensor(2.2301) 	 accuracy:  0.265625
    loss:  tensor(0.6034) 	 accuracy:  0.84375
    loss:  tensor(0.6776) 	 accuracy:  0.75
    loss:  tensor(0.6230) 	 accuracy:  0.828125
    loss:  tensor(0.4466) 	 accuracy:  0.875
    loss:  tensor(0.3114) 	 accuracy:  0.921875
    loss:  tensor(0.4596) 	 accuracy:  0.859375
    loss:  tensor(0.3602) 	 accuracy:  0.90625
    loss:  tensor(0.4485) 	 accuracy:  0.890625
    loss:  tensor(0.3115) 	 accuracy:  0.890625

### Put it all together in a training loop

```python
def score(x, y):
    y_pred = to_np(net2(V(x)))
    return np.sum(y_pred.argmax(axis=1) == to_np(y))/len(y_pred)
```

```python
# net2 = LogReg().cuda()
net2 = LogReg()
loss=nn.NLLLoss()
learning_rate = 1e-2
optimizer=optim.SGD(net2.parameters(), lr=learning_rate)
trn_batches = md.trn_ds.n // md.bs + int(md.trn_ds.n % md.bs > 0)
val_batches = md.val_ds.n // md.bs + int(md.val_ds.n % md.bs > 0)

for epoch in range(1):
    losses=[]
    dl = iter(md.trn_dl)
#     for t in range(len(dl)):
    for t in range(trn_batches):
        # Forward pass: compute predicted y and loss by passing x to the model.
        xt, yt = next(dl)
        y_pred = net2(V(xt))
        l = loss(y_pred, V(yt))
        losses.append(l)

        # Before the backward pass, use the optimizer object to zero all of the
        # gradients for the variables it will update (which are the learnable weights of the model)
        optimizer.zero_grad()

        # Backward pass: compute gradient of the loss with respect to model parameters
        l.backward()

        # Calling the step function on an Optimizer makes an update to its parameters
        optimizer.step()

    val_dl = iter(md.val_dl)
    val_scores = [score(*next(val_dl)) for i in range(val_batches)]
#     val_scores = [score(*next(val_dl)) for i in range(len(val_dl))]
    print(np.mean(val_scores))
```

    0.874104299363

# Stochastic Gradient Descent

Nearly all of deep learning is powered by one very important algorithm: **stochastic gradient descent (SGD)**. SGD can be seeing as an approximation of **gradient descent (GD)**. In GD you have to run through all the samples in your training set to do a single itaration. In SGD you use only a subset of training samples to do the update for a parameter in a particular iteration. The subset used in each iteration is called a batch or minibatch.

Now, instead of using the optimizer, we will do the optimization ourselves!

```python
# net2 = LogReg().cuda()
net2 = LogReg()
loss_fn=nn.NLLLoss()
lr = 1e-2
w,b = net2.l1_w,net2.l1_b
trn_batches = md.trn_ds.n // md.bs + int(md.trn_ds.n % md.bs > 0)
val_batches = md.val_ds.n // md.bs + int(md.val_ds.n % md.bs > 0)

for epoch in range(3):
    losses=[]
    dl = iter(md.trn_dl)
#     for t in range(len(dl)):
    for t in range(trn_batches):
        xt, yt = next(dl)
        y_pred = net2(V(xt))
#         l = loss(y_pred, Variable(yt).cuda())
        l = loss(y_pred, Variable(yt))
        losses.append(loss)

        # Backward pass: compute gradient of the loss with respect to model parameters
        l.backward()
        w.data -= w.grad.data * lr
        b.data -= b.grad.data * lr

        w.grad.data.zero_() # suffix underscore just means do the operation in place instead of creating new variable
        b.grad.data.zero_()

    val_dl = iter(md.val_dl)
    val_scores = [score(*next(val_dl)) for i in range(val_batches)]
#     val_scores = [score(*next(val_dl)) for i in range(len(val_dl))]
    print(np.mean(val_scores))
```

    0.871218152866
    0.885250796178
    0.893113057325

# Addenda

To see the number of parameters per layer in my neural network by calling the `.parameters()` method on my neural network.  This creates a generator object which will go through each layer,  If I then iterate through the generator I can extract the number of elements in each layer using `.numel()`

```python
t = [o.numel() for o in net.parameters()]
t, sum(t)
```

    ([7840, 10], 7850)

## L2 Regularization and weight decay

Remember that we calculate our parameters by minimizing a loss function, which has the general form of :

$$ \ell = \sum\limits_{i=1}^n\left(Y_i - \sum\limits_{j=1}^p X_{ij} w_j\right)^2 $$

One way to reduce the overfitting of our model is to penalize all non-zero weights.  We can do this by adding a term to our loss function:

$$ \ell = \sum\limits_{i=1}^n\left(Y_i - \sum\limits_{j=1}^p X_{ij} w_j\right)^2 + \ell_{reg}(w)$$

We don't want to penalize weights with a value zero, as that is the same as not having any parameter, just the degree to which the weights differ from zero.  Two standard ways of doing this are L1 regularization, also known as lasso regression (least absolute shrinkage and selection operator), and L2 regularization.  L1 regularization uses the absolute value of the weights and L2 uses the square of the weights.  In both cases you don't want to add the entire value of the weights to the loss function, just some small percentage, on the order of $10^{-4}$ to $10^{-6}$, which we can call $\alpha$.

$$ \ell_{L1} = \sum\limits_{i=1}^n\left(Y_i - \sum\limits_{j=1}^p X_{ij} w_j\right)^2 + \alpha\sum\limits_{j=1}^p\left|w_j\right| $$

$$ \ell_{L2} = \sum\limits_{i=1}^n\left(Y_i - \sum\limits_{j=1}^p X_{ij} w_j\right)^2 + \alpha\sum\limits_{j=1}^p w_j^2 $$

When working with neural networks we are often concerned with the gradient of the loss, which for L2 is easy to calculate

$$\Delta_{L2} = 2\alpha w $$

In neural networks, this L2 gradient is referred to as the weight decay.

Let's build a lsightly deeper simple neural network with and without weight decay

```python
net = nn.Sequential(
    nn.Linear(28*28, 100),
    nn.ReLU(),
    nn.Linear(100, 100),
    nn.ReLU(),
    nn.Linear(100, 10),
    nn.LogSoftmax()
)
md = ImageClassifierData.from_arrays(PATH, (x,y), (x_valid, y_valid))
loss=nn.NLLLoss()
metrics=[accuracy]
```

```python
opt=optim.SGD(net.parameters(), 1e-1, momentum=0.9)
```

```python
fit(net, md, n_epochs=3, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=3), HTML(value='')))

      1%|          | 4/782 [00:00<00:36, 21.48it/s, loss=0.294]

    epoch      trn_loss   val_loss   accuracy
        0      0.301842   0.24314    0.9398

      1%|          | 4/782 [00:00<00:37, 20.89it/s, loss=0.244]

        1      0.246357   0.264417   0.9415

        2      0.24186    0.250231   0.9451

    [0.25023104345500469, 0.94510000000000005]

```python
net = nn.Sequential(
    nn.Linear(28*28, 100),
    nn.ReLU(),
    nn.Linear(100, 100),
    nn.ReLU(),
    nn.Linear(100, 10),
    nn.LogSoftmax()
)
opt=optim.SGD(net.parameters(), 1e-1, momentum=0.9, weight_decay=1e-3)
```

```python
fit(net, md, n_epochs=3, crit=loss, opt=opt, metrics=metrics)
```

    HBox(children=(IntProgress(value=0, description='Epoch', max=3), HTML(value='')))

      1%|          | 4/782 [00:00<00:36, 21.27it/s, loss=0.274]

    epoch      trn_loss   val_loss   accuracy
        0      0.273734   0.244744   0.9298

      1%|          | 4/782 [00:00<00:37, 20.94it/s, loss=0.231]

        1      0.232462   0.275376   0.9212

        2      0.231492   0.191098   0.9475

    [0.19109784487709403, 0.94750000000000001]

You might expect the training loss to be worse with weight decay, but what might be happening here is that weight decay is making the loss function easier to optimize on a per epoch level, resulting in a lower training loss.  But in the end you would expect the training loss with weight decay to be worse (larger value) than the training loss without weight decay.

However, weight decay should improve the overall loss on the validation set, which it does in this case.  This is because the purpose of the weight decay is to decrease overfitting, which should make the model more generalizable.


```python
%pylab inline
import yt
yt.mylog.setLevel(50) #suppress yt outputs
```

    Populating the interactive namespace from numpy and matplotlib

To analyze the hdf5  files that the simulation outputs,we will be using yt, a well-supported Python package for visualizing volumetric, AMR data from astrophysical simulations. For more information, refer to the excellent [yt documentation](http://yt-project.org/docs/3.2/) and [series of notebook tutorials](http://yt-project.org/docs/dev/quickstart/index.html) that is much more comprehensive than what is covered here. In this notebook, we will be covering yt basics, as well as using some custom scripts used for analyzing the FLASH data in ``plotSim.py`` that I wrote.

```python
from scripts.plotSim import *
```

    /anaconda2/lib/python2.7/site-packages/matplotlib/__init__.py:1350: UserWarning:  This call to matplotlib.use() has no effect
    because the backend has already been chosen;
    matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
    or matplotlib.backends is imported for the first time.

      warnings.warn(_use_error_msg)

```python
cd ~/data_astroSim/data/
```

    /global/u2/d/dorislee/data_astroSim/data

## Exploratory Data Analysis

Loading in the dataset for the last timestep in our Rotating Sink Sphere simulation:

```python
ds = yt.load("sphere_hdf5_chk_0494")
```

By default, FLASH outputs its data in cgs (except in MHD..), so the code units in yt is actually in cgs.

```python
print ds.length_unit,ds.mass_unit,ds.time_unit
```

    1.0 cm 1.0 g 1.0 s

Often useful information about a simulation timestep output:

```python
print "Current simulation timestep:", ds.current_time
```

    Current simulation timestep: 4.94073212769e+12 code_time

Here is a list of variables that are stored in the simulation outputs, note that some variables (e.g mag) will not have data (since this is a hydro run):

```python
print map(lambda x:str(x[1]),filter(lambda x :x[0]=='flash',ds.field_list))
```

    ['dens', 'divb', 'eint', 'ener', 'gamc', 'game', 'gpol', 'gpot', 'grac', 'magp', 'magx', 'magy', 'magz', 'pden', 'pres', 'sgax', 'sgay', 'sgaz', 'sgxo', 'sgyo', 'sgzo', 'shok', 'temp', 'velx', 'vely', 'velz']

This is just a demo of the long list of variables that yt can derive from the few variables stored in the simulation as shown above:

```python
print map(lambda x: x[1],ds.field_info.keys())[:15]
```

    ['particle_position_cylindrical_z', 'cylindrical_theta', 'particle_angular_momentum_z', u'game', 'particle_radius', u'divb', 'total_energy', 'particle_position_cylindrical_radius', 'density_gradient_magnitude', 'particle_cylindrical_velocity_theta', 'pressure', 'particle_velocity_z', 'particle_position_spherical_radius', 'magnetic_field_strength', 'io_nn_velocity_x']

# SlicePlots

We many things we can do on the yt dataset ``ds``, such as making SlicePlots, create a volumetric rendering, or making radial profiles by cutting through the simulation box.

Here we make a SlicePlot of the density through the z direction

```python
slc = yt.SlicePlot(ds, "z","density")
slc.set_figure_size(5) #sets the size of the plot
slc.set_cmap("all","rainbow")
```

We can plot what the AMR grid looks like. In this simulation, we are using Jean's criterion based AMR refinement.  The AMR structure would look quite different if we used density based refinement (which refines based on density gradients across cells). Note that the grid lines that yt is drawing denotes where the *blocks* are __NOT__ where the *cells* are. (Recall that each block consists of 8 cells)

```python
slc.annotate_grids()
```

We can also do fancy things like zooming in and plotting the velocity vectors.

```python
slc.zoom(4)
slc.annotate_grids()
slc.annotate_velocity(normalize=True)
```

zlim defines the limits on the colorbar for the variable that you are plotting. This is sometimes useful when we are looking at data across many timesteps.

~~~python
slc.set_zlim("density",zmin,zmax)

~~~

```python
plot_dens(0,fname="sphere")
```

```python
plot_var(0,"temperature", "sphere")
plot_var(0,"pressure", "sphere")
```

Without fixing the colorbar,

```python
plot_dens(300,fname="sphere")
```

Fixing the colorbar,

```python
plot_dens(300,fname="sphere",zmin = 5e-21,zmax = 1e-17)
```

As shown above, fixing the colorbar allows us to compare our simulation against the initial conditions. For example we may be interested in seeing which areas of the sphere experience a density increase compared to the initial sphere, rather than by *how much* does the density increase. On the other hand, allowing the colorbar to vary enable us to see more detailed structures that might be saturated if we retained the initial color scheme. It is important to think about whether the visualization settings may affect what we see in the data.

```python
plot_dens(494,fname="sphere",zoom=8)
```

```python
plot_dens(494,fname="sphere",zoom=8,zmin = 5e-21,zmax = 1e-17)
```

# Adding user-defined parameters

Even though yt has a comprehensive list of derived variables , sometimes it is useful to define our own parameters. For example, Foster and Chevalier (1993) defines a set of non-dimensional units used for their plots, so in order to compare our simulation result to theirs, we must additionally convert into these nondimensional units.
\begin{align}
D = \frac{\rho}{\rho_c} = 9.09\times 10^{18}\rho
\\ m = \frac{GM\sqrt{4\pi G\rho_c}}{a^3}=8.54\times10^{-34} M
\\v = \frac{u}{a} = 3.48\times10^{-5} u
\\ \xi =\frac{\sqrt{4\pi G \rho_c}}{a} r = 1.0568\times10^{-17} r
\\ \tau = \sqrt{4\pi G \rho_c} t  = 3.036\times 10^{-13} t
\end{align}
where $\rho$, M, u, r,t are in cgs units.

```python
%pylab inline
import yt
yt.mylog.setLevel(50)
from yt.units import second, g, cm ,dyne
G = 6.674e-8*cm**3/second**2/g
rho_c = 1.1e-17*g/cm**3
conversion = sqrt(4*pi*G*rho_c)
aT  = 28730*cm/second # for 10K gas
ctr= 5e18*cm
def _radius(field,data):
    radius = sqrt((data["x"]-ctr)**2+(data["y"]-ctr)**2+(data["z"]-ctr)**2)
def _norm_r(field,data):
    return data["radius"]*conversion/aT
def _norm_v(field,data):
    return data["radial_velocity"]/aT
def _norm_d(field,data):
    return data["density"]/rho_c
yt.add_field("radius",function= _radius,units="cm")
yt.add_field("norm_r",function= _norm_r,units="")
yt.add_field("norm_v",function= _norm_v,units="")
yt.add_field("norm_d",function= _norm_d,units="")
ds = yt.load("../../data_astroSim/data/sphere_hdf5_chk_0000")
data = ds.all_data()
```

    Populating the interactive namespace from numpy and matplotlib

Now the units are non-dimensional

```python
print data["norm_r"]
print data["norm_v"]
print data["norm_d"]
```

    [ 908.41256519  903.6691589   898.97660951 ...,  898.97660951  903.6691589
      908.41256519] dimensionless
    [  9.89269385e-19   4.94634692e-19   1.48390408e-18 ...,   0.00000000e+00
      -4.94634692e-19   0.00000000e+00] dimensionless
    [ 0.00045455  0.00045455  0.00045455 ...,  0.00045455  0.00045455
      0.00045455] dimensionless

We can now use these non-dimensional, derived fields in a profile plot:

```python
sphere = ds.sphere("c", (0.5, "pc"))
prof = yt.ProfilePlot(sphere, "norm_r","norm_d")
prof.show()
```

# FixedResolutionBuffer

turns your AMR data into a 2D or 3D numpy array which you can then use for plotting in
If you are used to matplotlib or other plotting library
I find that generally plotting with --- yt is sufficient and usually better than whatever ----, but FRBs are especially useful for quantitatively analyzing the data. For example in scripts/----, M/R plots

```python
Inside, scripts/
```

```python
cd ~/proj/dlee/FLASH4.3/object/
```

    /global/project/projectdirs/astro250/dlee/FLASH4.3/object

```python
ls sphere*
```

    sphere.dat            sphere_hdf5_part_0000     sphere.log
    sphere_hdf5_chk_0000  sphere_hdf5_plt_cnt_0000

```python
plot_dens(0,"sphere")
```

```python
plot_var(0,"temperature", "sphere")
```

```python
plot_var(0,"pressure", "sphere")
```

tutorial/.ipynb_checkpoints/6-Data Analysis and visualization with yt-checkpoint.ipynb

tutorial/6-Data Analysis and visualization with yt.ipynb