**This notebook is an exercise in the [Computer Vision](https://www.kaggle.com/learn/computer-vision) course.  You can reference the tutorial at [this link](https://www.kaggle.com/ryanholbrook/convolution-and-relu).**

---

# Introduction #

In this exercise, you'll work on building some intuition around feature extraction. First, we'll walk through the example we did in the tutorial again, but this time, with a kernel you choose yourself. We've mostly been working with images in this course, but what's behind all of the operations we're learning about is mathematics. So, we'll also take a look at how these feature maps can be represented instead as arrays of numbers and what effect convolution with a kernel will have on them.

Run the cell below to get started!

```python
# Setup feedback system
from learntools.core import binder
binder.bind(globals())
from learntools.computer_vision.ex2 import *
```

# Apply Transformations #

The next few exercises walk through feature extraction just like the example in the tutorial. Run the following cell to load an image we'll use for the next few exercises.

```python
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
plt.rc('figure', autolayout=True)
plt.rc('axes', labelweight='bold', labelsize='large',
       titleweight='bold', titlesize=18, titlepad=10)
plt.rc('image', cmap='magma')

image_path = '../input/computer-vision-resources/car_illus.jpg'
image = tf.io.read_file(image_path)
image = tf.io.decode_jpeg(image, channels=1)
image = tf.image.resize(image, size=[400, 400])

plt.figure(figsize=(6, 6))
plt.imshow(tf.squeeze(image), cmap='gray')
plt.axis('off')
plt.show();
```

You can run this cell to see some standard kernels used in image processing.

```python
import learntools.computer_vision.visiontools as visiontools
from learntools.computer_vision.visiontools import edge, bottom_sobel, emboss, sharpen

kernels = [edge, bottom_sobel, emboss, sharpen]
names = ["Edge Detect", "Bottom Sobel", "Emboss", "Sharpen"]

plt.figure(figsize=(12, 12))
for i, (kernel, name) in enumerate(zip(kernels, names)):
    plt.subplot(1, 4, i+1)
    visiontools.show_kernel(kernel)
    plt.title(name)
plt.tight_layout()
```

# 1) Define Kernel #

Use the next code cell to define a kernel. You have your choice of what kind of kernel to apply. One thing to keep in mind is that the *sum* of the numbers in the kernel determines how bright the final image is. Generally, you should try to keep the sum of the numbers between 0 and 1 (though that's not required for a correct answer).

In general, a kernel can have any number of rows and columns. For this exercise, let's use a $3 \times 3$ kernel, which often gives the best results. Define a kernel with `tf.constant`.

```python
# YOUR CODE HERE: Define a kernel with 3 rows and 3 columns.
kernel = tf.constant([
     [-1, 1, -1],
    [1,  8, 1],
    [-1, 1, -1],
])
# Uncomment to view kernel
visiontools.show_kernel(kernel)

# Check your answer
q_1.check()
```

```python
# Lines below will give you a hint or solution code
#q_1.hint()
#q_1.solution()
```

Now we'll do the first step of feature extraction, the filtering step. First run this cell to do some reformatting for TensorFlow.

```python
# Reformat for batch compatibility.
image = tf.image.convert_image_dtype(image, dtype=tf.float32)
image = tf.expand_dims(image, axis=0)
kernel = tf.reshape(kernel, [*kernel.shape, 1, 1])
kernel = tf.cast(kernel, dtype=tf.float32)
```

# 2) Apply Convolution #

Now we'll apply the kernel to the image by a convolution. The *layer* in Keras that does this is `layers.Conv2D`. What is the *backend function* in TensorFlow that performs the same operation?

```python
# YOUR CODE HERE: Give the TensorFlow convolution function (without arguments)
conv_fn = tf.nn.conv2d

# Check your answer
q_2.check()
```

    <IPython.core.display.Javascript object>

Correct

```python
# Lines below will give you a hint or solution code
#q_2.hint()
#q_2.solution()
```

Once you've got the correct answer, run this next cell to execute the convolution and see the result!

```python
image_filter = conv_fn(
    input=image,
    filters=kernel,
    strides=1, # or (1, 1)
    padding='SAME',
)

plt.imshow(
    # Reformat for plotting
    tf.squeeze(image_filter)
)
plt.axis('off')
plt.show();
```

Can you see how the kernel you chose relates to the feature map it produced?

# 3) Apply ReLU #

Now detect the feature with the ReLU function. In Keras, you'll usually use this as the activation function in a `Conv2D` layer. What is the *backend function* in TensorFlow that does the same thing?

```python
# YOUR CODE HERE: Give the TensorFlow ReLU function (without arguments)
relu_fn = tf.nn.relu

# Check your answer
q_3.check()
```

```python
# Lines below will give you a hint or solution code
#q_3.hint()
#q_3.solution()
```

Once you've got the solution, run this cell to detect the feature with ReLU and see the result!

The image you see below is the feature map produced by the kernel you chose. If you like, experiment with some of the other suggested kernels above, or, try to invent one that will extract a certain kind of feature.

```python
image_detect = relu_fn(image_filter)

plt.imshow(
    # Reformat for plotting
    tf.squeeze(image_detect)
)
plt.axis('off')
plt.show();
```

In the tutorial, our discussion of kernels and feature maps was mainly visual. We saw the effect of `Conv2D` and `ReLU` by observing how they transformed some example images.

But the operations in a convolutional network (like in all neural networks) are usually defined through mathematical functions, through a computation on numbers. In the next exercise, we'll take a moment to explore this point of view.

Let's start by defining a simple array to act as an image, and another array to act as the kernel. Run the following cell to see these arrays.

```python
# Sympy is a python library for symbolic mathematics. It has a nice
# pretty printer for matrices, which is all we'll use it for.
import sympy
sympy.init_printing()
from IPython.display import display

image = np.array([
    [0, 1, 0, 0, 0, 0],
    [0, 1, 0, 0, 0, 0],
    [0, 1, 0, 0, 0, 0],
    [0, 1, 0, 0, 0, 0],
    [0, 1, 0, 1, 1, 1],
    [0, 1, 0, 0, 0, 0],
])

kernel = np.array([
    [1, -1],
    [1, -1],
])

display(sympy.Matrix(image))
display(sympy.Matrix(kernel))
# Reformat for Tensorflow
image = tf.cast(image, dtype=tf.float32)
image = tf.reshape(image, [1, *image.shape, 1])
kernel = tf.reshape(kernel, [*kernel.shape, 1, 1])
kernel = tf.cast(kernel, dtype=tf.float32)
```

# 4) Observe Convolution on a Numerical Matrix #

What do you see? The image is simply a long vertical line on the left and a short horizontal line on the lower right. What about the kernel? What effect do you think it will have on this image? After you've thought about it, run the next cell for the answer.

```python
# View the solution (Run this code cell to receive credit!)
q_4.check()
```

Now let's try it out. Run the next cell to apply convolution and ReLU to the image and display the result.

```python
image_filter = tf.nn.conv2d(
    input=image,
    filters=kernel,
    strides=1,
    padding='VALID',
)
image_detect = tf.nn.relu(image_filter)

# The first matrix is the image after convolution, and the second is
# the image after ReLU.
display(sympy.Matrix(tf.squeeze(image_filter).numpy()))
display(sympy.Matrix(tf.squeeze(image_detect).numpy()))
```

Is the result what you expected?

# Conclusion #

In this lesson, you learned about the first two operations a convolutional classifier uses for feature extraction: **filtering** an image with a **convolution** and **detecting** the feature with the **rectified linear unit**.

# Keep Going #

Move on to [**Lesson 3**](https://www.kaggle.com/ryanholbrook/maximum-pooling) to learn the final operation: **condensing** the feature map with **maximum pooling**!

---

*Have questions or comments? Visit the [Learn Discussion forum](https://www.kaggle.com/learn-forum) to chat with other Learners.*


# Dynamic Latent Variable Models

### Preliminaries

- Goal
  - Introduction to dynamic (=temporal) Latent Variable Models, including the Hidden Markov Model and Kalman filter.
- Materials
  - Mandatory
    - These lecture notes
  - Optional
    - Bishop pp.605-615 on Hidden Markov Models
    - Bishop pp.635-641 on Kalman filters
    - Faragher (2012), [Understanding the Basis of the Kalman Filter](./files/Faragher-2012-Understanding-the-Basis-of-the-Kalman-Filter.pdf)
    - Minka (1999), [From Hidden Markov Models to Linear Dynamical Systems](./files/Minka-1999-from-HMM-to-LDS.pdf)

### Example Problem

- We consider a one-dimensional cart position tracking problem, see  [Faragher 2012](./files/Faragher-2012-Understanding-the-Basis-of-the-Kalman-Filter.pdf).

- The hidden states are the position $z_t$ and velocity $\dot z_t$. We can apply an external acceleration/breaking force $u_t$. (Noisy) observations are represented by $x_t$.

- The equations of motions are given by

$$\begin{align*}
\begin{bmatrix} z_t \\ \dot{z_t}\end{bmatrix} &=  \begin{bmatrix} 1 & \Delta t \\ 0 & 1\end{bmatrix} \begin{bmatrix} z_{t-1} \\ \dot z_{t-1}\end{bmatrix} + \begin{bmatrix} (\Delta t)^2/2 \\ \Delta t\end{bmatrix} u_t + \mathcal{N}(0,\Sigma_z) \\
x_t &= \begin{bmatrix} z_t \\ \dot{z_t}\end{bmatrix} + \mathcal{N}(0,\Sigma_x)
\end{align*}$$

- Infer the position after 10 time steps.

### Dynamical Models

- Consider the _ordered_ observation sequence $x^T \triangleq \left(x_1,x_2,\ldots,x_T\right)$.

- We wish to develop a generative model
    $$ p( x^T \,|\, \theta)$$
that 'explains' the time series $x^T$.

- We cannot use the IID assumption $p( x^T  | \theta) = \prod_t p(x_t \,|\, \theta)$. In general, we _can_ use the [**chain rule**](https://en.wikipedia.org/wiki/Chain_rule_(probability) (a.k.a. the general product rule)

$$\begin{align*}
p(x^T) &= p(x_T|x^{T-1}) \,p(x^{T-1}) \\
  &=  p(x_T|x^{T-1}) \,p(x_{T-1}|x^{T-2}) \cdots p(x_2|x_1)\,p(x_1) \\
  &= p(x_1)\prod_{t=2}^T p(x_t\,|\,x^{t-1})
\end{align*}$$

- Generally, we will want to limit the depth of dependencies on previous observations. For example, the $M$th-order linear **Auto-Regressive** (AR) model
    $$\begin{align*}
  p(x_t\,|\,x^{t-1}) = \mathcal{N}\left(x_t \,\middle|\,  \sum_{m=1}^M a_m x_{t-m}\,,\sigma^2\,\right)
    \end{align*}$$
    limits the dependencies to the past $M$ samples.

### State-space Models

- A limitation of AR models is that they need a lot of parameters in order to create a flexible model. E.g., if $x_t$ is an $K$-dimensional discrete variable, then an $M$th-order AR model will have $K^{M-1}(K-1)$ parameters.

- Similar to our work on Gaussian Mixture models, we can create a flexible dynamic system by introducing _latent_ (unobserved) variables  $z^T \triangleq \left(z_1,z_2,\dots,z_T\right)$ (one $z_t$ for each observation $x_t$). In dynamic systems, $z_t$ are called _state variables_.

- A general **state space model** is defined by
$$\begin{align*}
 p(x^T,z^T) &= \underbrace{p(z_1)}_{\text{initial state}} \prod_{t=2}^T \underbrace{p(z_t\,|\,z^{t-1})}_{\text{state transitions}}\,\prod_{t=1}^T \underbrace{p(x_t\,|\,z_t)}_{\text{observations}}
\end{align*}$$

- A common assumption is to let state transitions be ruled by a _first-order Markov chain_ as
$$
 p(z_t\,|\,z^{t-1}) = p(z_t\,|\,z_{t-1})
$$

- Exercise: Show that in a Markovian state-space model, the observation sequence $x^T$ is not a first-order Markov chain, i.e., show that for the model
    $$\begin{align*}
 p(x^T,z^T) &= p(z_1) \prod_{t=2}^T p(z_t\,|\,z_{t-1})\,\prod_{t=1}^T p(x_t\,|\,z_t)
\end{align*}$$
the following statement holds:
    $$p(x_t\,|\,x_{t-1},x_{t-2}) \neq p(x_t\,|\,x_{t-1})\,.$$
In other words, the latent variables $z_t$ represent a memory bank for past observations beyond $t-1$.

### Hidden Markov Models and Linear Dynamical Systems

- A **Hidden Markov Model** (HMM) is a specific state-space model with discrete-valued state variables $Z_t$.

- E.g., $Z_t$ is a $K$-dimensional hidden binary 'class indicator' with transition probabilities $A_{jk} \triangleq p(z_{tk}=1\,|\,z_{t-1,j}=1)$, or equivalently
  $$p(z_t|z_{t-1}) = \prod_{k=1}^K \prod_{j=1}^K A_{jk}^{z_{t-1,j}z_{tk}}$$
which is usually accompanied by an initial state distribution $\pi_k \triangleq p(z_{1k}=1)$.

- The classical HMM has also discrete-valued observations but in pratice any (probabilistic) observation model $p(x_t|z_t)$ may be coupled to the hidden Markov chain.

- Another well-known state-space model with continuous-valued state variables $Z_t$ is the **(Linear) Gaussian Dynamical System** (LGDS), which is defined as

$$\begin{align*}
p(z_t\,|\,z_{t-1}) &= \mathcal{N}\left(\, A z_{t-1}\,,\,\Sigma_z\,\right) \\
p(x_t\,|\,z_t) &= \mathcal{N}\left(\, C z_t\,,\,\Sigma_x\,\right) \\
p(z_1) &= \mathcal{N}\left(\, \mu_1\,,\,\Sigma_1\,\right)
\end{align*}$$

- Note that the joint distribution over $\{(x_1,z_1),\ldots,(x_t,z_t)\}$ is a (large-dimensional) Gaussian distribution. This means that, in principle, every inference problem on the LGDS model also leads to a Gaussian distribution.

- HMM's and LGDS's (and variants thereof) are at the basis of a wide range of complex information processing systems, such as speech and language recognition, robotics and automatic car navigation, and even processing of DNA sequences.

### Kalman Filtering

- Technically, a [**Kalman filter**](https://en.wikipedia.org/wiki/Kalman_filter) is the solution to the recursive estimation (inference) of the hidden state $z_t$ based on past observations in an LGDS, i.e., Kalman filtering solves the problem $p(z_t\,|\,x^t)$ based on the previous estimate $p(z_{t-1}\,|\,x^{t-1})$ and a new observation $x_t$ (in the context of the given model specification of course).

- Let's infer the Kalman filter for a scalar linear Gaussian dynamical system:
$$\begin{align*}
    p(z_t\,|\,z_{t-1}) &= \mathcal{N}(z_t\,|\,a z_{t-1},\sigma_z^2) \tag{state transition}   \\
    p(x_t\,|\,z_t) &= \mathcal{N}(x_t\,|\,c z_t,\sigma_x^2) \tag{observation}
\end{align*}$$

- Kalman filtering comprises inferring $p(z_t\,|\,x^t)$ from a given prior estimate $p(z_{t-1}\,|\,x^{t-1})$ and a new observation $x_t$. Let us assume that
$$\begin{align}
p(z_{t-1}\,|\,x^{t-1}) = \mathcal{N}(z_{t-1} \,|\, \mu_{t-1}, \sigma_{t-1}^2) \tag{prior}
\end{align}$$

- Note that everything is Gaussian, so this is _in principle_ possible to execute inference problems analytically and the result will be a Gaussian posterior:

$$\begin{align*}
\underbrace{p(z_t\,|\,x^t)}_{\text{posterior}} &= p(z_t\,|\,x_t,x^{t-1}) \propto p(x_t,z_t\,|\,x^{t-1}) \\
  &\propto p(x_t\,|\,z_t) \,p(z_t\,|\,x^{t-1}) \\
  &= p(x_t\,|\,z_t) \, \sum_{z_{t-1}} p(z_t,z_{t-1}\,|\,x^{t-1}) \\
  &= \underbrace{p(x_t\,|\,z_t)}_{\text{observation}} \, \sum_{z_{t-1}} \underbrace{p(z_t\,|\,z_{t-1})}_{\text{state transition}} \, \underbrace{p(z_{t-1}\,|\,x^{t-1})}_{\text{prior}} \\
  &= \mathcal{N}(x_t\,|\,c z_t,\sigma_x^2) \sum_{z_{t-1}} \mathcal{N}(z_t\,|\,a z_{t-1},\sigma_z^2) \, \mathcal{N}(z_{t-1} \,|\, \mu_{t-1}, \sigma_{t-1}^2) \\
  &\propto \mathcal{N}\left(z_t\,\bigm| \,\frac{x_t}{c} ,\left(\frac{\sigma_x}{c}\right)^2\right) \times \mathcal{N}\left(z_t\, \bigm|\,a \mu_{t-1},\sigma_z^2 + \left(a \sigma_{t-1}\right)^2 \right) \\
  &= \mathcal{N}\left( z_t \,|\, \mu_t, \sigma_t^2\right)
\end{align*}$$
with
$$\begin{align*}
  \rho_t^2 &= \sigma_z^2 + a^2 \sigma_{t-1}^2 \tag{auxiliary variable}\\
  K_t &= \frac{c \rho_t^2}{c^2 \rho_t^2 + \sigma_x^2} \tag{'Kalman gain'} \\
  \mu_t &= a \mu_{t-1} + K_t \cdot \left( x_t - c a \mu_{t-1}\right) \tag{posterior mean}\\
  \sigma_t^2 &= \left( 1 - K_t \right) \rho_t^2 \tag{posterior variance}
\end{align*}$$

- Kalman filtering consists of computing/updating these four equations for each new observation ($x_t$).

### Kalman Filtering and the Cart Position Tracking Example Revisited

- The Kalman filter equations can also be derived for multidimensional state-space models. In particular, for the model
$$\begin{align*}
z_t &= A z_{t-1} + \mathcal{N}(0,\Gamma) \\
x_t &= C z_t + \mathcal{N}(0,\Sigma)
\end{align*}$$
the Kalman filter update equations are given by (see Bishop, pg.639)
$$\begin{align*}
P_t &= A V_{t-1} A^T + \Gamma &&\text{auxiliary variable}\\
K_t &= P_t C^T \cdot \left( \Sigma + C P_t C^T\right)^{-1} &&\text{Kalman gain vector} \\
\mu_t &= A \mu_{t-1} + K_t\cdot\left(x_t - C A \mu_{t-1} \right) &&\text{posterior state mean}\\
V_t &= \left(I-K_t C \right) V_{t-1}  &&\text{posterior state variance}
\end{align*}$$

- We can solve the cart tracking problem by implementing the Kalman filter, see the next code example.

```julia
using LinearAlgebra, PyPlot
include("scripts/cart_tracking_helpers.jl")

# Specify the model parameters
Δt = 1.0                     # assume the time steps to be equal in size
A = [1.0 Δt;
     0.0 1.0]
b = [0.5*Δt^2; Δt]
Σz = convert(Matrix,Diagonal([0.2*Δt; 0.1*Δt])) # process noise covariance
Σx = convert(Matrix,Diagonal([1.0; 2.0]))     # observation noise covariance;

# Generate noisy observations
n = 10                # perform 10 timesteps
z_start = [10.0; 2.0] # initial state
u = 0.2 * ones(n)     # constant input u
noisy_x = generateNoisyMeasurements(z_start, u, A, b, Σz, Σx);

m_z = noisy_x[1]                                    # initial predictive mean
V_z = A * (1e8*Diagonal(I,2) * A') + Σz             # initial predictive covariance

for t = 2:n
    global m_z, V_z, m_pred_z, V_pred_z
    #predict
    m_pred_z = A * m_z + b * u[t]                   # predictive mean
    V_pred_z = A * V_z * A' + Σz                    # predictive covariance
    #update
    gain = V_pred_z * inv(V_pred_z + Σx)            # Kalman gain
    m_z = m_pred_z + gain * (noisy_x[t] - m_pred_z) # posterior mean update
    V_z = (Diagonal(I,2)-gain)*V_pred_z             # posterior covariance update
end

plotCartPrediction2(m_pred_z[1], V_pred_z[1], m_z[1], V_z[1], noisy_x[n][1], Σx[1][1]);
```

### Extensions of Generative Gaussian Models

- Using the methods of the previous lessons, it is possible to create your own new models based on stacking Gaussian and categorical distributions in new ways:

### Recap Dynamical Models

- Dynamical systems do not obey the sample-by-sample independence assumption, but still can be specified, and state and parameter estimation equations can be solved by similar tools as for static models.

- Two of the more famous and powerful models with latent states include the hidden Markov model (with discrete states) and the Linear Gaussian dynamical system (with continuous states).

- For the LGDS, the Kalman filter is a well-known recursive state estimation procedure. The Kalman filter can be derived through Bayesian update rules on Gaussian distributions.

- If anything changes in the model, e.g., the state noise is not Gaussian, then you have to re-derive the inference equations again from scratch and it may not lead to an analytically pleasing answer.

- $\Rightarrow$ Generally, we will want to automate the inference process. This issue is discussed in the next lesson on inference by message passing in factor graphs.

```julia
open("../../styles/aipstyle.html") do f display("text/html", read(f, String)) end
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>

```julia

```


# Identifying ACR species with Gurobi

In a chemical network equipped with Mass Action kinetics we define chemical species with absolute concentration robustness (ACR) as those whose concentration at steady state is independent of initial conditions. For a given chemical network with stoichiometric matrix $S$, reaction rate $v_j = K_j \prod_i{x_i^{s_{ij}}}$ and given reaction rate constant, $K$, ranges, we can identify ACR species through the following optimization problem:

\begin{align}
    \begin{aligned}
      \label{eq:4}
      &\log{x_{min_i}}, \log{x_{max_i}} = \min,\max_{\substack{ \log{x} \in \rm I\!R^m, \\
                            v,w,\log{K} \in \rm I\!R^n_{\geq 0}}} \; \log{x_i}
      \\
      &\mathrm{s.t.}
      \\
      &1.\;Sv = 0
      \\
      &2.\;w_j = \log{v_j}
      \\
      &3.\;w_j = \log{K_j} + \sum_i s_{ij} \log{x_i}
      \\
      &4.\;\log{x_{min}} \leq \log{x} \leq \log{x_{max}}
      \\
      &5.\;v_{min} \leq v \leq v_{max}
      \\
      &6.\;\log{K_{min}} \leq \log{K} \leq \log{K_{max}}
    \end{aligned}
\end{align}

The key constraint here is constraint 2, in which we relate $w_j = \log{v_j}$. This constraint is implemented in the Gurobi solver with a piece-wise linear approximation of the log function, and it works quite well for large scale chemical networks. Let's try this out in a genome scale metabolic model of _Escherichia coli_. We will find ACR species without any additional constraint and for any positive values of parameters $K$.

```python
from gurobipy import GRB
from importlib import reload
import numpy as np
import pandas as pd
# from matplotlib import pyplot as plt
# import seaborn as sns
import parameters as par
import GEM_model
import gurobi_model

GEM = GEM_model.prepareGEM(path_to_GEM=f'{par.work_directory}/{par.model}',
                      carbon_source=par.carbon_source,
                      uptake_rate=par.uptake_rate,
                      loopless=False,
                      biomass_reaction_id=par.biomass_rxn_id)
GEM
```

    Using license file C:\Users\tinta\gurobi.lic
    Academic license - for non-commercial use only
    Read LP format model from file C:\Users\tinta\AppData\Local\Temp\tmpwu0t9xjb.lp
    Reading time = 0.05 seconds
    : 1147 rows, 4276 columns, 17168 nonzeros

        Name
        iML1515

        Memory address
        0x022b27004a48

        Number of metabolites
        1147

        Number of reactions
        2138

        Number of groups
        0

        Objective expression
        1.0*BIOMASS_Ec_iML1515_core_75p37M - 1.0*BIOMASS_Ec_iML1515_core_75p37M_reverse_35685

        Compartments
        cytosol, extracellular space, periplasm

## Some notes on results
Ok, so the idea was cool. Unfortunately, Gurobi is way to slow to solve this problem. Adding log constraints is fine I don't add the mass-action constraint $Sv=0$. However, requiring steady state plus $w = \log{v}$ is apparently too much to ask...

```python
reload(par)
reload(gurobi_model)
lp_model, variables = gurobi_model.buildLPModel(GEM)
logx_optimum = {}
for met in GEM.metabolites:
    print(met.id)
    logx_optimum[met.id] = {}

    lp_model.setObjective(variables['logx'][f'logx_{met.id}'], GRB.MINIMIZE)
    lp_model.update()
    lp_model.optimize()
    logx_optimum[met.id]['min'] = lp_model.objval

    lp_model.setObjective(variables['logx'][f'logx_{met.id}'], GRB.MAXIMIZE)
    lp_model.update()
    lp_model.optimize()
    logx_optimum[met.id]['max'] = lp_model.objval

```

```python

```


```python
from IPython.display import HTML
from IPython.display import display

display(HTML("<style>.container { width:70% !important; }</style>"))
```

```python
%matplotlib inline
import numpy as np, scipy, scipy.stats as stats, pandas as pd, matplotlib.pyplot as plt, seaborn as sns
import statsmodels, statsmodels.api as sm
import sympy, sympy.stats
import pymc3 as pm
import daft
import xarray, numba, arviz as az

pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
# pd.set_option('display.float_format', lambda x: '%.2f' % x)
np.set_printoptions(edgeitems=10)
np.set_printoptions(linewidth=1000)
np.set_printoptions(suppress=True)
np.core.arrayprint._line_width = 180

SEED = 42
np.random.seed(SEED)

sns.set()

import warnings
warnings.filterwarnings("ignore")
```

This blog post is part of the [Series: Monte Carlo Methods](https://weisser-zwerg.dev/posts/series-monte-carlo-methods/).

You can find this blog post on [weisser-zwerg.dev](https://weisser-zwerg.dev/posts/monte-carlo-markov-chain-monte-carlo/) or on [github](https://github.com/cs224/blog-series-monte-carlo-methods) as either [html](https://rawcdn.githack.com/cs224/blog-series-monte-carlo-methods/main/0020-markov-chain-monte-carlo.html) or via [nbviewer](https://nbviewer.jupyter.org/github/cs224/blog-series-monte-carlo-methods/blob/main/0020-markov-chain-monte-carlo.ipynb?flush_cache=true).

# Markov chain Monte Carlo (MCMC)

Table of Contents

1  Markov chain Monte Carlo

2  Impl

3  NumPyro

4  Resources

## Markov chain Monte Carlo

I will not explain the details of why Markov chain Monte Carlo (MCMC) works, because other people have done already a very good job at it. Have a look below at the [Resources](#Resources) section. I personally found the following two books very helpful, because they explain the mechanics of MCMC in a discrete set-up, which at least for me helps in building an intuition.

* [Machine Learning: A Bayesian and Optimization Perspective](https://www.amazon.com/-/de/dp/0128188030) by [Sergios Theodoridis](https://sergiostheodoridis.wordpress.com/)
    * Chapter 14.7 Markov Chain Monte Carlo Methods
* [Probabilistic Graphical Models: Principles and Techniques](https://www.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193) by Daphne Koller and Nir Friedman

But all the other mentioned references are very helpful, too.

A Markov chain starts with the initial distribution of starting states $\pi_0(X)$. If you start with a concrete single state then this will be a dirac-delta $\pi_0(X) = \delta(X=x_0)$. In addition you have a so called transition operator $\mathcal{T}(x\to x')$. This transition operator is a simple conditional probability just in reverse notation $\mathcal{T}(x\to x')\equiv p(x'\,|\,x)$. In addition this transitio operator is kalled a *kernel* $\mathcal{K}(x\to x')\equiv \mathcal{T}(x\to x')\equiv p(x'\,|\,x)$. If you then apply the transition operator / kernel several times in sequence and marginalize over all earlier state variables then you get a sequence of distributions

$
\begin{array}{rcl}
\pi_1(x')&=&\int\pi_0(x)\cdot p(x'\,|\,x)dx=\int\pi_0(x)\cdot \mathcal{T}(x\to x')dx=\int\pi_0(x)\cdot \mathcal{K}(x\to x')dx=\int p(x, x')dx
\end{array}
$

$\pi_0(X)\to \pi_1(X)\to .... \to \pi_N(X)$.

Just as a recap: the following vocabulary is typically used in conjunction with the conditions that have to hold to make MCMC work:

* [ergodic](https://en.wikipedia.org/wiki/Markov_chain#Ergodicity)
* [homogeneous](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf)
* [stationary](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf) / [invariant](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf)
* [detailed-balance](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf)
* [irreducible](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf)
* [aperiodic](https://www.robots.ox.ac.uk/~fwood/teaching/C19_hilary_2015_2016/mcmc.pdf)
* [regular](https://www.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193)

But in the end the goal is to construct a Markov chain that has the target distribution as its stationary (aka invariant) / equilibrium distribution and will converge to this equilibrium distribution in the long run no matter from where you start.

Stationarity is defined by:

$
\begin{array}{rcl}
\pi_S(X=x')&=&\displaystyle\sum_{x\in Val(X)}\pi_S(X=x)\mathcal{T}(x\to x')\\
&=&\displaystyle\int\pi_S(x)\mathcal{T}(x\to x')dx
\end{array}
$

The first line is for the discrete case and the second one is for the continuous case. The subscript $S$ in $\pi_S$ is for *stationary*.

So in our construction process of a transition operator for a target distribution you start by restricting yourself to transition operators that satisfy detailed balance w.r.t. a particular
target distribution, because then that distribution will be the stationary / invariant distribution.

The following equation is the detailed-balance equation:

$
\begin{array}{rcl}
\displaystyle\pi_S(X=x)\mathcal{T}(x\to x')&=&\displaystyle\pi_S(X=x')\mathcal{T}(x'\to x)\\
\end{array}
$

A Markov chain that respects detailed-balance is said to be reversible. Reversibility implies that $\pi_S$ is a stationary distribution of $\mathcal{T}$

In addition we use *homogeneity*. A Markov chain is *homogeneous* if the transition operators are the same (constant) for every transition we make $\mathcal{T}_1=\mathcal{T}_2=...=\mathcal{T}_N=\mathcal{T}$. It can be shown that a *homogeneous* Markov chain that possesses a *stationary* distribution (guaranteed via *detailed-balance*) will converge to the single *equilibrium* distribution from any starting point, subject only to weak restrictions on the stationary distribution and the transition probabilities ([Neal 1993](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.8183))

In the discrete case *regularity* (see [Probabilistic Graphical Models: Principles and Techniques](https://www.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193)) plus detailed-balance guarantee convergence to its stationary distribution.

My goal for this blog post is to look deeper into how to combine elements of algorithms in the sense of **fine-grained composable abstractions**
([FCA](https://web.archive.org/web/20130117175652/http://blog.getprismatic.com/blog/2012/4/5/software-engineering-at-prismatic.html)), which does not get a lot of attention in the other resources.

A good starting point for that is to look at the detailed balance equation.

## Impl

```python
# Flip coin 9 times and get 6 heads
samples = np.array([0,0,0,1,1,1,1,1,1])

def fn(a, b):
    return lambda p: stats.bernoulli(p).pmf(samples).prod() * stats.beta(a,b).pdf(p)

# convert from omega, kappa parametrization of the beta distribution to the alpha, beta parametrization
def ok2ab(omega, kappa):
    return omega*(kappa-2)+1,(1-omega)*(kappa-2)+1

```

```python
@numba.jit(nopython=True)
def bernoulli(p, samples):
    r = np.zeros_like(samples, dtype=numba.float64)
    for i in range(len(r)):
        if samples[i] < 0.5: # == 0
            r[i] = np.log(1-p)
        else: # == 1
            r[i] = np.log(p)
    return np.sum(r)
```

```python
bernoulli(0.001, samples)
```

```python
# verify that our implementation delivers the same results as the scipy implementation
stats.bernoulli(0.001).logpmf(samples).sum()
```

```python
@numba.jit(nopython=True)
def logpdf(p):
    return bernoulli(p,samples)

@numba.jit(nopython=True)
def mcmc(q_rvs, unif_rvs, trace, logpdf):
    p = 0.5
    for i in range(N):
        rv = q_rvs[i]
        unifrand = unif_rvs[i]
        p_new = p + rv
        log_hastings_ratio = logpdf(p_new) - logpdf(p) # is only correct, because rv is from a symmetric distribution otherwise the "Hastings q(y,x)/q(x,y) is missing"
        if log_hastings_ratio >= 0.0 or unifrand < np.exp(log_hastings_ratio):
            p = p_new
        trace[i] = p
    return trace
```

```python
N = 10000

q_rvs = stats.norm(0,0.3).rvs(size=N, random_state=np.random.RandomState(42))
unif_rvs = stats.uniform.rvs(size=N, random_state=np.random.RandomState(42))
trace = np.zeros(N)
mcmc(q_rvs, unif_rvs, trace, logpdf)
trace
```

```python
datadict = {'p': trace}
dataset = az.convert_to_inference_data(datadict)
dataset
```

```python
az.summary(dataset)
```

```python
az.plot_trace(dataset)
```

```python
def kdeplot(lds, parameter_name=None, x_min = None, x_max = None, ax=None, kernel='gau'):
    if parameter_name is None and isinstance(lds, pd.Series):
        parameter_name = lds.name
    kde = sm.nonparametric.KDEUnivariate(lds)
    kde.fit(kernel=kernel, fft=False, gridsize=2**10)
    if x_min is None:
        x_min = kde.support.min()
    else:
        x_min = min(kde.support.min(), x_min)
    if x_max is None:
        x_max = kde.support.max()
    else:
        x_max = max(kde.support.max(),  x_max)
    x = np.linspace(x_min, x_max,100)
    y = kde.evaluate(x)
    if ax is None:
        plt.figure(figsize=(6, 3), dpi=80, facecolor='w', edgecolor='k')
        ax = plt.subplot(1, 1, 1)
    ax.plot(x, y, lw=2)
    ax.set_xlabel(parameter_name)
    ax.set_ylabel('Density')
```

```python
plt.figure(figsize=(15, 8), dpi=80, facecolor='w', edgecolor='k')
ax = plt.subplot(1, 1, 1)
kdeplot(trace, x_min=0.0, x_max=1.0, ax=ax)
x = np.linspace(0.0,1.0,100)
y = stats.beta(1+6,1+3).pdf(x)
ax.plot(x,y)
```

```python
with pm.Model() as model:
    p = pm.Beta('p', 1.0, 1.0)
    yl = pm.Bernoulli('yl', p, observed=samples)
    prior = pm.sample_prior_predictive()
    posterior = pm.sample(return_inferencedata=True)
    posterior_pred = pm.sample_posterior_predictive(posterior)
```

```python
pm.summary(posterior)
```

```python
ldf = pd.DataFrame(datadict)
ldf['w'] = 1.0/len(ldf)
ldf = ldf.sort_values('p').set_index('p')
ldf['c'] = ldf['w'].cumsum()
ldf1 = ldf
ldf1
```

```python
ldf = posterior['posterior']['p'].loc[dict(chain=0)].to_dataframe()
ldf = ldf.drop('chain', axis=1)
ldf['w'] = 1.0/len(ldf)
ldf = ldf.sort_values('p').set_index('p')
ldf['c'] = ldf['w'].cumsum()
ldf2 = ldf
ldf2
```

```python
plt.figure(figsize=(15, 8), dpi=80, facecolor='w', edgecolor='k')
ax = plt.subplot(1, 1, 1)
x = np.linspace(0.0,1.0,100)
y = stats.beta(1+6,1+3).cdf(x)
ax.plot(x,y, label='exact')
ldf1.loc[0.0:1.0, 'c'].plot(ax=ax, label='self-made MCMC')
ldf2.loc[0.0:1.0, 'c'].plot(ax=ax, label='PyMC3')
ax.legend()
```

## NumPyro

```python
# pip install numpyro[cpu]
import numpyro, numpyro.distributions, numpyro.infer
import jax
numpyro.set_platform('cpu')
numpyro.set_host_device_count(4)
```

```python
def coin_flip_example(y=None):
    p = numpyro.sample('p', numpyro.distributions.Beta(1,1))
    numpyro.sample('obs', numpyro.distributions.Bernoulli(p), obs=y)
```

```python
nuts_kernel = numpyro.infer.NUTS(coin_flip_example)
sample_kwargs = dict(
    sampler=nuts_kernel,
    num_warmup=1000,
    num_samples=1000,
    num_chains=4,
    chain_method="parallel"
)
mcmc = numpyro.infer.MCMC(**sample_kwargs)
rng_key = jax.random.PRNGKey(0)
mcmc.run(rng_key, y=samples, extra_fields=('potential_energy',))
```

```python
mcmc.print_summary()
```

```python
pyro_trace = az.from_numpyro(mcmc)
pyro_trace
```

```python
az.summary(pyro_trace)
```

```python
az.plot_trace(pyro_trace)
```

## Resources

Text books:

* [Handbook of Markov Chain Monte Carlo](https://www.amazon.com/-/de/dp-B008GXJVF8/dp/B008GXJVF8/) by Steve Brooks, Andrew Gelman, Galin Jones, Xiao-Li Meng
* [Machine Learning: A Bayesian and Optimization Perspective](https://www.amazon.com/-/de/dp/0128188030) by [Sergios Theodoridis](https://sergiostheodoridis.wordpress.com/)
    * Chapter 14.7 Markov Chain Monte Carlo Methods
* [Probabilistic Graphical Models: Principles and Techniques](https://www.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193) by Daphne Koller and Nir Friedman
    * Chapter 12.3 Markov Chain Monkte Carlo Methods
* [Bayesian Reasoning and Machine Learning](https://www.amazon.com/Bayesian-Reasoning-Machine-Learning-Barber/dp/0521518148) by David Barber
    * Chapter 27.4 Markov chain Monte Carlo (MCMC)
* [Pattern Recognition and Machine Learning](https://www.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/1493938436) by Christopher M. Bishop
    * Chapter 11.2 Markov Chain Monte Carlo
* [Machine Learning: A Probabilistic Perspective](https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020/)
    * Chapter 24 Markov chain Monte Carlo (MCMC) inference
* [Doing Bayesian Data Analysis](https://www.amazon.com/-/de/dp/0124058884/): A Tutorial with R, JAGS, and Stan by [John Kruschke](http://doingbayesiandataanalysis.blogspot.com/)
    * Chapter 7 Markov Chain Monte Carlo
* [Statistical Rethinking](https://www.amazon.com/-/de/dp/036713991X): A Bayesian Course with Examples in R and STAN by [Richard McElreath](https://elevanth.org/blog/)
    * Chapter 9 Markov Chain Monte Carlo

Tutorial:

* [A Tutorial on Markov Chain Monte-Carlo and Bayesian Modeling](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3759243) by Martin B. Haugh

Blog posts:

* [Series of posts on implementing Hamiltonian Monte Carlo](https://discourse.pymc.io/t/series-of-posts-on-implementing-hamiltonian-monte-carlo/3138) by Colin Carroll
    * [Hamiltonian Monte Carlo from scratch](https://colindcarroll.com/2019/04/11/hamiltonian-monte-carlo-from-scratch/)
    * [Step Size Adaptation in Hamiltonian Monte Carlo](https://colindcarroll.com/2019/04/21/step-size-adaptation-in-hamiltonian-monte-carlo/)
    * [Choice of Symplectic Integrator in Hamiltonian Monte Carlo](https://colindcarroll.com/2019/04/28/choice-of-symplectic-integrator-in-hamiltonian-monte-carlo/)
    * [Pragmatic Probabilistic Programming](https://colcarroll.github.io/hmc_tuning_talk/)
    * [A tour of probabilistic programming language APIs](https://colcarroll.github.io/ppl-api/)
    * [minimc](https://github.com/ColCarroll/minimc) ~15 line Hamiltonian Monte Carlo implementation
    * [Hamiltonian Monte Carlo in PyMC3](https://colcarroll.github.io/hamiltonian_monte_carlo_talk/bayes_talk.html)
* [Markov Chains: Why Walk When You Can Flow?](https://elevanth.org/blog/2017/11/28/build-a-better-markov-chain/) by Richard McElreath
* [Bayesian Inference Algorithms: MCMC and VI](https://towardsdatascience.com/bayesian-inference-algorithms-mcmc-and-vi-a8dad51ad5f5) by Wicaksono Wijono

Papers:

* [Mixed Hamiltonian Monte Carlo for Mixed Discrete and Continuous Variables](https://arxiv.org/abs/1909.04852) by Guangyao Zhou
    * [Guangyao (Stannis) Zhou](https://stanniszhou.github.io/)
    * [mixed_hmc](https://github.com/StannisZhou/mixed_hmc) on GitHub
    * [MixedHMC](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.mixed_hmc.MixedHMC) for NumPyro
    * [YouTube](https://www.youtube.com/watch?v=ag44SuB0wB8)

```python

```


# Selección óptima de portafolios I

En la clase pasada vimos que:
- La LAC describe las posibles selecciones de riesgo-rendimiento entre un activo libre de riesgo y un activo riesgoso.
- Su pendiente es igual al radio de Sharpe del activo riesgoso.
- La asignación óptima de capital para cualquier inversionista es el punto tangente de la curva de indiferencia del inversionista con la LAC (depende de las preferencias particulares - aversión al riesgo).

Para todo lo anterior, supusimos que ya teníamos el portafolio óptimo (activo riesgoso).

En el siguiente análisis:

**Objetivos:**
- ¿Cuál es el portafolio óptimo de activos riesgosos?
 - ¿Cuál es el mejor portafolio de activos riesgosos?
 - Es un portafolio eficiente en media-varianza.
- Problema: dado un conjunto de activos riesgosos, ¿cómo construimos la mejor combinación?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Maximizando el radio de Sharpe

### ¿Qué pasa si tenemos dos activos riesgosos?

Cuando tenemos dos activos riesgosos, tenemos disponibles diferentes LAC. ¿Qué significan sus pendientes?

 Ver en el tablero.

Pregunta:
- ¿Qué es lo que se quiere?

**Conclusión:**
- el *portafolio tangente* de la LAC más pendiente;
- por tanto, maximiza el radio de Sharpe para todos los posibles portafolios de activos riesgosos;
- este portafolio se conoce como el portafolio eficiente en media-varianza (EMV).

**Idea principal: el portafolio óptimo de activos riesgosos es independiente de las preferencias del inversionista.**
- El portafolio EMV determina el portafolio óptimo de activos riesgosos.
- Todos tendremos el mismo portafolio de activos riesgosos (EMV), y lo combinaremos con el activo libre de reisgo de acuerdo con las preferencias de cada uno de nosotros (aversión al riesgo).
- La LAC combinando el activo libre de riesgo y el portafolio EMV, se vuelve el conjunto de portafolios eficientes.

Entonces, se deben seguir los siguientes pasos:
1. Crear la frontera media-varianza.
2. Encontrar el portafolio que maximize el radio de Sharpe (portafolio EMV).
3. Construir la frontera eficiente (LAC) del punto $(0,r_f)$ al punto $(E[r_s],\sigma_s)$ del portafolio EMV.
4. Combinar de acuerdo a sus preferencias.
___

## 2. Solución analítica del portafolio EMV: caso con dos activos.

Queremos solucionar el siguiente problema:

\begin{align}
\max_{w_1,w_2} &\quad \frac{E[r_p]-r_f}{\sigma_p}\\
\text{s.a.}    &\quad E[r_p]=w_1E[r_1]+w_2E[r_2]\\
               &\quad \sigma_p=\sqrt{w_1^2\sigma_1^2+w_2^2\sigma_2^2+2w_1w_2\rho_{12}\sigma_1\sigma_2}\\
               &\quad w_1+w_2=1, \quad w_1,w_2\geq0
\end{align}

el cual es equivalente a

\begin{align}
\max_{w_1} &\quad \frac{w_1E[r_1]+(1-w_1)E[r_2]-r_f}{\sqrt{w_1^2\sigma_1^2+(1-w_1)^2\sigma_2^2+2w_1(1-w_1)\rho_{12}\sigma_1\sigma_2}}\\
\text{s.a.}    &\quad 0\leq w_1\leq1
\end{align}

**Actividad.**
El anterior es un problema de maximizar una función de una variable en un dominio cerrado. No debaría representar dificultad.

Encontrar la solución analítica a este problema.

Quien primero lo haga, y salga a explicarlo al tablero, le subo alguna tarea, quiz o el primer examen a 100.

Deben llegar a:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

Si nadie lo ha hecho en 30 min., procederé a hacerlo yo.

**Nota:**
- así como obtuvimos una expresión para el peso del portafolio de mínima varianza con dos activos, obtenemos una expresión para el peso del portafolio Eficiente en Media-Varianza.
- Estas actividades son sin duda un buen ejercicio, y se pueden replicar usando técnicas de varias variables (multiplicadores de Lagrange) cuando se tengan más de dos activos.
- Sin embargo, la complejidad del problema crece considerablemente con el número de variables, y la solución analítica deja de ser viable cuando mencionamos que un portafolio bien diversificado consta aproximadamente de 50-60 activos.
- En esos casos, este problema se soluciona con rutinas numéricas que hagan la optimización por nosotros.
- Por eso, les enseño cómo resolver este problema con optimizadores numéricos, porque son una solución viable y escalable a más variables.

el mejor portafolio es el mismo para todos sin importar las preferencias y aversiones al riesgo

## 3. Ejemplo ilustrativo.

Retomamos el ejemplo de mercados de acciones en los países integrantes del $G5$: EU, RU, Francia, Alemania y Japón.

```python
# Importamos pandas y numpy
import pandas as pd
import numpy as np
```

```python
# Resumen en base anual de rendimientos esperados y volatilidades
annual_ret_summ = pd.DataFrame(columns=['EU', 'RU', 'Francia', 'Alemania', 'Japon'], index=['Media', 'Volatilidad'])
annual_ret_summ.loc['Media'] = np.array([0.1355, 0.1589, 0.1519, 0.1435, 0.1497])
annual_ret_summ.loc['Volatilidad'] = np.array([0.1535, 0.2430, 0.2324, 0.2038, 0.2298])

annual_ret_summ.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      Media
      0.1355
      0.1589
      0.1519
      0.1435
      0.1497

      Volatilidad
      0.1535
      0.243
      0.2324
      0.2038
      0.2298

```python
# Matriz de correlación
corr = pd.DataFrame(data= np.array([[1.0000, 0.5003, 0.4398, 0.3681, 0.2663],
                                    [0.5003, 1.0000, 0.5420, 0.4265, 0.3581],
                                    [0.4398, 0.5420, 1.0000, 0.6032, 0.3923],
                                    [0.3681, 0.4265, 0.6032, 1.0000, 0.3663],
                                    [0.2663, 0.3581, 0.3923, 0.3663, 1.0000]]),
                    columns=annual_ret_summ.columns, index=annual_ret_summ.columns)
corr.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      EU
      1.0000
      0.5003
      0.4398
      0.3681
      0.2663

      RU
      0.5003
      1.0000
      0.5420
      0.4265
      0.3581

      Francia
      0.4398
      0.5420
      1.0000
      0.6032
      0.3923

      Alemania
      0.3681
      0.4265
      0.6032
      1.0000
      0.3663

      Japon
      0.2663
      0.3581
      0.3923
      0.3663
      1.0000

Supondremos, además, que la tasa libre de riesgo es $r_f=5\%$.

```python
# Tasa libre de riesgo
rf = 0.05
```

Entonces, supondremos que tenemos disponibles los activos correspondientes a los mercados de acciones de EU y Japón, y en adición el activo libre de riesgo.

#### 1. Construir la frontera de mínima varianza

```python
# Vector de w variando entre 0 y 1 con n pasos
n=101
w=np.linspace(0,1,n)
# Rendimientos esperados individuales
# Activo1: EU, Activo2:Japon
e1=annual_ret_summ['EU']['Media']
e2=annual_ret_summ['Japon']['Media']
# Volatilidades individuales
s1=annual_ret_summ['EU']['Volatilidad']
s2=annual_ret_summ['Japon']['Volatilidad']
# Correlacion
r12=corr['EU']['Japon']
```

```python
# DataFrame de portafolios:
# 1. Índice: i
# 2. Columnas 1-2: w, 1-w
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
portafolios=pd.DataFrame(index=np.arange(n),columns=['w','1-w','E[r](ret)','vol','SR'])
portafolios['w']=w
portafolios['1-w']=1-w
portafolios['E[r](ret)']=w*e1+(1-w)*e2
portafolios['vol']=np.sqrt((w*s1)**2+((1-w)*s2)**2+2*w*(1-w)*r12*s1*s2)
portafolios['SR']=(portafolios['E[r](ret)']-rf)/portafolios['vol']
portafolios
```

      w
      1-w
      E[r](ret)
      vol
      SR

      0
      0.00
      1.00
      0.149700
      0.229800
      0.433856

      1
      0.01
      0.99
      0.149558
      0.227916
      0.436820

      2
      0.02
      0.98
      0.149416
      0.226041
      0.439814

      3
      0.03
      0.97
      0.149274
      0.224176
      0.442839

      4
      0.04
      0.96
      0.149132
      0.222322
      0.445894

      5
      0.05
      0.95
      0.148990
      0.220478
      0.448979

      6
      0.06
      0.94
      0.148848
      0.218645
      0.452094

      7
      0.07
      0.93
      0.148706
      0.216823
      0.455238

      8
      0.08
      0.92
      0.148564
      0.215012
      0.458411

      9
      0.09
      0.91
      0.148422
      0.213213
      0.461613

      10
      0.10
      0.90
      0.148280
      0.211426
      0.464843

      11
      0.11
      0.89
      0.148138
      0.209651
      0.468101

      12
      0.12
      0.88
      0.147996
      0.207889
      0.471387

      13
      0.13
      0.87
      0.147854
      0.206139
      0.474698

      14
      0.14
      0.86
      0.147712
      0.204403
      0.478036

      15
      0.15
      0.85
      0.147570
      0.202680
      0.481398

      16
      0.16
      0.84
      0.147428
      0.200971
      0.484785

      17
      0.17
      0.83
      0.147286
      0.199277
      0.488195

      18
      0.18
      0.82
      0.147144
      0.197597
      0.491627

      19
      0.19
      0.81
      0.147002
      0.195932
      0.495080

      20
      0.20
      0.80
      0.146860
      0.194282
      0.498553

      21
      0.21
      0.79
      0.146718
      0.192648
      0.502044

      22
      0.22
      0.78
      0.146576
      0.191031
      0.505553

      23
      0.23
      0.77
      0.146434
      0.189429
      0.509076

      24
      0.24
      0.76
      0.146292
      0.187845
      0.512613

      25
      0.25
      0.75
      0.146150
      0.186279
      0.516162

      26
      0.26
      0.74
      0.146008
      0.184730
      0.519721

      27
      0.27
      0.73
      0.145866
      0.183199
      0.523288

      28
      0.28
      0.72
      0.145724
      0.181688
      0.526860

      29
      0.29
      0.71
      0.145582
      0.180195
      0.530435

      ...
      ...
      ...
      ...
      ...
      ...

      71
      0.71
      0.29
      0.139618
      0.142081
      0.630751

      72
      0.72
      0.28
      0.139476
      0.141924
      0.630452

      73
      0.73
      0.27
      0.139334
      0.141806
      0.629973

      74
      0.74
      0.26
      0.139192
      0.141729
      0.629313

      75
      0.75
      0.25
      0.139050
      0.141693
      0.628472

      76
      0.76
      0.24
      0.138908
      0.141697
      0.627451

      77
      0.77
      0.23
      0.138766
      0.141742
      0.626250

      78
      0.78
      0.22
      0.138624
      0.141828
      0.624872

      79
      0.79
      0.21
      0.138482
      0.141954
      0.623317

      80
      0.80
      0.20
      0.138340
      0.142120
      0.621587

      81
      0.81
      0.19
      0.138198
      0.142327
      0.619687

      82
      0.82
      0.18
      0.138056
      0.142574
      0.617618

      83
      0.83
      0.17
      0.137914
      0.142860
      0.615385

      84
      0.84
      0.16
      0.137772
      0.143187
      0.612990

      85
      0.85
      0.15
      0.137630
      0.143552
      0.610440

      86
      0.86
      0.14
      0.137488
      0.143957
      0.607737

      87
      0.87
      0.13
      0.137346
      0.144401
      0.604887

      88
      0.88
      0.12
      0.137204
      0.144883
      0.601894

      89
      0.89
      0.11
      0.137062
      0.145403
      0.598765

      90
      0.90
      0.10
      0.136920
      0.145960
      0.595505

      91
      0.91
      0.09
      0.136778
      0.146555
      0.592120

      92
      0.92
      0.08
      0.136636
      0.147186
      0.588615

      93
      0.93
      0.07
      0.136494
      0.147854
      0.584996

      94
      0.94
      0.06
      0.136352
      0.148557
      0.581270

      95
      0.95
      0.05
      0.136210
      0.149296
      0.577443

      96
      0.96
      0.04
      0.136068
      0.150070
      0.573521

      97
      0.97
      0.03
      0.135926
      0.150877
      0.569509

      98
      0.98
      0.02
      0.135784
      0.151719
      0.565415

      99
      0.99
      0.01
      0.135642
      0.152593
      0.561244

      100
      1.00
      0.00
      0.135500
      0.153500
      0.557003

101 rows × 5 columns

```python
# Importar librerías de gráficos
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR
plt.figure(figsize=(7,5))
plt.scatter(portafolios['vol'],portafolios['E[r](ret)'],c=portafolios['SR'],cmap='RdYlBu')
plt.xlabel('Volatilidad')
plt.ylabel('rendimiento esperado')
plt.colorbar()
plt.show()
```

#### 2. Encontrar el portafolio que maximiza el radio de Sharpe (EMV)

Primero, encontramos este portafolio con la fórmula que obtuvimos:

```python
# Fórmula que obtuvimos
wEMV=((e1-rf)*s2**2-(e2-rf)*r12*s1*s2)/((e2-rf)*s1**2+(e1-rf)*s2**2-((e1-rf)+(e2-rf))*r12*s1*s2)
wEMV
```

    0.6983139170512034

Ahora sí, con la función scipy.optimize.minimize

```python
# Importar el módulo optimize
from scipy.optimize import minimize
```

```python
## Construcción de parámetros
## Activo 1: EU, Activo 2: Japon
# 1. Sigma: matriz de varianza-covarianza
sigma=np.array([[s1**2,     r12*s1*s2],
                [r12*s1*s2, s2**2]])
# 2. Eind: rendimientos esperados activos individuales
Eind=np.array([e1,e2])
```

```python
# Función objetivo (-SR) para maximizar
def obj_func(w,Eind,sigma,rf):
    Erp=Eind.dot(w)
    sp=np.sqrt(w.dot(sigma).dot(w))
    Sharpe=(Erp-rf)/sp
    return -Sharpe
```

```python
# Dato inicial
w0=np.ones(2)/2
# Cotas de las variables
bnds=((0,1),)*2
# Restricciones
cons=({'type': 'eq', 'fun': lambda w: np.sum(w)-1})
```

```python
# Optimización numérica
PortEMV=minimize(obj_func,w0,args=(Eind,sigma,rf),bounds=bnds,constraints=cons)
# Resultado
PortEMV
```

         fun: -0.6308725303331167
         jac: array([-0.35171176, -0.35029621])
     message: 'Optimization terminated successfully.'
        nfev: 17
         nit: 4
        njev: 4
      status: 0
     success: True
           x: array([0.69830749, 0.30169251])

Con lo anterior, podemos obtener datos de rendimiento esperado y volatilidad del portafolio EMV

```python
# Rendimiento esperado y volatilidad del portafolio EMV
wEMV=PortEMV.x
EpEMV=wEMV.dot(Eind)
spEMV=np.sqrt(wEMV.dot(sigma).dot(wEMV))
EpEMV,spEMV
```

    (0.1397840336419269, 0.14231723418757614)

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, y portafolio EMV
plt.figure(figsize=(7,5))
plt.scatter(portafolios['vol'],portafolios['E[r](ret)'],c=portafolios['SR'],cmap='RdYlBu')
plt.plot(spEMV,EpEMV,'*g',ms=10)

plt.xlabel('Volatilidad')
plt.ylabel('rendimiento esperado')
plt.colorbar()
plt.show()
```

#### 3. Construir LAC

Ahora, dibujamos la LAC, combinando el portafolio EMV con el activo libre de riesgo:

```python
# Vector de wp variando entre 0 y 1.5 con n pasos
wp=np.linspace(0,1.5,n)
```

```python
# DataFrame de CAL:
# 1. Índice: i
# 2. Columnas 1-2: wp, wrf
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
LAC=pd.DataFrame(index=np.arange(n),columns=['wp','wrf','E[r]','vol','SR'])
LAC['wp']=wp
LAC['wrf']=1-wp
LAC['E[r]']=wp*EpEMV+(1-wp)*rf
LAC['vol']=wp*spEMV
LAC['SR']=(LAC['E[r]']-rf)/LAC['vol']
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, portafolio EMV y LAC
plt.figure(figsize=(7,5))
plt.scatter(portafolios['vol'],portafolios['E[r](ret)'],c=portafolios['SR'],cmap='RdYlBu')
plt.plot(spEMV,EpEMV,'*g',ms=20)
plt.plot(LAC['vol'],LAC['E[r]'],'k--',lw=3)
plt.xlabel('Volatilidad')
plt.ylabel('rendimiento esperado')
plt.colorbar()
plt.xlim([.13,.16])
plt.ylim([.13,.15])
plt.show()
```

#### 4. Combinación óptima de acuerdo a preferencias

Con los datos anteriores, y la caracterización de aversión al riesgo, se escoge la combinación óptima entre el portafolio EMV y el activo libre de riesgo de acuerdo a:

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}.$$

```python
# Para gamma=7
(EpEMV-rf)/(7*(spEMV**2))

```

    0.6332658704683628

Created with Jupyter by Esteban Jiménez Rodríguez.


# Deep Reinforcement Learning Tutorial

## Introduction

### Types of ML
#### Supervised and unsupervised learning
The loss or objective function that we want to optimise for is known in both of these settings. For supervised learning we have a dataset of *(input, output)* pairs, our models make predictions for the output values based on the inputs and then a loss function scores how good a prediction this was compared to the *correct* output.

Similarly for unsupervised learning, while there isn't a predefined output label for each input, there is a known objective function that scores some derived property of the data. E.g. word embeddings are trained to minimise the distance between the vectors representing similar words, or for language modelling there is a loss function (softmax cross-entropy) scoring how well the model can predict each word in the dataset from the words that came before it.

#### Reinforcement learning
Unlike the previous two scenarios, there isn't a known objective function that we can directly optimise for. RL problems are usually framed as an agent interacting with an environment. The agent observes states, takes actions and recieves rewards from the environment but the inner workings or the true dynamics driving the environment cannot be directly observed.

The agent observes the current state of the environment and chooses and action, in response to which the environment provides a reward and the next state that result from the action. RL algorithms try to maximise the rewards for the agent by learning better and better choices of actions.

Both the agent and environment can be stochastic.

## The environment, states and actions

The environment is generally regarded as a black box in RL. As mentioned above the agent only observes the state $s_t$ returned by the environment and chooses and action $a_t$ to perform from among a pre-defined set of actions. After taking an action the environment is updated and returns the next state $s_{t+1}$ and a reward value $r_t$. This is repeated until the environment reaches a terminal state, at which point the environment is reset to some inital state and the process starts again. A series of steps from inital state to terminal state is called an episode.

## Rewards, policies and value functions

### Rewards and returns

The agent recieves a reward value after each step in the environment. The total return at a time $t$ is $R_t = \sum^\infty_{k=0} \gamma^k r_{t+k}$, the discounted sum of all future rewards using a discount factor $\gamma$ (e.g. 0.99). This way the return at each time step captures the rewards from subsequent actions and by keeping track of these an agent can choose between actions by considering which one it expects to get the best return for.

### Policies

A policy $\pi(s)$ is the probability of taking an action $a$ at state $s$, $\pi(s) = p(a|s)$. Some RL methods explicitly represent the policy and improve it directly but Q-learning uses what is called a Q-function that calculates the expected rewards of states and actions and only implicitly represents the notion of a policy.

### Experiences

As the agent ineracts with the environment it creates a sequence of states, actions and rewards, referred to as a trajectory $[s_0, a_0, r_0, s_1, a_1, r_1, ..., s_\$]$. Most RL algorithms work on subsequences of trajectories, for example the tuple $(s,a,r,s')$ covering a current state, chosen action, observed reward and the next state that resulted. These experiences could cover multiple time steps but here we will only consider a single step and its resulting next state.

### The Q function
For a given policy, the state-action value function $Q(s, a)$ represents the value of all future rewards from taking action $a$ at state $s$ and then continuing to the follow the policy.
\begin{align}
Q^\pi(s_t,a_t) & = \mathbb{E}\bigl[r_t + \gamma r_{t+1} + \gamma^2 r_{t+2} + ... \big|  \pi \bigr]\\
& = \mathbb{E}\bigl[r_t + \gamma \max_{a'}Q^\pi(s_{t+1},a') \big|  \pi \bigr]
\end{align}

$Q$ is the agent's best current estimate of the value of an action at the current state and can be recursively defined as the expected value of the current reward plus the discounted estimate of the best next action at the next state (resulting from the chosen action $a$). This function can be learnt iteratively by repeatedly playing the game to sample many $(s, a, r, s')$ experiences and then using those experiences to improve the estimates of $Q$.

The policy $\pi$ can be extracted from $Q$ by always taking the action with the maximal expected value at each time step.

### Q-learning loss

In Q-learning the loss is an expectation over $(s,a,r,s')$ experiences of the squared error between the current value estimate and the maximal value estimate at the next state.
\begin{equation*}
L_i(\theta_i) = \mathbb{E}_{(s,a,r,s')}\bigl[\bigl(r + \gamma \max_{a'}Q(s',a') - Q(s, a)\bigr)^2\bigr]
\end{equation*}

The Q function can be approximated by a neural network, which is usually configured to output the estimated value for all actions at the current state in one go: `action_values = model(curent_state)`. This neural network is updated using the loss function above. The $Q$ estimates improve as more and more experiences are sampled. This is because after a very strong reward signal has been experienced at $(s', a^\text{bad})$, the model will have knowledge of the bad value of $Q(s', a^\text{bad})$ the next time it experiences $(s,a,r,s')$, and this information therefore will be propagated backwards.

### Experience replay

If we just use the current experience $(s,a,r,s')$ to update our Q function then it could suffer from overfitting to just the local parts of the state space that the agent is currently in, as each experience will be highly correlated to those around it. The DQN paper introduced the idea of an experience replay buffer to try to overcome this by calculating each Q function update based on a large number of experiences at once. Each experience the agent observes while playing the game is added to the buffer and instead of calculating the loss for the current experience, a *batch* of experiences sampled from the buffer is used instead. The function is updated using the average loss across all examples in the batch.

## Simplified DQN algorithm

```
Initialise replay buffer D
Initialise Q function with random weights w
for episode = 1, M:
    Initialise the environment and the initial state
    for t = 1, T:
        With probability eps selct a random action a_t
        otherwise select a_t = argmax(Q(state)) over possible actions a
        Execute action a_t in the environment and observe next_state, r_t, done
        Store (state, a_t, r_t, next_state, done) in D
        Sample a batch_size batch of experiences (s_j, a_j, r_j, s_j+1, done) from D
        Set loss = 0
        For each experience in batch:
            set y = r_j if done else
                    r_j + gamma * max(Q(s_j+1))
            loss += (y_j - Q(s_j)[a_j])**2
        loss /= batch_size
        Perform a gradient update step on the loss wrt the weights w
        if done: break
```

## Further reading

* [Reinforcement Learning: An Introduction](http://incompleteideas.net/book/bookdraft2017nov5.pdf), Sutton and Barto, 2014. A text book covering almost all aspects of RL theory.
* [Human Level Control Through Deep Reinforcement Learning](https://deepmind.com/research/publications/human-level-control-through-deep-reinforcement-learning/), Mnih et al., 2015. The original paper that used deep convolutional networks to approximate the Q function and learn to play Atari games.
* [Asynchronous Methods for Deep Reinforcement Learning](https://arxiv.org/abs/1602.01783), Mnih et al., 2016. An alternative to Q-learning, this paper also includes lots of information on Q learning as well as introducing an asynchronous version of Actor-Critic, a powerful alternative to DQN.
* [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://arxiv.org/abs/1710.02298), Hessel et al., 2017. This paper throws together all the most promising advances since DQN and evaluates them. Also includes good introductions to DQN and the subsequent advances.

```python

```


```python
import os
import numpy as np
import math
import sympy as sp
from numpy.linalg import eig

Ka = sp.Matrix ([[1,0,-1,0],
                 [0,0,0,0],
                 [-1,0,1,0],
                 [0,0,0,0]
                 ])
Kb = 1/2* sp.Matrix ([[1,1,-1,-1],
                 [1,1,-1,-1],
                 [-1,-1,1,1],
                 [-1,-1,1,1]
                 ])
Kc = sp.Matrix ([[0,0,0,0],
                 [0,1,0,-1],
                 [0,0,0,0],
                 [0,1,0,-1]
                 ])

Kd =1/2* sp.Matrix ([[1,-1,1,-1],
                 [-1,1,-1,1],
                 [1,-1,1,-1],
                 [-1,1,-1,1]
                 ])
C1 = sp.Symbol('C_1')
C2 = sp.Symbol('C_2')
C3 = sp.Symbol('C_3')
C4 = sp.Symbol('C_4')
C5 = sp.Symbol('C_5')
C6 = sp.Symbol('C_6')
C7 = sp.Symbol('C_7')
C8 = sp.Symbol('C_8')
C9 = sp.Symbol('C_9')

F1x = sp.Symbol('F_1x')
F1y = sp.Symbol('F_1y')
F2x = sp.Symbol('F_2x')
F2y = sp.Symbol('F_2y')
F3x = sp.Symbol('F_3x')
F3y = sp.Symbol('F_3y')
F4x = sp.Symbol('F_4x')
F4y = sp.Symbol('F_4y')
F6x = sp.Symbol('F_6x')
F6y = sp.Symbol('F_6y')

u1x = sp.Symbol('u_1x')
u1y = sp.Symbol('u_1y')
u2x = sp.Symbol('u_2x')
u2y = sp.Symbol('u_2y')
u3x = sp.Symbol('u_3x')
u3y = sp.Symbol('u_3y')
u4x = sp.Symbol('u_4x')
u4y = sp.Symbol('u_4y')
u6x = sp.Symbol('u_6x')
u6y = sp.Symbol('u_6y')

H = sp.Symbol('H')

f_c = [H, 0]
d0 = [u1x, u1y, u2x, u2y, u3x, u3y, u4x, u4y, u6x, u6y]

items = [[1,1,2,Ka*C1],
         [2,1,3,Kb*C2],
         [3,2,3,Kc*C3],
         [4,2,4,Ka*C4],
         [5,3,4,Kd*C5],
         [6,3,5,Ka*C6],
         [7,4,5,Kc*C7],
         [8,4,6,Ka*C8],
         [9,5,6,Kd*C9]]

```

```python
Ksystem = sp.zeros(12,12)
j=0
for i in items:
  #print(i[3].shape)
  #print(Ksystem[1])

  Ksystem[(i[1]-1)*2*12+(i[1]-1)*2] += i[3][0]
  Ksystem[(i[1]-1)*2*12+(i[1]-1)*2+1] += i[3][1]
  Ksystem[(i[1]-1)*2*12+(i[2]-1)*2] += i[3] [2]
  Ksystem[(i[1]-1)*2*12+(i[2]-1)*2+1] += i[3] [3]
  Ksystem[(i[1]-1)*2*12+(i[1]-1)*2+12] += i[3] [4]
  Ksystem[(i[1]-1)*2*12+(i[1]-1)*2+13] += i[3] [5]
  Ksystem[(i[1]-1)*2*12+(i[2]-1)*2+12] += i[3] [6]
  Ksystem[(i[1]-1)*2*12+(i[2]-1)*2+13] += i[3] [7]

  Ksystem[(i[2]-1)*2*12+(i[1]-1)*2] += i[3][8]
  Ksystem[(i[2]-1)*2*12+(i[1]-1)*2+1] += i[3][9]
  Ksystem[(i[2]-1)*2*12+(i[2]-1)*2] += i[3] [10]
  Ksystem[(i[2]-1)*2*12+(i[2]-1)*2+1] += i[3] [11]
  Ksystem[(i[2]-1)*2*12+(i[1]-1)*2+12] += i[3] [12]
  Ksystem[(i[2]-1)*2*12+(i[1]-1)*2+13] += i[3] [13]
  Ksystem[(i[2]-1)*2*12+(i[2]-1)*2+12] += i[3] [14]
  Ksystem[(i[2]-1)*2*12+(i[2]-1)*2+13] += i[3] [15]

for k in range(0, Ksystem.rows):
  print(Ksystem.row(k))

string = ""
for i in range(0, Ksystem.rows):

  for j in range(0,Ksystem.cols):
    string += str(Ksystem[i*12+j])
    string += " & "
  string=string[0:len(string)-2]
  string+= "\\\\"
  string+= "\n"

print(string)

```

    Matrix([[C_1 + 0.5*C_2, 0.5*C_2, -C_1, 0, -0.5*C_2, -0.5*C_2, 0, 0, 0, 0, 0, 0]])
    Matrix([[0.5*C_2, 0.5*C_2, 0, 0, -0.5*C_2, -0.5*C_2, 0, 0, 0, 0, 0, 0]])
    Matrix([[-C_1, 0, C_1 + C_4, 0, 0, 0, -C_4, 0, 0, 0, 0, 0]])
    Matrix([[0, 0, 0, C_3, 0, -C_3, 0, 0, 0, 0, 0, 0]])
    Matrix([[-0.5*C_2, -0.5*C_2, 0, 0, 0.5*C_2 + 0.5*C_5 + C_6, 0.5*C_2 - 0.5*C_5, 0.5*C_5, -0.5*C_5, -C_6, 0, 0, 0]])
    Matrix([[-0.5*C_2, -0.5*C_2, 0, C_3, 0.5*C_2 - 0.5*C_5, 0.5*C_2 - C_3 + 0.5*C_5, -0.5*C_5, 0.5*C_5, 0, 0, 0, 0]])
    Matrix([[0, 0, -C_4, 0, 0.5*C_5, -0.5*C_5, C_4 + 0.5*C_5 + C_8, -0.5*C_5, 0, 0, -C_8, 0]])
    Matrix([[0, 0, 0, 0, -0.5*C_5, 0.5*C_5, -0.5*C_5, 0.5*C_5 + C_7, 0, -C_7, 0, 0]])
    Matrix([[0, 0, 0, 0, -C_6, 0, 0, 0, C_6 + 0.5*C_9, -0.5*C_9, 0.5*C_9, -0.5*C_9]])
    Matrix([[0, 0, 0, 0, 0, 0, 0, C_7, -0.5*C_9, -C_7 + 0.5*C_9, -0.5*C_9, 0.5*C_9]])
    Matrix([[0, 0, 0, 0, 0, 0, -C_8, 0, 0.5*C_9, -0.5*C_9, C_8 + 0.5*C_9, -0.5*C_9]])
    Matrix([[0, 0, 0, 0, 0, 0, 0, 0, -0.5*C_9, 0.5*C_9, -0.5*C_9, 0.5*C_9]])
    C_1 + 0.5*C_2 & 0.5*C_2 & -C_1 & 0 & -0.5*C_2 & -0.5*C_2 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0.5*C_2 & 0.5*C_2 & 0 & 0 & -0.5*C_2 & -0.5*C_2 & 0 & 0 & 0 & 0 & 0 & 0 \\
    -C_1 & 0 & C_1 + C_4 & 0 & 0 & 0 & -C_4 & 0 & 0 & 0 & 0 & 0 \\
    0 & 0 & 0 & C_3 & 0 & -C_3 & 0 & 0 & 0 & 0 & 0 & 0 \\
    -0.5*C_2 & -0.5*C_2 & 0 & 0 & 0.5*C_2 + 0.5*C_5 + C_6 & 0.5*C_2 - 0.5*C_5 & 0.5*C_5 & -0.5*C_5 & -C_6 & 0 & 0 & 0 \\
    -0.5*C_2 & -0.5*C_2 & 0 & C_3 & 0.5*C_2 - 0.5*C_5 & 0.5*C_2 - C_3 + 0.5*C_5 & -0.5*C_5 & 0.5*C_5 & 0 & 0 & 0 & 0 \\
    0 & 0 & -C_4 & 0 & 0.5*C_5 & -0.5*C_5 & C_4 + 0.5*C_5 + C_8 & -0.5*C_5 & 0 & 0 & -C_8 & 0 \\
    0 & 0 & 0 & 0 & -0.5*C_5 & 0.5*C_5 & -0.5*C_5 & 0.5*C_5 + C_7 & 0 & -C_7 & 0 & 0 \\
    0 & 0 & 0 & 0 & -C_6 & 0 & 0 & 0 & C_6 + 0.5*C_9 & -0.5*C_9 & 0.5*C_9 & -0.5*C_9 \\
    0 & 0 & 0 & 0 & 0 & 0 & 0 & C_7 & -0.5*C_9 & -C_7 + 0.5*C_9 & -0.5*C_9 & 0.5*C_9 \\
    0 & 0 & 0 & 0 & 0 & 0 & -C_8 & 0 & 0.5*C_9 & -0.5*C_9 & C_8 + 0.5*C_9 & -0.5*C_9 \\
    0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & -0.5*C_9 & 0.5*C_9 & -0.5*C_9 & 0.5*C_9 \\

```python
K_cc = Ksystem[8:10,8:10]
print(K_cc)
```

    Matrix([[C_6 + 0.5*C_9, -0.5*C_9], [-0.5*C_9, -C_7 + 0.5*C_9]])

```python
K_co1 = Ksystem[0:9,:]
K_co2 = Ksystem[10:12,:]
##K_co.append(Ksystem[11:12,:])

```

```python

for k in range(0, K_co.rows):
  print(K_co.row(k))
  print(len(K_co.row(k)))
```

    Matrix([[C_1 + 0.5*C_2, 0.5*C_2, -C_1, 0, -0.5*C_2, -0.5*C_2, 0, 0]])
    8
    Matrix([[0.5*C_2, 0.5*C_2, 0, 0, -0.5*C_2, -0.5*C_2, 0, 0]])
    8
    Matrix([[-C_1, 0, C_1 + C_4, 0, 0, 0, -C_4, 0]])
    8
    Matrix([[0, 0, 0, C_3, 0, -C_3, 0, 0]])
    8
    Matrix([[-0.5*C_2, -0.5*C_2, 0, 0, 0.5*C_2 + 0.5*C_5 + C_6, 0.5*C_2 - 0.5*C_5, 0.5*C_5, -0.5*C_5]])
    8
    Matrix([[-0.5*C_2, -0.5*C_2, 0, C_3, 0.5*C_2 - 0.5*C_5, 0.5*C_2 - C_3 + 0.5*C_5, -0.5*C_5, 0.5*C_5]])
    8
    Matrix([[0, 0, -C_4, 0, 0.5*C_5, -0.5*C_5, C_4 + 0.5*C_5 + C_8, -0.5*C_5]])
    8
    Matrix([[0, 0, 0, 0, -0.5*C_5, 0.5*C_5, -0.5*C_5, 0.5*C_5 + C_7]])
    8

```python

for k in range(0, K_co2.rows):
  print(K_co2.row(k))
  print(len(K_co2.row(k)))
```

    Matrix([[0, 0, 0, 0, 0, 0, -C_8, 0, 0.5*C_9, -0.5*C_9, C_8 + 0.5*C_9, -0.5*C_9]])
    12
    Matrix([[0, 0, 0, 0, 0, 0, 0, 0, -0.5*C_9, 0.5*C_9, -0.5*C_9, 0.5*C_9]])
    12

```python
K_c01 = np.concatenate((Ksystem[:,0:8], Ksystem[:, 10:12]), axis=1)
print(K_c01)
```

    [[C_1 + 0.5*C_2 0.5*C_2 -C_1 0 -0.5*C_2 -0.5*C_2 0 0 0 0]
     [0.5*C_2 0.5*C_2 0 0 -0.5*C_2 -0.5*C_2 0 0 0 0]
     [-C_1 0 C_1 + C_4 0 0 0 -C_4 0 0 0]
     [0 0 0 C_3 0 -C_3 0 0 0 0]
     [-0.5*C_2 -0.5*C_2 0 0 0.5*C_2 + 0.5*C_5 + C_6 0.5*C_2 - 0.5*C_5 0.5*C_5
      -0.5*C_5 0 0]
     [-0.5*C_2 -0.5*C_2 0 C_3 0.5*C_2 - 0.5*C_5 0.5*C_2 - C_3 + 0.5*C_5
      -0.5*C_5 0.5*C_5 0 0]
     [0 0 -C_4 0 0.5*C_5 -0.5*C_5 C_4 + 0.5*C_5 + C_8 -0.5*C_5 -C_8 0]
     [0 0 0 0 -0.5*C_5 0.5*C_5 -0.5*C_5 0.5*C_5 + C_7 0 0]
     [0 0 0 0 -C_6 0 0 0 0.5*C_9 -0.5*C_9]
     [0 0 0 0 0 0 0 C_7 -0.5*C_9 0.5*C_9]
     [0 0 0 0 0 0 -C_8 0 C_8 + 0.5*C_9 -0.5*C_9]
     [0 0 0 0 0 0 0 0 -0.5*C_9 0.5*C_9]]

```python
K_c01.shape

K_oo = np.concatenate((K_c01[0:8, :], K_c01[10:12, :]), axis = 0)
```

```python
K_oo
```

    array([[C_1 + 0.5*C_2, 0.5*C_2, -C_1, 0, -0.5*C_2, -0.5*C_2, 0, 0, 0, 0],
           [0.5*C_2, 0.5*C_2, 0, 0, -0.5*C_2, -0.5*C_2, 0, 0, 0, 0],
           [-C_1, 0, C_1 + C_4, 0, 0, 0, -C_4, 0, 0, 0],
           [0, 0, 0, C_3, 0, -C_3, 0, 0, 0, 0],
           [-0.5*C_2, -0.5*C_2, 0, 0, 0.5*C_2 + 0.5*C_5 + C_6,
            0.5*C_2 - 0.5*C_5, 0.5*C_5, -0.5*C_5, 0, 0],
           [-0.5*C_2, -0.5*C_2, 0, C_3, 0.5*C_2 - 0.5*C_5,
            0.5*C_2 - C_3 + 0.5*C_5, -0.5*C_5, 0.5*C_5, 0, 0],
           [0, 0, -C_4, 0, 0.5*C_5, -0.5*C_5, C_4 + 0.5*C_5 + C_8, -0.5*C_5,
            -C_8, 0],
           [0, 0, 0, 0, -0.5*C_5, 0.5*C_5, -0.5*C_5, 0.5*C_5 + C_7, 0, 0],
           [0, 0, 0, 0, 0, 0, -C_8, 0, C_8 + 0.5*C_9, -0.5*C_9],
           [0, 0, 0, 0, 0, 0, 0, 0, -0.5*C_9, 0.5*C_9]], dtype=object)

```python

```

    (12, 9)

```python
sum(Ksystem)

```

$\displaystyle 0$

```python
Kco1 =  Ksystem[8:10,:]
Kco = np.concatenate((Kco1[:,0:8], Kco1[:,10:]),axis=1)
Koc = np.transpose(Kco)
```

```python
print(Kco.shape)
```

    (2, 10)

```python
print(Koc.shape)

```

    (10, 2)

```python
Kco[0:7,:].shape
```

    (2, 12)

```python
 Ksystem[8:10,:].shape
```

    (2, 12)

```python
Kco[:,0:8]
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0 & - C_{6} & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & C_{7}\end{matrix}\right]$

```python
Kco[:,10:]
```

$\displaystyle \left[\begin{matrix}0.5 C_{9} & - 0.5 C_{9}\\- 0.5 C_{9} & 0.5 C_{9}\end{matrix}\right]$

```python
Kco = np.concatenate((Kco[:,0:8], Kco[:,10:]),axis=1)
```

```python

```

    [[0 0 0 0 -C_6 0 0 0 0.5*C_9 -0.5*C_9]
     [0 0 0 0 0 0 0 C_7 -0.5*C_9 0.5*C_9]]

    (2, 10)

```python
MK_cc = sp.Matrix(K_cc)
dc = np.dot(MK_cc.inv(), f_c - np.dot(Kco,d0))
```

```python
print(dc.shape)
```

    (2,)

```python
dc[0]
```

$\displaystyle - \frac{0.5 C_{9} \left(- C_{7} u_{4y} + 0.5 C_{9} u_{6x} - 0.5 C_{9} u_{6y}\right)}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}} + \frac{\left(1.0 C_{7} - 0.5 C_{9}\right) \left(C_{6} u_{3x} - 0.5 C_{9} u_{6x} + 0.5 C_{9} u_{6y} + H\right)}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}}$

```python
MK_cc = sp.Matrix(K_cc)
```

```python
MK_cc.inv()
```

$\displaystyle \left[\begin{matrix}\frac{1.0 C_{7} - 0.5 C_{9}}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}} & - \frac{0.5 C_{9}}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}}\\- \frac{0.5 C_{9}}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}} & \frac{- 1.0 C_{6} - 0.5 C_{9}}{1.0 C_{6} C_{7} - 0.5 C_{6} C_{9} + 0.5 C_{7} C_{9}}\end{matrix}\right]$


# Driving a skyrmion with spin-polarised current

**Author:** Weiwei Wang (2014)

**Edited:** Marijan Beg (2016)

The implemented equation in finmag with STT is [1,2],

\begin{equation}
\frac{\partial \mathbf{m}}{\partial t} = - \gamma \mathbf{m} \times \mathbf{H} + \alpha \mathbf{m} \times  \frac{\partial \mathbf{m}}{\partial t}  + u (\mathbf{j}_s \cdot \nabla) \mathbf{m} - \beta u [\mathbf{m}\times (\mathbf{j}_s \cdot \nabla)\mathbf{m}]
\end{equation}

where $\mathbf{j}_s$ is the current density. $u$ is the material parameter, and by default,

$$u=u_{ZL}=\frac{u_0}{1+\beta^2}$$

There is an option "using_u0" in sim.set_zhangli method, u=u0 if "using_u0 = True" and

$$u_0=\frac{g \mu_B P}{2 |e| M_s}=\frac{g \mu_B P a^3}{2 |e| \mu_s}$$

where $\mu_B=|e|\hbar/(2m)$ is the Bohr magneton, $P$ is the polarization rate, $e$ is the electron charge.

The implemented Landau-Lifshitz-Gilbert equation with Slonczewski spin-transfer torque is [3],

\begin{equation}
\frac{\partial \mathbf{m}}{\partial t} = - \gamma \mathbf{m} \times \mathbf{H} + \alpha \mathbf{m} \times  \frac{\partial \mathbf{m}}{\partial t}
+ \gamma \beta \epsilon (\mathbf{m} \times \mathbf{m}_p \times \mathbf{m})
\end{equation}

where

\begin{align*}
\beta&=\left|\frac{\hbar}{\mu_0 e}\right|\frac{J}{tM_\mathrm{s}}\,\,\, \mathrm{and}\\
\epsilon&=\frac{P\Lambda^2}{(\Lambda^2+1)+(\Lambda^2-1)(\mathbf{m}\cdot\mathbf{m}_p)}
\end{align*}

[1] S. Zhang and Z. Li, Roles of nonequilibrium conduction electrons on the magnetization dynamics of ferromagnets, Phys. Rev. Lett. 93, 127204 (2004).

[2] A. Thiaville, Y. Nakatani, J. Miltat and Y. Suzuki, Micromagnetic understanding of current-driven domain wall motion in patterned nanowires, Europhys. Lett. 69, 990 (2005).

[3] J. Xiao, A. Zangwill, and M. D. Stiles, “Boltzmann test of Slonczewski’s theory of spin-transfer torque,” Phys. Rev. B, 70, 172405 (2004).

## Skyrmion nucleation

Import the related modules and create a two-dimensional rectangular mesh:

```python
%matplotlib inline
import os
import matplotlib.pyplot as plt
import dolfin as df
import numpy as np
from finmag import Simulation as Sim
from finmag.energies import Exchange, DMI, UniaxialAnisotropy, Zeeman
from finmag.util.dmi_helper import find_skyrmion_center_2d
from finmag.util.helpers import set_logging_level
import finmag

mesh = df.RectangleMesh(df.Point(0, 0), df.Point(200, 40), 200, 40)
```

    [2016-07-05 12:15:41] INFO: Finmag logging output will be appended to file: '/home/mb4e10/.finmag/global.log'
    [2016-07-05 12:15:41] DEBUG: Building modules in 'native'...
    [2016-07-05 12:15:42] DEBUG: FinMag          f28296c990ff2e117b748be86dbe1c03d67bde7a
    [2016-07-05 12:15:42] DEBUG: Dolfin          1.6.0                Matplotlib      1.5.1
    [2016-07-05 12:15:42] DEBUG: Numpy           1.11.0               Scipy           0.15.1
    [2016-07-05 12:15:42] DEBUG: IPython         4.2.0                Python          2.7.6
    [2016-07-05 12:15:42] DEBUG: Paraview        4.0.1-1ubuntu1       Sundials        2.5.0
    [2016-07-05 12:15:42] DEBUG: Boost-Python    <unknown>            Linux           Ubuntu 14.04.4 LTS
    [2016-07-05 12:15:42] DEBUG: Registering debug signal handler. Press Ctrl-Z any time to stop execution and jump into the debugger.

We define a function to generate a skyrmion in the track,

```python
def m_init_one(pos):
    x, y = pos

    x0 = 50
    y0 = 20
    if (x-x0)**2 + (y-y0)**2 < 10**2:
        return (0, 0, -1)
    else:
        return (0, 0, 1)
```

Create function that can plot scalar field of one magnetisation component:

```python
def plot_2d_comp(sim, comp='z', title=None):
    """expects a simulation object sim and a component to plot. Component can be
    'x' or 'y' or 'z'

    Not optimised for speed.
    """
    finmag.logger.info("plot_2d_comp: at t = {:g}".format(sim.t))
    comps = {'x': 0, 'y': 1, 'z': 2}
    assert comp in comps, "print unknown component {}, we know: {}".format(comp, comp.keys())
    m = sim.get_field_as_dolfin_function('m')

    # get mesh coordinates for plotting
    coords = mesh.coordinates()
    mym = []
    for coord in coords:
        mym.append(m(coord))

    import matplotlib.pyplot as plt
    import matplotlib.tri as tri
    import numpy as np

    x = [ r[0] for r in coords]
    y = [ r[1] for r in coords]
    # extract i-ith component of magnetisation
    mi = [ m[comps[comp]] for m in mym]

    # Create the Triangulation; no triangles so Delaunay triangulation created.
    triang = tri.Triangulation(x, y)

    # tripcolor plot.
    plt.figure()
    plt.gca().set_aspect('equal')
    plt.tripcolor(triang, mi, shading='flat', cmap=plt.cm.rainbow)
    if title:
        plt.title(title)
    else:
        plt.title('Plot of {} component of m at t={:.3f}ns'.format(comp, sim.t * 1e9))
```

```python
sim = Sim(mesh, Ms=5.8e5, unit_length=1e-9, pbc=None)

sim.add(UniaxialAnisotropy(K1=6e5, axis=[0, 0, 1]))
sim.add(Exchange(A=1.5e-11))
sim.add(DMI(D=3e-3))
sim.add(Zeeman((0, 0, 1e5)))
sim.alpha = 0.5

sim.set_m(m_init_one)

sim.relax()
plot_2d_comp(sim, comp='z', title='relaxed magnetisation (z-comp)')
```

## Moving a skyrmion with Zhang-Li term

We apply a spin-polarised current in the $x$ direction.

```python
Jx = (-2e12, 0, 0)
```

Now, we can add a Zhang-Li term to the LLG equation.

```python
#We use the zhang-li spin-transfer torque with parameters that polarisation=0.5 and beta=0.01
sim.set_zhangli(Jx, P=0.5, beta=0.01, using_u0=False)

# every 0.1ns save vtk data
sim.schedule('save_vtk', every=1e-10, filename='vtks/m.pvd', overwrite=True)
# every 0.1ns save raw data
sim.schedule('save_m', every=1e-10, filename='npys/m.pvd', overwrite=True)
# every 0.1ns create plot for notebook
sim.schedule(plot_2d_comp, every=1e-10)

# now do the calculation (events scheduled above will be done automatically)
sim.run_until(0.5e-9)
```

## Slonczewski spin-transfer torque

If we want to move a skyrmion with perpendicular current (Slonczewski STT term), the following line should be used instead of `set_zhangli` method.

```python
#sim.set_stt(current_density=1e10, polarisation=0.5, thickness=0.4e-9, direction=(0,1,0))
```


[Sebastian Raschka](http://sebastianraschka.com)

- [Open in IPython nbviewer](http://nbviewer.ipython.org/github/rasbt/pattern_classification/blob/master/python_howtos/scikit_linear_classificationreate=1)

- [Link to this IPython notebook on Github](http://nbviewer.ipython.org/github/rasbt/pattern_classification/blob/master/machine_learning/scikit-learn/scikit_linear_classification.ipynb)

- [Link to the GitHub Repository pattern_classification](https://github.com/rasbt/pattern_classification)

```python
%load_ext watermark
%watermark -a 'Sebastian Raschka' -u -d -v -p numpy,scikit-learn,matplotlib
```

    Sebastian Raschka
    last updated: 2016-07-16

    CPython 3.5.1
    IPython 5.0.0

    numpy 1.11.0
    scikit-learn 0.17.1
    matplotlib 1.5.1

I would be happy to hear your comments and suggestions.
Please feel free to drop me a note via
[twitter](https://twitter.com/rasbt), [email](mailto:bluewoodtree@gmail.com), or [google+](https://plus.google.com/+SebastianRaschka).

# An Introduction to simple linear supervised classification using `scikit-learn`

In this introduction I want to give a brief overview of how Python's `scikit-learn` machine learning library can be used for simple linear classification.

# Sections

- [About the dataset](#About-the-dataset)
- [Reading in a dataset from a CSV file](#Reading-in-a-dataset-from-a-CSV-file)
- [Visualizing the Wine dataset](#Visualizing-the-Wine-dataset)
- [Splitting into training and test dataset](#Splitting-into-training-and-test-dataset)
- [Feature Scaling](#Feature-Scaling)
- [Introduction to Multiple Discriminant Analysis (MDA)](#MDA)
- [Classification via LDA](#LDA)
- [Stochastic Gradient Descent (SGD) as linear classifier](#SGD)

## About the dataset

[[back to top]](#Sections)

For the following tutorial, we will be working with the free "Wine" Dataset that is deposited on the UCI machine learning repository
(http://archive.ics.uci.edu/ml/datasets/Wine).

**Reference:**
Forina, M. et al, PARVUS - An Extendible Package for Data
Exploration, Classification and Correlation. Institute of Pharmaceutical
and Food Analysis and Technologies, Via Brigata Salerno,
16147 Genoa, Italy.

The Wine dataset consists of 3 different classes where each row correspond to a particular wine sample.

The class labels (1, 2, 3) are listed in the first column, and the columns 2-14 correspond to the following 13 attributes (features):

1) Alcohol
2) Malic acid
3) Ash
4) Alcalinity of ash
5) Magnesium
6) Total phenols
7) Flavanoids
8) Nonflavanoid phenols
9) Proanthocyanins
10) Color intensity
11) Hue
12) OD280/OD315 of diluted wines
13) Proline

An excerpt from the wine_data.csv dataset:

1,14.23,1.71,2.43,15.6,127,2.8,3.06,.28,2.29,5.64,1.04,3.92,1065
1,13.2,1.78,2.14,11.2,100,2.65,2.76,.26,1.28,4.38,1.05,3.4,1050
[...]
2,12.37,.94,1.36,10.6,88,1.98,.57,.28,.42,1.95,1.05,1.82,520
2,12.33,1.1,2.28,16,101,2.05,1.09,.63,.41,3.27,1.25,1.67,680
[...]
3,12.86,1.35,2.32,18,122,1.51,1.25,.21,.94,4.1,.76,1.29,630
3,12.88,2.99,2.4,20,104,1.3,1.22,.24,.83,5.4,.74,1.42,530

## Reading in a dataset from a CSV file

[[back to top]](#Sections)

Since it is quite typical to have the input data stored locally, as mentioned above, we will use the [`numpy.loadtxt`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html) function  now to read in the data from the CSV file.
(alternatively [`np.genfromtxt()`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html) could be used in similar way, it provides some additional options)

```python
import numpy as np

# reading in all data into a NumPy array
all_data = np.loadtxt(open("../../data/wine_data.csv","r"),
        delimiter=",",
        skiprows=0,
        dtype=np.float64
        )

# load class labels from column 1
y_wine = all_data[:,0]

# conversion of the class labels to integer-type array
y_wine = y_wine.astype(np.int64, copy=False)

# load the 14 features
X_wine = all_data[:,1:]

# printing some general information about the data
print('\ntotal number of samples (rows):', X_wine.shape[0])
print('total number of features (columns):', X_wine.shape[1])

# printing the 1st wine sample
float_formatter = lambda x: '{:.2f}'.format(x)
np.set_printoptions(formatter={'float_kind':float_formatter})
print('\n1st sample (i.e., 1st row):\nClass label: {:d}\n{:}\n'
          .format(int(y_wine[0]), X_wine[0]))

# printing the rel.frequency of the class labels
print('Class label frequencies')
print('Class 1 samples: {:.2%}'.format(list(y_wine).count(1)/y_wine.shape[0]))
print('Class 2 samples: {:.2%}'.format(list(y_wine).count(2)/y_wine.shape[0]))
print('Class 3 samples: {:.2%}'.format(list(y_wine).count(3)/y_wine.shape[0]))
```

    total number of samples (rows): 178
    total number of features (columns): 13

    1st sample (i.e., 1st row):
    Class label: 1
    [14.23 1.71 2.43 15.60 127.00 2.80 3.06 0.28 2.29 5.64 1.04 3.92 1065.00]

    Class label frequencies
    Class 1 samples: 33.15%
    Class 2 samples: 39.89%
    Class 3 samples: 26.97%

## Visualizing the Wine dataset

[[back to top]](#Sections)

There are endless way to visualize datasets for get an initial idea of how the data looks like. The most common ones are probably histograms and scatter plots.

Scatter plots are useful for visualizing features in more than just one dimension, for example to get a feeling for the correlation between particular features.
Unfortunately, we can't plot all 13 features here at once, since the visual cortex of us humans is limited to a maximum of three dimensions.

Below, we will create an example 2D-Scatter plot from the features "Alcohol content" and "Malic acid content".
Additionally, we will use the [`scipy.stats.pearsonr`](http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.pearsonr.html) function to calculate a Pearson correlation coefficient between these two features.

```python
%matplotlib inline
```

```python
from matplotlib import pyplot as plt
from scipy.stats import pearsonr

plt.figure(figsize=(10,8))

for label,marker,color in zip(
        range(1,4),('x', 'o', '^'),('blue', 'red', 'green')):

    # Calculate Pearson correlation coefficient
    R = pearsonr(X_wine[:,0][y_wine == label], X_wine[:,1][y_wine == label])
    plt.scatter(x=X_wine[:,0][y_wine == label], # x-axis: feat. from col. 1
                y=X_wine[:,1][y_wine == label], # y-axis: feat. from col. 2
                marker=marker, # data point symbol for the scatter plot
                color=color,
                alpha=0.7,
                label='class {:}, R={:.2f}'.format(label, R[0]) # label for the legend
                )

plt.title('Wine Dataset')
plt.xlabel('alcohol by volume in percent')
plt.ylabel('malic acid in g/l')
plt.legend(loc='upper right')

plt.show()
```

If we want to pack 3 different features into one scatter plot at once, we can also do the same thing in 3D:

```python
from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(111, projection='3d')

for label,marker,color in zip(
        range(1,4),('x', 'o', '^'),('blue','red','green')):

    ax.scatter(X_wine[:,0][y_wine == label],
               X_wine[:,1][y_wine == label],
               X_wine[:,2][y_wine == label],
               marker=marker,
               color=color,
               s=40,
               alpha=0.7,
               label='class {}'.format(label))

ax.set_xlabel('alcohol by volume in percent')
ax.set_ylabel('malic acid in g/l')
ax.set_zlabel('ash content in g/l')
plt.legend(loc='upper right')
plt.title('Wine dataset')

plt.show()
```

## Splitting into training and test dataset

[[back to top]](#Sections)

It is a typical procedure for machine learning and pattern classification tasks to split one dataset into two: a training dataset and a test dataset.
The training dataset is henceforth used to train our algorithms or classifier, and the test dataset is a way to validate the outcome quite objectively before we apply it to "new, real world data".

Here, we will split the dataset randomly so that 70% of the total dataset will become our training dataset, and 30% will become our test dataset, respectively.

```python
from sklearn.cross_validation import train_test_split
from sklearn import preprocessing

X_train, X_test, y_train, y_test = train_test_split(X_wine, y_wine,
     test_size=0.30, random_state=123)
```

Note that since this a random assignment, the original relative frequencies for each class label are not maintained.

```python
print('Class label frequencies')

print('\nTraining Dataset:')
for l in range(1,4):
    print('Class {:} samples: {:.2%}'.format(l, list(y_train).count(l)/y_train.shape[0]))

print('\nTest Dataset:')
for l in range(1,4):
    print('Class {:} samples: {:.2%}'.format(l, list(y_test).count(l)/y_test.shape[0]))
```

    Class label frequencies

    Training Dataset:
    Class 1 samples: 36.29%
    Class 2 samples: 42.74%
    Class 3 samples: 20.97%

    Test Dataset:
    Class 1 samples: 25.93%
    Class 2 samples: 33.33%
    Class 3 samples: 40.74%

## Feature Scaling

[[back to top]](#Sections)

Another popular procedure is to standardize the data prior to fitting the model and other analyses so that the features will have the properties of a standard normal distribution with

$\mu = 0$ and $\sigma = 1$

where $\mu$ is the mean (average) and $\sigma$ is the standard deviation from the mean, so that the standard scores of the samples are calculated as follows:

\begin{equation} z = \frac{x - \mu}{\sigma}\end{equation}

```python
std_scale = preprocessing.StandardScaler().fit(X_train)
X_train = std_scale.transform(X_train)
X_test = std_scale.transform(X_test)
```

```python
f, ax = plt.subplots(1, 2, sharex=True, sharey=True, figsize=(10,5))

for a,x_dat, y_lab in zip(ax, (X_train, X_test), (y_train, y_test)):

    for label,marker,color in zip(
        range(1,4),('x', 'o', '^'),('blue','red','green')):

        a.scatter(x=x_dat[:,0][y_lab == label],
                y=x_dat[:,1][y_lab == label],
                marker=marker,
                color=color,
                alpha=0.7,
                label='class {}'.format(label)
                )

    a.legend(loc='upper right')

ax[0].set_title('Training Dataset')
ax[1].set_title('Test Dataset')
f.text(0.5, 0.04, 'malic acid (standardized)', ha='center', va='center')
f.text(0.08, 0.5, 'alcohol (standardized)', ha='center', va='center', rotation='vertical')

plt.show()
```

## Linear Transformation & Classification: Multiple Discriminant Analysis (MDA)

[[back to top]](#Sections)

The main purposes of a Multiple Discriminant Analysis is to analyze the data to identify patterns to project it onto a subspace that yields a better separation of the classes. Also, the dimensionality of the dataset shall be reduced with minimal loss of information.

**The approach is very similar to a Principal Component Analysis (PCA), but in addition to finding the component axes that maximize the variance of our data, we are additionally interested in the axes that maximize the separation of our classes (e.g., in a supervised pattern classification problem)**

Here, our desired outcome of the multiple discriminant analysis is to project a feature space (our dataset consisting of n d-dimensional samples) onto a smaller subspace that represents our data "well" and has a good class separation. A possible application would be a pattern classification task, where we want to reduce the computational costs and the error of parameter estimation by reducing the number of dimensions of our feature space by extracting a subspace that describes our data "best".

#### Principal Component Analysis (PCA) Vs. Multiple Discriminant Analysis (MDA)

Both Multiple Discriminant Analysis (MDA) and Principal Component Analysis (PCA) are linear transformation methods and closely related to each other. In PCA, we are interested to find the directions (components) that maximize the variance in our dataset, where in MDA, we are additionally interested to find the directions that maximize the separation (or discrimination) between different classes (for example, in pattern classification problems where our dataset consists of multiple classes. In contrast two PCA, which ignores the class labels).

**In other words, via PCA, we are projecting the entire set of data (without class labels) onto a different subspace, and in MDA, we are trying to determine a suitable subspace to distinguish between patterns that belong to different classes. Or, roughly speaking in PCA we are trying to find the axes with maximum variances where the data is most spread (within a class, since PCA treats the whole data set as one class), and in MDA we are additionally maximizing the spread between classes.**

In typical pattern recognition problems, a PCA is often followed by an MDA.

If you are interested, you can find more information about the MDA in my IPython notebook
[Stepping through a Multiple Discriminant Analysis - using Python's NumPy and matplotlib](http://nbviewer.ipython.org/github/rasbt/pattern_classification/blob/master/dimensionality_reduction/projection/linear_discriminant_analysis.ipynb?create=1).

Like we did in the PCA section above, we will use a `scikit-learn` funcion, [`sklearn.lda.LDA`](http://scikit-learn.org/stable/modules/generated/sklearn.lda.LDA.html) in order to transform our training data onto 2 dimensional subspace, where MDA is basically the more generalized form of an LDA (Linear Discriminant Analysis):

```python
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
sklearn_lda = LDA(n_components=2)
sklearn_transf = sklearn_lda.fit_transform(X_train, y_train)

plt.figure(figsize=(10,8))

for label,marker,color in zip(
        range(1,4),('x', 'o', '^'),('blue', 'red', 'green')):

    plt.scatter(x=sklearn_transf[:,0][y_train == label],
                y=sklearn_transf[:,1][y_train == label],
                marker=marker,
                color=color,
                alpha=0.7,
                label='class {}'.format(label)
                )

plt.xlabel('vector 1')
plt.ylabel('vector 2')

plt.legend()
plt.title('Most significant singular vectors after linear transformation via LDA')

plt.show()
```

## Classification via LDA

[[back to top]](#Sections)

The LDA that we've just used in the section above can also be used as a simple linear classifier.

```python
# fit model
lda_clf = LDA()
lda_clf.fit(X_train, y_train)
LDA(n_components=None, priors=None)

# prediction
print('1st sample from test dataset classified as:', lda_clf.predict(X_test[0,:].reshape(1, -1)))
print('actual class label:', y_test[0])
```

    1st sample from test dataset classified as: [3]
    actual class label: 3

Another handy subpackage of sklearn is `metrics`. The [`metrics.accuracy_score`](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html), for example, is quite useful to evaluate how many samples can be classified correctly:

```python
from sklearn import metrics
pred_train = lda_clf.predict(X_train)

print('Prediction accuracy for the training dataset')
print('{:.2%}'.format(metrics.accuracy_score(y_train, pred_train)))
```

    Prediction accuracy for the training dataset
    100.00%

To verify that over model was not overfitted to the training dataset, let us evaluate the classifier's accuracy on the test dataset:

```python
pred_test = lda_clf.predict(X_test)

print('Prediction accuracy for the test dataset')
print('{:.2%}'.format(metrics.accuracy_score(y_test, pred_test)))
```

    Prediction accuracy for the test dataset
    98.15%

**Confusion Matrix**
As we can see above, there was a very low misclassification rate when we'd apply the classifier on the test data set. A confusion matrix can tell us in more detail which particular classes could not classified correctly.

	<colgroup width="60"></colgroup>
	<colgroup span="4" width="82"></colgroup>

		predicted class

		class 1
		class 2
		class 3

		actual class
		class 1
		True positives

		class 2

		True positives

		class 3

		True positives

```python
print('Confusion Matrix of the LDA-classifier')
print(metrics.confusion_matrix(y_test, lda_clf.predict(X_test)))
```

    Confusion Matrix of the LDA-classifier
    [[14  0  0]
     [ 1 17  0]
     [ 0  0 22]]

As we can see, one sample from class 2 was incorrectly labeled as class 1, from the perspective of class 1, this would be 1 "False Negative" or a "False Postive" from the perspective of class 2, respectively

## Stochastic Gradient Descent (SGD) as linear classifier

[[back to top]](#Sections)

Let us now compare the classification accuracy of the LDA classifier with a simple classification (we also use the probably not ideal default settings here) via stochastic gradient descent, an algorithm that minimizes a linear objective function.
More information about the `sklearn.linear_model.SGDClassifier` can be found [here](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html).

```python
from sklearn.linear_model import SGDClassifier

sgd_clf = SGDClassifier()
sgd_clf.fit(X_train, y_train)

pred_train = sgd_clf.predict(X_train)
pred_test = sgd_clf.predict(X_test)

print('\nPrediction accuracy for the training dataset')
print('{:.2%}\n'.format(metrics.accuracy_score(y_train, pred_train)))

print('Prediction accuracy for the test dataset')
print('{:.2%}\n'.format(metrics.accuracy_score(y_test, pred_test)))

print('Confusion Matrix of the SGD-classifier')
print(metrics.confusion_matrix(y_test, sgd_clf.predict(X_test)))
```

    Prediction accuracy for the training dataset
    100.00%

    Prediction accuracy for the test dataset
    96.30%

    Confusion Matrix of the SGD-classifier
    [[14  0  0]
     [ 1 17  0]
     [ 0  1 21]]