# Stochastic Differential Equations: Lab 1

```python
from IPython.core.display import HTML
css_file = 'https://raw.githubusercontent.com/ngcm/training-public/master/ipython_notebook_styles/ngcmstyle.css'
HTML(url=css_file)
```

<link href='http://fonts.googleapis.com/css?family=Open+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 1000px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1200px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Open Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Arvo', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Arvo', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Arvo', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Arvo', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Arvo', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>

This background for these exercises is article of D Higham, [*An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations*, SIAM Review 43:525-546 (2001)](http://epubs.siam.org/doi/abs/10.1137/S0036144500378302).
Higham provides Matlab codes illustrating the basic ideas at <http://personal.strath.ac.uk/d.j.higham/algfiles.html>, which are also given in the paper.

For random processes in `python` you should look at the `numpy.random` module. To set the initial seed (which you should *not* do in a real simulation, but allows for reproducible testing), see `numpy.random.seed`.

## Brownian processes

A *random walk* or *Brownian process* or *Wiener process* is a way of modelling error introduced by uncertainty into a differential equation. The random variable representing the walk is denoted $W$. A single realization of the walk is written $W(t)$. We will assume that

1. The walk (value of $W(t)$) is initially (at $t=0$) $0$, so $W(0)=0$, to represent "perfect knowledge" there;
2. The walk is *on average* zero, so $\mathbb{E}[W(t+h) - W(t)] = 0$, where the *expectation value* is
$$
  \mathbb{E}[W] = \int_{-\infty}^{\infty} t W(t) \, \text{d}t
$$
3. Any step in the walk is independent of any other step, so $W(t_2) - W(t_1)$ is independent of $W(s_2) - W(s_1)$ for any $s_{1,2} \ne t_{1,2}$.

These requirements lead to a definition of a *discrete* random walk: given the points $\{ t_i \}$ with $i = 0, \dots, N$ separated by a uniform timestep $\delta t$, we have - for a single realization of the walk - the definition
$$
\begin{align}
  \text{d}W_i &= \sqrt{\delta t} {\cal N}(0, 1), \\
  W_i &= \left( \sum_{j=0}^{i-1} \text{d}W_j \right), \\
  W_0 &= 0
\end{align}
$$
Here ${\cal N}(0, 1)$ means a realization of a normally distributed random variable with mean $0$ and standard deviation $1$: programmatically, the output of `numpy.random.randn`.

When working with discrete Brownian processes, there are two things we can do.

1. We can think about a *single realization* at different timescales, by averaging over more points. E.g.
$$
  W_i = \left( \sum_{j=0}^{i_1} \sum_{k=0}^{p} \text{d}W_{(p j + k)} \right)
$$
is a Brownian process with timestep $p \, \delta t$.
2. We can think about *multiple realizations* by computing a new set of steps $\text{d}W$, whilst at the same timestep.

Both viewpoints are important.

### Tasks

1. Simulate a single realization of a Brownian process over $[0, 1]$ using a step length $\delta t = 1/N$ for $N = 500, 1000, 2000$. Use a fixed seed of `100`. Compare the results.
2. Simulation different realizations of a Brownian process with $\delta t$ of your choice. Again, compare the results.

```python
%matplotlib inline
import numpy
from matplotlib import pyplot
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
rcParams['figure.figsize'] = (12,6)
from scipy.integrate import quad
```

Evaluate the function $u(W(t)) = \sin^2(t + W(t))$, where $W(t)$ is a Brownian process, on $M$ Brownian paths for $M = 500, 1000, 2000$. Compare the *average* path for each $M$.

The average path at time $t$ should be given by
$$
\begin{equation}
  \int_{-\infty}^{\infty} \frac{\sin(t+s)^2 \exp(-s^2 / 2t)}{\sqrt{2 \pi t}} \,\text{d}s.
\end{equation}
$$

```python
# This computes the exact solution!

t_int = numpy.linspace(0.005, numpy.pi, 1000)

def integrand(x,t):
    return numpy.sin(t+x)**2*numpy.exp(-x**2/(2.0*t))/numpy.sqrt(2.0*numpy.pi*t)

int_exact = numpy.zeros_like(t_int)
for i, t in enumerate(t_int):
    int_exact[i], err = quad(integrand, -numpy.inf, numpy.inf, args=(t,))
```

## Stochastic integrals

We have, in eg finite elements or multistep methods for IVPs, written the solution of differential equations in terms of integrals. We're going to do the same again, so we need to integrate random variables. The integral of a random variable *with respect to a Brownian process* is written
$$
  \int_0^t G(s) \, \text{d}W_s,
$$
where the notation $\text{d}W_s$ indicates that the step in the Brownian process depends on the (dummy) independent variable $s$.

We'll concentrate on the case $G(s) = W(s)$, so we're trying to integrate the Brownian process itself. If this were a standard, non-random variable, the answer would be
$$
  \int_0^t W(s) \, \text{d}W_s = \frac{1}{2} \left( W(t)^2 - W(0)^2 \right).
$$

When we approximate the quadrature numerically than we would split the interval $[0, T]$ into strips (subintervals), approximate the integral on each subinterval by picking a point inside the interval, evaluating the integrand at that point, and weighting it by the width of the subinterval. In normal integration it doesn't matter which point within the subinterval we choose.

In the stochastic case that is not true. We pick a specific point $\tau_i = a t_i + (1-a) t_{i-1}$ in the interval $[t_{i-1}, t_i]$. The value $a \in [0, 1]$ is a constant that says where within each interval we are evaluating the integrand. We can then approximate the integral by

\begin{equation}
  \int_0^T W(s) \, dW_s = \sum_{i=1}^N W(\tau_i) \left[ W(t_i) - W(t_{i-1}) \right] = S_N.
\end{equation}

Now we can compute (using that the expectation of the products of $W$ terms is the covariance, which is the minimum of the arguments)

\begin{align}
  \mathbb{E}(S_N) &= \mathbb{E} \left( \sum_{i=1}^N W(\tau_i) \left[ W(t_i) - W(t_{i-1}) \right] \right) \\
  &= \sum_{i=1}^N \mathbb{E} \left( W(\tau_i) W(t_i) \right) - \mathbb{E} \left( W(\tau_i) W(t_{i-1}) \right) \\
  &= \sum_{i=1}^N (\min\{\tau_i, t_i\} - \min\{\tau_i, t_{i-1}\}) \\
  &= \sum_{i=1}^N (\tau_i - t_{i-1}) \\
  &= (t - t_0) a.
\end{align}

The choice of evaluation point **matters**.

So there are multiple different stochastic integrals, each (effectively) corresponding to a different choice of $a$. The two standard choices are
There are two standard choices of stochastic integral.

1. Ito: choose $a=0$.
2. Stratonovich: choose $a=1/2$.

These lead to
$$
  \int_0^t G(s) \, \text{d}W_s \simeq_{\text{Ito}} \sum_{j=0}^{N-1} G(s_j, W(s_j)) \left( W(s_{j+1}) - W(s_j) \right) = \sum_{j=0}^{N-1} G(s_j) \text{d}W(s_{j})
$$
for the Ito integral, and
$$
  \int_0^t G(s) \, \text{d}W_s \simeq_{\text{Stratonovich}} \sum_{j=0}^{N-1} \frac{1}{2} \left( G(s_j, W(s_j)) + G(s_{j+1}, W(s_{j+1})) \right) \left( W(s_{j+1}) - W(s_j) \right) = \sum_{j=0}^{N-1} \frac{1}{2} \left( G(s_j, W(s_j)) + G(s_{j+1}, W(s_{j+1})) \right) \text{d}W(s_{j}).
$$
for the Stratonovich integral.

### Tasks

Write functions to compute the Itô and Stratonovich integrals of a function $h(t, W(t))$ of a *given* Brownian process $W(t)$ over the interval $[0, 1]$.

```python
def ito(h, trange, dW):
    """Compute the Ito stochastic integral given the range of t.

    Parameters
    ----------

    h : function
        integrand
    trange : list of float
        the range of integration
    dW : array of float
        Brownian increments
    seed : integer
        optional seed for the Brownian path
    Returns
    -------

    ito : float
        the integral
    """

    return ito
```

```python
def stratonovich(h, trange, dW):
    """Compute the Stratonovich stochastic integral given the range of t.

    Parameters
    ----------

    h : function
        integrand
    trange : list of float
        the range of integration
    dW : array of float
        the Brownian increments

    Returns
    -------

    stratonovich : float
        the integral
    """

    return stratonovich
```

Test the functions on $h = W(t)$ for various $N$. Compare the limiting values of the integrals.

## Euler-Maruyama's method

Now we can write down a stochastic differential equation.

The differential form of a stochastic differential equation is
$$
  \frac{\text{d}X}{\text{d}t} = f(X) + g(X) \frac{\text{d}W}{\text{d}t}
$$
and the comparable (and more useful) *integral form* is
$$
  \text{d}X = f(X) \, \text{d}t + g(X) \text{d}W.
$$
This has formal solution
$$
  X(t) = X_0 + \int_0^t f(X(s)) \, \text{d}s + \int_0^t g(X(s)) \, \text{d}W_s.
$$

We can use our Ito integral above to write down the *Euler-Maruyama method*

$$
  X(t+h) \simeq X(t) + h f(X(t)) + g(X(t)) \left( W(t+h) - W(t) \right) + {\cal{O}}(h^p).
$$

Written in discrete, subscript form we have

$$
  X_{n+1} = X_n + h f_n + g_n \, \text{d}W_{n}
$$

The order of convergence $p$ is an interesting and complex question.

### Tasks

Apply the Euler-Maruyama method to the stochastic differential equation

$$
\begin{equation}
  dX(t) = \lambda X(t) + \mu X(t) dW(t), \qquad X(0) = X_0.
\end{equation}
$$

Choose any reasonable values of the free parameters $\lambda, \mu, X_0$.

The exact solution to this equation is $X(t) = X(0) \exp \left[ \left( \lambda - \tfrac{1}{2} \mu^2 \right) t + \mu W(t) \right]$. Fix the timetstep and compare your solution to the exact solution.

Vary the timestep of the Brownian path and check how the numerical solution compares to the exact solution.

## Convergence

We have two ways of thinking about Brownian paths or processes.

We can fix the path (ie fix $\text{d}W$) and vary the timescale on which we're looking at it: this gives us a single random path, and we can ask how the numerical method converges for this single realization. This is *strong convergence*.

Alternatively, we can view each path as a single realization of a random process that should average to zero. We can then look at how the method converges as we average over a large number of realizations, *also* looking at how it converges as we vary the timescale. This is *weak convergence*.

Formally, denote the true solution as $X(T)$ and the numerical solution for a given step length $h$ as $X^h(T)$. The order of convergence is denoted $p$.

#### Strong convergence

$$
  \mathbb{E} \left| X(T) - X^h(T) \right| \le C h^{p}
$$
For Euler-Maruyama, expect $p=1/2$!.

#### Weak convergence

$$
  \left| \mathbb{E} \left( \phi( X(T) ) \right) - \mathbb{E} \left( \phi( X^h(T) ) \right) \right| \le C h^{p}
$$
For Euler-Maruyama, expect $p=1$.

### Tasks

Investigate the weak and strong convergence of your method, applied to the problem above.

```python

```

FEEG6016 Simulation and Modelling/09-Stochastic-DEs-Lab-1.ipynb


#### _Speech Processing Labs 2021: SIGNALS 1: Digital Signals: Sampling and Superposition_

```python
## Run this first!
%matplotlib inline
import sys
import matplotlib.pyplot as plt
import numpy as np

import cmath
from matplotlib.animation import FuncAnimation
from IPython.display import HTML
plt.style.use('ggplot')

from dspMisc import *
```

# Digital Signals: Sampling and Superposition

### Learning Outcomes
* Understand how we can approximate a sine wave with a specific frequency, given a specific sampling rate
* Understand how sampling rate limits the frequencies of sinusoids we can describe with discrete sequences
* Explain when aliasing will occurr and how this relates the sampling rate and the Nyquist frequency.
* Observe how compound waveforms can be described as a linear combination of phasors ('superposition')

### Background
* Topic Videos: Digital Signal, Short Term Analysis, Series Expansion
* [Interpreting the discrete fourier transform](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb)

#### Extra background (extension material)
* [Phasors, complex numbers and sinusoids](./signals-1-2a-digital-signals-complex-numbers.ipynb)

## 1 Introduction

In the class videos, you've seen that sound waves are changes in air pressure (amplitude) over time. In the notebook [interpreting the discrete fourier transform](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb), we saw that we can
decompose complex sound waves into 'pure tone' frequency components. We also saw that the output of the DFT was actually a sequence of complex numbers! In this notebook, we'll give a bit more background on the relationship
between complex numbers and sinusoids, and why it's useful to characterise sinusoids in the complex plane.

## 2 Phasors and Sinusoids: tl;dr

At this point, I should say that you can get a conceptual understanding of digital signal processing concepts without going through _all_ the math. We certainly won't be examining your knowledge of complex numbers or geometry in this class.  Of course, if you want to go further in understanding digital signal processing then you will have to learn a bit more about complex numbers, algebra, calculus and geometry than we'll touch upon here.

However, right now the main point that we'd like you to take away from this notebook is that we can conveniently represent periodic functions, like sine waves, in terms of **phasors**: basically what shown on the left hand side of the following gif:

You can think of the **phasor as an analogue clockface** with one moving hand. On the right hand side is one period of a 'pure tone' sinusoid, sin(t).

Now, we can think of every movement of the 'clockhand' (the phasor is actually this **vector**) as a step in time on the sinusoid graph: at every time step, the phasor (i.e., clockhand) rotates by some angle. If you follow the blue dots on both graphs, you should be able to see that the amplitude of the sinusoid matches the height of the clockhand on the phasor at each time step.

This gives us a different way of viewing the periodicity of $\sin(t)$.  The sinusoid starts to repeat itself when the phasor has done one full circle. So, rather than drawing out an infinite time vs amplitude graph, we can capture the behaviour of this periodic function in terms rotations with respect to this finite circle.

So, what's the connection with complex numbers?  Well, that blue dot on the phasor actually represents a complex number, and the dimensions of that graph are actually the **real** (horizontal) and **imaginary** (vertical) parts of that number. That is, a complex number of the form $a + jb$, where $a$ is the real part and $b$ is the imaginary part.  Quite conveniently, we can also express complex numbers in terms of a **magnitude** or radius $r$ (length of the clockhand) and a **phase angle** $\theta$ (angle of rotation from the point (1,0)) and an exponential.  So, we can write each point that the phasor hits in the form $re^{j\theta}$.  This will be familiar if you've had a look at the DFT formulae.

This relationship with complex numbers basically allows us to describe complicated periodic waveforms in terms of combinations of 'pure tone' sinusoids.  It turns out that maths for this works very elegantly using the phasor/complex number based representation.

The basic things you need to know are:

* A **sinusoid** (time vs amplitude, i.e. in the **time domain**) can be described in terms of a vector rotating around a circle (i.e. a phasor in the complex plane)
* The **phasor** vector (i.e., 'clockhand') is described by a complex number $re^{j\theta}$
* $re^{j\theta}$ is a point on a circle centered at (0,0) with radius $r$, $\theta$ degrees rotated from $(r,0)$ on the 2D plane.
    * the **magnitude** $r$ tells us what the peak amplitude of the corresponding sine wave is
    * the **phase angle** $\theta$ tells us how far around the circle the phasor has gone:
      * zero degrees (0 radians) corresponds to the point (r,0), while 90 degrees ($\pi/2$ radians) corresponds to the point (0,r)
* The vertical projection of the vector (onto the y-axis) corresponds to the amplitude of a **sine wave** $\sin(\theta)$
* The horizontal projection of the vector (onto the x-axis) corresponds to the amplitude of a **cosine wave** $\cos(\theta)$
* The **period** of these sine and cosine waves is the same as the time it takes to make one full circle of the phasor (in seconds). As such the **frequency** of the sine and cosine waves is the same as the frequency with which the phasor makes a full cycle (in cycles/second = Hertz).

If you take the maths on faith, you can see all of this just from the gif above. You'll probably notice in most phonetics text books, if they show this at all, they will just show the rotating phasor without any of the details.

If you want to know more about how this works, you can find a quick tour of these concepts in the (extension) notebook on [complex numbers and sinusoids](./sp-m1-2-digital-signals-complex-numbers).  But it's fine if you don't get all the details right now.  In fact, if you get the intuition behind from the phasor/sinusoid relationship above, it's fine to move on now to the rest of the content in this notebook.

## Changing the frequency of a sinusoid

So, we think of sine (and cosine) waves in terms of taking steps around a circle in the 2D (complex) plane.  Each of these 'steps' was represented by a complex number, $re^{j\theta}$ (the phasor) where the magnitude $r$ tells you the radius of the circle, and the phase angle $\theta$ tells you how far around the circle you are. When $\theta = 0$, means you are at the point (r,0), while $\theta = 90$ degrees means you are at the point (0,r).  There are 360 degrees (or 2$\pi$ radians) makes a complete cycle, i.e. when $\theta = 360$ degrees, you end up back at (r,0).

    It's often easier to deal with angles measured in radians rather than degrees.  The main thing to note is that:
    $$2\pi \text{ radians} = 360 \text{ degrees, i.e. 1 full circle }$$
Again, it may not seem obvious why we should want to use radians instead of the more familiar degrees.  The reason is that it makes dividing up a circle really nice and neat and so ends up making calculations much easier in the long run!

So that describes a generic sinusoid, e.g. $\sin(\theta)$, but now you might ask yourself how do we generate a generate a sine wave with a specific frequency $f$ Herzt (Hz=cycles/second)?

Let's take a concrete example, if we want a sinusoid with a frequency of $f=10$ Hz, that means:
* **Frequency:** we need to complete 10 full circles of the phasor in 1 second.
* **Period:** So, we have to complete 1 full cycle every 1/10 seconds (i.e. the period of this sinusoid $T=0.1$ seconds).
* **Angular velocity:** So, the phasor has to rotate at a speed of $2\pi/0.1 = 20\pi$ radians per second

So if we take $t$ to represent time, a sine wave with frequency 10 Hz has the form $\sin(20\pi t)$
* Check: at $t=0.1$ seconds we have $\sin(20 \times \pi \times 0.1) = \sin(2\pi)$, one full cycle.
* This corresponds to the phasor $e^{20\pi t j}$, where $t$ represents some point in time.

In general:
* A sine wave with peak amplitude R and frequency $f$ Hz is expressed as $R\sin(2 \pi f t)$
   * The amplitude of this sine wave at time $t$ corresponds to the imaginary part of the phasor $Re^{2\pi ftj}$.
* A cosine wave with peak amplitude R and frequency $f$ Hz is expressed as $\cos (2 \pi f t$)
   * The amplitude of this cosine wave at time $t$ corresponds to the real part of the phasor $Re^{2\pi ftj}$.

The term $2\pi f$ corresponds to the angular velocity, often written as $\omega$ which is measured in radians per second.

### Exercise
Q: What's the frequency of $\sin(2\pi t)$?

```python

```

## Frequency and Sampling Rate

The representation above assumes we're dealing with a continuous sinusoid, but since we're dealing with computers we need to think about digital (i.e. discrete) representations of waveforms.
So if we want to analyze a wave, we also need to sample it at a specific **sampling rate**, $f_s$.

For a given sampling rate $f_s$ (samples/second) we can work out the time between each sample, the **sampling period** as:

$$  t_s = \frac{1}{f_s}$$

The units of $t_s$ is seconds/sample.  That means that if we want a phasor to complete $f$ cycles/second, the angle between each sampled $\theta_s$ step will need to be a certain size in order to  complete a full cycle every $t_s$ seconds.

The units here help us figure this out: the desired frequency $f$ has units cycles/second. So, we can calculate what fraction of a complete cycle we need to take with each sample by multiplying $f$ with the sampling time $t_s$.
* $c_s = ft_s$.
* cycles/sample  = cycles/second x seconds/sample

We know each cycle is $2\pi$ radians (360 degrees), so we can then convert $c_s$ to an angle as follows:
* $ \theta_s = 2 \pi c_s $

### Exercise

Q: Calculate the period $t_s$ and angle $\theta_s$ between samples for a sine wave with frequency $f=8$ Hz and sampling rate of $f_s=64$

### Notes

```python

```

### Setting the Phasor Frequency

I've written a function `gen_phasors_vals_freq` that calculates the complex phasor values (`zs`), angles  (`thetas`) and time steps (`tsteps`) for a phasor with a given frequency `freq` over a given time period (`Tmin` to `Tmax`). In the following we'll use this to plot how changes in the phasor relate to changes in the corresponding sinusoid given a specific sampling rate (`sampling_rate`).

#### Example:

Let's look at a phasor and corresponding sine wave with frequency $f=2$ Hz (`freq`), given a sampling rate of $f_s=16$ (`sampling_rate`) over 4 seconds.

```python
## Our parameters:
Tmin = 0
Tmax = 4

freq = 2                 # cycles/second
sampling_rate = 16       # i.e, f_s above
t_step=1/sampling_rate   # i.e., t_s above
```

```python
## Get our complex values corresponding to the phasor with frequency freq
zs, thetas, tsteps = gen_phasor_vals_freq(Tmin=Tmin, Tmax=Tmax, t_step=t_step, freq=freq)

## Project to real and imaginary parts for plotting
Xs = np.real(zs)
Ys = np.imag(zs)

## generate the background for the plot: a phasor diagram on the left, a time v amplitude graph on the right
fig, phasor, sinusoid = create_anim_bkg(tsteps, thetas, freq)

## the phasor is plotted on the left on the left with a circle with radius 1 for reference
phasor.set_xlabel("Real values")
phasor.set_ylabel("Imaginary values")
# plot the points the phasor will "step on"
phasor.scatter(Xs, Ys)

## Plot our actual sampled sine wave in magenta on the right
sinusoid.plot(tsteps, Ys, 'o', color='magenta')
sinusoid.set_xlabel("Time (s)")
sinusoid.set_ylabel("Amplitude")
```

You should see two graphs above:
* On the left is the phasor diagram: the grey circle represents a phasor with magnitude 1, the red dots represents the points on the circle that the phasor samples between `tmin` and `tmax` given the `sampling_rate`.
* On the right is the time versus amplitude graph: The grey line shows a continuous sine wave with with frequency `freq`, the magenta dots show the points we actually sample between times `tmin` and `tmax` given the `sampling_rate`.

You can see that although we sample 64 points for the sine wave, we actually just hit the same 8 values per cycle on the phasor.

It's clearer when we animate it the phasor in time:

```python
## Now let's animate it!
## a helper to draw the 'clockhand' line
X, Y, n_samples = get_line_coords(Xs, Ys)
## initialize the animation
line = phasor.plot([], [], color='b', lw=3)[0]
sin_t = sinusoid.plot([], [], 'o', color='b')[0]
figs = (line, sin_t)

anim = FuncAnimation(
    fig, lambda x: anim_sinusoid(x, X=X, Y=Y, tsteps=tsteps, figs=figs), interval=600, frames=n_samples)

HTML(anim.to_html5_video())

```

### Exercise
Change the `freq` variable in the code below to investigate:

* What happens when the sine wave frequency (cycles/second) `freq` is set to `sampling_rate/2`?
* What happens when the frequency `freq` approaches the half the `sampling_rate`?
* What happens when the frequency `freq` equals the half the `sampling_rate`?
* What happens when the frequency `freq` is greater than `sampling_rate/2`

```python
## Example: Play around with these values
Tmax = 1
Tmin = 0

freq = 15   # cycles/second
sampling_rate = 16 # f_s above
t_step=1/sampling_rate

print("freq=%.2f cycles/sec, sampling rate=%.2f samples/sec, sampling period=%.2f sec" % (freq, sampling_rate, t_step) )

## Get our complex values corresponding to the sine wave
zs, thetas, tsteps = gen_phasor_vals_freq(Tmin=Tmin, Tmax=Tmax, t_step=t_step, freq=freq)

## Project to real and imaginary parts for plotting
Xs = np.real(zs)
Ys = np.imag(zs)

## generate the background
fig, phasor, sinusoid = create_anim_bkg(tsteps, thetas, freq)

## Plot the phasor samples
phasor.scatter(Xs, Ys)
phasor.set_xlabel("Real values")
phasor.set_ylabel("Imaginary values")

## Plot our actual sampled sine wave in magenta
sinusoid.plot(tsteps, Ys, 'o-', color='magenta')
sinusoid.set_xlabel("Time (s)")
sinusoid.set_ylabel("Amplitude")

## Animate the phasor and sinusoid
X, Y, n_samples = get_line_coords(Xs, Ys)

line = phasor.plot([], [], color='b', lw=3)[0]
sin_t = sinusoid.plot([], [], 'o', color='b')[0]
figs = (line, sin_t)
anim = FuncAnimation(
    fig, lambda x: anim_sinusoid(x, X=X, Y=Y, tsteps=tsteps, figs=figs), interval=600, frames=n_samples)

HTML(anim.to_html5_video())

```

###  Notes

```python

```

## Aliasing

If you change the frequency (`freq`) for the phasor to be higher than half the sampling rate , you'll see that the actual frequency of the sinusoid doesn't actually keep getting higher.  In fact, with `freq=8` the sine wave (i.e. projection of the vertical (imaginary) component) doesn't appear to have any amplitude modulation at all.  However, keen readers will note that for `sampling_rate=16` and `freq=8` in the example above, the real projection (i.e. cosine) would show amplitude modulations since $\cos(t)$ is 90 degree phase shifted relative to $\sin(t)$. The phasor `freq=15` appears to complete only one cycle per second, just like for `freq=1`, but appears to rotating the opposite way.

These are examples of **aliasing**: given a specific sampling rate there is a limit to which we can distinguish different frequencies because we simply can't take enough samples to show the difference!

In the example above, even though we are sampling from a 15 Hz wave for `freq=15`, we only get one sample per cycle and the overall sampled sequence looks like a 1 Hz wave. So, the fact that the phasor appears to rotate the opposite way to `freq=1` is because it's actually just the 15th step of the `freq=1` phasor.

In general, with a sampling rate of $f_s$ we can't distinguish between a sine wave of frequency $f_0$ and a sine wave of $f_0 + kf_s$ for any integer $k$.

This means that we can't actually tell the frequency of the underlying waveform based on the sample amplitudes alone.

The practical upshot of this is that for sampling rate $f_s$, the highest frequency we can actually sample is $f_s/2$, the **Nyquist Frequency**.  This is one of the most important concepts in digital signal processing and will effect pretty much all the methods we use.  It's why we see the mirroring effect in [the DFT output spectrum](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb). So, if you remember just one thing, remember this!

## Superposition

This use of phasors to represent sinusoids may seem excessively complex at the moment, but it actually gives us a nice way of visualizing what happens when we add two sine waves together, i.e. linear superposition.

We've seen how the Fourier Transform gives us a way of breaking down periodic waveforms (no matter how complicated) into a linear combination of sinusoids (cosine waves, specifically).  But if you've seen the actual DFT equations, you'll have noticed that each DFT output is actually is described in terms of phasors of specific frequencies (e.g. sums over $e^{-j \theta}$ values).  We can now get at least a visual idea of what this means.

Let's look at how can combining phasors can let us define complicated waveforms in a simple manner.

### Magnitude and Phase Modifications
First, let's note that we can easily change the magnitude and phase of a sine wave before adding it to others to make a complex waveform.

* We can change the magnitude of a sinusoidal component by multiplying all the values of that sinusoid by a scalar $r$.

* We can apply a phase shift of $\phi$ radians to $\sin(\theta)$ to gives us a sine wave of the form: $\sin(\theta + \phi)$.  It basically means we start our cycles around the unit circle at $e^{i\phi}$ instead of at $e^{i0} = 1 + i0 \mapsto (1,0)$

### Generating linear combinations of sinusoids

Let's plot some combinations of sinusoids.

First let's set the sampling rate and the start and end times of the sequence we're going to generate:

```python
## Some parameters to play with
Tmax = 2
Tmin = 0

sampling_rate = 16
t_step=1/sampling_rate
```

Now, let's create some phasors with different magnitudes, frequencies and phases. Here we create 2 phasors with magnitude 1 and no phase shift, one with `freq=2` Hz and another phasor with frequency `2*freq`.

We then add the two phasors values together at each timestep (`zs_sum` in the code below):

```python
## Define a bunch of sinusoids.  We can do this in terms of 3 parameters:
## (magnitude, frequency, phase)

## The following defines two sinusoids, both with magnitude (peak amplitude) 1 and the same phase (no phase shift)
## The second has double the frequency of the first:
freq=2
params = [(1, freq, 0), (1, 2*freq, 0)]

## Later: change these values and see what happens, e.g.
#params = [(1, freq, 0), (0.4, 5*freq, 0), (0.4, 5*freq, np.pi)]

phasor_list = []
theta_list = []
tsteps_list = []

## Generate a list of phasors for each set of (mag, freq, phase) parameters
for mag, freq, phase in params:
    ## Generate a phasor with frequency freq
    ## zs are the phasor values
    ## thetas are the corresponding angles for each value in zs
    ## tsteps are the corresponding time steps for each value in zs
    zs, thetas, tsteps = gen_phasor_vals_freq(Tmin=Tmin, Tmax=Tmax, t_step=t_step, freq=freq)

    ## Apply the phase_shift
    phase_shift = np.exp(1j*phase)

    ## scale by the magnitude mag - changes the peak amplitude
    zs = mag*zs*phase_shift

    ## Append the phasor to a list
    phasor_list.append(zs)

    ## The angle sequence and time sequence in case you want to inspect them
    ## We don't actually use them below
    theta_list.append(thetas)
    tsteps_list.append(tsteps)

## Superposition: add the individual phasors in the list together (all with the same weights right now)
zs_sum = np.zeros(len(tsteps_list[0]))
for z in phasor_list:
    zs_sum = zs_sum + z
```

Now, we can plot the sine (vertical) component of the individual phasors (on the right), ignoring the cosine (horizontal) component for the moment.

```python
## Plot the phasor (left) and the projection of the imaginary (vertical) component (right)
## cosproj would be the projection to the real axis, but let's just ignore that for now

fig, phasor, sinproj, cosproj = create_phasor_sinusoid_bkg(Tmin, Tmax, ymax=3, plot_phasor=True, plot_real=False, plot_imag=True,)

dense_tstep=0.001
for mag, freq, phase in params:
    ## We just want to plot the individual sinusoids (time v amplitude), so we ignore
    ## the complex numbers we've been using to plot the phasors
    _, dense_thetas, dense_tsteps = gen_phasor_vals_freq(Tmin, Tmax, dense_tstep, freq)
    sinproj.plot(dense_tsteps, mag*np.sin(dense_thetas+phase), color='grey')
```

Now plot the sum of the phasors (left) and the projected imaginary component in magenta (right) - that is, the sum of the sine components (in grey)

```python
## Plot sinusoids as sampled
Xlist = []
Ylist = []

## some hacks to get to represent the individual phasors as lines from the centre of a circle as well as points
for i, zs in enumerate(phasor_list):
    Xs_ = np.real(zs)
    Ys_ = np.imag(zs)
    X_, Y_, _ = get_line_coords(Xs_, Ys_)
    Xlist.append(X_)
    Ylist.append(Y_)

## Project the real and imaginary parts of the timewise summed phasor values
Xs = np.real(zs_sum)
Ys = np.imag(zs_sum)
Xline, Yline, _ = get_line_coords(Xs, Ys)

## plot the summed phasor values as 2-d coordinates (left)

## plot the sine projection of the phasor values in time (right)
sinproj.plot(tsteps_list[0], Ys, color='magenta')
fig
```

Now let's see an animation of how we're adding these phasors together!

```python
anim = get_phasor_animation(Xline, Yline, tsteps, phasor, sinproj, cosproj, fig, Xlist=Xlist, Ylist=Ylist, params=params)
anim
```

In the animation above you should see:
* the red circle represents the first phasor (`freq=2`)
* the blue circle represents the 2nd phasor (`freq=4`)
* In adding the the two phasors together, we add the corresponding vectors for each phasor at each point in time.

### Exercise:

* What happens when you add up two sinusoids with the same frequency but different magnitudes
    * e.g. `params = [(1, freq, 0), (2, freq, 0)]`

* What happens when you change the phase?
    * Can you find $\phi$ such that $\sin(\theta+\phi) = \cos(\theta)$ ?

* When do the individual sinusoids cancel each other out?

* Assume you have a compound sinusoid defined by the following params:
    * `params = [(1, freq, 0), (0.4, 5*freq, 0)]`
    * What sinusoid could you add to cancel the higher frequency component out while keeping the lower frequency one?

### Notes

```python

```

## Maths Perspective: The DFT equation as a sum of phasors

Now if you look at the mathematical form of the DFT, you can start to recognize this as representing a sequence of phasors of different frequencies, which have a real (cosine) and imaginary (sine) component.

The DFT is defined as follows:

* For input: $x[n]$, for $n=0..N-1$ (i.e. a time series of $N$ samples)
* We calculate an output of N complex numbers $\mapsto$ magnitude and phases of specific phasors:

Where the $k$th output, DFT[k], is calculated using the following equation:
$$
\begin{align}
DFT[k] &= \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi n}{N} k} \\
\end{align}
$$

Which is equivalent to the following (using Euler's rule):

$$
\begin{align}
DFT[k] &= \sum_{n=0}^{N-1} x[n]\big[\cos(\frac{2\pi n}{N} k) - j \sin(\frac{2\pi n}{N} k) \big]
\end{align}
$$

This basically says that each DFT output is the result of multiplying the $n$th input value $x[n]$ with the $n$th sample of a phasor (hence sine and cosine waves) of a specific frequency, and summing the result (hence the complex number output). The frequency of DFT[k] is $k$ times the frequency of DFT[1], where the frequency of DFT[1] depends on the input size $N$ and the sampling rate (as discussed the [this notebook](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb)). The sampling rate determines the time each phasor step takes, hence how much time it takes to make a full phasor cycle, hence what frequencies we can actually compare the input against.

The pointwise multiplication and summation is also known as a dot product (aka inner product).  The dot product between two vectors tells us how similar those two vectors are. So in a very rough sense, the DFT 'figures out' which frequency components are present in the input, by looking at how similar the input is to each of the N phasors represented in the DFT output.

There are two more notebooks on the DFT for this module, but both are extension material (not essential).

* [This notebook](./signals-1-3-discrete-fourier-transform-in-detail.ipynb) goes into more maths details but is purely extension (you can skip)
* [This notebook](./signals-1-4-more-interpreting-the-dft.ipynb) looks at a few more issues in interpreting the DFT

So, you can look at those if you want more details. Otherwise, we'll move onto the source-filter model in the second signals lab!

```python

```


# Triple Pendulum CHAOS!

*This notebook originally appeared as a [post](http://jakevdp.github.io/blog/2017/03/08/triple-pendulum-chaos/) on the blog [Pythonic Perambulations](http://jakevdp.github.io). The content is MIT licensed.*

Earlier this week a tweet made the rounds which features a video that nicely demonstrates chaotic dynamical systems in action:

A visualization of chaos: 41 triple pendulums with very slightly different initial conditions pic.twitter.com/CTiABFVWHW
— Fermat's Library (@fermatslibrary) March 5, 2017

*Edit: a reader pointed out that the original creator of this animation [posted it on reddit](https://www.reddit.com/r/mathpics/comments/4nd5h1/41_triple_pendulums_with_very_slightly_different/) in 2016.*

Naturally, I immediately wondered whether I could reproduce this simlulation in Python.
This post is the result.

This should have been pretty easy. After all, a while back I wrote a [Matplotlib Animation Tutorial](https://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/) containing a double pendulum example, simulated in Python using SciPy's [``odeint`` solver](https://docs.scipy.org/doc/scipy-0.17.0/reference/generated/scipy.integrate.odeint.html) and animated using matplotlib's [animation module](http://matplotlib.org/api/animation_api.html).
All we need to do is to extend that to a three-segment pendulum, and we're good to go, right?

Unfortunately, things are not so simple.
While the double pendulum equations of motion can be solved [relatively straightforwardly](http://scienceworld.wolfram.com/physics/DoublePendulum.html), the equations for a triple pendulum are much more involved.
For example, the appendix of [this document](https://www.nickeyre.com/images/triplependulum.pdf) lists the three coupled second-order differential equations that govern the motion of the a triple pendulum;
here's a screenshot of just the first of those three:

Yikes.

Fortunately, there are easier approaches than brute-force algebra, that rely on higher abstractions: one such approach is known as [Kane's Method](http://docs.sympy.org/dev/modules/physics/mechanics/kane.html).
This method still involves a significant amount of book-keeping for any but the most trivial problems, but the [Sympy package](http://sympy.org) has a nice implementation that handles the details for you.
This is the approach I took to simulate the triple pendulum, borrowing heavily from  [Gede et al. 2013](https://www.researchgate.net/publication/267490975_Constrained_Multibody_Dynamics_With_Python_From_Symbolic_Equation_Generation_to_Publication) who present a nice example of Sympy's API for applying Kane's Method.

## The Code

The following function defines and solves the equations of motion for a system of *n* pendulums, with arbitrary masses and lengths.
It's a bit long, but hopefully commented well enough that you can follow along.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

from sympy import symbols
from sympy.physics import mechanics

from sympy import Dummy, lambdify
from scipy.integrate import odeint

def integrate_pendulum(n, times,
                       initial_positions=135,
                       initial_velocities=0,
                       lengths=None, masses=1):
    """Integrate a multi-pendulum with `n` sections"""
    #-------------------------------------------------
    # Step 1: construct the pendulum model

    # Generalized coordinates and velocities
    # (in this case, angular positions & velocities of each mass)
    q = mechanics.dynamicsymbols('q:{0}'.format(n))
    u = mechanics.dynamicsymbols('u:{0}'.format(n))

    # mass and length
    m = symbols('m:{0}'.format(n))
    l = symbols('l:{0}'.format(n))

    # gravity and time symbols
    g, t = symbols('g,t')

    #--------------------------------------------------
    # Step 2: build the model using Kane's Method

    # Create pivot point reference frame
    A = mechanics.ReferenceFrame('A')
    P = mechanics.Point('P')
    P.set_vel(A, 0)

    # lists to hold particles, forces, and kinetic ODEs
    # for each pendulum in the chain
    particles = []
    forces = []
    kinetic_odes = []

    for i in range(n):
        # Create a reference frame following the i^th mass
        Ai = A.orientnew('A' + str(i), 'Axis', [q[i], A.z])
        Ai.set_ang_vel(A, u[i] * A.z)

        # Create a point in this reference frame
        Pi = P.locatenew('P' + str(i), l[i] * Ai.x)
        Pi.v2pt_theory(P, A, Ai)

        # Create a new particle of mass m[i] at this point
        Pai = mechanics.Particle('Pa' + str(i), Pi, m[i])
        particles.append(Pai)

        # Set forces & compute kinematic ODE
        forces.append((Pi, m[i] * g * A.x))
        kinetic_odes.append(q[i].diff(t) - u[i])

        P = Pi

    # Generate equations of motion
    KM = mechanics.KanesMethod(A, q_ind=q, u_ind=u,
                               kd_eqs=kinetic_odes)
    fr, fr_star = KM.kanes_equations(forces, particles)

    #-----------------------------------------------------
    # Step 3: numerically evaluate equations and integrate

    # initial positions and velocities – assumed to be given in degrees
    y0 = np.deg2rad(np.concatenate([np.broadcast_to(initial_positions, n),
                                    np.broadcast_to(initial_velocities, n)]))

    # lengths and masses
    if lengths is None:
        lengths = np.ones(n) / n
    lengths = np.broadcast_to(lengths, n)
    masses = np.broadcast_to(masses, n)

    # Fixed parameters: gravitational constant, lengths, and masses
    parameters = [g] + list(l) + list(m)
    parameter_vals = [9.81] + list(lengths) + list(masses)

    # define symbols for unknown parameters
    unknowns = [Dummy() for i in q + u]
    unknown_dict = dict(zip(q + u, unknowns))
    kds = KM.kindiffdict()

    # substitute unknown symbols for qdot terms
    mm_sym = KM.mass_matrix_full.subs(kds).subs(unknown_dict)
    fo_sym = KM.forcing_full.subs(kds).subs(unknown_dict)

    # create functions for numerical calculation
    mm_func = lambdify(unknowns + parameters, mm_sym)
    fo_func = lambdify(unknowns + parameters, fo_sym)

    # function which computes the derivatives of parameters
    def gradient(y, t, args):
        vals = np.concatenate((y, args))
        sol = np.linalg.solve(mm_func(*vals), fo_func(*vals))
        return np.array(sol).T[0]

    # ODE integration
    return odeint(gradient, y0, times, args=(parameter_vals,))
```

## Extracting Positions

The function above returns generalized coordinates, which in this case are the angular position and velocity of each pendulum segment, relative to vertical.
To visualize the pendulum, we need a quick utility to extract *x* and *y* coordinates from these angular coordinates:

```python
def get_xy_coords(p, lengths=None):
    """Get (x, y) coordinates from generalized coordinates p"""
    p = np.atleast_2d(p)
    n = p.shape[1] // 2
    if lengths is None:
        lengths = np.ones(n) / n
    zeros = np.zeros(p.shape[0])[:, None]
    x = np.hstack([zeros, lengths * np.sin(p[:, :n])])
    y = np.hstack([zeros, -lengths * np.cos(p[:, :n])])
    return np.cumsum(x, 1), np.cumsum(y, 1)
```

Finally, we can call this function to simulate a pendulum at a set of times *t*.
Here are the paths of a double pendulum over time:

```python
t = np.linspace(0, 10, 1000)
p = integrate_pendulum(n=2, times=t)
x, y = get_xy_coords(p)
plt.plot(x, y);
```

And here are the positions of a triple pendulum over time:

```python
p = integrate_pendulum(n=3, times=t)
x, y = get_xy_coords(p)
plt.plot(x, y);
```

## Pendulum Animations

The static plots above provide a bit of insight into the situation, but it's much more intuitive to see the results as an animation.
Drawing from the double pendulum code in my [Animation Tutorial](https://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/), here is a function to animate the pendulum's motion over time:

```python
from matplotlib import animation

def animate_pendulum(n):
    t = np.linspace(0, 10, 200)
    p = integrate_pendulum(n, t)
    x, y = get_xy_coords(p)

    fig, ax = plt.subplots(figsize=(6, 6))
    fig.subplots_adjust(left=0, right=1, bottom=0, top=1)
    ax.axis('off')
    ax.set(xlim=(-1, 1), ylim=(-1, 1))

    line, = ax.plot([], [], 'o-', lw=2)

    def init():
        line.set_data([], [])
        return line,

    def animate(i):
        line.set_data(x[i], y[i])
        return line,

    anim = animation.FuncAnimation(fig, animate, frames=len(t),
                                   interval=1000 * t.max() / len(t),
                                   blit=True, init_func=init)
    plt.close(fig)
    return anim
```

```python
anim = animate_pendulum(3)
```

In the notebook, you can embed the resulting animation using the following snippet:

```python
from IPython.display import HTML
HTML(anim.to_html5_video())
```

Embedding videos directly like this can lead to very large file sizes for notebooks (and blog posts!), so instead I've saved the animation and will load it with an HTML5 video tag.
This is less portable (because the video file is separate from the notebook) but avoids embedding the entire content of the video into the single notebook file.

```python
from IPython.display import HTML
# anim.save('triple-pendulum.mp4', extra_args=['-vcodec', 'libx264'])
HTML('<video controls loop src="http://jakevdp.github.io/videos/triple-pendulum.mp4" />')
```

<video controls loop src="http://jakevdp.github.io/videos/triple-pendulum.mp4" />

The nice part about the Kane method implemented above is that it readily generalizes to larger compound pendulums; for example, here is an animation of a 5-part "quintuple pendulum".
It takes a bit longer to run, but can be computed with the same code:

```python
anim = animate_pendulum(5)
# HTML(anim.to_html5_video())  # uncomment to embed
```

```python
# anim.save('quintuple-pendulum.mp4', extra_args=['-vcodec', 'libx264'])
HTML('<video controls loop src="http://jakevdp.github.io/videos/quintuple-pendulum.mp4" />')
```

<video controls loop src="http://jakevdp.github.io/videos/quintuple-pendulum.mp4" />

## And a Butterfly Flaps its Wings...

And now back to the motivation of this post: illustrating the chaotic results of small perturbations in initial conditions.
The following code animates any specified number of nearly identical pendulums, each with a starting location perturbed slightly (here by about one part in a million):

```python
from matplotlib import collections

def animate_pendulum_multiple(n, n_pendulums=41, perturbation=1E-6, track_length=15):
    oversample = 3
    track_length *= oversample

    t = np.linspace(0, 10, oversample * 200)
    p = [integrate_pendulum(n, t, initial_positions=135 + i * perturbation / n_pendulums)
         for i in range(n_pendulums)]
    positions = np.array([get_xy_coords(pi) for pi in p])
    positions = positions.transpose(0, 2, 3, 1)
    # positions is a 4D array: (npendulums, len(t), n+1, xy)

    fig, ax = plt.subplots(figsize=(6, 6))
    fig.subplots_adjust(left=0, right=1, bottom=0, top=1)
    ax.axis('off')
    ax.set(xlim=(-1, 1), ylim=(-1, 1))

    track_segments = np.zeros((n_pendulums, 0, 2))
    tracks = collections.LineCollection(track_segments, cmap='rainbow')
    tracks.set_array(np.linspace(0, 1, n_pendulums))
    ax.add_collection(tracks)

    points, = plt.plot([], [], 'ok')

    pendulum_segments = np.zeros((n_pendulums, 0, 2))
    pendulums = collections.LineCollection(pendulum_segments, colors='black')
    ax.add_collection(pendulums)

    def init():
        pendulums.set_segments(np.zeros((n_pendulums, 0, 2)))
        tracks.set_segments(np.zeros((n_pendulums, 0, 2)))
        points.set_data([], [])
        return pendulums, tracks, points

    def animate(i):
        i = i * oversample
        pendulums.set_segments(positions[:, i])
        sl = slice(max(0, i - track_length), i)
        tracks.set_segments(positions[:, sl, -1])
        x, y = positions[:, i].reshape(-1, 2).T
        points.set_data(x, y)
        return pendulums, tracks, points

    interval = 1000 * oversample * t.max() / len(t)
    anim = animation.FuncAnimation(fig, animate, frames=len(t) // oversample,
                                   interval=interval,
                                   blit=True, init_func=init)
    plt.close(fig)
    return anim

anim = animate_pendulum_multiple(3)
# HTML(anim.to_html5_video())  # uncomment to embed
```

```python
# anim.save('triple-pendulum-chaos.mp4', extra_args=['-vcodec', 'libx264'])
HTML('<video controls loop src="http://jakevdp.github.io/videos/triple-pendulum-chaos.mp4" />')
```

<video controls loop src="http://jakevdp.github.io/videos/triple-pendulum-chaos.mp4" />

Quite similar to the video that made the rounds on twitter, but the benefit here is that the result can be reproduced from scratch with the above code.

Thanks for reading!

*This post was written entirely in the IPython notebook.  You can
[download](http://jakevdp.github.io/downloads/notebooks/TriplePendulum.ipynb)
this notebook, or see a static view
[here](http://nbviewer.ipython.org/url/jakevdp.github.io/downloads/notebooks/TriplePendulum.ipynb).*


# Бифуркации $I_{Na,p}+I_K$ модели и интерактивные графики на фазовой плоскости

```python
# Графики будут сохраняться в сам документ:

%pylab inline

```

    Populating the interactive namespace from numpy and matplotlib

```python
from __future__ import division
```

```python
# Для интерактивных графиков, легко установить, если используете Anaconda
import bokeh
import bokeh.plotting as bp
from bokeh.models import ColumnDataSource, Range1d

from ipywidgets import interact
```

```python
bp.output_notebook()

```

    BokehJS successfully loaded.

```python
# Устанавливается командой `pip install PyDSTool`
import PyDSTool as dst
from PyDSTool.Toolbox import phaseplane as pp
```

## Модель $I_{Na,p}+I_K$

\begin{equation}
C\dot{V} = I - (\bar{g}_Kn(V-E_K) + \bar{g}_{Na}m_\infty(V)(V-E_{Na}) + gl(V-El))
\end{equation}

\begin{equation}
\tau_n n = (n_\infty(V) - n)
\end{equation}

\begin{equation}
x_\infty =  \frac{1}{1 + \exp(\frac{V_{1/2}-V}{k})}
\end{equation}

```python
napk_pset = dict(
    I = 0.0,
    El = -80.0,
    Ena = 60.0,
    Ek = -90.0,
    gl = 8.0,
    gna = 20.0,
    gk = 10.0,
    ntau = 5.0,
    minf_vhalf = -20.,
    minfk = 15.,
    ninf_vhalf = -25.,
    ninfk = 5.0
)
```

```python
pset_str = ';\n'.join(['{k}=dst.Par({v},"{k}")'.format(k=k,v=v) for k,v in napk_pset.items()])
print pset_str
exec(pset_str)
```

    ntau=dst.Par(5.0,"ntau");
    Ena=dst.Par(60.0,"Ena");
    Ek=dst.Par(-90.0,"Ek");
    El=dst.Par(-80.0,"El");
    I=dst.Par(0.0,"I");
    minf_vhalf=dst.Par(-20.0,"minf_vhalf");
    ninf_vhalf=dst.Par(-25.0,"ninf_vhalf");
    minfk=dst.Par(15.0,"minfk");
    gna=dst.Par(20.0,"gna");
    ninfk=dst.Par(5.0,"ninfk");
    gl=dst.Par(8.0,"gl");
    gk=dst.Par(10.0,"gk")

```python
# Переменные состояния
V = dst.Var('V')
n = dst.Var('n')
```

```python
# Параметризация зависимости x_inf от потенциала
boltzman = dst.Fun(1./(1. + dst.Exp(('Vhalf'- V)/'bk')), ['Vhalf','bk'], 'boltzman')
minf = boltzman(minf_vhalf,minfk)
ninf = boltzman(ninf_vhalf,ninfk)
```

```python
print minf
print 'At -10 mV:', minf.eval(V=-10)
```

    1.0/(1.0+Exp((minf_vhalf-V)/minfk))
    At -10 mV: 1.0/(1.0+Exp((minf_vhalf+10)/minfk))

```python
vtest = linspace(-89, 45, 250)
```

```python
plot(vtest, eval(str(minf.eval(V='vtest', Exp='np.exp', **napk_pset))))
plot(vtest, eval(str(ninf.eval(V='vtest', Exp='np.exp',**napk_pset))))
xlabel(u'V, mV'); legend(('$m_\infty$', '$n_\infty$'),loc='best')
```

```python
# Уравнение для активации К-тока
dn = (ninf-n)/ntau
```

```python
print dn
```

    ((1.0/(1.0+Exp((ninf_vhalf-V)/ninfk)))-n)/ntau

```python
# Символьное описание токов и уравнение для dV/dt

iK = gk*(V-Ek)*n
iNa = gna*(V-Ena)*minf
ileak = gl*(V-El)
dV = I - (ileak + iNa + iK)
print dV
```

    I-(gl*(V-El)+gna*(V-Ena)*1.0/(1.0+Exp((minf_vhalf-V)/minfk))+gk*(V-Ek)*n)

```python
print dV
```

    I-(gl*(V-El)+gna*(V-Ena)*1.0/(1.0+Exp((minf_vhalf-V)/minfk))+gk*(V-Ek)*n)

```python
Iinf = iNa + gk*(V-Ek)*ninf + ileak
vnull = (I - (ileak + iNa))/(gk*(V-Ek))
```

```python
print 'Iinf = ', Iinf
print 'V-nullcline:', vnull
```

    Iinf =  gna*(V-Ena)*1.0/(1.0+Exp((minf_vhalf-V)/minfk))+gk*(V-Ek)*1.0/(1.0+Exp((ninf_vhalf-V)/ninfk))+gl*(V-El)
    V-nullcline: (I-(gl*(V-El)+gna*(V-Ena)*1.0/(1.0+Exp((minf_vhalf-V)/minfk))))/(gk*(V-Ek))

```python
vnull_expr = str(vnull.eval(V='vtest',Exp='np.exp', **napk_pset))
```

```python
# Собственно, создаем модель и передаем ее PyDSTool
NapKmodel = dst.args(
    name = 'napk',
    pars = napk_pset,
    varspecs = {'V':dV, 'n':dn},
    tdomain=[0,250],
    xdomain=dict(V=[-150, 60], n=[0,1]),
    ics = {'V':-70,'n':0})

odeset = dst.Generator.Vode_ODEsystem(NapKmodel)
```

```python
# Посчитаем траекторию (динамику) для примера
traj = odeset.compute('test')
pts = traj.sample(dt=0.1)
```

```python
vnull_0 = eval(vnull_expr)
nnull_0 =eval(str(ninf.eval(V='vtest',Exp='exp',**napk_pset)))
Iinf_0 = eval(str(Iinf.eval(V='vtest',Exp='exp',**napk_pset)))

source_nulls = ColumnDataSource(data=dict(x=vtest,
                                          vnull=vnull_0,
                                          nnull=nnull_0,
                                          Iinf=Iinf_0))
source_traj = ColumnDataSource(data=dict(t=pts['t'],v=pts['V'],n=pts['n']))

def warp_eval(x,args):
    return eval(str(x.eval(args)))

def update(I=0, vhalf_n=-25, ntaux=0.2):
    I_before = napk_pset['I']
    vhalf_n_before = napk_pset['ninf_vhalf']
    napk_pset['I'] = I
    napk_pset['ninf_vhalf'] = vhalf_n
    napk_pset['ntau'] = ntaux

    args = dict(V='vtest',Exp='exp',**napk_pset)
    source_nulls.data['vnull'] = warp_eval(vnull,args)
    source_nulls.data['nnull'] = warp_eval(ninf,args)
    source_nulls.data['Iinf'] = warp_eval(Iinf,args)

    odeset.set(pars=napk_pset)
    traj = odeset.compute('test')
    pts = traj.sample(dt=0.025)
    source_traj.data['t'] = pts['t']
    source_traj.data['v'] = pts['V']
    source_traj.data['n'] = pts['n']

    bokeh.io.push_notebook()
```

```python
p1 = bp.figure(title=u"Фазовый портрет", plot_height=500, plot_width=500,
              x_axis_label='V', y_axis_label='n')
p1.line(vtest, y='vnull', source=source_nulls,  line_width=1.5, color='magenta',  legend="dV/dt=0")
p1.line(vtest, 'nnull', source=source_nulls, color='teal', line_width=1.5,legend='dn/dt=0')
p1.line(x='v',y='n', source=source_traj, line_width=1)
#p1.legend.orientation = "top_right"

p1.y_range = Range1d(-0.2,2)

p2 = bp.figure(title=u"Равновесный ток", plot_height=500, plot_width=500,
              x_axis_label='V', y_axis_label='Iinf')
p2.line(vtest, y='Iinf', source=source_nulls,  line_width=1,  color='black')
p2.y_range = Range1d(-100,100)

p3 = bp.figure(title=u"Динамика (V)", plot_width=900, plot_height=200)
p3.line(x='t',y='v', source=source_traj)

bp.show(bp.vplot(bp.hplot(p1,p2),p3))

```

    <bokeh.io._CommsHandle at 0x7f491641d9d0>

```python
interact(update, I=[-10, 40, 0.1], vhalf_n=[-40,-20], ntaux=[0.1,2,0.01])
```

todo: demonstrate bistability for SN

```python
p1.
```

INa,p+IK model with interactive plots.ipynb


# Physics 256
## Random Number Generators

http://www.idquantique.com/random-number-generation/

## Last Time

- Error scaling for high dimensional quadrature
- Monte Carlo Integration

## Today

- Generation and testing of pseudorandom numbers
- Tower sampling

## Setting up the Notebook

```python
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
plt.style.use('notebook');
%config InlineBackend.figure_format = 'retina'
colors = ["#2078B5", "#FF7F0F", "#2CA12C", "#D72827", "#9467BE", "#8C574B",
            "#E478C2", "#808080", "#BCBE20", "#17BED0", "#AEC8E9", "#FFBC79",
            "#98E08B", "#FF9896", "#C6B1D6", "#C59D94", "#F8B7D3", "#C8C8C8",
           "#DCDC8E", "#9EDAE6"]
```

## Generation of Random Numbers

### What is a random number?

- There is really no such thing, definitely not on a deterministic classical computer
- Loose term applied to a sequence of independent numbers drawn randomly from some distribution
- Typically we select integer or real values on some finite domain

What of the simplest ways to generate uniformly distrubted random numbers on $[0,1]$ is the **Linear Congruential Generator** (LCG).

Consider the map (recursion relation) which generates integers between $0$ and $m-1$:

\begin{equation}
X_{n+1} = (a X_n + c) \mod m
\end{equation}

where $a$ is known as the multiplier, $c$ is the increment and $m$ the modulus.  Starting from an initial **seed** value $X_0$ we generate the list of numbers:

\begin{align*}
X_0 &= \text{seed} \newline
X_1 &= (a X_0 + c) \mod m \newline
X_2 &= \{a [a (X_0 + c) \mod m] + c\} \mod m \newline
\vdots &
\end{align*}

Then, a uniform number $x_n \in \mathcal{U}_{[0,1]}$ can be computed as:

\begin{equation}
x_n = \frac{X_n}{m} .
\end{equation}

Programming challenge

Write a LCG function with $a=16807$, $c=0$, $m=2^{31}-1$ and $seed=332$ that generates a list of *pseudorandom* uniform numbers of length $N=1000$ on [0,1].

```python
def lcg_rand(a,c,m,seed,N=1):
    '''A linear congruential pseudrandom number generator'''
    x = np.zeros([N])
    X = seed
    x[0] = X/m
    for n in range(N-1):
        X = (a*X + c) % m
        x[n+1] = X/m
    return x

N = 1000
a,c,m,seed = 16807,0,2**31-1,332
m = 2**32-1
a = 7
c = 7
seed = 7
x = lcg_rand(a,c,m,seed,N)
print(x)
```

    [  1.62981451e-09   1.30385160e-08   9.28994268e-08   6.51925802e-07
       4.56511043e-06   3.19574028e-05   2.23703450e-04   1.56592578e-03
       1.09614821e-02   7.67303761e-02   5.37112635e-01   7.59788443e-01
       3.18519105e-01   2.29633734e-01   6.07436137e-01   2.52052964e-01
       7.64370746e-01   3.50595226e-01   4.54166586e-01   1.79166105e-01
       2.54162734e-01   7.79139143e-01   4.53974000e-01   1.77818004e-01
       2.44726027e-01   7.13082190e-01   9.91575334e-01   9.41027338e-01
       5.87191367e-01   1.10339570e-01   7.72376994e-01   4.06638960e-01
       8.46472723e-01   9.25309064e-01   4.77163447e-01   3.40144130e-01
       3.81008913e-01   6.67062395e-01   6.69436768e-01   6.86057377e-01
       8.02401638e-01   6.16811467e-01   3.17680267e-01   2.23761874e-01
       5.66333118e-01   9.64331831e-01   7.50322819e-01   2.52259736e-01
       7.65818152e-01   3.60727066e-01   5.25089464e-01   6.75626248e-01
       7.29383740e-01   1.05686179e-01   7.39803252e-01   1.78622769e-01
       2.50359383e-01   7.52515683e-01   2.67609780e-01   8.73268463e-01
       1.12879245e-01   7.90154719e-01   5.31083034e-01   7.17581243e-01
       2.30687023e-02   1.61480918e-01   1.30366427e-01   9.12564988e-01
       3.87954921e-01   7.15684448e-01   9.79113486e-03   6.85379456e-02
       4.79765621e-01   3.58359350e-01   5.08515448e-01   5.59608139e-01
       9.17256975e-01   4.20798824e-01   9.45591768e-01   6.19142379e-01
       3.33996657e-01   3.37976598e-01   3.65836189e-01   5.60853328e-01
       9.25973296e-01   4.81813073e-01   3.72691513e-01   6.08840592e-01
       2.61884145e-01   8.33189015e-01   8.32323108e-01   8.26261760e-01
       7.83832319e-01   4.86826238e-01   4.07783668e-01   8.54485680e-01
       9.81399762e-01   8.69798337e-01   8.85883612e-02   6.20118530e-01
       3.40829711e-01   3.85807979e-01   7.00655853e-01   9.04590973e-01
       3.32136814e-01   3.24957697e-01   2.74703881e-01   9.22927167e-01
       4.60490174e-01   2.23431220e-01   5.64018539e-01   9.48129775e-01
       6.36908427e-01   4.58358988e-01   2.08512915e-01   4.59590404e-01
       2.17132831e-01   5.19929817e-01   6.39508723e-01   4.76561061e-01
       3.35927427e-01   3.51491990e-01   4.60443931e-01   2.23107517e-01
       5.61752623e-01   9.32268360e-01   5.25878521e-01   6.81149649e-01
       7.68047544e-01   3.76332809e-01   6.34329664e-01   4.40307646e-01
       8.21535248e-02   5.75074675e-01   2.55227261e-02   1.78659085e-01
       2.50613593e-01   7.54295155e-01   2.80066086e-01   9.60462602e-01
       7.23238218e-01   6.26675263e-02   4.38672686e-01   7.07088027e-02
       4.94961620e-01   4.64731345e-01   2.53119414e-01   7.71835898e-01
       4.02851289e-01   8.19959026e-01   7.39713185e-01   1.77992299e-01
       2.45946098e-01   7.21622687e-01   5.13588123e-02   3.59511688e-01
       5.16581814e-01   6.16072701e-01   3.12508907e-01   1.87562349e-01
       3.12936445e-01   1.90555117e-01   3.33885822e-01   3.37200753e-01
       3.60405271e-01   5.22836899e-01   6.59858293e-01   6.19008055e-01
       3.33056384e-01   3.31394690e-01   3.19762833e-01   2.38339831e-01
       6.68378820e-01   6.78651741e-01   7.50562187e-01   2.53935307e-01
       7.77547153e-01   4.42830072e-01   9.98105048e-02   6.98673536e-01
       8.90714750e-01   2.35003254e-01   6.45022781e-01   5.15159469e-01
       6.06116283e-01   2.42813982e-01   6.99697875e-01   8.97885126e-01
       2.85195882e-01   9.96371177e-01   9.74598244e-01   8.22187710e-01
       7.55313972e-01   2.87197805e-01   1.03846388e-02   7.26924732e-02
       5.08847314e-01   5.61931200e-01   9.33518402e-01   5.34628813e-01
       7.42401690e-01   1.96811830e-01   3.77682808e-01   6.43779659e-01
       5.06457614e-01   5.45203297e-01   8.16423083e-01   7.14961580e-01
       4.73106443e-03   3.31174526e-02   2.31822170e-01   6.22755193e-01
       3.59286350e-01   5.15004450e-01   6.05031154e-01   2.35218080e-01
       6.46526560e-01   5.25685924e-01   6.79801470e-01   7.58610289e-01
       3.10272023e-01   1.71904164e-01   2.03329148e-01   4.23304035e-01
       9.63128245e-01   7.41897715e-01   1.93284005e-01   3.52988035e-01
       4.70916245e-01   2.96413718e-01   7.48960299e-02   5.24272211e-01
       6.69905479e-01   6.89338353e-01   8.25368472e-01   7.77579306e-01
       4.43055146e-01   1.01386024e-01   7.09702168e-01   9.67915175e-01
       7.75406225e-01   4.27843577e-01   9.94905041e-01   9.64335291e-01
       7.50347037e-01   2.52429258e-01   7.67004805e-01   3.69033639e-01
       5.83235474e-01   8.26483211e-02   5.78538249e-01   4.97677457e-02
       3.48374221e-01   4.38619551e-01   7.03368578e-02   4.92358006e-01
       4.46506047e-01   1.25542328e-01   8.78796295e-01   1.51574063e-01
       6.10184453e-02   4.27129119e-01   9.89903833e-01   9.29326834e-01
       5.05287842e-01   5.37014897e-01   7.59104283e-01   3.13729983e-01
       1.96109880e-01   3.72769164e-01   6.09384146e-01   2.65689027e-01
       8.59823188e-01   1.87623177e-02   1.31336226e-01   9.19353581e-01
       4.35475072e-01   4.83255056e-02   3.38278541e-01   3.67949787e-01
       5.75648508e-01   2.95395604e-02   2.06776925e-01   4.47438475e-01
       1.32069324e-01   9.24485269e-01   4.71396886e-01   2.99778205e-01
       9.84474346e-02   6.89132044e-01   8.23924310e-01   7.67470170e-01
       3.72291192e-01   6.06038343e-01   2.42268400e-01   6.95878801e-01
       8.71151606e-01   9.80612440e-02   6.86428710e-01   8.05000970e-01
       6.35006794e-01   4.45047559e-01   1.15332915e-01   8.07330408e-01
       6.51312860e-01   5.59190021e-01   9.14330147e-01   4.00311032e-01
       8.02177225e-01   6.15240578e-01   3.06684048e-01   1.46788334e-01
       2.75183416e-02   1.92628393e-01   3.48398749e-01   4.38791247e-01
       7.15387303e-02   5.00771114e-01   5.05397798e-01   5.37784589e-01
       7.64492122e-01   3.51444857e-01   4.60113999e-01   2.20797997e-01
       5.45585979e-01   8.19101856e-01   7.33712991e-01   1.35990937e-01
       9.51936558e-01   6.63555905e-01   6.44891338e-01   5.14239365e-01
       5.99675557e-01   1.97728904e-01   3.84102330e-01   6.88716310e-01
       8.21014168e-01   7.47099180e-01   2.29694265e-01   6.07859853e-01
       2.55018975e-01   7.85132826e-01   4.95929785e-01   4.71508500e-01
       3.00559499e-01   1.03916492e-01   7.27415448e-01   9.19081376e-02
       6.43356965e-01   5.03498754e-01   5.24491279e-01   6.71438953e-01
       7.00072675e-01   9.00508725e-01   3.03561074e-01   1.24927518e-01
       8.74492629e-01   1.21448406e-01   8.50138842e-01   9.50971898e-01
       6.56803287e-01   5.97623011e-01   1.83361077e-01   2.83527541e-01
       9.84692787e-01   8.92849513e-01   2.49946590e-01   7.49626131e-01
       2.47382917e-01   7.31680419e-01   1.21762936e-01   8.52340555e-01
       9.66383890e-01   7.64687232e-01   3.52810623e-01   4.69674366e-01
       2.87720562e-01   1.40439326e-02   9.83075295e-02   6.88152708e-01
       8.17068958e-01   7.19482707e-01   3.63789508e-02   2.54652657e-01
       7.82568603e-01   4.77980224e-01   3.45861572e-01   4.21031003e-01
       9.47217026e-01   6.30519182e-01   4.13634276e-01   8.95439936e-01
       2.68079554e-01   8.76556880e-01   1.35898160e-01   9.51287124e-01
       6.59009868e-01   6.13069081e-01   2.91483566e-01   4.03849653e-02
       2.82694758e-01   9.78863311e-01   8.52043177e-01   9.64302240e-01
       7.50115684e-01   2.50809792e-01   7.55668549e-01   2.89679845e-01
       2.77589161e-02   1.94312415e-01   3.60186904e-01   5.21308332e-01
       6.49158323e-01   5.44108263e-01   8.08757845e-01   6.61304917e-01
       6.29134420e-01   4.03940940e-01   8.27586581e-01   7.93106068e-01
       5.51742480e-01   8.62197362e-01   3.53815346e-02   2.47670744e-01
       7.33695209e-01   1.35866464e-01   9.51065252e-01   6.57456768e-01
       6.02197375e-01   2.15381629e-01   5.07671402e-01   5.53699817e-01
       8.75898720e-01   1.31291043e-01   9.19037303e-01   4.33261122e-01
       3.28278570e-02   2.29795000e-01   6.08565004e-01   2.59955030e-01
       8.19685208e-01   7.37796460e-01   1.64575222e-01   1.52026552e-01
       6.41858676e-02   4.49301075e-01   1.45107524e-01   1.57526683e-02
       1.10268679e-01   7.71880758e-01   4.03165307e-01   8.22157152e-01
       7.55100067e-01   2.85700470e-01   9.99903294e-01   9.99323057e-01
       9.95261397e-01   9.66829784e-01   7.67808488e-01   3.74659416e-01
       6.22615916e-01   3.58311414e-01   5.08179902e-01   5.57259316e-01
       9.00815210e-01   3.05706474e-01   1.39945317e-01   9.79617220e-01
       8.57320539e-01   1.24377571e-03   8.70643161e-03   6.09450229e-02
       4.26615162e-01   9.86306133e-01   9.04142936e-01   3.29000552e-01
       3.03003864e-01   1.21027051e-01   8.47189359e-01   9.30325514e-01
       5.12278597e-01   5.85950184e-01   1.01651288e-01   7.11559020e-01
       9.80913143e-01   8.66392000e-01   6.47440038e-02   4.53208028e-01
       1.72456201e-01   2.07193407e-01   4.50353849e-01   1.52476947e-01
       6.73386296e-02   4.71370409e-01   2.99592865e-01   9.71500599e-02
       6.80050421e-01   7.60352948e-01   3.22470640e-01   2.57294484e-01
       8.01061391e-01   6.07429741e-01   2.52008188e-01   7.64057315e-01
       3.48401208e-01   4.38808455e-01   7.16591834e-02   5.01614285e-01
       5.11299999e-01   5.79099992e-01   5.36999453e-02   3.75899619e-01
       6.31297335e-01   4.19081346e-01   9.33569425e-01   5.34985978e-01
       7.44901850e-01   2.14312949e-01   5.00190643e-01   5.01334504e-01
       5.09341532e-01   5.65390726e-01   9.57735082e-01   7.04145575e-01
       9.29019026e-01   5.03133182e-01   5.21932274e-01   6.53525923e-01
       5.74681461e-01   2.27702318e-02   1.59391624e-01   1.15741373e-01
       8.10189610e-01   6.71327270e-01   6.99290895e-01   8.95036267e-01
       2.65253871e-01   8.56777101e-01   9.97439706e-01   9.82077940e-01
       8.74545584e-01   1.21819091e-01   8.52733637e-01   9.69135463e-01
       7.83948246e-01   4.87637723e-01   4.13464061e-01   8.94248428e-01
       2.59738995e-01   8.18172964e-01   7.27210748e-01   9.04752373e-02
       6.33326662e-01   4.33286638e-01   3.30064700e-02   2.31045291e-01
       6.17317041e-01   3.21219289e-01   2.48535025e-01   7.39745177e-01
       1.78216238e-01   2.47513665e-01   7.32595656e-01   1.28169595e-01
       8.97187169e-01   2.80310182e-01   9.62171279e-01   7.35198952e-01
       1.46392667e-01   2.47486706e-02   1.73240696e-01   2.12684874e-01
       4.88794123e-01   4.21558862e-01   9.50912037e-01   6.56384260e-01
       5.94689822e-01   1.62828757e-01   1.39801301e-01   9.78609106e-01
       8.50263744e-01   9.51846209e-01   6.62923462e-01   6.40464237e-01
       4.83249658e-01   3.82747609e-01   6.79233262e-01   7.54632832e-01
       2.82429829e-01   9.77008802e-01   8.39061615e-01   8.73431305e-01
       1.14019138e-01   7.98133965e-01   5.86937755e-01   1.08564290e-01
       7.59950032e-01   3.19650224e-01   2.37551567e-01   6.62860967e-01
       6.40026772e-01   4.80187403e-01   3.61311822e-01   5.29182756e-01
       7.04279293e-01   9.29955056e-01   5.09685393e-01   5.67797753e-01
       9.74584274e-01   8.22089918e-01   7.54629427e-01   2.82405991e-01
       9.76841938e-01   8.37893569e-01   8.65254985e-01   5.67848992e-02
       3.97494296e-01   7.82460075e-01   4.77220525e-01   3.40543679e-01
       3.83805757e-01   6.86640297e-01   8.06482084e-01   6.45374588e-01
       5.17622120e-01   6.23354842e-01   3.63483898e-01   5.44387287e-01
       8.10711013e-01   6.74977093e-01   7.24839655e-01   7.38775893e-02
       5.17143127e-01   6.20001890e-01   3.40013233e-01   3.80092630e-01
       6.60648412e-01   6.24538884e-01   3.71772189e-01   6.02405326e-01
       2.16837286e-01   5.17861004e-01   6.25027030e-01   3.75189214e-01
       6.26324502e-01   3.84271515e-01   6.89900609e-01   8.29304267e-01
       8.05129872e-01   6.35909103e-01   4.51363724e-01   1.59546072e-01
       1.16822504e-01   8.17757526e-01   7.24302686e-01   7.01188052e-02
       4.90831638e-01   4.35821467e-01   5.07502686e-02   3.55251882e-01
       4.86763176e-01   4.07342235e-01   8.51395645e-01   9.59769514e-01
       7.18386602e-01   2.87062177e-02   2.00943525e-01   4.06604678e-01
       8.46232749e-01   9.23629246e-01   4.65404726e-01   2.57833087e-01
       8.04831609e-01   6.33821262e-01   4.36748835e-01   5.72418491e-02
       4.00692946e-01   8.04850621e-01   6.33954351e-01   4.37680459e-01
       6.37632113e-02   4.46342480e-01   1.24397365e-01   8.70781556e-01
       9.54708951e-02   6.68296267e-01   6.78073872e-01   7.46517106e-01
       2.25619743e-01   5.79338202e-01   5.53674167e-02   3.87571919e-01
       7.13003432e-01   9.91024026e-01   9.37168185e-01   5.60177296e-01
       9.21241072e-01   4.48687509e-01   1.40812563e-01   9.85687945e-01
       8.99815614e-01   2.98709297e-01   9.09650801e-02   6.36755562e-01
       4.57288938e-01   2.01022566e-01   4.07157963e-01   8.50105741e-01
       9.50740189e-01   6.55181323e-01   5.86269265e-01   1.03884856e-01
       7.27193995e-01   9.03579688e-02   6.32505783e-01   4.27540486e-01
       9.92783404e-01   9.49483831e-01   6.46386818e-01   5.24707729e-01
       6.72954104e-01   7.10678728e-01   9.74751101e-01   8.23257708e-01
       7.62803956e-01   3.39627691e-01   3.77393840e-01   6.41756880e-01
       4.92298160e-01   4.46087122e-01   1.22609858e-01   8.58269006e-01
       7.88304187e-03   5.51812947e-02   3.86269064e-01   7.03883453e-01
       9.27184172e-01   4.90289209e-01   4.32024462e-01   2.41712341e-02
       1.69198640e-01   1.84390485e-01   2.90733394e-01   3.51337600e-02
       2.45936321e-01   7.21554251e-01   5.08797609e-02   3.56158328e-01
       4.93108297e-01   4.51758078e-01   1.62306549e-01   1.36145843e-01
       9.53020906e-01   6.71146342e-01   6.98024399e-01   8.86170793e-01
       2.03195555e-01   4.22368885e-01   9.56582194e-01   6.96075361e-01
       8.72527528e-01   1.07692698e-01   7.53848888e-01   2.76942217e-01
       9.38595521e-01   5.70168650e-01   9.91180554e-01   9.38263881e-01
       5.67847172e-01   9.74930205e-01   8.24511438e-01   7.71580065e-01
       4.01060457e-01   8.07423202e-01   6.51962416e-01   5.63736913e-01
       9.46158394e-01   6.23108758e-01   3.61761306e-01   5.32329145e-01
       7.26304020e-01   8.41281414e-02   5.88896992e-01   1.22278943e-01
       8.55952601e-01   9.91668208e-01   9.41677458e-01   5.91742207e-01
       1.42195449e-01   9.95368143e-01   9.67577003e-01   7.73039024e-01
       4.11273168e-01   8.78912179e-01   1.52385258e-01   6.66968071e-02
       4.66877651e-01   2.68143559e-01   8.77004914e-01   1.39034401e-01
       9.73240808e-01   8.12685660e-01   6.88799622e-01   8.21597355e-01
       7.51181488e-01   2.58270415e-01   8.07892907e-01   6.55250352e-01
       5.86752463e-01   1.07267240e-01   7.50870679e-01   2.56094755e-01
       7.92663283e-01   5.48642984e-01   8.40500887e-01   8.83506207e-01
       1.84543452e-01   2.91804168e-01   4.26291742e-02   2.98404221e-01
       8.88295488e-02   6.21806843e-01   3.52647905e-01   4.68535337e-01
       2.79747362e-01   9.58231538e-01   7.07620766e-01   9.53345367e-01
       6.73417567e-01   7.13922972e-01   9.97460806e-01   9.82225642e-01
       8.75579498e-01   1.29056486e-01   9.03395403e-01   3.23767820e-01
       2.66374745e-01   8.64623215e-01   5.23625068e-02   3.66537549e-01
       5.65762844e-01   9.60339910e-01   7.22379372e-01   5.66556084e-02
       3.96589260e-01   7.76124823e-01   4.32873762e-01   3.01163329e-02
       2.10814332e-01   4.75700327e-01   3.29902290e-01   3.09316028e-01
       1.65212199e-01   1.56485396e-01   9.53977704e-02   6.67784395e-01
       6.74490764e-01   7.21435349e-01   5.00474472e-02   3.50332132e-01
       4.52324927e-01   1.66274489e-01   1.63921423e-01   1.47449960e-01
       3.21497202e-02   2.25048043e-01   5.75336305e-01   2.73541373e-02
       1.91478963e-01   3.40352740e-01   3.82469180e-01   6.77284261e-01
       7.40989828e-01   1.86928797e-01   3.08501583e-01   1.59511082e-01
       1.16577575e-01   8.16043027e-01   7.12301189e-01   9.86108321e-01
       9.02758252e-01   3.19307765e-01   2.35154354e-01   6.46080478e-01
       5.22563351e-01   6.57943458e-01   6.05604211e-01   2.39229477e-01
       6.74606341e-01   7.22244389e-01   5.57107267e-02   3.89975088e-01
       7.29825620e-01   1.08779338e-01   7.61455371e-01   3.30187597e-01
       3.11313180e-01   1.79192262e-01   2.54345838e-01   7.80420865e-01
       4.62946054e-01   2.40622381e-01   6.84356667e-01   7.90496672e-01
       5.33476709e-01   7.34336963e-01   1.40358745e-01   9.82511216e-01
       8.77578514e-01   1.43049601e-01   1.34720560e-03   9.43044084e-03
       6.60130875e-02   4.62091614e-01   2.34641302e-01   6.42489117e-01
       4.97423821e-01   4.81966747e-01   3.73767231e-01   6.16370620e-01
       3.14594342e-01   2.02160399e-01   4.15122793e-01   9.05859552e-01
       3.41016866e-01   3.87118064e-01   7.09826453e-01   9.68785170e-01
       7.81496190e-01   4.70473331e-01   2.93313321e-01   5.31932456e-02
       3.72352721e-01   6.06469046e-01   2.45283320e-01   7.16983244e-01
       1.88827112e-02   1.32178980e-01   9.25252861e-01   4.76770031e-01
       3.37390218e-01   3.61731528e-01   5.32120695e-01   7.24844870e-01
       7.39140907e-02   5.17398636e-01   6.21790455e-01   3.52533187e-01
       4.67732313e-01   2.74126192e-01   9.18883348e-01   4.32183440e-01
       2.52840812e-02   1.76988570e-01   2.38919994e-01   6.72439957e-01
       7.07079704e-01   9.49557927e-01   6.46905490e-01   5.28338434e-01
       6.98369040e-01   8.88583282e-01   2.20082977e-01   5.40580842e-01
       7.84065893e-01   4.88461252e-01   4.19228764e-01   9.34601347e-01
       5.42209432e-01   7.95466025e-01   5.68262176e-01   9.77835231e-01
       8.44846615e-01   9.13926309e-01   3.97484163e-01   7.82389140e-01
       4.76723984e-01   3.37067893e-01   3.59475252e-01   5.16326763e-01
       6.14287342e-01   3.00011393e-01   1.00079753e-01   7.00558269e-01
       9.03907887e-01   3.27355210e-01   2.91486472e-01   4.04053053e-02
       2.82837139e-01   9.79859975e-01   8.59019826e-01   1.31387846e-02
       9.19714936e-02   6.43800457e-01   5.06603198e-01   5.46222391e-01
       8.23556738e-01   7.64897170e-01   3.54280195e-01   4.79961364e-01]

We can test for uniformity by examining a histogram

```python
# the histogram of the data
n, bins, patches = plt.hist(x, 20, normed=1, ec='w')
plt.xlabel('x')
plt.ylabel('p(x)')
```

## Optimal Values for the LCG

1. $c$ is relatively prime to m
2. $b=a-1$ is a multiple of $p$ for every prime number $p$ dividing $m$
3. $b$ is a multiple of 4 if $m$ is a multiple of 4

Numerical Recipes suggests:

\begin{align*}
a &= 1664525 \newline
c &= 1013904223 \newline
m &= 2^{32}
\end{align*}

We can also test the overall statistics (but not correlations) by looking at the mean and variance over the uniform probability distribution $p(x) = 1$.

\begin{equation}
\langle x \rangle = \mu = \int_0^1 p(x) x dx
= \int_0^1 x dx
= \left. \frac{x^2}{2}\right \rvert_0^1
= \frac{1}{2}
\end{equation}

\begin{equation}
\sigma^2 = \langle(x-\mu)^2\rangle = \int_0^1 \left(x-\frac{1}{2}\right)^2 dx =  \int_0^1 \left(x^2 - x +\frac{1}{4}\right) dx
= \frac{1}{3} - \frac{1}{2} + \frac{1}{4} = \frac{1}{12}
\end{equation}

so $\sigma = \frac{1}{\sqrt{12}} \simeq 0.2886$

```python
a,c,m,seed = 1664525,1013904223,2**32,13523
x = lcg_rand(a,c,m,seed,10000)

print(np.average(x),np.std(x))
```

    0.50449334808 0.289062854736

We can also visually inspect for correlations by plotting $x_i vs. x_{i+1}$

```python
plt.figure(figsize=(5,5))
plt.plot(x[:-1],x[1:],'o', ms=3, mew=0)
plt.xlabel(r'$x_i$')
plt.ylabel(r'$x_{i+1}$')
```

## Tower Sampling

We can use the uniform distribution of (pseudo) random numbers in many ways, including sampling $N$ discrete events, each with their own probabilities $p_0,p_1,\ldots,p_{N-1}$.  Since something *must* happen, we know

\begin{equation}
\sum_{i=0}^{n-1} p_i = 1
\end{equation}

We can use our uniformly distributed random numbers $x\in \mathcal{U}_{[0,1]}$ to sample this discrete distribution by exploiting the fact that each event occupies a width $p_i$ in the probability interval. i.e. for a given random number $x$:

\begin{align*}
0 &\leftarrow 0 \le x < p_0 \newline
1 &\leftarrow p_0 \le x < p_0 + p_1 \newline
2 &\leftarrow p_0 + p_1 \le x < p_0 + p_1 + p_2 \newline
&\vdots \newline
N-1 &\leftarrow p_0 + \cdots + p_{N-2} \le x < 1 .
\end{align*}

Note that the relevant quantity here is the **cumulative probability**:

\begin{equation}
{P}_i = \sum_{k=0}^i p_k
\end{equation}

which for continuous distribution is:

\begin{equation}
{P}(x) = \int_{-\infty}^x p(x) dx
\end{equation}

In practice, we simply make a list of cumulative probabilites and figure out where to insert $x$.

```python
# Suppose we have 6 outcomes (eg. an unfair die) with the following propbabilites
p = [0.22181816, 0.16939565, 0.16688735, 0.06891783, 0.19622408, 0.17675693]

# generate the CDF
P = [np.sum(p[:i+1]) for i in range(len(p))]

plt.plot(P)
plt.xlabel('n')
plt.ylabel('P(n)')
plt.title('Cumulative Probability Distribution')
```

```python
P
```

    [0.22181815999999999,
     0.39121381,
     0.55810115999999999,
     0.62701898999999994,
     0.82324306999999997,
     1.0]

```python
# Generate N random numbers sampled according to the tower, searchsorted is *fast*
N = 1000000
events = np.searchsorted(P,np.random.random(N))
plt.plot(p,'o', mec='None')
plt.hist(events, bins=len(p), normed=True, range=(-0.5,len(p) - 0.5), ec='w')
plt.xlabel('x')
plt.ylabel('p(x)')
plt.xlim(-0.5,5.5)
plt.title('Tower Sampling')
```

## Sampling Continous Distributions

We can extend the *tower sampling* concept to any continuous probability distribution. Our starting point will be the uniform distribution which satisfies:

\begin{equation}
p(x) dx = \left \{
\begin{array}[rcl]
{}dx & ; & 0 \le x \le 1 \\
0 & ; & \text{otherwise}
\end{array}
\right.
\end{equation}

such that:

\begin{equation}
\int_{-\infty}^{\infty} p(x) dx = 1.
\end{equation}

Now, we want to sample some new random variables $y$ from some probability distribution $p(y)$.  This requires we identify a mapping $x\leftrightarrow y$ such that probability is conserved, i.e.

\begin{equation}
p(y) dy = p(x) dx \Rightarrow p(y) = \frac{dx}{dy}  p(x) .
\end{equation}

We can integrate both sides:

\begin{equation}
P(y) = \int_{-\infty}^y p(y') dy' = \int_{-\infty}^{y} \frac{dx}{dy'}  p(x) dy' = \int_0^y \frac{dx}{dy'} dy' = x(y)
\end{equation}

Therefore, if we can invert the CDF $P(y)$ we can get $y = P^{-1}(x)$ for a uniformly distributed $x$.

Let's see how this works for a few specific examples.

### Example 1
Generate a uniform random number $y$ on the domain $[a,b]$.

We have:
\begin{equation}
p(y) = \left \{
\begin{array}[rcl]
{} \frac{1}{b-a} & ; & a \le y \le b \\
0 & ; & \text{otherwise}
\end{array}
\right.
\end{equation}

So

\begin{equation}
x(y) =\int_a^y p(y') dy' = \int_a^y \frac{dy'}{b-a} = \left.\frac{y'}{b-a} \right \rvert_a^y = \frac{y-a}{b-a}
\end{equation}

so $y = (b-a)x + a$,  our well known result.

## Next Time:
What happens if we can't analytically invert $P(y)$?

```python

```


# Optimización media-varianza

## [Evaluación de medio término](http://cursos.iteso.mx/course/view.php?id=1480)

La **teoría de portafolios** es uno de los avances más importantes en las finanzas modernas e inversiones.
- Apareció por primera vez en un [artículo corto](https://www.math.ust.hk/~maykwok/courses/ma362/07F/markowitz_JF.pdf) llamado "Portfolio Selection" en la edición de Marzo de 1952 de "the Journal of Finance".
- Escrito por un desconocido estudiante de la Universidad de Chicago, llamado Harry Markowitz.
- Escrito corto (sólo 14 páginas), poco texto, fácil de entender, muchas gráficas y unas cuantas referencias.
- No se le prestó mucha atención hasta los 60s.

Finalmente, este trabajo se convirtió en una de las más grandes ideas en finanzas, y le dió a Markowitz el Premio Nobel casi 40 años después.
- Markowitz estaba incidentalmente interesado en los mercados de acciones e inversiones.
- Estaba más bien interesado en entender cómo las personas tomaban sus mejores decisiones cuando se enfrentaban con "trade-offs".
- Principio de conservación de la miseria. O, dirían los instructores de gimnasio: "no pain, no gain".
- Si queremos más de algo, tenemos que perder en algún otro lado.
- El estudio de este fenómeno era el que le atraía a Markowitz.

De manera que nadie se hace rico poniendo todo su dinero en la cuenta de ahorros. La única manera de esperar altos rendimientos es si se toma bastante riesgo. Sin embargo, riesgo significa también la posibilidad de perder, tanto como ganar.

Pero, ¿qué tanto riesgo es necesario?, y ¿hay alguna manera de minimizar el riesgo mientras se maximizan las ganancias?
- Markowitz básicamente cambió la manera en que los inversionistas pensamos acerca de esas preguntas.
- Alteró completamente la práctica de la administración de inversiones.
- Incluso el título de su artículo era innovador. Portafolio: una colección de activos en lugar de tener activos individuales.
- En ese tiempo, un portafolio se refería a una carpeta de piel.
- En el resto de este módulo, nos ocuparemos de la parte analítica de la teoría de portafolios, la cual puede ser resumida en dos frases:
 - No pain, no gain.
 - No ponga todo el blanquillo en una sola bolsa.

**Objetivos:**
- ¿Qué es la línea de asignación de capital?
- ¿Qué es el radio de Sharpe?
- ¿Cómo deberíamos asignar nuestro capital entre un activo riesgoso y un activo libre de riesgo?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Línea de asignación de capital

### 1.1. Motivación

El proceso de construcción de un portafolio tiene entonces los siguientes dos pasos:
1. Escoger un portafolio de activos riesgosos.
2. Decidir qué tanto de tu riqueza invertirás en el portafolio y qué tanto invertirás en activos libres de riesgo.

Al paso 2 lo llamamos **decisión de asignación de activos**.

Preguntas importantes:
1. ¿Qué es el portafolio óptimo de activos riesgosos?
 - ¿Cuál es el mejor portafolio de activos riesgosos?
 - Es un portafolio eficiente en media-varianza.
2. ¿Qué es la distribución óptima de activos?
 - ¿Cómo deberíamos distribuir nuestra riqueza entre el portafolo riesgoso óptimo y el activo libre de riesgo?
 - Concepto de **línea de asignación de capital**.
 - Concepto de **radio de Sharpe**.

Dos suposiciones importantes:
- Funciones de utilidad media-varianza.
- Inversionista averso al riesgo.

La idea sorprendente que saldrá de este análisis, es que cualquiera que sea la actitud del inversionista de cara al riesgo, el mejor portafolio de activos riesgosos es idéntico para todos los inversionistas.

Lo que nos importará a cada uno de nosotros en particular, es simplemente la desición óptima de asignación de activos.
___

### 1.2. Línea de asignación de capital

Sean:
- $r_s$ el rendimiento del activo riesgoso,
- $r_f$ el rendimiento libre de riesgo, y
- $w$ la fracción invertida en el activo riesgoso.

 Realizar deducción de la línea de asignación de capital en el tablero.

**Tres doritos después...**

#### Línea de asignación de capital (LAC):
$E[r_p]$ se relaciona con $\sigma_p$ de manera afín. Es decir, mediante la ecuación de una recta:

$$E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p.$$

- La pendiente de la LAC es el radio de Sharpe $\frac{E[r_s-r_f]}{\sigma_s}=\frac{E[r_s]-r_f}{\sigma_s}$,
- el cual nos dice qué tanto rendimiento obtenemos por unidad de riesgo asumido en la tenencia del activo (portafolio) riesgoso.

Ahora, la pregunta es, ¿dónde sobre esta línea queremos estar?
___

### 1.3. Resolviendo para la asignación óptima de capital

Recapitulando de la clase pasada, tenemos las curvas de indiferencia: **queremos estar en la curva de indiferencia más alta posible, que sea tangente a la LAC**.

 Ver en el tablero.

Analíticamente, el problema es

$$\max_{w} \quad E[U(r_p)]\equiv\max_{w} \quad E[r_p]-\frac{1}{2}\gamma\sigma_p^2,$$

donde los puntos $(\sigma_p,E[r_p])$ se restringen a estar en la LAC, esto es $E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p$ y $\sigma_p=w\sigma_s$. Entonces el problema anterior se puede escribir de la siguiente manera:

$$\max_{w} \quad r_f+wE[r_s-r_f]-\frac{1}{2}\gamma w^2\sigma_s^2.$$

 Encontrar la $w$ que maximiza la anterior expresión en el tablero.

**Tres doritos después...**

La solución es entonces:

$$w^\ast=\frac{E[r_s]-r_f}{\gamma\sigma_s^2}.$$

De manera intuitiva:
- $w^\ast\propto E[r_s-r_f]$: a más exceso de rendimiento que se obtenga del activo riesgoso, más querremos invertir en él.
- $w^\ast\propto \frac{1}{\gamma}$: mientras más averso al riesgo seas, menos querrás invertir en el activo riesgoso.
- $w^\ast\propto \frac{1}{\sigma_s^2}$: mientras más riesgoso sea el activo, menos querrás invertir en él.
___

## 2. Ejemplo de asignación óptima de capital: acciones y billetes de EU

Pongamos algunos números con algunos datos, para ilustrar la derivación que acabamos de hacer.

En este caso, consideraremos:
- **Portafolio riesgoso**: mercado de acciones de EU (representados en algún índice de mercado como el S&P500).
- **Activo libre de riesgo**: billetes del departamento de tesorería de EU (T-bills).

Tenemos los siguientes datos:

$$E[r_{US}]=11.9\%,\quad \sigma_{US}=19.15\%, \quad r_f=1\%.$$

Recordamos que podemos escribir la expresión de la LAC como:

\begin{align}
E[r_p]&=r_f+\left[\frac{E[r_{US}-r_f]}{\sigma_{US}}\right]\sigma_p\\
      &=0.01+\text{S.R.}\sigma_p,
\end{align}

donde $\text{S.R}=\frac{0.119-0.01}{0.1915}\approx0.569$ es el radio de Sharpe (¿qué es lo que es esto?).

Grafiquemos la LAC con estos datos reales:

```python
# Importamos librerías que vamos a utilizar
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
```

```python
# Datos
Ers = 0.119
ss = 0.1915
rf = 0.01
# Radio de Sharpe para este activo
RS = (Ers - rf) / ss
# Vector de volatilidades del portafolio (sugerido: 0% a 50%)
sp = np.linspace(0, 0.5)
# LAC
Erp = RS * sp + rf
```

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.axhline(y=0, color='k')
plt.axvline(x=0, color='k')
plt.plot(0, rf, 'og', ms=8, label='Activo libre de riesgo')
plt.plot(ss, Ers, 'ob', ms=8, label='Activo riesgoso')
plt.plot(sp, Erp, 'r', lw=2, label='LAC')
plt.grid()
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.legend(loc='best')
```

Bueno, y ¿en qué punto de esta línea querríamos estar?
- Pues ya vimos que depende de tus preferencias.
- En particular, de tu actitud de cara al riesgo, medido por tu coeficiente de aversión al riesgo.

Solución al problema de asignación óptima de capital:

$$\max_{w} \quad E[U(r_p)]$$

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}$$

Dado que ya tenemos datos, podemos intentar para varios coeficientes de aversión al riesgo:

```python
# importar pandas
import pandas as pd
```

```python
# Crear un DataFrame con los pesos, rendimiento
# esperado y volatilidad del portafolio óptimo
# entre los activos riesgoso y libre de riesgo
# cuyo índice sean los coeficientes de aversión
# al riesgo del 1 al 10 (enteros)
gamma = np.arange(1, 11, 1)
df = pd.DataFrame({'$\gamma$': gamma, '$w^*$': (Ers - rf) / (gamma * ss**2)})
```

```python
df
```

      $\gamma$
      $w^*$

      0
      1
      2.972275

      1
      2
      1.486137

      2
      3
      0.990758

      3
      4
      0.743069

      4
      5
      0.594455

      5
      6
      0.495379

      6
      7
      0.424611

      7
      8
      0.371534

      8
      9
      0.330253

      9
      10
      0.297227

¿Cómo se interpreta $w^\ast>1$?
- Cuando $0<w^\ast<1$, entonces $0<1-w^\ast<1$. Lo cual implica posiciones largas en el mercado de activos y en el activo libre de riesgo.
- Por el contrario, cuando $w^\ast>1$, tenemos $1-w^\ast<0$. Lo anterior implica una posición corta en el activo libre de riesgo (suponiendo que se puede) y una posición larga (de más del 100%) en el mercado de activos: apalancamiento.

# Anuncios parroquiales.

## 1. Segunda entrega Tarea 5 para el viernes.

## 2. Quiz la siguiente clase - Viernes 16 de Octubre 4pm.

## 3. La clase del viernes 16 de Octubre se anticipa para el jueves 15 de Octubre 4pm.

## 4. Pueden consultar sus calificaciones en el siguiente [enlace](https://docs.google.com/spreadsheets/d/10Ziyn-G6OjS4OlSc1lEbC0s-fBwkrA-rvN0MRggTDt4/edit?usp=sharing).

Created with Jupyter by Esteban Jiménez Rodríguez.


# Exercise 3
* Due: 27.1. at 14:00
* Max points: 8

## General rules
* **Submit your answers by using Moodle.** The link is https://moodle.jyu.fi/mod/assign/view.php?id=680507&forceview=1.
* Name your file as *lastname_ex3.xxx*. If you have multiple files, use *lastname_ex3_1.xxx, lastname_ex3_2.xxx* etc.
* **Remember to add comments to your answers and include also some testing part!**
* You will get feedback about your answers about 1-2 weeks from the due date
* The exercises will be given on the previous Wednesday's lecture

## Exercises

For all of the following problems, we study optimization problem
$$
\begin{align}
\min \qquad & x_1^2+x_2^2 + x_1+2x_2\\
\text{s.t.}\qquad &x_1,x_2\in\mathbb R
\end{align}
$$

1. **(3 points)** Implement a steepest descent algorithm, but do a golden section search to each new search direction. In other words, use line search to determine optimal step length. This means that you need to define a single-variable function to determine the step length. Set the length of the interval for doing line search as parameter of the method. Solve above problem using your method.
2. **(2 points)** Plot the steps of the steepest descent algorithm implemented at the class (with fixed step lengths) against the steps of a steepest descent algorithm that you implemented above. You can choose any starting point you wish, but not the optimum. Compare and analyze the results of both methods.
3. **(3 points)** Implement the Quasi-Newton method with DFP update as described e.g., at https://en.wikipedia.org/wiki/Davidon%E2%80%93Fletcher%E2%80%93Powell_formula. In other words, replace computation of the inverse Hessian in theSolve above problem using this method.