# Generating C code calling a third-party library
In this notebook we will look at how we can customize the `CCodePrinter` to generate C code which calls functions provided by a 3rd party library. For this example we will use [fastapprox](https://github.com/pmineiro/fastapprox) which is an open source [header only](https://en.wikipedia.org/wiki/Header-only) library containing fast approximate versions of transcendental functions. Will will look into using fastexp instead of exp.

```python
import sympy as sym
sym.init_printing()
```

```python
args = x, y, z = sym.symbols('x y z')
expr = x + sym.exp(2*y + x) + sym.exp(3*z)
expr
```

say that we need to evaluate this expression for vectors of values of x, y & z:

```python
import numpy as np
N = 4096
inp = np.linspace(10, 20, N), np.linspace(-20, -30, N), np.linspace(-9, -3, N)
```

```python
f = sym.lambdify(args, expr, modules=['numpy'])
f(*inp)
```

```python
%timeit f(*inp)
```

```python
from sympy.utilities.autowrap import ufuncify
```

```python
uf = ufuncify(args, expr)
```

```python
uf(*inp)
```

```python
%timeit uf(*inp)
```

The fact that lambdify is about as fast as our ufuncify tells us that we are bound by the time to evaluate expensive transcendental ``exp``. For those of you who know Cython you can see that in a Cython version as well:

```python
%load_ext cython
```

```python
from scipy2017codegen.templates import render_pyxbld
render_pyxbld('cy_f_mod', include_dirs=[np.get_include()])
```

```python
%%cython_pyximport cy_f_mod
from libc.math cimport exp
cimport numpy as cnp
import numpy as np
import cython

def cy_f(
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] x,
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] y,
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] z,
):
    cdef cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] out = np.empty(x.size)
    cdef double * _x = &x[0]
    cdef double * _y = &y[0]
    cdef double * _z = &z[0]
    cdef double * _out = &out[0]
    cdef int i
    if x.size != y.size or x.size != z.size:
        raise ValueError("Inconsistent length")
    for i in range(x.size):
        _out[i] = _x[i] + exp(2*_y[i] + _x[i]) + exp(3*_z[i])
    return out
```

```python
cy_f(*inp)
```

```python
%timeit cy_f(*inp)
```

So let's try to use ``fastexp`` from ``fastapprox``:

```python
import os
import scipy2017codegen
fastapprox_dir = os.path.join(os.path.dirname(scipy2017codegen.__file__), 'fastapprox')
print(''.join(open(os.path.join(fastapprox_dir, 'fastexp.h')).readlines()[62:67]))
```

```python
sym.ccode(expr, user_functions={'exp': 'fastexp'})
```

```python
render_pyxbld('cy_f_fastexp_mod', include_dirs=[np.get_include()])
```

```python
%%cython_pyximport cy_f_fastexp_mod
from libc.math cimport exp
cimport numpy as cnp
import numpy as np
import cython

cdef extern from "fastapprox/fastexp.h":
    float fastexp(float)

def cy_f_fastexp(
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] x,
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] y,
    cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] z,
):
    cdef cnp.ndarray[cnp.float64_t, ndim=1, mode='c'] out = np.empty(x.size)
    cdef double * _x = &x[0]
    cdef double * _y = &y[0]
    cdef double * _z = &z[0]
    cdef double * _out = &out[0]
    cdef int i
    if x.size != y.size or x.size != z.size:
        raise ValueError("Inconsistent length")
    for i in range(x.size):
        _out[i] = _x[i] + fastexp(2*_y[i] + _x[i]) + fastexp(3*_z[i])
    return out
```

```python
%timeit cy_f_fastexp
```

So that's a 10x speedup, of course at the cost of lost precision, but we are already assuming that was acceptable:

```python
cy_f_fastexp(*inp) - f(*inp)
```


```python
%matplotlib inline
```

Neural Transfer with PyTorch
============================
**Author**: `Alexis Jacq <https://alexis-jacq.github.io>`_

Introduction
------------

Welcome! This tutorial explains how to impletment the
`Neural-Style <https://arxiv.org/abs/1508.06576>`__ algorithm developed
by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge.

Neural what?
~~~~~~~~~~~~

The Neural-Style, or Neural-Transfer, is an algorithm that takes as
input a content-image (e.g. a tortle), a style-image (e.g. artistic
waves) and return the content of the content-image as if it was
'painted' using the artistic style of the style-image:

.. figure:: /_static/img/neural-style/neuralstyle.png
   :alt: content1

How does it work?
~~~~~~~~~~~~~~~~~

The principle is simple: we define two distances, one for the content
($D_C$) and one for the style ($D_S$). $D_C$ measures
how different the content is between two images, while $D_S$
measures how different the style is between two images. Then, we take a
third image, the input, (e.g. a with noise), and we transform it in
order to both minimize its content-distance with the content-image and
its style-distance with the style-image.

OK. How does it work?
^^^^^^^^^^^^^^^^^^^^^

Well, going further requires some mathematics. Let $C_{nn}$ be a
pre-trained deep convolutional neural network and $X$ be any
image. $C_{nn}(X)$ is the network fed by $X$ (containing
feature maps at all layers). Let $F_{XL} \in C_{nn}(X)$ be the
feature maps at depth layer $L$, all vectorized and concatenated
in one single vector. We simply define the content of $X$ at layer
$L$ by $F_{XL}$. Then, if $Y$ is another image of same
the size than $X$, we define the distance of content at layer
$L$ as follow:

\begin{align}D_C^L(X,Y) = \|F_{XL} - F_{YL}\|^2 = \sum_i (F_{XL}(i) - F_{YL}(i))^2\end{align}

Where $F_{XL}(i)$ is the $i^{th}$ element of $F_{XL}$.
The style is a bit less trivial to define. Let $F_{XL}^k$ with
$k \leq K$ be the vectorized $k^{th}$ of the $K$
feature maps at layer $L$. The style $G_{XL}$ of $X$
at layer $L$ is defined by the Gram produce of all vectorized
feature maps $F_{XL}^k$ with $k \leq K$. In other words,
$G_{XL}$ is a $K$\ x\ $K$ matrix and the element
$G_{XL}(k,l)$ at the $k^{th}$ line and $l^{th}$ column
of $G_{XL}$ is the vectorial produce between $F_{XL}^k$ and
$F_{XL}^l$ :

\begin{align}G_{XL}(k,l) = \langle F_{XL}^k, F_{XL}^l\rangle = \sum_i F_{XL}^k(i) . F_{XL}^l(i)\end{align}

Where $F_{XL}^k(i)$ is the $i^{th}$ element of
$F_{XL}^k$. We can see $G_{XL}(k,l)$ as a measure of the
correlation between feature maps $k$ and $l$. In that way,
$G_{XL}$ represents the correlation matrix of feature maps of
$X$ at layer $L$. Note that the size of $G_{XL}$ only
depends on the number of feature maps, not on the size of $X$.
Then, if $Y$ is another image *of any size*, we define the
distance of style at layer $L$ as follow:

\begin{align}D_S^L(X,Y) = \|G_{XL} - G_{YL}\|^2 = \sum_{k,l} (G_{XL}(k,l) - G_{YL}(k,l))^2\end{align}

In order to minimize in one shot $D_C(X,C)$ between a variable
image $X$ and target content-image $C$ and $D_S(X,S)$
between $X$ and target style-image $S$, both computed at
several layers , we compute and sum the gradients (derivative with
respect to $X$) of each distance at each wanted layer:

\begin{align}\nabla_{	extit{total}}(X,S,C) = \sum_{L_C} w_{CL_C}.\nabla_{	extit{content}}^{L_C}(X,C) + \sum_{L_S} w_{SL_S}.\nabla_{	extit{style}}^{L_S}(X,S)\end{align}

Where $L_C$ and $L_S$ are respectivement the wanted layers
(arbitrary stated) of content and style and $w_{CL_C}$ and
$w_{SL_S}$ the weights (arbitrary stated) associated with the
style or the content at each wanted layer. Then, we run a gradient
descent over $X$:

\begin{align}X \leftarrow X - \alpha \nabla_{	extit{total}}(X,S,C)\end{align}

Ok. That's enough with maths. If you want to go deeper (how to compute
the gradients) **we encourage you to read the original paper** by Leon
A. Gatys and AL, where everything is much better and much clearer
explained.

For our implementation in PyTorch, we already have everything
we need: indeed, with PyTorch, all the gradients are automatically and
dynamically computed for you (while you use functions from the library).
This is why the implementation of this algorithm becomes very
comfortable with PyTorch.

PyTorch implementation
----------------------

If you are not sure to understand all the mathematics above, you will
probably get it by implementing it. If you are discovering PyTorch, we
recommend you to first read this :doc:`Introduction to
PyTorch </beginner/deep_learning_60min_blitz>`.

Packages
~~~~~~~~

We will have recourse to the following packages:

-  ``torch``, ``torch.nn``, ``numpy`` (indispensables packages for
   neural networks with PyTorch)
-  ``torch.autograd.Variable`` (dynamic computation of the gradient wrt
   a variable)
-  ``torch.optim`` (efficient gradient descents)
-  ``PIL``, ``PIL.Image``, ``matplotlib.pyplot`` (load and display
   images)
-  ``torchvision.transforms`` (treat PIL images and transform into torch
   tensors)
-  ``torchvision.models`` (train or load pre-trained models)
-  ``copy`` (to deep copy the models; system package)

```python
from __future__ import print_function

import torch
import torch.nn as nn
from torch.autograd import Variable
import torch.optim as optim

from PIL import Image
import matplotlib.pyplot as plt

import torchvision.transforms as transforms
import torchvision.models as models

import copy
```

Cuda
~~~~

If you have a GPU on your computer, it is preferable to run the
algorithm on it, especially if you want to try larger networks (like
VGG). For this, we have ``torch.cuda.is_available()`` that returns
``True`` if you computer has an available GPU. Then, we can use method
``.cuda()`` that moves allocated proccesses associated with a module
from the CPU to the GPU. When we want to move back this module to the
CPU (e.g. to use numpy), we use the ``.cpu()`` method. Finally,
``.type(dtype)`` will be use to convert a ``torch.FloatTensor`` into
``torch.cuda.FloatTensor`` to feed GPU processes.

```python
use_cuda = torch.cuda.is_available()
dtype = torch.cuda.FloatTensor if use_cuda else torch.FloatTensor
```

Load images
~~~~~~~~~~~

In order to simplify the implementation, let's start by importing a
style and a content image of the same dimentions. We then scale them to
the desired output image size (128 or 512 in the example, depending on gpu
availablity) and transform them into torch tensors, ready to feed
a neural network:

.. Note::
    Here are links to download the images required to run the tutorial:
    `picasso.jpg <http://pytorch.org/tutorials/_static/img/neural-style/picasso.jpg>`__ and
    `dancing.jpg <http://pytorch.org/tutorials/_static/img/neural-style/dancing.jpg>`__.
    Download these two images and add them to a directory
    with name ``images``

```python
# desired size of the output image
imsize = 512 if use_cuda else 128  # use small size if no gpu

loader = transforms.Compose([
    transforms.Scale(imsize),  # scale imported image
    transforms.ToTensor()])  # transform it into a torch tensor

def image_loader(image_name):
    image = Image.open(image_name)
    image = Variable(loader(image))
    # fake batch dimension required to fit network's input dimensions
    image = image.unsqueeze(0)
    return image

style_img = image_loader("images/picasso.jpg").type(dtype)
content_img = image_loader("images/dancing.jpg").type(dtype)

assert style_img.size() == content_img.size(), \
    "we need to import style and content images of the same size"
```

Imported PIL images has values between 0 and 255. Transformed into torch
tensors, their values are between 0 and 1. This is an important detail:
neural networks from torch library are trained with 0-1 tensor image. If
you try to feed the networks with 0-255 tensor images the activated
feature maps will have no sense. This is not the case with pre-trained
networks from the Caffe library: they are trained with 0-255 tensor
images.

Display images
~~~~~~~~~~~~~~

We will use ``plt.imshow`` to display images. So we need to first
reconvert them into PIL images:

```python
unloader = transforms.ToPILImage()  # reconvert into PIL image

plt.ion()

def imshow(tensor, title=None):
    image = tensor.clone().cpu()  # we clone the tensor to not do changes on it
    image = image.view(3, imsize, imsize)  # remove the fake batch dimension
    image = unloader(image)
    plt.imshow(image)
    if title is not None:
        plt.title(title)
    plt.pause(0.001) # pause a bit so that plots are updated

plt.figure()
imshow(style_img.data, title='Style Image')

plt.figure()
imshow(content_img.data, title='Content Image')
```

Content loss
~~~~~~~~~~~~

The content loss is a function that takes as input the feature maps
$F_{XL}$ at a layer $L$ in a network fed by $X$ and
return the weigthed content distance $w_{CL}.D_C^L(X,C)$ between
this image and the content image. Hence, the weight $w_{CL}$ and
the target content $F_{CL}$ are parameters of the function. We
implement this function as a torch module with a constructor that takes
these parameters as input. The distance $\|F_{XL} - F_{YL}\|^2$ is
the Mean Square Error between the two sets of feature maps, that can be
computed using a criterion ``nn.MSELoss`` stated as a third parameter.

We will add our content losses at each desired layer as additive modules
of the neural network. That way, each time we will feed the network with
an input image $X$, all the content losses will be computed at the
desired layers and, thanks to autograd, all the gradients will be
computed. For that, we just need to make the ``forward`` method of our
module returning the input: the module becomes a ''transparent layer''
of the neural network. The computed loss is saved as a parameter of the
module.

Finally, we define a fake ``backward`` method, that just call the
backward method of ``nn.MSELoss`` in order to reconstruct the gradient.
This method returns the computed loss: this will be useful when running
the gradient descent in order to display the evolution of style and
content losses.

```python
class ContentLoss(nn.Module):

    def __init__(self, target, weight):
        super(ContentLoss, self).__init__()
        # we 'detach' the target content from the tree used
        self.target = target.detach() * weight
        # to dynamically compute the gradient: this is a stated value,
        # not a variable. Otherwise the forward method of the criterion
        # will throw an error.
        self.weight = weight
        self.criterion = nn.MSELoss()

    def forward(self, input):
        self.loss = self.criterion(input * self.weight, self.target)
        self.output = input
        return self.output

    def backward(self, retain_graph=True):
        self.loss.backward(retain_graph=retain_graph)
        return self.loss
```

.. Note::
   **Important detail**: this module, although it is named ``ContentLoss``,
   is not a true PyTorch Loss function. If you want to define your content
   loss as a PyTorch Loss, you have to create a PyTorch autograd Function
   and to recompute/implement the gradient by the hand in the ``backward``
   method.

Style loss
~~~~~~~~~~

For the style loss, we need first to define a module that compute the
gram produce $G_{XL}$ given the feature maps $F_{XL}$ of the
neural network fed by $X$, at layer $L$. Let
$\hat{F}_{XL}$ be the re-shaped version of $F_{XL}$ into a
$K$\ x\ $N$ matrix, where $K$ is the number of feature
maps at layer $L$ and $N$ the lenght of any vectorized
feature map $F_{XL}^k$. The $k^{th}$ line of
$\hat{F}_{XL}$ is $F_{XL}^k$. We let you check that
$\hat{F}_{XL} \cdot \hat{F}_{XL}^T = G_{XL}$. Given that, it
becomes easy to implement our module:

```python
class GramMatrix(nn.Module):

    def forward(self, input):
        a, b, c, d = input.size()  # a=batch size(=1)
        # b=number of feature maps
        # (c,d)=dimensions of a f. map (N=c*d)

        features = input.view(a * b, c * d)  # resise F_XL into \hat F_XL

        G = torch.mm(features, features.t())  # compute the gram product

        # we 'normalize' the values of the gram matrix
        # by dividing by the number of element in each feature maps.
        return G.div(a * b * c * d)
```

The longer is the feature maps dimension $N$, the bigger are the
values of the gram matrix. Therefore, if we don't normalize by $N$,
the loss computed at the first layers (before pooling layers) will have
much more importance during the gradient descent. We dont want that,
since the most interesting style features are in the deepest layers!

Then, the style loss module is implemented exactly the same way than the
content loss module, but we have to add the ``gramMatrix`` as a
parameter:

```python
class StyleLoss(nn.Module):

    def __init__(self, target, weight):
        super(StyleLoss, self).__init__()
        self.target = target.detach() * weight
        self.weight = weight
        self.gram = GramMatrix()
        self.criterion = nn.MSELoss()

    def forward(self, input):
        self.output = input.clone()
        self.G = self.gram(input)
        self.G.mul_(self.weight)
        self.loss = self.criterion(self.G, self.target)
        return self.output

    def backward(self, retain_graph=True):
        self.loss.backward(retain_graph=retain_graph)
        return self.loss
```

Load the neural network
~~~~~~~~~~~~~~~~~~~~~~~

Now, we have to import a pre-trained neural network. As in the paper, we
are going to use a pretrained VGG network with 19 layers (VGG19).

PyTorch's implementation of VGG is a module divided in two child
``Sequential`` modules: ``features`` (containing convolution and pooling
layers) and ``classifier`` (containing fully connected layers). We are
just interested by ``features``:

```python
cnn = models.vgg19(pretrained=True).features

# move it to the GPU if possible:
if use_cuda:
    cnn = cnn.cuda()
```

A ``Sequential`` module contains an ordered list of child modules. For
instance, ``vgg19.features`` contains a sequence (Conv2d, ReLU,
Maxpool2d, Conv2d, ReLU...) aligned in the right order of depth. As we
said in *Content loss* section, we wand to add our style and content
loss modules as additive 'transparent' layers in our network, at desired
depths. For that, we construct a new ``Sequential`` module, in wich we
are going to add modules from ``vgg19`` and our loss modules in the
right order:

```python
# desired depth layers to compute style/content losses :
content_layers_default = ['conv_4']
style_layers_default = ['conv_1', 'conv_2', 'conv_3', 'conv_4', 'conv_5']

def get_style_model_and_losses(cnn, style_img, content_img,
                               style_weight=1000, content_weight=1,
                               content_layers=content_layers_default,
                               style_layers=style_layers_default):
    cnn = copy.deepcopy(cnn)

    # just in order to have an iterable access to or list of content/syle
    # losses
    content_losses = []
    style_losses = []

    model = nn.Sequential()  # the new Sequential module network
    gram = GramMatrix()  # we need a gram module in order to compute style targets

    # move these modules to the GPU if possible:
    if use_cuda:
        model = model.cuda()
        gram = gram.cuda()

    i = 1
    for layer in list(cnn):
        if isinstance(layer, nn.Conv2d):
            name = "conv_" + str(i)
            model.add_module(name, layer)

            if name in content_layers:
                # add content loss:
                target = model(content_img).clone()
                content_loss = ContentLoss(target, content_weight)
                model.add_module("content_loss_" + str(i), content_loss)
                content_losses.append(content_loss)

            if name in style_layers:
                # add style loss:
                target_feature = model(style_img).clone()
                target_feature_gram = gram(target_feature)
                style_loss = StyleLoss(target_feature_gram, style_weight)
                model.add_module("style_loss_" + str(i), style_loss)
                style_losses.append(style_loss)

        if isinstance(layer, nn.ReLU):
            name = "relu_" + str(i)
            model.add_module(name, layer)

            if name in content_layers:
                # add content loss:
                target = model(content_img).clone()
                content_loss = ContentLoss(target, content_weight)
                model.add_module("content_loss_" + str(i), content_loss)
                content_losses.append(content_loss)

            if name in style_layers:
                # add style loss:
                target_feature = model(style_img).clone()
                target_feature_gram = gram(target_feature)
                style_loss = StyleLoss(target_feature_gram, style_weight)
                model.add_module("style_loss_" + str(i), style_loss)
                style_losses.append(style_loss)

            i += 1

        if isinstance(layer, nn.MaxPool2d):
            name = "pool_" + str(i)
            model.add_module(name, layer)  # ***

    return model, style_losses, content_losses
```

.. Note::
   In the paper they recommend to change max pooling layers into
   average pooling. With AlexNet, that is a small network compared to VGG19
   used in the paper, we are not going to see any difference of quality in
   the result. However, you can use these lines instead if you want to do
   this substitution:

   ::

       # avgpool = nn.AvgPool2d(kernel_size=layer.kernel_size,
       #                         stride=layer.stride, padding = layer.padding)
       # model.add_module(name,avgpool)

Input image
~~~~~~~~~~~

Again, in order to simplify the code, we take an image of the same
dimensions than content and style images. This image can be a white
noise, or it can also be a copy of the content-image.

```python
input_img = content_img.clone()
# if you want to use a white noise instead uncomment the below line:
# input_img = Variable(torch.randn(content_img.data.size())).type(dtype)

# add the original input image to the figure:
plt.figure()
imshow(input_img.data, title='Input Image')
```

Gradient descent
~~~~~~~~~~~~~~~~

As Leon Gatys, the author of the algorithm, suggested
`here <https://discuss.pytorch.org/t/pytorch-tutorial-for-neural-transfert-of-artistic-style/336/20?u=alexis-jacq>`__,
we will use L-BFGS algorithm to run our gradient descent. Unlike
training a network, we want to train the input image in order to
minimise the content/style losses. We would like to simply create a
PyTorch  L-BFGS optimizer, passing our image as the variable to optimize.
But ``optim.LBFGS`` takes as first argument a list of PyTorch
``Variable`` that require gradient. Our input image is a ``Variable``
but is not a leaf of the tree that requires computation of gradients. In
order to show that this variable requires a gradient, a possibility is
to construct a ``Parameter`` object from the input image. Then, we just
give a list containing this ``Parameter`` to the optimizer's
constructor:

```python
def get_input_param_optimizer(input_img):
    # this line to show that input is a parameter that requires a gradient
    input_param = nn.Parameter(input_img.data)
    optimizer = optim.LBFGS([input_param])
    return input_param, optimizer
```

**Last step**: the loop of gradient descent. At each step, we must feed
the network with the updated input in order to compute the new losses,
we must run the ``backward`` methods of each loss to dynamically compute
their gradients and perform the step of gradient descent. The optimizer
requires as argument a "closure": a function that reevaluates the model
and returns the loss.

However, there's a small catch. The optimized image may take its values
between $-\infty$ and $+\infty$ instead of staying between 0
and 1. In other words, the image might be well optimized and have absurd
values. In fact, we must perform an optimization under constraints in
order to keep having right vaues into our input image. There is a simple
solution: at each step, to correct the image to maintain its values into
the 0-1 interval.

```python
def run_style_transfer(cnn, content_img, style_img, input_img, num_steps=300,
                       style_weight=1000, content_weight=1):
    """Run the style transfer."""
    print('Building the style transfer model..')
    model, style_losses, content_losses = get_style_model_and_losses(cnn,
        style_img, content_img, style_weight, content_weight)
    input_param, optimizer = get_input_param_optimizer(input_img)

    print('Optimizing..')
    run = [0]
    while run[0] <= num_steps:

        def closure():
            # correct the values of updated input image
            input_param.data.clamp_(0, 1)

            optimizer.zero_grad()
            model(input_param)
            style_score = 0
            content_score = 0

            for sl in style_losses:
                style_score += sl.backward()
            for cl in content_losses:
                content_score += cl.backward()

            run[0] += 1
            if run[0] % 50 == 0:
                print("run {}:".format(run))
                print('Style Loss : {:4f} Content Loss: {:4f}'.format(
                    style_score.data[0], content_score.data[0]))
                print()

            return style_score + content_score

        optimizer.step(closure)

    # a last correction...
    input_param.data.clamp_(0, 1)

    return input_param.data
```

Finally, run the algorithm

```python
output = run_style_transfer(cnn, content_img, style_img, input_img)

plt.figure()
imshow(output, title='Output Image')

# sphinx_gallery_thumbnail_number = 4
plt.ioff()
plt.show()
```


# Mass Spring System:

In this example, we are going to study a simple mass-spring system. This example serves to demonstrate the visualization handling abilities of mechanics-visualization module which is still in development process

## Dependencies:

For this system, following python modules are required:

1) sympy=0.7.2

2) numpy=1.6.2

```
%load_ext autoreload
%autoreload 2
from sympy.physics.mechanics import *
from sympy import symbols,cos,Eq,sqrt,sin
import viz                       #This module is included with the notebook
from IPython.display import *
```

That is all we are going to need to carry out this visualization.

Now, we will create symbols for coordinates, speed, force etc of the mass(bob).

```
q = dynamicsymbols('q')                  # Generalized coordinate
u = dynamicsymbols('u')                  # Generalized speed
f = dynamicsymbols('f')                  # Net Force applied to the box
m = symbols('m')                         # Mass of body
g, k, t, L = symbols('g k t L')               # Gravity,spring constant, time and length of unstretched spring

```

Now we create an inertial reference frame, assign an origin, and set origin velocity to zero.

```
I = ReferenceFrame('I')            # Inertial reference frame
O = Point('O')                     # Origin point
O.set_vel(I, 0)                    # Origin's velocity is zero
```

Now we define the bob as a Particle ..

```
P = Point('P')                      # Center Of Mass of the box
P.set_pos(O, q * I.y)               # Set the position of P
P.set_vel(I, u * I.y )              # Set the velocity of P
bob = Particle('bob',P,m)

```

We assign forces acting on the Particle.
Coordinate axes are defined to be as : x-axis: running from left to right, y-axis: running from top to bottom, z-axis: running from behind the plane to above the plane.

Following the above notion, Gravity is taken to be in (+I.y) direction, and force due to spring(F=Kx), in -I.y direction ..

```
f = m*g* I.y - k*(q)*I.y    #Net force on bob.
force = [(P,f)]
```

Setting up equation of motion for the body:

```
diff_eq = [q.diff(t) - u]
```

Now, we call the Kane's Method to generate the equation of motion of the bob

```
kane = KanesMethod(I, q_ind=[q], u_ind=[u], kd_eqs=diff_eq) # Initialize the object
fr, frstar = kane.kanes_equations(force, [bob])
```

The Differential equation of the motion of bob is given by the expression: fr + frstar = 0, i.e. fr = frstar

```
fr, frstar
```

    ([-g*m + k*q(t)], [m*Derivative(u(t), t)])

Now, We need to solve this differential equation, for this we would be utilizing code generator sub-module in sympy mechanics, which is under development, and is not merged in the package.

First packing the parameters in a list.

```
parameters = [m, k, g, L]     #Mass, stiffness, gravity
parameter_vals = [1, 1, 9.8, 1]
```

Now we import the code generator module.

```
import code_gen
right_hand_side = code_gen.numeric_right_hand_side(kane, parameters)

```

The numeric_right_hand_side method takes the Kane's object and parameters in symbolic form, and returns the differential equation in a numeric form, which can be integrated using odeint() method in SciPy.

Now we integrate right_hand_side using odeint

```
from scipy.integrate import odeint
x0 = [1,0]        #Initial conditions, q=1,u=0.
t = [i*0.1 for i in range(0,10)]    #Taking 10 time intervals of 0.1 sec
numeric_vals = odeint(right_hand_side, x0, t, args=(parameter_vals,))
numeric_vals
```

    array([[ 1.        ,  0.        ],
           [ 1.04396334,  0.87853405],
           [ 1.17541411,  1.7482901 ],
           [ 1.39303891,  2.6005778 ],
           [ 1.69466326,  3.42688139],
           [ 2.07727345,  4.21894471],
           [ 2.53704657,  4.96885373],
           [ 3.06938873,  5.66911561],
           [ 3.66898093,  6.31273356],
           [ 4.32983223,  6.89327675]])

The variable contains the positions and velocities, we need only positions for this 1D simulation, so we separate them, and pack them alongwith time.

```
Q = []
for entries in numeric_vals:
    Q.append(entries[0])

simulation_coords = [Q,t]     #Packing positions and time in a single array
```

## Simulation:

For simulation and visualization, we are going to utilize, the Javascript rendering capabilities of IPython notebooks to render WebGL's.
We are using Three.js Javascript library for 3D visualizations.

```
viz.visualize(simulation_coords)
```

    <IPython.core.display.Javascript at 0x45dcd90>


# Spectral Estimation of Random Signals

*This jupyter/Python notebook is part of a [collection of notebooks](../index.ipynb) in the masters module [Digital Signal Processing](http://www.int.uni-rostock.de/Digitale-Signalverarbeitung.48.0.html), Comunications Engineering, Universität Rostock. Please direct questions and suggestions to <mailto:Sascha.Spors@uni-rostock.de>.*

## Parametric Methods

### Motivation

Non-parametric methods for the estimation of the power spectral density (PSD), like the [periodogram](periodogram.ipynb) or [Welch's method](welch_method.ipynb), don't rely on a-priori information about the process generating the random signal. Often some a-priori information is available that can be used to formulate a parametric model of the random process. The goal is then to estimate these parameters in order to characterize the random signal. Such techniques are known as *[parametric methods](https://en.wikipedia.org/wiki/Spectral_density_estimation#Parametric_estimation)* or *model-based methods*. The incorporation of a-priori knowledge can improve the estimation of the PSD significantly, as long as the underlying model is a valid description of the random process. The parametric model of the random process can also be used to generate random signals with a desired PSD.

### Process Models

For the remainder we assume weakly stationary real-valued random processes. For many applications the process can be modeled by a linear-time invariant (LTI) system

where $n[k]$ is [white noise](../random_signals/white_noise.ipynb) and $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ denotes the transfer function of the system. In general, the random signal $x[k]$ will be correlated as a result of the processing of the uncorrelated input signal $n[k]$ by the system $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. Due to the white noise assumption $\Phi_{nn}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = N_0$, the PSD of the random process is given as

\begin{equation}
\Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = N_0 \cdot | H(\mathrm{e}^{\,\mathrm{j}\,\Omega}) |^2
\end{equation}

Parametric methods model the system $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ by a limited number of parameters. These parameters are then estimated from $x[k]$, providing an estimate $\hat{H}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ of the transfer function. This estimate is then used to calculate the desired estimate $\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ of the PSD.

#### Autoregressive model

The [autoregressive](https://en.wikipedia.org/wiki/Autoregressive_model) (AR) model assumes a recursive system with a direct path. Its output relation is given as

\begin{equation}
x[k] = \sum_{n=1}^{N} a_n \cdot x[k-n] + n[k]
\end{equation}

where $a_n$ denote the coefficients of the recursive path and $N$ the order of the model. Its system function $H(z)$ is derived by $z$-transformation of the output relation

\begin{equation}
H(z) = \frac{1}{1 - \sum_{n=1}^{N} a_n z^{-n}}
\end{equation}

Hence, the AR model is a pole-only model of the system.

#### Moving average model

The [moving average](https://en.wikipedia.org/wiki/Moving-average_model) (MA) model assumes a non-recursive system. The output relation is given as

\begin{equation}
x[k] = \sum_{m=0}^{M-1} b_m \cdot n[k-m] = h[k] * n[k]
\end{equation}

with the impulse response of the system $h[k] = [ b_0, b_1, \dots, b_{M-1} ]$. The MA model is a finite impulse response (FIR) model of the random process. Its system function is given as

\begin{equation}
H(z) = \mathcal{Z} \{ h[k] \} = \sum_{m=0}^{M-1} b_m \; z^{-m}
\end{equation}

#### Autoregressive moving average model

The [autoregressive moving average](https://en.wikipedia.org/wiki/Autoregressive%E2%80%93moving-average_model) (ARMA) model is a combination of the AR and MA model. It constitutes a general linear process model. Its output relation is given as

\begin{equation}
x[k] = \sum_{n=1}^{N} a_n \cdot x[k-n] + \sum_{m=0}^{M-1} b_m \cdot n[k-m]
\end{equation}

Its system function reads

\begin{equation}
H(z) = \frac{\sum_{m=0}^{M-1} b_m \; z^{-m}}{1 - \sum_{n=1}^{N} a_n z^{-n}}
\end{equation}

### Parametric Spectral Estimation

The models above describe the synthesis of the samples $x[k]$ from the white noise $n[k]$. For spectral estimation only the random signal $x[k]$ is known and we are aiming at estimating the parameters of the model. This can be achieved by determining an analyzing system $G(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ such to decorrelate the signal $x[k]$

where $e[k]$ should be white noise. Due to its desired operation, the filter $G(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ is also denoted as *whitening filter*. The optimal filter $G(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ is given by the inverse system $\frac{1}{H(\mathrm{e}^{\,\mathrm{j}\,\Omega})}$. However, $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ is in general not known. But this nevertheless implies that our linear process model of $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ also applies to $G(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. Various techniques have been developed to estimate the parameters of the filter $G(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ such that $e[k]$ becomes decorrelated. For instance, by expressing the auto-correlation function (ACF) $\varphi_{xx}[\kappa]$ in terms of the model parameters and solving with respect to these. The underlying set of equations are known as [Yule-Walker equations](https://en.wikipedia.org/wiki/Autoregressive_model#Yule-Walker_equations).

Once the model parameters have been estimated, these can be used to calculate an estimate $\hat{G}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ of the analysis system. The desired estimate of the PSD is then given as

\begin{equation}
\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = \frac{\Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega})}{|\hat{G}(\mathrm{e}^{\,\mathrm{j}\,\Omega})|^2}
\end{equation}

where if $e[k]$ is white noise, $\Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = N_0$.

### Example

In the follwing example $n[k]$ is drawn from normal distributed white noise with $N_0 = 1$. The Yule-Walker equations are used to estimate the parameters of an AR model of $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. The implementation provided by `statsmodels.api.regression.yule_walker` returns the estimated AR coefficients of the system $H(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. These parameters are then used to numerically evaluate the estimated transfer function, resulting in $\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = 1 \cdot | \hat{H}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) |^2 $.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm
import scipy.signal as sig

K = 4096  # length of random signal
N = 3  # order of AR model
a = np.array((1, -1, .5))  # coefficients of AR model

# generate random signal n[k]
n = np.random.normal(size=K)

# AR model for random signal x[k]
x = np.zeros(K)
for k in np.arange(3, K):
    x[k] = a[0]*x[k-1] + a[1]*x[k-2] + a[2]*x[k-3] + n[k]

# estimate AR parameters by Yule-Walker method
rho, sigma = sm.regression.yule_walker(x, order=N, method='mle')

# compute true and estimated transfer function
Om, H = sig.freqz(1, np.insert(-a, 0, 1))
Om, He = sig.freqz(1, np.insert(-rho, 0, 1))
# compute PSD by Welch method
Om2, Pxx = sig.welch(x, return_onesided=True)

# plot PSDs
plt.figure(figsize=(10,5))
plt.plot(Om, np.abs(H)**2, label=r'$\Phi_{xx}(e^{j\Omega})$')
plt.plot(Om2*2*np.pi, .5*np.abs(Pxx), 'k-', alpha=.5 , label=r'$\hat{\Phi}_{xx}(e^{j\Omega})$ (Welch)')
plt.plot(Om, np.abs(He)**2, label=r'$\hat{\Phi}_{xx}(e^{j\Omega})$ (parametric)')

plt.xlabel(r'$\Omega$')
plt.axis([0, np.pi, 0, 25])
plt.legend();
```

**Exercise**

* Change the order `N` of the AR model used for estimation by the Yule-Walker equations. What happens if the order is smaller or higher than the order of the true system? Why?

* Change the number of samples `K`. Is the estimator consistent?

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


# Modelo del rendimiento de una cuenta de ahorro

> **¿Tiene el dinero el mismo valor a lo largo del tiempo?** La respuesta es *no*. Todos lo hemos vivido.

> Dos situaciones básicas:
1. Inflación: ¿Cuánto dinero necesitabas para comprar unas papas y un refresco hace 10 años? ¿Cuánto necesitas hoy?
2. Interés: no es lo mismo tener \$10000 MXN disponibles hoy a recibir \$10000 MXN en un año, pues los primeros pueden ser invertidos en un negocio o una cuenta bancaria para generar *interés*. Por lo tanto los \$10000 MXN disponibles hoy valen más que los \$10000 MXN que se recibirán en un año.

Referencia:
- Vidaurri Aguirre, Héctor Manuel. *Ingeniería económica básica*, ISBN: 978-607-519-017-4. (Disponible en biblioteca)

Referencias:
- http://www.sympy.org
- http://matplotlib.org
- http://www.numpy.org
- http://ipywidgets.readthedocs.io/en/latest/index.html

___
## Interés
Nos centraremos en como cambia el valor del dinero en el tiempo debido al **interés**. Existen dos tipos:

### Capitalización por interés simple
Este tipo de interés se calcula única y exclusivamente sobre la cantidad original que se invirtió. Como consecuencia, el interés generado no forma parte del dinero que se invierte, es decir, los intereses no ganan intereses.

Suponga que se tiene un capital inicial $C_0$ y se invierte a un plazo de $k$ periodos (pueden ser meses, trimestres, semestres, años...) a una tasa de **interés simple** por periodo $i$. Al final del primer periodo, el capital $C_1$ que se obtiene es:

$$C_1=C_0+iC_0=C_0(1+i).$$

De la misma manera, como el interés solo se calcula sobre el capital inicial, al final del segundo periodo, el capital $C_2$ que se obtiene es:

$$C_2=C_1+iC_0=C_0+iC_0+iC_0=C_0(1+2i).$$

Así, al final del $k-$ésimo periodo, el capital $C_k$ que se obtiene es:

$$C_k=C_{k-1}+iC_0=C_0+kiC_0=C_0(1+ki).$$

> **Ejemplo.** Suponga que se tiene un capital de \$10000 MXN, el cual se pone en un fondo de inversión que paga una tasa de interés simple del 0.8% mensual.

> Si se tiene una meta de ahorro de \$11000 MXN sin inversiones adicionales, ¿cuántos meses se debería dejar invertido el dinero?

```python
# Librería para cálculo numérico
import numpy as np

# Valores dados en el enunciado
C_0, meta, i = 10000, 11000, 0.008
# Despejamos k tal que C_k=meta
k = np.ceil((meta/C_0 - 1)/i)   # Notar el uso de la función ceil (no se puede tener un número no entero de periodos)
k = k.astype(int)               # Conversión a entero (para visualización)
C_k = C_0*(1+k*i)               # Cálculo del capital al final del periodo k
C_k = round(C_k, 2)             # Redondeo con dos cifras decimales

# Imprimimos respuesta en pantalla
print("El número de periodos que se debe dejar invertido el dinero es ", k,". Al final del periodo ", k,
      ", el capital es ", C_k, ".", sep="")
```

    El número de periodos que se debe dejar invertido el dinero es 13. Al final del periodo 13, el capital es 11040.0.

> **Actividad.**
> - ¿Qué pasa si el interés no es del 0.8% mensual sino del 1% mensual?
> - ¿Qué pasa si la meta no son \$11000 MXN si no \$12000 MXN?

```python
# Solución
```

> Una gráfica que nos permite ilustrar la situación anterior se puede realizar de la siguiente manera.

```python
# Librerías para gráficos
import matplotlib.pyplot as plt
# Para que se muestren las gráficas en la misma ventana
%matplotlib inline
# Librería para widgets de jupyter
from ipywidgets import *

def interes_simple(C_0, meta, i):
    # Despejamos k
    k = np.ceil((meta/C_0 - 1)/i)           # Notar el uso de la función ceil
    k = k.astype(int)                       # Conversión a entero
    C_k = C_0*(1+k*i)                       # Cálculo del capital al final del periodo k
    C_k = round(C_k, 2)                     # Redondeo con dos cifras decimales
    # Vector de periodos
    kk = np.linspace(0,k,k+1)
    # Vector de capitales por periodo
    CC = C_0*(1+kk*i)
    # Gráfico
    plt.figure(num=1); plt.clf()            # Figura 1, borrar lo que contenga
    plt.plot(kk, CC,'*',linewidth=3.0)      # Se grafica la evolución de los capitales
    plt.plot(kk,meta*np.ones(k+1),'--k')    # Se grafica la meta
    plt.xlabel('k')                         # Etiqueta eje x
    plt.ylabel('C_k')                       # Etiqueta eje y
    plt.grid(True)                          # Malla en la gráfica
    plt.show()                              # Mostrar la figura
    print("El número de periodos que se debe dejar invertido el dinero para llegar a la meta de ", meta," es ", k,
          ". Al final del periodo ", k,", el capital es ", C_k, ".", sep="")

interact_manual(interes_simple, C_0=fixed(10000), meta=(10000,12000,100), i=fixed(0.008));
```

    interactive(children=(IntSlider(value=11000, description='meta', max=12000, min=10000, step=100), Button(descr…

Como se esperaba, el capital en el $k-$ésimo periodo $C_k=C_0(1+ki)$ crece linealmente con $k$.

### Capitalización por interés compuesto
El capital que genera el interés simple permanece constante todo el tiempo de duración de la inversión. En cambio, el que produce el interés compuesto en un periodo se convierte en capital en el siguiente periodo. Esto es, el interés generado al final de un periodo se reinvierte para el siguiente periodo para también producir interés.

Suponga que se tiene un capital inicial $C_0$, y se va a ceder el uso de este capital por un periodo de tiempo determinado a una tasa de interés $i$. El capital que se obtiene al final del primer periodo $C_1$ se puede calcular por

$$C_1=C_0(1+i).$$

Si la anterior suma se vuelve a ceder a la misma tasa de interés, al final del periodo dos el capital $C_2$ es

$$C_2=C_1(1+i)=C_0(1+i)^2.$$

Si se repite el anterior proceso $k$ veces, el capital al final del $k-$ésimo periodo $C_k$ es

$$C_k=C_{k-1}(1+i)=C_0(1+i)^k.$$

**Referencia**:
- https://es.wikipedia.org/wiki/Inter%C3%A9s_compuesto.

> **Ejemplo.** Suponga que se tiene un capital de \$10000 MXN, el cual se pone en un fondo de inversión que paga una tasa de interés del 0.8% mensual.

> Si se tiene una meta de ahorro de \$11000 MXN sin inversiones adicionales, ¿cuántos meses se debería dejar invertido el dinero?

> Muestre una gráfica que ilustre la situación.

```python
def interes_compuesto(C_0, meta, i):
    # Despejamos k
    k = np.ceil(np.log(meta/C_0)/np.log(1+i))
    k = k.astype(int)
    C_k = C_0*(1+i)**k                      # Cálculo del capital al final del periodo k
    #C_k = round(C_k,2)                      # Redondeo con dos cifras decimales
    # Vector de periodos
    kk = np.linspace(0,k,k+1)
    # Vector de capitales por periodo
    CC = C_0*(1+i)**kk
    # Gráfico
    plt.figure(num=1); plt.clf()            # Figura 1, borrar lo que contenga
    plt.plot(kk, CC,'*',linewidth=3.0)      # Se grafica la evolución de los capitales
    plt.plot(kk,meta*np.ones(k+1),'--k')    # Se grafica la meta
    plt.xlabel('k')                         # Etiqueta eje x
    plt.ylabel('C_k')                       # Etiqueta eje y
    plt.grid(True)                          # Malla en la gráfica
    plt.show()                              # Mostrar la figura
    print("El número de periodos que se debe dejar invertido el dinero para llegar a la meta de ", meta," es ", k,
          ". Al final del periodo ", k,", el capital es ", C_k, ".", sep="")

interact_manual(interes_compuesto, C_0=fixed(10000), meta=(10000,12000,100), i=fixed(0.008));
```

    interactive(children=(IntSlider(value=11000, description='meta', max=12000, min=10000, step=100), Button(descr…

El capital en el $k-$ésimo periodo $C_k=C_0(1+i)^k$ crece de manera exponencial con $k$.

> **Actividad.**
> - Modificar el código anterior para dejar fija la meta de ahorro y variar la tasa de interés compuesta.

```python
interact_manual(interes_compuesto, C_0=fixed(10000), meta=fixed(11000), i=(0.01,0.05,0.01));
```

    interactive(children=(FloatSlider(value=0.02, description='i', max=0.05, min=0.01, step=0.01), Button(descript…

### Capitalización continua de intereses
La capitalización continua se considera un tipo de capitalización compuesta, en la que a cada instante de tiempo $t$ se se capitalizan los intereses. Es decir, la frecuencia de capitalización es infinita (o, equivalentemente, el periodo de capitalización tiende a cero).

Suponga que se tiene un capital inicial $C_0$, y que el capital acumulado en el tiempo $t$ es $C(t)$. Queremos saber cuanto será el capital pasado un periodo de tiempo $\Delta t$, dado que la tasa de interés efectiva para este periodo de tiempo es $i$. De acuerdo a lo anterior tenemos

$$C(t+\Delta t)=C(t)(1+i)=C(t)(1+r\Delta t),$$

donde $r=\frac{i}{\Delta t}$ es la tasa de interés instantánea. Manipulando la anterior expresión, obtenemos

$$\frac{C(t+\Delta t)-C(t)}{\Delta t}=r\; C(t).$$

Haciendo $\Delta t\to 0$, obtenemos la siguiente ecuación diferencial

$$\frac{d C(t)}{dt}=r\; C(t),$$

sujeta a la condición inicial (monto o capital inicial) $C(0)=C_0$.

La anterior, es una ecuación diferencial lineal de primer orden, para la cual se puede calcular la *solución analítica*.

```python
# Librería de cálculo simbólico
import sympy as sym

# Para imprimir en formato TeX
from sympy import init_printing; init_printing(use_latex='mathjax')

# Símbolos t(para el tiempo) y r(para el interés instantáneo)
#t, r = sym.symbols('t r')
# Otra forma de hacer lo anterior
sym.var('t r')

C = sym.Function('C')
```

```python
eqn = sym.Eq(sym.Derivative(C(t),t) - r*C(t), 0) # Ecuación diferencial
display(eqn) # Mostrar ecuación
sym.dsolve(eqn, C(t)) # Resolver
```

$$- r C{\left (t \right )} + \frac{d}{d t} C{\left (t \right )} = 0$$

$$C{\left (t \right )} = C_{1} e^{r t}$$

con $C_1=C_0$.

La equivalencia entre la tasa de interés compuesta $i$ y la tasa de interés instantánea $r$ viene dada por

$$e^r=1+i.$$

___
¿Cómo podemos calcular la *solución numérica*?

> **Ejemplo.** Suponga que se tiene un capital de \$10000 MXN, el cual se pone en un fondo de inversión que paga una tasa de interés del 0.8% mensual.

> Si se tiene una meta de ahorro de \$11000 MXN sin inversiones adicionales, ¿cuánto tiempo se debe dejar invertido el dinero?

> Muestre una gráfica que ilustre la situación.

```python
# Librerías para integración numérica
from scipy.integrate import odeint
```

```python
help(odeint)
```

    Help on function odeint in module scipy.integrate.odepack:

    odeint(func, y0, t, args=(), Dfun=None, col_deriv=0, full_output=0, ml=None, mu=None, rtol=None, atol=None, tcrit=None, h0=0.0, hmax=0.0, hmin=0.0, ixpr=0, mxstep=0, mxhnil=0, mxordn=12, mxords=5, printmessg=0, tfirst=False)
        Integrate a system of ordinary differential equations.

        .. note:: For new code, use `scipy.integrate.solve_ivp` to solve a
                  differential equation.

        Solve a system of ordinary differential equations using lsoda from the
        FORTRAN library odepack.

        Solves the initial value problem for stiff or non-stiff systems
        of first order ode-s::

            dy/dt = func(y, t, ...)  [or func(t, y, ...)]

        where y can be a vector.

        .. note:: By default, the required order of the first two arguments of
                  `func` are in the opposite order of the arguments in the system
                  definition function used by the `scipy.integrate.ode` class and
                  the function `scipy.integrate.solve_ivp`.  To use a function with
                  the signature ``func(t, y, ...)``, the argument `tfirst` must be
                  set to ``True``.

        Parameters
        ----------
        func : callable(y, t, ...) or callable(t, y, ...)
            Computes the derivative of y at t.
            If the signature is ``callable(t, y, ...)``, then the argument
            `tfirst` must be set ``True``.
        y0 : array
            Initial condition on y (can be a vector).
        t : array
            A sequence of time points for which to solve for y.  The initial
            value point should be the first element of this sequence.
        args : tuple, optional
            Extra arguments to pass to function.
        Dfun : callable(y, t, ...) or callable(t, y, ...)
            Gradient (Jacobian) of `func`.
            If the signature is ``callable(t, y, ...)``, then the argument
            `tfirst` must be set ``True``.
        col_deriv : bool, optional
            True if `Dfun` defines derivatives down columns (faster),
            otherwise `Dfun` should define derivatives across rows.
        full_output : bool, optional
            True if to return a dictionary of optional outputs as the second output
        printmessg : bool, optional
            Whether to print the convergence message
        tfirst: bool, optional
            If True, the first two arguments of `func` (and `Dfun`, if given)
            must ``t, y`` instead of the default ``y, t``.

            .. versionadded:: 1.1.0

        Returns
        -------
        y : array, shape (len(t), len(y0))
            Array containing the value of y for each desired time in t,
            with the initial value `y0` in the first row.
        infodict : dict, only returned if full_output == True
            Dictionary containing additional output information

            =======  ============================================================
            key      meaning
            =======  ============================================================
            'hu'     vector of step sizes successfully used for each time step.
            'tcur'   vector with the value of t reached for each time step.
                     (will always be at least as large as the input times).
            'tolsf'  vector of tolerance scale factors, greater than 1.0,
                     computed when a request for too much accuracy was detected.
            'tsw'    value of t at the time of the last method switch
                     (given for each time step)
            'nst'    cumulative number of time steps
            'nfe'    cumulative number of function evaluations for each time step
            'nje'    cumulative number of jacobian evaluations for each time step
            'nqu'    a vector of method orders for each successful step.
            'imxer'  index of the component of largest magnitude in the
                     weighted local error vector (e / ewt) on an error return, -1
                     otherwise.
            'lenrw'  the length of the double work array required.
            'leniw'  the length of integer work array required.
            'mused'  a vector of method indicators for each successful time step:
                     1: adams (nonstiff), 2: bdf (stiff)
            =======  ============================================================

        Other Parameters
        ----------------
        ml, mu : int, optional
            If either of these are not None or non-negative, then the
            Jacobian is assumed to be banded.  These give the number of
            lower and upper non-zero diagonals in this banded matrix.
            For the banded case, `Dfun` should return a matrix whose
            rows contain the non-zero bands (starting with the lowest diagonal).
            Thus, the return matrix `jac` from `Dfun` should have shape
            ``(ml + mu + 1, len(y0))`` when ``ml >=0`` or ``mu >=0``.
            The data in `jac` must be stored such that ``jac[i - j + mu, j]``
            holds the derivative of the `i`th equation with respect to the `j`th
            state variable.  If `col_deriv` is True, the transpose of this
            `jac` must be returned.
        rtol, atol : float, optional
            The input parameters `rtol` and `atol` determine the error
            control performed by the solver.  The solver will control the
            vector, e, of estimated local errors in y, according to an
            inequality of the form ``max-norm of (e / ewt) <= 1``,
            where ewt is a vector of positive error weights computed as
            ``ewt = rtol * abs(y) + atol``.
            rtol and atol can be either vectors the same length as y or scalars.
            Defaults to 1.49012e-8.
        tcrit : ndarray, optional
            Vector of critical points (e.g. singularities) where integration
            care should be taken.
        h0 : float, (0: solver-determined), optional
            The step size to be attempted on the first step.
        hmax : float, (0: solver-determined), optional
            The maximum absolute step size allowed.
        hmin : float, (0: solver-determined), optional
            The minimum absolute step size allowed.
        ixpr : bool, optional
            Whether to generate extra printing at method switches.
        mxstep : int, (0: solver-determined), optional
            Maximum number of (internally defined) steps allowed for each
            integration point in t.
        mxhnil : int, (0: solver-determined), optional
            Maximum number of messages printed.
        mxordn : int, (0: solver-determined), optional
            Maximum order to be allowed for the non-stiff (Adams) method.
        mxords : int, (0: solver-determined), optional
            Maximum order to be allowed for the stiff (BDF) method.

        See Also
        --------
        solve_ivp : Solve an initial value problem for a system of ODEs.
        ode : a more object-oriented integrator based on VODE.
        quad : for finding the area under a curve.

        Examples
        --------
        The second order differential equation for the angle `theta` of a
        pendulum acted on by gravity with friction can be written::

            theta''(t) + b*theta'(t) + c*sin(theta(t)) = 0

        where `b` and `c` are positive constants, and a prime (') denotes a
        derivative.  To solve this equation with `odeint`, we must first convert
        it to a system of first order equations.  By defining the angular
        velocity ``omega(t) = theta'(t)``, we obtain the system::

            theta'(t) = omega(t)
            omega'(t) = -b*omega(t) - c*sin(theta(t))

        Let `y` be the vector [`theta`, `omega`].  We implement this system
        in python as:

        >>> def pend(y, t, b, c):
        ...     theta, omega = y
        ...     dydt = [omega, -b*omega - c*np.sin(theta)]
        ...     return dydt
        ...

        We assume the constants are `b` = 0.25 and `c` = 5.0:

        >>> b = 0.25
        >>> c = 5.0

        For initial conditions, we assume the pendulum is nearly vertical
        with `theta(0)` = `pi` - 0.1, and is initially at rest, so
        `omega(0)` = 0.  Then the vector of initial conditions is

        >>> y0 = [np.pi - 0.1, 0.0]

        We will generate a solution at 101 evenly spaced samples in the interval
        0 <= `t` <= 10.  So our array of times is:

        >>> t = np.linspace(0, 10, 101)

        Call `odeint` to generate the solution.  To pass the parameters
        `b` and `c` to `pend`, we give them to `odeint` using the `args`
        argument.

        >>> from scipy.integrate import odeint
        >>> sol = odeint(pend, y0, t, args=(b, c))

        The solution is an array with shape (101, 2).  The first column
        is `theta(t)`, and the second is `omega(t)`.  The following code
        plots both components.

        >>> import matplotlib.pyplot as plt
        >>> plt.plot(t, sol[:, 0], 'b', label='theta(t)')
        >>> plt.plot(t, sol[:, 1], 'g', label='omega(t)')
        >>> plt.legend(loc='best')
        >>> plt.xlabel('t')
        >>> plt.grid()
        >>> plt.show()

```python
# Modelo de capitalización continua
def cap_continua(C, t, r):
    return r * C
```

```python
def interes_continuo(C_0, meta, r):
    # Despejamos t
    t = np.log(meta/C_0)/r
    # Vector de periodos
    tt = np.linspace(0,t,100)
    # Vector de capitales por periodo
    CC = odeint(cap_continua, C_0, tt, args = (r,))
    # Gráfico
    plt.figure(num=1); plt.clf()                # Figura 1, borrar lo que contenga
    plt.plot(tt, CC,'-',linewidth=3.0)          # Se grafica la evolución de los capitales
    plt.plot(tt,meta*np.ones(len(tt)),'--k')    # Se grafica la meta
    plt.xlabel('t')                             # Etiqueta eje x
    plt.ylabel('C(t)')                          # Etiqueta eje y
    plt.grid(True)                              # Malla en la gráfica
    plt.show()                                  # Mostrar la figura
    print("El tiempo que se debe dejar invertido el dinero para llegar a la meta de ", meta," es ", t, " meses.", sep="")

interact_manual(interes_continuo, C_0=fixed(10000), meta=(10000,20000,100), r=fixed(np.log(1+i)));
```

    interactive(children=(IntSlider(value=15000, description='meta', max=20000, min=10000, step=100), Button(descr…

### Actividad.
1. Resolver con sympy la siguientes ecuación diferencial:
$$\frac{dx}{dt}=\frac{x}{t+1}$$
2. Obtener la solución numérica.
3. Comparar.

```python
import sympy as sym
```

```python

```

```python
# Definimos variables
sym.var('t', real=True, positive=True)
x = sym.Function('x')
# Ecuacion diferencial
ode = sym.Eq(sym.Derivative(x(t),t)-x(t)/(t+1),0)
display(ode)
```

```python

```

```python

```

___
## Tabla de abonos
Como aplicación importante del concepto de interés compuesto se encuentra la creación de un modelo de cuenta de ahorro.

Referencia:
- Vidaurri Aguirre, Héctor Manuel. *Ingeniería económica básica*, ISBN: 978-607-519-017-4. (Disponible en biblioteca)
- http://pbpython.com/amortization-model.html

```python
# Librería de análisis de datos
import pandas as pd
# Librería para manipulación de fechas
from datetime import date

# Datos para la cuenta de ahorro
Tasa_interes = 0.04
Anos = 3
Abonos_ano = 12
Inicial = 10000
Meta = 50000
fecha_inicio = (date(2018,1,31))

# Cálculo de abonos mensuales iguales para llegar a la meta de ahorro en el tiempo deseado
Abono = (Tasa_interes/Abonos_ano)*(Meta - Inicial*(1+Tasa_interes/Abonos_ano)**(Anos*Abonos_ano))/((1+Tasa_interes/Abonos_ano)**(Anos*Abonos_ano)-1)

# Rango de fechas en la tabla
rng = pd.date_range(fecha_inicio, periods=Anos*Abonos_ano, freq='MS')
rng.name = "Fecha del abono"

# Columnas de abonos y extracto de cuenta en las fechas respectivas
df = pd.DataFrame(index=rng,columns=['Abono', 'Balance'], dtype='float')

# Para poner el indice con los periodos y no con las fechas
df.reset_index(inplace=True)
df.index += 1
df.index.name = "Periodo"  # Columna correspondiente al periodo

# Los abonos son iguales
df["Abono"] = Abono

# Vector indizador de los periodos de los abonos
index_vector = np.arange(1,len(rng)+1)
# Extracto de la cuenta mes a mes
df["Balance"] = Inicial*(1+Tasa_interes/Abonos_ano)**index_vector+Abono*(((1+Tasa_interes/Abonos_ano)**index_vector-1)/(Tasa_interes/Abonos_ano))

# Redondear con dos cifras decimales
df = df.round(2)
df

# La siguiente solución es también válida. Sin embargo, evitar el uso de 'for'

#df.loc[1, "Balance"] = Inicial*(1+Tasa_interes/Abonos_ano)**1 + df.loc[1, "Abono"]

#for i in range(2, len(df)+1):
#    # Get the previous balance as well as current payments
#    prev_balance = df.loc[i-1, "Balance"]

#    df.loc[i, "Balance"] = prev_balance*(1+Tasa_interes/Abonos_ano)**1 + df.loc[i, "Abono"]
```

      Fecha del abono
      Abono
      Balance

      Periodo

      1
      2018-02-01
      1014.29
      11047.63

      2
      2018-03-01
      1014.29
      12098.74

      3
      2018-04-01
      1014.29
      13153.37

      4
      2018-05-01
      1014.29
      14211.50

      5
      2018-06-01
      1014.29
      15273.17

      6
      2018-07-01
      1014.29
      16338.37

      7
      2018-08-01
      1014.29
      17407.13

      8
      2018-09-01
      1014.29
      18479.44

      9
      2018-10-01
      1014.29
      19555.33

      10
      2018-11-01
      1014.29
      20634.81

      11
      2018-12-01
      1014.29
      21717.89

      12
      2019-01-01
      1014.29
      22804.57

      13
      2019-02-01
      1014.29
      23894.88

      14
      2019-03-01
      1014.29
      24988.82

      15
      2019-04-01
      1014.29
      26086.41

      16
      2019-05-01
      1014.29
      27187.66

      17
      2019-06-01
      1014.29
      28292.58

      18
      2019-07-01
      1014.29
      29401.18

      19
      2019-08-01
      1014.29
      30513.47

      20
      2019-09-01
      1014.29
      31629.48

      21
      2019-10-01
      1014.29
      32749.20

      22
      2019-11-01
      1014.29
      33872.66

      23
      2019-12-01
      1014.29
      34999.86

      24
      2020-01-01
      1014.29
      36130.82

      25
      2020-02-01
      1014.29
      37265.55

      26
      2020-03-01
      1014.29
      38404.06

      27
      2020-04-01
      1014.29
      39546.37

      28
      2020-05-01
      1014.29
      40692.48

      29
      2020-06-01
      1014.29
      41842.41

      30
      2020-07-01
      1014.29
      42996.18

      31
      2020-08-01
      1014.29
      44153.79

      32
      2020-09-01
      1014.29
      45315.27

      33
      2020-10-01
      1014.29
      46480.61

      34
      2020-11-01
      1014.29
      47649.84

      35
      2020-12-01
      1014.29
      48822.96

      36
      2021-01-01
      1014.29
      50000.00

```python
plt.plot(df['Fecha del abono'], df['Balance']);
```

> **Tarea.**
> - Averiguar tasas de interés reales en algún banco y proyectar un ahorro mensual para que al terminar su carrera tengan $50000 MXN en su cuenta.
> - Hacer esto en un nuevo archivo, llamarlo Tarea6_ApellidoNombre.ipynb y subirlo a moodle.
> - Plazo: Jueves a las 23:00.

Created with Jupyter by Esteban Jiménez Rodríguez.


* Autor - Bruno Augusto Souza Oliveira

* e-mail - bruno.augusto.souza@outlook.com

* Linkedin - https://www.linkedin.com/in/brunoaugustosouza/

# Resumo

Iremos construir uma estratégia utilizando o modelo de fronteira eficiente criado por Harry Markowitz.  Este modelo nos permite analisar um portfólio de investimentos com base no risco e retorno da carteira, por meio dele, podemos combinar diferentes pesos para cada ativo, gerando diferentes portfólios e buscando, sempre que possível, maximizar nosso retorno enquanto minimizamos o risco. Faremos uma breve introdução da teoria do modelo e também do movimento Browniano geométrico, o qual será usado na nossa estratégia.

# Teoria - Fronteira Eficiente

Resumidamente, no modelo de Markowitz queremos maximizar uma função que envolve métricas de risco e de retorno. Definiremos o retorno de um portfólio com n ativos como:
$$R = \mu^T w$$

onde $w = [w_1, w_2, \dots, w_n]$ é o vetor de pesos, definido de tal forma que $\sum_{i=1}^n w_i = 1$ e, $\mu = [\mu_1, \mu_2, \dots, \mu_n]$ é o vetor de retorno dos n ativos. Em seguida, definimos a métrica de risco como:

$$w^T \Sigma w$$

onde $\Sigma$ é a matriz de variâncias e covariâncias. Iremos gerar um grande número de portfólios com diferentes pesos, fixando o risco e o retorno dos ativos teóricos, para por fim, podermos ter uma breve visualização do funcionamento do modelo.

```python
import pandas as pd
import os
import numpy as np
import matplotlib.pyplot as plt

def plot_portfolio(μ, Σ):
    """
    Creation of a markowitz bullet
    """
    n = len(Σ)
    PortfolioReturn = []
    PortfolioRisk = []
    for t in range(5000):
        w = np.random.rand(n)
        w /= np.sum(w)
        Risk = np.sqrt(np.dot(w.T, np.dot(Σ,w)))
        Return = np.dot(μ.T, w)
        PortfolioReturn.append(Return)
        PortfolioRisk.append(Risk)
    fig, ax = plt.subplots(figsize=(16,10), facecolor=(0.75, 0.75, 0.75))
    ax.plot(PortfolioRisk, PortfolioReturn, 'o', label='Portfolios')
    plt.xlabel('Standard deviation (Risk)')
    plt.ylabel('Mean (Returns)')
    plt.title('Markowitz Bullet')
    plt.grid()

#Number of assets
n_assets = 4

#Number of observatios (252 days = 1 year)
n_obs = 252*2

returns = np.random.randn(n_assets, n_obs) #Gerar n ativos com n retornos i.i.d.
means = returns.mean(axis=1)
risk = np.cov(returns)
plot_portfolio(means, risk)
```

Perceba que existe a possibilidade de otimizarmos os pesos em cada ativo pelo Risco x Retorno.

# Teoria - movimento Browniano geométrico

Um processo estocástico $\{S_t\}_{t \geq 0}$ pode ser modelado como um mBg (movimento Browniano geométrico) se satisfazer a  SDE:

$$dS_t = \mu S_t dt + \sigma S_t dB_t$$

Neste caso, a solução da SDE acima é dada por $S_t = S_0 e^{X(t)}$, onde $X(t) = \left(\mu - \frac{\sigma^2}{2} \right)t + \sigma B_t$ e $B_t$ é um movimento Browniano, ou seja, $X(t)$ é um movimento Browniano com drift $\left(\mu - \frac{\sigma^2}{2}\right)$.

## Propriedade

$\quad$ i. $\lim_{t \to +\infty} \mathbb{P}\left(S_t = +\infty \mid \mu - \frac{\sigma^2}{2} > 0\right) = 1$.

# Estratégia

O backtest abaixo tem a seguinte ideia, mensalmente, utilizando como universo os ativos que compoem o ibovespa no dia 20/10/2021, estimaremos os parâmetros do modelo movimento Browniano geométrico e iremos rankear os ativos do maior para o menor com base na seguinte fórmula:

$$\frac{\mu}{\sigma^2}$$

A ideia é que queremos os ativos com maior drift e menor volatilidade, e aproveitar-se da propriedade i citada anteriormente. Observe que $\mu$ e $\sigma$ são os parâmetros do modelo. Selecionaremos o top n ativos com maior ranking e aplicaremos a fronteira eficiente nesse conjunto, a estratégia fará o rebalanceamento dos ativos mensalmente. (Observe que a composição do ibovespa muda diariamente, então há um problema de look ahead bias)

```python
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)
from collections import OrderedDict

#library for filtering signals
import statsmodels.api as sm

#Numerical operations
import numpy as np
from numpy import log as ln
from math import floor, isnan

#Symbolic operations
import sympy as sy
from sympy import symbols, lambdify
from sympy.functions.elementary.exponential import exp

#Datetime operations
import datetime

#Import backtrader platform
import backtrader as bt
import backtrader.analyzers as btanalyzers

#Module to analyze portfolio
import pyfolio as pf

#Ploting library
import matplotlib.pyplot as plt

#operating system library
import os

#Optimizations
from scipy.optimize import minimize, Bounds, LinearConstraint
from scipy.stats import norm as N

#portfolio analyzer
import quantstats as qs

#data analysis library
import pandas as pd

#debugging library
from ipdb import set_trace

class Portfolio():
    """
        Class portfolio, retrieve optimal portfolio weights based on Markowitz efficient frontier model.

        Inputs:
            μ - assets log return vector
            Σ - assets variance-covariance matrix
            minimize_method - method to optimize (std = standard deviation, neg_sharpe = negative sharpe)
            long_only(True/False) - allows short position if False (not implemented)
    """

    def __init__(self, μ, Σ, premiumRisk=0, minimize_method='std', long_only=True):

        n = len(μ)

        self.n = n
        self.μ = μ
        self.Σ = Σ
        self.premiumRisk = 0
        self.n_days = 1
        self.minimize_method = minimize_method
        self.premiumRisk = premiumRisk
        self.set_long_only = long_only

    def n_bound(self,n):
        return [(0,None) for i in range(n)]

    def std(self, w, μ, Σ):
        return (np.dot(w.T, np.dot(Σ, w)))**0.5

    def neg_sharpe(self, w, μ, Σ):
        return -(np.dot(w.T,μ))/((np.dot(w.T, np.dot(Σ, w)))**0.5)

    def optimize(self):
        μ = self.μ
        Σ = self.Σ
        n = self.n
        if self.minimize_method=='std':
            f=self.std
        if self.minimize_method=='neg_sharpe':
            f=self.neg_sharpe

        #Initial Guess
        w = np.random.rand(n)
        w /= np.sum(w)

        #Model Constraints

        bounds = self.n_bound(n)
        constraints = LinearConstraint(np.ones(n), lb=1, ub=1)

        #Optimization
        optimization = minimize(f, w, args=(μ, Σ),
                        method='SLSQP',
                        bounds=bounds, constraints=constraints)

        #Risk, Return Calculations
        optW = optimization['x']
        optReturn = np.dot(optW.T,self.μ)
        optRisk = np.sqrt(np.dot(optW.T, np.dot(self.Σ, optW)))

        self.optRisk = optRisk
        self.optReturn = optReturn

        return optW, optRisk, optReturn

    def plot(self):
        #Creation of a markowitz graph representation
        n = self.n
        PortfolioReturn = []
        PortfolioRisk = []
        for t in range(10000):
            ω = np.random.rand(n)
            ω /= (np.sum(ω)*1)
            Return = np.dot(ω.T, self.μ)
            Risk = np.sqrt(np.dot(ω.T, np.dot(self.Σ, ω)))
            PortfolioReturn.append(Return)
            PortfolioRisk.append(Risk)
        fig, ax = plt.subplots(figsize=(16,10), facecolor=(0.75, 0.75, 0.75))

        #plot
        ax.plot(PortfolioRisk, PortfolioReturn, 'o', label='Portfolios')
        ax.plot(self.optRisk, self.optReturn, 'bo', label='Optimal Point')

        #aesthetics
        ax.set_facecolor((0.1, 0.1, 0.1))
        ax.grid(True)
        ax.set_ylabel('Expected Return')
        ax.set_xlabel('Standard Deviation')
        ax.legend()

        plt.show()

class gBm(bt.Indicator):
    """
        Geometric Brownian Motion model.
                dS_t = μ S_t dt + σ S_t dB_t

        Crescimento Exponencial.
        o modelo tem um fit melhor para growth stocks como AAPL, TSLA, MGLU3, PRIO3 etc i.e.
        para ativos que tem um Crescimento Exponencial
    """
    lines = ('m', 'σ', 'μ', 'var', 'sl')
    params = dict(size=2**9+1,
                  sl_quantil=0.1,
                  sl_look_forward=5)

    def __init__(self):
        self.addminperiod(self.p.size)

    def next(self):
        self.S = self.data.close.get(0, size=self.p.size)
        self.estimateParams()

        self.lines.m[0] = self.m
        self.lines.σ[0] = self.σ
        self.lines.μ[0] = self.μ
        self.lines.var[0] = self.Var(self.p.sl_look_forward)
        self.lines.sl[0] = self.q(self.p.sl_quantil, self.p.sl_look_forward)

    def estimateParams(self):
        """
        Referência:
        "Estimation of Geometric Brownian Motion Parameters for Oil Price Analysis" C. Jakob et al.
        """
        S = self.S
        X = np.diff(np.log(S), n=1)
        m = X.mean() #mean
        σ = X.std() #standard deviation
        μ = m + ((σ**2)/2) #drift
        n = len(S)

        self.m = m
        self.σ = σ
        self.μ = μ
        self.n = n

    def E(self, t):
        """
        Referência:
        Ross, Sheldon M. (2014). "Variations on Brownian Motion".
        Introduction to Probability Models (11th ed.).
        """
        S = self.S
        S0 = S[-1]
        μ = self.μ
        return S0*np.exp(μ*t)

    def Var(self, t):
        """
        Referência:
        Ross, Sheldon M. (2014). "Variations on Brownian Motion".
        Introduction to Probability Models (11th ed.).
        """
        S = self.S
        S0 = S[-1]
        μ = self.μ
        σ = self.σ
        return (S0**2)*np.exp(2*μ*t)*(np.exp((σ**2)*t) - 1)

    def q(self, p, t):
        """
         quantil de St/S0 o qual é definido como:
                q(p) = exp( (μ - σ**2/2)*t + σ*np.sqrt(t)*inv_Φ(p))
         p ∈ (0,1)
        """
        #assert p>0 and p<1
        #assert type(t)==int

        σ = self.σ
        μ = self.μ

        mean = (μ - (σ**2/2))*t
        var = σ**2*t
        return np.exp(mean +  np.sqrt(var)*N.ppf(p, 0, 1))

class momentumStrat(bt.Strategy):
    """
        strategy class
    """
    params = dict(gBmSize=2**8+1, #number of observations to use in backtest to estimate parameters
                  enableLog=False, #enable log of buy and sell of assets
                  exposure=1, #exposure to the market
                  rebal_monthday=1, #which day in month to do the rebalancing
                  stocks_holding=10) #Max number of stocks holding

    def __init__(self):
        self.order = None #variable to track pending orders
        acoes = self.datas

        #construct line objects of gBm data
        gBms = [gBm(d, size=self.p.gBmSize) for d in acoes]
        driftAcoes = [gBm.μ for gBm in gBms]
        volatAcoes = [gBm.σ for gBm in gBms]

        #construct a line object of ranks
        self.rank = {d:drift/volat for d, drift, volat in zip(acoes, driftAcoes, volatAcoes)}

        self.driftAcoes = {d:drift for d, drift in zip(acoes, driftAcoes)}
        self.volatAcoes = {d:volat for d, volat in zip(acoes, volatAcoes)}

        #create a timer to execute the strategy montly
        self.add_timer(
            when=bt.Timer.SESSION_START,
            monthdays=[self.p.rebal_monthday],
            monthcarry=True  # if a day isn't there, execute on the next
        )

    def notify_order(self, order):
        '''Notify if an orde was executed/submited or not'''

        if order.status in [order.Submitted, order.Accepted]:
            # Buy/Sell order submitted/accepted to/by broker - Nothing to do
            return

        # Check if an order has been completed
        # Attention: broker could reject order if not enough cash
        if order.status in [order.Completed]:
            if order.isbuy():
                self.log(
                    'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                    (order.executed.price,
                     order.executed.value,
                     order.executed.comm))

                self.buyprice = order.executed.price
                self.buycomm = order.executed.comm

            else:  # Sell

                self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                         (order.executed.price,
                          order.executed.value,
                          order.executed.comm))

            self.bar_executed = len(self)

        elif order.status==order.Canceled:
            self.log('Order Canceled. Price: {}'.format(order.price))
        elif order.status==order.Margin:
            self.log('Order Margin. Price: {}'.format(order.price))
        elif order.status==order.Rejected:
            self.log('Order Rejected. Price: {}'.format(order.price))

        self.order = None

    def log(self, txt, dt=None, doprint=False):
        '''Logging function fot this strategy'''
        if self.params.enableLog or doprint:
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))

    def getPortfolioParameters(self, datas:list):
        '''Get portfolio assets log returns vector and assets variance-covariance matrix'''
        S = np.column_stack(tuple(np.array(d).T for d in map(lambda x: x.close.get(0, self.p.gBmSize), datas)))
        S0 = S[0]
        X = np.diff(np.log(np.divide(S,S0)), n=1, axis=0)
        μ = X.mean(axis=0)
        Σ = np.cov(X, rowvar=False)

        n = len(datas)

        return μ, Σ

    def notify_timer(self, timer, when, *args, **kwargs):
        '''this function is called monthly, it is our strategy logic'''
        l = len(self)
        if l < self.p.gBmSize or self.order:
            return

        rank = {d:rank for (d, rank) in self.rank.items() if d.close.get(0, self.p.gBmSize)}

        ranks = sorted(
            rank.items(),  # get the (d, rank), pair
            key=lambda x: x[1][0],  # use rank (elem 1) and current time "0"
            reverse=True,  # highest ranked 1st ... please
        )

        top_ranks = ranks[:self.p.stocks_holding]
        datas = [data for (data, rank) in top_ranks]
        μ, Σ = self.getPortfolioParameters(datas)
        portfolio = Portfolio(μ, Σ, minimize_method='neg_sharpe')
        w, Risk, Return = portfolio.optimize()
        #portfolio.plot()

        opt_portfolio = [(data,weight) for (data,weight) in zip(datas,w)]
        current_positions = [d for d, pos in self.getpositions().items() if pos]

        #if we're not in market
        if len(current_positions)==0:
            for (d,weight) in opt_portfolio:
                if weight!=0:
                    self.order = self.order_target_percent(d, target=weight)
                    self.log('Buying: {} /---/ Rank: {:.4f} /---/ Weight: {:.3f}'.format(d._name,
                                                                                rank[d][0],
                                                                                weight))
            return

        #if we're in market
        else:

            # remove those no longer top ranked
            # do this first to issue sell orders and free cash
            for d in (d for d in current_positions if d not in datas):
                self.log('Closing: {} /---/ Rank: {:.4f}'.format(d._name, rank[d][0]))
                self.order_target_percent(d, target=0.0)

            # rebalance those already top ranked and still there
            for d in (d for d in current_positions if d in datas):
                i = datas.index(d)
                self.log('Rebalancing: {} /---/ Rank: {:.4f} /---/ Weight: {:.3f}'.format(d._name,
                                                                                rank[d][0],
                                                                                w[i]))
                self.order_target_percent(d, target=w[i])
                del datas[i]  # remove it, to simplify next iteration
                w = np.delete(w, i)

            # issue a target order for the newly top ranked stocks
            # do this last, as this will generate buy orders consuming cash
            for i,d in enumerate(datas):

                if w[i]!=0:
                    self.log('Buying: {} /---/ Rank: {:.4f} /---/ Weight: {:.3f}'.format(d._name,
                                                                                rank[d][0],
                                                                                w[i]))
                    self.order_target_percent(d, target=w[i])
                else:
                    continue

    def stop(self):
        self.log('Ending Value %.2f' %
                 (self.broker.getvalue()), doprint=True)

def runStrat(in_start, in_end, wf=False, best_set=None):
    '''create cerebro instance, set cash, add data, add strategy and run it'''
    cerebro = bt.Cerebro()

    #Set cash value
    cerebro.broker.set_cash(100000)

    #Set commission modelo
    cerebro.broker.setcommission(commission=0)

    cerebro.broker.set_checksubmit(False)

    data_font = 'data'

    #Adding data
    start=datetime.datetime.strptime(in_start, '%Y-%m-%d')
    end=datetime.datetime.strptime(in_end, '%Y-%m-%d')

    files = os.listdir(data_font)

    for file in files:
        #print(file)
        datapath='{}//'.format(data_font)+file
        data = bt.feeds.YahooFinanceCSVData(dataname=datapath,
                                            fromdate=start,
                                            todate=end,
                                            reverse=False)
        cerebro.adddata(data)

    #Add analyzers
    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
    cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='sharpe')

    #Add Strategy
    if wf:
        cerebro.addstrategy(momentumStrat, wf=wf)
    else:
        cerebro.addstrategy(momentumStrat)

    #Run Algo
    results = cerebro.run(maxcpus=1)

    #Plot results
    #cerebro.plot()

    return results

def analyzeStrat(results, live):
    '''analyze strategy results'''
    strat = results[0]
    pyfoliozer = strat.analyzers.getbyname('pyfolio')
    returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()

    # extend pandas functionality with metrics, etc.
    qs.extend_pandas()
    returns.index = pd.to_datetime(returns.index.astype(str).str[:10])
    qs.reports.full(returns, '^BVSP')

if __name__=='__main__':
    live = '2021-01-01'
    r = runStrat('2005-01-03', '2021-06-02')
    analyzeStrat(r, live)
```


# Non Linear Regression Analysis

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression because, as the name implies, linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example on python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

### Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Though Linear regression is very good to solve many problems, it cannot be used for all datasets. First recall how linear regression, could model a dataset. It models a linear relation between a dependent variable y and independent variable x. It had a simple equation, of degree 1, for example y = 2*(x) + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

Non-linear regressions are a relationship between independent variables $x$ and a dependent variable $y$ which result in a non-linear function modeled data. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and others. For example: $$ y = \log(x)$$

Or even, more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying logarithmic map from input $x$'s to output variable $y$. It is one of the simplest form of __log()__: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be polynomial representation of the $x$'s. In general form it would be written as
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

    2019-02-15 08:52:08 URL:https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv [1218/1218] -> "china_gdp.csv" [1]

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)

### Plotting the Dataset ###
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it deaccelerates slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model ###

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta_1(X-\beta_2)}}$$

$\beta_1$: Controls the curve's steepness,

$\beta_2$: Slides the curve on the x-axis.

### Building The Model ###
Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?
we can use __curve_fit__ which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, *popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

     beta_1 = 690.453017, beta_2 = 0.997207

Now we plot our resulting regresssion model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice
Can you calculate what is the accuracy of our model?

```python
# write your code here

# split data into train/test
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )

```

    Mean absolute error: 0.02
    Residual sum of squares (MSE): 0.00
    R2-score: 0.94

Double-click __here__ for the solution.

## Want to learn more?

IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: [SPSS Modeler](http://cocl.us/ML0101EN-SPSSModeler).

Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at [Watson Studio](https://cocl.us/ML0101EN_DSX)

### Thanks for completing this lesson!

Notebook created by: Saeed Aghabozorgi

Copyright © 2018 [Cognitive Class](https://cocl.us/DX0108EN_CC). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/).