subfactorial

sympy.functions.combinatorial.factorials.subfactorial

subfactorial
$$!n=n!\sum_{i=0}^n\frac{(-1)^i}{i!},\quad n\geq 0$$
$$!n=n!-\sum_{i=1}^n{n\choose i}\cdot{!(n-i)},\quad\ n \ge 1$$
recurrence relation
$$!n=(n-1)\left(!(n-2)+!(n-1)\right)$$

```python
from typing import TypeVar

import treefactorial

Numeric = TypeVar('Numeric', int, float, complex)
EMPTY_SUM = 0
MINUS_ONE = -1

factorial = treefactorial.factorial
#binom = treefactorial.binom

def binom(n: int, k: int) -> int:
    """Pure Python binomial coefficient, using treefactorial."""
    return int(factorial(n)/(factorial(k)*factorial(n - k)))

def round(number: Numeric) -> int:
    """Round a floating point number to nearest integer."""
    if number < 0:
        return int(number - 0.5)
    else:
        return int(number + 0.5)

def _subfactorial1(n: int) -> int:
    """Pure Python subfactorial.
    Also called derangement number or de Montmort number or rencontres numbers.
    Published by Remond de Montmort in 1713.
    """
    sum = EMPTY_SUM
    for i in range(n + 1):
        sum += MINUS_ONE**i/factorial(i)
    return round(factorial(n)*sum)

def _subfactorial(n: int) -> int:
    """Pure Python subfactorial.
    Also called derangement number or de Montmort number or rencontres numbers.
    Published by Remond de Montmort in 1713.
    """
    if not n:
        #return round(factorial(n)*MINUS_ONE**0/factorial(0)) #cutdown version of _subfactorial for n=0
        return _subfactorial1(n)
    sum = EMPTY_SUM
    for i in range(1, n + 1):
        sum += binom(n, i)*_subfactorial(n - i)
    return factorial(n) - sum

def subfactorial(number: int) -> int:
    """Pure Python subfactorial.
    Also called derangement number or de Montmort number or rencontres numbers.
    Published by Remond de Montmort in 1713.
    Algorithm using recurrence by Euler.
    """
    result = list()
    for n in range(11):
        result += [_subfactorial(n)]
    if number < 11:
        return result[number]
    for n in range(11, number + 1):
        result += [(n - 1)*(result[n - 2] + result[n - 1])]
    return result[number]
```

```python
for n in range(52):
    print(n, subfactorial(n))
```

    0 1
    1 0
    2 1
    3 2
    4 9
    5 44
    6 265
    7 1854
    8 14833
    9 133496
    10 1334961
    11 14684570
    12 176214841
    13 2290792932
    14 32071101049
    15 481066515734
    16 7697064251745
    17 130850092279664
    18 2355301661033953
    19 44750731559645106
    20 895014631192902121
    21 18795307255050944540
    22 413496759611120779881
    23 9510425471055777937262
    24 228250211305338670494289
    25 5706255282633466762357224
    26 148362637348470135821287825
    27 4005791208408693667174771274
    28 112162153835443422680893595673
    29 3252702461227859257745914274516
    30 97581073836835777732377428235481
    31 3025013288941909109703700275299910
    32 96800425246141091510518408809597121
    33 3194414033122656019847107490716704992
    34 108610077126170304674801654684367969729
    35 3801352699415960663618057913952878940514
    36 136848697178974583890250084902303641858505
    37 5063401795622059603939253141385234748764684
    38 192409268233638264949691619372638920453057993
    39 7503961461111892333037973155532917897669261726
    40 300158458444475693321518926221316715906770469041
    41 12306496796223503426182275975073985352177589230680
    42 516872865441387143899655590953107384791458747688561
    43 22225533213979647187685190410983617546032726150608122
    44 977923461415104476258148378083279172025439950626757369
    45 44006555763679701431616677013747562741144797778204081604
    46 2024301565129266265854367142632387886092660697797387753785
    47 95142173561075514495155255703722230646355052796477224427894
    48 4566824330931624695767452273778667071025042534230906772538913
    49 223774392215649610092605161415154686480227084177314431854406736
    50 11188719610782480504630258070757734324011354208865721592720336801
    51 570624700149906505736143161608644450524579064652151801228737176850

```python
import sympy
for n in range(51):
    print(n, subfactorial(n), sympy.functions.combinatorial.factorials.subfactorial(n))
```

    0 1 1
    1 0 0
    2 1 1
    3 2 2
    4 9 9
    5 44 44
    6 265 265
    7 1854 1854
    8 14833 14833
    9 133496 133496
    10 1334961 1334961
    11 14684570 14684570
    12 176214841 176214841
    13 2290792932 2290792932
    14 32071101049 32071101049
    15 481066515734 481066515734
    16 7697064251745 7697064251745
    17 130850092279664 130850092279664
    18 2355301661033953 2355301661033953
    19 44750731559645106 44750731559645106
    20 895014631192902121 895014631192902121
    21 18795307255050944540 18795307255050944540
    22 413496759611120779881 413496759611120779881
    23 9510425471055777937262 9510425471055777937262
    24 228250211305338670494289 228250211305338670494289
    25 5706255282633466762357224 5706255282633466762357224
    26 148362637348470135821287825 148362637348470135821287825
    27 4005791208408693667174771274 4005791208408693667174771274
    28 112162153835443422680893595673 112162153835443422680893595673
    29 3252702461227859257745914274516 3252702461227859257745914274516
    30 97581073836835777732377428235481 97581073836835777732377428235481
    31 3025013288941909109703700275299910 3025013288941909109703700275299910
    32 96800425246141091510518408809597121 96800425246141091510518408809597121
    33 3194414033122656019847107490716704992 3194414033122656019847107490716704992
    34 108610077126170304674801654684367969729 108610077126170304674801654684367969729
    35 3801352699415960663618057913952878940514 3801352699415960663618057913952878940514
    36 136848697178974583890250084902303641858505 136848697178974583890250084902303641858505
    37 5063401795622059603939253141385234748764684 5063401795622059603939253141385234748764684
    38 192409268233638264949691619372638920453057993 192409268233638264949691619372638920453057993
    39 7503961461111892333037973155532917897669261726 7503961461111892333037973155532917897669261726
    40 300158458444475693321518926221316715906770469041 300158458444475693321518926221316715906770469041
    41 12306496796223503426182275975073985352177589230680 12306496796223503426182275975073985352177589230680
    42 516872865441387143899655590953107384791458747688561 516872865441387143899655590953107384791458747688561
    43 22225533213979647187685190410983617546032726150608122 22225533213979647187685190410983617546032726150608122
    44 977923461415104476258148378083279172025439950626757369 977923461415104476258148378083279172025439950626757369
    45 44006555763679701431616677013747562741144797778204081604 44006555763679701431616677013747562741144797778204081604
    46 2024301565129266265854367142632387886092660697797387753785 2024301565129266265854367142632387886092660697797387753785
    47 95142173561075514495155255703722230646355052796477224427894 95142173561075514495155255703722230646355052796477224427894
    48 4566824330931624695767452273778667071025042534230906772538913 4566824330931624695767452273778667071025042534230906772538913
    49 223774392215649610092605161415154686480227084177314431854406736 223774392215649610092605161415154686480227084177314431854406736
    50 11188719610782480504630258070757734324011354208865721592720336801 11188719610782480504630258070757734324011354208865721592720336801

```python

```


# Lecture 13 - Advanced Curve Fitting: Gaussian Processes to Encode Prior Knowledge about Functions

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set_context('talk')
sns.set_style('white')
```

## Objectives

+ Express prior knowledge/beliefs about unknown functions using Gaussian process (GP).

+ Sample functions from the probability measure defined by GP.

## Readings

+ [Chapter 1 from C.E. Rasmussen's textbook on Gaussian processes](http://www.gaussianprocess.org/gpml/chapters/RW1.pdf).

+ (Optional video lecture) [Neil Lawrence's video lecture on Introduction to Gaussian processes](https://www.youtube.com/watch?v=ewJ3AxKclOg).

## Motivation: A fully Bayesian paradigm for curve fitting

Gaussian process regression is Bayesian regression on steroids.
However, understanding how it works requires a change of mind.
After a bit of practice it starts making sense.

Here is how it works:

+ Let's say that you have to learn some function $f(\cdot)$ from some space $\mathcal{X}$ to $\mathbb{R}$ (this could either be a supervised learning problem (regression or classification) or even an unsupervised learning problem.

+ You sit down and you think about $f(\cdot)$. What do you know about it? How large do you expect it be? How small do you expect it be? Is it continuous? Is it differentiable? Is it periodic? How fast does it change as you change its inputs?

+ You create a probability measure on the space of functions in which $f(\cdot)$ lives which is compatible with everything you know about it. Abusing mathematical notation a lot, let's write this probability measure as $p(f(\cdot))$. Now you can sample from it. Any sample you take is compatible with your prior beliefs. You cannot tell which one is better than any other. Any of them could be the true $f(\cdot)$.

+ Then, you get a little bit of data, say $\mathcal{D}$. You model the likelihood of the data, $p(\mathcal{D}|f(\cdot))$, i.e., you model how the data may have been generated if you knew $f(\cdot)$.

+ Finally, you use Bayes' rule to come up with your posterior probability measure over the space of functions:
$$
p(f(\cdot)|\mathcal{D}) \propto p(\mathcal{D}|f(\cdot)) p(f(\cdot)),
$$
which is simultaneously compatible with your prior beliefs and the data.
Again, we are abusing mathematical notation here sinc you cannot really write down the probability density corresponding to a random function.
But you get the point.

This is it.
As Persi Diaconis' said in an 1988 paper:

> Most people, even Bayesians, think that this sounds crazy when they first hear about it.

Where do Gaussian processes come in?
Well, it is just the equivalent of the multivariate Gaussian for function spaces.
It defines a probability measure on the space of functions that is centered about a mean (function) and shaped by a covariance (function).
In this lecture, we will show that the mean function and the covariance function is the place where you can encode your prior knowledge.
We will also show how you can sample functions from this probability measure.
Next time, we will show how you can condition these probability measures on observations.

## Some mathematical terminology

A *stochatic process* is just a collection of random variables that are labeled by some index: $\{X_i\}$ for some $i$ in a set $I$.
If the set $I$ is discrete, then we say that we have a discrete stochastic process.
If the set $I$ is continuous, then we have a continuous stochastic process.
We will mostly work with continuous stochastic processes in this class.

For example, $X_t = X(t)$ is a stochastic process parameterized by time.
You can also think of a stochastic process as a random function.
Here is how, you sample an $\omega$, you keep it fixed, and then $X(t,\omega)$ is just a function of time. (Remember what we learned in earlier lectures: a random variable is just a function from the event space to the real numbers).

Of course, you can have a stochastic process that is parameterized by space, for example $T(x,\omega)$, could be an unknown temperature field.
This is typically called a random field.
You can also have a stochastic process that is parameterized by both space and time, say $T(x,t,\omega)$.
This is a unknown spatiotemporal temperature field.
As a matter of fact, you can have a stochastic process parameterized by any continuous label you like.
It does not have to be space or time.
And also, you can have as many different labels as you like.

Gaussian processes are the simplest continuous stochastic processes you can have.

## Gaussian process

A Gaussian process (GP) is a generalization of a multivariate Gaussian distribution to
*infinite* dimensions.
It essentially defines a probability measure on a function space.
When we say that $f(\cdot)$ is a GP, we mean that it is a random variable that is actually
a function.
Mathematically, we write:
\begin{equation}
f(\cdot) \sim \mbox{GP}\left(m(\cdot), k(\cdot, \cdot) \right),
\end{equation}
where
$m:\mathbb{R}^d \rightarrow \mathbb{R}$ is the *mean function* and
$k:\mathbb{R}^d \times \mathbb{R}^d \rightarrow \mathbb{R}$ is the *covariance function*.
So, compared to a multivariate normal we have:

+ A random function $f(\cdot)$ instead of a random vector $\mathbf{x}$.
+ A mean function $m(\cdot)$ instead of a mean vector $\boldsymbol{\mu}$.
+ A covariance function $k(\cdot,\cdot)$ instead of a covariance matrix $\mathbf{\Sigma}$.

But, what does this definition actually mean? Actually, it gets its meaning from the multivariate Gaussian distribution. Here is how:

+ Let $\mathbf{x}_{1:n}=\{\mathbf{x}_1,\dots,\mathbf{x}_n\}$ be $n$ points in $\mathbb{R}^d$.
+ Let $\mathbf{f}\in\mathbb{R}^n$ be the outputs of $f(\cdot)$ on each one of the elements of $\mathbf{x}_{1:n}$, i.e.,
$$
\mathbf{f} =
\left(
\begin{array}{c}
f(\mathbf{x}_1)\\
\vdots\\
f(\mathbf{x}_n)
\end{array}
\right).
$$
+ The fact that $f(\cdot)$ is a GP with mean and covariance function $m(\cdot)$ and $k(\cdot,\cdot)$, respectively, *means* that the vector of outputs $\mathbf{f}$ at
the arbitrary inputs in $\mathbf{X}$ is the following multivariate-normal:
$$
\mathbf{f} | \mathbf{x}_{1:n}, m(\cdot), k(\cdot, \cdot) \sim \mathcal{N}\left(\mathbf{m}(\mathbf{x}_{1:n}), \mathbf{K}(\mathbf{x}_{1:n}, \mathbf{x}_{1:n}) \right),
$$
with mean vector:
$$
\mathbf{m}(\mathbf{x}_{1:n}) =
\left(
\begin{array}{c}
m(\mathbf{x}_1)\\
\vdots\\
m(\mathbf{x}_n)
\end{array}
\right),
$$
and covariance matrix:
$$
\mathbf{K}(\mathbf{x}_{1:n},\mathbf{x}_{1:n}) = \left(
\begin{array}{ccc}
k(\mathbf{x}_1,\mathbf{x}_1) & \dots & k(\mathbf{x}_1, \mathbf{x}_n)\\
\vdots & \ddots & \vdots\\
k(\mathbf{x}_n, \mathbf{x}_1) & \dots & k(\mathbf{x}_n, \mathbf{x}_n)
\end{array}
\right).
$$

Now that we have defined a Gaussian process (GP), let us talk about we encode our prior beliefs into a GP.
We do so through the mean and covariance functions.

### Interpretation of the mean function

What is the meaning of $m(\cdot)$?
Well, it is quite easy to grasp.
For any point $\mathbf{x}\in\mathbb{R}^d$, $m(\mathbf{x})$ should give us the value we believe is more probable for
$f(\mathbf{x})$.
Mathematically, $m(\mathbf{x})$ is nothing more than the expected value of the random variable $f(\mathbf{x})$.
That is:
\begin{equation}
m(\mathbf{x}) = \mathbb{E}[f(\mathbf{x})].
\end{equation}

The mean function can be any arbitrary function. Essentially, it tracks generic trends in the response as the input is varied. In practice, we try and make a suitable choice for the mean function that is easy to work with. Such choices include:

+ zero, i.e.,
$$
m(\mathbf{x}) = 0.
$$

+ a constant, i.e.,
$$
m(\mathbf{x}) = c,
$$
where $c$ is a parameter.

+ linear, i.e.,
$$
m(\mathbf{x}) = c_0 + \sum_{i=1}^dc_ix_i,
$$
where $c_i, i=0,\dots,d$ are parameters.

+ using a set of $m$ basis functions (generalized linear model), i.e.,
$$
m(\mathbf{x}) = \sum_{i=1}^mc_i\phi_i(\mathbf{x}),
$$
where $c_i$ and $\phi_i(\cdot)$ are parameters and basis functions.

+ generalized polynomial chaos (gPC), i.e.,
using a set of $d$ polynomial basis functions upto a given degree $\rho$
$m(\mathbf{x}) = \sum_{i=1}^{d}c_i\phi_i(\mathbf{x})$
where the basis functions $\phi_i$ are mutually orthonormal with respect to some
measure $\mu$:
$$
\int \phi_{i}(\mathbf{x}) \phi_{j}(\mathbf{x}) d\mu(\mathbf{x}) = \delta_{ij}
$$

+ and many other possibilities.

### Interpretation of the covariance function

What is the meaning of $k(\cdot, \cdot)$?
This concept is considerably more challenging than the mean.

Let's try to break it down:

+ Let $\mathbf{x}\in\mathbb{R}^d$. Then $k(\mathbf{x}, \mathbf{x})$ is the variance of the random variable $f(\mathbf{x})$, i.e.,
$$
\mathbb{V}[f(\mathbf{x})] = \mathbb{E}\left[\left(f(\mathbf{x}) - m(\mathbf{x}) \right)^2 \right].
$$
In other words, we believe that there is about $95\%$ probability that the value of
the random variable $f(\mathbf{x})$ fall within the interval:
$$
\left((m(\mathbf{x}) - 2\sqrt{k(\mathbf{x}, \mathbf{x})}, m(\mathbf{x}) + 2\sqrt{k(\mathbf{x},\mathbf{x})}\right).
$$

+ Let $\mathbf{x},\mathbf{x}'\mathbb{R}^d$. Then $k(\mathbf{x}, \mathbf{x}')$ tells us how the random variable $f(\mathbf{x})$ and
$f(\mathbf{x}')$ are correlated. In particular, $k(\mathbf{x},\mathbf{x}')$ is equal to the covariance
of the random variables $f(\mathbf{x})$ and $f(\mathbf{x}')$, i.e.,
$$
k(\mathbf{x}, \mathbf{x}') = \mathbb{C}[f(\mathbf{x}), f(\mathbf{x}')]
= \mathbb{E}\left[
\left(f(\mathbf{x}) - m(\mathbf{x})\right)
\left(f(\mathbf{x}') - m(\mathbf{x}')\right)
\right].
$$

Essentially, a covariance function (or covariance kernel) defines a nearness or similarity measure on the input space. We cannot choose any arbitrary function of two variables as a covariance kernel. How we go about choosing a covariance function is discussed in great detail [here](http://www.gaussianprocess.org/gpml/chapters/RW4.pdf). We briefly discuss some properties of covariance functions here and then we shall move onto a discussion of what kind of prior beliefs we can encode through the covariance function.

### Properties of the covariance function

+ There is one property of the covariance function that we can note right away.
Namely, that for any $\mathbf{x}\in\mathbb{R}^d$, $k(\mathbf{x}, \mathbf{x}) > 0$.
This is easly understood by the interpretation of $k(\mathbf{x}, \mathbf{x})$ as the variance
of the random variable $f(\mathbf{x})$.

+ $k(\mathbf{x}, \mathbf{x}')$ becomes smaller as the distance between $\mathbf{x}$ and $\mathbf{x}'$ grows.

+ For any choice of points $\mathbf{X}\in\mathbb{R}^{n\times d}$, the covariance matrix: $\mathbf{K}(\mathbf{X}, \mathbf{X})$ has
to be positive-definite (so that the vector of outputs $\mathbf{f}$ is indeed a multivariate
normal distribution).

### Encoding prior beliefs in the covariance function

+ **Modeling regularity**. The choice of the covariance function controls the regularity properties of the functions sampled from the probability induced by the GP. For example, if the covariance kernel chosen is the squared exponential kernel, which is infinitely differentiable, then the functions sampled from the GP will also be infinitely differentiable.

+ **Modeling invariance** If the covariance kernel is invariant w.r.t. a transformation $T$, i.e., $k(\mathbf{x}, T\mathbf{x}')=k(T\mathbf{x}, \mathbf{x}')=k(\mathbf{x}, \mathbf{x}')$ then samples from the GP will be invariant w.r.t. the same transformation.

+ Other possibilities include periodicity, additivity etc.

### Squared exponential covariance function

Squared expnential (SE) is the most commonly used covariance function.
Its formula is as follows:
$$
k(\mathbf{x}, \mathbf{x}') = v\exp\left\{-\frac{1}{2}\sum_{i=1}^d\frac{(x_i - x_i')^2}{\ell_i^2}\right\},
$$
where $v,\ell_i>0, i=1,\dots,d$ are parameters.
The interpretation of the parameters is as follows:

+ $v$ is known as the *signal strength*. The bigger it is, the more the GP $f(\cdot)$ will vary
about the mean.

+ $\ell_i$ is known as the *length scale* of the $i$-th input dimension of the GP.
The bigger it is, the smoother the samples of $f(\cdot)$ appear along the $i$-th input dimension.

Let's experiment with this for a while:

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns # Comment this out if you don't have it
sns.set_style('white')
sns.set_context('talk')
import GPy
# The input dimension
dim = 1
# The variance of the covariance kernel
variance = 1.
# The lengthscale of the covariance kernel
ell = 0.3
# Generate the covariance object
k = GPy.kern.RBF(dim, variance=variance, lengthscale=ell)
# Print it
print (k)
# and plot it
k.plot();
```

### Example 1: Plotting a covariance function
Remember:
> The covariance function $k(x,x')$ measures the similarity of $f(x)$ and $f(x')$.

The interactive tools provided, draw $k(\mathbf{x}, \mathbf{x}'=0)$ in one and two dimensions.
Use them to answer the following questions:
+ What is the intuitive meaning of $\ell$?
+ What is the intuitive meaning of $v$?
+ There are many other covariance functions that we could be using. Try changing ``RBF`` to ``Exponential``. What changes do you nottice.
+ Repeat the previous steps on a 2D covariance function.
+ If you still have time, try a couple of other covariances, e.g., ``Matern32``, ``Matern52``.
+ If you still have time, explore ``help(GPy.kern)``.

```python
from ipywidgets import interactive
def plot_kernel(variance=1., ell=0.3):
    k = GPy.kern.RBF(dim, variance=variance, lengthscale=ell)
    k.plot()
    plt.ylim(0, 10)
interactive(plot_kernel, variance=(1e-3, 10., 0.01), ell=(1e-3, 10., 0.01))
```

    interactive(children=(FloatSlider(value=1.0, description='variance', max=10.0, min=0.001, step=0.01), FloatSli…

```python
from ipywidgets import interactive
def plot_kernel(variance=1., ell1=0.3, ell2=0.3):
    k = GPy.kern.RBF(2, ARD=True, variance=variance,
                     lengthscale=[ell1, ell2])  # Notice that I just changed the dimension here
    k.plot()
interactive(plot_kernel, variance=(1e-3, 10., 0.01), ell1=(1e-3, 10., 0.01), ell2=(1e-3, 10., 0.01))
```

    interactive(children=(FloatSlider(value=1.0, description='variance', max=10.0, min=0.001, step=0.01), FloatSli…

### Example 2: Properties of the covariance matrix
Let $\mathbf{x}_{1:n}$ be an arbitrary set of input points. The covariance matrix $\mathbf{K}\in\mathbb{R}^{n\times n}$ defined by:
$$
\mathbf{K}\equiv\mathbf{K}(\mathbf{x}_{1:n}, \mathbf{x}_{1:n}) = \left(
\begin{array}{ccc}
k(\mathbf{x}_1,\mathbf{x}_1) & \dots & k(\mathbf{x}_1, \mathbf{x}_n)\\
\vdots & \ddots & \vdots\\
k(\mathbf{x}_n, \mathbf{x}_1) & \dots & k(\mathbf{x}_n, \mathbf{x}_n)
\end{array}
\right),
$$
must be [positive definite](https://en.wikipedia.org/wiki/Positive-definite_matrix). Mathematically this can be expressed in two equivalent ways:

+ For all vectors $\mathbf{v}\in\mathbb{R}^T$, we have:
$$
\mathbf{v}^t\mathbf{K}\mathbf{v} > 0,
$$
+ All the eigenvalues of $\mathbf{K}$ are positive.

Using the code provided:
+ Verify that the the sum of two covariance functions is a valid covariance function.
+ Verify that the product of two covariance functions is a valid covariance function.
+ Is the following function a covariance function:
$$
k(x, x') = k_1(x, x')k_2(x, x') + k_3(x, x') + k_4(x, x'),
$$
where all $k_i(x, x')$'s are covariance functions.
+ What about:
$$
k(x, x') = k_1(x, x') / k_2(x, x')?
$$

```python
# Number of dimensions
dim = 1

# Number of input points
n = 20

# The lengthscale
ell = .1

# The variance
variance = 1.

# The covariance function
k1 = GPy.kern.RBF(dim, lengthscale=ell, variance=variance)

# Draw a random set of inputs points in [0, 1]^dim
X = np.random.rand(n, dim)

# Evaluate the covariance matrix on these points
K = k1.K(X)

# Compute the eigenvalues of this matrix
eig_val, eig_vec = np.linalg.eigh(K)

# Plot the eigenvalues (they should all be positive)
print ('> plotting eigenvalues of K')
print ('> they must all be positive')
fig, ax = plt.subplots()
ax.plot(np.arange(1, n+1), eig_val, '.')
ax.set_xlabel('$i$', fontsize=16)
ax.set_ylabel('$\lambda_i$', fontsize=16)
```

```python
# Now create another (arbitrary) covariance function
k2 = GPy.kern.Exponential(dim, lengthscale=0.2, variance=2.1)

# Create a new covariance function that is the sum of these two:
k_new = k1 + k2

# Let's plot the new covariance
fig, ax = plt.subplots()
k1.plot(ax=ax, label='$k_1$')
k2.plot(ax=ax, label='$k_2$')
k_new.plot(ax=ax, label='$k_1 + k_2$')
plt.legend(fontsize=16);
```

```python
# If this is a valid covariance function, then it must
# be positive definite
# Compute the covariance matrix:
K_new = k_new.K(X)

# and its eigenvalues
eig_val_new, eig_vec_new = np.linalg.eigh(K_new)

# Plot the eigenvalues (they should all be positive)
print ('> plotting eigenvalues of K')
print ('> they must all be positive')
fig, ax = plt.subplots()
ax.plot(np.arange(1, n+1), eig_val_new, '.')
ax.set_xlabel('$i$', fontsize=16)
ax.set_ylabel('$\lambda_i$', fontsize=16);
```

### Example 3: Sampling from a Gaussian process

Samples from a Gaussian process are functions. But, functions are infinite dimensional objects?
We cannot sample directly from a GP....
However, if we are interested in the values of $f(\cdot)$ at any given set of test points $\mathbf{x}_{1:n} = \{\mathbf{x}_1,\dots,\mathbf{x}_b\}$, then we have that:
$$
\mathbf{f} | \mathbf{x}_{1:n} \sim \mathcal{N}\left(\mathbf{m}(\mathbf{x}_{1:n}), \mathbf{K}(\mathbf{x}_{1:n}, \mathbf{x}_{1:n}) \right),
$$
where all the quantities have been introduced above.
This is
What we are going to do is pick a dense set of points $\mathbf{x}_{1:n}\in\mathbb{R}^{n\times d}$
sample the value of the GP, $\mathbf{f} = (f(\mathbf{x}_1),\dots,f(\mathbf{x}_n))$ on these points.
We saw above that the probability density of $\mathbf{f}$ is just a multivariate normal
with a mean vector that is specified from the mean function and a covariance matrix
that is specified by the covariance function.
Therefore, all we need to know is how to sample from the multivariate normal.
This is how we do it:
+ Compute the Cholesky of $\mathbf{L}$:
$$
\mathbf{K} = \mathbf{L}\mathbf{L}^T.
$$
+ Draw $n$ random samples $\mathbf{z} = (z_1,\dots,z_n)$ independently from a standard normal.
+ Get one sample by:
$$
\mathbf{f} = \mathbf{m} + \mathbf{L}\mathbf{z}.
$$

```python
# To gaurantee reproducibility
np.random.seed(123456)

# Number of test points
num_test = 10

# Pick a covariance function
k = GPy.kern.RBF(input_dim=1, variance=1., lengthscale=.1)

# Pick a mean function
mean_func = lambda x: np.zeros(x.shape)

# Pick a bunch of points over which you want to sample the GP
X = np.linspace(0, 1, num_test)[:, None]

# Evaluate the mean function at X
m = mean_func(X)

# Compute the covariance function at these points
nugget = 1e-6 # This is a small number required for stability
C = k.K(X) + nugget * np.eye(X.shape[0])

# Compute the Cholesky of the covariance
# Notice that we need to do this only once
L = np.linalg.cholesky(C)

# Number of samples to take
num_samples = 3

# Take 3 samples from the GP and plot them:
fig, ax = plt.subplots()
# Plot the mean function
ax.plot(X, m)
for i in range(num_samples):
    z = np.random.randn(X.shape[0], 1)    # Draw from standard normal
    f = m + np.dot(L, z)                  # f = m + L * z
    ax.plot(X, f, color=sns.color_palette()[1], linewidth=1)
#ax.set_ylim(-6., 6.)
ax.set_xlabel('$x$', fontsize=16)
ax.set_ylabel('$y$', fontsize=16)
ax.set_ylim(-5, 5);
```

The solid line is the mean function and the dashed lines are 3 samples of   f . These don’t look like functions yet. This is because we have used only 10 test points to represent the GP.

### Questions

1. Edit the code above changing the number of test points ``num_test`` to 20, 50, 100. Rerun the example. How do your samples of   f look like now? Do they look more like functions to you? Imagine that the true nature of the GP appears when these test points become infinitely dense.

2. Edit the code above and change the random seed to an arbitrary integer (just make up one). Rerun the example and notice how the sampled functions change.

3. Edit the code above and change the variance first to 0.1 and then to 5 each time rerunning the example. Notice the values on the vertical axis of the plot. What happens to the sampled functions as you do this? What does the variance parameter of the SE control?

4. Edit the code above and now change the length-scale parameter first to 0.05 and then to 1. What happens to the sampled functions as you do this? What does the length- scale parameter of the SE control?

5. Now set the variance and the length-scale back to their original values (1. and 0.1, respectively). Edit the code and change the mean function to:
```
     mean_fun = lambda(x): 5 * x
```
Re-run the example. What do you observe? Try a couple more. For example, try:
```
     mean_fun = lambda(x): np.sin(5 * np.pi * x)
```

6. So far, all the samples we have seen are smooth. There is this theorem that says that the samples of the GP will be as smooth as the covariance function we use. Since the SE covariance is infinitely smooth, the samples we draw are infinitely smooth. The [Matern 3-2 covariance function](https://en.wikipedia.org/wiki/Matérn_covariance_function) is twice differentiable. Edit the code and
change ``RBF`` to ``Matern32``. Rerun the example. How smooth are the samples now?

7. The exponential covariance function is continuous but not differentiable. Edit the code and change ``RBF`` to ``Exponential``. Rerun the example. How smooth are the samples now?

8. The covariance function can also be used to model invariances. The periodic exponential covariance function is... a periodic covariance function. Edit line 29 and change ``RBF`` to
```
k = GPy.kern.PeriodicMatern32(input_dim=1, variance=500., lengthscale=0.01, period=0.1)
```
Rerun the example. Do you notice the periodic pattern?

9. How can you encode the information that there are two lengthscales in $f(\cdot)$. There are many ways to do this.
Try summing or multiplying covariance functions.


# Class

# LCR in series

Resonating frequency
\begin{equation}
f_0=\frac{1}{2\pi\sqrt{LC}}
\end{equation}

Quality factor
\begin{equation}
Q=\frac{1}{R}\sqrt{\frac{L}{C}}
\end{equation}

Inductive reactance
\begin{equation}
X_L=2\pi fL
\end{equation}

Capacitive reactance
\begin{equation}
X_C=\frac{1}{2\pi fC}
\end{equation}

Impedance
\begin{equation}
z=\sqrt{(X_L-X_C)^2+R^2}
\end{equation}

Phase angle
\begin{equation}
\phi=\tan^{-1}(\frac{X_L-X_C}{R})
\end{equation}

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
class lcr():
    def __init__(self,L,C,R,f):
        self.L     = L
        self.C     = C
        self.R     = R
        self.f     = f

    def fo(self):
        fr=(2*np.pi*np.sqrt(self.L*self.C))**-1
        return fr

    def Q(self):
        ql=np.sqrt(self.L/self.C)/self.R
        return ql

    def inductiveReactance(self):
        XL=2*np.pi*self.L*self.f
        return XL

    def capacitiveReactance(self):
        XC=1/(2*np.pi*self.C*self.f)
        return XC

    def impedance(self):
        imp=np.sqrt((self.inductiveReactance()-self.capacitiveReactance())**2+self.R**2)
        return imp

    def phase(self):
        pi=np.arctan((self.inductiveReactance()-self.capacitiveReactance())/self.R)*180/np.pi
        return pi    # degree

```

```python
l=0.1    #inductance in H
c=1e-6   #capacitance in F
r=10     # resistance in ohm
v=100    #voltage in V
```

```python
d=lcr(l,c,r,100)
print('Resonating frequency:',d.fo(),'Hz')
print('Q factor:',d.Q())
```

    Resonating frequency: 503.2921210448704 Hz
    Q factor: 31.622776601683796

```python
freq=np.arange(10,1500,10)
g=lcr(l,c,r,freq)
xl=g.inductiveReactance()
xc=g.capacitiveReactance()
z=g.impedance()
ph=g.phase()
I=v/z  # current
vr=I*r  #p.d.across reisitor
vl=I*xl  #p.d.across inductor
vc=I*xc  #p.d.across capacitor
power =v*I*np.cos(ph*np.pi/180)
```

```python
plt.plot(freq,xl,'m.',label='Inductive Reactance')
plt.plot(freq,xc,'c-.',label='Capacitive Reactance')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="Reactance(ohm)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.legend()
plt.tight_layout()
plt.savefig("image\LCR1.pdf", dpi = 600) # dpi dot per inch
plt.show()
plt.close()
```

```python
plt.plot(freq,vr,'g-',label='p.d. across R')
plt.plot(freq,vl,'b-',label='p.d. across L')
plt.plot(freq,vc,'r-',label='p.d. across C')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="P.d.(V)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.legend()
plt.tight_layout()
plt.savefig("image\LCR2.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
plt.plot(freq,z,'m^')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="Impedance(ohm)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.tight_layout()
plt.savefig("image\LCR3.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
plt.plot(freq,I,'c*')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="Current(A)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.tight_layout()
plt.savefig("image\LCR4.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
plt.plot(freq,np.cos(ph*np.pi/180),'k+')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="power factor(unitless)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.tight_layout()
plt.savefig("image\LCR7.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
plt.plot(freq,ph,'k+')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="phase angle(degree)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.tight_layout()
plt.savefig("image\LCR5.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
plt.plot(freq,power,'yo')
plt.xlabel(xlabel="Frequency(Hz)",fontsize=14,fontstyle='italic',color="black")
plt.ylabel(ylabel="Power(W)",fontsize=14,fontstyle='italic',color="black")
plt.title(label="LCR circuit",fontsize=20,fontweight='bold',color="red")
plt.tight_layout()
plt.savefig("image\LCR6.pdf", dpi = 600)
plt.show()
plt.close()
```

```python
data={}
data.update({'Freqency':freq,'in.rect.':xl,'cap.Rect':xc,'imp.':z,'curren':I,'phase':ph,'power':power})
```

```python
DF = pd.DataFrame(data)
DF
```

      Freqency
      in.rect.
      cap.Rect
      imp.
      curren
      phase
      power

      0
      10
      6.283185
      15915.494309
      15909.214267
      0.006286
      -89.963986
      0.000395

      1
      20
      12.566371
      7957.747155
      7945.187077
      0.012586
      -89.927886
      0.001584

      2
      30
      18.849556
      5305.164770
      5286.324672
      0.018917
      -89.891615
      0.003578

      3
      40
      25.132741
      3978.873577
      3953.753482
      0.025292
      -89.855085
      0.006397

      4
      50
      31.415927
      3183.098862
      3151.698800
      0.031729
      -89.818206
      0.010067

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      144
      1450
      911.061870
      109.762030
      801.362236
      0.124788
      89.285002
      0.155719

      145
      1460
      917.345055
      109.010235
      808.396673
      0.123702
      89.291224
      0.153021

      146
      1470
      923.628240
      108.268669
      815.420892
      0.122636
      89.297330
      0.150396

      147
      1480
      929.911425
      107.537124
      822.435099
      0.121590
      89.303323
      0.147842

      148
      1490
      936.194611
      106.815398
      829.439497
      0.120563
      89.309206
      0.145355

149 rows × 7 columns

```python
DF.to_csv("data/lcr.csv")  #save data in csv format in excel
```

```python

```


# Lab 5: RNNs for Time Series Prediction

------------------------------------------------------
*Deep Learning. Master in Information Health Engineering*

*Pablo M. Olmos pamartin@ing.uc3m.es*

------------------------------------------------------

In this notebook, you will deploy a simple probabilistic model using a RNN to infer the probability distribution of a time-series. Given a set of signals $\mathcal{D}=\{X^1,X_2,\ldots,X_N\}$ where
$$X^i = [X_0^i,X_1^i,\ldots,X_T^i]$$
is the $i$-th signal, we will train a probabilistic model of the form
$$p(X|X_0) = \prod_{t=0}^T p(X_t|X_{t-1},X_{:t-1})$$
where each factor is a Gaussian distribution with variance $\sigma^2$ and mean given by **the ouput of a RNN** with input $X_{t-1}$. Hence $X_{:t-1}$ in the conditional probability $p(X_t|X_{t-1},X_{:t-1})$ is embedded through the **RNN** state $\mathbf{h}_{t-1}$:

$$p(X_t|X_{t-1},X_{:t-1}) = \mathcal{N}\left(f_{RNN}(X_{t-1},\mathbf{h}_{t-1}),\sigma^2\right)$$

During training, for $t=1,\ldots,T$, we will sample $\hat{X}_t$ from $p(X_t|X_{t-1},X_{:t-1})$ and minimize the average square loss $\frac{1}{T}\sum_{t=1}^T(X_t-\hat{X}_t)^2$. Then we average again for all signals in the training set. Note that during training we **feed the RNN with the true values of the signal** $X^i = [X_0^i,X_1^i,\ldots,X_T^i]$.  This strategy is called [**teaching forcing**](https://papers.nips.cc/paper/2016/file/16026d60ff9b54410b3435b403afd226-Paper.pdf).

```python
import torch
from torch import nn
from torch import optim
import numpy as np
import time
import matplotlib.pyplot as plt

%matplotlib inline
%config InlineBackend.figure_format = 'retina'  #To get figures with high quality!

```

## Part I. Create a synthetic databse

We will generate $N$ target signals of length $T$ time steps. We generate each signal as one realization of the following autoregressive model
\begin{align}
X_{t}=c+\sum_{i=1}^{p} \varphi_{i} X_{t-i}+\varepsilon_{t}
\end{align}

```python
N = 1000 # Number of signals

T = 200

c = 0

phi_1 = 1

phi_2 = -1

phi_3 = 1

sigma = 1

X = np.zeros([N,T])

np.random.seed(23)

X[:,0] = c + np.random.randn(N,)*np.sqrt(sigma)

X[:,1] = c + phi_1 * X[:,0] + np.random.randn(N,)*np.sqrt(sigma)

X[:,2] = c + phi_1 * X[:,1] + phi_2 * X[:,0] + np.random.randn(N,)*np.sqrt(sigma)

X[:,3] = c + phi_1 * X[:,2] + phi_2 * X[:,1] + phi_3 * X[:,0] + np.random.randn(N,)*np.sqrt(sigma)

t = 4

while (t<T):

    X[:,t] = c + phi_1 * X[:,t-1] + phi_2 * X[:,t-2] + phi_3 * X[:,t-3] + np.random.randn(N,)*np.sqrt(sigma)

    t +=1

# Create targets

Y = X[:,1:] # all but the first
X = X[:,:-1] # all but the last piece of data

T -=1
```

The goal of the RNN will be to predict the value of the signal in the next time point given the current observation. Note that the noise in the model

$$p(X_t|X_{t-1},X_{:t-1}) = \mathcal{N}\left(f_{RNN}(X_{t-1},\mathbf{h}_{t-1}),\sigma^2\right)$$

will simply introduce an error that will prevent the model from overfitting.

Lets plot one of the signals versus the *target*, which is the same signal but shifted to the right ...

```python
# Plot the signal
plt.figure(figsize=(8,5))
plt.plot(np.arange(T), X[1,:T], 'r.--', label='input, x',ms=10) # x
plt.plot(np.arange(T), Y[1,:T], 'b.-', label='target, y',ms=10) # y

plt.legend(loc='best')

# Plot the signal (20 first steps)
plt.figure(figsize=(8,5))
plt.plot(np.arange(20), X[1,:20], 'r.--', label='input, x',ms=10) # x
plt.plot(np.arange(20), Y[1,:20], 'b.-', label='target, y',ms=10) # y

plt.legend(loc='best')
```

## Part II. Define RNN

Next, we define an RNN in PyTorch. We'll use `nn.RNN` to create an RNN layer, which takes in a number of parameters:
* **input_size** - the size of the input
* **hidden_dim** - the dimension of the RNN output and the hidden state
* **n_layers** - the number of layers that make up the RNN, typically 1-3; greater than 1 means that you'll create a **stacked RNN**

This is an example of a stacked RNN

If you take a look at the [RNN documentation](https://pytorch.org/docs/stable/nn.html#rnn), you will see that `nn.RNN` only provides the actual computation of the hidden states along time
\begin{align}
h_{t}=g \left(W_{i h} x_{t}+b_{i h}+W_{h h} h_{(t-1)}+b_{h h}\right)
\end{align}

Then we'll add a last, fully-connected layer to get the output size that we want. For simplicity, **the input to this dense layer is the state $h_t$ of the RNN**.

You have to pay special attention to the dimensions of the input/output tensors of the RNN. **Check the [RNN documentation](https://pytorch.org/docs/stable/nn.html#rnn)**.

The following class implements a class where
- An input signal of dimension `input_size` is processed by a RNN. As a result, we obtain a sequence of states $\mathbf{h}_{t}$, from $t=1$ to $t=T$.
- We process each state with a linear layer to estimate the output signal (of dimension `output_size`) at time $t$ from $\mathbf{h}_{t}$.
- We sum a Gaussian noise of variance `sigma` to the output of the linear layer.

> **Exercise**: complete the following code. Understand all steps, particularly those in the `forward` method.

```python
class RNN(nn.Module):
    def __init__(self, input_size, output_size, hidden_dim, n_layers,sigma):

        # input size -> Dimension of the input signal
        # outpusize -> Dimension of the output signal
        # hidden_dim -> Dimension of the rnn state
        # n_layers -> If >1, we are using a stacked RNN

        super().__init__()

        self.hidden_dim = hidden_dim

        self.input_size = input_size

        self.sigma = sigma

        # define an RNN with specified parameters
        # batch_first=True means that the first dimension of the input will be the batch_size
        self.rnn = nn.RNN(input_size=input_size, hidden_size=hidden_dim, num_layers=n_layers,
                          nonlinearity='relu',batch_first=True)

        # last, fully-connected layer
        self.fc1 = nn.Linear(# YOUR CODE HERE)

    def forward(self, x, h0=None):

        '''
        About the shape of the different tensors ...:

        - Input signal x has shape (batch_size, seq_length, input_size)
        - The initialization of the RNN hidden state h0 has shape (n_layers, batch_size, hidden_dim).
          If None value is used, internally it is initialized to zeros.
        - The RNN output (batch_size, seq_length, hidden_size). This output is the RNN state along time

        '''

        batch_size = x.size(0) # Number of signals N
        seq_length = x.size(1) # T

        # get RNN outputs
        # r_out is the sequence of states
        # hidden is just the last state (we will use it for forecasting)
        r_out, hidden = self.rnn(x, h0)

        # shape r_out to be (seq_length, hidden_dim) #UNDERSTANDING THIS POINT IS IMPORTANT!!
        r_out = r_out.reshape(-1, self.hidden_dim)

        output = self.fc1(r_out)

        noise = torch.randn_like(output)*sigma

        output += noise

        # reshape back to temporal structure
        output = output.reshape([-1,seq_length,1])

        return output, hidden

```

> **Exercise:** Instantiate the object RNN with the right parameters for our problem. Use `hidden_dim=32`, `n_layers=1` and `sigma=1`

```python
# YOUR CODE HERE
```

In the following code, we compute the model output using the `forward` method. Note that we use an all zero initial state.

> **Exercise**: Complete the following code. What are the dimensions of variables `h` and `o`? How are these dimensions related to the number of signals, hidden state of the RNN and signal duration?

```python
X_in = torch.Tensor(X).view([-1,T,1])

o,h = test_rnn.forward(# YOUR CODE HERE)

print(h.shape)

print(o.shape)

```

> **Exercise:** Complete the code for the following class, which extends `RNN` to include a training method.

Note that there is no mini-batch, we process all signals for every SGD iteration. You are free to the mini-batch training functionality.

```python
class RNN_extended(RNN):

    #Your code here

    def __init__(self, num_data_train, num_iter, sequence_length,
                 input_size, output_size, hidden_dim, n_layers, sigma, lr=0.001):

        super().__init__(input_size, output_size, hidden_dim, n_layers,sigma)

        self.hidden_dim = hidden_dim

        self.sequence_length = sequence_length

        self.num_layers = n_layers

        self.lr = lr #Learning Rate

        self.num_train = num_data_train #Number of training signals

        self.optim = optim.Adam(self.parameters(), self.lr)

        self.num_iter = num_iter

        self.criterion = #YOUR CODE HERE

        # A list to store the loss evolution along training

        self.loss_during_training = []

    def trainloop(self,x,y):

        # SGD Loop

        for e in range(int(self.num_iter)):

            self.optim.zero_grad()

            x = torch.Tensor(x).view([self.num_train,#YOUR CODE HERE ,1])

            y = torch.Tensor(y).view([self.num_train,#YOUR CODE HERE ,1])

            out,hid = self.forward(x)

            loss = self.criterion(#YOUR CODE HERE)

            loss.backward()

            # This code helps to avoid vanishing exploiting gradients in RNNs
            # `clip_grad_norm` helps prevent the exploding gradient problem in RNNs / LSTMs.
            nn.utils.clip_grad_norm_(self.parameters(), 2.0)

            self.optim.step()

            self.loss_during_training.append(loss.item()/self.num_iter)

            if(e % 50 == 0): # Every 10 iterations

                print("Iteration %d. Training loss: %f" %(e,self.loss_during_training[-1]))
```

> **Exercise:** Using only the first 100 values of every signal, train the RNN for 100 SGD iterations. Use `hidden_dim=32`, `n_layers=1`,`lr=0.005`, and `sigma=1`. Recall that the target signal is stored in the variable `Y`.

```python
T_train = 100

my_rnn = RNN_extended(num_data_train=X.shape[0],num_iter=200,sequence_length=T_train,
                     input_size=1,output_size=1,hidden_dim=32,n_layers=1,sigma=1.0,lr=0.005)
```

```python
my_rnn.trainloop(X[:,:T_train],Y[:,:T_train])
```

> **Exercise:** Plot the loss function along training

```python
plt.plot(my_rnn.loss_during_training,label='Training Loss')
plt.legend()
```

> **Exercise:** Compute the following plot, in which we plot one of the input signals, the target one, and the predicted by the RNN from t=50 to t=100

```python
# We first evaluate the model for the N signals up to time T_train = 1000
X_in = torch.Tensor(X[:,:T_train]).view(# YOUR CODE HERE)

o,h = my_rnn.forward(# YOUR CODE HERE)

output_rnn = o.detach().numpy().reshape([N,-1])

offset = 50

signal = 0 # From 1 to N (you can play with this)

# Plot the first training signal and the target
plt.figure(figsize=(8,5))
plt.plot(np.arange(T_train-offset,T_train,1), X[signal,T_train-offset:T_train], 'r.--', label='input, x',ms=10) # x
plt.plot(np.arange(T_train-offset,T_train,1), Y[signal,T_train-offset:T_train], 'b.-', label='target, y',ms=10) # x
plt.plot(np.arange(T_train-offset,T_train,1), output_rnn[signal,T_train-offset:T_train], 'k.-', label='RNN output',ms=10) # y

plt.legend(loc='best')
```

Observe that the prediction is pretty good!The RNN model have clearly learnt the dynamics of the dataset. In the previous experiment, note we have fed the RNN model with the **true** values of the signal, i.e. we have use the full signal $X$ to compute the sequence of states.

Using the model we have just trained, lets do now **forecasting**. Namely, we feed the RNN the output that we predicted and we do this recursively for as long as we want. This represents **sampling** from the probabilistic model

$$p(X|X_{:T_{train}}) = \prod_{t=T_{train}}^T \mathcal{N}\left(f_{RNN}(X_{t-1},\mathbf{h}_{t-1}),\sigma^2\right)$$

To do forecasting, note that we have to recursively call the `forward` method and feed the obtained RNN output and state as the entry and initial state for the next `forward` call. The following code would do the job:

```python
# We take the last RNN output
current_input = o[:,-1,:].view([N,1,1]) #Note that current input only contains one observation for each of the N signals
# We take the last RNN state
current_state = h

forecast_rnn = np.zeros([N,T-T_train])

for t in range(T-T_train):

    # ... and feed them as input and initial state

    current_input,current_state = my_rnn.forward(current_input,current_state)

    forecast_rnn[:,t] = current_input.detach().numpy().reshape([-1,])

final_rnn_reconstruct = np.hstack([output_rnn,forecast_rnn])

# We plot the signal and the target before and after forecasting

plt.plot(np.arange(0,T-1,1), Y[signal,:-1].reshape([-1]), 'b.-', label='target, y',ms=10)
plt.plot(np.arange(0,T-1,1), final_rnn_reconstruct[signal,:-1], 'g.-', label='RNN output',ms=10)
plt.plot([T_train,T_train],[np.min(Y[signal,:]),np.max(Y[signal,:])],'k--')
plt.legend()

print('Between t=0 and t=100, we feed the real values')
print('From t=100, we feed the estimated values (forecasting)')
```

Observe that, during forecasting, sometimes the model quickly diverges from the target. That is **expected** we are sampling from the generative model and it is likely that do not get exactly the same sample! Particularly in RNN, since they have short memory.

## LSTMs

Lets study how an LSTM would perform in this context. You can create a basic [LSTM layer](https://pytorch.org/docs/stable/nn.html#lstm) as follows

```python
self.lstm = nn.LSTM(input_size, n_hidden, n_layers,
                            dropout=drop_prob, batch_first=True)
```

where `input_size` is the number of characters this cell expects to see as sequential input, and `n_hidden` is the number of units in the hidden layers in the cell. If **stacked LSTMs (n_layers>1) are used** we can automatically add dropout between LSTM layers with te parameter `dropout` with a specified probability.

> **Exercise:** Complete the code for the following two classes

```python
class LSTM(nn.Module):
    def __init__(self, input_size, output_size, hidden_dim, n_layers,sigma,drop_prob):

        # input size -> Dimension of the input signal
        # outpusize -> Dimension of the output signal
        # hidden_dim -> Dimension of the rnn state
        # n_layers -> If >1, we are using a stacked RNN

        super().__init__()

        self.hidden_dim = hidden_dim

        self.input_size = input_size

        self.sigma = sigma

        # define an RNN with specified parameters
        # batch_first=True means that the first dimension of the input will be the batch_size
        self.lstm = nn.LSTM(input_size, hidden_dim, n_layers, dropout=drop_prob, batch_first=True)

        # last, fully-connected layer
        self.fc1 = # YOUR CODE HERE

    def forward(self, x, h0=None, valid=False):

        '''
        About the shape of the different tensors ...:

        - Input signal x has shape (batch_size, seq_length, input_size)
        - The initialization of the LSTM hidden state is a tuple, containing two tensors of dimensions
          (n_layers, batch_size, hidden_dim) each. The first tensor represents the LSTM hidden state
          cell states. We can use the None value so internally they are initialized with 0s.
        - The LSTM output shape is (batch_size, seq_length, hidden_size)

        '''

        # If we use stacked LSTMs, we have to control the evaluation mode due to the dropout between LSTMs
        if(valid):
            self.eval()
        else:
            self.train()

        batch_size = x.size(0) # Number of signals N
        seq_length = x.size(1) # T

        # get RNN outputs
        # r_out is the sequence of states
        # hidden is just the last state (we will use it for forecasting)

        r_out, hidden = self.lstm(x, h0)

        # shape r_out to be (seq_length, hidden_dim)
        r_out = r_out.reshape(-1, self.hidden_dim)

        output = self.fc1(# YOUR CODE HERE)

        noise = torch.randn_like(output)*sigma

        output += noise

        # reshape back to temporal structure
        output = output.reshape([-1,seq_length,1])

        return output, hidden

```

```python
class LSTM_extended(LSTM):

    #Your code here

    def __init__(self, num_data_train, num_iter, sequence_length,
                 input_size, output_size, hidden_dim, n_layers, sigma, drop_prob=0.3, lr=0.001):

        super().__init__(input_size, output_size, hidden_dim, n_layers,sigma,drop_prob)

        self.hidden_dim = hidden_dim

        self.sequence_length = sequence_length

        self.num_layers = n_layers

        self.lr = lr #Learning Rate

        self.num_train = num_data_train #Number of training signals

        self.optim = optim.Adam(self.parameters(), self.lr)

        self.num_iter = num_iter

        self.criterion = #YOUR CODE HERE

        # A list to store the loss evolution along training

        self.loss_during_training = []

    def trainloop(self,x,y):

        # SGD Loop

        for e in range(int(self.num_iter)):

            self.optim.zero_grad()

            x = #YOUR CODE HERE

            y = #YOUR CODE HERE

            out,hid = #YOUR CODE HERE

            loss = #YOUR CODE HERE

            loss.backward()

            # This code helps to avoid vanishing exploiting gradients in RNNs
            # `clip_grad_norm` helps prevent the exploding gradient problem in RNNs / LSTMs.
            nn.utils.clip_grad_norm_(self.parameters(), 2.0)

            self.optim.step()

            self.loss_during_training.append(loss.item()/self.num_iter)

            if(e % 50 == 0): # Every 10 iterations

                print("Iteration %d. Training loss: %f" %(e,self.loss_during_training[-1]))
```

> **Exercise:** Train the LSTM model for 500 iterations using the first 100 values of each signal. Use `hidden_dim=32`, `n_layers=1`, `lr=0.005`, and `sigma=1`. Recall that the target signal is stored in the variable `Y`.
Note that with only one layer, the dropout probability parameter does not play any role (you will get a warning actually).

```python
# YOUR CODE HERE
```

> **Exercise:** Complete the code to visualize the LSTM prediction of the next value of the signal

```python
# We first evaluate the model for the N signals up to time T_train = 1000
X_in = # YOUR CODE HERE

o,h = my_lstm.forward(# YOUR CODE HERE)

output_lstm = o.detach().numpy().reshape([N,-1])

offset = 50

signal = 6 # From 1 to N (you can play with this)

# Plot the first training signal and the target
plt.figure(figsize=(8,5))
plt.plot(np.arange(T_train-offset,T_train,1), X[signal,T_train-offset:T_train], 'r.--', label='input, x',ms=10) # x
plt.plot(np.arange(T_train-offset,T_train,1), Y[signal,T_train-offset:T_train], 'b.-', label='target, y',ms=10) # x
plt.plot(np.arange(T_train-offset,T_train,1), output_lstm[signal,T_train-offset:T_train], 'k.-', label='LSTM output',ms=10) # y

plt.legend(loc='best')
```

> **Exercise:** Complete the code to visualize the LSTM forecasting. Plot the LSTM vs RNN vs target for a few signals and discuss the results.

```python
# We take the last RNN output
current_input = o[:,-1,:].view([N,1,1]) #Note that current input only contains one observation for each of the N signals
# We take the last RNN state
current_state = h

forecast_lstm = np.zeros([N,T-T_train])

for t in range(T-T_train):

    # ... and feed them as input and initial state

    current_input,current_state = # YOUR CODE HERE

    forecast_lstm[:,t] = current_input.detach().numpy().reshape([-1,])

final_lstm_reconstruct = np.hstack([output_lstm,forecast_lstm])

# We plot the signal and the target before and after forecasting

signal = 6

plt.plot(np.arange(0,T-1,1), Y[signal,:-1].reshape([-1]), 'b.-', label='target, y',ms=10)
plt.plot(np.arange(0,T-1,1), final_lstm_reconstruct[signal,:-1], 'g.-', label='LSTM output',ms=10)
plt.plot(np.arange(0,T-1,1), final_rnn_reconstruct[signal,:-1], 'r-', label='RNN output',ms=10)
plt.plot([T_train,T_train],[np.min(Y[signal,:]),np.max(Y[signal,:])],'k--')
plt.legend()

print('Between t=0 and t=100, we feed the real values')
print('From t=100, we feed the estimated values (forecasting)')
```

You should observe that the LSTM is able to keep track of the real signal during forecasting for a longer period of time.

> **Exercise (Optional):** Train an LSTM with 3 layers and evaluate the model.

```python
# YOUR CODE HERE
```


# Jacobian Transpose Inverse Kinematics and Control

In this notebook, we'll demonstrate an extremely simple approach for computing basic inverse kinematics (IK) and controlling the position of some point on our robot using the Jacobian transpose.

For a brief technical introduction, see <https://groups.csail.mit.edu/drl/journal_club/papers/033005/buss-2004.pdf> or <https://homes.cs.washington.edu/~todorov/courses/cseP590/06_JacobianMethods.pdf>

```julia
using Pkg
Pkg.activate(@__DIR__)
pkg"instantiate"
pkg"precompile"
```

      Updating registry at `~/.julia/registries/General`
      Updating git-repo `https://github.com/JuliaRegistries/General.git`
    Precompiling project...

```julia
using RigidBodyDynamics
using StaticArrays

# Fix the random seed, so we get repeatable results
using Random
Random.seed!(42);
```

First we'll load our double pendulum robot from URDF:

```julia
urdf = joinpath(dirname(pathof(RigidBodyDynamics)), "..", "test", "urdf", "Acrobot.urdf")
mechanism = parse_urdf(urdf)
state = MechanismState(mechanism)
mechanism
```

    Spanning tree:
    Vertex: world (root)
      Vertex: upper_link, Edge: shoulder
        Vertex: lower_link, Edge: elbow
    No non-tree joints.

Now we choose a point on the robot to control. We'll pick the end of the second link, which is located 2m from the origin of the `lower_link` body:

```julia
body = findbody(mechanism, "lower_link")
point = Point3D(default_frame(body), 0., 0, -2)
```

    Point3D in "after_elbow": [0.0, 0.0, -2.0]

Let's visualize the mechanism and its attached point. For visualization, we'll use [MeshCatMechanisms.jl](https://github.com/JuliaRobotics/MeshCatMechanisms.jl) with [Blink.jl](https://github.com/JunoLab/Blink.jl).

(Note: the `#NBSKIP` comments are to skip these cells during `Pkg.test("RigidBodyDynamics")`)

```julia
#NBSKIP
using Pkg
Pkg.activate(joinpath(@__DIR__, "visualization"))
pkg"instantiate"
pkg"precompile"
using MeshCatMechanisms, Blink
Blink.AtomShell.isinstalled() || Blink.AtomShell.install();
```

      Updating registry at `~/.julia/registries/General`
      Updating git-repo `https://github.com/JuliaRegistries/General.git`
    Precompiling project...

```julia
#NBSKIP
# Create the visualizer
vis = MechanismVisualizer(mechanism, URDFVisuals(urdf))
# Render our target point attached to the robot as a sphere with radius 0.07
setelement!(vis, point, 0.07)
# Open the visualizer in a new Blink window
open(vis, Window());
```

## Inverse Kinematics

First, let's use the point jacobian to solve a simple inverse kinematics problem. Given a target location `desired` expressed in world frame, we want to find the joint angles `q` such that the `point` attached to the robot is at the desired location.

To do that, we'll iteratively update `q` by applying:

\begin{align}
\Delta q = \alpha \, J_p^\top \, \Delta p
\end{align}

where $\alpha$ is our step size (equivalent to a learning rate in gradient descent) and $\Delta p$ is the error in the position of our target point.

```julia
function jacobian_transpose_ik!(state::MechanismState,
                               body::RigidBody,
                               point::Point3D,
                               desired::Point3D;
                               α=0.1,
                               iterations=100)
    mechanism = state.mechanism
    world = root_frame(mechanism)

    # Compute the joint path from world to our target body
    p = path(mechanism, root_body(mechanism), body)
    # Allocate the point jacobian (we'll update this in-place later)
    Jp = point_jacobian(state, p, transform(state, point, world))

    q = copy(configuration(state))

    for i in 1:iterations
        # Update the position of the point
        point_in_world = transform(state, point, world)
        # Update the point's jacobian
        point_jacobian!(Jp, state, p, point_in_world)
        # Compute an update in joint coordinates using the jacobian transpose
        Δq = α * Array(Jp)' * (transform(state, desired, world) - point_in_world).v
        # Apply the update
        q .= configuration(state) .+ Δq
        set_configuration!(state, q)
    end
    state
end
```

    jacobian_transpose_ik! (generic function with 1 method)

To use our IK method, we just have to set our current state and choose a desired location for the tip of the robot's arm:

```julia
rand!(state)
```

```julia
#NBSKIP
set_configuration!(vis, configuration(state))
```

```julia
# Choose a desired location. We'll move the tip of the arm to
# [0.5, 0, 2]
desired_tip_location = Point3D(root_frame(mechanism), 0.5, 0, 2)
# Run the IK, updating `state` in place
jacobian_transpose_ik!(state, body, point, desired_tip_location)
```

    MechanismState{Float64, Float64, Float64, …}(…)

```julia
#NBSKIP
set_configuration!(vis, configuration(state))
```

```julia
# We asked for our point to be close to [0.5, 0, 2],
# but since the arm cannot move in the y direction at all
# we end up near [0.5, 0.25, 2] instead
transform(state, point, root_frame(mechanism))
```

    Point3D in "": [0.49993, 0.25, 1.969]

We can try varying the target and watching the IK solution change:

```julia
qs = typeof(configuration(state))[]

# Vary the desired x position from -1 to 1
for x in range(-1, stop=1, length=100)
    desired = Point3D(root_frame(mechanism), x, 0, 2)
    jacobian_transpose_ik!(state, body, point, desired)
    push!(qs, copy(configuration(state)))
end
```

```julia
#NBSKIP
ts = collect(range(0, stop=1, length=length(qs)))
setanimation!(vis, ts, qs)
```

    ┌ Warning: `getindex(itp::AbstractInterpolation{T, N}, i::Vararg{Number, N}) where {T, N}` is deprecated, use `itp(i...)` instead.
    │   caller = #setanimation!#4(::Int64, ::Bool, ::Int64, ::Function, ::MechanismVisualizer{MechanismState{Float64,Float64,Float64,TypeSortedCollections.TypeSortedCollection{Tuple{Array{Joint{Float64,Revolute{Float64}},1}},1}},MeshCat.Visualizer}, ::Array{Float64,1}, ::Array{SegmentedVector{JointID,Float64,Base.OneTo{JointID},Array{Float64,1}},1}) at animate.jl:46
    └ @ MeshCatMechanisms /home/twan/.julia/packages/MeshCatMechanisms/AHOtT/src/animate.jl:46

## Control

Now let's use the same principle to generate torques and actually control the robot. To make things more interesting, let's get the end of the robot's arm to trace out a circle.

```julia
circle_origin = SVector(0., 0.25, 2)
radius = 0.5
ω = 1.0  # radians per second at which the point should move in its circle
```

    1.0

```julia
#NBSKIP
using MeshCat
using GeometryTypes: Point
```

```julia
#NBSKIP

# Draw the circle in the viewer
θ = repeat(range(0, stop=2π, length=100), inner=(2,))[2:end]
cx, cy, cz = circle_origin
geometry = PointCloud(Point.(cx .+ radius .* sin.(θ), cy, cz .+ 0.5 .* cos.(θ)))
setobject!(vis[:circle], LineSegments(geometry, LineBasicMaterial()))
```

    MeshCat Visualizer with path /meshcat/circle

```julia
# This function will take in the parameters of the circle
# and the target point and return a function we can use
# as the controller. By wrapping the construction of the
# controller in this way, we avoid any issues with accessing
# non-const global variables.
function make_circle_controller(state::MechanismState,
                                body::RigidBody,
                                point::Point3D,
                                circle_origin::AbstractVector,
                                radius,
                                ω)
    mechanism = state.mechanism
    world = root_frame(mechanism)
    joint_path = path(mechanism, root_body(mechanism), body)
    Jp = point_jacobian(state, joint_path, transform(state, point, root_frame(mechanism)))
    v̇ = similar(velocity(state))

    function controller!(τ, t, state)
        desired = Point3D(world, circle_origin .+ radius .* SVector(sin(t / ω), 0, cos(t / ω)))
        point_in_world = transform_to_root(state, body) * point
        point_jacobian!(Jp, state, joint_path, point_in_world)
        Kp = 200
        Kd = 20
        Δp = desired - point_in_world
        v̇ .= Kp * Array(Jp)' * Δp.v .- 20 .* velocity(state)
        τ .= inverse_dynamics(state, v̇)
    end
end
```

    make_circle_controller (generic function with 1 method)

```julia
controller! = make_circle_controller(state, body, point, circle_origin, radius, ω)
```

    controller! (generic function with 1 method)

```julia
ts, qs, vs = simulate(state, 10, controller!);
```

Animate the resulting trajectory:

```julia
#NBSKIP
setanimation!(vis, ts, qs)
```

Now we can plot the behavior of the controller. The initial state is quite far from the target, so there's some significant overshoot early in the trajectory, but the controller eventually settles into tracking the desired circular path. This controller isn't very well-tuned, and we could certainly do better with a more advanced approach, but this is still a nice demonstration of a very simple control policy.

```julia
#NBSKIP
using Plots; gr()
```

    Plots.GRBackend()

```julia
#NBSKIP
xs = Float64[]
zs = Float64[]
for q in qs[1:100:end]  # Downsample by 100 just so the plot doesn't become a huge file
    set_configuration!(state, q)
    p = transform(state, point, root_frame(mechanism))
    push!(xs, p.v[1])
    push!(zs, p.v[3])
end

plot(xs, zs, xlim=(-1, 1), ylim=(1, 3), aspect_ratio=:equal, legend=nothing)
```

```julia

```

notebooks/Jacobian IK and Control/Jacobian IK and Control.ipynb


# The Discrete-Time Fourier Transform

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Theorems

The theorems of the discrete-time Fourier transform (DTFT) relate basic operations applied to discrete signals to their equivalents in the DTFT domain. They are of use to transform signals composed from modified [standard signals](../discrete_signals/standard_signals.ipynb), for the computation of the response of a linear time-invariant (LTI) system and to predict the consequences of modifying a signal or system by certain operations.

### Convolution Theorem

The [convolution theorem](https://en.wikipedia.org/wiki/Convolution_theorem) states that the DTFT of the linear convolution of two discrete signals $x[k]$ and $y[k]$ is equal to the scalar multiplication of their DTFTs $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ and $Y(e^{j \Omega}) = \mathcal{F}_* \{ y[k] \}$

\begin{equation}
\mathcal{F}_* \{ x[k] * y[k] \} = X(e^{j \Omega}) \cdot Y(e^{j \Omega})
\end{equation}

The theorem can be proven by introducing the [definition of the linear convolution](../discrete_systems/linear_convolution.ipynb) into the [definition of the DTFT](definition.ipynb) and changing the order of summation

\begin{align}
\mathcal{F} \{ x[k] * y[k] \} &= \sum_{k = -\infty}^{\infty} \left( \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot y[k - \kappa] \right) e^{-j \Omega k}  \\
&= \sum_{\kappa = -\infty}^{\infty} \left( \sum_{k = -\infty}^{\infty} y[k - \kappa] \, e^{-j \Omega k} \right) x[\kappa] \\
&= Y(e^{j \Omega}) \cdot \sum_{\kappa = -\infty}^{\infty} x[\kappa] \, e^{-j \Omega \kappa} \\
&= Y(e^{j \Omega}) \cdot X(e^{j \Omega})
\end{align}

The convolution theorem is very useful in the context of LTI systems. The output signal $y[k]$ of an LTI system is given as the convolution of the input signal $x[k]$ with its impulse response $h[k]$. Hence, the signals and the system can be represented equivalently in the time and frequency domain

Calculation of the system response by transforming the problem into the DTFT domain can be beneficial since this replaces the computation of the linear convolution by a scalar multiplication. The (inverse) DTFT is known for many signals or can be derived by applying the properties and theorems to standard signals and their transforms. In many cases this procedure simplifies the calculation of the system response significantly.

The convolution theorem can also be useful to derive the DTFT of a signal. The key is here to express the signal as convolution of two other signals for which the transforms are known. This is illustrated in the following example.

#### Transformation of the trapezoidal and triangular signal

The linear convolution of two [rectangular signals](../discrete_signals/standard_signals.ipynb#Rectangular-Signal) of lengths $N$ and $M$ defines a [signal of trapezoidal shape](../discrete_systems/linear_convolution.ipynb#Finite-Length-Signals)

\begin{equation}
x[k] = \text{rect}_N[k] * \text{rect}_M[k]
\end{equation}

Application of the convolution theorem together with the [DTFT of the rectangular signal](definition.ipynb#Transformation-of-the-Rectangular-Signal) yields its DTFT as

\begin{equation}
X(e^{j \Omega}) = \mathcal{F}_* \{ \text{rect}_N[k] \} \cdot \mathcal{F}_* \{ \text{rect}_M[k] \} =
e^{-j \Omega \frac{N+M-2}{2}} \cdot \frac{\sin(\frac{N \Omega}{2}) \sin(\frac{M \Omega}{2})}{\sin^2 ( \frac{\Omega}{2} )}
\end{equation}

The transform of the triangular signal can be derived from this result. The convolution of two rectangular signals of equal length $N=M$ yields the triangular signal $\Lambda[k]$ of length $2N - 1$

\begin{equation}
\Lambda_{2N - 1}[k] = \begin{cases} k + 1 & \text{for } 0 \leq k < N \\
2N - 1 - k & \text{for } N \leq k < 2N - 1 \\
0 & \text{otherwise}
\end{cases}
\end{equation}

From above result the DTFT of the triangular signal is derived by substitution of $N$ by $M$

\begin{equation}
\mathcal{F}_* \{ \Lambda_{2N - 1}[k] \} =
e^{-j \Omega (N-1)} \cdot \frac{\sin^2(\frac{N \Omega}{2}) }{\sin^2 ( \frac{\Omega}{2} )}
\end{equation}

Both the triangular signal and the magnitude of its DTFT are plotted for illustration

```python
import sympy as sym
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

N = 7

x = np.convolve(np.ones(N), np.ones(N), mode='full')
plt.stem(x)
plt.xlabel('$k$')
plt.ylabel('$x[k]$')

W = sym.symbols('Omega')
X = sym.exp(-sym.I*W * (N-1)) * sym.sin(N*W/2)**2 / sym.sin(W/2)**2
sym.plot(sym.Abs(X), (W, -5, 5), xlabel='$\Omega$',
         ylabel='$|X(e^{j \Omega})|$')
```

    <sympy.plotting.plot.Plot at 0x10cc6f048>

**Exercise**

* Change the length of the triangular signal in above example. How does its DTFT change?
* The triangular signal introduced above is of odd length $2N - 1$
    * Define a triangular signal of even length by convolving two rectangular signals
    * Derive its DTFT
    * Compare the DTFTs of a triangular signal of odd/even length

### Shift Theorem

The [shift of a signal](../discrete_signals/operations.ipynb#Shift) $x[k]$ can be expressed by a convolution with a shifted Dirac impulse

\begin{equation}
x[k - \kappa] = x[k] * \delta[k - \kappa]
\end{equation}

for $\kappa \in \mathbb{Z}$. This follows from the sifting property of the Dirac impulse. Applying the DTFT to the left- and right-hand side and exploiting the convolution theorem yields

\begin{equation}
\mathcal{F}_* \{ x[k - \kappa] \} = X(e^{j \Omega}) \cdot e^{- j \Omega \kappa}
\end{equation}

where $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$. Note that $\mathcal{F}_* \{ \delta(k - \kappa) \} = e^{- j \Omega \kappa}$ can be derived from the definition of the DTFT together with the sifting property of the Dirac impulse. Above relation is known as shift theorem of the DTFT.

Expressing the DTFT $X(e^{j \Omega}) = |X(e^{j \Omega})| \cdot e^{j \varphi(e^{j \Omega})}$ by its absolute value $|X(e^{j \Omega})|$ and phase $\varphi(e^{j \Omega})$ results in

\begin{equation}
\mathcal{F}_* \{ x[k - \kappa] \} = | X(e^{j \Omega}) | \cdot e^{j (\varphi(e^{j \Omega}) - \Omega \kappa)}
\end{equation}

Shifting of a signal does not change the absolute value of its spectrum but it subtracts the linear contribution $\Omega \kappa$ from its phase.

### Multiplication Theorem

The transform of a multiplication of two signals $x[k] \cdot y[k]$ is derived by introducing the signals into the definition of the DTFT, expressing the signal $x[k]$ by its spectrum $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ and rearranging terms

\begin{align}
\mathcal{F}_* \{ x[k] \cdot y[k] \} &= \sum_{k=-\infty}^{\infty} x[k] \cdot y[k] \, e^{-j \Omega k} \\
&= \sum_{k=-\infty}^{\infty} \left( \frac{1}{2 \pi} \int_{-\pi}^{\pi} X(e^{j \nu}) \, e^{j \nu k} \; d \nu \right) y[k] \, e^{-j \Omega k} \\
&= \frac{1}{2 \pi} \int_{-\pi}^{\pi} X(e^{j \nu}) \sum_{k=-\infty}^{\infty} y[k] \, e^{-j (\Omega - \nu) k} \; d\nu \\
&= \frac{1}{2 \pi} \int_{-\pi}^{\pi} X(e^{j \nu}) \cdot Y(e^{j (\Omega - \nu)}) d\nu
\end{align}

where $Y(e^{j \Omega}) = \mathcal{F}_* \{ y[k] \}$.

The [periodic (cyclic/circular) convolution](https://en.wikipedia.org/wiki/Circular_convolution) of two aperiodic signals $h(t)$ and $g(t)$ is defined as

\begin{equation}
h(t) \circledast_{T} g(t) = \int_{-\infty}^{\infty} h(\tau) \cdot g_\text{p}(t - \tau) \; d\tau
\end{equation}

where $T$ denotes the period of the convolution, $g_\text{p}(t) = \sum_{n=-\infty}^{\infty} g(t + n T)$ the periodic summation of $g(t)$ and $\tau \in \mathbb{R}$ an arbitrary constant. The periodic convolution is commonly abbreviated by $\circledast_{T}$. With $h_\text{p}(t)$ denoting the periodic summation of $h(t)$ the periodic convolution can be rewritten as

\begin{equation}
h(t) \circledast_{T} g(t) = \int_{\tau_0}^{\tau_0 + T} h_\text{p}(\tau) \cdot g_\text{p}(t - \tau) \; d\tau
\end{equation}

where $\tau_0 \in \mathbb{R}$ denotes an arbitrary constant. The latter definition holds also for two [periodic signals](../periodic_signals/spectrum.ipynb) $h(t)$ and $g(t)$ with period $T$.

Comparison of the DTFT of two multiplied signals with the definition of the periodic convolution reveals that the preliminary result above can be expressed as

\begin{equation}
\mathcal{F}_* \{ x[k] \cdot y[k] \} = \frac{1}{2\pi} \, X(e^{j \Omega}) \circledast_{2 \pi} Y(e^{j \Omega})
\end{equation}

The DTFT of a multiplication of two signals $x[k] \cdot y[k]$ is given by the periodic convolution of their transforms $X(e^{j \Omega})$ and $Y(e^{j \Omega})$ weighted with $\frac{1}{2 \pi}$. The periodic convolution has a period of $T = 2 \pi$. Note, the convolution is performed with respect to the normalized angular frequency $\Omega$.

Applications of the multiplication theorem include the modulation and windowing of signals. The former leads to the modulation theorem introduced later, the latter is illustrated by the following example.

**Example**

Windowing of signals is used to derive signals of finite duration from signals of infinite duration or to truncate signals to a shorter length. The signal $x[k]$ is multiplied by a weighting function $w[k]$ in order to derive the finite length signal

\begin{equation}
y[k] = w[k] \cdot x[k]
\end{equation}

Application of the multiplication theorem yields the spectrum $Y(e^{j \Omega}) = \mathcal{F}_* \{ y[k] \}$ of the windowed signal as

\begin{equation}
Y(e^{j \Omega}) = \frac{1}{2 \pi} W(e^{j \Omega}) \circledast X(e^{j \Omega})
\end{equation}

where $W(e^{j \Omega}) = \mathcal{F}_* \{ w[k] \}$ and $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$. In order to illustrate the consequence of windowing, a cosine signal $x[k] = \cos(\Omega_0 k)$ is truncated to a finite length using a rectangular signal

\begin{equation}
y[k] = \text{rect}_N[k] \cdot \cos(\Omega_0 k)
\end{equation}

where $N$ denotes the length of the truncated signal and $\Omega_0$ its normalized angular frequency. Using the DTFT of the [rectangular signal](definition.ipynb#Transformation-of-the-Rectangular-Signal) and the [cosine signal](properties.ipynb#Transformation-of-the-cosine-and-sine-signal) yields

\begin{align}
Y(e^{j \Omega}) &= \frac{1}{2 \pi} e^{-j \Omega \frac{N-1}{2}} \cdot \frac{\sin \left(\frac{N \Omega}{2} \right)}{\sin \left( \frac{\Omega}{2} \right)} \circledast \frac{1}{2} \left[ {\bot \!\! \bot \!\! \bot} \left( \frac{\Omega + \Omega_0}{2 \pi} \right) + {\bot \!\! \bot \!\! \bot} \left( \frac{\Omega - \Omega_0}{2 \pi} \right) \right] \\
&= \frac{1}{2} \left[ e^{-j (\Omega+\Omega_0) \frac{N-1}{2}} \cdot \frac{\sin \left(\frac{N (\Omega+\Omega_0)}{2} \right)}{\sin \left( \frac{\Omega+\Omega_0}{2} \right)} + e^{-j (\Omega-\Omega_0) \frac{N-1}{2}} \cdot \frac{\sin \left(\frac{N (\Omega-\Omega_0)}{2} \right)}{\sin \left( \frac{\Omega-\Omega_0}{2} \right)} \right]
\end{align}

The latter identity results from the sifting property of the Dirac impulse and the periodicity of both spectra. The signal $y[k]$ and its magnitude spectrum $|Y(e^{j \Omega})|$ are plotted for specific values of $N$ and $\Omega_0$.

```python
N = 20
W0 = 2*np.pi/10

k = np.arange(N)
x = np.cos(W0 * k)

plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel('$y[k]$')
```

    Text(0, 0.5, '$y[k]$')

```python
W = sym.symbols('Omega')

Y = 1/2 * ((sym.exp(-sym.I*(W+W0)*(N-1)/2) * sym.sin(N*(W+W0)/2) / sym.sin((W+W0)/2)) +
           (sym.exp(-sym.I*(W-W0)*(N-1)/2) * sym.sin(N*(W-W0)/2) / sym.sin((W-W0)/2)))

sym.plot(sym.Abs(Y), (W, -sym.pi, sym.pi),
         xlabel='$\Omega$', ylabel='$|Y(e^{j \Omega})|$')
```

    <sympy.plotting.plot.Plot at 0x118c97240>

**Exercise**

* Change the length $N$ of the signal by modifying the example. How does the spectrum change if you decrease or increase the length?

* What happens if you change the normalized angular frequency $\Omega_0$ of the signal?

* Assume a signal that is composed from a superposition of two finite length cosine signals with different frequencies. What qualitative condition has to hold that you can derive these frequencies from inspection of the spectrum?

### Modulation Theorem

The complex modulation of a signal $x[k]$ is defined as $e^{j \Omega_0 k} \cdot x[k]$ with $\Omega_0 \in \mathbb{R}$. The DTFT of the modulated signal is derived by applying the multiplication theorem

\begin{equation}
\mathcal{F}_* \left\{ e^{j \Omega_0 k} \cdot x[k] \right\} = \frac{1}{2 \pi} \cdot {\bot \!\! \bot \!\! \bot}\left( \frac{\Omega - \Omega_0}{2 \pi} \right) \circledast X(e^{j \Omega})
= X \big( e^{j \, (\Omega - \Omega_0)} \big)
\end{equation}

where $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$.  Above result states that the complex modulation of a signal leads to a shift of its spectrum. This result is known as modulation theorem.

**Example**

An example for the application of the modulation theorem is the
[downsampling/decimation](https://en.wikipedia.org/wiki/Decimation_(signal_processing)) of a discrete signal $x[k]$. Downsampling refers to lowering the sampling rate of a signal. The example focuses on the special case of removing every second sample, hence halving the sampling rate. The downsampling is modeled by defining a signal $x_\frac{1}{2}[k]$ where every second sample is set to zero

\begin{equation}
x_\frac{1}{2}[k] = \begin{cases}
x[k] & \text{for even } k \\
0 & \text{for odd } k
\end{cases}
\end{equation}

In order to derive the spectrum $X_\frac{1}{2}(e^{j \Omega}) = \mathcal{F}_* \{ x_\frac{1}{2}[k] \}$, the signal $u[k]$ is introduced where every second sample is zero

\begin{equation}
u[k] = \frac{1}{2} ( 1 + e^{j \pi k} ) = \begin{cases} 1 & \text{for even } k \\
0 & \text{for odd } k \end{cases}
\end{equation}

Using $u[k]$, the process of setting every second sample of $x[k]$ to zero can be expressed as

\begin{equation}
x_\frac{1}{2}[k] = u[k] \cdot x[k]
\end{equation}

Now the spectrum $X_\frac{1}{2}(e^{j \Omega})$ is derived by applying the multiplication theorem and introducing the [DTFT of the exponential signal](definition.ipynb#Transformation-of-the-Exponential-Signal). This results in

\begin{equation}
X_\frac{1}{2}(e^{j \Omega}) = \frac{1}{4 \pi} \left(  {\bot \!\! \bot \!\! \bot}\left( \frac{\Omega}{2 \pi} \right) +
{\bot \!\! \bot \!\! \bot}\left( \frac{\Omega - \pi}{2 \pi} \right) \right) \circledast X(e^{j \Omega}) =
\frac{1}{2} X(e^{j \Omega}) + \frac{1}{2} X(e^{j (\Omega- \pi)})
\end{equation}

where $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$. The spectrum $X_\frac{1}{2}(e^{j \Omega})$ consists of the spectrum of the original signal $X(e^{j \Omega})$ superimposed by the shifted spectrum $X(e^{j (\Omega- \pi)})$ of the original signal. This may lead to overlaps that constitute aliasing. In order to avoid aliasing, the spectrum of the signal $x[k]$ has to be band-limited to $-\frac{\pi}{2} < \Omega < \frac{\pi}{2}$ before downsampling.

### Parseval's Theorem

[Parseval's theorem](https://en.wikipedia.org/wiki/Parseval's_theorem) relates the energy of a discrete signal to its spectrum. The squared absolute value of a signal $x[k]$ represents its instantaneous power. It can be expressed as

\begin{equation}
| x[k] |^2 = x[k] \cdot x^*[k]
\end{equation}

where $x^*[k]$ denotes the complex conjugate of $x[k]$. Transformation of the right-hand side and application of the multiplication theorem results in

\begin{equation}
\mathcal{F}_* \{ x[k] \cdot x^*[k] \} = \frac{1}{2 \pi} \cdot X(e^{j \Omega}) \circledast_{2 \pi} X^*(e^{-j \Omega})
\end{equation}

Introducing the definition of the DTFT and the periodic convolution

\begin{equation}
\sum_{k = -\infty}^{\infty} x[k] \cdot x^*[k] \, e^{-j \Omega k}  =
\frac{1}{2 \pi} \int_{-\pi}^{\pi} X(e^{j \nu}) \cdot X^*(e^{j (\Omega - \nu)}) \; d\nu
\end{equation}

Setting $\Omega = 0$ followed by the substitution $\nu = \Omega$ yields Parseval's theorem

\begin{equation}
\sum_{k = -\infty}^{\infty} | x[k] |^2 = \frac{1}{2 \pi} \int_{-\pi}^{\pi} | X(e^{j \Omega}) |^2 \; d\Omega
\end{equation}

The sum over the samples of the squared absolute signal is equal to the integral over its squared absolute spectrum divided by $2 \pi$. Since the left-hand side represents the energy $E$ of the signal $x[k]$, Parseval's theorem states that the energy can be computed alternatively in the spectral domain by integrating over the squared absolute value of the spectrum.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.