## Nonlinear Dimensionality Reduction
G. Richards (2016, 2018), based on materials from Ivezic, Connolly, Miller, Leighly, and VanderPlas.

Today we will talk about the concepts of
* manifold learning
* nonlinear dimensionality reduction

Specifically using the following algorithms
* local linear embedding (LLE)
* isometric mapping (IsoMap)
* t-distributed Stochastic Neighbor Embedding (t-SNE)

Let's start by my echoing the brief note of caution given in Adam Miller's notebook: "astronomers will often try to derive physical insight from PCA eigenspectra or eigentimeseries, but this is not advisable as there is no physical reason for the data to be linearly and orthogonally separable".  Moreover, physical components are (generally) positive definite.  So, PCA is great for dimensional reduction, but for doing physics there are generally better choices.

While NMF "solves" the issue of negative components, it is still a linear process.  For data with non-linear correlations, an entire field, known as [Manifold Learning](http://scikit-learn.org/stable/modules/manifold.html) and [nonlinear dimensionality reduction]( https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction), has been developed, with several algorithms available via the [`sklearn.manifold`](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.manifold) module.

For example, if your data set looks like this:

Then PCA is going to give you something like this.

Clearly not very helpful!

What you were really hoping for is something more like the results below.  For more examples see
[Vanderplas & Connolly 2009](http://iopscience.iop.org/article/10.1088/0004-6256/138/5/1365/meta;jsessionid=48A569862A424ECCAEECE2A900D9837B.c3.iopscience.cld.iop.org)

## Local Linear Embedding

[Local Linear Embedding](http://scikit-learn.org/stable/modules/generated/sklearn.manifold.LocallyLinearEmbedding.html#sklearn.manifold.LocallyLinearEmbedding) attempts to embed high-$D$ data in a lower-$D$ space.  Crucially it also seeks to preserve the geometry of the local "neighborhoods" around each point.  In the case of the "S" curve, it seeks to unroll the data.  The steps are

Step 1: define local geometry
- local neighborhoods determined from $k$ nearest neighbors.
- for each point calculate weights that reconstruct a point from its $k$ nearest
neighbors via
$$
\begin{equation}
  \mathcal{E}_1(W) = \left|X - WX\right|^2,
\end{equation}
$$
where $X$ is an $N\times K$ matrix and $W$ is an $N\times N$ matrix that minimizes the reconstruction error.

Essentially this is finding the hyperplane that describes the local surface at each point within the data set. So, imagine that you have a bunch of square tiles and you are trying to tile the surface with them.

Step 2: embed within a lower dimensional space
- set all $W_{ij}=0$ except when point $j$ is one of the $k$ nearest neighbors of point $i$.
- $W$ becomes very sparse for $k \ll N$ (only $Nk$ entries in $W$ are non-zero).
- minimize
>$\begin{equation}
  \mathcal{E}_2(Y) = \left|Y - W Y\right|^2,
\end{equation}
$

with $W$ fixed to find an $N$ by $d$ matrix ($d$ is the new dimensionality).

Step 1 requires a nearest-neighbor search.

Step 2 requires an
eigenvalue decomposition of the matrix $C_W \equiv (I-W)^T(I-W)$.

LLE has been applied to data as diverse as galaxy spectra, stellar spectra, and photometric light curves.   It was introduced by [Roweis & Saul (2000)](https://www.ncbi.nlm.nih.gov/pubmed/11125150).

Skikit-Learn's call to LLE is as follows, with a more detailed example already being given above.

```python
import numpy as np
from sklearn.manifold import LocallyLinearEmbedding
X = np.random.normal(size=(1000,2)) # 1000 points in 2D
R = np.random.random((2,10)) # projection matrix
X = np.dot(X,R) # now a 2D linear manifold in 10D space
k = 5 # Number of neighbors to use in fit
n = 2 # Number of dimensions to fit
lle = LocallyLinearEmbedding(k,n)
lle.fit(X)
proj = lle.transform(X) # 100x2 projection of the data
```

See what LLE does for the digits data, using the 7 nearest neighbors and 2 components.

```python
# Execute this cell to load the digits sample
%matplotlib inline
import numpy as np
from sklearn.datasets import load_digits
from matplotlib import pyplot as plt
digits = load_digits()
grid_data = np.reshape(digits.data[0], (8,8)) #reshape to 8x8
plt.imshow(grid_data, interpolation = "nearest", cmap = "bone_r")
print(grid_data)
X = digits.data
y = digits.target
```

```python
#LLE
from sklearn.manifold import LocallyLinearEmbedding
k = 7 # Number of neighbors to use in fit
n = 2 # Number of dimensions to fit
lle = LocallyLinearEmbedding(k,n)
lle.fit(X)
X_reduced = lle.transform(X)

plt.scatter(X_reduced[:,0], X_reduced[:,1], c=y, cmap="nipy_spectral", edgecolor="None")
plt.colorbar()
```

## Isometric Mapping

is based on multi-dimensional scaling (MDS) framework.  It was introduced in the same volume of *Science* as the article above.  See [Tenenbaum, de Silva, & Langford (2000)](https://www.ncbi.nlm.nih.gov/pubmed/?term=A+Global+Geometric+Framework+for+Nonlinear+Dimensionality+Reduction).
Geodestic curves are used to recover non-linear structure.

In Scikit-Learn [IsoMap](http://scikit-learn.org/stable/modules/generated/sklearn.manifold.Isomap.html) is implemented as follows:

```python
# Execute this cell
import numpy as np
from sklearn.manifold import Isomap
XX = np.random.normal(size=(1000,2)) # 1000 points in 2D
R = np.random.random((2,10)) # projection matrix
XX = np.dot(XX,R) # X is a 2D manifold in 10D space
k = 5 # number of neighbors
n = 2 # number of dimensions
iso = Isomap(k,n)
iso.fit(XX)
proj = iso.transform(XX) # 1000x2 projection of the data
```

Try 7 neighbors and 2 dimensions on the digits data.

```python
# IsoMap
from sklearn.manifold import Isomap
k = 7 # Number of neighbors to use in fit
n = 2 # Number of dimensions to fit
iso = Isomap(k,n)
iso.fit(X)
X_reduced = iso.transform(X)

plt.scatter(X_reduced[:,0], X_reduced[:,1], c=y, cmap="nipy_spectral", edgecolor="None")
plt.colorbar()
```

## t-SNE

[t-distributed Stochastic Neighbor Embedding (t-SNE)](https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding) is not discussed in the book, Scikit-Learn does have a [t-SNE implementation](http://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html) and it is well worth mentioning this manifold learning algorithm too.  SNE itself was developed by [Hinton & Roweis](http://www.cs.toronto.edu/~fritz/absps/sne.pdf) with the "$t$" part being added by [van der Maaten & Hinton](http://jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf).  It works like the other manifold learning algorithms.

 Try it on the digits data.  You'll need to import `TSNE` from `sklearn.manifold`, instantiate it with 2 components, then do a `fit_transform` on the original data.

```python
# t-SNE
from sklearn.manifold import TSNE
tsne = TSNE(n_components=2)
X_reduced = tsne.fit_transform(X)

plt.scatter(X_reduced[:,0], X_reduced[:,1], c=y, cmap="nipy_spectral", edgecolor="None")
plt.colorbar()
```

You'll know if you have done it right if you understand Adam Miller's comment "Holy freakin' smokes.  That is magic.  (It's possible we just solved science)."

Personally, I think that some exclamation points may be needed in there!

What's even more illuminating is to make the plot using the actual digits to plot the points.  Then you can see why certain digits are alike or split into multiple regions.  Can you explain the patterns you see here?

```python
# Execute this cell
from matplotlib import offsetbox

#----------------------------------------------------------------------
# Scale and visualize the embedding vectors
def plot_embedding(X):
    x_min, x_max = np.min(X, 0), np.max(X, 0)
    X = (X - x_min) / (x_max - x_min)

    plt.figure()
    ax = plt.subplot(111)
    for i in range(X.shape[0]):
      #plt.text(X[i, 0], X[i, 1], str(digits.target[i]), color=plt.cm.Set1(y[i] / 10.), fontdict={'weight': 'bold', 'size': 9})
      plt.text(X[i, 0], X[i, 1], str(digits.target[i]), color=plt.cm.nipy_spectral(y[i]/9.))

    shown_images = np.array([[1., 1.]])  # just something big
    for i in range(digits.data.shape[0]):
        dist = np.sum((X[i] - shown_images) ** 2, 1)
        if np.min(dist) < 4e-3:
            # don't show points that are too close
            continue
        shown_images = np.r_[shown_images, [X[i]]]
        imagebox = offsetbox.AnnotationBbox(offsetbox.OffsetImage(digits.images[i], cmap=plt.cm.gray_r), X[i])
        ax.add_artist(imagebox)
    plt.xticks([]), plt.yticks([])

plot_embedding(X_reduced)
plt.show()
```

With the remainder of time in class today, play with the arguments of the algorithms that we have discussed this week and/or try running them on a different data set.  For example the iris data set or one of the other samples of data that are included with Scikit-Learn.  Or maybe have a look through some of these public data repositories:

- [https://github.com/caesar0301/awesome-public-datasets?utm_content=buffer4245d&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer](https://github.com/caesar0301/awesome-public-datasets?utm_content=buffer4245d&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer)
- [http://www.datasciencecentral.com/m/blogpost?id=6448529%3ABlogPost%3A318739](http://www.datasciencecentral.com/m/blogpost?id=6448529%3ABlogPost%3A318739)
- [http://www.kdnuggets.com/2015/04/awesome-public-datasets-github.html](http://www.kdnuggets.com/2015/04/awesome-public-datasets-github.html)


## **Importance sampling**

### **Exercise:**

Asset price under $\mathbb{Q}$ follows

$$S_{t} = S_{0} \exp \{\mu t + \sigma W_{t} \}.$$

Consider Digital put with its payoff

$$h(S_{T}) = I (S_{T} < S_{0} e^{-b}) .$$

We want to find the forward price:

$$v = \mathbb{E}^{\mathbb{Q}} [h(S_{T})].$$

Parameters are given as

$$r = 0.03, \sigma = 0.2, \mu = r - \frac{1}{2} \sigma^{2} = 0.01, T = 1, b = 0.39.$$

- Prove that the exact price is $0.02275$.
- Use OMC find the price.
- Use $IS(\alpha)$ find the price.
- Can you show your approach is optimal?
- Prove or demonstrate IS is more efficient to OMC.

```

```

## **Solution:**

(1) Prove that the exact price is $0.02275$.

Solution:

\begin{equation}
\begin{aligned}
v &= \mathbb{E}^{\mathbb{Q}} [h(S_{T})] \\
&= \mathbb{E}^{\mathbb{Q}} [I (S_{T} < S_{0} e^{-b})] \\
&= \mathbb{Q} (S_{T} < S_{0} e^{-b})\\
&= \mathbb{Q} (S_{0} \exp \{(r - \frac{1}{2} \sigma^{2}) T + \sigma W_{T}\} < S_{0} e^{-b}) \\
&= \mathbb{Q} (Z < - \frac{b + (r - \frac{1}{2} \sigma^{2}) T}{\sigma \sqrt{T}}) \\
&= \mathbb{Q} (Z < - 2) \\
&= \Phi(-2), \\
\end{aligned}
\end{equation}
where $Z \sim \mathcal N (0, 1)$ under probability measure $\mathbb{Q}$ and $\Phi(\cdot)$ is the culmulative distribution function of standard normal variable. Then we know the exact price of this forward is $\Phi(-2)$.

```
import numpy as np
import scipy.stats as ss

v = ss.norm.cdf(-2)
print("The exact value of this forward is:", v)
```

    The exact value of this forward is: 0.022750131948179195

```

```

(2) Use OMC find the price.

Solution:

Since we get $v= \Phi(-2) $ in (1), then we have the estimator

$$\hat{v}  \approx \frac{1}{n} \sum_{i = 1}^{n} I(X_{i} < -2),$$

where $X_{i} \sim \mathcal{N}(0, 1)$.

```
## Use monte carlo method to get the forward price

def monte_carlo_v(N):
  sum_v = 0
  for i in range(N):
    x = np.random.normal()
    if x < -2:
      sum_v += 1
  return sum_v/N
```

```
## Test the monte carlo method

v_hat = monte_carlo_v(10000)
print("The value of this forward by the monte carlo method is:", v_hat)
```

    The value of this forward by the monte carlo method is: 0.0233

```

```

(3) Use $IS(\alpha)$ find the price.

Solution:

For the importance sampling , we suppose $\phi_{\alpha}(\cdot)$ is the probability distribution function of the Gaussian random variable with mean $-\alpha$ and variance $1$. Then we have

\begin{equation}
\begin{aligned}
v &= \Phi(-2) \\
&= \int_{- \infty}^{-2} \phi_{0}(x) \, d x \\
&= \int_{- \infty}^{-2} \frac{\phi_{0}(x)}{\phi_{\alpha}(x)} \phi_{\alpha}(x) \, d x \\
&= \int_{- \infty}^{-2} e^{\frac{1}{2} \alpha^{2} + \alpha x} \phi_{\alpha} (x) \, d x \\
&= \mathbb{E} [I(Y < - 2) e^{\frac{1}{2} \alpha^{2} + \alpha Y} | Y \sim \phi_{\alpha}] \\
&\approx \frac{1}{n} \sum_{i = 1}^{n} [I(Y_{i} < - 2) e^{\frac{1}{2} \alpha^{2} + \alpha Y_{i}}]. \\
\end{aligned}
\end{equation}

```
## Use importance sampling method to get the forward price

def importance_sampling_v(N, alpha):
  sum_v = 0
  for i in range(N):
    y = np.random.normal(- alpha, 1)
    if y < -2:
      sum_v += np.exp(0.5 * alpha ** 2 + alpha * y)
  return sum_v / N
```

```
## Test the importance sampling method

v_hat = importance_sampling_v(10000, 4)
print("The value of this forward by the importance sampling method is:", v_hat)
```

    The value of this forward by the importance sampling method is: 0.022070269486359897

We can use the importance sampling method to generate a sequence of estimators, and then we can calculate the sample variance of this sequence. And the $\alpha$ which is correspondent to the minimum variance is the desired one. But this method may not very exactly.

```
estimate_list = []
variance_list = []
for i in range(10):
  for j in range(100):
    estimate_list.append(importance_sampling_v(1000, i))
  variance_list.append(np.var(estimate_list))

alpha_optimal1 = variance_list.index(min(variance_list))
print("The optimal choice of alpha is", alpha_optimal1)
```

    The optimal choice of alpha is 3

```
v_hat = importance_sampling_v(10000, alpha_optimal1)
print("The value of this forward by the importance sampling method is:", v_hat)
```

    The value of this forward by the importance sampling method is: 0.023079668609016963

```

```

(4) Can you show your approach is optimal?

Solution:

By the result we get from (3), we know that when $\alpha = 3$, the variance of the extimators by the importance sampling method is the minimum one. We can also calculate the varaince of the estimators. As

$$\hat{v} = \frac{1}{n} \sum_{i = 1}^{n} [I(Y_{i} < - 2) e^{\frac{1}{2} \alpha^{2} + \alpha Y_{i}}]$$

by the importance sampling method. We know that

\begin{equation}
\begin{aligned}
MSE(\hat{v}) &= var(\hat{v}) \\
&= \frac{1}{n} var(I(Y_{i} < - 2) e^{\frac{1}{2} \alpha^{2} + \alpha Y_{i}}) \\
&= \frac{1}{n} \{ \mathbb{E} [I(Y_{i} < - 2) e^{ \alpha^{2} + 2 \alpha Y_{i}}] - \Phi^{2}(-2) \}. \\
\end{aligned}
\end{equation}

As we have

\begin{equation}
\begin{aligned}
\mathbb{E} [I(Y_{i} < - 2) e^{ \alpha^{2} + 2 \alpha Y_{i}}] &= \int_{- \infty}^{-2} e^{\alpha^{2}+ 2 \alpha y}  \frac{1}{\sqrt{2 \pi}} e^{- \frac{(y + \alpha)^{2}}{2}} \, d y \\
&= \int_{- \infty}^{-2} \frac{1}{\sqrt{2 \pi}} e^{- \frac{y^{2} - \alpha y - \alpha^{2}}{2}} \, d y \\
&= \int_{- \infty}^{-2} \frac{1}{\sqrt{2 \pi}} e^{- \frac{(y - \alpha)^{2}}{2}}  e^{\alpha^{2}} \, d y  \\
&= e^{\alpha^{2}} \Phi(-2-\alpha),
\end{aligned}
\end{equation}

then we can get

$$MSE(\hat{v}) = \frac{1}{n} \{e^{\alpha^{2}} \Phi(-2-\alpha) - \Phi^{2}(-2)\}.$$

Thus we know that the desired $\alpha$ satisfies the equation:

$$2 \alpha \Phi(- 2 - \alpha) = \phi(-2 - \alpha).$$

```
## get the optimal alpha

mse_list = []
for i in range(10):
  mse_list.append(np.exp(i**2) * ss.norm.cdf(-2 - i) - ss.norm.cdf(-2) ** 2)

alpha_optimal2 = mse_list.index(min(mse_list))
print("The optimal choice of alpha is", alpha_optimal2)
```

    The optimal choice of alpha is 2

```
v_hat = importance_sampling_v(10000, alpha_optimal2)
print("The value of this forward by the importance sampling method is:", v_hat)
```

    The value of this forward by the importance sampling method is: 0.02335980963571127

```

```

(5) Prove or demonstrate IS is more efficient to OMC.

Solution:

By the Monte-Carlo method, we have

$$\hat{v_{1}} \approx \frac{1}{n} \sum_{i = 1}^{n} I(X_{i} < -2),$$

then the variance of this estimator is

\begin{equation}
\begin{aligned}
MSE(\hat{v_{1}}) &= var(\hat{v_{1}}) \\
&= \frac{1}{n} var(I(X_{i} < - 2)) \\
&= \frac{1}{n} \{ \mathbb{E} [I(X_{i} < - 2)] - (\mathbb{E} [I(X_{i} < - 2)])^{2} \} \\
&= \frac{1}{n} \{\Phi(-2) - (\Phi(-2))^{2}\}.
\end{aligned}
\end{equation}

For importance sampling method, we have proved that

$$MSE(\hat{v_{2}}) = \frac{1}{n} \{e^{\alpha^{2}} \Phi(-2-\alpha) - \Phi^{2}(-2)\},$$

hence we have

$$MSE(\hat{v_{1}}) - MSE(\hat{v_{2}}) = \frac{\Phi(-2) - e^{\alpha^{2}} \Phi(-2-\alpha)}{n}.$$

When $\alpha = 2$, the difference of MSE between these two method is

$$\frac{\Phi(-2) - e^{4} \Phi(-4)}{n}$$

```
## the difference of MSE between these two method

distance = ss.norm.cdf(-2) - np.exp(4) * ss.norm.cdf(-4)
print("The difference of MSE between these two method is:", distance)
```

    The difference of MSE between these two method is: 0.021020940734838522

Since $\frac{\Phi(-2) - e^{4} \Phi(-4)}{n} > 0$, we know that importance sampling method is more efficient to ordinary Monte-Carlo method.


```python
import cmath, random, numpy
import functools
import  matplotlib.pyplot as plt
import sys
import os
import math
from qutip import*

from sympy import*
#from sympsi import*
from scipy import optimize

from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import time
import math
from qutip import *
from qutip.ipynbtools import plot_animation
import numpy as np
import matplotlib.pyplot as plt
import qutip
%matplotlib inline
import matplotlib.pylab as plt
import matplotlib as mpl
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from IPython.display import display, Math, Latex
import cmath
from mpl_toolkits.axes_grid1 import AxesGrid
from scipy.special import factorial
```

```python
"""Define the operators for MZI - we will deal in Heisenberg """
T = Symbol('T')
xvec = np.arange(-30.,30.)*5./40
X,Y = np.meshgrid(xvec, xvec)  ##Some plotting params
N_dim = 35##Dimenstion of the Hilbert spac

a1 = destroy(N_dim) ##This is for single-photon field
a2 = destroy(N_dim) ##for coherent field
a3 = destroy(N_dim) ##for vacuum field
def n_choose_k(n,k):
    return factorial(n)/(factorial(n-k)*factorial(k))
def D(state,alpha):
    Rho_new=displace(N_dim,alpha)*state*displace(N_dim,alpha).dag()
    return Rho_new
'''Define a rotation in phase space, or phase shifter operation'''
def Phase(theta):
    b=-1j*theta*a1.dag()*a1;
    return b.expm()
'''Squeezing operation, inputs a density matrix and outputs the squeezed density matrix for squeezing parameter r'''
def Sq(state,r):
    Rho_new=squeeze(N_dim,r)*state*squeeze(N_dim,r).dag();
    return Rho_new
'''The function below creates a beamsplitter operation that acts on two modes.
The value for k determines what number Fock state could be filtered out of the first state
based on a single photon input for the second BS port, followed by single photon detection.'''
def BS_operator_filtering(a1, a2, k):
    theta_k = np.arctan(1/np.sqrt(k))
    T = np.sin(theta_k)*np.sin(theta_k)
    R = np.cos(theta_k)*np.cos(theta_k)
    print('I am filtering', k, 'and:', theta_k*180/math.pi)
    print('BS T is : ', T, 'and : ', R)
    b = theta_k*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))
    return b.expm()
def SSV_plus(r,alpha):
    state = ket2dm((displace(N_dim,alpha)+displace(N_dim,-alpha))*squeeze(N_dim,r)*fock(N_dim,0))
    norm_state = state/state.tr()
    return norm_state
def SSV_minus(r,alpha):
    state = ket2dm((displace(N_dim,alpha)-displace(N_dim,-alpha))*squeeze(N_dim,r)*fock(N_dim,0))
    norm_state = state/state.tr()
    return norm_state
def cat_plus(alpha):
    cat = (1/(np.sqrt(2)*np.sqrt(1+np.e**(-alpha*alpha.conj()))))*(coherent(N_dim,-alpha)+(coherent(N_dim,alpha)))
    return cat
def cat_minus(alpha):
    cat = (1/(np.sqrt(2)*np.sqrt(1-np.e**(-alpha*alpha.conj()))))*(-coherent(N_dim,-alpha)+(coherent(N_dim,alpha)))
    return cat
def pnr_resolution_detector(eta, click, n_truc):
    pi_n = 0;
    l = np.arange(click,n_truc)
    for i in l:
        pi_n +=  n_choose_k(i,click)*math.pow((1-eta),(i-click))*math.pow(eta,click)*fock(N_dim,i)*fock(N_dim,i).dag()
        #print("The final Povm element is: ", pi_0)
    return Qobj(pi_n)
def Fock_Filter_povm(in_state,in_fock,refl,num_det,eta,n_truc):
    Projector = tensor(pnr_resolution_detector(eta, num_det, n_truc),qeye(N_dim));
    Initial_state=tensor(in_state,ket2dm(fock(N_dim,in_fock)));
    theta_k=np.arccos(np.sqrt(refl));

    BS1= ((theta_k)*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))).expm()

    Rho=BS1*Initial_state*BS1.dag();

    Rho_filtered = ((Rho*Projector).ptrace(1))/((Rho*Projector).tr())
    '''The operation .ptrace(m) takes the partial trace over every mode EXCEPT m, where the numbering
    startes at 0.  So .ptrace(1) means you keep mode 1, which is actually the 2nd mode'''
    print('BS has reflectivity',refl,' and I am detecting the |',num,'> state, where my detector has efficiency', eta)
    return Rho_filtered
def Fock_Filter_prob(in_state,in_fock,refl,num_det,eta,n_truc):
    Projector = tensor(pnr_resolution_detector(eta, num_det, n_truc),qeye(N_dim));
    Initial_state=tensor(in_state,ket2dm(fock(N_dim,in_fock)));
    theta_k=np.arccos(np.sqrt(refl));

    BS1= ((theta_k)*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))).expm()

    Rho=BS1*Initial_state*BS1.dag();
    P=(Rho*Projector).tr()
    print('The probability of a sucessful detection is:',P)
    Rho_filtered = ((Rho*Projector).ptrace(1))/((Rho*Projector).tr())
    #Rho_filtered=Rho*Projector
    '''The operation .ptrace(m) takes the partial trace over every mode EXCEPT m, where the numbering
    startes at 0.  So .ptrace(1) means you keep mode 1, which is actually the 2nd mode'''
    print('BS has reflectivity',refl,' and I am detecting the |',num,'> state, where my detector has efficiency', eta)
    return Rho_filtered

def fid(state1,state2):
    F=np.absolute((state1.sqrtm()*state2*state1.sqrtm()).sqrtm().tr())
    return F

```

```python
#Variable definitions:
#delta = initial coherent state amplitude
# refl(1-3)= beamsplitter r^2 values at each step
# n(1-3) = number of photons detected at each detector
# beta = amplitude of final dispalcement to displace the state back
#alpha = amplitude of SSV state to be compared with.  Note, here
#this can be related to cat state amplitude by amp_cat=alpha/(Cosh[sq]-Sinh[sq])
# sq = the 'r' value, or squeezing parameter. This is the negative of the mathematica results.

'''Three-step check: P -> 0.00179605, 0.9836, r[3.] -> 0.520135, r[2.] -> 0.490869,
 r[1.] -> 0.637566, \[Alpha] -> 1.349,
 sq -> -0.476595, \[Beta] -> 2.03846, delta -> 3.53709, n1 -> 5,
 n2 -> 2, n3 -> 1}'''

refl1=0.637566**2; refl2=0.490869**2; refl3=0.520135**2;
delta=3.53709; beta=2.03846; sq=0.476595; alpha=1.349; n1=5;n2=2;n3=1;

#refl1=1/2; refl2=1/2; refl3=2/3;
#delta=np.sqrt(14); beta=np.sqrt(19/16); sq=0.476595; alpha=1.4; n1=5;n2=3;n3=1;

```

```python
ssv=SSV_minus(sq,alpha)

W_ssv=wigner(ssv,xvec,xvec);
eta=1;

first=Fock_Filter_prob(ket2dm(coherent(N_dim,delta)),1,refl1,n1,eta,N_dim)
W1=wigner(first,xvec,xvec);

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_ssv, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_ssv, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of ssv state')

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W1, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W1, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of filtered state')
fid = fidelity(ssv,first)
print('fidelity with ssv:',fid*fid)

fig, axes = plt.subplots(1, 3, figsize=(12,4))
bar0 = axes[0].bar(range(N_dim), first.diag())

lbl0 = axes[0].set_title("Filtered State")

lim0 = axes[0].set_xlim([-.5, 20])
bar1 = axes[1].bar(range(N_dim), ssv.diag())

lbl1 = axes[1].set_title("SSV")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl2 = axes[2].set_title("Initial Coherent State")

lim2 = axes[2].set_xlim([-.5, 20])

plt.show()
```

```python
second=Fock_Filter_prob(first,1,refl2,n2,eta,N_dim);
W2=wigner(second,xvec,xvec);

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W2, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W2, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of filtered state')
fid = fidelity(ssv,second)
print('fidelity with cat:',fid*fid)

fig, axes = plt.subplots(1, 3, figsize=(12,4))
bar0 = axes[0].bar(range(N_dim), second.diag())

lbl0 = axes[0].set_title("Twice Filtered State")

lim0 = axes[0].set_xlim([-.5, 20])
bar1 = axes[1].bar(range(N_dim), first.diag())

lbl1 = axes[1].set_title("First Filtered State")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl2 = axes[2].set_title("Initial Coherent State")

lim2 = axes[2].set_xlim([-.5, 20])
plt.show()
```

```python
third=Fock_Filter_prob(second,1,refl3,n3,eta,N_dim);
W3=wigner(third,xvec,xvec);

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W3, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W3, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of filtered state')
fid = fidelity(ssv,second)
print('fidelity with cat:',fid*fid)

fig, axes = plt.subplots(1, 4, figsize=(16,4))

bar0 = axes[0].bar(range(N_dim), third.diag())

lbl0 = axes[0].set_title("Third Filtered State")

lim0 = axes[0].set_xlim([-.5, 20])
bar1 = axes[1].bar(range(N_dim), second.diag())

lbl1 = axes[1].set_title("Second Filtered State")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), first.diag())

lbl2 = axes[2].set_title("First Filtered State")

lim2 = axes[2].set_xlim([-.5, 25])

bar3 = axes[3].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl3 = axes[3].set_title("Initial Coherent State")

lim3 = axes[3].set_xlim([-.5, 25])
plt.show()
```

```python
final_state=D(third,beta*(-1))

W_final=wigner(final_state,xvec,xvec)

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_ssv, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_ssv, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of squeezed cat state')

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_final, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_final, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Displaced Wigner function of |5>, |3> and |2> filtered coherent state')
fid = fidelity(ssv,final_state)
print('fidelity with ssv:',fid*fid)
plt.show()
fig, axes = plt.subplots(1, 5, figsize=(16,4))

bar0 = axes[0].bar(range(N_dim), first.diag())

lbl0 = axes[0].set_title("First Filtered")

lim0 = axes[0].set_xlim([-.5, 20])

bar1 = axes[1].bar(range(N_dim), second.diag())

lbl1 = axes[1].set_title("Second Filtered")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), third.diag())

lbl2 = axes[2].set_title("Third Filtered State")

lim2 = axes[2].set_xlim([-.5, 20])

bar3 = axes[3].bar(range(N_dim), final_state.diag())

lbl3 = axes[3].set_title("Final")

lim3 = axes[3].set_xlim([-.5, 8])

bar4 = axes[4].bar(range(N_dim), ssv.diag())

lbl4 = axes[4].set_title("SSV")

lim4 = axes[4].set_xlim([-.5, 10])
plt.show()

```

```python
from mpl_toolkits.axes_grid1 import AxesGrid

def shiftedColorMap(cmap, start=1, midpoint=0, stop=0, name='shiftedcmap'):
    '''
    Function to offset the "center" of a colormap. Useful for
    data with a negative min and positive max and you want the
    middle of the colormap's dynamic range to be at zero.

    Input
    -----
      cmap : The matplotlib colormap to be altered
      start : Offset from lowest point in the colormap's range.
          Defaults to 0.0 (no lower offset). Should be between
          0.0 and `midpoint`.
      midpoint : The new center of the colormap. Defaults to
          0.5 (no shift). Should be between 0.0 and 1.0. In
          general, this should be  1 - vmax / (vmax + abs(vmin))
          For example if your data range from -15.0 to +5.0 and
          you want the center of the colormap at 0.0, `midpoint`
          should be set to  1 - 5/(5 + 15)) or 0.75
      stop : Offset from highest point in the colormap's range.
          Defaults to 1.0 (no upper offset). Should be between
          `midpoint` and 1.0.
    '''
    cdict = {
        'red': [],
        'green': [],
        'blue': [],
        'alpha': []
    }

    # regular index to compute the colors
    reg_index = np.linspace(start, stop, 257)

    # shifted index to match the data
    shift_index = np.hstack([
        np.linspace(0, midpoint, 128, endpoint=False),
        np.linspace(midpoint, 1, 129, endpoint=True)
    ])

    for ri, si in zip(reg_index, shift_index):
        r, g, b, a = cmap(ri)

        cdict['red'].append((si, r, r))
        cdict['green'].append((si, g, g))
        cdict['blue'].append((si, b, b))
        cdict['alpha'].append((si, a, a))

    newcmap = cm.colors.LinearSegmentedColormap(name, cdict)
    plt.register_cmap(cmap=newcmap)

    return newcmap
#from matplotlib import rc,rcParams
#from pylab import *
```

```python
#activate latex text rendering
#rc('text', usetex=True)
#rc('axes', linewidth=2)
#rc('font', weight='bold')
#rcParams['text.latex.preamble'] = [r'\usepackage{sfmath} \boldmath']

xvec = np.arange(-26.,26.)*5./40
yvec = np.arange(-35.,35.)*5./40
X,Y = np.meshgrid(xvec, xvec)  ##Some plotting params
W=wigner(final_state,xvec,xvec);
orig_cmap = cm.seismic

shifted_cmap1 = shiftedColorMap(orig_cmap, midpoint=1-(W.max()/(W.max()-W.min())), name='shifted')

fig = plt.figure(figsize=(8,5))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot

ax1 = fig.add_subplot(1,1,1, projection='3d')
p = ax1.plot_surface(X, Y, W, rstride=1,cstride=1, cmap=shifted_cmap1, linewidth=0.5)
#lim=ax1.set_xlim([-4,2])
#lim=ax1.set_ylim([-1,1])

cb = fig.colorbar(p, shrink = 0.5)
cb.ax.tick_params(labelsize=12)
ax1.contour(X, Y, W, cmap=shifted_cmap1,linestyles="solid", offset=-0.35)
ax1.set_xlim([-4,4])
ax1.set_ylim([-4,4])
ax1.view_init(15, -40)

ax1.contour(X, Y, W,[0], zdir='y',cmap=cm.bwr,offset=4.5)
ax1.contour(X, Y, W,[0], zdir='x',cmap=cm.RdBu, offset=-4.5)
#plt.axis('off')
ax1.grid(b=False)
#plt.xlabel('P',fontsize=16,fontweight='heavy')
plt.xlabel('P',fontsize=16,fontweight='heavy')
plt.ylabel('X',fontsize=16,fontweight='heavy')
ax1.xaxis.set_tick_params(labelsize=12)
ax1.yaxis.set_tick_params(labelsize=12)
ax1.zaxis.set_tick_params(labelsize=12)

fig2 = plt.figure(figsize=(8,4))
ax1=fig2.add_subplot(1,2,1)
ax1.bar(range(N_dim),final_state.diag())
lim1 = ax1.set_xlim([-.5, 10.5])
lim1y = ax1.set_ylim([0,0.6])
plt.ylabel('Probability',fontsize='large')
plt.xlabel('Photon Number',fontsize='large')
plt.title('Approximate SSV')

ax2=fig2.add_subplot(1,2,2)
ax2.bar(range(N_dim),ssv.diag())
lim2 = ax2.set_xlim([-.5, 10.5])
lim2y = ax2.set_ylim([0,0.6])
plt.xlabel('Photon Number',fontsize='large')
plt.title('Ideal SSV')
#plt.subplots_adjust(wspace=0)
plt.show()
```

```python
'''Now plot the density matrix elements of the ideal state with that of
the approximate state resulting from photon-catalysis'''
s=np.abs((ssv.full()))
temp=np.delete(s,np.s_[10:],0)
state=np.delete(temp,np.s_[10:],1)
plt.imshow(state)
plt.colorbar()
plt.show()
```

```python
fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(221, projection='3d')
ax2 = fig.add_subplot(222, projection='3d')
ax3 = fig.add_subplot(223, projection='3d')

# fake data
_x = np.arange(10)
_y = np.arange(10)
_xx, _yy = np.meshgrid(_x, _y)
x, y = _xx.ravel(), _yy.ravel()

#top = x + y
diff=state.ravel()-state2.ravel()
bottom = np.zeros_like(top)
width = depth = 1

ax1.bar3d(x, y,0,1,1, np.abs(diff), shade=True)
ax1.set_title('Shaded')
ax1.set_zlim([0,.7])

ax2.bar3d(x, y, 0, 1,1, state.ravel(), shade=True)
ax2.set_title('Not Shaded')
ax2.set_zlim([0,.7])

ax3.bar3d(x, y, 0, 1,1, state2.ravel(), shade=True)
ax3.set_title('State')
ax3.set_zlim([0,.7])
ax1.view_init(55, -60)
ax2.view_init(55, -60)
ax3.view_init(55, -60)

plt.show()
```

```python
z=[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
```

```python
fig, ax = plt.subplots(1, 3, figsize=(15,4))
heatmap = ax[0].pcolor(state,vmin=0.0, vmax=0.6)
heatmap = ax[1].pcolor(state2,vmin=0.0, vmax=0.6)
heatmap = ax[2].pcolor(np.abs(state-state2),vmin=0.0, vmax=0.6)
cbar = plt.colorbar(heatmap)
cbar.ax.tick_params(labelsize=15)
ax[0].tick_params(labelsize=15)
ax[1].tick_params(labelsize=15)
ax[2].tick_params(labelsize=15)
plt.show()
```

```python
s=np.abs((ssv.full()))
temp=np.delete(s,np.s_[10:],0)
state=np.delete(temp,np.s_[10:],1)

s2=np.abs((final_state.full()))
temp1=np.delete(s2,np.s_[10:],0)
state2=np.delete(temp1,np.s_[10:],1)

#plt.imshow(state)
fig = plt.figure(figsize=(6,4))
plt.pcolormesh(state, vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()

#plt.imshow(state2)
fig = plt.figure(figsize=(6,4))
plt.pcolormesh(state2, vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()

fig = plt.figure(figsize=(6,4))
plt.pcolormesh(np.abs(state2-state), vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()
```

```python
s2=np.abs((final_state.full()))
temp1=np.delete(s2,np.s_[10:],0)
state2=np.delete(temp1,np.s_[10:],1)
plt.imshow(state2)
plt.colorbar()
plt.show()
```

```python
myarray=np.array([[5,1,5,77],[6,2,5,2],[7,3,5,3],[8,4,5,4],[8,9,6,2]])
new=np.delete(myarray,np.s_[2:],0)
final=np.delete(new,np.s_[2:],1)
```

```python

```

```python

```


# Signals and Sampling

In this notebook, we introduce signals and discuss the concept of sampling to convert a continuous into a discrete time axis. Using sinusoidal signals as example, we then cover phenomena such as aliasing and interference. This notebook is based on material from the
[FMP Notebook on Sampling](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C2/C2S2_DigitalSignalSampling.html) and the [FMP Notebook on Interference and Beating](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C2/C2S3_InterferenceBeating.html).

Signals

Sampling

Aliasing

Interference

Exercise 1: Beating

Exercise 2: Aliasing with Sinsuoids

Exercise 3: Visual Aliasing

## Signals

A **sound** is generated by a vibrating object such as the vocal cords of a singer, the string and soundboard of a violin, or the diaphragm of a kettledrum. In signal processing, such a sound is typically represented by a function or **signal** $f\colon\mathbb{R}\to\mathbb{R}$, which encodes the sound's air pressure changes over time. The signal is called **periodic** if high and low air pressures repeat in an alternating and regular fashion. In this case, the **period** of the signal is defined as the time required to complete a cycle. The **frequency**, measured in **Hertz** (Hz), is the reciprocal of the period. The prototype of such a periodic signal is a **sinusoid**, which is specified by its **frequency**, its **amplitude** (the peak deviation of the sinusoid from its mean), and its **phase** (determining where in its cycle the sinusoid is at time zero). In the following code cell, we provide a function for generating a sinusoid.

```python
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline

def generate_sinusoid(dur=1, amp=1, freq=1, phase=0, Fs=100):
    """Generation of sinusoid

    Notebook: PCP_signal.ipynb

    Args:
        dur: Duration (in seconds)
        amp: Amplitude of sinusoid
        freq: Frequency of sinusoid
        phase: Phase of sinusoid
        Fs: Sampling rate

    Returns:
        x: Signal
        t: Time axis (in seconds)
    """
    num_samples = int(Fs * dur)
    t = np.arange(num_samples) / Fs
    x = amp * np.sin(2 * np.pi * (freq * t - phase))
    return x, t

amp = 1.2
freq = 2
phase = 0.2
x, t = generate_sinusoid(dur=5, amp=amp, freq=freq, phase=phase, Fs=1000)

plt.figure(figsize=(8, 2.5))
plt.plot(t, x, color='blue', linewidth=2.0, linestyle='-')
plt.xlim([0, t[-1]])
plt.ylim([-amp*1.2, amp*1.2])
plt.xlabel('Time (seconds)')
plt.ylabel('Amplitude')
plt.title('Sinusoid with freq = %.1f, amp = %.1f, and phase = %.1f' % (freq, amp, phase))
plt.grid()
plt.tight_layout()
```

Interpreted as audio signal, one may listen to the sound of a sinusoid as long as its frequency lies in the human hearing range (roughly between $20$ and $20,000~\mathrm{Hz}$). To integrate an audio signal into a Jupyter notebook, we can use the module [`IPython.display`](https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html), which is an application programming interface (API) for displaying various tools in IPython. As for audio, one can use the class [`IPython.display.Audio`](https://ipython.org/ipython-doc/3/api/generated/IPython.display.html),  which creates an in-browser audio player. In the following code cell, this functionality is demonstrated using a sinusoid of $440~\mathrm{Hz}$.

```python
import IPython.display as ipd

Fs = 4000
dur = 1
x, t = generate_sinusoid(dur=dur, amp=1, freq=440, Fs=Fs)

ipd.display(ipd.Audio(x, rate=Fs))
```

<audio  controls="controls" >
    <source src="data:audio/wav;base64,UklGRmQfAABXQVZFZm10IBAAAAABAAEAoA8AAEAfAAACABAAZGF0YUAfAAAAAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrgAAllG6fSlwHi9z2O6TBIFiqPf3O0v5e9BzfzYs4HOYQoCyovbvlUS7eQF3qT0F6GGdAYBhnQXoqT0Bd7t5lUT277KiQoBzmCzgfzbQc/l7O0v392KoBIHuk3PYHi8pcLp9llEAAGquRoLXj+LQjScSbPx+nlcJCMW0B4QwjIHJ1B+NZ75/Tl0KEGu7RYb/iFfC+xefYv9/n2L7F1fC/4hFhmu7ChBOXb5/jWfUH4HJMIwHhMW0CQieV/x+EmyNJ+LQ149GgmquAACWUbp9KXAeL3PY7pMEgWKo9/c7S/l70HN/Nizgc5hCgLKi9u+VRLt5AXepPQXoYZ0BgGGdBeipPQF3u3mVRPbvsqJCgHOYLOB/NtBz+Xs7S/f3YqgEge6Tc9geLylwun2WUQAAaq5GgteP4tCNJxJs/H6eVwkIxbQHhDCMgcnUH41nvn9OXQoQa7tFhv+IV8L7F59i/3+fYvsXV8L/iEWGa7sKEE5dvn+NZ9QfgckwjAeExbQJCJ5X/H4SbI0n4tDXj0aCaq4AAJZRun0pcB4vc9jukwSBYqj39ztL+XvQc382LOBzmEKAsqL275VEu3kBd6k9BehhnQGAYZ0F6Kk9AXe7eZVE9u+yokKAc5gs4H820HP5eztL9/diqASB7pNz2B4vKXC6fZZRAABqrkaC14/i0I0nEmz8fp5XCQjFtAeEMIyBydQfjWe+f05dChBru0WG/4hXwvsXn2L/f59i+xdXwv+IRYZruwoQTl2+f41n1B+ByTCMB4TFtAkInlf8fhJsjSfi0NePRoJqrg==" type="audio/wav" />
    Your browser does not support the audio element.
</audio>

## Sampling

In digital signal processing, one often reduces a **continuous-time** (CT) signal $f\colon\mathbb{R}\to\mathbb{R}$ into a **discrete-time** (DT) signal $x\colon\mathbb{Z}\to\mathbb{R}$ by a procedure known as **equidistant sampling**. Fixing a positive real number $T>0$, the DT-signal $x$ is obtained by setting

\begin{equation}
   x(n):= f(n \cdot T)
\end{equation}

for $n\in\mathbb{Z}$. The value $x(n)$ is called the **sample** taken at time $t=n\cdot T$ of the original analog signal $f$. In short, this procedure is also called **$T$-sampling**. The number $T$ is referred to as the **sampling period** and the inverse $F_\mathrm{s}:=1/T$ as the **sampling rate**. The sampling rate specifies the number of samples per second and is measured in Hertz (Hz).

In the following code cell, we start with a CT-signal $f$ that is defined via linear interpolation of a DT-signal sampled at a high sampling rate (generated by the `generate_example_function`). In the plot, this CT-signal is visualized as black curve. Applying equidistant sampling (`sampling_equidistant`), we obtain the DT-signal visualized as the red stem plot.

```python
def generate_example_signal(dur=1, Fs=100):
    """Generate example signal

    Notebook: PCP_signal.ipynb

    Args:
        dur: Duration (in seconds) of signal to be generated
        Fs: Sampling rate

    Returns:
        x: Signal
        t: Time axis (in seconds)
    """
    N = int(Fs * dur)
    t = np.arange(N) / Fs
    x = 1 * np.sin(2 * np.pi * (1.9 * t - 0.3))
    x += 0.5 * np.sin(2 * np.pi * (6.1 * t - 0.1))
    x += 0.1 * np.sin(2 * np.pi * (20 * t - 0.2))
    return x, t

def sampling_equidistant(x_1, t_1, Fs_2, dur=None):
    """Equidistant sampling of interpolated signal

    Notebook: PCP_signal.ipynb

    Args:
        x_1: Signal to be interpolated and sampled
        t_1: Time axis (in seconds) of x_1
        Fs_2: Sampling rate used for equidistant sampling
        dur: Duration (in seconds) of sampled signal

    Returns:
        x_2: Sampled signal
        t_2: time axis (in seconds) of sampled signal
    """
    if dur is None:
        dur = len(t_1) * t_1[1]
    N = int(Fs_2 * dur)
    t_2 = np.arange(N) / Fs_2
    x_2 = np.interp(t_2, t_1, x_1)
    return x_2, t_2

Fs_1 = 100
x_1, t_1 = generate_example_signal(Fs=Fs_1, dur=2)

Fs_2 = 20
x_2, t_2 = sampling_equidistant(x_1, t_1, Fs_2)

plt.figure(figsize=(8, 2.2))
plt.plot(t_1, x_1, 'k')
plt.title('Original CT-signal')
plt.xlabel('Time (seconds)')
plt.ylim([-1.8, 1.8])
plt.xlim([t_1[0], t_1[-1]])
plt.tight_layout()

plt.figure(figsize=(8, 2.2))
plt.stem(t_2, x_2, linefmt='r', markerfmt='ro', basefmt='None', use_line_collection=True)
plt.plot(t_1, x_1, 'k', linewidth=1, linestyle='dotted')
plt.title(r'DT-Signal obtained by sampling with $F_\mathrm{s} = %.0f$' % Fs_2)
plt.xlabel('Time (seconds)')
plt.ylim([-1.8, 1.8])
plt.xlim([t_1[0], t_1[-1]])
plt.tight_layout()

plt.figure(figsize=(8, 2.2))
plt.stem(x_2, linefmt='r', markerfmt='ro', basefmt='None', use_line_collection=True)
plt.title(r'DT-Signal')
plt.xlabel('Time (samples)')
plt.ylim([-1.8, 1.8])
plt.xlim([0, len(t_2)])
plt.tight_layout()
```

## Aliasing

In general, sampling is a **lossy** operation in the sense that information is lost in this process and that the original CT-signal cannot be recovered from its sampled version. Only if the CT-signal has additional properties in terms of its frequency spectrum (it needs to be **bandlimited**), a perfect reconstruction is possible. This is the assertion of the famous **sampling theorem**. The sampling theorem also shows how the original CT-signal can be reconstructed by superimposing suitably shifted $\mathrm{sinc}$-functions weighted by the samples of the DT-signal.

Recap: Sampling Theorem

The sampling theorem, which is often associated with the names Harry Nyquist and Claude Shannon, states that a continuous-time (CT) signal that is bandlimited can be reconstructed perfectly under certain conditions. More precisely, a CT-signal $f\in L^2(\mathbb{R})$ is called $\Omega$-bandlimited if the Fourier transform $\hat{f}$ vanishes for $|\omega|>\Omega$ (i.e., $\hat{f}(\omega) = 0$ for $|\omega|>\Omega$). Let $f\in L^2(\mathbb{R})$ be an $\Omega$-bandlimited function and let $x$ be the $T$-sampled version of $f$ with $T:=1/(2\Omega)$ (i.e., $x(n)=f(nT)$, $n\in\mathbb{Z}$). Then $f$ can be reconstructed from $x$ by

$$
 f(t)=\sum_{n\in\mathbb{Z}}x(n)\mathrm{sinc}\left(\frac{t-nT}{T}\right)
 =\sum_{n\in\mathbb{Z}}f\left(\frac{n}{2\Omega}\right) \mathrm{sinc}\left(2\Omega t-n\right),
$$

where the $\mathrm{sinc}$-function is defined as

\begin{equation}
    \mathrm{sinc}(t):=\left\{\begin{array}{ll}
    \frac{\sin \pi t}{\pi t},&\mbox{ if $t\not= 0$,}\\
    1,&\mbox{ if $t= 0$.}
\end{array}\right.
\end{equation}

In other words, the CT-signal $f$ can be perfectly reconstructed from the DT-signal obtained by equidistant sampling if the bandlimit is no greater than half the sampling rate. This reconstruction based on the $\mathrm{sinc}$-function has been used in the function `reconstruction_sinc`.

Without additional properties, sampling may cause an effect known as **aliasing** where certain frequency components of the signal become indistinguishable. This effect is illustrated by the following figure. Using a high sampling rate, the original CT-signal can be reconstructed with high accuracy. However, when decreasing the sampling rate, the higher-frequency components are not captured well and only a coarse approximation of the original signal remains. This phenomenon is demonstrated by the following example.

```python
def reconstruction_sinc(x, t, t_sinc):
    """Reconstruction from sampled signal using sinc-functions

    Notebook: PCP_signal.ipynb

    Args:
        x: Sampled signal
        t: Equidistant discrete time axis (in seconds) of x
        t_sinc: Equidistant discrete time axis (in seconds) of signal to be reconstructed

    Returns:
        x_sinc: Reconstructed signal having time axis t_sinc
    """
    Fs = 1 / t[1]
    x_sinc = np.zeros(len(t_sinc))
    for n in range(0, len(t)):
        x_sinc += x[n] * np.sinc(Fs * t_sinc - n)
    return x_sinc

def plot_signal_reconstructed(t_1, x_1, t_2, x_2, t_sinc, x_sinc, figsize=(8, 2.2)):
    """Plotting three signals
    Notebook: PCP_signal.ipynb
    """
    plt.figure(figsize=figsize)
    plt.plot(t_1, x_1, 'k', linewidth=1, linestyle='dotted', label='Orignal signal')
    plt.stem(t_2, x_2, linefmt='r:', markerfmt='r.', basefmt='None', label='Samples', use_line_collection=True)
    plt.plot(t_1, x_sinc, 'b', label='Reconstructed signal')
    plt.title(r'Sampling rate $F_\mathrm{s} = %.0f$' % (1/t_2[1]))
    plt.xlabel('Time (seconds)')
    plt.ylim([-1.8, 1.8])
    plt.xlim([t_1[0], t_1[-1]])
    plt.legend(loc='upper right', framealpha=1)
    plt.tight_layout()
    plt.show()

Fs_2 = 40
x_2, t_2 = sampling_equidistant(x_1, t_1, Fs_2)
t_sinc = t_1
x_sinc = reconstruction_sinc(x_2, t_2, t_sinc)
plot_signal_reconstructed(t_1, x_1, t_2, x_2, t_sinc, x_sinc)

Fs_2 = 20
x_2, t_2 = sampling_equidistant(x_1, t_1, Fs_2)
t_sinc = t_1
x_sinc = reconstruction_sinc(x_2, t_2, t_sinc)
plot_signal_reconstructed(t_1, x_1, t_2, x_2, t_sinc, x_sinc)

Fs_2 = 10
x_2, t_2 = sampling_equidistant(x_1, t_1, Fs_2)
t_sinc = t_1
x_sinc = reconstruction_sinc(x_2, t_2, t_sinc)
plot_signal_reconstructed(t_1, x_1, t_2, x_2, t_sinc, x_sinc)
```

## Interference

In signal processing, **interference** occurs when a wave is superimposed with another wave of similar frequency. When a crest of one wave meets a crest of the other wave at some point, then the individual magnitudes add up for a certain period of time, which is known as **constructive interference**. Vice versa, when a crest of one wave meets a trough of the other wave, then the magnitudes cancel out for a certain period of time, which is known as **destructive interference**. We illustrate these phenomena in the following code cell.

```python
def plot_interference(t, x1, x2, figsize=(8, 2), xlim=None, ylim=None, title=''):
    """Plotting two signals and its superposition
    Notebook: PCP_signal.ipynb
    """
    plt.figure(figsize=(8, 2))
    plt.plot(t, x1, color='gray', linewidth=1.0, linestyle='-', label='x1')
    plt.plot(t, x2, color='cyan', linewidth=1.0, linestyle='-', label='x2')
    plt.plot(t, x1+x2, color='red', linewidth=2.0, linestyle='-', label='x1+x2')
    if xlim is None:
        plt.xlim([0, t[-1]])
    else:
        plt.xlim(xlim)
    if ylim is not None:
        plt.ylim(ylim)
    plt.xlabel('Time (seconds)')
    plt.ylabel('Amplitude')
    plt.title(title)
    plt.legend(loc='upper right')
    plt.tight_layout()
    plt.show()

dur = 5
x1, t = generate_sinusoid(dur=dur, Fs=100, amp=1, freq=1.05, phase=0.0)
x2, t = generate_sinusoid(dur=dur, Fs=100, amp=1, freq=0.95, phase=0.8)
plot_interference(t, x1, x2, xlim=[0, dur], ylim=[-2.2,2.2], title='Constructive Interference')

dur = 5
x1, t = generate_sinusoid(dur=dur, Fs=100, amp=1, freq=1.05, phase=0.0)
x2, t = generate_sinusoid(dur=dur, Fs=100, amp=1, freq=1.00, phase=0.4)
plot_interference(t, x1, x2, xlim=[0, dur], ylim=[-2.2,2.2], title='Destructive Interference')
```

## Exercises and Results

```python
import LibPCP.signal
show_result = True
```

Exercise 1: Beating

Let $f_1(t)=\sin(2\pi \omega_1 t)$ and $f_2(t)=\sin(2\pi \omega_2 t)$ be two sinusoids with distinct but nearby frequencies $\omega_1\approx\omega_2$. The superposition $f_1+f_2$ of these two sinusoids results in a function that looks like a single sine wave with a slowly varying amplitude, a phenomenon also known as beating. Mathematically, this phenomenon results from a trigonometric identity yielding

$$
\sin(2\pi \omega_1 t)+\sin(2\pi \omega_2 t)=
2\cos\left(2\pi\frac{\omega_1-\omega_2}{2}t\right)\sin\left(2\pi\frac{\omega_1+\omega_2}{2}t\right).
$$

If the difference $\omega_1-\omega_2$ is small, the cosine term has a low frequency compared to the sine term. As a result the signal $f_1+f_2$ can be seen as a sine wave of frequency $(\omega_1+\omega_2)/2$ with a slowly varying amplitude envelope of frequency $|\omega_1-\omega_2|$. Note that this rate is twice the frequency $(\omega_1-\omega_2)/2$ of the cosine term.

Use the functions generate_sinusoid and plot_interference to  illustrate the beating effect for various parameters $\omega_1$ and $\omega_2$.

In particular, consider $\omega_1=200$ and $\omega_2=203$ to generate a superimposed signal x at a sampling rate of Fs=4000. Listen to x using ipd.display(ipd.Audio(x, rate=Fs)).

```python
#<solution>
# Your Solution
#</solution>
```

```python
LibPCP.signal.exercise_beating(show_result=show_result)
```

Exercise 2: Aliasing with Sinsuoids

First, using x, t = generate_sinusoid(dur=2, Fs=128, freq=10), generate a sinusoid of frequency $\omega=10~\mathrm{Hz}$ at Fs=128 to mimic a CT-signal. Then, use the function sampling_equidistant to successively decrease the sampling rate using [64, 32, 20, 16, 12, 8, 4]. Visualize the result via plot_signal_reconstructed and discuss your observations.

```python
#<solution>
# Your Solution
#</solution>
```

```python
LibPCP.signal.exercise_aliasing_sinus(show_result=show_result)
```

Exercise 3: Visual Aliasing

Aliasing effects occur also in other areas such as the visual domain. One famous example is the wagon-wheel effect, where a wheel appears to rotate at a slower speed or even backwards when the speed is actually increasing. Identify some YouTube videos that illustrate this effect. Furthermore, integrate the video into your notebooks. To this end, do the following:

 Import the API for display tools in IPython: import IPython.display as ipd.

 Use the following commands: ipd.display, ipd.YouTubeVideo

```python
#<solution>
# Your Solution
#</solution>
```

```python
LibPCP.signal.exercise_aliasing_visual(show_result=show_result)
```


# Logistic Regression

- Logistic regression is a classification algorithm, don’t confuse with the name regression.
- Logistic Regression is basically Supervised ML algorithm that is used for __binary classification__
- In classification problem the values we predict take on only a small number of discrete values. For example, "yes or no", "cat or dog", "cat or dog or bird", etc.
- In logistic regression we estimate the probability of a certain discrete value.
- Our main goal is to __predict the probability of the discrete class labels__

Some examples of classification problems:
1. Spam detection
2. Detection of fraudulent transanctions
3. Malignant/Benign tumor

### Hypothesis for classification problem

In classification problem we want the probability of the outcomes. So we would like f(x) to be between 0 and 1.

\begin{equation}
0
\leq
f_\theta(x)
\leq
1
\end{equation}

* Mathematically, in Linear Regression __hypothesis function $f_{\theta}(x)$ can be >1 or <0__

* For Logistic Regression we need __hypothesis function which can satisfy the math rule:__ $0 \leq f_{\theta}(x)\leq 1$

#### Sigmoid function

Sigmoid function is used to restrict the f(x) between 0 and 1. Sigmoid function is given below:

$$\sigma(x)=\frac{1}{1+e^{-x}}$$

**Remark:** The derivative of logistic function can be written $\text{d}\sigma = \sigma (1-\sigma)$.

```python
'''Explore the signature of sigmoid function'''
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def sigmoid(x):
     y = 1 / (1 + np.exp(-x))
     return y

xdata = np.arange(-10, 10)
ydata = sigmoid(xdata)

plt.plot(xdata, ydata, 'o')
plt.show()
```

### Why do we need sigmoid function?

* This function will give us the __output value between [0,1]__

* When you __perform derivative of $e^{x}$ it gives you back $e^{x}$ and this function is the only known
  differentiable function with this mathematical property. That makes it more convenient for calculation.__

__Assumption of Logistic Regression:__

* Input space can be __separated into two nice ‘regions’, one for each class, by a defined boundary.__

* Your data must be linearly separable in __n dimensions__

### Intuitive Example to understand math behind LR

__Example / Problem statement: __

* We are provided a sample of 100 customers.

* We need to predict the probability whether a customer will buy (y) a particular magazine or not based on their
  age

* As you can see, we’ve a categorical outcome variable, we’ll use logistic regression.

#### Step 1:

To start with logistic regression, I’ll first write the simple linear regression equation with dependent variable

$$ \large Y = mx + C ------ (1) $$

Here,
   * Y = dependent variable ( user buy magazine or not)
   * m = slop
   * x  = independent variable ( Here I will take age of the readers for make it simple)
   * C = Y-intercept

#### Step 2:

So, final equation form based on our example will be

$$ \large Y_{readers\;will\;buy\; magazine\; or\; not}  = m * x_{age} + C ------(2)$$

* Now consider Y as function of probability. Why? because we need the chance (probability of the whether buyer will buy the magazine or not)

* This function is established using two things:
  * Probability of Success(p) means reader buys magazine
  * Probability of Failure(1-p) means reader doesn't buy magazine.

* Probability p should meet following criteria:
    1. It must always be positive (since p >= 0)
    2. It must always be less than equals to 1 (since p <= 1)

#### Step 3:

__Since probability must always be positive, we’ll put the linear equation in exponential form.__
For any value of slope and dependent variable, exponent of this equation will never be negative.

$$ \large p = exp(C + mx_{Age}) = e^{(C + mx_{Age})}    ------- (3) $$

Here,

* P  =  Probability of readers' buy magazine or not
* exp OR e =  exponential symbol OR Euler's number.

#### Step 4:

To make the probability less than 1, we must divide p by a number greater than p. This can simply be done by:

$$ \large p  =  \frac{exp(C + mx_{Age})}{exp(C + mx_{Age}) + 1}   =  \frac{ e^{(C + mx_{Age})}}{e^{(C + mx_{Age})} + 1 }   ----- (4)$$

#### Step 5:

Now by using equation 1, 2, 3 and 4 we can redefine the probability as:

 $$ p = \frac {e^{Y}}{ 1 + e^{Y}} ----- (5)$$

 * Here we have just __substitute the C + mx{Age} with Y__

 * Here p is the probability of success. This __Equation (5) is the Logit Function__

```

```

If p is the probability of success, 1-p will be the probability of failure which can be written as:

$$q = 1 - p = 1 - \frac {e^{Y}}{ 1 + e^{Y}}    --- (6)$$

* Here q is the probability of failure

#### Step 6:

Now let's __divide the equation 5 and 6__ so that we get following equation

$$ \frac {p}{1-p} = e^{Y}    --- (7)$$

#### Step 7:

After __taking log on both side__, So that we can get the value of Y

$$ \large log(\frac{p}{1-p}) = Y    --- (8) $$

Now, let's __put the value of Y in equation 8 from equation 2__

$$ \large log(\frac{p}{1-p}) = C + mx_{Age}   --- (9) $$

#### Notes:

* The __equation 9 is used in Logistic Regression and It is called Logit(p). __

* Here __(p/1-p) is called the Odd Ratio (OR) or Odds(p)__

* Whenever the __log of Odd Ratio is found to be positive, then the probability of success is always more than
  50%.__

* You can see __probability never goes below 0 and above 1. __

### Why we take odds instead of probabilities?

Let $P(X)$ denote the probability of an event X occurring. In that case, the __odds ratio (OR(X))__ is defined as

$$ \large \frac{P(X)}{1-P(X)}$$

* It is essentially the ratio of the probability of the event happening, vs. it not happening.

* It is clear that __probability and odds__ convey the __exact same information.__

* __But as $P(X)$ goes from 0 to 1, OR(X) goes from 0 to infinity__

* The advantage is that the __odds defined on 0 to infinity__ which map to __log-odds on -infinity to +infinity__

## Cost functions for Logistic Regression

#### Can we directly use cost function of linear regression which is Sum of sequared Errors?

* The Answer is __No, we can't__

* Linear regression uses mean squared error as its cost function. If this is used for logistic regression, then it
  will be generate a non-convex function of our input intercept values or parameter (theta) values

* Gradient Descent will converge into global minimum for function that is convex.

## Cost function for a value of f(x):

$$ \large \text{Cost function} (f_{\theta}(x))  = \begin{cases}
  -\log(f_{\theta}(x)) & \text{if $y = 1$} \\
  -\log(1\; -\; f_{\theta}(x) ) & \text{if $y = 0$}
\end{cases}$$

$$ \text{Cost function for entire data set with m observation} = J(\theta) = \frac{1}{m} \sum_{i=1}^{m} \text{Cost function} (h_{\theta}(x^{i},y^{i})) $$

### Combine both cases of Cost function in one equation

$$ \large \text{Cost function} (h_{\theta}(x),y)  =
  -y\log(h_{\theta}(x))-(1-y)\log(1\; -\; h_{\theta}(x) )
$$

$$\large J(\theta) = - \frac{1}{m}\sum_{i=1}^{m}[y^{(i)}\log(h_{\theta}(x^{(i)}) + (1-y^{(i)})\log(1-h_{\theta}(x^{(i)}))]$$

__Now we need to minimize this cost function $J(\theta)$__

### Gradient Descent

* In order to minimize the cost function we need to derive its __partial derivative with respect to $\theta$__

$$\large J(\theta) = - \frac{1}{m}\sum_{i=1}^{m}[y^{(i)}\log(\frac{1}{1+e^{-\theta x^{(i)}}}) + (1-y^{(i)})\log(1-\frac{1}{1+e^{-\theta x^{(i)}}})]$$

__Partial derivate that we use for updating our cost function or minimize our error function is given below__

$$\large \frac{\partial J}{\partial \theta}  = \frac{1}{m}\sum_{i=1}^{m}[x^{(i)}_{j}h_{\theta}(x^{(i)}) - y^{(i)}x^{(i)}_{j}]$$

#### Final Gradient descent equation is as below

__Repeat this until we get $min_{\theta}J(\theta)$:__

        {

$$ \large \theta_{j} := \theta_{j} -\alpha \sum_{i=0}^{m}[x^{(i)}_{j}h_{\theta}(x^{(i)}) - y^{(i)}x^{(i)}_{j}] $$

        }

__Here $\alpha$ is Learning Rate __

### Comparing Classification & Regression

|Properties| Supervised Classification (Logistic Regression) | Regression (Linear Regression)|
|:---:|:---: |:---:|
|  Output Type | Discrete Value (Class labels) | Continuous (number) |
|  What are you trying to achieve? | Decision Boundary or How to choose value for parameters theta | Best line of fit |
|  Evaluation | Accuracy | Sum of Sequared Error or $r^{2} Error$|

### Intution of logistic regression

Its about finding the decision boundary.

#### Scenario 1

```python
import matplotlib.pyplot as plt
x1 = np.random.randn(100)
y1 = np.random.randn(100)
x2 = np.random.randn(100)
y2 = np.random.randn(100) + 6
plt.plot(x1, y1, "+", x2, y2, "x")
plt.axis('equal')
plt.show()
```

#### Scenario 2

```python
x1 = np.random.randn(100)
y1 = np.random.randn(100) + 3
x2 = np.random.randn(100) + 3
y2 = np.random.randn(100)
plt.plot(x1, y1, "+", x2, y2, "x")
plt.axis('equal')
plt.show()
```

#### Scenario 3

```python
x1 = np.random.randn(100)
y1 = np.random.randn(100)
x3 = np.random.randn(25) + 3
y3 = np.random.randn(25)
x4 = np.random.randn(25) - 3
y4 = np.random.randn(25)
x5 = np.random.randn(25)
y5 = np.random.randn(25) + 3
x6 = np.random.randn(25)
y6 = np.random.randn(25) - 3
x3 = np.append(x3 , x5)
y3 = np.append(y3, y5)
x4 = np.append(x4, x6)
y4 = np.append(y4, y6)
x2 = np.append(x3, x4)
y2 = np.append(y3, y4)
#x2 = np.append(x3, x6)
#y2 = np.append(y2 , y6)
plt.plot(x1, y1, "+", x2, y2, "x")
plt.axis('equal')
plt.show()
```

### Digits classification example

```python
'''Load the dataset from sklearn'''
from sklearn.datasets import load_digits
digits = load_digits()
```

```python
'''Print the size of the data set'''
print("Data set size: ")
print ("Image Data Shape", digits.data.shape)
print("Label Data Shape", digits.target.shape)
```

    Data set size:
    Image Data Shape (1797, 64)
    Label Data Shape (1797,)

```python
import numpy as np
import matplotlib.pyplot as plt
```

#### Plot the figures

An image is represented by a matirx containing the pixel values of the image. The matrix has the same dimensions as the image.
A colour image has 3 matrices covering Red, Green and Blue spectrum. A grey scale image has only one matrix with values between 0 and 1.

```python
plt.figure(figsize=(20,4))
for index, (image, label) in enumerate(zip(digits.data[0:5], digits.target[0:5])):
    plt.subplot(1, 5, index + 1)
    plt.imshow(np.reshape(image, (8,8)), cmap=plt.cm.gray) # reshaping the image to 8 by 8 and specifying color map to grey scale
    plt.title('y =: %i\n' % label, fontsize = 20)
plt.show()
```

```python
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size=0.25, random_state=0)
```

```python
from sklearn.linear_model import LogisticRegression
classify_digits = LogisticRegression()
classify_digits.fit(x_train, y_train)
```

    /anaconda3/envs/forall/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
      FutureWarning)
    /anaconda3/envs/forall/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:460: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.
      "this warning.", FutureWarning)

    LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='warn',
              n_jobs=None, penalty='l2', random_state=None, solver='warn',
              tol=0.0001, verbose=0, warm_start=False)

```python
'''Plot first 10 test images'''
plt.figure(figsize=(9,4))
for index, (image, label) in enumerate(zip(x_test[0:10], y_test[0:10])):
    plt.subplot(2, 5, index + 1)
    plt.imshow(np.reshape(image, (8,8)), cmap=plt.cm.gray)
plt.show()
```

```python
'''Predict first test image'''
classify_digits.predict(x_test[0].reshape(1,-1))
```

    array([2])

```python
'''Predict the first 10 test images'''
classify_digits.predict(x_test[0:10])
```

    array([2, 8, 2, 6, 6, 7, 1, 9, 8, 5])

```python
'''Complete predicting the whole test set'''
predictions = classify_digits.predict(x_test)
```

```python
'''Score the prediction'''
score = classify_digits.score(x_test, y_test)
print("score = ", score)
```

    score =  0.9533333333333334

```python

```

```python

```


```python
# Models from Introduction to Algorithmic Marketing
# https://algorithmicweb.wordpress.com/
#
# Markov chain-based LTV model predicts customer lifetime value
# using the probabilities of transition between different customer states
```

```python
%matplotlib inline
import sympy as sy
import numpy as np
import matplotlib.pyplot as plt
```

```python
p1 = 0.7                # Probability of a purchase after 1 month of silence
p2 = 0.5                # Probability of a purchase after 2 months of silence
p3 = 0.3                # Probability of a purchase after 3 months of silence
r = 100                 # Revenue associated with one purchase, $
c = 5                   # Cost of one promotion
d = 0.001               # Inflation rate for one month
projection_horizon = 10 # LTV projection horizon, months
```

```python
# Transition probabilities
P = np.matrix([
[  p1,  1-p1,   0,    0   ],
[  p2,   0,    1-p2,  0   ],
[  p3,   0,     0,   1-p3 ],
[  0,    0,     0,    0]
])

# Expected revenue
G = np.matrix([
[  r-c  ], # If customer goes to the initial state (purchases), we get the revenue minus promotion cost
[  -c   ], # If customer is silent for one month, we send promotion and lose c dollars
[  -c   ], # If customer is silent for two months, we send promotion and lose c dollars
[  0    ]  # If customer is silent for three month, we stop to send promotions
])
```

```python
# Calculate and visualize the LTV. Each line on the plot corresponds to a customer state
#
# For example, the topmost line corresponds to the first state (just purchased)
# It indicates that the cumulative revenue from this cusomer is likely to grow over
# next projection_horizon months.
#
# The bottommost line corresponds to customers who are silent for 3 months
# This customers are considred lost, so this line is constantly zero
ltv_dynamics = [
   sum(
      (1/((1+d)**(t-1))) * (P**t)*G
      for t in range(1, T+1)
   ) for T in range(1, projection_horizon)
]
plt.plot(np.concatenate(ltv_dynamics, axis=1).T);
```

```python

```


```python
!pip install nangs
```

    Collecting nangs
      Downloading https://files.pythonhosted.org/packages/4c/cd/7eeca93e6088a85d18ff99897743aaf429c94c12049e56f2a1e8b2091656/nangs-0.1.2-py3-none-any.whl
    Installing collected packages: nangs
    Successfully installed nangs-0.1.2

```python
pip install --upgrade git+https://github.com/adantra/nangs.git
```

    Collecting git+https://github.com/adantra/nangs.git
      Cloning https://github.com/adantra/nangs.git to /tmp/pip-req-build-0txwxjz5
      Running command git clone -q https://github.com/adantra/nangs.git /tmp/pip-req-build-0txwxjz5
    Building wheels for collected packages: nangs
      Building wheel for nangs (setup.py) ... done
      Created wheel for nangs: filename=nangs-0.1.2-cp37-none-any.whl size=13860 sha256=c31b8b0729486e3cf39414c49ba3279f1759cb7081471c091ccc4e53d0bf9299
      Stored in directory: /tmp/pip-ephem-wheel-cache-i32_6lnc/wheels/2c/7f/7f/677931a26064faa3c2f5f1f88c96cae03fadc9cffb901d8b1e
    Successfully built nangs
    Installing collected packages: nangs
    Successfully installed nangs-0.1.2

```python
# autoreload nangs
%reload_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
!nvidia-smi -L
```

    GPU 0: Tesla P100-PCIE-16GB (UUID: GPU-5154ead1-ca7b-9003-94b6-ff8699cbe32b)

```python
#imports

import numpy as np
import matplotlib.pyplot as plt
import torch
import pandas as pd
import nangs
from nangs import *

device = "cuda" if torch.cuda.is_available() else "cpu"

nangs.__version__, torch.__version__
```

    ('0.1.2', '1.9.0+cu102')

```python
device

```

    'cuda'

```python
alpha=10
class Difussion2d(PDE):
    def computePDELoss(self, inputs, outputs):
        p = outputs[:, 0]
        grads = self.computeGrads(p, inputs)
        dpdx, dpdt = grads[:, 0], grads[:, 1]
        # second order derivatives
        dp2dx2 = self.computeGrads(dpdx, inputs)[:, 0]
        #dp2dy2 = self.computeGrads(dpdy, inputs)[:, 1]
        return {'pde':dpdt - dp2dx2}
```

```python
class NeumannX(Neumann):
    def computeBocoLoss(self, inputs, outputs):
        p = outputs[:, 0]
        grads = self.computeGrads(p, inputs)
        dpdx = grads[:, 0]
        return {'gradX': 100*dpdx}
```

```python
class PBC0(Neumann):
    def computeBocoLoss(self, inputs, outputs):
        pbc = 0
        p = outputs[:, 0]
        return {'PWF': 100*(p-pbc)}
```

```python
pde = Difussion2d(inputs=('x', 't'), outputs='p')
```

```python
def append(arr, values, axis=None):
    arr = np.asanyarray(arr)
    if axis is None:
        if arr.ndim != 1:
            arr = arr.ravel()
        values = np.ravel(values)
        axis = arr.ndim-1
    return np.concatenate((arr, values), axis=axis)
```

```python
# mesh

x = np.linspace(0,1,100)

t = np.linspace(0,1,100)
#t = append(t,np.linspace(0.2,1,10))
#t=np.array([0])
#tt=np.logspace(-2,0,20)
#t = append(t,tt)
#mesh = Mesh({'x': x, 't':t}, device=device)
#pde.set_mesh(mesh)
```

```python
#x = np.append(np.linspace(0,.01,100),x)

#t = np.append(np.linspace(0,.01,100),t)
#t = append(t,np.linspace(0.2,1,10))
#t=np.array([0])
#tt=np.logspace(-2,0,20)
#t = append(t,tt)
mesh2 = Mesh({'x': x, 't':t}, device=device)
pde.set_mesh(mesh2)
```

```python
from nangs import Dirichlet
```

```python
t0 = np.array([0])
```

```python
_x = np.meshgrid(x)
#p0 = np.sin(2*np.pi*_x)*np.sin(2*np.pi*_y)
```

```python

_x[0].shape
```

    (100,)

```python
p0=np.zeros((len(x)))
p0[:]=1
p0[0]=0
#p0[47:53,47:53]=1
```

```python
p0.shape
```

    (100,)

```python
initial_condition = Dirichlet({'x': x,'t':t0}, {'p': p0.reshape(-1)}, device=device, name="initial")
```

```python
p0.shape
```

    (100,)

```python
pde.add_boco(initial_condition)
#p0=np.ones(len(y))
#p1=np.zeros(len(y))
```

```python
pwf = PBC0({'x': np.array([0]),'t':t}, name='pwf', device=device)
#left = Dirichlet({'x': x, 'y': y}, {'p': p0.reshape(-1)}, device=device, name="left")
```

```python

pde.add_boco(pwf)
```

```python
#right = PBC0({'x': np.array([1]), 'y': y, 't':t}, name='right', device=device)
#right = Dirichlet({'x': x[-1], 'y': y}, {'p': p1.reshape(-1)}, device=device, name="right")
right =  NeumannX({'x': x[-1],'t':t}, name='right', device=device)
```

```python
pde.add_boco(right)
```

```python
#from nangs import MLP

BATCH_SIZE = 64
LR = 0.1e-2
EPOCHS = 2
NUM_LAYERS = 4
NUM_HIDDEN = 50

mlp = MLP(len(pde.inputs), len(pde.outputs), NUM_LAYERS, NUM_HIDDEN).to(device)
optimizer = torch.optim.Adam(mlp.parameters())
scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=LR, pct_start=0.1, total_steps=EPOCHS)
```

```python
mlp
```

    MLP(
      (mlp): Sequential(
        (0): Linear(in_features=2, out_features=50, bias=True)
        (1): Sequential(
          (0): GELU()
          (1): Linear(in_features=50, out_features=50, bias=True)
        )
        (2): Sequential(
          (0): GELU()
          (1): Linear(in_features=50, out_features=50, bias=True)
        )
        (3): Sequential(
          (0): GELU()
          (1): Linear(in_features=50, out_features=50, bias=True)
        )
        (4): Sequential(
          (0): GELU()
          (1): Linear(in_features=50, out_features=1, bias=True)
        )
      )
    )

```python

EPOCHS = 300
scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=LR, pct_start=0.1, total_steps=EPOCHS)
```

```python
pde.compile(mlp, optimizer, scheduler)
```

```python

%time hist = pde.solve(EPOCHS, BATCH_SIZE)
```

```python
hist['pde'][-1] #512
```

    0.010966393444687129

```python
hist['pde'][-1] #256
```

    0.006426302855834365

```python
hist['pde'][-1] #128
```

    0.002856906867421032

```python
hist['pde'][-1] #64 - 300 - 4
```

    0.002376913167921838

```python
hist['pde'][-1] #64 - 500 - 4
```

    0.0184016907884247

```python
hist['pde'][-1] #64 - 300 - 5
```

    0.007399362147115408

```python
hist['pde'][-1] #64 - 300 - 3
```

    0.002850979857241652

```python
hist['pde'][-1] #64 - 300 - 3
```

    0.0025286150037364405

```python
t = 0.0
x = np.linspace(0,1,100)
#y = np.linspace(0,1,100)

_x, _t = np.meshgrid(x, t)
eval_mesh = Mesh({'x': x,  't':t}, device=device)
p = pde.eval(eval_mesh)
p = p.cpu().numpy()
#p5 = p.reshape((_y.shape[0],_x.shape[1]))
```

```python
p,grad,loss=pde.eval_with_grad(eval_mesh)
```

```python
ppp=[]
loss = []
x = np.linspace(0.0,1.,10)
qq=[]
for t in np.linspace(0.01,1,12):
  #_x,_t = np.meshgrid(x,t)
  eval_mesh = Mesh({'x': x,  't':t}, device=device)
  pp,g,l = pde.eval_with_grad(eval_mesh)
  qq.append(g)
  loss.append(l)
  #pp = pp.cpu().numpy()
  ppp.append(pp)
```

```python
for l in loss:
  plt.plot(np.power(l,2))
```

```python
q0=[]
qL=[]
for q in qq:
  plt.plot(q.detach().cpu().numpy()[0])
  q0.append(q.detach().cpu().numpy()[0][0])
  qL.append(q.detach().cpu().numpy()[0][-1])
```

```python
plt.plot(q0)
```

```python
plt.plot(qL[10:])
```

```python
from sympy import Sum, factorial, oo, IndexedBase, Function, sin,exp
from sympy.abc import n
```

```python
def diff_sol(x,t,alpha=0.1):
    sol= Sum(4/((2*n + 1)*np.pi)*sin((2*n+1)*np.pi/2*x)*exp(-((2*n + 1)*np.pi/2)**2*t*alpha),(n,0,50))

    return sol #,sol_n
```

```python

out=pde.eval_with_grad(eval_mesh)
```

```python
out_np=out[1].detach().cpu().numpy()
```

```python
out_np.shape
```

    (100, 2)

```python
#xxx=np.linspace(0,1,10)
x = np.linspace(0,1,10)
for t in np.linspace(0.01,1,10):
    d=[]
    for xx in x:
        d.append(diff_sol(xx,t,alpha=1).doit())
    plt.plot(d,marker='o',color='y')
    eval_mesh = Mesh({'x': x,  't':t}, device=device)
    pp = pde.eval(eval_mesh)
    pp = pp.cpu().numpy()
    plt.plot(pp,marker='+',linestyle='None',color='black')
```

```python
x = np.linspace(0,1,10)
for t in np.linspace(0.01,1,10):
    d=[]
    for xx in x:
        d.append(diff_sol(xx,t,alpha=1).doit())
    plt.plot(d,marker='o',color='y')
    eval_mesh = Mesh({'x': x,  't':t}, device=device)
    pp = pde.eval(eval_mesh)
    pp = pp.cpu().numpy()
    plt.plot(pp,marker='+',linestyle='None',color='black')
```

```python
x = np.linspace(0,1,10)
for t in np.linspace(0.1,1,10):
    d=[]
    for xx in x:
        d.append(diff_sol(xx,t,alpha=1).doit())
    #plt.plot(d,marker='o',color='y')
    eval_mesh = Mesh({'x': x,  't':t}, device=device)
    pp = pde.eval(eval_mesh)
    pp = pp.cpu().numpy()
    plt.plot(pp.reshape(-1)-np.array(d),marker='o',linestyle='None')
```

```python
pp.reshape(-1)-np.array(d)
```

    array([9.82690835371614e-5, 1.91111856493176e-5, -0.000271523194577829,
           -0.000412734673227264, -0.000213420091990463, 0.000191083336638975,
           0.000605168526016372, 0.000920534883478999, 0.00109250133816152,
           0.00112512274501220], dtype=object)

```python
pp
```

    array([[9.8269084e-05],
           [1.8769128e-02],
           [3.6658801e-02],
           [5.3575788e-02],
           [6.9192886e-02],
           [8.2906298e-02],
           [9.4116032e-02],
           [1.0238577e-01],
           [1.0742913e-01],
           [1.0910217e-01]], dtype=float32)

```python

```


Traçar um esboço do gráfico e obter uma equação da parábola que satisfaça as condições dadas.

12. Foco: $F(2,0)$; Firetriz $d: x+2=0$

Arrumando a equação da diretriz

$x = -2$

Se a diretriz é dada por $-\frac{p}{2}$ e possui o valor $x = -2$ e o foco possui o valor $2$ para $x$ e sabendo que a distância do foco para o vértice e do vértice para a diretriz são iguais, temos que $V(0,0)$, para que isso seja verdadeiro.

$D : x = -2$

$F(2,0)$

Descobrindo o valor de $p$

$F = \frac{p}{2}$

$F = 2$

$\frac{p}{2} = 2$

$p = 4$

Montando a equação

$y^2 = 2px$

$y^2 = 2 \cdot 4 \cdot x$

$y^2 = 8x$

Gráfico da parábola

```python
from sympy import *
from sympy.plotting import plot_implicit
x, y = symbols("x y")
plot_implicit(Eq((y-0)**2, 8*(x+0)), (x,-10,10), (y,-10,10),
title=u'Gráfico da parábola', xlabel='x', ylabel='y');
```

Problemas Propostos. Pag. 172 - 175/12.ipynb