# Image deblurring

**Authors: M. Ravasi, D. Vargas, I. Vasconcelos**

Welcome to the second session of our **Solving large-scale inverse problems in Python with PyLops** tutorial!

Throughout this tutorial, we aim at developing the following aspects in image processing:

- We connect with the concepts learned in the previous session by defining a simple PyLops operator implementing a convolutional kernel for image processing
- Learn how to use build-in PyLops operators for image manipulation including, blurring, sharpening, and edge detection
- Demonstrate the versatility of the linear operators when implemented in the solution of inverse problems, in particular, the image deblurring problem
- Analyze the influence of noise in contrast with the performance of different solvers, least-squares, TV-Regularization, and FISTA

## Useful links

- Tutorial Github repository: https://github.com/mrava87/pylops_pydata2020

- PyLops Github repository: https://github.com/equinor/pylops

- PyLops reference documentation: https://pylops.readthedocs.io/en/latest/

# Deblurring Images

Compact digital cameras were introduced in the early seventies thanks to the development of CCD sensors allowing the recording and storage of images in digital format. The goal of an image process is to capture an accurate representation of a particular scene. However, any image is influenced by the optical system itself and the overall effect is a blurred image reconstruction, therefore, specialized algorithms performing deblurring need to be implemented in order to achieve a sharper image. Image deblurring is a method that aims at recovering the original sharp-image by removing effect caused by limited aperture, lens aberrations, defocus, and unintended motions. This can be done by defining a mathematical model of the blurring process with the idea of removing from the image the blurring effects.

Traditionally, the main assumption is that the blurring process is linear. When this is the case, a 2-dimensional convolutional model describes the imaging problem. The blurred Image (output) is the result of a point-spread function acting on a target sharp image (input). Such process is described by the following transformation,

\begin{equation}
g(x, y) = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} h(x-x_0,y-y_0) f(x_0, y_0) dx_0 dy_0.
\end{equation}

This operation can be discretized as follows

\begin{equation}
g[i,n] = \sum_{j=-\infty}^{\infty} \sum_{m=-\infty}^{\infty} h[i-j,n-m] f[j,m]
\end{equation}

and also implemented in the frequency domain using the convolution theorem

\begin{equation}
G(k_x, k_y) = H(k_x, k_y) F(k_x, k_y).
\end{equation}

Here, the spectrum of the point-spread function $H(k_x, k_y) = \mathscr{F} \{h(x,y)\}$ is called transfer function, similarly, $F(k_x, k_y) = \mathscr{F} \{f(x,y)\}$ is the spectrum of the sharp image. The previous definition is an specific case of the more general problem $\mathbf{y} =  \mathbf{A} \mathbf{x}$ (*forward operation*), where $\mathbf{A}:\mathbb{F}^m \to \mathbb{F}^n$ is a linear operator that maps a vector of size $m$ in the *model space* to a vector of size $n$ in the *data space*, as explained in the previous session. The (*adjoint operation*) is $\mathbf{x} = \mathbf{A}^H \mathbf{y}$. In this context, our deblurring operation is an inverse problem design to find a pseudoinverse, $\mathbf{A}^{-1}$, that aims at removing the effect of the operator $\mathbf{A}$ from the data $\mathbf{y}$ to retrieve the model $\mathbf{x}$, i.e., $\hat{\mathbf{x}} = \mathbf{A}^{-1} \mathbf{y}$.

This Jupyter Notebook is intended to guide you through the main steps in the implementation of the blurring and deblurring process using the pylops.signalprocessing.Convolve2D operator, in this case, we assume to know the structure of the point-spread function for an specific problem.

```python
# Run this when using Colab (will install the missing libraries)
# !pip install pylops scooby
```

```python
%matplotlib inline

import numpy as np
from skimage import data
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
from scipy.signal import convolve, correlate
import scooby

from pylops.optimization import leastsquares, sparsity
from pylops.signalprocessing import Convolve2D
from pylops import FirstDerivative, Laplacian
from pylops import LinearOperator
from pylops.utils import dottest
```

# A simple  deconvolution problem

The first problem we will solve with the aid of pylops is a simple one-dimensional convolution. We start by defining a pylops operator applying convolution with a given impulse response to a model vector. For simplicity, we select a *[ricker](https://en.wikipedia.org/wiki/Mexican_hat_wavelet)* wavelet and used it as a compact filter to be convolved with a 1D input signal. Assuming our input signal (model) is a time series constructed by superposition of three gaussian functions with different time shifts, the forward problem can be interpreted as a weighted sum of the function $f(\tau)$ at the moment $t$ with weights given by $h$.

The resulting signal (data), $g(t)$, is a 'blurred' shifted version of the input model mathematically computed by integration over time according to

\begin{equation}
g(t) = \int_{-\infty}^{\infty} h(t - \tau) f(\tau) d\tau
\end{equation}

Next, we solve the inverse problem (**deconvolution**) where we ask the question. What model vector would reproduce a given data set when convolved with a specific convolutional kernel?. Here we try to reconstruct an unknown model based on the given data (blue line in the figure) and impulse response (red line in the figure). The expected output of this operation should approximate our target model (black_line in the figure)

**Pylops convolution operator:**
A simple pylops operator is defined as a Child class that inherits from **[`pylops.LinearOperator`](https://pylops.readthedocs.io/en/latest/api/generated/pylops.LinearOperator.html#pylops.LinearOperator)**. We initilize the class by defining at least three atributes in the the class constructor; `shape`, `dtype`, and `explicit`. Next, we difine the forward mode operation through the method `_matvec`, and `_rmatvec` for the adjoint mode.

```python
# Defining a convolution Pylops operator
class my_convolve(LinearOperator):
    def __init__(self, A, dims, dtype="float64"):
        self.A = A
        self.dims = dims
        self.shape = (np.prod(self.dims), np.prod(self.dims))
        self.dtype = np.dtype(dtype)
        self.explicit = False

    def _matvec(self, x):
        x = np.reshape(x, self.dims)
        y = convolve(x, self.A, mode='same')
        return np.ndarray.flatten(y)

    def _rmatvec(self, x):
        x = np.reshape(x, self.dims)
        y = correlate(x, self.A, mode='same')
        return np.ndarray.flatten(y)
```

```python
t_min, t_max, nt = -4.0, 4.0, 10000
f_central = 15
t_delay = 1
t_delay_wav = np.array([0.0, 0.6, 1.2])[:, np.newaxis]

# Time axis
t = np.linspace(t_min, t_max, nt)
sigma = 1 / (np.pi * f_central) ** 2

# Model vector
wav = np.exp(-((t - t_delay_wav) ** 2) / sigma)
model_ = wav[0] - 1.5*wav[1] + wav[2]

# Impulse response - Convolutional Operator
impulse_response = (1 - (2 * (t - t_delay) ** 2) / sigma) * \
    np.exp(-((t - t_delay) ** 2) / sigma)

# Data vector
R_op = my_convolve(A=impulse_response, dims=(nt))
data_ = R_op * model_

# Model reconstruction - Inverse problem
model_reconstructed = sparsity.FISTA(Op=R_op,
                                    data=data_,
                                    eps=1e-1,
                                    niter=200)[0]

# Plotting input vectors
fig, ax = plt.subplots(1, 3, figsize=[10, 3], facecolor='w', edgecolor='k')
ax[0].plot(t, impulse_response, 'r', lw=1.5)
ax[0].set_xlabel('Time - (sec)')
ax[0].set_ylabel('Amplitude')
ax[0].set_xlim([0.8, 1.2])
ax[0].set_title("Operator")

ax[1].plot(t, model_, 'k', lw=1.5)
ax[1].set_xlabel('Time - (sec)')
ax[1].set_xlim([-0.1, 2.5])
ax[1].set_title("Model")

ax[2].plot(t, data_, 'b', lw=1.5)
ax[2].set_xlabel('Time - (sec)')
ax[2].set_xlim([-0.1, 2.5])
ax[2].set_title("Data")
fig.tight_layout()

# Plotting Model reconstruction vectors
fig = plt.figure(figsize=[4, 3], facecolor='w', edgecolor='k')
ax = fig.add_subplot(1, 1, 1)
ax.plot(t, model_, 'k', lw=3, label='Target Model')
ax.plot(t, model_reconstructed, '--g', lw=2, label='Reconstruction')
ax.set_ylabel('Amplitude')
ax.set_xlabel('Time - (sec)')
ax.set_xlim([-0.1, 1.5])
ax.set_title("Model Reconstruction")
ax.legend()
```

```python
R_op
```

# Image filtereing

PyLops comes with multiple build-in operators ready to use. In this section, we are using `pylops.signalprocessing.Convolve2D` to perform convolutions. First, we need to define a compact filter to construct the actual operator and then apply it to a pre-loaded image. Before we do that, let's first see if under the given parameters this linear operator passes the dot test. Once we make sure that this is the case, we can compute the convolution using NumPy matrix-like syntax.

In the cell below, we select the most simple point spread function, the identity kernel. It should not blur the image and the error of the output compared to the input should be very low (to machine precession)

Let's start by uploading an image. Here we use the **scikit-image** module skimage.data containing standard test images. The very first line selects the image of your preference to be used in the next cells, feel free to play around, and even upload your own pictures.

```python
image = data.camera()         # Load image from scikit-image
# image = data.binary_blobs()
# image = data.checkerboard()
ny, nx = image.shape

# Identity kernel
kernel = np.zeros([3, 3])
kernel[1, 1] = 1

# Define the PyLops blurring Operator
C_op = Convolve2D(N=ny*nx,
                  h=kernel,
                  offset=(kernel.shape[0]//2, kernel.shape[1]//2),
                  dims=(ny, nx))

# Dot test
dottest(C_op, ny*nx, ny*nx, verb=True)

img_blur = C_op * image.flatten()
img_blur = img_blur.reshape(image.shape)

# PLOTTING
fig, ax = plt.subplots(1, 3, figsize=(15, 5))
ax[0].imshow(image, cmap='gray')
ax[0].set_title('Input image')

ax[1].imshow(img_blur, cmap='gray')
ax[1].set_title('Response to identity')

him = ax[2].imshow(img_blur-image, cmap='gray')
ax[2].set_title('Error output-input')
fig.colorbar(him, ax=ax[2])
```

```python
print("image.shape {}".format(image.shape))
C_op
```

**EX: Write your own kernel.** A very common practice in digital image processing consists of applying filters as a way to highlight specific image features. One of the most useful is known as unsharp mask and is used attenuate low frequencies features while enhancing small contrast changes. A very simple version of this sharpening filter is given by the matrix kernel,

$$
\mathbf{A} =
\begin{bmatrix}
    0 & -1 & 0\\
    -1 & 5 & -1\\
    0 & -1 & 0
\end{bmatrix}
$$

Define a pylops operator for this mask and apply it to your preferred image. A code snippet for the plotting the input/output images is given below

```python
# %load -s Unsharp_Mask solutions/deblurring_sol.py
```

```python
# Unsharp_Mask()
```

#### Edge detection - Sobel operator

Edge detection is a process that tries to identify areas on an image where strong changes in intensity take place. A simple way of performing this task is by implementing a Sobel operator. This kind of mask approximates the directional derivates of an image along with directions $x$, and $y$. Vertical, $\mathbf{G_x}$, and Horizontal, $\mathbf{G_y}$, gradients are give by the 3x3 kernels

\begin{equation}
    \mathbf{G_x}=
    \begin{bmatrix}
        1 & 2 & 1\\
        0 & 0 & 0\\
        -1 & -2 & -1
    \end{bmatrix}
   \qquad\text{,}\qquad
    \mathbf{G_y} =
    \begin{bmatrix}
        1 & 0 & -1\\
        2 & 0 & -2\\
        1 & 0 & -1
    \end{bmatrix}
\end{equation}

note that each mask is designed to detect either horizontal or vertical changes. A Combination of these gradient approximations provides the total gradient magnitude, $\mathbf{G} = \sqrt{\mathbf{G_x}^2 + \mathbf{G_x}^2}$, at an specific pixel of the image.

As we did in the previous exercise, we now implement edge detection by defining vertical and horizontal gradient kernels, apply them to an image, and assemble the gradient magnitude.

```python
# vertical edge detector
kernel_v = np.array([[1, 0, -1],
                     [2, 0, -2],
                     [1, 0, -1]])

# horizontal edge detector
kernel_h = np.array([[1, 2, 1],
                     [0, 0, 0],
                     [-1, -2, -1]])

# Define the PyLops blurring Operator
Sobel_x_op = Convolve2D(N=ny*nx,
                        h=kernel_h,
                        offset=(kernel_h.shape[0]//2, kernel_h.shape[1]//2),
                        dims=(ny, nx))
Sobel_y_op = Convolve2D(N=ny*nx,
                        h=kernel_v,
                        offset=(kernel_v.shape[0]//2, kernel_v.shape[1]//2),
                        dims=(ny, nx))

Gx = Sobel_x_op * image.flatten()
Gy = Sobel_y_op * image.flatten()
Gx = Gx.reshape(image.shape)
Gy = Gy.reshape(image.shape)

# Gradient magnitude
G = np.sqrt(Gy**2 + Gy**2)

# PLOTTING
fig, ax = plt.subplots(1, 3, figsize=(15, 5))

ax[0].imshow(image, cmap='gray')
ax[0].set_title('Image')

ax[1].imshow(kernel_v, cmap='gray')
ax[1].set_title('vertical Kernel')

ax[2].imshow(G, cmap='gray')
ax[2].set_title('Image edges')

```

# Image Blurring

The Gaussian filter is an operator used to suppress noise and reduce the contrast of an image. In this case, an image is rendered smooth under the action of a Gaussian mask by removing the high-frequency components.  Therefore, we can understand a Gaussian blur as a low-pass filter, attenuating the high-frequency components in an image. Similarly, the maximum value of the weighted sum in the convolution located around the center of the distribution decreases as a function of distance. Mathematically the two-dimensional Gaussian function is defined as

\begin{equation}
G(x,y) = \frac{1}{2 \pi \sigma ^{2}} e^{- \frac{x^{2} + y^{2}}{2 \sigma ^{2}}}
\label{2dgaussian}
\end{equation}

where the standard deviation of the kernel is given for each axis and determines the effective extension of the filter.

In the following cell, we will prepare a data vector represented by a Gaussian-blurred image as preparation for the next section, where we will use the smoothed image together with the given kernel and try to remove the blurring effects by solving a deconvolution problem

```python
# Play around with this parameters
mx, my = 25, 15                 # 25, 15
sigma_x, sigma_y = 4.5, 2.5     # 4.5, 2.5

x = np.linspace(-mx//2, mx//2, mx)
y = np.linspace(-my//2, my//2, my)[..., None]
kernel_gauss = 1/(2*np.pi*sigma_x*sigma_y) * \
    np.exp(-(x**2/(2*sigma_x**2) + y**2/(2*sigma_y**2)))

# Define the PyLops blurring Operator
Gauss_op = Convolve2D(N=ny*nx,
                      h=kernel_gauss,
                      offset=(kernel_gauss.shape[0]//2,
                              kernel_gauss.shape[1]//2),
                      dims=(ny, nx))

img_gauss = Gauss_op * image.flatten()
img_gauss_ = img_gauss.reshape(image.shape)

# image spectrum
image_shift = np.fft.fftshift(np.fft.fft2(image))
image_spectrum = np.abs(image_shift)

# PSF spectrum - Transfer function
kernel_gauss_shift = np.fft.fftshift(np.fft.fft2(kernel_gauss))
kernel_gauss_spectrum = np.abs(kernel_gauss_shift)

# blurred-image spectrum
img_gauss_shift = np.fft.fftshift(np.fft.fft2(img_gauss_))
img_gauss_spectrum = np.abs(img_gauss_shift)

# PLOTTING
fig, ax = plt.subplots(2, 3, figsize=(9, 6))

ax[0, 0].imshow(image, cmap='gray')
ax[0, 0].set_title('Image')

ax[0, 1].imshow(kernel_gauss, cmap='gray')
ax[0, 1].set_title('Gaussian Kernel')

him2 = ax[0, 2].imshow(img_gauss_, cmap='gray')
ax[0, 2].set_title('Blurred Image')

ax[1, 0].imshow(image_spectrum, cmap='gray', norm=LogNorm(vmin=10))
ax[1, 0].set_title('Image Spectrum')

ax[1, 1].imshow(kernel_gauss_spectrum, cmap='gray')
ax[1, 1].set_title('Gaussian Kernel Spectrum')

him2 = ax[1, 2].imshow(img_gauss_spectrum, cmap='gray', norm=LogNorm(vmin=10))
ax[1, 2].set_title('Blurred Image Spectrum')
fig.tight_layout()
```

# Image Deblurring

Finally, we consider a standard deblurring case. Here we assume a picture was taken in real life; therefore, a whole set of potential artifacts are present in it. Our mission is to try and reconstruct a shaper image representation of the scene by removing the blurring effects. Luckily, we assume to have some information on the optical system and decided to implement an inverse problem. Depending on the application, different solvers are available, below we will compare the performance of some well-known algorithms to deblur the image and compare it with its original version.

- least squares inversion
- least squares - regularized inversion
- TV inversion
- FISTA - Fast Iterative Shrinkage-Thresholding Algorithm

A set of ready to use solvers are implemented in the PyLops module `pylops.optimization.leastsquares` and `pylops.optimization.sparsity`.

**least squares inversion**
\begin{equation}
J= ||\mathbf{d} - \mathbf{A} \mathbf{m}||_2^2
\end{equation}

```python
# least squares inversion
deblur_l2 = leastsquares.NormalEquationsInversion(Op=Gauss_op,
                                                  Regs=None,
                                                  data=img_gauss,
                                                  maxiter=40)
# Reshape image
deblur_l2 = deblur_l2.reshape(image.shape)
```

**least squares - regularized inversion**
\begin{equation}
J= ||\mathbf{d} - \mathbf{A} \mathbf{m}||_2^2 + ||\nabla \mathbf{m}||_2^2
\end{equation}

```python
# L2 regularization term - Second derivative
D2op = Laplacian(dims=(ny, nx), dtype=np.float)

# least squares - regularized inversion
deblur_l2_reg = leastsquares.RegularizedInversion(Op=Gauss_op,
                                                  Regs=[D2op],
                                                  data=img_gauss,
                                                  epsRs=[1e0],
                                                  show=True,
                                                  **dict(iter_lim=20))
# Reshape image
deblur_l2_reg = np.real(deblur_l2_reg.reshape(image.shape))
```

**TV inversion**
\begin{equation}
J= ||\mathbf{d} - \mathbf{A} \mathbf{m}||_2^2 + ||\mathbf{D}_x  \mathbf{m}||_1 + ||\mathbf{D}_y \mathbf{m}||_1
\end{equation}

```python
# L1 regularization term  - First derivative
Dop = [FirstDerivative(ny * nx, dims=(ny, nx), dir=0),
       FirstDerivative(ny * nx, dims=(ny, nx), dir=1)]

# TV inversion
deblur_tv = sparsity.SplitBregman(Op=Gauss_op,
                                  RegsL1=Dop,
                                  data=img_gauss,
                                  niter_outer=10,
                                  niter_inner=5,
                                  mu=1.8,
                                  epsRL1s=[1e-1, 1e-1],
                                  tol=1e-4,
                                  tau=1.,
                                  ** dict(iter_lim=5, damp=1e-4, show=True))[0]
# Reshape image
deblur_tv = deblur_tv.reshape(image.shape)
```

**FISTA inversion**
\begin{equation}
J= ||\mathbf{d} - \mathbf{A} \mathbf{m}||_2^2 + \epsilon||\mathbf{m}||_p
\end{equation}

```python
# FISTA inversion
deblur_fista = sparsity.FISTA(Op=Gauss_op,
                              data=img_gauss,
                              eps=1e-1,
                              niter=100,
                              show=True)[0]
# Reshape image
deblur_fista = deblur_fista.reshape(image.shape)
```

```python
# PLOTTING
fig, ax = plt.subplots(2, 3, sharex=True, sharey=True, figsize=(12, 8))

ax[0, 0].imshow(image, aspect='auto', cmap='gray')
ax[0, 0].set_title('Original Image')

ax[0, 1].imshow(deblur_l2, aspect='auto', cmap='gray')
ax[0, 1].set_title('LS-Inversion')

ax[0, 2].imshow(deblur_l2_reg, aspect='auto', cmap='gray')
ax[0, 2].set_title('Regularized LS-Inversion')

ax[1, 0].imshow(img_gauss_, aspect='auto', cmap='gray')
ax[1, 0].set_title('Blurred Image')

ax[1, 1].imshow(deblur_tv, aspect='auto', cmap='gray')
ax[1, 1].set_title('TV-Inversion')

ax[1, 2].imshow(deblur_fista, aspect='auto', cmap='gray')
ax[1, 2].set_title('FISTA Inversion')
fig.tight_layout()
```

# Image Deblurring - The effect of noise Noise

We have so far discussed the blurring of images. In practical image processing, observed images are commonly polluted with noise. The nature of the noise is influenced by multiple sources, and mathematically can be linear, nonlinear, multiplicative, and additive. In this exercise, we consider common additive noise, which can be included in our linear model by a term perturbing the blurred image

$$
\mathbf{B} = \mathbf{B_{blur}} + \mathbf{N}
$$

**EX: Adding noise.** Add Gaussian white noise to the blurred image with a given standard deviation and perform image deblurring to evaluate the performace of different solvers

```python
# %load -s Noisy_Inversion solutions/deblurring_sol.py
```

```python
# Noisy_Inversion()
```

# Recap

In this tutorial we have learned to:

- Define a simple PyLops operator implementing a convolutional kernel for image processing
- Use build-in PyLops operators for image manipulation including
- Solve the image deblurring problem using different inverse problems schemes
- Denoise a blurred image with the help of least-squares, TV-Regularization, and FISTA

```python
scooby.Report(core='pylops')
```


## Tutorial 2: Poisson Equation

In this tutorial, a set of equations will be developed to solve the Poisson equation $\Delta p = f$ using Jacobi as the linear solver, where $\Delta$ denotes the Laplace operator. The main objective is to apply the concepts presented in the previous tutorial to generate explicit equations that represent the application of an implicit method.

__Note 0__: If you want to check the interactive plots, do not try to use the notebook command "Restart and Run All Cells". For some reason, "%matplotlib" cells sometimes do not work correctly if you do that and the plots might not work as expected. If you want to restart the page, either use "Restart kernel and clear output" or reload the notebook.

The specialization of the Poisson equation for the two dimensional case is as follows:

$$\frac{\partial ^2 p}{\partial x^2} + \frac{\partial ^2 p}{\partial y^2} = f(x, y)$$

We will use Dirichlet boundary conditions $p = 0$ on all boundaries and the equation will be driven by the forcing term $f$ such that the solution is known and equal to $p(x, y) = \sin{(2 \pi x)} (1 - \cos{(2 \pi y)})$. Note that it respects the boundary condition when the grid is defined in the domain $[0, 1] \times [0, 1]$.

$$\frac{\partial ^2 p}{\partial x^2} + \frac{\partial ^2 p}{\partial y^2} = f(x, y) \implies
f(x, y) = -4 \pi^2 (1 - \cos{(2 \pi y)}) \sin{(2 \pi x)} + 4 \pi^2 \sin{(2 \pi x)} \cos{(2 \pi y)} =
4 \pi^2 \sin{(2 \pi x)} (2 \cos{(2 \pi y)} - 1)$$

__Note 1__: The solution is pre-set so that it can be further used as a reference to calculate the absolute and relative errors of the approximated solution obtained by the implicit methods.

First, the __Grid__ is defined, which will contain the numerical values for $p$ in the domain, and also the __TimeFunction__, which is the symbolic representation of the variable $p$ inside Devito. Even though the Poisson equation does not depend on a time dimension, we use time to represent the passage of the iterations of the implicit method.

__Note 2__: We will store the values of $p$ for all iterations by using the parameter "save" when creating the __TimeFunction__. This is very inefficient, but is necessary to make further analysis on how the solution is evolving after each iteration.

__Note 3__: It is possible to avoid using the class __TimeFunction__, which has some properties specific to problems that actually have a time dimension. This can be avoided by creating a __Dimension__ of iteration and replacing $p$ to be a __Function__. This is not recomended though, since the __TimeFunction__ class has some nice implementations on alternating buffers if you do not want to save all iterations.

```python
from devito import Grid, TimeFunction, Function, Dimension

shape = (50, 50)   # Grid size in number of points
extent = (1., 1.)  # Grid domain limits -> [0,1] x [0,1]

space_order = 2 # The spatial order when discretizing the PDE

nmax = 2000 # Maximum number of iterations of the implicit solver

grid = Grid(shape = shape, extent = extent) # The spatial grid

# The symbolic representation of the Poisson function
p = TimeFunction(name = 'p', grid = grid, space_order = space_order, save = nmax + 2)
```

We will now define the forcing term $f$ as a __Function__, since it is constant over all iterations. We also define the dimensions of the problem, namely $x$ and $y$ as the spatial dimensions and $t$ as the dimension of iteration. The initial condition is set to be random at each point, which will further show some of the properties of the Jacobi method.

```python
from devito import Function
from examples.cfd import plot_field
from numpy import pi, sin, cos, arange, meshgrid, linspace
from numpy.random import rand

# The symbolic representation of the Poisson source/forcing term
f = Function(name = 'f', grid = grid, space_order = space_order)

# The x and y variables represent the spatial dimensions in p(x, y)
x, y = grid.dimensions

# The dimension of iteration
# p(n = 0) is the initial condition, p(n = 1) is the result after one iteration
n = p.time_dim

# The initial condition is randomized between [-2, 2] = 4 * [0, 1] - 2
p.data[0, :, :] = 4 * rand(shape[0], shape[1]) - 2

# Calculate the values for the forcing term and the expected solution
X, Y = meshgrid(linspace(0., 1., shape[0], endpoint = True), linspace(0., 1., shape[1], endpoint = True),
                sparse = True)
f.data[:] = 4 * pi ** 2 * sin(2 * pi * X) * (2 * cos(2 * pi * Y) - 1)
solution = sin(2 * pi * X) * (1 - cos(2 * pi * Y))

plot_field(solution, xmax = 1., ymax = 1., view = (30, 225))
```

The figure above is a plot of the expected solution. We can now create the representation of the PDE inside devito.

```python
from devito import Eq

pde = Eq(p.laplace, f, subdomain = grid.interior)

print("%s = %s" % (pde.lhs.evaluate, pde.rhs.evaluate))
```

    -2.0*p(time, x, y)/h_y**2 + p(time, x, y - h_y)/h_y**2 + p(time, x, y + h_y)/h_y**2 - 2.0*p(time, x, y)/h_x**2 + p(time, x - h_x, y)/h_x**2 + p(time, x + h_x, y)/h_x**2 = f(x, y)

When using an iterative algorithm, it is useful to establish a stopping criteria, so that not all the iterations have to be executed. In this tutorial, we use the criteria of the euclidian norm between the difference of two consecutive iterations is less than a limit $\epsilon$:

$$\mathrm{stop\ if\ } ||p_{n+1}-p_n||_2 < \epsilon$$

To implement the stopping criteria, we need to define a variable to represent the norm of the difference between two iterations, which will be further compared with $\epsilon$ to check for the need to halt the execution of the iterations. Since the residual must be calculated after one full execution of an iteration, two temporary dimensions "xtemp" and "ytemp" must be created so that devito does not put both the residual calculation and the PDE equation in the same loop.

An auxiliary variable "niter" will also be used to store the value of the current iteration so that we can check how many iterations were executed.

__Note 4__: To represent a scalar ("residual" and "niter"), we must create a __Function__ which has a singular shape, in other words, it is seen by devito as an array with only one element. Therefore, we must specify a dimension for the array, even though there is actually no possible values for this dimension other than 0.

```python
from devito import DefaultDimension, ConditionalDimension
from numpy import int32

# Temporary dimensions used to separate the residual calculation from the PDE equation
# Note: The default value is passed so that devito knows what are the upper bounds for this dimension
xtemp = DefaultDimension(name = 'xt', default_value = p.shape[1] - 1)
ytemp = DefaultDimension(name = 'yt', default_value = p.shape[2] - 1)

xy_to_xtyt = [(x, xtemp), (y, ytemp)] # Used to convert an equation written in (x,y) to (xtemp, ytemp)

# The minimum residual to stop iterating (residual goal)
# Note: The Jacobi method converges slowly for this problem, choose a high enough epsilon
epsilon = 0.1

# The norm of the residuals to be compared with epsilon
residual = Function(name = 'residual', shape = (1,), dimensions = (Dimension(name = 'residual_dim'),))
residual.data[0] = 0

cn = ConditionalDimension(name = 'cn', parent = n, condition = (residual[0] >= epsilon), brk = True)

# The variable that will store which iteration we are currently executing
niter = Function(name = 'niter', shape = (1,), dimensions = (Dimension(name = 'niter_dim'),),
                 dtype = int32)
```

Finally we will define each of the equations to be passed to devito:

1. __Implicit Method Equation__ (iterEq): This is the equation of one iteration of the implicit method (Jacobi) applied to the partial differential equation $p_{n+1} = F(p_n)$. In this case, the Jacobi method consists of isolating the term p(x, y) from the discretized PDE and applying the resulting equation to each grid point.

2. __Residual Equation__ (resEq): This is the equation that calculates the residual after an iteration. It can be calculated as the norm of the difference between the previous and the current iterations.

3. __Iteration Counter Equation__ (cntEq): This equation is used to save the value of the current iteration so that we can know what was the last executed iteration. An alternative would be to pre-set the values of p.data[1:] to zero and then check for the first unmodified iteration, in which everything would still be  zero.

4. __Boundary Condition Equations__ (boundary): This are the equations for the Dirichlet boundary condition $p(x, 0) = p(x, 1) = p(0, y) = p(1, y) = 0$.

```python
from devito import solve, Inc
from sympy import symbols

a, b = symbols('a b') # Sympy symbols used to represent generic variables

iterEq = Eq(p.forward, solve(pde, p))

# Note: 'implicit_dims = [n]' is used to make the equation residual = 0 be dependent on the iteration dimension.
# This is necessary so devito puts this equation inside the iteration loop
resEq = [Eq(residual[0], 0, implicit_dims = [n]),
         Inc(residual[0], (p.forward - p).evaluate.subs(xy_to_xtyt) ** 2)]

# The conditional dimension 'cn' is used so that the stop criteria check is made when executing this equation
cntEq = Eq(niter[0], cn)

boundary = [Eq(p[n + 1, 0, y], 0.)]
boundary += [Eq(p[n + 1, x.symbolic_max, y], 0.)]
boundary += [Eq(p[n + 1, x, 0], 0.)]
boundary += [Eq(p[n + 1, x, y.symbolic_max], 0.)]

eqs = [iterEq] + boundary + resEq + [cntEq]
```

The final step is to create and run the Operator responsible for solving the PDE. After that the final result can be plotted.

```python
%matplotlib notebook
import matplotlib.pyplot as plt
from devito import Operator, configuration
from examples.implicit import plot_iterative

# The loop-splitting needs to be disabled since the operator will have a single iteration loop
configuration["loop-splitting"] = False

op = Operator(eqs) # Create the operator

op(time_M = nmax) # Run the operator

last_iter = niter.data[0] + 1 # +1 since the residual check happens AFTER one iteration

if last_iter == nmax:
    print("Failed to converge after %d iterations" % nmax)
else:
    print("Executed %d iterations to finally get a residual of %e" % (last_iter, residual.data[0]))

plot_iterative(p.data[0:last_iter], view = (30, 225), title = "Approximated Solution")
#plot_field(p.data[0], view = (30, 225))
#plot_field(p.data[12], view = (30, 225))
#plot_field(p.data[30], view = (30, 225))
#plot_field(p.data[100], view = (30, 225))
#plot_field(p.data[600], view = (30, 225))
```

    Operator `Kernel` ran in 0.01 s

    Executed 580 iterations to finally get a residual of 9.957591e-02

    <IPython.core.display.Javascript object>

As you can see by moving the iteration slider on the figure above, the Jacobi method is best when used to eliminate high frequencies in the solution. After only some initial iterations, the "noise" that was randomly generated already gets the basic form of the solution. Most of the iterations are then used to perfect the solution by slowly solving for the low frequencies. That is why the Jacobi and SOR methods are often used as pre-conditioners for other implicit methods.

Let's now plot the evolution of the residual and then plot the absolute error in each iteration.

```python
from numpy import sum, argmax, unravel_index, array

# Calculate the residual and absolute error at each grid point
I = arange(1, last_iter + 1)
resid = p.data[I, :, :] - p.data[I - 1, :, :]
error = p.data[0:last_iter + 1, :, :] - solution

# Calculate the total residual for each step
total_resid = []
for i in range(resid.shape[0]):
    total_resid.append(sum(resid[i, :, :] ** 2))

print("The last manually calculated residual %f should be approximately equal to the last devito residual %f" %
      (total_resid[-1], residual.data[0]))

last_error = error[last_iter, :, :]
max_error_idx = unravel_index(argmax(last_error), last_error.shape)
print("Maximum error at the last iteration is %e and happens at (x, y) = %s" %
      (last_error[max_error_idx], max_error_idx * array(grid.spacing[0], grid.spacing[1])))

# Plot the total residual
plt.figure()
plt.grid()
plt.xlabel("Iteration")
plt.ylabel("Total Residual")
plt.yscale('log')
plt.plot(I, total_resid)

# Plot the absolute error at each grid point
#plot_iterative(error, dynamic = True, title = "Absolute Error")

None
```


# Realization of Non-Recursive Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Quantization Effects

Numbers and numerical operations are represented with a finite numerical resolution in digital processors. The same holds for the amplitude values of signals and the algorithmic operations applied to them. Hence, the intended characteristics of a digital filter may deviate in practice due to the finite numerical resolution. The [double-precision floating point representation](https://en.wikipedia.org/wiki/Double-precision_floating-point_format) used in numerical environments like MATLAB or Python/numpy is assumed to be quasi-continuous. This representation serves therefore as reference for the evaluation of quantization effects.

This section investigates the consequences of quantization in non-recursive filters. We first take a look on the quantization of the filter coefficients, followed by the effects caused by a finite numerical resolution of the operations. The realization of non-recursive filters is subject to both effects.

### Quantization of Filter Coefficients

The output signal $y[k]$ of a non-recursive filter with a finite impulse response (FIR) $h[k]$ of length $N$ is given as

\begin{equation}
y[k] = h[k] * x[k] = \sum_{\kappa = 0}^{N-1} h[\kappa] \; x[k - \kappa]
\end{equation}

where $x[k]$ denotes the input signal. The quantized impulse response $h_Q[k]$ (quantized filter coefficients) is yielded by quantizing the impulse response $h[k]$

\begin{equation}
h_Q[k] = \mathcal{Q} \{ h[k] \} = h[k] + e[k]
\end{equation}

where $e[k] = h_Q[k] - h[k]$ denotes the [quantization error](../quantization/introduction.ipynb#Model-of-the-Quantization-Process). Introducing $h_Q[k]$ into above equation and rearranging results in

\begin{equation}
y_Q[k] = \sum_{\kappa = 0}^{N-1} h[\kappa] \; x[k - \kappa] + \sum_{\kappa = 0}^{N-1} e[\kappa] \; x[k - \kappa]
\end{equation}

The input signal $x[k]$ is filtered by the quantization noise $e[k]$ and superimposed to the desired output of the filter. The overall transfer function $H_Q(e^{j \Omega})$ of the filter with quantized filter coefficients is given as

\begin{equation}
H_Q(e^{j \Omega}) = \sum_{k=0}^{N-1} h[k] \; e^{-j \Omega k} + \sum_{k=0}^{N-1} e[k] \; e^{-j \Omega k}  = H(e^{j \Omega}) + E(e^{j \Omega})
\end{equation}

Hence, the quantization of filter coefficients results in a linear distortion of the desired frequency response. To some extent, this distortion can be incorporated into the design of the filter. However, the magnitude of the quantization error $| E(e^{j \Omega}) |$ cannot get arbitrarily small for a finite quantization step $Q$. This limits the achievable attenuation of a digital filter with quantized coefficients. It is therefore important to normalize the filter coefficients $h[k]$ before quantization in order to keep the relative power of the quantization noise small.

#### Example - Effect of coefficient quantization in a lowpass filter

The coefficients of a digital lowpass filter with a cutoff frequency of $\Omega_0 = \frac{\pi}{2}$ are quantized in the following example.

```python
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sig

w = 8  # wordlength of quantized coefficients
A = 1  # attenuation of filter coefficients
N = 256  # number of coefficients for filter
Q = 1/(2**(w-1))  # quantization stepsize

def uniform_midtread_quantizer(x, Q):
    '''Uniform mid-tread quantizer with limiter.'''
    # limiter
    x = np.copy(x)
    idx = np.where(x <= -1)
    x[idx] = -1
    idx = np.where(x > 1 - Q)
    x[idx] = 1 - Q
    # linear uniform quantization
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

# design lowpass
h = A * sig.firwin(N, .5)
# quantize coefficients
hQ = uniform_midtread_quantizer(h, Q)

# plot frequency response
Om, H = sig.freqz(h)
Om, HQ = sig.freqz(hQ)
Om, E = sig.freqz(hQ-h)

plt.figure(figsize=(10, 4))
plt.plot(Om, 20*np.log10(np.abs(H)),
         label=r'$| H(e^{j \Omega}) |$ in dB (Designed)')
plt.plot(Om, 20*np.log10(np.abs(HQ)),
         label=r'$| H_Q(e^{j \Omega}) |$ in dB (Quantized coefficients)')
plt.title('Magnitude of the transfer function w and w/o quantization of coefficients')
plt.xlabel(r'$\Omega$')
plt.axis([0, np.pi, -130, 10])
plt.legend(loc=3)
plt.grid()
```

**Exercise**

* Change the wordlength `w` of the quantized filter coefficients. How does the magnitude response $| H_Q(e^{j \Omega}) |$ of the quantized filter change?
* Change the attenuation `A` of the filter coefficients. What changes?
* Why does the magnitude response of the quantized filter $| H_Q(e^{j \Omega}) |$ deviate more from the magnitude response of the designed filter $| H(e^{j \Omega}) |$ in the frequency ranges with high attenuation?

Solution: The magnitude response of the quantized filter deviates more from the designed filter the shorter the wordlength `w` is chosen. This is due to the increasing quantization error for shorter wordlengths. The quantization error increases also in comparison to the filter coefficients if their amplitude (attenuation `A`) is small. The deviations from the designed filter are generally higher for high attenuations since the quantization error is additive $H_Q(e^{j \Omega}) = H(e^{j \Omega}) + E(e^{j \Omega})$, as derived above.

### Quantization of Signals and Operations

Besides the quantization of filter coefficients $h[k]$, also the quantization of the signals, state variables and operations has to be considered in a practical implementation of filters. The computation of the output signal $y[k] = h[k] * x[k]$ of a non-recursive filter by convolution involves multiplications and additions. In digital signal processors numbers are often represented in [fixed-point arithmetic](https://en.wikipedia.org/wiki/Fixed-point_arithmetic) using [two's complement](https://en.wikipedia.org/wiki/Two's_complement). When multiplying two numbers with a wordlength of $w$-bits in this representation the result would require $2w$-bits. Hence the result has to be requantized to $w$-bits. The rounding operation in the quantizer is often realized as truncation of the $w$ least significant bits. The resulting quantization error is known as [round-off error](https://en.wikipedia.org/wiki/Round-off_error). The addition of two numbers may fall outside the maximum/minimum values of the representation and may suffer from clipping. Similar considerations hold also for other number representations, like e.g. [floating point](https://en.wikipedia.org/wiki/Floating_point).

As for the [quantization noise](../quantization/linear_uniform_quantization_error.ipynb#Model-for-the-Quantization-Error), a statistical model for the round-off error in multipliers is used to quantify the average impact of round-off noise in a non-recursive filter.

#### Model for round-off errors in multipliers

As outlined above, multipliers require a requantization of the result in order to keep the wordlength constant. The multiplication of a quantized signal $x_Q[k]$ with a quantized factor $a_Q$ can be written as

\begin{equation}
y_Q[k] = \mathcal{Q} \{ a_Q \cdot x_Q[k] \} = a_Q \cdot x_Q[k] + e[k]
\end{equation}

where the round-off error $e[k]$ is defined as

\begin{equation}
e[k] = y_Q[k] - a_Q \cdot x_Q[k]
\end{equation}

This leads to the following model of a multiplier including round-off effects

The round-off error can be modeled statistically in the same way as quantization noise [[Zölzer](../index.ipynb#Literature)]. Under the assumption that the average magnitude of $a_Q \cdot x_Q[k]$ is much larger that the quantization step size $Q$, the round-off error $e[k]$ can be approximated by the following statistical model

1. The round-off error $e[k]$ is not correlated with the input signal $x_Q[k]$

2. The round-off error is [white](../random_signals/white_noise.ipynb)

    $$ \Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = \sigma_e^2 $$

2. The probability density function (PDF) of the round-off error is given by the zero-mean [uniform distribution](../random_signals/important_distributions.ipynb#Uniform-Distribution)

    $$ p_e(\theta) = \frac{1}{Q} \cdot \text{rect} \left( \frac{\theta}{Q} \right) $$

The variance (power) of the round-off error is [derived from its PDF](../random_signals/important_distributions.ipynb#Uniform-Distribution) as

\begin{equation}
\sigma_e^2 = \frac{Q^2}{12}
\end{equation}

#### Round-off noise

Using above model of a multiplier and discarding clipping, a straightforward realization of the convolution with quantized signals would be to requantize after every multiplication

\begin{equation}
\begin{split}
y_Q[k] &= \sum_{\kappa = 0}^{N-1} \mathcal{Q} \{ h_Q[\kappa] \; x_Q[k - \kappa] \} \\
&= \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] + e[\kappa] \\
&= \left( \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] \right) + e_\text{tot}[k]
\end{split}
\end{equation}

where $e_\text{tot}[k] = \sum_{\kappa = 0}^{N-1} e[\kappa]$. The round-off errors $e[\kappa]$ for each multiplication are uncorrelated to each other. The overall power of the round-off error is then given as

\begin{equation}
\sigma_{e, \text{tot}}^2 = N \cdot \frac{Q^2}{12}
\end{equation}

Many digital signal processors allow to perform the multiplications and additions in an internal register with double wordlength. In this case, only the result of the convolution sum has to be requantized

\begin{equation}
y_Q[k] = \mathcal{Q} \left\{ \sum_{\kappa = 0}^{N-1}  h_Q[\kappa] \; x_Q[k - \kappa] \right\} =
\left( \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] \right) + e[k]
\end{equation}

and the total power of the round-off noise in this case is

\begin{equation}
\sigma_{e, \text{tot}}^2 = \frac{Q^2}{12}
\end{equation}

It is evident that this realization is favorable due to the lower round-off noise, especially for filters with a large number $N$ of coefficients.

#### Example - Effect of requantization after each multiplication

The following example simulates the round-off noise of a non-recursive filter when requantization is performed after each multiplication. Clipping is not considered. The input signal $x[k]$ is drawn from a [uniform distribution](../random_signals/important_distributions.ipynb#Uniform-Distribution) with $a=-1$ and $b=1-Q$. Both the input signal and filter coefficients are quantized. The output signal $y[k]$ without requantization of the multiplications is computed, as well as the output signal $y_Q[k]$ with requantization. The statistical properties of the round-off noise $e[k] = y_Q[k] - y[k]$ are evaluated.

```python
w = 16  # wordlength of quantized coefficients/operations
N = 32  # number of coefficients for filter
L = 8192  # length of input signal
Q = 1/(2**(w-1))  # quantization stepsize

def uniform_midtread_quantizer(x, Q):
    '''Uniform mid-tread quantizer.'''
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

# random impulse response
h = np.random.uniform(size=N, low=-1, high=1)
hQ = uniform_midtread_quantizer(h, Q)
# input signal
x = np.random.uniform(size=L, low=-1, high=1-Q)
xQ = uniform_midtread_quantizer(x, Q)
# output signal by convolution
y = np.zeros(L+N-1)
yQ = np.zeros(L+N-1)
for k in np.arange(L):
    for kappa in np.arange(N):
        if (k-kappa) >= 0:
            y[k] += hQ[kappa] * xQ[k-kappa]
            yQ[k] += uniform_midtread_quantizer(hQ[kappa] * xQ[k-kappa], Q)

# overall round-off error
e = yQ - y

# estimate power of round-off error
sx = 10*np.log10(np.var(e))
print('Power of overall round-off noise is %f dB' % sx)
# estimated PDF of round-off error
pe, bins = np.histogram(e, bins=50, density=True, range=(-10*Q, 10*Q))
# estimate PSD of round-off error
nf, Pee = sig.welch(e, nperseg=128)

# plot statistical properties of error signal
plt.figure(figsize=(10, 6))

plt.subplot(121)
plt.bar(bins[:-1]/Q, pe*Q, width=20/len(pe))
plt.title('Estimated PDF of the round-off noise')
plt.xlabel(r'$\theta / Q$')
plt.ylabel(r'$\hat{p}_x(\theta) / Q$')
#plt.axis([-1, 1, 0, 1.2])

plt.subplot(122)
plt.plot(nf*2*np.pi, Pee*6/Q**2/N)
plt.title('Estimated PSD of the round-off noise')
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$\hat{\Phi}_{ee}(e^{j \Omega}) / \sigma_e^2$')
plt.axis([0, np.pi, 0, 2])
plt.grid()
```

**Exercise**

* Change the wordlength `w` and check if the $\sigma_e^2$ derived by numerical simulation is equal to its theoretic value derived above?
* Can you explain the shape of the estimated PDF for the round-off noise?

Solution: The power of the overall round-off noise was derived above as $\sigma_e^2 = N \frac{Q}{12}$. Using results from the [model of quantization noise for a uniform quantizer](../quantization/linear_uniform_quantization_error.ipynb#Model-for-the-Quantization-Error) this can be rewritten as $10 \log_{10} (\sigma_e^2) \approx 10 \log_{10} (N) - 10 \log_{10} (3) - 6.02 w$. Hence a doubling of the filter length results in approximately 3 dB more round-off noise at the filter output. The requantization noise of the individual multiplications can be modeled as uniformly distributed white noise. It follows from the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) that the superposition of these individual round-off errors results in a normal distribution.

#### Example - Comparison of round-off noise of conventional and fast convolution

The overall round-off noise in an FIR filter scales with the number of multiplications when rounding is performed after each of these. The fast convolution algorithm requires less multiplications and as a consequence also less rounding operations resulting in potentially less round-off noise at the filter output. This effect is illustrated by the following example. The conventional convolution and the fast convolution is computed for two rectangular signals of the same length $y[k] = A\cdot\text{rect}_L[k] * A\cdot\text{rect}_L[k]$. The round-off noise is computed by comparing the results to the theoretical result $A^2 L \cdot \Lambda_{2L -1}[k]$. Since we cannot change the representation of numbers nor the rounding operations in the given implementation of the FFT, a floating point representation of the signal amplitudes is used for this example. The amplitude $A$ has been chosen such to trigger rounding operations.

```python
L = 1024  # length of signals x[k]
A = 1/7  # amplitude of signal
M = 2*L-1

# generate signals
x = A * np.ones(L)
y = A**2 * L * sig.triang(M)

# linear convolution
y1 = np.convolve(x, x, 'full')
e1 = y - y1
# fast convolution
y2 = np.fft.irfft(np.fft.rfft(x, M+1) * np.fft.rfft(x, M+1))[0:M]
e2 = y - y2

plt.figure(figsize=(10, 4))
plt.plot(np.abs(e1), label='conventional convolution')
plt.plot(np.abs(e2), label='fast convolution')
plt.xlabel(r'$k$')
plt.ylabel(r'$|e[k]|$')
plt.legend()
plt.grid()
```

**Exercises**

* Change the length `L` of the signal. How does the round-off noise scale with length? Why?
* Extend above code to compute the power of the round-off noise for various lengths.

Solution: The round-off noise scales with the number of multiplications which itself scales by the length `L` of the signal. This also explains the shape of the round-off errors for the conventional convolution. The two rectangular functions overlap fully in the center which results in the maximum number of non-zero multiplications.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples*.


```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.special import factorial
from collections import Counter
```

### G-M tube high coltage

```python
#the amplitude, counting rate and background noise
A = np.array([0.440,0.586,0.624,0.728,0.808,0.952,1.002])
R = np.array([0.02,10.7,11.3,11.7,12.3,12.2,12.2])
N = np.array([0,.25,.37,.33,.46,.57,.53])
```

```python
mean = np.mean(R)
sd = np.std(R)
error_on_mean = (sd**2)/6
print("mean:",mean)
print("standard deviation:",sd)
print("error on the mean:", error_on_mean)

fig = plt.figure(figsize=(7,7))
#set the voltage from 780V to 900V
V=np.linspace(780,900,7)
plt.plot(V,A)

plt.title('Figure 2.1')
plt.xlabel('Voltage')
plt.ylabel('Amplitude')

plt.savefig("Amplitude_voltage.png")
```

```python
#voltage vs rate
fig = plt.figure(figsize=(7,7))
plt.plot(V,R-N)
plt.title('Figure 2.2')
plt.xlabel('Voltage')
plt.ylabel('Rate - Noise')
plt.savefig("Voltage_rate.png")
```

## Frequency Distribution

```python
#import the measured 100 rates
Rate = np.array([13,9,10,12,5,6,7,13,16,11,6,11,9,8,10,11,13,12,10,12,9,7,8,14,8,7,9,8,9,12,8,7,15,11,10,9,8,14,9,12,9,11,9,15,14,10,11,12,9,
              5,12,13,19,11,10,16,14,13,16,11,8,10,11,4,9,16,10,8,12,13,10,9,12,12,10,19,9,12,13,11,12,14,15,11,15,6,11,5,7,14,8,13,18,10,9
              ,12,11,12,14,10])
counts_of_rates = dict(Counter(sorted(Rate,key=int)))
print(counts_of_rates)
ratecounts = np.array([1,3,3,5,9,14,12,13,14,8,7,4,4,1,2])
print(ratecounts)
```

    {4: 1, 5: 3, 6: 3, 7: 5, 8: 9, 9: 14, 10: 12, 11: 13, 12: 14, 13: 8, 14: 7, 15: 4, 16: 4, 18: 1, 19: 2}
    [ 1  3  3  5  9 14 12 13 14  8  7  4  4  1  2]

```python
#caculate the mean of the rates
m = np.mean(Rate)
print("mean:",m)
```

    mean: 10.83

```python
#define a function of factorial
def fac(x):
    x = np.zeros(15)
    x[:] = np.linspace(4,19,15)
    fac = factorial(n,exact=False)
    return fac
```

```python
#define the function for poission distribution
def P(n):
    return ((m**n)*(np.exp(-m)))/fac(n)
```

```python
#plot the histgram and poission distribution
plt.figure(figsize=(7,7))
plt.hist(Rate,bins=15,label="F(n)")
n = np.linspace(4,19,15)
plt.plot(n,100*P(n),color="red",label="P(n)")
plt.legend()
plt.title('Figure 3.1')
plt.xlabel('Rate')
plt.ylabel('Frequency')
plt.savefig("Poission_of_counting.png")
```

```python
ratecounts - 100*P(n)
```

    array([-0.13473211,  0.42173075, -1.76393821, -2.40233238, -0.90891765,
            2.36253292, -0.16344536,  1.5555674 ,  4.21513825,  0.33785491,
            1.46808087,  0.296458  ,  1.68935414, -0.3493233 ,  1.25965045])

```python
#define function for X^2
chi_square = np.sum((ratecounts-100*P(n))**2/(100*P(n)))
print('chi_square:',chi_square)
print("confindence level:",0.9235)
```

    chi_square: 8.007176844614612
    confindence level: 0.9235

## Absorbation of gamma ray by lead

```python
#import the data
#x in density times thickness
x = np.array([30.71,26.89,22.52,18.59,13.88,7.41,2.74])
#intensity = rate
I = np.array([673/831,555/501,700/479,728/428,1120/401,1003/187,999/114])
#I not
I0 = 8.517
#background intensity = bacrate*1000
BI = 0.451
#the total error of measured and bacground, sqrt(sigmai^2+sigmab^2)
yerr = np.array([0.0459,0.0578,0.0646,0.0714,0.0900,0.1727,0.2793])
```

```python
#show that x and ln(I) have liner relationship
plt.figure(figsize=(7,7))
plt.plot(x,np.log(I-BI))
plt.title('Figure 4.1')
plt.xlabel('ρx')
plt.ylabel('lnI')
plt.savefig("linear_relation.png")
```

```python
#the model functiuon is ux+b
def func(x,u,b):
    return u*x + b

#liner I is ln(I-BI)
lnI = np.log(I-BI)
#the liner error is error over I
lin_yerr = yerr/(I-BI)

#define the terms needed to find u,b and find u,b
wi = 1/(lin_yerr)**2
W = np.sum(wi)
X_1 = np.sum(wi*x)
Y_1 = np.sum(wi*lnI)
X_2 = np.sum(wi*x**2)
Y_2 = np.sum(wi*lnI**2)
P = np.sum(wi*x*lnI)
delta = (W*X_2 - X_1**2)

u = (W*P - X_1*Y_1) / delta
b = (X_2*Y_1 - X_1*P) / delta
```

```python
#plot the data points and fitting line
plt.figure(figsize=(7,7))

plt.errorbar(x,lnI,lin_yerr,fmt=".",color="blue",label="data points")
plt.plot(x, func(x,u,b) ,color="red",label="fit")
plt.legend()
plt.title('Figure 4.2')
plt.xlabel('ρx')
plt.ylabel('ln(I-BI)')
plt.savefig("line_fitting.png")
print("mass absorption coefficient:",-u)
```

```python
# caculate the chi-square
chi_square = np.sum((lnI - func(x,u,b))**2/lin_yerr**2)
degree_freedom = 7-2
normalized_chisqr = chi_square/degree_freedom
print("chi-square:",chi_square)
print("normalized chi-square:", normalized_chisqr)
print("confindence level:",0.06289)
```

```python
from sympy.solvers import solve
from sympy import Symbol
z = Symbol('z')
w = solve(np.sum((lnI - func(x,z,b))**2/lin_yerr**2)-11.473266437659785,z)
print("uncertanity on u:", w-u)
```


```python
import numpy as np
from scipy import io
from scipy import sparse
from scipy.sparse import csgraph
from scipy import fftpack
from scipy import signal
from scipy import linalg
from matplotlib import pyplot as plt
from matplotlib import collections
import mpl_toolkits.mplot3d.art3d as art3d
import seaborn as sns
import igraph

import graph3d

%matplotlib inline
```

```python
sns.set_palette('husl')
```

### Load data

```python
# Minnesota roads graph
minn = io.loadmat('../data/minnesota.mat')

# Minnesota roads adjacency matrix
Aroads = minn['Problem'][0][0][2]

# Minnesota roads lat/lon coordinates
xy = minn['Problem'][0][0][8][0][0][0]
```

### Isolate largest connected component

```python
# Isolate largest connected component
numcomp, components = sparse.csgraph.connected_components(Aroads)
compsize = np.max(np.bincount(components))
biggestcomp = np.argmax(np.bincount(components))
in_component = np.where(components == biggestcomp)[0]
Aroads = Aroads.todense()
Aroads = Aroads[in_component][:, in_component]
Aroads = sparse.csc_matrix(Aroads)

# Get xy coordinates for plotting
xy = xy[in_component]
```

```python
# Data needed for plotting
links = np.where(Aroads.todense())
startcoords = xy[links[0]]
endcoords = xy[links[1]]
lines = [[tuple(x), tuple(y)] for x, y in zip(startcoords, endcoords)]
```

```python
# Graph laplacian of Minnesota roads
Lroads = csgraph.laplacian(Aroads)

# Eigenvalues and eigenvectors of graph laplacian
l, u = np.linalg.eigh(Lroads.todense())
u = np.asarray(u)
```

```python
# Define a kernel in the graph spectral domain
ghat = np.exp(-5*l)

# Convert kernel to vertex domain
g = u @ ghat
```

# Translation

#### Classical translation operation
\begin{equation}
      (T_vf)(t) = f(t - v) = (f * \delta_v)(t)
    \end{equation}

  We cannot directly generalize $(t - v)$ for a graph, so we consider
  instead the definition of translation as convolution with a Dirac delta.

#### Translation of a graph signal
\begin{equation}
     (T_n f)(i) = \sqrt{N} (f * \delta_n)(i) = \sqrt{N} \sum_{l=0}^{N-1} \hat{f}(\lambda_l) u_l^*(n) u_l(i)
    \end{equation}

Where:

\begin{equation}
  \delta_n =
  \begin{cases}
    1 \ if \ i=n \\
    0 \ otherwise
  \end{cases}
\end{equation}

```python
# Compute translated signal for t={0, 100, 150, 200, 2000, N-1}
g_t = {}

for n in (0, 100, 150, 200, 2000, -1):
    # n : index of vertex to localize around
    # w : index of eigenvalue in l
    g_t[n] = sum([ghat[w]*u[:, w][n]*u[:, w] for w in range(len(l))])
```

```python
# Plot translated signals
fig = plt.figure(figsize=(6,12))

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax0 = fig.add_subplot(4, 1, 1, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[0], fig=fig, ax=ax0, lc=lc)
ax0.set_title(r'$t=0$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax1 = fig.add_subplot(4, 1, 2, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[100], fig=fig, ax=ax1, lc=lc)
ax1.set_title(r'$t=100$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax2 = fig.add_subplot(4, 1, 3, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[200], fig=fig, ax=ax2, lc=lc)
ax2.set_title(r'$t=200$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax3 = fig.add_subplot(4, 1, 4, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[-1], fig=fig, ax=ax3, lc=lc)
ax3.set_title(r'$t=N$')

plt.tight_layout()
plt.savefig('../img/basic_operations_0.pdf', bbox_inches='tight')
```

# Filtering and convolution

Using some transfer function $\hat{h}$, we can filter an input signal as follows:

#### Classical frequency filtering
\begin{equation}
      f_{out}(t) = \mathcal{F}^{-1} \{ \hat{f}_{in}(\xi) \hat{h}(\xi)\}
\end{equation}

In the graph setting:

#### Graph filtering in the graph spectral domain
\begin{equation}
      \begin{split}
        \hat{h}(L) = U
        \begin{bmatrix}
          \hat{h}(\lambda_0) & & 0 \\
           & \ddots & \\
          0 & & \hat{h}(\lambda_{N-1})
        \end{bmatrix}
        U^*
      \\\\
      f_{out} = \hat{h}(L) f_{in}
      \end{split}
    \end{equation}

## Gaussian filter

```python
# Define a gaussian filter
def gaussian_filter(f, std=1):
    return np.exp(-(f**2 / (2*std**2)))
```

```python
# Perform frequency filtering
hhat = u @ np.diag(gaussian_filter(l, std=0.2)) @ u.T

# Generate a noisy graph signal
gnoisy = 4*g_t[150] + 0.03*np.random.randn(len(g))

# Filter noisy signal
gfiltered = hhat @ gnoisy
```

```python
# Plot the original, noisy and filtered signals
fig = plt.figure(figsize=(6,12))

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax0 = fig.add_subplot(3, 1, 1, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[150], fig=fig, ax=ax0, lc=lc)
ax0.set_title(r'$g$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax1 = fig.add_subplot(3, 1, 2, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], gnoisy, fig=fig, ax=ax1, lc=lc)
ax1.set_title(r'$g_{noisy}$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax2 = fig.add_subplot(3, 1, 3, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], gfiltered, fig=fig, ax=ax2, lc=lc)
ax2.set_title(r'$g_{filtered}$')

plt.tight_layout()
plt.savefig('../img/basic_operations_1.pdf', bbox_inches='tight')
```

## Remove high-frequency eigenvalues

```python
# Filter by removing high-frequency eigenvalues
hhat = u @ np.diag(np.where(l < l[100], l, 0)) @ u.T

# Filter noisy signal
gfiltered = hhat @ gnoisy
```

```python
# Plot the original, noisy and filtered signals
fig = plt.figure(figsize=(6,12))

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax0 = fig.add_subplot(3, 1, 1, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g_t[150], fig=fig, ax=ax0, lc=lc)
ax0.set_title(r'$g$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax1 = fig.add_subplot(3, 1, 2, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], gnoisy, fig=fig, ax=ax1, lc=lc)
ax1.set_title(r'$g_{noisy}$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax2 = fig.add_subplot(3, 1, 3, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], gfiltered, fig=fig, ax=ax2, lc=lc)
ax2.set_title(r'$g_{filtered}$')

plt.tight_layout()
plt.savefig('../img/basic_operations_2.pdf', bbox_inches='tight')
```

# Modulation

In simple terms, modulation is like a "translation" in the frequency domain.

#### Classical modulation
\begin{equation}
  \begin{split}
        \text{Time domain: } (M_\omega f)(t) = e^{2 \pi j \omega t} f(t) \\
        \text{Frequency domain: } \overline{M_\omega f}(\xi) = \hat{f}(\xi -
        \omega)
  \end{split}
\end{equation}

Replace complex exponential with a graph Laplacian eigenvector:

#### Graph modulation
\begin{equation}
  (M_kf)(i) = \sqrt{N} u_k(i) f(i)
\end{equation}

If a kernel $f$ is localized around 0 in the graph spectral domain, then
$\overline{M_kg}$ is localized around $\lambda_k$.

```python
def modulate(f, u, mod_freq):
    '''
    f : graph signal
    u : left eigenvectors of the laplacian
    mod_freq : frequency to modulate with
    '''
    N = len(g)
    return np.sqrt(N) * u[:, mod_freq] * g
```

```python
# Define a signal
g = g_t[150]

# Compute modulations
mod_20 = modulate(g, u, 20)
mod_100 = modulate(g, u, 100)
```

```python
# Plot the modulated signals
fig = plt.figure(figsize=(6,12))

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax0 = fig.add_subplot(3, 1, 1, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g, fig=fig, ax=ax0, lc=lc)
ax0.set_title(r'$g$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax1 = fig.add_subplot(3, 1, 2, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], mod_20, fig=fig, ax=ax1, lc=lc)
ax1.set_title(r'$g_{mod(20)}$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax2 = fig.add_subplot(3, 1, 3, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], mod_100, fig=fig, ax=ax2, lc=lc)
ax2.set_title(r'$g_{mod(100)}$')

plt.tight_layout()
plt.savefig('../img/basic_operations_3.pdf', bbox_inches='tight')
```

# Dilation

#### Classical dilation

\begin{equation}
  \begin{split}
        \text{Time domain: } (D_s f)(t) = \frac{1}{s} f \bigg( \frac{t}{s} \bigg) \\
        \text{Frequency domain: } \overline{D_s f}(\xi) = \hat{f}(s \xi)
  \end{split}
\end{equation}

Replace the frequency $\xi$ with an eigenvalue of the Laplacian.

#### Graph dilation

\begin{equation}
      (D_sf)(\lambda) = \hat{f} (s \lambda)
\end{equation}

#### Heat diffusion

Applying the power $\tau$ of a heat diffusion operator $e^{L}$ to any signal $f$ is equivalent to filtering with a dilated heat kernel:

\begin{equation}
      (e^{- \tau L}) f = \overline{(D_{\tau} g)}(L) f = f * (D_{\tau} g)
\end{equation}

```python
# Take matrix exponential of negative laplacian
E = linalg.expm(-np.asarray(Lroads.todense()))

# Filter signal with dilated heat kernel
ds_10 = np.linalg.matrix_power(E, 10) @ g
ds_100 = np.linalg.matrix_power(E, 100) @ g
```

```python
# Plot dilation over time
fig = plt.figure(figsize=(6,12))

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax0 = fig.add_subplot(3, 1, 1, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], g, fig=fig, ax=ax0, lc=lc, vmin=0, vmax=g.max())
ax0.set_title(r'$g$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax1 = fig.add_subplot(3, 1, 2, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], ds_10, fig=fig, ax=ax1, lc=lc,
       vmin=0, vmax=g.max())
ax1.set_title(r'$e^{- 10 L} g$')

lc = collections.LineCollection(lines, linewidths=1, colors='k')
ax2 = fig.add_subplot(3, 1, 3, projection='3d')
graph3d.Graph3d(xy[:,0], xy[:,1], ds_100, fig=fig, ax=ax2, lc=lc,
       vmin=0, vmax=g.max())
ax2.set_title(r'$e^{- 100 L} g$')

plt.tight_layout()
plt.savefig('../img/basic_operations_4.pdf', bbox_inches='tight')
```

```python

```


## Exercício Aula  18 - Otimização com Restrições

```python
import math
import matplotlib.pyplot as plt
import numpy as np
from sympy.solvers import solve
from sympy import Symbol
import pandas as pd
%matplotlib inline
```

# a) Problema de programação linear:

# b) Resolução gráfica:

## No gráfico abaixo, as linhas pontilhadas são restrições e a linha cheia é a função a ser maximizada (neste caso, o lucro).
## A linha pontilhada laranja é a restrição do estoque de matéria-prima.
## A linha pontilhada verde é a restrição do tempo de produção disponível, por semana.
## A linha pontilhada roxa é a restrição da capacidade de armazenamento do estoque.

## Para iniciar o problema, será assumido um lucro de 10k reais para analisar o comportamento. Assim, tem-se o seguinte gráfico:

```python
def f1(x):
    return 550 - x
def f2(x):
    return (40 - 0.04*x)/0.12
def f3(x):
    return (9500 - 20*x)/5
def f4(x):
    return 0
def lucro(x):
    return (10000-45*x)/20

x = Symbol('x')
x1, =  solve(f1(x)-f2(x))
x2, =  solve(f1(x)-f3(x))
x3, =  solve(f2(x)-f3(x))
x4, =  solve(f3(x)-f4(x))
x5 = 0
x6 = 0

y1 = f1(x1)
y2 = f1(x2)
y3 = f2(x3)
y4 = f3(x4)
y5 = 0
y6 = 333.333

plt.figure(figsize=(10,10))

plt.plot(x1,f1(x1),'go',markersize=5)
plt.plot(x2,f1(x2),'go',markersize=5)
#plt.plot(x3,f2(x3),'go',markersize=5)
plt.plot(x4,f4(x4),'go',markersize=5)
plt.plot(x5,y5,'go',markersize=5)
plt.plot(x6,y6,'go',markersize=5)

plt.fill([x1,x2,x3,x4,x5,x6,x1],[y1,y2,y3,y4,y5,y6,y1],'pink',alpha=0.5)

xr = np.linspace(0.5,550,100)
y1r = f1(xr)
y2r = f2(xr)
y3r = f3(xr)
ylucror = lucro(xr)

plt.plot(xr,y1r,'k--', color='yellowgreen', label='tempo útil para produção')
plt.plot(xr,y2r,'k--', color='mediumslateblue', label='armazenamento')
plt.plot(xr,y3r,'k--', color='darkorange', label='matéria-prima')
plt.plot(xr,ylucror, color='k', label='lucro')

plt.xlim(0,500)
plt.ylim(0,550)

plt.legend()
plt.show()
```

# O lucro é, neste gráfico, representado pela linha cheia. Desta forma, variando-se o valor do COEFICIENTE LINEAR DA RETA PRETA CHEIA (lucro) até encontrar a intersecção representada no gráfico com o ponto ótimo (que é aquele ponto ali mais à direita, onde duas retas pontilhadas se intersectam). Assim, pelo método gráfico, é possível concluir que o lucro máximo é de 22250 conforme o gráfico ABAIXO.

```python
def f1(x):
    return 550 - x
def f2(x):
    return (40 - 0.04*x)/0.12
def f3(x):
    return (9500 - 20*x)/5
def f4(x):
    return 0
def lucro(x):
    return (22250-45*x)/20

x = Symbol('x')
x1, =  solve(f1(x)-f2(x))
x2, =  solve(f1(x)-f3(x))
x3, =  solve(f2(x)-f3(x))
x4, =  solve(f3(x)-f4(x))
x5 = 0
x6 = 0

y1 = f1(x1)
y2 = f1(x2)
y3 = f2(x3)
y4 = f3(x4)
y5 = 0
y6 = 333.333

plt.figure(figsize=(10,10))

plt.plot(x1,f1(x1),'go',markersize=5)
plt.plot(x2,f1(x2),'go',markersize=5)
#plt.plot(x3,f2(x3),'go',markersize=5)
plt.plot(x4,f4(x4),'go',markersize=5)
plt.plot(x5,y5,'go',markersize=5)
plt.plot(x6,y6,'go',markersize=5)

plt.fill([x1,x2,x3,x4,x5,x6,x1],[y1,y2,y3,y4,y5,y6,y1],'pink',alpha=0.5)

xr = np.linspace(0.5,550,100)
y1r = f1(xr)
y2r = f2(xr)
y3r = f3(xr)
ylucror = lucro(xr)

plt.plot(xr,y1r,'k--', color='yellowgreen', label='tempo útil para produção')
plt.plot(xr,y2r,'k--', color='mediumslateblue', label='armazenamento')
plt.plot(xr,y3r,'k--', color='darkorange', label='matéria-prima')
plt.plot(xr,ylucror, color='k', label='lucro')

plt.xlim(0,500)
plt.ylim(0,550)

plt.legend()
plt.show()
```

## c) Resolução pelo método simplex:

# Tableau:

```python
index = ["Z", "S1", "S2", "S3"]
column_titles = ['Z','A','B','S1','S2','S3', 'Solução']
table_values = [[1,-45,-20,0,0,0,0],
                [0,1,1,1,0,0,550],
                [0,0.04,0.12,0,1,0,40],
                [0,20,5,0,0,1,9500]]

def table(index, table_values, column_titles):
    table = pd.DataFrame(table_values)
    table.index = [i for i in index]
    table.columns = [i for i in column_titles]
    return table

tableau = table(index, table_values, column_titles)
tableau
```

      Z
      A
      B
      S1
      S2
      S3
      Solução

      Z
      1
      -45.00
      -20.00
      0
      0
      0
      0

      S1
      0
      1.00
      1.00
      1
      0
      0
      550

      S2
      0
      0.04
      0.12
      0
      1
      0
      40

      S3
      0
      20.00
      5.00
      0
      0
      1
      9500

```python
matrix = np.array([[1,-45,-20,0,0,0,0],[0,1,1,1,0,0,550],[0,0.04,0.12,0,1,0,40],[0,20,5,0,0,1,9500]])
matrix = matrix.astype('float64')
```

# TRANSFORMANDO ESSA MATRIZ EM MATRIZ IDENTIDADE POR GAUSS JORDAN, É POSSIVEL DETERMINAR S1, S2 e S3 e depois, por fim, A e B.

```python
#código para pivotar a matriz inteira
for i in range(1,4):
    for j in range(3):
        if i > j:
            if matrix[i][j] != 0:
                temp = matrix[i][j]/matrix[j][j]
                for k in range(4):
                    matrix[i][k] -= temp*matrix[j][k]
```

```python
matrix = np.transpose(matrix)
```

```python
#código para pivotar a matriz inteira
for i in range(1,4):
    for j in range(3):
        if i > j:
            if matrix[i][j] != 0:
                temp = matrix[i][j]/matrix[j][j]
                for k in range(4):
                    matrix[i][k] -= temp*matrix[j][k]
```

```python
print(matrix)
```

    [[  1.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   1.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   8.00000000e-02   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00  -2.75000000e+01]
     [  0.00000000e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   1.00000000e+00]
     [  0.00000000e+00   5.50000000e+02   4.00000000e+01   9.50000000e+03]]

```python
matrix = np.transpose(matrix)
```

```python
print(matrix)
```

    [[  1.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   1.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   8.00000000e-02   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00  -2.75000000e+01]
     [  0.00000000e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   1.00000000e+00]
     [  0.00000000e+00   5.50000000e+02   4.00000000e+01   9.50000000e+03]]

## d) Resolução utilizando um pacote de software.

## Esta resolução está na planilha em anexo.

## e)

## Analisando a resolução gráfica, é possível perceber que a restrição laranja sendo deslocada para a direita (isto é, seu coeficiente linear, ou seja, o valor do estoque de matéria prima), a reta preta também irá para a direita, ou seja, o lucro aumentará.
## SENDO ASSIM, a resposta procurada é aumentar o estoque de matéria-prima.

# Análise

## As váriáveis de folga servem para avaliar quanto de um determinado recurso ainda está disponível. Se assumirem um valor negativo, a interpretação que se aplica é que está faltando aquele determinado recurso para a execução da tarefa. Valores positivos representam a sobra, e, por fim, quando a variável assume o valor zero, toda a quantidade em estoque foi consumida.


# Homework 1
*This notebook includes both coding and written questions. Please hand in this notebook file with all the outputs and your answers to the written questions.*

This assignment covers linear filters, convolution and correlation.

```python
# Setup
import numpy as np
import matplotlib.pyplot as plt
from time import time
from skimage import io

from __future__ import print_function

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading extenrnal modules
%load_ext autoreload
%autoreload 2
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

## Part 1: Convolutions
### 1.1 Commutative Property (10 points)
Recall that the convolution of an image $f:\mathbb{R}^2\rightarrow \mathbb{R}$ and a kernel $h:\mathbb{R}^2\rightarrow\mathbb{R}$ is defined as follows:
$$(f*h)[m,n]=\sum_{i=-\infty}^\infty\sum_{j=-\infty}^\infty f[i,j]\cdot h[m-i,n-j]$$

Or equivalently,
\begin{align}
(f*h)[m,n] &= \sum_{i=-\infty}^\infty\sum_{j=-\infty}^\infty h[i,j]\cdot f[m-i,n-j]\\
&= (h*f)[m,n]
\end{align}

Show that this is true (i.e. prove that the convolution operator is commutative: $f*h = h*f$).

**Your Answer:** *Write your solution in this markdown cell. Please write your equations in [LaTex equations](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Typesetting%20Equations.html).*

### 1.2 Shift Invariance (10 points)
Let $f$ be a function $\mathbb{R}^2\rightarrow\mathbb{R}$. Consider a system $f\xrightarrow{s}g$, where $g=(f*h)$ with some kernel $h:\mathbb{R}^2\rightarrow\mathbb{R}$. Also consider functions $f'(m,n) = f(m-m_0, n-n_0)$ and $g'(m,n) = g(m-m_0, n-n_0)$.

Show that $S$ defined by any kernel $h$ is a Shift Invariant system by showing that $g' = (f'*h)$.

**Your Answer:** *Write your solution in this markdown cell. Please write your equations in [LaTex equations](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Typesetting%20Equations.html).*

### 1.3 Implementation (30 points)

In this section, you will implement two versions of convolution:
- `conv_nested`
- `conv_fast`

First, run the code cell below to load the image to work with.

```python
# Open image as grayscale
img = io.imread('dog.jpg', as_gray=True)

# Show image
plt.imshow(img)
plt.axis('off')
plt.title("Isn't he cute?")
plt.show()
```

Now, implement the function **`conv_nested`** in **`filters.py`**. This is a naive implementation of convolution which uses 4 nested for-loops. It takes an image $f$ and a kernel $h$ as inputs and outputs the convolved image $(f*h)$ that has the same shape as the input image. This implementation should take a few seconds to run.

*- Hint: It may be easier to implement $(h*f)$*

We'll first test your `conv_nested` function on a simple input.

```python
from filters import conv_nested

# Simple convolution kernel.
kernel = np.array(
[
    [1,0,1],
    [0,0,0],
    [1,0,0]
])

# Create a test image: a white square in the middle
test_img = np.zeros((9, 9))
test_img[3:6, 3:6] = 1

# Run your conv_nested function on the test image
test_output = conv_nested(test_img, kernel)

# Build the expected output
expected_output = np.zeros((9, 9))
expected_output[2:7, 2:7] = 1
expected_output[5:, 5:] = 0
expected_output[4, 2:5] = 2
expected_output[2:5, 4] = 2
expected_output[4, 4] = 3

# Plot the test image
plt.subplot(1,3,1)
plt.imshow(test_img)
plt.title('Test image')
plt.axis('off')

# Plot your convolved image
plt.subplot(1,3,2)
plt.imshow(test_output)
plt.title('Convolution')
plt.axis('off')

# Plot the exepected output
plt.subplot(1,3,3)
plt.imshow(expected_output)
plt.title('Exepected output')
plt.axis('off')
plt.show()

# Test if the output matches expected output
assert np.max(test_output - expected_output) < 1e-10, "Your solution is not correct."
```

Now let's test your `conv_nested` function on a real image.

```python
from filters import conv_nested

# Simple convolution kernel.
# Feel free to change the kernel to see different outputs.
kernel = np.array(
[
    [1,0,-1],
    [2,0,-2],
    [1,0,-1]
])

out = conv_nested(img, kernel)

# Plot original image
plt.subplot(2,2,1)
plt.imshow(img)
plt.title('Original')
plt.axis('off')

# Plot your convolved image
plt.subplot(2,2,3)
plt.imshow(out)
plt.title('Convolution')
plt.axis('off')

# Plot what you should get
solution_img = io.imread('convoluted_dog.jpg', as_gray=True)
plt.subplot(2,2,4)
plt.imshow(solution_img)
plt.title('What you should get')
plt.axis('off')

plt.show()
```

Let us implement a more efficient version of convolution using array operations in numpy. As shown in the lecture, a convolution can be considered as a sliding window that computes sum of the pixel values weighted by the flipped kernel. The faster version will i) zero-pad an image, ii) flip the kernel horizontally and vertically, and iii) compute weighted sum of the neighborhood at each pixel.

First, implement the function **`zero_pad`** in **`filters.py`**.

```python
from filters import zero_pad

pad_width = 20 # width of the padding on the left and right
pad_height = 40 # height of the padding on the top and bottom

padded_img = zero_pad(img, pad_height, pad_width)

# Plot your padded dog
plt.subplot(1,2,1)
plt.imshow(padded_img)
plt.title('Padded dog')
plt.axis('off')

# Plot what you should get
solution_img = io.imread('padded_dog.jpg', as_gray=True)
plt.subplot(1,2,2)
plt.imshow(solution_img)
plt.title('What you should get')
plt.axis('off')

plt.show()
```

Next, complete the function **`conv_fast`** in **`filters.py`** using `zero_pad`. Run the code below to compare the outputs by the two implementations. `conv_fast` should run significantly faster than `conv_nested`.
Depending on your implementation and computer, `conv_nested` should take a few seconds and `conv_fast` should be around 5 times faster.

```python
from filters import conv_fast

t0 = time()
out_fast = conv_fast(img, kernel)
t1 = time()
out_nested = conv_nested(img, kernel)
t2 = time()

# Compare the running time of the two implementations
print("conv_nested: took %f seconds." % (t2 - t1))
print("conv_fast: took %f seconds." % (t1 - t0))

# Plot conv_nested output
plt.subplot(1,2,1)
plt.imshow(out_nested)
plt.title('conv_nested')
plt.axis('off')

# Plot conv_fast output
plt.subplot(1,2,2)
plt.imshow(out_fast)
plt.title('conv_fast')
plt.axis('off')

# Make sure that the two outputs are the same
if not (np.max(out_fast - out_nested) < 1e-10):
    print("Different outputs! Check your implementation.")
```

### Extra Credit 1 (10 points)
Devise a faster version of convolution and implement **`conv_faster`** in **`filters.py`**. You will earn extra credit only if the `conv_faster` runs faster (by a fair margin) than `conv_fast` **and** outputs the same result.

```python
from filters import conv_faster

t0 = time()
out_fast = conv_fast(img, kernel)
t1 = time()
out_faster = conv_faster(img, kernel)
t2 = time()

# Compare the running time of the two implementations
print("conv_fast: took %f seconds." % (t1 - t0))
print("conv_faster: took %f seconds." % (t2 - t1))

# Plot conv_nested output
plt.subplot(1,2,1)
plt.imshow(out_fast)
plt.title('conv_fast')
plt.axis('off')

# Plot conv_fast output
plt.subplot(1,2,2)
plt.imshow(out_faster)
plt.title('conv_faster')
plt.axis('off')

# Make sure that the two outputs are the same
if not (np.max(out_fast - out_faster) < 1e-10):
    print("Different outputs! Check your implementation.")
```

---
## Part 2: Cross-correlation

Cross-correlation of two 2D signals $f$ and $g$ is defined as follows:
$$(f\star{g})[m,n]=\sum_{i=-\infty}^\infty\sum_{j=-\infty}^\infty f[i,j]\cdot g[i-m,j-n]$$

### 2.1 Template Matching with Cross-correlation (12 points)
Suppose that you are a clerk at a grocery store. One of your responsibilites is to check the shelves periodically and stock them up whenever there are sold-out items. You got tired of this laborious task and decided to build a computer vision system that keeps track of the items on the shelf.

Luckily, you have learned in CS131 that cross-correlation can be used for template matching: a template $g$ is multiplied with regions of a larger image $f$ to measure how similar each region is to the template.

The template of a product (`template.jpg`) and the image of shelf (`shelf.jpg`) is provided. We will use cross-correlation to find the product in the shelf.

Implement **`cross_correlation`** function in **`filters.py`** and run the code below.

*- Hint: you may use the `conv_fast` function you implemented in the previous question.*

```python
from filters import cross_correlation

# Load template and image in grayscale
img = io.imread('shelf.jpg')
img_grey = io.imread('shelf.jpg', as_grey=True)
temp = io.imread('template.jpg')
temp_grey = io.imread('template.jpg', as_grey=True)

# Perform cross-correlation between the image and the template
out = cross_correlation(img_grey, temp_grey)

# Find the location with maximum similarity
y,x = (np.unravel_index(out.argmax(), out.shape))

# Display product template
plt.figure(figsize=(25,20))
plt.subplot(3, 1, 1)
plt.imshow(temp)
plt.title('Template')
plt.axis('off')

# Display cross-correlation output
plt.subplot(3, 1, 2)
plt.imshow(out)
plt.title('Cross-correlation (white means more correlated)')
plt.axis('off')

# Display image
plt.subplot(3, 1, 3)
plt.imshow(img)
plt.title('Result (blue marker on the detected location)')
plt.axis('off')

# Draw marker at detected location
plt.plot(x, y, 'bx', ms=40, mew=10)
plt.show()

```

#### Interpretation
How does the output of cross-correlation filter look? Was it able to detect the product correctly? Explain what problems there might be with using a raw template as a filter.

**Your Answer:** *Write your solution in this markdown cell.*

---
### 2.2 Zero-mean cross-correlation (6 points)
A solution to this problem is to subtract the mean value of the template so that it has zero mean.

Implement **`zero_mean_cross_correlation`** function in **`filters.py`** and run the code below.

```python
from filters import zero_mean_cross_correlation

# Perform cross-correlation between the image and the template
out = zero_mean_cross_correlation(img_grey, temp_grey)

# Find the location with maximum similarity
y,x = (np.unravel_index(out.argmax(), out.shape))

# Display product template
plt.figure(figsize=(30,20))
plt.subplot(3, 1, 1)
plt.imshow(temp)
plt.title('Template')
plt.axis('off')

# Display cross-correlation output
plt.subplot(3, 1, 2)
plt.imshow(out)
plt.title('Cross-correlation (white means more correlated)')
plt.axis('off')

# Display image
plt.subplot(3, 1, 3)
plt.imshow(img)
plt.title('Result (blue marker on the detected location)')
plt.axis('off')

# Draw marker at detcted location
plt.plot(x, y, 'bx', ms=40, mew=10)
plt.show()

```

You can also determine whether the product is present with appropriate scaling and thresholding.

```python
def check_product_on_shelf(shelf, product):
    out = zero_mean_cross_correlation(shelf, product)

    # Scale output by the size of the template
    out = out / float(product.shape[0]*product.shape[1])

    # Threshold output (this is arbitrary, you would need to tune the threshold for a real application)
    out = out > 0.025

    if np.sum(out) > 0:
        print('The product is on the shelf')
    else:
        print('The product is not on the shelf')

# Load image of the shelf without the product
img2 = io.imread('shelf_soldout.jpg')
img2_grey = io.imread('shelf_soldout.jpg', as_grey=True)

plt.imshow(img)
plt.axis('off')
plt.show()
check_product_on_shelf(img_grey, temp_grey)

plt.imshow(img2)
plt.axis('off')
plt.show()
check_product_on_shelf(img2_grey, temp_grey)
```

---
### 2.3 Normalized Cross-correlation (12 points)
One day the light near the shelf goes out and the product tracker starts to malfunction. The `zero_mean_cross_correlation` is not robust to change in lighting condition. The code below demonstrates this.

```python
from filters import normalized_cross_correlation

# Load image
img = io.imread('shelf_dark.jpg')
img_grey = io.imread('shelf_dark.jpg', as_grey=True)

# Perform cross-correlation between the image and the template
out = zero_mean_cross_correlation(img_grey, temp_grey)

# Find the location with maximum similarity
y,x = (np.unravel_index(out.argmax(), out.shape))

# Display image
plt.imshow(img)
plt.title('Result (red marker on the detected location)')
plt.axis('off')

# Draw marker at detcted location
plt.plot(x, y, 'rx', ms=25, mew=5)
plt.show()
```

A solution is to normalize the pixels of the image and template at every step before comparing them. This is called **normalized cross-correlation**.

The mathematical definition for normalized cross-correlation of $f$ and template $g$ is:
$$(f\star{g})[m,n]=\sum_{i,j} \frac{f[i,j]-\overline{f_{m,n}}}{\sigma_{f_{m,n}}} \cdot \frac{g[i-m,j-n]-\overline{g}}{\sigma_g}$$

where:
- $f_{m,n}$ is the patch image at position $(m,n)$
- $\overline{f_{m,n}}$ is the mean of the patch image $f_{m,n}$
- $\sigma_{f_{m,n}}$ is the standard deviation of the patch image $f_{m,n}$
- $\overline{g}$ is the mean of the template $g$
- $\sigma_g$ is the standard deviation of the template $g$

Implement **`normalized_cross_correlation`** function in **`filters.py`** and run the code below.

```python
from filters import normalized_cross_correlation

# Perform normalized cross-correlation between the image and the template
out = normalized_cross_correlation(img_grey, temp_grey)

# Find the location with maximum similarity
y,x = (np.unravel_index(out.argmax(), out.shape))

# Display image
plt.imshow(img)
plt.title('Result (red marker on the detected location)')
plt.axis('off')

# Draw marker at detcted location
plt.plot(x, y, 'rx', ms=25, mew=5)
plt.show()
```

## Part 3: Separable Filters

### 3.1 Theory (10 points)
Consider an $M_1\times{N_1}$ image $I$ and an $M_2\times{N_2}$ filter $F$. A filter $F$ is **separable** if it can be written as a product of two 1D filters: $F=F_1F_2$.

For example,
$$F=
\begin{bmatrix}
1 & -1 \\
1 & -1
\end{bmatrix}
$$
can be written as a matrix product of
$$F_1=
\begin{bmatrix}
1  \\
1
\end{bmatrix},
F_2=
\begin{bmatrix}
1 & -1
\end{bmatrix}
$$
Therefore $F$ is a separable filter.

Prove that for any separable filter $F=F_1F_2$,
$$I*F=(I*F_1)*F_2$$

**Your Answer:** *Write your solution in this markdown cell. Please write your equations in [LaTex equations](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Typesetting%20Equations.html).*

### 3.2 Complexity comparison (10 points)
Consider an $M_1\times{N_1}$ image $I$ and an $M_2\times{N_2}$ filter $F$ that is separable (i.e. $F=F_1F_2$).

(i) How many multiplication operations do you need to do a direct 2D convolution (i.e. $I*F$)?

(ii) How many multiplication operations do you need to do 1D convolutions on rows and columns (i.e. $(I*F_1)*F_2$)?

(iii) Use Big-O notation to argue which one is more efficient in general: direct 2D convolution or two successive 1D convolutions?

**Your Answer:** *Write your solution in this markdown cell. Please write your equations in [LaTex equations](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Typesetting%20Equations.html).*

Now, we will empirically compare the running time of a separable 2D convolution and its equivalent two 1D convolutions. The Gaussian kernel, widely used for blurring images, is one example of a separable filter. Run the code below to see its effect.

```python
# Load image
img = io.imread('dog.jpg', as_grey=True)

# 5x5 Gaussian blur
kernel = np.array(
[
    [1,4,6,4,1],
    [4,16,24,16,4],
    [6,24,36,24,6],
    [4,16,24,16,4],
    [1,4,6,4,1]
])

t0 = time()
out = conv_nested(img, kernel)
t1 = time()
t_normal = t1 - t0

# Plot original image
plt.subplot(1,2,1)
plt.imshow(img)
plt.title('Original')
plt.axis('off')

# Plot convolved image
plt.subplot(1,2,2)
plt.imshow(out)
plt.title('Blurred')
plt.axis('off')

plt.show()
```

In the below code cell, define the two 1D arrays (`k1` and `k2`) whose product is equal to the Gaussian kernel.

```python
# The kernel can be written as outer product of two 1D filters
k1 = None  # shape (5, 1)
k2 = None  # shape (1, 5)

### YOUR CODE HERE
pass
### END YOUR CODE

# Check if kernel is product of k1 and k2
if not  np.all(k1 * k2 == kernel):
    print('k1 * k2 is not equal to kernel')

assert k1.shape == (5, 1), "k1 should have shape (5, 1)"
assert k2.shape == (1, 5), "k2 should have shape (1, 5)"
```

We now apply the two versions of convolution to the same image, and compare their running time. Note that the outputs of the two convolutions must be the same.

```python
# Perform two convolutions using k1 and k2
t0 = time()
out_separable = conv_nested(img, k1)
out_separable = conv_nested(out_separable, k2)
t1 = time()
t_separable = t1 - t0

# Plot normal convolution image
plt.subplot(1,2,1)
plt.imshow(out)
plt.title('Normal convolution')
plt.axis('off')

# Plot separable convolution image
plt.subplot(1,2,2)
plt.imshow(out_separable)
plt.title('Separable convolution')
plt.axis('off')

plt.show()

print("Normal convolution: took %f seconds." % (t_normal))
print("Separable convolution: took %f seconds." % (t_separable))
```

```python
# Check if the two outputs are equal
assert np.max(out_separable - out) < 1e-10
```

```python

```