# Muography
#### Roland Grinis  -  Researcher at MIPT Nuclear Physics Methods lab  -  CTO at GrinisRIT (grinisrit.com)
#### Danila Riazanov - Student at MIPT,  JetBrains Research trainee

Code available within `NOA` [github.com/grinisrit/noa](https://github.com/grinisrit/noa) - Differentiable Programming Algorithms in `C++17` over [LibTorch](https://pytorch.org/cppdocs)

## Installation

The `conda` environment provided with the repository has all the required dependencies. For this particular tutorial we will need the following `python` packages:

```python
import torch
from torch.utils.cpp_extension import load
import matplotlib.pyplot as plt
```

Now we need to build and load `C++17/CUDA` extensions for `PyTorch`, set up the locations:

```python
!mkdir -p build
noa_location = '../..'
```

If you are running this on Google Colab, you need to clone `NOA` and set `noa_location` accordingly:
```python
!git clone https://github.com/grinisrit/noa.git
noa_location = 'noa'
```

Also, make sure that `ninja` and `g++-9` or higher are available. The following commands will do that for you:
```python
!pip install Ninja
!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt update
!apt install gcc-9 g++-9
!update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100 --slave /usr/bin/g++ g++ /usr/bin/g++-9
!apt upgrade -y
!gcc --version
!g++ --version
!nvcc --version
```
Finally, you get the extensions into `python` by calling `load`:

```python
muons = load(name='muons',
             build_directory='./build',
             sources=[f'{noa_location}/docs/pms/muons.cc'],
             extra_include_paths=[f'{noa_location}/include'],
             extra_cflags=['-Wall -Wextra -Wpedantic -O3 -std=c++17'],
             verbose=False)
```

```python
muons_cuda = load(name='muons_cuda',
             build_directory='./build',
             sources=[f'{noa_location}/docs/pms/muons.cu'],
             extra_include_paths=[f'{noa_location}/include'],
             extra_cflags=['-Wall -Wextra -Wpedantic -O3 -std=c++17'],
             extra_cuda_cflags=['-std=c++17 --extended-lambda'],
             verbose=False) if torch.cuda.is_available() else None
```

## Differential Cross-Section calculations

Differential cross-sections (DCS) are implemented in `<noa/pms/dcs.hh>` for `CPU` and `<noa/pms/dcs.cuh>` for `CUDA` within the namespace `noa::pms::dcs`.

Here, we demonstrate the calculations for muons passing through the standard rock. In `<noa/pms/constants.hh>` you will find:
```cpp
constexpr ParticleMass MUON_MASS = 0.10565839;     // GeV/c^2

constexpr AtomicElement<Scalar> STANDARD_ROCK =
            AtomicElement<Scalar>{
                    22.,       // Atomic mass in g/mol
                    0.1364E-6, // Mean Excitation in GeV
                    11 // Atomic number
    };
```

Let's get a range of kinetic and recoil energies:

```python
kinetic_energies = torch.linspace(1e-3, 1e6, 10000).double()
recoil_energies = 0.0505 * kinetic_energies
```

```python
kinetic_energies_gpu = kinetic_energies.cuda()
recoil_energies_gpu = recoil_energies.cuda()
```

```python
kinetic_energies_gpu[:5]
```

    tensor([1.0000e-03, 1.0001e+02, 2.0002e+02, 3.0003e+02, 4.0004e+02],
           device='cuda:0', dtype=torch.float64)

From now on, we shall get ourselves into the namespace:
```cpp
using namespace noa::pms;
```
On `CPU` the DCS computation kernel can be mapped on a tensor via the utility `dcs::vmap`. The user is expected to provide a `result` tensor with the same options as `kinetic_energies` which will get populated with the calculation values:
```cpp
const auto result = torch::zeros_like(kinetic_energies);
```

### Bremsstrahlung

Bremsstrahlung dominates other muon interaction processes in the region of catastrophic collisions (𝑣 ≥ 0.1 ), that is
at “moderate” muon energies above the kinematic limit for knock–on electron production. At high energies (𝐸 ≥ 1
TeV) this process contributes about 40% of the average muon energy loss.

##### Input:
   * K - The projectile initial kinetic energy, in GeV
   * q - The kinetic energy lost to the photon, in GeV
   * A - The mass number of the target atom, in g/mol
   * Z - The charge number of the target atom.
   * mu - The projectile rest mass, in GeV

##### Output:
    DCS (differential crros-section) in m^2/kg.

#### Definitions and Applicability

Theory from: https://pdg.lbl.gov/2014/AtomicNuclearProperties/adndt.pdf.

The formula for the differential cross section applies when:

* 𝐸 ≫ 𝜇, hence the ultrarelativistic approximation is used
* 𝐸 ≤ $10^{20}$ eV; above this energy, LPM suppression can be expected
* 𝑣 ≥ $10^{−6}$ ; below $10^{−6}$ Ter-Mikaelyan suppression takes place. However, in the latter region the cross section
of muon bremsstrahlung is several orders of magnitude less than that of other processes.
The Coulomb correction (for high 𝑍) is not included. However, existing calculations [AB97] show that for muon
bremsstrahlung this correction is small.

##### Formulae

\begin{equation}
\frac{\text{d}\sigma}{\text{d}\nu} = \left.\frac{\text{d}\sigma}{\text{d}\nu}\right|_\text{brems, elec} + \left.\frac{\text{d}\sigma}{\text{d}\nu}\right|_\text{brems, nucl}
\end{equation}

The differential cross section for muon bremsstrahlung from a (screened) nucleus:

\begin{equation}
\left.\frac{\text{d}\sigma}{\text{d}\nu}\right|_\text{brems, nucl} = \alpha(2Z\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)\Phi_n(\delta)
\end{equation}

To account for bremsstrahlung losses on atomic electrons, approximation for the contribution from electrons, taking into account electronic binding and recoil, is

\begin{equation}
\left.\frac{\text{d}\sigma}{\text{d}\nu}\right|_\text{brems, elec} = \alpha Z(2\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)\Phi_\text{in}(\delta)
\end{equation}

\begin{equation}
\frac{\text{d}\sigma}{\text{d}\nu} = \alpha Z(2\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)(\Phi_\text{in}(\delta) + Z\Phi_n(\delta) )
\end{equation}

Here $\nu = \frac{q}{E}$ is the fraction of the muon's energy transferred to the photon, and

\begin{equation}
\Phi(\delta) = \ln(\frac{BZ^{\frac{-1}{3}}(M_\mu + \delta(D_n\sqrt{e} - 2))}{(m_e + \delta \sqrt{e}BZ^{\frac{-1}{3}})D_n},
\end{equation}

where $D_n = 1.54A^{0.27}$ , $B = 182.7$ ($B = 202.4$ for hygrogen), $e = 2.7181...$ and

\begin{equation}
\delta = \frac{M_\mu^{2}\nu}{2E(1 - \nu)}
\end{equation}

\begin{equation}
\Phi_\text{in}(\delta) = \ln(\frac{M_\mu BZ^{-2/3} \sqrt{e}}{(m_e + \delta  BZ^{-2/3} \sqrt{e})(\frac{M_\mu \delta}{m_e^{2}} + \sqrt{e})}),
\end{equation}

where $B = 1429$ for all elements but hydrogen, where $B = 446$

For our calculations we use differential cross section, as
\begin{equation}
\frac{\text{d} \sigma}{\text{d} q} = \frac{\text{d}\sigma}{\text{d}\nu E}
\end{equation}
This formula takes the form:
\begin{equation}
\frac{\text{d}\sigma}{\text{d}q} = \frac{\alpha Z}{E}(2\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)(\Phi_\text{in}(\delta) + Z\Phi_n(\delta) )
\end{equation}
But, this DCS in $\frac{m^2}{Gev}$. We need to multiply by Avogadro's number, energy and divide by the mass number of the target atom to get DCS in $\frac{m^2}{kg}$.

The final formula:
\begin{equation}
\frac{\text{d}\sigma}{\text{d}q} = \frac{\alpha Z}{E}(2\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)(\Phi_\text{in}(\delta) + Z\Phi_n(\delta) ) \frac{N_a E}{A}
\end{equation}

And take into account, that A in g/mol

\begin{equation}
\frac{\text{d}\sigma}{\text{d}q} = \frac{\alpha Z}{E}(2\frac{m_e}{M_\mu})^{2}(\frac{4}{3}(\frac{1}{\nu} - 1) + \nu)(\Phi_\text{in}(\delta) + Z\Phi_n(\delta) ) \frac{1000 N_a E}{A}
\end{equation}

For `CPU` we have:
```cpp
dcs::vmap(dcs::pumas::bremsstrahlung)(
    result, kinetic_energies, recoil_energies, STANDARD_ROCK, MUON_MASS);
```
For `CUDA` we create the lambda function directly ourselves:
```cpp
dcs::pumas::cuda::vmap_bremsstrahlung(
    result, kinetic_energies, recoil_energies, STANDARD_ROCK, MUON_MASS);
```

```python
brems = muons.bremsstrahlung(kinetic_energies, recoil_energies)
brems[:5]
```

    tensor([3.5293e-04, 3.9395e-06, 4.0777e-06, 4.1341e-06, 4.1650e-06],
           dtype=torch.float64)

```python
brems_gpu = muons_cuda.bremsstrahlung(kinetic_energies_gpu, recoil_energies_gpu);
brems_gpu[:5]
```

    tensor([3.5293e-04, 3.9395e-06, 4.0777e-06, 4.1341e-06, 4.1650e-06],
           device='cuda:0', dtype=torch.float64)

```python
(brems - brems_gpu.cpu()).abs().sum()
```

    tensor(3.2179e-18, dtype=torch.float64)

```python
%timeit muons_cuda.bremsstrahlung(kinetic_energies_gpu, recoil_energies_gpu);
```

    155 µs ± 1.4 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

```python
%timeit muons.bremsstrahlung(kinetic_energies, recoil_energies);
```

    316 µs ± 15.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

### Pair Production

Direct electron pair production is one of the most important muon interaction processes. At TeV muon energies,
the pair production cross section exceeds those of other muon interaction processes over a range of energy transfers
between 100 MeV and 0.1𝐸𝜇. The average energy loss for pair production increases linearly with muon energy, and
in the TeV region this process contributes more than half the total energy loss rate.

To adequately describe the number of pairs produced, the average energy loss and the stochastic energy loss distribution,
the differential cross section behavior over an energy transfer range of 5 MeV ≤ 𝜖 ≤ 0.1 ·𝐸𝜇 must be accurately
reproduced. This is is because the main contribution to the total cross section is given by transferred energies 5 MeV
≤ 𝜖 ≤ 0.01 ·𝐸𝜇, and because the contribution to the average muon energy loss is determined mostly in the region
0.001 · 𝐸𝜇 ≤ 𝜖 ≤ 0.1 ·𝐸𝜇 .

##### Input:
   * Z - The charge number of the target atom.
   * A - The mass number of the target atom.
   * mass - The projectile rest mass, in GeV
   * K - The projectile initial kinetic energy.
   * q - The kinetic energy lost to the photon ($E - E^{\text{'}}$).

##### Output:
    DCS in m^2/kg.

#### Definitions and Applicability

Theory from https://geant4-userdoc.web.cern.ch/UsersGuides/PhysicsReferenceManual/fo/PhysicsReferenceManual.pdf (Page 151)

Coefficients for the Gaussian quadrature from:
https://pomax.github.io/bezierinfo/legendre-gauss.html.

The formula for the differential cross section applies when:
* $E_\mu \gg \mu $ $ ( E \geq 2 - 5 GeV)$ and $E_\mu \leq 10^{15} - 10^{17} eV $. If muon energies exceed this limit, the LPM (Landau Pomeranchuk Migdal) effect may become important, depending on the material.
*The muon energy transfer $q$ lies between $q_\text{min} = 4m_e$ and $q_\text{max} = E_\mu - \frac{3\sqrt{e}}{4} M_\mu Z^{1/3}$, although the formal lower limit is $q \gg 2m_e$, and the formal upper limit requires $E_\mu ^\text{'} \gg \mu$.
*$Z \leq 40 - 50$. For higher Z, the Coulomb correction is important but has not been sufficiently studied theoretically.

##### Formulae

\begin{equation}
\frac{\text{d}\sigma}{\text{d}q}(Z, A, E, q) = \frac{4}{3\pi} \frac{Z(Z + \xi(Z))}{A} N_A {(\alpha r_e)}^{2} (\frac{1 - \nu}{q}) \int_{p}[\Phi_e + (m_e/M_\mu)^{2}\Phi_\mu]dp,
\end{equation}

where $\Phi_{e,\mu} = B_{e,\mu} {L_{e,\mu}}^{'}$  and  $\Phi_{e,\mu} = 0 \text{  whenever  } \Phi_{e,\mu}<0$.

$B_e$ and $B_\mu$ do not depend on Z, A, and are given by

\begin{equation}
B_e = [(2 + \rho^{2})(1 + \beta) + \xi(3 + \rho^{2})]\ln(1 + \frac{1}{\xi}) + \frac{1 - \rho^{2} - \beta}{1 + \xi} - (3 + \rho^{2})
\approx \frac{1}{2\xi}[(3 - \rho^{2}) + 2\beta(z + \rho^{2})] \text{ for } \xi \geq 10 ^{3}
\end{equation}

\begin{equation}
B_\mu = [(1 + \rho^{2})(1 + \frac{3\beta}{2}) - \frac{1}{\xi}(1 + 2\beta)(1 - \rho^{2})]\ln(1 + \xi) + \frac{\xi(1 - \rho{2} - \beta)}{1 + \xi} + (1 + 2\beta)(1 - \rho^{2})
\approx \frac{\xi}{2}[(5 - \rho^{2}) + \beta(3 + \rho^{2})] \text{ for } \xi \leq 10^{-3}
\end{equation}

Also,

\begin{equation}
\xi = \frac{\mu^{2}\nu^{2}}{4m^{2}} \frac{(1 - \rho^{2})}{1 - \nu}; \text{ }
\beta = \frac{\nu^{2}}{2(1 - \nu)}
\end{equation}

\begin{equation}
L_e^\text{'} = \ln(\frac{A^{*}Z^{-1/3} \sqrt{(1 + \xi)(1 + Y_e)}}{1 + \frac{2m\sqrt{e}A*Z^{-1/3}(1 + \xi)(1 + Y_\mu)}{E\nu(1 - \rho^{2})}})  - \frac{1}{2}ln([1 + (\frac{3mZ^{1/3}}{2\mu})^{2}(1 + \xi)(1 + Y_e)])
\end{equation}

\begin{equation}
L_\mu ^\text{'} = \ln(\frac{(\frac{M_\mu}{m_e})A^{*}Z^{-1/3} \sqrt{(1 + \frac{1}{\xi})(1 + Y_e)}}{1 + \frac{2m\sqrt{e}A*Z^{-1/3}(1 + \xi)(1 + Y_\mu)}{E\nu(1 - \rho^{2})}})  - ln(\frac{3}{2} Z^{1/3}\sqrt{(1 + \frac{1}{\xi})(1 + Y_\mu)})
\end{equation}

For faster computing, the expressions for $L_{e, \mu} ^\text{'}$ are further algebraically transformed. The function $L_{e, \mu} ^\text{'}$ include the nuclear size correction in comparison with parameterization.

\begin{equation}
Y_e = \frac{5 - \rho^2 + 4\beta(1 + \rho^2)}{2(1 + 3\beta)\ln(3 + 1/\xi)) - \rho^2 - 2\beta(2 - \rho^2)}
\end{equation}

\begin{equation}
Y_\mu = \frac{4 + \rho^2 + 3\beta(1 + \rho^2)}{(1 + \rho^2)(\frac{3}{2} + 2\beta)\ln(3 + \xi) + 1 - \frac{3}{2}\rho^2}
\end{equation}

\begin{equation}
\rho_{max} = 1 - \frac{6M_\mu^2}{E^2(1 - \nu)})\sqrt{1 - \frac{4m_e}{E\nu}}
\end{equation}

```cpp
dcs::vmap(dcs::pumas::pair_production)(
            result, kinetic_energies, recoil_energies, STANDARD_ROCK, MUON_MASS);
```

```python
ppair = muons.pair_production(kinetic_energies, recoil_energies)
ppair[:5]
```

    tensor([0.0000e+00, 6.5366e-06, 7.3699e-06, 7.7919e-06, 8.0572e-06],
           dtype=torch.float64)

### Photonuclear

Input:
   * Z - The charge number of the target atom.
   * A - The mass number of the target atom.
   * ml - The projectile rest mass, in GeV
   * K - The projectile initial kinetic energy.
   * q - The kinetic energy lost to the photon.

Output:
    DCS in m^2/kg.

#### Definitions and Applicability

Theory from: https://arxiv.org/pdf/hep-ph/9712415.pdf, https://arxiv.org/pdf/hep-ph/0012350.pdf

The inelastic interaction of muons with nuclei is important at high muon energies (𝐸 ≥ 10 GeV), and at relatively high
energy transfers 𝜈 (𝜈/𝐸 ≥ 10−2). It is especially important for light materials and for the study of detector response
to high energy muons, muon propagation and muon-induced hadronic background. The average energy loss for this
process increases almost linearly with energy, and at TeV muon energies constitutes about 10% of the energy loss rate.

The main contribution to the cross section 𝜎(𝐸, 𝜈) and energy loss comes from the low 𝑄2–region ( 𝑄2 ≪ 1 GeV2).
In this domain, many simplifications can be made in the theoretical consideration of the process in order to obtain
convenient and simple formulae for the cross section. Most widely used are the expressions given by Borog and
Petrukhin [BP75], and Bezrukov and Bugaev [BB81]. Results from these authors agree within 10% for the differential
cross section and within about 5% for the average energy loss, provided the same photonuclear cross section, 𝜎𝛾𝑁, is
used in the calculations.

##### Formulae

The differential cross section can be written in the form

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{4\pi\alpha^2}{Q^4}\frac{F_2(x, Q^2)}{x}(1-y-\frac{Mxy}{2E} + (1 - \frac{2m_l^2}{Q^2})\frac{y^2(1 + \frac{4M^2x^2}{Q^2})}{2(1 + R(x, Q^2))})
\end{equation}

Where $F_2$ - a nucleon structure function

\begin{equation}
F_2(x, Q^2) = \frac{Q^2}{Q^2 + m_0^2} (F_2^P(x, Q^2) + F_2^R(x, Q^2))
\end{equation}

\begin{equation}
F_2 ^{R}(x, Q^{2}) = c_R(t)x_R^{a_R(t)}(1 - x)^{b_R(t)}
= c_R(t)e^{a_R(t)\ln x_R+ b_R(t)\ln(1 - x)}
\end{equation}

\begin{equation}
F_2 ^{P}(x, Q^{2}) = c_P(t)x_P^{a_P(t)}(1 - x)^{b_P(t)}
= c_P(t)e^{\ln(x_P^{a_P(t)}(1 - x)^{b_P(t)})}
= c_P(t)e^{\ln x_P^{a_P(t)}+ \ln(1 - x)^{b_P(t)}}
= c_P(t)e^{a_P(t)\ln x_P+ b_P(t)\ln(1 - x)}
\end{equation}

Take $x$ as $x = \frac{Q^2}{2Mq}$

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{4\pi\alpha^2 F_2}{Q^4} \frac{2Mq}{Q^2}(1 - y - \frac{My}{2E}\frac{Q^2}{2Mq} + (1 - \frac{2m_l^2}{Q^2})\frac{y^2(1 + (\frac{2MQ^2}{2MqQ})^2)}{2(1 + R)})
\end{equation}

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{8\pi\alpha^2F_2Mq}{Q^6}(1 - y - \frac{Q^2y}{4Eq} + (1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + (y\frac{Q}{q})^2}{2(1 + R)})
\end{equation}

Take into account, that $y = \frac{q}{E}$

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{8\pi\alpha^2F_2Mq}{Q^6}(1 - y - \frac{Q^2}{4E^2} + (1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{2(1 + R)})
\end{equation}

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{8\pi\alpha^2F_2Mq}{Q^6}(1 - y  + (1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{2(1 + R)}) - \frac{8\pi\alpha^2F_2Mq}{Q^6}\frac{Q^2}{4E^2}
\end{equation}

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{8\pi\alpha^2F_2Mq}{Q^6}(1 - y  + (1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{2(1 + R)}) - \frac{8\pi\alpha^2F_2Mq}{4Q^4E^2}
\end{equation}

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}x} = \frac{8\pi\alpha^2F_2Mq}{Q^2}((\frac{1 - y  + \frac{1}{2}(1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{1 + R}}{Q^4}) - \frac{1}{4Q^2E^2})
\end{equation}

Let's see on $x = \frac{Q^2}{2Mq}$

\begin{equation}
\frac{\text{d}x}{\text{d}q} = \frac{-Q^2}{2Mq^2}
\end{equation}

And
\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}q}\frac{2Mq^2}{-Q^2} = \frac{8\pi\alpha^2F_2Mq}{Q^2}((\frac{1 - y  + \frac{1}{2}(1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{1 + R}}{Q^4}) - \frac{1}{4Q^2E^2})
\end{equation}

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}q} = \frac{-4\pi\alpha^2F_2}{q}((\frac{1 - y  + \frac{1}{2}(1 - \frac{2m_l^2}{Q^2}) \frac{y^2 + \frac{Q^2}{E^2}}{1 + R}}{Q^4}) - \frac{1}{4Q^2E^2})
\end{equation}

So it's not enough correct equation. We should add some approximation factors. And finally we have:

\begin{equation}
\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}q} = \frac{c_f F_2}{q} (\frac{1-y+\frac{1}{2}(1 - \frac{2m_l^2}{Q^2}) \frac {y^2 + \frac{Q^2}{E^2}}{1 + R}}{Q^4} - \frac{1}{4E^2Q^2})
\end{equation}

with a constant factor $c_f = 2.603096 \cdot 10^{-35}$. Then integrate this equation using Gaussian Quadrature from $Q_\text{min}^2 = \frac{m_l^2y^2}{1 - y} $ to $Q_\text{max}^2 = 2MEy - ((M + m_\pi)^2 - M^2) $

\begin{equation}
\frac{\text{d}\sigma}{\text{d}q} = \int{\frac{\text{d}\sigma(x, Q^2)}{\text{d}Q^2\text{d}q}}\text{d}Q^2
\end{equation}

```cpp
dcs::vmap(dcs::pumas::photonuclear)(
            result, kinetic_energies, recoil_energies, STANDARD_ROCK, MUON_MASS);
```

```python
photonuc = muons.photonuclear(kinetic_energies, recoil_energies)
photonuc[:5]
```

    tensor([0.0000e+00, 2.2912e-06, 2.1304e-06, 2.0719e-06, 2.0427e-06],
           dtype=torch.float64)

### Ionisation

Input:
   * A - The charge number of the target atom.
   * I - The mean excitation of the target atom.
   * Z - The mass number of the target atom.
   * mu - The projectile rest mass, in GeV
   * K - The projectile initial kinetic energy.
   * q - The kinetic energy lost to the photon.

Output:
    DCS in m^2/kg.

### Definitions and Applicability

Theory from: Salvat et al., NIMB316 (2013) 144-159, Sokalski et al., Phys.Rev.D64 (2001) 074015 (MUM)

The differential cross section for ionisation is computed following Salvat et al., NIMB316 (2013) 144-159, considering only close interactions for DELs. In addition a radiative correction is applied according toSokalski et al., Phys.Rev.D64 (2001) 074015 (MUM).

\begin{equation}
\frac{\text{d}\sigma}{\text{d}q} = \frac{CEZ}{A(\frac{1}{2P_2} + \frac{P_2W_\text{max}}{E^2W_\text{max} - qP_2})}(1 + \Delta_{e\gamma}),
\end{equation}

where $P_2 = E^2 - M_\mu^2$, $W_{max} = \frac{2m_e P_2}{M_\mu^2 + m_e(m_e + 2E)}$ and $C = 1.535336*10^{-5}$

$\Delta_{e\gamma}$ - Radiative correction

\begin{equation}
\Delta_{e\gamma} = \frac{\alpha}{2\pi}\ln(1 + \frac{2\nu E}{m_e})(\ln(\frac{4E^2(1 - \nu)}{M_\mu^2}) - \ln(1 + \frac{2\nu E}{m_e}))
\end{equation}

If we take into account, that $\nu = \frac{q}{E}$

\begin{equation}
\Delta_{e\gamma} = \frac{\alpha}{2\pi}\ln(1 + \frac{2q}{m_e})(\ln(\frac{4E(E - q)}{M_\mu^2}) - \ln(1 + \frac{2q}{m_e}))
\end{equation}

```cpp
dcs::vmap(dcs::pumas::ionisation)(
            result, kinetic_energies, recoil_energies, STANDARD_ROCK, MUON_MASS);
```

```python
ionis = muons.ionisation(kinetic_energies, recoil_energies)
ionis[:5]
```

    tensor([0.0000e+00, 3.0168e-05, 1.5300e-05, 1.0284e-05, 7.7575e-06],
           dtype=torch.float64)

```python
def plot_recoil_energy(E, sample = 1000):
    recoil_energies = torch.zeros(sample).double()
    kinetic_energies = torch.full((sample,), E).double() #tensor with one energy level

    for i in range(sample):
        recoil_energies[i] = E*(0.05 + i*(1 - 0.05)/sample)

    brems = muons.bremsstrahlung(kinetic_energies, recoil_energies)
    ppair = muons.pair_production(kinetic_energies, recoil_energies)
    photonuc = muons.photonuclear(kinetic_energies, recoil_energies)
    ionis = muons.ionisation(kinetic_energies, recoil_energies)

    fig, ax = plt.subplots()
    ax.plot(recoil_energies.numpy(), brems.numpy(), label = 'bremsstrahlung')
    ax.plot(recoil_energies.numpy(), ppair.numpy(), label = 'pair production')
    ax.plot(recoil_energies.numpy(), photonuc, label = 'photonuclear')
    ax.plot(recoil_energies.numpy(), ionis.numpy(), label = 'ionisation')

    ax.set_xlabel('The kinetic energy lost to the photon, GeV', fontsize = 20)
    ax.set_ylabel('DCS, m^2/kg', fontsize = 20)
    ax.set_title(f"Energy loss at {E} GeV")

    fig.set_figwidth(20)
    fig.set_figheight(10)
    ax.legend()
    plt.show()
```

```python
plot_recoil_energy(1e-3)
```

```python
plot_recoil_energy(10.0)
```

```python
plot_recoil_energy(1e3)
```

```python
plot_recoil_energy(1e6)
```


# NRPy+'s Finite Difference Interface

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

### NRPy+ Source Code for this module: [finite_difference.py](../edit/finite_difference.py)

# Table of Contents \[Back to [top](#toc)\]
$$\label{toc}$$

This module is organized as follows

1. [Preliminaries](#fdd): Introduction to Finite Difference Derivatives
1. [Step 1](#fdmodule): The finite_difference NRPy+ module
    1. [Step 1.a](#fdcoeffs_func): The `compute_fdcoeffs_fdstencl()` function
        1. [Step 1.a.i](#exercise): Exercise: Using `compute_fdcoeffs_fdstencl()`
    1. [Step 1.b](#fdoutputc): The  `FD_outputC()` function
1. [Step 2](#latex_pdf_output): Output this module to $\LaTeX$-formatted PDF

# Preliminaries: Introduction to Finite Difference Derivatives \[Back to [top](#toc)\]
$$\label{fdd}$$

Suppose we have a *uniform* numerical grid in one dimension; say, the Cartesian $x$ direction. Since the grid is uniform, the spacing between successive grid points is $\Delta x$, and the position of the $i$th point is given by

$$x_i = x_0 + i \Delta x.$$

Then, given a function $u(x)$ on this uniform grid, we will adopt the notation

$$u(x_i) = u_i.$$

We wish to approximate derivatives of $u_i$ at some nearby point (in this tutorial, we will consider derivatives at one of the sampled points $x_i$) using [finite difference](https://en.wikipedia.org/wiki/Finite_difference). (FD) techniques.

FD techniques are usually constructed as follows:
* First, find the unique $N$th-degree polynomial that passes through $N+1$ sampled points of our function $u$ in the neighborhood of where we wish to find the derivative.
* Then, provided $u$ is smooth and properly-sampled, the $n$th derivative of the polynomial (where $n\le N-1$; *Exercise: Justify this inequality*) is approximately equal to the $n$th derivative of $u$. We call this the **$n$th-order finite difference derivative of $u$**.
* So long as the function $u$ is smooth and properly sampled, the relative error between the exact and the finite difference derivative $u^{(n)}$ will generally decrease as the polynomial degree or sampling density increases.

The $n$th finite difference derivative of $u(x)$ at $x=x_i$ can then be written in the form
$$u^{(n)}(x_i)_{\text{FD}} = \sum_{j=0}^{N} u_j a_j,$$
where the $a_j$'s are known as *finite difference coefficients*. So long as the $N$th-degree polynomial that passes through the $N+1$ points is unique, the corresponding set of $a_j$'s are unique as well.

There are multiple ways to compute the finite difference coefficients $a_j$, including solving for the $N$th-degree polynomial that passes through the function at the sampled points. However, the most popular and most straightforward way involves Taylor series expansions about sampled points near the point where we wish to evaluate the derivative.

**Recommended: Learn more about the algorithm NRPy+ adopts to automatically compute finite difference derivatives: ([How NRPy+ Computes Finite Difference Coefficients](Tutorial-How_NRPy_Computes_Finite_Difference_Coeffs.ipynb))**

# Step 1: The finite_difference NRPy+ module \[Back to [top](#toc)\]
$$\label{fdmodule}$$

The finite_difference NRPy+ module contains one parameter:

* **FD_CENTDERIVS_ORDER**: An integer indicating the requested finite difference accuracy order (not the order of the derivative) , where FD_CENTDERIVS_ORDER = [the size of the finite difference stencil in each direction, plus one].

The finite_difference NRPy+ module contains two core functions: `compute_fdcoeffs_fdstencl()` and `FD_outputC()`. The first is a low-level function normally called only by `FD_outputC()`, which computes and outputs finite difference coefficients and the numerical grid indices (stencil) corresponding to each coefficient:

## Step 1.a:  The `compute_fdcoeffs_fdstencl()` function \[Back to [top](#toc)\]
$$\label{fdcoeffs_func}$$

**compute_fdcoeffs_fdstencl(derivstring,FDORDER=-1)**:
* Output nonzero finite difference coefficients and corresponding numerical stencil as lists, using as inputs:
    * **derivstring**: indicates the precise type and direction derivative desired:
        * **Centered derivatives**, where the center of the finite difference stencil corresponds to the point where the derivative is desired:
            * For a first-order derivative, set derivstring to "D"+"dirn", where "dirn" is an integer denoting direction. For a second-order derivative, set derivstring to "DD"+"dirn1"+"dirn2", where "dirn1" and "dirn2" are integers denoting the direction of each derivative. Currently only $1 \le N \le 2$ supported (extension to higher-order derivatives is straightforward). Examples in 3D Cartesian coordinates (x,y,z):
                * the derivative operator $\partial_x^2$ corresponds to derivstring = "DD00"
                * the derivative operator $\partial_x \partial_y$ corresponds to derivstring = "DD01"
                * the derivative operator $\partial_z$ corresponds to derivstring = "D2"
        * **Up- or downwinded derivatives**, where the center of the finite difference stencil is *one gridpoint* up or down from where the derivative is requested.
            * Set derivstring to "upD"+"dirn" or "dnD"+"dirn", where "dirn" is an integer denoting direction. Example in 3D Cartesian coordinates (x,y,z):
                * the upwinded derivative operator $\partial_x$ corresponds to derivstring = "dupD0"
        * **Kreiss-Oliger dissipation derivatives**, where the center of the finite difference stencil corresponds to the point where the dissipation will be applied.
            * Set derivstring to "dKOD"+"dirn", where "dirn" is an integer denoting direction. Example in 3D Cartesian coordinates (x,y,z):
                * the Kreiss-Oliger derivative operator $\partial_z^\text{KO}$ corresponds to derivstring = "dKOD2"
    * **FDORDER**: an *optional* parameter that, if set to a positive even integer, overrides FD_CENTDERIVS_ORDER

Within NRPy+, `compute_fdcoeffs_fdstencl()` is only called from `FD_outputC()`. Regardless, this function provides a nice interface for evaluating finite difference coefficients, as shown below:

```python
# Import the finite difference module
import finite_difference as fin

fdcoeffs, fdstencl = fin.compute_fdcoeffs_fdstencl("dDD00")
print(fdcoeffs)
print(fdstencl)
```

    [-1/12, 4/3, -5/2, 4/3, -1/12]
    [[-2, 0, 0, 0], [-1, 0, 0, 0], [0, 0, 0, 0], [1, 0, 0, 0], [2, 0, 0, 0]]

Interpreting the output, notice first that $\texttt{fdstencl}$ is a list of coordinate indices, where up to 4 dimension indices are supported (higher dimensions are possible and can be straightforwardly added, though be warned about [The Curse of Dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality)).

Thus NRPy+ found that for some function $u$, the fourth-order accurate finite difference operator at point $x_{i0}$ is given by

$$[\partial_x u]^\text{FD4}_{i0} = \frac{1}{\Delta x} \left[ \frac{1}{12} \left(u_{i0-2,i1,i2,i3} - u_{i0+2,i1,i2,i3}\right) + \frac{2}{3} \left(-u_{i0-1,i1,i2,i3} + u_{i0+1,i1,i2,i3}\right)\right]$$

Notice also that multiplying by the appropriate power of $\frac{1}{\Delta x}$ term is up to the user of this function.

In addition, if the gridfunction $u$ exists on a grid that is less than four (spatial) dimensions, it is up to the user to truncate the additional index information.

### Step 1.a.i: Exercise: Using `compute_fdcoeffs_fdstencl()` \[Back to [top](#toc)\]
$$\label{exercise}$$

Using `compute_fdcoeffs_fdstencl()` write the necessary loops to output the finite difference coefficient tables in the Wikipedia article on [finite difference coefficients](https://en.wikipedia.org/wiki/Finite_difference_coefficients), for first and second centered derivatives (i.e., up to $\partial_i^2$)  up to eighth-order accuracy. [Solution, courtesy Brandon Clark](Tutorial-Finite_Difference_Derivatives-FDtable_soln.ipynb).

## Step 1.b: The  `FD_outputC()` function \[Back to [top](#toc)\]
$$\label{fdoutputc}$$

**FD_outputC(filename,sympyexpr_list)**: C code generator for finite-difference expressions.

C codes that evaluate expressions with finite difference derivatives on numerical grids generally consist of three  components, all existing within a loop over "interior" gridpoints; at a given gridpoint, the code must
1. Read gridfunctions from memory at all points needed to evaluate the finite difference derivatives or the gridfunctions themselves.
2. Perform arithmetic, including computation of finite difference stencils.
3. Write the output from the arithmetic to other gridfunctions.

To minimize cache misses and maximize potential compiler optimizations, it is generally recommended to segregate the above three steps. FD_outputC() first analyzes the input expressions, searching for derivatives of gridfunctions. The search is very easy, as NRPy+ requires a very specific syntax for derivatives:
* gf_dD0 denotes the first derivative of gridfunction "gf" in direction zero.
* gf_dupD0 denotes the upwinded first derivative of gridfunction "gf" in direction zero.
* gf_ddnD0 denotes the downwinded first derivative of gridfunction "gf" in direction zero.
* gf_dKOD2 denotes the Kreiss-Oliger dissipation operator of gridfunction "gf" in direction two.
Each time `FD_outputC()` finds a derivative (including references to the gridfunction directly \["zeroth"-order derivatives\]) in this way, it calls `compute_fdcoeffs_fdstencl()` to record the specific locations in memory from which the underlying gridfunction must be read to evaluate the appropriate finite difference derivative.

`FD_outputC()` then orders this list of points for all gridfunctions and points in memory, optimizing memory reads based on how the gridfunctions are stored in memory (set via parameter MemAllocStyle in the NRPy+ grid module). It then completes step 1.

For step 2, `FD_outputC()` exports all of the finite difference expressions, as well as the original expressions input into the function, to outputC() to generate the optimized C code. Step 3 follows trivally from just being careful with the bookkeeping in the above steps.

`FD_outputC()` takes two arguments:
* **filename**: Set to "stdout" to print to screen. Otherwise specify a filename.
* **sympyexpr_list**: A single named tuple or list of named tuples of type "lhrh", where the lhrh type refers to the simple structure:
    * **lhrh(left-hand side of equation, right-hand side of the equation)**

Time for an example: let's compute
$$
\texttt{output} = \text{phi_dDD00} = \partial_x^2 \phi(x,t),
$$
where $\phi$ is a function of space and time, though we only store its spatial values at a given time (*a la* the [Method of Lines](https://reference.wolfram.com/language/tutorial/NDSolveMethodOfLines.html), described & implemented in next the [Scalar Wave Equation module](Tutorial-Start_to_Finish-ScalarWave.ipynb)).

As detailed above, the suffix $\text{_dDD00}$ tells NRPy+ to construct the second finite difference derivative of gridfunction $\texttt{phi}$ with respect to coordinate $xx0$ (in this case $xx0$ is simply the Cartesian coordinate $x$). Here is the NRPy+ implementation:

```python
import sympy as sp
from outputC import *
import grid as gri
import indexedexp as ixp
import finite_difference as fin

# Set the spatial dimension to 1
par.set_paramsvals_value("grid::DIM = 1")

# Register the input gridfunction "phi" and the gridfunction to which data are output, "output":
phi, output = gri.register_gridfunctions("AUX",["phi","output"])

# Declare phi_dDD as a rank-2 indexed expression: phi_dDD[i][j] = \partial_i \partial_j phi
phi_dDD = ixp.declarerank2("phi_dDD","nosym")

# Set output to \partial_0^2 phi
output = phi_dDD[0][0]

# Output to the screen the core C code for evaluating the finite difference derivative
fin.FD_outputC("stdout",lhrh(lhs=gri.gfaccess("out_gf","output"),rhs=output))
```

    {
       /*
        * NRPy+ Finite Difference Code Generation, Step 1 of 2: Read from main memory and compute finite difference stencils:
        */
       /*
        *  Original SymPy expression:
        *  "const double phi_dDD00 = invdx0**2*(-5*phi/2 + 4*phi_i0m1/3 - phi_i0m2/12 + 4*phi_i0p1/3 - phi_i0p2/12)"
        */
       const double phi_i0m2 = aux_gfs[IDX2(PHIGF, i0-2)];
       const double phi_i0m1 = aux_gfs[IDX2(PHIGF, i0-1)];
       const double phi = aux_gfs[IDX2(PHIGF, i0)];
       const double phi_i0p1 = aux_gfs[IDX2(PHIGF, i0+1)];
       const double phi_i0p2 = aux_gfs[IDX2(PHIGF, i0+2)];
       const double phi_dDD00 = pow(invdx0, 2)*(-5.0/2.0*phi + (4.0/3.0)*phi_i0m1 - 1.0/12.0*phi_i0m2 + (4.0/3.0)*phi_i0p1 - 1.0/12.0*phi_i0p2);
       /*
        * NRPy+ Finite Difference Code Generation, Step 2 of 2: Evaluate SymPy expressions and write to main memory:
        */
       /*
        *  Original SymPy expression:
        *  "aux_gfs[IDX2(OUTPUTGF, i0)] = phi_dDD00"
        */
       aux_gfs[IDX2(OUTPUTGF, i0)] = phi_dDD00;
    }

Some important points about the above code:
* The gridfunction PHIGF samples some function $\phi(x)$ at discrete uniform points in $x$, labeled $x_i$ at all points $i\in [0,N]$, so that
$$\phi(x_i) = \phi_{i}=\text{in_gfs[IDX2(PHIGF, i)]}.$$
* For a *uniformly* sampled function with constant grid spacing (sample rate) $\Delta x$, $x_i$ is defined as $x_i = x_0 + i \Delta x$.
* The variable $\texttt{invdx0}$ must be defined by the user in terms of the uniform gridspacing $\Delta x$ as $\texttt{invdx0} = \frac{1}{\Delta x}$.
     * *Aside*: Why do we choose to multiply by $1/\Delta x$ instead of dividing the expression by $\Delta x$, which would seem much more straightforward?
         * *Answer*: as discussed in the [first part of the tutorial](Tutorial-Coutput__Parameter_Interface.ipynb), division of floating-point numbers on modern CPUs is far more expensive than multiplication, usually by a factor of ~3 or more.

# Step 2: Output this module to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Finite_Difference_Derivatives.pdf](Tutorial-Finite_Difference_Derivatives.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx Tutorial-Finite_Difference_Derivatives.ipynb
!pdflatex -interaction=batchmode Tutorial-Finite_Difference_Derivatives.tex
!pdflatex -interaction=batchmode Tutorial-Finite_Difference_Derivatives.tex
!pdflatex -interaction=batchmode Tutorial-Finite_Difference_Derivatives.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    [NbConvertApp] Converting notebook Tutorial-Finite_Difference_Derivatives.ipynb to latex
    [NbConvertApp] Writing 35911 bytes to Tutorial-Finite_Difference_Derivatives.tex
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode


*This notebook contains course material from [CBE30338](https://jckantor.github.io/CBE30338)
by Jeffrey Kantor (jeff at nd.edu); the content is available [on Github](https://github.com/jckantor/CBE30338.git).
The text is released under the [CC-BY-NC-ND-4.0 license](https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode),
and code is released under the [MIT license](https://opensource.org/licenses/MIT).*

< [Blending Tank Simulation](http://nbviewer.jupyter.org/github/jckantor/CBE30338/blob/master/notebooks/02.03-Blending-Tank-Simulation.ipynb) | [Contents](toc.ipynb) | [Hare and Lynx Population Dynamics](http://nbviewer.jupyter.org/github/jckantor/CBE30338/blob/master/notebooks/02.05-Hare-and-Lynx-Population-Dynamics.ipynb) >

# Continuous Product Blending

This case study is modeled after an actual process used to prepare an over-the-counter pediatric medication. The situation has been simplified to the dissolution of a single active ingredient into a suspension, and adapted to continouous rather than batch operation.

The process objective is to mix an active ingredient $A$ into a suspension $S$, providing enough time to fully mix and dissolve components coming from both streams. The feed concentration of $A$ is 200 grams/liter. Process requirements include:

* The final product concentration must be in the range 7.8 to 8.2 grams/liter.
* At all timses the mixer residence time, $\frac{V}{q_{out}}$, must remain between 84 and 108 hours.
* The volume cannot exceed the tank capacity of 15,000 liters, and cannot fall below the 8,000 liters required to fully cover the mixing blades.

The outlet from the tank is fed directly to a packaging line. The flow demand can vary due to changes in speed of the packaging equipment.

## Preliminary Control Analysis

### Parameters

| Quantity | Symbol | Value | Units |
|:-:|:-:|:-:|:-:|
| Feed Concentration |$c_{A,f}$ | 200 | g/liter |
| Maximum Tank Operating Capacity | $V$ | 15,000 | liters |
| Minimum Tank Operating Capacity | $V$ | 8,000 | liters |

### Control Objectives

* Maintain outlet concentration of $A$ at $8.0 \pm 0.2$ grams/liter.
* Maintain an average residence time (i.e., ratio $V/q_{out}$) of $96 \pm 12$ hours.
* Past operation has shown a liquid volume of 12,000 liters generally meets nominal demand. Demand, however, fluctuates and is determined by the downstream packaging line.

### Classification of Variables

What are the disturbances variables (DV), manipulated variables (MV), and controlled variables (CV)?

| Variable | Symbol | Classification |
|:-:|:-:|:-:|:-:|
| Outlet Flow | $q_{out}$ | DV |
| Feed Flow | $q_A$ | MV |
| Water Makeup Flow | $q_W$ | MV |
| Product Concentration | $c_A$ | CV |
| Volume | $V$ | CV |

In this case, we can assume the controlled variables are directly measureable.

### Degrees of Freedom Analysis

At steady-state the balance equations become algebraic equations

\begin{align}
0 & = \bar{q}_A + \bar{q}_S - \bar{q}_{out} \\
& \\
0 & = \bar{q}_A c_{A,f} - \bar{q}_{out}\bar{c}_A
\end{align}

There are a total of five variables in these two equations. So we need to find at least three more specifciations or constraints to determine values for these variables.

Additional pieces of information we have are the process specifications

\begin{align}
\frac{\bar{q}_{out}}{\bar{V}} & = 96 \mbox{ hours} \\
c_{A,f} & = 200 \mbox{ g/liter} \\
\bar{q}_{out} & = \mbox{ set by the downstream demand} \\
\bar{c}_A & = 8 \mbox{ g/liter}
\end{align}

This is a total of four new equations for steady-state.  The new equations introduced an additional variable $\bar{V}$. This provides six variables in six equations provided we know the downstream demand.

### Steady State

Let's assume a constant output demand $\bar{q}_{out}$ = 125 liters/hr.  The equations may then be solved in order:

\begin{align}
c_{A,f} & = 200 \mbox{ g/liter} \\
\bar{c}_A & = 8 \mbox{ g/liter} \\
\bar{V} & = 96\ \bar{q}_{out} = 12,000 \mbox{ liters}\\
\bar{q}_A & = \frac{\bar{q}_{out}\bar{c}_A}{c_{A,f}} = 5 \mbox{ liters/hr}\\
\bar{q}_S & = \bar{q}_{out} - \bar{q}_A = 120 \mbox{ liters/hr}
\end{align}

### Exercise 1. Determine Operating Range

Solve the steady-state equations for other values of $\bar{q}_{out}$. Given a maximum tank operating capacity of 15,000 liters, what is the largest possible value of $\bar{q}_{out}$ which still meets process specifications?  If the minimum tank operating capacity is 8,000 liters, what is the minimum value of $\bar{q}_{out}$?

## Modeling

The balance equations are

\begin{align}
\frac{dV}{dt} & = q_A + q_S - q_{out} \\
\frac{d(Vc_A)}{dt} & = q_A c_{A,f} - q_{out}c_A
\end{align}

where we have made the assumption of constant density among all of the streams, and of complete and uniform mixing in the stirred tank.

For simulation, it is useful to isolate the derivatives of $V$ and $c_A$ on the left-hand side of these differential equations. Using the chain rule

\begin{align}
\frac{dV}{dt} & = q_A + q_S - q_{out} \\
V\frac{dc_A}{dt} + c_A\frac{dV}{dt} & = q_A c_{A,f} - q_{out}c_A
\end{align}

Substituting the first equation into the second gives

\begin{align}
\frac{dV}{dt} & = q_A + q_S - q_{out} \\
V\frac{dc_A}{dt} + c_A\left(q_A + q_S - q_{out}\right) & = q_A c_{A,f} - q_{out}c_A
\end{align}

Rearranging terms, and dividing the second equation by $V$, gives us the final version of a dynamical model.

\begin{align}
\frac{dV}{dt} & = q_A + q_S - q_{out} \\
\frac{dc_A}{dt} & = \frac{q_A}{V} \left(c_{A,f} - c_A\right) - \frac{q_S}{V} c_A
\end{align}

## Simulation

### Step 1. Imports

We start the simulation by importing necessary Python libraries.

```python
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
%matplotlib inline
```

### Step 2. Parameter Values

It is good coding practice to include all relevant parameter values in single block of code. This makes it easier for users of your code to understand what values are relevant to the problem, and to adjust and maintain your code. No 'magic' numbers should appear later in your code.

```python
# parameters
caf       = 200       # g/liter
Vmax      = 15000     # liters
Vmin      = 8000      # liters

# process setpoints
ca_SP     = 8         # g/liter
rtime_SP  = 96        # residence time in hours

# nominal values of the disturbance variables
qout_bar  = 125
```

### Step 3. Computing Steady State

A good modeling practice is to begin with the determination of a nominal steady state. By nominal we mean operating conditions representative of typical, desired behavior of the process in question, and should be computed based on the parameter values given above.

```python
# nominal steady state at process setpoint
V_bar    = rtime_SP * qout_bar
ca_bar   = ca_SP
qa_bar   = ca_SP*qout_bar/caf
qs_bar   = qout_bar - qa_bar

print("Steady-State")
print("    V [liters]    = ", V_bar)
print("   ca [g/liter]   = ", ca_bar)
print("   qa [liters/hr] = ", qa_bar)
print("   qs [liters/hr] = ", qs_bar)
```

    Steady-State
        V [liters]    =  12000
       ca [g/liter]   =  8
       qa [liters/hr] =  5.0
       qs [liters/hr] =  120.0

### Step 4. Process Model Equations

In our case the model consists of a system of two differential equations. For simulation with the Python function `odeint`, the model is encapsulated into a Python function that accepts two arguments.

* `X` is a list of values for the model state, in this case `V` and `ca`.
* `t` a variable containing value corresponding to the current time.

```python
def deriv(X, t):
    V, ca = X
    dV = qa + qs - qout
    dca = qa*(caf - ca)/V - qs*ca/V
    return [dV, dca]
```

### Step 5. Numerical Solution of the Process Model

This block does the numerically intensive portion of the simulation. Here we establish values for all degrees of freedeom, specify initial conditions, and thne proceed to compute a history of the process behavior. The history is broken into discrete time step which is a bit more work now, but will later provide a means to test implementations of control algorithms.

For the first simulation, it usually a good idea to pick an initial condition that will yield a known result. In this case we start a simulation at steady state.

```python
# fix all degrees of freedom
qout = qout_bar      # outlet flow -- process disturbance variable.
qa = qa_bar          # inlet flow of A -- process manipulated variable
qs = qs_bar          # inlet flow of suspension -- process manipulated variable

# establish initial conditions
t  = 0
V  = V_bar
ca = ca_bar

# time step, and variable to store simulation record (or history)
dt = 1
history = [[t,V,ca]]

while t < 500:
    V, ca = odeint(deriv, [V, ca], [t, t+dt])[-1]
    t += dt
    history.append([t,V,ca])

history[0:10]
```

    [[0, 12000, 8],
     [1, 12000.0, 8.0],
     [2, 12000.0, 8.0],
     [3, 12000.0, 8.0],
     [4, 12000.0, 8.0],
     [5, 12000.0, 8.0],
     [6, 12000.0, 8.0],
     [7, 12000.0, 8.0],
     [8, 12000.0, 8.0],
     [9, 12000.0, 8.0]]

### Step 6. Visualization and Analysis

To simplify future simulations, here we define a function to plot the results stored in the simulation record. The function `plot_history(history, labels)` takes two arguments. The first argument is the data history recorded during the course of the simulation, where the first element is assumed to be time. The second argument is a list of labels corresponding to each of the recorded variables.

```python
def plot_history(history, labels):
    """Plots a simulation history."""
    history = np.array(history)
    t = history[:,0]
    n = len(labels) - 1
    plt.figure(figsize=(8,1.95*n))
    for k in range(0,n):
        plt.subplot(n, 1, k+1)
        plt.plot(t, history[:,k+1])
        plt.title(labels[k+1])
        plt.xlabel(labels[0])
        plt.grid()
    plt.tight_layout()

plot_history(history, ['t / hours','Volume','Concentration $c_A$'])
```

### Exercise 2. Explore Dynamical Behavior

Let's explore the behavior of this mixing tank under different assumptions.  We'll explore the following cases:

1. The startup concentration of $A$ is $c_A = 0$. How long before the mixer is producing an acceptable product?
2. The outlet flow increases by 10%.  How long before the product is out of spec?
3. The outlet flow decreases by 10%.  How long before the product is out of spec?

The following cell shows how you might get started with first question.

```python
# fix all degrees of freedom
qout = qout_bar      # outlet flow -- process disturbance variable.
qa = qa_bar          # inlet flow of A -- process manipulated variable
qs = qs_bar          # inlet flow of suspension -- process manipulated variable

# establish initial conditions
t  = 0
V  = V_bar
ca = 0         # <========= CHANGED TO AN INITIAL CONDITION OF ZERO

# time step, and variable to store simulation record (or history)
dt = 1
history = [[t,V,ca]]

while t < 500:
    V, ca = odeint(deriv, [V, ca], [t, t+dt])[-1]
    t += dt
    history.append([t,V,ca])

plot_history(history, ['t / hours','Volume','Concentration $c_A$'])
```

## Control Residence Time

From the exercises above, we know even small changes in the outlet flowrate results in substantial loss due to out-of-spec product.  Let's try a control strategy in which we change in the suspension flowrate, $q_S$, to compensate for excursions from the desired value of residence time.

We'll try proportional control where the change in $q_S$ is proportional to the difference of the residence time from the setpoint.

\begin{align}
q_S - \bar{q}_S = - K \left(\frac{V}{q_{out}} - \frac{\bar{V}}{\bar{q}_{out}}\right)
\end{align}

The negative sign is there because we expect a negative deviation in flowrate is needed to compensate for a positive deviation in residence time.  This can be written more directly as

\begin{align}
q_S = \bar{q}_S - K \left(\frac{V}{q_{out}} - \frac{\bar{V}}{\bar{q}_{out}}\right)
\end{align}

In the following cell we will decrease qout by 10%, and attempt to find a value for $K$ that provides satisfactory control.

```python
# fix all degrees of freedom
qout = qout_bar      # outlet flow -- process disturbance variable.
qa = qa_bar          # inlet flow of A -- process manipulated variable
qs = qs_bar          # inlet flow of suspension -- process manipulated variable

# establish initial conditions
t  = 0
V  = V_bar
ca = ca_bar

# time step, and variable to store simulation record (or history)
dt = 1
history = [[t, V, ca, V/qout, qs]]

qout = 0.9*qout
while t < 500:
    qs = qs_bar - 5*(V/qout - V_bar/qout_bar)
    V, ca = odeint(deriv, [V, ca], [t, t+dt])[-1]
    t += dt
    history.append([t, V, ca, V/qout, qs])

plot_history(history, ['t','V','ca','V/qout','qs'])
```

## Control Concentration

As shown above, it isn't enough to control only residence time. In fact, controlling residence time in response to a decrease in output causes the outlet concentration to increase beyond the product quality limits. We propose to fix this problem by controlling $q_A$, the inlet flow of $A$. A proportional control rule is given by

\begin{align}
q_A & = \bar{q}_A - K \left(c_A - \bar{c}_A \right)
\end{align}

where, again, we predict a decrease in $q_A$ is needed to compensate for a positive excursion in $c_A$.

```python
# fix all degrees of freedom
qout = qout_bar      # outlet flow -- process disturbance variable.
qa = qa_bar          # inlet flow of A -- process manipulated variable
qs = qs_bar          # inlet flow of suspension -- process manipulated variable

# establish initial conditions
t  = 0
V  = V_bar
ca = ca_bar

# time step, and variable to store simulation record (or history)
dt = 1
history = [[t, V, ca, V/qout, qs, qa]]

qout = 0.9*qout
while t < 500:
    qs = qs_bar - 5*(V/qout - V_bar/qout_bar)
    qa = qa_bar - 3*(ca - ca_bar)
    V, ca = odeint(deriv, [V, ca], [t, t+dt])[-1]
    t += dt
    history.append([t, V, ca, V/qout, qs, qa])

plot_history(history, ['t / hours','Volume','Concentration $c_A$','V/qout','qs','qa'])
```

## Exercise 3.

The algebraic formulae given above do not include any limits on flowrate.  Suppose the flowrate $q_s$ is limited to 0 to 200 liters/hour, and the flowrate $q_a$ is limited to 0 to 20 liters/hour. Modify the code above to incorporate those limits. Now try to find the minimum time necessary to bring the tank from initial concentration of 0 to state where it is producing an acceptable product.

## Proportional-Integral Control

The above control laws do a reasonable job of control, but do not return the system to a desired steady-state operating condition. How can we modify the control laws?

\begin{align}
q_A & = \bar{q}_A - K_p \left(c_A - \bar{c}_A \right) - K_i \int_0^t \left(c_A - \bar{c}_A \right) dt
\end{align}

The second term will integrate any constant offset to produce a constantly changing value.  So the only possible steady-state is when $c_A = \bar{c}_A$.

```python
# fix all degrees of freedom
qout = qout_bar      # outlet flow -- process disturbance variable.
qa = qa_bar          # inlet flow of A -- process manipulated variable
qs = qs_bar          # inlet flow of suspension -- process manipulated variable

# establish initial conditions
t  = 0
V  = V_bar
ca = ca_bar
error_sum = 0

# time step, and variable to store simulation record (or history)
dt = 1
history = [[t, V, ca, V/qout, qs, qa]]

qout = 0.9*qout
while t < 500:
    qs = qs_bar - 5*(V/qout - V_bar/qout_bar)

    error = (ca - ca_bar)
    error_sum += error * dt

    qa = qa_bar - 3*error - 0.4*error_sum
    V, ca = odeint(deriv, [V, ca], [t, t+dt])[-1]
    t += dt
    history.append([t, V, ca, V/qout, qs, qa])

plot_history(history, ['t / hours','Volume','Concentration $c_A$','V/qout','qs','qa'])
```

## Exercise 4.

Write an expression for the proportional-integral control of residence time.  Implement the control, and 'tune' the control constants until you get satisfactory feedback control for a 10% reduction in demand.  By satisfactory we mean:

* Meet product quality specifications at all times.
* There are no more than three observable peaks in any underdamped response.
* The decay ratio is less than 0.25.

Is it possible to meet these control specifications?

```python

```

< [Blending Tank Simulation](http://nbviewer.jupyter.org/github/jckantor/CBE30338/blob/master/notebooks/02.03-Blending-Tank-Simulation.ipynb) | [Contents](toc.ipynb) | [Hare and Lynx Population Dynamics](http://nbviewer.jupyter.org/github/jckantor/CBE30338/blob/master/notebooks/02.05-Hare-and-Lynx-Population-Dynamics.ipynb) >


```python
import sympy as sym
import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline
sym.init_printing()

Omega = sym.symbols('Omega', real=True)
k = sym.symbols('k', integer=True)

```

# Transformada de Fourier de tiempo discreto
La [Transformada de Fourier de tiempo discreto](https://en.wikipedia.org/wiki/Discrete-time_Fourier_transform) (DTFT) de una señal discreta $x[k]$ con $k \in \mathbb{Z}$ es:

\begin{equation}
X(e^{j \Omega}) = \sum_{k = -\infty}^{\infty} x[k] \, e^{-j \Omega k}
\end{equation}

donde $\Omega \in \mathbb{R}$ representa la frecuencia angular normalizada. Observe que $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ es una función compleja y continua que contiene la información de la señal discreta $x[k]$, por lo cual se le denomina el "espectro" de $x[k]$.

Distintos autores pueden definir esta transformada mediante otras expresiones y por lo tanto debe considerarse tales definiciones al interpretar sus propiedades.

Una condición suficiente pero no necesaria para que exista la DTFT emerge de la  [desigualdad triangular o desigualdad de Minkowski](https://en.wikipedia.org/wiki/Triangle_inequality). es:

\begin{equation}
\left|X(e^{j \Omega})\right| = \left| \sum_{k = -\infty}^{\infty} x[k] \, e^{-j \Omega k} \right|
\leq \sum_{k = -\infty}^{\infty} \left| x[k] \right| < \infty
\end{equation}

Por lo cual, si la señal $x[k]$ debe ser absolutamente sumable, existe su DTFT.

La DTFT es periódica con periodo $T_\text{p} = 2 \pi$,

\begin{equation}
X(e^{j \Omega}) = X(e^{j (\Omega + n \cdot 2 \pi)})
\end{equation}

con $n \in \mathbb{Z}$. Observe que  $e^{j \Omega k} = e^{j (\Omega + 2 \pi) k}$ representa oscilaciones puras y también es periódica, por lo que solo es necesario analizar un periodo del espectro de una señal periodica.

La transformada inversa $x[k] = \mathcal{F}_*^{-1} \{ X(e^{j \Omega}) \}$ se define como:

\begin{equation}
x[k] = \frac{1}{2 \pi} \int_{-\pi}^{\pi} X(e^{j \Omega}) \, e^{j \Omega k} \; d \Omega
\end{equation}

## Transformada del Impulso de Dirac $\delta[k]$

La tranasformada $\mathcal{F}_* \{ \delta[k] \}$ del [impulso de Dirac](../discrete_signals/standard_signals.ipynb#Dirac-Impulse) se obtiene al introducir $\delta[k]$ en la definición de DTFT

La señal impulso de Dirac discreta es:

$$
\delta[k] = \left\{\begin{matrix}
0 & k \neq 0 \\
1 & k = 0
\end{matrix}\right.
$$

\begin{split}
\mathcal{F}_* \{ \delta[k] \} &= \sum_{k = -\infty}^{\infty} \delta[k] \, e^{-j \Omega k} \\
& = ... +0e^{-j \Omega (-2)}+0e^{-j \Omega (-1)}+\underline{1}+0e^{-j \Omega 1}+0e^{-j \Omega 2}+... \\
&= 1
\end{split}

Esto indica que todas las frecuencias normalizadas tienen el mismo peso en la señal impulso de Dirac.

## Transformada del Impulso de Dirac desplazado $\delta[k- \kappa]$

La señal impulso de Dirac discreta es:

$$
\delta[k- \kappa] = \left\{\begin{matrix}
0 & k \neq \kappa \\
1 & k = \kappa
\end{matrix}\right.
$$

\begin{split}
\mathcal{F}_* \{ \delta[k] \} &= \sum_{k = -\infty}^{\infty} \delta[k] \, e^{-j \Omega k} \\
& = ... +0e^{-j \Omega (\kappa-2)}+0e^{-j \Omega (\kappa-1)}+1e^{-j \Omega (\kappa)}+0e^{-j \Omega (\kappa+1)}+0e^{-j \Omega (\kappa+2)}+... \\
&= 1e^{-j \Omega (\kappa)}
\end{split}

Esto indica que todas las frecuencias normalizadas tienen el mismo peso en la señal impulso de Dirac pero aparecen términos de desplazamientos temporales $e^{-j \Omega (\kappa)}$.

## Transformada de la señal Rectangular $\text{rect}_N[k]$

Considere una señal rectangular $\text{rect}_N[k]$ .

$$
\text{rect}_N[k] = \left\{\begin{matrix}
0 & k < 0 \\
1 & 0 \leq k < N \\
0 & k \geq N \\
\end{matrix}\right.
$$

\begin{split}
\mathcal{F}_* \{ \text{rect}[k] \} &= \sum_{k = -\infty}^{\infty} \text{rect}[k] \, e^{-j \Omega k} \\
&= \sum_{k = 0}^{N-1} e^{-j \Omega k} \\
&= e^{-j \Omega 0} + e^{-j \Omega 1} + e^{-j \Omega 3} + ... + e^{-j \Omega (N-2)} + e^{-j \Omega (N-1)}     \\
\end{split}

Observe que la serie resultante es una [serie geométrica finita](https://en.wikipedia.org/wiki/Geometric_series) de la forma:

$$ \sum_{k = 0}^{N} a r^k = a \frac{1-r^{N+1}}{1-r} $$

Por lo cual la transformada puede escribirse como:

\begin{split}
\mathcal{F}_* \{ \text{rect}[k] \} &= \sum_{k = 0}^{N-1} 1 (e^{-j \Omega})^k \\
&= 1 \frac{1-(e^{-j \Omega})^N}{1-e^{-j \Omega}} \\
&= 1 \frac{1-e^{-j N \Omega}}{1-e^{-j \Omega}} \\
\end{split}

```python
N = sym.symbols('N', integer=True, positive =True)

F_rect = sym.summation(sym.exp(-sym.I*Omega*k), (k, 0, N-1))
F_rect
```

$\displaystyle \begin{cases} N & \text{for}\: e^{- i \Omega} = 1 \\\frac{1 - e^{- i N \Omega}}{1 - e^{- i \Omega}} & \text{otherwise} \end{cases}$

```python
# Define una señal rectángular
Nr = 5 # Valor N
kr = range(-20, 20) # Rango de visualización
x = list(kr) # inicia valores de la señal
for ind,kx in enumerate(kr):
    if (kx>= 0) & ( kx < Nr):
        x[ind]=1
    else:
        x[ind]=0

plt.rcParams['figure.figsize'] = 9, 3
plt.stem(kr, x)
```

```python
X = sym.summation(sym.exp(-sym.I*Omega*k), (k, 0, Nr-1))
X
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.re(X), (Omega, -20, 20), xlabel=r'$\Omega$', ylabel=r'$Re( X(e^{j \Omega}) )$')
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.im(X), (Omega, -20, 20), xlabel=r'$\Omega$', ylabel=r'$Im( X(e^{j \Omega}) )$')
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.Abs(X), (Omega, -20, 20), xlabel=r'$\Omega$', ylabel=r'$| X(e^{j \Omega}) |$')
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.arg(X), (Omega, -20, 20), xlabel=r'$\Omega$', ylabel=r'$| X(e^{j \Omega}) |$')
```

¿Qué observa en las curvas de magnitud y la fase al  incrementar o disminuir el tamaño de la señal rectangular $N$?

## Transformada de la señal exponencial $e^{\Omega_0 k}\cdot \epsilon [k]$

La señal exponencial causal se define como:

$$
\left\{\begin{matrix}
0 & k < 0 \\
e^{\Omega_0 k} & k \geq 0
\end{matrix}\right.
$$

La transformada es:

\begin{split}
\mathcal{F}_* \{ e^{\Omega_0 k} \epsilon [k] \} &= \sum_{k = 0}^{\infty} e^{\Omega_0 k} \, e^{-j \Omega k} \\
&= \sum_{k = 0}^{\infty} e^{(\Omega_0 - \Omega j)k} \\
&= \frac{1}{1-e^{(\Omega_0 - \Omega j)}} \\
\end{split}

```python
# Define una señal rectángular
kr = range(-20, 20) # Rango de visualización
x = list(kr) # inicia valores de la señal
W0 = -0.5
for ind,kx in enumerate(kr):
    if (kx>= 0):
        x[ind]=np.exp(W0*kx)
    else:
        x[ind]=0

plt.rcParams['figure.figsize'] = 9, 3
plt.stem(kr, np.real(x))

```

```python
plt.rcParams['figure.figsize'] = 9, 3
plt.stem(kr, np.imag(x))
```

```python
X = 1/(1-sym.exp(W0-1j*(Omega)))
rW = (Omega, -2*sym.pi, 2*sym.pi)
X
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.re(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$Re( X(e^{j \Omega}) )$',
         ylim=(-1,3))
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.im(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$Im( X(e^{j \Omega}) )$',
         ylim=(-1,3))
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.Abs(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$| X(e^{j \Omega})|$',
         ylim=(-1,3))
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.arg(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$arg( X(e^{j \Omega}) )$')
```

## Transformada de la señal seno $\sin(\Omega_0 k)\cdot \epsilon [k]$

La señal seno causal se define como:

$$
\left\{\begin{matrix}
0 & k < 0 \\
\sin(\Omega_0 k) = \frac{e^{j\Omega_0 k}-e^{-j\Omega_0 k}}{2j} & k \geq 0
\end{matrix}\right.
$$

La transformada es:

\begin{split}
\mathcal{F}_* \{ \sin(\Omega_0 k)\cdot \epsilon [k] \} &= \mathcal{F}_* \{ \frac{e^{j\Omega_0 k}-e^{-j\Omega_0 k}}{2j}\}\\
&= \frac{\mathcal{F}_* \{e^{j\Omega_0 k}\}-\mathcal{F}_* \{e^{-j\Omega_0 k}\}}{2j}\\
&= \frac{\frac{1}{1-e^{(\Omega_0 - \Omega j)}} - \frac{1}{1-e^{(-\Omega_0 - \Omega j)}}}{2j} \\
\end{split}

```python
# Define una señal rectángular
kr = range(-20, 20) # Rango de visualización de instantes k
x = list(kr) # inicia valores de la señal
W0 = 0.1*np.pi # np.pi/4
for ind,kx in enumerate(kr):
    if (kx>= 0):
        x[ind]=np.sin(W0*kx)
    else:
        x[ind]=0

plt.rcParams['figure.figsize'] = 9, 3
plt.stem(kr, np.real(x))

```

```python
plt.rcParams['figure.figsize'] = 9, 3
plt.stem(kr, np.imag(x))
```

```python
X = (1/(1-sym.exp(1j*W0-1j*(Omega))) - 1/(1-sym.exp(-1j*W0-1j*(Omega))))/2j
rW = (Omega, -2*sym.pi, 2*sym.pi)
X = X.simplify()
X
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.re(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$Re( X(e^{j \Omega}) )$',
         ylim=(-10,10),
         adaptive = False)
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.im(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$Im( X(e^{j \Omega}) )$',
         ylim=(-1,10),
         adaptive = False)
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.Abs(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$| X(e^{j \Omega})|$',
         ylim=(-1,10),
         adaptive = False)

```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.arg(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$arg( X(e^{j \Omega}) )$',
         adaptive = False)
```

Analice los espectros de la señal seno con frecuencias:
- $1$
- $2$
- $\pi/2$
- $3\pi/2$

## Transformada de una señal arbitraria

```python
import soundfile as sf

x, fs = sf.read('datos/JPDP001.wav')# Lee un archivo .wav
x = x.T[0] #Selecciona solo un canal de grabación
sf.write('temp.wav',x,fs)
x = x[30:260:2]

plt.rcParams['figure.figsize'] = 9, 3
plt.stem(x)
len(x)
```

```python
X = 0
for ind,xk in enumerate(x):
    X = X + xk*sym.exp(1j*Omega*ind)
```

```python
plt.rcParams['figure.figsize'] = 9, 3
sym.plot(sym.Abs(X), rW,
         xlabel=r'$\Omega$',
         ylabel=r'$| X(e^{j \Omega})|$',
         ylim=(-1,8),
         adaptive = False)

```

- ¿Qué espera que pase si toma una señal de $440 000$ muestras?

## Propiedades

|            | $x[k]$ | $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ |
|:---|:---:|:---:|
| Linealidad | $A \, x_1[k] + B \, x_2[k]$ | $A \, X_1(e^{j \Omega}) + B \, X_2(e^{j \Omega})$ |
| Convolución | $x[k] * h[k]$ | $X(e^{j \Omega}) \cdot H(e^{j \Omega})$ |
| Multiplicación | $x[k] \cdot h[k]$ | $ X(e^{j \Omega}) * H(e^{j \Omega})$ |
| Desplazamiento | $x[k - \kappa]$ | $e^{-j \Omega \kappa} \cdot X(e^{j \Omega})$ |
| Modulación | $e^{j \Omega_0 k} \cdot x[k]$ | $X(e^{j (\Omega- \Omega_0)})$ |

Con $A, B \in \mathbb{C}$, $\Omega_0 \in \mathbb{R}$ y $\kappa \in \mathbb{Z}$.

```python

```


# Recurrent neural network and dynamical system analysis

In this tutorial, we will use supervised learning to train a recurrent neural network on a parametric working memory task, and analyze the trained network using dynamical system analysis.

[](https://colab.research.google.com/github/gyyang/nn-brain/blob/master/ParametricWorkingMemory.ipynb)

## Install dependencies

```python
# # If on Google Colab, uncomment to install neurogym to use cognitive tasks
# ! git clone https://github.com/gyyang/neurogym.git
# %cd neurogym/
# ! pip install -e .
```

## Defining a cognitive task

```python
import numpy as np
import matplotlib.pyplot as plt
import neurogym as ngym
```

```python
# Environment
task = 'DelayComparison-v0'
timing = {'delay': ('choice', [200, 400, 800, 1600, 3200]),
          'response': ('constant', 500)
         }
kwargs = {'dt': 100, 'timing': timing}
seq_len = 100

# Make supervised dataset
dataset = ngym.Dataset(task, env_kwargs=kwargs, batch_size=16,
                       seq_len=seq_len)

# A sample environment from dataset
env = dataset.env
# Visualize the environment with 2 sample trials
_ = ngym.utils.plot_env(env, num_trials=2, def_act=0)

# Network input and output size
input_size = env.observation_space.shape[0]
output_size = env.action_space.n
```

```python
inputs, target = dataset()
mask = target > 0
print(inputs.shape)   # (N_time, batch_size, N_neuron)
print(target.shape)   # (N_time, batch_size)
```

    (100, 16, 2)
    (100, 16)

```python
plt.plot(target[:, 0])
```

## Define a vanilla continuous-time recurrent network

Here we will define a continuous-time neural network but discretize it in time using the Euler method.
\begin{align}
    \tau \frac{d\mathbf{r}}{dt} = -\mathbf{r}(t) + f(W_r \mathbf{r}(t) + W_x \mathbf{x}(t) + \mathbf{b}_r).
\end{align}

This continuous-time system can then be discretized using the Euler method with a time step of $\Delta t$,
\begin{align}
    \mathbf{r}(t+\Delta t) = \mathbf{r}(t) + \Delta \mathbf{r} = \mathbf{r}(t) + \frac{\Delta t}{\tau}[-\mathbf{r}(t) + f(W_r \mathbf{r}(t) + W_x \mathbf{x}(t) + \mathbf{b}_r)].
\end{align}

```python
# Define networks
import torch
import torch.nn as nn
from torch.nn import init
from torch.nn import functional as F
import math

class CTRNN(nn.Module):
    """Continuous-time RNN.

    Args:
        input_size: Number of input neurons
        hidden_size: Number of hidden neurons

    Inputs:
        input: (seq_len, batch, input_size), network input
        hidden: (batch, hidden_size), initial hidden activity
    """

    def __init__(self, input_size, hidden_size, dt=None, **kwargs):
        super().__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.tau = 100
        if dt is None:
            alpha = 1
        else:
            alpha = dt / self.tau
        self.alpha = alpha
        self.oneminusalpha = 1 - alpha

        self.input2h = nn.Linear(input_size, hidden_size)
        self.h2h = nn.Linear(hidden_size, hidden_size)

    def init_hidden(self, input_shape):
        batch_size = input_shape[1]
        return torch.zeros(batch_size, self.hidden_size)

    def recurrence(self, input, hidden):
        """Recurrence helper."""
        pre_activation = self.input2h(input) + self.h2h(hidden)
        h_new = torch.relu(hidden * self.oneminusalpha +
                           pre_activation * self.alpha)
        return h_new

    def forward(self, input, hidden=None):
        """Propogate input through the network."""
        if hidden is None:
            hidden = self.init_hidden(input.shape).to(input.device)

        output = []
        steps = range(input.size(0))
        for i in steps:
            hidden = self.recurrence(input[i], hidden)
            output.append(hidden)

        output = torch.stack(output, dim=0)
        return output, hidden

class RNNNet(nn.Module):
    """Recurrent network model.

    Args:
        input_size: int, input size
        hidden_size: int, hidden size
        output_size: int, output size
        rnn: str, type of RNN, lstm, rnn, ctrnn, or eirnn
    """
    def __init__(self, input_size, hidden_size, output_size, **kwargs):
        super().__init__()

        # Continuous time RNN
        self.rnn = CTRNN(input_size, hidden_size, **kwargs)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        rnn_activity, _ = self.rnn(x)
        out = self.fc(rnn_activity)
        return out, rnn_activity
```

## Train the recurrent network on the decision-making task

```python
import torch.optim as optim

# Instantiate the network and print information
hidden_size = 64
net = RNNNet(input_size=input_size, hidden_size=hidden_size,
             output_size=output_size, dt=env.dt)
print(net)

# Use Adam optimizer
optimizer = optim.Adam(net.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()

running_loss = 0
running_acc = 0
for i in range(2000):
    inputs, labels_np = dataset()
    labels_np = labels_np.flatten()
    inputs = torch.from_numpy(inputs).type(torch.float)
    labels = torch.from_numpy(labels_np).type(torch.long)

    # in your training loop:
    optimizer.zero_grad()   # zero the gradient buffers
    output, _ = net(inputs)
    output = output.view(-1, output_size)
    loss = criterion(output, labels)
    loss.backward()
    optimizer.step()    # Does the update

    running_loss += loss.item()
    # Compute performance
    output_np = np.argmax(output.detach().numpy(), axis=-1)
    ind = labels_np > 0  # Only analyze time points when target is not fixation
    running_acc += np.mean(labels_np[ind] == output_np[ind])
    if i % 100 == 99:
        running_loss /= 100
        running_acc /= 100
        print('Step {}, Loss {:0.4f}, Acc {:0.3f}'.format(i+1, running_loss, running_acc))
        running_loss = 0
        running_acc = 0
```

    RNNNet(
      (rnn): CTRNN(
        (input2h): Linear(in_features=2, out_features=64, bias=True)
        (h2h): Linear(in_features=64, out_features=64, bias=True)
      )
      (fc): Linear(in_features=64, out_features=3, bias=True)
    )
    Step 100, Loss 0.2606, Acc 0.049
    Step 200, Loss 0.0488, Acc 0.134
    Step 300, Loss 0.0268, Acc 0.543
    Step 400, Loss 0.0197, Acc 0.728
    Step 500, Loss 0.0147, Acc 0.803
    Step 600, Loss 0.0113, Acc 0.839
    Step 700, Loss 0.0109, Acc 0.823
    Step 800, Loss 0.0079, Acc 0.878
    Step 900, Loss 0.0082, Acc 0.866
    Step 1000, Loss 0.0077, Acc 0.872
    Step 1100, Loss 0.0071, Acc 0.878
    Step 1200, Loss 0.0063, Acc 0.895
    Step 1300, Loss 0.0063, Acc 0.897
    Step 1400, Loss 0.0061, Acc 0.897
    Step 1500, Loss 0.0070, Acc 0.877
    Step 1600, Loss 0.0050, Acc 0.918
    Step 1700, Loss 0.0055, Acc 0.907
    Step 1800, Loss 0.0056, Acc 0.899
    Step 1900, Loss 0.0054, Acc 0.904
    Step 2000, Loss 0.0056, Acc 0.899

## Visualize neural activity for in sample trials

We will run the network for 100 sample trials, then visual the neural activity trajectories in a PCA space.

```python
import numpy as np
import gym

# Set delay to 3000ms for analysis
kwargs = {'timing': {'delay': ('constant', 3000)}}
env = gym.make(task, **kwargs)
env.reset(no_step=True)
env.timing

perf = 0
num_trial = 100
activity_dict = {}
trial_infos = {}
for i in range(num_trial):
    env.new_trial()
    ob, gt = env.ob, env.gt
    inputs = torch.from_numpy(ob[:, np.newaxis, :]).type(torch.float)
    action_pred, rnn_activity = net(inputs)
    rnn_activity = rnn_activity[:, 0, :].detach().numpy()
    activity_dict[i] = rnn_activity[env.start_ind['delay']:env.end_ind['delay']]
    trial_infos[i] = env.trial.copy()

# Concatenate activity for PCA
activity = np.concatenate(list(activity_dict[i] for i in range(num_trial)), axis=0)
print('Shape of the neural activity: (Time points, Neurons): ', activity.shape)

# Print trial informations
for i in range(5):
    print('Trial ', i, trial_infos[i])
```

    Shape of the neural activity: (Time points, Neurons):  (3000, 64)
    Trial  0 {'ground_truth': 2, 'fpair': (22, 14), 'f1': 14, 'f2': 22}
    Trial  1 {'ground_truth': 2, 'fpair': (26, 18), 'f1': 18, 'f2': 26}
    Trial  2 {'ground_truth': 1, 'fpair': (30, 22), 'f1': 30, 'f2': 22}
    Trial  3 {'ground_truth': 1, 'fpair': (34, 26), 'f1': 34, 'f2': 26}
    Trial  4 {'ground_truth': 1, 'fpair': (34, 26), 'f1': 34, 'f2': 26}

```python
# Compute PCA and visualize
from sklearn.decomposition import PCA

pca = PCA(n_components=2)
pca.fit(activity)
# print('Shape of the projected activity: (Time points, PCs): ', activity_pc.shape)
```

    PCA(copy=True, iterated_power='auto', n_components=2, random_state=None,
        svd_solver='auto', tol=0.0, whiten=False)

Transform individual trials and Visualize in PC space based on ground-truth color. We see that the neural activity is organized by stimulus ground-truth in PC1

```python
import matplotlib.pyplot as plt

fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(6, 3))
for i in range(num_trial):
    trial = trial_infos[i]
    activity_pc = pca.transform(activity_dict[i])

    color = 'red' if trial['ground_truth'] == 1 else 'blue'

    _ = ax1.plot(activity_pc[:, 0], activity_pc[:, 1], 'o-', color=color)

    if i < 1:
        _ = ax2.plot(activity_pc[:, 0], activity_pc[:, 1], 'o-', color=color)

ax1.set_xlabel('PC 1')
ax1.set_ylabel('PC 2')
```

## Dynamical system analysis

### Search for approximate fixed points
Here we search for approximate fixed points and visualize them in the same PC space. In a generic dynamical system,
\begin{align}
    \frac{d\mathbf{x}}{dt} = F(\mathbf{x}),
\end{align}
We can search for fixed points by doing the optimization
\begin{align}
    \mathrm{argmin}_{\mathbf{x}} |F(\mathbf{x})|^2.
\end{align}

```python
activity.shape
```

    (3000, 64)

```python
# Freeze for parameters in the recurrent network
for param in net.parameters():
    param.requires_grad = False

batch_size = 64

# Inputs should be the 0-coherence mean input during stimulus period
# This will be task-specific
input = np.tile([1, 0], (batch_size, 1))
input = torch.tensor(input, dtype=torch.float32)

# Here hidden activity is the variable to be optimized
# Initialized randomly for search in parallel (activity all positive)
# hidden_init = np.random.rand(batch_size, hidden_size)*3
hidden_init = activity[np.random.randint(activity.shape[0], size=(batch_size,))]
hidden_init = np.random.uniform(0.5, 1.5, size=hidden_init.shape) * hidden_init
hidden = torch.tensor(hidden_init, requires_grad=True, dtype=torch.float32)

# Use Adam optimizer
optimizer = optim.Adam([hidden], lr=0.01)
criterion = nn.MSELoss()

running_loss = 0
for i in range(10000):
    optimizer.zero_grad()   # zero the gradient buffers

    # Take the one-step recurrent function from the trained network
    new_h = net.rnn.recurrence(input, hidden)
    loss = criterion(new_h, hidden)
    loss.backward()
    optimizer.step()    # Does the update

    running_loss += loss.item()
    if i % 1000 == 999:
        running_loss /= 1000
        print('Step {}, Loss {:0.4f}'.format(i+1, running_loss))
        running_loss = 0
```

    Step 1000, Loss 0.0553
    Step 2000, Loss 0.0021
    Step 3000, Loss 0.0008
    Step 4000, Loss 0.0003
    Step 5000, Loss 0.0001
    Step 6000, Loss 0.0001
    Step 7000, Loss 0.0001
    Step 8000, Loss 0.0000
    Step 9000, Loss 0.0000
    Step 10000, Loss 0.0000

### Visualize the found approximate fixed points.

We see that they found an approximate line attrator, corresponding to our PC1, along which evidence is integrated during the stimulus period.

```python
fixedpoints = hidden.detach().numpy()
print(fixedpoints.shape)

# Plot in the same space as activity
plt.figure()
for i in range(5):
    activity_pc = pca.transform(activity_dict[i])
    trial = trial_infos[i]
    color = 'red' if trial['ground_truth'] == 0 else 'blue'
    plt.plot(activity_pc[:, 0], activity_pc[:, 1], 'o-',
             color=color, alpha=0.1)

# Fixed points are shown in cross
fixedpoints_pc = pca.transform(fixedpoints)
plt.plot(fixedpoints_pc[:, 0], fixedpoints_pc[:, 1], 'x')

plt.xlabel('PC 1')
plt.ylabel('PC 2')
```

### Computing the Jacobian and finding the line attractor

First we will compute the Jacobian.

```python
# index of fixed point to focus on
# choose one close to center by sorting PC1
i_fp = np.argsort(fixedpoints[:, 0])[int(fixedpoints.shape[0]/2)]

fp = torch.from_numpy(fixedpoints[i_fp])
fp.requires_grad = True

# Inputs should be the 0-coherence mean input during stimulus period
# This will be task-specific
input = torch.tensor([1, 0], dtype=torch.float32)
deltah = net.rnn.recurrence(input, fp) - fp

jacT = torch.zeros(hidden_size, hidden_size)
for i in range(hidden_size):
    output = torch.zeros(hidden_size)
    output[i] = 1.
    jacT[:,i] = torch.autograd.grad(deltah, fp, grad_outputs=output, retain_graph=True)[0]

jac = jacT.detach().numpy().T
```

Here we plot the direction of the eigenvector corresponding to the highest eigenvalue

```python
eigval, eigvec = np.linalg.eig(jac)
vec = np.real(eigvec[:, np.argmax(eigval)])
end_pts = np.array([+vec, -vec]) * 10
end_pts = pca.transform(fp.detach().numpy() + end_pts)

# Plot in the same space as activity
plt.figure()
for i in range(5):
    activity_pc = pca.transform(activity_dict[i])
    trial = trial_infos[i]
    color = 'red' if trial['ground_truth'] == 0 else 'blue'
    plt.plot(activity_pc[:, 0], activity_pc[:, 1], 'o-',
             color=color, alpha=0.1)

# Fixed points are shown in cross
fixedpoints_pc = pca.transform(fixedpoints)
plt.plot(fixedpoints_pc[:, 0], fixedpoints_pc[:, 1], 'x')

# Line attractor
plt.plot(end_pts[:, 0], end_pts[:, 1])

plt.xlabel('PC 1')
plt.ylabel('PC 2')
```

```python
# Plot distribution of eigenvalues in a 2-d real-imaginary plot
plt.figure()
plt.scatter(np.real(eigval), np.imag(eigval))
plt.plot([0, 0], [-1, 1], '--')
plt.xlabel('Real')
plt.ylabel('Imaginary')
```

# Supplementary Materials

Code for making publication quality figures as it appears in the paper.

```python
# Convert information into pandas dataframe
import pandas as pd
df = pd.DataFrame()
for i in range(len(trial_infos)):
    df = df.append(trial_infos[i], ignore_index=True)
# Example selection of conditions
# print(df[df['f1']==22])
```

```python
import matplotlib as mpl
plot_fp = True

# Plot in the same space as activity
# fig = plt.figure(figsize=(3, 3))
fig, axes = plt.subplots(1, 2, sharex=True, sharey=True, figsize=(4, 2))

for i in range(2):
    ax = axes[i]
    plot_fp = i == 1
#     ax = fig.add_axes([0.2, 0.2, 0.6, 0.6])

    colors = np.array([[27,158,119], [117,112,179], [217,95,2]])/255.

    # Search for two trials with similar conditions
    values = np.unique(df['f1'])

    color_intensity = [0.4, 0.7, 1.0, 1.3]
    cmap = mpl.cm.get_cmap('winter')
    if plot_fp:
        alpha = 0.2
    else:
        alpha = 1.0
    for i, val in enumerate(values):
        trials = df[df['f1']==val].index
        activity = np.mean(np.array([activity_dict[i] for i in trials]), axis=0)

        activity_pc = pca.transform(activity)
        label = '{:0.1f}'.format(val)
        color = cmap(i/len(values))
        ax.plot(activity_pc[:, 0], activity_pc[:, 1], 'o-',
                 color=color, ms=3, markeredgecolor='none',
                 lw=1, label=label, alpha=alpha)
        ax.plot(activity_pc[0, 0], activity_pc[0, 1], 'o-', alpha=alpha,
                marker='^', color=color, ms=5)

    if plot_fp:
        # Fixed points are shown in cross
        color = colors[2]
        fixedpoints_pc = pca.transform(fixedpoints)
        ax.plot(fixedpoints_pc[:, 0], fixedpoints_pc[:, 1], 'x', ms=3, color=color, alpha=0.3)

        # Line attractor
        ax.plot(fixedpoints_pc[i_fp, 0], fixedpoints_pc[i_fp, 1], 'x', ms=5, color=color, lw=1)
        ax.plot(end_pts[:, 0], end_pts[:, 1], color=color)
    else:
        ax.legend(title='Stimulus', loc='upper left', bbox_to_anchor=(1.0, 1.0), frameon=False)

    ax.set_xlabel('PC 1', fontsize=7)
    ax.set_ylabel('PC 2', fontsize=7)

    # plt.xlim([-5, 5])
    # plt.ylim([-1, 5])

    # Beautification
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    # ax.spines['left'].set_position(('data', -5))
    # ax.spines['bottom'].set_position(('data', -1.5))
plt.tight_layout()
plt.locator_params(nbins=2)
from pathlib import Path
# if plot_fp:
#     fname = Path('figures/lineattractors_parametricWM')
# else:
#     fname = Path('figures/rnndynamics_parametricWM')
fname = Path('figures/rnndynamics_parametricWM')
fig.savefig(fname.with_suffix('.pdf'), transparent=True)
fig.savefig(fname.with_suffix('.png'), dpi=300)
```

```python

```


Sascha Spors,
Professorship Signal Theory and Digital Signal Processing,
Institute of Communications Engineering (INT),
Faculty of Computer Science and Electrical Engineering (IEF),
University of Rostock,
Germany

# Tutorial Digital Signal Processing

**DFT to DTFT Interpolation with the Periodic Sinc Function**,
Winter Semester 2021/22 (Course #24505)

- lecture: https://github.com/spatialaudio/digital-signal-processing-lecture
- tutorial: https://github.com/spatialaudio/digital-signal-processing-exercises

Feel free to contact lecturer frank.schultz@uni-rostock.de

```python
import numpy as np
from matplotlib import pyplot as plt
from numpy.fft import fft, ifft
#from scipy.fft import fft, ifft
from scipy.special import diric
```

# DFT to DTFT Interpolation with the Periodic Sinc Function

The DFT spectrum $X[\mu]$ can be interpolated towards the DTFT spectrum $X(\Omega)$ with

\begin{equation}
X(\Omega)=\sum_{\mu=0}^{N-1}X[\mu]\cdot\mathrm{e}^{-\mathrm{j}\frac{\left(\Omega-\frac{2\pi}{N}\mu\right)(N-1)}{2}}\cdot\text{psinc}_N\left(\Omega-\frac{2\pi}{N}\mu\right).
\end{equation}

This interpolation implies:
- the DFT $X[\mu]$ stems from a signal $x[k]$ for which periodicity of $N$ is inherent, we assume the first period at $0\leq k\leq N-1$
- the DFT spectrum is discrete and $N$ periodic
- the DTFT spectrum stems from the signal $x[k]$ for $0\leq k\leq N-1$, for all other $k$ is $x[k]=0$, i.e. considering only one period
- the DTFT spectrum is continuous and $2\pi$ periodic, which means that the signal $x[k]$ can not be periodic

The interpolation kernel utilises the so-called **periodic sinc function**

\begin{align}
\text{psinc}_N(\Omega)=\begin{cases}\frac{1}{N}\cdot\frac{\sin\left(\frac{N}{2}\Omega\right)}{\sin\left(\frac{1}{2}\Omega\right)}&\text{for }\Omega\neq2\pi m\\
(-1)^{m(N-1)}&\text{for }\Omega=2\pi m\end{cases},\,\,m\in\mathbb{Z},
\end{align}

which is also known as aliased sinc and Dirichlet function.

Below, we give an example graph for $\text{psinc}_N(\Omega)$.
Note that the orange dots indicate the DFT eigenfrequencies.

```python
def dft2dtft(X, W):
    N = len(X)
    Xi = np.zeros(len(W), dtype='complex')
    for i, Omega in enumerate(W):
        for mu_dft in range(N):
            Xi[i] += X[mu_dft] *\
                np.exp(-1j/2*(Omega - 2*np.pi/N*mu_dft)*(N-1)) *\
                diric(Omega - 2*np.pi/N*mu_dft, N)
    return Xi
```

```python
K = 3
N_DFT = 2**K
Omega_DFT = np.arange(-2*N_DFT, 2*N_DFT) * 2*np.pi/N_DFT
N = 2**(K+4)
Omega = np.arange(-2*N, 2*N) * 2*np.pi/N
psinc_DFT = diric(Omega_DFT, N_DFT)
psinc = diric(Omega, N_DFT)

plt.plot(Omega, psinc, label='psinc function')
plt.plot(Omega_DFT, psinc_DFT, 'o', label='DFT bins')
plt.xlim(-4*np.pi, 4*np.pi)
plt.xticks(np.arange(-4, 5, 1)*np.pi)
plt.xlabel(r'$\Omega$')
plt.ylabel(r'psinc($\Omega$)')
plt.title('psinc for '+str(N_DFT)+'-point DFT')
plt.legend()
plt.grid(True)
```

## Task: DFT Analysis Using a Rectangular Window

A complex signal

- $x[k]=\mathrm{exp}(\mathrm{j}(\Omega k + \frac{\pi}{\pi}))$ with
- $\Omega=4\cdot\frac{2\pi}{N}$,
- $N=8$

is to be analysed with the DFT
\begin{align}
X[\mu]=\sum_{k=0}^{N-1}x[k]\cdot\mathrm{e}^{-\mathrm{j}\frac{2\pi}{N}k\mu}
\end{align}
for $0\leq k \leq N-1$.

Furthermore, assume that $x[k]$ results from continuous-time signal $x(t)$ using sampling frequency of $f_s=10$ Hz.

1. Plot the discrete-time signals that correspond to the DFT and the DTFT spectrum.

2. Calculate the DFT spectrum $X[\mu]$ of $x[k]$ and visualise the real and imaginary part as well as the magnitude and the phase of $X[\mu]$ over $0\leq\mu\leq N-1$.

3. Implement the above mentioned interpolation towards the DTFT and visualise the resulting magnitude spectra $|X[\mu]|$, $|X(\Omega)|$ over frequency axes $\mu$, $\Omega$ as well as the physical frequency $f$.

## Task 1: Generate Signal and Plot

```python
N = 8

Om = 4 * 2*np.pi/N  # play with the factor 4

k = np.arange(N)
x = np.exp(1j*(Om*k+1))  # pi/pi=1

koffs = [-2, -1, 1, 2]
plt.subplot(2, 1, 1)
plt.plot(k, x.real, 'C0o-', label='real', ms=5)
plt.plot(k, x.imag, 'C1o-', label='imag', ms=5)
# the DFT spectrum corresponds to a periodic signal of period N
# we exemplarily indicate this
for koffsi in koffs:
    plt.plot(k+koffsi*N, x.real, 'C0o-', ms=3)
    plt.plot(k+koffsi*N, x.imag, 'C1o-', ms=3)
plt.xlim(koffs[0]*N, koffs[-1]*N+N-1)
plt.xlabel(r'$k$')
plt.ylabel(r'$x_\mathrm{DFT}[k]$')
plt.legend()
plt.grid(True)
plt.subplot(2, 1, 2)
plt.plot(k, x.real, 'C0o-', ms=5)
plt.plot(k, x.imag, 'C1o-', ms=5)
# the DTFT spectrum corresponds to singular event of length N for k=0...N-1
# we exemplarily indicate this
for koffsi in koffs:
    plt.plot(k+koffsi*N, 0*x.real, 'C0o-', ms=3)
    plt.plot(k+koffsi*N, 0*x.imag, 'C1o-', ms=3)
plt.xlim(koffs[0]*N, koffs[-1]*N+N-1)
plt.xlabel(r'$k$')
plt.ylabel(r'$x_\mathrm{DTFT}[k]$')
plt.grid(True)
```

## Task 2: DFT and Plot

```python
mu = np.arange(N)
w = np.ones(N)

# optional playground for windowing
# see dft_windowing_tutorial.pdf for the equations of these windows
# note that the scipy.signal.windows with same name behave different !
# thus standardised names does NOT imply standardised characteristics
if False:
    # Hann, two unused zeros
    w = (1 - np.cos(2*np.pi/N*(k+1/2))) / 2
    # Hamming, put notch into first sidelobe, thus all zeros used
    w = 0.54 - 0.46 * np.cos(2*np.pi/N*(k+1/2))

X = fft(x*w)
```

```python
plt.figure(figsize=(8, 4))

plt.subplot(2, 2, 1)
plt.plot(mu, np.real(X), 'o')
plt.xlim(0, N-1)
plt.xlabel(r'$\mu$')
plt.ylabel(r'$\Re(X[\mu])$')
plt.grid(True)

plt.subplot(2, 2, 2)
plt.plot(mu, np.imag(X), 'o')
plt.xlim(0, N-1)
plt.xlabel(r'$\mu$')
plt.ylabel(r'$\Im(X[\mu])$')
plt.grid(True)

plt.subplot(2, 2, 3)
plt.plot(mu, np.abs(X), 'o')
plt.xlim(0, N-1)
plt.xlabel(r'$\mu$')
plt.ylabel(r'$|X[\mu]|$')
plt.grid(True)

# note that only the phase at mu=4 is a meaningful result (=1)
# all other values are due to numerical noise / trash
# since Re and Im are approx zero at these mu
plt.subplot(2, 2, 4)
plt.plot(mu, np.angle(X), 'o')
plt.xlim(0, N-1)
plt.xlabel(r'$\mu$')
plt.ylabel(r'$\angle X[\mu]$')
plt.grid(True)
```

## Task 3: DFT -> DTFT Interpolation, Plot

```python
fs = 10
Ni = 2**8
W = np.arange(Ni) * 2*np.pi/Ni  # Omega for which to interpolate to DTFT
f = W / (2*np.pi) * fs  # physical frequency based on W using fs
mui = W / (2*np.pi) * N  # frequency vector normalized to integer frequencies
Xi = dft2dtft(X, W)  # DTFT interpolation from DFT

plt.figure(figsize=(10, 10))

plt.subplot(3, 2, 1)
plt.plot(mu, np.abs(X), 'C0o', label='DFT')
plt.plot(mui, np.abs(Xi), 'C1-', label='DTFT')
plt.xlabel(r'$\mu$')
plt.ylabel('|X|')
plt.legend()
plt.grid(True)

plt.subplot(3, 2, 2)
plt.plot(mu, 20*np.log10(np.abs(X)), 'C0o', label='DFT')
plt.plot(mui, 20*np.log10(np.abs(Xi)), 'C1-', label='DTFT')
plt.ylim(-100, 20)
plt.xlabel(r'$\mu$')
plt.ylabel('20lg|X| / dB')
plt.grid(True)

plt.subplot(3, 2, 3)
plt.plot(2*np.pi/N*mu, np.abs(X), 'C0o', label='DFT')
plt.plot(W, np.abs(Xi), 'C1-', label='DTFT')
plt.xlabel(r'$\Omega$')
plt.ylabel('|X|')
plt.grid(True)

plt.subplot(3, 2, 4)
plt.plot(2*np.pi/N*mu, 20*np.log10(np.abs(X)), 'C0o', label='DFT')
plt.plot(W, 20*np.log10(np.abs(Xi)), 'C1-', label='DTFT')
plt.ylim(-100, 20)
plt.xlabel(r'$\Omega$')
plt.ylabel('20lg|X| / dB')
plt.grid(True)

plt.subplot(3, 2, 5)
plt.plot(mu*fs/N, np.abs(X), 'C0o', label='DFT')
plt.plot(f, np.abs(Xi), 'C1-', label='DTFT')
plt.xlabel(r'f / Hz')
plt.ylabel('|X|')
plt.grid(True)

plt.subplot(3, 2, 6)
plt.plot(mu*fs/N, 20*np.log10(np.abs(X)), 'C0o', label='DFT')
plt.plot(f, 20*np.log10(np.abs(Xi)), 'C1-', label='DTFT')
plt.ylim(-100, 20)
plt.xlabel(r'f / Hz')
plt.ylabel('20lg|X| / dB')
plt.grid(True)
```

We observe that 7 of 8 DFT coeefficients are precisely zero, and only $|X[\mu=4]|=8$. This is intentional and is expected for the chosen signal that oscillates with exactly a DFT eigenfrequency, here half of the sampling frequency.

The code above allows for two more playgrounds:

- We might change $\Omega=1\cdot\frac{2\pi}{N}$, $\Omega=2\cdot\frac{2\pi}{N}$, $\Omega=3\cdot\frac{2\pi}{N}$
and observe how the spectrum moves around the frequency axis, however the shape is preserved.
The main lobe precisely corresponds to the chosen $\mu$.

- We might change $\Omega=1.5\cdot\frac{2\pi}{N}$, $\Omega=2.5\cdot\frac{2\pi}{N}$, $\Omega=3.5\cdot\frac{2\pi}{N}$
and observe that - since these frequencies are **no** DFT eigenfrequencies - now all DFT coefficients exhibit energy. This suggests frequencies in the signal which are actually not there, but originate from cutting the signal to the chosen block size $N$. The effect is known as **leakage effect**.

- In task 2 we might optionally apply a window to the signal. The plots for task 3 then reveal how the leakage effect is reduced but the main lobe gets broader.
Since we have chosen a mono-frequent, complex oscillation as input signal to the DFT / DTFT, the actual window spectrum can be directly seen in the plots.

**Copyright**

The notebooks are provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebooks for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Frank Schultz, Digital Signal Processing - A Tutorial Featuring Computational Examples* with the URL https://github.com/spatialaudio/digital-signal-processing-exercises