# Gaussian process regression with varying output noise

This notebook shows how to construct a Gaussian process model where different noise is assumed for different data points. The model is:

\begin{align}
f(\cdot) &\sim \mathcal{GP}\big(0, k(\cdot, \cdot)\big) \\
y_i | f, x_i &\sim \mathcal N\big(y_i; f(x_i), \sigma^2_i\big)
\end{align}

We'll demonstrate two methods. In the first demonstration, we'll assume that the noise variance is known for every data point. We'll incorporate the known noise variances $\sigma^2_i$ into the data matrix $\mathbf Y$, make a likelihood that can deal with this structure, and implement inference using variational GPs with natural gradients.

In the second demonstration, we'll assume that the noise variance is not known, but we'd like to estimate it for different groups of data. We'll show how to construct an appropriate likelihood for this task and set up inference similarly to the first demonstration, with optimization over the noise variances.

```python
import numpy as np
import tensorflow as tf
import gpflow
from gpflow.ci_utils import ci_niter
from gpflow.optimizers import NaturalGradient
from gpflow import set_trainable
import matplotlib.pyplot as plt

%matplotlib inline
```

## Demo 1: known noise variances
### Generate synthetic data
We create a utility function to generate synthetic data, including noise that varies amongst the data:

```python
np.random.seed(1)  # for reproducibility

def generate_data(N=80):
    X = np.random.rand(N)[:, None] * 10 - 5  # Inputs, shape N x 1
    F = 2.5 * np.sin(6 * X) + np.cos(3 * X)  # Mean function values
    NoiseVar = 2 * np.exp(-((X - 2) ** 2) / 4) + 0.3  # Noise variances
    Y = F + np.random.randn(N, 1) * np.sqrt(NoiseVar)  # Noisy data
    return X, Y, NoiseVar

X, Y, NoiseVar = generate_data()
```

Here's a plot of the data, with error bars representing two standard deviations:

```python
fig, ax = plt.subplots(1, 1, figsize=(12, 6))
_ = ax.errorbar(
    X.squeeze(),
    Y.squeeze(),
    yerr=2 * (np.sqrt(NoiseVar)).squeeze(),
    marker="x",
    lw=0,
    elinewidth=1.0,
    color="C1",
)
```

### Make a Y matrix that includes the variances
We need to tell the GP model what the variance is for each data point. To do this, we'll concatenate the observations with the variances into a single data matrix:

```python
Y_data = np.hstack([Y, NoiseVar])
```

### Make a new likelihood

To cope with this data structure, we'll build a new likelihood. Note how the code extracts the observations `Y` and the variances `NoiseVar` from the data. For more information on creating new likelihoods, see [Likelihood design](../tailor/likelihood_design.ipynb). Here, we're implementing the `log_prob` function (which computes the log-probability of the data given the latent function) and `variational_expectations`, which computes the expected log-probability under a Gaussian distribution on the function, and is needed in the evaluation of the evidence lower bound (ELBO). Check out the docstring for the `Likelihood` object for more information on what these functions do.

```python
class HeteroskedasticGaussian(gpflow.likelihoods.Likelihood):
    def __init__(self, **kwargs):
        # this likelihood expects a single latent function F, and two columns in the data matrix Y:
        super().__init__(latent_dim=1, observation_dim=2, **kwargs)

    def _log_prob(self, F, Y):
        # log_prob is used by the quadrature fallback of variational_expectations and predict_log_density.
        # Because variational_expectations is implemented analytically below, this is not actually needed,
        # but is included for pedagogical purposes.
        # Note that currently relying on the quadrature would fail due to https://github.com/GPflow/GPflow/issues/966
        Y, NoiseVar = Y[:, 0], Y[:, 1]
        return gpflow.logdensities.gaussian(Y, F, NoiseVar)

    def _variational_expectations(self, Fmu, Fvar, Y):
        Y, NoiseVar = Y[:, 0], Y[:, 1]
        Fmu, Fvar = Fmu[:, 0], Fvar[:, 0]
        return (
            -0.5 * np.log(2 * np.pi)
            - 0.5 * tf.math.log(NoiseVar)
            - 0.5 * (tf.math.square(Y - Fmu) + Fvar) / NoiseVar
        )

    # The following two methods are abstract in the base class.
    # They need to be implemented even if not used.

    def _predict_log_density(self, Fmu, Fvar, Y):
        raise NotImplementedError

    def _predict_mean_and_var(self, Fmu, Fvar):
        raise NotImplementedError
```

### Put it together with Variational Gaussian Process (VGP)
Here we'll build a variational GP model with the previous likelihood on the dataset that we generated. We'll use the natural gradient optimizer (see [Natural gradients](natural_gradients.ipynb) for more information).

The variational GP object is capable of variational inference with any GPflow-derived likelihood. Usually, the inference is an inexact (but pretty good) approximation, but in the special case considered here, where the noise is Gaussian, it will achieve exact inference. Optimizing over the variational parameters is easy using the natural gradients method, which provably converges in a single step.

Note: We mark the variational parameters as not trainable so that they are not included in the `model.trainable_variables` when we optimize using the Adam optimizer. We train the variational parameters separately using the natural gradient method.

```python
# model construction
likelihood = HeteroskedasticGaussian()
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)
```

Notice that we specify num_latent_gps=1, as the VGP model would normally infer this from the shape of Y_data, but we handle the second column manually in the HeteroskedasticGaussian likelihood.

```python
natgrad = NaturalGradient(gamma=1.0)
adam = tf.optimizers.Adam()

set_trainable(model.q_mu, False)
set_trainable(model.q_sqrt, False)

for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
    adam.minimize(model.training_loss, model.trainable_variables)
```

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

plt.figure(figsize=(12, 6))
plt.plot(xx, mu, "C0")
plt.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
plt.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

plt.errorbar(
    X.squeeze(),
    Y.squeeze(),
    yerr=2 * (np.sqrt(NoiseVar)).squeeze(),
    marker="x",
    lw=0,
    elinewidth=1.0,
    color="C1",
)
_ = plt.xlim(-5, 5)
```

### Questions for the reader
1) What is the difference in meaning between the orange vertical bars and the blue regions in the prediction?

2) Why did we not implement `_conditional_mean` and `_conditional_variance` in the HeteroskedasticGaussian likelihood? What could be done here?

2) What are some better kernel settings for this dataset? How could they be estimated?

## Demo 2: grouped noise variances

In this demo, we won't assume that the noise variances are known, but we will assume that they're known in two groups. This example represents a case where we might know that an instrument has varying fidelity for different regions, but we do not know what those fidelities are.

Of course it would be straightforward to add more groups, or even one group per data point. We'll stick with two for simplicity.

```python
np.random.seed(1)  # for reproducibility and to make it independent from demo 1
```

### Generate data

```python
def generate_data(N=100):
    X = np.random.rand(N)[:, None] * 10 - 5  # Inputs, shape N x 1
    F = 2.5 * np.sin(6 * X) + np.cos(3 * X)  # Mean function values
    groups = np.where(X > 0, 0, 1)
    NoiseVar = np.array([0.02, 0.5])[groups]  # Different variances for the two groups
    Y = F + np.random.randn(N, 1) * np.sqrt(NoiseVar)  # Noisy data
    return X, Y, groups

X, Y, groups = generate_data()
```

```python
# here's a plot of the raw data.
fig, ax = plt.subplots(1, 1, figsize=(12, 6))
_ = ax.plot(X, Y, "kx")
```

### Data structure

In this case, we need to let the model know which group each data point belongs to. We'll use a similar trick to the above, stacking the group identifier with the data:

```python
Y_data = np.hstack([Y, groups])
```

### Build a likelihood

This time, we'll use a builtin likelihood, `SwitchedLikelihood`, which is a container for other likelihoods, and applies them to the first `Y_data` column depending on the index in the second. We're able to access and optimize the parameters of those likelihoods. Here, we'll (incorrectly) initialize the variances of our likelihoods to 1, to demonstrate how we can recover reasonable values for these through maximum-likelihood estimation.

```python
likelihood = gpflow.likelihoods.SwitchedLikelihood(
    [gpflow.likelihoods.Gaussian(variance=1.0), gpflow.likelihoods.Gaussian(variance=1.0)]
)
```

```python
# model construction (notice that num_latent_gps is 1)
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)
```

```python
for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
```

We've now fitted the VGP model to the data, but without optimizing over the hyperparameters. Plotting the data, we see that the fit is not terrible, but hasn't made use of our knowledge of the varying noise.

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

fig, ax = plt.subplots(1, 1, figsize=(12, 6))
ax.plot(xx, mu, "C0")
ax.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
ax.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

ax.plot(X, Y, "C1x", mew=2)
_ = ax.set_xlim(-5, 5)
```

### Optimizing the noise variances
Here we'll optimize over both the noise variance and the variational parameters, applying natural gradients interleaved with the Adam optimizer.

As before, we mark the variational parameters as not trainable, in order to train them separately with the natural gradient method.

```python
likelihood = gpflow.likelihoods.SwitchedLikelihood(
    [gpflow.likelihoods.Gaussian(variance=1.0), gpflow.likelihoods.Gaussian(variance=1.0)]
)
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)

set_trainable(model.q_mu, False)
set_trainable(model.q_sqrt, False)

for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
    adam.minimize(model.training_loss, model.trainable_variables)
```

### Plotting the fitted model

Now that the noise variances have been estimated, we can see the final model fit.
The predictive variance is higher on the left side of the plot, where we know that the data have different variance.
We'll plot the known underlying function in green to see how effectively we've recovered the ground truth.
We can also print the model to examine the estimated noise variances:

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

fig, ax = plt.subplots(1, 1, figsize=(12, 6))
ax.plot(xx, mu, "C0")
ax.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
ax.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

ax.plot(X, Y, "C1x", mew=2)
ax.set_xlim(-5, 5)
_ = ax.plot(xx, 2.5 * np.sin(6 * xx) + np.cos(3 * xx), "C2--")
```

```python
from gpflow.utilities import print_summary

print_summary(model, fmt="notebook")
```

name                                  	class    	transform       	prior  	trainable  	shape        	dtype  	value

VGP.kernel.variance                   	Parameter	Softplus        	       	True       	()           	float64	2.0487746335100305

VGP.kernel.lengthscales               	Parameter	Softplus        	       	True       	()           	float64	0.21743510667548904

VGP.likelihood.likelihoods[0].variance	Parameter	Softplus + Shift	       	True       	()           	float64	0.29148168577591194

VGP.likelihood.likelihoods[1].variance	Parameter	Softplus + Shift	       	True       	()           	float64	0.6055446990450606

VGP.q_mu                              	Parameter	Identity        	       	False      	(100, 1)     	float64	[[1.15102310e+00...

VGP.q_sqrt                            	Parameter	FillTriangular  	       	False      	(1, 100, 100)	float64	[[[2.06898404e-01, 0.00000000e+00, 0.00000000e+00...

## Further reading
To model the variance using a second GP, see the [Heteroskedastic regression notebook](heteroskedastic.ipynb).


```
!pip install einsteinpy
```

    Collecting einsteinpy
      Downloading https://files.pythonhosted.org/packages/35/bb/4e379b56b8cd089c90511027fea2a7cc54d7a6dc49af9ad66a7f39e99448/einsteinpy-0.3.0-py3-none-any.whl (111kB)
         |████████████████████████████████| 112kB 7.4MB/s
    Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (1.1.1)
    Requirement already satisfied: plotly>=4.0 in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (4.4.1)
    Requirement already satisfied: astropy in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (4.0.1.post1)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (1.18.5)
    Requirement already satisfied: numba!=0.49.0,>=0.46; implementation_name == "cpython" in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (0.48.0)
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (3.2.2)
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.6/dist-packages (from einsteinpy) (1.4.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.1->einsteinpy) (1.1.0)
    Requirement already satisfied: retrying>=1.3.3 in /usr/local/lib/python3.6/dist-packages (from plotly>=4.0->einsteinpy) (1.3.3)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from plotly>=4.0->einsteinpy) (1.15.0)
    Requirement already satisfied: llvmlite<0.32.0,>=0.31.0dev0 in /usr/local/lib/python3.6/dist-packages (from numba!=0.49.0,>=0.46; implementation_name == "cpython"->einsteinpy) (0.31.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from numba!=0.49.0,>=0.46; implementation_name == "cpython"->einsteinpy) (50.3.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->einsteinpy) (2.4.7)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->einsteinpy) (1.2.0)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->einsteinpy) (0.10.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->einsteinpy) (2.8.1)
    Installing collected packages: einsteinpy
    Successfully installed einsteinpy-0.3.0

```
import numpy as np
import pandas as pd
import matplotlib as plt
%matplotlib inline
```

# So mass of Cygnus X1 is 16 Solar mass = 3.182e+31

## Plotting Spacial Hypersurface Embedding for Schwarzschild Space-Time for Cygnus X1

```
from einsteinpy.hypersurface import SchwarzschildEmbedding
from einsteinpy.plotting import HypersurfacePlotter
from astropy import units as u
```

```
surface_obj = SchwarzschildEmbedding(3.182e+31 * u.kg)
```

```
surface = HypersurfacePlotter(embedding=surface_obj, plot_type='surface')
surface.plot()
surface.show()
```

## Shadow cast by an thin emission disk around Cygnus X1

```
from einsteinpy.utils import kerr_utils, schwarzschild_radius
```

```
M = 3.182e+31
scr = schwarzschild_radius(M * u.kg).value
# for nearly maximally rotating black hole
a1 = 0.499999*scr
# for ordinary black hole
a2 = 0.3*scr
```

#### Calculating the ergosphere and event horizon for spherical coordinates

```
ergo1, ergo2, hori1, hori2 = list(), list(), list(), list()
thetas = np.linspace(0, np.pi, 720)
for t in thetas:
    ergo1.append(kerr_utils.radius_ergosphere(M, a1, t, "Spherical"))
    ergo2.append(kerr_utils.radius_ergosphere(M, a2, t, "Spherical"))
    hori1.append(kerr_utils.event_horizon(M, a1, t, "Spherical"))
    hori2.append(kerr_utils.event_horizon(M, a2, t, "Spherical"))
ergo1, ergo2, hori1, hori2 = np.array(ergo1), np.array(ergo2), np.array(hori1), np.array(hori2)
```

#### Calculating the X, Y coordinates for plotting

```
Xe1, Ye1 = ergo1[:,0] * np.sin(ergo1[:,1]), ergo1[:,0] * np.cos(ergo1[:,1])
Xh1, Yh1 = hori1[:,0] * np.sin(hori1[:,1]), hori1[:,0] * np.cos(hori1[:,1])
Xe2, Ye2 = ergo2[:,0] * np.sin(ergo2[:,1]), ergo2[:,0] * np.cos(ergo2[:,1])
Xh2, Yh2 = hori2[:,0] * np.sin(hori2[:,1]), hori2[:,0] * np.cos(hori2[:,1])
```

#### Maximally Rotating

```
%matplotlib inline
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
# for maximally rotating black hole
ax.fill(Xh1, Yh1, 'b', Xe1, Ye1, 'r', alpha=0.3)
ax.fill(-1*Xh1, Yh1, 'b', -1*Xe1, Ye1, 'r', alpha=0.3)
```

#### Normally Rotating

```
%matplotlib inline
fig, ax = plt.subplots()
ax.fill(Xh2, Yh2, 'b', Xe2, Ye2, 'r', alpha=0.3)
ax.fill(-1*Xh2, Yh2, 'b', -1*Xe2, Ye2, 'r', alpha=0.3)
```

# The inner body represents event horizon and outer one represents ergosphere. It can be concluded that with decrease in angular momentum, radius of event horizon increases, and that of ergosphere decreases.

#

```

```


# Characterization of Discrete Systems in the Spectral Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Combination of Systems

The representation of systems with a complex structure as combination of simpler systems is often convenient for their analysis or synthesis. This section discusses three of the most common combinations, the series and parallel connection of systems as well as feedback loops. The latter is very important in control engineering.

### Concatenation

When two linear time-invariant (LTI) systems are combined in series by connecting the output of the first system to the input of a second system this is termed as *concatenation* of two systems. Denoting the impulse responses of the two systems by $h_1[k]$ and $h_2[k]$, the output signal $y[k]$ of the second system is given as

\begin{equation}
y[k] =  x[k] * h_1[k] * h_2[k]
\end{equation}

where $x[k]$ denotes the input signal of the first system. Applying a $z$-transform to the left- and right-hand side, and repeated application of the convolution theorem yields

\begin{equation}
Y(z) = \underbrace{H_1(z) \cdot H_2(z)}_{H(z)} \cdot X(z)
\end{equation}

It can be concluded that the concatenation of two systems can be regarded as one LTI system with the transfer function $H(z) = H_1(z) \cdot H_2(z)$. Hence, the following structures are equivalent

The extension to a concatenation of $N$ systems is straightforward. The overall transfer function is given by multiplication of all the individual transfer functions $H_n(z)$

\begin{equation}
H(z) = \prod_{n=1}^{N} H_n(z)
\end{equation}

Applications of concatenated systems include for instance the modeling of electroacoustic systems, wireless transmission systems and cascaded filters.

**Example - Concatenation of second-order sections**

Concatenation of LTI systems can be used to construct higher-order filters from lower-order prototypes. Such filters are known as *cascaded filters*. In digital signal processing, typically second-order systems are used as building blocks for higher-order systems. These blocks are termed second-order sections or [biquad filters](https://en.wikipedia.org/wiki/Digital_biquad_filter).

This is illustrated at the before introduced [second-order recursive LTI system](difference_equation.ipynb#Second-Order-System) with transfer function

\begin{equation}
H_0(z) = \frac{\frac{1}{2}}{1 - z^{-1} + \frac{1}{2} z^{-2}}
\end{equation}

Note, the transfer function has been normalized for unit gain at $z = e^{j \Omega} \vert_{\Omega = 0}$.
Concatenation of $N$ second-order filters leads to a filter with order $2 N$. Its transfer function reads

\begin{equation}
H_N(z) = \left(\frac{\frac{1}{2}}{1 - z^{-1} + \frac{1}{2} z^{-2}} \right)^N
\end{equation}

The resulting transfer function is illustrated by its logarithmic magnitude response for a varying number of cascaded filters. First the transfer function $H_N(s)$ is defined

```python
import sympy as sym
sym.init_printing()
%matplotlib inline

z = sym.symbols('z', complex=True)
W = sym.symbols('Omega', real=True)
N = sym.symbols('N', integer=True)

H0 = sym.Rational(1, 2) / (1 - z**(-1) + sym.Rational(1, 2)*z**(-2))
HN = H0**N
HN
```

The magnitude $|H_N(e^{j \Omega})|$ of the transfer function is shown for $N = \{1, 2, 3\}$ (red, green, blue line)

```python
HNa = 20*sym.log(sym.Abs(HN.subs(z, sym.exp(sym.I*W))))

p1 = sym.plot(HNa.subs(N, 1), (W, -sym.pi, sym.pi), xlabel='$\Omega$', ylabel='$| H_n(e^{j \Omega}) |$ in dB', line_color='r', show=False);
p2 = sym.plot(HNa.subs(N, 2), (W, -sym.pi, sym.pi), xlabel='$\Omega$', ylabel='$| H_n(e^{j \Omega}) |$ in dB', line_color='g', show=False);
p3 = sym.plot(HNa.subs(N, 3), (W, -sym.pi, sym.pi), xlabel='$\Omega$', ylabel='$| H_n(e^{j \Omega}) |$ in dB', line_color='b', show=False);
p1.extend(p2)
p1.extend(p3)
p1.show()
```

**Exercise**

* Compute the magnitude $|H_N(z)|$ and phase $\varphi(z)$ of the concatenated system.
* Using the result from the first exercise, how will the phase of the cascaded filter develop for an increasing number $N$ of cascaded filters?

### Parallel Connection

A structure where two LTI systems share the same input signal and their output signals are superimposed is called *parallel connection*. The overall output signal $y[k]$ is given as the superposition of the output signals of the individual systems

\begin{equation}
y[k] = h_1[k] * x[k] + h_2[k] * x[k]
\end{equation}

Applying a $z$-transform to the left- and right-hand side, exploiting the superposition principle, and convolution theorem yields

\begin{equation}
Y(z) = \underbrace{\left( H_1(z) + H_2(z)  \right)}_{H(z)} \cdot X(z)
\end{equation}

The overall transfer function $H(z)$ of a parallel connection of two systems is given as the superposition of the transfer functions of the individual systems. Hence, the following structures are equivalent

The extension to a parallel connection of $N$ systems is straightforward. The overall transfer function is given by superposition of all individual transfer functions $H_n(z)$

\begin{equation}
H(z) = \sum_{n=1}^{N} H_n(z)
\end{equation}

A prominent application of a parallel connection of systems are [filter banks](https://en.wikipedia.org/wiki/Filter_bank) as used in signal analysis and many lossy coding schemes.

### Feedback

The connection of two LTI systems, where the input of the second system is connected to the output of the first and the output of the second system is superimposed to the input of the first is called *feedback loop*. This structure is depicted in the following illustration (upper block diagram)

The output signal $y[k]$ is given as

\begin{equation}
y[k] = x[k] * h_1[k] + y[k] * h_2[k] * h_1[k]
\end{equation}

Applying a $z$-transform to the left- and right-hand side, exploiting the superposition principle and the convolution theorem, and rearrangement of terms yields

\begin{equation}
Y(z) = \frac{H_1(z)}{1 - H_1(z) \cdot H_2(z)} \cdot X(z)
\end{equation}

The overall transfer function $H(z)$ of the feedback loop is then given as

\begin{equation}
H(z) = \frac{H_1(z)}{1 - H_1(z) \cdot H_2(z)}
\end{equation}

This equivalence is depicted by the lower block diagram of above structure. Applications of feedback loops include [digital control systems](https://en.wikipedia.org/wiki/Control_system).

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


```python

```

```python
from lcapy import *
%matplotlib inline
```

```python
from matplotlib import pyplot as plt
from rfcircuit import RFCircuit
```

# RF SET series

```python

```

```python
mycirc = RFCircuit()
```

```python
mycirc.set_circuit(L('L') + (R('R') | C('C')))
```

```python
mycirc.draw_circuit()
```

```python
mycirc.args
```

    {'L': L, 'w': w, 'R': R, 'C': C}

```python
mycirc.Z
```

```python
mycirc.Gamma
```

```python
mycirc.wres
```

```python
mycirc.get_Reff(mycirc.wres[2])
```

    <function _lambdifygenerated(L, C, R)>

```python
mycirc.Reff
```

```python

```

```python
args={mycirc.args['C']:0.53e-12, mycirc.args['L']:390e-9}
```

```python
G=mycirc.get_Gamma(args)
```

```python
G
```

    <function _lambdifygenerated(w, R)>

```python
r=np.array([14.7e3, 100e3, 3e3])
```

```python
f=np.linspace(250e6,450e6,1000)
```

```python
data=G(2*np.pi*f[:,np.newaxis], r)
```

```python
fig, ax = plt.subplots()
ax.plot(f*1e-6, np.absolute(data))
axx=ax.twinx()
axx.plot(f*1e-6, np.angle(data))
ax.set_xlabel('$f$ (MHz)')
ax.set_ylabel('$\Gamma$')
axx.set_ylabel('$\phi$ (rad)')
```

```python

```

```python

```

```python

```

```python

```

# reflectometry parallel

```python
mycirc = RFCircuit()
```

```python
mycirc.set_circuit(C('Cc') + ( R('Rd') | C('C0') | L('L')))
```

```python
mycirc.draw_circuit()
```

```python
mycirc.draw_circuit(filename='parallel.svg')
```

```python
mycirc.args
```

    {'w': w, 'L': L, 'R_d': R_d, 'C_c': C_c, 'C_0': C_0}

```python
mycirc.Z
```

```python
from sympy.functions import re, im
re(mycirc.Z).simplify()
```

```python
im(mycirc.Z).simplify()
```

```python
mycirc.wres
```

```python
mycirc.get_Reff(mycirc.wres[1])
```

    <function _lambdifygenerated(R_d, C_0, C_c, L)>

```python
mycirc.Reff
```

```python
args={mycirc.args['L']:405e-9, mycirc.args['C_0']:480e-15, mycirc.args['R_d']:800e3}
gamma=mycirc.get_Gamma(args)
```

```python
gamma
```

    <function _lambdifygenerated(C_c, w)>

```python
plt.figure()
f=np.linspace(300e6, 370e6, 1000)
plt.plot(f*1e-6, np.absolute(gamma(77e-15, 2*np.pi*f)))
plt.xlabel('$|\Gamma$|')
plt.ylabel('$f$ (MHz)')
```

```python

```

```python

```

## dG vs Cc
* this mostly shows how I can match using Cc, separate of R and C0

```python
f=np.linspace(200e6, 400e6, 1000)
cc=np.linspace(1, 500, 501)*1e-15
```

```python
data=gamma(cc[:,np.newaxis], 2*np.pi*f)
matching=np.min(np.absolute(data), axis=1)
index=np.argmin(np.absolute(data), axis=1)

```

```python
data.shape
```

```python
plt.figure()
plt.plot(f*1e-6, np.absolute(gamma(77e-15, 2*np.pi*f)))
plt.ylabel('$|\Gamma|$')
plt.xlabel('$f$ (MHz)')
plt.plot(f*1e-6, np.absolute(data)[::50,:].transpose())

```

```python

```

```python
res=mycirc.get_wres(mycirc.wres[1], args) # need to choose correct resonance solution
```

```python
plt.figure()
plt.xlabel('$C_c$ (pF)')
plt.ylabel('$f_{res}$ (MHz)')
plt.plot(cc*1e12,res(cc)/2/np.pi*1e-6) # gives same results as finding minimum
```

```python

```

```python

```

```python
plt.figure()
plt.xlabel('$C_c$ (pF)')
plt.ylabel('$\Gamma_{min}$')
plt.plot(cc*1e12, np.absolute(Z(cc, res(cc)))) # matches Fig4b
```

```python
dG=mycirc.get_DeltaGamma(mycirc.args['C_0'], args)
```

```python
dG
```

    <function _lambdifygenerated(C_c, w)>

```python
plt.figure()
plt.xlabel('$C_c$ (pF)')
plt.ylabel('$\partial\Gamma/\partial C_0$')
plt.plot(cc, np.absolute(dG(cc, res(cc))))  # matches Fig4b
```

```python

```

```python

```

## dG vs Cc and C0

```python
c0=np.linspace(0.1, 0.9, 100)*1e-12
cc=np.linspace(1, 500, 200)*1e-15
```

```python
args={mycirc.args['L']:405e-9, mycirc.args['R_d']:800e3}
res=mycirc.get_wres(mycirc.wres[1], args)
dG=mycirc.get_DeltaGamma(mycirc.args['C_0'], args)
```

```python
res
```

    <function _lambdifygenerated(C_c, C_0)>

```python
res(c0, cc[:,np.newaxis]).shape
```

```python
dG
```

    <function _lambdifygenerated(C_c, w, C_0)>

```python
data=dG(cc[:,np.newaxis], res(c0, cc[:,np.newaxis]), c0)
```

```python
data.shape
```

```python

```

```python
plt.figure()
plt.xlabel('$C_0$ (pF)')
plt.ylabel('max $\partial\Gamma/\partial C_0$')
plt.plot(c0*1e12, np.max(np.absolute(data), axis=0)) # looks same as inset in Fig4d
```

```python
c0[[12, 47, 86]]*1e12 # select some values of C0
```

    array([ 0.1969697 ,  0.47979798,  0.79494949])

```python
plt.figure()
plt.xlabel('$C_c$ (pF)')
plt.ylabel('$\partial\Gamma/\partial C_0$')
plt.plot(cc*1e12, np.absolute(data)[:,[12, 47, 86]])
```

```python

```

```python

```

```python

```

## dG vs Cc and Rd

```python
rd=np.linspace(0.01, 2, 100)*1e6
cc=np.linspace(1, 500, 200)*1e-15
```

```python
args={mycirc.args['L']:405e-9, mycirc.args['C_0']:480e-15}
res=mycirc.get_wres(mycirc.wres[1], args)
dG=mycirc.get_DeltaGamma(mycirc.args['C_0'], args)
```

```python
res
```

    <function _lambdifygenerated(C_c, R_d)>

```python
res(cc[:,np.newaxis], rd).shape
```

```python
dG
```

    <function _lambdifygenerated(R_d, C_c, w)>

```python
data=dG(rd, cc[:,np.newaxis], res(cc[:,np.newaxis], rd))
```

```python
data.shape
```

```python

```

```python
plt.figure()
plt.xlabel('$R_d$ (MOhms)')
plt.ylabel('max $\partial\Gamma/\partial C_0$')
plt.plot(rd*1e-6, np.max(np.absolute(data), axis=0))
```

```python

```

```python
rd[[10, 40, 99]]*1e-6 # select values sof Rd
```

    array([ 0.2110101,  0.8140404,  2.       ])

```python
plt.figure()
plt.xlabel('$C_c$ (pF)')
plt.ylabel('$\partial\Gamma/\partial C_0$')
plt.plot(cc*1e12, np.absolute(data)[:,[10, 40, 99]])
```


# **Basic Python & Jupyter**

## **Load YT Video**

```python
#load yt video
from IPython.display import YouTubeVideo
YouTubeVideo("HW29067qVWk",560,315,rel=0)
```

## **Interactive Widget**

```python
from ipywidgets import *
import ipywidgets as widgets
import numpy as np

def f(x):
    return x

interact(f, x=10);
```

    interactive(children=(IntSlider(value=10, description='x', max=30, min=-10), Output()), _dom_classes=('widget-…

```python
def say_something(x):
    """
    Print the current widget value in short sentence
    """
    print(f'Widget says: {x}')

widgets.interact(say_something, x=[0, 1, 2, 3])
widgets.interact(say_something, x=(0, 10, 1))
widgets.interact(say_something, x=(0, 10, .5))
_ = widgets.interact(say_something, x=True)
```

    interactive(children=(Dropdown(description='x', options=(0, 1, 2, 3), value=0), Output()), _dom_classes=('widg…

    interactive(children=(IntSlider(value=5, description='x', max=10), Output()), _dom_classes=('widget-interact',…

    interactive(children=(FloatSlider(value=5.0, description='x', max=10.0, step=0.5), Output()), _dom_classes=('w…

    interactive(children=(Checkbox(value=True, description='x'), Output()), _dom_classes=('widget-interact',))

## **Plot**

```python
%matplotlib inline
import matplotlib.pyplot as plt

x = np.linspace(0,10)
y = np.sin(x)
z = np.cos(x)

plt.plot(x,y,'b',x,z,'r')
plt.xlabel('Radians');
plt.ylabel('Value');
plt.title('Plotting Demonstration')
plt.legend(['Sin','Cos'])
plt.grid()
```

## **Interactive Plot**

```python
x = np.linspace(0, 2 * np.pi)

def update(w = 1.0):
    fig = plt.figure()
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(x, np.sin(w * x))
    fig.canvas.draw()

interact(update);
```

    interactive(children=(FloatSlider(value=1.0, description='w', max=3.0, min=-1.0), Output()), _dom_classes=('wi…

## **Symbolic on Python**

```python
import sympy as sym

sym.var('P V n R T');

# Gas constant
R = 8.314        # J/K/gmol
R = R * 1000     # J/K/kgmol

# Moles of air
mAir = 1         # kg
mwAir = 28.97    # kg/kg-mol
n = mAir/mwAir   # kg-mol

# Temperature
T = 298

# Equation
eqn = sym.Eq(P*V,n*R*T)

# Solve for P
f = sym.solve(eqn,P)
print(f[0])

# Use the sympy plot function to plot
sym.plot(f[0],(V,1,10),xlabel='Volume m**3',ylabel='Pressure Pa')
```

```python

```


# The biharmonic equation on the Torus

The biharmonic equation is given as

$$
\nabla^4 u = f,
$$

where $u$ is the solution and $f$ is a function. In this notebook we will solve this equation  inside a torus with homogeneous boundary conditions $u(r=1)=u'(r=1)=0$ on the outer surface. We solve the equation with the spectral Galerkin method in curvilinear coordinates.

The torus is parametrized by

\begin{align*}
x(r, \theta, \phi) &= (R + r \cos \theta) \cos \phi \\
y(r, \theta, \phi) &= (R + r \cos \theta) \sin \phi \\
z(r, \theta, \phi) &= r \sin \theta
\end{align*}

where the Cartesian domain is $\Omega = \{(x, y, z): \left(\sqrt{x^2+y^2} - R^2\right)^2 + z^2  < 1\}$ and the computational domain is $(r, \theta, \phi) \in D = [0, 1] \times [0, 2\pi] \times [0, 2\pi]$. Hence $\theta$ and $\phi$ are angles which make a full circle, so that their values start and end at the same point, $R$ is the distance from the center of the tube to the center of the torus,
$r$ is the radius of the tube. Note that $\theta$ is the angle in the small circle (around its center), whereas $\phi$ is the angle of the large circle, around origo.

We start the implementation by importing necessary functionality from shenfun and sympy and then defining the coordinates of the surface of the torus.

```python
from shenfun import *
from shenfun.la import SolverGeneric2ND
import sympy as sp
from IPython.display import Math

N = 50
R = 3
r, theta, phi = psi = sp.symbols('x,y,z', real=True, positive=True)
rv = ((R + r*sp.cos(theta))*sp.cos(phi), (R + r*sp.cos(theta))*sp.sin(phi), r*sp.sin(theta))

B0 = FunctionSpace(N, 'L', bc='UpperDirichlet', domain=(0, 1))
B1 = FunctionSpace(N, 'F', dtype='D', domain=(0, 2*np.pi))
B2 = FunctionSpace(N, 'F', dtype='d', domain=(0, 2*np.pi))
T = TensorProductSpace(comm, (B0, B1, B2), coordinates=(psi, rv, sp.Q.positive(r*sp.cos(theta)+R)))
T.coors.sg
```

We use Fourier basis functions for the two periodic directions, and a Legendre basis that satisfies the homogeneous boundary conditions for the radial direction.

Note that `rv` represents the position vector $\vec{r}=x\mathbf{i} + y\mathbf{j} + z\mathbf{k}$ and that `T.hi` now contains the 3 scaling factors for the torus coordinates:

\begin{align*}
h_r &= \left|\frac{\partial \vec{r}}{\partial r}\right| = 1\\
h_{\theta} &= \left|\frac{\partial \vec{r}}{\partial \theta}\right| = r\\
h_{\phi} &= \left|\frac{\partial \vec{r}}{\partial \phi}\right| = r\cos \theta + R\\
\end{align*}

The covariant basis vectors used by shenfun are

```python
Math(T.coors.latex_basis_vectors(covariant=True, symbol_names={r: 'r', theta: '\\theta', phi: '\\phi'}))
```

Now check what the biharmonic operator looks like for the torus. Simplify equation using the integral measure $\sqrt{g}$, found as `T.coors.sg`

$$
\sqrt{g} = r (r \cos \theta + R)
$$

```python
u = TrialFunction(T)
v = TestFunction(T)
```

```python
du = div(grad(div(grad(u))))
g = sp.Symbol('g', real=True, positive=True)
replace = [(r*sp.cos(theta)+R, sp.sqrt(g)/r), (2*r*sp.cos(theta)+R, 2*sp.sqrt(g)/r-R)] # to simplify the look
Math((du*T.coors.sg**4).tolatex(symbol_names={r: 'r', theta: '\\theta', phi: '\\phi'}, replace=replace))
```

Glad you're not doing this by hand?

To solve this equation we need to get a variational form that is separable. To get a variational form we multiply the equation by a weight $\omega$ and the complex conjugate of a test function $\overline{v}$, and integrate over the domain by switching to computational coordinates

\begin{align*}
\int_{\Omega} \nabla^4 u\, \overline{v} \omega dV &= \int_{\Omega} f \, \overline{v} \omega dV \\
\int_{D} \nabla^4 u \, \overline{v} \omega \sqrt{g} dr d\theta d\phi &= \int_{D} f \, \overline{v} \omega \sqrt{g} dr d\theta d\phi
\end{align*}

<div class="alert alert-warning">
    Note that the functions in the last equation now actually are transformed to computational space, i.e., $u=u(\mathbf{x}(r, \theta, \phi))$ and the same for the rest. We should probably use a new name for the transformed functions, but we keep the same here to keep it simple. Whether the function is transformed or not should be evident from context. If the integral is in computational space, then the functions are transformed.
</div>

For Legendre and Fourier test functions the weight $\omega$ is normally a constant. However, we see that the denominator in some terms above contains $g^2=r^4(r\cos \theta +R)^4$. The term in parenthesis $(r\cos \theta +R)$ makes the variational form above unseparable. If, on the other hand, we change the weight $\omega$ to $g^{3/2}$, then the misbehaving denominator disappears and the variational form becomes separable.

$$
\int_{D} \nabla^4 u \, \overline{v} \, g^2 dr d\theta d\phi = \int_{D} f \, \overline{v} \, g^2 dr d\theta d\phi
$$

Alternatively, we can aim at only removing the $(r\cos \theta +R)$ term from the denominator by using weight $(r\cos \theta +R)^3$

$$
\int_{D} \nabla^4 u \, \overline{v} \, r (r\cos \theta +R)^4 dr d\theta d\phi = \int_{D} f \, \overline{v} \, r \,(r\cos \theta + R)^4 dr d\theta d\phi.
$$

The first actually leads to a coefficient matrix of fewer bands than the second. However, the condition number is larger and round-off errors more severe. In the code below we use the first approach by default, but it can be easily changed with the commented out code.

We verify the implementation by using a manufactured solution that satisfies the boundary conditions. Note that the Legendre basis `ShenBiPolar0Basis`, chosen using `bc='BiPolar0'` with the `Basis`, currently is the only function space in shenfun that can satisfy $u(r=1, \theta, \phi)=u'(r=1, \theta, \phi)=u'(0, \theta, \phi)=0$, where the latter is a pole condition inherited from [Shen's paper on cylindrical coordinates](https://epubs.siam.org/doi/abs/10.1137/S1064827595295301). With the current weights the pole condition is probably not needed.

```python
ue = sp.sin(theta*2)*sp.cos(4*phi)*(1-r)*r*sp.sin(sp.pi*r)+B0.sympy_basis(4, x=r)
f = div(grad(div(grad(u)))).tosympy(basis=ue, psi=psi)
fj = Array(T, buffer=f*T.sg**3)
#fj = Array(T, buffer=f*T.hi[2]**3)
f_hat = Function(T)
f_hat = inner(v, fj, output_array=f_hat)
#M = inner(v, div(grad(div(grad(u))))*T.hi[2]**3)
M = inner(v, div(grad(div(grad(u))))*T.sg**3)
#M = inner(div(grad(v*T.sg**3)), -div(grad(u)))

u_hat = Function(T)
sol = SolverGeneric2ND(M)
u_hat = sol(f_hat, u_hat)
uj = u_hat.backward()
uq = Array(T, buffer=ue)
print('Error =', np.linalg.norm(uj-uq))
```

Note that the variational form contains

```python
len(M)
```

```python
for i in range(len(M)):
    print(len(M[i].mats[0].keys()), len(M[i].mats[1].keys()))
```

tensorproduct matrices, but some differ only in the scales. The solver `SolverGeneric2ND` loops over and solves for one Fourier coefficient in the $\phi$-direction at the time, because all submatrices in the $\phi$-direction are diagonal. The matrices in the $\theta$-direction are not necessarily diagonal because of the weight $(r\cos \theta + 3)$. The sparsity pattern of the matrix can be inspected as follows

```python
import matplotlib.pyplot as plt
%matplotlib notebook
B = sol.diags(4, 'csr')
#print(np.linalg.cond(B.toarray()))
plt.spy(B, markersize=0.1)
plt.show()
```

A baded matrix in deed, but quite a large number of bands.

A slice of the solution can be visualized. Here we use $\phi=0$, because that lies in the Cartesian $x-z$-plane.

```python
M[0].mats[0]
```

```python
u_hat2 = u_hat.refine([N*3, N*3, N])
ur = u_hat2.backward()
us = ur.get((slice(None), slice(None), 0))
xx, yy, zz = u_hat2.function_space().local_cartesian_mesh(uniform=True)

# Wrap periodic plot around since it looks nicer
xp = np.hstack([xx[:, :, 0], xx[:, 0, 0][:, None]])
zp = np.hstack([zz[:, :, 0], zz[:, 0, 0][:, None]])
up = np.hstack([us, us[:, 0][:, None]])

# plot
plt.figure()
plt.contourf(xp, zp, up)
plt.colorbar()

```

Now print the solution at approximately half the radius

```python
from mayavi import mlab
u_hat3 = u_hat.refine([N, N*3, N*3])
ux = u_hat3.backward()
X = u_hat3.function_space().local_mesh(broadcast=True, uniform=True)
print('radius =',X[0][N//2,0,0])
```

Get the $\theta-\phi$ mesh for given radius

```python
xj = []
for rv in T.coors.coordinates[1]:
    xj.append(sp.lambdify(psi, rv)(X[0][N//2], X[1][N//2], X[2][N//2]))
xx, yy, zz = xj
us = ux[N//2]
```

Wrap around periodic direction to make it nicer

```python
xx = np.hstack([xx, xx[:, 0][:, None]])
yy = np.hstack([yy, yy[:, 0][:, None]])
zz = np.hstack([zz, zz[:, 0][:, None]])
us = np.hstack([us, us[:, 0][:, None]])
xx = np.vstack([xx, xx[0]])
yy = np.vstack([yy, yy[0]])
zz = np.vstack([zz, zz[0]])
us = np.vstack([us, us[0]])
mlab.figure(bgcolor=(1, 1, 1))
mlab.mesh(xx, yy, zz, scalars=us, colormap='jet')
mlab.show()
```

## Vector Laplacian

Finally, we look at the vector Laplacian and verify the following

$$
\nabla^2 \vec{u} = \nabla \nabla \cdot \vec{u} - \nabla \times \nabla \times \vec{u}
$$

The vector Laplace $\nabla^2 \vec{u}$ looks like:

```python
V = VectorSpace(T)
p = TrialFunction(V)
du = div(grad(p))
replace = [(r*sp.cos(theta)+R, sp.sqrt(g)/r), (2*r*sp.cos(theta)+R, 2*sp.sqrt(g)/r-R)] # to simplify the look
Math(du.tolatex(symbol_names={r: 'r', theta: '\\theta', phi: '\\phi'}, replace=replace))
```

And if we subtract $\nabla \nabla \cdot \vec{u} - \nabla \times \nabla \times \vec{u}$ we should get the zero vector.

```python
dv = grad(div(p)) - curl(curl(p))
dw = du-dv
dw.simplify()
Math(dw.tolatex(symbol_names={r: 'r', theta: '\\theta', phi: '\\phi'}))
```


# The $z$-Transform

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Theorems

The theorems of the $z$-transform relate basic operations applied to discrete signals to their equivalents in the spectral domain. They are of use for the computation of $z$-transforms of signals composed from modified [standard signals](../continuous_signals/standard_signals.ipynb) and for the computation of the response of systems to an input signal. The theorems allow further to predict the consequences of modifying a signal or system by certain operations.

### Convolution Theorem

The convolution theorem states that the $z$-transform of the linear convolution of two discrete signals $x[k]$ and $y[k]$ is equal to the scalar multiplication of their $z$-transforms $X(z)$ and $Y(z)$

\begin{equation}
\mathcal{Z} \{ x[k] * y[k] \} = X(z) \cdot Y(z)
\end{equation}

under the assumption that both transforms $X(z) = \mathcal{Z} \{ x[k] \}$ and $Y(z) = \mathcal{Z} \{ y[k] \}$ exist, respectively. The ROC of the linear convolution $x(t) * y(t)$ includes at least the intersection of the ROCs of $x[k]$ and $y[k]$

\begin{equation}
\text{ROC} \{ x[k] * y[k] \} \supseteq \text{ROC} \{ x[k] \} \cap \text{ROC} \{ y[k] \}
\end{equation}

The theorem can be proven by introducing the [definition of the linear convolution](../discrete_systems/linear_convolution.ipynb) into the [definition of the $z$-transform](definition.ipynb) and changing the order of summation

\begin{align}
\mathcal{Z} \{ x[k] * y[k] \} &= \sum_{k = -\infty}^{\infty} \left( \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot y[k - \kappa] \right) z^{-k} \\
&= \sum_{\kappa = -\infty}^{\infty} \left( \sum_{k = -\infty}^{\infty} y[k - \kappa] \, z^{-k} \right) x[\kappa] \\
&= Y(z) \cdot \sum_{\kappa = -\infty}^{\infty} x[\kappa] \, z^{-\kappa} \\
&= Y(z) \cdot X(z)
\end{align}

The convolution theorem is very useful in the context of linear time-invariant (LTI) systems. The output signal $y[k]$ of an LTI system is given as the convolution of the input signal $x[k]$ with its impulse response $h[k]$. The signals and their relation can be represented alternatively in the spectral domain using the convolution theorem. This leads to the following two equivalent representations of an LTI system

Calculation of the system response by transforming the problem into the spectral domain can be beneficial since this replaces the evaluation of the linear convolution by a scalar multiplication. In many cases this procedure simplifies the calculation of the system response significantly. The convolution theorem can also be useful to derive a $z$-transform. The key is here to express the signal as convolution of two other signals for which the $z$-transforms are known. This is illustrated by the following example.

**Example**

The [linear convolution of two rectangular signals](../discrete_systems/linear_convolution.ipynb#Finite-Length-Signals) $\text{rect}_N[k] * \text{rect}_M[k]$ results in a finite-length signal of trapezoidal shape. The $z$-transform of this signal is derived by application of the convolution theorem together with the [transform of the rectangular signal](definition.ipynb#Transformation-of-the-Rectangular-Signal). This results in

\begin{equation}
\mathcal{Z} \{ \text{rect}_N[k] * \text{rect}_M[k] \}
= \sum_{k=0}^{N-1} z^{-k} \cdot \sum_{k=0}^{M-1} z^{-k}
= \begin{cases}
\frac{1 - z^{-N}}{1 - z^{-1}} \cdot \frac{1 - z^{-M}}{1 - z^{-1}} & \text{for } z \neq 1 \\
N \cdot M & \text{for } z = 1
\end{cases}
\end{equation}

The ROC of the rectangular signal is $z \in \mathbb{C} \setminus \{ 0 \}$. The ROC of the convolution of two rectangular signals is also $z \in \mathbb{C} \setminus \{ 0 \}$, since in general no poles and zeros cancel out. The poles and zeros of $\mathcal{Z} \{ \text{rect}_N[k] * \text{rect}_M[k] \}$ are computed and illustrated in the following.

```python
import sympy as sym
from pole_zero_plot import pole_zero_plot
%matplotlib inline
sym.init_printing()

N = 5
M = 7

z = sym.symbols('z', complex=True)
k = sym.symbols('k', integer=True)

X = sym.summation(z**(-k), (k,0,N-1)) * sym.summation(z**(-k), (k,0,M-1))

poles = sym.roots(sym.denom(X) * z, z)
zeros = sym.roots(sym.numer(X) * z**(N+M-2), z)

pole_zero_plot(poles, zeros)
```

**Exercise**

* Determine the locations of the zeros of the $z$-transform $\mathcal{Z} \{ \text{rect}_N[k] \}$ of a rectangular signal.
* How are the locations of the zeros from above example related to these?

### Shift Theorem

The [shift of a signal](../discrete_signals/operations.ipynb#Shift) $x(k - \kappa)$ for $\kappa \in \mathbb{Z}$ can be expressed by the convolution of the signal $x[k]$ with a shifted Dirac impulse

\begin{equation}
x[k - \kappa] = x[k] * \delta[k - \kappa]
\end{equation}

This follows from the sifting property of the Dirac impulse. Applying a two-sided $z$-transform to the left- and right-hand side and exploiting the convolution theorem yields

\begin{equation}
\mathcal{Z} \{ x[k - \kappa] \} = X(z) \cdot z^{- \kappa}
\end{equation}

where $X(z) = \mathcal{Z} \{ x[k] \}$ is assumed to exist. Note that $\mathcal{Z} \{ \delta[k - \kappa] \} = z^{- \kappa}$ can be derived from the definition of the two-sided $z$-transform together with the sifting property of the Dirac impulse. The $z$-transform of a shifted signal is given by multiplying the $z$-transform of the original signal with $z^{-\kappa}$. The ROC of $x[k-\kappa]$ is given as the ROC of $x[k]$, whereby the points $0$ and $\infty$ have to be regarded separately due to the term $z^{- \kappa}$ in its $z$-transform

\begin{equation}
\text{ROC} \{ x[k - \kappa] \} \supseteq \text{ROC} \{ x[k] \} \setminus \{0, \infty \}
\end{equation}

This result is known as shift theorem of the $z$-transform. For a causal signal $x[k]$ and $\kappa > 0$ the shift theorem of the one-sided $z$-transform is equal to the shift theorem of the two-sided transform.

**Example**

Let's consider a system which delays a signal by a given number of samples. Its output signal $y[k] = \mathcal{H} \{ x[k] \}$ reads

\begin{equation}
y[k] = x[k-\kappa]
\end{equation}

where $\kappa \in \mathbb{N}$. Applying the two-sided $z$-transform to the left- and right-hand side yields

\begin{equation}
Y(z) = z^{-\kappa} \cdot X(z)
\end{equation}

where $Y(z) = \mathcal{Z} \{ y[k] \}$ and $X(z) = \mathcal{Z} \{ y[k] \}$. The transfer function $H(z)$ of the system can be deduced from the spectral representation of the system output $Y(z) = H(z) \cdot X(z)$ as

\begin{equation}
H(z) = z^{- \kappa}
\end{equation}

Delaying a signal is a frequently applied operation in discrete signal processing. Block diagrams illustrating signal processing algorithms frequently contain blocks that constitute delays. These blocks are commonly annotated by the corresponding transfer function $z^{- \kappa}$.

**Exercise**

* Draw a block diagramm illustrating the following difference equation $y[k] = x[k] + 2 x[k-1] + x[k-3]$.

### Modulation Theorem

The complex modulation of a signal $x[k]$ is defined as $z_0^{k} \cdot x[k]$ with $z_0 \in \mathbb{C} \setminus \{ 0 \}$. The $z$-transform of a modulated signal is derived by introducing it into the definition of the two-sided $z$-transform

\begin{equation}
\mathcal{Z} \left\{ z_0^k \cdot x[k] \right\} =
\sum_{k = -\infty}^{\infty} x[k] z_0^k \, z^{-k} =
\sum_{k = -\infty}^{\infty} x[k] \, \left( \frac{z}{z_0} \right)^{-k}
= X \left( \frac{z}{z_0} \right)
\end{equation}

where $X(z) = \mathcal{Z} \{ x[k] \}$. Modulation of the signal $x[k]$ leads to a complex scaling of the $z$-plane by the factor $z_0$. The complex scaling of $z$ can be expressed as

\begin{equation}
\frac{z}{z_0} = \frac{|z|}{|z_0|} \; e^{j (\arg(z) - \arg(z_0))}
\end{equation}

where $\arg(z)$ and $\arg(z_0)$ denote the argument (phase) of $z$ and $z_0$, respectively. It follows from this representation that the complex scaling by $z_0$ can be interpreted as a scaling and rotation of the $z$-plane.

The ROC is also complex scaled

\begin{equation}
\text{ROC} \{ z_0^k \cdot x[k] \} = \{z: \frac{z}{z_0} \in \text{ROC} \{ x[k] \} \}
\end{equation}

This relation is known as modulation theorem.

**Example**

The $z$-transform of an harmonic exponential signal $e^{j \Omega_0 k}$ of finite duration $N$ is derived the following. The signal can be expressed as $e^{j \Omega_0 k} \cdot \text{rect}_N[k]$ which allows to apply the modulation theorem with $z_0 = e^{j \Omega_0}$. Together with the [z-Transform of the rectangular signal](definition.ipynb#Transformation-of-the-Rectangular-Signal) this yields

\begin{equation}
\mathcal{Z} \{ e^{j \Omega_0 k} \cdot \text{rect}_N[k] \} = \begin{cases}
\frac{1 - \left( \frac{z}{e^{j \Omega_0}} \right)^{-N}}{1 - \left( \frac{z}{e^{j \Omega_0}} \right)^{-1}} & \text{for } z \neq e^{j \Omega_0} \\
N & \text{for } z = e^{j \Omega_0}
\end{cases}
\end{equation}

for $z \in \mathbb{C} \setminus \{ 0 \}$.

### Reversal

The two-sided $z$-transform of the reversed signal $x[-k]$ of $x[k]$ is given as

\begin{equation}
\mathcal{Z} \{ x[-k] \} = \sum_{k=-\infty}^{\infty} x[-k] \, z^{-k} = \sum_{k=-\infty}^{\infty} x[k] \, z^k =
\sum_{k=-\infty}^{\infty} x[k] \, \left( \frac{1}{z} \right)^{-k} = X \left( \frac{1}{z} \right)
\end{equation}

where $X(z) = \mathcal{Z} \{ x[k] \}$ denotes the $z$-transform of $x[k]$. The revesal of a discrete signal leads to an inversion of the complex frequency $z$ in its $z$-transform. The ROC is given as

\begin{equation}
\text{ROC} \{ x[-k] \} = \{z: \frac{1}{z} \in \text{ROC} \{ x[k] \} \}
\end{equation}

**Example**

The $z$-transform of the anticausal complex exponential signal $x[k] = - z_0^k \cdot \epsilon[k - 1]$ is derived by applying a series of transformations to the complex exponential signal $z_0[k] \cdot \epsilon[k]$. Reversal of the complex exponential signal yields

\begin{equation}
\mathcal{Z} \{ z_o^{-k} \cdot \epsilon[-k] \} = \frac{z^{-1}}{z^{-1} - z_0}
\end{equation}

Now the shift theorem is applied to the shifted signal

\begin{equation}
\mathcal{Z} \{ z_o^{-k-1} \cdot \epsilon[-k-1] \} = z \cdot \frac{z^{-1}}{z^{-1} - z_0}
\end{equation}

The factor $z_0^{-1}$ is a constant with respect to the $z$-transform. This yields

\begin{equation}
\mathcal{Z} \{ z_o^{-k} \cdot \epsilon[-k-1] \} = z_0 z \cdot \frac{z^{-1}}{z^{-1} - z_0}
\end{equation}

By noting that $z_o^{-k} = \left( \frac{1}{z_o} \right)^k$ we get

\begin{equation}
\mathcal{Z} \{ \left( \frac{1}{z_o} \right)^{k} \cdot \epsilon[-k-1] \} = z_0^{-1} z \cdot \frac{z^{-1}}{z^{-1} - z_0^{-1}} = \frac{z}{z_0 - z}
\end{equation}

From this, the $z$-transform of the anticausal complex exponential signal follows as

\begin{equation}
\mathcal{Z} \{ - z_0^{k} \cdot \epsilon[-k-1] \} =  \frac{z}{z - z_0}
\end{equation}

By considering the influence of above steps on the ROC, it is derived as

\begin{equation}
|z| < |z_0|
\end{equation}

Note that the $z$-transform of the anticausal exponential signal is equal to the transform of the causal exponential signal. However, both differ with respect to their ROCs. Above results can be confirmed by numerical evaluation of the definition of the $z$-transform

```python
z0 = sym.symbols('z0', complex=True)
sym.summation(z0**k * z**(-k), (k, -sym.oo, -1))
```

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


# Discrete Choice Problem

```python
%%capture --no-stderr
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from csrank import FATEDiscreteChoiceFunction
```

    Using TensorFlow backend.
    /home/pritha/anaconda3/envs/linenv/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
      warnings.warn(msg, category=DeprecationWarning)

## The Medoid DC Problem

In the medoid problem the goal of the discrete choice algorithms for the medoid problem is to find the most central object for the given set.
This problem is inspired by solving the task of finding a good representation of the given data using the most central point of the data points

We will generate a random dataset where each instance contains 5 objects and 2 features for easy plotting.

```python
from csrank import DiscreteChoiceDatasetGenerator
seed = 123
n_train = 10000
n_test = 10000
n_features = 2
n_objects = 5
gen = DiscreteChoiceDatasetGenerator(dataset_type='medoid', random_state=seed,
                                n_train_instances=n_train,
                                n_test_instances=n_test,
                                n_objects=n_objects,
                                n_features=n_features)
```

```python
X_train, Y_train, X_test, Y_test = gen.get_single_train_test_split()
```

Let us plot a random instance. The medoid is marked as M.

```python
def get_name(d):
    if d ==0:
        return ""
    else:
        return "M"
fig, ax = plt.subplots(figsize=(5,5))
inst = np.random.choice(n_train)
ax.scatter(X_train[inst][:, 0], X_train[inst][:, 1])
for i in range(n_objects):
    ax.text(X_train[inst, i, 0]+0.01,
            X_train[inst, i, 1]+0.01,
            s=get_name(int(Y_train[inst, i])))
```

## The FATE network
The first-aggregate-then-evaluate approach learns an embedding of each object and then aggregates that into a _context_:
\begin{equation}
	\mu_{C(\vec{x})} = \frac{1}{|C(\vec{x})|} \sum_{\vec{y} \in C(\vec{x})} \phi(\vec{y})
\end{equation}
and then scores each object $\vec{x}$ using a generalized utility function $U (\vec{x}, \mu_{C(\vec{x})})$.

```python
from keras.optimizers import SGD
from csrank import FATEObjectRanker
from csrank.losses import smooth_rank_loss
fate = FATEObjectRanker(
    n_object_features=n_features,
    loss_function=smooth_rank_loss,
    optimizer=SGD(lr=1e-4, nesterov=True, momentum=0.9))
```

We will run the training for only 10 epochs to get an idea of the convergence:

```python
fate.fit(X_train, Y_train, verbose=True, epochs=10)
```

    Train on 9000 samples, validate on 1000 samples
    Epoch 1/10
    9000/9000 [==============================] - 5s 597us/step - loss: 11.3359 - zero_one_rank_loss_for_scores_ties: 2.2760 - val_loss: 11.1825 - val_zero_one_rank_loss_for_scores_ties: 1.7870
    Epoch 2/10
    9000/9000 [==============================] - 0s 55us/step - loss: 11.1318 - zero_one_rank_loss_for_scores_ties: 1.6821 - val_loss: 11.0589 - val_zero_one_rank_loss_for_scores_ties: 1.5470
    Epoch 3/10
    9000/9000 [==============================] - 1s 56us/step - loss: 11.0235 - zero_one_rank_loss_for_scores_ties: 1.5189 - val_loss: 10.9597 - val_zero_one_rank_loss_for_scores_ties: 1.4140
    Epoch 4/10
    9000/9000 [==============================] - 0s 53us/step - loss: 10.9261 - zero_one_rank_loss_for_scores_ties: 1.3969 - val_loss: 10.8706 - val_zero_one_rank_loss_for_scores_ties: 1.3390
    Epoch 5/10
    9000/9000 [==============================] - 1s 56us/step - loss: 10.8475 - zero_one_rank_loss_for_scores_ties: 1.3219 - val_loss: 10.8015 - val_zero_one_rank_loss_for_scores_ties: 1.2610
    Epoch 6/10
    9000/9000 [==============================] - 0s 54us/step - loss: 10.7881 - zero_one_rank_loss_for_scores_ties: 1.2678 - val_loss: 10.7448 - val_zero_one_rank_loss_for_scores_ties: 1.1940
    Epoch 7/10
    9000/9000 [==============================] - 1s 56us/step - loss: 10.7408 - zero_one_rank_loss_for_scores_ties: 1.2316 - val_loss: 10.6945 - val_zero_one_rank_loss_for_scores_ties: 1.1680
    Epoch 8/10
    9000/9000 [==============================] - 0s 55us/step - loss: 10.6958 - zero_one_rank_loss_for_scores_ties: 1.1852 - val_loss: 10.6414 - val_zero_one_rank_loss_for_scores_ties: 1.1430
    Epoch 9/10
    9000/9000 [==============================] - 1s 59us/step - loss: 10.6492 - zero_one_rank_loss_for_scores_ties: 1.1494 - val_loss: 10.5997 - val_zero_one_rank_loss_for_scores_ties: 1.0930
    Epoch 10/10
    9000/9000 [==============================] - 1s 56us/step - loss: 10.6093 - zero_one_rank_loss_for_scores_ties: 1.1200 - val_loss: 10.5566 - val_zero_one_rank_loss_for_scores_ties: 1.0710

```python
scores = fate.predict_scores(X_test)
```

```python
from csrank.metrics_np import categorical_accuracy_np
from keras import backend as K
categorical_accuracy_np(Y_test, scores)
```

    0.0035

Not converged yet, but let us visualize the scores it assigns to test instances:

```python
fig, ax = plt.subplots(figsize=(5,5))
inst = np.random.choice(n_test)
ax.scatter(X_test[inst][:, 0], X_test[inst][:, 1])
for i in range(n_objects):
    ax.text(X_test[inst, i, 0]+0.01,
            X_test[inst, i, 1]+0.01,
            s=int(Y_test[inst, i]))
    ax.text(X_test[inst, i, 0]+0.01,
            X_test[inst, i, 1]-0.05,
            s='{:.1f}'.format(scores[inst][i]),
            color='b')
```


# Aula 3

## Episódio 4

```python
comprimento = 0
palavra = 'BOLA'
for letra in palavra:
    print(letra)
    comprimento = comprimento + 1

print('A palavra', palavra, 'tem', comprimento, 'letras')
```

    B
    O
    L
    A
    A palavra BOLA tem 4 letras

```python
len(palavra)
```

    4

```python
for i in range(1, 4):
    print(i)
```

    1
    2
    3

```python
print(5 ** 3)
```

    125

```python
x = 1
for _ in range(3):
    x = x * 5
print(x)
```

    125

```python
palavra = 'bola'
for n, letra in enumerate(palavra, 1001):
    print(n, '->', letra)
```

    1001 -> b
    1002 -> o
    1003 -> l
    1004 -> a

```python
for treco in [('laranja', 3.00), ('melancia', 4.5), ('banana', 1.95)]:
    print(treco)
```

    ('laranja', 3.0)
    ('melancia', 4.5)
    ('banana', 1.95)

```python
for fruta, preço in [('laranja', 3.00), ('melancia', 4.5), ('banana', 1.95)]:
    print(fruta, 'R$', preço)
```

    laranja R$ 3.0
    melancia R$ 4.5
    banana R$ 1.95

```python

```

```python
with open('letras.txt', 'w') as arq:
    for n, letra in enumerate(palavra, 1001):
        arq.write(str(n) + ' -> ' + letra + '\n')
```

```python
# arq.write?
```

### Exercícios

#### Exponenciação na unha

```python
5 ** 3
```

    125

```python
resultado = 1
expoente = 3
base = 5
for _ in range(expoente):
    resultado = resultado * base
resultado
```

    125

#### Invertendo uma String na unha

```python
original = 'Newton'
invertida = ''
for letra in original:
    invertida = letra + invertida
invertida
```

    'notweN'

#### Computando o Valor de um Polinômio

Vamos aproveitar para mostrar como escrever fórmulas em **MathJax**, um **LaTeX** simplificado que vem integrado ao **Jupyter Notebook** [(documentação)](https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Working%20With%20Markdown%20Cells.html#LaTeX-equations).

$$y = ax^3 + bx^2 + cx + d$$

Já que vamos receber os coeficientes em uma lista `c`, teremos algo mais parecido com isso:

\begin{equation}
y = c_3x^3 + c_2x^2 + c_1x + c_0
\end{equation}

O exemplo abaixo em Python corresponde a esse polinômio $y = 2 + 4x + 3x^2$

```python
x = 5
coefs = [2, 4, 3, 7]
y = coefs[0] * x**0 + coefs[1] * x**1 + coefs[2] * x**2 + coefs[3] * x**3
y
```

    972

```python
x = 5
coefs = [2, 4, 3, 7]
y = 0
for expoente, coeficiente in enumerate(coefs):
    y = y + coeficiente * x ** expoente
y
```

    972

## Episódio 5

```python
ímpares = [1, 3, 5, 7]
print(ímpares)
```

    [1, 3, 5, 7]

```python
ímpares = list(range(1, 8, 2))
print(ímpares)
```

    [1, 3, 5, 7]

```python
print('primeiro elemento:', ímpares[0])
print('último elemento:', ímpares[3])
print('elemento "-2":', ímpares[-2])
```

    primeiro elemento: 1
    último elemento: 7
    elemento "-2": 5

```python
for n in ímpares:
    print(n)
```

    1
    3
    5
    7

```python
nomes = ['Curie', 'Darwing', 'Turing']  # erro no nome de Darwin
print('nomes (valor inicial):', nomes)
nomes[1] = 'Darwin'  # corrigir o erro
print('final value of names:', nomes)
```

    nomes (valor inicial): ['Curie', 'Darwing', 'Turing']
    final value of names: ['Curie', 'Darwin', 'Turing']

```python
s = 'Python'
# s[1] = 'x'  # TypeError:  'str' object does not support item assignment
```

```python
salsa = ['peppers', 'onions', 'cilantro', 'tomatoes']
minha_salsa = salsa[:]
minha_salsa[0] = 'pimenta malagueta'
minha_salsa
```

    ['pimenta malagueta', 'onions', 'cilantro', 'tomatoes']

```python
salsa
```

    ['peppers', 'onions', 'cilantro', 'tomatoes']

```python
x = [10, 20, 30, 40]
```

```python
y = x
```

```python
y
```

    [10, 20, 30, 40]

```python
x[2] = 99
```

```python
y
```

    [10, 20, 99, 40]

```python

```


```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import fsolve
from scipy.integrate import quad
from scipy.special import eval_laguerre
import itertools
import copy
from image_matrix_helper import compute_master_list, imshow_list, rgb_map, color_to_rgb, list_to_matrix
import random
import time

nb_start = time.time()
```

## Simulation for the Derangement Model

In this notebook, we simulate a thermal system of particles of various colors binding onto a grid. We have $R$ different types of particles and particle of type $j$ has $n_j$ copies in the system. For the derangement model, particles can only exist \textit{on} thhe grid. Each particle type has a collection of "correct" locations on the grid. A particle of type $j$ attaches to this correct location with a Boltzmann factor of $\delta_j$. Here we want to use simulations of this system to affirm analytical calculations of the average number of correctly bound particles as functions of temperature.

### Numerical representations of analytical work

#### System Partition Function

The partition function for the system is
\begin{equation}
Z_{\boldsymbol{n}}(\{\beta\Delta_k\})  = \int^{\infty}_{0} dx\, e^{-x} \, \prod_{k=1}^{R}\left(e^{\beta \Delta_k}-1\right)^{n_k}\, L_{n_k} \left( \frac{x}{1-e^{\beta\Delta_k}} \right).
\label{eq:znr}
\end{equation}

```python
# delta function definition
delta_func = lambda Del, T: np.exp(Del/T)
```

```python
# parameters for the model going forward
np.random.seed(42)
Dels0 = np.random.randn(10)+2 # randomly generated delta from normal distribution
Nelems0 = np.random.randint(1,10,10) # randomly generated integers
```

```python
## integrand of partition function
integrand = lambda x, deltas, ms: np.exp(-x)*np.prod([eval_laguerre(m, x/(1-delta))*(1-delta**(-1))**m for m, delta in zip(ms, deltas)])

# test function
integrand(.40, np.exp(Dels0), Nelems0)
```

    0.03200812850398603

```python
## partition function for
ZN_betalambda = lambda deltas, ms: quad(integrand, args =  (deltas, ms), a= 0,b =np.inf)[0]
error_ZN  = lambda deltas, ms: quad(integrand, args =  (deltas, ms), a= 0,b =np.inf)[1]

# test function
ZN_betalambda(np.exp(Dels0), Nelems0)
```

    189563.4375182251

#### Calculating $\langle m \rangle$ exactly

From this partitition function, we can calculate the average number of elements in their correct placement through the equation

\begin{equation}
\langle m \rangle  = \sum_{j=1}^{R} n_{j} e^{\beta \Delta_j} \frac{Z_{\boldsymbol{n}_{j}}(\{\beta\Delta_k\}) }{Z_{\boldsymbol{n}}(\{\beta\Delta_k\})},
\label{eq:ord_zn}
\end{equation}

where $\boldsymbol{n}_{j}$ is $\boldsymbol{n}$ with 1 subtracted from the $j$th component: $\boldsymbol{n}_{j}= (n_1, \ldots, n_{j}-1, \ldots, n_R)$.

```python
# function to get n_j
mj_vals = lambda j, ms: np.array([ms[k] if k != j else ms[k]-1 for k in range(len(ms))])

num = 3
# test function
print('original n vector:', Nelems0)
print(f'(n vector)_j value for j={num}:', mj_vals(num, Nelems0))
```

    original n vector: [5 1 6 9 1 3 7 4 9 3]
    (n vector)_j value for j=3: [5 1 6 8 1 3 7 4 9 3]

```python
# average of ell
avg_m_exact = lambda T, Dels, ms: np.sum([ms[j]*ZN_betalambda(delta_func(Dels, T), mj_vals(j, ms))/ZN_betalambda(delta_func(Dels, T), ms) for j in range(len(ms))])
```

```python
# testing evaluation
avg_m_exact(1.0, Dels0, Nelems0)
```

    32.24972879913478

#### Thermal limits of disorder

At what temperature, will all the particles in the system settle into their correct location? In other words, with $\langle m \rangle$ representing the number of elements in their correct location, at what temperature does the system settle into the microstate  $\langle m \rangle = N$?"

In the accompanying paper, we found that this temperature $\beta_c = 1/k_BT_c$ is defined implicitly by

$$ 1 = \sum_{j=1}^R n_j e^{-\beta_c\Delta_j}$$

We also found the necessary (but not sufficient) condition for the system to settle into the correct macrostate is

$$ k_BT_c \leq \frac{1}{\ln N} \sum_{j=1}^r \frac{n_j}{N} \Delta_j$$

```python
# implicit thermal constraint

def therm_constr(beta,Dels, Ns):

    F =  1-np.sum(Ns*np.exp(-Dels*beta))

    return F

# critical temperature
kBTcrit = lambda Dels, Ns: 1.0/fsolve(therm_constr, x0 = 0.5, args = (Dels, Ns))[0]

# Test function
kBTcrit(Dels0, Nelems0)
```

    0.561392354979486

```python
# necessary condition for temperature
kBTlimit = lambda Dels, Ns : 1.0/(np.sum(Ns)*np.log(np.sum(Ns)))*np.sum(Dels*Ns)

# Test function
kBTlimit(Dels0, Nelems0)
```

    0.6809298387017404

```python
# checking inequality
kBTcrit(Dels0, Nelems0) < kBTlimit(Dels0, Nelems0)
```

    True

#### Large Temperature Limit

We found that in the limit of large temperature the total number of correctly ordered components has the behavior

$$ \lim_{T \to \infty} \langle m\rangle  = \dfrac{ \sum_{j=1}^R n_j^2}{ \sum_{j=1}^R n_j}.$$

```python
## ell limit
m_limit = lambda Ns : np.sum(Ns*Ns)/np.sum(Ns)
```

#### Equations of Large $N$ approximation

The order parameter for this system can be approximated as
\begin{align}
\langle m\rangle & =   \sum_{j=1}^R \frac{n_j }{1- \delta_j^{-1}} \frac{\displaystyle L_{n_j-1} \left( \bar{\sigma}_{j}\right)}{\displaystyle  L_{n_j} \left( \bar{\sigma}_{j} \right)},
\label{eq:jtox}
\end{align}
where $\bar{x}$ is defined from
\begin{equation}
N- \bar{x} = \sum_{j=1}^{R} n_j \frac{\displaystyle L_{n_j-1} \left( \bar{\sigma}_{j}\right)}{\displaystyle  L_{n_j} \left( \bar{\sigma}_{j} \right)}
\label{eq:barx}
\end{equation}
with $\bar{\sigma}_{j} \equiv{\bar{x}}/(1-\delta_j); \delta_j = e^{\Delta_j/T}$ for notational simplicity

```python
# function that determines xbar
def xbar_func(x, T, Dels, Ns):

    R = len(Ns)
    deltas_ = delta_func(Dels, T)
    sigmas_ = x/(1-deltas_)

    RHS = np.sum([Ns[j]*eval_laguerre(Ns[j]-1, sigmas_[j])/eval_laguerre(Ns[j], sigmas_[j]) for j in range(R)])

    LHS = np.sum(Ns) -x

    return LHS-RHS

# function that determs ellavge for large N
def avg_m_approx(T, Dels, Ns):

    R = len(Ns)
    xbar = fsolve(xbar_func, x0 = np.sum(Ns), args = (T, Dels, Ns))
    deltas_ = delta_func(Dels, T)
    sigma_bars = xbar/(1-deltas_)

    return np.sum([Ns[j]*deltas_[j]/(deltas_[j]-1)*eval_laguerre(Ns[j]-1, sigma_bars[j])/eval_laguerre(Ns[j], sigma_bars[j]) for j in range(R)] )
```

#### Example analytical plot

```python
# temperature vals
Tvals = np.linspace(0.1, 3.0, 50)

# parameters for the integral
np.random.seed(42)
Dels0 = np.random.randn(10)+2
n_vals = np.random.randint(1,10,10) # randomly generated integers

# computed averages for each temperature
avg_m_exact_vals = np.array([avg_m_exact(T, Dels0, Nelems0) for T in Tvals])
avg_m_approx_vals = np.array([avg_m_approx(T, Dels0, Nelems0) for T in Tvals])
```

```python
## plotting order parameters
ax = plt.subplot(111)
ax.plot(Tvals, avg_m_exact_vals/np.sum(Nelems0), label = 'Exact')
ax.plot(Tvals, avg_m_approx_vals/np.sum(Nelems0), label = 'Large N')
ax.set_ylabel(r'$\langle m \rangle$', fontsize = 18)
ax.set_xlabel(r'$k_BT$', fontsize = 18, labelpad = 10.5)
ax.grid(alpha = 0.5)
ax.axvline(x = kBTcrit(Dels0, Nelems0), color = 'k', linestyle = '--')
# ax.set_ylim([0,50])
# Hide the right and top spines
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)

# Only show ticks on the left and bottom spines
ax.yaxis.set_ticks_position('left')
ax.xaxis.set_ticks_position('bottom')
```

### Metropolis Hastings simulation code

#### Microstate transitions

```python
## permutation operator
def perm(init_ls):
    Ncomp = len(init_ls)
    i1 = int(random.choice(range(Ncomp)))
    i2 = int(random.choice(range(Ncomp)))

    ## new omega vector
    fin_ls = copy.deepcopy(init_ls)
    fin_ls[i2] = init_ls[i1]
    fin_ls[i1] = init_ls[i2]

    return(fin_ls)
```

#### Function to count the number of correct elements in a bound vector

```python
## number of correctly placed elements
def m_calc(vec, mstr_vec):

    num = 0

    for k in range(len(mstr_vec)):
        if mstr_vec[k] == vec[k]:
            num += 1

    return num
```

#### Logarithm of Botlzmann factor

The logarithm of the Botlzmann factor for a microstate (i.e., the temperature normalized negative energy of the microstate) is defined as

\begin{equation}
\beta E(\boldsymbol{m}) = \sum_{i=1}^R m_i \ln \delta_i.
\label{eq:sim_en}
\end{equation}

```python
# log of boltzmann factor
def log_boltz(bound_objs, mstr_vec, deltas, name_key):

    corr_log_factor = 0
    for j in range(len(bound_objs)):
        if bound_objs[j] == mstr_vec[j]:
            elem = bound_objs[j]
            key = name_key[elem]
            corr_log_factor+=np.log(deltas[key])

    return corr_log_factor
```

#### Checking logarithm of Boltzmann factor definition

```python
# defining name key
name_key0 = dict()
key_list = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', ]
for j in range(len(key_list)):
    name_key0[key_list[j]] = j

# random energies
np.random.seed(2)
q1 = np.random.rand(10)

# sample master list
sample_master = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', ]

print('Energy for original ordering:', log_boltz(sample_master, sample_master, np.exp(-q1), name_key0))
print('Checking energy value:', -np.sum(q1))
print('Number of correctly placed elements:', m_calc(sample_master, sample_master))
print('-----')
changed_list = perm(sample_master)
print('Energy after switching first and last values:', log_boltz(changed_list, sample_master, np.exp(-q1), name_key0))
print('Checking energy value:', -np.sum(q1*(np.array(changed_list)==np.array(sample_master))))
print('Number of correctly placed elements:', m_calc(changed_list, sample_master))
```

    Energy for original ordering: -3.58801017672414
    Checking energy value: -3.58801017672414
    Number of correctly placed elements: 10
    -----
    Energy after switching first and last values: -3.58801017672414
    Checking energy value: -3.58801017672414
    Number of correctly placed elements: 10

#### Metropolis Hastings algorithm

```python
### Metropolis Monte Carlo Algorithm

## loads uniform random sampling
runif = np.random.rand

def met_gen_derang(Niter, bound_objs, mstr_vec, deltas, name_key):
    '''
    #################################################################
    # function to sample using Metropolis
    #
    # n_iter:  number of iterations
    # initial_state: initial state for the start position for our chain
    # gamma: energy cost for incorrect component
    # temp: temperature
    ##################################################################
    '''
    # Initialize trace for state values
    state_vals = [0]*(Niter+1)

    # Set initial values
    state_vals[0] = bound_objs

    # Initialize acceptance counts
    # We can use this to tune our step size
    accepted = 0

    for i in range(Niter):

        # get current permutation
        current_bound_objs = state_vals[i]

        # proposed new permutation; generated from random integer sampling
        new_bound_objs = perm(current_bound_objs)

        #sets current log-probability
        log_current_prob = log_boltz(current_bound_objs, mstr_vec, deltas, name_key)

        # Calculate posterior log-probability with proposed value
        log_proposed_prob = log_boltz(new_bound_objs, mstr_vec, deltas, name_key)

        # Log-acceptance rate
        log_alpha = log_proposed_prob- log_current_prob

        # Sample a uniform random variate
        u = runif()

        # Test proposed value
        if np.log(u) <= log_alpha:
            # Accept
            state_vals[i+1] = new_bound_objs
            log_current_prob = log_proposed_prob
            accepted += 1
        else:
            # Stay put
            state_vals[i+1] = state_vals[i]

    # return our samples and the number of accepted steps
    return state_vals, accepted
```

#### Computing microstate average from simiulations

```python
def avg_m(state_vals, mstr_vec, n_mc):

    """
    Microstate average of number of correctly bound objects
    We only consider microstates near the end of theh chain to ensure
    that the system has equilibrated
    """

    length = int(n_mc/50)

    ls = [0]*length
    ls = np.array(ls)
    for k in range(length):
        ls[k] = m_calc(state_vals[n_mc-length+k], mstr_vec)

    return(np.mean(ls))
```

#### Image grid for completely correct configuration

```python
# defining master_list
master_list =compute_master_list()
# testing plot
imshow_list(master_list, title = 'Completely Correct Configuration');
# defining Nelems
Nelems = np.zeros(8)
key_list = list(rgb_map.keys())[:-1]
name_key_ = dict()
for j in range(len(key_list)):
    name_key_[key_list[j]] = j
    Nelems[j] = master_list.count(key_list[j])
```

```python
# displaying copy-number counts of the various elements
Nelems
```

    array([ 9.,  9., 10.,  5.,  7.,  6.,  3., 51.])

#### Simulating system

```python
## Generate lf for each temperature from .03 to 2.0 in npoints steps
t0 = time.time()

# number of steps for MC algortihm
Nmc = 20000

# binding energy parameters
R = 8
Del_bar, sigma_D = 4.0, 2.0
Dels = np.random.randn(R)*sigma_D+Del_bar

random.seed(0)

# initial monomer and dimer states;
# system in microstate of all correct dimers
bound_objs_0 = random.sample(master_list, len(master_list))
mstr_vec = copy.deepcopy(master_list)

# temperature limits
Tmin = .05
Tmax = 3.05

npoints = 15 #number of temperature values
navg = 5 # number of times we run simulation at each temperature; 50 in paper
temp_vals = np.linspace(Tmin, Tmax, npoints).tolist()

# list of dimer values
sim_m_vals = [0]*npoints

# accepted list
accepted_list = [0]*npoints

# saved list for plotting
saved_list = dict()

for k in range(npoints):

    final_m_vals = [0]*navg

    for j in range(navg):

        # make copy of initial monomer and dimer states
        bound_objs_copy = copy.deepcopy(bound_objs_0)

        # defining helper functions
        deltas_ = delta_func(Dels, temp_vals[k])

        # make copy of list
        master_list_copy = copy.deepcopy(master_list)

        # metroplois generator
        bound_list, accepted = met_gen_derang(Nmc, bound_objs_copy, mstr_vec, deltas_, name_key_)

        # calculate final j value
        final_m_vals[j] = avg_m(bound_list, master_list, Nmc)

    # saving averaged values
    sim_m_vals[k] = np.mean(np.array(final_m_vals))

    if (k+1)%5 ==0 or k ==0:
        saved_list[k] = ['white' if x=='-' else x for x in bound_list[-1]]

    time_prelim = time.time()
    print("Temperature Run:",str(k+1),"; Current Time:", round(time_prelim-t0,2),"secs")

t1 = time.time()
print("Total simulation run time: %secs" % (t1-t0))
print("------")
print()
```

    Temperature Run: 1 ; Current Time: 49.38 secs
    Temperature Run: 2 ; Current Time: 96.82 secs
    Temperature Run: 3 ; Current Time: 144.21 secs
    Temperature Run: 4 ; Current Time: 187.47 secs
    Temperature Run: 5 ; Current Time: 226.98 secs
    Temperature Run: 6 ; Current Time: 263.06 secs
    Temperature Run: 7 ; Current Time: 299.26 secs
    Temperature Run: 8 ; Current Time: 335.83 secs
    Temperature Run: 9 ; Current Time: 372.2 secs
    Temperature Run: 10 ; Current Time: 405.93 secs
    Temperature Run: 11 ; Current Time: 441.39 secs
    Temperature Run: 12 ; Current Time: 473.89 secs
    Temperature Run: 13 ; Current Time: 503.44 secs
    Temperature Run: 14 ; Current Time: 533.46 secs
    Temperature Run: 15 ; Current Time: 560.87 secs
    Total simulation run time: 560.8682470321655ecs
    ------

#### Simulated image grid at various temperatures

```python
# figure parameters
rows, cols, idx = 2, 2, 0
fig, axes = plt.subplots(nrows=rows, ncols=cols, figsize=(9,9))

# list of keys for saved snapshots of image
img_key_list = list(saved_list.keys())

for i in range(rows):
    for j in range(cols):
        if idx < 4:
            axes[i, j].imshow(color_to_rgb(list_to_matrix(saved_list[img_key_list[idx]])))
            ax = plt.gca()

            # making ticks invisible
            axes[i, j].set_xticks([])
            axes[i, j].set_yticks([])

            # Minor ticks
            axes[i, j].set_xticks(np.arange(-0.5, 11, 1), minor=True)
            axes[i, j].set_yticks(np.arange(-0.5, 10, 1), minor=True)
            axes[i, j].tick_params(axis='y', colors='red')

            # labeling images
            itimes = 'i'*(1+idx) if idx<3 else 'iv'

            # Gridlines based on minor ticks
            axes[i, j].grid(which='minor', color='w', linestyle='-', linewidth=3)
            axes[i, j].set_title(fr'({itimes})  $k_BT = {round(temp_vals[img_key_list[idx]],2)}$', fontsize = 18, y = -.2)

            # making spines invisible
            axes[i, j].spines['right'].set_visible(False)
            axes[i, j].spines['top'].set_visible(False)
            axes[i, j].spines['left'].set_visible(False)
            axes[i, j].spines['bottom'].set_visible(False)

            idx +=1

# plt.savefig('derang_grid_assembly_grid_plots.png', bbox_inches='tight', format = 'png')
plt.show()
```

#### Comparing analytical and simulation results

```python
plt.figure(figsize = (7,5))
ax = plt.subplot(111)

# plot of simulation results
plt.plot(temp_vals,sim_m_vals/np.sum(Nelems),
        label = f'Simulation',
        markersize = 7.5,
       marker = 'D',
       linestyle = '')

# plot of exact analytical results
m_exact_vals = [avg_m_exact(T, Dels, Nelems) for T in Tvals]
plt.plot(Tvals, m_exact_vals/np.sum(Nelems),linestyle = '--', linewidth = 3.0, label = f'Exact')

# plot of large N analytical results
m_largeN_vals = [avg_m_approx(T, Dels, Nelems) for T in Tvals]
plt.plot(Tvals, m_largeN_vals/np.sum(Nelems),linestyle = '--', linewidth = 3.0, label = f'Large $N$')

# plot of critical temperature
plt.axvline(x = kBTcrit(Dels, Nelems), color = 'k',  linestyle = '-.')

plt.legend(loc = (.5,0.05), fontsize = 12)
# plot formatting
ax.set_xlabel(r'$k_B T$', fontsize = 18)
plt.xlim([-0.01,3.1])
plt.ylim([0,1.1])
plt.ylabel(r'$\langle m \rangle/N$', fontsize = 18)
# plt.yaxis.set_label_coords(-0.1,.5)
plt.grid(alpha = 0.45)

# Hide the right and top spines
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)

# Only show ticks on the left and bottom spines
ax.yaxis.set_ticks_position('left')
ax.xaxis.set_ticks_position('bottom')

# increase label size
ax.tick_params(axis='both', which='major', labelsize=12)
ax.tick_params(axis='both', which='minor', labelsize=12)

ax.text(kBTcrit(Dels, Nelems)-.2, 0.25, r'$k_BT_{derang}$', color='black', fontsize = 15,
        bbox=dict(facecolor='white', edgecolor='none', pad=5.0))

for i in range(4):
    ax.text(temp_vals[img_key_list[i]], sim_m_vals[img_key_list[i]]/np.sum(Nelems)+.05,'('+'i'*(1+i)+')' if i<3 else '(iv)', fontsize = 14 )

# plt.savefig(f'derang_model.png', bbox_inches='tight', format = 'png')
plt.show()
```

```python
print('Total Notebook Runtime: %.3f mins' % ((time.time()-nb_start)/60))
```

    Total Notebook Runtime: 9.734 mins


```python
import numpy as np
import kwant

import sympy
sympy.init_printing(print_builtin=False)

from semicon import models, parameters
```

# Test methods of basic BulkModel class

## Simple case

```python
hamiltonian = kwant.continuum.sympify("A_x * k_x**2 + B_x * k_y**2")
model = models.Model(hamiltonian)
```

```python
model.hamiltonian
```

```python
R = np.array([[0, 1, 0], [-1, 0, 0], [0, 0, 1]])
model.rotate(R).hamiltonian
```

## Matrix structure

```python
tmp = "alpha_{0} * k_{0} * sigma_{0}"
ham_str = " + ".join(tmp.format(s) for s in ['x', 'y', 'z'])
R = np.array([[0, 1, 0], [-1, 0, 0], [0, 0, 1]])
```

### Reference approach

```python
sx, sy, sz = sympy.symbols('sigma_x sigma_y sigma_z')

hamiltonian = kwant.continuum.sympify(
    ham_str,
    locals={'sigma_x': sx, 'sigma_y': sy, 'sigma_z': sz}

)

model = models.Model(hamiltonian)
```

```python
model.hamiltonian
```

```python
act_on = (kwant.continuum.momentum_operators, [sx, sy, sz])
model = model.rotate(R, act_on=act_on)
```

```python
model.hamiltonian
```

```python
kwant.continuum.sympify(str(model.hamiltonian))
```

$$\left[\begin{matrix}\alpha_{z} k_{z} & - i \alpha_{x} k_{y} + \alpha_{y} k_{x}\\i \alpha_{x} k_{y} + \alpha_{y} k_{x} & - \alpha_{z} k_{z}\end{matrix}\right]$$

### Wrong approach (no information about the spin)

```python
model = models.Model(kwant.continuum.sympify(ham_str))
model.rotate(R).prettify(zero_atol=1e-6, nsimplify=True).hamiltonian
```

$$\left[\begin{matrix}\alpha_{z} k_{z} & \alpha_{x} k_{y} + i \alpha_{y} k_{x}\\\alpha_{x} k_{y} - i \alpha_{y} k_{x} & - \alpha_{z} k_{z}\end{matrix}\right]$$

### Correct approach (with information about the spin)

```python
model = models.Model(kwant.continuum.sympify(ham_str), spins=1/2)
model.rotate(R).prettify(zero_atol=1e-6, nsimplify=True).hamiltonian
```

$$\left[\begin{matrix}\alpha_{z} k_{z} & - i \alpha_{x} k_{y} + \alpha_{y} k_{x}\\i \alpha_{x} k_{y} + \alpha_{y} k_{x} & - \alpha_{z} k_{z}\end{matrix}\right]$$

```python
model.spin_operators
```

    array([[[ 0. +0.j ,  0.5+0.j ],
            [ 0.5+0.j ,  0. +0.j ]],

           [[ 0. +0.j ,  0. -0.5j],
            [ 0. +0.5j,  0. +0.j ]],

           [[ 0.5+0.j ,  0. +0.j ],
            [ 0. +0.j , -0.5+0.j ]]])

# ZincBlende model

```python
model = models.ZincBlende(
    bands=('gamma_8v', 'gamma_7v'),
    components=('foreman', )
)
```

```python
model.spins
```

    [1.5, 0.5]

```python
model.spin_operators.shape
```

    (3, 6, 6)

```python
model.hamiltonian[0, 1]
```

```python
model = model.rotate(R)
```

```python
model.hamiltonian[0, 1]
```

```python
model.prettify(decimals=10).hamiltonian[0, 1]
```

```python
model.prettify(zero_atol=1e-8, nsimplify=True).hamiltonian[0, 1]
```

# Parameters

### default databank can be provided

```python
model = models.ZincBlende(
    bands=('gamma_6c', 'gamma_8v', 'gamma_7v'),
    components=('foreman', ),
    default_databank='lawaetz',
)

model.parameters('InAs')
```

    {'E_0': 0.42, 'Delta_0': 0.38, 'P': 0.9197, 'g_c': 1.9387298407111935, 'gamma_1': 2.050284378198743, 'gamma_2': -0.43985781090063014, 'gamma_3': 0.4801421890993698, 'kappa': -1.1298578109006296, 'q': 0.04, 'gamma_0': -1.0115210754829604, 'E_v': 0, 'm_0': 510998.94609999994, 'phi_0': 4135.667662, 'mu_B': 5.7883818012e-05, 'hbar': 197.3269788}

### it can be overwritten or provided later as well

```python
db = parameters.DataBank('winkler')
model.parameters('InAs', databank=db)
```

    {'E_0': 0.418, 'Delta_0': 0.38, 'P': 0.9197, 'g_c': 1.960971886891162, 'gamma_1': 2.6959795187642825, 'gamma_2': -0.5520102406178573, 'gamma_3': 0.24798975938214163, 'kappa': -1.2520102406178584, 'q': 0.39, 'gamma_0': -1.0134532295192091, 'E_v': 0, 'm_0': 510998.94609999994, 'phi_0': 4135.667662, 'mu_B': 5.7883818012e-05, 'hbar': 197.3269788}

## workaround to spurious solutions can be easily applied

```python
InAs = model.parameters('InAs')
```

```python
InAs.renormalize(new_gamma_0=1)
```

    {'E_0': 0.42, 'Delta_0': 0.38, 'P': 0.8986682778019771, 'g_c': 1.1819199311235522, 'gamma_1': 2.8469263882910028, 'gamma_2': -0.04153680585450026, 'gamma_3': 0.8784631941454997, 'kappa': -0.7315368058544998, 'q': 0.04, 'gamma_0': 0.9999999999999964, 'E_v': 0, 'm_0': 510998.94609999994, 'phi_0': 4135.667662, 'mu_B': 5.7883818012e-05, 'hbar': 197.3269788}

```python
InAs.renormalize(new_P=0.8986682778019771)
```

    {'E_0': 0.42, 'Delta_0': 0.38, 'P': 0.8986682778019771, 'g_c': 1.1819199311235522, 'gamma_1': 2.8469263882910028, 'gamma_2': -0.04153680585450026, 'gamma_3': 0.8784631941454997, 'kappa': -0.7315368058544998, 'q': 0.04, 'gamma_0': 0.9999999999999964, 'E_v': 0, 'm_0': 510998.94609999994, 'phi_0': 4135.667662, 'mu_B': 5.7883818012e-05, 'hbar': 197.3269788}


# 2021-08-23 First Day
## CSCI-5636: Numerical solution of partial differential equations (PDE)

## Instructor: Jed Brown, jed.brown@colorado.edu, ECOT 824
## Meeting: Mon, Wed, Fri at 1:30-12:20 in ECEE 283 (and virtually)

# What are PDE?

## partial derivatives

* Gradient $\mathbf g(\mathbf x) = \nabla u(\mathbf x)$
  * scalar field to vector field
* Curl $\mathbf c(\mathbf x) = \nabla\times \mathbf g(\mathbf x)$
 * vector field to vector field
* Divergence $d(\mathbf x) = \nabla\cdot \mathbf c(\mathbf x)$
  * vector field to scalar field
* time derivatives: $\frac{\partial u(t, \mathbf x)}{\partial t}$

## what do they describe?

* heat transfer
* electrostatics, electromagnetics
* solid mechanics
* fluid mechanics
* quantum mechanics
* population dynamics

Find $u$ such that
\begin{align}
-\nabla\cdot\big(\kappa(\mathbf x) \nabla u(\mathbf x)\big) &= f(\mathbf x) & \mathbf x &\in \Omega \\
u(\mathbf x) &= g(\mathbf x) & \mathbf x &\in \Gamma_D \subset \partial\Omega \\
\kappa(\mathbf x) \nabla u(\mathbf x) \cdot \mathbf{\hat n}(\mathbf x) &= h(\mathbf x) & \mathbf x &\in \Gamma_N \subset\partial\Omega
\end{align}

# What does it mean to discretize a PDE?

Find $u$ such that
\begin{align}
-\nabla\cdot\big(\kappa(\mathbf x) \nabla u(\mathbf x)\big) &= f(\mathbf x) & \mathbf x &\in \Omega \\
u(\mathbf x) &= g(\mathbf x) & \mathbf x &\in \Gamma_D \subset \partial\Omega \\
\nabla u(\mathbf x) \cdot \mathbf{\hat n}(\mathbf x) &= h(\mathbf x) & \mathbf x &\in \Gamma_N \subset\partial\Omega
\end{align}

## How is it satisfied?
* At grid points with rules to approximate derivatives
  * Finite Difference / collocation
* A weak (integral) over elements with rules to reconstruct from element averages and define fluxes
  * Finite Volume
* A weak form over with solution and "test functions" in the same space
  * Finite Element / Galerkin

## How expensive is it?

* Suppose our domain is $\Omega = (0, 1)^3$

```julia
using Plots

# a common "good" discretiation
error(h) = min(1, 10*h^2)

function cost(h)
    n = 1/h # number of points per dimension
    N = n^3 # total number of grid points in 3D
    N
end

function cost2(h)
    N = cost(h)
    N * log2(1 + N)
end
```

    cost2 (generic function with 1 method)

```julia
plot([cost, cost2], error, .01, 1, xscale=:log10, yscale=:log10)
```

# What goes in? What comes out?

* Equations
  * Conservation
  * Variational principles
  * Materials
* Geometry
  * meshing needed?
* Boundary conditions
  * essential/Dirichlet
  * natural/Neumann
  * mixed
* Initial conditions

* Discrete solutions
  * Sampled on a grid
  * Time series
* Quantities of Interest (QoI)
  * Heat flux
  * Maximum temperature
  * Maximum stress
  * Lift, drag
* Eigensolutions
  * Resonant frequencies/modes
  * Orbitals

# Who works with PDE solvers?

* Numerical analysts
* Domain scientists and engineers
* Materials scientists
* HPC specialists
* Optimizers
* Statisticians

# What skills should they have?

Breakout groups: pick one role and make a list of essential and nice-to-have skills, plus at least one question. Pick one member to report out.

* ...

* ...

# Computer Science = Runnable Abstraction Science

### In the "grinder"
* Mathematical language and principles for each major class of method
  * What is covered by theory
  * When you're venturing off-trail
  * Analytic tools to predict and debug
* Realistic cost and exploitable structure
  * Performance on modern hardware
  * Algebraic solvers and time integrators
* Write/modify stand-alone code
* Select and use (parallel) libraries

### How can you trust the solution?
* Verification: solving the problem right
* Validation: solving the right problem

### Abstractions/collaboration
* Domain scientists, engineers
* Optimizers, statisticians
* Abstractions that reduce cognitive load
* Metrics/visualization for decisions

# On programming languages

## C
Reliable and popular for libraries (PETSc, etc.). "Simple", unsafe, capable of encapsulation.

## C++
Popular with applications and some libraries. Powerful, but complicated and unsafe.

## Fortran
The OG of numerical computing. Good for "array programming", but encapsulation is hard. Unsafe depending on dialect.

## Julia
For ground-up examples in class and in activities. Capable of high performance, expressive multiple dispatch, works well in a notebook. Library ecosystem for PDEs is limited, but rapidly improving.

## Python
Good access to JIT, libraries like FEniCS. Poor native performance, but good libraries to compiled code.

## Rust
New compiled language. Good performance, encapsulation, safety, static analysis. Poor libraries (so far).

You don't need to know any of these well and you can choose the language for your project.

# I won't grade and I won't waste your time

 My mom said, "basically, your professor is asking you to be an adult". That was too flexible.

  > In other classes, I didn’t feel like I could bounce back, but in this one I did.

# What about the letter grade?

* Collect a portfolio of the work and insights you're most proud of
* Individual meetings during the last week of the semester (mid-term preview)
* We'll have a conversation and you'll propose a grade based on your portfolio
  * I'll trust you.

*I can nudge upward when students are too modest -- pretty common. In rare cases, I may adjust down.*

# What I need from you: Growth Mindset

# How will the semester look?

## Lecture periods
* Refresh, introduce, activity/group discussion, reflect
* Pointers to further resources/activities

## In-class and homework activities
* (Short) coding, experiments, presentation of results
* Use the math and programming tools of the field
* Open-ended, go further occasionally

## Learning plans
* Write, track, and revise a personal learning plan.
* Meet/chat approx weekly to check in with partner

## Projects (second half of semester)
* Dig into community software for solving PDEs
* Short presentation on how the community works
 * Key stakeholders
 * Strengths and weaknesses
* Discuss and critique

## Community contribution
* Tutorial, documentation, performance study, comparison, new features, new application
* Reflect in a short presentation

# Tools

## Git and GitHub

* GitHub CLassroom to manage repositories
* Write using notebooks and markdown
* Review using GitHub tools and nbgrader

## Zulip

* math- and code-aware team chat
* live, informal
* thread managemnet so a question in class can spin off into a deeper asynchronous conversation

## Jupyter

* RISE slides available on website
* Activities with scaffolding
* Nbgrader for writing feedback
* Works with many languages; we'll mostly use Julia
* Collaborative mode (new)

## JupyterHub: [coding.csel.io](https://coding.csel.io)
* Nothing to install, persistent storage

# A note about COVID

* Boulder County cases are high and increasing
* Vaccinated can transmit
* I have a 1yo and 4yo in childcare
* They need to be tested at any symptoms
  * 1-2 days to result

* I'd prefer to do virtual office hours until 7-day average new cases drops below 20/day
* I'll likely be home with a kid and need to teach remotely or otherwise make it up to you