# Colour Temperature & Correlated Colour Temperature

The [colour temperature](http://en.wikipedia.org/wiki/Color_temperature) noted $T_c$ is the temperature of a Planckian radiator whose radiation has the same chromaticity as that of a given stimulus. [1]

The [correlated colour temperature](http://en.wikipedia.org/wiki/Color_temperature#Correlated_color_temperature) noted $T_{cp}$ and shortened to $CCT$ is the temperature of the Planckian radiator having the chromaticity nearest the chromaticity associated with the given spectral distribution on a diagram where the (CIE 1931 2° Standard Observer based) $u^\prime, \cfrac{2}{3}v^\prime$ coordinates of the Planckian locus and the test stimulus are depicted. [2]

The *CIE Standard Illuminant A*, *CIE Standard Illuminant D65* and *CIE Illuminant E* illuminants plotted in the *CIE 1960 UCS Chromaticity Diagram*:

```python
import colour

from colour.plotting import *
```

```python
colour_style();
```

```python
with colour.utilities.suppress_warnings(python_warnings=True):
    plot_planckian_locus_in_chromaticity_diagram_CIE1960UCS(['A', 'D65', 'E']);
```

```python
# Zooming into the *Planckian Locus*.
with colour.utilities.suppress_warnings(python_warnings=True):
    plot_planckian_locus_in_chromaticity_diagram_CIE1960UCS(
        ['A', 'D65', 'E'], bounding_box=[0.15, 0.35, 0.25, 0.45]);
```

The concept of correlated colour temperature should not be used if the chromaticity of the test source differs more than $\Delta C=5\cdot10^{-2}$ from the Planckian radiator with: [2]

$$
\Delta C= \Biggl[ \Bigl(u_t^\prime-u_p^\prime\Bigr)^2+\cfrac{4}{9}\Bigl(v_t^\prime-v_p^\prime\Bigr)^2\Biggr]^{1/2}
$$

where $u_t^\prime$, $u_p^\prime$ refer to the test source, $v_t^\prime$, $v_p^\prime$ to the Planckian radiator.

[Colour](https://github.com/colour-science/colour/) implements various methods for correlated colour temperature computation $T_{cp}$ from chromaticity coordinates $xy$ or $uv$ and chomaticity coordinates $xy$ or $uv$ computation from correlated colour temperature:

* Robertson (1968) correlated colour temperature $T_{cp}$ and $D_{uv}$ computation method by interpolation between isotemperature lines.
* Ohno (2013) correlated colour temperature $T_{cp}$ and $D_{uv}$ computation method by direct approach and combined triangular and parabolic solutions.
* McCamy (1992) correlated colour temperature $T_{cp}$ cubic approximation computation method.
* Hernandez-Andres, Lee and Romero (1999) correlated colour temperature $T_{cp}$ cubic approximation computation method.
* Krystek (1985) chomaticity coordinates $uv$ polynomial approximation computation method.
* Kang et al. (2002) chomaticity coordinates $xy$ cubic approximation computation method.
* *CIE Illuminant D Series* chomaticity coordinates $xy$ computation method.

## Robertson (1968) Method

Robertson (1968) method is based on $T_{cp}$ computation by linear interpolation between two adjacent members of a defined set of 31 isotemperature lines. [3]

In the *CIE 1960 UCS chromaticity diagram* the distance $d_i$ of the chromaticity point of given source ($u_s$, $v_s$) from each of the chromaticity point ($u_i$, $v_i$) through which the $i$th isotemperature line of slope $t_i$ passes is calculated as follows: [3]

$$
\begin{equation}
d_i=\cfrac{(v_s-v_i)-t_i(u_s-u_i)}{(1+t_i^2)^{1/2}}
\end{equation}
$$

The chromaticity point ($u_s$, $v_s$) is located between the adjacent isotemperature lines $j$ and $j + 1$ if $d_j/d_{j+1} < 0$

$$
\begin{equation}
T_c=\Biggl[\cfrac{1}{T_j}+\cfrac{\theta_1}{\theta_1+\theta_2}\biggl(\cfrac{1}{T_{j+1}}-\cfrac{1}{T_j}\biggr)\Biggr]^{-1}
\end{equation}
$$

where $\theta_1$ and $\theta_2$ are respectively the angles between the isotemperature lines $T_j$ and $T_{j+1}$ and the line joining ($u_s$, $v_s$) to their intersection. Since the isotemperature lines are narrow spaced $\theta_1$ and $\theta_2$ are small enough that one can set $\theta_1/\theta_2 = \sin\theta_1/\sin\theta_2$. The above equation can then be written:

$$
\begin{equation}
T_c=\Biggl[\cfrac{1}{T_j}+\cfrac{d_j}{d_j-d_{j+1}}\biggl(\cfrac{1}{T_{j+1}}-\cfrac{1}{T_j}\biggr)\Biggr]^{-1}
\end{equation}
$$

The `colour.uv_to_CCT_Robertson1968` definition is used to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ ($d_i$):

```python
colour.temperature.uv_to_CCT_Robertson1968((0.19783451566098664, 0.31221744678060825))
```

    array([  6.50303994e+03,   3.25561654e-03])

`colour.uv_to_CCT` definition is implemented as a wrapper for various correlated colour temperature computation methods:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'Robertson 1968')
```

    array([  6.50303994e+03,   3.25561654e-03])

> Note: `'robertson1968'` is defined as a convenient alias for `'Robertson 1968'`:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'robertson1968')
```

    array([  6.50303994e+03,   3.25561654e-03])

Converting from correlated colour temperature $T_{cp}$ and distance $D_{uv}$ to chomaticity coordinates $uv$:

```python
colour.CCT_to_uv((6503.03994225557, 0.0032556165414977167), 'Robertson 1968')
```

    array([ 0.19783447,  0.31221739])

```python
colour.CCT_to_uv((6503.03994225557, 0.0032556165414977167), 'robertson1968')
```

    array([ 0.19783447,  0.31221739])

## Ohno (2013) Method

Ohno (2013) presented new practical accurate methods to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ with an error of 1 $K$ in $T_{cp}$ range from 1000 to 20,000 and $\pm$0.03 in $D_{uv}$. [4]

### Triangular Solution

The correlated colour temperature is calculated by searching the closest point on the Planckian locus on the *CIE 1960 UCS chromaticity diagram* but without the complexity of Roberston (1968) method.

A table of coordinates ($U_i$, $V_i$) of Planckian locus (Planckian ($u$, $v$) table) in the estimated range of correlated colour temperature needed is generated and then the distance $d_i$ from the chromaticity coordinates ($U_x$, $V_x$) of a test light source is calculated.

The point $i = m$ is the point where $d_i$ is the smallest in the table ensuring that the correlated colour temperature to be obtained lies between $T_{m-1}$ and $T_{m+1}$.

The previous computation is repeated $n$ times through cascade expansion in order to reduce errors.

A triangle is then formed by the chromaticity point ($U_x$, $V_x$) of the test light soure and the chromaticity points on Planckian locus at $T_{m-1}$ and $T_{m+1}$. The blackbody temperature $T_x$ for the closest point to the line between $T_{m-1}$ and $T_{m+1}$ is calculated as follows: [4]

$$
\begin{equation}
T_x=T_{m-1}+(T_{m+1}-T_{m-1})\cdot\cfrac{x}{l}
\end{equation}
$$

with

$$
\begin{equation}
\begin{aligned}
x&=\cfrac{d_{m-1}^2-d_{m+1}^2+l^2}{2l}\\
l&=\sqrt{(u_{m+1}-u_{m-1})^2+(v_{m+1}-v_{m-1})^2}
\end{aligned}
\end{equation}
$$

$D_{uv}$ is then calculated as follows:

$$
\begin{equation}
D_{uv}=(d_{m-1}^2-x^2)^{1/2}\cdot sgn(v_x-v_{T_x})
\end{equation}
$$

with

$$
\begin{equation}
\begin{aligned}
v_{T_x}&=v_{m-1}+\{v_{m+1}-v_{m-1}\}\cdot x/l\\
SIGN(z)&=1\ for\ z \geq0\ and\ SIGN(z)=-1\ for\ z <0
\end{aligned}
\end{equation}
$$

Errors due to the non linearity of the correlated colour temperature scale on ($u$, $v$) coordinates are reduced by applying the following correction:

$$
\begin{equation}
T_{x,cor}=T_x\times 0.99991
\end{equation}
$$

This correction is not needed for Planckian ($u$, $v$) table with steps of 0.25% or smaller.

### Parabolic Solution

After finding $T_{m-1}$ and $T_{m+1}$ as described in the [triangular solution](#Triangular-Solution) method above, $d_{m−1}$, $d_m$, $d_{m+1}$ are fitted to a parabolic function. The polynomial is derived from $d_{m−1}$, $d_m$, $d_{m+1}$ and $T_{m−1}$, $T_m$, $T_{m+1}$ as: [4]

$$
\begin{equation}
d(T)=aT^2+bT+c
\end{equation}
$$

where

$$
\begin{equation}
\begin{aligned}
a&\ =[T_{m-1}(d_{m+1}-d_m)+T_m(d_{m-1}-d_{m+1})+T_{m+1}(d_m-d_{m-1})]\cdot X^{-1}\\
b&\ =-[T_{m-1}^2(d_{m+1}-d_m)+T_m^2(d_{m-1}-d_{m+1})+T_{m+1}^2(d_m-d_{m-1})]\cdot X^{-1}\\
c&\ =-[d_{m-1}(T_{m+1}-T_m)\cdot T_m\cdot T_{m+1}+d_m(T_{m-1}-T_{m+1})\cdot T_{m-1}\cdot T_{m+1}+d_{m+1}(T_m-T_{m-1})\cdot T_{m-1}\cdot T_m]\cdot X^{-1}
\end{aligned}
\end{equation}
$$

with

$$
\begin{equation}
X=(T_{m+1}-T_m)(T_{m-1}-T_{m+1})(T_m-T_{m-1})
\end{equation}
$$

The correlated colour temperature $T=T_x$ is then obtained as follows:

$$
\begin{equation}
T_X=-\cfrac{b}{2a}\qquad\because d^\prime(T)=2aT_x+b=0
\end{equation}
$$

The correction factor $T_{x,cor}$ for nonlinearity is applied as described in the [triangular solution](#Triangular-Solution) method.

$D_{uv}$ is then calculated as follows:

$$
\begin{equation}
D_{uv}=SIGN(v_x-v_{T_x})\cdot(aT_{x,cor}^2+bT_{x,cor}+c)
\end{equation}
$$

with

$$
\begin{equation}
SIGN(z)=1\ for\ z \geq0\ and\ SIGN(z)=-1\ for\ z <0
\end{equation}
$$

### Combined Solution

The parabolic solution works accurately except in on or near the Planckian locus. Taking triangular solution results for $|D_{uv}| < 0.002$ and the parabolic solution results for other regions solves that problem. [4]

The `colour.uv_to_CCT_Ohno2013` definition is used to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ ($d_i$):

```python
colour.temperature.uv_to_CCT_Ohno2013((0.19783451566098664, 0.31221744678060825))
```

    array([  6.50364955e+03,   3.20563638e-03])

Precision can be changed by passing a value to the *iterations* argument:

```python
for i in range(10):
    print(colour.temperature.uv_to_CCT_Ohno2013(
        (0.19783451566098664, 0.31221744678060825), iterations=i + 1))
```

    [  1.68848187e+04   2.66789445e-03]
    [  6.79456522e+03   3.46904097e-03]
    [  6.55383806e+03   3.48674101e-03]
    [  6.50607833e+03   3.20681676e-03]
    [  6.50374738e+03   3.20565144e-03]
    [  6.50364955e+03   3.20563638e-03]
    [  6.50364731e+03   3.20563610e-03]
    [  6.50364722e+03   3.20563604e-03]
    [  6.50364721e+03   3.20563264e-03]
    [  6.50364721e+03   3.20561976e-03]

Using the `colour.uv_to_CCT` wrapper definition:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'Ohno 2013')
```

    array([  6.50364955e+03,   3.20563638e-03])

> Note: `'ohno2013'` is defined as a convenient alias for `'Ohno 2013'`:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'ohno2013')
```

    array([  6.50364955e+03,   3.20563638e-03])

Converting from correlated colour temperature $T_{cp}$ and distance $D_{uv}$ to chomaticity coordinates $uv$:

```python
colour.CCT_to_uv((6503.03994225557, 0.0032556165414977167), 'Ohno 2013')
```

    array([ 0.19779726,  0.31225121])

```python
colour.CCT_to_uv((6503.03994225557, 0.0032556165414977167), 'Ohno 2013')
```

    array([ 0.19779726,  0.31225121])

## McCamy (1992) Method

McCamy (1992) proposed an equation to compute correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ by using a chromaticity epicenter ($x_e$, $y_e$) where the isotemperature lines in some of the correlated colour temperature range converge and the inverse slope of the line $n$ that connects it to $x$, $y$. [5]

The cubic approximation equation is defined as follows: [5]

$$
\begin{equation}
T_{cp}=-449n^3+3525n^2-6823.3n+5520.33
\end{equation}
$$

where

$$
\begin{equation}
n=\cfrac{x-x_e}{y-ye}\\
x_e=0.3320\qquad y_e=0.1858
\end{equation}
$$

The `colour.xy_to_CCT_mccamy` definition is used to calculate the correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.temperature.xy_to_CCT_McCamy1992((0.31271, 0.32902))
```

    6504.3893830489724

The `colour.xy_to_CCT` definition is implemented as a wrapper for various correlated colour temperature computation methods from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'McCamy 1992')
```

    6504.3893830489724

> Note: `'mccamy1992'` is defined as a convenient alias for `'McCamy 1992'`:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'mccamy1992')
```

    6504.3893830489724

## Hernandez-Andres, Lee and Romero (1999) Method

Hernandez-Andres, Lee and Romero (1999) extended McCamy (1992) work by using a second epicenter to extend the accuracy over a wider correlated colour temperature and chromaticity coordinates range ($3000$–$10^6K$). [6]

The new extended equation to calculate the correlated colour temperature $T_{cp}$ is defined as follows: [6]

$$
\begin{equation}
T_{cp}=A_0+A_1exp(-n/t_1)+A_2exp(-n/t_2)+A_3exp(-n/t_3)
\end{equation}
$$

where

$$
\begin{equation}
n=\cfrac{x-x_e}{y-ye}\\
\end{equation}
$$

with

| Constants | $T_{cp}$ Range ($K$) $3000$-$50,000$ | $T_{cp}$ Range ($K$) $50,000$-$8\times10^5$ |
|:---------:|:------------------------------------:|:-------------------------------------------:|
| $A_0$     | $-949.86315$                         | $36284.48953$                               |
| $A_1$     | $6253.80338$                         | $0.00228$                                   |
| $t_1$     | $0.92159$                            | $0.07861$                                   |
| $A_2$     | $28.70599$                           | $5.4535\times10^{-36}$                      |
| $t_2$     | $0.20039$                            | $0.01543$                                   |
| $A_3$     | $0.00004$                            |                                             |
| $t_3$     | $0.07125$                            |                                             |
| $x_e$     | $0.3366$                             | $0.3356$                                    |
| $y_e$     | $0.1735$                             | $0.1691$                                    |

The `colour.xy_to_CCT_hernandez` definition is used to calculate the correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.temperature.xy_to_CCT_Hernandez1999((0.31271, 0.32902))
```

    6500.0421533365825

Using the `colour.xy_to_CCT` wrapper definition:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'Hernandez 1999')
```

    6500.0421533365825

> Note: `'hernandez1999'` is defined as a convenient alias for `'Hernandez 1999'`:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'hernandez1999')
```

    6500.0421533365825

## Krystek (1985) Method

Krystek (1985) proposed a polynomial approximation valid from $1000K$ to $15000K$. [7]

The *CIE UCS* colourspace chromaticity coordinates $u$, $v$ are given by the following equations: [7]

$$
\begin{equation}
\begin{aligned}
u&\ =\cfrac{0.860117757 + 1.54118254 \times 10^{-4} T + 1.28641212 \times 10^{-7} T^2}{1 + 8.42420235 \times 10^{-4} T + 7.08145163 \times 10^{-7} T^2}\\
v&\ =\cfrac{0.317398726 + 4.22806245 \times 10^{-5} T + 4.20481691 \times 10^{-8} T^2}{1 - 2.89741816 \times 10^{-5} T + 1.61456053 \times 10^{-7} T^2}
\end{aligned}
\end{equation}
$$

The `colour.CCT_to_uv_Krystek1985` definition is used to calculate the *CIE UCS* colourspace chromaticity coordinates $u$, $v$ from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_uv_Krystek1985(6504.389383048972)
```

    array([ 0.18376696,  0.30934437])

Using the `colour.CCT_to_uv` wrapper definition:

```python
colour.CCT_to_uv(6504.389383048972, 'Krystek 1985')
```

    array([ 0.18376696,  0.30934437])

## Kang et al. (2002) Method

Kang et al. (2002) proposed an advanced colour-temperature control system for HDTV applications in the range from $1667K$ to $25000K$. [8]

The *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ are given by the following equations: [8]

$$
\begin{equation}
\begin{aligned}
x&\ =\begin{cases}-0.2661239\cfrac{10^9}{T_{cp}^3}-0.2343589\cfrac{10^6}{T_{cp}^2}+0.8776956\cfrac{10^3}{T_{cp}}+0.179910 & for\ 1667K\leq T_{cp}\leq4000k\\
-3.0258469\cfrac{10^9}{T_{cp}^3}+2.1070379\cfrac{10^6}{T_{cp}^2}+0.2226347\cfrac{10^3}{T_{cp}}+0.24039 & for\ 4000K\leq T_{cp}\leq25000k\end{cases}\\
y&\ =\begin{cases}-1.1063814x^3-1.34811020x^2+2.18555832x-0.20219683 & for\ 1667K\leq T_{cp}\leq2222k\\
-0.9549476x^3-1.37418593x^2+2.09137015x-0.16748867 & for\ 2222K\leq T_{cp}\leq4000k\\
3.0817580x^3-5.8733867x^2+3.75112997x-0.37001483 & for\ 4000K\leq T_{cp}\leq25000k\end{cases}
\end{aligned}
\end{equation}
$$

The `colour.CCT_to_xy_kang` definition is used to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_xy_Kang2002(6504.389383048972)
```

    array([ 0.313426  ,  0.32359597])

The `colour.CCT_to_xy` definition is implemented as a wrapper for various *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ computation from correlated colour temperature:

```python
colour.CCT_to_xy(6504.389383048972, 'Kang 2002')
```

    array([ 0.313426  ,  0.32359597])

> Note: `'kang2002'` is defined as a convenient alias for `'Kang 2002'`:

```python
colour.CCT_to_xy(6504.389383048972, 'kang2002')
```

    array([ 0.313426  ,  0.32359597])

## *CIE Illuminant D Series* Method

Judd et al. (1964) defined the following equations to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x_D$, $y_D$ of a *CIE Illuminant D Series*: [9]

$$
\begin{equation}
\begin{aligned}
x_D&\ =\begin{cases}-4,6070\cfrac{10^9}{T_{cp}^3}+2.9678\cfrac{10^6}{T_{cp}^2}+0.09911\cfrac{10^3}{T_{cp}}+0.244063 & for\ 4000K\leq T_{cp}\leq7000k\\
-2.0064\cfrac{10^9}{T_{cp}^3}+1.9018\cfrac{10^6}{T_{cp}^2}+0.24748\cfrac{10^3}{T_{cp}}+0.237040 & for\ 7000K\leq T_{cp}\leq25000k\end{cases}\\
y_D&\ =-3.000x_D^2+0.2.870x_D-0.275
\end{aligned}
\end{equation}
$$

The `colour.CCT_to_xy_CIE_D` definition is used to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ of a *CIE Illuminant D Series* from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_xy_CIE_D(6504.389383048972)
```

    array([ 0.31270775,  0.32911283])

Using the `colour.CCT_to_xy` wrapper definition:

```python
colour.CCT_to_xy(6504.389383048972, 'CIE Illuminant D Series')
```

    array([ 0.31270775,  0.32911283])

> Note: `'daylight'` is defined as a convenient alias for `'CIE Illuminant D Series'`:

```python
colour.CCT_to_xy(6504.389383048972, 'daylight')
```

    array([ 0.31270775,  0.32911283])

## Bibliography

1. ^ CIE. (n.d.). 17-231 colour temperature [Tc]. Retrieved from http://eilv.cie.co.at/term/231
2. ^ CIE. (n.d.). 17-258 correlated colour temperature [Tcp]. Retrieved from http://eilv.cie.co.at/term/258
3. ^ Wyszecki, G., & Stiles, W. S. (2000). DISTRIBUTION TEMPERATURE, COLOR TEMPERATURE, AND CORRELATED COLOR TEMPERATURE. In Color Science: Concepts and Methods, Quantitative Data and Formulae (pp. 224–229). Wiley. ISBN:978-0471399186
4. ^ Ohno, Y. (2014). Practical Use and Calculation of CCT and Duv. LEUKOS, 10(1), 47–55. doi:10.1080/15502724.2014.839020
5. ^ Wikipedia. (n.d.). Approximation. Retrieved June 28, 2014, from http://en.wikipedia.org/wiki/Color_temperature#Approximation
6. ^ Hernández-Andrés, J., Lee, R. L., & Romero, J. (1999). Calculating correlated color temperatures across the entire gamut of daylight and skylight chromaticities. Applied Optics, 38(27), 5703–5709. doi:10.1364/AO.38.005703
7. ^ Krystek, M. (1985). An algorithm to calculate correlated colour temperature. Color Research & Application, 10(1), 38–40. doi:10.1002/col.5080100109
8. ^ Kang, B., Moon, O., Hong, C., Lee, H., Cho, B., & Kim, Y. (2002). Design of advanced color: Temperature control system for HDTV applications. Journal of the Korean …, 41(6), 865–871. Retrieved from http://cat.inist.fr/?aModele=afficheN&cpsidt=14448733
9. ^ Wyszecki, G., & Stiles, W. S. (2000). CIE Method of Calculating D-Illuminants. In *Color Science: Concepts and Methods, Quantitative Data and Formulae* (pp. 145–146). Wiley. ISBN:978-0471399186


Sascha Spors,
Professorship Signal Theory and Digital Signal Processing,
Institute of Communications Engineering (INT),
Faculty of Computer Science and Electrical Engineering (IEF),
University of Rostock,
Germany

# Tutorial Digital Signal Processing

**Pole/Zeros Plots of Window Functions**,
Winter Semester 2021/22 (Course #24505)

- lecture: https://github.com/spatialaudio/digital-signal-processing-lecture
- tutorial: https://github.com/spatialaudio/digital-signal-processing-exercises

Feel free to contact lecturer frank.schultz@uni-rostock.de

```python
import numpy as np
from matplotlib import pyplot as plt
from matplotlib.patches import Circle
from scipy.signal import tf2zpk
from scipy.signal.windows import hann, hamming, chebwin, dpss, kaiser
from numpy.fft import fft, fftshift, ifft
```

```python
def zplane_plot(z, p, k, title_str):
    """Plot pole/zero/gain plot of discrete-time, linear-time-invariant system.

    Note that the for-loop handling might be not very efficient
    for very long FIRs

    z...array of zeros in z-plane
    p...array of poles in z-zplane
    k...gain factor

    taken from own work
    URL = ('https://github.com/spatialaudio/signals-and-systems-exercises/'
           'blob/master/sig_sys_tools.py')

    currently we don't use the ax input parameter, we rather just plot
    in hope for getting an appropriate place for it from the calling function
    """
    # draw unit circle
    Nf = 2**7
    Om = np.arange(Nf) * 2*np.pi/Nf
    plt.plot(np.cos(Om), np.sin(Om), 'C7')

    try:  # TBD: check if this pole is compensated by a zero
        circle = Circle((0, 0), radius=np.max(np.abs(p)),
                        color='C7', alpha=0.15)
        plt.gcf().gca().add_artist(circle)
    except ValueError:
        print('no pole at all, ROC is whole z-plane')

    zu, zc = np.unique(z, return_counts=True)  # find and count unique zeros
    for zui, zci in zip(zu, zc):  # plot them individually
        plt.plot(np.real(zui), np.imag(zui), ms=7,
                 color='C0', marker='o', fillstyle='none')
        if zci > 1:  # if multiple zeros exist then indicate the count
            plt.text(np.real(zui), np.imag(zui), zci)

    pu, pc = np.unique(p, return_counts=True)  # find and count unique poles
    for pui, pci in zip(pu, pc):  # plot them individually
        plt.plot(np.real(pui), np.imag(pui), ms=7,
                 color='C3', marker='x')
        if pci > 1:  # if multiple poles exist then indicate the count
            plt.text(np.real(pui), np.imag(pui), pci)

    plt.text(0, +1, 'k=%f' % k)
    plt.text(0, -1, 'ROC for causal: white')
    plt.axis('square')
    # plt.axis([-2, 2, -2, 2])
    plt.xlabel(r'$\Re\{z\}$')
    plt.ylabel(r'$\Im\{z\}$')
    plt.title(title_str)
    plt.grid(True)

def freq_resp(w, label_str, Nz=2**10):
    Omega = 2*np.pi/Nz*np.arange(Nz)
    W = np.zeros(Nz)
    W[0:N] = w
    W = fftshift(fft(W))
    W /= np.max(np.abs(W))
    W = np.abs(W)
    W[W < 10**(-100/20)] = 10**(-100/20)  # avoid log10(0)
    fig = plt.figure()
    ax1 = fig.add_subplot(111)
    ax2 = ax1.twiny()
    ax1.plot(Omega/np.pi*180-180, 20*np.log10(W), label=label_str)
    ax1.set_xlabel(r'$\Omega$ / deg')
    ax1.set_ylabel(r'normalized $|W(\Omega)|$')
    ax1.set_xticks(np.angle(z)*180/np.pi)
    ax1.set_yticks(np.arange(-100, 10, 10))
    ax1.set_xlim(-180, 180)
    ax1.set_ylim(-100, 0)
    ax1.legend()
    ax1.grid(True, which='both')
    ax2.plot(Omega-np.pi, 20*np.log10(W))
    ax2.set_xlabel(r'$\Omega$ / rad')
    ax2.set_xlim(-np.pi, np.pi);
```

```python
N = 8  # length of window
```

```python
# don't change, this is needed below repeatedly:
k = np.arange(N)
a = np.zeros(N)
a[0] = 1
```

For a length $N$ window $w[k]$ with
- $w[k]=0$ if $k<0$ and $k>N-1$

we have the z-transform of the window / finite length sequence

\begin{equation}
W(z) = \sum_{k=0}^{N-1} w[k] z^{-k} = w[0] + w[1] z^{-1} + w[2] z^{-2} + ... + w[N-1] z^{-(N-1)}
\end{equation}

We can rewrite

\begin{equation}
W(z) = \frac{w[0] z^{N-1} + w[1] z^{N-2} + w[2] z^{N-3} + ... + w[N-1]}{z^{N-1}}
\end{equation}

- This rational function has $N-1$ poles in the origin, i.e. $z=0$
- Furthermore, it has $N-1$ zeros somewhere in the complex z-plane

For real valued $w[k]$ the zeros are either complex conjugate or real-valued (they lie on the real axis in the z-plane).

For the windowing process we are interested in the DTFT spectrum, which can be obtained when evaluating $W(z)$ along the unit circle, thus as

\begin{equation}
W(\Omega) = W(z)\big|_{z=\mathrm{e}^{\mathrm{j}\Omega}}
\end{equation}

Window design is about meaningful, if not to say clever, positioning of $N-1$ zeros, such that a desired trade-off
between **mainlobe width** and **sidelobe magnitude** / **sidelobe decay rate** is achieved.
Naturally, this design task has many, many solutions. Some of them are more meaningful than others, which is why they
were published as special window designs over the last decades.

For windowing we actually always aim at very low sidelobes. Thus all potential zeros of $W(z)$ should used as best as possible: putting them exactly onto the unit circle the impact is largest in the DTFT spectrum, since notches are realized. Windows that follow this design rule, are commonly termed optimum windows. The design constraints for the trade-off then tell us where exactly to put these zeros on the unit circle.

In the examples below we discuss very simple cases for rather small $N$.

**Example 1**

Although, is has worst leakage effect, the rectangular window is actually an optimum window. The design criterion is simply: provide the most narrow main lobe that is possible for $N$. The price you pay is the highest sidelobe magnitude and very slow sidelobe decay rate.

**Example 2**

This is a! symmetric Hann window (in older literature misnamed as Hanning window, don't do this anymore!). We see that two zeros are real valued and not on the unit circle. They don't shape the window spectrum in optimum manner.

Note that for didactical convenience we use a symmetric window here. For practical spectral analysis we rather should use the flag `sym=False` for scipy windows.

**Example 3**

Idea of Hamming window:
- taking the above Hann window and moving the two zeros onto the unit circle (they must be conjugate complex).
- position of zeros such that they put a notch at the two sidelobes of the Hann window left and right of the main lobe

Mainlobe shape remains, sidelobe magnitude is overall reduced compared to the Hann window.

## Rectangular Window

```python
w = np.ones(N)
[z, p, gain] = tf2zpk(w, a)
# zeros
print('zeros: angle in deg', np.angle(z)*180/np.pi)
print('zeros: abs', np.abs(z))
# zeros are equiangularly aligned on! the unit circle
```

```python
zplane_plot(z, p, gain, 'Rect')
```

```python
freq_resp(w, 'Rect')
```

## Symmetric Hann Window

```python
# a symmetric Hann window, two zeros are not used on! the unit circle
w = (1 - np.cos(2*np.pi/N*(k+1/2))) / 2
print('my hann:', w)
[z, p, gain] = tf2zpk(w, a)
```

```python
# Note that our Hann window above differs from the numpy/scipy definitions:
# our window does start and end with a non-zero coefficient, while numpy/scipy
# versions start/end with zeros effectively using two samples less for
# signal analysis/filter design
w_tmpn = np.hanning(N)  # numpy, note: 'hanning' is an old naming to be avoided
w_tmps = hann(N)  # scipy version
w_tmp = 0.5 - 0.5 * np.cos(2*np.pi/(N-1)*(k))  # manual version
print('numpy == scipy?', np.allclose(w_tmpn, w_tmps))
print('equal to manual?', np.allclose(w_tmpn, w_tmp))
print('numpy/scipy hann:', w_tmp)
```

```python
# zeros
print('zeros: angle in deg', np.angle(z)*180/np.pi)
print('zeros: abs', np.abs(z))
```

```python
zplane_plot(z, p, gain, 'Hann')
```

```python
freq_resp(w, 'Hann')
```

We have two zeros that are **not** on the unit circle. If we put them **onto** the unit circle their influence with regard the DTFT (i.e. the frequency response) can be made stronger. By that we can optimize the window, for example to attenuate a certain sidelobe. This idea is pursued by the Hamming window, which is shown next.

## Symmetric Hamming Window

```python
# a symmetric Hamming window
# we put notches into the two sidelobes 'left/right' of the main lobe
# of the above defined symmetric Hann window,
# thus here all zeros are used on! the unit circle
# which improves side lobe attenuation
w = 0.54 - 0.46 * np.cos(2*np.pi/N*(k+1/2))
print('my hamming:', w)
[z, p, gain] = tf2zpk(w, a)
```

```python
# Note that our Hamming window above differs from the numpy/scipy definitions
w_tmpn = np.hamming(N)
w_tmps = hamming(N)
w_tmp = 0.54 - 0.46 * np.cos(2*np.pi/(N-1)*(k))
print('numpy == scipy?', np.allclose(w_tmpn, w_tmps))
print('equal to manual?', np.allclose(w_tmpn, w_tmp))
print('numpy/scipy hamming:', w_tmp)
```

```python
# +-112.03298762 is approx +-(90 + 135) / 2
# this is the angle of the first side lobe
print('zeros-> angle in deg:', np.angle(z)*180/np.pi)
# so put a zero in the middle of the 'first/last' two zeros
# now all zeros are on! the unit circle, leading to an optimum design

# zeros
print('zeros: angle in deg', np.angle(z)*180/np.pi)
print('zeros: abs', np.abs(z))
# all zeros on! the unit circle (since all np.abs(z) == 1)
```

```python
zplane_plot(z, p, gain, 'Hamming')
```

```python
freq_resp(w, 'Hamming')
```

**Copyright**

The notebooks are provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebooks for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Frank Schultz, Digital Signal Processing - A Tutorial Featuring Computational Examples* with the URL https://github.com/spatialaudio/digital-signal-processing-exercises


```python
import sympy.physics.mechanics as mech
from sympy import S,Rational,pi
import sympy as sp
```

```python
l,t,m,M,g,k,l0,w,r,lam, qref = sp.symbols(r'l t m M g k l0 w \rho \lambda \theta_{ref}')
q1, q2,q3,q4 = mech.dynamicsymbols('q1:5')
q1d, q2d, q3d, q4d = mech.dynamicsymbols('q1:5', 1)

# Create and initialize the reference frame
N = mech.ReferenceFrame('N')
pointN = mech.Point('N*')
pointN.set_vel(N, 0)

# Create the points
point1 = pointN.locatenew('p_1',q1*N.x)
point2 = pointN.locatenew('p_2',q2*N.x)
point3 = point1.locatenew('p_3', l*(sp.sin(q3)*N.x-sp.cos(q3)*N.y))
point4 = point2.locatenew('p_4', l*(sp.sin(q4)*N.x-sp.cos(q4)*N.y))

# Set the points' velocities
point1.set_vel(N, point1.pos_from(pointN).dt(N))
point2.set_vel(N, point2.pos_from(pointN).dt(N))
point3.set_vel(N, point3.pos_from(pointN).dt(N))
point4.set_vel(N, point4.pos_from(pointN).dt(N))

# Create the particles
particle1 = mech.Particle('P_1',point1,M)
particle2 = mech.Particle('P_2',point2,M)
particle3 = mech.Particle('P_3',point3,m)
particle4 = mech.Particle('P_4',point4,m)

# Set the particles' potential energy
particle1.potential_energy = 0
particle2.potential_energy = S.Half*k*((point2.pos_from(pointN)-point1.pos_from(pointN)).magnitude()-l0)**2
particle3.potential_energy = particle3.mass*g*q3
particle4.potential_energy = particle4.mass*g*q4

# Define forces not coming from a potential function
escapement3 = 2*lam*sp.Piecewise((-1, sp.Abs(q3) < qref), (1, sp.Abs(q3) >= qref))
escapement4 = 2*lam*sp.Piecewise((-1, sp.Abs(q4) < qref), (1, sp.Abs(q4) >= qref))

forces=[(point1, -2*r*point1.vel(N)),(point2, -2*r*point2.vel(N)),
        (point3,escapement3*point3.vel(N).normalize()),
        (point4,escapement4*point4.vel(N).normalize())]

# Construct the Lagrangian
L = mech.Lagrangian(N, particle1,particle2,particle3,particle4)

# Create the LagrangesMethod object
LM = mech.LagrangesMethod(L, [q1, q2,q3,q4], hol_coneqs=None, forcelist=forces, frame=N)

# Form Lagranges Equations
ELeqns = LM.form_lagranges_equations()
sp.simplify(ELeqns)

# # Holonomic Constraint Equations
# f_c = Matrix([q1**2 + q2**2 - L**2])
```

$\displaystyle \left[\begin{matrix}\begin{cases} \frac{2 \lambda \left(l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right) \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) - k \left(l_{0} - \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}} \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}} + \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \left(M \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)} + 2 \rho \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} + m \left(- l \sin{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{3}}{\left(t \right)} + \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)}\right)\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}}{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} & \text{for}\: \theta_{ref} > \left|{\operatorname{q_{3}}{\left(t \right)}}\right| \\\frac{- 2 \lambda \left(l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right) \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) - k \left(l_{0} - \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}} \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}} + \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \left(M \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)} + 2 \rho \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} + m \left(- l \sin{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{3}}{\left(t \right)} + \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)}\right)\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}}{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} & \text{otherwise} \end{cases}\\\begin{cases} \frac{2 \lambda \left(l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right) \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) + k \left(l_{0} - \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}} \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}} + \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \left(M \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)} + 2 \rho \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} + m \left(- l \sin{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{4}}{\left(t \right)} + \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)}\right)\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}}{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} & \text{for}\: \theta_{ref} > \left|{\operatorname{q_{4}}{\left(t \right)}}\right| \\\frac{- 2 \lambda \left(l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right) \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) + k \left(l_{0} - \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}} \sqrt{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right)^{2}} + \left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \left(M \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)} + 2 \rho \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} + m \left(- l \sin{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{4}}{\left(t \right)} + \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)}\right)\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}}{\left(\operatorname{q_{1}}{\left(t \right)} - \operatorname{q_{2}}{\left(t \right)}\right) \sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} & \text{otherwise} \end{cases}\\\begin{cases} \frac{2 \lambda l^{2} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} + \frac{2 \lambda l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} + g m + l^{2} m \frac{d^{2}}{d t^{2}} \operatorname{q_{3}}{\left(t \right)} + l m \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)} & \text{for}\: \theta_{ref} > \left|{\operatorname{q_{3}}{\left(t \right)}}\right| \\- \frac{2 \lambda l^{2} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} - \frac{2 \lambda l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{3}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{1}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{3}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{1}}{\left(t \right)}\right)^{2}}} + g m + l^{2} m \frac{d^{2}}{d t^{2}} \operatorname{q_{3}}{\left(t \right)} + l m \cos{\left(\operatorname{q_{3}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{1}}{\left(t \right)} & \text{otherwise} \end{cases}\\\begin{cases} \frac{2 \lambda l^{2} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} + \frac{2 \lambda l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} + g m + l^{2} m \frac{d^{2}}{d t^{2}} \operatorname{q_{4}}{\left(t \right)} + l m \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)} & \text{for}\: \theta_{ref} > \left|{\operatorname{q_{4}}{\left(t \right)}}\right| \\- \frac{2 \lambda l^{2} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} - \frac{2 \lambda l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}}{\sqrt{l^{2} \left(\frac{d}{d t} \operatorname{q_{4}}{\left(t \right)}\right)^{2} + 2 l \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d}{d t} \operatorname{q_{2}}{\left(t \right)} \frac{d}{d t} \operatorname{q_{4}}{\left(t \right)} + \left(\frac{d}{d t} \operatorname{q_{2}}{\left(t \right)}\right)^{2}}} + g m + l^{2} m \frac{d^{2}}{d t^{2}} \operatorname{q_{4}}{\left(t \right)} + l m \cos{\left(\operatorname{q_{4}}{\left(t \right)} \right)} \frac{d^{2}}{d t^{2}} \operatorname{q_{2}}{\left(t \right)} & \text{otherwise} \end{cases}\end{matrix}\right]$

```python
from numpy import array, linspace, sin, cos
from pydy.system import System
import numpy as np

sys = System(LM,constants={
    m:1.0,M:10.0,l:1.0,
    l0:1.0,k: 10.0,g:9.81,
    w:1.0,r:0.2,lam:0.1,
    qref:0.1},
             initial_conditions={
                 q1:0,q2:w,q3:np.pi/4,
                 q4:np.pi/5,q1d:0,q2d:0,
                 q3d:0,q4d:0},
             times=linspace(0.0, 10.0, 1000))
y = sys.integrate()
```

```python

```


```python
import matplotlib as mpl
mpl.rcParams.update({
    'font.size' : 9.0,
    'mathtext.fontset' : 'stix',
    'font.family': 'STIXGeneral',
    'text.usetex': True
})
import matplotlib.pyplot as plt
import numpy as np
%matplotlib widget
```

```python
import sympy as sym
from sympy.abc import s,t,x,y,z
from sympy.integrals import laplace_transform
from sympy.integrals import inverse_laplace_transform

# Laplace transform (t->s)
U = laplace_transform(5*t, t, s)
print('U')
print(U[0])
# Result: 5/s**2

# Inverse Laplace transform (s->t)
X = inverse_laplace_transform(U[0],s,t)
print('X')
print(X)
# Result: 5*t*Heaviside(t)

# Function
F = 5*(s+1)/(s+3)**2
print('F')
print(F)
# Result: (5*s + 5)/(s + 3)**2

# Partial fraction decomposition
G = sym.apart(F)
print('G')
print(G)
# Result: 5/(s + 3) - 10/(s + 3)**2

# denominator of transfer function
d1 = (s+1)*(s+3)*(s**2+3*s+1)

# expand polynomial
d2 = sym.expand(d1)
print('d2')
print(d2)
# Result: s**4 + 7*s**3 + 16*s**2 + 13*s + 3

# find roots
print(sym.roots(d2))
# Result: {-1: 1, -3: 1, -3/2 - sqrt(5)/2: 1, -3/2 + sqrt(5)/2: 1}
```

    U
    5/s**2
    X
    5*t*Heaviside(t)
    F
    (5*s + 5)/(s + 3)**2
    G
    5/(s + 3) - 10/(s + 3)**2
    d2
    s**4 + 7*s**3 + 16*s**2 + 13*s + 3
    {-1: 1, -3: 1, -3/2 - sqrt(5)/2: 1, -3/2 + sqrt(5)/2: 1}

```python

```


# Gibbs sampling for a 2D multivariate normal

## Outline

* **Part 1: Defining the conditional distributions**
* **Part 2: Writing a Gibbs sampler**
* **Part 3: Sanity check results against NumPy's built-in sampler**

## Takeaways

* Gibbs samplers are only possible if we can exactly define the conditional distributions (for some models this is easy, for many others this is quite challenging)
* When possible, Gibbs samplers are almost always a better choice than Random Walk or other proposal-based MCMC methods

# Setup

```python
import numpy as np
import pandas as pd
import scipy.stats
```

```python
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style("whitegrid")
sns.set_context("notebook", font_scale=1.25)
```

# Background

## Random variables

(We'll use 0-indexing so we have close alignment between math and python code)

We are interested in a 2-dimensional random variable $z = [z_0, z_1]$

Each entry $z_d$ is a real scalar: $z_d \in \mathbb{R}$

## Target distribution

Our target distribution is a multivariate Gaussian with known zero mean and a covariance matrix that establishes positive correlation between $z_0$ and $z_1$.

\begin{align}
p^*(z_0, z_1) =  \mathcal{N}\left(
    \left[ \begin{array}{c}
    0 \\ 0
    \end{array} \right],
    \left[
    \begin{array}{c c}
    1 & 0.8 \\
    0.8 & 2
    \end{array} \right] \right)
\end{align}

Our goal is to draw SAMPLES from this distribution, using the Gibbs sampling method.

Naturally, you could draw samples from this distribution just by calling `np.random.multivariate_normal` with the right mean and covariance parameters. This is simpler (of course), and we'll use this fact to *check our Gibbs sampler*, since both procedures should generate samples from the same distribution.

One way to think about what we're doing is to say, what if you were given a software library where it was easy to sample from a "univariate" normal, but not a *multivariate* normal. The Gibbs sampler here would be one way we could still draw *joint* samples.

## Gibbs Sampling algorithm sketch

To begin, we initialize the vector $z^0$ to *any valid value in the sample space*.

Then, at each iteration $t \in 0,1 , 2, \ldots$, we iteratively sample each entry of $z$ as follows:
\begin{align}
    z^{t+1}_0 &\sim p^* (z_0 | z_1 = z^t_1) \\
    z^{t+1}_1 &\sim p^* (z_1 | z_0 = z^{t+1}_0)
\end{align}

## Things to remember

This is a simple example to illustrate the idea of how Gibbs sampling works.

There are other "better" ways of sampling from a 2d normal.

# Part 1: Prepare for Gibbs sampling

## Exercise 1a: Using the code below, complete the method to sample from p(z0 | z1)

```python
# First, we used Bishop textbook formulas to compute the conditional mean/var
# See Eq 2.96 and 2.97
# https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf#page=110

# Start by defining parameters of the given target distribution
mean_2 = np.zeros(2)
cov_22 = np.asarray([[1.0, 0.8], [0.8, 2.0]])
lam_22 = np.linalg.inv(cov_22)
```

```python
# Implement Eq 2.96: Solve for the conditional variance
# Note that this is a CONSTANT no matter what value of z1 we're conditioning on

cond_v_of_z0 = 1.0 / lam_22[0,0] # grab diag entry of lam_22 relevant to 0
print(cond_v_of_z0)
```

    0.6799999999999999

```python
# Implement Eq 2.97: Solve for the conditional mean
# Note that this is a FUNCTION of the value of z1 we're conditioning on
def calc_cond_mean_of_z0(z1):
    return mean_2[0] - cond_v_of_z0 * lam_22[0,1] * z1 + cond_v_of_z0 * lam_22[0,1] * mean_2[1]
```

```python
calc_cond_mean_of_z0(np.linspace(-1, 1, 10))
```

    array([-0.4       , -0.31111111, -0.22222222, -0.13333333, -0.04444444,
            0.04444444,  0.13333333,  0.22222222,  0.31111111,  0.4       ])

```python
def draw_z0_given_z1(z1, random_state):
    ''' Sample a value of z[0] from its conditional given z[1]

    Args
    ----
    z1 : float
        Value of random variable z[1]
    random_state : numpy RandomState object
        Random number generator

    Returns
    -------
    z0_samp : float
        Sample from the conditional p(z[0] | z[1])
    '''

    cond_mean_of_z0 = 0.0 # TODO compute the conditional mean of z[0] given z[1]
    cond_var_of_z0  = 1.0 # TODO compute the conditional variance of z[0] given z[1]

    # Draw a value u from the Standard Normal (zero mean, unit variance)
    u_samp = random_state.randn()

    # Transform this value into a sample from Normal(mean_01, var_01)
    # Remember, if u ~ Normal(0, 1), a "standard" normal with mean 0 variance 1,
    # then using transform: x <- T(u), with T(u) = \mu + \sigma * u
    # we can say x ~ Normal(\mu, \sigma^2)
    z0_samp = u_samp # TODO update this line to perform the transformation
    return z0_samp
```

## Exercise 1b: Using the code below, complete the method to sample from p(z1 | z0)

Use the same process as above! Just flip the indices so you get the mean and variance for z1 instead of z0

```python
def draw_z1_given_z0(z0, random_state):
    ''' Sample a value of z[1] from its conditional given z[0]

    Args
    ----
    z0 : float
        Value of random variable z[0]
    random_state : numpy RandomState object
        Random number generator

    Returns
    -------
    z1_samp : float
        Sample from the conditional p(z[1] | z[0])
    '''
    ## First, use Bishop textbook formulas to compute conditional mean/var
    cond_mean_of_z1 = 0.0 # TODO compute the conditional mean of z[1] given z[0]
    cond_var_of_z1  = 1.0 # TODO compute the conditional variance of z[1] given z[0]

    # Draw a value u from the Standard Normal (zero mean, unit variance)
    u_samp = random_state.randn()

    # Transform this value into a sample from Normal(mean_01, var_01)
    # Remember, if u ~ Normal(0, 1), a "standard" normal with mean 0 variance 1,
    # then using transform: x <- T(u), with T(u) = \mu + \sigma * u
    # we can say x ~ Normal(\mu, \sigma^2)
    z1_samp = u_samp # TODO update this line to perform the transformation
    return z1_samp
```

# Part 2: Execute the Gibbs sampling algorithm

## Gibbs sampler algorithm

Perform 6000 iterations.

Discard the first 1000 as "not yet burned in".

```python
S = 6000
sample_list = list()
z_D = np.zeros(2)

random_state = np.random.RandomState(0) # reproducible random seeds

for t in range(S):
    z_D[0] = draw_z0_given_z1(z_D[1], random_state)
    z_D[1] = draw_z1_given_z0(z_D[0], random_state)

    if t > 1000:
        sample_list.append(z_D.copy()) # save copies so we get different vectors
```

```python
# convert list of samples to a 2d numpy array
z_samples_SD = np.vstack(sample_list)
```

## Visualize the sampled values for z0

```python
plt.plot(z_samples_SD[:,0])
plt.xlabel('sample iteration');
plt.ylabel('$z_0$')
plt.title('Sampled values of r.v. from Gibbs Sampler');
```

## Exercise 2a: Compute the empirical MEAN of your samples

You should estimate the mean of your samples, which is effectively:

$$
\frac{1}{S} \sum_{s=1}^S z^s
$$

If your sampler is correct, and you have enough samples, you should get *very close* to the true mean of your target distribution

```python
## TODO compute the mean of z_samples_SD
```

## Exercise 2b: Compute the empirical covariance of your samples

You should estimate the covariance of your samples, which is effectively:

$$
\frac{1}{S} \sum_{s=1}^S (z^s - \bar{z} )  (z^s - \bar{z} )^T
$$

If your sampler is correct, and you have enough samples, you should get *very close* to the true covariance of your target distribution

```python
## TODO compute the covariance of z_samples_SD. Hint: use np.cov
```

# Part 3: Compare to samples from built-in routines for 2D MVNormal sampling

## Draw 1000 samples from target distribution using `np.random.multivariate_normal`

```python
true_mu_2 = np.zeros(2)
true_cov_22 = np.asarray([[1.0, 0.8], [0.8, 2.0]])
true_samples_SD = random_state.multivariate_normal(true_mu_2, true_cov_22, size=S-1000)
```

## Visualize results of your Gibbs-sampler and Numpy's built-in sampler side-by-side

Do the results visually match?

```python
fig, ax_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, figsize=(10,4))

ax_grid[0].plot(z_samples_SD[:,0], z_samples_SD[:,1], 'k.')
ax_grid[0].set_title('Gibbs sampler')
ax_grid[0].set_aspect('equal', 'box');
ax_grid[0].set_xlabel('$z_0$');
ax_grid[0].set_ylabel('$z_1$');

ax_grid[1].plot(true_samples_SD[:,0], true_samples_SD[:,1], 'k.')
ax_grid[1].set_title('np.random.multivariate_normal')
ax_grid[1].set_aspect('equal', 'box');
ax_grid[1].set_xlim([-6, 6]);
ax_grid[1].set_ylim([-6, 6]);
ax_grid[1].set_xlabel('$z_0$');
ax_grid[1].set_ylabel('$z_1$');
```

```python

```


# Laboratório 5: Câncer e Tratamento Químico

### Referente ao capítulo 10

Queremos minimizar a densidade de um tumor em um organismo e os efeitos colaterais das drogas para o tratamento de câncer por quimioterapia em um período de tempo fixo. É assumido que o tumor tenha um crescimento Gompertzian. A hipótese *log-kill* de Skipper é utilizada para modelar a ocisão de células cancerosas. Ela diz que a morte de células devido aos químicos é proporcional à população de tumor.

Considere $N(t)$ a densidade normalizada do tumor no tempo $t$. Assim:

$$N'(t) = rN(t)\ln\left(\frac{1}{N(t)}\right) - u(t)\delta N(t)$$

onde $r$ é a taxa de crescimento do tumor, $\delta$ a magnitude da dose e $u(t)$ descreve a força do efeito da droga (famacocinética).

Escolhemos o funcional da seguinte maneira para minimizar tanto a densidade de tumor quanto os efeitos da droga. Supomos aqui que quanto maior a força do efeito da droga, maior seu efeito negativo, o que é razoável.

$$\min_u \int_0^T aN(t)^2 + u(t)^2 dt$$

Além disso, $u(t) \geq 0$ e $N(0) = N_0$.

## Condições Necessárias

### Hamiltoniano

$$
H = aN^2 + u^2 + \lambda\left[rN\ln\left(\frac{1}{N}\right) - u\delta N\right]
$$

### Equação adjunta

$$
\lambda '(t) = - H_N = -2aN - \lambda\left(r\ln\left(\frac{1}{N}\right) - rN\frac{1}{1/N}\frac{1}{N^2} - u\delta\right) = -2aN + \lambda\left[r-r\ln\left(\frac{1}{N}\right) + u\delta\right]
$$

### Condição de transversalidade

$$
\lambda(T) = 0
$$

### Condição de otimalidade

$$
H_u = 2u - \lambda \delta N
$$

Nesse caso temos apenas uma desigualdade, a inferior e, portanto, $H_u < 0$ não é possível. Temos que $\delta > 0$ dada sua interpretação e $N(t) > 0$, afinal sua derivada não é nem definida caso contrário. Como o problema é de minimização, temos que lembrar que as desigualdades ficam invertidas em relação a $H_u$.

$$
H_u > 0 \implies u^*(t) = 0 \implies \lambda \delta N < 0 \implies \lambda(t) < 0
$$
$$
H_u = 0 \implies 0 \le u^*(t) = \frac{\delta}{2}\lambda(t)N(t)
$$

Assim $u^*(t) = \max\left\{0, \frac{\delta}{2}\lambda(t)N(t)\right\}$

### Importanto as bibliotecas

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
import sympy as sp

import sys
sys.path.insert(0, '../pyscripts/')

from optimal_control_class import OptimalControl
```

### Usando a biblitoca sympy

```python
x_sp,u_sp,lambda_sp, r_sp, a_sp, delta_sp = sp.symbols('N u lambda r a delta')
H = a_sp*x_sp**2 + u_sp**2 + lambda_sp*(r_sp*x_sp*sp.log(1/x_sp) - u_sp*delta_sp*x_sp)
H
```

$\displaystyle N^{2} a + \lambda \left(- N \delta u + N r \log{\left(\frac{1}{N} \right)}\right) + u^{2}$

```python
print('H_x = {}'.format(sp.diff(H,x_sp)))
print('H_u = {}'.format(sp.diff(H,u_sp)))
print('H_lambda = {}'.format(sp.diff(H,lambda_sp)))
```

    H_x = 2*N*a + lambda*(-delta*u + r*log(1/N) - r)
    H_u = -N*delta*lambda + 2*u
    H_lambda = -N*delta*u + N*r*log(1/N)

Resolvendo para $H_u = 0$

```python
eq = sp.Eq(sp.diff(H,u_sp), 0)
sp.solve(eq,u_sp)
```

    [N*delta*lambda/2]

Aqui podemos descrever as funções necessárias para a classe.

```python
parameters = {'a': None, 'delta': None, 'r': None}

diff_state = lambda t, N, u, par: -N*par['delta']*u + N*par['r']*np.log(1/N)
diff_lambda = lambda t, N, u, lambda_, par: -2*N*par['a'] + lambda_*(par['delta']*u + par['r']*(1 - np.log(1/N)))
update_u = lambda t, N, lambda_, par: np.maximum(0, 0.5*par['delta']*lambda_*N)
```

## Aplicando a classe ao exemplo

Vamos fazer algumas exeperimentações. Sinta-se livre para variar os parâmetros. Nesse caso passaremos o limite inferior como padrão inicial. Note que `np.inf` deve ser escrito quando não há limite.

```python
problem = OptimalControl(diff_state, diff_lambda, update_u, bounds = [(0,np.inf)])
```

```python
N0 = 0.975
T = 20
parameters['a'] = 3
parameters['delta'] = 0.45
parameters['r'] = 0.3
```

```python
t,x,u,lambda_ = problem.solve(N0, T, parameters)
ax = problem.plotting(t,x,u,lambda_)
```

Note o formato do controle: Inicia com valor alto e depois reduz conforme se aproxima do dia 20. Isto é consistente com as práticas médicas atuais. Todavia, também observamos que o tumor tende a crescer após o dia 12. Por isso, vamos aumentar a importância de reduzir as células cancerosas, em troca de mais danos pelos remédios. Vamos notar que a força da droga é muito maior quando reduzimos a importância de seu efeito negativo. O formato das curvas é, entretanto, muito similar.

```python
parameters['a'] = 10
t,x,u,lambda_ = problem.solve(N0, T, parameters)
ax = problem.plotting(t,x,u,lambda_)
```

Vamos comparar o controle escolhido conforme diferentes valor iniciais da densidade $N_0$.

```python
parameters['a'] = 3
N0_values = [0.1, 0.5, 0.7, 0.9]
u_values = []
for N0 in N0_values:
    _,_,u,_ = problem.solve(N0, T, parameters)
    u_values.append(u)

fig = plt.figure(figsize = (10,5))
plt.xlabel("Tempo")
plt.ylabel("Químico")
plt.title("Quimioterapia utilizada")
for i, N0 in enumerate(N0_values):
    plt.plot(t, u_values[i],label = r'$N_0$ = {}'.format(N0))
plt.legend()
plt.grid(alpha = 0.5)
```

Podemos observar acima que apenas o estágio inicial do controle é afetado pela densidade inicial, até que atinge uma espécie de equilíbrio.

Outro ponto importante é o quanto a magnitude da dose influencia na dosagem do químico ao longo do tempo. Observe que quanto maior a magnitude, o valor inicial é mais alto, porém o decaimento é mais rápido, usando menos químico no total, como era esperado.

```python
N0 = 0.8
delta_values = [0.25, 0.5, 0.75]
u_values = []
for delta in delta_values:
    parameters['delta'] = delta
    _,_,u,_ = problem.solve(N0, T, parameters)
    u_values.append(u)

fig = plt.figure(figsize = (10,5))
plt.xlabel("Tempo")
plt.ylabel("Químico")
plt.title("Quimioterapia utilizada")
for i, delta in enumerate(delta_values):
    plt.plot(t, u_values[i],label = r'$\delta$ = {}'.format(delta))
plt.legend()
plt.grid(alpha = 0.5)
```

Podemos ver também diferentes valores temporais $T$ de duração do tratamento alteram o comportamento da quantidade de células cancerosas. Na verdade, o formato fica muito similar em todos os casos, variando o tempo de estabilidade no meio do caminho.

```python
parameters['delta'] = 0.4
T_values = [10, 20, 40, 80]
x_values = []
u_values = []
t_values = []
for T in T_values:
    t,x,u,_ = problem.solve(N0, T, parameters)
    t_values.append(t)
    x_values.append(x)
    u_values.append(u)

fig, ax = plt.subplots(2,2,figsize = (10,8))
fig.suptitle(r'Comparação temporal de $N(t)$')
for k in range(4):
    i = k//2
    j = k%2
    ax[i][j].set_ylabel(r"$N(t)$")
    ax[i][j].set_title("Densisade de células cancerosas")
    ax[i][j].plot(t_values[k], x_values[k])
    ax[i][j].grid(alpha = 0.5)

fig, ax = plt.subplots(2,2,figsize = (10,8))
fig.suptitle(r'Comparação temporal de $u(t)$')
for k in range(4):
    i = k//2
    j = k%2
    ax[i][j].set_ylabel(r"$u(t)$")
    ax[i][j].set_title("Força do efeito da quimioterapia")
    ax[i][j].plot(t_values[k], u_values[k], color = 'green')
    ax[i][j].grid(alpha = 0.5)
```

Você deve ter observado em todos os exemplos que $u(t)$ atinge 0 apenas pontualmente. É necessária a restrição $u(t) \ge 0$?

## Experimentação

```python
#N0 = 1
#T = 5
#parameters['r'] = 0.3
#parameters['a'] = 10
#parameters['delta'] = 0.4
#
#t,x,u,lambda_ = problem.solve(N0, T, parameters)
#roblem.plotting(t,x,u,lambda_)
```

### Este é o final do notebook


# Quantization of Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Introduction

[Digital signal processors](https://en.wikipedia.org/wiki/Digital_signal_processor) and general purpose processors can only perform arithmetic operations within a limited number range. So far we considered discrete signals with continuous amplitude values. These cannot be handled by processors in a straightforward manner. [Quantization](https://en.wikipedia.org/wiki/Quantization_%28signal_processing%29) is the process of mapping a continuous amplitude to a countable set of amplitude values. This refers also to the *requantization* of a signal from a large set of countable amplitude values to a smaller set. Scalar quantization is an instantaneous and memoryless operation. It can be applied to the continuous amplitude signal, also referred to as *analog signal* or to the (time-)discrete signal. The quantized discrete signal is termed as *digital signal*. The connections between the different domains is illustrated in the following for time dependent signals.

### Model of the Quantization Process

In order to quantify the effects of quantizing a continuous amplitude signal, a model of the quantization process is formulated. We restrict our considerations to a discrete real-valued signal $x[k]$. In order to map the continuous amplitude to a quantized representation the following model is used

\begin{equation}
x_Q[k] = g( \; \lfloor \, f(x[k]) \, \rfloor \; )
\end{equation}

where $g(\cdot)$ and $f(\cdot)$ denote real-valued mapping functions, and $\lfloor \cdot \rfloor$ a rounding operation. The quantization process can be split into two stages

1. **Forward quantization**
    The mapping $f(x[k])$ maps the signal $x[k]$ such that it is suitable for the rounding operation. This may be a scaling of the signal or a non-linear mapping. The result of the rounding operation is an integer number $\lfloor \, f(x[k]) \, \rfloor \in \mathbb{Z}$, which is termed as *quantization index*.

2. **Inverse quantization**
    The mapping $g(\cdot)$, maps the quantization index to the quantized value $x_Q[k]$ such that it is an approximation of $x[k]$. This may be a scaling or a non-linear operation.

The quantization error/quantization noise $e[k]$ is defined as

\begin{equation}
e[k] = x_Q[k] - x[k]
\end{equation}

Rearranging yields that the quantization process can be modeled by adding the quantization noise to the discrete signal

#### Example

In order to illustrate the introduced model, the quantization of one period of a sine signal is considered

\begin{equation}
x[k] = \sin[\Omega_0 k]
\end{equation}

using $f(x[k]) = 3 \cdot x[k]$ and $g(i) = \frac{1}{3} \cdot i$. The rounding is realized by the [nearest integer function](https://en.wikipedia.org/wiki/Nearest_integer_function). The quantized signal is then given as

\begin{equation}
x_Q[k] = \frac{1}{3} \cdot \lfloor \, 3 \cdot \sin[\Omega_0 k] \, \rfloor
\end{equation}

For ease of illustration the signals are not shown by stem plots.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

N = 1024  # length of signal

# generate signal
x = np.sin(2*np.pi/N * np.arange(N))
# quantize signal
xi = np.round(3 * x)
xQ = 1/3 * xi
e = xQ - x

# plot (quantized) signals
fig, ax1 = plt.subplots(figsize=(10,4))
ax2 = ax1.twinx()

ax1.plot(x, 'r', label=r'discrete signal $x[k]$')
ax1.plot(xQ, 'b', label=r'quantized signal $x_Q[k]$')
ax1.plot(e, 'g', label=r'quantization error $e[k]$')
ax1.set_xlabel('k')
ax1.set_ylabel(r'$x[k]$, $x_Q[k]$, $e[k]$')
ax1.axis([0, N, -1.2, 1.2])
ax1.legend()

ax2.set_ylim([-3.6, 3.6])
ax2.set_ylabel('quantization index')
ax2.grid()
```

**Exercise**

* Investigate the quantization noise $e[k]$. Is its amplitude bounded?
* If you would represent the quantization index (shown on the right side) by a binary number, how much bits would you need?
* Try out other rounding operations like `np.floor()` and `np.ceil()` instead of `np.round()`. What changes?

### Properties

Without knowledge of the quantization error $e[k]$, the signal $x[k]$ cannot be reconstructed exactly knowing only its quantization index or quantized representation $x_Q[k]$. The quantization error $e[k]$ itself depends on the signal $x[k]$. Therefore, quantization is in general an irreversible process. The mapping from $x[k]$ to $x_Q[k]$ is furthermore non-linear, since the superposition principle does not hold in general. Summarizing, quantization is an inherently irreversible and non-linear process.

### Applications

Quantization has widespread applications in Digital Signal Processing. For instance in

* [Analog-to-Digital conversion](https://en.wikipedia.org/wiki/Analog-to-digital_converter)
* [Lossy compression](https://en.wikipedia.org/wiki/Lossy_compression) of signals (speech, music, video, ...)
* Storage and Transmission ([Pulse-Code Modulation](https://en.wikipedia.org/wiki/Pulse-code_modulation), ...)

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


# Realization of Recursive Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing.

## Cascaded Structures

The realization of recursive filters with a high order may be subject to numerical issues. For instance, when the coefficients span a wide amplitude range, their quantization may require a small quantization step or may impose a large relative error for small coefficients. The basic concept of cascaded structures is to decompose a high order filter into a cascade of lower order filters, typically first and second order recursive filters.

### Decomposition into Second-Order Sections

The rational transfer function $H(z)$ of a linear time-invariant (LTI) recursive system can be [expressed by its zeros and poles](introduction.ipynb#Transfer-Function) as

\begin{equation}
H(z) = \frac{b_M}{a_N} \cdot \frac{\prod_{\mu=1}^{P} (z - z_{0\mu})^{m_\mu}}{\prod_{\nu=1}^{Q} (z - z_{\infty\nu})^{n_\nu}}
\end{equation}

where $z_{0\mu}$ and $z_{\infty\nu}$ denote the $\mu$-th zero and $\nu$-th pole of degree $m_\mu$ and $n_\nu$ of $H(z)$, respectively. The total number of zeros and poles is denoted by $P$ and $Q$.

The poles and zeros of a real-valued filter $h[k] \in \mathbb{R}$ are either single real valued or conjugate complex pairs. This motivates to split the transfer function into

* first order filters constructed from a single pole and zero
* second order filters constructed from a pair of conjugated complex poles and zeros

Decomposing the transfer function into these two types by grouping the poles and zeros into single poles/zeros and conjugate complex pairs of poles/zeros results in

\begin{equation}
H(z) = K \cdot \prod_{\eta=1}^{S_1} \frac{(z - z_{0\eta})}{(z - z_{\infty\eta})}
\cdot \prod_{\eta=1}^{S_2} \frac{(z - z_{0\eta}) (z - z_{0\eta}^*)} {(z - z_{\infty\eta})(z - z_{\infty\eta}^*)}
\end{equation}

where $K$ denotes a constant and $S_1 + 2 S_2 = N$ with $N$ denoting the order of the system. The cascade of two systems results in a multiplication of their transfer functions. Above decomposition represents a cascade of first- and second-order recursive systems. The former can be treated as a special case of second-order recursive systems. The decomposition is therefore known as decomposition into second-order sections (SOSs) or [biquad filters](https://en.wikipedia.org/wiki/Digital_biquad_filter). Using a cascade of SOSs the transfer function of the recursive system can be rewritten as

\begin{equation}
H(z) = \prod_{\mu=1}^{S} \frac{b_{0, \mu} + b_{1, \mu} \, z^{-1} + b_{2, \mu} \, z^{-2}}{1 + a_{1, \mu} \, z^{-1} + a_{2, \mu} \, z^{-2}}
\end{equation}

where $S = \lceil \frac{N}{2} \rceil$ denotes the total number of SOSs. These results state that any real valued system of order $N > 2$ can be decomposed into SOSs. This has a number of benefits

* quantization effects can be reduced by sensible grouping of poles/zeros, e.g. such that the spanned amplitude range of the filter coefficients is limited
* A SOS may be extended by a gain factor to further reduce quantization effects by normalization of the coefficients
* efficient and numerically stable SOSs serve as generic building blocks for higher-order recursive filters

### Example - Cascaded second-order section realization of a lowpass

The following example illustrates the decomposition of a higher-order recursive Butterworth lowpass filter into a cascade of second-order sections.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.markers import MarkerStyle
from matplotlib.patches import Circle
import scipy.signal as sig

N = 9  # order of recursive filter

def zplane(z, p, title='Poles and Zeros'):
    "Plots zero and pole locations in the complex z-plane"
    ax = plt.gca()

    ax.plot(np.real(z), np.imag(z), 'bo', fillstyle='none', ms = 10)
    ax.plot(np.real(p), np.imag(p), 'rx', fillstyle='none', ms = 10)
    unit_circle = Circle((0,0), radius=1, fill=False,
                         color='black', ls='solid', alpha=0.9)
    ax.add_patch(unit_circle)
    ax.axvline(0, color='0.7')
    ax.axhline(0, color='0.7')

    plt.title(title)
    plt.xlabel(r'Re{$z$}')
    plt.ylabel(r'Im{$z$}')
    plt.axis('equal')
    plt.xlim((-2, 2))
    plt.ylim((-2, 2))
    plt.grid()

# design filter
b, a = sig.butter(N, 0.2)
# decomposition into SOS
sos = sig.tf2sos(b, a, pairing='nearest')

# print filter coefficients
print('Coefficients of the recursive part \n')
print(['%1.2f'%ai for ai in a])
print('\n')
print('Coefficients of the recursive part of the individual SOS \n')
print('Section \t a1 \t\t a2')
for n in range(sos.shape[0]):
    print('%d \t\t %1.5f \t %1.5f'%(n, sos[n, 4], sos[n, 5]))

# plot pole and zero locations
plt.figure(figsize=(5,5))
zplane(np.roots(b), np.roots(a), 'Poles and Zeros - Overall')

plt.figure(figsize=(10, 7))
for n in range(sos.shape[0]):
    plt.subplot(231+n)
    zplane(np.roots(sos[n, 0:3]), np.roots(sos[n, 3:6]), title='Poles and Zeros - Section %d'%n)
plt.tight_layout()

# compute and plot frequency response of sections
plt.figure(figsize=(10,5))
for n in range(sos.shape[0]):
    Om, H = sig.freqz(sos[n, 0:3], sos[n, 3:6])
    plt.plot(Om, 20*np.log10(np.abs(H)), label=r'Section %d'%n)

plt.xlabel(r'$\Omega$')
plt.ylabel(r'$|H_n(e^{j \Omega})|$ in dB')
plt.legend()
plt.grid()
```

**Exercise**

* What amplitude range is spanned by the filter coefficients?
* What amplitude range is spanned by the SOS coefficients?
* Change the pole/zero grouping strategy from `pairing='nearest'` to `pairing='keep_odd'`. What changes?
* Increase the order `N` of the filter. What changes?

Solution: Inspecting both the coefficients of the recursive part of the original filter and of the individual SOS reveals that the spanned amplitude range is lower for the latter. The choice of the pole/zero grouping strategy influences the locations of the poles/zeros in the individual SOS, the spanned amplitude range of their coefficients and the transfer functions of the individual sections. The total number of SOS scales with the order of the original filter.


# Internal stability of Runge-Kutta methods

Internal stability deals with the growth of errors (such as roundoff) introduced at the Runge-Kutta stages during a single Runge-Kutta step.  It is usually important only for methods with a large number of stages, since that is when the internal amplification factors can be large.  An excellent explanation of internal stability is given in [this paper](http://oai.cwi.nl/oai/asset/1652/1652A.pdf).  Here we demonstrate some tools for studying internal stability in NodePy.

First, let's load a couple of RK methods:

```python
from nodepy import rk
import numpy as np
import matplotlib.pyplot as plt

rk4 = rk.loadRKM('RK44')
ssprk4 = rk.loadRKM('SSP104')
print(rk4)
print(ssprk4)
```

    Classical RK4
    The original four-stage, fourth-order method of Kutta
     0   |
     1/2 | 1/2
     1/2 |      1/2
     1   |           1
    _____|____________________
         | 1/6  1/3  1/3  1/6
    SSPRK(10,4)
    The optimal ten-stage, fourth order SSP Runge-Kutta method
     0    |
     1/6  | 1/6
     1/3  | 1/6   1/6
     1/2  | 1/6   1/6   1/6
     2/3  | 1/6   1/6   1/6   1/6
     1/3  | 1/15  1/15  1/15  1/15  1/15
     1/2  | 1/15  1/15  1/15  1/15  1/15  1/6
     2/3  | 1/15  1/15  1/15  1/15  1/15  1/6   1/6
     5/6  | 1/15  1/15  1/15  1/15  1/15  1/6   1/6   1/6
     1    | 1/15  1/15  1/15  1/15  1/15  1/6   1/6   1/6   1/6
    ______|____________________________________________________________
          | 1/10  1/10  1/10  1/10  1/10  1/10  1/10  1/10  1/10  1/10
          |       2/9               5/18  1/3                     1/6

## Absolute stability regions

First we can use NodePy to plot the region of absolute stability for each method.  The absolute stability region is the set

\begin{align} \label{1}
\{ z \in C : |\phi (z)|\le 1 \}
\end{align}

where $\phi(z)$ is the *stability function* of the method:

$1 + z b^T (I-zA)^{-1}$

If we solve $u'(t) = \lambda u$ with a given method, then $z=\lambda \Delta t$ must lie inside this region or the computation will be unstable.

```python
p,q = rk4.stability_function()
print(p)
h1 = rk4.plot_stability_region()
```

```python
p,q = ssprk4.stability_function()
print(p)
h2 = ssprk4.plot_stability_region()
```

# Internal stability

The stability function tells us by how much errors from one step are amplified in the next one.  This is important since we introduce truncation errors at every step.  However, we also introduce roundoff errors at the each stage within a step.  Internal stability tells us about the growth of those.  Internal stability is typically less important than (step-by-step) absolute stability for two reasons:

 - Roundoff errors are typically much smaller than truncation errors, so moderate amplification of them typically is not significant
 - Although the propagation of stage errors within a step is governed by internal stability functions, in later steps these errors are propagated according to the (principal) stability function

Nevertheless, in methods with many stages, internal stability can play a key role.

Questions: *In the solution of PDEs, large spatial truncation errors enter at each stage.  Does this mean internal stability becomes more significant?  How does this relate to stiff accuracy analysis and order reduction?*

## Internal stability functions

We can write the equations of a Runge-Kutta method compactly as

\begin{align}
y & = u^n e + h A F(y) \\
u^{n+1} & = u^n + h b^T F(y),
\end{align}

where $y$ is the vector of stage values, $u^n$ is the previous step solution, $e$ is a vector with all entries equal to 1, $h$ is the step size, $A$ and $b$ are the coefficients in the Butcher tableau, and $F(y)$ is the vector of stage derivatives.  In floating point arithmetic, roundoff errors will be made at each stage.  Representing these errors by a vector $r$, we have

$y = u^n e + h A F(y) + r.$

Considering the test problem $F(y)=\lambda y$ and solving for $y$ gives

$y = u^n (I-zA)^{-1}e + (I-zA)^{-1}r,$

where $z=h\lambda$.  Substituting this result in the equation for $u^{n+1}$ gives

$u^{n+1} = u^n (1 + zb^T(I-zA)^{-1}e) + zb^T(I-zA)^{-1}r = \psi(z) u^n + \theta(z)^T r.$

Here $\psi(z)$ is the *stability function* of the method, that we already encountered above.  Meanwhile, the vector $\theta(z)$ contains the *internal stability functions* that govern the amplification of roundoff errors $r$ within a step:

$\theta(z) = z b^T (I-zA)^{-1}.$

Let's compute $\theta$ for the classical RK4 method:

```python
theta=rk4.internal_stability_polynomials()
theta
```

    [poly1d([1/12, 1/6, 1/3, 0], dtype=object),
     poly1d([1/6, 1/3, 0], dtype=object),
     poly1d([1/6, 0], dtype=object)]

```python
for theta_j in theta:
    print(theta_j)
```

             3          2
    0.08333 x + 0.1667 x + 0.3333 x
            2
    0.1667 x + 0.3333 x

    0.1667 x

Thus the roundoff errors in the first stage are amplified by a factor $z^4/24 + z^3/12 + z^2/6 + z/6$, while the errors in the last stage are amplified by a factor $z/6$.

## Internal instability

Usually internal stability is unimportant since it relates to amplification of roundoff errors, which are very small.  Let's think about when things can go wrong in terms of internal instability.  If $|\theta(z)|$ is of the order $1/\epsilon_{machine}$, then roundoff errors could be amplified so much that they destroy the accuracy of the computation.  More specifically, we should be concerned if $|\theta(z)|$ is of the order $tol/\epsilon_{machine}$ where $tol$ is our desired error tolerance.  Of course, we only care about values of $z$ that lie inside the absolute stability region $S$, since internal stability won't matter if the computation is not absolutely stable.

We can get some idea about the amplification of stage errors by plotting the curves $|\theta(z)|=1$ along with the stability region.  Ideally these curves will all lie outside the stability region, so that all stage errors are damped.

```python
rk4.internal_stability_plot()
```

```python
ssprk4.internal_stability_plot()
```

For both methods, we see that some of the curves intersect the absolute stability region, so some stage errors are amplified.  But by how much?  We'd really like to know the maximum amplification of the stage errors under the condition of absolute stability.  We therefore define the *maximum internal amplification factor* $M$:

$M = \max_j \max_{z \in S} |\theta_j(z)|$

```python
print(rk4.maximum_internal_amplification())
print(ssprk4.maximum_internal_amplification())
```

    (1.6718324799475162, 0.0)
    (2.386110272389574, 0.6)

We see that both methods have small internal amplification factors, so internal stability is not a concern in either case.  This is not surprising for the method with only four stages; it is a surprisingly good property of the method with ten stages.

Questions: *Do SSP RK methods always (necessarily) have small amplification factors?  Can we prove it?*

Now let's look at some methods with many stages.

## Runge-Kutta Chebyshev methods

The paper of Verwer, Hundsdorfer, and Sommeijer deals with RKC methods, which can have very many stages.  The construction of these methods is implemented in NodePy, so let's take a look at them.  The functions `RKC1(s)` and `RKC2(s)` construct RKC methods of order 1 and 2, respectively, with $s$ stages.

```python
s = 4
rkc = rk.RKC1(s)
print(rkc)
```

    Runge-Kutta-Chebyshev (4,1)

     0    |
     1/16 | 1/16
     1/4  | 1/8   1/8
     9/16 | 3/16  1/4   1/8
    ______|________________________
          | 1/4   3/8   1/4   1/8

```python
rkc.internal_stability_plot()
```

It looks like there could be some significant internal amplification here.  Let's see:

```python
rkc.maximum_internal_amplification()
```

    (3.888032523476453, 4.0)

Nothing catastrophic.  Let's try a larger value of $s$:

```python
s = 20
rkc = rk.RKC1(s)
rkc.maximum_internal_amplification()
```

    (10.693352186111435, 20.0)

As promised, these methods seem to have good internal stability properties.  What about the second-order methods?

```python
s = 20
rkc = rk.RKC2(s)
rkc.maximum_internal_amplification()
```

    (29.805924630882497, 25.27)

Again, nothing catastrophic.  We could take $s$ much larger than 20, but the calculations get to be rather slow (in Python) and since we're using floating point arithmetic, the accuracy deteriorates.

Remark: *we could do the calculations in exact arithmetic using Sympy, but things would get even slower.  Perhaps there are some optimizations that could be done to speed this up.  Or perhaps we should use Mathematica if we need to do this kind of thing.*

Remark 2: *of course, for the RKC methods the internal stability polynomials are shifted Chebyshev polynomials.  So we could evaluate them directly in a stable manner using the three-term recurrence (or perhaps scipy's special functions library).  This would also be a nice check on the calculations above.*

## Other methods with many stages

Three other classes of methods with many stages have been implemented in NodePy:

 - SSP families
 - Integral deferred correction (IDC) methods
 - Extrapolation methods

### SSP Families

```python
s = 20
ssprk = rk.SSPRK2(s)
ssprk.internal_stability_plot()
ssprk.maximum_internal_amplification()
```

```python
s = 25 # # of stages
ssprk = rk.SSPRK3(s)
ssprk.internal_stability_plot()
ssprk.maximum_internal_amplification()
```

The SSP methods seem to have excellent internal stability properties.

### IDC methods

```python
p = 6 #order
idc = rk.DC(p-1)
print(len(idc))
idc.internal_stability_plot()
idc.maximum_internal_amplification()
```

IDC methods also seem to have excellent internal stability.

### Extrapolation methods

```python
p = 6 #order
ex = rk.extrap(p)
print(len(ex))
ex.internal_stability_plot()
ex.maximum_internal_amplification()
```

Not so good.  Let's try a method with even more stages (this next computation will take a while; go stretch your legs).

```python
p = 10 #order
ex = rk.extrap(p)
print(len(ex))
ex.maximum_internal_amplification()
```

    46

    (53764.206493020814, 13315.250793650794)

Now we're starting to see something that might cause trouble, especially since such high order extrapolation methods are usually used when extremely tight error tolerances are required.  Internal amplification will cause a loss of about 5 digits of accuracy here, so the best we can hope for is about 10 digits of accuracy in double precision.  Higher order extrapolation methods will make things even worse.  How large are their amplification factors?  (Really long calculation here...)

```python
pmax = 12
ampfac = np.zeros(pmax+1)
for p in range(1, pmax+1):
    ex = rk.extrap(p)
    ampfac[p] = ex.maximum_internal_amplification()[0]
    print(p, ampfac[p])
```

    1 0.0
    2 2.193928615790812
    3 6.173740624457357
    4 25.594706632818742
    5 114.62256593507632
    6 189.8568730592554
    7 629.1679641391079
    8 2541.5350046110575
    9 11602.92082338415
    10 53764.206493020814
    11 246176.07962544935
    12 332603.423069675

```python
plt.semilogy(ampfac, linewidth=3)
plt.xlabel(r"Order $p$")
plt.ylabel(r"Amplification factor")
```

We see roughly geometric growth of the internal amplification factor as a function of the order $p$.  It seems clear that very high order extrapolation methods applied to problems with high accuracy requirements will fall victim to internal stability issues.


---
### 테일러 전개

 - 원래의 형태와 비슷한 다항식
 - 직선의 방정식
      - $f(x_0) + \frac{df(x_0)}{dx}(x-x_0)$

 ##### 다변수 함수의 테일러 전개
  - $f(x_0,y_0) + \frac{\partial f(x_0,y_0)}{\partial x}(x-x_0)+\frac{\partial f(x_0,y_0)}{\partial y}(y-y_0)$

---
### SymPy

```python
import sympy

sympy.init_printing(use_latex='mathjax')
```

```python
x = sympy.symbols('x')
x
```

$\displaystyle x$

```python
f = x * sympy.exp(x)
f
```

$\displaystyle x e^{x}$

```python
# 미분
sympy.diff(f)
```

$\displaystyle x e^{x} + e^{x}$

```python
# 인수분해
sympy.simplify(sympy.diff(f))
```

$\displaystyle \left(x + 1\right) e^{x}$

```python
# 편미분 예제함수

x, y = sympy.symbols('x y')
f = x ** 2 + 4 * x * y + 4 * y ** 2
f
```

$\displaystyle x^{2} + 4 x y + 4 y^{2}$

```python
# 편미분 (미분할 변수를 지정해줘야함)

sympy.diff(f, x)
```

$\displaystyle 2 x + 4 y$

```python
sympy.diff(f, y)
```

$\displaystyle 4 x + 8 y$

```python
# 연쇄법칙

x, mu, sigma = sympy.symbols('x mu sigma')
f = sympy.exp((x-mu) ** 2/sigma**2)
f
```

$\displaystyle e^{\frac{\left(- \mu + x\right)^{2}}{\sigma^{2}}}$

```python
sympy.diff(f, x)
```

$\displaystyle \frac{\left(- 2 \mu + 2 x\right) e^{\frac{\left(- \mu + x\right)^{2}}{\sigma^{2}}}}{\sigma^{2}}$

```python
# 이차도함수
# 미분할 변수를 두 번 넣어주면 된다.

sympy.diff(f, x, x)
```

$\displaystyle \frac{2 \left(1 + \frac{2 \left(\mu - x\right)^{2}}{\sigma^{2}}\right) e^{\frac{\left(\mu - x\right)^{2}}{\sigma^{2}}}}{\sigma^{2}}$

---
#### 예제

```python
x, k, a, b = sympy.symbols('x k a b')  # k,a,b = 상수
```

```python
f = x**3-1
f
```

$\displaystyle x^{3} - 1$

```python
sympy.diff(f)
```

$\displaystyle 3 x^{2}$

```python
f = sympy.log(x**2-3*k)
f
```

$\displaystyle \log{\left(- 3 k + x^{2} \right)}$

```python
sympy.diff(f, x)
```

$\displaystyle \frac{2 x}{- 3 k + x^{2}}$

```python
x, a, b = sympy.symbols('x,a,b')
```

```python
f = sympy.exp(a*x**b)
f
```

$\displaystyle e^{a x^{b}}$

```python
sympy.simplify(sympy.diff(f, x))
```

$\displaystyle a b x^{b - 1} e^{a x^{b}}$

---
#### 연습문제 4.2.6

```python
x, y = sympy.symbols('x y')
```

```python
f = sympy.exp(x**2 + 2*y**2)
f
```

$\displaystyle e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, x)
```

$\displaystyle 2 x e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, y)
```

$\displaystyle 4 y e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, x, x)
```

$\displaystyle 2 \left(2 x^{2} + 1\right) e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, x, y)
```

$\displaystyle 8 x y e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, y, x)
```

$\displaystyle 8 x y e^{x^{2} + 2 y^{2}}$

```python
sympy.diff(f, y, y)
```

$\displaystyle 4 \left(4 y^{2} + 1\right) e^{x^{2} + 2 y^{2}}$