# Вариант 5.4

### Сингулярно-возмущенная система - модель двухлампового генератора Фрюгауфа.

Сингулярно-возмущенная система имеет вид:
\begin{cases}
\varepsilon \dot{x}_1 = - \alpha (y_1 - y_2) + \phi (x_1) - x_2 \\
\varepsilon \dot{x}_2 =   \alpha (y_1 - y_2) + \phi (x_2) - x_1 \\
\dot{y}_1 = x_1 \\
\dot{y}_2 = x_2
\end{cases}
где $\alpha > 0$ - константа порядка единицы, функция
$\phi (u) = - \tan (\pi u /2)$,
$x_1 (0) = x_2 (0) = 0$,
$y_1 (0) = 2$,
$y_2 (0) = 0$.

Будем рассматривать $\varepsilon = 10^{-3}$ и $\varepsilon = 10^{-6}$. Конечное время интегрирования системы $T_k = 20$.

Для численного решения используются ФДН-методы:

\begin{array}{lllll}
k = 2: &\dfrac{3}{2} y_{n+1} &-2 y_n &+\dfrac{1}{2} y_{n-1} & & &= h f_{n+1} \\
k = 3: &\dfrac{11}{6} y_{n+1} &-3 y_n &+\dfrac{3}{2} y_{n-1} &-\dfrac{1}{3} y_{n-2} & &= h f_{n+1} \\
k = 4: &\dfrac{25}{12} y_{n+1} &-4 y_n &+3 y_{n-1} &-\dfrac{4}{3} y_{n-2} &+\dfrac{1}{4} y_{n-3} &= h f_{n+1}
\end{array}
причем значения в недостающих точках доопределеяются с помощью метода Рунге-Кутты (таблица 2)

\begin{array}{c|cc}
\dfrac{2+\sqrt{2}}{2} & \dfrac{2+\sqrt{2}}{2} & 0 \\
\dfrac{2-\sqrt{2}}{2} & -\sqrt{2}             & \dfrac{2+\sqrt{2}}{2} \\
\hline \\
                      & 1/2                   & 1/2
\end{array}
Таблица 1

\begin{array}{c|cc}
\label{rk2}
\dfrac{3+\sqrt{3}}{6} & \dfrac{3+\sqrt{3}}{6} & 0 \\
\dfrac{3-\sqrt{3}}{6} & \dfrac{3-2\sqrt{3}}{6}& \dfrac{3+\sqrt{3}}{6} \\
\hline \\
                      & 1/2                   & 1/2
\end{array}
Таблица 2

Задание
1. Сравнить полученные результаты с результатами вычислений по однократно диагональным неявным методам Рунге-Кутты с двумя стадиями (второго порядка аппроксимации, асимптотически устойчивому, таблица 1 и третьего порядка аппроксимации, таблица 2)
2. Построить функции устойчивости всех используемых численных методов.

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
# Потом поменяем на 10**(-6) (требуется в задании)
epsilon = 10**(0)
# Положим alpha за 1
alpha = 1
# Зададим phi
phi = lambda u: -np.tan(np.pi*u/2)

# Зададим систему
'''
Переменные идут в следующем порядке:
    x_1==y[0], x_2==y[1], y_1==y[2], y_2==y[3]
Порядок тут важен!
'''
#'''
system_of_eqs = lambda t, y: [
    1/epsilon*(-alpha*(y[2] - y[3]) + phi(y[0]) - y[1]),
    1/epsilon*( alpha*(y[2] - y[3]) + phi(y[1]) - y[0]),
    y[0],
    y[1]
]
#'''
# Начальный вектор: y(0) = [...]
y0 = [0, 0, 2, 0]
```

Составим решение методом Рунге-Кутты второго и третьего порядка

```python
t = np.linspace(0, 20, 10**4)
```

```python
from scipy.integrate import solve_ivp
ivp_rk2_1 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='RK23', t_eval=t)
ivp_rk3_1 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='RK45', t_eval=t)
```

Построим методы ФДН с разным числом шагов и сравним их между собой:

```python
ivp_bdf2_1 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=5*10**-4,
                      params={'order': 2})
ivp_bdf3_1 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=10**-4,
                      params={'order': 3})
ivp_bdf4_1 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=10**-5,
                      params={'order': 4})
```

Для сравнения построим график невязки между методам для каждой из переменной $x_1$, $y_1$, $x_2$, $y_2$

```python
for i in range(4):
    plt.plot(ivp_bdf2_1.t, ivp_bdf2_1.y[i, :] - ivp_bdf3_1.y[i, :], label='Разница BDF2 и BDF3', )
    plt.plot(ivp_bdf3_1.t, ivp_bdf2_1.y[i, :] - ivp_bdf4_1.y[i, :], label='Разница BDF2 и BDF4', )
    plt.plot(ivp_bdf4_1.t, ivp_bdf3_1.y[i, :] - ivp_bdf4_1.y[i, :], label='Разница BDF3 и BDF4', )
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

Из графиков видно, что наибольшная невязка достигается между методами ФДН4 и ФДН2. Это обусловлено тем, что четырехшаговый метод оказывается более точным, чем двухшаговый. Переход от трехшагового метода к четырехшаговому сопроводился меньшим изменением невязки, нежели переход от двухшагового к трехшаговому. Вполне возможно, что для данной задачи выбор четырехшагового метода является оптимальным.

Построим на одном листе графики решения двумя методами РК и одним из предложенных ФДН для каждой из переменных.

```python
for i in range(4):
    plt.plot(ivp_rk2_1.t, ivp_rk2_1.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_1.t, ivp_bdf2_1.y[i, :], label='BDF2', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

```python
for i in range(4):
    plt.plot(ivp_rk2_1.t, ivp_rk2_1.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_1.t, ivp_bdf3_1.y[i, :], label='BDF3', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

```python
for i in range(4):
    plt.plot(ivp_rk2_1.t, ivp_rk2_1.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_1.t, ivp_bdf4_1.y[i, :], label='BDF4', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

Методы не дают принципиально различных результатов. Построим теперь разницу между методами РК и ФНД для двух-, трех- и четырехстадийного метода

```python
# Сравним метод Рунге-Кутты 3 с методом ФДН4
plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[2,:] - ivp_bdf4_1.y[2,:], label='Разница между РК 3 и ФДН4')

plt.legend()
plt.grid()

# Сравним метод Рунге-Кутты 3 с методом ФДН3
plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[2,:] - ivp_bdf3_1.y[2,:], label='Разница между РК 3 и ФДН3')

plt.legend()
plt.grid()

# Сравним метод Рунге-Кутты 3 с методом ФДН2
plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[2,:] - ivp_bdf2_1.y[2,:], label='Разница между РК 3 и ФДН2')

plt.legend()
plt.grid()
```

Видим, что метод двуххстадийный метод ФДН дает наибольшую невязку. Это связано с тем, что в области быстрого режима (в нашей задаче это диапазон $t \in [0, 2.5]$) ошибки методов, а следовательно, и их невязки, будут очень быстро расти при вычислениях. Невязка метода ФДН2 как раз показывает эту закономерность.

```python
# Сравним методы Рунге-Кутты между собой
plt.plot(ivp_rk3_1.t, ivp_rk3_1.y[2,:] - ivp_rk2_1.y[2,:], label='Разница между РК 3 и РК 2')

plt.legend()
plt.grid()
```

В случае метода Рунге-Кутты третьего порядка невязка принимает и отрицательные значения. Это означает, что метод третьего порядка более точен, если полагать, что ФДН дает лучшие результаты для заданной системы, поскольку осцилляция невязки означает, что метод РК третьего порядка быстрее приближается к решению задачи. Более того, абсолютная величина невзяки для такого РК меньше во всех точках, чем у метода РК второго порядка.

Для случая $\varepsilon = 10^{-6}$:

```python
# Потом поменяем на 10**(-6) (требуется в задании)
epsilon = 10**(0)
# Положим alpha за 1
alpha = 1
# Зададим phi
phi = lambda u: -np.tan(np.pi*u/2)

# Зададим систему
'''
Переменные идут в следующем порядке:
    x_1==y[0], x_2==y[1], y_1==y[2], y_2==y[3]
Порядок тут важен!
'''
#'''
system_of_eqs = lambda t, y: [
    1/epsilon*(-alpha*(y[2] - y[3]) + phi(y[0]) - y[1]),
    1/epsilon*( alpha*(y[2] - y[3]) + phi(y[1]) - y[0]),
    y[0],
    y[1]
]
#'''
# Начальный вектор: y(0) = [...]
y0 = [0, 0, 2, 0]
```

Составим решение методом Рунге-Кутты второго и третьего порядка

```python
from scipy.integrate import solve_ivp
ivp_rk2_2 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='RK23', t_eval=t)
ivp_rk3_2 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='RK45', t_eval=t)
```

Построим методы ФДН с разным числом шагов и сравним их между собой:

```python
ivp_bdf2_2 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=5*10**-4,
                      params={'order': 2})
ivp_bdf3_2 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=10**-4,
                      params={'order': 3})
ivp_bdf4_2 = solve_ivp(system_of_eqs, (min(t), max(t)), y0, method='BDF', t_eval=t, rtol=10**-5,
                      params={'order': 4})
```

Для сравнения построим график невязки между методам для каждой из переменной $x_1$, $y_1$, $x_2$, $y_2$

```python
for i in range(4):
    plt.plot(ivp_bdf2_2.t, ivp_bdf2_2.y[i, :] - ivp_bdf3_2.y[i, :], label='Разница BDF2 и BDF3', )
    plt.plot(ivp_bdf3_2.t, ivp_bdf2_2.y[i, :] - ivp_bdf4_2.y[i, :], label='Разница BDF2 и BDF4', )
    plt.plot(ivp_bdf4_2.t, ivp_bdf3_2.y[i, :] - ivp_bdf4_2.y[i, :], label='Разница BDF3 и BDF4', )
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

Из графиков видно, что наибольшная невязка тоже достигается между методами ФДН4 и ФДН2. Это обусловлено тем, что четырехшаговый метод оказывается более точным, чем двухшаговый. Переход от трехшагового метода к четырехшаговому сопроводился меньшим изменением невязки, нежели переход от двухшагового к трехшаговому. Вполне возможно, что для данной задачи выбор четырехшагового метода является оптимальным.

Построим на одном листе графики решения двумя методами РК и одним из предложенных ФДН для каждой из переменных.

```python
for i in range(4):
    plt.plot(ivp_rk2_2.t, ivp_rk2_2.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_2.t, ivp_bdf2_2.y[i, :], label='BDF2', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

```python
for i in range(4):
    plt.plot(ivp_rk2_2.t, ivp_rk2_2.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_2.t, ivp_bdf3_2.y[i, :], label='BDF3', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

```python
for i in range(4):
    plt.plot(ivp_rk2_2.t, ivp_rk2_2.y[i, :], label='РК 2 порядка', linestyle='--')
    plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[i, :], label='РК 3 порядка', linestyle='-.')
    plt.plot(ivp_bdf2_2.t, ivp_bdf4_2.y[i, :], label='BDF4', linestyle=':')
    plt.xlabel('t')
    plt.ylabel({0: r'$y_1$', 1: r'$y_2$', 2: r'$x_1$', 3: r'$x_2$'}[i])
    plt.grid()
    plt.legend()
    plt.show()
```

Методы также не дают принципиально различных результатов. Построим теперь разницу между методами РК и ФНД для двух-, трех- и четырехстадийного метода

```python
# Сравним метод Рунге-Кутты 3 с методом ФДН4
plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[2,:] - ivp_bdf4_2.y[2,:], label='Разница между РК 3 и ФДН4')

plt.legend()
plt.grid()

# Сравним метод Рунге-Кутты 3 с методом ФДН3
plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[2,:] - ivp_bdf3_2.y[2,:], label='Разница между РК 3 и ФДН3')

plt.legend()
plt.grid()

# Сравним метод Рунге-Кутты 3 с методом ФДН2
plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[2,:] - ivp_bdf2_2.y[2,:], label='Разница между РК 3 и ФДН2')

plt.legend()
plt.grid()
```

Видим, что метод двуххстадийный метод ФДН дает наибольшую невязку. Это связано с тем, что в области быстрого режима (в нашей задаче это диапазон $t \in [0, 2.5]$) ошибки методов, а следовательно, и их невязки, будут очень быстро расти при вычислениях. Невязка метода ФДН2 как раз показывает эту закономерность.

```python
# Сравним методы Рунге-Кутты между собой
plt.plot(ivp_rk3_2.t, ivp_rk3_2.y[2,:] - ivp_rk2_2.y[2,:], label='Разница между РК 3 и РК 2')

plt.legend()
plt.grid()
```

Сравнение методов Рунге-Кутты второго и третьего порядка показывает, что 3 порядок дает меньшую ошибку. Осцилляции в невязке присутствуют в обоих случаях.

В общем и целом, поведение графиков невязок схожее для обоих $\varepsilon=10^{-3}$ и $\varepsilon=10^{-6}$. Основное различие - абсолютные значения невязок, которые в случае $\varepsilon=10^{-6}$ оказались больше. Это объяснимо тем, что при приближении $\varepsilon \rightarrow 0$ к нулю задача будет становиться все более жесткой и даже неявные методы станут давать большую невязку.

Последний пункт задания - построить функции устойчивости для всех используемых методов. Напомним, что функцией устойчивости называется такая $R(z)$, что $y_{n+1} = R(z) y_n$, где $z = \lambda h$, а $\lambda$ определяется
уравнением $y' = \lambda y$.

### Методы ФДН

\begin{array}{lll}
k = 2: & \dfrac{3}{2} R^2 - 2R + \dfrac{1}{2} = h \dfrac{z}{h} R^2 &\Leftrightarrow & R = 2 \pm \sqrt{1 + 2z} \\
k = 3: & \dfrac{11}{6} R^3 - 3 R^2 + \dfrac{3}{2} R - \dfrac{1}{3} = h \dfrac{z}{h} R^3 & \Leftrightarrow
& R = R_3(z) \text{ (выражение опущено в виду его громоздкости)} \\
k = 4: & \dfrac{25}{12} R^4 - 4 R^3 + 3 R^2 - \dfrac{4}{3} R + \dfrac{1}{4} = h \dfrac{z}{h} R^4 & \Leftrightarrow
& R = R_4(z)
\end{array}

//Построим эти функции с помощью ЭВМ (наше счастье, что они умеют работать с такими вещами)

A-устойчивость: при $Re(z) < 0 \rightarrow |R| < 1$
L-устойчивость: А-устойчивость и выполение условия: $$ \lim_{z \rightarrow \infty} R(z) = 0$$

```python
from sympy import solve, symbols, Rational, I, lambdify

R, z = symbols('R z')

R_2 = (Rational(3, 2) - z)*R**2 - 2*R + Rational(1, 2)
R_3 = (Rational(11, 6) - z)*R**3 - 3*R**2 + Rational(3, 2)*R - Rational(1, 3)
R_4 = (Rational(25, 12) - z)*R**4 - 4*R**3 + 3*R**2 - Rational(4, 3)*R + Rational(1, 4)

# Решения
R_2_sols = solve(R_2, R)
R_3_sols = solve(R_3, R)
R_4_sols = solve(R_4, R)

methods_funcs = [
    lambdify(z, R_2_sols),
    lambdify(z, R_3_sols),
    lambdify(z, R_4_sols)
]
```

```python
import nodepy.linear_multistep_method as lm

bdf2 = lm.backward_difference_formula(2)
bdf3 = lm.backward_difference_formula(3)
bdf4 = lm.backward_difference_formula(4)

def show_A_area(method):
    method.plot_stability_region(N=100, color='green', bounds=[-1, 11, -7, 7])

show_A_area(bdf2)
```

```python
show_A_area(bdf3)
```

```python
show_A_area(bdf4)
```

По оси X отложена действительная часть, по оси Y мнимая. Видим, что с ростом порядка растет область устойчивости.

ФДН порядка 3 и 4 не могут быть А-устойчивыми согласно второму барьеру Далквиста. Для метода ФДН 2 порядка условие А-устойчивости не выполнено (для корня $R = 2 + \sqrt{1 + 2z}$ имеем $R(-0.5) = 2 > 1$).


[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## UE 2.3 Time Shifting and Time Mirroring of Signals

We clarify how to interpret time shifting and time mirroring of a signal $x(t)$ towards $x(-t + t_\text{Shift})$

```python
import sympy as sp
sp.init_printing()
t = sp.symbols('t', real=True)
MirrorAxis = sp.Symbol('MirrorAxis', real=True)
# test signals to be shifted and mirrored
if False:  # special case with no heaviside shift and toffs = 0 to start with
    xref = sp.exp(-t*3) * sp.Heaviside(t)
    toffs = 0  # delayed for >0, leading for <0
    tshift = 1  # delayed for >0, leading for <0
    ymax = 1
elif False:  # case with all different numbers, to see general behaviour
    xref = sp.exp(-t*3) * sp.Heaviside(t+5/4)
    toffs = 2  # delayed for >0, leading for <0
    tshift = 1  # delayed for >0, leading for <0
    ymax = 50
else:  # case for taks FBE36B0684 in sig_sys_ex_02.tex
    xref = sp.exp(-t) * sp.Heaviside(t)
    toffs = -1  # delayed for >0, leading for <0
    tshift = 3  # delayed for >0, leading for <0
    ymax = 1
```

## Method A: 1st Mirror, 2nd Shift

* blue: original signal
* red: mirrored signal
* green: mirrored and shifted signal, final result

```python
x1 = xref.subs(t, +(t - toffs))
x = xref.subs(t,  -(t - toffs))
x2 = xref.subs(t, -(t - toffs - tshift))

px1 = sp.plot(x1, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
              line_color='C0', legend=' ', show=False)
px = sp.plot(x, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
             line_color='C3', legend=' ', show=False)
px2 = sp.plot(x2, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
              line_color='C2', legend=' ', show=False)
pma = sp.plot_implicit(sp.Eq(MirrorAxis, toffs+tshift/2),
                       line_color='C1', legend=' ', show=False)

px1.extend(px)
px1.extend(px2)
px1.extend(pma)
px1.show()
```

```python
toffs  # time at which the mirror axis occurs for blue vs. red
```

## Method B: 1st Shift, 2nd Mirror

* blue: original signal
* red: shifted signal (delayed, i.e. shift to the right for $t_\text{Shift}>0$; leading, i.e. shift to the left for $t_\text{Shift}<0$)
* green: shifted and mirrored (along vertical axis at $t = t_\text{Offset} + t_\text{Shift}$) signal, final result

```python
x1 = xref.subs(t, +(t - toffs))
x = xref.subs(t,  +(t - toffs - tshift))
x2 = xref.subs(t, -(t - toffs - tshift))

px1 = sp.plot(x1, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
              line_color='C0', legend=' ', show=False)
px = sp.plot(x, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
             line_color='C3', legend=' ', show=False)
px2 = sp.plot(x2, (t, -4, 6), ylim=[0, ymax], ylabel='$x(t)$',
              line_color='C2', legend=' ', show=False)
pma = sp.plot_implicit(sp.Eq(MirrorAxis, toffs+tshift/2),
                       line_color='C1', legend=' ', show=False)

px1.extend(px)
px1.extend(px2)
px1.extend(pma)
px1.show()
```

```python
toffs + tshift  # time at which the mirror axis occurs for red vs. green
```

## Method C: direct

$x(t)$ can be directly mirrored along the vertical axis $t = t_\text{Offset} + \frac{t_\text{Shift}}{2}$. See the orange vertical mirroring axis in the plots for method A and B.

```python
toffs + tshift/2  # time at which the mirror axis occurs for blue vs. green
# this is the orange line in both plots above!
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


# (Theory) Quantum Ensemble as Simple Averaging

### Fixed $U_{(i,j)}$ for independent  quantum trajectories

This notebook describes the quantum circuit to obtain $4$ **independent** quantum trajectories in superposition considering a Quantum Ensemble of cosine classifiers (Section 4.1).

### (Step 1) State Preparation

For a 2-qubits $control$ register $(d=2)$, we can build an ensemble of $B=2^2$ classifiers. The $data$ register leverages the *qubit encoding strategy* where a single observation is encoded in a single qubit. In particular, given a dataset made up of $N$ observations $(x_i, y_i)_{i=1, \dots, N}$, where $x_i =(x_{i,1}, x_{i,2})$ is a $2$-dimensional vector and $y_i \in \{0,1\}$ is the binary target variable, the $data$ register encodes the whole dataset and is made of $2 \times N$ qubits:
$$
\begin{align}
    \text{data register: } \underset{features}{\big( \overset{4}{\underset{i=1}{\otimes}} \left| x_i \right\rangle \big)}\otimes  \underset{ labels}{\big( \overset{4}{\underset{i=1}{\otimes}} \left| y_i \right\rangle \big)}
\end{align}
$$
where the values $x_{i,1}$ and $x_{i,2}$ are encoded into the amplitudes of a single qubit:
$$
\begin{align}
    \left| x_i \right\rangle = x_{i,1}\left| 0 \right\rangle +  x_{i,2}\left| 1 \right\rangle
\end{align}
$$
and the two class of the target variable are represented by the two basis state of a single qubit, so if $\left| y_i \right\rangle=\left| 0 \right\rangle$ the $i$-th observation belongs to the class $0$. Otherwise if $\left| y_i \right\rangle=\left| 1 \right\rangle$ the $i$-th observation belongs to the class $1$.

*Qubit encoding strategy* allows to store a training set of $4$ observations using a $8$-qubits $data$ register. Furthermore, in order to obtain different observations in superposition entangled with the $control$ register it is possible to apply swap operations to move observations from one register to the other. In formulas, state preparation step leads to:

$$
\begin{align*}
    \left|\Phi_0\right\rangle &=
    \big( H^{\otimes 2} \otimes S_{(x,y)} \big)\left|0\right\rangle \otimes \left|0\right\rangle \otimes \left|0\right\rangle \nonumber \\
        & =
   \left|c_1\right\rangle \otimes \left|c_2\right\rangle \otimes \left|x\right\rangle \left|y\right\rangle \nonumber\\
    & =
    \frac{1}{\sqrt{2}}\big(\left|0\right\rangle+\left|1\right\rangle\big) \otimes \frac{1}{\sqrt{2}}\big(\left|0\right\rangle+\left|1\right\rangle\big) \otimes \left|x_0,x_1,x_2,x_3\right\rangle \left|y_0,y_1,y_2,y_3\right\rangle \end{align*}
$$

where $S_x$ is the routine which encodes in the amplitudes of a qubit a real vector $x$ and $H$ is the Hadamard transformation.

### (Step 2) Sampling in Superposition

The second step regards the generation of $2^d$ different transformations of the training set in superposition, each entangled with a state of the control register. To this end, $d$ steps are necessary, where each step consists in the entanglement of the $i$-th control qubit with two transformations of $\left|x,y\right\rangle$ based on two random unitaries, $U_{(i,1)}$ and $U_{(i,2)}$, for $i = 1,2$.

As shown in the **Appendix A**, *Sampling in Superposition* step leads to the following quantum state:
$$
\begin{align*}
\left|\Phi_{1}\right\rangle
=  \frac{1}{2}\Big[
 \hspace{.2em} & \left|00\right\rangle U_{(2,1)}U_{(1,1)}\left|x_0,x_1,x_2,x_3 \right\rangle \left| y_0,y_1,y_2,y_3\right\rangle
\nonumber \\ + &
\left|01\right\rangle U_{(2,1)}U_{(1,2)}\left|x_0,x_1,x_2,x_3\right\rangle \left|y_0,y_1,y_2,y_3\right\rangle
\nonumber \\ + &
\left|10\right\rangle U_{(2,2)}U_{(1,1)}\left|x_0,x_1,x_2,x_3\right\rangle \left|y_0,y_1,y_2,y_3\right\rangle
\nonumber \\ + &
\left|11\right\rangle U_{(2,2)}U_{(1,2)}\left|x_0,x_1,x_2,x_3\right\rangle \left| y_0,y_1,y_2,y_3\right\rangle
         \Big]
     %\nonumber \\
 %& \hspace{-2.75em} = \frac{1}{\sqrt{4}} \sum_{b=1}^{4} \ket{b} V_b\ket{x_0,x_1,x_2,x_3; y_0,y_1,y_2,y_3}
\end{align*}
$$

In order to obtain independend quantum trajectories we provide the following definition for $U_{(i,j)}$:
$$U_{(1,1)} = \text{swap}(x_0,x_2) \times \text{swap}(y_0,y_2)$$
$$U_{(1,2)} = \text{swap}(x_1,x_3) \times \text{swap}(y_1,y_3)$$
$$U_{(2,1)} = \mathbf{I} $$
$$U_{(2,2)} = \text{swap}(x_2,x_3) \times \text{swap}(y_2,y_3)$$

where $ \mathbf{I}$ is the identity matrix. Thus, the step of *Sampling in Superposition* leads to:

\begin{align*}
    \left|\Phi_{2}\right\rangle = \frac{1}{2}\Big[
    & \left|11\right\rangle \left|x_0, x_3, x_1, x_2\right\rangle  \left|y_0, y_3, y_1, y_2\right\rangle
    \\ + &
    \left|10\right\rangle \left|x_2, x_1, x_3, x_0\right\rangle  \left|y_2, y_1, y_3, y_0\right\rangle \nonumber\\
    \hspace{.1em}
    + &
    \left|01\right\rangle \left|x_0, x_3, x_2, x_1\right\rangle \left|y_0, y_3, y_2, y_1\right\rangle \\
    + &
    \left|00\right\rangle \left|x_2, x_1, x_0, x_3\right\rangle \left|y_2, y_1, y_0, y_3\right\rangle
    \Big]
\end{align*}

We can see that swap operations allows to entangle different observations (in terms of the indices of the qubits) to different state of the $control$ register. In particular, if considering the last qubit of the *features* and *labels* (sub-)registers, the above choice for $U_{(i,j)}$ guarantees that each quantum state of the control register is entangled with a different training observation. Using a compact representation:

\begin{align}
     \left|\Phi_{2^{'}}\right\rangle = \frac{1}{2}\Big[
    \left|11\right\rangle \left|x_2\right\rangle  \left|y_2\right\rangle
    +
    \left|10\right\rangle\left|x_0\right\rangle\left|y_0\right\rangle
    +
    \left|01\right\rangle\left|x_1\right\rangle\left|y_1\right\rangle
    +
    \left|00\right\rangle\left|x_3\right\rangle \left|y_3\right\rangle
    \Big] =
    \frac{1}{\sqrt{4}}\sum_{i=0}^{3}\left|i\right\rangle\left|x_i,y_i\right\rangle
\end{align}

Notice that, in this case the $i$-th basis state does not correspond to the integer representation of the binary state.

### (Step 3) Learning via interference

First, the $test$ register is initialised to encode the test set, $\tilde{x}$, considering also an additional register to store the final prediction:

\begin{align}
    (S_{\tilde{x}} \otimes \mathbb{1}) \left|0\right\rangle \left|0\right\rangle =\left|x^{(test)}\right\rangle \left|0\right\rangle
\end{align}

Then, the $data$ and $test$ registers interact via interference using the quantum version of the cosine classifier (gate $F$) to compute the estimates of the target variable:

\begin{align*}
    \left|\Phi_{f}\right\rangle
                = & \Big(\mathbb{1}^{\otimes 2} \otimes F \Big) \left|\Phi_{d}\right\rangle \nonumber \\
                = & (\mathbb{1}^{\otimes d} \otimes F )\Bigg[\frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\Bigg] \otimes \left|x^{(test)}\right\rangle \left|0\right\rangle  \nonumber \\
                = & \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\left|x^{(test)}\right\rangle \left|\hat{f}_b\right\rangle
\end{align*}

where $\hat{f}_b$ represents the $b$-th prediction for $\tilde{x}$ given the  $b$-th training set.

### (Step 4) Measurement

\begin{align*}
    \left\langle M \right\rangle = &
    \frac{1}{2^d}\sum_{b=1}^{2^d} \left\langle\hat{f}_b|M|\hat{f}_b\right\rangle =
    \frac{1}{2^d}\sum_{b=1}^{2^d}\left\langle M_b \right\rangle  \nonumber \\
    = & \frac{1}{B} \sum_{b=1}^B \hat{f}_b = \hat{f}_{bag}(\tilde{x}|x,y)
\end{align*}

## Quantum Implementation

```python
# Import pakages and functions
import sys
sys.path.insert(1, '../')

from Utils import *
from modeling import *
```

```python
# load the toy dataset
X_data, Y_data, x_test = load_data_custom()

# Generate the quantum circuit
qc = ensemble_fixed_U(X_data, Y_data, x_test)
```

```python
qc.draw(output='mpl', scale=.6,
        style={'fontsize':15, 'dpi':200})
```

notebooks/Quantum Ensemble - Independent Trajectories.ipynb


*Introduction*

This file enables a user to construct and manipulate geometric objects in $\mathbb{R}^3$. The constructions and manipulations are performed using a conformal model of $\mathbb{R}^3$. A user need not know much about the conformal model, as all constructions and manipulations are via the functions provided here.

My intent is that the functions are self documenting through their code and comments.

Vectors passed to the functions must be in conformal representation.
Exceptions: pt, which converts a 3D point to a conformal point, a 3D normal vector $\mathbf{n}$, and a 3D parallel bivector $\mathbf{B}$.

Objects returned by the functions are in conformal representation. Exception: tp, which returns a 3D point.

To start, pull down the "Cell" menu item and choose "Run All".

Comments and proposed changes or additions are welcome.

```python
# Conformal Model, Amsterdam convention.  Dorst et al. p. 361
from sympy import *
from galgebra.ga import Ga
from galgebra.mv import *
# from lt import *
# from sympy import *

cm3coords = (o,x,y,z,infty) = symbols('o 1 2 3 infty', real=True)
cm3g = '0 0 0 0 -1, 0 1 0 0 0, 0 0 1 0 0, 0 0 0 1 0, -1 0 0 0 0'
cm3 = Ga('o e_1 e_2 e_3 oo', g = cm3g, coords = cm3coords)
(eo, e1, e2, e3, eoo) = cm3.mv()
ep = eo - eoo/2  # ep^2 = +1  GACS 408
em = eo + eoo/2  # em^2 = -1
E = eo^eoo
Ga.dual_mode('Iinv+')
#cm3coords = (o,x,y,z,infty) = symbols('o x y z \infty', real=True)
#cm3 = Ga('o e_x e_y e_z \infty', g = cf3g, coords = cf3coords)
from IPython.display import display
```

```python
def pt(arg): # R^3 vector --> conformal point.
    if isinstance(arg,str):           # Return general 3D point
        v = cm3.mv(arg, 'vector')     # General conformal vector
        v = v + (v < eoo)*eo + (v < eo)*eoo  # 3D part
        v = eo + v + (v<v)*eoo/2
    elif arg == 0:
        v = eo
    elif (arg < eoo) == 0:    # Return point for 3D vector in arg
        v = eo + arg + (arg<arg)*eoo/2
    else: v = arg     # arg already in conformal representation
    return(v)
```

```python
def tp(arg): # conformal point --> R^3 vector
    if isinstance(arg,str):   # Return general 3D vector
        v = cm3.mv(arg, 'vector')
    else:                     # Return 3D vector part of arg
        v = arg
    v = v + (v < eoo)*eo + (v < eo)*eoo
    return(v)
```

```python
def normalize(v):
    if (v < eoo) == 0: # Normalize 3D vector
        return(v/sqrt((v<v).scalar()))
    else:  # Normalize conformal vector: set eo coeff to 1.
        return(-v/(v<eoo))
```

```python
def scalar(arg):
    return(cm3.mv(arg, 'scalar')) # Save user from typing all this
```

* Create direct representations of geometric objects *

```python
def round(*args):  # args are conformal points
    ans = args[0]
    for i in range(1,len(args)):
        ans = ans ^ args[i]
    return(ans)
```

```python
def flat(*args):   # args are conformal points
    return(round(*args) ^ eoo)
```

```python
def line(p,q): # If q is 3D, line thru p parallel to q returned
    return(flat(p,q))
```

```python
def plane(p,q,r):
    return(flat(p,q,r))
```

```python
def circle(p,q,r):
    return(round(p,q,r))
```

```python
def sphere(p,q,r,s):
    return(round(p,q,r,s))
```

* Create dual representations of geometric objects *

```python
def dualLine(p, B): # thru point p, orthogonal to 3D bivector B
    return(p < (B*eoo))  # A vector
```

```python
def dualPlane(p,n):    # n: GA^3 normal vector
    m = normalize(n)
    if isinstance(p,(int, long, float)):
        p = scalar(p)         # Python scalar -> GAlgebra scalar
    if (p!=0) and ((p<p)==0): # p: point on plane.
        return(p < (m^eoo))   # a vector
    else:                     # p: distance to origin.
        return(m + (p*eoo))   # a vector
```

```python
def dualSphere(c,rho):  # c:center.
    if isinstance(rho,(int, long, float)):
        rho = scalar(rho)   # Python scalar -> GAlgebra scalar
    if (rho!=0) and ((rho<rho)==0):  # rho: point on sphere
        return(rho < (c ^ eoo))
    else:                            # rho: radius.
        return(c - (rho*rho*eoo)/2)  # A vector
```

```python
def dualCircle(c,rho,n): # c:center. rho:radius. n:normal vector
    ds = dualSphere(c,rho)
    dp = dualPlane(c,n)
    return(ds^dp)          # A BIvector
```

*  Geometric operations *

```python
def translate(object,a3): # a3: 3D vector
    return(1 - a3*eoo/2)*object*(1 + a3*eoo/2)
```

```python
def rotate(object,itheta):
    return(exp(-itheta/2)*object*exp(itheta/2))
```

```python
def invert(p, norm=False):   # GACS 513
    ans = -(eo - eoo/2)*p*(eo - eoo/2)
    if norm:
        ans = normalize(ans)
    return(ans)
```

```python
# Reflect point p in hyperplane with normal 3D vector n.
def reflect(p,n):
    return(-n*p*(n/norm2(n)))
```

```python
# Can be considerably simplified: A Covariant Approach ..., 16
def dilate(p, alpha, norm = False):  # Dilate by alpha (> 0)
    ans = exp(E*ln(alpha)/2)*p*exp(-E*ln(alpha)/2)
    if norm:
        ans = normalize(ans)
    return(ans)
```

* Play *

```python

```


# 準ニュートン法
引き続き、非線形最適化の手法について学習していきましょう。
今回は、ニュートン法においてその計算が必要であったヘッセ行列(の逆行列) の近似を、計算の過程で並行して求めることで、ニュートン法のような高速な収束性を、より少ない計算量で行うことのできる、**準ニュートン法**について扱います。
準ニュートン法に於けるヘッセ行列(の逆行列) の近似公式はいくつも知られていますが、ここではAnaconda に標準で添付されているSciPy において提供される、BFGS(Broyden-Fletcher-Goldfarb-Shanno) 公式による準ニュートン法について、その生成点列の挙動を確認しましょう。

```python
import numpy as np
from scipy.optimize import line_search

def sdm(xk, iter=15):
    sequence = [xk]
    for k in range(iter):
        dk = -jac(xk)
        alpha = line_search(fun, jac, xk, dk)[0]
        xk = xk + alpha * dk
        sequence.append(xk)
    return np.array(sequence)
```

## 目的関数
今回も、前々回、前回と扱ってきた、
\begin{align*}
f(x_0, x_1):=\sin\left(\frac{1}{2}x_0^2-\frac{1}{4}x_1^2+3\right)\cos(2x_0+1-e^{x_1})
\end{align*}
を目的関数として考えましょう。
この関数は、前回確認した通り、下記のような複雑な形状をしていました。

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure(figsize=(8, 6), dpi=80)
ax = plt.axes(projection='3d')
X, Y = np.meshgrid(np.linspace(-1, 1, 100), np.linspace(-1, 1, 100))
Z = np.array([[np.sin((x ** 2) / 2 - (y ** 2 ) / 4 + 3) * np.cos(2 * x + 1 - np.exp(y)) for x, y in zip(vx, vy)] for vx, vy in zip(X, Y)])
ax.plot_surface(X, Y, Z, cmap='plasma')
```

この目的関数`fun` およびその勾配`jac` は、それぞれ下記のようにSymPy を用いてPython 関数として導出することができました。
後にNewton 法とも結果の比較を行うために、ヘッセ行列`hess` も同時に求めておきましょう。

```python
import numpy as np
from sympy import symbols, sin, cos, exp, diff

_x, _y = symbols('x, y')
F = sin((_x ** 2) / 2 - (_y ** 2) / 4 + 3) * cos(2 * _x + 1 - exp(_y))
FdX, FdY = diff(F, _x), diff(F, _y)
FdXdX, FdXdY = diff(FdX, _x), diff(FdX, _y)
FdYdX, FdYdY = diff(FdY, _x), diff(FdY, _y)

def fun(x):
    p = {_x: x[0], _y: x[1]}
    return float(F.subs(p))

def jac(x):
    p = {_x: x[0], _y: x[1]}
    return np.array([FdX.subs(p), FdY.subs(p)], dtype=np.float)

def hess(x):
    p = {_x: x[0], _y: x[1]}
    return np.array([[FdXdX.subs(p), FdXdY.subs(p)],
                      [FdYdX.subs(p), FdYdY.subs(p)]], dtype=np.float)
```

## BFGS 公式による準ニュートン法
それでは早速、実験をしてみましょう。
BFGS 公式による準ニュートン法は、下記の関数により実行することができます。
```python
from scipy.optimize import minimize
x = minimize(fun, x0, method='bfgs', jac=jac).x
```
前回の信頼領域法と同様に、``scipy.optimize`` モジュールの``minimize`` 関数より、最適化アルゴリズムを呼び出すことができます。
信頼領域法と異なり、ヘッセ行列を指定する必要はなく、目的関数``fun``、初期点``x0`` および勾配``jac`` を与えることで実行できます。
それでは早速、生成点列の挙動を確認してみましょう。
初期点は前回までと同様に、$x^{(0)}:=(-0.3, 0.2)^\top$ を与えるものとします。
ここでは、比較のため、最急降下法とニュートン法による生成点列も同時にプロットしてみましょう。

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize, line_search

def newton(xk, iter=15):
    sequence = [xk]

    return np.array(sequence)
X, Y = np.meshgrid(np.linspace(-1, 1, 100), np.linspace(-1, 1, 100))
plt.ylim(-1, 1)
plt.xlim(-1, 1)
Z = np.array([[np.sin((x ** 2) / 2 - (y ** 2 ) / 4 + 3) * np.cos(2 * x + 1 - np.exp(y)) for x, y in zip(vx, vy)] for vx, vy in zip(X, Y)])
plt.contour(X, Y, Z, cmap='plasma', levels=np.linspace(np.min(Z), np.max(Z), 15))

sequence = [np.array([-0.3, 0.2])]
for k in range(15):
    dk = -jac(sequence[-1])
    alpha = line_search(fun, jac, sequence[-1], dk)[0]
    sequence.append(sequence[-1] + alpha * dk)
sequence = np.array(sequence)
plt.plot(sequence[:, 0], sequence[:, 1], marker='o', label='sdm')

sequence = [np.array([-0.3, 0.2])]
for k in range(15):
    sequence.append(sequence[-1] + np.linalg.solve(hess(sequence[-1]), -jac(sequence[-1])))
sequence = np.array(sequence)
plt.plot(sequence[:, 0], sequence[:, 1], marker='v', label='newton')

sequence = [np.array([-0.3, 0.2])]
minimize(fun, sequence[0], method='bfgs', jac=jac, callback=lambda xk: sequence.append(xk))
sequence = np.array(sequence)
plt.plot(sequence[:, 0], sequence[:, 1], marker='^', label='bfgs')
plt.legend()
```

準ニュートン法による生成点列は、多少の振動こそ起きていますが、最急降下法(sdm) のような極端な振動はなく、しかしニュートン法(newton) のように発散はせずに、比較的効率的に最適解へ収束していることが見て取れます。

## 参考文献
  * 福島雅夫著『新版　数理計画入門』(朝倉書店; 2011)
  * 矢部博著『工学基礎　最適化とその応用』(数理工学社; 2006)
  * J. Nocedal, S. J. Wright: Numerical Optimization (2nd ed.), Springer (2006)
  * [Gradient descent - Wikipedia](https://en.wikipedia.org/wiki/Gradient_descent) (目的関数はこのページのものを使用しました。)


# Ball and Stick

*(Behrens et al. 2003)* proposed to represent the intra-axonal diffusion as a Stick - a cylinder with zero radius - and the extra-axonal diffusion as a Ball - an isotropic Gaussian compartment.

\begin{equation}
 E^{\textrm{Ball and}}_{\textrm{Sticks}}=\underbrace{f_h\overbrace{E_{iso}(\lambda_{\textrm{iso}})}^{\textrm{Ball}}}_{\textrm{Extra-Axonal}}+ \sum_{i=1}^{N}\underbrace{f_{i,r}\overbrace{E_r(\boldsymbol{\mu}_i|\lambda_\parallel)}^{\textrm{Stick}}}_{\textrm{Intra-Axonal}}
\end{equation}

For this first example, we will restrict ourselves to the case where $N$=2

# Human Connectome Project Example

```python
from dmipy.data import saved_data
scheme_hcp, data_hcp = saved_data.wu_minn_hcp_coronal_slice()
```

    This data slice originates from Subject 100307 of the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

```python
import matplotlib.pyplot as plt
import matplotlib.patches as patches
%matplotlib inline

fig, ax = plt.subplots(1)
ax.imshow(data_hcp[:, 0, :, 0].T, origin=True)
rect = patches.Rectangle((70,70),20,20,linewidth=1,edgecolor='r',facecolor='none')
ax.add_patch(rect)
ax.set_axis_off()
ax.set_title('HCP coronal slice B0 with ROI');
```

```python
data_hcp_small = data_hcp[70:90, :, 70:90]
```

## Generate Ball and Stick model

```python
from dmipy.signal_models import cylinder_models, gaussian_models
from dmipy.core.modeling_framework import MultiCompartmentModel
ball = gaussian_models.G1Ball()
stick1 = cylinder_models.C1Stick()
stick2 = cylinder_models.C1Stick()
BAS_mod = MultiCompartmentModel(models=[ball, stick1, stick2])
```

```python
BAS_mod.parameter_cardinality
```

    OrderedDict([('C1Stick_2_mu', 2),
                 ('G1Ball_1_lambda_iso', 1),
                 ('C1Stick_1_mu', 2),
                 ('C1Stick_1_lambda_par', 1),
                 ('C1Stick_2_lambda_par', 1),
                 ('partial_volume_0', 1),
                 ('partial_volume_1', 1),
                 ('partial_volume_2', 1)])

```python
BAS_mod.set_fixed_parameter('G1Ball_1_lambda_iso', 3e-9)
```

```python
BAS_mod.parameter_cardinality
```

    OrderedDict([('C1Stick_2_mu', 2),
                 ('C1Stick_1_mu', 2),
                 ('C1Stick_1_lambda_par', 1),
                 ('C1Stick_2_lambda_par', 1),
                 ('partial_volume_0', 1),
                 ('partial_volume_1', 1),
                 ('partial_volume_2', 1)])

## Fit Ball and Sticks model to HCP data

```python
BAS_fit_hcp = BAS_mod.fit(scheme_hcp, data_hcp_small)
```

    Using parallel processing with 8 workers.
    Setup brute2fine optimizer in 80.1930429935 seconds
    Fitting of 400 voxels complete in 80.4078359604 seconds.
    Average of 0.201019589901 seconds per voxel.

```python
from dipy.viz import fvtk
from dipy.viz.actor import slicer
import numpy as np
affine = np.eye(4)
affine[0,3] = -10
affine[1,3] = -10

volume1 = BAS_fit_hcp.fitted_parameters['partial_volume_1']
volume2 = BAS_fit_hcp.fitted_parameters['partial_volume_2']
volume_im = slicer(1. - BAS_fit_hcp.fitted_parameters['partial_volume_0'][:, 0, :, None],
                   interpolation='nearest', affine=affine, opacity=0.7)

peak_intensities = np.concatenate([volume1[..., None], volume2[..., None]], axis=-1)

ren = fvtk.ren()
peaks = BAS_fit_hcp.peaks_cartesian()
peaks_fvtk = fvtk.peaks(peaks, peak_intensities, scale=1.)
peaks_fvtk.RotateX(90)
peaks_fvtk.RotateZ(180)
peaks_fvtk.RotateY(180)
fvtk.add(ren, peaks_fvtk)
fvtk.add(ren, volume_im)
fvtk.record(ren=ren, size=[700, 700], out_path='dipy.png')
```

```python
import matplotlib.image as mpimg
img = mpimg.imread('dipy.png')
plt.figure(figsize=[10, 10])
plt.imshow(img[100:-97, 100:-85])
```

Seems like two sticks overparameterize the problem in the corpus callosum, but on the right we can see consistent crossings. Possibly MIX can do a slighly better job, but it looks like we need somethings that enforces sparsity in the volume fractions.

## References
- Behrens, Timothy EJ, et al. "Characterization and propagation of uncertainty in diffusion‐weighted MR imaging." Magnetic resonance in medicine 50.5 (2003): 1077-1088


[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Übung / Exercise 7 Sampling Reconstruction Filters

```python
import numpy as np
import matplotlib.pyplot as plt

def my_sinc(x):  # we rather use definition sinc(x) = sin(x)/x, thus:
    return np.sinc(x/np.pi)
```

Assume ideal Dirac comb sampling with $h_s(t) = \frac{1}{T} {\bot \!\! \bot \!\! \bot}(\frac{t}{T}) = \sum_{m=-\infty}^{+\infty} \delta(t-m T)$ and reconstruction with the following reconstruction filters.

**Ideal lowpass**

\begin{equation}
h_{ideal}(t) = \mathrm{sinc}(a t) \circ - \bullet H_{ideal}(\omega) = \frac{\pi}{|a|} \mathrm{rect}(\frac{\omega}{2 a})
\end{equation}

For $a = \frac{\pi}{T} > 0$, we get

\begin{equation}
h_{ideal}(t) = \mathrm{sinc}(\frac{\pi t}{T}) \circ - \bullet H_{ideal}(\omega) = T \mathrm{rect}(\frac{\omega T}{2 \pi}) = T \mathrm{rect}(\frac{\omega}{\omega_s}),
\end{equation}

for which the passband $|\omega|<\frac{\omega_s}{2} = \frac{\frac{2\pi}{T}}{2} = \frac{\pi}{T}$ holds.

**Spline 0th order** (nearest neighbor, zero order hold)

\begin{equation}
h_{r0}(t) = \mathrm{rect}(a t) \circ-\bullet H_{r0}(\omega) = \frac{1}{|a|} \mathrm{sinc}(\frac{\omega}{2 a})
\end{equation}

For $a = \frac{1}{T} > 0$, we get

\begin{equation}
h_{r0}(t) = \mathrm{rect}(\frac{t}{T}) \circ-\bullet H_{r0}(\omega) = T \mathrm{sinc}(\frac{\omega T}{2}).
\end{equation}

**Spline 1st order** (linear)

\begin{equation}
h_{r1}(t) = \Lambda(a t) \circ-\bullet H_{r1}(\omega) = \frac{1}{|a|} \mathrm{sinc}^2(\frac{\omega}{2 a})
\end{equation}

For $a = \frac{1}{T} > 0$, we get

\begin{equation}
h_{r1}(t) = \Lambda(\frac{t}{T}) \circ-\bullet H_{r1}(\omega) = T \mathrm{sinc}^2(\frac{\omega T}{2})
\end{equation}

```python
N = 3.5  # plot range

Ts = 1.23456789  # sampling intervall in s, dont care since we plot normalized

fs = 1/Ts  # samping frequency in Hz
ws = 2*np.pi/Ts  # angular sampling frequency in rad/s
print('ws/2 = ', ws/2, 'rad/s')
print('ws   = ', ws, 'rad/s')

t = np.arange(-N*Ts, N*Ts, Ts/100)
hsinc = my_sinc(np.pi*t/Ts)

w = np.arange(-N*ws, +N*ws, ws/100)
Hsinc = Ts * my_sinc(Ts*w/2) / Ts  # spline 0th order
Hsinc2 = Ts * my_sinc(Ts*w/2)**2 / Ts  # spline 1st order

plt.figure(figsize=(12, 5))

plt.subplot(1, 2, 1)
plt.plot(t/Ts, hsinc, 'C0', label=r'$\mathrm{sinc}(\pi\,\frac{t}{T_s})$')
plt.plot((t[0]/Ts, -(Ts/2)/Ts), (0, 0), 'C1')
plt.plot((+(Ts/2)/Ts, t[-1]/Ts), (0, 0), 'C1')
plt.plot((-(Ts/2)/Ts, -(Ts/2)/Ts), (0, 1), 'C1')
plt.plot((+(Ts/2)/Ts, +(Ts/2)/Ts), (0, 1), 'C1')
plt.plot((-(Ts/2)/Ts, +(Ts/2)/Ts), (1, 1), 'C1',
         label=r'$\mathrm{rect}(\frac{t}{T_s})$')
plt.plot((t[0]/Ts, -(Ts)/Ts), (0, 0), 'C2-.', lw=3)
plt.plot((+(Ts)/Ts, t[-1]/Ts), (0, 0), 'C2-.', lw=3)
plt.plot((-(Ts)/Ts, 0), (0, 1), 'C2-.', lw=3)
plt.plot((+(Ts)/Ts, 0), (0, 1), 'C2-.', lw=3, label=r'$\Lambda(\frac{t}{T_s})$')
plt.xlabel(r'$\frac{t}{T_s}$')
plt.ylabel(r'$h_r(t)$')
plt.title('Impulse responses of reconstruction filters')
plt.xlim(t[0]/Ts, t[-1]/Ts)
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(1, 2, 2)
plt.plot((w[0]/ws, -np.pi/Ts / ws), (0, 0), 'C0')
plt.plot((+np.pi/Ts / ws, w[-1]/ws), (0, 0), 'C0')
plt.plot((-np.pi/Ts / ws, -np.pi/Ts / ws), (0, Ts/Ts), 'C0')
plt.plot((+np.pi/Ts / ws, +np.pi/Ts / ws), (0, Ts/Ts), 'C0')
plt.plot((-np.pi/Ts / ws, +np.pi/Ts / ws), (Ts/Ts, Ts/Ts), 'C0',
         label=r'$T_s\,\mathrm{rect}(\frac{1}{\pi}\cdot\frac{\omega\,T_s}{2})$')
plt.plot(w/ws, Hsinc, 'C1',
         label=r'$T_s\,\mathrm{sinc}(\frac{\omega\,T_s}{2})$')
plt.plot(w/ws, Hsinc2, 'C2-.',
         label=r'$T_s\,\mathrm{sinc}^2(\frac{\omega\,T_s}{2})$', lw=3)
plt.xlabel(
    r'$\frac{\omega T_s}{2 \pi} = \frac{\Omega}{2 \pi} = \frac{f}{f_s}$')
plt.ylabel(r'$H_r(\mathrm{j}\omega)\,/\,T_s$')
plt.title('Fourier transforms of impulse responses')
plt.xlim(w[0]/ws, w[-1]/ws)
plt.legend(loc='upper left')
plt.grid(True)

plt.savefig('ReconstructionSplines.pdf')
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


```
from sympy import I, Matrix, symbols, Symbol, eye
from datetime import datetime
import numpy as np
import pandas as pd

```

```
'''
Rotas
'''
R1 = Matrix([[0,"L1p3",0,0,0,0],[0,0,"L1v1",0,0,0],[0,0,0,"L1p4",0,0],[0,0,0,0,"L1v3",0],[0,0,0,0,0,"L1v4"],[0,0,0,0,0,0]])
R2 = Matrix([[0,"L2p3",0,0,0,0],[0,0,"L2v2",0,0,0],[0,0,0,"L2p5",0,0],[0,0,0,0,"L2v3",0],[0,0,0,0,0,"L2v5"],[0,0,0,0,0,0]])
R3 = Matrix([[0,"L3p3",0,0,0,0],[0,0,"L3v5",0,0,0],[0,0,0,"L3p1",0,0],[0,0,0,0,"L3v3",0],[0,0,0,0,0,"L3v1"],[0,0,0,0,0,0]])
```

```
'''
Seções de bloqueio
'''
T1 = Matrix([[0, "p1"],["v1", 0]])
T2 = Matrix([[0, "p2"],["v2", 0]])
T3 = Matrix([[0, "p3"],["v3", 0]])
T4 = Matrix([[0, "p4"],["v4", 0]])
T5 = Matrix([[0, "p5"],["v5", 0]])
```

```
def kronSum(A,B):
  m = np.size(A,1)
  n = np.size(B,1)
  A = np.kron(A,np.eye(n))
  B = np.kron(np.eye(m),B)
  return A + B
```

```
momento_inicio = datetime.now()
'''
Algebra de rotas
'''
rotas = kronSum(R1,R2)
rotas = kronSum(rotas,R3)

'''
Algebra de seções
'''
secoes = kronSum(T1,T2)
secoes = kronSum(secoes,T3)
secoes = kronSum(secoes,T4)
secoes = kronSum(secoes,T5)
'''
Algebra de sistema
'''
sistema = np.kron(rotas, secoes)
# calcula tempo de processamento
tempo_processamento = datetime.now() - momento_inicio
```

```
sistema = pd.DataFrame(data=sistema,index=list(range(1,np.size(sistema,0)+1)), columns=list(range(1,np.size(sistema,1)+1)))
```

```
sistema.shape
```

    (6912, 6912)

```
print(tempo_processamento)
```

    0:01:10.109705

```
sistema
```

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      ...
      6873
      6874
      6875
      6876
      6877
      6878
      6879
      6880
      6881
      6882
      6883
      6884
      6885
      6886
      6887
      6888
      6889
      6890
      6891
      6892
      6893
      6894
      6895
      6896
      6897
      6898
      6899
      6900
      6901
      6902
      6903
      6904
      6905
      6906
      6907
      6908
      6909
      6910
      6911
      6912

      1
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1.0*L3p3*p5
      1.0*L3p3*p4
      0
      1.0*L3p3*p3
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      2
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1.0*L3p3*v5
      0
      0
      1.0*L3p3*p4
      0
      1.0*L3p3*p3
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      3
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1.0*L3p3*v4
      0
      0
      1.0*L3p3*p5
      0
      0
      1.0*L3p3*p3
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      4
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1.0*L3p3*v4
      1.0*L3p3*v5
      0
      0
      0
      0
      1.0*L3p3*p3
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      5
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1.0*L3p3*v3
      0
      0
      0
      0
      1.0*L3p3*p5
      1.0*L3p3*p4
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      6908
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      6909
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      6910
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      6911
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      6912
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

6912 rows × 6912 columns

```
sistema.loc[6858,6906]
```

    1.0*L3v1*p1

```
momento_inicio = datetime.now()
colunas = ['denode', 'paranode', 'aresta']
grafo = pd.DataFrame(columns=colunas)
r = 1
c = 1
for j in range(np.size(sistema,0)):
  for i in range(np.size(sistema,0)):
    if sistema.loc[r,c]==0 and c < np.size(sistema,0):
      c += 1
    elif c < np.size(sistema,0):
      grafo.loc[len(grafo)+1] = (r, c, sistema.loc[r,c])
      c += 1
    else:
      c = 1
      r += 1
tempo_processamento = datetime.now() - momento_inicio
print(tempo_processamento)
```

    0:19:39.230451

```
grafo
```

      denode
      paranode
      aresta

      1
      1
      34
      1.0*L3p3*p5

      2
      1
      35
      1.0*L3p3*p4

      3
      1
      37
      1.0*L3p3*p3

      4
      1
      41
      1.0*L3p3*p2

      5
      1
      49
      1.0*L3p3*p1

      ...
      ...
      ...
      ...

      86381
      6880
      6896
      1.0*L3v1*v1

      86382
      6880
      6904
      1.0*L3v1*v2

      86383
      6880
      6908
      1.0*L3v1*v3

      86384
      6880
      6910
      1.0*L3v1*v4

      86385
      6880
      6911
      1.0*L3v1*v5

86385 rows × 3 columns

```
grafo.to_csv('grafo.csv', sep=";")
```

```
from google.colab import files
files.download('grafo.csv')
```

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>


# Markov-Chain Monte Carlo (MCMC)

## Lotka-Volterra Model

\begin{equation}
\begin{split}
\frac{dx}{dt} =&  v-v^3-w+I_{ext}\\
\frac{dy}{dt} =& \frac{1}{\tau}(v-a-bw)
\end{split}
\end{equation}

```python
import argparse
import os

import matplotlib
import matplotlib.pyplot as plt

from jax.experimental.ode import odeint
import jax.numpy as jnp
from jax.random import PRNGKey

import numpyro
import numpyro.distributions as dist
from numpyro.examples.datasets import LYNXHARE, load_dataset
from numpyro.infer import MCMC, NUTS, Predictive

```

```python

def dz_dt(z, t, theta):
    """
    Lotka–Volterra equations. Real positive parameters `alpha`, `beta`, `gamma`, `delta`
    describes the interaction of two species.
    """
    u = z[0]
    v = z[1]
    alpha, beta, gamma, delta = theta[..., 0], theta[..., 1], theta[..., 2], theta[..., 3]
    du_dt = (alpha - beta * v) * u
    dv_dt = (-gamma + delta * u) * v

    return jnp.stack([du_dt, dv_dt])

def model(N, y=None):
    """
    :param int N: number of measurement times
    :param numpy.ndarray y: measured populations with shape (N, 2)
    """
    # initial population
    z_init = numpyro.sample("z_init", dist.LogNormal(jnp.log(10), 1), sample_shape=(2,))
    # measurement times
    ts = jnp.arange(float(N))
    # parameters alpha, beta, gamma, delta of dz_dt
    theta = numpyro.sample(
        "theta",
        dist.TruncatedNormal(low=0., loc=jnp.array([0.5, 0.05, 1.5, 0.05]),
                             scale=jnp.array([0.5, 0.05, 0.5, 0.05])))
    # integrate dz/dt, the result will have shape N x 2
    z = odeint(dz_dt, z_init, ts, theta, rtol=1e-5, atol=1e-3, mxstep=500)

    # measurement errors, we expect that measured hare has larger error than measured lynx
    sigma = numpyro.sample("sigma", dist.Exponential(jnp.array([1, 2])))

    # measured populations (in log scale)
    numpyro.sample( "y" , dist.Normal(jnp.log(z), sigma), obs=y)

```

```python
device = 'cpu'

num_samples = 1000
num_chains = 2
num_warmup = 200

numpyro.set_platform(device)
numpyro.set_host_device_count(num_chains)

_, fetch = load_dataset(LYNXHARE, shuffle=False)
year, data = fetch()  # data is in hare -> lynx order

# use dense_mass for better mixing rate
mcmc = MCMC(NUTS(model, dense_mass=True),
            num_warmup, num_samples, num_chains=num_chains, progress_bar= True)
mcmc.run(PRNGKey(1), N=data.shape[0], y=jnp.log(data))
mcmc.print_summary()

```

    sample: 100%|██████████| 2000/2000 [17:04<00:00,  1.95it/s, 1023 steps of size 4.44e-11. acc. prob=0.80]

                    mean       std    median      5.0%     95.0%     n_eff     r_hat
      sigma[0]      0.52      0.00      0.52      0.52      0.52      0.50      1.00
      sigma[1]      0.58      0.00      0.58      0.58      0.58      0.50      1.00
      theta[0]      3.72      0.00      3.72      3.72      3.72      0.50      1.00
      theta[1]      0.84      0.00      0.84      0.84      0.84      0.50      1.00
      theta[2]      1.55      0.00      1.55      1.55      1.55      0.50      1.00
      theta[3]      4.53      0.00      4.53      4.53      4.53      0.50      1.00
     z_init[0]     18.89      0.00     18.89     18.89     18.89      0.50      1.00
     z_init[1]     19.14      0.00     19.14     19.14     19.14      0.50      1.00

    Number of divergences: 0

```python
# predict populations
ax = plt.figure(figsize=(15.5, 7.2))
y_pred = Predictive(model, mcmc.get_samples())(PRNGKey(2), data.shape[0])["y"]
pop_pred = jnp.exp(y_pred)
mu, pi = jnp.mean(pop_pred, 0), jnp.percentile(pop_pred, (10, 90), 0)
plt.plot(year, data[:, 0], "ko", mfc="none", ms=4, label="true hare", alpha=0.67)
plt.plot(year, data[:, 1], "bx", label="true lynx")
plt.plot(year, mu[:, 0], "k-.", label="pred hare", lw=1, alpha=0.67)
plt.plot(year, mu[:, 1], "b--", label="pred lynx")
plt.fill_between(year, pi[0, :, 0], pi[1, :, 0], color="k", alpha=0.2)
plt.fill_between(year, pi[0, :, 1], pi[1, :, 1], color="b", alpha=0.3)
plt.gca().set(ylim=(0, 160), xlabel="year", ylabel="population (in thousands)")
plt.title("Posterior predictive (80% CI) with predator-prey pattern.")
plt.legend()
plt.tight_layout()
plt.show()

```

```python
mcmc.print_summary()
```

                    mean       std    median      5.0%     95.0%     n_eff     r_hat
      sigma[0]      0.52      0.00      0.52      0.52      0.52      0.50      1.00
      sigma[1]      0.58      0.00      0.58      0.58      0.58      0.50      1.00
      theta[0]      3.72      0.00      3.72      3.72      3.72      0.50      1.00
      theta[1]      0.84      0.00      0.84      0.84      0.84      0.50      1.00
      theta[2]      1.55      0.00      1.55      1.55      1.55      0.50      1.00
      theta[3]      4.53      0.00      4.53      4.53      4.53      0.50      1.00
     z_init[0]     18.89      0.00     18.89     18.89     18.89      0.50      1.00
     z_init[1]     19.14      0.00     19.14     19.14     19.14      0.50      1.00

    Number of divergences: 0

```python
params = mcmc.get_samples()['theta']

fig, ax = plt.subplots(4,2, figsize=(20, 40))

ax[0,0].plot(params[:num_samples,0]), ax[0,1].hist(params[:num_samples,0])
ax[1,0].plot(params[:num_samples,1]), ax[1,1].hist(params[:num_samples,1])
ax[2,0].plot(params[:num_samples,2]), ax[2,1].hist(params[:num_samples,2])
ax[3,0].plot(params[:num_samples,3]), ax[3,1].hist(params[:num_samples,3])

for i in range(4):
    ax[i,0].set_title('Trace plot')
    ax[i,0].set_xlabel('Chain Iter')
    ax[i,1].set_ylabel('Value')
    ax[i,1].set_ylabel('Freq')

plt.show()
```

```python
params.all
```

    (1000, 4)

```python

```