# Glosten-Milgrom Model for security market

## Deriving the model

```python
%matplotlib inline
#import the needed packages
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.figure import Figure
import sympy as sm
import pandas as pd

#from local file import plotterwindow
from plotter import PlotterWindow

#use pretty print
sm.init_printing(use_unicode=True)
```

we create the equation symbols needed

```python
mu_t = sm.symbols("mu_t")
mu_t1 = sm.symbols("mu_t-1")
b_t = sm.symbols("b_t")
a_t = sm.symbols("a_t")
p_t = sm.symbols("P_t")
v = sm.symbols("v")
v_h = sm.symbols("v^H")
v_l = sm.symbols("v^L")
d_t = sm.symbols("d_t")
sd_t = sm.symbols("s(d_t)")
s_a = sm.symbols("s_t^a")
s_b = sm.symbols("s_t^b")
theta_t = sm.symbols("theta_t")
theta_t1 = sm.symbols("theta_t-1")
pi = sm.symbols("pi")
beta_b = sm.symbols("beta_B")
beta_s = sm.symbols("beta_S")
spib = sm.symbols("Pi_buy")
spis = sm.symbols("Pi_sell")
theta_p=sm.symbols("theta^+")
theta_m=sm.symbols("theta^-")
```

There is a single market with 1 dealer. The dealer buys and sells a single security at the ask price $a_t$ and bid price $b_t$. In every time period t he takes a single order from a single trader. The order is denoted $d_t$ which takes the value 1 for buy order, -1 for a sale order and 0 for no order.

On the market there are two types of traders, Informed traders and liquidity traders. The informed traders have a high amount of information about the security and knows the true value of it. Further they seek to maximize their profit trading with the security, which leads to only buying if the security value higher than the ask price and only selling selling if the value is lower than the bid price.
Liquidity traders do not have information about the security or rather, they dont care. These traders buys or sells the security independent on the value of the security. Among liquidity traders you find traders who either seeks to diversify their portfolio, thus buy, or who needs to liquify some assets. The liquidity trader buys with probability $\beta_B$ and sells with probability $\beta_S$.

$\pi$ share of the traders is informed traders.

On the market there is unequal information. The informed traders have private information about the value of the security. The dealer only has access to public information about the market, such as the ratio of informed traders. The dealer however receives information with each trade order. An informed trader would never sell if $a_t<v$, therefore an unequal amount of buy orders would indicate $v=v^H$

Before we start on our model equations we make two assumptions:
 i) The dealer is risk neutral and is in a competitive.
 ii) there is no trade cost on orders on the market

To simplify our model we set the security v to be binary distributed taking the value $v^H$ or $v^L$, the superscript denoting high and low value.

The dealer has the belief $\theta_t$, which is his estimate of the probability $p(v=v^h)=\theta$. From his belief he finds his expected value of security v denoted $\mu_t$

```python
sm.Eq(mu_t,theta_t*v_h+(1-theta_t)*v_l)
```

The dealer sets his ask and bid price based on his expectition to v conditional on the information $\Omega_{t-1}$ and whether he expects a buy or sell order.
$$a_t=E(v\mid \Omega_{t-1} , d_t=1)$$
$$b_t=E(v\mid \Omega_{t-1} , d_t=-1)$$
Further, since the dealer is in a competitive market he sets the ask and bid price according to an expected null profit. Since the dealer does not know the true value of the security, he loses on every trade made with an informed trader. The deficit is made up by trading with liquidity traders. Based on his expectation to the security and his knowledge of the market, he derives the following profit functions

```python
sm.Eq(theta_t1*pi*(a_t-v_h)+beta_b*(1-pi)*(a_t-mu_t1), sm.symbols("Pi^buy_t"))
```

```python
sm.Eq((1-theta_t1)*pi*(v_l-b_t)+beta_s*(1-pi)*(mu_t1-b_t),sm.symbols("Pi^sell_t"))
```

From these profits he derives his optimal ask and bid price.

```python
sm.Eq(mu_t1+(pi*theta_t1*(1-theta_t1))/(pi*theta_t1+(1-pi)*beta_b)*(v_h-v_l), a_t)
```

```python
sm.Eq(mu_t1-(pi*(1-theta_t1)*(1-theta_t1))/(pi*theta_t1+(1-pi)*beta_s)*(v_h-v_l),b_t)
```

The ask and bid price equal the dealers expected value of the security with a markup and discount respectively. We denote the markup $s_t^a$ and the discount $s_t^b$

```python
sm.Eq((pi*theta_t1*(1-theta_t1))/(pi*theta_t1+(1-pi)*beta_b)*(v_h-v_l), s_a)
sm.Eq(mu_t1+s_a, a_t)
```

```python
sm.Eq((pi*(1-theta_t1)*(1-theta_t1))/(pi*theta_t1+(1-pi)*beta_s)*(v_h-v_l), s_b)
sm.Eq(mu_t1-s_b, b_t)
```

Depending on the trade order the dealer receives in period t, his beliefs about the value of the security in period t+1 is changed.
$$\theta_t^{+}\equiv P(v=v^H|\Omega_{t-1},d_t=1)$$
$$\theta_t^{-}\equiv P(v=v^L|\Omega_{t-1},d_t=-1)$$

```python
sm.Eq(((1+pi)*beta_b)/(pi*theta_t1+(1-pi)*beta_b),theta_p)
```

```python
sm.Eq(((1-pi)*beta_s)/(pi*(1-theta_t1)+(1-pi)*beta_s),theta_m)
```

With his updated beliefs about the security, the dealer updates his expectation of the value of the security for the period t+1. If the dealer recieved a buy order in in period t, his expectation $\mu$ can be shown to have the following relation
$$\mu^+_{t+1}-\mu_t=s_t^a$$
The dealers expectation for any order is therefore
$$\mu_t=\mu_{t-1}+s(d_t)d_t$$
with $s(d_t)\equiv \begin{cases} s_t^a & d_t=1 \\ s_t^b & d_t=-1 \end{cases}$

## Simulating the model

Solving the model equilibrium is unfortunately not straight forward. The model have local equilibria for every period, which maximize the profit for informed traders and ensure null profit for the dealer. In order to find the model equibrilium, we use the algorithm below.

**How to simulate the Glosten-Milgrom model**
1. choose start values and parameters. Set threshold value $\epsilon$
2. calculate value expectation, then ask/bid price.
3. determine trader type, then order type
4. update dealer beliefs
5. repeat step 2-4 until $\epsilon > spread = a_t - b_t$

The flow chart below shows more detailed steps to simulate our model. We followed the flow of the chart to create the function simulation()

In the code below we define the function simulation, which simulates our model in accordance with flow chart above.

```python
def simulation(distribution=(0,1), decision="v_h", ratio=0.2, uninformed=0.5, startvalue=0.5, iterations = 500, seed=5000, epsilon=10**-5, shockperiod = None, shock={}):
    #define constants
    v_l, v_h = distribution
    pi = ratio
    beta_b = uninformed
    beta_s = 1-beta_b
    shock = shock

    #determine realized value of v
    v = decision

    #allocate space
    values={}
    ratiovalues = []
    iteration = []
    thetavalues = np.empty(iterations)
    muvalues = np.empty(iterations)
    askvalues = np.empty(iterations)
    bidvalues = np.empty(iterations)
    gapvalues = np.empty(iterations)
    pivalues = np.empty(iterations)
    decisionvalues = np.empty(iterations)

    #simulation settings
    thetavalues[0]=startvalue
    theta_t1 = startvalue
    N = iterations
    np.random.seed(seed)

    break_index = 0
    for i in range(N):

        if i==shockperiod:
            if shock != {}:
                if "Public" not in shock:
                    if shock["Private"]==1:
                        v="v_h"
                    if shock["Private"]==0:
                        v="v_l"

                elif "Private" not in shock:
                    v_l, v_h = shock["Public"]
                    v = decision

                else:
                    v_l, v_h = shock["Public"]
                    if shock["Private"]==1:
                        v="v_h"
                    if shock["Private"]==0:
                        v="v_l"

        if v=="v_h":
            v=v_h
        elif v=="v_l":
            v=v_l

        mu_t1 = theta_t1*v_h+(1-theta_t1)*v_l
        muvalues[i] = mu_t1

        #calculate markup/discount
        s_a = (pi*theta_t1*(1-theta_t1))/(pi*theta_t1+(1-pi)*beta_b)*(v_h-v_l)
        s_b = (pi*theta_t1*(1-theta_t1))/(pi*(1-theta_t1)+(1-pi)*beta_s)*(v_h-v_l)

        #calculate ask/bid price
        askvalues[i] = a_t = mu_t1 + s_a
        bidvalues[i] = b_t = mu_t1 - s_b

        #calculate gap
        gapvalues[i] = gap_t = a_t - b_t

        #realize pi
        trader = np.random.binomial(1,pi)
        pivalues[i] = trader
        #if trader is informed
        if trader == 1:
            if v == v_h:
                if v_h>a_t:
                    d_t=1
            elif v == v_l:
                if v_l<b_t:
                    d_t=-1
            else:
                d_t=0

        #if trader is uninformed
        if trader == 0:
            buysell = np.random.binomial(1,beta_b)
            if buysell == 1:
                d_t = 1
            else:
                d_t = -1

        decisionvalues[i] = d_t

        #update theta
        if d_t == 1:
            theta_t = ((1+pi)*beta_b)/(pi*theta_t1+(1-pi)*beta_b)*theta_t1
            theta_t1 = theta_t
        elif d_t == -1:
            theta_t = ((1-pi)*beta_b)/(pi*(1-theta_t1)+(1-pi)*beta_b)*theta_t1
            theta_t1 = theta_t

        if i<iterations-1:
            thetavalues[i+1] = theta_t
        ratiovalues.append(str(ratio))
        iteration.append(int(i))

        #off by one error
        break_index=i+1
        if gap_t<epsilon or i == N-1:
            values.update({"Theta": theta_t,"Bid": b_t, "Ask": (a_t), "Mu": mu_t1, "Equilibrium period": break_index-1})
            break

    dataframe = pd.DataFrame()
    dataframe["Iteration"] = iteration
    dataframe["ratio"] = ratiovalues
    dataframe["theta"] = thetavalues[0:break_index]
    dataframe["mu"] = muvalues[0:break_index]
    dataframe["ask"] = askvalues[0:break_index]
    dataframe["bid"] = bidvalues[0:break_index]
    dataframe["spread"] = gapvalues[0:break_index]
    dataframe["trader"] = pivalues[0:break_index]
    dataframe["order"] = decisionvalues[0:break_index]

    return dataframe, values
```

The simulation function depends on several random distributions in order to simulate our models. Thus, running a single instance of the function will not create credible results. In order to get credible values for our equilibrium we create the function numericalsolution, which loops the simulation function N times with random seeds. The output is the mean values for the N different equilibria.

```python
def numericalsolution(N=1000, ratio=0.15):
    data = pd.DataFrame()

    for i in range(N):
        seed = int(np.random.uniform(0,10000))
        dataframe, values = simulation(distribution=(0,10), ratio=ratio, startvalue=0.5, iterations = 1001, seed=seed, epsilon=5**-5)

        for key in values:
            data.loc[i,key]=values[key]

    meanvalues = {}
    for key in list(data.columns):
        meanvalues[key]=data[key].mean()
    return meanvalues
```

```python
numericalsolution()
```

    {'Theta': 0.9999522089136563,
     'Bid': 9.99937042512405,
     'Ask': 9.999656044732731,
     'Mu': 9.999534654405569,
     'Equilibrium period': 216.922}

with a ratio 0.15 we reach equilibrium after 216 iterations on average. As expected $\theta_t$ converges towards 1, while ask price, bid price and expected value converges towards the true value of the security.

In order to examine the influence of the ratio of traders, we construct a for loop to run numericalsolution with $\pi \in [0.01;1]$.

The code below is quite taxing to run, so we have included a line to prevent you from running it. We have included a data file in our repository containing the data from running the code below, which we use to make the following graph. You can run the code by commenting out import nothing.py.

```python
#To prevent you from accidently running it we have include the line below.
import nothing.py

#make lists
ratiolist = []
periodlist = []

#loop the function numericalsolution
for i in range(99):
    num = ((i+1)/100)
    values = numericalsolution(ratio=num)

    ratiolist.append(num)
    periodlist.append(values["Equilibrium period"])

```

```python
#if you ran the above code, uncomment the three lines below to graph the data
#plt.plot(ratiolist,periodlist)
#plt.xlabel("ratio")
#plt.ylabel("Period")

#alternative:
#read data file
doomloopdata = pd.read_csv("./doomloopdata.csv")
doomloopdata.columns = ["periodlist","ratiolist"]

#plot the data
doomloopdata.plot("ratiolist","periodlist")
plt.xlabel("ratio")
plt.ylabel("Period")
```

The speed of convergence towards the equilibrium is heavily determined by the ratio of informed traders. In our model the dealer gets information about the true value of the security from the trade flow. With more informed traders on the market the amount of information in the trade flow is higher.

## Aspects of the model

#### Seed
In our simulation function we rely on random distributions to simulate our model. This short chapter will demonstrate the randomness of a single iteration of the simulation function. We call our function with alternating seeds but all other settings equal.

```python
#calling simulation function
thetarun1 = simulation(ratio=0.15, iterations=201,seed=4795)
thetarun2 = simulation(ratio=0.15, iterations=201,seed=6279)
thetarun3 = simulation(ratio=0.15, iterations=201,seed=6130)
thetarun4 = simulation(ratio=0.15, iterations=201,seed=9352)
thetarun5 = simulation(ratio=0.15, iterations=201,seed=5059)

#saving return data
thetadata1, thetavalues1 = thetarun1
thetadata2, thetavalues2 = thetarun2
thetadata3, thetavalues3 = thetarun3
thetadata4, thetavalues4 = thetarun4
thetadata5, thetavalues5 = thetarun5

fig = plt.figure(dpi=100)
ax2 = fig.add_subplot(1,1,1)
ax2.plot(thetadata1["theta"], label="1")
ax2.plot(thetadata2["theta"], label="2")
ax2.plot(thetadata3["theta"], label="3")
ax2.plot(thetadata4["theta"], label="4")
ax2.plot(thetadata5["theta"], label="")
ax2.grid(True)
ax2.legend()
ax2.set_title("Theta")
```

As we can see in the graph above, all the simulations converges towards equilibrium. The speed of convergence however is vastly different for the different seeds.

#### ratio of informed traders

We call the simulation for different values of $\pi$ and plot bid price, ask price and expected value $\mu_t$ in our custom interactive graphwindow class. In the graph window you are able move around in the plot and examine the development of the model for different $\pi$ values

```python
#call simulation function 5 times with different ratios
pirun1 = simulation(ratio=0.10, iterations=1001, seed=404)
pirun2 = simulation(ratio=0.15, iterations=1001, seed=404)
pirun3 = simulation(ratio=0.25, iterations=1001, seed=404)
pirun4 = simulation(ratio=0.5, iterations=201, seed=404)
pirun5 = simulation(ratio=0.9, iterations=201, seed=404)

#save data
pirundata1, pirunvalues1 = pirun1
pirundata2, pirunvalues2 = pirun2
pirundata3, pirunvalues3 = pirun3
pirundata4, pirunvalues4 = pirun4
pirundata5, pirunvalues5 = pirun5

#merge data
pirunmerged = pd.concat([pirundata1, pirundata2, pirundata3, pirundata4, pirundata5])

#call custom graphwindow class with graphtype=piplot
pirungraphwindow = PlotterWindow(data = pirunmerged, slicename = "ratio", xvariable = "Iteration", yvariablelist = ["mu","ask","bid"], graphtype ="piplot")
pirungraphwindow.start()
```

The code below opens another instance of our graphwindow. The plots are candlestick plots, which maps the difference in ask and bid price.

```python
#call custom graphwindow class with graphtype=candlestick
pirungraphwindow2 = PlotterWindow(data = pirunmerged, slicename = "ratio", graphtype = "candlestick")
pirungraphwindow2.start()
```

    C:\Users\Corfixen\Anaconda3\lib\site-packages\matplotlib\figure.py:98: MatplotlibDeprecationWarning:
    Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
      "Adding an axes using the same arguments as a previous axes "

As seen in both the piplot and candlestick graphwindow, the initial spread between ask and bid prices heavily depend on the ratio $\pi$. Higher values of $\pi$ leads to higher initial spread. The dealer have to compensate his prices for the higher amount of orders from informed traders to ensure the null profit

## Shocks to the model

Simply simulating the model is not as interesting. In this chapter we introduce several types of shocks to the market and graph the result. We would have preferred to run this chapter with a modified numericalsolution function to get better results. We settled on the simulation function due to the lower amount of computational power needed. As such the results are not as credible as they could be.

We simulate our model with three different types of shocks.
1. The first shock is a shock in private information. We change the realisation of the variable v, which only the informed traders know the true value of. A real-life example of this type of shock would be a trader recieving insider info from a firm.
2. The second shock is a public information shock. We change the upper and lower value for v. This information is available to all on the market. A real-life example of this type of shock would be the release of the annual report from a firm
3. We shock the model with both private and public information.

all of the shocks incur in the 90th period. To avoid naming conflict in our graphwindow we set a slighty different ratio for the different calls. The difference on the results are negligible

```python
#call simulation function with different shock types
shockrun1 = simulation(ratio=0.15001, iterations=1001, seed=404, shockperiod = 90, shock={"Private":0})
shockrun2 = simulation(ratio=0.15002, iterations=1001, seed=404, shockperiod = 90, shock={"Public":(0.3, 0.8)})
shockrun3 = simulation(ratio=0.15003, iterations=1001, seed=404, shockperiod = 90, shock={"Private":0, "Public":(0.3,0.8)})

#save results
shockdata1, shockvalues1 = shockrun1
shockdata2, shockvalues2 = shockrun2
shockdata3, shockvalues3 = shockrun3

#merge results
shockmerged = pd.concat([shockdata1,shockdata2,shockdata3])

#call custom graphwindow class with graphtype = piplot
shockgraphwindow = PlotterWindow(data = shockmerged, slicename = "ratio", xvariable = "Iteration", yvariablelist = ["mu","ask","bid"], graphtype="piplot")
shockgraphwindow.start()
```

all graph converges towards the same equilibrium before the shock.
1. After the shock the model slowly converges towards the new equilibrium, which is at $a_t, b_t, \mu_t \rightarrow v^L$. The dealer slowly receives information about the new equilibrium from the informed traders behavior. He adjusts his ask and bid price to make up for the new informed trader strategy.
2. The change in the prices happens instantly when the new information is made available. This behavior on the market suggests our model is semi-strong form efficient, since the market instantly adepts to the new public information.
3. With both shock types we see the instant respons to the public information. The reaction to the private information shock is rather slow, as with the first graph.

# Conclusion

In this paper we found the Gloston-milgrom model capable of adapting to public and private information. The model simulated the behavior of several types of traders and a market maker (the dealer), and showed their adaptions to the information avaliable. The model further shows this adaptation of information by the equilibrium prices converging on the true value of the security

The ratio of informed traders $\pi$ on the market have a huge impact on the speed of convergence and spread of ask and bid prices.

From the shocks we introduce to the model we conclude the model is semi-strong form efficient.


Construir o gráfico e encontrar o foco e uma equação da diretriz.

5. $y^2 - x = 0$

Arrumando a equação

$y^2 = x$

$2p = 1$,logo

$p = \frac{1}{2}$

Calculando o foco

$F = \frac{p}{2}$

$F = \frac{\frac{1}{2}}{2}$

$F = \frac{1}{2}\cdot \frac{1}{2}$

$F = \frac{1}{4}$

$F(\frac{1}{4},0)$

Calculando a diretriz

$d = -\frac{p}{2}$

$d = -(\frac{1}{4})$

$d : x = -\frac{1}{4}$

$V(0,0)$

$F(\frac{1}{4},0)$

Gráfico da parábola

```python
from sympy import *
from sympy.plotting import plot_implicit
x, y = symbols("x y")
plot_implicit(Eq((y-0)**2, 1*(x+0)), (x,-3,3), (y,-3,3),
title=u'Gráfico da parábola', xlabel='x', ylabel='y');
```

Problemas Propostos. Pag. 172 - 175/05.ipynb


# Critical Damping Regulator Design for Oscillator-like Systems
In this tutorial, we will introduce the `kontrol.regulator.feedback` module, and use `kontrol.regulator.feedback.critical_damping()` and `kontrol.regulator.feedback.proportional_derivative()` functions to algorithmically generate feedback controller that critically damps the system.

For critical damping of a oscillator, derivative controllers, i.e.
\begin{equation}
K_\mathrm{derivative}(s) = K_d s\,
\end{equation}
where $K_d$ is a the derivative gain, is sufficient for making a critically damped system.

Consider the damped-oscillator plant
\begin{equation}
P(s) = \frac{\omega_n^2}{s^2 + \frac{\omega_n}{Q} s + \omega_n^2}\,
\end{equation}
where $\omega_n$ is the resonance frequency (rad/s) of the oscillator, and $Q$ is the quality factor / Q-factor of the oscillator.

It's easy to show that if we would like to actively damp the oscillator to some desired Q value $Q_\mathrm{desired}$, then the corresponding required derivative gain is exactly
\begin{equation}
K_d = \frac{\frac{1}{Q_\mathrm{desired}}-\frac{1}{Q}}{\omega_n \lvert P(0)\rvert}\,,
\end{equation}
for $Q_\mathrm{desired} \geq 0.5$.

`kontrol.regulator.feedback.critical_damp()` has argument `method` with options `"calculated"` and `"optimized"`.

For critical damping, the desired Q value is $Q_\mathrm{crit}=0.5$.
Furthermore, if we assume that the the Q value of the system is much greater than 0.5, then the critical damping gain can simply be approximated by
\begin{equation}
K_d = K_\mathrm{crit} = \frac{2}{\omega_n \lvert P(0)\rvert}\,,
\end{equation}
which is extremely easy to compute.

For coupled oscillators systems with more than one modes, the critical derivative gain can't be calculated easily, and we cannot achieve critical damping for all modes using only simple derivative damping.
If we only aim to critically damp the dominant mode, then the critical gain approximation is still valid, as we shall see later.

To find the the exact critical derivative gain, the `method="optimized"` is a better approach. It algorithically finds the derivative gain that makes the closed-loop system has one less complex pole pairs. And it finds the gain such that the complex pole pairs just becomes two simple poles, i.e. critically damped.

In this tutorial, we will design (algorithmically) the control system to damp a coupled oscillator system with two modes, i.e.

\begin{equation}
P(s) = G_1\frac{\omega_1^2}{s^2 + \omega_1/Q_1 s + \omega_1^2} + G_2\frac{\omega_2^2}{s^2 + \omega_2/Q_2 s + \omega_2^2}\,,
\end{equation}

and we will compaire the open-loop plant with the closed-loop plant.
\begin{equation}
\frac{P(s)}{1+K(s)P(s)}\,\,
\end{equation}
which is the transfer function from external disturbance to displacement.

```python
import numpy as np
import control
import matplotlib.pyplot as plt
import kontrol.regulator

s = control.tf("s")
g1 = 100
w1 = 1
q1 = 100
g2 = 10
w2 = 3
q2 = 50

plant = g1*w1**2 / (s**2 + w1/q1*s + w1**2) + g2*w2**2 / (s**2 + w2/q2*s + w2**2)

f = np.logspace(-2, 1, 1000)
plt.figure(figsize=(10, 7))
plt.subplot(211, title="Bode plot")
plt.loglog(f, abs(plant(1j*2*np.pi*f)), label="Plant")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")

plt.subplot(212)
plt.semilogx(f, np.angle(plant(1j*2*np.pi*f)), label="Plant")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")
```

```python
plant
```

$$\frac{190 s^2 + 6.9 s + 990}{s^4 + 0.07 s^3 + 10 s^2 + 0.15 s + 9}$$

Here, we will make two feedback regulators from `kontrol.regulator.feedback.critical_damping()`, one for `method="calculated"`, another one for `method="optimized"`.

```python
k = kontrol.regulator.feedback.critical_damping(plant)
k
```

$$\frac{0.01671 s}{1}$$

```python
k_calculated = kontrol.regulator.feedback.critical_damping(plant, method="calculated")
k_optimized = kontrol.regulator.feedback.critical_damping(plant)  # "optimized" is the default method
print("k_calculated: ", k_calculated)
print("k_optimized: ", k_optimized)
```

    k_calculated:
    0.01818 s
    ---------
        1

    k_optimized:
    0.01671 s
    ---------
        1

As can be seen, the approximated critical gain is very close to the optimized one.

Now, we check the open-loop transfer functions.

```python
oltf_calculated = k_calculated * plant
oltf_optimized = k_optimized * plant

plt.figure(figsize=(10, 7))
plt.subplot(211, title="Bode plot")
plt.loglog(f, abs(oltf_calculated(1j*2*np.pi*f)), label="OLTF with calculated derivative controller")
plt.loglog(f, abs(oltf_optimized(1j*2*np.pi*f)), label="OLTF with optimized derivative controller")
plt.hlines(1, min(f), max(f), color="k", ls="--", label="Unity gain")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")

plt.subplot(212)
plt.semilogx(f, np.angle(oltf_calculated(1j*2*np.pi*f)), label="OLTF with calculated derivative controller")
plt.semilogx(f, np.angle(oltf_optimized(1j*2*np.pi*f)), label="OLTF with optimized derivative controller")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")
```

From now one we will consider the optimized version only.

Let's compare the open-loop and closed-loop frequency response.

```python
closed_loop = plant / (1+oltf_optimized)
k_overdamp = 2 * k_optimized
k_underdamp = 1/2 * k_optimized

closed_loop_overdamp = plant / (1 + k_overdamp*plant)
closed_loop_underdamp = plant / (1 + k_underdamp*plant)

plt.figure(figsize=(10, 7))
plt.subplot(211, title="Bode plot")
plt.loglog(f, abs(plant(1j*2*np.pi*f)), label="Open loop")
plt.loglog(f, abs(closed_loop_overdamp(1j*2*np.pi*f)), label="Closed loop, overdamping")
plt.loglog(f, abs(closed_loop_underdamp(1j*2*np.pi*f)), label="Closed loop, underdamping")
plt.loglog(f, abs(closed_loop(1j*2*np.pi*f)), label="Closed loop, critical damping")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")

plt.subplot(212)
plt.semilogx(f, np.angle(plant(1j*2*np.pi*f)), label="Open loop")
plt.semilogx(f, np.angle(closed_loop_overdamp(1j*2*np.pi*f)), label="Closed loop, overdamping")
plt.semilogx(f, np.angle(closed_loop_underdamp(1j*2*np.pi*f)), label="Closed loop, underdamping")
plt.semilogx(f, np.angle(closed_loop(1j*2*np.pi*f)), label="Close loop")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")
```

Let's show the root locus from 0 to `k_optimized` and you shall see how it works.

```python
k_max = k_optimized.num[0][0][0]
for k in np.linspace(0, k_max, 10):
    plt.plot((plant/(1+k*s*plant)).minreal().pole().real, (plant/(1+k*s*plant)).minreal().pole().imag, "kx")
```

Here we compare the poles as well to show that one of the modes is critically damped.
(Look at the last two poles)

```python
plant.pole()
```

    array([-0.03 +2.99985j  , -0.03 -2.99985j  , -0.005+0.9999875j,
           -0.005-0.9999875j])

```python
closed_loop.minreal().pole()
```

    array([-0.45683853+2.5338248j, -0.45683853-2.5338248j,
           -1.16528417+0.j       , -1.16510385+0.j       ])

Now let's show the step-response.
We will also compare the cases that are slightly off critical gain.

```python
t = np.linspace(0, 32, 1024)

_, step_open_loop = control.step_response(plant, T=t)
_, step_closed_loop_overdamp = control.step_response(closed_loop_overdamp, T=t)
_, step_closed_loop_underdamp = control.step_response(closed_loop_underdamp, T=t)

_, step_closed_loop = control.step_response(closed_loop, T=t)

plt.title("Step response")
plt.plot(t, step_open_loop, label="Open loop")
plt.plot(t, step_closed_loop_overdamp, label="Closed loop, overdamping")
plt.plot(t, step_closed_loop_underdamp, "--", label="Closed loop, underdamping")
plt.plot(t, step_closed_loop, label="Closed loop, critical damping")
plt.legend(loc=0)
plt.xlabel("Time (s)")
plt.show()
```


# Roll hull equation

```python
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
import sympy as sp
from sympy.plotting import plot as plot
from sympy.plotting import plot3d as plot3d
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
sp.init_printing()
from IPython.core.display import HTML
```

```python
import seaman.helpers
import seaman_symbol as ss
import roll_hull_equations as equations
import roll_hull_lambda_functions as lambda_functions
from bis_system import BisSystem
```

## Coordinate system

## Symbols

```python
from seaman_symbols import *
```

```python
HTML(ss.create_html_table(symbols=equations.roll_hull_equation.free_symbols))
```

## Roll equation

```python
equations.roll_hull_equation
```

```python
equations.roll_hull_equation_SI
```

### Plotting the foll hull moment equation

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-0.3,3,10)
df['u_w'] = 5.0
df['p'] = 0
df['r_w'] = 0.0
df['rho'] = 1025
df['L'] = 1.0
df['g'] = 9.81
df['disp'] = 23
df['K_ur'] = 1
df['K_uur'] = 1
df['K_uv'] = 1
df['K_uuv'] = 1
df['K_up'] = 1
df['K_p'] = 1
df['K_vav'] = 1
df['K_rar'] = 1
df['K_pap'] = 1

result = df.copy()
lambda_functions.K_h_function(**df)
result['mx'] = lambda_functions.K_h_function(**df)

result.plot(x = 'v_w',y = 'mx');
```

### Plotting with coefficients from a real seaman ship model

```python
import generate_input
shipdict = seaman.ShipDict.load('../../tests/test_ship.ship')
```

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-3,3,20)
df['rho'] = 1025.0
df['g'] = 9.81
df['u_w'] = 5.0
df['r_w'] = 0.0
df['p'] = 0

df_input = generate_input.add_shipdict_inputs(lambda_function=lambda_functions.K_h_function,
                                              shipdict = shipdict,
                                              df = df)
df_input
```

```python
result = df_input.copy()
result['mx'] = lambda_functions.K_h_function(**df_input)
```

```python
result.plot(x = 'v_w',y = 'mx');
```

## Real seaman++
Run real seaman in C++ to verify that the documented model is correct.

```python
import run_real_seaman
```

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-3,3,20)
df['rho'] = 1025.0
df['g'] = 9.81
df['u_w'] = 5.0
df['r_w'] = 0.0
df['p'] = 0

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.K_h_function,
                                                        shipdict = shipdict,
                                                        df = df,label = 'mx')

fig,ax = plt.subplots()
result_comparison.plot(x = 'v_w',y = ['mx','mx_seaman'],ax = ax)
ax.set_title('Drift angle variation');
```

```python
df = pd.DataFrame()
df['r_w'] = np.linspace(-0.3,0.3,20)
df['rho'] = 1025.0
df['g'] = 9.81
df['u_w'] = 5.0
df['v_w'] = 0.0
df['p'] = 0

shipdict2 = shipdict.copy()
shipdict2.lin_hull_coeff_data['kur'] = 0
#shipdict2.lin_hull_coeff_data['kuur'] = 0

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.K_h_function,
                                                        shipdict = shipdict2,
                                                        df = df,label = 'mx')
fig,ax = plt.subplots()
result_comparison.plot(x = 'r_w',y = ['mx','mx_seaman'],ax = ax)
ax.set_title('Yaw rate variation');
```

```python
np.sqrt(df_input['L'])/2
```

```python

```


```python
from sympy.physics.optics import RayTransferMatrix as RTM, FreeSpace as FS, ThinLens as TL
from sympy import symbols, Eq, init_printing, S, sqrt
init_printing()
import numpy as np
import matplotlib.pyplot as plt
from scipy.constants import c
```

```python
# Defining functions
ghz_cm = lambda x: c * 1e2 / x * 1e-9
cm_ghz = lambda x: c*1e-9*1e2/x
```

```python
d_out, d_in, w0_in, w0_out = symbols("d_out, d_in, w0_in, w0_out")
w, lam, w0, z, Z_r = symbols("w, lam, w0, z, Z_r")
M, E, f, f0 = symbols('M, E, f, f0')
f_p, f_m = symbols("f_p, f_m")
f_e, R1, R2 = symbols("f_e, R1, R2")
```

```python
# Beam radius
Wz = Eq(w, w0 * sqrt(1 + (z / Z_r) ** 2))
Wz
```

```python
# Input distance
eq_din_m = Eq(d_in, f - sqrt(f ** 2 - f0 ** 2) / M) # Negative sign
eq_din_p = Eq(d_in, f + sqrt(f ** 2 - f0 ** 2) / M) # Positive sign

# output distance
eq_dout_m = Eq(d_out, f - sqrt(f ** 2 - f0 ** 2) * M) # Negative sign
eq_dout_p = Eq(d_out, f + sqrt(f ** 2 - f0 ** 2) * M) # Positive sign
# NOTE: use same sign for output distance as chosed for input distane

eq_din_m, eq_din_p, eq_dout_m, eq_dout_p
```

```python
# Focal length
E = 1-M**-2
fp = Eq(f_p, Z_r*(d_in/(E*Z_r))*(1+sqrt(1-E*(1+(d_in/Z_r)**-2)))) # positive sign but output is negative focus
fm = Eq(f_m, Z_r*(d_in/(E*Z_r))*(1-sqrt(1-E*(1+(d_in/Z_r)**-2)))) # negative sign but output is positive focus

fp, fm
```

```python
magnification = 0.15/0.34

w0_ = 0.34
confocalLength = np.pi*w0_**2/ghz_cm(325)
d1 = 20

f1 = float(fp.subs({M:magnification, Z_r:confocalLength, d_in:d1}).rhs)
f2 = float(fm.subs({M:magnification, Z_r:confocalLength, d_in:d1}).rhs)

f1, f2
```

```python
fe = Eq(f_e, (R1*R2)/(R1+R2))
fe
```

```python
M_ellps = TL(fe.rhs)
M_ellps
```

```python
dout_direct = (
    (M_ellps.A * d_in + M_ellps.B) * (M_ellps.C * d_in + M_ellps.D)
    + (M_ellps.A * M_ellps.C * Z_r)
) / ((M_ellps.C * d_in + M_ellps.D) ** 2 + (M_ellps.C * Z_r) ** 2)

dout_direct
```

```python
dout_direct.subs({R1:f1, R2:f2, d_in:d1, Z_r:confocalLength})
```

```python
wout_direct = w0_in/((M_ellps.C * d_in + M_ellps.D) ** 2 + (M_ellps.C * Z_r) ** 2)
wout_direct
```

```python
wout_direct.subs({R1:f1, R2:f2, d_in:d1, Z_r:confocalLength, w0_in:w0_})
```

```python

```

```python

```

```python

```

```python

```

```python

```


# Canonical Correlation Analysis (CCA)

Example is taken from *Section 12.5.3, Machine Learning: A Probabilistic Perspective by Kevin Murphy.*

```python
from symgp import *
from sympy import *

from IPython.display import display, Math, Latex
```

## Set up shapes, variables and constants

We have two observed variables `x` and `y` of shapes `(D_x,1)` and `(D_y,1)` and the latent variables `z_s`, `z_x`, `z_y` of shapes `(L_o,1)`, `(L_x,1)` and `(L_y,1)`.

```python
# Shapes
D_x, D_y, L_o, L_x, L_y = symbols('D_x, D_y, L_o L_x L_y')

# Variables
x, y, z_s, z_x, z_y = utils.variables('x y z_{s} z_{x} z_{y}', [D_x, D_y, L_o, L_x, L_y])

# Constants
B_x, W_x, mu_x, B_y, W_y, mu_y = utils.constants('B_{x} W_{x} mu_{x} B_{y} W_{y} mu_{y}',
                                                 [(D_x,L_x), (D_x,L_o), D_x, (D_y,L_y), (D_y,L_o), D_y])
sig = symbols('\u03c3')  # Noise standard deviation
```

## Define the model

```python
# p(z_s), p(z_x), p(z_y)
p_zs = MVG([z_s],mean=ZeroMatrix(L_o,1),cov=Identity(L_o))
p_zx = MVG([z_x],mean=ZeroMatrix(L_x,1),cov=Identity(L_x))
p_zy = MVG([z_y],mean=ZeroMatrix(L_y,1),cov=Identity(L_y))

display(Latex(utils.matLatex(p_zs)))
display(Latex(utils.matLatex(p_zx)))
display(Latex(utils.matLatex(p_zx)))
```

\begin{align*}
p\left(\mathbf{z_{s}}\right)&= \mathcal{N}\left(\mathbf{z_{s}};\mathbf{m}_{\mathbf{z_{s}}},\mathbf{\Sigma}_{\mathbf{z_{s}}}\right)\\
\mathbf{m}_{\mathbf{z_{s}}} &= \mathbf{0}\\
\mathbf{\Sigma}_{\mathbf{z_{s}}} &= \mathbf{I}\\
\end{align*}

\begin{align*}
p\left(\mathbf{z_{x}}\right)&= \mathcal{N}\left(\mathbf{z_{x}};\mathbf{m}_{\mathbf{z_{x}}},\mathbf{\Sigma}_{\mathbf{z_{x}}}\right)\\
\mathbf{m}_{\mathbf{z_{x}}} &= \mathbf{0}\\
\mathbf{\Sigma}_{\mathbf{z_{x}}} &= \mathbf{I}\\
\end{align*}

\begin{align*}
p\left(\mathbf{z_{x}}\right)&= \mathcal{N}\left(\mathbf{z_{x}};\mathbf{m}_{\mathbf{z_{x}}},\mathbf{\Sigma}_{\mathbf{z_{x}}}\right)\\
\mathbf{m}_{\mathbf{z_{x}}} &= \mathbf{0}\\
\mathbf{\Sigma}_{\mathbf{z_{x}}} &= \mathbf{I}\\
\end{align*}

```python
# p(z)
p_z = p_zs*p_zx*p_zy

display(Latex(utils.matLatex(p_z)))
```

\begin{align*}
p\left(\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}\right)&= \mathcal{N}\left(\left[\begin{smallmatrix}\mathbf{z_{s}}\\\mathbf{z_{x}}\\\mathbf{z_{y}}\end{smallmatrix}\right];\mathbf{m}_{\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}},\mathbf{\Sigma}_{\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}}\right)\\
\mathbf{m}_{\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\mathbf{0}\\\mathbf{0}\\\mathbf{0}\end{smallmatrix}\right]\\
\mathbf{\Sigma}_{\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\mathbf{I}&\mathbf{0}&\mathbf{0}\\\mathbf{0}&\mathbf{I}&\mathbf{0}\\\mathbf{0}&\mathbf{0}&\mathbf{I}\end{smallmatrix}\right]\\
\end{align*}

```python
# p(x|z)
p_x_g_z = MVG([x],mean=B_x*z_x + W_x*z_s + mu_x,cov=sig**2*Identity(D_x),cond_vars=[z_x,z_s])

display(Latex(utils.matLatex(p_x_g_z)))
```

\begin{align*}
p\left(\mathbf{x}|\mathbf{z_{x}},\mathbf{z_{s}}\right)&= \mathcal{N}\left(\mathbf{x};\mathbf{m}_{\mathbf{x}|\mathbf{z_{x}},\mathbf{z_{s}}},\mathbf{\Sigma}_{\mathbf{x}|\mathbf{z_{x}},\mathbf{z_{s}}}\right)\\
\mathbf{m}_{\mathbf{x}|\mathbf{z_{x}},\mathbf{z_{s}}} &= \mathbf{mu_{x}} + \mathbf{B_{x}} \mathbf{z_{x}} + \mathbf{W_{x}} \mathbf{z_{s}}\\
\mathbf{\Sigma}_{\mathbf{x}|\mathbf{z_{x}},\mathbf{z_{s}}} &= \sigma^{2} \mathbf{I}\\
\end{align*}

```python
# p(y|z)
p_y_g_z = MVG([y],mean=B_y*z_y + W_y*z_s + mu_y,cov=sig**2*Identity(D_y),cond_vars=[z_y,z_s])

display(Latex(utils.matLatex(p_y_g_z)))
```

\begin{align*}
p\left(\mathbf{y}|\mathbf{z_{y}},\mathbf{z_{s}}\right)&= \mathcal{N}\left(\mathbf{y};\mathbf{m}_{\mathbf{y}|\mathbf{z_{y}},\mathbf{z_{s}}},\mathbf{\Sigma}_{\mathbf{y}|\mathbf{z_{y}},\mathbf{z_{s}}}\right)\\
\mathbf{m}_{\mathbf{y}|\mathbf{z_{y}},\mathbf{z_{s}}} &= \mathbf{mu_{y}} + \mathbf{B_{y}} \mathbf{z_{y}} + \mathbf{W_{y}} \mathbf{z_{s}}\\
\mathbf{\Sigma}_{\mathbf{y}|\mathbf{z_{y}},\mathbf{z_{s}}} &= \sigma^{2} \mathbf{I}\\
\end{align*}

## Obtain joint distribution p(x,y)

```python
# p(v|z) (p(x,y|z_s,z_x,z_y))   We denote v = (x;y) and z = (z_s;z_x;z_y)
p_v_g_z = p_x_g_z*p_y_g_z

display(Latex(utils.matLatex(p_v_g_z)))
```

\begin{align*}
p\left(\mathbf{x},\mathbf{y}|\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}\right)&= \mathcal{N}\left(\left[\begin{smallmatrix}\mathbf{x}\\\mathbf{y}\end{smallmatrix}\right];\mathbf{m}_{\mathbf{x},\mathbf{y}|\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}},\mathbf{\Sigma}_{\mathbf{x},\mathbf{y}|\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}}\right)\\
\mathbf{m}_{\mathbf{x},\mathbf{y}|\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\mathbf{mu_{x}} + \mathbf{B_{x}} \mathbf{z_{x}} + \mathbf{W_{x}} \mathbf{z_{s}}\\\mathbf{mu_{y}} + \mathbf{B_{y}} \mathbf{z_{y}} + \mathbf{W_{y}} \mathbf{z_{s}}\end{smallmatrix}\right]\\
\mathbf{\Sigma}_{\mathbf{x},\mathbf{y}|\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\sigma^{2} \mathbf{I}&\mathbf{0}\\\mathbf{0}&\sigma^{2} \mathbf{I}\end{smallmatrix}\right]\\
\end{align*}

```python
# p(v,z) (p(x,y,z_s,z_x,z_y))
p_v_z = p_v_g_z*p_z

display(Latex(utils.matLatex(p_v_z)))
```

\begin{align*}
p\left(\mathbf{x},\mathbf{y},\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}\right)&= \mathcal{N}\left(\left[\begin{smallmatrix}\mathbf{x}\\\mathbf{y}\\\mathbf{z_{s}}\\\mathbf{z_{x}}\\\mathbf{z_{y}}\end{smallmatrix}\right];\mathbf{m}_{\mathbf{x},\mathbf{y},\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}},\mathbf{\Sigma}_{\mathbf{x},\mathbf{y},\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}}\right)\\
\mathbf{m}_{\mathbf{x},\mathbf{y},\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\mathbf{mu_{x}}\\\mathbf{mu_{y}}\\\mathbf{0}\\\mathbf{0}\\\mathbf{0}\end{smallmatrix}\right]\\
\mathbf{\Sigma}_{\mathbf{x},\mathbf{y},\mathbf{z_{s}},\mathbf{z_{x}},\mathbf{z_{y}}} &= \left[\begin{smallmatrix}\sigma^{2} \mathbf{I} + \mathbf{B_{x}} \mathbf{B_{x}}^T + \mathbf{W_{x}} \mathbf{W_{x}}^T&\mathbf{W_{x}} \mathbf{W_{y}}^T&\mathbf{W_{x}}&\mathbf{B_{x}}&\mathbf{0}\\\mathbf{W_{y}} \mathbf{W_{x}}^T&\sigma^{2} \mathbf{I} + \mathbf{B_{y}} \mathbf{B_{y}}^T + \mathbf{W_{y}} \mathbf{W_{y}}^T&\mathbf{W_{y}}&\mathbf{0}&\mathbf{B_{y}}\\\mathbf{W_{x}}^T&\mathbf{W_{y}}^T&\mathbf{I}&\mathbf{0}&\mathbf{0}\\\mathbf{B_{x}}^T&\mathbf{0}&\mathbf{0}&\mathbf{I}&\mathbf{0}\\\mathbf{0}&\mathbf{B_{y}}^T&\mathbf{0}&\mathbf{0}&\mathbf{I}\end{smallmatrix}\right]\\
\end{align*}

```python
# p(v) (p(x,y))
p_v = p_v_z.marginalise([z_s,z_x,z_y])

display(Latex(utils.matLatex(p_v)))
```

\begin{align*}
p\left(\mathbf{x},\mathbf{y}\right)&= \mathcal{N}\left(\left[\begin{smallmatrix}\mathbf{x}\\\mathbf{y}\end{smallmatrix}\right];\mathbf{m}_{\mathbf{x},\mathbf{y}},\mathbf{\Sigma}_{\mathbf{x},\mathbf{y}}\right)\\
\mathbf{m}_{\mathbf{x},\mathbf{y}} &= \left[\begin{smallmatrix}\mathbf{mu_{x}}\\\mathbf{mu_{y}}\end{smallmatrix}\right]\\
\mathbf{\Sigma}_{\mathbf{x},\mathbf{y}} &= \left[\begin{smallmatrix}\sigma^{2} \mathbf{I} + \mathbf{B_{x}} \mathbf{B_{x}}^T + \mathbf{W_{x}} \mathbf{W_{x}}^T&\mathbf{W_{x}} \mathbf{W_{y}}^T\\\mathbf{W_{y}} \mathbf{W_{x}}^T&\sigma^{2} \mathbf{I} + \mathbf{B_{y}} \mathbf{B_{y}}^T + \mathbf{W_{y}} \mathbf{W_{y}}^T\end{smallmatrix}\right]\\
\end{align*}


# Riemann Problems

The code solves Riemann Problems for the relativistic Euler equations

$$
  \partial_t \begin{pmatrix} D \\ S_x \\ S_t \\ \tau \end{pmatrix} + \partial_x \begin{pmatrix} S_x \\ S_x v_x + p \\ S_t v_x \\ (\tau + p) v_x \end{pmatrix} = 0.
$$

For further details on this system of equations, see the [Living Review of Martí and Müller](http://computastrophys.livingreviews.org/Articles/lrca-2015-3/), particularly [section 3.1](http://computastrophys.livingreviews.org/Articles/lrca-2015-3/articlese3.html#x6-190003.1) for the equations and [section 8.5](http://computastrophys.livingreviews.org/Articles/lrca-2015-3/articlese8.html#x11-1150008.5) for the solution of the Riemann Problem.

The initial data is piecewise constant: two states $w_{L, R}$ are specified, each in terms of $w = (\rho_0, v_x, v_t, \epsilon)$, (the specific rest mass density, normal ($x$) and tangential ($t$) velocity components, and the specific internal energy). At $t=0$ the data is set by $w_L$ for $x<0$ and $w_R$ for $x>0$. Each state has associated with it an equation of state (EOS) to close the set of equations: the EOS does not need to be the same for each state.

## Code

To set up a Riemann problem, first set up a left and right state. Each state has its own equation of state. Here we use the first test from the [Test Bench section](http://computastrophys.livingreviews.org/Articles/lrca-2015-3/articlese6.html#x9-690006.3) of the [Living Review](http://computastrophys.livingreviews.org/Articles/lrca-2015-3/):

```python
from r3d2 import eos_defns, State, RiemannProblem
```

```python
eos = eos_defns.eos_gamma_law(5.0/3.0)
test_1_U_left = State(10.0, 0.0, 0.0, 2.0, eos, label="L")
test_1_U_right = State(1.0, 0.0, 0.0, 1.5e-6, eos, label="R")
test_1_rp = RiemannProblem(test_1_U_left, test_1_U_right)
```

The Riemann Problem will produce output directly in the notebook:

```python
test_1_rp
```

This output should be interpreted as follows. First it displays the initial states ${\bf U}_{L, R}$. It then gives the resulting wave pattern: in this case a left going rarefaction, a contact, and a right going shock. It then gives the pressure $p_*$ in the central states (the pressure and normal velocity do not jump across the contact). Next it gives the speeds of each wave. Finally, it gives the constant states between the left wave and the contact ${\bf U}_{*_L}$ and between the contact and the right state ${\bf U}_{*_R}$.

We can also display the output graphically in the notebook:

```python
from IPython.display import display_png
```

```python
display_png(test_1_rp)
```

The first (top left) plot is the wave pattern in the $(x, t)$ plane. Rarefactions are shaded with thin solid lines. Contacts are dashed lines. Discontinuities such as shocks are thick solid lines.

All the other plots show a single quantity as a function of space at a fixed time, or equivalently as a function of the characteristic variable $\xi = x / t$.

### Changing equation of state

There is no need for the two states to have the same equation of state. For example:

```python
eos_air = eos_defns.eos_gamma_law(1.4)
U_vary_eos_L = State(1.0, -0.5, 0.0, 2.0, eos, label="L")
U_vary_eos_R = State(1.0, +0.5, 0.0, 2.0, eos_air, label="R")
test_vary_eos_rp = RiemannProblem(U_vary_eos_L, U_vary_eos_R)
test_vary_eos_rp
```

```python
display_png(test_vary_eos_rp)
```

### Reactive problems

To include a reaction, make the equation of state of at least one of the states be a reactive EOS:

```python
q_unburnt = 0.1
gamma = 5/3
Cv = 1.0
t_ignition = 2
eos_burnt = eos_defns.eos_gamma_law(gamma)
eos_unburnt = eos_defns.eos_gamma_law_react(gamma, q_unburnt, Cv, t_ignition, eos_burnt)
U_reactive_left = State(1, 0, 0, 2, eos_burnt)
U_reactive_right = State(10, 0, 0, 3, eos_unburnt)
test_reactive_rp = RiemannProblem(U_reactive_left, U_reactive_right)
test_reactive_rp
```

```python
display_png(test_reactive_rp)
```


# Autograd

PyTorch's *Autograd* feature is part of what make PyTorch flexible and fast for building machine learning projects. It allows for the rapid and easy computation of multiple partial derivatives (also referred to as *gradients)* over a complex computation. This operation is central to backpropagation-based neural network learning.

The power of autograd comes from the fact that it traces your computation dynamically *at runtime,* meaning that if your model has decision branches, or loops whose lengths are not known until runtime, the computation will still be traced correctly, and you'll get correct gradients to drive learning. This, combined with the fact that your models are built in Python, offers far more flexibility than frameworks that rely on static analysis of a more rigidly-structured model for computing gradients.

## What Do We Need Autograd For?

A machine learning model is a *function*, with inputs and outputs. For this discussion, we'll treat the inputs a as an *i*-dimensional vector $\vec x$, with elements $x_{i}$. We can then express the model, *M*, as a vector-valued function of the input: *$\vec y$ = $\vec M$($\vec x$)*. (We treat the value of M's output as a vector because in general, a model may have any number of outputs.)

Since we'll mostly be discussing autograd in the context of training, our output of interest will be the model's loss. The *loss function* *L($\vec y$) = L($\vec M$($\vec x$))* is a single-valued scalar function of the model's output. This function expresses how far off our model's prediction was from a particular input's *ideal* output. *Note: After this point, we will often omit the vector sign where it should be contextually clear - e.g., $y$ instead of $\vec y$.*

In training a model, we want to minimize the loss. In the idealized case of a perfect model, that means adjusting its learning weights - that is, the adjustable parameters of the function - such that loss is zero for all inputs. In the real world, it means an iterative process of nudging the learning weights until we see that we get a tolerable loss for a wide variety of inputs.

How do we decide how far and in which direction to nudge the weights? We want to *minimize* the loss, which means making its first derivative with respect to the input equal to 0: $\frac{\partial L}{\partial x} = 0$.

Recall, though, that the loss is not *directly* derived from the input, but a function of the model's output (which is a function of the input directly), $\frac{\partial L}{\partial x}$ = $\frac{\partial {L({\vec y})}}{\partial x}$. By the chain rule of differential calculus, we have $\frac{\partial {L({\vec y})}}{\partial x}$ = $\frac{\partial L}{\partial y}\frac{\partial y}{\partial x}$ = $\frac{\partial L}{\partial y}\frac{\partial M(x)}{\partial x}$.

$\frac{\partial M(x)}{\partial x}$ is where things get complex. The partial derivatives of the model's outputs with respect to its inputs, if we were to expand the expression using the chain rule again, would involve many local partial derivatives over every multiplied learning weight, every activation function, and every other mathematical transformation in the model. The full expression for each such partial derivative is the sum of the products of the local gradient of *every possible path* through the computation graph that ends with the variable whose gradient we are trying to measure.

In particular, the gradients over the learning weights are of interest to us - they tell us *what direction to change each weight* to get the loss function closer to zero.

Since the number of such local derivatives (each corresponding to a separate path through the model's computation graph) will tend to go up exponentially with the depth of a neural network, so does the complexity in computing them. This is where autograd comes in: It tracks the history of every computation. Every computed tensor in your PyTorch model carries a history of its input tensors and the function used to create it. Combined with the fact that PyTorch functions meant to act on tensors each have a built-in implementation for computing their own derivatives, this greatly speeds the computation of the local derivatives needed for learning.

## A Simple Example

That was a lot of theory - but what does it look like to use autograd in practice?

Let's start with a straightforward example. First, we'll do some imports to let us graph our results:

```python
%matplotlib inline
```

```python
import torch

import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
import math
```

Next, we'll create an input tensor full of evenly spaced values on the interval $[0, 2{\pi}]$, and specify `requires_grad=True`. (Like most functions that create tensors, `torch.linspace()` accepts an optional `requires_grad` option.) Setting this flag means that in every computation that follows, autograd will be accumulating the history of the computation in the output tensors of that computation.

```python
a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
print(a)
```

    tensor([0.0000, 0.2618, 0.5236, 0.7854, 1.0472, 1.3090, 1.5708, 1.8326, 2.0944,
            2.3562, 2.6180, 2.8798, 3.1416, 3.4034, 3.6652, 3.9270, 4.1888, 4.4506,
            4.7124, 4.9742, 5.2360, 5.4978, 5.7596, 6.0214, 6.2832],
           requires_grad=True)

Next, we'll perform a computation, and plot its output in terms of its inputs:

```python
b = torch.sin(a)
plt.plot(a.detach(), b.detach())
```

Let's have a closer look at the tensor `b`. When we print it, we see an indicator that it is tracking its computation history:

```python
print(b)
```

    tensor([ 0.0000e+00,  2.5882e-01,  5.0000e-01,  7.0711e-01,  8.6603e-01,
             9.6593e-01,  1.0000e+00,  9.6593e-01,  8.6603e-01,  7.0711e-01,
             5.0000e-01,  2.5882e-01, -8.7423e-08, -2.5882e-01, -5.0000e-01,
            -7.0711e-01, -8.6603e-01, -9.6593e-01, -1.0000e+00, -9.6593e-01,
            -8.6603e-01, -7.0711e-01, -5.0000e-01, -2.5882e-01,  1.7485e-07],
           grad_fn=<SinBackward>)

This `grad_fn` gives us a hint that when we execute the backpropagation step and compute gradients, we'll need to compute the derivative of $sin(x)$ for all this tensor's inputs.

Let's perform some more computations:

```python
c = 2 * b
print(c)

d = c + 1
print(d)
```

    tensor([ 0.0000e+00,  5.1764e-01,  1.0000e+00,  1.4142e+00,  1.7321e+00,
             1.9319e+00,  2.0000e+00,  1.9319e+00,  1.7321e+00,  1.4142e+00,
             1.0000e+00,  5.1764e-01, -1.7485e-07, -5.1764e-01, -1.0000e+00,
            -1.4142e+00, -1.7321e+00, -1.9319e+00, -2.0000e+00, -1.9319e+00,
            -1.7321e+00, -1.4142e+00, -1.0000e+00, -5.1764e-01,  3.4969e-07],
           grad_fn=<MulBackward0>)
    tensor([ 1.0000e+00,  1.5176e+00,  2.0000e+00,  2.4142e+00,  2.7321e+00,
             2.9319e+00,  3.0000e+00,  2.9319e+00,  2.7321e+00,  2.4142e+00,
             2.0000e+00,  1.5176e+00,  1.0000e+00,  4.8236e-01, -3.5763e-07,
            -4.1421e-01, -7.3205e-01, -9.3185e-01, -1.0000e+00, -9.3185e-01,
            -7.3205e-01, -4.1421e-01,  4.7684e-07,  4.8236e-01,  1.0000e+00],
           grad_fn=<AddBackward0>)

Finally, let's compute a single-element output. When you call `.backward()` on a tensor with no arguments, it expects the calling tensor to contain only a single element, as is the case when computing a loss function.

```python
out = d.sum()
print(out)
```

    tensor(25.0000, grad_fn=<SumBackward0>)

Each `grad_fn` stored with our tensors allows you to walk the computation all the way back to its inputs with its `next_functions` property. We can see below that drilling down on this property on `d` shows us the gradient functions for all the prior tensors. Note that `a.grad_fn` is reported as `None`, indicating that this was an input to the function with no history of its own.

```python
print('d:')
print(d.grad_fn)
print(d.grad_fn.next_functions)
print(d.grad_fn.next_functions[0][0].next_functions)
print(d.grad_fn.next_functions[0][0].next_functions[0][0].next_functions)
print(d.grad_fn.next_functions[0][0].next_functions[0][0].next_functions[0][0].next_functions)
print('\nc:')
print(c.grad_fn)
print('\nb:')
print(b.grad_fn)
print('\na:')
print(a.grad_fn)
```

    d:
    <AddBackward0 object at 0x7fa00048dfd0>
    ((<MulBackward0 object at 0x7fa00048d3a0>, 0), (None, 0))
    ((<SinBackward object at 0x7fa00048dfd0>, 0), (None, 0))
    ((<AccumulateGrad object at 0x7fa00048d280>, 0),)
    ()

    c:
    <MulBackward0 object at 0x7fa0004a8070>

    b:
    <SinBackward object at 0x7fa0004a8070>

    a:
    None

With all this machinery in place, how do we get derivatives out? You call the `backward()` method on the output, and check the input's `grad` property to inspect the gradients:

```python
out.backward()
print(a.grad)
plt.plot(a.detach(), a.grad.detach())
```

Recall the computation steps we took to get here:

```
a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
b = torch.sin(a)
c = 2 * b
d = c + 1
out = d.sum()
```

Adding a constant, as we did to compute `d`, does not change the derivative. That leaves $c = 2 * b = 2 * sin(a)$, the derivative of which should be $2 * cos(a)$. Looking at the graph above, that's just what we see.

Be aware than only *leaf nodes* of the computation have their gradients computed. If you tried, for example, `print(c.grad)` you'd get back `None`. In this simple example, only the input is a leaf node, so only it has gradients computed.

## Autograd in Training

We've had a brief look at how autograd works, but how does it look when it's used for its intended purpose? Let's define a small model and examine how it changes after a single training batch. First, define a few constants, our model, and some stand-ins for inputs and outputs:

```python
BATCH_SIZE = 16
DIM_IN = 1000
HIDDEN_SIZE = 100
DIM_OUT = 10

class TinyModel(torch.nn.Module):

    def __init__(self):
        super(TinyModel, self).__init__()

        self.layer1 = torch.nn.Linear(1000, 100)
        self.relu = torch.nn.ReLU()
        self.layer2 = torch.nn.Linear(100, 10)

    def forward(self, x):
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        return x

some_input = torch.randn(BATCH_SIZE, DIM_IN, requires_grad=False)
ideal_output = torch.randn(BATCH_SIZE, DIM_OUT, requires_grad=False)

model = TinyModel()
```

One thing you might notice is that we never specify `requires_grad=True` for the model's layers. Within a subclass of `torch.nn.module`, it's assumed that we want to track gradients on the layers' weights for learning.

If we look at the layers of the model, we can examine the values of the weights, and verify that no gradients have been computed yet:

```python
print(model.layer2.weight[0][0:10]) # just a small slice
print(model.layer2.weight.grad)
```

    tensor([ 0.0244,  0.0843,  0.0378, -0.0212, -0.0114,  0.0351, -0.0460,  0.0561,
             0.0040,  0.0179], grad_fn=<SliceBackward>)
    None

Let's see how this changes when we run through one training batch. For a loss function, we'll just use the square of the Euclidean distance between our `prediction` and the `ideal_output`, and we'll use a basic stochastic gradient descent optimizer.

```python
optimizer = torch.optim.SGD(model.parameters(), lr=0.001)

prediction = model(some_input)

loss = (ideal_output - prediction).pow(2).sum()
print(loss)
```

    tensor(157.9841, grad_fn=<SumBackward0>)

Now, let's call `loss.backward()` and see what happens:

```python
loss.backward()
print(model.layer2.weight[0][0:10])
print(model.layer2.weight.grad[0][0:10])
```

    tensor([ 0.0244,  0.0843,  0.0378, -0.0212, -0.0114,  0.0351, -0.0460,  0.0561,
             0.0040,  0.0179], grad_fn=<SliceBackward>)
    tensor([ 2.8054,  1.4428,  4.5022,  4.4852, 10.5636,  8.2099,  7.4645,  1.9516,
             1.9949,  3.5033])

We can see that the gradients have been computed for each learning weight, but the weights remain unchanged, because we haven't run the optimizer yet. The optimizer is responsible for updating model weights based on the computed gradients.

```python
optimizer.step()
print(model.layer2.weight[0][0:10])
print(model.layer2.weight.grad[0][0:10])
```

    tensor([ 0.0216,  0.0829,  0.0332, -0.0257, -0.0220,  0.0269, -0.0535,  0.0541,
             0.0020,  0.0144], grad_fn=<SliceBackward>)
    tensor([ 2.8054,  1.4428,  4.5022,  4.4852, 10.5636,  8.2099,  7.4645,  1.9516,
             1.9949,  3.5033])

You should see that `layer2`'s weights have changed.

One important thing about the process: After calling `optimizer.step()`, you need to call `optimizer.zero_grad()`, or else every time you run `loss.backward()`, the gradients on the learning weights will accumulate:

```python
print(model.layer2.weight.grad[0][0:10])

for i in range(0, 5):
    prediction = model(some_input)
    loss = (ideal_output - prediction).pow(2).sum()
    loss.backward()

print(model.layer2.weight.grad[0][0:10])

optimizer.zero_grad()

print(model.layer2.weight.grad[0][0:10])
```

    tensor([ 2.8054,  1.4428,  4.5022,  4.4852, 10.5636,  8.2099,  7.4645,  1.9516,
             1.9949,  3.5033])
    tensor([ 4.6037, -1.8794,  6.1310, 20.7784, 36.1072, 11.4850, 43.7955,  1.5995,
             3.7064, -0.5014])
    tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

After running the cell above, you should see that after running `loss.backward()` multiple times, the magnitudes of most of the gradients will be much larger. Failing to zero the gradients before running your next training batch will cause the gradients to blow up in this manner, causing incorrect and unpredictable learning results.

## Turning Autograd Off and On

There are situations where you will need fine-grained control over whether autograd is enabled. There are multiple ways to do this, depending on the situation.

The simplest is to change the `requires_grad` flag on a tensor directly:

```python
a = torch.ones(2, 3, requires_grad=True)
print(a)

b1 = 2 * a
print(b1)

a.requires_grad = False
b2 = 2 * a
print(b2)
```

    tensor([[1., 1., 1.],
            [1., 1., 1.]], requires_grad=True)
    tensor([[2., 2., 2.],
            [2., 2., 2.]], grad_fn=<MulBackward0>)
    tensor([[2., 2., 2.],
            [2., 2., 2.]])

In the cell above, we see that `b1` has a `grad_fn` (i.e., a traced computation history), which is what we expect, since it was derived from a tensor, `a`, that had autograd turned on. When we turn off autograd explicitly with `a.requires_grad = False`, computation history is no longer tracked, as we see when we compute `b2`.

If you only need autograd turned off temporarily, a better way is to use the `torch.no_grad()`:

```python
a = torch.ones(2, 3, requires_grad=True) * 2
b = torch.ones(2, 3, requires_grad=True) * 3

c1 = a + b
print(c1)

with torch.no_grad():
    c2 = a + b

print(c2)

c3 = a * b
print(c3)
```

    tensor([[5., 5., 5.],
            [5., 5., 5.]], grad_fn=<AddBackward0>)
    tensor([[5., 5., 5.],
            [5., 5., 5.]])
    tensor([[6., 6., 6.],
            [6., 6., 6.]], grad_fn=<MulBackward0>)

`torch.no_grad()` can also be used as a function or method dectorator:

```python
def add_tensors1(x, y):
    return x + y

@torch.no_grad()
def add_tensors2(x, y):
    return x + y

a = torch.ones(2, 3, requires_grad=True) * 2
b = torch.ones(2, 3, requires_grad=True) * 3

c1 = add_tensors1(a, b)
print(c1)

c2 = add_tensors2(a, b)
print(c2)
```

    tensor([[5., 5., 5.],
            [5., 5., 5.]], grad_fn=<AddBackward0>)
    tensor([[5., 5., 5.],
            [5., 5., 5.]])

There's a corresponding context manager, `torch.enable_grad()`, for turning autograd on when it isn't already. It may also be used as a decorator.

Finally, you may have a tensor that requires gradient tracking, but you want a copy that does not. For this we have the `Tensor` object's `detach()` method - it creates a copy of the tensor that is *detached* from the computation history:

```python
x = torch.rand(5, requires_grad=True)
y = x.detach()

print(x)
print(y)
```

    tensor([0.5443, 0.0012, 0.8133, 0.8857, 0.7629], requires_grad=True)
    tensor([0.5443, 0.0012, 0.8133, 0.8857, 0.7629])

We did this above when we wanted to graph some of our tensors. This is because `matplotlib` expects a NumPy array as input, and the implicit conversion from a PyTorch tensor to a NumPy array is not enabled for tensors with requires_grad=True. Making a detached copy lets us move forward.

### Autograd and In-place Operations

In every example in this notebook so far, we've used variables to capture the intermediate values of a computation. Autograd needs these intermediate values to perform gradient computations. *For this reason, you must be careful about using in-place operations when using autograd.* Doing so can destroy information you need to compute derivatives in the `backward()` call. PyTorch will even stop you if you attempt an in-place operation on leaf variable that requires autograd, as shown below.

*Note: The following code cell throws a runtime error. This is expected.*

```python
a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
torch.sin_(a)
```

## Autograd Profiler

Autograd tracks every step of your computation in detail. Such a computation history, combined with timing information, would make a handy profiler - and autograd has that feature baked in. Here's a quick example usage:

```python
device = torch.device('cpu')
run_on_gpu = False
if torch.cuda.is_available():
    device = torch.device('cuda')
    run_on_gpu = True

x = torch.randn(2, 3, requires_grad=True)
y = torch.rand(2, 3, requires_grad=True)
z = torch.ones(2, 3, requires_grad=True)

with torch.autograd.profiler.profile(use_cuda=run_on_gpu) as prf:
    for _ in range(1000):
        z = (z / x) * y

print(prf.key_averages().table(sort_by='self_cpu_time_total'))
```

    -------  ---------------  ---------------  ---------------  ---------------  ---------------  ---------------
    Name     Self CPU total %  Self CPU total   CPU total %      CPU total        CPU time avg     Number of Calls
    -------  ---------------  ---------------  ---------------  ---------------  ---------------  ---------------
    mul      51.18%           7.985ms          51.18%           7.985ms          7.985us          1000
    div      48.82%           7.616ms          48.82%           7.616ms          7.616us          1000
    -------  ---------------  ---------------  ---------------  ---------------  ---------------  ---------------
    Self CPU time total: 15.601ms

The profiler can also label individual sub-blocks of code, break out the data by input tensor shape, and export data as a Chrome tracing tools file. For full details of the API, see the [documentation](https://pytorch.org/docs/stable/autograd.html#profiler)

## Advanced Topic: More Autograd Detail and the High-Level API

If you have a function with an n-dimensional input and m-dimensional output, $\vec{y}=f(\vec{x})$, the complete gradient is a matrix of the derivative of every output with respect to every input, called the *Jacobian:*

\begin{align}J=\left(\begin{array}{ccc}
   \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial y_{1}}{\partial x_{n}}\\
   \vdots & \ddots & \vdots\\
   \frac{\partial y_{m}}{\partial x_{1}} & \cdots & \frac{\partial y_{m}}{\partial x_{n}}
   \end{array}\right)\end{align}

If you have a second function, $l=g\left(\vec{y}\right)$ that takes m-dimensional input (that is, the same dimensionality as the output above), and returns a scalar output, you can express its gradients with respect to $\vec{y}$ as a column vector, $v=\left(\begin{array}{ccc}\frac{\partial l}{\partial y_{1}} & \cdots & \frac{\partial l}{\partial y_{m}}\end{array}\right)^{T}$ - which is really just a one-column Jacobian.

More concretely, imagine the first function as your PyTorch model (with potentially many inputs and many outputs) and the second function as a loss function (with the model's output as input, and the loss value as the scalar output).

If we multiply the first function's Jacobian by the gradient of the second function, and apply the chain rule, we get:

\begin{align}J^{T}\cdot v=\left(\begin{array}{ccc}
   \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial y_{m}}{\partial x_{1}}\\
   \vdots & \ddots & \vdots\\
   \frac{\partial y_{1}}{\partial x_{n}} & \cdots & \frac{\partial y_{m}}{\partial x_{n}}
   \end{array}\right)\left(\begin{array}{c}
   \frac{\partial l}{\partial y_{1}}\\
   \vdots\\
   \frac{\partial l}{\partial y_{m}}
   \end{array}\right)=\left(\begin{array}{c}
   \frac{\partial l}{\partial x_{1}}\\
   \vdots\\
   \frac{\partial l}{\partial x_{n}}
   \end{array}\right)\end{align}

*Note: You could also use the equivalent operation $v^{T}\cdot J$, and get back a row vector.*

The resulting column vector is the *gradient of the second function with respect to the inputs of the first* - or in the case of our model and loss function, the gradient of the loss with respect to the model inputs.

**`torch.autograd` is an engine for computing these products.** This is how we accumulate the gradients over the learning weights during the backward pass.

For this reason, the `backward()` call can *also* take an optional vector input. This vector represents a set of gradients over the tensor, which are multiplied by the Jacobian of the autograd-traced tensor that precedes it. Let's try a specific example with a small vector:

```python
x = torch.randn(3, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
    y = y * 2

print(y)
```

    tensor([-663.2701, 1493.5994,  789.5186], grad_fn=<MulBackward0>)

If we tried to call `y.backward()` now, we'd get a runtime error and a message that gradients can only be *implicitly* computed for scalar outputs. For a multi-dimensional output, autograd expects us to provide gradients for those three outputs that it can multiply into the Jacobian:

```python
v = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float) # stand-in for gradients
y.backward(v)

print(x.grad)
```

    tensor([1.0240e+02, 1.0240e+03, 1.0240e-01])

(Note that the output gradients are all related to powers of two - which we'd expect from a repeated doubling operation.)

### The High-Level API

There is an API on autograd that gives you direct access to important differential matrix and vector operations. In particular, it allows you to calculate the Jacobian and the *Hessian* matrices of a particular function for particular inputs. (The Hessian is like the Jacobian, but expresses all partial *second* derivatives.) It also provides methods for taking vector products with these matrices.

Let's take the Jacobian of a simple function, evaluated for a 2 single-element inputs:

```python
def exp_adder(x, y):
    return 2 * x.exp() + 3 * y

inputs = (torch.rand(1), torch.rand(1)) # arguments for the function
print(inputs)
torch.autograd.functional.jacobian(exp_adder, inputs)
```

    (tensor([0.7327]), tensor([0.8072]))

    (tensor([[4.1616]]), tensor([[3.]]))

If you look closely, the first output should equal $2e^x$ (since the derivative of $e^x$ is $e^x$), and the second value should be 3.

You can, of course, do this with higher-order tensors:

```python
inputs = (torch.rand(3), torch.rand(3)) # arguments for the function
print(inputs)
torch.autograd.functional.jacobian(exp_adder, inputs)
```

    (tensor([0.3723, 0.7282, 0.0756]), tensor([0.9602, 0.6986, 0.2260]))

    (tensor([[2.9022, 0.0000, 0.0000],
             [0.0000, 4.1428, 0.0000],
             [0.0000, 0.0000, 2.1570]]),
     tensor([[3., 0., 0.],
             [0., 3., 0.],
             [0., 0., 3.]]))

The `torch.autograd.functional.hessian()` method works identically (assuming your function is twice differentiable), but returns a matrix of all second derivatives.

There is also a function to directly compute the vector-Jacobian product, if you provide the vector:

```python
def do_some_doubling(x):
    y = x * 2
    while y.data.norm() < 1000:
        y = y * 2
    return y

inputs = torch.randn(3)
my_gradients = torch.tensor([0.1, 1.0, 0.0001])
torch.autograd.functional.vjp(do_some_doubling, inputs, v=my_gradients)
```

    (tensor([ -52.9634, -894.0098, 1167.3909]),
     tensor([5.1200e+01, 5.1200e+02, 5.1200e-02]))

The `torch.autograd.functional.jvp()` method performs the same matrix multiplication as `vjp()` with the operands reversed. The `vhp()` and `hvp()` methods do the same for a vector-Hessian product.

For more information, including preformance notes on the [docs for the functional API](https://pytorch.org/docs/stable/autograd.html#functional-higher-level-api)

## Advanced Topic: Higher-Order Derivatives
+ computing your own chain-rule derivatives with the high-level API


# The $z$-Transform

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Properties

The $z$-transform has a number of specific properties which can be concluded directly from its definition. The most important ones in the context of signals and systems are reviewed in the following.

### Linearity

As can be concluded from its definition, the $z$-transform is a linear operation. For two signals $x_1[k]$ and $x_2[k]$ with $z$-transforms $X_1(z) = \mathcal{Z} \{ x_1[k] \}$ and $X_2(z) = \mathcal{Z} \{ x_2[k] \}$ the following holds

\begin{equation}
\mathcal{Z} \{ A \cdot x_1[k] + B \cdot x_2[k] \} = A \cdot X_1(z) + B \cdot X_2(z)
\end{equation}

with $A, B \in \mathbb{C}$. The region of convergence (ROC) of the superposition $A x_1[k] + B x_2[k]$ is as least as large as the intersection of the ROCs of $x_1[k]$ and $x_2[k]$

\begin{equation}
\text{ROC} \{ A \cdot x_1[k] + B \cdot x_2[k] \} \supseteq \text{ROC} \{ x_1[k] \} \cap \text{ROC} \{ x_2[k] \}
\end{equation}

The $z$-transform of a weighted superposition of signals is equal to the weighted superposition of the individual $z$-transforms. This property is useful to derive the transform of signals that can be expressed as superposition of other signals for which the $z$-transform is known or can be calculated easier.

#### Transformation of the cosine and sine signal

The Laplace transform of the causal cosine $\epsilon[k] \cdot \cos(\Omega_0 k)$ and sine $\epsilon[k] \cdot \sin(\Omega_0 k)$ signal with $\Omega_0 \in \mathbb{R}$ is derived by expressing the cosine/sine as a superposition of harmonic exponential signals using [Euler's formula](https://en.wikipedia.org/wiki/Euler's_formula)

\begin{align}
\cos(\Omega_0 k) &= \frac{1}{2} \left( e^{j \Omega_0 k} + e^{-j \Omega_0 k} \right) \\
\sin(\Omega_0 k) &= \frac{1}{2j} \left( e^{j \Omega_0 k} - e^{-j \Omega_0 k} \right)
\end{align}

The $z$-transform of the harmonic exponential signal can be derived from the spectrum of the [causal complex exponential signal](definition.ipynb#Transformation-of-the-Causal-Complex-Exponential-Signal) as

\begin{equation}
\mathcal{Z} \{ \epsilon[k] \cdot e^{j \Omega_0 k} \} = \frac{z}{z - e^{j \Omega_0}} \qquad \text{for } |z| > 1
\end{equation}

Exploiting the linearity of the $z$-transform yields

\begin{equation}
\mathcal{Z} \{ \epsilon[k] \cdot \cos(\Omega_0 k) \} =
\frac{1}{2} \left( \frac{z}{z - e^{j \Omega_0}} + \frac{z}{z - e^{-j \Omega_0}} \right) =
\frac{z ( z - \cos(\Omega_0))}{z^2 - 2 z \cos(\Omega_0) + 1}
\qquad \text{for } |z| > 1
\end{equation}

and

\begin{equation}
\mathcal{Z} \{ \epsilon[k] \cdot \sin(\Omega_0 k) \} =
\frac{1}{2j} \left( \frac{z}{z - e^{j \Omega_0}} - \frac{z}{z - e^{-j \Omega_0}} \right) =
\frac{z \sin(\Omega_0)}{z^2 - 2 z \cos(\Omega_0) + 1}
\qquad \text{for } |z| > 1
\end{equation}

**Exercise**

* Determine the zeros and poles of above $z$-transforms. What specific symmetries do the zeros and poles show in the $z$-plane?

### Symmetry for Real-Valued Signals

The $z$-transform of a conjugated signal $x^*[k]$ is given as

\begin{equation}
\mathcal{Z} \{ x^*[k] \} = \sum_{k = -\infty}^{\infty} x^*[k] \, z^{-k}
= \left( \sum_{k = -\infty}^{\infty} x[k] \, (z^*)^{-k} \right)^*
= X^*(z^*)
\end{equation}

where $X(z) = \mathcal{Z} \{ x[k] \}$. The ROC of $x^*[k]$ is equal to the ROC of $x[k]$. Above result is used to derive the $z$-transform of the real part of the signal $x[k]$

\begin{equation}
\mathcal{Z} \left\{ \Re \{ x[k] \} \right\} = \mathcal{Z} \left\{ \frac{1}{2} ( x[k] + x^*[k] ) \right\} = \frac{1}{2} \left[ X(z) + X^*(z^*) \right]
\end{equation}

and imaginary part

\begin{equation}
\mathcal{Z} \left\{ \Im \{ x[k] \} \right\} = \mathcal{Z} \left\{ \frac{1}{2j} ( x[k] - x^*[k] ) \right\} = \frac{1}{2j} \left[ X(z) - X^*(z^*) \right]
\end{equation}

For a real-valued signal $x[k] \in \mathbb{R}$ with $\Im \{ x[k] \} = 0$ it follows that

\begin{equation}
X(z) = X^*(z^*)
\end{equation}

It can be concluded, that the $z$-transform of a real valued signal shows complex conjugate symmetry.

The poles and zeros of a [$z$-transform which can be expressed as a rational function](definition.ipynb#Representation) in $z$

\begin{equation}
X(s) = \frac{\sum_{m=0}^{M} \beta_m z^{-m}}{\sum_{n=0}^{N} \alpha_n z^{-n}} = K \cdot \frac{\prod_{\mu=0}^{Q} (z - z_{0 \mu})}{\prod_{\nu=0}^{P} (z - z_{\infty \nu})}
\end{equation}

with $Q=M-1$ and $P=N-1$ show also specific symmetries. From above condition for the $z$-transform of a real-valued signal $x[k] \in \mathbb{R}$, it can be concluded that the coefficients $\alpha_n, \beta_m$ of the numerator and denominator are real-valued. The [complex conjugate root theorem](https://en.wikipedia.org/wiki/Complex_conjugate_root_theorem) states that the roots of a polynomial with real valued coefficients are either real-valued or complex conjugate pairs. Hence, the zeros and poles of the $z$-transform $X(z)$ of a real-valued signal $x[k]$ are either real-valued or complex conjugate pairs.

**Example**

As example for a real-valued signal, the poles and zeros of the $z$-transform $X(z) = \mathcal{Z} \{ \epsilon[k] \cos(\Omega_0 k) \}$ are investigated. First the transform, as derived above, is defined in `SymPy`

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

z = sym.symbols('z', complex=True)
W0 = sym.symbols('Omega0', real=True)

X = z * (z - sym.cos(W0)) / (z**2 - 2*z*sym.cos(W0) + 1)
X
```

The roots of $X(z)$ are computed

```python
poles = sym.roots(sym.denom(X), z)
poles
```

as well as its zeros

```python
zeros = sym.roots(sym.numer(X), z)
zeros
```

Finally the pole-zero plot is plotted for $\Omega = \frac{\pi}{4}$

```python
from pole_zero_plot import pole_zero_plot

X2 = X.subs(W0, sym.pi/4)
pole_zero_plot(sym.roots(sym.denom(X2), z), sym.roots(sym.numer(X2), z))
```

**Copyright**

The notebooks are provided as [Open Educational Resource](https://de.wikipedia.org/wiki/Open_Educational_Resources). Feel free to use the notebooks for your own educational purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Lecture Notes on Signals and Systems* by Sascha Spors.


# Testing out the reparameterization trick

Just a simple implementation to test if it will be appropriate for the GLM, if it is, we can use Auto-Encoding Variational Bayes inference.

The basic premise is we can construct a differenctiable Monte-Carlo estimator,
$$
    \mathbb{E}_{q(z)}[f(z)] = \int q_{\theta}(z|x) f(z) dz
        \approx \frac{1}{L} \sum^L_{l=1} f(g_{\theta}(x, \epsilon^{(l)})),
$$
where
$$
    z^{(l)} = g_{\theta}(x, \epsilon^{(l)}) \qquad \text{and} \qquad \epsilon^{(l)} \sim p(\epsilon),
$$
that results in lower variance derivatives than Monte-Carlo sampling the derivatives using, e.g. variational black box methods.

## Test 1: $f(z)$ is a log-Normal

### Likelihood approximation

Let's start with a really simple example,
$$
\begin{align}
    f(z) &= \log \mathcal{N}(x|z, \sigma^2), \\
    q_\theta(z | x) &= \mathcal{N}(z | \mu, \lambda).
\end{align}
$$
We can solve this integral analytically,
$$
    \int \mathcal{N}(z | \mu, \lambda) \log \mathcal{N}(x|z, \sigma^2) dz
        = \log \mathcal{N}(x | \mu, \sigma^2) - \frac{\lambda}{2 \sigma^2}
$$
So we can test how this compares to the reparameterization trick results.
lets use the following deterministic function for reparameterization,
$$
    g_{(\mu, \lambda)}(\epsilon^{(l)}) = \mu + \sqrt{\lambda}\epsilon^{(l)}
$$
where
$$
    p(\epsilon) = \mathcal{N}(0, 1)
$$
Now let's test:
$$
    \log \mathcal{N}(x | \mu, \sigma^2) - \frac{\lambda}{2 \sigma^2} \stackrel{?}{\approx}
    \frac{1}{L} \sum^L_{l=1} \log \mathcal{N}(x|,g_{(\mu, \lambda)}(\epsilon^{(l)}), \sigma^2)
$$

```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as pl
pl.style.use('ggplot')

from scipy.stats import norm
from scipy.special import expit
from scipy.integrate import quadrature
from scipy.misc import derivative

from revrand.mathfun.special import softplus
from revrand.optimize import sgd, Adam

```

```python
# Initial values
x = 0
mu = 2
sigma = 3
lambd = 0.5
L = 50

```

```python
# The test
exact = norm.logpdf(x, loc=mu, scale=sigma) - lambd / (2 * sigma**2)
print("Exact expectation = {}".format(exact))

# Normal Monte Calo estimation
z = norm.rvs(loc=mu, scale=np.sqrt(lambd), size=(L,))
approx_mc = norm.logpdf(x, loc=z, scale=sigma)
print("MC Approx expectation = {} ({})".format(approx_mc.mean(), approx_mc.std()))

# Reparameterised Sampling
g = lambda e: mu + np.sqrt(lambd) * e
e = norm.rvs(loc=0, scale=1, size=(L,))
approx_re = norm.logpdf(x, loc=g(e), scale=sigma)

print("Reparameterized Approx expectation = {} ({})".format(approx_re.mean(), approx_re.std()))

```

    Exact expectation = -2.2675508218727822
    MC Approx expectation = -2.260564929445225 (0.12780830112435596)
    Reparameterized Approx expectation = -2.2630586927869594 (0.15937389700974838)

We would expect a trivial relationship here between exact monte-carlo and the reparameterization trick, since they are doing the same thing. Lets see if gradient estimates have lower variances now.

### Gradient approximation
Let's evaluate the exact gradient for $\mu$,
$$
    \frac{\partial}{\partial \mu} \left(\log \mathcal{N}(x | \mu, \sigma^2) - \frac{\lambda}{2 \sigma^2} \right) =
        \frac{1}{\sigma^2} (x - \mu)
$$
Now the approximation
$$
\begin{align}
    \frac{\partial}{\partial \mu} \left(
        \frac{1}{L} \sum^L_{l=1} \log \mathcal{N}(x|,g_{(\mu, \lambda)}(\epsilon^{(l)}), \sigma^2) \right) &=
        \frac{1}{L} \sum^L_{l=1} \frac{1}{\sigma^2} (x - g_{(\mu, \lambda)}(\epsilon^{(l)}))
        \frac{\partial g_{(\mu, \lambda)}(\epsilon^{(l)})}{\partial \mu}, \\
        &= \frac{1}{L} \sum^L_{l=1} \frac{1}{\sigma^2} (x - g_{(\mu, \lambda)}(\epsilon^{(l)})).
\end{align}
$$

```python
# A range of mu's
N = 100
mu = np.linspace(-5, 5, N)

# Exact
dmu = (x - mu) / sigma**2

# Approx
e = norm.rvs(loc=0, scale=1, size=(L, N))
approx_dmu = (x - gs(e)) / sigma**2
Edmu = approx_dmu.mean(axis=0)
Sdmu = approx_dmu.std(axis=0)

# plot
pl.figure(figsize=(15, 10))
pl.plot(mu, dmus, 'b', label='Exact')
pl.plot(mu, Edmu, 'r', label= 'Approx')
pl.fill_between(mu, Edmu - 2 * Sdmu, Edmu + 2 * Sdmu, edgecolor='none', color='r', alpha=0.3)
pl.legend()
pl.title("Derivatives of expected log Gaussian")
pl.xlabel('$\mu$')
pl.ylabel('$\partial f(z)/ \partial \mu$')
pl.show()

```

## Test 2: $f(z)$ is log Bernoulli

Now let's try the following function with the same posterior and $g$ as before,

$$
    f(z) = \log \text{Bern}(x | \text{logistic}(z)) = x z - \log(1 + exp(z))
$$

We can get an "exact" expectation using quadrature. First of all, likelihoods,

### Likelihood Approximation

```python
# Quadrature
def qlogp(z, mu):

    q = norm.pdf(z, loc=mu, scale=np.sqrt(lambd))
    logp = x * z - softplus(z)
    return q * logp

def quadELL(mu):
    return quadrature(qlogp, a=-10, b=10, args=(mu,))[0]

ELL = [quadELL(m) for m in mu]

# Reparam
e = norm.rvs(loc=0, scale=1, size=(L, N))
approx_ELL = x * g(e) - softplus(g(e))
EELL = approx_ELL.mean(axis=0)
SELL = approx_ELL.std(axis=0)

```

```python
# plot
pl.figure(figsize=(15, 10))
pl.plot(mu, ELL, 'b', label='Quadrature')
pl.plot(mu, EELL, 'r', label= 'Approx')
pl.fill_between(mu, EELL - 2 * SELL, EELL + 2 * SELL, edgecolor='none', color='r', alpha=0.3)
pl.legend()
pl.title("ELL with log Bernoulli")
pl.xlabel('$\mu$')
pl.ylabel('$\mathbb{E}[\log Bern(x | z)]$')
pl.show()

```

### Gradient approximation

$$
\begin{align}
    \frac{\partial}{\partial \mu} \mathbb{E}_q \left[\frac{\partial f(z)}{\partial \mu} \right]
    &\approx \frac{1}{L} \sum^L_{l=1} (x - \text{logistic}(g(\epsilon^{(l)})))
        \frac{\partial g(\epsilon^{(l)})}{\partial \mu} \\
    &= \frac{1}{L} \sum^L_{l=1} x - \text{logistic}(g(\epsilon^{(l)}))
\end{align}
$$

```python
# Quadrature
dmu = [derivative(quadELL, m) for m in mu]

# Reparam
e = norm.rvs(loc=0, scale=1, size=(L, N))
approx_dmu = x - expit(g(e))
Edmu = approx_dmu.mean(axis=0)
Sdmu = approx_dmu.std(axis=0)

```

```python
# plot
pl.figure(figsize=(15, 10))
pl.plot(mu, dmu, 'b', label='Quadrature')
pl.plot(mu, Edmu, 'r', label= 'Approx')
pl.fill_between(mu, Edmu - 2 * Sdmu, Edmu + 2 * Sdmu, edgecolor='none', color='r', alpha=0.3)
pl.legend()
pl.title("Derivative of $\mu$ with log Bernoulli")
pl.xlabel('$\mu$')
pl.ylabel('$\partial f(z)/ \partial \mu$')
pl.show()

```

## Optimisation test

Now let's see if we can optimise Expected log likelihood using SG!

```python
data = np.ones((100, 1), dtype=bool)
mu_rec = []

def ell_obj(mu, x, samples=100):
    e = norm.rvs(loc=0, scale=1, size=(samples,))
    g = mu + np.sqrt(lambd) * e

    ll = (x * g - softplus(g)).mean()
    dmu = (x - expit(g)).mean()

    mu_rec.append(float(mu))
    dmu_rec.append(float(dmu))

    return -ll, -dmu

res = sgd(ell_obj, x0=np.array([-4]), data=data, maxiter=1000, updater=Adam(), eval_obj=True)

```

```python
# plot
niter = len(mu_rec)

fig = pl.figure(figsize=(15, 10))
ax1 = fig.add_subplot(111)

ax1.plot(range(niter), res.norms, 'b', label='gradients')
ax1.plot(range(niter), res.objs, 'g', label='negative ELL')
ax1.set_ylabel('gradients/negative ELL')
ax1.legend()
for t in ax1.get_yticklabels():
    t.set_color('b')

ax2 = ax1.twinx()
ax2.set_ylabel('$\mu$')
ax2.plot(range(niter), mu_rec, 'r', label='$\mu$')
for t in ax2.get_yticklabels():
    t.set_color('r')

pl.show()

```

```python

```


# Homework 2 - Constructing & Analyzing Laplace Transforms for Reaction Networks

In this homework, you will construct a Laplace transform for a reaction network and predict its response to initial conditions.

The reaction network you will analyze is:

$S_1 \xrightarrow[\text{}]{k_1 S_1} S_2$,
$S_2 \xrightarrow[\text{}]{k_2 S_2} S_3$,
$S_3 \xrightarrow[\text{}]{k_3 S_3} S_2$,

In your computational studies, use $k_1 = 1$, $k_2 = 2$, $k_3 = 2.5$.
The initial conditions for $S_1 = 5$; all other species are initialized to 0.

You should submit your homework as a collab notebook with a separate section for each question.

**Questions**
1. (1 pt) Write Antimony code for the reaction network and simulate it under initial conditions.
1. (5 pt) Find the transfer function $G(s) = \frac{S_3(s)}{S_1 (s)}$. Show steps: (a) construct system equations; (b) convert to Laplace Transforms; and (c) calculate the transfer function.
You will get two points extra credit if you submit your work in LaTex.
1. (1 pt) What are the poles of $G(s)$? Is the system stable? Does it oscillate?
1. (3 pt) We can interpret the simulation in (1) as a response of the transfer function
to the input signal $S_1 (s) = \frac{5}{s + k_1}$. Show that the predicted value of $S_3$ at time
$\infty$ (the "final value") is consistent with the simulation in (1). (Hint: The final value of the Laplace transform $G(s)$
is $lim_{s\rightarrow \infty} s G(s)$.) Why is the final value less than the initial value of $S_1$
in terms of the symbolic parameters of the system?

# Preliminaries

```python
!pip install -q tellurium
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import tellurium as te
import sympy
```

# Helpful Hints

## LaTex Basics

1. All LaTex appears between dollar signs (``$``)
1. Subscripts: $x_2$
1. Superscript: $x^2$
1. Summation: $\sum_{n=0}^{\infty} e^{nt}$
1. Derivatives: $\dot{x}(t)$
1. Bold: ${\bf A}$
1. Fractions: $\frac{a}{b}$

## Steps in Constructing Laplace Transforms
1. Specify the input and output for the transfer function.
1. Write the system equations.
1. Convert to Laplace transforms.
1. Solve for the output LT in terms of the input LT and form the ratio.

# Q1: Write Antimony Code for the Reaction Network

```python
MODEL = """
J1: S1 -> S2; k1*S1
J2: S2 -> S3; k2*S2
J3: S3 -> S2; k3*S3
k1 = 1
k2 = 2
k3 = 2.5
k4 = 1
S1 = 5
S2 = 0
S3 = 0
"""
RR = te.loada(MODEL)
RR.simulate(0, 20)
RR.plot()
```

# Q2: Derive the transfer function $\frac{S_3(s)}{S_1(s)}$

## Write the system equations

\begin{eqnarray}
\dot{S}_1 & = & - k_1 S_1 \\
\dot{S}_2 & = &  k_1 S_1 - k_2 S_2 + k_3 S_3\\
\dot{S}_3 & = &  k_2 S_2 - k_3 S_3 \\
\end{eqnarray}

## Convert to Laplace Transforms

\begin{eqnarray}
s S_1 (s) & = & - k_1 S_1 (s) \\
s S_2 (s) & = &  k_1 S_1 (s) - k_2 S_2 (s) + k_3 S_3 (s) \\
s S_3 (s) & = &  k_2 S_2 (s) - k_3 S_3 (s) \\
\end{eqnarray}

## Solve for $\frac{S_3(s)}{S_1 (s)}$

```python
s, S1, S2, S3, k1, k2, k3 = sympy.symbols("s, S1, S2, S3, k1, k2, k3")
exprs = [s*S2 - k1*S1 + k2*S2 - k3*S3, s*S3 - k2*S2 + k3*S3]
sympy.solve(exprs, S3)
```

    []

```python
sympy.solve(s*S2 - k1*S1 + k2*S2 - k3*S3, S2)
```

    [(S1*k1 + S3*k3)/(k2 + s)]

```python
sympy.solve(s*S3 - k2*(S1*k1 + S3*k3)/(k2 + s) + k3*S3, S3)
```

    [S1*k1*k2/(s*(k2 + k3 + s))]

$\frac{S_3 (s)}{S_1 (s)} = \frac{k_1 k_2} {s^2+ (k_2 + k_3)s }$

# Q3: Poles of the Transfer Function

The poles are $-k_2, -k_3$.
The system is stable and does not oscillate.

# Q4: Response to the Input Signal

For $U(s) = \frac{5}{s + k_1}$, we want to find the final (converged value of) $U(s) \frac{S_3 (s)}{S_1 (s)}$.
\begin{eqnarray}
lim_{s \rightarrow 0} U(s) \frac{S_3 (s)}{S_1 (s)} & = & lim_{s \rightarrow 0}\frac{1}{s + k_1} \frac{k_1 k_2}{(s + k_2)(s + k_3)} \\
& = & \frac{1}{k_1} \frac{k_1 k_2}{ k_2 k_3} \\
& = & \frac{1}{ k_3} \\
\end{eqnarray}

In our study, $k_3 = 2.5$ and so $\frac{5}{k_3} = 2$, which is where $S_3$ converges in the simulation.
This is less than $S_1$ at time 0 because of the reversible reaction that results in some of the
mass staying in $S_2$.