# Lecture 2: Debugging, Control Statements, Functions
By Dr. Anders S. Christensen
`anders.christensen @ unibas.ch`

## Debugging Exercises:

First, we go through some common errors that you **will** encounter in this course!

I wrote the following bits of code, neither of them work!

```
# Error 1:

k = 0

# Sum all numbers from 0 to 9
for i in range(10):
  k = k + 1

 print(k)
```

```
# Error 2:

my_list = [1.0, 2.0, 3.0, 4.0, 5.0]

print("Length of list:", len(my_list))

print(my_list[1])
print(my_list[2])
print(my_list[3])
print(my_list[4])
print(my_list[5])
```

```
# Error 3:

y = [3.5, 2.1, -2.5, 1.2, 6.2, 5.2, -8.1]

# print y
print(y)

# print second entry in y
print(y(2))

print("Done!")
```

```
# Error 4:

for i in range(10)
  print(i)

print "Done!"
```

## Example 2.1: `if` and `else` statements

These statments can do things depending on a condition, for example:

```
a = 33
b = 200
if b > a:
  print("b is greater than a")
```

Or with branching:

```
a = 33
b = 20
if b > a:
  print("b is greater than a")
else:
  print("b is NOT greater than a")
```

Even more possibilites with `elif` (short for else+if):

```
a = 33
b = 20
if b > a:
  print("b is greater than a")
elif b == a:
  print("b is equal to a")
else:
  print("b is smaller than a")
```

### Other common checks:

```
a == b # Note two == signs!!
```

```
a >= b # Greater than or equal to
```

```
a > b # Greater than
```

```
a <= b # Smaller than or equal to
```

```
a < b # Smaller than
```

```
a != b # Not equal to
```

`True` and `False` are called "boolean"!

The tests above are from what is called "boolean algebra"

```
# Boolean as a variable

my_var = True
# var = False
print(my_var)

print(type(my_var))
```

What happens here?

```
if my_var:
  print("That was True")
```

Other keywords: `and`, `or`, `not`

```
a = 33
b = 20

if (a > b) and (a == 33):
  print("Complicated")
```

## Example 2.2: The `break` and `continue` statements

`Break` is used to break out of a `for` loop, for example when a condition is met:

```
# We only want to print up to i==5:

for i in range(100):

  if i > 5:
    break

  print(i)

```

The `continue` statment is used to skip only the remaining part of the current loop iteration:

```
# Print odd numbers:

for i in range(20):

  # print(i, i%2)

  # NOTE: % is the modulo operator
  if i % 2 == 0:
    continue

  print(i)
```

## Example 2.3: Defining Functions

Last time we used functions from NumPy to calcualte the exponential

```
import numpy as np

e = np.exp(1.0)

print(e)
```

We can define such functions ourself, for example
\begin{equation}
f\left(x\right) = x^{2}
\end{equation}

```
def square(x):
  f = x**2
  return f
```

Let's try our new function:

```
print(square(2))
```

### Demonstration: Define your own function!

Can you define a function such that?

\begin{equation}
f\left(x\right) = \sum_{n=0}^{10} x^{n}
\end{equation}

```
# Let's write something here!

```

#### Possible Solution:

```
def power_sum(x):

  f = 0

  for n in range(11):
    f = f + x**n

  return f

print(power_sum(11))
print(power_sum(12))
```

### **Note:**
A common error when defining functions is demonstrated here: What could be wrong??

```
def cubed(x):
  f = x**3

print(cubed(3))
```

## Code Demonstration: Heat Capacity as a function of the temperature + plotting

Same example as yesterday, but today we will look at a better code and end up with a plot.

The energy $\epsilon_n$ for a state of the harmonic oscillator is given by
\begin{equation}
\epsilon_n = n+\frac{1}{2},
\end{equation}
in units of $\hbar\omega$. We also assume that there are no degenerate states!

The probability for observing the state $n$ at a given temperature $\tau$ is given by:
\begin{equation}
p(n) = \frac{\exp{\left(-\epsilon_n / \tau \right)}}{Z},
\end{equation}

where $Z$ is the partition function (*Zustandssumme*)
\begin{equation}
Z = \sum_{n=0}^{\infty} \exp{\left(-\epsilon_n / \tau\right)}.
\end{equation}

The total *internal* energy $U$ is then given by:
\begin{equation}
U = \langle \epsilon \rangle = \sum_n \epsilon_n p(n)
\end{equation}

---

Another interesting property of this system is the *heat capactity* $C_V$ given by
\begin{align}
C_V &= \frac{\partial U}{\partial \tau} \biggl|_V \\
&= \frac{\langle \epsilon^2 \rangle - \langle \epsilon \rangle^2}{\tau^2}.
\end{align}

The expectation value $\langle \epsilon \rangle$ is given via the energy $U$:
\begin{align}
\langle \epsilon \rangle &= \sum_n \epsilon_n p(n) \\
&= \sum_n \left(n+\frac{1}{2}\right) p(n)
\end{align}

And $\langle \epsilon^2 \rangle$ is the second moment of the energy given by:
\begin{align}
\langle \epsilon^2 \rangle &= \sum_n \epsilon^2_n p(n) \\
&= \sum_n \left(n+\frac{1}{2}\right)^2 p(n)
\end{align}

#### Ugly code from last time:

```
# 7 temperatures
temperatures = [0.1, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0]
n_states = 1000

for tau in temperatures:

  # Calculate the partition function for tau
  Z = 0.0
  for n in range(n_states+1):
    Z = Z + np.exp(-(n + 0.5) / tau)

  # First moment of the energy
  U = 0.0

  # Second moment of the energy
  U2 = 0.0

  for n in range(n_states+1):

    epsilon = n + 0.5
    prob = np.exp(-epsilon/tau) / Z

    U = U + epsilon * prob
    U2 = U2 + epsilon**2 * prob

  Cv = (U2 - U**2) / tau

  print(tau, Cv)
```

### Example 1: Print the energy $\epsilon_n$ for each state up to $n=3$?

The energy $\epsilon_n$ for a state of the harmonic oscillator is given by:
\begin{equation}
\epsilon_n = n+\frac{1}{2}
\end{equation}

Last time:
```
for n in range(3+1):

  epsilon = n + 0.5

  print(n, epsilon)
```
Now, lets write a function for this:

```
# Energy for n'th state
def epsilon(n):
  e = n+0.5
  return e

print(epsilon(1))
```

    1.5

#### Example 2: What is the partition function $Z$

for $\tau$ = 2.0?

Recall that the partition function is given by:

\begin{equation}
Z(\tau) = \sum_{n=0}^{\infty} \exp{\left(-\frac{\epsilon_n}{\tau}\right)}
\end{equation}

Yesterday:
```
tau = 2.0
Z = 0.0

for n in range(10000):

  Z = Z + np.exp(-(n + 0.5) / tau)
```

```
# Import exponential function
import numpy as np

# Partition function at temperature Tau
def partition(tau):
  Z = 0

  for n in range(100):

      # Code from last time
      # Z = Z + np.exp(-(n + 0.5) / tau)

      Z = Z + np.exp(-epsilon(n) / tau)

  return Z

```

```
Z_2 = partition(2.0)

print(Z_2)
```

**Question:** When should we stop the loop??
*    Can we use `break`?

#### Example 3: What is the internal energy $U$ for $\tau$ = 2.0?

\begin{align}
\langle \epsilon \rangle &= \sum_n \epsilon_n p(n) \\
\langle \epsilon^2 \rangle &= \sum_n \epsilon_n^2 p(n)
\end{align}

Also recall that:

\begin{equation}
p(n, \tau) = \frac{\exp{\left(-\epsilon_n / \tau \right)}}{Z(\tau)},
\end{equation}

```
# Probability for n'th state at temperature Tau
def prob(n, tau):

  Z = partition(tau)

  p = np.exp(-epsilon(n) / tau) / Z

  return p

print( prob(0, 2.0) )
print( prob(1, 2.0) )
print( prob(4, 2.0) )
print( prob(100, 2.0) )
```

```
# Internal energy at temperature Tau
def internal_energy(tau):

  u = 0

  for n in range(100):
    u = u + epsilon(n) * prob(n, tau)

  return u

# Second moment of the energy at temperature Tau
def second_moment(tau):

  u2 = 0

  for n in range(100):
    u2 = u2 + epsilon(n)**2 * prob(n, tau)

  return u2

print( internal_energy(2.0))
print( second_moment(2.0))

```

### Example 4: The Heat capacity as a function of $\tau$

Definition of heat capacity $C_V$:
\begin{equation}
C_V = \frac{\langle \epsilon^2 \rangle - \langle \epsilon \rangle^2}{\tau^2}.
\end{equation}

```
def heat_capacity(tau):

  u = internal_energy(tau)
  u2 = second_moment(tau)

  cv = (u2 - u**2) / tau**2

  return cv

print(heat_capacity(2.0))
```

Finally print $C_V$ for every $\tau$:

```
temperatures = [0.1, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0]

for tau in temperatures:

  cv = heat_capacity(tau)
  print(tau, cv)

```

## Plotting everything with Pyplot/Matplotlib

First, how do we save the results in a list?

```
temperatures = [0.01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0]

# Initialize an empty list
heat_capacities = []

for tau in temperatures:

  # calculate Cv
  cv = heat_capacity(tau)

  # Add Cv to the list!
  heat_capacities.append(cv)

print(heat_capacities)

```

To plot a graph, we can use "pyplot" from matplotlib:

```
import matplotlib.pyplot as plt

# Construct the plot
plt.plot(temperatures, heat_capacities)

# Draw the plot
plt.show()
```

Matplotlib has tons of options for customization, for example labels, titles, etc.

Take a look at the matplotlib gallery: https://matplotlib.org/gallery.html

```
# Figure size in inches
plt.figure(figsize=(8, 8))

# Add labels to axes
plt.xlabel("Tau")
plt.ylabel("Heat Capacity")

# Set a descriptive title
plt.title("Quantum Harmonic Oscillator")

# Display a grid
plt.grid(True)

plt.plot(temperatures, heat_capacities)
plt.show()
```


# Conic Programming

**Originally Contributed by**: Arpit Bhatia

This tutorial is aimed at providing a simplistic introduction to conic programming using JuMP.

## What is a Cone?
A subset $C$ of a vector space $V$ is a cone if $\forall x \in C$ and positive scalars $\alpha$,
the product $\alpha x \in C$. A cone C is a convex cone if $\alpha x + \beta y \in C$,
for any positive scalars $\alpha, \beta$, and any $x, y \in C$.

## Conic Programming
Conic programming problems are convex optimization problems in which a convex function is minimized
over the intersection of an affine subspace and a convex cone.
An example of a conic-form minimization problems, in the primal form is:

$$
\begin{align}
& \min_{x \in \mathbb{R}^n} & a_0^T x + b_0 \\
& \;\;\text{s.t.} & A_i x + b_i & \in \mathcal{C}_i & i = 1 \ldots m
\end{align}
$$

The corresponding dual problem is:

$$
\begin{align}
& \max_{y_1, \ldots, y_m} & -\sum_{i=1}^m b_i^T y_i + b_0 \\
& \;\;\text{s.t.} & a_0 - \sum_{i=1}^m A_i^T y_i & = 0 \\
& & y_i & \in \mathcal{C}_i^* & i = 1 \ldots m
\end{align}
$$

where each $\mathcal{C}_i$ is a closed convex cone and $\mathcal{C}_i^*$ is its dual cone.

## Some of the Types of Cones Supported by JuMP

```julia
using JuMP
using ECOS
```

By this point we have used quite a few different solvers.
To find out all the different solvers and their supported problem types, check out the
[solver table](http://www.juliaopt.org/JuMP.jl/v0.19.0/installation/#Getting-Solvers-1) in the docs.

### Second-Order Cone
The Second-Order Cone (or Lorenz Cone) of dimension $n$ is of the form:

$$
Q^n = \{ (t,x) \in \mathbb{R}^\mbox{n} : t \ge ||x||_2 \}
$$

A Second-Order Cone rotated by $\pi/4$ in the $(x_1,x_2)$ plane is called a Rotated Second-Order Cone.
It is of the form:

$$
Q_r^n = \{ (t,u,x) \in \mathbb{R}^\mbox{n} : 2tu \ge ||x||_2^2, t,u \ge 0 \}
$$

These cones are represented in JuMP using the MOI sets `SecondOrderCone` and `RotatedSecondOrderCone`.

#### Example: Euclidean Projection on a Hyperplane
For a given point $u_{0}$ and a set $K$, we refer to any point $u \in K$
which is closest to $u_{0}$ as a projection of $u_{0}$ on $K$.
The projection of a point $u_{0}$ on a hyperplane $K = \{u | p' \cdot u = q\}$ is given by

$$
\begin{align*}
& \min & ||u - u_{0}|| \\
& \;\;\text{s.t.} & p' \cdot u = q  \\
\end{align*}
$$

```julia
u0 = rand(10)
p = rand(10)
q = rand();
```

We can model the above problem as the following conic program:

$$
\begin{align*}
& \min & t \\
& \;\;\text{s.t.} & p' \cdot u = q \\
& & (t, u - u_{0}) \in Q^{n+1}
\end{align*}
$$

On comparing this with the primal form of a conic problem we saw above,

$$
\begin{align*}
& x = (t , u) &\\
& a_0 = e_1 &\\
& b_0 = 0 &\\
& A_1 = (0, p) &\\
& b_1 = -q &\\
& C_1 = \mathbb{R}_- &\\
& A_2 = 1 &\\
& b_2 = -(0, u_0) &\\
& C_2 = Q^{n+1} &
\end{align*}
$$

Thus, we can obtain the dual problem as:

$$
\begin{align*}
& \max & y_1 + (0, u_0)^T y_2 \\
& \;\;\text{s.t.} & e_1 - (0,p)^T y_1 - y_2 = 0 \\
& & y_1 \in \mathbb{R}_- \\
& & y_2 \in Q^{n+1}
\end{align*}
$$

```julia
model = Model(with_optimizer(ECOS.Optimizer, printlevel = 0))
@variable(model, u[1:10])
@variable(model, t)
@objective(model, Min, t)
@constraint(model, [t, (u - u0)...] in SecondOrderCone())
@constraint(model, u' * p == q)
optimize!(model)
```

    ECOS 2.0.5 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

    It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
     0  +0.000e+00  -0.000e+00  +1e+01  4e-01  2e-04  1e+00  6e+00    ---    ---    1  1  - |  -  -
     1  +1.534e-01  +1.559e-01  +9e-01  5e-02  2e-05  9e-02  5e-01  0.9177  5e-04   2  1  1 |  0  0
     2  +9.409e-01  +9.665e-01  +7e-02  6e-03  2e-06  3e-02  5e-02  0.9533  6e-02   2  2  2 |  0  0
     3  +9.743e-01  +9.746e-01  +8e-04  6e-05  2e-08  4e-04  6e-04  0.9890  1e-04   1  1  1 |  0  0
     4  +9.747e-01  +9.747e-01  +9e-06  7e-07  2e-10  4e-06  7e-06  0.9890  1e-04   1  1  1 |  0  0
     5  +9.747e-01  +9.747e-01  +1e-07  8e-09  2e-12  5e-08  7e-08  0.9890  1e-04   1  1  1 |  0  0
     6  +9.747e-01  +9.747e-01  +1e-09  8e-11  2e-14  5e-10  8e-10  0.9890  1e-04   2  1  1 |  0  0

    OPTIMAL (within feastol=8.4e-11, reltol=1.1e-09, abstol=1.1e-09).
    Runtime: 0.000177 seconds.

```julia
@show objective_value(model);
@show value.(u);
```

    objective_value(model) = 0.9747011153011627
    value.(u) = [0.659213, 0.309677, 0.511505, 0.285757, 0.146255, 0.241544, -0.352419, -0.0583324, -0.107894, 0.511019]

```julia
e1 = [1, zeros(10)...]
dual_model = Model(with_optimizer(ECOS.Optimizer, printlevel = 0))
@variable(dual_model, y1 <= 0)
@variable(dual_model, y2[1:11])
@objective(dual_model, Max, q * y1 + [0, u0...]' * y2)
@constraint(dual_model, e1 - [0, p...] .* y1 - y2 .== 0)
@constraint(dual_model, y2 in SecondOrderCone())
optimize!(dual_model)
```

    ECOS 2.0.5 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

    It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
     0  +0.000e+00  -0.000e+00  +4e+00  4e-01  4e-01  1e+00  2e+00    ---    ---    1  1  - |  -  -
     1  -5.671e+00  -1.407e+00  +4e-01  3e-01  3e-01  5e+00  2e-01  0.8790  2e-03   1  1  1 |  0  0
     2  -1.487e+00  -1.569e+00  +1e-01  9e-02  4e-02  3e-02  5e-02  0.8344  1e-01   2  1  1 |  0  0
     3  -9.975e-01  -9.795e-01  +5e-03  3e-03  2e-03  2e-02  3e-03  0.9786  4e-02   2  2  2 |  0  0
     4  -9.750e-01  -9.748e-01  +6e-05  4e-05  2e-05  2e-04  3e-05  0.9890  1e-04   2  1  1 |  0  0
     5  -9.747e-01  -9.747e-01  +6e-07  4e-07  2e-07  3e-06  4e-07  0.9890  1e-04   2  1  1 |  0  0
     6  -9.747e-01  -9.747e-01  +7e-09  4e-09  2e-09  3e-08  4e-09  0.9890  1e-04   1  1  1 |  0  0

    OPTIMAL (within feastol=4.4e-09, reltol=7.1e-09, abstol=6.9e-09).
    Runtime: 0.000099 seconds.

```julia
@show objective_value(dual_model);
```

    objective_value(dual_model) = 0.9747011467702701

We can also have an equivalent formulation using a Rotated Second-Order Cone:

$$
\begin{align*}
& \min & t \\
& \;\;\text{s.t.} & p' \cdot u = q \\
& & (t, 1/2, u - u_{0})\in Q_r^{n+2}
\end{align*}
$$

```julia
model = Model(with_optimizer(ECOS.Optimizer, printlevel = 0))
@variable(model, u[1:10])
@variable(model, t)
@objective(model, Min, t)
@constraint(model, [t, 0.5, (u - u0)...] in RotatedSecondOrderCone())
@constraint(model, u' * p == q)
optimize!(model)
```

    ECOS 2.0.5 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

    It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
     0  -7.846e-17  -3.254e-02  +9e+00  4e-01  2e-02  1e+00  5e+00    ---    ---    1  1  - |  -  -
     1  +2.476e-02  +2.543e-02  +6e-01  4e-02  1e-03  7e-02  4e-01  0.9304  2e-04   1  1  1 |  0  0
     2  +6.892e-01  +9.125e-01  +3e-01  5e-02  1e-03  3e-01  2e-01  0.8037  4e-01   1  2  2 |  0  0
     3  +9.190e-01  +9.297e-01  +7e-03  2e-03  3e-05  1e-02  5e-03  0.9731  1e-04   1  1  1 |  0  0
     4  +9.371e-01  +9.412e-01  +2e-03  4e-04  9e-06  5e-03  1e-03  0.7864  7e-02   1  1  1 |  0  0
     5  +9.450e-01  +9.479e-01  +7e-04  2e-04  5e-06  3e-03  6e-04  0.7406  3e-01   1  1  1 |  0  0
     6  +9.483e-01  +9.490e-01  +2e-04  6e-05  1e-06  8e-04  2e-04  0.7802  5e-02   1  1  1 |  0  0
     7  +9.500e-01  +9.500e-01  +1e-05  3e-06  6e-08  3e-05  8e-06  0.9715  3e-02   1  1  1 |  0  0
     8  +9.500e-01  +9.500e-01  +1e-06  3e-07  6e-09  3e-06  8e-07  0.9062  3e-03   1  1  1 |  0  0
     9  +9.500e-01  +9.500e-01  +4e-08  1e-08  3e-10  2e-07  4e-08  0.9890  4e-02   1  1  1 |  0  0
    10  +9.500e-01  +9.500e-01  +1e-09  4e-10  9e-12  6e-09  1e-09  0.9702  1e-04   2  1  1 |  0  0

    OPTIMAL (within feastol=4.3e-10, reltol=1.2e-09, abstol=1.2e-09).
    Runtime: 0.000125 seconds.

```julia
@show value.(u);
```

    value.(u) = [0.659213, 0.309677, 0.511505, 0.285757, 0.146255, 0.241544, -0.352419, -0.0583324, -0.107894, 0.511019]

The difference here is that the objective in the case of the Second-Order Cone is $||u - u_{0}||_2$,
while in the case of a Rotated Second-Order Cone is $||u - u_{0}||_2^2$.
However, the value of x is the same for both.

### Exponential Cone

An Exponential Cone is a set of the form:

$$
K_{exp} = \{ (x,y,z) \in \mathbb{R}^3 : y \exp (x/y) \le z, y > 0 \}
$$

It is represented in JuMP using the MOI set `ExponentialCone`.

#### Example: Entropy Maximization
As the name suggests, the entropy maximization problem consists of maximizing the entropy function,
$H(x) = -x\log{x}$ subject to linear inequality constraints.

$$
\begin{align*}
& \max & - \sum_{i=1}^n x_i \log x_i \\
& \;\;\text{s.t.} & \mathbf{1}' x = 1 \\
& & Ax \leq b
\end{align*}
$$

We can model this problem using an exponential cone by using the following transformation:

$$
t\leq -x\log{x} \iff t\leq x\log(1/x)  \iff (1, x, t) \in K_{exp}
$$

Thus, our problem becomes,

$$
\begin{align*}
& \max & 1^Tt \\
& \;\;\text{s.t.} & Ax \leq b \\
& & 1^T x = 1 \\
& & (1, x_i, t_i) \in K_{exp} && \forall i = 1 \ldots n \\
\end{align*}
$$

```julia
n = 15;
m = 10;
A = randn(m, n);
b = rand(m, 1);

model = Model(with_optimizer(ECOS.Optimizer, printlevel = 0))
@variable(model, t[1:n])
@variable(model, x[1:n])
@objective(model, Max, sum(t))
@constraint(model, sum(x) == 1)
@constraint(model, A * x .<= b )
# Cannot use the exponential cone directly in JuMP, hence we use MOI to specify the set.
@constraint(model, con[i = 1:n], [1, x[i], t[i]] in MOI.ExponentialCone())

optimize!(model);
```

    ECOS 2.0.5 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

    It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
     0  +0.000e+00  +6.662e+00  +6e+01  1e+00  8e-01  1e+00  1e+00    ---    ---    0  0  - |  -  -
     1  +5.698e+01  +2.834e+02  +2e+01  2e+01  2e+01  1e+02  4e-01  0.6305  7e-02   1  1  1 |  0  0
     2  -2.863e+01  +1.302e+01  +1e+01  7e-01  6e-01  4e+01  3e-01  0.7833  6e-01   1  1  1 |  0  1
     3  -8.836e+01  +3.194e+01  +5e+00  9e-01  7e-01  1e+02  1e-01  0.5949  9e-02   1  2  2 |  2  2
     4  -5.666e+02  +1.039e+02  +2e+00  1e+00  1e+00  7e+02  6e-02  0.7833  2e-01   1  1  1 |  3  1
     5  -4.486e+03  +3.220e+02  +7e-01  3e+00  2e+00  5e+03  1e-02  0.9791  2e-01   1  1  1 |  4  0
     6  -1.438e+04  +3.825e+02  +1e-01  3e+00  2e+00  1e+04  3e-03  0.7815  1e-02   1  1  1 |  1  1
     7  -5.045e+04  -5.628e+02  +3e-02  2e+00  2e+00  5e+04  6e-04  0.7826  1e-02   2  1  1 |  1  1
     8  -1.894e+05  -7.687e+03  +6e-03  2e+00  1e+00  2e+05  1e-04  0.7833  9e-03   2  1  1 |  1  1
     9  -7.396e+05  -4.634e+04  +1e-03  2e+00  1e+00  7e+05  3e-05  0.7833  9e-03   1  1  1 |  1  1
    10  -2.955e+06  -2.334e+05  +3e-04  2e+00  1e+00  3e+06  5e-06  0.7833  9e-03   1  1  1 |  1  1
    11  -1.197e+07  -1.090e+06  +5e-05  2e+00  1e+00  1e+07  1e-06  0.7833  9e-03   2  1  1 |  1  1
    12  -1.110e+08  -1.198e+07  +7e-06  3e+00  2e+00  1e+08  1e-07  0.9791  1e-01   2  0  0 |  3  0
    13  -3.626e+08  -4.717e+07  +1e-06  3e+00  2e+00  3e+08  3e-08  0.7833  1e-02   2  0  0 |  1  1
    14  -1.728e+09  -2.822e+08  +4e-07  6e+00  3e+00  1e+09  7e-09  0.9791  3e-01   1  0  0 |  5  0
    15  -6.699e+09  -1.226e+09  +6e-08  6e+00  2e+00  5e+09  1e-09  0.7833  1e-02   1  0  0 |  1  1
    16  -6.766e+10  -1.311e+10  +2e-08  2e+01  6e+00  5e+10  3e-10  0.9791  3e-01   1  0  0 |  5  0

    UNBOUNDED (within feastol=3.5e-09).
    Runtime: 0.000772 seconds.

```julia
@show objective_value(model);
```

    objective_value(model) = 6.765923931549602e10

### Positive Semidefinite Cone
The set of Positive Semidefinite Matrices of dimension $n$ form a cone in $\mathbb{R}^n$.
We write this set mathematically as

$$
\mathcal{S}_{+}^n = \{ X \in \mathcal{S}^n \mid z^T X z \geq 0, \: \forall z\in \mathbb{R}^n \}.
$$

A PSD cone is represented in JuMP using the MOI sets
`PositiveSemidefiniteConeTriangle` (for upper triangle of a PSD matrix) and
`PositiveSemidefiniteConeSquare` (for a complete PSD matrix).
However, it is prefferable to use the `PSDCone` shortcut as illustrated below.

#### Example: Largest Eigenvalue of a Symmetric Matrix
Suppose $A$ has eigenvalues $\lambda_{1} \geq \lambda_{2} \ldots \geq \lambda_{n}$.
Then the matrix $t I-A$ has eigenvalues $t-\lambda_{1}, t-\lambda_{2}, \ldots, t-\lambda_{n}$.
Note that $t I-A$ is PSD exactly when all these eigenvalues are non-negative,
and this happens for values $t \geq \lambda_{1} .$
Thus, we can model the problem of finding the largest eigenvalue of a symmetric matrix as:

$$
\begin{align*}
\lambda_{1} = \max t \\
\text { s.t. } t I-A \succeq 0
\end{align*}
$$

```julia
using LinearAlgebra
using CSDP

A = [3 2 4;
     2 0 2;
     4 2 3]

model = Model(with_optimizer(CSDP.Optimizer, printlevel = 0))
@variable(model, t)
@objective(model, Min, t)
@constraint(model, t .* Matrix{Float64}(I, 3, 3) - A in PSDCone())

optimize!(model)
```

```julia
@show objective_value(model);
```

    objective_value(model) = 8.000000000000036

## Other Cones and Functions
For other cones supported by JuMP, check out the
[MathOptInterface Manual](http://www.juliaopt.org/MathOptInterface.jl/dev/apimanual/#Standard-form-problem-1).
A good resource for learning more about functions which can be modelled using cones is the
MOSEK Modeling Cookbook[[1]](#c1).

### References

1. MOSEK Modeling Cookbook — MOSEK Modeling Cookbook 3.1. Available at: https://docs.mosek.com/modeling-cookbook/index.html.


# Control

In this notebook we want to control the chaos in the Henon map. The Henon map is defined by

$$
\begin{align}
x_{n+1}&=1-ax_n^2+y_n\\
y_{n+1}&=bx_n
\end{align}.
$$

```python
from plotly import offline as py
from plotly import graph_objs as go

py.init_notebook_mode(connected=True)
```

### Fixed points

First we need to find the fixed points of the Henon map. From $y_n=y_{n+1}=bx_n$ we can elliminate $y_n$ in the first equation. The quadratic equation obtained after ellimination with $x_n=x_{n+1}$ yields,
$$
\begin{align}
x^*=\frac{b-1\pm\sqrt{4a+(b-1)^2}}{2a},
&&
y^*=bx^*,
\end{align}
$$
as the fixed points of the Henon map.

```python
def henon_map(x0, y0, a, b, N):
    x = [x0]
    y = [y0]

    for i in range(N):
        xn = x[-1]
        yn = y[-1]

        x.append(1 - a * xn**2 + yn)
        y.append(b * xn)

    return x, y

def fixed_points(a, b):
    u = (b - 1) / (2 * a)
    v = np.sqrt(4 * a + (b - 1)**2) / (2 * a)

    x1 = u - v
    x2 = u + v

    y1 = b * x1
    y2 = b * x2

    return [(x1, y1), (x2, y2)]
```

```python
((xf1, yf1), (xf2, yf2)) = fixed_points(a=1.4, b=0.3)

radius = 0.1

layout = go.Layout(
    title='Henon Attractor',
    xaxis=dict(title='x'),
    yaxis=dict(title='y', scaleanchor='x'),
    showlegend=False,
    shapes=[
        {
            'type': 'circle',
            'xref': 'x',
            'yref': 'y',
            'x0': xf1 + radius,
            'y0': yf1 + radius,
            'x1': xf1 - radius,
            'y1': yf1 - radius,
            'line': { 'color': 'gray' },
        },
        {
            'type': 'circle',
            'xref': 'x',
            'yref': 'y',
            'x0': xf2 + radius,
            'y0': yf2 + radius,
            'x1': xf2 - radius,
            'y1': yf2 - radius,
        },
    ]
)

x = []
y = []

for i in range(50):
    x0, y0 = np.random.uniform(0.2, 0.8, 2)

    xx, yy = henon_map(x0, y0, a=1.4, b=0.3, N=100)

    if np.abs(xx[-1]) < 10 and np.abs(yy[-1]) < 10:
        x += xx
        y += yy

figure = go.Figure([
    go.Scatter(x=x, y=y, mode='markers', marker=dict(size=3))
], layout)

py.iplot(figure)
```

So the second fixed point (positive sign) sits on the attractor.

```python
def fixed_point(a, b):
    return fixed_points(a, b)[1]

fixed_point(a=1.4, b=0.3)
```

    (0.6313544770895047, 0.1894063431268514)

We assume that coordinates and parameters are sufficiently close such that the following Taylor expansion is valid,$$
\boldsymbol{x}_{n+1}
=
\boldsymbol{F}\left(\boldsymbol{x}^*,\boldsymbol{r}_0\right)
+
\frac{d\boldsymbol{F}}{d\boldsymbol{x}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}\left(\boldsymbol{x}_n-\boldsymbol{x}^*\right)
+
\frac{d\boldsymbol{F}}{d\boldsymbol{r}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}\left(\boldsymbol{r}_n-\boldsymbol{r}_0\right).$$

In the regime where these linear approximations are valid we can use, $$
\Delta\boldsymbol{r}_n
=
\gamma\left(\boldsymbol{x}_n-\boldsymbol{x}^*\right). $$
Further introducing $\Delta\boldsymbol{x}_n=\boldsymbol{x}_n-\boldsymbol{x}^*$ we can rewrite the map as, $$
\Delta\boldsymbol{x}_{n+1}
=
\underbrace{\left(
\frac{d\boldsymbol{F}}{d\boldsymbol{x}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}
+
\frac{d\boldsymbol{F}}{d\boldsymbol{r}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}
\right)}_{A}
\Delta\boldsymbol{x}_n.$$

The Jacobians are $$
\begin{align}
\frac{d\boldsymbol{F}}{d\boldsymbol{x}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}
=
\begin{pmatrix}
-2 a_0 x^* & 1 \\
b_0 & 0
\end{pmatrix},
&&
\frac{d\boldsymbol{F}}{d\boldsymbol{r}_n}\Bigr|_{\boldsymbol{x}^*,\boldsymbol{r}_0}
=
\begin{pmatrix}
-{x^*}^2 & 0 \\
0 & x^*
\end{pmatrix}
\end{align}. $$

Thus the matrix $A$ reads, $$
A
=
\begin{pmatrix}
-2a_0x^*-\gamma{x^*}^2 & 1 \\
b_0 & \gamma x^*
\end{pmatrix}.
$$ The optimal value for $\gamma$ can be found for $0=A\Delta\boldsymbol{x}_n$.

```python
def eigenvector(a, b):
    xf, yf = fixed_point(a, b)

    A = np.array([
        [-2 * a * xf - xf**2, 1],
        [b, xf]
    ])

    return u-v, u+v

eigenvalues(a=1.4, b=0.3)
```

    (-1.9237388581534067, 0.15594632230279393)

The Jacobian of the Henon map close to $(x^*,a_0,b_0)$ is given through, $$
\begin{pmatrix}
-2 a_0 x^* & 1 \\
b_0 & 0
\end{pmatrix},$$
and has eigenvalues $$\lambda=-a_0\left[x^*\pm\sqrt{{x^*}^2+b_0/a_0^2}\right]$$

```python
fixed_point(a=1.4, b=0.3)
```

    (0.6313544770895047, 0.1894063431268514)


# Realization of Recursive Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Cascaded Structures

The realization of recursive filters with a high order may be subject to numerical issues. For instance, when the coefficients span a wide amplitude range, their quantization may require a small quantization step or may impose a large relative error for small coefficients. The basic concept of cascaded structures is to decompose a high order filter into a cascade of lower order filters, typically first and second order recursive filters.

### Decomposition into Second-Order Sections

The rational transfer function $H(z)$ of a linear time-invariant (LTI) recursive system can be [expressed by its zeros and poles](introduction.ipynb#Transfer-Function) as

\begin{equation}
H(z) = \frac{b_M}{a_N} \cdot \frac{\prod_{\mu=1}^{P} (z - z_{0\mu})^{m_\mu}}{\prod_{\nu=1}^{Q} (z - z_{\infty\nu})^{n_\nu}}
\end{equation}

where $z_{0\mu}$ and $z_{\infty\nu}$ denote the $\mu$-th zero and $\nu$-th pole of degree $m_\mu$ and $n_\nu$ of $H(z)$, respectively. The total number of zeros and poles is denoted by $P$ and $Q$.

The poles and zeros of a real-valued filter $h[k] \in \mathbb{R}$ are either single real valued or conjugate complex pairs. This motivates to split the transfer function into

* first order filters constructed from a single pole and zero
* second order filters constructed from a pair of conjugated complex poles and zeros

Decomposing the transfer function into these two types by grouping the poles and zeros into single poles/zeros and conjugate complex pairs of poles/zeros results in

\begin{equation}
H(z) = K \cdot \prod_{\eta=1}^{S_1} \frac{(z - z_{0\eta})}{(z - z_{\infty\eta})}
\cdot \prod_{\eta=1}^{S_2} \frac{(z - z_{0\eta}) (z - z_{0\eta}^*)} {(z - z_{\infty\eta})(z - z_{\infty\eta}^*)}
\end{equation}

where $K$ denotes a constant and $S_1 + 2 S_2 = N$ with $N$ denoting the order of the system. The cascade of two systems results in a multiplication of their transfer functions. Above decomposition represents a cascade of first- and second-order recursive systems. The former can be treated as a special case of second-order recursive systems. The decomposition is therefore known as decomposition into second-order sections (SOSs) or [biquad filters](https://en.wikipedia.org/wiki/Digital_biquad_filter). Using a cascade of SOSs the transfer function of the recursive system can be rewritten as

\begin{equation}
H(z) = \prod_{\mu=1}^{S} \frac{b_{0, \mu} + b_{1, \mu} \, z^{-1} + b_{2, \mu} \, z^{-2}}{1 + a_{1, \mu} \, z^{-1} + a_{2, \mu} \, z^{-2}}
\end{equation}

where $S = \lceil \frac{N}{2} \rceil$ denotes the total number of SOSs. These results state that any real valued system of order $N > 2$ can be decomposed into SOSs. This has a number of benefits

* quantization effects can be reduced by sensible grouping of poles/zeros, e.g. such that the spanned amplitude range of the filter coefficients is limited
* A SOS may be extended by a gain factor to further reduce quantization effects by normalization of the coefficients
* efficient and numerically stable SOSs serve as generic building blocks for higher-order recursive filters

### Example - Cascaded second-order section realization of a lowpass

The following example illustrates the decomposition of a higher-order recursive Butterworth lowpass filter into a cascade of second-order sections.

```python
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.markers import MarkerStyle
from matplotlib.patches import Circle
import scipy.signal as sig
%matplotlib inline

N = 9  # order of recursive filter

def zplane(z, p, title='Poles and Zeros'):
    "Plots zero and pole locations in the complex z-plane"
    ax = plt.gca()

    ax.plot(np.real(z), np.imag(z), 'bo', fillstyle='none', ms=10)
    ax.plot(np.real(p), np.imag(p), 'rx', fillstyle='none', ms=10)
    unit_circle = Circle((0, 0), radius=1, fill=False,
                         color='black', ls='solid', alpha=0.9)
    ax.add_patch(unit_circle)
    ax.axvline(0, color='0.7')
    ax.axhline(0, color='0.7')

    plt.title(title)
    plt.xlabel(r'Re{$z$}')
    plt.ylabel(r'Im{$z$}')
    plt.axis('equal')
    plt.xlim((-2, 2))
    plt.ylim((-2, 2))
    plt.grid()

# design filter
b, a = sig.butter(N, 0.2)
# decomposition into SOS
sos = sig.tf2sos(b, a, pairing='nearest')

# print filter coefficients
print('Coefficients of the recursive part \n')
print(['%1.2f' % ai for ai in a])
print('\n')
print('Coefficients of the recursive part of the individual SOS \n')
print('Section \t a1 \t\t a2')
for n in range(sos.shape[0]):
    print('%d \t\t %1.5f \t %1.5f' % (n, sos[n, 4], sos[n, 5]))

# plot pole and zero locations
plt.figure(figsize=(5, 5))
zplane(np.roots(b), np.roots(a), 'Poles and Zeros - Overall')

plt.figure(figsize=(10, 7))
for n in range(sos.shape[0]):
    plt.subplot(231+n)
    zplane(np.roots(sos[n, 0:3]), np.roots(sos[n, 3:6]),
           title='Poles and Zeros - Section %d' % n)
plt.tight_layout()

# compute and plot frequency response of sections
plt.figure(figsize=(10, 5))
for n in range(sos.shape[0]):
    Om, H = sig.freqz(sos[n, 0:3], sos[n, 3:6])
    plt.plot(Om, 20*np.log10(np.abs(H)), label=r'Section %d' % n)

plt.xlabel(r'$\Omega$')
plt.ylabel(r'$|H_n(e^{j \Omega})|$ in dB')
plt.legend()
plt.grid()
```

    Coefficients of the recursive part

    ['1.00', '-5.39', '13.38', '-19.96', '19.62', '-13.14', '5.97', '-1.78', '0.31', '-0.02']

    Coefficients of the recursive part of the individual SOS

    Section 	 a1 		 a2
    0 		 -0.50953 	 0.00000
    1 		 -1.04232 	 0.28838
    2 		 -1.11568 	 0.37905
    3 		 -1.25052 	 0.54572
    4 		 -1.46818 	 0.81477

**Exercise**

* What amplitude range is spanned by the filter coefficients?
* What amplitude range is spanned by the SOS coefficients?
* Change the pole/zero grouping strategy from `pairing='nearest'` to `pairing='keep_odd'`. What changes?
* Increase the order `N` of the filter. What changes?

Solution: Inspecting both the coefficients of the recursive part of the original filter and of the individual SOS reveals that the spanned amplitude range is lower for the latter. The choice of the pole/zero grouping strategy influences the locations of the poles/zeros in the individual SOS, the spanned amplitude range of their coefficients and the transfer functions of the individual sections. The total number of SOS scales with the order of the original filter.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples*.


```python
from sympy import *
# from sympy.abc import *
from IPython.display import display
init_printing()
```

# SymPy

## Symbolic Computation
Free, Open Source, Python
- solve equations - simplify expressions
- compute derivatives, integrals, limits
- work with matrices, - plotting & printing
- code gen - physics - statitics - combinatorics
- number theory - geometry - logic

----

## Modules
[SymPy Core](http://docs.sympy.org/latest/modules/core.html) - [Combinatorics](http://docs.sympy.org/latest/modules/combinatorics/index.html) - [Number Theory](http://docs.sympy.org/latest/modules/ntheory.html) - [Basic Cryptography](http://docs.sympy.org/latest/modules/crypto.html) - [Concrete Maths](http://docs.sympy.org/latest/modules/concrete.html) - [Numerical Evaluation](http://docs.sympy.org/latest/modules/evalf.html) - [Code Gen](http://docs.sympy.org/latest/modules/codegen.html) - [Numeric Computation](http://docs.sympy.org/latest/modules/numeric-computation.html) - [Functions](http://docs.sympy.org/latest/modules/functions/index.html) - [Geometry](http://docs.sympy.org/latest/modules/geometry/index.html) - [Holonomic Functions](http://docs.sympy.org/latest/modules/holonomic/index.html) - [Symbolic Integrals](http://docs.sympy.org/latest/modules/integrals/integrals.html) - [Numeric Integrals](http://docs.sympy.org/latest/modules/integrals/integrals.html#numeric-integrals) - [Lie Algebra](http://docs.sympy.org/latest/modules/liealgebras/index.html) - [Logic](http://docs.sympy.org/latest/modules/logic.html) - [Matricies](http://docs.sympy.org/latest/modules/matrices/index.html) - [Polynomials](http://docs.sympy.org/latest/modules/polys/index.html) - [Printing](http://docs.sympy.org/latest/modules/printing.html) - [Plotting](http://docs.sympy.org/latest/modules/plotting.html) - [Pyglet Plotting](http://docs.sympy.org/latest/modules/plotting.html#module-sympy.plotting.pygletplot) - [Assumptions](http://docs.sympy.org/latest/modules/assumptions/index.html) - [Term Rewriting](http://docs.sympy.org/latest/modules/rewriting.html) - [Series Module](http://docs.sympy.org/latest/modules/series/index.html) - [Sets](http://docs.sympy.org/latest/modules/sets.html) - [Symplify](http://docs.sympy.org/latest/modules/simplify/simplify.html) - [Hypergeometrtic](http://docs.sympy.org/latest/modules/simplify/hyperexpand.html) - [Stats](http://docs.sympy.org/latest/modules/stats.html) - [ODE](http://docs.sympy.org/latest/modules/solvers/ode.html) - [PDE](http://docs.sympy.org/latest/modules/solvers/pde.html) - [Solvers](http://docs.sympy.org/latest/modules/solvers/solvers.html) - [Diophantine](http://docs.sympy.org/latest/modules/solvers/diophantine.html) - [Inequality Solvers](http://docs.sympy.org/latest/modules/solvers/inequalities.html) - [Solveset](http://docs.sympy.org/latest/modules/solvers/solveset.html) - [Tensor](http://docs.sympy.org/latest/modules/tensor/index.html) - [Utilities](http://docs.sympy.org/latest/modules/utilities/index.html) - [Parsing Input](http://docs.sympy.org/latest/modules/parsing.html) - [Calculus](http://docs.sympy.org/latest/modules/calculus/index.html) - [Physics](http://docs.sympy.org/latest/modules/physics/index.html) - [Categrory Theory](http://docs.sympy.org/latest/modules/categories.html) - [Differential Geometry](http://docs.sympy.org/latest/modules/diffgeom.html) - [Vector](http://docs.sympy.org/latest/modules/vector/index.html)

----
## Simple Expressions

```python
# declare variable first
x, y = symbols('x y')

# Declare expression
expr = x + 3*y

# Print expressions
print("expr     =", expr)
print("expr + 1 =", expr + 1)
print("expr - x =", expr - x) # auto-simplify
print("x * expr =", x * expr)
```

    expr     = x + 3*y
    expr + 1 = x + 3*y + 1
    expr - x = 3*y
    x * expr = x*(x + 3*y)

----
## Substitution

```python
x = symbols('x')

expr = x + 1
print(expr)
display(expr)
```

```python
# Evaluate expression at a point
print("expr(2)=", expr.subs(x, 2))
```

    expr(2)= 3

```python
# Replace sub expression with another sub expression
# 1. For expressions with symmetry
x, y = symbols('x y')
expr2 = x ** y
expr2 = expr2.subs(y, x**y)
expr2 = expr2.subs(y, x**x)
display(expr2)
```

```python
# 2. Controlled simplifcation
expr3 = sin(2*x) + cos(2*x)
print("expr3")
display(expr3)
print(" ")

print("expand_trig(expr3)")
display(expand_trig(expr3))
print(" ")

print("use this to only expand sin(2*x) if desired")
print("expr3.subs(sin(2*x), 2*sin(x)*cos(x))")
display(expr3.subs(sin(2*x), 2*sin(x)*cos(x)))
```

```python
# multi-substitute
expr4 = x**3 + 4*x*y - z
args = [(x,2), (y,4), (z,0)]
expr5 = expr4.subs(args)
display(expr4)
print("args = ", args)
display(expr5)
```

```python
expr6 = x**4 - 4*x**3 + 4 * x ** 2 - 2 * x + 3
args = [(x**i, y**i) for i in range(5) if i%2 == 0]
display(expr6)
print(args)
display(expr6.subs(args))
```

----
## Equality & Equivalence

```python
# do not use == between symbols and variables, will return false

x = symbols('x')
x+1==4
```

    False

```python
# Create a symbolic equality expression
expr2 = Eq(x+1, 4)
print(expr2)
display(expr2)

print("if x=3, then", expr2.subs(x,3))
```

```python
# two equivalent formulas
expr3 = (x + 1)**2 # we use pythons ** exponentiation (instead of ^)
expr4 = x**2 + 2*x + 1
eq34 = Eq(expr3, expr4)

print("expr3")
display(expr3)

print("                        ≡   expr4")
display(expr4)
print("")

print("(expr3 == expr4)        => ", expr3 == expr4)
print("(these are equivalent, but not the same symbolically)")
print("")

print("Equal by negating, simplifying and comparing to 0")
print("expr3 - expr4           => ", expr3 - expr4)
print("simplify(expr3-expr4)==0=> ", simplify(expr3 - expr4)==0 )
print("")

print("Equals (test by evaluating 2 random points)")
print("expr3.equals(expr4)     => ", expr3.equals(expr4))

```

----
## SymPy Types & Casting

```python
print( "1                     =", type(1) )
print( "1.0                   =", type(1.0) )
print( "Integer(1)            =", type(Integer(1)) )
print( "Integer(1)/Integer(3) =", type(Integer(1)/Integer(3)) )
print( "Rational(0.5)         =", type(Rational(0.5)) )
print( "Rational(1/3)         =", type(Rational(1,3)) )
```

    1                     = <class 'int'>
    1.0                   = <class 'float'>
    Integer(1)            = <class 'sympy.core.numbers.One'>
    Integer(1)/Integer(3) = <class 'sympy.core.numbers.Rational'>
    Rational(0.5)         = <class 'sympy.core.numbers.Half'>
    Rational(1/3)         = <class 'sympy.core.numbers.Rational'>

```python
# string to SymPy
sympify("x**2 + 3*x - 1/2")
```

----

## Evaluating Expressions

```python
# evaluate as float using .evalf(), and N
display( sqrt(8) )
display( sqrt(8).evalf() )
display( sympy.N(sqrt(8)) )
```

```python
# evaluate as float to nearest n decimals
display(sympy.pi)
display(sympy.pi.evalf(100))
```

----

## SymPy Types

#### Number Class
[Number](http://docs.sympy.org/latest/modules/core.html#number) - [Float](http://docs.sympy.org/latest/modules/core.html#float) - [Rational](http://docs.sympy.org/latest/modules/core.html#rational) - [Integer](http://docs.sympy.org/latest/modules/core.html#integer) - [RealNumber](http://docs.sympy.org/latest/modules/core.html#realnumber)

#### Numbers
[Zero](http://docs.sympy.org/latest/modules/core.html#zero) - [One](http://docs.sympy.org/latest/modules/core.html#one) - [Negative One](http://docs.sympy.org/latest/modules/core.html#negativeone) - [Half](http://docs.sympy.org/latest/modules/core.html#half) - [NaN](http://docs.sympy.org/latest/modules/core.html#nan) - [Infinity](http://docs.sympy.org/latest/modules/core.html#infinity) - [Negative Infinity](http://docs.sympy.org/latest/modules/core.html#negativeinfinity) - [Complex Infinity](http://docs.sympy.org/latest/modules/core.html#complexinfinity)

#### Constants
[E (Transcedental Constant)](http://docs.sympy.org/latest/modules/core.html#exp1) - [I (Imaginary Unit)](http://docs.sympy.org/latest/modules/core.html#imaginaryunit) - [Pi](http://docs.sympy.org/latest/modules/core.html#pi) - [EulerGamma (Euler-Mascheroni constant)](http://docs.sympy.org/latest/modules/core.html#eulergamma) - [Catalan (Catalan's Constant)](http://docs.sympy.org/latest/modules/core.html#catalan) - [Golden Ratio](http://docs.sympy.org/latest/modules/core.html#goldenratio)

### Rational Numbers

```python
# Rational Numbers
expr_rational = Rational(1)/3
print("expr_rational")
display( type(expr_rational) )
display( expr_rational )

eval_rational = expr_rational.evalf()
print("eval_rational")
display( type(eval_rational) )
display( eval_rational )

neval_rational = N(expr_rational)
print("neval_rational")
display( type(neval_rational) )
display( neval_rational )
```

### Complex Numbers

```python
# Complex Numbers supported.
expr_cplx = 2.0 + 2*sympy.I

print("expr_cplx")
display( type(expr_cplx) )
display( expr_cplx )

print("expr_cplx.evalf()")
display( type(expr_cplx.evalf()) )
display( expr_cplx.evalf() )

print("float() - errors")
print(" ")
# this errors complex cannot be converted to float
#display( float(sym_cplx) )

print("complex() - evaluated to complex number")
display( complex(expr_cplx) )
display( type(complex(expr_cplx)) )
```

```python
# Partial Evaluation if cannot be evaluated as float
display( (sympy.pi*x**2 + x/3).evalf(2) )
```

```python
# use substitution in evalf
expr = cos(2*x)
expr.evalf(subs={x:2.4})
```

```python
# sometimes there are round-offs smaller than the desired precision
one = cos(1)**2 + sin(1)**2
display( (one-1).evalf() )
# chop=True can remove these errors
display( (one-1).evalf(chop=True) )
```

```python
import sys
'gmpy2' in sys.modules.keys()
```

    True

```python

```


# Homework 6: Perturbed QM Oscillator (20 points)

Group Members: Julius Franke (el442, juliusttf@gmail.com), Erik Meister (kd400, erik.meister@me.com), Eugen Dizer (qo452, eugen9898@web.de)

Due on Friday, 05.06.2020.

```python
#Load standard libraries
import numpy as np
import matplotlib.pyplot as plt
from scipy.linalg import eigh
%matplotlib inline
```

Calculate the eigenvalues of the perturbed quantum mechanical harmonic oscillator for $n = 0 \dots 9$ by approximating the operators in Hilbert space by matrices with finite dimension in the range $N = 15 \dots 30$.

The dimensionless Hamiltonian reads

\begin{align}
h = \frac{H}{\hbar \omega} &= \left( \frac{1}{2} \Pi^2 + \frac{1}{2} Q^2 + \lambda Q^4 \right) \\
\left(h\right)_{nm} &= \left(h_0\right)_{nm} + \lambda \left(Q^4\right)_{nm}
\end{align}

where $\left(h_0\right)_{nm} = \left(n + \frac{1}{2}\right) \delta_{nm}$ is the unperturbed Hamiltonian.

### 1. Determine the matrix form of $Q^4$ using
### $$Q_{nm} = \frac{1}{\sqrt{2}} \left(\sqrt{n+1} \delta_{n, m-1} + \sqrt{n} \delta_{n, m+1} \right) .$$

The operator $Q$ can be written in terms of creation an annihilation operators, $a^{\dagger}$ and $a$, as follows:

$$ Q = \frac{1}{\sqrt{2}} \left(a + a^{\dagger} \right) .$$

With the commutator relations $\left[a , a^{\dagger}\right] = 1$, $\left[a , N\right] = a$ and $\left[a^{\dagger} , N\right] = - a^{\dagger}$, where $N = a^{\dagger} a$ is the number operator, one obtains

$$Q^2 = \frac{1}{2} \left( a^2 + 2N + 1 + \left(a^{\dagger}\right)^2 \right)$$
and finally
$$Q^4 = \frac{1}{4} \left( a^4 + (4N + 6)a^2 + (6N^2 + 6N + 3) + (4N - 2)\left(a^{\dagger}\right)^2 + \left(a^{\dagger}\right)^4 \right).$$
In matrix form this is
\begin{align}
\left(Q^4\right)_{nm} = \frac{1}{4} \Big( &\sqrt{(n+1)(n+2)(n+3)(n+4)} \ \delta_{n, m-4} \\
&+ (4n + 6)\sqrt{(n+1)(n+2)} \ \delta_{n, m-2} \\
&+ (6n^2 + 6n + 3) \ \delta_{n,m} \\
&+ (4n - 2)\sqrt{n(n-1)} \ \delta_{n, m+2}  \\
&+ \sqrt{n(n-1)(n-2)(n-3)} \ \delta_{n, m+4} \Big).
\end{align}

```python
#Implementation of h for arbitrary matrix size N
def unperturbed_hamiltonian(N):
    matrix = np.zeros(shape = (N,N))
    for i in range(N):
        matrix[i][i] = i + 1/2

    return matrix

#Implementation of Q
def Q(N):
    matrix = np.zeros(shape = (N,N))
    for i in range(N-1):
        matrix[i][i+1] = np.sqrt(i+1)
        matrix[i+1][i] = np.sqrt(i+1)

    return matrix / np.sqrt(2)
```

One can obtain the matrix representation of $Q^4$ just by taking the matrix $Q$ to the power of 4:

```python
#Simple way to get Q^4 directly out of Q
def Q4(N):
    N += 2
    Q2 = np.matmul(Q(N),Q(N))
    Q4 = np.matmul(Q2,Q2)

    return Q4[:-2,:-2]
```

Due to the finite matrix size, one gets some artefacts at the border of the matrix, so one has the shorten the output. But in general this leads to the same result as the implementation of $Q^4$ via the analytic formula:

```python
#Q^4 out of the calculation with a and a*
def Q4_formula(N):
    matrix = np.zeros(shape = (N,N))
    for i in range(N):
        matrix[i][i] = 6*i**2 + 6*i + 3
    for i in range(N-2):
        matrix[i][i+2] = (4*i + 6) * np.sqrt((i+1)*(i+2))
        matrix[i+2][i] = (4*i + 6) * np.sqrt((i+1)*(i+2))
    for i in range(N-4):
        matrix[i][i+4] = np.sqrt((i+1)*(i+2)*(i+3)*(i+4))
        matrix[i+4][i] = np.sqrt((i+1)*(i+2)*(i+3)*(i+4))

    return matrix / 4
```

The matrix representation of the full Hamiltonian is thus given by:

```python
def hamiltonian(lamb, N):
    return unperturbed_hamiltonian(N) + lamb * Q4_formula(N)
```

### 2. Compute the eigenvalues of $(h)_{nm}$ for the parameter $\lambda = 0.1$ as function of the matrix size $(N = 15 \dots 30)$. Demonstrate that your program works properly, just listing the eigenvalues is not sufficient.

```python
#Calculate the eigenvalues for matrix size N and lambda = 0.1
def eigenvalue(N):
    return eigh(hamiltonian(0.1, N))[0]

#Take the first 10 eigenvalues for each matrix size N between 15 and 30
eigenvalues = []
for i in range(15, 31):
    eigenvalues.append(eigenvalue(i)[:10])

eigenvalues = np.array(eigenvalues)

N = np.linspace(15, 30, 16)
for i in range(10):
    plt.plot(N, eigenvalues[:,i], label = "n = {}".format(i))

plt.title("First 9 eigenvalues as function of matrix size")
plt.xlabel("Matrix Size N")
plt.legend(bbox_to_anchor=(1, 1))
plt.show()
```

The first 10 eigenvalues are:

```python
print(eigenvalue(30)[:10])
```

    [  0.55914633   1.76950264   3.13862431   4.62888281   6.2203009
       7.89976723   9.65784064  11.4873175   13.38247933  15.33868582]

To demonstrate that the program works properly consider both sides of the eigenvalue equation: $h \psi = \varepsilon \psi$.

```python
#Choose lambda = 0.1 and N = 4. Demonstrate for the 1st eigenvalue.
matrix = hamiltonian(0.1, 4)
eigenval = eigh(matrix)[0][1]
eigenvec = eigh(matrix)[1][:,1]
```

```python
np.matmul(matrix, eigenvec)
```

    array([ 0.        , -1.74592707,  0.        ,  0.29665452])

```python
eigenval * eigenvec
```

    array([ 0.        , -1.74592707,  0.        ,  0.29665452])

They coincide indeed!

Although it has some problems with larger vectors and very small numbers:

```python
matrix = hamiltonian(0.1, 10)
eigenval = eigh(matrix)[0][1]
eigenvec = eigh(matrix)[1][:,1]
```

```python
print(np.matmul(matrix, eigenvec))
```

    [ -4.25668881e-16  -1.74303346e+00  -5.29792232e-15   3.04383239e-01
       3.73369659e-15  -9.24936594e-03   1.44781430e-16  -1.58016563e-02
       2.06962047e-15   5.94612392e-03]

```python
print(eigenval * eigenvec)
```

    [ -4.64158363e-16  -1.74303346e+00  -3.14329111e-15   3.04383239e-01
       1.47341771e-15  -9.24936594e-03  -2.94683542e-16  -1.58016563e-02
       1.96455695e-16   5.94612392e-03]

### 3. Calculate the eigenvalues analytically using the linearized form of the equation, i.e. consider only the terms on the diagonal.

Take the diagonal elements of $h_0$ and $Q^4$. The eigenvalues are then just the diagonal entries:

```python
eigenvals = []

for i in range(10):
    h0 = i + 1/2
    q4 = 0.1*(6*i**2 + 6*i +3)/4
    eigenvals.append(h0 + q4)

print(eigenvals)
```

    [0.575, 1.875, 3.475, 5.375, 7.575, 10.075, 12.875, 15.975, 19.375, 23.075000000000003]

In comparison, the numerically calculated eigenvalues with the full Hamiltonian:

```python
print(eigenvalue(30)[:10])
```

    [  0.55914633   1.76950264   3.13862431   4.62888281   6.2203009
       7.89976723   9.65784064  11.4873175   13.38247933  15.33868582]

One can see that the approximation is relatively good for the first few eigenvalues but fails more and more for larger eigenvalues. This is due to the relatively large $\lambda$ value. The smaller $\lambda$, the better is the perturbative approach.


## Black-Scholes with Devito

Black-Scholes is a model used to determine the theoretical value of a call or a put option in finance. A way that traders might use it is to buy an option priced under the Black-Scholes value and sell options priced higher than the calculated value. Larger firms might use the model to hedge, or attempt to eliminate risk, by buying and selling the underlying asset in just the right way. Here we solve the partial differential equation and calculate theoretical "fair" values for an option from one year to expiration to the expiration date.

Symbol definitions:

* $S(t)$ is the price of the underlying asset at time $t$
* $V(t,S)$ is the value of the option at time $t$ and stock price $S$
* $K$ is the strike price of the option
* $r$ is the annualized risk-free interest rate
* $\sigma$ is the standard deviation of the stock's returns
* $t$ is the time, in years, to expiration.

```python
from devito import (Eq, Grid, TimeFunction, Operator, solve, Constant,
                    SpaceDimension, configuration, SubDomain)

from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d.axis3d import Axis
import matplotlib.pyplot as plt
import matplotlib as mpl
from matplotlib import cm

from sympy.stats import Normal, cdf
import numpy as np
import time as timer

%matplotlib inline
mpl.rc('font', size=14)

configuration["log-level"] = 'INFO'

## Constants
# The strike price of the option
K = 100.0

# The annualized risk free interest rate
r = 0.12

# The standard deviation of the stock's returns
sigma = 0.1

# The S (price of the asset) range to process
smin = 70
smax = 130

# If you want to try some different problems, uncomment these lines

## Example 2
# K = 10.0
# r = 0.1
# sigma = 0.2
# smin = 0.0
# smax = 20.0

## Example 3
# K = 100.0
# r = 0.05
# sigma = 0.25
# smin = 50.0
# smax = 150.0

# Amount of padding to proccess left/right (hidden from graphs)
padding = 10
smin -= padding
smax += padding

# Extent calculations
tmax = 1.0
dt0 = 0.0005
ds0  = 1.0
nt = (int)(tmax / dt0) + 1
ns = int((smax - smin) / ds0) + 1

shape = (ns, )
origin =(smin, )
spacing = (ds0, )
extent = int(ds0 * (ns - 1))

print("dt,tmax,nt;", dt0,tmax,nt)
print("shape;   ", shape)
print("origin;  ", origin)
print("spacing; ", spacing)
print("extent;  ", extent)
```

    dt,tmax,nt; 0.0005 1.0 2001
    shape;    (81,)
    origin;   (60,)
    spacing;  (1.0,)
    extent;   80

## Black Scholes initial boundary value problem
$$
- \frac{\partial}{\partial t} V(t,s) + r\ s\ \frac{\partial}{\partial s} V(t,s) + \frac{1}{2}\
    \sigma^2\ s^2\ \frac{\partial^2}{\partial s^2} V(t,s) - r\ V(t,s) = 0
$$

This differs from the regular Black Scholes PDE since our $\frac{\partial}{\partial t} V(t,s)$ is negative. This is because instead of using a terminal condition, we are using an initial condition and solving backward in time.

#### Update equation

In our update equation, we solve for $v.\text{forward} = v{\left(t + dt,s \right)}$.

Because $$-\frac{\partial}{\partial t} V(t,s) = \displaystyle \frac{v{\left(t,s \right)}}{dt} - \frac{v{\left(t + dt,s \right)}}{dt}$$

Our update equation will be:

$$v{\left(t + dt,s \right)} = v{\left(t,s \right)} + dt \left( r\ s\ \frac{\partial}{\partial s} V(t,s) + \frac{1}{2}\ \sigma^2\ s^2\ \frac{\partial^2}{\partial s^2} V(t,s) - r\ V(t,s) \right)$$

We give this to Devito to solve for the next timestep of the problem. Our finite difference order of accuracy will be 4, meaning we will use two points off center to approximate the derivatives.

#### Initial and boundary conditions

The boundary conditions are the following:

Call option:

* For very large S: $V\left(t,S\right) \approx S$
* For very small S: $V\left(t,S\right) \approx 0$

Put option:

* For very large S: $V\left(t,S\right) \approx 0$
* For very small S: $V\left(t,S\right) \approx K e^{-r(T-t)}$

In this example we are solving for a call option. Because we have a set S range we are solving the problem in (smin to smax), we don't need to worry about either of these boundary conditions. The initial condition will be:

$$V(t=T,S)=\max(S-K,0)$$

Where T is the time at expiration. For us this is flipped, so we set this condition at $T = 0$.

When solving this problem, most approaches typically convert to the heat equation with a variable change of $x=\ln S$, which gives a simpler PDE which is computationaly easier to solve. However, with the power of Devito, we can just solve the Black Scholes PDE directly.

```python
s = SpaceDimension(name='s', spacing=Constant(name='h_s', value=spacing[0]))
grid = Grid(shape=shape, origin=origin, extent=extent, dimensions=(s, ))

so = 2
v        = TimeFunction(name='v',        grid=grid, space_order=so, time_order=1, save=nt)
v_no_bc  = TimeFunction(name='v_no_bc',  grid=grid, space_order=so, time_order=1, save=nt)

t,s = v.dimensions
ds = s.spacing
dt = t.spacing

# Derivatives
vds = v.ds(side="centered")
vds2 = v.ds2

# Centered der eq
pde_centered = -v.dt + r*(smin+ds0*s) * vds + 0.5 * sigma**2 * (smin+ds0*s)**2 * vds2 - r*v

# Time update equations
update_centered = solve(pde_centered, v.forward)
```

## Boundary conditions

Because this problem has non zero boundary conditions, we need to add a boundary condition on the right so that when the equation refers to out of bounds values it has a good approximation. Because the slope on the right is linear, a Neumann boundary condition is a good choice. Because our order of accuracy is 4, we need to implement the condition for two points past the right edge.

ASCII representation of what happens at the boundary:

                          |  o  v(t,smax+1)
                          | /
                          |/
                          o  v(t,smax)
                         /|
                        / |
          v(t,smax-1)  X  |
                      /   | BC: constant slope for v when s >= smax
                     /    | need 2 points for a 4th order derivative
       v(t,smax-2)  X     |
                   /      |   slope = v(t,smax-1) - v(t,smax-2)
                  /       |   v(t,smax)   = v(t,smax-1) + slope
    v(t,smax-2)  X        |   v(t,smax+1) = v(t,smax)   + slope
                          |
         interior         |        boundary
        (s < smax)      smax      (s >= smax)

```python
# Equations with Neumann boundary conditions
eq       = [Eq(v[t,extent], v[t,extent-1]+(v[t,extent-1]-v[t,extent-2])),
            Eq(v[t,extent+1], v[t,extent]+(v[t,extent-1]-v[t,extent-2])),
            Eq(v.forward, update_centered)]
eq_no_bc = [Eq(v.forward, update_centered)]

op       = Operator(eq, subs=v.grid.spacing_map)
op_no_bc = Operator(eq_no_bc, subs=v_no_bc.grid.spacing_map)

# Initial conditions

for i in range(shape[0]):
    v.data[0, i]       = max((smin + ds0 * i) - K, 0)
    v_no_bc.data[0, i] = max((smin + ds0 * i) - K, 0)
```

```python
#NBVAL_IGNORE_OUTPUT

# Run our operators
startDevito = timer.time()

# Apply operator
op.apply(dt=dt0)

endDevito = timer.time()

# Apply operator without boundary condition
op_no_bc.apply(dt=dt0, v=v_no_bc)
```

    Operator `Kernel` run in 0.01 s
    Operator `Kernel` run in 0.01 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
                         PerfEntry(time=0.00023999999999999944, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

```python
#NBVAL_IGNORE_OUTPUT

# Get an appropriate ylimit
slice_smax = v.data[:,int(smax-smin-padding)]
ymax = max(slice_smax) + 2

# Plot
s = np.linspace(smin, smax, shape[0])
plt.figure(figsize=(12,10), facecolor='w')

time = [1*nt//5, 2*nt//5, 3*nt//5, 4*nt//5, 5*nt//5-1]
colors = ["blue", "green", "gold", "darkorange", "red"]

# initial conditions
plt.plot(s, v_no_bc.data[0,:], '-', color="black", label='initial condition', linewidth=1)

for i in range(len(time)):
    plt.plot(s, v_no_bc.data[time[i],:], '-', color=colors[i], label='t='+str(time[i]*dt0), linewidth=1.5)

plt.xlim([smin+padding,smax-padding])
plt.ylim([0,ymax])

plt.legend(loc=2)
plt.grid(True)
plt.xlabel("S")
plt.ylabel("Value")
plt.title("Value of a call option at various times, no boundary conditions")
plt.tight_layout()
```

Here is what our plot looks like without boundary conditions. Because on the right side the derivatives try to read data that is not defined, the lines for the value of our option fall incorrectly towards zero.

```python
#NBVAL_IGNORE_OUTPUT

# Get an appropriate ylimit
slice_smax = v.data[:,int(smax-smin-padding)]
ymax = max(slice_smax) + 2

# Plot
s = np.linspace(smin, smax, shape[0])
plt.figure(figsize=(12,10), facecolor='w')

time = [1*nt//5, 2*nt//5, 3*nt//5, 4*nt//5, 5*nt//5-1]
colors = ["blue", "green", "gold", "darkorange", "red"]

# initial conditions
plt.plot(s, v.data[0,:], '-', color="black", label='initial condition', linewidth=1)

for i in range(len(time)):
    plt.plot(s, v.data[time[i],:], '-', color=colors[i], label='t='+str(time[i]*dt0), linewidth=1.5)

plt.xlim([smin+padding,smax-padding])
plt.ylim([0,ymax])

plt.legend(loc=2)
plt.grid(True)
plt.xlabel("S")
plt.ylabel("Value")
plt.title("Value of a call option at various times")
plt.tight_layout()
```

## Plotting results

Above is a plot of our results. If the stock price is below the strike price at expiration, the option is worthless and thus the value is 0. At stock prices above the strike price, the value of the option is $S - K$.

The red line is the value of the option 1 year from expiration. Orange is 0.8 years from expiration, yellow is 0.6 years from expiration, green is 0.4 years from expiration, and blue is 0.2 years from expiration. The black line is the initial condition (the value of the option at expiration).

Below we will plot the option value versus the stock value and the time to expiration as a surface.

```python
#NBVAL_IGNORE_OUTPUT

from mpl_toolkits.mplot3d import Axes3D

# Trim the padding off smin and smax
trim_data = v.data[:, padding:-padding]

# Our axes
tt = np.linspace(0.0, dt0*(nt-1), nt)
ss = np.linspace(smin+padding, smax-padding, shape[0]-padding*2)

hf = plt.figure(figsize=(12,12))
ha = plt.axes(projection='3d')

# 45 degree viewpoint
ha.view_init(elev=25, azim=-45)

ha.set_xlim3d([0.0,1.0])
ha.set_ylim3d([smin+padding,smax-padding])
ha.set_zlim3d([0,ymax])

ha.set_xlabel('Time to expiration', labelpad=12, fontsize=16)
ha.set_ylabel('Stock value', labelpad=12, fontsize=16)
ha.set_zlabel('Value of call', labelpad=12, fontsize=16)

X, Y = np.meshgrid(tt, ss)  # `plot_surface` expects `x` and `y` data to be 2D
ha.plot_surface(X, Y, np.transpose(trim_data), cmap='viridis', edgecolor='none')

hf.suptitle("Value surface from the Black-Scholes formula", fontsize=22)
plt.tight_layout()
plt.show()
```

Above is the option value plotted as a surface. Lighter colors represent higher value.

## Using the derived formula for Black-Scholes at a point for comparison

Let's take a look at the Black-Scholes derivation which gives us the value of a option at a point.

First some definitions. N(d) is the cumulative distribution function of the standard normal distribution.

$$N(d) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^d e^{-\frac{1}{2}x^2} dx$$

The formula for a call option is (please see the references below for more information):

$$C(S,t) = SN(d_1) - Ke^{-r(T - t)} N(d_2)$$

Where $d_1$ and $d_2$ are the following

$$d_1 = \frac{\ln \left(\frac{S}{K} \right) + \left(r + \frac{\sigma^2}{2} \right)(T - t)}{\sigma \sqrt{T - t}}$$

$$d_2 = d_1 - \sigma \sqrt{T - t} = \frac{\ln \left(\frac{S}{K} \right) + \left(r - \frac{\sigma^2}{2}\right)(T - t)}{\sigma \sqrt{T - t}}$$

This formula is implemented below in the *call_value_bs* function.
We run it on the same problem with 5 time values to compare our results from the formula to our results from solving the PDE with Devito.

```python
#NBVAL_IGNORE_OUTPUT

# Derived formula for Black Scholes call from
# https://aaronschlegel.me/black-scholes-formula-python.html
def call_value_bs(S, K, T, r, sigma):
    N = Normal('x', 0.0, 1.0)

    d1 = (np.log(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * np.sqrt(T))
    d2 = (np.log(S / K) + (r - 0.5 * sigma ** 2) * T) / (sigma * np.sqrt(T))

    call = (S * cdf(N)(d1) - K * np.exp(-r * T) * cdf(N)(d2))
    return call

startBF = timer.time()

# Calculate truth and compare to our solution
time = [0.20, 0.40, 0.60, 0.80, 0.99]
results = []

for t_call in time:
    call_price = []
    for x in range(int(smin), int(smax)+1):
        call_price.append(call_value_bs(x, K, t_call, r, sigma).evalf(5))
    results.append(call_price)

endBF = timer.time()

print("devito pde    timesteps: %12.6s, %12.6fs runtime" % (nt-1, endDevito - startDevito))
print("call_value_bs timesteps: %12.6s, %12.6fs runtime" % (len(time), endBF - startBF))

s2 = np.linspace(smin, smax, shape[0])
plt.figure(figsize=(12,10))

colors = ["blue", "green", "gold", "darkorange", "red"]
plt.plot(s2, v.data[0,:], '-', color="black", label='initial condition', linewidth=1)

for i in range(len(time)):
    plt.plot(s2, results[i],    ':', color=colors[i], label='truth t='+str(time[i]), linewidth=3)
    plt.plot(s2, v.data[int(time[i]*nt),:], '-', color=colors[i], label='pde t='+str(time[i]), linewidth=1)

plt.xlim([smin+padding,smax-padding])
plt.ylim([0,ymax])

plt.legend()
plt.grid(True)
plt.xlabel("stock value")
plt.ylabel("call value")
plt.title("Call black scholes")
plt.tight_layout()
```

The graph above shows the results from *call_value_bs* as a dotted line and the results from solving the PDE with devito as a solid line.

Comparing runtime of this function to Devito's runtime is like comparing apples to oranges, but here is a quick comparison anyway. On a machine with a Ryzen 1800x with 8 cores, with 10000 time values ranging from 0.0 to 1.0, Devito takes **0.042 seconds** to run the problem. With 5 time values between 0 and 1, running this function on the same problem takes about 4 seconds. This gives a reasonable idea of Devito's performance in comparison to the alternative solution, i.e. stupid fast.

```python
#NBVAL_IGNORE_OUTPUT

# Plot the l2 norm of the formula and our solution over time
t_range = np.linspace(dt0,1.0,50)
x_range     = range(padding, smax-smin-padding*2, 1)
vals = []

for t in t_range:
    l2 = 0.0
    for x in x_range:
        truth = call_value_bs(x+smin, K, t, r, sigma)
        val   = v.data[int(t*(nt-1)), x]
        l2   += (truth - val)**2

    rms = np.sqrt(np.float64(l2 / len(x_range)))
    vals.append(rms)

plt.figure(figsize=(12,10))
plt.plot(t_range, np.array(vals))
```

```python
#NBVAL_IGNORE_OUTPUT

np.mean(vals)
```

    0.00581731890853893

Above is a plot of the RMS error of our results from *call_value_bs* and Devito at a time t to expiration, which stays below 0.02 at all time values. The mean RMS error over all time is 0.0073, which is well below a tenth of a percent error.

## References

* http://www.math.unl.edu/~sdunbar1/MathematicalFinance/Lessons/BlackScholes/Solution/solution.pdf
* https://aaronschlegel.me/black-scholes-formula-python.html


# Unconstrained Scalar Optimization

Introductory calculus courses introduce the minimization (or maximization) of a function of a single variable. Given a function $f(x)$, find values $x^*$ such that $f(x^*) \leq f(x)$ (or $f(x^*) \geq f(x)$) for all $x$ in an interval containing $x^*$. Such points are called local optima.

If the derivative exists at all points in a given interval, then the local optima are found by solving for values $x^*$ that satisfy

\begin{align}
f'(x^*) = 0
\end{align}

Let's see how this is put to work in the context of process engineering.

## Application: Maximizing Production of a Reaction Intermediate

A desired product $B$ is produced as intermediate in a series reaction

\begin{align}
A \overset{k_A}{\longrightarrow} B \overset{k_B}{\longrightarrow} C
\end{align}

where $A$ is a raw material and $C$ is a undesired by-product. The reaction operates at temperature where the rate constants are $k_A = 0.5\ \mbox{min}^{-1}$ and $k_A = 0.1\ \mbox{min}^{-1}$. The raw material is available as a solution with concenration $C_{A,f} = 2.0\ \mbox{moles/liter}$.

A 100 liter tank is avialable to run the reaction. Below we will answer the following questions:

1. If the goal is obtain the maximum possible concentration of $B$, and the tank is operated as a continuous stirred tank reactor, what should be the flowrate?

2. What is the production rate of $B$ at maximum concentration?

## Mathematical Model for a Continuous Stirred Tank Reactor

The reaction dynamics for an isothermal continuous stirred tank reactor with a volume $V = 40$ liters and feed concentration $C_{A,f}$ are modeled as

\begin{align}
V\frac{dC_A}{dt} & = q(C_{A,f} - C_A) - V k_A C_A \\
V\frac{dC_B}{dt} & = - q C_B + V k_A C_A - V k_B C_B
\end{align}

At steady-state the material balances become

\begin{align}
0 & = q(C_{A,f} - \bar{C}_A) - V k_A \bar{C}_A \\
0 & = - q \bar{C}_B + V k_A \bar{C}_A - V k_B \bar{C}_B
\end{align}

which can be solved for $C_A$

\begin{align}
\bar{C}_A & = \frac{qC_{A,f}}{q + Vk_A} \\
\end{align}

and then for $C_B$

\begin{align}
\bar{C}_B & = \frac{q V k_A C_{A,f}}{(q + V k_A)(q + Vk_B)}
\end{align}

The numerator is first-order in flowrate $q$, and the denominator is quadratic.  This is consistent with an intermediate value of $q$ corresponding to a maximum concentration $\bar{C}_B$.

The next cell plots $\bar{C}_B$ as a function of flowrate $q$.

```
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

V = 40     # liters
kA = 0.5   # 1/min
kB = 0.1   # l/min
CAf = 2.0  # moles/liter

def cstr(q):
    return q*V*kA*CAf/(q + V*kB)/(q + V*kA)

q = np.linspace(0,30,200)
plt.plot(q, cstr(q))
plt.xlabel('flowrate q / liters per minute')
plt.ylabel('concentration C_B / moles per liter')
plt.title('Outlet concentration for a CSTR')
plt.show()
```

We see that, for the parameters given, there is an optimal flowrate somewhere between 5 and 10 liters per minute.

## Analytical Solution using Calculus

As it happens, this problem has an interesting analytical solution that can be found by hand, and which can be used to check the accuracy of numerical solutions. Setting the first derivative of $\bar{C}_B$ to zero,

\begin{align}
\left.\frac{d\bar{C}_B}{dq}\right|_{q^*} = \frac{V k_A C_{A,f}}{(q^* + V k_A)(q^* + Vk_B)} - \frac{q^* V k_A C_{A,f}}{(q^* + V k_A)^2(q^* + Vk_B)} - \frac{q^* V k_A C_{A,f}}{(q^* + V k_A)(q^* + Vk_B)^2} = 0
\end{align}

Clearing out the non-negative common factors yields

\begin{align}
1 - \frac{q^*}{(q^* + V k_A)} - \frac{q^*}{(q^* + Vk_B)} = 0
\end{align}

and multiplying by the non-negative denominators produces

\begin{align}
{q^*}^2 + q^*V(k_A + k_B) + V^2k_Ak_B - q^*(q^* + Vk_B) - q^*(q^* + Vk_A) = 0
\end{align}

Expanding these expressions followed by arithmetic cancelations gives the final result

\begin{align}
q^*  =  V\sqrt{k_Ak_B}
\end{align}

which shows the optimal dilution rate, $\frac{q^*}{V}$, is equal the geomtric mean of the rate constants.

```
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

V = 40     # liters
kA = 0.5   # 1/min
kB = 0.1   # l/min
CAf = 2.0  # moles/liter

qmax = V*np.sqrt(kA*kB)
CBmax = cstr(qmax)
print('Flowrate at maximum CB = ', qmax, 'liters per minute.')
print('Maximum CB =', CBmax, 'moles per liter.')
print('Productivity = ', qmax*CBmax, 'moles per minute.')
```

    Flowrate at maximum CB =  8.94427190999916 liters per minute.
    Maximum CB = 0.9549150281252629 moles per liter.
    Productivity =  8.541019662496845 moles per minute.

## Numerical Solution with Pyomo

This problem can also be solved using Pyomo to create a model instance. First we make sure that Pyomo and ipopt are installed, then we proceed with the model specification and solution.

```
!pip install -q pyomo
!wget -N -q "https://ampl.com/dl/open/ipopt/ipopt-linux64.zip" && unzip -o -q ipopt-linux64
ipopt_executable = '/content/ipopt'
```

```
from pyomo.environ import *

V = 40     # liters
kA = 0.5   # 1/min
kB = 0.1   # l/min
CAf = 2.0  # moles/liter

# create a model instance
m = ConcreteModel()

# create the decision variable
m.q = Var(domain=NonNegativeReals)

# create the objective
m.CBmax = Objective(expr=m.q*V*kA*CAf/(m.q + V*kB)/(m.q + V*kA), sense=maximize)

# solve using the nonlinear solver ipopt
SolverFactory('ipopt', executable=ipopt_executable).solve(m)

# print solution
print('Flowrate at maximum CB = ', m.q(), 'liters per minute.')
print('Maximum CB =', m.CBmax(), 'moles per liter.')
print('Productivity = ', m.q()*m.CBmax(), 'moles per minute.')
```

    Flowrate at maximum CB =  8.944271964904415 liters per minute.
    Maximum CB = 0.9549150281252627 moles per liter.
    Productivity =  8.541019714926698 moles per minute.

One advantage of using Pyomo for solving problems like these is that you can reduce the amount of algebra needed to prepare the problem for numerical solution. This not only minimizes your work, but also reduces possible sources of error in your solution.

In this example, the steady-state equations are

\begin{align}
0 & = q(C_{A,f} - \bar{C}_A) - V k_A \bar{C}_A \\
0 & = - q \bar{C}_B + V k_A \bar{C}_A - V k_B \bar{C}_B
\end{align}

with unknowns $C_B$ and $C_A$. The modeling strategy is to introduce variables for the flowrate $q$ and these unknowns, and introduce the steady state equations as constraints.

```
from pyomo.environ import *

V = 40     # liters
kA = 0.5   # 1/min
kB = 0.1   # l/min
CAf = 2.0  # moles/liter

# create a model instance
m = ConcreteModel()

# create the decision variable
m.q  = Var(domain=NonNegativeReals)
m.CA = Var(domain=NonNegativeReals)
m.CB = Var(domain=NonNegativeReals)

# equations as constraints
m.eqn = ConstraintList()
m.eqn.add(0 == m.q*(CAf - m.CA) - V*kA*m.CA)
m.eqn.add(0 == -m.q*m.CB + V*kA*m.CA - V*kB*m.CB)

# create the objective
m.CBmax = Objective(expr=m.CB, sense=maximize)

# solve using the nonlinear solver ipopt
SolverFactory('ipopt', executable=ipopt_executable).solve(m)

# print solution
print('Flowrate at maximum CB = ', m.q(), 'liters per minute.')
print('Maximum CB =', m.CBmax(), 'moles per liter.')
print('Productivity = ', m.q()*m.CBmax(), 'moles per minute.')
```

    Flowrate at maximum CB =  8.944272002876573 liters per minute.
    Maximum CB = 0.9549150281377385 moles per liter.
    Productivity =  8.541019751298471 moles per minute.

```

```


# SEIRHVD model example
## Work in progress (equations not ready)
\begin{align}
\dot{S} & = S_f - \alpha\beta\frac{SI}{N+k_I I+k_R R} + r_{R\_S} R\\
\dot{E} & = E_f + \alpha\beta\frac{SI}{N+k_I I+k_R R} - E\frac{1}{t_{E\_I}} \\
\dot{I} & = I_f + E\frac{1}{t_{E\_I}} - I\frac{1}{t_{I\_R}} \\
\dot{R} & = R_f + I\frac{1}{t_{I\_R}} - r_{I\_R} R\\
\end{align}
Where:
* $S:$ Susceptible
* $E:$ Exposed
* $I:$ Infectious
* $R:$ Removed
* $\alpha:$ Mobilty
* $\beta:$ Infection rate
* $N:$ Total population
* $t_{E\_I}:$ # Transition time between exposed and infectious
* $t_{I\_R}:$ # Transition time between infectious and recovered
* $r_{R\_S}:$ Immunity loss rate ($\frac{1}{t_{R\_S}}$)
* $S_f,E_f,I_f,R_f:$ External flux
* $k_I:$ Infected saturation
* $k_R:$ Immunity shield

```python
# Util libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
# Adding lib paths

# cv19 libraries
from cv19gm.models.seirhvd import SEIRHVD
from cv19gm.utils import cv19functions
```

```python
# For pop-up plots execute this code (optional)
import platform
OS = platform.system()

if OS == 'Linux':
    %matplotlib tk
    print('Linux')
elif OS == 'Windows':
    %matplotlib qt
    print('Windows')
elif OS == 'Darwin':
    %matplotlib tk
    print('Mac (Funciona?)')
```

# Variable CFR

## Discrete change
* pH_r de 0.7 a 0.6
* pH_D de 0.3 a 0.4

```python
pH_R = cv19functions.events(values=[0.7,0.5],days=[[0,30],[30,500]])
pH_D = cv19functions.events(values=[0.3,0.5],days=[[0,30],[30,500]])
```

```python
%%capture
# Input configuration file
config = 'cfg/SEIRHVD.toml'
# Build simulation object
model1 = SEIRHVD(config = config, H_cap=4000, pH_R = pH_R,pH_D = pH_D)
# Simulate (solve ODE)
model1.solve()
```

```python
t = np.linspace(0,50,1000)
plt.plot(t,100*pH_R(t),label='pH_R')
plt.plot(t,100*pH_D(t),label='pH_D')
plt.plot(t,100*pH_D(t)*model1.pE_Icr(t),label='CFR')
plt.legend(loc=0)
plt.title('CFR change (%)')
plt.show()
```

```python
t = model1.t
plt.plot(t,100*model1.pH_R(t),label='pH_R')
plt.plot(t,100*model1.pH_D(t),label='pH_D')
plt.plot(t,100*model1.CFR,label='CFR')
plt.xlim(0,50)
plt.legend(loc=0)
plt.title('CFR change (%)')
plt.show()
```

* Nota: la transicion pareciera ocurrir en 2 días, pero tiee que ver con la resolución en la cual se entregan los datos, la transición que utiliza el integrador es "instantánea".

```python
# Plot matplotlib
fig, axs = plt.subplots(figsize=(13,9),linewidth=5,edgecolor='black',facecolor="white")
axs2 = axs.twinx()

axs.plot(model1.t,model1.D_d,color='tab:red',label='Daily deaths')
axs.set_ylabel('Deaths',color='tab:red')
axs.tick_params(axis='y', labelcolor='tab:red')

t = model1.t
axs2.plot(t,100*pH_D(t)*model1.pE_Icr(t),color='tab:blue',label='CFR')
axs2.set_ylabel('CFR',color='tab:blue')
axs2.tick_params(axis='y',labelcolor='tab:blue')

axs.set_xlim(0,200)
axs2.set_xlim(0,200)
fig.legend(loc=8)
fig.suptitle('CFR vs Deaths')
fig.show()
```

## Continious change
* pH_r de 0.7 a 0.6
* pH_D de 0.3 a 0.4

```python
pH_R = cv19functions.sigmoidal_transition(t_init=20,t_end=40,initvalue = 0.7, endvalue = 0.5)
pH_D = cv19functions.sigmoidal_transition(t_init=20,t_end=40,initvalue = 0.3, endvalue = 0.5)
```

```python
%%capture
# Input configuration file
config = 'cfg/SEIRHVD.toml'
# Build simulation object
model2 = SEIRHVD(config = config, H_cap=4000, pH_R = pH_R,pH_D = pH_D)
# Simulate (solve ODE)
model2.solve()
```

```python
t = model2.t
plt.plot(t,100*model2.pH_R(t),label='pH_R')
plt.plot(t,100*model2.pH_D(t),label='pH_D')
plt.plot(t,100*model2.CFR,label='CFR')
plt.xlim(0,50)
plt.legend(loc=0)
plt.title('CFR change (%)')
plt.show()
```

```python
# Plot matplotlib
fig, axs = plt.subplots(figsize=(13,9),linewidth=5,edgecolor='black',facecolor="white")
axs2 = axs.twinx()

axs.plot(model2.t,model2.D_d,color='tab:red',label='Daily deaths')
axs.set_ylabel('Deaths',color='tab:red')
axs.tick_params(axis='y', labelcolor='tab:red')

t = model2.t
axs2.plot(t,100*pH_D(t)*model2.pE_Icr(t),color='tab:blue',label='CFR')
axs2.set_ylabel('CFR',color='tab:blue')
axs2.tick_params(axis='y',labelcolor='tab:blue')

axs.set_xlim(0,200)
axs2.set_xlim(0,200)
fig.legend(loc=8)
fig.suptitle('CFR vs Deaths')
fig.show()
```

# Access CFR value

## As a variable

```python
model2.CFR
```

## As part of the pandas array

```python
model2.results['CFR']
```

```python

```


原点からずれた点を中心に,、近似的に円状に分布する点を生成する関数

```
import numpy as np
from typing import List

# 再現性のためシードは固定
np.random.seed(42)

def get_point() -> List[float]:
    r = 10. + np.random.normal(0,0.5)
    center = [0.1, -0.2]

    theta = np.random.uniform(0, 2*np.pi)
    return [r*np.cos(theta)+center[0], r*np.sin(theta)+center[1]]
get_point()
```

    [-1.0569801383343806, -10.382839472714007]

データ生成

```
data_list = []
for k in range(50):
    data_list.append(get_point())

data = np.array(data_list)
display(data)
```

    array([[ -7.98315673,  -5.96929055],
           [  9.33204952,   3.32724182],
           [  6.69030475,  -7.56479591],
           [ 10.79948867,   1.19159928],
           [ 10.29868946,  -2.15149162],
           [  4.15625806,   8.68296843],
           [  4.27350919,   9.1851486 ],
           [ -9.109716  ,   3.99706926],
           [ -2.21657133,   8.54161628],
           [ -2.38464942,   8.96267771],
           [ -6.68197306,   7.36121701],
           [  3.43754038,  10.00070309],
           [ -9.74759427,  -1.08310126],
           [ -7.4901633 ,  -6.28438016],
           [  4.91521092,   8.62757303],
           [ -3.79506153,  -9.08325957],
           [ -9.06563139,   3.41877536],
           [  9.86088502,   1.94247449],
           [  8.07500899,  -5.3089931 ],
           [ -3.83649296,   9.43839288],
           [ -9.21503473,  -1.38080745],
           [  9.26594555,  -1.97330238],
           [  1.68806597, -10.17278033],
           [  8.9072666 ,   5.27293834],
           [  3.45773688,   9.31034319],
           [ -7.50764529,   6.2008276 ],
           [ -1.21724559,   9.56096783],
           [ -1.78697224,   9.38571829],
           [-10.0519712 ,  -2.99071315],
           [  9.16743522,   4.38788794],
           [  9.87418916,  -1.00714034],
           [  9.75572408,   0.13515419],
           [  4.22034633,  -9.64632447],
           [  1.50116975, -10.62173011],
           [  9.45330571,   4.49524165],
           [  6.34954217,  -7.46134416],
           [ -6.93578852,  -7.08690491],
           [ -3.54899074,   8.85265299],
           [ -4.40792606,   8.62216698],
           [  8.00077347,  -6.9725982 ],
           [-10.41581035,   1.65471822],
           [  0.78936974, -10.35745169],
           [ -8.86897823,  -3.83459445],
           [ -9.78043412,  -1.62094406],
           [ -9.58404455,   4.54107004],
           [ -3.85143315,   9.03355517],
           [ -9.73621677,  -0.7302051 ],
           [ -8.26326779,   5.07930838],
           [  0.45494418, -10.37236564],
           [  8.77471984,  -4.30215981]])

プロット

```
import matplotlib.pyplot as plt

# アスペクト比 1:1
plt.axes().set_aspect('equal', 'datalim')
# 散布図
plt.scatter(data[:,0],data[:,1])
plt.show()
```

このデータを中心・半径が未知の円として非線形フィッティング。
MSSは
\begin{equation}
  f(x_0,y_0,r) = \frac{1}{N}\sum_{k=1}^N
    \left[\frac{(x_i-x_0)^2+(y_i-y_0)^2}{r^2} - 1\right]^2
\end{equation}
となる。
ただし計算の都合上、無次元で平方根が含まれない形になっている。
この関数の最適化を最急降下法で行う。

```
def residue(x0, y0, r) -> float:
    result = 0.
    for k in range(len(data)):
        result += (((data[k,0]-x0)**2+(data[k,1]-y0)**2)/r**2-1)**2
    return result/len(data)
residue(0.1, -0.2, 10.)
```

    0.005860849564807571

```
def back_tracking():
    rho = 0.5
```


# Workshop 12: Introduction to Numerical ODE Solutions
*Source: http://phys.csuchico.edu/ayars/312 *

**Submit this notebook to bCourses to receive a grade for this Workshop.**

Please complete workshop activities in code cells in this iPython notebook. The activities titled **Practice** are purely for you to explore Python. Some of them may have some code written, and you should try to modify it in different ways to understand how it works. Although no particular output is expected at submission time, it is _highly_ recommended that you read and work through the practice activities before or alongside the exercises. However, the activities titled **Exercise** have specific tasks and specific outputs expected. Include comments in your code when necessary. Enter your name in the cell at the top of the notebook. The workshop should be submitted on bCourses under the Assignments tab.

$\color{red}{\large\text{Comments}}$
Ex 1: 4 pts
Ex 2: 3 pts
Ex 3: 3 pts

```python
# Run this cell before preceding
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
```

### Exercise 1:

The damped harmonic oscillator (DHO) satisfies the following differential equation:

$$\frac{d^2x}{dt^2}+\frac{c}{m}\frac{dx}{dt}+\frac{k}{m}x = 0$$

It differs from the previous example by the addition of the $(c/m) dx/dt$ term. Like we did above, we can unwrap this second-order ODE into two first-order ODEs using two separate variables $x(t)$ and $v(t)$

\begin{align}
x' &= v \\
v' &= -\frac{c}{m}v - \frac{k}{m}x
\end{align}

1. Like in the example above, write down the update rules for $x_i$ and $v_i$.
1. Then write some code to implement your rules to estimate a numerical solution for $x(t)$ and $v(t)$ for a given initial condition $x_0$ and $v_0$ (you can assume $t_0 = 0$ like above).
1. Plot your results for $x(t)$ and $v(t)$ and make sure that they make sense. You may use the code in the example as a template.

*Hint*: Recall that the qualitative behavior of the oscillator is different depending on the (dimensionless) value of the ratio

$$\frac{(c/m)^2}{k/m}$$

So you should be able to see the effect of this by trying out different values for $c/m$ and $k/m$.

```python
# Use Euler method to solve coupled first order ODE
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt

# Initial conditions
x_0 = 1.0
v_0 = 0.0

# Number of timesteps
T = 1000
dt = 0.1 #size of time step (Delta t)

def Euler(t0, x0, v0, T, dt, cm, km):
    x_data = np.zeros(T)
    v_data = np.zeros(T)

    t_data = np.arange(T) * dt + t0
    x_data[0] = x0
    v_data[0] = v0

    for i in range(1,T):
        x_data[i] = x_data[i-1] + v_data[i-1] * dt
        v_data[i] = v_data[i-1] + (-cm * v_data[i - 1] - km * x_data[i-1]) * dt

    return t_data, x_data, v_data

km = 0.3 # value of k / m
cms = [.3, 1.095] # values of c/m
labels = ['underdamped', 'critically damped']

plt.figure(figsize=(8,8))
x_0 = 1.0

plt.subplot(211)
plt.ylabel("Position")

plt.title("Position of the mass on damped spring")

plt.subplot(212)
plt.ylabel("Velocity")
plt.xlabel("Time")

plt.title("Velocity of the mass on damped spring")

for i, cm in enumerate(cms):

# Analytical solutions for (x(t), v(t)) assuming x_0 = 1.0, v_0 = 0, t_0 = 0
    D = x_0/np.sqrt(1 - cm**2/(4*km))
    y = cm/2
    w = (km - 1/4 * cm**2)**(1/2)
    phi = np.arctan2(-y, w)

    t_data, x_data, v_data = Euler(0.0, x_0, v_0, T, dt, cm, km)
    analytical_x = D * np.exp(-y * t_data) * np.cos(w*t_data + phi)
    analytical_v = -y * D * np.exp(-y * t_data) * np.cos(w*t_data + phi) \
    - w * D * np.exp(-y*t_data) * np.sin(w*t_data + phi)

    plt.subplot(211)
    plt.plot(t_data, x_data, label="numerical " + labels[i])
    plt.plot(t_data, analytical_x,label="analytical " + labels[i])
    plt.legend()

    plt.subplot(212)
    plt.plot(t_data, v_data, label="numerical " + labels[i])
    plt.plot(t_data, analytical_v, label="analytical " + labels[i])
    plt.legend()

overdamped_cm = 2
t_data, x_data, v_data = Euler(0.0, x_0, v_0, T, dt, overdamped_cm, km)
plt.subplot(211)
plt.plot(t_data, x_data, label="numerical overdamped")
plt.legend()

plt.subplot(212)
plt.plot(t_data, v_data, label="numerical overdamped")
plt.legend()

```

The ratio $\frac{cm^2}{km}$ determines whether the oscillator is underdamped, critically damped, or overdamped

## But wait...

But you know that for a closed system, like the SHO, we actually have a special constraint on the system--the total energy (kinetic + potential) must be constant! So at every point of our solution, we should check whether this is true. How do we evaluate the total energy?

$$E = T + U = \frac{1}{2}mv^2 + \frac{1}{2}kx^2$$

Let's define a rescaled energy $\tilde{E}$ as $(1/m)E$:

$$\tilde{E} = \frac{1}{2}v^2 + \frac{1}{2}\frac{k}{m} x^2$$

### Exercise 2:

1. Copy the code from the example using the SHO above, in which we solved the SHO using the Euler Method. Add code to calculate the rescaled energy $\tilde{E}_i$ for each time step.
1. Plot $\tilde{E}(t)$ vs. the time. Does the energy stay constant, fluctuate around some constant value, or does it diverge/decay?

```python
# Use Euler method to solve coupled first order ODE
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import scipy.optimize as sp
plt.rcParams['font.size'] = 14

km = 0.3 # value of k / m

# Initial conditions
x_0 = 1.0
v_0 = 0.0

# Number of timesteps
T = 1000
dt = 0.1 #size of time step (Delta t)

def Euler(t0, x0, v0, T, dt):
    x_data = np.zeros(T)
    v_data = np.zeros(T)
    E_data = np.zeros(T)

    t_data = np.arange(T) * dt + t0
    x_data[0] = x0
    v_data[0] = v0
    E_data[0] = 1/2 * v0**2 + 1/2 * km * x0**2

    for i in range(1,T):
        x_data[i] = x_data[i-1] + v_data[i-1] * dt
        v_data[i] = v_data[i-1] + (-km * x_data[i-1]) * dt
        E_data[i] = 1/2 * v_data[i]**2 + 1/2 * km * x_data[i]**2

    return t_data, x_data, v_data, E_data

t_data, x_data, v_data, E_data = Euler(0.0, x_0, v_0, T, dt)

def linear(x, a, b):
    return a * x + b

par, _ = sp.curve_fit(linear, t_data, np.log(E_data))

plt.plot(t_data, E_data, 'b', linewidth = 8, label = 'eulor method energy')
plt.plot(t_data, np.exp(linear(t_data, *par)), 'r', linewidth = 3, label = '$E = {:.2f}e^{{{:.2f}t}}$'.format(np.exp(par[1]),par[0]))
plt.xlabel('time')
plt.ylabel('energy')
plt.legend()
```

* the Euler method yields an exponentially increasing energy

## Euler-Cromer/Symplectic Euler Method

There exists a way to keep the energy fluctuations from growing, using a just a slight variant of the update rules described above. This update rule is called the

\begin{align}
v_i &= v_{i-1} + \left(-\frac{k}{m}x_{i-1}\right)\Delta t \\
x_i &= x_{i-1} + v_{i} \Delta t
\end{align}

In this version, you use the approximate velocity at time $t_i$ instead of the velocity at time $t_{i-1}$ to calculate $x_i$.

### Exercise 3:

1. Modify the code from Exercise 2 to instead implement the update rule in the Euler-Cromer method. You can either modify the it in-place or copy it to the cell below and modify it.
1. Now run your code to calculate and plot $x(t)$, $v(t)$, and $\tilde{E}(t)$. Does the energy stay constant, fluctuate around some constant value, or does it diverge/decay?

```python
# Use Euler method to solve coupled first order ODE
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams['font.size'] = 14

km = 0.3 # value of k / m

# Initial conditions
x_0 = 1.0
v_0 = 0.0

# Number of timesteps
T = 1000
dt = 0.1 #size of time step (Delta t)

def Euler(t0, x0, v0, T, dt):
    x_data = np.zeros(T)
    v_data = np.zeros(T)
    E_data = np.zeros(T)

    t_data = np.arange(T) * dt + t0
    x_data[0] = x0
    v_data[0] = v0
    E_data[0] = 1/2 * v0**2 + 1/2 * km * x0**2

    for i in range(1,T):
        v_data[i] = v_data[i-1] + (-km * x_data[i-1]) * dt
        x_data[i] = x_data[i-1] + v_data[i] * dt
        E_data[i] = 1/2 * v_data[i]**2 + 1/2 * km * x_data[i]**2

    return t_data, x_data, v_data, E_data

t_data, x_data, v_data, E_data = Euler(0.0, x_0, v_0, T, dt)
print('Amplitude = {:.3f}'.format((np.max(E_data) - E_data[0])))
print('Maximum fractional error error = {:.2f}'.format((np.max(E_data) - E_data[0])/E_data[0]))
plt.plot(t_data, E_data)
plt.xlabel('time')
plt.ylabel('energy')
```

* the energy oscillates about an equilibrium value of .15 with an amplitude of .004
* the maximum fractional error in the energy is .03