# PAA Project, a.y. 2015/2016
## Average number of checks for Quicksort

Authors:
- Tommaso Papini, 5537529 ([mail][mail-tom] [Github][git-tom])
- Gabriele Bani, 5719258 ([mail][mail-gab] [Github][git-gab])

[mail-tom]: mailto:tommaso.papini1@stud.unifi.it
[git-tom]: https://github.com/oddlord
[mail-gab]: mailto:gabriele.bani@stud.unifi.it
[git-gab]: https://github.com/brnibani

### Introduction
The goal of this notebook is to provide an explanation on how to get the formula to calculate the average number of checks done by the [Quicksort][quicksort] algorithm on arrays of length $n$.

In particular this notebook will be very [symbolic computation][symb-comp] oriented, which means that most of the passages needed to get to the final form of our formula will be done using a symbolic computation package for [Python][python], called [sympy][sympy].

With sympy we'll be able to rewrite and manipulate our recurrence to finally obtain the goal function that computes the average number of checks, given the array length $n$.

[quicksort]: https://en.wikipedia.org/wiki/Quicksort
[symb-comp]: https://en.wikipedia.org/wiki/Symbolic_computation
[python]: https://www.python.org/
[sympy]: http://www.sympy.org/en/index.html

### Some initial imports
Before with start with the cool stuff, let's first import the basic packages we're gonna need for the rest of the notebook!

```python
import itertools
import matplotlib.pyplot as plt
import numpy as np
import sympy
from sympy import *
from sympy.abc import x, n, z, t, j, k
from sympy.combinatorics import *
import timeit

%matplotlib inline
init_printing()
```

The call $init\_printing()$ is to allow sympy outputs to be displayed with LaTeX style.

### Quicksort implementation with checks counting
Here we provide the Quicksort implementation that will be used for the rest of the notebook.

As you might know, the number of checks/swaps might vary greatly from one implementation to another of the same algorithm, so it's mandatory to set in stone right away which specific implementation we're gonna use.

In this implementation the inputs $v$, $l$ and $r$ represents, respectively, the array to be sorted and the left and right indexes of the array.
The pivot is always the rightmost element of the array, i.e. $v[r]$.

Also, we included a checks counting, so that later on we can also run some simulation to gather some actual number to compare with the function output (and check if the results are consistent!).
The algorithm will return the number of checks done $nc$.

```python
def quick(v,l,r):
    nc = 0;
    if (r - l) >= 0:
        p = v[r]
        i = l - 1
        j = r
        while i < j:
            i += 1
            nc += 1
            while v[i] < p:
                i += 1
                nc += 1
            j -= 1
            nc += 1
            while l <= j and p < v[j]:
                j -= 1
                nc += 1
            if i < j:
                a = v[i]
                v[i] = v[j]
                v[j] = a
        a = v[i]
        v[i] = p
        v[r] = a
        nc += quick(v,l,i-1)
        nc += quick(v,i+1,r)
    return nc
```

### Closed form function
Let's define the closed form of the function we're looking for, just to give a glimpse of what we're doing with a few examples.

The function is
$$C_n = 2(n+1)(H_{n+1}-1),$$
where $C_n$ is the average checks with arrays of length $n$ and $H_{n}$ is the $n$-th [harmonic number][harm-num].

[harm-num]: https://en.wikipedia.org/wiki/Harmonic_number

```python
def checks_quicksort(n):
    return 2*(n+1)*(harmonic(n+1).evalf()-1)
```

Getting this closed form of $C_n$ is the main goal of this notebook.

### A couple of examples
We provide here a couple of examples where we compare the result of the closed form function defined above with the actual number of checks done by our implementation of Quicksort on a random array.

```python
random_perm = np.random.permutation(20)
print ("Random array: %s" % (random_perm))
checks = quick(random_perm,0,19)
print ("Array after Quicksort: %s" % (random_perm))
print ("Quicksort count: %.5f; closed-form: %.5f" % (checks, checks_quicksort(20)))
```

    Random array: [16 17  8  1  7 19 14 15 18  9  2 13  0 11  5  3 12  6 10  4]
    Array after Quicksort: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]
    Quicksort count: 110.00000; closed-form: 111.10507

```python
random_perm = np.random.permutation(4)
print ("Random array: %s" % (random_perm))
checks = quick(random_perm,0,3)
print ("Array after Quicksort: %s" % (random_perm))
print ("Quicksort count: %.5f; closed-form: %.5f" % (checks, checks_quicksort(4)))
```

    Random array: [0 1 3 2]
    Array after Quicksort: [0 1 2 3]
    Quicksort count: 12.00000; closed-form: 12.83333

### Main recurrence
Let the dance commence!

The first thing we need is to define a recurrence that describes the average number of checks done by our Quicksort implementations.

Why a recurrence, you might ask? because the Quicksort is a recursive algorithm and recurrences are the mathematical counterpart of recursive algorithms: a recursive algorithm is described by a recurrence and a recurrence can be evaluated by running a recursive algorithm.

But evaluating a recurrence can be sometimes very expensive, in terms of both space and time. So what we usually try to do is to **solve** the recurrence, which means try to rewrite and manipulate the recurrence until we obtain a closed form.
Since the closed form is more direct and does not involve calling the same function over and over multiple times, it's often preferrable to the recurrence evaluation as it'll save precious resources.

To obtain the closed form of the recurrence that describes the average number of checks in the Quicksort is the goal of this notebook.
In particular we'll see two different approaches to get from the recurrence to the closed form: the *difference method* and the *genetaring function method*.

#### Getting the recurrence
So let's start with defining the recurrence that describes the average number of checks done by our Quicksort implementation.

How many checks does a single Quicksort call do?
$$n+1$$
In fact we can notice that our algorithm does a check each time the indexes $i$ and $j$ are increased/decreased. Since these indexes have to scan all the elements that are not the pivot until they cross, we have $n-1$ checks (for $n-1$ elements that are not the pivot) plus $2$ checks to let the indexes cross, for a total of $n+1$.

Now we have to consider the checks done by the two recursive calls.
If the pivot is $k$, then the checks of the two calls will be given by
$$C_{k-1}+C_{n-k}$$

But since we're analyzing the average number of checks, we have to consider every possible configuration we can find. The number of checks done by the two recursive calls depends on the value of the pivot, i.e. the rightmost element, that can have $n$ different values (from $1$ to $n$).

Also, among the $n!$ possible permutations, only $(n-1)!$ have $k$ in the last position. This means that the probability of having a combination with $k$ as pivot is:
$$\frac{(n-1)!}{n!}=\frac{1}{n}$$

If we put everything together we finally obtain the recurrence:
$$C_n = n+1+\frac{1}{n}\sum_{k=1}^{n}(C_{k-1}+C_{n-k})$$
With initial contition $C_0 = 0$.

Before we can move on to the next step, we can notice that inside the summation, each recursive call appears exactly twice and also that the actual indexes of the recursive calls range from $0$ to $n-1$. This means that:
$$\sum_{k=1}^{n}(C_{k-1}+C_{n-k}) = 2\sum_{k=0}^{n-1}C_k$$

That said, we can rewrite the recurrence in a much simpler way:
$$C_n = n+1 + \frac{2}{n}\sum_{k=0}^{n-1}C_k$$

#### Python version
Below we provide a python implementation of the above recurrence, so that we can later compare it with the results of the closed form function and the actual checks from the algoritms.

```python
def C_py(n):
    if n == 0:
        return 0
    else:
        Cn = 0.0
        for k in range(0, n):
            Cn += C_py(k)
        return n+1+(2.0/n)*Cn
```

```python
random_perm = np.random.permutation(20)
quicksort_count = quick(random_perm,0,19)
checks = checks_quicksort(20)
rec_quicksort = C_py(20)
print ("Quicksort count: %.5f; closed-form: %.5f; recurrence: %.5f" % (quicksort_count, checks, rec_quicksort))
```

    Quicksort count: 106.00000; closed-form: 111.10507; recurrence: 111.10507

#### sympy version
Let's start using sympy, the python package for symbolic computation.

We start by defining the recurrence as seen above, in sympy sintax.

```python
Cn = Function('C_n')
base_rec = Eq(Cn(n), n+1+(2/n)*Sum(Cn(k), (k, 0, n-1)))
base_rec
```

Please observe that, in order to manipulate this succession with sympy, we have to express $C_n$ as a function, i.e. with the parenthesis notation. Also, to distinguish it from a similar notation that we'll introduce further on the notebook, we'll also keep the subscript $n$.

So, from this point on, we'll assume that $C_k = C_n(k)$.

#### Are we lucky?
sympy provides a function called *rsolve*, which tries to automatically solve a recurrence and calculate it's closed form.

Let's try to see if we are lucky and our recurrence is easy enough to be solved automatically by sympy.

```python
print (rsolve(base_rec, Cn(n), {Cn(0):0}))
```

    None

Nope... :(

So let's begin with the first approach, the *difference method*.

### Recurrence solving: difference method
The first method we're gonna use to solve the recurrence is called *difference method*.

The idea of the difference method is to express the recurrence in two generic successive points, let them be $n-1$ and $n$, and then express the difference of the recurrence in these two point, with the hope to obtain a simpler recurrence.

#### Auxiliary function to apply the same operation between a number and the two sides of an Eq
Before we start, let's define an useful auxiliary function that lets us apply a binary operation $op$ to the two sides of an equation $eq$ (first argument of $op$) and a number $n$ (second argument of $op$).

```python
def op_eq_n(eq, n, op):
    return Eq(op(eq.lhs, n), op(eq.rhs, n))
```

#### Multiply by $n$ both sides
We use that auxiliary function right away, to multiply both sides of the equation by $n$.

```python
base_rec2 = op_eq_n(base_rec, n, lambda x,y: x*y).simplify()
base_rec2
```

#### Auxiliary function to factor a Sum and substitute
Since in sympy there's currently no supported feature to factor constants out of Sums, we need to implement a small function that does that, as suggested [here][factor-sum].

[factor-sum]: https://github.com/sympy/sympy/issues/7902

```python
def factor_sum_subs(eq, old_sum):
    con, dep = factor_terms(old_sum.function.as_independent(*old_sum.variables));
    new_sum=con*Sum(dep, *old_sum.args[1:])
    return eq.subs(old_sum, new_sum)
```

#### Let's use it to factor that $2$ out of the Sum
That $2$ inside the summation is quite ugly, let's take it out with $factor\_sum\_subs$!

```python
base_rec3 = factor_sum_subs(base_rec2, Sum(2*Cn(k), (k, 0, n-1)))
base_rec3
```

#### The above is valid for every value of $n$, so it's valid for $n-1$ as well
Here we simply substitute $n$ with $n-1$ in the recurrence. This is legit since the recurrence we defined is valid for all $n$.

```python
base_rec3_prev = base_rec3.subs(n, n-1)
base_rec3_prev
```

#### Auxiliary function to apply an operation to two Eqs sidewise
Now we need an auxiliary function that is similar to $op_eq_n$ but where the operation $op$ is applied between two equations sidewise and not between an equation and a number.

```python
def op_eqs(eq1, eq2, op):
    return Eq(op(eq1.lhs, eq2.lhs), op(eq1.rhs, eq2.rhs))
```

#### Application of the difference method
In this passage we apply the idea of the *difference method*: we have the recurrence expressed in $n$ and in $n-1$ and we express the difference between the two now.

```python
diff_rec = op_eqs(base_rec3, base_rec3_prev, lambda x,y: x-y).simplify()
diff_rec
```

#### Auxiliary function to extract the last term of a summation and substitute inside an Eq
Now that we need some advanced manipulation power, we write a function that extracts the last term out of a summation and substitute the new summation (plus the last term explicited) inside an equation.

```python
def extract_last_term_subs(eq, old_sum):
    a = Wild('a')
    i = Wild('i')
    f = Wild('f')
    l = Wild('l')
    match = old_sum.match(Sum(a, (i, f, l)))
    new_sum = Sum(a, (i, f, l-1)).xreplace(match) + a.xreplace(match).subs(match[i], match[l])
    return eq.subs(old_sum, new_sum)
```

#### Extract term $n-1$ and simplify summations
With the above function, we can now extract the term with index $n-1$ from the second summation. This way, the two resulting summations in the recurrence will be the opposite one another and will simplify mutually.

```python
diff_rec2 = extract_last_term_subs(diff_rec, Sum(Cn(k), (k, 0, n-1))).simplify().doit()
diff_rec2
```

#### Move $(n-1)C(n-1)$ to the other side and group by $C(n-1)$
Here we move the term $(n-1)C(n-1)$ to the right side of the equation so we can factor by $C(n-1)$.

```python
new_rec = op_eq_n(diff_rec2, (n-1)*Cn(n-1), lambda x,y: x+y).simplify().factor(Cn(n-1))
new_rec
```

#### Divide both sides by $n(n+1)$

```python
div_rec  = apart(op_eq_n(new_rec, n*(n+1), lambda x,y: x/y), Cn(n-1))
div_rec
```

#### Let's unfold!
We can notice now that the last term $\frac{1}{n}C_n(n-1)$ is exactly the term of the left side of the equation but calculated on the previous point.
So we can now calculate the same recurrence for $\frac{1}{n}C_n(n-1)$ and substitute to $\frac{1}{n}C_n(n-1)$.

```python
div_rec2 = div_rec.subs(Cn(n-1)/n, div_rec.subs(n, n-1).rhs)
div_rec2
```

We can do the same with $\frac{C_n(n-2)}{n-1}$.

```python
div_rec3 = div_rec2.subs(Cn(n-2)/(n-1), div_rec.subs(n, n-2).rhs)
div_rec3
```

And so on and so forth...

#### All the way to $C_n(0)$
We unfold the summation as shown above until we reach $C_n(0)$, i.e. when we obtain
$$\frac{C_n(n)}{n+1} = \frac{2}{n+1} + \frac{2}{n} + \frac{2}{n-1} + \dots + \frac{2}{n-n+2} + \frac{C_n(0)}{n-n+1}.$$
Unfortunately sympy does not provide any summation unfold operator, so we are bound to manually rewrite the recurrence.

```python
div_rec_final = factor_sum_subs(Eq(div_rec.lhs, Sum(2/k, (k, 2, n+1))+Cn(0)/(n-n+1)), Sum(2/k, (k, 2, n+1)))
div_rec_final
```

#### But $C_n(0)=0$
We know that $C_n(0) = 0$, as it's the initial condition we set for our recurrence.

```python
div_rec_final2 = div_rec_final.subs(Cn(0), 0)
div_rec_final2
```

#### Auxiliary function to add a missing first term in a summation and substitute inside an Eq
Now we need another auxiliary function that adds a missing first term inside a summation, adding that same term but with opposite sign outside the summation to balance everything.

```python
def add_missing_first_term_subs(eq, old_sum):
    a = Wild('a')
    i = Wild('i')
    f = Wild('f')
    l = Wild('l')
    match = old_sum.match(Sum(a, (i, f, l)))
    new_sum = Sum(a, (i, f-1, l)).xreplace(match) - a.xreplace(match).subs(match[i], match[f]-1)
    return eq.subs(old_sum, new_sum)
```

#### Let's add the missing first term to the summation above
We can now add the missing first term to the previous summation. The reason we do this will soon be clear.

```python
div_rec_final3 = add_missing_first_term_subs(div_rec_final2, Sum(1/k, (k, 2, n+1))).factor()
div_rec_final3
```

#### That summation above is the harmonic series!
By definition of [harmonic series][harm-series], we have that
$$\sum_{k=1}^{n} \frac{1}{k} = H_n,$$
so we can rewrite the summation above as $H_{n+1}.$

[harm-series]: https://en.wikipedia.org/wiki/Harmonic_series_(mathematics)

```python
div_rec_final_harm = div_rec_final3.doit().rewrite(harmonic).factor()
div_rec_final_harm
```

#### Let's finally obtain the closed form
With this last step, we finally obtain the closed form of the function $C_n(n)$ that calculates the average number of checks done by the Quicksort on arrays of length $n$ by multiplying both sides by $(n+1)$.

```python
closed_form_diff = op_eq_n(div_rec_final_harm,n+1,lambda x,y: x*y).factor()
closed_form_diff
```

### Recurrence solving: generating function method
The second method we see is the *[generating function][gen-func] method*.

The idea of this method is to calculate a closed form of the function $\mathcal{G}(C_n(n), t)$ through the application of a finite number of rules, or principles, that applies to generating function.

Once we have the closed form of $\mathcal{G}(C_n(n), t)$ we have two choices: we either stop there and use the series expansion of $\mathcal{G}(C_n(n), t)$ to check if the coefficients coincide with $C_n(n)$, for some fixed $n$, or we can go even further and apply the operator $[t^n]$, called "coefficient of", to $\mathcal{G}(C_n(n), t)$, in order to obtain the closed form for $C_n(n)$.

[gen-func]: https://en.wikipedia.org/wiki/Generating_function

#### Let's start again from our base recurrence

```python
base_rec
```

#### Multiply both sides by $n$, as before

```python
base_rec3
```

#### Let's define the $\mathcal{G}$ function
We need to define the *generating function* operator $\mathcal{G}$.

$\mathcal{G}$ is a function in two variables, a numeric succession, like our $C_n(n)$, and a placeholder symbol $t$, such that:
$$\mathcal{G}(C_n(n), t) = \sum_{k=0}^{+\infty}C_n(n)\cdot t^k.$$

```python
G = Function('\mathcal{G}');
G(Cn(n), t)
```

#### Identity principle
The *identity principle* states that if
$$a_n = b_n \;\forall n,$$
then
$$\mathcal{G}(a_n, t) = \mathcal{G}(b_n, t).$$

```python
def identity_principle(eq):
    return Eq(G(eq.lhs, t), G(eq.rhs, t))
```

#### Apply the identity principle to our recurrence
We can easily check that the successions on each side of our recurrence are equal for each $n$ (even for $n=0$ we get $0=0$, so no adjustment is needed). This means we can apply the identity principle and move on to the generating function world.

```python
g_rec = identity_principle(base_rec3)
g_rec
```

#### Let's define the linearity property of $G$
One of the basic properties of generating functions is the *linearity*, which states that
$$\mathcal{G}(\alpha a_n + \beta b_n, t) = \alpha\mathcal{G}(a_n, t) + \beta\mathcal{G}(b_n, t).$$

```python
def linearity(eq, g):
    x = Wild('x', exclude=[n])
    a = Wild('a')
    y = Wild('y', exclude=[n])
    b = Wild('b')
    match = g.match(G(x*a+y*b, t))
    return eq.subs(g, x*G(a, t)+y*G(b, t)).xreplace(match)
```

#### And apply it to the right side
Let's apply the linearity principle to the right side of the equation in order to break the inner sum into two generating functions (and also bring that $2$ outside of the $\mathcal{G}$).

```python
g_rec2 = linearity(g_rec, g_rec.rhs)
g_rec2
```

#### Apply the linearity again to $\mathcal{G}(n^2+n)$
Here we apply the linearity again on $\mathcal{G}(n^2+n)$ to break it into two smaller generating functions.

```python
g_rec3 = linearity(g_rec2, G(n**2 + n, t))
g_rec3
```

#### Derivation property
The *derivation property* states that
$$\mathcal{G}(n\cdot a_n, t) = t\cdot \mathcal{G}'(a_n, t),$$
with $\mathcal{G}'(a_n, t)$ derivate of $\mathcal{G}(a_n, t)$ for $t$.

```python
def derivation(eq, g):
    a = Wild('a')
    match = g.match(G(n*a, t))
    return eq.subs(g, t*G(a, t).diff(t)).xreplace(match)
```

#### Let's use it on the left side
We notice that the left side of the equation matches with the derivation property premises, so we apply it.

```python
g_rec4 = derivation(g_rec3, g_rec3.lhs)
g_rec4
```

#### New notation for $\mathcal{G}(C_n(n), t)$
We introduce now a new notation for $\mathcal{G}(C_n(n), t)$ to achieve a better readability.

```python
Ct = Function('C_t')
Ct(t)
```

#### Substitution function for $G(C_n(n)) = C_t(t)$
We also provide a handy function to substitute the new notation when possible.

```python
def ct_subs(eq):
    return eq.subs(G(Cn(n), t), Ct(t))
```

#### Let's substitute $G(C_n(n))$ for a better readability
We substitute here the new notation just introduced.

```python
g_rec5 = ct_subs(g_rec4)
g_rec5
```

#### Couple of function to substitute known generating functions
Without going into details and demonstration, we provide here the substitution functions for two known generating functions:
$$\mathcal{G}(n, t) = \frac{t}{(1-t)²}$$
$$\mathcal{G}(n², t) = \frac{t(1+t)}{(1-t)³}$$

```python
def known_n(eq):
    return eq.subs(G(n, t), t/((1-t)**2))

def known_n_squared(eq):
    return eq.subs(G(n**2, t), (t*(1+t))/((1-t)**3))
```

#### Let's use them both!
We substitute here the known results for these two generating functions.

```python
g_rec6 = known_n(known_n_squared(g_rec5))
g_rec6
```

#### Backward shift
The *backward shift property* states that
$$\mathcal{G}(a_{n-s}, t) = t^s\cdot\mathcal{G}(a_n, t),$$
with $s$ negative shift.

```python
def backward_shift(eq, g, s):
    a = Wild('a')
    match = g.match(G(a, t))
    return eq.subs(g, (t**s)*G(a, t)).xreplace(match).subs(n, n+s)
```

#### Apply the backward shift property to the last term of the right side
Since we want the summation on the right side to go up to $n$ (and not $n-1$), we apply to it the backward shift property with $s=1$.

```python
g_rec7 = backward_shift(g_rec6, G(Sum(Cn(k), (k, 0, n-1)), t), 1)
g_rec7
```

#### Euler method implementation (particular case of convolution)
The *Euler property* states that
$$\mathcal{G}(\sum_{k=0}^n a_n, t) = \frac{1}{1-t}\mathcal{G}(a_n).$$
The Euler property is a particular case of the more general *[convolution property][conv-prop]* with $b_n = 1 \;\forall n$.

[conv-prop]: https://en.wikipedia.org/wiki/Generating_function#Convolution

```python
def euler(eq, g):
    a = Wild('a')
    i = Wild('i')
    f = Wild('f')
    l = Wild('l')
    match = g.match(G(Sum(a, (i, f, l)), t))
    return eq.subs(g, (1/(1-t))*G(a, t)).xreplace(match).subs(match[i], n)
```

#### Let's apply Euler on the last term of the right side
We can now eliminate the summation on the right side by applying Euler.

```python
g_rec8 = ct_subs(euler(g_rec7, G(Sum(Cn(k), (k, 0, n)), t)))
g_rec8
```

#### Simplify $t$ on both sides

```python
g_rec9 = op_eq_n(g_rec8, t, lambda x,y: x/y).simplify()
g_rec9
```

#### Are we lucky?
sympy provides the function $dsolve$ to solve differential equations.

Let's see if we are lucky enough to have our differential equation solved automatically by sympy.

```python
g_solved = dsolve(g_rec9).factor()
g_solved
```

Yeah! :D

$dsolve$ actually worked! Now we only need a few rewriting steps (and figure out what value should be assigned to the integration constant $C_1$) and we're done!

#### Auxiliary function to bring a coefficient inside a $log$
Before we can move on to the final step, we need to rewrite a bit that $log$ and in particular we want to reverse the sign of the $log$ argument, $t-1$, by applying some basic $log$ properties.

Let's first define a function to bring a coefficient inside a $log$ as power of its argument.

```python
def power_in_log_subs(eq, log_term):
    a = Wild('a')
    b = Wild('b')
    match = log_term.match(a*log(b))
    return eq.subs(log_term, log(b**a)).xreplace(match)
```

#### Let's bring that $-2$ inside that $log$
We use here that function to bring the $-2$ multiplying the $log$ inside as power of its argument $t-1$.

```python
g_solved2 = power_in_log_subs(g_solved, -2*log(t-1))
g_solved2
```

#### Auxiliary function to reverse an even power
Now we write a function that reverse the sign of an even power, using the property that $a^{2x}=(-a)^{2x}$.

```python
def reverse_even_power_subs(eq, power):
    a = Wild('a')
    b = Wild('b')
    match = power.match(a**(2*b))
    return eq.subs(power, (-a)**(2*b)).xreplace(match)
```

#### Let's reverse the sign of $(t-1)^2$
We can now use this function to reverse the sign of that $t-1$ at the denominator inside the $log$.

```python
g_solved3 = reverse_even_power_subs(g_solved2, (t-1)**2)
g_solved3
```

#### Auxiliary function to bring a power outside a $log$
Here we define a specular function, that brings a power outside a $log$ as coefficient. For convenience, we pass the power we want to bring out as an argument.
```java
// ToFix: keep the original sign of a without manual change
```

```python
def power_out_log_subs(eq, log_term, p):
    a = Wild('a')
    b = Wild('b')
    match = log_term.match(log(a**p))
    match[a] = 1/(1-t) # ToFix: this!
    return eq.subs(log_term, p*log(a)).xreplace(match)
```

#### Let's bring the $2$ outside the $log$ again
We take that $2$ outside again (keeping the $-1$ inside as power).

```python
g_solved4 = power_out_log_subs(g_solved3, log(1/((1-t)**2)),2)
g_solved4
```

#### Let's find $C_1$
In order to find the correct value for the constant $C_1$ we solve the equation for $C_1$ in $0$ (since $C_n(0)$ is the only value known of our succession and $C_n(0)=C_t(0)=0$).

To do so, we express the equation in $t=0$ and the use the function $solve$ offered by sympy.

```python
C1_value = solve(g_solved4.subs(t, 0).subs(Ct(0), 0), dict=True)[0]
C1_value
```

#### And let's substitute the value found into our equation
Finally, if we substitute the value found to $C_1$ in our equation we obtain the closed form of $C_t(t)$.

```python
g_solved5 = g_solved4.xreplace(C1_value)
g_solved5
```

#### Known harmonic generating function
We can now observe that $\frac{log(\frac{1}{1-t})}{1-t}$ is the generating function of the Harmonic series. I.e.:
$$\frac{log(\frac{1}{1-t})}{1-t} = \mathcal{G}(H_n, t).$$
So let's define a function that substitute the generating function of the Harmonic series to that fraction.

```python
def known_g_harmonic(eq):
    return eq.subs((log(1/(-t+1)))/(-t+1), G(harmonic(n), t))
```

#### Substitute with the generating function of the Harmonic series
We finally substitute the known generating function of the Harmonic series for a better readability.

```python
g_solved6 = known_g_harmonic(g_solved5)
g_solved6
```

#### Let's check with some coefficient

```python
g_solved5.rhs.series(n=21).evalf()
```

Seems like it's working!

#### Coefficient extraction
Without getting our hands dirty with the "coefficient of" operator, let's define a simple function for coefficient extraction by converting the series expansion into a polynomial.

```python
def get_coeff(eq, n):
    poly = Poly(eq.rhs.series(n=n+1).removeO(), t)
    coeffs = poly.coeffs()
    return coeffs[0]
```

```python
get_coeff(g_solved5, 20).evalf()
```

Yep, it works! :)

### Test & comparison
In this last section we provide some test so we can easily compare the three methods seen (direct recurrence evaluation, closed form evaluation and series expansion) with some simulation with our Quicksort implementation.

#### Tests definition
We define here the routine to run a test. Basically we first run the Quicksort algorithm with checks counting on several permutations (every possible permutation for n <= 8, otherwise 10000 random permutations) and then we calculate the checks by evaluating the recurrence, then by evaluating the closed form function and finally by extracting the coefficient from the series expansion of the generating function.

We then run some tests for some values of $n$.

```python
def run_test(ns, times, results, i):
    sum_nc = 0
    if i <= 8: # Exhaustive Testing for the QuickSort algorithm
        all_perms = list(itertools.permutations(range(1, i+1)))
        N = len(all_perms)
        for perm in all_perms:
            sum_nc += quick(list(perm), 0, i-1)
    else: # Random Testing for the QuickSort algorithm
        N = 10000 # Number of random arrays to be tested
        for j in range(0, N):
            random = np.random.permutation(i)
            sum_nc = sum_nc + quick(random, 0, i-1)
    mean = sum_nc/N

    s1 = timeit.default_timer()
    recurrence = C_py(i)
    s2 = timeit.default_timer()
    closed = closed_form_diff.subs(n,i).rhs.evalf()
    s3 = timeit.default_timer()
    g_cn = get_coeff(g_solved5, i)
    s4 = timeit.default_timer()

    ns.append(i)
    results[0].append(mean)
    results[1].append(recurrence)
    results[2].append(closed)
    results[3].append(g_cn)
    times[0].append(s2-s1)
    times[1].append(s3-s2)
    times[2].append(s4-s3)

ns = []
times = [[], [], []]
results = [[], [], [], []]

for i in range(1, 21):
    run_test(ns, times, results, i)
```

#### Results comparison
We show here a comparison of the results yielded by the different approaches on the tests we ran.

```python
for i in range(0, len(ns)):
    print ("n = %02d | Algorithm count: %07.3f | Recurrence: %07.3f | \
Closed form: %07.3f | G(C_n): %07.3f"
         % (ns[i], results[0][i], results[1][i], results[2][i], results[3][i]))
```

    n = 01 | Algorithm count: 002.000 | Recurrence: 002.000 | Closed form: 002.000 | G(C_n): 002.000
    n = 02 | Algorithm count: 005.000 | Recurrence: 005.000 | Closed form: 005.000 | G(C_n): 005.000
    n = 03 | Algorithm count: 008.000 | Recurrence: 008.667 | Closed form: 008.667 | G(C_n): 008.667
    n = 04 | Algorithm count: 012.000 | Recurrence: 012.833 | Closed form: 012.833 | G(C_n): 012.833
    n = 05 | Algorithm count: 017.000 | Recurrence: 017.400 | Closed form: 017.400 | G(C_n): 017.400
    n = 06 | Algorithm count: 022.000 | Recurrence: 022.300 | Closed form: 022.300 | G(C_n): 022.300
    n = 07 | Algorithm count: 027.000 | Recurrence: 027.486 | Closed form: 027.486 | G(C_n): 027.486
    n = 08 | Algorithm count: 032.000 | Recurrence: 032.921 | Closed form: 032.921 | G(C_n): 032.921
    n = 09 | Algorithm count: 038.000 | Recurrence: 038.579 | Closed form: 038.579 | G(C_n): 038.579
    n = 10 | Algorithm count: 044.000 | Recurrence: 044.437 | Closed form: 044.437 | G(C_n): 044.437
    n = 11 | Algorithm count: 050.000 | Recurrence: 050.477 | Closed form: 050.477 | G(C_n): 050.477
    n = 12 | Algorithm count: 056.000 | Recurrence: 056.683 | Closed form: 056.683 | G(C_n): 056.683
    n = 13 | Algorithm count: 063.000 | Recurrence: 063.044 | Closed form: 063.044 | G(C_n): 063.044
    n = 14 | Algorithm count: 069.000 | Recurrence: 069.547 | Closed form: 069.547 | G(C_n): 069.547
    n = 15 | Algorithm count: 076.000 | Recurrence: 076.183 | Closed form: 076.183 | G(C_n): 076.183
    n = 16 | Algorithm count: 082.000 | Recurrence: 082.945 | Closed form: 082.945 | G(C_n): 082.945
    n = 17 | Algorithm count: 089.000 | Recurrence: 089.824 | Closed form: 089.824 | G(C_n): 089.824
    n = 18 | Algorithm count: 096.000 | Recurrence: 096.814 | Closed form: 096.814 | G(C_n): 096.814
    n = 19 | Algorithm count: 103.000 | Recurrence: 103.910 | Closed form: 103.910 | G(C_n): 103.910
    n = 20 | Algorithm count: 111.000 | Recurrence: 111.105 | Closed form: 111.105 | G(C_n): 111.105

Below here, we can see the same results plotted. As we can see, with enough random permutations the Quicksort count estimation is quite accurate, while the other three methods are exactly the same.

```python
plt.rcParams['figure.figsize'] = 15, 5
plt.title("Number of Checks as n increases")
plt.xlabel("n")
plt.ylabel("#Checks")
plt.plot(ns, results[1], c='black', ls='--')
plt.plot(ns, results[2], c='green', marker='s', ls='')
plt.plot(ns, results[3], c='blue', marker='s',mew=2, fillstyle='none', ls='')
plt.plot(ns, results[0], c='red', marker='o', ls='')
plt.axis([0, 30, 0, 120])
plt.legend(['Recurrence', 'Closed form', 'G(C_n)', 'Algorithm count'])
plt.show()
```

#### Evaluation time comparison
Let's compare also the evaluation time needed to evaluate the recurrence, the closed form function and to calculate the generating function series expantion and extract the coefficient needed.

```python
for i in range(0, len(ns)):
    print ("n = %02d | Recurrence: %.5f s| \
Closed form: %.5f s| G(C_n): %.5f s"
           % (ns[i], times[0][i], times[1][i], times[2][i]))
```

    n = 01 | Recurrence: 0.00001 s| Closed form: 0.00429 s| G(C_n): 0.10582 s
    n = 02 | Recurrence: 0.00000 s| Closed form: 0.00238 s| G(C_n): 0.17118 s
    n = 03 | Recurrence: 0.00001 s| Closed form: 0.00347 s| G(C_n): 0.26183 s
    n = 04 | Recurrence: 0.00001 s| Closed form: 0.00330 s| G(C_n): 0.34894 s
    n = 05 | Recurrence: 0.00001 s| Closed form: 0.00249 s| G(C_n): 0.47132 s
    n = 06 | Recurrence: 0.00003 s| Closed form: 0.00294 s| G(C_n): 0.56125 s
    n = 07 | Recurrence: 0.00004 s| Closed form: 0.00284 s| G(C_n): 0.75981 s
    n = 08 | Recurrence: 0.00007 s| Closed form: 0.00247 s| G(C_n): 0.86279 s
    n = 09 | Recurrence: 0.00019 s| Closed form: 0.00261 s| G(C_n): 1.04230 s
    n = 10 | Recurrence: 0.00024 s| Closed form: 0.00342 s| G(C_n): 1.10192 s
    n = 11 | Recurrence: 0.00051 s| Closed form: 0.00291 s| G(C_n): 1.43626 s
    n = 12 | Recurrence: 0.00115 s| Closed form: 0.00359 s| G(C_n): 1.53465 s
    n = 13 | Recurrence: 0.00191 s| Closed form: 0.00237 s| G(C_n): 1.78319 s
    n = 14 | Recurrence: 0.00375 s| Closed form: 0.00255 s| G(C_n): 1.98631 s
    n = 15 | Recurrence: 0.00848 s| Closed form: 0.00319 s| G(C_n): 2.22234 s
    n = 16 | Recurrence: 0.01470 s| Closed form: 0.00245 s| G(C_n): 2.60340 s
    n = 17 | Recurrence: 0.03232 s| Closed form: 0.00256 s| G(C_n): 2.87335 s
    n = 18 | Recurrence: 0.06420 s| Closed form: 0.00254 s| G(C_n): 3.34817 s
    n = 19 | Recurrence: 0.12601 s| Closed form: 0.00287 s| G(C_n): 3.80211 s
    n = 20 | Recurrence: 0.24046 s| Closed form: 0.00317 s| G(C_n): 4.08211 s

As before, let's plot the evaluation times to have a better understanting.

As expected, evaluating the recurrence seems to take more time than evaluating the closed form, although not too much, at least for small $n$. We can expect to see the recurrence evaluation time "explode" for bigger values of $n$.

It's particularly interesting to notice how the computation of the series expansion of $\mathcal{G}(C_n)$ explodes right away, making this method unusable for bigger values of $n$. So we can conclude from these tests that, if we really want to use the generating function method, then we have to also try to apply the "coefficient of" operator and try to obtain the closed form of the function, because calculating the series expansion gets too heavy with bigger $n$.

```python
plt.title("Time spent (in seconds) as n increases")
plt.xlabel("n")
plt.ylabel("Time (sec.)")
plt.plot(ns, times[0], c='black', marker='s', ls='--')
plt.plot(ns, times[1], c='green', marker='o', ls='--')
plt.plot(ns, times[2], c='blue', marker='D', ls='--')
plt.axis([0, 30, -1, 6])
plt.legend(['Recurrence','Closed form','G(C_n)'])
plt.show()
```


# How to derive moment equations in Compartor
$$
\def\n{\mathbf{n}}
\def\x{\mathbf{x}}
\def\N{\mathbb{\mathbb{N}}}
\def\X{\mathbb{X}}
\def\NX{\mathbb{\N_0^\X}}
\def\C{\mathcal{C}}
\def\Jc{\mathcal{J}_c}
\def\DM{\Delta M_{c,j}}
\newcommand\diff{\mathop{}\!\mathrm{d}}
\def\Xc{\mathbf{X}_c}
\def\Yc{\mathbf{Y}_c}
\newcommand{\muset}[1]{\dot{\{}#1\dot{\}}}
$$

```python
# initialize sympy printing (for latex output)
from sympy import init_printing, Symbol
init_printing()

# import functions and classes for compartment models
from compartor import *
```

## Moment declaration

The purpose of Compartor is to generate the moment equations: these are a system of ODEs that describes the time evolution of a set of expected population moments.

To denote a particular moment, use the `Moment` class. It takes as constructor arguments the integer exponents in each species. For instance, the following list

```python
moments = [
    Moment(0),
    Moment(0)**2,
    Moment(1),
    Moment(1)**2
]
display(moments)
```

denotes 4 moment expressions of a system comprising $D=1$ chemical species. Instead, this list

```python
moments = [
    Moment(0,0),
    Moment(0,0)**2,
    Moment(1,0),
    Moment(1,0)**2,
    Moment(0,1),
    Moment(0,1)**2,
    Moment(1,0)*Moment(0,1)
]
display(moments)
```

collects 7 moment expressions of a system of dimensionality $D=2$.

Compartor offers two main modalities to handle the derivation of the moment equations for a given compartment population model.

The user can choose a fully automated derivation, where possible further moment dependencies and moment closures are managed internally by the tool. The automated usage is usually recommended, especially when the user seeks for a one-shot solution to create a closed system of moment equations.

Alternatively, more experienced users might opt for a step-by-step analysis of the moment equations, which permits a more gradual control on the included moment statistics and on the application of moment closures.

In this notebook, we illustrate both automated and manual derivation for the case study shown in the paper.

First, we specify the model (see (1) HOWTO - define the model).

```python
x = Content('x')
y = Content('y')

# Intake Distribution
pi_I = OutcomeDistribution.Poisson(Symbol('\pi_{I}(y; \lambda)'),y[0],Symbol('\lambda'))

Intake      = TransitionClass( {}        -to> [(y[0],0)],   'k_I', pi=pi_I, name='I')
Fusion      = TransitionClass( [x] + [y] -to> [x+y],        'k_F',          name='F')
Conversion  = TransitionClass( [x]       -to> [x + (-1,1)], 'k_c', x[0],    name='c')
Degradation = TransitionClass( [x]       -to> [x + (0,-1)], 'k_d', x[1],    name='d')

transitions = [ Intake, Fusion, Conversion, Degradation]

display_transition_classes(transitions)
```

$\displaystyle \begin{align} \emptyset&\overset{h_{I}}{\longrightarrow}\left[\left( {y}_{0}, \  0\right)\right] && h_{I} = \pi_{I}(y; \lambda) k_{I}\\\left[x\right] + \left[y\right]&\overset{h_{F}}{\longrightarrow}\left[x + y\right] && h_{F} = \frac{k_{F} \left(n{\left(y \right)} - \delta_{x y}\right) n{\left(x \right)}}{\delta_{x y} + 1}\\\left[x\right]&\overset{h_{c}}{\longrightarrow}\left[\left( -1, \  1\right) + x\right] && h_{c} = k_{c} n{\left(x \right)} {x}_{0}\\\left[x\right]&\overset{h_{d}}{\longrightarrow}\left[\left( 0, \  -1\right) + x\right] && h_{d} = k_{d} n{\left(x \right)} {x}_{1} \end{align}$

## Automated derivation of moment equations
The function `automated_moment_equations()` handles automatically the process of moment identification, equation generation and moment closure for a provided model.

`automated_moment_equations()` takes the following arguments:
* the number `D` of chemical species of the system
* the set of transition classes defining the model
* optionally, a list of `Moments` that must be included the system of moment equations.

When the optional set of moments is not given, `automated_moment_equations()` outputs the smallest closed system of moment equations containing at least the total number number and the total amount of chemical species. Let's call `automated_moment_equations()` on the `transition` of the case study:

```python
equations = automated_moment_equations(2, transitions)
```

Computed moment equations for desired moments $\left< N\right> $, $\left< M^{\left(1, 0\right)}\right> $, and $\left< M^{\left(0, 1\right)}\right> $.

Equations were iteratively added for $\left< N^{2}\right> $.

Gamma closures were substituted for $\left< N^{3}\right> $.

The function outputs a summary of the operations executed in the derivation and returns the final system of moment equations. To display the derived equations, use the `display_moment_equations()` function.

```python
display_moment_equations(equations)
```

Optionally, a list of moments can be passed to `automated_moment_equations()` to require them to be included in the system of moment equations.

```python
desired_moments = [Moment(0,0), Moment(0,1)**2]
equations = automated_moment_equations(2, transitions, desired_moments)
```

Computed moment equations for desired moments $\left< N\right> $ and $\left< {\left(M^{\left(0, 1\right)}\right)}^{2}\right> $.

Equations were iteratively added for $\left< M^{\left(0, 1\right)}\right> $, $\left< N^{2}\right> $, $\left< M^{\left(1, 0\right)}\right> $, $\left< M^{\left(0, 1\right)} M^{\left(1, 0\right)}\right> $, and $\left< {\left(M^{\left(1, 0\right)}\right)}^{2}\right> $.

Gamma closures were substituted for $\left< N^{3}\right> $.

```python
display_moment_equations(equations)
```

In the latter case, further moments and more closures were required to converge to a closed system of equations.

Internally, the function `automated_moment_equations()` works by applying in iterative fashion the different steps that would occur in the manual derivation. For advanced users, it is then useful to get an idea of the step-by-step derivation of the moment equations as reported in the following.

## Step-by-step derivation of moment equations

In the manual derivation, the user needs to specify a set of moment of interest. Considering again the case study, we define a set of moments which we aim to characterize:

```python
moments = [
    Moment(0,0),
    Moment(0,0)**2,
    Moment(1,0),
    Moment(1,0)**2,
    Moment(0,1),
    Moment(0,1)**2,
    Moment(1,0)*Moment(0,1)
]
display(moments)
```

Then, call the function `compute_moment_equations()` to generate the moment ODES for the given model and the desired set of moments

```python
equations = compute_moment_equations(transitions, moments)
display_moment_equations(equations)
```

The `display_moment_equations()` function outputs a warning when the system is not closed, and which moments are missing. (This output can be suppressed by passing the optional second argument `display_moment_equations(equations, print_missing=False)`).

These "missing" moments are moment expressions whose expectation occurs on the right-hand side of an equation but which are not governed by the system. They can also be computed using the `get_missing_moments()` function.

```python
missing = get_missing_moments(equations)
missing
```

At this point, the user can decide to add the missing moments to the current list (e.g. `moments += list(missing)`) and reiterate the derivation by calling `compute_moment_equations()`, or to apply a moment closure to replace the missing moments with functions of those already available.

### Moment closures

The moment equations are usually not in closed form for a large class of models. In those cases, expanding the moment dynamics does not help to converge to a closed system of equations. Thus, it is necessary to resort to moment closure schemes.

A moment closure is an expression that replaces moments of higher order with functions of moments of lower order. Thus, an open system of equations can be converted into an approximated closed system of equations. In Compartor, the following closure functions can be applied to a list of `missing` moments:
* `gamma_closures()` applies a third-order multivariate Gamma closure,
* `meanfield_closures()` applies a meanfield closure,
* `hybrid_closures()` applies first the Gamma closure and then the meanfield closure to those moments not treatable by the Gamma closure.

In the case above, the Gamma closure is suited to close the third-order missing moment. Thus, we can compute the closure

```python
closures = gamma_closures(missing)
display_closures(closures)
```

and substitute it in `equations` with the function `substitute_closures()`

```python
equations = substitute_closures(equations, closures)
display_moment_equations(equations)
```

The system of equations is now closed and ready to be solved.

The usage of the meanfield and the hybrid closure is analogous, but typically is associated with a greater approximation error. This is why the function `automated_moment_equations()`, which was introduced in the previous section, prioritizes the Gamma closure and only resorts to meanfield (thus, the hybrid scheme) when the Gamma closure is not applicable.

(2) HOWTO - derive moment equations.ipynb


# Machine Learning: Basic Principles

## Model Validation and Selection

## Learning goals

Remember the main components of a machine learning problem:

1. **data points** which are characterized by features $\mathbf{x} \in \mathcal{X}$ and labels $y\in \mathcal{Y}$,

2. a **hypothesis space** $\mathcal{H}$ consisting of maps ("predictors") $h: \mathcal{X} \rightarrow \mathcal{Y}$ from features $\mathbf{x}\in\mathcal{X}$ to a predicted label $\hat{y}=h(\mathbf{x})\in \mathcal{Y}$ and

3. a **loss function** (such as squared error loss or the logistic loss), which measures the quality of a predictor.

In this exercise you will learn a principled approach for **model selection**, i.e., how to choose the best (in a certain sense) hypothesis space $\mathcal{H}$ out of a whole list of different hypothesis spaces $\mathcal{H}^{(1)},\mathcal{H}^{(2)},\ldots$. The key idea behind this approach is to use different data points for choosing a predictor $\hat{h}^{(d)}$ out of a hypothesis space $\mathcal{H}^{(d)}$ (via empiricial risk minimization) and for assessing (validating) the quality of the predictor $\hat{h}^{(d)} \in \mathcal{H}^{(d)}$. The data points used for choosing (learning) the predictor $\hat{h}^{(d)}$ out of $\mathcal{H}^{(d)}$ is referred to as the **training set** $\mathbb{X}^{(\rm train)}$, while the data points used to assess the quality of $\hat{h}^{(d)}$ is referred to as the **validation set** or **cross-validation set** $\mathbb{X}^{(\rm val)}$.

We will also learn about **regularization** as a "soft" variant of model selection. The basic idea is to regularize the empirical risk minimization problem by adding a term that quantifies the anticipated increase in the validation error (compared to the training error) due to the "complexity" of a particular predictor (e.g., a polynomial of high degree).

The concept of model selection and regularization is best understood by working through a particular example. To this end, we will (re-)consider the problem of predicting cryptocurrencies (see Round 2 - "Regression").

### Exercise Content
1. Predicting Cryptocurrencies
2. Model Selection
3. Regularization

### Keywords
 `(Cross-) Validation`, `Model Selection`,`Regularization`,`Training Set`,`Validation Set`,

##  1. Predicting Cryptocurrencies

Let us consider again (see Round 2 "Regression") the problem of predicting the closing price of the cryptocurrency Ethereum based on the closing price of Bitcoin on the same day. However, we will only consider a **subset** of the historic data about Ethereum and Bitcoin used in Round 2 - "Regression". This subset of $N=20$ data points $\mathbb{X} = \{ (x^{(i)},y^{(i)}) \}_{i=1}^{N}$ is stored in the file *BTC_ETH_round4.csv*.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display

#Read the data
df=pd.read_csv('BTC_ETH_round4.csv')
x=df.Bitcoin.values #Bitcoin values
y=df.Ethereum.values #Ethereum values

#Reshape the data.
#This is really important for the matrix multiplications later on!
x=np.reshape(x,(len(x),1))
y=np.reshape(y,(len(y),1))

#Plot the data
plt.figure(figsize=(6, 6))
plt.scatter(x, y)
plt.title(r'$\bf{Figure\ 1.}$ Bitcoin vs Ethereum')
plt.xlabel('Bitcoin')
plt.ylabel('Ethereum')
plt.show()
print(len(x))
```

    <Figure size 600x600 with 1 Axes>

    20

## 2 Model Selection

As observed in Round 2 - "Regression" (see also the course book Sec. 2.1.3), the relation between the Ethereum price $y$ and the Bitcoin price $x$ is highly non-linear. Therefore it is useful to consider a hypothesis space which is constituted by polynomial functions of degree $d$:

\begin{equation}
\mathcal{H}^{(d)}_{\rm poly} = \{ h^{(\mathbf{w})}(\cdot): \mathbb{R} \rightarrow \mathbb{R}: h^{(\mathbf{w})}(x) = \sum_{r=1}^{d+1} w_{r} x^{r-1} \mbox{, with some } \mathbf{w} =(w_{1},\ldots,w_{d+1})^{T} \in \mathbb{R}^{d+1} \}.
\label{eq6}
\tag{6}
\end{equation}

As discussed in the course book (Section 3.2 "Polynomial Regression"), polynomial regression is equivalent to combining linear regression with a feature map $\phi: \mathbb{R} \rightarrow \mathbb{R}^{d+1}$. In particular, we transform the features $x$ (Bitcoin closing price) of the data points to the feature vector

\begin{equation}
\phi(x) = (x^{0},\ldots,x^{d})^{T} \in \mathbb{R}^{d+1}.
\label{eq7}
\tag{7}
\end{equation}

Using this new feature vector $\mathbf{x}=\phi(x)$, we can represent any polynomial map $h \in \mathcal{H}^{(d)}_{\rm poly}$ as

\begin{equation}
h^{(\mathbf{w})}(x) = \mathbf{w}^{T} \phi(x) \mbox{ with some weight vector } \mathbf{w} \in \mathbb{R}^{d+1}.
\label{eq8}
\tag{8}
\end{equation}

Consider a particular choice for the maximum polynomial degree $d=2$. We can find a good polynomial predictor $\hat{h}^{(d)}$ by minimizing the empirical risk over some labeled data points. However, we will not use the entire dataset $\mathbb{X}$ from Section 1 for this empirical risk minimization but only a subset, the **training set**

\begin{equation}
\mathbb{X}^{(\rm train)} = \{ (x^{(2)},y^{(2)}), (x^{(4)},y^{(4)}), (x^{(6)},y^{(6)}),\ldots,(x^{(N)},y^{(N)}) \}.
\end{equation}

This training set $\mathbb{X}^{(\rm train)}$, of size $N_{\rm train} = N/2$, is obtained by selecting every other data point of the original data set $\mathbb{X}$. The remaining data points in $\mathbb{X}$ will be used as the **validation set**:

\begin{equation}
\mathbb{X}^{(\rm val)}= \{(x^{(1)},y^{(1)}),(x^{(3)},y^{(3)}),(x^{(19)},y^{(19)})\}.
\end{equation}

For each choide of $d$, which corresponds to a particular hypothesis space $\mathcal{H}^{(d)}$, we learn the optimal predcitor $\hat{h}^{(d)}$ by solving

\begin{align}
\hat{h}^{(d)} & = {\rm arg} \min_{h \in \mathcal{H}^{(d)}_{\rm poly}} (1/N_{\rm train}) \sum_{i=2,4,\ldots,20} \big(y^{(i)} - h(x^{(i)})\big)^2.
\label{eq9}
\tag{9}
\end{align}

This optimization problem can be rewritten using the paramtrization \eqref{eq7} as

\begin{align}
\hat{\mathbf{w}}^{(d)} & = {\rm arg} \min_{\mathbf{w} \in \mathbb{R}^{d+1}} (1/N_{\rm train}) \sum_{i=2,4,\ldots,20} \big(y^{(i)} - \mathbf{w}^{T}\phi \big(x^{(i)}\big) \big)^2.
\label{eq10}
\tag{10}
\end{align}

Given the optimal weight vector $\hat{\mathbf{w}}^{(d)}$ which solves \eqref{eq10}, we obtain the optimal predictor $\hat{h}^{(d)}$ (which solves \eqref{eq9}) by $\hat{h}^{(d)}(x)  = \big( \hat{\mathbf{w}}^{(d)} \big)^{T} \phi(x)$.

The optimization problem \eqref{eq10} can be solved either using a closed-form expression (under certain conditions) involving the feature matrix $\mathbf{X}=\big(\phi\big(x^{(2)}\big),\phi\big(x^{(4)}\big),\ldots,\phi\big(x^{(20)}\big)\big)^{T} \in \mathbb{R}^{10\times(d+1)}$ or using gradient descent (with a sufficiently small step size). The minimum objective value in \eqref{eq9} and \eqref{eq10}, obtained for degree $d$ is the **training error**

\begin{equation}
E_{\rm train}^{(d)}= (1/N_{\rm train}) \sum_{i=2,4,\ldots,20} \big(y^{(i)} - \big( \hat{\mathbf{w}}^{(d)} \big)^{T}\phi\big(x^{(i)}\big) \big)^2.
\end{equation}

After determining the predictors $\hat{h}^{(d)}$, one predictor for each choice of the maximum degree $d$, we evaluate the empirical risk incurred by $\hat{h}^{(d)}$ over the validation set (the **validation error**):

\begin{equation}
E_{\rm val}^{(d)} = (1/N_{\rm val}) \sum_{i=1,3,\ldots,19} \big(y^{(i)} - \big( \hat{\mathbf{w}}^{(d)} \big)^{T}\phi\big(x^{(i)}\big) \big)^2.
\end{equation}

We then choose the degree $d$ for which the validation error $E_{\rm val}^{(d)}$ is smallest.

It is important to note that we do not use the training error $E_{\rm train}^{(d)}$ for choosing the degree $d$. In general it is not a good idea to choose the hypothesis space based on how small the training error is. This is because for very large hypothesis spaces, like polynomials with a large degree, we can always find a predictor in it which **by accident** is able to fit the training data $\mathbb{X}^{(\rm train)}$ well. However, such a predictor will perform poorly when applied to other data points, such as the data points in the validation set $\mathbb{X}^{(\rm val)}$.

- The Python function `trainValErrors(x_train,y_train,x_val,y_val,degrees)` below takes as input the training data, validation data and a list of degrees. It returns 3 vectors which contain the resulting optimal weight *w_opts*, the training error, *training_errors* and the validation error, *validation_errors* for each of the degree values.
- We can reuse many functions used for Polynomial Regression in Round 2.
- Note that the optimal weights are obtained only using the training data
- After optimal weight vector has been found, we compute the empirical risk separately for the training data and validation data yielding the training error and the validation error

```python
def fit(x, y):
    #Calculate the optimal weight vector
    w_opt = np.dot(np.dot(np.linalg.inv(np.dot(np.transpose(x), x)),np.transpose(x)), y)
    return w_opt

def feature_mapping(x, degree):
    #compute the feature map for the given degree
    polynomial_features = np.column_stack([x**d for d in range(0,degree)])
    return polynomial_features

def polynomialRegression(x, y, degree):
    X = feature_mapping(x,degree)
    w_opt=fit(X,y)
    return w_opt

def predict(X, w_opt):
    #predict the labels
    y_pred = np.dot(X,w_opt)
    return y_pred

def empirical_risk(X, y, w_opt):
    empirical_error = np.mean(np.power(np.subtract(np.dot(X,w_opt), y),2))
    return empirical_error

#Split the data into a training and validation set.
#Don't change this.
x_train=x[1::2]
x_val=x[0::2]
y_train=y[1::2]
y_val=y[0::2]

#the degrees we want to loop over
degrees=[1,3,4]

def trainValErrors(x_train,y_train,x_val,y_val,degrees):
    #compute the optimal weight, training and validation error for each degree in degrees
    w_opts = []
    training_errors=[]
    validation_errors=[]
    for d in degrees:
        w_opt=polynomialRegression(x_train, y_train, d)
        w_opts.append(w_opt)
        training_errors.append(empirical_risk(feature_mapping(x_train,d),y_train,w_opt))
        validation_errors.append(empirical_risk(feature_mapping(x_val,d),y_val,w_opt))
    return w_opts, training_errors,validation_errors

#compute the training and validation errors and display them
w_opts, training_errors, validation_errors = trainValErrors(x_train,y_train,x_val,y_val,degrees)
df_degrees=pd.DataFrame(data={'d':degrees,'E_train':training_errors,'E_val':validation_errors})
display(df_degrees)

fig = plt.figure(figsize=(16, 8))
ax = fig.add_subplot(121)

plt.scatter(x_train, y_train, color='black', label='Training data')
plt.scatter(x_val,y_val,color='red',label='Validation data')
for i in range(len(degrees)):
    plt.plot(np.linspace(0, 1, 50), predict(feature_mapping(np.linspace(0, 1, 50),degrees[i]),w_opts[i]), label='degree = %d' %degrees[i])

plt.title(r'$\bf{Figure\ 2.}$ Polynomial Regression for different degrees')
plt.xlabel('Bitcoin')
plt.ylabel('Ethereum')
plt.ylim(0,1)
plt.xlim(0,1)
plt.legend()

#plot the training and validation errors for the different degrees
ax = fig.add_subplot(122)
plt.plot(degrees,training_errors,color='black',label='Training error')
plt.plot(degrees,validation_errors,color='red',label='Validation error')

plt.title(r'$\bf{Figure\ 3.}$ Training and validation error for different degrees')
plt.ylabel('Empirical error')
plt.xlabel('Degree')
plt.xticks(degrees)
plt.legend()

plt.show()
```

##  3 Regularization

In Section 2, we have tried to find the best hypothesis space within a sequence of increasing spaces $\mathcal{H}^{(1)} \subseteq \mathcal{H}^{(2)} \subseteq \ldots \subseteq \mathcal{H}^{(d_{\rm max})}$. The space hypothesis space $\mathcal{H}^{(d)}$ is constituted by polynomials with maximum degree $d$ (see (6)). Indeed, the space of polynomials with maximum degree $d=1$ is a subset of the space of all polynomials with maximum degree $d=2$.

The simple (but powerful) idea behind model selection is to use the validation errors obtained for the different hypothesis spaces and picking the one resulting in the smallest validation error.
An alternative approach to model selection is to use a fixed but large hypothesis space, e..g, $\mathcal{H}^{(d_{\rm max})}$ with large degree $d_{\rm max}$, and to **regularize** the corresponding empirical risk minimization problem (10). The idea of regularization is to add a second term (the regularization term) to the objective function in (10). This second terms takes into account that the validation error might be larger than the training error for a highly complex predictor.

Note that in polynomial regression, we can parametrize the predictors as $h^{(\mathbf{w})}(x)= \mathbf{w}^{T} \phi(x) = \sum_{l=0}^{d_{\rm max}} w_{l+1} x^{l}$ with some weight vector $\mathbf{w} \in \mathbb{R}^{d+1}$. The entries $w_{l}$ of the weight vector $\mathbf{w}=(w_{1},\ldots,w_{d+1})^{T}$ are the cefficients of the powers $x^{0},x^{1},\ldots,x^{d}$ of the feature $x$ (e.g., the Bitcoin closing price). It is reasonable to measure the complexity of $h^{(\mathbf{w})}$ by the average size of the coeffcients $w_{l}$. Indeed, for many large coefficients $w_{l}$, the polynomial map $h^{(\mathbf{w})}(x)$ tends to be more ''wobbly''. In particular, we will use the squared norm $\| \mathbf{w} \|^{2}_{2} = w_{1}^{2}+\ldots+w_{d+1}^{2}$ of the weight vector for the regularization term to be added in (10). This results in the following **regularized empirical risk minimization problem**

\begin{equation}
\hat{\mathbf{w}}^{(\lambda)} = {\rm arg} \min\limits_{\mathbf{w} \in \mathbb{R}^{d_{\rm max}+1}} (1/N_{\rm train}) \sum_{i=2,4,\ldots,20} (y^{(i)} - \mathbf{w}^{T} \phi(x^{(i)}))^{2}+\lambda \| \mathbf{w} \|^{2}_{2}.
\label{eq12}
\tag{12}
\end{equation}

The resulting weight vector $\hat{\mathbf{w}}^{(\lambda)}$ is then used to construct a predictor $\hat{h}^{(\lambda)}$ as

\begin{equation}
\hat{h}^{(\lambda)}(x) = \big( \hat{\mathbf{w}}^{(\lambda)}\big)^{T} \phi(x).
\label{eq13}
\tag{13}
\end{equation}

The constant $\lambda \geq 0$ is a tuning parameter and controls the "effective degree" of the resulting predictor $\hat{h}^{(\lambda)}(x)$. Using a large value of $\lambda$ in (12), results in a predictor $\hat{h}^{(\lambda)}(x)$ which resembles a polynomial of small degree (say, $d=2$). In contrast, using a very small value of $\lambda$ in (12), will typically produce a predictor $\hat{h}^{(\lambda)}$ which resembles a high-degree polynomial (with $d \approx d_{\rm max}$).

In a certain sense, we replace (or approximate) the computation of the validation error of $h^{(\mathbf{w})}$  by adding the regularization term $\lambda \| \mathbf{w} \|^{2}_{2}$ to the training error in (12). Indeed, the component $\lambda \| \mathbf{w} \|^{2}_{2}$ accounts for the anticipated increase in validation error due to the complexity ("wobbliness") of the predictor $h^{(\mathbf{w})}(x)= \mathbf{w}^{T} \phi(x)$.

Similar to linear regression, we can derive a closed-form solution for the optimal weights in \eqref{eq12} as

\begin{align}
\hat{\mathbf{w}}^{(\lambda)}& = \text{argmin}_{\mathbf{w} \in  \mathbb{R}^{d_{\rm max}+1}} \big[(1/N_{\rm train}) \|\mathbf{y} - \mathbf{X} \mathbf{w} \|^{2}_{2} + \lambda \| \mathbf{w} \|^{2}_{2}\big] \nonumber \\[3mm]
& = (1/N_{\rm train})( (1/N_{\rm train}) \mathbf{X}^{T} \mathbf{X} + \lambda \mathbf{I})^{-1} \mathbf{X}^{T} \mathbf{y}.
\label{eq14}
\tag{14}
\end{align}

Here, we used the feature matrix $\mathbf{X} = \big(\phi(x^{(2)}),\phi(x^{(4)}),\ldots,\phi(x^{(20)}) \big)^{T} \in \mathbb{R}^{10 \times (d_{\rm max}+1)}$ and the label vector $\mathbf{y}=\big(y^{(2)},y^{(4)},\ldots,y^{(20)}\big)^{T} \in \mathbb{R}^{10}$.

It remains to specify the choice of the regularization parameter $\lambda$ in \eqref{eq12}. One option is to try out different values for $\lambda$, compute the weight vector $\hat{\mathbf{w}}^{(\lambda)}$ by solving the problem \eqref{eq12} and evaluate the validation error of the corresponding predictor $h^{(\lambda)}$ (see \eqref{eq13}):

\begin{equation}
E_{\rm val}^{(\lambda)} = (1/N_{\rm val}) \sum_{i=1,3,\ldots,19} (y^{(i)} - \mathbf{w}^{T} \phi(x^{(i)}))^{2}.
\end{equation}

We then pick the value for $\lambda$ which results in the smallest validation error $E_{\rm val}^{(\lambda)}$.

1. The Python function `regularizedFit()` below takes as input $\mathbf{X}$, $\mathbf{y}$ and $\lambda$ and returns the optimal weight vector according to (Eq. \ref{eq14}).
    * You can look at the numpy functions being used in `fit()` implemented in Section 2.
    * You can use `np.eye()` to generate an identity matrix
2. The Python function `regularizedPolynomialRegression()` takes as input $\mathbf{x}$,$\mathbf{y}$,$\lambda$ and the degree. It returns the optimal weight vector.
    * Use `feature_mapping()` from Section 2 to get the feature matrix.
    * Use `regularizedFit()` to compute the optimal weight vector.
3. The Python function `trainValErrorsRegularization()` takes as inputs the training data, validation data, a list of values for lambdas and the maximum polynomial degree. It returns three vectors which contain, for every value of lambda, the optimal weight *w_opts*, the training error *training_errors* and the validation error *validation_errors*.
4. We can largely reuse `trainValErrors()` from Section 2. The difference is that we now loop over the different lambda values, instead of the different degrees.

```python
def regularizedFit(X, y,l=0):
    N=X.shape[0]
    w_opt = 1/N*np.dot(np.dot(np.linalg.inv(1/N*np.dot(np.transpose(X), X)+l*np.eye(X.shape[1])),np.transpose(X)), y)
    return w_opt

def regularizedPolynomialRegression(x, y, l=0,degree=2):
    X = feature_mapping(x,degree)
    w_opt=regularizedFit(X,y,l)
    return w_opt

#specify the degree
degree=6
#specify list of values for lambda to be considered
lambdas = [0,0.5,2]

def trainValErrorsRegularization(x_train,y_train,x_val,y_val,lambdas=[0],degree=2):
    #compute the optimal weight, training and validation error for each lambda
    w_opts = []
    training_errors=[]
    validation_errors=[]
    for l in lambdas:
        w_opt=regularizedPolynomialRegression(x_train, y_train, l,degree)
        w_opts.append(w_opt)
        training_errors.append(empirical_risk(feature_mapping(x_train,degree),y_train,w_opt))
        validation_errors.append(empirical_risk(feature_mapping(x_val,degree),y_val,w_opt))
    return w_opts, training_errors,validation_errors

#compute the training and validation errors and display them
w_opts_reg, training_errors_reg, validation_errors_reg = trainValErrorsRegularization(x_train,y_train,x_val,y_val,lambdas,degree=degree)
df_lambdas=pd.DataFrame(data={'Lambdas':lambdas,'Training errors':training_errors_reg,'Validation errors':validation_errors_reg})
display(df_lambdas)

fig = plt.figure(figsize=(16, 8))
ax = fig.add_subplot(121)
plt.scatter(x_train, y_train, color='black', label='Training data')
plt.scatter(x_val,y_val,color='red',label='Validation data')

for i in range(len(lambdas)):
    plt.plot(np.linspace(0, 1, 50), predict(feature_mapping(np.linspace(0, 1, 50),degree),w_opts_reg[i]), label='$ \lambda=%s$' %str(lambdas[i]))

plt.title(r'$\bf{Figure\ 4.}$ Regularized Polynomial Regression')
plt.xlabel('Bitcoin')
plt.ylabel('Ethereum')
plt.ylim(0,1)
plt.xlim(0,1)
plt.legend()

#plot the training and validation errors for the different values of lambda
ax=fig.add_subplot(122)
plt.plot(lambdas,training_errors_reg,color='black',label='Training error')
plt.plot(lambdas,validation_errors_reg,color='red',label='Validation error')

plt.title(r'$\bf{Figure\ 5.}$ Training and validation error for different lambdas')
plt.xlabel('Lambda')
plt.ylabel('Empirical error')
plt.xticks(lambdas)
plt.legend()
plt.show()

```

```python

```

Round 4 - Model Validation and Selection.ipynb


```python
import cirq
from cirq.contrib.svg import SVGCircuit
import sympy
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import networkx as nx
import itertools
```

```python
# Let us initialize a random graph G, with random connections
n_random = 0

# The total number of Nodes
n=4
G=nx.Graph()
G.add_nodes_from(range(0,n))

elist =[]
for k in range(0,n):
    n1,n2,w12 = (k,(k+1)%n,1.0)

    if(n1!=n2):
        elist.append((n1,n2,1.0))
G.add_weighted_edges_from(elist)

# Add custom edges
for k in range(0,n_random):
    n1,n2 = np.random.randint(0,n),np.random.randint(0,n)
    if(n1!=n2):
        G.add_edge(n1,n2,weight=1.0)

colors = ['b' for node in G.nodes()]
pos = nx.spring_layout(G)
default_axes = plt.axes(frameon=True)
nx.draw_networkx(G, node_color=colors, node_size=600, alpha=.8, ax=default_axes, pos=pos)
```

```python
p=1

# Generate a list of symbolic arguments
beta_list = ["b_"+str(i) for i in range(p)]
gamma_list = ["gamma_"+str(i) for i in range(p)]

# Convert them to symbolic numpy objects
betas = sympy.symbols(beta_list)
gammas = sympy.symbols(gamma_list)

# Define qubits on the grid.
qubits = [cirq.LineQubit(i) for i in range(n)]
#qubits = [cirq.GridQubit(i, 0) for i in range(n)]

# create the first part of the circuit
hadamard_operators = [cirq.H(qubits[i]) for i in range(len(qubits))]

# The mixers
mixer_operators = [cirq.rx(2*betas[0]).on(qubits[i]) for i in range(len(qubits))]

quadratic_operators = []
pk = 0
# The quadratic terms
for edge in G.edges():
        k,l = edge[0],edge[1]
        quadratic_operators.append(cirq.ZZPowGate(exponent= -gammas[0],global_shift=-0.5).on(qubits[k],qubits[l]))

#measurements = cirq.Moment([cirq.measure(qubits[i],key=str(i)) for i in range(len(qubits))])
measurements = cirq.measure(*qubits, key='m')

# Create the circuit from this
circuit = cirq.Circuit((hadamard_operators,quadratic_operators,mixer_operators,measurements))

SVGCircuit(circuit)
```

```python
def linear_transform(x):
    '''
    This function is used to simplify some calculations.

    x: Bitstring consisting of  {0 or 1} values

    mapping: 2*x-1

    returns:
    x_prime: numpy array with entries that are -1 or 1
    '''

    N=len(x)
    x_prime = np.zeros(N)

    for k in range(N):
        x_prime[k] = 2*int(x[k])-1

    return x_prime

def compute_cost(x,G):
    '''
    The cost of a bitstring x using the max-cut
    cost function

    C = sum_{i,j} 1/2*(1-xi*xj)

    where xi,xj are either -1 or 1

    G: Graph object
    x: bitstring consisting of characters that are either 0 or 1.
    '''
    x_prime = linear_transform(x)

    c=0.0

    for edge in G.edges():
        k,l = edge[0],edge[1]
        c+= 0.5*(1.0-x_prime[k]*x_prime[l])

    return c

def expectation_value(results,G):
    '''

    '''
    E = 0.0
    count = 0

    bitstrings = result.measurements['m']

    for x in bitstrings:
        Ek = compute_cost(x,G)
        E+=Ek
        count+=1

    E = E/count

    return E

def count_instances(N,bitstrings):
    '''

    '''

    # The dictionary to store the results
    results = {}

    # The values that can be taken by Zi
    index = [0,1]

    keys = itertools.product(index,repeat=N)

    labels = []
    label_count = []

    for state in list(keys):

        # Convert the state to integer array
        Z_state =  np.array([state[i] for i in range(len(state))]).astype(int)

        # label the state
        state_label = ''.join(Z_state.astype('str'))

        state_count = 0

        for i in range(len(bitstrings)):

            # retrieve the ith-bitstring
            xi = bitstrings[i]

            if(np.array_equal(Z_state,xi)==True):
                state_count+=1

        labels.append(state_label)
        label_count.append(state_count)

    results['labels'] = labels
    results['counts'] = label_count
    results['probability'] = np.array(label_count)/np.sum(np.array(label_count))
    return results
```

```python
n_repetitions = 3000

resolved_params = {}

max_E = -1e10
best_results = None

M = np.zeros(())

for b in np.linspace(-2*np.pi,2*np.pi,40):
    for g in np.linspace(-3.0,3.0,40):

        for beta in betas:
            resolved_params[beta] =  b

        for gamma in gammas:
            resolved_params[gamma] =  g

        simulator = cirq.Simulator()

        # Resolve all of the parameters
        resolver = cirq.ParamResolver(resolved_params)

        result = simulator.run(circuit, resolver,repetitions=n_repetitions)

        E_bg =expectation_value(result,G)

        if(E_bg > max_E):
            max_E = E_bg
            b0 = b
            g0 = g
            best_results = result

print('Best results: beta={}, gamma={}, E={}'.format(b0,g0,max_E))
```

    Best results: beta=1.1277512089809507, gamma=1.7692307692307692, E=3.0233333333333334

```python
E = []

bitstrings = best_results.measurements['m']

for x in bitstrings:
    E.append(compute_cost(x,G))

plt.clf()
#plt.figure(figsize=(10,10))
plt.hist(E,density=True)
plt.show()
```

```python
circuit_result = count_instances(N=n,bitstrings= best_results.measurements['m'])

plt.figure(figsize=(10,10))
plt.bar(np.arange(len(circuit_result['labels'])), circuit_result['probability'], tick_label=circuit_result['labels'])
plt.xticks(rotation=90)
plt.show()
```

```python

```


# 3D Plotting

```python
from pathlib import Path
```

```python
from IPython.display import Latex
from matplotlib.pyplot import *
import matplotlib.cm
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from sympy import latex, simplify, solve, symbols
```

```python
exec((Path('../../../py/startup.py').read_text()))
```

    hello world

```python
X = np.linspace(-12, 12, 128)
Y = X
X,Y = np.meshgrid(X,Y)
```

```python
fig, ax = subplots(subplot_kw={"projection": "3d"})
Z = np.sin(np.sqrt(X**2 + Y**2))
surface = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm, linewidth=1, antialiased=True)
fig.colorbar(surface, shrink=.5, aspect=5)
xlabel("$x$")
ylabel("$y$")
grid(True)

show()
```

```python
import mpl_toolkits.mplot3d as mp3d
public(mp3d)
```

    Axes3D  art3d  axes3d  axis3d  proj3d

```python
public(mp3d.art3d)
```

    Line3D             Poly3DCollection          mpath
    Line3DCollection   PolyCollection            mtext
    LineCollection     Text3D                    np
    Normalize          artist                    patch_2d_to_3d
    Patch              get_dir_vector            patch_collection_2d_to_3d
    Patch3D            juggle_axes               pathpatch_2d_to_3d
    Patch3DCollection  line_2d_to_3d             poly_collection_2d_to_3d
    PatchCollection    line_collection_2d_to_3d  proj3d
    Path3DCollection   lines                     rotate_axes
    PathCollection     math                      text_2d_to_3d
    PathPatch3D        mcolors

```python

```


### DEMDP11

# Monetary Policy Model

A central bank must set nominal interest rate so as to minimize deviations of inflation rate and GDP gap from established targets.

A monetary authority wishes to control the nominal interest rate $x$ in order to minimize the variation of the inflation rate $s_1$ and the gross domestic product (GDP) gap $s_2$ around specified targets $s^∗_1$ and $s^∗_2$, respectively. Specifically, the authority wishes to minimize expected discounted stream of weighted squared deviations

\begin{equation}
L(s) = \frac{1}{2}(s − s^∗)'\Omega(s − s^∗)
\end{equation}

where $s$ is a $2\times 1$ vector containing the inflation rate and the GDP gap, $s^∗$ is a $2\times 1$ vector of  targets, and $\Omega$ is a $2 \times 2$ constant positive definite matrix of preference weights. The inflation rate and the GDP gap are a joint controlled exogenous linear Markov process

\begin{equation}
s_{t+1} = \alpha + \beta s_t + \gamma x_t + \epsilon_{t+1}
\end{equation}

where $\alpha$ and $\gamma$ are $2 \times 1$ constant vectors, $\beta$ is a $2 \times 2$ constant matrix, and $\epsilon$ is a $2 \times 1$ random vector with mean zero. For institutional reasons, the nominal interest rate $x$ cannot be negative. **What monetary policy minimizes the sum of current and expected future losses?**

This is an infinite horizon, stochastic model with time $t$ measured in years. The state vector $s \in \mathbb{R}^2$
contains the inflation rate and the GDP gap. The action variable $x \in [0,\infty)$ is the nominal interest rate. The state transition function is $g(s, x, \epsilon) = \alpha + \beta s + \gamma x + \epsilon$

In order to formulate this problem as a maximization problem, one posits a reward function that equals the negative of the loss function $f(s,x) = −L(s)$

The sum of current and expected future rewards satisfies the Bellman equation

\begin{equation}
V(s) = \max_{0\leq x}\left\{-L(s) + \delta + E_\epsilon V\left(g(s,x,\epsilon)\right)\right\}
\end{equation}

Given the structure of the model, one cannot preclude the possibility that the nonnegativity constraint on the optimal nominal interest rate will be binding in certain states. Accordingly, the shadow-price function $\lambda(s)$ is characterized by the Euler conditions

\begin{align}
    \delta\gamma'E_\epsilon \lambda\left(g(s,x,\epsilon)\right) &= \mu \\
    \lambda(s) &= -\Omega(s-s^*) + \delta\beta'E_\epsilon \lambda\left(g(s,x,\epsilon)\right)
\end{align}

where the nominal interest rate $x$ and the long-run marginal reward $\mu$ from increasing the nominal interest rate must satisfy the complementarity condition
\begin{equation}
x \geq 0, \qquad \mu \leq 0, \qquad x > 0 \Rightarrow \mu = 0
\end{equation}

It follows that along the optimal path

\begin{align}
    \delta\gamma'E_\epsilon \lambda_{t+1} &= \mu_t \\
    \lambda_t &= -\Omega(s_t-s^*) + \delta\beta'E_\epsilon \lambda_{t+1}\\
    x \geq 0, \qquad \mu \leq 0, &\qquad x > 0 \Rightarrow \mu = 0
\end{align}

Thus, in any period, the nominal interest rate is reduced until either the long-run marginal

reward or the nominal interest rate is driven to zero.

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from compecon import BasisChebyshev, DPmodel, BasisSpline, qnwnorm, demo
import pandas as pd
pd.set_option('display.float_format',lambda x: f'{x:.3f}')
```

### Model Parameters

```python
α   = np.array([[0.9, -0.1]]).T             # transition function constant coefficients
β    = np.array([[-0.5, 0.2], [0.3, -0.4]])  # transition function state coefficients
γ   = np.array([[-0.1, 0.0]]).T             # transition function action coefficients
Ω   = np.identity(2)                        # central banker's preference weights
ξ   = np.array([[1, 0]]).T                  # equilibrium targets
μ   = np.zeros(2)                           # shock mean
σ   = 0.08 * np.identity(2),                # shock covariance matrix
δ   = 0.9                                   # discount factor
```

### State Space

There are two state variables: 'GDP gap' = $s_0\in[-2,2]$ and 'inflation'=$s_1\in[-3,3]$.

```python
n = 21
smin = [-2, -3]
smax = [ 2,  3]

basis = BasisChebyshev(n, smin, smax, method='complete',
                       labels=['GDP gap', 'inflation'])
```

### Action space

There is only one action variable x: the nominal interest rate, which must be nonnegative.

```python
def bounds(s, i, j):
    lb  = np.zeros_like(s[0])
    ub  = np.full(lb.shape, np.inf)
    return lb, ub
```

### Reward Function

```python
def reward(s, x, i, j):
    s = s - ξ
    f = np.zeros_like(s[0])
    for ii in range(2):
        for jj in range(2):
            f -= 0.5 * Ω[ii, jj] * s[ii] * s[jj]
    fx = np.zeros_like(x)
    fxx = np.zeros_like(x)
    return f, fx, fxx
```

### State Transition Function

```python
def transition(s, x, i, j, in_, e):
    g = α + β @ s + γ @ x + e
    gx = np.tile(γ, (1, x.size))
    gxx = np.zeros_like(s)
    return g, gx, gxx
```

The continuous shock must be discretized. Here we use Gauss-Legendre quadrature to obtain nodes and weights defining a discrete distribution that matches the first 6 moments of the Normal distribution (this is achieved with m=3 nodes and weights) for each of the state variables.

```python
m   = [3, 3]
[e,w] = qnwnorm(m,μ,σ)
```

### Model structure

```python
bank = DPmodel(basis, reward, transition, bounds,
               x=['interest'], discount=δ, e=e, w=w)
```

Compute Unconstrained Deterministic Steady-State

```python
bank_lq = bank.lqapprox(ξ,0)

sstar = bank_lq.steady['s']
xstar = bank_lq.steady['x']
```

If Nonnegativity Constraint Violated, Re-Compute Deterministic Steady-State

```python
if xstar < 0:
    I = np.identity(2)
    xstar = 0.0
    sstar = np.linalg.solve(np.identity(2) - β, α)

frmt = '\t%-21s = %5.2f'
print('Deterministic Steady-State')
print(frmt % ('GDP Gap', sstar[0]))
print(frmt % ('Inflation Rate', sstar[1]))
print(frmt % ('Nominal Interest Rate', xstar))
```

    Deterministic Steady-State
    	GDP Gap               =  0.61
    	Inflation Rate        =  0.06
    	Nominal Interest Rate =  0.00

### Solve the model
We solve the model by calling the `solve` method in `bank`. On return, `sol` is a pandas dataframe with columns *GDP gap*, *inflation*, *value*, *interest*, and *resid*. We set a refined grid `nr=5` for this output.

```python
S = bank.solve(nr=5)
```

    Solving infinite-horizon model collocation equation by Newton's method
    iter change       time
    ------------------------------
       0       6.7e+00    0.3994
       1       9.9e-01    1.1321
       2       9.1e-04    1.7787
       3       8.3e-07    2.2782
       4       7.7e-09    2.7687
    Elapsed Time =    2.77 Seconds

To make the 3D plots, we need to reshape the columns of `sol`.

```python
S3d = {x: S[x].reshape((5*n,5*n)) for x in S.columns}
```

This function will make all plots

```python
def makeplot(series,zlabel,zticks,title):
    fig = plt.figure(figsize=[8,5])
    ax = fig.gca(projection='3d')
    ax.plot_surface(S3d['GDP gap'], S3d['inflation'], S3d[series], cmap=cm.coolwarm)
    ax.set_xlabel('GDP gap')
    ax.set_ylabel('Inflation')
    ax.set_zlabel(zlabel)
    ax.set_xticks(np.arange(-2,3))
    ax.set_yticks(np.arange(-3,4))
    ax.set_zticks(zticks)
    ax.set_title(title)
```

###  Optimal policy

```python
fig1 = makeplot('interest', 'Nomianal Interest Rate',
               np.arange(0,21,5),'Optimal Monetary Policy')
```

### Value function

```python
fig2 = makeplot('value','Value',
                np.arange(-12,S['value'].max(),4),'Value Function')
```

### Residuals

```python
fig3 = makeplot('resid','Residual',
                [-1.5e-3, 0, 1.5e3],'Bellman Equation Residual')
plt.ticklabel_format(style='sci', axis='z', scilimits=(-1,1))
```

## Simulating the model

We simulate 21 periods of the model starting from $s=s_{\min}$, 10000 repetitions.

```python
T = 21
nrep = 10_000
data = bank.simulate(T, np.tile(np.atleast_2d(smax).T,nrep))
```

```python
subdata = data[data['time']==T][['GDP gap', 'inflation', 'interest']]
stats =pd.DataFrame({'Deterministic Steady-State': [*sstar.flatten(), xstar],
              'Ergodic Means': subdata.mean(),
              'Ergodic Standard Deviations': subdata.std()})
stats.T
```

      GDP gap
      inflation
      interest

      Deterministic Steady-State
      0.608
      0.059
      0.000

      Ergodic Means
      0.591
      0.058
      0.301

      Ergodic Standard Deviations
      0.315
      0.334
      0.694

### Simulated State and Policy Paths

```python
subdata = data[data['_rep'].isin(range(3))]
opts = dict(spec='r*', offset=(0, -15), fs=11, ha='right')
gdpstar, infstar, intstar = stats['Ergodic Means']

def simplot(series,ylabel,yticks,steady):
    fig = demo.figure('Simulated and Expected ' + ylabel,'Period', ylabel,[0, T + 0.5])
    plt.plot(data[['time',series]].groupby('time').mean())
    plt.plot(subdata.pivot('time','_rep',series),lw=1)
    plt.xticks(np.arange(0,24,4))
    plt.yticks(yticks)
    demo.annotate(T, steady, f'Expected {series}\n = {steady:.2f}', **opts)
    return fig
```

```python
fig4 = simplot('GDP gap','GDP gap',np.arange(smin[0],smax[0]+1),gdpstar)
```

```python
fig5 = simplot('inflation', 'Inflation Rate',np.arange(smin[1],smax[1]+1),infstar)
```

```python
fig6 = simplot('interest','Nominal Interest Rate',np.arange(-2,5),intstar)
```

```python
#demo.savefig([fig1,fig2,fig3,fig4,fig5,fig6])
```

notebooks/dp/11 Monetary Policy Model.ipynb


# 07 Solution: Sine Series

* *Computational Physics*:  2.5

## Problem: Summing Series: sin(x)

Evaluate the $\sin$ function from its series representation
$$
\sin x = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \dots
$$

A naive algorithm is to sum the series up to the $N$th term:
$$
\sin x \approx \sum_{n=1}^N \frac{(-1)^{n-1} x^{2n-1}}{(2n - 1)!}
$$

Problems:

- How to decide when to stop summing?
- Division of large terms (overflows!)
- Powers and factorials are very expensive to compute.

Better approach: Build up series terms $a_n$ using previous term $a_{n-1}$ through a recursion:

\begin{align}
a_n &= a_{n-1} \times q_n\\
a_n &= \frac{(-1)^{n-1} x^{2n-1}}{(2n - 1)!} = \frac{(-1)^{n-2} x^{2n-3}}{(2n - 3)!} \frac{-x^2}{(2n - 1)(2n - 2)}\\
a_n & = a_{n-1} \frac{-x^2}{(2n - 1)(2n - 2)}
\end{align}

Accuracy of this approach? Not clear in absolute terms but we can make the assumption that the error is approximately the last term in the sum, $a_N$. Hence we can strive to make the relative error smaller than the machine precision
$$
\left| \frac{a_N}{\sum_{n=1}^N a_n} \right| < \epsilon_m
$$

```python
import numpy as np
```

```python
def sin_series(x, eps=1e-16):
    """Calculate sin(x) to precision eps with recursion.

    Arguments
    ---------
    x : float
        argument of sin(x)
    eps : float, optional
        desired precision

    Returns
    -------
    func_value, N

    where func_value = sin(x) and N is the number
    of terms included in the approximation.
    """
    if x == 0:
        return 0., 1
    a_n = sumN = x   # a_1 is the sum
    n = 2            # start with calculating a_2
    while abs(a_n/sumN) > eps:
        a_n *= -x*x/((2*n - 1)*(2*n - 2))
        sumN += a_n
        n += 1
    return sumN, n-1
```

Bonus problem: the really bad implementation:

```python
import math
def sin_series_bad(x, eps=1e-16):
    """Calculate sin(x) to precision eps... BADLY"""
    if x == 0:
        return 0., 1
    a_n = x
    n = 1
    sumN = 0.
    while True:
        a_n = (-1)**(n-1) * x**(2*n-1) / math.factorial(2*n-1)
        sumN += a_n
        if abs(a_n/sumN) <=  eps:
            break
        n += 1
    return sumN, n
```

```python
sin_series_bad(10*np.pi)
```

    (0.0006695609195825486, 61)

```python
sin_series(10*np.pi, eps=1e-16)
```

    (-0.0003048974510408005, 61)

Test the implementation against the "exact" numpy function `np.sin()`.

Report
1. `x`
2. maximum `n`
3. `sin_series(x)`
4. relative error `abs(sin_series(x) - sin(x))/abs(sin(x))`

Plot against `x` the quantities above and also `sin(x)`.
* `x` - `sin_series(x)`
* `x` - `sin(x)`
* `x` - max `n`
* `x` - relative error (semilogy plot!)

For a range of numbers look at `Xsmall` and `Xlarge`:

```python
Xsmall = np.pi*np.arange(-2, 2.05, 0.05)
Xlarge = np.pi*np.arange(-50, 50.05, 0.1)
```

Implementation (modularized compared to the problem so that I can use it for `sin_series()` and `sin_series_bad()`):

```python
import numpy as np
def test_sin(x):
    y, nmax = sin_series(x)
    return compare_to_sin(x, y, nmax)

def test_sin_bad(x):
    y, nmax = sin_series_bad(x)
    return compare_to_sin(x, y, nmax)

def compare_to_sin(x, y, nmax):
    y0 = np.sin(x)
    delta = y - y0
    if y0 != 0:
        relative_error = delta/y0
    else:
        relative_error = 0
    # print(x, y, y0, delta, relative_error)
    #return x, y, y0, delta, relative_error
    return x, nmax, y, relative_error, delta
```

The functions do not work as numpy ufuncs on whole arrays; therefore, loop over all input values $x$ like the following code:

```python
results = np.array([test_sin(x) for x in Xsmall])
```

Plot the approximation and the exact solution:

```python
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
def test_plot_sine(X, filename="sine_error.pdf"):
    results = np.array([test_sin(x) for x in X])
    return plot_results(results, filename)

def test_plot_bad_sine(X, filename="sine_bad_error.pdf"):
    results = np.array([test_sin_bad(x) for x in X])
    return plot_results(results, filename)

def plot_results(results, filename):
    fig = plt.figure(figsize=(6, 8))

    ax1 = fig.add_subplot(4,1,1)
    ax1.plot(results[:, 0], results[:, 2], 'k-', lw=1, label="series")
    ax1.plot(results[:, 0], np.sin(results[:, 0]), 'g--', lw=2, label="sin x")
    ax1.legend(loc="best")

    ax2 = fig.add_subplot(4,1,2)
    ax2.plot(results[:, 0], results[:, 1], label="max n")
    ax2.legend(loc="best")

    ax3 = fig.add_subplot(4,1,3)
    ax3.semilogy(results[:, 0], results[:, 3], label="rel.error")
    ax3.legend(loc="best")

    ax4 = fig.add_subplot(4, 1, 4)
    ax4.semilogy(results[:, 0], results[:, 4], label="abs.error")
    ax4.legend(loc="best")

    fig.suptitle("sine series approximation")

    fig.savefig(filename)
    print("saved to file {0}".format(filename))
```

### Recursive implementation

Now test the two ranges of numbers and write to different files:

For a few oscillations near $x=0$ the approximation appears to hold pretty well with relative errors near machine precision.

```python
test_plot_sine(Xsmall, filename="sine_error_Xsmall.pdf")
```

But for larger $|x|$, the approximation breaks down and round-off error completely swamps the result:

```python
test_plot_sine(Xlarge, filename="sine_error_Xlarge.pdf")
```

#### More exploration

```python
test_plot_sine(np.pi*np.arange(-5, 5, 0.05))
```

```python
test_plot_sine(np.pi*np.arange(-15, 15, 0.05))
```

### Direct implementation
Let's see how the (supposedly) bad implementation holds up:

```python
test_plot_bad_sine(Xsmall, filename="sine_bad_error_Xsmall.pdf")
```

For the small range, the direct implementation performs similarly to the recursive one.

```python

```

The `Xlarge` range overflows with an `OverflowError`

```python
test_plot_bad_sine(Xlarge, filename="sine_bad_error_Xlarge.pdf")
```

so we choose a smaller one:

```python
X10 = np.pi * np.arange(-15, 15.1, 0.05)
test_plot_bad_sine(X10, filename="sine_bad_error_X10.pdf")
```

In this range, the direct implementation and the recursive one perform similarly badly (although the direct one is much slower).

### Speed comparison

The recursive implementation is faster:

```python
%timeit sin_series(-10/3*np.pi)
```

    10.3 µs ± 129 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

```python
%timeit sin_series_bad(-10/3*np.pi)
```

    32.8 µs ± 571 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

```python
sin_series(-10/3*np.pi)
```

    (0.866025403783719, 28)

```python
sin_series_bad(-10/3*np.pi)
```

    (0.8660254037851793, 28)

## Conclusion

The recursive implementation is better because it is faster and does not suffer from overflows. However, in terms of accuracy it is as bad as the direct implementation. Both fail for $|x| > 10\pi$.

```python

```