# MAST30032: Biological Modelling and Simulation

Author: Stephen Zhang

## Discrete-time models

Discrete-time models in general concern _difference equations_. We will focus on _first order_ difference equations, which describe sequences $\{x_n\}$ such that:

$$x_{n+1} \leftarrow f(x_n)$$

The function $f(\cdot)$ is called the _map_.

### Solution of first-order linear difference equations

Consider an equation with the general form

$$x_{n+1} = ax_n + P(x), a \in \mathbb{R}$$

We rewrite this as $x_{n+1} - ax_n = P(n)$, then split into _homogeneous_ and _inhomogeneous_ equations:
\begin{align}
    x_{n+1} - ax_n &= 0 \quad &\text{homogeneous}\\
    x_{n+1} - ax_n &= P(n) \quad &\text{inhomogeneous}\\
\end{align}

A simple solution (verify this) of the homogeneous solution is ${x_n}^h = Ca^n$. We need to **guess** some $Q(n)$, a solution to the inhomogeneous equation.

The **overall** solution is given by superposition:

$$x_n = {x_n}^h + {x_n}^i = Ca^n + Q(n)$$

### Behaviour of systems of the form $x_{n+1} = rx_n$

Consider the case $x_{n+1} = rx_n$ which has solution $x_n = r^tx_0$. There are **seven** classes

1. $r = 0$: solution is zero except possibly $x_0$
2. $r = 1$: solution is constant everywhere
3. $r = -1$: solution oscillates between $+x_0$ and $-x_0$
4. $r \in (0, 1)$: $x_n \to 0$ as $n \to \infty$ monotonically
5. $r \in (-1, 0)$: $x_n \to 0$ as $n \to \infty$ by bounded oscillation
6. $r > 1$: $x_n \to \infty$ as $n \to \infty$ monotonically
7. $r < -1$: $|x_n| \to \infty$ as $n \to \infty$ and $x_n$ oscillates unboundedly

### Analysis by cobwebbing

Cobwebbing is a graphical method for determining the long-term behaviour of a discrete dynamical system. Consider the general case $x_{n+1} = f(x_n)$. Fixed points of our system are solutions of $x = f(x)$.

That is, they will be the intersection between the graph of $y = f(x)$ and $y = x$.

Here, we see the cobweb of $x_{n+1} = \sqrt{x}$ starting from $x_0 = 0.6$. Observe that we converge towards $x = 1$, one of the fixed points of our dynamical system. We call $x = 1$ **stable**, because starting from $x = 0.6$ results in attraction to $x = 1$. On the other hand, $x = 0$ is an **unstable** fixed point, since starting close to $x = 0$ will result in repulsion (see below).

Cobwebbing can show us situations that result in oscillatory behaviour around a fixed point, such as when $f(x) = 1-0.5x^2$ and $x_0 = 0.91$:

The results observed from cobwebbing are formalised as _linear stability analysis_.

### General linear stability analysis

Consider the dynamical system described by:

$$u_{t+1} = f(u_t ; r)$$

Here, $r$ is some parameter in the model (its importance will show later!)

Let us first determine the _stable (fixed) points_ of our system - such points are solutions to the equation $u = f(u; r)$. Let $u^*$ be such a stable point.

Clearly, if $u_0 = u^*$ then $u_t = u^*$ everywhere. This is not interesting - we want to know about the behaviour of our system _close_ to $u^*$. Let us thus consider some perturbation $\delta_n$ away from our fixed point - we would like to know the _fate_ of this perturbation as it propagates with the evolution of time in our system.

Consider using the Taylor series expansion:

\begin{align}
    u^* + \delta_{n+1} &= f(u^* + \delta_n; r)\\
    &= f(u^*) + \delta_nf'(u^*) + \mathcal{O}(\delta_n^2)
\end{align}

Since $\delta_n \ll 1$, we have

$$u^* + \delta_{n+1} = \underbrace{(u^*)}_{ = u^*} + \delta_nf'(u^*)$$

Thus

$$\delta_{n+1} = f'(u^*)\delta_n$$

*** Eigenvalues of steady states***

We call $\lambda = f'(u^*)$ the _eigenvalue_ of the system associated with the steady state $u^*$.

Clearly, if $|\lambda| < 1$, then $\delta_{n} \to 0$ as $n \to \infty$; that is, $u^*$ is said to be stable.
* Specifically, if $-1 < \lambda < 0$, we obtain _oscillatory_ behaviour.

Consider the case $\lambda = \pm 1$. This corresponds to the points in our parameter space where the nature of our steady state *changes*.

Consider the **quadratic map** specified by $f(x) = rx(1-x)$.

Solving for steady states, we obtain:
\begin{align*}
    x^*_1 &= 0\\
    x^*_2 &= 1 - \dfrac{1}{r}\\
\end{align*}

Compute the derivative at these points using $f'(x) = r\left( 1-x\right) -rx$

\begin{align*}
    f'(x^*_1) &= r\\
    f'(x^*_2) &= 2-r
\end{align*}

$x^*_1 = 0$ is stable for $|r| < 1$. $x^*_2 = 1 - 1/r$ is stable for $1<r<3$. We can thus plot the **stable** solutions to our system as a function of $r$.

```python
import matplotlib as mp
import matplotlib.pyplot as plt
plt.figure()
x = [0.01*i for i in range(0, 300)]
y = [0 for i in range(0, 100)] + [1 - 1/(0.01*i) for i in range(100, 300)]
plt.plot(x, y)
plt.title("Plot of stable fixed points for map $f(x) = rx(1-x)$ for $r \in [0, 3]$")
```

```python
from iterate import iterate
r = 0.5
x = iterate(30, lambda x: r*x*(1-x), x0 = 0.5)

plt.figure()
plt.scatter([i[0] for i in x], [i[1] for i in x])
plt.title("Plot of iterative solution with r = 0.5")
plt.ylim(-0.1, 1)

r = 1.5
x = iterate(30, lambda x: r*x*(1-x), x0 = 0.01)
plt.figure()
plt.scatter([i[0] for i in x], [i[1] for i in x])
plt.ylim(0, 1)
plt.title("Plot of iterative solution with r = 1.5")

r = 3.1
x = iterate(30, lambda x: r*x*(1-x), x0 = 0.01)
plt.figure()
plt.scatter([i[0] for i in x], [i[1] for i in x])
plt.ylim(0, 1)
plt.title("Plot of iterative solution with r = 3.1")
```

We may programmatically determine the stable points of our system as a function of $r$. The resulting plot is a **bifurcation plot**

```python
from iterate import iterate
import numpy as np

x0 = 0.1
round_prec = 5
points = 500

# finding attracting fixed points by iterating for 2000 cycles, taking last 500
# iterates and then binning values

plt.figure()

r_values = np.linspace(0, 4, num = points)

for r in r_values:
    stable_points = set([])

    v = iterate(2000, lambda x: r*x*(1-x), x0 = x0)
    t = v[-500:]
    t = [round(i[1], round_prec) for i in t]

    for i in t:
        stable_points.add(i)

    plt.scatter([r, ]*len(stable_points), list(stable_points), s = 0.1)

plt.title('Bifurcation plot for $x_{n+1} = rx_n(1-x_n)$, with $0 < r < 4$')
plt.show()
```

In the last case where $r = 3.1$, we see that our steady state is a **2-cycle** that alternates between 2 values. We say that $r$ increased past the bifurcation point $r = 3$.

How does this arise?

For $r > 3$, consider some fixed point of our map, $u^* = f(u^*)$. Clearly, $u^* = f(f(u^*))$. That is,

$$u^* = f^2(u^*)$$

We can consider solutions to the second-iterate map.

$$r^2(1-u^*)u^*(1-r(1-u^*)u^*) = u^*$$

We can attempt to solve this equation and recover the solutions:
\begin{align*}
    u^* &= 0 &\text{for } |r| < 1\\
    u^* &= 1 - 1/r &\text{for } 1 < r < 3\\
    u^* &= \alpha, \beta &\text{for } r > 3
\end{align*}

We can now consider stability under the second-iterate map $f^2$, so we consider $\mathrm{d}_x f^2$ at our fixed points to determine where they are stable.

Following on from this, we may consider the next iteration of our map, $f^3$, and so on.

In general, to investigate the stability of specific points on our map, we may consider:

$$u^* = f^m(u^*) \Rightarrow \lambda = \dfrac{\partial f^m}{\partial u}(u^*) = \Pi_{i = 0}^{m-1} f'(u_i)$$

### Systems of difference equations

Definition: a *k*th order discrete system of difference equations is described by an expression of the form:

$$X_{t+k} = f(X_{t+k-1}, X_{t+k-2}, ..., X_t ; t)$$

#### 'Reduction of order' for systems of difference equations

A system of order $k > 1$ can be reduced to a first order system by augmenting the number of variables:

Consider

$$y_{t+2} = g(y_{t+1}, y_t)$$

We set $u_t = y_{t+1}$ and $v_t = y_t$. This gives:

\begin{align*}
    u_{t+1} &= g(u_t, v_t)\\
    v_{t+1} &= u_t
\end{align*}

This is a *first order* system that can be solving methods we will presently show.

#### Analysis of linear systems of difference equations

A linear system of difference equations can be expressed in the form

$$\vec{x}_{n+1} = \mathbf{A}\vec{x}_n + \vec{c}$$

Where $\vec{x}_n = (x_{1,n}, x_{2,n}, ..., x_{m, n})$ and $\mathbf{A}$ is a matrix of constants, $\vec{c}$ is a vector of constants.

Consider a certain solution where $\vec{x}$ takes the form $\vec{x}_n = (A\lambda^n, B\lambda^n) = \lambda^n \vec{x}_0$. Thus we achieve the following:

\begin{align*}
\lambda^{n+1}\vec{x}_0 &= \mathbf{A} \lambda^n \vec{x}_0\\
(\mathbf{A} - \lambda \mathbf{I}) \vec{x}_0 &= \mathbf{0}
\end{align*}

Clearly in this situation, $\vec{x}_0 \ne \vec{0}$ in general so we require $\mathrm{det}(\mathbf{A} - \lambda\mathbf{I}) = 0$, i.e. $\lambda$ must be an **eigenvalue** of $\mathbf{A}$.

Consequently for our system to undergo the behaviour $\vec{x}_{n+1} = \lambda \vec{x}_n$, $\vec{x}_0$ must be *an eigenvector* corresponding to the eigenvalue $\lambda$.

Our solution will converge $\vec{x}_n \to \vec{0}$ if $|\lambda| < 1$, and diverge if $|\lambda| > 1$.

***What happens in the general case where $\vec{x}_0$ is a general vector?***

Assuming $\mathbf{A}$ is nonsingular ($\Rightarrow$ $\mathbf{A}$ is of full rank), the rank-nullity theorem implies it will have a 'non-degenerate eigenspace'.

Thus, for any $\vec{x}_n$ we may write an expansion in basis vectors:

$$\vec{x}_n = \sum_i k_i \vec{e}_i$$

Thus

$$\vec{x}_{n+1} = \mathbf{A}\vec{x}_n = \mathbf{A}\sum_i k_i\vec{e}_i = \sum_i \lambda_ik_i\vec{e}_i$$

From this it is clear that:

* To ensure that $\vec{x}_n \to \vec{0}$, $|\lambda_i| < 1$ for **all** our eigenvalues.
* If **one** of our eigenvalues is $>1$, one of our eigenvector expansion terms will explode.

#### Nonlinear systems

We can consider the most general case of our problem:

\begin{equation}
\begin{bmatrix}x_{n+1}\\y_{n+1}\end{bmatrix} = \begin{bmatrix}f(x_n, y_n)\\g(x_n, y_n)\end{bmatrix} \Rightarrow \vec{x}_{n+1} = \mathbf{F}(\vec{x}_n)
\end{equation}

Consider some fixed point $\vec{x}^*$ of our system, and take some small perturbation $\vec{\delta}_n$ with $|\vec{\delta}_n| \ll 1$

As with univariate linear stability analysis, we can consider the behaviour of our system around the stable state.

\begin{align*}
    \vec{x}^* + \vec{\delta}_{n+1} &= \mathbf{F}(\vec{x}^* + \vec{\delta}_n)\\
    \vec{x}^* + \vec{\delta}_{n+1} &= \vec{x}^* + \mathbf{DF}(\vec{x}^*)\vec{\delta}_n + \vec{e}\\
    \vec{\delta}_{n+1} &\approx \mathbf{DF}(\vec{x}^*)\vec{\delta}_n
\end{align*}

Here, $\mathbf{DF}$ is the derivative matrix

\begin{equation}
\mathbf{DF} = \begin{bmatrix}f_x&f_y\\g_x&g_y\end{bmatrix}
\end{equation}

We can apply the stability analysis from the linear case to deduce that our fixed point will be **stable** if for **all** the eigenvalues of $\mathbf{DF}$ we have $\lambda_i < 1$.

### Applications - Verhulst process (discrete logistic)

Consider the discrete logistic equation:

$$x_{n+1} = rx_n(1-x_n)$$

(a) We plot $x_n$ for $r = 1.9, 2.9$ against the long term population size $(r-1)/r$.

```python
from math import *
for r in [1.9, 2.9, 3.2, 3.5, 3.55, 3.571, 3.89]:
    n_iter = 2000
    v = iterate(n_iter, lambda x: r*x*(1-x), x0 = 0.2)
    plt.figure()
    plt.scatter([log10(i[0]+1) for i in v], [i[1] for i in v], marker = "+")
    plt.plot([0, log10(n_iter+1)], [(r-1)/r, ]*2, c = 'r', marker = "+")
    plt.title('$r = %f$' % r)
```

Consider the Beverton-Holt model, which behaves nicer:

$$x_{n+1} = \dfrac{r}{1 + ((r-1)/K)x_n}x_n$$

```python
import numpy as np
from colour import Color
red = Color("red")

n_runs = 100

colors = list(red.range_to(Color("blue"), n_runs))

K = 0.8

n_iter = 300

def f(x):
    return (r/(1+((r-1)/K)*x))*x

for (r, col) in zip(np.linspace(0, 5, n_runs), colors):
    v = iterate(n_iter, f, x0 = 0.2)
    plt.scatter([log(i[0]+1) for i in v], [i[1] for i in v], marker = "+", c = col.rgb)
```

## Population genetics

### Hardy-Weinberg Equilibrium

The Hardy-Weinberg law states that the genetic variation in a (large) population assuming random mating will remain constant from generation to generation, excluding disturbing factors.

* Allele frequencies **never** change
* Genotype frequencies change **no more than once**

Derivation:

Consider a population with an genotype frequency at step $n$ as follows:
\begin{align*}
f^{AA}(n) : f^{Aa}(n) : f^{aa}(n)\\
\end{align*}

Let $p_n = f^{A}(n)$ and $q_n = f^{a}(n) = 1-p_n$ be our allele frequencies.

Since we assume random mating, each breeding step allows for **random mixing of alleles**. This allows us to reason that our new genotype frequencies in step $n+1$ will be:

\begin{align*}
    f^{AA}(n+1) : f^{Aa}(n+1) : f^{aa}(n+1) = p_n^2 : 2p_nq_n : q_n^2
\end{align*}

Hence our allele frequencies in generation $(n+1)$ will be:

\begin{align*}
    p_{n+1} &= f^{AA}(n) + \dfrac{1}{2}f^{Aa}(n)
            &= p_n^2 + p_nq_n\\
            &= p_n(\underbrace{p_n + q_n}_{ = 1})
            &= p_n
\end{align*}

Thus, $p_{n+1} = p_n \Rightarrow q_{n+1} = q_n$, i.e. our allele frequencies never change.

Since we have  $f^{AA}(n+1) : f^{Aa}(n+1) : f^{aa}(n+1) = p_n^2 : 2p_nq_n : q_n^2$, we also see that our genotype frequencies will be constant for all $n$, except for possibly $n = 0$. i.e. our genotype frequencies change at most once.

### Selection

What happens when we factor selection into our model? Consider the case where:
* Genotypes {AA, Aa} $\to$ phenotype A*
* Genotypes {aa} $\to$ phenotype a*

That is, A is the dominant allele, and a is recessive.

Let $\alpha$ be the proportion of individuals with phenotype A* that survive from one generation to the next.
Let $\gamma$ be the proportion of individuals with phenotype a* that survive from one generation to the next.

Clearly we have $0 \le \alpha, \gamma \le 1$.

Suppose we begin with some phenotype ratios $p_n, q_n$ in one generation. Assuming random mating, and assuming that selection is carried out such that only the correct proportion of adults survive, we have our new genotype frequencies for generation $(n+1)$.

\begin{align*}
    p_n^2 : 2p_nq_n : q_n^2 \quad \text{ before selection }\\
    \alpha p_n^2 : 2\alpha p_nq_n : \gamma q_n^2 \quad \text { after selection }
\end{align*}

Thus, our allele frequencies are:

\begin{align*}
    f^A(n+1) &= \dfrac{\alpha p_n^2 + \alpha p_nq_n}{\alpha p_n^2 + 2\alpha p_n q_n + \gamma q_n^2}\\
             &= \dfrac{\alpha p_n}{\alpha p_n^2 + 2\alpha p_n(1-p_n) + \gamma (1-p_n)^2}
\end{align*}

So

$$p_{n+1} = \dfrac{\alpha p_n}{\alpha p_n^2 + 2\alpha p_n(1-p_n) + \gamma (1-p_n)^2}$$

Letting $\alpha = \gamma$ (no selection), we recover that $p_{n+1} = p_n$.

We can determine two fixed points for our map: $p^* = 0, 1$.

Linear stability analysis provides us that

\begin{align*}
    f'(0) &= \alpha/\gamma\\
    f'(1) &= 1
\end{align*}

We observe that $p^* = 0$ is stable when $|\alpha/\gamma| < 1 \Rightarrow \alpha < \gamma$. That is, when {AA, Aa} is **less fit** than {aa}, we will have $f^A \to 0$ as $n \to \infty$.

For $p^* = 1$, we cannot use LSA to draw conclusions on its stability since $f' = 1$

### Selection for 3 phenotypes

Consider the previous analysis, but now we have 3 phenotypes AA*, Aa*, and aa*, associated with the 'survival fractions' $\alpha, \beta\, \gamma$.

Our allele frequencies thus become:

$$\alpha p_n^2 : 2\beta p_n q_n : \gamma q_n^2$$

Carrying out similar analysis as previously, we obtain

$$p_{n+1} = \dfrac{\alpha p_n^2 + \beta p_n(1-p_n)}{\alpha p_n^2 + 2\beta p_n (1-p_n) + \gamma (1-p_n)^2}$$

This is the **Fisher-Haldane-Wright** equation.

We can normalize our weights $\alpha : \beta : \gamma \equiv 1 : \hat{\beta} : \hat{\gamma}$ and thus obtain

$$p_{n+1} = \dfrac{p_n^2 + \hat{\beta}p_n(1-p_n)}{p_n^2 + 2\hat{\beta}p_n(1-p_n) + \hat{\gamma}(1-p_n)^2}$$

This reduces our 3-parameter model to a 2-parameter model. Now take $\hat\gamma = 1 -s$. $s$ is called the *selection coefficient*. Also, take $\hat\beta = 1 - hs$.

By writing $\alpha p_n^2 + 2\beta p_n (1-p_n) + \gamma (1-p_n)^2  = \bar{w}$, we have

$$p_{n+1} = \dfrac{p_n^2 + (1-hs)p_nq_n}{\bar w}$$

#### Analysis

We can solve for our equilibria, and we find our stable points to be $p^* = \underbrace{0, 1}_{\text{always solutions}}
, \dfrac{h-1}{2h-1}$.

Of interest is the case where $p^* = \dfrac{h-1}{2h-1}, h \ne 1/2$.

We thus note that the $h, s$ parameterization of our problem is valuable because it allows the stable solutions to be determined by only $h$!

We now require $0 \le (h-1)/(2h-1) \le 1$ in order for this stable point to have biological meaning. Observing a plot of the function, we see that we have allowable values of $p^*$ for:

* $0 \le h \le 1$: no allowable $p^*$, fixed points are only 0, 1
* $h \ge 1$: $p^* = (h-1)/(2h-1) < 1/2$ is a fixed point (in addition to 0, 1)
* $h \le 0$: $p^* = (h-1)/(2h-1) > 1/2$ is a fixed point (in addition to 0, 1)

Clearly, $h$ can occupy values $h > 1$ (Aa less fit than AA, aa) or $h < 0$ (Aa more fit than AA, aa).

Linear stability analysis yields the following:

* $\lambda(p^* = 0) = \dfrac{1-hs}{1-s}$
* $\lambda(p^* = 1) = 1-hs$
* $\lambda(p^* = (h-1)/(2h-1))$ is quite complicated!

```python
from iterate import iterate

s = 0.3
h = 0.1

def f(p):
    return (p**2 + (1-h*s)*(1-p)*p)/(p**2 + 2*(1-h*s)*p*(1-p) + (1-s)*(1-p)**2)

n_iter = 200

x = iterate(n_iter, lambda x: f(x), x0 = 0.1)

plt.figure()
plt.scatter([i[0] for i in x], [i[1] for i in x])
plt.plot([0, n_iter], [1, ]*2)
plt.plot([0, n_iter], [0, ]*2)
plt.title("FHW equation iteration for $h = 0.1, s = 0.3$")

h = -1.0
s = 0.2
# Here h < 0 and so we are dealing with a heterozygote advantage

x = iterate(n_iter, lambda x: f(x), x0 = 0.05)
plt.figure()
plt.scatter([i[0] for i in x], [i[1] for i in x])
plt.plot([0, n_iter], [1, ]*2)
plt.plot([0, n_iter], [0, ]*2)
plt.plot([0, n_iter], [(h-1)/(2*h-1), ]*2)
plt.title("FHW equation iteration for $h = -1.0, s= 0.2$")
```

MAST30032 - Biological Modelling and Simulation Semester 1 2018.ipynb


```python
import matplotlib.pyplot as plt

def model():
    """Solve u'' = -1, u(0)=0, u'(1)=0."""
    import sympy as sym
    x, c_0, c_1, = sym.symbols('x c_0 c_1')
    u_x = sym.integrate(1, (x, 0, x)) + c_0
    u = sym.integrate(u_x, (x, 0, x)) + c_1
    r = sym.solve([u.subs(x,0) - 0,
                   sym.diff(u,x).subs(x, 1) - 0],
                  [c_0, c_1])
    u = u.subs(c_0, r[c_0]).subs(c_1, r[c_1])
    u = sym.simplify(sym.expand(u))
    return u

def midpoint_rule(f, M=100000):
    """Integrate f(x) over [0,1] using M intervals."""
    from numpy import sum, linspace
    dx = 1.0/M                       # interval length
    x = linspace(dx/2, 1-dx/2, M)    # integration points
    return dx*sum(f(x))

def check_integral_b():
    from numpy import pi, sin
    for i in range(12):
        exact = 2/(pi*(2*i+1))
        numerical = midpoint_rule(
            f=lambda x: sin((2*i+1)*pi*x/2))
        print((i, abs(exact - numerical)))

def sine_sum(x, N):
    s = 0
    from numpy import pi, sin, zeros
    u = [] # u[k] is the sum i=0,...,k
    k = 0
    for i in range(N+1):
        s += - 16.0/((2*i+1)**3*pi**3)*sin((2*i+1)*pi*x/2)
        u.append(s.copy())  # important with copy!
    return u

def plot_sine_sum():
    from numpy import linspace
    x = linspace(0, 1, 501)  # coordinates for plot
    u = sine_sum(x, N=10)
    u_e = 0.5*x*(x-2)
    N_values = 0, 1, 10
    for k in N_values:
        plt.plot(x, u[k])
    plt.plot(x, u_e)
    plt.legend(['N=%d' % k for k in N_values] + ['exact'],
               loc='upper right')
    plt.xlabel('$x$');  plt.ylabel('$u$')
    plt.savefig('tmpc.png'); plt.savefig('tmpc.pdf')

def check_integral_d():
    from numpy import pi, sin
    for i in range(24):
        if i % 2 == 0:
            exact = 2/(pi*(i+1))
        elif (i-1) % 4 == 0:
            exact = 2*2/(pi*(i+1))
        else:
            exact = 0
        numerical = midpoint_rule(
            f=lambda x: sin((i+1)*pi*x/2))
        print((i, abs(exact - numerical)))

def check_integral_d_sympy_answer():
    from numpy import pi, sin
    for i in range(12):
        exact = 2/(pi*(i+1))
        numerical = midpoint_rule(
            f=lambda x: sin((i+1)*pi*x/2))
        print((i, abs(exact - numerical)))

def sine_sum_d(x, N):
    s = 0
    from numpy import pi, sin, zeros
    u = []  # u[k] is the sum i=0,...,k
    k = 0
    for i in range(N+1):
        if i % 2 == 0:       # even i
            s +=   - 16.0/((i+1)**3*pi**3)*sin((i+1)*pi*x/2)
        elif (i-1) % 4 == 0:   # 1, 5, 9, 13, 17
            s += - 2*16.0/((i+1)**3*pi**3)*sin((i+1)*pi*x/2)
        else:
            s += 0
        u.append(s.copy())
    return u

def plot_sine_sum_d():
    from numpy import linspace
    x = linspace(0, 1, 501)  # coordinates for plot
    u = sine_sum_d(x, N=20)
    u_e = 0.5*x*(x-2)
    N_values = 0, 1, 2, 3, 20
    for k in N_values:
        plt.plot(x, u[k])
    plt.plot(x, u_e)
    plt.legend(['N=%d' % k for k in N_values] + ['exact'],
               loc='upper right')
    plt.xlabel('$x$');  plt.ylabel('$u$')
    #plt.axis([0.9, 1, -0.52, -0.49])
    plt.savefig('tmpd.png'); plt.savefig('tmpd.pdf')

if __name__ == '__main__':
    import sys
    print((model()))
    print('sine 2*i+1 integral:')
    check_integral_b()
    print('sine i+1 integral, sympy answer:')
    check_integral_d_sympy_answer()
    print('sine i+1 integral:')
    check_integral_d()
    #sys.exit(0)
    plot_sine_sum()
    plt.figure()
    plot_sine_sum_d()
    plt.show()
```

```python

```

Data Science and Machine Learning/Machine-Learning-In-Python-THOROUGH/EXAMPLES/FINITE_ELEMENTS/INTRO/EXERCICES/05_CABLE_SIN.ipynb


# Riemann Zeta Functions $\zeta$

The simplest representation is

$$ \zeta(s) = \sum_{n=1}^{\infty} \frac{1}{n^s} $$

Often in physics problems, $s$ will be integer. On the real line with $s>1$ we have an alternative definition:

$$ \zeta(s) = \frac{1}{\Gamma(s)} \int_0^{\infty} \frac{u^{s-1}}{e^u - 1} du $$

This is an integral that is needed in quite common problems such as blackbody radiation. Turning it around slightly:

$$ \int_0^{\infty} \frac{x^{\alpha}}{e^x - 1} dx = \Gamma(\alpha + 1) \zeta(\alpha + 1) $$

$\zeta(s)$ is defined over the complex plane, apart from some singularities (such as $s=1$). This is important in number theory and has spawned a vast literature, not immediately comprehensible to most of us.

Does $\zeta(s)$ have a simple representation for integer $s$? Some do, some don't:

```python
from IPython.display import Math
from sympy import zeta, latex
from sympy.abc import x, s

for i in range(-7, 7):
    zeta_i = latex(zeta(i))
    display(Math('\zeta({}) = {}'.format(i, zeta_i)))
```

$\displaystyle \zeta(-7) = \frac{1}{240}$

$\displaystyle \zeta(-6) = 0$

$\displaystyle \zeta(-5) = - \frac{1}{252}$

$\displaystyle \zeta(-4) = 0$

$\displaystyle \zeta(-3) = \frac{1}{120}$

$\displaystyle \zeta(-2) = 0$

$\displaystyle \zeta(-1) = - \frac{1}{12}$

$\displaystyle \zeta(0) = - \frac{1}{2}$

$\displaystyle \zeta(1) = \tilde{\infty}$

$\displaystyle \zeta(2) = \frac{\pi^{2}}{6}$

$\displaystyle \zeta(3) = \zeta\left(3\right)$

$\displaystyle \zeta(4) = \frac{\pi^{4}}{90}$

$\displaystyle \zeta(5) = \zeta\left(5\right)$

$\displaystyle \zeta(6) = \frac{\pi^{6}}{945}$

For real $s$, Python has a few issues with $s < 1$. The function `scipy.special.zeta(x)` returns `nan` for $s \le 1$. A related function `scipy.special.zetac(x)` is more flexible, but the values returned are offset to $\zeta(s) - 1$ and the documentation warns it returns `nan` for $x < -30.8148$.

```python
import numpy as np
import matplotlib.pyplot as plt

plt.rcParams.update({'font.size': 16})

import scipy.special as sp
xlims = (-20, 20)
x = np.linspace(xlims[0], xlims[1], 500)
zetas = sp.zetac(x) + 1
# display(zetas)

plt.figure(figsize=(9, 9))
plt.plot(x, zetas)

plt.xlim(xlims)
plt.ylim((-0.5, 1.5))
plt.xlabel('$x$')
plt.ylabel('$\\zeta(x)$')
plt.title('Plot of the zetas function for real x')
plt.grid(True)
```

## References:

- MathWorld, http://mathworld.wolfram.com/RiemannZetaFunction.html
- Wikipedia, https://en.wikipedia.org/wiki/Riemann_zeta_function

```python

```


```python
import sympy as sym
from IPython.display import display, Math, Latex
import pandas as pd
import numpy as np
def func_str(function):
    return str(function)
    function_str = str(function)
    function_str = function_str.replace("**", "^")
    return function_str
class Iterative:
    def __init__(self, function, *args):
        self.variables = []
        self.function = function
        self.data_dict = {"iterate":[],"G_val":[]}
        for variable in args:
            self.variables.append(variable)
            self.data_dict["der_" + str(variable)] = []
            self.data_dict[str(variable)] = []
        self.partial_deravatives()

    def partial_deravatives(self):
         self.deravatives = []
         i = 0
         for variable in self.variables:
             self.deravatives.append(sym.diff(self.function, variable))
             display(Math(r"\frac{\partial G(\mathbf{Y_k})}{\partial y_%d}" % (i+1) + " = " + str(self.deravatives[i])))
             i += 1

    def calc_limit_state(self, numerical_vars):
        temp_dict = {}
        to_show = "G("
        for index,variable in enumerate(self.variables):
            temp_dict[variable] =numerical_vars[index]
            if index != (len(self.variables)-1):
                to_show += str(variable) + " = " + str(numerical_vars[index]) + ", "
            else:
                to_show += str(variable) + " = " + str(numerical_vars[index]) + ")"
        display(Math(to_show))
        return self.function.subs(temp_dict)

    def calc_gradient(self, numerical_vars):
        gradient = []
        temp_dict = {}
        for index, variable in enumerate(self.variables):
            temp_dict[variable] = numerical_vars[index]

        for deravative in self.deravatives:
            gradient.append(deravative.subs(temp_dict))

        return gradient

    def make_dataframe(self):
        return pd.DataFrame(data=self.data_dict)

    def iterate(self):
        i = 0
        self.ys = []
        while(True):
            if(i == 0):
                for j in range(len(self.variables)):
                    self.ys.append(0)
            self.data_dict["iterate"].append(i)
            i += 1
            gradient = np.array(self.calc_gradient(self.ys))
            limit_state = self.calc_limit_state(self.ys)
            ys = np.array(self.ys).reshape(-1,1)
            gradient_measurement = sum(gradient**2)
            self.ys = (1/gradient_measurement)*(gradient.dot(ys)[0] - limit_state) * gradient
            self.data_dict["G_val"].append(limit_state)
            for index in range(len(self.variables)):
                self.data_dict["der_" + str(self.variables[index])].append(gradient[index])
                self.data_dict[str(self.variables[index])].append(ys[index][0])

            if abs(limit_state) < 0.001:
                break

    def get_beta(self):
        self.beta = (sum(self.ys**2))**0.5
        display(Math(r"\beta = " + str(self.beta)))
        return self.beta

y_1, y_2 = sym.symbols('y_1 y_2')

G =  10*y_1 - 4*y_2 + 12

iterative = Iterative(G, y_1, y_2)
iterative.iterate()
iterative.get_beta()
iterative.make_dataframe().round(3)
```

$\displaystyle \frac{\partial G(\mathbf{Y_k})}{\partial y_1} = 10$

$\displaystyle \frac{\partial G(\mathbf{Y_k})}{\partial y_2} = -4$

$\displaystyle G(y_1 = 0, y_2 = 0)$

$\displaystyle G(y_1 = -30/29, y_2 = 12/29)$

$\displaystyle \beta = 1.11417202906231$

      iterate
      G_val
      der_y_1
      y_1
      der_y_2
      y_2

      0
      0
      12
      10
      0
      -4
      0

      1
      1
      0
      10
      -30/29
      -4
      12/29


# K-Nearest Neighbors Algorithm

* Last class, we introduced the probabilistic generative classifier.
* As discussed, the probabilistic generative classifier requires us to assume a parametric form for each class (e.g., each class is represented by a multi-variate Gaussian distribution, etc..).  Because of this, the probabilistic generative classifier is a *parametric* approach.
* Parametric approaches have the drawback that the functional parametric form needs to be decided/assumed in advance and, if chosen poorly, might be a poor model of the distribution that generates the data resulting in poor performance.
* Non-parametric methods are those that do not assume a particular generating distribution for the data.  The $K$-nearest neighbors algorithm is one example of a non-parametric classifier.

* Nearest neighbor methods compare a test point to the $k$ nearest training data points and then estimate an output value based on the desired/true output values of the $k$ nearest training points
* Essentially, there is no ''training'' other than storing the training data points and their desired outputs
* In test, you need to: (1) determine which $k$ training data points are closest to the test point; and (2) determine the output value for the test point
* In order to find the $k$ nearest neighbors in the training data, you need to define a *similarity measure* or a *dissimilarity measure*.  The most common dissimilarity measure is Euclidean distance.
    * Euclidean distance: $d_E = \sqrt{\left(\mathbf{x}_1-\mathbf{x}_2\right)^T\left(\mathbf{x}_1-\mathbf{x}_2\right)}$
    * City block distance: $d_C = \sum_{i=1}^d \left| x_{1i} - x_{2i} \right|$
    * Mahalanobis distance: $\left(\mathbf{x}_1-\mathbf{x}_2\right)^T\Sigma^{-1}\left(\mathbf{x}_1-\mathbf{x}_2\right)$
    * Cosine angle similarity: $\cos \theta = \frac{\mathbf{x}_1^T\mathbf{x}_2}{\left\|\mathbf{x}_1\right\|_2^2\left\|\mathbf{x}_2\right\|_2^2}$
    * and many more...

* If you are doing classification, once you find the $k$ nearest neighbors to your test point in the training data, then you can determine the class label of your test point using (most commonly) *majority vote*
* If there are ties, they can be broken randomly or using schemes like applying the label of the closest data point in the neighborhood
*  Of course, there are MANY modifications you can make to this.  A common one is to weight the votes of each of the nearest neighbors by their distance/similarity measure value.  If they are closer, they get more weight.

```python
# Reference for some code: http://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html

import numpy as np
from matplotlib import pyplot as plt
from matplotlib.colors import ListedColormap
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_moons, make_circles, make_classification
from sklearn import neighbors
%matplotlib inline

#figure params
h = .02  # step size in the mesh
figure = plt.figure(figsize=(17, 9))

#set up classifiers
n_neighbors = 10
classifiers = []
classifiers.append(neighbors.KNeighborsClassifier(n_neighbors, weights='uniform'))
classifiers.append(neighbors.KNeighborsClassifier(n_neighbors, weights='distance'))
names = ['K-NN_Uniform', 'K-NN_Weighted']

#Put together datasets
n_samples = 30
X, y = make_classification(n_samples, n_features=2, n_redundant=0, n_informative=2,
                           random_state=0, n_clusters_per_class=1)
rng = np.random.RandomState(2)
X += 2 * rng.uniform(size=X.shape)
linearly_separable = (X, y)

datasets = [make_moons(n_samples, noise=0.3, random_state=0),
            make_circles(n_samples,  noise=0.2, factor=0.5, random_state=1),
            linearly_separable]

i = 1
# iterate over datasets
for X, y in datasets:
    # preprocess dataset, split into training and test part
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.8)  #split into train/test folds

    #set up meshgrid for figure
    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))

    # just plot the dataset first
    cm = plt.cm.RdBu
    cm_bright = ListedColormap(['#FF0000', '#0000FF'])
    ax = plt.subplot(len(datasets), len(classifiers) + 1, i)
    # Plot the training points
    ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright)
    # and testing points
    ax.scatter(X_test[:, 0], X_test[:, 1], marker='+', c=y_test, cmap=cm_bright, alpha=0.6)
    ax.set_xlim(xx.min(), xx.max())
    ax.set_ylim(yy.min(), yy.max())
    ax.set_xticks(())
    ax.set_yticks(())
    i += 1

    # iterate over classifiers
    for name, clf in zip(names, classifiers):
        ax = plt.subplot(len(datasets), len(classifiers) + 1, i)
        clf.fit(X_train, y_train)
        score = clf.score(X_test, y_test)

        # Plot the decision boundary. For that, we will assign a color to each
        # point in the mesh [x_min, x_max]x[y_min, y_max].
        Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])

        # Put the result into a color plot
        Z = Z.reshape(xx.shape)
        ax.contourf(xx, yy, Z, cmap=cm, alpha=.8)

        # Plot also the training points
        ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright)
        # and testing points
        ax.scatter(X_test[:, 0], X_test[:, 1], marker='+', c=y_test, cmap=cm_bright,
                   alpha=0.4)

        ax.set_xlim(xx.min(), xx.max())
        ax.set_ylim(yy.min(), yy.max())
        ax.set_xticks(())
        ax.set_yticks(())
        ax.set_title(name)
        ax.text(xx.max() - .3, yy.min() + .3, ('%.2f' % score).lstrip('0'),
                size=15, horizontalalignment='right')
        i += 1

figure.subplots_adjust(left=.02, right=.98)
plt.show()
```

# Error and Evaluation Metrics

* A key step in machine learning algorithm development and testing is determining a good error and evaluation metric.

* Evaluation metrics help us to estimate how well our model is trained and it is important to pick a metric that matches our overall goal for the system.

* Some common evaluation metrics include precision, recall, receiver operating curves, and confusion matrices.

### Classification Accuracy and Error

* Classification accuracy is defined as the number of correctly classified samples divided by all samples:

\begin{equation}
\text{accuracy} = \frac{N_{cor}}{N}
\end{equation}
where $N_{cor}$ is the number of correct classified samples and $N$ is the total number of samples.

* Classification error is defined as the number of incorrectly classified samples divided by all samples:

\begin{equation}
\text{error} = \frac{N_{mis}}{N}
\end{equation}
where $N_{mis}$ is the number of misclassified samples and $N$ is the total number of samples.

* Suppose there is a 3-class classification problem, in which we would like to classify each training sample (a fish) to one of the three classes (A = salmon or B = sea bass or C = cod).

* Let's assume there are 150 samples, including 50 salmon, 50 sea bass and 50 cod.  Suppose our model misclassifies 3 salmon, 2 sea bass and 4 cod.

* Prediction accuracy of our binary classification model is calculated as:

\begin{equation}
\text{accuracy} = \frac{47+48+46}{50+50+50} = \frac{47}{50}
\end{equation}

* Prediction error is calculated as:

\begin{equation}
\text{error} = \frac{N_{mis}}{N} = \frac{3+2+4}{50+50+50} = \frac{3}{50}
\end{equation}

### Confusion Matrices

* A confusion matrix summarizes the classification accuracy across several classes. It shows the ways in which our classification model is confused when it makes predictions, allowing visualization of the performance of our algorithm. Generally, each row represents the instances of an actual class while each column represents the instances in a predicted class.

* If our classifier is trained to distinguish between salmon, sea bass and cod, then we can summarize the prediction result in the confusion matrix as follows:

| Actual/Predicted | Salmon | Sea bass | Cod  |
| --- | --- | --- | --- |
| Salmon | 47 | 2 | 1 |
| Sea Bass | 2 | 48 | 0 |
| Cod | 0 | 0 | 50 |

* In this confusion matrix, of the 50 actual salmon, the classifier predicted that 2 are sea bass, 1 is cod incorrectly and 47 are labeled salmon correctly. All correct predictions are located in the diagonal of the table. So it is easy to visually inspect the table for prediction errors, as they will be represented by values outside the diagonal.

### TP, FP, TN, and FN

* True positive (TP): correctly predicting event values
* False positive (FP): incorrectly calling non-events as an event
* True negative (TN): correctly predicting non-event values
* False negative (FN): incorrectly labeling events as non-event

* Precision is also called positive predictive value.

\begin{equation}
\text{Precision} = \frac{\text{TP}}{\text{TP}+\text{FP}}
\end{equation}

* Recall is also called true positive rate, probability of detection

\begin{equation}
\text{Recall} = \frac{\text{TP}}{\text{TP}+\text{FN}}
\end{equation}

* Fall-out is also called false positive rate, probability of false alarm.

\begin{equation}
\text{Fall-out} = \frac{\text{FP}}{\text{N}}= \frac{\text{FP}}{\text{FP}+\text{TN}}
\end{equation}

* *Consider the salmon/non-salmon classification problem, what are the TP, FP, TN, FN values?*

| Actual/Predicted | Salmon | Non-Salmon  |
| --- | --- | --- |
| Salmon | 47 | 3 |
| Non-Salmon | 2 | 98 |

### ROC curves

* The Receiver Operating Characteristic (ROC) curve is a plot between the true positive rate (TPR) and the false positive rate (FPR), where the TPR is defined on the $y$-axis and FPR is defined on the $x$-axis.

* $TPR = TP/(TP+FN)$ is defined as ratio between true positive prediction and all real positive samples. The definition used for $FPR$ in a ROC curve is often problem dependent.  For example, for detection of targets in an area, FPR may be defined as the ratio between the number of false alarms per unit area ($FA/m^2$).  In another example, if you have a set number of images and you are looking for targets in these collection of images, FPR may be defined as the number of false alarms per image.  In some cases, it may make the most sense to simply use the Fall-out or false positive rate.

* Given a binary classifier and its threshold, the (x,y) coordinates of ROC space can be calculated from all the prediction result.  You trace out a ROC curve by varying the threshold to get all of the points on the ROC.

* The diagonal between (0,0) and (1,1) separates the ROC space into two areas, which are left up area and right bottom area. The points above the diagonal represent good classification (better than random guess) which below the diagonal represent bad classification (worse than random guess).

* *What is the perfect prediction point in a ROC curve?*

### MSE and MAE

* *Mean Square Error* (MSE) is the average of the squared error between prediction and actual observation.

* For each sample $\mathbf{x}_i$, the prediction value is $y_i$ and the actual output is $d_i$. The MSE is

\begin{equation}
MSE = \sum_{i=1}^n \frac{(d_i - y_i)^2}{n}
\end{equation}

* *Root Mean Square Error* (RMSE) is simply the square root the MSE.

\begin{equation}
RMSE = \sqrt{MSE}
\end{equation}

* *Mean Absolute Error* (MAE) is the average of the absolute error.
\begin{equation}
MAE = \frac{1}{n} \sum_{i=1}^n \lvert d_i - y_i \rvert
\end{equation}

```python

```

08_KNN/08_Non-parametric K-Nearest Neighbors Algorithm.ipynb


Imports and set magics:

```python
import numpy as np
import matplotlib.pyplot as plt
import ipywidgets as widgets
from scipy import linalg
from scipy import optimize
import sympy as sm

# To print nicely:
sm.init_printing(use_unicode=True)

# autoreload modules when code is run
get_ipython().run_line_magic('load_ext', 'autoreload')
get_ipython().run_line_magic('autoreload', '2')
```

# The basic Solow model

The basic Solow model is as follows:

\begin{align}
Y_t = AK_t^{\alpha}L_t^{1-\alpha} \quad \quad (1) \\ \\
S_t = sY_t \quad \quad (2) \\ \\
K_{t+1} - K_t = S_t -\delta K_t \quad \quad (3) \\ \\
L_{t+1} = (1-n)L_t \quad \quad (4)
\end{align}

We also define the following equations:
\begin{align}
y_t = \frac{Y_t}{L_t} = Ak_t^{\alpha} \quad \quad (5) \\ \\
k_t = \frac{K_t}{L_t} \quad \quad (6)
\end{align}

## Defining symbols

First we define the necessary variables so we can use sympy:

```python
Y = sm.symbols("Y_t")
y = sm.symbols("y_t")
K = sm.symbols("K_t")
k = sm.symbols("k_t")
r = sm.symbols("r_t")
w = sm.symbols("w_t")
s = sm.symbols("s")
n = sm.symbols("n")
g = sm.symbols("g")
A = sm.symbols("A")
L = sm.symbols("L_t")
alpha = sm.symbols("alpha")
delta = sm.symbols("delta")
kstar = sm.symbols("k_t^*")
ystar = sm.symbols("y^*")
```

## Finding FOC:

```python
r_foc = sm.simplify(sm.diff(A*K**alpha*L**(1-alpha),K))
sm.Eq(r, r_foc)
```

```python
w_foc = sm.simplify(sm.diff(A*K**alpha*L**(1-alpha),L))
sm.Eq(w, w_foc)
```

## Solving for steady state

We insert (1) in (2) and inserting this into (3). Then we divide by $L_t$ and solve the equation knowing that in steady state $k_{t+1} = k_t = k^*$. The equation we are solving can then be simplified to: $sAk^{\alpha} - (n-\delta)k = 0$

```python
ksteady = sm.solve(s*A*k**alpha-(n-delta)*k, k)[0]
sm.Eq(kstar , ksteady)
```

Inserting this result into (5) gives us $y^*$:

```python
y_star = sm.simplify(A*ksteady**alpha)
sm.Eq(ystar, y_star)
```

## Simulating the basic Solow model

```python
def simulate_basic(alpha,delta,s,n,A,T):
    """
    Args:
    alpha (float): Capital income share
    delta (float): Capital depreciation rate
    s (float): Savings rate
    n (float): Labor force growth rate
    A (float): Total factor productivity
    T (int): Periods to simulate through

    Returns:
    A plot showing how capital per worker accumulates over time"""

    #First we make a list for our future values of capital accumulation to be in
    k_path = [0]

    #Then we make a list which will contain the values for the 45-degree-line
    Degreeline = [0]

    #We generate the 45-degree-line represeting capital per worker depreciation
    for t in range(1,T):
        line = (n + delta)*t
        Degreeline.append(line)

    #We generate the capital accumulation
    for t in range(1,T):
        k_plus = s*A*t**alpha
        k_path.append(k_plus)

    #Plotting the two lines
    plt.figure(figsize=(5,5))
    plt.plot(k_path[:T], label='$sAk_t^{\u03B1}$')
    plt.plot(Degreeline[:T], label = '$(n+\delta)k_t$')
    plt.xlim(0,T)
    plt.ylim(0,Degreeline[-1])
    plt.xlabel('$k_t$')
    plt.grid(True)
    plt.legend()

    return plt.show()
```

Example on a simulation with reasonable values:

```python
simulate_basic(1/3,0.05,0.3,0.01,1,50)
```

Below, you can try changing the parametres yourself by moving the sliders to see how different parameter values changes the steady state.

```python
widgets.interact(simulate_basic,
                alpha = widgets.FloatSlider(description = '$\u03B1$' , min = 0 , max = 0.9 , step = 0.05 , value = 0.3),
                delta = widgets.FloatSlider(description = '$\u03B4$' , min = 0 , max = 0.1 , step = 0.01 , value = 0.05),
                s = widgets.FloatSlider(description = 's' , min = 0 , max = 0.8 , step = 0.05 , value = 0.3),
                n = widgets.FloatSlider(description ='$n$', min = 0 , max = 0.1 , step = 0.005, value = 0.01),
                A = widgets.fixed(1),
                T = widgets.IntSlider(description='$T$' , min = 0, max = 200, step = 10, value = 50))
```

    interactive(children=(FloatSlider(value=0.3, description='$α$', max=0.9, step=0.05), FloatSlider(value=0.05, d…

    <function __main__.simulate_basic(alpha, delta, s, n, A, T)>

# The Solow-model with land

The Solow-model with land is as follows:
\begin{align}
Y_t = K_t^{\alpha}(A_tL_t)^{\beta}X^{\kappa}, \quad \alpha + \beta + \kappa = 1. \quad \quad (7) \\ \\
K_{t+1} = sY_t+(1-\delta)K_t, \quad K_0 \text{ given}. \quad \quad (8) \\ \\
L_{t+1} = (1+n)L_t, \quad n≥0, \quad L_0 \text{ given}. \quad \quad (9) \\ \\
A_{t+1} = (1+g)A_t, \quad g≥0, \quad A_0 \text{ given}. \quad \quad (10)
\end{align}
* $K_t$ is capital
* $L_t$ is labor (growing with a constant rate of $n$)
* $A_t$ is technology (growing with a constant rate of $g$)
* $X$ is land (constant)

* $s$ is the savings rate (which is assumed to be constant)

**Saving** is a constant fraction of GDP
$$ S_t = sY_t,\,s\in(0,1) $$
Defining the following variables:

$$ \text{Capital-output ratio:} \quad z_t=\frac{K_t}{Y_t} $$

$$ \text{Capital per worker:} \quad k_t=\frac{K_t}{L_t} $$

$$ \text{Land per worker:} \quad x_t=\frac{X_t}{L_t} $$

$$ \text{Output per worker:} \quad y_t=\frac{Y_t}{L_t} $$

$$ y_t = k_t^\alpha A_t^\beta x_t^\kappa \quad \quad (11)$$

## Defining symbols

First we define all necessary **symbols** so we can use sympy:

```python
z = sm.symbols('z_t')
v = sm.symbols('v_t')
beta = sm.symbols('beta')
kappa = sm.symbols('kappa')
zstar = sm.symbols('z^*')
x = sm.symbols('x_t')
X = sm.symbols('X')
A0 = sm.symbols('A_0')
L0 = sm.symbols('L_0')
```

**Defining our Cobb-Douglas production function as a sympy function:**

```python
Cobb_Douglas = K**alpha*(A*L)**beta*X*kappa
```

## Finding FOC:

```python
r_foc_land = sm.simplify(sm.diff(Cobb_Douglas,K))
sm.Eq(r, r_foc_land)
```

```python
w_foc_land = sm.simplify(sm.diff(Cobb_Douglas,L))
sm.Eq(w, w_foc_land)
```

```python
v_foc_land = sm.simplify(sm.diff(Cobb_Douglas,X))
sm.Eq(v, v_foc_land)
```

# Steady state

We are working with the transition equation for the capital-output ratio (z)

$$ z_{t+1} = \left(\frac{1}{(1+n)(1+g)}\right)^\beta [(s+(1-\delta)z_t)]^{1-\alpha} z_t^\alpha $$

Solving for z to find z* where we use that in steady state $z_{t+1} = z_t = z^*$

```python
# Solving the z equation using Sympy only works for certain values of alpha.
# We choose alpha to be 0.5. Setting alpha to 0.5 is perhaps too high,
# as imperical studies using a Solow model with land suggests that alpha is closer to 0.2,
# but we only made the calculations work with 0.5
alpha = 0.5

# Calculating the capital-ouput ratio (z) for the chosen value of alpha
z_eq = sm.Eq(zstar, (1/((1+n)*(1+g)))**beta * (s+(1-delta)*zstar)**(1-alpha)*zstar**alpha)
z_solve = sm.simplify(sm.solve(z_eq, zstar)[1])
zfinal = sm.Eq(zstar, z_solve)
zfinal
```

$z^*$ can also be reduced to the following for a given $\alpha$ value

$$ z^* = \frac{s}{[(1+n)(1+g)]^{\frac{\beta}{1-\alpha}}-(1-\delta)} $$

We create a function that gives us steady state value of $z^*$

```python
ss_z = sm.lambdify((beta, delta, n, g, s), z_solve)
def steady_state_z(beta, delta, n, g, s):

    """Returns steady state values for capital-output ratio (z)
    Args:
    alpha (float): Capital income share
    beta  (float): Labor income share
    kappa (float): Land income share
    delta (float): Capital depreciation rate
    n (float): Labor force growth rate
    g (float): Technological growth rate
    s (float): Savings rate
    A0 (float): Initital techonological level
    X (float): Land (square kilometers)
    L0 (float): Inital labor force

    Returns:
    Steady state values for z"""

    ssz = ss_z(beta, delta, n, g, s)

    return ssz
```

```python
ssz_value = steady_state_z(0.2,0.05,0.01,0.02,0.3)
print('Steady state for z is ' + str(round(ssz_value,2)))
```

    Steady state for z is 4.84

We are finding $y^*$ by using (11):

$$ y_t = k_t^\alpha A_t^\beta x_t^\kappa $$

We devide by $y_t^\alpha$ and find:

$$ y_t^{1-\alpha} = z_t^\alpha A_t^\beta x_t^\kappa $$

Isolating $y_t$ and inserting $z^*$ gives us $y_t^*$:

$$ y_t^* = (z^*)^{\frac{\alpha}{\beta+\kappa}} A_t^{\frac{\beta}{\beta+\kappa}} x_t^{\frac{\kappa}{\beta+\kappa}} $$

Deriving $y^*$ by using SymPy where we still use the alpha set to 0.5

```python
y_eq = sm.Eq(ystar**(1-alpha), zstar**alpha*A**beta*x**kappa)
y_solve = sm.simplify(sm.solve(y_eq, ystar))
y_solve
```

We create a function to find the steady state of $y_t^*$ where we allow the user to choose both X and $L_t$

```python
ss_y = sm.lambdify((A, beta, kappa, x, zstar), y_solve)
def steady_state_y(A, beta, kappa, X, L):

    """Returns steady state values for capital-output ratio (z) and output (y)
    Args:
    beta  (float): Labor income share
    kappa (float): Land income share
    X (float): Land (square kilometers)
    A (float): Techonological level
    L (float): Labor force

    Returns:
    Steady state values for z and y"""

    ssy = ss_y(A, beta, kappa, X/L, ssz_value)

    return ssy
```

```python
ssy_value = steady_state_y(1,0.6,0.2,43094,2917000)[0] # Here we use the Danish numbers of land and labor force
print('Steady state for y is ' + str(round(ssy_value,2)))
```

    Steady state for y is 0.9

# Numerical optimization

We are deriving the equation for the change in capital per effective worker, and using this to find the steady state level of capital.

We start by dividing (7) by $A_tL_t$ which gives us: $$\tilde{y}_t = \tilde{k}_t^\alpha \tilde{x}^\kappa$$

Then we divide (8) by $A_{t+1} L_{t+1}$ which gives us: $$\tilde{k}_{t+1} = \frac{1}{(1 + n)(1 + g)}(s\tilde{y}_t + (1 - \delta)\tilde{k}_t)$$

Inserting $\tilde{y}_t$ and ignoring $\tilde{x}^\kappa$ as it is just a scalar:
$$\tilde{k}_{t+1} = \frac{1}{(1 + n)(1 + g)}(s\tilde{k}_t^\alpha + (1 - \delta)\tilde{k}_t)$$

Finally we subtract $\tilde{k}_t$ to get an expression of the change in the capital stock:
$$\tilde{k}_{t+1} - \tilde{k}_t= \frac{1}{(1 + n)(1 + g)}(s\tilde{k}_t^\alpha - (n + g + \delta + ng)\tilde{k}_t) \quad \quad (12)$$

In steady state this change will be 0. So, we use (12) to find the value of $\tilde{k}_t$ where the change in capital is 0 as this will be the steady state level of capital.

**Defining parameter values used for optimization:**

```python
g = 0.02
alpha = 0.5
delta = 0.05
n = 0.01
s = 0.3
```

**Defining (12) which we will use to find the steady state level of capital:**

```python
f = lambda k: 1/((1 + n)*(1 + g)) * (s*k**alpha - (n + g + delta + n*g)*k)
```

**Plotting the change in k as a function of k:**

```python
# creating a vector with different values of k
k_vec = np.linspace(0,15,100)

# calculating the change in k for different values of k
k_change_vec = np.zeros(100)
for i,k in enumerate(k_vec):
    k_change_vec[i] = f(k)

# plotting the change in k as a function of k and marking the steady state level of capital
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(k_vec,k_change_vec)
ax.plot(14, 0, marker='o')
ax.grid()
ax.set_xlabel(r'$\tilde{k}_t$', fontsize = 15)
ax.set_ylabel(r'change in $\tilde{k}_t$', fontsize = 15)
ax.set_title(r'Change in $\tilde{k}_t$ as a function of $\tilde{k}_t$', fontsize = 15);
```

**Defining a function using the bisection method:**

**Algorithm:** `bisection()`

1. Set $a_0 = a$ and $b_0 = b$.
2. Compute $f(m_0)$ where $m_0 = (a_0 + b_0)/2$ is the midpoint.
3. Determine the next sub-interval $[a_1,b_1]$:
  * If $f(a_0)f(m_0) < 0$ (different signs) then $a_1 = a_0$ and $b_1 = m_0$ (i.e. focus on the range $[a_0,m_0]$).
  * If $f(m_0)f(b_0) < 0$ (different signs) then $a_1 = m_0$ and $b_1 = b_0$ (i.e. focus on the range $[m_0,b_0]$).
4. Repeat step 2 and step 3 until $f(m_n) < \epsilon$.

**The bisection function comes from the notebook from lecture 9:**

```python
def bisection(f,a,b,max_iter=50,tol=1e-6,full_info=True):
    """ bisection

    Solve equation f(x) = 0 for a <= x <= b.

    Args:

        f (function): function
        a (float): left bound
        b (float): right bound
        tol (float): tolerance on solution

    Returns:

        m (float): root

    """

    # test inputs
    if f(a)*f(b) >= 0:
        print("bisection method fails.")
        return None

    # step 1: initialize
    _a = a
    _b = b
    a = np.zeros(max_iter)
    b = np.zeros(max_iter)
    m = np.zeros(max_iter)
    fm = np.zeros(max_iter)
    a[0] = _a
    b[0] = _b

    # step 2-4: main
    i = 0
    while i < max_iter:

        # step 2: midpoint and associated value
        m[i] = (a[i]+b[i])/2
        fm[i] = f(m[i])

        # step 3: determine sub-interval
        if abs(fm[i]) < tol:
            break
        elif f(a[i])*fm[i] < 0:
            a[i+1] = a[i]
            b[i+1] = m[i]
        elif f(b[i])*fm[i] < 0:
            a[i+1] = m[i]
            b[i+1] = b[i]
        else:
            print("bisection method fails.")
            return None

        i += 1

    if full_info:
        return m,i,a,b,fm
    else:
        return m[i],i
```

**Calculating steady state using the bisection function:**

```python
m,i,a,b,fm = bisection(f,1,15)
ssk = m[i]
print(f'Iteration nr. {i}: steady state level of capital per effective worker is = {ssk:.2f}')
```

    Iteration nr. 16: steady state level of capital per effective worker is = 13.99


# Derivations for `constrsphtrig.py`: _Constructible spherical trigonometry in Python_

This document outlines the mathematical derivations underlying a number of algorithms in `constrsphtrig.py`.
We represent a point on the unit 2-sphere using spherical coordinates $p=(\phi_p,\theta_p)$, where $-\pi/2\leq\phi\leq\pi/2$ is its latitude and $-\pi<\theta\leq\pi$ its longitude.
$d_{ab}$ gives the distance between two points, and $\angle_{abc}$ the angle between three points.

For a spherical triangle with vertices $a,b,c$, the
[spherical cosine rule](https://en.wikipedia.org/wiki/Spherical_law_of_cosines)
gives

$$
    \cos d_{ab} = \cos d_{ac} \cos d_{bc} + \sin d_{ac} \sin d_{bc} \cos\angle _{acb} ~.
$$

Given any two points $a,b$ on the unit sphere, the
[geodesic distance](https://en.wikipedia.org/wiki/Great-circle_distance)
is

$$
    \cos d_{ab} = \sin\phi_a \sin\phi_b + \cos\phi_a \cos\phi_b \cos(\theta_a-\theta_b) ~,
$$

while the
[Cartesian coordinates](https://en.wikipedia.org/wiki/Spherical_coordinate_system#Cartesian_coordinates)
of any point $p$ on the sphere are

$$
    \mathbf{p} = (x_p, y_p, z_p)
    = (\cos\phi_p \cos\theta_p, \cos\phi_p \sin\theta_p, \sin\phi_p) ~.
$$

## `ConstrSphPoint.ang(self, lon_e=zero)`: _Returns the angle between `self` and the equator at a given longitude._

Given a point $p$ and a point on the equator $q$, compute $\angle_{pqq'} := \angle_{(\phi_q,\theta_q) (0,\theta_q) (0,\theta_q+\pi/2)}$.
Assume that $p\not\in\{q,(0,\theta_q+\pi)\}$.
Since $d_{q'q}=\pi/2$, the spherical cosine rule yields

$$
    \cos\angle_{pqq'}
    = \frac{\cos d_{pq'}}{\sin d_{pq}} ~.
$$

The geodesic distance formula then gives the necessary data to obtain the numerator and denominator, noting that $\phi_q=\phi_{q'}=0$:

\begin{align}
    \cos d_{pq'} &= \cos\phi_p \sin(\theta_p-\theta_q) ~, \\
    \cos d_{pq} &= \cos\phi_p \cos(\theta_p-\theta_q) ~.
\end{align}

## `ConstrSphPoint.linear(a, b, d_ac, d_bc)`: _Solves the linear problem in spherical geometry._

_Given two points of a spherical triangle and distances from these points to
        the third point, this function returns the list of possible third points $c$._

Here, we solve for $x_c, y_c$ linearly in terms of $z_c$ and then apply the nonlinear condition $\mathbf{c}\cdot\mathbf{c}=1$.
The geodesic distance formula expressed in Cartesian coordinates succinctly yields $\cos d_{ab} = \mathbf{a}\cdot\mathbf{b} = \mathbf{a}^T \mathbf{b}$, so that

$$
\begin{pmatrix}
    \cos d_{ac} \\ \cos d_{bc}
\end{pmatrix}
= \begin{pmatrix}
    \mathbf{a}^T \\ \mathbf{b}^T
\end{pmatrix} \mathbf{c}
= \begin{pmatrix}
    x_a & y_a \\
    x_b & y_b
\end{pmatrix}
\begin{pmatrix}
    x_c \\ y_c
\end{pmatrix}
+
\begin{pmatrix}
    z_a \\ z_b
\end{pmatrix}z_c
~.
$$

Thus

$$
\begin{pmatrix}
    x_c \\ y_c
\end{pmatrix}
=
\begin{pmatrix}
    x_a & y_a \\
    x_b & y_b
\end{pmatrix}^{-1}
\left(
\begin{pmatrix}
    \cos d_{ac} \\ \cos d_{bc}
\end{pmatrix}
- \begin{pmatrix}
    z_a \\ z_b
\end{pmatrix}z_c
\right)
$$

Denoting

$$
    M :=
\begin{pmatrix}
    x_a & y_a \\
    x_b & y_b
\end{pmatrix}, ~
\mathbf{d'} :=
\begin{pmatrix}
    \cos d_{ac} \\ \cos d_{bc}
\end{pmatrix}, ~
\mathbf{z'} :=
\begin{pmatrix}
    z_a \\ z_b
\end{pmatrix}
$$

allows one to express the above as

$$\begin{equation}
\begin{pmatrix}
    x_c \\ y_c
\end{pmatrix}
=
\frac{1}{\det M}
\begin{pmatrix}
    y_b & -y_a \\
    -x_b & x_a
\end{pmatrix}^{-1}
(\mathbf{d'} - \mathbf{z'}z_c) ~. \tag{1}
\end{equation}$$

Applying the unit spherical condition $\mathbf{c}\cdot\mathbf{c}=1$ gives

$$
    \frac{1}{\det^2 M} \left(
        (y_b,-y_a)\cdot(\mathbf{d'}-\mathbf{z'}z_c)~^2 +
        (-x_b,x_a)\cdot(\mathbf{d'}-\mathbf{z'}z_c)~^2
    \right)
    + z_c^2 = 1 ~,
$$

which is a quadratic equation in $z_c$.
Collecting its coefficients, this quadratic equation has the form $\alpha_2 z_c^2 - 2 \alpha_1 z_c + \alpha_0 = 0$, where

$$
\begin{align}
    \alpha_2 &:= \alpha_{yz}^2 + \alpha_{xz}^2 + \det\phantom{}^2 M \\
    \alpha_1 &:= \alpha_{yd}\alpha_{yz} + \alpha_{xd}\alpha_{xz} \\
    \alpha_0 &:= \alpha_{yd}^2 + \alpha_{xd}^2 - \det\phantom{}^2 M \\
    \alpha_{yu} &:= (y_b, -y_a) \cdot \mathbf{u'} \\
    \alpha_{xu} &:= (-x_b, x_a) \cdot \mathbf{u'}
\end{align}
$$

and $u\in\{d,z\}$.
The solution to this quadratic equation yields

$$
z_c = \frac{\alpha_1 \pm \sqrt{\alpha_1^2 - \alpha_2\alpha_0}}{\alpha_2} \tag{2} ~.
$$

Substituting $(2)$ into $(1)$ recovers the Cartesian coordinates of $c$.
Since the latitude always satisfies $\cos\phi\geq0$, it can be recovered directly from the last Cartesian component:
$$
    \phi_c = \verb|ConstrAngle|\left(\mathrm{sgn}~z_c, \sqrt{1-z_c^2}\right) ~.
$$

The longitude satisfies $\cos\theta=\frac{x}{\cos\phi}, \sin\theta=\frac{y}{\cos\phi}$ so that

$$
\theta_c = \verb|ConstrAngle|\left(\mathrm{sgn}~y_c, \frac{x_c}{\sqrt{1-z_c^2}}\right)
$$

### Edge cases

The above derivation is not applicable when $\det M=0$, which occurs when $\cos\phi_p=0$ for some $p\in\{a,b\}$ (one of the points is on a pole) or $\sin(\theta_b-\theta_a)=0$ (both points lie on the same or opposite line of longitude).
<!--This edge case is collectively defined by the property

$$
\cos\phi_a\cos\phi_b\sin(\theta_b-\theta_a)=0 ~. \tag{3}
$$
-->

Without loss of generality, assume that $\cos\phi_a=0$ or $\sin(\theta_b-\theta_a)=0$.
The geodesic distance formula gives

$$
\begin{pmatrix}
    \cos d_{ac} \\ \cos d_{bc}
\end{pmatrix}
= \underbrace{
\begin{pmatrix}
    \sin\phi_a & m\cos\phi_a \\
    \sin\phi_b & \cos\phi_b
\end{pmatrix}
}_{N}
\begin{pmatrix}
    \sin\phi_c \\ \cos\phi_c\cos(\theta_b-\theta_c)
\end{pmatrix}
$$

where $m=\cos(\theta_a-\theta_c)/\cos(\theta_b-\theta_c)$ is relevant only when $\sin(\theta_b-\theta_a)=0$, and attains the value $1$ if $\theta_a=\theta_b$, and $-1$ if $\theta_a=\pi+\theta_b$.
The matrix $N$ is guaranteed to have nonzero determinant under our assumptions and is thus invertible, so that

$$
\begin{pmatrix}
    \sin\phi_c \\ \cos\phi_c\cos(\theta_b-\theta_c)
\end{pmatrix}
=
\frac{1}{\det N}
\begin{pmatrix}
    \cos\phi_b & -m\cos\phi_a \\
    -\sin\phi_b & \sin\phi_a
\end{pmatrix}
\begin{pmatrix}
    \cos d_{ac} \\ \cos d_{bc}
\end{pmatrix} ~,
$$

from which one can read off $\sin\phi_c$ directly.
Since the cosine of a latitude is always non-negative,

$$
\phi_c = \verb|ConstrAngle|\left(\mathrm{sgn}\sin\phi_c, \sqrt{1-\sin^2\phi_c}\right) ~.
$$

The latter matrix equation also allows one to read off $\cos(\theta_b-\theta_c)$ directly, so that

$$
\theta_b-\theta_c = \begin{cases}
\verb|ConstrAngle|\left(\pm1, (0,1)\cdot N^{-1}\mathbf{d'}\right) \qquad& \text{if $\cos(\theta_b-\theta_c)\not\in\{-1,1\}$,} \\
\verb|ConstrAngle|\left(1, (0,1)\cdot N^{-1}\mathbf{d'}\right) & \text{otherwise.}
\end{cases}
$$

One can then unambiguously obtain the possible values of $\theta_c$ by subtracting the previous expression from $\theta_b$.

```python

```


### ContinuousWalkBot-v0

State

$$
\mathbf{x} = \begin{bmatrix}
x \\
y \\
v_x \\
v_y
\end{bmatrix}
$$

control

$$
\mathbf{u} = \begin{bmatrix}
a_x \\
a_y
\end{bmatrix}
$$

subject to constraints

$$
\begin{align}
 0 \leq x \leq 10 \\
 0 \leq y \leq 10 \\
 -10 \leq v_x \leq 10 \\
 -10 \leq v_y \leq 10 \\
 -2 \leq a_x \leq 2 \\
 -2 \leq a_y \leq 2
\end{align}
$$

and cost function

$$
Q = \begin{bmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & 0\\
0 & 0 & 0 & 1
\end{bmatrix},\,
R = \begin{bmatrix}
0 & 0 \\
0 & 0
\end{bmatrix}
$$

State matrix

$$
A = \begin{bmatrix}
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{bmatrix}
$$

Input matrix

$$
B = \begin{bmatrix}
0 & 0 \\
0 & 0 \\
1 & 0 \\
0 & 1 \\
\end{bmatrix}
$$

```python
import numpy as np
import sys
sys.path.append('../..')
from wizluk.envs.walker import WalkBot
```

    /usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters
    Using TensorFlow backend.

```python
model = WalkBot(0,'walker')
```

```python
model.add_bounds('x', 0.0, 10.0)
model.add_bounds('y', 0.0, 10.0)
model.add_bounds('vx', -10.0, 10.0)
model.add_bounds('vy', -10.0, 10.0)
model.add_bounds('ax', -2.0, 2.0)
model.add_bounds('ay', -2.0, 2.0)
```

```python
x = np.array([ [x] for x in [2.0, 8.0, -1.0140,0.2664]])
```

```python
print(x)
```

    [[ 2.    ]
     [ 8.    ]
     [-1.014 ]
     [ 0.2664]]

```python
u = np.zeros((2,1))
```

```python
print(u)
```

    [[0.]
     [0.]]

```python
x + 0.5*(model._state_matrix.dot(x)+model._input_matrix.dot(u))
```

    array([[ 1.493 ],
           [ 8.1332],
           [-1.014 ],
           [ 0.2664]])

```python
model.simulate_evolution(x,u, dt=0.5)
```

    array([[ 1.493 ],
           [ 8.1332],
           [-1.014 ],
           [ 0.2664]])

```python

```


# Gate F - Classification via Interference

$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[1]{\left\langle{#1}\right\rangle}$$

### (Step 3) Learning via interference
The prediction for the test set has to be computed based on the following transformation:

\begin{align}
    \left|\Phi_{f}\right\rangle
                = & \Big(\mathbb{1}^{\otimes 2} \otimes F \Big) \left|\Phi_{d}\right\rangle \nonumber \\
                = & (\mathbb{1}^{\otimes d} \otimes F )\Bigg[\frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\Bigg] \otimes \left|x^{(test)}\right\rangle \left|0\right\rangle  \nonumber \\
                = & \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\left|x^{(test)}\right\rangle \left|f_b\right\rangle
\end{align}

where $f_b$ represents the $b$-th prediction for $\tilde{x}$ given the  $b$-th training set.

## Cosine Classifier

Given a training point $(x_{b}, y_{b})$ at random, the probability of $y^{(\text{test})}$ and $y_{b}$ being equal is a function of the similarity between $x^{(\text{test})}$ and $x_{b}$.

Predict a similar target class for similar input features
\begin{align}\label{equation:cosine classifier}
    Pr\left(y^{(\text{test})} = y_{b}\right) = \frac{1}{2}+\frac{\left[d\left(x_{b}, x^{(\text{test})}\right)\right]^2}{2}
\end{align}

Final classification rule:
 \begin{align} \label{classification_rule}
    y^{(\text{test})} =
    \begin{cases} y_{b}, & \mbox{if } Pr\left(y^{(\text{test})} = y_{b}\right) > \frac{1}{2} \\ 1- y_{b}
                      & \mbox{otherwise }
    \end{cases}
\end{align}

## Cosine Classifier (II)

+ the prediction depends on the squared cosine distance between the two vectors

+ max dissimilarity occurs when training and test observations are orthogonal

        $\rightarrow$ uniform probability distribution in the two classes for $y^{(test)}$

+ the cosine classifier performs well only if training and test belongs to the same class

## Quantum Cosine Classifier

The quantum circuit that implements the cosine classifier encodes data into three different registers: the training vector $x_b$, the training label $y_b$ and the test point $x^{(test)}$. One more qubit is used to store the prediction.

The algorithm is made of three steps: **State Preparation**, **Swap test**, **Controlled-X**

```python
import sys
sys.path.insert(1, '../')
from Utils import *
from modeling import *

x_train = [random.uniform(-1, 1), random.uniform(0, 1)]
x_train_norm = normalize_custom(x_train)
x_test = [random.uniform(-1, 1), random.uniform(0, 1)]
x_test_norm = normalize_custom(x_test)
class0 = [1,0]

qc = quantum_cosine_classifier(x_train_norm, x_test_norm, class0)
qc.draw(output='mpl', scale=.7)
```

### Step 1: State Preparation

Data are encoded into three different quantum registers. Since each set of data ($x$ and $y$) is stored separately, the state preparation routine can be performed independently for each qubit.

$$ \ket{\Phi_1} = \Big( S_{x_{b}} \otimes S_{x^{(test)}} \otimes S_{y_{b}} \otimes  \mathbb{1}  \Big) \ket{0} \ket{0} \ket{0} \ket{0}  = \ket{x_{b}} \ket{x^{(test)}} \ket{y_{b}} \ket{0}
$$

where $S_{(x)}$ is the routine which encodes in the amplitudes of a qubit a $2$-dimensional real vector $x$.

### Step 2: Execution of the swap test

In the second step, the swap-test transforms the amplitudes of the qubit $y^{(test)}$ as a function of the squared cosine distance:

\begin{align}
\ket{\Phi_2} & =
\big(\mathbb{1} \otimes \mathbb{1} \otimes \mathbb{1} \otimes H\big)
\big(\text{cswap} \otimes \mathbb{1} \otimes C \big)
\big(\mathbb{1} \otimes \mathbb{1} \otimes \mathbb{1} \otimes H\big)
\ket{x_{b}} \ket{x^{(test)}} \ket{y_{b}} \ket{0} \nonumber \\
   & = \frac{1}{2}
   \Big(
   \big( \ket{x_{b}} \ket{x^{(test)}} + \ket{x^{(test)}} \ket{x_{b}} \big)\ket{y_{b}} \ket{0} +
  \big(\ket{x_{b}} \ket{x^{(test)}} - \ket{x^{(test)}} \ket{x_{b}} \big)\ket{y_{b}} \ket{1} \Big)
\end{align}

where $H$ is the Hadamard gate, cswap is the controlled-swap operation which uses the fourth qubit (position of gate *C*) as control  qubit to swap $\ket{x_{b}}$ and $\ket{x^{(test)}}$.
After the execution of the swap test the probability to redout the basis state $\ket{0}$, that is the probability for the *test* observation to be classified in class $0$ is:
\begin{align}
    P(y^{(test)} = \left|0\right\rangle) =
    \frac{1}{2}+\frac{|\langle x_{b}|x^{(test)} \rangle|^2}{2}
\end{align}

### Step 3: Controlled Pauli-*X* gate

The third step consists in applying a controlled Pauli-$X$ rotation using the control qubit that encodes the label of the training vector. This implies that $y^{(test)}$ is left untouched if  $x_{b}$ belongs to the class $0$. Otherwise,
the amplitudes of the $y^{(test)}$ qubit are exchanged,
and the probability $P(y^{(test)} = 1)$ is higher as the similarity between the two vectors increases.

$$
\left|\Phi_3\right\rangle = \big(\mathbb{1} \otimes \mathbb{1} \otimes \text{C-X} \big) \left|\Phi_2\right\rangle
$$

Thus, the probability amplitudes of the two basis states of $y^{(test)}$ lead to a classification rule that is equivalent to the classic cosine classifier.

## Behaviour of Quantum Cosine Classifier

To study the behaviour of the quantum cosine classifier, $1000$ random datasets are generated. Each dataset is made  of a training point $(x_{b}, y_{b})$ and a test point $x^{(test)}$

```python
n_shots = 100 # n° of shots for the quantum circuit
n=1000 # n° of datasets

x = [] # vector that stores classical cosine prediction
x_err = [] # cosine distance between vectors

P0 = [] # list of values if the training obs belgons to class 1
P1 = [] # list of values if the training obs belgons to class 1

for i in np.arange(n):
    '''Random generated dataset'''
    x_train = [random.uniform(-1, 1), random.uniform(0, 1)]
    x_train_norm = normalize_custom(x_train)
    x_test = [random.uniform(-1, 1), random.uniform(0, 1)]
    x_test_norm = normalize_custom(x_test)

    '''Compute cosine distance and append it to x'''
    d_cos = cosine_similarity([x_train], [x_test])[0][0]
    x.append(d_cos)

    '''If x_train belongs to class 0'''
    qc = quantum_cosine_classifier(x_train_norm, x_test_norm, [1,0])
    r = exec_simulator(qc, n_shots)
    P_q = r['0']/n_shots
    P0.append(P_q)
    d_cos_err = np.sqrt(2*P_q-1)
    x_err.append( d_cos_err )

P0 = np.array(P0)
x = np.array(x)
x_err = np.array(x_err)

order = x.argsort()

x = x[order[::-1]]
P0 = P0[order[::-1]]
x_err = x_err[order[::-1]]

err = [ abs(x1 - x2) for (x1, x2) in zip(abs(x), x_err)]
err = np.array(err)
P1 = 1-P0
```

```python
quantum_cos_random_data(x, P0, P1, err)
```

presentation/2. Gate F - Quantum Cosine Classifier.ipynb


# Summary - Logistic Regression Algorithm
> summary and useful points to know about LR
- toc: true
- badges: true
- comments: true
- image: https://www.jeremyjordan.me/content/images/2017/06/Screen-Shot-2017-06-10-at-9.41.25-AM.png
- author: Uday Paila
- categories: [Logistic Regression, classification]

## Terminology
$N$ - Number of data points
$X_q$ - query data points
$X_{qn}$ - nth query data point
$X$ - input train data
$D$ - dimensionality of data
$C$ - Number of classes
$C_i$ - i^th class
$N_k$ - K nearst neighbors
$m$ - Number of epochs in SGD

## Algorithm

We have to optimize the log loss value. We can do this using the Gradient Descent. optimization problem is

\begin{align}
\min_{W, b} \sum_{i=1}^N \log(e^{- y_i (X_i^T W + b)} + 1)\\
\text{ here }  C = 2 \text{ and } y_i = +1 \text{ or } -1
\end{align}

once after optimization, we will get the W, b where loss $L(W,b)$ is minimum. We can predict the class probability of a query point using W, b as below.

\begin{align}
P(y=1 | X_{qn}, W, b) = \frac{1}{1+e^{-(WX_{qn}+b)}}
\end{align}

We can re-write the above same loss formulation as below,

\begin{align}
\min_{W_{1},W_{2}...W_{C}, b}- \frac{1}{N} \sum_{i=0}^{N-1} \sum_{C=0}^{C-1} y_{i,c} \log p_{i,c}\\
y_{i,c} = 1 \text{if sample i has label C taken from a set of C labels else 0} \\
p_{i,c} = \frac{1}{1+e^{-(W_{C}X_{i}+b)}}
\end{align}

Once after calculating W vectors, we can get C probabilities using $p_{i, c}$ and then classify the given query point as maximum probability class.

We can write the final formulation with regularization as below

\begin{align}
\min_{W, b} C * \sum_{i=1}^N L(W,b) + \text{regularization_term}
\end{align}

$$\text{or}$$
\begin{align}
\min_{W, b}  \sum_{i=1}^N L(W,b) + \lambda * \text{regularization_term}
\end{align}

We can handle multi-class classification in 3 ways,
1. Using one-vs-rest approach and first formulation of loss
2. Using one-vs-one approach and first formulation of loss
3. Multinomial logistic regression - 2nd formulation of loss and softmax function in the place of sigmoid function to get C class probabilities.

You can read more about LR in [this](https://medium.com/@udaybhaskarpaila/math-behind-logistic-regression-3d7d53ec3091) blog and [this](https://medium.com/@udaybhaskarpaila/everything-you-need-to-know-about-logistic-regression-18e740be87a0) blog

## Useful points to know

- If we want to use logistic regression for multi-class classification, we have to use one-vs-rest/one-vs-one/multinomial.

- Time complexity of training is $O(mCND)$ and prediction is $O(CD)$. Prediction time is very less so we can use logistic regression for low latency applications.

- We can train the model using the iterative algorithm so there is no need to load total data into RAM (We can load chunk by chunk if we have huge data). We can use this model in online training too. Check the `partial_fit` function in `sklearn` implementation.

- It assumes
    - Observations should not come from repeated measurements.
    - None or Little Multicollinearity
    - The linearity of independent variables and Log odds - It requires features that are linearly related to the log-odds/logits, i.e $log(\frac{P}{1-P})$

- We can parallelize the multi-class logistic regression using one-vs-rest in sklearn. If you want for binary classification, use data parallelization, and accumulate gradients.

- It is a linear model, it cannot classify non-linear data. If you have non-linear data, do feature engineering and try to get linear data.

- Higher dimensional data (not Huge) may lead to linear separable sometimes.

- An increase in the feature value always leads to an increase or decrease in the target outcome(not probability, to logit/log-odds) so, it is a monotone model to log-odds/logits. Please check the interpretability below to know more about logit.

- It won't consider the interaction between the features. We have to create the interaction features if we need it. More interaction features may lead to less interpretability.

## Hyperparameters
- C is the main hyperparameter we have (this is different from C classes we are using)
- High C value means less regularization strength.
- C is a multiplicative constant to loss term. While optimization we have to find the minimum value of the loss(Log loss+regularization). If we increase the C value, Log loss has to decrease so that final loss will decrease, i.e log loss will get very near to zero if C increases, so it may be overly certain about training data, so it overfits.
- The below image contains decision hyperplanes for each value of C. You can observe that increasing the C value is reducing the regularization and overfitting to the data.

- In the above figure, we are getting similar classification results for all C greater than 0.5 but, increasing the C value reduces the regularization strength and increases the magnitude of weight vector values. You can check that in the below figure.

- If two hyperplanes are giving similar results, get the best hyperplane based on the magnitude of weights. low weight is better because it gives more regularization and it won't overestimate the probability value.

>Important: Let's take two planes. plane1 is 1.5f1 + 2.4f2 + 1 = 0, plane2 is 4.5f1 + 7.2f2 + 3 = 0. Both planes are mathematically same, only weight values are changed, but for a query point (0, 0), plane1 will give a probability of 0.73 and plane2 will give a probability of 0.95. But, the distance of the point (0,0) from both planes is the same that is 0.35 only. so large weights lead to overestimation of probabilities even though they are near to the plane.

## Interpretability
From logistic regression prediction probability function, we can write

\begin{align}
log\left(\frac{P(y=1)}{1-P(y=1)}\right)=log\left(\frac{P(y=1)}{P(y=0)}\right)=W_{0}+W_{1}x_{1}+\ldots+W_{d}x_{d}
\end{align}

\begin{align}
odds = \left(\frac{P(y=1)}{1-P(y=1)}\right)=\left(\frac{P(y=1)}{P(y=0)}\right)=e^{W_{0}+W_{1}x_{1}+\ldots+W_{d}x_{d}}
\end{align}

If odds = 3 then $P(y=1)$ is thrice as high as $P(y=0)$

If the feature $x_{j}$ is changed by n unit and the ratio of odds after the change and before the change is

\begin{align}
\frac{odds_{x_j+n}}{odds}=\frac{e^{\left(W_{0}+W{1}x_{1}+\ldots+W_{j}(x_{j}+n)+\ldots+W_{d}x_{d}\right)}}{e^{\left(W_{0}+W_{1}x_{1}+\ldots+W_{j}x_{j}+\ldots+W_{d}x_{d}\right)}}
\end{align}

\begin{align}
\frac{odds_{x_j+1}}{odds}=e^{\left(W_{j}(x_{j}+n)-W_{j}x_{j}\right)}=e^{\left(nW_j\right)}
\end{align}

If we increase the feature $x_{j}$ by n unit, the odds change by factor of $e^{nW_j}$

We can interpret the odds. If odds = k, $P(y=1) = k*P(y=0)$

## References
1. [AppliedAICourse.com](https://www.appliedaicourse.com/)
2. https://stackexchange.com/
3. An Introduction to Statistical Learning - Book
4. https://www.jeremyjordan.me/content/images/2017/06/Screen-Shot-2017-06-10-at-9.41.25-AM.png

## Getting In Touch
Please feel free to get in touch with us:
- [AppliedAICourse.com](https://www.appliedaicourse.com/)
- Author - [@Uday Paila](https://www.linkedin.com/in/uday-paila-1a496a84/)

```python

```


# US Production Data for RBC Modeling

```python
import pandas as pd
import numpy as np
import fredpy as fp
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
pd.plotting.register_matplotlib_converters()
```

```python
# Load API key
fp.api_key = fp.load_api_key('fred_api_key.txt')

# Download nominal GDP, nominal personal consumption expenditures, nominal
# gross private domestic investment, the GDP deflator, and an index of hours
# worked in the nonfarm business sector produced by the BLS. All data are
# from FRED and are quarterly.
gdp = fp.series('GDP')
cons = fp.series('PCEC')
invest = fp.series('GPDI')
hours = fp.series('HOANBS')
defl = fp.series('GDPDEF')

# Make sure that all of the downloaded series have the same data ranges
gdp,cons,invest,hours,defl = fp.window_equalize([gdp,cons,invest,hours,defl])

# Compute real GDP, real consumption, real investment
gdp.data = gdp.data/defl.data*100
cons.data = cons.data/defl.data*100
invest.data = invest.data/defl.data*100

# Print units
print('Hours units:   ',hours.units)
print('Deflator units:',defl.units)
```

    Hours units:    Index 2012=100
    Deflator units: Index 2012=100

Next, compute the quarterly capital stock series for the US using the perpetual inventory method. The discrete-time Solow growth model is given by:

\begin{align}
Y_t & = A_tK_t^{\alpha}L_t^{1-\alpha} \tag{1}\\
C_t & = (1-s)Y_t \tag{2}\\
Y_t & = C_t + I_t \tag{3}\\
K_{t+1} & = I_t + (1-\delta)K_t \tag{4}\\
A_{t+1} & = (1+g)A_t \tag{5}\\
L_{t+1} & = (1+n)L_t \tag{6}.
\end{align}

Here the model is assumed to be quarterly so $n$ is the *quarterly* growth rate of labor hours, $g$ is the *quarterly* growth rate of TFP, and $\delta$ is the *quarterly* rate of depreciation of the capital stock. Given a value of the quarterly depreciation rate $\delta$, an investment series $I_t$, and an initial capital stock $K_0$, the law of motion for the capital stock, Equation (4), can be used to compute an implied capital series. But we don't know $K_0$ or $\delta$ so we'll have to *calibrate* these values using statistics computed from the data that we've already obtained.

Let lowercase letters denote a variable that's been divided by $A_t^{1/(1-\alpha)}L_t$. E.g.,

\begin{align}
y_t = \frac{Y_t}{A_t^{1/(1-\alpha)}L_t}\tag{7}
\end{align}

Then (after substituting consumption from the model), the scaled version of the model can be written as:

\begin{align}
y_t & = k_t^{\alpha} \tag{8}\\
i_t & = sy_t \tag{9}\\
k_{t+1} & = i_t + (1-\delta-n-g')k_t,\tag{10}
\end{align}

where $g' = g/(1-\alpha)$ is the growth rate of $A_t^{1/(1-\alpha)}$. In the steady state:

\begin{align}
k & = \left(\frac{s}{\delta+n+g'}\right)^{\frac{1}{1-\alpha}} \tag{11}
\end{align}

which means that the ratio of capital to output is constant:

\begin{align}
\frac{k}{y} & = \frac{s}{\delta+n+g'} \tag{12}
\end{align}

and therefore the steady state ratio of depreciation to output is:

\begin{align}
\overline{\delta K/ Y} & = \frac{\delta s}{\delta + n + g'} \tag{13}
\end{align}

where $\overline{\delta K/ Y}$ is the long-run average ratio of depreciation to output. We can use Equation (13) to calibrate $\delta$ given $\overline{\delta K/ Y}$, $s$, $n$, and $g'$.

Furthermore, in the steady state, the growth rate of output is constant:

\begin{align}
\frac{\Delta Y}{Y} & = n + g' \tag{14}
\end{align}

1. Assume $\alpha = 0.35$.
2. Calibrate $s$ as the average of ratio of investment to GDP.
3. Calibrate $n$ as the average quarterly growth rate of labor hours.
4. Calibrate $g'$ as the average quarterly growth rate of real GDP minus n.
5. Calculate the average ratio of depreciation to GDP $\overline{\delta K/ Y}$ and use the result to calibrate $\delta$. That is, find the average ratio of Current-Cost Depreciation of Fixed Assets (FRED series ID: M1TTOTL1ES000) to GDP (FRED series ID: GDPA). Then calibrate $\delta$ from the following steady state relationship:
\begin{align}
\delta & = \frac{\left( \overline{\delta K/ Y} \right)\left(n + g' \right)}{s - \left( \overline{\delta K/ Y} \right)} \tag{15}
\end{align}
6. Calibrate $K_0$ by asusming that the capital stock is initially equal to its steady state value:
\begin{align}
K_0 & = \left(\frac{s}{\delta + n + g'}\right) Y_0 \tag{16}
\end{align}

Then, armed with calibrated values for $K_0$ and $\delta$, compute $K_1, K_2, \ldots$ recursively. See Timothy Kehoe's notes for more information on the perpetual inventory method:

http://users.econ.umn.edu/~tkehoe/classes/GrowthAccountingNotes.pdf

```python
# Set the capital share of income
alpha = 0.35

# Average saving rate
s = np.mean(invest.data/gdp.data)

# Average quarterly labor hours growth rate
n = (hours.data[-1]/hours.data[0])**(1/(len(hours.data)-1)) - 1

# Average quarterly real GDP growth rate
g = ((gdp.data[-1]/gdp.data[0])**(1/(len(gdp.data)-1)) - 1) - n

# Compute annual depreciation rate
depA = fp.series('M1TTOTL1ES000')
gdpA = fp.series('gdpa')

gdpA = gdpA.window([gdp.data.index[0],gdp.data.index[-1]])
gdpA,depA = fp.window_equalize([gdpA,depA])

deltaKY = np.mean(depA.data/gdpA.data)
delta = (n+g)*deltaKY/(s-deltaKY)

# print calibrated values:
print('Avg saving rate:        ',round(s,5))
print('Avg annual labor growth:',round(4*n,5))
print('Avg annual gdp growth:  ',round(4*g,5))
print('Avg annual dep rate:    ',round(4*delta,5))

# Construct the capital series. Note that the GPD and investment data are reported on an annualized basis
# so divide by 4 to get quarterly data.
capital = np.zeros(len(gdp.data))
capital[0] = gdp.data[0]/4*s/(n+g+delta)

for t in range(len(gdp.data)-1):
    capital[t+1] = invest.data[t]/4 + (1-delta)*capital[t]

# Save in a fredpy series
capital = fp.to_fred_series(data = capital,dates =gdp.data.index,units = gdp.units,title='Capital stock of the US',frequency='Quarterly')

# plot the computed capital series
plt.plot(capital.data.index,capital.data,'-',lw=3,alpha = 0.7)
plt.ylabel(capital.units)
plt.title(capital.title)
plt.grid()
```

```python
# Compute TFP
tfp = gdp.data/capital.data**alpha/hours.data**(1-alpha)
tfp = fp.to_fred_series(data = tfp,dates =gdp.data.index,units = gdp.units,title='TFP of the US',frequency='Quarterly')

# Plot the computed capital series
plt.plot(tfp.data.index,tfp.data,'-',lw=3,alpha = 0.7)
plt.ylabel(tfp.units)
plt.title(tfp.title)
plt.grid()
```

```python
# Convert each series into per capita using civilian pop 16 and over
gdp = gdp.per_capita(civ_pop=True)
cons = cons.per_capita(civ_pop=True)
invest = invest.per_capita(civ_pop=True)
hours = hours.per_capita(civ_pop=True)
capital = capital.per_capita(civ_pop=True)

# Put GDP, consumption, and investment in units of thousands of dollars per person
gdp.data = gdp.data*1000
cons.data = cons.data*1000
invest.data = invest.data*1000
capital.data = capital.data*1000

# Scale hours per person to equal 100 on October (Quarter III) of 2012
hours.data = hours.data/hours.data.loc['2012-10-01']*100

# Make sure TFP series has same length as the rest (since the .per_capita() function may affect the date range.
tfp,gdp = fp.window_equalize([tfp,gdp])

# Compute and plot log real GDP, log consumption, log investment, log hours
gdp_log = gdp.log()
cons_log = cons.log()
invest_log = invest.log()
hours_log = hours.log()
capital_log = capital.log()
tfp_log = tfp.log()

# HP filter to isolate trend and cyclical components
gdp_log_cycle,gdp_log_trend = gdp_log.hp_filter()
cons_log_cycle,cons_log_trend = cons_log.hp_filter()
invest_log_cycle,invest_log_trend = invest_log.hp_filter()
hours_log_cycle,hours_log_trend = hours_log.hp_filter()
capital_log_cycle,capital_log_trend = capital_log.hp_filter()
tfp_log_cycle,tfp_log_trend = tfp_log.hp_filter()
```

```python
# Create a DataFrame with actual and trend data
data = pd.DataFrame({
        'gdp':gdp.data,
        'gdp_trend':np.exp(gdp_log_trend.data),
        'gdp_cycle':gdp_log_cycle.data,
        'consumption':cons.data,
        'consumption_trend':np.exp(cons_log_trend.data),
        'consumption_cycle':cons_log_cycle.data,
        'investment':invest.data,
        'investment_trend':np.exp(invest_log_trend.data),
        'investment_cycle':invest_log_cycle.data,
        'hours':hours.data,
        'hours_trend':np.exp(hours_log_trend.data),
        'hours_cycle':hours_log_cycle.data,
        'capital':capital.data,
        'capital_trend':np.exp(capital_log_trend.data),
        'capital_cycle':capital_log_cycle.data,
        'tfp':tfp.data,
        'tfp_trend':np.exp(tfp_log_trend.data),
        'tfp_cycle':tfp_log_cycle.data,
    },index = gdp.data.index)

columns_ordered =[]
names = ['gdp','consumption','investment','hours','capital','tfp']
for name in names:
    columns_ordered.append(name)
    columns_ordered.append(name+'_trend')

data[columns_ordered].to_csv('../Csv/rbc_data_actual_trend.csv')

# Create a DataFrame with actual, trend, and cycle data
columns_ordered =[]
names = ['gdp','consumption','investment','hours','capital','tfp']
for name in names:
    columns_ordered.append(name)
    columns_ordered.append(name+'_trend')
    columns_ordered.append(name+'_cycle')

data[columns_ordered].to_csv('../Csv/rbc_data_actual_trend_cycle.csv')
```


# Introduction

The temperature measurement is done with the following circuit:

see also [Circuit Simulation](http://falstad.com/circuit/circuitjs.html?ctz=CQAgjCAMB0l3BWcMBMcUHYMGZIA4UA2ATmIxEIiQBZsAoANxGMJG0L2dZal+shAJY2XgJgI6AJy4yUCbqwFhIhFFJByFG+Ww68U1fus0z+AnkrzxI6s6bgaD+tPFsOUTu+05LrNgO4y3tqswQEhupxeejYADiB2HtSCYCiOyRBidPF2PNQ8PJlQ2REFoXpFNkA)

The value of the thermistor is defined as $ R_T=R_N \cdot e^{B \cdot \left(\frac{1}{T} - \frac{1}{T_N}\right)} $ with $ T_N = 298.15 $.

This results in: $ T = \frac{B \cdot T_N}{B + \ln\left( \frac{R_T}{R_N} \right) \cdot T_N} $

*Please note: Temperature values must be given in Kelvin.*

```python
from IPython.display import Markdown as md
import numpy as np
from matplotlib import pyplot as plt
from scipy.optimize import curve_fit

plt.rcParams['figure.figsize'] = [16,8]
```

```python
pt1000 = np.array([[-20, -15, -10, -5, 0,5,10,15,20,25,30],[8194, 6256, 4825, 3758, 2954, 2342,1872,1508,1224,1000,823]])
R_N = 1000
T_N = 298.15
```

```python
def func(T, B):
    return R_N * np.exp(B*(1/T - 1/T_N))

popt, pcov = curve_fit(func, pt1000[0]+273.15, pt1000[1])
B=popt[0]
x_data = np.arange(-30, 41, 1)
y_data = func(x_data+273.15, *popt)

plt.plot(x_data, y_data, 'b-', label='model (exponential fit)')

stdevs = np.sqrt(np.diag(pcov))

plt.plot(pt1000[0], pt1000[1], 'ro', label='sample data')
plt.legend()
plt.grid(color='grey', linestyle=':', linewidth=1)
plt.title('Thermistor values');
```

```python
md("Based on model data we get $B={:.3f} \pm {:.2f}$.".format(B, stdevs[0]))
```

Based on model data we get $B=3528.009 \pm 0.04$.

## Calculating the temperature

To actually calculate the current temperature from a given ADC-value we need to

1. get a general idea what the formula looks like
2. sample data

```python
raw = np.array([787, 780, 773, 754, 749, 729, 717, 722, 725, 731.3, 746.6, 754.9, 763.7, 748.7, 742.5, 715.6, 699.5, 694, 677, 688, 672, 659, 644.7, 666.5, 655.7, 609.5, 626, 745, 809, 810.3, 761.4, 765.5, 777.1, 786.45, 793, 794.4, 796.6, 799.4, 822.1, 823.8, 824.2, 801.1, 716.7, 710.7, 706.7, 691.5, 588, 599.3])
temp = np.array([28.1, 27, 26.7, 24.4, 24, 21.5, 20.3, 20.7, 21, 21.6, 23.5, 24.4, 25.5, 23.7, 23, 20.1, 18.6, 18.3, 16.5, 17.5, 15.9, 14.6, 13.3, 15.5, 14.3, 10.1, 11.5, 23.1, 31.3, 31.5, 25.2, 25.8, 27.3, 28.5, 29, 29.4, 29.8, 30.1, 33.2, 33.5, 33.5, 30.2, 20.3, 19.5, 19.1, 17.4, 7.7, 8.8])+273.15
```

### Linear approach

$ T = m \cdot \mathit{raw} + b $

```python
# linear approach
def func(x, m, b):
    return m * x + b

popt, pcov = curve_fit(func, raw, temp)
a=popt[0]
b=popt[1]

x_data = np.arange(500, 901, 10)
y_data = func(x_data, *popt)

plt.rcParams['figure.figsize'] = [16,8]

plt.plot(x_data, y_data-273.15, 'b-', label='linear-fit')

stdevs = np.sqrt(np.diag(pcov))

upper = func(x_data, popt[0]+stdevs[0], popt[1]+stdevs[1])-273.15
lower = func(x_data, popt[0]-stdevs[0], popt[1]-stdevs[1])-273.15

plt.fill_between(x_data, upper, lower, color="grey", alpha=0.15, label='model std-bounds')

plt.plot(raw, temp-273.15, 'ro', label='data')
plt.rcParams['figure.figsize'] = [16,8]
plt.grid(color='grey', linestyle=':', linewidth=1)

plt.legend();
plt.axis([500, 900, 0, 40]);
plt.title('Raw to Temperature (linear)');
plt.xlabel('ADC raw value');
plt.ylabel('Temperature (°C)');
```

```python
md("Based on linear model we get $m={:.3f} \pm {:.3f}$ and $b={:.3f} \pm {:.3f}$.".format(a, stdevs[0], b, stdevs[1]))
```

Based on linear model we get $m=0.109 \pm 0.001$ and $b=215.785 \pm 0.991$.

```python
a, b
```

    (0.10890629242812568, 215.78480075579688)

```python
def func(x, m, b):
    return m * x + b

x_data = raw
residuals = temp-func(x_data, 0.103, 219.74);

ss_res = np.sum(residuals**2)
ss_tot = np.sum((temp-np.mean(temp))**2)
r_squared = 1 - (ss_res / ss_tot)

plt.rcParams['figure.figsize'] = [8,4]
plt.axhline( color='black');
plt.plot(x_data, residuals, 'bo', label='residuals');
plt.grid(color='grey', linestyle=':', linewidth=1)

plt.title('Residuals (R²={:.3})'.format(r_squared));
plt.xlabel('ADC raw value');
plt.ylabel('Temperature (°C)');
```

```python
np.sum(residuals**3)
```

    37.530818144541016

### Logarithmic approach

Based on

$$ T = \frac{B \cdot T_N}{B + \ln\left( \frac{R_T}{R_N} \right) \cdot T_N} $$

$$ U_2 = V_{CC} \cdot R_2 \cdot \frac{R_4}{(R_1+R_2+R_3) \cdot R_4 + R_T \cdot \left(R_1+ R_2+ R_3 + R_4 \right)} $$

transforms to
$$
\begin{align}
 \frac{U_2}{V_{CC} \cdot R_2 \cdot R_4} &= \frac{1}{(R_1+R_2+R_3) \cdot R_4 + R_T \cdot (R_1+R_2+R_3+R_4)} \\
 \frac{V_{CC} \cdot R_2 \cdot R_4}{U_2} &= (R_1+R_2+R_3) \cdot R_4 + R_T \cdot (R_1+R_2+R_3+R_4) \\
 \frac{V_{CC} \cdot R_2 \cdot R_4}{U_2} - (R_1+R_2+R_3) \cdot R_4 &=  R_T \cdot (R_1+R_2+R_3+R_4) \\
 R_T &= \frac{V_{CC} \cdot R_2 \cdot R_4}{U_2 \cdot (R_1+R_2+R_3+R_4)} - \frac{R_4 \cdot (R_1+R_2+R_3)}{(R_1+R_2+R_3+R_4)} \\
 R_T &= \frac{R_4}{(R_1+R_2+R_3+R_4)} \cdot \left(\frac{V_{CC} \cdot R_2}{U_2} - (R_1+R_2+R_3) \right)
\end{align}
 $$

Model:
$$
\begin{align}
R_{T_{Model}}(U_2, a, b) &= \frac{a}{U_2} + b \\
T_{Model}(U_2, a, b) &= \frac{B \cdot T_N}{B + \ln\left( \frac{\frac{a}{U_2}+b}{R_N} \right) \cdot T_N}
\end{align}
$$

```python
# logarithmic approach
def func(x, a, b):
    RT = a/x+b
    return B * T_N / (B + np.log(RT/R_N)*T_N)

popt, pcov = curve_fit(func, raw, temp, absolute_sigma=True)
a=popt[0]
b=popt[1]

x_data = np.arange(500, 901, 10)
y_data = func(x_data, *popt);

plt.rcParams['figure.figsize'] = [16,8]
plt.plot(x_data, y_data-273.15, 'b-', label='model ($R_T-based$ logarithm)');

stdevs = np.sqrt(np.diag(pcov))

upper = func(x_data, popt[0]+stdevs[0], popt[1]+stdevs[1])-273.15
lower = func(x_data, popt[0]-stdevs[0], popt[1]-stdevs[1])-273.15

plt.fill_between(x_data, upper, lower, color="grey", alpha=0.15, label='model std-bounds')

plt.plot(raw, temp-273.15, 'ro', label='sample data');
plt.rcParams['figure.figsize'] = [16,8]
plt.grid(color='grey', linestyle=':', linewidth=1)

plt.legend();
plt.axis([500, 900, 0, 40]);
plt.title('Raw to Temperature');
plt.xlabel('ADC raw value');
plt.ylabel('Temperature (°C)');
```

```python
md("Based on logarithmic model, we get $a={:.3f} \pm {:.3f}$% and $b={:.3f} \pm {:.3f}$%.".format(popt[0], abs(stdevs[0]/popt[0]*100), popt[1], abs(stdevs[1]/popt[1]*100)))
```

Based on logarithmic model, we get $a=2687025.966 \pm 2.484$% and $b=-2538.487 \pm 3.458$%.

```python
def func(x, a, b):
    RT = a/x+b
    return B * T_N / (B + np.log(RT/R_N)*T_N)

x_data = raw
residuals = temp-func(x_data, 2622921.367, -2449.817);

plt.rcParams['figure.figsize'] = [8,4]
plt.axhline( color='black');
plt.plot(x_data, residuals, 'bo', label='residuals');
plt.grid(color='grey', linestyle=':', linewidth=1)

plt.title('Residuals');
plt.xlabel('ADC raw value');
plt.ylabel('Temperature (°C)');
```

```python
np.sum(residuals**3)
```

    0.31313480676243405

```python

```


# Importo los paquetes

```python
from sympy import Symbol,integrate,sin
```

# Ejercicio propuesto para resolver

$$c_n = \frac{2}{L}\left[\underbrace{\int_0^{\frac{L}{2}}\frac{2c}{L}x\sin{\left(\frac{n\pi}{L}x\right)dx}}_{f_1}+\underbrace{\int_{\frac{L}{2}}^{L} \frac{2c}{L}(L-x)\sin{\left(\frac{n\pi}{L}x\right)}dx}_{f_2} \right] $$

# Defino los simbolos necesarios

```python
x=Symbol("x")
L=Symbol("L",positive=True)
c=Symbol("C")
n=Symbol("n",integer=True,positive=True)
pi=Symbol("\u03c0",positive=True)

# Funciones para integrar

f_1=((2*c*x)/L)*sin((n*pi*x)/L)
f_2=((2*c)/L)*(L-x)*sin((n*pi*x)/L)
```

# Resuelvo las integrales

```python
integrate(f_1,(x,0,L/2))
```

```python
integrate(f_2,(x,L/2,L))
```