# Calculus review

```python
import datetime
import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm
import pandas as pd
import matplotlib.pyplot as plt
from sympy import symbols, diff, solve, integrate, dsolve
from sympy.utilities.lambdify import lambdify
from sympy.solvers.solveset import linsolve
import warnings
warnings.filterwarnings("ignore")
```

The following notebook is a review of differential and difference equations with examples in python.

The equations and examples are from the following book I highly recommend using to brush up on mathamtics commonly used in economics coursework:
- Dowling, E. T. (2012). Introduction to mathematical economics. McGraw-Hill.
    - [Amazon link](https://www.amazon.com/Schaums-Introduction-Mathematical-Economics-Outlines/dp/0071762515/ref=sr_1_7?dchild=1&keywords=mathematics+economics&qid=1593200726&sr=8-7)

# Table of contents
- [1. First-order equations](#First-order-equations)
    - [1.1 Differential equations](#1.1-Differential-equations)
    - [1.2 Difference equations](#1.2-Difference-equations)

- [2. Second-order equations](#2.-Second-order-equations)
     - [2.1 Differential equations](#2.1-Differential-equations)
     - [2.2 Difference equations](#2.2-Difference-equations)

- [3. Simultaneous equations](#3.-Simultaneous-equations)
    - [3.1 Differential equations](#3.1-Differential-equations)
    - [3.2 Difference equations](#3.2-Difference-equations)

- [4. Calculus of variations](#4.-Calculus-of-variations)

- [5. Optimal control theory](#5.-Optimal-control-theory)

# Intro
## I.1 Differential equations

- `Differential equation:` is an equation that relates one or more functions and their derivatives.

Definition from: [wikipedia](https://en.wikipedia.org/wiki/Differential_equation#:~:text=In%20mathematics%2C%20a%20differential%20equation,a%20relationship%20between%20the%20two.)

## I.2 Difference equations
- `Difference equation:` an equation that recursively defines a sequence or multidimensional array of values if one or more initial terms are given
    - each further term of the sequence or array is defined as a function of the preceding terms.
-   A difference equation is a specific type of recurrence relation

Definition from: [wikipedia](https://en.wikipedia.org/wiki/Recurrence_relation#Relationship_to_difference_equations_narrowly_defined)

## I.3 Linear difference equation
- `Linear difference equation:`
-  A polynomial that is linear
    - Means that each of its terms has degree 0 or 1.
- Usually the context is the evolution of some variable over time, with the current time period or discrete moment in time denoted as t, one period earlier denoted as t − 1, one period later as t + 1

- Used in modeling such time series
- In econometric applications, linear difference equations are modeled with stochastic terms
    - Examples include: AR, VAR, ARMA

Definition from: [wikipedia](hhttps://en.wikipedia.org/wiki/Linear_difference_equation)

# 1. First-order equations

## 1.1 Differential equations

```python
y_dt, y = symbols('y_dt y', integer=True)
def differential(f):
    #Steady state solutions
    ss = solve(f, y)

    #Critical point
    critical = solve(diff(f,y))

    #Check for concavity
    if diff(f,y,y)<0:
        print("Concave with max")
    else:
        print("Not concave")

    #Plot
    x = np.linspace(-1,5,101)
    f_lamb = lambdify(y, f)
    yf = f_lamb(x)
    plt.plot(x,yf)
    plt.axvline(critical, color='red', label='Critical point')
    plt.axvline(ss[0], color='green', label='Steady state')
    plt.axvline(ss[1], color='green', label='Steady state')
    plt.legend(loc='upper center', bbox_to_anchor=(0.5, -0.05),fancybox=True, shadow=True, ncol=3)

    print('If dy*/dy>0 arrow points right ')
    arrow_args = dict(fc="k", ec="k", head_width=.6, lw=1,
            alpha=0.6, head_length=.4)

    plt.arrow(ss[0],0, -.5, 0, **arrow_args) # x, y, dx, dy
    plt.arrow(ss[0],0, .5, 0, **arrow_args) # x, y, dx, dy

    plt.arrow(ss[1]+1, 0, -.5, 0, **arrow_args) # x, y, dx, dy
    plt.arrow(ss[1]-1, 0, .5, 0, **arrow_args) # x, y, dx, dy
```

### 1.1.1 Python example: Differential equations and phase diagrams
$$Y=8y-2y^2$$

```python
f = 8*y - 2*y**2
differential(f)
```

## 1.2 Difference equations

#### Basic definition
- Relationship between dependent variable and a lagged variable changing at discrete intervals of time

### 1.2.1 General formula

$$y_t = b y_{t+1}+a \\
\begin{cases}
y_t = (y_0-\frac{a}{1-b})b^t+ \frac{a}{1-b} & \text{when} \neq 1\\
y_t = y_0+at & \text{when} = 1
\end{cases}
$$

When:
- b>0: non-osscilating
- b<0: osccilating
- |b|>1: explodes
- |b|<0: converges

#### 1.2.1.1 Python example: FOD-General formula

```python
def first_difference(y0, b, a, length):
    yt = []
    for t in range(0, length+1):
        if b!=1:
            yt.append((y0-((a)/(1-b)))*(b)**t + (a/(1-b)))
        else:
            yt.append(y0+a*t)
    return yt
y0, a, length = [5,16,5]
b = [2, 1, .2, -.2, -1, -1.5]
for bs in b:
    yt = first_difference(y0, bs, a, length)
    t = np.arange(0,length+1, 1)
    plt.plot(t, yt, label = bs)
    plt.ylim([-5,20])
plt.axhline(y0,color='pink',label ='y0')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.ylabel(r'$y_t$')
plt.xlabel("t")
plt.title('First order difference equations');
```

### 1.2.2 Economic example: First-order-difference
- Reference: User create code derived from [Quantecon](https://python-programming.quantecon.org/python_oop.html)

 $$k_{t+1} = \frac{[(s z k^α_t) + (1 - δ)k_t]}{(1 + n)}$$
 - s: Savings
 - n: Depreciation rate
 - $\alpha_t$: Labor share in period t
 - z: Productivity
 - $\delta$: Depreciation rate
 - $k_t$: Capital stock in period t

#### 1.2.2.1 Solow steady state

```python
def solow(n, s, δ, α, z, kt_0, ts_len):
    k = np.zeros([ts_len])
    k[0] = kt_0
    for t in range(ts_len-1):
        k[t+1] = (s * z * k[t]**α + (1 - δ) * k[t]) / (1 + n)
    return k
```

```python
def solow_ss(n, s, δ, α, z):
    return ((s * z) / (n + δ))**(1 / (1 - α))
```

```python
n, s, δ, α, z, kt_0, ts = [.05, 0.25, .1, .3, 2, 1, 60]
for k in [1,8]:
    capital_path = solow(n, s, δ, α, z, k, ts)
    time_path = np.linspace(0,ts,ts)
    plt.plot(time_path, capital_path, label=r'$K_0={}$'.format(k))
plt.plot(time_path, np.repeat(solow_ss(n,s,δ, α, z), len(time_path)), label='Steady state')
plt.legend();
```

# 2. Second-order equations

## 2.1 Differential equations
$$y\prime\prime(t) + b1 y\prime(t)+b2 y(t)=a$$

### 2.1.1 Sympy example

```python
import sympy as sp

t = sp.symbols('t')
y = sp.Function('y')
diffeq = sp.Eq(y(t).diff(t, t) + y(t).diff(t, t,t)+ y(t))
diffeq
```

$\displaystyle y{\left(t \right)} + \frac{d^{2}}{d t^{2}} y{\left(t \right)} + \frac{d^{3}}{d t^{3}} y{\left(t \right)} = 0$

### 2.1.2 Python example: second-order differential equation

Example:
$$y\prime\prime(t) - 5 y\prime(t)+4 y(t)=2$$

```python
k1, k2, e,t = symbols('k1 k2 e t', integer=True)

def second_differential(b1, b2, a, y, dy):
    #Particular equation:
    if b2!=0:
        particular_integral = a/b2
    if b2 == 0 or b1 !=0:
        particular_integral = (a/b1)*t
    if b1==0 or b2==0:
        particular_integral = (a/2)*t**2

    #Eigenvalues
    r1 = (-b1+(np.sqrt(b1**2-4*b2)))/2
    r2 = (-b1- (np.sqrt(b1**2-4*b2)))/2

    #Complementary solution
    comp_solu = k1*e**(r2*t)    +   k2*e**(r1*t) + particular_integral
    comp_solu_deriv = diff(comp_solu, t)

    #UNFINISHED
    #Definite solution
    def_solu1 = comp_solu-y
    def_solu2 = comp_solu_deriv-dy
    #def_solu = linsolve([def_solu1, def_solu2],(k1,k2))

    return comp_solu
```

```python
second_differential(-5, 4, 2, 5.5, 11)
```

$\displaystyle e^{1.0 t} k_{1} + e^{4.0 t} k_{2} - 0.4 t$

## 2.2 Difference equations
#### Basic definition
- Specifies the state $x_t$ as a function of the state at two previous times

$$y_t+b1y_{t-1}+ b2y_{t-2}=a$$

### 2.2.1 Particular solution

```python
k1, k2, e, t, y = symbols('k1 k2 e t y', integer=True)
def second_difference(b1, b2,a):
    #Particular equation:
    if b2+b1!=-1:
        particular_solu = (a)/(1+b1+b2)
    if b2+b1 == -1 and b1 !=-2:
        particular_solu = ((a)/(2+b1))*t
    if b2+b1==-1 and b1==-2:
        particular_solu = (a/2)*t**2

    return particular_solu
```

#### 2.2.1.1 Ex1: Particular solution

$$y_t-10y_{t-1}+ 16y_{t-2}=14$$

```python
b1,b2,a = [-10,16, 14]
second_difference(b1,b2,a)
```

    2.0

#### 2.2.1.2 Ex2: Particular solution

$$y_t-6y_{t-1}+ 5y_{t-2}=12$$

```python
second_difference(-6, 5, 12)
```

$\displaystyle - 3.0 t$

$$y_t-2y_{t-1}+ y_{t-2}=8$$

```python
second_difference(-2, 1, 8)
```

$\displaystyle 4.0 t^{2}$

### 2.2.2 Complementary function

```python
A1, A2, e, t, y = symbols('A1 A2 e t y', integer=True)
def second_difference(b1, b2,a):
    #Particular equation:
    if b2+b1!=-1:
        particular_solu = (a)/(1+b1+b2)
    if b2+b1 == -1 and b1 !=-2:
        particular_solu = ((a)/(2+b1))*t
    if b2+b1==-1 and b1==-2:
        particular_solu = (a/2)*t**2

    #Eigenvalues
    eigvalues = np.sort((-b1+np.array([+1,-1])*(np.sqrt(b1**2-4*b2)))/2)

    #Complementary function
    comp_func = A1*eigvalues[0]**t+A2*eigvalues[1]**t+particular_solu
    return comp_func
```

#### 2.2.2.1 Ex1: Particular solution

$$y_t-10y_{t-1}+ 16y_{t-2}=14$$

```python
second_difference(-10,16, 14)
```

$\displaystyle 2.0^{t} A_{1} + 8.0^{t} A_{2} + 2.0$

#### 2.2.2.2 Ex2: Particular solution

$$y_t-6y_{t-1}+ 5y_{t-2}=12$$

```python
second_difference(-6, 5, 12)
```

$\displaystyle 1.0^{t} A_{1} + 5.0^{t} A_{2} - 3.0 t$

### 2.2.3 General solution

```python
A1, A2, e, t, y = symbols('A1 A2 e t y', integer=True)
def second_difference(b1, b2,a, y0, y1):
    #Particular equation:
    if b2+b1!=-1:
        particular_solu = (a)/(1+b1+b2)
    if b2+b1 == -1 and b1 !=-2:
        particular_solu = ((a)/(2+b1))*t
    if b2+b1==-1 and b1==-2:
        particular_solu = (a/2)*t**2

    #Eigenvalues
    eigvalues = np.sort((-b1+np.array([+1,-1])*(np.sqrt(b1**2-4*b2)))/2)

    #Complementary function
    comp_func = A1*eigvalues[0]**t+A2*eigvalues[1]**t+particular_solu

    #General solution
    general_solu_1 =  comp_func.subs({t:0})-y0
    general_solu_2 =  comp_func.subs({t:1})-y1
    general_solu = list(linsolve([general_solu_1, general_solu_2],(A1,A2)))[0]
    final_answer = general_solu[0]*eigvalues[0]**t+general_solu[1]*eigvalues[1]**t+particular_solu

    #Check answer by plugging t into equation to find y0
    #Should be equal
    assert final_answer.subs({t:0})==y0
    assert final_answer.subs({t:1})==y1

    #Stability conditions
    if abs(max(eigvalues))>1:
        print('Time path diverges')
    else:
        print('Time path converges')

    return final_answer

second_difference(-10,16, 14, 10, 36)
```

    Time path diverges

$\displaystyle 5.0 \cdot 2.0^{t} + 3.0 \cdot 8.0^{t} + 2.0$

## 2.3 Economic example: Second-order difference
- Reference: User create code derived from [Quantecon](https://python-programming.quantecon.org/python_oop.html)

###  2.3.1 The Samuelson Multiplier-Accelerator

##### Consumption function
$$
C_t = a Y_{t-1} + \gamma \tag{1}
$$

Where:
- a: Marginal propensity to consume (MPC)

##### Investment function
$$
I_t = b (Y_{t-1} - Y_{t-2}) \tag{2}
$$

Where:
- b: Accelerator coefficient

##### GDP identity
$$
Y_t = C_t + I_t + G_t \tag{3}
$$

##### Second-order linear difference equation: Combine 1, 2, 3

$$
Y_{t+2} - \rho_1 Y_{t+1} - \rho_2 Y_t  = 0 \tag{6}
$$
Where:
- $ \rho_1 = (a+b) $
- $ \rho_2 = -b $.

#### 2.3.1.1 Python example: Samuelson Multiplier-Accelerator

```python
def sam(y_0=100, y_1=80, α=.92, β=.5, γ=10, time_series=60):
    #Path
    ρ1 = α + β
    ρ2 = -β
    y = np.zeros([time_series])
    y[0] = y_0
    y[1] = y_1
    for t in range(time_series-2):
        y[t+2] = ρ1 * y[t+1] + ρ2 * y[t] + γ
    plt.plot(np.linspace(0,time_series,time_series), y)
sam(y_0=100, y_1=120, α=.92, β=.5, γ=10, time_series=60)
```

#### 2.3.1.2 Python example: Stabilty conditions--Samuelson Multiplier-Accelerator

```python
def sam_stability(α=.92, β=.5):
    roots = []
    ρ1 = α + β
    ρ2 = -β
    print(f'ρ_1 is {ρ1}')
    print(f'ρ_2 is {ρ2}')
    discriminant = ρ1 ** 2 + 4 * ρ2
    roots.append((-ρ1 + np.sqrt(discriminant)) / 2)
    roots.append((-ρ1 - np.sqrt(discriminant)) / 2)
    if ρ2 > 1 + ρ1 or ρ2 < -1:
        print('Explosive oscillations')
    elif ρ1 + ρ2 > 1:
        print('Explosive growth')
    elif discriminant < 0:
        print('Complex roots\nModulus less than one\nTherefore damped oscillations')
    else:
        print('Roots real \nAbsolute values are less than one \nTherefore smooth convergence to a steady state')
```

```python
sam_stability(α=.92, β=.5)
```

    ρ_1 is 1.42
    ρ_2 is -0.5
    Roots real
    Absolute values are less than one
    Therefore smooth convergence to a steady state

# 3. Simultaneous equations

## 3.1 Differential equations

$$
\dot{y_1} = a_{11}y_1 + a_{12}y_2+b_1 \\
\dot{y_2} = a_{21}y_1 + a_{22}y_2+b_2
$$

#### Expressed in matrix form
$$
\begin{bmatrix}
\dot{y_1}\\
\dot{y_2}
\end{bmatrix}
=
\begin{bmatrix}
a_{11} & a_{12}\\
a_{21} & a_{22}
\end{bmatrix}
\begin{bmatrix}
y_1\\
y_2
\end{bmatrix} +
\begin{bmatrix}
b_1\\
b_2
\end{bmatrix}
$$

#### Complementary solution
$$y_c = \sum_{i=1}^{n} k_i C_i e^{r_i t}$$
Where:
- $C_i$ = Eigen vector
- $r_i$ = EIgenvalue

#### Calculate eigenvalues
$$r_i = \frac{Tr(A) \pm \sqrt{Tr(A)^2 - 4|A|}}{2}$$

#### Stability conditions
- all $r_i<0$ dynamically stable
- $r_i>0$ unstable

### 3.1.1 Python example: Differential equations

```python
import numpy as np
from sympy import symbols, diff, solve
from sympy.solvers.solveset import linsolve

k1, k2, e,t = symbols('k1 k2 e t', integer=True)

def simultaneous_differential(matrix, solu, ys):
    assert sum(np.diag(A))==np.trace(A)

    #Eigenvalues
    trace = np.trace(A)
    det = round(np.linalg.det(A),0)
    eig_values = np.round((np.sort((trace+np.array([+1,-1])*np.sqrt(trace**2-(4*det)))/2)),1)

    #Eigvenvectors
    eig_vecs = []
    for i in eig_values:
        vec = matrix-np.diag(np.repeat(i,2))
        c1 = 1 #Euclidean distance condition
        c2 = c1/((-vec[0,1])/vec[0,0])
        eigvector = np.array([c1,c2]).reshape(-1,1)
        eig_vecs.append(eigvector)
    eig_matrix = np.concatenate(eig_vecs, axis=1)

    #Complementary solution
    comp_solus = []
    for i in range(np.shape(eig_matrix)[1]):
        y_complementary = eig_matrix[i,0]*k1*e**(eig_values[0]*t)+ eig_matrix[i,1]*k2*e**(eig_values[1]*t)
        comp_solus.append(y_complementary)

    #Complete general solution
    intertemporal_solu = np.linalg.solve(A,b)
    general_solus = []
    for solu, comp in zip(intertemporal_solu, comp_solus):
        general_solus.append(comp+solu)

   #Definite solution
    initial_conditions = []
    for idx, (y, solu) in enumerate(zip(ys,intertemporal_solu)):
        k = eig_matrix[idx,0]*k1+ eig_matrix[idx,1]*k2+solu-y
        initial_conditions.append(k)
    ks = list(linsolve([initial_conditions[0], initial_conditions[1]],(k1,k2)))[0]

    definite_solu = []
    for ans,intertemp in zip(comp_solus,intertemporal_solu):
           definite_solu.append(ans.subs({k1:ks[0], k2:ks[1]})+intertemp)

    return eig_values, eig_matrix, comp_solus, general_solus, definite_solu
```

#### 3.1.1.1 Complementary solutions

```python
A = np.array([[5,-.5],[-2,5]])
b = np.array([[12], [24]])
ys = [12,4]
simultaneous_differential(A, b, ys)[2][0]
```

$\displaystyle 1.0 e^{4.0 t} k_{1} + 1.0 e^{6.0 t} k_{2}$

```python
simultaneous_differential(A, b, ys)[2][1]
```

$\displaystyle 2.0 e^{4.0 t} k_{1} - 2.0 e^{6.0 t} k_{2}$

#### 3.1.1.2 Complete general solutions

```python
simultaneous_differential(A, b, ys)[3][0]
```

$\displaystyle 1.0 e^{4.0 t} k_{1} + 1.0 e^{6.0 t} k_{2} + 3.0$

```python
simultaneous_differential(A, b, ys)[3][1]
```

$\displaystyle 2.0 e^{4.0 t} k_{1} - 2.0 e^{6.0 t} k_{2} + 6.0$

#### 3.1.1.3 Definite solutions

```python
simultaneous_differential(A, b, ys)[4][0]
```

$\displaystyle 4.0 e^{4.0 t} + 5.0 e^{6.0 t} + 3.0$

```python
simultaneous_differential(A, b, ys)[4][1]
```

$\displaystyle 8.0 e^{4.0 t} - 10.0 e^{6.0 t} + 6.0$

## 3.2 Difference equations

```python
from sympy import symbols, diff, solve
from sympy.solvers.solveset import linsolve

k1, k2,t = symbols('k1 k2 t', integer=True)

def simultaneous_difference(matrix, solu, x0, y0):
    assert sum(np.diag(A))==np.trace(A)

    #Eigenvalues
    trace = np.trace(A)
    det = round(np.linalg.det(A),0)
    r1 = (trace + np.sqrt(trace**2-(4*det)))/2 #Eigenvalue finder
    r2 = (trace - np.sqrt(trace**2-(4*det)))/2 #Eigenvalue finder
    if abs(r1)>0 and abs(r2)>0:
        print('Time path is divergent')
    else:
        print("Time path is convergent")

    #Eigvenvectors
    # Eigenvector 1
    vec1 = matrix-np.diag(np.repeat(r1,2))
    c1 = 1 #Euclidean distance condition
    c2 = c1/((-vec1[1,1])/vec1[1,0])
    eigvector_1 = np.array([c1,c2]).reshape(-1,1)

    # Eigenvector 2
    vec2 = matrix-np.diag(np.repeat(r2,2))
    c1 = 1 #Euclidean distance condition
    c2 = c1/((-vec2[0,1])/vec2[0,0])
    eigvector_2 = np.array([c1,c2]).reshape(-1,1)

    #Complementary solution
    eig_matrix = np.hstack([eigvector_1, eigvector_2])
    xt = k2*(eig_matrix[0,1])*r2**t    +   k1*eig_matrix[0,0]*r1**t
    yt = k2*(eig_matrix[1,1])*r2**t    +   k1*eig_matrix[1,0]*r1**t

    #Complete general solution
    solu1, solu2 = np.linalg.inv(np.eye(2)-matrix)@solu
    y1_c = xt + solu1
    y2_c = yt + solu2

    return (r1, r2), eigvector_1, eigvector_2, xt, yt, y1_c, y2_c
```

### 3.2.1 Simultaneous differential equations

- Pg. 434, 19.3

```python
A = np.array([[-4, 1],[2,-3]])
b = np.array([[12], [6]])
x0, y0 = [16,8]
# General solutions
yt = simultaneous_difference(A, b, x0, y0)[6]
xt = simultaneous_difference(A, b, x0, y0)[5]
```

    Time path is divergent
    Time path is divergent

```python
xt
```

$\displaystyle 1.0 \left(-2.0\right)^{t} k_{1} + 1.0 \left(-5.0\right)^{t} k_{2} + 3.0$

```python
yt
```

$\displaystyle 2.0 \left(-2.0\right)^{t} k_{1} - 1.0 \left(-5.0\right)^{t} k_{2} + 3.0$

# 4. Calculus of variations

## 4.1 Basics
Dynamic optimization:
- max or min of an integral

$$\int^T_0 F[t,x(t), x^*(t)]dt$$

## 4.2 Distance between two points

$$ s= \int^{t1}_{t0}=\sqrt{1+(\frac{dx}{dt})^2 dt}$$

## 4.3 Euler's equation

$$\int^{t1}_{t0} F[t,x(t), x^*(t)]dt$$

To optimize the above function the necessary condition is Euler's equation

$$\frac{\partial F}{\partial x} = \frac{d}{dt}\Big(\frac{\partial F}{\partial \dot{x}}\Big)$$

### 4.3.1 Sympy: Euler equation

```python
t,x = sp.symbols('t, x')
xdot = sp.Function('x')
f = 6*x**2*e**(3*t)+4*t*(xdot(t))
f
```

$\displaystyle 6 e^{3 t} x^{2} + 4 t x{\left(t \right)}$

### 4.3.1.1 $\frac{\partial F}{\partial \dot{x}}$

```python
df_dx_dot = diff(f, xdot(t))
df_dx_dot
```

$\displaystyle 4 t$

### 4.3.1.2 $\frac{d}{dt}\Big(\frac{\partial F}{\partial \dot{x}}\Big)$

```python
rhs = diff(df_dx_dot,t)
rhs
```

$\displaystyle 4$

### 4.3.1.3 $\frac{\partial F}{\partial x}$

```python
df_dx = diff(f,x)
df_dx
```

$\displaystyle 12 e^{3 t} x$

### 4.3.1.4 Euler equation solution: $X(t)=$

```python
solve(df_dx-rhs, x)
```

    [e**(-3*t)/3]

## 4.5 Sufficiency conditions

$$D=\begin{bmatrix}
F_{xx}& F_{xx^*} \\
F_{x^*x}& F_{x^*x^*}
\end{bmatrix}$$

Where:
- $|D_1| = F_{xx}<0$
    - Stricly concave

- $|D_2| = |D|>0$
    - Global max

## 4.6 Dynamic optimization s.j.t constraints

$$\int^T_0 F[t,x(t), \dot{x}(t)]dt\\
s.j.t\\
\int^T_0 G[t,x(t), \dot{x}(t)]dt\\
=\int^T_0 (F+\lambda G)dt$$

Necessary condition for extremal dynamic optimization (Euler equation):

$$\frac{\partial H}{\partial x} = \frac{d}{dt}\Big(\frac{\partial H}{\partial \dot{x}}\Big)$$

### 4.6.1 Sympy example: Dynamic optimization
- Isoperimetric problem to determine a curve with a given perimeter

- Find curve y of length k enclosing a max area of A

$$
A=\frac{1}{2} \int(x\dot{y}-y)dx \\
\int_{x0}^{x1}\sqrt{1+\dot{y}^2} dx = k\\
\int_{x0}^{x1}\frac{1}{2}\Big[(x\dot{y}-y)+\lambda\sqrt{1+\dot{y}^2}\Big]dx
$$

$$
\frac{\partial H}{\partial x} = \frac{d}{dt}\Big(\frac{\partial H}{\partial \dot{x}}\Big)
$$

### 4.6.1.1 Sympy

```python
t,y,x, lamb = sp.symbols('t y x lambda')
ydot = sp.Function('y')
h = (1/2)*(x*ydot(t)-y)+lamb*sp.sqrt(1+ydot(t)**2)
h
```

$\displaystyle \lambda \sqrt{y^{2}{\left(t \right)} + 1} + 0.5 x y{\left(t \right)} - 0.5 y$

```python
diff(h,y)== -diff(diff(h, ydot(t)),x)
```

    True

# 5. Optimal control theory

## 5.1 Basics
- More powerful than calculus of variations because it can manage:
    - Corner solutions

#### Aim of optimal control is to find the optimal path of:
   - y(t): control variable
   - x(t): State variable

$$
\max J = \int^T_0 F[x(t),y(t),t]dt\\
s.j.t \\
\dot{x} = G[x(t),y(t),t]dt\\
x(0)=x_0 \\
x(T)= x_T
$$

## 5.2 Hamiltonian

$$
H[x(t),y(t),\lambda(t),t]= F[x(t),y(t),t] +\lambda(t) g[x(t),y(t),t]\\
$$

- $\lambda(t)$ = Costate variable
    - Estimates the marginal value/shadow price of thestate variable x(t)

### 5.2.1 Necessary conditions of Hamiltonian
$$
\frac{\partial h}{\partial y} = 0 \tag{1}
$$

#### Max principle
$$
\dot{\lambda} = -\frac{\partial h}{\partial x} \tag{2.a}
$$

$$
\dot{x} = \frac{\partial h}{\partial\lambda} \tag{2.b}
$$

#### Boundary conditions

$$
x(0)= x_0 \tag{3.a}
$$

$$
x(T)= x_T \tag{3.b}
$$

### 5.2.2 Sympy example: Hamiltonian

$$
\int^3_0 (4x-5y^2)dt\\
\dot{x} = 8y \\
x(0) = 2 \\
x(3) = 117.2
$$

```python
t, y, x, lamb = sp.symbols('t y x lambda')
ydot = sp.Function('y')
xdot = sp.Function('x')
lambdot = sp.Function('lambda')
H = 4*x-5*y**2+(lamb*8*y)
H
```

$\displaystyle 8 \lambda y + 4 x - 5 y^{2}$

#### 5.2.2.1 Maxium principle

```python
# 1.
one = diff(H,y)
one = solve(one, y)
one
```

    [4*lambda/5]

```python
# 2.a
two_a = - diff(H, x) #dot lamb
dot_lamb_eq = Eq(two_a, lambdot(t).diff())
dot_lamb_eq
```

```python
# 2.b
two_b = diff(H, lamb) #dot x
dot_x_eq = Eq(two_b, xdot(t).diff())
dot_x_eq
```

```python
# one into dot_x
dot_x = two_b.subs({y:4*lamb/5})
dot_x_eq = Eq(dot_x, xdot(t).diff())
dot_x_eq
```

#### 5.2.2.2 Solve for state variables $x(t)$ & co-state $\lambda(t)$

```python
costate = dsolve(dot_lamb)
costate
```

```python
dot_ex_eq = dot_x_eq.subs({lamb: lambs.rhs})
dot_ex_eq
```

```python
state = dsolve(dot_ex_eq)
state
```

#### 5.2.2.3 Boundary conditions

```python
#When state(0)=2
state_0 = state.subs({t:0})
state_0
```

```python
#C2
c2 = solve(state_0.rhs-2)
c2
```

```python
# When state(3)=117.2
state_3 = state.subs({t:3})
state_3
```

```python
c1 = 12
c1
```

#### 5.2.2.4 Solve state variable

```python
state
```

$$
x(t) = -12.8t^2+76.8t+2 \tag{state}
$$
$$
\lambda(t) = -4t+12 \tag{costate}
$$

#### 5.2.2.5 Two methods to solve control y(t)
1. Plug in costate to equation one

$$y(t)=.8(-4t+12) = -3.2t+9.6$$

2. Take deriv of state var

$$\dot{x} = -25.6t+76.8$$

Sub $\dot{x}$

$$y(t) = -3.2t+9.6\tag{control}$$

#### 5.2.2.6 Solve endpoints

$$
y(0)=9.6\\
y(3)=0
$$

## 5.3 Sufficiency conditions

$$D=\begin{bmatrix}
F_{xx}& F_{xy} \\
F_{yx}& F_{yy}
\end{bmatrix}$$

Where:
- $|D_1| = F_{xx}<0$
- $|D_2| = |D|>0$
    - Concave

## 5.4 Optimal control with free endpoints

Boundary conditions of Hamiltonian are different

$$
\frac{\partial h}{\partial y} = 0 \tag{1}
$$

#### Max principle
$$
\dot{\lambda} = -\frac{\partial h}{\partial x} \tag{2.a}
$$

$$
\dot{x} = \frac{\partial h}{\partial\lambda} \tag{2.b}
$$

#### Boundary conditions

$$
x(0)= x_0 \tag{3.a}
$$

Different than transversality condition
$$
x(T)= 0 \tag{3.b}
$$

## 5.5 Inequality constraints in endpoints

$$
\begin{cases}
\lambda(t) = 0 & \text{when} \ \ x^*(t)> x_{min}\\
\lambda(t) \geq 0  & \text{when} \ \ x^*(t)=x_{min}
\end{cases}
$$

## 5.6 Current valued Hamiltonian

Optimal control with discounting

$$
\max J = \int^T_0 e^{-pt} F[x(t),y(t),t]dt\\
s.j.t \\
\dot{x} = G[x(t),y(t),t]dt\\
$$

$$
H= e^{-pt}F[x(t),y(t),t] +\lambda(t) g[x(t),y(t),t]\\
$$

Let $\mu(t) = \lambda(t)e^{pt}$:

Form new current valued Hamiltonian
$$
He^{pt}= H_c = F[x(t),y(t),t] +mu(t) g[x(t),y(t),t]\\
$$

### 5.6.1 Sufficient conditions

Unchanged
$$
\frac{\partial H}{\partial y} = 0 \tag{1}
$$

#### Max principle
New to $H_c$
$$
\dot{\mu} = p\mu-\frac{\partial H}{\partial x} \tag{2.a}
$$

New to $H_c$
$$
\dot{x} = \frac{\partial H}{\partial\mu} \tag{2.b}
$$

#### Boundary conditions

Unchanged
$$
x(0)= x_0 \tag{3.a}
$$

New to $H_c$
$$
\mu(t)e^{pt}= 0 \tag{3.b}
$$


# Visualizing stationary paths of a  functional

Last revised: 02-Feb-2019 by Dick Furnstahl [furnstahl.1@osu.edu]

Consider the functional

$\begin{align}
  S = \int_{x_1}^{x_2} f[y(x), y'(x), x] \, dx
\end{align}$

with $y_1 = y(x_1)$ and $y_2 = y(x_2)$ fixed.  We denote by $y^*(x)$ the path that minimizes $S$ (or, more generally, makes it stationary).  Then we consider the class of candidate paths $y(x)$ given by

$\begin{align}
  y(x) = y^*(x) + \alpha \eta(x)
\end{align}$

where $\eta(x)$ is some function that vanishes at the endpoints: $\eta(x_1) = \eta(x_2) = 0$.  We can derive the Euler-Lagrange equations by minimizing $S(\alpha)$ with respect to $\alpha$.

Here we visualize this problem by considering a particular $S$, choosing among some possible $\eta(x)$ definitions, and seeing how $S$ is minimized with respect to $\alpha$.  We will also allow for an incorrect determination of $y^*(x)$, in which case we expect that the minimum alpha will give us a reasonable reproduction of the true $y^*(x)$.  The variation of $\alpha$ and the choice of functions will be made using widgets from `ipywidgets`.

## Looking at a plot of the functional evaluation versus $\alpha$

\[We'll use `%matplotlib notebook` so that we can modify figures without redrawing them.\]

```python
%matplotlib notebook
```

```python
import numpy as np
import matplotlib.pyplot as plt

import ipywidgets as widgets
from IPython.display import display

```

### Functional from Taylor problem 6.9

This problem states: "Find the equation of the path from the origin $O$ to the point $P(1,1)$ in the $xy$ plane that makes the integral $\int_O^P (y'^2 + y y' + y^2)$ stationary.  The answer from solving the Euler-Lagrange equation is $y^*(x) = \sinh(x)/\sinh(1)$.

```python
def y_star(x):
    """Path that minimizes the functional in Taylor problem 6.9."""
    return np.sinh(x) / np.sinh(1.)
```

```python
delta_x = 0.001
x_pts = np.arange(0., 1., delta_x)

fig = plt.figure(figsize=(6,3),
                 num='Visualizing stationary paths of a functional')
ax1 = fig.add_subplot(1,2,1)
ax2 = fig.add_subplot(1,2,2)

def setup_figure():
    ax1.set_title('Show paths')
    ax1.plot(x_pts, y_star(x_pts), color='black', lw=2)
    ax1.set_xlabel('x')
    ax1.set_ylabel('y(x)')

    ax2.set_title('Evaluate functional')
    ax2.set_xlabel(r'$\alpha$')
    ax2.set_ylabel('functional')
    ax2.set_xlim(-0.4, 0.4)
    ax2.set_ylim(1.5, 3.)
    #ax2.axvline(0., color='black', alpha=0.3)
    ax2.axhline(evaluate_functional(x_pts, y_star(x_pts)),
                                    color='black', alpha=0.3)

    fig.tight_layout()

def evaluate_functional(x_pts, y_pts):
    """Given arrays of x and y points, evaluate the functional from 6.9."""
    # The numpy gradient function takes the derivative of an array y_pts
    #  that is a function of points x in x_pts.
    y_deriv_pts = np.gradient(y_pts, x_pts)
    f = y_deriv_pts**2 + y_pts * y_deriv_pts + y_pts**2

    # Use the numpy trapezoid rule (trapz) to do the integral over f.
    return np.trapz(f, x_pts)

def make_path(alpha, ax1_passed, ax2_passed,
              base_function='exact', eta_function='sine'):
    """Given a base function, which may be the exact y^*(x) or a guess that
       is not correct, generate and plot the path corresponding to adding
       alpha*eta(x) to the base function, with eta(x) chosen among some
       functions that vanish at the endpoints in x.
    """
    # map x_pts to zero to 1 (it may already be there)
    x_mapped_pts = (x_pts - x_pts[0]) / (x_pts[-1] - x_pts[0])

    # Choices for the base function
    if (base_function == 'exact'):
        base = lambda x : y_star(x)
    elif (base_function == 'guess 1'):
        base = lambda x : np.sinh(2.*x) / np.sinh(2.)
    elif (base_function == 'guess 2'):
        base = lambda x : x**3

    if (eta_function == 'sine'):
        eta = lambda x : np.sin(np.pi * x)
    elif (eta_function == 'parabola'):
        eta = lambda x : 4. * x * (1. - x)

    y_new_pts = base(x_pts) + alpha * eta(x_mapped_pts)

    ax1_passed.plot(x_pts, y_new_pts, color='red', lw=1)
    ax2_passed.plot(alpha, evaluate_functional(x_pts, y_new_pts), '.',
                    color='red')

def reset_graph(event):
    ax1.clear()
    ax2.clear()
    setup_figure()

button = widgets.Button(
    description='reset graph'
)
button.on_click(reset_graph)

widgets.interact(make_path,
                 alpha=widgets.FloatSlider(min=-1., max=1., step=.05,
                                           value=0.0, description=r'$\alpha$',
                                           continuous_update=False),
                 ax1_passed=widgets.fixed(ax1),
                 ax2_passed=widgets.fixed(ax2),
                 base_function=widgets.Dropdown(options=['exact', 'guess 1',
                                                         'guess 2'],
                                                value='exact',
                                                description='base function'),
                 eta_function=widgets.Dropdown(options=['sine', 'parabola'],
                                               value='sine',
                                               description=r'$\eta(x)$')
                )

setup_figure()
button
```

    <IPython.core.display.Javascript object>

    interactive(children=(FloatSlider(value=0.0, continuous_update=False, description='$\\alpha$', max=1.0, min=-1…

    Button(description='reset graph', style=ButtonStyle())

```python

```


# Illumination profile

In this notebook, we validate several of the expressions in the paper related to the illumination profile due to a point source on the projected disk of a sphere.

```python
%matplotlib inline
```

```python
%run notebook_setup.py
```

```python
import matplotlib.pyplot as plt
import numpy as np
import ipywidgets as widgets
from ipywidgets import HBox, VBox
from IPython.display import display
import sympy

np.seterr(invalid="ignore");
```

The illumination at a point on the surface is $I = \text{max}(0, \cos\theta_i)$, where $\theta_i$ is the angle between the surface normal and the incident light. We can compute the illumination profile on the surface of the sky-projected sphere given the illumination source Cartesian coordinate vector $\mathbf{s}$ as follows:

```python
def compute_illumination_from_s(s, res=300):

    # Construct a uniform grid over the unit disk
    x, y = np.meshgrid(np.linspace(-1, 1, res), np.linspace(-1, 1, res))
    z = np.sqrt(1 - x ** 2 - y ** 2)
    x = x.reshape(1, -1)
    y = y.reshape(1, -1)
    z = z.reshape(1, -1)

    # The surface normal at a point is easy in Cartesian coords:
    n = np.vstack([x, y, z])

    # Compute the cosine of the angle of incidence.
    # It's just the dot product between the source vector
    # and the surface normal:
    cos_theta_i = np.dot(s, n)

    # The illumination profile is proportional to max(0, cos(theta_i))
    I = np.maximum(0, cos_theta_i)

    # Normalize to get the units correct (see below)
    rs = np.sqrt(s[0] ** 2 + s[1] ** 2 + s[2] ** 2)
    I /= (np.pi * rs ** 2)

    # Reshape to image form
    I = I.reshape(res, res)
    return x, y, I
```

As we mention in the paper, we can also compute it as a function of the parameters of the terminator ellipse, $b$ and $\theta$, as well as the distance to the source, $r_s$:

```python
def compute_illumination_from_b_theta_rs(b, theta, rs, res=300):

    # Construct a uniform grid over the unit disk
    x, y = np.meshgrid(np.linspace(-1, 1, res), np.linspace(-1, 1, res))
    z = np.sqrt(1 - x ** 2 - y ** 2)
    x = x.reshape(1, -1)
    y = y.reshape(1, -1)
    z = z.reshape(1, -1)

    # Complement of b
    bc = np.sqrt(1 - b ** 2)

    # Polynomial illumination
    I = -bc * np.sin(theta) * x + bc * np.cos(theta) * y - b * z

    # Remove the nightside, where the polynomial goes negative
    I = np.maximum(0, I)

    # Normalize to get the units correct (see below)
    I /= (np.pi * rs ** 2)

    # Reshape to image form
    I = I.reshape(res, res)
    return x, y, I
```

Let's visualize this for different values of $\mathbf{s}$ using a Jupyter widget. We'll show that our expression computed from $b$ and $\theta$ is equivalent:

```python
@widgets.interact(xs=(-1.0, 1.0), ys=(-1.0, 1.0), zs=(-1.0, 1.0), res=widgets.fixed(300))
def visualize(xs=-0.3, ys=0.2, zs=0.3, res=300):

    # Set up the figure
    fig, ax = plt.subplots(1, 2, figsize=(8, 4))
    for axis in ax:
        axis.set_xticks([])
        axis.set_yticks([])
        axis.set_frame_on(False)
    ax[0].set_xlabel("from source vector")
    ax[1].set_xlabel(r"from $b$ and $\theta$")

    # Compute and plot the illumination given a source vector
    x, y, I = compute_illumination_from_s([xs, ys, zs], res=res)
    im = ax[0].imshow(I, origin="lower", extent=(-1, 1, -1, 1), cmap="Greys_r")

    # Get the semi-minor axis of the ellipse
    rs = np.sqrt(xs ** 2 + ys ** 2 + zs ** 2)
    b = -zs / rs

    # Get the orientation of the ellipse on the sky
    theta = -np.arctan2(xs, ys)

    # Compute and plot the illumination given b and theta
    x, y, I = compute_illumination_from_b_theta_rs(b, theta, rs, res=res)
    im = ax[1].imshow(I, origin="lower", extent=(-1, 1, -1, 1), cmap="Greys_r")

    # The equation for an ellipse parallel to the x axis
    x0 = np.linspace(-1, 1, 1000)
    y0 = b * np.sqrt(1 - x0 ** 2)

    # Rotate it by theta
    x = x0 * np.cos(theta) - y0 * np.sin(theta)
    y = x0 * np.sin(theta) + y0 * np.cos(theta)

    # Plot the ellipse
    ax[0].plot(x, y, "C1")
    ax[1].plot(x, y, "C1");
```

    interactive(children=(FloatSlider(value=-0.3, description='xs', max=1.0, min=-1.0), FloatSlider(value=0.2, des…

The orange line is the day/night terminator, computed from the expressions in the text.

We also argue in the text that we must normalize the intensity by $\frac{1}{\pi r_s^2}$ to get the units correct, where $r_s$ is the distance to the source (in units where the radius of the illuminated body, i.e., the planet, is 1). To see why, let's look at our body at full phase (corresponding to $b = -1$ and any value of $\theta$), illuminated by a point source unit radius away. (The source is therefore sitting right above the surface of the body, which is unphysical for so many reasons, but never mind that for now).

```python
b = -1
theta = 0
rs = 1
```

```python
fig, ax = plt.subplots(1, figsize=(4, 4))
x, y, I = compute_illumination_from_b_theta_rs(b, theta, rs, res=1000)
im = ax.imshow(I, origin="lower", extent=(-1, 1, -1, 1), cmap="Greys_r")
plt.colorbar(im, shrink=0.75);
```

Let's integrate this intensity profile numerically to compute the *flux* measured by the observer. We will do this by summing the values of the intensity over the disk times the albedo (which we asssume to be unity) and times the tiny area of each pixel. If there are $N$ pixels, each one has area
$dA = \frac{2}{\sqrt{N}}\times\frac{2}{\sqrt{N}} = \frac{4}{N}$:

```python
dA = 4.0 / len(I.flat)
dA
```

    4e-06

The total flux is thus (at two significant figures, the precision of the numerical integration)

```python
f = np.nansum(I) * dA
print("{:.2f}".format(f))
```

    0.67

At higher precision, it's clear that [the answer](https://www.wolframalpha.com/input/?i=integrate+sqrt%281+-+x%5E2+-+y%5E2%29%2Fpi+from+y+%3D+-sqrt%281+-+x%5E2%29+to+y+%3D+sqrt%281+-+x%5E2%29+and+x+%3D+-1+to+x+%3D+1) is exactly $\frac{2}{3}$. This is equal to the geometric albedo of a Lambert sphere, which means that a Lambert sphere is $2/3$ as bright as a disk of the same (projected) size, illuminated with the same intensity.

## The illumination operator

Since we can write the (dayside portion of the) illumination profile as a polynomial, we can construct a linear operator $\mathbf{I}$ that weights an albedo map expressed in the polynomial basis on the sphere. From the original `starry` paper, we may compute the terms in the polynomial basis from

```python
def get_ijk(n):
    """Get the exponents of x, y, z i the nth term of the polynomial basis."""
    l = int(np.floor(np.sqrt(n)))
    m = n - l * l - l
    mu = l - m
    nu = l + m
    if mu % 2 == 0:
        i = mu // 2
        j = nu // 2
        k = 0
    else:
        i = (mu - 1) // 2
        j = (nu - 1) // 2
        k = 1
    return i, j, k

def poly_basis(x, y, z, deg):
    """Return the polynomial basis evaluated at `x`, `y`, `z`."""
    N = (deg + 1) ** 2
    try:
        B = np.zeros((len(x * y * z), N))
    except TypeError:
        B = np.zeros((1, N), dtype="object")
    for n in range(N):
        i, j, k = get_ijk(n)
        B[:, n] = x ** i * y ** j * z ** k
    return B
```

For reference, here's the basis up to total degree $l = 3$:

```python
x, y, z = sympy.symbols("x y z")
list(poly_basis(x, y, z, 3)[0])
```

    [1,
     x,
     z,
     y,
     x**2,
     x*z,
     x*y,
     y*z,
     y**2,
     x**3,
     x**2*z,
     x**2*y,
     x*y*z,
     x*y**2,
     y**2*z,
     y**3]

The illumination polynomial may be written in this basis as

```python
def get_i(b, theta, rs):
    bc = np.sqrt(1 - b ** 2)
    return np.array([0, -bc * np.sin(theta), -b, bc * np.cos(theta)]) / (np.pi * rs ** 2)
```

Let's construct the illumination operator for a degree $l = 1$ map, as in the example presented in the paper. This operator will transform a degree $l = 1$ map in the polynomial basis to a degree $l = 2$ *illuminated* map, still in the polynomial basis.

```python
def get_I(b, theta, rs):
    i = get_i(b, theta, rs)
    I = np.zeros((9, 4))
    I[:, 0] = [i[0], i[1], i[2], i[3], 0, 0, 0, 0, 0]
    I[:, 1] = [0, i[0], 0, 0, i[1], i[2], i[3], 0, 0]
    I[:, 2] = [i[2], 0, i[0], 0, -i[2], i[1], 0, i[3], -i[2]]
    I[:, 3] = [0, 0, 0, i[0], 0, 0, i[1], i[2], i[3]]
    return I
```

Here's what that looks like for a particular value of the inputs:

```python
b = -0.25
theta = 30 * np.pi / 180
rs = 1
I = get_I(b, theta, rs)
print(I)
```

    [[ 0.          0.          0.07957747  0.        ]
     [-0.15410111  0.          0.          0.        ]
     [ 0.07957747  0.          0.          0.        ]
     [ 0.26691095  0.          0.          0.        ]
     [ 0.         -0.15410111 -0.07957747  0.        ]
     [ 0.          0.07957747 -0.15410111  0.        ]
     [ 0.          0.26691095  0.         -0.15410111]
     [ 0.          0.          0.26691095  0.07957747]
     [ 0.          0.         -0.07957747  0.26691095]]

This matrix will weight a polynomial map by the function

```python
res = 300
x, y, illum = compute_illumination_from_b_theta_rs(b, theta, rs, res=res)
plt.imshow(illum, origin="lower", extent=(-1, 1, -1, 1), cmap="Greys_r")
plt.gca().axis("off");
```

As a concrete demonstration, say I have the following $l = 1$ representation of the albedo in the polynomial basis:

```python
p = np.array([1.0, -0.5, 0, -0.5])
```

To see what that looks like, evaluate the polynomial basis on the projected disk and dot it into our vector:

```python
x, y = np.meshgrid(np.linspace(-1, 1, res), np.linspace(-1, 1, res))
z = np.sqrt(1 - x ** 2 - y ** 2)
x = x.flatten()
y = y.flatten()
z = z.flatten()
basis = poly_basis(x, y, z, 1)
image = basis.dot(p).reshape(res, res)
plt.imshow(image, origin="lower", cmap="plasma", vmin=0)
plt.gca().axis("off");
```

Let's illuminate this sphere by applying our illumination operator to $p$:

```python
Ip = I.dot(p)
```

And now let's visualize it like we did above. Note that we need to re-compute the basis, this time at $l=2$.

```python
basis = poly_basis(x, y, z, 2)
image_illum = basis.dot(Ip).reshape(res, res)
plt.imshow(image_illum, origin="lower", cmap="plasma", vmin=0)
plt.gca().axis("off");
```

We can check that we get the exact same result by multiplying the unilluminated `image` by the illumination profile `illum`:

```python
plt.imshow(image * illum, origin="lower", cmap="plasma", vmin=0)
plt.gca().axis("off");
```

## A (very important) caveat

As we discuss in the text, the illumination profile isn't strictly a polynomial; it's a piecewise function of a polynomial. The nightside intensity is, of course, zero everywhere. We swept this under the rug by setting `vmin=0` in the plotting above. If we remove that, we see that our map illuminated with the operator $\mathbf{I}$ isn't quite physical:

```python
plt.imshow(
    image_illum,
    origin="lower",
    cmap="RdBu_r",
    extent=(-1, 1, -1, 1),
    vmin=np.nanmin(image_illum),
    vmax=-np.nanmin(image_illum),
)
plt.colorbar()

x0 = np.linspace(-1, 1, 1000)
y0 = b * np.sqrt(1 - x0 ** 2)
x = x0 * np.cos(theta) - y0 * np.sin(theta)
y = x0 * np.sin(theta) + y0 * np.cos(theta)
plt.plot(x, y, "k--")
plt.gca().axis("off");
```

We switched the colormap to make it easier to see where things go negative (everything that is blue). The terminator is also indicated.

So that's not good. This, in fact, is why reflected light curves are so hard to compute! We discuss at length in the paper how to skirt this issue. The polynomial illumination operator is extremely useful, as long as we are always careful to exclude the unphysical nightside from the integration region when computing fluxes.


# Simple 1-D Burger's POD-ROM

The problem we are solving is, for $ x\in [0,100] $ and $t\in [0, T]$:

\begin{align}\label{eq:burgers}
\begin{split}
    w_t + \frac{1}{2}\left(w^2\right)_x &= 0.02 e^{\mu_2 x}, \\
    w(0, t, \mu) &= \mu_1, \\
    w(x,0,\mu) &= 1.
\end{split}
\end{align}

```python
# These are required
from time import perf_counter
import numpy as np
import matplotlib.pyplot as plt

# This is only necessary to load the previously prepared data
from torch import load
```

### This is the class managing the simulation.

Computes ROM and (optionally) FOM with finite differences.  Supports loading data from my file or generating it from a list of parameter pairs.

```python
class BurgersROM:

    def __init__(self):
        pass

    def POD_from_data(self, file, n):
        '''
        Loads snapshot data from my BurgersData file.
        Computes POD approximation after subtracting initial condition.
        '''
        data = load(file)
        S = data['S']
        M = data['M']

        tStart = perf_counter()
        for i in range(12):
            IC = np.ones(S.shape[1])
            IC[0] = M[101*i][1]
            S[101*i:101*(i+1)] = S[101*i:101*(i+1)] - IC.reshape(1,-1)
        self.POD = np.linalg.svd(S.T)[0][:,:n]
        tEnd = perf_counter()

        print(f'POD time is {tEnd-tStart}')

    def POD_from_parameters(self, N, Nt, t_max, mu_list, n):
        '''
        Computes a POD approximation from a user-defined list of parameters.
        '''
        sols = [0 for mu in mu_list]
        u0 = np.ones(N)
        for i, mu in enumerate(mu_list):
            sols[i] = self.solve_FOM(N, Nt, t_max, mu, verbose=False)
            u0[0] = mu[0]
            sols[i] = sols[i] - u0.reshape(-1,1)
        snapshots = np.concatenate(sols, axis=1)
        self.POD = np.linalg.svd(snapshots)[0][:,:n]

    def solve_FOM(self, N, Nt, t_max, mu, I=[0,100], verbose=True):
        '''
        Computes the (N x Nt) FOM solution at parameter mu.
        '''
        def f(u): return 0.5 * u**2  # Convenience function

        # Define parameters
        a, b = I
        dt = t_max / Nt
        x = np.linspace(a, b, N)
        dx = (b - a) / N

        # Define forcing.
        g = 0.02 * np.exp(mu[1] * x)

        # Set up the initial solution values.
        u0 = np.ones_like(x)
        u0[0] = mu[0]  # BC at left endpoint

        # Initialize quantities.
        U = np.zeros((Nt+1, N))
        U[0] = u0

        # Implementation of the numerical method.
        FOMtElapsed = 0
        for i in range(1, Nt+1):

            # Upwind conservative.
            FOMtStart = perf_counter()
            u = U[i-1]
            uNew = u
            uNew[1:] = u[1:] + dt * (-1 / dx * (f(u[1:]) - f(u[:-1])) + g[1:])

            FOMtEnd = perf_counter()
            FOMtElapsed = FOMtElapsed + FOMtEnd - FOMtStart

            # Save the latest result.
            U[i] = uNew

        if verbose is True: print(f'FOM time is {FOMtElapsed}')
        return U.T

    def assemble(self, mu, I=[0,100]):
        '''
        Assembles ROM system at parameter mu.
        Uses POD basis computed earlier.
        '''
        a, b = I
        N = self.POD.shape[0]
        x = np.linspace(a, b, N)
        dx = (b - a) / N

        def tridiag(a, b, c, k1=-1, k2=0, k3=1):
            return np.diag(a, k1) + np.diag(b, k2) + np.diag(c, k3)

        tStart = perf_counter()

        # Precompute quantities for ROM.
        PODinv = self.POD.T  # Since POD comes from unitary mtx
        self.gRed = PODinv @ (0.02 * np.exp(mu[1] * x))

        fdiag = np.insert(np.ones(N-1), 0, 0)
        FD = tridiag(-np.ones(N-1), fdiag, np.zeros(N-1)) / dx
        fd = PODinv @ FD  # low-dim FD matrix

        u0 = np.ones_like(x)
        u0[0] = mu[0]  # BC at left endpoint

        self.aVec = fd @ (0.5*u0**2)  # rank-1 term in (u^2)_x

        self.Bmat = fd @ np.diag(u0) @ self.POD  # rank-2 term in (u^2)_x

        outprod = np.tensordot(self.POD, self.POD, axes=0)
        temp = 0.5 * outprod.diagonal(axis1=0, axis2=2).transpose((2,1,0))
        self.Ctens = np.einsum('ij,jkl', fd, temp)  # rank-3 term in (u^2)_x

        tEnd = perf_counter()
        print(f'assembly time is {tEnd-tStart}')

    def solve_ROM(self, Nt, t_max, I=[0,100]):
        '''
        Forward Euler to solve the pre-assembled ROM.
        '''
        # Define parameters
        a, b = I
        dt = t_max / Nt
        N = self.POD.shape[0]
        x = np.linspace(a, b, N)
        dx = (b-a) / N

        # Initialize quantities.
        uRed0 = np.zeros(self.POD.shape[1])
        U = np.zeros((Nt+1, self.POD.shape[1]))
        U[0] = uRed0

        # Implementation of the numerical method.
        ROMtElapsed = 0
        for i in range(1,Nt+1):
            ROMtStart = perf_counter()

            uRed = U[i-1]
            uRed2 = np.tensordot(uRed, uRed, axes=0)
            uRedNew = uRed + dt * (self.gRed - self.aVec - self.Bmat @ uRed \
                                   - np.einsum('ijk,jk', self.Ctens, uRed2))

            ROMtEnd = perf_counter()
            ROMtElapsed = ROMtElapsed + ROMtEnd - ROMtStart

            #  Save the latest result.
            U[i] = uRedNew

        print(f'ROM time is {ROMtElapsed}')
        return U.T

    def reconstruct(self, approx, mu):
        '''
        Reconstruct approximation to the FOM solution at mu from approx.
        '''
        u0 = np.ones(self.POD.shape[0])
        u0[0] = mu[0]
        return u0.reshape(-1,1) + self.POD @ approx

```

## Testing

We test the convergence under refinement of the reduced dimension.  First, we use the POD basis computed from my saved data.

```python
x = np.linspace(0, 100, 256)
mu = [2.75, 0.00275]

p=50

rom = BurgersROM()
sol = rom.solve_FOM(256, 100, 10, mu)

rom.POD_from_data('datasets/burgersData', 3)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol4 = rom.reconstruct(qqq, mu)

rom.POD_from_data('datasets/burgersData', 10)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol10 = rom.reconstruct(qqq, mu)

rom.POD_from_data('datasets/burgersData', 30)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol30 = rom.reconstruct(qqq, mu)

fig = plt.figure(1, figsize=(8, 8))
ax = fig.add_subplot(1, 1, 1)
ax.xaxis.label.set_size(24)
ax.yaxis.label.set_size(24)
ax.tick_params(axis='x', labelsize=18)
ax.tick_params(axis='y', labelsize=18)
ax.set_xlabel('Position $x$', labelpad=15)
ax.set_ylabel('Solution $w$', labelpad=15)
ax.plot(x, sol[:,p])
ax.plot(x, sol4[:,p])
ax.plot(x, sol10[:,p])
ax.plot(x, sol30[:,p])
plt.show()
```

### Testing from parameters

Now, we test the basis computed from some random parameters.

```python
mu_list = np.array([0.5, 0.001]) + np.random.rand(10,2) * [3.5, 0.003]
x = np.linspace(0, 100, 256)

mu = [2.75, 0.00275]

p=50

rom = BurgersROM()
sol = rom.solve_FOM(256, 100, 10, mu)

rom.POD_from_parameters(256, 100, 10, mu_list, 3)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol4 = rom.reconstruct(qqq, mu)

rom.POD_from_parameters(256, 100, 10, mu_list, 10)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol10 = rom.reconstruct(qqq, mu)

rom.POD_from_parameters(256, 100, 10, mu_list, 30)
rom.assemble(mu)
qqq = rom.solve_ROM(100, 10)
sol30 = rom.reconstruct(qqq, mu)

fig = plt.figure(1, figsize=(8, 8))
ax = fig.add_subplot(1, 1, 1)
ax.xaxis.label.set_size(24)
ax.yaxis.label.set_size(24)
ax.tick_params(axis='x', labelsize=18)
ax.tick_params(axis='y', labelsize=18)
ax.set_xlabel('Position $x$', labelpad=15)
ax.set_ylabel('Solution $w$', labelpad=15)
ax.plot(x, sol[:,p])
ax.plot(x, sol4[:,p])
ax.plot(x, sol10[:,p])
ax.plot(x, sol30[:,p])
plt.show()
```


```python
import sympy as sy
```

```python
Ru, Rv = sy.symbols("R_u R_v")
r, s, v = sy.symbols("r s v")

K = sy.Matrix([
                [ (1-v) * Ru, (1-v) * (1-r) * Rv],
                [ v * (1-s) * Ru, v * (1-r) *(1-s)* Rv],
              ])
vec = K.eigenvects()[-1][-1][-1]
vec /= sum(vec)
sy.simplify(vec)
```

$\displaystyle \left[\begin{matrix}\frac{1 - v}{- s v + 1}\\\frac{v \left(1 - s\right)}{- s v + 1}\end{matrix}\right]$

```python
K
```

$\displaystyle \left[\begin{matrix}R_{u} \left(1 - v\right) & R_{v} \left(1 - r\right) \left(1 - v\right)\\R_{u} v \left(1 - s\right) & R_{v} v \left(1 - r\right) \left(1 - s\right)\end{matrix}\right]$

```python
for eigenvalue, algebraic_multiplicity, eigenvectors in K.eigenvects():
    print(f'{eigenvalue=}, {algebraic_multiplicity=}')
    print(sy.simplify(eigenvectors[0]/sum(eigenvectors[0])))
```

    eigenvalue=0, algebraic_multiplicity=1
    Matrix([[R_v*(r - 1)/(R_u + R_v*r - R_v)], [R_u/(R_u + R_v*r - R_v)]])
    eigenvalue=-R_u*v + R_u + R_v*r*s*v - R_v*r*v - R_v*s*v + R_v*v, algebraic_multiplicity=1
    Matrix([[(1 - v)/(-s*v + 1)], [v*(1 - s)/(-s*v + 1)]])

```python
K
```

$\displaystyle \left[\begin{matrix}R_{u} \left(1 - v\right) & R_{v} \left(1 - r\right) \left(1 - v\right)\\R_{u} v \left(1 - s\right) & R_{v} v \left(1 - r\right) \left(1 - s\right)\end{matrix}\right]$

```python
(K.dot(vec))
```

    [-R_u*(1 - v)**2/((s*v - v)*(-(1 - v)/(s*v - v) + 1)) + R_v*(1 - r)*(1 - v)/(-(1 - v)/(s*v - v) + 1),
     -R_u*v*(1 - s)*(1 - v)/((s*v - v)*(-(1 - v)/(s*v - v) + 1)) + R_v*v*(1 - r)*(1 - s)/(-(1 - v)/(s*v - v) + 1)]

```python
sy.simplify(sum(K.dot(vec)))
```

$\displaystyle - R_{u} v + R_{u} + R_{v} r s v - R_{v} r v - R_{v} s v + R_{v} v$

```python

```


# CHAPTER 21 - Deep Learning
### George Tzanetakis, University of Victoria

## WORKPLAN

The section number is based on the 4th edition of the AIMA textbook and is the suggested
reading for this week. Each list entry provides just the additional sections. For example the Expected reading include the sections listed under Basic as well as the sections listed under Expected. Some additional readings are suggested for Advanced.

1. Basic: Sections **21.1**, **21.2**, , and **Summary**
2. Expected: Same as Basic plus **21.3**
3. Advanced: All the chapter including bibligraphical and historical notes

# Deep Learning

Understanding deep learing starts from the simple feed-forward netowrk with input layer, hidden layers, and output layer. Each node/unit in a layer is collected with weights to every node in the next layer. The resulting weighted sum is then processed by a nonlinear activition function to produce output. So one can think of the entire network as a sequence of applying vector-matrix multiplication followed by non-linar activations.
We can add +1 to the input so that total weighted sum can be non-zero even when all the output of the previous layer/inputs of the current layer are all zero.

\begin{equation}
a_j = g_j(\mathbf{w}^T \mathbf{x})
\end{equation}

where $\mathbf(w)$ is the vector of weights leading into unit $j$, and $\mathbf x$ is the vector of inputs to unit $j$.

Activation is non-linear otherwise any composition of units would still represent a linear function. The nonlinearity is what allows sufficiently large networks of units to represent arbitrary functions.

There are different activation functions:

1. Logistic or sigmoid
\begin{equation}
\sigma(x) = 1 / (1 + e^{-x})
\end{equation}
2. Rectified linear unit
\begin{equation}
ReLU(x) = max(0,x)
\end{equation}
3. Softplus
\begin{equation}
softplus(x) = log(1 + e^{x})
\end{equation}
4. tanh

Vector form of network:
\begin{equation}
h_w(\mathbf x) = g^{2}(\mathbf{W}^{2}g^{1}(\mathbf W^{(1)}\mathbf x))
\end{equation}

For supervised learning training we can use **gradient descent** i.e calculate of the gradient of the loss function with respect to the weights, and then adjust the weights along the gradient direction to reduce the loss.

Example loss function where $y$ is the ground truth value and $\hat y$ is the output prediction of the network.
\begin{equation}
Loss(h_{w}) = (y-\hat y)^2
\end{equation}

The idea is that for each training sample (or mini-batch of samples) we can calcuate the gradient with respect to the loss function and propagate the error backwards from the output layer through the hidden layers, and eventually to the input layer. This algorithm is called **back propagtion**.

## Input encoding

Boolean attributes are typically encoded as $0$ for False and $1$ for True. Numeric attributes whether integer
or real-valued are typically used as is or sometimes mapped onto a log-scale. Networks used with images have array-like internal structures that aim to reflect the semantics of adjacency of pixels. Categorial values are usually encoded with **one-hot-encoding** to avoid numerical adjacency issues.

## Output layers and loss functions

In most deep learning applications, it is more common to interpret the output value $\mathbf{\hat y}$ as probabilities and to use the **negative log likelihood** as the loss function.

\begin{equation}
\mathbf{w^*} = \underset{w}{\operatorname{argmin}} - \sum_{j=1}^{N} logP_w(\mathbf{y_j}|\mathbf{x_j})
\end{equation}

Without going into details this is called the cross-entropy loss in Deep Learning literature.
For binary classification problems a **sigmoid output** layer will do what we want and output
probabilities. For multiclass problems we can use a **softmax** layer which outputs a vector of non-negative
numbers that sum up to 1. For regression problems we can use a linear output layer without any activation function. This corresponds to doing a classical linear regression at the output layer after the multiple non-linear transforms.

## Hidden Layers

From 1985-2010 internal nodes typically used sigmoid and tanh activation functions almost exclusively. From around 20210 onwards the ReLU and softplus become more popular, partly because they are believed to avoid the problem of vanishing gradients. Experiments suggest that deep and narrow networks learn better than shallow and wide given a fixed total number of weights.

There is little understanding at the moment as to why some structures seem to work better than others for some particular problem.

```python
import torch
import torchvision
import torchvision.transforms as transforms

```

```python
transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
```

    Files already downloaded and verified
    Files already downloaded and verified

```python
import matplotlib.pyplot as plt
import numpy as np

# functions to show an image

def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()

# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(batch_size)))

```

```python
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1) # flatten all dimensions except batch
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()
```

```python
import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
```

```python
for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')
```

    [1,  2000] loss: 2.212
    [1,  4000] loss: 1.874
    [1,  6000] loss: 1.695
    [1,  8000] loss: 1.580
    [1, 10000] loss: 1.509
    [1, 12000] loss: 1.474
    [2,  2000] loss: 1.370
    [2,  4000] loss: 1.384
    [2,  6000] loss: 1.352
    [2,  8000] loss: 1.325
    [2, 10000] loss: 1.309
    [2, 12000] loss: 1.293
    Finished Training

```python
dataiter = iter(testloader)
images, labels = dataiter.next()

# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))
```

```python
PATH = './cifar_net.pth'
torch.save(net.state_dict(), PATH)

net = Net()
net.load_state_dict(torch.load(PATH))

outputs = net(images)
_, predicted = torch.max(outputs, 1)

print(outputs)
print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]
                              for j in range(4)))
```

    tensor([[-1.1878, -1.4310,  0.4186,  3.3092,  0.0564,  2.9314, -0.3943,  0.2575,
             -0.8055, -1.9552],
            [ 3.6265,  6.8741, -1.3167, -2.1925, -2.9995, -3.3881, -3.6827, -4.1026,
              5.5275,  3.6994],
            [ 1.8977,  2.9827, -0.7546, -0.4158, -1.4942, -1.7446, -2.1207, -2.1412,
              2.4460,  2.5054],
            [ 3.5295,  1.4411,  1.0033, -1.6429, -0.9217, -3.1417, -2.8556, -3.6819,
              6.4549,  1.1515]], grad_fn=<AddmmBackward>)
    Predicted:    cat   car   car  ship

```python
import torch

a = torch.tensor([2., 3.], requires_grad=True)
b = torch.tensor([6., 4.], requires_grad=True)

Q = 3*a**3 - b**2
external_grad = torch.tensor([1., 1.])
Q.backward(gradient=external_grad)

print(9*a**2)
print(a.grad)
```

    tensor([36., 81.], grad_fn=<MulBackward0>)
    tensor([36., 81.])

```python

```

## Differentiation using computers

1. Symbolic differentiation: Automatic manipulation of mathematical expressions to get derivatives
    * Input and output are mathematical expressions
    * Used in Mathematica, Maple, Sympy, etc.

2.  Numeric differentiation: Approximating derivatives by finite differences:

\begin{equation}
\frac{\partial f(x_1, \dots, x_N}{\partial x_i)} = \frac{f(x_1, \dots, x_i+h, \dots, x_N)}{2h}
\end{equation}

∂xi
f(x1, . . . , xN) = lim
h→0
f(x1, . . . , xi + h, . . . , xN) − f(x1, . . . , xi − h, . . . , xN)
2h

• Automatic differentiation (AD): A method to get exact derivatives
efficiently, by storing information as you go forward that you can reuse
as you go backwards
• Takes code that computes a function and returns code that computes the
derivative of that function.
• “The goal isn’t to obtain closed-form solutions, but to be able to write a
program that efficiently computes the derivatives.”
• Autograd, Torch Autograd

```python
import autograd.numpy as np
from autograd import elementwise_grad as egrad
import matplotlib.pyplot as plt

x = np.linspace(-31.4,31.4, 256)
sinc = lambda x: np.sin(x) / x
plt.figure(figsize=(12,7))
plt.title("sinc function and derivatives", fontsize=24)
my_fn = sinc

for ii in range(5):
    plt.plot(x, my_fn(x), lw=3, label="d{} sinc(x)/dx{}".format(ii,ii))
    plt.legend(fontsize=18)
    plt.axis([-32, 32, -0.50, 1.2])
    my_fn = egrad(my_fn)
```

## Differentiable programming

The idea of computing using building blocks that can be differentiated orinated in Deep Learning but has found applications in other areas. For example it is possible to model chains of digital signal processing effects
using differentiable DSP.

## Convolutional neural networks

In order to preserve local adjacency information, the first hidden layer should receive input from only a small, local region of the image. In addition, we would like approximate **spatial invariance** at small to moderate scales - i.e an edge or an eye should look the same if it appears in a different, small region of the image.

Constraining the $l$ weights connecting a local region to a unit in the hidden layer to be the same for each hidden unit (i.e $w_{1,i}, \dots w_{l,i}$ should be the same as $w_{1,j}, \dots, w_{l, j}$ for hidden unit $i$ and $j$. A pattern of weights that is replicated across multiple local regions is called a **kernel** and the process of applying the **kernel** to the pixels of an image is called **convolution**. Note that in DSP this is called cross-correlation and convolution means something else.

Let's illustrate **convolution** with an example in 1D. Consider the sequence $5,6,6,2,5,6,5$ and the **kernel**
$+1,-1,+1$. This kernel will detect (produce higher values) when there is a "dark" pixel. The result of applying
the kernel at the first position of the sequence will be: $5-6+6 = 5$. The kernel is then moved by **stride** pixels. For example if the **stride** is two the next output value will be $6-2+5=9$. If the stride is one the next output value will be $6-6+2=2$.

Convolution is a linear operation and therefore we can propagate gradients through it, just like we did with fully connected networks.

CNNs were inspired originally by models of the visual cortex proposed in neuroscience. In those models, the **receptive field** of a neuron is the portion of the sensory input that can affect that neuron's activation.

```python
# Example of 1D convolution with kernel size 3 and stride 2

a = np.array([[+1,-1,+1,0,0,0,0],[0,0,+1,-1,+1,0,0], [0,0,0,0,+1,-1,+1]])

b = np.array([[5,6,6,2,5,6,5]])
c = np.matmul(a,b.T)
print(a)
print(b.T)
print('=')
print(c)

```

    [[ 1 -1  1  0  0  0  0]
     [ 0  0  1 -1  1  0  0]
     [ 0  0  0  0  1 -1  1]]
    [[5]
     [6]
     [6]
     [2]
     [5]
     [6]
     [5]]
    =
    [[5]
     [9]
     [4]]

## Pooling layer

A **pooling** layer in a neural network summarizes a set of adjacent units from the preceding layer with a single value. **Average-pooling** computes the average value of its $l$ inputs. Average pooling facilitates multiscale recognition. It also reduces the number of weights required in subsequent layers, leading to lower computational cost and prossibly faster learning. **Max pooling** computes the maximum value of its $l$ inputs. Max-pooling acts as a kind of logical disjunction, saying that a feature exists somewhere in the unit's receptive field.

In a image classification network the final layer of the network will be a softmax with $c$ output units. The early layers of the CNN are image-sized, so somewhere in between there must be significant reductions in layer size. Convolutional layers and pooling layers with stride larger than 1 all serve to reduce a layer size.

### Tensors

Multi-dimensional arrays of any dimension - they keep track of the "shape" of the data as it progress through layers of the network. Describing the CNN in terms of tensors and tensor operators, a deep learning package can generate compiled code that is highly optimized for the underlying computational substrate.

Support we are traying on $256 \times 256$ RGB images with a minibatch size of $64$. The input will be a 4-dimensional tensor of size $256 \times 256 \times 3 \times 64$. The we applyu 96 kernel of size $5 \times 5 \times 3$ with a stride of $2$ in both $x$ and $y$ dimensions. This gives as an output tensor of size
$128 \times 128 \times 96 \times 64$. Such a tensor is called a **feature map** - note no dedicated color
channels but color information has been incorporated if the learning algorithm finds it useful for the final
predictions of the network.

Graphical Processing units (GPUs) are specialized hardware for graphics operations that can be used
to perform tensor operations. Tensor processing units (TPUs) are specialized hardware for computing tensor
operations that optimize for speed and throughput rather than high numerical precision.

### Residual networks

## Learning Algorithms

Standard gradient descent with $\alpha$ learning rate. The loss L is defined with respect to the entire training set.

\begin{equation}
\mathbf{w} \leftarrow \mathbf{w} - \alpha \nabla_{w} L(\mathbf w)
\end{equation}

Stochastic Gradient Descent (SGD), the loss $L$ is defined with respect to a minibatch of $m$ examples chosen randomly at each step.

Empirical considerations:

1. Small-mini batch size helps escape small local minima and computational cost of each weight update is a small constant, independent of training set size.
2. The gradient contribution of each training example in the SGD minibatch can be computed independently, the minibatch size is often chosen so as to take maximum advantage of hardware parallelism in GPUs or TPUs.
3. Learning rate needs to decrease over time. Choosing the right schedule is usually a matter of trial and error.
4. Care must be taken to mitigate numerical instabilities that may arise due to overflow, underflow, and rounding error.

Process of learning stops when there are diminishing returns.

### Batch normalization

### Generalization

Approaches to improving generalization in deep learning include:

1. Choosing the right architecture, varying number of layers, connectivity, and types of nodes
2. Penalizing large weights
3. Randomly perturing the values passing through the network during training
4. Data augmentation

Deeper (and narrow) networks tend to do better than shallow and wide networks for the same number of weights.
Deep learning works well with high-dimensional data such as images, video, speech. They have to a large extent
replaced preprocessing approaches that extracted features that prevailed prior to 2010.

**Weight decay** encourages weights to become small in some ways enforcing **reguralization** i.e limiting the complexity of the model.

**Dropout**

At each step of training, dropout applies one step of back-propagation learning to a new version of the network that is created by deactivating a randomly chosen subset of the units.

1. Introduce noise that provides robustness
2. Approximation of large ensemble of thinned networks
3. Paying attention to all features of the example rather than focusing on just a few

Usually makes it harder to fit the training set, it is usually necessary to use a larger model and to train it for more iterations.

**Data augmentation**

Provide multiple version of the same input (for example for images: add noise, crop, rotate, scale) to increase the size of the training data and the robustness to the various transformations applied.

### Graduate student descent

Incremental exploratory work carried out by graduate students to figure out which architectures
work best for which problems.

## Recurrent neural networks

Recurrent neural networks (RNNs) are distinct from feedforward networks in that they allow cycles in the computational graph. Each cycle has a delay i.e units may take as input a value computed from their own output at an earlier steop in the computation. This allows the RNN to have internal state or **memory**.

## Unsupervised learning and transfer learning

Unsupervised learning:

1. Representation learning
2. Generative modeling

Joint model $P_w(\mathbf x,\mathbf z)$, where $\mathbf z$ is a set of latent, unobserved variables that represent the content of the data $\mathbf x$ in some way.

A learned probability model achieves both representation learning (it has constructed meaningful $z$ vectors from the raw $\mathbf x$ vectors) and generative modelingL if we integrate $\mathbf z$ out of $P_{w}(\mathbf x,\mathbf z)$ we obtain $P_{w}(\mathbf x)$.

1. Probabilistic PCA
2. Autoencoders
3. Deep autoregressive models
4. Generative adversarial networks
5. Unsupervised translation
6. Transfer learning and multitask learning

```python

```


```python
import numpy as np
from toolz.curried import pipe, curry, identity, merge_with, merge
from toolz.curried import map as map_
import matplotlib.pyplot as plt
import math

%matplotlib inline
```

# Cross-Entropy Loss Backpropagation

Let $f_{ij}$ be the final scores from the last final layer. $i$ is over the samples ($1$ to $N$) and $j$ is over the categories. The cross-entropy loss is given by,

$$L_D = \frac{\sum_i L_i}{N}$$

where $L_D$ is the total loss not including the regularization loss and $L_i$ is the loss for each sample given by,

$$L_i = -\log \left(p_{iy_i} \right)$$

where $p_{ik}$ are the probabilities for each category per sample and $y_i$ are the actual categories per sample. The $p_{ik}$ is given by,

$$p_{ik} = \frac{ e^{f_{ik}} }{ \sum_m e^{f_{im}}}$$

To calculate the backprop,

\begin{align}
\frac{\partial L_i}{ \partial f_{ij}} &= \frac{\partial}{ \partial f_{ij}} \left[ \log\left(p_{iy_i} \right)\right] \\
&=\frac{1}{p_{iy_i}} \frac{\partial}{ \partial f_{ij}} \left[ p_{iy_i}  \right] \\
&=\frac{1}{p_{iy_i} } \left[ p_{iy_i}\delta_{jy_i} - p_{iy_i} p_{ij} \right] \\
&=\delta_{jy_i} - p_{ij}
\end{align}

So,

$$ \frac{\partial L_D}{ \partial f_{ij}} = \frac{1}{N}\left(p_{ij} - \delta_{iy_i}\right)$$

```python
## Example from https://cs231n.github.io/optimization-2/

class Node:
    def __init__(self, *nodes):
        self.nodes = nodes
        self.loss = None

    def reset(self):
        self.loss = None
        map_(lambda x: x.reset(), self.nodes)

    def calc(self):
        if self.loss is None:
            self.loss = self.f_(*list(map_(lambda x: x.calc(), self.nodes)))
        return self.loss

    def backprop(self, dvalue=1.0):
        values = list(map_(lambda x: x.calc(), self.nodes))
        dvalues = self.df_(dvalue, *values)
        list(map_(lambda x: x[0].backprop(x[1]), zip(self.nodes, dvalues)))

    def get_dvalues(self):
        return merge(list(map_(lambda x: x.get_dvalues(), self.nodes)))

    def df_(self, dvalue, *values):
        return list(map_(lambda x: x * dvalue, self.df(*values)))

    def f_(self, *values):
        return self.f(*values)

    def __mul__(self, other):
        return Mul(self, other)

    def __add__(self, other):
        return Add(self, other)

class Leaf(Node):
    def __init__(self, value, name):
        super().__init__()
        self.f = lambda: value
        self.name = name
        self.dvalue = 0.0

    def get_dvalues(self):
        return {self: self.dvalue}

    def backprop(self, dvalue=1.0):
        self.dvalue += dvalue

    def __repr__(self):
        return f'Leaf({self.name})'

    def reset(self):
        self.dvalue = 0.0
        self.loss = None

class Add(Node):
    f = staticmethod(lambda a, b: a + b)
    df = staticmethod(lambda a, b: (1, 1))

class Div(Node):
    f = staticmethod(lambda a, b: a / b)
    df = staticmethod(lambda a, b: (b, -a / b**2))

class Mul(Node):
    f = staticmethod(lambda a, b: a * b)
    df = staticmethod(lambda a, b: (b, a))

class Sig(Node):
    f = staticmethod(lambda a: 1 / (1 + math.exp(-a)))
    df = staticmethod(lambda a: (Sig.f(a)**2 * math.exp(-a),))

x0 = Leaf(-1, 'x0')
x1 = Leaf(-2, 'x1')
w0 = Leaf(2, 'w0')
w1 = Leaf(-3, 'w1')
w2 = Leaf(-3, 'w2')

out = Sig(w0 * x0 + w1 * x1 + w2 + w0 * x0)
out.reset()
print(out.calc())
print(out.backprop())
print(out.get_dvalues())

```

    0.2689414213699951
    None
    {Leaf(w0): -0.3932238664829636, Leaf(x0): 0.7864477329659272, Leaf(w1): -0.3932238664829636, Leaf(x1): -0.5898357997244454, Leaf(w2): 0.1966119332414818}

```python
# implement https://cs231n.github.io/neural-networks-case-study/

class Weight(Leaf):
    def __init__(self, N0, N1, name):
        super().__init__(0.01 * np.random.randn(N0, N1), name)

class B(Leaf):
    def __init__(self, N, name):
        super().__init__(0.01 * np.zeros((1, N)), name)

class NN(Node):
    @staticmethod
    def f(x, w, b):
        return np.dot(x, w) + b

    def df_(self, dvalue, x, w, b):
        return (
            np.dot(dvalue, w.T),
            np.dot(x.T, dvalue),
            np.sum(dvalue, axis=0, keepdims=True)
        )

class Loss(Node):
    def __init__(self, y, *nodes):
        self.y = y
        self.reg = 1e-3
        super().__init__(*nodes)

    def f_(self, scores, w):
        N = scores.shape[0]
        exp_scores = np.exp(scores)
        self.probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
        correct_logprobs = -np.log(self.probs[range(N), self.y])
        return np.sum(correct_logprobs) / N + 0.5 * self.reg * np.sum(w * w)

    def df_(self, dvalue, scores, w):
        self.calc()
        N = self.y.shape[0]
        delta = np.zeros_like(self.probs)
        delta[range(N), self.y] = 1.0
        return (
            dvalue * (self.probs - delta) / N,
            dvalue * self.reg * w
        )

Nsample = 10

Nfeature = 4
Nclass = 3

y = np.random.randint(0, Nclass, Nsample)
X = Leaf(np.zeros((Nsample, Nfeature)), 'X')
W = Weight(Nfeature, Nclass, 'W')
b = B(Nclass, 'b')

print(Loss(y, NN(X, W, b), W).calc())
net = Loss(y, NN(X, W, b), W)
net.calc()
net.reset()
net.backprop()
print(net.get_dvalues())

```

    1.0986131449120995
    {Leaf(X): array([[-1.01610256e-04, -5.85234330e-04,  8.43932694e-04,
            -1.76053207e-04],
           [-1.01610256e-04, -5.85234330e-04,  8.43932694e-04,
            -1.76053207e-04],
           [ 5.38159528e-04, -6.32784578e-04, -8.18745209e-04,
             5.10521293e-05],
           [-4.36549272e-04,  1.21801891e-03, -2.51874843e-05,
             1.25001078e-04],
           [-4.36549272e-04,  1.21801891e-03, -2.51874843e-05,
             1.25001078e-04],
           [-1.01610256e-04, -5.85234330e-04,  8.43932694e-04,
            -1.76053207e-04],
           [-1.01610256e-04, -5.85234330e-04,  8.43932694e-04,
            -1.76053207e-04],
           [ 5.38159528e-04, -6.32784578e-04, -8.18745209e-04,
             5.10521293e-05],
           [-1.01610256e-04, -5.85234330e-04,  8.43932694e-04,
            -1.76053207e-04],
           [-4.36549272e-04,  1.21801891e-03, -2.51874843e-05,
             1.25001078e-04]]), Leaf(W): array([[ 1.79033195e-05,  2.12527097e-05,  1.15056217e-05],
           [ 2.94872037e-06, -1.50838120e-05,  3.42422285e-06],
           [ 3.34596425e-06,  1.20371660e-05,  1.99727433e-05],
           [ 1.87388100e-06, -1.13666184e-06, -3.97172363e-07]]), Leaf(b): array([[-0.16666667,  0.03333333,  0.13333333]])}

```python
def get_data():
    N = 100 # number of points per class
    D = 2 # dimensionality
    K = 3 # number of classes
    X = np.zeros((N*K,D)) # data matrix (each row = single example)
    y = np.zeros(N*K, dtype='uint8') # class labels
    for j in range(K):
      ix = range(N*j,N*(j+1))
      r = np.linspace(0.0,1,N) # radius
      t = np.linspace(j*4,(j+1)*4,N) + np.random.randn(N)*0.2 # theta
      X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]
      y[ix] = j
    return X, y

Xdata, ydata = get_data()
print(Xdata.shape)
print(ydata.shape)
plt.scatter(Xdata[:, 0], Xdata[:, 1], c=ydata, s=40, cmap=plt.cm.Spectral)
plt.show()

```

```python
fnetwork = Loss(ydata, NN(Xdata, Wdata, b), W)

```


```python
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

```python
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
from numpy import *
from matplotlib.patches import FancyArrowPatch
from mpl_toolkits.mplot3d import proj3d
font = {'size': 14}
matplotlib.rc('font', **font)
```

```python
class Arrow3D(FancyArrowPatch):
    def __init__(self, xs, ys, zs, *args, **kwargs):
        FancyArrowPatch.__init__(self, (0,0), (0,0), *args, **kwargs)
        self._verts3d = xs, ys, zs
    def draw(self, renderer):
        xs3d, ys3d, zs3d = self._verts3d
        xs, ys, zs = proj3d.proj_transform(xs3d, ys3d, zs3d, renderer.M)
        self.set_positions((xs[0],ys[0]),(xs[1],ys[1]))
        FancyArrowPatch.draw(self, renderer)
```

```python
from sympy import symbols, Matrix
from sympy.parsing.sympy_parser import parse_expr
```

```python
vec2 = parse_expr('Matrix([1, 1])')
vec3 = parse_expr('Matrix([1, 1, 1])')
vec2, vec3
```

    (Matrix([
     [1],
     [1]]), Matrix([
     [1],
     [1],
     [1]]))

```python
V = np.array([vec2[0].evalf(), vec2[1].evalf()]).astype('float')
V
```

    array([1., 1.])

```python
U = np.array([vec3[0].evalf(), vec3[1].evalf(), vec3[2].evalf()]).astype('float')
U
```

    array([1., 1., 1.])

```python
plt.figure(figsize=(6,5))

magn = np.sqrt(V[0]**2 + V[1]**2)

plt.xlim([-2,2])
plt.ylim([-2,2])

plt.axis('equal')
X = [np.cos(x) for x in np.linspace(0, 2*np.pi, 64)]
Y = [np.sin(y) for y in np.linspace(0, 2*np.pi, 64)]

plot(X,Y, c='darkgray', lw=3)

annotate("", xy=(V[0]/magn, V[1]/magn), xytext=(0, 0),arrowprops=dict(arrowstyle="->", color="purple", lw=3))
axhline(0, ls='dashed', alpha=0.33, c='gray', lw=3)
axvline(0, ls='dashed', alpha=0.33, c='gray', lw=3)
```

```python

```

Applied Math/Y2S2/.ipynb_checkpoints/vector plot 2d and 3d-checkpoint.ipynb


## First Assignment

#### 1) Apply the appropriate string methods to the **x** variable (as '.upper') to change it exactly to: "$Dichlorodiphenyltrichloroethane$".

```python
x = "DiClOrod  IFeNi  lTRicLOr oETaNo  DiChlorod iPHeny lTrichL oroEThaNe"
```

```python
x[0]+x[1:].lower().replace(" ", "")[27:]
```

    'Dichlorodiphenyltrichloroethane'

#### 2) Assign respectively the values: 'word', 15, 3.14 and 'list' to variables A, B, C and D in a single line of code. Then, print them in that same order on a single line separated by a space, using only one print statement.

```python
A, B, C, D = 'word', 15, 3.14, 'list' ;  print(A, B, C, D)
```

    word 15 3.14 list

#### 3) Use the **input()** function to receive an input in the form **'68.4 1.71'**, that is, two floating point numbers in a line separated by space. Then, assign these numbers to the variables **w** and **h** respectively, which represent an individual's weight and height (hint: take a look at the '.split()' method). With this data, calculate the individual's Body Mass Index (BMI) from the following relationship:

\begin{equation}
BMI = \dfrac{weight}{height^2}
\end{equation}

```python
w, h = input("Adam's weight and height:").split(); bmi = (float(w)/float(h) ** 2); print(bmi)
```

    Adam's weight and height:68.4 1.71
    23.39181286549708

#### This value can also be classified according to ranges of values, following to the table below. Use conditional structures to classify and print the classification assigned to the individual.

<\center>

(source: https://healthtravelguide.com/bmi-calculator/)

```python
bmi = 23.39181286549708

if bmi < 18.5 :
  print ("Underweight")
elif 18.5 <= bmi < 24.9:
    print ("Normal weight")
elif 25.0 <= bmi < 29.9:
    print ("Pre-obesity")
elif 30.0 <= bmi < 34.9:
    print ("Obesity class I")
elif 35.0 <= bmi < 39.9:
    print ("Obesity class II")
elif 40 <= bmi :
      print ("Obesity class III")

```

    Normal weight

#### 4) Receive an integer as an input and, using a loop, calculate the factorial of this number, that is, the product of all the integers from one to the number provided.

```python
num=int(input("Enter a number to find factorial: "))

factorial=1

if num<0:
   print("Factorial does not defined for negative integer");
elif(num==0):
    print("The factorial of 0 is 1");
else:
  while(num>0):
        factorial=factorial*num

        num=num-1

  print("factorial of the given number is: ")
  print(factorial)
```

    Enter a number to find factorial: 5
    factorial of the given number is:
    120

#### 5) Using a while loop and the input function, read an indefinite number of integers until the number read is -1. Present the sum of all these numbers in the form of a print, excluding the -1 read at the end.

```python
x=int(input("Enter a number: "))

while x != -1 :
  print(x, "is bigger than -1")
  if x > -1 :
    x = x -1
  else :
    x = x + 1
```

    Enter a number: 5
    5 is bigger than -1
    4 is bigger than -1
    3 is bigger than -1
    2 is bigger than -1
    1 is bigger than -1
    0 is bigger than -1

#### 6) Read the **first name** of an employee, his **amount of hours worked** and the **salary per hour** in a single line separated by commas. Next, calculate the **total salary** for this employee and show it to two decimal places.

```python
def weeklyPaid(employee, hours_worked, wage_per_hour):
    if hours_worked > 40:
        return 40 * wage_per_hour + (hours_worked - 40) * wage_per_hour * 1.5
    else:
        return hours_worked * wage_per_hour

employee = "Matilde"
hours_worked = 50
wage_per_hour = 120

pay = weeklyPaid(employee, hours_worked, wage_per_hour)

print(f"Total gross pay of {employee} : EUR {pay:.2f} ")
```

    Total gross pay of Matilde : EUR 6600.00

#### 7) Read three floating point values **A**, **B** and **C** respectively. Then calculate itens a, b, c, d and e:

```python
A, B, C = 1.1, 2.2, 3.3
```

    a) the area of the triangle rectangle with A as the base and C as the height.

```python
base = int(A)
height = int(C)
Area = (base * height) / 2

print("The area of the triangle rectangle =", Area)
```

    The area of the triangle rectangle = 1.5

    b) the area of the circle of radius C. (pi = 3.14159)

```python
radius = int(C)
pi = 3.14159
Area = pi * radius * radius
print("Area of the circle = " + str(Area))
```

    Area of the circle = 28.274309999999996

    c) the area of the trapezoid that has A and B for bases and C for height.

```python
base_1 = int(A)
base_2 = int(B)
height = int(C)
Area = (base_1+base_2)*height/2
print("Area of the trapezoid = " + str(Area))
```

    Area of the trapezoid = 4.5

    d) the area of the square that has side B.

```python
side = int(B)
Area = side ** 2
print("Area of the square = " + str(Area))
```

    Area of the square = 4

    e) the area of the rectangle that has sides A and B.

```python
width = int(A)
height = int(B)
Area = width * height
print("The area of the rectangle = " + str(Area))
```

    The area of the rectangle = 2

#### 8) Read **the values a, b and c** and calculate the **roots of the second degree equation** $ax^{2}+bx+c=0$ using [this formula](https://en.wikipedia.org/wiki/Quadratic_equation). If it is not possible to calculate the roots, display the message **“There are no real roots”**.

```python
a, b, c = 0.23, 1.45, 3.77

r = b**2 - 4*a*c

if r > 0:
    num_roots = 2
    x1 = (((-b) + sqrt(r))/(2*a))
    x2 = (((-b) - sqrt(r))/(2*a))
    print("There are 2 roots: %f and %f" % (x1, x2))
elif r == 0:
    num_roots = 1
    x = (-b) / 2*a
    print("There is one root: ", x)
else:
    num_roots = 0
    print("There are no real roots.")

```

    There are no real roots.

#### 9) Read four floating point numerical values corresponding to the coordinates of two geographical coordinates in the cartesian plane. Each point will come in a line with its coordinates separated by space. Then calculate and show the distance between these two points.

(obs: $d=\sqrt{(x_1-x_2)^2 + (y_1-y_2)^2}$)

```python
x1, x2, y1, y2 = 1.11, 2.32, 4.56, 5.23

result= ((((x2 - x1 )**2) + ((y2-y1)**2) )**0.5)

print("distance(d) between",(x1,x2),"and",(y1,y2),"is : ",result)
```

    distance(d) between (1.11, 2.32) and (4.56, 5.23) is :  1.3831124321616086

#### 10) Read **two floating point numbers** on a line that represent **coordinates of a cartesian point**. With this, use **conditional structures** to determine if you are at the origin, printing the message **'origin'**; in one of the axes, printing **'x axis'** or **'y axis'**; or in one of the four quadrants, printing **'q1'**, **'q2**', **'q3'** or **'q4'**.

```python
def quadrant(x, y):
	if (x > 0 and y > 0):
		print ("lies in First quadrant")
	elif (x < 0 and y > 0):
		print ("lies in Second quadrant")
	elif (x < 0 and y < 0):
		print ("lies in Third quadrant")
	elif (x > 0 and y < 0):
		print ("lies in Fourth quadrant")
	elif (x == 0 and y > 0):
		print ("lies at positive y axis")
	elif (x == 0 and y < 0):
		print ("lies at negative y axis")
	elif (y == 0 and x < 0):
		print ("lies at negative x axis")
	elif (y == 0 and x > 0):
		print ("lies at positive x axis")
	else:
		print ("lies at origin")

x = 1
y = 1
quadrant(x, y)
```

    lies in First quadrant

#### 11) Read an integer that represents a phone code for international dialing.
#### Then, inform to which country the code belongs to, considering the generated table below:
(You just need to consider the first 10 entries)

```python
import pandas as pd
df = pd.read_html('https://en.wikipedia.org/wiki/Telephone_numbers_in_Europe')[1]
df = df.iloc[:,:2]
df.head(20)
```

      Country
      Country calling code

      0
      Austria
      43

      1
      Belgium
      32

      2
      Bulgaria
      359

      3
      Croatia
      385

      4
      Cyprus
      357

      5
      Czech Republic
      420

      6
      Denmark
      45

      7
      Estonia
      372

      8
      Finland
      358

      9
      France
      33

      10
      Germany
      49

      11
      Greece
      30

      12
      Hungary
      36

      13
      Iceland
      354

      14
      Ireland
      353

      15
      Italy
      39

      16
      Latvia
      371

      17
      Liechtenstein
      423

      18
      Lithuania
      370

      19
      Luxembourg
      352

```python
Country_calling_code = int(input('Enter a number: '))
CountryCall_dict = {43:'Austria',
             32:'Belgium',
             359:'Bulgaria',
             385:'Croatia',
             357:'Cyprus',
             420:'Czech Republic',
             45:'Denmark',
             372:'Estonia',
             359:'Finland',
             33:'France',
             49: 'Germany'}

if Country_calling_code in CountryCall_dict.keys():
    print(CountryCall_dict[Country_calling_code])
else:
    print('No result.')
```

    Enter a number: 43
    Austria

#### 12) Write a piece of code that reads 6 numbers in a row. Next, show the number of positive values entered. On the next line, print the average of the values to one decimal place.

```python
import statistics

data1 = [1, 3, 4, 5, 7, 9]

x = statistics.mean(data1)
print(f"Mean is: {x:.1f}")
```

    Mean is: 4.8

#### 13) Read an integer **N**. Then print the **square of each of the even values**, from 1 to N, including N, if applicable, arranged one per line.

```python
def printSquares(n):

    square = 0; prev_x = 0;
    for x in range(1, n+1):
        square = (square + x + prev_x)
        print(square, end = " ")
        prev_x = x

n = 6;
printSquares(n)
```

    1 4 9 16 25 36

#### 14) Using **input()**, read an integer and print its classification as **'even / odd'** and **'positive / negative'** . The two classes for the number must be printed on the same line separated by a space. In the case of zero, print only **'null'**.

```python
num = int(input("Enter a number: "))
if num >= 0:
   if num == 0:
       print("Zero")
   else:
       print("Positive number")
else:
   print("Negative number")

if (num % 2) == 0:
   print("{0} is Even".format(num))
else:
   print("{0} is Odd".format(num))
```

    Enter a number: 10
    Positive number
    10 is Even

## Challenge
#### 15) Ordering problems are recurrent in the history of programming. Over time, several algorithms have been developed to fulfill this function. The simplest of these algorithms is the [**Bubble Sort**](https://en.wikipedia.org/wiki/Bubble_sort), which is based on comparisons of elements two by two in a loop of passes through the elements. Your mission, if you decide to accept it, will be to input six whole numbers ramdonly ordered. Then implement the **Bubble Sort** principle to order these six numbers **using only loops and conditionals**.
#### At the end, print the six numbers in ascending order on a single line separated by spaces.

```python

```


# CLOSED LOOP SYSTEMS

This notebook is a companion to the slides for understanding closed loop systems.

# Preliminaries

```python
!pip install -q control
!pip install -q tellurium
!pip install -q controlSBML

import control
import controlSBML as ctl
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sympy
import tellurium as te
```

```python
TIMES = ctl.makeSimulationTimes(0, 5, 500)
```

# Helpers

```python
def plotStep(tf, is_plot=True):
    """
    Plots a transfer function step response

    Parameters
    ----------
    tf: control.TransferFunction
    """
    result = control.forced_response(tf, T=TIMES, U=1)
    if is_plot:
        plt.plot(result.t, result.outputs)

# TESTS
Gs = control.TransferFunction([2], [1, 3])
plotStep(Gs, is_plot=False)
print("OK!")
```

    OK!

# Antimony Model

$\$S_1 \xrightarrow{k_1 S_1} S_2$,
$S_2 \xrightarrow{k_2 S_3} S_3$,
$S_3 \xrightarrow{k_3 S_3} S_2$,
$S_2 \xrightarrow{k_2 S_3} \emptyset$,

```python
# Constants
CONSTANT_DCT = {"k1": 1, "k2": 2, "k3": 3, "k4": 4}
s = sympy.Symbol("s")
REF = 10
```

```python
MODEL = """
$S1 -> S2; k1*$S1
J1: S2 -> S3; k2*S2
J2: S3 -> S2; k3*S3
J3: S2 -> ; k4*S2

k1 = 0
k2 = 0
k3 = 0
k4 = 0
$S1 = 10
S2 = 0
S3 = 0
S4 = 0
"""
RR = te.loada(MODEL)
for k in CONSTANT_DCT.keys():
    RR[k] = CONSTANT_DCT[k]
RR.simulate()
RR.plot()
```

We are going to control ``S3`` by changing ``S2``. That is, $U(s) = S_2  (s)$
and $Y(s) = S_3 (s)$.

# Simulating the Closed Loop System

```python
ctlsb = ctl.ControlSBML(RR,  input_names=["S2"], output_names=["S3"])
```

```python
sys = ctlsb.makeNonlinearIOSystem("sys")
```

```python
ctlsb.makeStateSpace()
```

\[
\left(\begin{array}{rll|rll}
-3\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&2\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}\\
\hline
1\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&0\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}\\
\end{array}\right)
\]

```python
def controller_outfcn(_, __, u, param_dct):
    k = param_dct.get("k", 2)
    ref = param_dct.get("ref", REF)
    return k*([REF] - u[0])
#
controller = control.NonlinearIOSystem(
    None, controller_outfcn, inputs=['in'], outputs=['out'],
    name="controller")
```

```python
# Create the closed loop system
closed_loop = control.interconnect(
  [sys, controller],       # systems
  connections=[
    ['sys.S2', 'controller.out'],
    ['controller.in',  'sys.S3'],
  ],
  inplist=["controller.in"],
  outlist=["sys.S2", "sys.S3"],
)
```

```python
param_dct = {"k": 5, "ref": REF}
result = control.input_output_response(closed_loop, T=TIMES, params=param_dct)
plt.plot(TIMES, result.outputs[0], TIMES, result.outputs[1])
plt.legend(["S2", "S3"])
```

```python
result.outputs[1][-2:]
```

    array([7.68988725, 7.68994905])

```python
type(control.feedback(controller, sys))
```

    control.iosys.InterconnectedSystem

The output is biased! We wanted $S_3$ to be 10, but it's 7.6.

# Analyzing the Cause of Bias

Using direct calculation, I have calculated the transfer function $G(s) = \frac{k_2}{s + k_3}$.

```python
numerator = CONSTANT_DCT["k2"]
denominator = [1, CONSTANT_DCT["k3"]]
tf = control.TransferFunction(numerator, denominator)
tf
```

$$\frac{2}{s + 3}$$

```python
G = 2/(s + 3)
G
```

$\displaystyle \frac{2}{s + 3}$

```python
C = sympy.symbols("k")
```

We calculate the transfer function for the closed loop system, $H(s) = \frac{Y(s)}{R(s)}$.

# Constructing Transfer Functions

In the following, we use $G(s) = \frac{2}{s + 3}$ and $C(s) = k$ (proportional control).

```python
Gs = control.TransferFunction([2], [1, 3])
Gs
```

$$\frac{2}{s + 3}$$

```python
plotStep(Gs)
```

The original system, $G(s)$, settles around time 1.

```python
k = 5
Cs = control.TransferFunction([k], [1])
Cs.dcgain()
```

    5.0

For the filter, we use a low pass filter.

$F(s) = \frac{k_f}{s + k_f}$. Note that the DC Gain of $F(s)$ is 1.

```python
kf = 5
Fs = control.TransferFunction([kf], [1, kf])
Fs.dcgain()
```

    1.0

We use the above transfer functions to construct the various closed loop transfer functions.
Note that you can multiply and divide transfer function objects.

```python
HRYs = control.minreal(Cs*Gs/(1 + Cs*Gs))
HRYs
```

    1 states have been removed from the model

$$\frac{10}{s + 13}$$

```python
HRYs.pole()
```

    array([-13.])

```python
HRYs.dcgain()
```

    0.7692307692307693

```python
plotStep(HRYs)
```

The closed loop system settles around time 0.5.

Now we include the filter

```python
HRYs = control.minreal(Cs*Gs/(1 + Cs*Gs*Fs))
HRYs
```

    1 states have been removed from the model

$$\frac{10 s + 50}{s^2 + 8 s + 65}$$

```python
HRYs.dcgain()
```

    0.7692307692307698

```python
plotStep(HRYs)
```

```python
HREs = control.minreal(1/(1 + Cs*Gs*Fs))
HREs
```

    0 states have been removed from the model

$$\frac{s^2 + 8 s + 15}{s^2 + 8 s + 65}$$

We moved the pole of the original system from -3 to -13.