# Checkpoint 1

### Read This First
**1. Use the constants provided in the cell below. Do not use your own constants.**

**2. Put the code that produces the output for a given task in the specific cell indicated. You are welcome to add as many cells as you like for imports, function definitions, variables, etc. Additional cells need to be in the proper order such that your code runs the first time through.**

The Coulomb law is given by:

$
\Large
\begin{align}
F(r) = -\frac{e^{2}}{4 \pi \epsilon_{0} r^{2}} \left( \frac{r}{r_{0}} \right)^{\alpha},
\end{align}
$

where $r_{0}$ is the Bhor radius, given by:

$
\Large
\begin{align}
r_{0} = \frac{4 \pi \epsilon_{0} \hbar^{2}}{m e^{2}}.
\end{align}
$

The electric potential is given by:

$
\Large
\begin{align}
V(r) = \int_{r}^{\infty} F(r^{\prime}) dr^{\prime}
\end{align}
$

Use the following constants:
* $\frac{\hbar^{2}}{2m} = 0.0380998\ nm^{2} eV$ (called `c1` below)
* $\frac{e^{2}}{4 \pi \epsilon_{0}} = 1.43996\ nm\ eV$ (called `c2` below)
* $r_{0} = 0.0529177\ nm$ (called `r0` below)
* Planck constant $h = 6.62606896\times10^{-34} J s$ (`h`)
* Speed of light $c = 299792458\ m/s$ (`c`)

```python
# add imports here
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from scipy.integrate import quad
from scipy.linalg import eigvalsh_tridiagonal
from scipy.stats import linregress
```

```python
plt.rcParams['figure.figsize'] = (10, 6)
plt.rcParams['font.size'] = 14
```

```python
# Constants (use these)
c1 = 0.0380998 # nm^2 eV
c2 = 1.43996 # nm eV
r0 = 0.0529177 # nm
h  = 6.62606896e-34 # J s
c  = 299792458. # m/s
hc = 1239.8419 # eV nm
```

## Task 1

Write a code that calculates $V(r)$ numerically for $\alpha = 0.01$ and plots it for $r$ = 0.01...1 nm. Remember to label the axes.

```python
def get_force(r, alpha=0):
    """
    Method: calculates the electrostatic force between a proton and
    an electron with charge ±e respectively

    :param r:  radial distance
    :param alpha:  perturbation
    :return force: value for force as a float
    """
    force = -c2 * 1 / (r)**2 * (r / r0)**alpha
    return force

def potential_numerical(r, alpha=0):
    """
    Method: calculates the electrostatic potential for a pairwise
    system through the integration of the force between the pair

    :param radii:  numPy array of radial distances
    :param alpha:  perturbation
    :return potential: numPy array of potential values
    """
    potential = quad(get_force, r, np.inf, args=(alpha))[0]
    return potential
```

```python
N = 13000  # Number of integration steps
radii = np.linspace(0.01, 1.5, num=N)  # Defining radial distances (rmax = 1.5nm)
# rmax = 1.5nm gets good contribution of the energy from BOTH n=1 and n=2 energy levels
potential = np.array([]) # Forming potential array
for i in range(radii.size):
    potential = np.append(potential, potential_numerical(radii[i], alpha=0)) # Getting potential

# Plotting electrostatic potential vs radial distance for H atom
plt.title("Electrostatic Potential vs. Radial Distance")
plt.xlabel("Radial Distance (nm)")
plt.xlim(right=1.0)
plt.ylabel("Potential (eV)")
plt.plot(radii, potential)
plt.show()
```

## Task 2

In addition to (1), the test below will compare the analytic expression for $𝑉(r)$ with the numerically obtained values for $r$ = 0.01,0.02...1 nm. The biggest absolute difference $diff = max |V_{exact}(r) − V_{numerical}(r)|$ must be smaller than 10$^{−5}$ eV. There is nothing else for you to do.

```python
# We will call your function for one value of r and alpha = 0.01. There will be more tests!
potential_numerical(0.5, 0.01)
```

    -2.975081858428647

## Task 3

In addition to (2), calculate the first 2 energy levels (eigenvalues of $H$) for $\alpha = 0, 0.01$ and print out the values in eV. The values must be accurate to 0.01 eV. This requires sufficiently large $r_{max}$ and $N$. Plot the difference $\Delta E$ between the two energies for $\alpha = 0, 0.01$. Remember to label the axes.

```python
def energy_levels(alpha):
    """
    Method: determines the Hamiltonian using sparse matrices
    where H = -hbar**2/2m * (d/dr)**2 + V(r)

    :param radii: numPy array of radial distances
    :param dr: step
    :return E1,E2: the two lowest eigenvalues
    """
    # Constructing Hamiltonian
    dr_array = np.arange(dr, radii.size * dr + dr, dr)
    pot_diag = np.array([])
    for i in range(dr_array.size):
        pot_diag = np.append(pot_diag, potential_numerical(dr_array[i], alpha))

    H_diag = (- c1 / (dr)**2) * np.full(radii.size, -2) + pot_diag
    off_diag =(- c1 / (dr)**2)   * np.full(radii.size - 1, 1)

    # Finding eigenvalues of tridiagonal Hamiltonian matrix
    eigvals = eigvalsh_tridiagonal(H_diag, off_diag, select='i', select_range=(0,1))
    np.sort(np.real(eigvals))
    E1, E2 = eigvals[0], eigvals[1]
    return E1, E2
```

```python
dr = max(radii)/N
delta_E = np.array([])
alpha_values = np.arange(0.0, 0.01, 0.001)

# Calculating delta E for different alpha values
for alpha in alpha_values:
    E1, E2 = energy_levels(alpha)
    delta_E = np.append(delta_E, E2-E1)

# Plotting delta_E vs. alpha
plt.title("En=2 - En=1 for the Hydrogen atom with a perturbed potential")
plt.xlabel("Alpha (Perturbation)")
plt.ylabel("Delta E (eV)")
plt.plot(alpha_values, delta_E)
plt.show()

```

```python
# Print out the energy levels for alpha = 0, 0.01.
e_levels_0 = energy_levels(0.0)
e_levels_0_01 = energy_levels(0.01)
print ("alpha = 0.00:", e_levels_0)
print ("alpha = 0.01:", e_levels_0_01)
```

    alpha = 0.00: (-13.605598566607691, -3.401402280845947)
    alpha = 0.01: (-13.807382891866881, -3.5346022779978856)

## Task 4

In addition to (3), assuming that the transition between the 1st excited and the ground state corresponds to the wavelength $\lambda = 121.5 \pm 0.1$ nm, what is the maximum value of $\alpha_{max} > 0$ consistent with this measurement (i.e., the largest $\alpha_{max} > 0$ such that the predicted and measured wavelengths differ by less than 0.1 nm)?

```python
def find_alpha_max():
    # Only need delta_E_max to find alpha_max
    # delta_E max corresponds to minimum wavelength between n=2 and n=1 levels
    lambda_min = 121.4
    delta_E_max = hc / lambda_min

    # Finding equation of line from plot above
    r = linregress(alpha_values, delta_E)
    # Finding corresponding alpha_max to delta_E_max from straight line
    alpha_max = (delta_E_max - r.intercept)/r.slope
    return alpha_max
```

```python
# Run the function and print alpha_max.
alpha_max = find_alpha_max()
print ("alpha_max:", alpha_max)
```

    alpha_max: 0.0012803032280778062

## Task 5

Improve the accuracy of the computation of the two energy levels to 0.001 eV and find $\alpha_{max}$ assuming the wavelength $\lambda = 121.503 \pm 0.01$ nm.

```python
### TASK 5
def energy_levels_improved(alpha):
    """
    Method: determines the Hamiltonian using sparse matrices
    where H = -hbar**2/2m * (d/dr)**2 + V(r)

    :param radii: numPy array of radial distances
    :param dr: step
    :return E1,E2: the two lowest eigenvalues
    """
    dr_array = np.arange(dr, radii.size * dr + dr, dr)
    pot_diag = np.array([])
    for i in range(dr_array.size):
        pot_diag = np.append(pot_diag, potential_numerical(dr_array[i], alpha))

    H_diag = (- c1 / (dr)**2) * np.full(radii.size, -2) + pot_diag
    off_diag =(- c1 / (dr)**2)   * np.full(radii.size - 1, 1)

    eigvals = eigvalsh_tridiagonal(H_diag, off_diag, select='i', select_range=(0,1))
    np.sort(np.real(eigvals))
    E1, E2 = eigvals[0], eigvals[1]
    return E1, E2

def find_alpha_max_improved():
    # Only need delta_E_max to find alpha_max
    # delta_E max corresponds to minimum wavelength between n=2 and n=1 levels
    # Using old accuray of ±0.001nm as I think the code is fast enough
    lambda_min = 121.502
    delta_E_max = hc / lambda_min

    # Finding equation of line from plot above
    r = linregress(alpha_values, delta_E)
    # Finding corresponding alpha_max to delta_E_max from straight line
    alpha_max = (delta_E_max - r.intercept)/r.slope
    return alpha_max
```

```python
# Run the function and print alpha_max.
e_levels_0_01 = energy_levels_improved(0.01)
print ("alpha = 0.01:", e_levels_0_01)
# Run the function and print alpha_max.
alpha_max_improved = find_alpha_max_improved()
print ("alpha_max:", alpha_max_improved)
```

    alpha = 0.01: (-13.807382891866881, -3.5346022779978856)
    alpha_max: 2.8704299245945143e-05

## Task 6

How would one achieve the same accuracy with significantly smaller matrices? Hint: can we represent $R$ from Eq. (1) as a linear combination of functions that solve the "unperturbed" equation, and translate this into an eigenproblem for a certain $N \times N$ matrix, with $N < 100$?

```python
### TASK 6
def energy_levels_best(alpha):
    # Remove the line that says "raise NotImplementedError"
    # YOUR CODE HERE
    raise NotImplementedError()
    return E1, E2

def find_alpha_max_best():
    # Remove the line that says "raise NotImplementedError"
    # YOUR CODE HERE
    raise NotImplementedError()
    return alpha_max
```

```python
# Run the function and print alpha_max.
e_levels_0_01 = energy_levels_best(0.01)
print ("alpha = 0.01:", e_levels_0_01)
# Run the function and print alpha_max.
alpha_max_best = find_alpha_max_best()
print ("alpha_max:", alpha_max_best)
```

```python

```


# Solutions

## Question 1

> `1`. Using recursion, obtain the first 10 terms of the following sequences:

> `1`. $\left\{\begin{array}{l}a_1 = 1,\\a_n = 3a_{n - 1}, n > 1\end{array}\right.$

```python
def get_sequence_a(n):
    """
    Return the sequence a.
    """
    if n == 1:
        return 1
    return 3 * get_sequence_a(n - 1)

[get_sequence_a(n) for n in range(1, 11)]
```

    [1, 3, 9, 27, 81, 243, 729, 2187, 6561, 19683]

> `2`. $\left\{\begin{array}{l}b_1 = 3,\\b_n = 6b_{n - 1}, n > 1\end{array}\right.$

```python
def get_sequence_b(n):
    """
    Return the sequence b.
    """
    if n == 1:
        return 1
    return 6 * get_sequence_b(n - 1)

[get_sequence_b(n) for n in range(1, 11)]
```

    [1, 6, 36, 216, 1296, 7776, 46656, 279936, 1679616, 10077696]

> `3`. $\left\{\begin{array}{l}c_1 = 3,\\c_n = 6c_{n - 1} + 3, n > 1\end{array}\right.$

```python
def get_sequence_c(n):
    """
    Return the sequence c.
    """
    if n == 1:
        return 3
    return 6 * get_sequence_c(n - 1) + 3

[get_sequence_c(n) for n in range(1, 11)]
```

    [3, 21, 129, 777, 4665, 27993, 167961, 1007769, 6046617, 36279705]

> `4`. $\left\{\begin{array}{l}d_0 = 3,\\d_n = \sqrt{d_{n - 1}} + 3, n > 0\end{array}\right.$

```python
import sympy as sym

def get_sequence_d(n):
    """
    Return the sequence c.
    """
    if n == 0:
        return 3
    return sym.sqrt(get_sequence_d(n - 1)) + 3

[get_sequence_d(n) for n in range(1, 11)]
```

    [sqrt(3) + 3,
     sqrt(sqrt(3) + 3) + 3,
     sqrt(sqrt(sqrt(3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3,
     sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(sqrt(3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3) + 3]

We could use a `sqrt` from a different library. Choosing to use `sympy` as it
ensures the result is exact although not necessarily readable. Here is an
approximate approach:

```python
import math

def get_sequence_d(n):
    """
    Return the sequence c.
    """
    if n == 0:
        return 3
    return math.sqrt(get_sequence_d(n - 1)) + 3

[get_sequence_d(n) for n in range(1, 11)]
```

    [4.732050807568877,
     5.175327747161075,
     5.2749346687676715,
     5.296722592906612,
     5.301460969233807,
     5.302490167022176,
     5.302713652850084,
     5.302762178960321,
     5.3027727154368325,
     5.302775003216084]

2. Using recursion, obtain the first 5 terms of the Fibonacci sequence:

    $$\left\{\begin{array}{l}
        a_0 = 0,\\
        a_1 = 1,\\
        a_n = a_{n - 1} + a_{n - 2}, n \geq 2\end{array}\right.
    $$

```python
def get_fibonacci(n):
    """
    Return the nth term of the Fibonacci sequence
    """
    if n == 0:
        return 0
    if n == 1:
        return 1
    return get_fibonacci(n - 1) + get_fibonacci(n - 2)

[get_fibonacci(n) for n in range(10)]
```

    [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

## Question 3

> `3`. A 40 year building programme for new houses began in Oldtown in the year
> 1951 (Year 1) and finished in 1990 (Year 40).

> The number of houses built each year form an arithmetic sequence with first
> term $a$ and common difference $d$.

> Given that 2400 new houses were built in 1960 and 600 new houses were built in
> 1990, find:

> `1`. The value of $d$.

An arithmetic sequence with first term $a$ and common difference $d$ is a
sequence of the form:

$$
  \left\{
    \begin{array}{l}
       a_1 = a,\\
       a_n = a_{n - 1} + d
    \end{array}
  \right.
$$

We will write a function to express this:

```python
def get_arithmetic_sequence(n, first_term, common_difference):
    """
    Return the nth term of an arithmetic sequence with give first_term and
    common common_difference.
    """
    if n == 1:
        return first_term
    return (
        get_arithmetic_sequence(n - 1, first_term, common_difference)
        + common_difference
    )
```

We know that $a_{10}=2400$ and $a_{40}=600$ we can write down equations that
represent this:

```python
a = sym.Symbol("a")
d = sym.Symbol("d")

a_10_equation = sym.Eq(
    get_arithmetic_sequence(n=10, first_term=a, common_difference=d), 2400
)
a_10_equation
```

$\displaystyle a + 9 d = 2400$

```python
a_40_equation = sym.Eq(
    get_arithmetic_sequence(n=40, first_term=a, common_difference=d), 600
)
a_40_equation
```

$\displaystyle a + 39 d = 600$

We will solve the first equation for $a$:

```python
sym.solveset(a_10_equation, a)
```

$\displaystyle \left\{2400 - 9 d\right\}$

We substitute this in to the other equation and solve it for $d$:

```python
sym.solveset(a_40_equation.subs({a: 2400 - 9 * d}), d)
```

$\displaystyle \left\{-60\right\}$

> `2`. The value of $a$.

We can substitute that value for $d$ back in to the expression for $a$:

```python
(2400 - 9 * d).subs({d: -60})
```

$\displaystyle 2940$

> `3`. The total number of houses built in Oldtown over 40 years.

```python
sum(
    get_arithmetic_sequence(n=n, first_term=2940, common_difference=-60)
    for n in range(1, 41)
)
```

    70800

## Question 4

> `4`. A sequence is given by:

> $$ \left\{\begin{array}{l}
        x_1 = 1\\
        x_{n + 1}= x_n(p + x_n), n > 1
        \end{array}\right.
    $$

> for $p\ne0$.

> `1`. Find $x_2$ in terms of $p$.

We start by defining a function:

```python
def get_sequence(n, p):
    """
    Return the nth term of the sequence x_n for a given value of p
    """
    if n == 1:
        return 1
    return get_sequence(n - 1, p) * (p + get_sequence(n - 1, p))
```

Using this we can answer the question:

```python
p = sym.Symbol("p")

x_2 = get_sequence(n=2, p=p)
x_2
```

$\displaystyle p + 1$

> `2`. Show that $x_3=1+3p+2p^2$.

```python
p = sym.Symbol("p")

x_3 = get_sequence(n=3, p=p)
sym.expand(x_3)
```

$\displaystyle 2 p^{2} + 3 p + 1$

> `3`. Given that $x_3=1$, find the value of $p$

```python
equation = sym.Eq(x_3, 1)
sym.solveset(equation, p)
```

$\displaystyle \left\{- \frac{3}{2}, 0\right\}$

As $p\ne0$ this gives us that $p=-\frac{3}{2}$.


# Integración numérica

Integrar, al contrario de derivar, no es una operación *algorítmica*, es decir, no existe una manera de, dada una función, calcular su integral definida en un intervalo.

Debido a esto, integrar analíticamente es mucho más dificil que derivar analíticamente. Más aún, existen funciones cuyas integrales, aunque existan, no se pueden expresar de manera simple.

Para atacar todos estos problemas, en análisis numérico se han desarrollado distintos métodos para calcular la integral de una función.

## Planteamiento del problema

Dada una función $f:[a,b] \to \mathbb{R}$ continua a pedazos (tiene a lo más un número numerable de discontinuidades), quiero calcular $\int_{a}^{b} f(s) ds$, que corresponde al área algebraica entre la función y el eje x

## ¿Cómo calculo una integral numéricamente?

Primero debemos recordar la definición de una integral sobre el intervalo $[a,b]$.

### Particiones regulares
Sea $P = \{x_0=a, x_1, \ldots, x_{n-1}, x_n = b\}$. Decimos que $P$ es una **partición regular** de $[a,b]$ si y solo si, para toda $i$ se cumple que

1. $x_{i} < x_{i+1}$

2.  $x_{i+1} - x_{i} = h = \frac{b-a}{n}$

En este ejemplo, la partición tiene $n+1$ puntos y $n$ subintervalos

### Sumas de Riemann

Primero la integral se aproxima mediante la suma de las áreas de varios rectángulos cuyas bases están sobre una partición regular del invervalo $[a,b]$

$$
\int_{a}^{b} f(s) ds \approx \sum_{i=1}^{n} f(\xi_i)(x_{i} - x_{i-1}) = h \sum_{i=1}^{n}  f(\xi_i)
$$

con $\xi_i \in [x_{i-1},x_{i}]$. Notemos que la altura del rectángulo, dada por $f(\xi_i)$, es arbitraria. Para tener la integral, simplemente tomamos el límite cuando el número de puntos de la partición tiende a infinito:

$$
\int_{a}^{b} f(s) ds = \lim_{n \to \infty} \sum_{i=1}^{n} f(\xi_i)(x_{i} - x_{i-1}) = \lim_{n \to \infty} h \sum_{i=1}^{n}  f(\xi_i)
$$

con $\xi_i \in [x_{i-1},x_{i}]$

### Sumas importantes

Si f es continua, sabemos que en cualquier intervalo cerrado alcanza su máximo y su mínimo. Así, podemos tomar como alturas de los rectángulos los valores máximos y mínimos de la función y así obtener:

#### Suma superior:

$\xi_i = \text{argmax}_{x\in[x_{i-1},x_i]}{f(x)}$, $f(\xi_i) = \max_{x \in [x_{i-1},x_i]}{f(x)}$

#### Suma inferior:

$\xi_i = \text{argmin}_{x\in[x_{i-1},x_i]}{f(x)}$, $f(\xi_i) = \min_{x \in [x_{i-1},x_i]}{f(x)}$

## ¿Podemos implementar una suma superior o inferior en la computadora?

Generalmente **NO**: como ya vimos, encontrar minimos y maximos no es sencillo y eso se tendría que hacer para cada subintervalo de la partición ($n$ veces), lo que haría la integración muy tardada.

## ¿Cómo calculamos entonces la integral?

Ya que no podemos hacer sumas superiores o inferiores, lo más sencillo es tomar $\xi_i$ de manera que sea fácil calcular la integral en la computadora:

Recordamos que $\xi_i \in [x_{i-1},x_i]$

### Primera opción: $\xi_i = x_{i}$ (Integral rectangular hacia adelante)

$$
\int_{a}^{b} f(s) ds \approx h \sum_{i=1}^{n}  f(\xi_i) =  h \sum_{i=1}^{n}  f(x_i)
$$

Tomar el punto $\xi_i = x_{i}$ corresponde a aproximar la integral como se muestra en la siguiente figura

## Preludio computacional: diferencia entre los indices de la partición y de la lista de Julia

Muchas veces, cuando planteamos un problema de manera matemática, la notación del problema puede incluir índices que nosotros también quisiéramos utilizar. Sin embargo, es importante recordar que **los índices en un objeto iterable siempre empiezan en 1**. Para el caso de la partición, se cumple que:

Partición: $xs = \{x_0, x_1, x_2 \ldots, x_{n-1},x_n\}$

Si queremos representar todos los puntos de la partición en un arreglo, debemos notar que tendrán la siguiente correspondencia

$xs[1] = x_0$

$xs[2] = x_1$

.
.
.

$xs[i] = x_{i-1}$

$xs[i+1] = x_{i}$

Es fundamental que dicha correspondencia se respete para no tener problemas entre lo teórico y lo computacional

```julia
function intAdelante(f,a,b,n)
    # hacer la particion regular con n+1 puntos, es decir, n subintervalos
    xs = range(a,stop = b, length = n+1)
    # hacer la suma de Riemann
    integral = 0
    for i in range(1,stop=n)
        # xs[i+1] es el x_i de la partición
        integral = integral + f(xs[i+1])
    end
    # multiplicamos por h = (b-a)/n
    integral = (b-a)/n * integral
    return integral
end
```

    intAdelante (generic function with 1 method)

Podemos probar nuestra implementación en la función $f(x) = 3x^2$

Analíticamente, sabemos que

$$
\int_0^2 3x^2 dx= x^3 \mid_0^2 = 2^3 - 0^3 = 8
$$

```julia
function prueba(x)
    return 3*x^2
end
```

    prueba (generic function with 1 method)

```julia
# con 10 subintervalos, o rectángulos, la aproximación es mala
println(intAdelante(prueba,0,2,10))
# tomando más mejora
println(intAdelante(prueba,0,2,100))
println(intAdelante(prueba,0,2,1000))
```

    9.24
    8.120399999999998
    8.012004000000006

```julia
using Plots
```

```julia
# seleccionamos ns espaciadas logaritmicamente
# aux guarda los exponentes de las ns
aux = range(1,stop=4,length=30)
println("exponentes: ")
println(collect(aux))
# ns va a guardar mis ns (los distintos numeros de subintervalos)
# todos los elementos de n deben ser enteros, por lo que debemos redondearlos hacia arriba con la función
# ceil (función techo) y luego convertirlos a enteros con la función Int64()
ns = [Int64(ceil(10^k)) for k in aux]
println("numeros de  subintervalos: ")
println(collect(ns))
ys = [abs(intAdelante(prueba,0,2,n)-8) for n in ns]
scatter(ns,ys,title="Error absoluto",label="Integral hacia adelante",xscale=:log10,yscale=:log10)
```

    exponentes:
    [1.0, 1.103448275862069, 1.206896551724138, 1.3103448275862069, 1.4137931034482758, 1.5172413793103448, 1.6206896551724137, 1.7241379310344827, 1.8275862068965518, 1.9310344827586208, 2.0344827586206895, 2.1379310344827585, 2.2413793103448274, 2.3448275862068964, 2.4482758620689653, 2.5517241379310347, 2.6551724137931036, 2.7586206896551726, 2.8620689655172415, 2.9655172413793105, 3.0689655172413794, 3.1724137931034484, 3.2758620689655173, 3.3793103448275863, 3.4827586206896552, 3.586206896551724, 3.689655172413793, 3.793103448275862, 3.896551724137931, 4.0]
    numeros de  subintervalos:
    [10, 13, 17, 21, 26, 33, 42, 53, 68, 86, 109, 138, 175, 222, 281, 357, 453, 574, 728, 924, 1173, 1488, 1888, 2396, 3040, 3857, 4894, 6211, 7881, 10000]

## Ejercicio 1

Resuelve las siguientes dos integrales utilizando la regla de integración hacia adelante:

1. $$\int_{0}^{1} \frac{\arctan{\sqrt{x^2+2}}}{\left(\sqrt{x^2+2}\right)(x^2+1)} \; dx = \frac{5\pi^2}{96}$$

2. $$\int_{0}^{\pi} \ln{\left(1 - 2 e \cos{x}+ e^2 \right) } \; dx = 2\pi$$

$e$ es el número de Euler.

Usando el resultado que se muestra, grafica el error absoluto como función del número de puntos de la partición $n$ y discute cuál es la tasa de convergencia del método al valor analítico.

## Preludio matemático: integrales como funciones

Aunque sabemos que la integral de una función es un número real, podemos definir una función a partir de una integral de la siguiente forma:

$$
F(x) = \int_{a}^{x} f(s) \; ds
$$

Así, podemos utilizar nuestra función para integrar numéricamente también para construir una función cuya definición esté dada por una integral. Por ejemplo:

$$
\ln{x} = \int_{1}^x \frac{1}{s} \; ds
$$

```julia
# definimos una función auxiliar para nuestro logaritmo integrado numéricamente
function inv(s)
    return 1/s
end
# calculamos el logaritmo como una integral
function lnIntegral(x,n)
    return intAdelante(inv,1,x,n)
end
```

    lnIntegral (generic function with 1 method)

```julia
print("Valor exacto: ")
println(log(exp(1)))
print("valor aproximado: ")
println(lnIntegral(exp(1),100))
```

    Valor exacto: 1.0
    valor aproximado: 0.9945904672391623

```julia
# dominio de la gráfica
xs = range(1,stop=10,length=100)
# subintervalos de la integral
n = 50
ys1 = [log(x) for x in xs]
ys2 = [lnIntegral(x,n) for x in xs]
plot(xs,ys1,label="Logaritmo original",title="Integrales como funciones")
plot!(xs,ys2,label="Definición con integral numérica")
```

## Ejercicio 2
La función $\text{erf}(x)$ se define de la siguiente forma
    \begin{equation}
        \text{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-t^2} dt
    \end{equation}
 Define una función `miErf(x,n)` que regresa el valor la función $\text{erf}(x)$ al calcular la integral utilizando la regla de integración rectangular hacia adelante con $n$ subintervalos.

## Otras opciones para definir integrales mediante sumas de Riemman:

### Segunda opción: $\xi_i = x_{i-1}$ (Integral rectangular hacia atras)

$$
\int_{a}^{b} f(s) ds \approx h \sum_{i=1}^{n}  f(\xi_i) =  h \sum_{i=1}^{n}  f(x_{i-1})
$$

## Ejercicio 3

Implementa una función `intAtrás(f,a,b,n)` regrese el valor la integral $\int_a^b f(s) \; ds$ calculado utilizando la fórmula de la integral rectangular hacia atrás con $n$ subintervalos. Prueba tu función integrando la función  `prueba`.

### Tercera opción: tomar el punto medio $\xi_i = \frac{x_{i-1} + x_{i}}{2}$ (Integral rectangular Centrada o regla rectangular)

$$
\int_{a}^{b} f(s) ds \approx h \sum_{i=1}^{n}  f(\xi_i) =  h \sum_{i=1}^{n}  f\left( \frac{x_{i-1} + x_{i}}{2} \right)
$$

## Ejercicio 4

Implementa una función `intCentrada(f,a,b,n)` regrese el valor la integral $\int_a^b f(s) \; ds$ calculado utilizando la fórmula de la regla centrada con $n$ subintervalos. Prueba tu función integrando la función  `prueba`.

## Ejercicio 5

Repite el ejercicio 1 pero ahora utilizando las reglas de integración del ejercicio 3 y 4. ¿la tasa de convergencia mejora con las nuevas reglas?

## Ejercicio 6

Repite el ejercicio 2 pero ahora utilizando las reglas de integración del ejercicio 3 y 4.

## Aproximación más compleja: regla trapezoidal

Notemos que todas las aproximaciones rectangulares de la integral, por su construcción a partir de sumas de Riemann, aproximaban a la función $f(x)$ en cada intervalo $[x_i,x_{i+1}]$ como una función constante, es decir, un **polinomio de grado 0**.

Una aproximación más compleja para la función en dicho intervalo sería aproximar a la función no como una constante si no como una recta (o un **polinomio de grado 1**) que pasa por los puntos $(x_i,f(x_i))$ y $(x_{i+1},f(x_{i+1}))$. Así, el area de la integral se aproxima como la siguiente figura

## Ejercicio 7

Deduce la fórmula para el valor de $\int_{x_{i}}^{x_{i+1}} f(s) \; ds$ cuando aproximamos la función como una recta. Es decir, da la fórmula para el área del trapecio en un intervalo $[x_i,x_{i+1}]$ de la figura anterior. La fórmula solo debe depender de $x_i,f(x_i), x_{i+1},f(x_{i+1})$ y $y$

**Sugerencia:** Las áreas siempre son valores positivos, por lo que debes de tratar por separado los casos $f(x_{i}) \geq f(x_{i+1})$ y   $f(x_{i}) \leq f(x_{i+1})$.

Aunque trates ambos casos por separado, la fórmula obtenida **debe de ser la misma para ambos**.

### Planteamiento de la regla

Podemos explotar el ejercicio para crear una nueva regla de integración. Recordando que, dada una partición regular de orden $n$, se cumple que

$$
\int_{a}^b f(s) \; ds = \sum_{i=0}^{n-1} \int_{x_{i}}^{x_{i+1}} f(s) \; ds
$$

Podemos entonces utilizar la fórmula de $\int_{x_{i}}^{x_{i+1}} f(s) \; ds$ que encontramos en el ejercicio anterior para calcular la integral en todo el intervalo $[a,b]$. A este método de cálculo de la integral se le llama la **regla trapezoidal**

## Ejercicio 8

Escribe una función `intTrapezoidal(f,a,b,n)` que regresa el valor de la integral $\int_{a}^b f(s) \; ds$ calculado utilizando la regla trapezoidal. Es decir, que para cada integral $\int_{x{i}}^{x_{i+1}} f(s) \; ds$ se calcula como el area de un trapecio.

## Ejercicio 9

Repite el ejercicio 1 utilizando ahora la regla de integración trapezoidal. ¿Mejora la tasa de convergencia?

## Aproximación avanzada: regla de Simpson

Una regla todavía más avanzada para integrar consiste en ahora aproximar la función $f(x)$ como un **polinomio de grado 2** en un subintervalo de la partición. Sin embargo, para aproximar dicha función por un polinomio, como vimos en la clase de interpolación, necesitamos conocer tres puntos por los que pasa la función.

Para todo lo que sigue, supondremos que el orden $n$ de la partición tomada para aproximar la integral es un número **par**. Recordando el notebook de interpolación, sabemos que el polinomio de grado 2 $P_i(x)$ que pasa por los puntos $(x_{i-1},f(x_{i-1}))$, $(x_i,f(x_i))$ y $(x_{i+1},f(x_{i+1}))$ esta dado por la ecuación

\begin{align}
P_i(x) &= f(x_{i-1}) \frac{(x-x_{i})(x-x_{i+1})}{(x_{i-1}-x_{i})(x_{i-1}-x_{i+1})} \\
 &+ f(x_{i}) \frac{(x-x_{i-1})(x-x_{i+1})}{(x_{i}-x_{i-1})(x_{i}-x_{i+1})} \\
 &+ f(x_{i+1}) \frac{(x-x_{i-1})(x-x_{i})}{(x_{i+1}-x_{i-1})(x_{i+1}-x_{i})} \\
\end{align}

Así, podemos entonces aproximar la integral en el subintervalo $[x_{i-1},x_{i+1}]$ como

$$
\int_{x_{i-1}}^{x_{i+1}} f(s) \; ds \approx \int_{x_{i-1}}^{x_{i+1}} P_i(s) \;ds
$$

### Ejercicio 10

Demuestra, realizando la integración analíticamente, que

$$
\int_{x_{i-1}}^{x_{i+1}} P_i(s) \; ds = h \left(\frac{f(x_{i-1})}{3} + \frac{4 f(x_{i})}{3} + \frac{f(x_{i+1})}{3} \right)
$$

### Planteamiento de la regla

Podemos ahora, después de demostrar la fórmula del ejercicio 9, utilizar dicha fórmula para implementar una regla de integración, notando que

$$
\int_{a}^b f(s) \; ds = \sum_{i=0}^{(n-2)/2} \int_{x_{2i}}^{x_{2i+2}} f(s) \; ds
$$

A esta regla de integración se le conoce como **regla de Simpson**

## Ejercicio 11

Implementa una función `intSimpson(f,a,b,n)` que regrese el valor de la integral $\int_{a}^b f(s) \; ds $ calculado mediante regla de Simpson.

## Ejercicio 12

Repite el ejercicio 1 ahora utilizando la regla de Simpson y compara con los otros resultados. ¿Qué tanto mejora la tasa de convergencia?

## La visión general

Integrar funciones es un problema que surge todo el tiempo a la hora de hacer cómputo científico. Muchísimos problemas se descomponen, desde el punto de vista teórico, en hacer integrales (electromagnetismo, por ejemplo). Existen, debido a eso, una infinidad de reglas y aproximaciones para hacer integrales de funciones de manera numérica.

El mayor desafío es cuando tenemos ya no integrales en una dimensión, sino integrales en varias dimensiones (a veces más de 20). En esos casos, las aproximaciones aquí mencionadas se pueden volver inútiles.

Las reglas rectangulares no son muy usadas en aplicaciones reales pues convergen a una velocidad muy pequeña. Las regla de Simpson y la trapezoidal, por el contrario, son mucho más usadas.

Existe una manera formal de estimar el error de las reglas de integración mostradas aquí. No quise enseñárselos por que, aunque sea importante, es más largo y complicado que los errores de derivación numérica.

Pueden consultar como calcular dichos errores en el libro de Stickler (*Basic Concepts in Computational Physics*)

```julia

```


# Circular hole probllem

In this notebook an analytic solution for a disk-shaped hole is constructed.

The domain defined as
$$
    \Omega = \{ x \in \mathbb{R^2} : 0 < a < |x| < A\}
$$
Let
$$
    \Gamma^r = \{ x \in \mathbb{R^2} : |x| = r\},
$$
then $\partial\Omega = \Gamma^a \cup \Gamma^A$.

## Poisson

Sought is the solution to the problem
$$
\begin{aligned}
    \nabla^2 u = 0, & \quad x \in \Omega, \\
    \frac{\partial u}{\partial r} = 0, & \quad x \in \Gamma^a, \\
    u = g, & \quad x \in \Gamma^A, \\
\end{aligned}
$$
where the Laplacian in polar coordinates is
$$
    \nabla^2 = \frac{\partial^2}{\partial r^2}
               + \frac{1}{r}\frac{\partial}{\partial r}
               + \frac{1}{r^2}\frac{\partial^2}{\partial \theta^2}
$$
and $g$ is chosen such that the solution is easy to construct :-)

```python
from sympy import *
from util import lua_code
```

```python
# Laplacian in polar coordinates
def laplacian(f):
    return f.diff(r, 2) + f.diff(r)/r + f.diff(t, 2)/r**2
```

Solution ansatz:

```python
r, t = symbols('r theta')
a, B = symbols('a B')
u = r**2 * sin(2*t) + B * sin(2*t) / r**2
u
```

$\displaystyle \frac{B \sin{\left(2 \theta \right)}}{r^{2}} + r^{2} \sin{\left(2 \theta \right)}$

Check Laplacian

```python
simplify(laplacian(u))
```

$\displaystyle 0$

Check inner circle boundary

```python
u.diff(r).subs(r, a)
```

$\displaystyle - \frac{2 B \sin{\left(2 \theta \right)}}{a^{3}} + 2 a \sin{\left(2 \theta \right)}$

Hence,

```python
u = u.subs(B, a**4)
u.diff(r).subs(r, a)
```

$\displaystyle 0$

We set $g=u$ on the outer circle boundary.

Gradient:

```python
u_x = simplify(u.diff(r) * cos(t) - u.diff(t) * sin(t) / r)
u_x
```

$\displaystyle - \frac{2 a^{4} \sin{\left(3 \theta \right)}}{r^{3}} + 2 r \sin{\left(\theta \right)}$

```python
u_y = simplify(u.diff(r) * sin(t) + u.diff(t) * cos(t) / r)
u_y
```

$\displaystyle \frac{2 a^{4} \cos{\left(3 \theta \right)}}{r^{3}} + 2 r \cos{\left(\theta \right)}$

## Elasticity

For a plane-strain problem, we assume there exists a biharmonic function $\phi$ such that [1]

\begin{align}
    2\mu u_r &= -\frac{\partial \phi}{\partial r} + (1-\nu)r\frac{\partial\psi}{\partial \theta} \\
    2\mu u_\theta &= -\frac{1}{r}\frac{\partial \phi}{\partial \theta} + (1-\nu)r^2\frac{\partial\psi}{\partial r} \\
    \sigma_{rr} &= \frac{1}{r}\frac{\partial \phi}{\partial r} +
    \frac{1}{r^2}\frac{\partial^2 \phi}{\partial \theta^2} \\
    \sigma_{\theta\theta} &= \frac{\partial^2 \phi}{\partial r^2} \\
    \sigma_{r\theta} &= -\frac{\partial }{\partial r} \left(
      \frac{1}{r}\frac{\partial \phi}{\partial\theta}
    \right) \\
\end{align}
where $\mu$ is the shear modulus, $\nu$ is Poisson's ratio, and $\phi$ and $\psi$ are related by
$$
\nabla^2\phi = \frac{\partial}{\partial r}\left(r\frac{\partial\psi}{\partial\theta}\right)
$$

Remark: Plane-stress is modelled in [1], therefore the formula are adjusted here for plane-strain.

[1] "Stress Singularites Resulting From Various Boundary Conditions in Angular Corners of Plates in Extension", M. L. Williams, 1952, Journal of Applied Mechanics, 19 (4), pp. 526-528.

Sought is the solution to the problem
$$
\begin{aligned}
    \sigma_{rr} = 0, & \quad x \in \Gamma^a, \\
    \sigma_{r\theta} = 0, & \quad x \in \Gamma^a, \\
    u_r = g_r, & \quad x \in \Gamma^A, \\
    u_\theta = g_\theta, & \quad x \in \Gamma^A, \\
\end{aligned}
$$
for $\phi,\psi$ such that the equations of elasticity are satisfied.

Solution ansatz (from J.R. Barber, Elasticity, 2002):

```python
r, t = symbols('r theta')
a, S, mu, nu = symbols('a S mu nu')
A = S*a**2
B = -S*a**4/2
phi = -S * r**2 * sin(2*t) / 2 + A * sin(2*t) + B * sin(2*t) / r**2
phi
```

$\displaystyle - \frac{S a^{4} \sin{\left(2 \theta \right)}}{2 r^{2}} + S a^{2} \sin{\left(2 \theta \right)} - \frac{S r^{2} \sin{\left(2 \theta \right)}}{2}$

Check that phi is biharmonic.

```python
laplacian(simplify(laplacian(phi)))
```

$\displaystyle 0$

Compute stresses

```python
srr = simplify(phi.diff(r) / r + phi.diff(t, 2) / r**2)
stt = simplify(phi.diff(t, 2))
srt = simplify(-(phi.diff(t) / r).diff(r))
```

Check boundary condition on inner circle.

```python
(srr.subs(r, a), srt.subs(r, a))
```

    (0, 0)

Find $\psi$ from compatbility condition, disregarding integration constants.

```python
psi = integrate(integrate(simplify(laplacian(phi)), r) / r, t)
psi
```

$\displaystyle - \frac{2 S a^{2} \cos{\left(2 \theta \right)}}{r^{2}}$

Check compatbility condition.

```python
simplify((r*psi.diff(t)).diff(r) - laplacian(phi))
```

$\displaystyle 0$

Compute displacement

```python
ur = simplify(-phi.diff(r) + (1 - nu) * r * psi.diff(t)) / (2*mu)
ut = simplify(-phi.diff(t) / r + (1 - nu) * r**2 * psi.diff(r)) / (2*mu)
```

```python
ur
```

$\displaystyle \frac{S \left(- a^{4} - 4 a^{2} r^{2} \left(\nu - 1\right) + r^{4}\right) \sin{\left(2 \theta \right)}}{2 \mu r^{3}}$

```python
lua_code(ur)
```

    '(1.0/2.0)*S*(-a^4 - 4*a^2*r^2*(nu - 1) + r^4)*math.sin(2*theta)/(mu*r^3)'

```python
ut
```

$\displaystyle \frac{S \left(a^{4} - 4 a^{2} \nu r^{2} + 2 a^{2} r^{2} + r^{4}\right) \cos{\left(2 \theta \right)}}{2 \mu r^{3}}$

```python
lua_code(ut)
```

    '(1.0/2.0)*S*(a^4 - 4*a^2*nu*r^2 + 2*a^2*r^2 + r^4)*math.cos(2*theta)/(mu*r^3)'

We set $g_r=u_r$ and $g_\theta = u_\theta$ on the outer circle boundary.

Compute Jacobian

```python
J = {}
J['ur_r'] = simplify(ur.diff(r))
J['ur_t'] = simplify(ur.diff(t))
J['ut_r'] = simplify(ut.diff(r))
J['ut_t'] = simplify(ut.diff(t))
```

```python
for key, value in J.items():
    print(f'local {key} = {lua_code(value)}')
```

    local ur_r = (1.0/2.0)*S*(3*a^4 + 4*a^2*nu*r^2 - 4*a^2*r^2 + r^4)*math.sin(2*theta)/(mu*r^4)
    local ur_t = -S*(a^4 + 4*a^2*r^2*(nu - 1) - r^4)*math.cos(2*theta)/(mu*r^3)
    local ut_r = (1.0/2.0)*S*(-3*a^4 + 4*a^2*nu*r^2 - 2*a^2*r^2 + r^4)*math.cos(2*theta)/(mu*r^4)
    local ut_t = S*(-a^4 + 4*a^2*nu*r^2 - 2*a^2*r^2 - r^4)*math.sin(2*theta)/(mu*r^3)

```python

```


# Session 5: Gradient Descent

## House sale-value prediction using the Boston housing dataset

------------------------------------------------------
*ATDST, 2017-2018*

*Pablo M. Olmos olmos@tsc.uc3m.es*

------------------------------------------------------

## Importing Packages

```python
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from sklearn import linear_model

%matplotlib inline
```

Today, we will continue with the example we used for session 2: Predicting house values using the average number of rooms in the [Boston housing dataset](https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.names).

Attribute Information can be found [here](https://www.kaggle.com/c/boston-housing).

### Loading Data

We will manage the database using [Panda's library and Dataframes](http://pandas.pydata.org/pandas-docs/stable/tutorials.html)

```python
housing_data=pd.read_csv('./Boston_train.csv')
```

We divide the whole data set in **80% training** and **20% test**:

```python
N = len(housing_data)

N_train = np.round(0.8 * N).astype(np.int32)

np.random.seed(seed=10) #To fix the random seed

mask = np.random.permutation(len(housing_data))

regression_data_frame = housing_data[['rm','medv']].iloc[list(mask[0:N_train])]

X_0 = np.array(regression_data_frame['rm'])
Y = np.array(regression_data_frame['medv'])

regression_data_frame_test = housing_data[['rm','medv']].iloc[list(mask[N_train:-1])]

X_0_test = np.array(regression_data_frame_test['rm'])
Y_test = np.array(regression_data_frame_test['medv'])

```

# Numerical Optimization with Gradient Descend

As we know, the Ridge regression optimization problem
$$\boldsymbol{\theta}_\lambda = \arg \min_{\theta} \frac{1}{N} \left[\sum_{i=1}^{N} (y^{(i)}-\boldsymbol{\theta}^T\mathbf{x}^{(i)})^2 + \lambda \sum_{j=1}^{D+1} \theta_j^2\right],$$
can be solved using the **normal equation**
$$\boldsymbol{\theta}_\lambda = (\mathbf{X}^T\mathbf{X} + \mathbf{D}_\lambda)^{-1}\mathbf{X}^T\mathbf{y},$$

However, as we advanced in Session 2, computing the normal equation has many important drawbacks:
- You need to keep the full target matrix $\mathbf{X}_{N\times (D+1)}$ in memory, which can be huge in large datasets!
- You need to invert the matrix $(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}$ $\Rightarrow$ $\mathcal{O}(D^3)$ complexity.
- For small data sets ($N\leq D$), $(\mathbf{X}^T\mathbf{X})^{-1}$ can be non-invertible!
- Once you get new data, how do you update $\boldsymbol{\theta}^*$?

Today, we will learn how to apply a very simple algorithm to perform numerical optimization: **gradient descent** (GD). GD is one of the cornerstones of **convex optimization**, but it is also widely used for **non-convex optimization** for instance in [deep learning with neural networks](https://www.youtube.com/watch?v=IHZwWFHWa-w). Be aware that GD is probably the simplest numerical optimization algorithm  and that there is a whole [field](https://web.stanford.edu/group/sisl/k12/optimization/#!index.md) devoted to optimize numerically a function.

In a nutshell, GD tries to iteratively converge to the minimum of a function $f(\boldsymbol{\theta})$ by iteratively applying the following update rule:

$$\boldsymbol{\theta}_{\ell+1} = \boldsymbol{\theta}_\ell - \alpha \nabla f(\boldsymbol{\theta})|_{\boldsymbol{\theta}_\ell},$$

where $\nabla(\cdot)$ is the gradient operator and $\alpha$ is the learning rate. **Setting the learning rate** is a problem in general. Despite we won't cover the topic in detail and just restrict to simple checks, be aware that there are many modifications to GD that attempt to authomatically tune $\alpha$, like the [line search](https://people.maths.ox.ac.uk/hauser/hauser_lecture2.pdf) method.

#### Check out this [beautiful post](http://www.benfrederickson.com/numerical-optimization/)!

___

# Solving the Ridge regression problem using GD

If we define

$$ J(\boldsymbol{\theta}) = \frac{1}{N} \left[\sum_{i=1}^{N} (y^{(i)}-\boldsymbol{\theta}^T\mathbf{x}^{(i)})^2 + \lambda \sum_{j=1}^{D+1}\theta_j^2\right],$$

then it is easy to check that, for $m>0$:

\begin{align}
\frac{\partial J(\boldsymbol{\theta})}{\partial \theta_m}= \frac{2}{N} \left[\lambda\theta_m-\sum_{i=1}^{N} x_m^{(i)}\left(y^{(i)}-\boldsymbol{\theta}^T\mathbf{x}^{(i)}\right)\right]= \frac{2}{N} \left[\lambda\theta_m - \mathbf{e}^T \mathbf{X}_{:,m}\right],
\end{align}
where $\mathbf{e}=\mathbf{y}-\boldsymbol{\theta}^T \mathbf{X}$ is the error vector and $\mathbf{X}_{:,m}$ is the $m$-th column of the normalized training feature matrix. For $m=0$ the first term is not present, since the intercept is not regularized:
\begin{align}
\frac{\partial J(\boldsymbol{\theta})}{\partial \theta_m}= \frac{-2}{N} \left[\sum_{i=1}^{N} x_m^{(i)}\left(y^{(i)}-\boldsymbol{\theta}^T\mathbf{x}^{(i)}\right)\right]= \frac{-2}{N} \left[\mathbf{e}^T \mathbf{X}_{:,m}\right],
\end{align}

Note that in both cases ** the gradient vanishes when the error is zero**.

Lets now program the GD method in a modular way. First, we incorporate some useful functions from the ** previous session**. They will be included as a Python library file.

```python
import Ridge_functions as ridge   # You can find details in provided file S4_Ridge_functions.py
```

## Compute Gradient

** First ** you have to program a function that computes the gradient and takes as input arguments
- The feature matrix $\mathbf{X}$
- The error vector $\mathbf{e}$
- The value of $\lambda$
- The current value of $\boldsymbol{\theta}$
and returns as output the $(D+1)$-dimensional gradient of $J(\boldsymbol{\theta})$.

```python
def compute_gradient(feature_matrix,error,l,T):
    ## YOUR CODE HERE

    gradient = l*T - error @ feature_matrix
    gradient[0] -= l*T[0]
    gradient *= 2.0/feature_matrix.shape[0]

    return(gradient)
```

## Numerical evaluation of the gradient

Before moving forward, let's run a numerical check to verify that the gradient you compute in the function above is correct.

1) Fix a given arbitrary value of $\boldsymbol{\theta}^o$.

2) Introduce a small distortion in one of the components of $\boldsymbol{\theta}$. For instance, $\theta_m^+=\theta^o_m$ for $m\neq 1$ and $\theta^+_1 = \theta^o_1+\epsilon$, where $\epsilon=10^{-3}$. Evaluate the cost function $J(\boldsymbol{\theta}^+)$.

3) Define a new vector $\boldsymbol{\theta}^-$ such that $\theta_m^-=\theta^o_m$ for $m\neq 1$ and $\theta^-_1 = \theta^o_1-\epsilon$, where $\epsilon=10^{-3}$. Evaluate the cost function $J(\boldsymbol{\theta}^-)$.

4) Verify that

$$\frac{\partial J(\boldsymbol{\theta})}{\partial \theta_1} \approx \frac{J(\boldsymbol{\theta}^+)-J(\boldsymbol{\theta}^-)}{2\epsilon}$$,
where the derivative in the left hand side is given by the second component of the gradient computed using your function above. **Repeat the experiment using the first component of the gradient, i.e., the derivative w.r.t. $\theta_0$**.

```python
# Lets fix a degree
deg = 2
l= 1.0
np.random.seed(seed=10) #To fix the random seed

T = np.random.rand(3)

# This is how we would compute the cost function at T and obtain the normalized train feature matrix
# J_train,_,F_train,_ = ridge.eval_J_Ridge_given_T(X_0,X_0_test,deg,Y,Y_test,l,T)

# Your code here
mod = np.zeros(T.shape)
epsilon = 1e-2
index = 0
mod[index] += epsilon

T_p = T + mod
T_m = T - mod

J_p,_,F_train,_ = ridge.eval_J_Ridge_given_T(X_0,X_0_test,deg,Y,Y_test,l,T_p)
J_m,_,_,_ = ridge.eval_J_Ridge_given_T(X_0,X_0_test,deg,Y,Y_test,l,T_m)

error = (Y - ridge.LS_evaluate(F_train,T))

gradient = compute_gradient(F_train,error,l,T)

print("The gradient at position %d is %f" %(index,gradient[index]))

print("The approximate gradient at position %d is %f" %(index,(J_p-J_m)/(2*epsilon))) # Your code here

```

    The gradient at position 0 is -44.204727
    The approximate gradient at position 0 is -44.198928

## Implementing GD

Now, complete the following function to estimate the Ridge solution using gradient descend. The inputs are:

- The normalized train feature matrix $\mathbf{X}$
- The target vector $\mathbf{Y}$
- The initial value of theta $\boldsymbol{\theta}$
- The step size $\alpha$
- The stopping tolerance: a threshold that halts the algorithm when the norm of the gradient is below this limit
- The maximum number of iterations

```python
def regression_gradient_descent(F_train, Y, T_0, l, step_size, tolerance, iter_max,verbose=True,period_verbose=1000):
    converged = False
    T = np.array(T_0) # make sure it's a numpy array
    it=0

    while not converged:

        # First, compute the error vector
        error= (Y - ridge.LS_evaluate(F_train,T)) #YOUR CODE HERE

        # Second compute the gradient vector
        gradient= compute_gradient(F_train,error,l,T) #YOUR CODE HERE

        # Finally, update the theta vector
        T=T- step_size * gradient #YOUR CODE HERE

        grad_norm=np.linalg.norm(gradient)

        if(verbose==True and it % period_verbose ==0):
            J_Ridge = ridge.J_error_L2(Y,ridge.LS_evaluate(F_train,T),T,l)
            print ("Iterations = %d" %(it))
            print ("Gradient norm %f" %(grad_norm))
            print ("Ridge cost function %f" %(J_Ridge))

        if grad_norm < tolerance:
            converged = True
        elif it > iter_max:
            converged = True
        else:
            it=it+1

    if(converged==True):
        J_Ridge = ridge.J_error_L2(Y,ridge.LS_evaluate(F_train,T),T,l)
        print ("Iterations = %d" %(it))
        print ("Gradient norm %f" %(grad_norm))
        print ("Ridge cost function %f" %(J_Ridge))

    return(T)
```

## Evaluate solution and compare with normal equation

Run an example to verify that the GD solution is close to the one predicted by the normal equation. Investigate the effect of the step size.

```python
# Lets fix a degree
deg = 5
l= 1.0
T = np.random.rand(6)

#Cost function at initial T and feature matrix
J_0,_,F_train,_ = ridge.eval_J_Ridge_given_T(X_0,X_0_test,deg,Y,Y_test,l,T)

step_size = 1e-02
iter_max = 2e4
tolerance = 1e-03
period= 1e04

T_opt = regression_gradient_descent(F_train, Y, T, l, step_size, tolerance, iter_max,verbose=True,period_verbose=period)

T_normal = ridge.Ridge_solution(F_train,Y,l)

plt.stem(T_normal,'r',label='Normal Equation')
plt.plot(T_opt,'b',label='Gradient Descent')
plt.legend()
```

```python

```

```python

```

Notebooks/Session 5 Gradient Descent/S5-Gradiend Descend (Complete).ipynb


## Examples

```python
import numpy as np
import sys
sys.path.append('../..')
from utils.graph_utils import digraph_from_numpy_array
```

### Probability Tables Example

```python
# Country probabilities
cnt = np.array([0.88, 0.08, 0.04])
# Mother tongue given country probabilities
mt_given_cnt = np.array([[.95, .7, .6],
                         [.04, .3, .0],
                         [.01, .0, .4]])
```

```python
# Matrix containing the joint probabilities at its each entry
# This is not a matrix product
cnt_and_mt = mt_given_cnt*cnt
cnt_and_mt
```

    array([[ 0.836 ,  0.056 ,  0.024 ],
           [ 0.0352,  0.024 ,  0.    ],
           [ 0.0088,  0.    ,  0.016 ]])

```python
# Sum over the mother tongues to get country probabilities
np.sum(cnt_and_mt, axis=0)
```

    array([ 0.88,  0.08,  0.04])

```python
# Sum over the countries to get mother tongue probabilities
np.sum(cnt_and_mt, axis=1)
```

    array([ 0.916 ,  0.0592,  0.0248])

### Example 1.3 (Obtaining the Desired Conditional Probability Using Marginalizations)
$
\begin{align}
p(B \mid K) &= \sum_m p(B, m \mid K) & \quad\quad \text{Marginalize over b} \\
            &= \sum_m \frac{p(B, m, K)}{p(K)} & \quad\quad \text{Conditional to joint} \\
            &= \frac{\sum_m p(B, m, K)}{p(K)} & \quad\quad \text{Sum to nominator} \\
            &= \frac{\sum_m p(B, m, K)}{\sum_b \sum_m p(K \mid b, m)p(b, m)} & \quad\quad \text{Marginalize over m and b} \\
            &= \frac{\sum_m p(B, m, K)}{\sum_b \sum_m p(K \mid b, m)p(b)p(m)} & \quad\quad \text{m and b are independent} \\
            &= \frac{\sum_m p(B, m, K)}{\sum_b p(b) \sum_m p(K \mid b, m)p(m)} & \quad\quad \text{p(b) to outer sum} \\
            &= \frac{\sum_m p(K \mid B, m)p(B, m)}{\sum_b p(b) \sum_m p(K \mid b, m)p(m)} & \quad\quad \text{Joint to conditional} \\
            &= \frac{\sum_m p(K \mid B, m)p(B)p(m)}{\sum_b p(b) \sum_m p(K \mid b, m)p(m)} & \quad\quad \text{m and b are independent} \\
            &= \frac{p(B) \sum_m p(K \mid B, m)p(m)}{\sum_b p(b) \sum_m p(K \mid b, m)p(m)} & \quad\quad \text{p(b) to outer sum} \\
\end{align}
$

### Exercise 1.11 (Soft XOR Gate)

```python
def soft_xor(a, b):
    """
    Stochastically outputs 1 depending on a and b.
    """
    high_prob = np.array([0.1, 0.99, 0.8, 0.25])
    rand_num = np.random.ranf()
    if a == 0 and b == 0:
        threshold_index = 0
    elif a == 0 and b == 1:
        threshold_index = 1
    elif a == 1 and b == 0:
        threshold_index = 2
    elif a == 1 and b == 1:
        threshold_index = 3

    if rand_num <= high_prob[threshold_index]:
        return 1
    return 0
```

```python
num_tries = 10000
sum(soft_xor(0, 1) for _ in range(num_tries))/num_tries
```

    0.9907

### Exercise 1.12 (Hamburgers)

```python
p_kj = 1/100000
p_h_given_kj = 0.9

### 1.
p_h = 0.5
p_kj_given_h = (p_h_given_kj*p_kj)/p_h
print('Question 1')
print('p(KJ|Hamburger Eater) = {:f}'.format(p_kj_given_h))
print()

### 2.
p_h = 0.001
p_kj_given_h = (p_h_given_kj*p_kj)/p_h
print('Question 2')
print('p(KJ|Hamburger Eater) = {:f}'.format(p_kj_given_h))
```

    Question 1
    p(KJ|Hamburger Eater) = 0.000018

    Question 2
    p(KJ|Hamburger Eater) = 0.009000

### Exercise 1.13 (Two Dice)

```python
# Prior
p_sa = np.full(6, 1/6)
p_sb = np.full(6, 1/6)
prior = np.outer(p_sa, p_sb)
print('Prior = P(sa, sb) = P(sa)P(sb)')
print(prior)
print()

# Likelihood
indices_1d = np.reshape(np.arange(36), (6, 6))
dice_sums = (indices_1d//6 + 1) + (indices_1d%6 + 1)
likelihood = (dice_sums == 9).astype(int)
print('Likelihood = P(t = 9 | sa, sb)')
print(likelihood)
print()

# Normalizer
joint = prior*likelihood
print('Joint = P(t = 9, sa, sb) = Likelihood*Prior')
print(joint)
print()
normalizer = np.sum(joint)

# Posterior
posterior = likelihood*prior/normalizer  # joint/normalizer
print('Posterior = P(sa, sb | t = 9)')
print(posterior)
```

    Prior = P(sa, sb) = P(sa)P(sb)
    [[ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]
     [ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]
     [ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]
     [ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]
     [ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]
     [ 0.02777778  0.02777778  0.02777778  0.02777778  0.02777778  0.02777778]]

    Likelihood = P(t = 9 | sa, sb)
    [[0 0 0 0 0 0]
     [0 0 0 0 0 0]
     [0 0 0 0 0 1]
     [0 0 0 0 1 0]
     [0 0 0 1 0 0]
     [0 0 1 0 0 0]]

    Joint = P(t = 9, sa, sb) = Likelihood*Prior
    [[ 0.          0.          0.          0.          0.          0.        ]
     [ 0.          0.          0.          0.          0.          0.        ]
     [ 0.          0.          0.          0.          0.          0.02777778]
     [ 0.          0.          0.          0.          0.02777778  0.        ]
     [ 0.          0.          0.          0.02777778  0.          0.        ]
     [ 0.          0.          0.02777778  0.          0.          0.        ]]

    Posterior = P(sa, sb | t = 9)
    [[ 0.    0.    0.    0.    0.    0.  ]
     [ 0.    0.    0.    0.    0.    0.  ]
     [ 0.    0.    0.    0.    0.    0.25]
     [ 0.    0.    0.    0.    0.25  0.  ]
     [ 0.    0.    0.    0.25  0.    0.  ]
     [ 0.    0.    0.25  0.    0.    0.  ]]

### Exercise 1.19.2

$T : \text{Random variable representing the weather today}$

$Y : \text{Random variable representing the weather yesterday}$

Then, we can draw the table for $P(T \mid Y)$.

|   P(T $|$ Y)    | Y = Rainy | Y = Sunny |
| --------------- | --------- | --------- |
| T = Rainy       |     0.7   |     0.6   |
| T = Sunny       |     0.3   |     0.4   |

This conditional probability table corresponds to the following Markov Chain.

```python
# state transition matrix
T = np.array([[0.7, 0.3],
              [0.6, 0.4]])
digraph_from_numpy_array(T, labels=['Rainy', 'Sunny'], edge_fmt='{:.1f}')
```

To answer the second part of the question, we need the steady state transition matrix of this Markov Chain which can be found by multiplying the initial transition matrix with itself sufficiently many times.

```python
T_s = T
for i in range(5):
    T_s = T_s@T
    print('T^{}'.format(i + 2))
    display(digraph_from_numpy_array(T_s, labels=['Rainy', 'Sunny'], edge_fmt='{:.5f}'))
    print('='*20)
```

    T^2

    ====================
    T^3

    ====================
    T^4

    ====================
    T^5

    ====================
    T^6

    ====================


## Equations

Find the derivative with respect to x of the following functions. **Additionally, find the derivative with respect to a of function h.**
Provide your answers on this sheet. Where you took more than one step to arrive at the final result, please include the important steps. You may fill out the sheet by hand and submit a scanned version.

$
\begin{align}
f(x) = 3x^2
\end{align}
$

$
\begin{align}
g(x) = (x+8)^2
\end{align}
$

$
\begin{align}
h(x) = ax^3+\frac{1}{2}x^8
\end{align}
$

$
\begin{align}
k(x) = x^{501}+3x^7 - \frac{1}{2}x^6+x^5+2x^3+3x^2-1   % Note 501 is between {} to display correctly
\end{align}
$

## Solutions

#### df/dx

$
\begin{align}
\frac{df}{dx} &= \frac{d}{dx}(3x^2)	&\\
&= 3.2x^{2-1}	&\\
&= 3.2x  &\\
&= 6x
\end{align}
$

#### dg/dx

$
\begin{align}
\frac{dg}{dx} &=  \frac{d}{dx} (x+8)^2 	&\\
&=   \frac{d}{dx} (x^2+16x+64) 	&\\
&=   (2x^{2-1}+16x^0+0) 	&\\
&=   (2x+16) 	&\\
&=   2(x+8)
\end{align}
$

#### dk/dx

$
\begin{align}
\frac{dk}{dx} &= \frac{d}{dx} (x^{501}+3x^7 - \frac{1}{2}x^6+x^5+2x^3+3x^2-1) 	&\\
&= (501x^{501-1})+3(7x^{7-1}) - \frac{1}{2}(6x^{6-1})+(5x^{5-1})+2(3x^{3-1})+3(2x^{2-1})-0 	&\\
&= 501x^{500} + 3.7x^6 - \frac{1}{2}.6x^5 + 5x^4 + 2.3x^2 + 3.2x 	&\\
&= 501x^{500} + 21x^6 - 3x^5 + 5x^4 + 6x^2 + 6x
\end{align}
$

#### ∂h/∂x

$
\begin{align}
\frac{\partial h}{\partial x} &=  \frac{\partial}{\partial x} (ax^3+\frac{1}{2}x^8)  &\\
&= a(3x^2)+\frac{1}{2}(8x^7)  &\\
&= 3ax^2+4x^7
\end{align}
$

#### ∂h/∂a

$
\begin{align}
\frac{\partial h}{\partial a} &= \frac{\partial}{\partial a} (ax^3+\frac{1}{2}x^8) 	&\\
&= (x^3+0) 	&\\
&= x^3
\end{align}
$


# Matemática: Introdução ao Cálculo de Derivadas - Parte 01

# Variação Instantânea de uma Função

## **Exemplo 01**

O modelo de oscilação de um amortecedor é descrito pela seguinte função:

**Distância entre as extremidades**

$ g(x) = e^{\sqrt{X}}cos(x)$

**Função arbitrária para comparação**

$ f(x) = x + 1$

## Definindo Funções

```
from sympy import *
import math
from math import *
init_printing()

var('x')

# Trabalhando em Radianos
def g(x): return ( e ** ( sqrt( x ) )) * ( cos( x ) )
def f(x): return x + 1

print(g(0))
print(f(0))

```

    1.0
    1

## Variação Média da Função

A variação média de uma função f(x), dentro do intervalo [a,b] é definida por:

$
\dfrac{\Delta f(x)}{\Delta x} = \dfrac{f(b)-f(a)}{b-a}
$

```
def variacao_f(a,b): return (f(b)-f(a))/(b -a)
def variacao_g(a,b): return (g(b)-g(a))/(b -a)

print(variacao_f(0,5.39))
print(variacao_g(0,5.39))
```

    1.0
    0.9999863432970383

### Utilizando Bokeh para plotar gráficos

```
from bokeh.plotting import figure, output_notebook, show
from bokeh.io import curdoc, show
from bokeh.io import output_notebook
from bokeh.models.tools import HoverTool

output_notebook()

import numpy as np

x = list()
y = list()

for i in np.arange(0,20, step=0.1):
  x.append(i)
  y.append(g(i))

# Gerando canvas
fig = figure(title='Gráfico com Bokeh',
             x_axis_label='x',
             y_axis_label='y',
             width=600,
             height=600)
# Desenhando a linha
fig.line(x, y,
         line_alpha=1,
         legend_label='g (x)',
         line_width=3)

x2 = list()
y2 = list()

for i in np.arange(0,20, step=0.1):
  x2.append(i)
  y2.append(f(i))

# Desenhando a linha
fig.line(x2, y2,
         line_alpha=1,
         legend_label='f (x)',
         line_width=3,
         color= 'green')

fig.add_tools(HoverTool(
    tooltips='<font face="Arial" size="3"> x = @x{0.00}, y = @y{0.00}</font>',
    mode='vline'
))

show(fig)

```

## Definindo Variação Instantânea

A variação instantânea de f(x) corresponde ao limite da variação média da f(x) quando $ \Delta x$ tende a 0:

$ \lim_{\Delta x \to 0} \left[\dfrac{f(x + \Delta x)-f(x)}{\Delta x}\right]$

```
var ('x delta')

def h(x): return x**3 + 3*x**2 +1
def i(x,delta): return (h(x+delta)-h(x))/delta

i(x,delta)
```

```
# Testando a função
display(i(1,0.55))
display(i(1,3))
display(i(1,2))
display(i(1,0.5))
```

```
def variacao_inst(x): return limit(i(x,delta),delta,0)

# Testando a função por x
# Variação instantânea é chamada de derivada
display(variacao_inst(1))
display(variacao_inst(3))
display(variacao_inst(2))
display(variacao_inst(9))
```

# Derivada e Variação Instantânea

## **Definição**

A variação instantânea de uma função f(x) leva um nome particular: ela chama-se a derivada da função f(x) no ponto x, e é simbolizada como:

$ \dfrac{\partial f(x)}{\partial x} = \lim_{\Delta x \to 0} \left[\dfrac{f(x + \Delta x)-f(x)}{\Delta x}\right]$

##**Calculando a primeira derivada**

Assumindo  a função

$ f(x) = x^2$

Podemos perceber o seguinte comportamento na variação instantânea

$(a+b)^2 = a^2 + 2ab + b^2$

Logo:

$f(x + \Delta x) = (x + \Delta x)^2 = x^2 + 2x\Delta x + \Delta x ^2 $

Assim temos:

$f(x + \Delta x) = (x + \Delta x)^2 = x^2 + 2x\Delta x + \Delta x ^2  - x^2 = 2x\Delta x + \Delta x ^2 $

Substituindo:

 $\lim_{\Delta x \to 0} \left[\dfrac{2x\Delta x + \Delta x ^2}{\Delta x}\right] = \lim_{\Delta x \to 0} \left[2x+ \Delta x \right] $

Por fim,  se $\Delta x $ tende a 0, obtemos:

 $ \lim_{\Delta x \to 0} 2x +  \lim_{\Delta x \to 0} \Delta x =  2x +  0  = 2x $

 **Resultado do Cálculo:**

## $ 2x$

```
def f(x): return x**2
def i(x,delta): return (f(x+delta)-f(x))/delta
def variacao_inst(x): return limit(i(x,delta),delta,0)

# Resultado da função criada
print('Resultado Função Criada:')
display(variacao_inst(x))

print()

print('Resultado Função Numpy:')
# Resultado da função existente no numpy
display(diff(f(x)))
```

## **Regra da Derivada de potência**

$ \dfrac{\partial x^n}{\partial x} = nx ^{n-1}$

# A derivada como reta tangente

## **Definição**

Uma reta tangente a um ponto de uma curva é aquela reta que enconsta na curva, naquele ponto, sem cortar a curva no mesmo.

## Conceito de reta

A noção de reta ou linha reta foi introduzida por matemáticos antigos para representar objetos retos com largura e profundidade desprezíveis. As retas são uma idealização de tais objetos.

Com a equação reduzida da reta, é possível calcular quais são os pontos que pertencem a essa reta e quais não pertencem.

Forma reduzida:
$ y(x) = ax + b$

onde:

*   a --> Coeficiente Angular
*   b --> Coeficiente Linear
*   x --> Variável Independente

## **Declarando funções**

```
def f(x): return ((3)*(x**2)) + 1
def r1(x): return (6*(x-1))+4
def r2(x): return 1
```

## **Plotando gráfico**

```
x,y,y2,y3 = list(),list(),list(),list()

for i in np.arange(-1,4, step=0.1):
  x.append(i)
  y.append(f(i))
  y2.append(r1(i))
  y3.append(r2(i))

# Gerando canvas
fig = figure(title='Reta Tangente',
             x_axis_label='x',
             y_axis_label='y',
             width=600,
             height=600)

# Desenhando a linha
fig.line(x, y,
         line_alpha=1,
         legend_label='f (x)',
         line_width=3)

# Desenhando a linha
fig.line(x, y2,
         line_alpha=1,
         legend_label='r1 (x)',
         line_width=3,
         color= 'green')

# Desenhando a linha
fig.line(x, y3,
         line_alpha=1,
         legend_label='r2 (x)',
         line_width=3,
         color= 'red')

fig.add_tools(HoverTool(
    tooltips='<font face="Arial" size="3"> x = @x{0.00}, y = @y{0.00}</font>',
    mode='vline'
))

show(fig)

```

# Determinando Tangentes Usando a Derivada

## **Questão do tópico**

Dada a curva f(x), como podemos determinar as suas tangentes?

## **Resposta**

Calcular as derivadas de f(x). A derivada de uma curva f(x) em um ponto escolhido, determina o coeficiente angular de inclinação da reta que é tangente naquele ponto.

## **Exemplo**
Dada a função f(x) abaixo, calcule o coeficiente angular das retas tangentes a essa curva , nos pontos $ x= 1$ e $ x=0$
.

#### **Função**

$f(x) = 3x^2 + 1$

#### **Resolvendo:**

Regra da derivada de potência:

$ \dfrac{\partial x^n}{\partial x} = nx ^{n-1}$

 A derivada de uma constante é 0, logo:

$ \dfrac{\partial  f }{\partial x} = 2 (3x^2) ^{2-1} = 6x$

#### **Resposta:**

No ponto $x = 0$, o resultado da derivada é 0.

No ponto $x = 1$, o resultado da derivada é 6.

```
def f(x): return (3*(x**2)) + 1
def i(x,delta): return (f(x+delta)-f(x))/delta
def variacao_inst(x): return limit(i(x,delta),delta,0)

# Resultado da função criada
print('Resultado Função Criada:')
display(variacao_inst(0))

print()

# Resultado da função criada
print('Resultado Função Criada:')
display(variacao_inst(1))

print()

print('Resultado Função Numpy:')
# Resultado da função existente no numpy
var('x')
derivada = f(x).diff(x)
display(derivada)

# Resultado da função existente no numpy
print('Equação:')
Derivative(f(x))

```

### Plotando gráfico

## **Equação da reta tangente:**

$ y(x) = \left (\dfrac{df}{dx} \right) (x_0) (x-x_0) + f(x_0) $

```
def r1(x): return x
def r2(x): return ( 6 * ( x - 1 ) ) + f(1)

x,y,y2,y3 = list(),list(),list(),list()

for i in np.arange(0,4, step=0.1):
  x.append(i)
  y.append(f(i))
  y2.append(r1(1))
  y3.append(r2(i))

# Gerando canvas
fig = figure(title='Reta Tangente',
             x_axis_label='x',
             y_axis_label='y',
             width=600,
             height=600)

# Desenhando a linha
fig.line(x, y,
         line_alpha=1,
         legend_label='f (x)',
         line_width=3)

# Desenhando a linha
fig.line(x, y2,
         line_alpha=1,
         legend_label='r1 (x)',
         line_width=3,
         color= 'green')

# Desenhando a linha
fig.line(x, y3,
         line_alpha=1,
         legend_label='r2 (x)',
         line_width=3,
         color= 'red')

fig.add_tools(HoverTool(
    tooltips='<font face="Arial" size="3"> x = @x{0.00}, y = @y{0.00}</font>',
    mode='vline'
))

show(fig)

```

# Propriedades das Derivadas

*   As propriedades da derivadas são obtidas das propriedades dos limites

## Derivada da Soma de Funções

A derivada da soma de funções é igual a soma das derivadas de cada função

## Derivada da Diferença de Funções

A derivada da diferença de funções é igual a diferença das derivadas de cada função

## Derivada de uma Constante * f(x)

A derivada de uma constante vezes a função é igual à derivada da função vezes a constante


# Modeling ODE with only absorted solar radiation to predict the change in temperature for 200 years from 1850 and visualizing the model

```python
#from sympy import solve
import matplotlib.pyplot as plt
import numpy as np
from scipy.integrate import solve_ivp

S = 1368
α = 0.3
absorbed_solar_radiation = S*(1 - α)/4
temp0 = 14
C = 51.0

F = lambda t, s: (1/C) * absorbed_solar_radiation
t_span = [0, 210]
s0 = [temp0]
t_eval = np.arange(0, 210, 50)
sol = solve_ivp(F, t_span, s0, t_eval=t_eval)
sol
```

      message: 'The solver successfully reached the end of the integration interval.'
         nfev: 32
         njev: 0
          nlu: 0
          sol: None
       status: 0
      success: True
            t: array([  0,  50, 100, 150, 200])
     t_events: None
            y: array([[ 14.        , 248.70588235, 483.41176471, 718.11764706,
            952.82352941]])
     y_events: None

```python
plt.figure(figsize=(12, 4))
plt.subplot(121)
plt.plot(sol.t, sol.y[0])
plt.title('Absorbing Solar Radiation (only)')
plt.xlabel('Years from 1850')
plt.ylabel('Temperature °C')
plt.show()
```

# Extending the model with thermal radiation and visualizing it

```python
B = 1.3
start_temp = 14 #0->28, default: 14
F1 = lambda t, s: (1/C) * B * (temp0-s)
t_span = [0, 210]
s0 = [start_temp]
t_eval = np.arange(0, 210, 50)
sol = solve_ivp(F1, t_span, s0, t_eval=t_eval)
sol
```

      message: 'The solver successfully reached the end of the integration interval.'
         nfev: 62
         njev: 0
          nlu: 0
          sol: None
       status: 0
      success: True
            t: array([  0,  50, 100, 150, 200])
     t_events: None
            y: array([[14., 14., 14., 14., 14.]])
     y_events: None

```python
plt.figure(figsize=(12, 4))
plt.subplot(121)
plt.plot(sol.t, sol.y[0])
plt.title('Energy Balance Model (Healthy Earth)')
plt.xlabel('Years from start')
plt.ylabel('Temperature °C')
plt.ylim(0, 30)
plt.show()
```

# Extending the model with Greenhouse Effects and visualizing it

```python
def calc_greenhouse_effect(CO2):
  return forcing_coef*np.log(CO2/CO2_PreIndust)

forcing_coef = 5.0
CO2_PreIndust = 280.0
calc_greenhouse_effect(CO2_PreIndust * (1 + np.power((15/220), 3)))
```

    0.0015845535506302491

```python
F2 = lambda t, s: (1/C) * (B * (temp0-s) + calc_greenhouse_effect(CO2_PreIndust * (1 + np.power((t/220), 3))))
sol = solve_ivp(F2, [0, 210], [start_temp], t_eval=np.arange(0, 210, 50))
sol
```

      message: 'The solver successfully reached the end of the integration interval.'
         nfev: 74
         njev: 0
          nlu: 0
          sol: None
       status: 0
      success: True
            t: array([  0,  50, 100, 150, 200])
     t_events: None
            y: array([[14.        , 14.00769389, 14.14788232, 14.57018498, 15.3856948 ]])
     y_events: None

```python
plt.figure(figsize=(12, 4))
plt.subplot(121)
plt.plot(sol.t, sol.y[0])
plt.title('Model with CO₂')
plt.xlabel('Years from 1850')
plt.ylabel('Temperature °C')
plt.ylim(10, 20)
plt.show()
```

```python
tmp_CO2 = []
for i in range(1850, 2021):
  t_year = i - 1850
  CO2_from_1850 = CO2_PreIndust * (1 + np.power((t_year/220), 3))
  tmp_CO2.append(CO2_from_1850)
tmp_CO2
```

```python
plt.figure(figsize=(12, 4))
plt.subplot(121)
plt.plot(range(1850, 2021), tmp_CO2)
plt.show()
```

# Compare with NASA

```python
import pandas as pd

url ='https://data.giss.nasa.gov/gistemp/graphs/graph_data/Global_Mean_Estimates_based_on_Land_and_Ocean_Data/graph.txt'
df = pd.read_csv(url, skiprows=3, sep='\s+').drop(0)
df
```

      Year
      No_Smoothing
      Lowess(5)

      1
      1880
      -0.15
      -0.08

      2
      1881
      -0.07
      -0.12

      3
      1882
      -0.10
      -0.15

      4
      1883
      -0.17
      -0.19

      5
      1884
      -0.27
      -0.23

      ...
      ...
      ...
      ...

      137
      2016
      1.02
      0.88

      138
      2017
      0.92
      0.91

      139
      2018
      0.85
      0.95

      140
      2019
      0.98
      0.98

      141
      2020
      1.02
      1.01

141 rows × 3 columns

      <button class="colab-df-convert" onclick="convertToInteractive('df-420212da-e6ee-4a2f-b03e-1dd1fc9be7fa')"
              title="Convert this dataframe to an interactive table."
              style="display:none;">

  <svg xmlns="http://www.w3.org/2000/svg" height="24px"viewBox="0 0 24 24"
       width="24px">
    <path d="M0 0h24v24H0V0z" fill="none"/>
    <path d="M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z"/><path d="M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z"/>
  </svg>
      </button>

```python
df['Year'] = df['Year'].astype('float64')
df['No_Smoothing'] = df['No_Smoothing'] + 14.15
df
```

      Year
      No_Smoothing
      Lowess(5)

      1
      1880.0
      14.00
      -0.08

      2
      1881.0
      14.08
      -0.12

      3
      1882.0
      14.05
      -0.15

      4
      1883.0
      13.98
      -0.19

      5
      1884.0
      13.88
      -0.23

      ...
      ...
      ...
      ...

      137
      2016.0
      15.17
      0.88

      138
      2017.0
      15.07
      0.91

      139
      2018.0
      15.00
      0.95

      140
      2019.0
      15.13
      0.98

      141
      2020.0
      15.17
      1.01

141 rows × 3 columns

      <button class="colab-df-convert" onclick="convertToInteractive('df-90886347-c33d-4a6e-b9b4-5347137d8bb4')"
              title="Convert this dataframe to an interactive table."
              style="display:none;">

  <svg xmlns="http://www.w3.org/2000/svg" height="24px"viewBox="0 0 24 24"
       width="24px">
    <path d="M0 0h24v24H0V0z" fill="none"/>
    <path d="M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z"/><path d="M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z"/>
  </svg>
      </button>

```python
BB = 1.3 #[0.0, 4.0]
CC = 51.0 #[10.0, 200.0]
F3 = lambda t, s: (1/CC) * (BB * (temp0-s) + calc_greenhouse_effect(CO2_PreIndust * (1 + np.power((t/220), 3))))
solp4 = solve_ivp(F3, [0, 171], [start_temp], t_eval=np.arange(0, 171, 1))
solp4
```

      message: 'The solver successfully reached the end of the integration interval.'
         nfev: 62
         njev: 0
          nlu: 0
          sol: None
       status: 0
      success: True
            t: array([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,
            13,  14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,
            26,  27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,
            39,  40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,
            52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,
            65,  66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,
            78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,
            91,  92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103,
           104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
           117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
           130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
           143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155,
           156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168,
           169, 170])
     t_events: None
            y: array([[14.        , 14.        , 14.00000003, 14.00000018, 14.00000057,
            14.0000014 , 14.00000289, 14.00000534, 14.00000907, 14.00001444,
            14.0000219 , 14.0000319 , 14.00003714, 14.00002733, 14.00000454,
            13.99997096, 13.99992878, 13.99988014, 13.99982717, 13.99977201,
            13.99971674, 13.99966346, 13.99961422, 13.99957108, 13.99953606,
            13.99951117, 13.9994984 , 13.99949974, 13.99951713, 13.99955252,
            13.99960782, 13.99968494, 13.99978576, 13.99991215, 14.00006596,
            14.00024901, 14.00046312, 14.00071009, 14.00099168, 14.00130967,
            14.00166578, 14.00206174, 14.00249926, 14.00298001, 14.00350568,
            14.0040779 , 14.00469832, 14.00536854, 14.00609016, 14.00686476,
            14.00769389, 14.00857911, 14.00952193, 14.01052387, 14.0115864 ,
            14.01271099, 14.01389911, 14.01515218, 14.01647162, 14.01785882,
            14.01931518, 14.02084203, 14.02244074, 14.02411263, 14.025859  ,
            14.02768115, 14.02958034, 14.03155782, 14.03361485, 14.03575262,
            14.03797235, 14.0402752 , 14.04266236, 14.04513495, 14.04769411,
            14.05034095, 14.05307655, 14.055902  , 14.05881835, 14.06182663,
            14.06492786, 14.06812304, 14.07141316, 14.07479919, 14.07828206,
            14.08186271, 14.08554205, 14.08932097, 14.09320035, 14.09718104,
            14.10126388, 14.1054497 , 14.10973929, 14.11413344, 14.11863292,
            14.12323847, 14.12795083, 14.13277071, 14.1376988 , 14.14273578,
            14.14788232, 14.15313904, 14.15850657, 14.16398551, 14.16957646,
            14.17527999, 14.18109663, 14.18702692, 14.19307138, 14.19923051,
            14.20550478, 14.21189465, 14.21840124, 14.22502766, 14.23177706,
            14.23865248, 14.24565686, 14.25279302, 14.26006368, 14.26747146,
            14.27501886, 14.28270829, 14.29054204, 14.2985223 , 14.30665115,
            14.31493057, 14.32336243, 14.33194849, 14.3406904 , 14.34958973,
            14.3586479 , 14.36786626, 14.37724604, 14.38678836, 14.39649424,
            14.40636459, 14.41640023, 14.42660183, 14.43697001, 14.44750524,
            14.4582079 , 14.46907827, 14.48011651, 14.49132268, 14.50269674,
            14.51423853, 14.52594779, 14.53782417, 14.54986718, 14.56207625,
            14.57445069, 14.58698971, 14.59969241, 14.6125578 , 14.62558475,
            14.63877205, 14.65211839, 14.66562232, 14.67928231, 14.69309672,
            14.70706379, 14.72118169, 14.73544843, 14.74986196, 14.76442009,
            14.77912055, 14.79396095, 14.80893879, 14.82405148, 14.8392963 ,
            14.85467045]])
     y_events: None

```python
plt.figure(figsize=(16, 4))
plt.subplot(121)
plt.plot(df['Year'].tolist(), df['No_Smoothing'].tolist(), label='NASA Observations')
plt.plot(range(1850, 2021), solp4.y[0], label='Predicted Temperature from model')
plt.xlabel('Years')
plt.ylabel('Temp °C')
plt.legend()
plt.show()
```

# Improving the model

```python
def calc_alpha(T, alpha_0=0.3, alpha_i=0.5, delta_T=10.0):
  if T < -delta_T:
    return alpha_i
  elif -delta_T <= T < delta_T:
    return alpha_i + (alpha_0 - alpha_i) * (T + delta_T) / (2 * delta_T)
  elif T >= delta_T:
    return alpha_0

F_final = lambda t, s: ((1/85.0) * (BB * (temp0 - s) + \
                       calc_greenhouse_effect(CO2_PreIndust * (1 + np.power((t/220), 3))))) * ((1/85.0) * (S * (1 - calc_alpha(t))/4))
solp_final = solve_ivp(F_final, [0, 171], [start_temp], t_eval=np.arange(0, 171, 1))
solp_final
```

      message: 'The solver successfully reached the end of the integration interval.'
         nfev: 80
         njev: 0
          nlu: 0
          sol: None
       status: 0
      success: True
            t: array([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,
            13,  14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,
            26,  27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,
            39,  40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,
            52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,
            65,  66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,
            78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,
            91,  92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103,
           104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
           117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
           130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
           143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155,
           156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168,
           169, 170])
     t_events: None
            y: array([[14.        , 14.        , 14.00000006, 14.00000029, 14.00000091,
            14.00000219, 14.00000452, 14.00000835, 14.00001423, 14.00002278,
            14.00003471, 14.00005083, 14.00005407, 14.00002087, 13.99995628,
            13.99986551, 13.99975369, 13.99962581, 13.99948676, 13.9993413 ,
            13.99919411, 13.99904971, 13.99891255, 13.99878694, 13.99867707,
            13.99858704, 13.99852083, 13.99848229, 13.99847517, 13.99850311,
            13.99856962, 13.99867811, 13.99883187, 13.99903408, 13.99928781,
            13.99959599, 13.99996148, 14.00038699, 14.00087513, 14.0014284 ,
            14.00204917, 14.00273972, 14.00350219, 14.00433863, 14.00525097,
            14.00624101, 14.00731045, 14.00846087, 14.00969376, 14.01101046,
            14.01241221, 14.01390015, 14.01547529, 14.01713853, 14.01889066,
            14.02073235, 14.02266417, 14.02468655, 14.02679982, 14.02900422,
            14.03129984, 14.03368667, 14.03616459, 14.03873336, 14.04139263,
            14.04414194, 14.04698071, 14.04990824, 14.05292372, 14.05602625,
            14.05921478, 14.06248816, 14.06584514, 14.06928434, 14.07280426,
            14.07640331, 14.08007977, 14.08383181, 14.08765748, 14.09155473,
            14.09552138, 14.09955514, 14.10365362, 14.1078143 , 14.11203455,
            14.11631164, 14.12064183, 14.12502717, 14.12947985, 14.13401192,
            14.13863493, 14.14335987, 14.1481972 , 14.15315684, 14.15824818,
            14.16348008, 14.16886084, 14.17439823, 14.18009951, 14.18597137,
            14.19201997, 14.19825095, 14.20466941, 14.21127988, 14.2180864 ,
            14.22509244, 14.23230096, 14.23971435, 14.24733449, 14.25516272,
            14.26319984, 14.2714461 , 14.27990123, 14.28856441, 14.29743431,
            14.30650903, 14.31578616, 14.32526272, 14.33493523, 14.34479966,
            14.35485143, 14.36508544, 14.37549605, 14.38607707, 14.39682179,
            14.40772295, 14.41877277, 14.42996292, 14.44128454, 14.45272822,
            14.46428403, 14.4759415 , 14.48768961, 14.49951682, 14.51141103,
            14.52335965, 14.53536231, 14.54750225, 14.55980046, 14.57226096,
            14.58488746, 14.59768334, 14.61065168, 14.62379525, 14.63711647,
            14.65061747, 14.66430006, 14.67816573, 14.69221564, 14.70645066,
            14.72087133, 14.73547786, 14.75027016, 14.76524782, 14.78041012,
            14.795756  , 14.8112841 , 14.82699275, 14.84287995, 14.85894338,
            14.87518043, 14.89158813, 14.90816323, 14.92490215, 14.94180099,
            14.95885554, 14.97606126, 14.99341332, 15.01090654, 15.02853544,
            15.04629424]])
     y_events: None

```python
plt.figure(figsize=(16, 4))
plt.subplot(121)
plt.plot(df['Year'].tolist(), df['No_Smoothing'].tolist(), label='NASA Observations')
plt.plot(range(1850, 2021), solp_final.y[0], label='Predicted Temperature from improved model')
plt.xlabel('Years')
plt.ylabel('Temp °C')
plt.legend()
plt.show()
```


# Optimizing Stiffness Constants

Suppose we have $n$ blocks, each of width $w$, at positions $x\in\mathbb{R}^n$.
The blocks are positioned between two walls at $0$ and $l$.
The leftmost block is connected to the left wall via a spring with stiffness coefficient $k_1$,
the rightmost block is connected to the right wall via a spring with stiffness coefficient $k_{n+1}$,
and the $i$th block is connected to the $(i+1)$th block via a spring with stiffness coefficient $k_{i+1}$.
The equilibrium position of all the blocks can be found by solving the optimization problem
\begin{equation}
\begin{array}{ll}
\mbox{minimize} & \frac{1}{2}k_1x_1^2 + \frac{1}{2}k_2(x_2-x_1)^2 + \ldots + \frac{1}{2}k_{n+1}(l-x_n)^2\\[.2cm]
\mbox{subject to} & x_1 \geq w/2, \quad x_n \leq l - w/2, \\
& x_i - x_{i-1} \geq w, \quad i=2,\ldots,n-1,
\end{array}
\label{eq:prob}
\end{equation}
with variable $x$ and solution denoted $x^\star$.
The objective is the potential energy of the system, and the constraints express
the fact that the blocks have a width, and cannot penetrate each other or the walls.

```python
# NOTE: this notebook requires ffmpeg, which can be installed on ubuntu with "sudo apt install ffmpeg"
# and on mac with "brew install ffmpeg"
import cvxpy as cp
import numpy as np
import matplotlib.pyplot as plt
import torch
from cvxpylayers.torch import CvxpyLayer
import matplotlib.patches as patches
from matplotlib import animation, rc
rc('animation', html='html5')
```

We can easily set up this problem as a CVXPY problem, with parameter $k$ and variable $x$:

```python
n = 5
l = 1
w = .05

k = cp.Parameter(n + 1, nonneg=True)
x = cp.Variable(n + 2)
objective = cp.sum(cp.multiply(k, .5 * cp.square(cp.diff(x))))
constraints = [x[0] == 0, x[-1] == l] + [x[i] - x[i-1] >= w for i in range(1, n+2)]
prob = cp.Problem(cp.Minimize(objective), constraints)
```

We can easily convert this problem to a `torch` `CvxpyLayer` in one line.
The layer maps stiffness coefficients to block positions, or $x^\star(k)$.

```python
prob_tch = CvxpyLayer(prob, [k], [x])
```

Our goal is to tune the stiffness coefficients such that the block positions are close to target positions,
or solve the problem
\begin{equation}
\begin{array}{ll}
\mbox{minimize} & \|x^\star(k) - x^\mathrm{targ}\|_2^2,
\end{array}
\end{equation}
with variable $k\in\mathbb{R}^{n+1}$.

```python
torch.manual_seed(0)
k = torch.ones(n+1, dtype=torch.double, requires_grad=True)
x_targ, _ = torch.sort(torch.rand(n, dtype=torch.double))
```

We can (approximately) solve this problem via gradient descent:

```python
x, = prob_tch(k)
x_np = x.detach().numpy()

fig, ax = plt.subplots()
ax.set_xlim((0, 1))
ax.set_ylim((0, w*4))
rects = []
for i in range(n):
    plt.axvline(x_targ[i].item(), c='k')
    rect = patches.Rectangle((x_np[i + 1] - w/2, 0), w, w, linewidth=1, edgecolor='k', facecolor='none')
    ax.add_patch(rect)
    rects.append(rect)

def animate(i):
    k.grad = None
    x, = prob_tch(k)
    x_np = x.detach().numpy()
    loss = (x[1:-1] - x_targ).pow(2).sum()
    loss.backward()
    k.data = k.data - .1 * k.grad.data
    k.data[k.data < 0] = 0.0
    for i in range(n):
        rects[i].xy = (x_np[i + 1] - w/2, 0)
    return rects

anim = animation.FuncAnimation(fig, animate,
                               frames=300, interval=50, blit=True)
anim
```

```python

```


# ODEs in Python

Solving Ordinary Differential Equations in Python is similar to other languages is done by some sort of numerical method. Fortunately for us, Python and other high-level languages such as MATLAB take away much of the leg work required to solve a differential equation and allow us to get to the heart of a problem.

Although many First-Order ODEs have analytical solutions, we'll begin here considering the syntax for ODE solvers in many languages is tricky.

## The RL Circuit

A problem that is often presented in a introductory Physics class is that of the RL circuit (which is short for a resistor-inductor circuit). The equation for current in just such a circuit is

\begin{equation}
    L \frac{d I}{d t} + R I = \mathcal{E}.
\end{equation}

The solution to this equation is readily solvable to show

\begin{equation}
    I = \frac{\mathcal{E}}{R} \left( 1 - e^{-t/\tau} \right)
\end{equation}

where $\tau=L/R$ and the initial current is assumed to be zero.

## ODEs in Python

In this class, we will make use for the [scipy.integrate toolbox](https://docs.scipy.org/doc/scipy/reference/integrate.html). Primarily, we will be accessing the [solve_ivp](https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_ivp.html#scipy.integrate.solve_ivp) (for *initial* value problems) and the [solve_bvp](https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_bvp.html#scipy.integrate.solve_bvp) (for *boundary* value problems).
Through these two functions, Scipy seeks the best type of solver for a particular system. We'll import these and other necessary elements below.

```python
# Python Imports

# 3rd Party Numerical Imports
import numpy as np
from scipy.integrate import solve_ivp as ivp

# 3rd Party Plotting Utilities
from matplotlib import pyplot as plt
```

In general, require the user to do some algrbra to get a differential equation into a set of equations that describe the derivatives. Understanding that this is difficult to comprehend as a simple statement, we'll demonstrate with the RL differential equation by solving for the derivative. First, we'll get every term in units of current by dividing by $R$.

\begin{equation}
    \frac{L}{R} \frac{d I}{d t} + I = \frac{\mathcal{E}}{R}
\end{equation}

which then we can rewrite as

\begin{equation}
    \frac{d I}{d t} = \frac{R}{L} \left( \frac{\mathcal{E}}{R} - I \right)
\end{equation}

or

\begin{equation}
    \frac{d I}{d t} = \frac{I_0 - I}{\tau}
\end{equation}

Now, the ODE solver in Python expects as its first argument a function that packages all the necessary information about the problem we are trying to solve. And, the specified function is meant to take the independent variable as the first argument ($t$ in this case), the dependent variables as a tuple in the second argument ($I$ in this case) and any other variables after that. To see this visually, check the example and input below.

```python
def odefunc(indepVar, depVars, *otherArgs)
```

Fortunately for us, First-Order ODEs only have one independent and one dependent variable which makes the function easier to write.

```python
# The RL Equation
def rleq(t, current, emf, resist, induct):

    # Simply return the RHS of the Equation shown above
    return (emf - resist*current)/induct
```

At this point, we are ready to begin testing this equation against different initial conditions. Before doing so, let's take a look at the necessary arguments for Scipy's IVP solver. Their documentation states that the function call looks like

```python
solve_ivp(odefunc, tSpan, y0, args=None, tEval=None)
```

The function call takes other arguments which we will not utilize here. However, those listed above are explained below.

* `odefunc`: This represents the name of the function to be evaluated. The IVP solver will call this function to create the solution.
* `tSpan`: This is meant to be a two-tuple meant to represent the start and stop times.
* `y0`: The initial condition(s) of the system as a tuple or list
* `args`: A tuple of arguments meant to be passed on to the `odefunc`. Note: **they must be in the same order as they appear in the `odefunc`**.
* `tEval`: Used to specify specific times to evaluate.

Now that we're aware of these, let's solve the charging problem under various initial conditions.

```python
# Set the values of the variables
# We'll set everything to 1 to allow for greater numerical precision.
R = E = L = 1

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
I0 = [0]

# Set Time Specs
tSpan = (0, 5)
tEval = np.linspace(tSpan[0], tSpan[1], 100)

# Solve the Problem
sol = ivp(rleq, tSpan, I0, args=(E, R, L), t_eval=tEval)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(sol.t, sol.y[0], linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_xlabel('t/$\\tau$', fontsize=14)
_ = ax.set_ylabel('$I/I_0$', fontsize=14)
_ = ax.set_title('Charging RL Circuit with no Intial Current', fontsize=16)
_ = ax.grid(True)
```

Now, let's assume that the initial current is twice that than the current carrying capacity of the emf.

```python
# Set the values of the variables
# We'll set everything to 1 to allow for greater numerical precision.
R = E = L = 1

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
I0 = [2]  # Note the change here

# Set Time Specs
tSpan = (0, 5)
tEval = np.linspace(tSpan[0], tSpan[1], 100)

# Solve the Problem
sol = ivp(rleq, tSpan, I0, args=(E, R, L), t_eval=tEval)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(sol.t, sol.y[0], linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_ylim(-0.1, 2.1)
_ = ax.set_xlabel('t/$\\tau$', fontsize=14)
_ = ax.set_ylabel('$I/I_0$', fontsize=14)
_ = ax.set_title('Charging RL Circuit with 2x Intial Current', fontsize=16)
_ = ax.grid(True)
```

Likewise, we can solve the discharging equation where we assume the emf has been removed from the system and that the current begins with some value $I_0 = \mathcal{E}/R$.

```python
# Set the values of the variables
# We'll set everything to 1 to allow for greater numerical precision.
R = L = 1
E = 0  # Note the change here

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
I0 = [1]  # Note the change here

# Set Time Specs
tSpan = (0, 5)
tEval = np.linspace(tSpan[0], tSpan[1], 100)

# Solve the Problem
sol = ivp(rleq, tSpan, I0, args=(E, R, L), t_eval=tEval)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(sol.t, sol.y[0], linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_xlabel('t/$\\tau$', fontsize=14)
_ = ax.set_ylabel('$I/I_0$', fontsize=14)
_ = ax.set_title('Discharging RL Circuit with Intial Current', fontsize=16)
_ = ax.grid(True)
```

## Coupled, First-Order Differential Equations

When it comes to solving coupled or high-order differential equations, the functions and their calls become a little more complicated. Now, the dependent variables must be accessed as a list within the function call. To demonstrate, we'll use the same problem within the Scipy documentation: the Lotka-Volterra (Predator-Prey) Equations which are written as

\begin{align}
    \frac{d x}{d t} &= \alpha x - \beta x y \\
    \frac{d x}{d t} &= \delta x y - \gamma y
\end{align}

where $x$ represents the number or prey and $y$ represents the number of predators.

Fortunately for us, these equations are already in a form that Scipy will accept. Therefore, all we have to do is program it into our system.

```python
# The Predator-Prey Equation
def lveq(t: float, orgNums: tuple, a: float, b: float, c: float, d: float):

    # Unpack
    prey, pred = orgNums

    # Return the Solutions
    sol = (
        a*prey - b*prey*pred,
        d*pred*prey - c*pred
    )
    return sol
```

```python
# Set the values of the variables
# Varibles set based on Scipy docs
a = 1.5
b = 1.0
c = 3.0
d = 1.0
args = (a, b, c, d)

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
initOrgCnts = [10, 5]

# Set Time Specs
tSpan = (0, 15)
tEval = np.linspace(tSpan[0], tSpan[1], 300)

# Solve the Problem
sol = ivp(lveq, tSpan, initOrgCnts, args=args, dense_output=True)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(tEval, sol.sol(tEval).T, linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_xlabel('Time', fontsize=14)
_ = ax.set_ylabel('Organism Counts', fontsize=14)
_ = ax.set_title('LV System', fontsize=16)
_ = ax.grid(True)
_ = ax.legend(['Prey', 'Predator'], fontsize=12)
```

## Assignment

### First Order Equation

Find a numerical solution to the differential equation

\begin{equation}
    2 \frac{d y}{d t} - \frac{y}{t} - \gamma \frac{t^2}{y} = 0
\end{equation}

with the initial condition $y(1) = 2$ and $\gamma = 1$. Plot this against the known solution

\begin{equation}
    y(t) = \sqrt{\frac{x^3 + 7x}{2}}
\end{equation}

Write the function for Scipy below.

```python
def diffeq(t, y, g):

    # Return the Solution
    return y/(2*t) + g*t**2 / (2*y)
```

Write the code to solve the equation here.

```python
# Set the values of the variables
# Varibles set based on Scipy docs
g = 1
args = (g,)

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
inits = [2]

# Set Time Specs
tSpan = (1, 15)
tEval = np.linspace(tSpan[0], tSpan[1], 300)

# Solve the Problem
sol = ivp(diffeq, tSpan, inits, args=args, t_eval=tEval)
yExact = np.sqrt((tEval**3 + 7*tEval)/2)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(sol.t, sol.y[0], linewidth=2)
_ = ax.plot(tEval, yExact, '--', linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_xlabel('Time', fontsize=14)
_ = ax.set_ylabel('y(t)', fontsize=14)
_ = ax.set_title('Differential Equation Assignment', fontsize=16)
_ = ax.grid(True)
_ = ax.legend(['Numerical', 'Exact'], fontsize=12)
```

### First Order Coupled Equations

When tanks of fluid are connected to each other such that the water is cycled through the system, we also get coupled differential equations. Your assignment is to solve the following differential equations numerically which represent the flow of water from one tank to another.

\begin{align}
    \dot{x}_1 &= -a x_1         + c x_3, \\
    \dot{x}_2 &=  a x_1 - b x_2,         \\
    \dot{x}_3 &=          b x_2 - c x_3
\end{align}

for $a=1/6$, $b=1/3$ and $c=1/6$. Also, the initial conditions for the system are $x_1(0) = 0$, $x_2(0) = 2$ and $x_3(0) = 3$.

```python
def coupleddiffeq(t, X, a, b, c):

    # Unpack
    x1, x2, x3 = X

    # Return Solution
    return (
        c*x3 - a*x1,
        a*x1 - b*x2,
        b*x2 - c*x3
    )
```

And write the code to solve the differential equation below.

```python
# Set the values of the variables
# Varibles set based on Scipy docs
a = 1/6
b = 1/3
c = 1/6
args = (a, b, c)

# Set the initial value of the current
# Assumed to be in units of I/I0
# Keep in mind that Scipy expects a list even if there's only one value
inits = [0, 2, 3]

# Set Time Specs
tSpan = (0, 20)
tEval = np.linspace(tSpan[0], tSpan[1], 300)

# Solve the Problem
sol = ivp(coupleddiffeq, tSpan, inits, args=args, t_eval=tEval)

# Plot the Solution
fig, ax = plt.subplots(figsize=(10, 7))
_ = ax.plot(sol.t, sol.y.T, linewidth=2)
_ = ax.set_xlim(tSpan)
_ = ax.set_xlabel('Time', fontsize=14)
_ = ax.set_ylabel('x, y', fontsize=14)
_ = ax.set_title('Differential Equation Assignment - Coupled', fontsize=16)
_ = ax.grid(True)
_ = ax.legend(['$x_1$', '$x_2$', '$x_3$'], fontsize=12)
```

If you solved/setup this problem correctly, then you should find that that fluid in each tank comes to an equilibrium point about 15 seconds in and that this equilibrium should have ratio $2:1:2$ for $x_1:x_2:x_3$.


```python
import numpy as np
```

```python
A = np.array([[2,-3,1],[1,-2,1],[3,1,-2]])
B = np.array([[2],[7],[10]])
```

```python
np.linalg.inv(A).dot(B)
```

    array([[19.5],
           [24.5],
           [36.5]])

```python
from sympy import *
A1 = Matrix(A)
A1.rref()
```

    (Matrix([
     [1, 0, 0],
     [0, 1, 0],
     [0, 0, 1]]), (0, 1, 2))

```python
Aug = np.array([[2,-3,1,2],[1,-2,1,7],[3,1,-2,10]])

Aug2 = Matrix(Aug)
Aug2.rref()
```

    (Matrix([
     [1, 0, 0, 39/2],
     [0, 1, 0, 49/2],
     [0, 0, 1, 73/2]]), (0, 1, 2))

```python
np.column_stack([A,B])
```

    array([[ 2, -3,  1,  2],
           [ 1, -2,  1,  7],
           [ 3,  1, -2, 10]])

```python
v1 = np.array([1,2,-1,2])
v2 = np.array([-3,3,1,0])
v3 = np.array([-1,7,-1,4])

```

```python
Matrix(np.vstack([v1,v2,v3])).rref()
```

    (Matrix([
     [1, 0, -5/9, 2/3],
     [0, 1, -2/9, 2/3],
     [0, 0,    0,   0]]), (0, 1))

# Finding r

```python
m = Matrix(np.vstack([v1,2*v2,v2,v1+v2,v3,v1-v3]))
(m.T).rref()
```

    (Matrix([
     [1, 0,   0,   1,   2,   -1],
     [0, 1, 1/2, 1/2, 1/2, -1/2],
     [0, 0,   0,   0,   0,    0],
     [0, 0,   0,   0,   0,    0]]), (0, 1))