#  **II Escuela de Verano en Física Computacional**

Este material está sujeto a los términos y condiciones de la licencia [Creative Commons CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Se permite el uso gratuito para cualquier propósito no comercial.

*También puede consultar la última versión de este notebook en nuestro [repositorio](https://github.com/PCPUNMSM) y los videos de clase [en nuestro canal de Youtube](https://www.youtube.com/channel/UCze8vzQLcplutz0nWDNjFCA).*

#  Lista de Ejercicios

### Problema 1:  Raíces de ecuaciones

La siguiente fórmula nos describe la velocidad ascendente v de un cohete:

$$v(t)=uln\left(\frac{m}{m-qt}\right)-gt$$

Donde:

* u = módulo de la velocidad del combustible expulsado en relación con el cohete

* m = masa inicial del cohete (en t=0)

* g = módulo aceleración de la gravedad

* t = tiempo

Dado:

$$u= 950m/s,\quad m=2x10^5 kg, \quad q=3x10^3 kg/s, \quad g=9.8m/s^2, \quad t \in [10,25]$$

a) Defina y utilice 3 métodos para hallar el tiempo que tarda el cohete en alcanzar la velocidad de 500 m/s.

b) Resuelva analíticamente y compare resultados con los 3 métodos utilizados.

c) Elabore gráficas y detalle su análisis.

```python
import numpy as np
import matplotlib.pyplot as plt

u = 950 #m/s
m = 2E5 #kg
q = 3E3 #kg/s
g = 9.8 #m/s^2

def v(t):
    return  u*np.log(m/(m-q*t))-g*t

def f(t):
    # esta es la función v(t)-500, porque queremos encontrar la raíz, i.e. el tiempo t* tal que v(t*)-500=0 o bien
    # v(t*)=500.
    return v(t)-500
#graficamos y notamos que la raíz está aproximadamente en t=40

t_plot = np.linspace(20,50,500)
plt.plot(t_plot,v(t_plot))
plt.grid(True)
plt.title('Velocidad de un cohete')
plt.xlabel('Tiempo (s)')
plt.ylabel('Velocidad (m/s)')
plt.show()

def dfdt(t):
    h = 1E-5
    return (f(t+h/2)-f(t-h/2))/h #diferencia central

# Cuando se alcance la precisión, los programas se detienen
accuracy = 1E-10

# Método de Newton
def newton_method(t):
    delta = 1.0
    while abs(delta) > accuracy:
        delta = f(t)/dfdt(t)
        t -= delta
    return t

t_newton = newton_method(40)
print(f"Según método de Newton, el cohete alcanza la velocidad de 500m/s en el tiempo {t_newton}s")

#método secante
def secant_method(t,t0):
    delta = 1.0
    while abs(delta) > accuracy:
        dfdt = (f(t)-f(t0))/(t-t0)
        delta = f(t)/dfdt
        t0 = t
        t -= delta
    return t

t_secant = secant_method(30,60)
print(f"Según método secante, el cohete alcanza la velocidad de 500m/s en el tiempo {t_secant}s")

# bisección
def biseccion(f, t1,t2,accuracy): #f(t1) y f(t2) deben tener signos distintos
    while abs(t1-t2) > accuracy:
        t = (t1+t2)/2
        if np.sign(f(t))==np.sign(f(t1)):
            t1 = t
        elif np.sign(f(t))==np.sign(f(t2)):
            t2 = t
    return t

t_bisec = biseccion(f,30,60,accuracy)
print(f"Según método de bisección, el cohete alcanza la velocidad de 500m/s en el tiempo {t_bisec}s")

## Analíticamente, de acuerdo con Mathematica, el resultado es t=40.8159 s, de manera que las soluciones aquí dadas se aproximan bastante.
## Hacer el valor de la precisión (accuracy) más pequeño puede ayudar a encontrar soluciones más cercanas, aunque estas son bastante buenas.

```

Me tomé la libertad de escribir mis propios programas (con ayuda del libro de Mark Newman "Computational Physics") para practicar un poco e investigar más a fondo las ventajas y desventajas de cada uno de estos métodos.

Analíticamente, de acuerdo con Mathematica, el resultado es $t=40.8159 s$, de manera que las soluciones aquí dadas se aproximan bastante.
Hacer el valor de la precisión (accuracy) más pequeño puede ayudar a encontrar soluciones más cercanas, aunque estas son bastante buenas.
Lo que hicimos fue definir la función $f(t)=v(t)-500$ y hallar la raíz mínimo, ya que esto da el tiempo $t_0$ tal que $f(t_0)=v(t_0)-500=0$, o bien, $v(t_0)=500$, que es lo que queremos (ignoro unidades por simplicidad).

El método de bisección (o búsqueda binaria), aquí utilizado, es un método robusto. Uno específica un intervalo en el que desea encontrar la solución a una ecuación. En este caso tuvimos que graficar para saber grosso modo en qué región se encuentra dicha solución. Supongámos que nuestros puntos iniciales $t_1$ y $t_2$ se encuentran a una distancia $\delta$; la distancia se disminuye a la mitad a cada paso de la búsqueda binaria, por lo que después de $N$ pasos, la distancia es $\delta/2^N$. Si tenemos, por ejemplo, puntos a una distancia inicial de $\delta=10^{10}$ con una precisión de $10^{-10}$, el número de pasos necesario para encontrar la raíz es $67$, que no requiere mucho tiempo de cómputo. Este método tiene sus desventajas, por ejemplo, si entre los puntos iniciales hay un número par de raíces. Por otro lado, a veces no es posible graficar para buscar un punto que a grosso modo sea cercano a la solución.

El método de Newton requiere un solo valor inicial. Basado en este valor, usa la pendiente en esa posición para extrapolar y hallar un segundo punto que usualmente será más cercano a la solución que el primero (si uno tiene mala suerte, puede que sea más lejano). El método de Newton tiene converfencia cuadrática, lo que implica que converge extremadamente rápido. Las desventajas son que requiere que conozcamos la derivada de la función (podemos usar derivación numérica) y que no siempre converge (esto depende del valor de la derivada).

El método de la secante arregla algunos problemas que se tienen en el de Newton, pues si no conocemos la derivada simplemente podemos calcularla numéricamente usando la secante (diferencia finita) en lugar de la tangente (derivada). Son formas similares; el método de la secante también converge con rapidez y tiene las mismas desventajas. También puede generalizarse a la solución de ecuaciones simultáneas no lineales, aunque la generalización puede ser complicada.

### Problema 2: Interpolación

Dadas las funciones:

1. $f (x) = sin(x^2) $

2. $f (x) = e^{sin (x)}$

3. $f (x) = \frac{0.2}{(x-3.2)^2 + 0.04}$

a) Estudiar el error en dos métodos de interpolación (de su elección) con 10,20 y 50 datos (uniformemente distribuidos) en el intervalo [0.0, 5.0] y compare los resultados.

b) Hallar los órdenes de convergencia, elabore su gráfico con el eje x en base 2 y detallar su análisis.

c) Añade una perturbación sinusoidal ($\delta g(x)=sin(x)$) del orden de $10^{-4},10^{-5}, 10^{-6}$ con distribución homogénea, grafice y detalle su análisis, ¿cuál es el error que generan con respecto a aproximación hecha en "2.a)"?

```python
# import numpy as np
# import numpy.polynomial.polynomial as poly
# import matplotlib.pyplot as plt
# import sympy as sym

# # Interpolacion de Lagrange
# def f1(x):
#   return np.sin(x**2)

# def f2(x):
#   return np.exp(np.sin(x))

# def f3(x):
#   return 0.2/((x-3.2)**2+0.04)

# def lagrange(f,n):
#   # n es número de puntos y f la función
#   xi = np.linspace(0.0,5.0,n)
#   fi = f(xi)
#   # PROCEDIMIENTO

#   n = len(xi)
#   x = sym.Symbol('x') #variable simbólica
#   polinomio = 0
#   divisorL = np.zeros(n, dtype = float)

#   for i in range(0,n,1):
#       # Termino de Lagrange
#       numerador = 1
#       denominador = 1
#       for j  in range(0,n,1):
#           if (j!=i):
#               numerador = numerador*(x-xi[j])
#               denominador = denominador*(xi[i]-xi[j])
#       terminoLi = numerador/denominador

#       polinomio = polinomio + terminoLi*fi[i]
#       divisorL[i] = denominador

#   # simplifica el polinomio
#   #Dado un polinomio, expand() lo pondrá en una forma canónica de una suma de monomios.
#   polisimple = polinomio.expand()

#   # para evaluación numérica
#   px = sym.lambdify(x,polisimple)

#   # Puntos para la gráfica
#   muestras = 101
#   a = np.min(xi)
#   b = np.max(xi)
#   pxi = np.linspace(a,b,muestras)
#   pfi = px(pxi)

#   # SALIDA
#   print(f'Polinomio de Lagrange para función {f.__name__} con {n} datos: ')
#   print(polisimple)

#   # Gráfica
#   plt.plot(xi,fi,'o', label = 'Puntos')
#   plt.plot(pxi,pfi, label = 'Polinomio')
#   plt.legend()
#   plt.xlabel('xi')
#   plt.ylabel('fi')
#   plt.grid()
#   plt.title('Interpolación Lagrange')
#   plt.show()

# for n in [10,20,50]:
#   for func in [f1,f2,f3]:
#     lagrange(func,n)
```

### Problema 3: Derivación numérica

En 1928, George Gamow propuso una teoría muy exitosa del decaímiento $\alpha$, según la cual la partícula $α$ se mueve libremente dentro del núcleo y se emite después de atravesar la barrera potencial entre ella y el núcleo hijo. En otras palabras, la partícula $α$, de carga $Z_2$, cuya energía es $E$, está atrapada en un pozo potencial de radio R por la barrera potencial:

$$V(r)=\frac{Z_1Z_2e^2}{4\pi\epsilon_0 r}$$

para $r>R$.

$Z_1 =Z-2, \quad Z_2=2$, donde Z es la carga del núcleo de radio R.

$r$: es el radio fuera del núcleo ($r>R$ y con A, número de masa) donde:

$$R = 1.5\times 10^{-15}\,A^{1/3}\,{\rm m}=2.0\times 10^{-15}\,Z_1^{\,1/3}\,{\rm m}$$

a) Calcule la fuerza que experimenta la partícula y analice $\alpha$ para $r\in [R,3R]$, utilizando los tres esquemas de diferencias finitas.

b) Comparar los esquemas con la solución analítica, calcular los errores, órdenes de convergencia y detallar su análisis.

```python
import numpy as np
import matplotlib.pyplot as plt

# Sistema internacional
epsilon0 = 8.85418782E-12
e = 1.60217662E-19

Z = 20 # Número de protones
Z2 = 2
Z1 = Z-2
R = 2E-15*Z1**(1/3)
N = 2
h0 = R*1*10**(-N)
# Este valor de h0 quiere decir que por cada valor de r,
# para evaluar la derivada estudiaremos dicho valor +
# 1/10^n del valor de R

def V(r):
    return Z1*Z2*e**2/(4*np.pi*epsilon0*r)

def derivative(f,a,method='centrada',h=h0):
    if method == 'centrada':
        return (f(a + h) - f(a - h))/(2*h)
    elif method == 'adelante':
        return (f(a + h) - f(a))/h
    elif method == 'atrás':
        return (f(a) - f(a - h))/h
    else:
        raise ValueError("Los métodos pueden ser 'centrada', 'adelante' o 'atrás'.")

# Derivada analítica de potencial V para calcular los errores
def DV(r):
    return -Z1*Z2*e**2/(4*np.pi*epsilon0*r**2)

# Nuestro dominio va de R a 3R
r = np.linspace(R,3*R,100)
DV_c = derivative(V,r)
DV_ad = derivative(V,r,method='adelante')
DV_at = derivative(V,r,method='atrás')

plt.plot(r,DV_c)
plt.plot(r,DV_ad)
plt.plot(r,DV_at)
plt.plot(r,DV(r))
plt.legend(['DF centrada','DF hacia adelante','DF hacia atrás','Analítica'])
plt.title('Modelo de Gamow')
plt.ylabel('Fuerza')
plt.xlabel('Distancia')
plt.grid()
plt.show()

# Calculamos errores comparando derivada real y derivadas numéricas
c_err = DV(r)-DV_c
ad_err = DV(r)-DV_ad
at_err = DV(r)-DV_at

plt.plot(r,c_err)
plt.plot(r,ad_err)
plt.plot(r,at_err)
plt.legend(['DF centrada','DF hacia adelante','DF hacia atrás'])
plt.title('Errores de truncamiento para distintas DFs')
plt.ylabel('$O(h^2)$')
plt.xlabel('θ')
plt.grid()
plt.show()

h_l=[] #lista de tamaños de pasos
errc_l=[]
errad_l=[]
errat_l=[]
for i in range(10,10000,100):
  a = R
  b = 3*R
  h = 1/i*R #tamaño del paso
  N = round((b-a)/h) #redondear el resultado, el cual será la cantidad de datos
  r = np.linspace(a,b,N)
  DV_c=derivative(V,r, h=h)
  DV_ad=derivative(V,r, h=h,method='adelante')
  DV_at=derivative(V,r, h=h,method='atrás')
  #tomamos los máximos errores máximos por iteración sobre el dominio [a,b]
  c_err=np.max(np.abs(DV(r)-DV_c))
  ad_err=np.max(np.abs(DV(r)-DV_ad))
  at_err=np.max(np.abs(DV(r)-DV_at))

  errc_l.append(c_err)
  errad_l.append(ad_err)
  errat_l.append(at_err)
  h_l.append(h)

plt.loglog(h_l,errc_l)
plt.loglog(h_l,errad_l)
plt.loglog(h_l,errat_l)

plt.legend(['DF centrada','DF hacia adelante','DF hacia atrás'])
plt.title('log(Error absoluto) vs log(tamaño de paso)')
plt.xlabel('log(h)')
plt.ylabel('$log(e_{abs})$')
plt.grid(True, which="both", ls="-")
plt.show()
```

La diferencia central es similar a la diferencia hacia atrás y hacia adelante salvo por el hecho de que los dos puntos están posicionados simétricamente alrededor del valor del cual queremos conocer la derivada. El método de derivada central es tanto más adecuado. Por ejemplo, si $f(x)$ y $f'''(x)$ son del orden de la unidad, el valor ideal de $h$ será típicamente de $10^{-5}$, pero el error mismo será del orden de $10^{-10}$, bastante bueno para la mayoría de nuestros propósitos. Este comportamiento se observa en las gráficas: para el mismo valor de $h$ los órdenes de convergencia para la diferencia hacia atrás y hacia adelante son prácticamente iguales, pero para la diferencia central es, en el peor de los casos, un par de órdenes de magnitud mejor. Claro que una aproximación a orden superior en Taylor daría mejores resultados, pero ya que buscamos un balance entre precisión y tiempo de cómputo, una diferencia central es el mejor acercamiento a este orden. Es evidente que cuando $h$ es grande, ambos métodos fallarán bastante. Lo importante es notar cómo la diferencia central es una mejor elección independientemente del valor de $h$. En la gráfica de los errores de truncamiento observamos que para valores grandes de $r$ los tres métodos son buenos; sin embargo, cuando $r$ es cercano a $R$, los métodos de diferencia hacia atrás y hacia adelante muestran errores de orden cuadrático relativamente grandes. Esto puede ser más evidente en otras funciones y, como se ve en este caso, principalmente cerca de divergencias.

### Problema 4: Integración numérica

La carga a partir de la densidad radial de carga lineal, $\lambda$, sobre el área de la sección transversal del cilindro, está definida por:

$$q_{total}=\int_A \lambda(r)dA(r)$$

donde, dA es el diferencial de área de la base del cilindro:

$$q_{total}=\int_A \lambda(r)(2\pi r dr)$$

Sabemos que la densidad tiene la forma:

$$\lambda(r)=\lambda_0 \left(1-\frac{r^2}{a^2} \right)$$

Y el campo eléctrico:

$$\vec{E}=\frac{kq_{total}}{d^2}\hat r$$

d: distancia de la superficie al espacio a evaluar el campo eléctrico. ($d\gt r$)

a) Determina la distribución de la carga y campo eléctrico (a distancias $d \in \{ r+r/4,r+r/2\}$ ) en función al radio utilizando 3 métodos de integración.

b) Obtener la solución analítica, comparar los errores de distintos métodos y detallar análisis.

```python
import numpy as np
import matplotlib.pyplot as plt
import random as random

l_0 = 1
k = 1
a = 1 # radio del cilindro
# Hago 1 estos valores ya que no los proporcionaron
x1 = 0 # límites de integración desde 0 hasta el radio del cilindro
x2 = a # que elegimos como 1

def l(r):
    return l_0*(1-r**2/a**2)

# Integrando
def f(r):
    return l(r)*2*np.pi*r

#montecarlo
def montecarlo(f,b):
    n = 100 #número de subintervalos del intervalo [a, b]
    N = 100 #número de muestras
    x = np.zeros(n+1) #valores de x
    h = (x2-x1)/n #tamaño del paso
    total = 0
    random.seed(55)
    for i in range(n):
        x[i+1] = x[i] + h #actualizando el valor de x
        f_mean = 0 #initialización
        for k in range(N):
            x_k = random.uniform(x[i],x[i+1]) #número aleatorio dentro del subintervalo
            f_mean += f(x_k)
        sub_area = (h/N)*f_mean #promedio para el sub-intervalo
        total += sub_area #promedio del intervalo
    return total

# método del trapezoide
def int_trapezoide(f,n,x2):
    N = 2**(n-1) # num de divisiones
    h = (x2-x1)/N # tamaño de divisiones
    if n == 1:
        return 0.5*h*(f(x1)+f(x2))
    else:
        I = 1/2*int_trapezoide(f,n-1,x2)
        for k in range(1,N,2):
            I += h*f(x1+k*h)
        return I

# método de simpson
def int_simpson(f,n,x2):
    N = 2**(n-1) #num de divisiones
    h = (x2-x1)/N #tamaño de las divisiones
    def t(n):
        N = 2**(n-1)
        h = (x2-x1)/N #tamaño de las divisiones
        t = 0
        for k in range(1,N,2):
            t += 2/3*f(x1+k*h)
        return t
    def s(n):
        if n == 1:
            return 1/3*(f(x1)+f(x2))
        else:
            return s(n-1)+t(n-1)
    return h*(s(n)+2*t(n))

# tomamos k=1 y el radio del cilindro como r=1
def E_1(d,method):
    if method == 'trapezoide':
        dist_carga = int_trapezoide(f,8,x2)
    elif method == 'simpson':
        dist_carga = int_simpson(f,8,x2)
    elif method == 'montecarlo':
        dist_carga = montecarlo(f,x2)
    return k*dist_carga/d**2

print(f'La carga segun método trapezoide es {int_trapezoide(f,8,x2)}')
print(f'La carga segun método simpson es {int_simpson(f,8,x2)}')
print(f'La carga segun método montecarlo es {montecarlo(f,x2)}\n')

# No entendí muy bien pero supongo que aquí se refieren a que
# r es el radio del cilindro?

# Distancia
d = [1+1/4,1+1/2]
# Sol analítica para distancia 1 y distancia 2
sol_analitica = [1.00531,0.698132]
for method in ['trapezoide','simpson','montecarlo']:
  for i,val in enumerate(d):
        print(f'El campo elécrico a una distancia {val} del origen es {E_1(val,method)} usando el método {method}')
        print (f'Error = 1-sol. obtenida/sol.analítica={abs(1-E_1(val,method)/sol_analitica[i])}\n')
```

    La carga segun método trapezoide es 1.5707004529956536
    La carga segun método simpson es 1.5707963267948963
    La carga segun método montecarlo es 1.57089292714826

    El campo elécrico a una distancia 1.25 del origen es 1.0052482899172184 usando el método trapezoide
    Error = 1-sol. obtenida/sol.analítica=6.138413303513612e-05

    El campo elécrico a una distancia 1.5 del origen es 0.6980890902202905 usando el método trapezoide
    Error = 1-sol. obtenida/sol.analítica=6.146370558779424e-05

    El campo elécrico a una distancia 1.25 del origen es 1.0053096491487337 usando el método simpson
    Error = 1-sol. obtenida/sol.analítica=3.489980864301856e-07

    El campo elécrico a una distancia 1.5 del origen es 0.6981317007977317 usando el método simpson
    Error = 1-sol. obtenida/sol.analítica=4.285754962030097e-07

    El campo elécrico a una distancia 1.25 del origen es 1.0053714733748864 usando el método montecarlo
    Error = 1-sol. obtenida/sol.analítica=6.114867542006408e-05

    El campo elécrico a una distancia 1.5 del origen es 0.6981746342881155 usando el método montecarlo
    Error = 1-sol. obtenida/sol.analítica=6.106909311642816e-05

La regla del trapezoide permite calcular integrales aproximando las funciones a orden lineal en intervalos pequeños (en este caso definidos por la distancia $h$). Esto explica su nombre. Es una aproximación no muy buena porque para funciones que cambian rápidamente en intervalos pequeños puede dar errores grandes. El método de Simpson, por su parte, simplemente mejora esta idea al considerar funciones cuadráticas y no sólo lineales entre los intervalos definidos por el programa. En ambos casos se puede hacer el cálculo más preciso al incrementar el número de divisiones. Comparando los programas arriba vemos que el método de Simpson es ligeramente más complicado que la regla del trapezoide, pero no por mucho.

Ha de tenerse en cuenta que nuestras integrales son sólo aproximaciones. En primer lugar, está un error de redondeo; sin embargo, es más importante el error de aproximación.  El error de la regla del trapezoide lo da la fórmula de Euler-Maclaurin, que es de orden cuadrático (y el siguiente orden es cuádrico porque los términos con $h^3$ se cancelan). Hay poco interés en hacer $h$ tan pequeño que el error de aproximación se vuelva despreciable a comparación del error de redondeo; esto sólo hace lento al programa sin mejorar la precisión. Es por eso que tenemos que ser cuidadosos al escoger $h$. Para la regla de Simspon, se llega a que el error es del orden de $h^4$, que es significativamente mejor que el caso del trapezoide. Hay casos aislados en los que la regla del trapezoide puede ser mejor que la de Simpson, por lo que es útil conocer ambos métodos.

El método de Monte Carlo, por su parte, utiliza el concepto de aleatoriedad. Se toma un punto al azar, que puede caer por debajo o por encima de la curva. Si se hace esto con muchos puntos (digamos $N$) y $k$ de ellos caen debajo de la curva dentro de un rectángulo de área $A$, la integral queda aproximada por $I \approx kA/N$. La principal desventaja de Monte Carlo es que no da resultados muy precisos, pues requiere un gran poder computaacional para ello. Sin embargo, es útil en funciones patológicas en las que aproximar a orden lineal o cuadrático no da ninguna ventaja. Para ser más específicos, el error varía con $N$ como $N^{1/2}$, de modo que la tasa de convergencia es bastante baja.

Para este caso particular ya que es una función sencilla de integrar, vemos que con poco esfuerzo (i.e. pocos pasos para cada método) se llegan a resultados prácticamente iguales al analítico y entre sí. (El integrando va como $r-r^3/a^2$.) El error es más pequeño para el método Simpson —orden $10^{-7}$—, como es de esperarse por lo ya argumentado. Los errores de la regla del trapezoide y Monte Carlo son bastante similares —orden $10^{-5}$—.

### Problema 5:

Considerar un aro horizontal sin rozamiento de radio R. En este aro enhebro tres objetos con masas idénticas m, y, entre los objetos, tres muelles idénticos, cada uno con una masa constante de fuerza k.

Sabemos que, la energía cinética es:

$$T=\frac{1}{2}mR^2(\dot{\phi_1^2}+\dot{\phi_2^2}+\dot{\phi_3^2})$$

y la energía potencial:

$$U=\frac{1}{2}kR^2 [(\phi_1-\phi_2)^2+(\phi_2-\phi_3)^2+(\phi_3-\phi_1)^2]$$

Resultado: Sistema de EDOs

$$m\frac{d^2\phi_1}{dt^2}=-k(\phi_1-\phi_2)-k(\phi_1-\phi_3)$$

$$m\frac{d^2\phi_2}{dt^2}=-k(\phi_2-\phi_3)-k(\phi_2-\phi_1)$$

$$m\frac{d^2\phi_3}{dt^2}=-k(\phi_3-\phi_1)-k(\phi_3-\phi_2)$$

a) Encontrar las ecuaciones de movimiento para cada masa usando las ecuaciones de Euler-Lagrange. *(ejemplo en PCP_sesión5.ipynb)*

b) Resolver numéricamente el sistema de ecuaciones con los métodos de Euler y RK de orden 4. Para los datos: $\phi_1=\phi_2/2,\quad \phi_2=\phi_3/2,\quad \phi_3=\pi/10 \quad $ y $\quad \phi_1'=1,\quad \phi_2'=1,\quad \phi_3'=1, \quad R=m=k=1$

c) Mostrar los resultados y explicar la dinámica del sistema en base al resultado de la simulación.

```python
import numpy as np

steps = 300     # número de pasos de tiempo calculados
endtime = 6.0   # fin de la simulación
R=m=k=1

#########################################3
#EULER

# crear matrices 1D de pasos de longitud +1 para tiempo (t), posición (x), velocidad (v)
t = np.zeros(steps+1)
phi1 = np.zeros(steps+1)
phi2 = np.zeros(steps+1)
phi3 = np.zeros(steps+1)
v1 = np.zeros(steps+1)
v2 = np.zeros(steps+1)
v3 = np.zeros(steps+1)

# inicializar variables en el tiempo = 0.0
phi3[0] = np.pi/10.0
phi2[0] = phi3[0]/2
phi1[0] = phi2[0]/2
v1[0] = 1.0
v2[0] = 1.0
v3[0] = 1.0
t[0] = 0.0

dt = endtime / float(steps)
for i in range (steps):
   t[i+1] = t[i] + dt
   phi1[i+1] = phi1[i] + v1[i]*dt
   v1[i+1] = v1[i] +(-k/m*(phi1[i]-phi2[i])-k/m*(phi1[i]-phi3[i]))*dt
   phi2[i+1] = phi2[i] + v2[i]*dt
   v2[i+1] = v2[i] +(-k/m*(phi2[i]-phi3[i])-k/m*(phi2[i]-phi1[i]))*dt
   phi3[i+1] = phi3[i] + v3[i]*dt
   v3[i+1] = v3[i] +(-k/m*(phi3[i]-phi1[i])-k/m*(phi3[i]-phi2[i]))*dt

plt.plot(t, phi1)
plt.plot(t,phi2)
plt.plot(t,phi3)
plt.legend(['phi1','phi2','phi3'])
plt.title('Método de Euler')
plt.xlabel('Tiempo')
plt.ylabel('Ángulo')
plt.show()

#########################################
# RUNGE KUTTA
# cambio el nombre de las variables phi_i a x,y,z para no confundirme
def f(r,t):
    x,y,z,X,Y,Z = tuple(r)
    fx = X
    fy = Y
    fz = Z
    fX = -k/m*(x-y)-k/m*(x-z)
    fY = -k/m*(y-x)-k/m*(y-z)
    fZ = -k/m*(z-x)-k/m*(z-y)
    return np.array([fx,fy,fz,fX,fY,fZ],float)

N = 200 # no. de puntos
a = 0.0 # tiempo inicial
b = 6.0 # tiempo final
h = (b-a)/N # divisiones

# inicializar variables en el tiempo = 0.0
r = np.array([np.pi/(10*4),np.pi/(10*2),np.pi/10,1.0,1.0,1.0],float)
phi1points = []
phi2points = []
phi3points = []
tpoints = np.arange(a,b,h)
for t in tpoints:
    phi1points.append(r[0])
    phi2points.append(r[1])
    phi3points.append(r[2])
    k1 = h*f(r,t)
    k2 = h*f(r+1/2*k1,t+1/2*h)
    k3 = h*f(r+1/2*k2,t+1/2*h)
    k4 = h*f(r+k3,t+h)
    r += 1/6*(k1+2*k2+2*k3+k4)

plt.plot(tpoints,phi1points)
plt.plot(tpoints,phi2points)
plt.plot(tpoints,phi3points)
plt.legend(['phi1','phi2','phi3'])
plt.title('Método RK 4')
plt.xlabel('Tiempo')
plt.ylabel('Ángulo')
plt.show()
```

Las ecuaciones de Euler Lagrange son
$$\frac{\partial L}{\partial \phi_i}=\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{\phi_i}}\right),$$
donde el lagrangiano está dado por
$$L=T-U=\frac{1}{2}mR^2(\dot{\phi_1^2}+\dot{\phi_2^2}+\dot{\phi_3^2})-\frac{1}{2}kR^2 [(\phi_1-\phi_2)^2+(\phi_2-\phi_3)^2+(\phi_3-\phi_1)^2].$$

Observamos que la energía cinética es puramente función de la velocidad angular puesto que el radio es fijo.

Sólo la energía cinética depende de $\dot{\phi_i}$ y sólo la energía potencial depende de $\phi_i$, de donde se obtiene fácilmente que
$$\frac{\partial L}{\partial \phi_1}=-kR^2[(\phi_1-\phi_2)-(\phi_3-\phi_1)]=-kR^2(\phi_1-\phi_2)-kR^2(\phi_1-\phi_3),$$
y análogamente para las otras dos variables. Asimismo,
$$\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{\phi_1}}\right)=mR^2\frac{d}{dt}(\dot\phi_1)=mR^2\frac{d^2 \phi_1}{dt^2}.$$

Por tanto, la ecuación de movimiento es
$$m\frac{d^2\phi_1}{dt^2}=-k(\phi_1-\phi_2)-k(\phi_1-\phi_3),$$
y equivalentemente para $\phi_2$ y $\phi_3$:
$$m\frac{d^2\phi_2}{dt^2}=-k(\phi_2-\phi_3)-k(\phi_2-\phi_1),$$
$$m\frac{d^2\phi_3}{dt^2}=-k(\phi_3-\phi_1)-k(\phi_3-\phi_2).$$

Estas ecuaciones describen la dinámica de la desviación de los ángulos $\phi_i$ a partir de la posición de equilibrio. Inicialmente, perturbamos el sistema de manera que
$$\phi_3(t=0)=\pi/10, \quad \phi_2(t=0)=\pi/20, \quad \phi_1(t=0)=\pi/40,$$
i.e., que el objeto más desviado de su posición de equilibrio es el tercero, mientras que la desviación de los otros dos es igual la mitad y un cuarto respectivamente.

También consideramos que las velocidades iniciales son
$\dot{\phi}_i(t=0)=1$ para los tres objetos. Lo que ambos métodos indican gráficamente es que los tres objetos que conforman el sistema oscilan periódicamente alrededor de una posición de equilibrio. La amplitud del tercero es la más grande, y esto es consecuencia de que sufre la mayor desviación inicial (el sistema de ecuaciones es de hecho simétrico para las tres variables angulares, de modo que habríamos obtenido resultados análogos permutando los valores iniciales). Identificando que $\theta +2\pi$ equivale a $\theta$, cada cierto tiempo el sistema regresa a la posición inicial, lo cual es resultado de la linealidad (pequeños desplazamientos desde la posición de equilibrio).

Si pensamos la solución general como combinación lineal lineal de modos tenemos:

a) La primera solución es una simple rotación, es decir, no hay oscilaciones.
b )En la segunda solución, dos masas oscilan en fase y la tercera con fase opuesta.
c) En la tercera solución una masa es estacionaria mientras que las otras dos masas oscilan con fase opuesta.

Problema 6: (Opcional)

De las sesiones brindadas, su propia experiencia y preferencias: Definir un problema a resolver con el enfoque numérico, explique a detalle cada consideración que realice y explique sus conclusiones.

El problema puede ser la implementación de un modelo estándar discutido en libros de física (i.e. obtener el gráfico típico de estructura de ionización del átomo de Hidrógeno.), proyecto que esté desarrollando en su formación o complementar cálculos de las sesiones.

```python
from cmath import exp,sqrt,sin,pi
import matplotlib.pyplot as plt
from numpy import linspace,empty

#d=n*pi/alpha
slit_sep = 20 #mu m
alpha = pi/slit_sep
num_slits = 10
wavelength = 0.5 #mu m
f = 100 #cm de manera que si x se mide en cm x*u/lambda*f es adimensional
#x en cm, u en mu m, lambda en mu m, f en cm
#x va de -10cm a 10cm

def q(u): #u se mide en mu m
   return sin(alpha*u)**2

def integrand(u,x):
    return sqrt(q(u))*exp(1j*2*pi*x*u/(wavelength*f))

# regla del trapezoide
def int_trapezoid(f,x,n):
    a = -(num_slits*slit_sep)/2 #numero de rendijas*separacion de rendijas/2
    b = (num_slits*slit_sep)/2
    N = 2**(n-1)
    h = (b-a)/N
    if n == 1:
        return 0.5*h*(f(a,x)+f(b,x))
    else:
        I = 1/2*int_trapezoid(f,x,n-1)
        for k in range(1,N,2):
            I += h*f(a+k*h,x)
        return I

def I(x):
    return abs(int_trapezoid(integrand,x,12))**2

x_vals = linspace(-5,5,500)
I_vals = list(map(I,x_vals))

plt.title('Intensidad en la pantalla como función de la distancia')
plt.xlabel('x')
plt.ylabel('I(x)')
plt.plot(x_vals,I_vals)
plt.grid(True)
plt.show()

I_array = empty([100, 500], float)
for k in range(100):
    I_array[k, :] = I_vals

plt.imshow(I_array,vmax=3000)
plt.gray()
plt.show()
```

Para este ejercicio libre decidí resolver el problema 5.19 del libro de Mark Newman http://www-personal.umich.edu/~mejn/cp/exercises.html
Quisiera mencionar que estuve leyendo este libro a la par del curso y algunas cosas las tomé de ahí para resolver estos ejercicios. Por eso tal vez perciban cierta nomenclatura diferente a lo que se vio en clase.

En este problema se estudia la intensidad de luz difractada sobre una pantalla como función de la distancia.
En resumen, el problema da una función (intensidad de difracción) y pide lo siguiente:

"Use su función en un programa para calcular y graficar la intensidad del patrón de difracción producido por una rejilla que tiene diez rendijas en total, si la luz incidente tiene una longitud de onda
$\lambda = 500 nm$. Suponga que la lente tiene una distancia focal de 1 metro y la pantalla mide $10 cm$ (...) Cree una visualización de cómo se vería el patrón de difracción en la pantalla usando la función imshow"

**Por último, debe de rellenar la siguiente encuesta de satisfacción:** https://forms.gle/4WU3ovvP7gJMj8GA7 (ingresar con el correo que se inscribió).

Enviar sus respuestas a nuestro correo: pythonclubforphysicists@gmail.com

**OJO: El correo debe de tener como asunto:**
**"Ejercicios_IIEVFC"**

*Caso contrario su solución puede quedar sin evaluar debido a la gran cantidad de correos.*

## Donaciones

Puedes donar una vez en el siguiente enlace (Ko-Fi):

*Click en la imagen.*


```python
import sympy as sy
import numpy as np
```

```python
a, b, c, d, e, f, g, x = sy.symbols('a, b, c, d, e, f, g, x')
```

```python
sine = a*x**6 + b*x**5 + c*x**4 + d*x**3 + e*x**2 + f*x + g
```

```python
sine_d = sy.diff(sine, x)
```

```python
sine_i = sy.integrate(sine, x)
```

```python
the_system = [
    sine_i.subs(x, sy.pi / 2) - sine_i.subs(x, 0) - 1,
    sine_d.subs(x, 0) - 1,
    sine_d.subs(x, sy.pi / 2),
    sine_d.subs(x, sy.pi) + 1,
    sine.subs(x, 0),
    sine.subs(x, sy.pi/2) - 1,
    sine.subs(x, sy.pi)
]
```

```python
res = sy.solve(the_system, (a, b, c, d, e, f, g))
```

```python
for var, exp in res.items():
    print(var, exp, sy.srepr(exp))
```

```python
[x for x in dir(sy) if 'numpy' in x]
```

```python
sy.list2numpy([exp for var, exp in res.items()])
```

```python
np.array([exp for var, exp in res.items()])
```

```python
[x for x in dir(np) if 'sympy' in x]
```

```python
sine.subs(a, res[a]).subs(b, res[b]).subs(c, res[c])\
    .subs(d, res[d]).subs(e, res[e]).subs(f, res[f])\
    .subs(g, res[g]).__repr__()
```

```python
res[a]
```

```python
point = sy.geometry.Point3D(0,1,2)
```

```python
point.direction_ratio()
```

```python
import pytest
```

```python
pytest.param
```

```python
pytest.xfail
```

```python

```


```python
import numpy as np
import scipy as sp
import sympy as smp
import matplotlib.pyplot as plt
from scipy.misc import derivative
```

# Symbolic Case

You know the formula explicitly but don't really feel like using a pencil to take the derivative

$$f(x) = ... $$

**Example:** $$f(x) = e^{-a\sin(x^2)}\cdot \sin(b^x) \cdot \ln(c\sin^2(x)/x)$$

First Define all your symbols in sympy

```python
x, a, b, c = smp.symbols('x a b c', real=True)
```

```python
x**2+smp.exp(a)
```

$\displaystyle x^{2} + e^{a}$

```python
x, a, b, c = smp.symbols('x a b c', real=True)
f = smp.exp(-a*smp.sin(x**2)) * smp.sin(b**x) * smp.log(c*smp.sin(x)**2 /x)
```

Compute derivatives using `smp.diff(f, x)` where $f$ is the function you want to take the derivative of and $x$ is the variable you are taking the derivative with respect to.

```python
dfdx = smp.diff(f, x)
dfdx
```

$\displaystyle - 2 a x e^{- a \sin{\left(x^{2} \right)}} \log{\left(\frac{c \sin^{2}{\left(x \right)}}{x} \right)} \sin{\left(b^{x} \right)} \cos{\left(x^{2} \right)} + b^{x} e^{- a \sin{\left(x^{2} \right)}} \log{\left(b \right)} \log{\left(\frac{c \sin^{2}{\left(x \right)}}{x} \right)} \cos{\left(b^{x} \right)} + \frac{x \left(\frac{2 c \sin{\left(x \right)} \cos{\left(x \right)}}{x} - \frac{c \sin^{2}{\left(x \right)}}{x^{2}}\right) e^{- a \sin{\left(x^{2} \right)}} \sin{\left(b^{x} \right)}}{c \sin^{2}{\left(x \right)}}$

Can take the nth derivative $d^n f/dx^n$ by putting the optional argument at the end `smp.diff(f,x,n)`.

```python
d4fdx4 = smp.diff(f, x, 4)
#d4fdx4
```

Can then compute numerical values by plugging in numbers

```python
d4fdx4.subs([(x,4),(a,1),(b,2),(c,3)]).evalf()
```

$\displaystyle -73035.8044625845$

Can also convert to a numerical function for plotting

```python
d4fdx4_f = smp.lambdify((x,a,b,c), d4fdx4)
```

Define $x$ and $y$ arrays using the numerical function above

```python
x = np.linspace(1,2,100)
```

```python
x = np.linspace(1,2,100)
y = d4fdx4_f(x, a=1, b=2, c=3)
```

Plot

```python
plt.plot(x,y)
plt.ylabel('$d^4 f / dx^4$', fontsize=24)
plt.xlabel('$x$', fontsize=24)
```

# Numerical Case

Your given data like such

```python
x, y = np.loadtxt('../data/sample_data1.txt')
```

```python
plt.plot(x, y, 'o--')
```

There are a few ways to go about this.

**1: The basic way**

```python
dydx = np.gradient(y,x)
```

```python
plt.plot(x,y, 'o--', label='$y(x)$')
plt.plot(x,dydx, 'o--', label='$y\'(x)$')
plt.legend()
```

This way works fine if the data is smooth but not if the data is noisy:

```python
x, y = np.loadtxt('../data/sample_data2.txt')
dydx = np.gradient(y,x)
```

Plot the noisey data

```python
fig, ax = plt.subplots(1, 2, figsize=(10,3))
ax[0].plot(x,y, label='$y(x)$')
ax[1].plot(x,dydx, label='$y\'(x)$', color='r')
[a.legend() for a in ax]
plt.show()
```

Noise gets amplified in the derivative! This leads us to technique 2:

**2. The Clever Way**

Smooth data then take derivative. Consider the following new covid cases per day data

```python
x, y = np.loadtxt('../data/coviddata.txt')
dydx = np.gradient(y,x)
```

Taking the derivative naively gives a bad result

```python
fig, ax = plt.subplots(1, 2, figsize=(10,3))
ax[0].plot(x,y, label='$y(x)$')
ax[1].plot(x,dydx, label='$y\'(x)$', color='r')
[a.legend() for a in ax]
plt.show()
```

Smooth the data by convolving it with a rectangle

```python
filt = np.ones(15)/15
```

```python
y_smooth = np.convolve(y, filt, mode='valid')
dysdx = np.gradient(y_smooth, x[7:-7])
```

Plot

```python
fig, ax = plt.subplots(1, 2, figsize=(10,3))
ax[0].plot(x,y, label='$y(x)$')
ax[0].plot(x[7:-7], y_smooth, label=r'$y_{{smooth}}(x)$')
ax[1].plot(x,dydx, label='$y\'(x)$', color='r')
ax[1].plot(x[7:-7],dysdx, label='$y_{smooth}\'(x)$', color='purple')
ax[1].set_ylim(-100,120)
ax[1].grid()
[a.legend() for a in ax]
[a.set_xlabel('Time [Days]') for a in ax]
ax[0].set_ylabel('Cases per Day')
ax[1].set_ylabel('$\Delta$ (Cases per Day) / $\Delta t$')
fig.tight_layout()
plt.show()
```

# Quasi-Symbolic Case

In this case you know your function $f(x) = ...$ but the function is not given by a typical expression. For example.

$$f(u) = \text{max} \left\{ \left|e^{-x_iu^2} - y_i\right| \right\}  $$

for some array of $x_i$'s and $y_i$'s

```python
x = np.linspace(0, 1, 500)
y = np.exp(-x*2.15**2) + 0.1*np.random.randn(len(x))
```

```python
plt.scatter(x,y)
plt.xlabel('$x_i$', fontsize=20)
plt.ylabel('$y_i$', fontsize=20)
plt.show()
```

Define function

```python
def f(u):
    return max(np.abs(np.exp(-x*u**2) -y))
```

Compute the function values

```python
u = np.linspace(0,10,40)
f_u = np.vectorize(f)(u)
```

Plot

```python
plt.plot(u, f_u, 'o--')
plt.xlabel('$u$', fontsize=20)
plt.ylabel('$f(u)$', fontsize=20)
plt.show()
```

You could always just take the derivative of the numerical array `f_u`, but there is a specific derivative function better assigned for this

Compute derivative

```python
dfdu = np.vectorize(derivative)(f, u, dx=1e-6)
```

Plot

```python
plt.plot(u, dfdu)
```


```python
from sympy import symbols, cos, sin, pi, simplify, trigsimp, expand_trig, pprint
from sympy.matrices import Matrix
```

```python
q1, q2, q3, q4, q5, q6= symbols('q1:7')
```

```python
print(pi)
```

    pi

```python
def pose(theta, alpha, a, d):

  r11, r12 = cos(theta), -sin(theta)
  r23, r33 = -sin(alpha), cos(alpha)
  r21 = sin(theta) * cos(alpha)
  r22 = cos(theta) * cos(alpha)
  r31 = sin(theta) * sin(alpha)
  r32 = cos(theta) * sin(alpha)
  y = -d * sin(alpha)
  z = d * cos(alpha)

  T = Matrix([
    [r11, r12, 0.0, a],
    [r21, r22, r23, y],
    [r31, r32, r33, z],
    [0.0, 0.0, 0.0, 1]
  ])

  T = simplify(T)

  return T
```

```python
def print_pretty_matrix(T):

  T11, T12, T13, T14 = str(T[0, 0]), str(T[0, 1]), str(T[0, 2]), str(T[0, 3])
  T21, T22, T23, T24 = str(T[1, 0]), str(T[1, 1]), str(T[1, 2]), str(T[1, 3])
  T31, T32, T33, T34 = str(T[2, 0]), str(T[2, 1]), str(T[2, 2]), str(T[2, 3])
  T41, T42, T43, T44 = str(T[3, 0]), str(T[3, 1]), str(T[3, 2]), str(T[3, 3])

  s1 = '|{:20.20}|{:20.20}|{:20.20}|{:30.30}|'.format(T11, T12, T13, T14)
  s2 = '|{:20.20}|{:20.20}|{:20.20}|{:30.30}|'.format(T21, T22, T23, T24)
  s3 = '|{:20.20}|{:20.20}|{:20.20}|{:30.30}|'.format(T31, T32, T33, T34)
  s4 = '|{:20.20}|{:20.20}|{:20.20}|{:30.30}|'.format(T41, T42, T43, T44)

  print('\n', s1, '\n', s2, '\n', s3, '\n', s4, '\n')

```

```python
d90 = pi / 2

T01 = pose(q1, 0, 0, 0.75)
T12 = pose(q2 - d90, -d90, 0.35, 0)
T23 = pose(q3, 0, 1.25, 0)
T34 = pose(q4, -d90, -0.054, 1.5)
T45 = pose(q5, d90, 0, 0)
T56 = pose(q6, -d90, 0, 0)
T6g = pose(0, 0, 0, 0.303)

print("T01")
print_pretty_matrix(T01)

print("T12")
print_pretty_matrix(T12)

print("T23")
print_pretty_matrix(T23)

print("T34")
print_pretty_matrix(T34)

print("T45")
print_pretty_matrix(T45)

print("T56")
print_pretty_matrix(T56)

print("T6g")
print_pretty_matrix(T6g)

```

    T01

     |cos(q1)             |-sin(q1)            |0.0                 |0                             |
     |sin(q1)             |cos(q1)             |0                   |0                             |
     |0                   |0                   |1                   |0.750000000000000             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T12

     |sin(q2)             |cos(q2)             |0.0                 |0.350000000000000             |
     |0                   |0                   |1                   |0                             |
     |cos(q2)             |-sin(q2)            |0                   |0                             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T23

     |cos(q3)             |-sin(q3)            |0.0                 |1.25000000000000              |
     |sin(q3)             |cos(q3)             |0                   |0                             |
     |0                   |0                   |1                   |0                             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T34

     |cos(q4)             |-sin(q4)            |0.0                 |-0.0540000000000000           |
     |0                   |0                   |1                   |1.50000000000000              |
     |-sin(q4)            |-cos(q4)            |0                   |0                             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T45

     |cos(q5)             |-sin(q5)            |0.0                 |0                             |
     |0                   |0                   |-1                  |0                             |
     |sin(q5)             |cos(q5)             |0                   |0                             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T56

     |cos(q6)             |-sin(q6)            |0.0                 |0                             |
     |0                   |0                   |1                   |0                             |
     |-sin(q6)            |-cos(q6)            |0                   |0                             |
     |0.0                 |0.0                 |0.0                 |1                             |

    T6g

     |1                   |0                   |0.0                 |0                             |
     |0                   |1                   |0                   |0                             |
     |0                   |0                   |1                   |0.303000000000000             |
     |0.0                 |0.0                 |0.0                 |1                             |

```python
a, b = symbols('a b')
print(expand_trig(cos(a - b)))
```

    sin(a)*sin(b) + cos(a)*cos(b)

```python
print(trigsimp(sin(a)*sin(b) + cos(a)*cos(b)))
```

    cos(a - b)

```python
T03 = simplify(T01 * T12 * T23)
R03 = T03[:3, :3]
```

```python
print()
print("T03")
print_pretty_matrix(T03)

print()
print("R03")
pprint(R03)
```

    T03

     |sin(q2 + q3)*cos(q1)|cos(q1)*cos(q2 + q3)|-sin(q1)            |(1.25*sin(q2) + 0.35)*cos(q1) |
     |sin(q1)*sin(q2 + q3)|sin(q1)*cos(q2 + q3)|cos(q1)             |(1.25*sin(q2) + 0.35)*sin(q1) |
     |cos(q2 + q3)        |-sin(q2 + q3)       |0                   |1.25*cos(q2) + 0.75           |
     |0                   |0                   |0                   |1                             |

    R03
    ⎡sin(q₂ + q₃)⋅cos(q₁)  cos(q₁)⋅cos(q₂ + q₃)  -sin(q₁)⎤
    ⎢                                                    ⎥
    ⎢sin(q₁)⋅sin(q₂ + q₃)  sin(q₁)⋅cos(q₂ + q₃)  cos(q₁) ⎥
    ⎢                                                    ⎥
    ⎣    cos(q₂ + q₃)         -sin(q₂ + q₃)         0    ⎦

```python
print("R03")
print(R03)
```

    R03
    Matrix([
    [sin(q2 + q3)*cos(q1), cos(q1)*cos(q2 + q3), -sin(q1)],
    [sin(q1)*sin(q2 + q3), sin(q1)*cos(q2 + q3),  cos(q1)],
    [        cos(q2 + q3),        -sin(q2 + q3),        0]])

```python
T36 = T34*T45*T56
pprint(T36)
```

    ⎡-sin(q₄)⋅sin(q₆) + cos(q₄)⋅cos(q₅)⋅cos(q₆)  -sin(q₄)⋅cos(q₆) - sin(q₆)⋅cos(q₄
    ⎢
    ⎢             sin(q₅)⋅cos(q₆)                             -sin(q₅)⋅sin(q₆)
    ⎢
    ⎢-sin(q₄)⋅cos(q₅)⋅cos(q₆) - sin(q₆)⋅cos(q₄)  sin(q₄)⋅sin(q₆)⋅cos(q₅) - cos(q₄)
    ⎢
    ⎣                    0                                           0

    )⋅cos(q₅)  -sin(q₅)⋅cos(q₄)  -0.054⎤
                                       ⎥
                   cos(q₅)        1.5  ⎥
                                       ⎥
    ⋅cos(q₆)   sin(q₄)⋅sin(q₅)     0   ⎥
                                       ⎥
                      0            1   ⎦

```python
R36 = T36[:3, :3]
pprint(R36)
```

    ⎡-sin(q₄)⋅sin(q₆) + cos(q₄)⋅cos(q₅)⋅cos(q₆)  -sin(q₄)⋅cos(q₆) - sin(q₆)⋅cos(q₄
    ⎢
    ⎢             sin(q₅)⋅cos(q₆)                             -sin(q₅)⋅sin(q₆)
    ⎢
    ⎣-sin(q₄)⋅cos(q₅)⋅cos(q₆) - sin(q₆)⋅cos(q₄)  sin(q₄)⋅sin(q₆)⋅cos(q₅) - cos(q₄)

    )⋅cos(q₅)  -sin(q₅)⋅cos(q₄)⎤
                               ⎥
                   cos(q₅)     ⎥
                               ⎥
    ⋅cos(q₆)   sin(q₄)⋅sin(q₅) ⎦

```python
R36f = Matrix([[-sin(q4)*sin(q6) + cos(q4)*cos(q5)*cos(q6), -sin(q4)*cos(q6) - sin(q6)*cos(q4)*cos(q5), -sin(q5)*cos(q4)],
  [                           sin(q5)*cos(q6),                           -sin(q5)*sin(q6),          cos(q5)],
  [-sin(q4)*cos(q5)*cos(q6) - sin(q6)*cos(q4),  sin(q4)*sin(q6)*cos(q5) - cos(q4)*cos(q6),  sin(q4)*sin(q5)]])
```

```python
print(R36 == R36f)
```

    True


# Introduction to Programming for Engineers

## Python 3

# 06 Calculus with Sympy and Numpy
## CLASS MATERIAL

 1. Symbolic Mathematics

 2. Differentiation

 3. Integration

 4. Ordinary Differential Equations (ODE)

5. Review Exercises

### Lesson Goal

- differentiation and integration using python
- solving ordinary differential equations in python

### Fundamental programming concepts
Often, it can be difficult to know how to begin setting out a familiar mathematical concept in code.

This class will show how to arrange operations for differentiation, integration and solving ordinary differntial equations.

Many of the practical situations that engineers have to analyse involve quantities that change with respect to time or another parameter.

 - Temperature
 - Voltage
 - Torque
 - Chemical species
 - Population

To solve these types of problems, we usually need to use differentiation, integration or find solutions to differential equations.

# 1. Symbolic Mathematics

In the last class we learned how to represent an equation symbolically.

```python
import numpy as np
import sympy as sp
from sympy import solve, symbols, Symbol, Function, Eq, pprint, init_printing

# This initialises pretty printing
init_printing()
from IPython.display import display
```

To write a symbolic equation, we first create variables to store the symbols we want to use:

```python
a, b, c, x = symbols('a, b, c, x')
```

We then write an equation, using the variable names.

$$f(x) = ax^2 + bx + c$$

```python
f = a*x**2 + b*x + c
```

# 2. Differentiation

 2.1 Symbolic Differentiation

 2.2 Differentiating Polynomial Functions

__Differentiation :__ A measures of the change of the function value (output value) with respect to a change in some input value.

## 2.1 Symbolic Differentiation

`sympy.diff` differentiates an equation to give a general solution with respect to a specified variable.

The inputs to `sympy.diff` are the function to differentiate and the specified variable.

```python
from sympy import diff
```

```python
f = a*x**2 + b*x + c
```

$$f(x) = ax^2 + bx + c$$

$f'(x)$
(first derivative w.r.t $x$)

```python
print(diff(f, x))
```

    2*a*x + b

$$f(a) = ax^2 + bx + c$$

$f'(a)$
(first derivative w.r.t $a$)

```python
print(diff(f, a))

```

    x**2

$$f(x) = ax^2 + bx + c$$

We can specify the order of differentiation required.

$f''(x)$
(second derivative w.r.t $x$)

```python
print(diff(f, x, 2))

```

    2*a

$$f(x) = ax^2 + bx + c$$

We can substitute numerical values to find the exact solution:

Find $f'(a), \quad x=2$

```python
print(diff(f, a))

diff(f, a).subs([(x, 2)])

```

$$f(x) = ax^2 + bx + c$$

Find $f'(x), \quad a=2, b=4$

```python
print(diff(f, x))

diff(f, x).subs([(a, 2), (b, 4), (x, 3)])
```

The following example uses the trigonometric cosine function.

$$f(x) = cos(x)$$

We need to import a *symbolic function* to handle this operation:
>`sympy.cos`

Documentation for elementary symbolic functions including trigonometric, hyperbolic, and sqrt can be found here:
https://docs.sympy.org/latest/modules/functions/elementary.html#exp

We can use `diff` to find higher order derivatives.

Example: Find $f''(x)$ (the second derivative of $f(x) = cos(x)$).

```python
# argumets : function, differentaition variable, order of differentiation

print(diff( sp.cos(x), x, 2 ) )
```

    -cos(x)

## 2.2 Differentiating Polynomial Functions

We can quickly find derivatives of polynomial functions with defined coefficients, by using the  numpy function, `polyder`

Example:
    $$p(x) = x^3 + x^2 + x^1 + 1$$

Find $p'(x)$

```python
p = np.poly1d([1,1,1,1])

pd = np.polyder(p)

print(pd)
```

       2
    3 x + 2 x + 1

A numerical value can be substited in to find an exact solution.

The derivative returend by `polyder` behaves like a function and takes a value of x as an argument.

Find $p'(2)$

```python
pd(2.)
```

`polyder` can be be used to solve higher order derivatives by giving a second argument:

```python
# second derivative of p
print(np.polyder(p, 2))

# third derivative of p
print(np.polyder(p, 3))

# fourth derivative of p
print(np.polyder(p, 4))
```

    6 x + 2

    6

    0

# 3. Integration

 3.1 Symbolic Integration

 3.2  Indefinate Integrals

 3.3 Definate Integrals

 3.4 Non-symbolic Integration

 3.5 Trapezoid Estimation Method

Often we need to __integrate__ a function over some range.

For example, $S$ is the integral of a function $f(x)$ from a to b

$$S = \int_{a}^{b} f(x) dx$$

This area or integral can be estimated from the sum of trapezoid strips used to represent the curve.

Example: Integrate the following function with respect to $x$:
$$f(x) = x^2$$

limits $x=0$ to $x=1$.

\begin{align*}
S &= \int_{0}^{1} x^2 dx\\
  &= \frac{1}{3}x^3 \biggr\rvert_0^{1}\\
  &= \frac{1}{3}(1)^3 - \frac{1}{3}(0)^3\\
  &= \frac{1}{3}
\end{align*}

## 3.1 Symbolic Integration

`sympy.integrate`

There are two kinds of integrals, definite (limits defined) and indefinite (limits not defined).

```python
from sympy import integrate
```

## 3.2 Indefinate Integrals

$$f(x) = ax^2 + bx + c$$

```python
a, b, c, x = symbols('a, b, c, x')
f = a*x**2 + b*x + c
```

The inputs to `sympy.integrate` are the integrand $f$ and the integration variable $x$.

$$f(x) = ax^2 + bx + c$$

Find : $\int f(x)dx$

$$f(x) = ax^2 + bx + c$$

$$\int f(x)dx = \frac{ax^3}{3} + \frac{bx^2}{2} + cx + C$$

$C = $ constant of integration

Notice that SymPy does not include the constant of integration.

```python
print(integrate(f, x))
```

    a*x**3/3 + b*x**2/2 + c*x

`pprint` (pretty print) can be used to display the symbolic expression in a way that is easier to read.

```python
from sympy import pprint

pprint(integrate(f, x))
```

       3      2
    a⋅x    b⋅x
    ──── + ──── + c⋅x
     3      2

Multiple/nested integrals can be performed

$$f(x) = ax^2 + bx + c$$

$$\int \int f(x)dx dx = \frac{ax^4}{12} + \frac{bx^3}{6} + \frac{cx^2}{2} + Cx + D$$

$C, D = $ constants of integration

```python
print( integrate(f, (x), (x)) )
```

    a*x**4/12 + b*x**3/6 + c*x**2/2

Multiple/nested integrals can have different dimensions:

$$g(x,y) = x y $$

```python
y = symbols('y')

g = x*y

pprint( integrate(g, (x), (y)) )
```

     2  2
    x ⋅y
    ─────
      4

## 3.3 Definate Integrals

To find a definite integral, parse the argument:
 `(integration_variable, lower_limit, upper_limit)`.

$$f(x) = ax^2 + bx + c$$

\begin{align*}
\int_{x=0}^{1} f(x) &= \frac{ax^3}{3} + \frac{bx^2}{2} + cx \biggr\rvert_0^{1}\\
                    &= \frac{a(1)^3}{3} + \frac{b(1)^2}{2} + c(1)  - \left( \frac{a(0)^3}{3} + \frac{b(0)^2}{2} + c(0) \right) \\
                    &= \frac{a}{3} + \frac{b}{2} + c
\end{align*}

```python
print(integrate(f, (x, 0, 1))) # definite integral from x=0..1
```

    a/3 + b/2 + c

Substitute in numerical values to find exact solution.

$f(x) = ax^2 + bx + c$

Find $\int_{x=0}^{1} f(x)dx$

$a=b=c=2 $

```python
integrate(f, (x, 0, 1)).subs([(a, 2),
                              (b, 2),
                              (c, 2)])

```

The following example uses the exponential function.

$$\int e^{-x} dx = - e^{-x} + C$$

We need to import a *symbolic function* to handle this operation:
>`sympy.exp`

Documentation for elementary symbolic functions including trigonometric, hyperbolic, and sqrt can be found here:
https://docs.sympy.org/latest/modules/functions/elementary.html#exp

```python
integrate( sp.exp(-x) )
```

The definite integral has infinity as the upper limit.

\begin{align*}
\int_{x=0}^{\infty} e^{-x}dx &= -e^{-x}\biggr\rvert_0^{\infty}\\
  &= 1
\end{align*}

We need to import symbolic infinity:
>`sympy.oo`

```python
from sympy import oo as oo

integrate(sp.exp(-x), (x, 0, oo))
```

Again, we can compute multiple/nested integrals
\begin{align*}
\int\int_{0}^{1} x^2 \; dxdx &= \frac{1}{3}
\end{align*}

```python
f = x**2

pprint( integrate(f, (x, 0, 1), (x, 0, 1)) )
```

    1/3

Again, we can compute multiple/nested integrals
\begin{align*}
\int_{0}^{1}\int_{0}^{1} xy \; dxdy &= \frac{1}{4}
\end{align*}

```python
y = symbols('y')

g = x*y

pprint( integrate(g, (x, 0, 1), (y, 0, 1)) )
```

    1/4

## 3.4 Non-symbolic Integration

We can also integrate non-symbolically.

The scipy function `quad` is very convenient for solving integration problems:

Input arguments :
- function to integrate
- lower limit
- upper limit

Returns:
- integral
- estimated absolute error in the result

```python
from scipy.integrate import quad
```

\begin{align*}
S &= \int_{0}^{1} x^2 dx\\
  &= \frac{1}{3}x^3 \biggr\rvert_0^{1}\\
  &= \frac{1}{3}(1)^3 - \frac{1}{3}(0)^3\\
  &= \frac{1}{3}
\end{align*}

```python
# first create a function to describe the integrand
def integrand(x):
    return x**2

# create two variables to store the integral and the error
ans, err = quad(integrand, 0, 1)

print(ans)
print(f'The estimated error is {err}')
```

    0.33333333333333337
    The estimated error is 3.700743415417189e-15

### Example :  Water flowing into a tank.

Water flows into a tank at flow rate that changes with time.

$v'$ is the rate of change of the *volume flow rate* (L/s) of water in the tank.

$$ v' =\frac{dv}{dt}=\alpha t $$

where $\alpha = 1\textrm{L/s}$

__How much liquid flows into the tank over 10 seconds?__

Initial flow rate:

$v_0=5\textrm{L/s,} \qquad t_0=0$

__Solution__

We can see that at

\begin{align*}
v_1 &= v_0 + \frac{dv}{dt}\\
    &= v_0 + \alpha t \\
\end{align*}

To find the total __volume__ of liquid the enters the tank in the first $10$ seconds:

$$V = \int_{0}^{10} v_0 + \alpha t \quad dt$$

```python
from scipy.integrate import quad

t0=0.0
tfinal = 10.0
v0=5       # L/s
alpha = 1  # L/s

# create a function to describe the integrand
def integrand(t):
    return v0 + alpha * t

V, err = quad(integrand, t0, tfinal)

print(f'{V} L flowed into the tank in 10 seconds')
print(f'The estimated error is {err}')
```

    100.0 L flowed into the tank in 10 seconds
    The estimated error is 1.1102230246251565e-12

We can check the solution:

\begin{align*}
V &= \int_{0}^{10} v_0 + \alpha t \quad dt\\
  &= v_0(t) + \frac{1}{2}\alpha t^2 \biggr\rvert_0^{10}\\
  &=100
\end{align*}

## 3.5 Trapezoid Estimation Method

Often we have a data set (e.g. from an experiemnt) rather than a function that we want to ingtegrate.

One way to deal with this is to fit a function to the data and then integrate the function.

Another way is to use the __trapezoid method__ to integrate the array of raw data.

The region under the graph of the function between two data points is approximated as a trapezoid.

The area of the region is approximiated as  the area of teh trapezoid.

```python
import numpy as np

x = np.array([0, 0.5, 1, 1.5, 2])

y = np.array([5, 6, 1, 1.5, 2])

print(np.trapz(y, x))

# We can do this manually to check how the function works
means = (y[1:] + y[:-1]) / 2 # mean of each pair of y values
diffs = (x[1:] - x[:-1])     # difference between each pair of x values
print(sum(means * diffs))    # sum areas
```

    6.0
    6.0

# 4. Ordinary Differential Equations (ODEs)

 4.1 Analytical Solution to ODE

 4.2  Changing the Subject using `solve`

 4.3  Solving Higher Order ODE

 4.4  Numerial Solutions to  ODEs

__Ordinary differential equation__

A differential equation containing one or more functions of __one__ independent variable and its derivatives.

Example:
*Given $f'(x)$, find $f(x)$.*

## 4.1 Analytical Solutions to ODE using Symbolic Math

The function `sympy.dsolve` can be used to find the solution to any ordinary differential equation.

```python
from sympy import dsolve, Eq, sin, cos
```

Syntax to represent $f'(x)$

        f(x).diff(x)

Syntax to represent $f''(x)$

        f(x).diff(x, x)

In the next examples we will:
1. solve ODE symbolically to find the general solution, $f(x)$
2. substitute an initial condition to find constants of the general solution
3. find the value of the function for a specific input variable.

This is how to calculate the solution analytically using symbolic mathematics:

a) Define the variables and functions in the equation symbolically

b) Use `Eq` from the `sympy` package to define the differential equation.

c) Use `dsolve` to find the general solution to the equation by solving the equation symbolically.

d) Find the value of the constant of ingtegration $C1$.

e) Substitute $C1$ into the general solution to find the exact solution to the equation.

#### Solving a First Order ODE

Let's start with an example we used earlier:

$$f'(x) = cos(x)$$

The solution to the equation is :

$$f(x) = sin(x)$$

We create a variable to represent:
- the function $f(x)$
- the input variable $x$

```python
f = Function('f')
x = Symbol('x')
```

We create a Sympy `Eq` equation.

The first argument (left hand side of equation) is equal to the second argument (right hand side of equation).

In other words, `Eq` gives the name `ode` to the expression:

$$f'(x) = cos(x)$$

$f$ and $x$ are the function and variable that we have created symbolic representations  of: `f` and `x`.

```python
ode = Eq( f(x).diff(x), cos(x) )
```

### Finding the General Solution
`dsolve` : Solves ordinary differential equation eq for function f(x).

We can find the general solution, with unknown constant, `C1`.

```python
# General solution
gen_sol = dsolve(ode, f(x))

display(gen_sol)
```

The package `display` can also be used to achieve more readable print formatting for mathematical equations:

```python
from IPython.display import display
```

```python
display(gen_sol)
```

### Finding constant C1
Initial value problem.

Use $x=0, \quad f(x)=0$ to find `C1`.

```python
C_1 = gen_sol.subs([(x,    0),       # x = 0 --> substitute 0 for x
                    (f(0), 0)])      # f(0) = 0 --> substitute 0 for f(0)

display(gen_sol)
display(C_1)
```

##### What does this code do?

`C_1` is a Sympy __`Eq`__ (equation) object.

(__Note :__ `C_1` is different from the integration constant `C1`).

In other words `C_1` is the name given to the expression `C1 = 0`

`subs` takes a list of the substitutions we want to make as arguments.

Each substitution is given as a tuple: *(original value, value to substitute in)*

`gen_sol.subs` : The original value is replaced by the value to substitute in wherever it appears in the general solution.

In the example shown, the substitution reveals the value of `C1`.

Substitute the revealed value into the general solution for `C1`.

When doing this, note if `C1` appears on left or right of equation `C_1`

```python
# Method 1, C1 rhs
ode_sol = gen_sol.subs([(C_1.rhs, C_1.lhs)])

# Method 2, C1 lhs
# ode_sol = gen_sol.subs([(C_1.lhs, C_1.rhs)])

display (ode_sol)
```

##### What does this code do?

>`ode_sol = gen_sol.subs([(C_1.rhs, C_1.lhs)])`

`subs` takes a list of the substitutions we want to make as arguments.

Each substitution is given as a tuple: *(original value, value to substitute in)*

`gen_sol.subs` : The original value is replaced by the value to substitute in wherever it appears in the general solution.

This code substitites the value 0 into the general solution for `C1`.

### Finding the value of the function for a specific input variable.

Now we have found the value of `C1`, we can use `subs` again to find the value of $f(x)$ at any input value $x$.

```python
spec_sol = ode_sol.subs([(x, np.pi/2)])
display(spec_sol)
```

The value of $f(\frac{\pi}{2})$ can then be accessed as a numerical value.

```python
print(spec_sol.rhs)          # .rhs methid used to access value of function

print(type(spec_sol.rhs))    # value is a floating point number

print(int(spec_sol.rhs) * 2) # value can be cast as int, used in aithmetic operations etc
```

    1.00000000000000
    <class 'sympy.core.numbers.Float'>
    2

## 4.2 Changing the Subject using `solve`

__Look carefully at this next example.__

Solve the following eqation to find $v(t)$:

$$\frac{dv}{dt} = -t^2 - e^m$$

Initial condition $ v(1) = 5 $

In this case, when we substitue the initial condition into the general solution, the subject of equation returned is not the constant we are trying to find.

We must rearrange the equation to make `C1` the subject:

```python
# create symbolic equation
t, m, C1 = symbols("t m C1")
v = Function("v")
eqn = Eq(v(t).diff(t), -t**2 - sp.exp(m))
display(eqn)
```

```python
# find the general solution
gen_sol = dsolve(eqn, v(t))
display(gen_sol)

# substitute in initial condition
cnts = gen_sol.subs([(t,    1),       # x = 0 --> subs 0 for x
                     (v(1), 5)])      # f(0) = 0 --> subs 0 for f(0)

display(cnts)
```

We can now use `sympy.solve`(imported earlier) to solve the equation for `C1`:

```python
# solve the equation for C1
C_1 = solve([cnts], (C1))
display(C_1)
```

We can now substitute `C1` into the general solution as before

```python
# substitue the value of C1 into the general solution
ode_sol = gen_sol.subs(C_1)
display(ode_sol)
```

Again, the complete solution can be used to find v(t) for specific value of t e.g. $\frac{\pi}{2}$

```python
spec_sol = ode_sol.subs([(t, np.pi/2)])
display(spec_sol)
```

## 4.3 Solving Higher Order ODE

Taking another expression containing higher order derivatives,

we can write the equation:

$$A=f''(x) =12x^2$$

```python
f = Function('f')
x = Symbol('x')

A = Eq(f(x).diff(x, x), 12*x**2)
```

The highest order derivative in this expression is a second order devrivative.

The general solution will therefore have two constants to solve.

We need two initial conditions.

```python
# General solution
gen_sol = dsolve(A, f(x))
display(gen_sol)
```

Find the constants by substitution.

$f(0)=0$

$f(1)=3$

$f'(1)=6$

```python
cntsA = gen_sol.subs([(x,    0),       # x = 0 --> subs 0 for x
                    (f(0), 0)])      # f(0) = 0 --> subs 0 for f(0)

cntsB = gen_sol.subs([(x,    1),       # x = 1 --> subs 0 for x
                    (f(1), 3)])      # f(1) = 3 --> subs 0 for f(0)

display(cntsA)
display(cntsB)
```

Create symbolic representations of the variables.

This time we use `sympy.solve` to solve the coupled equations `cnd0` and `cnd1` for constants `C1, C2`:

```python
C1, C2 = symbols("C1, C2")  # generic constants

C1C2_sol = solve([cntsA, cntsB], (C1, C2))
display(C1C2_sol)
```

Substitute the two constants into the general solution.

```python
ode2_sol = gen_sol.subs(C1C2_sol)
display(ode2_sol)
```

## 4.4 Numerial Solutions to  ODEs

The function `scipy.integrate.odeint` estimates the solution to first order differential equations.

In contrast to `dsolve`, this gives the numerical solution for a range of input values, rather than a symbolic expression, solved analytically.

```python
from scipy.integrate import odeint
```

The function `odeint` takes a user-defined function as an argument.

This input function should define the derivative you want to solve in the form:

$f' = \frac{df}{dt} = g(t)$

###### Example

Solve $f' = cos(t)$

Initial value : $f(0)=0$

Create a user-defined function to represent the differential equation:

```python
def dfdt(x, t):
    return np.cos(t)
```

Two additional arguments are needed:

- the initial value of x
- the value(s) of t at which to evaluate x, starting with the initial value
- (if `dx_dt` takes any other argumemts they are entered as a tuple as the third argument)

```python
f0 = 0                    # the initial value
ts = np.linspace(0,5,100) # the value(s) of t at which to evaluate x

# odeint returns x at each value of t
fs = odeint(dfdt, # user-defined function
            f0,   # initial value
            ts)   # value(s) of t at which to evaluate x

fs = np.array(fs).flatten() # flatten to 1D array
```

The function `odeint` returns an $n\times 1$ array (2D column).

To plot the output we have to "flatten" it using `flatten` to a 1D array.

Plot the function

```python
import matplotlib.pyplot as plt
%matplotlib inline

# plot the numerical solution
plt.plot(ts, fs, 'c', label='numerical solution');

# plot the analytical solution to check your answer
plt.plot(ts, np.sin(ts), 'b:', label='analytical solution'); # Recall: f'(t) = cos(t), f(t) = sin(t)

plt.xlabel("t")
plt.ylabel("f")
plt.legend(loc='best')
```

The numerical and analytical solutions agree.

Lastly we can add the symbolically function we geernated earlier:

```python
sym_sol = []

for t in ts:
    sym_sol.append(ode_sol.subs([(x,    t)]))       # x = t --> subs t for x

```

Plot the function

```python
plt.xlabel("t")
plt.ylabel("f")
plt.plot(ts, np.sin(ts), 'c', label='analytical solution');
plt.plot(ts, fs, 'r--', label='numerical solution');
plt.plot(ts, fs, 'b:', label='symbolic solution');
plt.legend(loc='best')

```

All solutions agree.

https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.integrate.odeint.html

`odeint` can be used to solve:
- first order ordinary differerntial equations (ODEs)
- systems of coupled first order ODEs

In other words, `odeint` can only solve equations/systems of equations of the form:

$$\frac{dy}{dt}=F(y,t)$$

To solve a higher order ODE, we can reformulate it as a system of first order ODEs.

Suppose we have a second order ODE (e.g. equation for damped simple harmonic motion):

$$y'' + 2y' + 2y = cos(2x)$$

Initial conditions:
$ y(0) = 0, \; \; y'(0) = 0 $

We can rewrite this as two first order ODEs by defining a new dependent variable, $z$.

\begin{align*}
z &= y'\\
z' + 2z + 2y &= cos(2x)\\
\end{align*}

Initial conditions:
$ z(0) = y(0) = 0 $

Now we can solve this system of ODEs using `odeint` exactly as we did for a single ODE

```python
def dU_dx(U, x):
    """
    U is a vector such that U[0] = y and U[1] = z.
    Returns [y', z']
    """
    return [U[1],
            -2*U[1] - 2*U[0] + np.cos(2*x)]

U0 = [0, 0]                   # the initial value
xs = np.linspace(0, 10, 200)  # the value(s) of x at which to evaluate y

# odeint returns y,z at each value of x
Us = odeint(dU_dx, # user-defined function
            U0,    # initial value
            xs)    # value(s) of t at which to evaluate x

print(Us)
ys = Us[:,0]  # y is the first column of the solution U = [y, z]T
```

```python
plt.xlabel("x")
plt.ylabel("y")
plt.title("Damped harmonic oscillator")
plt.plot(xs,ys);
```

# 5. Review Exercises

Compete the exercises below.

## Review Exercise 1 : Differentiation

$$f(x) = cos(x) + bx + c$$

Use symbolic math to find the second derivative of $f(x)$

```python
# Review Exercise 1 : Differentiation

```

## Review Exercise 2 : Integration
$$f(x) = cos(x) + bx + c$$

Find: $\int_{0}^{1} f(x) \; dx$:

1. By using symbolic mathemathics to find the indefinite integral then substituting in the limits of integration to find the definite integral.
2. Solve the definite integral without using symbolic mathematics (where $b=2$, $c=5$)

```python
# Review Exercise 2 : Symbolically

```

```python
# Review Exercise 2 : Non Symbolically

```

## Review Exercise 3 : Estimating the integral of exerimental data

Use the trapezoid method to integrate the array of raw experimental `y` data with respect to indpendent variable `x`.

```
x = np.array([0, 10, 15, 30, 35, 50, 75, 80])

y = np.array([5, 6, 1, 1.5, 2, 5, 6, 1])
```

```python
# Review Exercise 3

```

## Review Exercise 4: Solving ordinary differntial equations.

During braking, a car’s velocity is given by $v = v_{0} e^{−t/R}$.

$v = \frac{dD}{dt}$

Initial condition: distance, $D(0) = 0$

Solve the equtaion using symbolic mathematics to show that the distance travelled while braking as a function of time can be expressed as:

$$D(t) =  Rv_0 - R v_0 e^{−t/R} $$

a) Define variables as symbols sybolically. Define function `D` (distance).

b) Use `Eq` from the `sympy` package to define the differential equation.

c) Use `dsolve` to find the general solution to the equation by solving the equation symbolically.

d) Find the value of the constant of ingtegration $C1$.

e) Substitute $C1$ into the general solution to find the exact solution to the equation.

```python
# Review Exercise 4
```

```python
# Review Exercise 4

```

## Review Exercise 5 : Solving ordinary differntial equations.

$\frac{dy}{dx} + y = x$

Initial value: $y(0) = 1$

Find the numerical solution for $y$ using `odeint` for the range $x = [0, 5]$

```python
# Review Exercise 5
```

```python
# Review Exercise 5

```

```python

```


# Lorenz Equations

A demonstration of reproducible research.

```python
import numpy as np
from scipy import integrate

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.colors import cnames

```

```python
%matplotlib inline
```

## Background

In the early 1960s, [Edward Lorenz](https://en.wikipedia.org/wiki/Edward_Norton_Lorenz), a mathemtaician and meteorologist was studying convection.  Considering a 2-dimensional flow of fluid of uniform depth with an imposed vertical temperature difference.

Simplifying a more general set of equations for convection, Lorenz derived:

$$
\begin{align}
\dot{x} & = \sigma(y-x) \\
\dot{y} & = \rho x - y - xz \\
\dot{z} & = -\beta z + xy
\end{align}
$$

Where

* x is proportional to the itensity of convective motion
* y is proportional to the temperature difference of ascending and descending currents
* z is proportional to the distortion of thr vertical temperature profile
* $\sigma$ is the [Prandtl Number](https://en.wikipedia.org/wiki/Prandtl_number): the ratio of momentum diffusivity (Kinematic viscosity) and thermal diffusivity.
* $\rho$ is the [Rayleigh Number](https://en.wikipedia.org/wiki/Rayleigh_number): ratio of buoyancy and viscosity forces multiplied by the ratio of momentum and thermal diffusivities.
* $\beta$ is a geometric factor.

For more information on the physical meanings, see [this answer](http://physics.stackexchange.com/questions/89880/physical-interpretation-of-the-lorenz-system).  A typical value of the three parameters is $\sigma=10, \beta=8/3, \rho=28$

## Define the equations

```python
def dx(x,y, sigma):
    return sigma*(y-x)
```

```python
def dy(x, y, z, rho):
    return x*(rho-z) - y
```

```python
def dz(x, y, z, beta):
    return x*y-beta*z
```

Now create a function which returns the time derivative. To be able to integrate this numerically, it must accept a time argument t0.

```python
def lorenz_deriv(point, t0, sigma=10, beta=2.666, rho=28):
    """Compute the time-derivative of a Lorentz system

    Arguments;
        point : (x,y,z) values, tuple or list of length 3
        t0    : time value
        sigma : Prandtl number, default=10
        beta  : geometric factor, default=2.666
        rho   : Rayleigh number, default=28

    Returns, the derivative (dx, dy, dt) calculated at point"""
    x = point[0]
    y = point[1]
    z = point[2]
    return [dx(x, y, sigma), dy(x, y, z, rho), dz(x, y, z, beta)]
```

Create a series of timesteps to integrate over.

```python
max_time = 100
t = np.linspace(0, max_time, int(250*max_time))
```

## Integrate numerically

Lorenz simulated the behaviour of these equations on an [LGP-30](https://en.wikipedia.org/wiki/LGP-30), a "desktop" machine weighing >300Kg, and taking tape input.  Since simulations took a long time, he would often print out intermediate results and restart the simulations from somewhere in the middle.  The intermediate results were truncated to 3 decimal places, which lead to his famous discovery...

First simulate the system with a low value of $\rho$, meaning conduction is favoured over convection.

```python
x0 = 3.0
y0 = 15.0
z0 = 0
sigma = 10
beta = 2.666
rho = 10
epsilon = 0.001
```

Here we use the `scipy.integrate.odeint` function, which uses the [LSODA](http://www.oecd-nea.org/tools/abstract/detail/uscd1227) solver.

```python
r1 = integrate.odeint(lorenz_deriv, (x0, y0, z0), t, args=(sigma, beta, rho))
```

And redo the simulation with slightly different initial conditions...

```python
r2 = integrate.odeint(lorenz_deriv, (x0, y0+epsilon, z0), t, args=(sigma, beta, rho))
```

Plot the results, examine intensity of convection over time...

```python
fig = plt.figure(figsize=(15,5))
ax1 = fig.add_subplot(211)
ax2 = fig.add_subplot(212)
ax1.plot(t, r1[:,0], 'r-', linewidth=2)
ax2.plot(t, r2[:,0], 'b-', linewidth=2)
```

Not so interesting. Steady state solution shows that convection doesn't occur.  In this case, making a small change to the initial conditions doesn't matter. Plot a scatter of results from the two runs.

```python
plt.scatter(r1[:,0], r2[:,0], marker='.')
```

Show how x, y, and z evolve over time

```python
fig = plt.figure(figsize=(7,7))
ax = fig.add_axes([0, 0, 1, 1], projection='3d')
x, y, z = r1.T
ax.plot(x, y, z, 'r-', linewidth=1)
x, y, z = r2.T
lines = ax.plot(x, y, z, 'b-', linewidth=1)
```

## Exercise

**Rerun the code with a more realistic value of $\rho=28$.  **

## The Lorenz Attractor

We can explore in more detail what happens over different ranges of starting points

```python
N = 5
sigma = 10
beta = 2.666
rho = 28

# generate random initial conditions uniform(-15, 15)
x0 = -15 + 30 * np.random.random((N, 3))
x0
```

    array([[ -5.04117765, -11.17065497,  -1.36463646],
           [ 12.80126419, -12.38458801,  -2.72177287],
           [-14.99277182,   0.24928451,  -9.76713759],
           [ -5.45408094,  14.60035494,  10.50024973],
           [ 10.92013321, -13.57775883, -13.42161216]])

```python
results = np.asarray([integrate.odeint(lorenz_deriv, x0i, t) for x0i in x0])

fig = plt.figure(figsize=(7,7))
ax = fig.add_axes([0, 0, 1, 1], projection='3d')

# choose a different color for each trajectory
colors = plt.cm.jet(np.linspace(0, 1, N))

for i in range(N):
    x, y, z = results[i,:,:].T
    ax.plot(x, y, z, '-', c=colors[i])

plt.show()

```

## Conlcusions

The Lorenz equations behave *chaoticly*, exhibiting extreme *sensitivity to inital conditions* at certain parameter values.

The values of x, y, and z tend toward two regions, representing two semi-stable states. Values can remain for some time in the same region, then suddenly flip into another state.

Broad aspects of the system can be predicted, but exact details such as when convection will begin and end are impossible to predict beyond a certain timescale.  Because of the feedbacks between variables, even tiny deviations in initial conditions will grow over time.


# Rayleigh quotient
Consider a linear least squares problem. Given a vector $\vec{b} \in \mathrm{R}^m$ and a matrix $A \in \mathrm{R} ^{m\times n}$, the best fit to $\vec{b}$ from the range of $A$ is $A\vec{x}$, where $\vec{x}$ is the linear least square solution and can be found through solving the normal equation

$$\newcommand{\RR}{\mathbb{R}}\newcommand{\ffrac}{\displaystyle \frac}\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}\Tran{A}A\vec{x} = \Tran{A}\vec{b}$$

Now we let $A$ be a real symmetric $n \times n$ matrix. Given an arbitary vector $\vec{x} \in \RR^n$, which may not be the eigenvector of $A$, we try to best fit that to $A\vec{x}$ among all multiple of $\vec{x}$.

$$\alpha^* = \arg \min_{\alpha\in \RR} \left\| \vec{x}\alpha - A\vec{x} \right\|_2$$

so that the solution $\alpha^*$ satisfies the equation:

$$\Tran{\vec{x}}\vec{x} \alpha^* = \Tran{\vec{x}}A\vec{x} \Longrightarrow \alpha^* = \frac{\Tran{\vec{x}}A\vec{x}} {\Tran{\vec{x}}\vec{x}}$$

And it's obvious that if $\vec{x}$ is indeed an eigenvector of matrix $A$, then

$$\alpha^* = \frac{\Tran{\vec{x}}A\vec{x}} {\Tran{\vec{x}}\vec{x}} = \frac{\Tran{\vec{x}}\lambda\vec{x}} {\Tran{\vec{x}}\vec{x}} = \lambda$$

And in particular, given a real symmetric matrix $A \in \RR^{n \times n}$, we call **Rayleigh quotient** of $A$ at any vector $\vec{x} \in \RR^n$

$$r(\vec{x})= \frac{\Tran{\vec{x}}A\vec{x}} {\Tran{\vec{x}}\vec{x}}$$
***

Then we do Taylor expansion of multiple variables.

$$r\left(\vec{x}\right) = r\left(\vec{q}\right) + \nabla \Tran{r\left(\vec{q}\right)} \left( \vec{x} - \vec{q} \right) + \frac{\Tran{\left( \vec{x} - \vec{q} \right)} \nabla^2 r\left(\tilde{\vec{q}}\right)\left( \vec{x} - \vec{q} \right)} {2}$$

Here $\vec{q}$ is an eigenvector of $A$ corresponding to eigenvalue $\lambda$. $\nabla r\left(\vec{q}\right)$ is the Jacobian of $r\left(\vec{x}\right)$ at $\vec{x} = \vec{q}$; $\nabla^2 r\left(\tilde{\vec{q}}\right)$ is the Hessian matrix of the function $r\left(\vec{x}\right)$ at $\tilde{\vec{q}}$ which is close to $\vec{q}$.

$\nabla r\left(\vec{q}\right)$ is a column vector, and contains the partial drivative of $r\left(\vec{x}\right)$ with respect to the components of $\vec{x}$. From the definition $r(\vec{x})= \ffrac{\Tran{\vec{x}}A\vec{x}} {\Tran{\vec{x}}\vec{x}}$, we have

$$
\frac{\partial r\left(\vec{x}\right)} {\partial x_k} = \frac{\frac{\partial \left(\Tran{\vec{x}}A\vec{x} \right)} {\partial {x_{k}}} \cdot \left(\Tran{\vec{x}}\vec{x}\right) - \left(\Tran{\vec{x}} A \vec{x}\right) \cdot \frac{ \partial \left(\Tran{\vec{x}}\vec{x}\right)} {\partial {x_{k}}}} {\left( \Tran{\vec{x}}\vec{x} \right)^2}
$$

One way to derive the formula for $\ffrac{\partial \left(\Tran{\vec{x}}A\vec{x} \right)} {\partial {x_{k}}}$ is to decomposite it completely, another way is shown below

$$\begin{align}
\frac{\partial \left(\Tran{\vec{x}}A\vec{x} \right)} {\partial {x_{k}}} &= \frac{\partial \left(\Tran{\vec{x}}A \right)} {\partial {x_{k}}} \cdot \vec{x} + \Tran{\vec{x}} \cdot \ffrac{\partial \left(A\vec{x} \right)} {\partial {x_{k}}} \\
&= 2 \cdot \frac{\partial \left(\Tran{\vec{x}}A \right)} {\partial {x_{k}}} \cdot \vec{x} \\
&= 2 \cdot \frac{\partial \left(
\left[
\begin{array}{cccc}
\Tran{\vec{x}} \vec{a}_1^c & \Tran{\vec{x}} \vec{a}_2^c & \cdots & \Tran{\vec{x}} \vec{a}_n^c
\end{array}
\right]
\right)} {\partial {x_{k}}} \cdot \vec{x} \\
&= 2 \cdot \left[
\begin{array}{cccc}
a_{k1} & a_{k2} & \cdots & a_{kn}
\end{array}
\right] \cdot \vec{x} \\
&= 2\cdot \vec{a}_k ^r \cdot \vec{x} \\
&= 2 \cdot k\text{-th row of }A\vec{x}
\end{align}$$

Then we have

$$
\begin{align}
\frac{\partial r\left(\vec{x}\right)} {\partial x_k} &= \frac{\frac{\partial \left(\Tran{\vec{x}}A\vec{x} \right)} {\partial {x_{k}}} \cdot \left(\Tran{\vec{x}}\vec{x}\right) - \left(\Tran{\vec{x}} A \vec{x}\right) \cdot \frac{ \partial \left(\Tran{\vec{x}}\vec{x}\right)} {\partial {x_{k}}}} {\left( \Tran{\vec{x}}\vec{x} \right)^2} \\
&= \frac{2\cdot \left( A \vec{x} \right)_{k\text{-th row}}\left(\Tran{\vec{x}}\vec{x} \right) - \left(\Tran{\vec{x}} A \vec{x}\right)\cdot \left( 2x_k \right)} {\left( \Tran{\vec{x}}\vec{x} \right)^2} \\
&= \frac{2} {\Tran{\vec{x}}\vec{x}} \left( A\vec{x} - r\left(\vec{x}\right) \vec{x} \right)_{k\text{-th row}}
\end{align}$$

So that

$$\nabla r\left( \vec{x} \right) = \Tran{\left[
\begin{array}{cccc}
\frac{\partial r\left(\vec{x}\right)} {\partial x_1}&\frac{\partial r\left(\vec{x}\right)} {\partial x_2} & \cdots & \frac{\partial r\left(\vec{x}\right)} {\partial x_n}
\end{array}
\right]} = \frac{2} {\Tran{\vec{x}}\vec{x}} \left( A\vec{x} - r\left(\vec{x}\right) \vec{x} \right)$$

We can plug in eigenvector $\vec{q}$ for check

$$\nabla r\left( \vec{q} \right) = \frac{2} {\Tran{\vec{q}}\vec{q}} \left( A\vec{q} - \lambda \vec{q} \right) = \vec{0}$$

Then the Taylor expansion of $r\left( \vec{q} \right)$ at $\vec{q}$ becomes

$$\boxed{ r\left( \vec{q} \right) = \lambda + \frac{\Tran{\left( \vec{x} - \vec{q} \right)} \nabla^2 r\left(\tilde{\vec{q}}\right)\left( \vec{x} - \vec{q} \right)} {2} }$$

Assuming that $\left\| \nabla^2 r\left( \tilde{\vec{q}} \right) \right\|$ is bounded by a constant, then we have

$$\left| r\left( \vec{x} \right) - \lambda \right| = O \left( \left\| \vec{x} - \vec{q} \right\|^2 \right)$$

$Conclusion$

Given a symmetric matrix $A$, if $\vec{x}$ is an approximation of an eigenvector of $A$, then its Rayleigh quotient $r\left(\vec{x}\right)$ approximates the corresponding eigenvalue of $A$. Besides, the accuracy is even better than the approximation of the eigenvector.

What's more, if $\vec{x}$ is normalized, $i.e.$, if $\left\|\vec{x}\right\|_2 = 1$, then the Rayleigh quotient of $\vec{x}$ is $r\left( \vec{x} \right) = \Tran{\vec{x}}A \vec{x}$.

# A few properties of eigenvalues
## Inversion
If $A$ is nonsingular and $A\vec{x} = \lambda \vec{x}$, for $\vec{x} \neq \vec{0}$, then $A^{-1} \vec{x} = \ffrac{1} {\lambda} \vec{x}$. In another word, if $\lambda$ is an eigenvalue of $A$, then $\lambda^{-1}$ is the eigenvalue of $A^{-1}$, and they share the same eigenvectors.

## Powers
If $A\vec{x} = \lambda \vec{x}$, then for any integer $k$, we have $A^k \vec{x} = \lambda^k \vec{x}$, $i.e.$,  taking the $k\text{-th}$ power of a matrix also takes the $k\text{-th}$ power of its eigenvalues, while the eigenvectors are the same.

## Shift
If $A\vec{x} = \lambda \vec{x}$, the for any value $\sigma$, $\left( A - \sigma I\right) \vec{x} = \left( \lambda  - \sigma \right) \vec{x}$. Thus, the eigenvalues of the matrix $A - \sigma I$ are shifted from the eigenvalues of
$A$ by $\sigma$, and again their eigenvectors are unchanged.

# Power iteration
Given a real symmetrix $n\times n$ matrix $A$, it has $n$ real eigenvalues $\lambda_1,\lambda_2,\dots, \lambda_n$ and a set of orthonormal eigenvectors $\vec{q}_1, \vec{q}_2, \dots, \vec{q}_n$, which satisfy $A\vec{q}_i = \lambda_i \vec{q}_i$ for $i = 1,2,\dots, n$. And by default, the eigenvalues follow the order $\left|\lambda_1\right| \geq \left|\lambda_2\right| \geq \cdots \geq \left|\lambda_n\right|$.

Since now $\vec{q}_1, \vec{q}_2, \dots, \vec{q}_n$ is also a base of $n\text{-dimensional}$ vector space, so that any vector $\vec{v}^{(0)}$ can be written as

$$
\begin{array}{rl}
\vec{v}^{(0)}\!\!\! & =\; a_1 \vec{q}_1 + a_2 \vec{q}_2 + \cdots a_n \vec{q}_n \\
A\vec{v}^{(0)}\!\!\! & =\;  A \left( a_1 \vec{q}_1 + a_2 \vec{q}_2 + \cdots a_n \vec{q}_n \right) \\
& =\;  a_1 \lambda_1 \vec{q}_1 + a_2 \lambda_2 \vec{q}_2 + \cdots a_n \lambda_n \vec{q}_n \\
A^2\vec{v}^{(0)}\!\!\! & =\;  A \left( a_1 \lambda_1 \vec{q}_1 + a_2 \lambda_2 \vec{q}_2 + \cdots a_n \lambda_n \vec{q}_n \right)\\
& =\;  a_1 \lambda_1^2 \vec{q}_1 + a_2 \lambda_2^2 \vec{q}_2 + \cdots a_n \lambda_n^2 \vec{q}_n \\
& \;  \vdots \\
A^k\vec{v}^{(0)}\!\!\! &= \;  A \left( a_1 \lambda_1^{k-1} \vec{q}_1 + a_2 \lambda_2^{k-1} \vec{q}_2 + \cdots a_n\lambda_n^{k-1} \vec{q}_n \right) \\
& =\;  a_1 \lambda_1^k \vec{q}_1 + a_2 \lambda_2^k \vec{q}_2 + \cdots a_n \lambda_n^k \vec{q}_n \\
& =\;  a_1 \lambda_1^k \left( \vec{q}_1 + \frac{a_2} {a_1} \left( \frac{\lambda_2} {\lambda_1} \right)^k \vec{q}_2 + \cdots + \frac{a_n} {a_1} \left( \frac{\lambda_n} {\lambda_1}\right)^k \vec{q}_n \right)
\end{array}
$$

Now we need two stronger prerequisites:
1. $\left|\lambda_1\right| > \left|\lambda_2\right| \geq \cdots \geq \left|\lambda_n\right|$
2. the initial vector $\vec{v}^{(0)}$ has a nonzero component in the direction of eigenvector $\vec{q}_1$.

$$\lim_{k \to \infty} \frac{A^k \vec{v}^{(0)}} {a_1 \lambda_1^k} \to \vec{q}_1$$

And the speed of convergence is determined by

$$\left\| \frac{A^k \vec{v}^{(0)}} {a_1 \lambda_1^k} - \vec{q}_1 \right\| = O\left( \left| \frac{\lambda_2} {\lambda_1} \right|^k \right)$$

meaning that the convergence will be faster if $\left| \lambda_2 \right|$ is much smaller than $\left| \lambda_1 \right|$, and slower if $\left| \lambda_2 \right|$ is closer to $\left| \lambda_1 \right|$.

In addition, we can also see that $\left\| \ffrac{A^{k-1} \vec{v}^{(0)}} {a_1 \lambda_1^{k-1}} - \vec{q}_1 \right\| = O\left( \left| \ffrac{\lambda_1} {\lambda_2} \right|^{k-1} \right)$, so that

$$\left\| \frac{A^k \vec{v}^{(0)}} {a_1 \lambda_1^k} - \vec{q}_1 \right\| = \left\| \frac{A^{k-1} \vec{v}^{(0)}} {a_1 \lambda_1^{k-1}} - \vec{q}_1 \right\| \cdot O\left( \left| \frac{\lambda_2} {\lambda_1} \right| \right)$$

which implies that *the convergence rate is linear* and is reduced by a factor of $\ffrac{\lambda_2} {\lambda_1}$ after each step. So that we define the sequence to approximate the eigenvector corresponding the largest eigenvalue of $A$.

$$
\begin{align}
\text{Initial } \vec{v}^{(0)}, \left\| \vec{v}^{(0)}\right\| &= 1 \\
\vec{v}^{(1)} &= \frac{A \vec{v}^{(0)}} {\left\| A \vec{v}^{(0)} \right\|} \\
\vec{v}^{(2)} &= \frac{A \vec{v}^{(1)}} {\left\| A \vec{v}^{(1)} \right\|} \\
& \vdots \\
\vec{v}^{(k)} &= \frac{A \vec{v}^{(0)}} {\left\| A \vec{v}^{(0)} \right\|} \\
& \vdots \\
\end{align}
$$

so that $\vec{v}^{(k)}$ converges to the eigenvector $\vec{q}_1$ corresponding the largest eigenvalue of $A$.

$Algorithm$

Given an initial $\vec{v}^{(0)}$ with $\left\| \vec{v}^{(0)}\right\| = 1$

```
for k = 1:n
    v = A v ^{(k − 1)}
    v ^{(k)} = v / \|v\|
    \lambda ^{(k)} = v^{(k)}' A v^{(k)}
end
```

We can check $\left\| A \vec{v}^{(k)} - \lambda^{(k)}\vec{v}^{(k)} \right\|$ so that jump out of the loop when it's sufficiently small.
***
$$\left\| \vec{v}^{(k)} - \vec{q}_1 \right\| = \left\| \vec{v}^{(k-1)} - \vec{q}_1 \right\|\cdot O\left( \left| \frac{\lambda_2} {\lambda_1} \right| \right) \Longrightarrow \left\| \vec{v}^{(k)} - \vec{q}_1 \right\| = O\left( \left| \frac{\lambda_2} {\lambda_1} \right|^{k} \right)$$

$$\left| \lambda^{(k)} - \lambda_1 \right| = \left| \lambda^{(k-1)} - \lambda_1 \right|\cdot O\left( \left| \frac{\lambda_2} {\lambda_1} \right|^{2} \right) \Longrightarrow \left| \lambda^{(k)} - \lambda_1 \right| = O\left( \left| \frac{\lambda_2} {\lambda_1} \right|^{2k} \right)$$

$\dagger$ Here there's a square and it's because the conclusion from the Rayleigh Quotient. Same for the Rayleigh quotient iteration.$\Join$

# Inverse iteration
This is used to approximate an eigenvector corresponding to the eigenvalue closest to an initially specified value.

Given real value $\mu$, the eigenvalues of the matrix $\left( A - \mu I \right)^{-1}$ are simply $\left( \lambda_i - \mu \right)^{-1}$ where $\lambda_i$ are the eigenvalues of $A$. Assuming that $\lambda_j$ is the closest one to $\mu$, and the next one $\lambda_l$, $i.e.$, we have the following relation

$$\left| \frac{1} {\lambda_j - \mu} \right| > \left| \frac{1} {\lambda_l - \mu} \right| \geq \cdots$$

So we apply the power iteration to matrix $\left( A - \mu I \right)^{-1}$ and it will converge to the eigenvector of the matrix $\left( A - \mu I \right)^{-1}$ corresponding to its largest eigenvalue $\frac{1} {\lambda_j- \mu}$, which is the exact eigenvector $\lambda_j$ of $A$.

$Algorithm$

Given an initial $\vec{v}^{(0)}$ with $\left\| \vec{v}^{(0)}\right\| = 1$, and a number $\mu$

```
for k = 1:n
    v = (A - \mu I) ^{-1} v ^{(k − 1)}
    v ^{(k)} = v / \|v\|
    \lambda ^{(k)} = v^{(k)}' A v^{(k)}
end
```
***
And with same logic we have the following

$$\left\| \vec{v}^{(k)} - \vec{q}_j \right\| = \left\| \vec{v}^{(k-1)} - \vec{q}_j \right\|\cdot O\left( \left| \frac{\lambda_j - \mu} {\lambda_l - \mu} \right| \right) \Longrightarrow \left\| \vec{v}^{(k)}-\vec{q}_j \right\| = O\left( \left| \frac{\lambda_j - \mu} {\lambda_l - \mu} \right|^{k} \right)$$

$$\left| \lambda^{(k)} - \lambda_j \right| = \left| \lambda^{(k-1)} - \lambda_j \right|\cdot O\left( \left| \frac{\lambda_j - \mu} {\lambda_l - \mu} \right|^{2} \right) \Longrightarrow \left| \lambda^{(k)} - \lambda_j \right| = O\left( \left| \frac{\lambda_j - \mu} {\lambda_l - \mu} \right|^{2k} \right)$$

This process is more expensive because more calculations are required when solving the linear system, the first step of the iteration, $\vec{v} = \left(A - \mu I \right) ^{-1} \cdot \vec{v} ^{(k − 1)} \Longleftrightarrow \left(A - \mu I \right)\vec{v} = \vec{v} ^{(k − 1)}$. But luckly we only need to implement $LU$or Cholesky  factorization for once the entire time.

# Rayleigh quotient iteration
In the inverse iteration, $\lambda^{(k)}$ converges to $\lambda_j$ and the convergence rate depends on $\mu$. The convergence will become faster if $\mu$ is much closer to $\lambda_j$.

So that we replace $\mu$ with the latest $\lambda^{(k)}$ at each step of the inverse iteration.

$Algorithm$

Given an initial $\vec{v}^{(0)}$ with $\left\| \vec{v}^{(0)}\right\| = 1$, and a number $\mu$

```
\lambda ^{(0)} = v^{(0)}' A v^{(0)}

for k = 1:n
    v = (A - \lambda ^{(k-1)} I) ^{-1} v ^{(k − 1)}
    v ^{(k)} = v / \|v\|
    \lambda ^{(k)} = v^{(k)}' A v^{(k)}
end
```
***
$$\begin{align}
\left\| \vec{v}^{(k)} - \vec{q}_j \right\| &= \left\| \vec{v}^{(k-1)} - \vec{q}_j \right\|\cdot O\left( \left| \frac{\lambda_j - \mu} {\lambda_l - \mu} \right| \right) \\
&= \left\| \vec{v}^{(k-1)} - \vec{q}_j \right\|\cdot O\left( \left| \frac{\lambda_j - \lambda ^{(k-1)}} {\lambda_l - \lambda ^{(k-1)}} \right| \right) \\
&= \left\| \vec{v}^{(k-1)}-\vec{q}_j \right\|\cdot O\left( \left| {\lambda_j- \lambda ^{(k-1)}} \right| \right) \\
&= O\left( \left\| \vec{v}^{(k-1)}-\vec{q}_j \right\|^3 \right)
\end{align}$$

$$\left|\lambda^{k}-\lambda_j\right| = O\left( \left\| \vec{v}^{(k)}-\vec{q}_j \right\|^2 \right) = O\left( \left\| \vec{v}^{(k-1)}-\vec{q}_j \right\|^6\right)$$

And still it's more expensive because at each step you need to apply the $LU$ or Cholesky factorization to solve the linear system.

# Computational cost
If $A$ is an $m \times m$ dense matrix, then the computational cost of the [power iteration](#Power-iteration) for each step is $O(m^2)$ for multiplying a matrix with a vector. For the inverse iteration, one $LU$ factorization of $A-\lambda I$ costs $O(m^3)$ ops, which only needs to be computed once in the inverse iteration, and then in each step of the iteration, the cost is $O(m^2)$ for the forward and backward substitutions. For the Rayleigh quotient iteration, the cost for each step is $O(m^3)$ for solving
the system of linear equations.

However, the computation cost can be greatly reduced, especially for the Rayleigh quotient iteration, if we first reduce the symmetric matrix $A$ to a tridiagonal for by a sequence of symmetric orthogonal transformations which does not change the eigenvalues of $A$.

# Reduction to Tridiagonal form

We know that for a real symmetric matrix $A$, it has an eigenvalue decomposition $A = Q \Lambda Q^{\mathrm{T}}$, where $\Lambda$ is a diagonal matrix which contains the eigenvalues of $A$, and $Q$ is an orthogonal matrix which contains the eigenvectors of $A$.

However in general we can only multiply a sequence of orthogonal matrices to the left and their transpose to the right of $A$ such that the result is a tridiagonal matrix. And such orthogonal matrices can be determined by the
using Householder reflector.

$$\underset{Q^{\mathrm{T}}}{\underbrace{Q_{m-2}\cdots Q_{2}Q_{1}}} \;\; A \;\; \underset{Q}{\underbrace{Q_{1}^{\mathrm{T}}Q_{2}^{\mathrm{T}}\cdots Q_{m-2}^{\mathrm{T}}}} = T = \left[ \begin{array}{ccccc}
\times & \times & & & \\
\times & \times & \times & & \\
& \times & \times & \times & \\
& & \times & \times & \times \\
& & & \times & \times
\end{array}
\right]$$

We can also see that $A$ is similar to $T$ and they have the same eigenvalues.

$Algorithm$

```
for k = 1 to m - 2
    x = A_{k+1:m,k}
    v_{k} = x + sign( x_{1} ) \| x \|_{2} e_{1}
    v_{k} = v_{k} / \| v_{k} \|_{2}
    A_{k+1:m,k:m} = A_{k+1:m,k:m} - 2 v_{k} ( v_{k}' A_{k+1:m,k:m} )
    A_{k:m,k+1:m} = A_{k:m,k+1:m}  2 ( A_{k:m,k+1:m} v_{k} ) v_{k}'
end
```

In general, an eigenvalue problem solver first apply the above algorithm to reduce $A$ into a tridiagonal form $T$. Then an iteration after that. And for the reduction part the cost for each iteration is also $O\left(m^{2}\right)$

Computational/Intro to Numerical Computing/Note_Chap09_Power iteration, inverse iteration, and Rayleigh quotient iteration.ipynb