# 18 PDEs: Waves – Students

(See *Computational Physics* Ch 21 and *Computational Modeling* Ch 6.5.)

## Background: waves on a string

Assume a 1D string of length $L$ with mass density per unit length $\rho$ along the $x$ direction. It is held under constant tension $T$ (force per unit length). Ignore frictional forces and the tension is so high that we can ignore sagging due to gravity.

### 1D wave equation
The string is displaced in the $y$ direction from its rest position, i.e., the displacement $y(x, t)$ is a function of space $x$ and time $t$.

For small relative displacements $y(x, t)/L \ll 1$ and therefore small slopes $\partial y/\partial x$ we can describe $y(x, t)$ with a *linear* equation of motion:

Newton's second law applied to short elements of the string with length $\Delta x$ and mass $\Delta m = \rho \Delta x$: the left hand side contains the *restoring force* that opposes the displacement, the right hand side is the acceleration of the string element:

\begin{align}
\sum F_{y}(x) &= \Delta m\, a(x, t)\\
T \sin\theta(x+\Delta x) - T \sin\theta(x) &= \rho \Delta x \frac{\partial^2 y(x, t)}{\partial t^2}
\end{align}

The angle $\theta$ measures by how much the string is bent away from the resting configuration.

Because we assume small relative displacements, the angles are small ($\theta \ll 1$) and we can make the small angle approximation

$$
\sin\theta \approx \tan\theta = \frac{\partial y}{\partial x}
$$

and hence

\begin{align}
T \left.\frac{\partial y}{\partial x}\right|_{x+\Delta x} - T \left.\frac{\partial y}{\partial x}\right|_{x}  &= \rho \Delta x \frac{\partial^2 y(x, t)}{\partial t^2}\\
\frac{T \left.\frac{\partial y}{\partial x}\right|_{x+\Delta x} - T \left.\frac{\partial y}{\partial x}\right|_{x}}{\Delta x} &= \rho \frac{\partial^2 y}{\partial t^2}
\end{align}

or in the limit $\Delta x \rightarrow 0$ a linear hyperbolic PDE results:

\begin{gather}
\frac{\partial^2 y(x, t)}{\partial x^2} = \frac{1}{c^2} \frac{\partial^2 y(x, t)}{\partial t^2}, \quad c = \sqrt{\frac{T}{\rho}}
\end{gather}

where $c$ has the dimension of a velocity. This is the (linear) **wave equation**.

### General solution: waves

General solutions are propagating waves:

If $f(x)$ is a solution at $t=0$ then

$$
y_{\mp}(x, t) = f(x \mp ct)
$$

are also solutions at later $t > 0$.

Because of linearity, any linear combination is also a solution, so the most general solution contains both right and left propagating waves

$$
y(x, t) = A f(x - ct) + B g(x + ct)
$$

(If $f$ and/or $g$ are present depends on the initial conditions.)

In three dimensions the wave equation is

$$
\boldsymbol{\nabla}^2 y(\mathbf{x}, t) - \frac{1}{c^2} \frac{\partial^2 y(\mathbf{x}, t)}{\partial t^2} = 0\
$$

### Boundary and initial conditions

* The boundary conditions could be that the ends are fixed

  $$y(0, t) = y(L, t) = 0$$

* The *initial condition* is a shape for the string, e.g., a Gaussian at the center

  $$
  y(x, t=0) = g(x) = y_0 \frac{1}{\sqrt{2\pi\sigma}} \exp\left[-\frac{(x - x_0)^2}{2\sigma^2}\right]
  $$

  at time 0.
* Because the wave equation is *second order in time* we need a second initial condition, for instance, the string is released from rest:

  $$
  \frac{\partial y(x, t=0)}{\partial t} = 0
  $$

  (The derivative, i.e., the initial displacement velocity is provided.)

### Analytical solution
Solve (as always) with *separation of variables*.

$$
y(x, t) = X(x) T(t)
$$

and this yields the general solution (with boundary conditions of fixed string ends and initial condition of zero velocity) as a superposition of normal modes

$$
y(x, t) = \sum_{n=0}^{+\infty} B_n \sin k_n x\, \cos\omega_n t,
\quad \omega_n = ck_n,\ k_n = n \frac{2\pi}{L} = n k_0.
$$

(The angular frequency $\omega$ and the wave vector $k$ are determined from the boundary conditions.)

The coefficients $B_n$ are obtained from the initial shape:

$$
y(x, t=0) = \sum_{n=0}^{+\infty} B_n \sin n k_0 x = g(x)
$$

In principle one can use the fact that $\int_0^L dx \sin m k_0 x \, \sin n k_0 x = \pi \delta_{mn}$ (orthogonality) to calculate the coefficients:

\begin{align}
\int_0^L dx \sin m k_0 x \sum_{n=0}^{+\infty} B_n \sin n k_0 x &= \int_0^L dx \sin(m k_0 x) \, g(x)\\
\pi \sum_{n=0}^{+\infty} B_n \delta_{mn} &= \dots \\
B_m &= \pi^{-1} \dots
\end{align}

(but the analytical solution is ugly and I cannot be bothered to put it down here.)

## Numerical solution

1. discretize wave equation
2. time stepping: leap frog algorithm (iterate)

Use the central difference approximation for the second order derivatives:

\begin{align}
\frac{\partial^2 y}{\partial t^2} &\approx \frac{y(x, t+\Delta t) + y(x, t-\Delta t) - 2y(x, t)}{\Delta t ^2} = \frac{y_{i, j+1} + y_{i, j-1} - 2y_{i,j}}{\Delta t^2}\\
\frac{\partial^2 y}{\partial x^2} &\approx \frac{y(x+\Delta x, t) + y(x-\Delta x, t) - 2y(x, t)}{\Delta x ^2} = \frac{y_{i+1, j} + y_{i-1, j} - 2y_{i,j}}{\Delta x^2}
\end{align}

and substitute into the wave equation to yield the *discretized* wave equation:

$$
\frac{y_{i+1, j} + y_{i-1, j} - 2y_{i,j}}{\Delta x^2} = \frac{1}{c^2} \frac{y_{i, j+1} + y_{i, j-1} - 2y_{i,j}}{\Delta t^2}
$$

#### Student activity: derive the finite difference version of the 1D wave equation
Re-arrange so that the future terms $j+1$ can be calculated from the present $j$ and past $j-1$ terms:

$$
? = ?
$$

Use $\beta := \frac{c}{\Delta x/\Delta t}$ to write your solution.

This is the time stepping algorithm for the wave equation.

## Numerical implementation

```python
# if you have plotting problems, try
# %matplotlib inline
%matplotlib notebook

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

plt.style.use('ggplot')
```

Implement the time stepping algorithm in the code below. Look for sections `# TODO`.

```python
L = 0.5  # m
Nx = 50
Nt = 100

Dx = L/Nx
# TODO: choose Dt
Dt =   # s

rho = 1.5e-2   # kg/m
tension = 150  # N

c = np.sqrt(tension/rho)

# TODO: calculate beta
beta =
beta2 =

print("c = {0} m/s".format(c))
print("Dx = {0} m,  Dt = {1} s, Dx/Dt = {2} m/s".format(Dx, Dt, Dx/Dt))
print("beta = {}".format(beta))

X = np.linspace(0, L, Nx+1)  # need N+1!

def gaussian(x, y0=0.05, x0=L/2, sigma=0.1*L):
    return y0/np.sqrt(2*np.pi*sigma) * np.exp(-(x-x0)**2/(2*sigma**2))

# displacements at j-1, j, j+1
y0 = np.zeros_like(X)
y1 = np.zeros_like(y0)
y2 = np.zeros_like(y0)

# save array
y_t = np.zeros((Nt+1, Nx+1))

# boundary conditions
# TODO: set boundary conditions
y2[:] = y0

# initial conditions: velocity 0, i.e. no difference between y0 and y1
y0[1:-1] = y1[1:-1] = gaussian(X)[1:-1]

# save initial
t_index = 0
y_t[t_index, :] = y0
t_index += 1
y_t[t_index, :] = y1

for jt in range(2, Nt):
    # TODO: time stepping algorithm

    t_index += 1
    y_t[t_index, :] = y2
    print("Iteration {0:5d}".format(jt), end="\r")
else:
    print("Completed {0:5d} iterations: t={1} s".format(jt, jt*Dt))

```

### 1D plot
Plot the output in the save array `y_t`. Vary the time steps that you look at with `y_t[start:end]`.

We indicate time by color changing.

```python
ax = plt.subplot(111)
ax.set_prop_cycle("color", [plt.cm.viridis_r(i) for i in np.linspace(0, 1, len(y_t))])
ax.plot(X, y_t.T);
```

### 1D Animation
For 1D animation to work in a Jupyter notebook, use

```python
%matplotlib notebook
```

If no animations are visible, restart kernel and execute the `%matplotlib notebook` cell as the very first one in the notebook.

We use `matplotlib.animation` to look at movies of our solution:

```python
import matplotlib.animation as animation
```

The `update_wave()` function simply re-draws our image for every `frame`.

```python
y_limits = 1.05*y_t.min(), 1.05*y_t.max()

fig1 = plt.figure(figsize=(5,5))
ax = fig1.add_subplot(111)
ax.set_aspect(1)

def update_wave(frame, data):
    global ax, Dt, y_limits
    ax.clear()
    ax.set_xlabel("x (m)")
    ax.set_ylabel("y (m)")
    ax.plot(X, data[frame])
    ax.set_ylim(y_limits)
    ax.text(0.1, 0.9, "t = {0:3.1f} ms".format(frame*Dt*1e3), transform=ax.transAxes)

wave_anim = animation.FuncAnimation(fig1, update_wave, frames=len(y_t), fargs=(y_t,),
                                    interval=30, blit=True, repeat_delay=100)

```

### 3D plot
(Uses functions from previous lessons.)

```python
def plot_y(y_t, Dt, Dx, step=1):
    T, X = np.meshgrid(range(y_t.shape[0]), range(y_t.shape[1]))
    Y = y_t.T[X, T]    # intepret index 0 as "t" and index 1 as "x", but plot x along axis 1 and t along axis 2
    fig = plt.figure()
    ax = fig.add_subplot(111, projection="3d")
    ax.plot_wireframe(X*Dx, T*Dt*step, Y)
    ax.set_ylabel(r"time $t$ (s)")
    ax.set_xlabel(r"position $x$ (m)")
    ax.set_zlabel(r"displacement $y$ (m)")
    fig.tight_layout()
    return ax

def plot_surf(y_t, Dt, Dx, step=1, filename=None, offset=-1, zlabel=r'displacement',
             elevation=40, azimuth=-20, cmap=plt.cm.coolwarm):
    """Plot y_t as a 3D plot with contour plot underneath.

    Arguments
    ---------
    y_t : 2D array
          displacement y(t, x)
    filename : string or None, optional (default: None)
          If `None` then show the figure and return the axes object.
          If a string is given (like "contour.png") it will only plot
          to the filename and close the figure but return the filename.
    offset : float, optional (default: 20)
          position the 2D contour plot by offset along the Z direction
          under the minimum Z value
    zlabel : string, optional
          label for the Z axis and color scale bar
    elevation : float, optional
          choose elevation for initial viewpoint
    azimuth : float, optional
          chooze azimuth angle for initial viewpoint
    """

    t = np.arange(y_t.shape[0], dtype=int)
    x = np.arange(y_t.shape[1], dtype=int)
    T, X = np.meshgrid(t, x)
    Y = y_t.T[X, T]

    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    surf = ax.plot_surface(X*Dx, T*Dt*step, Y, cmap=cmap, rstride=1, cstride=1, alpha=1)
    cset = ax.contourf(X*Dx, T*Dt*step, Y, 20, zdir='z', offset=offset+Y.min(), cmap=cmap)

    ax.set_xlabel('x')
    ax.set_ylabel('t')
    ax.set_zlabel(zlabel)
    ax.set_zlim(offset + Y.min(), Y.max())

    ax.view_init(elev=elevation, azim=azimuth)

    cb = fig.colorbar(surf, shrink=0.5, aspect=5)
    cb.set_label(zlabel)

    if filename:
        fig.savefig(filename)
        plt.close(fig)
        return filename
    else:
        return ax
```

```python
plot_y(y_t, Dt, Dx, step)
```

```python
plot_surf(y_t, Dt, Dx, step, offset=0, cmap=plt.cm.coolwarm)
```

## von Neumann stability analysis: Courant condition

Assume that the solutions of the discretized equation can be written as normal modes

$$
y_{m,j} = \xi(k)^j e^{ikm\Delta x}, \quad t=j\Delta t,\ x=m\Delta x
$$

The time stepping algorith is stable if

$$
|\xi(k)| < 1
$$

Insert normal modes into the discretized equation

$$
y_{i,j+1} = 2(1 - \beta^2)y_{i,j} - y_{i, j-1} + \beta^2 (y_{i+1,j} + y_{i-1,j}), \quad
\beta := \frac{c}{\Delta x/\Delta t}
$$

and simplify (use $1-\cos x = 2\sin^2\frac{x}{2}$):

$$
\xi^2 - 2(1-2\beta^2 s^2)\xi + 1 = 0, \quad s=\sin(k\Delta x/2)
$$

The characteristic equation has roots

$$
\xi_{\pm} = 1 - 2\beta^2 s^2 \pm \sqrt{(1-2\beta^2 s^2)^2 - 1}.
$$

It has one root for

$$
\left|1-2\beta^2 s^2\right| = 1,
$$

i.e., for

$$
\beta s = 1
$$

We have two real roots for

$$
\left|1-2\beta^2 s^2\right| < 1 \\
\beta s > 1
$$

but one of the roots is always $|\xi| > 1$ and hence these solutions will diverge and not be stable.

For

$$
\left|1-2\beta^2 s^2\right| ≥ 1 \\
\beta s ≤ 1
$$

the roots will be *complex conjugates of each other*

$$
\xi_\pm = 1 - 2\beta^2s^2 \pm i\sqrt{1-(1-2\beta^2s^2)^2}
$$

and the *magnitude*

$$
|\xi_{\pm}|^2 =  (1 - 2\beta^2s^2)^2 - (1-(1-2\beta^2s^2)^2) = 1
$$

is unity: Thus the solutions will not grow and will be *stable* for

$$
\beta s ≤ 1\\
\frac{c}{\frac{\Delta x}{\Delta t}} \sin\frac{k \Delta x}{2} ≤ 1
$$

Assuming the "worst case" for the $\sin$ factor (namely, 1), the **condition for stability** is

$$
c ≤ \frac{\Delta x}{\Delta t}
$$

or

$$
\beta ≤ 1.
$$

This is also known as the **Courant condition**. When written as

$$
\Delta t ≤ \frac{\Delta x}{c}
$$

it means that the time step $\Delta t$ (for a given $\Delta x$) must be *smaller than the time that the  wave takes to travel one grid step*.

```python

```


# Introducción a la Computación Científica con Python

Versión original en inglés de J.R. Johansson (robert@riken.jp) http://dml.riken.jp/~rob/

Traducido/Adaptado por [G.F. Rubilar](http://google.com/+GuillermoRubilar).

La última versión de estos [notebooks de IPython](http://ipython.org/notebook.html) está disponible en [http://github.com/gfrubi/clases-python-cientifico](http://github.com/gfrubi/clases-python-cientifico).

La última versión del original (en inglés) de estos [notebooks de IPython](http://ipython.org/notebook.html) está disponible en [http://github.com/jrjohansson/scientific-python-lectures](http://github.com/jrjohansson/scientific-python-lectures).

Los otros notebooks de esta serie están listados en [http://jrjohansson.github.com](http://jrjohansson.github.com).

```python
%matplotlib inline
```

## Introducción

Existen dos sistemas de álgebra simbólica para Python:

* [SymPy](http://sympy.org/en/index.html) - Un módulo que puede ser usado en cualquier programa Python, o bien en una sesión de IPython, que incluye poderosas herramientas para cálculo simbólico.
* [Sage](http://www.sagemath.org/) - Sage es un sistema completo y poderoso que intenta suministrar un sistema de código abierto que compita con Mathematica and Maple. Sage no es un módulo de Python, sino un ambiente de cálculo simbólico que usa Python como su lenguaje de programación.

Sage es más poderoso que SymPy en algunos aspectos, pero ambos ofreces una lista completa de funcionalidades de cálculo simbólico. La ventaja de SymPy es que es un módulo normal de Python y se integra muy bien en un notebook de IPython.

En esta clase veremos cómo usar SymPy en un notebook de IPython.

Para comenzar a usar SymPy en un programa Python o en un notebook, importamos el módulo `sympy`:

```python
from sympy import *
```

Para que los resultados sean formateados en $\LaTeX$ podemos usar:

```python
init_printing(use_latex=True)
```

## Variables simbólicas

En SymPy podemos crear símbolos para las variables con las que deseamos trabajar. Podemos crear un nuevo símbolo usando la clase `Symbol`:

```python
x = Symbol('x')
```

```python
(pi + x)**2
```

```python
# forma alternativa de definir (varios) símbolos
a, b, c = symbols("a, b, c")
```

```python
type(a)
```

    sympy.core.symbol.Symbol

Podemos agregar algunas propiedades a los símbolos cuando son creados:

```python
x = Symbol('x', real=True)
```

```python
x.is_imaginary
```

    False

```python
x = Symbol('x', positive=True)
```

```python
x > 0
```

### Números complejos

La unidad imaginaria es denotada por `I` en Sympy.

```python
1+1*I
```

```python
I**2
```

```python
(x * I + 1)**2
```

### Números racionales

Existen tres tipos distintos de números en SymPy: `Real`, `Rational`, `Integer`:

```python
r1 = Rational(4,5)
r2 = Rational(5,4)
```

```python
r1
```

```python
r1+r2
```

```python
r1/r2
```

### Evaluación numérica

SymPy usa una librería para trabajar con números con precisión arbitraria, y tiene expresiones SymPy  predefinidas para varias constantes matemáticas, tales como: `pi`, `e` y `oo` para el infinito.

Para evaluar numéricamente una expresión podemos usar la función `evalf` (o bien `N`). Ésta usa un argumento `n` que especifíca el número de cifras significativas.

```python
pi.evalf(n=50)
```

```python
y = (x + pi)**2
```

```python
N(y, 5) # equivalente a evalf
```

Cuando evaluamos numéricamente expresiones a menudo deseamos substituir un símbolo por un valor numérico. En SymPy hacemos esto usando la función `subs`:

```python
y.subs(x, 1.5)
```

```python
N(y.subs(x, 1.5))
```

La función `subs` también puede ser usada para substituir símbolos y expresiones:

```python
y.subs(x, a+pi)
```

También podemos combinar la evaluación numérica de expresiones con arreglos NumPy:

```python
import numpy
```

```python
x_vec = numpy.arange(0, 10, 0.1)
```

```python
y_vec = numpy.array([N(((x + pi)**2).subs(x, xx)) for xx in x_vec])
```

```python
from matplotlib.pyplot import *
plot(x_vec, y_vec);
```

Sin embargo, este tipo de evaluación numérica puede ser muy lenta, y existe una forma mucho más eficiente de realizar la misma tarea: Usar la función `lambdify` para "mapear" una expresión de Sympy a una función que es mucho más eficiente para la evaluación numérica:

```python
f = lambdify([x], (x + pi)**2, 'numpy')  # el primer argumento es una lista de variables de las que la función f dependerá: en este caso sólo x -> f(x)
```

```python
type(f)
```

    function

```python
y_vec = f(x_vec)  # ahora podemos pasar directamente un arreglo Numpy. Así f(x) es evaluado más eficientemente
```

La mayor eficiencia de usar funciones "lambdificadas" en lugar de usar evalación numérica directa puede ser significativa, a menudo de varios órdenes de magnitud. Aún en este sencillo ejemplo obtenemos un aumento de velocidad importante:

```python
%%timeit

y_vec = numpy.array([N(((x + pi)**2).subs(x, xx)) for xx in x_vec])
```

    100 loops, best of 3: 19.1 ms per loop

```python
%%timeit

y_vec = f(x_vec)
```

    The slowest run took 18.93 times longer than the fastest. This could mean that an intermediate result is being cached
    1000000 loops, best of 3: 1.49 µs per loop

## Manipulaciones algebráicas

Uno de los usos principales de un sistema de cálculo simbólico es que realiza manipulaciones algebráicas de expresiones. Por ejemplo, si queremos expandir un producto, factorizar una expresión, o simplificar un resultado. En esta sección presentamos las funciones para realizar estas operaciones básicas en SymPy.

### Expand and factor

Primeros pasos en la manipulación algebráica

```python
(x+1)*(x+2)*(x+3)
```

```python
expand((x+1)*(x+2)*(x+3))
```

La función `expand` acepta varias argumentos clave con los que se puede indicar qué tipo de expansión deseamos realizar. Por ejemplo, para expandir expresiones trigonométricas, usamos el argumento clave `trig=True`:

```python
sin(a+b)
```

```python
expand(sin(a+b), trig=True)
```

Ver `help(expand)` para una descripción detallada de los distintos tipos de expansiones que la función `expand` puede realizar.

También podemos factorizar expresiones, usando la función `factor` de SymPy:

```python
factor(x**3 + 6 * x**2 + 11*x + 6)
```

### Simplify

La función `simplify` intenta simplificar una expresión usando distinta técnicas. Existen también alternativas más específicas a la función `simplify`: `trigsimp`, `powsimp`, `logcombine`, etc.

El uso básico de estas funciones en el siguiente:

```python
# simplify expande un producto
simplify((x+1)*(x+2)*(x+3))
```

```python
# simplify usa identidades trigonometricas
simplify(sin(a)**2 + cos(a)**2)
```

```python
simplify(cos(x)/sin(x))
```

## apart and together

Podemos también manipular expresiones simbólicas que involucran fracciones usando las funciones `apart` y `together`. La primera de estas funciones separa una fracción en sus correspondientes fracciones parciales; la segunda hace todo lo contrario.

```python
f1 = 1/((a+1)*(a+2))
```

```python
f1
```

```python
apart(f1)
```

```python
f2 = 1/(a+2) + 1/(a+3)
```

```python
f2
```

```python
together(f2)
```

Usualmente `Simplify` combina las fracciones, pero no factoriza:

```python
simplify(f2)
```

## Cálculo

Además de realizar manipulaciones algebráicas, SimPy puede realizar operaciones de cálculo, tales como derivar y derivar expresiones.

### Derivación

Derviar es usualmente algo simple. Usamos la función `diff`. El primer argumento es una expresión que será derivada, y el segundo argumento es el símbolo respecto al cual se realizará la derivada:

```python
y
```

```python
diff(y**2, x)
```

Para calcular derivadas de orden superior podemos usar:

```python
diff(y**2, x, x)
```

o bien

```python
diff(y**2, x, 2) # hace lo mismo
```

To calculate the derivative of a multivariate expression, we can do:

```python
x, y, z = symbols("x,y,z")
```

```python
f = sin(x*y) + cos(y*z)
```

$\frac{d^3f}{dxdy^2}$

```python
diff(f, x, 1, y, 2)
```

### Integration

Integration is done in a similar fashion:

```python
f
```

```python
integrate(f, x)
```

By providing limits for the integration variable we can evaluate definite integrals:

```python
integrate(f, (x, -1, 1))
```

and also improper integrals

```python
integrate(exp(-x**2), (x, -oo, oo))
```

Remember, `oo` is the SymPy notation for inifinity.

### Sums and products

We can evaluate sums and products using the functions: 'Sum'

```python
n = Symbol("n")
```

```python
Sum(1/n**2, (n, 1, 10))
```

```python
Sum(1/n**2, (n,1, 10)).evalf()
```

```python
Sum(1/n**2, (n, 1, oo)).evalf()
```

Products work much the same way:

```python
Product(n, (n, 1, 10)) # 10!
```

### Limits

Limits can be evaluated using the `limit` function. For example,

```python
limit(sin(x)/x, x, 0)
```

We can use 'limit' to check the result of derivation using the `diff` function:

```python
f
```

```python
diff(f, x)
```

$\displaystyle \frac{\mathrm{d}f(x,y)}{\mathrm{d}x} = \frac{f(x+h,y)-f(x,y)}{h}$

```python
h = Symbol("h")
```

```python
limit((f.subs(x, x+h) - f)/h, h, 0)
```

OK!

We can change the direction from which we approach the limiting point using the `dir` keywork argument:

```python
limit(1/x, x, 0, dir="+")
```

```python
limit(1/x, x, 0, dir="-")
```

### Series

Series expansion is also one of the most useful features of a CAS. In SymPy we can perform a series expansion of an expression using the `series` function:

```python
series(exp(x), x)
```

By default it expands the expression around $x=0$, but we can expand around any value of $x$ by explicitly include a value in the function call:

```python
series(exp(x), x, 1)
```

And we can explicitly define to which order the series expansion should be carried out:

```python
series(exp(x), x, 1, 10)
```

The series expansion includes the order of the approximation, which is very useful for keeping track of the order of validity when we do calculations with series expansions of different order:

```python
s1 = cos(x).series(x, 0, 5)
s1
```

```python
s2 = sin(x).series(x, 0, 2)
s2
```

```python
expand(s1 * s2)
```

If we want to get rid of the order information we can use the `removeO` method:

```python
expand(s1.removeO() * s2.removeO())
```

But note that this is not the correct expansion of $\cos(x)\sin(x)$ to $5$th order:

```python
(cos(x)*sin(x)).series(x, 0, 6)
```

## Linear algebra

### Matrices

Matrices are defined using the `Matrix` class:

```python
m11, m12, m21, m22 = symbols("m11, m12, m21, m22")
b1, b2 = symbols("b1, b2")
```

```python
A = Matrix([[m11, m12],[m21, m22]])
A
```

```python
b = Matrix([[b1], [b2]])
b
```

With `Matrix` class instances we can do the usual matrix algebra operations:

```python
A**2
```

```python
A * b
```

And calculate determinants and inverses, and the like:

```python
A.det()
```

```python
A.inv()
```

## Solving equations

For solving equations and systems of equations we can use the `solve` function:

```python
solve(x**2 - 1, x)
```

```python
solve(x**4 - x**2 - 1, x)
```

System of equations:

```python
solve([x + y - 1, x - y - 1], [x,y])
```

In terms of other symbolic expressions:

```python
solve([x + y - a, x - y - c], [x,y])
```

## Quantum mechanics: noncommuting variables

How about non-commuting symbols? In quantum mechanics we need to work with noncommuting operators, and SymPy has a nice support for noncommuting symbols and even a subpackage for quantum mechanics related calculations!

```python
from sympy.physics.quantum import *
```

### States

We can define symbol states, kets and bras:

```python
Ket('psi')
```

```python
Bra('psi')
```

```python
u = Ket('0')
d = Ket('1')

a, b = symbols('alpha beta', complex=True)
```

```python
phi = a * u + sqrt(1-abs(a)**2) * d; phi
```

```python
Dagger(phi)
```

```python
Dagger(phi) * d
```

Use `qapply` to distribute a mutiplication:

```python
qapply(Dagger(phi) * d)
```

```python
qapply(Dagger(phi) * u)
```

### Operators

```python
A = Operator('A')
B = Operator('B')
```

Check if they are commuting!

```python
A * B == B * A
```

```python
expand((A+B)**3)
```

```python
c = Commutator(A,B)
c
```

We can use the `doit` method to evaluate the commutator:

```python
c.doit()
```

We can mix quantum operators with C-numbers:

```python
c = Commutator(a * A, b * B)
c
```

To expand the commutator, use the `expand` method with the `commutator=True` keyword argument:

```python
c = Commutator(A+B, A*B)
c.expand(commutator=True)
```

```python
Dagger(Commutator(A, B))
```

```python
ac = AntiCommutator(A,B)
```

```python
ac.doit()
```

#### Example: Quadrature commutator

Let's look at the commutator of the electromagnetic field quadatures $x$ and $p$. We can write the quadrature operators in terms of the creation and annihilation operators as:

$\displaystyle x = (a + a^\dagger)/\sqrt{2}$

$\displaystyle p = -i(a - a^\dagger)/\sqrt{2}$

```python
X = (A + Dagger(A))/sqrt(2)
X
```

```python
P = -I * (A - Dagger(A))/sqrt(2)
P
```

Let's expand the commutator $[x,p]$

```python
Commutator(X, P).expand(commutator=True).expand(commutator=True)
```

Here we see directly that the well known commutation relation for the quadratures

$[x,p]=i$

is a directly related to

$[A, A^\dagger]=1$

(which SymPy does not know about, and does not simplify).

For more details on the quantum module in SymPy, see:

* http://docs.sympy.org/0.7.2/modules/physics/quantum/index.html
* http://nbviewer.ipython.org/urls/raw.github.com/ipython/ipython/master/docs/examples/notebooks/sympy_quantum_computing.ipynb

## Further reading

* http://sympy.org/en/index.html - The SymPy projects web page.
* https://github.com/sympy/sympy - The source code of SymPy.
* http://live.sympy.org - Online version of SymPy for testing and demonstrations.


# Solvers

```python
from sympy import *
from sympy.solvers.solveset import solveset
init_printing()
```

For each exercise, fill in the function according to its docstring.

```python
a, b, c, d, x, y, z, t = symbols('a b c d x y z t')
f, g, h = symbols('f g h', cls=Function)
```

## Algebraic Equations

Write a function that computes the [quadratic equation](http://en.wikipedia.org/wiki/Quadratic_equation).

```python
def quadratic():
    return ???
quadratic()
```

Write a function that computes the general solution to the cubic $x^3 + ax^2 + bx + c$.

```python
def cubic():
    return ???
cubic()
```

## Differential Equations

A population that grows without bound is modeled by the differential equation

$$f'(t)=af(t)$$

Solve this differential equation using SymPy.

```python

```

If the population growth is bounded, it is modeled by

$$f'(t) = f(t)(1 - f(t))$$

Solve this differential equation using SymPy.

```python

```


# Sympy

```python
import sympy as sp

# See: http://docs.sympy.org/latest/tutorial/printing.html
sp.init_printing()
```

See:
* http://docs.sympy.org/latest/tutorial/
* http://docs.sympy.org/0.7.0/tutorial.html

## Live shell

http://live.sympy.org/

## Make symbols

Make one symbol:

```python
x = sp.symbols("x")
```

Make several symbols at once:

```python
x, y, z = sp.symbols("x y z")
```

## Main symbols

```python
sp.pi
```

```python
sp.I
```

```python
sp.oo
```

## Main functions

```python
sp.sqrt(x)
```

```python
x**2
```

The natural logarithm function `ln`:

```python
sp.log(x)
```

```python
sp.exp(x)
```

### Trigonometric functions

```python
sp.cos(x)
```

```python
sp.sin(x)
```

```python
sp.tan(x)
```

```python
sp.cosh(x)
```

```python
sp.sinh(x)
```

```python
sp.tanh(x)
```

```python
sp.acos(x)
```

```python
sp.asin(x)
```

```python
sp.atan(x)
```

```python
sp.acosh(x)
```

```python
sp.asinh(x)
```

```python
sp.atanh(x)
```


```python
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"

import gzip
import numpy as np
import tensorflow as tf
from typing import Tuple
```

You've already seen how to train a neural network using Keras in [module 1](../intro-machine-learning-keras) — in this notebook, we'll re-implement the training loop in TensorFlow. This will help you understand what goes on under the hood a bit better, will give you the opportunity to customize the training loop if you want, and will enable you to debug it.

We'll start by including code that gives us the datasets and model that we'll use in the remainder of this notebook. We will use the same FashionMNIST dataset and data loading code as in [module 1](../intro-machine-learning-keras), so feel free to re-visit that module if something is not clear, or take a look [at the source code](https://github.com/MicrosoftDocs/tensorflow-learning-path/blob/main/intro-tf/tintro.py).

```python
!wget -Nq https://raw.githubusercontent.com/MicrosoftDocs/tensorflow-learning-path/main/intro-tf/tintro.py
from tintro import *
```

As we mentioned in module 1, the goal of training the neural network is to find parameters $W$ and $b$ that minimize the loss function, which measures the difference between the actual and predicted labels. We also mentioned that we can think of the neural network as the function $\ell$ below, and that we use an optimization algorithm to find the parameters $W$ and $b$ that minimize this function.

$$
\mathrm{loss} = \ell(X, y, W, b)
$$

Let's now dig deeper into what this optimization algorithm might look like. There are many types of optimization algorithms, but in this tutorial we'll cover only the simplest one: the gradient descent algorithm. To implement gradient descent, we iteratively improve our estimates of $W$ and $b$ according to the update formulas below, until the gradients are smaller than a pre-defined threshold $\epsilon$ (or for a pre-defined number of times):

$$
\begin{align}
  W &:= W - \alpha \frac{\partial \ell}{\partial W} \\
  b &:= b - \alpha \frac{\partial \ell}{\partial b}
\end{align}
$$

The parameter $\alpha$ is typically referred to as the "learning rate," and will be defined later in the code.

When doing training, we pass a mini-batch of data as input, perform a sequence of calculations to obtain the loss, then propagate back through the network to calculate the derivatives used in the gradient descent formulas above. Once we have the derivatives, we can update the values of the network's parameters $W$ and $b$ according to the formulas. This sequence of steps is the backpropagation algorithm. By performing these calculations several times, our parameters get updated repeatedly, getting more and more accurate each time.

In Keras, when we called the function [`fit`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit), the backpropagation algorithm was executed several times. Here, we'll start by understanding the code that reflects a single pass of the backpropagation algorithm:

- A forward pass through the model to compute the predicted value, `y_prime = model(X, training=True)`
- A calculation of the loss using a loss function, `loss = loss_fn(y, y_prime)`
- A backward pass from the loss function through the model to calculate derivatives, `grads = tape.gradient(loss, model.trainable_variables)`
- A gradient descent step to update $W$ and $b$ using the derivatives calculated in the backward pass, `optimizer.apply_gradients(zip(grads, model.trainable_variables))`

Here's the complete code:

```python
def fit_one_batch(X, y, model, loss_fn, optimizer) -> Tuple[tf.Tensor, tf.Tensor]:
  with tf.GradientTape() as tape:
    y_prime = model(X, training=True)
    loss = loss_fn(y, y_prime)

  grads = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(grads, model.trainable_variables))

  return (y_prime, loss)
```

Notice that the code above ensures that the forward calculations are within the `GradientTape`'s scope, just as we saw in the previous notebook. This makes it possible for us to ask the tape for the gradients.

The code above works for a single mini-batch, which is typically much smaller than the full set of data (in this sample we use a mini-batch of size 64, out of 60,000 training data items). But we want to execute the backpropagation algorithm for the full set of data. We can do so by iterating through the `Dataset` we created earlier, which, as we saw in module 1, returns a mini-batch per iteration. There are two critical lines in the code below: the `for` loop and the call to the `fit_one_batch` function. The rest of the code just prints the accuracy and loss as the model is being trained.

```python
def fit(dataset: tf.data.Dataset, model: tf.keras.Model, loss_fn: tf.keras.losses.Loss,
optimizer: tf.optimizers.Optimizer) -> None:
  batch_count = len(dataset)
  loss_sum = 0
  correct_item_count = 0
  current_item_count = 0
  print_every = 100

  for batch_index, (X, y) in enumerate(dataset):
    (y_prime, loss) = fit_one_batch(X, y, model, loss_fn, optimizer)

    y = tf.cast(y, tf.int64)
    correct_item_count += (tf.math.argmax(y_prime, axis=1) == y).numpy().sum()

    batch_loss = loss.numpy()
    loss_sum += batch_loss
    current_item_count += len(X)

    if ((batch_index + 1) % print_every == 0) or ((batch_index + 1) == batch_count):
      batch_accuracy = correct_item_count / current_item_count * 100
      print(f'[Batch {batch_index + 1:>3d} - {current_item_count:>5d} items] accuracy: {batch_accuracy:>0.1f}%, loss: {batch_loss:>7f}')
```

A complete iteration over all mini-batches in the dataset is called an "epoch." In this sample, we restrict the code to just five epochs for quick execution, but in a real project you would want to set it to a much higher number (to achieve better predictions). The code below also shows the creation of the loss function and optimizer, which we discussed in module 1.

```python
learning_rate = 0.1
batch_size = 64
epochs = 5

(train_dataset, test_dataset) = get_data(batch_size)

model = NeuralNetwork()

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
optimizer = tf.optimizers.SGD(learning_rate)

print('\nFitting:')
for epoch in range(epochs):
  print(f'\nEpoch {epoch + 1}\n-------------------------------')
  fit(train_dataset, model, loss_fn, optimizer)
```

    Fitting:

    Epoch 1
    -------------------------------
    [Batch 100 -  6400 items] accuracy: 62.9%, loss: 0.705272
    [Batch 200 - 12800 items] accuracy: 68.9%, loss: 0.743024
    [Batch 300 - 19200 items] accuracy: 71.5%, loss: 0.531616
    [Batch 400 - 25600 items] accuracy: 73.6%, loss: 0.617099
    [Batch 500 - 32000 items] accuracy: 74.7%, loss: 0.561903
    [Batch 600 - 38368 items] accuracy: 75.7%, loss: 0.925934
    [Batch 700 - 44768 items] accuracy: 76.6%, loss: 0.632339
    [Batch 800 - 51168 items] accuracy: 77.3%, loss: 0.370619
    [Batch 900 - 57568 items] accuracy: 77.9%, loss: 0.568127
    [Batch 938 - 60000 items] accuracy: 78.1%, loss: 0.627154

    Epoch 2
    -------------------------------
    [Batch 100 -  6400 items] accuracy: 83.6%, loss: 0.487360
    [Batch 200 - 12800 items] accuracy: 83.4%, loss: 0.428791
    [Batch 300 - 19200 items] accuracy: 83.3%, loss: 0.486824
    [Batch 400 - 25600 items] accuracy: 83.4%, loss: 0.391912
    [Batch 500 - 32000 items] accuracy: 83.5%, loss: 0.424991
    [Batch 600 - 38368 items] accuracy: 83.6%, loss: 0.515074
    [Batch 700 - 44768 items] accuracy: 83.6%, loss: 0.485272
    [Batch 800 - 51168 items] accuracy: 83.6%, loss: 0.345185
    [Batch 900 - 57568 items] accuracy: 83.7%, loss: 0.466114
    [Batch 938 - 60000 items] accuracy: 83.8%, loss: 0.271606

    Epoch 3
    -------------------------------
    [Batch 100 -  6400 items] accuracy: 85.3%, loss: 0.462108
    [Batch 200 - 12800 items] accuracy: 85.0%, loss: 0.370773
    [Batch 300 - 19200 items] accuracy: 85.2%, loss: 0.467257
    [Batch 400 - 25600 items] accuracy: 85.1%, loss: 0.564032
    [Batch 500 - 32000 items] accuracy: 85.1%, loss: 0.419473
    [Batch 600 - 38400 items] accuracy: 85.1%, loss: 0.444543
    [Batch 700 - 44768 items] accuracy: 85.1%, loss: 0.477494
    [Batch 800 - 51168 items] accuracy: 85.2%, loss: 0.428996
    [Batch 900 - 57568 items] accuracy: 85.2%, loss: 0.377139
    [Batch 938 - 60000 items] accuracy: 85.2%, loss: 0.331598

    Epoch 4
    -------------------------------
    [Batch 100 -  6400 items] accuracy: 85.8%, loss: 0.400852
    [Batch 200 - 12800 items] accuracy: 85.5%, loss: 0.399611
    [Batch 300 - 19200 items] accuracy: 85.6%, loss: 0.363247
    [Batch 400 - 25600 items] accuracy: 85.7%, loss: 0.337134
    [Batch 500 - 32000 items] accuracy: 85.8%, loss: 0.366630
    [Batch 600 - 38368 items] accuracy: 85.7%, loss: 0.488763
    [Batch 700 - 44768 items] accuracy: 85.7%, loss: 0.640464
    [Batch 800 - 51168 items] accuracy: 85.8%, loss: 0.422981
    [Batch 900 - 57568 items] accuracy: 85.9%, loss: 0.242606
    [Batch 938 - 60000 items] accuracy: 85.9%, loss: 0.532112

    Epoch 5
    -------------------------------
    [Batch 100 -  6400 items] accuracy: 86.5%, loss: 0.347542
    [Batch 200 - 12800 items] accuracy: 86.4%, loss: 0.427709
    [Batch 300 - 19200 items] accuracy: 86.5%, loss: 0.335535
    [Batch 400 - 25600 items] accuracy: 86.5%, loss: 0.185783
    [Batch 500 - 32000 items] accuracy: 86.6%, loss: 0.307705
    [Batch 600 - 38400 items] accuracy: 86.6%, loss: 0.274786
    [Batch 700 - 44800 items] accuracy: 86.6%, loss: 0.315632
    [Batch 800 - 51168 items] accuracy: 86.6%, loss: 0.267578
    [Batch 900 - 57568 items] accuracy: 86.6%, loss: 0.252907
    [Batch 938 - 60000 items] accuracy: 86.6%, loss: 0.485074


# The generalized model

The equation of the generalized model for 1 species can be written as follows:

$
\dot{X} = X\left[\left(r+b X\right)-\left(a + c b X\right)X\right],
$

```python
from sympy.abc import x, y
from sympy.solvers import solve
from sympy import *
```

```python
# We define the symbols for the equations and functiones
a,b,r,c = symbols('a b r c')
f = Function('f')
```

```python
# We define the function to solve

eq = r+(b-a)*x - c*b*x**2
f= x*eq

```

We calculate the fixed points

```python
s1,s2,s3=solve(f,x)
print('fixed points =',(s1,s2,s3))
```

    fixed points = (0, (-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c), (-a + b + sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c))

We calculate the Jacobian of the function

```python
dfdx = f.diff(x)
print("f'(x) =", simplify(dfdx))
```

    f'(x) = -2*a*x - 3*b*c*x**2 + 2*b*x + r

We evaluate the jacobian in the fixed points

```python
L1=dfdx.subs(x,s1)
L1
```

    r

```python
L2=dfdx.subs(x,s2)
L2
```

    r + (-a + b)*(-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c) - (-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))**2/(4*b*c) + (-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r)/(2*b*c)

```python
L3=simplify(dfdx.subs(x,s3))
L3
```

    -(a**2 - 2*a*b - a*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r) + b**2 + 4*b*c*r + b*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c)

```python
# We can evaluate the eigenvalues with the numerical values of $r,c,b,a$
L3.subs(r,0.1).subs(b,-0.01).subs(a,0.001).subs(c,0.1)
```

    -0.1395 - 0.0918681119866954*I

```python
# Another way to calculate the Jacobian Matrix and the eigenvalues.
equilibria = solve(f,x)
eqMat = Matrix([f])
Mat = Matrix([x])
jacMat =eqMat.jacobian(Mat)
print('Jacobian %s' % jacMat)
print('---------------------')

# iterate through list of equilibria
for item in equilibria:
    eqmat = jacMat.subs([ (x, item)])
    print('The eigenvalues for the fixed point (%s) is %s:'
          %(item, eqmat.eigenvals()))

    print('-------------------------------------------')

```

    Jacobian Matrix([[-b*c*x**2 + r + x*(-a + b) + x*(-a - 2*b*c*x + b)]])
    ---------------------
    The eigenvalues for the fixed point (0) is {r: 1}:
    -------------------------------------------
    The eigenvalues for the fixed point ((-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c)) is {(-a**2 + 2*a*b - a*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r) - b**2 - 4*b*c*r + b*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c): 1}:
    -------------------------------------------
    The eigenvalues for the fixed point ((-a + b + sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c)) is {(-a**2 + 2*a*b + a*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r) - b**2 - 4*b*c*r - b*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c): 1}:
    -------------------------------------------

```python
eqmat = jacMat.subs([ (x, s2)])
eqmat.refine(Q.positive(b)).refine(Q.negative(r))
```

    Matrix([[r + (-a + b)*(-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))/(2*b*c) - (-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))**2/(4*b*c) + (-a + b - sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r))*sqrt(a**2 - 2*a*b + b**2 + 4*b*c*r)/(2*b*c)]])

```python

```


# Stochastic Differential Equation (SDEs)

Continous-state dynamical systems lead determinitical Ordinary Differential Equations systems (ODEs). Stochastic versions of this model are Stochastic Differential Equations (SDEs).

## Fitzhugh Nagumo Model

\begin{equation}
\begin{split}
\frac{dv}{dt} =&  v-v^3-w+I_{ext}\\
\frac{dw}{dt} =& \frac{1}{\tau}(v-a-bw)
\end{split}
\end{equation}

```python
import numpy as np
import pandas as pd

def fitzhugh_nagumo(x, t, a, b, tau, I):
    """
        Fitzhugh-Nagumo model.
    """
    v, w = x

    dvdt = v-v**3-w+I
    dwdt = (v-a-b*w)/tau

    return np.array([dvdt, dwdt])
```

```python
from functools import partial
from scipy.integrate import odeint

def integration_SDE(model, noise_flow, y0, t) :
    ''' Euler integration.
        y(t) = f(Y(t),t)dt + g(Yt,t)dBt
        y(0) = y0
    '''
    y = np.zeros((len(t),len(y0)))
    y[0] = y0

    for n, dt in enumerate(np.diff(t),1):
        y[n] = y[n-1] + model(y[n-1],dt) * dt + model(y[n-1],dt) * np.random.normal(0,np.sqrt(dt))

    return y

n_runs = 10
t_span = np.linspace(0, 1000, num=10000)
brownian_noise = lambda y,t: 0.01

initial_conditions = [(-0.5,-0.1), [0, -0.16016209760708508]]

I_ext = [0, 0.19, 0.22, 0.5]

import matplotlib.pyplot as plt
fig, ax = plt.subplots(len(I_ext), 1, figsize=(15, 10*len(I_ext)))

for idx, current_ext in enumerate(I_ext):
    # Evaluate fitzhugh_nagumo model with specified pameters a, b, tau, I in param
    param = {'a': -0.3, 'b':1.4, 'tau':20, 'I':current_ext}

    model = partial(fitzhugh_nagumo, **param)
    ic = initial_conditions[1]

    sde_solutions = np.zeros((10000,2,n_runs))
    for i in range(n_runs):
        sde_solutions[:,:,i] = integration_SDE(model,
                                      brownian_noise,
                                      y0=ic,
                                      t=t_span)

    ode_solution = odeint(model, y0=ic, t=t_span)

    v_sde, w_sde = (sde_solutions[:,0,:], sde_solutions[:,1,:])

    # Drop nans in case stochastic run results in ill solution
    v_sde = pd.DataFrame(v_sde).dropna(axis=1).to_numpy()
    v_ode, w_ode = (ode_solution[:,0], ode_solution[:,1])

    ax[idx].plot(t_span, v_ode, label='V - ODE', color='k')
    ax[idx].plot(t_span, np.median(v_sde, 1), label=r'Median V - SDE', color='r', linestyle='-.')
    ax[idx].plot(t_span, v_sde, color='r', linestyle='-.', alpha=0.2)
    ax[idx].set_xlabel('Time (ms)')
    ax[idx].set_ylabel('Membrane Potential (mV)')
    ax[idx].set_title(r'External Stimulus $I_e=${}'.format(param['I']))
    ax[idx].set_ylim([-2,2])
    ax[idx].legend()

```

```python

```


For a particle in a harmonic potential with a damping rate due to the enviroment $\Gamma$, a natural frequency $\omega_0$ and a corresponding trap stiffness $k_0 = m\omega_0^2$ with a mass $m$ and an external force acting on the particle $F_{th}$ the equation of motion for 1 degree of freedom is:

$$ m\ddot{x}(t) + m\Gamma \dot{x}(t) + k_0 x(t) = F_{th}(t)$$

replacing $k_0$ with $m\omega^2$ we get

$$ \ddot{x}(t) + \Gamma \dot{x}(t) + \omega_0^2 x(t) = \dfrac{F_{th}(t)}{m}$$

If we then use the Fourier relationship

$$ \dfrac{d^n f(t)}{dt^n} = (i\omega)^n \tilde{f}(\omega) $$

Then the Fourier transform of the equation of motion is:

$$ (i\omega)^2 \tilde{x}(\omega) + \Gamma_0(i\omega)\tilde{x}(\omega) + \omega_0^2 \tilde{x}(\omega) = \dfrac{\tilde{F_{th}}(\omega)}{m} $$

which is equal to:

$$ -\omega^2 \tilde{x}(\omega) + \Gamma_0(i\omega)\tilde{x}(\omega) + \omega_0^2 \tilde{x}(\omega) = \dfrac{\tilde{F_{th}}(\omega)}{m} $$

Rearranging this for $\tilde{x}(\omega)$ we get:

$$ \tilde{x}(\omega) = \dfrac{1}{m} \dfrac{\tilde{F_{th}}(\omega)}{\omega_0^2-\omega^2 + i(\Gamma_0\omega)} $$

# Derivation of the form of the PSD

The formula for the power spectral density (the Fourier transform of the autocorrelation function) is:

$$ S_{xx}(\omega) = \int_{-\infty}^{+\infty} \langle x(t) x(t+\tau)^* \rangle e^{-i\omega \tau} d\tau $$

From https://journals.aps.org/rmp/pdf/10.1103/RevModPhys.86.1391 using the Wiener-Khinchin theorem.

And the formula for the reverse Fourier transform is

$$ x(t) = \dfrac{1}{\sqrt{2\pi}} \int_{-\infty}^{+\infty} \tilde{x}(\omega)e^{+i\omega t} d\omega $$

$$ S_{xx}(\omega) = \int_{-\infty}^{\infty} d\tau e^{-i\omega \tau} \langle \dfrac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} d\omega_1 \tilde{x}(\omega_1)e^{+i\omega_1 t} \dfrac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} d\omega_2 \tilde{x}(\omega_2)^*e^{+i\omega_2 (t + \tau)} \rangle $$

$$ S_{xx}(\omega) = \dfrac{1}{2\pi} \int_{-\infty}^{\infty} d\tau e^{-i\omega \tau} \langle \int_{-\infty}^{\infty} d\omega_1 \dfrac{1}{m} \dfrac{\tilde{F_{th}}(\omega_1)}{\omega_0^2-\omega_1^2 + i(\Gamma_0\omega_1)} e^{+i\omega_1 t} \int_{-\infty}^{\infty} d\omega_2 \dfrac{1}{m} \dfrac{\tilde{F_{th}}(\omega_2)}{\omega_0^2-\omega_2^2 - i(\Gamma_0\omega_2)} e^{+i\omega_2 (t+ \tau)} \rangle $$

$$ S_{xx}(\omega) = \dfrac{1}{2\pi m^2} \int_{-\infty}^{\infty} d\tau \int_{-\infty}^{\infty} d\omega_1 \int_{-\infty}^{\infty} d\omega_2  e^{-i\omega \tau} e^{+i\omega_2 t} e^{+i\omega_2 \tau)} e^{+i\omega_1 t} \dfrac{\langle \tilde{F_{th}}(\omega_1) \tilde{F_{th}}(\omega_2) \rangle}{(\omega_0^2-\omega_1^2 + i(\Gamma_0\omega_1))\cdot(\omega_0^2-\omega_2^2 - i(\Gamma_0\omega_2))}  $$

Now using $\dfrac{1}{2\pi} \int_{-\infty}^{\infty} e^{-i\omega\tau} e^{i\omega_2\tau} d\tau = \dfrac{1}{2\pi} \int_{-\infty}^{\infty} e^{i(\omega_2-\omega)\tau} d\tau = \delta(\omega_2 - \omega)$ from the definition of the delta function

$$ S_{xx}(\omega) = \dfrac{1}{m^2} \int_{-\infty}^{\infty} d\omega_1 \int_{-\infty}^{\infty} d\omega_2 e^{+i\omega_2 t} e^{+i\omega_1 t} \dfrac{\langle \tilde{F_{th}}(\omega_1) \tilde{F_{th}}(\omega_2) \rangle}{(\omega_0^2-\omega_1^2 + i(\Gamma_0\omega_1))\cdot(\omega_0^2-\omega_2^2 - i(\Gamma_0\omega_2))} \delta(\omega_2 - \omega) $$

$$ S_{xx}(\omega) = \dfrac{1}{m^2} \int_{-\infty}^{\infty} d\omega_1 e^{+i\omega t} e^{+i\omega_1 t} \dfrac{\langle \tilde{F_{th}}(\omega_1) \tilde{F_{th}}(\omega) \rangle}{(\omega_0^2-\omega_1^2 + i(\Gamma_0\omega_1))\cdot(\omega_0^2-\omega^2 - i(\Gamma_0\omega))}  $$

If we now use the result, which is derived below [here](#derivation), that:

$$ \langle \tilde{F_{th}}(\omega_1) \tilde{F_{th}}(\omega) \rangle = \sqrt{2\pi} \delta(\omega_1 + \omega) \tilde{f}(\dfrac{\omega_1 - \omega}{2}) $$

Where $\tilde{f}$ is the Fourier transform of the 2 point correlation function of the thermal noise $F_{th}$.

We get:

$$ S_{xx}(\omega) = \dfrac{1}{m^2} \int_{-\infty}^{\infty} d\omega_1 e^{+i(\omega + \omega_1) t} \dfrac{\delta(\omega_1 + \omega) \sqrt{2\pi} \tilde{f}(\frac{\omega_1 - \omega}{2})}{(\omega_0^2-\omega_1^2 + i(\Gamma_0\omega_1))\cdot(\omega_0^2-\omega^2 - i(\Gamma_0\omega))}  $$

Using this result we get (using $\omega_1 = - \omega$ for each term $\omega_1$ inside the integral - due to the delta function):

$$ S_{xx}(\omega) = \dfrac{1}{m^2} \dfrac{\sqrt{2\pi} \tilde{f}(-\omega)}{(\omega_0^2-\omega^2 - i(\Gamma_0\omega))\cdot(\omega_0^2-\omega^2 + i(\Gamma_0\omega))}  $$

which when expanded comes to:

$$ S_{xx}(\omega) = \dfrac{\sqrt{2\pi}}{m^2} \dfrac{\tilde{f}(-\omega)}{\omega_0^4 - 2\omega_0^2 \omega^2 + \omega^4 + \Gamma_0^2\omega^2} = \dfrac{\sqrt{2\pi}}{m^2} \dfrac{1}{m^2} \dfrac{\tilde{f}(-\omega)}{(\omega_0^2 - \omega^2)^2 + (\Gamma_0\omega)^2}  $$

$\tilde{f}$, as mentioned earilier, is the Fourier transform of the 2 point correlation function of the thermal noise $F_{th}$. Taking this to be constant is equivalent to assuming that the noise is Markovian i.e. that it is uncorrelated from time $t$ to time $t + \tau$.

This is the case because the reverse Fourier transform of constant $\tilde{f}$ results in $f(s) = \dfrac{1}{\sqrt{2\pi}}\int{\tilde{f}e^{-i\omega s} d\omega} = \tilde{f}\delta(s)$ which is the two-point correlation function in time for Markovian noise. That is to say, it is uncorrelated in time.

therefore we get the result that $$ S_{xx}(\omega) = \dfrac{\sqrt{2\pi}}{m^2}  \dfrac{\tilde{f}}{(\omega_0^2-\omega^2)^2 + (\Gamma_0\omega)^2} $$

## Derivation of

$$ \langle \tilde{F_{th}}(\omega_1) \tilde{F_{th}}(\omega) \rangle = \sqrt{2\pi} \delta(\omega_1 + \omega) \tilde{f}(\dfrac{\omega_1 - \omega}{2}) $$

$$ \langle \tilde{h}(\omega_1) \tilde{h}(\omega_2) \rangle = \dfrac{1}{2\pi} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} dt_1 dt_2 \langle h(t_1)h(t_2) \rangle e^{-i\omega_1 t_1} e^{-i\omega_2 t_2} $$

Using the Fourier transform $\tilde{h}(\omega) = \dfrac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} x(t) e^{-i\omega t} dt$

Defining
$$ \langle h(t_1)h(t_2) \rangle = f(t_1 - t_2) $$
which is the two-point correlation function of the thermal noise $F_{th}$.

we get

$$  = \dfrac{1}{2\pi} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} dt_1 dt_2 f(t_1-t_2) e^{-i\omega_1 t_1} e^{-i\omega_2 t_2} $$

Now defining $t_+ = t_1 + t_2$ and $t_- = t_1 - t_2$ and replacing $t_1$ with $\dfrac{1}{2}(t_+ + t_-)$ and $t_2$ with $\dfrac{1}{2}(t_+ - t_-)$ in the integral we get:

$$ = \dfrac{1}{4\pi} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} dt_+ dt_- f(t_-) e^{-\frac{1}{2}i\omega_1 (t_+ + t_-)} e^{-\frac{1}{2}i\omega_2 (t_+ - t_-)} $$

$$ = \dfrac{1}{4\pi} \int_{-\infty}^{\infty} dt_+ e^{-\frac{1}{2}i t_+ (\omega_1 + \omega_2)} \int_{-\infty}^{\infty} dt_- e^{\frac{1}{2}i t_- (\omega_2 - \omega_1)} f(t_-)  $$

$$ = \dfrac{1}{2\pi} \int_{-\infty}^{\infty} dt_+ e^{i t_+ (\frac{-\omega_1 - \omega_2}{2})} \dfrac{1}{2} \int_{-\infty}^{\infty} dt_- e^{-i t_- (\frac{\omega_1 - \omega_2}{2})} f(t_-)  $$

By taking the first term to be
$$\dfrac{1}{2\pi} \int_{-\infty}^{\infty} dt_+ e^{i t_+ (\frac{-\omega_1 - \omega_2}{2})} = \delta(\dfrac{-\omega_1 - \omega_2}{2}) $$
from the definition of the delta function

And the second term to be
$$\int_{-\infty}^{\infty} dt_- e^{-i t_- (\frac{\omega_1 - \omega_2}{2})} f(t_-) = \sqrt{2\pi} \tilde{f}(\dfrac{\omega_1-\omega_2}{2}) $$
from the definition of the Fourier transform

$$ \delta(\dfrac{-\omega_1 - \omega_2}{2}) \sqrt{\dfrac{\pi}{2}} \tilde{f}(\dfrac{\omega_1-\omega_2}{2}) $$

$\delta(\dfrac{-\omega_1 - \omega_2}{2})$ is equivalent to $2\delta(\omega_1 + \omega_2)$ which leaves us with:

$$ \sqrt{2\pi} \delta(\omega_1 + \omega_2) \tilde{f}(\dfrac{\omega_1 - \omega_2}{2}) $$

# Derivation of the constant $\tilde{f}$

The form of the constant $\tilde{f}$ can be derived readily by realising that the integral of a power spectral density yields the positional variance of the oscillator, namely $\langle x^2 \rangle$. As used in  https://link.aps.org/doi/10.1103/RevModPhys.86.1391

Therefore

$$ \langle x^2 \rangle = \int_0^{\infty} S_{xx}(\omega) \dfrac{d\omega}{2\pi} $$

$$ = \tilde{f} \dfrac{\sqrt{2\pi}}{m^2} \dfrac{1}{2\pi} \int_{0}^{\infty} \dfrac{1}{(\omega_0^2-\omega^2)^2 + (\Gamma_0\omega)^2} d\omega $$

$$ = \dfrac{1}{\sqrt{2\pi}} \dfrac{\tilde{f}}{m^2}  \int_{0}^{\infty} \dfrac{1}{(\omega_0^2-\omega^2)^2 + (\Gamma_0\omega)^2} d\omega $$

This integral can be calculated like so:

-----------------------------------------------------------

**Preliminaries**

Using $a^2+b^2=(a+ib)(a-ib)$, we get

\begin{align}
\left(\omega_0^2-\omega^2\right)^2+(\Gamma_0\omega)^2
&=\left(\omega^2+i\Gamma_0\omega-\omega_0^2\right)\left(\omega^2-i\Gamma_0\omega-\omega_0^2\right)
\end{align}

The roots for the two quadratics are
$$
\omega=\frac{\pm i\Gamma_0\pm\sqrt{4\omega_0^2-\Gamma_0^2}}2
$$
Thus, re-pairing the roots gives

\begin{align}
&\left(\omega_0^2-\omega^2\right)^2+(\Gamma_0\omega)^2\\
&=\left(\omega^2+\omega\sqrt{4\omega_0^2-\Gamma_0^2}+\omega_0^2\right)\left(\omega^2-\omega\sqrt{4\omega_0^2-\Gamma_0^2}+\omega_0^2\right)\\
&=\left[\left(\omega+\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2\right]\left[\left(\omega-\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2\right]
\end{align}

Then [Partial Fractions](https://en.wikipedia.org/wiki/Partial_fraction_decomposition) gives

\begin{align}
&\frac1{\left(\omega_0^2-\omega^2\right)^2+(\Gamma_0\omega)^2}\\[12pt]
&=\frac1{2\omega_0^2}\left[\frac{1+\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\left(\omega+\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2}
+\frac{1-\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\left(\omega-\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2}\right]
\end{align}

***
**Integration**

Due to the evenness of the integrand,

\begin{align}
&\int_0^\infty\frac1{\left(\omega_0^2-\omega^2\right)^2+(\Gamma_0\omega)^2}\,\mathrm{d}\omega\\[6pt]
&=\frac12\int_{-\infty}^\infty\frac1{\left(\omega_0^2-\omega^2\right)^2+(\Gamma_0\omega)^2}\,\mathrm{d}\omega\\
&=\frac1{4\omega_0^2}\int_{-\infty}^\infty\left[{\small\frac{1+\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\left(\omega+\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2}
+\frac{1-\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\left(\omega-\sqrt{\omega_0^2-\tfrac14\Gamma_0^2}\right)^2+\tfrac14\Gamma_0^2}}\right]\,\mathrm{d}\omega\\
&=\frac1{4\omega_0^2}\int_{-\infty}^\infty\left[{\small\frac{\frac12+\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\omega^2+\tfrac14\Gamma_0^2}
+\frac{\frac12-\frac\omega{\sqrt{4\omega_0^2-\Gamma_0^2}}}{\omega^2+\tfrac14\Gamma_0^2}}\right]\,\mathrm{d}\omega\\[6pt]
&=\frac1{4\omega_0^2}\int_{-\infty}^\infty\frac{\mathrm{d}\omega}{\omega^2+\tfrac14\Gamma_0^2}\\[12pt]
&=\frac\pi{2\omega_0^2\Gamma_0}
\end{align}

---------------------------------------------------------------

This results in the following form for

$$ \langle x^2 \rangle = \sqrt{\dfrac{\pi}{2}} \dfrac{\tilde{f}}{2m^2\omega_0^2\Gamma_0} $$

Assuming that the system is in thermal equilibrium and the equipartition theorem applies to this system.

From the equipartition theorem $$ \dfrac{1}{2}k_BT_{c.m.} = \dfrac{1}{2}m\omega_0^2 \langle x^2 \rangle $$ we get:

$$ \dfrac{1}{2}k_BT_{c.m.} = \dfrac{1}{2}m\omega_0^2 \sqrt{\dfrac{\pi}{2}} \dfrac{\tilde{f}}{2m^2\omega_0^2\Gamma_0} = \sqrt{\dfrac{\pi}{2}} \dfrac{\tilde{f}}{2m\Gamma_0} $$

$$ \tilde{f} = \sqrt{\dfrac{2}{\pi}} 2k_B T_{c.m.} m \Gamma_0 $$

# Final form of the PSD

Plugging this into the previous result

$$ S_{xx}(\omega) = \dfrac{\sqrt{2\pi}}{m^2}  \dfrac{\tilde{f}}{(\omega_0^2-\omega^2)^2 + (\Gamma_0\omega)^2} $$

we get:

$$ S_{xx}(\omega) = \dfrac{4 k_B T_{c.m.}}{m} \dfrac{\Gamma_0}{(\omega_0^2-\omega^2)^2 + (\Gamma_0\omega)^2} $$

```python

```


# What if the Earth collapses to a black hole, instantaneously?

## Premise
You're standing on the Earth, when it suddenly collapses to a black hole.
Do you fall into the resulting black hole, the size of a peanut?

## Assumptions
You don't die immediately from this mysterious event, so you can experience your fate.
This means:

- This is a non-violent event: no gamma ray burst or accretion disk spewing sterilizing x-rays.
- Ergo, the atmosphere and any resulting debris are inside the black hole. No accretion disk.
- You're wearing a space suit. (Since there's no atmosphere.)
- The resulting black hole has the same mass and angular momentum as the Earth had had.

Also, assume that you're on the equator at the time, since values like radius and rotational speed are easy to find.
(And this is the particular scenario descibed in a podcast.)

```python
import sympy
sympy.init_printing()

upsilon, mu, r, a, T, R_g, c = sympy.symbols("upsilon mu r a T R_g c")

speed = sympy.sqrt(mu * (2/r - 1/a))
speedEq = sympy.Eq(upsilon, speed)

# Solve for "a".
axis = sympy.solve(speedEq, a)[0]
axisEq = sympy.Eq(a, axis)

period = sympy.sqrt(a**3/mu)*2*sympy.pi
periodEq = sympy.Eq(T, period)

schwarzschild = 2 * mu / c**2
schwarzschildEq = sympy.Eq(R_g, schwarzschild)
```

"r" is distance between the astronaut and the black hole.
Equation for speed, given semimajor axis ("a"):

```python
speedEq
```

Apogee initial conditions (r,v):
where r = Earth radius at equator, and v = Earth rotation speed at equator.
Now, solve for the length of semimajor axis:

```python
axisEq
```

Equation for period of an orbit, given semimajor axis:

```python
periodEq
```

Schwarzschild radius:

```python
schwarzschildEq
```

```python
# Earth's standard gravitational parameter (mu = G*M)
muVal = 3.986e14        # units: m^3 / s^2

# Equatorial radius
rVal = 6378137.         # units: m

# Equatorial rotation speed (apogee speed)
upsilonVal = 465.1      # units: m/s

# Speed of light
cVal = 299792458        # units: m/s

# Calculate some key values.
axisFun = sympy.lambdify((mu, r, upsilon), axis)
axisVal = axisFun(muVal, rVal, upsilonVal)
print(f"semimajor axis(a):\t{axisVal:.0f} meters\t(or {axisVal/1000:.3f} km)")

periodFun = sympy.lambdify((a, mu, upsilon), period)
periodVal = periodFun(axisVal, muVal, upsilonVal)
print(f"orbital period(T):\t{periodVal:.0f} secs\t(or {periodVal/60:.2f} mins)")

perigeeVal = 2*axisVal - rVal
print(f"perigee (closest):\t{perigeeVal:.0f} meters\t(or {perigeeVal/1000:.3f} km)")

speedFun = sympy.lambdify((mu, r, a), speed)
perigeeSpeed = speedFun(muVal, perigeeVal, axisVal)
print(f"perigee (fastest):\t{perigeeSpeed:.0f} m/s\t(or {perigeeSpeed/1000:.3f} km/s)")

schwarzschildFun = sympy.lambdify((mu, c), schwarzschild)
blackholeRadius = schwarzschildFun(muVal, cVal)
print(f"blackhole radius :\t{blackholeRadius:.4f} meters\t(or {blackholeRadius*1000:.1f} millimeters)")

```

    semimajor axis(a):	3194597 meters	(or 3194.597 km)
    orbital period(T):	1797 secs	(or 29.95 mins)
    perigee (closest):	11058 meters	(or 11.058 km)
    perigee (fastest):	268272 m/s	(or 268.272 km/s)
    blackhole radius :	0.0089 meters	(or 8.9 millimeters)

## Tidal Forces
Assume that you are two meters tall in your space suit. The worst case is if you're oriented along the semimajor axis at perigee.

```python
# Assume a two meter tall astronaut, and her center of mass is about halfway.
alpha, r_1, r_2 = sympy.symbols("alpha r_1 r_2")
r1Val = perigeeVal + 1          # one meter farther from black hole
r2Val = perigeeVal - 1          # one meter closer to the black hole
accel = mu/r_2**2 - mu/r_1**2
accelEq = sympy.Eq(alpha, accel)
accelEq
```

```python
g = 9.807                       # Earth's acceleration of gravity in m/s/s

accelFun = sympy.lambdify((mu, r_1, r_2), accel)
accelVal = accelFun(muVal, r1Val, r2Val)
tidalVal = accelVal/g

print(f"tidal acceleration:\t{accelVal:.0f} m/s/s\t(or {tidalVal:.0f} g)")
print("This is the difference between head and foot; each end experiences half that.")
```

    tidal acceleration:	1179 m/s/s	(or 120 g)
    This is the difference between head and foot; each end experiences half that.

## Conclusion
You'll not just fall into the peanut-sized black hole. The closest you'll get is about 11 kilometers.

But -- you'd be moving at almost 270 km/s, while turning the corner very, VERY, quickly. This is a highly eccentric orbit.

So, assuming you're 2m tall: the tidal difference between head and foot could be about 120g. Ouch!

However!!! In the worst case scenario, you would experience these extreme accelerations for only a fraction of a second.
It might be survivable, like experiencing a shock wave on Earth.

In fact, it's so quick that there wouldn't be time for the gravity gradient to align you with "down". So, if your space suit has a jetpack, your best use of it would be to make sure you were not oriented along the semimajor axis. (Even so, you're likely to come out of each close pass spinning.) Unfortunately, you would need to deal with this every 30 mins or so -- until you get rescued.

# Diagrams of the Highly Eccentric Orbit

```python
# We also need to calculate the semiminor axis for drawing the following diagrams.
b, c_f = sympy.symbols("b c_f")
minor = sympy.sqrt(a**2 - c_f**2)
minorEq = sympy.Eq(b, minor)
minorEq
```

```python
# Distance from ellipse center to either focus in meters
centerVal = (rVal - perigeeVal)/2

# Length of semiminor axis in meters
minorFun = sympy.lambdify((c_f, a), minor)
minorVal = minorFun(centerVal, axisVal)
print(f"semiminor axis(b):\t{minorVal:.0f} meters\t(or {minorVal/1000:.3f} km)")
```

    semiminor axis(b):	265570 meters	(or 265.570 km)

```python
# Support for diagrams.
import numpy
import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse
from matplotlib.offsetbox import OffsetImage, AnnotationBbox
import matplotlib.image as img
im = img.imread('space-suit-icon.jpg')

plt.rc('font', size=15) # bigger font, by default
scale = 1000            # Report distances in km, rather than meters
x_siz = 16              # width of the display canvas

# Size of the diagram, in km
x_max, y_max = 7000, 7500
x_min, y_min = -7000, -1000
y_siz = x_siz * (y_max-y_min)/(x_max-x_min)

fig, ax = plt.subplots(figsize=(x_siz, y_siz))
ax.set_title("Earth's Sudden Collapse Into A Black Hole (astronaut is not to scale)")
ax.set_ylabel('km')

# Define a circle for the Earth.
theta = numpy.linspace(0, 2*numpy.pi, 121)
r = rVal/scale          # units: km
x = r*numpy.cos(theta)
y = r*numpy.sin(theta)

# Dashed line circle, and arrows pointing in
ax.plot(x, y, 'k--')
ax.quiver(x, y, -x, -y, alpha=0.1)

# Astronaut (not to scale)
imagebox = OffsetImage(im, zoom=0.1)
ab = AnnotationBbox(imagebox, (0, r+400), frameon=False)
ax.add_artist(ab)

# Scaled values, using km
apoSpeed  = upsilonVal/scale
periSpeed = perigeeSpeed/scale
majorDist = axisVal/scale
minorDist = minorVal/scale
periDist  = perigeeVal/scale

# Arrowed boxes with text for speeds
bbox_props = dict(fc=(0.8, 0.9, 0.9), ec="b", lw=2)
t = ax.text(500, r + 400, f"{apoSpeed:.3f} km/s", ha="left", va="center", rotation=0, size=15, bbox=bbox_props)
bb = t.get_bbox_patch()
bb.set_boxstyle("rarrow", pad=0.6)
t = ax.text(0, -400, f"{periSpeed:.0f} km/s", ha="center", va="top", rotation=0, size=15, bbox=bbox_props)
bb = t.get_bbox_patch()
bb.set_boxstyle("larrow", pad=0.6)

# New orbit ellipse
ellipse = Ellipse((0, centerVal/scale), 2*majorDist, 2*minorDist, angle=90, alpha=0.1)
ax.add_patch(ellipse)

# Black hole indicator as red dot -- (dot for black hole is not to scale)
plt.scatter([0], [0], color="r")
plt.text(0, 100, "black hole", ha='center', va='bottom')
plt.text(-1000, axisVal/1000, f" apogee   {r:.0f}km", ha='center', va='center')
plt.plot([-800,-1000,-1000,-800], [0,0,r,r], color='g', lw=2)

plt.axis([x_min, x_max, y_min, y_max])
plt.grid(color='lightgray', linestyle='--')
plt.show()
```

```python
x_max, y_max = 40, 30
x_min, y_min = -40, -20
y_siz = x_siz * (y_max-y_min)/(x_max-x_min)

fig, ax = plt.subplots(figsize=(x_siz, y_siz))
ax.set_title("Zoom In: Black Hole Perigee (astronaut is not to scale)")
ax.set_ylabel('km')

imagebox = OffsetImage(im, zoom=0.1)
ab = AnnotationBbox(imagebox, (0, -perigeeVal/1000), frameon=False)
ax.add_artist(ab)

t = ax.text(10, -periDist, f"{periSpeed:.0f} km/s", ha="left", va="center", rotation=0, size=15, bbox=bbox_props)
bb = t.get_bbox_patch()
bb.set_boxstyle("larrow", pad=0.6)

bbox_props2 = dict(fc=(1, 0.95, 0.95), ec="r", lw=1)
t = ax.text(-5, -periDist-.2, f"{tidalVal/2:.0f} g", ha="center", va="top", rotation=90, size=12, bbox=bbox_props2)
bb = t.get_bbox_patch()
bb.set_boxstyle("larrow")
t = ax.text(-5, -periDist+.2, f"{tidalVal/2:.0f} g", ha="center", va="bottom", rotation=90, size=12, bbox=bbox_props2)
bb = t.get_bbox_patch()
bb.set_boxstyle("rarrow")

ellipse = Ellipse((0, centerVal/scale), 2*majorDist, 2*minorDist, angle=90, alpha=0.1)
ax.add_patch(ellipse)

plt.scatter([0], [0], color="r")
plt.text(0, 0.5, "black hole", ha='center', va='bottom')
plt.text(5, -periDist/2, f" perigee   {periDist:.1f} km", ha='center', va='bottom')
plt.plot([4,5,5,4], [0,0,-periDist,-periDist], color='g', lw=2)
plt.text(-6, -periDist, "tide", ha='right', va='center', rotation=90)

plt.axis([x_min, x_max, y_min, y_max])
plt.grid(color='lightgray', linestyle='--')
plt.show()
```

# Author's Notes

## The astronaut is a pendulum.
My intuition tells me that our astronaut is likely to be spun around by tidal forces every time she encounters perigee, depending on orientation.

But I'm not yet sure how to evaluate the **likely** vs **worst-case** spin scenarios.

## GR effects were not considered.
I have not yet looked into how to evaluate the effect of the following on subsequent orbits:

- GR orbital precession
- frame-dragging (due to Earth's angular momentum)

This would be really cool to investigate. I'd need to do some research for follow-up.
Given the magnitude for change in size, the black hole would likely be spinning very fast, indeed.

## The Moon's effect may not be negligible.
Everything described above only considers the **Earth**!!
But the Earth and the Moon actually **both** orbit about the barycenter (combined center of mass).

If all of the Earth's mass collapsed into its equivalent-mass black hole, this orbital dynamic would not change.
The resulting black hole would trace out a circle every 28 days with 4700 km radius.
In the 15 minutes that it would take for our astronaut to fall towards the black hole, the black hole will have been displaced by almost **11 km**!!

This would not likely change the orbit analysis by much since most of the acceleration takes place over a tiny time frame, but it would complicate any calculation of orbital precession, particularly if we consider latitudes away from the equator.

## Final surprise: black hole spin

So: how fast could the black hole be spinning? (Our previous analysis did not need this info.)

According to https://en.wikipedia.org/wiki/Moment_of_inertia_factor, the Earth's moment of inertia factor is about 0.3307.
I have no idea what this factor would be for a black hole; it's just a constant for different mass distributions.

```python
from IPython.display import Latex, Markdown
```

```latex
%%latex
angular moment of inertia: $I$ = (0.3307) $M$ $R^2$, (for Earth) $\\$
angular momentum: $L = I$ $\omega$,  (where $\omega$ is angular velocity) $\\$
So roughly: $\omega$ $\propto$ $\frac{1}{R^2}$, ignoring relativistic effects $\\$
```

angular moment of inertia: $I$ = (0.3307) $M$ $R^2$, (for Earth) $\\$
angular momentum: $L = I$ $\omega$,  (where $\omega$ is angular velocity) $\\$
So roughly: $\omega$ $\propto$ $\frac{1}{R^2}$, ignoring relativistic effects $\\$

```python
ratio = rVal/blackholeRadius
Markdown(f"As the Earth's radius ($R$) shrinks: {rVal:.3e} $\\ \mapsto\\ $ {blackholeRadius:.4f} meters,\n\n"+
    f"the corresponding tangent velocity grows: {upsilonVal:.1f} $\\ \mapsto\\ $ {upsilonVal*ratio*ratio:.3e} m/s.\n\n"+
    f"That's orders of magnitude faster than the speed of light ($c$): {cVal:.3e} m/s!")
```

As the Earth's radius ($R$) shrinks: 6.378e+06 $\ \mapsto\ $ 0.0089 meters,

the corresponding tangent velocity grows: 465.1 $\ \mapsto\ $ 2.405e+20 m/s.

That's orders of magnitude faster than the speed of light ($c$): 2.998e+08 m/s!

### Getting the answer
I suppose such calculations could be handled with relativistic equations to get a precise figure, where momentum (mass) must rise as the speed gets asymptotically closer to the speed of light.

But for our purposes, we don't need this: we already have a good enoungh answer for such extreme values: **our black hole is spinning at *nearly* the speed of light**.

If we don't relax the assumption (on conservation of Earth's rotational angular momentum), the Ergosphere and frame-dragging effects could be very interesting.

### Follow-up questions:
- How big is the Ergosphere?
- How could frame-dragging affect the tidal force felt by our astronaut?
- Would frame-dragging mean our astronaut could be kicked into a significantly higher orbit?
  If so, she could use her jetpack (thrusting away from the black hole) to gain orbital
  angular momentum at perigee, and then wait for the next apogee (now farther out)
  to improve orbit eccentricity, using her remaining Δv more efficiently.

```python

```


# Introduction to Linear Regression

## Learning Objectives

After doing this section, you'll understand the following:
- Concepts behind Linear Regression
- Differences between Simple Linear Regression and Multiple Linear Regression
- Applications of Linear Regression

## Import Libraries

We begin with the following code to import the necessary libraries.

```python
%matplotlib inline

import matplotlib.pyplot as plt
import numpy as np
```

## Overview of Linear Regression

Linear Regression has been around for 200 years (since 1805-1809). Linear Regression is one of the fundamental machine learning algorithms due to its simplicity and interpretation. It is used to study the relationship between a dependent variable and one or more independent variables by fitting a linear equation to observed data. You may ask that real-world data are never linear, so why do we need to learn Linear Regression?

Despite its simplicity, Linear Regression is extremely useful both conceptually and practically. Linear Regression is a good starting point for regression tasks where you can use it to predict or visualize the relationship between different variables. In addition, it can be extended to model more complicated patterns in data. You will learn about these later in this course.

In the following contents, we will start with a walk-through of the mathematics behind two types of Linear Regression (i.e., Simple Linear Regression and Multiple Linear Regression), before moving on to build your own model and extend it to account real-world data.

## Simple Linear Regression

Simple Linear Regression gets it adjective "simple" because it considers only one independent variable. In constrast, Multiple Linear Regression, which we'll study later, considers two or more independent variables.

Simple Linear Regression assumes that the relationship between an independent variable $x$ and a dependent variable $y$ is linear. It can be modeled as a straight-line:

$$y = b + a x$$
where, $b$ is commonly known as the *intercept*, and $a$ is commonly known as the *slope*.

Here is an example of a straight-line fit to the data:

```python
celsius_values = np.linspace(0, 100, 10)
fahrenheit_values = (9 / 5) * celsius_values + 32

plt.scatter(celsius_values, fahrenheit_values, alpha=0.7)
plt.plot(celsius_values, fahrenheit_values)
plt.xlabel("Celsius")
plt.ylabel("Fahrenheit");
```

We can see that the relationship between degrees Fahrenheit and degrees Celsius is linear. It actually can be formulate as follows:

$$F = \frac{9}{5}C + 32$$

Here are some examples of other linear relationships that you might familiar with:
- Conversion between centimeters and inches
- Exchange rates between Japanese Yen and other currencies
- Diameter of a given circle and the circle’s  circumference

In the above figure, it is important to note that the observed data points fall directly on a line. However, real data are dirty and noisy, and the relationship between the variables is not perfect. Let's consider the following data, which is scattered about a line with a slope of -3 and an intercept of 5.

```python
num_data_points = 100

# get a new instance of random numbers generator
rng = np.random.default_rng(seed=12345)

# generate x
x = rng.random(size=(num_data_points, 1))

# generate y using the defined slope, intercept, and random noise
slope = -3
intercept = 5
y = (slope * x + intercept) + rng.random(size=(num_data_points, 1))

print("Shape of x:", x.shape)
print("The first 5 rows of x:", x[:5])
print()
print("Shape of y:", y.shape)
print("The first 5 rows of y:", y[:5])

plt.scatter(x, y, alpha=0.7)
plt.plot(x, slope * x + intercept);
```

In such noisy data, instead of trying to find a "perfect" fitting line, we will try to find a "best" fitting line to represent the relationship between variables. The equation for the best fitting line is:

$$y \approx \hat{y} = w_0 + w_1 x$$

The "hat" on $y$ is used to indicate that this is an estimate and is not perfectly correct. The variation between an observed data point $(x^{(i)}, y^{(i)})$ and its predicted data point $(x^{(i)}, \hat{y}^{(i)})$ is **prediction error**, or **residual error** $\varepsilon_i$

$$\varepsilon^{(i)} = y^{(i)} - \hat{y}^{(i)}$$

We can use NumPy method `polyfit` to fit the above noisy data and construct the best fitting line.

```python
w_1, w_0 = np.polyfit(x[:, 0], y[:, 0], deg=1)
print(f"Best fitting line: y_hat = {w_0} + {w_1}*x")

plt.scatter(x, y, alpha=0.7)
plt.plot(x, w_0 + w_1 * x, color="green", label="Best fitting line")

residuals = y - (w_0 + w_1 * x)
plt.bar(x[:, 0], residuals[:, 0], width=0.01, label="Residual errors")

plt.legend();
```

Each observed data point will have a residual. The goal of finding a best fitting line is to minimize these residuals by picking the right $w_0$ and $w_1$. This goal can be formulate as follows:

$$\underset{w_0, w_1}{argmin}\mathcal{L}(w_0, w_1)$$

where $\mathcal{L}(w_0, w_1)$ can be the sum of the squared residual errors (or **least squares criterion**).

$$\mathcal{L}(w_0, w_1) = \frac{1}{2}\sum^m_{i=1}(y^{(i)} - \hat{y}^{(i)})^2$$

where $m$ is the number of observed data points. $\mathcal{L}$ is also called the loss function.

## Multiple Linear Regression

In machine learning, we will generally have various input variables to determine the outcome. In such cases, we will use Multiple Linear Regression to model the relationship between a depedent $y$ variable and $n$ $x$-variables, the equation is written as:

$$y \approx \hat{y} = w_0 + w_1 x_1 + ... + w_j x_j + ... + w_n x_n$$

where, $w_0, w_1, ..., w_n$ are the coefficients. In machine learning, $w_0$ is commonly known as *bias*, $w_1, ..., w_n$ is commonly known as *model weights*.

The above equation can be visualize as a plane or hyper-plane in higher dimensions.

```python
num_data_points = 100

# get a new instance of random numbers generator
rng = np.random.default_rng(seed=12345)

# generate x_1 and x_2
x_1 = rng.random(size=(num_data_points, 1))
x_2 = rng.random(size=(num_data_points, 1))

# initialize w_0, w_1, and w_2
w_0, w_1, w_2 = 2, 3, 5

# compute y
y = (w_0 + w_1 * x_1 + w_2 * x_2) + rng.random(size=(num_data_points, 1))

print("Shape of x_1:", x.shape)
print("The first 5 rows of x_1:", x_1[:5])
print()
print("Shape of x_2:", x_2.shape)
print("The first 5 rows of x:", x_2[:5])
print()
print("Shape of y:", y.shape)
print("The first 5 rows of y:", y[:5])

fig = plt.figure(figsize=(8, 6))
axis = plt.axes(projection="3d")
axis.set_xlabel(r"$x_1$")
axis.set_ylabel(r"$x_2$")
axis.set_zlabel(r"$y$")
axis.scatter(x_1, x_2, y)

x_1_mesh, x_2_mesh = np.meshgrid(np.linspace(0, 1, 5), np.linspace(0, 1, 5))
y_hat_mesh = w_0 + w_1 * x_1_mesh + w_2 * x_2_mesh
axis.plot_surface(
    x_1_mesh,
    x_2_mesh,
    y_hat_mesh,
    cmap="Oranges",
    alpha=0.4,
)
axis.azim = 10
axis.elev = 30
```

Each $x_j$ variable can be an independent variable or a transformation of independent variables (such as the square of an independent variable or the multiplication of two independent variables). Allowing non-linear transformation of independent variables enables the Multiple Linear Regression to represent non-linear relationships between the dependent variable and independent variables.

### Matrix Formulation of Multiple Linear Regression

In the Multiple Linear Regression, because of the potentially large number of independent variables, it is more efficient to use matrices to define the regression model.

Let's start with the simple case first. Consider the following Simple Linear Regression function:

$$\hat{y}^{(i)} = w_0 + w_1 x^{(i)} \quad \text{for } i=1,...,m$$

where $m$ is the number of observed data points. If we let $i=1,...,m$, we can obtain $m$ equations:

$$
\hat{y}^{(1)} = w_0 + w_1 x^{(1)} \\
\hat{y}^{(2)} = w_0 + w_1 x^{(2)} \\
\vdots \\
\hat{y}^{(m)} = w_0 + w_1 x^{(m)}
$$

We can formulate the above Simple Linear Regression equations in matrix notation:

$$
\begin{bmatrix}
  \hat{y}^{(1)} \\
  \hat{y}^{(2)} \\
  \vdots \\
  \hat{y}^{(m)}
\end{bmatrix} =
  \begin{bmatrix}
    1 & x^{(1)} \\
    1 & x^{(2)} \\
    \vdots \\
    1 & x^{(m)}
  \end{bmatrix} \begin{bmatrix}
    w_0 \\
    w_1
  \end{bmatrix}
$$

which, can be further reduce to a short and simple equation:

$$
  \pmb{\hat{y}} = \pmb{X}\pmb{w}
$$

where $\pmb{\hat{y}}$ is an $m \times 1$ column vector, $\pmb{X}$ is an $m \times 2$ matrix, and $\pmb{w}$ is a $2 \times 1$ column vector.

Using matrix notation, we can easily extend the above equation to Multiple Linear Regression with $n$ $x$-variables.

$$
\begin{align}
  \pmb{\hat{y}} &= \pmb{X}\pmb{w} \\
  \begin{bmatrix}
    \hat{y}^{(1)} \\
    \hat{y}^{(2)} \\
    \vdots \\
    \hat{y}^{(m)}
  \end{bmatrix} &=
    \begin{bmatrix}
      1 & x^{(1)}_1 & x^{(1)}_2 & \cdots & x^{(1)}_n \\
      1 & x^{(2)}_1 & x^{(2)}_2 & \cdots & x^{(1)}_n \\
      \vdots \\
      1 & x^{(m)}_1 & x^{(m)}_2 & \cdots & x^{(m)}_n
    \end{bmatrix} \begin{bmatrix}
      w_0 \\
      w_1 \\
      \vdots \\
      w_n
    \end{bmatrix}
\end{align}
$$

where $\pmb{\hat{y}}$ is an $m \times 1$ column vector, $\pmb{X}$ is an $m \times (n+1)$ matrix, and $\pmb{w}$ is a $(n+1) \times 1$ column vector.

### Estimate Bias and Weights

The goal of Linear Regression is to find a best fitting line by finding the values of $\pmb{w}$ parameters to minimize the loss function $\mathcal{L}(\pmb{w})$. The sum of squared residual errors is commonly used and formulated as follows:

$$
\begin{align}
  \mathcal{L}(\pmb{w}) &= \frac{1}{2}\sum^m_{i=1}(y^{(i)} - \hat{y}^{(i)})^2 \\
  &= \frac{1}{2} \lVert \pmb{y} - \pmb{\hat{y}} \lVert ^2
\end{align}
$$

Usually, we will perform an optimization algorithm (e.g., gradient descent) for finding the best $w$ parameters to minimize the loss function $\mathcal{L}(\pmb{w})$. You can learn about these algorithms in another course on our platform. In this course, we will solve this optimization problem in a different approach.

We further rewritten the loss function as follows:

$$
\begin{align}
  \mathcal{L}(\pmb{w}) &= \frac{1}{2} (\pmb{y} - \pmb{\hat{y}})^\top (\pmb{y} - \pmb{\hat{y}}) \\
  &= \frac{1}{2} (\pmb{y}^\top \pmb{y} - 2 \pmb{y}^\top \pmb{\hat{y}}
    + \pmb{\hat{y}}^\top \pmb{\hat{y}}) \\
  &= \frac{1}{2} (\pmb{y}^\top \pmb{y} - 2 \pmb{y}^\top \pmb{X} \pmb{w}
    + \pmb{X}^\top \pmb{w}^\top \pmb{X} \pmb{w}) \\
\end{align}
$$

Since the loss function is convex, the best $\pmb{w}$ lies at gradient zero. Therefore, to minimize $\mathcal{L}(\pmb{w})$, we set its partial derivative with respect to $\pmb{w}$ to 0. The solution satisfies:

$$
\begin{align}
  &\frac{\partial}{\partial \pmb{w}} \mathcal{L}(\pmb{w}) &= &0 \\
  \Leftrightarrow  &\frac{1}{2} (- 2 \pmb{y}^\top \pmb{X} + 2 \pmb{X}^\top \pmb{X} \pmb{w}) &= & 0 \\
  \Leftrightarrow  &\pmb{X}^\top \pmb{X} \pmb{w} &= &\pmb{X}^\top \pmb{y} \\
  \Leftrightarrow  &\pmb{w} &= &(\pmb{X}^\top \pmb{X})^{-1} \pmb{X}^\top \pmb{y}
\end{align}
$$

The optimization problem is solved! We now have an equation that can help us compute $\pmb{w}$ parameters directly from the observed data. The equation is also known as the normal equation.

$$\pmb{w} = (\pmb{X}^\top \pmb{X})^{-1} \pmb{X}^\top \pmb{y} $$

Congratulation! You now have a brief understanding of Linear Regression. Let's move
to the next section where we will guide you on how to utilize the normal equation to implement a Linear Regression model by yourself.