# Ordinary Differential Equation Solvers: Runge-Kutta Methods
### Christina Lee
### Category: Numerics

So what's an Ordinary Differential Equation?

Differential Equation means we have some equation (or equations) that have derivatives in them.

The ordinary part differentiates them from partial differential equations (the ones with curly $\partial$ derivatives).  Here, we only have one independent variable (let's call it $t$), and one or more dependent  variables (let's call them $x_1, x_2, ...$).  In partial differential equations, we can have more than one independent variable.

This ODE can either be written as a system of the form
$$
\frac{d x_1}{dt}=f_1(t,x_1,x_2,...,x_n)
$$
$$
\frac{d x_2}{dt}=f_2(t,x_1,x_2,...,x_n)
$$
...
$$
\frac{d x_n}{dt}=f_n(t,x_1,x_n,...,x_n)
$$
or a single n'th order ODE of the form
$$
f_n(t,x) \frac{d^n x}{dt^n}+...+f_1(t,x) \frac{dx}{dt}+f_0(t,x)=0
$$
that can be rewritten in terms of a system of first order equations by performing variable substitutions such as
$$
\frac{d^i x}{dt^i}=x_i
$$

Though STEM students such as I have probably spent thousands of hours pouring of ways to analytically solve both ordinary and partial differential equations, unfortunately, the real world is rarely so kind as to provide us with an exactly solvable differential equation.  At least for interesting problems.

We can sometimes approximate the real world as an exactly solvable situation, but for the situation we are interested in, we have to turn to numerics.  I'm not saying those thousand different analytic methods are for nothing.  We need an idea ahead of time of what the differential equation should be doing, to tell if it's misbehaving or not.  We can't just blindly plug and chug.

Today will be about introducing four different methods based on Taylor expansion to a specific order, also known as Runge-Kutta Methods.  We can improve these methods with adaptive stepsize control, but that is a topic for another time, just like the other modern types of solvers such as Richardson extrapolation and predictor-corrector.

Nonetheless, to work with ANY computational differential equation solver, you need to understand the fundamentals of routines like Euler and Runge-Kutta, their error propagation, and where they can go wrong. Otherwise, you might misinterpret the results of more advanced methods.

WARNING: If you need to solve a troublesome differential equation for a research problem, use a package, like [DifferentialEquations](https://github.com/JuliaDiffEq/DifferentialEquations.jl).  These packages have much better error handling and optimization.

Let's first add our plotting package and colors.

```julia
using Plots
gr()
```

    Plots.GRBackend()

We will be benchmarking our solvers on one of the simplest and most common ODE's,

\begin{equation}
\frac{d}{d t}x=x \;\;\;\;\;\;\; x(t)=C e^t
\end{equation}

Though this only has one dependent variable, we want to structure our code so that we can accommodate a series of dependent variables, $x_1,x_2,...,x_n$, and their associated derivative functions.  Therefore, we create a function for each dependent variable, and then `push` it into an array declared as type `Function`.

```julia
function f1(t::Float64,x::Array{Float64,1})
    return x[1]
end
f=Function[]
push!(f,f1)

```

    1-element Array{Function,1}:
     f1

### Euler's Method

First published in Euler's Instutionum calculi integralis in 1768, this method gets a lot of milage, and if you are to understand anything, this method is it.

We march along with step size $h$, and at each new point, calculate the slope.  The slope gives us our new direction to travel for the next $h$.

We can determine the error from the Taylor expansion of the function
$$
x_{n+1}=x_n+h f(x_n,t) + \mathcal{O}(h^2).
$$
In case you haven't seen it before, the notation $\mathcal{O}(x)$ stands for "errors of the order x".
Summing over the entire interval, we accumuluate error according to
$$N\mathcal{O}(h^2)= \frac{x_f-x_0}{h}\mathcal{O}(h^2)=h, $$
making this a first order method.  Generally, if a technique is $n$th order in the Taylor expansion for one step, its $(n-1)$th order over the interval.

```julia
function Euler(f::Array{Function,1},t0::Float64,x::Array{Float64,1},h::Float64)
    d=length(f)
    xp=copy(x)
    for ii in 1:d
        xp[ii]+=h*f[ii](t0,x)
    end

    return t0+h,xp
end
```

    Euler (generic function with 1 method)

```julia

```

## Implicit Method or Backward Euler

If $f(t,x)$ has a form that is invertible, we can form a specific expression for the next step.  For example, if we use our exponential,
\begin{equation}
x_{n+1}=x_n+ h f(t_{n+1},x_{n+1})
\end{equation}
\begin{equation}
x_{n+1}-h x_{n+1}=x_n
\end{equation}
\begin{equation}
x_{n+1}=\frac{x_n}{1-h}
\end{equation}

This expression varies for each differential equation and only exists if the function is invertible.

```julia
function Implicit(f::Array{Function,1},t0::Float64,x::Array{Float64,1},h::Float64)
    return t0+h,[ x[1]/(1-h) ]
end
```

    Implicit (generic function with 1 method)

## 2nd Order Runge-Kutta

So in the Euler Method, we could just make more, tinier steps to achieve more precise results. Here, we make bettter steps.  Each step itself takes more work than a step in the first order methods, but we win by having to perform fewer steps.

This time, we are going to work with the Taylor expansion up to second order,
\begin{equation}
x_{n+1}=x_n+h f(t_n,x_n) + \frac{h^2}{2} f^{\prime}(t_n,x_n)+ \mathcal{O} (h^3).
\end{equation}

Define
\begin{equation}
k_1=f(t_n,x_n),
\end{equation}
so that we can write down the derivative of our $f$ function, and the second derivative (curvature), of our solution,
\begin{equation}
f^{\prime}(t_n,x_n)=\frac{f(t_n+h/2,x_n+h k_1/2)-k_1}{h/2}+\mathcal{O}(h^2).
\end{equation}
Plugging this expression back into our Taylor expanion, we get a new expression for $x_{n+1}$
\begin{equation}
x_{n+1}=x_n+hf(t_n+h/2,x_n+h k_1/2)+\mathcal{O}(h^3)
\end{equation}

We can also interpret this technique as using the slope at the center of the interval, instead of the start.

```julia
function RK2(f::Array{Function,1},t0::Float64,x::Array{Float64,1},h::Float64)
    d=length(f)

    xp=copy(x)
    xk1=copy(x)

    for ii in 1:d
        xk1[ii]+=f[ii](t0,x)*h/2
    end
    for ii in 1:d
        xp[ii]+=f[ii](t0+h/2,xk1)*h
    end

    return t0+h,xp
end
```

    RK2 (generic function with 1 method)

## 4th Order Runge-Kutta
Wait! Where's 3rd order? There exists a 3rd order method, but I only just heard about it while fact-checking for this post.  RK4 is your dependable, multi-purpose workhorse, so we are going to skip right to it.

$$
k_1= f(t_n,x_n)
$$
$$
k_2= f(t_n+h/2,x_n+h k_1/2)
$$
$$
k_3 = f(t_n+h/2, x_n+h k_2/2)
$$
$$
k_4 = f(t_n+h,x_n+h k_3)
$$
$$
x_{n+1}=x_n+\frac{h}{6}\left(k_1+2 k_2+ 2k_3 +k_4 \right)
$$
I'm not going to prove here that the method is fourth order, but we will see numerically that it is.

Note: I premultiply the $h$ in my code to reduce the number of times I have to multiply $h$.

```julia
function RK4(f::Array{Function,1},t0::Float64,x::Array{Float64,1},h::Float64)
    d=length(f)

    hk1=zeros(Float64,length(x))
    hk2=zeros(Float64,length(x))
    hk3=zeros(Float64,length(x))
    hk4=zeros(Float64,length(x))

    for ii in 1:d
        hk1[ii]=h*f[ii](t0,x)
    end
    for ii in 1:d
        hk2[ii]=h*f[ii](t0+h/2,x+hk1/2)
    end
    for ii in 1:d
        hk3[ii]=h*f[ii](t0+h/2,x+hk2/2)
    end
    for ii in 1:d
        hk4[ii]=h*f[ii](t0+h,x+hk3)
    end

    return t0+h,x+(hk1+2*hk2+2*hk3+hk4)/6
end
```

    RK4 (generic function with 1 method)

This next function merely iterates over a certain number of steps for a given method.

```julia
function Solver(f::Array{Function,1},Method::Function,t0::Float64,
        x0::Array{Float64,1},h::Float64,N::Int64)
    d=length(f)
    ts=zeros(Float64,N+1)
    xs=zeros(Float64,d,N+1)

    ts[1]=t0
    xs[:,1]=x0

    for i in 2:(N+1)
        ts[i],xs[:,i]=Method(f,ts[i-1],xs[:,i-1],h)
    end

    return ts,xs
end
```

    Solver (generic function with 1 method)

```julia
N=1000
xf=10
t0=0.
x0=[1.]
dt=(xf-t0)/N

tEU,xEU=Solver(f,Euler,t0,x0,dt,N);
tIm,xIm=Solver(f,Implicit,t0,x0,dt,N);
tRK2,xRK2=Solver(f,RK2,t0,x0,dt,N);
tRK4,xRK4=Solver(f,RK4,t0,x0,dt,N);

xi=tEU
yi=exp.(xi);

errEU=reshape(xEU[1,:],N+1)-yi
errIm=reshape(xIm[1,:],N+1)-yi
errRK2=reshape(xRK2[1,:],N+1)-yi;
errRK4=reshape(xRK4[1,:],N+1)-yi;
```

```julia
plot(tEU,xEU[1,:],label="Euler")
plot!(tIm,xIm[1,:],label="Implicit")
plot!(tRK2,xRK2[1,:],label="RK2")
plot!(tRK4,xRK4[1,:],label="RK4")
plot!(xi,yi,label="Exact")
plot!(xlabel="Independent Variable",ylabel="Dependent variable",title="Comparing Methods")
```

```julia
plot(xi,errEU,label="Euler")
plot!(xi,errIm,label="Implicit")
plot!(xi,errRK2,label="RK2")
plot!(xi,errRK4,label="RK4")
plot!(xlabel="Independent Variable",ylabel="Error",title="Comparison of error scaling")
```

## Scaling of the Error

I talked above about the error scaling either as $h,h^2$, or $h^4$.  I won't just talk but here will numerically demonstrate the relationship as well.  For a variety of different step sizes, the below code calculates the final error for each method.  Then we will plot the error and see how it scales.

```julia
t0=0.
tf=1.
dx=tf-t0
x0=[1.]

dt=collect(.001:.0001:.01)

correctans=exp(tf)
errfEU=zeros(Float64,length(dt))
errfIm=zeros(Float64,length(dt))
errfRK2=zeros(Float64,length(dt))
errfRK4=zeros(Float64,length(dt))

for ii in 1:length(dt)
    N=round(Int,dx/dt[ii])
    dt[ii]=dx/N

    tEU,xEU=Solver(f,Euler,t0,x0,dt[ii],N);
    tIm,xIm=Solver(f,Implicit,t0,x0,dt[ii],N);
    tRK2,xRK2=Solver(f,RK2,t0,x0,dt[ii],N);
    tRK4,xRK4=Solver(f,RK4,t0,x0,dt[ii],N);

    errfEU[ii]=xEU[1,end]-correctans
    errfIm[ii]=xIm[1,end]-correctans
    errfRK2[ii]=xRK2[1,end]-correctans
    errfRK4[ii]=xRK4[1,end]-correctans
end
```

```julia
plot(x->errfEU[end]*x/.01,dt[1],dt[end],label="Fitted line",
    linewidth=10)
plot!(dt,errfEU,label="Error",linewidth=3)

plot!(xlabel="step size",ylabel="final error",
    title="Linear Error of Euler Method")
```

```julia
plot(x->errfIm[end]*x/.01,dt[1],dt[end],label="Fitted line",
    linewidth=10)
plot!(dt,errfIm,label="Error",linewidth=3)
plot!(xlabel="step size",ylabel="final error",
    title="Linear Error of Implicit Method")

```

```julia
plot(x->errfRK2[end]*(x/.01)^2,dt[1],dt[end],label="Quadratic",
    linewidth=10)
plot!(dt,errfRK2,label="Error",linewidth=3)

plot!(xlabel="step size",ylabel="final error",
    title="Quadratic Error of Runge-Kutta 2 Method")
```

```julia
plot(x->errfRK4[end]*(x/.01)^4,dt[1],dt[end],label="Quartic",
    linewidth=10)
plot!(dt,errfRK4,label="Error",linewidth=3)

plot!(xlabel="step size",ylabel="final error",
    title="Quartic Error of Runge-Kuuta 4 Method")
```

## Arbitrary Order
While I have presented 4 concrete examples, many more exist. For any choice of variables $a_i, \beta_{i,j},a_i$ that fulfill
$$
x_{n+1}=x_n+h\left(\sum_{i=1}^s a_i k_i \right)+ \mathcal{O}(h^p)
$$
with
$$
k_i=f\left(t_n+\alpha_i h,x_n+h\left(\sum_{j=1}^s \beta_{ij} k_j \right) \right)
$$
we have a Runge-Kutta method of order $p$, where $p\geq s$.  The Butcher tableau provides a set of consistent coefficients.

Stay tuned for when we tuned these routines to the stiff van der Pol equations!

```julia

```


# Elementary functions 2

Now that we know what an inverse function is, let us think about the inverse functions of elementary functions we explored in a previous notebook.

## Roots

```python
import numpy as np
import matplotlib.pyplot as plt

x1 = np.linspace(0, 10, 101)
x2 = np.linspace(-10, 10, 101)

fig, ax = plt.subplots(1, 2, figsize=(10, 5))

ax[0].plot(x1, x1**2, label=r'$G(x^2)$')
ax[0].plot(x1, np.sqrt(x1), label=r'$G(x^{1/2})$')
ax[0].set_xlim(0, 10)
ax[0].set_ylim(0, 10)

ax[1].plot(x2, x2**3, label=r'$G(x^3)$')
ax[1].plot(x2, np.cbrt(x2), label=r'$G(x^{1/3})$')
ax[1].set_ylim(-10, 10)
ax[1].set_yticks([-10, -5, 5, 10])

for i in range(2):
    ax[i].plot(x2, x2, '--k', alpha=0.2)
    ax[i].text(7.5, 7, 'y=x', alpha=0.5)
    ax[i].set_aspect('equal')
    ax[i].spines['left'].set_position('zero')
    ax[i].spines['bottom'].set_position('zero')
    ax[i].spines['right'].set_visible(False)
    ax[i].spines['top'].set_visible(False)
    ax[i].legend(loc='best')

plt.show()
```

## Logarithmic functions

The exponential function $f: \mathbb{R} \to (0, + \infty), f(x)=a^x$ is an injection because it is strictly increasing and it is a surjection because its image is $f(\mathbb{R}) = (0, + \infty)$. Therefore, it is a bijection and the inverse function $f^{-1} : (0, + \infty) \to \mathbb{R}$ exists.

We denote this inverse function as $\log_a x := f^{-1}(x)$ and we call it the **logarithm with base a**.

We call a logarithm with base **$e$** the **natural logarithm** and define it as:

$$ \ln x := \log_e x $$

```{admonition} Properties

Domain: $(0, + \infty)$, range: $\mathbb{R}$

$$ \log_a x = y \implies x = a^y $$

$$ (f \circ f^{-1})(y) = a^{\log_a y} = y, \forall y > 0 $$
$$ (f^{-1} \circ f)(x) = \log_a a^x = x, \forall x \in \mathbb{R} $$

For all $x, y \in (0, + \inft), a, b, c \in (0, 1) \cup (1, + \infty)$:

$$ \log_a 1 = 0 $$

Since the exponential function maps a sum to a product, i.e. $a^{x+y} = a^x a^y$, the logarithmic function maps a product to a sum:

$$ \begin{align}
\log_a (xy) & = \log_a x + \log_a y \\
\log_a \frac{x}{y} & = \log_a x - \log_a y
\end{align}$$

$$ \begin{align}
\log_a x^\alpha & = \alpha \log_a x \\
\log_{a^\beta} x & = \frac{1}{\beta} \log_a x
\end{align} $$

Change of base:

$$ \log_b c = \frac{\log_a c}{\log_a b} $$

```

```python
x1 = np.linspace(-10, 10, 201)
x2 = np.linspace(0.001, 10, 301)

fig, ax = plt.subplots(1, 2, figsize=(10, 5))

ax[0].plot(x1, 0.5**x1, label=r'$0.5^x$')
ax[0].plot(x2, np.log(x2)/np.log(0.5), label=r'$\log_{0.5}x$')
ax[0].set_title('0 < a < 1')

ax[1].plot(x1, np.exp(x1), label=r'$e^x$')
ax[1].plot(x2, np.log(x2), label=r'$\ln x$')
ax[1].set_title('a > 1')

for i in range(2):
    ax[i].plot(x1, x1, '--k', alpha=0.2)
    ax[i].set_aspect('equal')
    ax[i].spines['left'].set_position('zero')
    ax[i].spines['bottom'].set_position('zero')
    ax[i].spines['right'].set_visible(False)
    ax[i].spines['top'].set_visible(False)
    ax[i].legend(loc='upper right')
    ax[i].set_ylim(-4, 10)
    ax[i].set_xlim(-4, 10)
```

## Inverse trigonometric functions

```{index} Inverse trigonometric functions (arcus)
```

**Inverse trigonometric functions** are also called **arcus functions** so we denote them with the prefix $\text{arc-}$, e.g. $\arcsin x$. They are also often denoted as $\sin^{-1} x$.

Periodic functions are not injections so they do not have inverses. Therefore, we need to restrict them to parts of their natural domain where they are strictly monotonic - here they are injections.

Then trigonometric functions with the following domain restrictions are bijections:

$$ \begin{aligned}
& \left. \sin \right|_{[-\frac{\pi}{2}, \frac{\pi}{2}]} : [-\frac{\pi}{2}, \frac{\pi}{2}] \to [-1, 1] \\
& \left. \cos \right|_{[0, \pi]} : [0, \pi] \to [-1, 1] \\
& \left. \tan \right|_{(-\frac{\pi}{2}, \frac{\pi}{2})} : (-\frac{\pi}{2}, \frac{\pi}{2}) \to \mathbb{R} \\
& \left. \csc \right|_{(-\frac{\pi}{2}, 0) \cup (0, \frac{\pi}{2})} : (-\frac{\pi}{2}, 0) \cup (0, \frac{\pi}{2}) \to (- \infty, -1) \cup (-1, + \infty) \\
& \left. \sec \right|_{(0, \frac{\pi}{2}) \cup (\frac{\pi}{2}, \pi)} : (0, \frac{\pi}{2}) \cup (\frac{\pi}{2}, \pi) \to (- \infty, -1) \cup (-1, + \infty) \\
& \left. \cot \right|_{(0, \pi)} : (0, \pi) \to \mathbb{R}
\end{aligned}$$

so we can define their inverse functions:

$$ \begin{aligned}
& \arcsin : [-1, 1] \to [-\frac{\pi}{2}, \frac{\pi}{2}] \\
& \arccos : [-1, 1] \to [0, \pi] \\
& \arctan : \mathbb{R} \to (-\frac{\pi}{2}, \frac{\pi}{2}) \\
& \text{arccsc} : (- \infty, -1) \cup (-1, + \infty) \to (-\frac{\pi}{2}, 0) \cup (0, \frac{\pi}{2}) \\
& \text{arcsec} : (- \infty, -1) \cup (-1, + \infty) \to (0, \frac{\pi}{2}) \cup (\frac{\pi}{2}, \pi) \\
& \text{arccot} : \mathbb{R} \to (0, \pi)
\end{aligned}$$

```python
# for demonstration purposes will suppress warnings when plotting
import warnings; warnings.simplefilter('ignore')

ax = [0, 0, 0]

fig = plt.figure(constrained_layout=True, figsize=(10, 8))
gs = plt.GridSpec(2, 2, width_ratios=[1.3, 3], height_ratios=[1, 1])

ax[0] = fig.add_subplot(gs[:, 0])
ax[1] = fig.add_subplot(gs[0, 1])
ax[2] = fig.add_subplot(gs[1, 1])

x = np.linspace(-1, 1, 101)
ax[0].plot(x, np.arcsin(x), label=r'$\arcsin x$')
ax[0].plot(x, np.arccos(x), label=r'$\arccos x$')

# will include 0 to separate lines joining
x = np.concatenate((np.linspace(-5, -1, 100), [0], np.linspace(1, 5, 100)))
ax[1].plot(x, np.arcsin(1/x), label=r'arccsc $x$')
ax[1].plot(x, np.arccos(1/x), label=r'arcsec $x$')

x = np.linspace(-10, 10, 201)
ax[2].plot(x, np.arctan(x), label=r'$\arctan x$')
ax[2].plot(x, np.pi/2 - np.arctan(x), label=r'arccot $x$')

for i in range(3):
    ax[i].spines['left'].set_position('zero')
    ax[i].spines['bottom'].set_position('zero')
    ax[i].spines['right'].set_visible(False)
    ax[i].spines['top'].set_visible(False)
    ax[i].legend(loc='best')
    ax[i].set_yticks(np.pi * np.array([-2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2]))
    ax[i].set_yticklabels(['$-2\pi $', r'$-\frac{3}{2} \pi $', r'$-\pi$',
                    r'$-\frac{1}{2}\pi$', '0', r'$\frac{1}{2}\pi$',
                    r'$\pi$', r'$\frac{3}{2}\pi$', r'$2\pi$'])

ax[0].set_xlim(-1.2, 1.2)
ax[0].set_ylim(-1.8, 3.3)

ax[1].set_xlim(-5, 5)
ax[1].set_ylim(-1.7, 3.3)

ax[2].set_ylim(-1.7, 3.3)

plt.show()
```

```{admonition} Properties

For a more complete list of properties and their visualisation, visit [Wikipedia](https://en.wikipedia.org/wiki/Inverse_trigonometric_functions#Relationships_between_trigonometric_functions_and_inverse_trigonometric_functions). Here we will name a few.

Complementary angles:

$$ \arccos x = \frac{\pi}{2} - \arcsin x, \quad \text{arccot} x = \frac{\pi}{2} - \arctan x, \quad \text{arccsc} x = \frac{\pi}{2} - \text{arcsec} x $$

Reciprocal arguments:

$$ \begin{aligned}
\arccos \left( \frac{1}{x} \right) = \text{arcsec} x, \quad & \quad \text{arcsec} \left( \frac{1}{x} \right) = \arccos x \\
\arcsin \left( \frac{1}{x} \right) = \text{arccsc} x, \quad & \quad \text{arccsc} \left( \frac{1}{x} \right) = \arcsin x \\
\text{if } x > 0: \arctan \left( \frac{1}{x} \right) = \text{arccot} x, \quad & \quad \text{arccot} \left( \frac{1}{x} \right) = \arctan x \\
\text{if } x < 0: \arctan \left( \frac{1}{x} \right) = \text{arccot} x - \pi, \quad & \quad \text{arccot} \left( \frac{1}{x} \right) = \arctan x + \pi \\
\end{aligned} $$

```

## Inverse hyperbolic functions

```{index} Inverse hyperbolic functions (area)
```

**Inverse hyperbolic functions** are also called **area functions** so we denote them with the prefix $\text{ar-}$, e.g. $\arsinh x$. They are also often denoted as $\sinh^{-1} x$, for example.

Hyperbolic functions

$$ \begin{align}
\sinh : \mathbb{R} \to \mathbb{R} \\
\left. \cosh \right|_{[0, + \infty)} : [0, + \infty) \to [1, + \infty) \\
\tanh : \mathbb{R} \to (-1, 1) \\
\coth : \mathbb{R} \backslash \{ 0 \} \to (- \infty, -1) \cup (1, + \infty)
\end{align} $$

are bijections, so we can define their inverses:

$$ \begin{align}
\text{arsinh} := \sinh ^{-1} : \mathbb{R} \to \mathbb{R} \\
\text{arcosh} := \left ( \left. \cosh \right|_{[0, + \infty)} \right)^{-1} : [1, + \infty) \to [0, + \infty) \\
\text{artanh} := \tanh^{-1} : (-1, 1) \to \mathbb{R} \\
\text{arcoth} := \coth^{-1} : (- \infty, -1) \cup (1, + \infty) \to \mathbb{R} \backslash \{ 0 \}
\end{align} $$

where

$$
\text{arsinh} x = \ln ( x + \sqrt{x^2 + 1} ), \qquad \text{arcosh} x = \ln (x + \sqrt{x^2 - 1}), \\
\text{artanh} x = \frac{1}{2} \ln \left( \frac{1+x}{1-x} \right ), \qquad \text{arcoth} x = \frac{1}{2} \ln \left( \frac{x+1}{x-1} \right )
$$

```{admonition} Derivation of arcosh $x$

For demonstration purposes let us derive the above equation for $\text{arcosh} x$. Recall that $\cosh x = \frac{e^x + e^{-x}}{2}$. Let that equal some $y$, so that $\frac{e^x + e^{-x}}{2} = y$ or $e^x + e^{-x} - 2y = 0$. Now let $u = e^x$ and multiply both sides by $u$. We get:

$$ u^2 - 2uy + 1 = 0., $$

which is a simple quadratic equation with solutions:

$$ u_{1,2} = y \pm \sqrt{y^2 - 1}. $$

But since $u = e^x > 0$ the only possible solution is

$$ u = e^x = y + \sqrt{y^2 - 1}. $$

We take the natural logarithm of both sides:

$$ x = \ln \left ( y + \sqrt{y^2 -1} \right ) $$

```

```python
fig = plt.figure(figsize=(10, 8))
ax = plt.gca()

x = np.linspace(-4, 4, 201)
plt.plot(x, np.log(x + np.sqrt(x**2 + 1)), label=r'arsinh $x$')

x = np.linspace(1, 4, 101)
plt.plot(x, np.log(x + np.sqrt(x**2 - 1)), label=r'arcosh $x$')

x = np.linspace(-0.999, 0.999, 200)
plt.plot(x, np.log((1+x)/(1-x)) / 2, label=r'artanh $x$')

# will include 0 to avoid joining lines, this returns an error
x = np.concatenate((np.linspace(-4, -1.001, 100), [0], np.linspace(1.001, 4, 100)))
plt.plot(x, np.log((x+1)/(x-1)) / 2, label=r'artanh $x$')

ax.spines['left'].set_position('zero')
ax.spines['bottom'].set_position('zero')
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)
ax.set_aspect('equal')
ax.legend(loc='best')

plt.show()
```

```python

```


# Mixture of Gaussian and Uniform Distribution

First we define a PDF:

```
from sympy import *
%matplotlib inline
init_printing()
sigma, sigma1, p, mu, mu1, x, a, b  = symbols('sigma, sigma1, p, mu, mu1, x, a, b')
sigma2, mu2 = symbols('sigma2, mu2')
pdf = p * (1 / sigma1) * (exp((-1/2)*((x-mu1)/sigma1)**2)/sqrt(2* pi)) + (1-p) * (1/(b-a))
pdf
```

Let's compute the CDF:

```
cdf = integrate(pdf,x)
cdf = simplify(cdf)
cdf
```

Now lets define mu and solve for mu1:

```
mu_simpli = p * mu1 + (1-p) * ((1/2) * (a+b))
mu1 = solve(Eq(mu_simpli, mu), mu1)
mu1
```

```
sigma_simpli = p * ( (mu1**2) + (sigma1**2)) + (1-p) * ((mu2**2) + (sigma2**2))
sigma1 = solve(Eq(sigma_simpli, sigma), sigma1)
sigma1
```

Now we can calculate sigma1 and mu1 with the simplified versions of mu and sigma

```
sigma1 = solve(Eq(sigma_simpli, sigma), sigma1)
mu1 = solve(Eq(mu_simpli, mu), mu1)
othersigma =  sigma1 = ( sigma / p) ** (1 / sqrt(2))/2
sigma1, othersigma
```

Let's try it with a mixture of two gaussians:

```
pdf = p * (1 / sigma1) * (exp((-1/2)*((x-mu1)/sigma1)**2)/sqrt(2* pi)) + (1-p) * (1 / sigma2) * (exp((-1/2)*((x-mu2)/sigma2)**2)/sqrt(2* pi))
pdf
```

```
cdf = integrate(pdf,x)
```

```
cdf = simplify(cdf)
cdf
```

```
f = pdf

```

```
mu2 = (1/2) * (a+b)
```

```
sigma2 = (1/12)*(b-a)**2
```

```
delta1 = mu1 - mu
delta2 = mu2 - mu
```

```
mean = p* mu1 + (1-p)*mu2
mu1 = mean
```

```
variance_simpli = p * (mu1**2 + sigma1**2) + (1-p) * (mu2**2 + sigma2**2)
variance_simpli
```

```
sigma1 = solve(Eq(variance_simpli, sigma), sigma1)
```

```
sigma1, mu1
```

```

```


# Orbit games

We consider energy plots and orbital solutions in polar coordinates for the general potential energy

$\begin{align}
   U(r) = k r^n
\end{align}$

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
```

```python
# Change the common font size
font_size = 14
plt.rcParams.update({'font.size': font_size})
```

```python
class Orbit:
    """
    Potentials and associated differential equations for central force motion
    with the potential U(r) = k r^n.
    """

    def __init__(self, ang_mom, n, k=1, mu=1):
        self.ang_mom = ang_mom
        self.n = n
        self.k = k
        self.mu = mu

    def U(self, r):
        """Potential energy of the form U = kr^n."""
        return self.k * r**self.n

    def Ucf(self, r):
        """Centrifugal potential energy"""
        return self.ang_mom**2 / (2. * self.mu * r**2)

    def Ueff(self, r):
        """Effective potential energy"""
        return self.U(r) + self.Ucf(r)

    def U_deriv(self, r):
        """dU/dr"""
        return self.n * self.k * r**(self.n - 1)

    def Ucf_deriv(self, r):
        """dU_cf/dr"""
        return -2. * self.ang_mom**2 / (2. * self.mu * r**3)

    def Ueff_deriv(self, r):
        """dU_eff/dr"""
        return self.U_deriv(r) + self.Ucf_deriv(r)

    def dy_dt(self, t, y):
        """
        This function returns the right-hand side of the diffeq:
        [dr/dt d^2r/dt^2 dphi/dt]

        Parameters
        ----------
        t : float
            time
        y : float
            3-component vector with y[0] = r(t), y[1] = dr/dt, y[2] = dphi/dt

        """
        return [ y[1],
                -1./self.mu * self.Ueff_deriv(y[0]),
                self.ang_mom / (self.mu * y[0]**2) ]

    def solve_ode(self, t_pts, r_0, r_dot_0, phi_0,
                  abserr=1.0e-8, relerr=1.0e-8):
        """
        Solve the ODE given initial conditions.
        For now use odeint, but we have the option to switch.
        Specify smaller abserr and relerr to get more precision.
        """
        y = [r_0, r_dot_0, phi_0]
        solution = solve_ivp(self.dy_dt, (t_pts[0], t_pts[-1]),
                             y, t_eval=t_pts,
                             atol=abserr, rtol=relerr)
        r, r_dot, phi = solution.y
        return r, r_dot, phi

    def energy(self, t_pts, r, r_dot):
        """Evaluate the energy as a function of time"""
        return (self.mu/2.) * r_dot**2 + self.Ueff(r)
```

```python
def start_stop_indices(t_pts, plot_start, plot_stop):
    start_index = (np.fabs(t_pts-plot_start)).argmin()  # index in t_pts array
    stop_index = (np.fabs(t_pts-plot_stop)).argmin()  # index in t_pts array
    return start_index, stop_index
```

# Gravity: $n = -1$

```python
n = -1
k = -3.
ang_mom = 1.
o1 = Orbit(ang_mom, n=n, k=k, mu=1)

fig_2 = plt.figure(figsize=(7,5))
ax_2 = fig_2.add_subplot(1,1,1)

r_pts = np.linspace(0.001, 3., 200)
U_pts = o1.U(r_pts)
Ucf_pts = o1.Ucf(r_pts)
Ueff_pts = o1.Ueff(r_pts)

ax_2.plot(r_pts, U_pts, linestyle='dashed', color='blue', label='U(r)')
ax_2.plot(r_pts, Ucf_pts, linestyle='dotted', color='green', label='Ucf(r)')
ax_2.plot(r_pts, Ueff_pts, linestyle='solid', color='red', label='Ueff(r)')

ax_2.set_xlim(0., 3.)
ax_2.set_ylim(-5., 5.)
ax_2.set_xlabel('r')
ax_2.set_ylabel('U(r)')
ax_2.set_title(f'$n = {n},\ \ k = {k},\ \  l = {ang_mom}$')
ax_2.legend(loc='upper center')

ax_2.axhline(0., color='black', alpha=0.3)

fig_2.tight_layout()

fig_2.savefig('Gravitation_orbit_1.png')

```

## Orbit (time dependence)

We'll directly solve the equations for r(t) and phi(t).

```python
# Plotting time
t_start = 0.
t_end = 10.
delta_t = 0.01

t_pts = np.arange(t_start, t_end+delta_t, delta_t)

# Initial conditions
r_0 = 0.2
r_dot_0 = 0.
phi_0 = 0.0
r_pts, r_dot_pts, phi_pts = o1.solve_ode(t_pts, r_0, r_dot_0, phi_0)

c = o1.ang_mom**2 / (np.abs(o1.k) * o1.mu)
epsilon = c / r_0 - 1.
energy_0 = o1.mu/2. * r_dot_0**2 + o1.Ueff(r_0)
print(f'energy = {energy_0:.2f}')
print(f'eccentricity = {epsilon:.2f}')
```

```python
fig_4 = plt.figure(figsize=(8,8))

overall_title = 'Gravitational orbit:  ' + \
                rf' $n = {o1.n},$' + \
                rf' $k = {o1.k:.1f},$' + \
                rf' $l = {o1.ang_mom:.1f},$' + \
                rf' $r_0 = {r_0:.1f},$' + \
                rf' $\dot r_0 = {r_dot_0:.2f},$' + \
                rf' $\phi_0 = {phi_0:.2f}$' + \
                '\n'     # \n means a new line (adds some space here)
fig_4.suptitle(overall_title, va='baseline')

ax_4a = fig_4.add_subplot(2,2,1)
ax_4a.plot(t_pts, r_pts, color='black')
ax_4a.set_xlabel(r'$t$')
ax_4a.set_ylabel(r'$r$')
ax_4a.set_title('Time dependence of radius')

ax_4b = fig_4.add_subplot(2,2,2)
ax_4b.plot(t_pts, phi_pts/(2.*np.pi), color='black')
ax_4b.plot(t_pts, phi_pts/(2.*np.pi)%1, color='red')
ax_4b.set_xlabel(r'$t$')
ax_4b.set_ylabel(r'$\phi/2\pi$')
ax_4b.set_title(r'Time dependence of $\phi$')

ax_4c = fig_4.add_subplot(2,2,3)
ax_4c.plot(r_pts*np.cos(phi_pts), r_pts*np.sin(phi_pts), color='black')
ax_4c.set_xlabel(r'$x$')
ax_4c.set_ylabel(r'$y$')
ax_4c.set_aspect(1)
ax_4c.set_title('Cartesian plot')

ax_4d = fig_4.add_subplot(2,2,4, polar=True)
ax_4d.plot(phi_pts, r_pts, color='black')
ax_4d.set_title('Polar plot', pad=20.)

fig_4.tight_layout()

```

## Energy versus time

```python
E_tot_pts = o1.energy(t_pts, r_pts, r_dot_pts)
E_tot_0 = E_tot_pts[0]
E_tot_rel_pts = np.abs((E_tot_pts - E_tot_0)/E_tot_0)

print(f'    t        E_tot        rel. error')
for t, E_tot, E_tot_rel in zip(t_pts, E_tot_pts, E_tot_rel_pts):
    print(f'{t:8.5f}  {E_tot:8.5e}  {E_tot_rel:12.4e}')
```

```python
fig_5 = plt.figure(figsize=(6,6))

overall_title = 'Gravitational orbit:  ' + \
                rf' $n = {o1.n},$' + \
                rf' $k = {o1.k:.1f},$' + \
                rf' $l = {o1.ang_mom:.1f},$' + \
                rf' $r_0 = {r_0:.1f},$' + \
                rf' $\dot r_0 = {r_dot_0:.2f},$' + \
                rf' $\phi_0 = {phi_0:.2f}$' + \
                '\n'     # \n means a new line (adds some space here)
fig_5.suptitle(overall_title, va='baseline')

ax_5a = fig_5.add_subplot(1,1,1)
ax_5a.semilogy(t_pts, np.abs(E_tot_pts), color='black', label=r'$E(t)$')
ax_5a.semilogy(t_pts, E_tot_rel_pts, color='red', label=r'$\Delta E(t)$')
ax_5a.set_xlabel(r'$t$')
ax_5a.set_ylabel(r'Energy')
ax_5a.set_title('Change in energy with time')
ax_5a.legend(loc='center right')

fig_5.tight_layout()
```

```python

```

```python

```


# Finding the intersection of two functions

This is the **master** branch version. We suggest you first try the [**student**](https://github.com/stephenbeckr/numerical-analysis-class/blob/student/Demos/Ch2_Intersection_GraphingCalculator.ipynb) branch version, which has open-ended questions.

By the way, if the "Open in Colab" link doesn't work, try right-clicking the link and choose "open in new tab".

### Finding intersections of two functions
This is something you might have done in high school on a graphing calculator (the particular functions we're using are inspired by this [post about using the TI-84+](https://www.dummies.com/education/graphing-calculators/how-to-find-points-of-intersection-on-the-ti-84-plus/)).

Let's find the intersection of
$$ f(x) = .2(x-2)x(x+4)  \quad\text{and}\quad g(x) = .5 x $$

First, let's plot the functions:
(for plotting, here's a list of [style sheets](https://matplotlib.org/3.3.1/gallery/style_sheets/style_sheets_reference.html) to use with `plt.style.use`, and [general matplotlib cheatsheets](https://github.com/matplotlib/cheatsheets); if you want to make the axes through the origin, like in the TI-84 picture, this [stackoverflow post](https://stackoverflow.com/a/25689340) gives several methods)

```
import numpy as np

# basic plots
%matplotlib inline

# allows interative plots, so you can zoom in!
# However, can be a bit buggy
#%matplotlib notebook

#%matplotlib ipympl # see https://medium.com/@1522933668924/using-matplotlib-in-jupyter-notebooks-comparing-methods-and-some-tips-python-c38e85b40ba1

import matplotlib.pyplot as plt

plt.style.use('seaborn-notebook') # just a personal preference for the style

# == First, define the function f
f = lambda x : .2*(x-2)*x*(x+4)

# == Now, define g
g = lambda x : .5*x

# == Now, plot

x   = np.linspace(-10,10,100)
```

```
fig, ax = plt.subplots()
ax.plot( x, f(x) )
ax.plot( x, g(x) )
ax.set_ylim(-10,10)
plt.xticks(range(-10,11,5))
plt.yticks(range(-10,11,5))

# Adjust it to look a bit more like the TI-84 image
# Alternatively, see https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.axvline
ax.spines['left'].set_position('zero')
ax.spines['right'].set_color('none')
ax.spines['bottom'].set_position('zero')
ax.spines['top'].set_color('none')
ax.minorticks_on()
fig.show() # Optional in jupyter
```

## Now, let's find all the roots (it seems we have three of them, looking at the plots).

Use the builtin root finder from `scipy` (see [`scipy.optimize` documentation](https://docs.scipy.org/doc/scipy/reference/optimize.html#root-finding)) called `root_scalar`

You can either specify the method you want, or it will pick a default method for you, based on what you supply, e.g.,
- if you supply an **interval** containing the root, using `bracket = [a,b]`, then it will use one of Brent's methods; you can also request using the bisection method
- if you supply an **initial guess** using `x0 = 1.2` (for example), then it will try something like Newton's method. However, you need to give it a bit more information:
  - to actually use **Newton's method** you have to specify a function that computes the derivative, using `fprime = [insert name-of-derivative-function]`
  - you can use the **secant method** which doesn't need the derivative, but then you have to specify a second initial point via `x1 = 1.3` (for example)

```
from scipy.optimize import root_scalar

output = root_scalar( lambda x : f(x) - g(x), x0 = -4, x1 = -4.1)
r1     = output.root
print('Found first root at x={} and y={}\n'.format(r1,f(r1)) ) # see https://pyformat.info/ for print statement formatting
output
```

    Found first root at x=-4.3911649915626345 and y=-2.19558249578132

          converged: True
               flag: 'converged'
     function_calls: 7
         iterations: 6
               root: -4.3911649915626345

```
output = root_scalar( lambda x : f(x) - g(x), x0 = 0, x1 = .1)
r2     = output.root
print('Found second root at x={} and y={}\n'.format(r2,f(r2)) ) # see https://pyformat.info/ for print statement formatting
output
```

    Found second root at x=0.0 and y=-0.0

          converged: True
               flag: 'converged'
     function_calls: 3
         iterations: 2
               root: 0.0

```
output = root_scalar( lambda x : f(x) - g(x), x0 = 2.1, x1 = 2.2)
r3     = output.root
print('Found third root at x={} and y={}\n'.format(r3,f(r3)) ) # see https://pyformat.info/ for print statement formatting
output
```

    Found third root at x=2.3911649915626345 and y=1.1955824957813186

          converged: True
               flag: 'converged'
     function_calls: 7
         iterations: 6
               root: 2.3911649915626345

```
ax.plot( r1, f(r1), 'o',MarkerSize=12 )
ax.plot( r2, f(r2), 'o',MarkerSize=12 )
ax.plot( r3, f(r3), 'o',MarkerSize=12 )
fig
```

### Compare accuracy with `roots`

Be careful: despite a similar sounding name, [`numpy.roots`](https://numpy.org/doc/stable/reference/generated/numpy.roots.html) is **only for polynomials**.

With a bit of pen-and-paper manipulation, you can cast the above intersection problem as one of finding the roots of a polynomial.  Do this, and then find the roots via `numpy.roots`.

- How can you evaluate the accuracy of your answer?
- Which method (`scipy.optimize.root_scalar` vs `numpy.roots`) is more accurate?

```
# Make the polynomial we want to find roots of (i.e., find the coefficients)

coeff = np.poly( (2,0,-4) )  # coefficients of (x-2)*x*(x+4)
# and don't forget to multiply by 0.2
coeff = 0.2*coeff
# and subtract the g(x) function, so -.5*x.  This is the 2nd-to-last coefficient
coeff[-2] -=.5
coeff
```

    array([ 0.2,  0.4, -2.1,  0. ])

```
r_numpy = np.roots( coeff )
r_numpy
```

    array([-4.39116499,  2.39116499,  0.        ])

```
r_scipy = np.array( (r1,r3,r2 ))  # changed ordering to match np.roots output

residual = lambda x : abs( f(x) - g(x) )
for r_n, r_s in zip(r_numpy,r_scipy):
    print("At root {:5.2f}, the residual using np.roots is {:.2e}, and using root_scalar is {:.2e}".format( r_n, residual(r_n), residual(r_s)) )
```

    At root -4.39, the residual using np.roots is 2.66e-15, and using root_scalar is 2.66e-15
    At root  2.39, the residual using np.roots is 2.22e-16, and using root_scalar is 1.33e-15
    At root  0.00, the residual using np.roots is 0.00e+00, and using root_scalar is 0.00e+00

Conclusion: in this example, both root finding libraries work just fine

## Let's repeat, for a different function

Again, let's pick a polynomial since we can find the true roots by hand

Let's do $$ f(x) = (x-1) \cdot (x-(1+10^{-8}))^2. $$

Do we expect any conditioning issues?

Yes!  This is almost a double-root at $x=1$, so very shallow slope.  If we perturb $f$ upward a bit, it won't have any roots at all!

### Try to find the roots using the polynomial root-finding routines (`np.roots`)

```

trueRoots = (1,1+1e-8)
coeff = np.poly( trueRoots  )
f     = lambda x : np.polyval( coeff, x )

r_numpy = np.roots( coeff )
r_numpy
```

    array([1.+1.49011612e-08j, 1.-1.49011612e-08j])

How did it go?

We got imaginary numbers! This means there were no real roots. But there are! It's just a hard problem, and very ill-conditioned.

The plot of the function will help us

```
fig, ax = plt.subplots(1,2);
x = np.linspace(0,2,100)
ax[0].plot( x, f(x) );

# Zoom in a bit
x = np.linspace(1-1e-8,1+5e-8,100);
ax[1].plot( x, f(x) );
ax[1].spines['bottom'].set_position('zero')
ax[1].spines['top'].set_color('none')
ax[1].spines['right'].set_color('none')
f( x[9] )

# from sympy import init_printing
# init_printing()
# list(zip( range(20), f( x[:20] ).tolist() ))

for xx, fx in zip( x[:20], f( x[:20] ) ):
  print("At x = 1 + {:5.2e}, f(x) is {:5.2e}".format(xx-1,fx) )
```

The table of values shows us the problem!  Within our limited precision, **many** values of x give us a root, since the nearest floating point number is $0$

## Let's try finding the roots via a generic rootfinding method (i.e., ignoring the fact that we have a polynomial)

Call [`root_scalar`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.root_scalar.html#scipy.optimize.root_scalar).

```
coeff_deriv = np.polyder( coeff )
deriv   = lambda x : np.polyval( coeff_deriv, x )
root_scalar( f, x0 = 1.1, fprime = deriv )
```

          converged: True
               flag: 'converged'
     function_calls: 46
         iterations: 23
               root: 1.0000000162008909

Let's repeat, but with a different evaluation of $f$ (taking advantage that we already know it in factored form)

```
f       = lambda x : (x-trueRoots[0])*(x-trueRoots[1]) # different
coeff_deriv = np.polyder( coeff )
deriv   = lambda x : np.polyval( coeff_deriv, x )
out = root_scalar( f, x0 = 1.1, fprime = deriv )
out
```

          converged: True
               flag: 'converged'
     function_calls: 46
         iterations: 23
               root: 1.0000000176119155

```
# What is the relative error?
abs( out.root - trueRoots[1] )/abs(trueRoots[1])
```

```
# We have about 8 digits of accuracy, roughly
```


```python
from varform1D import *
import sympy as sym
import numpy as np

def phi_factory(name, N, highest_derivative=2):
    """
    Generate N+1 basis functions (phi) on [0,1] which vanish
    on the boundary. Differentiate the functions up to
    (and including) highest_derivative.
    """
    x = sym.Symbol('x')
    from sympy import sin, cos, pi
    if name == 'sines':
        phi = {0: [sin(pi*(i+1)*x) for i in range(N+1)]}
    elif name == 'poly':
        phi = {0: [x**(i+1)*(1-x) for i in range(N+1)]}
    elif name == 'poly2':
        phi = {0: [x**(i+2)*(1-x) for i in range(N+1)]}
    elif name == 'poly3':
        phi = {0: [(1-x)**(i+1) for i in range(N+1)]}
    elif name == 'Lagrange':
        from Lagrange import Lagrange_polynomials
        # Return all Lagrange polynomials and strip off
        # boundary polynomials in application code (if necessary)
        phi = {0: Lagrange_polynomials(x, N, [0,1], 'uniform')[0]}
    elif name == 'Lagrange_Cheb':
        from Lagrange import Lagrange_polynomials
        phi = {0: Lagrange_polynomials(x, N, [0,1], 'Chebyshev')}

    # Compute derivatives of the basis functions
    for d in range(1, highest_derivative+1):
        phi[d] = [sym.diff(phi[0][i], x, d) for i in range(len(phi[0]))]
    return phi

def case0(f, N=3):
    B = 1 - x**3
    dBdx = sym.diff(B, x)

    # Compute basis functions and their derivatives
    phi = {0: [x**(i+1)*(1-x) for i in range(N+1)]}
    phi[1] = [sym.diff(phi_i, x) for phi_i in phi[0]]

    def integrand_lhs(phi, i, j):
        return phi[1][i]*phi[1][j]

    def integrand_rhs(phi, i):
        return f*phi[0][i] - dBdx*phi[1][i]

    Omega = [0, 1]

    u_bar, _ = solver(integrand_lhs, integrand_rhs, phi, Omega,
                      verbose=True, symbolic=True)
    u = B + u_bar
    print(('solution u:', sym.simplify(sym.expand(u))))

    # Calculate analytical solution

    # Solve -u''=f by integrating f twice
    f1 = sym.integrate(f, x)
    f2 = sym.integrate(f1, x)
    # Add integration constants
    C1, C2 = sym.symbols('C1 C2')
    u_e = -f2 + C1*x + C2
    # Find C1 and C2 from the boundary conditions u(0)=0, u(1)=1
    s = sym.solvers.solve([u_e.subs(x,0) - 1, u_e.subs(x,1) - 0], [C1, C2])
    # Form the exact solution
    u_e = -f2 + s[C1]*x + s[C2]
    print(('analytical solution:', u_e))
    #print 'error:', u - u_e  # many terms - which cancel
    print(('error:', sym.expand(u - u_e)))

def case1(N, basis='sines'):
    """
    Solve -u''=f(x) on [0,1], u(0)=u(1)=0. f(x)=2.
    Method: least-squares, Galerkin, collocation.
    """

    f = 2
    D = 0; L = 1

    def integrand_lhs_LS(phi, i, j):
        return -phi[2][i]*phi[2][j]

    def integrand_rhs_LS(phi, i):
        return f*phi[2][i]

    def integrand_lhs_G2(phi, i, j):
        return -phi[0][i]*phi[2][j]

    def integrand_rhs_G2(phi, i):
        return f*phi[0][i]

    def integrand_lhs_G1(phi, i, j):
        return phi[1][i]*phi[1][j]

    def integrand_rhs_G1(phi, i):
        return f*phi[0][i]

    def term_lhs_co(phi, x, i, j):
        return -phi[2][j](x[i])

    def term_rhs_co(phi, x, i):
        return 2

    Omega = [0, 1]
    phi = phi_factory(basis, N, 2)
    u = {}  # dict of solutions corresponding to different methods
    u['LS'] = solver(integrand_lhs_LS, integrand_rhs_LS, phi, Omega)[0]
    u['G1'] = solver(integrand_lhs_G1, integrand_rhs_G1, phi, Omega)[0]
    u['G2'] = solver(integrand_lhs_G2, integrand_rhs_G2, phi, Omega)[0]
    u['exact'] = x*(D + L - x)
    # Test different collocation points
    points = []
    if N == 0:
        points.extend([sym.Rational(1,2), 0.1, 0.9])
    else:
        points.append(np.linspace(0.1, 0.9, N+1))  # uniformly distributed
        for seed in 2, 10:
            np.random.seed(seed)
            points.append(np.random.uniform(0, 1, size=N+1))
    for k, p in enumerate(points):
        u['co'+str(k)] = collocation(term_lhs_co, term_rhs_co, phi, p)
    import pprint; pprint.pprint(u)
    comparison_plot(u, [0, 1])

def case2(N):
    """
    Solve -u''=f(x) on [0,1], u'(0)=C, u(1)=D.
    Method: Galerkin only.
    """
    x = sym.Symbol('x')
    f = 2
    D = 2; E = 3;
    L = 1  # basis function factory restricted to [0,1]
    D = sym.Symbol('D')
    C = sym.Symbol('C')

    # u exact
    f1 = sym.integrate(f, x)
    f2 = sym.integrate(f1, x)
    C1, C2 = sym.symbols('C1 C2')
    u = -f2 + C1*x + C2
    BC1 = sym.diff(u,x).subs(x, 0) - C
    BC2 = u.subs(x,1) - D
    s = sym.solve([BC1, BC2], [C1, C2])
    u_e = -f2 + s[C1]*x + s[C2]

    def diff_eq(u, x):
        eqs =  {'diff': -sym.diff(u, x, x) - f,
                'BC1': sym.diff(u, x).subs(x, 0) - C,
                'BC2': u.subs(x, L) - D}
        for eq in eqs:
            eqs[eq] = sym.simplify(eqs[eq])

    print(('Check of exact solution:', diff_eq(u_e, x)))

    def integrand_lhs(phi, i, j):
        return phi[1][i]*phi[1][j]

    B = D*x/L
    dBdx = sym.diff(B, x)

    def integrand_rhs(phi, i):
        return f*phi[0][i] - dBdx*phi[1][i]

    boundary_lhs = None # not used here

    def boundary_rhs(phi, i):
        return -C*phi[0][i].subs(x, 0)

    Omega = [0, L]
    phi = phi_factory('poly3', N, 1)
    #phi = phi_factory('Lagrange', N, 1)
    print((phi[0]))
    u = {'G1': solver(integrand_lhs, integrand_rhs, phi, Omega,
                      boundary_lhs, boundary_rhs, verbose=True)[0] + B,
         'exact': u_e}
    print(('numerical solution:', u['G1']))
    print(('simplified:', sym.simplify(u['G1'])))
    print(('u exact', u['exact']))
    # Change from symblic to numerical computing for plotting.
    # That is, replace C and D symbols by numbers
    # (comparison_plot takes the expressions with x to functions of x
    # so C and D must have numbers).
    for name in u:
        u[name] = u[name].subs(C, 2).subs(D, -2)
    print(('u:', u))
    C = 2;  D = -2  # Note that these are also remembered by u_e
    comparison_plot(u, [0, 1])

def case3(N, a=1, a_symbols={}, f=0, f_symbols={},
          basis='poly', symbolic=True, B_type='linear'):
    """
    Solve -(a(x)u)'=0 on [0,1], u(0)=1, u(1)=0.
    Method: Galerkin.
    a and f must be sympy expressions with x as the only symbol
    (other symbols in a gives very long symbolic expressions in
    the solution and is of little value).
    """
    # Note: a(x) with symbols
    #f = sym.Rational(10,7)  # for a=1, corresponds to f=0 when a=1/(2+10x)

    """
    def a(x):  # very slow
        return sym.Piecewise((a1, x < sym.Rational(1,2)),
                            (a2, x >= sym.Rational(1,2)))

    def a(x):  # cannot be treated by sympy or wolframalpha.com
        return 1./(a1 + a2*x)

    # Symbolic a(x) makes large expressions...
    def a(x):
        return 2 + b*x
    def a(x):
        return 1/(2 + b*x)

    def a(x):
        return 1/(2 + 10*x)

    b = sym.Symbol('b')
    def a(x):
        return sym.exp(b*x)
    """
    if f == 0:
        h = sym.integrate(1/a, x)
        h1 = h.subs(x, 1)
        h0 = h.subs(x, 0)
        u_exact = 1 - (h-h0)/(h1-h0)
    else:
        # Assume a=1
        f1 = sym.integrate(f, x)
        f2 = sym.integrate(f1, x)
        C1, C2 = sym.symbols('C1 C2')
        u = -f2 + C1*x + C2
        BC1 = u.subs(x,0) - 1
        BC2 = u.subs(x,1) - 0
        s = sym.solve([BC1, BC2], [C1, C2])
        u_exact = -f2 + s[C1]*x + s[C2]
    print(('u_exact:', u_exact))

    def integrand_lhs(phi, i, j):
        return a*phi[1][i]*phi[1][j]

    def integrand_rhs(phi, i):
        return f*phi[0][i] - a*dBdx*phi[1][i]

    boundary_lhs = boundary_rhs = None # not used here

    Omega = [0, 1]
    if B_type == 'linear':
        B = 1 - x
    elif B_type == 'cubic':
        B = 1 - x**3
    elif B_type == 'sqrt':
        B = 1 - sym.sqrt(x)
    else:
        B = 1 - x
    if basis == 'poly':
        phi = phi_factory('poly', N, 1)
    elif basis == 'Lagrange':
        phi = phi_factory('Lagrange', N, 1)
        print(('len phi:', len(phi)))
        B = phi[0][0]*1 + phi[0][-1]*0
        phi[0] = phi[0][1:-1]
        phi[1] = phi[1][1:-1]
    elif basis == 'sines':
        phi = phi_factory('sines', N, 1)
    else:
        raise ValueError('basis=%s must be poly, Lagrange or sines' % basis)
    print(('Basis functions:', phi[0]))

    dBdx = sym.diff(B, x)

    verbose = True if symbolic else False
    phi_sum, _ = solver(integrand_lhs, integrand_rhs, phi, Omega,
                        boundary_lhs, boundary_rhs, verbose=verbose,
                        symbolic=symbolic)
    print(('sum c_j*phi_j:', phi_sum))
    name = 'numerical, N=%d' % N
    u = {name: phi_sum + B, 'exact': sym.simplify(u_exact)}
    print(('Numerical solution:', u[name]))
    if verbose:
        print(('...simplified to', sym.simplify(u[name])))
        print(('...exact solution:', sym.simplify(u['exact'])))

    f_str = str(f).replace(' ', '')
    a_str = str(a).replace(' ', '')
    filename = 'DaDu=-%s_a=%s_N%s_%s.eps' % (f_str, a_str, N, basis)
    # Change from symblic to numerical computing for plotting.
    all_symbols = {}
    all_symbols.update(a_symbols)
    all_symbols.update(f_symbols)
    if all_symbols:
        for s in all_symbols:
            value = all_symbols[s]
            print(('symbol', s, 'gets value', value))
            u[name] = u[name].subs(s, value)
            u['exact'] = u['exact'].subs(s, value)
    print(('Numerical u_exact formula before plot:', u_exact))
    comparison_plot(u, [0, 1], filename)

def comparison_plot(u, Omega, filename='tmp.eps'):
    """
    Plot the solution u(x) (a sympy expression with x as the only
    symbol - all other symbols must have been substituted by numbers)
    and the exact solution u_e (which is a Python function that can
    take an array x and return the values of the exact solution).
    Omega is a 2-tuple/list with the domain's lower and upper limit.
    """
    x = sym.Symbol('x')
    resolution = 401
    xcoor = np.linspace(Omega[0], Omega[1], resolution)
    for name in u:
        u[name] = sym.lambdify([x], u[name], modules="numpy")
        u[name] = u[name](xcoor)
    legends = []
    for name in u:
        plt.plot(xcoor, u[name])
        legends.append(name)
    plt.legend(legends)
    plt.savefig(filename)

x, b = sym.symbols('x b')

#case1(8, 'sines')
#case2(1)
#case3(4)
#case3(N=3, a=sym.exp(b*x), f=0, basis='poly',
#      a_symbols={b: 8}, f_symbols={})
#case3(N=2, a=1, f=b, basis='poly', f_symbols={b: 10}, B_type='cubic')

#case0(f=b, N=1)
#case0(f=x**6, N=7)
case2(1)

```

```python

```

Data Science and Machine Learning/Machine-Learning-In-Python-THOROUGH/EXAMPLES/FINITE_ELEMENTS/INTRO/SRC/18_EX_VARFORM1D.ipynb


```python
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
import numpy as np
import matplotlib.pyplot as plt
from hottbox.core import Tensor
```

[Return to Table of Contents](./0_Table_of_contents.ipynb)

# Tensors and basic operations

A tensor is a multi-dimenaional array of data where each dimension is conventionally referred to as the **mode**. Tensor order is defined by the number of its modes which is equivivalent to the number of indices required to identify a particular entry of a multi-dimensional array. For ease of visualisation and compact notation, we consider a third order tensor $\mathbf{\underline{X}} \in \mathbb{R}^{I \times J \times K}$. Thus, an element of such a tensor, $\mathbf{\underline{X}} \in \mathbb{R}^{I \times J \times K}$, can be written in a general form as:

$$ x_{ijk} = \mathbf{\underline{X}}[i, j, k]$$

> **Note:** In order to be consistent with Python indexing, count of modes and elements within starts from zeros.

Creation of a tensor starts with a formation of a multidimensional array of data.

```python
# Create 3-d array of data
array_3d = np.arange(24).reshape((2, 3, 4))

# Create tensor
tensor = Tensor(array_3d)

# Result preview
print(tensor)
tensor.data
```

    This tensor is of order 3 and consists of 24 elements.
    Sizes and names of its modes are (2, 3, 4) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[[ 0,  1,  2,  3],
            [ 4,  5,  6,  7],
            [ 8,  9, 10, 11]],

           [[12, 13, 14, 15],
            [16, 17, 18, 19],
            [20, 21, 22, 23]]])

## **Assigment 1**

1. Create a tensor of order 5 (of any shape) and with all its values equal to 5. Then, calculate its Frobenious norm.
2. Construct several tensors of orders varying from 3 to 20, with the size of each dimension equal to 2. Plot the relation between an order of a tensor and the number of its elements.
3. Construct several tensors of order 3 with the size of each dimension varying from 3 to 15. Plot the relation between the size of dimensions of a tensor and the number of its elements.
4. Analyse and compare the so obtained graphs.

### Solution: Part 1

```python
# Create a tensor
array_5d = np.full((2, 3, 2, 3, 2), 5)

tensor_5 = Tensor(array_5d)

print(tensor_5)
tensor_5.data
```

    This tensor is of order 5 and consists of 72 elements.
    Sizes and names of its modes are (2, 3, 2, 3, 2) and ['mode-0', 'mode-1', 'mode-2', 'mode-3', 'mode-4'] respectively.

    array([[[[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]],

            [[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]],

            [[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]]],

           [[[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]],

            [[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]],

            [[[5, 5],
              [5, 5],
              [5, 5]],

             [[5, 5],
              [5, 5],
              [5, 5]]]]])

```python
# Calculate Frobenious norm
tensor_5.frob_norm
```

    42.42640687119285

### Solution: Part 2

```python
# Create tensors and collect information
elements = []
order = []
for i in range(3,21):
    lst = [2]*i
    array = np.full(lst, 1)
    tensor = Tensor(array)
    order.append(tensor.order)
    elements.append(tensor.size)
```

```python
# Plot results
plt.plot(order,elements)

plt.xlabel(r'Tensor Order',fontsize=16)
plt.ylabel(r'Number of Elements',fontsize=16)
plt.title(r"Number of Elements against Tensor order with size 2",fontsize=16)
```

### Solution: Part 3

```python
# Create tensors and collect information
size = []
ele = []
for i in range(3,17):
    array = np.full((i,i,i), 1)
    tensor = Tensor(array)
    size.append(i)
    ele.append(tensor.size)
```

```python
# Plot results
plt.plot(size,ele)

plt.xlabel(r'Tensor Order',fontsize=16)
plt.ylabel(r'Number of Elements',fontsize=16)
plt.title(r"Number of Elements against Tensor size with order 3",fontsize=16)
```

### Solution: Part 4

The number of elements against tensor order gives an exponential increase, whereas the number of elements against tensor size gives a more gradual increase. This is due to the fact that for the order, the number of elements in this case correspond to 2^(order). For the size, the number of elements is equal to (size x 3) as it is of order 3. Increasing the order, can drastically increase the number of elements present in the tensor.

# Most common tensor substructures

1. A **fiber** is a vector obtained by fixing all but one of the indices, e.g.  $\mathbf{\underline{X}}[i,:,k]$ is the mode-2 fiber $\mathbf{\underline{X}}$ (usually refered to as row fiber).

1. Fixing all but two of the indices yields a matrix called a **slice** of a tensor, e.g. $\mathbf{\underline{X}}[:,:,k]$ is the mode-[1,2] slice (usually refered to as the frontal slice).

**Note:** The same principles and definitions can be applied to a tensor of an arbitrarily large order. In addition, you can obtain a **subtensor** by fixing at least three indices and letting the remaining ones vary.

```python
# Mode-2 fiber
tensor.data[0,1,:]
```

    array([4, 5, 6, 7])

```python
# Frontal slice
tensor.data[0,:,:]
```

    array([[ 0,  1,  2,  3],
           [ 4,  5,  6,  7],
           [ 8,  9, 10, 11]])

## **Assigment 2**

1. Create a tensor of order 4 which consists of 81 elements and display all its horizontal slices and any five row fibers.
2. Create a tensor of order 3 and shape `(4,2,5)`, where the values of each consequent frontal slice are doubled.

### Solution: Part 1

```python
# Create a tensor of order 4
array_4d = np.arange(81).reshape((3, 3, 3, 3))
tensor_4 = Tensor(array_4d)
```

```python
# Display slices
for i in range(0,3):
    print(tensor_4.data[i,:,:,:])
```

    [[[ 0  1  2]
      [ 3  4  5]
      [ 6  7  8]]

     [[ 9 10 11]
      [12 13 14]
      [15 16 17]]

     [[18 19 20]
      [21 22 23]
      [24 25 26]]]
    [[[27 28 29]
      [30 31 32]
      [33 34 35]]

     [[36 37 38]
      [39 40 41]
      [42 43 44]]

     [[45 46 47]
      [48 49 50]
      [51 52 53]]]
    [[[54 55 56]
      [57 58 59]
      [60 61 62]]

     [[63 64 65]
      [66 67 68]
      [69 70 71]]

     [[72 73 74]
      [75 76 77]
      [78 79 80]]]

```python
# Display fibers
for p in range(0,3):
    print(tensor_4.data[1,:,2,p])

for i in range(0,2):
    print(tensor_4.data[1,:,i,2])
```

    [33 42 51]
    [34 43 52]
    [35 44 53]
    [29 38 47]
    [32 41 50]

### Solution: Part 2

```python
# Create a tensor of order 3
array3 = np.full((4,2,5), 1)
for k in range(1,5):
    array3[:,:,k] = 2*array3[3,1,k-1]

tensor3 = Tensor(array3)
for i in range(0,5):
    print(tensor3.data[:,:,i])
```

    [[1 1]
     [1 1]
     [1 1]
     [1 1]]
    [[2 2]
     [2 2]
     [2 2]
     [2 2]]
    [[4 4]
     [4 4]
     [4 4]
     [4 4]]
    [[8 8]
     [8 8]
     [8 8]
     [8 8]]
    [[16 16]
     [16 16]
     [16 16]
     [16 16]]

# Unfolding

Conventionally, unfolding is considered to be a process of element mapping from a tensor to a matrix. In other words, it arranges the mode-$n$ fibers of a tensor to be the matrix columns. Thus, the mode-$n$ unfolding is denoted as:

$$\mathbf{\underline{A}} \xrightarrow{n} \mathbf{A}_{(n)}$$

Thus, this operations requires to specify a mode along which a tensor will be unfolded. For a third order tensor, a visual representation of such operation is as following

In `hottbox` this functionality is available through the corresponding methods of the `Tensor` object, for instance:

```python
# Create 3-d array of data
array_3d = np.arange(24).reshape((2, 3, 4))

# Create tensor
tensor = Tensor(array_3d)

# Unfolding of a tensor
tensor.unfold(mode=0)

# Result preview
print(tensor)
tensor.data
```

    This tensor is of order 2 and consists of 24 elements.
    Sizes and names of its modes are (2, 12) and ['mode-0', 'mode-1_mode-2'] respectively.

    array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11],
           [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])

## **Assigment 3**

1. Create a tensor, $\mathbf{\underline{A}}$, of order 4, in such a way that its version unfolded along the second mode would have sizes `(5,72)`, that is $\mathbf{A}_{(n)} \in  \mathbb{R}^{5 \times 72}$. Validate your results by displaying the unfolded tensor.

### Solution : Part 1

```python
# Create a tensor
array4 = np.arange(360).reshape((8, 5, 3, 3))

tensor4 = Tensor(array4)

tensor4.unfold(mode=1)
```

    This tensor is of order 2 and consists of 360 elements.
    Sizes and names of its modes are (5, 72) and ['mode-1', 'mode-0_mode-2_mode-3'] respectively.

```python
# Display unfolded version
print(tensor4)
tensor4.data
```

    This tensor is of order 2 and consists of 360 elements.
    Sizes and names of its modes are (5, 72) and ['mode-1', 'mode-0_mode-2_mode-3'] respectively.

    array([[  0,   1,   2,   3,   4,   5,   6,   7,   8,  45,  46,  47,  48,
             49,  50,  51,  52,  53,  90,  91,  92,  93,  94,  95,  96,  97,
             98, 135, 136, 137, 138, 139, 140, 141, 142, 143, 180, 181, 182,
            183, 184, 185, 186, 187, 188, 225, 226, 227, 228, 229, 230, 231,
            232, 233, 270, 271, 272, 273, 274, 275, 276, 277, 278, 315, 316,
            317, 318, 319, 320, 321, 322, 323],
           [  9,  10,  11,  12,  13,  14,  15,  16,  17,  54,  55,  56,  57,
             58,  59,  60,  61,  62,  99, 100, 101, 102, 103, 104, 105, 106,
            107, 144, 145, 146, 147, 148, 149, 150, 151, 152, 189, 190, 191,
            192, 193, 194, 195, 196, 197, 234, 235, 236, 237, 238, 239, 240,
            241, 242, 279, 280, 281, 282, 283, 284, 285, 286, 287, 324, 325,
            326, 327, 328, 329, 330, 331, 332],
           [ 18,  19,  20,  21,  22,  23,  24,  25,  26,  63,  64,  65,  66,
             67,  68,  69,  70,  71, 108, 109, 110, 111, 112, 113, 114, 115,
            116, 153, 154, 155, 156, 157, 158, 159, 160, 161, 198, 199, 200,
            201, 202, 203, 204, 205, 206, 243, 244, 245, 246, 247, 248, 249,
            250, 251, 288, 289, 290, 291, 292, 293, 294, 295, 296, 333, 334,
            335, 336, 337, 338, 339, 340, 341],
           [ 27,  28,  29,  30,  31,  32,  33,  34,  35,  72,  73,  74,  75,
             76,  77,  78,  79,  80, 117, 118, 119, 120, 121, 122, 123, 124,
            125, 162, 163, 164, 165, 166, 167, 168, 169, 170, 207, 208, 209,
            210, 211, 212, 213, 214, 215, 252, 253, 254, 255, 256, 257, 258,
            259, 260, 297, 298, 299, 300, 301, 302, 303, 304, 305, 342, 343,
            344, 345, 346, 347, 348, 349, 350],
           [ 36,  37,  38,  39,  40,  41,  42,  43,  44,  81,  82,  83,  84,
             85,  86,  87,  88,  89, 126, 127, 128, 129, 130, 131, 132, 133,
            134, 171, 172, 173, 174, 175, 176, 177, 178, 179, 216, 217, 218,
            219, 220, 221, 222, 223, 224, 261, 262, 263, 264, 265, 266, 267,
            268, 269, 306, 307, 308, 309, 310, 311, 312, 313, 314, 351, 352,
            353, 354, 355, 356, 357, 358, 359]])

# Mode-n product

The mode-$n$ product is a multiplication of a tensor  by a matrix along the $n^{th}$ mode of a tensor. This essentially means that each mode-$n$ fiber should be multiplied by this matrix. Mathematically, this can be expressed as:

$$\mathbf{\underline{X}} \times_n \mathbf{A} = \mathbf{\underline{Y}} \quad \Leftrightarrow  \quad \mathbf{Y}_{(n)} = \mathbf{A} \mathbf{X}_{(n)}  $$

```python
I, J, K = 2, 3, 4
J_new = 5

# Create tensor
array_3d = np.arange(I * J * K).reshape(I, J ,K)
X = Tensor(array_3d)

# Create matrix
A = np.arange(J_new * J).reshape(J_new, J)

# Performs Mode-n product
Y = X.mode_n_product(A, mode=1, inplace=False)

# Preview of resulting tensor
print(Y)

# Preview of its unfolded version
Y.unfold(mode=1, inplace=False).data
```

    This tensor is of order 3 and consists of 40 elements.
    Sizes and names of its modes are (2, 5, 4) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    40

## **Assignment 4**

1. Perform the mode-$n$ product of a 3-rd order tensor, $\mathbf{\underline{X}}$, with a matrix, $\mathbf{A} \in  \mathbb{R}^{5 \times 4}$, provided that the resulting tensor, $\mathbf{\underline{Y}}$, consists of `100` elements. Validate your results.

> **Note:** You are encouraged to experiment with the optional parameter `inplace` and to investigate its effect on the result.

### Solution: Part 1

```python
# Create original tensor
I, J, K = 10, 4, 2

array_3d = np.arange(I * J * K).reshape(I, J ,K)
X = Tensor(array_3d)
```

```python
# Create matrix
J_new = 5
A = np.arange(J_new * J).reshape(J_new, J)
```

```python
# Perform mode-n product
Y = X.mode_n_product(A, mode=1, inplace=False)
```

```python
# Validate equivalence between the left and right hand side
print(Y)
Y.unfold(mode=1, inplace=False).data
```

    This tensor is of order 3 and consists of 100 elements.
    Sizes and names of its modes are (10, 5, 2) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[  28,   34,   76,   82,  124,  130,  172,  178,  220,  226,  268,
             274,  316,  322,  364,  370,  412,  418,  460,  466],
           [  76,   98,  252,  274,  428,  450,  604,  626,  780,  802,  956,
             978, 1132, 1154, 1308, 1330, 1484, 1506, 1660, 1682],
           [ 124,  162,  428,  466,  732,  770, 1036, 1074, 1340, 1378, 1644,
            1682, 1948, 1986, 2252, 2290, 2556, 2594, 2860, 2898],
           [ 172,  226,  604,  658, 1036, 1090, 1468, 1522, 1900, 1954, 2332,
            2386, 2764, 2818, 3196, 3250, 3628, 3682, 4060, 4114],
           [ 220,  290,  780,  850, 1340, 1410, 1900, 1970, 2460, 2530, 3020,
            3090, 3580, 3650, 4140, 4210, 4700, 4770, 5260, 5330]])

# Properties of mode-n product

1. For distinct modes in a series of multiplications, the order of the multiplication is irrelevent:

    $$\mathbf{\underline{X}} \times_n \mathbf{A} \times_m \mathbf{B} = \mathbf{\underline{X}} \times_m \mathbf{B} \times_n \mathbf{A} \quad (m \neq n)$$

1. However, this does not hold if the modes are the same :

    $$\mathbf{\underline{X}} \times_n \mathbf{A} \times_n \mathbf{B} = \mathbf{\underline{X}} \times_n (\mathbf{B}\mathbf{A})$$

## **Assignment 5**

1. Validate both of the above properties empirically
2. Prove the second property analytically

> **Note:** Be aware of consequences of the optional parameter `inplace` (its default value is set to `True`).

### Solution: Part 1

```python
# Create tensor
I, J, K = 2, 3, 4
J_new, K_new = 5, 6

array_3d = np.arange(I * J * K).reshape(I, J ,K)
X = Tensor(array_3d)
```

```python
# Create matrices
A = np.arange(J_new * J).reshape(J_new, J)
B = np.arange(K_new * K).reshape(K_new, K)

B1 = np.arange(K_new * J_new).reshape(K_new, J_new)
```

```python
# Validate property 1
Y1 = X.mode_n_product(A, mode=1, inplace=False).mode_n_product(B, mode=2, inplace=False)
Y2 = X.mode_n_product(B, mode=2, inplace=False).mode_n_product(A, mode=1, inplace=False)
np.testing.assert_array_equal(Y1.data, Y2.data)
print('The underlying data arrays are equal for all of them.')
```

    The underlying data arrays are equal for all of them.

```python
# Validate property 2
Y4 = X.mode_n_product(A, mode=1, inplace=False).mode_n_product(B1, mode=1, inplace=False)
Y5 = X.mode_n_product(np.dot(B1,A), mode=1, inplace=False)
np.testing.assert_array_equal(Y4.data, Y5.data)
print('The underlying data arrays are equal for all of them.')
```

    The underlying data arrays are equal for all of them.

### Solution: Part 2

Use [LaTeX syntax](https://stackoverflow.com/questions/13208286/how-to-write-latex-in-ipython-notebook) for analytical deriviations for the second property below.

```latex
%%latex
The n-mode product is defined as:
    \begin{equation}
           (\mathbf{\underline{X}} \times_{n} \mathbf{A}) = \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}}A_{ji_{n}}
    \end{equation}
Therefore we have:
\begin{equation}
    \mathbf{\underline{X}} \times_{n} \mathbf{A} \times_{n} \mathbf{B} = \sum_{j_{n} = 1}^{J_{N}}(\sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}}A_{j_{n}i_{n}})B_{kj_{n}} = \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}} \sum_{j_{n} = 1}^{J_{N}} A_{j_{n}i_{n}}B_{kj_{n}} =  \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}} \sum_{j_{n} = 1}^{J_{N}} B_{kj_{n}}A_{j_{n}i_{n}} =  \mathbf{\underline{X}} \times_{n} (\mathbf{BA})
\end{equation}
This corresonds to the affirmation of property 2
```

The n-mode product is defined as:
    \begin{equation}
           (\mathbf{\underline{X}} \times_{n} \mathbf{A}) = \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}}A_{ji_{n}}
    \end{equation}
Therefore we have:
\begin{equation}
    \mathbf{\underline{X}} \times_{n} \mathbf{A} \times_{n} \mathbf{B} = \sum_{j_{n} = 1}^{J_{N}}(\sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}}A_{j_{n}i_{n}})B_{kj_{n}} = \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}} \sum_{j_{n} = 1}^{J_{N}} A_{j_{n}i_{n}}B_{kj_{n}} =  \sum_{i_{n}=1}^{I_{N}} \underline{X}_{i_{1},i_{2}...i_{N}} \sum_{j_{n} = 1}^{J_{N}} B_{kj_{n}}A_{j_{n}i_{n}} =  \mathbf{\underline{X}} \times_{n} (\mathbf{BA})
\end{equation}

```python

```


# The Discrete Fourier Transform

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Fast Convolution

The linear convolution of signals is a basic building block in many practical applications. The straightforward convolution of two finite-length signals $x[k]$ and $h[k]$ has considerable numerical complexity. This has led to the development of various algorithms that realize the convolution with lower complexity. The basic concept of the *fast convolution* is to exploit the [convolution theorem](theorems.ipynb#Convolution-Theorem) of the discrete Fourier transform (DFT). This theorem states that the periodic convolution of two signals is equal to a scalar multiplication of their spectra. The scalar multiplication has considerably less numerical operations that the convolution. The transformation of the signals can be performed efficiently by the [fast Fourier transform](fast_fourier_transform.ipynb) (FFT).

Since the scalar multiplication of the spectra realizes a periodic convolution, special care has to be taken to realize a linear convolution in the spectral domain. The equivalence between linear and periodic convolution is discussed in the following.

### Equivalence of Linear and Periodic Convolution

The [linear convolution](../discrete_systems/linear_convolution.ipynb#Finite-Length-Signals) of a causal signal $x_L[k]$ of length $L$ with a causal signal $h_N[k]$ of length $N$ reads

\begin{equation}
y[k] = x_L[k] * h_N[k] = \sum_{\kappa = 0}^{L-1} x_L[\kappa] \; h_N[k - \kappa] = \sum_{\kappa = 0}^{N-1} h_N[\kappa] \; x_L[k - \kappa]
\end{equation}

The resulting signal $y[k]$ is of finite length $M = N+L-1$. Without loss of generality it is assumed in the following that $N \leq L$. The computation of $y[k]$ for $k=0,1, \dots, M-1$ requires $M \cdot N$ multiplications and $M \cdot (N-1)$ additions. The computational complexity of the convolution is consequently [on the order of](https://en.wikipedia.org/wiki/Big_O_notation) $\mathcal{O}(M \cdot N)$.

The [periodic (or circular/cyclic) convolution](https://en.wikipedia.org/wiki/Circular_convolution) of two signals $x_L[k]$ and $h_N[k]$ is defined as

\begin{equation}
x_L[k] \circledast_P h_N[k] = \sum_{\kappa=0}^{P-1} \tilde{x}_P[k - \kappa] \; \tilde{h}_P[\kappa]
\end{equation}

where $\circledast_P$ denotes the periodic convolution with period $P$. The periodic summations $\tilde{x}_P[k]$ of $x[k]$ and $\tilde{h}_P[k]$ of $h[k]$ with period $P$ are defined as

\begin{align}
\tilde{x}_P[k] &= \sum_{\nu = -\infty}^{\infty} x_L[\nu \cdot P + k] \\
\tilde{h}_P[k] &= \sum_{\nu = -\infty}^{\infty} h_N[\nu \cdot P + k]
\end{align}

The result of the circular convolution has a period of $P$.

To compute the linear convolution by a periodic convolution, one has to take care that the result of the linear convolution fits into one period of the periodic convolution. Hence, the periodicity has to be chosen as $P \geq M$ where $M = N+L-1$. This can be achieved by zero-padding of $x_L[k]$ to the total length $M$ resulting in the signal $x_M[k]$ of length $M$ which is defined as

\begin{equation}
x_M[k] = \begin{cases}
x_L[k] & \text{for } 0 \leq k < L \\
0 & \text{for } L \leq k < M
\end{cases}
\end{equation}

and similar for $h_N[k]$ resulting in the zero-padded signal $h_M[k]$ which is defined as

\begin{equation}
h_M[k] = \begin{cases}
h_N[k] & \text{for } 0 \leq k < N \\
0 & \text{for } N \leq k < M
\end{cases}
\end{equation}

Using these signals, the linear and periodic convolution are equivalent for the first $M$ samples $k = 0,1,\dots, M-1$

\begin{equation}
x_L[k] * h_N[k] = x_M[k] \circledast_M h_M[k]
\end{equation}

#### Example

The following example computes the linear, periodic and linear by periodic convolution of two signals $x[k] = \text{rect}_L[k]$ and $h[k] = \text{rect}_N[k]$.

```python
import numpy as np
import matplotlib.pyplot as plt
from tools import cconv
%matplotlib inline

L = 8  # length of signal x[k]
N = 10  # length of signal h[k]
P = 14 # periodicity of periodic convolution

# generate signals
x = np.ones(L)
h = np.ones(N)

# linear convolution
y1 = np.convolve(x, h, 'full')
# periodic convolution
y2 = cconv(x, h, int(P))
# linear convolution via periodic convolution
xp = np.append(x, np.zeros(N-1))
hp = np.append(h, np.zeros(L-1))
y3 = cconv(xp, hp, L+N-1)

# plot results
def plot_signal(x):
    plt.stem(x)
    plt.xlabel('$k$')
    plt.ylabel('$y[k]$')
    plt.xlim([0, N+L])
    plt.gca().margins(y=0.1)

plt.figure(figsize = (10, 8))
plt.subplot(3,1,1)
plot_signal(y1)
plt.title('Linear convolution')

plt.subplot(3,1,2)
plot_signal(y2)
plt.title('Periodic convolution with period $P=%d$'%P)

plt.subplot(3,1,3)
plot_signal(y3)
plt.title('Linear convolution as periodic convolution')
plt.tight_layout()
```

**Exercise**

* Change the lengths `L`, `N` and `P` and check how the results for the different convolutions change.

### The Fast Convolution Algorithm

Using the above derived equality of the linear and periodic convolution one can express the linear convolution $y[k] = x_L[k] * h_N[k]$ by the DFT as

$$ y[k] = \text{IDFT}_M \{ \; \text{DFT}_M\{ x_M[k] \} \cdot \text{DFT}_M\{ h_M[k] \} \; \} $$

The resulting algorithm is composed of the following steps

1. Zero-padding of the two input signals $x_L[k]$ and $h_N[k]$ to at least a total length of $M \geq N+L-1$

2. Computation of the DFTs $X[\mu]$ and $H[\mu]$ using a FFT of length $M$

3. Multiplication of the spectra $Y[\mu] = X[\mu] \cdot H[\mu]$

4. Inverse DFT of $Y[\mu]$ using an inverse FFT of length $M$

The algorithm requires two DFTs of length $M$, $M$ complex multiplications and one IDFT of length $M$. On first sight this does not seem to be an improvement, since one DFT/IDFT requires $M^2$ complex multiplications and $M \cdot (M-1)$ complex additions. The overall numerical complexity is hence in the order of $\mathcal{O}(M^2)$. The DFT can be realized efficiently by the [fast Fourier transformation](fast_fourier_transform.ipynb) (FFT), which lowers the number of numerical operations for each DFT/IDFT significantly. The actual gain depends on the particular implementation of the FFT. Many FFTs are most efficient for lengths which are a power of two. It therefore can make sense, in terms of the number of numerical operations, to choose $M$ as a power of two instead of the shortest possible length $N+L-1$. In this case, the numerical complexity of the radix-2 algorithm is on the order of $\mathcal{O}(M \log_2 M)$.

The introduced algorithm is known as *fast convolution* due to its computational efficiency when realized by the FFT. For real valued signals $x[k] \in \mathbb{R}$ and $h[k] \in \mathbb{R}$ the number of numerical operations can be reduced further by using a real valued FFT.

#### Example

The implementation of the fast convolution algorithm is straightforward. In the following example the fast convolution of two real-valued signals $x[k] = \text{rect}_L[k]$ and $h[k] = \text{rect}_N[k]$ is shown. The real valued FFT/IFFT is consequently used. Most implementations of the FFT include the zero-padding to a given length $M$, e.g as in `numpy` by `numpy.fft.rfft(x, M)`.

```python
L = 8  # length of signal x[k]
N = 10  # length of signal h[k]

# generate signals
x = np.ones(L)
h = np.ones(N)

# fast convolution
M = N+L-1
y = np.fft.irfft(np.fft.rfft(x, M)*np.fft.rfft(h, M))

# show result
plt.figure(figsize=(10, 3))
plt.stem(y)
plt.xlabel('k')
plt.ylabel('y[k]')
```

    Text(0, 0.5, 'y[k]')

### Benchmark

It was already argued that the numerical complexity of the fast convolution is considerably lower due to the usage of the FFT. As measure, the gain in terms of execution time with respect to the linear convolution is evaluated in the following. Both algorithms are executed for the convolution of two real-valued signals $x_L[k]$ and $h_N[k]$ of length $L=N=2^n$ for $n \in \mathbb{N}$. The length of the FFTs/IFFT was chosen as $M=2^{n+1}$. The results depend heavily on the implementation of the FFT and the hardware used. Note that the execution of the following cell may take some time.

```python
import timeit

n = np.arange(17)  # lengths = 2**n to evaluate
reps = 100  # number of repetitions for timeit

gain = np.zeros(len(n))
for N in n:
    length = 2**N
    # setup environment for timeit
    tsetup = 'import numpy as np; from numpy.fft import rfft, irfft; \
            x=np.random.randn(%d); h=np.random.randn(%d)' % (length, length)
    # direct convolution
    tc = timeit.timeit('np.convolve(x, x, "full")', setup=tsetup, number=reps)
    # fast convolution
    tf = timeit.timeit('irfft(rfft(x, %d) * rfft(h, %d))' % (2*length, 2*length), setup=tsetup, number=reps)
    # speedup by using the fast convolution
    gain[N] = tc/tf

# show the results
plt.barh(n, gain, log=True)
plt.plot([1, 1], [-1, n[-1]+1], 'r-')
plt.yticks(n, 2**n)
plt.xlabel('Gain of fast convolution')
plt.ylabel('Length of signals')
plt.title('Comparison of execution times between direct and fast convolution')
plt.grid()
```

**Exercise**

* For which lengths is the fast convolution faster than the linear convolution?
* Why is it slower below a given signal length?
* Is the trend of the gain as expected from above considerations?

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


# Characterization of Systems in the Time Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Analysis of a Damped Spring Pendulum

The damped [spring pendulum](https://en.wikipedia.org/wiki/Spring_pendulum) is an example for a mechanical system that can be modeled by a linear ordinary differential equation (ODE) with constant coefficients. In view of the theory of signals and systems it hence can be interpreted as a linear time-invariant (LTI) system. The mechanical properties of the damped spring pendulum are analyzed by using the theory of LTI systems. The underlying mechanical setup is depicted in the following

A compact mass $m$ is mounted on a spring with stiffness $k$ which is connected to the ground. A damper with viscous damping coefficient $c$ is mounted parallel to the spring to model the friction present in the system. It is assumed that the movement of the mass over time is restricted to the vertical axis, here denoted by $y$. It is further assumed that the mass is in its idle position for $t<0$. The pretension of the spring by the mass can be neglected this way. It is additionally assumed that the mass is not moving for $t<0$. Denoting the displacement of the mass over time with $y(t)$, these initial conditions are formulated as $y(t) = 0$ and $\frac{d y(t)}{dt} = 0$ for $t<0$.

The normalized values $m = 0.1$, $c = 0.1$, $k = 2.5$ are used for illustration in the following.

### Differential Equation

The differential equation of the mechanical system is derived by considering the force equilibrium at the mass

\begin{equation}
F_\text{S}(t) + F_\text{F}(t) + F_\text{I}(t) = F_\text{E}(t)
\end{equation}

where $F_\text{E}(t)$ denotes an external force acting onto the mass, the other forces are derived in the sequel. The force $F_\text{S}(t)$ induced by the spring is given by [Hooke's law](https://en.wikipedia.org/wiki/Hooke%27s_law)

\begin{equation}
F_\text{S}(t) = k y(t)
\end{equation}

Its common to model the frictional force $F_\text{F}(t)$ induced by the damper as being proportional to the velocity of the mass

\begin{equation}
F_\text{F}(t) = c \frac{d y(t)}{dt}
\end{equation}

The inertial force $F_\text{I}(t)$ due to the acceleration of the mass is given as

\begin{equation}
F_\text{I}(t) = m \frac{d^2 y(t)}{dt^2}
\end{equation}

Introducing the forces into the force equilibrium yields the differential equation describing the displacement of the damped spring pendulum

\begin{equation}
m \frac{d^2 y(t)}{dt^2} + c \frac{d y(t)}{dt} + k y(t) = F_\text{E}(t)
\end{equation}

as a consequence of the external force.
Above equation constitutes an ODE with constant coefficients. It can be interpreted as an LTI system with the external force as input signal $x(t) = F_\text{E}(t)$ and the displacement of the mass as output signal $y(t)$.

### Comparison to Passive Electrical Networks

Comparing the ODEs of the damped spring pendulum and the [second-order analog low-pass](http://localhost:8888/notebooks/systems_time_domain/network_analysis.ipynb#Differential-Equation) yields that both constitute second-order ODEs with constant coefficients. Dividing the ODE of the second-order analog low pass by $C$ results in

\begin{equation}
L \frac{d^2 u_\text{o}(t)}{dt^2} + R \frac{d u_\text{o}(t)}{dt} + \frac{1}{C} u_\text{o}(t) = \frac{1}{C} u_\text{i}(t)
\end{equation}

where $u_\text{i}(t)$ and $u_\text{o}(t)$ denote the in- and output voltage of the analog circuit.
Comparison with above ODE of the spring pendulum yields the [equivalence of both systems](https://en.wikipedia.org/wiki/System_equivalence) for

| | 2nd-order low-pass | spring pendulum |
|:---|:---|:---|
| input signal $x(t)$ | $u_\text{i}(t) = F_\text{E}(t) C$ | $F_\text{E}(t) = \frac{u_\text{i}(t)}{C}$ |
| output signal $y(t)$ | $u_\text{o}(t)$ | $y(t)$ |
|  | $L = m$ | $m = L$ |
|  | $R = c$ | $c = R$ |
|  | $C = \frac{1}{k}$ | $k = \frac{1}{C}$ |

Note, the equivalence between mechanical systems described by ODEs with constant coefficients and analog circuits was used to simulate such systems by [analog computers](https://en.wikipedia.org/wiki/Analog_computer).

### Impulse Response

The LTI system corresponding to the pendulum can be characterized by its [impulse response](impulse_response.ipynb) $h(t)$. It is defined as the output of the system for a Dirac Delta impulse $x(t) = \delta(t)$ at the input. Physically this can be approximated by hitting the mass (very shortly and forceful). The impulse response characterizes the movement $y(t)$ of the mass after such an event.

First the ODE of the spring pendulum is defined in `SymPy`

```python
import sympy as sym
%matplotlib inline
sym.init_printing()

t, m, c, k = sym.symbols('t m c k', real=True)
x = sym.Function('x')(t)
y = sym.Function('y')(t)

ode = sym.Eq(m*y.diff(t, 2) + c*y.diff(t) + k*y, x)
ode
```

The normalized values of the physical constants are stored in a dictionary for ease of later substitution

```python
mck = {m: 0.1, c: sym.Rational('.1'), k: sym.Rational('2.5')}
mck
```

The impulse response is calculated by explicit solution of the ODE.

```python
solution_h = sym.dsolve(
    ode.subs(x, sym.DiracDelta(t)).subs(y, sym.Function('h')(t)))
solution_h
```

The integration constants $C_1$ and $C_2$ have to be determined from the initial conditions $y(t) = 0$ and $\frac{d y(t)}{dt} = 0$ for $t<0$.

```python
integration_constants = sym.solve((solution_h.rhs.limit(
    t, 0, '-'), solution_h.rhs.diff(t).limit(t, 0, '-')), ['C1', 'C2'])
integration_constants
```

Substitution of the values for the integration constants $C_1$ and $C_2$ into the result from above yields the impulse response of the spring pendulum

```python
h = solution_h.subs(integration_constants)
h
```

The impulse response is plotted for the specific values of $m$, $c$ and $k$ given above

```python
sym.plot(h.rhs.subs(mck), (t, 0, 12), ylabel=r'h(t)');
```

### Transfer Function

For an exponential input signal $x(t) = e^{s t}$, the [transfer function](eigenfunctions.ipynb#Transfer-Function) $H(s)$ represents the weight of the exponential output signal $y(t) = H(s) \cdot e^{s t}$. The transfer function is derived by introducing $x(t)$ and $y(t)$ into the ODE and solving for $H(s)$

```python
s = sym.symbols('s')
H = sym.Function('H')(s)

H, = sym.solve(ode.subs(x, sym.exp(s*t)).subs(y, H*sym.exp(s*t)).doit(), H)
H
```

The transfer characteristic of an LTI system for harmonic exponential signals $e^{j \omega t} = \cos(\omega t) + j \sin(\omega t)$ is of special interest in the analysis of resonating systems. It can be derived from $H(s)$ by substituting the complex frequency $s$ with $s = j \omega$. The resulting transfer function $H(j \omega)$ provides the attenuation and phase the system adds to an harmonic input signal.

```python
w = sym.symbols('omega', real=True)

Hjw = H.subs(s, sym.I * w)
Hjw
```

The magnitude of the transfer function $|H(j \omega)|$ is plotted for the specific values of the elements given above

```python
sym.plot(abs(Hjw.subs(mck)), (w, -15, 15),
         ylabel=r'$|H(j \omega)|$', xlabel=r'$\omega$');
```

When inspecting the magnitude of the transfer function it becomes evident that the damped spring pendulum shows resonances (maxima) for two specific angular frequencies. These resonance frequencies $\omega_0$ are calculated by inspecting the extreme values of $|H(j \omega)|$. First the derivative of $|H(j \omega)|$ with respect to $\omega$ is computed and set to zero

```python
extrema = sym.solve(sym.Eq(sym.diff(abs(Hjw), w),0),w)
extrema
```

For the maxima of the transfer function only the 2nd and 3rd extrema are of interest

```python
w0 = extrema[1:3]
w0
```

The resonance frequencies are computed for the specific values of $m$, $c$ and $k$ given above

```python
[w00.subs(mck) for w00 in w0]
```

The phase of the transfer function $\varphi(j \omega)$ is computed and plotted for the specific values of the elements given above

```python
phi = sym.arg(Hjw)
sym.plot(phi.subs(mck), (w, -15, 15),
         ylabel=r'$\varphi(j \omega)$', xlabel=r'$\omega$');
```

**Exercise**

* Change the viscous damping coefficient $c$ of the spring pendulum and investigate how the magnitude and phase of the transfer function $H(j \omega)$ changes.

* How does the frequency of the damped harmonic oscillation in the impulse response relates to the resonance frequency?

### Application: Vibration Isolation

An application of above example is the design of [vibration isolation](https://en.wikipedia.org/wiki/Vibration_isolation) by a damped spring pendulum. A typical example is a rotating machinery with mass $m$ which has some sort of imbalance. Assuming that the imbalance can be modeled as a rotating mass, the external force $F_\text{E}(t)$ is given by the vertical component of its [centrifugal force](https://en.wikipedia.org/wiki/Centrifugal_force)

\begin{equation}
F_\text{E}(t) = F_0 \sin(\omega t) = F_0 \cdot \Im \{e^{j \omega t} \}
\end{equation}

where $\omega$ denotes the angular frequency of the rotating machinery and

\begin{equation}
F_0 = m_\text{I} r \omega^2
\end{equation}

the amplitude of the force with $m_\text{I}$ denoting the mass of the imbalance and $r$ the radius of its circular orbit. Since $e^{j \omega t}$ is an eigenfunction of the LTI system, the resulting displacement is then given as

\begin{equation}
y(t) = F_0 \cdot \Im \{e^{j \omega t} H(j \omega) \}
\end{equation}

The aim of vibration isolation is to keep the magnitude of the displacement as low as possible.

**Exercise**

* Compute and plot the displacement for given $m_\text{I}$ and $r$.

* For which angular frequencies $\omega$ is the magnitude of the displacement largest? How is the phase relation between the external force $F_\text{E}(t)$ and displacement $y(t)$ at these frequencies?

* How should the resonance frequencies $\omega_0$ of the spring pendulum be chosen in order to get a good vibration isolation for a machine rotating with angular frequency $\omega$? How is the phase relation between the external force $F_\text{E}(t)$ and displacement $y(t)$ at this frequency?

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.