# PHYS 5640 - Project 3
## Steven Stetzler

## The Fermi-Pasta-Ulam-Tsingou Nonlinear Lattice

We wish to use the method of **Velocity-Verlet Integration** to study the Fermi-Pasta-Ulam-Tsingou (FPTU) nonlinear chain problem. The FPTU problem consits of $N + 2$ particles, each with a mass $m$, on a $1D$ chain. Springs of spring constant $k$ connect each particle to its two adjacent neighbors. Each particle $n$ on this chain follows the following differential equation:

\begin{equation}
\tag{1}
m \ddot{u}_n = k \left( u_{n+1} + u_{n-1} - 2 u_n \right) + \alpha \left[ \left( u_{n+1} - u_n \right)^2 - \left( u_n - u_{n-1} \right)^2 \right]
\end{equation}

The endpoints of the chain are fixed: $u_0 = u_{N+1} = 0$ for all time $t$. $\alpha$ is a paramter that tells us how strong the non-linearity of the system is.

### (1) First consider the linear chain problem with $\alpha = 0$. Show that the normal modes of the chain are standing waves with wavevector $q = m\pi/(N + 1)$.

To show this, we first assume that, given the translational symmetry of the system, that the equations of motion of each of the particles is given by

$$
u_n(t) = A(t) e^{iqn}
$$

where $q$ is the wavevector. Plugging this solution into Eq. 1 (with $\alpha = 0$), we obtain

$$
m \ddot{A} e^{iqn} = k \left( Ae^{iq(n+1)} + Ae^{iq(n-1)} - 2 A e^{iqn} \right) \\
m \ddot{A} e^{iqn} = k A e^{iqn} \left( e^{iq} + e^{-iq} - 2 \right) \\
m \ddot{A} = 2 k A ( \cos(q)  - 1 )
$$

substituting in the trigometric identity

$$
2 \sin^2\left(\frac{q}{2}\right) = 1 - \cos(q)
$$

we obtain

$$
m \ddot{A} = - 4 k \sin^2\left(\frac{q}{2}\right) A \\
\ddot{A} = -4 \frac{k}{m} \sin^2\left(\frac{q}{2}\right) A \\
\ddot{A} = - w_q^2 A
$$

which can be solved easily for $A$:

$$
A = e^{i w_q t}
$$

where $w_q = 2 \sqrt{\frac{k}{m}} \sin\left(\frac{q}{2}\right)$ are the eigen-frequencies of the system. The eigen-modes of the system can be found by considering now solutions to $u_n$ of the form

\begin{equation}
\tag{2}
u_n(t) = A(t) e^{iqn} + B(t) e^{-iqn}
\end{equation}

which represent a forward and backward moving wave. We will now impose the boundary conditions of the system: $u_0(t) = u_{N+1}(t) = 0$

$$
u_0(t) = A(t) + B(t) = 0 \\
\rightarrow A(t) = -B(t)
$$

$$
u_{N+1}(t) = A(t) e^{iq(N+1)} + B(t) e^{-iq(N+1)} \\
u_{N+1}(t) = A(t) \left( e^{iq(N+1)} - e^{-iq(N+1)} \right) \\
u_{N+t}(t) = A(t) 2 i \sin(q(N+1)) = 0 \\
\rightarrow q(N+1) = m \pi \\
\rightarrow q = \frac{m\pi}{N+1}
$$

Plugging in what we've found from imposing boundary conditions to the solution given by Eq. 2 we obtain

$$
u_n^{(m)}(t) = A(t) 2 i \sin\left(\frac{mn\pi}{N+1} \right) \\
\rightarrow \xi_n^{(m)}(t) = \tilde{A}(t) \sin\left(\frac{mn\pi}{N+1} \right)
$$

giving the eigen-modes of the system $\xi_n^{(m)}$ which we can normalize by imposing the restriction that

$$
\sum_{n=1}^{N} \lvert \xi_n^{(m)} \rvert^2= 1 \\
\sum_{n=1}^{N} \lvert \tilde{A}(t) \sin\left(\frac{mn\pi}{N+1} \right) \rvert^2 = 1 \\
\rightarrow \lvert \tilde{A}(t) \rvert^2 = \frac{1}{\sum_{n=1}^{N} \sin^2\left(\frac{mn\pi}{N+1} \right)}
$$

Now, we must evaluate the sum $\sum_{n=1}^{N} \sin^2\left(\frac{mn\pi}{N+1} \right)$. First we will use the trigonometric identity $\sin^2(q) = \frac{1}{2} \left( 1 - \cos(2 q) \right)$ to obtain the sum:

$$
\frac{1}{2} \left( \sum_{n=1}^{N} 1 - \cos\left( \frac{2mn\pi}{N+1} \right) \right) = \frac{1}{2} \left( N - \sum_{n=1}^{N} \cos\left( \frac{2mn\pi}{N+1} \right) \right)
$$

which now only requires us to compute the sum $\sum_{n=1}^{N} \cos\left( \frac{2mn\pi}{N+1} \right)$. We can compute this sum by first looking at another sum which is similar:

$$
\sum_{k=0}^{n-1} \cos(2\pi f k) + i \sin(2 \pi f k) = \sum_{k=0}^{n-1} e^{2\pi i f k}
$$

The right hand side of this equation can be computed using the formula for a geometric series: $ \sum_{k=0}^{n-1} r^n = \frac{r^n - 1}{r - 1}$:

$$
\sum_{k=0}^{n-1} e^{2\pi i f k} = \frac{e^{2\pi i f n} - 1}{e^{2\pi i f} - 1} \\
= e^{\pi i f (n - 1)} \frac{e^{\pi i f n} - e^{-\pi i f n}}{e^{\pi i f} - e^{-\pi i f}} \\
= \left( \cos(\pi f (n - 1)) + i \sin(\pi f (n - 1)) \right) \frac{\sin(\pi f n)}{\sin(\pi f)}
$$

equating the real and imaginary parts in the expression above with the expression we started with we obtain

\begin{equation}
\tag{3}
\sum_{k=0}^{n-1} \cos(2\pi f k) = \cos(\pi f (n - 1)) \frac{\sin(\pi f n)}{\sin(\pi f)}
\end{equation}

We can make this result useful for us by expressing our sum as

$$
\sum_{n=1}^{N} \cos\left( \frac{2mn\pi}{N+1} \right) = \left( \sum_{n=0}^{N} \cos\left( 2 \pi f n  \right) \right) - 1
$$

by re-expressing $\frac{m}{N+1} = f$ and subtracting off the first term in the series (starting at $n = 0$). Quoting the result from Eq. 3 we obtain

$$
\sum_{n=1}^{N} \cos\left( \frac{2mn\pi}{N+1} \right) = \cos\left(\frac{mN\pi}{N+1} \right) \frac{\sin\left(m\pi\frac{N+1}{N+1}\right)}{\sin\left(\frac{m\pi}{N+1}\right)} - 1 = 0 - 1 = -1
$$

since the term $\sin\left(m\pi\frac{N+1}{N+1}\right) = \sin(m\pi) = 0 $ since $m \in Z$. Finally, this gives

$$
\sum_{n=1}^{N} \sin^2\left(\frac{mn\pi}{N+1} \right) = \frac{1}{2} \left( N - \sum_{n=1}^{N} \cos\left( \frac{2mn\pi}{N+1} \right) \right) = \frac{1}{2}\left(N + 1\right)
$$

This gives us the final normalization for $\xi_{n}^{(m)}$ of

$$
A = \frac{1}{\sqrt{\frac{1}{2}\left(N + 1\right)}} = \sqrt{\frac{2}{N+1}}
$$

giving the normalized eigen-modes with corresponding eigen-frequencies of

$$
\xi_n^{(m)} = \sqrt{\frac{2}{N+1}} \sin\left(\frac{mn\pi}{N+1} \right), \qquad w_q = 2 \sqrt{\frac{k}{m}} \sin\left(\frac{m\pi}{2\left(N+1\right)}\right)
$$

### (2)  Implement the Velocity-Verlet algorithm to solve the FPUT chain.

The Velocity-Verlet algorithm allows one to integrate equations of motion by computing the position, velocity, and acceleration of the particles at every time step of cadence $\Delta t$ of a simulation given only initial conditions for the particles $x(t=0)$, $\dot{x}(t = 0)$, and some function for the acceleration of the particles: $\ddot{x}(t)$. At the next time $t + \Delta t$ of a simulation, the position, velocity, and acceleration of a particle is given in terms of its current position, velocity, and acceleration:

\begin{equation}
\tag{4}
\overrightarrow{x}(t + \Delta t) = \overrightarrow{x}(t) + \overrightarrow{v}(t) \Delta t + \frac{1}{2} \overrightarrow{a}(t)\Delta t^2 \\
\overrightarrow{a}(t + \Delta t) = \ddot{\overrightarrow{x}}(t + \Delta t) \\
\overrightarrow{v}(t + \Delta t) = \overrightarrow{v}(t) + \frac{1}{2} \left(\overrightarrow{a}(t) + \overrightarrow{a}(t + \Delta t)  \right) \Delta t
\end{equation}

A generalized Velocity-Verlet is implemented in the function `velocity_verlet()`.

We simulate the motion of each of the particles in the FPUT chain using Velocity-Vertlet given the initial condition $u_n(t = 0) = \mathcal{A} \xi_n^{(1)} $ and $\dot{u}_n(t=0) = 0$ where $\mathcal{A}$ is a parameter that governs the strength of the initial perturbation of the chain. The acceleration of the particles is governed by Eq. 1. This initial condition represents pulling the chain into its first excited mode ($m = 1$) and then releasing it from rest. At each step of the simulation we compute both the *potential* and *kinetic* energies $E_p$ and $E_k$ as follows

\begin{equation}
\tag{5}
E_p = \sum_{n=0}^{N} \frac{k}{2} \left( u_{n+1} - u_n \right)^2, \qquad E_k = \sum_{n=1}^{N} \frac{m \dot{u}_n^2}{2}
\end{equation}

```python
# Import useful libraries
import numpy as np
from matplotlib import rcParams
import matplotlib.pyplot as plt
```

```python
# Plotting parameters
rcParams['figure.figsize'] = [12, 12]
rcParams['font.size'] = 24
rcParams['axes.labelsize'] = 28
rcParams['axes.labelpad'] = 15
```

```python
# A generalized velocity verlet that takes as arguments the initial position and velocities of a
# set of particles and evolves them on the time scale delta_t using the function acceleration
# This function computes the values of any observables at each time step passed to it using the funcs list
def velocity_verlet(x, v, delta_t, num_iter, acceleration=lambda x : 0, funcs=[], **kwargs):
    ret_vals = [[] for i in range(len(funcs))]
    x_now, v_now = x, v
    for i in range(num_iter):
        for func, ret_val in zip(funcs, ret_vals):
            ret_val.append(func(x_now, v_now, **kwargs))

        a_now = acceleration(x_now, **kwargs)

        x_next = x_now + v_now * delta_t + 0.5 * a_now * delta_t**2

        a_next = acceleration(x_next, **kwargs)

        v_next = v_now + 0.5 * (a_now + a_next) * delta_t

        x_now, v_now = x_next, v_next

        print("{:0.1f}% done      ".format(100 * (i + 1) / num_iter), end='\r')

    return ret_vals

# Compute the acceleration felt by each particle in the chain
def accel(u, k=1, m=1, alpha=0, **kwargs):
    a = np.zeros(len(u))
    # Particles to the right
    u_shift_up = np.roll(u, -1)
    # Particles to the left
    u_shift_down = np.roll(u, 1)

    # Eq. (1)
    a = (k * (u_shift_up + u_shift_down - 2 * u) + alpha * ((u_shift_up - u)**2 - (u - u_shift_down)**2)) / m

    # Impose boundary conditions
    a[0] = 0
    a[len(u) - 1] = 0

    return a

# Kinetic energy of the chain
# Eq. (5)
def KE(u, u_d, m=1, **kwargs):
    return 0.5 * m * np.sum((u_d**2))#[1:len(u) - 1])**2)

# Potential energy of the chain
# Eq. (5)
def PE(u, u_d, k=1, **kwargs):
    u_shift = np.roll(u, -1)
    return 0.5 * k * np.sum(((u_shift - u)**2))#[1:len(u) - 1])**2)
```

We will run a simulation of the FPUT chain using the following parameters: there $N = 36$ movable particles in the chain, $m = k = 1$, $\alpha = 0$ (the chain is linear), and the strength of the initial perturbation is $\mathcal{A} = 0.2$. The initial condition of the chain is $u_n(t = 0) = \mathcal{A} \xi_n^{(1)} $ and $\dot{u}_n(t=0) = 0$.

```python
def run_sim_one(N, A, k, m, delta_t, num_iter, **kwargs):
    u = A * np.sqrt(2 / (N + 1)) * np.sin(np.arange(N + 2) * np.pi / (N + 1))
    u[0] = 0
    u[N + 1] = 0
    u_d = np.zeros(N + 2)

    rets = velocity_verlet(u, u_d, delta_t, num_iter, acceleration=accel, funcs=[lambda x, v, **kwargs : x, lambda x, v, **kwargs : v, KE, PE], **kwargs)

    return rets
```

```python
N = 36
A = 0.2
k, m = 1, 1
w_1 = 2 * np.sqrt(k / m) * np.sin(1 * np.pi / (2 * (N + 1)))

delta_t = 0.01
num_iter = int(2 * 2 * np.pi / (w_1 * delta_t))
t_1 = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_one(N, A, k, m, delta_t, num_iter)

E_k_1 = np.array(ret_vals[2])
E_p_1 = np.array(ret_vals[3])
E_total_1 = E_k_1[0] + E_p_1[0]
E_error_1 = np.abs(E_total_1 - (E_k_1 + E_p_1)) / E_total_1
E_max_1 = np.max(E_error_1)
```

    100.0% done

We plot here the *potential* and *kinetic* energy of the chain as a function of time, exploring how energy oscillates in the system. We find that while energy is swapped between potential and kinetic, the total energy of the system remains constant.

```python
plt.plot(t_1, E_k_1, label='KE', color='c', lw=3)
plt.plot(t_1, E_p_1, label='PE', color='forestgreen', lw=3)
plt.plot(t_1, np.ones(len(E_k_1)) * (E_k_1[0] + E_p_1[0]), label='Total E', color='purple', lw=3)

plt.xlabel(r"$\omega^{(1)} t / 2 \pi$")
plt.ylabel(r"$E$", rotation=0)

plt.legend(loc='right', bbox_to_anchor=(1.25, .5), ncol=1, fancybox=True, shadow=True)
plt.show()
```

We observe that the energy of the system oscillates between potential and kinetic energy with a period of $\pi / \omega^{(1)}$.

Next, we explore how the energy of the system fluctuates on small timescales given a few different values of $\Delta t$. We explore the total energy fluctuation for $\Delta t = 0.01, 0.02, \text{ and } 0.04$.

```python
delta_t = 0.02
num_iter = int(4 * np.pi / (w_1 * delta_t))
t_2 = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_one(N, A, k, m, delta_t, num_iter)

E_k = np.array(ret_vals[2])
E_p = np.array(ret_vals[3])
E_total = E_k[0] + E_p[0]
E_error_2 = np.abs(E_total - (E_k + E_p)) / E_total
E_max_2 = np.max(E_error_2)
```

    100.0% done

```python
delta_t = 0.04
num_iter = int(4 * np.pi / (w_1 * delta_t))
t_3 = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_one(N, A, k, m, delta_t, num_iter)

E_k = np.array(ret_vals[2])
E_p = np.array(ret_vals[3])
E_total = E_k[0] + E_p[0]
E_error_3 = np.abs(E_total - (E_k + E_p)) / E_total
E_max_3 = np.max(E_error_3)
```

    100.0% done

```python
plt.plot(t_1, E_error_1, label=r'$\delta t = 0.01$', color='purple', lw=3)
plt.plot(t_2, E_error_2, label=r'    = 0.02', color='forestgreen', lw=3)
plt.plot(t_3, E_error_3, label=r'    = 0.04', color='c', lw=3)

plt.ylabel(r"$\left( E_0 - E \right)/E_0$")
plt.xlabel(r"$\omega^{(1)} t / 2 \pi$")

plt.legend(loc='right', bbox_to_anchor=(1.25, .90), ncol=1, fancybox=True, shadow=True)
plt.show()
```

We find that the error is of order $\lvert E_0 - E \rvert / E_0 \sim 10^{-6}$ and that the error oscillates with a period of $\pi/w^{(1)}$.

Next we explore how the maximum error of the energy scales with $\Delta t$ by finding the peak error within a simulation of the chain for values of $\Delta t = 0.01, 0.02, 0.04, 0.08, 0.16, 0.32, \text{ and } 0.64$.

```python
delta_ts = [0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64]
E_maxs = []
for delta_t in delta_ts:
    num_iter = int(4 * np.pi / (w_1 * delta_t))

    ret_vals = run_sim_one(N, A, k, m, delta_t, num_iter)

    E_k = np.array(ret_vals[2])
    E_p = np.array(ret_vals[3])
    E_total = E_k[0] + E_p[0]
    E_error = np.abs(E_total - (E_k + E_p)) / E_total
    E_max = np.max(E_error)
    E_maxs.append(E_max)
```

    100.0% done

```python
plt.loglog(delta_ts, 0.00175 * np.array(delta_ts)**2, linestyle='--', color='cyan', label=r'$C \times \Delta t^2$')
plt.loglog(delta_ts, E_maxs, linestyle='None', color='purple', marker='o', markersize=9)

plt.xlabel(r"$\Delta t$")
plt.ylabel(r"$\left(E_0 - E \right) / E_0$")

plt.legend(fancybox=True, shadow=True)
plt.show()
```

### (3) Linear chain with localized perturbations

Now, instead of exciting a single mode of the chain (previously just the $m = 1$ mode), we will excite the chain with some localized perturbation. Specifically, we will use the initial conditions $u_n(t=0) = 0$ and $\dot{u}_n(t=0) = \mathcal{B} \exp \left( \frac{- (n - N / 2)^2}{\sigma_0^2} \right)$ where $\mathcal{B}$ is a constant characterizing the strength of this perturbation and $\sigma_0$ characterizes the width of the perturbation. This initial perturbation is equivalent to having a chain at rest in its equilibrium position and then giving it a small, gaussian-shaped kick at the center of the chain. To investigate this perturbation, we define a "local energy" as

\begin{equation}\tag{6}
E_L(n) = \frac{m\dot{u}_n^2}{2} + \frac{k}{4} \left[ \left( u_{n+1} - u_n\right)^2 + \left( u_n - u_{n-1}\right)^2 \right]
\end{equation}

which measures both the kinetic and potential energy stored in particle $n$.

```python
# The local energy stored in the chain
# Eq. (6)
def LE(u, u_d, m=1, k=1, **kwargs):
    u_shift_up = np.roll(u, -1)
    u_shift_down = np.roll(u, 1)
    E = np.zeros(len(u))
    E = (m / 2) * u_d**2 + (k / 4) * ((u_shift_up - u)**2 + (u - u_shift_down)**2)
    E[0] = 0
    E[len(u) - 1] = 0
    return E

# Simulates a linear chain with a localized gaussian perturbation centered at the middle of the chain
def run_sim_two(N, B, k, m, sigma, delta_t, num_iter, **kwargs):
    u = np.zeros(N + 2)
    u_d = np.zeros(N + 2)

    u_d[1:N + 1] = B * np.exp( -(np.arange(N) - N / 2)**2 / sigma**2 )

    rets = velocity_verlet(u, u_d, delta_t, num_iter, acceleration=accel, funcs=[lambda x, v, **kwargs : x, lambda x, v, **kwargs : v, KE, PE, LE], **kwargs)

    return rets
```

We run a simulation of the chain with the initial perturbation given above with the paramters $N = 100$, $m = k = 1$, $\Delta_t = 0.04$, $\mathcal{B} = 0.8$, and $\sigma_0 = 10$.

```python
N = 100
B = 0.8
sigma = 10
m, k = 1, 1

delta_t = 0.04
num_iter = int(4 * np.pi / (w_1 * delta_t))
t = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_two(N, B, k, m, sigma, delta_t, num_iter)
```

    100.0% done

```python
E_l = np.array(ret_vals[4])

plt.plot(E_l[0], label=r'$\omega_1 t / 2 pi = {:0.0f}$'.format(0 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[125], label=r'     = {:0.3f}'.format(125 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[200], label=r'     = {:0.3f}'.format(200 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[300], label=r'     = {:0.3f}'.format(300 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[500], label=r'     = {:0.3f}'.format(500 * delta_t * w_1 / (2 * np.pi)))

plt.xlabel(r"$n$")
plt.ylabel(r"$E(n)$")

plt.legend(fancybox=True, shadow=True)
plt.show()
```

We find that the initial pertubation spreads into two symmetric waves moving to the left and to the right along the chain. These wavefronts move with velocity $v = \sqrt{k/m}$.

### (3 - bonus) Investigate how the energy spreads in a nonlinear FPU chain ($\alpha \neq 0$)

We can also explore how the initial perturbation distributes over the chain for a non-linear chain ($\alpha \neq 0$). In the following simulation, we repeat the above simulation with similar parameters, except we increase the time precision to $\delta_t = 0.01$ and change the non-linearity coefficient to $\alpha = 0.5$.

```python
N = 100
B = 0.8
sigma = 10
m, k = 1, 1

delta_t = 0.01
num_iter = int(4 * np.pi / (w_1 * delta_t))
t = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_two(N, B, k, m, sigma, delta_t, num_iter, alpha=0.5)
```

    100.0% done

```python
E_l = np.array(ret_vals[4])

plt.plot(E_l[0], label=r'$\omega_1 t / 2 pi = {:0.0f}$'.format(0 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[500], label=r'     = {:0.3f}'.format(500 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[800], label=r'     = {:0.3f}'.format(800 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[1200], label=r'     = {:0.3f}'.format(1200 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[2000], label=r'     = {:0.3f}'.format(2000 * delta_t * w_1 / (2 * np.pi)))
plt.xlabel(r"$n$")

plt.ylabel(r"$E(n)$")
plt.legend(fancybox=True, shadow=True)
plt.show()
```

We observe similar behavior to the linear case: two waves propogating to the left and to the right from the initial perturbation. However, the propogation is not symmetric: the wave moving to the right seems to have a larger amplitude than the wave moving to the right as more energy is stored in the particles on the right side of the chain.

We can bump up the non-linearity by increasing $\alpha$ to $\alpha = 1$ and increasing the time resolution to $\Delta t = 0.005$.

```python
N = 100
B = 0.8
sigma = 10
m, k = 1, 1

delta_t = 0.005
num_iter = int(4 * np.pi / (w_1 * delta_t))
t = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_two(N, B, k, m, sigma, delta_t, num_iter, alpha=1.0)
```

    12.4% done

    /usr/local/lib/python3.4/dist-packages/ipykernel_launcher.py:45: RuntimeWarning: overflow encountered in square
    /usr/local/lib/python3.4/dist-packages/ipykernel_launcher.py:7: RuntimeWarning: overflow encountered in square
      import sys
    /usr/local/lib/python3.4/dist-packages/ipykernel_launcher.py:34: RuntimeWarning: overflow encountered in square
    /usr/local/lib/python3.4/dist-packages/ipykernel_launcher.py:34: RuntimeWarning: invalid value encountered in subtract
    /usr/local/lib/python3.4/dist-packages/ipykernel_launcher.py:51: RuntimeWarning: overflow encountered in square

    100.0% done

```python
E_l = np.array(ret_vals[4])

plt.plot(E_l[0], label=r'$\omega_1 t / 2 pi = {:0.0f}$'.format(0 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[1000], label=r'     = {:0.3f}'.format(1000 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[1600], label=r'     = {:0.3f}'.format(1600 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[2400], label=r'     = {:0.3f}'.format(2400 * delta_t * w_1 / (2 * np.pi)))
plt.plot(E_l[4000], label=r'     = {:0.3f}'.format(4000 * delta_t * w_1 / (2 * np.pi)))
plt.xlabel(r"$n$")

plt.xlabel(r"$n$")
plt.ylabel(r"$E(n)$")
plt.legend(fancybox=True, shadow=True)
plt.show()
```

We observe similar, yet more extreme behavior than in the $\alpha = 0.5$ case: two wavefronts propogating to the left and right, but the right obtaining preferential energy from the initial perturbation. By increasing $\alpha$ we've increasing the asymmetry in the left and right moving waves.

### (4) Energy distribution in a nonlinear FPUT chain

Next, we explore how the energy from an initial perturbation that excites only the first $m = 1$ mode. We will observe how the energy of the system spreads among each of the normal modes of oscillation. We will define the mode amplitude as

\begin{equation}\tag{7}
Q_m(t) = \sum_{n=1}^{N} u_n(t) \xi_n^{(m)} = \sqrt{\frac{2}{N + 1}} \sum_{n=1}^{N} u_n(t) \sin\left( \frac{mn\pi}{N+1} \right)
\end{equation}

and the energy stored in the $m^{th}$ mode as

\begin{equation}\tag{8}
E_m(t) = \frac{m}{2} \left( \dot{Q}_m^2 + \left(w^{(m)} Q_m\right)^2 \right)
\end{equation}

```python
# The mode energy of the chain
# Eq. (8)
def ME(u, u_d, m=1, num_modes=4, **kwargs):
    N = len(u) - 2
    w = 2 * np.sqrt(k / m) * np.sin(np.arange(1, num_modes + 1) * np.pi / (2 * (N + 1)))
    mn = np.outer(np.arange(1, num_modes + 1), np.arange(len(u)))
    # The mode amplitude
    # Eq. (7)
    Q = np.sqrt(2 / (N + 1)) * np.sum(u * np.sin(mn * np.pi / (N + 1)), axis=1)
    # The time derivative of the mode amplitude
    # Time derivative of Eq. (7)
    Q_d = np.sqrt(2 / (N + 1)) * np.sum(u_d * np.sin(mn * np.pi / (N + 1)), axis=1)
    return (m / 2) * (Q_d**2 + (w**2) * Q**2)

# Runs a simulation of the chain with an initial perturbation exciting only the first mode of oscillation
def run_sim_three(N, B, k, m, delta_t, num_iter, **kwargs):
    u = A * np.sqrt(2 / (N + 1)) * np.sin(np.arange(N + 2) * np.pi / (N + 1))
    u[0] = 0
    u[N + 1] = 0
    u_d = np.zeros(N + 2)

    rets = velocity_verlet(u, u_d, delta_t, num_iter, acceleration=accel, funcs=[lambda x, v, **kwargs : x, lambda x, v, **kwargs : v, KE, PE, ME], **kwargs)

    return rets
```

We run the simulation of the non-linear chain with parameters $N = 36$, $\Delta t = 0.02$, $m = k = 1$, $\mathcal{A} = 0.5$, and the non-linear parameter $\alpha = 1.25$. The initial conditions of the chain are $u_n(t = 0) = \mathcal{A} \xi_n^{(1)} $ and $\dot{u}_n(t=0) = 0$.

```python
N = 36
A = 0.5
alpha = 1.25
m, k = 1, 1

delta_t = 0.02
num_iter = int(280 * 2 * np.pi / (w_1 * delta_t))
t = np.arange(num_iter) * delta_t * w_1 / (2 * np.pi)

ret_vals = run_sim_three(N, A, k, m, delta_t, num_iter, alpha=alpha)

x = np.array(ret_vals[0])
v = np.array(ret_vals[1])
E_k = np.array(ret_vals[2])
E_p = np.array(ret_vals[3])
E_m = np.array(ret_vals[4])
```

    100.0% done

```python
plt.plot(t, E_k + E_p, label='Total Energy')

for i in range(0, 4):
    plt.plot(t, E_m[:, i], label='mode {}'.format(i + 1))

plt.xlabel(r"$\omega_1 t / 2 \pi$")

plt.legend(loc='right', bbox_to_anchor=(1.25, .5), ncol=1, fancybox=True, shadow=True)
plt.show()
```

We observe that initially, all energy is in the first mode of oscillation, which is the only mode initially excited. We expect that the energy will drift among the modes and eventually reach a distribution consistent with the equipartition of energy. However, in the plot above, we see that after some time, almost all of the energy of the system returns to the first mode!


```python
%load_ext autoreload
%autoreload 2

import utils
from sympy import sqrt
import sympy as sy
```

# Exercise 4 Problem 1

```python
dh_params = [
    [0, 0, 0, "theta_1"],
    [-90, "l_1", "d_2", 0],
]

# Joint center in coordinate frame of previous joint
joint_points = [
    sy.Matrix([0, 0, 0]),
    sy.Matrix([sy.Symbol("l_1"), sy.Symbol("d_2"), 0]),
]

# Center of mass of each link
m_center_points = [
    sy.Matrix([sy.Symbol("l_1")/2, 0, 0]),
    sy.Matrix([0, 0, sy.Symbol("l_2")])
]

# Set acceleration at base to negative gravitational accel
v_dot_0 = sy.Matrix([sy.Symbol("g"), 0, 0])

# Link mass
link_m = [sy.Symbol("m_1"), sy.Symbol("m_2")]

# Link inertia tensor
link_I = [
    sy.diag(sy.Symbol("I_{xx1}"), sy.Symbol("I_{yy1}"), sy.Symbol("I_{zz1}")),
    sy.diag(sy.Symbol("I_{xx2}"), sy.Symbol("I_{yy2}"), sy.Symbol("I_{zz2}"))
]
```

```python
tau = utils.newton_euler(dh_params, joint_points, m_center_points, v_dot_0, link_m, link_I)
```

    Forward phase:
    i = 0:

$\displaystyle {}^1\dot{v}_1 = \left[\begin{matrix}g \cos{\left(\theta_{1} \right)}\\- g \sin{\left(\theta_{1} \right)}\\0\end{matrix}\right]$

$\displaystyle {}^1\omega_1 = \left[\begin{matrix}0\\0\\\dot\theta_{1}\end{matrix}\right]$

$\displaystyle {}^1\dot{\omega}_1 = \left[\begin{matrix}0\\0\\\ddot\theta_{1}\end{matrix}\right]$

$\displaystyle {}^1\dot{v}_{C_1} = \left[\begin{matrix}g \cos{\left(\theta_{1} \right)}\\- g \sin{\left(\theta_{1} \right)}\\0\end{matrix}\right]$

$\displaystyle {}^1F_1 = \left[\begin{matrix}m_{1} \left(- \frac{\dot\theta_{1}^{2} l_{1}}{2} + g \cos{\left(\theta_{1} \right)}\right)\\m_{1} \left(\frac{\ddot\theta_{1} l_{1}}{2} - g \sin{\left(\theta_{1} \right)}\right)\\0\end{matrix}\right]$

$\displaystyle {}^1N_1 = \left[\begin{matrix}0\\0\\I_{zz1} \ddot\theta_{1}\end{matrix}\right]$

    i = 1:

$\displaystyle {}^2\dot{v}_2 = \left[\begin{matrix}- \ddot\theta_{1} d_{2} - \dot\theta_{1}^{2} l_{1} - 2 \dot\theta_{1} \dot{d_2} + g \cos{\left(\theta_{1} \right)}\\0\\\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - g \sin{\left(\theta_{1} \right)}\end{matrix}\right]$

$\displaystyle {}^2\omega_2 = \left[\begin{matrix}0\\- \dot\theta_{1}\\0\end{matrix}\right]$

$\displaystyle {}^2\dot{\omega}_2 = \left[\begin{matrix}0\\- \ddot\theta_{1}\\0\end{matrix}\right]$

$\displaystyle {}^2\dot{v}_{C_2} = \left[\begin{matrix}- \ddot\theta_{1} d_{2} - \dot\theta_{1}^{2} l_{1} - 2 \dot\theta_{1} \dot{d_2} + g \cos{\left(\theta_{1} \right)}\\0\\\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - g \sin{\left(\theta_{1} \right)}\end{matrix}\right]$

$\displaystyle {}^2F_2 = \left[\begin{matrix}m_{2} \left(- \ddot\theta_{1} d_{2} - \ddot\theta_{1} l_{2} - \dot\theta_{1}^{2} l_{1} - 2 \dot\theta_{1} \dot{d_2} + g \cos{\left(\theta_{1} \right)}\right)\\0\\m_{2} \left(\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - \dot\theta_{1}^{2} l_{2} - g \sin{\left(\theta_{1} \right)}\right)\end{matrix}\right]$

$\displaystyle {}^2N_2 = \left[\begin{matrix}0\\- I_{yy2} \ddot\theta_{1}\\0\end{matrix}\right]$

    Backwards phase:
    i = 2:

$\displaystyle {}^2f_2 = \left[\begin{matrix}m_{2} \left(- \ddot\theta_{1} d_{2} - \ddot\theta_{1} l_{2} - \dot\theta_{1}^{2} l_{1} - 2 \dot\theta_{1} \dot{d_2} + g \cos{\left(\theta_{1} \right)}\right)\\0\\m_{2} \left(\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - \dot\theta_{1}^{2} l_{2} - g \sin{\left(\theta_{1} \right)}\right)\end{matrix}\right]$

$\displaystyle {}^2n_2 = \left[\begin{matrix}0\\- I_{yy2} \ddot\theta_{1} - l_{2} m_{2} \left(\ddot\theta_{1} d_{2} + \ddot\theta_{1} l_{2} + \dot\theta_{1}^{2} l_{1} + 2 \dot\theta_{1} \dot{d_2} - g \cos{\left(\theta_{1} \right)}\right)\\0\end{matrix}\right]$

    i = 1:

$\displaystyle {}^1f_1 = \left[\begin{matrix}- \frac{m_{1} \left(\dot\theta_{1}^{2} l_{1} - 2 g \cos{\left(\theta_{1} \right)}\right)}{2} - m_{2} \left(\ddot\theta_{1} d_{2} + \ddot\theta_{1} l_{2} + \dot\theta_{1}^{2} l_{1} + 2 \dot\theta_{1} \dot{d_2} - g \cos{\left(\theta_{1} \right)}\right) \cos{\left(\theta_{1} \right)}\\\frac{m_{1} \left(\ddot\theta_{1} l_{1} - 2 g \sin{\left(\theta_{1} \right)}\right)}{2} - m_{2} \left(\ddot\theta_{1} d_{2} + \ddot\theta_{1} l_{2} + \dot\theta_{1}^{2} l_{1} + 2 \dot\theta_{1} \dot{d_2} - g \cos{\left(\theta_{1} \right)}\right) \sin{\left(\theta_{1} \right)}\\m_{2} \left(\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - \dot\theta_{1}^{2} l_{2} - g \sin{\left(\theta_{1} \right)}\right)\end{matrix}\right]$

$\displaystyle {}^1n_1 = \left[\begin{matrix}\left(I_{yy2} \ddot\theta_{1} + l_{2} m_{2} \left(\ddot\theta_{1} d_{2} + \ddot\theta_{1} l_{2} + \dot\theta_{1}^{2} l_{1} + 2 \dot\theta_{1} \dot{d_2} - g \cos{\left(\theta_{1} \right)}\right)\right) \sin{\left(\theta_{1} \right)}\\- \left(I_{yy2} \ddot\theta_{1} + l_{2} m_{2} \left(\ddot\theta_{1} d_{2} + \ddot\theta_{1} l_{2} + \dot\theta_{1}^{2} l_{1} + 2 \dot\theta_{1} \dot{d_2} - g \cos{\left(\theta_{1} \right)}\right)\right) \cos{\left(\theta_{1} \right)}\\I_{zz1} \ddot\theta_{1} + \frac{l_{1} m_{1} \left(\ddot\theta_{1} l_{1} - 2 g \sin{\left(\theta_{1} \right)}\right)}{4}\end{matrix}\right]$

    Joint torques/ forces:

$\displaystyle \tau = \left[\begin{matrix}\tau_{1}\\\tau_{2}\end{matrix}\right] = \left[\begin{matrix}I_{zz1} \ddot\theta_{1} + \frac{l_{1} m_{1} \left(\ddot\theta_{1} l_{1} - 2 g \sin{\left(\theta_{1} \right)}\right)}{4}\\m_{2} \left(\ddot\theta_{1} l_{1} + \ddot{d_2} - \dot\theta_{1}^{2} d_{2} - \dot\theta_{1}^{2} l_{2} - g \sin{\left(\theta_{1} \right)}\right)\end{matrix}\right]$

```python

```


```python
from __future__ import division
from sympy import *
init_session(quiet=True)
```

```python
T,delta,rho_r,b_m,c_m,a_m,R_u = symbols('T,delta,rho_r,b_m,c_m,a_m,R_u')
W = symbols('W', cls=Function)(delta)

alphar = -log(1-delta*rho_r*(b_m-c_m)) - sqrt(2)*a_m/(4*R_u*T*b_m)*log(W);
display(alphar)
for ndelta in range(1,5):
    ss = simplify(diff(alphar, delta, ndelta))
    display(ss)

W =(1+delta*rho_r*(b_m*(1+sqrt(2)+c_m))) / (1+delta*rho_r*(b_m*(1-sqrt(2)+c_m)))
for ndelta in range(1,5):
    display(diff(W,delta,ndelta))
```

```python

```

doc/notebooks/Sympy derivatives of alphar from VTPR.ipynb


# Matrix generation

## Init symbols for *sympy*

```python
from sympy import *
from geom_util import *
from sympy.vector import CoordSys3D
N = CoordSys3D('N')
alpha1, alpha2, alpha3 = symbols("alpha_1 alpha_2 alpha_3", real = True, positive=True)
init_printing()

%matplotlib inline

%reload_ext autoreload
%autoreload 2
%aimport geom_util
```

### Lame params

```python
H1=symbols('H1')
H2=S(1)
H3=S(1)

H=[H1, H2, H3]
DIM=3
dH = zeros(DIM,DIM)
for i in range(DIM):
    for j in range(DIM):
        if (i == 0 and j != 1):
            dH[i,j]=Symbol('H_{{{},{}}}'.format(i+1,j+1))

dH

```

### Metric tensor

${\displaystyle \hat{G}=\sum_{i,j} g^{ij}\vec{R}_i\vec{R}_j}$

```python
G_up = getMetricTensorUpLame(H1, H2, H3)
```

${\displaystyle \hat{G}=\sum_{i,j} g_{ij}\vec{R}^i\vec{R}^j}$

```python
G_down = getMetricTensorDownLame(H1, H2, H3)
```

### Christoffel symbols

```python
DIM=3

G_down_diff = MutableDenseNDimArray.zeros(DIM, DIM, DIM)
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):

            G_down_diff[i,i,k]=2*H[i]*dH[i,k]

GK = getChristoffelSymbols2(G_up, G_down_diff, (alpha1, alpha2, alpha3))
GK
```

### Gradient of vector

$
\left(
\begin{array}{c}
\nabla_1 u_1 \\ \nabla_2 u_1 \\ \nabla_3 u_1 \\
\nabla_1 u_2 \\ \nabla_2 u_2 \\ \nabla_3 u_2 \\
\nabla_1 u_3 \\ \nabla_2 u_3 \\ \nabla_3 u_3 \\
\end{array}
\right)
=
B \cdot
\left(
\begin{array}{c}
u_1 \\
\frac { \partial u_1 } { \partial \alpha_1} \\
\frac { \partial u_1 } { \partial \alpha_2} \\
\frac { \partial u_1 } { \partial \alpha_3} \\
u_2 \\
\frac { \partial u_2 } { \partial \alpha_1} \\
\frac { \partial u_2 } { \partial \alpha_2} \\
\frac { \partial u_2 } { \partial \alpha_3} \\
u_3 \\
\frac { \partial u_3 } { \partial \alpha_1} \\
\frac { \partial u_3 } { \partial \alpha_2} \\
\frac { \partial u_3 } { \partial \alpha_3} \\
\end{array}
\right)
= B \cdot D \cdot
\left(
\begin{array}{c}
u^1 \\
\frac { \partial u^1 } { \partial \alpha_1} \\
\frac { \partial u^1 } { \partial \alpha_2} \\
\frac { \partial u^1 } { \partial \alpha_3} \\
u^2 \\
\frac { \partial u^2 } { \partial \alpha_1} \\
\frac { \partial u^2 } { \partial \alpha_2} \\
\frac { \partial u^2 } { \partial \alpha_3} \\
u^3 \\
\frac { \partial u^3 } { \partial \alpha_1} \\
\frac { \partial u^3 } { \partial \alpha_2} \\
\frac { \partial u^3 } { \partial \alpha_3} \\
\end{array}
\right)
$

```python
def row_index_to_i_j_grad(i_row):
    return i_row // 3, i_row % 3

B = zeros(9, 12)
B[0,1] = S(1)
B[1,2] = S(1)

B[2,3] = S(1)

B[3,5] = S(1)
B[4,6] = S(1)
B[5,7] = S(1)

B[6,9] = S(1)
B[7,10] = S(1)
B[8,11] = S(1)

for row_index in range(9):
    i,j=row_index_to_i_j_grad(row_index)
    B[row_index, 0] = -GK[i,j,0]
    B[row_index, 4] = -GK[i,j,1]
    B[row_index, 8] = -GK[i,j,2]

B
```

### Physical coordinates

$u_i=u_{[i]} H_i$

```python
P=zeros(12,12)
P[0,0]=H[0]
P[1,0]=dH[0,0]
P[1,1]=H[0]
P[2,0]=dH[0,1]
P[2,2]=H[0]
P[3,0]=dH[0,2]
P[3,3]=H[0]

P[4,4]=H[1]
P[5,4]=dH[1,0]
P[5,5]=H[1]
P[6,4]=dH[1,1]
P[6,6]=H[1]
P[7,4]=dH[1,2]
P[7,7]=H[1]

P[8,8]=H[2]
P[9,8]=dH[2,0]
P[9,9]=H[2]
P[10,8]=dH[2,1]
P[10,10]=H[2]
P[11,8]=dH[2,2]
P[11,11]=H[2]
P=simplify(P)
P
```

```python
B_P = zeros(9,9)

for i in range(3):
    for j in range(3):

        row_index = i*3+j

        B_P[row_index, row_index] = 1/(H[i]*H[j])

Grad_U_P = simplify(B_P*B*P)
Grad_U_P
```

### Strain tensor

$
\left(
\begin{array}{c}
\varepsilon_{11} \\
\varepsilon_{22} \\
\varepsilon_{33} \\
2\varepsilon_{12} \\
2\varepsilon_{13} \\
2\varepsilon_{23} \\
\end{array}
\right)
=
\left(E + E_{NL} \left( \nabla \vec{u} \right) \right) \cdot
\left(
\begin{array}{c}
\nabla_1 u_1 \\ \nabla_2 u_1 \\ \nabla_3 u_1 \\
\nabla_1 u_2 \\ \nabla_2 u_2 \\ \nabla_3 u_2 \\
\nabla_1 u_3 \\ \nabla_2 u_3 \\ \nabla_3 u_3 \\
\end{array}
\right)$

```python
E=zeros(6,9)
E[0,0]=1
E[1,4]=1
E[2,8]=1
E[3,1]=1
E[3,3]=1
E[4,2]=1
E[4,6]=1
E[5,5]=1
E[5,7]=1
E
```

```python
StrainL=simplify(E*Grad_U_P)
StrainL
```

```python
def E_NonLinear(grad_u):
    N = 3

    du = zeros(N, N)

    #    print("===Deformations===")

    for i in range(N):
        for j in range(N):
            index = i*N+j
            du[j,i] = grad_u[index]

    #    print("========")

    I = eye(3)

    a_values = S(1)/S(2) * du * G_up

    E_NL = zeros(6,9)
    E_NL[0,0] = a_values[0,0]
    E_NL[0,3] = a_values[0,1]
    E_NL[0,6] = a_values[0,2]

    E_NL[1,1] = a_values[1,0]
    E_NL[1,4] = a_values[1,1]
    E_NL[1,7] = a_values[1,2]

    E_NL[2,2] = a_values[2,0]
    E_NL[2,5] = a_values[2,1]
    E_NL[2,8] = a_values[2,2]

    E_NL[3,1] = 2*a_values[0,0]
    E_NL[3,4] = 2*a_values[0,1]
    E_NL[3,7] = 2*a_values[0,2]

    E_NL[4,0] = 2*a_values[2,0]
    E_NL[4,3] = 2*a_values[2,1]
    E_NL[4,6] = 2*a_values[2,2]

    E_NL[5,2] = 2*a_values[1,0]
    E_NL[5,5] = 2*a_values[1,1]
    E_NL[5,8] = 2*a_values[1,2]

    return E_NL

%aimport geom_util
u=getUHat3DPlane(alpha1, alpha2, alpha3)

# u=getUHatU3Main(alpha1, alpha2, alpha3)

gradu=B*u

E_NL = E_NonLinear(gradu)*B

E_NL

```

```python
%aimport geom_util

u=getUHatU3MainPlane(alpha1, alpha2, alpha3)

gradup=Grad_U_P*u

# e=E*gradup
# e

E_NLp = E_NonLinear(gradup)*gradup

simplify(E_NLp)
```

## Virtual work

```python
%aimport geom_util
C_tensor = getIsotropicStiffnessTensor()
C = convertStiffnessTensorToMatrix(C_tensor)
C
```

```python
StrainL.T*C*StrainL*H1
```

### Tymoshenko theory

$u_1 \left( \alpha_1, \alpha_2, \alpha_3 \right)=u\left( \alpha_1 \right)+\alpha_3\gamma \left( \alpha_1 \right) $

$u_2 \left( \alpha_1, \alpha_2, \alpha_3 \right)=0 $

$u_3 \left( \alpha_1, \alpha_2, \alpha_3 \right)=w\left( \alpha_1 \right) $

$ \left(
\begin{array}{c}
u_1 \\
\frac { \partial u_1 } { \partial \alpha_1} \\
\frac { \partial u_1 } { \partial \alpha_2} \\
\frac { \partial u_1 } { \partial \alpha_3} \\
u_2 \\
\frac { \partial u_2 } { \partial \alpha_1} \\
\frac { \partial u_2 } { \partial \alpha_2} \\
\frac { \partial u_2 } { \partial \alpha_3} \\
u_3 \\
\frac { \partial u_3 } { \partial \alpha_1} \\
\frac { \partial u_3 } { \partial \alpha_2} \\
\frac { \partial u_3 } { \partial \alpha_3} \\
\end{array}
\right) = T \cdot
\left(
\begin{array}{c}
u \\
\frac { \partial u } { \partial \alpha_1} \\
\gamma \\
\frac { \partial \gamma } { \partial \alpha_1} \\
w \\
\frac { \partial w } { \partial \alpha_1} \\
\end{array}
\right) $

```python
T=zeros(12,6)
T[0,0]=1
T[0,2]=alpha3
T[1,1]=1
T[1,3]=alpha3
T[3,2]=1

T[8,4]=1
T[9,5]=1
T
```

```python
D_p_T = StrainL*T
simplify(D_p_T)
```

```python
u = Function("u")
t = Function("theta")
w = Function("w")

u1=u(alpha1)+alpha3*t(alpha1)
u3=w(alpha1)

gu = zeros(12,1)
gu[0] = u1
gu[1] = u1.diff(alpha1)
gu[3] = u1.diff(alpha3)

gu[8] = u3
gu[9] = u3.diff(alpha1)

gradup=Grad_U_P*gu

# E_NLp = E_NonLinear(gradup)*gradup

# simplify(E_NLp)
# gradup=Grad_U_P*gu

# o20=(K*u(alpha1)-w(alpha1).diff(alpha1)+t(alpha1))/2
# o21=K*t(alpha1)
# O=1/2*o20*o20+alpha3*o20*o21-alpha3*K/2*o20*o20
# O=expand(O)
# O=collect(O,alpha3)
# simplify(O)

StrainNL = E_NonLinear(gradup)*gradup
StrainL*gu+simplify(StrainNL)
```

### Square theory

$u^1 \left( \alpha_1, \alpha_2, \alpha_3 \right)=u_{10}\left( \alpha_1 \right)p_0\left( \alpha_3 \right)+u_{11}\left( \alpha_1 \right)p_1\left( \alpha_3 \right)+u_{12}\left( \alpha_1 \right)p_2\left( \alpha_3 \right) $

$u^2 \left( \alpha_1, \alpha_2, \alpha_3 \right)=0 $

$u^3 \left( \alpha_1, \alpha_2, \alpha_3 \right)=u_{30}\left( \alpha_1 \right)p_0\left( \alpha_3 \right)+u_{31}\left( \alpha_1 \right)p_1\left( \alpha_3 \right)+u_{32}\left( \alpha_1 \right)p_2\left( \alpha_3 \right) $

$ \left(
\begin{array}{c}
u^1 \\
\frac { \partial u^1 } { \partial \alpha_1} \\
\frac { \partial u^1 } { \partial \alpha_2} \\
\frac { \partial u^1 } { \partial \alpha_3} \\
u^2 \\
\frac { \partial u^2 } { \partial \alpha_1} \\
\frac { \partial u^2 } { \partial \alpha_2} \\
\frac { \partial u^2 } { \partial \alpha_3} \\
u^3 \\
\frac { \partial u^3 } { \partial \alpha_1} \\
\frac { \partial u^3 } { \partial \alpha_2} \\
\frac { \partial u^3 } { \partial \alpha_3} \\
\end{array}
\right) = L \cdot
\left(
\begin{array}{c}
u_{10} \\
\frac { \partial u_{10} } { \partial \alpha_1} \\
u_{11} \\
\frac { \partial u_{11} } { \partial \alpha_1} \\
u_{12} \\
\frac { \partial u_{12} } { \partial \alpha_1} \\
u_{30} \\
\frac { \partial u_{30} } { \partial \alpha_1} \\
u_{31} \\
\frac { \partial u_{31} } { \partial \alpha_1} \\
u_{32} \\
\frac { \partial u_{32} } { \partial \alpha_1} \\
\end{array}
\right) $

```python
L=zeros(12,12)
h=Symbol('h')
p0=1/2-alpha3/h
p1=1/2+alpha3/h
p2=1-(2*alpha3/h)**2

L[0,0]=p0
L[0,2]=p1
L[0,4]=p2

L[1,1]=p0
L[1,3]=p1
L[1,5]=p2

L[3,0]=p0.diff(alpha3)
L[3,2]=p1.diff(alpha3)
L[3,4]=p2.diff(alpha3)

L[8,6]=p0
L[8,8]=p1
L[8,10]=p2

L[9,7]=p0
L[9,9]=p1
L[9,11]=p2

L[11,6]=p0.diff(alpha3)
L[11,8]=p1.diff(alpha3)
L[11,10]=p2.diff(alpha3)

L
```

```python
D_p_L = StrainL*L
simplify(D_p_L)
```

```python
p0_2=p0*p0
p01=p0*p1
p02=p0*p2
p1_2=p1*p1
p12=p1*p2
p2_2=p2*p2

p0_2i=integrate(p0_2, (alpha3, -h/2, h/2))
p01i=integrate(p01, (alpha3, -h/2, h/2))
p02i=integrate(p02, (alpha3, -h/2, h/2))
p1_2i=integrate(p1_2, (alpha3, -h/2, h/2))
p12i=integrate(p12, (alpha3, -h/2, h/2))
p2_2i=integrate(p2_2, (alpha3, -h/2, h/2))

# p0_2i = simplify(p0_2i)
# p01i = expand(simplify(p01i))
# p02i = expand(simplify(p02i))
# p1_2i = expand(simplify(p1_2i))
# p12i = expand(simplify(p12i))
# p2_2i = expand(simplify(p2_2i))

```

```python
p0_2i
```

```python
p01i
```

```python
p02i
```

```python
p1_2i
```

```python
p12i
```

```python
p2_2i
```

```python
1/6
```

```python
Ct=getOrthotropicStiffnessTensor()
C=convertStiffnessTensorToMatrix(Ct)
LC=zeros(6,9)
LC[0,0]=p0
LC[0,1]=p1
LC[0,2]=p2

LC[2,3]=p0
LC[2,4]=p1
LC[2,5]=p2

LC[4,6]=p0
LC[4,7]=p1
LC[4,8]=p2

e = LC.T*C*LC
integrate(e, (alpha3, -h/2, h/2))
```

```python

```

## Mass matrix

```python
rho=Symbol('rho')
B_h=zeros(3,12)
B_h[0,0]=1
B_h[1,4]=1
B_h[2,8]=1
M=simplify(rho*P.T*B_h.T*G_up*B_h*P)
M
```

```python
M_p = L.T*M*L
integrate(M_p, (alpha3, -h/2, h/2))

```

```python
omega, t=symbols("\omega, t")
c=cos(omega*t)
c2=cos(omega*t)*cos(omega*t)
c3=cos(omega*t)*cos(omega*t)*cos(omega*t)
T=2*pi/omega
# omega*T/4
integrate(c, (t, 0, T/4))/T
```


```python
from __future__ import division
import numpy as np
import matplotlib.pyplot as plt
import numpy.matlib as mat
```

**Probabilistic Movement Primitives**
===================================

In this notebook we will write a class called **ProMP** which includes all the information required to define a ProMP as well as important methods which allow the manipulation of the MPs to show the potential of the probabilistic approach.

_______________________________________________________________________________

For stroke-based movements, the mean of a ProMP is given by a weighted sum of $N$ Gaussians. These Gaussian basis functions have the following form:

\begin{equation}
b_i(t):=\mathrm{exp}\left(-\frac{(t-c_i)^2}{2h}\right),\;i=1,\ldots,N
\tag{1}
\end{equation}

These functions are commonly normalized to improve regression

\begin{equation}
\phi_i(t):=\frac{b_i(t)}{\sum_{j=1}^Nb_j(t)}
 \tag{2}
\end{equation}

The function $z(t)$ allows temporal modulation, so that our basis functions are given by:

\begin{equation}
\phi_i(z)=\phi_i(z(t))
 \tag{3}
\end{equation}

We work in discrete time, so we define a time step d$t$. By default, we work with a time lapse of 1s, therefore, if $T$ steps are needed, d$t=1/T$. We then introduce $\mathbf{\Phi}_t:=(\phi_1(t),\ldots,\phi_N(t))\in R^{N\times 1}$, which includes the evaluation of all basis functions in a time step.

The **BasisFuncGauss** function creates and evaluates $N$ basis Gaussian functions with a bandwidth $h$. Input $f$ refers to the linear modulation factor, so that the modulation is given by $z(t):=ft$. Then, the Gaussian functions $\phi_i(t)$ are evaluated as $\phi_i(ft)$. $dt$ represents the time step. The functions have modes at $c_i$, evenly spread along $[0,1/f]$. The output of this function is a $T\times N$array for which every row corresponds to the vector $\mathbf{\Phi}_t$ for every time step.

In the code cell below, write instructions that will normalize the basis functions in $F$ in line 19 (use eq(2)).

```python
def BasisFuncGauss(N, h, f, dt):
    """
    Evaluates Gaussian basis functions in [0,1/f]
    N = number of basis functions
    h = bandwidth
    dt = time step
    f = modulation factor
    """
    tf = 1/f;
    T = int(round(tf/dt+1))
    Phi = np.zeros((T,N))
    for z in range(0,T):
        t = z*dt
        phi = np.zeros((1, N))
        for k in range(1,N+1):
            c = (k-1)/(N-1)
            phi[0,k-1] = np.exp(-(f*t - c)*(f*t - c)/(2*h))
        Phi[z,:N] = phi[0, :N]
    # -- INSERT YOUR CODE HERE --
    return Phi #[TxN]

```

In a ProMP, at time step $t$, a joint variable $q$ is modeled as:

\begin{equation}
q_t = \mathbf{\Phi}_t^{\mathrm{T}}\boldsymbol w + \epsilon_q
 \tag{4}
\end{equation}

where $\epsilon_q$ adds zero-mean Gaussian observation noise with variance $\Sigma_q$. It follows that the probability of observing $q_t$ is represented by:

\begin{equation}
p(q_t|\boldsymbol w)=\mathcal{N}\left(q_t\,\big|\,\mathbf{\Phi}^{\mathrm{T}}_t\boldsymbol w, \mathbf{\Sigma}_q\right)
 \tag{5}
\end{equation}

Since $\Sigma_q$ is the same for every time step, the values $q_t$ are taken from independent and identical distributions, i.i.d. Hence, the probability of observing a trajectory $\tau:=\{q_1,\ldots,q_T\}$ is given by:

\begin{equation}
p(q_t|\boldsymbol w):=\prod_{t=1}^Tp(q_t|\boldsymbol w)
 \tag{6}
\end{equation}

However, since parameters $\boldsymbol w$ are to be learnt from data, we also assume such parameters are taken from a distribution $\boldsymbol w\sim p(\boldsymbol w|\theta)=\mathcal{N}(\boldsymbol w|\mu_{\boldsymbol w},\mathbf{\Sigma}_{\boldsymbol w})$. We therefore would like to have a predictive distribution of $q_t$ which does not depend on $\boldsymbol w$, but on $\theta:=(\mu_{\boldsymbol w},\mathbf{\Sigma}_{\boldsymbol w})$. This is done buy marginalizing $\boldsymbol w$ out in the distribution as follows:

\begin{equation}
p(q_t|\theta) = \int\mathcal{N}(q_t\,\big|\,\mathbf{\Phi}^{\mathrm{T}}_t\boldsymbol w,\,\Sigma_q)\mathcal{N}(\boldsymbol w\,\big|\,\mu_{\boldsymbol w},\,\Sigma_{\boldsymbol w})\mathrm{d}\boldsymbol w
 = \mathcal{N}(q_t\,\big|\,\mathbf{\Phi}^{\mathrm{T}}_t\boldsymbol w,\,\Sigma_q+\mathbf{\Phi}^{\mathrm{T}}_t\mathbf{\Sigma}_{\boldsymbol w}\mathbf{\Phi}_t)
  \tag{7}
\end{equation}

In the *ProMP* class, a *ProMP* object is initialized by the number of basis functions $N$, the bandwidth in the basis functions $h$, the time step d$t$, the covariance of the original distributions on $q$, *covQ* ($\Sigma_q$), the mean of the weights, $Wm$ ($\mu_{\boldsymbol w}$), and the covariance of the weights, *covW* ($\mathbf{\Sigma}_{\boldsymbol w}$).

When initialized, the class also defines other convenience variables. *Phi* is the $T\times N$ matrix containing the basis functions evaluated for every time step and with a default modulation factor equal to 1; $Qm$ ($\mu_{q,t}$) is the mean values of $q$ for every time step; and *cov* is the covariance of the marginal distribution on $q$ for every time step.

In the code cell below, write instructions that will calculate the covariance matrix $cov$  in line 13 (use eq(7)).

```python
def __init__(self, N, h, dt, covQ, Wm, covW):
    self.N = N
    self.h = h
    self.dt = dt
    self.covQ = covQ
    self.Wm = Wm #[Nx1]
    self.covW = covW #[NxN]
    self.Phi = BasisFuncGauss(N,h,1,dt) #[TxN]
    self.T,_ = self.Phi.shape
    self.Qm = np.matmul(self.Phi,self.Wm) #[Tx1]
    self.cov = np.zeros((self.T,1)) #[Tx1]
    for i in range(0,self.T):
        self.cov[i,0] = # -- write your code here --
```

One of the big advantages of the ProMPs approach is the inclusion of new via-points. Say we wish to add a new via-point $q_{t^{*}}^{*}$ at time $t^{*}$ which is to be observed with an uncertainty of $\Sigma_{q^{*}}$. We simply have to obtain a posterior distribution over $\boldsymbol w$ by affecting our prior distribution with the new information given by $q_{t^{*}}^{*}$ and $\Sigma_{q^{*}}$. We thus apply Bayes theorem as follows:

\begin{equation}
p(\boldsymbol w|q_{t^{*}}^{*},\Sigma_{q^{*}})\propto\mathcal{N}\left(q_{t^{*}}^{*}\,\big|\,\mathbf{\Phi}^{\mathrm{T}}_{t^{*}}\boldsymbol w,\,\Sigma_{q^{*}}\right)p(\boldsymbol w)
 \tag{8}
\end{equation}

The result of applying the Bayes theorem leads to the following mean and variance of the posterior distribution:

\begin{equation}
\mu_{\boldsymbol w}^{[\mathrm{new}]} = \mu_{\boldsymbol w}+\mathbf{\Sigma}_{\boldsymbol w}\mathbf{\Phi}_{t^{*}}(\Sigma_q^{*}+\mathbf{\Phi}_{t^{*}}^{\mathrm{T}}\mathbf{\Sigma}_{\boldsymbol w}\mathbf{\Phi}_{t^{*}})^{-1}(q_{t^{*}}^{*}-\mathbf{\Phi}^{\mathrm{T}}_{t^{*}}\mu_{\boldsymbol w})
 \tag{9}
\end{equation}

\begin{equation}
\mathbf{\Sigma}_{\boldsymbol w}^{[\mathrm{new}]} = \mathbf{\Sigma}_{\boldsymbol w}-\mathbf{\Sigma}_{\boldsymbol w}\mathbf{\Phi}_{t^{*}}(\Sigma_q^{*}+\mathbf{\Phi}_{t^{*}}^{\mathrm{T}}\mathbf{\Sigma}_{\boldsymbol w}\mathbf{\Phi}_{t^{*}})^{-1}\mathbf{\Phi}_{t^{*}}^{\mathrm{T}}\mathbf{\Sigma}_{\boldsymbol w}
\end{equation}

The *condition* method conditions a ProMP to the new observation *Qstar* ($q^{*}_{t^{*}}$) at time *tstar* ($t^{*}$) with precision of observation *covQstar* ($\Sigma_{q^{*}}$).

Complete the below code cell to compute *Wm* and *covW* follwing Eq. (9). Use variable *Phit* ($\Phi_{t^{*}}$) which is defined inside the function..

```python
 def condition(self, tstar, Qstar, covQstar):
    Phit = np.transpose(self.Phi[tstar-1:tstar,:])
    self.Wm = # -- INSERT YOUR CODE HERE ---
    self.covW = # -- INSERT YOUR CODE HERE ---
    self.Qm = np.matmul(self.Phi,self.Wm)
    for i in range(0,self.T):
        self.cov[i,0] = self.covQ + np.matmul(np.array([self.Phi[i,:]]),np.matmul(self.covW,np.transpose(np.array([self.Phi[i,:]]))))
```

Temporal modulation is carried out by means of the previously defined phase function $z(t)$.

Member function *modulate* allows temporal modulation of a ProMP. Only linear time modulation is supported by this class. Modulation is given by the *factor* ($f$) input variable which is originally set to 1 when a ProMP is initialized. The phase function is then given by $z(t)=ft$. Note that $dt$ is constant throughout the code, hence, temporal modulation modifies $T$.

```python
 def modulate(self, factor):
    self.Phi = BasisFuncGauss(self.N,self.h,factor,self.dt) #[TxN]
    self.T,_ = self.Phi.shape # new T
    self.Qm = np.matmul(self.Phi,self.Wm) #[Tx1]
    self.cov = np.zeros((self.T,1)) #[Tx1]
    for i in range(0,self.T):
        self.cov[i,0] = self.covQ + np.matmul(np.array([self.Phi[i,:]]),np.matmul(self.covW,np.transpose(np.array([self.Phi[i,:]]))))
```

The *printMP* method inside the *ProMP* class plots the mean *Qm* ($\mu_q$) against time. It also plots two standard deviations above and below $\mu_q$ in order to show the marginal distribution at every time step. The *name* value refers to the title of the plot.

To complete the code cell bellow, compute the $T\times 1$ arrays *upper* and *lower* which contain two standard deviations above and below the mean $\mu_q$, respectively, for every time step. Remember that we store the covariance at every time step in the member variable *cov*.

```python
def printMP(self, name):
    t = np.arange(0, self.T*self.dt, self.dt)
    upper = # -- INSERT YOUR CODE HERE ---
    lower = # -- INSERT YOUR CODE HERE ---
    plt.plot(t,self.Qm)
    plt.fill_between(t, upper[:,0], lower[:,0], color = 'k', alpha = 0.1)
    plt.title(name)
    plt.show()
```

We would like to be able to blend different MPs into a single movement. For example, given two ProMPs, each having an important via-point, we wish to blend them so that our new ProMP crosses both via-points. We can do this by activation and deactivation of the ProMPs. We define the activation functions $\alpha^{[1]}(t)$ and $\alpha^{[2]}(t)$, with values $\alpha_{t}^{[1]}, \alpha_{t}^{[2]}\in[0,1]$ at each time step $t=1,\ldots,T$. When the value of an activation function reaches 0, the corresponding ProMP is fully deactivated. Oppositely, when the value reaches 1, the ProMP is fully activated. With these tools, the result of blending two ProMPs with predictive distributions $p^{[i]}(q_i)=\mathcal{N}(q_t\,|\,\mu_{q,t}^{[i]},\Sigma_t^{[i]})$, $i=1,2$ yields to a new ProMP described by $p^{*}(q_t)=\mathcal{N}(q_t\,|\,\mu_{q,t}^{*},\Sigma_t^{*})$, where:

\begin{equation}
\Sigma_t^{*} = (\frac{\alpha_t^{[1]}}{\Sigma_t^{[1]}} +\frac{\alpha_t^{[2]}}{\Sigma_t^{[2]}})^{-1}
 \tag{10}
\end{equation}

\begin{equation}
\mu_{q,t}^{*} = \Sigma_t^{*}(\frac{\alpha_t^{[1]}}{\Sigma_t^{[1]}}\mu_{q,t}^{[1]} + \frac{\alpha_t^{[2]}}{\Sigma_t^{[2]}}\mu_{q,t}^{[2]})
\end{equation}

The **blend** function blends two **ProMPs** objects, **MP1** and **MP2**, according to the activation functions **alpha1** ($\alpha_t^{[1]}$) and **alpha2** ($\alpha_t^{[2]}$). Vectors **alpha1** and **alpha2** contain the values of the activation functions for every time step. It is assumed that both ProMPs have the same value for **T**, hence, **alpha1** and **alpha2** must be arrays of **T** elements.

Using Eq. (10), complete the code above by writing the expressions for $cov12$ and $Qm12$, which correspond to the covariance $\Sigma_t^{*}$ and mean $\mu_{q,t}^{*}$ of the resulting ProMP at every time step. In order to be compatible with the $ProMP$ class,$cov12$ and $Qm12$ must be arrays with shapes $(T,1)$.

```python
def blend(MP1,MP2,alpha1,alpha2):
    """
    blends two MPs
        MP1, MP2 = ProMP objects to blend
        alpha1, alpha2 = activation functions for each respective MP [Tx1]
    """
    a1 = np.transpose(np.array([alpha1])) #[Tx1]
    a2 = np.transpose(np.array([alpha2])) #[Tx1]
    cov12 = # -- INSERT YOUR CODE HERE ---
    Qm12 = # -- INSERT YOUR CODE HERE ---
    M12 = ProMP(MP1.N,MP1.h,MP1.dt,MP1.covQ,np.zeros((MP1.N,1)),np.zeros((MP1.N,MP1.N)))
    M12.cov = cov12
    M12.Qm = Qm12
    return M12
```

Now that we defined the methods we can define ProMP class. Replace your written method in the corresponding section.

```python
class ProMP:
    """
    ProMP class
        N = number of basis functions
        h = bandwidth of basis functions
        dt = time step
        covQ = variance of original p(Q)
        Wm = mean of weights [Nx1]
        covW = variance of weights [NxN]
    internal:
        Phi = basis functions evaluated for every step [TxN]
        Qm = mean of Q [Nx1]
        cov = variance of p(Q|Wm) for every step  [Tx1]
    methods:
        condition (conditions an MP for a new viapoint)
            tstar = step of viapoint
            Qstar = value of Q of the viapoint
            covTstar = uncertainty of observation
        modulate (linear time modulation of the MP)
            factor = factor of linear modulation, z = factor*t
        printMP (plots an MP showing a standar deviation above and below)
            name = title of the plot
    """
    def __init__(self, N, h, dt, covQ, Wm, covW):
        self.N = N
        self.h = h
        self.dt = dt
        self.covQ = covQ
        self.Wm = Wm #[Nx1]
        self.covW = covW #[NxN]
        self.Phi = BasisFuncGauss(N,h,1,dt) #[TxN]
        self.T,_ = self.Phi.shape
        self.Qm = np.matmul(self.Phi,self.Wm) #[Tx1]
        self.cov = np.zeros((self.T,1)) #[Tx1]
        for i in range(0,self.T):
            self.cov[i,0] = # -- write your code here --

    def condition(self, tstar, Qstar, covQstar):
        Phit = np.transpose(self.Phi[tstar-1:tstar,:])
        self.Wm = # -- INSERT YOUR CODE HERE ---
        self.covW = # -- INSERT YOUR CODE HERE ---
        self.Qm = np.matmul(self.Phi,self.Wm)
        for i in range(0,self.T):
            self.cov[i,0] = self.covQ + np.matmul(np.array([self.Phi[i,:]]),np.matmul(self.covW,np.transpose(np.array([self.Phi[i,:]]))))

    def modulate(self, factor):
        self.Phi = BasisFuncGauss(self.N,self.h,factor,self.dt) #[TxN]
        self.T,_ = self.Phi.shape # new T
        self.Qm = np.matmul(self.Phi,self.Wm) #[Tx1]
        self.cov = np.zeros((self.T,1)) #[Tx1]
        for i in range(0,self.T):
            self.cov[i,0] = self.covQ + np.matmul(np.array([self.Phi[i,:]]),np.matmul(self.covW,np.transpose(np.array([self.Phi[i,:]]))))

    def printMP(self, name):
        t = np.arange(0, self.T*self.dt, self.dt)
        upper = # -- INSERT YOUR CODE HERE ---
        lower = # -- INSERT YOUR CODE HERE ---
        plt.plot(t,self.Qm)
        plt.fill_between(t, upper[:,0], lower[:,0], color = 'k', alpha = 0.1)
        plt.title(name)
        plt.show()

```

Now we will use the samples of weights $\boldsymbol w$ obtained in 5 observations. So we assume that the learning process has been completed. With this data, a ProMP for a single joint variable $q$ is created. We will carry out the following experiments:

*   Condition the ProMP to a first via-point
*   Condition the resulting ProMP from the previous experiment to a second via-point

*   Obtain two ProMPs after conditioning the original MP to each via-point, separately, and blend the two results
*   Time-modulate the original ProMP to different phases

Array $Wsamples$ includes the values of the $N = 15$ weights after 5 observations. $\mu_{\boldsymbol w}$ is computed from these observations and stored in $Wmean$. Similarly, $\mathbf{\Sigma}_{\boldsymbol w}$ is saved in $Wcov$. Using these values, three identical objects of the class $ProMP$ are defined.

```python
 # 15 weights obtained from 5 observations
Wsamples = np.array([[0.0141,0.0130,0.0038,0.0029,0.0143],
                      [0.0044,0.2025,0.0178,0.0703,0.0143],
                      [0.0388,0.1042,0.0531,0.0854,0.1479],
                      [0.0025,0.0321,0.0235,0.0495,0.0086],
                      [0.0810,0.0178,0.1500,0.0310,0.0843],
                      [0.0658,0.1258,0.0488,0.1650,0.1398],
                      [0.1059,0.0821,0.0116,0.2260,0.0531],
                      [0.0032,0.0952,0.0305,0.2220,0.0025],
                      [0.2031,0.1665,0.1430,0.0842,0.0656],
                      [0.0491,0.1543,0.1232,0.1505,0.0049],
                      [0.1914,0.0525,0.0783,0.0009,0.0292],
                      [0.0584,0.1035,0.0830,0.0305,0.1452],
                      [0.0157,0.1713,0.2550,0.0695,0.0051],
                      [0.2106,0.0630,0.0942,0.0086,0.1512],
                      [0.0959,0.2093,0.1388,0.0566,0.0819]])

Wmean = np.transpose([np.mean(Wsamples, axis=1)])
Wcov = np.cov(Wsamples)
N,_ = Wsamples.shape

T = 100
dt = 1/(T-1)

# Define MPs
MP1 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP2 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP3 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
```

$MP2$ is conditioned to point $Qstar1$, at time step $T$, with an uncertainty of the observation equal to $covQstar1$. The desired via-point, the original mean and the new conditioned mean are then plotted.

```python
# New desired point
tstar1 = 100
Qstar1 = MP2.Qm[100-1]+ 0.03
covQstar1 = 1e-6
MP2.condition(tstar1,Qstar1,covQstar1)

t = np.arange(0, 1+dt, dt)

# Plot original mean, and mean of conditioned MP
plt.figure()
plt.plot(t,MP1.Qm, 'r--', t, MP2.Qm, 'k')
plt.plot(tstar1/(MP1.T), Qstar1,'ro')
plt.title('Coditioning for point 1')
plt.show()

# Print MP conditioned to point 1
MP2.printMP('Coditioning for point 1')
```

The resulting MP is now conditioned to a second via-point, $Qstar2$, at time step $tstar2$. The resulting ProMP must cross both points.

```python
# Second desired point
tstar2 = 30
Qstar2 = 0.11
covQstar2 = 1e-6
MP2.condition(tstar2,Qstar2,covQstar2)

plt.figure()
plt.plot(t,MP1.Qm, 'r--', t, MP2.Qm, 'k')
plt.plot(tstar1/(MP1.T), Qstar1,'ro')
plt.plot(tstar2/(MP1.T), Qstar2,'ro')
plt.title('Coditioning for point 1 and point 2')
plt.show()

MP2.printMP('Coditioning for point 1 and point 2')
```

MPs $MP1$ and $MP2$ are conditioned to points 1 and 2, respectively. Then both are blended. The result of the blending must have both points as via-points.

To complete the code, come up with an expression for the $T-$element array ***alpha1*** which contains the values $\alpha_t^{[1]}$ of the activation function for **MP1**. Since tstar1 = 100 and tstar2 = 30, $\alpha_t^{[1]}$ must be equal (or close) to 1 at $t = 100$ but must be equal (or close) to 0 at $t = 30$, and there must be a smooth transition between these values. $\alpha_t^{[2]}$ is simply computed by $alpha2 = -alpha1+1$ in the following line of the code.

For this, find suitable values for $A$, $B$ and $C$ in the following function:
\begin{equation}
\alpha^{[1]}(t):=A\mathrm{tanh}\left(B\left(\frac{t-C}{T}\right)\right)+A
\end{equation}
The figure below shows an example of such a function using $B=30$, but you can play with this value and see how it affects the result.

The program then plots the ProMPs with the following results:

```python
 # Blending: MP1 is conditioned to point 1, MP2 is conditioned to point 2, MP12 is the result of blending both
MP1 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP2 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP1.condition(tstar1,Qstar1,covQstar1)
MP2.condition(tstar2,Qstar2,covQstar2)
MP1.printMP('MP1: Coditioning for point 1')
MP2.printMP('MP2: Coditioning for point 2')
# Activation functions for blending
alpha1 = # -- INSERT YOUR CODE HERE --
alpha2 = -alpha1+1
MP12 = blend(MP1,MP2,alpha1,alpha2)
MP12.printMP('blending of MP1 and MP2')
```

Given three initially identical MPs, MP1, MP2 and MP3, MP2 is modulated with a factor of 0.75, i.e. $z(t)=0.75t$, and MP3 with a factor 1.5, i.e. $z(t)=1.5t$. MP1 is left unmodified.

```python
#time modulation
MP1 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP2 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP3 = ProMP(N,0.02,dt,1e-6,Wmean,Wcov)
MP2.modulate(0.75)
MP3.modulate(1.5)

t1 = np.arange(0, MP1.T*dt, dt)
t2 = np.arange(0, MP2.T*dt, dt)
t3 = np.arange(0, MP3.T*dt, dt)

# Plot original mean, and mean of conditioned MP
plt.figure()
plt.plot(t1,MP1.Qm, 'k')
plt.plot(t2,MP2.Qm, 'b')
plt.plot(t3,MP3.Qm, 'g')
plt.title('Time modulation')
plt.show()

MP1.printMP('modulation factor = 1')
MP2.printMP('modulation factor = 0.75')
MP3.printMP('modulation factor = 1.5')
```


```python
import numpy as np
import numpy.random as nr
import sympy as sy
import typing

import IPython.display as disp

```

```python
sy.init_printing()

```

# Ex 02 009 Inverse matrix by Gauss-Jordan Elimination

* Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong, Mathematics For Machine Learning, Cambridge University Press, 2020, ISBN 978-1108455145.

Let's think about a 4x4 matrix as follows.

```python
A44 = np.array([
    [1, 0, 2, 0],
    [1, 1, 0, 0],
    [1, 2, 0, 1],
    [1, 1, 1, 1],
])

```

```python
A44

```

Let's also think about an identity matrix of the same size.

```python
I44 = np.identity(A44.shape[0], dtype=A44.dtype)

```

```python
I44

```

Let's augment both matrices.

```python
AX44 = np.hstack([A44, I44]) * 1.0

```

```python
AX44

```

## Gauss Jordan Elimination for `numpy.ndarray`

Following cell imports an implementation of the Gauss Jordan Elimination.

```python
import gauss_jordan

```

Let's apply to the augmented matrix.

```python
A44_inv_GJ = gauss_jordan.elimination(AX44)

```

```python
A44_inv_GJ

```

```python
A44_inv_GJ @ A44

```

```python
import numpy.testing as nt

nt.assert_array_almost_equal(A44_inv_GJ @ A44, np.array(np.identity(A44.shape[0])))

```

## References

* Gilbert Strang. 18.06 Linear Algebra. Spring 2010. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA.

## Final Bell

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import statsmodels.api as sm
import statsmodels.formula.api as smf
import scipy.stats as st
import sympy as sp
import datetime
import scipy as sp

sns.set_style("whitegrid")
```

```python
df = pd.read_csv(
    "/Users/sean/Desktop/Github/Parametric-Portfolio-Policy/price.csv"
).set_index("Date")

df_price = pd.pivot_table(df, values="adjusted", index=df.index, columns=df["symbol"])
mktcap = pd.pivot_table(df, values="mktcap", index=df.index, columns=df["symbol"])

have_null = df_price.columns[df_price.isna().any()]
have_null.append(df_price.columns[df_price.isna().any()])
df_price = df_price.drop(columns=have_null)
ret = np.log(df_price).diff()
ret = ret.iloc[1::, :]

have_null = mktcap.columns[mktcap.isna().any()]
have_null.append(mktcap.columns[mktcap.isna().any()])
mktcap = mktcap.drop(columns=have_null)
mktcap = mktcap.iloc[1::, :]

m12 = ret.rolling(12).sum()
m12 = m12.iloc[11::, :]
ret = ret.iloc[12::, :]
mktcap = mktcap.iloc[11::, :]
```

```python
ret_rowmean = ret.mean(axis=1)
mktcap_rowmean = mktcap.mean(axis=1)
m12_rowmean = m12.mean(axis=1)
ret_std = ret.std(axis=1)
mkt_std = mktcap.std(axis=1)
m12_std = m12.std(axis=1)
d = {
    "Mean(ret)": ret_rowmean,
    "Mean(mktcap)": mktcap_rowmean,
    "Mean(m12)": m12_rowmean,
    "Sd(ret)": ret_std,
    "Sd(mktcap)": mkt_std,
    "Sd(m12_std)": m12_std,
}
DF = pd.DataFrame(d)
print(DF.iloc[1::, :])

fig = plt.figure(figsize=(20, 10))
DF = DF.reset_index()
for i in range(1, 7):
    plt.subplot(2, 3, i)
    DF.iloc[:, i].plot()
plt.show()

fig = plt.figure(figsize=(20, 10))
plt.subplot(2, 2, 1)
plt.plot(ret.iloc[:, 0:10])
plt.ylabel("monthly returns")
plt.subplot(2, 2, 2)
plt.plot(m12.iloc[:, 0:10])
plt.ylabel("12 month returns")
plt.subplot(2, 2, 3)
plt.plot(mktcap.iloc[:, 0:10])
plt.ylabel("market captalization")
plt.show()
```

```python
def Scale(y, c=True, sc=True):

    """
    If ‘scale’ is
    ‘TRUE’ then scaling is done by dividing the (centered) columns of
    ‘x’ by their standard deviations if ‘center’ is ‘TRUE’, and the
    root mean square otherwise.  If ‘scale’ is ‘FALSE’, no scaling is
    done.

    The root-mean-square for a (possibly centered) column is defined
    as sqrt(sum(x^2)/(n-1)), where x is a vector of the non-missing
    values and n is the number of non-missing values.  In the case
    ‘center = TRUE’, this is the same as the standard deviation, but
    in general it is not.
    """
    x = y.copy()

    if c:
        x -= x.mean()
    if sc and c:
        x /= x.std()
    elif sc:
        x /= np.sqrt(x.pow(2).sum().div(x.count() - 1))
    return x

scaled_mktcap = pd.DataFrame(Scale(mktcap.T))
scaled_m12 = pd.DataFrame(Scale(m12.T))
scaled_mktcap = scaled_mktcap.T
scaled_mktcap = scaled_mktcap.iloc[0:109, :]
scaled_m12 = scaled_m12.T
scaled_m12 = scaled_m12.iloc[0:109, :]

def PPS(x, wb, nt, ret, m12, mktcap, rr):
    wi = wb + nt * (x[0] * m12 + x[1] * mktcap)
    wret = (wi * ret).sum(axis=1)
    ut = ((1 + wret) ** (1 - rr)) / (1 - rr)
    u = -(ut.mean())
    return u

Scaled_m12 = scaled_m12.reset_index()
Scaled_m12 = Scaled_m12.drop("Date", axis=1)
Scaled_mktcap = scaled_mktcap.reset_index()
Scaled_mktcap = Scaled_mktcap.drop("Date", axis=1)
Ret = ret.reset_index()
Ret = Ret.drop("Date", axis=1)

nt = wb = 1 / np.shape(ret)[1]
rr = 5

res_save = []
weights = []
x0 = np.array([0, 0])
for i in range(0, 60):
    opt = sp.optimize.minimize(
        PPS,
        x0,
        method="BFGS",
        args=(
            wb,
            nt,
            Ret.iloc[0 : 48 + i, :],
            Scaled_m12.iloc[0 : 48 + i, :],
            Scaled_mktcap.iloc[0 : 48 + i, :],
            rr,
        ),
    )
    print("The {} window".format(i + 1))
    print("The value:", opt["x"])
    res_save.append(opt["x"])
    w = wb + nt * (
        opt["x"][0] * Scaled_m12.iloc[i + 48, :]
        + opt["x"][1] * Scaled_mktcap.iloc[i + 48, :]
    )
    print(w)
    weights.append(w)

index = ret.index[49:110]
char_df = pd.DataFrame(res_save, index=index, columns=["m12", "mktcap"])

fig = plt.figure(figsize=(12, 8))

plt.subplot(2, 1, 1)
char_df["m12"].plot()
plt.title("m12")
plt.ylabel("Parameter Value")
plt.subplot(2, 1, 2)
char_df["mktcap"].plot()
plt.title("mktcap")
plt.ylabel("Parameter Value")
plt.show()
```

```python
weights = pd.DataFrame(weights)
ret_fit = (weights * Ret.tail(60)).sum(axis=1)
ret_EW = (nt * Ret.tail(60)).sum(axis=1)
acc_fit = ret_fit.cumsum()
acc_fitvalue = acc_fit.values
acc_fitvalue = acc_fitvalue[1::]
acc_EW = ret_EW.cumsum()
acc_EWvalue = acc_EW.values

acc_ret = ret.tail(60).cumsum()
top100 = (np.argsort(-acc_ret.tail(1))).iloc[0, :].values[0:100]

acc_top100 = acc_ret.iloc[:, top100].mean(axis=1)
acc_top100value = acc_top100.values

acc_df = pd.DataFrame(index=index)
acc_df["opt"] = acc_fitvalue
acc_df["EW"] = acc_EWvalue
acc_df["top100"] = acc_top100value
acc_df.plot(figsize=(12, 8))
plt.ylabel("Cumulative Return")
plt.show()
```

```python
weights.index = index
lv = (weights[weights < 0]).sum(axis=1).reindex(index)
lv.plot(figsize=(12,8))
plt.ylabel('Leverage')
plt.gca().invert_yaxis()
```

```python
def PPS1(x, wb, nt, ret, m12, mktcap, rr):
    wi = wb + nt*(x[0] * m12 + x[1]* mktcap)
    wret = (wi*ret).sum(axis=1)
    ut = ((1+wret)**(1-rr))/(1-rr)
    u = - (ut.mean())
    Lv = abs((wi[wi<0]).sum(axis=1))
    pen = abs((Lv[Lv>2]*Lv).sum())
    return (u+pen)

### 然后重新进行这个最优化的过程
nt = wb = 1/np.shape(ret)[1]
rr = 5
guess = np.array([0,0])
res_save_pen = []
weights_pen = []
for i in range(0,60):
    opt = sp.optimize.minimize(PPS1, guess, method='BFGS', args=(wb, nt, Ret.iloc[0:48+i,:],
                                                            Scaled_m12.iloc[0:48+i,:], Scaled_mktcap.iloc[0:48+i,:],
                                                            rr))
    print('The {} window'.format(i+1))
    print('The value:', opt['x'])
    res_save_pen.append(opt['x'])
    w = wb + nt*(opt['x'][0]*Scaled_m12.iloc[i + 48,:] + opt['x'][1]*Scaled_mktcap.iloc[i + 48,:])
    print(w)
    weights_pen.append(w)
```

    The 1 window
    The value: [-0.9543712  -7.51789321]
    symbol
    A       0.009796
    AAL     0.006203
    AAP     0.013160
    ABC     0.009796
    ABT    -0.014175
              ...
    XRAY    0.009570
    XRX     0.012427
    YUM     0.007635
    ZBH     0.005968
    ZION    0.009151
    Name: 48, Length: 453, dtype: float64
    The 2 window
    The value: [-0.89505728 -7.83816766]
    symbol
    A       0.011205
    AAL     0.006027
    AAP     0.013753
    ABC     0.008845
    ABT    -0.014034
              ...
    XRAY    0.009966
    XRX     0.012934
    YUM     0.007842
    ZBH     0.005912
    ZION    0.010260
    Name: 49, Length: 453, dtype: float64
    The 3 window
    The value: [-0.48689843 -7.34678018]
    symbol
    A       0.010066
    AAL     0.006205
    AAP     0.011681
    ABC     0.008699
    ABT    -0.012701
              ...
    XRAY    0.009384
    XRX     0.011298
    YUM     0.006425
    ZBH     0.006613
    ZION    0.010823
    Name: 50, Length: 453, dtype: float64
    The 4 window
    The value: [-0.64507729 -7.75819671]
    symbol
    A       0.010805
    AAL     0.007953
    AAP     0.012876
    ABC     0.008058
    ABT    -0.014206
              ...
    XRAY    0.010371
    XRX     0.011840
    YUM     0.008138
    ZBH     0.006754
    ZION    0.011259
    Name: 51, Length: 453, dtype: float64
    The 5 window
    The value: [-1.15099938 -7.67249017]
    symbol
    A       0.010954
    AAL     0.010012
    AAP     0.012940
    ABC     0.007971
    ABT    -0.012607
              ...
    XRAY    0.011225
    XRX     0.011943
    YUM     0.010322
    ZBH     0.006837
    ZION    0.009402
    Name: 52, Length: 453, dtype: float64
    The 6 window
    The value: [-1.68124977 -7.46513938]
    symbol
    A       0.011585
    AAL     0.010517
    AAP     0.011624
    ABC     0.006552
    ABT    -0.009921
              ...
    XRAY    0.011947
    XRX     0.012137
    YUM     0.008823
    ZBH     0.008189
    ZION    0.007264
    Name: 53, Length: 453, dtype: float64
    The 7 window
    The value: [-1.12886332 -7.37765617]
    symbol
    A       0.010348
    AAL     0.006840
    AAP     0.012036
    ABC     0.007806
    ABT    -0.010286
              ...
    XRAY    0.010435
    XRX     0.009906
    YUM     0.007829
    ZBH     0.005553
    ZION    0.008157
    Name: 54, Length: 453, dtype: float64
    The 8 window
    The value: [-1.13222598 -7.64585915]
    symbol
    A       0.008805
    AAL     0.008144
    AAP     0.012348
    ABC     0.007223
    ABT    -0.009049
              ...
    XRAY    0.010317
    XRX     0.009833
    YUM     0.007737
    ZBH     0.006401
    ZION    0.009194
    Name: 55, Length: 453, dtype: float64
    The 9 window
    The value: [-0.55950277 -7.27429434]
    symbol
    A       0.008221
    AAL     0.007912
    AAP     0.011148
    ABC     0.007859
    ABT    -0.008423
              ...
    XRAY    0.009875
    XRX     0.010236
    YUM     0.007233
    ZBH     0.006912
    ZION    0.010772
    Name: 56, Length: 453, dtype: float64
    The 10 window
    The value: [-0.33417868 -7.4661312 ]
    symbol
    A       0.008702
    AAL     0.008734
    AAP     0.010505
    ABC     0.008456
    ABT    -0.010743
              ...
    XRAY    0.009368
    XRX     0.010708
    YUM     0.008169
    ZBH     0.006556
    ZION    0.011258
    Name: 57, Length: 453, dtype: float64
    The 11 window
    The value: [ 0.28183591 -6.93942583]
    symbol
    A       0.008406
    AAL     0.009908
    AAP     0.010094
    ABC     0.009166
    ABT    -0.010787
              ...
    XRAY    0.008408
    XRX     0.011042
    YUM     0.005620
    ZBH     0.006292
    ZION    0.010748
    Name: 58, Length: 453, dtype: float64
    The 12 window
    The value: [ 0.44626173 -7.53510475]
    symbol
    A       0.008757
    AAL     0.010897
    AAP     0.011049
    ABC     0.010055
    ABT    -0.011623
              ...
    XRAY    0.008851
    XRX     0.012301
    YUM     0.005947
    ZBH     0.006719
    ZION    0.011375
    Name: 59, Length: 453, dtype: float64
    The 13 window
    The value: [ 0.21475763 -7.33125765]
    symbol
    A       0.008399
    AAL     0.009315
    AAP     0.010708
    ABC     0.009559
    ABT    -0.010920
              ...
    XRAY    0.009044
    XRX     0.011273
    YUM     0.006813
    ZBH     0.006280
    ZION    0.011117
    Name: 60, Length: 453, dtype: float64
    The 14 window
    The value: [ 0.35647075 -7.04497741]
    symbol
    A       0.008684
    AAL     0.010276
    AAP     0.010760
    ABC     0.009534
    ABT    -0.011572
              ...
    XRAY    0.008737
    XRX     0.011132
    YUM     0.006284
    ZBH     0.006330
    ZION    0.010832
    Name: 61, Length: 453, dtype: float64
    The 15 window
    The value: [-0.34156057 -7.36808611]
    symbol
    A       0.008452
    AAL     0.005541
    AAP     0.009265
    ABC     0.009324
    ABT    -0.010259
              ...
    XRAY    0.010069
    XRX     0.010804
    YUM     0.007402
    ZBH     0.006481
    ZION    0.011125
    Name: 62, Length: 453, dtype: float64
    The 16 window
    The value: [-1.12894638 -7.1357366 ]
    symbol
    A       0.007640
    AAL     0.000472
    AAP     0.007585
    ABC     0.009240
    ABT    -0.008447
              ...
    XRAY    0.011353
    XRX     0.008358
    YUM     0.007322
    ZBH     0.005647
    ZION    0.011410
    Name: 63, Length: 453, dtype: float64
    The 17 window
    The value: [-0.81722137 -6.89477617]
    symbol
    A       0.008094
    AAL     0.000421
    AAP     0.007465
    ABC     0.007562
    ABT    -0.009274
              ...
    XRAY    0.009931
    XRX     0.008798
    YUM     0.007109
    ZBH     0.004878
    ZION    0.012771
    Name: 64, Length: 453, dtype: float64
    The 18 window
    The value: [-0.61805065 -7.4662651 ]
    symbol
    A       0.008531
    AAL     0.001656
    AAP     0.008190
    ABC     0.009161
    ABT    -0.011005
              ...
    XRAY    0.010508
    XRX     0.010500
    YUM     0.007207
    ZBH     0.005607
    ZION    0.013237
    Name: 65, Length: 453, dtype: float64
    The 19 window
    The value: [-1.07253798 -7.3507675 ]
    symbol
    A       0.007848
    AAL    -0.000262
    AAP     0.007109
    ABC     0.007098
    ABT    -0.012250
              ...
    XRAY    0.010811
    XRX     0.008349
    YUM     0.010342
    ZBH     0.005998
    ZION    0.014104
    Name: 66, Length: 453, dtype: float64
    The 20 window
    The value: [-1.09725638 -7.27340231]
    symbol
    A       0.009274
    AAL    -0.002141
    AAP     0.005466
    ABC     0.007745
    ABT    -0.011832
              ...
    XRAY    0.011207
    XRX     0.009082
    YUM     0.010213
    ZBH     0.006622
    ZION    0.014275
    Name: 67, Length: 453, dtype: float64
    The 21 window
    The value: [-1.20463587 -7.30623249]
    symbol
    A       0.009732
    AAL     0.000280
    AAP     0.005213
    ABC     0.007586
    ABT    -0.012802
              ...
    XRAY    0.011640
    XRX     0.009047
    YUM     0.009725
    ZBH     0.005696
    ZION    0.013095
    Name: 68, Length: 453, dtype: float64
    The 22 window
    The value: [-1.05775068 -7.3488353 ]
    symbol
    A       0.010189
    AAL     0.000680
    AAP     0.006607
    ABC     0.007036
    ABT    -0.012865
              ...
    XRAY    0.010732
    XRX     0.008921
    YUM     0.008780
    ZBH     0.004704
    ZION    0.013533
    Name: 69, Length: 453, dtype: float64
    The 23 window
    The value: [-0.36066043 -6.88830993]
    symbol
    A       0.008844
    AAL     0.003805
    AAP     0.008679
    ABC     0.007753
    ABT    -0.011775
              ...
    XRAY    0.009036
    XRX     0.010272
    YUM     0.007715
    ZBH     0.005612
    ZION    0.012138
    Name: 70, Length: 453, dtype: float64
    The 24 window
    The value: [-0.22940295 -7.39457501]
    symbol
    A       0.009713
    AAL     0.004205
    AAP     0.009307
    ABC     0.008413
    ABT    -0.013375
              ...
    XRAY    0.009760
    XRX     0.011207
    YUM     0.008220
    ZBH     0.005852
    ZION    0.012432
    Name: 71, Length: 453, dtype: float64
    The 25 window
    The value: [ 0.8992754  -6.72786514]
    symbol
    A       0.006676
    AAL     0.008470
    AAP     0.011095
    ABC     0.010017
    ABT    -0.012066
              ...
    XRAY    0.008631
    XRX     0.011319
    YUM     0.006750
    ZBH     0.005998
    ZION    0.008203
    Name: 72, Length: 453, dtype: float64
    The 26 window
    The value: [ 0.28392595 -6.59925357]
    symbol
    A       0.008292
    AAL     0.006869
    AAP     0.009556
    ABC     0.008552
    ABT    -0.012514
              ...
    XRAY    0.009103
    XRX     0.010796
    YUM     0.006642
    ZBH     0.005753
    ZION    0.010163
    Name: 73, Length: 453, dtype: float64
    The 27 window
    The value: [ 0.78404651 -6.81165642]
    symbol
    A       0.008221
    AAL     0.007892
    AAP     0.010089
    ABC     0.010814
    ABT    -0.012619
              ...
    XRAY    0.009227
    XRX     0.011074
    YUM     0.006475
    ZBH     0.006357
    ZION    0.009282
    Name: 74, Length: 453, dtype: float64
    The 28 window
    The value: [-0.54128339 -6.82099629]
    symbol
    A       0.009183
    AAL     0.005158
    AAP     0.009549
    ABC     0.004164
    ABT    -0.013078
              ...
    XRAY    0.009267
    XRX     0.012098
    YUM     0.006405
    ZBH     0.006042
    ZION    0.012199
    Name: 75, Length: 453, dtype: float64
    The 29 window
    The value: [ 0.34662373 -6.68019346]
    symbol
    A       0.008413
    AAL     0.007014
    AAP     0.009818
    ABC     0.008346
    ABT    -0.013047
              ...
    XRAY    0.009101
    XRX     0.010238
    YUM     0.006336
    ZBH     0.005716
    ZION    0.010710
    Name: 76, Length: 453, dtype: float64
    The 30 window
    The value: [ 0.85234258 -6.96163978]
    symbol
    A       0.008166
    AAL     0.006488
    AAP     0.010366
    ABC     0.010293
    ABT    -0.013785
              ...
    XRAY    0.009569
    XRX     0.009539
    YUM     0.006617
    ZBH     0.005986
    ZION    0.011310
    Name: 77, Length: 453, dtype: float64
    The 31 window
    The value: [ 2.19320172 -6.83281968]
    symbol
    A       0.007381
    AAL     0.006241
    AAP     0.014019
    ABC     0.011802
    ABT    -0.012907
              ...
    XRAY    0.010837
    XRX     0.005715
    YUM     0.009292
    ZBH     0.005281
    ZION    0.010619
    Name: 78, Length: 453, dtype: float64
    The 32 window
    The value: [ 1.79537923 -6.77731469]
    symbol
    A       0.007205
    AAL     0.007392
    AAP     0.012571
    ABC     0.011528
    ABT    -0.012126
              ...
    XRAY    0.010348
    XRX     0.006325
    YUM     0.008313
    ZBH     0.006669
    ZION    0.010912
    Name: 79, Length: 453, dtype: float64
    The 33 window
    The value: [ 0.47547663 -6.73730621]
    symbol
    A       0.008426
    AAL     0.007540
    AAP     0.009667
    ABC     0.008358
    ABT    -0.010877
              ...
    XRAY    0.009283
    XRX     0.009804
    YUM     0.006776
    ZBH     0.006200
    ZION    0.010736
    Name: 80, Length: 453, dtype: float64
    The 34 window
    The value: [ 0.2527376 -6.7844879]
    symbol
    A       0.008728
    AAL     0.006716
    AAP     0.008856
    ABC     0.007964
    ABT    -0.011194
              ...
    XRAY    0.008669
    XRX     0.010144
    YUM     0.007304
    ZBH     0.005991
    ZION    0.010755
    Name: 81, Length: 453, dtype: float64
    The 35 window
    The value: [ 0.29066476 -6.98488281]
    symbol
    A       0.008574
    AAL     0.006842
    AAP     0.009412
    ABC     0.007964
    ABT    -0.011482
              ...
    XRAY    0.008712
    XRX     0.010249
    YUM     0.007222
    ZBH     0.006223
    ZION    0.011115
    Name: 82, Length: 453, dtype: float64
    The 36 window
    The value: [ 0.4278985  -6.78403067]
    symbol
    A       0.008485
    AAL     0.006068
    AAP     0.008933
    ABC     0.007805
    ABT    -0.011113
              ...
    XRAY    0.008689
    XRX     0.009713
    YUM     0.007184
    ZBH     0.005869
    ZION    0.010628
    Name: 83, Length: 453, dtype: float64
    The 37 window
    The value: [ 2.29416469 -6.73882514]
    symbol
    A       0.009586
    AAL     0.003528
    AAP     0.008921
    ABC     0.007641
    ABT    -0.010201
              ...
    XRAY    0.012359
    XRX     0.005832
    YUM     0.008310
    ZBH     0.004739
    ZION    0.010832
    Name: 84, Length: 453, dtype: float64
    The 38 window
    The value: [ 0.25870832 -6.83953171]
    symbol
    A       0.008524
    AAL     0.006622
    AAP     0.009102
    ABC     0.007674
    ABT    -0.009501
              ...
    XRAY    0.008445
    XRX     0.010163
    YUM     0.006789
    ZBH     0.005915
    ZION    0.010772
    Name: 85, Length: 453, dtype: float64
    The 39 window
    The value: [ 0.28167413 -7.11858694]
    symbol
    A       0.008788
    AAL     0.006668
    AAP     0.009469
    ABC     0.007688
    ABT    -0.010360
              ...
    XRAY    0.008980
    XRX     0.010644
    YUM     0.006835
    ZBH     0.005922
    ZION    0.011122
    Name: 86, Length: 453, dtype: float64
    The 40 window
    The value: [ 0.53057859 -7.15373277]
    symbol
    A       0.008871
    AAL     0.006587
    AAP     0.009945
    ABC     0.007087
    ABT    -0.009339
              ...
    XRAY    0.009532
    XRX     0.010526
    YUM     0.006638
    ZBH     0.005992
    ZION    0.011193
    Name: 87, Length: 453, dtype: float64
    The 41 window
    The value: [ 0.24376097 -7.21245867]
    symbol
    A       0.008697
    AAL     0.008104
    AAP     0.009582
    ABC     0.008257
    ABT    -0.009051
              ...
    XRAY    0.008973
    XRX     0.011052
    YUM     0.006644
    ZBH     0.005538
    ZION    0.011241
    Name: 88, Length: 453, dtype: float64
    The 42 window
    The value: [ 0.20362262 -7.0020921 ]
    symbol
    A       0.008587
    AAL     0.008363
    AAP     0.009134
    ABC     0.008185
    ABT    -0.008354
              ...
    XRAY    0.008638
    XRX     0.010880
    YUM     0.006461
    ZBH     0.005534
    ZION    0.010708
    Name: 89, Length: 453, dtype: float64
    The 43 window
    The value: [ 0.32475113 -7.16884235]
    symbol
    A       0.008688
    AAL     0.007860
    AAP     0.009041
    ABC     0.007946
    ABT    -0.010856
              ...
    XRAY    0.008741
    XRX     0.010950
    YUM     0.006433
    ZBH     0.005658
    ZION    0.010789
    Name: 90, Length: 453, dtype: float64
    The 44 window
    The value: [ 2.45031232 -6.73942657]
    symbol
    A       0.012074
    AAL     0.003559
    AAP     0.003614
    ABC     0.002297
    ABT    -0.012481
              ...
    XRAY    0.009683
    XRX     0.007911
    YUM     0.007228
    ZBH     0.008090
    ZION    0.009393
    Name: 91, Length: 453, dtype: float64
    The 45 window
    The value: [ 2.44946219 -6.70916986]
    symbol
    A       0.012863
    AAL     0.002929
    AAP    -0.000604
    ABC     0.001058
    ABT    -0.010213
              ...
    XRAY    0.009094
    XRX     0.008755
    YUM     0.006417
    ZBH     0.009956
    ZION    0.010230
    Name: 92, Length: 453, dtype: float64
    The 46 window
    The value: [ 0.8066508  -6.90408464]
    symbol
    A       0.009328
    AAL     0.005825
    AAP     0.005934
    ABC     0.005904
    ABT    -0.009069
              ...
    XRAY    0.007686
    XRX     0.011051
    YUM     0.007925
    ZBH     0.005979
    ZION    0.011271
    Name: 93, Length: 453, dtype: float64
    The 47 window
    The value: [ 1.9989516  -6.93433456]
    symbol
    A       0.007866
    AAL     0.007750
    AAP     0.008009
    ABC     0.001776
    ABT    -0.011869
              ...
    XRAY    0.005811
    XRX     0.006836
    YUM     0.009523
    ZBH     0.005027
    ZION    0.015024
    Name: 94, Length: 453, dtype: float64
    The 48 window
    The value: [ 2.105427   -6.89989775]
    symbol
    A       0.007651
    AAL     0.006401
    AAP     0.008819
    ABC    -0.000168
    ABT    -0.012122
              ...
    XRAY    0.004715
    XRX     0.004157
    YUM     0.008488
    ZBH     0.004084
    ZION    0.017764
    Name: 95, Length: 453, dtype: float64
    The 49 window
    The value: [ 1.95877808 -6.95462894]
    symbol
    A       0.009451
    AAL     0.005981
    AAP     0.006889
    ABC     0.004536
    ABT    -0.009261
              ...
    XRAY    0.004463
    XRX     0.008928
    YUM     0.007352
    ZBH     0.005553
    ZION    0.018154
    Name: 96, Length: 453, dtype: float64
    The 50 window
    The value: [ 1.89673657 -6.96843745]
    symbol
    A       0.009912
    AAL     0.005427
    AAP     0.006366
    ABC     0.005631
    ABT    -0.009739
              ...
    XRAY    0.005291
    XRX     0.010067
    YUM     0.007015
    ZBH     0.005658
    ZION    0.019677
    Name: 97, Length: 453, dtype: float64
    The 51 window
    The value: [ 1.99938929 -6.89226094]
    symbol
    A       0.010766
    AAL     0.005299
    AAP     0.004932
    ABC     0.006220
    ABT    -0.009718
              ...
    XRAY    0.005854
    XRX     0.008001
    YUM     0.005747
    ZBH     0.005681
    ZION    0.018394
    Name: 98, Length: 453, dtype: float64
    The 52 window
    The value: [ 1.97334252 -6.93377389]
    symbol
    A       0.010992
    AAL     0.008923
    AAP     0.004637
    ABC     0.005274
    ABT    -0.008064
              ...
    XRAY    0.006752
    XRX     0.010715
    YUM     0.006809
    ZBH     0.003758
    ZION    0.015173
    Name: 99, Length: 453, dtype: float64
    The 53 window
    The value: [ 2.41738242 -6.45571581]
    symbol
    A       0.010607
    AAL     0.013811
    AAP     0.002419
    ABC     0.010013
    ABT    -0.007119
              ...
    XRAY    0.005339
    XRX     0.009071
    YUM     0.008058
    ZBH     0.002000
    ZION    0.015455
    Name: 100, Length: 453, dtype: float64
    The 54 window
    The value: [ 2.28343609 -6.66178159]
    symbol
    A       0.010607
    AAL     0.016154
    AAP    -0.000340
    ABC     0.009116
    ABT    -0.007370
              ...
    XRAY    0.006171
    XRX     0.010739
    YUM     0.007767
    ZBH     0.003947
    ZION    0.019010
    Name: 101, Length: 453, dtype: float64
    The 55 window
    The value: [ 1.20740194 -7.09144041]
    symbol
    A       0.008948
    AAL     0.009867
    AAP     0.004297
    ABC     0.008580
    ABT    -0.009893
              ...
    XRAY    0.007241
    XRX     0.011272
    YUM     0.006761
    ZBH     0.004086
    ZION    0.014783
    Name: 102, Length: 453, dtype: float64
    The 56 window
    The value: [ 0.3533308  -7.05726374]
    symbol
    A       0.007961
    AAL     0.007993
    AAP     0.008918
    ABC     0.008645
    ABT    -0.010275
              ...
    XRAY    0.008650
    XRX     0.011324
    YUM     0.006138
    ZBH     0.005926
    ZION    0.011240
    Name: 103, Length: 453, dtype: float64
    The 57 window
    The value: [ 0.31230485 -7.1939908 ]
    symbol
    A       0.008075
    AAL     0.007977
    AAP     0.009436
    ABC     0.009075
    ABT    -0.011149
              ...
    XRAY    0.008950
    XRX     0.011431
    YUM     0.006439
    ZBH     0.006075
    ZION    0.011350
    Name: 104, Length: 453, dtype: float64
    The 58 window
    The value: [ 0.28801909 -7.09472482]
    symbol
    A       0.007829
    AAL     0.007473
    AAP     0.009242
    ABC     0.009206
    ABT    -0.010418
              ...
    XRAY    0.008754
    XRX     0.010966
    YUM     0.006363
    ZBH     0.006430
    ZION    0.010729
    Name: 105, Length: 453, dtype: float64
    The 59 window
    The value: [ 0.24804893 -7.14193385]
    symbol
    A       0.007925
    AAL     0.007185
    AAP     0.009286
    ABC     0.009089
    ABT    -0.010648
              ...
    XRAY    0.008841
    XRX     0.011218
    YUM     0.006094
    ZBH     0.006895
    ZION    0.010373
    Name: 106, Length: 453, dtype: float64
    The 60 window
    The value: [ 0.54386191 -7.07371611]
    symbol
    A       0.008645
    AAL     0.006943
    AAP     0.007084
    ABC     0.008990
    ABT    -0.009673
              ...
    XRAY    0.008761
    XRX     0.011584
    YUM     0.006559
    ZBH     0.006777
    ZION    0.010154
    Name: 107, Length: 453, dtype: float64

```python
weights_pen = pd.DataFrame(weights_pen)
ret_fit_pen = (weights_pen*Ret.tail(60)).sum(axis=1)
ret_EW_pen = (nt*Ret.tail(60)).sum(axis=1)
acc_fit_pen = ret_fit_pen.cumsum()
acc_fitpenvalue = acc_fit_pen.values
acc_EW_pen = ret_EW_pen.cumsum()
acc_EWpenvalue = acc_EW_pen.values

acc_df_pen = pd.DataFrame()
acc_df_pen['opt'] = acc_fitvalue
acc_fitpenvalue = acc_fitpenvalue[1::]
acc_df_pen['opt_pen'] = acc_fitpenvalue
acc_df_pen['EW'] = acc_EWpenvalue
acc_df_pen['top100'] = acc_top100value

acc_df_pen.index = index
acc_df_pen.plot(figsize=(12,8))
plt.ylabel('Cumulative Return')
```

```python

```


```python
%matplotlib widget
```

## Python Package Imports

```python
%load_ext autoreload
%autoreload 2
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
import matplotlib.cm as cm
from IPython.display import display, Math, clear_output

import sympy
from sympy import *
from sympy.physics.vector import ReferenceFrame, CoordinateSym
from sympy.vector import CoordSys3D, divergence, curl

import ipyvolume as ipv
import time
from ipywidgets import Output

import helpers

np.seterr(divide='ignore', invalid='ignore');
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

## Demonstrate a surface (or mesh or wireframe) plot.
### The input data to a 3D surface plot is 3 vectors (i.e. x,y,z). These vectors define a surface; the surface is not a vector field, nor is it a scalar value defined at all points in space.

```python
# Create the data.
dphi, dtheta = np.pi/250.0, pi/250.0
[phi,theta] = np.mgrid[0:np.pi+dphi*1.5:dphi,0:2*np.pi+dtheta*1.5:dtheta]
m0 = 4; m1 = 3; m2 = 2; m3 = 3; m4 = 6; m5 = 2; m6 = 6; m7 = 4;
r = np.sin(m0*phi)**m1 + np.cos(m2*phi)**m3 + np.sin(m4*theta)**m5 + np.cos(m6*theta)**m7
x = r*np.sin(phi)*np.cos(theta)
y = r*np.cos(phi)
z = r*np.sin(phi)*np.sin(theta)

# And surface/mesh plots, showing surfaces or wireframes.
ipv.clear()
ipv.figure()
ipv.plot_mesh(x, y, z, wireframe=False)
ipv.squarelim()
ipv.show()
```

    A Jupyter Widget

## Demonstrate a quiver plot, which shows a vector at each position
### The input data to a 3D quiver plot is:
1. a sequence of three vectors which define the position of each point **and**
2. a sequence of three vectors which define the magnitude and direction at each point.

```python
# quiver plot, vector at each position
x, y, z, u, v, w = np.random.random((6, 1000))*2-1

print('Shape of x: {}'.format(np.shape(x)[0]))
print('Shape of u: {}'.format(np.shape(u)[0]))

quiver = ipv.quickquiver(x, y, z, u, v, w, size=5)
ipv.show()
```

    Shape of x: 1000
    Shape of u: 1000

    A Jupyter Widget

## A quiver plot with sliders

```python
x, y, z, u, v, w = np.random.random((6, 1000))*2-1
selected = np.random.randint(0, 1000, 100)
ipv.figure()
quiver = ipv.quiver(x, y, z, u, v, w, size=5, size_selected=8, selected=selected)

from ipywidgets import FloatSlider, ColorPicker, VBox, jslink

size = FloatSlider(min=0, max=30, step=0.1)
size_selected = FloatSlider(min=0, max=30, step=0.1)
color = ColorPicker()
color_selected = ColorPicker()
jslink((quiver, 'size'), (size, 'value'))
jslink((quiver, 'size_selected'), (size_selected, 'value'))
jslink((quiver, 'color'), (color, 'value'))
jslink((quiver, 'color_selected'), (color_selected, 'value'))
VBox([ipv.gcc(), size, size_selected, color, color_selected])
```

    A Jupyter Widget


# Frequent-Pattern Analysis

**CS5483 Data Warehousing and Data Mining**
$\def\abs#1{\left\lvert #1 \right\rvert}
\def\Set#1{\left\{ #1 \right\}}
\def\mc#1{\mathcal{#1}}
\def\M#1{\boldsymbol{#1}}
\def\R#1{\mathsf{#1}}
\def\RM#1{\boldsymbol{\mathsf{#1}}}
\def\op#1{\operatorname{#1}}
\def\E{\op{E}}
\def\d{\mathrm{\mathstrut d}}
$
___

$\def\abs#1{\left\lvert #1 \right\rvert}
\def\Set#1{\left\{ #1 \right\}}
\def\mc#1{\mathcal{#1}}
\def\M#1{\boldsymbol{#1}}
\def\R#1{\mathsf{#1}}
\def\RM#1{\boldsymbol{\mathsf{#1}}}
\def\op#1{\operatorname{#1}}
\def\E{\op{E}}
\def\d{\mathrm{\mathstrut d}}
$

```python
%reset -f
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
# produce vector inline graphics
from IPython.display import set_matplotlib_formats
set_matplotlib_formats('svg')
from ipywidgets import interact
import ipywidgets as widgets
import weka.core.jvm as jvm
from weka.core.converters import Loader
from weka.associations import Associator
import javabridge
```

## Association Rule Mining using Weka

We will conduct the market-basket analysis on the supermarket dataset in Weka.

### Transaction data

Each instance of the dataset is a transaction, i.e., the purchase of items in a supermarket by a customer. Mathematically, the dataset can be represented by

\begin{align}
D &:= \Set{T_i}_{i=1}^{n} \quad \text{where}\\
T_i&\subseteq \mathcal{I},
\end{align}

and $\mathcal{I}$ is the collection of all items. A transaction $T_i$ is simply a subset of items.

Using the Explorer interface, load the `supermarket.arff` dataset in Weka.

Note that most attribute contains only one possible value, namely `t`. Click the button `Edit...` to open the data editor. Observe that most attributes have missing valu:

In `supermarket.arff`:
- Each attribute specified by `@attribute` can be a product category, a department, or a product with one possible value `t`:
```
...
@attribute 'grocery misc' { t}
@attribute 'department11' { t}
@attribute 'baby needs' { t}
@attribute 'bread and cake' { t}
...
```
- The last attribute `'total'` has two possible values `{low, high}`:
```
@attribute 'total' { low, high} % low < 100
```

To understand the dataset further:
1. Select the `Associate` tab. By default, `Apriori` is chosen as the `Associator`.
1. Open the `GenericObjectEditor` and check that there is a parameter called `treatZeroAsMissing`. Hover the mouse pointer over the parameter to see more details.
1. Run the Apriori algorithm with different choices of the parameter `treatZeroAsMissing`. Observe the diference in the generated rules.

**Exercise** Explain what `t` and `?` means in the dataset when we set `treatZeroAsMissing` to `True` and `False` respectively.

*Hint:* See the [documentation](https://weka.sourceforge.io/doc.dev/weka/associations/Apriori.html) of `Apriori` `Associator`.

YOUR ANSWER HERE

### Association rules

An association rule can be represent as

\begin{align}
A \implies B,
\end{align}

which means that a transaction contains all items in $B$ if it contains all items in $A$, i.e.,

\begin{align}
\underbrace{A\subseteq T}_{\text{premise}} \implies \underbrace{B\subseteq T}_{\text{consequence}}
\end{align}

for transaction $T\in D$.

We will use [`python-weka-wrapper`](https://github.com/fracpete/python-weka-wrapper3-examples/blob/master/src/wekaexamples/associations/apriori_output.py) for illustration. To load the dataset:

```python
jvm.start()
from weka.core.converters import Loader
loader = Loader(classname="weka.core.converters.ArffLoader")
weka_data_path = 'https://raw.githubusercontent.com/Waikato/weka-3.8/master/wekadocs/data/'
dataset = loader.load_url(
    weka_data_path +
    'supermarket.arff')  # use load_file to load from file instead
```

To apply the apriori algorithm with the default settings:

```python
from weka.associations import Associator

apriori = Associator(classname="weka.associations.Apriori")
apriori.build_associations(dataset)
apriori
```

**Exercise** Explain what the first rule means according to the notation $A\implies B$.

*Hint:* Note that `biscuits=t` and `total=high` are both regarded as items. Since `total` has to possible values, it is associated with two items, the other one being `total=low`.

The first rule means that if a customer buys biscuits, frozen foods, fruit, and has a high total value, he/she will buy bread and cake.

To retrieve the rules as a list, and print the first rule:

```python
rules = list(apriori.association_rules())
rules[0]
```

To obtain the set $A$ (in premise) and $B$ (in consequence):

```python
rules[0].premise, rules[0].consequence
```

```python
premise_support = rules[0].premise_support
total_support = rules[0].total_support

```

The apriori algorithm returns rules with large enough support:
\begin{align}
\op{support}(A \implies B) &= \op{support}(A \cup B) :=
\frac{\op{count}(A \cup B)}{|D|}\quad \text{where}\\
\op{count(A \cup B)} &:= \abs{\Set{T\in D|T\supseteq A\cup B}}.
\end{align}

In words, support is the fraction of transactions containing both $A$ and $B$.

For the first rule, the number 723 at the end of the rule corresponds to the total support count $\op{count}(A\cup B)$.

**Exercise** Assign to `support` the (fractional) support for the first rule (`rules[0]`).

*Hint:* In `python-weka-wrapper`, you can use the properties `total_support` and `total_transactions` of `rules[0]`.

```python
# YOUR CODE HERE
raise NotImplementedError()
support
```

```python
# tests
```

By default, the rules are ranked by confidence.

\begin{align}
\op{confidence}(A\implies B) &:= \frac{\op{support(A \cup B)}}{\op{support(A)}},
\end{align}

where the denominator $\op{support}(A)$ is the support of the premise. It gives, out of the transactions containing $A$, the fraction of transactions containing $B$.

`<conf:(0.92)> lift:(1.27) lev:(0.03) conv:(3.35)` at the end of the first rule indicates with that confidence is used for ranking the rules and the rule has a confidence of 0.92. In `python-weka-wrapper`, we can print different metrics as follows:

```python
for n, v in zip(rules[0].metric_names, rules[0].metric_values):
    print(f'{n}: {v:.3g}')
```

**Exercise** Assign to `premise_support` the support count $\op{count}(A)$ of the premise for the first rule.

```python
# YOUR CODE HERE
raise NotImplementedError()
premise_support
```

```python
# tests
```

Lift is another rule quality measure defined as

\begin{align}
\op{lift}(A\implies B) &:= \frac{\op{confidence}(A\implies B)}{\op{support(B)}}\\
&= \frac{\op{support(A \cup B)}}{\op{support(A)}\op{support(B)}}.
\end{align}

$\op{support}(B)$ in the denominator is the support of the consequence, which can be rewritten as

\begin{align}
\op{confidence}(\emptyset \implies B) &= \frac{\op{support}(B)}{\op{support}(\emptyset)} = \op{support}(B)
\end{align}

and so

\begin{align}
\op{lift}(A\implies B) &:= \frac{\op{confidence}(A\implies B)}{\op{confidence}(\emptyset \implies B)}.
\end{align}

In other words, lift is the fractional increase in confidence by imposing the premise.

**Exercise** In Weka, we can change the parameter `metricType` to rank the rule according to `Lift` instead of `Confidence`. Run the algorithm again with `metricType = Lift`. Assign to `lift` the maximum lift achieved. For `python-weka-wrapper`, you can specify the option as follows:

```Python
apriori_lift = Associator(classname="weka.associations.Apriori", options=['-T', '1'])
...
```
where the value `1` corresponds to `Lift`.

```python
# YOUR CODE HERE
raise NotImplementedError()
lift
```

```python
# tests
```

**Exercise** What is the relationship between the first and second rule? Explain your answer based on the symmetry of lift.

From the [documentation](https://weka.sourceforge.io/doc.dev/weka/associations/Apriori.html), the apriori algorithm in Weka reduces the minimum support until it obtains a specified number (default: 10) of rules with specified minimum metric value for the metric type.

**Exercise** Explain why the maximum lift obtained by ranking the rules using `Lift` is smaller than 1.27, which is the lift obtained before by ranking rules using `Confidence`.

YOUR ANSWER HERE