# Minimum Trajectories

## Overview
Notebook and code to derive minimum trajectories. This notebook starts with minimum snap trajectories between 2 points and then explores multi-waypoint minimum snap trajectories. Finally, we finish off with lower derivative minimum trajectories (jerk, acceleration and velocity), which are derivied in a similar fashion but with less coefficients and boundary conditions.

**DISCLAIMER:** I only have a loose understanding of the math involved - please feel free to submit a pull request with a better/accurate explanation.

To control a quadcopter through a give set of waypoints, we desire a smooth trajectory. The criteria of smoothness can be defined as minimizing the rate of change of the input or minimizing a lower derivative of position.

[Calculus of variations](https://en.wikipedia.org/wiki/Calculus_of_variations) can be used to minimize a lower derivative function to achieve the desired smoothness: "Functions that maximize or minimize functionals may be found using the [Euler–Lagrange](https://en.wikipedia.org/wiki/Euler%E2%80%93Lagrange_equation) equation of the calculus of variations".

For a function $x(t)$, the stationary values of the functional:

$I[x] = \int_{t=t_s}^{t_e} \mathcal{L}(t, x, \dot{x}, \ddot{x}, \ldots, x^{(n-1)}, x^{(n)})\,dt$

can be obtained from the Euler–Lagrange equation:

$\dfrac{\partial\mathcal{L}}{\partial{x}} - \dfrac{d}{dt}\dfrac{\partial\mathcal{L}}{\partial{\dot{x}}} + \dfrac{d^2}{dt^2}\dfrac{\partial\mathcal{L}}{\partial{\ddot{x}}} - \ldots + (-1)^k\dfrac{d^n}{dt^n}\dfrac{\partial\mathcal{L}}{\partial{x^{(n)}}} = 0$

To find the shortest distance:

$min I[x] = \int_{t=t_s}^{t_e} \mathcal{L}(t, x, \dot{x})\,dt = \int_{t=t_s}^{t_e} (\dot{x})^2\, dt$

Satisfying the requirements of the Euler–Lagrange equation:

$\dfrac{\partial\mathcal{L}}{\partial{x}} - \dfrac{d}{dt}\dfrac{\partial\mathcal{L}}{\partial{\dot{x}}} = 0$ resulting in $\ddot{x} = 0$

This means we wish to find a polynomial that has its 2nd time derivative equal to 0, or a polynomial with 2 coefficients:

$x = c_1t + c_0$

and in general ( **TODO:** need a reference/explanation here as to why the functional is $(x^{(n)})^2$ ):

$min I[x] = \int_{t=t_s}^{t_e} (x^{(n)})^2\, dt \\
x^{(2n)} = 0$

and we will need a polynomial with degree $2n - 1$ to satisfy the Euler-Lagrange requirement.

Summary of the minimum trajectory types for values of $n$:

| $n$ | Minimum Type | Euler–Lagrange | Poly. Degree | Num. Coeffs. |
| --- | ------------ | -------------- | ------------ | ------------ |
| 1   | Velocity     | $\ddot{x} = 0$ | 1            | 2            |
| 2   | Acceleration | $x^{(4)} = 0$  | 3            | 4            |
| 3   | Jerk         | $x^{(6)} = 0$  | 5            | 6            |
| 4   | Snap         | $x^{(8)} = 0$  | 7            | 8            |

## Minimum Snap Trajectory
A minimum snap trajectory can be used to create smooth trajectories to control a quadcopter since the control system for a quadcopter is a fourth order system.

We want a trajectory defining position of the form:
$x(t) = c_7t^7 + c_6t^6 + c_5t^5 + c_4t^4 + c_3t^3 + c_2t^2 + c_1t + c_0$

Once we derive the coefficients, we can then differentiate the resulting polynomial to get the desired velocity and acceleration polynomials, and use these to calculate the desired position, velocity, and accelerations at any time $t$ to give us a minimum snap trajectory, that we can then feed into our position controller.

### Generating a Trajectory for 2 Points
For a set of 2 points a and b where the time at position a is $t_s$, and the time at position b is $t_e$, we assume that the velocity and all further time deriviates will be zero (i.e. the object following the trajectory starts and ends at rest):

|               | Position      | Velocity | Acceleration | Jerk |
| ------------- | ------------- | -------- | ------------ | ---- |
| $t = t_s$     | $a$           | 0        | 0            | 0    |
| $t = t_e$     | $b$           | 0        | 0            | 0    |

This means we will have 8 unknowns and will need 8 equations to determine the coefficients of the polynomial that will give us the minimum snap trajectory over the given time period. We will be able to evaluate the position, velocity, acceleration, and jerk equations at the start and end times to generate the required equations. Note that this will be in 1-dimension, but we can repeat the process for each dimension - e.g. if we want to generate a trajectory in 3D space, the process will be the same for x, y, and z and we can re-use the A matrix for each dimension.

The equations are as follows:
$c_{7}t_s^7 + c_{6}t_s^6 + c_{5}t_s^5 + c_{4}t_s^4 + c_{3}t_s^3 + c_{2}t_s^2 + c_{1}t_s + c_{0} = a \\
 c_{7}t_e^7 + c_{6}t_e^6 + c_{5}t_e^5 + c_{4}t_e^4 + c_{3}t_e^3 + c_{2}t_e^2 + c_{1}t_e + c_{0} = b \\
 7c_{7}t_s^6 + 6c_{6}t_s^5 + 5c_{5}t_s^4 + 4c_{4}t_s^3 + 3c_{3}t_s^2 + 2c_{2}t_s + c_{1} = 0 \\
 7c_{7}t_e^6 + 6c_{6}t_e^5 + 5c_{5}t_e^4 + 4c_{4}t_e^3 + 3c_{3}t_e^2 + 2c_{2}t_e + c_{1} = 0 \\
 42c_{7}t_s^5 + 30c_{6}t_s^4 + 20c_{5}t_s^3 + 12c_{4}t_s^2 + 6c_{3}t_s + 2c_{2} = 0 \\
 42c_{7}t_e^5 + 30c_{6}t_e^4 + 20c_{5}t_e^3 + 12c_{4}t_e^2 + 6c_{3}t_e + 2c_{2} = 0 \\
 210c_{7}t_s^4 + 120c_{6}t_s^3 + 60c_{5}t_s^2 + 24c_{4}t_s + 6c_{3} = 0 \\
 210c_{7}t_e^4 + 120c_{6}t_e^3 + 60c_{5}t_e^2 + 24c_{4}t_e + 6c_{3} = 0$

We can form an 8x8 matrix A using the above equations, and we can let x be a column vector of the coefficients we wish to find, and b will be a column vector of the RHS. We can then form the system $Ax = b$ and then we can solve for $x = A^{-1}b$

$x = \begin{bmatrix}
c_0 \\
c_1 \\
c_2 \\
c_3 \\
c_4 \\
c_5 \\
c_6 \\
c_7
\end{bmatrix}$ ,
$A = \begin{bmatrix}
1 & t_s & t_s^2 & t_s^3 & t_s^4 & t_s^5 & t_s^6 & t_s^7 \\
1 & t_e & t_e^2 & t_e^3 & t_e^4 & t_e^5 & t_e^6 & t_e^7 \\
0 & 1 & 2t_s & 3t_s^2 & 4t_s^3 & 5t_s^4 & 6t_s^5 & 7t_s^6 \\
0 & 1 & 2t_e & 3t_e^2 & 4t_e^3 & 5t_e^4 & 6t_e^5 & 7t_e^6 \\
0 & 0 & 2 & 6t_s & 12t_s^2 & 20t_s^3 & 30t_s^4 & 42t_s^5 \\
0 & 0 & 2 & 6t_e & 12t_e^2 & 20t_e^3 & 30t_e^4 & 42t_e^5 \\
0 & 0 & 0 & 6 & 24t_s & 60t_s^2 & 120t_s^3 & 210t_s^4 \\
0 & 0 & 0 & 6 & 24t_e & 60t_e^2 & 120t_e^3 & 210t_e^4
\end{bmatrix}$ ,
$b = \begin{bmatrix}
a \\
b \\
0 \\
0 \\
0 \\
0 \\
0 \\
0
\end{bmatrix}$

```python
import sympy as sp
import numpy as np
import matplotlib.pyplot as plt

sp.init_printing()
```

```python
t, T = sp.symbols('t, T')

# define our position and time derivatives
pos = t**7 + t**6 + t**5 + t**4 + t**3 + t**2 + t + 1
vel = pos.diff(t)
acc = vel.diff(t)
jerk = acc.diff(t)
snap = jerk.diff(t)
crac = snap.diff(t)
pop = crac.diff(t)

# print all the equations
eqs = [pos, vel, acc, jerk, snap, crac, pop]
for eq in eqs:
    display(eq)
```

```python
# define matrix print function, courtesy of: https://gist.github.com/braingineer/d801735dac07ff3ac4d746e1f218ab75
def matprint(mat, fmt="g"):
    col_maxes = [max([len(("{:"+fmt+"}").format(x)) for x in col]) for col in mat.T]
    for x in mat:
        for i, y in enumerate(x):
            print(("{:"+str(col_maxes[i])+fmt+"}").format(y), end="  ")
        print("")
```

```python
# since we want to find 8 coefficients, we will need 8 equations, so we can evaluate pos, velocity, acceleration and jerk at time = 0 and time = T
# we will solve the linear system Ax = b - each row of A will be the coefficients of the evaluated polynomials at the boundary conditions,
# where the lowest index in the row, 0, corresponds to coefficient 0
numcoeffs = 8
A = np.zeros((numcoeffs, numcoeffs))

# our time range for this polynomial
startt = 1
endt = 6

# let's assume we are starting from (1, 2) and moving to (12, 9) with the above time range
start = (1, 2)
end = (12, 9)

def coeffs_for_time(equations, numcoeffs, time):
    retval = np.zeros((len(equations), numcoeffs))

    for ii, eq in enumerate(equations):
        # pull out the coefficients - ordered from highest to lowest poly order, e.g.: t^3, t^2, t^1, t^0
        coeffs = eq.as_poly().all_coeffs()

        # apply time
        exp = len(coeffs) - 1
        for idx in range(len(coeffs)):
            coeffs[idx] *= time**exp
            exp -= 1

        # pad out with zeros if some of the coefficients are 0
        while len(coeffs) < numcoeffs:
            coeffs.append(0)

        # reverse them to match up with our coeffs vector
        retval[ii] = list(reversed(coeffs))
    return retval

half = int(numcoeffs/2)
A[0:half,:] = coeffs_for_time(eqs[0:half], numcoeffs, startt)
A[half:,:] = coeffs_for_time(eqs[0:half], numcoeffs, endt)

# print our A matrix
print('A = [')
matprint(A)
print(']')
```

    A = [
    1  1   1    1     1     1      1       1
    0  1   2    3     4     5      6       7
    0  0   2    6    12    20     30      42
    0  0   0    6    24    60    120     210
    1  6  36  216  1296  7776  46656  279936
    0  1  12  108   864  6480  46656  326592
    0  0   2   36   432  4320  38880  326592
    0  0   0    6   144  2160  25920  272160
    ]

```python
# so now we can create our b vectors
bx = np.zeros((numcoeffs, 1))
by = np.zeros((numcoeffs, 1))

bx[0] = start[0]
bx[half] = end[0]
by[0] = start[1]
by[half] = end[1]

# print out our b vectors
print('bx = [')
matprint(bx)
print(']\n\nby = [')
matprint(by)
print(']')
```

    bx = [
     1
     0
     0
     0
    12
     0
     0
     0
    ]

    by = [
    2
    0
    0
    0
    9
    0
    0
    0
    ]

```python
# now we can solve for the coefficients
bxc = np.linalg.solve(A, bx)
byc = np.linalg.solve(A, by)
print('bxc = [')
matprint(bxc)
print(']\n\nbyc = [')
matprint(byc)
print(']')
```

    bxc = [
      1.96378
     -4.25779
      7.45114
     -6.50496
      2.93216
    -0.650496
     0.068992
    -0.002816
    ]

    byc = [
      2.61331
      -2.7095
      4.74163
     -4.13952
      1.86592
    -0.413952
     0.043904
    -0.001792
    ]

```python
plot_colors = ['r', 'b', 'g', 'y', 'm', 'c'] * 2
derivative_labels = ['position', 'velocity', 'acceleration', 'jerk', 'snap', 'crack', 'pop']

# define a plotting function
def plot_all(tvals, xpoly, ypoly, numderivatives=3):
    plt.figure(figsize=(20, 8*numderivatives))

    numplots = 2*numderivatives
    numcols = 2
    label_idx = 0

    for plot_idx in range(1, numderivatives*2+1, 2):
        dlabel = derivative_labels[label_idx]

        # plot x
        xvals = [xpoly.eval(tval) for tval in tvals]
        plt.subplot(numplots, numcols, plot_idx)
        plt.plot(tvals, xvals, plot_colors[plot_idx-1])
        plt.ylabel(f'X {dlabel}')
        plt.grid(True)

        plot_idx += 1

        # plot y
        yvals = [ypoly.eval(tval) for tval in tvals]
        plt.subplot(numplots, numcols, plot_idx)
        plt.plot(tvals, yvals, plot_colors[plot_idx-1])
        plt.ylabel(f'Y {dlabel}')
        plt.grid(True)

        # compute next derivative
        xpoly = xpoly.diff(t)
        ypoly = ypoly.diff(t)

        label_idx += 1

    plt.show()
```

```python
# create the polynomials so we can differentiate them to get our various commanded position, velocity and accelleration curves
bxpoly = sp.Poly(reversed(bxc.transpose()[0]), t)
bypoly = sp.Poly(reversed(byc.transpose()[0]), t)

# create our t values over the specified range
tvals = np.linspace(startt, endt, 100)

# display the position, velocity and acceleration curves
plot_all(tvals, bxpoly, bypoly, numderivatives=6)
```

#### Non-Zero Time Derivative Boundary Conditions
Trajectories can also be generated by assuming non-zero boundary conditions for velocity, and further time derivatives. One use case could be the desire for the trajectory to smoothly transition to/from a steady-state velocity or even start a some velocity and end at another velocity (in which case the acceleration and jerk would continue to be 0 at the boundaries).

By assigning these non-zero velocity values in our b vectors, we can generate the minimum snap trajectory that transitions to these velocities.

This is also tackled in the next section for multi-waypoint trajectories where all that is required are the desired waypoints and times of arrival, and then the intermediate velocities and further time derivative values are calculated automatically.

```python
# init our b vectors
bx = np.zeros((numcoeffs, 1))
by = np.zeros((numcoeffs, 1))

# assign the start and end positions and velocities - in this case we start with intial velocities of (0,0) and end with (4,3)
# NOTE: if you wanted to start with non-zero velocities, then you would assign non-zero values to bx[1] and by[1]
bx[0] = start[0]
bx[half] = end[0]
bx[half+1] = 4  # we want our end velocity in the x direction to be 4 m/s
by[0] = start[1]
by[half] = end[1]
by[half+1] = 3  # we want our end velocity in the y direction to be 3 m/s

# now we can solve for the coefficients
bxc = np.linalg.solve(A, bx)
byc = np.linalg.solve(A, by)

# create the polynomials so we can differentiate them to get our various commanded position, velocity and accelleration curves
bxpoly = sp.Poly(reversed(bxc.transpose()[0]), t)
bypoly = sp.Poly(reversed(byc.transpose()[0]), t)

# create our t values over the specified range
tvals = np.linspace(startt, endt, 100)

# display the position, velocity and acceleration curves
plot_all(tvals, bxpoly, bypoly)
```

### Multi-point Trajectories

For trajectories that contain $n$ points, we will need to generate $n-1$ polynomials. This means we will need to find $8(n-1)$ coefficients - where each set of 8 corresponds to one segment. So we must form $8(n-1)$ equations in order to solve for the coefficients, where coefficients for segment $i$ are:
$c_{i,7}, c_{i,6}, c_{i,5}, c_{i,4}, c_{i,3}, c_{i,2}, c_{i,1}, c_{i,0}$

The boundary conditions are slightly different in this case, since we don't want the trajectory to stop at intermediary points but smoothly traverse through them - so we want to ensure that the transitions between polynomials is continuous.

For a set of points, we will also assume we have a desired time when each point should be reached. For a set of n points, we will also have n times, where the following condition is satisifed:
$t_{i-1} < t_i < t_{i+1}$

For all segments, we know that the polynomial must pass through the start and end point for that segment. This gives us $2(n-1)$ equations, so for any segment $i$ with point $a_i$:
$c_{i,7}t_i^7 + c_{i,6}t_i^6 + c_{i,5}t_i^5 + c_{i,4}t_i^4 + c_{i,3}t_i^3 + c_{i,2}t_i^2 + c_{i,1}t_i + c_{i,0} = a_i \\
c_{i,7}t_{i+1}^7 + c_{i,6}t_{i+1}^6 + c_{i,5}t_{i+1}^5 + c_{i,4}t_{i+1}^4 + c_{i,3}t_{i+1}^3 + c_{i,2}t_{i+1}^2 + c_{i,1}t_{i+1} + c_{i,0} = a_{i+1}$

Each segment, except for the last, will have the condition that at it's end time, the velocity and all its further time derivatives, will equal the velocity, and all its further time derivatives of the following segment's start time. If we use velocity, acceleration, jerk, snap, crackle and pop, we will have a further $6(n-2)$ equations:
$7c_{i,7}t_{i+1}^6 + 6c_{i,6}t_{i+1}^5 + 5c_{i,5}t_{i+1}^4 + 4c_{i,4}t_{i+1}^3 + 3c_{i,3}t_{i+1}^2 + 2c_{i,2}t_{i+1} + c_{i,1} = 7c_{i+1,7}t_{i+1}^6 + 6c_{i+1,6}t_{i+1}^5 + 5c_{i+1,5}t_{i+1}^4 + 4c_{i+1,4}t_{i+1}^3 + 3c_{i+1,3}t_{i+1}^2 + 2c_{i+1,2}t_{i+1} + c_{+1i,1} \\
42c_{i,7}t_{i+1}^5 + 30c_{i,6}t_{i+1}^4 + 20c_{i,5}t_{i+1}^3 + 12c_{i,4}t_{i+1}^2 + 6c_{i,3}t_{i+1} + 2c_{i,2} = 42c_{i+1,7}t_{i+1}^5 + 30c_{i+1,6}t_{i+1}^4 + 20c_{i+1,5}t_{i+1}^3 + 12c_{i+1,4}t_{i+1}^2 + 3c_{i+1,3}t_{i+1} + 2c_{i+1,2} \\
210c_{i,7}t_{i+1}^4 + 120c_{i,6}t_{i+1}^3 + 60c_{i,5}t_{i+1}^2 + 24c_{i,4}t_{i+1} + 6c_{i,3} = 210c_{i+1,7}t_{i+1}^4 + 120c_{i+1,6}t_{i+1}^3 + 60c_{i+1,5}t_{i+1}^2 + 24c_{i+1,4}t_{i+1} + 6c_{i+1,3} \\
840c_{i,7}t_{i+1}^3 + 360c_{i,6}t_{i+1}^2 + 120c_{i,5}t_{i+1} + 24c_{i,4} = 840c_{i+1,7}t_{i+1}^3 + 360c_{i+1,6}t_{i+1}^2 + 120c_{i+1,5}t_{i+1} + 24c_{i+1,4} \\
2520c_{i,7}t_{i+1}^2 + 720c_{i,6}t_{i+1} + 120c_{i,5} = 2520c_{i+1,7}t_{i+1}^2 + 720c_{i+1,6}t_{i+1} + 120c_{i+1,5} \\
5040c_{i,7}t_{i+1} + 720c_{i,6}t_{i+1} = 5040c_{i+1,7}t_{i+1} + 720c_{i+1,6}$

or, by moving all terms to LHS:

$7c_{i,7}t_{i+1}^6 + 6c_{i,6}t_{i+1}^5 + 5c_{i,5}t_{i+1}^4 + 4c_{i,4}t_{i+1}^3 + 3c_{i,3}t_{i+1}^2 + 2c_{i,2}t_{i+1} + c_{i,1} - 7c_{i+1,7}t_{i+1}^6 - 6c_{i+1,6}t_{i+1}^5 - 5c_{i+1,5}t_{i+1}^4 - 4c_{i+1,4}t_{i+1}^3 - 3c_{i+1,3}t_{i+1}^2 - 2c_{i+1,2}t_{i+1} - c_{i+1,1} = 0 \\
42c_{i,7}t_{i+1}^5 + 30c_{i,6}t_{i+1}^4 + 20c_{i,5}t_{i+1}^3 + 12c_{i,4}t_{i+1}^2 + 6c_{i,3}t_{i+1} + 2c_{i,2} - 42c_{i+1,7}t_{i+1}^5 - 30c_{i+1,6}t_{i+1}^4 - 20c_{i+1,5}t_{i+1}^3 - 12c_{i+1,4}t_{i+1}^2 - 3c_{i+1,3}t_{i+1} - 2c_{i+1,2} = 0 \\
210c_{i,7}t_{i+1}^4 + 120c_{i,6}t_{i+1}^3 + 60c_{i,5}t_{i+1}^2 + 24c_{i,4}t_{i+1} + 6c_{i,3} - 210c_{i+1,7}t_{i+1}^4 - 120c_{i+1,6}t_{i+1}^3 - 60c_{i+1,5}t_{i+1}^2 - 24c_{i+1,4}t_{i+1} - 6c_{i+1,3} = 0 \\
840c_{i,7}t_{i+1}^3 + 360c_{i,6}t_{i+1}^2 + 120c_{i,5}t_{i+1} + 24c_{i,4} - 840c_{i+1,7}t_{i+1}^3 - 360c_{i+1,6}t_{i+1}^2 - 120c_{i+1,5}t_{i+1} - 24c_{i+1,4} = 0 \\
2520c_{i,7}t_{i+1}^2 + 720c_{i,6}t_{i+1} + 120c_{i,5} - 2520c_{i+1,7}t_{i+1}^2 - 720c_{i+1,6}t_{i+1} - 120c_{i+1,5} = 0 \\
5040c_{i,7}t_{i+1} + 720c_{i,6}t_{i+1} - 5040c_{i+1,7}t_{i+1} - 720c_{i+1,6} = 0$

We now only need 6 more equations which we can get from the first and last segments:
* the first segment will have zero velocity, acceleration and jerk for its start point - this gives us 3 more equations
  $7c_{0,7}t_0^6 + 6c_{0,6}t_0^5 + 5c_{0,5}t_0^4 + 4c_{0,4}t_0^3 + 3c_{0,3}t_0^2 + 2c_{0,2}t_0 + c_{0,1} = 0 \\
  42c_{0,7}t_0^5 + 30c_{0,6}t_0^4 + 20c_{0,5}t_0^3 + 12c_{0,4}t_0^2 + 6c_{0,3}t_0 + 2c_{0,2} = 0 \\
  210c_{0,7}t_0^4 + 120c_{0,6}t_0^3 + 60c_{0,5}t_0^2 + 24c_{0,4}t_0 + 6c_{0,3} = 0$
* the last segment will have zero velocity, acceleration and jerk for its its end point. This gives us 3 more equation
  $7c_{n-1,7}t_{n-1}^6 + 6c_{n-1,6}t_{n-1}^5 + 5c_{n-1,5}t_{n-1}^4 + 4c_{n-1,4}t_{n-1}^3 + 3c_{n-1,3}t_{n-1}^2 + 2c_{n-1,2}t_{n-1} + c_{n-1,1} = 0 \\
  42c_{n-1,7}t_{n-1}^5 + 30c_{n-1,6}t_{n-1}^4 + 20c_{n-1,5}t_{n-1}^3 + 12c_{n-1,4}t_{n-1}^2 + 6c_{n-1,3}t_{n-1} + 2c_{n-1,2} = 0 \\
  210c_{n-1,7}t_{n-1}^4 + 120c_{n-1,6}t_{n-1}^3 + 60c_{n-1,5}t_{n-1}^2 + 24c_{n-1,4}t_{n-1} + 6c_{n-1,3} = 0$

Once we have the above equations evaluated at the correct times, we can again use linear algebra to determine the set of polynomial coefficients to give us a minimum snap trajectory over multiple points, where the intermediary points will transition smoothly.

```python
# setup our desired trajectory points and times of arrival
points = [(1, 2), (12, 9), (20, 6), (25, 0)]
times = [1, 6, 11, 16]

# initialize the matrix and vectors
n = numcoeffs * (len(points) - 1)
A = np.zeros((n, n))
bx = np.zeros((n, 1))
by = np.zeros((n, 1))

# fill in 3 equations for first segment - velocity, acceleration and jerk are all equal to 0 at start time
nextrow = 0
A[nextrow:nextrow+3, 0:numcoeffs] = coeffs_for_time([vel, acc, jerk], numcoeffs, times[0])
nextrow += 3

# fill in 3 equations for last segment - velocity, acceleration and jerk are all equal to 0 at end time
A[nextrow:nextrow+3, n-numcoeffs:n] = coeffs_for_time([vel, acc, jerk], numcoeffs, times[-1])
nextrow += 3

# for all segments...
for idx, startp in enumerate(points[0:-1]):
    endp = points[idx+1]
    startt = times[idx]
    endt = times[idx+1]

    # fill in 2 equations for start and end point passing through the poly
    # start point
    col = idx * numcoeffs
    A[nextrow:nextrow+1, col:col+numcoeffs] = coeffs_for_time([pos], numcoeffs, startt)
    bx[nextrow] = startp[0]
    by[nextrow] = startp[1]
    nextrow += 1

    # end point
    A[nextrow:nextrow+1, col:col+numcoeffs] = coeffs_for_time([pos], numcoeffs, endt)
    bx[nextrow] = endp[0]
    by[nextrow] = endp[1]

    nextrow += 1

# for all segments, except last...
for idx in range(len(points) - 2):
    endt = times[idx+1]

    # fill in 6 equations for velocity, acceleration, jerk, snap, crackle and pop to ensure they are the same through the transition point
    # evaluate both poly's at the end point
    col = idx * numcoeffs
    A[nextrow:nextrow+6, col:col+numcoeffs] = coeffs_for_time([vel, acc, jerk, snap, crac, pop], numcoeffs, endt)
    col += numcoeffs

    # negate endt coefficients since we move everything to the lhs
    A[nextrow:nextrow+6, col:col+numcoeffs] = -coeffs_for_time([vel, acc, jerk, snap, crac, pop], numcoeffs, endt)
    nextrow += 6
```

```python
# now we can solve for the coefficients
bxc = np.linalg.solve(A, bx)
byc = np.linalg.solve(A, by)
```

```python
# create our array of position polys for each segment
xpolys = []
ypolys = []
for segment in range(len(points) - 1):
    offset = segment*numcoeffs
    xpolys.append(sp.Poly(reversed(bxc[offset:offset+numcoeffs].transpose()[0]), t))
    ypolys.append(sp.Poly(reversed(byc[offset:offset+numcoeffs].transpose()[0]), t))
```

```python
# now we can iterate over each of the found polynomials to get our full trajectory
samples = 50
tvals = np.linspace(times[0], times[-1], samples*(len(points) - 1))
xpos = []
ypos = []
xvel = []
yvel = []
xacc = []
yacc = []
for idx in range(len(points) - 1):
    # pull out the tvals for this segment
    currtvals = tvals[np.argwhere((tvals >= times[idx]) & (tvals <= times[idx+1]))].transpose()[0]

    # get the correct poly for this segment
    xpoly = xpolys[idx]
    ypoly = ypolys[idx]

    # evaluate at each tval for position, velocity and acceleration
    xpos += [xpoly.eval(tval) for tval in currtvals]
    ypos += [ypoly.eval(tval) for tval in currtvals]
    xvel += [xpoly.diff(t).eval(tval) for tval in currtvals]
    yvel += [ypoly.diff(t).eval(tval) for tval in currtvals]
    xacc += [xpoly.diff(t).diff(t).eval(tval) for tval in currtvals]
    yacc += [ypoly.diff(t).diff(t).eval(tval) for tval in currtvals]

# plot the results
plt.figure(figsize=(20, 24))

plt.subplot(6, 2, 1)
plt.plot(tvals, xpos, 'r')
plt.ylabel('X position (m)')
plt.grid(True)

plt.subplot(6, 2, 2)
plt.plot(tvals, ypos, 'b')
plt.ylabel('Y position (m)')
plt.grid(True)

plt.subplot(6, 2, 3)
plt.plot(tvals, xvel, 'g')
plt.ylabel('X velocity (m/s)')
plt.grid(True)

plt.subplot(6, 2, 4)
plt.plot(tvals, yvel, 'y')
plt.ylabel('Y velocity (m/s)')
plt.grid(True)

plt.subplot(6, 2, 5)
plt.plot(tvals, xacc, 'm')
plt.xlabel('time (s)')
plt.ylabel('X acceleration (m/s^2)')
plt.grid(True)

plt.subplot(6, 2, 6)
plt.plot(tvals, yacc, 'c')
plt.xlabel('time (s)')
plt.ylabel('Y acceleration (m/s^2)')
plt.grid(True)

plt.show()
```

## Minimum Jerk Trajectory

We want a trajectory defining position of the form:
$x(t) = c_5t^5 + c_4t^4 + c_3t^3 + c_2t^2 + c_1t + c_0$

Once we derive the coefficients, we can then differentiate the resulting polynomial to get the desired velocity and acceleration polynomials, and use these to calculate the desired position, velocity, and accelerations at any time $t$ to give us a minimum jerk trajectory, that we can then feed into our position controller.

This is the same process as for minimum snap trajectory, so we will define a new plotting function and rely on the class base trajectory generator included elsewhere in this git repository.

```python
# define a plotting function to process results from the class based generator
def plot_all_for_class(tvals, xvalues, yvalues):
    numderivatives = xvalues.shape[0]
    plt.figure(figsize=(20, 8*numderivatives))

    numplots = 2*numderivatives
    numcols = 2
    label_idx = 0

    currderivative = 0

    for plot_idx in range(1, numderivatives*2+1, 2):
        dlabel = derivative_labels[label_idx]

        # plot x
        xvals = xvalues[currderivative]
        plt.subplot(numplots, numcols, plot_idx)
        plt.plot(tvals, xvals, plot_colors[plot_idx-1])
        plt.ylabel(f'X {dlabel}')
        plt.grid(True)

        plot_idx += 1

        # plot y
        yvals = yvalues[currderivative]
        plt.subplot(numplots, numcols, plot_idx)
        plt.plot(tvals, yvals, plot_colors[plot_idx-1])
        plt.ylabel(f'Y {dlabel}')
        plt.grid(True)

        label_idx += 1
        currderivative += 1

    plt.show()
```

```python
# add in our class based trajectory generator path so we can use that moving forward
import sys
sys.path.insert(0, '../')

from trajectory import MininumTrajectory, TrajectoryType

numderivatives = 4

minjerk = MininumTrajectory(TrajectoryType.JERK)
minjerk.generate(points, times, numderivatives=numderivatives)

tvals = np.linspace(times[0], times[-1], samples*(len(points) - 1))

xvalues = np.zeros((numderivatives+1, len(tvals)))
yvalues = np.zeros((numderivatives+1, len(tvals)))

for tidx, tval in enumerate(tvals):
    out = minjerk.getvalues(tval)
    xout = out[0]
    yout = out[1]

    for idx in range(numderivatives+1):
        xvalues[idx][tidx] = xout[idx]
        yvalues[idx][tidx] = yout[idx]

plot_all_for_class(tvals, xvalues, yvalues)
```

## Minimum Acceleration Trajectory

We want a trajectory defining position of the form:
$x(t) = c_3t^3 + c_2t^2 + c_1t + c_0$

```python
numderivatives = 2

minacc = MininumTrajectory(TrajectoryType.ACCELERATION)
minacc.generate(points, times, numderivatives=numderivatives)

tvals = np.linspace(times[0], times[-1], samples*(len(points) - 1))

xvalues = np.zeros((numderivatives+1, len(tvals)))
yvalues = np.zeros((numderivatives+1, len(tvals)))

for tidx, tval in enumerate(tvals):
    out = minacc.getvalues(tval)
    xout = out[0]
    yout = out[1]

    for idx in range(numderivatives+1):
        xvalues[idx][tidx] = xout[idx]
        yvalues[idx][tidx] = yout[idx]

plot_all_for_class(tvals, xvalues, yvalues)
```

## Minimum Velocity Trajectory

We want a trajectory defining position of the form:
$x(t) = c_1t + c_0$

```python
numderivatives = 1

minvel = MininumTrajectory(TrajectoryType.VELOCITY)
minvel.generate(points, times, numderivatives=numderivatives)

tvals = np.linspace(times[0], times[-1], samples*(len(points) - 1))

xvalues = np.zeros((numderivatives+1, len(tvals)))
yvalues = np.zeros((numderivatives+1, len(tvals)))

for tidx, tval in enumerate(tvals):
    out = minvel.getvalues(tval)
    xout = out[0]
    yout = out[1]

    for idx in range(numderivatives+1):
        xvalues[idx][tidx] = xout[idx]
        yvalues[idx][tidx] = yout[idx]

plot_all_for_class(tvals, xvalues, yvalues)
```

```python

```


## Review Calculus using by Python

Consider a sequence of n numbers $x_0, x_1, \cdots x_{n-1}$. We will start our index at 0, to remain in accordance with Python/Numpy's index system. $x_0$ is the first number in the sequence, $x_1$ is the second number in the sequence, and so forth, so $x_j$ is the general $j+1$ number in the sequence. We will utilize this $j$ index in our summation notation. Suppose we were to calculate the sum of all $n$ of these numbers in the sequence. We write that this sum is equivalent to

Let's parse this equation. $\Sigma$ is the summation sign, indicating that we are summing a sequence. $j$ is the index of summation that is being iterated over; $j$ is being used to subscript $x$. Then, we start our summation at 0 because the lower bound on our sum is set as $j=0$. The upper bound for our sum, or our stopping point is written as $n-1$, which is equivalent to writing $j=n-1$ on top of the $\Sigma$ symbol. Then, $x_j$ indicates the quantitiy that we are summing. One way to think about this is in terms of a for-loop. This summation concept is equivalent to a for loop for all integers in the range from the lower bound to the upper bound, indexed into the sequence $x$. Then, the code for our previous sum is:

```python

```

Then, you can run through a few more examples of summation notation.

Example1:

```python

```

Example2:

```python

```

Example3:

```python

```

Then, we can also sum over several different sequences. Consider the sequences A and B, where A consists of m values $a_0, a_1, \cdots, a_{m-1}$ and B contains n values $b_0, b_1, \cdots b_{n-1}$. Then, we can calculate the sum of the product of each value in A with each value in B (a sum over $m\times n$ products). Because the $i$ index only appears in association with A, and the $j$ index with B, we can group these summations.

```python
import numpy as np
```

```python
A = np.random.rand(5)
```

```python
B = np.random.rand(7)
```

```python

```

Note that the following does not hold

because this effectively treats the index $i$ in the first term independently from the $i$ in the second term of the product. Notice that in the right side of the equation, we could have interchanged the index $i$ in the second summation with $j$, without changing any of the mathematics. The sum on the left represents the summation of $m$ terms of $a_{i}^2$, whereas the sum on the right represents the summation of $m \times m$ terms - products between all possible pairs of A's terms.

```python
import numpy as np
```

```python
A = np.random.rand(5)
```

```python

```

Note that typically when someone writes $\sum_{i} x_i$, this is just the sum of all values in the sequence X, and is the same as writing $\sum_{i=0}^{n-1} x_i$, where you'll usually know the value of n so you can still compute the sum.

## Matrices

Then suppose we have a m by n matrix that contains all products of the values in both sequence A and sequence B such that the matrix value of $M$ at index $(i,j)$ is $a_i \cdot b_j$.

You can form this matrix in numpy via an "outer product":

```python
import numpy as np
```

```python

```

```python

```

```python

```

```python

```

You can use this matrix to visualize several possible uses of summations. For example, suppose $m=n$. Then, if we express the sequences A and B as column vectors, the dot product of the two vectors would be the sum of the diagonal of M, or $\sum_{k=0}^{n-1} a_k \cdot b_k$. Furthermore, we can take the sum of a row using the sum $\sum_{i=0}^{n-1} a_k \cdot b_i$ for the $k^{th}$ row of the matrix, and similarly take the sum of a column with $\sum_{i=0}^{n-1} a_i \cdot b_k$ for the $k^{th}$ column of the matrix.

```python
k = 2
```

```python
# sum over the kth column:
```

```python
# sum over the kth row:
```

## Kronecker Delta

To make notation for working with summations (particularly in considering the matrix format) even simpler, we can use the Kronecker delta function, named after Prussian mathematician Leonard Kronecker. We use $\delta_{i,j}$ to denote the Kronecker delta function, defined as: \begin{equation} \begin{cases} \delta{i, j}= 0 & \text{if } i \neq j\
\delta{i,j}=1 & \text{if } i=j
\end{cases} \end{equation}

See that a Kronecker-delta can "collapse" a sum. Let $j$ be an integer between 0 and $n-1$:

See also that the identity matrix, $I$, can be written as $I_{i,j}=\delta_{i,j}$.

As an exercise, write a python function that behaves as a kronecker delta, and include it in a for-loop that is computing a sum. Verify that it does indeed collapse the sum.

## Partial Derivatives

Partial derivatives are used in multivariable functions, in which we essentially derive with respect to one of these variables and treat the remaining variables as constants. For example:

$$ f(x,y) = 6 x^2 y^3 $$ $$ \frac{\partial}{\partial x} f(x,y) = 12x y^3 $$ $$ \frac{\partial}{\partial y} f(x,y) = 18 x^2 y^2 $$

So, what if we want to take a partial derivative of a sum?

Suppose that we have two vectors containing variables:

Then, in our previous section we saw that $\vec{x} \cdot \vec{y} = \sum_{i=0}^{n-1} x_i \cdot y_i$. What happens when we take the partial derivative of this sum with respect to the variable $x_{j}$? Let $f = \vec{x} \cdot \vec{y}$, then

It is critical to see that, because $x_i$ and $x_j$ are distinct variables (unless $i = j$), \begin{equation} \begin{cases} \frac{\partial x_i}{\partial x_j} = 0 & \text{if } i \neq j\
\frac{\partial x_i}{\partial x_j} = 1 & \text{if } i=j
\end{cases} \end{equation} and thus $\frac{\partial x_i}{\partial x_j} = \delta_{i,j}$.

Thus we can simplify our sum by collapsing it via the kronecker-delta:

Defining $\frac{\partial f}{\partial \vec{x}} = [\frac{\partial f}{\partial x_0}, \cdots , \frac{\partial f}{\partial x_j}, \cdots, \frac{\partial f}{\partial x_{n-1}}]$, we can see from our above result that

Take a little bit of time to think through the above expression, and make sure you understand why it is true, writing out matrices to help your understanding. This is just one example of how we can apply some of the calculus we already know to vectors, leading to vector calculus, a pillar for linear algebra! This is especially important for deriving expressions used in back-propagation in machine learning.

The end of document


# Complex numbers and operation on complex numbers

From _A.G. Sveshnokov, A.N. Tikhonov (1982). The theory of functions of a complex variable. Section 1.1_.

## The concept of a complex number

A complex number $z$ is characterized by a pair of real numbers $(a, b)$. The first number
$a$ of the pair $(a, b)$ is called the _real part_ of the complex number $z$ and is denoted
by $a =\mathrm{Re}\, z$; the second number $b$ of the pair $(a, b)$ is called the
_imaginary part_ of the complex number $z$ and is symbolized by $b=\mathrm{Im}\, z$.

Two complex numbers $z_1 = (a_1, b_1)$ and $z_2 = (a_2, b_2)$ are equal only when both the real
and imaginary parts are equal, that is, $z_1 = z_2$ only when $a_1 = a_2$ and $b_1 = b_2$.

## Operations on complex numbers

Let us now define algebraic operations involving complex numbers.

The sum of two complex numbers $z_1 = (a_1, b_1)$ and $z_2 = (a_2, b_2)$ is a complex number
$z = (a, b)$, where $a = a_1 + a_2$, $b = b_1 + b_2$. The commutative and associative laws
for addition, $z_1 + z_2 = z_2 + z_1$ and $z_1 + (z_2 + z_3) = (z_1 + z_2) + z_3$, hold true.
As in the domain of real numbers, zero is a complex number $0$ such that the sum of it and any
complex number $z$ is equal to $z$, that is, $z + 0 = z$. There is a unique complex number
$0 = (0, 0)$ that possesses this property.

The _product_ of the complex numbers $z_1 = (a_1, b_1)$ and $z_2 = (a_2, b_2)$ is a complex
number $z = (a, b)$ such that $a = a_1 a_2 - b_1 b_2$, $b = a_1 b_2 + a_2 b_1$. In this
definition of a product, we find that the commutative $[z_1 z_2 = z_2 z_1]$, associative
$[z_1 (z_2\cdot z_3) = (z_1\cdot z_2) z_3]$  and distributive $[(z_1 + z_2) z_3 = z_1 z_3 + z_2 z_3]$
laws hold.

Let us include the real numbers in the set of complex numbers and regard the real number $a$ as the
complex number $a = (a, 0)$. Then, as follows from the definition of the operations of addition
and miltiplication, the familiar rules involving real numbers hold true for complex numbers as well.
Thus, the set of complex numbers is regarded as an extension of the set of real numbers. Note that
multiplication by a real unit $(1, 0)$ does not change a complex number: $z\cdot 1 = z$.

A complex number of the form of the form $z = (0, b)$ is called a _pure imaginary_ number (or just
imaginary number) and is symbolized as $z = ib$. The pure imaginary number $(0, b) = ib$ may be
regarded as the product of the imaginary unit $(0, 1)$ and real number $(b, 0)$. The imaginary
unit is commonly denoted by the symbol $(0, 1) = i$. By virtue of the definition of a product
of complex numbers, the following relation holds true: $i\cdot i = i^2 = -1$. It enables one
to attribute a direct algebraig meaning to the _real-imaginary form_ of a complex number:

$$ z = (a, b) = a + ib\, ,$$

and perform operations of addition and multiplication of complex numbers in accordance with the
usual rules of the algebra of polynomials.

The complex number $\bar{z} = a - ib$ is said to be the complex _conjugate_ of $z = a + ib$.

The operation of substraction of complex numbers is defined as the inverse operation of addition.
A complex number $z = a + ib$ is termed the _difference_ between the complex numbers $z_1 = a_1 + ib_1$
and $z_2 = a_2 + ib_2$ if $a = a_1 - a_2$, $b = b_1 - b_2$.

The operation of dividing complex numbers is defined as the inverse operation of multiplication.
A complex number $z  = a + ib$ is called the quotient of the complex numbers $z_1 = a_1 + ib_1$ and
$z_2 = a_2 + ib_2 \neq = 0$ if $z_1 = z\cdot z_2$, whence it follows that the real part $a$ and the
imaginary part $b$ of the quotient $z$ are found from the linear system of algebraic equations

\begin{align}
  &a_2 a - b_2 b = a_1\\
  &b_2 a + a_2 b = b_1
\end{align}

with the determinant $a_2^2 + b_2^2$ different from zero. Solving this system, we get

$$z = \frac{z_1}{z_2} = \frac{a_1 a_2 + b_1 b_2}{a_2^2 + b_2^2} + i \frac{b_1 a_2 - a_1 b_2}{a_2^2 + b_2^2}\, .$$

## Operations on complex numbers using SymPy

Let us repeat the operations described above using SymPy.

```python
from sympy import *
init_printing()
```

```python
a, a1, a2, a3 = symbols("a a1 a2 a3", real=True)
b, b1, b2, b3 = symbols("b b1 b2 b3", real=True)
```

In SymPy the imaginary unit is represented by ``I``.

```python
I**2
```

```python
z1 = a1 + b1*I
z2 = a2 + b2*I
z3 = a3 + b3*I
display(z1)
display(z2)
display(z3)
```

### Real and imaginary parts

```python
# Real part
display(re(z1))
display(re(z2))
```

```python
# Imaginary part
display(im(z1))
display(im(z2))
```

### Addition

```python
display(z1 + z2)
display(re(z1 + z2))
display(im(z1 + z2))
```

Addition is commutative

```python
display(z1 + z2)
display(z2 + z1)
```

Addition is also associative

```python
display(z1 + (z2 + z3))
display((z1 + z2) + z3)
```

### Multiplication

```python
display(z1 * z2)
display(re(z1 * z2))
display(im(z1 * z2))
```

Multiplication is commutative

```python
display(z1 * z2)
display(z2 * z1)
```

Multiplication is associative

```python
display(expand(z1 * (z2 * z3)))
display(expand((z1 * z2) * z3))
```

Multiplication is distributive over addition

```python
display(expand((z1 + z2) * z3))
display(expand(z1 * z3 + z2 * z3))
```

### Complex conjugate

```python
display(z1)
display(conjugate(z1))
```

### Difference

```python
display(z1 - z2)
display(re(z1 - z2))
display(im(z1 - z2))
```

### Division

```python
display(z1 / z2)
display(factor(re(z1 / z2)))
display(factor(im(z1 / z2)))
```

## The geometric interpretation of complex numbers

The study of complex numbers is greatly facilitated by interpretating them geometrically.
Insofar as a complex number is defined as a pair of real numbers, it is natural to depict
the complex number $z = a + ib$ as a point in the $x$, $y$-plane with Cartesian coordinates
$x = a$ and $y = b$. The number $z=0$ corresponds to the origin of the plane. We shall henceforward
call this the _complex plane_; the axis of the abscissas is the _real_ axis, the axis of
the ordinates is the _imaginary_ axis of the complex plane. We have thus established a
reciprocal one-to-one correspondence between the set of all complex numbers and the set of
points the complex plane, and also between the set of al complex numbers $z = a + ib$ and the
set of free vectors, the projections $x$ and $y$ of which on the axis of abscissas and the
axis of ordinates are, respectively, equal to $a$ and $b$.

There is another extremely important form of representing complex numbers. It is possible to
define the position of a point in the plane by means of polar coordinates $(\rho, \varphi)$, where
$\rho$ is the distance of the point from the origin, and $\varphi$ is the angle between the
radius vector with the positive direction of the axis of abscissas. The positive direction of the
variation of the angle $\varphi$ is counterclockwise $(-\infty < \varphi < \infty)$. Taking
advantage of the relationship between Cartesian and polar coordinates

\begin{align}
  &x = \rho \cos\varphi\\
  &y = \rho \sin\varphi\, ,
\end{align}

we get the so-called _trigonometric form_ (or polar form) of a complex number:

$$z = \rho(\cos\varphi + i\sin\varphi)\, .$$

```python
Abs(z1)
```

```python

```

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./styles/custom_barba.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
/* Based on Lorena Barba template available at: https://github.com/barbagroup/AeroPython/blob/master/styles/custom.css*/
@font-face {
font-family: "Computer Modern";
src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}
div.cell{
width:800px;
margin-left:16% !important;
margin-right:auto;
}
h1 {
font-family: 'Alegreya Sans', sans-serif;
}
h2 {
font-family: 'Fenix', serif;
}
h3{
font-family: 'Fenix', serif;
margin-top:12px;
margin-bottom: 3px;
}
h4{
font-family: 'Fenix', serif;
}
h5 {
font-family: 'Alegreya Sans', sans-serif;
}
div.text_cell_render{
font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
line-height: 135%;
font-size: 120%;
width:600px;
margin-left:auto;
margin-right:auto;
}
.CodeMirror{
font-family: "Source Code Pro";
font-size: 90%;
}
/* .prompt{
display: None;
}*/
.text_cell_render h1 {
font-weight: 200;
font-size: 50pt;
line-height: 100%;
color:#CD2305;
margin-bottom: 0.5em;
margin-top: 0.5em;
display: block;
}
.text_cell_render h5 {
font-weight: 300;
font-size: 16pt;
color: #CD2305;
font-style: italic;
margin-bottom: .5em;
margin-top: 0.5em;
display: block;
}
.warning{
color: rgb( 240, 20, 20 )
}
</style>

```python

```


# Exponential-family belief updating

## Reduction of Bayes' theorem to mean updating

### Mean updating

The mean $\bar{x}_n$ of a set $\{x_1, x_2, \ldots, x_n\}$ is

$$
\bar{x}_n := \frac{1}{n} \sum_{i=1}^{n} x_i
$$

For example, $x_i$ could be observations made sequentially. If we now add a new element $x_{n+1}$ to our set, the new mean $\bar{x}_{n+1}$ is

$$
\bar{x}_{n+1} = \bar{x}_n + \frac{1}{n+1} \left( x_{n+1} - \bar{x}_n \right)
$$

*Proof:*

$$
\begin{align}
\bar{x}_{n+1} &= \frac{1}{n+1} \sum_{i=1}^{n+1} x_i \\
&= \frac{1}{n+1} \left( x_{n+1} + n \cdot \frac{1}{n} \sum_{i=1}^{n} x_i \right) \\
&= \frac{1}{n+1} \left( x_{n+1} + n \bar{x}_n \right) \\
&= \frac{1}{n+1} \left( x_{n+1} - \bar{x}_n + (n+1) \bar{x}_n \right) \\
&= \bar{x}_n + \frac{1}{n+1} \left( x_{n+1} - \bar{x}_n \right)
\end{align}
$$

### Bayes' theorem

Bayes' theorem is an immediate consequence of the product rule of probability and states that the posterior is the likelihood times the prior divided by the model evidence.

$$
p\left( \vartheta \middle| x \right) = \frac{p\left( x \middle| \vartheta \right) p\left( \vartheta \right)}{p\left( x \right)}
= \frac{p\left( x \middle| \vartheta \right) p\left( \vartheta \right)}{\int_\Omega p\left( x \middle| \vartheta' \right) p\left( \vartheta' \right) \mathrm{d}\vartheta'},
$$

where $\Omega$ is the support of $p(\vartheta)$.

### Exponential families of probability distributions

A family of probability distributions is an *exponential family* if and only if it can be written in the following form

$$
p\left( x \middle| \vartheta \right) = f_x(\vartheta) := h(x) \exp\left( \eta(\vartheta) \cdot T(x) - B(\vartheta) \right),
$$

where $x$ is a vector-valued observation, $\vartheta$ is a parameter vector, $h(x)$ is a normalization constant, $\eta(\vartheta)$ is the so-called 'natural' parameter vector, $T(x)$ is the sufficient statistic vector, and $B(\vartheta)$ is a scalar function.

### Bayes' theorem applied to exponential families

When the likelihood belongs to an exponential family, applying Bayes' theorem becomes extremely simple with an appropriate prior. We choose a prior with a natural interpretation, implying that we have made $\nu \in \mathbb{R}$ observations with sufficient statistics $\xi$:

$$
p\left( \vartheta \middle| \xi, \nu \right) = g_{\xi, \nu}(\vartheta) := z\left( \xi, \nu \right) \exp\left( \nu\left( \eta \left( \vartheta \right) \cdot \xi - B(\vartheta) \right) \right),
$$

where $z(\xi, \nu)$ is a normalization constant:

$$
z(\xi, \nu) := \left( \int_\Omega \exp\left( \nu\left( \eta \left( \vartheta \right) \cdot \xi - B(\vartheta) \right) \right) \mathrm{d}\vartheta \right)^{-1}.
$$

The *hyperpriors* $\xi$ and $\nu$ parameterize the prior $g_{\xi, \nu}$ of $\vartheta$. This prior is *conjugate* to the likelihood; that is, the posterior has the same form $g_{\xi, \nu}$. When passing from the prior to the posterior after a new observation $x$, all that needs to be updated are the hyperpriors $\xi$ and $\nu$:

$$
\begin{align}
\nu &\leftarrow \nu+1 \\
\xi &\leftarrow \xi + \frac{1}{\nu+1} \left( T(x) - \xi \right)
\end{align}
$$

*Proof:*

$$
\begin{align}
p\left( \vartheta \middle| x, \xi, \nu \right) &\propto p\left( x \middle| \vartheta \right) p\left( \vartheta \middle| \xi, \nu \right) \\\\
&= f_x(\vartheta)g_{\xi, \nu}(\vartheta) \\\\
&= h(x) \exp\left( \eta(\vartheta) \cdot T(x) - B(\vartheta) \right) z\left( \xi, \nu \right) \exp\left( \nu\left( \eta \left( \vartheta \right) \cdot \xi - B(\vartheta) \right) \right) \\\\
&\propto \exp\left( \eta(\vartheta) \cdot \left( T(x) + \nu \xi \right) - (\nu+1) B(\vartheta) \right) \\\\
&= \exp\left( (\nu+1) \left( \eta(\vartheta) \cdot \frac{1}{\nu+1} \left( T(x) + \nu \xi \right) - B(\vartheta) \right)\right) \\\\
&= \exp\left( (\nu+1) \left( \eta(\vartheta) \cdot \left( \xi + \frac{1}{\nu+1} \left( T(x) - \xi \right) \right) - B(\vartheta) \right)\right) \\\\\\
\Longrightarrow p\left( \vartheta \middle| x, \xi, \nu \right) &= g_{\xi', \nu'}(\vartheta) \qquad \text{with} \qquad \nu' = \nu+1 \qquad \text{and} \qquad \xi' = \xi + \frac{1}{\nu+1} \left( T(x) - \xi \right)
\end{align}
$$

Since the prior can

## Gaussian with unknown mean and known precision

### Likelihood

The likelihood is

$$
\begin{aligned}
p\left( x \middle| \vartheta \right) &= h\left( x \right) \exp\left( \eta\left( \vartheta \right) \cdot T(x) - B(\vartheta) \right) \\
&= \sqrt{\frac{\tau}{2\pi}} \exp \left( -\frac{\tau}{2}\left( x -\mu \right)^2 \right) \\
&= \mathcal{N} \left( x ; \mu, \tau^{-1} \right)
\end{aligned}
$$

with

$$
\begin{aligned}
h( x ) &: = \sqrt{\frac{\tau}{2\pi}} \exp\left(-\frac{\tau}{2} x^2\right),\\
\vartheta &:= \mu,\\
\eta (\vartheta) &: = \tau\mu,\\
T(x) &: = x,\\
B(\vartheta) &:= \frac{\tau}{2}\mu^2.
\end{aligned}
$$

### Prior

The prior is

$$
p\left( \vartheta \middle| \xi, \nu \right) = z\left( \xi, \nu
\right) \exp\left( \nu\left( \eta \left( \vartheta \right) \cdot \xi -
B(\vartheta) \right) \right),
$$

with normalization factor

$$
z(\xi, \nu) := \left( \int_\Omega \exp\left( \nu\left( \eta \left( \vartheta \right) \cdot \xi -
B(\vartheta) \right) \right) \mathrm{d}\vartheta \right)^{-1}.
$$

With the definitions above, this means

$$
\begin{aligned}
z(\xi, \nu)^{-1} &= \int_{-\infty}^\infty \exp \left( \nu \left( \tau \mu \xi - \frac{1}{2}\tau \mu^2 \right) \right) \mathrm{d}\mu \\
&= \int_{-\infty}^\infty \exp \left( -\frac{\nu\tau}{2} \left( \mu^2 - 2 \mu \xi \right) \right) \mathrm{d}\mu \\
&= \int_{-\infty}^\infty \exp \left( -\frac{\nu\tau}{2} \left( \left( \mu - \xi \right)^2 - \xi^2 \right) \right) \mathrm{d}\mu \\
&= \exp \left( \frac{\nu\tau}{2} \xi^2 \right) \int_{-\infty}^\infty \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi \right)^2 \right) \mathrm{d}\mu \\
&= \sqrt{\frac{2\pi}{\nu \tau}} \exp \left( \frac{\nu\tau}{2} \xi^2 \right)
\end{aligned}
$$

and

$$
\begin{aligned}
p\left( \vartheta \middle| \xi, \nu \right) &= z(\xi, \nu) \exp \left( \frac{\nu\tau}{2} \xi^2 \right) \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi \right)^2 \right) \\
&= \sqrt{\frac{\nu \tau}{2\pi}} \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi \right)^2 \right) \\
&= \mathcal{N} \left( \mu ; \xi, (\nu\tau)^{-1} \right)
\end{aligned}
$$

### Prior predictive distribution

Marginalizing the joint over the parameter $\vartheta$ gives us the prior predictive distribution

$$
\begin{aligned}
p\left( x | \xi, \nu \right) &= \int_\Omega p(x | \vartheta) p(\vartheta | \xi, \nu) \mathrm{d}\vartheta \\
&= \int_{-\infty}^\infty \mathcal{N} \left( x ; \mu, \tau^{-1} \right) \mathcal{N} \left( \mu ; \xi, (\nu\tau)^{-1} \right) \mathrm{d}\mu \\
&= \mathcal{N} \left( x ; \xi, \tau^{-1} + (\nu\tau)^{-1} \right) \\
&= \mathcal{N} \left( x ; \xi, \left( \frac{\nu\tau}{\nu+1} \right)^{-1} \right).
\end{aligned}
$$

The third equality is a consequence of the fact that compounding two Gaussians gives a Gaussian whose variance is the sum of those of the compounded Gaussians. A more tedious derivation gives the same result:

$$
\begin{aligned}
p\left( x | \xi, \nu \right) &= \int_\Omega p(x | \vartheta) p(\vartheta | \xi, \nu) \mathrm{d}\vartheta \\
&= \int_{-\infty}^\infty \mathcal{N} \left( x ; \mu, \tau^{-1} \right) \mathcal{N} \left( \mu ; \xi, (\nu\tau)^{-1} \right) \mathrm{d}\mu \\
&= \int_{-\infty}^\infty \sqrt{\frac{\tau}{2\pi}} \exp \left( -\frac{\tau}{2}\left( x -\mu \right)^2 \right) \sqrt{\frac{\nu \tau}{2\pi}} \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi \right)^2 \right) \mathrm{d}\mu \\
&= \frac{\sqrt{\nu} \tau}{2\pi} \int_{-\infty}^\infty \exp \left( -\frac{\tau}{2}\left( x -\mu \right)^2 \right)  \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi \right)^2 \right) \mathrm{d}\mu \\
&= \frac{\sqrt{\nu} \tau}{2\pi} \int_{-\infty}^\infty \exp \left( -\frac{\tau}{2}\left( (\nu+1)\mu^2 -2(x+\nu\xi)\mu + x^2 +\nu\xi^2 \right) \right) \mathrm{d}\mu \\
&= \frac{\sqrt{\nu} \tau}{2\pi} \exp \left( -\frac{\tau}{2}\left( x^2 +\nu\xi^2 \right) \right) \int_{-\infty}^\infty \exp \left( -\frac{(\nu+1) \tau}{2}\left( \left( \mu - \frac{x+\nu\xi}{\nu+1} \right)^2 - \left( \frac{x+\nu\xi}{\nu+1} \right)^2 \right) \right) \mathrm{d}\mu \\
&= \frac{\sqrt{\nu} \tau}{2\pi} \exp \left( -\frac{\tau}{2}\left( x^2 +\nu\xi^2 - \frac{(x+\nu\xi)^2}{\nu+1} \right) \right) \int_{-\infty}^\infty \exp \left( -\frac{(\nu+1) \tau}{2}\left( \left( \mu - \frac{x+\nu\xi}{\nu+1} \right)^2 \right) \right) \mathrm{d}\mu \\
&= \frac{\sqrt{\nu} \tau}{2\pi} \exp \left( -\frac{\tau}{2}\left( x^2 +\nu\xi^2 - \frac{(x+\nu\xi)^2}{\nu+1} \right) \right) \sqrt{\frac{2\pi}{(\nu+1)\tau}} \\
&= \sqrt{\frac{\nu\tau}{2\pi(\nu+1)}} \exp \left( -\frac{\tau}{2}\left( x^2 +\nu\xi^2 - \frac{(x+\nu\xi)^2}{\nu+1} \right) \right) \\
&= \sqrt{\frac{\nu\tau}{2\pi(\nu+1)}} \exp \left( -\frac{\tau}{2}\left( x^2 - \frac{x^2}{\nu+1} - \frac{2\nu\xi x}{\nu+1}- \frac{\nu^2\xi^2}{\nu+1} +\nu\xi^2 \right) \right) \\
&= \sqrt{\frac{\nu\tau}{2\pi(\nu+1)}} \exp \left( -\frac{\tau}{2}\left( x^2 \left( 1  - \frac{1}{\nu+1} \right) - 2\frac{\nu}{\nu+1}\xi x - \frac{\nu}{\nu+1} \left( +\nu\xi^2 + (\nu+1)\xi^2 \right) \right) \right) \\
&= \sqrt{\frac{\nu\tau}{2\pi(\nu+1)}} \exp \left( -\frac{\nu\tau}{2(\nu+1)}(x - \xi)^2 \right) \\
&= \mathcal{N} \left( x ; \xi, \left( \frac{\nu\tau}{\nu+1} \right)^{-1} \right).
\end{aligned}
$$

## Gaussian with mean and precision unknown

### Likelihood

Whether precision is known or unknown does not affect the likelihood. It still is

$$
\begin{aligned}
p\left( x \middle| \vartheta \right) &= h\left( x \right) \exp\left( \eta\left( \vartheta \right) \cdot T(x) - B(\vartheta) \right) \\
&= \sqrt{\frac{\tau}{2\pi}} \exp \left( -\frac{\tau}{2}\left( x -\mu \right)^2 \right) \\
&= \mathcal{N} \left( x ; \mu, \tau^{-1} \right),
\end{aligned}
$$

but this time with

$$
\begin{aligned}
h( x ) &: = \frac{1}{\sqrt{2\pi}},\\
\vartheta &:= \left( \mu, \tau \right)^\intercal \\
\eta (\vartheta) &: = \left( \tau\mu, -\tau/2 \right)^\intercal \\
T(x) &: = \left( x, x^2 \right)^\intercal \\
B(\vartheta) &:= \frac{1}{2} \left( \tau \mu^2 - \ln \tau \right).
\end{aligned}
$$

### Prior

With

$$
\begin{aligned}
z(\xi, \nu)^{-1} &= \int_0^\infty \int_{-\infty}^\infty \exp \left( \nu \left( \tau \mu \xi_1 - \frac{1}{2}\tau \xi_2 - \frac{1}{2}\tau \mu^2  + \frac{1}{2} \ln \tau \right) \right) \mathrm{d}\mu \mathrm{d}\tau \\
&= \int_0^\infty \tau^\frac{\nu}{2} \int_{-\infty}^\infty \exp \left( \frac{\nu\tau}{2} \left( 2\mu\xi_1 - \xi_2 - \mu^2 \right) \right) \mathrm{d}\mu \mathrm{d}\tau \\
&= \int_0^\infty \tau^\frac{\nu}{2} \exp \left( -\frac{\nu\tau}{2} \xi_2 \right) \int_{-\infty}^\infty \exp \left( -\frac{\nu\tau}{2} \left( \mu^2 - 2\mu\xi_1 \right) \right) \mathrm{d}\mu \mathrm{d}\tau \\
&= \int_0^\infty \tau^\frac{\nu}{2} \exp \left( -\frac{\nu\tau}{2} \left( \xi_2 - \xi_1^2 \right) \right) \int_{-\infty}^\infty \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi_1 \right)^2 \right) \mathrm{d}\mu \mathrm{d}\tau \\
&= \int_0^\infty \tau^\frac{\nu}{2} \exp \left( -\frac{\nu\tau}{2} \left( \xi_2 - \xi_1^2 \right) \right) \sqrt{\frac{2\pi}{\nu\tau}} \mathrm{d}\tau \\
&= \sqrt{\frac{2\pi}{\nu}} \int_0^\infty \tau^\frac{\nu-1}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \tau \right) \mathrm{d}\tau \\
&= \sqrt{\frac{2\pi}{\nu}} \frac{\Gamma \left( \frac{\nu+1}{2} \right)}{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2}} \\
\end{aligned}
$$

we get

$$
\begin{aligned}
p\left( \vartheta \middle| \xi, \nu \right) &= z(\xi, \nu) \tau^\frac{\nu}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \tau \right) \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi_1 \right)^2 \right) \\
&= \frac{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \tau^\frac{\nu-1}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \tau \right) \sqrt{\frac{\nu\tau}{2\pi}} \exp \left( -\frac{\nu\tau}{2} \left( \mu - \xi_1 \right)^2 \right) \\
&= \mathcal{N} \left( \mu ; \xi_1, (\nu\tau)^{-1} \right) \text{Gamma} \left( \tau; \frac{\nu+1}{2}, \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right). \\
\end{aligned}
$$

### Prior predictive distribution

$$
\begin{aligned}
p\left( x | \xi, \nu \right) &= \int_\Omega p(x | \vartheta) p(\vartheta | \xi, \nu) \mathrm{d}\vartheta \\
&= \int_0^\infty \int_{-\infty}^\infty \mathcal{N} \left( x ; \mu, \tau^{-1} \right) \mathcal{N} \left( \mu ; \xi_1, (\nu\tau)^{-1} \right) \text{Gamma} \left( \tau; \frac{\nu+1}{2}, \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right) \mathrm{d}\mu \mathrm{d}\tau \\
&= \int_0^\infty \mathcal{N} \left( x ; \xi_1, \left( \frac{\nu\tau}{\nu+1} \right)^{-1} \right) \text{Gamma} \left( \tau; \frac{\nu+1}{2}, \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right) \mathrm{d}\tau \\
&= \int_0^\infty \sqrt{\frac{\nu\tau}{2\pi(\nu+1)}} \exp \left( -\frac{\nu\tau}{2(\nu+1)}(x - \xi_1)^2 \right) \frac{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \tau^\frac{\nu-1}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \tau \right) \mathrm{d}\tau \\
&= \sqrt{\frac{\nu}{2\pi(\nu+1)}} \frac{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \int_0^\infty \exp \left( -\frac{\nu\tau}{2(\nu+1)}(x - \xi_1)^2 \right) \tau^\frac{\nu}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \tau \right) \mathrm{d}\tau \\
&= \sqrt{\frac{\nu}{2\pi(\nu+1)}} \frac{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \int_0^\infty \tau^\frac{\nu}{2} \exp \left( -\frac{\nu}{2} \left( \xi_2 - \xi_1^2 + \frac{(x - \xi_1)^2}{\nu+1} \right) \tau \right) \mathrm{d}\tau \\
&= \sqrt{\frac{\nu}{2\pi(\nu+1)}} \frac{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 \right) \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \frac{\Gamma \left( \frac{\nu}{2} + 1\right)}{\left( \frac{\nu}{2} \left( \xi_2 - \xi_1^2 + \frac{(x - \xi_1)^2}{\nu+1} \right) \right)^{\frac{\nu}{2} + 1}} \\
&= \sqrt{\frac{1}{\pi(\nu+1)}} \frac{\left( \xi_2 - \xi_1^2 \right)^\frac{\nu+1}{2} }{\Gamma \left( \frac{\nu+1}{2} \right)} \frac{\Gamma \left( \frac{\nu}{2} + 1\right)}{\left( \xi_2 - \xi_1^2 + \frac{(x - \xi_1)^2}{\nu+1} \right)^{\frac{\nu}{2} + 1}} \\
&= \sqrt{\frac{1}{\pi(\nu+1)\left( \xi_2 - \xi_1^2 \right)}} \frac{\Gamma \left( \frac{\nu+2}{2} \right)}{\Gamma \left( \frac{\nu+1}{2} \right)} \left( \frac{\xi_2 - \xi_1^2}{\xi_2 - \xi_1^2 + \frac{(x - \xi_1)^2}{\nu+1}} \right)^{\frac{\nu+2}{2}} \\
&= \sqrt{\frac{1}{\pi(\nu+1)\left( \xi_2 - \xi_1^2 \right)}} \frac{\Gamma \left( \frac{\nu+2}{2} \right)}{\Gamma \left( \frac{\nu+1}{2} \right)} \left( 1 + \frac{(x - \xi_1)^2}{(\nu+1) \left( \xi_2 - \xi_1^2 \right)} \right)^{-\frac{\nu+2}{2}} \\
&= \text{Student's-}t \left( x; \nu+1, \xi_1, \xi_2 - \xi_1^2 \right) \\
\end{aligned}
$$

This motivates the definition of the *normal-predictive* distribution $\mathcal{NP}$:

$$
\mathcal{NP} \left( x; \xi_1, \xi_2, \nu \right) :=  \sqrt{\frac{1}{\pi(\nu+1)\left( \xi_2 - \xi_1^2 \right)}} \frac{\Gamma \left( \frac{\nu+2}{2} \right)}{\Gamma \left( \frac{\nu+1}{2} \right)} \left( 1 + \frac{(x - \xi_1)^2}{(\nu+1) \left( \xi_2 - \xi_1^2 \right)} \right)^{-\frac{\nu+2}{2}}
$$

## Code and examples

```R
dnp <- function(x, xi1, xi2, nu) {
    # Mean
    mu <- xi1
    # 'Standard deviation'
    sigma <- sqrt(xi2 - xi1^2)
    # Degrees of freedom
    df <- nu + 1
    # Argument
    a <- (x - mu)/sigma

    # Density
    d <- 1/sigma*dt(a, df)
    return(d)
}
```

```R
dnp(1, 1, 2, 7)
```

0.386699020961393

```R
dt(1, 3)
```

0.206748335783172

```R
library(ggplot2)
```

```R
options(repr.plot.width = 4, repr.plot.height = 3)
ggplot(data.frame(x = c(-4, 4)), aes(x = x)) +
    stat_function(fun = dt, args = list(df = 1)) +
    stat_function(fun = dnp, args = list(xi1 = 1, xi2 = 2, nu = 200), colour = "red")
```

```R
# Define hyperprior update function
ud <- function(xi, nu, x, T = identity) {
    xi <- xi + 1/(1 + nu)*(T(x) - xi)
    return(xi)
}
```

```R
# Define sufficient statistics of univariate
# Gaussian with both parameters unknown
Tnorm <- function(x) {
    T <- c(x, x^2)
    return(T)
}
```

```R
# Set parameters of sampling distribution
mux <- 5
sdx <- 1/4 # ie pix == 16

# Set hyperpriors to define prior
xi <- c(0, 1/8)
nu <- 1

# Sample n observations
n <- 10001
xs <- rnorm(n, mux, sdx)

# Hyperprior update loop
xis <- xi
nus <- nu
for (i in 1:n) {
    xi <- ud(xi, nu, xs[i], Tnorm)
    xis <- rbind(xis, xi)
    nu <- nu + 1
    nus <- append(nus, nu)
}
```

```R
# Set up plot
options(repr.plot.width = 9, repr.plot.height = 3)
p <- ggplot(data.frame(x = c(-4, 8)), aes(x = x))

# Plot prior predictive
p <- p + stat_function(fun = dnp, args = list(xi1 = xis[1,1], xi2 = xis[1,2], nu = nus[1]), n = 401, colour = "black")

# Plot every mth posterior predictive
m <- 200
for (i in seq(2, n + 1, m)) {
    p <- p + stat_function(fun = dnp, args = list(xi1 = xis[i,1], xi2 = xis[i,2], nu = nus[i]), n = 401, colour = "grey")
}

# Plot sampling distribution
p <- p + stat_function(fun = dnorm, args = list(mean = mux, sd = sdx), n = 401, colour = "red")

# Make the plot
p
```

```R
gauss_dens <- function(x, mu, la) {
    T <- c(x, x^2)
    h <- 1/sqrt(2*pi)
    eta <- c(la*mu, -la/2)
    B <- (la*mu^2 - log(la))/2
    p <- drop(h*exp(eta %*% T - B))

    return(p)
}
```

```R
gauss_dens(3,0,1)
```

0.00443184841193801

```R
dnorm(3)
```

0.00443184841193801

```R

```


### Analytic Solutions to Variational Bayes

The general setting for variational Bayesian techniques begins from Bayesian inference, where we wish to infer a posterior.

\begin{align}
p(z|x) = \frac{p(x|z)p(z)}{p(x)}
\end{align}

Often times the marginal $p(x)$ is too complicated to derive analytically or calcuate computationally. In this case, we call it 'intractable'. When this happens we may wish to approximate the posterior $p(z|x)$. A common technique involves determining a surrogate distribution $q(z|x)$ that is as close as possible to the true posterior. We can measure the closeness between two distributions with the well known Kullback Leibler (KL) divergence. In this case we use the reverse KL divergence.

\begin{align}
D_{\text{KL}}(q(z|x)||p(z|x)) = \int_z q(z|x) \log{\frac{q(z|x)}{p(z|x)}} \text{d}z
\end{align}

However, posing the problem this way has not helped us yet as the KL retains the unknown posterior within it. In other words, we need the thing we are trying to approximate, to approximate it. There is a fix. Let us rewrite the KL term as follows...

\begin{align}
D_{\text{KL}}(q(z|x)||p(z|x)) &= \int_z q(z|x) \log{\frac{q(z|x)}{p(z|x)}} \text{d}z \\
&= \int_z q(z|x) \log{\frac{q(z|x)p(x)}{p(z|x)p(x)}} \text{d}z \\
&= \int_z q(z|x) \log{\frac{q(z|x)}{p(x,z)}} \text{d}z + \int_z q(z|x) \log{p(x)} \text{d}z \\
&= \int_z q(z|x) \log{\frac{q(z|x)}{p(x,z)}} \text{d}z + \log{p(x)} \\
\end{align}

We wish to make the right hand side as small as possible. Recall that the data generating distribution $p(x,z)$ is considered fixed and $x$ is considered given, or observed, therefore $\log{p(x)}$ is a constant. Minimizing the right hand side then amounts to minimizing the integral where we've replaced the posterior with the joint which is more often available. However, before we move on let us make a quick observation from information theory. As Claude Shannon demonstrated, KL divergence is greater than or equal to zero, therefore we know that

\begin{align}
\int_z q(z|x) \log{\frac{q(z|x)}{p(x,z)}} \text{d}z + \log{p(x)} &\ge 0 \\
\log{p(x)} &\ge -\int_z q(z|x) \log{\frac{q(z|x)}{p(x,z)}} \text{d}z \\
\log{p(x)} &\ge \mathcal{L}[q]
\end{align}
We see that the functional $\mathcal{L}[q]$ is a lower bound for the marginal likelihood, or model evidence. For this reason $\mathcal{L}$ is often referred to as the *evidence lower bound* or *ELBO* for short. We will adopt this nomenclature.

Our initial objective was to minimize the reverse KL divergence between the true posterior and our surrogate distribution $q(z|x)$. We have now reformulated that problem to maximizing the ELBO. As posed thus far, the maximum for $\mathcal{L}$ is when $\mathcal{L} = \log{p(x)}$, in which case the KL divergence would be zero and the optimal variational distribution will equal the true posterior $q(z|x) = p(z|x)$. This is rarely possible, so assumptions are made regarding $q(z|x)$ that constrain it to a family of distributions which simplifies the analysis. Under such assumptions, the distribution $q$ is no longer free to take any form and we can not guarantee that our optimal $q^*(z|x)$ equals the posterior. It is in this sense that our technique becomes an approximation. Albeit, there are times when the true posterior lives within the family of our variational distribution $q(z|x)$ and an exact solution is possible. The larger the family, the more likely this is to happen. Though, as the family increases in size, the complexity of the problem often does as well. Therefore, one might consider variational inference as the art of maximizing the expressive power of your model while minimizing the complexity of the resulting analysis. In any case, we often have to settle for a reasonable approximation to the posterior, in the KL sense.

### Mean Field Model

One simplifying assumption that is made to constrain the variational family is that the distribution $q(z|x)$ factorizes over the latent variables $q(z|x) = q(z_1|x)q(z_2|x)\cdots q(z_n|x)$. The assumption is often referred to as the *mean field* approximation, inspired by physics.

**(expand on this in the future with your seminar notes)**

### Parametric Families

Another technique, sometimes used in conjuction with the former, is to assume a parametric family for our variational distribution $q_{\phi}(z|x)$, parameterized by $\phi$. The ELBO simplifies to a function of the variational parameters $\mathcal{L}(\phi)$. This can sometimes be solved analytically but is most often approached computationally. In this form standard stochastic optimization techniques can be applied.

\begin{align}
\mathcal{L}(\phi;x) &= -\int_z q_{\phi}(z|x)  \log{\frac{q_{\phi}(z|x)}{p(x,z)}} \text{d}z \\
&\approx \frac{1}{L} \sum_i^L \log{p(x,z_i)} - \log{q_{\phi}(z_i|x)}
\end{align}

where $z_i \sim q_{\phi}(z|x)$. From here, one takes the pathwise derivative of the ELBO with respect to the variational parameters $\phi$ and optimizes with different samples $x \sim p(x)$.

### Variational Autoencoders

For variational autoencoders we rewrite the ELBO as follows

\begin{align}
\mathcal{L}[q] &= -\int_z q(z|x) \log{\frac{q(z|x)}{p(x,z)}} \text{d}z \\
&= \int_z q(z|x) \log{\frac{p(x|z)p(z)}{q(z|x)}} \text{d}z \\
&= -D_{\text{KL}}(q(z|x)||p(z)) + \mathbb{E}_{q(z|x)} \big[ \log{p(x|z)} \big]\\
\end{align}

By specifying different assumptions about $q(z|x)$, $p(x|z)$ and $p(z)$ we will arrive at different algorithms. For example if we choose parametric forms amenable to analysis we can derive closed form solutions for the lower bound. These simplifications remove the need for Monte Carlo estimation. Let us examine the most common example. We begin by parameterizing the encoder $q_{\phi}(z|x)$ as a diagonal covariance Gaussian $\mathcal{N}(z|x; \mu,\sigma^2 \cdot I)$ where $\sigma^2 \in \mathbb{R}^N$ and assume $p(z)$ is a centered isotropic Gaussian $p(z) = \mathcal{N}(z; 0,I)$ (i.e. parameterless).

\begin{align}
q_{\phi}(z|x) &= (2\pi )^{-\frac{N}{2}} \big( \prod_i \sigma_i^2 \big )^{-\frac{1}{2}} \text{exp}\big\{-\frac{1}{2}(z-\mu)^T(\sigma^2 \cdot I)^{-1}(z-\mu)\big\} \\
\end{align}
\begin{align}
p(z) &= (2\pi )^{-\frac{N}{2}} \text{exp}\big\{-\frac{z^T z}{2}\} \\
\log{p(z)} &= -\frac{N}{2}\log{2\pi} -\frac{z^T z}{2} \\
\end{align}

With these definitions we can evaluate the negative KL term as

\begin{align}
-D_{\text{KL}}(q_{\phi}(z|x)||p(z)) &= \int_z q_{\phi}(z|x) \log{\frac{p(z)}{q_{\phi}(z|x)}} \text{d}z \\
&= \int_z q_{\phi}(z|x) \log{p(z)} \text{d}z - \int_z q_{\phi}(z|x) \log{q_{\phi}(z|x)} \text{d}z \\
&= \int_z \bigg((2\pi )^{-\frac{N}{2}} \big( \prod_i \sigma_i^2 \big )^{-\frac{1}{2}} \text{exp}\big\{-\frac{1}{2}(z-\mu)^T(\sigma^2 \cdot I)^{-1}(z-\mu)\big\}\bigg)\bigg(-\frac{N}{2}\log{2\pi} -\frac{z^T z}{2}\bigg)\text{d}z + H\big[q_{\phi}(z|x)\big]
\end{align}

where we have identified the term on the right as the entropy $H$ which has a known closed form solution for the multivariate Gaussian case.

\begin{align}
H\big[\mathcal{N}(x; \mu,\Sigma)\big] &= \frac{1}{2} \log{\text{det}(2 \pi e \Sigma)} \\
\end{align}

Therefore in our case

\begin{align}
H\big[\mathcal{N}(z|x; \mu,\sigma^2 \cdot I)\big] &= \frac{1}{2} \log\big(\text{det}(2 \pi e \sigma^2 \cdot I)\big) \\
&= \frac{1}{2} \log\big((2 \pi e)^N \prod_i \sigma_i^2\big) \\
&= \frac{N}{2} \log(2 \pi e) + \frac{1}{2} \sum_i \log(\sigma_i^2) \\
\end{align}

Now to address the more daunting integral

\begin{align}
& \int_z q_{\phi}(z|x) \log{p(z)} \text{d}z \\
=& \int_z \bigg((2\pi )^{-\frac{N}{2}} \big( \prod_i \sigma_i^2 \big )^{-\frac{1}{2}} \text{exp}\big\{-\frac{1}{2}(z-\mu)^T(\sigma^2 \cdot I)^{-1}(z-\mu)\big\}\bigg)\bigg(-\frac{N}{2}\log{2\pi} -\frac{z^T z}{2}\bigg)\text{d}z
\end{align}

If we expand the inner term and factor out the constants we see that

\begin{align}
& \int_z \bigg((2\pi )^{-\frac{N}{2}} \big( \prod_i \sigma_i^2 \big )^{-\frac{1}{2}} \text{exp}\big\{-\frac{1}{2}(z-\mu)^T(\sigma^2 \cdot I)^{-1}(z-\mu)\big\}\bigg)\bigg(-\frac{N}{2}\log{2\pi} -\frac{z^T z}{2}\bigg)\text{d}z \\
=&  -\frac{N}{2}\log{2\pi} \int_z q_{\phi}(z|x) \text{d}z  -\int_z \frac{z^T z}{2}\bigg((2\pi )^{-\frac{N}{2}} \big( \prod_i \sigma_i^2 \big )^{-\frac{1}{2}} \text{exp}\big\{-\frac{1}{2}(z-\mu)^T(\sigma^2 \cdot I)^{-1}(z-\mu)\big\}\bigg) \text{d}z \\
\end{align}

---

assume we have only $M$ observations from the marginal data generating distribution $p(x)$. In this case we must not only parameterize the variational distribution $q_{\phi}(z|x)$ but also the true likelihood $p_{\theta}(x|z)$. Furthermore, we

\begin{align}
\nabla_{\phi} \mathcal{L}(\phi;x) &= -\nabla_{\phi} \int_z q_{\phi}(z|x)  \log{\frac{q_{\phi}(z|x)}{p(x,z)}} \text{d}z \\
&=  -\nabla_{\phi} \int_z q_{\phi}(z|x) \log{q_{\phi}(z|x)} \text{d}z + \int_z \nabla_{\phi} q_{\phi}(z|x) \log{p(x,z)} \text{d}z \\
&=  -\nabla_{\phi} \text{H}[q_{\phi}(z|x)] + \int_z \nabla_{\phi} q_{\phi}(z|x) \log{p(x,z)} \text{d}z
\end{align}

where we chose to represent the first integral as the entropy $\text{H}$ because many parametrics models have known closed form solutions for the entropy and this will simplify things to come.


# Entropy and Cross-Entropy

Created by Andres Segura-Tinoco
Created on Mar 15, 2021

Source: https://en.wikipedia.org/wiki/Entropy_(information_theory)

```
import math
```

**Entropy** is a measure of unpredictability, so its value is inversely related to the compression capacity of a chain of symbols.
\begin{align}
    Entropy(X) = H(X) = \sum_{i=1}^n P(x_i) \log_{2} \frac{1}{P(x_i)} \tag{1}
\end{align}

The **Cross-Entropy** of the distribution $q$ relative to a distribution $p$ over a given set is defined as follows:
\begin{align}
    Cross-Entropy(p, q) = H(p, q) = \sum_{i=1}^n P(x_i) \log_{2} \frac{1}{Q(x_i)} \tag{2}
\end{align}

```
# Probability distributions P of x set
p_x = [0.1, 0.2, 0.15, 0.15, 0.4]
sum(p_x)
```

    1.0

```
# Calculate entropy of Px
h_x = 0
for x in p_x:
  h_x -= x * math.log2(x)
h_x
```

    2.1464393446710153

```
# Probability distributions Qgu of x set
q_x = [0.08, 0.24, 0.13, 0.17, 0.38]
sum(q_x)
```

    1.0

```
# Calculate cross-entropy of Qx relative to Px
ce_x = 0
for p, q in zip(p_x, q_x):
  ce_x += p * math.log2(1/q)
ce_x
```

    2.1595073003443446

```
# Validation
h_x < ce_x
```

    True

You can contact me on Twitter | GitHub | LinkedIn


# Solving some system of equations

```python
import numpy as np
```

```python
# Solving following system of linear equation
# 3x + 2y = 7
# 2x + 3y = 9

x = np.array([[3, 2],[2,3]])
y = np.array([7, 8])

print(np.linalg.solve(x,y))
```

    [ 1.  2.]

```python
# Solving following system of linear equation
# 9x - 17y = -20
# -13x + 7y = -94

x = np.array([[9, -17],[-13,7]])
y = np.array([-20, -94])

print(np.linalg.solve(x,y))
```

    [ 11.   7.]

```python
# 5x - 2y = -13
# 4x + 5y = -6

x = np.array([[5, -2],[4,5]])
y = np.array([-13, -6])

print(np.linalg.solve(x,y))
```

    [-2.33333333  0.66666667]

```python
# 5x + 7y = 11
# 20x - 18y = 39

x = np.array([[5, 7],[20,-18]])
y = np.array([11, 39])

print(np.linalg.solve(x,y))
```

    [ 2.04782609  0.10869565]

```python
# 3x - 2y + z = 7
# x + y + z = 2
# 3x - 2y - z = 3

x = np.array([[3, -2, 1],[1, 1, 1],[3, -2, -1]])
y = np.array([7, 2, 3])

print(np.linalg.solve(x,y))
```

    [ 1. -1.  2.]

```python
# 5x - 2y = -13
# 4x + 5y = -6

from sympy import *

x, y = symbols(['x', 'y'])
system = [Eq(5*x - 2*y, -7),
          Eq(4*x + 5*y, -8)]

solutions = solve(system, [x, y])
print(solutions)
```

    {x: -17/11, y: -4/11}

```python
# 3x - 2y + z = 7
# x + y + z = 2
# 3x - 2y - z = 3

from sympy import *

x, y, z = symbols(['x', 'y', 'z'])
system = [Eq(3*x - 2*y + z, 7),
          Eq(x + y + z, 2),
          Eq(3*x - 2*y - z, 3)]

solutions = solve(system, [x, y, z])
print(solutions)
```

    {x: 1, y: -1, z: 2}

```python
x = np.array([[3, -2, 1],[1, 1, 1],[3, -2, -1]])
y = np.array([7, 2, 3])

# linalg.solve is the function of NumPy to solve a system of linear scalar equations
print("Solutions:\n", np.linalg.solve(x, y))
```

    Solutions:
     [ 1. -1.  2.]

## LU decomposition with SciPy

```python
# LU decomposition with SciPy
import scipy.linalg as linalg # Package for LU decomposition

x = np.array([[3, -2, 1],[1, 1, 1],[3, -2, -1]])
y = np.array([7, 2, 3])

LU = linalg.lu_factor(x)
x = linalg.lu_solve(LU, y)
print("Solutions:\n",x)
```

    Solutions:
     [ 1. -1.  2.]

```python
import scipy
x = scipy.array([[3, -2, 1],[1, 1, 1],[3, -2, -1]])
P, L, U = scipy.linalg.lu(x)
```

```python
print("x:\n", x)
print("-"*50)
print("P:\n", P)
print("-"*50)
print("L:\n", L)
print("-"*50)
print("U:\n", U)
```

    x:
     [[ 3 -2  1]
     [ 1  1  1]
     [ 3 -2 -1]]
    --------------------------------------------------
    P:
     [[ 1.  0.  0.]
     [ 0.  1.  0.]
     [ 0.  0.  1.]]
    --------------------------------------------------
    L:
     [[ 1.          0.          0.        ]
     [ 0.33333333  1.          0.        ]
     [ 1.          0.          1.        ]]
    --------------------------------------------------
    U:
     [[ 3.         -2.          1.        ]
     [ 0.          1.66666667  0.66666667]
     [ 0.          0.         -2.        ]]

## Euclidean

```python
from scipy.spatial import distance
a = (1, 2, 3)
b = (4, 5, 6)
euc_dist = distance.euclidean(a, b)

print("Euclidean Distance:", euc_dist)
```

    Euclidean Distance: 5.19615242271

## Hadamard Product

```python
a = np.array([[1,2],[3,4]])
b = np.array([[5,6],[7,8]])
hp = np.multiply(a,b)

print("Hadamard Product:\n", hp)
```

    Hadamard Product:
     [[ 5 12]
     [21 32]]

```python
# Another method
a * b
```

    array([[ 5, 12],
           [21, 32]])

```python
# Another method
np.multiply(a,b)
```

    array([[ 5, 12],
           [21, 32]])

## Dot Product

```python
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

dp = x @ y
print('Dot Product:\n', dp)
```

    Dot Product:
     [[19 22]
     [43 50]]

```python
# Another Method
np.dot(x,y)
```

    array([[19, 22],
           [43, 50]])

##  Dot product of vectors

Finding the product of the summation of two vectors and the output will be a single vector.

```python
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
dotproduct = sum(i*j for i,j in zip(x,y))

print('Dot product is : ' , dotproduct)
```

    Dot product is :  [26 44]

```python
x = [1,2,3,4]
y = [5,6,7,8]
dotproduct = sum(i*j for i,j in zip(x,y))

print('Dot product is : ' , dotproduct)
```

    Dot product is :  70

## Inner Product

```python
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

ip = np.inner(x,y)
print('Inner Product:\n', ip)
```

    Inner Product:
     [[17 23]
     [39 53]]


```python
%pylab inline
import seaborn
```

    Populating the interactive namespace from numpy and matplotlib

```python
from plotting import plot_trajectory
```

# Joint Space Trajectories

## Point-to-Point Motion

### Cubic Polynomial

\begin{align}
q(t) &= a_3 t^3 + a_2 t^2 + a_1 t + a_0 \\
\dot{q}(t)  &= 3 a_3 t^2 + 2 a_2 t + a_1 \\
\ddot{q}(t) &= 6 a_3 t + 2 a_2
\end{align}

\begin{align}
a_0 &= q_i \\
a_1 &= \dot{q}_i \\
a_3 t_f^3 + a_2 t_f^2 + a_1 t_f + a_0 &= q_f \\
3 a_3 t_f^2 + 2 a_2 t_f + a_1 &= \dot{q}_f
\end{align}

\begin{align}
\begin{pmatrix} q_i \\ q_f \\ \dot q_i \\ \dot q_f \end{pmatrix} =
\begin{pmatrix}
  0 & 0 & 0 & 1 \\
  t^3 & t^2 & t & 1 \\
  0 & 0 & 1 & 0 \\
  3t^2 & 2t & 1 & 0
  \end{pmatrix}
  \begin{pmatrix} a_3 \\ a_2 \\ a_1 \\ a_0 \end{pmatrix}
\end{align}

```python
def cubic_trajectory(current_position, target_position,
                     current_velocity, target_velocity,
                     duration_in_seconds):
    trajectories = []
    t = duration_in_seconds
    xs = linspace(0,t)
    for qi, qf, dqi, dqf in zip(current_position, target_position,
                                current_velocity, target_velocity):
        A = np.array([[0.0,0.0,0.0,1.0],
                      [t**3, t**2, t, 1.],
                      [0.0, 0.0, 1.0, 0.0],
                      [3.0 * t**2, 2*t, 1.0, 0.0]])

        b = np.array([qi, qf, dqi, dqf])
        x = np.linalg.solve(A,b)

        qs = np.polyval(x, xs)
        dqs = np.polyval([3. * x[0], 2. * x[1], x[2]], xs)
        ddqs = np.polyval([6. * x[0], 2. * x[1]], xs)

        trajectories.append((qs, dqs, ddqs))
    return trajectories
```

```python
qi = [0,np.pi]
qf = [np.pi,0]
dqi = [0,0]
dqf = [0,0]
trajectories = cubic_trajectory(qi, qf, dqi, dqf, 50)
```

```python
plot_trajectory(trajectories[0], iscubic=True)
```

### Cubic Polynomial

```python
def quintic_trajectory(current_position, target_position, current_velocity,
                       target_velocity, current_acceleration,
                       target_acceleration, duration_in_seconds):
    trajectories = []
    t = duration_in_seconds
    xs = linspace(0, t)
    for qi, qf, dqi, dqf, ddqi, ddqf in zip(current_position, target_position,
                                            current_velocity, target_velocity,
                                            current_acceleration, target_acceleration):
        A = np.array(
            [[0.0, 0.0, 0.0, 0.0, 0.0, 1.0],
             [t**5, t**4, t**3, t**2, t, 1.0],
             [0.0, 0.0, 0.0, 0.0, 1.0, 0.0],
             [5. * t**4, 4. * t**3, 3. * t**2, 2. * t, 1., 0.0],
             [0.0, 0.0, 0.0, 2.0, 0.0, 0.0],
             [20. * t**3, 12. * t**2, 6. * t, 2., 0.0, 0.0]])

        b = np.array([qi, qf, dqi, dqf, ddqi, ddqf])
        x = np.linalg.solve(A, b)

        qs = np.polyval(x, xs)
        dqs = np.polyval([5. * x[0], 4. * x[1], 3. * x[2], 2. * x[3], x[4]], xs)
        ddqs = np.polyval([20. * x[0], 12. * x[1], 6. * x[2], 2. * x[3]], xs)

        trajectories.append((qs, dqs, ddqs))
    return trajectories
```

```python
qi = [0,np.pi]
qf = [np.pi,0]
dqi = [0,0]
dqf = [0,0]
ddqi = [0,0]
ddqf = [0,0]
trajectories = quintic_trajectory(qi, qf, dqi, dqf, ddqi, ddqf, 50)
```

```python
plot_trajectory(trajectories[0], iscubic=False)
```


## Introduction
-----
You (an electrical engineer) wish to determine the resistance of an electrical component by using Ohm's law. You remember from your high school circuit classes that $$V = RI$$ where $V$ is the voltage in volts, $R$ is resistance in ohms, and $I$ is electrical current in amperes. Using a multimeter, you collect the following data:

| Current (A) | Voltage (V) |
|-------------|-------------|
| 0.2         | 1.23        |
| 0.3         | 1.38        |
| 0.4         | 2.06        |
| 0.5         | 2.47        |
| 0.6         | 3.17        |

Your goal is to
1. Fit a line through the origin (i.e., determine the parameter $R$ for $y = Rx$) to this data by using the method of least squares. You may assume that all measurements are of equal importance.
2. Consider what the best estimate of the resistance is, in ohms, for this component.

## Getting Started
----

First we will import the neccesary Python modules and load the current and voltage measurements into numpy arrays:

```python
import numpy as np
from numpy.linalg import inv
import matplotlib.pyplot as plt

# Store the voltage and current data as column vectors.
I = np.mat([0.2, 0.3, 0.4, 0.5, 0.6]).T
V = np.mat([1.23, 1.38, 2.06, 2.47, 3.17]).T
```

Now we can plot the measurements - can you see the linear relationship between current and voltage?

```python
plt.scatter(np.asarray(I), np.asarray(V))

plt.xlabel('Current (A)')
plt.ylabel('Voltage (V)')
plt.grid(True)
plt.show()
```

```python

```

## Estimating the Slope Parameter
----
Let's try to estimate the slope parameter $R$ (i.e., the resistance) using the least squares formulation from Module 1, Lesson 1 - "The Squared Error Criterion and the Method of Least Squares":

\begin{align}
\hat{R} = \left(\mathbf{H}^T\mathbf{H}\right)^{-1}\mathbf{H}^T\mathbf{y}
\end{align}

If we know that we're looking for the slope parameter $R$, how do we define the matrix $\mathbf{H}$ and vector $\mathbf{y}$?

```python
# Define the H matrix, what does it contain?
H = I

H_transpose = H.transpose()
inverse = inv(np.dot(H_transpose, H))

# Now estimate the resistance parameter.
# R = ...
y = V
R = inverse * H_transpose * y

print('The slope parameter (i.e., resistance) for the best-fit line is:')
print(R)
R = R[0, 0]
print(R)

```

    The slope parameter (i.e., resistance) for the best-fit line is:
    [[5.13444444]]
    5.134444444444445

## Plotting the Results
----
Now let's plot our result. How do we relate our linear parameter fit to the resistance value in ohms?

```python
I_line = np.arange(0, 0.8, 0.1)
V_line = R*I_line

plt.scatter(np.asarray(I), np.asarray(V))
plt.plot(I_line, V_line)
plt.xlabel('current (A)')
plt.ylabel('voltage (V)')
plt.grid(True)
plt.show()
```

If you have implemented the estimation steps correctly, the slope parameter $\hat{R}$ should be close to the actual resistance value of $R = 5~\Omega$. However, the estimated value will not match the true resistance value exactly, since we have only a limited number of noisy measurements.

State Estimation and Localization for Self-Driving Cars/week_1/Programming_excersie/C2M1L1.ipynb


# 15.7. Analyzing a nonlinear differential system — Lotka-Volterra (predator-prey) equations

```python
from sympy import *
init_printing(pretty_print=True)

var('x y')
var('a b c d', positive=True)
```

```python
f = x * (a - b * y)
g = -y * (c - d * x)
```

```python
solve([f, g], (x, y))
```

```python
(x0, y0), (x1, y1) = _
```

```python
M = Matrix((f, g))
M
```

```python
J = M.jacobian((x, y))
J
```

```python
M0 = J.subs(x, x0).subs(y, y0)
M0
```

```python
M0.eigenvals()
```

```python
M1 = J.subs(x, x1).subs(y, y1)
M1
```

```python
M1.eigenvals()
```


```python
import sympy
```

```python
x = sympy.symbols("x")
slope = 3

# Phi-sides
phi1 = 3*x
phi2 = 2-3*x
phi3 = 3*x-1
phi4 = 3-3*x
```

```python
interval1 = sympy.integrate(slope**2, (x, 0, sympy.Rational(1, 3)))
interval2 = sympy.integrate(slope**2, (x, sympy.Rational(1, 3), sympy.Rational(2, 3)))

interval3 = sympy.integrate(phi1**2, (x, 0, sympy.Rational(1, 3)))
interval4 = sympy.integrate(phi2**2, (x, sympy.Rational(1, 3), sympy.Rational(2, 3)))

K_11 = interval1 + interval2 + interval3 + interval4
K_11
```

$\displaystyle \frac{56}{9}$

```python
interval1 = sympy.integrate(-slope*slope, (x, sympy.Rational(1, 3), sympy.Rational(2, 3)))
interval2 = sympy.integrate(phi2*phi3, (x, sympy.Rational(1, 3), sympy.Rational(2, 3)))

K_12 = interval1 + interval2
K_12
```

$\displaystyle - \frac{53}{18}$

```python
# Because of symmetry

sympy.Matrix([[K_11, K_12], [K_12, K_11]])
```

$\displaystyle \left[\begin{matrix}\frac{56}{9} & - \frac{53}{18}\\- \frac{53}{18} & \frac{56}{9}\end{matrix}\right]$


Author: Drishika Nadella

Date: 4th March 2021

```python
import numpy as np
from sympy import *
```

```python
def func(x):
    return x*(x-1)
```

```python
def derivative(x, delta):
    f_ = (func(x+delta) - func(x))/delta
    return f_
```

```python
# Analytical derivative

x = Symbol('x')
y = func(x)
yprime = y.diff(x)

f = lambdify(x, yprime, 'numpy')
f(1)
```

    1

```python
print(derivative(1, 10**-2))
```

    1.010000000000001

```python
print(derivative(1, 10**-4))
```

    1.0000999999998899

```python
print(derivative(1, 10**-6))
```

    1.0000009999177333

```python
print(derivative(1, 10**-8))
```

    1.0000000039225287

```python
print(derivative(1, 10**-10))
```

    1.000000082840371

```python
print(derivative(1, 10**-12))
```

    1.0000889005833413

```python
print(derivative(1, 10**-14))
```

    0.9992007221626509

Until $10^{-8}$, the accuracy got better, and then it got worse again. This is because of the multiplication of very small floating point numbers being multiplied with each other, which increases the round-off error.


```python
%load_ext autoreload
%autoreload 2

# Needed imports
from abtest import utils # local utils; see utils.py
import pandas as pd
import numpy as np
from statsmodels.stats import proportion as ssp
from scipy import stats

%matplotlib inline
from matplotlib import pyplot as plt
plt.style.use('ggplot')
import seaborn as sns

# Some settings for the notebook
%precision 5
np.set_printoptions(precision=4)
pd.set_option('precision',4)
```

The focus of this section is on the following case.
The experiment consists of two groups: control and variant.
Each user can either convert or not and the metric of interest is the conversion rate.
Set the null and alternative hypotheses as follows:

$$\begin{align}
H_0:\quad & p_c = p_v \\
H_a:\quad & p_c \neq p_v
\end{align}$$

As before, you can start with a simulation of an experiment.

```python
exp = utils.generate_experiment(seed=43, N=10000, control_cr=0.3, variant_cr=0.3)
exp
```

      Converted
      Visited
      CR_pct

      Control
      3090
      10000
      30.90

      Variant
      2952
      10000
      29.52

The question you will answer in this section is:
> Is the difference witnessed between the proportions "real" or just due to randomness?

This is the time to introduce the *two-sided $Z$-test*

## Two-sided $Z$-test

Recall that $p_c$ is the conversion rate (or proportion) for the control group and $p_v$ is the one for the variant; both are, in reality, *unknown!*
Next, $\hat{p}_c$ and $\hat{p}_v$ are the rates witnessed in the experiment.
In the table above they are given as percentages in the column `CR_pct`.

### By hand

Under the hood, for each group you obtain a vector $c = (c_1, c_2, \ldots, c_N)$ and $v = (v_1, v_2, \ldots, v_N)$ where $c_i, v_i \in \{0,1\}$ and $N$ is the number of visitors in the group (here assumed to be the same).
From this, we have that:

$$
\begin{align}
\hat{p}_c = \frac{\sum c_i}{N} \\
\hat{p}_v = \frac{\sum v_i}{N}
\end{align}
$$

In other words, the witnessed proportions are means of the values observed.
Therefore, by the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem), both are approximately normally distributed.
Thus also their difference $\hat{p}_c - \hat{p}_v$.
In this case, the following  $z$-statistic is to be used [[ref]](https://onlinecourses.science.psu.edu/stat414/node/268)

$$z = \frac{\hat{p}_c - \hat{p}_v}{\sqrt{\frac{\hat{p}(1-\hat{p})}{c_t} + \frac{\hat{p}(1-\hat{p})}{v_t} }} $$

Where

$$\hat{p} = \frac{c_t \hat{p}_c + v_t \hat{p}_v}{c_t + v_t}$$

is the *pooled* proportion where $c_t$ and $v_t$ are the sample sizes of the control and variant groups, respectively.
You can find the above formula implemented in `utils.manual_z_score`; and you can compute the statistic for the data generated for the experiment above.

Note that in this computation the *pooled* version of the variant is used. An un-pooled computation can also be used, but it is less recommended [[ref]](https://stats.stackexchange.com/a/17205/54320).

```python
z_score = utils.manual_z_score(exp)
z_score
```

    2.12516

It is time to better understand what this score means.
The obtained $Z$-score indicates where the results of the experiment lies on the normal distribution:

```python
x = np.arange(-5, 5, step=0.1)
z_score = utils.manual_z_score(exp)
x = np.append(x, [-z_score, z_score])
x = np.sort(x)
y = stats.norm.pdf(x)

fig, ax = plt.subplots(1, 1)
ax.plot(x , y)
ax.fill_between(x[x <= -abs(z_score)],
                y1=y[x <= -abs(z_score)], alpha=0.5, facecolor='blue');
ax.fill_between(x[x >= abs(z_score)],
                y1=y[x >= abs(z_score)], alpha=0.5, facecolor='blue')
plt.axvline(z_score, color='gray')
plt.axvline(-z_score, color='gray')
plt.text(z_score + 0.1, 0.2, '$|Z|$', rotation=90)
plt.text(-z_score + 0.1, 0.2, '$-|Z|$', rotation=90)
plt.title('Normal distribution with $\mu = 0$ and $\sigma^2 = 1$');
```

Recall that the question that has to be answered is:
> What is the likelihood to witness even more extreme results than those witnessed in the experiment?

This can be answered by computing the area to the left of $-|Z|$ and to the right of $|Z|$ (shaded in the plot above).
By symmetry it is enough to compute the area to the left of $-|Z|$ and multiply the result by $2$.
The [cumulative distribution function (cdf)](https://en.wikipedia.org/wiki/Cumulative_distribution_function) of the normal distribution can be used:

```python
2 * stats.norm.cdf(-abs(z_score))
```

    0.03357

Say hello to the mighty $p$-value!
This value tells you how likely is it to witness even more extreme difference given the null hypothesis holds.
Recall, that $H_0$ is that the two proportions are the same.
You can now convince yourself that this is indeed the meaning of the $p$-value.
Do so by running a simulation: run $M$ identical experiments and check in how many of them you witness more extreme results.

```python
M = 1000
N = 10000
base_cr_diff = abs((exp.Converted / exp.Visited).diff()[-1])
np.random.seed(1492)
seeds = np.random.randint(0, 2**32-1, size=(M))
control_cr = 0.3 # 30%
variant_cr = 0.3 # 30%
experiments = [
    utils.generate_experiment(
        seed=seed, N=N, control_cr=control_cr, variant_cr=variant_cr
    ) for seed in seeds]
cr_diffs = pd.Series([(exp.Converted / exp.Visited).diff()[-1] for exp in experiments])

print(
    "In ~{}% of the simulations the difference between the proportions\n"
    "is at least {:03.3f} (the one in the base experiment)".format(
        np.round(100 * np.sum(np.abs(cr_diffs) > base_cr_diff) / M, decimals=2),
        base_cr_diff
    ))
```

    In ~3.5% of the simulations the difference between the proportions
    is at least 0.014 (the one in the base experiment)

#### Summary

You obtained the results of a (simulated) experiment (with random seed $43$) where the underlaying proportions are predefined.
Due to pure randomness you witness a difference between the conversion rates which might be considered big:
$$
100 \cdot \frac{30.90 - 29.52}{29.52} = 4.675\%
$$

But is it really that big? If you had ran more experiments, what is the likelihood that you would witness an even bigger difference?
The $p$-value as computed above, answers this very question.
It is now a question of interpretation; can you live peacefully knowing that this extremity is to be expected in $\sim 3.38\%$ of the cases?
In all other cases the difference is smaller!
Are you taking a risk by deciding based on this finding?
Later on you will learn how to better evaluate the validity (or *power*) of the test.

### The Python way

Now, once you hopefully have deeper understanding of the meaning of the $p$-value, you should know how to easily compute it.
The easiest way is to use [`statsmodels.stats.proportions.proportions_ztest`](http://www.statsmodels.org/dev/generated/statsmodels.stats.proportion.proportions_ztest.html):

```python
# ssp.proportions_ztest returns a tuple contaiing two values
res_ztest = ssp.proportions_ztest(exp.Converted, exp.Visited, alternative='two-sided')
print("z-score = %f" % res_ztest[0])
print("p-value = %f" % res_ztest[1])
```

    z-score = 2.125162
    p-value = 0.033573

The values above are exactly the $Z$-score and the $p$-value as we computed by hand above.
At this point it is worthy to mention also the $\chi^2$-test which is implemented in [`scipy.stats.chi2_contingency`](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.chi2_contingency.html).
For this to work, the aggregation of the data should be a little altered:

```python
# In this case the function returns also the number of degrees of freedom and
# the expected frequencies
res_chi2 = stats.contingency.chi2_contingency(
    pd.concat([exp.Converted, exp.Visited - exp.Converted], axis=1),
    correction=False) # Keeping the default True would yield different resutls
                      # due to the Yates’ correction
print("z-score = %f" % res_chi2[0])
print("p-value = %f" % res_chi2[1])
```

    z-score = 4.516315
    p-value = 0.033573

You should notice that this approach yields the *same* $p$-value but a different score.
The math behind this is beyond the scope of this post.


## $$a_0 = \frac{2}{T}\int_a^b{f(x)}dx$$
## $$a_n =  \frac{2}{T}\int_a^b{f(x)\;cos(n\omega x)dx}$$
## $$b_n =  \frac{2}{T}\int_a^b{f(x)\;sin(n\omega x)dx}$$

```python
from sympy import *
import sympy as sp
import matplotlib.pyplot as plt
import numpy as np
import handcalcs.render
plt.style.use('dark_background')

x, t = symbols('x t', real = True)
n = symbols('n', real = True, positive = True, integer = True)
T = sp.symbols('T', real = True, positive = True)
a = symbols('a', real = True, positive = True, constant = True)
number_of_terms = 9
```

# Define function, free variable and range

```python
f = sp.Piecewise((x, x>0), (-x, x<0))
free_var = x
period = [-2, 2]
T = period[1]-period[0]
T
```

    4

# Calculation of coefficients

```python
a_0 = (2/T) * integrate(f, (free_var, period[0], period[1]))
a_0 = a_0.simplify()

a_n =(2/T) * (integrate(f*cos(n*2*pi*free_var/T), (free_var,period[0], period[1])))
a_n = a_n.simplify()

b_n =(2/T) * integrate(f*sin(n*2*pi/T*free_var), (free_var, period[0], period[1]))
b_n = b_n.simplify()
```

```python
a_0.factor()
a_0.factor()
```

$\displaystyle 2.0$

```python
%%render
a_0
a_n
b_n
```

\[
\begin{aligned}
a_{0} &= \displaystyle 2.0 \;
\\[10pt]
a_{n} &= \displaystyle \frac{4.0 \left(\left(-1\right)^{n} - 1\right)}{\pi^{2} n^{2}} \;
\\[10pt]
b_{n} &= \displaystyle 0 \;
\end{aligned}
\]

# Calculation of whole series upto desired terms

```python
a_new = 0
b_new = 0
a1 = cos(n*free_var) * a_n
b1 = sin(n*free_var) * b_n
for i in range(number_of_terms,0, -1):
    a_new += a1.subs(n, i)
    b_new += b1.subs(n, i)
ser = (a_0/2) + a_new + b_new
```

# The whole series

```python
ser
```

$\displaystyle - \frac{8.0 \cos{\left(x \right)}}{\pi^{2}} - \frac{0.888888888888889 \cos{\left(3 x \right)}}{\pi^{2}} - \frac{0.32 \cos{\left(5 x \right)}}{\pi^{2}} - \frac{0.163265306122449 \cos{\left(7 x \right)}}{\pi^{2}} - \frac{0.0987654320987654 \cos{\left(9 x \right)}}{\pi^{2}} + 1.0$