# Optimization

> Marcos Duarte
> Laboratory of Biomechanics and Motor Control ([http://demotu.org/](http://demotu.org/))
> Federal University of ABC, Brazil

## Definition

> `If there occur some changes in nature, the amount of action necessary for this change must be as small as possible.` Maupertuis (sec XVIII)

Optimization is the process of finding the best value from possible alternatives with regards to a certain criteria [Wikipedia](http://en.wikipedia.org/wiki/Mathematical_optimization).
Usually, such best value is the value that maximizes or minimizes the criteria. In this context, to solve an optimization problem is to find the maximum or minimum of a function. To solve an optimization problem, we first have to model the problem and define the objective, the variables, and the constraints of the problem. In optimization, these terms are usually defined as:

1. Cost or objective function: a function describing what we want to optimize.
2. Design variable(s): variables that will be manipulated to optimize the cost function.
3. Constraint functions: constrain the solution to a set of possible values.

Read more about that in [Introduction to Optimization](http://neos-guide.org/content/optimization-introduction) from the  [NEOS Guide](http://neos-guide.org/).

## Maxima and minima

In mathematics, the maximum and minimum of a function are the largest and smallest values that the function takes at a point either within a neighborhood (local) or on the function entire domain  (global) ([Wikipedia](http://en.wikipedia.org/wiki/Maxima_and_minima)). For a function of one variable, if the maximum or minimum of a function is not at the limits of the domain and if at least the first and second derivatives of the function exist, a maximum and minimum can be found as the point where the first derivative of the function is zero. If the second derivative on that point is positive, then it's a minimum, if it is negative, it's a maximum.

Figure. Maxima and minima of a function of one variable.

### Maximum range of projectile motion

What is the optimal angle to throw a projectile to achieve the greatest distance?

The equations of motion for a projectile launched with an angle $\theta$ with the horizontal (consider constant gravitational acceleration, $g$) and neglecting the air resistance are:

$$ \begin{array}{l l}
x(t) = x_0 + v_0\cos(\theta)\:t \\
y(t) = y_0 + v_0\sin(\theta)\:t - \frac{g\:t^2}{2} \\
\\
v_x(t) = v_0\cos(\theta) \\
v_y(t) = v_0\sin(\theta) - g\:t \\
\\
a_x(t) = 0 \\
a_y(t) = -g
\end{array} $$

**Time of flight**
The time of flight can be calculated from the equation for the vertical velocity and using the properties that at the maximum height, the vertical velocity is zero and the time of rising is equal to the time of falling:

$$ t_{flight} = \frac{2v_0\sin(\theta)}{g} $$

**Range**
The range, $R$, is the horizontal distance reached by the projectile with respect to the point of release:

$$ \begin{array}{l l}
R = v_0\cos(\theta)\:t_{flight} \\
\\
R = \frac{v_0^2\sin(2\theta)}{g}
\end{array} $$

The maximum range is obtained by finding the angle that maximizes the equation above. In the context of optimization, the equation for $R$ is the cost function, $\theta$ is the design variable, and the  constraint is that feasible angles are in the interval $[0,\: 90^o]$.
The solution of this problem is trivial, we know by inspection that the maximum range is given when $\sin(2\theta)$ is maximum, and for that, $\theta=45^o$. But let's show the solution with the  approach using derivatives:

$$ \frac{\mathrm{d} R}{\mathrm{d} \theta} = \frac{2v_0^2\cos(2\theta)}{g} $$

Find the value of $\theta$ where this derivative is zero:

$$ \frac{2v_0^2\cos(2\theta)}{g} = 0 \quad \Rightarrow \quad \theta=45^o $$

We can check this value corresponds to a maximum if the second derivative of $R$ at this value is negative:

$$ \frac{\mathrm{d}^2 R}{\mathrm{d} \theta^2} = -\frac{4v_0^2\sin(2\theta)}{g} $$

Which indeed is negative at $\theta=45^o$.

We can use Sympy to solve the steps of this problem. Let's import the necessary Python libraries and customize the environment for that:

```python
import numpy as np
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['lines.linewidth'] = 3
matplotlib.rcParams['font.size'] = 13
matplotlib.rcParams['lines.markersize'] = 5
matplotlib.rc('axes', grid=False, labelsize=14, titlesize=16, ymargin=0.05)
matplotlib.rc('legend', numpoints=1, fontsize=11)

import sympy as sym
from sympy.plotting import plot
#from sympy.interactive import printing
#printing.init_printing()
from IPython.display import display
from IPython.core.display import Math
```

Let's create symbolic variables, define the equation, differentiate it, and find the maximum value:

```python
v0, g, theta = sym.symbols('v0 g theta')
R = v0**2*sym.sin(2*theta)/g
Rdiff = sym.diff(R, theta)
display(Math(sym.latex('R =') + sym.latex(R)))
display(Math(sym.latex(r'\frac{\mathrm{d}R}{\mathrm{d}\theta} =') + sym.latex(Rdiff)))
ang = sym.solve(Rdiff, theta)
display(Math(sym.latex('Sols:') + sym.latex(ang)))
display(Math(sym.latex(r'R(\theta=\pi/4)=') +
             sym.latex(sym.N(R.subs({v0: 10, g: 9.8, theta: sym.pi/4}), 4))))
plot(R.subs({v0: 10, g: 9.8}), (theta, 0, sym.pi/2),
     xlabel=r'$\theta(rad)$', ylabel=r'$R$');
```

### Maximum volume of a box

We want to make a box from a square cardboard with side $a$ such that its volume should be maximum. What is the optimal distance where the square cardbox should be cut and folded to make a box with maximum volume?

Figure. A box to be made from a cardboard such that its volume should be maximum. Where we should cut?

If the distance where to cut and fold the cardbox is $b$, see figure above, the volume of the box will be:

$$ \begin{array}{l l}
V(b) = b(a-2b)(a-2b) \\
\\
V(b) = a^2b - 4ab^2 + 4b^3
\end{array} $$

The expression for $V$ is the cost function, $b$ is the design variable, and the constraint is that feasible values of $b$ are in the interval $[0, a/2]$.
The first derivative of $V$ is:

$$ \frac{\mathrm{d}V}{\mathrm{d}b} = a^2 - 8ab + 12b^2  $$

We have to find the values for $b$ where this derivative is zero. Let's use Sympy for that:

```python
a, b = sym.symbols('a b')
V = b*(a - 2*b)*(a - 2*b)
Vdiff = sym.expand(sym.diff(V, b))
roots = sym.solve(Vdiff, b)
display(Math(sym.latex('V =') + sym.latex(V)))
display(Math(sym.latex('\\frac{\mathrm{d}V}{\mathrm{d}b} =') + sym.latex(Vdiff)))
display(Math(sym.latex('Roots:') + sym.latex(roots)))
```

$$V =b \left(a - 2 b\right)^{2}$$

$$\frac{\mathrm{d}V}{\mathrm{d}b} =a^{2} - 8 a b + 12 b^{2}$$

$$Roots:\left [ \frac{a}{6}, \quad \frac{a}{2}\right ]$$

Discarding the solution $b=a/2$ (where $V=0$), $b=a/6$ results in the maximum volume. We can check that by ploting the volume of the cardbox for $a=1$ and $b: [0,\:0.5]$:

```python
plot(V.subs({a: 1}), (b, 0, .5), xlabel='b', ylabel='V')
display(Math(sym.latex('V_{a=1}^{max} =') +
             sym.latex(sym.N(V.subs({b: a/6}).subs({a: 1}), 2))))
```

The examples above are trivial problems of optimization; there was only one design variable in each case and the cost functions, although nonlinear, were simple. When there is more than one design variable (the cost function depends on more than one variable), the general idea of finding minimum and maximum values where the derivatives are zero still holds, but the problem gets more complicated. For an introduction on that, particularly applied to biomechanics, see chapter 4 (start on page 609) of Nigg and Herzog (2006).

Let's see now a classical problem in biomechanics where optimization is useful and there is more than one design variable.

## The distribution problem in biomechanics

Using the inverse dynamics approach in biomechanics, we can determine the net force and torque acting on a joint if we know the external forces on the segments and the kinematics and inertial properties of the segments. But with this approach we are unable to determine the individual muscles forces that  created such torque, as expressed in the following equation:

$$ M_{total} = M_1 + M_2 + \dots + M_n = r_1F_1 + r_2F_2 + \dots + r_nF_n $$

where $r_i$ is the moment arm of the force $F_i$ that generates a torque $M_i$, a parcel of the (known) total torque $M_{total}$.

Even if we know the moment arm of each muscle (e.g., from cadaveric data or from image analysis), the equation above has $n$ unknowns. Because there is more than one muscle that potentially created such torque, there are more unknowns than equations, and the problem is undetermined. So, the problem is how to find how the torque is distributed among the muscles of that joint.

One solution is to consider that we (biological systems) optimize our effort in order to minimize energy expenditure, stresses on our tissues, fatigue, etc. The principle of least action, stated in the opening of this text, is an allusion that optimization might be ubiquitous in nature. With this rationale, let's solve the distribution problem in biomechanics using optimization and find the minimum force of each muscle necessary to complete a given task.

The following cost functions have been proposed to solve the distribution problem in biomechanics:

$$ \begin{array}{l l}
\sum_{i=1}^N F_i \quad &\text{(e.g., Seireg and Arkivar, 1973)}
\\
\sum_{i=1}^N F_i^2 \quad &
\\
\sum_{i=1}^N \left(\frac{F_i}{pcsa_i}\right)^2 \quad &\text{(e.g., Crowninshield and Brand, 1981)}
\\
\sum_{i=1}^N \left(\frac{F_i}{M_{max,i}}\right)^3 \quad &\text{(e.g., Herzog, 1987)}
\end{array} $$

Where $pcsa_i$ is the physiological cross-sectional area of muscle $i$ and $M_{max,i}$ is the maximum torque muscle $i$ can produce.
Each muscle force $F_i$ is a design variable and the following constraints must be satisfied:

$$ \begin{array}{l l}
0 \leq F_i \leq F_{max}
\\
\sum_{i=1}^N r_i \times F_i = M
\end{array} $$

Let's apply this concept to solve a distribution problem in biomechanics.

### Muscle force estimation

Consider the following main flexors of the elbow joint (see figure below): biceps long head, biceps short head, and brachialis. Suppose that the elbow net joint torque determined using inverse dynamics is 20 Nm (flexor). How much each of these muscles contributed to the net torque?

Figure. A view in OpenSim of the arm26 model showing three elbow flexors (Biceps long and short heads and Brachialis).

For the optimization, we will need experimental data for the moment arm, maximum moment, and pcsa of each muscle. Let's import these data from the OpenSim arm26 model:

```python
# time elbow_flexion BIClong BICshort BRA
r_ef = np.loadtxt('./../data/r_elbowflexors.mot', skiprows=7)
f_ef = np.loadtxt('./../data/f_elbowflexors.mot', skiprows=7)
```

The maximum isometric force of these muscles are defined in the arm26 model as: Biceps long head: 624.3 N, Biceps short head: 435.56 N, and Brachialis: 987.26 N. Let's compute the mamimum torques that each muscle could produce considering a static situation at the different elbow flexion angles:

```python
m_ef = r_ef*1
m_ef[:, 2:] = r_ef[:, 2:]*f_ef[:, 2:]
```

And let's visualize these data:

```python
labels = ['Biceps long head', 'Biceps short head', 'Brachialis']
fig, ax = plt.subplots(nrows=1, ncols=3, sharex=True, figsize=(10, 4))
ax[0].plot(r_ef[:, 1], r_ef[:, 2:])
#ax[0].set_xlabel('Elbow angle $(\,^o)$')
ax[0].set_title('Moment arm (m)')
ax[1].plot(f_ef[:, 1], f_ef[:, 2:])
ax[1].set_xlabel('Elbow angle $(\,^o)$', fontsize=16)
ax[1].set_title('Maximum force (N)')
ax[2].plot(m_ef[:, 1], m_ef[:, 2:])
#ax[2].set_xlabel('Elbow angle $(\,^o)$')
ax[2].set_title('Maximum torque (Nm)')
ax[2].legend(labels, loc='best', framealpha=.5)
ax[2].set_xlim(np.min(r_ef[:, 1]), np.max(r_ef[:, 1]))
plt.tight_layout()
plt.show()
```

These data don't have the pcsa value of each muscle. We will estimate the pcsa considering that the amount of maximum muscle force generated per area is constant and equal to $50N/cm^2$. Consequently, the pcsa (in $cm^2$) for each muscle is:

```python
a_ef = np.array([624.3, 435.56, 987.26])/50  # 50 N/cm2
```

### Static versus dynamic optimization

We can solve the distribution problem separately for each angle (instant) of the elbow; we will refer to that as static optimization. However, there is no guarantee that when we analyze all these solutions across the range of angles, they will be the best solution overall. One reason is that static optimization ignores the time history of the muscle force. Dynamic optimization refers to the optimization over a period of time. For such, we will need to input a cost function spanning the entire period of time at once. Dynamic optimization usually has a higher computational cost than static optimization.

For now, we will solve the present problem using static optimization.

### Solution of the optimization problem

For the present case, we are dealing with a problem of minimization, multidimensional (function of several variables), nonlinear, constrained, and we can't assume that the cost function is convex. Numerical optimization is hardly a simple task. There are many different algorithms and public and commercial software for performing optimization. For instance, look at [NEOS Server](http://www.neos-server.org/neos/), a free internet-based service for solving numerical optimization problems.
We will solve the present problem using the [scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html#module-scipy.optimize) package which provides several optimization algorithms. We will use the function `minimize`:

```python
scipy.optimize.minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)
"""Minimization of scalar function of one or more variables."""
```

Now, let's write Python functions for each cost function:

```python
from scipy.optimize import minimize
```

```python
def cf_f1(x):
    """Cost function: sum of forces."""
    return x[0] + x[1] + x[2]

def cf_f2(x):
    """Cost function: sum of forces squared."""
    return x[0]**2 + x[1]**2 + x[2]**2

def cf_fpcsa2(x, a):
    """Cost function: sum of squared muscle stresses."""
    return (x[0]/a[0])**2 + (x[1]/a[1])**2 + (x[2]/a[2])**2

def cf_fmmax3(x, m):
    """Cost function: sum of cubic forces normalized by moments."""
    return (x[0]/m[0])**3 + (x[1]/m[1])**3 + (x[2]/m[2])**3
```

Let's also define the Jacobian for each cost function (which is an optional parameter for the optimization):

```python
def cf_f1d(x):
    """Derivative of cost function: sum of forces."""
    dfdx0 = 1
    dfdx1 = 1
    dfdx2 = 1
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_f2d(x):
    """Derivative of cost function: sum of forces squared."""
    dfdx0 = 2*x[0]
    dfdx1 = 2*x[1]
    dfdx2 = 2*x[2]
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_fpcsa2d(x, a):
    """Derivative of cost function: sum of squared muscle stresses."""
    dfdx0 = 2*x[0]/a[0]**2
    dfdx1 = 2*x[1]/a[1]**2
    dfdx2 = 2*x[2]/a[2]**2
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_fmmax3d(x, m):
    """Derivative of cost function: sum of cubic forces normalized by moments."""
    dfdx0 = 3*x[0]**2/m[0]**3
    dfdx1 = 3*x[1]**2/m[1]**3
    dfdx2 = 3*x[2]**2/m[2]**3
    return np.array([dfdx0, dfdx1, dfdx2])
```

Let's define initial values:

```python
M = 20  # desired torque at the elbow
iang = 69  # which will give the closest value to 90 degrees
r  = r_ef[iang, 2:]
f0 = f_ef[iang, 2:]
a  = a_ef
m  = m_ef[iang, 2:]
x0 = f_ef[iang, 2:]/10  # far from the correct answer for the sum of torques
print('M =', M)
print('x0 =', x0)
print('r * x0 =', np.sum(r*x0))
```

    M = 20
    x0 = [ 57.51311369  36.29974032  89.6470056 ]
    r * x0 = 6.62200444607

Inequality constraints (such as boundaries in our problem) can be entered with the parameter `bounds` to the `minimize` function:

```python
bnds = ((0, f0[0]), (0, f0[1]), (0, f0[2]))
```

Equality constraints (such as the sum of torques should equals the desired torque in our problem), as well as inequality constraints, can be entered with the parameter `constraints` to the `minimize` function (and we can also opt to enter the Jacobian of these constraints):

```python
# use this in combination with the parameter bounds:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)})
```

```python
# to enter everything as constraints:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[0]-x[0],
         'jac' : lambda x, r, f0, M: np.array([-1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[1]-x[1],
         'jac' : lambda x, r, f0, M: np.array([0, -1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[2]-x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, -1]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[0],
         'jac' : lambda x, r, f0, M: np.array([1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[1],
         'jac' : lambda x, r, f0, M: np.array([0, 1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, 1]), 'args': (r, f0, M)})
```

Although more verbose, if all the Jacobians of the constraints are also informed, this alternative seems better than informing bounds for the optimization process (less error in the final result and less iterations).

Given the characteristics of the problem, if we use the function `minimize` we are limited to the SLSQP (Sequential Least SQuares Programming) solver.

Finally, let's run the optimization for the four different cost functions and find the optimal muscle forces:

```python
f1r = minimize(fun=cf_f1, x0=x0, args=(), jac=cf_f1d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

    Optimization terminated successfully.    (Exit mode 0)
                Current function value: 409.59266009953336
                Iterations: 7
                Function evaluations: 7
                Gradient evaluations: 7

```python
f2r = minimize(fun=cf_f2, x0=x0, args=(), jac=cf_f2d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

    Optimization terminated successfully.    (Exit mode 0)
                Current function value: 75657.38479127169
                Iterations: 5
                Function evaluations: 6
                Gradient evaluations: 5

```python
fpcsa2r = minimize(fun=cf_fpcsa2, x0=x0, args=(a,), jac=cf_fpcsa2d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

    Optimization terminated successfully.    (Exit mode 0)
                Current function value: 529.9639777695737
                Iterations: 11
                Function evaluations: 11
                Gradient evaluations: 11

```python
fmmax3r = minimize(fun=cf_fmmax3, x0=x0, args=(m,), jac=cf_fmmax3d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

    Optimization terminated successfully.    (Exit mode 0)
                Current function value: 1075.1388931664733
                Iterations: 12
                Function evaluations: 12
                Gradient evaluations: 12

Let's compare the results for the different cost functions:

```python
dat = np.vstack((np.around(r*100,1), np.around(a,1), np.around(f0,0), np.around(m,1)))
opt = np.around(np.vstack((f1r.x, f2r.x, fpcsa2r.x, fmmax3r.x)),1)
er = ['-', '-', '-', '-',
      np.sum(r*f1r.x)-M, np.sum(r*f2r.x)-M, np.sum(r*fpcsa2r.x)-M, np.sum(r*fmmax3r.x)-M]
data = np.vstack((np.vstack((dat, opt)).T, er)).T

from pandas import DataFrame
rows = ['$\text{Moment arm}\;[cm]$', '$pcsa\;[cm^2]$', '$F_{max}\;[N]$', '$M_{max}\;[Nm]$',
        '$\sum F_i$', '$\sum F_i^2$', '$\sum(F_i/pcsa_i)^2$', '$\sum(F_i/M_{max,i})^3$']
cols = ['Biceps long head', 'Biceps short head', 'Brachialis', 'Error in M']
df = DataFrame(data, index=rows, columns=cols)
print('\nComparison of different cost functions for solving the distribution problem')
df
```

    Comparison of different cost functions for solving the distribution problem

      Biceps long head
      Biceps short head
      Brachialis
      Error in M

      $\text{Moment arm}\;[cm]$
      4.9
      4.9
      2.3
      -

      $pcsa\;[cm^2]$
      12.5
      8.7
      19.7
      -

      $F_{max}\;[N]$
      575.0
      363.0
      896.0
      -

      $M_{max}\;[Nm]$
      28.1
      17.7
      20.4
      -

      $\sum F_i$
      215.4
      194.2
      0.0
      0.0

      $\sum F_i^2$
      184.7
      184.7
      86.1
      0.0

      $\sum(F_i/pcsa_i)^2$
      201.7
      98.2
      235.2
      0.0

      $\sum(F_i/M_{max,i})^3$
      241.1
      120.9
      102.0
      -3.5527136788e-15

## Comments

The results show that the estimations for the muscle forces depend on the cost function used in the optimization. Which one is correct? This is a difficult question and it's dependent on the goal of the actual task being modeled. Glitsch and Baumann (1997) investigated the effect of different cost functions on the optimization of walking and running and the predicted muscles forces were compared with the electromyographic activity of the corresponding muscles of the lower limb. They found that, among the analyzed cost functions, the minimization of the sum of squared muscle stresses resulted in the best similarity with the actual electromyographic activity.

In general, one should always test different algorithms and different initial values before settling for the solution found. Downey (2011), Kitchin (2013), and Kiusalaas (2013) present more examples on numerical optimization. The [NEOS Guide](http://neos-guide.org/) is a valuable source of information on this topic and [OpenOpt](http://openopt.org/) is a good alternative software for numerical optimization in Python.

## Exercises

1. Regarding the distribution problem for the elbow muscles presented in this text:
    a. Test different initial values for the optimization.
    b. Test other values for the elbow angle where the results are likely to change.

2. In an experiment to estimate forces of the elbow flexors, through inverse dynamics it was found an elbow flexor moment of 10 Nm. Consider the following data for maximum force (F0), moment arm (r), and pcsa (A) of the brachialis, brachioradialis, and biceps brachii muscles: F0 (N): 1000, 250, 700; r (cm): 2, 5, 4; A (cm$^2$): 33, 8, 23, respectively (data from Robertson et al. (2013)).
    a. Use static optimization to estimate the muscle forces.
    b. Test the robustness of the results using different initial values for the muscle forces.
    c. Compare the results for different cost functions.

## References

- Crowninshield RD, Brand RA (1981) [A physiologically based criterion of muscle force prediction in locomotion](http://www.ncbi.nlm.nih.gov/pubmed/7334039). Journal of Biomechanics, 14, 793–801.
- Downey AB (2011) [Physical Modeling in MATLAB](http://greenteapress.com/matlab/). Green Tea Press.
- Herzog W (1987) [Individual muscle force estimations using a non-linear optimal design](http://www.ncbi.nlm.nih.gov/pubmed/3682873). J Neurosci Methods, 21, 167-179.
- Glitsch U, Baumann W (1997) [The three-dimensional determination of internal loads in the lower extremity](http://www.ncbi.nlm.nih.gov/pubmed/9456380). Journal of Biomechanics, 30, 1123–1131.
- Kitchin J (2013) [pycse - Python Computations in Science and Engineering](http://kitchingroup.cheme.cmu.edu/pycse/).
- Kiusalaas (2013) [Numerical methods in engineering with Python 3](http://books.google.com.br/books?id=aJkXoxxoCoUC). 3rd edition. Cambridge University Press.
- Nigg BM and Herzog W (2006) [Biomechanics of the Musculo-skeletal System](https://books.google.com.br/books?id=hOIeAQAAIAAJ&dq=editions:ISBN0470017678). 3rd Edition. Wiley.
- Robertson G, Caldwell G, Hamill J, Kamen G (2013) [Research Methods in Biomechanics](http://books.google.com.br/books?id=gRn8AAAAQBAJ). 2nd Edition. Human Kinetics.
- Seireg A, Arvikar RJ (1973) [A mathematical model for evaluation of forces in lower extremeties of the musculo-skeletal system](http://www.ncbi.nlm.nih.gov/pubmed/4706941). Journal of Biomechanics, 6,  313–322, IN19–IN20, 323–326.


# Discrete Signals

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Elementary Operations

Operations like superposition, shifting and flipping can be used to construct signals with a more complex structure than by the [standard signals](standard_signals.ipynb) alone. In the following, a set of elementary operations is introduced  that are frequently used in discrete signal processing for this purpose. Note that the equivalent operation to the [temporal scaling of a continuous signal](../continuous_signals/operations.ipynb#Temporal-Scaling) is not defined for a discrete signal.

### Superposition

The weighted superposition $x[k]$ of two signals $x_1[k]$ and $x_2[k]$ is given as

\begin{equation}
x[k] = A \cdot x_1[k] + B \cdot x_2[k]
\end{equation}

with the complex weights $A, B \in \mathbb{C}$.

**Example**

The following example illustrates the superposition of two harmonic signals $x(t) = A \cdot \cos[\Omega_1 k] + B \cdot \cos[\Omega_2 k]$ with weights $A$, $B$ and normalized frequencies $\Omega_1$ and $\Omega_2$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

k = np.arange(0, 51)
x = np.cos(2 * np.pi / 10 * k) + 2 * np.cos(2 * np.pi / 15 * k)

plt.figure(figsize=(6, 3))
plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel('$x[k]$')
plt.gca().margins(y=0.1)
```

### Shift

The shift of a signal $s[k]$ by the index $\kappa$ is defined as

\begin{equation}
x[k] = s[k-\kappa]
\end{equation}

with $\kappa \in \mathbb{Z}$. The signal $s[k]$ is

* shifted to the right for $\kappa > 0$
* shifted to the left for $\kappa < 0$

The shift of a signal is a frequently applied operation in discrete signal processing. For instance for the description of systems by linear difference equations with constant coefficients. For a discrete signal which has been derived by [temporal sampling from a continuous signal](../sampling/ideal.ipynb), the shift can be interpreted as [temporal shift](../continuous_signals/operations.ipynb#Temporal-Shift) by the time $\tau = \kappa \cdot T$ where $T$ denotes the sampling interval.

**Example**

In order to illustrate the shifting of signals, the construction of a [sawtooth signal](https://en.wikipedia.org/wiki/Sawtooth_wave) by a superposition of shifted ramp signals $k \cdot \text{rect}_N[k]$ is shown. The sawtooth signal is given as periodic continuation of the ramp signal

\begin{equation}
x[k] = \sum_{\nu = -\infty}^{\infty} (k - \nu \cdot N) \cdot \text{rect}_N[k - \nu \cdot N]
\end{equation}

The signal can be computed efficiently using the [modulo operation](https://en.wikipedia.org/wiki/Modulo_operation)

\begin{equation}
x[k] = k \bmod N
\end{equation}

which is illustrated in the following

```python
def sawtooth(k, N):
    return np.mod(k, N)

k = np.arange(-10, 40)
x = sawtooth(k, 10)

plt.figure(figsize=(6, 3))
plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel('$x[k]$')
plt.gca().margins(y=0.1)
```

### Flipping

The flipping of a signal $s[k]$ is defined as

\begin{equation}
x[k] = s[\kappa - k]
\end{equation}

with $\kappa \in \mathbb{Z}$. The flipping operation can also be represented as a reversal of the index $k$ of the signal $s[k]$ followed by a shift of $\kappa$ of the reversed signal, as $s[\kappa - k] = s[- (k - \kappa)]$. The operation can interpreted geometrically as a mirroring of the signal $s[k]$ at the vertical axis $k = \frac{\kappa}{2}$.

For $\kappa = 0$ this results in a reversal of the signal. The reversal can be interpreted as time-reversal for a discrete signal which has been derived by temporal sampling from a continuous signal.

**Example**

The following example illustrates the temporal flipping of the sawtooth signal $x[k]$ introduced above for $\kappa = 3$.

```python
x = sawtooth(3 - k, 10)

plt.figure(figsize=(6, 3))
plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel('$x[k]$')
plt.gca().margins(y=0.1)
```

**Excercise**

* Realize the temporal flipping by splitting it into two operations:
    1. time-reversal and
    2. temporal shift.

### Copyright

  To the extent possible under law,
  the person who associated CC0
  with this work has waived all copyright and related or neighboring
  rights to this work.


```python
import numpy as np
from scipy.linalg import lu_factor, lu_solve
import matplotlib.pyplot as plt
import matplotlib as mpl
```

# Paramentros de entrada

```python
f      =  -2
dudx_0 =  -2
u3     =  0.0
```

## No 1

\begin{equation}
k_{11} = \int_0^1 \frac{dN_1}{dx} \frac{dN_1}{dx} dx =  \int_0^{1/2} \frac{dN_1}{dx} \frac{dN_1}{dx} dx =  \int_0^{1/2} (-2) (-2) dx = (-2) (=2) \left(\frac{1}{2} - 0 \right) = 2
\end{equation}

\begin{equation}
k_{12} = \int_0^1 \frac{dN_1}{dx} \frac{dN_2}{dx} dx =  \int_0^{1/2} \frac{dN_1}{dx} \frac{dN_2}{dx} dx =  \int_0^{1/2} (-2) (2) dx = (-2) (2) \left(\frac{1}{2} - 0 \right) = -2
\end{equation}

\begin{equation}
k_{13} = \int_0^1 \frac{dN_1}{dx} \frac{dN_3}{dx} dx = 0
\end{equation}

\begin{equation}
f_{1} = \int_0^1 f N_1dx + \frac{du}{dx}(1) N_1(1) - \frac{du}{dx}(0) N_1(0) =  \frac{f}{4} - \frac{du}{dx}(0)
\end{equation}

```python
k11 =  2.0
k12 = -2.0
k13 =  0.0
f1 = f/4 - dudx_0
```

## No 2

\begin{equation}
k_{21} = \int_0^1 \frac{dN_2}{dx} \frac{dN_1}{dx} dx = 2
\end{equation}

\begin{equation}
k_{22} = \int_0^1 \frac{dN_2}{dx} \frac{dN_2}{dx} dx =  \int_0^{1/2} \frac{dN_2}{dx} \frac{dN_2}{dx} dx +  \int_{1/2}^{1} \frac{dN_2}{dx} \frac{dN_2}{dx} dx =  \int_0^{1/2} (2) (2) dx + \int_{1/2}^{1} (-2) (-2) dx = 2 + 2 = 4
\end{equation}

\begin{equation}
k_{23} = \int_0^1 \frac{dN_2}{dx} \frac{dN_3}{dx} dx = \int_{1/2}^1 \frac{dN_2}{dx} \frac{dN_3}{dx} dx = \int_{1/2}^1 (2) (-2) dx = -2
\end{equation}

\begin{equation}
f_{2} = \int_0^1 f N_2dx + \frac{du}{dx}(1) N_2(1) - \frac{du}{dx}(0) N_2(0) =  \frac{f}{2}
\end{equation}

```python
k21 = k12
k22 = 4.0
k23 = -2.0
f2 = f/2
```

# Sistema de equações

\begin{equation}
\begin{bmatrix}
 k_{11} & k_{12}\\
 k_{21} & k_{22}
\end{bmatrix}
*
\begin{bmatrix}
u_1\\
u_2
\end{bmatrix}
=
\begin{bmatrix}
f_1 - k_{13} * u_3\\
f_2 - k_{23} * u_3
\end{bmatrix}
\end{equation}

## Matriz de Coeficiente Real

```python
K = np.array([
             [k11, k12],
             [k21, k22],
            ])

K
```

    array([[ 2., -2.],
           [-2.,  4.]])

## Vetor de forças

```python
F = np.array([
              f1 - k13 * u3,
              f2 - k23 * u3
            ])

F
```

    array([ 1.5, -1. ])

```python
lu, piv = lu_factor(K)
u1, u2 = lu_solve((lu, piv), F)
```

```python
u_numerico_coef = [ u1, u2, u3]
u_numerico_coef
```

    [1.0, 0.25, 0.0]

```python
x_malha = [0, 0.5, 1.0]
x_malha
```

    [0, 0.5, 1.0]

# Solução Exata

Solução
$$
u(x) = x ^ 2 - 2 x + 1
$$

Derivada da solução

$$
\frac{du}{dx} = 2 x - 2
$$

```python
def u_analitico(x):
  return x**2 - 2.0 * x + 1.0

def dudx_analitico(x):
  return 2.0 * x - 2.0
```

# Solução númerica

Aproximação

$$u(x) = N_1(x) u_1 + N_2(x) u_2 + N_3(x) u_3$$

**Funções de interpolação:**

* $N_1$:

$$
N_1(x) = \begin{cases}
                      &1 * -2 x     &\text{ se } & 0 < x < 1/2 \\
                      &0            &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

* $N_2$:

$$
N_2(x) = \begin{cases}
                      &2 x     &\text{ se } & 0 < x < 1/2 \\
                      &2 - 2 x &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

* $N_3$:

$$
N_3(x) = \begin{cases}
                      &0             &\text{ se } & 0 < x < 1/2 \\
                      &2 x - 1     &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

```python
def u_numerico(x, u_numerico_coef, x_malha):

  u1, u2, u3 = u_numerico_coef

  x1, x2, x3 = x_malha

  # 0 < x < 1/2
  if x1 <= x < x2:
    N1 = 1.0 - 2.0 * x
    N2 = 2.0 * x
    N3 = 0.0

    # 1/2 < x < 1
  elif x2 <= x <= x3:
    N1 = 0.0
    N2 = 2.0 * ( 1.0 - x)
    N3 = 2*x - 1.0

  return N1*u1 + N2 * u2 + N3 *u3

```

**Solução númerica:**

$$
\frac{du}{dx}(x) = \frac{dN_1}{dx}(x) u_1 + \frac{dN_2}{dx}(x) u_2 + \frac{dN_3}{dx}(x) u_3
$$

**Funções de interpolação:**

* $\frac{dN_1}{dx}$:

$$
\frac{dN_1}{dx} = \begin{cases}
                      &-2     &\text{ se } & 0 < x < 1/2 \\
                      &0      &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

* $N_2$:

$$
\frac{dN_2}{dx} = \begin{cases}
                      & 2        &\text{ se } & 0 < x < 1/2 \\
                      &-2        &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

* $N_3$:

$$
\frac{dN_3}{dx} = \begin{cases}
                      &0             &\text{ se } & 0 < x < 1/2 \\
                      &2             &\text{ se } & 1/2 < x < 1
          \end{cases}
$$

```python
def dudx_numerico(x, u_numerico_coef, x_malha):

  u1, u2, u3 = u_numerico_coef

  x1, x2, x3 = x_malha

  # 0 < x < 1/2
  if x1 <= x < x2:
    dN1dx = -2.0
    dN2dx =  2.0
    dN3dx =  0.0

    # 1/2 < x < 1
  elif x2 <= x <= x3:
    dN1dx =  0.0
    dN2dx = -2.0
    dN3dx =  2.0

  return dN1dx * u1 + dN2dx * u2 + dN3dx *u3
```

# Plotando os resultados

```python
x = np.linspace(0, 1, 50)
```

```python
u_exato = [ u_analitico(xi) for xi in x  ]
dudx_exato = [ dudx_analitico(xi) for xi in x  ]
```

```python
u_num = [ u_numerico(xi, u_numerico_coef, x_malha) for xi in x  ]
dudx_num = [ dudx_numerico(xi, u_numerico_coef, x_malha) for xi in x  ]
```

```python
mpl.rcParams['figure.figsize'] = (20, 10)
#
fig, (ax1, ax2) = plt.subplots(ncols = 2)
#
ax1.set_title('Solução', fontsize = 18)
ax1.plot(x, u_exato, label = 'Analito')
ax1.plot(x, u_num  , label = 'Numerico')
ax1.set_ylabel('u(x)', fontsize = 14)
ax1.set_xlabel('x', fontsize = 14)
#
ax2.set_title('Derivada', fontsize = 18)
ax2.plot(x, dudx_exato)
ax2.plot(x, dudx_num)
ax2.set_ylabel(r'$\frac{du}{dx}(x)$', fontsize = 14)
ax2.set_xlabel('x', fontsize = 14)
#
ax1.grid(ls = '--')
ax2.grid(ls = '--')
#
ax1.legend(fontsize=14)
plt.show()
```

```python

```


# Free Vibrations of an Uniform Beam

A uniform beam, its length $L$, its stiffness $EJ=\text{const}$ and its
unit mass $\bar m=\text{const}$ is clamped at $x=0$ and is simply
supported at $x=L$.

The beam is in a condition of static equilibrium (see figure) under a
couple $W=w\frac{EJ}{L}$ applied at $x=L$ when, at time $t=0$, the
external load is suddenly released.

Determine the modal responses for the first 3 modes and plot the
bending moment $M_b(0, t)$ in the interval $0 \le \omega_0 t \le 5$,
where $\omega_0^2=\frac{EJ}{\bar mL^4}$

## Solution

We are going to

- recall the general form of the eigenfunctions of an uniform beam, solutions of the indefinite eq. of equilibrium,
- determine the eigenfunctions of the actual problem applying the boundary conditions,
    - determine the wavenumbers,
    - determine the frequencies of free vibration,
- determine the static equilibrium position of the beam (i.e., the initial conditions) using the moment-curvature relationship and
    - determine the coefficients of the eigenfunctions expansion of said initial condition,
- determine the modal responses (free vibrations following given initial conditions),
- write the bending moment as a sum of second derivatives of the modal responses,
    - specialize for $t=0$ and discuss the problems arising from our particular boundary conditions,
    - specialize for $x=0$ and plot for $0\le\omega_0t\le10$ and discuss the role of the high frequency components with respect to the effects of damping.

### The eigenfunctions

The eigenfunctions of a uniform beam have the form
$$\phi_n = A_n\sin\beta_nx + B_n\cos\beta_nx + C_n\sinh\beta_nx + D_n\cosh\beta_nx,$$
where the _wave number_ $\beta$ is associated to the relevant frequency of vibration by the relationship
$$\beta_n^4=\frac{\bar{m}\omega_n^2}{EJ}
\qquad\mathrm{or,\ multiplying\ by\ L^4,}\qquad
(\beta_nL)^4=\omega_n^2\frac{\bar{m}L^4}{EJ} = \frac{\omega_n^2}{\omega_0^2}.$$

#### The boundary conditions, determination of wavenumbers and frequencies

The boundary conditions for our problem are
\begin{align*}
    \phi_n(0) &= 0 & \phi'_n(0)  &=0\\
    \phi_n(L) &= 0 & -EJ\phi''_n(L) &=0
\end{align*}
From the first two equations we find that $B_n+D_n=0$ and $A_n+C_n=0$, solving for $C_n$ and $D_n$ and substituting in the general integral we have

\begin{align*}
    \phi_n &= A_n(\sin\beta_nx-\sinh\beta_nx) + B_n(\cos\beta_nx-\cosh\beta_nx)\\
    \phi''_n &= -A_n\lambda^2_n(\sin\beta_nx+\sinh\beta_nx) - B_n\lambda_n^2(\cos\beta_nx+\cosh\beta_nx)
\end{align*}

The boundary conditions in $L$ can be written, after some
simplifications, in terms of a homogeneous linear system
$$
\begin{bmatrix}
  sin\beta_nL-\sinh\beta_nL&\cos\beta_nL-\cosh\beta_nL\\
  sin\beta_nL+\sinh\beta_nL&\cos\beta_nL+\cosh\beta_nL
\end{bmatrix}\,
\begin{Bmatrix}
  A_n\\B_N\end
{Bmatrix} =
\begin{Bmatrix}
  0\\0
\end{Bmatrix}
$$

Equating the determinant of the coefficent matrix to zero gives
$$
2 \cosh{\beta_n L} \sin{\beta_n L} - 2 \cos{\beta_n L} \sinh{\beta_n L}=0
\quad\leftrightarrow\quad
\sin{\beta_n L}=\cos{\beta_n L} \tanh{\beta_n L}.$$

Because for moderately large arguments it is $\tanh\beta L\simeq1$
we can write that the determinant is (almost) equal to zero when
$\sin\beta_n L=\cos\beta_n L$, i.e., when $\beta\,L = \pi/4 + n\pi$.

It is useful to plot
$\sin{\beta_n L}-\cos{\beta_n L} \tanh{\beta_n L}$ and
$\sin{\beta_n L}\cos{\beta_n L}$ against $\beta L/\pi$,

```python
BetaL = linspace(0,5,501)
det_exact  = sin(pi*BetaL)-cos(pi*BetaL)*tanh(pi*BetaL)
det_approx = sin(pi*BetaL)-cos(pi*BetaL)
```

```python
plt.plot(BetaL, det_exact,  label='exact')
plt.plot(BetaL, det_approx, label='approx')
plt.xticks([0]+[n+0.25 for n in (1,2,3,4)])
plt.title('The determinant of the homogeneous system as a function of $\\beta L$')
plt.xticks((0.00, 1.00, 1.25, 2.00, 2.25, 3.00, 3.25, 4.00, 4.25, 5.00),
           ('0', '1', '1.25', '2', '2.25', '3', '3.25', '4', '4.25', '5'))
plt.legend(); plt.xlabel(r'$\beta L/\pi$');
```

Not interested in the trivial solution $\beta_nL=0$, it's apparent that the roots are $\beta_nL\simeq(n+1/4)\pi$ but we can compute numerically more precise approximations to the roots of the determinant and eventually the frequencies of vibration, $\omega_n^2 = (\beta_nL)^4\omega_0^2$.

```python
N = 3
roots = array([newton(lambda x: sin(x)-cos(x)*tanh(x), (r+1.25)*pi)
               for r in range(N)])
BetaL = roots
w1 = BetaL**2
w2 = w1**2
```

```python
dL(r'\noindent The adimensional wavenumbers',
   groupalign(roots, 3, variable='\\beta_{%d}L'),
   r'$\text{The adimensional frequencies}$',
   groupalign(w1, 3, variable=r'\frac{\omega_{%d}}{\omega_0}'),
   r'$\text{The adimensional squared frequencies}$',
   groupalign(w2, 3, variable=r'\frac{\omega^2_{%d}}{\omega^2_0}'))
```

\noindent The adimensional wavenumbers \begin{align*}\beta_{1}L&=3.9266,&\beta_{2}L&=7.06858,&\beta_{3}L&=10.2102.\end{align*} $\text{The adimensional frequencies}$ \begin{align*}\frac{\omega_{1}}{\omega_0}&=15.4182,&\frac{\omega_{2}}{\omega_0}&=49.9649,&\frac{\omega_{3}}{\omega_0}&=104.248.\end{align*} $\text{The adimensional squared frequencies}$ \begin{align*}\frac{\omega^2_{1}}{\omega^2_0}&=237.721,&\frac{\omega^2_{2}}{\omega^2_0}&=2496.49,&\frac{\omega^2_{3}}{\omega^2_0}&=10867.6.\end{align*}

#### The eigenfunctions

It is $\phi_n = (\sin\beta_nx - \sinh\beta_nx) A_n + (\cos\beta_nx - \cosh\beta_nx) B_n$, with (from the displacement boundary condition in $L$)

$$ (\sin\beta_nL - \sinh\beta_nL) A_n + (\cos\beta_nL - \cosh\beta_nL) B_n =
\mathcal{C}_{A_n} A_n + \mathcal{C}_{B_n} B_n =0
$$

```python
CA, CB = sin(roots)-sinh(roots), cos(roots)-cosh(roots)
A = 1.0 ; B = -CA*A/CB

xi = linspace(0, 1, 1001)
b_xi = outer(xi, roots)
phi = A*(sin(b_xi)-sinh(b_xi)) + B*(cos(b_xi)-cosh(b_xi))
```

```python
tlbl = '$\\tau=\\omega_0t$'
xlbl = '$\\xi=x/L$'

plt.legend(plt.plot(xi, phi), [r'$\phi_{%d}$'%n for n in range(1,7)], loc=0, ncol=3)
plt.xlabel(xlbl)
plt.title('The first %d eigenfunctions'%N )
rev_y()
```

### Static equilibrium position

We put in evidence an hyperstatic reaction $Z=z\,EJ/L$, the bending moment is $M = (x-L) \, Z/L + x \, W/L$ ; the curvature is
$y'' = -EJ\,M = -(x-L)\,z/L^2 - x\,w/L^2$ ; integrating $y'= -(x-L)^2/2\,z/L^2 -x^2/2\,w/L^2 + A$ and
$y = -(x-L)^3/6\,z/L^2 -x^3/6\,w/L^2 + Ax + B$.

Imposing the boundary conditions $y(0) = zL/6 + B = 0$ and $y'(0)=-z/2 + A=0$ we have $A=z/2$ and $B=-zL/6$

Substituting in $y$ it is $y=-(x-L)^3/6\,z/L^2 -x^3/6\,w/L^2 + x/2\,z - L/6\,z$. evaluating in $x=L$ and imposing a zero displacement we have
$y(L) = 0 =  (1/2 - 1/6) \, zL - 1/6\, wL \rightarrow z = w/2$.

Substituting again in $y$ and simplifying we have
$$y(x) = \frac{wL}4 \frac{x^2L-x^3}{L^3} = L\,\frac{\xi^2-\xi^3}{4}\,w.$$

```python
y = (1-xi)*xi*xi/4
plt.plot(xi, y, label='$y_\mathrm{stat}/wL$')
plt.title('Normalized static displacement')
plt.legend() ; plt.xlabel(xlbl)
rev_y()
```

#### Modal expansion of static displacement

We can represent the (known) static displacements in terms of a weighted sum of eigenfunctions
$$y(x)=\sum_1^\infty \eta_n\phi_n(x),$$
whose coefficients can be easily determined by exploiting the orthogonality relation of the eigenfunctions:
$$\int_0^L \bar m(x)\,\phi_m y \,dx=\delta_{m,n}\,m_n\,\eta_n
\rightarrow \eta_n=\frac{\int_0^L \bar m(x)\,\phi_n y \,dx}{\int_0^L \bar m(x)\,\phi_n^2 \,dx}.$$

```python
modal_mass =  array([trapz(phi_i**2, dx=0.001) for phi_i in phi.T])
modmass_eta = array([trapz(phi_i*y,  dx=0.001) for phi_i in phi.T])
eta = modmass_eta/modal_mass
```

```python
dL(groupalign(eta, 3, variable='\\eta_{%d}', fmt='%+g'))
```

\begin{align*}\eta_{1}&=+0.0240387,&\eta_{2}&=-0.00399941,&\eta_{3}&=+0.00132873.\end{align*}

Plotting the approximation to the static displacements obtained using a few eigenfunctions is not really useful, but it's so easy... We plot the approximant and the difference between the static displacements and the approximant.

```python
y3 = phi@eta
plt.plot(xi, y3); plt.xlabel(xlbl)
plt.title(r'Approximant to $y_\mathrm{stat}$ using %d terms'%N)
rev_y()
plt.show()
plt.plot(xi, y-y3) ; plt.xlabel(xlbl)
plt.title(r'Difference $y_\mathrm{stat} - \sum_1^{%d} \eta_n\phi_n(\beta_n x)$'%N)
rev_y()
```

### The Modal Responses

We know that the initial velocity is zero, the modal displacements are equal to $\eta_i$ and so the modal responses are simply
$$\frac{q_n(t)}{wL} = \eta_n \cos\omega_nt = \eta_n \cos(\beta_n^2L^2\omega_0t)$$

```python
responses = [r'\frac{q_{%d}}{wL} &= %+g \cos(%g\omega_0t)'%(n, eta_n, w_n)
               for n, eta_n, w_n in zip(count(1), eta, w1)]
align_res = r',\\\\'.join(',&'.join(res for res in row if res) for row in grouper(responses, 2))
dL(r'\begin{align}', align_res, r'.\end{align}')
```

\begin{align} \frac{q_{1}}{wL} &= +0.0240387 \cos(15.4182\omega_0t),&\frac{q_{2}}{wL} &= -0.00399941 \cos(49.9649\omega_0t),\\\\\frac{q_{3}}{wL} &= +0.00132873 \cos(104.248\omega_0t) .\end{align}

### The Bending Moment $M(0, t)$

It is $M = -EJ y''$, or

$$  M = -EJ\,wL \sum \eta_n\frac{1}{L^2}\frac{d^2\phi_n(\beta_nL\,\xi)}{d\xi^2}\cos\omega_it =
     -\left(\sum\eta_n\frac{d^2\phi_n(\beta_nL\,\xi)}{d\xi^2}\cos\omega_it\right) W.
$$

Because
$$
  \left.-\frac{d^2\phi_n(\beta_nL\,\xi)}{d\xi^2}\right|_{\xi=0} =
  \left.\beta_n^2L^2\left(\strut
    (\sin\beta_n\xi + \sinh\beta_n\xi) A_n +
    (\cos\beta_n\xi + \cosh\beta_n\xi)B_n\right)
  \right|_{\xi=0} = 2\frac{\omega_n}{\omega_0}B_n$$

we can finally write $M(0, t) = 2 W \sum \frac{\omega_n}{\omega_0}\,\eta_n\,B_n \cos(\frac{\omega_n}{\omega_0}\,\omega_0t)$

```python
dL(r'$$\frac{M(0)}{W} =', ' '.join(
    r'%+.6g\cos(%.2f\omega_0t)'%(coef, w) for coef, w in zip(w1*eta*B*2, w1)), r'+\ldots$$')
```

$$\frac{M(0)}{W} = -0.740691\cos(15.42\omega_0t) +0.399659\cos(49.96\omega_0t) -0.277035\cos(104.25\omega_0t) +\ldots$$

#### Static bending moment

Before proceding with $M(0,t)$ it is interesting to compute the approximation to $M(x, 0)$, that is _how well_ we are approximating the bending moment at the beginning of our analysis.
\begin{multline}
  M(x, 0) = -EJ \, wL \sum \eta_n\,\frac{d^2\phi_n}{d\,x^2} \cos(\omega_n 0) = \\
  = W \sum \frac{\omega_n}{\omega_0}\eta_n\left(\strut
    (\sin\beta_n\xi + \sinh\beta_n\xi) A_n +(\cos\beta_n\xi + \cosh\beta_n\xi)B_n
  \right)
\end{multline}

```python
# d2y is the NEGATIVE of the second spatial derivative of \phi
d2y = A*(sin(b_xi)+sinh(b_xi)) + B*(cos(b_xi)+cosh(b_xi))
M_stat = d2y@(eta*BetaL**2)
```

```python
# plot the derivatives
plt.legend(plt.plot(xi, d2y), (r"$-\phi''_{%d}$"%(n) for n, _ in enumerate(d2y, 1)), ncol=3)
plt.xlabel(xlbl)
plt.ylabel(r'$-d^2\phi/d\xi^2$')
plt.title('The second spatial derivatives of the mode shapes')
rev_y()
plt.show()

# plot the static M(x)/W (from -0.5 to +1) and the approximation using N modes
plt.plot(xi, M_stat, label=r'$\sum_1^{%d}M_{\mathrm{stat, }n}$'%N)
plt.plot(xi, 0.5*(xi-1)+xi, label=r'$M_\mathrm{stat}$')
plt.xlabel(xlbl)
plt.ylabel(r'$M\, /\, W$')
plt.title('Comparison of the static bending moment and its approximation')
plt.legend(ncol=3)
rev_y()
M_stat[0]
```

As you can see, we are in trouble inasmuch the bending moment in $x=L$ _can not_ be approximated correctly,
as we are summing contributions of the type $\phi''(L) \equiv 0$. Ask your favourite search engine about "Gibbs phenomenon".

#### The response in terms of bending at the fixed support
We need an 1D array with the instants of (dimensionless) time, a 2D array with the arguments of the cosines (i.e., time multiplied by the frequencies), a 2D array with the cosines of said arguments and finally we multiply the cosines by the coefficients

```python
t = linspace(0, 5, 5001)
wt = outer(t, BetaL**2)
cwt = cos(wt)
M = cwt@(2*B*eta*BetaL**2)
```

```python
if 1:
    plt.plot(t, M)
    plt.title('Bending moment @ x=0, with $\\zeta=0.0\\%$')
    plt.xlabel(r'$\tau=\omega_0t$')
    plt.ylabel(r'$M(0)/W$')
    plt.ylim((-1.5, 1.5))
    plt.yticks((-1, -0.5, 0, 1))
    rev_y()

```

Let's see what happens if we take into account a very small value of viscous damping.

Similar to what we have done for MDoF systems, we assume that the modal shapes are not affected by the presence of damping and that the modal response is $$q_i(t) = \exp(-\zeta\omega_it) (A_i\sin\omega_{D,i}t + B_i\cos\omega_{D,i}t).$$
In our case, $q_i(0) = q_{o,i}$ and $\dot{q}_i(0)=0$, it is $B_i = q_{o,i}$ and $A_i = \zeta B_i/\sqrt{1-\zeta^2}$; for small $\zeta$, we can approximately write $\omega_{D,i}\simeq\omega_i$ and $A_i\simeq\zeta B_i$.

Let's say $\zeta=0.5\%$ and compute and plot the (approximated) response

```python
zeta = 0.005
swt = sin(wt)
Md = (exp(-zeta*wt)*(zeta*swt+cwt))@(2*B*eta*BetaL**2)
```

```python
if 1:
    plt.plot(t, Md)
    plt.title('Bending moment @ x=0, with $\\zeta=0.5\\%$')
    plt.xlabel(r'$\tau=\omega_0t$')
    plt.ylabel(r'$M(0)/W$')
    plt.ylim((-1.5, 1.5))
    plt.yticks((-1, -0.5, 0, 1))
    rev_y()
```

As you can see, the effects of viscous damping are really important for the high frequency modal components, that is because the decrement
$$
  \exp(-\zeta\omega_nt) = \exp(-\zeta\frac{2\pi}{T_n}t) =
  e^{-2\pi\zeta}e^\frac{t}{T_n} = e^{-2\pi\zeta}e^{n_\text{cycles}}.
$$
is proportional to the _exponential_ of the number of cycles involved...

Here we stop with this printed solution, but if you like you can download [the notebook](nbiewer.jupyter.com/...) that is the source of this paper and as an added bonus you'll receive an interactive plotting widget that lets you investigate the variation of the response with respect to the value of the damping ratio.  A further bonus (well, possibly so...) is the code that produced the plots and the mathematical displays, code that's been _hidden_ in this paper to streamline the narrative but could be interesting from a different point of view.

```python
def p(z=0.5):
    zeta = z/100.0
    M = (exp(-zeta*wt)*(zeta*swt+cwt))@(2*B*eta*BetaL**2)
    plt.plot(t, M)
    plt.title('Bending moment @ x=0, with $\\zeta=%.1f\\%%$'%z)
    plt.xlabel(r'$\tau=\omega_0t$')
    plt.ylabel(r'$M(0)/W$')
    plt.ylim((-1.5, 1.5))
    plt.yticks((-1, -0.5, 0, 1))
    rev_y()
interact(p, z=(0, 2.0)) ;
```

    interactive(children=(FloatSlider(value=0.5, description='z', max=2.0), Output()), _dom_classes=('widget-inter…

## The Initialization Cells

I have placed the inizialization cells here, rather than at the beginning of the notebook, to not disturb the flow of the narrative. We have all the necessary imports, plus the definition of some helper functions, mostly connected to the formatting  of the results...

### The import  statements

```python
%matplotlib inline
import numpy as np
from numpy import array, cos, cosh, exp, linspace, outer
from numpy import pi, sin, sinh, sqrt, tanh, trapz
import matplotlib.pyplot as plt
from scipy.optimize import newton
from jupyterthemes import jtplot
from itertools import count, zip_longest
from ipywidgets import interact
```

### Configuration statements and function definitions

```python
jtplot.style(context='paper', fscale=1.5, figsize=(15, 3))
def rev_y(): plt.ylim(plt.ylim()[::-1])

def dL(*l):
    from IPython.display import Latex
    display(Latex(' '.join(l)))

def grouper( iterable, N):
    return zip_longest(*([iter(iterable)]*N))

def groupalign(seq, N, variable='x_{%d}', fmt='%g', closing='.'):
    beg = r'\begin{align*}'
    end = closing + r'\end{align*}'
    xfmt = variable + '&=' + fmt
    g = grouper(seq, N)
    body = r',\\'.join(
        ',&'.join(
            xfmt%(1+i+N*j, x) for i, x in enumerate(items) if x != None)
        for j, items in enumerate(g))
    return ''.join((beg, body, end))
```


# 1. 1D example

$\begin{align}
    f_{simulator}\colon\mathbb{R}^{N\times D} &\to\mathbb{R}^{N} \\
    X &\mapsto \mathbf{y}
\end{align}$

```python
import numpy as np
import torch
```

```python
# set logger and enforce reproducibility
from GPErks.log.logger import get_logger
from GPErks.utils.random import set_seed
log = get_logger()
seed = 8
set_seed(seed)  # reproducible sampling
```

**1D function example**: Forrester et al. (2008)

$f(x) = (6x - 2)^2 \sin(12x - 4)$

```python
# function to learn (normally a high-dimensional, expensive deterministic model)
from GPErks.utils.test_functions import forrester
f = lambda x: forrester(x)
D = 1
```

```python
# build dataset
from GPErks.gp.data.dataset import Dataset
dataset = Dataset.build_from_function(
    f,
    D,
    n_train_samples=10,
    n_test_samples=10,
    design="srs",
    seed=seed,
    l_bounds=[0],
    u_bounds=[1]  # can put None if, as in this case, parameters range in [0, 1]
)
```

**Gaussian process emulator (GPE):**

$f(\mathbf{x}) = h(\mathbf{x}) + g(\mathbf{x})$

**deterministic part:**

$h(\mathbf{x}) := \beta_0 + \beta_1 x_1 + \dots + \beta_{D} x_{D}$

**stochastic part:**

$\begin{align}
    &g(\mathbf{x})\sim\mathcal{GP}(\mathbf{0},\,k_{\text{SE}}(d(\mathbf{x},\,\mathbf{x}'))) \\
    &k_{\text{SE}}(d(\mathbf{x},\,\mathbf{x}')) := \sigma_f^2\, e^{-\frac{1}{2}\,d(\mathbf{x},\,\mathbf{x}')} \\
    &d(\mathbf{x},\,\mathbf{x}') := (\mathbf{x}-\mathbf{x}')^\mathsf{T}\,\Lambda\,(\mathbf{x}-\mathbf{x}')
\end{align}$

**likelihood:**

$y=f(\mathbf{x}) + \varepsilon,\quad \varepsilon\sim\mathcal{N}(0,\,\sigma_n^2)$

```python
# choose likelihood
from gpytorch.likelihoods import GaussianLikelihood
likelihood = GaussianLikelihood()
```

```python
# choose mean function
from gpytorch.means import LinearMean
mean_function = LinearMean(input_size=dataset.input_size)
```

```python
# choose covariance function (kernel)
from gpytorch.kernels import RBFKernel, ScaleKernel
kernel = ScaleKernel(RBFKernel(ard_num_dims=dataset.input_size))
```

```python
# choose metrics
from torchmetrics import MeanSquaredError, R2Score
metrics = [MeanSquaredError(), R2Score()]
```

```python
# define experiment
from GPErks.gp.experiment import GPExperiment
experiment = GPExperiment(
    dataset,
    likelihood,
    mean_function,
    kernel,
    n_restarts=3,
    metrics=metrics,
    seed=seed,  # reproducible training
    learn_noise=True  # y = f(x) + e, e ~ N(0, sigma^2I)
)
```

```python
# choose training options: device + optimizer
device = "cuda" if torch.cuda.is_available() else "cpu"
optimizer = torch.optim.Adam(experiment.model.parameters(), lr=0.1)
```

```python
# train model
from GPErks.train.emulator import GPEmulator
emulator = GPEmulator(experiment, device)
emulator.train(optimizer)
```

```python
# inference on stored test set
x_test = dataset.X_test
y_test = dataset.y_test

y_mean, y_std = emulator.predict(x_test)

for metric in metrics:
    print( metric(
        torch.from_numpy(y_mean), torch.from_numpy(y_test)
        ).item()
    )
```

```python
# perk n.1: automatic inference
from GPErks.perks.inference import Inference
inference = Inference(emulator)
inference.summary()  # can be retrieved from inference.scores_dct
print( inference.scores_dct )
```

```python
# nice plotting
x_train = dataset.X_train
y_train = dataset.y_train

xx = np.linspace(dataset.l_bounds[0], dataset.u_bounds[0], 1000)
yy_mean, yy_std = emulator.predict(xx)
yy_true = f(xx)

import matplotlib.pyplot as plt
height = 9.36111
width = 5.91667
fig, axis = plt.subplots(1, 1, figsize=(4*width/3, height/2))

axis.plot(xx, yy_true, c="C0", ls="--", label="true function")

CI = 2
axis.plot(xx, yy_mean, c="C0", label="predicted mean")
axis.fill_between(
    xx, yy_mean - CI * yy_std, yy_mean + CI * yy_std, color="C0", alpha=0.15, label="~95% CI"
)
axis.scatter(x_train, y_train, fc="C0", ec="C0", label="training data")

axis.scatter(x_test, y_test, fc="none", ec="C0", label="testing data")

axis.legend(loc="best")
fig.tight_layout()
plt.show()
```

```python
# check testing points
inference.plot()
```

```python
# draw samples from the posterior distribution
y_mean, y_std = emulator.predict(x_test)
print(y_mean.shape)
print(y_std.shape)
```

```python
y_samples = emulator.sample(x_test, n_draws=5)
print(y_samples.shape)
```

```python
y_samples = emulator.sample(xx, n_draws=5)

fig, axis = plt.subplots(1, 1, figsize=(4*width/3, height/2))

for i, ys in enumerate(y_samples):
    axis.plot(xx, ys, lw=0.8, label=f"posterior sample #{i+1}", zorder=1)

axis.plot(xx, yy_mean, c="k", lw=2, ls="--", label="posterior mean", zorder=2)
axis.scatter(x_train, y_train, fc="k", ec="k", label="training data", zorder=2)

axis.legend(loc="best")
fig.tight_layout()
plt.show()
```


# Modélisation d'un vol d'une fusée

## 1. Introduction

Nous allons étudier la trajectoire de vol d'une fusée dans l'air. En utilisant le langage Python nous allons la calculer numériquement. L'équation de la fusé est decrite comme suit :

\begin{align}
\frac{dh}{dt}& = v \\
(m_s+m_p) \frac{dv}{dt}& = -(m_s+m_p)g + \dot{m}_pv_e - \frac{1}{2}\rho v|v|AC_D
\end{align}

$m_s = 50kg$ est la masse de fusée hors carburant

$g = 9.81 \frac{m}{s^2}$ est l'accélération du à la gravité supposée constante

$\rho = 1.091 \frac{kg}{m^3}$ est la densité de l'air supposée constante

$A = \pi r^2$ est l'air de la section transverse de la fusée et vaut $r = 0.5 m$

$v_e = 325 \frac{m}{s}$ est la vitesse d'éjection du gaz à la sortie de la tuyère

$C_D = 0.15 $ est le coefficient de trainer

On suposera aussi que la masse du carburant enbarqué évolue selon la loi
$$mp = mp - \int^t_0 20 d\tau$$ où 20 est la consomation de kilo de carburant par seconde.

## 2.Résolution

Comme la résolution analytique est peu commode nous resolverons le problème précédent de manière numérique.

Pour résoudre numériquement ce problème, nous utiliserons la méthode d'Euler progressive. La descrétisation est comme suit :

$$ h_{n+1} = h_n + f(h_n) \Delta t $$
$$ v_{n+1} = v_n + g(v_n) \Delta t $$
Ce qui donnera au final :
$$ v_{n+1} = \-g + frac { 20 ve - 0.5 \rho |v_n|v_n A Cd }{ms+mp} \Delta t + v_n $$
$$ h_{n+1} = v_n \Delta t + h_n $$

Nous importons quelques libraires pour effectués cette résolution. La première est numpy est une libraire avec des fonctionnalités orientées calculs. De cette librairie nous allons utiliser les numpy arrays. Ces types de données ont l'avantage par rapport aux listes directement implémentées dans python d'avoir des fonctionnalités sumplémentaires.
Ensuite nous utiliserons matplotlib pour afficher les graphiques

```python
"""
importation des librairies
"""

import numpy
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
"""
initialisation des variables
"""
step = 0.1
ms = 50
g = 9.81
p = 1.091
r = 0.5
A = numpy.pi*(r**2)
ve = 325
Cd = 0.15
mp =  100
hn = 0
vi = 0
i = 0

first_estimate = 40 #estimation du temps
len_descrit_vec = int(first_estimate/step)+1 # donne la tailme des vecteurs

t = numpy.linspace(0,first_estimate,int(len_descrit_vec))
v = numpy.zeros(len_descrit_vec)
h = numpy.zeros(len_descrit_vec)
m = numpy.zeros(len_descrit_vec)
```

On definit dans le cadre précédent les variables nécessaires au bon fonctionnement de la boucle ainsi que les variables globales introduite dans l'équation décrivant le système. les vecteurs t,v,h,m sont repectivement les vecteurs qui receveront les valeurs calculers pendant l'execution de la boucle. La taille de ces vecteurs est calculé sur le nombre d'itération necessaire pour calculer toute la simulation. Ce nombre d'itération est la taille de l'intervalle de la simulation  divisée par la taille du step (pas d'intégration). La taille de l'intervalle a été estimé en post traitement. En effet il est relativement compliqué de traité un vecteur de taille non-prédéfinie avec une numpy array.

```python
thmax =0
tvmx =0
hmax =0
hvmx = 0
vmx =0
tsol =0
vsol = 0

for i in range(1,int(len_descrit_vec)):
    if hn<0:
        break
    hn = vi * step + hn
    if t[i] > 5: # cette boucle if est la pour tenir compte de la masse variable en effet apres 5 seconde
                 #il n'y a plus de carburant
        mp = 0;
        vi = (-g+(-0.5*vi*p*abs(vi)*A*Cd)/(ms+mp))*step + vi
    else :
        vi = ((-1)*g+20*ve/(ms+mp)-(0.5*vi*p*abs(vi)*A*Cd)/(ms+mp))*step + vi
        mp = mp - 20*step

    if hn>hmax:
        hmax = hn
        tHmax = t[i]
    if vi > vmx:
        vmx = vi
        tvmx = t[i]
        hvmx = hn
    if hn<0:
        tsol = t[i]
        vsol = vi

    v[i] = vi
    h[i] = hn
    m[i] = mp

"""
Impression des resultats
"""

print("vitesse est maximale en "+str(tvmx)+" s et vaut "+str(vmx)+ " m/s où la hauteur vaut " + str(hvmx), "m.")
print("la hauteur est maximale en t = "+ str(tHmax) +" et vaut h = "+str(hmax))
print("la fusee touche le sol en t = " + str((tsol)))
print("avec la vitesse " +str(vsol))
```

    vitesse est maximale en 5.0 s et vaut 232.1061334126179 m/s où la hauteur vaut 523.5228342918476 m.
    la hauteur est maximale en t = 15.7 et vaut h = 1334.1829454303067
    la fusee touche le sol en t = 37.1
    avec la vitesse -86.00683498380592

Le code se déroule de la manière suivante. On va calculer la valeur suivante de la table en se basant sur la précédente en utilisant la méthode d'Euleur. Les variables definies avant la boucle sont là pour stocker la valeur des différentes information que l'on cherche tel que la hauteur maximale ainsi de suite.

Les valeurs initiales sont données par le condition initiale de notre équation différentielles. Ainsi h(0) = 0 et v(0) = 0.

A propos de la boucle, elle se décompose en 3 étages de conditions (if). Le premier _if_ est là pour vérifier que le sol est atteint ou pas. En effet si la hauteur devient négative cela signifie bien que l'on a touché le sol.
Le deuxième étage de condition est là pour vérifier l'état du carburant. La vitesse cessera de croitre si il n'y a plus de carburant en toute logique.
Enfin la dernière série de codition est là pour trouver le maximum des différentes données. Il exite une autre méthode utilisant la fonction numpy.argmax mais cela reviendrait à la même chose.

## 3.Graphiques et interprétation des résultats

```python
"""
Impression du graphique hauteur
"""

plt.figure(figsize=(10,5))
plt.plot(t,h,label="hauteur")
plt.grid(True)
plt.title("hauteur de la fusée en fonction du temps")
plt.xlabel('t (s)')
plt.ylabel('h (m)')
plt.legend()
plt.show()
```

Analysons le graphique de la hauteur. On observe bien la première phase du vol où la fusée accélère. Cela se traduit graphiquement par une concavité négative. Physiquement, nous avant la même intuition vu que on continue à éjecter du carburant durant cette partie. Après 5 sec, nous changeons de concavité pour atteindre un maximum. Puis la fusée vas descendre en accélerant du à la gravité. Enfin en t = 37.1 s la fusée touche le sol ce qui explique pourquoi la hauteur est nul après ce temps-là.

```python
"""
Impression du graphique vitesse
"""
plt.figure(figsize=(10,5))
plt.plot(t,v, label= "vitesse")
plt.xlabel('t (s)')
plt.ylabel('v (m/s)')
plt.title("vitesse de la fusée en fonction du temps")
plt.grid(True)
plt.legend()
plt.show()
```

Regardons à présent le graphique de la vitesse en fonction du temps. On observe bien que le maximum de la vitesse et le point d'inflexion de la hauteur coïncide. Ce qui est un bon point car nous retrouvons bien le vient entre la vitesse et la hauteur la première étant la dérivée de la seconde. Un deuxième point intéressant est d'observé que la vitesse ne tends pas vers $ - \infty$. Nous avons modélisés des forces de frotements dans nos calculs. Cette diminution de la pente de la vitesse est s'interprète comme un equilibre entre la gravité et les frottements qui ont tendance à s'opposer à cette dernière. Un petit bémol quant à la justesse physique de la simaltion est que la vitesse passe brusquement de -86 m/s à 0. Ceci est normal car nous coupons brusquement la simulation lorsque la fusée touche le sol.

Un autre point de l'analyse imporant est de vérifier si notre solution a bien convergé. Regardons le tableau ci-dessous:

| step | hmax (m)| vmax(m/s) |
|-|-|-|
|0,1|1334,2|232.1|
|0,01|1348,8|232.4|
|0,001|1350,2|232.4|
|0,0001|1350,4|232.4||

On voit que que l'on a une solution qui a convergé avec un pas de 0.001 . Mais comme on voit que malgré l'utilsation d'un pas de temps assez grossié on a une erreur inférieur au pourcent ce qui est acceptable pour une simulation numérique. Ces différences sont dues à la propagation des érreurs d'apprximation à chaque itération.

## 5. Conclusion

Nous avons vérifié nos resultats avec ceux du site et ils sont bons.

Au travers de cette exercice, nous avons appris à utiliser de méthodes numériques pour résoudre des équations différentielles difficiles à résoudre analytiquement ainsi que des outils de visualisations graphiques.

```python

```


```python
import torch
import numpy as np
from matplotlib import pyplot as plt
from torch.distributions.utils import broadcast_all
from probabll.distributions.utils import mc_kl, mc_entropy
from torch.distributions.kl import kl_divergence
```

Let's import some densities with support over $(0, 1)$

```python
from torch.distributions.uniform import Uniform
from probabll.distributions import RightTruncatedExponential
from probabll.distributions import Kumaraswamy
```

# Mixture of Discrete and Continuous over [0,1)

This distribution is a mixture of a point masses at $0$ and a continuous distribution over the support $(0, 1)$.

The distribution function looks like this:

\begin{equation}
f(x|\alpha, \beta) = \alpha \delta(x) + (1-\alpha) g(x|\beta) \mathbf 1_{(0, 1)}(x)
\end{equation}

where $0 < \alpha < 1$, $g(x|\beta)$ is a (properly normalised) density over $(0,1)$, and $\mathbf 1_{(0, 1)}(x)$ is the indicator function.

```python
from probabll.distributions.mixdc import MixtureD0C01
```

```python
p = MixtureD0C01(
    probs0=torch.full([1], 0.2),
    cont=RightTruncatedExponential(rate=torch.ones(1), upper=1.))
print(p.p0, p.pc)
```

    tensor([0.2000]) tensor([0.8000])

```python
x = p.rsample(torch.Size([10]))
x
```

    tensor([[0.3865],
            [0.2682],
            [0.5663],
            [0.0000],
            [0.0557],
            [0.4937],
            [0.5800],
            [0.0000],
            [0.3457],
            [0.2662]])

```python
_= plt.hist(p.rsample(torch.Size([1000])).numpy(), bins=50)
```

```python
p2 = MixtureD0C01(
    probs0=torch.full([1], 0.5),
    cont=RightTruncatedExponential(rate=torch.ones(1) * 1., upper=1.))
print(p2.p0, p2.pc)
```

    tensor([0.5000]) tensor([0.5000])

```python
_= plt.hist(p2.rsample(torch.Size([1000])).numpy(), bins=50)
```

```python
mc_kl(p, p2, 1000000)
```

    tensor([0.1929])

```python
kl_divergence(p, p2)
```

    tensor([0.1927])

```python
mc_entropy(p, 1000000)
```

    tensor([0.4673])

```python
p.entropy()
```

    tensor([0.4679])

# Mixture of Discrete and Continuous over [0,1]

This distribution is a mixture of a point masses at $0$, another at $1$, and a continuous distribution over the support $(0, 1)$.

The distribution function looks like this:

\begin{equation}
f(x|\alpha, \beta) = \alpha_0 \delta(x) + \alpha_1 \delta(1 - x) + (1-\alpha_0-\alpha_1) g(x|\beta) \mathbf 1_{(0, 1)}(x)
\end{equation}

where $0 < \alpha_i < 1$, $g(x|\beta)$ is a (properly normalised) density over $(0,1)$, $\mathbf 1_{(0, 1)}(x)$ the indicator function.

```python
from probabll.distributions.mixdc import MixtureD01C01
```

```python
probs = torch.from_numpy(np.array([0.3, 0.5, 0.2])).float()
probs
```

    tensor([0.3000, 0.5000, 0.2000])

```python
p = MixtureD01C01(
    cont=Uniform(torch.zeros([1]), torch.ones([1])),
    probs=probs
)
```

```python
x = p.rsample(torch.Size([10]))
x
```

    tensor([[1.0000],
            [1.0000],
            [1.0000],
            [0.0637],
            [0.0452],
            [1.0000],
            [1.0000],
            [1.0000],
            [0.0000],
            [0.0000]])

```python
_= plt.hist(p.rsample(torch.Size([1000])).numpy(), bins=50)
```

```python
q = MixtureD01C01(
    cont=Uniform(torch.zeros([1]), torch.ones([1])),
    probs=torch.from_numpy(np.array([0.2, 0.5, 0.3])).float()
)
```

```python
p.entropy()
```

    tensor([1.0297])

```python
q.entropy()
```

    tensor([1.0297])

```python
mc_entropy(p, 1000000)
```

    tensor([1.0298])

```python
kl_divergence(p, q)
```

    tensor([0.0405])

```python
mc_kl(p, q, 1000000)
```

    tensor([0.0407])

```python
- torch.log(p.cont.cdf(1) - p.cont.cdf(0))
```

    tensor([-0.])

# Prior and Posterior in VI

Let's specify a prior

\begin{equation}
    \alpha_0 \delta(x) + \alpha_1 \delta(1-x) + (1-\alpha_0-\alpha_1) \mathcal U(x|0, 1) \mathbf 1_{(0,1)}(x)
\end{equation}

and an approximate posterior
\begin{equation}
    \alpha_0 \delta(x) + \alpha_1 \delta(1-x) + (1-\alpha_0-\alpha_1) \text{Exp}_{(0,1)}(x|\lambda) \mathbf 1_{(0,1)}(x)
\end{equation}

where $\text{Exp}_{(0,1)}(\lambda)$ is an RightTruncatedExponential rv.

We provide support to KL for pairs like that, where the densities are mixtures D01C01 (or D0C01) and the prior uses a Uniform density over $(0, 1)$. They are particularly simple due to the uniform prior. Whenever the density used in the approximate posterior admits a closed-form entropy, we have the entire expression in closed form!

```python
p = MixtureD01C01(
    cont=Uniform(torch.zeros([1]), torch.ones([1])),
    probs=torch.from_numpy(np.array([0.2, 0.5, 0.3])).float()
)
```

```python
q1 = MixtureD01C01(
    cont=RightTruncatedExponential(rate=torch.full([1], 10.), upper=1.),
    probs=torch.from_numpy(np.array([0.3, 0.5, 0.2])).float()
)
```

```python
_= plt.hist(q1.rsample(torch.Size([1000])).numpy(), bins=50)
```

```python
kl_divergence(q1, p)
```

    tensor([0.3012])

```python
mc_kl(q1, p, 1000000)
```

    tensor([0.3001])

```python
q1.entropy()
```

    tensor([0.7690])

```python
mc_entropy(q1, 1000000)
```

    tensor([0.7696])

Another possible pair:

\begin{equation}
    \alpha_0 \delta(x) + \alpha_1 \delta(1-x) + (1-\alpha_0-\alpha_1) \mathcal U(x|0, 1) \mathbf 1_{(0,1)}(x)
\end{equation}

and an approximate posterior
\begin{equation}
    \alpha_0 \delta(x) + \alpha_1 \delta(1-x) + (1-\alpha_0-\alpha_1) \text{Kumaraswamy}(x|a, b) \mathbf 1_{(0,1)}(x)
\end{equation}

```python
q2 = MixtureD01C01(
    cont=Kumaraswamy(a=torch.full([1], 5.), b=torch.full([1], 5.)),
    probs=torch.from_numpy(np.array([[0.3, 0.5, 0.2], [0.2, 0.5, 0.3]])).float()
)
```

```python
_= plt.hist(q2.rsample(torch.Size([1000])).numpy(), bins=50)
```

```python
kl_divergence(q2, p)
```

    tensor([0.1590, 0.1777])

```python
mc_kl(q2, p, 1000000)
```

    tensor([0.1637, 0.1774])

```python
q2.entropy()
```

    tensor([0.9112, 0.8520])

```python
mc_entropy(q2, 1000000)
```

    tensor([0.9069, 0.8512])

```python

```


# Ekman spiral derivation
Upon Reynolds-averaging, the primitive equations for the atmosphere can be written as

$$ \frac{\partial u}{\partial t} + u_i \frac{\partial u}{\partial x_i} - fv = -\frac{1}{\rho}\frac{\partial P}{\partial x} - \frac{\partial \overline{u'u'}}{\partial x} - \frac{\partial \overline{u'v'}}{\partial y} - \frac{\partial \overline{u'w'}}{\partial z} $$
$$ \frac{\partial v}{\partial t} + u_i \frac{\partial v}{\partial x_i} + fu = -\frac{1}{\rho}\frac{\partial P}{\partial y}  - \frac{\partial \overline{v'u'}}{\partial x} - \frac{\partial \overline{v'v'}}{\partial y} - \frac{\partial \overline{v'w'}}{\partial z} $$

If we neglect large-scale advection and subsidence and assume stationarity, the first two terms cancel. The pressure gradient term can be eliminated by substituting the expressions for geostropich balance,

$$ fv_g = \frac{1}{\rho}\frac{\partial P}{\partial x} $$
$$ -fu_g = \frac{1}{\rho}\frac{\partial P}{\partial y} $$

Finally, we assume that the momentum fluxes in the last terms can be approximated with first order closure,

$$ \overline{u'w'} = - K \frac{\partial u}{\partial z}$$
$$ \overline{v'w'} = - K \frac{\partial v}{\partial z}$$

We could do likewise for the horizontal fluxes, but considering that horizontal gradients are generally small compared to vertical gradients, we also discard these terms. This leaves us with

$$ -fv = -fv_g + \frac{\partial}{\partial z} \left(K \frac{\partial u}{\partial z}\right)$$
$$ fu = fu_g + \frac{\partial}{\partial z} \left(K \frac{\partial v}{\partial z}\right)$$

We can rewrite this as

$$ -f(v-v_g) = \frac{\partial}{\partial z} \left(K \frac{\partial u}{\partial z}\right)$$
$$ f(u-u_g) = \frac{\partial}{\partial z} \left(K \frac{\partial v}{\partial z}\right)$$

These coupled equations define the Ekman spiral. If K would be constant, the expressions are somewhat simplified

$$ K\frac{\partial^2u}{\partial z^2} = -f(v-v_g) $$
$$ K\frac{\partial^2v}{\partial z^2} = f(u-u_g) $$

An analytical solution for this system can be found by defining a complex velocity $ W = u + iv $, such that the two equations can be combined into one. To this end, the meridional equation is multiplied by $i = \sqrt{-1}$ and subsequently added to the lateral equation:

$$ iK\frac{\partial^2v}{\partial z^2} = if(u-u_g) $$
$$ K\frac{\partial^2u}{\partial z^2}+iK\frac{\partial^2v}{\partial z^2} = -f(v-v_g) + if(u-u_g) $$
$$ K\frac{\partial^2W}{\partial z^2} = if(u+iv) - if(u_g-iv_g)$$
$$ \frac{\partial^2W}{\partial z^2} - \frac{if}{K}W + \frac{if}{K}W_g = 0$$

This is an inhomogeneous, second order differential equation, which can be solved in two steps. First, we find a particular integral, by realizing that one solution to the problem would be that z is independent of height. This solution is given by

$$  \frac{if}{K}W = \frac{if}{K}W_g $$

from which it follows that $ W = W_g $. Then the homogeneous part of the differential equation is solved by substituting a trial solution of the form $W(z) = A \exp(\lambda z)$, which gives:

$$ \frac{\partial^2W}{\partial z^2} = \frac{if}{K}W $$
$$ \lambda^2 A \exp(\lambda z) = \frac{if}{K} A \exp(\lambda z) $$
$$ \lambda^2 = \frac{if}{K} $$
$$ \lambda = \pm \sqrt{\frac{if}{K}} $$
$$ \lambda = \pm (1+i)\sqrt{\frac{f}{2K}} = \pm (1+i) \gamma $$

where $ \sqrt{i} = \frac{1+i}{\sqrt{2}} $ is used in the last step and $\gamma^{-1}$ is known as the 'Ekman depth'. Substituting these roots in the solution gives

$$ W(z) = A \exp((\gamma + i\gamma) z) + B \exp((-\gamma - i\gamma) z) $$

Now, the first term on the right-hand side, grows exponentially with z. This is not a physical solution, since we expect $W$ to converge to $W_g$ at the top of the Ekman layer, and therefore this term must be discarded. This leaves us with the physical solution

$$ W(z) = W_g + B \exp((-\gamma - i\gamma) z) $$

where the particular solution $W = W_g$ has been substituted in the solution of the homogeneous equation. To find B, we insert the boundary condition $ W(0) = 0$, which yields

$$ W_g + B \exp(0) = 0 $$

yielding $B = -W_g$ and the complete solution

$$ W(z) = W_g \left[1 - \exp(-\gamma z)\exp(-i\gamma z) \right]$$

Using Euler's formula $\exp(-ix) = \cos(x) -i \sin(x)$ we can split the complex exponential terms in a real and imaginary part

$$ u(z) + iv(z) = ( u_g + iv_g ) \left[1 - \exp(-\gamma z)\cos(\gamma z) + i\exp(-\gamma z)\sin(\gamma z) \right]$$
$$ u(z) + iv(z) = u_g - u_g \exp(-\gamma z)\cos(\gamma z) + iu_g \exp(-\gamma z)\sin(\gamma z) + iv_g - iv_g\exp(-\gamma z)\cos(\gamma z) - v_g \exp(-\gamma z)\sin(\gamma z)$$

so that we can finally retrieve the equations for u and v seperately

$$ u(z) = u_g - u_g \exp(-\gamma z)\cos(\gamma z) - v_g \exp(-\gamma z)\sin(\gamma z) $$
$$ v(z) = v_g - v_g \exp(-\gamma z)\cos(\gamma z) + u_g \exp(-\gamma z)\sin(\gamma z)  $$

Let's implement this system in Python.

```python
# Import packages
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Set parameters
ug = 10.; vg = 0.
f = 1e-4; K = 10.
gamma = np.sqrt(f/(2*K))

def uwind(ug,vg,gamma,z):
    return ug - ug*np.exp(-gamma*z)*np.cos(gamma*z) - vg*np.exp(-gamma*z)*np.sin(gamma*z)

def vwind(ug,vg,gamma,z):
    return vg - vg*np.exp(-gamma*z)*np.cos(gamma*z) + ug*np.exp(-gamma*z)*np.sin(gamma*z)

z = np.arange(2001)
u = uwind(ug,vg,gamma,z)
v = vwind(ug,vg,gamma,z)

f,ax = plt.subplots(1,2,figsize=(12,5))
ax[0].plot(u,z,v,z)
ax[1].plot(u,v)
plt.show()
```

Alternatively, we may try to solve the system numerically. We can start with an initial assumption for the profile and iteratively solve for u and v. Let's keep K constant first and then try it with z-dependent K where we can use the fixed-K profile as initial guess.

The expressions for u and v with time dependency reitroduced, are:

$$ \frac{\partial u}{\partial t} = f(v-v_g)+\frac{\partial}{\partial z}\left[K \frac{\partial u}{\partial z} \right] $$
$$ \frac{\partial v}{\partial t} = -f(u-u_g)+\frac{\partial}{\partial z}\left[K \frac{\partial v}{\partial z} \right] $$

```python
# Import packages
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Defining functions
def uwind(f,v,vg,K,u,dt):
    coriolis = f*(v-vg)
    d_u = np.gradient(u)
    d_uw = K*np.gradient(d_u)
    tendency = coriolis+d_uw
    new_u = u + tendency*dt
    return new_u

def vwind(f,u,ug,K,v,dt):
    coriolis = -f*(u-ug)
    d_v = np.gradient(v)
    d_vw = np.gradient(K*d_v)
    tendency = coriolis+d_vw
    new_v = v + tendency*dt
    return new_v

# Set parameters
ug = 10.; vg = 0.
f = 1e-4; K = 10.
dt = 0.1

z = np.arange(2000)+1
u = np.log(z/0.002)
v = 0.3*np.log(z/0.002)

plt.plot(u,z,v,z)

for i in range(20000):
  u = uwind(f,v,vg,K,u,dt)
  v = vwind(f,u,ug,K,v,dt)

plt.plot(u,z,v,z)

plt.show()

```

## Realistic K-profile
The K-profile is not constant in reality. Rather, it has the form

$$ K(z) = \frac{\kappa u_* z}{1+\alpha \frac{z}{L}}\left(1-\frac{z}{h}\right)^2 $$

This looks like:

```python
# Import packages
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def kprofile(z):
    '''
    Bert's notes
    '''
    kappa = 0.4 # Von Karman constant
    ust = 0.3 # just a guess
    alpha = 5. # stability function
    L = 500. # Obukhov length
    h = float(len(z))
    return kappa*ust*z/(1+alpha*z/L)*(1-z/h)**2

z = np.arange(2001)
K = kprofile(z)

plt.plot(K,z)
plt.show
```

In general, K varies with height and stability:

$$ K(z) = \frac{\kappa u_* z}{1+\alpha \frac{z}{L}}\left(1-\frac{z}{h}\right)^2 $$

where the Businger-Dyer function for stable stratification is recognizable in the denominator. Our aim is to find a z-averaged K-function that is still stability-dependent. As such, we seek to integrate

$$ \overline{K} =  \int_0^1 K\left(\frac{z}{h}\right) \, d\left(\frac{z}{h}\right) $$

which can be written as

$$ \overline{K} = \frac{\kappa u_*}{h} \int_0^1 \frac{x \left(1-x\right)^2}{1+\alpha \frac{h}{L}x} \, dx$$

with $x = z/h$. Expanding the numerator and defining $\alpha' = \alpha h/l$ gives

$$ \overline{K} = \frac{\kappa u_*}{h} \int_0^1 \frac{x^3-2x^2+x}{1+\alpha'x} \, dx$$

The integral in this equation can be solved by successive partial integration:

$$
\begin{align}
 & \int \frac{x^3-2x^2+x}{1+\alpha'x} \, dx \\
&= \frac{(x^3-2x^2+x)}{\alpha'} \ln(1+\alpha'x) - \int (3x^2-4x+1)(1+\alpha'x)^{-1} \,dx \\
&= \frac{(x^3-2x^2+x)}{\alpha'} \ln(1+\alpha'x) - \left[\frac{(3x^2-4x+1)}{\alpha'} \ln(1+\alpha'x) -\int (6x-4)(1+\alpha'x)^{-1} \,dx \right]\\
&= \frac{(x^3-2x^2+x)}{\alpha'} \ln(1+\alpha'x) - \left[\frac{(3x^2-4x+1)}{\alpha'} \ln(1+\alpha'x) - \left[\frac{(6x-4)}{\alpha'} \ln(1+\alpha'x) -\int 6(1+\alpha'x)^{-1} \,dx \right]\right]\\
&= \frac{(x^3-2x^2+x)}{\alpha'} \ln(1+\alpha'x) - \left[\frac{(3x^2-4x+1)}{\alpha'} \ln(1+\alpha'x) - \left[\frac{(6x-4)}{\alpha'} \ln(1+\alpha'x) - \frac{6}{\alpha'} \ln(1+\alpha'x)\right]\right]\\
&= \frac{x^3-5x^2+11x-11}{\alpha'} \ln(1+\alpha'x)
\end{align}
$$

```python
# Import packages
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def kmean(alpha,x):
    return 0.4*0.3*2000.*(x**3-5*x**2+11*x-11)/alpha*np.log(1+alpha*x)

h = 2000.
L = 800.
alpha = 4.7*h/L
z = np.arange(0,1,0.01)
ahh = kmean(alpha,z)

plt.plot(ahh,z)
plt.show

print 'integral value is',0.4*0.3*2000.*kmean(alpha,1.)-0.4*0.3*2000.*kmean(alpha,0.)

```

```python

```


We start by bringing in the Python libraries that we will be using for this code. We will be using Torch, which will enable us to run the code in the GPU for faster processing as compared to the CPU. Torch also assists us during backpropagation, using Autograd that does differentiation and stores the gradients that can be called when required. PyTorch also enables us to create neural networks very easily, and PyTorch together with the Numpy, Pandas and Scipy libraries makes it very powerful for Neural Networks calculations.

## Neural Networks

Neural networks are the building blocks for deep learning. They are made up of neurons or units. Each weighted unit sums up the units of the previous layers and passes the sum through an activation function to get the neuron's output. Some activation functions used in this code are sigmoid, ReLU (Rectified Linear Units), Softmax, tanH (hyperbolic tangent) just to name a few.

A simple neural network would be depicted as shown below:

$$
\begin{align}
y &= f(w_1 x_1 + w_2 x_2 + w_3 x_3 + b) \\
y &= f\left(\sum_i w_i x_i +b \right)
\end{align}
$$

The linear transformation done on each unit is given by the dot product of the input vector and the weight matrix:

$$
h = \begin{bmatrix}
x_1 \, x_2 \cdots  x_n
\end{bmatrix}
\cdot
\begin{bmatrix}
           w_{11}  w_{21}\\
           w_{12}  w_{22}\\
           \vdots \\
           w_{1n}  w_{2n}
\end{bmatrix}
$$

We begin by importing the libraries that we will be using in our code:
* torch - used to bring the torch package which will be used in creating the neural network layers, weihts and biases, as well as the backpropagation using autograd.
* nn - this module will be used for easy neural network creation.
* optim - used to optimize the neural network to adjust the weights to the optimal weights to give us the intended labels.

```python
import torch
from torch import nn, optim
from torch import optim
import numpy as np
```

We're going to build a small neural network with only one example as shown below. Our goal is to build a network that will give the correct output `[0.0,1.0,0.0]`

```python
x = torch.tensor([0.1,0.2,0.7])
x = x.reshape(1,3)
print(x.shape)
print(x)
```

    torch.Size([1, 3])
    tensor([[0.1000, 0.2000, 0.7000]])

Our output will be given by the vector below:

```python
labels = torch.tensor([0,1,0])
labels = labels.reshape(1,3)
labels = labels
print(labels.shape)
print(labels)

```

    torch.Size([1, 3])
    tensor([[0, 1, 0]])

Here is where the neural network is made and all the fun begins! I will go through the code line by line.

```python
class Network(nn.Module):
```

Using the `nn.Module` combined with `__super().__init__()` creates a class that will assist us with tracking of the neural network methods and attributes. The class name can be changed to anything, although it is mandatory to inherit it from the `nn.Module`.

```python
self.hidden = nn.Linear(3,3)
```

This line creates a 3 X 3 matrix for the first hidden layer. `nn.Linear(3,3)` creates a linear transformation $x\mathbf{W} + b$, with 3 inputs and 3 outputs and assigns it to `self.hidden`. This module also creates the weights and biases which will be used in the feedforward using the `forward` method.

The subsequent line, `self.hidden2 = nn.Linear(3,3)` performs the same function as the first hidden layer. Here we generate a 3 X 3 tensor for the second hidden layer. A linear transformation creates the weights and bias tensors and assigns them to `self.hidden2`. This will later be used in the `forward` method.

```python
self.output = nn.Linear(3,3)
```
This creates a linear transformation in the output of size 3 X 3 to be used to generate the output of the neural network. This will be used in the softmax activation function in the output layer.

```python
self.hidden.weight = torch.nn.Parameter(torch.tensor([[0.1,0.3,0.4],[0.2,0.2,0.3],[0.3,0.7,0.9]]))
```

This line of code creates the exact weights that will be used in the code. This was done to enable comparison between the code and the hand calculations already performed.

`self.hidden2.weight and self.output.weight` tensors are also created to explicitly define the tensors for the second hidden layer and the output layer of the neural network.

```python
self.sigmoid = nn.Sigmoid()
self.relu = nn.ReLU()
self.softmax = nn.Softmax(dim=1)
```
These operations are the sigmoid, Rectified Linear Unit (ReLU) and the softmax activation functions. The softmax is used in the output where we will be generating 3 classes for the 3 output units. The `dim=1` is used to ensure that the summation is done across the columns, as opposed to across the rows. The softmax summation should always add to 1 as this is a propability summation.

Some things to note :
* The softmax activation function can ONLY be used in the output layer.
* The softmax function and the Logistic/sigmoid function yield the same results when the number of classes = 2, therefore softmax is a generalization of the sigmoid function.

```python
def forward (self,x):
```

PyTorch neural networks require a `forward` method defined to be able to perform the feedforward step. This is achieved by taking a tensor `x` and taking it through all the operation defined in the `__init__` method.

```python
x = self.hidden(x)

x = self.relu(x)

x = self.hidden2(x)

x = self.sigmoid(x)

x = self.output(x)

x = self.softmax(x)
```

Here we pass the input tensor `x` through the hidden layes, hidden layer 2, the sigmoid the output layer and finally the softmax function. The operations must be sequenced correctly in the forward method to ensure that the operations are done sequentially as expected.

```python
class Network(nn.Module):
    def __init__(self):
        super().__init__()

        # Inputs to hidden layer linear transformation
        self.hidden = nn.Linear(3, 3)
        self.hidden2 = nn.Linear(3, 3)
        self.output = nn.Linear(3, 3)

        self.hidden.weight = torch.nn.Parameter(torch.tensor([[0.1,0.3,0.4],[0.2,0.2,0.3],[0.3,0.7,0.9]]))
        self.hidden2.weight = torch.nn.Parameter(torch.tensor([[0.2,0.3,0.6],[0.3,0.5,0.4],[0.5,0.7,0.8]]))
        self.output.weight = torch.nn.Parameter(torch.tensor([[0.1,0.3,0.5],[0.4,0.7,0.2],[0.8,0.2,0.9]]))

        # Define sigmoid activation, ReLU and softmax output
        self.sigmoid = nn.Sigmoid()
        self.relu = nn.ReLU()
        self.softmax = nn.Softmax(dim=1)

    def forward(self, x):
        # Pass the input tensor through each of our operations
        x = self.hidden(x)

        x = self.relu(x)

        x = self.hidden2(x)

        x = self.sigmoid(x)

        x = self.output(x)

        x = self.softmax(x)

        return x

```

We can now create a `Network` object `model = Network()`

An epoch in Machine Learning is the number of times we want to pass through the same data to optimize the weights to reduce the loss. For this neural network, we will use 50 epochs.

The next thing after the feedforward is the calculation of the loss of the neural network. We will be using the `nn.CrosEntropyloss()` criterion. This criterion is a combination of the `nn.LogSoftmax()` and `nn.NLLLoss`. The NLLLoss stands for the Negative Log Likelihood Loss. It is used when softmax function is being used in the output layer of the neural network. This loss maximizes the likelihood of the correct label while reducing the other classes to 0.

`loss.backward` performs the backpropagation and calculates the gradients in each weight and bias.

PyTorch has a module, autograd that enables us to automatically calculate the gradients of the tensors. We can use this to calculate the gradients with all of our parameters to the loss. Autograd tracks the operations performed during the feedforward, then calculates the gradient dugin backpropagation. Autograd, basically does differentiation, for those who are familiar with math Calculus using chain rule. During backpropagation, it is important to ensure that the gradient calculations are turned on by setting `requires_grad = True` on a tensor. This can be done during the tensor creation with `required_grad` keyword, or any time using `y.required_grad_True)`.

Gradient can be trurned off in a block of code using `torch.no_grad()` or globally on the entire code using `torch.set_grad_enables(True|False)`.

To train the network, we use the [`optim` package](https://pytorch.org/docs/stable/optim.html). For this network, we use the `optim.Adam` package. There are a lot of other packages we can use such as `optim.SGD` where the opimizer used Stochastic gradient descent for optimization. It is critical to zero out the gradient in each epoch using `optimizer.zero_grad()` to ensure that the gradients are not accumulated for each training .For this network, the learning rate, `lr=0.09` was adequate enough to ensure that we got to a global minumum as fast as possible.

```python
model = Network()
epochs = 50
for t in range (1,epochs):
    criterion = nn.CrossEntropyLoss()
    logps = model(x)
    print(labels)
    loss = criterion(logps, torch.max(labels, 1)[1])
    loss.backward()
    optimizer = optim.Adam(model.parameters(), lr=0.09)
    optimizer.zero_grad()
    optimizer.step()
    print(loss)
    print(model.forward(x))
```

    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)
    tensor([[0, 1, 0]])
    tensor(1.2714, grad_fn=<NllLossBackward>)
    tensor([[0.2725, 0.1738, 0.5537]], grad_fn=<SoftmaxBackward>)

Below I take you through a quick example showing how autograd calculates and performs backpropagation.

```python
a = torch.randn(2,2, requires_grad=True)
print(a)
```

    tensor([[-0.3680, -1.1311],
            [ 0.1723, -1.1959]], requires_grad=True)

```python
y = a**3
print(y)
```

    tensor([[-0.0498, -1.4470],
            [ 0.0051, -1.7103]], grad_fn=<PowBackward0>)

Here, we see that `y` is created with the a power operation `PowBackward0`

```python
m = y**2
print(m)
```

    tensor([[2.4846e-03, 2.0937e+00],
            [2.6132e-05, 2.9250e+00]], grad_fn=<PowBackward0>)

The `y` then goes through another power operation where it is squared.

```python
print(m.grad_fn)
```

    <PowBackward0 object at 0x12027d990>

Autograd tracks all the operations and calculates the gradients for each operation during backpropagation. Next we take a mean operation to get a scalar value from m.

```python
z = m.mean()
print(z)
```

    tensor(0.1634, grad_fn=<MeanBackward0>)

```python
print(a.grad)
#print((a**2)/3)
```

    None

We can see that there are no gradients since no backpropagation has been done at this point.The gradient after the `z.backward()` results in exactly the same results from the math differentiation with respect to a:
$$
\frac{\partial z}{\partial a} = \frac{\partial}{\partial a}\left[\frac{1}{n}\sum_i^n (a_i^3)^2\right] = \frac{3a^5}{2}
$$

```python
z.backward()
print(a.grad)
print((3*(a**5))/2)
```

    tensor([[ 1.6507e-04,  4.4403e-01],
            [-3.4622e-03,  7.2903e-01]])
    tensor([[ 1.6507e-04,  4.4403e-01],
            [-3.4622e-03,  7.2903e-01]], grad_fn=<DivBackward0>)

The results from the autograd's `z.backward()`yields the same result as the differentiation using chain rule from Calculus.