# Inverse dynamics (2D) for gait analysis

> Marcos Duarte, Renato Naville Watanabe
> [Laboratory of Biomechanics and Motor Control](http://pesquisa.ufabc.edu.br/bmclab/)
> Federal University of ABC, Brazil

Contents

1  Python setup

2  Forward and inverse dynamics

3  Estimation of joint force and moments of force by inverse dynamics

3.1  Free body diagrams

3.2  Equations of motion

3.3  The recursive approach for inverse dynamics of multi-body systems

3.4  Python function invdyn2d.py

3.5  Experimental data

3.6  Load data file

2.7  Data filtering

3.8  Data selection

3.9  Plot file data

3.10  Body-segment parameters

3.11  Kinematic calculations

3.12  Plot joint angles

3.13  Inverse dynamics calculations

3.14  Load files with true joint forces and moments of force

3.15  Plot calculated variables and their true values

4  Contribution of each term to the joint force and moment of force

4.1  Quasi-static analysis

4.2  Neglecting the acceleration and mass (weight) of the segments

4.3  WARNING: the calculated resultant joint force is not the actual joint reaction force!

5  Conclusion

6  Further reading

7  Video lectures on the Internet

8  Problems

9  References

## Python setup

```python
# import the necessary libraries
import numpy as np
from numpy import cross
from IPython.display import IFrame
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_context('notebook', font_scale=1.2, rc={"lines.linewidth": 2})
import sys
sys.path.insert(1, r'./../functions')
```

## Forward and inverse dynamics

With respect to the equations of motion to determine the dynamics of a system, there are two general approaches: forward (or direct) and inverse dynamics. For example, consider the solution of Newton's second law for a particle. If we know the force(s) and want to find the trajectory, this is **forward dynamics**. If instead, we know the trajectory and want to find the force(s), this is **inverse dynamics**:

Figure. The equation of motion and the forward and inverse dynamics approaches.

In Biomechanics, in a typical movement analysis of the human body using inverse dynamics, we would measure the positions of the segments and measure the external forces, calculate the segments' linear and angular acceleration, and find the internal net force and moment of force at the joint using the equations of motion. In addition, we could estimate the muscle forces (if we solve the redundancy problem of having more muscles than joints).
Using forward dynamics, the muscle forces would be the inputs and the trajectories of the segments would be the outputs. The figure below compares the forward and inverse dynamics approaches.

Figure. Inverse dynamics and Forward (or Direct) dynamics approaches for movement analysis (adapted from Zajac and Gordon, 1989).

## Estimation of joint force and moments of force by inverse dynamics

Let's estimate the joint force and moments of force at the lower limb during locomotion using the inverse dynamics approach.
We will model the lower limbs at the right side as composed by three rigid bodies (foot, leg, and thigh) articulated by three hinge joints (ankle, knee, and hip) and perform a two-dimensional analysis.

### Free body diagrams

The [free body diagrams](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/FreeBodyDiagram.ipynb) of the lower limbs are:

Figure. Free body diagrams of the lower limbs for a gait analysis. GRF is the resultant ground reaction force applied on the foot at the center of pressure, COP, position.

### Equations of motion

The equilibrium equations for the forces and moments of force around the center of mass are:

For body 1 (foot):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_1 + m_1\mathbf{g} + \mathbf{GRF} = m_1\mathbf{a}_1 \\
\mathbf{M}_1 + \mathbf{r}_{cmp1}\times\mathbf{F}_1 + \mathbf{r}_{cmCOP}\times\mathbf{GRF} = I_1\mathbf{\alpha}_1
\label{}
\end{array}
\end{equation}

For body 2 (leg):

\begin{equation}
\begin{array}{l}
\mathbf{F}_2 + m_2\mathbf{g} - \mathbf{F}_1 = m_2\mathbf{a}_2 \\
\mathbf{M}_2 + \mathbf{r}_{cmp2}\times\mathbf{F}_2 + \mathbf{r}_{cmd2}\times-\mathbf{F}_{1} - \mathbf{M}_1 = I_2\mathbf{\alpha}_2
\label{}
\end{array}
\end{equation}

For body 3 (thigh):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_3 + m_3\mathbf{g} - \mathbf{F}_2 = m_3\mathbf{a}_3 \\
\mathbf{M}_3 + \mathbf{r}_{cmp3}\times\mathbf{F}_3 + \mathbf{r}_{cmd3}\times-\mathbf{F}_{2} - \mathbf{M}_2 = I_3\mathbf{\alpha}_3
\label{}
\end{array}
\end{equation}

Where $p$ and $d$ stands for proximal and distal joints (with respect to the fixed extremity), $\mathbf{r}_{cmji}$ is the position vector from the center of mass of body $i$ to the joint $j$, $COP$ is the center of pressure, the position of application of the resultant ground reaction force (GRF), $\mathbf{\alpha}$ is the angular acceleration, and $g$ is the acceleration of gravity

Note that the pattern of the equations is the same for the three segments: distal and proximal forces and moments of force and the weight force are present in all segments.
The only exception is with the foot in contact with the ground. As the ground only pushes the foot, it can not generate a moment of force over the foot. Because of that we model the interaction foot-ground as a resultant ground reaction force (GRF) applied on the foot at the COP position.

Both GRF and COP quantities are measured with a force platform and are assumed as known quantities.
Because of that the system of equations above is only solvable if we start by the body 1, from bottom to top.

The system of equations above is simple and straightforward to solve, it is just a matter of being systematic.
We start by segment 1, find $\mathbf{F}_1$ and $\mathbf{M}_1$, substitute these values on the equations for segment 2, find $\mathbf{F}_2$ and $\mathbf{M}_2$, substitute them in the equations for segment 3 and find $\mathbf{F}_3$ and $\mathbf{M}_3\:$:

For body 1 (foot):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_1 &=& m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF} \\
\mathbf{M}_1 &=& I_1\mathbf{\alpha}_1 - \mathbf{r}_{cmp1}\times\big(m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF}\big) - \mathbf{r}_{cmCOP}\times\mathbf{GRF}
\label{}
\end{array}
\end{equation}

For body 2 (leg):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_2 &=& m_1\mathbf{a}_1 + m_2\mathbf{a}_2 - (m_1+m_2)\mathbf{g} - \mathbf{GRF} \\
\mathbf{M}_2 &=& I_1\mathbf{\alpha}_1 + I_2\mathbf{\alpha}_2 - \mathbf{r}_{cmp2}\times\big(m_1\mathbf{a}_1 + m_2\mathbf{a}_2 - (m_1+m_2)\mathbf{g} - \mathbf{GRF}\big)\, + \\
&\phantom{=}& \mathbf{r}_{cgd2}\times\big(m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF}\big) - \mathbf{r}_{cmp1}\times\big(m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF}\big)\, - \\
&\phantom{=}& \mathbf{r}_{cmCOP}\times\mathbf{GRF}
\label{}
\end{array}
\end{equation}

For body 3 (thigh):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_3 &=& m_1\mathbf{a}_1 + m_2\mathbf{a}_2 + m_3\mathbf{a}_3 - (m_1+m_2+m_3)\mathbf{g} - \mathbf{GRF} \\
\mathbf{M}_3 &=& I_1\mathbf{\alpha}_1 + I_2\mathbf{\alpha}_2 + I_3\mathbf{\alpha}_3\, - \\
&\phantom{=}& \mathbf{r}_{cmp3}\times\big(m_1\mathbf{a}_1 + m_2\mathbf{a}_2 + m_3\mathbf{a}_3\, - (m_1+m_2+m_3)\mathbf{g} - \mathbf{GRF}\big)\, + \\
&\phantom{=}& \mathbf{r}_{cmd3}\times\big(m_1\mathbf{a}_1 + m_2\mathbf{a}_2 - (m_1+m_2)\mathbf{g} - \mathbf{GRF}\big)\, - \\
&\phantom{=}& \mathbf{r}_{cmp2}\times\big(m_1\mathbf{a}_1 + m_2\mathbf{a}_2 - (m_1+m_2)\mathbf{g} - \mathbf{GRF}\big)\, + \\
&\phantom{=}& \mathbf{r}_{cmd2}\times\big(m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF}\big)\, - \\
&\phantom{=}& \mathbf{r}_{cmp1}\times\big(m_1\mathbf{a}_1 - m_1\mathbf{g} - \mathbf{GRF}\big)\, - \\
&\phantom{=}& \mathbf{r}_{cmCOP}\times\mathbf{GRF}
\label{}
\end{array}
\end{equation}

### The recursive approach for inverse dynamics of multi-body systems

The calculation above is tedious, error prone, useless, and probably it's wrong.

To make some use of it, we can clearly see that forces act on far segments, which are not directly in contact with these forces. In fact, this is true for all stuff happening on a segment: note that $\mathbf{F}_1$ and $\mathbf{M}_1$ are present in the expression for $\mathbf{F}_3$ and $\mathbf{M}_3$ and that the acceleration of segment 1 matters for the calculations of segment 3.

Instead, we can use the power of computer programming (like this one right now!) and solve these equations recursively hence they have the same pattern. Let's do that.

For body 1 (foot):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_1 = m_1\mathbf{a}_1 - m_1\mathbf{g} -  \mathbf{GRF} \\
\mathbf{M}_1 = I_1\mathbf{\alpha}_1 - \mathbf{r}_{cmp1}\times\mathbf{F}_1 - \mathbf{r}_{cmCOP}\times\mathbf{GRF}
\label{}
\end{array}
\end{equation}

For body 2 (leg):

\begin{equation}
\begin{array}{l}
\mathbf{F}_2  = m_2\mathbf{a}_2 -  m_2\mathbf{g} + \mathbf{F}_1\\
\mathbf{M}_2 = I_2\mathbf{\alpha}_2 - \mathbf{r}_{cmp2}\times\mathbf{F}_2 +\mathbf{r}_{cmd2}\times\mathbf{F}_{1} + \mathbf{M}_1
\label{}
\end{array}
\end{equation}

For body 3 (thigh):

\begin{equation}
\begin{array}{l l}
\mathbf{F}_3  = m_3\mathbf{a}_3 - m_3\mathbf{g} + \mathbf{F}_2\\
\mathbf{M}_3  = I_3\mathbf{\alpha}_3 - \mathbf{r}_{cmp3}\times\mathbf{F}_3 + \mathbf{r}_{cmd3}\times\mathbf{F}_{2} + \mathbf{M}_2
\label{}
\end{array}
\end{equation}

### Python function `invdyn2d.py`

We could write a function that it would have as inputs the body-segment parameters, the kinematic data, and the distal joint force and moment of force and output the proximal joint force and moment of force.
Then, we would call this function for each segment, starting with the segment that has a free extremity or that has the force and moment of force measured by some instrument (i,e, use a force plate for the foot-ground interface).
This function would be called in the following manner:

```python
    Fp, Mp = invdyn2d(rcm, rd, rp, acm, alfa, mass, Icm, Fd, Md)
```

So, here is such function:

```python
# %load ./../functions/invdyn2d.py
"""Two-dimensional inverse-dynamics calculations of one segment."""

__author__ = 'Marcos Duarte, https://github.com/demotu/BMC'
__version__ = 'invdyn2d.py v.2 2015/11/13'

def invdyn2d(rcm, rd, rp, acm, alpha, mass, Icm, Fd, Md):
    """Two-dimensional inverse-dynamics calculations of one segment

    Parameters
    ----------
    rcm   : array_like [x,y]
            center of mass position (y is vertical)
    rd    : array_like [x,y]
            distal joint position
    rp    : array_like [x,y]
            proximal joint position
    acm   : array_like [x,y]
            center of mass acceleration
    alpha : array_like [x,y]
            segment angular acceleration
    mass  : number
            mass of the segment
    Icm   : number
            rotational inertia around the center of mass of the segment
    Fd    : array_like [x,y]
            force on the distal joint of the segment
    Md    : array_like [x,y]
            moment of force on the distal joint of the segment

    Returns
    -------
    Fp    : array_like [x,y]
            force on the proximal joint of the segment (y is vertical)
    Mp    : array_like [x,y]
            moment of force on the proximal joint of the segment

    Notes
    -----
    To use this function recursevely, the outputs [Fp, Mp] must be inputed as
    [-Fp, -Mp] on the next call to represent [Fd, Md] on the distal joint of the
    next segment (action-reaction).

    This code was inspired by a similar code written by Ton van den Bogert [1]_.
    See this notebook [2]_.

    References
    ----------
    .. [1] http://isbweb.org/data/invdyn/index.html
    .. [2] http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/GaitAnalysis2D.ipynb

    """

    g = 9.80665  # m/s2, standard acceleration of free fall (ISO 80000-3:2006)
    # Force and moment of force on the proximal joint
    Fp = mass*acm - Fd - [0, -g*mass]
    Mp = Icm*alpha - Md - cross(rd-rcm, Fd) - cross(rp-rcm, Fp)

    return Fp, Mp
```

The inverse dynamics calculations are implemented in only two lines of code at the end, the first part of the code is the help on how to use the function. The help is long because it's supposed to be helpful :), see the [style guide for NumPy/SciPy documentation](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard).

The real problem is to measure or estimate the experimental variables: the body-segment parameters, the ground reaction forces, and the kinematics of each segment. For such, it is necessary some expensive equipments, but they are typical in a biomechanics laboratory, such the the [BMClab](http://pesquisa.ufabc.edu.br/bmclab).

### Experimental data

Let's work with some data of kinematic position of the segments and ground reaction forces in order to compute the joint forces and moments of force.
The data we will work are in fact from a computer simulation of running created by Ton van den Bogert. The nice thing about these data is that as a simulation, the true joint forces and moments of force are known and we will be able to compare our estimation with these true values.
All the data can be downloaded from a page at the [ISB website](http://isbweb.org/data/invdyn/index.html):

```python
IFrame('http://isbweb.org/data/invdyn/index.html', width='100%', height=400)
```

### Load data file

```python
# load file with ground reaction force data
grf = np.loadtxt('./../data/all.frc') # [Fx, Fy, COPx]
# load file with kinematic data
kin = np.loadtxt('./../data/all.kin') # [Hip(x,y), knee(x,y), ankle(x,y), toe(x,y)]
freq = 10000
time = np.linspace(0, grf.shape[0]/freq, grf.shape[0])
```

### Data filtering

```python
# this is simulated data with no noise, filtering doesn't matter
if False:
    # filter data
    from scipy.signal import butter, filtfilt
    # Butterworth filter
    b, a = butter(2, (10/(freq/2)))
    for col in np.arange(grf.shape[1]-1):
        grf[:, col]  = filtfilt(b, a, grf[:, col])
    b, a = butter(2, (10/(freq/2)))
    for col in np.arange(kin.shape[1]):
        kin[:, col] = filtfilt(b, a, kin[:, col])
```

### Data selection

```python
# heel strike occurs at sample 3001
time = time[3001 - int(freq/40):-int(freq/20)]
grf  = grf[3001 - int(freq/40):-int(freq/20), :]
kin  = kin[3001 - int(freq/40):-int(freq/20), :]
```

### Plot file data

```python
# plot data
hfig, hax = plt.subplots(2, 2, sharex = True, squeeze=True, figsize=(9, 5))
hax[0, 0].plot(time, grf[:, [0, 1]], linewidth=2)
hax[0, 0].legend(('Fx','Fy'), frameon=False)
hax[0, 0].set_ylabel('Force [N]')
hax[0, 1].plot(time, grf[:, 2], linewidth=2)
hax[0, 1].legend(['COPx'], frameon=False)
hax[0, 1].set_ylabel('Amplitude [m]')
hax[1, 0].plot(time, kin[:, 0::2], linewidth=2)
hax[1, 0].legend(('Hip x','Knee x','Ankle x','Toe x'), frameon=False)
hax[1, 0].set_ylabel('Amplitude [m]')
hax[1, 1].plot(time, kin[:, 1::2], linewidth=2)
hax[1, 1].legend(('Hip y','Knee y','Ankle y','Toe y'), frameon=False)
hax[1, 1].set_ylabel('Amplitude [m]')
hax[1, 0].set_xlabel('Time [s]'), hax[1, 1].set_xlabel('Time [s]')
plt.tight_layout()
plt.show()
```

### Body-segment parameters

```python
# body-segment parameters [thigh, shank, foot]
mass = [6.85, 2.86, 1.00]                 # mass [kg]
Icm  = [0.145361267, 0.042996389, 0.0200] # rotational inertia [kgm2]
cmpr = [0.4323725, 0.4334975, 0.0]        # CM [m] wrt. prox. joint [frac. segment len]

```

### Kinematic calculations

```python
# Kinematic data
# center of mass position of the thigh, shank, foot segments
rcm = np.hstack((kin[:, [0,1]] + cmpr[0]*(kin[:, [2,3]] - kin[:, [0,1]]),
                 kin[:, [2,3]] + cmpr[1]*(kin[:, [4,5]] - kin[:, [2,3]]),
                 kin[:, [4,5]] + cmpr[2]*(kin[:, [6,7]] - kin[:, [4,5]])))

# center of mass linear acceleration of the thigh, shank, foot segments
acm = np.diff(rcm, n=2, axis=0)*freq*freq
acm = np.vstack((acm, acm[-1, :], acm[-1, :]))

# thigh, shank, foot segment angle
ang = np.vstack((np.arctan2(kin[:, 1] - kin[:, 3], kin[:, 0] - kin[:, 2]),
                 np.arctan2(kin[:, 3] - kin[:, 5], kin[:, 2] - kin[:, 4]),
                 np.arctan2(kin[:, 5] - kin[:, 7], kin[:, 4] - kin[:, 6]))).T

# hip, knee, and ankle joint angles
angj = np.vstack((-(ang[:, 0]-ang[:, 1]),
                  np.unwrap(ang[:, 1] - ang[:, 2] + np.pi/2))).T*180/np.pi

# thigh, shank, foot segment angular acceleration
aang = np.diff(ang, n=2, axis=0)*freq*freq
aang = np.vstack((aang, aang[-1, :], aang[-1, :]))
```

### Plot joint angles

```python
# plot hip, knee, and ankle joint angles
hfig, (hax1, hax2) = plt.subplots(2, 1, sharex = True, squeeze=True, figsize=(10, 5))
hax1.plot(time, angj[:, 0], linewidth=2, label='Knee')
hax1.legend(frameon=False, loc='upper left'), hax1.grid()
hax2.plot(time, angj[:, 1], linewidth=2, label='Ankle')
hax2.legend(frameon=False, loc='upper left'), hax2.grid()
hax1.set_ylabel('Joint angle $[^o]$')
hax2.set_ylabel('Joint angle $[^o]$')
hax2.set_xlabel('Time [s]')
plt.tight_layout()
plt.show()
```

### Inverse dynamics calculations

```python
# inverse dynamics
# invdyn2d(rcm, rd, rp, acm, alpha, mass, Icm, Fd, Md)
from invdyn2d import invdyn2d

# ankle
Fa, Ma = invdyn2d(rcm[:,(4,5)], grf[:,(2,2)]*[1,0], kin[:,(4,5)],
                    acm[:,(4,5)], aang[:, 2], mass[2], Icm[2],
                    grf[:, (0, 1)], 0)
# knee
Fk, Mk = invdyn2d(rcm[:,(2,3)], kin[:,(4,5)], kin[:,(2,3)],
                    acm[:,(2,3)], aang[:,1], mass[1], Icm[1],
                    -Fa, -Ma)
# hip
Fh, Mh = invdyn2d(rcm[:,(0,1)], kin[:,(2,3)], kin[:,(0,1)],
                    acm[:,(0,1)], aang[:,0], mass[0], Icm[0],
                    -Fk, -Mk)

# magnitude of the calculated hip, knee, and ankle resultant joint force
Fam = np.sqrt(np.sum(np.abs(Fa)**2, axis=-1))
Fkm = np.sqrt(np.sum(np.abs(Fk)**2, axis=-1))
Fhm = np.sqrt(np.sum(np.abs(Fh)**2, axis=-1))
```

### Load files with true joint forces and moments of force

```python
# load file with true joint forces and moments of force
forces  = np.loadtxt('./../data/all.fmg') # [Hip, knee, ankle]
moments = np.loadtxt('./../data/all.mom') # [Hip, knee, ankle]
#heel strike occurs at sample 3001
forces  = forces[3001-int(freq/40):-int(freq/20), :]
moments = moments[3001-int(freq/40):-int(freq/20), :]
```

### Plot calculated variables and their true values

Let's plot these data but because later we will need to plot similar plots, let's create a function for the plot to avoid repetition of code:

```python
def plotdata(time, Fh, Fk, Fa, Mh, Mk, Ma, forces, moments, stitle):
    # plot hip, knee, and ankle moments of force
    hfig, hax = plt.subplots(3, 2, sharex = True, squeeze=True, figsize=(11, 6))
    # forces
    hax[0, 0].plot(time, Fh, label='invdyn'), hax[0, 0].set_title('Hip')
    hax[1, 0].plot(time, Fk), hax[1, 0].set_title('Knee')
    hax[2, 0].plot(time, Fa), hax[2, 0].set_title('Ankle')
    hax[1, 0].set_ylabel('Joint force [N]')
    hax[2, 0].set_xlabel('Time [s]')
    # moments of force
    hax[0, 1].plot(time, Mh), hax[0, 1].set_title('Hip')
    hax[1, 1].plot(time, Mk), hax[1, 1].set_title('Knee')
    hax[2, 1].plot(time, Ma), hax[2, 1].set_title('Ankle')
    hax[1, 1].set_ylabel('Moment of Force [Nm]')
    hax[2, 1].set_xlabel('Time [s]')
    # true joint forces and moments of force
    hax[0, 0].plot(time,  forces[:, 0], 'r--', label='True')
    hax[0, 0].legend(frameon=False)
    hax[1, 0].plot(time,  forces[:, 1], 'r--')
    hax[2, 0].plot(time,  forces[:, 2], 'r--')
    hax[0, 1].plot(time, moments[:, 0], 'r--')
    hax[1, 1].plot(time, moments[:, 1], 'r--')
    hax[2, 1].plot(time, moments[:, 2], 'r--')
    plt.suptitle(stitle, fontsize=16)
    for x in hax.flat:
        x.locator_params(nbins=5); x.grid()
    plt.show()

plotdata(time, Fhm, Fkm, Fam, Mh, Mk, Ma, forces, moments,
         'Inverse dynamics: estimated versus true values')
```

The results are very similar; only a small part of the moments of force is different because of some noise.

## Contribution of each term to the joint force and moment of force

Let's see what happens with the joint forces and moments of force when we neglect the contribution of some terms in the inverse dynamics analysis of these data.

### Quasi-static analysis
Consider the case where the segment acceleration is neglected:

```python
# ankle
[Fast, Mast] = invdyn2d(rcm[:,(4,5)], grf[:,(2,2)]*[1,0], kin[:,(4,5)],
                        acm[:,(4,5)]*0, aang[:,2]*0, mass[2], Icm[2],
                        grf[:,(0,1)], 0)
# knee
[Fkst, Mkst] = invdyn2d(rcm[:,(2,3)], kin[:,(4,5)], kin[:,(2,3)],
                        acm[:,(2,3)]*0, aang[:,1]*0, mass[1], Icm[1],
                        -Fast, -Mast)
# hip
[Fhst, Mhst] = invdyn2d(rcm[:,(0,1)], kin[:,(2,3)], kin[:,(0,1)],
                        acm[:,(0,1)]*0, aang[:,0]*0, mass[0], Icm[0],
                        -Fkst, -Mkst)

# magnitude of the calculated hip, knee, and ankle resultant joint force
Fastm = np.sqrt(np.sum(np.abs(Fast)**2, axis=-1))
Fkstm = np.sqrt(np.sum(np.abs(Fkst)**2, axis=-1))
Fhstm = np.sqrt(np.sum(np.abs(Fhst)**2, axis=-1))

plotdata(time, Fhstm, Fkstm, Fastm, Mhst, Mkst, Mast, forces, moments,
         'Inverse dynamics: quasis-static approach versus true values')
```

This is not a pure static analysis because part of the ground reaction forces still reflects the body accelerations (were the body completely static, the ground reaction force should be equal to the body weight in magnitude).

### Neglecting the acceleration and mass (weight) of the segments

Consider the case where besides the acceleration, the body-segment parameters are also neglected.
This means that the joint loads are due only to the ground reaction forces (which implicitly include contributions due to the acceleration and the body-segment weights).

```python
# ankle
[Fagrf, Magrf] = invdyn2d(rcm[:, (4,5)], grf[:,(2,2)]*[1,0], kin[:,(4,5)],
                          acm[:,(4,5)]*0, aang[:,2]*0, 0, 0, grf[:,(0,1)], 0)
# knee
[Fkgrf, Mkgrf] = invdyn2d(rcm[:,(2,3)], kin[:,(4,5)], kin[:,(2,3)],
                          acm[:,(2,3)]*0, aang[:,1]*0, 0, 0, -Fagrf, -Magrf)
# hip
[Fhgrf, Mhgrf] = invdyn2d(rcm[:,(0,1)], kin[:,(2,3)], kin[:,(0,1)],
                          acm[:,(0,1)]*0, aang[:, 0]*0, 0, 0, -Fkgrf, -Mkgrf)

# magnitude of the calculated hip, knee, and ankle resultant joint force
Fagrfm = np.sqrt(np.sum(np.abs(Fagrf)**2, axis=-1))
Fkgrfm = np.sqrt(np.sum(np.abs(Fkgrf)**2, axis=-1))
Fhgrfm = np.sqrt(np.sum(np.abs(Fhgrf)**2, axis=-1))

plotdata(time, Fhgrfm, Fkgrfm, Fagrfm, Mhgrf, Mkgrf, Magrf, forces, moments,
         'Inverse dynamics: ground-reaction-force approach versus true values')
```

Neglecting all the accelerations and the weight of the segments means that the only external force that actuates on the system is the ground reaction force, which although is only actuating at the foot-ground interface it will be transmitted to the other segments through the joint forces.  Because of that, the joint forces on the ankle, knee, and hip will simply be minus the ground reaction force. Note that the forces shown above for the three joints are the same and equal to:

\begin{equation}
\begin{array}{l}
\sqrt{GRF_x^2+GRF_y^2}
\label{}
\end{array}
\end{equation}

These simplifications also mean that the moments of force could have been simply calculated as the cross product between the vector position of the the COP in relation to the joint and the GRF vector:

\begin{equation}
\begin{array}{l}
\mathbf{M_{a}} = -\mathbf{cross}(\mathbf{COP}-\mathbf{r_{a}},\,\mathbf{GRF}) \\
\mathbf{M_{k}} = -\mathbf{cross}(\mathbf{COP}-\mathbf{r_{k}},\,\mathbf{GRF}) \\
\mathbf{M_{h}} = -\mathbf{cross}(\mathbf{COP}-\mathbf{r_{h}},\,\mathbf{GRF})
\label{}
\end{array}
\end{equation}

Where $\mathbf{r_{i}}\;$ is the position vector of joint $i$.

Let's calculate the variables in this way:

```python
Fhgrfm2 = Fkgrfm2 = Fagrfm2 = np.sqrt(np.sum(np.abs(-grf[:,(0,1)])**2, axis=-1))
Magrf2  = -np.cross(grf[:,(2,2)]*[1,0]-kin[:,(4,5)], grf[:,(0,1)])
Mkgrf2  = -np.cross(grf[:,(2,2)]*[1,0]-kin[:,(2,3)], grf[:,(0,1)])
Mhgrf2  = -np.cross(grf[:,(2,2)]*[1,0]-kin[:,(0,1)], grf[:,(0,1)])

plotdata(time, Fhgrfm2, Fkgrfm2, Fagrfm2, Mhgrf2, Mkgrf2, Magrf2, forces, moments,
         'Inverse dynamics: ground-reaction-force approach versus true values II')
```

### WARNING: the calculated resultant joint force is not the actual joint reaction force!

In the Newton-Euler equations based on the free body diagrams we represented the consequences of all possible muscle forces on a joint as a net muscle torque and all forces acting on a joint as a resultant joint reaction force. That is, all forces between segments were represented as a resultant force that doesn't generate torque and a moment of force that only generates torque.
This is an important principle in mechanics of rigid bodies as we saw before.
However, this principle creates the unrealistic notion that the sum of forces is applied directly on the joint (which has no further implication for a rigid body), but it is inaccurate for the understanding of the local effects on the joint. So, if we are trying to understand the stress on the joint or mechanisms of joint injury, the forces acting on the joint and on the rest of the segment must be considered individually.

## Conclusion

For these data set of 'running' (remember this is simulated data), in the estimation of the forces and moments of force at the hip, knee, and ankle joints in a two-dimensional analysis, to not consider the segment acceleration and/or the mass of the segments had no effect on the ankle variables, a small effect on the knee, and a large effect on the hip.
This is not surprising; during the support phase, ankle and knee have small movements and the mass of the segments only start to have a significant contribution for more proximal and heavy segments such as the thigh.

Don't get disappointed thinking that all this work for drawing the complete FBDs and their correspondent equations was a waste of time.
Nowadays, the state of the art and the demand for higher accuracy in biomechanics is such that such simplifications are usually not accepted.

## Further reading

- [Gait Analysis on Wikipedia](https://en.wikipedia.org/wiki/Gait_analysis);
- [Gait analysis: clinical facts](https://www.ncbi.nlm.nih.gov/pubmed/27618499);
- [Gait Analysis Methods: An Overview of Wearable and Non-Wearable Systems, Highlighting Clinical Applications](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3958266/);
- [Avaliação Biomecânica da Corrida no BMClab (in Portuguese)](http://pesquisa.ufabc.edu.br/bmclab/servicos/rba-2/).

## Video lectures on the Internet

- Understanding & Analyzing Gait For The Clinician - series: [Introduction](https://youtu.be/x1JoaGgyKX0), [Patient Assessment](https://youtu.be/Z0QNkLshQUk), [Intro To Computer-Based 3-D Analysis](https://youtu.be/g0OcCLTQM_Y), [Basic Musculoskeletal Biomechanics](https://youtu.be/KsdrmyxOyxM), [The Gait Cycle](https://youtu.be/96nLX6sm9Yw);
- [How to benefit from a Gait Analysis | Runners Need](https://youtu.be/rxkX7qGtIEI).

## Problems

1. Search the Internet for actual experimental data from a gait analysis of a runner and compare with the simulated data used in this notebook.
2. Collect or search for some experimental data from a movement analysis and perform inverse dynamics to determine joint forces and torques.
3. Imagine that you have to perform a similar analysis but of the upper limb during throwing a ball. What would have to change in the approach described in this notebook?

## References

- Ruina A, Rudra P (2019) [Introduction to Statics and Dynamics](http://ruina.tam.cornell.edu/Book/index.html). Oxford University Press.
- Winter DA (2009) [Biomechanics and motor control of human movement](http://books.google.com.br/books?id=_bFHL08IWfwC). 4 ed. Hoboken, EUA: Wiley.
- Zajac FE, Gordon ME (1989) [Determining muscle's force and action in multi-articular movement](https://github.com/BMClab/BMC/blob/master/refs/zajac89.pdf). Exercise and Sport Sciences Reviews, 17, 187-230.
- Zatsiorsky VM (2202) [Kinetics of human motion](http://books.google.com.br/books?id=wp3zt7oF8a0C&lpg=PA571&ots=Kjc17DAl19&dq=ZATSIORSKY%2C%20Vladimir%20M.%20Kinetics%20of%20human%20motion&hl=pt-BR&pg=PP1#v=onepage&q&f=false). Champaign, IL: Human Kinetics.


## The GIBBON (General-purpose Information-Based Bayesian OptimisatioN) acquisition function

A particularly intuitive and empirically effective class of acquisition functions has arisen based on information theory. Information-theoretic Bayesian Optimisation (BO) seeks to reduce uncertainty in the location of high-performing areas of the search space, as measured in terms of differential entropy. BoTorch already supports information-theoretic BO through an implementation of the Max-value Entropy Search (MES) acquisition function [1] (see the [Max-Value Entropy tutorial](./max_value_entropy) for details), which makes evaluations that reduce uncertainty in the maximum value attained by the objective function. However, in order to support batch and multi-fidelity BO, our implementation of MES employs numerical integrations and fantasy observations  (i. e., we generate one point each time and when we try to generate the 𝑖-th point of a batch, we condition the models on the 𝑖−1 points generated prior to this). Unfortunately,  Each of these calculations can can add significantly to the computational overhead incurred by BO.

In this notebook, we provide an information-theoretic acquisition function for tasks where objective function query costs are not large enough to overshadow significant optimisation overheads known as General-purpose Information-Based Bayesian OptimisatioN (GIBBON) [2]. In this tutorial, we present a very high-level overview of GIBBON and demonstrate its use within BoTorch.

### Calculating GIBBON

Following a principled information-theoretic construction, the GIBBON acquisition function measures the utility of evaluating a candidate batch of $B$ points $\{\textbf{x}\}_{i=1}^B$ as
\begin{align}
    \alpha_{\text{GIBBON}}(\{\textbf{x}\}_{i=1}^B)
    &= \frac{1}{2}\log |C| + \sum_{i=1}^B  \hat{\alpha}_{\text{MES}}(\textbf{x}_i)
\end{align}
where $|C|$ is the determinant of the $B\times B$ correlation matrix between the batch elements and $\hat{\alpha}_{\text{MES}}$ is an analytical approximation of the standard (non-batch) MES acquisition function. The GIBBON acquisition function forms a lower bound on the exact (but intractable) batch MES function and is consequently referred to as the `qLowerBoundMaxValueEntropy` in BoTorch. Crucially, GIBBON can be computed in closed-form and so incurs substantially lower overheads than batch MES via fantasies.

### Interpretating GIBBON
Note that the above decomposition of GIBBON has two terms and each has a helpful intuitive justification. In particular, the first term encourages diversity within the batch (achieving high values for points with low predictive correlation), whereas the second term ensures that evaluations are targeted in areas of the search space that provide large amounts of information about the maximum value attained by the objective function.

<br>
__References__

[1] [Wang, Z., Jegelka, S., _Max-value Entropy Search for Efficient Bayesian Optimization._ arXiv:1703.01968v3, 2018](https://arxiv.org/abs/1703.01968)

[2] [Moss, M., et al., _GIBBON: General-purpose Information-Based Bayesian Optimisation._  arXiv:2102.03324, 2020](https://arxiv.org/abs/2102.03324)

```python
import os
SMOKE_TEST = os.environ.get("SMOKE_TEST")
```

### 1. Setting up a toy model
We will fit a standard SingleTaskGP model on noisy observations of the synthetic 2D SixHumpCamel function.

```python
import math
import torch

from botorch.test_functions import SixHumpCamel
from botorch.fit import fit_gpytorch_model
from botorch.models import SingleTaskGP
from botorch.utils.transforms import standardize, normalize
from gpytorch.mlls import ExactMarginalLogLikelihood

torch.manual_seed(123456)

bounds = torch.tensor(SixHumpCamel._bounds).T
bounds_norm = torch.tensor([[0.,0.],[1.,1.]])
train_X = bounds[0] + (bounds[1] - bounds[0]) * torch.rand(5, 2)
train_Y = SixHumpCamel(negate=True)(train_X).unsqueeze(-1)

train_X = normalize(train_X, bounds=bounds)
train_Y = standardize(train_Y + 0.05 * torch.randn_like(train_Y))

model = SingleTaskGP(train_X, train_Y)
mll = ExactMarginalLogLikelihood(model.likelihood, model)
fit_gpytorch_model(mll, max_retries=10);
```

### 2. Defining the GIBBON acquisition function

GIBBON is implemented in BoTorch as `qLowerBoundMaxValueEntropy` and supports pending points through its `X_pending` argument. Required arguments for the constructor are `model` and `candidate_set` (the discretized candidate points in the design space that will be used to draw max value samples). There are also other optional parameters, such as number of max value samples. Just like in our implementation of MES, two different sampling algorithms are supported for the max value samples: discretized Thompson sampling and Gumbel sampling (the default choice).

```python
from botorch.acquisition.max_value_entropy_search import qLowerBoundMaxValueEntropy

candidate_set_size = 1000 if not SMOKE_TEST else 5
candidate_set = torch.rand(candidate_set_size, bounds_norm.size(1), device=bounds.device, dtype=bounds.dtype)
qGIBBON = qLowerBoundMaxValueEntropy(model, candidate_set)

```

### 3. Optimizing the GIBBON acquisition function to get the next candidate points

In order to obtain the next candidate point(s) to query, we need to optimize the acquisition function over the design space. For $q=1$ case, we can simply call the `optimize_acqf` function in the library. For $q>1$,  we greedily build batches using sequential optimization.

```python
from botorch.optim import optimize_acqf

NUM_RESTARTS = 10 if not SMOKE_TEST else 2
RAW_SAMPLES = 512 if not SMOKE_TEST else 4

# for q = 1
candidates, acq_value = optimize_acqf(
    acq_function=qGIBBON,
    bounds=bounds,
    q=1,
    num_restarts=NUM_RESTARTS,
    raw_samples=RAW_SAMPLES,
)
candidates, acq_value
```

    (tensor([[ 0.1204, -0.0163]]), tensor(0.0085))

```python
from botorch.optim import optimize_acqf

# for q = 2, sequential optimsiation
candidates, acq_value = optimize_acqf(
    acq_function=qGIBBON,
    bounds=bounds,
    q=2,
    num_restarts=NUM_RESTARTS,
    raw_samples=RAW_SAMPLES,
    sequential=True,
)
candidates, acq_value
```

    (tensor([[ 0.1193, -0.0160],
             [ 1.4334,  0.4397]]),
     tensor([0.0085, 0.0082]))

### 4. Comparing GIBBON with other acquisition functions

We now perform an illustrative comparison between GIBBON and the other low-cost acquisition functions implemented in BoTorch. We plot points chosen by each of the acquisition functions, each acquisition function's surface.

#### Sequential BO (q=1)

Firstly, we investigate GIBBON in the purely sequential case, comparing agaisnt MES, Expected Improvement (EI) and Probability of Improvement (PI). We see that GIBBON provides a very high-quality approximation of MES, choosing essentially the same location.

```python
from botorch.acquisition import ExpectedImprovement, ProbabilityOfImprovement, qMaxValueEntropy
import matplotlib.pyplot as plt

# prep different acqusition functions
acqs = {}
candidate_set = torch.rand(10000, bounds.size(1), device=bounds.device, dtype=bounds.dtype)
acqs['GIBBON'] = qLowerBoundMaxValueEntropy(model, candidate_set)
acqs['MES'] = qMaxValueEntropy(model, candidate_set)
acqs['EI'] = ExpectedImprovement(model,best_f=train_Y.max())
acqs['PI'] = ProbabilityOfImprovement(model,best_f=train_Y.max())

# prep grid to evaluate acq functions
n = 100 if not SMOKE_TEST else 2
xv, yv = torch.meshgrid([torch.linspace(0, 1, n), torch.linspace(0, 1, n)])
test_x = torch.stack([xv.reshape(n*n, 1), yv.reshape(n*n, 1)], -1)

# eval and maximise acq functions
evals = {}
candidates = {}
for acq in acqs.keys():
    evals[acq] = acqs[acq](test_x).detach().reshape(n, n)
    candidates[acq],_ = optimize_acqf(
        acq_function=acqs[acq],
        bounds=bounds_norm,
        q=1,
        num_restarts=5,
        raw_samples=100
    )

# plot acqusition function values and chosen points
fig, (ax1, ax2, ax3, ax4) = plt.subplots(nrows=1, ncols=4, sharex=True, sharey=True, figsize=(10,5))
ax1.contourf(xv,yv,evals['GIBBON'],levels=20)
ax1.scatter(candidates['GIBBON'][:,0],candidates['GIBBON'][:,1],marker="X",c="r")
ax1.set_title("GIBBON")
ax2.contourf(xv,yv,evals['MES'],levels=20)
ax2.scatter(candidates['MES'][:,0],candidates['MES'][:,1],marker="X",c="r")
ax2.set_title("MES")
ax3.contourf(xv,yv,evals['EI'],levels=20)
ax3.scatter(candidates['EI'][:,0],candidates['EI'][:,1],marker="X",c="r")
ax3.set_title("EI")
ax4.contourf(xv,yv,evals['PI'],levels=20)
ax4.scatter(candidates['PI'][:,0],candidates['PI'][:,1],marker="X",c="r")
ax4.set_title("PI")
fig.text(0.5, -0.1, 'x_1', ha='center')
fig.text(-0.1, 0.5, 'x_2', va='center')
```

#### Batch BO (q=3)

For the batch BO case, GIBBON selects similar points to MES but with an order-of-magnitude lower computational overhead, i.e perfoming information-theoretic BO at the cost of much simpler acqusition functions like EI and PI. We stress that this gap in computational overhead between GIBBON and MES grows substantially as the optimisation progresses (see [2]).

```python
from botorch.acquisition import qNoisyExpectedImprovement, qProbabilityOfImprovement
from time import time

# prep different acqusition functions
acqs = {}
candidate_set = torch.rand(10000, bounds.size(1), device=bounds.device, dtype=bounds.dtype)
acqs['GIBBON'] = qLowerBoundMaxValueEntropy(model, candidate_set)
acqs['MES'] = qMaxValueEntropy(model, candidate_set)
acqs['EI'] = qNoisyExpectedImprovement(model, train_X)
acqs['PI'] = qProbabilityOfImprovement(model, best_f=train_Y.max())

# prep grid to evaluate acq functions
n = 100 if not SMOKE_TEST else 2
xv, yv = torch.meshgrid([torch.linspace(0, 1, n), torch.linspace(0, 1, n)])
test_x = torch.stack([xv.reshape(n*n, 1), yv.reshape(n*n, 1)], -1)

# eval and maximise acq functions
evals = {}
candidates = {}
times = {}
for acq in acqs.keys():
    evals[acq] = acqs[acq](test_x).detach().reshape(n, n)
    t_0 = time()
    candidates[acq],_ = optimize_acqf(
        acq_function=acqs[acq],
        bounds=bounds_norm,
        q=3,
        num_restarts=5,
        raw_samples=100,
        sequential=True
    )
    times[acq] = time() - t_0

# plot acqusition function values and chosen points
fig, (ax1, ax2, ax3, ax4) = plt.subplots(nrows=1, ncols=4, sharex=True, sharey=True, figsize=(10,5))
ax1.contourf(xv,yv,evals['GIBBON'],levels=20)
ax1.scatter(candidates['GIBBON'][:,0],candidates['GIBBON'][:,1],marker="X",c="r")
ax1.set_title("GIBBON")
ax2.contourf(xv,yv,evals['MES'],levels=20)
ax2.scatter(candidates['MES'][:,0],candidates['MES'][:,1],marker="X",c="r")
ax2.set_title("MES")
ax3.contourf(xv,yv,evals['EI'],levels=20)
ax3.scatter(candidates['EI'][:,0],candidates['EI'][:,1],marker="X",c="r")
ax3.set_title("EI")
ax4.contourf(xv,yv,evals['PI'],levels=20)
ax4.scatter(candidates['PI'][:,0],candidates['PI'][:,1],marker="X",c="r")
ax4.set_title("PI")
fig.text(0.5, -0.1, 'x_1', ha='center')
fig.text(-0.1, 0.5, 'x_2', va='center')

# plot computational overheads
plt.figure()
heights = [times[acq] for acq in acqs.keys()]
plt.bar(acqs.keys(), heights)
plt.ylabel("Computation Time")
plt.xlabel("Acquisition Function")
```


# Comparison of Descent Algorithms for Optimization

## Outline

The goal of this notebook is to implement and compare the performance of two common first-order optimization algorithms: Gradient Descent (GD) and Stochastic Gradient Descent (SGD).

For concreteness, we consider minimizing the cost of a construction project. Suppose we are building a pricing model for laying down new fibre optics high-speed internet cables under a particular region. The model takes as input a pair of coordinates $(x, y)$, and contains two parameters, $\lambda_1, \lambda_2$. Given a coordinate $(x, y)$, and model parameters, the loss in revenue corresponding to the price model at location $(x, y)$ is given by

$$
L(x, y, \lambda_1, \lambda_2) = 0.000045\lambda_2^2 y - 0.000098\lambda_1^2 x + 0.003926\lambda_1 x\exp\left\{\left(y^2 - x^2\right)\left(\lambda_1^2 + \lambda_2^2\right)\right\}
$$

After loading and describing the data, we first use level curves and a 3D plot to visually inspect the loss function.

Following this, we provide implementations of GD and SGD, visually determining convergence in both instances.

Finally, we compare the performance of both algorithms. Specifically, we consider efficiency, sensitivity to learning rate, and sensitivity to initialization.

```python
# Import relevant libraries
import numpy as np
import numpy.linalg as LA
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import time
import warnings
from itertools import product

# Aesthetics and other options
plt.style.use('seaborn-notebook')
sns.set_style('darkgrid')
warnings.simplefilter('ignore')
%matplotlib inline
```

## Description of the Data
The data contained in `coordinates.csv` contains the set of coordinates $\{(x_n, y_n)\}$ configured on the curve $y^2 - x^2 = -0.1$. We wish to find parameters $\lambda_1, \lambda_2$ that minimize the net loss over the entire dataset of coordinates $(x_n, y_n)$.

```python
# Load data
df = pd.read_csv('data/coordinates.csv')
x, y = np.array(df.columns.astype('float')), df.values[0]
```

## Visualization of Loss Function

```python
# Function to minimize
def loss_function(λ_1, λ_2):
    global x, y
    loss = 0.000045 * λ_2**2 * y - 0.000098 * λ_1**2 * x \
           + 0.003926 * λ_1 * x * np.exp((y**2-x**2)*(λ_1**2+λ_2**2))
    return np.sum(loss)
L = np.vectorize(loss_function)

# Grid of values to plot
λ_1, λ_2 = np.meshgrid(np.linspace(-5, 5, 100), np.linspace(-5, 5, 100))
z = L(λ_1, λ_2)

# Main plot
fig = plt.figure(figsize=(14, 6))

ax1 = fig.add_subplot(121)
contour = ax1.contour(λ_1, λ_2, z)
ax1.clabel(contour, inline=1, fontsize=10)
ax1.set_xlabel(r'$\lambda_1$')
ax1.set_ylabel(r'$\lambda_2$')
ax1.set_title('Contour Plot of Loss Function')

ax2 = fig.add_subplot(122, projection='3d')
ax2.plot_trisurf(λ_1.flatten(), λ_2.flatten(), z.flatten(), linewidth=0.2, antialiased=True, color='skyblue', alpha=0.5)
ax2.set_xlabel(r'$\lambda_1$')
ax2.set_ylabel(r'$\lambda_2$')
ax2.set_zlabel(r'$L(\lambda_1, \lambda_2)$')
ax2.set_title('3D-Plot of Loss Function');
```

## Implementation of Gradient Descent

We first calculate $\nabla L$:

$$
\begin{align}
\frac{\partial L}{\partial \lambda_1} &= 0.003926 x \text{exp}((\lambda_1^2+\lambda_2^2)(y^2-x^2)) - 0.000196\lambda_1 x - 0.007852\lambda_1^2 x (x^2-y^2) \text{exp}((\lambda_1^2+\lambda_2^2)(y^2-x^2)) \\
\frac{\partial L}{\partial \lambda_2} &= 0.00009 \lambda_2 y - 0.007852 \lambda_1 \lambda_2 x (x^2-y^2) \text{exp}((\lambda_1^2 + \lambda_2^2)(y^2-x^2))
\end{align}
$$

```python
# Function implementing gradient descent
def GD(λ, η=1.0, ε=1e-3):
    global x, y
    m = len(y)
    λ_optimal = np.array([2.05384, 0])
    n_iters = 0
    loss = []
    times = []

    # Helper function to calculate the gradient at each iteration
    def grad_L(λ_1, λ_2):
        dλ_1 = np.sum(0.003926 * x * np.exp((λ_1**2+λ_2**2)*(y**2-x**2)) - 0.000196 * λ_1 * x - 0.007852 * λ_1**2 * x  \
               * (x**2-y**2) * np.exp((λ_1**2+λ_2**2)*(y**2-x**2)))
        dλ_2 = np.sum(0.00009 * λ_2 * y - 0.007852 * λ_1 * λ_2 * x * (x**2-y**2) * np.exp((λ_1**2 + λ_2**2) * (y**2-x**2)))
        return np.array([dλ_1, dλ_2])

    # Main function
    while LA.norm(λ_optimal - λ) > ε:
        loss.append((1/m) * L(λ[0], λ[1]))
        start = time.time()
        λ = λ - (1/m) * η * grad_L(λ[0], λ[1])
        times.append(time.time()-start)
        n_iters += 1
    return np.mean(times), n_iters, loss, λ

# Initial values
λ_init = np.array([2, 1])

# Run gradient descent
avg_time_per_update_gd, n_iters_gd, loss_gd, λ_gd = GD(λ_init)
print(f"We obtain λ = [{λ_gd[0]:.4f}, {λ_gd[1]:.4f}] after {n_iters_gd} iterations with a learning rate of η = 1.0")
print(f"Average time per update: {avg_time_per_update_gd:.8f} seconds")

# Plot values of loss function per iteration to determine convergence
plt.plot(np.arange(1, n_iters_gd+1), loss_gd)
plt.xlabel('Number of Iterations')
plt.ylabel('Loss');
```

From the above, we see that the gradient descent algorithm converges after 43544 iterations. We also see that the returned parameter vector agrees with our visualization of the loss function - that the global minimum of $L$ appears to be attained at $\lambda_1 \approx 2.05, \lambda_2 \approx 0$.

## Implementation of Stochastic Gradient Descent

```python
# Set random seed for reproducibility
np.random.seed(207)

# Function implementing stochastic gradient descent
def SGD(λ, η=1.0, batch_size=1, ε=1e-3):
    global x, y
    x_copy, y_copy = x.copy(), y.copy()
    m = len(y)
    λ_optimal = np.array([2.05384, 0])
    inds = np.arange(m)
    n_iters = 0
    loss = []
    times = []

    # Helper function to calculate the gradient at each iteration
    def grad_L(λ_1, λ_2):
        dλ_1 = np.sum(0.003926 * x * np.exp((λ_1**2+λ_2**2)*(y**2-x**2)) - 0.000196 * λ_1 * x - 0.007852 * λ_1**2 * x  \
               * (x**2-y**2) * np.exp((λ_1**2+λ_2**2)*(y**2-x**2)))
        dλ_2 = np.sum(0.00009 * λ_2 * y - 0.007852 * λ_1 * λ_2 * x * (x**2-y**2) * np.exp((λ_1**2 + λ_2**2) * (y**2-x**2)))
        return np.array([dλ_1, dλ_2])

    # Main function
    while LA.norm(λ_optimal-λ) > ε:
        sample_inds = np.random.choice(inds, replace=False, size=batch_size)
        x, y = x[sample_inds], y[sample_inds]
        start = time.time()
        λ = λ - (1/batch_size) * η * grad_L(λ[0], λ[1])
        times.append(time.time()-start)
        x, y = x_copy, y_copy
        loss.append((1/m) * L(λ[0], λ[1]))
        n_iters += 1
    return np.mean(times), n_iters, loss, λ

# Run stochastic gradient descent
avg_time_per_update_sgd, n_iters_sgd, loss_sgd, λ_sgd = SGD(λ_init)
print(f"We obtain λ = [{λ_sgd[0]:.4f}, {λ_sgd[1]:.4f}] after {n_iters_sgd} iterations with a learning rate of η = 1.0")
print(f"Average time per update: {avg_time_per_update_sgd:.8f} seconds")

# Plot values of loss function per iteration to determine convergence
plt.plot(np.arange(1, n_iters_sgd+1), loss_sgd)
plt.xlabel('Number of Iterations')
plt.ylabel('Loss');
```

From the above, we see that the gradient descent algorithm converges after 63711 iterations. We also see that the returned parameter vector agrees with the solution obtained in the previous section.

## Comparison of Algorithms

**Average Time per Update**

- Gradient Descent: 0.00033216 seconds

- Stochastic Gradient Descent: 0.00003211 seconds

We note that, on average, SGD performs each update faster than that of GD. This result is expected since, in the case of SGD, each update uses a smaller subset of the data.

**Number of Iterations until Convergence**

- Gradient Descent: 43544 iterations

- Stochastic Gradient Descent: 63711 iterations

Gradient descent reaches convergence in a fewer number of iterations than that of SGD. Again, this result is expected since, in the case of GD, each update is more "meaningful" in that it incorporates all of the available data.

#### Learning Rate Sensitivity - Gradient Descent

```python
# Learning rates
learning_rates = [1000, 100, 1, 0.1]

# Perform gradient descent for each value of η
num_iters_gd, loss_gd = [], []
for rate in learning_rates:
    _, n_iters, loss, _ = GD(λ_init, η=rate)
    num_iters_gd.append(n_iters)
    loss_gd.append(loss)

# Report results
for i in range(4):
    print(f"η = {learning_rates[i]}: {num_iters_gd[i]} iterations")

# Plot results:
fig, ax = plt.subplots(ncols=4, figsize=(20, 8))

fig.tight_layout(pad=5)

for i in range(4):
    ax[i].plot(np.arange(1, num_iters_gd[i] + 1), loss_gd[i])
    ax[i].set(
        xlabel='Number of Iterations',
        ylabel='Loss',
        title=f'η = {learning_rates[i]}'
    )
```

#### Learning Rate Sensitivity - Stochastic Gradient Descent

```python
# Set random seed for reproducibility
np.random.seed(207)

# Perform stochastic gradient descent for each value of η
num_iters_sgd, loss_sgd = [], []
for rate in learning_rates:
    _, n_iters, loss, _ = SGD(λ_init, η=rate)
    num_iters_sgd.append(n_iters)
    loss_sgd.append(loss)

# Report results
for i in range(4):
    print(f"η = {learning_rates[i]}: {num_iters_sgd[i]} iterations")

# Plot results:
fig, ax = plt.subplots(ncols=4, figsize=(20, 8))

fig.tight_layout(pad=5)

for i in range(4):
    ax[i].plot(np.arange(1, num_iters_sgd[i] + 1), loss_sgd[i])
    ax[i].set(
        xlabel='Number of Iterations',
        ylabel='Loss',
        title=f'η = {learning_rates[i]}'
    )
```

From the above, we see that SGD is more sensitive to the learning rate. In particular, we note that, in the case of SGD, convergence is not achieved when $\eta$ equals 1000.

Other observed results are expected - namely, larger values of $\eta$ result in fewer iterations until convergence (for non-pathalogical cases) for both GD and SGD.

#### Sensitivity to Initialization

```python
# Parameters to consider
initial_values = [
    np.array([-2.47865, 0]),
    np.array([-3.0, 0]),
    np.array([-5.0, 0]),
    np.array([-10.0, 0])
]
rates = [10, 50]
grid = list(product(initial_values, rates))
```

From the above grid of values, convergence to the global minimum was only achieved via stochastic gradient descent with $\eta$ equal to 50. This suggests that gradient descent is more sensitive to initialization - in this particular instance, the algorithm is "trapped" at a local stationary point (as seen in the contour plot from the first section). Indeed, this corresponds to what we know about SGD: for sufficiently large η, the algorithm can "climb out of" local minima/maxima when the objective function is nonconvex.


# Символьные вычисления

Представим, вы студент технической специальности и только начинаете заниматься математическим анализом, но уже немного знаете Python. Тогда я горячо рекомендую изучить документацию библиотеки [sympy](https://docs.sympy.org/) — она сделает освоение матанализа более интереснее, вам будет легче проводить и проверять расчеты.

```python
from sympy import *

x, y, z, t = symbols('x y z t')
```

# Действия с выражениями

```python
expr = x + 2*y
x*expr
```

$\displaystyle x \left(x + 2 y\right)$

```python
expand(x*expr)
```

$\displaystyle x^{2} + 2 x y$

```python
simplify((x-1)*(x+1))
```

$\displaystyle x^{2} - 1$

# Пределы

```python
limit(sin(x)/x, x, 0)
```

$\displaystyle 1$

```python
expr = x**2/exp(x)
limit(expr, x, oo)
```

$\displaystyle 0$

# Производные

```python
diff(sin(x)*exp(x), x)
```

$\displaystyle e^{x} \sin{\left(x \right)} + e^{x} \cos{\left(x \right)}$

```python
expr = exp(x*y*z)
diff(expr, x, x, y, z)
```

$\displaystyle y z \left(x^{2} y^{2} z^{2} + 5 x y z + 4\right) e^{x y z}$

# Первообразные и интегралы

```python
integrate(1/x, x)
```

$\displaystyle \log{\left(x \right)}$

```python
integrate(exp(x)*sin(x) + exp(x)*cos(x), x)
```

$\displaystyle e^{x} \sin{\left(x \right)}$

```python
integrate(sin(x**2), (x, -oo, oo))
```

$\displaystyle \frac{\sqrt{2} \sqrt{\pi}}{2}$

```python
integrate(exp(-x**2 - y**2), (x, -oo, oo), (y, -oo, oo))
```

$\displaystyle \pi$

# Решение дифференциальных уравнений

```python
y = Function('y')
t = symbols('t')
dsolve(Eq(y(t).diff(t, t)-y(t), exp(t)))
```

$\displaystyle y{\left(t \right)} = C_{2} e^{- t} + \left(C_{1} + \frac{t}{2}\right) e^{t}$

# Специальные функции

```python
simplify(gamma(x)/gamma(x-2))
```

$\displaystyle \left(x - 2\right) \left(x - 1\right)$

# Задачи линейной алгебры

```python
M = Matrix([[1, 2, 3], [3, 2, 1]])
M
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$

```python
N = Matrix([0, 1, 1])
M*N
```

$\displaystyle \left[\begin{matrix}5\\3\end{matrix}\right]$

```python
X = Matrix([[1, 2], [2, 2]])
X.eigenvals()
```

    {3/2 - sqrt(17)/2: 1, 3/2 + sqrt(17)/2: 1}

```python
X.eigenvects()
```

    [(3/2 - sqrt(17)/2,
      1,
      [Matrix([
       [-sqrt(17)/4 - 1/4],
       [                1]])]),
     (3/2 + sqrt(17)/2,
      1,
      [Matrix([
       [-1/4 + sqrt(17)/4],
       [                1]])])]

# Работа с LaTeX и MathML

```python
print(latex(Integral(cos(x)**2, (x, 0, pi))))
```

    \int\limits_{0}^{\pi} \cos^{2}{\left(x \right)}\, dx

$\int\limits_{0}^{\pi} \cos^{2}{\left(x \right)}\, dx$