## Model Based Submodular Selection
Author: Jacob Schreiber <jmschreiber91@gmail.com>

Submodular selection is the task of identifying a representative subset of samples from a large set, and apricot focuses on the use of these algorithms to identify a good subset of data that can be used for the purpose of training machine learning models. However, submodular functions can also be coupled with a trained machine learning model and a feature attribution algorithm in order to identify the subset of features that the machine learning model thinks are the most important for training.

In order to understand how this works, let's review the feature based submodular algorithm that's implemented in apricot. These functions greedily add samples to the growing subset that utilize a diversity of features. The equation that the feature based functions optimize is as follows:

\begin{equation}
f(X) = \sum\limits_{u \in U} w_{u} \phi_{u} \left( \sum\limits_{x \in X} m_{u}(x_{u}) \right)
\end{equation}

In this equation, $U$ is the set of features, or dimensions, of a sample, and $u$ refers to a specific feature. $X$ refers to the original data set that we are selecting from and $x$ refers to a single sample from that data set. $w$ is a vector of weights that indicate how important each feature is, with $w_{u}$ being a scalar referring to how important feature $u$ is. Frequently these weights are uniform. $\phi$ refers to a set of saturating functions, such as $sqrt(X)$ or $log(X + 1)$, that have diminishing returns the larger X gets.

The default implementation in apricot for $m_{u}$ is the identify function $m_{u}(x_{u}) = x_{u}$, meaning that a sample will have the most gain when it has a high feature value in a feature that is not well represented with high values in the already included samples. What if, instead of using the raw feature value, we used the feature ~attribution~ as determined by some attribution algorithm? This would give us a subset of samples that gave a diversity of importanes.

```python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')
import shap

from sklearn.linear_model import LogisticRegression
from shap import LinearExplainer
from apricot import FeatureBasedSelection
```

    Populating the interactive namespace from numpy and matplotlib

Let's start off with creating two clusters of data and seeing what happens when we use a feature based function off of them natively.

```python
X = numpy.concatenate([numpy.random.normal((7, 8), 1, size=(100, 2)),
                       numpy.random.normal((10, 5), 1, size=(100, 2))])

y = numpy.concatenate([numpy.zeros(100), numpy.ones(100)])

Xi, yi = FeatureBasedSelection(20).fit_transform(X, y)
```

```python
plt.figure(figsize=(8, 6))
plt.title("Submodular selection on Gaussian blobs", fontsize=16)
plt.scatter(X[:100, 0], X[:100, 1], marker='+', label='Negative Class')
plt.scatter(X[100:, 0], X[100:, 1], marker='+', label='Positive Class')
plt.scatter(Xi[yi == 0, 0], Xi[yi == 0, 1], label='Selected Negative Samples')
plt.scatter(Xi[yi == 1, 0], Xi[yi == 1, 1], label="Selected Positive Samples")

plt.legend(fontsize=14, loc=(1, 0.3))
plt.show()
```

That's unfortunate. It looks like the selected samples are just those with the highest feature values. This is not entirely true due to the non-linearity that's applied during selection, but an approximation is that as you run a diagonal line with slope y = -x down from the top right corner of the plot, it selected the first 20 points that it hits. This is a downside of using a feature based function natively.

Well, our goal here was not to select on the feature naively, but rather to identify features that a trained machine learning model thought were important. However, this term "important" can have two meanings. It can be the samples that are the most obvious samples from one or another class, perhaps the most representative, or it can be those that are on the boundary. Let's look at the first case first. In order to calculate feature importances we will use the package shap.

The first thing to do is to train our model. In our case let's train a logistic regression model.

```python
model = LogisticRegression().fit(X, y)
```

Now we need to gt the attributions. These values sum up to the final prediction and atribute to each feature the importance of that measurement.

```python
X_shap = LinearExplainer(model, X).shap_values(X)

plt.figure(figsize=(12, 4))
plt.subplot(121)
plt.title("Original Data", fontsize=14)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10)
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10)

plt.subplot(122)
plt.title("Attributions", fontsize=14)
plt.scatter(X_shap[y == 0, 0], X_shap[y == 0, 1], s=10)
plt.scatter(X_shap[y == 1, 0], X_shap[y == 1, 1], s=10)
plt.xlabel("x-axis attribution", fontsize=14)
plt.ylabel("y-axis attribution", fontsize=14)
plt.show()
```

Unfortunately these values correspond to the log odds and so can be negative when they are predicting the negative class. We can correct for this by taking the absolute value of the attribution. That will ensure that the values are entirely positive and that the higher in magnitude they are, the most important that feature was for the prediction.

```python
X_shap = numpy.abs(X_shap)

plt.scatter(X_shap[y == 0, 0], X_shap[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X_shap[y == 1, 0], X_shap[y == 1, 1], s=10, label="Positive Samples")
plt.xlabel("|x-axis attribution|", fontsize=14)
plt.ylabel("|y-axis attribution|", fontsize=14)
plt.legend(fontsize=14, loc=(1.01, 0.4))
plt.show()
```

In this absolute attribution space we will still be selecting values as if you were running a line with slope y = -x down from the top of the plot, but these transformed values should be more amenable to that selection in that manner. Now, let's select the 20 most important samples according to the model. We can compare this to the samples that the model is most confident in, as ranked by maximum predicted probability.

```python
Xi1, yi1 = FeatureBasedSelection(20).fit_transform(X, y)

selector = FeatureBasedSelection(20)
selector.fit_transform(X_shap)
Xi2 = X[selector.indices]
yi2 = y[selector.indices]

y_pred = model.predict_proba(X).max(axis=1)
idx = numpy.argsort(y_pred)[::-1][:20]
Xi3 = X[idx]
yi3 = y[idx]

plt.figure(figsize=(14, 4))
plt.subplot(131)
plt.title("Selection in the original space", fontsize=16)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi1[yi1 == 0, 0], Xi1[yi1 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi1[yi1 == 1, 0], Xi1[yi1 == 1, 1], s=20, label="Positive Selected Samples")

plt.subplot(132)
plt.title("Selection in the transformed space", fontsize=16)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi2[yi2 == 0, 0], Xi2[yi2 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi2[yi2 == 1, 0], Xi2[yi2 == 1, 1], s=20, label="Positive Selected Samples")

plt.subplot(133)
plt.title("Selection in the transformed space", fontsize=16)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi3[yi3 == 0, 0], Xi3[yi3 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi3[yi3 == 1, 0], Xi3[yi3 == 1, 1], s=20, label="Positive Selected Samples")
plt.legend(fontsize=14, loc=(1, 0.5))
plt.show()
```

It certainly looks like using the absolute attributions in this manner is yielding samples that are more representative of the most confident samples. However, it's unclear what the differences between using attributions versus using the model predictions alone correspond to, and whether one is better than the other in this simple case.

Another way that one might want to select samples is based on the *least confident samples*, i.e., identifying those samples that lie near the decision boundary. This can be particularly useful for the identification of samples that a model would be uncertain about. We can use a similar procedure, but after taking the absolute value of the attribution values, we multiply all attributions for a sample by the *minimum predicted class probability*. This has the effect of reducing values for samples where the model is very confident, while preserving values when the model is uncertain. The submodular selection algorithm will then preferentially select samples that have a diversity of features being relevant for the prediction, but only for uncertain samples.

```python
X_shap = LinearExplainer(model, X).shap_values(X)
X_shap = (numpy.abs(X_shap).T * model.predict_proba(X).min(axis=1)).T

Xi1, yi1 = FeatureBasedSelection(20).fit_transform(X, y)

selector = FeatureBasedSelection(20)
selector.fit_transform(X_shap)
Xi2 = X[selector.indices]
yi2 = y[selector.indices]

y_pred = model.predict_proba(X).min(axis=1)
idx = numpy.argsort(y_pred)[::-1][:20]
Xi3 = X[idx]
yi3 = y[idx]

plt.figure(figsize=(14, 4))
plt.subplot(131)
plt.title("Selection on feature values", fontsize=14)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi1[yi1 == 0, 0], Xi1[yi1 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi1[yi1 == 1, 0], Xi1[yi1 == 1, 1], s=20, label="Positive Selected Samples")

plt.subplot(132)
plt.title("Selection on feature attributions", fontsize=14)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi2[yi2 == 0, 0], Xi2[yi2 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi2[yi2 == 1, 0], Xi2[yi2 == 1, 1], s=20, label="Positive Selected Samples")

plt.subplot(133)
plt.title("Least confident samples", fontsize=16)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=10, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], s=10, label="Positive Samples")
plt.scatter(Xi3[yi3 == 0, 0], Xi3[yi3 == 0, 1], s=20, label="Negative Selected Samples")
plt.scatter(Xi3[yi3 == 1, 0], Xi3[yi3 == 1, 1], s=20, label="Positive Selected Samples")
plt.legend(fontsize=14, loc=(1, 0.3))
plt.show()
```

It looks like, again, using the attributions is much more meaningful tha running submodular selection on the original feature values. One could argue that it looks like performing submodular selection on the attributions yields a more diverse sampling near the decision boundary than using the those samples that the model is the least confident about, but it's a bit subjective to say that.

Let's try to solidify a situation in which submodular selection on the attribution values is better than selecting the least confident samples. In this case, we'll build a data set where samples overlap significant in one part of the decision boundary and not in another portion.

```python
numpy.random.seed(2)

X = numpy.concatenate([numpy.random.uniform((5.6, 5), (8, 6.3), size=(100, 2)),
                       numpy.random.uniform((4, 5), (5.8, 8), size=(200, 2)),
                       numpy.random.uniform(6, 8.1, size=(300, 2))])

y = numpy.concatenate([numpy.zeros(300), numpy.ones(300)])

plt.scatter(X[y == 0, 0], X[y == 0, 1], s=5, label="Negative Samples")
plt.scatter(X[y == 1, 0], X[y == 1, 1], c='r', s=5, label="Positive Samples")
plt.legend(fontsize=14, loc=(1, 0.4))
plt.xlim(4, 8)
plt.ylim(5, 8)
plt.show()
```

On this data set we can see that the blue and red samples overlap on the portion of the decision boundary where the y-axis separates the two classes, but that the samples don't overlap on the portion where the x-axis separates the two classes. A machine learning model built on this data will be less sure about the overlapping samples

```python
from sklearn.svm import SVC
from shap import KernelExplainer

model = SVC(probability=True).fit(X, y)
explainer = KernelExplainer(model.predict_proba, X)
X_shap = explainer.shap_values(X)
X_shap = (numpy.abs(X_shap[1] - X_shap[0]).T * model.predict_proba(X).min(axis=1)).T

selector = FeatureBasedSelection(50)
selector.fit_transform(X_shap)
Xi2 = X[selector.indices]

y_pred = model.predict_proba(X).min(axis=1)
idx = numpy.argsort(y_pred)[::-1]

xx, yy = np.meshgrid(np.arange(4, 8.1, 0.1), np.arange(5, 9.1, 0.1))
Z = model.predict(np.c_[xx.ravel(), yy.ravel()])

# Put the result into a color plot
Z = Z.reshape(xx.shape)
```

    100%|██████████| 600/600 [00:05<00:00, 115.12it/s]

Now let's plot the attributions for each sample, the decision boundaries, and the selected samples using submodular selection versus the least confident samples.

```python
plt.figure(figsize=(14, 4))
plt.subplot(131)
plt.title("Feature Attributions", fontsize=16)
plt.scatter(X_shap[y == 0, 0], X_shap[y == 0, 1], s=5)
plt.scatter(X_shap[y == 1, 0], X_shap[y == 1, 1], c='r', s=5)
plt.scatter(X_shap[selector.indices][:, 0], X_shap[selector.indices][:, 1], c='g', s=20)
plt.xlabel("x-axis attribution", fontsize=12)
plt.ylabel("y-axis attribution", fontsize=12)

plt.subplot(132)
plt.title("Selection on Attribution", fontsize=16)
plt.contourf(xx, yy, Z, cmap='RdBu_r', linewidths=0.3, alpha=0.25)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=5)
plt.scatter(X[y == 1, 0], X[y == 1, 1], c='r', s=5)
plt.scatter(Xi2[:, 0], Xi2[:, 1], s=20, color='g', label="Selected Samples")
plt.xlim(4, 8)
plt.ylim(5, 8)

plt.subplot(133)
plt.title("Least confident samples", fontsize=16)

plt.contourf(xx, yy, Z, cmap='RdBu_r', linewidths=0.3, alpha=0.25)
plt.scatter(X[y == 0, 0], X[y == 0, 1], s=5)
plt.scatter(X[y == 1, 0], X[y == 1, 1], c='r', s=5)
plt.scatter(X[idx[:50], 0], X[idx[:50], 1], s=20, color='#FF6600', label="Selected Samples")
plt.xlim(4, 8)
plt.ylim(5, 8)
plt.savefig("img/attributionselection.png")
plt.show()
```

It looks like here that submodular selection identifies six samples from the x-axis dominated portion of the selection whereas the least confident samples selects only a single one.


# Multibody dynamics of one and two-link systems

> Marcos Duarte
> Laboratory of Biomechanics and Motor Control ([http://demotu.org/](http://demotu.org/))
> Federal University of ABC, Brazil

The human body is composed of multiple interconnected segments (which can be modeled as rigid or flexible) and each segment may have translational and rotational movement. The part of mechanics for the study of movement and forces of interconnected bodies is called [multibody system](http://en.wikipedia.org/wiki/Multibody_system) or multibody dynamics.

There are different approaches to deduce the kinematics and dynamics of such bodies, the most common are the [Newton-Euler](http://en.wikipedia.org/wiki/Newton%E2%80%93Euler_equations) and the [Langrangian](http://en.wikipedia.org/wiki/Lagrangian_mechanics) formalisms. The Newton-Euler formalism is based on the well known Newton-Euler equations. The Langrangian formalism uses the [principle of least action](http://en.wikipedia.org/wiki/Principle_of_least_action) and describes the movement based on [generalized coordinates](http://en.wikipedia.org/wiki/Generalized_coordinates), a set of parameters (typically, a convenient minimal set) to describe the configuration of the system taking into account its constraints. For a system with multiple bodies and several constraints, e.g., the human body, it is easier to describe the dynamics of such system using the Langrangian formalism.

Zajac and Gordon (1989) and Zajac (1993) offer excellent discussions about applying multibody system concepts to understanding human body movement.

Next, we will study two simple problems of multibody systems in the context of biomechanics which we can handle well using the Newton-Euler approach.
First a planar one-link system (which is not a multibody), which can represent the movement of one limb of the body or the entire body as a single inverted pendulum.
Second, a planar two-link system, which can represent the movement of two segments of the body, e.g., upper arm and forearm.

## Newton-Euler equations

For a two-dimensional movement in the $XY$ plane, the Newton-Euler equations are:

\begin{align}
\left\{ \begin{array}{l l}
\sum F_X &=& m \ddot{x}_{cm} \\
\\
\sum F_Y &=& m \ddot{y}_{cm} \\
\\
\sum M_Z &=& I_{cm} \ddot{\alpha}_Z
\end{array} \right.
\label{}
\end{align}

Where the movement is described around the body center of mass ($cm$).
$(F_X,\,F_Y)$ and $M_Z$ are, respectively, the forces and moment of forces (torques) acting on the body.
$(\ddot{x}_{cm},\,\ddot{y}_{cm})$ and $\ddot{\alpha}_Z$ are, respectively, the linear and angular accelerations.
$I_{cm}$ is the body moment of inertia around the body center of mass at the $Z$ axis.

Let's use Sympy to derive some of the characteristics of the systems.

```python
import sympy as sym
from sympy import Symbol, symbols, cos, sin, Matrix, simplify
from sympy.vector import CoordSys3D
from sympy.physics.mechanics import dynamicsymbols, mlatex, init_vprinting
init_vprinting()
from IPython.display import display, Math

eq = lambda lhs, rhs: display(Math(lhs + '=' + mlatex(rhs)))
eq = lambda lhs, rhs: display(Math(r'\begin{array}{l l}' + lhs +
                                   '&=&' + mlatex(rhs) + r'\end{array}'))
```

## One-link system

Let's study the dynamics of a planar inverted pendulum as a model for the movement of a human body segment with an external force acting on the segment (see Figure 1).

Figure 1. Planar inverted pendulum (one link attached to a fixed body by a hinge joint in a plane) with a joint actuators and corresponding free body diagram as a model of a human body segment. See text for notation convention.

The following notation convention will be used for this problem:

 - $L$ is the length of the segment.
 - $d$ is the distance from the joint of the segment to its center of mass position.
 - $m$ is the mass of the segment.
 - $g$ is the gravitational acceleration (+).
 - $\alpha$ is the angular position of the joint w.r.t. horizontal, $\ddot{\alpha_i}$ is the corresponding angular acceleration.
 - $I$ is the moment of inertia of the segment around its center of mass position.
 - $F_{r}$ is the joint reaction force.
 - $F_{e}$ is the external force acting on the segment.
 - $T$ is the joint moment of force (torque).

In the case of a human body segment, muscles responsible for the movement of the segment are represented as a single pair of antagonistic joint actuators (e.g., flexors and extensors). We will consider that all joint torques are generated only by these muscles (we will disregard the torques generated by ligaments and other tissues) and the total or net joint torque will be the sum of the torques generated by the two muscles:

\begin{equation}
T \quad=\quad T_{net} \quad=\quad T_{extension} - T_{flexion}
\label{}
\end{equation}

Where we considered the extensor torque as positive (counter-clockwise). In what follows, we will determine only the net torque and we will be unable to decompose the net torque in its components.

### Kinetics

From the free body diagram, the Newton-Euler equations for the planar inverted pendulum are (writing the equation for the torques around the center of mass):

\begin{equation}
\begin{array}{l l}
F_{r,x} + F_{e,x} & = & m\ddot{x} \\
\\
F_{r,y} - mg + F_{e,y} & = & m\ddot{y} \\
\\
T + dF_{r,x}\sin\alpha - dF_{r,y}\cos\alpha - (L-d)F_{e,x}\sin\alpha + (L-d)F_{e,y}\cos\alpha & = & I\ddot{\alpha}
\end{array}
\label{}
\end{equation}

However, manually placing the terms in the Newton-Euler equations as we did above where we calculated the signs of the cross products is error prone. We can avoid this manual placing by treating the quantities as vectors and express them in matricial form:

\begin{equation}
\begin{array}{l l}
\vec{\mathbf{F}}_r + \vec{\mathbf{F}}_g + \vec{\mathbf{F}}_e &=& m\ddot{\vec{\mathbf{r}}} \\
\\
\vec{\mathbf{T}} + \vec{\mathbf{r}}_{cm,j} \times \vec{\mathbf{F}}_r + \vec{\mathbf{r}}_{cm,e} \times \vec{\mathbf{F}}_e &=& I\ddot{\vec{\mathbf{\alpha}}}
\end{array}
\label{}
\end{equation}

Where:

\begin{equation}
\begin{array}{l l}
\begin{bmatrix} F_{rx} \\ F_{ry} \\ 0 \end{bmatrix}  + \begin{bmatrix} 0 \\ -g \\ 0 \end{bmatrix}  + \begin{bmatrix} F_{ex} \\ F_{ey} \\ 0 \end{bmatrix} &=& m\begin{bmatrix} \ddot{x} \\ \ddot{y} \\ 0 \end{bmatrix} , \quad \begin{bmatrix} \hat{i} \\ \hat{j} \\ \hat{k} \end{bmatrix}
\\
\begin{bmatrix} 0 \\ 0 \\ T_z \end{bmatrix}  + \begin{bmatrix} -d\cos\alpha \\ -d\sin\alpha \\ 0 \end{bmatrix}  \times \begin{bmatrix} F_{rx} \\ F_{ry} \\ 0 \end{bmatrix} + \begin{bmatrix} (L-d)\cos\alpha \\ (L-d)\sin\alpha \\ 0 \end{bmatrix}  \times \begin{bmatrix} F_{ex} \\ F_{ey} \\ 0 \end{bmatrix} &=& I_z\begin{bmatrix} 0 \\ 0 \\ \ddot{\alpha} \end{bmatrix} , \quad \begin{bmatrix} \hat{i} \\ \hat{j} \\ \hat{k} \end{bmatrix}
\end{array}
\label{}
\end{equation}

Note that $\times$ represents the cross product, not matrix multiplication. Then, both in symbolic or numeric manipulation we would use the cross product function to perform part of the calculations.
There are different computational tools that can be used for the formulation of the equations of motion. For instance, Sympy has a module, [Classical Mechanics](http://docs.sympy.org/dev/modules/physics/mechanics/), and see [this list](http://real.uwaterloo.ca/~mbody/#Software) for other software.
Let's continue with the explicit manual formulation of the equations for now.

We can rewrite the equation for the moments of force in a form that doesn't explicitly involve the joint reaction force expressing the moments of force around the joint center:

\begin{equation}
T - mgd\cos\alpha - LF_{e,x}\sin\alpha + LF_{e,y}\cos\alpha \quad=\quad I_o\ddot{\alpha}
\label{}
\end{equation}

Where $I_o$ is the moment of inertia around the joint, $I_o=I_{cm}+md^2$, using the parallel axis theorem.

The torque due to the joint reaction force does not appear on this equation; this torque is null because by the definition the reaction force acts on the joint. If we want to determine the joint torque and we know the kinematics, we perform inverse dynamics:

\begin{equation}
T \quad=\quad I_o\ddot{\alpha} + mgd \cos \alpha + LF_{e,x}\sin\alpha - LF_{e,y}\cos\alpha
\label{}
\end{equation}

If we want to determine the kinematics and we know the joint torque, we perform direct dynamics:

\begin{equation}
\ddot{\alpha} \quad=\quad I_o^{-1}[T - mgd \cos \alpha - LF_{e,x}\sin\alpha + LF_{e,y}\cos\alpha ]
\label{}
\end{equation}

The expression above is a second-order differential equation which typically is solved numerically. So, unless we are explicitly interested in estimating the joint reaction forces, we don't need to use them for calculating the joint torque or simulate movement. Anyway, let's look at the kinematics of this problem to introduce some important concepts which will be needed later.

### Kinematics

A single planar inverted pendulum has one degree of freedom, the rotation movement of the segment around the pin joint. In this case, if the angular position $\alpha(t)$ is known, the coordinates $x(t)$ and $y(t)$ of the center of mass and their derivatives can be readily determined (a process referred as [forward kinematics)](http://nbviewer.jupyter.org/github/demotu/BMC/blob/master/notebooks/KinematicChain.ipynb#Forward-and-inverse-kinematics):

```python
t = Symbol('t')
d, L = symbols('d L', positive=True)
a = dynamicsymbols('alpha')
```

```python
x, y = d*cos(a), d*sin(a)
xd, yd = x.diff(t), y.diff(t)
xdd, ydd = xd.diff(t), yd.diff(t)

eq(r'x', x)
eq(r'\dot{x}', xd)
eq(r'\ddot{x}', xdd)
eq(r'y', y)
eq(r'\dot{y}', yd)
eq(r'\ddot{y}', ydd)
```

$$\begin{array}{l l}x&=&d \operatorname{cos}\left(\alpha\right)\end{array}$$

$$\begin{array}{l l}\dot{x}&=&- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}\end{array}$$

$$\begin{array}{l l}\ddot{x}&=&- d \operatorname{sin}\left(\alpha\right) \ddot{\alpha} - d \operatorname{cos}\left(\alpha\right) \dot{\alpha}^{2}\end{array}$$

$$\begin{array}{l l}y&=&d \operatorname{sin}\left(\alpha\right)\end{array}$$

$$\begin{array}{l l}\dot{y}&=&d \operatorname{cos}\left(\alpha\right) \dot{\alpha}\end{array}$$

$$\begin{array}{l l}\ddot{y}&=&- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}^{2} + d \operatorname{cos}\left(\alpha\right) \ddot{\alpha}\end{array}$$

The terms in $\ddot{x}$ and $\ddot{y}$ proportional to $\dot{\alpha}^2$ are components of the centripetal acceleration on the body. As the name suggests, the [centripetal](http://en.wikipedia.org/wiki/Centripetal_force) acceleration is always directed to the center (towards the joint) when the segment is rotating. See the notebook [Kinematic chain](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/KinematicChain.ipynb) for more on that.

We could also use the methods of the Sympy physics/mechanics module and explicitly create a coordinate system in 3-D space, which will employ the versors $\hat{\mathbf{i}}, \hat{\mathbf{j}}, \hat{\mathbf{k}}$ for representing the vector components:

```python
G = CoordSys3D('')
r = d*cos(a)*G.i + d*sin(a)*G.j + 0*G.k
rd = r.diff(t)
rdd = r.diff(t, 2)

eq(r'\vec{\mathbf{r}}', r)
eq(r'\dot{\vec{\mathbf{r}}}', rd)
eq(r'\ddot{\vec{\mathbf{r}}}', rdd)
```

$$\begin{array}{l l}\vec{\mathbf{r}}&=&(d \cos{\left (\alpha{\left (t \right )} \right )})\mathbf{\hat{i}_{}} + (d \sin{\left (\alpha{\left (t \right )} \right )})\mathbf{\hat{j}_{}}\end{array}$$

$$\begin{array}{l l}\dot{\vec{\mathbf{r}}}&=&(- d \sin{\left (\alpha{\left (t \right )} \right )} \frac{d}{d t} \alpha{\left (t \right )})\mathbf{\hat{i}_{}} + (d \cos{\left (\alpha{\left (t \right )} \right )} \frac{d}{d t} \alpha{\left (t \right )})\mathbf{\hat{j}_{}}\end{array}$$

$$\begin{array}{l l}\ddot{\vec{\mathbf{r}}}&=&(- d \left(\sin{\left (\alpha{\left (t \right )} \right )} \frac{d^{2}}{d t^{2}} \alpha{\left (t \right )} + \cos{\left (\alpha{\left (t \right )} \right )} \left(\frac{d}{d t} \alpha{\left (t \right )}\right)^{2}\right))\mathbf{\hat{i}_{}} + (- d \left(\sin{\left (\alpha{\left (t \right )} \right )} \left(\frac{d}{d t} \alpha{\left (t \right )}\right)^{2} - \cos{\left (\alpha{\left (t \right )} \right )} \frac{d^{2}}{d t^{2}} \alpha{\left (t \right )}\right))\mathbf{\hat{j}_{}}\end{array}$$

But for now, let's continue writing the components ourselves.

As an exercise, let's go back to the Newton-Euler equation for the sum of torques around the center of mass where the torques due to the joint reaction forces are explicit.
From the equation for the the sum of forces, hence we have expressions for the linear accelerations, we can isolate the reaction forces and substitute them on the equation for the torques. With a little help from Sympy:

```python
m, I, g = symbols('m I g', positive=True)
Fex, Fey = symbols('F_ex F_ey')
add = a.diff(t, 2)
```

```python
Frx = m*xdd - Fex
Fry = m*ydd + m*g - Fey
eq(r'F_{rx}', Frx)
eq(r'F_{ry}', Fry)
```

$$\begin{array}{l l}F_{rx}&=&- F_{ex} + m \left(- d \operatorname{sin}\left(\alpha\right) \ddot{\alpha} - d \operatorname{cos}\left(\alpha\right) \dot{\alpha}^{2}\right)\end{array}$$

$$\begin{array}{l l}F_{ry}&=&- F_{ey} + g m + m \left(- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}^{2} + d \operatorname{cos}\left(\alpha\right) \ddot{\alpha}\right)\end{array}$$

```python
T = I*add - d*sin(a)*Frx + d*cos(a)*Fry + (L-d)*sin(a)*Fex - (L-d)*cos(a)*Fey
eq(r'T', T)
```

$$\begin{array}{l l}T&=&F_{ex} \left(L - d\right) \operatorname{sin}\left(\alpha\right) - F_{ey} \left(L - d\right) \operatorname{cos}\left(\alpha\right) + I \ddot{\alpha} - d \left(- F_{ex} + m \left(- d \operatorname{sin}\left(\alpha\right) \ddot{\alpha} - d \operatorname{cos}\left(\alpha\right) \dot{\alpha}^{2}\right)\right) \operatorname{sin}\left(\alpha\right) + d \left(- F_{ey} + g m + m \left(- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}^{2} + d \operatorname{cos}\left(\alpha\right) \ddot{\alpha}\right)\right) \operatorname{cos}\left(\alpha\right)\end{array}$$

This equation for the torques around the center of mass of only one rotating segment seems too complicated. The equation we derived before for the torques around the joint was much simpler. However, if we look at the terms on this last equation, we can simplify most of them. Let's use Sympy to simplify this equation:

```python
T = simplify(T)
eq(r'T', T)
```

$$\begin{array}{l l}T&=&F_{ex} L \operatorname{sin}\left(\alpha\right) - F_{ey} L \operatorname{cos}\left(\alpha\right) + I \ddot{\alpha} + d^{2} m \ddot{\alpha} + d g m \operatorname{cos}\left(\alpha\right)\end{array}$$

And we are back to the more simple equation we've seen before. The first two terms on the right side are the torque due to the external force, the third and fourth are the moment of inertia around the joint (use the theorem of parallel axis) times the acceleration, and the last term is the gravitational torque.

But what happened with all the other terms in the equation?

First, the terms proportional to the angular acceleration were just components from each direction of the 'inertial' torque that when summed resulted in $md^2\ddot{\alpha}$.
Second, the terms proportional to $\dot{\alpha}^2$ are components of the torque due to the centripetal force (acceleration). But the centripetal force passes through the joint as well as through the center of mass, i.e., it has zero lever arm and this torque should be zero. Indeed, when summed these terms are canceled out.

### The Jacobian matrix

Another way to deduce the velocity and acceleration of a point at the rotating link is to use the [Jacobian matrix](http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant) (see [Kinematic chain](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/KinematicChain.ipynb)).
Remember that in the context of kinematic chains, the Jacobian relates changes in the joint space to changes in the Cartesian space. The Jacobian is a matrix of all first-order partial derivatives of the linear position vector of the endpoint with respect to the angular position vector.
For the center of mass of the planar one-link system, this means that the Jacobian matrix is:

\begin{equation}
\mathbf{J} \quad=\quad
\begin{bmatrix}
\dfrac{\partial x}{\partial \alpha} \\
\dfrac{\partial y}{\partial \alpha} \\
\end{bmatrix}
\label{}
\end{equation}

```python
r = Matrix([x, y])
J = r.diff(a)
eq(r'\mathbf{J}', J)
```

$$\begin{array}{l l}\mathbf{J}&=&\left[\begin{matrix}- d \operatorname{sin}\left(\alpha\right)\\d \operatorname{cos}\left(\alpha\right)\end{matrix}\right]\end{array}$$

And Sympy has a function to calculate the Jacobian:

```python
J = r.jacobian([a])
eq(r'\mathbf{J}', J)
```

$$\begin{array}{l l}\mathbf{J}&=&\left[\begin{matrix}- d \operatorname{sin}\left(\alpha\right)\\d \operatorname{cos}\left(\alpha\right)\end{matrix}\right]\end{array}$$

The linear velocity of a point in the link will be given by the product between the Jacobian of the kinematic link and its angular velocity:

\begin{equation}
\vec{\mathbf{v}} \quad=\quad \mathbf{J} \dot{\vec{\alpha}}
\label{}
\end{equation}

Using Sympy, the linear velocity of the center of mass is:

```python
vel = J*a.diff(t)
eq(r'\begin{bmatrix} \dot{x} \\ \dot{y} \end{bmatrix}', vel)
```

$$\begin{array}{l l}\begin{bmatrix} \dot{x} \\ \dot{y} \end{bmatrix}&=&\left[\begin{matrix}- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}\\d \operatorname{cos}\left(\alpha\right) \dot{\alpha}\end{matrix}\right]\end{array}$$

And the linear acceleration of a point in the link will be given by the derivative of this last expression:

\begin{equation}
\vec{\mathbf{a}} \quad=\quad \dot{\mathbf{J}} \dot{\vec{\alpha}} + \mathbf{J} \ddot{\vec{\alpha}}
\label{}
\end{equation}

And using Sympy again, the linear acceleration of the center of mass is:

```python
acc = (J*a.diff(t)).diff(t)
eq(r'\begin{bmatrix} \ddot{x} \\ \ddot{y} \end{bmatrix}', acc)
```

$$\begin{array}{l l}\begin{bmatrix} \ddot{x} \\ \ddot{y} \end{bmatrix}&=&\left[\begin{matrix}- d \operatorname{sin}\left(\alpha\right) \ddot{\alpha} - d \operatorname{cos}\left(\alpha\right) \dot{\alpha}^{2}\\- d \operatorname{sin}\left(\alpha\right) \dot{\alpha}^{2} + d \operatorname{cos}\left(\alpha\right) \ddot{\alpha}\end{matrix}\right]\end{array}$$

Same expressions as before.

We can also use the Jacobian matrix to calculate the torque due to a force on the link:

\begin{equation}
T \quad=\quad \mathbf{J}^T \begin{bmatrix} F_{ex} \\ F_{ey} \end{bmatrix}
\label{}
\end{equation}

```python
Te = J.T*Matrix((Fex, Fey))
eq(r'T_e', Te[0])
```

$$\begin{array}{l l}T_e&=&- F_{ex} d \operatorname{sin}\left(\alpha\right) + F_{ey} d \operatorname{cos}\left(\alpha\right)\end{array}$$

Where in this case we considered that the force was applied to the center of mass, just because we already had the Jacobian calculated at that position.

We could simulate the movement of this one-link system for a typical human movement to understand the magnitude of these physical quantities.
The reader is invited to that now. We will perform this simulation for a two-link system next

## Two-link system

Let's study the dynamics of a planar double inverted pendulum (see Figure 2) as a model of two interconnected segments in the human body with an external force acting on the distal segment. Once again, we will consider that there are muscles around each joint and they generate torques.

Figure 2. Planar double inverted pendulum connected by hinge joints with joint actuators and corresponding free body diagrams. See text for notation convention.

The following notation convention will be used for this problem:
 - Subscript $i$ runs 1 or 2 meaning first (most proximal) or second joint when referring to angles, joint moments, or joint reaction forces, or meaning first or second segment when referring to everything else.
 - $L_i$ is the length of segment $i$.
 - $d_i$ is the distance from the proximal joint of segment $i$ to its center of mass position.
 - $m_i$ is the mass of segment $i$.
 - $g$ is the gravitational acceleration (+).
 - $\alpha_i$ is the angular position of joint $i$ in the joint space, $\ddot{\alpha_i}$ is the corresponding angular acceleration.
 - $\theta_i$ is the angular position of joint $i$ in the segmental space w.r.t. horizontal, $\theta_1=\alpha_1$ and $\theta_2=\alpha_1+\alpha_2$.
 - $I_i$ is the moment of inertia of segment $i$ around its center of mass position.
 - $F_{ri}$ is the reaction force at joint $i$.
 - $F_{e}$ is the external force acting on the distal segment.
 - $T_i$ is the moment of force (torque) at joint $i$.

Hence we know we will need the linear accelerations for solving the Newton-Euler equations, let's deduce them first.

```python
t = Symbol('t')
d1, d2, L1, L2 = symbols('d1, d2, L_1 L_2', positive=True)
a1, a2 = dynamicsymbols('alpha1 alpha2')
a1d, a2d = a1.diff(t), a2.diff(t)
a1dd, a2dd = a1.diff(t, 2), a2.diff(t, 2)
```

### Kinematics

Once again, if the angular positions $\alpha_1(t)$ and $\alpha_2(t)$ are known, the coordinates $(x_1(t), y_1(t))$ and $(x_2(t), y_2(t))$ and their derivatives can be readily determined (by forward kinematics):

#### Link 1

```python
x1, y1 = d1*cos(a1), d1*sin(a1)
x1d, y1d = x1.diff(t), y1.diff(t)
x1dd, y1dd = x1d.diff(t), y1d.diff(t)

eq(r'x_1', x1)
eq(r'\dot{x_1}', x1d)
eq(r'\ddot{x_1}', x1dd)
eq(r'y_1', y1)
eq(r'\dot{y_1}', y1d)
eq(r'\ddot{y_1}', y1dd)
```

$$\begin{array}{l l}x_1&=&d_{1} \operatorname{cos}\left(\alpha_{1}\right)\end{array}$$

$$\begin{array}{l l}\dot{x_1}&=&- d_{1} \operatorname{sin}\left(\alpha_{1}\right) \dot{\alpha}_{1}\end{array}$$

$$\begin{array}{l l}\ddot{x_1}&=&- d_{1} \operatorname{sin}\left(\alpha_{1}\right) \ddot{\alpha}_{1} - d_{1} \operatorname{cos}\left(\alpha_{1}\right) \dot{\alpha}_{1}^{2}\end{array}$$

$$\begin{array}{l l}y_1&=&d_{1} \operatorname{sin}\left(\alpha_{1}\right)\end{array}$$

$$\begin{array}{l l}\dot{y_1}&=&d_{1} \operatorname{cos}\left(\alpha_{1}\right) \dot{\alpha}_{1}\end{array}$$

$$\begin{array}{l l}\ddot{y_1}&=&- d_{1} \operatorname{sin}\left(\alpha_{1}\right) \dot{\alpha}_{1}^{2} + d_{1} \operatorname{cos}\left(\alpha_{1}\right) \ddot{\alpha}_{1}\end{array}$$

#### Link 2

```python
x2, y2 = L1*cos(a1) + d2*cos(a1+a2), L1*sin(a1) + d2*sin(a1+a2)
x2d, y2d = x2.diff(t), y2.diff(t)
x2dd, y2dd = x2d.diff(t), y2d.diff(t)

eq(r'x_2', x2)
eq(r'\dot{x_2}', x2d)
eq(r'\ddot{x_2}', x2dd)
eq(r'y_2', y2)
eq(r'\dot{y_2}', y2d)
eq(r'\ddot{y_2}', y2dd)
```

$$\begin{array}{l l}x_2&=&L_{1} \operatorname{cos}\left(\alpha_{1}\right) + d_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}\dot{x_2}&=&- L_{1} \operatorname{sin}\left(\alpha_{1}\right) \dot{\alpha}_{1} - d_{2} \left(\dot{\alpha}_{1} + \dot{\alpha}_{2}\right) \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}\ddot{x_2}&=&- L_{1} \operatorname{sin}\left(\alpha_{1}\right) \ddot{\alpha}_{1} - L_{1} \operatorname{cos}\left(\alpha_{1}\right) \dot{\alpha}_{1}^{2} - d_{2} \left(\dot{\alpha}_{1} + \dot{\alpha}_{2}\right)^{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right) - d_{2} \left(\ddot{\alpha}_{1} + \ddot{\alpha}_{2}\right) \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}y_2&=&L_{1} \operatorname{sin}\left(\alpha_{1}\right) + d_{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}\dot{y_2}&=&L_{1} \operatorname{cos}\left(\alpha_{1}\right) \dot{\alpha}_{1} + d_{2} \left(\dot{\alpha}_{1} + \dot{\alpha}_{2}\right) \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}\ddot{y_2}&=&- L_{1} \operatorname{sin}\left(\alpha_{1}\right) \dot{\alpha}_{1}^{2} + L_{1} \operatorname{cos}\left(\alpha_{1}\right) \ddot{\alpha}_{1} - d_{2} \left(\dot{\alpha}_{1} + \dot{\alpha}_{2}\right)^{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right) + d_{2} \left(\ddot{\alpha}_{1} + \ddot{\alpha}_{2}\right) \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

Inspecting the equations above, we see a new kind of acceleration, proportional to $\dot{\alpha_1}\dot{\alpha_2}$. This acceleration is due to the [Coriolis effect](http://en.wikipedia.org/wiki/Coriolis_effect) and is  present only when there are movement in the two joints.

### Jacobian matrix for the two-link system

The Jacobian matrix for the two-link system w.r.t. the center of mass of the second link is:

```python
r2 = Matrix([[x2, y2]])
J2 = r2.jacobian([a1, a2])
eq(r'\mathbf{J}', J2)
```

$$\begin{array}{l l}\mathbf{J}&=&\left[\begin{matrix}- L_{1} \operatorname{sin}\left(\alpha_{1}\right) - d_{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right) & - d_{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right)\\L_{1} \operatorname{cos}\left(\alpha_{1}\right) + d_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right) & d_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{matrix}\right]\end{array}$$

### Kinetics

From the free body diagrams, the Newton-Euler equations for links 1 and 2 of the planar double inverted pendulum are:

\begin{equation}
\begin{array}{l l}
F_{r2x} + F_{e,x} &=& m_2\ddot{x}_{2} \\
\\
F_{r2y} - m_2g + F_{e,y} &=& m_2\ddot{y}_{2} \\
\\
T_2 + d_2F_{r2x}\sin(\alpha_1+\alpha_2) - d_2F_{r2y}\cos(\alpha_1+\alpha_2) - (L_2-d_2)F_{e,x}\sin(\alpha_1+\alpha_2) - (L_2-d_2)F_{e,y}\cos(\alpha_1+\alpha_2) &=& I_{2}(\ddot{\alpha}_1+\ddot{\alpha}_2) \\
\\
F_{r1x} - F_{r2x} &=& m_1\ddot{x}_{1} \\
\\
F_{r1y} - F_{r2y} - m_1g &=& m_1\ddot{y}_{1} \\
\\
T_1 - T_2 + d_1F_{r1x}\sin\alpha_1 - d_1F_{r1y}\cos\alpha_1 + (L_1-d_1)F_{r2x}\sin\alpha_1 - (L_1-d_1)F_{r2y}\cos\alpha_1 &=& I_{1}\ddot{\alpha}_1
\end{array}
\label{}
\end{equation}

If we want to determine the joint torques and we know the kinematics of the links, the inverse dynamics approach, we isolate the joint torques in the equations above, start solving for link 2 and then link 1. To determine the kinematics knowing the joint torques, the direct dynamics approach, we isolate the joint angular accelerations in the equations above and solve the ordinary differential equations.

Let's express the equations for the torques substituting the terms we know:

```python
m1, m2, I1, I2, g = symbols('m_1, m_2, I_1 I_2 g', positive=True)
Fex, Fey = symbols('F_ex F_ey')
```

```python
# link 2
Fr2x = m2*x2dd - Fex
Fr2y = m2*y2dd + m2*g - Fey
T2 = I2*(a1dd+a2dd) - d2*Fr2x*sin(a1+a2) + d2*Fr2y*cos(a1+a2) + (L2-d2)*Fex*sin(a1+a2) - (L2-d2)*Fey*cos(a1+a2)
T2 = simplify(T2)
# link 1
Fr1x = m1*x1dd + Fr2x
Fr1y = m1*y1dd + Fr2y + m1*g
T1 = I1*a1dd + T2 - d1*Fr1x*sin(a1) + d1*Fr1y*cos(a1) - (L1-d1)*Fr2x*sin(a1) + (L1-d1)*Fr2y*cos(a1)
T1 = simplify(T1)
```

The expressions for the joint moments of force (torques) are:

```python
eq(r'T_1', T1)
eq(r'T_2', T2)
```

$$\begin{array}{l l}T_1&=&F_{ex} L_{1} \operatorname{sin}\left(\alpha_{1}\right) + F_{ex} L_{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right) - F_{ey} L_{1} \operatorname{cos}\left(\alpha_{1}\right) - F_{ey} L_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right) + I_{1} \ddot{\alpha}_{1} + I_{2} \ddot{\alpha}_{1} + I_{2} \ddot{\alpha}_{2} + L_{1}^{2} m_{2} \ddot{\alpha}_{1} - 2 L_{1} d_{2} m_{2} \operatorname{sin}\left(\alpha_{2}\right) \dot{\alpha}_{1} \dot{\alpha}_{2} - L_{1} d_{2} m_{2} \operatorname{sin}\left(\alpha_{2}\right) \dot{\alpha}_{2}^{2} + 2 L_{1} d_{2} m_{2} \operatorname{cos}\left(\alpha_{2}\right) \ddot{\alpha}_{1} + L_{1} d_{2} m_{2} \operatorname{cos}\left(\alpha_{2}\right) \ddot{\alpha}_{2} + L_{1} g m_{2} \operatorname{cos}\left(\alpha_{1}\right) + d_{1}^{2} m_{1} \ddot{\alpha}_{1} + d_{1} g m_{1} \operatorname{cos}\left(\alpha_{1}\right) + d_{2}^{2} m_{2} \ddot{\alpha}_{1} + d_{2}^{2} m_{2} \ddot{\alpha}_{2} + d_{2} g m_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

$$\begin{array}{l l}T_2&=&F_{ex} L_{2} \operatorname{sin}\left(\alpha_{1} + \alpha_{2}\right) - F_{ey} L_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right) + I_{2} \ddot{\alpha}_{1} + I_{2} \ddot{\alpha}_{2} + L_{1} d_{2} m_{2} \operatorname{sin}\left(\alpha_{2}\right) \dot{\alpha}_{1}^{2} + L_{1} d_{2} m_{2} \operatorname{cos}\left(\alpha_{2}\right) \ddot{\alpha}_{1} + d_{2}^{2} m_{2} \ddot{\alpha}_{1} + d_{2}^{2} m_{2} \ddot{\alpha}_{2} + d_{2} g m_{2} \operatorname{cos}\left(\alpha_{1} + \alpha_{2}\right)\end{array}$$

There is an elegant form to display the equations for the torques using generalized coordinates, $q=[\alpha_1, \alpha_2]^T$ and grouping the terms proportional to common quantities in matrices, see for example, Craig (2005, page 180), Pandy (2001), and Zatsiorsky (2002, page 383):

\begin{equation}
\tau \quad=\quad M(q)\ddot{q} + C(q,\dot{q}) + G(q) + E(q,\dot{q})
\label{}
\end{equation}

Where, for this two-link system:
- $\tau$ is a matrix (2x1) of joint torques;
- $M$ is the mass or inertia matrix (2x2);
- $\ddot{q}$ is a matrix (2x1) of angular accelerations;
- $C$ is a matrix (2x1) of [centipetal](http://en.wikipedia.org/wiki/Centripetal_force) and [Coriolis](http://en.wikipedia.org/wiki/Coriolis_effect) torques;
- $G$ is a matrix (2x1) of  gravitational torques;
- $E$ is a matrix (2x1) of external torques.

Let's use Sympy to display the equations for the torques in this new form:

```python
T1, T2 = T1.expand(), T2.expand()
q1, q2 = dynamicsymbols('q_1 q_2')
q1d, q2d = q1.diff(t), q2.diff(t)
q1dd, q2dd = q1.diff(t, 2), q2.diff(t, 2)
T1 = T1.subs({a1:q1, a2:q2, a1d:q1d, a2d:q2d, a1dd:q1dd, a2dd:q2dd})
T2 = T2.subs({a1:q1, a2:q2, a1d:q1d, a2d:q2d, a1dd:q1dd, a2dd:q2dd})
```

```python
M = Matrix(((simplify(T1.coeff(q1dd)), simplify(T1.coeff(q2dd))),
            (simplify(T2.coeff(q1dd)), simplify(T2.coeff(q2dd)))))
C = Matrix((simplify(T1.coeff(q1d**2)*q1d**2 + T1.coeff(q2d**2)*q2d**2 + T1.coeff(q1d*q2d)*q1d*q2d),
            simplify(T2.coeff(q1d**2)*q1d**2 + T2.coeff(q2d**2)*q2d**2 + T2.coeff(q1d*q2d)*q1d*q2d)))
G = Matrix((simplify(T1.coeff(g)*g),
            simplify(T2.coeff(g)*g)))
E = Matrix((simplify(T1.coeff(Fex)*Fex + T1.coeff(Fey)*Fey),
            simplify(T2.coeff(Fex)*Fex + T2.coeff(Fey)*Fey)))

display(Math(r'\begin{eqnarray}\tau&\quad=\quad&\begin{bmatrix}\tau_1\\ \tau_2\\ \end{bmatrix} \\' +
             r'M(q)&\quad=\quad&' + mlatex(M) + r'\\' +
             r'\ddot{q}&\quad=\quad&' + mlatex(Matrix((q1dd, q2dd))) + r'\\' +
             r'C(q,\dot{q})&\quad=\quad&' + mlatex(C) + r'\\' +
             r'G(q)&\quad=\quad&' + mlatex(G) + r'\\' +
             r'E(q,\dot{q})&\quad=\quad&' + mlatex(E) + r'\end{eqnarray}'))
```

$$\begin{eqnarray}\tau&\quad=\quad&\begin{bmatrix}\tau_1\\ \tau_2\\ \end{bmatrix} \\M(q)&\quad=\quad&\left[\begin{matrix}I_{1} + I_{2} + L_{1}^{2} m_{2} + 2 L_{1} d_{2} m_{2} \operatorname{cos}\left(q_{2}\right) + d_{1}^{2} m_{1} + d_{2}^{2} m_{2} & I_{2} + L_{1} d_{2} m_{2} \operatorname{cos}\left(q_{2}\right) + d_{2}^{2} m_{2}\\I_{2} + L_{1} d_{2} m_{2} \operatorname{cos}\left(q_{2}\right) + d_{2}^{2} m_{2} & I_{2} + d_{2}^{2} m_{2}\end{matrix}\right]\\\ddot{q}&\quad=\quad&\left[\begin{matrix}\ddot{q}_{1}\\\ddot{q}_{2}\end{matrix}\right]\\C(q,\dot{q})&\quad=\quad&\left[\begin{matrix}- L_{1} d_{2} m_{2} \left(2 \dot{q}_{1} + \dot{q}_{2}\right) \operatorname{sin}\left(q_{2}\right) \dot{q}_{2}\\L_{1} d_{2} m_{2} \operatorname{sin}\left(q_{2}\right) \dot{q}_{1}^{2}\end{matrix}\right]\\G(q)&\quad=\quad&\left[\begin{matrix}g \left(L_{1} m_{2} \operatorname{cos}\left(q_{1}\right) + d_{1} m_{1} \operatorname{cos}\left(q_{1}\right) + d_{2} m_{2} \operatorname{cos}\left(q_{1} + q_{2}\right)\right)\\d_{2} g m_{2} \operatorname{cos}\left(q_{1} + q_{2}\right)\end{matrix}\right]\\E(q,\dot{q})&\quad=\quad&\left[\begin{matrix}F_{ex} \left(L_{1} \operatorname{sin}\left(q_{1}\right) + L_{2} \operatorname{sin}\left(q_{1} + q_{2}\right)\right) - F_{ey} \left(L_{1} \operatorname{cos}\left(q_{1}\right) + L_{2} \operatorname{cos}\left(q_{1} + q_{2}\right)\right)\\L_{2} \left(F_{ex} \operatorname{sin}\left(q_{1} + q_{2}\right) - F_{ey} \operatorname{cos}\left(q_{1} + q_{2}\right)\right)\end{matrix}\right]\end{eqnarray}$$

With this convention, to perform inverse dynamics we would calculate:

\begin{equation}
\tau \quad=\quad M(q)\ddot{q} + C(q,\dot{q}) + G(q) + E(q,\dot{q})
\label{}
\end{equation}

And for direct dynamics we would solve the differential equation:

\begin{equation}
\ddot{q} \quad=\quad M(q)^{-1} \left[\tau - C(q,\dot{q}) - G(q) - E(q,\dot{q}) \right]
\label{}
\end{equation}

The advantage of calculating analytically the derivatives of the position vector as function of the joint angles and using the notation above is that each term that contributes to each joint torque or acceleration can be easily identified.

#### Coupling (or interaction) effects

The two terms off the main diagonal in the inertia matrix (which are the same) and the centripetal and Coriolis terms represent the effects of the movement (nonzero velocity) of one joint over the other. These torques are referred as coupling or interaction effects (see for example Hollerbach and Flash (1982) for an application of this concept in the study of the motor control of the upper limb movement).

#### Planar double pendulum

Using the same equations above, one can represent a planar double pendulum (hanging from the top, not inverted) considering the angles $\alpha_1$ and $\alpha_2$ negative, e.g., at $\alpha_1=-90^o$ and $\alpha_2=0$ the pendulum is hanging vertical.

#### WARNING: $F_r$ is not the actual joint reaction force!

For these two examples, in the Newton-Euler equations based on the free body diagrams we represented the consequences of all possible muscle forces on a joint as a net muscle torque and all forces acting on a joint as a resultant joint reaction force. That is, all forces between segments were represented as a resultant force that doesn't generate torque and a force couple (or free moment) that only generates torque. This is an important principle in mechanics of rigid bodies, see for example [this text](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/FreeBodyDiagram.ipynb). However, this principle creates the unrealistic notion that the sum of forces is applied directly on the joint (which has no further implication for a rigid body), but it is inaccurate for the understanding of the local effects on the joint. So, if we are trying to understand the stress on the joint or mechanisms of joint injury, the forces acting on the joint and on the rest of the segment must be considered individually.

#### Determination of muscle force

The torque $T$ exerted by a muscle is given by the product between the muscle-tendon moment arm $r$ and its force $F$. For the human body, there is more than one muscle crossing a joint and several joints. In such case, the torques due to the muscles are expressed in the following matrix form considering $n$ joints and $m$ muscles:

\begin{eqnarray}
\begin{bmatrix} T_1 \\ \vdots \\ T_n \end{bmatrix} &\quad=\quad& \begin{bmatrix} r_{11} & \cdots & r_{1m} \\ \vdots & \ddots & \vdots \\ r_{n1} & \cdots & r_{nm} \end{bmatrix} \begin{bmatrix} F_1 \\ \vdots \\ F_m \end{bmatrix}
\label{}
\end{eqnarray}

Where $r_{nm}$ is the moment arm about joint $n$ of the muscle $m$.
In the example of the two-link system, we sketched two uniarticular muscles for each of the two joints, consequently:

\begin{eqnarray}
\begin{bmatrix} T_1 \\ T_2 \end{bmatrix} &\quad=\quad& \begin{bmatrix} r_{1,ext} & -r_{1,flex} & 0 & 0 \\ 0 & 0 & r_{1,ext} & -r_{1,flex} \end{bmatrix} \begin{bmatrix} F_{1,ext} \\ F_{1,flex} \\ F_{2,ext} \\ F_{2,flex} \end{bmatrix}
\label{}
\end{eqnarray}

Note the opposite signs for the moment arms of the extension and flexion muscles hence they generate opposite torques. We could have represented the opposite signs in the muscle forces instead of in the moment arms.

The moment arm of a muscle varies with the motion of the joints it crosses. In this case, using the [virtual work principle](http://en.wikipedia.org/wiki/Virtual_work) the moment arm can be given by (Sherman et al., 2013; Nigg and Herzog, 2006, page 634):

\begin{equation}
r(q) \quad=\quad \dfrac{\partial L_{MT}(q)}{\partial q}
\label{}
\end{equation}

Where $L_{MT}(q)$ is the length of the muscle-tendon unit expressed as a function of angle $q$.

For the simulation of human movement, muscles can be modeled as [Hill-type muscles](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/MuscleSimulation.ipynb), the torques they generate are given by the matrix above, and this matrix is entered in the ODE for a multibody system dynamics we deduced before:

\begin{equation}
\ddot{q} \quad=\quad M(q)^{-1} \left[R_{MT}(q)F_{MT}(a,L_{MT},\dot{L}_{MT}) - C(q,\dot{q}) - G(q) - E(q,\dot{q}) \right]
\label{}
\end{equation}

Where $R_{MT}$ and $F_{MT}$ are matrices for the moment arms and muscle-tendon forces, respectively.
This ODE is then solved numerically given initial values; but this problem is far from trivial for a simulation with several segments and muscles.

## Numerical simulation of inverse dynamics

Let's simulate a voluntary movement of the upper limb using the planar two-link system as a model in order to visualize the contribution of each torque term.

We will ignore the muscle dynamics and we will calculate the joint torques necessary to move the upper limb from one point to another under the assumption that the movement is performed with the smoothest trajectory possible. I.e., the movement is performed with a [minimum-jerk trajectory](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/MinimumJerkHypothesis.ipynb), a hypothesis about control of voluntary movements proposed by Flash and Hogan (1985).

Once we determine the desired trajectory, we can calculate the velocity and acceleration of the segments and combine with anthropometric measures to calculate the joint torques necessary to move the segments. This means we will perform inverse dynamics.

Let's simulate a slow (4 s) and a fast (0.5 s) movement of the upper limb starting at the anatomical neutral position (upper limb at the side of the trunk) and ending with the upper arm forward at horizontal and elbow flexed at 90 degrees.

First, let's import the necessary Python libraries and customize the environment:

```python
import numpy as np
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['lines.linewidth'] = 3
matplotlib.rcParams['font.size'] = 13
matplotlib.rcParams['lines.markersize'] = 5
matplotlib.rc('axes', grid=True, labelsize=14, titlesize=16, ymargin=0.05)
matplotlib.rc('legend', numpoints=1, fontsize=11)
import sys
sys.path.insert(1, r'./../functions')  # add to pythonpath
```

Let's take the anthropometric data from Dempster's model (see [Body segment parameters](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/BodySegmentParameters.ipynb)):

```python
height, mass = 1.70,               70  # m, kg
L1n, L2n     = 0.188*height,       0.253*height
d1n, d2n     = 0.436*L1n,          0.682*L2n
m1n, m2n     = 0.0280*mass,        0.0220*mass
rg1n, rg2n   = 0.322,              0.468
I1n, I2n     = m1n*(rg1n*L1n)**2,  m2n*(rg2n*L2n)**2
```

Considering these lengths, the initial and final positions of the endpoint (finger tip) for the simulated movement will be:

```python
xi, yi = 0, -L1n-L2n
xf, yf = L1n, L2n
gn = 9.81  # gravity acceleration m/s2
```

### Slow movement

```python
duration = 4  # seconds
```

The endpoint minimum jerk trajectory will be (see [Kinematic chain in a plane (2D)](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/KinematicChain.ipynb)):

```python
from minjerk import minjerk
```

```python
time, rlin, vlin, alin, jlin = minjerk([xi, yi], [xf, yf], duration=duration)
```

Let's find the joint angles to produce this minimum-jerk trajectory (inverse kinematics):

```python
from invkin2_2d import invkin
```

```python
rang = invkin(time, rlin, L1=L1n, L2=L2n)
```

For the joint torques, we need to calculate the angular velocity and acceleration. Let's do that using numerical differentiation:

```python
def diff_c(ang, duration):
    """Numerical differentiations using the central difference for the angular data.
    """
    # central difference (f(x+h)-f(x-h))/(2*h)
    dt = duration/(ang.shape[0]-1)
    vang = np.empty_like(rang)
    aang = np.empty_like(rang)
    vang[:, 0] = np.gradient(rang[:, 0], dt)
    vang[:, 1] = np.gradient(rang[:, 1], dt)
    aang[:, 0] = np.gradient(vang[:, 0], dt)
    aang[:, 1] = np.gradient(vang[:, 1], dt)

    _, ax = plt.subplots(1, 3, sharex=True, figsize=(10, 3))
    ax[0].plot(time, rang*180/np.pi)
    ax[0].legend(['Ang 1', 'Ang 2'], framealpha=.5, loc='best')
    ax[1].plot(time, vang*180/np.pi)
    ax[2].plot(time, aang*180/np.pi)
    ylabel = [r'Displacement [$\mathrm{^o}$]', r'Velocity [$\mathrm{^o/s}$]',
              r'Acceleration [$\mathrm{^o/s^2}$]']
    for i, axi in enumerate(ax):
        axi.set_xlabel('Time [$s$]')
        axi.set_ylabel(ylabel[i])
        axi.xaxis.set_major_locator(plt.MaxNLocator(4))
        axi.yaxis.set_major_locator(plt.MaxNLocator(4))
    plt.tight_layout()
    plt.show()

    return vang, aang

vang, aang = diff_c(rang, duration)
```

```python
def dyna(time, L1n, L2n, d1n, d2n, m1n, m2n, gn, I1n, I2n, q1, q2, rang, vang, aang, Fexn, Feyn, M, C, G, E):
    """Numerical calculation and plot for the torques of a planar two-link system.
    """
    from sympy import lambdify, symbols

    Mfun  = lambdify((I1, I2, L1, L2, d1, d2, m1, m2, q1, q2), M, 'numpy')
    Mn    = Mfun(I1n, I2n, L1n, L2n, d1n, d2n, m1n, m2n, rang[:, 0], rang[:, 1])
    M00   = Mn[0, 0]*aang[:, 0]
    M01   = Mn[0, 1]*aang[:, 1]
    M10   = Mn[1, 0]*aang[:, 0]
    M11   = Mn[1, 1]*aang[:, 1]
    Q1d, Q2d = symbols('Q1d Q2d')
    dicti = {q1.diff(t, 1):Q1d, q2.diff(t, 1):Q2d}
    C0fun = lambdify((L1, d2, m2, q2, Q1d, Q2d), C[0].subs(dicti), 'numpy')
    C0    = C0fun(L1n, d2n, m2n, rang[:, 1], vang[:, 0], vang[:, 1])
    C1fun = lambdify((L1, d2, m2, q2, Q1d, Q2d), C[1].subs(dicti), 'numpy')
    C1    = C1fun(L1n, d2n, m2n, rang[:, 1], vang[:, 0], vang[:, 1])
    G0fun = lambdify((L1, d1, d2, m1, m2, g, q1, q2), G[0], 'numpy')
    G0    = G0fun(L1n, d1n, d2n, m1n, m2n, gn, rang[:, 0], rang[:, 1])
    G1fun = lambdify((L1, d1, d2, m1, m2, g, q1, q2), G[1], 'numpy')
    G1    = G1fun(L1n, d1n, d2n, m1n, m2n, gn, rang[:, 0], rang[:, 1])
    E0fun = lambdify((L1, L2, q1, q2, Fex, Fey), E[0], 'numpy')
    E0    = E0fun(L1n, L2n, rang[:, 0], rang[:, 1], 0, 0)
    E1fun = lambdify((L1, L2, q1, q2, Fex, Fey), E[1], 'numpy')
    E1    = E1fun(L1n, L2n, rang[:, 0], rang[:, 1], Fexn, Feyn)

    fig, ax = plt.subplots(1, 2, sharex=True, squeeze=True, figsize=(10, 4))
    ax[0].plot(time, M00+M01)
    ax[0].plot(time, C0)
    ax[0].plot(time, G0)
    ax[0].plot(time, E0)
    #ax[0].plot(time, M00+M01+C0+G0, 'k:', linewidth=4)
    ax[0].set_ylabel(r'Torque [Nm]')
    ax[0].set_title('Joint 1')
    ax[1].plot(time, M10+M11, label='Mass/Inertia')
    ax[1].plot(time, C1, label='Centripetal/Coriolis       ')
    ax[1].plot(time, G1, label='Gravitational')
    ax[1].plot(time, E1, label='External')
    #ax[1].plot(time, M10+M11+C1+G1, 'k:', linewidth=4, label='Muscular (sum)')
    ax[1].set_title('Joint 2')
    fig.legend(framealpha=.5, bbox_to_anchor=(1.15, 0.95), fontsize=12)
    for i, axi in enumerate(ax):
        axi.set_xlabel('Time [$s$]')
        axi.xaxis.set_major_locator(plt.MaxNLocator(4))
        axi.yaxis.set_major_locator(plt.MaxNLocator(4))
    plt.show()

    return M00, M01, M10, M11, C0, C1, G0, G1, E0, E1
```

```python
Fexn, Feyn = 0, 0
M00, M01, M10, M11, C0, C1, G0, G1, E0, E1 = dyna(time, L1n, L2n, d1n, d2n, m1n, m2n, gn, I1n, I2n,
                                                  q1, q2, rang, vang, aang, Fexn, Feyn, M, C, G, E)
T1a = M00+M01+C0+G0+E0
T2a = M10+M11+C1+G1+E1
```

The joint torques essentially compensate the gravitational torque.

### Fast movement

Let's see what is changed for a fast movement:

```python
duration = 0.5  # seconds
time, rlin, vlin, alin, jlin = minjerk([xi, yi], [xf, yf], duration=duration)
rang = invkin(time, rlin, L1=L1n, L2=L2n)
vang, aang = diff_c(rang, duration)
M00, M01, M10, M11, C0, C1, G0, G1, E0, E1 = dyna(time, L1n, L2n, d1n, d2n, m1n, m2n, gn, I1n, I2n,
                                                  q1, q2, rang, vang, aang, Fexn, Feyn, M, C, G, E)
T1b = M00+M01+C0+G0+E0
T2b = M10+M11+C1+G1+E1
```

The interaction torques are larger than the gravitational torques for most part of the movement.

### Fast movement in the horizontal plane

Let's simulate a fast movement in the horizontal plane:

```python
gn = 0  # gravity acceleration m/s2
M00, M01, M10, M11, C0, C1, G0, G1, E0, E1 = dyna(time, L1n, L2n, d1n, d2n, m1n, m2n, gn, I1n, I2n,
                                                  q1, q2, rang, vang, aang, Fexn, Feyn, M, C, G, E)
T1b = M00+M01+C0+G0+E0
T2b = M10+M11+C1+G1+E1
```

## Numerical simulation of direct dynamics

Remember that for direct dynamics we want to solve the following differential equation:

\begin{equation}
\ddot{q} \quad=\quad M(q)^{-1} \left[\tau - C(q,\dot{q}) - G(q) - E(q,\dot{q}) \right]
\label{}
\end{equation}

Let's use the Euler method for solving this equation numerically.

First, transforming the equation above into a system of two first-order ODE:

\begin{equation}
\left\{
\begin{array}{l l}
\dfrac{\mathrm{d} q}{\mathrm{d}t} &=& \dot{q}, \quad &q(t_0) = q_0
\\
\dfrac{\mathrm{d} \dot{q}}{\mathrm{d}t} &=& M(q)^{-1} \left[\tau - C(q,\dot{q}) - G(q) - E(q,\dot{q}) \right], \quad &\dot{q}(t_0) = \dot{q}_0
\end{array}
\right.
\label{}
\end{equation}

Second, we would write a function for the calculation of the system states and another function for the Euler method.

Third, now joint torques are the input to the system; just to cast out nines, let's choose as input torques the output torques of the inverse dynamics solution we calculated before.

Fourth, plot everything.

Easy peasy.

## Exercises

1. Derive the equations of motion for a single pendulum (not inverted).
2. Derive the equations of motion for a double pendulum (not inverted).
3. For the one-link system, simulate a typical trajectory to calculate the joint torque (i.e., perform inverse dynamics).
4. For the one-link system, simulate a typical joint torque to calculate the trajectory (i.e., perform direct dynamics).
5. Consider the double pendulum moving in the horizontal plane and with no external force. Find out the type of movement and which torque terms are changed when:
  a) $\dot{\alpha}_1=0^o$
  b) $\alpha_2=0^o$
  c) $\dot{\alpha}_2=0^o$
  d) $2\alpha_1+\alpha_2=180^o$ (hint: a two-link system with this configuration is called polar manipulator)
6. Derive the equations of motion and the torque terms using angles in the segmental space $(\theta_1,\,\theta_2)$.
7. Run the numerical simulations for the torques with different parameters.

## References

- Craig JJ (2005) [Introduction to Robotics: Mechanics and Control](http://books.google.com.br/books?id=MqMeAQAAIAAJ). 3rd Edition. Prentice Hall.
- Flash T, Hogan N (1985) [The coordination of arm movements: an experimentally confirmed mathematical model](http://www.jneurosci.org/cgi/reprint/5/7/1688.pdf). Journal of Neuroscience, 5, 1688-1703.
- Hollerbach JM, Flash T (1982) [Dynamic interactions between limb segments during planar arm movement](http://link.springer.com/article/10.1007%2FBF00353957). Biological Cybernetics, 44, 67-77.
- Nigg BM and Herzog W (2006) [Biomechanics of the Musculo-skeletal System](https://books.google.com.br/books?id=hOIeAQAAIAAJ&dq=editions:ISBN0470017678). 3rd Edition. Wiley.
- Pandy MG (2001) [Computer modeling and simulation](https://drive.google.com/open?id=0BxbW72zV7WmUbXZBR2VRMnF5UTA&authuser=0). Annu. Rev. Biomed. Eng., 3, 245–73.
- Sherman MA, Seth A, Delp SL (2013) [What is a moment arm? Calculating muscle effectiveness in biomechanical models using generalized coordinates](http://simtk-confluence.stanford.edu:8080/download/attachments/3376330/ShermanSethDelp-2013-WhatIsMuscleMomentArm-Final2-DETC2013-13633.pdf?version=1&modificationDate=1369103515834) in Proc. ASME Int. Design Engineering Technical Conferences (IDETC), Portland, OR, USA.
- Zajac FE (1993) [Muscle coordination of movement: a perspective](http://e.guigon.free.fr/rsc/article/Zajac93.pdf). J Biomech., 26, Suppl 1:109-24.
- Zajac FE, Gordon ME (1989) [Determining muscle's force and action in multi-articular movement](https://drive.google.com/open?id=0BxbW72zV7WmUcC1zSGpEOUxhWXM&authuser=0). Exercise and Sport Sciences Reviews, 17, 187-230.
- Zatsiorsky VM (2002) [Kinetics of human motion](http://books.google.com.br/books?id=wp3zt7oF8a0C). Human Kinetics.


# Structure Learning

> In the last two sessions we studied how to estimate the parameters of both Bayesian Networks and Markov Networks. We made a strong assumption that we know the network structure in advance.
>
> In this session we will takcle the task of learning in situations when we don't know the structure of the Bayesian network in advance.

> **Objetives:**
> - To understand the maximum likelihood score for structure learning in Bayesian Networks.
> - To study the BIC score for structure learning in Bayesian Networks.

> **References:**
> - Probabilistic Graphical Models: Principles and Techniques, By Daphne Koller and Nir Friedman. Ch. 18.
> - Mastering Probabilistic Graphical Models Using Python, By Ankur Ankan and Abinash Panda. Ch. 5.
> - Probabilistic Graphical Models Specialization, offered through Coursera. Prof. Daphne Koller.

___

## 1. Overview

### 1.1. Why would we be interested in learning a structure?

- To learn a model for new queries, when the domain expertise is not enough.

- For structure discovery, when inferring network structure is a goal itself.

### 1.2. Importance of accurate structure

Let's assume that the true model for some situation is the following:

```python
from IPython.display import Image
```

```python
Image("figures/AccurateStructure.png")
```

*What happens if an arc is missing?*

```python
Image("figures/AccurateStructure1.png")
```

- The model encodes inccorrect independencies.
- The correct distribution $P^*$ cannot be learned.
- However, it could generalize better :)

*What happens if an arc is added?*

```python
Image("figures/AccurateStructure2.png")
```

- The model encodes spurious dependencies.
- Can correctly learn the correct distribution $P^*$.
- More parameters to learn.
- In general, leads to worse generalization.

### 1.3. Score-based learning

To carry out the structure learning, we define a structure that evaluates how well a structure matches the data:

```python
Image("figures/ScoreBasedLearning.png")
```

Then, we search for the structure that maximizes the score.

## 2. Likelihood score

### 2.1. Introduction

The idea under the likelihood score is to find the structure $(\mathcal{G}, \theta)$ to maximize the likelihood:

$$\mathrm{score}_L (\mathcal{G}: \mathcal{D}) = l((\hat{\theta}, \mathcal{G}): \mathcal{D}) = \log \mathcal{L}((\hat{\theta}, \mathcal{G}): \mathcal{D})$$

where $\hat{\theta} = \theta_{MLE}$ is the MLE of the parameters given $\mathcal{G}$ and $\mathcal{D}$.

**Example:**

Consider the two random variables $X$ and $Y$.

First, consider the graph structure $\mathcal{G}_0:$

```python
Image("figures/Example1.png")
```

The likelihood score is:

$$\mathrm{score}_L(\mathcal{G}_0: \mathcal{D}) =  \sum_{d=1}^{M} \left(\log\hat{\theta}_{x[d]} + \log\hat{\theta}_{x[d]}\right)$$

Now, consider the graph structure $\mathcal{G}_1:$

```python
Image("figures/Example2.png")
```

The likelihood score is:

$$\mathrm{score}_L(\mathcal{G}_1: \mathcal{D}) = \sum_{d=1}^{M} \left(\log\hat{\theta}_{x[d]} + \log\hat{\theta}_{y[d]|x[d]}\right)$$

Now, let's compute the difference between them:

\begin{align}
\mathrm{score}_L(\mathcal{G}_1: \mathcal{D}) - \mathrm{score}_L(\mathcal{G}_0: \mathcal{D}) & = \sum_{d=1}^{M}\left(\log\hat{\theta}_{y[d]|x[d]} - \log\hat{\theta}_{y[d]}\right) \\
& = \sum_{x,y} M(x, y) \log\hat{\theta}_{y|x} - \sum_{y} M(y) \log\hat{\theta}_{y} \\
& = M \left(\sum_{x,y} \hat{P}(x, y) \log\hat{P}(x| y) - \sum_{y} \hat{P}(y) \log\hat{P}(y)\right),
\end{align}

where $\hat{P}$ is the empirical distribution (frequency counts).

Moreover, recall that $\sum_{x} \hat{P}(x, y) = \hat{P}(y)$. Then,

\begin{align}
\mathrm{score}_L(\mathcal{G}_1: \mathcal{D}) - \mathrm{score}_L(\mathcal{G}_0: \mathcal{D}) & = M \left(\sum_{x,y} \hat{P}(x, y) \log\hat{P}(x| y) - \sum_{x,y} \hat{P}(x,y) \log\hat{P}(y)\right) \\
& = M \sum_{x,y} \hat{P}(x, y) \left(\log\hat{P}(x| y) - \log\hat{P}(y)\right) \\
& = M \sum_{x,y} \hat{P}(x, y) \log\frac{\hat{P}(x| y)}{\hat{P}(y)} \\
& = M I_{\hat{P}} (X; Y),
\end{align}

where $I_{\hat{P}} (X; Y)$ is **the mutual information** between the variables $X$ and $Y$ w.r.t. the empirical distribution $\hat{P}$.

Intuitively, the mutual information $I_{\hat{P}} (X; Y)$ measures how close the variables $X$ and $Y$ are to the independence.

### 2.2. General decomposition

The above is not only for this simple example. In fact, one can easily show that:

$$\mathrm{score}_L(\mathcal{G}: \mathcal{D}) = M \sum_{i=1}^{n} I_{\hat{P}}(X_i, \mathrm{Pa}X_i) - M \sum_{i=1}^{n} H_{\hat{P}}(X_i),$$

where:

- $I_{\hat{P}}(X; Y) = \sum_{x,y} P(x, y) \log \frac{P(x,y)}{P(x)P(y)}$ is the mutual information.

- $H_{\hat{P}}(X) = - \sum_{x} P(x) \log P(x)$ is the entropy.

Note that the entropy $H_{\hat{P}}(X)$ is independent of the graph structure $\mathcal{G}$.

Then, the score is higher if the nodes $X_i$ is correlated with its parents.

**Limitations:**

Following the above, note that the difference of having an arc with not having it is:

$$\mathrm{score}_L(\mathcal{G}_1: \mathcal{D}) - \mathrm{score}_L(\mathcal{G}_0: \mathcal{D}) = M I_{\hat{P}} (X, Y).$$

Some comments:
- The mutual information is always nonnegative: $I_{\hat{P}} (X, Y) \geq 0.$
- It equals zero if and onlyt if $X$ and $Y$ are independent (in the empirical distribution $\hat{P}$).
- Even if $X \perp Y$ in the true distribution $P^*$, almost always $I_{\hat{P}} (X, Y) > 0$.
- Thus, adding edges can't hurt this score, and almost always helps.

### The maximum likelihood score is maximized for fully connected networks: overfitting.

### Because of the above, this score is never used (it is not implemented in `pgmpy`)

```python
# Import numpy and pandas

```

```python
# Generate some data (X, Y)

```

```python
# Wrap data into a data frame

```

```python
# Empirical probabilities
# P(X)

# P(Y)

# P(X, Y)

# P(X)P(Y)

```

```python
# Scores
# X    Y

# X -> Y

# X <- Y

```

```python
# Select best score

```

## 3. BIC score

How can we avoid overfitting?
- We can restrict the hypothesis space:
  - Restrict number of parents or parameters.

- We can penalyze the complexity.

An explicit penalization of the complexity is done by the **Bayesian Information Criterion (BIC) score**:

$$\mathrm{score}_{BIC}(\mathcal{G}: \mathcal{D}) = \mathrm{score}_{L}(\mathcal{G}: \mathcal{D}) - \frac{\log M}{2} \mathrm{Dim}[\mathcal{G}],$$

where $\mathrm{Dim}[\mathcal{G}]$ is the number of independent parameters implied by the structure $\mathcal{G}$.

This score directly represents the tradeoff between fit to the data and model complexity.

**Asymptotic consistency:**

We have that:

$$\mathrm{score}_{BIC}(\mathcal{G}: \mathcal{D}) = M \sum_{i=1}^{n} I_{\hat{P}}(X_i, \mathrm{Pa}X_i) - M \sum_{i=1}^{n} H_{\hat{P}}(X_i) - \frac{\log M}{2} \mathrm{Dim}[\mathcal{G}]$$

The mutual information grows linearly with $M$ whereas the complexity grows logarithmically with $M$.

> As $M\to\infty$, more emphasis is given to fit to the data than to model complexity.
>
> Thus, as $M\to\infty$, ($\hat{P} \to P^*$) the true structure $\mathcal{G}^*$ will maximize the score.

```python
# Import pgmpy.estimators.BicScore, pgmpy.estimators.ExhaustiveSearch, pgmpy.models.BayesianModel

```

```python
# Instantiate BicScore

```

```python
# Models
# X    Y

# X -> Y

# X <- Y

```

```python
# Scores
# X    Y

# X -> Y

# X <- Y

```

```python
# Select best score

```

The search space of BNs is exponential in the number of variables and the BIC scoring function allow for local maxima.

The first property makes exhaustive search intractable for all but very small networks, the second prohibits efficient local optimization algorithms to always find the optimal structure.

Thus, identifiying the ideal structure is often not tractable. Despite these bad news, heuristic search strategies often yields good results.

If only few nodes are involved (read: less than 5), ExhaustiveSearch can be used to compute the score for every DAG and returns the best-scoring one:

```python

```

```python
# Exhaustive search

```

```python
# Print all scores and edges

```

```python
# Best model

```

If more nodes are involved, one needs to switch to heuristic search.

HillClimbSearch implements a greedy local search that starts from the DAG start (default: disconnected DAG) and proceeds by iteratively performing single-edge manipulations that maximally increase the score. The search terminates once a local maximum is found.

```python
# Import pgmpy.estimators.HillClimbSearch

```

```python
# Create some data with dependencies

```

```python
# Hill Climb search

```

```python
# Best model

```

```python
# Edges

```

The search correctly identifies the independencies.

# Recommended lecture:

## Section 18.3.2 Bayesian score (page 794)

<footer id="attribution" style="float:right; color:#808080; background:#fff;">
Created with Jupyter by Esteban Jiménez Rodríguez.
</footer>


```
%load_ext autoreload
%autoreload 2

%matplotlib inline
```

```
#export
from exp.nb_06 import *
```

## ConvNet

Let's get the data and training interface from where we left in the last notebook.

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=5899)

```
x_train,y_train,x_valid,y_valid = get_data()

x_train,x_valid = normalize_to(x_train,x_valid)
train_ds,valid_ds = Dataset(x_train, y_train),Dataset(x_valid, y_valid)

nh,bs = 50,512
c = y_train.max().item()+1
loss_func = F.cross_entropy

data = DataBunch(*get_dls(train_ds, valid_ds, bs), c)
```

```
mnist_view = view_tfm(1,28,28)
cbfs = [Recorder,
        partial(AvgStatsCallback,accuracy),
        CudaCallback,
        partial(BatchTransformXCallback, mnist_view)]
```

```
nfs = [8,16,32,64,64]
```

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs)
```

```
%time run.fit(2, learn)
```

    train: [1.024234921875, tensor(0.6730, device='cuda:0')]
    valid: [0.2444910400390625, tensor(0.9262, device='cuda:0')]
    train: [0.162599970703125, tensor(0.9502, device='cuda:0')]
    valid: [0.10074585571289063, tensor(0.9698, device='cuda:0')]
    CPU times: user 3.78 s, sys: 1.61 s, total: 5.39 s
    Wall time: 6.37 s

## Batchnorm

### Custom

Let's start by building our own `BatchNorm` layer from scratch.

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=6018)

```
class BatchNorm(nn.Module):
    def __init__(self, nf, mom=0.1, eps=1e-5):
        super().__init__()
        # NB: pytorch bn mom is opposite of what you'd expect
        self.mom,self.eps = mom,eps
        self.mults = nn.Parameter(torch.ones (nf,1,1))
        self.adds  = nn.Parameter(torch.zeros(nf,1,1))
        self.register_buffer('vars',  torch.ones(1,nf,1,1))
        self.register_buffer('means', torch.zeros(1,nf,1,1))

    def update_stats(self, x):
        m = x.mean((0,2,3), keepdim=True)
        v = x.var ((0,2,3), keepdim=True)
        self.means.lerp_(m, self.mom)
        self.vars.lerp_ (v, self.mom)
        return m,v

    def forward(self, x):
        if self.training:
            with torch.no_grad(): m,v = self.update_stats(x)
        else: m,v = self.means,self.vars
        x = (x-m) / (v+self.eps).sqrt()
        return x*self.mults + self.adds
```

```
def conv_layer(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    # No bias needed if using bn
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=not bn),
              GeneralRelu(**kwargs)]
    if bn: layers.append(BatchNorm(nf))
    return nn.Sequential(*layers)
```

```
#export
def init_cnn_(m, f):
    if isinstance(m, nn.Conv2d):
        f(m.weight, a=0.1)
        if getattr(m, 'bias', None) is not None: m.bias.data.zero_()
    for l in m.children(): init_cnn_(l, f)

def init_cnn(m, uniform=False):
    f = init.kaiming_uniform_ if uniform else init.kaiming_normal_
    init_cnn_(m, f)

def get_learn_run(nfs, data, lr, layer, cbs=None, opt_func=None, uniform=False, **kwargs):
    model = get_cnn_model(data, nfs, layer, **kwargs)
    init_cnn(model, uniform=uniform)
    return get_runner(model, data, lr=lr, cbs=cbs, opt_func=opt_func)
```

We can then use it in training and see how it helps keep the activations means to 0 and the std to 1.

```
learn,run = get_learn_run(nfs, data, 0.9, conv_layer, cbs=cbfs)
```

```
with Hooks(learn.model, append_stats) as hooks:
    run.fit(1, learn)
    fig,(ax0,ax1) = plt.subplots(1,2, figsize=(10,4))
    for h in hooks[:-1]:
        ms,ss = h.stats
        ax0.plot(ms[:10])
        ax1.plot(ss[:10])
        h.remove()
    plt.legend(range(6));

    fig,(ax0,ax1) = plt.subplots(1,2, figsize=(10,4))
    for h in hooks[:-1]:
        ms,ss = h.stats
        ax0.plot(ms)
        ax1.plot(ss)
```

```
learn,run = get_learn_run(nfs, data, 1.0, conv_layer, cbs=cbfs)
```

```
%time run.fit(3, learn)
```

    train: [0.27833810546875, tensor(0.9105, device='cuda:0')]
    valid: [0.1912491943359375, tensor(0.9386, device='cuda:0')]
    train: [0.08977265625, tensor(0.9713, device='cuda:0')]
    valid: [0.09156090698242188, tensor(0.9716, device='cuda:0')]
    train: [0.06145498046875, tensor(0.9810, device='cuda:0')]
    valid: [0.09970919799804688, tensor(0.9707, device='cuda:0')]
    CPU times: user 3.71 s, sys: 584 ms, total: 4.29 s
    Wall time: 4.29 s

### Builtin batchnorm

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=6679)

```
#export
def conv_layer(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=not bn),
              GeneralRelu(**kwargs)]
    if bn: layers.append(nn.BatchNorm2d(nf, eps=1e-5, momentum=0.1))
    return nn.Sequential(*layers)
```

```
learn,run = get_learn_run(nfs, data, 1., conv_layer, cbs=cbfs)
```

```
%time run.fit(3, learn)
```

    train: [0.27115255859375, tensor(0.9206, device='cuda:0')]
    valid: [0.1547997314453125, tensor(0.9496, device='cuda:0')]
    train: [0.07861462890625, tensor(0.9755, device='cuda:0')]
    valid: [0.07472044067382813, tensor(0.9776, device='cuda:0')]
    train: [0.0570835498046875, tensor(0.9818, device='cuda:0')]
    valid: [0.0673232666015625, tensor(0.9813, device='cuda:0')]
    CPU times: user 3.37 s, sys: 747 ms, total: 4.12 s
    Wall time: 4.12 s

### With scheduler

Now let's add the usual warm-up/annealing.

```
sched = combine_scheds([0.3, 0.7], [sched_lin(0.6, 2.), sched_lin(2., 0.1)])
```

```
learn,run = get_learn_run(nfs, data, 0.9, conv_layer, cbs=cbfs
                          +[partial(ParamScheduler,'lr', sched)])
```

```
run.fit(8, learn)
```

    train: [0.2881914453125, tensor(0.9116, device='cuda:0')]
    valid: [0.5269224609375, tensor(0.8394, device='cuda:0')]
    train: [0.153792421875, tensor(0.9551, device='cuda:0')]
    valid: [0.462953662109375, tensor(0.8514, device='cuda:0')]
    train: [0.087637158203125, tensor(0.9736, device='cuda:0')]
    valid: [0.07029829711914062, tensor(0.9788, device='cuda:0')]
    train: [0.049282421875, tensor(0.9853, device='cuda:0')]
    valid: [0.08710025024414063, tensor(0.9740, device='cuda:0')]
    train: [0.0356986328125, tensor(0.9888, device='cuda:0')]
    valid: [0.07853966674804687, tensor(0.9773, device='cuda:0')]
    train: [0.0268300439453125, tensor(0.9918, device='cuda:0')]
    valid: [0.04807376098632812, tensor(0.9870, device='cuda:0')]
    train: [0.0219412109375, tensor(0.9939, device='cuda:0')]
    valid: [0.04363873901367187, tensor(0.9882, device='cuda:0')]
    train: [0.018501048583984374, tensor(0.9951, device='cuda:0')]
    valid: [0.04355916137695313, tensor(0.9877, device='cuda:0')]

## More norms

### Layer norm

From [the paper](https://arxiv.org/abs/1607.06450): "*batch normalization cannot be applied to online learning tasks or to extremely large distributed models where the minibatches have to be small*".

General equation for a norm layer with learnable affine:

$$y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta$$

The difference with BatchNorm is
1. we don't keep a moving average
2. we don't average over the batches dimension but over the hidden dimension, so it's independent of the batch size

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=6717)

```
class LayerNorm(nn.Module):
    __constants__ = ['eps']
    def __init__(self, eps=1e-5):
        super().__init__()
        self.eps = eps
        self.mult = nn.Parameter(tensor(1.))
        self.add  = nn.Parameter(tensor(0.))

    def forward(self, x):
        m = x.mean((1,2,3), keepdim=True)
        v = x.var ((1,2,3), keepdim=True)
        x = (x-m) / ((v+self.eps).sqrt())
        return x*self.mult + self.add
```

```
def conv_ln(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=True),
              GeneralRelu(**kwargs)]
    if bn: layers.append(LayerNorm())
    return nn.Sequential(*layers)
```

```
learn,run = get_learn_run(nfs, data, 0.8, conv_ln, cbs=cbfs)
```

```
%time run.fit(3, learn)
```

    train: [nan, tensor(0.1321, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    train: [nan, tensor(0.0986, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    train: [nan, tensor(0.0986, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    CPU times: user 4.56 s, sys: 862 ms, total: 5.42 s
    Wall time: 5.42 s

*Thought experiment*: can this distinguish foggy days from sunny days (assuming you're using it before the first conv)?

### Instance norm

From [the paper](https://arxiv.org/abs/1607.08022):

The key difference between **contrast** and batch normalization is that the latter applies the normalization to a  whole batch of images instead for single ones:

\begin{equation}\label{eq:bnorm}
    y_{tijk} =  \frac{x_{tijk} - \mu_{i}}{\sqrt{\sigma_i^2 + \epsilon}},
    \quad
    \mu_i = \frac{1}{HWT}\sum_{t=1}^T\sum_{l=1}^W \sum_{m=1}^H x_{tilm},
    \quad
    \sigma_i^2 = \frac{1}{HWT}\sum_{t=1}^T\sum_{l=1}^W \sum_{m=1}^H (x_{tilm} - mu_i)^2.
\end{equation}

In order to combine the effects of instance-specific normalization and batch normalization, we propose to replace the latter by the *instance normalization* (also known as *contrast normalization*) layer:

\begin{equation}\label{eq:inorm}
    y_{tijk} =  \frac{x_{tijk} - \mu_{ti}}{\sqrt{\sigma_{ti}^2 + \epsilon}},
    \quad
    \mu_{ti} = \frac{1}{HW}\sum_{l=1}^W \sum_{m=1}^H x_{tilm},
    \quad
    \sigma_{ti}^2 = \frac{1}{HW}\sum_{l=1}^W \sum_{m=1}^H (x_{tilm} - mu_{ti})^2.
\end{equation}

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=7114)

```
class InstanceNorm(nn.Module):
    __constants__ = ['eps']
    def __init__(self, nf, eps=1e-0):
        super().__init__()
        self.eps = eps
        self.mults = nn.Parameter(torch.ones (nf,1,1))
        self.adds  = nn.Parameter(torch.zeros(nf,1,1))

    def forward(self, x):
        m = x.mean((2,3), keepdim=True)
        v = x.var ((2,3), keepdim=True)
        res = (x-m) / ((v+self.eps).sqrt())
        return res*self.mults + self.adds
```

```
def conv_in(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=True),
              GeneralRelu(**kwargs)]
    if bn: layers.append(InstanceNorm(nf))
    return nn.Sequential(*layers)
```

```
learn,run = get_learn_run(nfs, data, 0.1, conv_in, cbs=cbfs)
```

```
%time run.fit(3, learn)
```

    train: [nan, tensor(0.0986, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    train: [nan, tensor(0.0986, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    train: [nan, tensor(0.0986, device='cuda:0')]
    valid: [nan, tensor(0.0991, device='cuda:0')]
    CPU times: user 4.46 s, sys: 718 ms, total: 5.18 s
    Wall time: 5.18 s

*Question*: why can't this classify anything?

Lost in all those norms? The authors from the [group norm paper](https://arxiv.org/pdf/1803.08494.pdf) have you covered:

### Group norm

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=7213)

*From the PyTorch docs:*

`GroupNorm(num_groups, num_channels, eps=1e-5, affine=True)`

The input channels are separated into `num_groups` groups, each containing
``num_channels / num_groups`` channels. The mean and standard-deviation are calculated
separately over the each group. $\gamma$ and $\beta$ are learnable
per-channel affine transform parameter vectorss of size `num_channels` if
`affine` is ``True``.

This layer uses statistics computed from input data in both training and
evaluation modes.

Args:
-    num_groups (int): number of groups to separate the channels into
-    num_channels (int): number of channels expected in input
-    eps: a value added to the denominator for numerical stability. Default: 1e-5
-    affine: a boolean value that when set to ``True``, this module
        has learnable per-channel affine parameters initialized to ones (for weights)
        and zeros (for biases). Default: ``True``.

Shape:
- Input: `(N, num_channels, *)`
- Output: `(N, num_channels, *)` (same shape as input)

Examples::

    >>> input = torch.randn(20, 6, 10, 10)
    >>> # Separate 6 channels into 3 groups
    >>> m = nn.GroupNorm(3, 6)
    >>> # Separate 6 channels into 6 groups (equivalent with InstanceNorm)
    >>> m = nn.GroupNorm(6, 6)
    >>> # Put all 6 channels into a single group (equivalent with LayerNorm)
    >>> m = nn.GroupNorm(1, 6)
    >>> # Activating the module
    >>> output = m(input)

## Fix small batch sizes

### What's the problem?

When we compute the statistics (mean and std) for a BatchNorm Layer on a small batch, it is possible that we get a standard deviation very close to 0. because there aren't many samples (the variance of one thing is 0. since it's equal to its mean).

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=7304)

```
data = DataBunch(*get_dls(train_ds, valid_ds, 2), c)
```

```
def conv_layer(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=not bn),
              GeneralRelu(**kwargs)]
    if bn: layers.append(nn.BatchNorm2d(nf, eps=1e-5, momentum=0.1))
    return nn.Sequential(*layers)
```

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs)
```

```
%time run.fit(1, learn)
```

    train: [2.35619296875, tensor(0.1649, device='cuda:0')]
    valid: [2867.7198, tensor(0.2604, device='cuda:0')]
    CPU times: user 1min 32s, sys: 835 ms, total: 1min 33s
    Wall time: 1min 33s

### Running Batch Norm

To solve this problem we introduce a Running BatchNorm that uses smoother running mean and variance for the mean and std.

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=7516)

```
class RunningBatchNorm(nn.Module):
    def __init__(self, nf, mom=0.1, eps=1e-5):
        super().__init__()
        self.mom,self.eps = mom,eps
        self.mults = nn.Parameter(torch.ones (nf,1,1))
        self.adds = nn.Parameter(torch.zeros(nf,1,1))
        self.register_buffer('sums', torch.zeros(1,nf,1,1))
        self.register_buffer('sqrs', torch.zeros(1,nf,1,1))
        self.register_buffer('batch', tensor(0.))
        self.register_buffer('count', tensor(0.))
        self.register_buffer('step', tensor(0.))
        self.register_buffer('dbias', tensor(0.))

    def update_stats(self, x):
        bs,nc,*_ = x.shape
        self.sums.detach_()
        self.sqrs.detach_()
        dims = (0,2,3)
        s = x.sum(dims, keepdim=True)
        ss = (x*x).sum(dims, keepdim=True)
        c = self.count.new_tensor(x.numel()/nc)
        mom1 = 1 - (1-self.mom)/math.sqrt(bs-1)
        self.mom1 = self.dbias.new_tensor(mom1)
        self.sums.lerp_(s, self.mom1)
        self.sqrs.lerp_(ss, self.mom1)
        self.count.lerp_(c, self.mom1)
        self.dbias = self.dbias*(1-self.mom1) + self.mom1
        self.batch += bs
        self.step += 1

    def forward(self, x):
        if self.training: self.update_stats(x)
        sums = self.sums
        sqrs = self.sqrs
        c = self.count
        if self.step<100:
            sums = sums / self.dbias
            sqrs = sqrs / self.dbias
            c    = c    / self.dbias
        means = sums/c
        vars = (sqrs/c).sub_(means*means)
        if bool(self.batch < 20): vars.clamp_min_(0.01)
        x = (x-means).div_((vars.add_(self.eps)).sqrt())
        return x.mul_(self.mults).add_(self.adds)
```

```
def conv_rbn(ni, nf, ks=3, stride=2, bn=True, **kwargs):
    layers = [nn.Conv2d(ni, nf, ks, padding=ks//2, stride=stride, bias=not bn),
              GeneralRelu(**kwargs)]
    if bn: layers.append(RunningBatchNorm(nf))
    return nn.Sequential(*layers)
```

```
learn,run = get_learn_run(nfs, data, 0.4, conv_rbn, cbs=cbfs)
```

```
%time run.fit(1, learn)
```

    train: [0.157932021484375, tensor(0.9511, device='cuda:0')]
    valid: [0.0986408935546875, tensor(0.9730, device='cuda:0')]
    CPU times: user 16.5 s, sys: 1.57 s, total: 18.1 s
    Wall time: 18.1 s

This solves the small batch size issue!

### What can we do in a single epoch?

Now let's see with a decent batch size what result we can get.

[Jump_to lesson 10 video](https://course19.fast.ai/videos/?lesson=10&t=8068)

```
data = DataBunch(*get_dls(train_ds, valid_ds, 32), c)
```

```
learn,run = get_learn_run(nfs, data, 0.9, conv_rbn, cbs=cbfs
                          +[partial(ParamScheduler,'lr', sched_lin(1., 0.2))])
```

```
%time run.fit(1, learn)
```

    train: [0.1573110546875, tensor(0.9521, device='cuda:0')]
    valid: [0.09242745971679688, tensor(0.9818, device='cuda:0')]
    CPU times: user 16.6 s, sys: 1.52 s, total: 18.1 s
    Wall time: 18.2 s

## Export

```
nb_auto_export()
```

    <IPython.core.display.Javascript object>

```

```


```python
import sympy
from phasor.utilities.ipynb.displays import *
from phasor.utilities.ipynb.ipy_sympy import *
import scipy.linalg

import numpy.testing as np_test
import declarative as decl
from declarative.bunch import (
    DeepBunch,
)

#import numpy as np

from phasor import system
from phasor import readouts
from phasor.utilities.print import pprint
```

```python
db = DeepBunch()
db.environment.F_AC.order = 1
from phasor.models.SQZ.AOMTestStand import VCOTest
sys = system.BGSystem(
    ctree = db,
    F_AC = 0*1e3,
)
sys.own.test = VCOTest()
db = sys.ctree_shadow()
#print(sys.test.ktp.ctree_as_yaml())
print(sys.test.AC_I.AC_sensitivity)
print(sys.test.AC_Q.AC_sensitivity)
print(sys.test.AC_IQ.AC_sensitivity)
print(sys.test.AC_QI.AC_sensitivity)
```

    (0.5+0j)
    (0.5+0j)
    0j
    0j

```python

```