# Center of Mass and Moment of Inertia

> Renato Naville Watanabe, Marcos Duarte
> [Laboratory of Biomechanics and Motor Control](http://pesquisa.ufabc.edu.br/bmclab/)
> Federal University of ABC, Brazil

An animation of a biomechanical analysis of Daiane dos Santos executing the Dos Santos I movement in artistic gymnast: a piked double Arabian (a half twist into a double front flip in a piked position).
I: While her body translates and rotates in varied ways, the trajectory of the body center of mass is always a parabola during the aerial phases (minus some experimental error).
II: To execute the double front flip at the last jump, she increases the body angular speed by flexing the hips (piked position), which decreases the body moment of inertia — she knows the law of conservation of angular momentum.
Note the view is not from a right angle to the sagittal plane (Image from BMClab).

Contents

1  Python setup

2  Center of mass

2.1  Set of particles

2.1.1  Example: Two particles

2.2  Rigid body

2.2.1  Example: Bar

2.2.2  Example: Triangle

2.3  Center of mass of a complex system

2.3.1  Example: Three bars

3  Center of gravity

4  Geometric center

5  Moment of rotational inertia

5.1  Set of particles

5.1.1  Example: System with two particles

5.2  Rigid body

5.2.1  Example: Bar

5.3  Radius of gyration

5.3.1  Example: Bar

5.4  Parallel axis theorem

5.4.1  Example: Bar

5.5  Moment of inertia of a complex system

5.5.1  Example: Eight bars

5.6  Matrix of Inertia

5.6.1  Principal axes

5.6.2  Example: Cylinder

5.7  The parallel axis theorem for rigid bodies in three dimensions

5.8  Moment of rotational inertia and area moment of inertia

6  Further reading

7  Video lectures on the internet

8  Problems

9  References

## Python setup

```python
# import the necessary libraries
import numpy as np
import sympy as sym
from IPython.display import display, Math
from sympy.vector import CoordSys3D
```

Code for using LaTeX commands, see https://github.com/jupyterlab/jupyterlab/pull/5997 and https://texfaq.org/FAQ-patch.

\renewcommand{\require}[1]{}
\newcommand{\require}[1]{}

$\require{begingroup}\require{renewcommand}$
$\gdef\hat#1{\widehat{\mathbf{#1}}}$

$\require{begingroup}\require{newcommand}$
$\gdef\vecb#1{\vec{\bf{#1}}}$

\vskip-\parskip
\vskip-\baselineskip

Execute this cell to format a versor with a hat and bold font or edit it to add custom commands.

## Center of mass

### Set of particles

The center of mass of a set of particles is defined as the point (position) where the sum of the vectors linking this point to each particle, weighted by its mass, is zero. By [mass](https://en.wikipedia.org/wiki/Mass) we mean the inertial mass, a quantitative measure of an object's resistance to linear acceleration.

Consider the set of particles shown below.

Figure. The center of mass of a set of particles.

The sum of the vectors linking the center of mass to each particle is:

\begin{equation}
\begin{array}{l}
\sum\limits_{i=1}^nm_i{\vec{r}_{i/cm}} &= \sum\limits_{i=1}^nm_i\left({\vec{r}_i} - {\vec{r}_{cm}}\right) \\
&= \sum\limits_{i=1}^nm_i{\vec{r}_i} - \sum\limits_{i=1}^nm_i{\vec{r}_{cm}} \\
&= \sum\limits_{i=1}^nm_i{\vec{r}_i} - {\vec{r}_{cm}}\sum\limits_{i=1}^nm_i
\label{}
\end{array}
\end{equation}

where $n$ is the number of particles.

Now, we equal this sum to zero and isolate ${\vec{r}_{cm}}$:

\begin{equation}
\begin{array}{l}
\sum\limits_{i=1}^nm_i{\vec{r}_i} - {\vec{r}_{cm}}\sum\limits_{i=1}^nm_i = 0 \quad \longrightarrow \\
\begin{aligned}
\vec{r}_{cm} &= \dfrac{\sum\limits_{i=1}^nm_i{\vec{r}_i}}{\sum\limits_{i=1}^nm_i} &\phantom{=} \\
&= \dfrac{\sum\limits_{i=1}^nm_i{\vec{r}_i}}{m_T}
\end{aligned}
\label{}
\end{array}
\end{equation}

where $m_T$ is the total mass of the particles.

#### Example: Two particles

Consider two particles with masses $m_1$ and $m_2$, the center of mass of the system is:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm} &= \dfrac{m_1\vec{r}_1 + m_2\vec{r}_2}{m_1+m_2} \\
&= \dfrac{m_1\vec{r}_1 + m_2\vec{r}_1 - m_2\vec{r}_1 + m_2\vec{r}_2}{m_1+m_2} \\
&= \dfrac{(m_1+m_2)\vec{r}_1}{m_1+m_2} + \dfrac{m_2(\vec{r}_2 - \vec{r}_1)}{m_1+m_2} \\
\vec{r}_{cm} &= \vec{r}_1 + \dfrac{m_2}{m_1+m_2}(\vec{r}_2 - \vec{r}_1)
\label{}
\end{array}
\end{equation}

Can you guess what is the expression above if we rewrite it in relation to vector $\vec{r}_{2}$, i.e., $\vec{r}_{cm} = \vec{r}_{2} + \ldots$ ?

### Rigid body

For a rigid body, the center of mass is defined as the point where the integral of the vectors  linking this point to each differential part of mass, weighted by this differential mass, is zero.

Figure. The center of mass of a rigid body.

This integral is:

\begin{equation}
\begin{array}{l}
\int\limits_{B} {\vec{r}_{/cm}} \mathrm d m &= \int\limits_{B}(\vec{r}-\vec{r}_{cm}) \mathrm d m \\
&= \int\limits_{B} {\vec{r}}\,\mathrm d m - \int\limits_{B}{\vec{r}_{cm}} \mathrm d m \\
&= \int\limits_{B} {\vec{r}}\,\mathrm d m - {\vec{r}_{cm}}\int\limits_{B}\, \mathrm d m
\label{}
\end{array}
\end{equation}

Now we equal this integral to zero and isolate $\vec{r}_{cm}$:

\begin{equation}
\begin{array}{l}
\int\limits_{B} \vec{r}\,\mathrm d m - \vec{r}_{cm}\int\limits_{B}\, \mathrm d m = 0 \longrightarrow \\
\vec{r}_{cm} = \dfrac{\int\limits_{B}{\vec{r}}\,\mathrm d m}{\int\limits_{B}\, \mathrm d m}  = \dfrac{ \int\limits_{B} \vec{r}\,\mathrm d m}{m_B}
\label{eq:cm}
\end{array}
\end{equation}

where $m_B$ is the mass of the body.

#### Example: Bar

Let's calculate the center of mass of a homogeneous (with equal density) bar shown below.

Figure. A bar.

\begin{equation}
\vec{r}_{cm} = \frac{ \int \vec{r}\,\mathrm d m}{m}
\end{equation}

The mass of the bar is:

\begin{equation}
m = \rho l
\end{equation}

The integral can be computed as:

\begin{equation}
\int \vec{r}\,\mathrm d m = \rho\int\limits_0^l x\,\mathrm d x =\rho\frac{l^2}{2}
\end{equation}

So, the center of mass of the bar is:

\begin{equation}
\vec{r}_{cm} = \dfrac{\rho\dfrac{l^2}{2}}{\rho l} = \dfrac{l}{2}
\end{equation}

The center of mass of a homogeneous bar is in the middle of the bar.

A key aspect in the calculation of the integral above is to transform the differential of the variable mass $\mathrm d m$ into the differential of the variable displacement $\mathrm d x$ (or area or volume for a surface or a solid) because the body's density is known.

Let's use Sympy, a symbolic library in Python, to solve this integral.
A definite integral of integrand `f` with respect to variable `x` over interval `[a,b]` can be calculated with the function `integrate`:

```python
sym.integrate(f, (x, a, b))
```

```python
# Helping function
def print2(lhs, rhs):
    """Rich display of Sympy expression as lhs = rhs."""
    display(Math(r'{} = '.format(lhs) + sym.latex(sym.simplify(rhs, ratio=1.7))))
```

```python
# define variables
x = sym.symbols('x')
rho, ell = sym.symbols('rho ell', positive=True)
m = rho*ell
# solve the integral
rcm = rho * sym.integrate(x, (x, 0, ell))/m

print2('\\vec{r}_{cm}', rcm)
```

$\displaystyle \vec{r}_{cm} = \frac{\ell}{2}$

#### Example: Triangle

Let's compute the center of mass of the triangle shown below.

Figure. A triangle.

The center of mass is given by Eq.(\ref{eq:cm}):

\begin{equation}
\vec{r}_{cm} = \frac{\int \vec{r}\,\mathrm d m}{m}
\label{eq:int_area}
\end{equation}

The mass of the triangle w.r.t. its density and dimensions is equal to:

\begin{equation}
m = \rho A = \frac{\rho bh}{2}
\end{equation}

In Eq.(\ref{eq:int_area}), the differential of the variable mass $\mathrm d m$ will be transformed into the differential of the variable area $\mathrm d A$, which in turn is calculated as $\mathrm d x \mathrm d y$, resulting in a double integral:

\begin{equation}
\int \vec{r}\,\mathrm d m = \rho\int \vec{r}\,\mathrm d A = \rho \int\limits_x \int\limits_y (x\hat{i} + y\hat{j})\,\mathrm d y \mathrm d x
\label{}
\end{equation}

The integral can be computed by separating it into two parts: one part we integrate in the $x$ direction from $0$ to $p$, and the other part from $p$ to $b$. The integration in the $y$ direction will be from $0$ to $\frac{xh}{p}$ in the first part and from $0$ to $\frac{(b-x)h}{b-p}$ in the second part:

\begin{equation}
\int \vec{r}\,\mathrm d m = \rho\left[\int\limits_0^p\int\limits_0^\frac{xh}{p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x + \int\limits_p^b\int\limits_0^\frac{(b-x)h}{b-p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x\right]
\label{}
\end{equation}

So, the center of mass of the triangle will be:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm} &= \dfrac{\rho\left[\int\limits_0^p\int\limits_0^\frac{xh}{p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x + \int\limits_p^b\int\limits_0^\frac{(b-x)h}{b-p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x\right]}{\dfrac{\rho bh}{2}} \\
&= \dfrac{2\left[\int\limits_0^p\int\limits_0^\frac{xh}{p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x + \int\limits_p^b\int\limits_0^\frac{(b-x)h}{b-p} (x\hat{i} + y\hat{j}) \,\mathrm d y\mathrm d x\right]}{bh}
\label{}
\end{array}
\end{equation}

The integral above will be solved using the Symbolic library of the Python.

```python
# define variables
G = CoordSys3D('')
x, y = sym.symbols('x, y')
rho, h, p, b = sym.symbols('rho, h, p, b', positive=True)
# solve the integral
# for x
xcm = 2*(sym.integrate(sym.integrate(x, (y, 0, x*h/p)), (x, 0, p)) +
         sym.integrate(sym.integrate(x, (y, 0, (b-x)*h/(b-p))), (x, p, b)))/(b*h)
# for y
ycm = 2*(sym.integrate(sym.integrate(y, (y, 0, x*h/p)), (x, 0, p)) +
         sym.integrate(sym.integrate(y, (y, 0, (b-x)*h/(b-p))), (x, p, b)))/(b*h)

print2('x_{cm}', xcm)
print2('y_{cm}', ycm)
```

$\displaystyle x_{cm} = \frac{b}{3} + \frac{p}{3}$

$\displaystyle y_{cm} = \frac{h}{3}$

Finally, the center of mass of the triangle is:

```python
print2('\\vec{r}_{cm}', xcm*G.i + ycm*G.j)
```

$\displaystyle \vec{r}_{cm} = (\frac{b}{3} + \frac{p}{3})\mathbf{\hat{i}_{}} + (\frac{h}{3})\mathbf{\hat{j}_{}}$

### Center of mass of a complex system

Now, we will consider a set of $n$ bodies. The center of mass of this set of bodies can be computed by integrating the Eq.(\ref{eq:cm}) over all bodies:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm} &= \dfrac{ \int\limits_{B1,B2,\ldots,Bn}\vec{r}\,\mathrm d m}{m_{B1}+m_{B2}+ \ldots + m_{Bn}} \\
&= \dfrac{\int\limits_{B1}\vec{r}\,\mathrm d m+\int\limits_{B2}\vec{r}\,dm+\ldots+\int\limits_{Bn}\vec{r}\,\mathrm d m}{m_{B1}+m_{B2}+\ldots+ m_{Bn}} \\
&= \dfrac{\dfrac{\int\limits_{B1}\vec{r}\,\mathrm d m}{m_{B1}}m_{B1} + \dfrac{\int\limits_{B2}\vec{r}\,\mathrm d m}{m_{B2}}m_{B2} +\ldots+ \dfrac{\int\limits_{Bn}\vec{r}\,\mathrm d m}{m_{Bn}}m_{Bn}}{m_{B1}+m_{B2}+\ldots+ m_{Bn}} \\
&= \dfrac{\vec{r}_{cm_1} m_{B1} + \vec{r}_{cm_2}m_{B2} +\ldots+ \vec{r}_{cm_n}m_{Bn}}{m_{B1}+m_{B2}+\ldots+ m_{Bn}} \\
\vec{r}_{cm} &= \dfrac{\vec{r}_{cm_1} m_{B1} + \vec{r}_{cm_2}m_{B2} +\ldots+ \vec{r}_{cm_n}m_{Bn}}{m_T}
\label{}
\end{array}
\end{equation}

where $\vec{r}_{cm_i}$ is  the center of mass of the body $i$, $m_{Bi}$ is the mass of the body $i$ and $m_T$ is the total mass of all bodies.
The expression above shows that we can interpret each body as a particle with its mass and position.

#### Example: Three bars

Let's compute the center of mass of the system shown below.

Figure. A system with three bars.

The system can be seen as a set of three bars; we can compute the center of mass by considering each bar as a particle located at its respective center of mass.

We have previously computed the center of mass of a homogeneous bar as being at its center. So, the center of mass of each bar of the system is:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm_1} &= -\frac{l_1}{2}\hat{j} \\
\vec{r}_{cm_2} &= \frac{l_2}{2}\sin(\theta_1)\hat{i}-\frac{l_2}{2}\cos(\theta_1)\hat{j} \\
\vec{r}_{cm_3} &= l_2\sin(\theta_1)\hat{i}-l_2\cos(\theta_1)\hat{j}+\frac{l_3}{2}\sin(\theta_2)\hat{i}-\frac{l_3}{2}\cos(\theta_2)\hat{j} \\
&= \left(l_2\sin(\theta_1)+\frac{l_3}{2}\sin(\theta_2) \right)\hat{i} + \left(l_2\cos(\theta_1)-\frac{l_3}{2}\cos(\theta_2) \right)\hat{j}
\label{}
\end{array}
\end{equation}

So, the center of mass of the system is:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm} &= \dfrac{m_1 \vec{r}_{cm_1}+m_2 \vec{r}_{cm_2}+m_3 \vec{r}_{cm_3}}{m_1+m_2+m_3} \\
&= \dfrac{-m_1\frac{l_1}{2}\hat{j} + m_2(\frac{l_2}{2}\sin(\theta_1)\hat{i}-\frac{l_2}{2}\cos(\theta_1)\hat{j})+m_3 \left[\left(l_2\sin(\theta_1)+\frac{l_3}{2}\sin(\theta_2) \right)\hat{i} + \left(l_2\cos(\theta_1)-\frac{l_3}{2}\cos(\theta_2) \right)\hat{j} \right]}{m_1+m_2+m_3} \\
&= \dfrac{m_2\frac{l_2}{2}\sin(\theta_1)\hat{i}+m_3 \left(l_2\sin(\theta_1)+\frac{l_3}{2}\sin(\theta_2) \right)\hat{i}}{m_1+m_2+m_3}+\dfrac{-m_1\frac{l_1}{2}\hat{j} - m_2\frac{l_2}{2}\cos(\theta_1)\hat{j}+m_3 \left(l_2\cos(\theta_1)-\frac{l_3}{2}\cos(\theta_2) \right)\hat{j}}{m_1+m_2+m_3} \\
\vec{r}_{cm} &= \dfrac{m_2\frac{l_2}{2}\sin(\theta_1)+m_3 \left(l_2\sin(\theta_1)+\frac{l_3}{2}\sin(\theta_2)\right)}{m_1+m_2+m_3}\hat{i}+\dfrac{-m_1\frac{l_1}{2} - m_2\frac{l_2}{2}\cos(\theta_1)+m_3 \left(l_2\cos(\theta_1)-\frac{l_3}{2}\cos(\theta_2) \right)}{m_1+m_2+m_3}\hat{j}
\label{}
\end{array}
\end{equation}

## Center of gravity

Center of gravity of a body is the point where the moment of force caused by the gravitational force in the whole body relative to this point is zero.
For the body $B$, this the moment of force is:

\begin{equation}
\vec{M}_0 = \int\limits_{B} \vec{r}_B \, \times \, \vec{g} \, \mathrm d m
\end{equation}

If the acceleration of gravity being applied to the whole body is the same (for all practical purposes, in Biomechanics we can consider it the same in the whole body), the gravity vector can go out of the integral:

\begin{equation}
\begin{array}{l}
\vec{M}_0 &= \int\limits_{B}\vec{r}_B \, \mathrm d m\, \times\, \vec{g} \\
&= \int\limits_{B}(\vec{r} - \vec{r}_{cm})\,\mathrm d m\, \times\, \vec{g} \\
&= \left(\int\limits_{B}\vec{r} \, \mathrm d m -\int\limits_{B}\vec{r}_{G} \, \mathrm d m \, \right) \times \, \vec{g}
\label{}
\end{array}
\end{equation}

Now, we equal this moment to zero and isolate $\vec{r}_G$:

\begin{equation}
\begin{array}{l}
\left(\int\limits_{B}\vec{r}\,\mathrm d m -\int\limits_{B}\vec{r}_G\,\mathrm d m\right) \times\,\vec{g} = 0 \longrightarrow \\
\int\limits_{B}\vec{r}\,\mathrm d m -\int\limits_{B}\vec{r}_G\,\mathrm d m = 0 \longrightarrow \\
\int\limits_{B}\vec{r}\,\mathrm d m -\vec{r}_G\int\limits_{B}\,\mathrm d m = 0\,\,\,\,\,\, \longrightarrow \\
\vec{r}_G = \dfrac{ \int\limits_{B}\vec{r}\,\mathrm d m}{\int\limits_{B}\,\mathrm d m} = \dfrac{ \int\limits_{B}\vec{r}\,\mathrm d m}{m_B}
\label{}
\end{array}
\end{equation}

where $m_B$ is the mass of the body.

Note that in this case, when the gravitational acceleration is constant, the center of gravity $\vec{r}_G$ is equal to the center of mass $\vec{r}_{cm}$.

## Geometric center

For a rigid body, the geometric center is defined as the point where the integral of the vectors linking this point to each differential part of mass is zero.

This integral for a body $B$ is:

\begin{equation}
\begin{array}{l}
\int\limits_{B} \vec{r}_{/gc} \mathrm d V &= \int\limits_{B} (\vec{r} - \vec{r}_{gc}) \mathrm d V \\
&= \int\limits_{B} \vec{r}\, \mathrm d V - \int\limits_{B}\vec{r}_{gc}\,  \mathrm d V \\
&= \int\limits_{B} \vec{r}\, \mathrm d V - \vec{r}_{gc}\int\limits_{B}\, \mathrm d V
\label{}
\end{array}
\end{equation}

Now we equal this integral to zero and isolate $\vec{r}_{gc}$:

\begin{equation}
\begin{array}{l}
\int\limits_{B} {\vec{r}}\,\mathrm d V - {\vec{r}_{gc}}\int\limits_{B}\, \mathrm d V  = 0 \longrightarrow \\
\vec{r}_{gc} = \dfrac{ \int\limits_{B} \vec{r}\,\mathrm d V}{\int\limits_{B}\, \mathrm d V}  = \dfrac{ \int\limits_{B} \vec{r}\,\mathrm d V}{V}
\label{eq:gc}
\end{array}
\end{equation}

where $V$ is the volume of the body.
Note that when the body has a constant density $\rho$, the center of mass is equal to the geometric center:

\begin{equation}
\begin{array}{l}
\vec{r}_{cm} &= \dfrac{ \int\limits_{B} \vec{r}\,\mathrm d m}{m_B} \\
&= \dfrac{ \int\limits_{B} \vec{r}\rho\,\mathrm d V}{\rho V} \\
&= \dfrac{ \rho\int\limits_{B} \vec{r}\,\mathrm d V}{\rho V} \\
&= \dfrac{ \int\limits_{B} \vec{r}\,\mathrm d V}{V} \\
\vec{r}_{cm} &= \vec{r}_{gc}
\label{}
\end{array}
\end{equation}

## Moment of rotational inertia

The moment of rotational inertia (or moment of inertia or rotational inertia), equivalent to mass for linear motion, is a quantitative measure of the resistance to rotational acceleration around an axis of a distribution of mass in space.

Consider the linear momentum of a particle in motion, by definition:

\begin{equation}
\vec{p} = m\vec{v}
\label{}
\end{equation}

The angular momentum of a particle in rotational motion, by definition is:

\begin{equation}
\vec{H}_{/O} = \vec{r} \times \vec{p}
\label{}
\end{equation}

where $\vec{r}$ is the position of the particle relative to the chosen origin $O$.

Figure. A particle at position r with linear momentum and angular momentum relative to origin O.

For simplicity (later we will generalize this problem), let's consider that the particle rotates at a fixed plane and at a fixed distance, $r$, to point $O$ (i.e., circular motion). In this case, the magnitude of the angular momentum is:

\begin{equation}
\begin{array}{l}
H_{/O} &= r \, p \\
&= r \, m \, v \\
&= r \, m \, \omega \, r \\
H_{/O} &= m \, r^2 \, \omega
\end{array}
\label{}
\end{equation}

where $\omega$ is the angular speed of rotation.

Equivalent to the linear momentum, defined as mass (inertia) times velocity, in the equation above, the term in the right-hand side that multiplies the angular velocity is defined as the rotational inertia or moment of inertia of the particle around axis passing through point $O$:

\begin{equation}
I_O = m r^2
\label{}
\end{equation}

The value of the moment of inertia is a single positive scalar for a planar rotation or a tensor (a 3×3 inertia matrix) for a three-dimensional rotation (we will see that later). Its dimension is $[M][L]^2$ and its SI unit is $kgm^2$.

### Set of particles

For a system of $n$ particles, the moment of inertia around a fixed axis passing by point $O$ for the case of planar rotation will be simply the sum of the moment of inertia of each particle:

\begin{equation}
I_O = \sum_{i=1}^n m_{i}r_{i/O}^2
\label{eg:I}
\end{equation}

#### Example: System with two particles

Calculate the moment of inertia of a system composed of two particles, each with 2-kg mass and spaced by 1 m, rotating around an axis perpendicular to the line that joins the two particles and passing through: a) center of mass of the system and b) one of the masses.

The center of mass of the system is at the center (centroid) of the system. Using this position as the origin of the system, the moment of inertia of the system around its center of mass is:

$$ I_{cm} = 2 \times (0.5)^2 + 2 \times (0.5)^2 = 1 \, kgm^2 $$

And around one of the masses:

$$ I_{m_1} = 2 \times (0)^2 + 2 \times (1)^2 = 2 \, kgm^2 $$

### Rigid body

In analogy to the determination of the center of mass, the moment of inertia of a rigid body around an axis passing through a point $O$ in the case of a planar rotation is given by the integral of the squared distance over the distribution of mass of the body:

\begin{equation}
\begin{array}{l}
I_O = \int\limits r^2_{/O} \,\mathrm d m \\
I_O = \int\limits (x^2_{/O}+y^2_{/O})\,\mathrm d m
\label{}
\end{array}
\end{equation}

For planar movements, we usually compute the moment of inertia relative to the $z$ axis (the axis perpendicular to the plane) and the point to compute the moment of inertia is the body center of mass (later we will see a simple form to calculate the moment of inertia around any axis parallel to the axis passing through the body center of mass). So, a common notation is:

\begin{equation}
\begin{array}{l}
I^{cm}_{zz} = \int\limits (x^2_{/cm}+y^2_{/cm})\,\mathrm d m
\label{eq:inmomzzo}
\end{array}
\end{equation}

The double $z$ in fact has a special meaning that will be clear when we consider possible rotations in three-dimensions.

#### Example: Bar

Let's compute the moment of inertia relative to the center of mass for the same bar we computed its center of mass.

The moment of inertia relative to the center of mass is:

\begin{equation}
\begin{array}{l}
I_{zz}^{cm} &= \int\limits x_{/cm}^2\,\mathrm d m \\
&= \int\limits_{-l/2}^{l/2} x^2\rho\,\mathrm d x \\
&= \rho\left.\left(\dfrac{x^3}{3}\right)\right|_{-l/2}^{l/2} \\
&= \rho\left(\dfrac{l^3}{24}+\dfrac{l^3}{24}\right) \\
&= \rho\dfrac{l^3}{12} \\
I_{zz}^{cm} &= \dfrac{ml^2}{12}
\label{}
\end{array}
\end{equation}

Or using the Sympy library:

```python
# define variables
x = sym.symbols('x')
m1, rho, ell = sym.symbols('m, rho, ell', positive=True)
m = rho*ell
# solve the integral
Icm = rho * sym.integrate(x**2, (x, -ell/2, ell/2))/m

print2('I_{zz}^{cm}', Icm*m1)
```

$\displaystyle I_{zz}^{cm} = \frac{\ell^{2} m}{12}$

### Radius of gyration

Radius of gyration is defined as the distance that a particle would be from the a point $O$ to have the same moment of inertia that the body has. So, the radius of gyration is defined as:

\begin{equation}
\begin{array}{l}
r_{gyr} = \sqrt{\dfrac{I_{zz}^{cm}}{m}}
\label{}
\end{array}
\end{equation}

#### Example: Bar

For a homogeneous bar with length $l$, the radius of gyration is:

\begin{equation}
\begin{array}{l}
r_{gyr} = \sqrt{\dfrac{\dfrac{ml^2}{12}}{m}}=\sqrt{\dfrac{l^2}{12}} = \dfrac{l\sqrt{3}}{6}
\label{}
\end{array}
\end{equation}

This means that for the bar, the moment of inertia around an axis passing by its center of mass is the same as of a particle with equal mass but located at $r_{gyr}$ from this axis.

### Parallel axis theorem

If we have computed the moment of inertia relative to the center of mass for an axis, for example the $z$ axis, and now want to compute the moment of inertia relative to another point $O$ for an axis parallel to the first, there is an expression to aid the computation of this moment of inertia.

In the figure below, the axis is perpendicular to the plane.

Figure. The parallel axis theorem for the calculation of the moment of inertia of a body.

The moment of inertia relative to the axis passing through the point O is given by:

\begin{equation}
\begin{array}{l}
I_{zz}^O &= \int\limits_B x_{/O}^2+y_{/O}^2\,\mathrm d m \\
&= \int\limits_B (x_{/cm}+x_{cm/O})^2+(y_{/cm}+y_{cm/O})^2\,\mathrm d m \\
&= \int\limits_B x_{/cm}^2+2x_{/cm}x_{cm/O}+x_{cm/O}^2+y_{/cm}^2+2y_{/cm}y_{cm/O}+y_{cm/O}^2\,\mathrm d m \\
&=\underbrace{\int\limits_B x_{/cm}^2+y_{/cm}^2\,\mathrm d m}_{I_{zz}^{cm}} +\int\limits_B 2x_{/cm}x_{cm/O}\,\mathrm d m+\int\limits_B x_{cm/O}^2+y_{cm/O}^2\,\mathrm d m + \int\limits_B 2y_{/cm}y_{cm/O}\,\mathrm d m \\
&= I_{zz}^{cm} +2x_{cm/O}\underbrace{\int\limits_B x_{/cm}\,\mathrm d m}_{0}+\underbrace{\vphantom{\int\limits_B}(x_{cm/O}^2+y_{cm/O}^2)}_{d^2}\underbrace{\int\limits_B \,\mathrm d m \vphantom{\Bigg)}}_{m_B} + 2y_{cm/O} \underbrace{\int\limits_B y_{/cm}\,\mathrm d m}_{0} \\
I_{zz}^{O} &=  I_{zz}^{cm} + m_B\,d^2
\label{}
\end{array}
\end{equation}

The terms $\int\limits_B x_{/cm}\,\mathrm d m$ and $\int\limits_B y_{/cm}\,\mathrm d m$ are equal to zero because of the definition of center of mass. The term $d$ is the distance between the two axes.
Note that this theorem is valid only for parallel axes and the original axis passes through the center of mass.

#### Example: Bar

Let's compute the moment of inertia of a bar relative to one of its extremities, by using the parallel axis theorem.

\begin{equation}
\begin{array}{l}
I_{zz}^O &= I_{zz}^{cm} + m\left(\dfrac{l}{2}\right)^2 \\
&= \dfrac{ml^2}{12} + \dfrac{ml^2}{4} = \dfrac{ml^2}{3}
\label{}
\end{array}
\end{equation}

Verify that for the example where we calculated the moment of inertia of a system with two particles in relation to axes passing by different points, we could have used the parallel axis theorem.

### Moment of inertia of a complex system

To compute the moment of inertia of a set o $n$ bodies relative to a point $O$ for a given axis (for example, the $z$ axis), we must apply the Eq.(\ref{eq:inmomzzo}) for all bodies:

\begin{equation}
\begin{array}{l}
I_{zz}^{O} &= \int\limits_{B1,B2,\ldots,Bn} x_{/O}^2+y_{/O}^2\,\mathrm d m \\
&= \int\limits_{B1} x_{/O}^2+y_{/O}^2\,\mathrm d m +\int\limits_{B2} x_{/O}^2+y_{/O}^2\,\mathrm d m +\ldots+\int\limits_{Bn} x_{/O}^2+y_{/O}^2\,\mathrm d m
\label{}
\end{array}
\end{equation}

Now, using the parallel axis theorem, we can write the equation above in terms of the moment of inertia relative to the center of mass of each body:

\begin{equation}
\begin{array}{l}
I_{zz}^{O} &= I_{zz_{B1}}^{cm_1} + m_{B1}.||\vec{r}_{cm_1/O}||^2 +I_{zz_{B2}}^{cm_2} + m_{B2}.||\vec{r}_{cm_2/O}||^2  +\ldots + I_{zz_{Bn}}^{cm_n} + m_{Bn}.||\vec{r}_{cm_n/O}||^2
\label{eq:paral}
\end{array}
\end{equation}

where $I_{zz_{Bi}}^{cm_i}$ is the moment of inertia of the body $i$ relative to its center of mass.

#### Example: Eight bars

Let's compute the moment of inertia of the set of eight equal and homogeneous bars depicted in the figure below relative to its center of mass.

Figure. Moment of inertia of a system with eight bars.

By symmetry, the center of mass of this system is in the point $O$. So, to compute the moment of inertia relative to this point, we must use the parallel axis theorem to each bar, and then sum the results, as in Eq.(\ref{eq:paral}).

\begin{equation}
\begin{array}{l}
I^{cm}_{zz}={} & I^{cm1}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm2}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm3}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm4}_{zz}+m\left(\frac{l}{2}\right)^2 + \\
& I^{cm5}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm6}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm7}_{zz}+m\left(\frac{l}{2}\right)^2+I^{cm8}_{zz}+m\left(\frac{l}{2}\right)^2
\label{}
\end{array}
\end{equation}

The moment of inertia of a bar relative to its center of mass is $\frac{ml^2}{12}$.
So the moment of inertia of the system relative to its center of mass is:

\begin{equation}
\begin{array}{l}
I^{cm}_{zz} = 8\cdot\left[\dfrac{ml^2}{12}+m\left(\dfrac{l}{2}\right)^2\right] = \dfrac{8ml^2}{3}
\label{}
\end{array}
\end{equation}

Verify that for this example, we also could have explored even more the symmetry of the system and computed the total moment of inertia as the sum of the moment of inertia of four bars, each with length $2l$ and mass $2m$, each rotating around its center.

### Matrix of Inertia

Let's find the expression for the moment of inertia of a three-dimensional problem for a system rotating around an axis where we can't reduce the problem to a possible planar rotation. But let's continue considering that the axis of rotation passes through the center of mass (one can always use the theorem of parallel axis).

In this case, the angular momentum for a system composed of a set of particles is:

\begin{equation}
\begin{array}{l}
\vec{H}_{cm} &= \sum \vec{r}_{i/cm} \times \vec{p}_i \\
&= \sum \vec{r}_{i/cm} \times m_i\vec{v}_i \\
&= \sum \vec{r}_{i/cm} \times m_i(\vec{\omega}_i \times \vec{r}_{i/cm}) \\
\vec{H}_{cm} &= \sum m_i (\vec{r}_{i/cm} \times (\vec{\omega}_i \times \vec{r}_{i/cm}))
\label{}
\end{array}
\end{equation}

Calculating the vector triple product (mnemonic: abc = bac - cab) and separating the terms for each component of the angular momentum:

\begin{equation}
\begin{array}{r}
H_{x/cm} &=& \sum m_i \big(&(y_i^2+z_i^2)\omega_x\hat{i} &-x_iy_i\omega_y\hat{j} &-x_iz_i\omega_z\hat{k} \big) \\
H_{y/cm} &=& \sum m_i \big(&-y_ix_i\omega_x\hat{i} &(z_i^2+x_i^2)\omega_y\hat{j} &-y_iz_i\omega_z\hat{k} \big) \\
H_{z/cm} &=& \sum m_i \big(&-z_ix_i\omega_x\hat{i} &-z_iy_i\omega_y\hat{j} &(x_i^2+y_i^2)\omega_z\hat{k} \big)
\end{array}
\label{}
\end{equation}

Once again, in analogy to the linear momentum, we can write the equation above as $\vec{H}_{cm} = I_{cm} \vec{w}$, which in matrix form is:

\begin{equation}
\begin{bmatrix}
H_x \\ H_y \\ H_z
\end{bmatrix} =
\begin{bmatrix}
I_{xx} & I_{xy} & I_{xz} \\
I_{yx} & I_{yy} & I_{yz} \\
I_{zx} & I_{zx} & I_{zz}
\end{bmatrix}  \cdot
\begin{bmatrix}
\omega_x \\ \omega_y \\ \omega_z
\end{bmatrix}
\label{}
\end{equation}

The matrix with the moment of inertia terms in the equation above is referred as the matrix of inertia (or  inertia matrix or inertia tensor):

\begin{equation}
I^{cm} = \begin{bmatrix}
&\sum m_i (y_i^2+z_i^2) &-\sum m_i x_iy_i &-\sum m_i x_iz_i \\
&-\sum m_i y_ix_i &\sum m_i (z_i^2+x_i^2) &-\sum m_i y_iz_i \\
&-\sum m_i z_ix_i &-\sum m_i z_iy_i &\sum m_i (x_i^2+y_i^2)
\end{bmatrix}
\label{}
\end{equation}

Equivalently, for a rigid body the matrix of inertia is:

\begin{equation}
I^{cm} = \begin{bmatrix}
&\int (y_i^2+z_i^2)\,\mathrm d m &-\int x_iy_i\,\mathrm d m &-\int x_iz_i\,\mathrm d m \\
&-\int y_ix_i\,\mathrm d m &\int (z_i^2+x_i^2)\,\mathrm d m &-\int y_iz_i\,\mathrm d m \\
&-\int z_ix_i\,\mathrm d m &-\int z_iy_i\,\mathrm d m &\int (x_i^2+y_i^2)\,\mathrm d m
\end{bmatrix}
\label{}
\end{equation}

It's usual to refer to the matrix of inertia simply by:

\begin{equation}
I^{cm} = \begin{bmatrix}
I_{xx} & I_{xy} & I_{xz} \\
I_{yx} & I_{yy} & I_{yz} \\
I_{zx} & I_{zx} & I_{zz}
\end{bmatrix}
\label{}
\end{equation}

Note that in the notation of subscripts with two axes, the first subscript refers to the component (axis) of the angular momentum, and the second subscript refers to the component (axis) of the angular velocity.

#### Principal axes

If the axes of rotation to calculate the moment of inertia are aligned with the axes of symmetry of the body passing through the center of mass (referred as principal axes), the terms off the main diagonal of the matrix of inertia are all zero because, by definition of center of mass, the mass is equally distributed around these axes of symmetry for each pair of coordinates considered in the integration. The terms off-diagonal are called product of inertia. In this case, the matrix of inertia becomes:

\begin{equation}
I^{cm} = \begin{bmatrix}
I_1 & 0 & 0 \\
0 & I_2 & 0 \\
0 & 0 & I_3
\end{bmatrix}
\label{}
\end{equation}

where $I_1$, $I_2$ and $I_3$ are the moments of inertia around each of the principal axes of the body.

This is a common strategy employed in biomechanics; in motion analysis we construct a base for each segment such that this base is aligned to the principal axes of the segment, and we do that for every instant the segment moves.

#### Example: Cylinder

Consider a cylinder with constant density shown in the figure below and compute the moment of inertia around the three axes relative to the center of mass.
The mass of this cylinder is $m = \rho h \pi R^2$ and, by symmetry, the moment of inertia is in the center (centroid) of the cylinder.

Figure. Moment of inertia of a cylinder.

The easiest approach to this problem is to use cylindrical coordinates $\theta, r, z$. In the integrals, the differential term $\mathrm d m$ will be replaced by $\rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z$ and the limits of the integral will be $[0,2\pi]$, $[0,R]$, $[-h/2,h/2]$.

First, around the $z$ axis:

\begin{equation}
\begin{array}{l}
I_{zz}^{cm} &= \int\limits_B (x_{/cm}^2+y_{/cm}^2)\,\mathrm d m \\
&= \int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r^2 \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
&= \rho \int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r^3  \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
&= \rho \int\limits_{-h/2}^{h/2}\int\limits_0^R 2\pi r^3  \,\mathrm d r\,\mathrm d z \\
&= \rho \int\limits_{-h/2}^{h/2} 2\pi \frac{R^4}{4}  \,dz \\
&=  \rho \pi \frac{R^4}{2}h \\
I_{zz}^{cm} &= \dfrac{mR^2}{2}
\label{}
\end{array}
\end{equation}

Now, around the $x$ axis:

\begin{equation}
\begin{array}{l}
I_{xx}^{cm} &= \int\limits_B (z_{/cm}^2+y_{/cm}^2)\,\mathrm d m \\
&= \int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} (z^2+r^2\sin^2(\theta)) \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{xx}^{cm} &= \rho\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} (r z^2+r^3\sin^2(\theta))   \,\mathrm d \theta\,\mathrm d r\,\mathrm d z
\label{}
\end{array}
\end{equation}

And by symmetry, the moment of inertia around the axis $y$ passing by the center of mass is equal to the moment of inertia around the axis $x$.

We will solve this integral using the Sympy library:

```python
# define variables
h, R, rho, m1 = sym.symbols('h, R, rho, m', positive=True)
theta, r, z = sym.symbols('theta, r, z')
m = rho*h*sym.pi*R**2
# solve the integral
Ixx = rho*sym.integrate(sym.integrate(sym.integrate(r*z**2 + (r**3)*sym.sin(theta)**2,
                                                    (theta, 0, 2*sym.pi)),
                                      (r, 0, R)),
                        (z, -h/2, h/2))/m
Ixxcm = sym.simplify(m1*Ixx)

print2('I^{cm}_{xx}=I^{cm}_{yy}', Ixxcm)
```

$\displaystyle I^{cm}_{xx}=I^{cm}_{yy} = \frac{m \left(3 R^{2} + h^{2}\right)}{12}$

**The product of inertia terms**

By inspecting the problem, we know that because of symmetry of the axes w.r.t. the cylinder, the product of inertia terms of the matrix of inertia should be all zero. Let's confirm that.
Here are the integrals:

\begin{equation}
\begin{array}{l}
I_{xy}^{cm} &= -\int x_{/cm}y_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r\cos\theta \,r\sin\theta \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{yx}^{cm} &= -\int y_{/cm}x_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r\sin\theta \,r\cos\theta \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{xz}^{cm} &= -\int x_{/cm}z_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r\cos\theta \,z \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{zx}^{cm} &= -\int z_{/cm}x_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} z \, r\cos\theta  \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{yz}^{cm} &= -\int y_{/cm}z_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} r\sin\theta \, z \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z \\
I_{zy}^{cm} &= -\int z_{/cm}y_{/cm}\,\mathrm d m \\
&= -\int\limits_{-h/2}^{h/2}\int\limits_0^R\int\limits_0^{2\pi} z \, \,r\sin\theta \rho r \,\mathrm d \theta\,\mathrm d r\,\mathrm d z
\label{}
\end{array}
\end{equation}

The product of inertia terms are indeed all zero.

### The parallel axis theorem for rigid bodies in three dimensions

The parallel axis theorem for rigid bodies in three dimensions is (see page 1083 of Ruina and Pratap, (2019)):

\begin{equation}
\begin{array}{l}
I^O = I^{cm} + m\begin{bmatrix}
& y_{cm/O}^2+z_{cm/O}^2 &-x_{cm/O}y_{cm/O}      &-x_{cm/O}z_{cm/O} \\
&-y_{cm/O}x_{cm/O}      & z_{cm/O}^2+x_{cm/O}^2 &-y_{cm/O}z_{cm/O} \\
&-z_{cm/O}x_{cm/O}      &-z_{cm/O}y_{cm/O}      & x_{cm/O}^2+y_{cm/O}^2
\end{bmatrix}
\label{}
\end{array}
\end{equation}

where $x_{cm/O}, y_{cm/O}, z_{cm/O}$ are the $x, y, z$ coordinates of the center of mass with respect to a coordinate system whose origin is located at point $O$.

The terms $I^O_{i,j}$ of the inertia tensor above can be represented in a shorter form as:

\begin{equation}
\begin{array}{l}
I^O_{ij} = I^{cm}_{ij} + m\left(|\vec{r}_{cm/O}|^2\delta_{ij} - r_{i,cm/O}r_{j,cm/O}\right)
\label{}
\end{array}
\end{equation}

where $\vec{r}_{cm/O} = x_{cm/O} \hat{i} + y_{cm/O}\hat{j} + z_{cm/O}\hat{k}$, $\delta_{ij}$ is the Kronecker delta, and $i,j$ varies from 1 to 3 (which also span the $x, y, z$ coordinates).

### Moment of rotational inertia and area moment of inertia

Another related quantity to moment of rotational inertia is the area moment of inertia; the latter is more used in the field of strength of materials. The area moment of inertia is a property of the shape of a body, it expresses the difficulty to deflect, bend or stress this body, and is given by:

\begin{equation}
\begin{array}{l}
I = \int\limits r^2\,\mathrm d A
\label{}
\end{array}
\end{equation}

where $\mathrm d A$ is the differential of area.

One can see that the area moment of inertia has no mass term; it has dimension $[L]^4$ and SI unit $m^4$.
To salient the difference between these two quantities, the moment of rotational inertia is also known as mass moment of inertia.

## Further reading

- Center of mass: Read pages pages 145-160 of the [Ruina and Pratap (2019)](http://ruina.tam.cornell.edu/Book/index.html);
- Moment of inertia: Read pages 771-781 and 1081-1089 of the [Ruina and Pratap (2019)](http://ruina.tam.cornell.edu/Book/index.html), read the chapter 17 of the [Hibbeler's book](https://drive.google.com/file/d/1sDLluWCiBCog2C11_Iu1fjv-BtfVUxBU/view) (available in the Classroom).

## Video lectures on the internet

- Center of mass:
  - Khan Academy: [definition](https://www.khanacademy.org/science/ap-physics-1/ap-linear-momentum/center-of-mass-ap/v/center-of-mass), [definição (in Portuguese)](https://pt.khanacademy.org/science/physics/linear-momentum/center-of-mass/v/center-of-mass);
  - MIT OpenCourseWare: [definition](https://youtu.be/ol1COj0LACs), [of three objects](https://youtu.be/-b0dFcebPcs), [of a continuous system](https://youtu.be/e548hRYcXlg), [of extended objects](https://ocw.mit.edu/courses/physics/8-01sc-classical-mechanics-fall-2016/week-5-momentum-and-impulse/17.4-center-of-mass-of-a-system-of-extended-objects), [of a uniform rod](https://youtu.be/CFh3gu-z_rc);
- Moment of inertia:
  - Khan Academy: [definition](https://www.khanacademy.org/science/physics/torque-angular-momentum/torque-tutorial/v/more-on-moment-of-inertia), [definição (in Portuguese)](https://pt.khanacademy.org/science/physics/torque-angular-momentum/torque-tutorial/v/more-on-moment-of-inertia);
  - MIT OpenCourseWare: [definition](https://youtu.be/0QF_uCgZW4Y), [of a rod](https://youtu.be/1AJbVRQTZlA), [of a disc](https://youtu.be/BPnbq6BobdA), [parallel axis theorem](https://youtu.be/r2Qb0vsxa8Y), [of a sphere](https://youtu.be/QmCQUBSsKwQ), [matrix of inertia](https://youtu.be/lT-GIGebbNc);
 - [Difference between mass and area moments of inertia](https://youtu.be/Bls5KnQOWkY).

## Problems

1. Solve problems 2.2.5, 2.2.7, 16.3.11, 16.3.15 from Ruina and Pratap (2019).
2. Determine the center of mass position and moment of inertia (around the principal axes of the object) for (adopt mass equal 1 kg):
  1. A planar disc with radius $r$.
  2. A planar square with side $a$.
  3. A planar ellipse with semi-axes $a$ and $b$.
  4. A sphere with radius $r$.
  5. A cube with side $a$.
  6. An ellipsoid with semi-axes $a$, $b$ and $c$ (see http://scienceworld.wolfram.com/physics/MomentofInertiaEllipsoid.html).
3. Calculate the matrix of inertia for the following systems:
  1. A particle with mass 1 kg located at $[1, 2, 0]\,m$ in relation to the origin $[0, 0, 0]\,m$.
  2. A set of two particles, each with mass 1 kg, located at $[1, 2, 0]\,m$ and $[-1, -2, 0]\,m$ in relation to the set center of mass.
4. Check examples 17.1 e 17.2 from Hibbeler's book.

## References

- Ruina A, Rudra P (2019) [Introduction to Statics and Dynamics](http://ruina.tam.cornell.edu/Book/index.html). Oxford University Press.
- R. C. Hibbeler (2010) [Engineering Mechanics Dynamics](https://drive.google.com/file/d/1sDLluWCiBCog2C11_Iu1fjv-BtfVUxBU/view). 12th Edition. Pearson Prentice Hall.


#  NMA Model Types Tutorial 2: "How" models

In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced. That is, the models will tell us something about the *mechanism* underlying the physiological phenomenon.

Our objectives:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video: "How" models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='yWPQsBud4Cc', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=yWPQsBud4Cc

## Setup

**Don't forget to execute the hidden cells!**

```python
#@title Imports
import matplotlib.pyplot as plt
import numpy as np
import scipy.stats as stats

import ipywidgets as widgets
```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='limegreen')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
```

## The Linear Integrate-and-Fire Neuron

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value of 1, a spike is emitted, the neuron resets $V_m$ back to 0, and the process continues.

#### Spiking Inputs

We now have a model for the neuron dynamics. Next we need to consider what form the input $I$ will take. How should we represent the presynaptic neuron firing behavior providing the input coming into our model nueuron? We learned previously that a good approximation of spike timing is a Poisson random variable, so we can do that here as well

\begin{align}
  I \sim Poisson(\lambda)
\end{align}

where $\lambda$ is the average rate of incoming spikes.

### Exercise: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ of the linear integrate-and-fire model neuron. The rest of the code to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` method below. The value for $\lambda$ needed for the Poisson random variable is named `rate`.

TIP: The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    #######################################################################
    ## TODO for students: compute dv, then remove the NotImplementedError #
    #######################################################################
    # dv = ...
    raise NotImplementedError("Student excercise: compute the change in membrane potential")

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
# v, spike_times = lif_neuron()
# plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

v, spike_times = lif_neuron()
with plt.xkcd():
  plot_neuron_stats(v, spike_times)
```

### Parameter Exploration

Here's an interactive demo that shows how the model behavior changes for different parameter values.

**Remember to enable the demo by running the cell.**

```python
#@title Linear Integrate-and-Fire Model Neuron Explorer

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-4, max=-1),
  rate=widgets.IntSlider(10, min=1, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

## Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes, but the actual spiking behavior did not line up with our expectations of exponentially distributed ISIs. This means we need to refine our model!

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease is upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendancy of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable, giving us

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim Poisson(\lambda_{exc}) \\
I_{inh} \sim Poisson(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the rates of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    ############################################################
    ## Students: compute dv and remove the NotImplementedError #
    ############################################################
    # dv = ...
    raise NotImplementedError("Student excercise: compute the change in membrane potential")

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
# v, spike_times = lif_neuron_inh()
# plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times
```

```python
v, spike_times = lif_neuron_inh()
plot_neuron_stats(v, spike_times)
```

### Parameter Exploration

Like last time, you can now explore how you LIF model behaves when the various parameters of the system are changed.

```python
#@title LIF Model Neuron with Inhibitory Inputs Explorer

def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-2, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-2, max=0),
                  exc_rate=widgets.IntSlider(10, min=1, max=20),
                  inh_rate=widgets.IntSlider(10, min=1, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video: Balanced inputs
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='buXEQPp9LKI', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=buXEQPp9LKI

## Appendix

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $𝑉_{𝑟𝑒𝑠𝑡}$ is the resting potential, and  𝐼  is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these properties to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the overall behavior, though these too can be manipulated to achieve different dynamics.


## Histograms of Oriented Gradients (HOG)

As we saw with the ORB algorithm, we can use keypoints in images to do keypoint-based matching to detect objects in images. These type of algorithms work great when you want to detect objects that have a lot of consistent internal features that are not affected by the background. For example, these algorithms work well for facial detection because faces have a lot of consistent internal features that don’t get affected by the image background, such as the eyes, nose, and mouth. However, these type of algorithms don’t work so well when attempting to do more general object recognition, say for example, pedestrian detection in images. The reason is that people don’t have consistent internal features, like faces do, because the body shape and style of every person is different (see Fig. 1). This means that every person is going to have a different set of internal features, and so we need something that can more generally describe a person.

Fig. 1. - Pedestrians.

One option is to try to detect pedestrians by their contours instead.  Detecting objects in images by their contours (boundaries) is very challenging because we have to deal with the difficulties brought about by the contrast between the background and the foreground. For example, suppose you wanted to detect a pedestrian in an image that is walking in front of a white building and she is wearing a white coat and black pants (see Fig. 2). We can see in Fig. 2, that since the background of the image is mostly white, the black pants are going to have a very high contrast, but the coat, since it is white as well, is going to have very low contrast. In this case, detecting the edges of pants is going to be easy but detecting the edges of the coat is going to be very difficult. This is where **HOG** comes in. HOG stands for **Histograms of Oriented Gradients** and it was first introduced by Navneet Dalal and Bill Triggs in 2005.

Fig. 2. - High and Low Contrast.

The HOG algorithm works by creating histograms of the distribution of gradient orientations in an image and then normalizing them in a very special way. This special normalization is what makes HOG so effective at detecting the edges of objects even in cases where the contrast is very low. These normalized histograms are put together into a feature vector, known as the HOG descriptor, that can be used to train a machine learning algorithm, such as a Support Vector Machine (SVM), to detect objects in images based on their boundaries (edges). Due to its great success and reliability, HOG has become one of the most widely used algorithms in computer vison for object detection.

In this notebook, you will learn:

* How the HOG algorithm works
* How to use OpenCV to create a HOG descriptor
* How to visualize the HOG descriptor.

# The HOG Algorithm

As its name suggests, the HOG algorithm, is based on creating histograms from the orientation of image gradients. The HOG algorithm is implemented in a series of steps:

1. Given the image of particular object, set a detection window (region of interest) that covers the entire object in the image (see Fig. 3).

2. Calculate the magnitude and direction of the gradient for each individual pixel in the detection window.

3. Divide the detection window into connected *cells* of pixels, with all cells being of the same size (see Fig. 3). The size of the cells is a free parameter and it is usually chosen so as to match the scale of the features that want to be detected. For example, in a 64 x 128 pixel detection window, square cells 6 to 8 pixels wide are suitable for detecting human limbs.

4. Create a Histogram for each cell, by first grouping the gradient directions of all pixels in each cell into a particular number of orientation (angular) bins; and then adding up the gradient magnitudes of the gradients in each angular bin (see Fig. 3). The number of bins in the histogram is a free parameter and it is usually set to 9 angular bins.

5. Group adjacent cells into *blocks* (see Fig. 3). The number of cells in each block is a free parameter and all blocks must be of the same size. The distance between each block (known as the stride) is a free parameter but it is usually set to half the block size, in which case you will get overlapping blocks (*see video below*).  The HOG algorithm has been shown empirically to work better with overlapping blocks.

6. Use the cells contained within each block to normalize the cell histograms in that block (see Fig. 3). If you have overlapping blocks this means that most cells will be normalized with respect to different blocks (*see video below*). Therefore, the same cell may have several different normalizations.

7. Collect all the normalized histograms from all the blocks into a single feature vector called the HOG descriptor.

8. Use the resulting HOG descriptors from many images of the same type of object to train a machine learning algorithm, such as an SVM, to detect those type of objects in images. For example, you could use the HOG descriptors from many images of pedestrians to train an SVM to detect pedestrians in images. The training is done with both positive a negative examples of the object you want detect in the image.

9. Once the SVM has been trained, a sliding window approach is used to try to detect and locate objects in images.  Detecting an object in the image entails finding the part of the image that looks similar to the HOG pattern learned by the SVM.

Fig. 3. - HOG Diagram.

Vid. 1. - HOG Animation.

# Why The HOG Algorithm Works

As we learned above, HOG creates histograms by adding the magnitude of the gradients in particular orientations in localized portions of the image called *cells*.  By doing this we guarantee that stronger gradients will contribute more to the magnitude of their respective angular bin, while the effects of weak and randomly oriented gradients resulting from noise are minimized. In this manner the histograms tell us the dominant gradient orientation of each cell.

### Dealing with contrast

Now, the magnitude of the dominant orientation can vary widely due to variations in local illumination and the contrast between the background and the foreground.

To account for the background-foreground contrast differences, the HOG algorithm tries to detect edges locally.  In order to do this, it defines groups of cells, called **blocks**, and normalizes the histograms using this local group of cells. By normalizing locally, the HOG algorithm can detect the edges in each block very reliably; this is called **block normalization**.

In addition to using block normalization, the HOG algorithm also uses overlapping blocks to increase its performance. By using overlapping blocks, each cell contributes several independent components to the final HOG descriptor, where each component corresponds to a cell being normalized with respect to a different block. This may seem redundant but, it has been shown empirically that by normalizing each cell several times with respect to different local blocks, the performance of the HOG algorithm increases dramatically.

### Loading Images and Importing Resources

The first step in building our HOG descriptor is to load the required packages into Python and to load our image.

We start by using OpenCV to load an image of a triangle tile. Since, the `cv2.imread()` function loads images as BGR we will convert our image to RGB so we can display it with the correct colors. As usual we will convert our BGR image to Gray Scale for analysis.

```python
import cv2
import numpy as np
import matplotlib.pyplot as plt

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Load the image
image = cv2.imread('./images/triangle_tile.jpeg')

# Convert the original image to RGB
original_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Convert the original image to gray scale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Print the shape of the original and gray scale images
print('The original image has shape: ', original_image.shape)
print('The gray scale image has shape: ', gray_image.shape)

# Display the images
plt.subplot(121)
plt.imshow(original_image)
plt.title('Original Image')
plt.subplot(122)
plt.imshow(gray_image, cmap='gray')
plt.title('Gray Scale Image')
plt.show()
```

# Creating The HOG Descriptor

We will be using OpenCV’s `HOGDescriptor` class to create the HOG descriptor. The parameters of the HOG descriptor are setup using the `HOGDescriptor()` function. The parameters of the `HOGDescriptor()` function and their default values are given below:

`cv2.HOGDescriptor(win_size = (64, 128),
                  block_size = (16, 16),
                  block_stride = (8, 8),
                  cell_size = (8, 8),
                  nbins = 9,
                  win_sigma = DEFAULT_WIN_SIGMA,
                  threshold_L2hys = 0.2,
                  gamma_correction = true,
                  nlevels = DEFAULT_NLEVELS)`

Parameters:

* **win_size** – *Size*
Size of detection window in pixels (*width, height*). Defines the region of interest. Must be an integer multiple of cell size.

* **block_size** – *Size*
Block size in pixels (*width, height*). Defines how many cells are in each block. Must be an integer multiple of cell size and it must be smaller than the detection window. The smaller the block the finer detail you will get.

* **block_stride** – *Size*
Block stride in pixels (*horizontal, vertical*). It must be an integer multiple of cell size. The `block_stride` defines the distance between adjecent blocks, for example, 8 pixels horizontally and 8 pixels vertically. Longer `block_strides` makes the algorithm run faster (because less blocks are evaluated) but the algorithm may not perform as well.

* **cell_size** – *Size*
Cell size in pixels (*width, height*). Determines the size fo your cell. The smaller the cell the finer detail you will get.

* **nbins** – *int*
Number of bins for the histograms. Determines the number of angular bins used to make the histograms. With more bins you capture more gradient directions. HOG uses unsigned gradients, so the angular bins will have values between 0 and 180 degrees.

* **win_sigma** – *double*
Gaussian smoothing window parameter. The performance of the HOG algorithm can be improved by smoothing the pixels near the edges of the blocks by applying a Gaussian spatial window to each pixel before computing the histograms.

* **threshold_L2hys** – *double*
L2-Hys (Lowe-style clipped L2 norm) normalization method shrinkage. The L2-Hys method is used to normalize the blocks and it consists of an L2-norm followed by clipping and a renormalization. The clipping limits the maximum value of the descriptor vector for each block to have the value of the given threshold (0.2 by default). After the clipping the descriptor vector is renormalized as described in *IJCV*, 60(2):91-110, 2004.

* **gamma_correction** – *bool*
Flag to specify whether the gamma correction preprocessing is required or not. Performing gamma correction slightly increases the performance of the HOG algorithm.

* **nlevels** – *int*
Maximum number of detection window increases.

As we can see, the  `cv2.HOGDescriptor()`function supports a wide range of parameters. The first few arguments (`block_size, block_stride, cell_size`, and `nbins`) are probably the ones you are most likely to change. The other parameters can be safely left at their default values and you will get good results.

In the code below, we will use the `cv2.HOGDescriptor()`function to set the cell size, block size, block stride, and the number of bins for the histograms of the HOG descriptor. We will then use `.compute(image)`method to compute the HOG descriptor (feature vector) for the given `image`.

```python
# Specify the parameters for our HOG descriptor

# Cell Size in pixels (width, height). Must be smaller than the size of the detection window
# and must be chosen so that the resulting Block Size is smaller than the detection window.
cell_size = (6, 6)

# Number of cells per block in each direction (x, y). Must be chosen so that the resulting
# Block Size is smaller than the detection window
num_cells_per_block = (2, 2)

# Block Size in pixels (width, height). Must be an integer multiple of Cell Size.
# The Block Size must be smaller than the detection window
block_size = (num_cells_per_block[0] * cell_size[0],
              num_cells_per_block[1] * cell_size[1])

# Calculate the number of cells that fit in our image in the x and y directions
x_cells = gray_image.shape[1] // cell_size[0]
y_cells = gray_image.shape[0] // cell_size[1]

# Horizontal distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (x_cells - num_cells_per_block[0]) / h_stride = integer.
h_stride = 1

# Vertical distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (y_cells - num_cells_per_block[1]) / v_stride = integer.
v_stride = 1

# Block Stride in pixels (horizantal, vertical). Must be an integer multiple of Cell Size
block_stride = (cell_size[0] * h_stride, cell_size[1] * v_stride)

# Number of gradient orientation bins
num_bins = 9

# Specify the size of the detection window (Region of Interest) in pixels (width, height).
# It must be an integer multiple of Cell Size and it must cover the entire image. Because
# the detection window must be an integer multiple of cell size, depending on the size of
# your cells, the resulting detection window might be slightly smaller than the image.
# This is perfectly ok.
win_size = (x_cells * cell_size[0] , y_cells * cell_size[1])

# Print the shape of the gray scale image for reference
print('\nThe gray scale image has shape: ', gray_image.shape)
print()

# Print the parameters of our HOG descriptor
print('HOG Descriptor Parameters:\n')
print('Window Size:', win_size)
print('Cell Size:', cell_size)
print('Block Size:', block_size)
print('Block Stride:', block_stride)
print('Number of Bins:', num_bins)
print()

# Set the parameters of the HOG descriptor using the variables defined above
hog = cv2.HOGDescriptor(win_size, block_size, block_stride, cell_size, num_bins)

# Compute the HOG Descriptor for the gray scale image
hog_descriptor = hog.compute(gray_image)
```

    The gray scale image has shape:  (250, 250)

    HOG Descriptor Parameters:

    Window Size: (246, 246)
    Cell Size: (6, 6)
    Block Size: (12, 12)
    Block Stride: (6, 6)
    Number of Bins: 9

# Number of Elements In The HOG Descriptor

The resulting HOG Descriptor (feature vector), contains the normalized histograms from all cells from all blocks in the detection window concatenated in one long vector. Therefore, the size of the HOG feature vector will be given by the total number of blocks in the detection window, multiplied by the number of cells per block, times the number of orientation bins:

\begin{equation}
\mbox{total_elements} = (\mbox{total_number_of_blocks})\mbox{ } \times \mbox{ } (\mbox{number_cells_per_block})\mbox{ } \times \mbox{ } (\mbox{number_of_bins})
\end{equation}

If we don’t have overlapping blocks (*i.e.* the `block_stride`equals the `block_size`), the total number of blocks can be easily calculated by dividing the size of the detection window by the block size. However, in the general case we have to take into account the fact that we have overlapping blocks. To find the total number of blocks in the general case (*i.e.* for any `block_stride` and `block_size`), we can use the formula given below:

\begin{equation}
\mbox{Total}_i = \left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right)\left( \frac{\mbox{window_size}_i}{\mbox{block_size}_i} \right) - \left [\left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right) - 1 \right]; \mbox{  for  } i = x,y
\end{equation}

Where Total$_x$, is the total number of blocks along the width of the detection window, and Total$_y$, is the total number of blocks along the height of the detection window. This formula for Total$_x$ and Total$_y$, takes into account the extra blocks that result from overlapping. After calculating Total$_x$ and Total$_y$, we can get the total number of blocks in the detection window by multiplying Total$_x$  $\times$ Total$_y$. The above formula can be simplified considerably because the `block_size`, `block_stride`, and `window_size`are all defined in terms of the `cell_size`. By making all the appropriate substitutions and cancelations the above formula reduces to:

\begin{equation}
\mbox{Total}_i = \left(\frac{\mbox{cells}_i - \mbox{num_cells_per_block}_i}{N_i}\right) + 1\mbox{  }; \mbox{  for  } i = x,y
\end{equation}

Where cells$_x$ is the total number of cells along the width of the detection window, and cells$_y$, is the total number of cells along the height of the detection window. And $N_x$ is the horizontal block stride in units of `cell_size` and $N_y$ is the vertical block stride in units of `cell_size`.

Let's calculate what the number of elements for the HOG feature vector should be and check that it matches the shape of the HOG Descriptor calculated above.

```python
# Calculate the total number of blocks along the width of the detection window
tot_bx = np.uint32(((x_cells - num_cells_per_block[0]) / h_stride) + 1)

# Calculate the total number of blocks along the height of the detection window
tot_by = np.uint32(((y_cells - num_cells_per_block[1]) / v_stride) + 1)

# Calculate the total number of elements in the feature vector
tot_els = (tot_bx) * (tot_by) * num_cells_per_block[0] * num_cells_per_block[1] * num_bins

# Print the total number of elements the HOG feature vector should have
print('\nThe total number of elements in the HOG Feature Vector should be: ',
      tot_bx, 'x',
      tot_by, 'x',
      num_cells_per_block[0], 'x',
      num_cells_per_block[1], 'x',
      num_bins, '=',
      tot_els)

# Print the shape of the HOG Descriptor to see that it matches the above
print('\nThe HOG Descriptor has shape:', hog_descriptor.shape)
print()
```

    The total number of elements in the HOG Feature Vector should be:  40 x 40 x 2 x 2 x 9 = 57600

    The HOG Descriptor has shape: (57600, 1)

# Visualizing The HOG Descriptor

We can visualize the HOG Descriptor by plotting the histogram associated with each cell as a collection of vectors. To do this, we will plot each bin in the histogram as a single vector whose magnitude is given by the height of the bin and its orientation is given by the angular bin that its associated with. Since any given cell might have multiple histograms associated with it, due to the overlapping blocks, we will choose to average all the histograms for each cell to produce a single histogram for each cell.

OpenCV has no easy way to visualize the HOG Descriptor, so we have to do some manipulation first in order to visualize it. We will start by reshaping the HOG Descriptor in order to make our calculations easier.  We will then compute the average histogram of each cell and finally we will convert the histogram bins into vectors. Once we have the vectors, we plot the corresponding vectors for each cell in an image.

The code below produces an interactive plot so that you can interact with the figure. The figure contains:
* the grayscale image,
* the HOG Descriptor (feature vector),
* a zoomed-in portion of the HOG Descriptor, and
* the histogram of the selected cell.

**You can click anywhere on the gray scale image or the HOG Descriptor image to select a particular cell**. Once you click on either image a *magenta* rectangle will appear showing the cell you selected. The Zoom Window will show you a zoomed in version of the HOG descriptor around the selected cell; and the histogram plot will show you the corresponding histogram for the selected cell. The interactive window also has buttons at the bottom that allow for other functionality, such as panning, and giving you the option to save the figure if desired. The home button returns the figure to its default value.

**NOTE**: If you are running this notebook in the Udacity workspace, there is around a 2 second lag in the interactive plot. This means that if you click in the image to zoom in, it will take about 2 seconds for the plot to refresh.

```python
%matplotlib notebook

import copy
import matplotlib.patches as patches

# Set the default figure size
plt.rcParams['figure.figsize'] = [9.8, 9]

# Reshape the feature vector to [blocks_y, blocks_x, num_cells_per_block_x, num_cells_per_block_y, num_bins].
# The blocks_x and blocks_y will be transposed so that the first index (blocks_y) referes to the row number
# and the second index to the column number. This will be useful later when we plot the feature vector, so
# that the feature vector indexing matches the image indexing.
hog_descriptor_reshaped = hog_descriptor.reshape(tot_bx,
                                                 tot_by,
                                                 num_cells_per_block[0],
                                                 num_cells_per_block[1],
                                                 num_bins).transpose((1, 0, 2, 3, 4))

# Print the shape of the feature vector for reference
print('The feature vector has shape:', hog_descriptor.shape)

# Print the reshaped feature vector
print('The reshaped feature vector has shape:', hog_descriptor_reshaped.shape)

# Create an array that will hold the average gradients for each cell
ave_grad = np.zeros((y_cells, x_cells, num_bins))

# Print the shape of the ave_grad array for reference
print('The average gradient array has shape: ', ave_grad.shape)

# Create an array that will count the number of histograms per cell
hist_counter = np.zeros((y_cells, x_cells, 1))

# Add up all the histograms for each cell and count the number of histograms per cell
for i in range (num_cells_per_block[0]):
    for j in range(num_cells_per_block[1]):
        ave_grad[i:tot_by + i,
                 j:tot_bx + j] += hog_descriptor_reshaped[:, :, i, j, :]

        hist_counter[i:tot_by + i,
                     j:tot_bx + j] += 1

# Calculate the average gradient for each cell
ave_grad /= hist_counter

# Calculate the total number of vectors we have in all the cells.
len_vecs = ave_grad.shape[0] * ave_grad.shape[1] * ave_grad.shape[2]

# Create an array that has num_bins equally spaced between 0 and 180 degress in radians.
deg = np.linspace(0, np.pi, num_bins, endpoint = False)

# Each cell will have a histogram with num_bins. For each cell, plot each bin as a vector (with its magnitude
# equal to the height of the bin in the histogram, and its angle corresponding to the bin in the histogram).
# To do this, create rank 1 arrays that will hold the (x,y)-coordinate of all the vectors in all the cells in the
# image. Also, create the rank 1 arrays that will hold all the (U,V)-components of all the vectors in all the
# cells in the image. Create the arrays that will hold all the vector positons and components.
U = np.zeros((len_vecs))
V = np.zeros((len_vecs))
X = np.zeros((len_vecs))
Y = np.zeros((len_vecs))

# Set the counter to zero
counter = 0

# Use the cosine and sine functions to calculate the vector components (U,V) from their maginitudes. Remember the
# cosine and sine functions take angles in radians. Calculate the vector positions and magnitudes from the
# average gradient array
for i in range(ave_grad.shape[0]):
    for j in range(ave_grad.shape[1]):
        for k in range(ave_grad.shape[2]):
            U[counter] = ave_grad[i,j,k] * np.cos(deg[k])
            V[counter] = ave_grad[i,j,k] * np.sin(deg[k])

            X[counter] = (cell_size[0] / 2) + (cell_size[0] * i)
            Y[counter] = (cell_size[1] / 2) + (cell_size[1] * j)

            counter = counter + 1

# Create the bins in degress to plot our histogram.
angle_axis = np.linspace(0, 180, num_bins, endpoint = False)
angle_axis += ((angle_axis[1] - angle_axis[0]) / 2)

# Create a figure with 4 subplots arranged in 2 x 2
fig, ((a,b),(c,d)) = plt.subplots(2,2)

# Set the title of each subplot
a.set(title = 'Gray Scale Image\n(Click to Zoom)')
b.set(title = 'HOG Descriptor\n(Click to Zoom)')
c.set(title = 'Zoom Window', xlim = (0, 18), ylim = (0, 18), autoscale_on = False)
d.set(title = 'Histogram of Gradients')

# Plot the gray scale image
a.imshow(gray_image, cmap = 'gray')
a.set_aspect(aspect = 1)

# Plot the feature vector (HOG Descriptor)
b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
b.invert_yaxis()
b.set_aspect(aspect = 1)
b.set_facecolor('black')

# Define function for interactive zoom
def onpress(event):

    #Unless the left mouse button is pressed do nothing
    if event.button != 1:
        return

    # Only accept clicks for subplots a and b
    if event.inaxes in [a, b]:

        # Get mouse click coordinates
        x, y = event.xdata, event.ydata

        # Select the cell closest to the mouse click coordinates
        cell_num_x = np.uint32(x / cell_size[0])
        cell_num_y = np.uint32(y / cell_size[1])

        # Set the edge coordinates of the rectangle patch
        edgex = x - (x % cell_size[0])
        edgey = y - (y % cell_size[1])

        # Create a rectangle patch that matches the the cell selected above
        rect = patches.Rectangle((edgex, edgey),
                                  cell_size[0], cell_size[1],
                                  linewidth = 1,
                                  edgecolor = 'magenta',
                                  facecolor='none')

        # A single patch can only be used in a single plot. Create copies
        # of the patch to use in the other subplots
        rect2 = copy.copy(rect)
        rect3 = copy.copy(rect)

        # Update all subplots
        a.clear()
        a.set(title = 'Gray Scale Image\n(Click to Zoom)')
        a.imshow(gray_image, cmap = 'gray')
        a.set_aspect(aspect = 1)
        a.add_patch(rect)

        b.clear()
        b.set(title = 'HOG Descriptor\n(Click to Zoom)')
        b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
        b.invert_yaxis()
        b.set_aspect(aspect = 1)
        b.set_facecolor('black')
        b.add_patch(rect2)

        c.clear()
        c.set(title = 'Zoom Window')
        c.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 1)
        c.set_xlim(edgex - cell_size[0], edgex + (2 * cell_size[0]))
        c.set_ylim(edgey - cell_size[1], edgey + (2 * cell_size[1]))
        c.invert_yaxis()
        c.set_aspect(aspect = 1)
        c.set_facecolor('black')
        c.add_patch(rect3)

        d.clear()
        d.set(title = 'Histogram of Gradients')
        d.grid()
        d.set_xlim(0, 180)
        d.set_xticks(angle_axis)
        d.set_xlabel('Angle')
        d.bar(angle_axis,
              ave_grad[cell_num_y, cell_num_x, :],
              180 // num_bins,
              align = 'center',
              alpha = 0.5,
              linewidth = 1.2,
              edgecolor = 'k')

        fig.canvas.draw()

# Create a connection between the figure and the mouse click
fig.canvas.mpl_connect('button_press_event', onpress)
plt.show()
```

    The feature vector has shape: (57600, 1)
    The reshaped feature vector has shape: (40, 40, 2, 2, 9)
    The average gradient array has shape:  (41, 41, 9)

    <IPython.core.display.Javascript object>

# Understanding The Histograms

Let's take a look at a couple of snapshots of the above figure to see if the histograms for the selected cell make sense. Let's start looking at a cell that is inside a triangle and not near an edge:

Fig. 4. - Histograms Inside a Triangle.

In this case, since the triangle is nearly all of the same color there shouldn't be any dominant gradient in the selected cell. As we can clearly see in the Zoom Window and the histogram, this is indeed the case. We have many gradients but none of them clearly dominates over the other.

Now let’s take a look at a cell that is near a horizontal edge:

Fig. 5. - Histograms Near a Horizontal Edge.

Remember that edges are areas of an image where the intensity changes abruptly. In these cases, we will have a high intensity gradient in some particular direction. This is exactly what we see in the corresponding histogram and Zoom Window for the selected cell.  In the Zoom Window, we can see that the dominant gradient is pointing up, almost at 90 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 90-degree bin in the histogram to dominate strongly over the others. This is in fact what we see.

Now let’s take a look at a cell that is near a vertical edge:

Fig. 6. - Histograms Near a Vertical Edge.

In this case we expect the dominant gradient in the cell to be horizontal, close to 180 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 170-degree bin in the histogram to dominate strongly over the others. This is what we see in the histogram but we also see that there is another dominant gradient in the cell, namely the one in the 10-degree bin. The reason for this, is because the HOG algorithm is using unsigned gradients, which means 0 degrees and 180 degrees are considered the same. Therefore, when the histograms are being created, angles between 160 and 180 degrees, contribute proportionally to both the 10-degree bin and the 170-degree bin. This results in there being two dominant gradients in the cell near the vertical edge instead of just one.

To conclude let’s take a look at a cell that is near a diagonal edge.

Fig. 7. - Histograms Near a Diagonal Edge.

To understand what we are seeing, let’s first remember that gradients have an *x*-component, and a *y*-component, just like vectors. Therefore, the resulting orientation of a gradient is going to be given by the vector sum of its components. For this reason, on vertical edges the gradients are horizontal, because they only have an x-component, as we saw in Figure 4. While on horizontal edges the gradients are vertical, because they only have a y-component, as we saw in Figure 3. Consequently, on diagonal edges, the gradients are also going to be diagonal because both the *x* and *y* components are non-zero.  Since the diagonal edges in the image are close to 45 degrees, we should expect to see a dominant gradient orientation in the 50-degree bin. This is in fact what we see in the histogram but, just like in Figure 4., we see there are two dominant gradients instead of just one. The reason for this is that when the histograms are being created, angles that are near the boundaries of bins, contribute proportionally to the adjacent bins.  For example, a gradient with an angle of 40 degrees, is right in the middle of the 30-degree and 50-degree bin. Therefore, the magnitude of the gradient is split evenly into the 30-degree and 50-degree bin. This results in there being two dominant gradients in the cell near the diagonal edge instead of just one.

Now that you know how HOG is implemented, in the workspace you will find a notebook named *Examples*. In there, you will be able set your own paramters for the HOG descriptor for various images. Have fun!


Solving Inhomogeneous LTI Differential Equations

# Preliminaries

```python
from common_python.LTIModel.LTISolver import LTISolver
from common_python.sympy import sympyUtil as su

import sympy
import numpy as np
from sympy import init_printing
init_printing()
import tellurium as te
import matplotlib.pyplot as plt
```

## Latex Macros
Macro definitions

* change in species concentrations: \dxdt - how $x$ changes with respect to $t$
$\newcommand{\dxdt}[]{ \frac{d {\bf x}}{d t}}$: $\dxdt$

* change in parameter values: \dpdt - how $p$ changes with respect to $t$
$\newcommand{\dpdt}[]{ \frac{d {\bf p}}{d t}}$: $\dpdt$

* total change in reaction rate: \dv
$\newcommand{\dv}[] {d {\bf v} }$: $\dv$

* total change in state: \dx
$\newcommand{\dx}[] {d {\bf x} }$: $\dx$

* change in reaction rate w.r.t. $\bf{x}$: \dvdx -
$\newcommand{\dvdx}[]{ \frac{\partial {\bf v}}{\partial {\bf x}}}$ $\dvdx$

* change in reaction rate w.r.t. $\bf{p}$: \dvdp -
$\newcommand{\dvdp}[]{ \frac{\partial {\bf v}}
{\partial {\bf p}}}$ $\dvdp$

# Helper Functions

```python
def addSymbols(symbolStr):
  """
  Adds the list of symbols in the blank separated string.
  """
  symbols = symbolStr.split(" ")
  globalDct = globals()
  for idx, symbol in enumerate(symbols):
    globalDct[symbol] = sympy.Symbol(symbol)

# Tests
variables = ["aa", "bbb", "fruit"]
symbolStr = " ".join(variables)
addSymbols(symbolStr)
for variable in variables:
  assert(variable in globals())
  delStmt = "del %s" % variable
  exec(delStmt)
```

```python
def setSymbolValue(symbol, value):
  globalDct = globals()
  globalDct[symbol] = value
```

```python
def mkVector(name, numRow):
  # Create the solution vector. The resulting vector is in the global name space.
  symbols = ["%s_%d" % (name, n) for n in range(numRow)]
  symbolStr = " ".join(symbols)
  addSymbols(symbolStr)
  setSymbolValue(name, sympy.Matrix([ [s] for s in symbols]))

# Test
numRow = 5
mkVector("q", numRow)
assert(len(q) == 5)
assert("q" in globals().keys())
del q
```

# General Approach

Find ${\bf x}(t)$ such that
${\bf x}^{\prime}(t) = {\bf A} {\bf x}(t) + {\bf r}(t)$ for ${\bf A}$ a matrix of constants
and ${\bf r}(t)$ a vector of time-varying terms.

Approach
1. ${\bf x}(t) = {\bf x}_c (t) + {\bf x}_p (t)$, where $c$ indicates the complementary solution for the homogeneous system and $p$ indicates a particular solution for the inhomogeneous system.

## Homogeneous Solution (Complementary Equation)

1. The homogeneous equation is ${\bf x}^{\prime} (t) = {\bf A}{\bf x}(t)$.
1. Trial solution for homogeneous system ${\bf x}_c(t) = {\bf v} e^{\lambda t}$, where ${\bf v}$ is a vector of constants.
1. Substituting the trial solution $\lambda {\bf v} e^{\lambda t} = {\bf A}{\bf v} e^{\lambda t} \equiv (A - {\bf I} \lambda){\bf v} = {\bf 0}$. Thus, $\lambda$ is an eigenvalue of ${\bf A}$ and ${\bf v}$ is is eigenvector.
1. So, a solution to the homogeneous equation the homogeneous equation has the form
$\sum_n c_n {\bf v}_n e^{\lambda_n t}$,
where $\lambda_n, {\bf v}_n$ are eigenvalue, eigenvector combinations.
1. We define ${\bf X}_c (t) =
\begin{pmatrix}
{\bf v}_1 e^{\lambda_1 t} & \cdots & {\bf v}_N e^{\lambda_N t}
\end{pmatrix}
$
1. Given ${\bf x}(t_0)$, we can find values for the $c_n$. Consider $t_0 = 0$.
Note that
   \begin{equation*}
    {\bf X}_c (0)
   \begin{pmatrix}
   c_1 \\
   \vdots \\
   c_N
   \end{pmatrix}
   = {\bf x} (0),
   \end{equation*}
   which is a system of linear equations in $c_n$.

## Particular Solution

As is standard practice, we solve for the particular solution.
Let $u_n(t)$ be a function associated with the $n$-th variable such that

$
\begin{align*}
{\bf x}_p & = &
{\bf X}_c (t)
\begin{pmatrix}
u_1 (t) \\
\vdots \\
u_N (t)
\end{pmatrix} \\
& = & {\bf X}_c  (t){\bf u} (t)
\end{align*}.
$

Because ${\bf x}_p$ is a solution,
$
\begin{align*}
{\bf x}_p^{\prime}
=
{\bf A} {\bf x}_p
+ {\bf r}(t) .
\end{align*}
$
Substituting,
$
\begin{align*}
\left(
  {\bf X}_c (t) {\bf u}(t)
\right)^{\prime} & = &
A
  {\bf X}_c (t) {\bf u}(t)
+ {\bf r}(t) \\
{\bf X}^{\prime}_c (t) {\bf u}(t) + {\bf X}_c (t) {\bf u}^{\prime}(t)
 & = & A {\bf X}_c {\bf u}(t) + {\bf r} (t) \\
{\bf X}_c (t) {\bf u}^{\prime} (t) & = &  {\bf r} (t) \\
{\bf u}^{\prime} (t) & = & \left( {\bf X}_c (t) \right)^{-1} {\bf r}(t) \\
{\bf u}(t) & = & \int \left( {\bf X}_c (t) \right)^{-1} {\bf r}(t) .
\end{align*}
$

Some notes:
1. Note that any ${\bf u} (t)$ satisfying the derivative
equation is sufficient.
1. Are we guaranteed that ${\bf X}_c (t)$ is invertible?
Does it suffice that the $\{ {\bf v}_n \}$ are independent (since they are orthogonal eigenvectors if the eigenvalues
are distinct)?

# Implementing the General Solution

## Complementary solution

```python
def init():
  addSymbols("Amat k1 k2 k3 mat t X0 x y z")

init()
assert("Amat" in globals())
```

```python
addSymbols("Amat mat k1 k2 k3 t X_0")
```

```python
Amat = sympy.Matrix( [[0, 0, 0 ], [k1, -k2, 0], [0, k2, -k3]])
Amat
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0\\k_{1} & - k_{2} & 0\\0 & k_{2} & - k_{3}\end{matrix}\right]$

```python
def mkEigenMatrix(Amat, subs={}):
  """
  Creates a numpy matrix of eigenvalues using the substitutions indicated.

  Parameters
  ----------
  Amat: sympy.Matrix
  subs: dict
  """
  # Obtain the eigenvectors
  eigenVecs = Amat.eigenvects()
  eigenVecs = [v[2][0] for v in eigenVecs]
  for idx, symbol in enumerate([k1, k2, k3]):
    eigenVecs = [v.subs(symbol, idx + 1) for v in eigenVecs]
  vec = np.array(eigenVecs)
  vec = [float(v) for v in vec.flatten()]
  vec = np.reshape(vec, (3,3) )

```

```python
eigenVecs = Amat.eigenvects()
eigenVecs = [v[2][0] for v in eigenVecs]
for idx, symbol in enumerate([k1, k2, k3]):
  eigenVecs = [v.subs(symbol, idx + 1) for v in eigenVecs]
vec = np.array(eigenVecs)
vec = [float(v) for v in vec.flatten()]
vec = np.reshape(vec, (3,3) )
```

```python
mat = Amat.copy()
for idx, symbol in enumerate([k1, k2, k3]):
  mat = mat.subs(symbol, idx + 1)
mat

```

$\displaystyle \left[\begin{matrix}0 & 0 & 0\\1 & -2 & 0\\0 & 2 & -3\end{matrix}\right]$

```python
M = sympy.Matrix(((k1, 1, 1, 1), (1, 1, 2, 3)))

system = A, b = M[:, :-1], M[:, -1]
# b = sympy.Matrix([x, y, z])

#sympy.linsolve(system, *b)
```

```python
def solveLTI(AMat, initialArr, rArr=None):
  """
  Symbolically solves the linear time invariant system: x'(t) = A*x(t) + r(t)
  Note: the symbol t must be defined in globals()

  Parameters
  ----------
  AMat: sympy.Matrix (N X N)
      A marix
  initialArr: sympy.Matrix (N X 1)
      Initial values
  rArr: sympy.Matrix (N X 1)
      r matrix in the differential equation

  Returns
  -------
  sympy.Matrix
  """
  X = "x"
  numRow = AMat.rows
  vectors = []
  if not "t" in globals():
    raise RuntimeError("Must define the symbol 't'.")
  results = AMat.eigenvects()
  for result in results:
    eigen = result[0]
    term = sympy.exp(eigen*t)
    vectors.append(result[2][0] * term)
  solutionMat = sympy.Matrix(vectors)
  solutionMat = solutionMat.reshape(numRow, numRow)
  solutionMat = solutionMat.transpose()
  # Find the coefficients for the eigenvectors
  smat = solutionMat.subs(t, 0) # evaluate at time 0
  # Solve the linear system
  system = smat, initialArr
  mkVector(X, numRow)
  result = sympy.linsolve(system, *x) # returns a finite set, not a vector
  coefVec = sympy.Matrix(result.args[0])
  fundamentalMat = solutionMat*sympy.diag(*coefVec)
  # Solution
  if rArr is not None:
    dCoefMat = fundamentalMat.inv()*rArr
    coefMat = sympy.integrate(dCoefMat, t)
    solutionMat = fundamentalMat * dCoefMat
  else:
    solutionMat = fundamentalMat * sympy.ones(numRow, cols=1)
  return solutionMat

# Tests
result = solveLTI(Amat, initialArr=[X_0, 0, 0])
result
```

$\displaystyle \left[\begin{matrix}X_{0}\\\frac{X_{0} k_{1} \left(-1 + \frac{k_{3}}{k_{2}}\right) e^{- k_{2} t}}{k_{2} - k_{3}} + \frac{X_{0} k_{1}}{k_{2}}\\- \frac{X_{0} k_{1} k_{2} e^{- k_{3} t}}{k_{2} k_{3} - k_{3}^{2}} + \frac{X_{0} k_{1} e^{- k_{2} t}}{k_{2} - k_{3}} + \frac{X_{0} k_{1}}{k_{3}}\end{matrix}\right]$

```python
vec = result.copy()
for subPair in [(k1, 1), (k2, 2), (k3, 3), (X_0, 1), (t, 0.5)]:
  vec = vec.subs(subPair[0], subPair[1])
vec.evalf()
```

$\displaystyle \left[\begin{matrix}1.0\\0.316060279414279\\0.114207332260844\end{matrix}\right]$

## Particular Solution

```python
def calcSolutionCoefs(fundamentalMat, rMat):
  """
  Calculates the coefficients for the solutions using variation of
  parameters. Assumes that integration is done w.r.t. "t", the time variable.

  Parameters
  ----------
  fundamentalMat: sympy.Matrix (N X N)
      fundamental matrix
  rMat: sympy.Matrix (N X 1)
      inhomogeneous term in the LTI system

  Returns
  -------
  coefMat: sympy.Mat (N X 1)
      coefficients to use for the terms in the fundamental matrix
  """
  dCoefMat = fundamentalMat.inv()*rMat
  coefMat = sympy.integrate(dCoefMat, t)
```

```python
def solveLTI(Amat, initialVals, stepVals):
  """
  Solves the linear, time invariant system of differential equations
  with the specified step inputs at time 0.

  Parameters
  ----------
  Amat: Matrix
      A matrix.
      Dimension: N X N, for N state variables
  initialVals: Vector
      Initial values of state variables.
      Dimension: N
  stepVals: Vector
      Step input to state variables at time 0

  Returns
  -------
  Vector of expressions
      Dimension: N
      Function of t
  """
```

# Numerical Studies

## Two Species Linear Network

$X_0 \xrightarrow{k_0} x_1 \xrightarrow{k_1} x_2 \xrightarrow{k_2} X_1$, for fixed species $X_0, X_1$

```python
model = """
J0: $X0 -> x1; k0*X0
J1: x1 -> x2; k1*x1
J2: x2 -> ; k2*x2

k0 = 1
k1 = 1
k2 = 2

X0 = 1
X1 = 3
x1 = 0
x2 = 0
"""

rr = te.loada(model)
rr.plot(rr.simulate())
```

```python
su.addSymbols("k0 k1 k2")
aMat = sympy.Matrix([ [   0,    0,    0],
                       [   k0,  -k1,   0],
                       [   0,    k1, -k2],
                   ])
initialVec = sympy.Matrix([1, 0, 0])
solver = LTISolver(aMat, initialVec)
solver.plot(0, 5, 100, subs={k0:1, k1: 1, k2: 2})
```

## FFL

$X \xrightarrow{k_0} x$, $x \xrightarrow{k_1} y \xrightarrow{k_2} z$,  $x \xrightarrow{k_3} z$, $z \xrightarrow{k_0}$

```python
model = """
J0: $x_0 -> x_1; k0*x_0
J1: x_1 -> x_2; k1*x_1
J2: x_2 -> x_3; k2*x_2
J3: x_1 -> x_3; k3*x_1
J4: x_3 ->  ; k0*x_3

k0 = 1
k1 = 1
k2 = 2
k3 = 3

x_0 = 1
x_1 = 0
x_2 = 0
x_3 = 0
"""

rr = te.loada(model)
rr.plot(rr.simulate(0, 5, 100))
```

```python
rr.x_2
```

```python
su.addSymbols("k0 k1 k2 k3 x_0")
aMat = sympy.Matrix([ [   0,           0,   0,  0],
                       [   k0,  -(k1 + k3),  0,  0],
                       [   0,          k1, -k2,  0],
                       [   0,          k3,  k2, -k0],
                       ])
initialVec = sympy.Matrix([x_0, 0, 0, 0])
solver = LTISolver(aMat, initialVec)
solver.solve()
subs={k0:1, k1: 1, k2: 2, k3: 3, x_0: 1}
solver.plot(0, 5, 100, subs=subs)
```

```python
solver.fundamentalMat
```

$\displaystyle \left[\begin{matrix}1.0 & 0 & 0 & 0\\\frac{k_{0}}{k_{1} + k_{3}} & 0 & 0 & \frac{\left(- k_{0} k_{1} + k_{0} k_{2} - k_{0} k_{3} + k_{1}^{2} - k_{1} k_{2} + 2.0 k_{1} k_{3} - k_{2} k_{3} + k_{3}^{2}\right) e^{- t \left(k_{1} + k_{3}\right)}}{k_{1} k_{2} - k_{1} k_{3} + k_{2} k_{3} - k_{3}^{2}}\\\frac{k_{0} k_{1}}{k_{2} \left(k_{1} + k_{3}\right)} & 0 & \frac{\left(k_{0} - k_{2}\right) e^{- k_{2} t}}{k_{2}} & \frac{k_{1} \left(k_{0} - k_{1} - k_{3}\right) e^{- t \left(k_{1} + k_{3}\right)}}{k_{1} k_{2} - k_{1} k_{3} + k_{2} k_{3} - k_{3}^{2}}\\1.0 & 1.0 e^{- k_{0} t} & 1.0 e^{- k_{2} t} & 1.0 e^{- t \left(k_{1} + k_{3}\right)}\end{matrix}\right]$

```python
# Squared error difference between predicated and actual values
v = su.evaluate(solver.fundamentalMat[:, 0], subs=subs) - sympy.Matrix([rr.x_0, rr.x_1, rr.x_2, rr.x_3])
v.dot(v)
```

```python
solver.coefHomogeneousVec
```

$\displaystyle \left[\begin{matrix}1.0 x_{0}\\\frac{1.0 k_{0} k_{3} x_{0} - 1.0 k_{1} k_{2} x_{0} - 1.0 k_{2} k_{3} x_{0}}{1.0 k_{0}^{2} - 1.0 k_{0} k_{1} - 1.0 k_{0} k_{2} - 1.0 k_{0} k_{3} + 1.0 k_{1} k_{2} + 1.0 k_{2} k_{3}}\\- \frac{1.0 k_{0} k_{1} x_{0}}{1.0 k_{0} k_{1} - 1.0 k_{0} k_{2} + 1.0 k_{0} k_{3} - 1.0 k_{1} k_{2} + 1.0 k_{2}^{2} - 1.0 k_{2} k_{3}}\\\frac{1.0 k_{0} k_{2} x_{0} - 1.0 k_{0} k_{3} x_{0}}{1.0 k_{0} k_{1} - 1.0 k_{0} k_{2} + 1.0 k_{0} k_{3} - 1.0 k_{1}^{2} + 1.0 k_{1} k_{2} - 2.0 k_{1} k_{3} + 1.0 k_{2} k_{3} - 1.0 k_{3}^{2}}\end{matrix}\right]$

```python
[e.val for e in solver.eigenCollection.eigenInfos]
```

    [0, -k0, -k2, -k1 - k3]

## Complex Eigenvalues

**Compare analysis with simulation.**
1. Evaluate non-complementary solutions.

notebooks/Solving Inhomogeneous LTI Differential Equations.ipynb