# Tutorial 1: Geometric view of data
**Week 1, Day 5: Dimensionality Reduction**

**By Neuromatch Academy**

__Content creators:__ Alex Cayco Gajic, John Murray

__Content reviewers:__ Roozbeh Farhoudi, Matt Krause, Spiros Chavlis, Richard Gao, Michael Waskom, Siddharth Suresh, Natalie Schaworonkow, Ella Batty

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

*Estimated timing of tutorial: 50 minutes*

In this notebook we'll explore how multivariate data can be represented in different orthonormal bases. This will help us build intuition that will be helpful in understanding PCA in the following tutorial.

Overview:
 - Generate correlated multivariate data.
 - Define an arbitrary orthonormal basis.
 - Project the data onto the new basis.

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/kaq2x/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

```python
# @title Video 1: Geometric view of data
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Af4y1R78w", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="THu9yHnpq9I", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

---
# Setup

```python
# Imports

import numpy as np
import matplotlib.pyplot as plt
```

```python
# @title Figure Settings
import ipywidgets as widgets  # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
# @title Plotting Functions

def plot_data(X):
  """
  Plots bivariate data. Includes a plot of each random variable, and a scatter
  plot of their joint activity. The title indicates the sample correlation
  calculated from the data.

  Args:
    X (numpy array of floats) :   Data matrix each column corresponds to a
                                  different random variable

  Returns:
    Nothing.
  """

  fig = plt.figure(figsize=[8, 4])
  gs = fig.add_gridspec(2, 2)
  ax1 = fig.add_subplot(gs[0, 0])
  ax1.plot(X[:, 0], color='k')
  plt.ylabel('Neuron 1')
  plt.title('Sample var 1: {:.1f}'.format(np.var(X[:, 0])))
  ax1.set_xticklabels([])
  ax2 = fig.add_subplot(gs[1, 0])
  ax2.plot(X[:, 1], color='k')
  plt.xlabel('Sample Number')
  plt.ylabel('Neuron 2')
  plt.title('Sample var 2: {:.1f}'.format(np.var(X[:, 1])))
  ax3 = fig.add_subplot(gs[:, 1])
  ax3.plot(X[:, 0], X[:, 1], '.', markerfacecolor=[.5, .5, .5],
           markeredgewidth=0)
  ax3.axis('equal')
  plt.xlabel('Neuron 1 activity')
  plt.ylabel('Neuron 2 activity')
  plt.title('Sample corr: {:.1f}'.format(np.corrcoef(X[:, 0], X[:, 1])[0, 1]))
  plt.show()

def plot_basis_vectors(X, W):
  """
  Plots bivariate data as well as new basis vectors.

  Args:
    X (numpy array of floats) :   Data matrix each column corresponds to a
                                  different random variable
    W (numpy array of floats) :   Square matrix representing new orthonormal
                                  basis each column represents a basis vector

  Returns:
    Nothing.
  """

  plt.figure(figsize=[4, 4])
  plt.plot(X[:, 0], X[:, 1], '.', color=[.5, .5, .5], label='Data')
  plt.axis('equal')
  plt.xlabel('Neuron 1 activity')
  plt.ylabel('Neuron 2 activity')
  plt.plot([0, W[0, 0]], [0, W[1, 0]], color='r', linewidth=3,
           label='Basis vector 1')
  plt.plot([0, W[0, 1]], [0, W[1, 1]], color='b', linewidth=3,
           label='Basis vector 2')
  plt.legend()
  plt.show()

def plot_data_new_basis(Y):
  """
  Plots bivariate data after transformation to new bases.
  Similar to plot_data but with colors corresponding to projections onto
  basis 1 (red) and basis 2 (blue). The title indicates the sample correlation
  calculated from the data.

  Note that samples are re-sorted in ascending order for the first
  random variable.

  Args:
    Y (numpy array of floats):   Data matrix in new basis each column
                                 corresponds to a different random variable

  Returns:
    Nothing.
  """
  fig = plt.figure(figsize=[8, 4])
  gs = fig.add_gridspec(2, 2)
  ax1 = fig.add_subplot(gs[0, 0])
  ax1.plot(Y[:, 0], 'r')
  plt.xlabel
  plt.ylabel('Projection \n basis vector 1')
  plt.title('Sample var 1: {:.1f}'.format(np.var(Y[:, 0])))
  ax1.set_xticklabels([])
  ax2 = fig.add_subplot(gs[1, 0])
  ax2.plot(Y[:, 1], 'b')
  plt.xlabel('Sample number')
  plt.ylabel('Projection \n basis vector 2')
  plt.title('Sample var 2: {:.1f}'.format(np.var(Y[:, 1])))
  ax3 = fig.add_subplot(gs[:, 1])
  ax3.plot(Y[:, 0], Y[:, 1], '.', color=[.5, .5, .5])
  ax3.axis('equal')
  plt.xlabel('Projection basis vector 1')
  plt.ylabel('Projection basis vector 2')
  plt.title('Sample corr: {:.1f}'.format(np.corrcoef(Y[:, 0], Y[:, 1])[0, 1]))
  plt.show()
```

---
# Section 1: Generate correlated multivariate data

```python
# @title Video 2: Multivariate data
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1xz4y1D7ES", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="jcTq2PgU5Vw", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video describes the covariance matrix and the multivariate normal distribution.

 Click here for text recap of video

To gain intuition, we will first use a simple model to generate multivariate data. Specifically, we will draw random samples from a *bivariate normal distribution*. This is an extension of the one-dimensional normal distribution to two dimensions, in which each $x_i$ is marginally normal with mean $\mu_i$ and variance $\sigma_i^2$:

\begin{align}
x_i \sim \mathcal{N}(\mu_i,\sigma_i^2).
\end{align}

Additionally, the joint distribution for $x_1$ and $x_2$ has a specified correlation coefficient $\rho$. Recall that the correlation coefficient is a normalized version of the covariance, and ranges between -1 and +1:

\begin{align}
\rho = \frac{\text{cov}(x_1,x_2)}{\sqrt{\sigma_1^2 \sigma_2^2}}.
\end{align}

For simplicity, we will assume that the mean of each variable has already been subtracted, so that $\mu_i=0$ for both $i=1$ and $i=2$. The remaining parameters can be summarized in the covariance matrix, which for two dimensions has the following form:

\begin{align}
{\bf \Sigma} =
\begin{pmatrix}
 \text{var}(x_1) & \text{cov}(x_1,x_2) \\
 \text{cov}(x_1,x_2) &\text{var}(x_2)
\end{pmatrix}.
\end{align}

In general, $\bf \Sigma$ is a symmetric matrix with the variances $\text{var}(x_i) = \sigma_i^2$ on the diagonal, and the covariances on the off-diagonal. Later, we will see that the covariance matrix plays a key role in PCA.

## Coding Exercise 1: Draw samples from a distribution

We have provided code to draw random samples from a zero-mean bivariate normal distribution with a specified covariance matrix (`get_data`). Throughout this tutorial, we'll imagine these samples represent the activity (firing rates) of two recorded neurons on different trials. Fill in the function below to calculate the covariance matrix given the desired variances and correlation coefficient. The covariance can be found by rearranging the equation above:

\begin{align}
\text{cov}(x_1,x_2) = \rho \sqrt{\sigma_1^2 \sigma_2^2}.
\end{align}

```python
# @markdown Execute this cell to get helper function `get_data`

def get_data(cov_matrix):
  """
  Returns a matrix of 1000 samples from a bivariate, zero-mean Gaussian.

  Note that samples are sorted in ascending order for the first random variable

  Args:
    cov_matrix (numpy array of floats): desired covariance matrix

  Returns:
    (numpy array of floats) : samples from the bivariate Gaussian, with each
                              column corresponding to a different random
                              variable
  """

  mean = np.array([0, 0])
  X = np.random.multivariate_normal(mean, cov_matrix, size=1000)
  indices_for_sorting = np.argsort(X[:, 0])
  X = X[indices_for_sorting, :]

  return X

help(get_data)
```

    Help on function get_data in module __main__:

    get_data(cov_matrix)
        Returns a matrix of 1000 samples from a bivariate, zero-mean Gaussian.

        Note that samples are sorted in ascending order for the first random variable

        Args:
          cov_matrix (numpy array of floats): desired covariance matrix

        Returns:
          (numpy array of floats) : samples from the bivariate Gaussian, with each
                                    column corresponding to a different random
                                    variable

```python
def calculate_cov_matrix(var_1, var_2, corr_coef):
  """
  Calculates the covariance matrix based on the variances and correlation
  coefficient.

  Args:
    var_1 (scalar)          : variance of the first random variable
    var_2 (scalar)          : variance of the second random variable
    corr_coef (scalar)      : correlation coefficient

  Returns:
    (numpy array of floats) : covariance matrix
  """

  #################################################
  ## TODO for students: calculate the covariance matrix
  # Fill out function and remove
  #raise NotImplementedError("Student excercise: calculate the covariance matrix!")
  #################################################

  # Calculate the covariance from the variances and correlation
  cov = corr_coef * np.sqrt(var_1 * var_2)

  cov_matrix = np.array([[var_1, cov], [cov, var_2]])

  return cov_matrix

# Set parameters
np.random.seed(2020)  # set random seed
variance_1 = 1
variance_2 = 1
corr_coef = 0.8

# Compute covariance matrix
cov_matrix = calculate_cov_matrix(variance_1, variance_2, corr_coef)

# Generate data with this covariance matrix
X = get_data(cov_matrix)

# Visualize
plot_data(X)
```

```python
# to_remove solution

def calculate_cov_matrix(var_1, var_2, corr_coef):
  """
  Calculates the covariance matrix based on the variances and correlation
  coefficient.

  Args:
    var_1 (scalar)          : variance of the first random variable
    var_2 (scalar)          : variance of the second random variable
    corr_coef (scalar)      : correlation coefficient

  Returns:
    (numpy array of floats) : covariance matrix
  """

  # Calculate the covariance from the variances and correlation
  cov = corr_coef * np.sqrt(var_1 * var_2)

  cov_matrix = np.array([[var_1, cov], [cov, var_2]])

  return cov_matrix

# Set parameters
np.random.seed(2020)  # set random seed
variance_1 = 1
variance_2 = 1
corr_coef = 0.8

# Compute covariance matrix
cov_matrix = calculate_cov_matrix(variance_1, variance_2, corr_coef)

# Generate data with this covariance matrix
X = get_data(cov_matrix)

# Visualize
with plt.xkcd():
  plot_data(X)
```

## Interactive Demo 1: Correlation effect on data

We'll use the function you just completed but now we can change the correlation coefficient via slider. You should get a feel for how changing the correlation coefficient affects the geometry of the simulated data.

1.   What effect do negative correlation coefficient values have?
2.   What correlation coefficient results in a circular data cloud?

Note that we sort the samples according to neuron 1's firing rate, meaning the plot of neuron 1 firing rate over sample number looks clean and pretty unchanging when compared to neuron 2.

```python
# @markdown Execute this cell to enable widget

def _calculate_cov_matrix(var_1, var_2, corr_coef):

  # Calculate the covariance from the variances and correlation
  cov = corr_coef * np.sqrt(var_1 * var_2)

  cov_matrix = np.array([[var_1, cov], [cov, var_2]])

  return cov_matrix

@widgets.interact(corr_coef = widgets.FloatSlider(value=.2, min=-1, max=1, step=0.1))
def visualize_correlated_data(corr_coef=0):
  variance_1 = 1
  variance_2 = 1

  # Compute covariance matrix
  cov_matrix = _calculate_cov_matrix(variance_1, variance_2, corr_coef)

  # Generate data with this covariance matrix
  X = get_data(cov_matrix)

  # Visualize
  plot_data(X)
```

    interactive(children=(FloatSlider(value=0.2, description='corr_coef', max=1.0, min=-1.0), Output()), _dom_clas…

```python
# to_remove explanation

"""

1) Negative correlation coefficients reverse the direction of the relationship: now higher neuron 1
activities are associated with lower neuron 2 activities.

2) A correlation coefficient of 0 (no correlation) results in a circular looking data cloud.

"""
```

    '\n\n1) Negative correlation coefficients reverse the direction of the relationship: now higher neuron 1\nactivities are associated with lower neuron 2 activities.\n\n2) A correlation coefficient of 0 (no correlation) results in a circular looking data cloud.\n\n'

---
# Section 2: Define a new orthonormal basis

*Estimated timing to here from start of tutorial: 20 min*

```python
# @title Video 3: Orthonormal bases
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1wT4y1E71g", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="PC1RZELnrIg", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video shows that data can be represented in many ways using different bases. It also explains how to check if your favorite basis is orthonormal.

 Click here for text recap of video

Next, we will define a new orthonormal basis of vectors ${\bf u} = [u_1,u_2]$ and ${\bf w} = [w_1,w_2]$. As we learned in the video, two vectors are orthonormal if:

1.   They are orthogonal (i.e., their dot product is zero):
\begin{align}
{\bf u\cdot w} = u_1 w_1 + u_2 w_2 = 0
\end{align}
2.   They have unit length:
\begin{align}
||{\bf u} || = ||{\bf w} || = 1
\end{align}

In two dimensions, it is easy to make an arbitrary orthonormal basis. All we need is a random vector ${\bf u}$, which we have normalized. If we now define the second basis vector to be ${\bf w} = [-u_2,u_1]$, we can check that both conditions are satisfied:
\begin{align}
{\bf u\cdot w} = - u_1 u_2 + u_2 u_1 = 0
\end{align}
and
\begin{align}
{|| {\bf w} ||} = \sqrt{(-u_2)^2 + u_1^2} = \sqrt{u_1^2 + u_2^2} = 1,
\end{align}
where we used the fact that ${\bf u}$ is normalized. So, with an arbitrary input vector, we can define an orthonormal basis, which we will write in matrix by stacking the basis vectors horizontally:

\begin{align}
{{\bf W} } =
\begin{pmatrix}
 u_1 & w_1 \\
 u_2 & w_2
\end{pmatrix}.
\end{align}

## Coding Exercise 2: Find an orthonormal basis

In this exercise you will fill in the function below to define an orthonormal basis, given a single arbitrary 2-dimensional vector as an input.

**Steps**
* Modify the function `define_orthonormal_basis` to first normalize the first basis vector $\bf u$.
* Then complete the function by finding a basis vector $\bf w$ that is orthogonal to $\bf u$.
* Test the function using initial basis vector ${\bf u} = [3,1]$. Plot the resulting basis vectors on top of the data scatter plot using the function `plot_basis_vectors`. (For the data, use  $\sigma_1^2 =1$, $\sigma_2^2 =1$, and $\rho = .8$).

```python
def define_orthonormal_basis(u):
  """
  Calculates an orthonormal basis given an arbitrary vector u.

  Args:
    u (numpy array of floats) : arbitrary 2-dimensional vector used for new
                                basis

  Returns:
    (numpy array of floats)   : new orthonormal basis
                                columns correspond to basis vectors
  """

  #################################################
  ## TODO for students: calculate the orthonormal basis
  # Fill out function and remove
  #raise NotImplementedError("Student excercise: implement the orthonormal basis function")
  #################################################

  # Normalize vector u
  u = u / np.linalg.norm(u)

  # Calculate vector w that is orthogonal to w
  w = np.array([-u[1], u[0]])

  # Put in matrix form
  W = np.column_stack([u, w])

  return W

# Set up parameters
np.random.seed(2020)  # set random seed
variance_1 = 1
variance_2 = 1
corr_coef = 0.8
u = np.array([3, 1])

# Compute covariance matrix
cov_matrix = calculate_cov_matrix(variance_1, variance_2, corr_coef)

# Generate data
X = get_data(cov_matrix)

# Get orthonomal basis
W = define_orthonormal_basis(u)

# Visualize
plot_basis_vectors(X, W)
```

```python
# to_remove solution

def define_orthonormal_basis(u):
  """
  Calculates an orthonormal basis given an arbitrary vector u.

  Args:
    u (numpy array of floats) : arbitrary 2-dimensional vector used for new
                                basis

  Returns:
    (numpy array of floats)   : new orthonormal basis
                                columns correspond to basis vectors
  """

  # Normalize vector u
  u = u / np.sqrt(u[0] ** 2 + u[1] ** 2)

  # Calculate vector w that is orthogonal to w
  w = np.array([-u[1], u[0]])

  # Put in matrix form
  W = np.column_stack([u, w])

  return W

# Set up parameters
np.random.seed(2020)  # set random seed
variance_1 = 1
variance_2 = 1
corr_coef = 0.8
u = np.array([3, 1])

# Compute covariance matrix
cov_matrix = calculate_cov_matrix(variance_1, variance_2, corr_coef)

# Generate data
X = get_data(cov_matrix)

# Get orthonomal basis
W = define_orthonormal_basis(u)

# Visualize
with plt.xkcd():
  plot_basis_vectors(X, W)
```

---
# Section 3: Project data onto new basis

*Estimated timing to here from start of tutorial: 35 min*

```python
# @title Video 4: Change of basis
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1LK411J7NQ", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="Mj6BRQPKKUc", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Finally, we will express our data in the new basis that we have just found. Since $\bf W$ is orthonormal, we can project the data into our new basis using simple matrix multiplication :

\begin{align}
{\bf Y = X W}.
\end{align}

We will explore the geometry of the transformed data $\bf Y$ as we vary the choice of basis.

## Coding Exercise 3: Change to orthonormal basis
In this exercise you will fill in the function below to change data to a orthonormal basis.

**Steps**
* Complete the function `change_of_basis` to project the data onto the new basis.
* Plot the projected data using the function `plot_data_new_basis`.
* What happens to the correlation coefficient in the new basis? Does it increase or decrease?
* What happens to variance?

```python
def change_of_basis(X, W):
  """
  Projects data onto new basis W.

  Args:
    X (numpy array of floats) : Data matrix each column corresponding to a
                                different random variable
    W (numpy array of floats) : new orthonormal basis columns correspond to
                                basis vectors

  Returns:
    (numpy array of floats)    : Data matrix expressed in new basis
  """

  #################################################
  ## TODO for students: project the data onto o new basis W
  # Fill out function and remove
  #raise NotImplementedError("Student excercise: implement change of basis")
  #################################################

  # Project data onto new basis described by W
  Y = X @ W

  return Y

# Project data to new basis
Y = change_of_basis(X, W)

# Visualize
plot_data_new_basis(Y)
```

```python
# to_remove solution

def change_of_basis(X, W):
  """
  Projects data onto new basis W.

  Args:
    X (numpy array of floats) : Data matrix each column corresponding to a
                                different random variable
    W (numpy array of floats) : new orthonormal basis columns correspond to
                                basis vectors

  Returns:
    (numpy array of floats)    : Data matrix expressed in new basis
  """

  # Project data onto new basis described by W
  Y = X @ W

  return Y

# Project data to new basis
Y = change_of_basis(X, W)

# Visualize
with plt.xkcd():
  plot_data_new_basis(Y)
```

## Interactive Demo 3: Play with the basis vectors
To see what happens to the correlation as we change the basis vectors, run the cell below. The parameter $\theta$ controls the angle of $\bf u$ in degrees. Use the slider to rotate the basis vectors.

1.   What happens to the projected data as you rotate the basis?
2.   How does the correlation coefficient change? How does the variance of the projection onto each basis vector change?
3.   Are you able to find a basis in which the projected data is **uncorrelated**?

```python
# @markdown Make sure you execute this cell to enable the widget!

def refresh(theta=0):
  u = [1, np.tan(theta * np.pi / 180)]
  W = define_orthonormal_basis(u)
  Y = change_of_basis(X, W)
  plot_basis_vectors(X, W)
  plot_data_new_basis(Y)

_ = widgets.interact(refresh, theta=(0, 90, 5))
```

    interactive(children=(IntSlider(value=0, description='theta', max=90, step=5), Output()), _dom_classes=('widge…

```python
# to_remove explanation

"""

1) As you rotate the basis vectors, the look of the cloud of projected data changes.
Specifically, the correlations and variances of the projected data change.

2) As the angle increases, the correlation coefficient decreases, goes to 0, then starts
to become negative. It changes from 0.8 to -.8 when the angle is changed by 90 degrees.

With theta of 0, the data projected onto the two basis vectors has equal variances of 1.
As theta increases, these variances become unequal: the projected data has larger and larger variance
for basis vector 1 and smaller variance for basis vector 2. Past theta of 45, the trend reverses and
the variances start becoming more similar.

3) If theta is 45 degrees, the projected data is uncorrelated.

"""
```

---
# Summary

*Estimated timing of tutorial: 50 minutes*

- In this tutorial, we learned that multivariate data can be visualized as a cloud of points in a high-dimensional vector space. The geometry of this cloud is shaped by the covariance matrix.

- Multivariate data can be represented in a new orthonormal basis using the dot product. These new basis vectors correspond to specific mixtures of the original variables - for example, in neuroscience, they could represent different ratios of activation  across a population of neurons.

- The projected data (after transforming into the new basis) will generally have a different geometry from the original data. In particular, taking basis vectors that are aligned with the spread of cloud of points decorrelates the data.

* These concepts - covariance, projections, and orthonormal bases - are key for understanding PCA, which we be our focus in the next tutorial.

---
# Notation

\begin{align}
x_i &\quad \text{data point for dimension } i\\
\mu_i &\quad \text{mean along dimension } i\\
\sigma_i^2 &\quad \text{variance along dimension } i \\
\bf u, \bf w &\quad \text{orthonormal basis vectors}\\
\rho &\quad \text{correlation coefficient}\\
\bf \Sigma &\quad \text{covariance matrix}\\
\bf X &\quad \text{original data matrix}\\
\bf W &\quad \text{projection matrix}\\
\bf Y &\quad \text{transformed data}\\
\end{align}

```python

```


Universidade Federal do Rio Grande do Sul (UFRGS)
Programa de Pós-Graduação em Engenharia Civil (PPGEC)

# PEC00144: Experimental Methods in Civil Engineering

### Part I: Analysis
[2.   Dimensional analysis and similarity](#section_2)

   [2.1. Quantities, units, dimensions](#section_21)
   [2.2. Dataframe for scales computation](#section_22)
   [2.3. The Theorem of Pi's by Vaschy-Buckinghan](#section_23)
   [2.4. Change of base units](#section_24)
   [2.5. Change of units is like change of scale](#section_25)
   [2.6. Famous Pi-Numbers](#section_26)
   [2.7. Assignment](#section_27)

---
_Prof. Marcelo M. Rocha, Dr.techn._ [(ORCID)](https://orcid.org/0000-0001-5640-1020)
_Porto Alegre, RS, Brazil_

```python
# Importing Python modules required for this notebook
# (this cell must be executed with "shift+enter" before any other Python cell)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

```

## 2. Dimensional analysis and similarity

### 2.1. Quantities, units, dimensions

Prof. G.I. Barenblatt has written in his book _Dimensional Analysis_ (1987):

>_In fact, the idea on which dimensional analysis is based is very simple, and can be
understood by all: physical laws do not depend on arbitrariness in the choice of basic
units of measurement. An important conclusion can be drawn from this simple idea
using simple arguments: the functions that express physical laws must possess a
certain fundamental property, which in mathematics is called generalized homogeneity
or symmetry. This property allows the number of arguments in these functions to be
reduced, thereby making it simpler to obtain them (by calculating them or determining
them experimentally). This is, in fact, the entire content of dimensional analysis - there
is nothing more to it._

In other words, the units resulting from both sides of an equation must be the same.
Although simple, this idea has allowed great scientists like Newton, Rayleigh, Maxwell,
Reynolds, Kolmogorov, Cauchy, and many others, to proposed many fundamental
physical principles.

To illustrate how the law of _dimensional homogeneity_ can be applied, let us
investigate the free vibration frequency, $f_{\rm n}$, of an undamped mass-spring
system with one degree of freedom.

The frequency $f_{\rm n}$ could be derived from mechanical principles like dynamic
equilibrium equations, but we take the alternative dimensional path instead.
The system above _may_ involve several physical quantities like:

* the instantaneous position, $u$,
* the oscillating mass, $m$,
* the time, $t$,
* the displacement amplitude, $u_0$,
* the spring linear stiffness, $k$, and also
* the free vibration frequency, $f_{\rm n}$.

The completeness of this set of quantities is essential for the success of our
dimensional analysis. The analysis process requires an investigation of possible
relationships among the _candidate_ quantities above and the frequency $f_{\rm n}$.
Let us take a look at their _units of measurement_:

\begin{align*}
[ u ]        &= [L]^{1}[M]^{0}[T]^{0}  \\
[ m ]        &= [L]^{0}[M]^{1}[T]^{0}  \\
[ t ]        &= [L]^{0}[M]^{0}[T]^{1}  \\
[ u_0 ]      &= [L]^{1}[M]^{0}[T]^{0}  \\
[ k ]        &= [L]^{0}[M]^{1}[T]^{-2} \\
[ f_{\rm n}] &= [L]^{0}[M]^{0}[T]^{-1}
\end{align*}

where the brackets mean "units of". One can observe that all measurement units have been
written in terms of only three _fundamental units_: length, $[L]$, mass, $[M]$, and time, $[T]$.
Most principles in solid mechanics and structural dynamics may be expressed with these
three units only. Eventually the temperature, $\Theta$, may be also required.
For the sake of simplicity, hereafter brackets will be ommited.

The quantities directly subordinated to the fundamental units are called _fundamental
quantities_, which are the length, the mass and the time, while other units and quantities
are called _derived_ units and quantities, like stiffness, frequency, acceleration, etc.
The dependency of derived units on the fundamental units are expressed through _dimensional
formulae_, as shown above, which are always _power-law monomials_. The exponents applyed to the
fundamental units in these formulae are called _dimensions_ or _dimension exponents_.

In the case of using the _International System_ (SI), the three fundamental units are the
meter (m), the kilogram (kg), and the second (s). In the example above this means that:

\begin{align*}
[ u ]        &= {\rm m}^{1}{\rm kg}^{0}{\rm s}^{0} = {\rm m}  \\
[ m ]        &= {\rm m}^{0}{\rm kg}^{1}{\rm s}^{0} = {\rm kg} \\
[ t ]        &= {\rm m}^{0}{\rm kg}^{0}{\rm s}^{1} = {\rm s}  \\
[ u_0 ]      &= {\rm m}^{1}{\rm kg}^{0}{\rm s}^{0} = {\rm m}  \\
[ k ]        &= {\rm m}^{0}{\rm kg}^{1}{\rm s}^{-2} = {\rm kg}/{\rm s}^{2} = {\rm N/m} \\
[ f_{\rm n}] &= {\rm m}^{0}{\rm kg}^{0}{\rm s}^{-1} = 1/{\rm s} = {\rm Hz}
\end{align*}

### 2.2. Dataframe for scales computation

To facilitate the scales calculations as presented in this _notebook_, let us now import
a ``pandas`` dataframe with physical quantities commonly used in engineering mechanics and
structural dynamics, aerodynamics, and fluid and soil mechanics:

```python
# Please Google "pandas read_excel()"

DimData = pd.read_excel('resources/DimData.xlsx',
                         header     =  0,
                         index_col  =  0,
                         sheet_name = 'DimData')

print(DimData)

```

                           descriptor   latex  L  M  T
    a                    Acceleration       a  1  0 -2
    α            Angular acceleration  \alpha  0  0 -2
    ω               Angular frequency  \omega  0  0 -1
    A                            Area       A  2  0  0
    EA                Axial stiffness      EA  1  1 -2
    EI         Beam bending stiffness      EI  3  1 -2
    GAs          Beam shear stiffness    GA_s  1  1 -2
    ρ                         Density    \rho -3  1  0
    μ               Dynamic viscosity     \mu -1  1 -1
    F                           Force       F  1  1 -2
    q           Force per unit length       q  0  1 -2
    f                       frequency       f  0  0 -1
    υ             Kinematic viscosity     \nu  2  0 -1
    L                          Length       L  1  0  0
    m                            Mass       m  0  1  0
    im   Mass inertia per unit length     i_m  1  1  0
    μA             Mass per unit area   \mu_A -2  1  0
    μL           Mass per unit length   \mu_L -1  1  0
    M                          Moment       M  2  1 -2
    I               Moment of inertia       I  4  0  0
    W               Resistent modulus       W  3  0  0
    Im        Rotational mass inertia     I_m  2  1  0
    k                Spring stiffness       k  0  1 -2
    σ                          Stress  \sigma -1  1 -2
    t                            Time       t  0  0  1
    v                        Velocity       v  1  0 -1
    c                 Viscous damping       c  0  1 -1
    V                          Volume       V  3  0  0

Other quantities may be easily included in this dataframe by editing the Excel file
``DimData.xlsx``. Each entry in the table corresponds to a dimensional quantity, being the last
three columns the respective dimension exponents in the dimensional formulae.
Quantities like angles, strains, Poison's coefficient and ratio of critical damping are
by definition non-dimensional, so there is no need of including them in this table.
Relations between length measures, the so-called _form factors_, must also not be included.

Observe that, for the sake of clearness, we are using Unicode
variable names, like greek letters, which is a remarkable Python's feature.

To access the $LMT$ exponents of a set of quantities, let us say the ones selected for
the oscillator problem above, the dataframe ```loc``` view may be used:

```python
par    = ['L', 'm', 't', 'L', 'k', 'f']   # selected dimensional quantities
LMT    = ['L', 'M', 'T']                  # three last columns of dataframe

DimMat = DimData.loc[par, LMT]            # the dimensional matrix

print(DimMat)
```

       L  M  T
    L  1  0  0
    m  0  1  0
    t  0  0  1
    L  1  0  0
    k  0  1 -2
    f  0  0 -1

According to ``pandas`` module, a subset of a dataframe is itself also a dataframe.
If only the dimension exponents are to be retrieved, the attribute ``values`` must be called,
returning a ``numpy`` 2D array.
This array will be required in the following sections, where we show how the information provided in the dataframe  ``DimData`` can be used to carry out units and scales changes for designing reduced scale models.

```python
print(DimMat.values)
```

    [[ 1  0  0]
     [ 0  1  0]
     [ 0  0  1]
     [ 1  0  0]
     [ 0  1 -2]
     [ 0  0 -1]]

Observe that the dataframe indices are the codes to access a specific quantity,
while the following first column, which header is ``descriptor``, corresponds to a
description to help identification.
The dataframe also brings a column named ``latex`` with the $\LaTeX$ code for eventual automatic
plot labelling.

### 2.3. The Vaschy-Buckingham's $\Pi$-Theorem

The Vaschy-Buckingham's $\Pi$-Theorem may be expressed as follows:

>Let us assume that a given physical phenomenon involves $n$ _dimensional quantities_, denoted as $p_1, p_2, \dots, p_n$. For this phenomenon a mathematical model may be expressed as a general relation:

\begin{equation}
 f\left (  p_1, p_2, \dots, p_n \right ) = 0
\end{equation}

>This relation may be replaced by another one with $r = n - k$ _non-dimensional quantities_:

\begin{equation}
 F\left (  \Pi_1, \Pi_2, \dots, \Pi_r \right ) = 0
\end{equation}

where $k$ is the number of fundamental units required to compose the units of all dimensional quantities involved. The $\Pi$-numbers have the general form:

\begin{equation}
\Pi = p_1^{\alpha_1} \; p_2^{\alpha_2} \; \dots \; p_n^{\alpha_n}
\end{equation}

where the exponents $\vec{\alpha} = \left [ \alpha_1 \; \alpha_2 \; \dots \; \alpha_n \right ]$ are such that the condition of non-dimensionality must always be fulfilled.

Recalling that in the present context the only required fundamental units are $L$, $M$, and $T$,
then $k = 3$. A demonstration of this theorem can be found in the literatura and will not be
replicated here.

To apply this theorem, let us assemble a _dimensional matrix_, $\bf D$, with one column for
each fundamental unit and one row for each quantity involved in the analysis. The
dataframe ``DimMat`` defined in the previous section is an computational example of a
dimensional matrix.  To build up $\Pi$-numbers we propose now some vector, $\vec{\alpha}$,
such that (for $k = 3$):

\begin{equation}
\vec{\alpha}\,{\bf D} = \left[0 \hspace{5mm} 0 \hspace{5mm} 0 \right]
\end{equation}

There is no general rule for specifying $\vec{\alpha}$.
The practical principle that guides any proposition is some possible physical meaning.
If the dimensional analysis subject is not well known, relevant $\Pi$-numbers will
always result from some educated guess.
Even the choice of a relevant set of quantities is not always obvious, for it may
both exceed or lack completeness.
At this point, one can clearly see that _dimensional analysis is not a panacea for
model proposition_ and a good measure of experience, or even luck, may be required.

Going back to the oscillator problem, possible $\Pi$-numbers for the involved quantities are:

\begin{align*}
 \Pi_1 &= u^1 u_0^{-1} \\
 \Pi_2 &= t^1 \; f_{\rm n}^{1} \\
 \Pi_3 &= m^{-1/2} \; k^{1/2} \; f_{\rm n}^{-1} = \frac{\sqrt{k/m}}{f_{\rm n}}
\end{align*}

where we can se that $\Pi_3$ is clearly the most interesting, with

$$\vec{\alpha}_3 = \left [ 0 \;\; {-1/2} \;\;\; 0 \;\;\;  0 \;\; 1/2  \;\; {-1} \right]$$

The product $\vec{\alpha}_3{\bf D}$  evidentiates its non-dimensionality:

```python
par =          ['L',  'm', 't', 'L', 'k', 'f']   # selected dimensional quantities
α3  = np.array([ 0,  -1/2,  0,   0,  1/2, -1 ])

print(np.matmul(α3, DimMat.values))
```

    [0. 0. 0.]

Representing physical phenomena as relationships among non-dimensional quantities
certainly implies independence of the chosen units system, but it does not ensure
independence of length scale, which is the holy grail of any mathematical model.
On the other hand, models may eventually include the length scale dependence itself
whenever enough data is available.

For the oscillator example above, it can be shown both experimentally or from physical
principles, that $\Pi_3 \approx 2\pi$ for small $u_0$. This constant results is, within some
limits, also (length) scale independent.
The whole point here is that this result _can be experimentally obtained through dimensional
analysis of experimental outcomes even if the applicable principles of physics were
not known in advance_.
This justifies the importance of dimensional analsys, for it has allowed great cientists
to propose models with $\Pi$-numbers governing many important phenomena. Some remarkable
examples will be shown in section 2.6.

### 2.4. Change of base units

Dimensional formulae allow us to calculate derived units after a change of the base units.
One can, for instance, change from S.I. to British unit system and use the dimensional
formulae to calculate the unit equivalence of any required physical quantity. In this
section it is shown that the same calculation for changing the system of units can also
be used for changing scales and designing experimental reduced models.

We recall that the choice of $L$, $M$, and $T$ as fundamental base units, although now
internationally acknowledged, is in some way arbitrary. However, it is always possible
to replace these three quantities and calculate a new dimensional matrix for other
three quantities according to our convenience.

Assuming that the base units $L$, $M$, and $T$ ($LMT$ for short) are to be replaced by
a new set of units $A$, $B$ e $C$ ($ABC$ for short). The dimensional matrix for
this new base can always be written as:

\begin{align*}
 A &= L^{a_{L}} M^{a_{M}} T^{a_{T}} \\
 B &= L^{b_{L}} M^{b_{M}} T^{b_{T}} \\
 C &= L^{c_{L}} M^{c_{M}} T^{c_{T}}
\end{align*}

As an example, let us propose a new base by selecting three quantities from the dataframe
```DimData```. For instance, let us take $A = L$ (length),  $B = \rho$ (density), and $C = a$ (acceleration):

```python
ABC  = ['L', 'ρ', 'a']
base = DimData.loc[ABC, LMT]

print(base)
```

       L  M  T
    L  1  0  0
    ρ -3  1  0
    a  1  0 -2

The new base _must include quantities with linearly independent dimensions_, only!
It is not possible, for instance, to choose length, area and volume, for their
dimensions are multiples one of each other.

Now we take logarithms from both sides of the equation above:

\begin{align*}
\log A &= \log \left ( L^{a_{L}} M^{a_{M}} T^{a_{T}} \right ) = a_{L}\log L + a_{M}\log M + a_{T}\log T \\
\log B &= \log \left ( L^{b_{L}} M^{b_{M}} T^{b_{T}} \right ) = b_{L}\log L + b_{M}\log M + b_{T}\log T \\
\log C &= \log \left ( L^{c_{L}} M^{c_{M}} T^{c_{T}} \right ) = c_{L}\log L + c_{M}\log M + c_{T}\log T
\end{align*}

In matrix notation:

$$\left [ \begin{array}{ccc}
           a_{L}& a_{M}& a_{T} \\
           b_{L}& b_{M}& b_{T} \\
           c_{L}& c_{M}& c_{T}
          \end{array} \right ]
  \left [ \begin{array}{ccc}
          \log L \\
          \log M \\
          \log T
          \end{array} \right ] =
  \left [ \begin{array}{ccc}
          \log A \\
          \log B \\
          \log C
          \end{array} \right ] $$

By isolating the vector with the fundamental base it gives:

$$\left [ \begin{array}{ccc}
          \log L \\
          \log M \\
          \log T
          \end{array} \right ] =
  \left [ \begin{array}{ccc}
           a_{A}& a_{B}& a_{C} \\
           b_{A}& b_{B}& b_{C} \\
           c_{A}& c_{B}& c_{C}
          \end{array} \right ]
  \left [ \begin{array}{ccc}
          \log A \\
          \log B \\
          \log C
          \end{array} \right ] $$

where:

$$\left [ \begin{array}{ccc}
        a_{A}& a_{B}& a_{C} \\
        b_{A}& b_{B}& b_{C} \\
        c_{A}& c_{B}& c_{C}
          \end{array} \right ] =
  \left [ \begin{array}{ccc}
          a_{L}& a_{M}& a_{T} \\
          b_{L}& b_{M}& b_{T} \\
          c_{L}& c_{M}& c_{T}
          \end{array} \right ] ^{-1} $$

This is the dimension matrix of base $LMT$ according to the new base $ABC$.
Now it becomes clear why the quantities in the new base must have linearly independent
dimensions, otherwise the matrix inversion is not possible.
Going back to the monomial form we get:

\begin{align*}
 L &= A^{a_{A}} B^{a_{B}} C^{a_{C}} \\
 M &= A^{b_{A}} B^{b_{B}} C^{b_{C}} \\
 T &= A^{c_{A}} B^{c_{B}} C^{c_{C}}
\end{align*}

The inversion of the dimension matrix is carried out in Python with the ```numpy```
module as follows:

```python
i_base = np.linalg.inv(base)

i_base
```

    array([[ 1. , -0. , -0. ],
           [ 3. ,  1. ,  0. ],
           [ 0.5, -0. , -0.5]])

Now we take a look in what happens with the dimensions of any derived quantity, $X$,
after the base change. The dimensional formula using the fundamental base is

$$ X = L^{\alpha_{L}} M^{\beta_{M}} T^{\gamma_{T}} $$

Replacing the previous result yields:

$$ X = \left ( A^{a_{A}} B^{a_{B}} C^{a_{C}} \right )^{\alpha_{L}}
       \left ( A^{b_{A}} B^{b_{B}} C^{b_{C}} \right )^{\beta_{M}}
       \left ( A^{c_{A}} B^{c_{B}} C^{c_{C}} \right )^{\gamma_{T}} $$

Rearranging terms:

$$ X = A^{(a_{A}\alpha_{L} + b_{A}\beta_{M} + c_{A}\gamma_{T})}
       B^{(a_{B}\alpha_{L} + b_{B}\beta_{M} + c_{B}\gamma_{T})}
       C^{(a_{C}\alpha_{L} + b_{C}\beta_{M} + c_{C}\gamma_{T})}
     = A^{\alpha_{A}} B^{\beta_{B}} C^{\gamma_{C}} $$

Finally, the dimensions of $X$ in the new base $ABC$ can be calculated as:

$$\left [ \begin{array}{ccc}
          \alpha_{A} \\
          \beta_{B} \\
          \gamma_{C}
          \end{array} \right ]  =
  \left [ \begin{array}{ccc}
           a_{A}& a_{B}& a_{C} \\
           b_{A}& b_{B}& b_{C} \\
           c_{A}& c_{B}& c_{C}
          \end{array} \right ] ^{T}
  \left [ \begin{array}{ccc}
          \alpha_{L} \\
          \beta_{M} \\
          \gamma_{T}
          \end{array} \right ]  $$

This matrix multiplication can be carried out with ``numpy``. We keep the computational
strategy previously adopted and use the result to create a new ``pandas`` dataframe
from scratch:

```python
NewMat = pd.DataFrame(data    = np.matmul(DimMat.values, i_base),
                      index   = DimMat.index,
                      columns = ABC)
print(NewMat)
```

         L    ρ    a
    L  1.0  0.0  0.0
    m  3.0  1.0  0.0
    t  0.5  0.0 -0.5
    L  1.0  0.0  0.0
    k  2.0  1.0  1.0
    f -0.5  0.0  0.5

This new dimension matrix contains the dimensions of the selected quantities in the new base.
These dimensions can be used for changing the unit system in the much same way it can be used
for changing scales.

### 2.5. Changing units is like changing scales

Como já mencionado, uma das aplicações da mudança de base é a mudança de unidades das grandezas fundamentais. A nova matriz dimensional nos permite calcular o efeito desta mudança nas grandezas derivadas. Por exemplo, vamos admitir que desejamos utilizar a milha nautica ao invés do metro como unidade de comprimento, e a hora ao invés do segundo como unidade do tempo. Vejamos como isso repercute na unidade de velocidade, sendo que a fórmula de cálculo é a mesma que foi anteriormente utilizada para a dimensão:

\begin{equation}
 X = A^{\alpha_{A}} B^{\beta_{B}} C^{\gamma_{C}}
\end{equation}

For instance, let us take a change from meters per second (m/s) to knots (nautical miles per hour).
The units for the fundamental quantities are:

```python
λ_L    =  1852/1                   # 1852 meters is 1 nautical mile
λ_m    =  1/1                      # mass does not matter for velocity
λ_t    =  3600/1                   # 3600 seconds is 1 hour

scales =  np.tile([λ_L, λ_m, λ_t],(1,1))
print(scales)
```

    [[1.852e+03 1.000e+00 3.600e+03]]

Now we get the dimensions of velocity and apply the equation:

```python
Dim_v  = DimData.loc[['v'], LMT]           # LMT dimensions of velocity must be as list
λ_v    = np.prod(scales**Dim_v, axis=1);   # applies formula e returns also a vector

print('Velocity of 1 knot is equal to {0:5.3f}m/s.'.format(λ_v[0]))

```

    Velocity of 1 knot is equal to 0.514m/s.

what means that 1m/s corresponds to approximately 1.94knots. This is not a very exciting
result, for the same number could have been obtained simply dividing 3600 by 1852.
However, the main goal here is to demonstrate a computational procedure that is
as much concise as general.
Furthermore, the same procedure can be used to calculate scale changes of any dimensional
quantities, with scales being specified for quantities other then the fundamental ones.
This calculation is required for designing reduced scale models, as will be shown in
next class.

As an example, let us calculate the scale for velocity with scale changes being specified
for length, $L$, density, $\rho$, and acceleration, $a$. We assume that only the length
scale is changed, with $\lambda_L = 1:160$. This is a typical situation of a
reduced scale model tested in a wind tunnel, where fluid density (air) and gravity
cannot be changed.

In Python, the new base and the scales definition is done as follows:

```python
ABC    =  ['L', 'ρ', 'a']         # define quantities for the new base
base   =  DimData.loc[ABC, LMT]   # get dimensional matrix
i_base =  np.linalg.inv(base)     # get LMT dimensions in the new base

λ_L    =  1/160                   # length scale for the reduced model
λ_ρ    =  1/1                     # density remains unchanged (same fluid)
λ_a    =  1/1                     # acceleration remains unchanged (same gravity)

scales =  np.tile([λ_L, λ_ρ, λ_a],(4,1))
print(scales)

```

    [[0.00625 1.      1.     ]
     [0.00625 1.      1.     ]
     [0.00625 1.      1.     ]
     [0.00625 1.      1.     ]]

Now we change base, calculate new dimensions and apply scales:

```python
DimMat = DimData.loc[['t','F','σ','v'], LMT]     # LMT dimensions of velocity must be as list

NewMat = pd.DataFrame(data    = np.matmul(DimMat.values, i_base),
                      index   = DimMat.index,
                      columns = ABC)

λ_vec  = np.prod(scales**NewMat, axis=1);

print('Velocity in model scale is {0:6.4f} times the prototype velocity.'.format(λ_vec[3]))

1/λ_vec
```

    Velocity in model scale is 0.0791 times the prototype velocity.

    t    1.264911e+01
    F    4.096000e+06
    σ    1.600000e+02
    v    1.264911e+01
    dtype: float64

This means, for instance, that a wind speed of 1m/s applied on the model corresponds to a
wind speed of 12.65m/s applied on the real structure (also called the _prototype_),
as long as all other scales are equally observed for the model construction and testing.

It can be observed that the base $LMT$, with order $k = 3$, allows the _specification
of at most_ three scales for selected quantities, while all other quantities will have
their scales constrained by this choice. This fact must be kept in mind for designing
reduced scale models that are expected to provide informations that are useful in any
scale.

### 2.6. Famous $\Pi$-numbers

A análise dimensional permitiu que grandes cientistas apresentassem ao mundo números adimensionais que regem fenômenos importantes, tais como o número de Reynolds (forças de arrasto em corpos imersos em fluidos), de Froude (relação entre forças gravitacionais e inerciais), ou de Strouhal (frequência de desprendimento de vórtices alternados).

#### _Reynold's Number_

Um dos exemplos mais citados sobre uso de adimensionais para descrever fenômenos físicos é a relação entre o coeficiente de arrasto e o número de Reynolds no estudo do escoamento de fluidos em torno de corpos sólidos. O coeficiente de arrasto, $C_{\rm D}$, é uma relação entre a força de arrasto, $F_{\rm D}$ é a força aerodinâmica, $\rho V^2 A/2$, onde $\rho$ é a massa específica do fluido, $V$ é a velocidade do escoamento, e $A$ é a área exposta de referência (geralmente a seção perpendicular ao escoamento):

$$ C_{\rm D} = \frac{F_{\rm D}}{\frac{1}{2} \rho V^2 A}$$

Já o número de Reynolds representa uma relação entre as forças inerciais e as forças de viscosidade em um fluido, definido como:

$$ {\rm Re} = \frac{V \, D}{\nu}$$

onde $D$ é uma dimensão de referência da área exposta (perpendicular à direção do escoamento), e $\nu$ é a viscosidade cinemática do fluido ($\nu \approx 1.48 \times 10^5 {\rm m^2/s}$ para o ar na temperatura ambiente). A figura abaixo mostra o que resulta de uma investigação experimental da relação entre estes dois adimensionais para uma esfera.

Observa-se, neste caso, que a relação apresentada graficamente pode ser usada em qualquer escala, independentemente de existirem modelos preditivos baseados em princípios físicos. Aqui o fenômeno é descrito apenas em termos de adimensionais.

### 2.7. Assignment

1. Pesquise por um número $\Pi$ importante (Reynolds, Froude, Strouhal, Cauchy, Mach, Scruton, etc.), escreva a matriz dimensional das grandezas envolvidas, confirme a adimensionalidade usando a fórmula $\vec{\alpha}\,{\bf D}$, e descreva exemplos de aplicação.
2. Defina um problema de seu interesse, para o qual exista uma expressão dimensional conhecida e utilizada na prática. Reformule a expressão em termos de números $\Pi$ propostos. Apresente algum resultado (simulado ou disponível na literatura) na forma de gráficos relacionando estes números.

```python

```


Dynamics of offset diff drive vehicle

Equations are from the paper written by Masayoshi Wada etal.

https://www.jstage.jst.go.jp/article/jrsj1983/18/8/18_8_1166/_pdf

This notebook is written by Yosuke Matsusaka

# Preparation

We use sympy to transform the equation:

```python
!sudo pip install -U sympy
```

    DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
    WARNING: The directory '/home/developer/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
    Requirement already up-to-date: sympy in /usr/local/lib/python2.7/dist-packages (1.5.1)
    Requirement already satisfied, skipping upgrade: mpmath>=0.19 in /usr/local/lib/python2.7/dist-packages (from sympy) (1.1.0)

```python
from sympy import *
```

```python
init_printing()
```

# Symbols

Following symbols are used to describe internal state of the vehicle:

```python
(r, s, W, theta) = symbols(r"r s W \theta_s")
```

$r$ : radius of each wheels

$W$: wheel separation

$s$ : wheel offset

$\theta_s$ : current angle of the steering joint (described later)

Joint space parameters:

```python
(wr, wl, ws) = symbols('w_r w_l w_s')
uv = Matrix([wr, wl, ws])
```

$w_r$ : angular velocity of the right wheel joint

$w_l$ : angular velocity of the left wheel joint

$w_s$: angular velocity of the steering joint

Cartesian space parameters:

```python
(vx, vy, vz) = symbols('v_x v_y v_z')
xv = Matrix([vx, vy, vz])
```

$v_x$ : x axes velocity

$v_y$ : y axes velocity

$v_z$ : z axes velocity

# Dynamics of diff drive vehicle

```python
diff_drive = Matrix([
    [r/2, r/2],
    [0, 0],
    [r/W, -r/W]
])
Eq(xv, MatMul(diff_drive, Matrix(uv[0:2]), evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}v_{x}\\v_{y}\\v_{z}\end{matrix}\right] = \left[\begin{matrix}\frac{r}{2} & \frac{r}{2}\\0 & 0\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right] \left[\begin{matrix}w_{r}\\w_{l}\end{matrix}\right]$

x axes velocity is generated from the combination of left and right wheel velocities.

z axes velocity is generated from the difference of left and right wheel velocities.

We can natually understand that we cannot directly obtain y axes velocity from the mechanism.

# Dynamics of offset diff drive vehicle

Here, we apply offset to the diff drive vehicle described above.

We first, create offset transformation matrix:

```python
offset = Matrix([
    [1, 0, 0],
    [0, 1, s],
    [0, 0, 1]
])
offset
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & s\\0 & 0 & 1\end{matrix}\right]$

Apply offset to the jacobian matrix:

```python
offset_diff_drive = offset * diff_drive
Eq(offset_diff_drive, MatMul(offset, diff_drive, evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}\frac{r}{2} & \frac{r}{2}\\\frac{r s}{W} & - \frac{r s}{W}\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right] = \left[\begin{matrix}1 & 0 & 0\\0 & 1 & s\\0 & 0 & 1\end{matrix}\right] \left[\begin{matrix}\frac{r}{2} & \frac{r}{2}\\0 & 0\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right]$

Now, the equation will be as below:

```python
Eq(xv, MatMul(offset_diff_drive, Matrix(uv[0:2]), evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}v_{x}\\v_{y}\\v_{z}\end{matrix}\right] = \left[\begin{matrix}\frac{r}{2} & \frac{r}{2}\\\frac{r s}{W} & - \frac{r s}{W}\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right] \left[\begin{matrix}w_{r}\\w_{l}\end{matrix}\right]$

What is interesting here, is we get y axes velocity by just adding the offset!

By calculating the inverse matrix, we can realize omnidirectional movement from this mechanism:

```python
inv_offset_diff_drive = Matrix(offset_diff_drive[0:2, :]).inv()
Eq(Matrix(uv[0:2]), MatMul(inv_offset_diff_drive, Matrix(xv[0:2]), evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}w_{r}\\w_{l}\end{matrix}\right] = \left[\begin{matrix}\frac{1}{r} & \frac{W}{2 r s}\\\frac{1}{r} & - \frac{W}{2 r s}\end{matrix}\right] \left[\begin{matrix}v_{x}\\v_{y}\end{matrix}\right]$

Unfortunately z axes velocity is defined by holonomic constraints and not controllable by us. But we will add a steering joint to our vehicle to realize complete omnidirectional movement.

# Dynamics of offset diff drive vehicle with a steering joint

Here, we will add a steering joint to our vehicle to get controllable z axes velocity:

```python
rotate = Matrix([
    [cos(theta), -sin(theta), 0],
    [sin(theta), cos(theta), 0],
    [0, 0, 1]
])
rotate
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta_s \right)} & - \sin{\left(\theta_s \right)} & 0\\\sin{\left(\theta_s \right)} & \cos{\left(\theta_s \right)} & 0\\0 & 0 & 1\end{matrix}\right]$

By applying the rotation, jacobian matrix will be as follows:

```python
abs_offset_diff_drive = rotate * offset_diff_drive
Eq(abs_offset_diff_drive, MatMul(rotate, offset_diff_drive, evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}\frac{r \cos{\left(\theta_s \right)}}{2} - \frac{r s \sin{\left(\theta_s \right)}}{W} & \frac{r \cos{\left(\theta_s \right)}}{2} + \frac{r s \sin{\left(\theta_s \right)}}{W}\\\frac{r \sin{\left(\theta_s \right)}}{2} + \frac{r s \cos{\left(\theta_s \right)}}{W} & \frac{r \sin{\left(\theta_s \right)}}{2} - \frac{r s \cos{\left(\theta_s \right)}}{W}\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right] = \left[\begin{matrix}\cos{\left(\theta_s \right)} & - \sin{\left(\theta_s \right)} & 0\\\sin{\left(\theta_s \right)} & \cos{\left(\theta_s \right)} & 0\\0 & 0 & 1\end{matrix}\right] \left[\begin{matrix}\frac{r}{2} & \frac{r}{2}\\\frac{r s}{W} & - \frac{r s}{W}\\\frac{r}{W} & - \frac{r}{W}\end{matrix}\right]$

Final jacobian matrix is as follows:

```python
fwd_dynamics = BlockMatrix([abs_offset_diff_drive, Matrix([0, 0, -1])]).as_explicit()
Eq(xv,MatMul(fwd_dynamics, uv, evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}v_{x}\\v_{y}\\v_{z}\end{matrix}\right] = \left[\begin{matrix}\frac{r \cos{\left(\theta_s \right)}}{2} - \frac{r s \sin{\left(\theta_s \right)}}{W} & \frac{r \cos{\left(\theta_s \right)}}{2} + \frac{r s \sin{\left(\theta_s \right)}}{W} & 0\\\frac{r \sin{\left(\theta_s \right)}}{2} + \frac{r s \cos{\left(\theta_s \right)}}{W} & \frac{r \sin{\left(\theta_s \right)}}{2} - \frac{r s \cos{\left(\theta_s \right)}}{W} & 0\\\frac{r}{W} & - \frac{r}{W} & -1\end{matrix}\right] \left[\begin{matrix}w_{r}\\w_{l}\\w_{s}\end{matrix}\right]$

Inverse jacobian matrix will let us know how to calculate each joint velocity to realize omnidirectional movement:

```python
inv_dynamics = fwd_dynamics.inv().simplify()
Eq(uv, MatMul(inv_dynamics, xv, evaluate=False), evaluate=False)
```

$\displaystyle \left[\begin{matrix}w_{r}\\w_{l}\\w_{s}\end{matrix}\right] = \left[\begin{matrix}\frac{- \frac{W \sin{\left(\theta_s \right)}}{2} + s \cos{\left(\theta_s \right)}}{r s} & \frac{\frac{W \cos{\left(\theta_s \right)}}{2} + s \sin{\left(\theta_s \right)}}{r s} & 0\\\frac{\frac{W \sin{\left(\theta_s \right)}}{2} + s \cos{\left(\theta_s \right)}}{r s} & \frac{- \frac{W \cos{\left(\theta_s \right)}}{2} + s \sin{\left(\theta_s \right)}}{r s} & 0\\- \frac{\sin{\left(\theta_s \right)}}{s} & \frac{\cos{\left(\theta_s \right)}}{s} & -1\end{matrix}\right] \left[\begin{matrix}v_{x}\\v_{y}\\v_{z}\end{matrix}\right]$

```python

```