# Well-documented Demonstration Python Multidimensional Scaling Workflow

## Based on Spatial Data for California and Texas Cities

### The University of Texas at Austin, PGE 2020 SURI, Undergraduation Research
### Alan Scherman, Undergraduate Student, Rice University, UT PGE SURI 2020
### Supervised by:
### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

***
### Introduction
**Multidimensional scaling (MDS)** consists in a powerful tool widely used in inferential machine learning to *improve visualization and detection of patterns in highly dimensional, multivariate data sets*. MDS produces a **lower dimension dissimilarity-based projection** (i.e. usually a scatter plot in $2D$) of the high-dimension data set (where each dimension represents an additional feature or variable of measurement). A MDS projection may suggest clustering or ordering patterns between events that would have been otherwise undetectable in high dimensionality space. Geoscientists and statisticians treasure MDS because of its *independence of any distribution assumptions or knowledge of data set values* (i.e. MDS relies only on a distance matrix which details dissimilarity between observations).

***
### Theory
The beauty of MDS rests on its structural simplicity. If imagined as a function, MDS only requires a distance matrix $D$ (also known as *proximity matrix*) which describes pairwise dissimilarities as input and produces a lower dimensional projection with the best possible representation of the high dimensional discrepancies in $2D$ format. However, the MDS process and results contain some noteworthy peculiarities as explained below:

 - The input dataset can have the same dimensionality as the output projection (e.g. $2D$ data set to $2D$ map), in which case the projection is a perfect representation of the dissimilarities;
 - The distance matrix $D$ may also be computed from pairwise similarity data, in which case the similarities are converted into dissimilarities by the relationship $d_{ij} = \sqrt{s_{ii}+s_{jj}-2s_{ij}}$ where $d$ is events dissimilarity, $s$ is events similarity, and $i$ and $j$ refer to event row and column, respectively;
 - The MDS projection is independent of distribution assumptions;
 - The MDS projection is not unique because it is invariant to translation and rotation (hence MDS assumes centered features).

#### Goodness-of-fit / loss function
In most applications, MDS projects a data set from a higher to a lower set of dimensions (e.g. a $4$ feature data set onto $2D$ Cartesian coordinates), in which case some discrepancy between original event distances and projected event distances is expected. Therefore, *the best MDS projection reduces this loss function so that the projection is as accurate to the original model as is possible*. This loss function minimalization is essential to validate any assumptions drawn from the projection.

One of the most common loss functions or goodness-of-fit measures is Kruskal's First Stress:

\begin{equation}
\sigma_{K} = \sqrt{\frac{\sum_{i}^{n}\sum_{j,i\neq j}^{n}(x_{ij}-d_{ij})^2}{\sum_{j,i\neq j}^{n}(d_{ij})^2}}
\end{equation}

where $d$ is the original multidimensional dissimilarity, $x$ is the projected MDS distance, and $i$ and $j$ are the rows and the columns of events (note that the projected MDS matrix and $D$ have equal dimensions). Kruskal suggested in his 1964 paper on MDS the following stress thresholds to evaluate the quality of a MDS projection:

|Stress ($\sigma_{K}$)|Goodness-of-Fit|
|:-:|:-:|
|$\geq 0.200$| Poor|
|$\geq 0.100$| Fair|
|$\geq 0.050$| Good|
|$\geq 0.025$| Excellent|
|$0.000$| Perfect|

However, note that these are arbitrary thresholds determined from testing experience and that there are no rigorously-defined thresholds. Depending on the MDS application, technical expertise might say much more about the validity of a MDS model than the table values above.

Several algorithms which minimize stress, strain, and other loss functions have been proposed since the inception of MDS. Among these, the *SMACOF* algorithm proposed by Kruskal presents itself as a reliable and flexible iterative solution. Nevertheless, the details regarding these optimization algorithms are beyond the scope of this introductory demonstration. More information can be found under the References section.

#### Classical or Metric MDS
As its name foreshadows, classical or metric MDS refers to the first MDS algorithm proposed by W.S. Torgerson in his 1952 inaugural paper on MDS. This type of MDS assumes that the dissimilarities in $D$ are true Euclidean distances which can be computed from any data set by:

\begin{equation}
d_{ij}=\sqrt{\sum_{k}^{p}(\sum_{i}^{n} \sum_{j}^{n} (x_{ik}-x_{jk})^2)}
\end{equation}

where $d_{ij}$ is the Euclidean distance between events $i$ and $j$, $x_{ik}$ and $x_{jk}$ refer to the value of events $i$ and $j$, respectively, in the $k$ dimension, $n$ is the total number of observations, and $p$ is the total number of features in the data set.

Note that this method is also known as metric MDS because the most common dissimilarity measure, Euclidean distance or L2-norm, satisifies the following criteria for a **metric**:

Given points $x$, $y$, and $z$ in any real dimensional space, a dissimilarity measure $d$ is a metric if it satisfies:
1. $d(x,y) \geq 0$
2. $d(x,y) = 0$ iff $x=y$
3. $d(x,y) = d(y,x)$
4. $d(x,z) \leq d(x,y) + d(y,z)$

Some sources describe classical and metric MDS as different procedures since the latter has a weighting term applied to it. However, because classical and metric MDS share nearly all other steps, we present metric MDS as follows:

1. Compute $D$, an $p-by-p$ square matrix whose entries correspond to the Euclidean distances between any pair of observations (note that $D$ is symmetric and has a zeros diagonal);

2. Calculate $m$, a $p-by-1$ vector whose entries correspond to the weight of each observation (if all weights are equal, then $m_{i} = 1/p$);

3. Determine the centering matrix $C$ by $C = I - 1*m^{T}$, where $I$ is the identity matrix, $1$ is the unity vector, and $m^{T}$ is the transverse of $m$;

4. Calculate the cross-product matrix $S$, where $S = \frac{-1}{2}C*D*C^{T}$

5. Perform the eigendecomposition of $S$ so that the descending-ordered eigenvalues form the diagonal of a null matrix $\Lambda$ and the corresponding normalized eigenvectors are the columns of a matrix $U$;

6. Find the projected MDS coordinates in matrix $F$ by $F = M^{\frac{-1}{2}}*U*\Lambda^{\frac{1}{2}}$, where $M$ is the null matrix with $m$ as the main diagonal, $U$ is the normalized eigenvectors matrix, and $\Lambda$ is the null matrix with the eigenvalues as the main diagonal;

7. Plot the MDS coordinates into the selected number of dimensions; thus, to project an observation into a $2D$ Cartesian plane, take the values in the intersection between the observation's row and the first 2 columns and plot them in different orthogonal coordinates.

An interesting property of the *MDS dimensions is that their variance is equal to that of the eigenvalues of $S$*. Then, because the eigenvalues were sorted in descending order on step $5$, the first MDS dimension (first column of $F$) describes the most variance of the distance data, and the second MDS dimension (second column of $F$) describes the second most variance of the distance data.

#### Non-metric MDS
Non-metric MDS provides a more appealing solution to data sets whose *dissimilarities are not Euclidean distances or whose feature values are not known or quantified*. For instance, psychological studies employ non-metric MDS much more often than the metric counterpart because their data sets usually contain preferences or other subjective answer types (ordinal data). Non-metric MDS relies on other *more complex algorithms* which again lay beyond the scope of this demonstration. Nevertheless, it worth making two remarks about non-metric MDS: 1) the dissimilarities in $D$ are not equivalent to the original distances; rather, they are related by an unknown weakly monotonic function which also accounts for measurement errors and distortions; and 2) non-metric MDS has different versions of loss functions (e.g. strain).

***
### Objective
This demonstration will perform metric MDS on a 4-feature data set (distance to state centroid, population, median income, and mean travel time to work) of 10 Texan and 10 Californian cities. We will also compute Kruskal's stress and apply *sklearn.manifold.MDS* methods to compare our projection with an optimized, non-metric MDS solution.

***
### Load required libraries
We will use some standard Python packages to import, manage, and plot our cities data set. These should come previously installed if you have Anaconda or other similar environment software.

```python
import numpy as np                                            # To manage NumPy arrays
import pandas as pd                                           # To import and print summary stats for data set
from matplotlib import pyplot as plt                          # Plotting
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.
***
### Import sample data

Let us start off by importing our data from the *.csv* file to a *pandas.DataFrame*:

The dataset is available [here](https://github.com/GeostatsGuy/GeoDataSets/blob/master/CA_TX_cities_data.csv). In the https://github.com/GeostatsGuy/GeoDataSets/ respository on GitHub.

```python
df = pd.read_csv('CA_TX_cities_data.csv', sep=';')
df.head()
```

      Cities
      Distance (km)
      Population (people)
      Median Income (USD)
      Travel to Work (min)

      0
      Amarillo
      477
      203245
      52543
      17.8

      1
      Austin
      202
      974581
      67462
      24.3

      2
      Corpus Christi
      449
      326162
      55709
      19.2

      3
      Dallas
      279
      1358066
      50100
      27.0

      4
      El Paso
      678
      681877
      45656
      22.8

Let's check the summary statistics.

```python
df.describe().transpose()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Distance (km)
      20.0
      315.30
      142.894551
      57.0
      230.50
      267.00
      404.500
      678.0

      Population (people)
      20.0
      910789.25
      923586.575625
      91364.0
      344314.25
      606726.50
      1105862.750
      3979576.0

      Median Income (USD)
      20.0
      63680.95
      17685.658080
      37735.0
      51100.00
      58855.50
      70781.750
      104552.0

      Travel to Work (min)
      20.0
      24.68
      5.068022
      17.0
      21.15
      24.35
      27.675
      33.3

In case you are unfamiliar with Texan or Californian geography, rows 0-9 correspond to Texan cities and rows 10-19 correspond to Californian cities. These cities were chosen simply because they were the first to come up in a quick Google search. All data is taken from the most recent US Census Bureau information (see source under References section).

The values in the Distances column refer to the Euclidean (straight-line) distance between a city and its state geographical centroid. Therefore, Texan cities have their distance measured to Texas's centroid and Californian cities have their distance measured to California's centroid. The tool used to calculate the centroids and the distances can also be found under the References section.

Note that the features are scaled in different orders of magnitude, so we will standardize the values for each feature. Otherwise, the larger magnitude features will outweigh the impact on the distance between events of smaller magnitude features.

In order to standardize, let us take advantage of the *pandas* package *mean* and *std* methods. We will standardize each feature to obtain standard normal distributions (Gaussian):

```python
# Recall: standardized value = (original value - mean)/(standard deviation)

df['Distance (km)'] = (df['Distance (km)'] - df['Distance (km)'].mean())/df['Distance (km)'].std()

df['Population (people)'] = (df['Population (people)'] - df['Population (people)'].mean())/df['Population (people)'].std()

df['Median Income (USD)'] = (df['Median Income (USD)'] - df['Median Income (USD)'].mean())/df['Median Income (USD)'].std()

df['Travel to Work (min)'] = (df['Travel to Work (min)'] - df['Travel to Work (min)'].mean())/df['Travel to Work (min)'].std()

df
```

      Cities
      Distance (km)
      Population (people)
      Median Income (USD)
      Travel to Work (min)

      0
      Amarillo
      1.131604
      -0.766083
      -0.629773
      -1.357532

      1
      Austin
      -0.792892
      0.069070
      0.213792
      -0.074980

      2
      Corpus Christi
      0.935655
      -0.632997
      -0.450758
      -1.081290

      3
      Dallas
      -0.254033
      0.484282
      -0.767908
      0.457772

      4
      El Paso
      2.538235
      -0.247851
      -1.019185
      -0.370953

      5
      Fort Worth
      -0.554955
      -0.017967
      -0.250256
      0.497235

      6
      Houston
      0.760701
      1.545494
      -0.709103
      0.536699

      7
      Midland
      -0.345010
      -0.829580
      0.908253
      -1.160216

      8
      San Antonio
      -0.512966
      0.686327
      -0.718150
      -0.055248

      9
      Waco
      -0.757902
      -0.835293
      -1.467062
      -1.416726

      10
      Anaheim
      0.578748
      -0.606791
      0.325804
      0.753746

      11
      Bakersfield
      -0.771898
      -0.570216
      -0.075821
      -0.410417

      12
      Fresno
      -1.807627
      -0.410588
      -0.932504
      -0.568269

      13
      Los Angeles
      0.347809
      3.322684
      -0.299449
      1.325961

      14
      Oakland
      -0.414991
      -0.517286
      0.269204
      1.582472

      15
      Sacramento
      -0.554955
      -0.430025
      -0.295434
      0.181530

      16
      Santa Barbara
      -0.051087
      -0.887221
      0.628591
      -1.515384

      17
      San Diego
      1.565490
      0.555510
      0.665797
      -0.114443

      18
      San Francisco
      -0.331013
      -0.031659
      2.310971
      1.700861

      19
      San Jose
      -0.708914
      0.120190
      2.292991
      1.089182

The new data frame is now standardized by feature. Note that because we adopted a Gaussian distribution for each feature, the mean for every column is $0$ and the standard deviation is $1$. Still, this data frame alone without further statistical analysis provides little insight into typical Texan and Californian city behavior. In addition, we cannot tell for sure if any Texan or Californian town behaves more like a city in another state (i.e. outliers within state).
***
### Metric Multidimensional Scaling
With our sample data standardized, we can start our metric MDS calculations. We will follow the steps described under the Classical or Metric MDS subsection.

#### Compute $D$
Our imported data is currently a *pandas.DataFrame* of dimensions $20-by-4$. The first step in computing the distance matrix is to convert this dataframe into NumPy format and then apply the Euclidean distance formula to every pair of events. In other words, we want to find every possible distance between two observations in our data set.

```python
cities = df['Cities'].to_numpy()                                            # Store city names in np.array

df_2 = df.drop(labels='Cities',axis=1)
X = df_2.to_numpy(dtype=np.double)

D = np.zeros(shape=(np.shape(X)[0],np.shape(X)[0]))                         # Pre-allocate D with 20x20 zeros matrix

for i in range(len(D)):                                                     # Span rows
    for j in range(len(D)):                                                 # Span columns
        if i==j:                                                            # Distance of event to itself is null
            continue
        else:
            D[i,j] = np.sqrt((X[j,0]-X[i,0])**2+(X[j,1]-X[i,1])**2+\
                             (X[j,2]-X[i,2])**2+(X[j,3]-X[i,3])**2)         # Euclidean distance

print('First row of D is:\n',D[0,:])
print('First column of D is:\n',D[:,0])
plt.imshow(D)
plt.title('City Dissimilarity'); plt.xlabel('Cities ($i$)'); plt.ylabel('Cities, ($j$)');
plt.xticks(np.linspace(0,19,20),np.arange(1,21,1)); plt.yticks(np.linspace(0,19,20),np.arange(1,20,1)); plt.colorbar()
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.3, top=2.0, wspace=0.5, hspace=0.3)
```

The full matrix $D$ was not printed out for conciseness. Still, we can verify our resulting matrix by checking if the entries in the first row and first column (or any same ordinal pair of row and column) are identical (since $D$ is symmetric). Also, note that we saved the names under the 'Cities' column for future use.

#### Calculate $m$
For the sake of simplicity, we will adopt a uniform weighting term to all observations. However, depending on the type of analysis, it would be reasonable to imagine heavier masses to more populous or higher income cities.

```python
m = np.ones(shape=(len(D),1))*(1/len(D))
print('City weights, m = ' + str(np.matrix.transpose(m)) + '.')
print('Sum of the data sample weights = ' + str(round(np.sum(m),2)) + '.')
```

    City weights, m = [[0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05
      0.05 0.05 0.05 0.05 0.05 0.05]].
    Sum of the data sample weights = 1.0.

#### Determine centering matrix $C$
There is no secret here, we just need to ensure we are creating $C$ with the right dimensions (i.e. $20-by-20$):

```python
C = np.identity(n=len(m)) - np.full(shape=(len(m),len(m)),fill_value=m)
plt.imshow(C)
plt.title('Centering Matrix'); plt.xlabel('Cities ($i$)'); plt.ylabel('Cities, ($j$)');
plt.xticks(np.linspace(0,19,20),np.arange(1,21,1)); plt.yticks(np.linspace(0,19,20),np.arange(1,20,1));
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.3, top=2.0, wspace=0.5, hspace=0.3); plt.colorbar()
```

#### Calculate cross-product matrix $S$
Note that matrix multiplication is associative, so the order in which the multiplications are performed is irrelevant.

```python
S = -0.5*np.matmul(C,np.matmul(D,np.transpose(C)))
C = np.identity(n=len(m)) - np.full(shape=(len(m),len(m)),fill_value=m)
plt.imshow(S)
plt.title('Cross Product Matrix'); plt.xlabel('Cities ($i$)'); plt.ylabel('Cities, ($j$)');
plt.xticks(np.linspace(0,19,20),np.arange(1,21,1)); plt.yticks(np.linspace(0,19,20),np.arange(1,20,1));
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.3, top=2.0, wspace=0.5, hspace=0.3); plt.colorbar()
```

#### Perform the eigendecomposition of $S$
We will now do the eigendecomposition of $S$ (i.e. find the eigenvalues and the corresponding eigenvectors). For such task we will employ the *numpy.linalg.eig* function, which outputs a tuple with the ordered eigenvalues in one vector and the normalized corresponding eigenvectors in another array. Recall that $S$, which is $20-by-20$ can have at most 20 distinct eigenvalues. Here another small difference between metric and non-metric MDS factors in: the former disregards any negative eigenvalues as errors. Let us check our eigenvalues:

```python
eigen = np.linalg.eig(S)
print('Eigenvalues are:\n',eigen[0])
```

    Eigenvalues are:
     [ 5.70446814e+00  4.62045049e+00  3.90401069e+00  2.16091355e+00
      1.52491297e+00  1.04931104e+00  9.41608227e-01  8.21025808e-01
      7.53974017e-01 -2.58736710e-16  5.69896220e-01  5.43041391e-01
      4.80631715e-01  1.87121164e-01  1.81061562e-01  2.32970779e-01
      2.46747370e-01  3.15060005e-01  3.79947760e-01  3.48284601e-01]

The printed array shows that $S$ has 20 eigenvalues but one of them (the 10th in the ordered vector) is essentially zero (it is represented as an extremely small negative number for computational reasons). Let us adjust this eigenvalue to zero and proceed:

```python
U = eigen[1]

Lambda = np.where(eigen[0]<=0,0,eigen[0])
print('Eigenvalues are:\n',Lambda)
```

    Eigenvalues are:
     [5.70446814 4.62045049 3.90401069 2.16091355 1.52491297 1.04931104
     0.94160823 0.82102581 0.75397402 0.         0.56989622 0.54304139
     0.48063172 0.18712116 0.18106156 0.23297078 0.24674737 0.31506
     0.37994776 0.3482846 ]

#### Find the projected MDS coordinates
This next step looks daunting because of the multiple matrix operations and the seemingly incoherent output (a $20-by-20$ matrix with MDS coordinates). With regards to the first concern, remember that matrix multiplication is associative and that any square root or inverse operations precede multiplication. Also, note that this step requires us to build two matrices whose diagonals are vectors we previously calculated. These are the mass matrix ($M$) whose diagonal is the vector $m$, and the eigenvalue matrix ($\Lambda$) whose diagonal is the vector of eigenvalues. All other non-diagonal entries are kept as null values. Thus:

```python
scores = np.matmul(np.sqrt(np.linalg.inv(np.diag(m[:,0]))),np.matmul(U,np.sqrt(np.diag(Lambda))))
```

We call the output of this step *scores*. Each column in *scores* descends from an eigenvalue, and so it indicates some percentage of the variance in the distances matrix ($D$). Because we have already ordered the eigenvalues, the first column of *scores* contains the coordinates of the observations in the MDS dimension with the most variance in distance. Likewise, the second column of *scores* contains the coordinates of the observations in the MDS dimension with the second most variance in distance. This trend repeats itself for the remaining columns of *scores*.

#### Plot the MDS coordinates
To plot the MDS transformation, we first need to recall what is the purpose of this MDS projection. We originally had a data set of 20 events against 4 features and we seek to visualize this data set behavior in a more friendly 2 dimensional space. Hence, we will take the first two columns of *scores* to plot the projected data in $2D$.

```python
x_proj = scores[:,0]
y_proj = scores[:,1]

#plt.style.use('ggplot')
fig1 = plt.figure()
plt.scatter(x_proj[0:10],y_proj[0:10], color='red', label='Texan cities',s=150,alpha=0.75)
plt.scatter(x_proj[10:20],y_proj[10:20], color='blue', label='Californian cities',s=150,alpha=0.75)

for i, txt in enumerate(cities):
    plt.annotate(txt, (x_proj[i]+0.05, y_proj[i]+0.1), size=12, style='italic')

plt.title('4 Feature, 2D MDS Projection of California and Texas cities')
plt.xlabel('First MDS dimension',size=20)
plt.ylabel('Second MDS dimension',size=20)
legend = plt.legend(loc = 'center left'); legend.get_frame().set_linewidth(2)
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.5, top=1.7, wspace=0.5, hspace=0.3)
plt.show()
```

It is important to remark that the *absolute coordinates of the observations in the MDS projected space have no statistical meaning*. Metric MDS strives to preserve as best as possible the original multidimensional distances between data points in a lower-dimensional space.

At a first glance, the MDS projection above looks pretty disappointing. Other than some curious pairwise observations (e.g. both state capitals - Austin and Sacramento - and cities in oil hotspots - Midland and Bakersfield - are close together), the projection has both groups of cities overlapping for the most part. No distinct clusters can be identified. Even the pairwise observations can be explained by a social scientist with enough data and real-world understanding.

Let us dig a little deeper and, in addition to classifying the events by their state, let us also scale their sizes by feature intensity:

```python
fig, [[sct1, sct2], [sct3, sct4]] = plt.subplots(2,2,figsize=(30,25))

# Normalizing feature observations to 0-1 range
s1 = (X[:,0] - min(X[:,0]))/(max(X[:,0] - min(X[:,0])))+0.05
s2 = (X[:,1] - min(X[:,1]))/(max(X[:,1] - min(X[:,1])))+0.05
s3 = (X[:,2] - min(X[:,2]))/(max(X[:,2] - min(X[:,2])))+0.05
s4 = (X[:,3] - min(X[:,3]))/(max(X[:,3] - min(X[:,3])))+0.05

# MDS scaled by distance feature
sct1.scatter(x_proj[0:10],y_proj[0:10],s=1000*s1[0:10],label='Texan cities',color='red',alpha=0.75)
sct1.scatter(x_proj[10:20],y_proj[10:20],s=1000*s1[10:20],label='Californian cities',color='blue',alpha=0.75)
sct1.set_title('MDS Projection w/ events scaled by Distance feature',size=25)
sct1.set_xlabel('First MDS Dimension',size=18)
sct1.set_ylabel('Second MDS Dimension',size=18)
legend = sct1.legend(loc='center left',fontsize=15); legend.get_frame().set_linewidth(2)

# MDS scaled by population feature
sct2.scatter(x_proj[0:10],y_proj[0:10],s=1000*s2[0:10],label='Texan cities',color='red',alpha=0.75)
sct2.scatter(x_proj[10:20],y_proj[10:20],s=1000*s2[10:20],label='Californian cities',color='blue',alpha=0.75)
sct2.set_title('MDS Projection w/ events scaled by Population feature',size=25)
sct2.set_xlabel('First MDS Dimension',size=18)
sct2.set_ylabel('Second MDS Dimension',size=18)
legend = sct2.legend(loc='center left',fontsize=15); legend.get_frame().set_linewidth(2)

# MDS scaled by median income feature
sct3.scatter(x_proj[0:10],y_proj[0:10],s=1000*s3[0:10],label='Texan cities',color='red',alpha=0.75)
sct3.scatter(x_proj[10:20],y_proj[10:20],s=1000*s3[10:20],label='Californian cities',color='blue',alpha=0.75)
sct3.set_title('MDS Projection w/ events scaled by Median Income feature',size=25)
sct3.set_xlabel('First MDS Dimension',size=18)
sct3.set_ylabel('Second MDS Dimension',size=18)
legend = sct3.legend(loc='center left',fontsize=15); legend.get_frame().set_linewidth(2)

# MDS scaled by travel to work feature
sct4.scatter(x_proj[0:10],y_proj[0:10],s=1000*s4[0:10],label='Texan cities',color='red',alpha=0.75)
sct4.scatter(x_proj[10:20],y_proj[10:20],s=1000*s4[10:20],label='Californian cities',color='blue',alpha=0.75)
sct4.set_title('MDS Projection w/ events scaled by Travel to Work feature',size=25)
sct4.set_xlabel('First MDS Dimension',size=18)
sct4.set_ylabel('Second MDS Dimension',size=18)
legend = sct4.legend(loc='center left',fontsize=15); legend.get_frame().set_linewidth(2)

# Add name tags to cities
for i,txt in enumerate(cities):
    sct1.annotate(txt, (x_proj[i], y_proj[i]), size=18, style='italic')
    sct2.annotate(txt, (x_proj[i], y_proj[i]), size=18, style='italic')
    sct3.annotate(txt, (x_proj[i], y_proj[i]), size=18, style='italic')
    sct4.annotate(txt, (x_proj[i], y_proj[i]), size=18, style='italic')
```

The feature-scaled graphs reveal some clusters or at least tendencies in the original data set. For the distance-scaled plot, cities with greater distance to state centroid tend to group in the bottom half of the plot (remember this grouping could exist elsewhere because MDS is blind to orientation). For the population-scaled map, population seems to increase as one moves diagonally to the bottom left corner and to decrease towards the top right corner. For the income-scaled plot, median income appears to increase as one moves up (increases second MDS dimension in this orientation). For the travel time scaled plot, cities with higher mean travel time to work seem to congregate above the diagonal going from the bottom left to the top right.

Let us also experiment removing one of the features (e.g. distance to state centroid) and running metric MDS again. The steps are the same from before, but everything after the distance matrix with exception of the weight vector and the centering matrix needs to be recalculated:

```python
X2 = np.delete(X,0,1)                                                        # Delete first column (distances feature)

D2 = np.zeros(shape=(np.shape(X2)[0],np.shape(X2)[0]))                       # Pre-allocate D2 with 20x20 zeros matrix

for i in range(len(D2)):                                                     # Span rows
    for j in range(len(D2)):                                                 # Span columns
        if i==j:                                                             # Distance of event to itself is null
            continue
        else:
            D2[i,j] = np.sqrt((X2[j,0]-X2[i,0])**2+(X2[j,1]-X2[i,1])**2+\
                             (X2[j,2]-X2[i,2])**2)

# m and C are the same

# Cross-product matrix
S2 = -0.5*np.matmul(C,np.matmul(D2,np.transpose(C)))

# Find eigenvalues and eigenvectors
eigen2 = np.linalg.eig(S2)
Lambda2 = np.where(eigen2[0]<=0,0,eigen2[0])
U2 = eigen2[1]

# Eigendecomposition
scores2 = np.matmul(np.sqrt(np.linalg.inv(np.diag(m[:,0]))),np.matmul(U2,np.sqrt(np.diag(Lambda2))))

# Assigning MDS coordinates
x_proj2 = scores2[:,0]
y_proj2 = scores2[:,1]

# Plotting both projections
fig, [sct1,sct2] = plt.subplots(1,2,figsize=(24,12))

sct1.scatter(x_proj[0:10],y_proj[0:10], color='red', label='Texan cities',s=100,alpha=0.75)
sct1.scatter(x_proj[10:20],y_proj[10:20], color='blue', label='Californian cities',s=100,alpha=0.75)

sct1.set_title('4 Feature, 2D MDS Projection \n of California and Texas cities',size=25)
sct1.set_xlabel('First MDS dimension',size=20)
sct1.set_ylabel('Second MDS dimension',size=20)
legend = sct1.legend(loc='center left',fontsize=15); legend.get_frame().set_linewidth(2)

sct2.scatter(x_proj2[0:10],y_proj2[0:10], color='red', label='Texan cities',s=100,alpha=0.75)
sct2.scatter(x_proj2[10:20],y_proj2[10:20], color='blue', label='Californian cities',s=100,alpha=0.75)

sct2.set_title('3 Feature, 2D MDS Projection \n of California and Texas cities',size=25)
sct2.set_xlabel('First MDS dimension',size=20)
sct2.set_ylabel('Second MDS dimension',size=20)
legend = sct2.legend(fontsize=15); legend.get_frame().set_linewidth(2)

for i, txt in enumerate(cities):
    sct1.annotate(txt, (x_proj[i], y_proj[i]), size=17, style='italic')
    sct2.annotate(txt, (x_proj2[i], y_proj2[i]), size=17, style='italic')
```

The new 3-feature MDS projection shows a rather different global arrangement of the cities, but preserves many of the projected distances between them. For some pairs of cities (e.g. San Jose and San Francisco or Corpus Christi and Amarillo), the removal of the distance feature actually reveals more similarities between them. However, for other pairs like Anaheim and Dallas, the deletion of the distance variable increases the projected dissimilarity between cities. Lastly, the new MDS projection reveals more similarities between cities that are geographically situated far away from state centroids (e.g. El Paso) and others. Note that because California and Texas are both territorially large states with cities relatively well spread along their areas, the impact of removing the distance feature is limited. Nevertheless, if another smaller state were present, like Alabama or Illinois, removing the distance feature could prove meaningful in the appearance of clusters and identification of trends.

Nevertheless, one might argue that finding patterns in data is just a matter of digging deep enough. The tendencies observed might just be correlations and so we must not jump the hedge and draw causation relations between a feature and MDS clusters. Last but not least, we must compute the goodness-of-fit for this MDS projection so we know how concrete our inferences are.

***
### Kruskal Stress
The plots above provided neat visualizations and some pattern inferences about our $4D$ data set, but we must evaluate how well the projected MDS distances reflect the original dissimilarities between events in higher-dimensional space. For such task we will compute Kruskal's Stress using the formula provided under the Theory section.

However, note that the Kruskal Stress formula requires projected pairwise distances. Currently, we only have the *scores* matrix which represents projected coordinates and not pairwise distances. We can obtain the projected distances with some for-loops and the Euclidean distance formula:

```python
proj_dists = np.zeros(shape=(np.shape(D)))

for i in range(len(D)):
    for j in range(len(D)):
        if i==j:
            continue                                                                     # Distance of event to itself is null
        else:
            proj_dists[i,j] = np.sqrt((scores[j,0]-scores[i,0])**2 + (scores[j,1]-scores[i,1])**2)        # Euclidean distance
```

Now, we can apply the Kruskal Stress formula where the $x_{ij}$ values come from *proj_dists* and the $d_{ij}$ values from the $D$ matrix:

```python
num = 0; den = 0                                            # Preallocating numerator and denominator

for i in range(len(D)):                                     # Span rows
    for j in range(len(D)):                                 # Span columns
        if i==j:
            continue
        else:
            num += (proj_dists[i,j] - D[i,j])**2
            den += (D[i,j])**2

stress = np.sqrt(num/den)
print('Kruskal First Stress is: ',stress)
```

    Kruskal First Stress is:  0.69690231212473

If we take the conventional threshold values set by Kruskal (see Theory section), this metric MDS projection ranks poorly (*stress* $\geq 0.2$). In other words, there is some significant distortion between the sets of original and MDS distances. This unevenness may disqualify some of our pattern inferences made in the previous section and lead us to reconsider if metric MDS is the best analysis methodology at our disposal.

Another form of determining if the projection is accurate is to plot the projected pairwise distances (stored in *proj_dists*) against the original pairwise distances (stored in *D*) in a scatter plot. A zero stress projection would make the scattered data exhibit linear behavior (i.e. data points would follow $y=x$ line). In our case:

```python
#plt.style.use('fivethirtyeight')

plt.scatter(D,proj_dists,label='Projected vs Original Distances',color='red')
plt.plot(2*np.max(D)*np.array([0,1]),2*np.max(D)*np.array([0,1]),color='black',linewidth=2,label=r'Null Stress ($y=x$)')

plt.title('Projected vs Original Pairwise Distances for Metric MDS');
plt.xlabel('Original Euclidean Distance',color='black'); plt.xlim([0,10]); plt.ylim([0,10])
plt.ylabel('Projected Euclidean Distance',color='black');
plt.legend(loc='lower right');
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.5, top=1.7, wspace=0.5, hspace=0.3)
```

The data in the scatter plot exhibit somewhat linear behavior, but not about the null stress line $y=x$. The more the data moves away from the null stress line, the less preserved are the original dissimilarities between events in higher-dimensional space.
***
### sklearn.manifold.MDS
Still, we can validate our metric MDS projection by comparing it against a non-metric MDS projection generated by *sklearn.manifold.MDS*. We will not dive into the details of this MDS tool, but it is important to state that *sklearn* contains powerful algorithms capable of iteratively optimizing the projection's Kruskal stress. Let us import the MDS module and look at its docstring:

```python
from sklearn.manifold import MDS
print(MDS.__doc__)
```

    Multidimensional scaling

        Read more in the :ref:`User Guide <multidimensional_scaling>`.

        Parameters
        ----------
        n_components : int, optional, default: 2
            Number of dimensions in which to immerse the dissimilarities.

        metric : boolean, optional, default: True
            If ``True``, perform metric MDS; otherwise, perform nonmetric MDS.

        n_init : int, optional, default: 4
            Number of times the SMACOF algorithm will be run with different
            initializations. The final results will be the best output of the runs,
            determined by the run with the smallest final stress.

        max_iter : int, optional, default: 300
            Maximum number of iterations of the SMACOF algorithm for a single run.

        verbose : int, optional, default: 0
            Level of verbosity.

        eps : float, optional, default: 1e-3
            Relative tolerance with respect to stress at which to declare
            convergence.

        n_jobs : int or None, optional (default=None)
            The number of jobs to use for the computation. If multiple
            initializations are used (``n_init``), each run of the algorithm is
            computed in parallel.

            ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.
            ``-1`` means using all processors. See :term:`Glossary <n_jobs>`
            for more details.

        random_state : int, RandomState instance or None, optional, default: None
            The generator used to initialize the centers.  If int, random_state is
            the seed used by the random number generator; If RandomState instance,
            random_state is the random number generator; If None, the random number
            generator is the RandomState instance used by `np.random`.

        dissimilarity : 'euclidean' | 'precomputed', optional, default: 'euclidean'
            Dissimilarity measure to use:

            - 'euclidean':
                Pairwise Euclidean distances between points in the dataset.

            - 'precomputed':
                Pre-computed dissimilarities are passed directly to ``fit`` and
                ``fit_transform``.

        Attributes
        ----------
        embedding_ : array-like, shape (n_samples, n_components)
            Stores the position of the dataset in the embedding space.

        stress_ : float
            The final value of the stress (sum of squared distance of the
            disparities and the distances for all constrained points).

        Examples
        --------
        >>> from sklearn.datasets import load_digits
        >>> from sklearn.manifold import MDS
        >>> X, _ = load_digits(return_X_y=True)
        >>> X.shape
        (1797, 64)
        >>> embedding = MDS(n_components=2)
        >>> X_transformed = embedding.fit_transform(X[:100])
        >>> X_transformed.shape
        (100, 2)

        References
        ----------
        "Modern Multidimensional Scaling - Theory and Applications" Borg, I.;
        Groenen P. Springer Series in Statistics (1997)

        "Nonmetric multidimensional scaling: a numerical method" Kruskal, J.
        Psychometrika, 29 (1964)

        "Multidimensional scaling by optimizing goodness of fit to a nonmetric
        hypothesis" Kruskal, J. Psychometrika, 29, (1964)

There is a lot of information in the docstring above, but the relevant parts to us are: (1) the parameter *n_components*, which establishes the dimensionality of the MDS projection; and (2) the attribute *fit_transform()*, which performs an operation similar to step 6 in the metric MDS subsection to obtain the *scores* coordinates matrix.

```python
n_components = 2                                                                 # MDS projection dimensionality

embedding = MDS(n_components=2,random_state=20348, dissimilarity='precomputed')  # dissimilarity parameter shows D is already known
nm_scores = embedding.fit_transform(D)

print('MDS projected coordinates matrix (nm_matrix) has dimensions: ', np.shape(nm_scores))
```

    MDS projected coordinates matrix (nm_matrix) has dimensions:  (20, 2)

Note that we assign a seed (*random_state*) argument to the MDS command. This will ensure that we will always get the same orientation for the MDS projection. The *nm_scores* matrix (similar to the *scores* matrix) has the expected dimensions and it indicates the projected event coordinates. Then:

```python
nm_x_proj = nm_scores[:,0]
nm_y_proj = nm_scores[:,1]

plt.figure(figsize=(14,10))
plt.scatter(nm_x_proj[0:10],nm_y_proj[0:10], color='red', label='Texan cities')
plt.scatter(nm_x_proj[10:20],nm_y_proj[10:20], color='blue', label='Californian cities')

for i, txt in enumerate(cities):
    plt.annotate(txt, (nm_x_proj[i], nm_y_proj[i]), size=15, style='italic')

plt.title('Nonmetric MDS 2D Projection of California and Texas cities',size=25)
plt.xlabel('First MDS dimension',size=20)
plt.ylabel('Second MDS dimension',size=20)
legend = plt.legend(); legend.get_frame().set_linewidth(2)
plt.show()
```

An untrained eye might conclude upon seeing this nonmetric MDS projection that the metric equivalent we computed was completely erroneous. Nonetheless, recall from the Theory section that **MDS projections are invariant to translation and rotation**, so we can adjust the rotation of either metric or nonmetric MDS projection to have matching orientations. Note that in the nonmetric projection Los Angeles is in the bottom right corner and El Paso in the top right, while in the metric projection the former is in the bottom left and the latter in the bottom right. Hence, if we **rotate the nonmetric MDS projection clockwise by 90$^{\circ}$** (or equivalently rotate the metric projection counterclockwise by 90 $^{\circ}$), then both projections will have the same orientation. Let us see an overlapped plot of both metric and nonmetric MDS where the *squares represent metric MDS events* and the *circles represent nonmetric MDS events*.

```python
plt.figure(figsize=(16,12))

nm_scores = np.matmul(nm_scores,np.array([[0,-1],[1,0]]))       # Rotate nonmetric projection by 90 degrees clockwise
nm_x_proj = nm_scores[:,0]; nm_y_proj = nm_scores[:,1]

for i, txt in enumerate(cities):
    plt.scatter(nm_x_proj[i],nm_y_proj[i], color='purple', marker='o',s=120)    # Plot nonmetric MDS
    plt.scatter(x_proj[i],y_proj[i], color='green', marker='s',s=150)           # Plot metric MDS
    plt.annotate(txt, (nm_x_proj[i], nm_y_proj[i]), size=12, style='italic')
    plt.annotate(txt, (x_proj[i], y_proj[i]), size=12, style='italic')

plt.title('Nonmetric vs Metric MDS 2D Projection of California and Texas cities',size=25);
plt.xlabel('First MDS dimension',size=20);
plt.ylabel('Second MDS dimension',size=20);
plt.legend(('Nonmetric MDS', 'Metric MDS'));
```

The overlapping plot validates the locations of most metric MDS projections. For instance, the pairs Corpus Christi and Amarillo and San Francisco and San Jose remain close neighbors in both projections. Observe that the cities farther away from the main cluster (e.g. Los Angeles and El Paso) show more discrepancies between projections. Furthermore, the nonmetric projections seem to concentrate more about the MDS plane's origin whilst the metric events seem more scattered. Nonetheless, the projections have many subtle arrangement differences which depending on the MDS application might prove relevant or not.

Again, the nonmetric MDS does not reveal any extraordinary clusters or tendencies that could lead to astounding discoveries. Yet, some interesting patterns and individual city behavior can be drawn. First, four out of Texas's 5 most populated cities (Austin, Dallas, Fort Worth, and San Antonio) stand close to one another in the middle of the projection. Houston appears as the outlier, behaving more like San Diego or Los Angeles than its fellow Texan cities. Second, Santa Barbara and Anaheim (both in California) behave much alike other Texan cities. Third, El Paso sits far away from other Texan cities (and from much else too).

Finally, let us compute the Kruskal stress of the nonmetric MDS projection:

```python
nm_proj_dists = np.zeros(shape=(np.shape(D)))

for i in range(len(D)):
    for j in range(len(D)):
        if i==j:
            continue                                                               # Distance of event to itself is null
        else:
            nm_proj_dists[i,j] = np.sqrt((nm_scores[j,0]-nm_scores[i,0])**2 + \
                                         (nm_scores[j,1]-nm_scores[i,1])**2)       # Euclidean distance

num = 0; den = 0                                                                   # Preallocating numerator and denominator

for i in range(len(D)):                                                            # Span rows
    for j in range(len(D)):                                                        # Span columns
        if i==j:
            continue
        else:
            num += (nm_proj_dists[i,j] - D[i,j])**2
            den += (D[i,j])**2

stress = np.sqrt(num/den)
print('Kruskal First Stress is: ',stress)
```

    Kruskal First Stress is:  0.16322637055453249

And the projected pairwise distances against the original pairwise distances scatter plot:

```python
plt.figure(figsize=(16,12))

plt.scatter(D,nm_proj_dists,label='Projected vs Original Distances')
plt.plot(np.max(D)*np.array([0,1]),np.max(D)*np.array([0,1]),color='black',linewidth=2,label='Null Stress (y=x)')

plt.title('Projected versus Original Pairwise Distances for Nonmetric MDS',size=25);
plt.xlabel('Original Euclidean Distance',size=20);
plt.ylabel('Projected Euclidean Distance',size=20);
plt.legend();
```

This time the linear trend formed by the projected versus original pairwise distances is clear. It is not a perfect fit because the Kruskal stress is not null, but the linear trend about $y=x$ attests to the fact that the nonmetric MDS projection preserves the dissimilarity better than its metric counterpart.

***
### References
This workflow covered a range of topics pertaining to MDS theory, demographic and geographic data, and Python utilities for MDS. There is so much more to learn about these areas, so please check out the list below for further information:

1. [Metric MDS step-by-step](https://personal.utdallas.edu/~herve/Abdi-MDS2007-pretty.pdf)
2. [Stress measurements](https://ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Multidimensional_Scaling.pdf)
3. [MDS overview, SMACOF algorithm, and MDS in R](https://mran.microsoft.com/snapshot/2018-05-08/web/packages/smacof/vignettes/smacof.pdf)
4. [Interpreting MDS projections](http://www.analytictech.com/borgatti/mds.htm)
5. [Dr. Michael Pyrcz's lecture on MDS](https://www.youtube.com/watch?v=Yt0o8ukIOKU)
6. [Distance matrices](https://medium.com/swlh/euclidean-distance-matrix-4c3e1378d87f)
7. [US Census Bureau data](https://www.census.gov/programs-surveys/sis/resources/data-tools/quickfacts.html)
8. [List of US State Centroids](https://en.wikipedia.org/wiki/List_of_geographic_centers_of_the_United_States)
9. [sklearn.manifold.MDS Documentation](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.MDS.html)

#### Work was supervised by Professor Pyrcz as part of the Summer Undergraduate Research Project (SURI) at The University of Texas at Austin.  See below for more information about Professor Pyrcz.

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


# SGA - Sampling Discrete Structures

**Notebook:**
[](https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/DL2/sampling/introduction.ipynb)
[](https://colab.research.google.com/github/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/DL2/sampling/introduction.ipynb)
**Author:**
Adeel Pervez

In these tutorials we discuss methods for sampling discrete variables from unstructured vectors to more structured objects such as subsets, permutations and graphs which can be incorporated in differentiable models.

At the foundation of these methods are continuous relaxations for discrete (binary or categorical) random variables.
So the first part of the tutorial gives an introduction to sampling form discrete distributions with the Gumbel-Softmax trick.
We use the method to train a variational autoencoder with categorical latent variables.

## Categorical Sampling with Gumbel-Argmax

Suppose we are given a categorical distribution with $C$ values as weights $w_i \in (0,\infty)$.
We would like to obtain a sample from this distribution.
The probablity of each category $c_i$ is given by the following softmax distribution

$$p_i = \frac{\exp(\log(w_i))}{\sum_j \exp(\log(w_j))}$$

The Gumbel-Argmax method for sampling this distribution is the following:
Sample $U_k \sim Uniform(0,1)$ iid and compute $r_k = \log\alpha_k -log(-\log U_k)$. Then choose the index $i$ of the maximum $r_k$ (ie take the argmax) and return the 1-hot vector with the $i$th index set to 1 and the rest to 0.
The form of noise $-log(-\log U_k)$ added to form $r_k$ has a Gumbel distribution whence the method gets its name. The cumulative distribution function of the Gumbel distribution (with location 0 and scale 1) is given as

$$F(z) = \exp(\exp(-z))$$

You can take a look at a proof that this indeed samples from the softmax distribution [here](https://lips.cs.princeton.edu/the-gumbel-max-trick-for-discrete-distributions/).

In short sampling a categorical variable with the Gumbel reparameterization proceeds as follows.

1. Given weights $w_i$ compute $r_i = w_i+g_i$ where $g_i$ are iid Gumbel samples
2. Argmax: Return index of largest $r_i$ as a 1-hot vector

## Softmax Relaxation

The above procedure still cannot be used in a differentiable model since the argmax operation has zero gradient except at points of discontinuity.
So instead we use the `softmax` as a differentiable approximation to the argmax.
In order to control the approximation we introduce a tunable temperature hyperparameter $\tau$ which controls how far the softmax outputs are from being 1-hot.

$$p_i = \frac{\exp(r_i/\tau)}{\sum_j \exp(r_j/\tau)}$$

The following figure ([Paper Link](https://arxiv.org/abs/1611.01144)) shows the effect of temperature on the distribution and samples.

A smaller temperature indicates a tighter approximation and the larger the temperature the looser the approximation.
Of course, if the temperature is too small we wouldn't be able to train the model since the gradients would be very small.
On the other hand, a large temperature would make the categorical outputs very far from being discrete, so it is important to choose an appropriate temperature for the problem at hand.
One possiblity is to slowly anneal the temperature from large to small so that close to the end of training the relaxed categorical outputs are closed to discrete.
In practice, however, the temperature is often kept fixed during each training trial and tuned with cross-validation.

With the softmax relaxation the sampling then proceeds as follows

1. Given weights $w_i$ compute $r_i = w_i+g_i$ where $g_i$ are iid Gumbel samples
2. Apply softmax with temperature to obtain a relaxed categorical sample

## Categorical VAE

As an example of the Gumbel Softmax relaxation we show a VAE with a categorical variable latent space for MNIST. The latent space has the structure of a vector of categorical variables each with the same fixed number of categories.
In the following example the latent space has 30 categorical variables each of dimension 10.
Since this is a VAE we also need to define a prior on the latent space which we define to be the uniform categorical distribution.

The following implementation uses code from [here](https://github.com/YongfeiYan/Gumbel_Softmax_VAE) with minor modification.

We being with the required imports and hyperparameter definitions.

```python
import numpy as np

import torch
import torch.nn.functional as F
from torch import nn, optim
from torch.nn import functional as F
from torchvision import datasets, transforms
from torchvision.utils import save_image
from torch.distributions.one_hot_categorical import OneHotCategorical

import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

cuda=True
```

```python
batch_size = 100
epochs = 10
latent_dim = 30
categorical_dim = 10
temp = 1.0
```

### Gumbel Sampling

We now define the Gumbel-Softmax sampling routines. The `sample_gumbel` function samples scale 0 location 1 Gumbel variables by sampling uniform random variables in $U(0,1)$ and computing $-\log(-\log(U(0,1))$.
The categorical parameters are input as unnormalized log probabilities.
The `gumbel_softmax_sample` function adds the Gumbel noise to the logits, applies the temperature and the softmax function.
In the `gumbel_softmax` function we also add evaluation code which simply returns a sample (unrelaxed) from the categorical distribution parameterized by `logits`.

```python
def sample_gumbel(shape, eps=1e-20):
    U = torch.rand(shape)
    if cuda:
        U = U.cuda()
    return -torch.log(-torch.log(U + eps) + eps)

def gumbel_softmax_sample(logits, temperature):
    y = logits + sample_gumbel(logits.size())
    return F.softmax(y / temperature, dim=-1)

def gumbel_softmax(logits, temperature, evaluate=False):
    if evaluate:
        d =  OneHotCategorical(logits=logits.view(-1, latent_dim, categorical_dim))
        return d.sample().view(-1, latent_dim * categorical_dim)

    y = gumbel_softmax_sample(logits, temperature)
    return y.view(-1, latent_dim * categorical_dim)
```

### VAE model
Now we define the VAE model.
The encoder computes the categorical probability parameters from which relaxed categorical variables can be sampled and passed into the decoder.

```python
class VAE_gumbel(nn.Module):
    def __init__(self, temp):
        super(VAE_gumbel, self).__init__()

        self.fc1 = nn.Linear(784, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, latent_dim * categorical_dim)

        self.fc4 = nn.Linear(latent_dim * categorical_dim, 256)
        self.fc5 = nn.Linear(256, 512)
        self.fc6 = nn.Linear(512, 784)

        self.relu = nn.ReLU()
        self.sigmoid = nn.Sigmoid()

    def encode(self, x):
        h1 = self.relu(self.fc1(x))
        h2 = self.relu(self.fc2(h1))
        return self.relu(self.fc3(h2))

    def decode(self, z):
        h4 = self.relu(self.fc4(z))
        h5 = self.relu(self.fc5(h4))
        return self.sigmoid(self.fc6(h5))

    def forward(self, x, temp, evaluate=False):
        q = self.encode(x.view(-1, 784))
        q_y = q.view(q.size(0), latent_dim, categorical_dim)
        z = gumbel_softmax(q_y, temp, evaluate)
        return self.decode(z), F.softmax(q_y, dim=-1).reshape(*q.size())
```

### KL Divergence
Since this is a VAE we also need to compute a KL divergence of the latent probability distribution with the uniform prior $p(x) = 1/C$ for all $x$ where $C$ is the total number of categories.

\begin{align}
KLD(q||p) &= E_q\left[\log\frac{q(x)}{p(x)}\right]\\
        &= \sum_{i=1}^{C} q(x_i) \log (C \cdot q(x_i))
\end{align}

Here $q(x)$ is the latent probability distribution

Finally we compute the reconstruction loss of the input as the binary cross entropy between the reconstruction parameters and the input image and add that to the KL divergence to get the VAE loss.

```python
def loss_function(recon_x, x, qy):
    BCE = F.binary_cross_entropy(recon_x, x.view(-1, 784), size_average=False) / x.shape[0]

    log_ratio = torch.log(qy * categorical_dim + 1e-20)
    KLD = torch.sum(qy * log_ratio, dim=-1).mean()

    return BCE + KLD
```

Next we build the model and train

```python
model = VAE_gumbel(temp)
if cuda:
    model.cuda()
optimizer = optim.Adam(model.parameters(), lr=1e-3)
```

```python
kwargs = {'num_workers': 1, 'pin_memory': True} if cuda else {}
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('./data/MNIST', train=True, download=True,
                   transform=transforms.ToTensor()),
    batch_size=batch_size, shuffle=True, **kwargs)
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('./data/MNIST', train=False, transform=transforms.ToTensor()),
    batch_size=batch_size, shuffle=True, **kwargs)
```

    /home/apervez/anaconda3/envs/pytorch1.9_2/lib/python3.7/site-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1623448224956/work/torch/csrc/utils/tensor_numpy.cpp:180.)
      return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)

For training we compute both the relaxed and the unrelaxed objective.
The unrelaxed objective is not itself used for training.
However, since that's the objective we want to improve, it's a good idea to also evaluate it so that we can observe how far the relaxed objective is from the actual objective.
This also allows us to get an idea of how low or high to set the temperature so that the relaxed objective is not too far from the true objective while achieving reasonable training performance.

```python
def train(epoch):
    model.train()
    train_loss = 0
    train_loss_unrelaxed = 0
    #temp = args.temp
    for batch_idx, (data, _) in enumerate(train_loader):
        if cuda:
            data = data.cuda()
        optimizer.zero_grad()
        recon_batch, qy = model(data, temp)
        loss = loss_function(recon_batch, data, qy)
        loss.backward()
        train_loss += loss.item() * len(data)
        optimizer.step()

        #Unrelaxed training objective for comparison
        recon_batch_eval, qy_eval = model(data, temp, evaluate=True)
        loss_eval = loss_function(recon_batch_eval, data, qy_eval)
        train_loss_unrelaxed += loss_eval.item() * len(data)

    print('Epoch: {} Average loss relaxed: {:.4f} Unrelaxed: {:.4f}'.format(
        epoch, train_loss / len(train_loader.dataset) , train_loss_unrelaxed / len(train_loader.dataset)))

```

```python
def test(epoch):
    model.eval()
    test_loss = 0
    for i, (data, _) in enumerate(test_loader):
        if cuda:
            data = data.cuda()
        recon_batch, qy = model(data, temp, evaluate=True)
        test_loss += loss_function(recon_batch, data, qy).item() * len(data)
    test_loss /= len(test_loader.dataset)
    print('Eval loss: {:.4f}'.format(test_loss))

```

Finally we can run the training.
You can try training with different values of the temperature to see how that affects the relaxed objective relative to the true one.

```python
def run():
    for epoch in range(1, epochs + 1):
        train(epoch)
        test(epoch)
run()
```

    /home/apervez/anaconda3/envs/pytorch1.9_2/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
      warnings.warn(warning.format(ret))

    Epoch: 1 Average loss relaxed: 197.1857 Unrelaxed: 199.8391
    Eval loss: 178.9625
    Epoch: 2 Average loss relaxed: 154.2037 Unrelaxed: 163.6063
    Eval loss: 152.1406
    Epoch: 3 Average loss relaxed: 136.7130 Unrelaxed: 148.5179
    Eval loss: 144.5594
    Epoch: 4 Average loss relaxed: 129.4279 Unrelaxed: 141.8772
    Eval loss: 138.8380
    Epoch: 5 Average loss relaxed: 124.0670 Unrelaxed: 137.2079
    Eval loss: 135.5893
    Epoch: 6 Average loss relaxed: 120.2097 Unrelaxed: 133.9328
    Eval loss: 131.6892
    Epoch: 7 Average loss relaxed: 117.3797 Unrelaxed: 131.4627
    Eval loss: 130.6390
    Epoch: 8 Average loss relaxed: 115.1937 Unrelaxed: 129.7081
    Eval loss: 128.6939
    Epoch: 9 Average loss relaxed: 113.3871 Unrelaxed: 128.1377
    Eval loss: 128.0002
    Epoch: 10 Average loss relaxed: 111.9536 Unrelaxed: 127.0920
    Eval loss: 126.7100

### Generating Samples
We can now generate some samples from the trained decoder.
For this we sample some uniform categorical variables from the prior and pass them into the decoder.

```python
def generate_samples():
    #generate uniform probability vector
    model.eval()
    probs = torch.ones([64, latent_dim, categorical_dim])*(1/categorical_dim)
    cat_samples = OneHotCategorical(probs=probs.cuda()).sample().view(-1, latent_dim*categorical_dim)
    output = model.decode(cat_samples)
    return output.view(-1,28,28).detach().cpu().numpy()

```

```python
samples = generate_samples()
```

```python
def show_gray_image_grid(imgs, x=2, y=5, size=(8,8), path=None, save=False):
    fig, axs = plt.subplots(x, y, figsize=size)
    axs = axs.flatten()
    for img, ax in zip(imgs, axs):
        ax.imshow(np.squeeze(img), cmap='gray')
        #ax.imshow(img, cmap='gray')
        ax.set_axis_off()

    if save:
        plt.savefig(path)
    else:
        plt.show()

show_gray_image_grid(samples, 8,8)
```

## Gumbel Straight-Through

In some instances we want to train models with discrete variables but do not want to pass relaxed values as inputs.
This might be the case where we want to optimize a function that cannot be defined for relaxed inputs and must use categorical/discrete inputs.
One heuristic for such cases is the Straight-Through estimator.
Here given some pre-activation $y$ for which we want the gradient we compute the sample $z$ using the non-differentiable sampling operations such as with categorical or Bernoulli sampling.
Then we compute the downstream function $f$ on the hard sample.
Then in the backward pass we ignore the non-differentiable operation and pass the gradient relative to $z$ back as the gradient relative to $y$. That is we set.

$$\partial_y f := \partial_z f.$$

This is the straight-through gradient and gives a biased estimate of the gradient, but in many cases this can often allow us to train models with discrete variables.

Here we use the straight-through gradient with Gumbel-Softmax relaxation.
In this case we compute the Gumbel-Softmax relaxation as before.
From the relaxation we compute a 1-hot vector where the index with the largest value is set to 1 and all others are set to 0.
This is used as the discrete value in the downstream network and we use the straight-through gradient in the backward pass.

Given a hard vector `y_hard` and a soft vector `y` there is a well-known trick that can be used to incorporate straight-through gradients.
We compute

```
y = (y_hard - y).detach() + y
```

which simply uses the `y_hard` in the forward pass but the gradient relative to `y` in the backward pass.

```python
def gumbel_softmax(logits, temperature, evaluate=False, hard=False):
    if evaluate:
        d =  OneHotCategorical(logits=logits.view(-1, latent_dim, categorical_dim))
        return d.sample().view(-1, latent_dim * categorical_dim)

    y = gumbel_softmax_sample(logits, temperature)

    if hard:
        #Straight-through gradient
        #takes the index of the largest and insert a 1.
        #all others are set to 0 obtaining a 1-hot vector.
        shape = logits.size()
        _, k = y.max(-1)

        y_hard = torch.zeros_like(logits)
        y_hard = y_hard.zero_().scatter_(-1, k.view(shape[:-1] + (1,)), 1.0)

        #This a trick to use the 1-hot value in the forward pass and the
        #relaxed gradient in the backward pass
        y = (y_hard - y).detach() + y

    return y.view(-1, latent_dim * categorical_dim)

```

As an exercise you can train the VAE model given above using Gumbel-Straight-Through by using the above function in the model definition and setting `hard=True`.

In the next tutorials we will see how the basic Gumbel relaxation method can be used to differentiably sample other discrete structures such as subsets, permutations and graphs.

## References

[Categorical Reparameterization with Gumbel-Softmax](https://arxiv.org/abs/1611.01144)

[The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://arxiv.org/abs/1611.00712)


# Phipp's Potential
> "Phipp, Conservation of Energy, odeint"

- toc: false
- branch: master
- badges: true
- comments: true
- author: JHM
- categories: [fastpages, jupyter]

Here we examine Phipp's potential $P=\frac{1}{r} \sqrt{1-\frac{r'}{c}}$. It's clear that Phipp's potential imposes an upper bound on the velocities of particles in free fall relative to $P$. But while Phipp's potential guarantees subluminal velocities, it does not appear to satisfy a conservation of energy, and the quantity $T+P$ is not constant along trajectories.

The Phipp potential can induce negative effective mass at small distances, similar to the case of Weber's potential. However the critical radius depends on the relative velocity of the particles. By contrast Weber's critical radius depends only on the mass $m$ and $c^2$.

These two facts are demonstrated in the cells below, and demonstrates some severe disadvantages to Phipp's potential. We are therefore inclined to investigate Weber's potential as a model of electrodynamics.

```python
from sympy import *
t=symbols('t')
m=symbols('m')
c=symbols('c')
r=Function('r')(t)
P=Function('P')(r,t)
F=Function('F')(r,t)

P=(r**-1)*sqrt(1-r.diff(t)/c)
F=(-1)*(P.diff(t))*((r.diff(t))**-1)

pprint(simplify(P))
print()
pprint(simplify(F))  ## symbolic computation of the Force law.
```

          ______________
         ╱     d
        ╱  c - ──(r(t))
       ╱       dt
      ╱    ────────────
    ╲╱          c
    ────────────────────
            r(t)

                                     2
                                    d
                              r(t)⋅───(r(t))
                                     2
    ⎛    d       ⎞ d               dt
    ⎜c - ──(r(t))⎟⋅──(r(t)) + ──────────────
    ⎝    dt      ⎠ dt               2
    ────────────────────────────────────────
             ______________
            ╱     d
           ╱  c - ──(r(t))
          ╱       dt         2    d
     c⋅  ╱    ──────────── ⋅r (t)⋅──(r(t))
       ╲╱          c              dt

```python
## import the basic packages.
import numpy as np
from scipy.integrate import odeint, solve_ivp
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

m=0.5 ## mass of second test particle
c=1.0 ## speed of light constant in Weber's potential

## Kinetic Energy: we use the naive expression for vis viva. Is it correct?
def T(vx, vy, vz):
    return m*(vx*vx+vy*vy+vz*vz)/2

## Phipp's Potential Energy
def P(x,y,z,vx,vy,vz):
    r=(x*x + y*y + z*z)**0.5
    rdot=(x*vx+y*vy+z*vz)/r

    return (r**-1)*(1-rdot/c)**0.5

## Integrating equations of motion relative Phipp's Force Law.

def phipp(t, state):
    x, y, z, vx, vy, vz = state

    r=(x*x + y*y + z*z)**0.5
    rdot=(x*vx+y*vy+z*vz)/r

    A=r**-2
    B=(1-rdot)**0.5

    C=(m-
       (
           2*r*rdot*((1-rdot)**0.5)
       )**-1
       )**-1

    dxdt = vx
    dydt = vy
    dzdt = vz

    dvxdt = (x/r)*A*B*C
    dvydt = (y/r)*A*B*C
    dvzdt = (z/r)*A*B*C

    return [dxdt, dydt, dzdt, dvxdt, dvydt, dvzdt]

t_span = (0.0, 1000.0)
t = np.arange(0.0, 1000.0, 0.001)

y0=[10, 0.3, 0, -0.2, -0.9, .1]  ## initial state of the system y0=[x, y, z, vx, vy, vz]

result = odeint(phipp, y0, t, tfirst=True)

Energy=T(y0[3], y0[4], y0[5] ) + P(y0[0],y0[1],y0[2],y0[3],y0[4],y0[5])

print('The initial total energy T+P is equal to:', Energy)
print('The luminal energy is:', m*c*c/2)
print('The energy is subliminal:', Energy < m*c**2 /2)

r = ( y0[0]*y0[0] + y0[1]*y0[1] + y0[2]*y0[2])**0.5
rdot = ( y0[0]*y0[3] + y0[1]*y0[4] + y0[2]*y0[5] ) /r

print('Phipp`s critical radius is equal to:',(m-
       (
        2*r*rdot*((1-rdot)**0.5)
       )**-1)**-1)
## N.B. Phipp's critical radius depends on the distance and velocity!

print('The initial radius r and velocity r` is within Phipp`s critical distance:', r < (m*c*c)**-1      )

fig = plt.figure()
ax = fig.add_subplot(1, 2, 1, projection='3d')
ax.plot(result[:, 0],
        result[:, 1],
        result[:, 2])
ax.set_title("position")

ax = fig.add_subplot(1, 2, 2, projection='3d')
ax.plot(result[:, 3],
        result[:, 4],
        result[:, 5])
ax.set_title("velocity")

```

```python
import matplotlib.pyplot
import pylab

def rho(x,y,z):
  return (x*x+y*y+z*z)**0.5

v_list=[]
for j in range(4000):
  v_list.append(
      (rho(result[j,0], result[j,1], result[j,2]),
       T(result[j,3], result[j,4], result[j,5]))
         )

P_list=[]
for j in range(4000):
    P_list.append(
        (rho(result[j,0], result[j,1], result[j,2]),
       P(result[j,0],result[j,1],result[j,2],result[j,3],result[j,4],result[j,5])+T(result[j,3], result[j,4], result[j,5]))
    )

prelist1 = list(zip(*v_list))
pylab.scatter(list(prelist1[0]),list(prelist1[1]))
pylab.show()

prelist2 = list(zip(*P_list))
pylab.scatter(list(prelist2[0]),list(prelist2[1]))
pylab.show()
```

```python

```


# mobrob - A4

Implementieren Sie eine Beacon-Navigation unter Verwendung des Kalman-Filter; benutzen Sie hierfür ebenfalls den differentiellen Roboter.
Es wird ein "Kompass" benötigt. Der Kompass liefert einen Winkel relativ zur X-Achse des Welt-KS, die Messung ist verrauscht.
Desweiteren muss ein Bildsensor implementiert werden, der Markierungen (Beacons) aus dem Bild detektiert.
Die Beacons sind an bekannten Orten im Raum angebracht ($X_{Li}$, $Y_{Li}$).
Gemessen wird jeweils der Winkel (Azimuth) relativ zum Rob.-KS für ein zufällig ausgewähltes Beacon, die Winkelmessung ist ebenfalls verrauscht!
Da die Identität der Beacons nicht ermittelt werden kann, ist die Verwendung eines "innovation gates" notwendig.
Sollte ein Match nicht eindeutig möglich sein, wird die Messung verworfen.

## Ablauf

*   Der Roboter bewegt sich, aktualisiert Positionsschätzung und deren Kovarianz anhand des Odometriemodells
($X_{k,k−1}$).
*   Gemessen wird jeweils der Winkel zu einem zufällig ausgewählten Beacon. Der Kalman-Filter
aktualisiert die Schätzung ($X_{k,k}$,$C_{k,k}$).
*   Basisbreite (Abstand der Räder): $b$
*   Geschwindigkeiten der Räder: $\mathbf{v} = \left( v_L , v_R\right)^{\text{T}}$
*   Zustand (Pose im Weltkoordinatensystem): $\mathbf{X}_k = (x_k , y_k , \theta_k )^\text{T}$
*   Zustandsschätzung: $\mathbf{X}_{k,k} \overset{\text{Korrektur(Kalman)}}{\longleftarrow} \mathbf{X}_{k,k-1} \overset{\text{Vorhersage(Odom.-Modell)}}{\longleftarrow} \mathbf{X}_{k-1,k-1}$
*   Kovarianz der Positionsschätzung: $\mathbf{C}_{\text{P},k,k}$

## Beacon-Peilung

Messgleichung: Ausgewählt wird Beacon $i$ aus der Karte $\alpha_k^i=\arctan\left({\frac{Y_{Li}-Y_k}{X_{Li}-X_k}}\right)-\theta_k - v_k^\text{P} = h^{\text{P}i}(\mathbf{X}_k,v_k^\text{P})$ mit $v_k^\text{P} \sim \mathcal{N}(0,\sigma_\text{P}^2)$

## Kompass

Messgleichung: $\beta_k = \theta_k + v_k^\text{K} = h^\text{K} (\theta_k , v_k^\text{K})$ mit $v_k^\text{K} \sim \mathcal{N}(0,\sigma_\text{K}^2)$

# Lösung

Dies sind die Lösungsschritte der Lokalisation anhand des Kalman-Filters mit Innovation-Gate.

## Odometriegleichung

Verfahren nach Euler-Colaz für einen Roboter mit differentialer Lenkung:

\begin{align}
\mathbf{X}_{k+1} =
  \begin{pmatrix}
    X_k\\
    Y_k\\
    \theta_k
  \end{pmatrix}
  +
  \begin{pmatrix}
    \frac{s_{Lk} + s_{Rk} }{2} \cos \left( \theta_k + \frac{s_{Rk} -s_{Lk}}{2b}\right)\\
    \frac{s_{Lk} + s_{Rk} }{2} \sin \left( \theta_k + \frac{s_{Rk} -s_{Lk}}{2b}\right)\\
    \frac{s_{Rk} - s_{Lk} }{b}
  \end{pmatrix}
  \text{ mit } s_{Lk}=v_{Lk}\Delta t \text{ und } s_{Rk}=v_{Rk}\Delta t
\end{align}

Fehlerfortpflanzung der Odometrie:

\begin{align}
\mathbf{C}_{\text{P},k+1,k} = \mathbf{F}_{\text{P},k+1} \mathbf{C}_{\text{P},k,k} \mathbf{F}_{\text{P},k+1}^{\text{T}} + \underbrace{\mathbf{F}_{\text{S},k+1} \mathbf{C}_{\text{S},k+1} \mathbf{F}_{\text{S},k+1}^{\text{T}}}_{\mathbf{Q}_{k+1}}
\end{align}

Notiz:

$\mathbf{F}_{\text{P},k+1}$ ist die Jacobi-Matrix nach der Position $\mathbf{X}$ im Arbeitspunkt $k+1$ mit $\mathbf{F}_{\text{P},k+1} = \frac{\partial\mathbf{X}_{k+1}}{\partial\mathbf{X}}$

$\mathbf{F}_{\text{S},k+1}$ ist die Jacobi-Matrix nach der Geschwindigkeit $\mathbf{v}$ im Arbeitspunkt $k+1$ mit $\mathbf{F}_{\text{S},k+1} = \frac{\partial\mathbf{X}_{k+1}}{\partial\mathbf{v}}$

$\mathbf{C}_{\text{S},k+1}$ ist die Unsicherheit der zurückgelegten Wegstrecke, welche mit der Geschwindigkeit $\mathbf{v}$ skaliert (Die Konstanten $(k_1,k_2)$ müssen messtechnisch ermittelt werden) mit $\mathbf{C}_{\text{S},k+1}=\text{diag}\left(k_1 \left| s_{Lk+1}\right|, k_2 \left| s_{Rk+1}\right| \right)$

$\mathbf{Q}_{k+1}$ wird als die verkürzte Schreibweise für die Unsicherheit der Wegänderung verwendet, mit $\mathbf{Q}_{k+1} = \mathbf{F}_{\text{S},k+1} \mathbf{C}_{\text{S},k,k} \mathbf{F}_{\text{S},k+1}^{\text{T}}$

## Sensordaten Einlesen

Diese Messgleichungen dienen der Simulation der Messungen zum Zeitpunkt $k+1$. Die Werte, die mit einem $'$ versehen sind, sind demnach ground truth Werte vom Simulator.

### Beacon Messung

Sensormesswert wird von einem zufälligen Beacon $i$ unter dem Winkel $\alpha^i_{k+1}$ gemessen eingelesen:

\begin{align}
  &\tilde{z}^\text{P}_{k+1} = \alpha^i_{k+1} = \arctan\left({\frac{Y'_{Li}-Y'_{k+1}}{X'_{Li}-X'_{k+1}}}\right)-\theta'_{k+1} - v_{k+1}^\text{P} = h^{\text{P}i}(\mathbf{X}'_{k+1},v_{k+1}^\text{P})
\end{align}

mit $v_{k+1}^\text{P} \sim \mathcal{N}(0,\sigma_\text{P}^2)$.

und $i \sim$ Menge aller Beacons.

### Kompass Messung

Die Kompassmessung ist eine verrauschte Messung der ground truth Daten.

\begin{align}
  &\tilde{z}^\text{K}_{k+1} = \beta_{k+1} = \theta'_{k+1} + v_{k+1}^\text{K} = h^{\text{K}i}(\mathbf{X}'_{k+1},v_{x+1}^\text{K})
\end{align}

mit $v_{k+1}^\text{K} \sim \mathcal{N}(0,\sigma_\text{K}^2)$

## Berechnung einer Messvorhersage für alle Beacons

Hier müssen nun die Hypothesen $\hat{z}_{i,k+1}$ berechnet werden, unter welchem Winkel der Roboter an der Position $\mathbf{X}_{k+1}$ das Beacon $i$ messen kann.

\begin{align}
  \hat{z}_{i,k+1} = \arctan\left({\frac{Y_{Li}-Y_{k+1}}{X_{Li}-X_{k+1}}}\right)-\theta_k = h^{i}(\mathbf{X}_{k+1},0)\\
\end{align}

## Kalman-Gain

Der Kalman-Gain wird anhand eines Innovation-Gates berechnet.

### Berechnung der Jakobi-Matrix der Messgleichungen:

\begin{align}
  \mathbf{H}^i_{k+1} = \left. \frac{\partial (h^{\text{P}i}, h^{\text{K}i})^{\text{T}}}{ \partial \mathbf{X} }\right|_{\mathbf{X}_{k+1}}
  =
  \left(
  \begin{matrix}
      \frac{ Y_{Li}-Y_{k+1} }{(X_{Li}-X_{k+1})^2 + (Y_{Li}-Y_{k+1})^2}  &  \frac{ X_{Li}-X_{k+1} }{(X_{Li}-X_{k+1})^2 + (Y_{Li}-Y_{k+1})^2} & -1\\
      0 & 0 & 1
  \end{matrix}
  \right)
\end{align}

\begin{align}
  \mathbf{V}_{k+1} = \left. \frac{\partial (h^{\text{P}i}, h^{\text{K}i})^{\text{T}}}{ \partial \left( v^{\text{P}}, v^{\text{K}} \right)^{\text{T}} }\right|_{\mathbf{X}_{k+1}}
  =
  \left(
  \begin{matrix}
      -1 & 0\\
      0 & 1
  \end{matrix}
  \right)
\end{align}

### Berechnung der Fehlerfortpflanzung der Messgleichung

Angabe der Unsicherheitsmatrix $\mathbf{N}_{k+1}$, welche vom System vorgegeben ist:

\begin{align}
  \mathbf{N}_{k+1} = \mathbf{N} =
  \left(
  \begin{matrix}
      \sigma_{\text{P}}^2 & 0\\
      0 & \sigma_{\text{K}}^2
  \end{matrix}
  \right) = \text{const.}
  \forall k
\end{align}

Fehlerfortpflanzung:

\begin{align}
  \mathbf{R}_{k+1} = \mathbf{V}_{k+1} \mathbf{N} \mathbf{V}_{k+1}^{\text{T}}
  =
  \left(
  \begin{matrix}
      \sigma_{\text{P}}^2 & 0\\
      0 & \sigma_{\text{K}}^2
  \end{matrix}
  \right)
  = \text{const.}
  \forall k
\end{align}

Notiz:

Die Unsicherheiten des Rauschens gehen additiv in die Messterme ein.
Somit bleibt auch das Rauschen zwischen dem Kompass und der Beacon-Peilung zum einen immer unkorreliert, und zum anderen konstant über die Zeit.

### Innovation Gate

Definition der Innovation:
\begin{align}
    \nu_{i,k+1} =
    \left(
    \begin{matrix}
        \nu^{\text{P}}_{i,k+1}\\
        \nu^{\text{K}}_{i,k+1}
    \end{matrix}
    \right)
    =
    \left(
    \begin{matrix}
        \tilde{z}^{\text{P}}_{k+1} - \hat{z}_{i,k+1}\\
        \tilde{z}^{\text{K}}_{k+1} - \theta_{k+1}
    \end{matrix}
    \right)
\end{align}

Berechnung der Kovarianz der Innovation für das Beacon $i$:

\begin{align}
    \mathbf{C}_{\nu_{i,k+1}} = \mathbf{H}^i_{k+1}  \mathbf{C}_{\text{P},k+1,k}  \mathbf{H}^{i\text{T}}_{k+1} + \mathbf{R}_{k+1}
\end{align}

Vergleich der Innovation einer Messgleichung vom Beacon $i$ zu einem festen, definierten Gate-Wert $g$:

\begin{align}
    \nu_{i,k+1}^{\text{T}} \mathbf{C}_{\nu_{i,k+1}}^{-1} \nu_{i,k+1} \le g^2
\end{align}

Falls die Gleichung für einen Beacon $i$ wahr ist, so wird dieser der Menge $\mathcal{M}_{k+1}$ hinzugefügt

### Kalman-Gain

Falls anhand des Innovation-Gates ein einziger Match $\mathcal{M}_{k+1}$ gefunden wurde, wird der Kalman-Gain berechnet.
Wird kein oder mehrer Matches $\mathcal{M}_{k+1}$ gefunden, wird der Kalman-Gain verworfen.

\begin{align}
    \mathbf{K}_{k+1} = \left\lbrace
    \begin{matrix}
        \mathbf{C}_{\text{P},k+1,k} \mathbf{H}^{i\text{T}}_{k+1}  \mathbf{C}_{\nu_{i,k+1}}^{-1} & \text{falls }|\mathcal{M}_{k+1}| = 1\\
        \mathbf{0}  & \text{falls }|\mathcal{M}_{k+1}| \neq 1
    \end{matrix}
    \right.
\end{align}

## Aktualisierung der Position und Unsicherheit

Falls das Matching eines Beacons $i$ anhand des Innovation-Gates erfolgreich war, so wird die Sensormessung beim Update berücksichtigt.
Falls es jedoch nicht erfolgreich war, so wird die Messung verworfen und lediglich die Odometrie verwendet.

### Update: Position

\begin{align}
    \mathbf{X}_{k+1,k+1} = \mathbf{X}_{k+1,k} + \mathbf{K}_{k+1} \nu_{i,k+1}
\end{align}

### Update: Kovarianz

\begin{align}
    \mathbf{C}_{\text{P},k+1,k+1} = \mathbf{C}_{\text{P},k+1,k} - \mathbf{K}_{k+1} \mathbf{C}_{\nu_{i,k+1}} \mathbf{K}_{k+1}^{\text{T}}
\end{align}

## Iteration

Setze nun $k+1 \rightarrow k$ und beginne wieder im Odometrieschritt.