# Music Machine Learning - feature-based mining (nearest neighbors)

### Author: Philippe Esling (esling@ircam.fr)

In this course we will cover
1. A [quick recap](#recap) on the idea of feature-based learning
2. An introduction to [Nearest Neighbors](#nn) for querying
3. An implementation of [classification](#classification) using k-NN
4. A discussion on the [evaluation](#evaluation) of classification results

## Feature-based learning

This tutorials corresponds to the same slides following the introductory developments performed in the [previous tutorial](01a_machine_learning.ipynb). As we have seen previously, we can compute a whole set of features out of audio waveforms. These features were used in the traditional approaches of machine learning, which performed _feature-based learning_. If we look at the space induced by these features, we can clearly see that similar examples are somewhat grouped together.

Therefore, a naive way to approach the problem would be to look at examples in our training set that are near a new (unseen) file. Based on the features computed, we will implement a simple *querying* and *classification* system based on [Nearest Neighbors](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm).

```python
import numpy as np
from matplotlib import pyplot as plt
from matplotlib import gridspec
from mpl_toolkits.mplot3d import Axes3D
```

## Nearest-neighbors

In this tutorial, we will cover the simplest querying and classification algorithm, namely the *$k$-Nearest Neighbor* method. The idea is for a given (usually *unknown*) element, to find its closest neighbor(s), based on the distances between this element and the known dataset for a given set of features. Formally, given a set of elements $e_{i}$, $i\in\left[1,N\right]$ and their corresponding features $\mathbf{f_{i,m}}\in\mathbb{R}^{d}$ (which denotes the $m^{th}$ feature of the $i^{th}$ element, which can be $d$-dimensional), we will need to compute a distance measure $\mathcal{D}\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)$ between the features of elements of the dataset. This distance will express the dissimilarity between two features. For the first two questions of the tutorial, we will simply consider that the dissimilarity between features is expressed by their Euclidean $(l_{2})$ distance.

$$
\begin{equation}
\mathcal{D}\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)=\sqrt{\sum_{d=1}^{D}\left(f_{i,m}^{d}-f_{j,m}^{d}\right)^{2}}
\end{equation}
$$

Given distances for each feature, we need to *merge* these various dissimilarities and then select the nearest neighbor of a particular element by computing

$$
\begin{equation}
kNN\left(e_{i}\right)=\overset{k}{\underset{j\neq i}{argmin}}\left(\frac{1}{M}\sum_{m=1}^{M}\left(\mathcal{D}\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)\right)\right)
\end{equation}
$$

***

**Questions**

  1. What problems can arise from $n$-dimensional audio features?
  2. Based on the selection equation, what constraints are implicitly made on the distances?
  3. Does the Euclidean distance seems like a sound way to measure the similarity between temporal features?

***

```python
from helper_data import import_dataset, compute_transforms, compute_features
class_path = 'data/classification'
# 0.1 - Import the classification dataset
data_struct = import_dataset(class_path, 'classification')
# 0.2 - Pre-process the audio to obtain spectral transforms
data_struct = compute_transforms(data_struct)
# 0.3 - Compute a set of temporal and spectral features
data_struct = compute_features(data_struct)
```

## Querying

First, we can use the nearest-neighbor idea to devise a very simple *querying* system. This type of method is typically used in many systems such as *Query By Humming (QBH)* softwares (similar to [Shazam](http://www.shazam.com/)). As previously, we provide a baseline code in the main script. First, we create a $ N \times M $ distance matrix `data_matrix` corresponding to the $M$ features of the $ N $ elements of the datasets. We selected here only the *spectral_centroid_mean, SpectralFlatnessMean* and *SpectralSkewnessMean* features.

```python
used_features = ['spectral_centroid_mean', 'loudness_mean', 'spectral_contrast_mean']
# Create a data matrix only from selected features
data_matrix = np.zeros((len(data_struct["filenames"]), len(used_features)))
for f in range(len(used_features)):
    data_matrix[:, f] = data_struct[used_features[f]]
# Avoid NaN and inf values
data_matrix = np.nan_to_num(data_matrix)
# Normalize the matrix (unit-variance)
for f in range(len(used_features)):
    data_matrix[:,f] = (data_matrix[:, f] - np.mean(data_matrix[:, f])) / np.std(data_matrix[:, f])
```

Then, after your code is filled, the `dist` matrix should contain the mean distances (eventually, for various types of distances), which will then be sorted to the `nn_ids` vector allowing to select the nearest neighbors.

***

**Exercise**

  1. Compute the set of distances between a random element and the rest of the dataset.
  2. Complete the plotting code in order to plot the element and its 10 nearest neighbors.
  3. Check that you obtain plots similar to those displayed below.

***

```python
from helper_plot import hdr_plot_style
hdr_plot_style()
# Create a vector of colors
color_vect = plt.cm.inferno_r.colors[1::10]
# Q 1.1.1 - Perform some 10-NN queries and plot the result
for test in range(10):
    select_id = int(np.random.randint(low=0, high=len(data_struct["filenames"]), size=1))
    select_mask = [True] * len(data_struct["filenames"])
    select_mask[select_id] = False
    tmp_copy = data_matrix[select_mask]
    tmp_class = data_struct["classes"][select_mask]

    ######################
    # YOUR CODE GOES HERE
    ######################

    # Performing a slightly complicated plot
    fig = plt.figure(figsize=(12,8))
    ax = fig.add_subplot(111, projection='3d')
    for c in range(len(data_struct["class_names"])):
        cur_points = (data_struct["classes"] == c);
        curColor = color_vect[c]
        s = ax.scatter(xs=data_matrix[cur_points, 0], ys=data_matrix[cur_points, 1], zs=data_matrix[cur_points, 2], s=80, c=np.array(curColor).reshape(1, -1));
    s = ax.scatter(tmp_copy[nn_ids[:10], 0], tmp_copy[nn_ids[:10], 1], tmp_copy[nn_ids[:10], 2], s=100, linewidths=6, edgecolor='w');
    s.set_edgecolors = s.set_facecolors = lambda *args:None
    s = ax.scatter(data_matrix[select_id, 0], data_matrix[select_id, 1], data_matrix[select_id, 2], s=200, linewidths=6, edgecolor='w');
    for t in range(10):
        fD = (tmp_copy[nn_ids[t], :] - data_matrix[select_id, :]) * 4;
        s = ax.text(tmp_copy[nn_ids[t], 0] + fD[0], tmp_copy[nn_ids[t], 1] + fD[1], tmp_copy[nn_ids[t], 2] + fD[2], data_struct["class_names"][tmp_class[nn_ids[t]]], fontsize=22, fontweight='bold', horizontalalignment='center');
        s.set_edgecolors = s.set_facecolors = lambda *args:None; s.set_zorder(200)
        lines = ax.plot([tmp_copy[nn_ids[t], 0], tmp_copy[nn_ids[t], 0] + (fD[0] * 0.8)], [tmp_copy[nn_ids[t], 1], tmp_copy[nn_ids[t], 1] + (fD[1] * 0.8)], [tmp_copy[nn_ids[t], 2], tmp_copy[nn_ids[t], 2] + (fD[2] * 0.8)], linewidth=2, c='w')
        for l in lines:
            l.set_zorder(199)
    ax.set_xlabel('Centroid'); ax.set_xlim(-2, 2); ax.set_ylabel('Loudness'); ax.set_ylim(-2, 3); ax.set_zlabel('Contrast'); ax.set_zlim(-3, 2);
    ax.set_title('True class : ' + data_struct["class_names"][data_struct["classes"][select_id]]);
    ax.dist = 10
    plt.show()
```

#### Going further

Of course, based on the types of features, different distance measures can be used, such as any of the $l_{p}$ distances (a generalization of the Euclidean distance)

$$
l_{p}\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)=\sqrt[p]{\sum_{d=1}^{D}\left|f_{i,m}^{d}-f_{j,m}^{d}\right|^{p}}
$$

the *Cosine* distance

$$
cosine\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)=1-\frac{\mathbf{f_{i,m}}\cdot\mathbf{f_{j,m}}}{\left\Vert \mathbf{f_{i,m}}\right\Vert \left\Vert \mathbf{f_{j,m}}\right\Vert }=1-\frac{\sum_{d=1}^{D}f_{i,m}^{d}f_{j,m}^{d}}{\sum_{d=1}^{D}\left(f_{i,m}^{d}\right)^{2}\sum_{d=1}^{D}\left(f_{j,m}^{d}\right)^{2}}
$$

or the *correlation* distance.

$$
correlation\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)=1-\frac{\mathbf{\left(f_{i,m}-\mu_{\mathbf{f_{i,m}}}\right)}\cdot\left(\mathbf{f_{j,m}}-\mu_{\mathbf{f_{j,m}}}\right)}{\left\Vert \mathbf{f_{i,m}-\mu_{\mathbf{f_{i,m}}}}\right\Vert \left\Vert \mathbf{f_{j,m}}-\mu_{\mathbf{f_{j,m}}}\right\Vert }
$$

The same observation holds for the way we decided to "merge" the different distances. By looking at these given definitions, start by thinking about the following questions.

***

**Exercise**

  4. Implement the $l_{p}$, *Cosine* and *correlation* distances
  5. Try the same piece of code by varying the distances and the `used_features`.
  6. What can you tell about the discriminative power of features?
  7. What other steps should we perform on the features?
  8. (Optional) Extend your code to include temporal features
  9. (Optional) Extend your code to a fully functional *Query-By-Example* (QBE) system.

***

```python
######################
# YOUR CODE GOES HERE
######################
```

## Classification

For the second part of this tutorial, we will rely on the same technique (computing the distance of a selected point to the rest of the dataset) in a *classification* framework. The overarching idea behind $k$-NN classification is that elements from a same class should have similar properties in the *feature space*. Hence, the closest feature to those of an element should be from elements of its right class. These types of approaches are usually termed as *distance-based* classification methods. The skeleton for this algorithm is provided in the `knn_classify` function.

```Python
def knn_classify(data_struct, test_sample, k, normalize, useL1dist):
    """
    % This function is used for classifying an unknown sample using the kNN
    % algorithm, in its multi-class form.
    %
    % Arguments :
    % - data_struct  : the data structure
    % - test_sample  : the input sample id to be classified
    % - k           : the k (number of neighbors) parameter
    % - dist_type    : type of distance ['Euclidean', 'Manhattan', 'Cosine'] (optional)
    % - normalize   : use class priors to weight results (optional)
    % Returns :
    % - probas      : an array that contains the classification probabilities for each class
    % - winner_class : the label of the winner class
    """
    return probas, winner_class
```

The algorithm will globally look quite similar to the previous one. However, this time we will compute the $k$ Nearest Neighbors *for each of the classes separately*, which will allow to consider the resulting distance vectors as probabilities.

Hence, we will compute for the set of classes $\mathcal{C}_{t}$ the vector of distances, and select the $k$ closest elements per class.

$$
\begin{equation}
kNN_{\mathcal{C}_{t}}\left(e_{i}\right)=\overset{k}{\underset{j\in\mathcal{C}_{t} \wedge j\neq i}{argmin}}\left(\frac{1}{M}\sum_{m=1}^{M}\left(\mathcal{D}\left(\mathbf{f_{i,m}},\mathbf{f_{j,m}}\right)\right)\right)
\end{equation}
$$

Then, in order to consider the distances as probabilities, we compute for each class the mean distance of its $k$ nearest neighbors and normalize these distances across classes

$$
p_{\mathcal{C}_{t}}\left(e_{i}\right)=\frac{1}{k}\sum_{j=1}^{k}kNN_{\mathcal{C}_{t}}\left(e_{i}\right)
$$

In the `knn_classify` function, we store in `testFeatures` the vector of features from the element we are trying to classify, and construct a cell of features for each class in the `class_feats` cell.

***

**Exercise**

  1. Update the `knn_classify` code to perform the basic k-NN classification function
  2. Run the algorithms for both 1-NN and 5-NN evaluation
  3. Plot the different confusion matrix to visually check the accuracies (you should obtain the values displayed in the following figure).
  4. Extend the code to take various distances into account (argument `dist_type`)
  5. What is the use of "class weighting" (argument `normalize`)?
  6. Implement the class weighting system and evaluate its effect
  4. Perform the same classification with various K and features to evaluate the properties and qualities of different parametrizations.
  5. (Optional) Automatize the evaluation of various configurations.

***

```python
#%%
# 1.2 - K-Nearest Neighbors classification based on static properties
#
nb_classes = len(data_struct["class_names"])
nb_files = len(data_struct["filenames"])
correct_elements_5NN = 0
confusion_matrix_5NN = np.zeros((nb_classes, nb_classes))
probas_matrix_5NN = np.zeros((nb_files, nb_classes))

#
# function [probas, winner_class] = knn_classify(data_struct, test_sample, k, normalize, useL1dist);
#
def knn_classify(data_struct, test_sample, usedFeatures, k=1, dist_type='Euclidean', normalize=0):
    # Keep the number of classes
    n_classes = len(data_struct["class_names"]);
    # Initialization of class properties
    n_dims = np.zeros(n_classes)
    n_samples = np.zeros(n_classes)
    class_feats = [];
    # Create a data matrix with the features of all data
    data_matrix = np.zeros((len(data_struct["filenames"]), len(usedFeatures)))
    for f in range(len(usedFeatures)):
        data_matrix[:, f] = np.array(data_struct[used_features[f]]);
        # Remove NaN and Inf
        data_matrix[:, f] = np.nan_to_num(data_matrix[:, f])
        data_matrix[:, f] = data_matrix[:, f] / np.max(data_matrix[:, f])
    # Retrieve the features of the test samples
    testFeatures = data_matrix[test_sample, :]
    # dist{i} will the vector of distance of the testing sample to all the samples of i-th class
    dist = []
    # We create a vector of "inf" distances for each class and a cell of features
    for i in range(n_classes):
        # Retrieve the class points
        cur_points = np.linspace(0, len(data_struct["classes"])-1, len(data_struct["classes"]), dtype=int)[data_struct["classes"] == i]
        # Eventually remove the current sample
        cur_points = cur_points[cur_points != test_sample];
        n_samples[i] = len(cur_points);
        n_dims[i] = len(usedFeatures);
        # Fill the dist cell with "inf" values
        dist.append(np.inf * np.ones(int(np.max(n_samples))));
        # Keep the cell of class features
        class_feats.append(data_matrix[cur_points, :])
        class_feats[i] = np.nan_to_num(class_feats[i])
    # Compute the distance vectors for each class
    for i in range(n_classes):

        #################
        # YOUR CODE GOES HERE
        #################

    kAll = np.zeros((n_classes, 1))
    # Compute the mean distance value for k neighbors
    for j in range(k):

        #################
        # YOUR CODE GOES HERE
        #################

    # Normalize the class probabilities

    #################
    # YOUR CODE GOES HERE
    #################

    # Retrieve the winning class

    #################
    # YOUR CODE GOES HERE
    #################

    return probas, winner_class

# Q 1.2.1 Perform 1-NN and 5-NN classification
for f in range(nb_files):
    # Perform 5-NN classification
    probas, winClass = knn_classify(data_struct, f, ['spectral_centroid_mean', 'loudness_mean', 'spectral_contrast_mean'], 5);
    if (winClass == data_struct["classes"][f]):
        correct_elements_5NN = correct_elements_5NN + 1;
    confusion_matrix_5NN[data_struct["classes"][f], winClass] = confusion_matrix_5NN[data_struct["classes"][f], winClass] + 1;
    probas_matrix_5NN[f, :] = probas[0, :];
# Q 1.2.3 - Plotting the confusion matrices
plt.figure(figsize=(10, 8))
plt.imshow(confusion_matrix_5NN)
#plt.set_cmap(plt.jet)
plt.title('1-NN Confusion')
for c in range(confusion_matrix_5NN.shape[0]):
    plt.text(c, c, '%.3f'%(confusion_matrix_5NN[c, c] / np.sum(confusion_matrix_5NN[c, :])), fontsize=22, fontweight='bold', color=[1, 0, 0], horizontalalignment='center');
plt.yticks(range(len(data_struct["class_names"])), data_struct["class_names"])
plt.xticks(range(len(data_struct["class_names"])), data_struct["class_names"]);
```

## Evaluation

When proposing machine learning algorithms, the fundamental aspects lies in correctly evaluating their performances. Depending on the application, methods, dataset and even the nature of corresponding data, a plethora of evaluation measures can be used. We highly recommend the following articles for those interested in machine learning, so that you develop your critical mind and do not limit yourself to narrow evaluations (by relying on statistical tests) and also that you avoid **cherry picking**

  * Demšar, J. (2006).   *Statistical comparisons of classifiers over multiple data sets.*   The Journal of Machine Learning Research, 7, 1-30.   [PDF](http://machinelearning.wustl.edu/mlpapers/paper_files/Demsar06.pdf)
  * Sturm, B. L. (2013).   *Classification accuracy is not enough.*   Journal of Intelligent Information Systems, **41**(3), 371-406. [PDF](http://vbn.aau.dk/files/70797941/Sturm20121030.pdf)
  * Keogh, E., & Kasetty, S. (2003).   *On the need for time series data mining benchmarks: a survey and empirical demonstration.*   Data Mining and knowledge discovery, **7**(4), 349-371. [PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.2240&rep=rep1&type=pdf)

However, for the scope of this tutorial, we will limit ourselves to typical measures minimally required to evaluate your classifier. Overall, the most important aspects of evaluation lies in different ways of comparing the *real labels* (ground truth) to the *assigned labels* (picked by the classifier).

  * The **confusion matrix** is computed simply by counting the occurences in which a particular instance of a real label (row) is classified to an assigned label (column). This code is already provided in the starter code, and all the following measures can be derived directly from it.
  * The **overall accuracy** is computed as the ratio of correctly classified examples divided by the complete number of examples.
  * The (per-class) **precision** defines the ratio of examples correctly assigned to a class divided by the number of instances assigned to that class by the classifier.
  * The (per-class) **recall** defines the ratio of examples correctly assigned to a class divided by the number of instances really belonging to that class.
  * The **F1 measure** is defined as the ratio between the geometric and harmonic means between the precision and recall measures.

You can implement these measures by simply completing the starter code. If you have doubts about the implementation of these measures, you can check the corresponding [Wikipedia article](https://en.wikipedia.org/wiki/Precision_and_recall)

***

**Exercise**

  1. Implement the *accuracy*, *recall*, *precision* and *F1 measure*
  2. Evaluate the previous algorithms with your new measures.
  3. Perform an automatization of the previous evaluations.
  4. Run the automatic evaluation for different K, distances and features.
  5. Plot the corresponding results for comparison

***

```python
#%%
# 1.3 - Evaluating classification accuracy with various measures
#
# Type of normalization
normalizationMode = 2
# number of classes
nClasses = confusion_matrix_1NN.shape[0]
# confusion matrix normalization:

######################
# YOUR CODE GOES HERE
######################
if (normalizationMode == 1):
    print('Standard normalization')
elif (normalizationMode == 2):
    print('Row-wise normalization')
else:
    print('No normalization')
```

```python
# compute overal accuracy
# compute class precision
# compute class recall
# compute class F1 measure

maxK = 10
classPrecisions = np.zeros((maxK, nClasses))
classRecalls = np.zeros((maxK, nClasses))
# Q 1.3.4 - Evaluate per-class precisions and recalls for various K
for k in range(maxK):
    confusionMatrix = np.zeros(nb_classes)

    ######################
    # YOUR CODE GOES HERE
    ######################
```

```python
#%% Plot the evaluations for different K

######################
# YOUR CODE GOES HERE
######################
```


# Vertical Line Test

## 1.1 Create two graphs, one that passes the vertical line test and one that does not.

```
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
matplotlib.style.use('ggplot')

theta = np.linspace(0, 2*np.pi, 100)

r = np.sqrt(1.0)

x1 = r*np.cos(theta)
x2 = r*np.sin(theta)

fig, ax = plt.subplots(1)

ax.plot(x1, x2)
ax.set_aspect(1)

plt.xlim(-1.25,1.25)
plt.ylim(-1.25,1.25)
xcoords = [0.3]
# colors for the lines
colors = ['k']

for xc,c in zip(xcoords,colors):
    plt.axvline(x=xc, label='line at x = {}'.format(xc), c=c)

plt.grid(linestyle='--')

plt.title('Does Not Pass Verticle Line Test', fontsize=8)

plt.savefig("plot_circle_matplotlib_01.png", bbox_inches='tight')

plt.show()
```

```
import matplotlib.pyplot as plt

vector_1 = [1, 2]
vector_2 = [2, 0]
xcoords = [0.5]
colors = ['b']

for xc,c in zip(xcoords,colors):
    plt.axvline(x=xc, label='line at x = {}'.format(xc), c=c)

plt.arrow(0,0, vector_1[0], vector_1[1],head_width=.05, head_length=0.05, color ='red')
plt.arrow(0,0, vector_2[0], vector_2[1],head_width=.05, head_length=0.05, color ='green')
plt.xlim(-1,3)
plt.ylim(-1,3)
plt.title("Passes the Verticle Line Test")
plt.show()
```

## 1.2 Why are graphs that don't pass the vertical line test not considered "functions?"

They are not considered functions because the vector lines lie on top of the other.

# Functions as Relations

## 2.1 Which of the following relations are functions? Why?

\begin{align}
\text{Relation 1: } \{(1, 2), (3, 2), (1, 3)\}
\\
\text{Relation 2: } \{(1, 3), (2, 3), (6, 7)\}
\\
\text{Relation 3: } \{(9, 4), (2, 1), (9, 6)\}
\\
\text{Relation 4: } \{(6, 2), (8, 3), (6, 4)\}
\\
\text{Relation 5: } \{(2, 6), (2, 7), (2, 4)\}
\end{align}

2 because its the only one that does not lay on top of the others

# Functions as a mapping between dimensions

## 3.1 for the following functions what is the dimensionality of the domain (input) and codomain (range/output)?

\begin{align}
m(𝑥_1,𝑥_2,𝑥_3)=(x_1+x_2, x_1+x_3, x_2+x_3)
\\
n(𝑥_1,𝑥_2,𝑥_3,𝑥_4)=(x_2^2 + x_3, x_2x_4)
\end{align}

Function m has a 3 dimensional input to a 3 dimensiona output.

Function n has a 4 dimensional input to a 2 dimensional output.

## 3.2 Do you think it's possible to create a function that maps from a lower dimensional space to a higher dimensional space? If so, provide an example.

I do not think that a function maps a lower dimensions space to a higher dimensionsal space.

m(x_1,x_2) = (x_1+x_2,x_1+2,x_2+3)

# Vector Transformations

## 4.1 Plug the corresponding unit vectors into each function. Use the output vectors to create a transformation matrix.

\begin{align}
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\2 x_2 - x_1 \\  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\2 x_2 - x_1 + 3x_3 \\ 5x_1 - 2x_3 + x_2  \end{bmatrix}
\end{align}

```

```

## 4.2 Verify that your transformation matrices are correct by choosing an input matrix and calculating the result both via the traditional functions above and also via vector-matrix multiplication.

```

```

# Eigenvalues and Eigenvectors

## 5.1 In your own words, give an explanation for the intuition behind eigenvalues and eigenvectors.

```
eigenvectors are the “axes” of the transformation represented by the matrix
```

# The Curse of Dimensionality

## 6.1 What are some of the challenges of working with high dimensional spaces?

The callenges of working with high dimensioal spaces, is that it's more difficult to explore the data and create visualizations.

## 6.2 What is the rule of thumb for how many observations you should have compared to parameters in your model?

###The number of observations to have oberved compared

###to parameter in the model should be 5.

# Principal Component Analysis

## 7.1 Code for loading and cleaning the 2013 national dataset from the [Housing Affordability Data System (HADS)](https://www.huduser.gov/portal/datasets/hads/hads.html) --housing data, can be found below.

## Perform PCA on the processed dataset `national_processed` (Make sure you standardize your data!) and then make a scatterplot of PC1 against PC2. Some of our discussion and work around PCA with this dataset will continue during tomorrow's lecture and assignment.

Not only does this dataset have decent amount columns to begin with (99), but in preparing the data for PCA we have also [one-hot-encoded](https://hackernoon.com/what-is-one-hot-encoding-why-and-when-do-you-have-to-use-it-e3c6186d008f#targetText=One%20hot%20encoding%20is%20a,the%20entry%20in%20the%20dataset.) all of the categorical variables. This has the effect of creating a new column for each individual category of each categorical variable. After processing this dataset has 64738 columns. --Das a lot of columns.

Don't worry too much about the mechanics of one-hot encoding right now, you will learn and experiment with a whole bunch of categorical encoding approaches in unit 2.

The code below will read in the dataset and perform the one-hot encoding of the categorical variables. Start adding your PCA code at the bottom of the provided code.

```
from urllib.request import urlopen
from zipfile import ZipFile
from io import BytesIO
import os.path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Read Natinal Data
national_url = 'https://www.huduser.gov/portal/datasets/hads/hads2013n_ASCII.zip'
national_file = 'thads2013n.txt'

if os.path.exists(national_file):
    national = pd.read_csv(national_file)
else:
    z_national = urlopen(national_url)
    zip_national = ZipFile(BytesIO(z_national.read())).extract(national_file)
    national = pd.read_csv(zip_national)

print(national.shape)
national.head()
```

    (64535, 99)

      CONTROL
      AGE1
      METRO3
      REGION
      LMED
      FMR
      L30
      L50
      L80
      IPOV
      BEDRMS
      BUILT
      STATUS
      TYPE
      VALUE
      VACANCY
      TENURE
      NUNITS
      ROOMS
      WEIGHT
      PER
      ZINC2
      ZADEQ
      ZSMHC
      STRUCTURETYPE
      OWNRENT
      UTILITY
      OTHERCOST
      COST06
      COST12
      COST08
      COSTMED
      TOTSAL
      ASSISTED
      GLMED
      GL30
      GL50
      GL80
      APLMED
      ABL30
      ...
      COST08RELPOVCAT
      COST08RELFMRPCT
      COST08RELFMRCAT
      COST12RELAMIPCT
      COST12RELAMICAT
      COST12RELPOVPCT
      COST12RELPOVCAT
      COST12RELFMRPCT
      COST12RELFMRCAT
      COSTMedRELAMIPCT
      COSTMedRELAMICAT
      COSTMedRELPOVPCT
      COSTMedRELPOVCAT
      COSTMedRELFMRPCT
      COSTMedRELFMRCAT
      FMTZADEQ
      FMTMETRO3
      FMTBUILT
      FMTSTRUCTURETYPE
      FMTBEDRMS
      FMTOWNRENT
      FMTCOST06RELPOVCAT
      FMTCOST08RELPOVCAT
      FMTCOST12RELPOVCAT
      FMTCOSTMEDRELPOVCAT
      FMTINCRELPOVCAT
      FMTCOST06RELFMRCAT
      FMTCOST08RELFMRCAT
      FMTCOST12RELFMRCAT
      FMTCOSTMEDRELFMRCAT
      FMTINCRELFMRCAT
      FMTCOST06RELAMICAT
      FMTCOST08RELAMICAT
      FMTCOST12RELAMICAT
      FMTCOSTMEDRELAMICAT
      FMTINCRELAMICAT
      FMTASSISTED
      FMTBURDEN
      FMTREGION
      FMTSTATUS

      0
      '100003130103'
      82
      '3'
      '1'
      73738
      956
      15738
      26213
      40322
      11067
      2
      2006
      '1'
      1
      40000
      -6
      '1'
      1
      6
      3117.394239
      1
      18021
      '1'
      533
      1
      '1'
      169.000000
      213.750000
      648.588189
      803.050535
      696.905247
      615.156712
      0
      -9
      73738
      15738
      26213
      40322
      51616.6
      20234.571429
      ...
      4
      72.898038
      2
      48.402635
      2
      290.250487
      4
      84.001102
      2
      37.077624
      2
      222.339102
      4
      64.346936
      2
      '1 Adequate'
      '-5'
      '2000-2009'
      '1 Single Family'
      '2 2BR'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '1 LTE 50% FMR'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '.'
      '2 30% to 50%'
      '-5'
      '-5'

      1
      '100006110249'
      50
      '5'
      '3'
      55846
      1100
      17165
      28604
      45744
      24218
      4
      1980
      '1'
      1
      130000
      -6
      '1'
      1
      6
      2150.725544
      4
      122961
      '1'
      487
      1
      '1'
      245.333333
      58.333333
      1167.640781
      1669.643405
      1324.671218
      1058.988479
      123000
      -9
      55846
      17165
      28604
      45744
      55846.0
      19911.400000
      ...
      4
      120.424656
      3
      103.094063
      6
      275.768999
      4
      151.785764
      3
      65.388468
      4
      174.909320
      3
      96.271680
      2
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '4 4BR+'
      '1 Owner'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '3 GT FMR'
      '4 60 - 80% AMI'
      '4 60 - 80% AMI'
      '6 100 - 120% AMI'
      '4 60 - 80% AMI'
      '7 120% AMI +'
      '.'
      '1 Less than 30%'
      '-5'
      '-5'

      2
      '100006370140'
      53
      '5'
      '3'
      55846
      1100
      13750
      22897
      36614
      15470
      4
      1985
      '1'
      1
      150000
      -6
      '1'
      1
      7
      2213.789404
      2
      27974
      '1'
      1405
      1
      '1'
      159.000000
      37.500000
      1193.393209
      1772.627006
      1374.582175
      1068.025168
      28000
      -9
      55846
      13750
      22897
      36614
      44676.8
      19937.500000
      ...
      4
      124.962016
      3
      109.452905
      6
      458.339239
      4
      161.147910
      3
      65.946449
      4
      276.153890
      4
      97.093197
      2
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '4 4BR+'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '4 60 - 80% AMI'
      '5 80 - 100% AMI'
      '6 100 - 120% AMI'
      '4 60 - 80% AMI'
      '4 60 - 80% AMI'
      '.'
      '3 50% or More'
      '-5'
      '-5'

      3
      '100006520140'
      67
      '5'
      '3'
      55846
      949
      13750
      22897
      36614
      13964
      3
      1985
      '1'
      1
      200000
      -6
      '1'
      1
      6
      2364.585097
      2
      32220
      '1'
      279
      1
      '1'
      179.000000
      70.666667
      1578.857612
      2351.169341
      1820.442900
      1411.700224
      0
      -9
      55846
      13750
      22897
      36614
      44676.8
      17875.000000
      ...
      4
      191.827492
      3
      161.926709
      7
      673.494512
      4
      247.752301
      3
      97.224801
      5
      404.382763
      4
      148.756610
      3
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '3 3BR'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '6 100 - 120% AMI'
      '7 120% AMI +'
      '7 120% AMI +'
      '5 80 - 100% AMI'
      '4 60 - 80% AMI'
      '.'
      '1 Less than 30%'
      '-5'
      '-5'

      4
      '100007130148'
      26
      '1'
      '3'
      60991
      737
      14801
      24628
      39421
      15492
      2
      1980
      '1'
      1
      -6
      -6
      '2'
      100
      4
      2314.524902
      2
      96874
      '1'
      759
      5
      '2'
      146.000000
      12.500000
      759.000000
      759.000000
      759.000000
      759.000000
      96900
      0
      60991
      14801
      24628
      39421
      48792.8
      16651.125000
      ...
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      '1 Adequate'
      'Central City'
      '1980-1989'
      '5 50+ units'
      '2 2BR'
      '2 Renter'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '7 120% AMI +'
      '0 Not Assisted'
      '1 Less than 30%'
      '-5'
      '-5'

5 rows × 99 columns

```
# Look at datatypes
# a lot of object datatypes even though they seem to be strings of numbers.
national.dtypes
```

    CONTROL                 object
    AGE1                     int64
    METRO3                  object
    REGION                  object
    LMED                     int64
    FMR                      int64
    L30                      int64
    L50                      int64
    L80                      int64
    IPOV                     int64
    BEDRMS                   int64
    BUILT                    int64
    STATUS                  object
    TYPE                     int64
    VALUE                    int64
    VACANCY                  int64
    TENURE                  object
    NUNITS                   int64
    ROOMS                    int64
    WEIGHT                 float64
    PER                      int64
    ZINC2                    int64
    ZADEQ                   object
    ZSMHC                    int64
    STRUCTURETYPE            int64
    OWNRENT                 object
    UTILITY                float64
    OTHERCOST              float64
    COST06                 float64
    COST12                 float64
                            ...
    COSTMedRELAMICAT         int64
    COSTMedRELPOVPCT       float64
    COSTMedRELPOVCAT         int64
    COSTMedRELFMRPCT       float64
    COSTMedRELFMRCAT         int64
    FMTZADEQ                object
    FMTMETRO3               object
    FMTBUILT                object
    FMTSTRUCTURETYPE        object
    FMTBEDRMS               object
    FMTOWNRENT              object
    FMTCOST06RELPOVCAT      object
    FMTCOST08RELPOVCAT      object
    FMTCOST12RELPOVCAT      object
    FMTCOSTMEDRELPOVCAT     object
    FMTINCRELPOVCAT         object
    FMTCOST06RELFMRCAT      object
    FMTCOST08RELFMRCAT      object
    FMTCOST12RELFMRCAT      object
    FMTCOSTMEDRELFMRCAT     object
    FMTINCRELFMRCAT         object
    FMTCOST06RELAMICAT      object
    FMTCOST08RELAMICAT      object
    FMTCOST12RELAMICAT      object
    FMTCOSTMEDRELAMICAT     object
    FMTINCRELAMICAT         object
    FMTASSISTED             object
    FMTBURDEN               object
    FMTREGION               object
    FMTSTATUS               object
    Length: 99, dtype: object

```
# check for null values
national.isnull().sum().any()
```

    False

```
# check for number of categorical vs numeric columns
cat_cols = national.columns[national.dtypes=='object']
num_cols = national.columns[national.dtypes!='object']

print(f'{len(cat_cols)} categorical columns')
print(f'{len(num_cols)} numerical columns')
```

    32 categorical columns
    67 numerical columns

```
# We're making a copy of our data in case we mess something up.
national_processed = national.copy()

# Categorically Encode our Variables:
# They need to all be numeric before we do PCA.
# https://pbpython.com/categorical-encoding.html

# Cast categorical columns to "category" data type
national_processed[cat_cols] = national_processed[cat_cols].astype('category')

national_processed.dtypes
```

    CONTROL                category
    AGE1                      int64
    METRO3                 category
    REGION                 category
    LMED                      int64
    FMR                       int64
    L30                       int64
    L50                       int64
    L80                       int64
    IPOV                      int64
    BEDRMS                    int64
    BUILT                     int64
    STATUS                 category
    TYPE                      int64
    VALUE                     int64
    VACANCY                   int64
    TENURE                 category
    NUNITS                    int64
    ROOMS                     int64
    WEIGHT                  float64
    PER                       int64
    ZINC2                     int64
    ZADEQ                  category
    ZSMHC                     int64
    STRUCTURETYPE             int64
    OWNRENT                category
    UTILITY                 float64
    OTHERCOST               float64
    COST06                  float64
    COST12                  float64
                             ...
    COSTMedRELAMICAT          int64
    COSTMedRELPOVPCT        float64
    COSTMedRELPOVCAT          int64
    COSTMedRELFMRPCT        float64
    COSTMedRELFMRCAT          int64
    FMTZADEQ               category
    FMTMETRO3              category
    FMTBUILT               category
    FMTSTRUCTURETYPE       category
    FMTBEDRMS              category
    FMTOWNRENT             category
    FMTCOST06RELPOVCAT     category
    FMTCOST08RELPOVCAT     category
    FMTCOST12RELPOVCAT     category
    FMTCOSTMEDRELPOVCAT    category
    FMTINCRELPOVCAT        category
    FMTCOST06RELFMRCAT     category
    FMTCOST08RELFMRCAT     category
    FMTCOST12RELFMRCAT     category
    FMTCOSTMEDRELFMRCAT    category
    FMTINCRELFMRCAT        category
    FMTCOST06RELAMICAT     category
    FMTCOST08RELAMICAT     category
    FMTCOST12RELAMICAT     category
    FMTCOSTMEDRELAMICAT    category
    FMTINCRELAMICAT        category
    FMTASSISTED            category
    FMTBURDEN              category
    FMTREGION              category
    FMTSTATUS              category
    Length: 99, dtype: object

```
# Replace all category cell values with their numeric category codes
for col in cat_cols:
  national_processed[col] = national_processed[col].cat.codes

print(national_processed.shape)
national_processed.head()
```

```
# Now we only ahve numeric columns (ints and floats)
national_processed.dtypes
```

```
national_processed.mean()
```

```
national_processed.std()
```

```
z=(national_processed-national_processed.mean())/national_processed.std()
z
```

# Stretch Goals

## 1) Perform further data exploration on the HADS national dataset (the version before we one-hot encoded it) Make scatterplots and see if you can see any resemblance between the original scatterplots and the plot of the principal components that you made in 7.1.

(You may or may not not see very much resemblance depending on the variables you choose, and that's ok!)

## 2) Study "Scree Plots" and then try and make one for your PCA dataset. How many principal conponents do you need to retain in order for your PCs to contain 90% of the explained variance?

We will present this topic formally at the beginning of tomorrow's lecture, so if you figure this stretch goal out, you're ahead of the game.

## 3) Explore further the intuition behind eigenvalues and eigenvectors by creating your very own eigenfaces:

Prioritize self-study over this stretch goal if you are not semi-comfortable with the topics of PCA, Eigenvalues, and Eigenvectors.

You don't necessarily have to use this resource, but this will get you started:
[Eigenface Tutorial](https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-a-simple-face-detector-with-pca-svd-in-python/)


# Generating C code for the right-hand-side of the scalar wave equation, in ***curvilinear*** coordinates, using a reference metric formalism

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

**Notebook Status:**  Validated

**Validation Notes:** This tutorial notebook has been confirmed to be self-consistent with its corresponding NRPy+ module, as documented [below](#code_validation). In addition, all expressions have been validated against a trusted code (the [original SENR/NRPy+ code](https://bitbucket.org/zach_etienne/nrpy)).

### NRPy+ Source Code for this module: [ScalarWaveCurvilinear/ScalarWaveCurvilinear_RHSs.py](../edit/ScalarWaveCurvilinear/ScalarWaveCurvilinear_RHSs.py)

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

0. [Preliminaries](#prelim): Reference Metrics and Picking Best Coordinate System to Solve the PDE
1. [Example](#example): The scalar wave equation in spherical coordinates
1. [Step 1](#contracted_christoffel): Contracted Christoffel symbols $\hat{\Gamma}^i = \hat{g}^{ij}\hat{\Gamma}^k_{ij}$ in spherical coordinates, using NRPy+
1. [Step 2](#rhs_scalarwave_spherical): The right-hand side of the scalar wave equation in spherical coordinates, using NRPy+
1. [Step 3](#code_validation): Code Validation against `ScalarWave.ScalarWaveCurvilinear_RHSs`  NRPy+ Module
1. [Step 5](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Preliminaries: Reference Metrics and Picking Best Coordinate System to Solve the PDE \[Back to [top](#toc)\]
$$\label{prelim}$$

Recall from [NRPy+ tutorial notebook on the Cartesian scalar wave equation](Tutorial-ScalarWave.ipynb), the scalar wave equation in 3D Cartesian coordinates is given by

$$\partial_t^2 u = c^2 \nabla^2 u \text{,}$$
where $u$ (the amplitude of the wave) is a function of time and Cartesian coordinates in space: $u = u(t,x,y,z)$ (spatial dimension as-yet unspecified), and subject to some initial condition
$$u(0,x,y,z) = f(x,y,z),$$

with suitable (sometimes approximate) spatial boundary conditions.

To simplify this equation, let's first choose units such that $c=1$. Alternative wave speeds can be constructed
by simply rescaling the time coordinate, with the net effect being that the time $t$ is replaced with time in dimensions of space; i.e., $t\to c t$:

$$\partial_t^2 u = \nabla^2 u.$$

As we learned in the [NRPy+ tutorial notebook on reference metrics](Tutorial-Reference_Metric.ipynb), reference metrics are a means to pick the best coordinate system for the PDE we wish to solve. However, to take advantage of reference metrics requires first that we generalize the PDE. In the case of the scalar wave equation, this involves first rewriting in [Einstein notation](https://en.wikipedia.org/wiki/Einstein_notation) (with implied summation over repeated indices) via

$$(-\partial_t^2 + \nabla^2) u = \eta^{\mu\nu} u_{,\ \mu\nu} = 0,$$

where $u_{,\mu\nu} = \partial_\mu \partial_\nu u$, and $\eta^{\mu\nu}$ is the contravariant flat-space metric tensor with components $\text{diag}(-1,1,1,1)$.

Next we apply the "comma-goes-to-semicolon rule" and replace $\eta^{\mu\nu}$ with $\hat{g}^{\mu\nu}$ to generalize the scalar wave equation to an arbitrary reference metric $\hat{g}^{\mu\nu}$:

$$\hat{g}^{\mu\nu} u_{;\ \mu\nu} = \hat{g}^{\mu\nu} \hat{\nabla}_{\mu} \hat{\nabla}_{\nu} u = 0,$$

where $\hat{\nabla}_{\mu}$ denotes the [covariant derivative](https://en.wikipedia.org/wiki/Covariant_derivative) with respect to the reference metric basis vectors $\hat{x}^{\mu}$, and $\hat{g}^{\mu \nu} \hat{\nabla}_{\mu} \hat{\nabla}_{\nu} u$ is the covariant
[D'Alembertian](https://en.wikipedia.org/wiki/D%27Alembert_operator) of $u$.

For example, suppose we wish to model a short-wavelength wave that is nearly spherical. In this case, if we were to solve the wave equation PDE in Cartesian coordinates, we would in principle need high resolution in all three cardinal directions. If instead we chose spherical coordinates centered at the center of the wave, we might need high resolution only in the radial direction, with only a few points required in the angular directions. Thus choosing spherical coordinates would be far more computationally efficient than modeling the wave in Cartesian coordinates.

Let's now expand the covariant scalar wave equation in arbitrary coordinates. Since the covariant derivative of a scalar is equivalent to its partial derivative, we have
\begin{align}
0 &= \hat{g}^{\mu \nu} \hat{\nabla}_{\mu} \hat{\nabla}_{\nu} u \\
&= \hat{g}^{\mu \nu} \hat{\nabla}_{\mu} \partial_{\nu} u.
\end{align}

$\partial_{\nu} u$ transforms as a one-form under covariant differentiation, so we have
$$\hat{\nabla}_{\mu} \partial_{\nu} u = \partial_{\mu} \partial_{\nu} u - \hat{\Gamma}^\tau_{\mu\nu} \partial_\tau u,$$
where

$$\hat{\Gamma}^\tau_{\mu\nu} = \frac{1}{2} \hat{g}^{\tau\alpha} \left(\partial_\nu \hat{g}_{\alpha\mu} + \partial_\mu \hat{g}_{\alpha\nu} - \partial_\alpha \hat{g}_{\mu\nu} \right)$$
are the [Christoffel symbols](https://en.wikipedia.org/wiki/Christoffel_symbols) associated with the reference metric $\hat{g}_{\mu\nu}$.

Then the scalar wave equation is written:
$$0 = \hat{g}^{\mu \nu} \left( \partial_{\mu} \partial_{\nu} u - \hat{\Gamma}^\tau_{\mu\nu} \partial_\tau u\right).$$

Define the contracted Christoffel symbols:
$$\hat{\Gamma}^\tau = \hat{g}^{\mu\nu} \hat{\Gamma}^\tau_{\mu\nu}.$$

Then the scalar wave equation is given by
$$0 = \hat{g}^{\mu \nu} \partial_{\mu} \partial_{\nu} u - \hat{\Gamma}^\tau \partial_\tau u.$$

The reference metrics we adopt satisfy
$$\hat{g}^{t \nu} = -\delta^{t \nu},$$
where $\delta^{t \nu}$ is the [Kronecker delta](https://en.wikipedia.org/wiki/Kronecker_delta). Therefore the scalar wave equation in curvilinear coordinates can be written
\begin{align}
0 &= \hat{g}^{\mu \nu} \partial_{\mu} \partial_{\nu} u - \hat{\Gamma}^\tau \partial_\tau u \\
&= -\partial_t^2 u + \hat{g}^{i j} \partial_{i} \partial_{j} u - \hat{\Gamma}^i \partial_i u \\
\implies \partial_t^2 u &= \hat{g}^{i j} \partial_{i} \partial_{j} u - \hat{\Gamma}^i \partial_i u,
\end{align}
where repeated Latin indices denote implied summation over *spatial* components only. This module implements the bottom equation for arbitrary reference metrics satisfying $\hat{g}^{t \nu} = -\delta^{t \nu}$. To gain an appreciation for what NRPy+ accomplishes automatically, let's first work out the scalar wave equation in spherical coordinates by hand:

# Example: The scalar wave equation in spherical coordinates \[Back to [top](#toc)\]
$$\label{example}$$

For example, the spherical reference metric is written

$$\hat{g}_{\mu\nu} = \begin{pmatrix}
-1 & 0 & 0 & 0 \\
 0 & 1 & 0 & 0 \\
 0 & 0 & r^2 & 0 \\
 0 & 0 & 0 & r^2 \sin^2 \theta \\
\end{pmatrix}.
$$

Since the inverse of a diagonal matrix is simply the inverse of the diagonal elements, we can write
$$\hat{g}^{\mu\nu} = \begin{pmatrix}
-1 & 0 & 0 & 0 \\
 0 & 1 & 0 & 0 \\
 0 & 0 & \frac{1}{r^2} & 0 \\
 0 & 0 & 0 & \frac{1}{r^2 \sin^2 \theta} \\
\end{pmatrix}.$$

The scalar wave equation in these coordinates can thus be written
\begin{align}
0 &= \hat{g}^{\mu \nu} \partial_{\mu} \partial_{\nu} u - \hat{\Gamma}^\tau \partial_\tau u \\
&= \hat{g}^{tt} \partial_t^2 u + \hat{g}^{rr} \partial_r^2 u + \hat{g}^{\theta\theta} \partial_\theta^2 u  + \hat{g}^{\phi\phi} \partial_\phi^2 u - \hat{\Gamma}^\tau \partial_\tau u \\
&= -\partial_t^2 u + \partial_r^2 u + \frac{1}{r^2} \partial_\theta^2
u + \frac{1}{r^2 \sin^2 \theta} \partial_\phi^2 u - \hat{\Gamma}^\tau \partial_\tau u\\
\implies \partial_t^2 u &= \partial_r^2 u + \frac{1}{r^2} \partial_\theta^2
u + \frac{1}{r^2 \sin^2 \theta} \partial_\phi^2 u - \hat{\Gamma}^\tau \partial_\tau u
\end{align}

The contracted Christoffel symbols
$\hat{\Gamma}^\tau$ can then be computed directly from the metric $\hat{g}_{\mu\nu}$.

It can be shown (exercise to the reader) that the only nonzero
components of $\hat{\Gamma}^\tau$ in static spherical polar coordinates are
given by
\begin{align}
\hat{\Gamma}^r &= -\frac{2}{r} \\
\hat{\Gamma}^\theta &= -\frac{\cos\theta}{r^2 \sin\theta}.
\end{align}

Thus we have found the Laplacian in spherical coordinates is simply:

\begin{align}
\nabla^2 u &=
\partial_r^2 u + \frac{1}{r^2} \partial_\theta^2 u + \frac{1}{r^2 \sin^2 \theta} \partial_\phi^2 u - \hat{\Gamma}^\tau \partial_\tau u\\
&= \partial_r^2 u + \frac{1}{r^2} \partial_\theta^2 u + \frac{1}{r^2 \sin^2 \theta} \partial_\phi^2 u +  \frac{2}{r} \partial_r u + \frac{\cos\theta}{r^2 \sin\theta} \partial_\theta u
\end{align}
(cf. http://mathworld.wolfram.com/SphericalCoordinates.html; though note that they defined the angle $\phi$ as $\theta$ and $\theta$ as $\phi$.)

# Step 1: Contracted Christoffel symbols $\hat{\Gamma}^i = \hat{g}^{ij}\hat{\Gamma}^k_{ij}$ in spherical coordinates, using NRPy+ \[Back to [top](#toc)\]
$$\label{contracted_christoffel}$$

Let's next use NRPy+ to derive the contracted Christoffel symbols
$$\hat{g}^{ij} \hat{\Gamma}^k_{ij}$$
in spherical coordinates, where $i\in\{1,2,3\}$ and $j\in\{1,2,3\}$ are spatial indices.

As discussed in the [NRPy+ tutorial notebook on reference metrics](Tutorial-Reference_Metric.ipynb), several reference-metric-related quantities in spherical coordinates are computed in NRPy+ (provided the parameter **`reference_metric::CoordSystem`** is set to **`"Spherical"`**), including the inverse spatial spherical reference metric $\hat{g}^{ij}$ and the Christoffel symbols from this reference metric $\hat{\Gamma}^{i}_{jk}$.

```python
import sympy as sp              # SymPy: The Python computer algebra package upon which NRPy+ depends
import NRPy_param_funcs as par  # NRPy+: Parameter interface
import grid as gri              # NRPy+: Functionality for handling numerical grids
import indexedexp as ixp        # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import reference_metric as rfm  # NRPy+: Reference metric support

# reference_metric::CoordSystem can be set to Spherical, SinhSpherical, SinhSphericalv2,
#                           Cylindrical, SinhCylindrical, SinhCylindricalv2, etc.
#                           See reference_metric.py and NRPy+ tutorial notebook on
#                           reference metrics for full list and description of how
#                           to extend.
par.set_parval_from_str("reference_metric::CoordSystem","Spherical")
par.set_parval_from_str("grid::DIM",3)

rfm.reference_metric()

contractedGammahatU = ixp.zerorank1()
for k in range(3):
    for i in range(3):
        for j in range(3):
            contractedGammahatU[k] += rfm.ghatUU[i][j] * rfm.GammahatUDD[k][i][j]

for k in range(3):
    print("contracted GammahatU["+str(k)+"]:")
    print(sp.simplify(contractedGammahatU[k]))
    # Sadly pretty_print results in garbage output in the generated PDF at the bottom of this notebook.
#     sp.pretty_print(sp.simplify(contractedGammahatU[k]))
    if k<2:
        print("\n\n")
```

    contracted GammahatU[0]:
    -2/xx0

    contracted GammahatU[1]:
    -1/(xx0**2*tan(xx1))

    contracted GammahatU[2]:
    0

# Step 2: The right-hand side of the scalar wave equation in spherical coordinates, using NRPy+ \[Back to [top](#toc)\]
$$\label{rhs_scalarwave_spherical}$$

Following our [implementation of the scalar wave equation in Cartesian coordinates](Tutorial-ScalarWave.ipynb), we will introduce a new variable $v=\partial_t u$ that will enable us to split the second time derivative into two first-order time derivatives:

\begin{align}
\partial_t u &= v \\
\partial_t v &= \hat{g}^{ij} \partial_{i} \partial_{j} u - \hat{\Gamma}^i \partial_i u.
\end{align}

Adding back the sound speed $c$, we have a choice of a single factor of $c$ multiplying both right-hand sides, or a factor of $c^2$ multiplying the second equation only. We'll choose the latter:

\begin{align}
\partial_t u &= v \\
\partial_t v &= c^2 \left(\hat{g}^{ij} \partial_{i} \partial_{j} u - \hat{\Gamma}^i \partial_i u\right).
\end{align}

Now let's generate the C code for the finite-difference representations of the right-hand sides of the above "time evolution" equations for $u$ and $v$. Since the right-hand side of $\partial_t v$ contains implied sums over $i$ and $j$ in the first term, and an implied sum over $k$ in the second term, we'll find it useful to split the right-hand side into two parts

\begin{equation}
\partial_t v = c^2 \left(
{\underbrace {\textstyle \hat{g}^{ij} \partial_{i} \partial_{j} u}_{\text{Part 1}}}
{\underbrace {\textstyle -\hat{\Gamma}^i \partial_i u}_{\text{Part 2}}}\right),
\end{equation}

and perform the implied sums in two pieces:

```python
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import grid as gri               # NRPy+: Functions having to do with numerical grids
import finite_difference as fin  # NRPy+: Finite difference C code generation module
import reference_metric as rfm   # NRPy+: Reference metric support
from outputC import lhrh         # NRPy+: Core C code output module
```

```python
# The name of this module ("scalarwave") is given by __name__:
thismodule = __name__

# Step 0: Read the spatial dimension parameter as DIM.
DIM = par.parval_from_str("grid::DIM")

# Step 1: Set the finite differencing order to 4.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER",4)

# Step 2a: Reset the gridfunctions list; below we define the
#          full complement of gridfunctions needed by this
#          tutorial. This line of code enables us to re-run this
#          tutorial without resetting the running Python kernel.
gri.glb_gridfcs_list = []
# Step 2b: Register gridfunctions that are needed as input
#          to the scalar wave RHS expressions.
uu, vv = gri.register_gridfunctions("EVOL",["uu","vv"])

# Step 3a: Declare the rank-1 indexed expression \partial_{i} u,
#          Derivative variables like these must have an underscore
#          in them, so the finite difference module can parse the
#          variable name properly.
uu_dD = ixp.declarerank1("uu_dD")

# Step 3b: Declare the rank-2 indexed expression \partial_{ij} u,
#          which is symmetric about interchange of indices i and j
#          Derivative variables like these must have an underscore
#          in them, so the finite difference module can parse the
#          variable name properly.
uu_dDD = ixp.declarerank2("uu_dDD","sym01")

# Step 4: Define the C parameter wavespeed. The `wavespeed`
#         variable is a proper SymPy variable, so it can be
#         used in below expressions. In the C code, it acts
#         just like a usual parameter, whose value is
#         specified in the parameter file.
wavespeed = par.Cparameters("REAL",thismodule,"wavespeed", 1.0)

# Step 5: Define right-hand sides for the evolution.
uu_rhs = vv
# Step 5b: The right-hand side of the \partial_t v equation
#          is given by:
#          \hat{g}^{ij} \partial_i \partial_j u - \hat{\Gamma}^i \partial_i u.
#          ^^^^^^^^^^^^ PART 1 ^^^^^^^^^^^^^^^^ ^^^^^^^^^^ PART 2 ^^^^^^^^^^^
vv_rhs = 0
for i in range(DIM):
    # PART 2:
    vv_rhs -= contractedGammahatU[i]*uu_dD[i]
    for j in range(DIM):
        # PART 1:
        vv_rhs += rfm.ghatUU[i][j]*uu_dDD[i][j]

vv_rhs *= wavespeed*wavespeed

# Step 6: Generate C code for scalarwave evolution equations,
#         print output to the screen (standard out, or stdout).
fin.FD_outputC("stdout",
               [lhrh(lhs=gri.gfaccess("rhs_gfs","uu"),rhs=uu_rhs),
                lhrh(lhs=gri.gfaccess("rhs_gfs","vv"),rhs=vv_rhs)])
```

    {
       /*
        * NRPy+ Finite Difference Code Generation, Step 1 of 2: Read from main memory and compute finite difference stencils:
        */
       /*
        *  Original SymPy expressions:
        *  "[const double uu_dD0 = invdx0*(-2*uu_i0m1_i1_i2/3 + uu_i0m2_i1_i2/12 + 2*uu_i0p1_i1_i2/3 - uu_i0p2_i1_i2/12),
        *    const double uu_dD1 = invdx1*(-2*uu_i0_i1m1_i2/3 + uu_i0_i1m2_i2/12 + 2*uu_i0_i1p1_i2/3 - uu_i0_i1p2_i2/12),
        *    const double uu_dDD00 = invdx0**2*(-5*uu/2 + 4*uu_i0m1_i1_i2/3 - uu_i0m2_i1_i2/12 + 4*uu_i0p1_i1_i2/3 - uu_i0p2_i1_i2/12),
        *    const double uu_dDD11 = invdx1**2*(-5*uu/2 + 4*uu_i0_i1m1_i2/3 - uu_i0_i1m2_i2/12 + 4*uu_i0_i1p1_i2/3 - uu_i0_i1p2_i2/12),
        *    const double uu_dDD22 = invdx2**2*(-5*uu/2 + 4*uu_i0_i1_i2m1/3 - uu_i0_i1_i2m2/12 + 4*uu_i0_i1_i2p1/3 - uu_i0_i1_i2p2/12)]"
        */
       const double uu_i0_i1_i2m2 = in_gfs[IDX4(UUGF, i0,i1,i2-2)];
       const double uu_i0_i1_i2m1 = in_gfs[IDX4(UUGF, i0,i1,i2-1)];
       const double uu_i0_i1m2_i2 = in_gfs[IDX4(UUGF, i0,i1-2,i2)];
       const double uu_i0_i1m1_i2 = in_gfs[IDX4(UUGF, i0,i1-1,i2)];
       const double uu_i0m2_i1_i2 = in_gfs[IDX4(UUGF, i0-2,i1,i2)];
       const double uu_i0m1_i1_i2 = in_gfs[IDX4(UUGF, i0-1,i1,i2)];
       const double uu = in_gfs[IDX4(UUGF, i0,i1,i2)];
       const double uu_i0p1_i1_i2 = in_gfs[IDX4(UUGF, i0+1,i1,i2)];
       const double uu_i0p2_i1_i2 = in_gfs[IDX4(UUGF, i0+2,i1,i2)];
       const double uu_i0_i1p1_i2 = in_gfs[IDX4(UUGF, i0,i1+1,i2)];
       const double uu_i0_i1p2_i2 = in_gfs[IDX4(UUGF, i0,i1+2,i2)];
       const double uu_i0_i1_i2p1 = in_gfs[IDX4(UUGF, i0,i1,i2+1)];
       const double uu_i0_i1_i2p2 = in_gfs[IDX4(UUGF, i0,i1,i2+2)];
       const double vv = in_gfs[IDX4(VVGF, i0,i1,i2)];
       const double FDPart1_Rational_2_3 = 2.0/3.0;
       const double FDPart1_Rational_1_12 = 1.0/12.0;
       const double FDPart1_Rational_5_2 = 5.0/2.0;
       const double FDPart1_Rational_4_3 = 4.0/3.0;
       const double FDPart1_1 = -uu_i0_i1p2_i2;
       const double FDPart1_2 = -FDPart1_Rational_5_2*uu;
       const double uu_dD0 = invdx0*(FDPart1_Rational_1_12*(uu_i0m2_i1_i2 - uu_i0p2_i1_i2) + FDPart1_Rational_2_3*(-uu_i0m1_i1_i2 + uu_i0p1_i1_i2));
       const double uu_dD1 = invdx1*(FDPart1_Rational_1_12*(FDPart1_1 + uu_i0_i1m2_i2) + FDPart1_Rational_2_3*(-uu_i0_i1m1_i2 + uu_i0_i1p1_i2));
       const double uu_dDD00 = ((invdx0)*(invdx0))*(FDPart1_2 + FDPart1_Rational_1_12*(-uu_i0m2_i1_i2 - uu_i0p2_i1_i2) + FDPart1_Rational_4_3*(uu_i0m1_i1_i2 + uu_i0p1_i1_i2));
       const double uu_dDD11 = ((invdx1)*(invdx1))*(FDPart1_2 + FDPart1_Rational_1_12*(FDPart1_1 - uu_i0_i1m2_i2) + FDPart1_Rational_4_3*(uu_i0_i1m1_i2 + uu_i0_i1p1_i2));
       const double uu_dDD22 = ((invdx2)*(invdx2))*(FDPart1_2 + FDPart1_Rational_1_12*(-uu_i0_i1_i2m2 - uu_i0_i1_i2p2) + FDPart1_Rational_4_3*(uu_i0_i1_i2m1 + uu_i0_i1_i2p1));
       /*
        * NRPy+ Finite Difference Code Generation, Step 2 of 2: Evaluate SymPy expressions and write to main memory:
        */
       /*
        *  Original SymPy expressions:
        *  "[rhs_gfs[IDX4(UUGF, i0, i1, i2)] = vv,
        *    rhs_gfs[IDX4(VVGF, i0, i1, i2)] = wavespeed**2*(2*uu_dD0/xx0 + uu_dD1*sin(2*xx1)/(2*xx0**2*sin(xx1)**2) + uu_dDD00 + uu_dDD11/xx0**2 + uu_dDD22/(xx0**2*sin(xx1)**2))]"
        */
       const double FDPart3_0 = (1.0/((xx0)*(xx0)));
       const double FDPart3_1 = FDPart3_0/((sin(xx1))*(sin(xx1)));
       rhs_gfs[IDX4(UUGF, i0, i1, i2)] = vv;
       rhs_gfs[IDX4(VVGF, i0, i1, i2)] = ((wavespeed)*(wavespeed))*(FDPart3_0*uu_dDD11 + (1.0/2.0)*FDPart3_1*uu_dD1*sin(2*xx1) + FDPart3_1*uu_dDD22 + 2*uu_dD0/xx0 + uu_dDD00);
    }

# Step 3: Code Validation against `ScalarWave.ScalarWaveCurvilinear_RHSs`  NRPy+ Module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the RHSs of the Curvilinear Scalar Wave equation (i.e., uu_rhs and vv_rhs) between

1. this tutorial and
2. the NRPy+ [ScalarWave.ScalarWaveCurvilinear_RHSs](../edit/ScalarWaveCurvilinear/ScalarWaveCurvilinear_RHSs.py) module.

By default, we analyze the RHSs in Spherical coordinates, though other coordinate systems may be chosen.

```python
# Step 7: We already have SymPy expressions for uu_rhs and vv_rhs in
#         terms of other SymPy variables. Even if we reset the list
#         of NRPy+ gridfunctions, these *SymPy* expressions for
#         uu_rhs and vv_rhs *will remain unaffected*.
#
#         Here, we will use the above-defined uu_rhs and vv_rhs to
#         validate against the same expressions in the
#         ScalarWaveCurvilinear/ScalarWaveCurvilinear module,
#         to ensure consistency between the tutorial and the
#         module itself.
#
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []

# Step 8: Call the ScalarWaveCurvilinear_RHSs() function from within the
#         ScalarWaveCurvilinear/ScalarWaveCurvilinear_RHSs.py module,
#         which should do exactly the same as in Steps 1-6 above.
import ScalarWave.ScalarWaveCurvilinear_RHSs as swcrhs
swcrhs.ScalarWaveCurvilinear_RHSs()

# Step 9: Consistency check between the tutorial notebook above
#         and the ScalarWaveCurvilinear_RHSs() function from within the
#         ScalarWaveCurvilinear/ScalarWaveCurvilinear_RHSs.py module.
print("Consistency check between ScalarWaveCurvilinear tutorial and NRPy+ module:")
if sp.simplify(uu_rhs - swcrhs.uu_rhs) != 0:
    print("TEST FAILED: uu_ID_SphericalGaussian - swid.uu_ID = "+str(sp.simplify(uu_rhs - swcrhs.uu_rhs))+"\t\t (should be zero)")
    sys.exit(1)
if sp.simplify(vv_rhs - swcrhs.vv_rhs) != 0:
    print("TEST FAILED: vv_ID_SphericalGaussian - swid.vv_ID = "+str(sp.simplify(vv_rhs - swcrhs.vv_rhs))+"\t\t (should be zero)")
    sys.exit(1)
print("TESTS PASSED!")
```

    Consistency check between ScalarWaveCurvilinear tutorial and NRPy+ module:
    TESTS PASSED!

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-ScalarWaveCurvilinear.pdf](Tutorial-ScalarWaveCurvilinear.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-ScalarWaveCurvilinear")
```

    Created Tutorial-ScalarWaveCurvilinear.tex, and compiled LaTeX file to PDF
        file Tutorial-ScalarWaveCurvilinear.pdf


# Materialien zu zufall

Autor: Holger Böttcher - hbomat@posteo.de

## Aufgaben 10 - Zum Geburtstagsproblem

Siehe dazu

Elemente der Mathematik

Leistungskurs Stochastik

Schroedel Verlag 2003

S. 52-53

Es soll die Handhabung von Python-Mitteln zur Bearbeitung der bei Schroedel

betrachteten Aufgaben demonstriert werden

#### Klassisches Geburtstagsproblem
Das klassische Geburtstagsproblem lautet:

23 Personen werden zufällig ausgewählt. Lohnt es sich darauf zu wetten, dass

unter diesen mindestens zwei sind, die am gleichen Tag Geburtstag haben?

Die Antwort ist

$\qquad$ Ja. $\quad$    Die Wahrscheinlichkeit für dieses Ereignis ist größer als 0.5

Die Berechnung der Wahrscheinlichkeit erfolgt über das Gegenereignis

"keine zwei Personen haben am gleichen Tag Geburtstag"

$P(\text{ mindestens zwei der 23 Personen haben am gleichen Tag Geburtstag} ) = $

$\quad = 1 - \frac{365 \, \times \, \dots \, \, \times \, 343}{365^{23}} = 0.5073$

```python
%run zufall/start
```

Bei der Berechnung kann die Funktion

```python
def p(k):
    m = 1
    for i in range(365-k+1, 365):
        m *= i/365
    return 1 - m
```

benutzt werden, die bei  $k = 23$  den obigen Wert ergibt

```python
p(23)
```

$$0.5072972343239853$$

Das Ergebnis widerspricht der Intuition vieler. Eventuell besteht die Vorstellung,

dass die Wahrscheinlichkeit linear mit der Anzahl der Personen wächst und einen

Wert über $0.5$ erst bei einem größeren Wert von $k$ erreicht. Die folgende Grafik

zeigt die Abhängigkeit der Wahrscheinlichkeit für das Zusammentreffen von minde-

stens zwei Geburtstagen von der Personenzahl. Es ist ersichtlich, dass eine solche

Annahme nicht richtig ist

(nach G. von Randow, Das Ziegenproblem, Rowohlt 1992/2004, S. 44)

Die Anweisungen zur Erzeugung der Grafik mit matplotlib sind bewußt im Notebook

belassen worden

```python
import numpy as np
import matplotlib.pyplot as plt

plt.close('all')
plt.figure(figsize=(4, 2.5))

x = np.array(range(60))
y = np.array([p(int(k)) for k in x])
plt.plot(x, y, color='r', lw=0.8)

z = np.linspace(0, 60, 100)
plt.plot(z, 0.009*z, color='g', lw=0.8)

plt.plot([23, 23], [0, p(23)], color='k', lw=0.8, linestyle='dotted')
plt.plot([0, 23], [p(23), p(23)], color='k', lw=0.8, linestyle='dotted')

plt.plot([47, 47], [0, p(47)], color='b', lw=0.8, linestyle='dotted')
plt.plot([0, 47], [p(47), p(47)], color='b', lw=0.8, linestyle='dotted')

plt.xlabel('Personen')
plt.ylim(0, 1.1)

plt.show()
```

Die grüne Linie entspricht der Annahme eines linearen Ansteigens der Wahr-

scheinlichkeit. Es sind außerdem die Personenzahlen markiert, bei denen die

Wahrscheinlichkeit $0.5$ (schwarz) bzw.$ 0.95$ (blau) erstmals übersteigt

```python
p(22), p(23), p(46), p(47)
```

$$\left ( 0.47569530766254997, \quad 0.5072972343239853, \quad 0.9482528433672548, \quad 0.9547744028332994\right )$$

#### Allgemeines Problem der Wiederholungen bei Zufallsversuchen
Ein Zufallsexperiment mit $m$ möglichen gleichwahrscheinlichen Ergebnissen wird

$k$-mal durchgeführt $(k = 1, 2, \dots,  m+1)$

Es soll der Wert  $k_{50}$  bestimmt werden, dass die Wahrscheinlichkeit für mindestens

eine Wiederholung größer ist als $50\%$

Die folgende Tabelle (s. Schroedel, Aufgabe 5) zeigt diese Werte für verschiedene $m$

(der letzte Wert entspricht dem Lottospiel 6 aus 49)

  <col width='55'>
  <col width='55'>
  <col width='55'>
  <col width='55'>
  <col width='55'>
  <col width='55'>
  <col width='60'>
  <col width='60'>
  <col width='60'>
  <row align='right'>

     $m$
     400
     900
     1600
     2500
     10000
     250000
     1000000
     13983816

     $k_{50}$
     24
     36
     48
     60
     119
     589
     1178
     4404
  </tr

Die Bestätigung der Tabellenwerte (Aufgabe 5a) kann analog zur oben verwende-

ten Funktion $p$ mittels einer Funktion $f$ erfolgen

```python
def f(m, k):
    q = 1
    for i in range(m-k+1, m):
        q *= i/m
    return 1 - q
```

Die Funktion wurde so gestaltet, dass  Zahlen zwischen 0 und 1 zu multiplizie-

ren sind und nicht ganze Zahlen  (was bei großen $m$ und $k$ schnell zum Speicher-

überlauf des Rechners führt). Bei $m = 365$ ergibt sie die selben Werte wie die

Funktion $p$

```python
f(365, 23), p(23), f(365, 47), p(47)
```

$$\left ( 0.5072972343239853, \quad 0.5072972343239853, \quad 0.9547744028332994, \quad 0.9547744028332994\right )$$

Die Tabellenwerte werden bestätigt, indem (bei gegebenem $m$) der angegebene $k_{50}$-

Wert und der um 1 verringerte Wert als Argumente angegeben werden, womit man

sich vom Erreichen der Grenze 0.5 überzeugt; z.B.

```python
f(400, 24), f(400, 23)
```

$$\left ( 0.5053593788661789, \quad 0.47518236484475196\right )$$

```python
f(10000, 119),  f(10000, 118)
```

$$\left ( 0.505836993838501, \quad 0.49993624148805993\right )$$

```python
f(13983816, 4404), f(13983816, 4403)
```

$$\left ( 0.5001280323435917, \quad 0.49997059109240716\right )$$

Eine weitere kleine Funktion berechnet für einen beliebigen $m$-Wert den zugehörigen

$k_{50}$-Wert

```python
def k50(m):
    k = 1
    while f(m, k) < 0.5:
        k += 1
    return k
```

```python
k50(250000)
```

$$589$$

Die bei Schroedel angegebene Faustregel lautet:

Führt man $k_{50} \approx 1.2 \cdot \sqrt{m}$ Zufallsversuche durch, dann ist die Wahrscheinlichkeit, dass

mindestens zwei Ergebnisse gleich sind, bereits über 50%

Die bei Schroedel angegebene Grafik einschließlich der in der Faustregel verwendeten

Funktion kann dann so erzeugt werden

```python
m = range(1, 1000, 20)
k50werte = [k50(i) for i in m]
```

```python
plt.close('all')
plt.figure(figsize=(6, 2.5))
plt.plot(m, k50werte, 'k.')
z = np.linspace(0, 1000, 100)
plt.plot(z, 1.2*np.sqrt(z), color='b', lw=0.8)
plt.xlabel('$m$', fontsize=12)
plt.ylabel('$k_{50}$', fontsize=12)

plt.show()
```

Der Vergleich der unten berechneten Näherungswerte mit den exakten Tabellenwer-

ten zeigt eine gute Approximation durch die Funktion  (Aufgabe 5b)

```python
[(m, (1.2*np.sqrt(m))) for m in (400, 900, 1600, 2500, 10000, 250000, 1000000, 13983816)]
```

$$\left [ \left ( 400, \quad 24.0\right ), \quad \left ( 900, \quad 36.0\right ), \quad \left ( 1600, \quad 48.0\right ), \quad \left ( 2500, \quad 60.0\right ), \quad \left ( 10000, \quad 120.0\right ), \quad \left ( 250000, \quad 600.0\right ), \quad \left ( 1000000, \quad 1200.0\right ), \quad \left ( 13983816, \quad 4487.39290011\right )\right ]$$

Bei $k = 2 \cdot \sqrt{m}$ erreicht die Wahrscheinlichkeit für mindestens zwei gleiche Ergebnis-

se fast 90%  (Aufgabe 5c)

Damit die Funktion $f$ auch für irrationale $k$ benutzt werden kann, wird

sie leicht modifiziert

```python
from sympy import ceiling

def ff(m, k):
    q = 1
    for i in range(ceiling(m-k+1), m):
        q *= i/m
    return 1 - q
```

```python
[ff(m, 2*np.sqrt(m)) for m in (400, 900, 1600, 2500, 10000, 250000, 1000000, 13983816)]
```

$$\left [ 0.8670200951442043, \quad 0.8662119073603859, \quad 0.8658167163216178, \quad 0.8655823510626799, \quad 0.8651196385197779, \quad 0.8647550910142474, \quad 0.8647098661620823, \quad 0.8646052358027866\right ]$$

```python

```