Note on Econometrics

Haixi Li

## Contents
- [Finite Sample Property](#Finite-Sample_Property)
- [Large Sample Theory](#Large-Sample-Theory)
- [Time Series Analysis](#Time-Series-Analysis)
- [Inverting Lag Polynomials](#Inverting-Lag-Polynomials)

## Assumption for Classic Linear Regression
The classic linear regression model starts with a few assumptions
#### Assumption 1.1 Linearity
\begin{equation}
y_{t} = X'\beta + \epsilon_{t}
\end{equation}
#### Assumption 1.2 Strict Exogeneity
\begin{equation}
E(\epsilon_{i}\mid X) = 0 \text{ for } i=1,2,...,n.
\end{equation}
#### Assumption 1.3 No Multicollinerity
The rank of the $n\times K$ data matrix, $X$ is $K$ with probability 1.
#### Assumption 1.4 Spherical Error Variance
homoskedasticity
\begin{equation}
E(\epsilon_{i}^{2}\mid X)=\sigma^{2} \text{ for } i=1,2,...,n
\end{equation}
no correlation betwee observations
\begin{equation}
E(\epsilon_{i}\epsilon_{j}) = 0 \, (i,j=1,2,...,n. i \neq j)
\end{equation}
#### Assumption 1.5 Normality of the Error Term
The distribution of $\epsilon$ conditional on $X$ is jointly normal.

Assumption 1.5 together with 1.2 and 1.4, we have
\begin{equation}
\epsilon \mid X \sim \mathcal{N}(0,\sigma^{2}I_{n})
\end{equation}
Given the 1.2 exogeneity assumption, we can conclude that the distribution of $\epsilon$ is independent of $X$. Therefore, in particular, the marginal or unconditional distribution of $epsilon$ is
\begin{equation}
\epsilon \sim \mathcal{N}(0,\sigma^{2}I_{n})
\end{equation}

## Finite-Sample Properties of OLS
#### Unbiasedness
under the assumption 1.1-1.3 $E(\hat{\beta}\mid X)=\beta$
\begin{proof}

$E(\hat{\beta}-\beta\mid X)=E((X'X)^{-1}X'y-\beta\mid X)$

$E((X'X)^{-1}X'(X\beta+\epsilon)-\beta \mid X)$

$E((X'X)^{-1}X'\epsilon \mid X)=(X'X)^{-1}X'E(\epsilon\mid X)=0$
\end{proof}
#### Expression for the Variance
under the assumption 1.1-1.4 $var(\hat{\beta}\mid X)=\sigma^{2}(X'X)^{-1}$.
\begin{proof}
\end{proof}

## Errors vs Residuals
Errors pertain to the true data generating process (DGP), whereas residuals are what is left over after having estimated your model. In truth, assumptions like normality, homoscedasticity, and independence apply to the errors of the DGP, not your model's residuals.

## Finite Sample Property
The finite sample property rely on distributional assumptions to derive the distribution for estimators. Large sample theory, on the contrast, due to Central Limit Theorom, can relax distributional assumptions.

## Large Sample Theory

## Time Series Analysis
A white noise process $\varepsilon_{t}$ is a zero mean and covariance stationary process with no serial correlation:

\begin{eqnarray}
E(\varepsilon_{t}) &=& 0\\
E(\varepsilon_{t}^{2}) &=&\sigma^{2}\\
E(\varepsilon_{t}\varepsilon_{t-j}) &=& 0 \, \text{for} \, j \neq 0.
\end{eqnarray}

A very import class of covariance stationary processes, called linear processes, can be created by taking a moving average of a white noise process.
We say a value is random, that means the variable has a probability distribution associated with $t$.
The reason for this is we have not other

\begin{equation}
\alpha + \beta =\sum x_{i} \label{eq2}
\end{equation}

We will refer this equation as $\ref{eq2}$ as the second equation.

## Inverting Lag Polynomials
The $pth$ degree of lag operator

\begin{equation}
\phi(L) = 1 - \phi_{1}L -\phi_{2}L^{2}-...-\phi_{p}L^{P} \label{eq1}
\end{equation}

is invertable if the p-th degree polynomial equation in $z$

\begin{equation}
\phi(z) = 0 \, \text{where} \, \phi(z)\equiv 1-\phi_{1}z-\phi_{2}z^{2}-...-\phi_{p}z^{p}
\end{equation}

are greater than 1 in absolute value.
we are referring $\ref{eq1}$
### Large Sample Theory
#### Convergence in Probability
A sequence of random scalar variables $\{ z_{n}\}$ converges in probability to a constant $\alpha$, if for any $\varepsilon > 0$,
\begin{equation}
\lim_{n \rightarrow \infty} \text{Prob}(\mid z_{n} - \alpha\mid > \varepsilon) = 0.
\end{equation}
The constant $\alpha$ is called the probability limit of $z_{n}$. Each $z_{n}$ is a random variable, hence has an distribution associated with it. As $n$ goes to infinity, the distribution will collapses around $\alpha$. The probability mass will center around $\alpha$.

## Time Series
### Basic Concept
##### Stochastic Process
A stochastic process is a sequence of random variables. $\{z_{t}\}$ is a sequence of random variables, then its realization is a sequence of real number.

The fundamental problem of time series analysis is we can observe the realization of the process only once. If the all the realization of the process comes from the same distribution, then we can still learn from the time series about the distribution which is the realization of the same distribution at different time.

Furthermore, if the process is not too persistent, that each realization represent some information others don't have. This property is called ergodicity.
##### Ststionarity
Strick stationarity means the probability distribution does not change across the stochastic process.
Weak stationarity suggest $E(z_{t}) \, \text{does not depend on t}$

\begin{equation}
cov(z_{t},z_{t-j}) \, \text{exists, in finite, and depends only on j but not on t}
\end{equation}
##### Ergodic
Heuristically, a stationary process is ergodic if it is asymptotically independent, that is, if any two random variables positioned far apart in the sequence are almost independently distributed. Ergodic stationary will be an integral ingredient in developing large-sample theory because of the following property.

```python

```


# FFM234, Klassisk fysik och vektorfält - Veckans tal

**Christin Rhen och Christian Forssén, Institutionen för  fysik, Chalmers**

Date: **Aug 10, 2019**

## Uppgift 2.4.14

Ett kroklinjigt koordinatsystem $u,v,w$ ges av sambanden

$$
u=r(1-\cos\theta),\nonumber
$$

$$
\begin{equation}
    v=r(1+\cos\theta),\label{uvw} \tag{1}
\end{equation}
$$

$$
w=\varphi,\nonumber
$$

där $r,\theta,\varphi$ är sfäriska koordinater. Visa att systemet är ortogonalt och beräkna dess skalfaktorer. Hur ser gradientoperatorn $\nabla$ och ortvektorn $\vec r$ ut i $u,v,w$-systemet?

**Answer.**
Skalfaktorer:

$$
\begin{align*}
h_u  &=  \frac{1}{2} \sqrt{\frac{u+v}{u}} \\
h_v  &=  \frac{1}{2} \sqrt{\frac{u+v}{v}} \\
h_w  &=  \sqrt{uv}
\end{align*}
$$

Gradient:

$$
\nabla = \frac{2}{\sqrt{u+v}}\left(\sqrt{u} \hat u
\frac{\partial}{\partial u} + \sqrt{v} \hat v
\frac{\partial}{\partial v}\right) + \frac{1}{\sqrt{uv}} \hat w
\frac{\partial}{\partial w}
$$

Ortsvektor:

$$
\vec{r} = \frac{\sqrt{u+v}}{2}(\sqrt{u} \hat u + \sqrt{v} \hat v)
$$

**Solution.**
**Enhetsvektorer**
Enhetsvektorer ges av (långt upp på sida 12 i kurskompendiet)

$$
\begin{equation}
    \vec e_i=h_i\nabla u_i.
\label{eq:ehat} \tag{2}
\end{equation}
$$

Här är $u,v,w$ en funktion av sfäriska koordinater, så vi använder den sfäriska gradienten (kurskompendium ekvation 2.14):

$$
\hat u\propto\nabla u= \left(\hat r\frac{\partial}{\partial r}+\hat\theta\frac 1r\frac{\partial}{\partial \theta}+\hat\varphi\frac1{r\sin\theta}\frac{\partial}{\partial \varphi}\right)u\nonumber
$$

$$
\begin{equation}
    =(1-\cos\theta)\hat r+\sin\theta\hat\theta,
\label{_auto1} \tag{3}
\end{equation}
$$

$$
\begin{equation}
    \hat v\propto\nabla v=(1+\cos\theta)\hat r-\sin\theta\hat\theta,
\label{_auto2} \tag{4}
\end{equation}
$$

$$
\begin{equation}
    \hat w\propto\nabla w=\hat\varphi\frac1{r\sin\theta}.
\label{_auto3} \tag{5}
\end{equation}
$$

**Ortogonalitet**
Det är lättast att kontrollera om ett nytt system är ortogonalt om vi har dess basvektorer uttryckta i ett annat, mer välkänt, system. Därför låter vi här $\hat u, \hat v,\hat w$ fortsätta vara en funktion av sfäriska koordinater och basvektorer, och räknar ut skalärprodukterna mellan dem. Vi ser direkt att $\hat u\cdot \hat w=\hat v\cdot\hat w=0$. För $\hat u$ och $\hat v$:

$$
\hat u\cdot\hat v\propto(1-\cos\theta)(1+\cos\theta)-\sin^2\theta\nonumber
$$

$$
\begin{equation}
    =1-\cos^2\theta-\sin^2\theta=0.
\label{_auto4} \tag{6}
\end{equation}
$$

Alltså är $u,v,w$-systemet ortogonalt.

**Skalfaktorer och enhetsvektorer**
Från ekvation ([eq:ehat](#eq:ehat)) inser vi att skalfaktorerna $h_i$ är inversen av $|\nabla u_i|$. För enkelhetens skull fortsätter vi räkna i termer av sfäriska koordinater, och översätter till $u,v,w$ i slutet. Notera att $u+v=2r$. Vi får

$$
h_u=\left((1-\cos\theta)^2+\sin^2\theta\right)^{-1/2}=\left(2-2\cos\theta\right)^{-1/2}\nonumber
$$

$$
\begin{equation}
    =\sqrt{\frac{u+v}{4u}},
\label{_auto5} \tag{7}
\end{equation}
$$

$$
h_v=\left((1+\cos\theta)^2+\sin^2\theta\right)^{-1/2}=\left(2+2\cos\theta\right)^{-1/2}\nonumber
$$

$$
\begin{equation}
    =\sqrt{\frac{u+v}{4v}},
\label{_auto6} \tag{8}
\end{equation}
$$

$$
\begin{equation}
    h_w=\sqrt{r^2\sin^2\theta}=\sqrt{r^2(1-\cos^2\theta)}=\sqrt{uv}.
\label{_auto7} \tag{9}
\end{equation}
$$

Kombinerar vi skalfaktorerna (i sfäriska koordinater) och de onormerade enhetvektorerna uträknade ovan så får vi

$$
\begin{equation}
    \hat u= \frac{1-\cos\theta}{\sqrt{2-2\cos\theta}}\hat r+\frac{\sin\theta}{\sqrt{2-2\cos\theta}}\hat\theta,
\label{_auto8} \tag{10}
\end{equation}
$$

$$
\begin{equation}
    \hat v= \frac{1+\cos\theta}{\sqrt{2+2\cos\theta}}\hat r-\frac{\sin\theta}{\sqrt{2+2\cos\theta}}\hat\theta,
\label{_auto9} \tag{11}
\end{equation}
$$

$$
\begin{equation}
    \hat w=\hat\varphi.
\label{eq:enhetsvektorer} \tag{12}
\end{equation}
$$

**Gradientoperator**
För ortogonala koordinatsystem gäller att (kurskompendium ekvation 2.13)

$$
\begin{equation}
    \nabla\phi=\sum_i\vec e_i\frac 1{h_i}\frac{\partial\phi}{\partial u_i},
\label{_auto10} \tag{13}
\end{equation}
$$

där $\phi$ är ett godtyckligt skalärfält. I vårt $u,v,w$-system blir detta

$$
\begin{equation}
    \nabla\phi=\hat u\frac 1{h_u}\frac{\partial\phi}{\partial u}+\hat v\frac 1{h_v}\frac{\partial\phi}{\partial v}+\hat w\frac 1{h_w}\frac{\partial\phi}{\partial w}.
\label{_auto11} \tag{14}
\end{equation}
$$

Sätter vi in skalfaktorerna vi beräknat ovan är det lätt att identifiera gradientoperatorn

$$
\begin{equation}
    \nabla = \hat u\sqrt{\frac{4u}{u+v}}\frac{\partial}{\partial u}+\hat v\sqrt{\frac{4v}{u+v}}\frac{\partial}{\partial v}+\hat w\frac 1{\sqrt{uv}}\frac{\partial}{\partial w}.
\label{_auto12} \tag{15}
\end{equation}
$$

Notera att den inversa skalfaktorn alltid kommer före partialderivatan! Skriver man dem i fel ordning ska plötsligt även skalfaktorn deriveras, och det blir fel.

**Ortsvektor**
I sfäriska koordinater är ortsvektorn

$$
\begin{equation}
    \vec r=r\hat r.
\label{_auto13} \tag{16}
\end{equation}
$$

Vi har redan noterat att $r=(u+v)/2$; vi behöver nu också uttrycka $\hat r$ som en funktion av $\hat u,\hat v, \hat w$.

En smart linjärkombination av enhetsvektorerna ([eq:enhetsvektorer](#eq:enhetsvektorer)):

$$
\begin{equation}
    \sqrt{2-2\cos\theta}\hat u+\sqrt{2+2\cos\theta}\hat v=2\hat r,
\label{_auto14} \tag{17}
\end{equation}
$$

som översatt till nya koordinater betyder att

$$
\begin{equation}
    \hat r=\frac12\left(\frac1{h_u}\hat u+\frac1{h_v}\hat v\right)=\sqrt{\frac u{u+v}}\hat u+\sqrt{\frac v{u+v}}\hat v.
\label{_auto15} \tag{18}
\end{equation}
$$

Vi får alltså ortsvektorn

$$
\vec r=r\hat r\nonumber
$$

$$
=\frac{u+v}2\left(\sqrt{\frac u{u+v}}\hat u+\sqrt{\frac v{u+v}}\hat v\right)\nonumber
$$

$$
\begin{equation}
    =\tfrac12\sqrt{u+v} \left( \sqrt u\hat u + \sqrt v\hat v \right).
\label{_auto16} \tag{19}
\end{equation}
$$


# Tarea 3: Diseño Avanzado de Algoritmos

## Spark: k-means

### Daniel San Martín

### Spark

Apache Spark es un framework open-source de computación en clúster de propósito general y orientado a la velocidad. Proporciona APIs en Java, Scala, Python y R.

```python
# Python version 3.7.3
import os
import numpy as np # Version 1.16.4
import pandas as pd # Version 0.24.2
import matplotlib.pyplot as plt # Version 3.1.0
from mpl_toolkits.mplot3d import Axes3D
from sklearn.decomposition import PCA # Version 0.21.2
from sklearn.preprocessing import StandardScaler
from sklearn.metrics.pairwise import cosine_similarity, cosine_distances
import time
import warnings
warnings.filterwarnings("ignore")
```

```python
DATA_DIR = 'data/dataset.csv'
```

### Funciones auxiliares

Implementación de funciones para la representacion de datos, graficar y obtener resultados.

```python
# Get data as numpy array or dictionary
def dataRepresentation(data, matrix=True):
    M, N = np.max(data['source']), np.max(data['target'])
    m = np.zeros((M, N))
    d = dict()

    # Read data and create the representation
    for index, row in data.iterrows():
        s, t, r = row['source'], row['target'], row['rating']
        m[s-1, t-1] = r
        if s not in d:
            d[s] = np.zeros(N)
        d[s][t-1] = r

    if matrix: # Return as numpy array
        # Remove rows/cols with only zeros
        m = m[~np.all(m == 0, axis=1)]
        m = m[:,~np.all(m == 0, axis=0)]
        return m
    else: # Return as dictionary
        return d
```

```python
# Results boxplots
def boxplot(data, times=True):
    N = len(data)
    fig, axes = plt.subplots(1, N, figsize=(15, 5), sharey=True)
    for k in range(N):
        axes[k].boxplot(data[k])
        axes[k].set_title("K = " + str((k+1)*10))
        axes[k].grid(True)
    plt.tight_layout()
    plt.show()
```

```python
# Scatter for 2D projection of data using PCA (only exploratory)
def scatter(data, means=None, lims=[[-10, 20], [-10, 80]]):
    fig = plt.figure(figsize=(14, 4))

    # All data
    plt.subplot(1, 2, 1)
    plt.scatter(data[:,0], data[:,1])
    if means is not None:
        plt.scatter(means[:,0], means[:,1], marker="*")
    plt.grid(True)
    plt.xlabel("PC 1")
    plt.ylabel("PC 2")

    # "Zoom"
    plt.subplot(1, 2, 2)
    plt.scatter(data[:,0], data[:,1])
    if means is not None:
        plt.scatter(means[:,0], means[:,1], marker="*")
    plt.xlim(lims[0])
    plt.ylim(lims[1])

    plt.grid(True)
    plt.xlabel("PC 1")
    plt.ylabel("PC 2")
    plt.show()
```

```python
def histograms(sim, dis):
    # Remove duplicated data for symmetry of similarity/distance matrix
    data_sim, data_dis = [], []
    M, N = sim.shape
    for i in range(M):
        for j in range(i+1, N):
            data_sim.append(sim[i, j])
            data_dis.append(dis[i, j])
    data_sim = np.array(data_sim)
    data_dis = np.array(data_dis)

    # Plots
    fig = plt.figure(figsize=(14, 4))

    # Similarity
    plt.subplot(1, 2, 1)
    plt.hist(data_sim, bins=20)
    plt.grid(True)
    plt.title("Cosine Similarity")

    # Distance
    plt.subplot(1, 2, 2)
    plt.hist(data_dis, bins=20)
    plt.grid(True)
    plt.title("Cosine Distance")
    plt.show()
```

```python
# Get results filenames
def getFiles(path, K):
    names = list()
    for k in K:
        path_k = path + str(k) + "/"
        for _, _, files in os.walk(path_k):
            for file in files:
                if '.txt' in file:
                    names.append(path_k + file)
    return names
```

```python
# Get data from results files
def getResults(path, K):
    file_names = getFiles(path, K)
    k_10, k_20, k_30, k_40, k_50 = [], [], [], [], []
    for name in file_names:
        if '/10/' in name: result = np.genfromtxt(name, delimiter=','); k_10.append(result)
        elif '/20/' in name: result = np.genfromtxt(name, delimiter=','); k_20.append(result)
        elif '/30/' in name: result = np.genfromtxt(name, delimiter=','); k_30.append(result)
        elif '/40/' in name: result = np.genfromtxt(name, delimiter=','); k_40.append(result)
        elif '/50/' in name: result = np.genfromtxt(name, delimiter=','); k_50.append(result)
    return k_10, k_20, k_30, k_40, k_50
```

### Formato de entrada

Se entrega un dataset en csv, donde cada fila contiene la siguiente información: ID_SOURCE, ID_TARGET, RATING
donde:
* ID_SOURCE: ID de usuario origen
* ID_TARGET: ID del usuario destino
* RATING: calificación del trust de ID_SOURCE sobre ID_TARGET, en $\{-10,10\}$

#### Descripción básica

Estadísticas generales de los datos.

```python
data = pd.read_csv(DATA_DIR, header=None, names=['source', 'target', 'rating'])
```

```python
data.describe()
```

      source
      target
      rating

      count
      24186.000000
      24186.000000
      24186.000000

      mean
      864.029314
      1051.093815
      1.463946

      std
      1592.085638
      1871.765504
      2.903656

      min
      1.000000
      1.000000
      -10.000000

      25%
      58.000000
      66.000000
      1.000000

      50%
      238.000000
      279.000000
      1.000000

      75%
      898.000000
      1068.000000
      2.000000

      max
      7604.000000
      7604.000000
      10.000000

### Algoritmo k–means

- Defina la función parse_vector para que lea el dataset y cree una RDD o DataFrame en Spark de vectores de usuarios. La representación vectorial de los usuarios corresponde a vectores donde cada componente es el rating. Por ejemplo, para el vector $u$ del usuario $i$, la componente $j$ corresponde a $RATING(i,j)$. Su RDD de vectores debe tener $n-1$ slices, donde $n$ es el número de cores de su máquina.

En el pre-procesamiento de datos se utilizó `dataRepresentation(data)` para forma convertir los datos según lo solicitado.

```python
#dr = dataRepresentation(data)
#np.savetxt('data/pro.csv', dr, fmt='%d', delimiter=',')
dr = np.genfromtxt('./data/pro.csv', delimiter=',')
pd.DataFrame(dr).describe()
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      ...
      3744
      3745
      3746
      3747
      3748
      3749
      3750
      3751
      3752
      3753

      count
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      ...
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.000000
      3286.00000
      3286.000000

      mean
      0.230676
      0.223676
      0.185636
      0.178941
      0.118685
      0.118077
      0.112599
      0.104991
      0.097687
      0.087645
      ...
      -0.019477
      -0.020998
      -0.027389
      -0.033475
      -0.033475
      -0.036214
      -0.036519
      -0.045648
      -0.06482
      -0.191114

      std
      0.858505
      1.169709
      0.819615
      0.903702
      0.787597
      0.739449
      0.886279
      0.652539
      0.743852
      0.523063
      ...
      0.599531
      0.462786
      0.522707
      0.759780
      0.759780
      0.811014
      0.696942
      0.717927
      1.01813
      1.446770

      min
      0.000000
      0.000000
      -2.000000
      0.000000
      -10.000000
      0.000000
      -10.000000
      0.000000
      -10.000000
      -10.000000
      ...
      -10.000000
      -10.000000
      -10.000000
      -10.000000
      -10.000000
      -10.000000
      -10.000000
      -10.000000
      -10.00000
      -10.000000

      25%
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      ...
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.00000
      0.000000

      50%
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      ...
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.00000
      0.000000

      75%
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      ...
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.00000
      0.000000

      max
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      8.000000
      ...
      7.000000
      2.000000
      0.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.000000
      10.00000
      10.000000

8 rows × 3754 columns

### Para tener alguna idea...

```python
ds = StandardScaler().fit_transform(dr)
X = np.copy(ds)
pca = PCA(n_components=2)
fit = pca.fit(X)
```

```python
dpca = pca.transform(X)
```

```python
scatter(dpca)
```

La función `parse_vector(filename, n)` recibe el nombre del archivo y la cantidad de cores de la máquina. Toma cada línea del archivo y aplica un *split* retornando un vector con $N_{c}$ `Doubles`, donde $N_{c}$ es el número de componentes del vector. Luego retorna el `RDD` de vectores con $n-1$ slices.
```scala
def parse_vector(filename: String, n: Int) = {
    val lines = sc.textFile(filename, n - 1)
    val data = lines.map(l =>
        Vector(l.split(',').map(_.toDouble))
    ).cache()
    data
}
```

```python
!sysctl hw.physicalcpu hw.logicalcpu
```

    hw.physicalcpu: 2
    hw.logicalcpu: 4

El valor escogido como número de cores fue $4$.

- Defina la función similitud de coseno en los primeros dos cuadrantes del plano cartesiano para que pueda medir la similitud entre dos vectores cuyas componentes tienen valores en el intervalo discreto $\{-10, 10\}$.

### Similitud del Coseno [1]

La similitud del Coseno calcula la similaridad como el producto punto normalizado de dos vectores $\mathbf{x}_i$ y $\mathbf{x}_j$:

\begin{equation}
    Sim(\mathbf{x}_i,\mathbf{x}_j) = \frac{\mathbf{x}_i\cdot\mathbf{x}_j}{||\mathbf{x}_i||\,||\mathbf{x}_j||}
\end{equation}

La función en `Scala` recibe dos vectores $\mathbf{x}$, $\mathbf{y}$ y retorna $Sim(\mathbf{x},\mathbf{y})$
```scala
def cosineSimilarity(x: Vector[Double], y: Vector[Double]): Double = {
    val denom = norm(x) * norm(y)
    val dotProduct = x dot y
    if (denom == 0.0) {
      0.0
    } else {
      dotProduct / denom
    }
}
```

Para el análisis previo se implementaría en `Python`de la siguiente forma, recibiendo como parámetro la matriz de datos.
```Python
def cos_sim(X):
    M, N = X.shape
    sim = np.zeros((M, M))
    for i in range(M):
        for j in range(M):
            sim[i, j] = np.dot(X[i], X[j]) / (np.linalg.norm(X[i]) * np.linalg.norm(X[j]))
    return sim
```
Ahora, dado que computacionalmente es muy costoso, se utiliza la función ```cosine_similarity(X)``` que proporciona la biblioteca ```sklearn.metrics.pairwise``` y que se encuentra optimizada.

### Distancia del Coseno

Dado que queremos analizar la "cercanía" de los vectores de usuarios, utilizaremos la distancia del coseno definida como

\begin{equation}
    Dist(\mathbf{x}_i,\mathbf{x}_j) = 1 - Sim(\mathbf{x}_i,\mathbf{x}_j)
\end{equation}

Dado que el cómputo es igual de costoso que la similitud del coseno, se utilizará la función `cosine_distances(X)` de `sklearn.metrics.pairwise`. Para el caso de `Scala` esta función se encuentra implementada como `cosineDistance(x,y)` dentro de la biblioteca `breeze.linalg.functions`.

- Implemente el algoritmo k-means en Spark para que corra sobre el dataset entregado. Puede reusar el código comentado en clases. Extiéndalo para que pueda trabajar sobre los datos de esta tarea.

Primero, una breve descripción del algoritmo.

## k-means clustering [2]

Dado un conjunto de datos $(x_1, x_2, ..., x_n)$, $x_i\in\mathbb{R}^d, \, i=1, ..., n$, $k$-means busca particionar los $n$ datos en $k$ ($\leq n$) conjuntos $S = {S_1, S_2, ..., S_k}$ minimizando el la varianza entre *clusters*. Formalmente, el objetivo es encontrar
\begin{equation}
{\underset {\mathbf {S} }{\operatorname {arg\,min} }}\sum _{i=1}^{k}\sum _{\mathbf {x} \in S_{i}}\left\|\mathbf {x} -{\boldsymbol {\mu }}_{i}\right\|^{2}={\underset {\mathbf {S} }{\operatorname {arg\,min} }}\sum _{i=1}^{k}|S_{i}|\operatorname {Var} S_{i}
\end{equation}

donde $\mu_i$ es la media de los puntos en $S_i$. Esto es equivalente a minimizar la desviación al cuadrado entre pares de puntos del mismo *cluster*:

\begin{equation}
{\underset {\mathbf {S} }{\operatorname {arg\,min} }}\sum _{i=1}^{k}\,{\frac {1}{2|S_{i}|}}\,\sum _{\mathbf {x} ,\mathbf {y} \in S_{i}}\left\|\mathbf {x} -\mathbf {y} \right\|^{2}
\end{equation}

Notar que para este trabajo se utiliza la *Distancia del Coseno* en vez de la desviación al cuadrado.

### Algoritmo

Dado un conjunto inicial de $k$ medias $m_1^{(1)}, ..., m_k^{(1)}$, el algoritmo funciona alternando los siguientes dos pasos:

1. **Asignación**: Asignar cada dato al *cluster* cuya media tenga la menor distancia, esto es intuitivamente la media "más cercana".

\begin{equation}
S_{i}^{(t)}={\big \{}x_{p}:Dist(x_{p},m_{i}^{(t)})\leq Dist(x_{p},m_{j}^{(t)})\ \forall j,1\leq j\leq k{\big \}}
\end{equation}

donde cada $x_{p}$ es asignado exactamente a un $S^{(t)}$, aun si este pudiera ser asignado a dos o más conjuntos.

2. **Actualización**: Calcular la nueva media (centroide) de los datos en los nuevos *clusters*.

\begin{equation}
    m_{i}^{(t+1)}={\frac {1}{\left|S_{i}^{(t)}\right|}}\sum _{x_{j}\in S_{i}^{(t)}}x_{j}
\end{equation}

### Implementación en Scala

```scala
// Conjunto inicial de k medias
val kPoints = data.takeSample(withReplacement=false, k, System.currentTimeMillis).toArray
var tempDist = 1.0
var iter = 0
while (tempDist > convergeDist) {
    // Asignacion de cada dato a un respectivo cluster
    val closest = data.map(p => (closestPoint(p, kPoints), (p, 1)))
    val pointStats = closest.reduceByKey{ case ((p1, c1), (p2, c2)) => (p1 + p2, c1 + c2) }

    // Actualizacion de los centroides
    val newPoints = pointStats.map{ pair =>
		(pair._1, pair._2._1 * (1.0 / pair._2._2))
	}.collectAsMap()

    // Revisar si hay 'convergencia' comparando los centroides de iter-1 con iter
    tempDist = 0.0
    for (i <- 0 until k) {
        tempDist += cosineDistance(kPoints(i), newPoints(i))
    }
    for (newP <- newPoints) kPoints(newP._1) = newP._2
    iter += 1
}
```

## Experimentación.

1.- Compute el histograma de similitudes entre los vectores ```Sim(ui,uj)```. Considere 20 bins para su histograma. Usando el histograma, defina un valor adecuado para convDist. Justifique su decisión.

```python
sim = cosine_similarity(dr)
dis = cosine_distances(dr)
```

```python
histograms(sim, dis)
```

Dado que la mayoría de puntos analizados se encuentran cercanos al valor $0$ (o $1$ si vemos el gráfico de la derecha), podemos suponer que los *rankings* de cada usuario no se parecen por lo tanto se escoge un valor de $convDist=0.001$. Dicho de otra forma, el algoritmo debería converger más o menos lento ya que los vectores tienden a ser ortogonales. Por esta razón se utiliza un valor relativamente pequeño para $convDist$ para obtener resultados en un tiempo prudente.

2.- Corra su algoritmo con los siguientes valores de $k = \{10, 20, 30, 40, 50\}$. Corra cada configuración 5 veces, registrando el número de iteraciones y el tiempo. Para cada valor de $k$, haga un box plot del tiempo de ejecución y otro del número de iteraciones. Comente sus resultados.

```python
def experiment(k, convDist=0.001):
    start = time.time()
    !bash run.sh "./data/pro.csv" $k $convDist 2>/dev/null
    end = time.time()
    e_t = end - start
    print("Elapsed time: ", e_t)
    return e_t
```

### "Live Demo"

```python
ld = experiment(10, 0.1)
```

    Iterations: 5
    Elapsed time:  55.13049912452698

```python
K = list(range(10, 51, 10))
times = np.zeros((len(K), 5))
```

### $k=10$

```python
times[0, 0] = experiment(10)
```

    Iterations: 16
    Elapsed time:  96.68977403640747

```python
times[0, 1] = experiment(10)
```

    Iterations: 17
    Elapsed time:  132.7415590286255

```python
times[0, 2] = experiment(10)
```

    Iterations: 10
    Elapsed time:  81.51960802078247

```python
times[0, 3] = experiment(10)
```

    Iterations: 31
    Elapsed time:  168.8739368915558

```python
times[0, 4] = experiment(10)
```

    Iterations: 24
    Elapsed time:  185.396479845047

### $k=20$

```python
times[1, 0] = experiment(20)
```

    Iterations: 15
    Elapsed time:  111.96453619003296

```python
times[1, 1] = experiment(20)
```

    Iterations: 15
    Elapsed time:  160.4166340827942

```python
times[1, 2] = experiment(20)
```

    Iterations: 15
    Elapsed time:  221.09233283996582

```python
times[1, 3] = experiment(20)
```

    Iterations: 18
    Elapsed time:  203.48249697685242

```python
times[1, 4] = experiment(20)
```

    Iterations: 11
    Elapsed time:  164.61798572540283

### $k=30$

```python
times[2, 0] = experiment(30)
```

    Iterations: 18
    Elapsed time:  274.56231904029846

```python
times[2, 1] = experiment(30)
```

    Iterations: 12
    Elapsed time:  187.47768115997314

```python
times[2, 2] = experiment(30)
```

    Iterations: 11
    Elapsed time:  204.5162308216095

```python
times[2, 3] = experiment(30)
```

    Iterations: 9
    Elapsed time:  201.44304585456848

```python
times[2, 4] = experiment(30)
```

    Iterations: 11
    Elapsed time:  189.88382387161255

### $k=40$

```python
times[3, 0] = experiment(40)
```

    Iterations: 10
    Elapsed time:  231.36989784240723

```python
times[3, 1] = experiment(40)
```

    Iterations: 16
    Elapsed time:  354.4967269897461

```python
times[3, 2] = experiment(40)
```

    Iterations: 15
    Elapsed time:  330.2719519138336

```python
times[3, 3] = experiment(40)
```

    Iterations: 12
    Elapsed time:  267.59576201438904

```python
times[3, 4] = experiment(40)
```

    Iterations: 8
    Elapsed time:  178.10608291625977

### $k=50$

```python
times[4, 0] = experiment(50)
```

    Iterations: 9
    Elapsed time:  240.53037309646606

```python
times[4, 1] = experiment(50)
```

    Iterations: 11
    Elapsed time:  301.00403881073

```python
times[4, 2] = experiment(50)
```

    Iterations: 13
    Elapsed time:  353.3671660423279

```python
times[4, 3] = experiment(50)
```

    Iterations: 12
    Elapsed time:  444.3880958557129

```python
times[4, 4] = experiment(50)
```

    Iterations: 11
    Elapsed time:  403.9678330421448

```python
#np.savetxt('./data/output/times.csv', times, fmt='%f', delimiter=',')
```

```python
res = getResults('./data/output/', K)
```

```python
iterations = np.array([
    [int(r[0,0]) for r in res[0]],
    [int(r[0,0]) for r in res[1]],
    [int(r[0,0]) for r in res[2]],
    [int(r[0,0]) for r in res[3]],
    [int(r[0,0]) for r in res[4]]
])
```

## Boxplots

### Tiempos de ejecución

```python
boxplot(times)
```

Es clara la necesidad de más tiempo de cómputo a medida que se incrementa el valor $k$. En la etapa de asignación es necesario que se recorran más conjuntos a medida que $k$ crece.

### Número de iteraciones

```python
boxplot(iterations, False)
```

En este caso notamos que el número de iteraciones no cambia mucho al variar el valor de $k$. Puede tener relación al valor $convDist$ que es fijo para todos los valores de $k$. La diferencia entre $k=10$ y $k>10$ posiblemente sea debido a que para mayor valor de $k$ es "más simple" asignar un punto a determinado clúster porque existen más grupos al cual asignarlos.

### Ejemplo de experimentos

```python
sc = StandardScaler().fit(dr)
```

```python
e_10 = pca.transform(sc.transform(res[0][0][1:]))
```

```python
scatter(dpca, e_10, [[-1, 5], [-5, 10]])
```

```python
e_10_2 = pca.transform(sc.transform(res[0][-1][1:]))
```

```python
scatter(dpca, e_10_2, [[-1, 5], [-5, 10]])
```

```python
e_50 = pca.transform(sc.transform(res[4][-1][1:]))
```

```python
scatter(dpca, e_50)
```

Al parecer el algoritmo agrupa los datos correctamente. Se puede notar la diferencia de centroides obtenidos debido a la inicialización de estos.

## Conclusiones

En esta tarea implementamos el algoritmo $k$-means para la manipulación de una cantidad considerable de datos utilizando el *Framework Spark*. Se obtuvieron resultados interesantes sobre todo en los tiempos de ejecución considerando el volumen de datos con el que se trabajó.

Las principal desventaja de este algoritmo es la sensibilidad a la selección de centroides iniciales, es decir, se obtienen distintos resultados según la inicialización que se utilice. Para este ejemplo se inicializó con una muestra aleatoria de los datos.

El procesamiento de grandes volumenes de datos es una problemática latente en la actualidad, por lo que herramientas como *Spark* nos permiten manipularlos de una forma un poco más cómoda. Principalmente esta herramienta evita el manejo explícito de técnicas de alto rendimiento como programación paralela o distribuida.

La mayor dificultad de este trabajo estuvo en la configuración de las herramientas y dependencias para el funcionamiento del *Framework*.

## Referencias

1. https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html
2. https://en.wikipedia.org/wiki/K-means_clustering


[Sebastian Raschka](http://sebastianraschka.com)

[Link to](https://github.com/rasbt/pattern_classification) the GitHub repository [pattern_classification](https://github.com/rasbt/pattern_classification)

```python
%load_ext watermark
```

```python
%watermark -a 'Gopala KR' -d -u -v -p scikit-learn,numpy,matplotlib
```

    Gopala KR
    last updated: 2018-01-28

    CPython 3.6.3
    IPython 6.2.1

    scikit-learn 0.19.0
    numpy 1.13.1
    matplotlib 2.0.2

    /srv/venv/lib/python3.6/site-packages/watermark/watermark.py:155: DeprecationWarning: Importing scikit-learn as `scikit-learn` has been depracated and will not be supported anymore in v1.7.0. Please use the package name `sklearn` instead.
      DeprecationWarning)

I would be happy to hear your comments and suggestions.
Please feel free to drop me a note via
[twitter](https://twitter.com/rasbt), [email](mailto:bluewoodtree@gmail.com), or [google+](https://plus.google.com/+SebastianRaschka).

# About Feature Scaling: Z-Score Standardization and Min-Max Scaling

## Sections

- [About standardization](#About-standardization)
- [About Min-Max scaling / "normalization"](#About-Min-Max-scaling-normalization)
- [Standardization or Min-Max scaling?](#Standardization-or-Min-Max-scaling?)
- [Standardizing and normalizing - how it can be done using scikit-learn](#Standardizing-and-normalizing---how-it-can-be-done-using-scikit-learn)
- [Bottom-up approaches](#Bottom-up-approaches)
- [The effect of standardization on PCA in a pattern classification task](#The-effect-of-standardization-on-PCA-in-a-pattern-classification-task)

## About standardization

[[back to top](#Sections)]

The result of **standardization** (or **Z-score normalization**) is that the features will be rescaled so that they'll have the properties of a standard normal distribution with

$\mu = 0$ and $\sigma = 1$

where $\mu$ is the mean (average) and $\sigma$ is the standard deviation from the mean; standard scores (also called ***z*** scores) of the samples are calculated as follows:

\begin{equation} z = \frac{x - \mu}{\sigma}\end{equation}

Standardizing the features so that they are centered around 0 with a standard deviation of 1 is not only important if we are comparing measurements that have different units, but it is also a general requirement for many machine learning algorithms. Intuitively, we can think of gradient descent as a prominent example
(an optimization algorithm often used in logistic regression, SVMs, perceptrons, neural networks etc.); with features being on different scales, certain weights may update faster than others since the feature values $x_j$ play a role in the weight updates

$$\Delta w_j = - \eta \frac{\partial J}{\partial w_j} = \eta \sum_i (t^{(i)} - o^{(i)})x^{(i)}_{j},$$

so that

$$w_j := w_j + \Delta w_j,$$
where $\eta$ is the learning rate, $t$ the target class label, and $o$ the actual output.
Other intuitive examples include K-Nearest Neighbor algorithms and clustering algorithms that use, for example, Euclidean distance measures -- in fact, tree-based classifier are probably the only classifiers where feature scaling doesn't make a difference.

To quote from the [`scikit-learn`](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html) documentation:

*"Standardization of a dataset is a common requirement for many machine learning estimators: they might behave badly if the individual feature do not more or less look like standard normally distributed data (e.g. Gaussian with 0 mean and unit variance)."*

## About Min-Max scaling

[[back to top](#Sections)]

An alternative approach to Z-score normalization (or standardization) is the so-called **Min-Max scaling** (often also simply called "normalization" - a common cause for ambiguities).
In this approach, the data is scaled to a fixed range - usually 0 to 1.
The cost of having this bounded range - in contrast to standardization - is that we will end up with smaller standard deviations, which can suppress the effect of outliers.

A Min-Max scaling is typically done via the following equation:

\begin{equation} X_{norm} = \frac{X - X_{min}}{X_{max}-X_{min}} \end{equation}

## Z-score standardization or Min-Max scaling?

[[back to top](#Sections)]

*"Standardization or Min-Max scaling?"* - There is no obvious answer to this question: it really depends on the application.

For example, in clustering analyses, standardization may be especially crucial in order to compare similarities between features based on certain distance measures. Another prominent example is the Principal Component Analysis, where we usually prefer standardization over Min-Max scaling, since we are interested in the components that maximize the variance (depending on the question and if the PCA computes the components via the correlation matrix instead of the covariance matrix; [but more about PCA in my previous article](http://sebastianraschka.com/Articles/2014_pca_step_by_step.html)).

However, this doesn't mean that Min-Max scaling is not useful at all! A popular application is image processing, where pixel intensities have to be normalized to fit within a certain range (i.e., 0 to 255 for the RGB color range). Also, typical neural network algorithm require data that on a 0-1 scale.

## Standardizing and normalizing - how it can be done using scikit-learn

[[back to top](#Sections)]

Of course, we could make use of NumPy's vectorization capabilities to calculate the z-scores for standardization and to normalize the data using the equations that were mentioned in the previous sections. However, there is an even more convenient approach using the preprocessing module from one of Python's open-source machine learning library [scikit-learn](http://scikit-learn.org ).

```python
%watermark -a 'Sebastian Raschka' -d -p scikit-learn,numpy,pandas,matplotlib -v
```

    /srv/venv/lib/python3.6/site-packages/watermark/watermark.py:155: DeprecationWarning: Importing scikit-learn as `scikit-learn` has been depracated and will not be supported anymore in v1.7.0. Please use the package name `sklearn` instead.
      DeprecationWarning)

    Sebastian Raschka 2018-01-28

    CPython 3.6.3
    IPython 6.2.1

    scikit-learn 0.19.0
    numpy 1.13.1
    pandas 0.20.3
    matplotlib 2.0.2

[More information](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/ipython_magic/watermark.ipynb) about the `watermark` magic command extension.

For the following examples and discussion, we will have a look at the free "Wine" Dataset that is deposited on the UCI machine learning repository
(http://archive.ics.uci.edu/ml/datasets/Wine).

**Reference:**
Forina, M. et al, PARVUS - An Extendible Package for Data
Exploration, Classification and Correlation. Institute of Pharmaceutical
and Food Analysis and Technologies, Via Brigata Salerno,
16147 Genoa, Italy.

Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

The Wine dataset consists of 3 different classes where each row correspond to a particular wine sample.

The class labels (1, 2, 3) are listed in the first column, and the columns 2-14 correspond to 13 different attributes (features):

1) Alcohol
2) Malic acid
...

#### Loading the wine dataset

```python
import pandas as pd
import numpy as np

df = pd.io.parsers.read_csv(
    'https://raw.githubusercontent.com/rasbt/pattern_classification/master/data/wine_data.csv',
     header=None,
     usecols=[0,1,2]
    )

df.columns=['Class label', 'Alcohol', 'Malic acid']

df.head()
```

      Class label
      Alcohol
      Malic acid

      0
      1
      14.23
      1.71

      1
      1
      13.20
      1.78

      2
      1
      13.16
      2.36

      3
      1
      14.37
      1.95

      4
      1
      13.24
      2.59

As we can see in the table above, the features **Alcohol** (percent/volumne) and **Malic acid** (g/l) are measured on different scales, so that ***Feature Scaling*** is necessary important prior to any comparison or combination of these data.

#### Standardization and Min-Max scaling

```python
from sklearn import preprocessing

std_scale = preprocessing.StandardScaler().fit(df[['Alcohol', 'Malic acid']])
df_std = std_scale.transform(df[['Alcohol', 'Malic acid']])

minmax_scale = preprocessing.MinMaxScaler().fit(df[['Alcohol', 'Malic acid']])
df_minmax = minmax_scale.transform(df[['Alcohol', 'Malic acid']])
```

```python
print('Mean after standardization:\nAlcohol={:.2f}, Malic acid={:.2f}'
      .format(df_std[:,0].mean(), df_std[:,1].mean()))
print('\nStandard deviation after standardization:\nAlcohol={:.2f}, Malic acid={:.2f}'
      .format(df_std[:,0].std(), df_std[:,1].std()))
```

    Mean after standardization:
    Alcohol=-0.00, Malic acid=-0.00

    Standard deviation after standardization:
    Alcohol=1.00, Malic acid=1.00

```python
print('Min-value after min-max scaling:\nAlcohol={:.2f}, Malic acid={:.2f}'
      .format(df_minmax[:,0].min(), df_minmax[:,1].min()))
print('\nMax-value after min-max scaling:\nAlcohol={:.2f}, Malic acid={:.2f}'
      .format(df_minmax[:,0].max(), df_minmax[:,1].max()))
```

    Min-value after min-max scaling:
    Alcohol=0.00, Malic acid=0.00

    Max-value after min-max scaling:
    Alcohol=1.00, Malic acid=1.00

#### Plotting

```python
%matplotlib inline
```

```python
from matplotlib import pyplot as plt

def plot():
    plt.figure(figsize=(8,6))

    plt.scatter(df['Alcohol'], df['Malic acid'],
            color='green', label='input scale', alpha=0.5)

    plt.scatter(df_std[:,0], df_std[:,1], color='red',
            label='Standardized [$N  (\mu=0, \; \sigma=1)$]', alpha=0.3)

    plt.scatter(df_minmax[:,0], df_minmax[:,1],
            color='blue', label='min-max scaled [min=0, max=1]', alpha=0.3)

    plt.title('Alcohol and Malic Acid content of the wine dataset')
    plt.xlabel('Alcohol')
    plt.ylabel('Malic Acid')
    plt.legend(loc='upper left')
    plt.grid()

    plt.tight_layout()

plot()
plt.show()
```

The plot above includes the wine datapoints on all three different scales: the input scale where the alcohol content was measured in volume-percent (green), the standardized features (red), and the normalized features (blue).
In the following plot, we will zoom in into the three different axis-scales.

```python
fig, ax = plt.subplots(3, figsize=(6,14))

for a,d,l in zip(range(len(ax)),
               (df[['Alcohol', 'Malic acid']].values, df_std, df_minmax),
               ('Input scale',
                'Standardized [$N  (\mu=0, \; \sigma=1)$]',
                'min-max scaled [min=0, max=1]')
                ):
    for i,c in zip(range(1,4), ('red', 'blue', 'green')):
        ax[a].scatter(d[df['Class label'].values == i, 0],
                  d[df['Class label'].values == i, 1],
                  alpha=0.5,
                  color=c,
                  label='Class %s' %i
                  )
    ax[a].set_title(l)
    ax[a].set_xlabel('Alcohol')
    ax[a].set_ylabel('Malic Acid')
    ax[a].legend(loc='upper left')
    ax[a].grid()

plt.tight_layout()

plt.show()
```

## Bottom-up approaches

[[back to top](#Sections)]

Of course, we can also code the equations for standardization and 0-1 Min-Max scaling "manually". However, the scikit-learn methods are still useful if you are working with test and training data sets and want to scale them equally.

E.g.,

std_scale = preprocessing.StandardScaler().fit(X_train)
X_train = std_scale.transform(X_train)
X_test = std_scale.transform(X_test)

Below, we will perform the calculations using "pure" Python code, and an more convenient NumPy solution, which is especially useful if we attempt to transform a whole matrix.

Just to recall the equations that we are using:

Standardization: \begin{equation} z = \frac{x - \mu}{\sigma} \end{equation}

with mean:

\begin{equation}\mu = \frac{1}{N} \sum_{i=1}^N (x_i)\end{equation}

and standard deviation:

\begin{equation}\sigma = \sqrt{\frac{1}{N} \sum_{i=1}^N (x_i - \mu)^2}\end{equation}

Min-Max scaling: \begin{equation} X_{norm} = \frac{X - X_{min}}{X_{max}-X_{min}} \end{equation}

### Pure Python

```python
# Standardization

x = [1,4,5,6,6,2,3]
mean = sum(x)/len(x)
std_dev = (1/len(x) * sum([ (x_i - mean)**2 for x_i in x]))**0.5

z_scores = [(x_i - mean)/std_dev for x_i in x]

# Min-Max scaling

minmax = [(x_i - min(x)) / (max(x) - min(x)) for x_i in x]
```

### NumPy

```python
import numpy as np

# Standardization

x_np = np.asarray(x)
z_scores_np = (x_np - x_np.mean()) / x_np.std()

# Min-Max scaling

np_minmax = (x_np - x_np.min()) / (x_np.max() - x_np.min())
```

### Visualization

Just to make sure that our code works correctly, let us plot the results via matplotlib.

```python
%matplotlib inline
```

```python
from matplotlib import pyplot as plt

fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(nrows=2, ncols=2, figsize=(10,5))

y_pos = [0 for i in range(len(x))]

ax1.scatter(z_scores, y_pos, color='g')
ax1.set_title('Python standardization', color='g')

ax2.scatter(minmax, y_pos, color='g')
ax2.set_title('Python Min-Max scaling', color='g')

ax3.scatter(z_scores_np, y_pos, color='b')
ax3.set_title('Python NumPy standardization', color='b')

ax4.scatter(np_minmax, y_pos, color='b')
ax4.set_title('Python NumPy Min-Max scaling', color='b')

plt.tight_layout()

for ax in (ax1, ax2, ax3, ax4):
    ax.get_yaxis().set_visible(False)
    ax.grid()

plt.show()
```

## The effect of standardization on PCA in a pattern classification task

[[back to top](#Sections)]

Earlier, I mentioned the Principal Component Analysis (PCA) as an example where standardization is crucial, since it is "analyzing" the variances of the different features.
Now, let us see how the standardization affects PCA and a following supervised classification on the whole wine dataset.

In the following section, we will go through the following steps:

- Reading in the dataset
- Dividing the dataset into a separate training and test dataset
- Standardization of the features
- Principal Component Analysis (PCA) to reduce the dimensionality
- Training a naive Bayes classifier
- Evaluating the classification accuracy with and without standardization

### Reading in the dataset

[[back to top](#Sections)]

```python
import pandas as pd

df = pd.io.parsers.read_csv(
    'https://raw.githubusercontent.com/rasbt/pattern_classification/master/data/wine_data.csv',
    header=None,
    )
```

### Dividing the dataset into a separate training and test dataset

[[back to top](#Sections)]

In this step, we will randomly divide the wine dataset into a training dataset and a test dataset where the training dataset will contain 70% of the samples and the test dataset will contain 30%, respectively.

```python
from sklearn.cross_validation import train_test_split

X_wine = df.values[:,1:]
y_wine = df.values[:,0]

X_train, X_test, y_train, y_test = train_test_split(X_wine, y_wine,
    test_size=0.30, random_state=12345)
```

    /srv/venv/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
      "This module will be removed in 0.20.", DeprecationWarning)

### Feature Scaling - Standardization

[[back to top](#Sections)]

```python
from sklearn import preprocessing

std_scale = preprocessing.StandardScaler().fit(X_train)
X_train_std = std_scale.transform(X_train)
X_test_std = std_scale.transform(X_test)
```

### Dimensionality reduction via Principal Component Analysis (PCA)

[[back to top](#Sections)]

Now, we perform a PCA on the standardized and the non-standardized datasets to transform the dataset onto a 2-dimensional feature subspace.
In a real application, a procedure like cross-validation would be done in order to find out what choice of features would yield a optimal balance between "preserving information" and "overfitting" for different classifiers. However, we will omit this step since we don't want to train a perfect classifier here, but merely compare the effects of standardization.

```python
from sklearn.decomposition import PCA

# on non-standardized data
pca = PCA(n_components=2).fit(X_train)
X_train = pca.transform(X_train)
X_test = pca.transform(X_test)

# om standardized data
pca_std = PCA(n_components=2).fit(X_train_std)
X_train_std = pca_std.transform(X_train_std)
X_test_std = pca_std.transform(X_test_std)
```

Let us quickly visualize how our new feature subspace looks like (note that class labels are not considered in a PCA - in contrast to a Linear Discriminant Analysis - but I will add them in the plot for clarity).

```python
%matplotlib inline
```

```python
from matplotlib import pyplot as plt

fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(10,4))

for l,c,m in zip(range(1,4), ('blue', 'red', 'green'), ('^', 's', 'o')):
    ax1.scatter(X_train[y_train==l, 0], X_train[y_train==l, 1],
        color=c,
        label='class %s' %l,
        alpha=0.5,
        marker=m
        )

for l,c,m in zip(range(1,4), ('blue', 'red', 'green'), ('^', 's', 'o')):
    ax2.scatter(X_train_std[y_train==l, 0], X_train_std[y_train==l, 1],
        color=c,
        label='class %s' %l,
        alpha=0.5,
        marker=m
        )

ax1.set_title('Transformed NON-standardized training dataset after PCA')
ax2.set_title('Transformed standardized training dataset after PCA')

for ax in (ax1, ax2):

    ax.set_xlabel('1st principal component')
    ax.set_ylabel('2nd principal component')
    ax.legend(loc='upper right')
    ax.grid()
plt.tight_layout()

plt.show()
```

### Training a naive Bayes classifier

[[back to top](#Sections)]

We will use a naive Bayes classifier for the classification task. If you are not familiar with it, the term "naive" comes from the assumption that all features are "independent".
All in all, it is a simple but robust classifier based on Bayes' rule

Bayes' Rule:

\begin{equation} P(\omega_j|x) = \frac{p(x|\omega_j) * P(\omega_j)}{p(x)} \end{equation}

where

- ω:  class label
- *P(ω|x)*: the posterior probability
- *p(x|ω)*: prior probability (or likelihood)

and the **decsion rule:**

Decide $ \omega_1 $ if $ P(\omega_1|x) > P(\omega_2|x) $ else decide $ \omega_2 $.

\begin{equation}
\Rightarrow \frac{p(x|\omega_1) * P(\omega_1)}{p(x)} > \frac{p(x|\omega_2) * P(\omega_2)}{p(x)}
\end{equation}

I don't want to get into more detail about Bayes' rule in this article, but if you are interested in a more detailed collection of examples, please have a look at the [Statistical Patter Classification](https://github.com/rasbt/pattern_classification#statistical-pattern-recognition-examples) in my pattern classification repository.

```python
from sklearn.naive_bayes import GaussianNB

# on non-standardized data
gnb = GaussianNB()
fit = gnb.fit(X_train, y_train)

# on standardized data
gnb_std = GaussianNB()
fit_std = gnb_std.fit(X_train_std, y_train)
```

### Evaluating the classification accuracy with and without standardization

[[back to top](#Sections)]

```python
from sklearn import metrics

pred_train = gnb.predict(X_train)

print('\nPrediction accuracy for the training dataset')
print('{:.2%}'.format(metrics.accuracy_score(y_train, pred_train)))

pred_test = gnb.predict(X_test)

print('\nPrediction accuracy for the test dataset')
print('{:.2%}\n'.format(metrics.accuracy_score(y_test, pred_test)))
```

    Prediction accuracy for the training dataset
    81.45%

    Prediction accuracy for the test dataset
    64.81%

```python
pred_train_std = gnb_std.predict(X_train_std)

print('\nPrediction accuracy for the training dataset')
print('{:.2%}'.format(metrics.accuracy_score(y_train, pred_train_std)))

pred_test_std = gnb_std.predict(X_test_std)

print('\nPrediction accuracy for the test dataset')
print('{:.2%}\n'.format(metrics.accuracy_score(y_test, pred_test_std)))
```

    Prediction accuracy for the training dataset
    96.77%

    Prediction accuracy for the test dataset
    98.15%

As we can see, the standardization prior to the PCA definitely led to an decrease in the empirical error rate on classifying samples from test dataset.

```python

```

```python

```

```python

```

```python

```

```python
test complete; Gopal
```


# Columns Static Solution - Loads Combos
## *Matteo Franzoi* - Anno Accademico 2019/2020
### matricola 166788 (triennale)
---

```python
    import math
#import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
from sympy import *
from sympy import MatrixSymbol, Matrix
import numpy as np
from engineering_notation import EngNumber
from decimal import Decimal
import pandas as pd
```

```python
2.75+3.5+3.1+3.1-(.04+.16+.05)
```

    12.2

```python
3.1-(.04+.16+.05)
```

    2.85

```python
25*.3*.3*12.2
```

    27.45

---
## P27
### SLU

```python
N1max_slu = 1.3*(4*21.15+27.45+101.52+90.24+101.536+99.50)+1.5*(80.09+130.284+146.59+127.68)+1.5*(67.96+0*14.1+56.40+95.17+110.54+.6*3.9)
N1max_slu
```

    1881.8808000000001

```python
N1min_slu = 1.0*(4*21.15+27.45+101.52+90.24+101.536+99.50)+.8*(80.09+130.284+146.59+127.68)+0*(67.96+0*14.1+.7*56.40+.7*95.17+.7*110.54)-1.5*3.9
N1min_slu
```

    886.7112000000001

```python
l1 = 3.1 - .04 - .16 - .05
l2 = 3.1
l3 = 3.5
l4 = 2.75
```

```python
s = np.array([0,l1, l1, l1+l2, l1+l2, l1+l2+l3, l1+l2+l3, l1+l2+l3+l4])
```

```python
NG1 = np.array([101.52+21.15, 101.52+21.15+25*.3**2 * l1, 101.52+21.15+90.24+21.15+25*.3**2 * l1, 101.52+3*21.15+90.24+25*.3**2 * (l1+l2), 101.52+3*21.15+90.24+25*.3**2 * (l1+l2) +101.536, 101.52+4*21.15+90.24+25*.3**2 * (l1+l2+l3) +101.536, 101.52+4*21.15+90.24+25*.3**2 * (l1+l2+l3) +101.536 + 99.50,  101.52+4*21.15+90.24+25*.3**2 * (l1+l2+l3+l4) +101.536 + 99.50])
```
plt.figure(figsize=(20,10))
plt.plot(NG1,s,12.2, color='#0f0f0f80')
plt.fill_between(NG1,12.2,s, color='#0f0f0f80')
plt.xlim(122, 507)
plt.ylim(0, l1+l2+l3+l4)
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$NG1(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.axhline(0, color='black')
plt.title('NG1', fontsize='22')
plt.show()

```python
NG2 = np.array([80.09, 80.09, 80.09+130.284, 80.09+130.284, 80.09+130.284+146.59, 80.09+130.284+146.59, 80.09+130.284+146.59 + 127.68, 80.09+130.284+146.59 + 127.68])
```
plt.figure(figsize=(20,10))
plt.plot(NG2,s,12.2, color='#0f0f0f80')
plt.fill_between(NG2,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(NG2), np.max(NG2))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$NG2(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.axhline(0, color='black')
plt.title('NG2', fontsize='22')
plt.show()

```python
NQ = np.array([67.96+0*14.1+.6*3.9, 67.96+0*14.1+.6*3.9, 67.96+0*14.1+56.40+.6*3.9, 67.96+0*14.1+56.40+.6*3.9, 67.96+0*14.1+.7*56.40+95.17+.6*3.9, 67.96+0*14.1+.7*56.40+95.17+.6*3.9, 67.96+0*14.1+56.40+95.17+110.54+.6*3.9, 67.96+0*14.1+56.40+95.17+110.54+.6*3.9])
```
plt.figure(figsize=(20,10))
plt.plot(NQ,s,12.2, color='#0f0f0f80')
plt.fill_between(NQ,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(NQ), np.max(NQ))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$NQ(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.axhline(0, color='black')
plt.title('NQ', fontsize='22')
plt.show()

```python
N1max_slu = 1.3*NG1 + 1.5*NG2 + 1.5*NQ
N1max_slu
```

    array([ 385.056  ,  393.39225,  818.22525,  854.78775, 1324.04455,
           1361.77705, 1873.83705, 1881.8808 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N1max_slu,s,12.2, color='#0f0f0f80')
plt.fill_between(N1max_slu,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1max_slu), np.max(N1max_slu))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1max_slu[0]),float(N1max_slu[3]), float(N1max_slu[5]), float(N1max_slu[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P27\nCombinazione SLU', fontsize='22')
plt.savefig('export/img/P27_maxAxialLoad_slu.jpg')
plt.show()
```

```python
N1min_slu = 1*NG1 + .8*NG2 + 0*NQ - 1.5*3.9
print(N1min_slu)
```

    [180.892  187.3045 402.9217 431.0467 649.8547 678.8797 880.5237 886.7112]

```python
plt.figure(figsize=(20,10))
plt.plot(N1min_slu,s,12.2, color='#0f0f0f80')
plt.fill_between(N1min_slu,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1min_slu), np.max(N1min_slu))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1min_slu[0]),float(N1min_slu[3]), float(N1min_slu[5]), float(N1min_slu[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Aione assiale minima (di compressione) sul pilastro P27\nCombinazione SLU', fontsize='22')
plt.savefig('export/img/P27_minAxialLoad_slu.jpg')
plt.show()
```

---
### SLE RARA

```python
N1max_sleRara = (4*21.15+27.45+101.52+90.24+101.536+99.50)+(80.09+130.284+146.59+127.68)+(67.96+0*14.1+56.40+95.17+110.54+.6*3.9)
N1max_sleRara

```

    1321.9

```python
N1max_sleRara = NG1 + NG2 + NQ
N1max_sleRara
```

    array([ 273.06  ,  279.4725,  577.5465,  605.6715,  932.0475,  961.0725,
           1315.7125, 1321.9   ])

```python
plt.figure(figsize=(20,10))
plt.plot(N1max_sleRara,s,12.2, color='#0f0f0f80')
plt.fill_between(N1max_sleRara,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1max_sleRara), np.max(N1max_sleRara))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1max_sleRara[0]),float(N1max_sleRara[3]), float(N1max_sleRara[5]), float(N1max_sleRara[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P27\nCombinazione SLE RARA', fontsize='22')
plt.savefig('export/img/P27_maxAxialLoad_sleRara.jpg')
plt.show()
```

```python
N1min_sleRara = (4*21.15+27.45+101.52+90.24+101.536+99.50)+(80.09+130.284+146.59+127.68)+(.5*67.96+0*14.1+56.40+95.17+110.54-3.9)
N1min_sleRara
```

    1281.68

```python
N1min_sleRara = NG1 + NG2 + np.array([.5*67.96+0*14.1-3.9, .5*67.96+0*14.1-3.9, .5*67.96+0*14.1+56.40-3.9, .5*67.96+0*14.1+56.40-3.9, .5*67.96+0*14.1+56.40+95.17-3.9, .5*67.96+0*14.1+56.40+95.17-3.9, .5*67.96+0*14.1+56.40+95.17+110.54-3.9, .5*67.96+0*14.1+56.40+95.17+110.54-3.9])
N1min_sleRara
```

    array([ 232.84  ,  239.2525,  537.3265,  565.4515,  908.7475,  937.7725,
           1275.4925, 1281.68  ])

```python
plt.figure(figsize=(20,10))
plt.plot(N1min_sleRara,s,12.2, color='#0f0f0f80')
plt.fill_between(N1min_sleRara,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1min_sleRara), np.max(N1min_sleRara))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1min_sleRara[0]), float(N1min_sleRara[3]), float(N1min_sleRara[5]), float(N1min_sleRara[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale minima (di compressione) sul pilastro P27\nCombinazione SLE RARA', fontsize='22')
plt.savefig('export/img/P27_minAxialLoad_sleRara.jpg')
plt.show()
```

---
### SLE FREQUENTE

```python
N1max_sleFreq = (4*21.15+27.45+101.52+90.24+101.536+99.50)+(80.09+130.284+146.59+127.68)+(0.2*67.96+0*14.1+.5*56.40+.5*95.17+.7*110.54+0*3.9)
N1max_sleFreq
```

    1156.245

```python
N1max_sleFreq = NG1 + NG2 + np.array([.2*67.96+0*14.1+0*3.9, .2*67.96+0*14.1+0*3.9, .2*67.96+0*14.1+0*3.9 +.5*56.40, .2*67.96+0*14.1+0*3.9 +.5*56.40,  .2*67.96+0*14.1+0*3.9 +.5*56.40+.5*95.17, .2*67.96+0*14.1+0*3.9 +.5*56.40+.5*95.17, .2*67.96+0*14.1+0*3.9 +.5*56.40+.5*95.17+.7*110.54, .2*67.96+0*14.1+0*3.9 +.5*56.40+.5*95.17+.7*110.54])
np.max(N1max_sleFreq)
```

    1156.245

```python
plt.figure(figsize=(20,10))
plt.plot(N1max_sleFreq,s,12.2, color='#0f0f0f80')
plt.fill_between(N1max_sleFreq,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1max_sleFreq), np.max(N1max_sleFreq))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1max_sleFreq[0]),float(N1max_sleFreq[3]), float(N1max_sleFreq[5]), float(N1max_sleFreq[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P27\nCombinazione SLE FREQUENTE', fontsize='22')
plt.savefig('export/img/P27_maxAxialLoad_sleFreq.jpg')
plt.show()
```

```python
N1min_sleFreq = (4*21.15+27.45+101.52+90.24+101.536+99.50)+(80.09+130.284+146.59+127.68)+(0*67.96+0*14.1+.3*56.40+.3*95.17+.6*110.54-0.2*3.9)
N1min_sleFreq
```

    1100.505

```python
N1min_sleFreq = NG1 + NG2 + np.array([0*67.96+0*14.1-0.2*3.9, 0*67.96+0*14.1-0.2*3.9, 0*67.96+0*14.1-0.2*3.9 +.3*56.40, 0*67.96+0*14.1-0.2*3.9 +.3*56.40,  0*67.96+0*14.1-0.2*3.9 +.3*56.40+.5*95.17, 0*67.96+0*14.1-0.2*3.9 +.3*56.40+.3*95.17, 0*67.96+0*14.1-0.2*3.9 +.3*56.40+.3*95.17+.6*110.54, 0*67.96+0*14.1-0.2*3.9 +.3*56.40+.3*95.17+.6*110.54])
N1min_sleFreq
```

    array([ 201.98  ,  208.3925,  466.9865,  495.1115,  790.8225,  800.8135,
           1094.3175, 1100.505 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N1min_sleFreq,s,12.2, color='#0f0f0f80')
plt.fill_between(N1min_sleFreq,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1min_sleFreq), np.max(N1min_sleFreq))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1min_sleFreq[0]),float(N1min_sleFreq[3]), float(N1min_sleFreq[5]), float(N1min_sleFreq[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale minima (di compressione) sul pilastro P27\nCombinazione SLE FREQUENTE', fontsize='22')
plt.savefig('export/img/P27_minAxialLoad_sleFreq.jpg')
plt.show()
```

---
### SLE QUASI PERMANENTE

```python
N1_sleQP = (4*21.15+27.45+101.52+90.24+101.536+99.50)+(80.09+130.284+146.59+127.68)+(0*67.96+0*14.1+.3*56.40+.3*95.17+.6*110.54+0*3.9)
N1_sleQP
```

    1101.285

```python
N1_sleQP = NG1 + NG2 +  np.array([0*67.96+0*14.1+0*3.9, 0*67.96+0*14.1+0*3.9, 0*67.96+0*14.1+.3*56.40+0*3.9, 0*67.96+0*14.1+.3*56.40+0*3.9, 0*67.96+0*14.1+.3*56.40+.3*95.17+0*3.9, 0*67.96+0*14.1+.3*56.40+.3*95.17+0*3.9, 0*67.96+0*14.1+.3*56.40+.3*95.17+.6*110.54+0*3.9, 0*67.96+0*14.1+.3*56.40+.3*95.17+.6*110.54+0*3.9])
np.max(N1_sleQP)
```

    1101.285

```python
plt.figure(figsize=(20,10))
plt.plot(N1_sleQP,s,12.2, color='#0f0f0f80')
plt.fill_between(N1_sleQP,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N1_sleQP), np.max(N1_sleQP))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N1_sleQP[0]), float(N1_sleQP[3]), float(N1_sleQP[5]), float(N1_sleQP[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale (di compressione) sul pilastro P27\nCombinazione SLE QUASI PERMANENTE', fontsize='22')
plt.savefig('export/img/P27_axialLoad_sleQP.jpg')
plt.show()
```

---
## P36
### SLU

```python
N2max_slu = 1.3*(27.45 + 15 +10.8 + 15+9.60 + 33.375+9.60+52.80+72+26.25)+1.5*(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+1.5*(7.23+0*1.5+.6*.414+6+.7*9+.7*66+80.85+.6*2.28+80)
N2max_slu
```

    1067.1171

```python
NG1 = np.array([15 + 10.8, 15 + 10.8+ 25*.3**2 * l1, 15 + 10.8+15+9.6+ 25*.3**2 * l1, 15 + 10.8+15+9.60 +25*.3**2 * (l1+l2), 10.8+15+9.60+33.375+9.6+52.80 +25*.3**2 * (l1+l2), 15 + 10.8+15+9.60+33.375+9.60+52.80+25*.3**2 * (l1+l2+l3), 15 + 10.8+15+9.60+33.375+9.60+ 52.80+72+26.25+25*.3**2 * (l1+l2+l3), 15 + 10.8+15+9.60+33.375+9.60+52.80+72+26.25+25*.3**2 * (l1+l2+l3+l4)])

NG2 = np.array([8.52, 8.52, 8.52+13.86+41.92,8.52+13.86+41.92, 8.52+13.86+41.92+13.86+36.55+40.48, 8.52+13.86+41.92+13.86+36.55+40.48, 8.52+13.86+41.92+13.86+36.55+40.48+92.40, 8.52+13.86+41.92+13.86+36.55+40.48+92.40])

NQ = np.array([7.23+0*1.5+.6*.414, 7.23+0*1.5+.6*.414, 7.23+0*1.5+.6*.414 + 6, 7.23+0*1.5+.6*.414 + 6, 7.23+0*1.5+.6*.414 + 6 + .7*9+.7*66+80.85+.6*2.28, 7.23+0*1.5+.6*.414 + 6 + .7*9+.7*66+80.85+.6*2.28, 7.23+0*1.5+.6*.414 + 6 + .7*9+.7*66+80.85+.6*2.28 + 80, 7.23+0*1.5+.6*.414 + 6 + .7*9+.7*66+80.85+.6*2.28 + 80])
```

```python
N2max_slu = 1.3*NG1 + 1.5*NG2 + 1.5*NQ
N2max_slu
```

    array([  57.5376 ,   65.87385,  190.52385,  199.59135,  643.01085,
            672.74835, 1059.07335, 1067.1171 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N2max_slu,s,12.2, color='#0f0f0f80')
plt.fill_between(N2max_slu,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2max_slu), np.max(N2max_slu))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2max_slu[0]),float(N2max_slu[3]), float(N2max_slu[5]), float(N2max_slu[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P36\nCombinazione SLU', fontsize='22')
plt.savefig('export/img/P36_maxAxialLoad_slu.jpg')
plt.show()
```

```python
N2min_slu = 1*NG1 + 0.8*NG2 + 0*NQ -1.5*np.array([.414, .414, .414, .414, .414+2.28, .414+2.28, .414+2.28, .414+2.28])
N2min_slu
```

    array([ 31.995 ,  38.4075, 107.6315, 114.6065, 264.6735, 287.5485,
           459.7185, 465.906 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N2min_slu,s,12.2, color='#0f0f0f80')
plt.fill_between(N2min_slu,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2min_slu), np.max(N2min_slu))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2min_slu[0]),float(N2min_slu[3]),float(N2min_slu[5]), float(N2min_slu[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale minima (di compressione) sul pilastro P36\nCombinazione SLU', fontsize='22')
plt.savefig('export/img/P36_minAxialLoad_slu.jpg')
plt.show()
```

### SLE RARA

```python
N2max_sleRara = (27.45 + 15+10.8 + 15+9.60 + 33.375+9.60+52.80 +72+26.25)+(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+(7.23+0*1.5+.6*.414+.6+.7*9+.7*66+80.85+.6*2.28+80)
N2max_sleRara
```

    742.2614

```python
N2max_sleRara = NG1 + NG2 + NQ
N2max_sleRara
```

    array([ 41.7984,  48.2109, 134.5909, 141.5659, 447.9489, 470.8239,
           741.4739, 747.6614])

```python
plt.figure(figsize=(20,10))
plt.plot(N2max_sleRara,s,12.2, color='#0f0f0f80')
plt.fill_between(N2max_sleRara,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2max_sleRara), np.max(N2max_sleRara))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2max_sleRara[0]),float(N2max_sleRara[3]), float(N2max_sleRara[5]), float(N2max_sleRara[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P36\nCombinazione SLE RARA', fontsize='22')
plt.savefig('export/img/P36_maxAxialLoad_sleRara.jpg')
plt.show()
```

```python
N2min_sleRara = (27.45 + 15+10.8 + 15+9.60 + 33.375+9.60+52.80+72+26.25)+(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+(.5*7.23+0*1.5+.414+6+.7*9+.7*66+.5*80.85+2.28+80)
N2min_sleRara
```

    704.6990000000001

```python
N2min_sleRara = NG1 + NG2 + np.array([.5*7.23+0*1.5+.414, .5*7.23+0*1.5+.414, .5*7.23+0*1.5+.414 + 6, .5*7.23+0*1.5+.414 + 6, .5*7.23+0*1.5+.414 + 6 + .7*9+.7*66+.5*80.85+2.28, 7.5*.23+0*1.5+.414 + 6 + .7*9+.7*66+.5*80.85+2.28, .5*7.23+0*1.5+.414 +6 + .7*9+.7*66+.5*80.85+2.28 + 80, .5*7.23+0*1.5+.414 + 6 + .7*9+.7*66+.5*80.85+2.28 + 80])
N2min_sleRara
```

    array([ 38.349 ,  44.7615, 131.1415, 138.1165, 404.9865, 425.9715,
           698.5115, 704.699 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N2min_sleRara,s,12.2, color='#0f0f0f80')
plt.fill_between(N2min_sleRara,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2min_sleRara), np.max(N2min_sleRara))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2min_sleRara[0]),float(N2min_sleRara[3]),float(N2min_sleRara[5]),float(N2min_sleRara[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale minima (di compressione) sul pilastro P36\nCombinazione SLE RARA', fontsize='22')
plt.savefig('export/img/P36_minAxialLoad_sleRara.jpg')
plt.show()
```

### SLE FREQUENTE

```python
N2max_sleFreq = (27.45 + 15+10.8 + 15+9.60 + 33.375+9.60+52.80 +72+26.25)+(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+(.2*7.23+0*1.5+0*.414+.3*6+.3*9+.3*66+.2*80.85+0*2.28+.6*80)
N2max_sleFreq
```

    609.3810000000001

```python
N2max_sleFreq = NG1 + NG2 + np.array([.2*7.23+0*1.5+0*.414, .2*7.23+0*1.5+0*.414, .2*7.23+0*1.5+0*.414 + .3*6, .2*7.23+0*1.5+0*.414 + .3*6, .2*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+.2*80.85+0*2.28, .2*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+.2*80.85+0*2.28, .2*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+.2*80.85+0*2.28 + .6*80, .2*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+.2*80.85+0*2.28+.6*80])
N2max_sleFreq
```

    array([ 35.766 ,  42.1785, 124.3585, 131.3335, 341.6685, 364.5435,
           603.1935, 609.381 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N2max_sleFreq,s,12.2, color='#0f0f0f80')
plt.fill_between(N2max_sleFreq,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2max_sleFreq), np.max(N2max_sleFreq))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2max_sleFreq[0]),float(N2max_sleFreq[3]),float(N2max_sleFreq[5]), float(N2max_sleFreq[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale massima (di compressione) sul pilastro P36\nCombinazione SLE FREQUENTE', fontsize='22')
plt.savefig('export/img/P36_maxAxialLoad_sleFreq.jpg')
plt.show()
```

```python
N2min_sleFreq = (27.45 + 15+10.8 + 15+9.60 + 33.375+9.60+52.80 +72+26.25)+(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+(0*7.23+0*1.5+.2*.414+.3*6+.3*9+.3*66+0*80.85+.2*2.28+.6*80)
N2min_sleFreq
```

    592.3038

```python
N2min_sleFreq = NG1 + NG2 + np.array([0*7.23+0*1.5+.2*.414, 0*7.23+0*1.5+.2*.414, 0*7.23+0*1.5+.2*.414 + .3*6, 0*7.23+0*1.5+.2*.414 + .3*6, 0*7.23+0*1.5+.2*.414 + .3*6 + .3*9+.3*66+0*80.85+0.2*2.28, 0*7.23+0*1.5+.2*.414 + .3*6 + .3*9+.3*66+0*80.85+0.2*2.28, 0*7.23+0*1.5+.2*.414 + .3*6 + .3*9+.3*66+0*80.85+.2*2.28+.6*80, 0*7.23+0*1.5+.2*.414 + .3*6 + .3*9+.3*66+0*80.85+.2*2.28 + .6*80])
N2min_sleFreq
```

    array([ 34.4028,  40.8153, 122.9953, 129.9703, 324.5913, 347.4663,
           586.1163, 592.3038])

```python
plt.figure(figsize=(20,10))
plt.plot(N2min_sleFreq,s,12.2, color='#0f0f0f80')
plt.fill_between(N2min_sleFreq,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2min_sleFreq), np.max(N2min_sleFreq))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2min_sleFreq[0]),float(N2min_sleFreq[3]),float(N2min_sleFreq[5]),float(N2min_sleFreq[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale minima (di compressione) sul pilastro P36\nCombinazione SLE FREQUENTE', fontsize='22')
plt.savefig('export/img/P36_minAxialLoad_sleFreq.jpg')
plt.show()
```

### SLE QUASI PERMANENTE

```python
N2min_sleQP = (27.45 + 15+10.8 + 15+9.60 + 33.375+9.60+52.80 +72+26.25)+(8.52+13.86+41.92+13.86+36.55+40.48+92.40)+(0*7.23+0*1.5+0*.414+.3*6+.3*9+.3*66+0*80.85+0*2.28+.6*80)
N2min_sleQP
```

    591.765

```python
N2_sleQP = NG1 + NG2 + np.array([0*7.23+0*1.5+0*.414, 0*7.23+0*1.5+0*.414, 0*7.23+0*1.5+0*.414 + .3*6, 0*7.23+0*1.5+.0*.414 + .3*6, 0*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+0*80.85+0*2.28, 0*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+0*80.85+0.2*28, 0*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+0*80.85+0*2.28+.6*80, 0*7.23+0*1.5+0*.414 + .3*6 + .3*9+.3*66+0*80.85+0*2.28 + .6*80])
N2_sleQP
```

    array([ 34.32  ,  40.7325, 122.9125, 129.8875, 324.0525, 352.5275,
           585.5775, 591.765 ])

```python
plt.figure(figsize=(20,10))
plt.plot(N2_sleQP,s,12.2, color='#0f0f0f80')
plt.fill_between(N2_sleQP,12.2,s, color='#0f0f0f80')
plt.ylim(0, l1+l2+l3+l4)
plt.xlim(np.min(N2_sleQP), np.max(N2min_sleFreq))
plt.gca().invert_yaxis()
plt.grid()
plt.ylabel(r'$s\,[m]$', fontsize='14')
plt.xlabel(r'$N(s)\,[kN]$', fontsize='14')
plt.yticks([0,l1, l1+l2, l1+l2+l3, l1+l2+l3+l4])
plt.xticks([float(N2_sleQP[0]),float(N2_sleQP[3]),float(N2_sleQP[5]),float(N2_sleQP[7])])
plt.xticks(fontsize=16)
plt.yticks(fontsize=16)
plt.axhline(0, color='black')
plt.title('Azione assiale (di compressione) sul pilastro P36\nCombinazione SLE QUASI PERMANENTE', fontsize='22')
plt.savefig('export/img/P36_axialLoad_sleQP.jpg')
plt.show()
```

---


# Two Formulations of Maxwell's equations in Cartesian Coordinates

## Authors: Patrick Nelson & Zach Etienne
### Formatting improvements courtesy Brandon Clark

[comment]: <> (Abstract: TODO)

**Notebook Status:**  Self-Validated

**Validation Notes:** This tutorial notebook has been confirmed to be self-consistent with its corresponding NRPy+ module, as documented below, [here](#code_validation_sys1) and [here](#code_validation). **Additional validation tests may have been performed, but are as yet, undocumented. (TODO)**

### NRPy+ Source Code for this module:
* [Maxwell/MaxwellCartesian_Evol.py](../edit/Maxwell/MaxwellCartesian_Evol.py)
* [Maxwell/MaxwellCartesian_ID.py](../edit/Maxwell/MaxwellCartesian_ID.py)

## Introduction:

This tutorial will draw on previous work done by Ian Ruchlin on [Maxwell's equations in Curvilinear Coordinates](Tutorial-MaxwellCurvilinear.ipynb), which itself drew on the two formulations described in [Illustrating Stability Properties of Numerical Relativity in Electrodynamics](https://arxiv.org/abs/gr-qc/0201051). This will be done to aid construction of an Einstein Toolkit thorn, which will itself be built in the [Tutorial-ETK_thorn-Maxwell](Tutorial-ETK_thorn-Maxwell.ipynb) tutorial notebook.

The construction of our equations here will be nearly identical; however, by assuming Cartesian coordinates, we are able to make several simplifications and eliminate the need for [reference_metric.py](../edit/reference_metric.py) as a dependency.

We will begin with their System I. While the Curvilinear version of this code assumed flat spacetime, we will be constructing the equations in a general spacetime. This allows us to simply replace the reference metric "hatted" quantities with their general counterparts.

# Table of Contents:
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Import core NRPy+ modules and set NRPy+ parameters
1. [Step 2](#laplacian): Constructing the Covariant Laplacian
1. [Step 3](#violation): Measuring the Constraint violation
1. [Step 4](#code_validation_sys1): Code Validation against `Maxwell.MaxwellCartesian_Evol` NRPy+ Module (System I)
1. [Step 5](#code_validation_sys2): Code Validation against `Maxwell.MaxwellCartesian_Evol` NRPy+ Module (System II)
1. [Step 6](#id): Constructing the Initial Data
1. [Step 7](#code_validation): Code Validation against `Maxwell.MaxwellCartesian_ID`  NRPy+ Module
1. [Step 8](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Import core NRPy+ modules and set NRPy+ parameters \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

```python
#Step P1: Import needed Python modules
import NRPy_param_funcs as par
import indexedexp as ixp
import grid as gri
import finite_difference as fin
from outputC import *

#Step P2: Name this Python module
thismodule = "MaxwellCartesian"

#Step 0: Set the spatial dimension parameter to 3.
par.set_parval_from_str("grid::DIM", 3)
DIM = par.parval_from_str("grid::DIM")

# Step 1: Set the finite differencing order to 4.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER", 4)

# Step 2: Register gridfunctions that are needed as input.
psi = gri.register_gridfunctions("EVOL", ["psi"])

# Step 3a: Declare the rank-1 indexed expressions E_{i}, A_{i},
#          and \partial_{i} \psi. Derivative variables like these
#          must have an underscore in them, so the finite
#          difference module can parse the variable name properly.
ED = ixp.register_gridfunctions_for_single_rank1("EVOL", "ED")
AD = ixp.register_gridfunctions_for_single_rank1("EVOL", "AD")
psi_dD = ixp.declarerank1("psi_dD")
x,y,z = gri.register_gridfunctions("AUX",["x","y","z"])

## Step 3b: Declare the conformal metric tensor and its first
#           derivative. These are needed to find the Christoffel
#           symbols, which we need for covariant derivatives.
gammaDD = ixp.register_gridfunctions_for_single_rank2("AUX","gammaDD", "sym01") # The AUX or EVOL designation is *not*
                                                                                # used in diagnostic modules.
gammaDD_dD = ixp.declarerank3("gammaDD_dD","sym01")
gammaDD_dDD = ixp.declarerank4("gammaDD_dDD","sym01_sym23")

gammaUU = ixp.declarerank3("gammaUU","sym01")
detgamma = gri.register_gridfunctions("AUX",["detgamma"])
gammaUU, detgamma = ixp.symm_matrix_inverter3x3(gammaDD)
gammaUU_dD = ixp.declarerank3("gammaDD_dD","sym01")

# Define the Christoffel symbols
GammaUDD = ixp.zerorank3(DIM)
for i in range(DIM):
    for k in range(DIM):
        for l in range(DIM):
            for m in range(DIM):
                GammaUDD[i][k][l] += (sp.Rational(1,2))*gammaUU[i][m]*\
                                     (gammaDD_dD[m][k][l] + gammaDD_dD[m][l][k] - gammaDD_dD[k][l][m])

# Step 3b: Declare the rank-2 indexed expression \partial_{j} A_{i},
#          which is not symmetric in its indices.
#          Derivative variables like these must have an underscore
#          in them, so the finite difference module can parse the
#          variable name properly.
AD_dD = ixp.declarerank2("AD_dD", "nosym")

# Step 3c: Declare the rank-3 indexed expression \partial_{jk} A_{i},
#          which is symmetric in the two {jk} indices.
AD_dDD = ixp.declarerank3("AD_dDD", "sym12")

# Step 4: Calculate first and second covariant derivatives, and the
#         necessary contractions.
# First covariant derivative
# D_{j} A_{i} = A_{i,j} - \Gamma^{k}_{ij} A_{k}
AD_dcovD = ixp.zerorank2()
for i in range(DIM):
    for j in range(DIM):
        AD_dcovD[i][j] = AD_dD[i][j]
        for k in range(DIM):
            AD_dcovD[i][j] -= GammaUDD[k][i][j] * AD[k]
```

# Step 2: Constructing the Covariant Laplacian \[Back to [top](#toc)\]
$$\label{laplacian}$$

One particular difficulty we will encounter here is taking the covariant Laplacian of a vector. This will here take the form of $D_j D^j A_i$. We will start with the outer derivative after lowering the index on the second operator. So, we see that
\begin{align}
D_j D^j A_i &= D_j (\gamma^{jk} D_k A_i) \\
&= (D_k A_i) D_j \gamma^{jk} + \gamma^{jk} D_j D_k A_i \\
&= \gamma^{jk} [\partial_j (D_k A_i) - \Gamma^l_{ij} D_k A_l - \Gamma^l_{jk} D_l A_i],
\end{align}
dropping the first term from the second line because $D_j \gamma^{jk} = 0$ Next, we will again apply the covariant derivative to $A_i$. First, however, we should consider that
\begin{align}
D_k A_i &= \partial_k A_i - \Gamma^l_{ik} A_l \\
& = \partial_k A_i - \gamma^{lm} \Gamma_{mik} A_l \\
& = \partial_k A_i - \Gamma_{mik} A^m, \\
\end{align}
where $\Gamma_{ljk} = \frac{1}{2} (\partial_k \gamma_{lj} + \partial_j \gamma_{kl} - \partial_l \gamma_{jk})$ is the Christoffel symbol of the first kind. Note how we were able to use the raising operator to switch the height of two indices; we will use this in upcoming steps. Thus, our expression becomes
\begin{align}
D_j D^j A_i &= \gamma^{jk} [\partial_j \partial_k A_i - \partial_j (\Gamma_{lik} A^l) - \Gamma^l_{ij} \partial_k A_l + \Gamma^m_{ij} \Gamma_{lmk} A^l - \Gamma^l_{jk} \partial_l A_i + \Gamma^m_{jk} \Gamma_{lim} A^l] \\
&= \gamma^{jk} [\partial_j \partial_k A_i - \underbrace{\partial_j (\Gamma_{lik} A^l)}_{\text{sub-term}} - \Gamma^l_{ij} \partial_k A_l + \Gamma^m_{ij} \Gamma^l_{mk} A_l - \Gamma^l_{jk} \partial_l A_i + \Gamma^m_{jk} \Gamma^l_{im} A_l].
\end{align}
Let's focus on the underbraced sub-term for a moment. Expanding this using the product rule and the definition of the Christoffel symbol,
\begin{align}
\partial_j (\Gamma_{lik} A^l) &= A^l \partial_j \Gamma_{lik} + \Gamma_{lik} \partial_j A^l \\
&= A^l \partial_j (\partial_k \gamma_{li} + \partial_i \gamma_{kl} - \partial_l \gamma_{ik}) + \Gamma_{lik} \partial_j (\gamma^{lm} A_m) \\
&= A^l (\gamma_{li,kj} + \gamma_{kl,ij} - \gamma_{ik,lj}) + \Gamma_{lik} (\gamma^{lm} A_{m,j} + A_m \gamma^{lm}{}_{,j}), \\
\end{align}
where commas in subscripts denote partial derivatives.

So, the Laplacian becomes
\begin{align}
D_j D^j A_i &= \gamma^{jk} [A_{i,jk} -
\underbrace{A^l (\gamma_{li,kj} + \gamma_{kl,ij} - \gamma_{ik,lj})}_{\text{Term 1}} +
\underbrace{\Gamma_{lik} (\gamma^{lm} A_{m,j} + A_m \gamma^{lm}{}_{,j})}_{\text{Term 2}} -
\underbrace{(\Gamma^l_{ij} A_{l,k} + \Gamma^l_{jk} A_{i,l})}_{\text{Term 3}} +
\underbrace{(\Gamma^m_{ij} \Gamma^l_{mk} A_l + \Gamma ^m_{jk} \Gamma^l_{im} A_l)}_{\text{Term 4}}]; \\
\end{align}
we will now begin to contruct these terms individually.

```python
# First, we must construct the lowered Christoffel symbols:
# \Gamma_{ijk} = \gamma_{il} \Gamma^l_{jk}
# And raise the index on A:
# A^j = \gamma^{ij} A_i
GammaDDD = ixp.zerorank3()
AU = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        AU[j] += gammaUU[i][j] * AD[i]
        for k in range(DIM):
            for l in range(DIM):
                GammaDDD[i][j][k] += gammaDD[i][l] * GammaUDD[l][j][k]

# Covariant second derivative (the bracketed terms):
# D_j D^j A_i = \gamma^{jk} [A_{i,jk} - A^l (\gamma_{li,kj} + \gamma_{kl,ij} - \gamma_{ik,lj})
#               + \Gamma_{lik} (\gamma^{lm} A_{m,j} + A_m \gamma^{lm}{}_{,j})
#               - (\Gamma^l_{ij} A_{l,k} + \Gamma^l_{jk} A_{i,l})
#               + (\Gamma^m_{ij} \Gamma^l_{mk} A_l + \Gamma ^m_{jk} \Gamma^l_{im} A_l)
AD_dcovDD = ixp.zerorank3()
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            AD_dcovDD[i][j][k] = AD_dDD[i][j][k]
            for l in range(DIM):
                # Terms 1 and 3
                AD_dcovDD[i][j][k] -= AU[l] * (gammaDD_dDD[l][i][k][j] + gammaDD_dDD[k][l][i][j] - \
                                               gammaDD_dDD[i][k][l][j]) \
                                    + GammaUDD[l][i][j] * AD_dD[l][k] + GammaUDD[l][j][k] * AD_dD[i][l]
                for m in range(DIM):
                    # Terms 2 and 4
                    AD_dcovDD[i][j][k] += GammaDDD[l][i][k] * (gammaUU[l][m] * AD_dD[m][j] + AD[m] * gammaUU_dD[l][m][j]) \
                                        + GammaUDD[m][i][j] * GammaUDD[l][m][k] * AD[l] \
                                        + GammaUDD[m][j][k] * GammaUDD[l][i][m] * AD[l]

# Covariant divergence
# D_{i} A^{i} = \gamma^{ij} D_{j} A_{i}
DivA = 0
# Gradient of covariant divergence
# DivA_dD_{i} = \gamma^{jk} A_{k;\hat{j}\hat{i}}
DivA_dD = ixp.zerorank1()
# Covariant Laplacian
# LapAD_{i} = \gamma^{jk} A_{i;\hat{j}\hat{k}}
LapAD = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        DivA += gammaUU[i][j] * AD_dcovD[i][j]
        for k in range(DIM):
            DivA_dD[i] += gammaUU[j][k] * AD_dcovDD[k][j][i]
            LapAD[i]   += gammaUU[j][k] * AD_dcovDD[i][j][k]

# Step 5: Define right-hand sides for the evolution.
ArhsD = ixp.zerorank1()
ErhsD = ixp.zerorank1()
for i in range(DIM):
    ArhsD[i] = -ED[i] - psi_dD[i]
    ErhsD[i] = -LapAD[i] + DivA_dD[i]
psi_rhs = -DivA

# Step 6: Generate C code for System I Maxwell's evolution equations,
#         print output to the screen (standard out, or stdout).
#lhrh_list = []
#for i in range(DIM):
#    lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "AD" + str(i)), rhs=ArhsD[i]))
#    lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "ED" + str(i)), rhs=ErhsD[i]))
#lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "psi"), rhs=psi_rhs))

#fin.FD_outputC("stdout", lhrh_list)
```

# Step 3: Measuring the Constraint violation \[Back to [top](#toc)\]
$$\label{violation}$$

To evaluate our results, we will need to measure how much our simulation differs from what should be physically possible. We will do this with the constraint equation $D_i E^i = 4 \pi \rho_e$; specifically, we will measure the constraint violation
\begin{align}
\mathcal{C} &= D_i E^i - 4 \pi \rho_e \\
&= D_i (\gamma^{ij} E_j) - 4 \pi \rho_e \\
&= \gamma^{ij} D_i E_j + E_j D_i \gamma^{ij} - 4 \pi \rho_e \\
&= \gamma^{ij} D_i E_j,
\end{align}
since the covariant derivative of the metric tensor is $0$ and $\rho_e=0$ in free space. So, $\mathcal{C} = \gamma^{ij} (E_{j,i} - \Gamma^b_{ij} E_b)$, which will be valid for both systems.

```python
ED_dD = ixp.declarerank2("ED_dD","nosym")
Cviolation = gri.register_gridfunctions("AUX", ["Cviolation"])
Cviolation = sp.sympify(0)
for i in range(DIM):
    for j in range(DIM):
        Cviolation += gammaUU[i][j] * ED_dD[j][i]
        for b in range(DIM):
            Cviolation -= gammaUU[i][j] * GammaUDD[b][i][j] * ED[b]
```

# Step 4: Code Validation against `Maxwell.MaxwellCartesian_Evol` NRPy+ Module (System I) \[Back to [top](#toc)\]
$$\label{code_validation_sys1}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the RHSs of Maxwell's equations (in System I) between

1. this tutorial and
2. the NRPy+ [Maxwell.MaxwellCartesian](../edit/Maxwell/MaxwellCartesian_Evol.py) module.

```python
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []

# Step 18: Call the MaxwellCartesian_Evol() function from within the
#          Maxwell/MaxwellCartesian_Evol.py module,
#          which should do exactly the same as in Steps 1-16 above.
import Maxwell.MaxwellCartesian_Evol as mwevol
par.set_parval_from_str("System_to_use","System_I")
mwevol.MaxwellCartesian_Evol()

print("Consistency check between MaxwellCartesian tutorial and NRPy+ module: ALL SHOULD BE ZERO.")

print("psi_rhs - mwevol.psi_rhs = " + str(psi_rhs - mwevol.psi_rhs))
for i in range(DIM):

    print("ArhsD["+str(i)+"] - mwevol.ArhsD["+str(i)+"] = " + str(ArhsD[i] - mwevol.ArhsD[i]))
    print("ErhsD["+str(i)+"] - mwevol.ErhsD["+str(i)+"] = " + str(ErhsD[i] - mwevol.ErhsD[i]))

print("Cviolation - mwevol.Cviolation = " + str(Cviolation - mwevol.Cviolation))
```

    Warning: System I is less stable!
    Consistency check between MaxwellCartesian tutorial and NRPy+ module: ALL SHOULD BE ZERO.
    psi_rhs - mwevol.psi_rhs = 0
    ArhsD[0] - mwevol.ArhsD[0] = 0
    ErhsD[0] - mwevol.ErhsD[0] = 0
    ArhsD[1] - mwevol.ArhsD[1] = 0
    ErhsD[1] - mwevol.ErhsD[1] = 0
    ArhsD[2] - mwevol.ArhsD[2] = 0
    ErhsD[2] - mwevol.ErhsD[2] = 0
    Cviolation - mwevol.Cviolation = 0

We will now build the equations for System II.

```python
# We inherit here all of the definitions from System I, above

# Step 7a: Register the scalar auxiliary variable \Gamma
Gamma = gri.register_gridfunctions("EVOL", ["Gamma"])

# Step 7b: Declare the ordinary gradient \partial_{i} \Gamma
Gamma_dD = ixp.declarerank1("Gamma_dD")

# Step 8a: Construct the second covariant derivative of the scalar \psi
# \psi_{;\hat{i}\hat{j}} = \psi_{,i;\hat{j}}
#                        = \psi_{,ij} - \Gamma^{k}_{ij} \psi_{,k}
psi_dDD = ixp.declarerank2("psi_dDD", "sym01")
psi_dcovDD = ixp.zerorank2()
for i in range(DIM):
    for j in range(DIM):
        psi_dcovDD[i][j] = psi_dDD[i][j]
        for k in range(DIM):
            psi_dcovDD[i][j] += - GammaUDD[k][i][j] * psi_dD[k]

# Step 8b: Construct the covariant Laplacian of \psi
# Lappsi = ghat^{ij} D_{j} D_{i} \psi
Lappsi = 0
for i in range(DIM):
    for j in range(DIM):
        Lappsi += gammaUU[i][j] * psi_dcovDD[i][j]

# Step 9: Define right-hand sides for the evolution.
ArhsD = ixp.zerorank1()
ErhsD = ixp.zerorank1()
for i in range(DIM):
    ArhsD[i] = -ED[i] - psi_dD[i]
    ErhsD[i] = -LapAD[i] + Gamma_dD[i]
psi_rhs = -Gamma
Gamma_rhs = -Lappsi

# Step 10: Generate C code for System II Maxwell's evolution equations,
#          print output to the screen (standard out, or stdout).
#lhrh_list = []
#for i in range(DIM):
#    lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "AD" + str(i)), rhs=ArhsD[i]))
#    lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "ED" + str(i)), rhs=ErhsD[i]))
#lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "psi"), rhs=psi_rhs))
#lhrh_list.append(lhrh(lhs=gri.gfaccess("rhs_gfs", "Gamma"), rhs=Gamma_rhs))

#fin.FD_outputC("stdout", lhrh_list)
```

# Step 5: Code Validation against `Maxwell.MaxwellCartesian_Evol` NRPy+ Module (System II) \[Back to [top](#toc)\]
$$\label{code_validation_sys2}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the RHSs of Maxwell's equations (in System II) between

1. this tutorial and
2. the NRPy+ [Maxwell.MaxwellCartesian](../edit/Maxwell/MaxwellCartesian_Evol.py) module.

```python
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []

# Step 18: Call the MaxwellCartesian_Evol() function from within the
#          Maxwell/MaxwellCartesian_Evol.py module,
#          which should do exactly the same as in Steps 1-16 above.

par.set_parval_from_str("System_to_use","System_II")
mwevol.MaxwellCartesian_Evol()

print("Consistency check between MaxwellCartesian tutorial and NRPy+ module: ALL SHOULD BE ZERO.")

print("psi_rhs - mwevol.psi_rhs = " + str(psi_rhs - mwevol.psi_rhs))
print("Gamma_rhs - mwevol.Gamma_rhs = " + str(Gamma_rhs - mwevol.Gamma_rhs))
for i in range(DIM):

    print("ArhsD["+str(i)+"] - mwevol.ArhsD["+str(i)+"] = " + str(ArhsD[i] - mwevol.ArhsD[i]))
    print("ErhsD["+str(i)+"] - mwevol.ErhsD["+str(i)+"] = " + str(ErhsD[i] - mwevol.ErhsD[i]))

print("Cviolation - mwevol.Cviolation = " + str(Cviolation - mwevol.Cviolation))
```

    Consistency check between MaxwellCartesian tutorial and NRPy+ module: ALL SHOULD BE ZERO.
    psi_rhs - mwevol.psi_rhs = 0
    Gamma_rhs - mwevol.Gamma_rhs = 0
    ArhsD[0] - mwevol.ArhsD[0] = 0
    ErhsD[0] - mwevol.ErhsD[0] = 0
    ArhsD[1] - mwevol.ArhsD[1] = 0
    ErhsD[1] - mwevol.ErhsD[1] = 0
    ArhsD[2] - mwevol.ArhsD[2] = 0
    ErhsD[2] - mwevol.ErhsD[2] = 0
    Cviolation - mwevol.Cviolation = 0

# Step 6: Constructing the Initial Data \[Back to [top](#toc)\]
$$\label{id}$$

Now that we have evolution equations in place, we must construct the initial data that will be evolved by the solver of our choice. We will start from the analytic solution to this system of equations, given in [Illustrating Stability Properties of Numerical Relativity in Electrodynamics](https://arxiv.org/abs/gr-qc/0201051) as
\begin{align}
A^{\hat{\phi}} &= \mathcal{A} \sin \theta \left( \frac{e^{-\lambda v^2}-e^{-\lambda u^2}}{r^2} - 2 \lambda \frac{ve^{-\lambda v^2}-ue^{-\lambda u^2}}{r} \right), \\
\end{align}
for vanishing scalar potential $\psi$, where $\mathcal{A}$ gives the amplitude, $\lambda$ describes the size of the wavepacket, $u = t+r$, and $v = t-r$. Other components of this field are $0$.To get initial data, then, we simply set $t=0$; since $\psi=0$, $E_i = \partial_t A_i$. Thus, our initial data becomes the equations
\begin{align}
A^{\hat{\phi}} &= 0 \\
E^{\hat{\phi}} &= 8 \mathcal{A} r \sin \theta \lambda^2 e^{-\lambda r^2} \\
\psi &= 0
\end{align}
where the non-$\hat{\phi}$ components are set to 0. We still will need to convert $E^i$ in spherical-like coordinates and then lower its index. Using the standard transformations for coordinates and unit vectors,
\begin{align}
E^{\hat{x}} &= -\frac{y E^{\hat{\phi}}(x,y,z)}{\sqrt{x^2+y^2}} \\
E^{\hat{y}} &= \frac{x E^{\hat{\phi}}(x,y,z)}{\sqrt{x^2+y^2}} \\
E^{\hat{z}} &= 0. \\
\end{align}
We can lower the index in the usual way.

For system II, we will also need to set initial data for $\Gamma$. Since $\Gamma = -\partial_t \psi$ and we have chosen $\psi(t=0) = 0$, $\Gamma(t=0) = 0$.

```python
# Step 1: Declare free parameters intrinsic to these initial data
amp,lam = par.Cparameters("REAL",thismodule,
                          ["amp","lam"],
                          [1.0,1.0]) # __name__ = "MaxwellCartesian_ID", this module's name

# Step 2: Set the initial data
AidD = ixp.zerorank1()

EidD = ixp.zerorank1()
EidU = ixp.zerorank1()
# Set the coordinate transformations:
radial = sp.sqrt(x*x + y*y + z*z)
polar = sp.atan2(sp.sqrt(x*x + y*y),z)
EU_phi = 8*amp*radial*sp.sin(polar)*lam*lam*sp.exp(-lam*radial*radial)
EidU[0] = -(y * EU_phi)/sp.sqrt(x*x + y*y)
EidU[1] =  (x * EU_phi)/sp.sqrt(x*x + y*y)
# The z component (2)is zero.
for i in range(DIM):
    for j in range(DIM):
        EidD[i] += gammaDD[i][j] * EidU[j]

psi_ID = sp.sympify(0)
Gamma_ID = sp.sympify(0)
```

# Step 7: Code Validation against `Maxwell.MaxwellCartesian_ID`  NRPy+ Module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the initial data we intend to use between

1. this tutorial and
2. the NRPy+ [Maxwell.MaxwellCartesian_ID](../edit/Maxwell/MaxwellCartesian_ID.py) module.

Since the initial data is identical between the two systems for $E_i$, $A_i$, and $\psi$, so checking system I should be redundant; we will do it anyways, to be sure.

```python
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []
par.set_parval_from_str("System_to_use","System_I")

import Maxwell.MaxwellCartesian_ID as mwid
mwid.MaxwellCartesian_ID()

print("System I consistency check between MaxwellCartesian tutorial and NRPy+ module;\n ALL SHOULD BE ZERO:")

print("psi_ID - mwid.psi_ID = " + str(psi_ID - mwid.psi_ID))
for i in range(DIM):
    print("AidD["+str(i)+"] - mwid.AidD["+str(i)+"] = " + str(AidD[i] - mwid.AidD[i]))
    print("EidD["+str(i)+"] - mwid.EidD["+str(i)+"] = " + str(EidD[i] - mwid.EidD[i]))
```

    System I consistency check between MaxwellCartesian tutorial and NRPy+ module;
     ALL SHOULD BE ZERO:
    psi_ID - mwid.psi_ID = 0
    AidD[0] - mwid.AidD[0] = 0
    EidD[0] - mwid.EidD[0] = 0
    AidD[1] - mwid.AidD[1] = 0
    EidD[1] - mwid.EidD[1] = 0
    AidD[2] - mwid.AidD[2] = 0
    EidD[2] - mwid.EidD[2] = 0

Finally, we will repeat the check with system II initial data.

```python
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []
par.set_parval_from_str("System_to_use","System_II")

mwid.MaxwellCartesian_ID()

print("System II consistency check between MaxwellCartesian tutorial and NRPy+ module;\n ALL SHOULD BE ZERO:")

print("psi_ID - mwid.psi_ID = " + str(psi_ID - mwid.psi_ID))
print("Gamma_ID - mwid.Gamma_ID = " + str(Gamma_ID - mwid.Gamma_ID))
for i in range(DIM):
    print("AidD["+str(i)+"] - mwid.AidD["+str(i)+"] = " + str(AidD[i] - mwid.AidD[i]))
    print("EidD["+str(i)+"] - mwid.EidD["+str(i)+"] = " + str(EidD[i] - mwid.EidD[i]))
```

    System II consistency check between MaxwellCartesian tutorial and NRPy+ module;
     ALL SHOULD BE ZERO:
    psi_ID - mwid.psi_ID = 0
    Gamma_ID - mwid.Gamma_ID = 0
    AidD[0] - mwid.AidD[0] = 0
    EidD[0] - mwid.EidD[0] = 0
    AidD[1] - mwid.AidD[1] = 0
    EidD[1] - mwid.EidD[1] = 0
    AidD[2] - mwid.AidD[2] = 0
    EidD[2] - mwid.EidD[2] = 0

# Step 8: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-MaxwellCartesian.pdf](Tutorial-MaxwellCartesian.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx --log-level='WARN' Tutorial-MaxwellCartesian.ipynb
!pdflatex -interaction=batchmode Tutorial-MaxwellCartesian.tex
!pdflatex -interaction=batchmode Tutorial-MaxwellCartesian.tex
!pdflatex -interaction=batchmode Tutorial-MaxwellCartesian.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode