```python
import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 3

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Jan. 31 at **12:30pm**.

## Exercise: Income and Growth Across Countries

The data file `cross_country_gdp_pc.csv` contains annual data on GDP per capita for over 100 countries and is available here: https://github.com/letsgoexploring/econ126/raw/master/Data/Csv/cross_country_gdp_pc.csv. The data are was constructed from the Penn World Table (https://www.rug.nl/ggdc/productivity/pwt/).

In this exercise, you will:

1. Analyze some basic facts about GDP per capita across the world.
2. Compute the average annual growth rate of GDP per capita for each country in the data.
3. Compute a linear regression of average GDP per capita growth on GDP per capita in the initial year.

Follow the instructions in the following cells.

### Part (a)

```python
# Create a variable called 'data' that stores the data in the file 'cross_country_gdp_pc.csv' in a DataFrame
# Make sure that 'Year', the left-most column of the data, is set to be the index column

# Print the first 5 rows of data

```

```python
# Each column contains data for a specific country. Print the number of countries (e.g., the length of data.columns)

```

```python
# Each column contains data for a specific year. Print the number of countries (e.g., the length of data.index)

```

```python
# Print the values of the initial year and the final year in the data

```

```python
# Create a variable called 'data_initial' that is equal to the first row of the data DataFrame

# Create a variable called 'data_final' that is equal to the last row of the data DataFrame

```

```python
# Use the sort_values() method of data_initial to sort the Series in ascending order.

# Print the name and GDP per capita of the countries with the 10 lowest GDPs per capita in the initial year of the data

```

```python
# Use the sort_values() method of data_final to sort the Series in ascending order.

# Print the name and GDP per capita of the countries with the 10 lowest GDPs per capita in the final year of the data

```

```python
# Print the names of the countries that are among the 10 with lowest GDPs per capita in BOTH the initial year and
# the final year.

```

```python
# Print the name and GDP per capita of the countries with the 10 highest GDPs per capita in the initial year of the data

```

```python
# Print the name and GDP per capita of the countries with the 10 highest GDPs per capita in the final year of the data

```

```python
# Print the names of the countries that are among the 10 with highest GDPs per capita in BOTH the initial year
# and the final year

```

### Part (b)

Let $y_t$ denotes GDP per capita for some country in some year $t$ and let $g$ denotes the average annual growth in GDP per capita between years 0 and $T$. $g$ is defined by:
\begin{align}
y_T & = (1+g)^T y_0
\end{align}
which implies:
\begin{align}
g & = \left(\frac{y_T}{y_0}\right)^{1/T} - 1
\end{align}
We can use this equation to compute the aveage growth rate of GDP per capita for each country in our data. Note that for our data, $T$ is equal to `len(data.index)-1`.

```python
# Create a variable called 'growth_rates' that contains the average growth rate of each country in the data.
# NOTE: You do not need to re-sort the variables data_initial and data_final. If their indices don't align,
# Pandas will automatically sort their indices alphabetically.

# Use the sort_values() method of growth_rates to sort the Series in ascending order.

```

```python
# Print the average growth rate of GDP per capita for all countries rounded to 4 decimal places

```

```python
# Print the standard deviation of the growth rate of GDP per capita for all countries rounded to 4 decimal places

```

```python
# Print the name and growth rates of the countries with the 10 lowest growth rates of GDP per capita

```

```python
# Print the name and growth rates of the countries with the 10 highest growth rates of GDP per capita

```

```python
# Print the names of the countries that were BOTH among the 10 with lowest GDPs per capita in the initial year and
# among the 10 with the highest growth of GDP per capita.

```

```python
# Use the sort_index() method of data_initial to alphabetize the index [e.g., data_initial = data_initial.sort_index()]

# Use the sort_index() method of growth_rates to alphabetize the index

# Construct a scatter plot with:
#     1. GDP per capita in the initial year on the horizontal axis
#     2. Average growth rate of GDP per capita on the vertical axis
#     3. Size of scatter plot markers at least 50
#     4. Opacity of scatter plot markers no greater than 0.5
#     5. x-axis limit: [0,20000]
#     6. Suitable title and labels for the axes

```

**Question**

1. Describe in words the relationship between GDP per capita in the initial year and average growth of GDP per capita. How is the relationship different for low income countries and high income countries?

**Answer**

1.

### Part (c)

In this part you will use OLS to estimate the relationship between GDP per capita in the initial year and average growth of GDP per capita.

\begin{align}
g_i & = \beta_0 + \beta_{\text{1}}y_i + \epsilon_i,
\end{align}

where $g_i$ is the average annual growth rate of country $i$, $y_i$ is country $i$'s initial GDP per capita, and $\epsilon_i$ is the residual of the regression.

```python

# Create a variable called 'y' that is equal to the average growth rates of the countries

# Create a variable called x' that is equal to the GDP per capita of the countries in the initial year

# Use the Statsmodels function add_constant() to add a constant column to the variable x

# Create a variable called 'model' equal to the output of the Statsmodels OLS function regressing y upon x

# Create a variable called 'results' that stores the fitted model (i.e., the output of the fit() method of model)

# Print the summary of results generated by the summary() method of results

```

**Questions**

1. Interpret the estimate for the coefficient on $y_i$.
2. Interpret the estimate for the constant. Where did we see this number earlier?
3. Interpret the R-squared of the regression.

**Answers**

1.

2.

3.

## Exercise: Income and Growth Across US States

The data file `state_income_data.csv` contains annual data on income per capita for the 48 continental states, Washington D.C., and the United States as a whole and is available here: https://raw.githubusercontent.com/letsgoexploring/economic-data/master/us-convergence/csv/state_income_data.csv.

In this exercise, you will:

1. Analyze some basic facts about income per capita among US states.
2. Compute the average annual growth rate of income per capita for each state in the data.
3. Compute a linear regression of average income per capita growth on income per capita in the initial year.

Follow the instructions in the following cells.

### Part (a)

```python
# Create a variable called 'state_df' that stores the data in the file 'cross_country_gdp_pc.csv' in a DataFrame
# Make sure that 'Year', the left-most column of the data, is set to be the index column

# Print the first 5 rows of data

```

The DataFrame method `dropna()` returns a DataFrame that omits rows that contain missing (NaN means *not a number*) values. The following command will remove the rows with missing values from the DataFrame `df`:

    df =  df.dropna()

The DataFrame method `drop()` returns a DataFrame that omits rows or columns with a given label. The following command will remove the column named `'LABEL'` from the DataFrame `df`:

    df = df.drop('LABEL',axis=1)

The argument `axis=1` means to look among the columns for `'LABEL'`. If we had wanted to remove a row with the index value `'LABEL'`, then the axis argument would have been: `axis=0`.

```python
# Remove the rows with missing data from state_df

# Remove the column named 'United States' from state_df

# Set state_df equal to itself divided by 1000 so that its units are in thousands of dollars

# Print the first 5 rows of data

```

```python
# Create a variable called 'state_growth_rates' that contains the average growth rate of each state in the data.

```

```python
# Construct a scatter plot with:
#     1. GDP per capita in the initial year on the horizontal axis
#     2. Average growth rate of GDP per capita on the vertical axis
#     3. Size of scatter plot markers at least 50
#     4. Opacity of scatter plot markers no greater than 0.5
#     5. y-axis limit: [0.015,0.035]
#     6. Suitable title and labels for the axes

```

### Part (b)

Like you did for the country data above, use OLS to estimate the relationship between income per capita in the initial year and average growth of income per capita.

\begin{align}
g_i & = \beta_0 + \beta_{\text{1}}y_i + \epsilon_i,
\end{align}

where $g_i$ is the average annual growth rate of country $i$, $y_i$ is state $i$'s initial income per capita, and $\epsilon_i$ is the residual of the regression.

```python
# Create a variable called 'y' that is equal to the average growth rates of the states

# Create a variable called x' that is equal to the income per capita of the states in the initial year

# Use the Statsmodels function add_constant() to add a constant column to the variable x

# Create a variable called 'model' equal to the output of the Statsmodels OLS function regressing y upon x

# Create a variable called 'results' that stores the fitted model (i.e., the output of the fit() method of model)

# Print the summary of results generated by the summary() method of results

```

**Questions**

1. Interpret the estimate for the coefficient on $y_i$.
2. Interpret the R-squared of the regression.
3. Why do you think that initial income and average growth are so strongly correlated in the state data but not so in the country data? (If you're not sure, referesh your knowledge about the difference between *conditional* and *unconditional* convergence: https://en.wikipedia.org/wiki/Convergence_(economics)#Types_of_Convergence)

**Answers**

1.

2.

3.


# Python を使ってネットワーク科学の問題を解決しよう
Chapter 2. Python での数式処理

----

明治大学理工学部情報科学科
飯塚 秀明

## SymPy の導入
Python で数式処理(数式の表現やその式変形、微分積分など) を行うには、SymPy を用います。
SymPy を使用するには、まず冒頭で
```python
from sympy import *
init_printing()
```
とすることで、必要な機能の読み込みと初期化を行います。
二行目の`init_printing()` は、以降で行う計算の結果として得られた数式を見やすく表示させるために必要です。
Python 上で処理の対象とする変数は、`symbols()` 関数を用いて生成します。
例えば、変数$x$, $y$, $z$ を用いて数式処理を行う場合、最低限書かなければならないプログラム(スケルトンコード、などとも呼びます) は下記のようになります。
（スケルトンコードなので、実行しても何も出てこないのが正常です。）

**【課題】下の注ぎ口を実行して、スケルトンコードが問題なく動作することを確かめてみましょう。**
**【課題】下の注ぎ口の最後の行に、変数$x$, $y$, $z$ を用いた多項式を記述し、SymPy での数式構築をしてみましょう。**

```python
from sympy import *
init_printing()
x, y, z = symbols("x y z")
```

## SymPy を用いた微分
SymPy を用いて構築した数式を微分するには、
```python
[数式].diff([文字])
```
と記述します。

**【課題】下の注ぎ口を実行して、先のラグランジュ関数$L$ を微分してみましょう。**
**【課題】下の注ぎ口の最後の行を編集して、$x_0$ 以外の変数についても微分できるかを確認してみましょう。**

```python
from sympy import *
init_printing()
x0, x1, x2, l0, l1 = symbols("x_0 x_1 x_2 l_0 l_1")

C0, C1 = 2, 3
L = -(log(x0) + log(x1) + log(x2)) + l0 * (x0 + x2 - C0) + l1 * (x1 + x2 - C1)

L.diff(x0)
```

## 連立方程式を解く
連立方程式
\begin{align}
  \begin{cases}
    \text{[式1]} = 0 \\
    \text{[式2]} = 0 \\
    \quad\vdots \\
    \text{[式$n$]} = 0 \\
  \end{cases}
\end{align}
の解を求めるには、
```python
solve([
    [式1],
    [式2],
    ...,
    [式n]
])
```
と記述します。

**【課題】下の注ぎ口を適切に編集して、前回の連立方程式を解いてみましょう。**
\begin{align}
  \begin{cases}
    3x + 2y = 60 \\
    2x + 3y = 60
  \end{cases}
\end{align}

```python
from sympy import *
init_printing()
x, y = symbols("x y")

solve([
    ???
])
```

## 演習
**【課題】きょう勉強したことを用いて、ラグランジュ関数$L$をSymPy を使って微分し、KKT 条件に基づく連立不等式を解いてみよう。**
\begin{align}
  L(x_0, x_1, x_2, \lambda_0, \lambda_1):=-(\log x_0 + \log x_1 + \log x_2) + \lambda_0(x_0+x_2-c_0) + \lambda_1(x_1+x_2-c_1)
\end{align}

※非負の変数を表現するときには、`symbols` 関数の第２引数に`nonnegative=True` オプションを与えます。すなわち、
```python
x, y, z = symbols("x y z", nonnegative=True)
```
のようにします。

```python

```


# Worksheet 2

```
%matplotlib inline
```

```
import ngcm_utils as ngcm
```

## Question 1

Perform the $LU$ decomposition of

\begin{equation}
  A = \begin{pmatrix} 1 & 3 \\ 4 & 16 \end{pmatrix}.
\end{equation}

Use both standard factorisation methods.

### Answer 1

The general algorithm is laid out in the notes. In particular, when calculating the next entry, we will need to either use the formula

\begin{equation}
  u_{k j} = \left( a_{k j} - \sum_{s = 1}^{k - 1} \ell_{k s} u_{s j} \right) / \ell_{k k}
\end{equation}
or
\begin{equation}
  \ell_{i k} = \left( a_{i k} - \sum_{s = 1}^{k - 1} \ell_{i s} u_{s k} \right) / u_{k k}.
\end{equation}

#### Doolittle factorisation

First consider the Doolittle factorisation, where the diagonal elements of $L$, $\ell_{k k}$ are set to 1.

Build the first entry of $U$ using

\begin{equation}
  u_{1 1} = \left( a_{1 1} - \sum_{s = 1}^{0} \ell_{1 s} u_{s 1} \right) / \ell_{1 1} = a_{1 1} = 1.
\end{equation}

Then work along the first row of $U$, finding

\begin{equation}
  u_{1 2} = \left( a_{1 2} - \sum_{s = 1}^{0} \ell_{1 s} u_{s 2} \right) / \ell_{1 1} = a_{1 2} = 3.
\end{equation}

We can then work down the first column of $L$, finding

\begin{equation}
  \ell_{2 1} = \left( a_{2 1} - \sum_{s = 1}^{0} \ell_{2 s} u_{s 1} \right) / u_{1 1} = a_{2 1} / u_{1 1} = 4.
\end{equation}

Finally we need the bottom right element of $U$, finding

\begin{equation}
  u_{2 2} = \left( a_{2 2} - \sum_{s = 1}^{1} \ell_{2 s} u_{s 2} \right) / \ell_{2 2} = a_{2 2} - \ell_{2 1} u_{1 2} = 16 - 4 \times 3 = 4.
\end{equation}

Hence we have

\begin{equation}
  L = \begin{pmatrix} 1 & 0 \\ 4 & 1 \end{pmatrix} \quad U = \begin{pmatrix} 1 & 3 \\ 0 & 4 \end{pmatrix}.
\end{equation}

#### Crout factorisation

Next consider the Crout factorisation, where the diagonal elements of $U$, $u_{k k}$ are set to 1.

Build the first entry of $L$ using

\begin{equation}
  \ell_{1 1} = \left( a_{1 1} - \sum_{s = 1}^{0} \ell_{1 s} u_{s 1} \right) / u_{1 1} = a_{1 1} = 1.
\end{equation}

We can then work along the first row of $U$, finding

\begin{equation}
  u_{1 2} = \left( a_{1 2} - \sum_{s = 1}^{0} \ell_{1 s} u_{s 2} \right) / \ell_{1 1} = a_{1 2} = 3.
\end{equation}

Then work down the first column of $L$, finding

\begin{equation}
  \ell_{2 1} = \left( a_{2 1} - \sum_{s = 1}^{0} \ell_{2 s} u_{s 1} \right) / u_{1 1} = a_{2 1} / u_{1 1} = 4.
\end{equation}

Finally we need the bottom right element of $L$, finding

\begin{equation}
  \ell_{2 2} = \left( a_{2 2} - \sum_{s = 1}^{1} \ell_{2 s} u_{s 2} \right) / u_{2 2} = a_{2 2} - \ell_{2 1} u_{1 2} = 16 - 4 \times 3 = 4.
\end{equation}

Hence we have

\begin{equation}
  L = \begin{pmatrix} 1 & 0 \\ 4 & 4 \end{pmatrix} \quad U = \begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix}.
\end{equation}

Note that in this case only one entry between the matrices has been switched: in general the results may be very different.

## Question 2

Write out the Thomas algorithm for a tridiagonal system.

### Answer 2

We assume that the tridiagonal matrix $A$ has diagonal entries $b_i , i = 1, . . . , N$, sub-diagonal entries $a_j , j = 1, . . . , N − 1$, and super-diagonal entries $c_j , j = 1, . . . , N − 1$. We assume that we are trying to solve the problem

\begin{equation}
  A {\bf x} = {\bf f},
\end{equation}

which has size $N$.

The Thomas algorithm is essentially Gaussian elimination; forward elimination is used to get an upper triangular problem, and then back substitution used to find the answer. The tridiagonal nature of the problem means that it is sufficiently simple to write it out explicitly.

The first step is the forward elimination process. This yields the system

\begin{equation}
  B {\bf x} = {\bf y},
\end{equation}

where $B$ has entries only on the diagonal, and these entries are all 1, and on the super-diagonal,
which are written as $c_j /d_j , j = 1, . . . , N − 1$. The vectors ${\bf d}, {\bf y}$ are given by the procedure

1. At the first step $d_1 = b_1$ and $y_1 = f_1 /d_1$ ;
2. At the $k$-th step $d_k = b_k − a_{k−1} c_{k−1} / d_{k−1}$ and $y_k = (f_k − y_{k−1} a_{k−1} ) / d_k$.

We can then solve this simplified system using back substitution to find

1. At the first step $x_N = y_N$;
2. In reverse order, $k = N − 1, . . . , 1$ we have $x_k = y_k − x_{k+1} c_k / d_k$.

## Question 3

Write down the general framework for iterative methods for linear systems. Give the convergence matrix. If the linear system uses the matrix $A$ above, will an iterative method converge? [Hint: remember what to do with the diagonal entries]

### Answer 3

The system to be solved is

\begin{equation}
  A {\bf x} = {\bf b},
\end{equation}

with $\det(A) = 0$. We first scale the problem such that all diagonal entries of $A$ are 1. We split the coefficient matrix $A$ into the matrices $N$, $P$, such that

\begin{equation}
  A = N − P
\end{equation}

where $\det(N) = 0$. We therefore have that

\begin{equation}
  N {\bf x} = P {\bf x} + {\bf b}
\end{equation}

and we use this to write an iteration scheme as

\begin{equation}
  N {\bf x}^{(n)} = P {\bf x}^{(n−1)} + {\bf b}, \quad n = 1, 2, 3, \dots
\end{equation}

where we start from some arbitrary initial guess $x^{(0)}$.

The convergence matrix $M$ is given by $M = N^{−1} P$. Convergence is guaranteed if the spectral radius of $M$, which is the largest eigenvalue (in magnitude), is less than one. For the matrix given in the first part we have the rescaled $A$ is

\begin{equation}
  A = \begin{pmatrix} 1 & 3 \\ 4 & 16 \end{pmatrix}.
\end{equation}

Clearly to talk about convergence we need to first decide how to split $A$ into $N$, $P$ matrices. For simplicity we consider Jacobi’s method for which $N = I$ and hence we have

\begin{align}
  && N & = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \\
  && P & = \begin{pmatrix} 0 & -3 \\ -1/4 & 0 \end{pmatrix}, \\
  && M & = N^{-1} P \\
  &&   & = P \\
  &&   & = \begin{pmatrix} 0 & -3 \\ -1/4 & 0 \end{pmatrix} \\
  \implies && \rho(M) & = \max_i | \lambda_i | \\
  &&   & = \max \left\{ \pm \sqrt{3/4} \right\} \\
  &&   & = \sqrt{3/4} \\
  &&   & < 1.
\end{align}

Therefore Jacobi will converge for this matrix.

## Question 4

Check which of the matrices on this sheet are diagonally dominant.

### Answer 4

For (strict) diagonal dominance we need the absolute value of the diagonal coefficient to be (strictly) greater than the sum of the absolute values of all other coefficients in the row.

For $A$ this is not true; for the first row we have $1 \not{>} 3$, although dominance holds for the second row.

For $B$ it is not true; for the second row we have $28 \not{>} 24 + 53$, although dominance holds for the other rows.

For $C$ it is not true; for the first row it fails, although dominance holds for the other rows.

## Question 5

Briefly explain what is meant by quadrature methods based on polynomial interpolation.

### Answer 5

*Standard exam question; see, e.g., 07/08.*

The function $f(x)$ to be integrated is replaced by an interpolating function, in this case a polynomial of degree $n$, that interpolates it at $n + 1$ *nodes* $x_j , j = 0, 1, \dots , n$. In general, when constructing a compound quadrature formula, we may split the interval into subintervals, each subinterval containing $n + 1$ nodes, and use a polynomial interpolating function on each subinterval. The integral of the function is approximated by the integral of the (piecewise polynomial) interpolating function.

## Question 6

Write down the contraction mapping theorem. Check that $g(x) = \cos(x)$ is contracting on the unit interval.

### Answer 6

The contraction mapping theorem is

> If $g(x)$ is a contraction mapping in an interval $I = [a, b]$ then there exists one and only one fixed point of the map in $[a, b]$.

In order to state we need the definition of a contraction mapping, which is

> **Definition** - A continuous map $g(x)$ from an interval $I = [a, b] \subseteq {\mathbb R}$ into ${\mathbb R}$ is contracting (or a contraction mapping) if
>
> 1. the image of $I$ is contained in $I$:
> \begin{align}
    && g(I) &\subseteq I \\
    \Leftrightarrow && g(x) & \in I \quad \forall x \in I.
  \end{align}
> 2. the function $g(x)$ is Lipschitz continuous in $I$ with Lipschitz constant $L < 1$:
> \begin{equation}
    |g(x) − g(y)| \leq L |x − y| \quad \forall x, y \in I.
  \end{equation}

Given this we want to check $g(x) = \cos(x)$ on $I = [0, 1]$. We have that $g$ is continuous and differentiable, so it must be Lipschitz continuous. We know that $L < \max_x \in I | g(x) |$ so we look at $g = − \sin(x)$. We note that the extrema of $g$ occur when $g = − \cos(x) = 0$ which is at $x = (n + 1/2) \pi$, of which none occur in the interval $[0, 1]$. We also note that $g$ is monotonic on the interval. Therefore the extreme values of $g$ are taken at the ends of the interval and are $[0, − \sin(1)]$, which are both less than 1 in absolute value. So $L < 1$.

We also need to check that $g(I) \subseteq I$. Again we note that $g$ is monotonic on the interval. Hence we only need to check that $g(0) = 1 \in I$ and $g(1) = \cos(1) \in I$, both of which hold. Therefore the map is contracting.

## Coding Question 1

Write a code to do $LU$ decomposition. Check the decomposition of

\begin{equation}
  B = \begin{pmatrix} 64 & 8 & 48 \\ 24 & 28 & 53 \\ 32 & 49 & 91 \end{pmatrix}.
\end{equation}

First we should note that python has an $LU$ decomposition algorithm in `scipy.linalg.lu` or `scipy.linalg.lu_factor`. The results produced by that algorithm will not necessarilly match those produced from the algorithm in the notes, as the python implementation includes pivoting (which is the best thing to do in a practical code, but not considered here).

Instead we re-implement the algorithm without pivoting. We will use the Doolittle factorisation.

### Answer Coding Question 1

```
import numpy as np
import scipy.linalg as la

import Worksheet2_Functions as w2
```

```
ngcm.highlight_source("Worksheet2_Functions", "LU_decomposition")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def LU_decomposition(A):
    """Perform LU decomposition using the Doolittle factorisation."""

    L = np.zeros_like(A)
    U = np.zeros_like(A)
    N = np.size(A, 0)

    for k in range(N):
        L[k, k] = 1
        U[k, k] = (A[k, k] - np.dot(L[k, :k], U[:k, k])) / L[k, k]
        for j in range(k+1, N):
            U[k, j] = (A[k, j] - np.dot(L[k, :k], U[:k, j])) / L[k, k]
        for i in range(k+1, N):
            L[i, k] = (A[i, k] - np.dot(L[i, :k], U[:k, k])) / U[k, k]

    return L, U

```
B = np.array([[64.0, 8.0, 48.0], [24.0, 28.0, 53.0], [32.0, 49.0, 91.0]])
[L, U] = w2.LU_decomposition(B)
print("The LU decomposition of\n\n{}\n\nis\n\n{}\n\n{}\n\n".format(B, L, U))
print("Cross check product:\n{}".format(np.dot(L, U)))

# Also check the result from theory question 1.
A = np.array([[1.0, 3.0], [4.0, 16.0]])
[L, U] = w2.LU_decomposition(A)
print("\n\n\nThe LU decomposition of\n\n{}\n\nis\n\n{}\n\n{}\n\n"\
      .format(A, L, U))
print("Cross check product:\n{}".format(np.dot(L, U)))
```

    The LU decomposition of

    [[ 64.   8.  48.]
     [ 24.  28.  53.]
     [ 32.  49.  91.]]

    is

    [[ 1.     0.     0.   ]
     [ 0.375  1.     0.   ]
     [ 0.5    1.8    1.   ]]

    [[ 64.   8.  48.]
     [  0.  25.  35.]
     [  0.   0.   4.]]

    Cross check product:
    [[ 64.   8.  48.]
     [ 24.  28.  53.]
     [ 32.  49.  91.]]

    The LU decomposition of

    [[  1.   3.]
     [  4.  16.]]

    is

    [[ 1.  0.]
     [ 4.  1.]]

    [[ 1.  3.]
     [ 0.  4.]]

    Cross check product:
    [[  1.   3.]
     [  4.  16.]]

## Coding Question 2

Implement the Thomas Algorithm for a tridiagonal system.

Note that we shall test this on the tridiagonal matrix constructed from $B$ above, i.e. on the system

\begin{equation}
  \begin{pmatrix} 62 & 8 & 0 \\ 24 & 28 & 53 \\ 0 & 49 & 91 \end{pmatrix} {\bf x} = \begin{pmatrix} 3 \\ 2 \\ 1 \end{pmatrix}.
\end{equation}

### Answer Coding Question 2

```
ngcm.highlight_source("Worksheet2_Functions", "ThomasAlgorithm")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def ThomasAlgorithm(a, b, c, f):
    """Implement the Thomas algorithm to solve A x = f.
    The vectors a, b, c are the sub-diagonal, diagonal and super-diagonal
    vectors of the original matrix A."""

    # Make copies of the input
    aa = a.copy()
    bb = b.copy()
    cc = c.copy()

    x = np.zeros_like(f)
    d = np.zeros_like(f)
    d[:] = f[:]
    N = len(f)
    for k in range(1, N):
        m = aa[k-1] / bb[k-1]
        bb[k] -= m * cc[k-1]
        d[k] -= m * d[k-1]
    x[-1] = d[-1] / bb[-1]
    for k in range(N-2, -1, -1):
        x[k] = (d[k] - cc[k] * x[k+1]) / bb[k]

    return x

```
D = np.array([[62.0, 8.0, 0.0], [24.0, 28.0, 53.0], [0.0, 49.0, 91.0]])
b = np.array([3.0, 2.0, 1.0])
x = w2.ThomasAlgorithm(np.diag(D, -1), np.diag(D), np.diag(D, +1), b)
print("Solution of A x = f where\n\nA = {}\n\nf = {}\n\n is\n\nx = {}".\
      format(D, b, x))
```

    Solution of A x = f where

    A = [[ 62.   8.   0.]
     [ 24.  28.  53.]
     [  0.  49.  91.]]

    f = [ 3.  2.  1.]

     is

    x = [ 0.05748415 -0.07050219  0.04895173]

## Coding Question 3

Implement the Jacobi method for linear systems. Investigate the behaviour of the method for the problem

\begin{equation}
  B {\bf x} = \begin{pmatrix} 3 \\ 2 \\ 1 \end{pmatrix}
\end{equation}

starting from the trivial initial guess ${\bf x}^{(0)} = {\bf 0}$. Should this be expected? [Hint: Check the convergence theorem]

In addition and for comparison, try the problem using the matrix

\begin{equation}
  C = \begin{pmatrix} \frac{119}{108} & -\frac{14}{27} & -\frac{8}{9} \\ \frac{7}{54} & \frac{46}{27} & \frac{7}{9} \\ \frac{5}{108} & \frac{1}{27} & \frac{23}{17}  \end{pmatrix}.
\end{equation}

### Answer Coding Question 3

```
ngcm.highlight_source("Worksheet2_Functions", "Jacobi")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def Jacobi(A, b, tolerance = 1.e-10, MaxSteps = 100):
    """Solve the linear system A x = b using Jacobi's method,
    starting from the trivial initial guess."""

    x = np.zeros_like(b)

    Anorm = A.copy()
    bnorm = b.copy()
    n = len(b)

    for i in range(n):
        bnorm[i] /= A[i, i]
        Anorm[i, :] /= A[i, i]

    # Compute the split
    N = np.eye(n)
    P = N - Anorm
    AL = la.tril(P)
    AU = la.triu(P)

    # Compute the convergence matrix and check its spectral radius
    M = np.dot(la.inv(N), P)
    eigenvalues, eigenvectors = la.eig(M)
    rho = np.amax(np.absolute(eigenvalues))
    if (rho > 1):
        print("Jacobi will not converge as the"\
            " largest eigenvalue of the convergence matrix is {}".format(rho))

    for j in range(MaxSteps):
        x_old = x.copy()
        x = bnorm + np.dot(AL + AU, x)
        if (la.norm(x - x_old) < tolerance):
            print "Jacobi converged in ", j, " iterations."
            break

    return x

```
B = np.array([[64.0, 8.0, 48.0], [24.0, 28.0, 53.0], [32.0, 49.0, 91.0]])
C = np.array([[119.0/108.0, -14.0/27.0, -8.0/9.0], \
              [7.0/54.0, 46.0/27.0, 7.0/9.0], \
              [5.0/108.0, 1.0/27.0, 23.0/18.0]])
b = np.array([3.0, 2.0, 1.0])

x1 = w2.Jacobi(B, b)
print("Jacobi method solution (fails to converge!) of A x = f where"\
      "\n\nA = {}\n\nf = {}\n\n is\n\nx = {}".format(B, b, x1))
print("True solution is\n\nx = {}\n\n".format(la.solve(B, b)))
x2 = w2.Jacobi(C, b)
print("Jacobi method solution of A x = f where"\
      "\n\nA = {}\n\nf = {}\n\n is\n\nx = {}".format(C, b, x2))
print("True solution is\n\nx = {}".format(la.solve(C, b)))

```

    Jacobi will not converge as the largest eigenvalue of the convergence matrix is 1.31061408063
    Jacobi method solution (fails to converge!) of A x = f where

    A = [[ 64.   8.  48.]
     [ 24.  28.  53.]
     [ 32.  49.  91.]]

    f = [ 3.  2.  1.]

     is

    x = [ -5.57434317e+09  -1.42774902e+10  -7.36150183e+09]
    True solution is

    x = [ 0.34078125  0.76125    -0.51875   ]

    Jacobi converged in  19  iterations.
    Jacobi method solution of A x = f where

    A = [[ 1.10185185 -0.51851852 -0.88888889]
     [ 0.12962963  1.7037037   0.77777778]
     [ 0.0462963   0.03703704  1.27777778]]

    f = [ 3.  2.  1.]

     is

    x = [ 3.52592593  0.61481481  0.63703704]
    True solution is

    x = [ 3.52592593  0.61481481  0.63703704]

We note that the original problem failed to converge, as indicated by the spectral radius of the convergence matrix.

## Coding Question 4

Implement the Gauss-Seidel method, applying it to the convergent problem above. Compare the convergence rate to Jacobi.

### Answer Coding Question 4

```
ngcm.highlight_source("Worksheet2_Functions", "GaussSeidel")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def GaussSeidel(A, b, tolerance = 1.e-10, MaxSteps = 100):
    """Solve the linear system A x = b using the Gauss-Seidel method,
    starting from the trivial initial guess."""

    x = np.zeros_like(b)

    Anorm = A.copy()
    bnorm = b.copy()
    n = len(b)

    for i in range(n):
        bnorm[i] /= A[i, i]
        Anorm[i, :] /= A[i, i]

    # Compute the split
    D = np.eye(n)
    AL = la.tril(D - Anorm)
    AU = la.triu(D - Anorm)
    N = np.eye(n) - AL
    P = AU

    # Compute the convergence matrix and check its spectral radius
    M = np.dot(la.inv(N), P)
    eigenvalues, eigenvectors = la.eig(M)
    rho = np.amax(np.absolute(eigenvalues))
    if (rho > 1):
        print("Gauss-Seidel will not converge as the"\
              " largest eigenvalue of the convergence matrix is {}".format(rho))

    for j in range(MaxSteps):
        x_old = x.copy()
        for i in range(n):
            x[i] = bnorm[i] + np.dot(AL[i, :], x) + np.dot(AU[i, :], x_old)
        if (la.norm(x - x_old) < tolerance):
            print("Gauss-Seidel converged in {} iterations.".format(j))
            break

    return x

```
C = np.array([[119.0/108.0, -14.0/27.0, -8.0/9.0], \
              [7.0/54.0, 46.0/27.0, 7.0/9.0], \
              [5.0/108.0, 1.0/27.0, 23.0/18.0]])
b = np.array([3.0, 2.0, 1.0])

x = w2.GaussSeidel(C, b)
print("Gauss-Seidel method solution of A x = f where"\
      "\n\nA = {}\n\nf = {}\n\n is\n\nx = {}".format(C, b, x))
print("True solution is\n\nx = {}".format(la.solve(C, b)))
```

    Gauss-Seidel converged in 12 iterations.
    Gauss-Seidel method solution of A x = f where

    A = [[ 1.10185185 -0.51851852 -0.88888889]
     [ 0.12962963  1.7037037   0.77777778]
     [ 0.0462963   0.03703704  1.27777778]]

    f = [ 3.  2.  1.]

     is

    x = [ 3.52592593  0.61481481  0.63703704]
    True solution is

    x = [ 3.52592593  0.61481481  0.63703704]

As expected, the number of iterations to convergence is considerably reduced. Note that we could reduce the number of operations taken by the Gauss-Seidel algorithm - the dot products in the update term do not need to range over the entire row, as many of the entries are zero.

## Coding Question 5

Implement the chord method to find the root of

\begin{equation}
  f(x) = \tan (x) - e^{-x}, \quad x \in [0, 1].
\end{equation}

### Answer Coding Question 5

```
ngcm.highlight_source("Worksheet2_Functions", "chord")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def chord(f, m, x0, tolerance = 1e-10, MaxSteps = 100):
    """Implement the chord method to find the root of the equation f(x) = 0,
    starting from the initial guess x^{(0)} = x0."""

    x = np.zeros(MaxSteps)
    x[0] = x0

    # Set up the map g
    g = lambda x: x - m * f(x)

    for i in range(1, MaxSteps):
        x[i] = g(x[i-1])
        if (np.absolute(f(x[i])) < tolerance):
            break
    return x[:i+1]

Note that we have already defined the function whose root we wish to find in worksheet 1:

```
import Worksheet1_Functions as w1
ngcm.highlight_source("Worksheet1_Functions", "fn_worksheet1_q4")
```

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">

  <title></title>
  <meta http-equiv="content-type" content="text/html; charset=None">

def fn_worksheet1_q4(x):
    """Simple function defined in question, f(x) = tan(x) - exp(-x)."""

    return np.tan(x) - np.exp(-x)

```
import matplotlib.pyplot as plt

# Note that over the interval [0, 1] f' is between 2 and 4. We need 0 < m f' < 2, so choose m = 0.5
# We will also start from x = 0.

x = w2.chord(w1.fn_worksheet1_q4, 0.5, 0.0)
print("The root is approximately {} where f is {} after {} steps.\n"\
    .format(x[-1], w1.fn_worksheet1_q4(x[-1]), len(x)))

fig = plt.figure(figsize = (12, 8), dpi = 50)
plt.semilogy(range(len(x)-1), np.absolute(x[:-1] - x[-1]), 'kx')
plt.xlabel('Iteration', size = 16)
plt.ylabel('$|x_i - x_{final}|$', size = 16)
plt.show()
```

```
from IPython.core.display import HTML
def css_styling():
    styles = open("../../IPythonNotebookStyles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

> (The cell above executes the style for this notebook. It closely follows the style used in the [12 Steps to Navier Stokes](http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/) course.)


#  Numerical Implementation

## Preamble
* Define "vec" command for LaTeX $\newcommand{vec}[1]{\boldsymbol{#1}}$
* Define "mat" command for LaTeX $\newcommand{mat}[1]{\mathbb{#1}}$
* Define "set" command for LaTeX $\newcommand{set}[1]{\mathcal{#1}}$
* Define "grad" command for LaTeX $\newcommand{grad}{\nabla}$

## Discretized Grid Dynamics

### Spatial Discretization

The basic field equation to be solved for the grid dynamics is as follows:

\begin{equation}
    \ddot{s}
    + 2\left(\zeta\omega - \zeta^\prime\omega^\prime \nabla^2_\xi \right)\dot{s}
    + \left(\omega^2 - \omega^{\prime 2} \nabla^2_\xi \right)s
    = 2\zeta\omega\dot{s}_s
    + \omega^2 s_s
\end{equation}

Our discretization of this equation will be cell-centered to match the discretization scheme of DPLR, which is where we ultimately expect to implement this algorithm. Furthermore, since we do not require high accuracy numerics or or strong conservation properties, we will use second order finite differences to approximate the spatial derivatives:

\begin{equation}
    \nabla^2_\xi s = s_{i-1,j} + s_{i+1,j} + s_{i,j-1} + s_{i,j+1} - 4s_{i,j} = -\left(4s_i - \sum_{j \in \mathcal{J}_i} s_j  \right)
\end{equation}

Where $\set{J}_i$ is the set cells neighboring the $i$-th cell in the mesh. Applying the discretization, we arrive at the following set of coupled, ordinary differential equations:

\begin{equation}
    \ddot{s}_i
    + 2\left(\zeta\omega + 4\zeta^\prime\omega^\prime\right) \dot{s}_i
    + \left(\omega^2 + 4\omega^{\prime 2} \right) s_i
    - 2\zeta^\prime\omega^\prime \sum \dot{s}_j
    - \omega^{\prime 2}\sum s_j
    = 2\zeta\omega\dot{s}_{s,i}
    + \omega^2 s_{s,i}
\end{equation}

Let $\vec{S}_i = [ s_i, \dot{s}_i ]^T $. With this notation, we can re-write our second order differential equation as system of first order equations.

\begin{equation}
    \dot{\vec{S}}_i
    + \left(\mat{U} + \mat{K} + 4\mat{K}^\prime \right)\vec{S}_i
    - \sum \mat{K}^\prime \vec{S}_j
    = \mat{K}\vec{S}_s
\end{equation}

Where

\begin{align}
    \mat{U} & = \left[\begin{array}{cc}
        0 & 1 \\
        0 & 0 \\
    \end{array}\right] \\
    \mat{K} & = \left[\begin{array}{cc}
        0 & 0 \\
        \omega^2 & 2\zeta\omega \\
    \end{array}\right] \\
    \mat{K}^\prime & = \left[\begin{array}{cc}
        0 & 0 \\
        \omega^{\prime 2} & 2\zeta^\prime\omega^\prime \\
    \end{array}\right] \\
\end{align}

### Temporal Discretization
DLPR uses implicit time integration to circumvent the CFL restrictions associated with grids that have very fine near-wall spacing. The integration engine uses the 1st-order accurate Backward Euler scheme with the option of modifying the residual to achieve 2nd order accuracy. Since our grid dynamics equation is linear with constant coefficients, deriving the implicit time advancement equation is trivial.

\begin{equation}
    \frac{1}{\Delta t} \Delta \vec{S}_i
    + \left( \mat{U} + \mat{K} + 4\mat{K}^\prime \right)
      \left( \vec{S}_i + \Delta\vec{S}_i \right)
    - \sum \mat{K}^\prime \left( \vec{S}_j + \Delta \vec{S}_j \right)
    = \mat{K} \left( \vec{S}_s + \Delta \vec{S}_s \right)
\end{equation}

\begin{equation}
    \left(\frac{\mat{I}}{\Delta t} + \mat{U} + \mat{K} + 4\mat{K}^\prime \right) \Delta \vec{S}_i
    - \sum \mat{K}^\prime \Delta \vec{S}_j
    - \mat{K} \Delta \vec{S}_s
    = \vec{F}_i\left(\vec{S},\vec{S}_s\right)
\end{equation}

Where

\begin{equation}
    \vec{F}_i\left(\vec{S},\vec{S}_s\right)
    = \mat{K}\vec{S}_s
    + \sum \mat{K}^\prime \vec{S}_j
    - \left(\mat{U} + \mat{K} - 4\mat{K}^\prime \right) \vec{S}_i
\end{equation}

In the case of a stationary shock where $\Delta\vec{S}_s = 0$, the equation above degenerates into a block-pentadiagonal linear system. However, while iterating to steady state, the shock will in fact move and change velocity depending on the local flow solution near the shock front. Therefore this is (one of) the terms whereby the linear solve for the grid dynamics couples into the linear solve for the flow dynamics.

## Discretized Fluid Dynamics

### Spatial Discretization
The fluid dynamics of for the problem are governed by the Navier Stokes equations, which can be written in integral form for an arbitrarily deforming control volume as follows:

\begin{equation}
    \frac{d}{dt} \int_{V(t)} \vec{U} dV + \int_{\partial V(t)} \left( \mat{F} - \vec{U}\dot{\vec{x}}^T \right) d\vec{A} = 0
\end{equation}

Where, the flux matrix $\mat{F}$ is

\begin{equation}
    \mat{F} = \left[ \vec{F}_z, \vec{F}_y, \vec{F}_z \right]
\end{equation}

A typical finite volume code will solve this conservation equation for each cell in the computational grid. Consider the $i$-th cell in a regular hexahedral mesh with the volume-average state $\vec{U}_i$. In this situation, the integral of the flux across the boundary of the control volume will be approximated using a midpoint quadrature on each of the six faces as follows:

\begin{equation}
    \frac{d}{dt}\vec{U}_iV_i
    + \sum_{j\in\set{J}_i} \left( \mat{F}_j - \tilde{\vec{U}}_j\dot{\vec{x}}_j^T \right) \vec{A}_j = 0
\end{equation}

### Temporal Discretization
As with the grid dynamics, we will apply Backward Euler time advancement to the discretized fluid equations.

\begin{equation}
    \frac{1}{\Delta t} \left(
        \vec{U}_i^{n+1}V_i^{n+1} - \vec{U}_i^nV_i^n
    \right)
    + \sum_{j \in \set{J}_i} \left(
        \mat{F}^{n+1}_j - \tilde{\vec{U}}_j^{n+1} {\vec{x}}_j^{n+1,T}
    \right) \vec{A}_j^{n+1}
    = 0
\end{equation}

Introducing a linear approximatation for all $n+1$ terms, we arrive at the following coupled time advancement equation for the flow state in each computational cells:

\begin{equation}
    \frac{1}{\Delta t} \left(
        \Delta\vec{U}_i V_i + \vec{U}_i\Delta V_i
    \right)
    + \sum_{j \in \set{J}_i}
    \left(
        \Delta\mat{F}_j
        - \Delta \left(
            \tilde{\vec{U}}_j {\dot{\vec{x}}_j^T}
        \right)
    \right) \vec{A}_j
    +
    \left(
        \mat{F}_j - \tilde{\vec{U}}_j {\dot{\vec{x}}_j^T}
    \right) \Delta\vec{A}_j
    = \vec{R}_i(\vec{U},\vec{x})
\end{equation}

Where

\begin{equation}
    \vec{R}_i(\vec{U},\vec{x}) = -\sum_{j \in \set{J}_i} \left(
        \mat{F}_j - \tilde{\vec{U}}_j \dot{\vec{x}}_j^{T}
    \right) \vec{A}_j
\end{equation}

In the above equation, three terms appear in the summation over the faces. The first term represents the standard linearization of the flux matrix with respect to the state variables. This term is already implemented in DPLR and would be present in most CFD codes that use implicit time advancement. The second term arises due to changes in the grid velocity, while the third arises from changes in the shape of each grid cell.

### Linearization

#### Flux Matrix Linearization
In the general case, the flux matrix is a non-linear function of the reconstructed fluid state on either side of the cell face, $\vec{U}_j^L,\vec{U}_j^R$ and the reconstructed gradients, $\nabla\vec{U}_j^L,\nabla\vec{U}_j^R$. These reconstructions are in turn computed from a non-linear combination of the cell-averaged data in neighboring cells and may take into account the geometry of the grid. To begin, consider the variation in one column of the flux matrix. Via the chain rule, we have:

\begin{align}
    \Delta \vec{F} &= \frac{\partial \vec{F}}{\partial \vec{U}^L} \left[
                          \sum_{s \in \set{S}_L}
                          \frac{\partial\vec{U}^L}{\partial \vec{U}_s} \Delta \vec{U}_s
                          +
                          \sum_{v \in \set{V}_L}
                          \frac{\partial\vec{U}^L}{\partial \vec{x}_v} \Delta \vec{x}_v
                      \right] \\
                   &+ \frac{\partial \vec{F}}{\partial \grad\vec{U}^L} \left[
                          \sum_{t \in \set{T}_L}
                          \frac{\partial\grad\vec{U}^L}{\partial \vec{U}_t} \Delta \vec{U}_t
                          +
                          \sum_{w \in \set{W}_L}
                          \frac{\partial\grad\vec{U}^L}{\partial \vec{x}_w} \Delta \vec{x}_w
                      \right] \\
                   &+ \frac{\partial \vec{F}}{\partial \vec{U}^R} \left[
                          \sum_{s \in \set{S}_R}
                          \frac{\partial\vec{U}^R}{\partial \vec{U}_s} \Delta \vec{U}_s
                          +
                          \sum_{v \in \set{V}_R}
                          \frac{\partial\vec{U}^R}{\partial \vec{x}_v} \Delta \vec{x}_v
                      \right] \\
                   &+ \frac{\partial \vec{F}}{\partial \grad\vec{U}^R} \left[
                          \sum_{t \in \set{T}_R}
                          \frac{\partial\grad\vec{U}^R}{\partial \vec{U}_t} \Delta \vec{U}_t
                          +
                          \sum_{v \in \set{W}_R}
                          \frac{\partial\grad\vec{U}^R}{\partial \vec{x}_w} \Delta \vec{x}_w
                      \right] \\
\end{align}

| $\;$Variable$\;$ | Description |
| :--------------: | ----------- |
| $\set{S}_L, \set{S}_R$ | Set of state vectors used to reconstruct $\vec{U}^L,\vec{U}^R$ |
| $\set{V}_L, \set{V}_R$ | Set of grid vertices used to reconstruct $\vec{U}^L,\vec{U}^R$ |
| $\set{T}_L, \set{T}_R$ | Set of state vectors used to reconstruct $\grad\vec{U}^L,\grad\vec{U}^R$ |
| $\set{W}_L, \set{W}_R$ | Set of grid vertices used to reconstruct $\grad\vec{U}^L,\grad\vec{U}^R$ |

Typically, a single reconstruction scheme will be used for all states in $\vec{U}$; similarly, the same scheme will typically be used for all gradients in $\grad\vec{U}$. In this scenario, the matrices $\partial\vec{U}^L/\partial\vec{U}_s$, $\partial\vec{U}^L/\partial\vec{x}_v$, etc. will collapse to scalar values. However, that's still a lot of terms that must be evaluated to exactly linearize the flux matrix.

Having noted that a precise linearization is possible for an arbirary flux matrix and reconstruction scheme, I'm going to focus on linearizing the Euler (inviscid) flux matrix assuming 1st order accurate state reconstruction. That's all I will need for my initial proof-of-concept implementation. With this approach  $\vec{U}^L=\vec{U}_i$ and $\vec{U}^R=\vec{U}_{j}$ at the $j$-th face; there is no functional dependence on $\grad\vec{U}^L,\grad\vec{U}^R$. Therefore,

\begin{equation}
    \Delta\vec{F}_j = \frac{\partial \vec{F}}{\partial \vec{U}^L} \Delta\vec{U}_i
                    + \frac{\partial \vec{F}}{\partial \vec{U}^R} \Delta\vec{U}_j
\end{equation}

#### Grid Motion Flux
Next let us consider the linearization of the flux due to grid motion, $\vec{U}_j\dot{\vec{x}}_j$. To evalute this term we take the interface state to be an average of the left and right states:

\begin{equation}
    \tilde{\vec{U}}_j = \frac{1}{2}\left(\vec{U}^L + \vec{U}^R\right)
                      = \frac{1}{2}\left(\vec{U}_i + \vec{U}_j\right)
\end{equation}

Next, to evaluate the grid velocity, we will assume that each quadrilateral face is defined using bi-linear interpolation of it's vertices. Assuming $\vec{t} \in [-1,1] \times [-1,1]$:

\begin{align}
    4\vec{x}_j(\vec{t}) &= (1 - t_1)(1 - t_2)\;\vec{x}_{j1} \\
                        &+ (1 + t_1)(1 - t_2)\;\vec{x}_{j2} \\
                        &+ (1 + t_1)(1 + t_2)\;\vec{x}_{j3} \\
                        &+ (1 - t_1)(1 + t_2)\;\vec{x}_{j4}
\end{align}

Taking the derviative of this expression and evaluating at the midpoint of the face yields the following expression for $\dot{\vec{x}}_j$:

\begin{equation}
    \dot{\vec{x}}_j = \frac{1}{4} \sum_{k=1}^4 \dot{\vec{x}}_{jk}
\end{equation}

The linearization of the product $\vec{U}_j\dot{\vec{x}}_j$ may then be written as:

\begin{equation}
    \Delta\left(\tilde{\vec{U}}_j\dot{\vec{x}}_j\right) =
        \frac{1}{2}\left( \Delta\vec{U}_i + \Delta\vec{U}_j \right)\dot{\vec{x}}_j
        + \frac{1}{4}\tilde{\vec{U}_j} \sum_{k=1}^4 \Delta\dot{\vec{x}}_{jk}
\end{equation}

#### Face Area Linearization
Formally, the directed face area, $\vec{A}_j$, is is defined as

\begin{equation}
    \vec{A}_j = \int_{\set{F}_j} \vec{n} dA
    = \int_{-1}^1 \int_{-1}^1 \vec{n} |J| dt_1 dt_2
    = \int_{-1}^1 \int_{-1}^1
        \frac{\partial \vec{x}_j}{\partial t_1}
        \times
        \frac{\partial \vec{x}_j}{\partial t_2}
      dt_1 dt_2
\end{equation}

Differentiating the coordinate transformation in the previous section

\begin{equation}
    4\frac{\partial \vec{x}_j}{\partial t_1}
        = (1-t_2)(\vec{x}_{j2} - \vec{x}_{j1})
        + (1+t_2)(\vec{x}_{j3} - \vec{x}_{j4})
\end{equation}
\begin{equation}
    4\frac{\partial \vec{x}_j}{\partial t_2}
        = (1-t_1)(\vec{x}_{j4} - \vec{x}_{j1})
        + (1+t_1)(\vec{x}_{j3} - \vec{x}_{j2})
\end{equation}

With some algebra, the cross product of these two vectors can be written as:

\begin{align}
    16 \frac{\partial \vec{x}_j}{\partial t_1}
       \times
       \frac{\partial \vec{x}_j}{\partial t_2}
    &= (1 - t_1)(1 - t_2)\;\vec{n}_1
     + (1 + t_1)(1 - t_2)\;\vec{n}_2 \\
    &+ (1 + t_1)(1 + t_2)\;\vec{n}_3
     + (1 - t_1)(1 + t_2)\;\vec{n}_4
\end{align}

Which is to say that the normal at any point on the face is obtained by interpolating the normal vector at each vertex. Integrating this function over the face yields:

\begin{equation}
    4\vec{A}_j = \sum_{k=1}^4 \vec{n}_k
               = \sum_{k=1}^4 (\vec{x}_{k+1} - \vec{x}_k) \times (\vec{x}_{k-1} - \vec{x}_k)
\end{equation}

Consider the variation of the sum with respect to vertex $\vec{x}_v$:

\begin{equation}
    4 \Delta\vec{A}_j
      = \Delta\vec{x}_{ v } \times (\vec{x}_{v-2} - \vec{x}_{v-1})
      - \Delta\vec{x}_{ v } \times (\vec{x}_{v-1} - \vec{x}_{ v })
      - (\vec{x}_{v+1} - \vec{x}_{ v }) \times \Delta\vec{x}_{ v }
      + (\vec{x}_{v+2} - \vec{x}_{v+1}) \times \Delta\vec{x}_{ v }
\end{equation}

This expression can be simplified by exchanging the order of the first two cross products and collecting terms. If one then sums the linear sensitivity for all four vertices, the following linearization is obtained:

\begin{equation}
    2 \Delta\vec{A}_j
    = \sum_{k=1}^4 (\vec{x}_{k-1} - \vec{x}_{k+1}) \times \Delta\vec{x}_k
    = \sum_{k=1}^4 \set{C}(\vec{x}_{k-1} - \vec{x}_{k+1}) \Delta\vec{x}_k
\end{equation}

In the above equation, the $\set{C}(\vec{x})$ operator returns the cross product equivilent matrix of $\vec{x}$, i.e. a matrix with the elements of $\vec{x}$ distributed such that $\set{C}(\vec{x})\,\vec{y} = \vec{x}\times\vec{y}$.

#### Volume Linearization
The final term that we need to linearize is the change in volume. To do this, condsider the application of our discrete flow equation to a static, spatially uniform flow field. In this cases, the flux matrix is identical at all quadrature points and $\vec{U}_i = \tilde{\vec{U}}_j$. Since the integral of a constant flux over a closed surface is zero, we obtain the Geometric Conservation Law, or GCL:

\begin{equation}
    \frac{d}{dt}V_i = \sum_{j\in\set{J}_i} \dot{\vec{x}}_j^T \vec{A}_j
\end{equation}

There is a considerable amount of published literature on the GCL and how best to deal with it. This literature is generally focused on time-accurate calculations with the goal of updating the mesh in such a way as to retain second-order time accuracy. However, second order accuracy is not required in our application, so we will approximate the change in volume using the standard Backward Euler update:

\begin{align}
    \frac{1}{\Delta t}\Delta V_i
        &= \sum_{j\in\set{J}_i}
           \dot{\vec{x}}_j^T \vec{A}_j +
           \Delta\dot{\vec{x}}_j^T \vec{A}_j +
           \dot{\vec{x}}_j^T \Delta\vec{A}_j \\
        &= \sum_{j\in\set{J}_i} \left[
           \dot{\vec{x}}_j^T \vec{A}_j +
           \frac{1}{4} \sum_{k=1}^4 \Delta\dot{\vec{x}}_{jk}^T \vec{A}_j +
           2\dot{\vec{x}}_j^T \set{C}(\vec{x}_{j,k-1} - \vec{x}_{j,k+1})
           \Delta\vec{x}_{jk}
        \right]
\end{align}

#### Grid Node Position
In the preceeding section, all linearizations were written in terms of the change in position and velocity of the vertices defining a computational cell. However, the motion of the cell vertices are derived from the motion of the $\xi_3=N_s$ isosurface and the clustering function used to distribute grid nodes along each $\zeta_1,\zeta_3$-gridline.

Let $s_j,\;j \in 1...4$ be the curvilinear position of the $\xi_3=N_s$ isosurface at the midpoint of the four cell centers surrounding the $i$-th gridline of the mesh (these are the unknowns of the grid motion equation above). Furthermore, define

* $s_{MAX}$: The total arc length of the $i$-th gridline, starting from the outer boundary
* $s_{MIN}$: The curvilinear position along the $i$-th gridline of the $\xi_3$-isosurface
* $s_k$: The curvilinear position of the $k$-th grid point along the $i$-th gridline

Note that even though arc length is measured inward from the outer boundary, grid points are ordered such that grid points radiate outward from the surface of the vehicle.

The distribution function, $\set{D}$, takes as input the grid point location in computational space, $\xi_{3,k}$, and return the position of the point along the line. This position is normalized by the distance between the vehicle surface and the $\xi_3$-isosurface such that $\set{D} = 0$ yields a point on the vehicle surface:

\begin{equation}
    s_k = s_{MAX} - \left(s_{MAX} - s_{MIN} \right) \set{D}(\xi_{3,k})
\end{equation}

We have a few choices for the grid distribution function: uniform, geometric, hyperbolic tangent, etc. Since our proof of concept will assume an inviscid flow field, we can utilize a uniform grid distribution as shown below.

\begin{equation}
    \set{D}(\xi_3) = \frac{\xi_3-1}{N_s}
\end{equation}

The nodal coodinates for each point in the computational grid are then computed via arc-length  interpolation of the background grid:

\begin{equation}
    \vec{x}_{ik}
    = \set{X}_i(s_k)
    = \set{X}_i\left(
        s_{MAX} - \left(s_{MAX} - s_{MIN} \right) \set{D}(\xi_{3,k})
      \right)
\end{equation}

This relationship provides the basis for coupling the linear update equations for the grid and flowfield. Linearizing our relationship, we find that:

\begin{equation}
    \Delta \vec{x}_{ik}
    = \frac{\partial \set{X}_i}{\partial s} \frac{\partial s}{\partial s_{MIN}} \Delta s_{MIN}
    = \vec{t}_k \set{D}_k \Delta s_{MIN}
\end{equation}
Where

| Symbol         | Definition                                             |
|:--------------:|--------------------------------------------------------|
| $\vec{t}_{ik}$ | Tangent vector for gridline $i$ at $\vec{x}_k$         |
| $\set{D}_k$    | Value of the grid distribution function at $\vec{x}_k$ |

Since the linear update equation for the $\xi_3=N_s$ isosurface is written in terms of cell-centered values, we must interpolate these updates from the cell centers to the grid lines to compute $\Delta s_{MIN}$. A simple linear interpolation in the computational domain is sufficient:

\begin{equation}
    \Delta s_{MIN} = \frac{1}{4}\sum_j \Delta s_j
\end{equation}

Combining the above equations:

\begin{equation}
    \Delta \vec{x}_{ik}
    = \vec{t}_k \set{D}_k \sum_j \Delta s_j
\end{equation}

#### Grid Node Velocity
Consider again the mapping from computational space to physical space via interpolation of the background grid.

\begin{equation}
    \vec{x}_{ik}
    = \set{X}_i\left(
        s_{MAX} - \left(s_{MAX} - s_{MIN} \right) \set{D}(\xi_{3,k})
      \right)
\end{equation}

This can be differntiated to find the velocity of each grid point in terms of the rate of change of the $s_{MIN}$:

\begin{align}
    \dot{\vec{x}}_{ik}
    &= \frac{\partial \set{X}_i}{\partial s}
       \frac{\partial s}{\partial s_{MIN}}
       \dot{s}_{MIN} \\
    &= \vec{t}_k \set{D}_k \dot{s}_{MIN} \\
    &= \vec{t}_k \set{D}_k \sum_j \dot{s}_j
\end{align}

When linearizing this relationship, we must take into consideration the fact that our gridlines may be curved, and therefore $\vec{t}_k$ will change as $s_{MIN}$ changes. Therefore, we have:

\begin{equation}
    \Delta \dot{\vec{x}}_{ik}
    = \vec{\kappa}_k\set{D}_k \dot{s}_{MIN} \sum_j \Delta{s}_j
    + \vec{t}_k \set{D}_k \sum_j \Delta \dot{s}_j
\end{equation}

With the paramter $\vec{\kappa}_k$ representing the curvature of the gridlines at the location of the $k$-th gridpoint.

### Background Grid Metrics
The final details required to implement this method relate to the manner in which we interpolate the background grid. We have already established that we will be performing arc-length interpolation and will require tangent and curvature vectors for each gridline.

First, let us establish how to compute arc length along a given gridline. If we assume that the background grid is defined using a mesh of discrete points (as opposed to some analytical mapping), that means that a spline of some kind is required to create a continuous, differentiable mapping from $\zeta_3$ to physical space. There are several strategies that can be employed here:

1. Catmul-Rom Cubic Spline: This $C^1$ continuous spline is easy to compute on the fly using finite differences to estimate the nodal derivatives. However, since we are not $C^2$ continuous, jumps in the curvature parameter will occur (which may or may not be a problem). Furthermore, there if there are any kinks in the background grid, undesireable local extrema may occur in the interpolant.

2. Akima Cubic Spline: A more elaborate $C^1$ spline that is robust to kinks and discontinuities in the grid. Like Catmul-Rom, can still be computed on-the-fly and will exhibit jumps in the curvature.

3. Classic Cubic Spline: $C^2$ continuous, so guarantees smooth curvature metrics. However, computing the derivatives requires a tridiagonal solve so this method is not amenable to on-the-fly computation. Rather, we would precompute the slopes and store them, doubling the required storage. However, the background mesh can be fairly coarse in the wall normal direction, so this may not be a deal breaker.

4. Catmul-Rom Quintic Spline: $C^2$ continuous by construction and easy to compute on the fly. If memory is a concern, this is a nice option, but would be even more prone to local extrema  on non-smooth grids. An interesting option, but would complicate the lookup process.

Generally speaking, I think any of these method will work. The Akima


# Dataset generator for a mixture of pseudo-Voigt shaped peaks + baseline + noise

Depending on the parameter settings, the notebook generates a dataset of a mixture of pseudo-Voigt shaped peaks,
which also can contain a baseline (either y-offset, linear or kwadratic) and normal random noise.

The lines are generated according to the following formula:

\begin{equation}
\large y = f(x) = C_s \cdot \sum\limits_{m=1}^M \big[A_m \cdot f_{pseudo-Voigt}(x)\big] + f_{baseline}(x) + \epsilon
\end{equation}

where:

\begin{equation}
\large f_{baseline}(x) = a_{0} + a_{1}\cdot{x} + a_{2}\cdot{x^2} \\
\large f_{pseudo-Voigt}(x) = \eta \cdot \frac{\sigma_m^2}{(x-\mu_m)^2 + \sigma_m^2} + (1 - \eta) \cdot e^{-\frac{(x-\mu_m)^2}{2\cdot\sigma_m^2}}
\end{equation}

```python
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import pymc3 as pm
import arviz as az
import seaborn as sns

#az.style.use('arviz-darkgrid')

print('Running on PyMC3 v{}'.format(pm.__version__))
```

    WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

    Running on PyMC3 v3.8

## Import local modules

```python
import sys
sys.path.append('../modules')
import utils as utl
import datagen as dg
```

# Set dataset parameters

```python
# list of wavelengths (x-values)
xval = [i for i in range(200, 400, 2)]

# number of samples in dataset (observations)
samples = 15

# number of peaks in the spectrum
npeaks = 4

# peakshape weight factor (0 = Gauss, 1 = Lorentz)
eta = 1

# baseline type (none/offset/linear/quadratic)
btype = "none"

# light scatter effect (yes/no)
scatter = "no"
```

# Generate data and plot

```python
# generate the datasets
ldata  = []
lpeaks = []
nsets  = 10
for i in range(nsets):
    df, peaks, _ = dg.data_generator(xvalues=xval, nsamples=samples, npeaks=npeaks,
                                      peakshape=eta, scatter=scatter, tbaseline=btype)
    ldata.append(df)
    lpeaks.append(peaks)
```

```python
# plot generated datasets
_, ax = plt.subplots(5, 2, figsize=(12, 8), constrained_layout=True)
ax = np.ravel(ax)
for idx, data in enumerate(ldata):
    X = data.columns
    Y = data[X].values
    mu = lpeaks[idx]
    for i in range(len(data)):
        ax[idx].plot(X, Y[i], "-", alpha=.5)
    for j in range(len(mu)):
        ax[idx].axvline(mu[j], linestyle='--', color='gray', alpha=.5)
    ax[idx].set_title("#{}".format(idx+1))
```

```python
# plot and save single dataset
peak = 6
_, ax = plt.subplots(1, 1, figsize=(7, 5), constrained_layout=True)
ax = np.ravel(ax)
for idx, data in enumerate(ldata[peak-1:peak]):
    ax[idx].set(xlabel='Wavelength ($cm^{-1}$)', ylabel='Intensity (a.u.)')
    X = data.columns
    Y = data[X].values
    mu = lpeaks[peak-1]
    for i in range(len(data)):
        ax[idx].plot(X, Y[i], "-", alpha=.5)
    for j in range(len(mu)):
        ax[idx].axvline(mu[j], linestyle='--', color='gray', alpha=.5)
plt.savefig('dataset_plot.png', dpi=150)
```

# Save datasets

```python
# save datasets to .csv
#for i in range(nsets):
#    ldata[i].to_csv('./datagen_%02d.csv' % (i+1), index=False)
```


```python
# ===== Programming exercise ======
# =================================
```

# The lorentz model is given as follow:
\begin{equation}
\left\{
\begin{split}
x'(t) & = \sigma(y-x)\\
y'(t) & = rx - y - xz \\
z'(t) & = xy - bz
\end{split}
\right.
\end{equation}
Let us implement the model using coding

```python
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
from mpl_toolkits import mplot3d
import seaborn as sns
import matplotlib as cm
```

```python
def F(m,m1,t):
    [x,y,z] = m1
    dxdt = m[0]*(y-x)
    dydt = m[1]*x - y - x*z
    dzdt = x*y - m[2]*z
    return np.array([dxdt,dydt,dzdt])
```

```python
def RK4(F,x0,t):
    h=t[1] - t[0]
    m1=np.zeros([len(t),len(x0)])
    m1[0] = x0
    for i in range(len(t)-1):
        k1 = F(m,m1[i],t[i])
        k2 = F(m, m1[i]+h*(k1/2),t[i]+(h/2))
        k3 = F(m, m1[i]+h*(k2/2),t[i]+(h/2))
        k4 = F(m, m1[i]+h*k3,t[i]+h)
        m1[i+1] = m1[i] + (h/6)*(k1+2*k2+2*k3+k4)
    return m1
```

```python
sigma =10
r=28
b=8/3
x0 = [8,0,30]
t = np.linspace(-20,20,1000)
m = [sigma,r,b]
Result = RK4(F,x0,t)
Result
```

    array([[ 8.        ,  0.        , 30.        ],
           [ 5.31263079, -0.13726712, 26.92803343],
           [ 3.57961011,  0.2760398 , 24.20684875],
           ...,
           [-3.1771216 ,  0.91678253, 27.35220855],
           [-1.86222268,  0.68924193, 24.50532831],
           [-1.07210399,  0.40095795, 21.99366711]])

```python
z = Result[:,2]
x = Result[:,0]
y = Result[:,1]
```

```python
fig = plt.figure(figsize = (16,8))
ax = plt.axes(projection='3d')
ax.plot3D(x, y, z, 'y', linewidth = 3)
ax.set_title('Lorentz dynamic')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
```

#  Plotting in two dimension

```python
plt.plot(x, y)
```

```python
plt.plot(x, z)
```

```python
plt.plot(z, y)
```

```python

```


## Classification using kNN

**Import the required libraries**

```python
import re
import math
import numpy as np
import pandas as pd
import string
import scipy as sp
import nltk
import time
import operator
from random import randrange
from sklearn import preprocessing

import warnings
warnings.filterwarnings('ignore')
```

**The following cell contains a class of methods to calculate distance between two points using various techniques**

**Formula to calculate Eucledian distance:**

<math>\begin{align}D(x, y) = \sqrt{ \sum_i (x_i - y_i) ^ 2 }\end{align}</math>

**Formula to calculate Manhattan Distance:**

<math>\begin{align}D(x, y) = \sum_i |x_i - y_i|\end{align}</math>

**Formula to calculate Hamming Distance:**

<math>\begin{align}D(x, y) = \frac{1}{N} \sum_i \delta_{x_i, y_i}\end{align}</math>

```python
class distanceMetrics:
    '''
    Description:
        This class contains methods to calculate various distance metrics
    '''
    def __init__(self):
        '''
        Description:
            Initialization/Constructor function
        '''
        pass

    def euclideanDistance(self, vector1, vector2):
        '''
        Description:
            Function to calculate Euclidean Distance

        Inputs:
            vector1, vector2: input vectors for which the distance is to be calculated
        Output:
            Calculated euclidean distance of two vectors
        '''
        self.vectorA, self.vectorB = vector1, vector2
        if len(self.vectorA) != len(self.vectorB):
            raise ValueError("Undefined for sequences of unequal length.")
        distance = 0.0
        for i in range(len(self.vectorA)-1):
            distance += (self.vectorA[i] - self.vectorB[i])**2
        return (distance)**0.5

    def manhattanDistance(self, vector1, vector2):
        """
        Desription:
            Takes 2 vectors a, b and returns the manhattan distance
        Inputs:
            vector1, vector2: two vectors for which the distance is to be calculated
        Output:
            Manhattan Distance of two input vectors
        """
        self.vectorA, self.vectorB = vector1, vector2
        if len(self.vectorA) != len(self.vectorB):
            raise ValueError("Undefined for sequences of unequal length.")
        return np.abs(np.array(self.vectorA) - np.array(self.vectorB)).sum()

    def hammingDistance(self, vector1, vector2):
        """
        Desription:
            Takes 2 vectors a, b and returns the hamming distance
            Hamming distance is meant for discrete-valued vectors, though it is a
            valid metric for real-valued vectors.
        Inputs:
            vector1, vector2: two vectors for which the distance is to be calculated
        Output:
           Hamming Distance of two input vectors
        """
        self.vectorA, self.vectorB = vector1, vector2
        if len(self.vectorA) != len(self.vectorB):
            raise ValueError("Undefined for sequences of unequal length.")
        return sum(el1 != el2 for el1, el2 in zip(self.vectorA, self.vectorB))

```

```python
class kNNClassifier:
    '''
    Description:
        This class contains the functions to calculate distances
    '''
    def __init__(self,k = 3, distanceMetric = 'euclidean'):
        '''
        Description:
            KNearestNeighbors constructor
        Input
            k: total of neighbors. Defaulted to 3
            distanceMetric: type of distance metric to be used. Defaulted to euclidean distance.
        '''
        pass

    def fit(self, xTrain, yTrain):
        '''
        Description:
            Train kNN model with x data
        Input:
            xTrain: training data with coordinates
            yTrain: labels of training data set
        Output:
            None
        '''
        assert len(xTrain) == len(yTrain)
        self.trainData = xTrain
        self.trainLabels = yTrain

    def getNeighbors(self, testRow):
        '''
        Description:
            Train kNN model with x data
        Input:
            testRow: testing data with coordinates
        Output:
            k-nearest neighbors to the test data
        '''

        calcDM = distanceMetrics()
        distances = []
        for i, trainRow in enumerate(self.trainData):
            if self.distanceMetric == 'euclidean':
                distances.append([trainRow, calcDM.euclideanDistance(testRow, trainRow), self.trainLabels[i]])
            elif self.distanceMetric == 'manhattan':
                distances.append([trainRow, calcDM.manhattanDistance(testRow, trainRow), self.trainLabels[i]])
            elif self.distanceMetric == 'hamming':
                distances.append([trainRow, calcDM.hammingDistance(testRow, trainRow), self.trainLabels[i]])
            distances.sort(key=operator.itemgetter(1))

        neighbors = []
        for index in range(self.k):
            neighbors.append(distances[index])
        return neighbors

    def predict(self, xTest, k, distanceMetric):
        '''
        Description:
            Apply kNN model on test data
        Input:
            xTest: testing data with coordinates
            k: number of neighbors
            distanceMetric: technique to calculate distance metric
        Output:
            predicted label
        '''
        self.testData = xTest
        self.k = k
        self.distanceMetric = distanceMetric
        predictions = []

        for i, testCase in enumerate(self.testData):
            neighbors = self.getNeighbors(testCase)
            output= [row[-1] for row in neighbors]
            prediction = max(set(output), key=output.count)
            predictions.append(prediction)

        return predictions
```

```python
def printMetrics(actual, predictions):
    '''
    Description:
        This method calculates the accuracy of predictions
    '''
    assert len(actual) == len(predictions)
    correct = 0
    for i in range(len(actual)):
        if actual[i] == predictions[i]:
            correct += 1
    return (correct / float(len(actual)) * 100.0)
```

```python
class kFoldCV:
    '''
    This class is to perform k-Fold Cross validation on a given dataset
    '''
    def __init__(self):
        pass

    def crossValSplit(self, dataset, numFolds):
        '''
        Description:
            Function to split the data into number of folds specified
        Input:
            dataset: data that is to be split
            numFolds: integer - number of folds into which the data is to be split
        Output:
            split data
        '''
        dataSplit = list()
        dataCopy = list(dataset)
        foldSize = int(len(dataset) / numFolds)
        for _ in range(numFolds):
            fold = list()
            while len(fold) < foldSize:
                index = randrange(len(dataCopy))
                fold.append(dataCopy.pop(index))
            dataSplit.append(fold)
        return dataSplit

    def kFCVEvaluate(self, dataset, numFolds, *args):
        '''
        Description:
            Driver function for k-Fold cross validation
        '''
        knn = kNNClassifier()
        folds = self.crossValSplit(dataset, numFolds)
        scores = list()
        for fold in folds:
            trainSet = list(folds)
            trainSet.remove(fold)
            trainSet = sum(trainSet, [])
            testSet = list()
            for row in fold:
                rowCopy = list(row)
                testSet.append(rowCopy)

            trainLabels = [row[-1] for row in trainSet]
            trainSet = [train[:-1] for train in trainSet]
            knn.fit(trainSet,trainLabels)

            actual = [row[-1] for row in testSet]
            testSet = [test[:-1] for test in testSet]

            predicted = knn.predict(testSet, *args)

            accuracy = printMetrics(actual, predicted)
            scores.append(accuracy)

        print('*'*20)
        print('Scores: %s' % scores)
        print('*'*20)
        print('\nMaximum Accuracy: %3f%%' % max(scores))
        print('\nMean Accuracy: %.3f%%' % (sum(scores)/float(len(scores))))
```

```python
def readData(fileName):
    '''
    Description:
        This method is to read the data from a given file
    '''
    data = []
    labels = []

    with open(fileName, "r") as file:
        lines = file.readlines()
    for line in lines:
        splitline = line.strip().split(',')
        data.append(splitline)
        labels.append(splitline[-1])
    return data, labels
```

### Hayes-Roth Data

```python
trainFile = 'Datasets/HayesRoth/hayes-roth.data'

trainData, trainLabel = readData(trainFile)

trainFeatures = []
for row in trainData:
    index = row[1:]
    temp = [int(item) for item in index]
    trainFeatures.append(temp)

trainLabels = [int(label) for label in trainLabel]
```

**Create an object for k-Fold cross validation class**

```python
kfcv = kFoldCV()
```

**Call the Evaluation function of kFCV class**

*kfcv.kFCVEvaluate(data, foldCount, neighborCount, distanceMetric)*

```python
kfcv.kFCVEvaluate(trainFeatures, 10, 3, 'euclidean')
```

    ********************
    Scores: [76.92307692307693, 46.15384615384615, 61.53846153846154, 53.84615384615385, 69.23076923076923, 53.84615384615385, 53.84615384615385, 46.15384615384615, 84.61538461538461, 53.84615384615385]
    ********************

    Maximum Accuracy: 84.615385%

    Mean Accuracy: 60.000%

```python
kfcv.kFCVEvaluate(trainFeatures, 10, 3, 'manhattan')
```

    ********************
    Scores: [69.23076923076923, 84.61538461538461, 76.92307692307693, 69.23076923076923, 76.92307692307693, 46.15384615384615, 46.15384615384615, 69.23076923076923, 76.92307692307693, 53.84615384615385]
    ********************

    Maximum Accuracy: 84.615385%

    Mean Accuracy: 66.923%

```python
kfcv.kFCVEvaluate(trainFeatures, 10, 3, 'hamming')
```

    ********************
    Scores: [30.76923076923077, 84.61538461538461, 61.53846153846154, 76.92307692307693, 61.53846153846154, 46.15384615384615, 69.23076923076923, 46.15384615384615, 61.53846153846154, 61.53846153846154]
    ********************

    Maximum Accuracy: 84.615385%

    Mean Accuracy: 60.000%

### Car Evaluation Data

```python
carFile = 'Datasets/CarEvaluation/car.data'

carData, carLabel = readData(carFile)
df = pd.DataFrame(carData)
df = df.apply(preprocessing.LabelEncoder().fit_transform)
carFeatures = df.values.tolist()
carLabels = [car[-1] for car in carFeatures]
```

```python
kfcv.kFCVEvaluate(carFeatures, 10, 3, 'euclidean')
```

    ********************
    Scores: [62.2093023255814, 62.7906976744186, 61.627906976744185, 63.372093023255815, 63.372093023255815, 70.34883720930233, 57.55813953488372, 64.53488372093024, 69.18604651162791, 55.23255813953488]
    ********************

    Maximum Accuracy: 70.348837%

    Mean Accuracy: 63.023%

```python
kfcv.kFCVEvaluate(carFeatures, 10, 3, 'manhattan')
```

    ********************
    Scores: [87.20930232558139, 90.11627906976744, 90.11627906976744, 86.04651162790698, 87.79069767441861, 90.11627906976744, 89.53488372093024, 87.79069767441861, 87.79069767441861, 89.53488372093024]
    ********************

    Maximum Accuracy: 90.116279%

    Mean Accuracy: 88.605%

```python
kfcv.kFCVEvaluate(carFeatures, 10, 3, 'hamming')
```

    ********************
    Scores: [83.72093023255815, 80.23255813953489, 84.30232558139535, 90.11627906976744, 84.88372093023256, 86.04651162790698, 81.97674418604652, 87.20930232558139, 87.20930232558139, 83.72093023255815]
    ********************

    Maximum Accuracy: 90.116279%

    Mean Accuracy: 84.942%

### Breast Cancer Data

```python
cancerFile = 'Datasets/BreastCancer/breast-cancer.data'

cancerData, cancerLabel = readData(cancerFile)
cdf = pd.DataFrame(cancerData)
cdf = cdf.apply(preprocessing.LabelEncoder().fit_transform)
cancerFeatures = cdf.values.tolist()
cancerLabels = [cancer[-1] for cancer in cancerFeatures]
```

```python
kfcv.kFCVEvaluate(cancerFeatures, 10, 3, 'euclidean')
```

    ********************
    Scores: [67.85714285714286, 89.28571428571429, 78.57142857142857, 60.71428571428571, 53.57142857142857, 78.57142857142857, 78.57142857142857, 60.71428571428571, 75.0, 85.71428571428571]
    ********************

    Maximum Accuracy: 89.285714%

    Mean Accuracy: 72.857%

```python
kfcv.kFCVEvaluate(cancerFeatures, 10, 3, 'manhattan')
```

    ********************
    Scores: [78.57142857142857, 60.71428571428571, 78.57142857142857, 78.57142857142857, 53.57142857142857, 85.71428571428571, 75.0, 64.28571428571429, 75.0, 64.28571428571429]
    ********************

    Maximum Accuracy: 85.714286%

    Mean Accuracy: 71.429%

```python
kfcv.kFCVEvaluate(cancerFeatures, 10, 3, 'hamming')
```

    ********************
    Scores: [71.42857142857143, 71.42857142857143, 85.71428571428571, 67.85714285714286, 64.28571428571429, 78.57142857142857, 57.14285714285714, 57.14285714285714, 89.28571428571429, 78.57142857142857]
    ********************

    Maximum Accuracy: 89.285714%

    Mean Accuracy: 72.143%

```python

```