# Time Series - II (AR Models)

An Auto Regressive (AR) model is a regression which depends linearly on the previous terms. An auto regressive model of order p is a regression model that linearly depends on p previous terms.

\begin{align}
x_{t} &= \alpha_{1}.x_{t-1} + ... + \alpha_{p}.x_{t-p} + w_{t}
\end{align}

In the above equation, p represents the number of previous terms (lags) used in the model, a is the regressive coefficient and w is the white noise in the term which cannot be explained with the help of previous terms.

```python
#Importing the required packages

import pandas as pd
import numpy as np

from matplotlib import pyplot as plt
%matplotlib inline

import statsmodels.tsa.api as smt
import statsmodels.api as sm
import scipy.stats as scs
import statsmodels.stats as sms

import warnings
warnings.filterwarnings('ignore')
```

```python
#Defining a function to visualize and analyze the time series

def tsplot(y, lags=None, figsize=(15, 10), style='bmh'):
    '''
    Prepares a (3,2) dimensional plot for the visualization of time series values, autocorrelation and partial
    autocorrelation plots and QQ and probability plots for comparision with normal distribution.

    Args:
    y: time series values
    lags: How many lagging values are to be considered.
    '''
    if not isinstance(y, pd.Series):
        y = pd.Series(y)
    with plt.style.context(style):
        fig = plt.figure(figsize=figsize)

        layout = (3, 2)
        ts_ax = plt.subplot2grid(layout, (0, 0), colspan=2)
        acf_ax = plt.subplot2grid(layout, (1, 0))
        pacf_ax = plt.subplot2grid(layout, (1, 1))
        qq_ax = plt.subplot2grid(layout, (2, 0))
        pp_ax = plt.subplot2grid(layout, (2, 1))

        y.plot(ax=ts_ax)
        ts_ax.set_title('Time Series Analysis Plots')
        smt.graphics.plot_acf(y, lags=lags, ax=acf_ax, alpha=0.05)
        smt.graphics.plot_pacf(y, lags=lags, ax=pacf_ax, alpha=0.05)
        sm.qqplot(y, line='s', ax=qq_ax)
        qq_ax.set_title('QQ Plot')
        scs.probplot(y, sparams=(y.mean(), y.std()), plot=pp_ax)

        plt.tight_layout()
    return
```

**Simulating AR(1) process with a = 0.6 **

```python
np.random.seed(1)
n_samples = int(1000)
a = 0.6
x = w = np.random.normal(size=n_samples)

for t in range(n_samples):
    x[t] = a*x[t-1] + w[t]

_ = tsplot(x, lags=30)
```

The distribution of our sampled AR1 model data is normal, as it should be as we sampled the data from normal distribution. There is significant correlation between lagged values visible in the ACF plot.

PACF plots are used to identify the extent of the lag in an autoregressive mode. If we find no significant correlation in a PACF plot after lag $k$, an AR(k) model is usually a good fit. Looking at this chart, we can hypothesize that a AR(1) model should fit.

**Fitting data with Python's statsmodels package **

Now we will use python's statsmodels packages and try to fit an AR model on the data we obtained in the last step to estimate the alpha coefficient and order.

```python
model = sm.tsa.AR(x).fit(maxlag=30, ic='aic', trend='nc')
est_order = sm.tsa.AR(x).select_order( maxlag=30, ic='aic', trend='nc')

true_order = 1
print('\nalpha estimate: %3.5f | order_estimate %s'%(model.params, est_order))
print('\ntrue alpha = %s | true order = %s'%(a, true_order))
```

    alpha estimate: 0.58227 | order_estimate 1

    true alpha = 0.6 | true order = 1

Using statsmodels packages, we have found the value of alpha very near to that of its real value used to generate the data. Also, we were able to identify the order of the AR model accurately.

**Simulating AR(2) process with $a_{1}$ = 0.666 and $a_{2}$ = -0.333**

```python
n = int(1000)
alphas = np.array([.666, -.333])
betas = np.array([0.])

# Python requires us to specify the zero-lag value which is 1
# Also note that the alphas for the AR model must be negated
# We also set the betas for the MA equal to 0 for an AR(p) model
# For more information see the examples at statsmodels.org
ar = np.r_[1, -alphas]
ma = np.r_[1, betas]

ar2 = smt.arma_generate_sample(ar=ar, ma=ma, nsample=n)
_ = tsplot(ar2, lags=30)
```

```python
#Fitting the model
max_lag = 10
mdl = smt.AR(ar2).fit(maxlag=max_lag, ic='aic', trend='nc')
est_order = smt.AR(ar2).select_order( maxlag=max_lag, ic='aic', trend='nc')

true_order = 2
print('\ncoef estimate: %3.4f %3.4f | order estimate %s'%(mdl.params[0],mdl.params[1],est_order))
```

    coef estimate: 0.6760 -0.3393 | order estimate 2

In the above models, we are explicitly choosing a lag of 10. If we don't specify the maximum lag that we want to consider, then the model will itself try to estimate the lags. In this process, we will end with estimating many more lags than the actual case because of indirect dependencies (see below). For Example, our AR(1) model just depends on the previous value $x_{t}$ (lag = 1), but model will include many more lags as $x_{t}$ depends on $x_{t-1}$ and in turn $x_{t-2}$ and so on indirectly.

This inclusion of indirect dependencies is not a good thing, as we want to have as few parameters in the model as possible. Including all the parameters overfit the data.

### Naive fit of the model

Fitting a model naively will estimate a lot of parameters. In the below model on AR(2) data, the model estimates a lot of parameters/lags due to indirect dependencies as discussed above.

```python
model2 = smt.AR(ar2).fit()
print('Parameters')
print(model2.params)
print('Standard Error')
print(model2.bse)
```

    Parameters
    [ 3.57345632e-02  6.67118523e-01 -3.40041205e-01  9.01909128e-03
     -1.11520048e-02  2.14435279e-02 -4.83381895e-02  3.28287551e-02
     -2.38531001e-02 -4.90031065e-04 -3.12735473e-02  5.21112625e-02
     -5.85303137e-02  2.17108859e-02 -5.58739239e-02  2.79663351e-02
     -6.45697812e-03 -2.90269682e-02  3.79624698e-02  1.12370300e-02
      2.54545125e-03  5.79634021e-02]
    Standard Error
    [0.03330767 0.03229192 0.03882052 0.04032568 0.04032737 0.04031148
     0.04030305 0.04029053 0.04027444 0.04025271 0.04022384 0.04018031
     0.0402171  0.04026555 0.04027908 0.04028844 0.04027529 0.04028007
     0.04028697 0.04034915 0.03870137 0.03212717]

By observing the ACF and PACF plots, we can infer that first two lags are useful. We can also AIC (Akaike Information Criterion) to test the useful lags. Python does that for usu when we use 'maxlags' and 'ic' arguments in the fit() function.  We want a model with smallest AIC, so we compute the AIC for all the models that we want to consider and we select the model with the smallest AIC. This model minimizes the information loss and select the best number of parameters.

```python
N = 10
AIC = np.zeros((N,1))

for i in range(N):
    model = smt.AR(ar2)
    model = model.fit(maxlag = (i+1))
    AIC[i] = model.aic

AIC_min = np.min(AIC)
model_min = np.argmin(AIC)

print("Minimum number of parameters in minimum AIC model %s" %(model_min + 1))
```

    Minimum number of parameters in minimum AIC model 2

**Evaluating Residuals **

The final step after fitting the model is to evaluate its residual behaviour. We want out residuals to be white noise because if it is not then that means there is still some pattern in it which could be extracted and there is a scope of making our model better.

```python
# Checking the normality of the residuals

from statsmodels.stats.stattools import jarque_bera

score, pvalue, _, _ = jarque_bera(mdl.resid)

if pvalue < 0.10:
    print('We have reason to suspect the residuals are not normally distributed.')
else:
    print('The residuals seem normally distributed.')

tsplot(mdl.resid, lags=30)
```

The residuals looks like randomly distributed in the above plots, indicating that the model explains the data well.

## Application of the above principles

In this section, we apply the above technique on the log returns of TSLA (Tesla Inc.) stock data and see how good can we explain the changes. We use last 5 years EOD stock price data of the TSLA stock which could be downloaded from Yahoo Finance.

**What are Log returns? **

Rate of Return is profit on an investment or a particular stock over a period of time, expressed as a proportion of the original investment. We use rate of return instead of absolute difference of the stock price in order to normalize the return with respect to the stock price. Log returns or Logarithmic rate of return is the log of the return for two dates.

For Example, if a stock is priced at 3.570 USD per share at the close on one day, and at 3.575 USD per share at the close the next day, then the logarithmic return is: ln(3.575/3.570) = 0.0014, or 0.14%

There are multiple reasons to work with log returns and this blog post summarizes everything beautifully: https://quantivity.wordpress.com/2011/02/21/why-log-returns/

```python
#Downloading the data from the directory

tsla_market_data = pd.read_csv("C:\\Users\ku.kulshrestha\Downloads\TSLA.csv")
print(tsla_market_data.head(5))
```

             Date        Open        High         Low       Close   Adj Close  \
    0  2013-06-20  104.650002  107.129997   99.449997  100.650002  100.650002
    1  2013-06-21  103.699997  103.699997   97.500000   99.550003   99.550003
    2  2013-06-24   96.500000  102.870003   95.300003  101.489998  101.489998
    3  2013-06-25  103.099998  104.199997  100.550003  102.400002  102.400002
    4  2013-06-26  103.800003  105.870003  102.660004  105.720001  105.720001

         Volume
    0  10106500
    1  11718600
    2   7119800
    3   5848700
    4   6602600

Since we would only be dealing with the price at the end of the day, hence we only need to take care about Close column of the above dataset and then calculate the log returns of the closing price.

```python
tsla_ts_data = tsla_market_data[['Date', 'Close']]

#Adding a new col in the dataframe
tsla_ts_data['Log_return'] = np.nan
tsla_ts_data.head()
for i in range(1, len(tsla_ts_data)):
    tsla_ts_data.loc[i, 'Log_return'] = np.log(np.divide(tsla_ts_data.loc[i,'Close'], tsla_ts_data.loc[i-1, 'Close']))

#removing the first row as Log_return cannot be defined for it
tsla_ts_data = tsla_ts_data.dropna()
tsla_ts_data.head()
```

      Date
      Close
      Log_return

      1
      2013-06-21
      99.550003
      -0.010989

      2
      2013-06-24
      101.489998
      0.019300

      3
      2013-06-25
      102.400002
      0.008926

      4
      2013-06-26
      105.720001
      0.031907

      5
      2013-06-27
      109.250000
      0.032845

```python
#Plotting the distribution of the log returns
tsla_ts_data.Log_return.plot(figsize = (10,8))
```

Now we have our data ready, we will try to fit a simple AR(p) model on the Log_return data, find out the best order and plot the residuals of the model.

```python
max_lag = 30
mdl = smt.AR(tsla_ts_data.Log_return.iloc[1:]).fit(maxlag=max_lag, ic='aic', trend='nc')
est_order = smt.AR(tsla_ts_data.Log_return.iloc[1:]).select_order(maxlag=max_lag, ic='aic', trend='nc')

print('best estimated lag order = %s'%(est_order))

_ = tsplot(mdl.resid, lags=max_lag)
```

This study suggests us that AR(1) model best explains the data. There are also some non significant peaks on 16th and 19th lags on the PACF plot of the residuals. This indicates that there is probability more complexity in the data hidden which our simple linear model is not able to explain.

Below test of normality of the residuals also confirms the same.

```python
from statsmodels.stats.stattools import jarque_bera

score, pvalue, _, _ = jarque_bera(mdl.resid)

print('Score is: ', score)

if pvalue < 0.10:
    print('We have reason to suspect the residuals are not normally distributed.')
else:
    print('The residuals seem normally distributed.')
```

    Score is:  1006.8068128169832
    We have reason to suspect the residuals are not normally distributed.

Jarque Bera test is generally used to check for the normality of the data. A gentle introduction for the same can be found here: http://www.statisticshowto.com/jarque-bera-test/


# Linear Regression from scratch
> Linear regression from scratch using Pytorch on Swedish Auto Insurance dataset.
- author: "Axel Mendoza"
- categories: [linear-regression, insurance-data, pytorch, from-scratch]
- toc: false
- comments: true
- badges: true
- image: images/lines.jpg

This blog post is about implementing linear regression from scratch using Pytorch.
We will train the model on insurance data to predict the total payment received by a customer in regards to his number of claims.

For more details, the swedish auto insurance dataset contains the following attributes:
- X = number of claims
- Y = total payment for all the claims in thousands of Swedish Kronor for geographical zones in Sweden

```python
import os
import sys
import torch
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
```

### Reading Excel file with Pandas

```python
data_path = os.path.join('data', 'slr06.xls')
df = pd.read_excel(data_path, encoding_override="cp1252")
df.head()
```

    *** No CODEPAGE record, no encoding_override: will use 'ascii'

      X
      Y

      0
      108
      392.5

      1
      19
      46.2

      2
      13
      15.7

      3
      124
      422.2

      4
      40
      119.4

### Data Visualization Using Seaborn

```python
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5))
ax1.set_title('Distribution of number of claims')
ax2.set_title('Distribution of total payment')

sns.distplot(df.X, bins=50, hist=True, ax=ax1)
sns.distplot(df.Y, ax=ax2)
```

Two outliers are present between 100 and 125 number of claims.
We will keep them in the training set.

```python
sns.heatmap(df.corr(), annot=True)
```

The correlation heatmap and the plot below highlight that the number of claims and total payment are very highly correlated.

```python
plt.xlabel('Number of claims')
plt.ylabel('Total payment')
plt.scatter(df['X'], df['Y'])
```

### Training a Least Square Regressor

Convert the data to pytorch and separate the data into train & test.

```python
data = df.to_numpy()
X = torch.from_numpy(data[:, 0]).float().unsqueeze(1)
y = torch.from_numpy(data[:, 1]).float().unsqueeze(1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=50)
```

We will train a least square regressor on one dimension using the above formula:
$$
    \begin{align}
    y &= b_0 + b_1x \\
    b_1 &= \frac{\text{Cov}(x, y)}{Var(x)} = \frac{\sum_i{(x_i - \mathbb{E}[x]) * (y_i - \mathbb{E}[y])}}{\sum_i{(x_i - \mathbb{E}[x])^2}}
    \end{align}
$$
where the expectation $\mathbb{E}[x]$ of a vector of random variables is its mean.

```python
def linear_regression_1d(X, y):
    """Trains a linear regression on 1D data

    Args:
        X: A numpy array for the training samples
        y: A numpy array for the labels of each sample
    """
    X_m = X.mean(dim=0)
    y_m = y.mean(dim=0)

    X_c = (X - X_m)

    # Compute covariance and variance
    covar = (X_c * (y - y_m)).sum(dim=0)
    var = X_c.pow(2).sum(dim=0)

    # Divide covariance by variance
    b_1 = covar / var

    # Get bias
    b_0 = y_m - b_1 * X_m.sum(dim=0)
    return b_0, b_1
```

We are using Pytorch for the matrix calculus.

### Plot Regression Line

```python
b_0, b_1 = linear_regression_1d(X_train, Y_train)
plt.scatter(X_train, Y_train, marker='*')
plt.scatter(X_test, Y_test, marker='.', color='red')

x = [int(elt) for elt in range(0, int(X.cpu().numpy().max()))]
y = b_0.numpy() + b_1.numpy() * x
plt.plot(x, y, color='red')

ax = plt.gca()
ax.set_xlabel('Number of claims')
ax.set_ylabel('Total payment')
plt.show()
```

Because both the data and target are very highly correlated, the regression line fits pretty well the data distribution.

### Compute Mean Square Error

```python
pred_train = b_0 + b_1 * X_train
err_train = Y_train - pred_train
mse_train = err_train.T.mm(err_train) / Y_train.shape[0]

pred_test = b_0 + b_1 * X_test
err_test = Y_test - pred_test
mse_test = err_test.T.mm(err_test) / Y_test.shape[0]

print('Train MSE:\t', mse_train.item())
print('Test MSE:\t', mse_test.item())
```

    Train MSE:	 1241.5714111328125
    Test MSE:	 1279.6766357421875

The train and test MSE are close to each other.
The model neither underfit nor overfit.

### Conclusion
Linear regression is a simple and interpretable model that fits very well the purpose of a baseline.
As we have seen during this experiment, if the features are highly correlated to the target, linear regression performs pretty well.


# How

## Create a matrix

We create a matrix using the `sympy.Matrix` tool. We combine this with
square brackets `[]` which we nest so that every row is also inside square
brackets.

````{tip}
```
sympy.Matrix([values])
```
````

For example, the following creates the matrix:

$$
    B = \begin{pmatrix}
            1 & 2 & 3 & 4\\
            5 & 6 & 7 & 8\\
            9 & 10 & 11 & 12
        \end{pmatrix}
$$

```python
import sympy as sym

B = sym.Matrix([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
B
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3 & 4\\5 & 6 & 7 & 8\\9 & 10 & 11 & 12\end{matrix}\right]$

```{attention}
It is possible to write the code in a more readable way as long as an incomplete
line ends with an open bracket:
```

```
B = sym.Matrix(
    [
        [1, 2, 3, 4],
        [5, 6, 7, 8],
        [9, 10, 11, 12]
    ]
)
```

## Calculate the determinant of a matrix

To calculate the determinant of a matrix, we use the `.det` tool. For example to
calculate the determinant of:

````{tip}
```
matrix = sympy.Matrix([values])
matrix.det()
```
````

For example, the determinant of the following matrix:

$$
    \begin{pmatrix}
    1 & 5\\
    5 & 1
    \end{pmatrix}
$$

```python
matrix = sym.Matrix([[1, 5], [5, 1]])
matrix.det()
```

$\displaystyle -24$

## Calculate the inverse of a matrix

To calculate the inverse of a matrix, we use the `.inv` tool.

````{tip}
```
matrix = sympy.Matrix([values])
matrix.inv()
```
````

For example to
calculate the inverse of:

$$
    \begin{pmatrix}
        1 / 2 & 1\\
        5     & 0
    \end{pmatrix}
$$

```python
matrix = sym.Matrix([[sym.S(1) / 2, 1], [5, 0]])
matrix.inv()
```

$\displaystyle \left[\begin{matrix}0 & \frac{1}{5}\\1 & - \frac{1}{10}\end{matrix}\right]$

## Multiply matrices by a scalar

To multiple a matrix by a scalar we use the `*` operator. For example to
multiply the following matrix by $6$:

$$
    \begin{pmatrix}
        1 / 5 & 1\\
        1 & 1
    \end{pmatrix}
$$

```python
matrix = sym.Matrix([[sym.S(1) / 5, 1], [1, 1]])
6 * matrix
```

$\displaystyle \left[\begin{matrix}\frac{6}{5} & 6\\6 & 6\end{matrix}\right]$

## Add matrices together

To add matrices we use the `+` operator. For example to compute:

$$
    \begin{pmatrix}
        1 / 5 & 1\\
        1 & 1
    \end{pmatrix} +
    \begin{pmatrix}
        4 / 5 & 0\\
        0 & 0
    \end{pmatrix}
$$

```python
matrix = sym.Matrix([[sym.S(1) / 5, 1], [1, 1]])
other_matrix = sym.Matrix([[sym.S(4) / 5, 0], [0, 0]])
matrix + other_matrix
```

$\displaystyle \left[\begin{matrix}1 & 1\\1 & 1\end{matrix}\right]$

## Multiply matrices together

To multiply matrices together we use the `@` operator. For example to compute:

$$
    \begin{pmatrix}
        1 / 5 & 1\\
        1 & 1
    \end{pmatrix}
    \begin{pmatrix}
        4 / 5 & 0\\
        0 & 0
    \end{pmatrix}
$$

```python
matrix @ other_matrix
```

$\displaystyle \left[\begin{matrix}\frac{4}{25} & 0\\\frac{4}{5} & 0\end{matrix}\right]$

## How to create a vector

A vector is essentially a matrix with a single row or column. For example to
create the vector:

$$
    \begin{pmatrix}
    3 \\
    2 \\
    1
    \end{pmatrix}
$$

```python
vector = sym.Matrix([[3], [2], [1]])
vector
```

$\displaystyle \left[\begin{matrix}3\\2\\1\end{matrix}\right]$

## How to solve a linear system

To solve a given linear system that can be represented in matrix form, we create
the corresponding matrix and vector and use the matrix inverse. For example to
solve the following equations:

$$
    \begin{array}
        a x + 2y = 3\\
        3x + y + 2z = 4\\
        - y + z = 1\\
    \end{array}
$$

```python
A = sym.Matrix([[1, 2, 0], [3, 1, 2], [0, -1, 1]])
b = sym.Matrix([[3], [4], [1]])
A.inv() @ b
```

$\displaystyle \left[\begin{matrix}- \frac{5}{3}\\\frac{7}{3}\\\frac{10}{3}\end{matrix}\right]$


# Vibration modes of membranes with convex polygonal shape

Nicolás Guarín Zapata

## Description

The idea is to find the modes of vibration for membranes with (convex) polygonal shape. These are found as eigenvalues for the [Helmholtz equation](http://en.wikipedia.org/wiki/Helmholtz_equation)
$$\left(\nabla^2 + \frac{\omega^2}{c^2}\right) u(x,y) \equiv \left(\nabla^2 + k^2\right) u(x,y) = 0\ \forall\ (x,y)\in\Omega\enspace ,$$
with boundaries fixed, i.e.
$$u(x,y)=0\ \forall\ (x,y) \in \partial\Omega \enspace .$$

This quation is common in Mathematical Physics, e.g., in the solution of Acoustics, Quantum Mechanics and Electromagnetism problems [[1]](#References).

We have an equivalent formulation for this problem, i.e., a [variational](http://en.wikipedia.org/wiki/Calculus_of_variations) (energy) formulation. Starting from
the energy (functional) $E$,

$$ E = U + k^2T \equiv \underbrace{\int\limits_\Omega \nabla u \cdot \nabla u\ dx\ dy}_{U} +  k^2\underbrace{\int\limits_\Omega u^2 dx\ dy}_{T} \enspace ,$$

that is equivalent to the original differential equation -under some assumptions [[2]](#References).

This problem has analytical solution for some shapes like rectangles, circles and ellipses. So, here we use an approximate method to find the solutions: the [Ritz method](http://en.wikipedia.org/wiki/Ritz_method). The Ritz method is the very same method used in most of Finite Element Formulations. I will describe the method in here, but it should not be read as a formal one, it is more like a cartoon-ish formulation.

Let's propose a solution of the form
$$\hat{u}(x,y) = \sum_{n=1}^{N} c_{n} f_n(x,y) \enspace ,$$

where $f_n(x,y)$ are known functions, they should be a subset of a complete basis for the space of solutions... but, for this notebook they are just polynomials (due to ease in integrate them). So, what we don't know in this equations are the coefficients. Good thing there is a theorem that says that the solution of the problem is an extremum (minimum in this case) of the functional. In our case, that means solving the system of equations
$$\frac{\partial E}{\partial c_{n}} = 0 \enspace .$$
What lead us to

$$[K]\lbrace\mathbf{c}\rbrace = k^2[M]\lbrace\mathbf{c}\rbrace \enspace ,$$

and whe know that

$$\left[\frac{\partial^2 U}{\partial c_i \partial c_j }\right]\lbrace \mathbf{c}\rbrace = k^2 \left[\frac{\partial^2 T}{\partial c_i \partial c_j}\right]\lbrace \mathbf{c}\rbrace \enspace .$$

Than can be explicitly, written as

$$K_{ij} = \int\limits_\Omega \nabla f_i \cdot \nabla f_j \ dx\ dy$$

and

$$M_{ij} = \int\limits_\Omega f_i f_j dx\ dy\, .$$

Hence, we have the formula for computing the components of the (stiffness and mass) matrices and then solve the resulting eigenvalue problem.

## Computing the integrals

The last ingredient missing is the calculation of the integral $U$ and $T$. We want to compute the integral of known functions $f_n(x,y)$; the main problem is the domain (in other words, the limits of the integrals). The following image shows a (convex) heptagon and a subdivision in triangles that use the centroid as one of the vertices. The idea is to subdivide the integral into integrals over the non-overlapping triangles.

To achieve that (easily) we can transform the domain of integration into a _canonical_ domain, as depicted in the next image.

For this simple case the transformation is given by
$$\begin{pmatrix}x\\ y \end{pmatrix} = \mathbf{T}\begin{pmatrix}r\\ s \end{pmatrix} \equiv [J]\begin{pmatrix}r\\ s \end{pmatrix}  + \begin{pmatrix}x_A\\ y_A \end{pmatrix} \enspace ,$$
with
$$[J] = \begin{bmatrix} x_B - x_A &x_C - x_A\\ y_B - y_A &y_C - y_A \end{bmatrix} \enspace .$$
And the inverse transformation reads
$$\begin{pmatrix}r\\ s \end{pmatrix} = \mathbf{T}^{-1}\begin{pmatrix}x\\ y \end{pmatrix} \equiv [J^{-1}]\begin{pmatrix}x\\ y \end{pmatrix}  + \frac{1}{\det J}\begin{pmatrix}x_A y_C - x_C y_A\\ x_By_A - x_A y_B \end{pmatrix} \enspace ,$$
where
$$[J^{-1}] = \frac{1}{\det J}\begin{bmatrix}  y_C - y_A &x_A - x_C\\ y_A - y_B &x_B - x_A \end{bmatrix} \enspace .$$

Then, to compute the integrals we [tranform the domain and the functions of interest](http://en.wikipedia.org/wiki/Integration_by_substitution), i.e., $u^2(x,y)$ and $\nabla u(x,y) \cdot \nabla u(x,y)$. And the integrals over one of the triangles are expressed as
$$\int\limits_{\Omega_k} u^2(x,y)\ dx\ dy = \int\limits_{0}^{1}\int\limits_{0}^{1-s} u^2(\phi(r,s))\ \det J\ dr\ ds \enspace ,$$
and
$$\int\limits_{\Omega_k} [\nabla u(x,y))]^2\ dx\ dy = \int\limits_{0}^{1}\int\limits_{0}^{1-s} [\nabla u(\phi(r,s))]^2\ \det J\ dr\ ds \enspace ,$$
in this case we are first computing $\nabla u(x,y)$ and later making the change of variable $(x,y) = \phi(r,s)$, it can be done the other way but that implies the use of the [chain rule](http://en.wikipedia.org/wiki/Chain_rule). Thus, we loop over the triangles, compute the Jacobian transform the functions, solve each integral and add it up.

## Algorithm

```python
%matplotlib notebook
```

```python
import numpy as np
from scipy.linalg import eigh
from sympy import (symbols, lambdify, init_printing, cos, sin,
                   pi, expand, Matrix, diff, integrate, Poly)
from sympy.plotting import plot3d
from sympy.utilities.lambdify import lambdify
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.path import Path
```

```python
x, y, r, s= symbols('x y r s')
init_printing()
```

### Compute the polygon

The polygon is defined as a set of nodes and its conectivity

```python
nsides = 6
poly = [[cos(2*k*pi/nsides), sin(2*k*pi/nsides)] for k in range(0, nsides)]
#poly = [[-1,-1],[1,-1],[1,1],[-1,1]]  # Simplest square
npts = len(poly)
lines = [[k,0] if k==npts-1 else [k,k+1] for k in range(npts)]
centroid = [sum([poly[k][0] for k in range(npts)]), sum([poly[k][1] for k in range(npts)])]
```

### Polygon plot

```python
plt.fill(np.array(poly)[:,0], np.array(poly)[:,1], fill=False, ec='k', lw=2, hatch='/')
plt.plot(np.array(poly)[:,0], np.array(poly)[:,1], lw=0, marker='o', ms=8,
         mfc="white", mec="black")
plt.axis('image');
plt.xlim(-1.2, 1.2), plt.ylim(-1.2, 1.2);
```

### Boundary conditions

We need our function to satisfy the boundary conditions, i.e,
$$u(\text{boundary}) = 0 \enspace ,$$
this can be convoluted for a general polygon. The easy way to do it is define a polynomial that is exactly zero when evaluated at the boundary, namely
$$b(x,y) = \prod_{i=1}^{n-1}\left[y - y_i + (x_i - x)\left(\frac{y_i - y_{i+1}}{x_i - x_{i+1}}\right)\right]\left[y- y_n + (x_n - x)\left(\frac{y_n - y_1}{x_n - x_1}\right)\right] \enspace ,$$
being $(x_i,y_i)$ the coordinates of each vertex.

```python
# Polynomial defining the boundaries
def b(x,y,n):
    prod = 1
    for k in range(0, n):
        prod = prod * ((y - poly[lines[k][0]][1])*(poly[lines[k][0]][0] - poly[lines[k][1]][0]) -
            (x - poly[lines[k][0]][0])*(poly[lines[k][0]][1] - poly[lines[k][1]][1]))
    return prod.expand()

bound = b(x, y, npts)
```

And the boundary function looks like...

```python
b_num = lambdify((x,y), bound, "numpy")
X,Y = np.mgrid[-1:1:50j, -1:1:50j]
Z = b_num(X,Y)
```

```python
fig = plt.figure(figsize=(10, 4))
ax = fig.add_subplot(121, projection='3d')
ax.plot_surface(X, Y, Z, cmap="RdYlBu", lw=0.2, edgecolor="black",
                vmin=-2, vmax=2)
surf = ax2 = fig.add_subplot(122)
cont = ax2.contourf(X, Y, Z, 12, cmap="RdYlBu", vmin=-2, vmax=2)
ax2.fill(np.array(poly)[:,0], np.array(poly)[:,1], fill=False, ec='k', lw=1)
ax2.axis("image");
```

    <IPython.core.display.Javascript object>

### The function

The function is then given by the product of a boundary function and the linear combinations of (nonredundant) functions over the domain
$$\hat{u}(x,y) = b(x,y)\sum\limits_{n=0}^{N-1} c_{n} W_{n}(x,y) \enspace .$$

The terms $W_{n}(x,y)$ are functions that should be linear independent, preferably a complete basis on the space solution. In our case we choose polynomials since we want them to be easilly integrable.

```python
def w_fun(x, y, m, n):
    """ Trial function. """
    c = symbols('c:%d' % (m*n)) # This is the way of define the coefficients c_i
    w = []
    for i in range(0, m):
        for j in range(0, n):
            w.append(x**i * y**j)

    return w, c

def u_fun(x, y, m, n):
    """ Complete function. Contains the boundary and trial functions. """
    w, c = w_fun(x, y, m, n)
    return [b(x, y, npts) * phi for phi in w ], c

m = 2
n = 2
u, c = u_fun(x, y, m, n)
```

### Matrices computation

```python
dudx = [diff(u[k], x) for k in range(len(c))]
dudy = [diff(u[k], y) for k in range(len(c))]
```

```python
Kaux = Matrix(m*n, m*n, lambda ii, jj: dudx[ii]*dudx[jj] + dudy[ii]*dudy[jj])
Maux = Matrix(m*n, m*n, lambda ii, jj: u[ii]*u[jj])
K = Matrix(m*n, m*n, lambda i,j: 0)
M = Matrix(m*n, m*n, lambda i,j: 0)
```

```python
for j in range(len(lines)):
    A = [poly[lines[j][0]][0], poly[lines[j][0]][1]]
    B = [poly[lines[j][1]][0], poly[lines[j][1]][1]]
    C = [centroid[0], centroid[1]]
    J = Matrix([[B[0] - A[0], C[0] - A[0]],
                [B[1] - A[1], C[1] - A[1]]])
    detJ = J.det()
    trans = J * Matrix([[r],[s]]) + Matrix(A)
    for row in range(m*n):
        for col in range(row, m*n):
            K_inte = Kaux[row, col].subs({x:trans[0], y:trans[1]})
            M_inte = Maux[row, col].subs({x:trans[0], y:trans[1]})
            K_inte = integrate(K_inte*detJ, (r, 0, 1-s), (s, 0, 1))
            M_inte = integrate(M_inte*detJ, (r, 0, 1-s), (s, 0, 1))
            K[row, col] += K_inte
            M[row, col] += M_inte
            if row != col:
                K[col, row] += K_inte
                M[col, row] += M_inte
```

So far, everything was done in an analytical fashion. This cannot be the case for the solution of eigenvalue problems, since they [need to](http://en.wikipedia.org/wiki/Abel%E2%80%93Ruffini_theorem) be solved in an iterative way. Thus, we convert our analytical matrices to numpy arrays and proceed

```python
Kn = np.array(K).astype(np.float64)
Mn = np.array(M).astype(np.float64)
```

```python
vals, vecs = eigh(Kn, Mn, eigvals=(0,3))
vals
```

    array([ 7.26184612, 18.19131025, 18.19131025, 33.03588824])

### Plot of the modes

They are note pretty neat since they are plotting the polynomials outside their region...

```python
X,Y = np.mgrid[-1:1:200j, -1:1:200j]
verts = np.array(poly, float)
```

```python
path = Path(verts)
mask = 0*X;
for i in range(X.shape[0]):
    for j in range(X.shape[1]):
        mask[i,j] = path.contains_point((X[i,j],Y[i,j]))
```

```python
for i in range(len(vals)):
    U = sum(vecs[j, i]*u[j] for j in range(m*n))
    vecU = lambdify((x,y), U, "numpy")
    Z = vecU(X,Y)*mask
    Z_max = Z.max()
    Z_max = max (Z_max, -Z.min())
    plt.figure(figsize=(4, 4))
    plt.title(r"$k^2=%.2f$" % vals[i], size=16);
    plt.fill(np.array(poly)[:,0], np.array(poly)[:,1], fill=False, ec='k', lw=1)
    plt.contourf(X, Y, Z, 12, cmap="RdYlBu", vmin=-1.2, vmax=1.2)
    plt.axis("image")
    plt.colorbar();
```

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>

## References

1. Arfken, George B., and Hans J. Weber. Mathematical Methods For Physicists International Student Edition. Academic press, 2005.
2. Reddy, J. N. "Applied Functional Analysis And Variational Methods In Engineering,  Mcgraw-Hill College Pa." (1986): 546.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('../styles/custom_barba.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import integrate
import sympy as sm
```

```python
def dxdt(x,t):
    return np.sin(x)

u = np.linspace(-10, 10,25)

soln=integrate.odeint(dxdt,y0=[1],t=u)
soln2=integrate.odeint(dxdt,y0=[np.pi],t=u)
plt.plot(soln)
plt.plot(soln2)
```

```python
#Find stable fixed points and classify their stability
x = sm.symbols("x")

# differential equations
B = x**2-1
# use sympy's way of setting equations to zero
BEqual = sm.Eq(B, 0)

# compute equilibria (fixed points, extrema)
fp = sm.solve(BEqual,x)
print('The equilibria of this system are %s' %fp)
```

    The equilibria of this system are [-1, 1]


# Harmonic oscillator with computer algebra

## Harmonic oscillator
This material includes the derivation of harmonic oscillator solutions using Sage in the following classic cases:

- free oscillator
- free damped oscillator
- forced oscillator with damping: resonance phenomenon

### What is a harmonic oscillator?

The harmonic oscillator is a point particle with mass $m$ in the force field, which depends linearly on the position. The restoring force to the equilibrium position depends linearly on the amount of the deflection. An example of such a force may be the reaction of a flexible body applying Hook's law. In other words, we have a point particle of  mass $m$ in a square potential:

$$ U(x) = \frac{1}{2}k x^2$$
The  Newton equation for this system reads:
$$ m a = -\frac{dU(x)}{dx} = - kx,$$
and because acceleration $a$ is the second derivative of the position with respect to time one gets the differential equation:
$$ m \ddot x   =  -kx.$$
This is the equation of motion for the harmonic oscillator. Another method of obtaining it is to substitute the force derived from the linear span - that is the Hook's law $\vec F = -k \vec x$, to equate $m\vec a = \vec F$.

### Small oscillations

The harmonic oscillator is an extremely important model that appears in many real situations, both in classical and quantum systems. We now show why? Note that for any potential $U(x)$, which at some point $x_0$, say $x_0=0$,  has a local minimum, its Taylor series takes the form:

$$ U(x) = U(0) + U'(0)x + \frac{1}{2}U''(0) x^2+ ...$$

Since we have assumed that at $x_0=0$ the potential has a local minimum, the derivative at this point is zero, i.e.,  $U'(0) = 0$ and therefore:

$$ U(x) = U(0) + \frac{1}{2}U''(0) x^2+ ...$$

The Taylor's series can be approximated by keeping only the first two non-vanishing terms. The first term $U(0)$ is constant and can be neglected because the corresponding force does not depend on this constant.  As a result, the potential is approximated by the expression:

$$ U(x) = \frac{1}{2}U''(0) x^2.$$

The potential is identical to the potential of the harmonic oscillator and  the second derivative of the potential at the point of its minimum $U"(0)$ can be identified with the elastic constant $k$. So we see that the movement of the particle in the vicinity of the potential minimum  can be approximated by a harmonic oscillator. This fact is the reason why the harmonic oscillator is such frequently used in physics.

## Free oscillator

Consider the Newton equation for a harmonic oscillator in a vacuum (there is no dissipation,no a frictional force) It has a form:
$$  m \ddot x   =  -kx.$$
It is conveniet to convert this equation to the form
$$  \ddot x   =  -\omega_0^2x,$$
where $\omega_0 = \sqrt{\frac{k}{m}}$ is a positive number.

It is a linear second order  ordinary differential equation with constant coefficients. Equations of this class can be easily solved - assuming the form of a solution and substituting it into the Newton  equation. With the computer algebra system included in SageMath, we can simplify this procedure using the desolve function:

```python
load('cas_utils.sage')
```

```python
var('omega0 x0')
assume(omega0>0)

var('t')
X = function('X')(t)

osc = diff(X,t,2) == -omega0^2*X
showmath( osc )
```

Because we use a set of variables for operations in expressions containing derivatives, the above cell gives us a mathematical formula of the differential equation representing the harmonic oscillator.

With this expression, we can use computer algebra to solve the differential equation:

```python
phi_anal = desolve(osc,dvar=X,ivar=t,show_method=True)
showmath(phi_anal)
```

First, we see that we must assume that $\omega_0$ is non-zero. Otherwise, the solution would have a different form.

Secondly, we see that the so-called a general solution depends on two constants $K_1$ and $K_2$. We can determine these two constants knowing the position and velocity of the oscillator at some given time $t_0$. Because the equation of motion do not depend explicitly on time, the solution $x(t) = x(t, t_0) = x(t-t_0) depends on the time difference $t-t_0. Therefore, without the limitation of generality, $t=0$ can be used as an initial moment.

Sage can also determine constants. If at the moment $t=0$  the oscillator is  in the rest state: $x(0)=x_0$ and $v(0)=0$, then we have:

```python
phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,x0,0])
showmath(phi_anal)
```

If, at $t=0$, the initial conditions are:  $x(0)=0$ and $v(0)=v_0$, then we have:

```python
var('v0 x0')
phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,0,v0])
showmath(phi_anal)
```

Finally, for a general case when  $x(0)=x_0$ and $v(0)=v_0$ the result reads

```python
var('v0 x0')
phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,x0,v0])
showmath(phi_anal)
```

We can also assume specific numerical values $x(0)=2$ and $v(0)=3$. Then

```python
phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,2,3])
showmath(phi_anal)
```

Let's plot this solution with initial conditions $x_0=2$  $v_0=3$ for  $\omega_0=1$. We see that the solution is tangent to $3/\omega_0$ line that passes the point $(0,2)$:

```python
plot(phi_anal.subs(omega0==1),(t,0,5),figsize=4)+\
 plot( 3*t+2,(t,-1,1),color='gray')+\
 point([0,2],color='red')
```

We can examine how the solution depends on the parameters $\omega_0$ and $x_0$ (with $v_0=0$).

```python
#@interact
def free_oscilator_xt(w0=slider(0.1,3,0.1,default=1),x0=slider(0.1,3,0.1,default=1)):
    phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,1,0])
    p = plot(phi_anal.subs({omega0:1}),(t,0,7),color='gray')
    phi_anal = desolve(osc,dvar=X,ivar=t,ics=[0,x0,0])
    p += plot(phi_anal.subs({omega0:w0}),(t,0,7))
    p.show( ticks=[[0,pi/2,pi,3/2*pi,2*pi],1/2],tick_formatter="latex",figsize=(6,2))
```

```python
free_oscilator_xt(w0=1, x0=1)
free_oscilator_xt(w0=3, x0=1)
free_oscilator_xt(w0=1/2, x0=1)
```

## Damped oscillator
As we have seen from previous considerations, the free harmonic oscillator exhibits oscillations  infinitively long. In real situations,  systems are in contact with its surroundings (environment) which introduces a dissiption process. In other words, the system is subjected to damping forces, which cause the oscillations  to decay in time. It is the so called damped oscillator. A special case of the friction force, which is taken into account when testing the motion of the oscillator - is the frictional force  proportional to the velocity:
$$ \vec F_D = - \Gamma \vec v, $$
where $\Gamma$ is a damping (friction) constant. It is so called the Stokes force or the drag force
Such friction occurs, for example, when the ball is in the liquid with small Reynolds numbers and is called viscous friction. If the Reynolds number is large, then the friction depends in a more complicated way on the velocity. E.g. in aerodynamics,  a quadratic relationship is a good model.

Consideration of a linearly rate-dependent friction force has the basic advantage that, despite adding an additional term  to the equation of motion, we still  deal with a linear differential equation with constant coefficients and a full mathematical analysis of solutions can be carried out. So we have the equation:

$$m \ddot x   + \Gamma \dot x  + kx  =  0.$$

Let's divide this equation by $m$ and introduce the rescaled damping constant $\gamma=\Gamma/m$  and
 $\omega_0 =k/m$. Then

$$\ddot x   + \gamma \dot x  + \omega_0^2 x  =  0.$$

From  theory of linear differential equations we know that we should consider the roots of a characteristic polynomial  for a given equation and depending on their nature we obtain various different solutions. It can be easily done assuming that the solution of the Newton equation in an exponent $e^{k t}$ and substitute this form into ODE:

```python
var('k t')
var('omega omega0')
var('g', latex_name='\gamma')

f = exp(k*t)
eq = f.diff(t,2)+g*f.diff(t)+omega0^2*f
showmath(eq)
```

Now we can factorize this expression:

```python
eq.factor().show()
```

It is zero for all $t$ only when ${\gamma} k + k^{2} + \omega_{0}^{2}=0$. This is a quadratic equation which has the following solutions:

```python
showmath( (eq.factor()*exp(-k*t)).solve(k) )
```

There are three different values of the determinant:

$$\Delta = \sqrt{\gamma^2-4 \omega_0^2}$$

for which the solution has qualitatively different properties:

 - Two complex roots - damped oscillations.
 - Two real negative roots - strongly damped motion, no oscillation.
 - One degenerate root  ($\Delta=0$ ) - critical vibrations and  it can have one maximum.

### Two complex roots:
Since both $\gamma$ and $\omega_0$ are positive and the expression for $\Delta$  factorizes:

$$ \gamma^2-4 \omega_0^2 = ( \gamma-2 \omega_0)( \gamma+2 \omega_0),$$

the expression under square root will be negative if and only if

\begin{equation}
\label{eq:osc_1}
\gamma-2 \omega_0 < 0
\end{equation}

```python
var('omega omega0')
var('g', latex_name='\gamma')

forget()
assume(g-2*omega0<0)
assume(g>0)
assume(omega0>0)
show( assumptions() )

osc = diff(X,t,2) == -g*diff(X,t)-omega0^2*X

showmath(osc)
```

```python
phi_anal = desolve(osc,dvar=X,ivar=t)
showmath(phi_anal)
```

In Sage the above solution contains two  constants which have no "handles":

```python
phi_anal.variables()
```

It means that expession like `.subs({_K1:2})` will throw an exception.

We can however define symbolic variables for all constants in solution. Note that those constants start with  underscore character, thus we can do it automatically:

```python
[var(str(s)) for s in phi_anal.variables() if str(s).startswith("_")]
```

Now we have `_K1` variable available, and we can use it in substitutions:

```python
_K1
```

Since the solution is in the form of a periodic function multiplied by exponent, we might want to draw the envelope:

```python
var('t')
pars={_K1:11,_K2:2,omega0:1,g:.51}
A = sqrt(_K1^2+_K2^2).subs(pars)
plot( phi_anal.subs(pars), (t,0,15), figsize=(10,2)) + \
    plot( (A*exp(-1/2*g*t)).subs(pars), (t,0,15),color='gray' ) +\
    plot( (-A*exp(-1/2*g*t)).subs(pars), (t,0,15),color='gray' )
```

**Where did the formula for A come from?**

We can use the trigonometric identity  which allow to add $\sin$ and $\cos$ functions with the same frequency:
$$\sqrt{2} \sin\left(\frac{1}{4} \, \pi + {x}\right)=\cos\left({x}\right) + \sin\left({x}\right)$$

$$\sqrt{a^{2} + b^{2}} \sin\left({x} + \arctan\left(\frac{b}{a}\right)\right)$$

We have:

```python
phi_osc = phi_anal.coefficient(e^(-1/2*g*t))
showmath(phi_osc)
```

We can transform the linear combination of the $\sin$ and $\cos$ functions into one function with a different amplitude. In order to accomplish this in Sage, we can use  wildcard substitutions.

```python
w0 = SR.wild(0)
w1 = SR.wild(1)
w2 = SR.wild(2)
sub3 =  { w1*sin(w0)+w2*cos(w0):sqrt(w1^2+w2^2)*sin(w0+arctan(w2/w1)) }
```

```python
showmath(phi_osc.subs(sub3))
```

Alternatively, you can match the solution to a bit more complicated pattern:

```python
w0 = SR.wild(0)
w1 = SR.wild(1)
w2 = SR.wild(2)
w3 = SR.wild(3)
sub4 =  { w3*(w1*sin(w0)+w2*cos(w0)):w3*sqrt(w1^2+w2^2)*sin(w0+arctan(w2/w1)) }
```

```python
showmath(phi_anal.subs(sub4))
```

On the other hand if have expression in the form of  $\sin(x+\phi) e^{-x}$, then the exponent term is an envelope:

```python
plot(sin(x)*exp(-x/3) ,(x,0,10),figsize=(6,2))+\
 plot([exp(-x/3),-exp(-x/3)],(x,0,10),color='red')
```

we have therefore:

```python
a,b = 2,1.2
plot([(a*sin(x)+b*cos(x))*exp(-x/3),sqrt(a^2+b^2)*exp(-x/3)],\
     (x,0,10), figsize=(6,2))
```

### Two real roots

If the determinant $\Delta$ is positive (but non-zero), i.e.:

\begin{equation}
\label{eq:osc_2}
\gamma-2 \omega_0 > 0,
\end{equation}

then we have:

```python
var('omega omega0')
var('g', latex_name='\gamma')
forget()
assume(g-2*omega0>0)
assume(g>0)
assume(omega0>0)
show( assumptions() )
osc = diff(X,t,2) == -g*diff(X,t)-omega0^2*X
showmath(osc)
```

```python
phi_anal = desolve(osc,dvar=X,ivar=t)
showmath(phi_anal)
```

We see that the solution does not contain periodic functions but  is the sum of two exponents with negative arguments.

Task: Prove that the argument of the exponent
$k_{1} e^{\left(-\frac{1}{2} \, {\left(\gamma - \sqrt{\gamma^{2} - 4 \, \omega_{0}^{2}}\right)} t\right)}$,
for $t>0$, is negative.

In this case, solutions are decaying without oscillations, however, for certain parameters it is possible that a single extremum will occur:

```python
var('t')
plot( phi_anal.subs({_K1:-1,_K2:1,omega0:1,g:2.1}), (t,0,15), figsize=(10,2))
```

```python
var('t')
plot( phi_anal.subs({_K1:-1,_K2:0,omega0:1,g:2.1}), (t,0,15), figsize=(10,2))
```

### The degenerate case

Consider the case when the determinant is zero,

\begin{equation}
\label{eq:osc_3}
\gamma-2 \omega_0 = 0
\end{equation}

Let us have a look how the general solution looks:

```python
var('omega omega0')
var('g', latex_name='\gamma')

forget()
assume(g-2*omega0==0)
assume(g>0)
assume(omega0>0)
show( assumptions() )
osc = diff(X,t,2) == -g*diff(X,t)-omega0^2*X
showmath(osc)
phi_anal = desolve(osc, dvar=X, ivar=t)
showmath(phi_anal)
```

This case is called:  critically damped.

```python
var('t')
plot( phi_anal.subs({_K1:-1,_K2:1,g:1}), (t,0,15), figsize=(10,2))
```

### Dissipated power

If we know analytical solutions then  one can evaluate  how much of system energy is dissipated to the environment. For a given value of  $\omega_0$ we might expect that power remains constant in two cases: $\gamma=0$ and $\gamma\to\infty$. Let's check it out - first we can obtain analytical solutions in oscilating and damped regimes. We will take initial condition $x_0=1$ and $v_0=1$:

```python
forget()
assume(g>2*omega0)
assume(g>0)
assume(omega0>0)
x_damped = desolve(osc, dvar=X, ivar=t, ics=[0,1,0] )

forget()
assume(g<2*omega0)
assume(g>0)
assume(omega0>0)
x_oscil = desolve(osc, dvar=X, ivar=t, ics=[0,1,0] )
```

Now, let's define functions which characterize dissipated power $P$  (it is energy or work per unit time):

\begin{equation}
\label{eq:power_harm}
P = F_{friction} v = (\gamma v) v =  \gamma v^2
\end{equation}

and total energy (for $m=1$):

\begin{equation}
\label{eq:E_harm}
E_{tot} = \frac{1}{2}v^2 + \frac{1}{2} \omega_0 x^2
\end{equation}

```python
E = lambda x: 1/2 * x.diff(t)^2 + 1/2*omega0*x^2
P = lambda x: g * (x.diff(t))^2
```

Now we can plot power  and trajectory as a function of time:

```python
plt_power = plot([P(x_damped).subs({omega0:1,g:g_}) for g_ in [2.01,4,10]],\
     (t,0,10), color='black')
plt_power += plot([P(x_oscil).subs({omega0:1,g:g_}) for g_ in [.1,1,1.99]],\
      (t,0,10), color='red')
plt_power.show(figsize=(6,2))
```

In the above figure, the dissipated power  $P$ is shown for selected values of the damping constant.

```python
plt_E = plot([E(x_damped).subs({omega0:1,g:g_}) for g_ in [2.01,4,10,1e3]],\
             (t,0,10),color='black')
plt_E += plot([E(x_oscil).subs({omega0:1,g:g_}) for g_ in [1e-3,.1,1,1.99]],\
              (t,0,10),color='red')
plt_E.show(figsize=(6,2))
```

In the above figure, the total energy $E_{tot}$ is depicted for various damping constants.

```python
plt_x = plot([(x_damped).subs({omega0:1,g:g_}) for g_ in [2.01,4,10]],\
             (t,0,10),color='black')
plt_x += plot([(x_oscil).subs({omega0:1,g:g_}) for g_ in [.1,1,1.99]],\
              (t,0,10),color='red')
plt_x.show(figsize=(6,2))
```

The above figure shows the solutions $x(t)$ for different damping constant.

## Forced harmonic oscillator
Now, we take into account the driving in the form of a time-dependent periodic force $\sin(\omega t)$.   In this case, the equation of motion  contains a term independent of the position $x(t)$, but explicitely dependent  on time. Such an equation is a linear non-homogeneous differential equation and we can give its analytical  solutions.
Let's see how they look:

```python
var('a omega omega0')
var('g', latex_name='\gamma')
forget()
assume(g-2*omega0<0)

Phi = function('Phi')(t)
assume(g>0)
assume(omega0>0)
osc = diff(Phi,t,2)+ g*diff(Phi,t) + omega0^2*Phi -a*sin(omega*t)

showmath(osc)
```

```python
phi_anal,method = desolve(osc,dvar=Phi,ivar=t,show_method=True)
print(method)
showmath(phi_anal)
```

It can be seen that we have a solution which is a sum of two terms: a general solution of a free damped oscillator, hence a homogeneous equation (contains $k_1$ and $k_2$ constants) and a solution that does not contain free constants. It is  called a special solution to the heterogeneous equation. It can be noted that this solution is the one that survives in the limit $t\to\infty$.

```python
r_sz = phi_anal.operands()[1]
showmath(r_sz)
```

Let's transform a special solution to the  form: $$A\sin(\omega t+\phi).$$
To do this, we extract  the numerator and denominator:

```python
expr_denom = r_sz.denominator()
expr = r_sz.numerator()
showmath([expr,expr_denom])
```

The numerator is a sum of $\sin$ and $\cos$ with different amplitudes. The formula: $$a \sin\left(x\right) + b \cos\left(x\right) =\sqrt{a^{2} + b^{2}} \sin\left(x + \arctan\left(\frac{b}{a}\right)\right)$$ can be used
for this purpose to put with wildcards (an. Wildcards):

```python
w0 = SR.wild(0)
w1 = SR.wild(1)
w2 = SR.wild(2)
w3 = SR.wild(3)
sub3 =  { w1*sin(w0)+w2*cos(w0):sqrt(w1^2+w2^2)*sin(w0+arctan(w2/w1)) }
```

Let's check how the substitution works on the formula:

```python
var('a b x')
assume(a>0)
(a*sin(x)+b*cos(x)).subs(sub3).show()
```

and expand the obtained formula to check:

```python
assume(a>0)
(a*sin(x)+b*cos(x)).subs(sub3).full_simplify().show()
```

Everything looks nice, but we have a classic problem to choose the function's branch. Let's look at the graph of the $\tan(x)$ function:

```python
print (arctan(10)-pi).n(),(arctan(-10)-pi).n()

plot(tan(x),(x,-4.,4),detect_poles='show',figsize=5,ymin=-10,ymax=10).show()
sum([plot(arctan(x)+n*pi,(x,-10.,10),figsize=(5,3)) for n in range(-1,2)])
```

We should take the branch $y\in -\pi .. 0$ and the function in Sage takes $y\in -\pi ..\pi$. Therefore, it is better to use the two-argument `arctan2`:

```python
w4 = SR.wild(4)
w5 = SR.wild(5)
sub3a = {arctan(w4/w5):(arctan2(w4,w5)-pi)}
```

Returning to our oscillator, we can use the substitution:

*Note* `collect (sin (omega * t))` is needed for the pattern to be in a form in which the substitution can be used.

```python
expr = r_sz.numerator().collect(sin(omega*t))
showmath(expr)
```

```python
expr = expr.subs(sub3)
showmath(expr)
```

```python
expr = expr.subs(sub3a)
showmath(expr)
```

and in all its glory our formula is presented as:

```python
r_szczegolne = (expr/expr_denom).canonicalize_radical()
showmath(r_szczegolne)
```

If for some reason we would like to pick its coefficients then we have:

```python
w0 = SR.wild(0)
w1 = SR.wild(1)
m = r_szczegolne.match(w0*sin(w1))
showmath(m)
```

check:

```python
showmath(w0.subs(m)*sin(w1.subs(m)))
```

Thus, the amplitude of the oscillator in the limit $t\to\infty$  is:

```python
A = -w0.subs(m)
showmath(A)
```

It is a famous formula related to the phenomenon of resonance. If the damping is weak then in the case of $\omega_0 \to \omega$, the amplitude of the special solution will increase to very large values, and will be infinite if the system is not damped. This means that the energy from the external source can be pumped in particularly efficient way  if the frequency of the external force is consistent with the eigen-frequency of the system.

The strength of this phenomenon was seen by soldiers passing over the bridge in Angers (http://en.wikipedia.org/wiki/Angers_Bridge), which had just its own frequency equal to the frequency of the military march. The bridge during the march of the column of the army, stared to swing to such an amplitude that it was destroyed.

```python
phase = w1.subs(m)-omega*t-pi
showmath(phase)
```

The maximal amplitude depends on the attenuation coefficient:

```python
#@interact
def plot_A_phase(g_=slider(0.01,1,0.01,label="$\gamma$",default=0.2)):
    pars = {g:g_,a:1,omega0:1}
    print ( g^2*omega^2 + omega^4 - 2*omega^2*omega0^2 + omega0^4 ).subs(pars)
    p = plot(A.subs(pars),(omega,0,10))
    p += plot(phase.subs(pars),(omega,0,10),color='red',detect_poles="show")
    p.show(figsize=(6,2))
```

```python
plot_A_phase(g_=0.1),plot_A_phase(g_=1)
```

### Numerical analysis

We can also numerically integrate the equation of motion for the harmonic oscillator. It requires to know all parameters and initial conditions numerically before, and it does not allow for easy analysis of the generic properties of the solutions. On the other hand numerical procedure can be applied easily to system of ODE which do not have analytical solution. Here, let us compare results obtained from numerical solution with $t\to\infty$ formula:

```python
var('phid',latex_name=r'\dot\varphi')
var('phi',latex_name=r'\varphi')

```

```python
rhs = solve(osc,diff(Phi(t), t, 2))[0].rhs()
#@interact
def plot_A_traj(g_=slider(0.01,2.2,0.01,label="$\gamma$",default=0.2),\
                omega_=slider(0.01,5.134,0.01,label="$\omega$",default=1.4)):
    pars = {g:g_,a:1,omega0:1}
    print ( g^2*omega^2 + omega^4 - 2*omega^2*omega0^2 + omega0^4 ).subs(pars)
    p = plot(A.subs(pars),(omega,0,10),figsize=(10,2))
    p += plot(phase.subs(pars),(omega,0,10),color='red',detect_poles="show")
    pars = {omega:omega_,omega0:1,a:1,g:g_}

```

```python
pars = {omega:1+0.1,omega0:1,a:0.2,g:.2}
ode = [phid,rhs.subs({Phi:phi,diff(Phi,t):phid}).subs(pars)]
times = srange(0,80,0.001)
ics = [0.0,2.1]
sol = desolve_odeint(ode,ics,times,[phi,phid])
line( zip(times,sol[::1,0]),figsize=(10,2) ) + \
 plot( r_szczegolne.subs(pars), (t,0,80),color='red') +\
 plot( (a*sin(omega*t)).subs(pars), (t,0,80),color='gray')
```

```python
rhs = solve(osc,diff(Phi(t), t, 2))[0].rhs()
#@interact
def plot_A_traj(g_=slider(0.01,2.2,0.01,label="$\gamma$",default=0.2),\
                omega_=slider(0.01,5.134,0.01,label="$\omega$",default=1.4)):
    pars = {g:g_,a:1,omega0:1}
    print ( g^2*omega^2 + omega^4 - 2*omega^2*omega0^2 + omega0^4 ).subs(pars)
    p = plot(A.subs(pars),(omega,0,10),figsize=(10,2))
    p += plot(phase.subs(pars),(omega,0,10),color='red',detect_poles="show")
    pars = {omega:omega_,omega0:1,a:1,g:g_}
    ode = [phid,rhs.subs({Phi:phi,diff(Phi,t):phid}).subs(pars)]

    times=srange(0,80,0.001)
    ics=[0.0,2.1]
    sol = desolve_odeint(ode,ics,times,[phi,phid])
    r_szczegolne2 = a*sin(omega*t +pi- \
                          arctan2(-g*omega,(omega^2 - omega0^2)))/\
           sqrt(g^2*omega^2+ omega^4 - 2*omega^2*omega0^2 + omega0^4)
    p2 = line( zip(times,sol[::1,0]),figsize=(10,2) ) + \
     plot( r_szczegolne.subs(pars), (t,0,80),color='red') +\
     plot( (a*sin(omega*t)).subs(pars), (t,0,80),color='gray')
    p += point([omega_,0])
    p.show()
    p2.show()
```

```python
plot_A_traj(g_=0.2,omega_=1.)
plot_A_traj(g_=4.2,omega_=1.)
```

We can further analyze the formula for resonance and e.g. find its maximum in $omega$:

```python
showmath(A.diff(omega))
```

```python
sol = solve(A.diff(omega), omega)
showmath(sol)
```

```python
omega_max = sol[1].rhs()
showmath(omega_max)
```

```python
showmath( (omega_max^2).factor() )
```

```python
p=[]
for g_ in srange(0.1,1,0.1)+srange(1,1.41,.2):
    pars = {g:g_,a:1,omega0:1}
    omega_max_v =  omega_max.subs(pars).n()
    if omega_max_v.is_real():
        p.append( point( (omega_max_v,A.subs(pars).subs(omega==omega_max_v) ),color='red' ) )
    p.append(  plot(A.subs(pars),(omega,0,3)) )
sum(p).show(ymax=11)
```

In above formula we see that in the weak damping limit the resonance frequency tends to internal frequency of the oscillator. But with increased damping we position moves towards lower values of frequency. Also it can be easily seen that for

$$ -\gamma^2+2 \omega_0^2 <0$$

because:

$$ -\gamma^2+2 \omega_0^2 = (\sqrt 2 \omega_0-\gamma)(\sqrt 2 \omega_0+\gamma)$$

for

$$ \sqrt 2 \omega_0 < \gamma, $$

the resonance disappears - i.e. there is not maxima in $A(\omega)$ dependence.

```python
pars = {a:1,omega0:1}
plot3d( A.subs(pars), (omega,0,3),(g,0.2,1) ).show(viewer='tachyon',figsize=4)
```

```python
pars = {a:1,omega0:1}
contour_plot( A.subs(pars), (omega,0,3),(g,0.2,1) )
```

###  Absorbed power

We can also analyze how much power is absorbed by the periodically driven harmonic oscillator. We can use the solution $t\to\infty$ which has a form:

```python
showmath( r_szczegolne )
```

Dissipated power is equal to: force times the velocity, so:

```python
P = g*(r_szczegolne.diff(t))^2
```

We want to integrate the power over one period of oscillations

```python
showmath( P.integrate(t,0,2*pi) )
```

We can plot this formula together with an expression for amplitude.

```python
p=[]
for g_ in [0.25,0.5,1,2,4]:
    pars = {g:g_,a:1,omega0:1}
    assume(omega>0)
    omega_max_v =  omega_max.subs(pars).n()
    if omega_max_v.is_real():
        p.append( point( (omega_max_v,A.subs(pars).subs(omega==omega_max_v) ),color='red' ) )
    p.append(  plot(A.subs(pars),(omega,0,3), color='gray') )
    p.append(  plot(P.subs(pars).integrate(t,0,2*pi/omega)/2,(omega,0,3),color='red') )
sum(p).show(figsize=(6,3))
```

\newpage


```
# Se importan las librerias para calculo simbolico
from IPython.display import display

from sympy import var, simplify, collect, expand, solve, sin, cos, Matrix, eye, diff, Function, expand_power_base, exp
from sympy.physics.mechanics import mlatex, mechanics_printing
mechanics_printing()
```

```
var("s, t, lamda")
```

```
A = Matrix([[-2, 3, 0], [0, 1, 0], [0, 1, -1]])
```

```
(s*eye(3) - A).inv()
```

```
exp(A*t)
```

```
exp(A*t).subs(t, 0)
```

```
exp(A*t).diff(t) == A*exp(A*t)
```

    True

```
(Matrix([[1, -1, 1], [1, 1, 1], [1, -2, 4]]).inv()*Matrix([[exp(-t)], [exp(t)], [exp(-2*t)]])).T
```

```
(Matrix([[1, -1, 1], [1, 1, 1], [1, -2, 4]]).inv()*Matrix([[exp(-t)], [exp(t)], [exp(-2*t)]]))
```

IPythonNotebooks/Sistemas lineales/Prueba matriz exponencial.ipynb


# Computing the capacity of a cube with a re-entrant corner

### Background

The capacity $\text{cap}(\Omega)$ of an isolated conductor $\Omega\subset\mathbb{R}^3$ with boundary $\Gamma$ measures its ability to store charges. It is defined as the ratio of the total surface equilibrium charge relative to its surface potential value. To compute the capacity, we need to solve the following exterior Laplace problem for the equilibrium potential $u$ with unit surface value:
$$
\begin{align}
-\Delta u &= 0\quad\text{in }\Omega^\text{+},\\
u &= 1\quad\text{on }\Gamma,\\
|u(\mathbf{x})| &=\mathcal{O}\left(|\mathbf{x}|^{-1}\right)\quad\text{as }|\mathbf{x}|\rightarrow\infty.
\end{align}
$$
Here $\Omega^\text{+}$ is the domain exterior to $\Omega$.
The total surface charge of an isolated conductor is given by Gauss law as
$$
\text{cap}(\Omega)=-\epsilon_0\int_{\Gamma}\frac{\partial u}{\partial\nu}(\mathbf{x})\,\mathrm{d}\mathbf{x}.
$$
$\nu(\mathbf{x})$ is the outward pointing normal direction for $\mathbf{x}\in\Gamma$, and $\epsilon_0$ is the electric constant with value $\epsilon_0\approx 8.854\times 10^{-12}\,{\rm F/m}$. In the following we will use the normalized capacity $\text{cap}^*(\Omega)=-\frac{1}{4\pi}\int_{\Gamma}\frac{\partial u}{\partial\nu}\,d\mathbf{x}$. The normalized capacity has the value $1$ for the unit sphere.

Using Green's representation theorem and noting that the exterior Laplace double layer potential is zero for constant densities, we can represent the solution $u$ as
$$
u(\mathbf{x}) = -\int_{\Gamma} g(\mathbf{x},\mathbf{y})\phi(\mathbf{y})\,\mathrm{d}\mathbf{y} \quad\text{for all }\mathbf{x}\in\Omega^\text{+},
$$
where $\phi:={\partial u}/{\partial\nu}$ is the normal derivative of the exterior solution $u$ and $g(\mathbf{x},\mathbf{y}):=\frac{1}{4\pi|\mathbf{x}-\mathbf{y}|}$ is the Green's function of the 3D Laplacian. By taking boundary traces, we arrive at the following boundary integral equation of the first kind.
$$
1 = -\int_{\Gamma} g(\mathbf{x},\mathbf{y})\phi(\mathbf{y})\,\mathrm{d}\mathbf{y} =: -\mathsf{V}\phi(\mathbf{x})\quad\text{for all }\mathbf{x}\in\Gamma.
$$
The normalized capacity is now simply given by
$$
\text{cap}^*(\Omega) = -\frac{1}{4\pi}\int_\Gamma \phi(\mathbf{x}) \,\mathrm{d}\mathbf{x}.
$$

### Implementation

We start with the usual imports.

```python
import bempp.api
import numpy as np
```

The grid re-entrant cube is predefined in the shapes module. By default it refines towards the singular corner. As function space on the grid we choose a simple space of piecewise constant functions.

```python
grid = bempp.api.shapes.reentrant_cube(h=0.02, refinement_factor=1)
space = bempp.api.function_space(grid, "DP", 0)
```

Next, we define the right-hand side.

```python
@bempp.api.real_callable
def one_fun(x, n, domain_index, res):
    res[0] = 1

rhs = bempp.api.GridFunction(space, fun=one_fun)
```

The following code defines the left-hand side single-layer boundary operator.

```python
op = bempp.api.operators.boundary.laplace.single_layer(space, space, space)
```

We use GMRES to solve the system. To improve convergence we use a strong form discretisation that automatically preconditions with the mass matrix.

```python
sol, _, iteration_count = bempp.api.linalg.gmres(op, rhs, use_strong_form=True, return_iteration_count=True)

print("Number of iterations: {0}".format(iteration_count))
```

    scipy/sparse/linalg/dsolve/linsolve.py:296: SparseEfficiencyWarning: splu requires CSC matrix format
      warn('splu requires CSC matrix format', SparseEfficiencyWarning)

    Number of iterations: 35

To obtain the capacity we simply integrate the solution across the boundary.

```python
normalized_capacity = 1./ ( 4 * np.pi) * sol.integrate()[0]
print("The normalized capacity is {0}.".format(normalized_capacity))
```

    The normalized capacity is 0.6455178938316151.


Summarizing your Data with Descriptive Statistics

Image Source : https://www.onlinebooksreview.com/articles/best-probability-and-statistics-books

## What is Descriptive Statistics ?
***

Descriptive Statistics, provides a summary of your dataset giving a measure of the centre, dispersion and shape of your data. Here the data is described as a sample of the whole population, and there are no  inferences made from the sample to the whole population, unlike Inferential Statistics, in which we model the data on the basis of probability theory.

In this notebook, we will be focussing on three key elements of Descriptive Statistics :

- Measures Of Central Tendency
    - Mean
    - Median
    - Mode

- Measures Of Spread
    - Range
    - Outliers
    - Interquantile Range
    - Variance

- Dependence
    - Correlation v/s Causation

## What Are We Going To Learn Today ?
***
- **Dataset for use**
    - **house-price prediction** dataset
    - `.shape()` of data
    - Loading `SalePrice` column
- **Mean**
    - What would be the Central Value ?  - MEAN
    - Mean - Mathematical Repesentation
    - Mean `SalePrice`
    - Disadvantage of Mean
- **Median**
    - Median - Odd Number of Observations
    - Median - Even Number of Observations
- **Median and Inter-Quartile Range (IQR)**
    - IQR - Intuition
    - IQR
    - IQR - A boxplot view
- **Outliers**
    - Outliers - IQR way
    - Outlier,mean,median
    - Box & whisker diagram for SalePrice
    - Outliers Found !
- **What we didn't look into ! - Skewness**
    - Plot histogram of data
    - Analysing histogram
- **Mode**
- **Plotting the Mean, Median & Mode for SalePrice column**
    - When is the mean the best measure of central tendency?
    - When is mode the best measure of central tendency?
        - What is nominal data ?
    - When is the median the best measure of central tendency?
    - What is the most appropriate measure of central tendency when the data has outliers?
- **Spread of the data**
    - Mean squared distance from the mean
    - Variance
    - Standard Deviation
- **Correlation**
    - Calculating Correlation Coefficient
    - Correltion Explained
    - Interpretation
    - Correlation is not equal to Causation

## Dataset
***
First things first, we will be using the **housing-price prediction** dataset named **"train.csv"** in this notebook, to work on with.

So let's start by loading the data !

```python
import pandas as pd
data = pd.read_csv("../data/train.csv") # read.csv() is used for rendering a csv in Pandas

data.head()                             # Recollect .head() returns top 5 rows.
```

      Id
      MSSubClass
      MSZoning
      LotFrontage
      LotArea
      Street
      Alley
      LotShape
      LandContour
      Utilities
      ...
      PoolArea
      PoolQC
      Fence
      MiscFeature
      MiscVal
      MoSold
      YrSold
      SaleType
      SaleCondition
      SalePrice

      0
      1
      60
      RL
      65.0
      8450
      Pave
      NaN
      Reg
      Lvl
      AllPub
      ...
      0
      NaN
      NaN
      NaN
      0
      2
      2008
      WD
      Normal
      208500

      1
      2
      20
      RL
      80.0
      9600
      Pave
      NaN
      Reg
      Lvl
      AllPub
      ...
      0
      NaN
      NaN
      NaN
      0
      5
      2007
      WD
      Normal
      181500

      2
      3
      60
      RL
      68.0
      11250
      Pave
      NaN
      IR1
      Lvl
      AllPub
      ...
      0
      NaN
      NaN
      NaN
      0
      9
      2008
      WD
      Normal
      223500

      3
      4
      70
      RL
      60.0
      9550
      Pave
      NaN
      IR1
      Lvl
      AllPub
      ...
      0
      NaN
      NaN
      NaN
      0
      2
      2006
      WD
      Abnorml
      140000

      4
      5
      60
      RL
      84.0
      14260
      Pave
      NaN
      IR1
      Lvl
      AllPub
      ...
      0
      NaN
      NaN
      NaN
      0
      12
      2008
      WD
      Normal
      250000

5 rows × 81 columns

### **`.shape()`** of the data
***

So above is what our data looks like. Let's check out the shape of the dataset.

```python
print(data.shape)
```

    (1460, 81)

That is 1460 rows/observations and 81 columns/features.

## Loading the `SalePrice` column
***

The last column  **SalePrice**  which is also the predictor for this dataset is what we are interested in. Let's start by loading the first 15 values of this column.

```python
Sales_price = pd.Series(data['SalePrice'])
Sales_price.head(15)
```

    0     208500
    1     181500
    2     223500
    3     140000
    4     250000
    5     143000
    6     307000
    7     200000
    8     129900
    9     118000
    10    129500
    11    345000
    12    144000
    13    279500
    14    157000
    Name: SalePrice, dtype: int64

## What does it show ?
***
- If you look at the values, they are spread all over.
- Some houses are ~120,000 dollars and some are over ~200,000
- AND THIS IS JUST IN 15 OBSERVATIONS OF THE DATA

## What would be the Central Value ?  - MEAN
***
- It would be a good thing to find out the value around which all the prices are centered.
- **Arithmetic Mean** also known as **Average** or just **Mean**, would give us a value central to all the observations and is calculated as :

        Mean = Sum of all Observations / No. of Observations

## Mean - Mathematical Representation
***
Mathematically, mean can be applied on a large scale to almost any number of observations one wants to consider and is represented as:
                $$ \bar{x} = \frac{1}{N}\sum_{i=1}^{N}x_i $$

Where,

-       N = Total number of observations
-       x_i = 'i'th observation (for i= 1,.....,N)

The advantage of mean is that it takes every value into account, easy to calculate and further analysis calculation can be done easily as it is a unique value of the dataset.

## Mean `SalePrice`
***

Let's find out the mean for the **SalePrice** column

```python
import numpy as np
mean = np.mean(Sales_price)
print(mean)
```

    180921.19589041095

- There we go. Our mean is around 180,922 dollars!
- Now we atleast get a fair idea of what to expect.
- But is the mean a value we can fairly latch on to ?

## Disadvantage of Mean
***

- Finding mean is not a good approach as the **'Mean is often affected by Outliers'** or in simple words if there are some observations larger or smaller than majority of the other observations then the mean tends to deviate towards these values.

- To generalize it if the distribution of datasets is skewed(troubled by outliers), we do not choose mean. Here we will have to go for **Median**.

## Median
***

- The **Median** is the nothing but the middle value when all values are arranged in Ascending Order!

- **It is the value above which lies the upper half of the data and below which lies the lower half. In other words, it is the middle value of a data set.**

## Median - Odd Number of Observations
***

- If number of values is ODD – then the median is given as:
     - Median = (N+1)/2 th position element.

- Let's understand this with an example
- Example – Find the median of the values – 110, 90, 40, 50, 125, 65, 100
     - Arranging values in ascending order – 40, 50, 65, 90, 100, 110, 125
     - Here N = 7 (Odd), Median = (7+1)/2 = 4th position element = 90

## Median - Even Number of Observations
***
- If number of values is EVEN then median will be *average* of  **(N/2)th and ((N/2) +1)th** element

- Example – Find the median of the values –10, 15, 20, 23, 24, 25, 25, 27, 30, 32, 40, 45
    - Here N = 12 (Even), Median = Avg of 12/2  = 6th and (12/2)+1 = 7th element =  (25 + 25)/2 = 25.
    - Note that the values are already arranged in Ascending Order

## Median and Inter-Quartile Range (IQR)
***
- Taking the concept of median a step further, we can define the Inter - Quartile Range.
- IQR is a measure of variability and is based on dividing a data set into quartiles.
- Quartile is the division of a set of observations into four intervals based on the values of the data.

## Inter-Quartile Range - Intuition
***

Image Source : http://resizeandsave.online/dappy-January_21_6.html

## Inter-Quartile Range
***

- Basically from the image above: Quartiles divide a rank-ordered data set into four equal parts. The values that divide each part are called the first, second, and third quartiles where
- Q1 is the "middle" value in the first half of the rank-ordered data set. Also called lower-quartile
- Q2 is the median value in the set.
- Q3 is the "middle" value in the second half of the rank-ordered data set. Also called upper-quartile

- The Inter-Quartile Range is the difference between the upper and lower quartiles:
    - Interquartile Range, IQR = Q3 - Q1 (it is the central 50% of data)
    - Where:
        - Q1 = Median of First half of observations arranged in Ascending Order
        - Q3 = Median of 2nd Half of observations arranged in Ascending Order

## Inter-Quartile Range - A Boxplot view
***
The figure below is known as a *Boxplot* or *Box & Whisker Diagram* and depending on our data, it also looks like this:
***

Image Source : Slide 18/31 on https://slideplayer.com/slide/5257962/

## Outliers
***

- The Boxplot above shows some additional observations below MINIMUM and above MAXIMUM. These are **Outliers**.
- There are many ways to mathematically represent or define outliers. One such method is using IQR.

## Outliers - The IQR way
***

- If a data point is 1.5xIQR below the first quartile (Q1) or 1.5xIQR above the third quartile (Q3) then it is an outlier.

- These shown as Black Dots in the boxplot above

- **"*Outliers*" have an effect on the Mean!**

- Let's try and visually understand what this means

## Outliers, Mean and Median
***
Let's consider some scores of Pre-school children in their Reflexes Test
***

Image Source : https://www.hackerearth.com/blog/machine-learning/descriptive-statistics-python-numpy/

## Outliers, Mean and Median
***
- We see that the mean is affected by these "outliers" and does not give a good estimate of all the values
- The first two students from the **right** are outliers since their score is quite far from that of the majority of the class.
- Even though 8 out of 10 students scored below 6, the mean is **6**!
- In this case, taking the median is a much better Estimate as **3.25** represents a good estimate of how the class performed as a whole!

Let's code and find out what the median and IQR for our **SalePrice** column is

```python
import numpy as np
median = np.median(Sales_price)
print(median)

q1 = Sales_price.quantile(0.25) # upper quartile  } Note: The fuction is .quantile() with 'n'
q3 = Sales_price.quantile(0.75) # lower quartile  }       not .quartile() with 'r'
print("Q1:", q1)
print("Q3:", q3)
print("IQR:", q3 - q1)
```

    163000.0
    Q1: 129975.0
    Q3: 214000.0
    IQR: 84025.0

**Mean** = 180921.19589041095

**Median** = 163000.0

That is almost a difference of around 20000. This set of observations is definitely skewed. We'll find out what that means in a while.

## Box & Whisker Diagram for SalePrice
***

Let's do a boxplot for our predictor that is **SalePrice** and get a sense of the outliers if any

```python
import matplotlib.pyplot as plt
%matplotlib inline

plt.boxplot(Sales_price)
plt.show()
```

## Outliers Found !
***
 - Woah ! there are quite a few outliers.
 - Let's code and find out how many ?

```python
## Enter Code to find no. of outliers

outlier_lower_limit = q1 - 1.5*(q3 - q1)
outlier_upper_limit = q3 + 1.5*(q3 - q1)
print(outlier_lower_limit)
print(outlier_upper_limit)
```

    3937.5
    340037.5

```python
lower_limit_outliers = Sales_price[Sales_price < outlier_lower_limit].count()
upper_limit_outliers = Sales_price[Sales_price > outlier_upper_limit].count()
print("lower_limit_outliers:", lower_limit_outliers)
print("upper_limit_outliers:", upper_limit_outliers)
print("total outliers:", upper_limit_outliers + lower_limit_outliers)
```

    lower_limit_outliers: 0
    upper_limit_outliers: 61
    total outliers: 61

## What we didn't look into ! - Skewness
***
 - Although the Box & Whisker plot is a good method to get an intution on the number of outliers; another good way to find out if we need to consider the median is by checking the: Skewness in distribution.
 - We know that the Median is **more robust** to outliers than the mean, thus giving us a better estimate of the "*Central Tendency*" of our dataset!
 - But what is Skewness?
 - Let's build our intuition

## Plot a Histogram of the data
***
 - Let's plot a histogram, which is nothing but a Frequency Plot of the no. of times the values in our particular data are observed!
 - These values are arranged in "bins"
 - Each "bin" is a range of values and their height (or y-values) determine **The number of times values in THAT range appear in our dataset!**
 - Let's code and have a look at the Histogram of our predictor

```python
plt.hist(Sales_price, bins=60)
plt.show()
```

## Analyzing the Histogram
***
 - By just looking, we can tell that the Histogram is not peaked around the center of ranges
 - The *peak* is towards the **left** of the plot, as the rightmost figure below.

 ***

Image Source : http://worldunseen.club/positively-skewed-distribution-math/positively-skewed-distribution-math-a-comparison-of-the-mean-median-and-mode-math-terminology-normal-distribution-math-solver-with-steps/

## Analyzing the Histogram
***
- Thus, we see that our Histogram is "*Positively Skewed*"

- We can see different examples of Skewness from the image on the previous slide and see how Mean, and the Median are affected in each distribution

## Mode
***
 - Simply put:  Mode is the most frequently occurring data point in the given data set.
 - We arrange the data in increasing order before finding the mode
 - Example The following is an ordered array of the values from below

7.00 11.00 14.25 15.00 15.00 15.50 19.00 19.00 19.00 19.00 21.00 22.00 23.00 24.00 25.00 27.00 27.00 28.00 34.22 43.25

 - This grouping makes it easier to see that 19.00 is the most frequently occurring number. In the case of a tie for the most frequently occurring value,two modes are listed.Then the data are said to be bimodal.

```python
mode = Sales_price.mode()
print(mode[0])
```

    140000

## Plotting the Mean, Median & Mode for SalePrice column
***

```python
## plot the hist with mean median and mode - This needs to be checked!

plt.figure(figsize=(10, 6))
plt.hist(Sales_price, bins=40)
plt.plot([mode]*300, range(300), label='mode')
plt.plot([median]*300, range(300), label='median')
plt.plot([mean]*300, range(300), label='mean')
plt.ylim(0, 250)
plt.legend()
plt.show()
```

## When is the mean the best measure of central tendency?
***

- The mean is usually the best measure of central tendency to use when your data distribution is continuous and symmetrical, such as when your data is normally distributed.

- However, it all depends on what you are trying to show from your data.

- You will learn about Normal Distributions in Inferential Statistics

## When is the mode the best measure of central tendency?
***

- The mode is the least used of the measures of central tendency

- The mode will be the best measure of central tendency (as it is the only one appropriate to use) when dealing with nominal data.

- The mean and/or median are usually preferred when dealing with all other types of data, but this does not mean it is never used with these data types

### What is nominal data?
***

- Nominal variables are categorical variables that have two or more categories, but which do not have an intrinsic order.

- For example
    - a real estate agent could classify their types of property into distinct categories such as houses, condos, co-ops or bungalows. So "type of property" is a nominal variable with 4 categories called houses, condos, co-ops and bungalows.

- Of note, the different categories of a nominal variable can also be referred to as groups or levels of the nominal variable.

## When is the median the best measure of central tendency?

- The median is usually preferred to other measures of central tendency when your data set is skewed (i.e., forms a skewed distribution) or you are dealing with ordinal data.

- We already know what skewed data is!

- However, the mode can also be appropriate in these situations, but **IS NOT** as commonly used as the median.

## What is the most appropriate measure of central tendency when the data has outliers?

- The median is usually preferred in these situations because the value of the mean can be distorted by the outliers.

- However, it will depend on how influential the outliers are. If they do not significantly distort the mean, using the mean as the measure of central tendency will usually be preferred.

### In a normally distributed data set, which is greatest: mode, median or mean?

- If the data set is perfectly normal, the mean, median and mean are equal to each other (i.e., the same value).

### For any data set, which measures of central tendency have only one value?

- The median and mean can only have one value for a given data set. The mode can have more than one value

## Spread of the data
***
 - Let's choose the value 250,000 from the SalePrice column and check how far this value is from the mean when compared to other points in the data set
 - We measure this as follows:
             (250,000 - mean)/Random Variation
 - We know the mean, we found that before
 - What is Random Variation?
     - It's nothing but the *Average variation of the data from the mean*
 - How do we measure this?

## To start off
***

 - Range of data is simply:
     - Max Value of Data - Min Value of data
 - Let's check the Range of John's data

```python
Range = np.max(Sales_price)-np.min(Sales_price)
Range
```

    720100

## Mean squared distance from the mean
***
 - Now if we square the differences between mean and every observation and take a sum, we get:
 - We get  =
   $$ {\frac{1}{N}\sum\limits_{i = 1}^N {\left( {x_i - \bar x} \right)^2 } } $$
 - THIS IS CALLED THE VARIANCE!
 - It's the mean squared distance of the observation from the mean

## Variance - Formula
***
- Therefore variance is calculated as:

 $$ {σ}^2 (Variance) = {\frac{1}{N}\sum\limits_{i = 1}^N {\left( {x_i - \bar x} \right)^2 } } $$

- It is denoted by ${σ}^2$

```python
variance = Sales_price.var()
print(variance)
```

    6311111264.297451

## Need for Standard Deviation
***
 - In our case, Variance is calculated in terms of ${dollars}^2$
 - This isn't really practical as the mean of the Prices is expressed in "dollars"
 - Therefore, we can simply square our Variance and obtain

## Standard Deviation
***
- It is *squareroot of the mean squared distance from the mean*
- It is denoted by σ

 $$ σ = \sqrt{\frac{1}{N}\sum\limits_{i = 1}^N {\left( {x_i - \bar x} \right)^2 } } $$

Let's find out the Standard Deviation for our data

```python
from math import sqrt

std = sqrt(variance)
print(std)
```

    79442.50288288663

## Spread of Data - Back to where we began
***
- Let's check what John originally set out to check
- How far the Price of this house is from the mean when compared to other points in the data set
- We want to check this:

(250,000 - mean)/Random Variation

- The Standard Deviation is a good measure of spread of Random Variation in a given data
- Thus we want to check:

(250,000 - mean)/Standard Deviation

```python
(250000 - mean)/std
```

    0.8695446593799336

## Importance of Standard Deviation
***
- If we know that our data is *Normally Distributed*, we can confidently say that:
   - ~68% of the data is within one Std. Dev. from the mean
   - ~95% of the data is within 2 Std. Dev. from the mean
   - ~99.7% of the data is within 3 Std Dev from the mean
    ***

## Correlation
***
 - Now what if we want to know how the price is affected by different factors such as the Living Room area, Garage Area, etc which are some of the other columns/features in our dataset.
 - This is nothing but Correlation!

## Calculating the Correlation Coefficient
***
- A good way to determine this correlation is to use an already existing method known as Pearson Correlation Coefficient
- Correlation Coefficient is a measure of **linear reationship** between two variables
- It always outputs in the range of [-1,1]
- This is a bit complicated but easy when you break it down

$$ r = \frac{N\sum xy - (\sum x)(\sum y)}{\sqrt {[N\sum x^2 -(\sum x)^2][N\sum y^2 -(\sum y)^2]}} $$

$
\begin{align}
Where :
\end{align}
$

$
\begin{align}
N = Number\thinspace of\thinspace pairs\thinspace of\thinspace scores
\end{align}
$

$
\begin{align}
\sum xy = sum\thinspace of\thinspace the\thinspace products\thinspace of\thinspace paired\thinspace scores\thinspace
\end{align}
$

$
\begin{align}
\sum x = sum\thinspace of\thinspace x\thinspace scores\thinspace
\end{align}
$

$
\begin{align}
\sum y = sum\thinspace of\thinspace y\thinspace scores\thinspace
\end{align}
$

$
\begin{align}
\sum x^2 = sum\thinspace of\thinspace squared\thinspace x\thinspace scores\thinspace
\end{align}
$

$
\begin{align}
\sum y^2 = sum\thinspace of\thinspace squared\thinspace y\thinspace scores\thinspace
\end{align}
$

## Correlation Explained
***
- Understanding values:
   - 1 indicates a strong positive relationship.
   - -1 indicates a strong negative relationship.
   - A result of zero indicates a non-linear relationship or no relationship at all.
***

Image Source : https://www.statisticshowto.datasciencecentral.com/direction-of-association/

## Interpretation
***
 - A correlation coefficient of 1 means that for every positive increase of 1 in one variable, there is a positive increase of 1 in the other.
 - A correlation coefficient of -1 means that for every positive increase of 1 in one variable, there is a negative decrease of 1 in the other.
 - Zero means that for every increase, there isn’t a positive or negative increase. The two just aren’t related.

## Let's find out...
***
- What is the Correlation between the Sales price and the Living Room Area?

- Let's find out. This should be easy.

```python
living_room_area = data.GrLivArea
np.corrcoef(Sales_price, living_room_area)[0,1]  # Returns Pearson product-moment correlation coefficients.
```

    0.7086244776126522

## More on Correlation
***
The absolute value of the correlation coefficient gives us the relationship strength. The larger the number, the stronger the relationship. For example, |-.75| = .75, which has a stronger relationship than .65.

 - Also, we can creatively plot "heatmaps" in Python to make our plot fancy and see the Correlation trends between Several variables!

 - This is a very handy trick used by Data Scientists All over the world!

 - The next slide displays the code where The Correlation trends amongst the "Lot Area", Living Room Area, Garage Area and Sales Price of the house are captured!

```python
#considering 4 continous variable and finding the correlation
import seaborn as sns
%matplotlib inline
x = data[['LotArea','GrLivArea','GarageArea','SalePrice']]
corr = x.corr()      # Computes pairwise correlation of columns excluding NaNs
sns.heatmap(corr)
print(corr)
```

## Correlation doesn't imply Causation

However, correlation does not imply causation. There may be, for example, an unknown factor that influences both variables similarly.

Causation indicates that one event is the result of the occurrence of the other event; i.e. there is a causal relationship between the two events. This is also referred to as cause and effect.

A statistically significant correlation has been reported, for example, between yellow cars and a lower incidence of accidents. That does not indicate that yellow cars are safer, but just that fewer yellow cars are involved in accidents. A third factor, such as the personality type of the purchaser of yellow cars, is more likely to be responsible than the color of the paint itself.

```python
df = pd.read_csv('../data/weather_2012.csv')
df.head()
```

      Date/Time
      Temp (C)
      Dew Point Temp (C)
      Rel Hum (%)
      Wind Spd (km/h)
      Visibility (km)
      Stn Press (kPa)
      Weather

      0
      2012-01-01 00:00:00
      -1.8
      -3.9
      86
      4
      8.0
      101.24
      Fog

      1
      2012-01-01 01:00:00
      -1.8
      -3.7
      87
      4
      8.0
      101.24
      Fog

      2
      2012-01-01 02:00:00
      -1.8
      -3.4
      89
      7
      4.0
      101.26
      Freezing Drizzle,Fog

      3
      2012-01-01 03:00:00
      -1.5
      -3.2
      88
      6
      4.0
      101.27
      Freezing Drizzle,Fog

      4
      2012-01-01 04:00:00
      -1.5
      -3.3
      88
      7
      4.8
      101.23
      Fog

```python

```

##  Mini-Challenge - 1
***
In the weather dataset, find the mean and median for Wind Speed and Temperature columns

```python

```

##  Mini-Challenge - 2
***
In the weather dataset, find the mode for the Weather column

```python

```

##  Mini-Challenge - 3
***
Find out the skewness, if any of the Wind Speed and Temperature columns by plotting a histogram

```python

```

##  Mini-Challenge - 4
***
Find out the variance and standard deviation for Dew Point Temperature column.

```python

```

##  Mini-Challenge - 5
***
Find out the correlation between all the continuous features in the weather dataset and plot a heatmap using Seaborn.

```python

```

# In-session Recap Time
***
- Mean
- Median
- Spread
    - Range
    - Interquantile Range
- Mode
- Variance
- Correlation v/s Causation

# Thank You
***

### Next Session: Basic Probability, Hypothesis Testing, t-test e.t.c
For more queries - Reach out to [academics@greyatom.com](academics@greyatom.com)

```python

```