# Lambda School Data Science - Ridge Regression

Regularize your way to a better tomorrow.

# Lecture

Data science depends on math, and math is generally focused on situations where:

1. a solution exists,
2. the solution is unique,
3. the solution's behavior changes continuously with the initial conditions.

These are known as [well-posed problems](https://en.wikipedia.org/wiki/Well-posed_problem), and are the sorts of assumptions so core in traditional techniques that it is easy to forget about them. But they do matter, as there can be exceptions:

1. no solution - e.g. no $x$ such that $Ax = b$
2. multiple solutions - e.g. several $x_1, x_2, ...$ such that $Ax = b$
3. "chaotic" systems - situations where small changes in initial conditions interact and reverberate in essentially unpredictable ways - for instance, the difficulty in longterm predictions of weather (N.B. not the same thing as longterm predictions of *climate*) - you can think of this as models that fail to generalize well, because they overfit on the training data (the initial conditions)

Problems suffering from the above are called ill-posed problems. Relating to linear algebra and systems of equations, the only truly well-posed problems are those with a single unique solution.

Think for a moment - what would the above plot look like if there was no solution? If there were multiple solutions? And how would that generalize to higher dimensions?

# Well-Posed problems in Linear Algebra

A lot of what you covered with linear regression was about getting matrices into the right shape for them to be solvable in this sense. But some matrices just won't submit to this, and other problems may technically "fit" linear regression but still be violating the above assumptions in subtle ways.

[Overfitting](https://en.wikipedia.org/wiki/Overfitting) is in some ways a special case of this - an overfit model uses more features/parameters than is "justified" by the data (essentially by the *dimensionality* of the data, as measured by $n$ the number of observations). As the number of features approaches the number of observations, linear regression still "works", but it starts giving fairly perverse results. In particular, it results in a model that fails to *generalize* - and so the core goal of prediction and explanatory power is undermined.

How is this related to well and ill-posed problems? It's not clearly a no solution or multiple solution case, but it does fall in the third category - overfitting results in fitting to the "noise" in the data, which means the particulars of one random sample or another (different initial conditions) will result in dramatically different models.

## Two Equations with Two Unknowns (well-posed)

\begin{align}
x-y = -1
\end{align}
\begin{align}
3x+y = 9
\end{align}
\begin{align}
\begin{bmatrix}
1 & -1 \\
3 & 1
\end{bmatrix}
\begin{bmatrix}
x \\
y
\end{bmatrix}
=
\begin{bmatrix}
-1 \\
9
\end{bmatrix}
\end{align}

```
import numpy as np
A = np.array([[1, -1], [3, 1]])
b = [[-1],[9]]

solution = np.linalg.solve(A, b)
print(solution)

x = solution[0][0]
y = solution[1][0]

print('\n')
print("x:", x)
print("y:", y)
```

    [[2.]
     [3.]]

    x: 2.0
    y: 3.0

## Two Equations with Three Unknowns (not well-posed)
\begin{align}
x-y+z = -1
\end{align}
\begin{align}
3x+y-2z = 9
\end{align}
\begin{align}
\begin{bmatrix}
1 & -1 & 1 \\
3 & 1 & -2
\end{bmatrix}
\begin{bmatrix}
x \\
y \\
z
\end{bmatrix}
=
\begin{bmatrix}
-1 \\
9
\end{bmatrix}
\end{align}

```
import numpy as np
A = np.array([[1, -1, 1], [3, 1, -2]])
b = [[-1],[9]]

solution = np.linalg.solve(A, b)
print(solution)
```

You can reduce these formulas, but there is no single solution, there are infinitely many solutions where the solution to at least one of these variables must be a function of the other variables.

Example:

<https://www.youtube.com/watch?v=tGPSEXVYw_o>

# Generalization in Machine Learning

The goal of machine learning is to end up with a model that can predict well on new data that it has never seen before. This is sometimes called "out of sample accuracy". This is what we are simulating when we do a train-test-split. We allow or model to fit to the training dataset and then we test its ability to generalize to new data by evaluating its accuracy on a test dataset. We want models that will be usable on new data indefinitely that way we can train them once and then reap the rewards of accurate predictions for a long time to come.

## Underfitting
An underfit model will not perform well on the test data and will also not generalize to new data. Because of this, we can usually detect it easily (it just performs poorly in all situations). Because it's easy to identify we either remedy it quickly or move onto new methods.

```
X = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20])
y = np.array([10,9,8,7,6,5,4,3,2,1,1,2,3,4,5,6,7,8,9,10])

import matplotlib.pyplot as plt
plt.scatter(X,y)
plt.show()
```

```
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)
```

```
plt.scatter(X_train, y_train)
plt.show()
```

```
plt.scatter(X_test, y_test)
plt.show()
```

```
from sklearn.linear_model import LinearRegression

X_train = X_train.reshape(-1, 1)

model = LinearRegression().fit(X_train, y_train)
model.score(X_train, y_train)

beta_0 = model.intercept_
beta_1 = model.coef_[0]

print("Slope Coefficient: ", beta_1)
print("\nIntercept Value: ", beta_0)

plt.scatter(X_train, y_train)
y_hat = [beta_1*x + beta_0 for x in X]
plt.plot(x, y_hat)
plt.show()
```

```
plt.scatter(X_test, y_test)
y_hat = [beta_1*x + beta_0 for x in X]
plt.plot(x, y_hat)
plt.show()
```

## Overfitting

Lets explore the problem of overfitting (and possible remedy - Ridge Regression) in the context of some housing data.

```
import pandas as pd
from sklearn.datasets import load_boston
from sklearn.preprocessing import scale

# Load and Scale the Data
boston = load_boston()
boston.data = scale(boston.data)  # Very helpful for regularization!

# Put it in a dataframe
df = pd.DataFrame(boston.data, columns=boston.feature_names)
df['Price'] = boston.target
df.head()
```

      CRIM
      ZN
      INDUS
      CHAS
      NOX
      RM
      AGE
      DIS
      RAD
      TAX
      PTRATIO
      B
      LSTAT
      Price

      0
      -0.419782
      0.284830
      -1.287909
      -0.272599
      -0.144217
      0.413672
      -0.120013
      0.140214
      -0.982843
      -0.666608
      -1.459000
      0.441052
      -1.075562
      24.0

      1
      -0.417339
      -0.487722
      -0.593381
      -0.272599
      -0.740262
      0.194274
      0.367166
      0.557160
      -0.867883
      -0.987329
      -0.303094
      0.441052
      -0.492439
      21.6

      2
      -0.417342
      -0.487722
      -0.593381
      -0.272599
      -0.740262
      1.282714
      -0.265812
      0.557160
      -0.867883
      -0.987329
      -0.303094
      0.396427
      -1.208727
      34.7

      3
      -0.416750
      -0.487722
      -1.306878
      -0.272599
      -0.835284
      1.016303
      -0.809889
      1.077737
      -0.752922
      -1.106115
      0.113032
      0.416163
      -1.361517
      33.4

      4
      -0.412482
      -0.487722
      -1.306878
      -0.272599
      -0.835284
      1.228577
      -0.511180
      1.077737
      -0.752922
      -1.106115
      0.113032
      0.441052
      -1.026501
      36.2

## preprocessing.scale(x) does the same thing as preprocessing.StandardScaler()

The difference is that `.scale(x)` is a function (lowercase naming convention)

`StandardScaler()` is a class (uppercase naming convention) with some extra functionality, they will both scale our data equally well.

```
#?scale # Like StandardScaler but less functionality
```

```
from sklearn.preprocessing import StandardScaler

# Load and scale the data
boston = load_boston()
scaler = StandardScaler()
boston.data = scaler.fit_transform(boston.data)

# Put it in a dataframe
df = pd.DataFrame(boston.data, columns=boston.feature_names)
df['Price'] = boston.target
df.head()
```

      CRIM
      ZN
      INDUS
      CHAS
      NOX
      RM
      AGE
      DIS
      RAD
      TAX
      PTRATIO
      B
      LSTAT
      Price

      0
      -0.419782
      0.284830
      -1.287909
      -0.272599
      -0.144217
      0.413672
      -0.120013
      0.140214
      -0.982843
      -0.666608
      -1.459000
      0.441052
      -1.075562
      24.0

      1
      -0.417339
      -0.487722
      -0.593381
      -0.272599
      -0.740262
      0.194274
      0.367166
      0.557160
      -0.867883
      -0.987329
      -0.303094
      0.441052
      -0.492439
      21.6

      2
      -0.417342
      -0.487722
      -0.593381
      -0.272599
      -0.740262
      1.282714
      -0.265812
      0.557160
      -0.867883
      -0.987329
      -0.303094
      0.396427
      -1.208727
      34.7

      3
      -0.416750
      -0.487722
      -1.306878
      -0.272599
      -0.835284
      1.016303
      -0.809889
      1.077737
      -0.752922
      -1.106115
      0.113032
      0.416163
      -1.361517
      33.4

      4
      -0.412482
      -0.487722
      -1.306878
      -0.272599
      -0.835284
      1.228577
      -0.511180
      1.077737
      -0.752922
      -1.106115
      0.113032
      0.441052
      -1.026501
      36.2

```
df.shape
```

    (506, 14)

## OLS Baseline Model

```
# Let's try good old least squares!
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

X = df.drop('Price', axis='columns')
y = df.Price

lin_reg = LinearRegression().fit(X, y)
mean_squared_error(y, lin_reg.predict(X))
```

    21.894831181729206

That seems like a pretty good score, but...

Chances are this doesn't generalize very well. You can verify this by splitting the data to properly test model validity.

```
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=43)
lin_reg_split = LinearRegression().fit(X_train, y_train)
print(mean_squared_error(y, lin_reg_split.predict(X)))
print(mean_squared_error(y_test, lin_reg_split.predict(X_test)))

# An overfit model performs well on training data and poorly on test data
```

    22.347018673376052
    26.273991426429014

Oops! 💥 - You have overfitting if you are fitting well to training data, but not generalizing well to test data.

### What can we do?

- Use fewer features - sure, but it can be a lot of work to figure out *which* features, and (in cases like this) there may not be any good reason to really favor some features over another.
- Get more data! This is actually a pretty good approach in tech, since apps generate lots of data all the time (and we made this situation by artificially constraining our data). But for case studies, existing data, etc. it won't work.
- **Regularize!**

## Regularization just means "add bias"

OK, there's a bit more to it than that. But that's the core intuition - the problem is the model working "too well", so fix it by making it harder for the model!

It may sound strange - a technique that is purposefully "worse" - but in certain situations, it can really get results.

What's bias? In the context of statistics and machine learning, bias is when a predictive model fails to identify relationships between features and the output. In a word, bias is *underfitting*.

We want to add bias to the model because of the [bias-variance tradeoff](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff) - variance is the sensitivity of a model to the random noise in its training data (i.e. *overfitting*), and bias and variance are naturally (inversely) related. Increasing one will always decrease the other, with regards to the overall generalization error (predictive accuracy on unseen data).

Visually, the result looks like this:

The blue line is overfit, using more dimensions than are needed to explain the data and so much of the movement is based on noise and won't generalize well. The green line still fits the data, but is less susceptible to the noise - depending on how exactly we parameterize "noise" we may throw out actual correlation, but if we balance it right we keep that signal and greatly improve generalizability.

### Look carefully at the above plot and think of ways you can quantify the difference between the blue and green lines...

```
# Now with regularization via ridge regression
from sklearn.linear_model import Ridge

ridge_reg = Ridge().fit(X, y)
mean_squared_error(y, ridge_reg.predict(X))
```

    21.895862166800143

```
# The score is a bit worse than OLS - but that's expected (we're adding bias)
# Let's try split

ridge_reg_split = Ridge(alpha=0).fit(X_train, y_train) # need to set alpha param to change form OLS. Higher alpha, the more bias
mean_squared_error(y_test, ridge_reg_split.predict(X_test))
```

    26.273991426429053

```
# A little better (to same test split w/OLS) - can we improve it further?
# We just went with defaults, but as always there's plenty of parameters
help(Ridge)
```

    Help on class Ridge in module sklearn.linear_model.ridge:

    class Ridge(_BaseRidge, sklearn.base.RegressorMixin)
     |  Linear least squares with l2 regularization.
     |
     |  Minimizes the objective function::
     |
     |  ||y - Xw||^2_2 + alpha * ||w||^2_2
     |
     |  This model solves a regression model where the loss function is
     |  the linear least squares function and regularization is given by
     |  the l2-norm. Also known as Ridge Regression or Tikhonov regularization.
     |  This estimator has built-in support for multi-variate regression
     |  (i.e., when y is a 2d-array of shape [n_samples, n_targets]).
     |
     |  Read more in the :ref:`User Guide <ridge_regression>`.
     |
     |  Parameters
     |  ----------
     |  alpha : {float, array-like}, shape (n_targets)
     |      Regularization strength; must be a positive float. Regularization
     |      improves the conditioning of the problem and reduces the variance of
     |      the estimates. Larger values specify stronger regularization.
     |      Alpha corresponds to ``C^-1`` in other linear models such as
     |      LogisticRegression or LinearSVC. If an array is passed, penalties are
     |      assumed to be specific to the targets. Hence they must correspond in
     |      number.
     |
     |  fit_intercept : boolean
     |      Whether to calculate the intercept for this model. If set
     |      to false, no intercept will be used in calculations
     |      (e.g. data is expected to be already centered).
     |
     |  normalize : boolean, optional, default False
     |      This parameter is ignored when ``fit_intercept`` is set to False.
     |      If True, the regressors X will be normalized before regression by
     |      subtracting the mean and dividing by the l2-norm.
     |      If you wish to standardize, please use
     |      :class:`sklearn.preprocessing.StandardScaler` before calling ``fit``
     |      on an estimator with ``normalize=False``.
     |
     |  copy_X : boolean, optional, default True
     |      If True, X will be copied; else, it may be overwritten.
     |
     |  max_iter : int, optional
     |      Maximum number of iterations for conjugate gradient solver.
     |      For 'sparse_cg' and 'lsqr' solvers, the default value is determined
     |      by scipy.sparse.linalg. For 'sag' solver, the default value is 1000.
     |
     |  tol : float
     |      Precision of the solution.
     |
     |  solver : {'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga'}
     |      Solver to use in the computational routines:
     |
     |      - 'auto' chooses the solver automatically based on the type of data.
     |
     |      - 'svd' uses a Singular Value Decomposition of X to compute the Ridge
     |        coefficients. More stable for singular matrices than
     |        'cholesky'.
     |
     |      - 'cholesky' uses the standard scipy.linalg.solve function to
     |        obtain a closed-form solution.
     |
     |      - 'sparse_cg' uses the conjugate gradient solver as found in
     |        scipy.sparse.linalg.cg. As an iterative algorithm, this solver is
     |        more appropriate than 'cholesky' for large-scale data
     |        (possibility to set `tol` and `max_iter`).
     |
     |      - 'lsqr' uses the dedicated regularized least-squares routine
     |        scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative
     |        procedure.
     |
     |      - 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses
     |        its improved, unbiased version named SAGA. Both methods also use an
     |        iterative procedure, and are often faster than other solvers when
     |        both n_samples and n_features are large. Note that 'sag' and
     |        'saga' fast convergence is only guaranteed on features with
     |        approximately the same scale. You can preprocess the data with a
     |        scaler from sklearn.preprocessing.
     |
     |      All last five solvers support both dense and sparse data. However,
     |      only 'sag' and 'saga' supports sparse input when `fit_intercept` is
     |      True.
     |
     |      .. versionadded:: 0.17
     |         Stochastic Average Gradient descent solver.
     |      .. versionadded:: 0.19
     |         SAGA solver.
     |
     |  random_state : int, RandomState instance or None, optional, default None
     |      The seed of the pseudo random number generator to use when shuffling
     |      the data.  If int, random_state is the seed used by the random number
     |      generator; If RandomState instance, random_state is the random number
     |      generator; If None, the random number generator is the RandomState
     |      instance used by `np.random`. Used when ``solver`` == 'sag'.
     |
     |      .. versionadded:: 0.17
     |         *random_state* to support Stochastic Average Gradient.
     |
     |  Attributes
     |  ----------
     |  coef_ : array, shape (n_features,) or (n_targets, n_features)
     |      Weight vector(s).
     |
     |  intercept_ : float | array, shape = (n_targets,)
     |      Independent term in decision function. Set to 0.0 if
     |      ``fit_intercept = False``.
     |
     |  n_iter_ : array or None, shape (n_targets,)
     |      Actual number of iterations for each target. Available only for
     |      sag and lsqr solvers. Other solvers will return None.
     |
     |      .. versionadded:: 0.17
     |
     |  See also
     |  --------
     |  RidgeClassifier : Ridge classifier
     |  RidgeCV : Ridge regression with built-in cross validation
     |  :class:`sklearn.kernel_ridge.KernelRidge` : Kernel ridge regression
     |      combines ridge regression with the kernel trick
     |
     |  Examples
     |  --------
     |  >>> from sklearn.linear_model import Ridge
     |  >>> import numpy as np
     |  >>> n_samples, n_features = 10, 5
     |  >>> np.random.seed(0)
     |  >>> y = np.random.randn(n_samples)
     |  >>> X = np.random.randn(n_samples, n_features)
     |  >>> clf = Ridge(alpha=1.0)
     |  >>> clf.fit(X, y) # doctest: +NORMALIZE_WHITESPACE
     |  Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,
     |        normalize=False, random_state=None, solver='auto', tol=0.001)
     |
     |  Method resolution order:
     |      Ridge
     |      _BaseRidge
     |      abc.NewBase
     |      sklearn.linear_model.base.LinearModel
     |      abc.NewBase
     |      sklearn.base.BaseEstimator
     |      sklearn.base.RegressorMixin
     |      builtins.object
     |
     |  Methods defined here:
     |
     |  __init__(self, alpha=1.0, fit_intercept=True, normalize=False, copy_X=True, max_iter=None, tol=0.001, solver='auto', random_state=None)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |
     |  fit(self, X, y, sample_weight=None)
     |      Fit Ridge regression model
     |
     |      Parameters
     |      ----------
     |      X : {array-like, sparse matrix}, shape = [n_samples, n_features]
     |          Training data
     |
     |      y : array-like, shape = [n_samples] or [n_samples, n_targets]
     |          Target values
     |
     |      sample_weight : float or numpy array of shape [n_samples]
     |          Individual weights for each sample
     |
     |      Returns
     |      -------
     |      self : returns an instance of self.
     |
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |
     |  __abstractmethods__ = frozenset()
     |
     |  ----------------------------------------------------------------------
     |  Methods inherited from sklearn.linear_model.base.LinearModel:
     |
     |  predict(self, X)
     |      Predict using the linear model
     |
     |      Parameters
     |      ----------
     |      X : array_like or sparse matrix, shape (n_samples, n_features)
     |          Samples.
     |
     |      Returns
     |      -------
     |      C : array, shape (n_samples,)
     |          Returns predicted values.
     |
     |  ----------------------------------------------------------------------
     |  Methods inherited from sklearn.base.BaseEstimator:
     |
     |  __getstate__(self)
     |
     |  __repr__(self)
     |      Return repr(self).
     |
     |  __setstate__(self, state)
     |
     |  get_params(self, deep=True)
     |      Get parameters for this estimator.
     |
     |      Parameters
     |      ----------
     |      deep : boolean, optional
     |          If True, will return the parameters for this estimator and
     |          contained subobjects that are estimators.
     |
     |      Returns
     |      -------
     |      params : mapping of string to any
     |          Parameter names mapped to their values.
     |
     |  set_params(self, **params)
     |      Set the parameters of this estimator.
     |
     |      The method works on simple estimators as well as on nested objects
     |      (such as pipelines). The latter have parameters of the form
     |      ``<component>__<parameter>`` so that it's possible to update each
     |      component of a nested object.
     |
     |      Returns
     |      -------
     |      self
     |
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from sklearn.base.BaseEstimator:
     |
     |  __dict__
     |      dictionary for instance variables (if defined)
     |
     |  __weakref__
     |      list of weak references to the object (if defined)
     |
     |  ----------------------------------------------------------------------
     |  Methods inherited from sklearn.base.RegressorMixin:
     |
     |  score(self, X, y, sample_weight=None)
     |      Returns the coefficient of determination R^2 of the prediction.
     |
     |      The coefficient R^2 is defined as (1 - u/v), where u is the residual
     |      sum of squares ((y_true - y_pred) ** 2).sum() and v is the total
     |      sum of squares ((y_true - y_true.mean()) ** 2).sum().
     |      The best possible score is 1.0 and it can be negative (because the
     |      model can be arbitrarily worse). A constant model that always
     |      predicts the expected value of y, disregarding the input features,
     |      would get a R^2 score of 0.0.
     |
     |      Parameters
     |      ----------
     |      X : array-like, shape = (n_samples, n_features)
     |          Test samples. For some estimators this may be a
     |          precomputed kernel matrix instead, shape = (n_samples,
     |          n_samples_fitted], where n_samples_fitted is the number of
     |          samples used in the fitting for the estimator.
     |
     |      y : array-like, shape = (n_samples) or (n_samples, n_outputs)
     |          True values for X.
     |
     |      sample_weight : array-like, shape = [n_samples], optional
     |          Sample weights.
     |
     |      Returns
     |      -------
     |      score : float
     |          R^2 of self.predict(X) wrt. y.

How to tune alpha? For now, let's loop and try values.

(For longterm/stretch/next week, check out [cross-validation](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.RidgeCV.html#sklearn.linear_model.RidgeCV).)

```
alphas = []
mses = []

for alpha in range(0, 200, 1):
  ridge_reg_split = Ridge(alpha=alpha).fit(X_train, y_train)
  mse = mean_squared_error(y_test, ridge_reg_split.predict(X_test))
  print(alpha, mse)
  alphas.append(alpha)
  mses.append(mse)
```

    0 26.273991426429053
    1 26.192201358877668
    2 26.118328007697226
    3 26.051117952293595
    4 25.989569283205444
    5 25.93287356811407
    6 25.880372753122625
    7 25.831526788692837
    8 25.785889053385123
    9 25.743087513207442
    10 25.702810145277628
    11 25.66479356379559
    12 25.628814073392263
    13 25.59468057863174
    14 25.56222892458644
    15 25.53131734932431
    16 25.50182280665663
    17 25.473637974726547
    18 25.44666880864133
    19 25.420832527348388
    20 25.396055949160893
    21 25.372274108781504
    22 25.349429102822004
    23 25.327469121742823
    24 25.30634763462536
    25 25.286022699825878
    26 25.266456379775086
    27 25.24761424230921
    28 25.229464934192976
    29 25.211979815108453
    30 25.19513264248022
    31 25.178899299197408
    32 25.163257557659424
    33 25.14818687468406
    34 25.13366821272317
    35 25.11968388357409
    36 25.106217411385522
    37 25.093253412260974
    38 25.080777488180427
    39 25.068776133307583
    40 25.057236651039766
    41 25.046147080399017
    42 25.035496130566347
    43 25.02527312253186
    44 25.015467936977434
    45 25.006070967630855
    46 24.997073079433843
    47 24.98846557095439
    48 24.980240140548986
    49 24.972388855844812
    50 24.96490412616673
    51 24.95777867758141
    52 24.951005530271846
    53 24.944577977990345
    54 24.9384895693689
    55 24.9327340908919
    56 24.92730555135946
    57 24.92219816768905
    58 24.917406351921144
    59 24.912924699309215
    60 24.9087479773882
    61 24.904871115926827
    62 24.901289197679855
    63 24.897997449864803
    64 24.8949912362963
    65 24.892266050117623
    66 24.889817507075655
    67 24.88764133929069
    68 24.885733389477625
    69 24.88408960557926
    70 24.882706035776263
    71 24.88157882384208
    72 24.880704204813682
    73 24.880078500952195
    74 24.879698117969724
    75 24.879559541500758
    76 24.87965933379892
    77 24.879994130641087
    78 24.8805606384229
    79 24.881355631430836
    80 24.882375949277577
    81 24.88361849448833
    82 24.88508023022692
    83 24.886758178151386
    84 24.888649416389928
    85 24.89075107762813
    86 24.89306034730016
    87 24.895574461876226
    88 24.898290707239912
    89 24.90120641714914
    90 24.90431897177517
    91 24.907625796314402
    92 24.911124359668285
    93 24.914812173186736
    94 24.918686789471128
    95 24.92274580123304
    96 24.92698684020521
    97 24.93140757610152
    98 24.93600571562298
    99 24.94077900150688
    100 24.945725211616683
    101 24.950842158070053
    102 24.95612768640294
    103 24.96157967476758
    104 24.96719603316249
    105 24.972974702692664
    106 24.97891365485829
    107 24.985010890870456
    108 24.99126444099231
    109 24.99767236390434
    110 25.004232746092597
    111 25.010943701258537
    112 25.017803369749362
    113 25.02480991800798
    114 25.0319615380414
    115 25.03925644690685
    116 25.04669288621445
    117 25.05426912164612
    118 25.061983442489353
    119 25.069834161185728
    120 25.077819612893087
    121 25.085938155060855
    122 25.09418816701804
    123 25.102568049573183
    124 25.111076224625787
    125 25.119711134788766
    126 25.128471243021377
    127 25.137355032272303
    128 25.14636100513223
    129 25.155487683495902
    130 25.164733608232837
    131 25.174097338866744
    132 25.183577453263027
    133 25.193172547324206
    134 25.2028812346929
    135 25.212702146462046
    136 25.222633930892243
    137 25.232675253135735
    138 25.24282479496694
    139 25.25308125451928
    140 25.26344334602802
    141 25.273909799578966
    142 25.284479360862818
    143 25.29515079093497
    144 25.305922865980495
    145 25.31679437708437
    146 25.32776413000649
    147 25.338830944961526
    148 25.349993656403374
    149 25.361251112814
    150 25.37260217649681
    151 25.384045723373994
    152 25.39558064278813
    153 25.407205837307533
    154 25.418920222535693
    155 25.430722726924202
    156 25.442612291589445
    157 25.45458787013271
    158 25.466648428463827
    159 25.478792944627976
    160 25.491020408635883
    161 25.50332982229701
    162 25.515720199055906
    163 25.528190563831558
    164 25.540739952859465
    165 25.5533674135368
    166 25.56607200427009
    167 25.578852794325684
    168 25.591708863682904
    169 25.604639302889613
    170 25.617643212920317
    171 25.63071970503678
    172 25.643867900650903
    173 25.657086931189966
    174 25.670375937964163
    175 25.68373407203625
    176 25.697160494093474
    177 25.71065437432154
    178 25.724214892280617
    179 25.737841236783435
    180 25.751532605775324
    181 25.765288206216116
    182 25.779107253964067
    183 25.792988973661497
    184 25.8069325986223
    185 25.820937370721257
    186 25.83500254028498
    187 25.849127365984653
    188 25.863311114730404
    189 25.87755306156723
    190 25.891852489572667
    191 25.906208689755893
    192 25.920620960958406
    193 25.93508860975623
    194 25.949610950363557
    195 25.964187304537848
    196 25.97881700148633
    197 25.99349937777395
    198 26.008233777232597
    199 26.023019550871716

```
from matplotlib.pyplot import scatter
scatter(alphas, mses);
```

## What's the intuition? What are we doing?

The `alpha` parameter corresponds to the weight being given to the extra penalty being calculated by [Tikhonov regularization](https://en.wikipedia.org/wiki/Tikhonov_regularization) (this parameter is sometimes referred to as $\lambda$ in the context of ridge regression).

Normal linear regression (OLS) minimizes the **sum of square error of the residuals**.

Ridge regression minimizes the **sum of square error of the residuals** *AND* **the squared slope of the fit model, times the alpha parameter**.

This is why the MSE for the first model in the for loop (`alpha=0`) is the same as the MSE for linear regression - it's the same model!

As `alpha` is increased, we give more and more penalty to a steep slope. In two or three dimensions this is fairly easy to visualize - beyond, think of it as penalizing coefficient size. Each coefficient represents the slope of an individual dimension (feature) of the model, so ridge regression is just squaring and summing those.

So while `alpha=0` reduces to OLS, as `alpha` approaches infinity eventually the penalty gets so extreme that the model will always output every coefficient as 0 (any non-zero coefficient resulting in a penalty that outweighs whatever improvement in the residuals), and just fit a flat model with intercept at the mean of the dependent variable.

Of course, what we want is somewhere in-between these extremes. Intuitively, what we want to do is apply an appropriate "cost" or penalty to the model for fitting parameters, much like adjusted $R^2$ takes into account the cost of adding complexity to a model. What exactly is an appropriate penalty will vary, so you'll have to put on your model comparison hat and give it a go!

PS - scaling the data helps, as that way this cost is consistent and can be added uniformly across features, and it is simpler to search for the `alpha` parameter.

### Bonus - magic! ✨

Ridge regression doesn't just reduce overfitting and help with the third aspect of well-posed problems (poor generalizability). It can also fix the first two (no unique solution)!

```
df_tiny = df.sample(10, random_state=27)
print(df_tiny.shape)
X = df_tiny.drop('Price', axis='columns')
y = df_tiny.Price

lin_reg = LinearRegression().fit(X, y)
lin_reg.score(X, y)  # Perfect multi-collinearity!
# NOTE - True OLS would 💥 here
# scikit protects us from actual error, but still gives a poor model
```

    (10, 14)

    1.0

```
ridge_reg = Ridge().fit(X, y)
ridge_reg.score(X, y)  # More plausible (not "perfect")
```

    0.9760119331942763

```
# Using our earlier test split
mean_squared_error(y_test, lin_reg.predict(X_test))
```

    103.04429449784261

```
# Ridge generalizes *way* better (and we've not even tuned alpha)
mean_squared_error(y_test, ridge_reg.predict(X_test))
```

    41.79869373639458

```
# e.g. (x1^2 + x2^2 + ...) * alpha is the extra penalty from Ridge
```

```
from sklearn.linear_model import RidgeCV
ridgecv = RidgeCV(alphas=[1e-3, 1e-2, 1e-2, 1, 10, 100, 1000]).fit(X, y)

```

## And a bit more math

The regularization used by Ridge Regression is also known as **$L^2$ regularization**, due to the squaring of the slopes being summed. This corresponds to [$L^2$ space](https://en.wikipedia.org/wiki/Square-integrable_function), a metric space of square-integrable functions that generally measure what we intuitively think of as "distance" (at least, on a plane) - what is referred to as Euclidean distance.

The other famous norm is $L^1$, also known as [taxicab geometry](https://en.wikipedia.org/wiki/Taxicab_geometry), because it follows the "grid" to measure distance like a car driving around city blocks (rather than going directly like $L^2$). When referred to as a distance this is called "Manhattan distance", and can be used for regularization (see [LASSO](https://en.wikipedia.org/wiki/Lasso_(statistics%29), which [uses the $L^1$ norm](https://www.quora.com/What-is-the-difference-between-L1-and-L2-regularization-How-does-it-solve-the-problem-of-overfitting-Which-regularizer-to-use-and-when)).

All this comes down to - regularization means increasing model bias by "watering down" coefficients with a penalty typically based on some sort of distance metric, and thus reducing variance (overfitting the model to the noise in the data). It gives us another lever to try and another tool for our toolchest!

## Putting it all together - one last example

The official scikit-learn documentation has many excellent examples - [this one](https://scikit-learn.org/stable/auto_examples/linear_model/plot_ols_ridge_variance.html#sphx-glr-auto-examples-linear-model-plot-ols-ridge-variance-py) illustrates how ridge regression effectively reduces the variance, again by increasing the bias, penalizing coefficients to reduce the effectiveness of features (but also the impact of noise).

```
Due to the few points in each dimension and the straight line that linear regression uses to follow these points as well as it can, noise on the observations will cause great variance as shown in the first plot. Every line’s slope can vary quite a bit for each prediction due to the noise induced in the observations.

Ridge regression is basically minimizing a penalised version of the least-squared function. The penalising shrinks the value of the regression coefficients. Despite the few data points in each dimension, the slope of the prediction is much more stable and the variance in the line itself is greatly reduced, in comparison to that of the standard linear regression
```

```
# Code source: Gaël Varoquaux
# Modified for documentation by Jaques Grobler
# License: BSD 3 clause

import numpy as np
import matplotlib.pyplot as plt

from sklearn import linear_model

X_train = np.c_[.5, 1].T
y_train = [.5, 1]
X_test = np.c_[0, 2].T

np.random.seed(0)

classifiers = dict(ols=linear_model.LinearRegression(),
                   ridge=linear_model.Ridge(alpha=.1))

for name, clf in classifiers.items():
    fig, ax = plt.subplots(figsize=(4, 3))

    for _ in range(6):
        this_X = .1 * np.random.normal(size=(2, 1)) + X_train
        clf.fit(this_X, y_train)

        ax.plot(X_test, clf.predict(X_test), color='gray')
        ax.scatter(this_X, y_train, s=3, c='gray', marker='o', zorder=10)

    clf.fit(X_train, y_train)
    ax.plot(X_test, clf.predict(X_test), linewidth=2, color='blue')
    ax.scatter(X_train, y_train, s=30, c='red', marker='+', zorder=10)

    ax.set_title(name)
    ax.set_xlim(0, 2)
    ax.set_ylim((0, 1.6))
    ax.set_xlabel('X')
    ax.set_ylabel('y')

    fig.tight_layout()

plt.show()
```

Between the first and the second graph, we have decreased the slope (penalized or watered-down coefficients), but we have less variance between our lines)

# Assignment

Following is data describing characteristics of blog posts, with a target feature of how many comments will be posted in the following 24 hours.

https://archive.ics.uci.edu/ml/datasets/BlogFeedback

Investigate - you can try both linear and ridge. You can also sample to smaller data size and see if that makes ridge more important. Don't forget to scale!

Focus on the training data, but if you want to load and compare to any of the test data files you can also do that.

Note - Ridge may not be that fundamentally superior in this case. That's OK! It's still good to practice both, and see if you can find parameters or sample sizes where ridge does generalize and perform better.

When you've fit models to your satisfaction, answer the following question:

```
Did you find cases where Ridge performed better? If so, describe (alpha parameter, sample size, any other relevant info/processing). If not, what do you think that tells you about the data?
```
**I found an alpha that made the model perform only slightly better with the test data. Although I didn't find anything truly insightful with this dataset, I see how Ridge Regression is useful for building generalizable models. Also, I found it interesting that the RidgeCV model was not able to find a better alpha than I found by using a for loop.**

You can create whatever plots, tables, or other results support your argument. In this case, your target audience is a fellow data scientist, *not* a layperson, so feel free to dig in!

### Cleaning Data

```
pd.set_option("display.max_columns", None)
pd.set_option("display.max_rows", None)
df = pd.read_csv('blogData_train.csv', header=None)
df.head()
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      135
      136
      137
      138
      139
      140
      141
      142
      143
      144
      145
      146
      147
      148
      149
      150
      151
      152
      153
      154
      155
      156
      157
      158
      159
      160
      161
      162
      163
      164
      165
      166
      167
      168
      169
      170
      171
      172
      173
      174
      175
      176
      177
      178
      179
      180
      181
      182
      183
      184
      185
      186
      187
      188
      189
      190
      191
      192
      193
      194
      195
      196
      197
      198
      199
      200
      201
      202
      203
      204
      205
      206
      207
      208
      209
      210
      211
      212
      213
      214
      215
      216
      217
      218
      219
      220
      221
      222
      223
      224
      225
      226
      227
      228
      229
      230
      231
      232
      233
      234
      235
      236
      237
      238
      239
      240
      241
      242
      243
      244
      245
      246
      247
      248
      249
      250
      251
      252
      253
      254
      255
      256
      257
      258
      259
      260
      261
      262
      263
      264
      265
      266
      267
      268
      269
      270
      271
      272
      273
      274
      275
      276
      277
      278
      279
      280

      0
      40.30467
      53.845657
      0.0
      401.0
      15.0
      15.52416
      32.44188
      0.0
      377.0
      3.0
      14.044226
      32.615417
      0.0
      377.0
      2.0
      34.567566
      48.475178
      0.0
      378.0
      12.0
      1.479934
      46.18691
      -356.0
      377.0
      0.0
      1.076167
      1.795416
      0.0
      11.0
      0.0
      0.400491
      1.078097
      0.0
      9.0
      0.0
      0.377559
      1.07421
      0.0
      9.0
      0.0
      0.972973
      1.704671
      0.0
      10.0
      0.0
      0.022932
      1.521174
      -8.0
      9.0
      0.0
      2.0
      2.0
      0.0
      2.0
      2.0
      0.0
      0.0
      0.0
      0.0
      0.0
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0

      1
      40.30467
      53.845657
      0.0
      401.0
      15.0
      15.52416
      32.44188
      0.0
      377.0
      3.0
      14.044226
      32.615417
      0.0
      377.0
      2.0
      34.567566
      48.475178
      0.0
      378.0
      12.0
      1.479934
      46.18691
      -356.0
      377.0
      0.0
      1.076167
      1.795416
      0.0
      11.0
      0.0
      0.400491
      1.078097
      0.0
      9.0
      0.0
      0.377559
      1.07421
      0.0
      9.0
      0.0
      0.972973
      1.704671
      0.0
      10.0
      0.0
      0.022932
      1.521174
      -8.0
      9.0
      0.0
      6.0
      2.0
      4.0
      5.0
      -2.0
      0.0
      0.0
      0.0
      0.0
      0.0
      35.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      2
      40.30467
      53.845657
      0.0
      401.0
      15.0
      15.52416
      32.44188
      0.0
      377.0
      3.0
      14.044226
      32.615417
      0.0
      377.0
      2.0
      34.567566
      48.475178
      0.0
      378.0
      12.0
      1.479934
      46.18691
      -356.0
      377.0
      0.0
      1.076167
      1.795416
      0.0
      11.0
      0.0
      0.400491
      1.078097
      0.0
      9.0
      0.0
      0.377559
      1.07421
      0.0
      9.0
      0.0
      0.972973
      1.704671
      0.0
      10.0
      0.0
      0.022932
      1.521174
      -8.0
      9.0
      0.0
      6.0
      2.0
      4.0
      5.0
      -2.0
      0.0
      0.0
      0.0
      0.0
      0.0
      35.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      3
      40.30467
      53.845657
      0.0
      401.0
      15.0
      15.52416
      32.44188
      0.0
      377.0
      3.0
      14.044226
      32.615417
      0.0
      377.0
      2.0
      34.567566
      48.475178
      0.0
      378.0
      12.0
      1.479934
      46.18691
      -356.0
      377.0
      0.0
      1.076167
      1.795416
      0.0
      11.0
      0.0
      0.400491
      1.078097
      0.0
      9.0
      0.0
      0.377559
      1.07421
      0.0
      9.0
      0.0
      0.972973
      1.704671
      0.0
      10.0
      0.0
      0.022932
      1.521174
      -8.0
      9.0
      0.0
      2.0
      2.0
      0.0
      2.0
      2.0
      0.0
      0.0
      0.0
      0.0
      0.0
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0

      4
      40.30467
      53.845657
      0.0
      401.0
      15.0
      15.52416
      32.44188
      0.0
      377.0
      3.0
      14.044226
      32.615417
      0.0
      377.0
      2.0
      34.567566
      48.475178
      0.0
      378.0
      12.0
      1.479934
      46.18691
      -356.0
      377.0
      0.0
      1.076167
      1.795416
      0.0
      11.0
      0.0
      0.400491
      1.078097
      0.0
      9.0
      0.0
      0.377559
      1.07421
      0.0
      9.0
      0.0
      0.972973
      1.704671
      0.0
      10.0
      0.0
      0.022932
      1.521174
      -8.0
      9.0
      0.0
      3.0
      1.0
      2.0
      2.0
      -1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      34.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      27.0

```
#df.dtypes
#df.isnul().sum()
```

```
df.shape
```

    (52397, 281)

```
# Scaling data with sklean scale function
df = pd.DataFrame(scale(df))
df.head()
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      135
      136
      137
      138
      139
      140
      141
      142
      143
      144
      145
      146
      147
      148
      149
      150
      151
      152
      153
      154
      155
      156
      157
      158
      159
      160
      161
      162
      163
      164
      165
      166
      167
      168
      169
      170
      171
      172
      173
      174
      175
      176
      177
      178
      179
      180
      181
      182
      183
      184
      185
      186
      187
      188
      189
      190
      191
      192
      193
      194
      195
      196
      197
      198
      199
      200
      201
      202
      203
      204
      205
      206
      207
      208
      209
      210
      211
      212
      213
      214
      215
      216
      217
      218
      219
      220
      221
      222
      223
      224
      225
      226
      227
      228
      229
      230
      231
      232
      233
      234
      235
      236
      237
      238
      239
      240
      241
      242
      243
      244
      245
      246
      247
      248
      249
      250
      251
      252
      253
      254
      255
      256
      257
      258
      259
      260
      261
      262
      263
      264
      265
      266
      267
      268
      269
      270
      271
      272
      273
      274
      275
      276
      277
      278
      279
      280

      0
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.0
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.0
      0.8758
      -0.140659
      0.578108
      0.92603
      0.0
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.337079
      -0.229670
      -0.249138
      -0.346756
      0.012103
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -1.199772
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      0.0
      -0.045171
      -0.037836
      -0.152885

      1
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.0
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.0
      0.8758
      -0.140659
      0.578108
      0.92603
      0.0
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.301070
      -0.229670
      -0.178225
      -0.315136
      -0.045635
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      0.010755
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      2.199274
      -0.440071
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      0.0
      -0.045171
      -0.037836
      -0.179406

      2
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.0
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.0
      0.8758
      -0.140659
      0.578108
      0.92603
      0.0
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.301070
      -0.229670
      -0.178225
      -0.315136
      -0.045635
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      0.010755
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      2.199274
      -0.440071
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      0.0
      -0.045171
      -0.037836
      -0.179406

      3
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.0
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.0
      0.8758
      -0.140659
      0.578108
      0.92603
      0.0
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.337079
      -0.229670
      -0.249138
      -0.346756
      0.012103
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -1.199772
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      0.0
      -0.045171
      -0.037836
      -0.152885

      4
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.0
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.0
      0.8758
      -0.140659
      0.578108
      0.92603
      0.0
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.328076
      -0.247051
      -0.213681
      -0.346756
      -0.031200
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -0.037666
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      -0.448969
      2.277320
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      0.0
      -0.045171
      -0.037836
      0.536657

```
# df.mean()
# df.std()
```

```
# Dropping columns with all 0's
df = df.drop(columns=[12, 32, 37, 277])
```

```
df.head()
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      33
      34
      35
      36
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      135
      136
      137
      138
      139
      140
      141
      142
      143
      144
      145
      146
      147
      148
      149
      150
      151
      152
      153
      154
      155
      156
      157
      158
      159
      160
      161
      162
      163
      164
      165
      166
      167
      168
      169
      170
      171
      172
      173
      174
      175
      176
      177
      178
      179
      180
      181
      182
      183
      184
      185
      186
      187
      188
      189
      190
      191
      192
      193
      194
      195
      196
      197
      198
      199
      200
      201
      202
      203
      204
      205
      206
      207
      208
      209
      210
      211
      212
      213
      214
      215
      216
      217
      218
      219
      220
      221
      222
      223
      224
      225
      226
      227
      228
      229
      230
      231
      232
      233
      234
      235
      236
      237
      238
      239
      240
      241
      242
      243
      244
      245
      246
      247
      248
      249
      250
      251
      252
      253
      254
      255
      256
      257
      258
      259
      260
      261
      262
      263
      264
      265
      266
      267
      268
      269
      270
      271
      272
      273
      274
      275
      276
      278
      279
      280

      0
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.8758
      -0.140659
      0.578108
      0.92603
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.337079
      -0.229670
      -0.249138
      -0.346756
      0.012103
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -1.199772
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      -0.045171
      -0.037836
      -0.152885

      1
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.8758
      -0.140659
      0.578108
      0.92603
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.301070
      -0.229670
      -0.178225
      -0.315136
      -0.045635
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      0.010755
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      2.199274
      -0.440071
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      -0.045171
      -0.037836
      -0.179406

      2
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.8758
      -0.140659
      0.578108
      0.92603
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.301070
      -0.229670
      -0.178225
      -0.315136
      -0.045635
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      0.010755
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      2.199274
      -0.440071
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      -0.045171
      -0.037836
      -0.179406

      3
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.8758
      -0.140659
      0.578108
      0.92603
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.337079
      -0.229670
      -0.249138
      -0.346756
      0.012103
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -1.199772
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      2.227325
      -0.439113
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      -0.045171
      -0.037836
      -0.152885

      4
      0.010876
      0.112877
      -0.052468
      0.138521
      -0.139108
      0.009598
      0.116182
      -0.020836
      0.368246
      -0.119031
      -0.00031
      0.113595
      0.374998
      -0.11243
      -0.004944
      0.129936
      -0.052468
      0.245038
      -0.161788
      0.080132
      0.119797
      -0.480089
      0.390503
      0.140192
      0.598229
      1.06407
      -0.029263
      0.781584
      -0.324229
      0.581057
      0.931022
      0.8758
      -0.140659
      0.578108
      0.92603
      0.876995
      -0.022591
      0.573526
      1.073282
      -0.029263
      0.763658
      -0.249724
      0.499465
      0.885857
      -0.898355
      0.910838
      -0.015447
      -0.328076
      -0.247051
      -0.213681
      -0.346756
      -0.031200
      -0.320777
      -0.21839
      -0.206545
      -0.310545
      -0.010048
      -0.037666
      -0.747296
      -0.040546
      -0.461678
      -0.008738
      -0.017477
      -0.739905
      -0.432851
      -0.564119
      -0.049292
      -0.058549
      -0.190451
      -0.008738
      -0.025482
      -0.069517
      -0.008738
      -0.128719
      -0.037607
      -0.536624
      -0.008738
      -0.008738
      -0.057888
      -0.039591
      -0.013816
      -0.017477
      -0.14596
      -0.013816
      -0.108441
      -0.259927
      -0.011559
      -0.017477
      -0.015753
      -0.06924
      -0.008738
      -0.013816
      -0.313762
      -0.008738
      -0.020495
      -0.019046
      -0.033289
      -1.226344
      -0.287567
      -0.176444
      -0.229929
      -0.055344
      -0.035782
      -0.05447
      -0.128028
      -0.011559
      -0.160884
      -0.013816
      -0.020495
      -0.017477
      -0.419249
      -0.268361
      -0.145755
      -0.040782
      -0.111464
      -0.231901
      -0.440503
      -0.198537
      -0.554032
      -0.020956
      -0.011559
      -0.102996
      -0.021848
      -0.079002
      -0.123168
      -0.011559
      -0.008738
      -0.070618
      -0.008738
      -0.024331
      -0.205193
      -0.167503
      -0.222754
      -0.062978
      -0.17477
      -0.581809
      -0.288615
      -0.284011
      -0.049679
      -1.36741
      -0.088478
      -0.060961
      -0.055517
      -0.071432
      -0.015753
      -0.008738
      -0.047711
      -0.605361
      -0.045658
      -0.22762
      -0.182952
      -0.011559
      -0.008738
      -0.092655
      -0.708757
      -0.354479
      -0.034971
      -0.008738
      -0.173081
      -0.1669
      -0.075118
      -0.036313
      -0.008738
      -0.035782
      -0.016348
      -0.008738
      -0.441769
      -0.02899
      -0.008738
      -0.017477
      -0.131525
      -0.054821
      -0.034971
      -0.062978
      -0.020495
      -0.008738
      -0.322745
      -0.210391
      -0.099729
      -0.28846
      -0.341206
      -0.019046
      -0.126791
      -0.149668
      -0.075885
      -0.122367
      -0.008738
      -0.349066
      -0.128873
      -0.386666
      -0.421449
      -0.14459
      -0.13115
      -0.289468
      -0.008738
      -0.017477
      -0.008738
      -0.061589
      -0.452126
      -0.02899
      -0.042394
      -0.102712
      -0.064488
      -0.232631
      -0.110406
      -0.035782
      -0.600006
      -0.051013
      -0.008738
      -0.936565
      -0.022281
      -0.064488
      -0.261138
      -0.024331
      -0.047104
      -0.168224
      -0.047711
      -0.210044
      -0.037861
      -0.011559
      -0.017477
      -0.080936
      -0.747271
      -0.079733
      -0.396779
      -0.039591
      -0.364666
      -0.065377
      -0.801457
      -0.431047
      -0.209052
      -0.037352
      -0.008738
      -0.07356
      -0.049292
      -0.030595
      -0.042619
      -0.167443
      -0.121156
      -0.008738
      -0.011559
      -0.144728
      -1.328428
      -0.375136
      -0.526081
      -0.088149
      -0.008738
      -0.123884
      -0.080216
      -0.013816
      -0.16324
      -0.122045
      -0.015753
      -0.209499
      -0.047911
      -0.058712
      -0.051201
      -0.092341
      -0.017477
      -0.358559
      -0.361906
      -0.400283
      -0.445005
      -0.448969
      2.277320
      -0.399224
      -0.43518
      -0.448846
      -0.454696
      2.272362
      -0.427399
      -0.326158
      -0.312402
      -0.08286
      -0.045171
      -0.037836
      0.536657

```
df.describe()
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      33
      34
      35
      36
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      135
      136
      137
      138
      139
      140
      141
      142
      143
      144
      145
      146
      147
      148
      149
      150
      151
      152
      153
      154
      155
      156
      157
      158
      159
      160
      161
      162
      163
      164
      165
      166
      167
      168
      169
      170
      171
      172
      173
      174
      175
      176
      177
      178
      179
      180
      181
      182
      183
      184
      185
      186
      187
      188
      189
      190
      191
      192
      193
      194
      195
      196
      197
      198
      199
      200
      201
      202
      203
      204
      205
      206
      207
      208
      209
      210
      211
      212
      213
      214
      215
      216
      217
      218
      219
      220
      221
      222
      223
      224
      225
      226
      227
      228
      229
      230
      231
      232
      233
      234
      235
      236
      237
      238
      239
      240
      241
      242
      243
      244
      245
      246
      247
      248
      249
      250
      251
      252
      253
      254
      255
      256
      257
      258
      259
      260
      261
      262
      263
      264
      265
      266
      267
      268
      269
      270
      271
      272
      273
      274
      275
      276
      278
      279
      280

      count
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04
      5.239700e+04

      mean
      -2.999339e-14
      2.269494e-14
      -1.710480e-15
      4.823368e-14
      4.230365e-14
      1.527395e-14
      -1.166116e-14
      4.016341e-15
      -5.115664e-14
      2.183072e-14
      4.556417e-14
      7.114238e-15
      -6.862485e-14
      9.852016e-14
      2.954071e-14
      -1.439280e-14
      -1.710480e-15
      -1.068626e-14
      -2.488936e-14
      5.220517e-14
      -6.607316e-15
      -8.034301e-14
      -4.312229e-14
      1.038993e-13
      2.257929e-14
      -1.973137e-14
      -1.817588e-14
      1.541509e-14
      6.136412e-14
      2.358986e-15
      -1.012004e-14
      4.679601e-14
      6.039364e-14
      1.096884e-14
      -1.530391e-14
      2.727849e-14
      -8.382246e-15
      -6.040213e-16
      3.414644e-15
      -1.817588e-14
      1.428379e-14
      -6.491413e-15
      1.670632e-14
      1.143433e-14
      -3.813867e-14
      -5.226630e-15
      4.589135e-15
      -2.490023e-14
      3.995568e-15
      3.295040e-14
      7.850419e-15
      2.735857e-15
      4.619432e-15
      5.869972e-15
      8.298290e-15
      5.420309e-15
      1.957919e-16
      1.125246e-16
      -4.075256e-15
      -8.459624e-16
      7.592856e-15
      -5.166826e-15
      -3.905852e-15
      4.161621e-15
      1.189900e-14
      1.443581e-14
      -1.371667e-16
      -6.398477e-17
      6.386558e-14
      -5.166826e-15
      1.803868e-14
      -1.833356e-14
      -5.166826e-15
      -9.494559e-14
      2.733319e-15
      -7.238797e-15
      -5.166826e-15
      -5.166826e-15
      5.754488e-15
      -6.560771e-15
      1.773376e-15
      1.549916e-15
      6.290142e-15
      3.094736e-15
      1.520575e-15
      9.325958e-15
      -1.382420e-15
      9.953243e-15
      9.501511e-15
      7.508800e-15
      -5.166826e-15
      1.377950e-15
      -2.674426e-15
      -5.166826e-15
      4.765765e-15
      1.066869e-14
      1.415416e-14
      2.521762e-14
      -6.131600e-15
      1.243153e-15
      -1.906255e-15
      7.374528e-16
      2.085732e-14
      -1.491054e-15
      -4.696327e-15
      3.857455e-15
      7.740689e-16
      -1.012574e-16
      -3.591727e-15
      -7.408865e-15
      -3.708092e-15
      -2.609292e-15
      -4.113120e-15
      2.899046e-15
      4.056459e-15
      3.393875e-14
      1.320498e-14
      -2.098049e-14
      -1.303308e-14
      3.157645e-15
      -1.637361e-15
      -1.593334e-14
      -1.278590e-14
      2.442141e-14
      2.616290e-14
      5.944726e-15
      -5.166826e-15
      1.382260e-14
      -5.166826e-15
      -8.283811e-15
      4.200763e-14
      -1.291797e-14
      2.768787e-15
      -2.197275e-14
      -8.075963e-15
      -1.954119e-14
      8.806649e-16
      9.005166e-16
      -7.240777e-16
      -1.067036e-15
      5.359723e-15
      -7.034289e-15
      -2.946521e-15
      -1.425332e-14
      1.150850e-14
      -5.166826e-15
      -6.474155e-16
      5.877828e-15
      -1.723314e-15
      2.086427e-15
      3.472278e-15
      6.885782e-16
      -5.166826e-15
      5.673282e-15
      1.116135e-15
      8.963600e-15
      2.723731e-14
      -5.166826e-15
      3.395785e-15
      3.146892e-15
      -1.212311e-14
      4.492683e-15
      -5.166826e-15
      1.332341e-16
      -4.570494e-15
      -5.166826e-15
      2.250104e-14
      -8.557426e-15
      -5.166826e-15
      -8.610890e-15
      -2.140622e-14
      3.942475e-14
      3.195103e-14
      8.830431e-15
      -1.911279e-15
      -5.166826e-15
      1.013922e-14
      -4.520195e-15
      3.169006e-15
      -6.807676e-15
      -1.458127e-14
      1.142918e-14
      -2.158706e-14
      -1.064646e-15
      6.271090e-16
      -9.480375e-15
      -5.166826e-15
      -3.786321e-15
      -1.208005e-14
      -5.536103e-15
      -4.094521e-16
      1.000182e-15
      -1.624211e-14
      1.252555e-14
      -5.166826e-15
      -4.490591e-15
      -5.166826e-15
      3.020594e-16
      1.684316e-14
      -1.487124e-15
      -1.223192e-14
      -3.762787e-15
      7.457439e-15
      7.815113e-15
      -7.986642e-16
      3.467925e-14
      9.832999e-15
      1.387951e-15
      -5.166826e-15
      3.194475e-14
      2.233387e-15
      5.426598e-15
      -9.186153e-15
      4.110175e-15
      1.856884e-14
      3.223886e-15
      7.870568e-15
      3.062504e-15
      4.866367e-15
      1.254876e-15
      -5.058999e-16
      -8.121706e-15
      1.453447e-14
      3.438897e-14
      4.841848e-15
      -1.883260e-15
      3.696244e-15
      -8.368332e-15
      8.757563e-15
      6.532725e-15
      5.836701e-15
      1.990290e-15
      -5.166826e-15
      -2.322733e-14
      -2.042617e-14
      -5.739070e-15
      1.333620e-14
      -6.594116e-15
      9.503242e-15
      -5.166826e-15
      -2.525975e-17
      1.417995e-15
      -6.462802e-14
      -1.255927e-15
      1.530829e-14
      -7.155662e-15
      -5.166826e-15
      -9.950375e-15
      -3.620639e-15
      -7.576825e-17
      -9.403859e-15
      3.431679e-14
      8.258159e-16
      -1.035342e-14
      1.410021e-15
      -1.293122e-15
      1.043379e-15
      -3.876974e-15
      4.522697e-15
      -8.066111e-15
      -7.630027e-15
      4.305666e-16
      5.683470e-15
      -1.572755e-15
      -3.174128e-15
      -3.631908e-16
      -4.893404e-15
      2.326300e-15
      2.806023e-15
      -1.989447e-15
      -2.350538e-15
      1.776135e-14
      -1.213259e-14
      1.155235e-14
      -1.559042e-14
      4.658095e-16
      -9.420359e-15

      std
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00
      1.000010e+00

      min
      -4.985293e-01
      -7.505960e-01
      -5.246776e-02
      -7.698984e-01
      -3.546302e-01
      -4.717580e-01
      -7.246376e-01
      -2.083560e-02
      -8.049481e-01
      -2.452511e-01
      -4.902656e-01
      -7.096359e-01
      -8.017981e-01
      -2.495668e-01
      -5.214705e-01
      -7.876236e-01
      -5.246776e-02
      -7.646644e-01
      -3.572827e-01
      -4.600447e-01
      -6.924700e-01
      -3.825012e+00
      -7.909064e-01
      -4.127246e+01
      -5.217605e-01
      -8.145337e-01
      -2.926304e-02
      -7.923241e-01
      -3.242290e-01
      -5.330336e-01
      -7.645764e-01
      -8.331290e-01
      -1.406588e-01
      -5.261425e-01
      -7.510599e-01
      -8.286613e-01
      -2.259097e-02
      -5.083576e-01
      -7.960591e-01
      -2.926304e-02
      -7.740821e-01
      -2.497243e-01
      -2.996388e+00
      -7.273545e-01
      -3.545066e+00
      -8.264828e-01
      -2.699525e+01
      -3.550828e-01
      -2.644305e-01
      -2.491379e-01
      -3.678365e-01
      -1.814657e+01
      -3.207774e-01
      -2.183901e-01
      -2.065451e-01
      -3.105447e-01
      -1.715956e+01
      -1.683982e+00
      -7.472960e-01
      -4.054644e-02
      -4.616777e-01
      -8.737624e-03
      -1.747725e-02
      -7.399055e-01
      -4.328508e-01
      -5.641186e-01
      -4.929191e-02
      -5.854858e-02
      -1.904510e-01
      -8.737624e-03
      -2.548163e-02
      -6.951747e-02
      -8.737624e-03
      -1.287194e-01
      -3.760707e-02
      -5.366239e-01
      -8.737624e-03
      -8.737624e-03
      -5.788850e-02
      -3.959076e-02
      -1.381619e-02
      -1.747725e-02
      -1.459602e-01
      -1.381619e-02
      -1.084411e-01
      -2.599265e-01
      -1.155912e-02
      -1.747725e-02
      -1.575333e-02
      -6.923973e-02
      -8.737624e-03
      -1.381619e-02
      -3.137620e-01
      -8.737624e-03
      -2.049507e-02
      -1.904594e-02
      -3.328904e-02
      -1.226344e+00
      -2.875665e-01
      -1.764442e-01
      -2.299292e-01
      -5.534404e-02
      -3.578179e-02
      -5.446982e-02
      -1.280282e-01
      -1.155912e-02
      -1.608835e-01
      -1.381619e-02
      -2.049507e-02
      -1.747725e-02
      -4.192489e-01
      -2.683608e-01
      -1.457555e-01
      -4.078188e-02
      -1.114637e-01
      -2.319007e-01
      -4.405033e-01
      -1.985366e-01
      -5.540322e-01
      -2.095589e-02
      -1.155912e-02
      -1.029958e-01
      -2.184844e-02
      -7.900227e-02
      -1.231678e-01
      -1.155912e-02
      -8.737624e-03
      -7.061771e-02
      -8.737624e-03
      -2.433079e-02
      -2.051931e-01
      -1.675033e-01
      -2.227541e-01
      -6.297839e-02
      -1.747696e-01
      -5.818093e-01
      -2.886154e-01
      -2.840114e-01
      -4.967947e-02
      -1.367410e+00
      -8.847841e-02
      -6.096115e-02
      -5.551725e-02
      -7.143200e-02
      -1.575333e-02
      -8.737624e-03
      -4.771050e-02
      -6.053614e-01
      -4.565751e-02
      -2.276196e-01
      -1.829519e-01
      -1.155912e-02
      -8.737624e-03
      -9.265534e-02
      -7.087572e-01
      -3.544788e-01
      -3.497053e-02
      -8.737624e-03
      -1.730808e-01
      -1.669003e-01
      -7.511770e-02
      -3.631261e-02
      -8.737624e-03
      -3.578179e-02
      -1.634816e-02
      -8.737624e-03
      -4.417689e-01
      -2.899049e-02
      -8.737624e-03
      -1.747725e-02
      -1.315255e-01
      -5.482116e-02
      -3.497053e-02
      -6.297839e-02
      -2.049507e-02
      -8.737624e-03
      -3.227445e-01
      -2.103906e-01
      -9.972882e-02
      -2.884602e-01
      -3.412056e-01
      -1.904594e-02
      -1.267906e-01
      -1.496678e-01
      -7.588471e-02
      -1.223669e-01
      -8.737624e-03
      -3.490663e-01
      -1.288726e-01
      -3.866662e-01
      -4.214492e-01
      -1.445905e-01
      -1.311496e-01
      -2.894682e-01
      -8.737624e-03
      -1.747725e-02
      -8.737624e-03
      -6.158877e-02
      -4.521261e-01
      -2.899049e-02
      -4.239363e-02
      -1.027115e-01
      -6.448784e-02
      -2.326309e-01
      -1.104059e-01
      -3.578179e-02
      -6.000061e-01
      -5.101297e-02
      -8.737624e-03
      -9.365650e-01
      -2.228134e-02
      -6.448784e-02
      -2.611378e-01
      -2.433079e-02
      -4.710392e-02
      -1.682245e-01
      -4.771050e-02
      -2.100441e-01
      -3.786068e-02
      -1.155912e-02
      -1.747725e-02
      -8.093601e-02
      -7.472711e-01
      -7.973278e-02
      -3.967785e-01
      -3.959076e-02
      -3.646664e-01
      -6.537706e-02
      -8.014570e-01
      -4.310467e-01
      -2.090516e-01
      -3.735175e-02
      -8.737624e-03
      -7.356025e-02
      -4.929191e-02
      -3.059483e-02
      -4.261894e-02
      -1.674431e-01
      -1.211562e-01
      -8.737624e-03
      -1.155912e-02
      -1.447280e-01
      -1.328428e+00
      -3.751356e-01
      -5.260809e-01
      -8.814917e-02
      -8.737624e-03
      -1.238844e-01
      -8.021618e-02
      -1.381619e-02
      -1.632401e-01
      -1.220451e-01
      -1.575333e-02
      -2.094987e-01
      -4.791101e-02
      -5.871246e-02
      -5.120066e-02
      -9.234053e-02
      -1.747725e-02
      -3.585591e-01
      -3.619064e-01
      -4.002833e-01
      -4.450046e-01
      -4.489690e-01
      -4.391126e-01
      -3.992237e-01
      -4.351795e-01
      -4.488462e-01
      -4.546955e-01
      -4.400708e-01
      -4.273988e-01
      -3.261578e-01
      -3.124023e-01
      -8.285969e-02
      -4.517081e-02
      -3.783608e-02
      -1.794060e-01

      25%
      -4.696404e-01
      -6.669788e-01
      -5.246776e-02
      -7.042022e-01
      -3.546302e-01
      -4.441133e-01
      -6.449386e-01
      -2.083560e-02
      -7.364858e-01
      -2.452511e-01
      -4.632285e-01
      -6.327895e-01
      -7.331256e-01
      -2.495668e-01
      -4.942066e-01
      -7.018973e-01
      -5.246776e-02
      -7.085698e-01
      -3.572827e-01
      -2.776947e-01
      -6.205933e-01
      -5.284048e-01
      -7.313659e-01
      1.401921e-01
      -5.217605e-01
      -8.145337e-01
      -2.926304e-02
      -7.923241e-01
      -3.242290e-01
      -5.330336e-01
      -7.645764e-01
      -8.331290e-01
      -1.406588e-01
      -5.261425e-01
      -7.510599e-01
      -8.286613e-01
      -2.259097e-02
      -5.083576e-01
      -7.960591e-01
      -2.926304e-02
      -7.740821e-01
      -2.497243e-01
      -5.219322e-01
      -7.273545e-01
      -6.777956e-01
      -8.264828e-01
      -1.544734e-02
      -3.550828e-01
      -2.644305e-01
      -2.491379e-01
      -3.678365e-01
      -3.120027e-02
      -3.207774e-01
      -2.183901e-01
      -2.065451e-01
      -3.105447e-01
      -1.004809e-02
      -1.006087e+00
      -7.229131e-01
      -4.054644e-02
      -4.616777e-01
      -8.737624e-03
      -1.747725e-02
      -7.399055e-01
      -4.328508e-01
      -5.641186e-01
      -4.929191e-02
      -5.854858e-02
      -1.904510e-01
      -8.737624e-03
      -2.548163e-02
      -6.951747e-02
      -8.737624e-03
      -1.287194e-01
      -3.760707e-02
      -5.366239e-01
      -8.737624e-03
      -8.737624e-03
      -5.788850e-02
      -3.959076e-02
      -1.381619e-02
      -1.747725e-02
      -1.459602e-01
      -1.381619e-02
      -1.084411e-01
      -2.599265e-01
      -1.155912e-02
      -1.747725e-02
      -1.575333e-02
      -6.923973e-02
      -8.737624e-03
      -1.381619e-02
      -3.137620e-01
      -8.737624e-03
      -2.049507e-02
      -1.904594e-02
      -3.328904e-02
      -1.226344e+00
      -2.875665e-01
      -1.764442e-01
      -2.299292e-01
      -5.534404e-02
      -3.578179e-02
      -5.446982e-02
      -1.280282e-01
      -1.155912e-02
      -1.608835e-01
      -1.381619e-02
      -2.049507e-02
      -1.747725e-02
      -4.192489e-01
      -2.683608e-01
      -1.457555e-01
      -4.078188e-02
      -1.114637e-01
      -2.319007e-01
      -4.405033e-01
      -1.985366e-01
      -5.540322e-01
      -2.095589e-02
      -1.155912e-02
      -1.029958e-01
      -2.184844e-02
      -7.900227e-02
      -1.231678e-01
      -1.155912e-02
      -8.737624e-03
      -7.061771e-02
      -8.737624e-03
      -2.433079e-02
      -2.051931e-01
      -1.675033e-01
      -2.227541e-01
      -6.297839e-02
      -1.747696e-01
      -5.818093e-01
      -2.886154e-01
      -2.840114e-01
      -4.967947e-02
      -1.367410e+00
      -8.847841e-02
      -6.096115e-02
      -5.551725e-02
      -7.143200e-02
      -1.575333e-02
      -8.737624e-03
      -4.771050e-02
      -6.053614e-01
      -4.565751e-02
      -2.276196e-01
      -1.829519e-01
      -1.155912e-02
      -8.737624e-03
      -9.265534e-02
      -7.087572e-01
      -3.544788e-01
      -3.497053e-02
      -8.737624e-03
      -1.730808e-01
      -1.669003e-01
      -7.511770e-02
      -3.631261e-02
      -8.737624e-03
      -3.578179e-02
      -1.634816e-02
      -8.737624e-03
      -4.417689e-01
      -2.899049e-02
      -8.737624e-03
      -1.747725e-02
      -1.315255e-01
      -5.482116e-02
      -3.497053e-02
      -6.297839e-02
      -2.049507e-02
      -8.737624e-03
      -3.227445e-01
      -2.103906e-01
      -9.972882e-02
      -2.884602e-01
      -3.412056e-01
      -1.904594e-02
      -1.267906e-01
      -1.496678e-01
      -7.588471e-02
      -1.223669e-01
      -8.737624e-03
      -3.490663e-01
      -1.288726e-01
      -3.866662e-01
      -4.214492e-01
      -1.445905e-01
      -1.311496e-01
      -2.894682e-01
      -8.737624e-03
      -1.747725e-02
      -8.737624e-03
      -6.158877e-02
      -4.521261e-01
      -2.899049e-02
      -4.239363e-02
      -1.027115e-01
      -6.448784e-02
      -2.326309e-01
      -1.104059e-01
      -3.578179e-02
      -6.000061e-01
      -5.101297e-02
      -8.737624e-03
      -9.365650e-01
      -2.228134e-02
      -6.448784e-02
      -2.611378e-01
      -2.433079e-02
      -4.710392e-02
      -1.682245e-01
      -4.771050e-02
      -2.100441e-01
      -3.786068e-02
      -1.155912e-02
      -1.747725e-02
      -8.093601e-02
      -7.472711e-01
      -7.973278e-02
      -3.967785e-01
      -3.959076e-02
      -3.646664e-01
      -6.537706e-02
      -8.014570e-01
      -4.310467e-01
      -2.090516e-01
      -3.735175e-02
      -8.737624e-03
      -7.356025e-02
      -4.929191e-02
      -3.059483e-02
      -4.261894e-02
      -1.674431e-01
      -1.211562e-01
      -8.737624e-03
      -1.155912e-02
      -1.447280e-01
      -1.328428e+00
      -3.751356e-01
      -5.260809e-01
      -8.814917e-02
      -8.737624e-03
      -1.238844e-01
      -8.021618e-02
      -1.381619e-02
      -1.632401e-01
      -1.220451e-01
      -1.575333e-02
      -2.094987e-01
      -4.791101e-02
      -5.871246e-02
      -5.120066e-02
      -9.234053e-02
      -1.747725e-02
      -3.585591e-01
      -3.619064e-01
      -4.002833e-01
      -4.450046e-01
      -4.489690e-01
      -4.391126e-01
      -3.992237e-01
      -4.351795e-01
      -4.488462e-01
      -4.546955e-01
      -4.400708e-01
      -4.273988e-01
      -3.261578e-01
      -3.124023e-01
      -8.285969e-02
      -4.517081e-02
      -3.783608e-02
      -1.794060e-01

      50%
      -3.641699e-01
      -4.402479e-01
      -5.246776e-02
      -4.029059e-01
      -2.971575e-01
      -3.430581e-01
      -4.382153e-01
      -2.083560e-02
      -4.284057e-01
      -2.031779e-01
      -3.570950e-01
      -4.319623e-01
      -4.240996e-01
      -2.495668e-01
      -3.753796e-01
      -4.833774e-01
      -5.246776e-02
      -4.227545e-01
      -3.084090e-01
      -2.359626e-01
      -4.374431e-01
      4.453395e-01
      -4.273960e-01
      1.401921e-01
      -4.086387e-01
      -4.017941e-01
      -2.926304e-02
      -5.061590e-01
      -3.242290e-01
      -4.129555e-01
      -3.742671e-01
      -4.533669e-01
      -1.406588e-01
      -4.160729e-01
      -3.887390e-01
      -4.496266e-01
      -2.259097e-02
      -4.053517e-01
      -3.888882e-01
      -2.926304e-02
      -4.665341e-01
      -2.497243e-01
      -5.219322e-01
      -3.797885e-01
      4.250008e-01
      -4.404116e-01
      -1.544734e-02
      -3.280763e-01
      -2.644305e-01
      -2.491379e-01
      -3.467561e-01
      -1.676571e-02
      -3.207774e-01
      -2.183901e-01
      -2.065451e-01
      -3.105447e-01
      -1.004809e-02
      1.075490e-02
      -2.598997e-01
      -4.054644e-02
      -4.616777e-01
      -8.737624e-03
      -1.747725e-02
      -7.399055e-01
      -4.328508e-01
      -5.641186e-01
      -4.929191e-02
      -5.854858e-02
      -1.904510e-01
      -8.737624e-03
      -2.548163e-02
      -6.951747e-02
      -8.737624e-03
      -1.287194e-01
      -3.760707e-02
      -5.366239e-01
      -8.737624e-03
      -8.737624e-03
      -5.788850e-02
      -3.959076e-02
      -1.381619e-02
      -1.747725e-02
      -1.459602e-01
      -1.381619e-02
      -1.084411e-01
      -2.599265e-01
      -1.155912e-02
      -1.747725e-02
      -1.575333e-02
      -6.923973e-02
      -8.737624e-03
      -1.381619e-02
      -3.137620e-01
      -8.737624e-03
      -2.049507e-02
      -1.904594e-02
      -3.328904e-02
      8.154322e-01
      -2.875665e-01
      -1.764442e-01
      -2.299292e-01
      -5.534404e-02
      -3.578179e-02
      -5.446982e-02
      -1.280282e-01
      -1.155912e-02
      -1.608835e-01
      -1.381619e-02
      -2.049507e-02
      -1.747725e-02
      -4.192489e-01
      -2.683608e-01
      -1.457555e-01
      -4.078188e-02
      -1.114637e-01
      -2.319007e-01
      -4.405033e-01
      -1.985366e-01
      -5.540322e-01
      -2.095589e-02
      -1.155912e-02
      -1.029958e-01
      -2.184844e-02
      -7.900227e-02
      -1.231678e-01
      -1.155912e-02
      -8.737624e-03
      -7.061771e-02
      -8.737624e-03
      -2.433079e-02
      -2.051931e-01
      -1.675033e-01
      -2.227541e-01
      -6.297839e-02
      -1.747696e-01
      -5.818093e-01
      -2.886154e-01
      -2.840114e-01
      -4.967947e-02
      7.313095e-01
      -8.847841e-02
      -6.096115e-02
      -5.551725e-02
      -7.143200e-02
      -1.575333e-02
      -8.737624e-03
      -4.771050e-02
      -6.053614e-01
      -4.565751e-02
      -2.276196e-01
      -1.829519e-01
      -1.155912e-02
      -8.737624e-03
      -9.265534e-02
      -7.087572e-01
      -3.544788e-01
      -3.497053e-02
      -8.737624e-03
      -1.730808e-01
      -1.669003e-01
      -7.511770e-02
      -3.631261e-02
      -8.737624e-03
      -3.578179e-02
      -1.634816e-02
      -8.737624e-03
      -4.417689e-01
      -2.899049e-02
      -8.737624e-03
      -1.747725e-02
      -1.315255e-01
      -5.482116e-02
      -3.497053e-02
      -6.297839e-02
      -2.049507e-02
      -8.737624e-03
      -3.227445e-01
      -2.103906e-01
      -9.972882e-02
      -2.884602e-01
      -3.412056e-01
      -1.904594e-02
      -1.267906e-01
      -1.496678e-01
      -7.588471e-02
      -1.223669e-01
      -8.737624e-03
      -3.490663e-01
      -1.288726e-01
      -3.866662e-01
      -4.214492e-01
      -1.445905e-01
      -1.311496e-01
      -2.894682e-01
      -8.737624e-03
      -1.747725e-02
      -8.737624e-03
      -6.158877e-02
      -4.521261e-01
      -2.899049e-02
      -4.239363e-02
      -1.027115e-01
      -6.448784e-02
      -2.326309e-01
      -1.104059e-01
      -3.578179e-02
      -6.000061e-01
      -5.101297e-02
      -8.737624e-03
      -9.365650e-01
      -2.228134e-02
      -6.448784e-02
      -2.611378e-01
      -2.433079e-02
      -4.710392e-02
      -1.682245e-01
      -4.771050e-02
      -2.100441e-01
      -3.786068e-02
      -1.155912e-02
      -1.747725e-02
      -8.093601e-02
      -7.472711e-01
      -7.973278e-02
      -3.967785e-01
      -3.959076e-02
      -3.646664e-01
      -6.537706e-02
      -8.014570e-01
      -4.310467e-01
      -2.090516e-01
      -3.735175e-02
      -8.737624e-03
      -7.356025e-02
      -4.929191e-02
      -3.059483e-02
      -4.261894e-02
      -1.674431e-01
      -1.211562e-01
      -8.737624e-03
      -1.155912e-02
      -1.447280e-01
      7.527693e-01
      -3.751356e-01
      -5.260809e-01
      -8.814917e-02
      -8.737624e-03
      -1.238844e-01
      -8.021618e-02
      -1.381619e-02
      -1.632401e-01
      -1.220451e-01
      -1.575333e-02
      -2.094987e-01
      -4.791101e-02
      -5.871246e-02
      -5.120066e-02
      -9.234053e-02
      -1.747725e-02
      -3.585591e-01
      -3.619064e-01
      -4.002833e-01
      -4.450046e-01
      -4.489690e-01
      -4.391126e-01
      -3.992237e-01
      -4.351795e-01
      -4.488462e-01
      -4.546955e-01
      -4.400708e-01
      -4.273988e-01
      -3.261578e-01
      -3.124023e-01
      -8.285969e-02
      -4.517081e-02
      -3.783608e-02
      -1.794060e-01

      75%
      1.087578e-02
      4.912830e-01
      -5.246776e-02
      3.129560e-01
      -1.391077e-01
      2.430874e-02
      4.598334e-01
      -2.083560e-02
      3.993651e-01
      -1.611047e-01
      2.049629e-02
      4.451662e-01
      4.062127e-01
      -1.809983e-01
      1.393228e-02
      5.431944e-01
      -5.246776e-02
      3.598988e-01
      -1.617881e-01
      -5.424211e-02
      4.078520e-01
      7.761153e-01
      4.218396e-01
      1.401921e-01
      -3.001386e-03
      3.590039e-01
      -2.926304e-02
      6.385015e-01
      -3.242290e-01
      3.395931e-02
      3.289212e-01
      6.859193e-01
      -1.406588e-01
      4.080070e-02
      3.081313e-01
      6.874773e-01
      -2.259097e-02
      -2.366779e-02
      3.357360e-01
      -2.926304e-02
      6.098837e-01
      -2.497243e-01
      1.143563e-01
      2.724983e-01
      8.661194e-01
      7.178020e-01
      -1.544734e-02
      -1.300287e-01
      -1.949104e-01
      -1.959530e-01
      -1.464925e-01
      -2.331149e-03
      -3.207774e-01
      -2.183901e-01
      -2.065451e-01
      -3.105447e-01
      -1.004809e-02
      9.791761e-01
      2.906825e-01
      -4.054644e-02
      -4.616777e-01
      -8.737624e-03
      -1.747725e-02
      1.351524e+00
      -4.328508e-01
      -5.641186e-01
      -4.929191e-02
      -5.854858e-02
      -1.904510e-01
      -8.737624e-03
      -2.548163e-02
      -6.951747e-02
      -8.737624e-03
      -1.287194e-01
      -3.760707e-02
      -5.366239e-01
      -8.737624e-03
      -8.737624e-03
      -5.788850e-02
      -3.959076e-02
      -1.381619e-02
      -1.747725e-02
      -1.459602e-01
      -1.381619e-02
      -1.084411e-01
      -2.599265e-01
      -1.155912e-02
      -1.747725e-02
      -1.575333e-02
      -6.923973e-02
      -8.737624e-03
      -1.381619e-02
      -3.137620e-01
      -8.737624e-03
      -2.049507e-02
      -1.904594e-02
      -3.328904e-02
      8.154322e-01
      -2.875665e-01
      -1.764442e-01
      -2.299292e-01
      -5.534404e-02
      -3.578179e-02
      -5.446982e-02
      -1.280282e-01
      -1.155912e-02
      -1.608835e-01
      -1.381619e-02
      -2.049507e-02
      -1.747725e-02
      -4.192489e-01
      -2.683608e-01
      -1.457555e-01
      -4.078188e-02
      -1.114637e-01
      -2.319007e-01
      -4.405033e-01
      -1.985366e-01
      -5.540322e-01
      -2.095589e-02
      -1.155912e-02
      -1.029958e-01
      -2.184844e-02
      -7.900227e-02
      -1.231678e-01
      -1.155912e-02
      -8.737624e-03
      -7.061771e-02
      -8.737624e-03
      -2.433079e-02
      -2.051931e-01
      -1.675033e-01
      -2.227541e-01
      -6.297839e-02
      -1.747696e-01
      1.718776e+00
      -2.886154e-01
      -2.840114e-01
      -4.967947e-02
      7.313095e-01
      -8.847841e-02
      -6.096115e-02
      -5.551725e-02
      -7.143200e-02
      -1.575333e-02
      -8.737624e-03
      -4.771050e-02
      1.651906e+00
      -4.565751e-02
      -2.276196e-01
      -1.829519e-01
      -1.155912e-02
      -8.737624e-03
      -9.265534e-02
      1.410920e+00
      -3.544788e-01
      -3.497053e-02
      -8.737624e-03
      -1.730808e-01
      -1.669003e-01
      -7.511770e-02
      -3.631261e-02
      -8.737624e-03
      -3.578179e-02
      -1.634816e-02
      -8.737624e-03
      -4.417689e-01
      -2.899049e-02
      -8.737624e-03
      -1.747725e-02
      -1.315255e-01
      -5.482116e-02
      -3.497053e-02
      -6.297839e-02
      -2.049507e-02
      -8.737624e-03
      -3.227445e-01
      -2.103906e-01
      -9.972882e-02
      -2.884602e-01
      -3.412056e-01
      -1.904594e-02
      -1.267906e-01
      -1.496678e-01
      -7.588471e-02
      -1.223669e-01
      -8.737624e-03
      -3.490663e-01
      -1.288726e-01
      -3.866662e-01
      -4.214492e-01
      -1.445905e-01
      -1.311496e-01
      -2.894682e-01
      -8.737624e-03
      -1.747725e-02
      -8.737624e-03
      -6.158877e-02
      -4.521261e-01
      -2.899049e-02
      -4.239363e-02
      -1.027115e-01
      -6.448784e-02
      -2.326309e-01
      -1.104059e-01
      -3.578179e-02
      1.666650e+00
      -5.101297e-02
      -8.737624e-03
      1.067732e+00
      -2.228134e-02
      -6.448784e-02
      -2.611378e-01
      -2.433079e-02
      -4.710392e-02
      -1.682245e-01
      -4.771050e-02
      -2.100441e-01
      -3.786068e-02
      -1.155912e-02
      -1.747725e-02
      -8.093601e-02
      1.338202e+00
      -7.973278e-02
      -3.967785e-01
      -3.959076e-02
      -3.646664e-01
      -6.537706e-02
      1.247728e+00
      -4.310467e-01
      -2.090516e-01
      -3.735175e-02
      -8.737624e-03
      -7.356025e-02
      -4.929191e-02
      -3.059483e-02
      -4.261894e-02
      -1.674431e-01
      -1.211562e-01
      -8.737624e-03
      -1.155912e-02
      -1.447280e-01
      7.527693e-01
      -3.751356e-01
      -5.260809e-01
      -8.814917e-02
      -8.737624e-03
      -1.238844e-01
      -8.021618e-02
      -1.381619e-02
      -1.632401e-01
      -1.220451e-01
      -1.575333e-02
      -2.094987e-01
      -4.791101e-02
      -5.871246e-02
      -5.120066e-02
      -9.234053e-02
      -1.747725e-02
      -3.585591e-01
      -3.619064e-01
      -4.002833e-01
      -4.450046e-01
      -4.489690e-01
      -4.391126e-01
      -3.992237e-01
      -4.351795e-01
      -4.488462e-01
      -4.546955e-01
      -4.400708e-01
      -4.273988e-01
      -3.261578e-01
      -3.124023e-01
      -8.285969e-02
      -4.517081e-02
      -3.783608e-02
      -1.528852e-01

      max
      1.369070e+01
      8.220507e+00
      1.060778e+02
      3.860550e+00
      1.852514e+01
      1.325396e+01
      8.593561e+00
      1.061189e+02
      3.626426e+00
      2.449380e+01
      1.479007e+01
      8.471447e+00
      3.643182e+00
      4.006867e+01
      1.594517e+01
      1.021391e+01
      1.060778e+02
      4.396039e+00
      2.065838e+01
      7.430508e+00
      8.808215e+00
      8.430137e-01
      3.665230e+00
      5.745770e+01
      8.795458e+00
      4.620362e+00
      1.066348e+02
      3.500153e+00
      1.245092e+01
      8.240364e+00
      6.726747e+00
      3.724016e+00
      7.163699e+00
      8.247979e+00
      6.721882e+00
      3.719754e+00
      5.634403e+01
      8.933224e+00
      4.640826e+00
      1.066348e+02
      3.224041e+00
      1.579094e+01
      2.917154e+01
      7.521431e+00
      8.661194e-01
      3.613336e+00
      5.394416e+01
      1.804534e+01
      2.448475e+01
      2.499597e+01
      1.999581e+01
      2.050918e+01
      1.887421e+01
      2.713537e+01
      2.734876e+01
      1.735015e+01
      1.971189e+01
      1.802334e+00
      1.443147e+01
      2.466308e+01
      2.166013e+00
      1.144476e+02
      5.721724e+01
      1.351524e+00
      2.310265e+00
      1.772677e+00
      2.028731e+01
      1.707983e+01
      5.250695e+00
      1.144476e+02
      3.924396e+01
      1.438487e+01
      1.144476e+02
      7.768835e+00
      2.659074e+01
      1.863503e+00
      1.144476e+02
      1.144476e+02
      1.727459e+01
      2.525842e+01
      7.237886e+01
      5.721724e+01
      6.851182e+00
      7.237886e+01
      9.221592e+00
      3.847241e+00
      8.651177e+01
      5.721724e+01
      6.347865e+01
      1.444258e+01
      1.144476e+02
      7.237886e+01
      3.187129e+00
      1.144476e+02
      4.879223e+01
      5.250464e+01
      3.003992e+01
      8.154322e-01
      3.477456e+00
      5.667516e+00
      4.349166e+00
      1.806879e+01
      2.794718e+01
      1.835879e+01
      7.810780e+00
      8.651177e+01
      6.215677e+00
      7.237886e+01
      4.879223e+01
      5.721724e+01
      2.385218e+00
      3.726327e+00
      6.860805e+00
      2.452069e+01
      8.971529e+00
      4.312191e+00
      2.270131e+00
      5.036855e+00
      1.804949e+00
      4.771929e+01
      8.651177e+01
      9.709133e+00
      4.576986e+01
      1.265786e+01
      8.119006e+00
      8.651177e+01
      1.144476e+02
      1.416075e+01
      1.144476e+02
      4.110019e+01
      4.873458e+00
      5.970030e+00
      4.489255e+00
      1.587846e+01
      5.721819e+00
      1.718776e+00
      3.464818e+00
      3.520986e+00
      2.012904e+01
      7.313095e-01
      1.130219e+01
      1.640389e+01
      1.801242e+01
      1.399933e+01
      6.347865e+01
      1.144476e+02
      2.095975e+01
      1.651906e+00
      2.190220e+01
      4.393296e+00
      5.465917e+00
      8.651177e+01
      1.144476e+02
      1.079269e+01
      1.410920e+00
      2.821043e+00
      2.859551e+01
      1.144476e+02
      5.777649e+00
      5.991602e+00
      1.331244e+01
      2.753864e+01
      1.144476e+02
      2.794718e+01
      6.116897e+01
      1.144476e+02
      2.263627e+00
      3.449407e+01
      1.144476e+02
      5.721724e+01
      7.603089e+00
      1.824113e+01
      2.859551e+01
      1.587846e+01
      4.879223e+01
      1.144476e+02
      3.098426e+00
      4.753065e+00
      1.002719e+01
      3.466683e+00
      2.930784e+00
      5.250464e+01
      7.887019e+00
      6.681466e+00
      1.317789e+01
      8.172145e+00
      1.144476e+02
      2.864785e+00
      7.759604e+00
      2.586210e+00
      2.372765e+00
      6.916086e+00
      7.624882e+00
      3.454611e+00
      1.144476e+02
      5.721724e+01
      1.144476e+02
      1.623673e+01
      2.211772e+00
      3.449407e+01
      2.358845e+01
      9.736003e+00
      1.550680e+01
      4.298655e+00
      9.057485e+00
      2.794718e+01
      1.666650e+00
      1.960286e+01
      1.144476e+02
      1.067732e+00
      4.488061e+01
      1.550680e+01
      3.829396e+00
      4.110019e+01
      2.122966e+01
      5.944438e+00
      2.095975e+01
      4.760905e+00
      2.641262e+01
      8.651177e+01
      5.721724e+01
      1.235544e+01
      1.338202e+00
      1.254189e+01
      2.520298e+00
      2.525842e+01
      2.742232e+00
      1.529589e+01
      1.247728e+00
      2.319934e+00
      4.783508e+00
      2.677251e+01
      1.144476e+02
      1.359430e+01
      2.028731e+01
      3.268526e+01
      2.346375e+01
      5.972177e+00
      8.253807e+00
      1.144476e+02
      8.651177e+01
      6.909515e+00
      7.527693e-01
      2.665703e+00
      1.900848e+00
      1.134441e+01
      1.144476e+02
      8.072040e+00
      1.246631e+01
      7.237886e+01
      6.125945e+00
      8.193689e+00
      6.347865e+01
      4.773299e+00
      2.087203e+01
      1.703216e+01
      1.953100e+01
      1.082948e+01
      5.721724e+01
      2.788940e+00
      2.763145e+00
      2.498230e+00
      2.247168e+00
      2.227325e+00
      2.277320e+00
      2.504861e+00
      2.297902e+00
      2.227935e+00
      2.199274e+00
      2.272362e+00
      2.339735e+00
      3.066000e+00
      3.201001e+00
      9.448112e+01
      6.461474e+01
      8.738533e+01
      3.758626e+01

### Defining Variables and Splitting Data

```
# Making independent and dependent variables
X = df.drop(columns=[280])
y = df[280]
```

```
# Splitting data
X_train, X_test, Y_train, Y_test = train_test_split(X, y, random_state=42)
```

### Linear Regression

```
# Linear regression instance and fitting model to train data
lin_reg = LinearRegression().fit(X_train, Y_train)
```

```
# Printing MSE for train and test datasets -- this model appears to be overfit bc train did better than test
print('Train MSE:', (mean_squared_error(Y_train, lin_reg.predict(X_train))))
print('Test MSE:', (mean_squared_error(Y_test, lin_reg.predict(X_test))))
```

    Train RMSE: 0.6128522709278815
    Test RMSE: 0.7115032395058142

```
# The score is better for the train data as well
print('Train Score:', lin_reg.score(X_train, Y_train))
print('Test Score:', lin_reg.score(X_test, Y_test))
```

    Train Score: 0.3756861890770966
    Test Score: 0.32562913832820084

### Ridge Regression

```
# Fitting model to training data
ridge_reg = Ridge(alpha=0).fit(X_train, Y_train)

# Looking at mean squared error of training data
print('Train MSE:', (mean_squared_error(Y_train, ridge_reg.predict(X_train))))
print('Test MSE:', (mean_squared_error(Y_test, ridge_reg.predict(X_test))))

# The result is essentially the same as the linear regression mse bc alpha is set to 0
```

    Train MSE: 0.6136220506010315
    Test MSE: 0.7125536615957923

```
alphas = []
mses = []

for alpha in np.arange(400, 600, 1):
  ridge_reg_split = Ridge(alpha=alpha).fit(X_train, Y_train)
  mse = mean_squared_error(Y_test, ridge_reg_split.predict(X_test))
  print(alpha, mse)
  alphas.append(alpha)
  mses.append(mse)
```

    400 0.7100100783232121
    401 0.7100098700513476
    402 0.7100096634043153
    403 0.7100094583783956
    404 0.710009254969877
    405 0.710009053175058
    406 0.7100088529902454
    407 0.7100086544117551
    408 0.710008457435912
    409 0.7100082620590499
    410 0.7100080682775106
    411 0.7100078760876463
    412 0.7100076854858163
    413 0.7100074964683897
    414 0.7100073090317437
    415 0.7100071231722643
    416 0.7100069388863466
    417 0.7100067561703934
    418 0.7100065750208165
    419 0.7100063954340364
    420 0.710006217406482
    421 0.7100060409345906
    422 0.7100058660148078
    423 0.7100056926435878
    424 0.710005520817393
    425 0.7100053505326948
    426 0.7100051817859719
    427 0.7100050145737126
    428 0.7100048488924121
    429 0.7100046847385746
    430 0.710004522108713
    431 0.7100043609993478
    432 0.7100042014070072
    433 0.7100040433282288
    434 0.7100038867595577
    435 0.7100037316975467
    436 0.7100035781387577
    437 0.7100034260797601
    438 0.7100032755171316
    439 0.7100031264474574
    440 0.7100029788673313
    441 0.7100028327733555
    442 0.7100026881621387
    443 0.710002545030299
    444 0.7100024033744624
    445 0.7100022631912618
    446 0.710002124477339
    447 0.7100019872293434
    448 0.710001851443932
    449 0.7100017171177699
    450 0.7100015842475305
    451 0.710001452829894
    452 0.7100013228615495
    453 0.7100011943391934
    454 0.7100010672595294
    455 0.7100009416192702
    456 0.710000817415135
    457 0.7100006946438514
    458 0.7100005733021545
    459 0.7100004533867872
    460 0.7100003348945004
    461 0.7100002178220521
    462 0.7100001021662083
    463 0.7099999879237426
    464 0.7099998750914361
    465 0.7099997636660774
    466 0.7099996536444634
    467 0.7099995450233979
    468 0.7099994377996924
    469 0.7099993319701663
    470 0.7099992275316457
    471 0.7099991244809652
    472 0.7099990228149667
    473 0.709998922530499
    474 0.7099988236244192
    475 0.7099987260935914
    476 0.7099986299348873
    477 0.7099985351451855
    478 0.7099984417213734
    479 0.7099983496603437
    480 0.7099982589589993
    481 0.7099981696142479
    482 0.7099980816230059
    483 0.7099979949821967
    484 0.7099979096887515
    485 0.7099978257396082
    486 0.7099977431317124
    487 0.7099976618620168
    488 0.7099975819274816
    489 0.7099975033250747
    490 0.70999742605177
    491 0.7099973501045498
    492 0.7099972754804035
    493 0.7099972021763273
    494 0.709997130189325
    495 0.7099970595164071
    496 0.7099969901545925
    497 0.7099969221009057
    498 0.7099968553523794
    499 0.7099967899060532
    500 0.7099967257589733
    501 0.7099966629081949
    502 0.7099966013507776
    503 0.7099965410837902
    504 0.7099964821043074
    505 0.7099964244094121
    506 0.7099963679961931
    507 0.7099963128617471
    508 0.7099962590031774
    509 0.7099962064175944
    510 0.7099961551021157
    511 0.7099961050538659
    512 0.7099960562699762
    513 0.709996008747585
    514 0.7099959624838381
    515 0.7099959174758876
    516 0.709995873720893
    517 0.7099958312160204
    518 0.7099957899584431
    519 0.7099957499453408
    520 0.7099957111739009
    521 0.709995673641317
    522 0.7099956373447898
    523 0.7099956022815267
    524 0.7099955684487425
    525 0.7099955358436579
    526 0.7099955044635016
    527 0.7099954743055076
    528 0.709995445366918
    529 0.7099954176449813
    530 0.7099953911369521
    531 0.7099953658400928
    532 0.7099953417516719
    533 0.7099953188689648
    534 0.7099952971892532
    535 0.7099952767098263
    536 0.7099952574279793
    537 0.7099952393410144
    538 0.7099952224462402
    539 0.7099952067409725
    540 0.7099951922225329
    541 0.7099951788882501
    542 0.70999516673546
    543 0.7099951557615036
    544 0.7099951459637298
    545 0.7099951373394935
    546 0.7099951298861563
    547 0.7099951236010865
    548 0.7099951184816586
    549 0.7099951145252537
    550 0.7099951117292592
    551 0.7099951100910694
    552 0.7099951096080855
    553 0.7099951102777136
    554 0.709995112097368
    555 0.7099951150644683
    556 0.7099951191764408
    557 0.7099951244307189
    558 0.7099951308247407
    559 0.7099951383559526
    560 0.7099951470218067
    561 0.7099951568197606
    562 0.7099951677472794
    563 0.7099951798018339
    564 0.7099951929809014
    565 0.7099952072819659
    566 0.7099952227025166
    567 0.7099952392400503
    568 0.7099952568920688
    569 0.7099952756560814
    570 0.7099952955296028
    571 0.7099953165101538
    572 0.7099953385952623
    573 0.7099953617824617
    574 0.7099953860692915
    575 0.7099954114532976
    576 0.7099954379320323
    577 0.709995465503054
    578 0.7099954941639265
    579 0.7099955239122204
    580 0.7099955547455127
    581 0.7099955866613857
    582 0.7099956196574282
    583 0.7099956537312351
    584 0.709995688880407
    585 0.709995725102551
    586 0.7099957623952801
    587 0.709995800756213
    588 0.7099958401829747
    589 0.7099958806731961
    590 0.7099959222245144
    591 0.7099959648345717
    592 0.7099960085010175
    593 0.7099960532215063
    594 0.7099960989936986
    595 0.7099961458152609
    596 0.7099961936838661
    597 0.7099962425971917
    598 0.709996292552923
    599 0.709996343548749

```
# from matplotlib.pyplot import scatter
# scatter(alphas, mses);
```

```
# Fitting model to training data
ridge_reg = Ridge(alpha=552).fit(X_train, Y_train) # Putting in alpha that minimized error the most

# Looking at mean squared error of training data
print('Train MSE:', (mean_squared_error(Y_train, ridge_reg.predict(X_train))))
print('Test MSE:', (mean_squared_error(Y_test, ridge_reg.predict(X_test))))

# It still overfits the data
```

    Train MSE: 0.6139200153365536
    Test MSE: 0.7099951096080855

```
from sklearn.linear_model import RidgeCV
alpha_range = np.arange(1, 1000, 1)

ridgecv = RidgeCV(alphas=alpha_range).fit(X_train, Y_train)
#ridgecv.score(X, y)
```

```
print('Train MSE:', (mean_squared_error(Y_train, ridgecv.predict(X_train))))
print('Test MSE:', (mean_squared_error(Y_test, ridgecv.predict(X_test))))

# Interesting. RidgeCV didn't find a better alpha than I found above
```

    Train MSE: 0.6142879568808577
    Test MSE: 0.7100805737027849

```

```

# Resources and stretch goals

Resources:
- https://www.quora.com/What-is-regularization-in-machine-learning
- https://blogs.sas.com/content/subconsciousmusings/2017/07/06/how-to-use-regularization-to-prevent-model-overfitting/
- https://machinelearningmastery.com/introduction-to-regularization-to-reduce-overfitting-and-improve-generalization-error/
- https://towardsdatascience.com/ridge-and-lasso-regression-a-complete-guide-with-python-scikit-learn-e20e34bcbf0b
- https://stats.stackexchange.com/questions/111017/question-about-standardizing-in-ridge-regression#111022

Stretch goals:
- Revisit past data you've fit OLS models to, and see if there's an `alpha` such that ridge regression results in a model with lower MSE on a train/test split
- Yes, Ridge can be applied to classification! Check out [sklearn.linear_model.RidgeClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.RidgeClassifier.html#sklearn.linear_model.RidgeClassifier), and try it on a problem you previous approached with a different classifier (note - scikit LogisticRegression also automatically penalizes based on the $L^2$ norm, so the difference won't be as dramatic)
- Implement your own function to calculate the full cost that ridge regression is optimizing (the sum of squared residuals + `alpha` times the sum of squared coefficients) - this alone won't fit a model, but you can use it to verify cost of trained models and that the coefficients from the equivalent OLS (without regularization) may have a higher cost