# Regression Week 2:

# Exercise 1-1
# Regression Week 2: Multiple Regression (Interpretation)

The goal of this first notebook is to explore multiple regression and feature engineering with existing sklearn Create functions.

In this notebook you will use data on house sales in King County to predict prices using multiple regression. You will:
* Use sklearn to do some feature engineering
* Use built-in sklearn Create functions to compute the regression weights (coefficients/parameters)
* Given the regression weights, predictors and outcome write a function to compute the Residual Sum of Squares
* Look at coefficients and interpret their meanings
* Evaluate multiple models via RSS

# Load libraries

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import math

from sklearn.model_selection import train_test_split
from sklearn import linear_model
```

# Load in house sales data

Dataset is from house sales in King County, the region where the city of Seattle, WA is located.

```python
dtype_dict = {'bathrooms':float, 'waterfront':int, 'sqft_above':int, 'sqft_living15':float, 'grade':int, 'yr_renovated':int, 'price':float, 'bedrooms':float, 'zipcode':str, 'long':float, 'sqft_lot15':float, 'sqft_living':float, 'floors':str, 'condition':int, 'lat':float, 'date':str, 'sqft_basement':int, 'yr_built':int, 'id':str, 'sqft_lot':int, 'view':int}

sales = pd.read_csv("data/week2_kc_house_data.csv")
sales.head()
```

      id
      date
      price
      bedrooms
      bathrooms
      sqft_living
      sqft_lot
      floors
      waterfront
      view
      ...
      grade
      sqft_above
      sqft_basement
      yr_built
      yr_renovated
      zipcode
      lat
      long
      sqft_living15
      sqft_lot15

      0
      7129300520
      20141013T000000
      221900.0
      3
      1.00
      1180
      5650
      1.0
      0
      0
      ...
      7
      1180
      0
      1955
      0
      98178
      47.5112
      -122.257
      1340
      5650

      1
      6414100192
      20141209T000000
      538000.0
      3
      2.25
      2570
      7242
      2.0
      0
      0
      ...
      7
      2170
      400
      1951
      1991
      98125
      47.7210
      -122.319
      1690
      7639

      2
      5631500400
      20150225T000000
      180000.0
      2
      1.00
      770
      10000
      1.0
      0
      0
      ...
      6
      770
      0
      1933
      0
      98028
      47.7379
      -122.233
      2720
      8062

      3
      2487200875
      20141209T000000
      604000.0
      4
      3.00
      1960
      5000
      1.0
      0
      0
      ...
      7
      1050
      910
      1965
      0
      98136
      47.5208
      -122.393
      1360
      5000

      4
      1954400510
      20150218T000000
      510000.0
      3
      2.00
      1680
      8080
      1.0
      0
      0
      ...
      8
      1680
      0
      1987
      0
      98074
      47.6168
      -122.045
      1800
      7503

5 rows × 21 columns

```python
print('len Sales: %d'%(len(sales)))
```

    len Sales: 21613

# Split data into training and testing.
* Split

        80% train data and

        20% test data.

```python
train_data,test_data = train_test_split(sales,test_size=0.2)
print('len Train: %d\nlen Test: %d'%(len(train_data),len(test_data)))
```

    len Train: 17290
    len Test: 4323

# Learning a multiple regression model
Recall we can use the following code to learn a multiple regression model predicting 'price' based on the following features:
example_features = ['sqft_living', 'bedrooms', 'bathrooms'] on training data with the following code:

(Aside: We set validation_set = None to ensure that the results are always the same)

```python
example_features = ['sqft_living', 'bedrooms', 'bathrooms']

example_model = linear_model.LinearRegression()
example_model.fit(train_data[example_features],train_data['price'])
```

    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)

Now that we have fitted the model we can extract the regression weights (coefficients) as an sklearn as follows:

```python
example_weight_summary = example_model.coef_
print(example_weight_summary)
```

    [   312.28759548 -63374.36308577   6893.18662492]

# Making Predictions

In the gradient descent notebook we use numpy to do our regression. In this book we will use existing sklearn Create functions to analyze multiple regressions.

Recall that once a model is built we can use the .predict() function to find the predicted values for data we pass. For example using the example model above:

```python
example_predictions = example_model.predict(train_data[example_features])

print(example_predictions[0]) # should be 271789.505878
```

    1038520.9319986172

# Compute RSS

Now that we can make predictions given the model, let's write a function to compute the RSS of the model. Complete the function below to calculate RSS given the model, data, and the outcome.

```python
def get_residual_sum_of_squares(model, data, m_features, outcome):
    # First get the predictions
    predictions = model.predict(data[m_features])

    # Then compute the residuals/errors
    residuals_errors = outcome - predictions

    # Then square and add them up
    residuals_errors_square = residuals_errors*residuals_errors
    RSS = residuals_errors_square.sum()

    return(RSS)
```

Test your function by computing the RSS on TEST data for the example model:

```python
rss_example_train = get_residual_sum_of_squares(example_model, test_data, example_features, test_data['price'])
print(rss_example_train) # should be 2.7376153833e+14
```

    303947419523376.2

# Create some new features

Although we often think of multiple regression as including multiple different features (e.g. # of bedrooms, squarefeet, and # of bathrooms) but we can also consider transformations of existing features e.g. the log of the squarefeet or even "interaction" features such as the product of bedrooms and bathrooms.

You will use the logarithm function to create a new feature. so first you should import it from the math library.

```python
from math import log
```

Next create the following 4 new features as column in both TEST and TRAIN data:
* bedrooms_squared = bedrooms\*bedrooms
* bed_bath_rooms = bedrooms\*bathrooms
* log_sqft_living = log(sqft_living)
* lat_plus_long = lat + long
As an example here's the first one:

```python
train_data['bedrooms_squared'] = train_data['bedrooms'].apply(lambda x: x**2)
test_data['bedrooms_squared'] = test_data['bedrooms'].apply(lambda x: x**2)
```

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      """Entry point for launching an IPython kernel.
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

```python
# create the remaining 3 features in both TEST and TRAIN data

# bed_bath_rooms = bedrooms*bathrooms
train_data['bed_bath_rooms'] = train_data['bedrooms']*train_data['bathrooms']
test_data['bed_bath_rooms'] = test_data['bedrooms']*test_data['bathrooms']

# log_sqft_living = log(sqft_living)
train_data['log_sqft_living'] = train_data['sqft_living'].apply(lambda x: log(x))
test_data['log_sqft_living'] = test_data['sqft_living'].apply(lambda x: log(x))

# lat_plus_long = lat + long As an example here's the first one:
train_data['lat_plus_long'] = train_data['lat']+train_data['long']
test_data['lat_plus_long'] = test_data['lat']+test_data['long']
```

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:4: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      after removing the cwd from sys.path.
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:5: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      """
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:8: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:9: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      if __name__ == '__main__':
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:12: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      if sys.path[0] == '':
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:13: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      del sys.path[0]

* Squaring bedrooms will increase the separation between not many bedrooms (e.g. 1) and lots of bedrooms (e.g. 4) since
$$
\begin{align}
1^2 = 1
\end{align}
$$
but
$$
\begin{align}
4^2 = 16. \end{align}
$$
Consequently this feature will mostly affect houses with many bedrooms.
* bedrooms times bathrooms gives what's called an "interaction" feature. It is large when *both* of them are large.
* Taking the log of squarefeet has the effect of bringing large values closer together and spreading out small values.
* Adding latitude to longitude is totally non-sensical but we will do it anyway (you'll see why)

```python
train_data[['bedrooms','bathrooms','lat','long','bedrooms_squared','bed_bath_rooms','log_sqft_living','lat_plus_long']].head()
```

      bedrooms
      bathrooms
      lat
      long
      bedrooms_squared
      bed_bath_rooms
      log_sqft_living
      lat_plus_long

      8028
      3
      2.50
      47.5950
      -122.173
      9
      7.5
      8.185907
      -74.5780

      11967
      2
      1.00
      47.5532
      -122.280
      4
      2.0
      6.917706
      -74.7268

      15743
      4
      1.75
      47.6209
      -122.302
      16
      7.0
      7.791523
      -74.6811

      5750
      5
      1.00
      47.6605
      -122.324
      25
      5.0
      7.154615
      -74.6635

      10630
      4
      1.50
      47.7501
      -122.302
      16
      6.0
      7.489971
      -74.5519

**Quiz Question: What is the mean (arithmetic average) value of your 4 new features on TEST data? (round to 2 digits)**

**Answer:**

```python
print('Mean of TestData---(bedrooms squared: %d'%test_data['bedrooms_squared'].mean())
print('Mean of TestData-----(bed bath rooms: %d '%test_data['bed_bath_rooms'].mean())
print('Mean of TestData----(log sqft living: %d '%test_data['log_sqft_living'].mean())
print('Mean of TestData------(lat plus long: %d '%test_data['lat_plus_long'].mean())
```

    Mean of TestData---(bedrooms squared: 12
    Mean of TestData-----(bed bath rooms: 7
    Mean of TestData----(log sqft living: 7
    Mean of TestData------(lat plus long: -74

# Learning Multiple Models

Now we will learn the weights for three (nested) models for predicting house prices. The first model will have the fewest features the second model will add one more feature and the third will add a few more:
* Model 1: squarefeet, # bedrooms, # bathrooms, latitude & longitude
* Model 2: add bedrooms\*bathrooms
* Model 3: Add log squarefeet, bedrooms squared, and the (nonsensical) latitude + longitude

```python
model_1_features = ['sqft_living', 'bedrooms', 'bathrooms', 'lat', 'long']
model_2_features = model_1_features + ['bed_bath_rooms']
model_3_features = model_2_features + ['bedrooms_squared', 'log_sqft_living', 'lat_plus_long']
```

Now that you have the features, learn the weights for the three different models for predicting target = 'price' using linear_model.LinearRegression(). and look at the value of the weights/coefficients:

```python
# Learn the three models: (don't forget to set validation_set = None)
# MODEL 1
model_1 = linear_model.LinearRegression()
model_1.fit(train_data[model_1_features],train_data['price'])

# MODEL 2
model_2 = linear_model.LinearRegression()
model_2.fit(train_data[model_2_features],train_data['price'])

# MODEL 3
model_3 = linear_model.LinearRegression()
model_3.fit(train_data[model_3_features],train_data['price'])
```

    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)

```python
# Examine/extract each model's coefficients:
model_1_weight_summary = model_1.coef_
model_2_weight_summary = model_2.coef_
model_3_weight_summary = model_3.coef_

print('Model1 Coeficients:')
print(model_1_weight_summary)

print('\nModel2 Coeficients:')
print(model_2_weight_summary)

print('\nModel3 Coeficients:')
print(model_3_weight_summary)
```

    Model1 Coeficients:
    [ 3.08887695e+02 -5.83106922e+04  1.64003544e+04  6.56212809e+05
     -3.04971323e+05]

    Model2 Coeficients:
    [ 3.04530899e+02 -1.02903339e+05 -5.62387893e+04  6.53407715e+05
     -2.92481427e+05  2.12575454e+04]

    Model3 Coeficients:
    [ 5.14465125e+02  2.71898262e+04  8.02273301e+04  5.32003957e+05
     -4.04032857e+05 -1.23095455e+04 -4.64241039e+03 -5.30058064e+05
      1.27971100e+05]

**Quiz Question: What is the sign (positive or negative) for the coefficient/weight for 'bathrooms' in model 1?**

    **Answer:** Coeficient of 'bathroom' ---> Positive (3th)

**Quiz Question: What is the sign (positive or negative) for the coefficient/weight for 'bathrooms' in model 2?**

    **Answer:** Negative (3th)

Think about what this means.

# Comparing multiple models

Now that you've learned three models and extracted the model weights we want to evaluate which model is best.

First use your functions from earlier to compute the RSS on TRAINING Data for each of the three models.

```python
# Compute the RSS on TRAINING data for each of the three models and record the values:
rss_model_1_train = get_residual_sum_of_squares(model_1, train_data, model_1_features, train_data['price'])
rss_model_2_train = get_residual_sum_of_squares(model_2, train_data, model_2_features, train_data['price'])
rss_model_3_train = get_residual_sum_of_squares(model_3, train_data, model_3_features, train_data['price'])

print('RSS Model 1: %d'%(rss_model_1_train))
print('\nRSS Model 2: %d'%(rss_model_2_train))
print('\nRSS Model 3: %d'%(rss_model_3_train))
```

    RSS Model 1: 940940329132490

    RSS Model 2: 934349514420330

    RSS Model 3: 885348573944778

**Quiz Question: Which model (1, 2 or 3) has lowest RSS on TRAINING Data?** Is this what you expected?

    **Answer:** Model 3 is better than others.

Now compute the RSS on TEST data for each of the three models.

```python
# Compute the RSS on TESTING data for each of the three models and record the values:
rss_model_1_test = get_residual_sum_of_squares(model_1, test_data, model_1_features, test_data['price'])
rss_model_2_test = get_residual_sum_of_squares(model_2, test_data, model_2_features, test_data['price'])
rss_model_3_test = get_residual_sum_of_squares(model_3, test_data, model_3_features, test_data['price'])

print('RSS Model 1: %d'%(rss_model_1_test))
print('\nRSS Model 2: %d'%(rss_model_2_test))
print('\nRSS Model 3: %d'%(rss_model_3_test))
```

    RSS Model 1: 252310966056068

    RSS Model 2: 247600046018032

    RSS Model 3: 255242885877590

## Result
 **Quiz Question: Which model (1, 2 or 3) has lowest RSS on TESTING Data?** Is this what you expected? Think about the features that were added to each model from the previous.

* RSS Model-1 on TEST data: 219742130498805
* RSS Model-2 on TEST data: 216715801956578
* RSS Model-3 on TEST data: 209668128559972

The RSS of the Model_3 is the lowest.

> #### Model_3 is better than other models.

# Exercise 1- 2
# Regression Week 2: Multiple Regression (gradient descent)

In the first notebook we explored multiple regression using sklearn Create. Now we will use sklearn Create along with numpy to solve for the regression weights with gradient descent.

In this notebook we will cover estimating multiple regression weights via gradient descent. You will:
* Add a constant column of 1's to a dataframe to account for the intercept
* Convert an dataframe into a Numpy array
* Write a predict_output() function using Numpy
* Write a numpy function to compute the derivative of the regression weights with respect to a single feature
* Write gradient descent function to compute the regression weights given an initial weight vector, step size and tolerance.
* Use the gradient descent function to estimate regression weights for multiple features

# Load in house sales data

Dataset is from house sales in King County, the region where the city of Seattle, WA is located.

```python
sales_2 = pd.read_csv("data/week2_kc_house_data.csv")
dtype_dict = {'bathrooms':float, 'waterfront':int, 'sqft_above':int, 'sqft_living15':float, 'grade':int, 'yr_renovated':int, 'price':float, 'bedrooms':float, 'zipcode':str, 'long':float, 'sqft_lot15':float, 'sqft_living':float, 'floors':str, 'condition':int, 'lat':float, 'date':str, 'sqft_basement':int, 'yr_built':int, 'id':str, 'sqft_lot':int, 'view':int}

sales_2.head()
```

      id
      date
      price
      bedrooms
      bathrooms
      sqft_living
      sqft_lot
      floors
      waterfront
      view
      ...
      grade
      sqft_above
      sqft_basement
      yr_built
      yr_renovated
      zipcode
      lat
      long
      sqft_living15
      sqft_lot15

      0
      7129300520
      20141013T000000
      221900.0
      3
      1.00
      1180
      5650
      1.0
      0
      0
      ...
      7
      1180
      0
      1955
      0
      98178
      47.5112
      -122.257
      1340
      5650

      1
      6414100192
      20141209T000000
      538000.0
      3
      2.25
      2570
      7242
      2.0
      0
      0
      ...
      7
      2170
      400
      1951
      1991
      98125
      47.7210
      -122.319
      1690
      7639

      2
      5631500400
      20150225T000000
      180000.0
      2
      1.00
      770
      10000
      1.0
      0
      0
      ...
      6
      770
      0
      1933
      0
      98028
      47.7379
      -122.233
      2720
      8062

      3
      2487200875
      20141209T000000
      604000.0
      4
      3.00
      1960
      5000
      1.0
      0
      0
      ...
      7
      1050
      910
      1965
      0
      98136
      47.5208
      -122.393
      1360
      5000

      4
      1954400510
      20150218T000000
      510000.0
      3
      2.00
      1680
      8080
      1.0
      0
      0
      ...
      8
      1680
      0
      1987
      0
      98074
      47.6168
      -122.045
      1800
      7503

5 rows × 21 columns

If we want to do any "feature engineering" like creating new features or adjusting existing ones we should do this directly using the dataframe as seen in the other Week 2 notebook. For this notebook, however, we will work with the existing features.

# Convert to Numpy Array
Although dataframe offer a number of benefits to users (especially when using Big Data and built-in ... Create functions) in order to understand the details of the implementation of algorithms it's important to work with a library that allows for direct (and optimized) matrix operations. Numpy is a Python solution to work with matrices (or any multi-dimensional "array").

Recall that the predicted value given the weights and the features is just the dot product between the feature and weight vector. Similarly, if we put all of the features row-by-row in a matrix then the predicted value for *all* the observations can be computed by right multiplying the "feature matrix" by the "weight vector".

First we need to take the dataframe of our data and convert it into a 2D numpy array (also called a matrix). To do this we can use Panda's .as_matrix() to convert the dataframe into a numpy matrix.

Now we will write a function that will accept an Pandas dataframe, a list of feature names (e.g. ['sqft_living', 'bedrooms']) and an target feature e.g. ('price') and will return two things:
* A numpy matrix whose columns are the desired features plus a constant column (this is how we create an 'intercept')
* A numpy array containing the values of the output

With this in mind, complete the following function (where there's an empty line you should write a line of code that does what the comment above indicates)

```python
def get_numpy_data(dataframe, features, output):
    dataframe['constant'] = 1 # this is how you add a constant column to an dataframe

    # add the column 'constant' to the front of the features list so that we can extract it along with the others:
    features = ['constant'] + features # this is how you combine two lists

    # select the columns of data given by the features list into the dataframe features_data (now including constant):
    features_dataframe = dataframe[features]

    # the following line will convert the features_data into a numpy matrix:
    feature_matrix = features_dataframe.as_matrix()

    # assign the column of data associated with the output to the Array output_array
    output_dataframe = dataframe[output]

    # the following will convert the Array into a numpy array by first converting it to a list
    output_array = output_dataframe.as_matrix()

    return(feature_matrix, output_array)
```

For testing let's use the 'sqft_living' feature and a constant as our features and price as our output:

```python
(example_features, example_output) = get_numpy_data(sales_2, ['sqft_living'], 'price') # the [] around 'sqft_living' makes it a list

print("Example_features: ")
print(example_features[0,:]) # this accesses the first row of the data the ':' indicates 'all columns'
print("\nExample_output: ")
print(example_output[0]) # and the corresponding output
```

    Example_features:
    [   1 1180]

    Example_output:
    221900.0

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

# Predicting output given regression weights
Suppose we had the weights [1.0, 1.0] and the features [1.0, 1180.0] and we wanted to compute the predicted output
$$
\begin{align}
1.0*1.0 + 1.0*1180.0 = 1181.0
\end{align}
$$
this is the dot product between these two arrays. If they're numpy arrayws we can use np.dot() to compute this:

```python
my_weights = np.array([1., 1.])    # the example weights
my_features = example_features[0,] # we'll use the first data point
predicted_value = np.dot(my_features, my_weights)

print("Predicted Value: %d"%(predicted_value))
```

    Predicted Value: 1181

np.dot() also works when dealing with a matrix and a vector. Recall that the predictions from all the observations is just the RIGHT (as in weights on the right) dot product between the features *matrix* and the weights *vector*. With this in mind finish the following predict_output function to compute the predictions for an entire matrix of features given the matrix and the weights:

```python
def predict_output(feature_matrix, weights):
    # assume feature_matrix is a numpy matrix containing the features as columns and weights is a corresponding numpy array
    # create the predictions vector by using np.dot()
    predictions = np.dot(feature_matrix, weights)

    return(predictions)
```

If you want to test your code run the following cell:

```python
test_predictions = predict_output(example_features, my_weights)

print("Test_prediction[0]: %d"%(test_predictions[0])) # should be 1181.0
print("Test_prediction[1]: %d"%(test_predictions[1])) # should be 2571.0
```

    Test_prediction[0]: 1181
    Test_prediction[1]: 2571

# Computing the Derivative
We are now going to move to computing the derivative of the regression cost function. Recall that the cost function is the sum over the data points of the squared difference between an observed output and a predicted output.

Since the derivative of a sum is the sum of the derivatives we can compute the derivative for a single data point and then sum over data points. We can write the squared difference between the observed output and predicted output for a single point as follows:

$$
\begin{align}
(w[0]*[CONSTANT] + w[1]*[feature_1] + ... + w[i] *[feature_i] + ... +  w[k]*[feature_k] - output)^2
\end{align}
$$

Where we have k features and a constant. So the derivative with respect to weight w[i] by the chain rule is:

$$
\begin{align}
2*(w[0]*[CONSTANT] + w[1]*[feature_1] + ... + w[i] *[feature_i] + ... +  w[k]*[feature_k] - output)* [feature_i]
\end{align}
$$

The term inside the paranethesis is just the error (difference between prediction and output). So we can re-write this as:

$$
\begin{align}
2*error*[feature_i]
\end{align}
$$

That is, the derivative for the weight for feature i is the sum (over data points) of 2 times the product of the error and the feature itself. In the case of the constant then this is just twice the sum of the errors!

Recall that twice the sum of the product of two vectors is just twice the dot product of the two vectors. Therefore the derivative for the weight for feature_i is just two times the dot product between the values of feature_i and the current errors.

With this in mind complete the following derivative function which computes the derivative of the weight given the value of the feature (over all data points) and the errors (over all data points).

```python
def feature_derivative(errors, feature):
    # Assume that errors and feature are both numpy arrays of the same length (number of data points)
    # compute twice the dot product of these vectors as 'derivative' and return the value
    derivative = 2*np.dot(errors, feature)
    return(derivative)
```

To test your feature derivartive run the following:

```python
(example_features, example_output) = get_numpy_data(sales, ['sqft_living'], 'price')
my_weights = np.array([0., 0.]) # this makes all the predictions 0
test_predictions = predict_output(example_features, my_weights)
# just like dataframes 2 numpy arrays can be elementwise subtracted with '-':
errors = test_predictions - example_output # prediction errors in this case is just the -example_output
feature = example_features[:,0] # let's compute the derivative with respect to 'constant', the ":" indicates "all rows"
derivative = feature_derivative(errors, feature)

print(derivative)
print(-np.sum(example_output)*2) # should be the same as derivative
```

    -23345850016.0
    -23345850016.0

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

# Gradient Descent
Now we will write a function that performs a gradient descent. The basic premise is simple. Given a starting point we update the current weights by moving in the negative gradient direction. Recall that the gradient is the direction of *increase* and therefore the negative gradient is the direction of *decrease* and we're trying to *minimize* a cost function.

The amount by which we move in the negative gradient *direction*  is called the 'step size'. We stop when we are 'sufficiently close' to the optimum. We define this by requiring that the magnitude (length) of the gradient vector to be smaller than a fixed 'tolerance'.

With this in mind, complete the following gradient descent function below using your derivative function above. For each step in the gradient descent we update the weight for each feature befofe computing our stopping criteria

```python
from math import sqrt # recall that the magnitude/length of a vector [g[0], g[1], g[2]] is sqrt(g[0]^2 + g[1]^2 + g[2]^2)
```

```python
def regression_gradient_descent(feature_matrix, output, initial_weights, step_size, tolerance):
    converged = False
    weights = np.array(initial_weights) # make sure it's a numpy array
    while not converged:
        # compute the predictions based on feature_matrix and weights using your predict_output() function
        predictions = predict_output(feature_matrix, weights)

        # compute the errors as predictions - output
        errors = predictions - output

        gradient_sum_squares = 0 # initialize the gradient sum of squares
        # while we haven't reached the tolerance yet, update each feature's weight
        for i in range(len(weights)): # loop over each weight
            # Recall that feature_matrix[:, i] is the feature column associated with weights[i]
            # compute the derivative for weight[i]:
            derivative = feature_derivative(errors, feature_matrix[:, i])

            # add the squared value of the derivative to the gradient sum of squares (for assessing convergence)
            gradient_sum_squares += (derivative**2)

            # subtract the step size times the derivative from the current weight
            weights[i] -= (step_size * derivative)

        # compute the square-root of the gradient sum of squares to get the gradient magnitude:
        gradient_magnitude = sqrt(gradient_sum_squares)
        if gradient_magnitude < tolerance:
            converged = True
    return(weights)
```

A few things to note before we run the gradient descent. Since the gradient is a sum over all the data points and involves a product of an error and a feature the gradient itself will be very large since the features are large (squarefeet) and the output is large (prices). So while you might expect "tolerance" to be small, small is only relative to the size of the features.

For similar reasons the step size will be much smaller than you might expect but this is because the gradient has such large values.

# Running the Gradient Descent as Simple Regression
First let's split the data into training and test data.

```python
train_data,test_data = train_test_split(sales_2,test_size=0.2)
```

Although the gradient descent is designed for multiple regression since the constant is now a feature we can use the gradient descent function to estimate the parameters in the simple regression on squarefeet. The folowing cell sets up the feature_matrix, output, initial weights and step size for the first model:

```python
# let's test out the gradient descent
simple_features = ['sqft_living']
my_output = 'price'

(simple_feature_matrix, output) = get_numpy_data(train_data, simple_features, my_output)

initial_weights = np.array([-47000., 1.])
step_size = 7e-12
tolerance = 2.5e7
```

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

Next run your gradient descent with the above parameters.

```python
test_weight = regression_gradient_descent(simple_feature_matrix, output, initial_weights, step_size, tolerance)

print(test_weight)
```

    [-46999.88427043    282.48042519]

How do your weights compare to those achieved in week 1 (don't expect them to be exactly the same)?

**Quiz Question: What is the value of the weight for sqft_living -- the second element of ‘simple_weights’ (rounded to 1 decimal place)?**

Use your newly estimated weights and your predict_output() function to compute the predictions on all the TEST data (you will need to create a numpy array of the test feature_matrix and test output first:

```python
(test_simple_feature_matrix, test_output) = get_numpy_data(test_data, simple_features, my_output)
```

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

Now compute your predictions using test_simple_feature_matrix and your weights from above.

```python
test_predictions = predict_output(test_simple_feature_matrix, test_weight)

print(test_predictions)
```

    [306100.64722172 871061.49760915 534909.79162863 ... 458640.07682632
     667675.59146968 362596.73226046]

**Quiz Question: What is the predicted price for the 1st house in the TEST data set for model 1 (round to nearest dollar)?**

```python
print(test_predictions[0])
```

    306100.6472217162

Now that you have the predictions on test data, compute the RSS on the test data set. Save this value for comparison later. Recall that RSS is the sum of the squared errors (difference between prediction and output).

```python
test_residuals = test_output - test_predictions
test_RSS = (test_residuals * test_residuals).sum()

print(test_RSS)
```

    291220853705378.5

# Running a multiple regression
Now we will use more than one actual feature. Use the following code to produce the weights for a second model with the following parameters:

```python
model_features = ['sqft_living', 'sqft_living15'] # sqft_living15 is the average squarefeet for the nearest 15 neighbors.
my_output = 'price'
(feature_matrix, output) = get_numpy_data(train_data, model_features, my_output)
initial_weights = np.array([-100000., 1., 1.])
step_size = 4e-12
tolerance = 1e9
```

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

Use the above parameters to estimate the model weights. Record these values for your quiz.

```python
weight_2 = regression_gradient_descent(feature_matrix, output, initial_weights, step_size, tolerance)

print(weight_2)
```

    [-9.99999546e+04  2.42341529e+02  6.88997536e+01]

Use your newly estimated weights and the predict_output function to compute the predictions on the TEST data. Don't forget to create a numpy array for these features from the test set first!

```python
(test_feature_matrix, test_output) = get_numpy_data(test_data, model_features, my_output)

test_predictions_2 = predict_output(test_feature_matrix, weight_2)

print(test_predictions_2)
```

    [280783.67886893 891553.28695681 526688.14035549 ... 453676.95449708
     670215.55326846 351299.90592349]

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:11: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      # This is added back by InteractiveShellApp.init_path()
    C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

**Quiz Question: What is the predicted price for the 1st house in the TEST data set for model 2 (round to nearest dollar)?**

```python
print(test_predictions_2[0])
```

    280783.67886893265

What is the actual price for the 1st house in the test data set?

```python
# print(test_data['price'][0])
test_data['price'].iloc[0]
```

    155000.0

**Quiz Question: Which estimate was closer to the true price for the 1st house on the TEST data set, model 1 or model 2?**

Now use your predictions and the output to compute the RSS for model 2 on TEST data.

```python
test_residuals_2 = test_output - test_predictions_2
test_RSS_2 = (test_residuals_2**2).sum()

print(test_RSS_2)
```

    287039655170051.94

## Result
 **Quiz Question: Which model (1 or 2) has lowest RSS on all of the TEST data? **
* RSS Model-1 on TEST data: 930653102866614
* RSS Model-2 on TEST data: 329573350672128

The RSS of the Model_2 is lower than the Model_1.

> #### Model_2 is the better model.


In this notebook, using the Qiskit library, we will explore the basics routines in quantum computing, getting to know how to build a quantum circuit, inputing classical and quantum bits (qubits), attaching common gates, and also how to obtain measures from it, in order to use it for computational purposes.

This is going to be the foundation of our familiarity with the library, and once we've got the hang of it, we may use those basic building blocks to implement more intricate quantum algorithms.

```python
from qiskit import *
%matplotlib inline
```

### Building a quantum circuit

First, we are going to create two registers, one for qubits and one for classical bits. The ```ClassicalRegister``` will allow us to retrieve outcomes from the measurement operations on the circuit. Moreover, by passing a single integer onto the argument of those classes, we will be saying how many bits (quantum or classic) we want to have on our setup.

```python
qr = QuantumRegister(2) # a quantum register with 2 qubits
cr = ClassicalRegister(2) # a classical register with 2 bits
```

Now, we may start to build a circuit, in order to perform some desired operation. To do so, we need to input the qubits and classical bits we've created into a ```QuantumCircuit``` structure. Besides that, you can always call the ```draw()```  method for this class in order to visualize what the circuit is looking like. Throughout this notebook, I will always draw the diagram whenever the circuit has undergone some changes, hoping this can make the procedure easier to read.

```python
qc = QuantumCircuit(qr, cr) # initializing the quantum circuit using the qubits
                            # and classical bits from qr and cr registers
qc.draw(output = 'mpl')
```

Right now, our circuit doesn't do anything, it's just an old identity operator on the quantum states represented by the qubits.

⭐ By the way, the input qubits are initialized, by default, in the basis state $\left\lvert 0 \right\rangle.$

In order to make things a little bit more interesting, we need to _effectively_ build a circuit, i.e. introducing gates (operators) that will induce changes to the qubits. As an example, let us build a simple, yet intriguing, circuit. I will not spoil what it exactly does right now... Be carried away by the mystery! 😉

Okay, so our system will consist of only two gates: a Hadamard and a CNOT. First, we'll apply the Hadamard gate onto the $q_{0}^{(0)}$ qubit, right after its input line, and then, we'll implement the quantum CNOT gate onto the qubit pair $\left( q_{0}^{(0)}, q_{0}^{(1)} \right)$, using the first one as a _control_ qubit and the other one as a _target_ qubit.

```python
qc.h(qr[0]) # implementing Hadamard to q0_0 qubit
qc.cx(qr[0], qr[1]) # adding CNOT to the pair q0_0 (control) and q0_1 (target)
qc.draw(output = 'mpl')
```

Now, in order to be able to evaluate how it works, we need to perform a measurement on the qubits in our quantum register. The outcomes of the circuit are then stored into the bits of the classical register.

```python
qc.measure(qr, cr)
qc.draw(output = 'mpl')
```

There we have it: our first quantum circuit. In the next steps, we are going to run this circuit (in two different manners!) and see what results we get. However, before that, I would like to present some things.

Firstly, I wanna show that the above diagram can be drawn in some (slightly) distinct ways, which I find particularly easier to understand, as we can explicitly see where things come from (i.e. the qubits initial states) and where they go to (i.e. which classical bits the measurement results went to).

```python
# the *initial_state* parameter is False by default; set it to True if you wanna see them;
# the *cregbundle* parameter is True by default, it ""joins"" the diagrams of classical bits
# in the same register; set it to False to see them individually.
qc.draw(output = 'mpl', initial_state = True, cregbundle = False)
```

And secondly, I want to analytically determine the final state of this circuit and, once we've got it, probabilities can be calculated.

#### Theoretical Evaluation: Qubit Pair Final State & Outcomes Probabilities

Our quantum register comprises two quantum bits, respectively denoted $q_{0}^{(0)}$ and $q_{0}^{(1)}$. Their corresponding initial quantum states are going to be generically represented by the _kets_ $\left\lvert \psi_{0}^{(0)} \right\rangle$ and $\left\lvert \psi_{0}^{(1)} \right\rangle$; hence, the initial state of the 2-qubit input can be written as the tensor product of those vectors,

$$\left\lvert \hspace{0.8pt} \Psi_{i} \hspace{0.5pt} \right\rangle = \left\lvert \hspace{1.2pt} \psi_{0}^{(0)} \right\rangle \otimes \, \left\lvert \hspace{1.2pt} \psi_{0}^{(1)} \right\rangle \, .$$

Now, we introduce it into the ```qc``` circuit and pass it through the gates. As we said earlier, the circuit works as follows: apply the Hadamard gate onto the $q_{0}^{(0)}$ qubit and then implement the quantum CNOT gate onto the qubit pair $\left(\textrm{control} = q_{0}^{(0)}, \textrm{target} = q_{0}^{(1)} \right)$. Schematically, the procedural steps along the quantum circuit are:

$$ \left\lvert \hspace{1.2pt} \psi_{0}^{(0)} \right\rangle \otimes \, \left\lvert \hspace{1.2pt} \psi_{0}^{(1)} \right\rangle \,\, \longrightarrow \,\, \left( \hat{\mathrm{H}} \left\lvert \hspace{1.2pt} \psi_{0}^{(0)} \right\rangle \right) \otimes \, \left\lvert \hspace{1.2pt} \psi_{0}^{(1)} \right\rangle \,\, \longrightarrow \,\, \hat{U}_{\textrm{CNOT}} \left[ \left( \hat{\mathrm{H}} \left\lvert \hspace{1.2pt} \psi_{0}^{(0)} \right\rangle \right) \otimes \, \left\lvert \hspace{1.2pt} \psi_{0}^{(1)} \right\rangle \right] , $$

where $\hat{\mathrm{H}}$ is the Hadamard operator and $\hat{U}_{\textrm{CNOT}}$ is the CNOT operator. Therefore, the final quantum state of our qubit pair is given by

$$ \left\lvert \hspace{0.8pt} \Psi_{\hspace{-1pt} f} \hspace{0.5pt} \right\rangle = \, \hat{U}_{\textrm{CNOT}} \left[ \left( \hat{\mathrm{H}} \left\lvert \hspace{1.2pt} \psi_{0}^{(0)} \right\rangle \right) \otimes \, \left\lvert \hspace{1.2pt} \psi_{0}^{(1)} \right\rangle \right] . $$

So, at this stage, given particular initial states for the qubits $q_{0}^{(0)}$ and $q_{0}^{(1)}$, one is able to easily determine $\left\lvert \hspace{0.8pt} \Psi_{\hspace{-1pt} f} \hspace{0.5pt} \right\rangle$ from the equation above. As laid out before, both qubits are initialized in the $\left\lvert 0 \right\rangle$ basis state of their associated Hilbert spaces: $\left\lvert \psi_{0}^{(0)} \right\rangle = \left\lvert 0 \right\rangle \in \mathcal{H}^{(0)}$ and $\left\lvert \psi_{0}^{(1)} \right\rangle = \left\lvert 0 \right\rangle \in \mathcal{H}^{(1)}$. Thus,

$$\left\lvert \hspace{0.8pt} \Psi_{i} \hspace{0.5pt} \right\rangle = \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle \,\,\, \longrightarrow \,\,\,\, \left\lvert \hspace{0.8pt} \Psi_{\hspace{-1pt} f} \hspace{0.5pt} \right\rangle = \, \hat{U}_{\textrm{CNOT}} \left[ \left( \hat{\mathrm{H}} \left\lvert 0 \right\rangle \right) \otimes \, \left\lvert 0 \right\rangle \right] . $$

From now on, by applying specific representations of the operators involved and performing simple matrix algebra, we can easily obtain the final state,

\begin{align}
&\left\lvert \hspace{0.8pt} \Psi_{\hspace{-1pt} f} \hspace{0.5pt} \right\rangle = \, \hat{U}_{\textrm{CNOT}} \left[ \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle + \left\lvert 1 \right\rangle  \Big) \otimes \, \left\lvert 0 \right\rangle \right] = \, \hat{U}_{\mathrm{CNOT}}\left[ \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle + \left\lvert 1 \right\rangle \otimes \left\lvert 0 \right\rangle \Big)  \right] =\\[13pt]
&= \, \hat{U}_{\mathrm{CNOT}} \left[ \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle \oplus \left\lvert 0 \right\rangle \Big) \right] = \Big( \mathbb{1}_{2} \oplus \sigma^{1} \Big) \left[ \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle \oplus \left\lvert 0 \right\rangle \Big) \right] =  \frac{1}{\sqrt{2}} \Big( \mathbb{1}_{2} \left\lvert 0 \right\rangle \oplus \sigma^{1} \left\lvert 0 \right\rangle \Big) = \\[15pt]
&\hspace{9.6cm}= \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle \oplus \left\lvert 1 \right\rangle \Big) = \frac{1}{\sqrt{2}} \Big( \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle + \left\lvert 1 \right\rangle \otimes \left\lvert 1 \right\rangle \Big) \, .
\end{align}

In summary,

$$\left\lvert \hspace{0.8pt} \Psi_{\hspace{-1pt} f} \hspace{0.5pt} \right\rangle = \frac{1}{\sqrt{2}} \, \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle + \frac{1}{\sqrt{2}} \, \left\lvert 1 \right\rangle \otimes \left\lvert 1 \right\rangle . $$

 🎉 SURPRISE! 🎉

The output state of our 2-qubit circuit is a Bell state, which is one example of an entangled quantum state. Therefore, by acting onto the qubits $q_{0}^{(0)}$ and $q_{0}^{(1)}$, initially living in separable states, the ```qc``` circuit was able to entangle them. So, the quantum circuit we've built here can be used to generate entanglement in a qubit pair.

From the theoretical standpoint, explicitly knowing the final state, we conclude that the only possible observations, i.e. measurement outcomes, correspond to the states $\left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle$ and $\left\lvert 1 \right\rangle \otimes \left\lvert 1 \right\rangle$. Therefore, when performing simulations of the ideal ```qc``` circuit, we then expect to only see $\left\lvert 0 0 \right\rangle$ and $\left\lvert 1 1 \right\rangle$ in our results. Moreover, we can also predict the respective probabilities for each of those outcomes:

\begin{align}
&\mathbb{P}\big[ \textrm{ observing } \left\lvert 0 0 \right\rangle \big] = \left\lvert \Big( \left\langle 0 \right\rvert \otimes \left\langle 0 \right\rvert  \Big) \left( \frac{1}{\sqrt{2}} \, \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle + \frac{1}{\sqrt{2}} \, \left\lvert 1 \right\rangle \otimes \left\lvert 1 \right\rangle \right) \right\rvert^{2} = \left\lvert \frac{1}{\sqrt{2}} \right\rvert^{2} = \frac{1}{2} \, ,\\[13pt]
& \mathbb{P}\big[ \textrm{ observing } \left\lvert 1 1 \right\rangle \big] = \left\lvert \Big( \left\langle 1 \right\rvert \otimes \left\langle 1 \right\rvert  \Big) \left( \frac{1}{\sqrt{2}} \, \left\lvert 0 \right\rangle \otimes \left\lvert 0 \right\rangle + \frac{1}{\sqrt{2}} \, \left\lvert 1 \right\rangle \otimes \left\lvert 1 \right\rangle \right) \right\rvert^{2} = \left\lvert \frac{1}{\sqrt{2}} \right\rvert^{2} = \frac{1}{2} \, .
\end{align}

 Hence, $\mathbb{P}\big[ \textrm{ observing } \left\lvert 0 0 \right\rangle \big] = \mathbb{P}\big[ \textrm{ observing } \left\lvert 1 1 \right\rangle \big] = 50 \%$

### Running the quantum circuit

Now that we've built our system and know what to expect from it, we can run it and see if everything checks out. In order to do so, we are going to execute the ```qc``` circuit by two separate means.

#### 1st Approach: Simulation

Numerical simulations of quantum circuits are supported by the Qiskit library via the ```Aer``` provider, which has several high-performance simulator backends. You can use it to simulate not only ideal quantum circuits but noisy ones as well. So, be sure to take a look at the documentation.

Here, however, we are only going to see how you can perform a simulation in order to generate the countings necessary to see the measument outcomes and its corresponding probabilities. First, we'll implement the ```get_backend()``` method from ```Aer``` and store it in a variable, representing the particular backend in which we are going to run the simulation.

```python
sim = Aer.get_backend('qasm_simulator') # 'qasm_simulator' is just one example of a backend
                                        # you can get a list of possibilities by running
                                        # the following -> Aer.backends()
```

Once we've selected our desired simulation method, we just need to ```execute``` the circuit using the specific backend stored in the ```sim``` variable. In fact, you can simply run the command ```execute(qc, backend = sim)``` and the simulation will be done, but, we are just interested in some simple results: the outcome probabilities. Therefore, we will call the ```result()``` method on ```execute``` and only care about what this method returns.

```python
sim_results = execute(qc, backend = sim).result()
sim_results # just calling it so you can see what this looks like
```

    Result(backend_name='qasm_simulator', backend_version='0.8.2', qobj_id='d52fc42d-66ec-4738-8bb1-e1a9ef05ee8e', job_id='3e7d3eb2-e2a5-4096-abfd-f35de0159519', success=True, results=[ExperimentResult(shots=1024, success=True, meas_level=MeasLevel.CLASSIFIED, data=ExperimentResultData(counts={'0x0': 515, '0x3': 509}), header=QobjExperimentHeader(clbit_labels=[['c0', 0], ['c0', 1]], creg_sizes=[['c0', 2]], global_phase=0.0, memory_slots=2, metadata=None, n_qubits=2, name='circuit-0', qreg_sizes=[['q0', 2]], qubit_labels=[['q0', 0], ['q0', 1]]), status=DONE, seed_simulator=2951969917, metadata={'parallel_state_update': 4, 'parallel_shots': 1, 'measure_sampling': True, 'method': 'stabilizer', 'fusion': {'enabled': False}}, time_taken=0.004830147)], date=2021-07-10T22:27:10.932922, status=COMPLETED, status=QobjHeader(backend_name='qasm_simulator', backend_version='0.8.2'), metadata={'mpi_rank': 0, 'time_taken': 0.0052109700000000005, 'max_gpu_memory_mb': 0, 'max_memory_mb': 4096, 'parallel_experiments': 1, 'num_mpi_processes': 1, 'omp_enabled': True}, time_taken=0.005632877349853516)

Ok, once we've got our simulation results, we can retrieve some functions from ```qiskit.visualization``` that will help us... you know... visualize the data. In particular, we are going to import the ```plot_histogram``` function and use it to summarize the countings of each observed state that were stored in ```sim_results```. Those counts can be obtained by calling the ```get_counts()``` method.

```python
from qiskit.visualization import plot_histogram
counting_measured_states = sim_results.get_counts()
plot_histogram(counting_measured_states)
```

As we can see from the above plot, the only quantum states measured from the ```qc``` circuit simulation were $\left\lvert 0 0 \right\rangle$ and $\left\lvert 1 1 \right\rangle$, with respective probabilities $0.503$ and $0.497$. It strongly agrees with our expectations from the theoretical perspective. The differences we see in the simulation-obtained probabilities are not a concern, since this is just a consequence of the finite amount of observations performed. If we make more and more shots during the simulation, those values are both going to converge to $0.5$. So, it seems the computational resources we've implemented are working just fine. 😊

#### 2nd Approach: Experimental,

##### $\hspace{5cm}$ a.k.a. send to IBM and run it on a real quantum computer

Yeah... Impressive, right? This is really a wonderful thing we can do. The coding necessary to do this is just as simple as (and analogous to) the one we did for the simulation. However, you must have an account on IBM Quantum in order to enjoy this feature.

If you already have an account saved in your local disk, you just need load your credentials, using the command ```IBMQ.load_account()```. On the other hand, if you just recently created an IBMQ account or doesn't have your account saved, you must enable it during each notebook session before a practical use; you can do this by running ```IBMQ.enable_account(<YOUR TOKEN>)```, passing your account token as an argument of the function.

```python
IBMQ.enable_account(<YOUR TOKEN>) # or just run `IBMQ.load_account()` if your credentials
                                  # are already saved locally
```

Once the account is loaded/enabled in the session, we just need to pass who is our provider and which backend we want to run the circuit. In the simulation case, the provider was ```Aer```, already built-in on the library. In the experimental case, the provider is going to be ```'ibm-q'``` and must be retrieved from the ```IBMQ``` section, as we need to connect to their server. With respect to the backends, the ```Aer``` ones represent different simulation methods, while the ```IBMQ``` ones represent individual (quantum computing) hardware machines.

```python
provider = IBMQ.ibmq.get_provider('ibm-q') # getting the provider from IBM
real_quantum_computer = provider.get_backend('ibmq_5_yorktown') # getting the backend of the
                                                                # 'ibmq_5_yorktown' machine
experiment = execute(qc, backend = real_quantum_computer) # now, we send the qc circuit
                                                          # to be executed at the backend
                                                          # stored in real_quantum_computer
```

How can we know if our circuit was compiled? Has it started running yet?

Well, those questions can be answered by a particular tool in the library: the ```job_monitor```. As the name implies, this function will return us with some information about the job process we sent to the IBM hardware.

```python
from qiskit.tools.monitor import job_monitor # retrieving job_monitor from the appropriate sector
job_monitor(experiment) # getting the monitor to tell us about the
                        # execution stored in the `experiment` variable
```

    Job Status: job has successfully run

Once our request has made its way into the quantum computer, after standing in a queue, it will be compiled and we'll just need to wait. After the job is completed, we may get the results and visualize them in the same manner as we did for the simulation.

```python
experiment_results = experiment.result()
plot_histogram(experiment_results.get_counts())
```

There we have it! A histogram from _real_ experimental measurements. However, it seems a little bit odd when compared to the simulated case and to the theoretical predictions. Indeed, we see that the states $\left\lvert 0 0 \right\rangle$ and $\left\lvert 1 1 \right\rangle$ are the most probable of being observed, however, there is some outcomes coming from $\left\lvert 0 1 \right\rangle$ and $\left\lvert 1 0 \right\rangle$. So, what's that?

Well, the thing is: real quantum computers are far from being ideal. Our simulation was preset to run the circuit in an ideal scenario, and under the same assumptions underlying our theoretical evaluation. In reality, there is a lot of noise coming from the (undesired) interaction of the systems that constitute the quantum computer with the environment, i.e. outside world. As technology progresses, we hope that the noisy signal will be attenuated.

A first dive into the hands-on basics.ipynb


# Table of Contents

1  Propriedades da Convolução

1.1  Translação por um impulso

1.2  Resposta ao impulso

1.3  Decomposição

1.3.1  Visualizando as imagens:

# Propriedades da Convolução

A convolução possui várias propriedades que são úteis tanto para o melhor entendimento
do seu funcionamento como de uso prático. Aqui são ilustradas três propriedades: translação
por impulso, resposta ao impulso e decomposição do núcleo da convolução.

```python
# importando a função a ser utilizada nesse tutorial
import numpy as np
import sys,os
ea979path = os.path.abspath('../../')
if ea979path not in sys.path:
    sys.path.append(ea979path)
import ea979.src as ia
```

## Translação por um impulso

Quando o núcleo da composição é composto de apenas um único valor um e os demais zeros, a
imagem resultante será a translação da imagem original pelas coordenadas do valor não zero do núcleo.
No exemplo a seguir, o núcleo da convolução consiste do valor 1 na coordenada (19,59). Assim, a imagem
resultante ficara deslocada de 19 pixels para baixo e 59 para a direita. Observe que como estamos
tratando as imagens como infinitas com valores zeros fora do retângulo da imagem, esta translação faz
com que o retângulo da imagem aumente e vários valores iguais a zero sejam agora visíveis.

```python
import numpy as np

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import sys,os
os.chdir('../data')
f = mpimg.imread('cameraman.tif')

h = np.zeros((20,60))
h[19,59] = 1
nb = ia.nbshow(3)
nb.nbshow(f,'entrada')
g = ia.conv(f,h)
nb.nbshow(g.astype(np.uint8),'entrada translada de (20,60)')
nb.nbshow()

```

entrada

entrada translada de (20,60)

## Resposta ao impulso
Quando a imagem é formada por um único pixel de valor 1, o resultado da convolução é o núcleo da convolução. Esta propriedade permite que se visualize o núcleo da convolução. Se você souber que existe algum software que possui um filtro linear invariante à translação e você não sabe qual é o seu núcleo, basta aplicá-lo numa imagem com um único pixel igual a 1. O resultado do filtro revelará o seu núcleo. Na ilustração a seguir, uma imagem com vários impulsos é criada. Após aplicar a convolução com um filtro qualquer, é possível visualizar o núcleo sendo repetido em cada lugar do impulso.

```python
import numpy as np

#gerando imagem com pulsos
# 1 pulso a cada 4 linhas e 4 colunas
f = np.zeros((4,4))
f[3,3]= 1
f = np.tile(f,(2,2))
print('Matriz com impulsos:\n',f)

#gerando  filtro
h = np.array([ [1,2,3],[4,5,6],[7,8,9]])
print('\nNucleo do Filtro:\n',h)

g = ia.conv(f,h)
print('\nVisualização do núcleo após aplicar o fitro sobre a matriz com pulsos:\n',g)

```

    Matriz com impulsos:
     [[ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  1.  0.  0.  0.  1.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  1.  0.  0.  0.  1.]]

    Nucleo do Filtro:
     [[1 2 3]
     [4 5 6]
     [7 8 9]]

    Visualização do núcleo após aplicar o fitro sobre a matriz com pulsos:
     [[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  1.  2.  3.  0.  1.  2.  3.]
     [ 0.  0.  0.  4.  5.  6.  0.  4.  5.  6.]
     [ 0.  0.  0.  7.  8.  9.  0.  7.  8.  9.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
     [ 0.  0.  0.  1.  2.  3.  0.  1.  2.  3.]
     [ 0.  0.  0.  4.  5.  6.  0.  4.  5.  6.]
     [ 0.  0.  0.  7.  8.  9.  0.  7.  8.  9.]]

```python
import numpy as np

print('Aplicando a resposta ao impulso numa imagem real para ilustrar o seu comportamento')

#gerando imagem com pulsos
f = np.zeros((40,40))
f[20,20]= 1
f = np.tile(f,(10,10))
f = ia.normalize(f)
nb.nbshow(f, 'imagem original')

#gerando  filtro - circulo de raio 50
r,c = np.indices( (40, 40) )
h = ((r-20)**2 + (c-20)**2 < 20**2)
h = ia.normalize(h)
nb.nbshow(h, 'nucleo')

g = ia.conv(f,h)

nb.nbshow(ia.normalize(g), 'resposta ao impulso')
nb.nbshow()
```

    Aplicando a resposta ao impulso numa imagem real para ilustrar o seu comportamento

imagem original

nucleo

resposta ao impulso

## Decomposição
A propriedade da associatividade da convolução é dada por:

\begin{align}
f*h_{eq} = f*(h1*h2) = (f*h1)*h2
\end{align}

Se conseguirmos decompor um núcleo de modo que ele seja o resultado da convolução de dois núcleos mais simples, esta propriedade permite um ganho computacional se a convolução for aplicada por cada núcleo separadamente. A seguir é ilustrado o caso do núcleo que faz a soma dos pixels numa janela quadrada de 10 pixels de lado. Se a convolução for feita com o quadrado 10 x 10, serão 100 operações feitas na convolução. Se o núcleo for decomposto em dois núcleos uma linha e uma coluna de 10 pixels cada, cada convolução precisará de 10 operações, totalizando 20 operações ao todo. Observe a diferença no tempo de processamento destes dois casos.

```python
import numpy as np

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import sys,os
os.chdir('../data')
f = mpimg.imread('cameraman.tif')

h1 = np.ones((1,10))
h2 = np.ones((10,1))

h = ia.conv(h1,h2)
print('Nucleo original h=\n',h)
print('\nTempo de processamento 10 x 10:')
%timeit ia.conv(f,h)

f2 = ia.conv(f,h1)
print('\nNucleo decomposto\nh1=\n',h1,'\nh2=\n',h2)
print('\nTempo de processamento 10 horizontal e 10 vertical:')
%timeit ia.conv(f,h1), ia.conv(f2,h2)
```

Note a grande diferença no tempo de execução com o nucleo original e com o nucleo separado

### Visualizando as imagens:

```python
f1 = ia.conv(f,h)
f3= ia.conv(f2,h2)
nb.nbshow(ia.normalize(f1), 'filtragem pela soma na janela 10x10 (f1)')
nb.nbshow(ia.normalize(f3), 'filtragem pela soma na janela 10 horizontal e 10 vertical separadas (f3)')
nb.nbshow()
print('f1 é igual f3?\nMaxima diferença entre f1 e f3:', np.max(np.abs(f1-f3)) )
```

filtragem pela soma na janela 10x10 (f1)

filtragem pela soma na janela 10 horizontal e 10 vertical separadas (f3)

    f1 é igual f3?
    Maxima diferença entre f1 e f3: 0.0

```python

```


# Phase plane plot

# Purpose
Exploring the possibilities with *phase plane plots*

# Methodology
Quickly describe assumptions and processing steps.

# WIP - improvements
(WORK IN PROGRESS)
Use this section only if the notebook is not final.

Notable TODOs:
* todo 1
* todo 2
* todo 3

## Results
Describe and comment the most important results.

# Suggested next steps
State suggested next steps, based on results obtained in this notebook.

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import rolldecayestimators.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables

```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

```python
from scipy.integrate import solve_ivp
from sympy.physics.vector import dynamicsymbols
from sympy.matrices import Matrix
```

## Matplotlib example

```python
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec

w = 3
Y, X = np.mgrid[-w:w:100j, -w:w:100j]
U = -1 - X**2 + Y
V = 1 + X - Y**2
speed = np.sqrt(U**2 + V**2)

fig,ax=plt.subplots()
ax.streamplot(X, Y, U, V, density=[0.5, 1])
ax.set_title('Varying Density')
```

## Example from book (CH2: Flows on the line)
Logistic equation ($r=1$, $K=1$):
$$\dot{x} = x\cdot(1-x)$$

```python
def logistic_equation(t,x):
    xd1 = x*(1-x)
    return xd1

```

```python
dt = 0.5
t = np.arange(0,10,dt)
n_x = 15
x = np.linspace(0,2,n_x)
```

```python
T = np.tile(t,(n_x,1))
T.shape
```

    (15, 20)

```python
X = np.tile(x,(len(t),1)).T
X.shape
```

    (15, 20)

```python
dT = dt*np.ones((n_x,len(t)))
dT.shape
```

    (15, 20)

```python
dXdT = logistic_equation(t=T,x=X)
```

```python
x0s = [2,1,0.5,0.03,0.03]
t0s = [0,0,5,0,3]
xs = []
sims = OrderedDict()

for t0,x0 in zip(t0s,x0s):
    t_ = np.linspace(t0,t[-1],100)
    t_span = (t_[0],t_[-1])
    result = solve_ivp(fun = logistic_equation, t_span=t_span,  y0=[x0], t_eval=t_)

    sim = {
        't':t_,
        'x':result.y[0],
    }
    key = (t0,x0)
    sims[key]=sim

```

```python
fig,ax=plt.subplots()

# Normalize the arrows:
lengths = np.sqrt(dT**2 + dXdT**2)
U = dT / lengths
V = dXdT / lengths

ax.quiver(T, X, U, V)

for key,sim in sims.items():
    label = 't0=%0.1f, x0=%0.1f' % key
    ax.plot(sim['t'],sim['x'], label=label)

ax.set_ylabel('x')
ax.set_xlabel('t [s]')
ax.legend();
```

## Example from book (CH5: Linear systems)
Spring-mass system:

```python
m,k,omega = sp.symbols('m k omega')
x,v = dynamicsymbols('x v')
eq = sp.Eq(m*x.diff().diff() + k*x,0)
Math(vlatex(eq))
```

$\displaystyle k x + m \ddot{x} = 0$

```python
eq_xdiff = sp.Eq(v,x.diff())
Math(vlatex(eq_xdiff))
```

$\displaystyle v = \dot{x}$

```python
eq_v = eq.subs(x.diff().diff(),v.diff())
eq_v_diff = sp.Eq(v.diff(),sp.solve(eq_v,v.diff())[0])
Math(vlatex(eq_v_diff))
```

$\displaystyle \dot{v} = - \frac{k x}{m}$

```python
eq_omega = sp.Eq(omega**2,k/m)
eq_omega
```

$\displaystyle \omega^{2} = \frac{k}{m}$

```python
eq_v_diff_omega = eq_v_diff.subs(eq_omega.rhs,eq_omega.lhs)
Math(vlatex(eq_v_diff_omega))
```

$\displaystyle \dot{v} = - \omega^{2} x$

```python
state = Matrix([x,v])
Math(vlatex(state))
```

$\displaystyle \left[\begin{matrix}x\\v\end{matrix}\right]$

```python
eq_state_diff = sp.Eq(state.diff(), Matrix([sp.solve(eq_xdiff,x.diff()),
                            sp.solve(eq_v_diff_omega,v.diff())]))

Math(vlatex(eq_state_diff))
```

$\displaystyle \left[\begin{matrix}\dot{x}\\\dot{v}\end{matrix}\right] = \left[\begin{matrix}v\\- \omega^{2} x\end{matrix}\right]$

```python
A = Matrix([[0, 1],[-omega**2,0]])
A
```

$\displaystyle \left[\begin{matrix}0 & 1\\- \omega^{2} & 0\end{matrix}\right]$

```python
A*state
```

$\displaystyle \left[\begin{matrix}v{\left(t \right)}\\- \omega^{2} x{\left(t \right)}\end{matrix}\right]$

```python
np.array([[0,1],[-omega_**2,0]]).dot(state).shape
```

    (2, 100)

```python
def update(t,state, omega=1):

    state_diff = np.array([[0,1],[-omega**2,0]]).dot(state)

    return state_diff
```

```python
N = 10
v_ = np.linspace(-1.0,1.0,N)
x_ = np.linspace(-1.0,1.0,N)
omega_ = 1.0
X,V=np.meshgrid(x_,v_)
state = np.array([X.flatten(),V.flatten()])

state_diff = update(0, state, omega=omega_)
fig,ax=plt.subplots()
ax.quiver(X, V, state_diff[0,:], state_diff[1,:])

x0s = [-1.0,-1,0]
v0s = [0.0,0.5,0]
xs = []
sims = OrderedDict()

t_ = np.linspace(0,1.9*np.pi/omega_,100)

for x0,v0 in zip(x0s,v0s):

    t_span = (t_[0],t_[-1])
    result = solve_ivp(fun = update, t_span=t_span,  y0=[x0,v0], t_eval=t_, args=(omega_,))
    ax.plot(result.y[0,:],result.y[1,:])
    ax.plot(x0,v0,'go')

ax.grid(True)
ax.set_xlabel('x [m]')
ax.set_ylabel('v [m/s]')
```

I was puzzled that you could plot time derivatives in the vector field. Plotting $\frac{dx}{dt}$ on the x-axis, which has physical unit meters must be wrong. But I think instead that each arrow is a small time step, so that each derivative has been integrated to obtain the correct physical unit. And in this linear case the integration has been $\frac{dx}{dt} \cdot \Delta t$, so that each arrow has the slope:
\begin{bmatrix}
\frac{dx}{dt} \\
\frac{dv}{dt}
\end{bmatrix}

## Roll decay

```python
Math(vlatex(equations.roll_decay_equation_cubic))
```

$\displaystyle A_{44} \ddot{\phi} + \left(B_{1} + B_{2} \left|{\dot{\phi}}\right| + B_{3} \dot{\phi}^{2}\right) \dot{\phi} + \left(C_{1} + C_{3} \phi^{2} + C_{5} \phi^{4}\right) \phi = 0$

```python
x1, x2 = dynamicsymbols('x1 x2')

```

```python
Math(vlatex(x1.diff()))
```

$\displaystyle \dot{x}_{1}$

```python
roll_decay_equation_cubic_canonical = equations.roll_decay_equation_cubic.subs([
    (symbols.phi_dot,x2),
    (symbols.phi,x1),
])
```

```python
Math(vlatex(roll_decay_equation_cubic_canonical))
```

$\displaystyle A_{44} \dot{x}_{2} + \left(B_{1} + B_{2} \left|{x_{2}}\right| + B_{3} x^{2}_{2}\right) x_{2} + \left(C_{1} + C_{3} x^{2}_{1} + C_{5} x^{4}_{1}\right) x_{1} = 0$

```python
eq_acceleration_quadratic = sp.Eq(x2.diff(),
                        sp.solve(roll_decay_equation_cubic_canonical,x2.diff())[0])

```

```python
Math(vlatex(eq_acceleration_quadratic))
```

$\displaystyle \dot{x}_{2} = - \frac{B_{1} x_{2} + B_{2} x_{2} \left|{x_{2}}\right| + B_{3} x^{3}_{2} + C_{1} x_{1} + C_{3} x^{3}_{1} + C_{5} x^{5}_{1}}{A_{44}}$

```python
accelaration_quadratic_lambda = lambdify(sp.solve(eq_acceleration_quadratic,x2.diff())[0])
```

```python
accelaration_quadratic_lambda
```

    <function _lambdifygenerated(A_44, B_1, B_2, B_3, C_1, C_3, C_5, x1, x2)>

```python
class RollDecayCubic():

    def __init__(self,A_44, B_1, B_2, B_3, C_1, C_3, C_5,**kwargs):
        self.parameters = {
            'A_44':A_44,
            'B_1':B_1,
            'B_2':B_2,
            'B_3':B_3,
            'C_1':C_1,
            'C_3':C_3,
            'C_5':C_5,
        }

    def time_step(self,t,states):

        x1 = states[0]
        x2 = states[1]
        x21d = accelaration_quadratic_lambda(**self.parameters, x1=x1, x2=x2)
        d_states_dt = np.array([x2, x21d])

        return d_states_dt

    def simulate(self,t,phi0=np.deg2rad(10),phi1d0=0):

        initial_state = [phi0,phi1d0]

        t_span = [t[0], t[-1]]

        result = solve_ivp(fun=simulation.time_step, t_span=t_span,  y0=initial_state, t_eval=t)
        assert result.success
        df_result = pd.DataFrame(index=result.t, data=result.y.T, columns = ['phi','phi1d'])
        return df_result
```

```python
parameters = {
 'B_2A': 0.052059778389283974,
 'C_3A': -3.1821206941051803,
 'C_1A': 6.088581805080991,
 'B_3A': 0.0028870139489360486,
 'C_5A': 396.2046786411697,
 'B_1A': 0.05572815192117509,
 'score': 0.9992140495309503,
 'phi_start': 0.16293739741722763,
 'phi_stop': 0.036667136224551956,
 'omega0_fft': 2.4755750032144674,
 'A_44': 135.00020663291045,
 'B_2': 7.02808083981686,
 'C_3': -429.586951235060,
 'C_1': 821.959801787313,
 'B_3': 0.389747479658461,
 'C_5': 53487.7134854838,
 'B_1': 7.52331202462886,
 'omega0': 2.4675051783291138}

simulation = RollDecayCubic(**parameters)
```

```python
phi0 = np.deg2rad(9.335625196852943)
phi01d = 0
t = np.arange(24.39,72.73,0.01)
df_result = simulation.simulate(t=t,phi0=phi0, phi1d0=phi01d)
```

```python
fig,ax=plt.subplots()
df_result['phi_deg'] = np.rad2deg(df_result['phi'])
df_result.plot(y='phi_deg', ax=ax)
ax.grid(True)
```

```python
fig,ax=plt.subplots()
df_result.plot(x='phi', y='phi1d', ax=ax)
#ax.set_aspect('equal', 'box')
```

```python

```


# Simplified Ikeda regression

```python
from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)
```

```python
%matplotlib inline
%load_ext autoreload
%autoreload 2
```

```python
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
import numpy as np
import matplotlib.pyplot as plt
from pylab import rcParams
rcParams['figure.figsize'] = 15, 5

from rolldecayestimators.simplified_ikeda import calculate_roll_damping
from rolldecayestimators import equations
import sympy as sp
from rolldecayestimators import symbols
from rolldecayestimators.substitute_dynamic_symbols import lambdify
from mdldb.tables import Run
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from sklearn.pipeline import Pipeline

from rolldecay import database

import data
import copy
from rolldecay import database
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
```

```python
df_ikeda = database.load('rolldecay_simplified_ikeda',limit_score=0.5,
                         exclude_table_name='rolldecay_exclude')
#mask = (df_ikeda['ship_speed']==0)   ## Zero speed!
#df_ikeda = df_ikeda.loc[mask].copy()
```

```python
df_ikeda.describe()
```

```python
df_ikeda['score'].mean()
```

```python
B1_zeta_lambda = lambdify(sp.solve(equations.zeta_B1_equation, symbols.B_1)[0])
B2_d_lambda = lambdify(sp.solve(equations.d_B2_equation, symbols.B_2)[0])

def linearize(result):

    g=9.81
    rho=1000
    m = result.Volume*rho/(result.scale_factor**3)
    #result['B_1'] = B1_zeta_lambda(GM=result.gm, g=g, m=m, omega0=result.omega0,
    #                                 zeta=result.zeta)
    #result['B_2'] = B2_d_lambda(GM=result.gm, g=g, m=m, omega0=result.omega0, d=result.d)

    factor=1.0  # Factor
    phi_a = result.phi_start.abs()/factor  # Radians
    B_e_lambda=lambdify(sp.solve(equations.B_e_equation, symbols.B_e)[0])
    result['B_e'] = B_e_lambda(B_1=result['B_1'], B_2=result['B_2'], omega0=result.omega0,
                                 phi_a=phi_a)

    return result
```

```python
#df_ikeda = linearize(df_ikeda)
```

```python
df_ikeda.describe()
```

```python
df_ikeda['score'].hist(bins=30)
```

```python
df_direct = database.load('rolldecay_quadratic_b',limit_score=0.7)
mask = (df_direct['ship_speed']==0)   ## Zero speed!
df_direct = df_direct.loc[mask].copy()
df_direct = linearize(df_direct)
mask = (df_direct['B_e'] < df_direct['B_e'].quantile(q=0.98))
df_direct=df_direct.loc[mask]
```

```python
df_direct['score'].hist(bins=30)
```

```python
df_direct['B_e'].hist(bins=30)
```

```python
df_ikeda['B_e'].hist(bins=30)
```

```python
df_compare = pd.merge(left=df_ikeda, right=df_direct, how='left', left_index=True, right_index=True,
                     suffixes=('_ikeda',''))
#df_compare.dropna(inplace=True,subset=['B_e','B_e_ikeda'])
```

```python
fig,ax=plt.subplots()
df_compare.plot(x='B_e', y='B_e_ikeda', style='o', alpha=0.5, ax=ax)
xlim=ax.get_xlim()
ylim=ax.get_ylim()
ax.plot([0,xlim[1]],[0,xlim[1]],'r-')
ax.set_xlim(xlim)
ax.set_ylim(ylim)
ax.set_aspect('equal', 'box')
ax.set_xlabel('$B_e$ (Model test)')
ax.set_ylabel('$B_e$ Ikeda')
```

```python
fig,ax=plt.subplots()
N=20
bins = np.linspace(df_compare['B_e_ikeda'].min(),df_compare['B_e_ikeda'].max(),N)
df_compare['B_e'].hist(bins=bins, ax=ax, label='linear')
df_compare['B_e_ikeda'].hist(bins=bins, ax=ax, label='linear', alpha=0.5)

```

```python

```

```python

```


# CHEM 1000 - Spring 2022
Prof. Geoffrey Hutchison, University of Pittsburgh

## 7 Maclaurin and Taylor Series

Chapter 7 in [*Mathematical Methods for Chemists*](http://sites.bu.edu/straub/mathematical-methods-for-molecular-science/)

By the end of this session, you should be able to:
- Understand the concepts behind Maclaurin and Taylor series approximations
- Use Maclaurin and Taylor expansions for exp, log, sin, cos, and other functions

### Series

Last time, we saw that infinite series can be useful for chemistry and physics. Many times, we can either use a ***series limit*** for mathematical purposes. We can also use finite approximations from ***partial sums*** to estimate a more difficult or complex system.

In particular, we used a ***power series*** to approximate $e^x$ and saw that related power series could be derived for other functions.

We will now use two new kinds of series to approximate functions.

Mathematical functions that are continuous and infinitely differentiable may be  represented in terms of the function’s derivatives at a single point.

The ***Maclaurin series*** represents a function as power series in x with coefficients depending on the derivatives of the function at x = 0.

The ***Taylor series*** represents a function in terms of derivatives of the function at an arbitrary point x0 and powers of the displacement x - x0.

We'll start with the Maclaurin series, since the derivations are a bit "cleaner."

### Maclaurin Series

While we saw a few functions evaluated as power series, we would like to know how to express *any* function as a power series...

$$
f(x)=\sum_{n=0}^{\infty} a_{n} x^{n}=a_{0}+a_{1} x+a_{2} x^{2}+\ldots
$$

The key question, of course is how to calculate the coefficients $a_0$, $a_1$, etc.

Since we will perform this approximation near $x = 0$ we can easily get the first coefficient $a_0$:

$$
f(0) = a_0 + a_1\cdot 0 + a_2\cdot 0 + \ldots
$$

We can get the *next* coefficient with the first derivative:

$$
f'(0) = a_1 + 2 a_2 \cdot 0 + \ldots
$$

Not surprisingly, we can continue this with second derivatives, third derivatives, etc.

$$
\left.\frac{d^{2}}{d x^{2}} f(x)\right|_{x=0}=2 a_{2}
$$

$$
\left.\frac{d^{3}}{d x^{3}} f(x)\right|_{x=0}=2 \cdot 3 a_{3}
$$

So in general, the coefficients will all be:

$$
a_{n}=\left.\frac{1}{n !} \frac{d^{n}}{d x^{n}} f(x)\right|_{x=0}
$$

This may seem like a boring and tedious procedure. On the other hand, it means that we can express *any* function as a power series around $x = 0$, which can be very useful.

We will see throughout the course that there are several methods to approximate any function (even something amazingly complex) as an infinite series of simple "basis" functions.

```python
# Let's do some examples
from sympy import init_session
init_session()
```

We already know the power series for $e^x$ - we saw it last class:

$$
e^{x}=\sum_{n=0}^{\infty} \frac{1}{n !} x^{n}=1+x+\frac{1}{2 !} x^{2}+\frac{1}{3 !} x^{3}+\ldots
$$

Let's see how the Maclauren series works and if we can derive this power series for $e^x$. Remember that what we need are the $a_n$:

```python
f = exp(x)
print('a0, ', f.subs(x, 0))
print('a1, ', 1*diff(f, x).subs(x, 0))
print('a2, ', 1/2*diff(f, x, 2).subs(x, 0))
```

This is obviously boring, because the derivative of $e^x$ is *always* $e^x$ and at $x=0$ the number is *always* one.

$$
a_{n}=\left.\frac{1}{n !} \frac{d^{n}}{d x^{n}} f(x)\right|_{x=0}=\frac{1}{n !} \times 1
$$

Thus, it's not a surprise that the Maclaurin series for $e^x$ is:

$$
e^{x}=\sum_{n=0}^{\infty} \frac{1}{n !} x^{n}=1+x+\frac{1}{2 !} x^{2}+\frac{1}{3 !} x^{3}+\ldots
$$

Let's do a more interesting example for $\sin x$:

```python
f = sin(x)

print('a0, ', f.subs(x, 0))
print('a1, ', 1/factorial(1)*diff(f, x).subs(x, 0))
print('a2, ', 1/factorial(2)*diff(f, x, 2).subs(x, 0))
print('a3, ', 1/factorial(3)*diff(f, x, 3).subs(x, 0))
```

Let's think about what this means.

- constant: $\sin 0 = 0$
- first derivative: $d/dx = \cos x$ and of course $\cos 0 = 1$
- second derivative: $d^2/dx^2 = -\sin x$ so 0 again.
- third derivative: $d^3/dx^3 = -\cos x$ so $a_3 = \frac{1}{3!} \times -1 = \frac{-1}{6}$

Okay, I obviously could keep going by hand, but larger terms clearly call for a loop:

```python
# or more fun...
for n in range(6):
    print(n, 1/factorial(n)*diff(f, x, n).subs(x, 0))
```

Notice that for even orders, the resulting derivative will be $\sin x$ and thus the coefficient is zero.

I want to plot these...

```python
# Let's plot it!
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use('./chem1000.mplstyle')
```

```python
x = np.arange(-2*np.pi,2*np.pi,0.1)

plt.plot(x, np.sin(x), color='blue', label='sin() function')
# zero order is zero b/c sin(0) = 0
plt.plot(x, 1/factorial(1) * x, label='Taylor - 1st order')
# second order is the same b/c sin(0) = 0
plt.plot(x, 1/factorial(1) * x - 1/factorial(3)*x**3, label='Taylor - 3rd order')
# fourth order is the same again
plt.plot(x, 1/factorial(1) * x - 1/factorial(3)*x**3 + 1/factorial(5)*x**5, label='Taylor - 5th order')

plt.ylim([-7,4])
plt.legend()
plt.show()
```

Notice that as we add higher terms, we get an increasing accuracy of the Taylor series expansion for $\sin x$.

Of course, Sympy can do this too...

```python
x = symbols('x')
f = sin(x)

# Sympy already knows how to do a Maclaurin or Taylor expension of a function
#  around x0 up to (but not including) order 'n'
# expression.series(x, x0, n)
# if x0 is omitted, it's assumed to be around x0 = 0 (Maclaurin series)
# if n is omitted, it's assumed to be n=6
f.series(x, x0 = 0, n=6)

# the "O(x**4)" expression means that terms above x**4 are omitted
```

```python
x = symbols('x')
g = cos(x)
g.series(x, x0 = 0, n=4)
```

### Quick Approximations

The Maclaurin series is useful for some quick / easy approximations for functions near $x=0$:

- $e^x \approx 1+x+\frac{x^2}{2}$ - or even just $1 + x$
- $\sin x \approx x$
- $\cos x \approx 1 - \frac{x^2}{2}$

Obviously, these aren't very good outside some region of convergence, but when doing mental math, they're useful shortcuts.

### Taylor Series

The Maclaurin series derivation is only for $x = 0$, but what if we want to expand around a different point (e.g., an optimum bond length isn't going to be 0.0 Å...

It's not hard to imagine what we need to do:
- Write the power series with ($x - x_0$)
- Evaluate the derivatives at $x_0$ instead of zero.

That's about it. The math notation looks a bit messy, but:

$$
f(x)=\sum_{n=0}^{\infty} a_{n}\left(x-x_{0}\right)^{n}=a_{0}+a_{1}\left(x-x_{0}\right)+a_{2}\left(x-x_{0}\right)^{2}+\ldots
$$

### Example log(1)

We can't take the natural log of zero. So let's consider:

$$
f(x) = \ln x
$$

around $x_0 = 1$. Well, that's going to be:

$$
\ln (x)=\ln (1)+\left.\frac{d}{d x} \ln (x)\right|_{x=1}(x-1)+\left.\frac{1}{2 !} \frac{d^{2}}{d x^{2}} \ln (x)\right|_{x=1}(x-1)^{2}+\ldots
$$

Remember that the derivative of $\ln x = 1/x$:

- $\ln 1 = 0$
- $1/x = 1$
- $-1/x^2 = -1$
- $2/x^3 = 2$
- (etc.)

So the formula looks something like:
$$
\ln \approx 0+\left.\frac{1}{x}\right|_{x=1}(x-1)+\frac{1}{2 !}\left[\frac{-1}{x^{2}}\right]_{x=1}(x-1)^{2}+\frac{1}{3 !}\left[\frac{2}{x^{3}}\right]_{x=1}(x-1)^{3}+\ldots
$$

Then doing a little clean-up:
$$
\begin{aligned}
\ln (x) &=(x-1)-\frac{1}{2 !}(x-1)^{2}+\frac{2}{3 !}(x-1)^{3}-\frac{2 \cdot 3}{4 !}(x-1)^{4}+\ldots \\
&=(x-1)-\frac{1}{2}(x-1)^{2}+\frac{1}{3}(x-1)^{3}-\frac{1}{4}(x-1)^{4}+\ldots
\end{aligned}
$$

Putting it slightly differently for small deviations $x$:

$$
\ln (1+x)=x-\frac{1}{2} x^{2}+\frac{1}{3} x^{3}-\frac{1}{4} x^{4}+\cdots=\sum_{n=1}^{\infty}(-1)^{n+1} \frac{1}{n} x^{n}
$$

```python
# plot it!
x = np.arange(-0.5,2,0.1)

plt.plot(x, np.log(x+1), color='red', label='ln(x+1)')
plt.plot(x, x, label='1st order')
plt.plot(x, x - x**2/2, label='2nd order')
plt.plot(x, x - x**2/2 + x**3/3, label='3rd order')
plt.legend()
plt.show()
```

We previously talked about the Morse potential energy function:

$$
V(r)=\varepsilon\left[1-e^{-\beta\left(r-r_{0}\right)}\right]^{2}=\varepsilon\left[1-2 e^{-\beta\left(r-r_{0}\right)}+e^{-2 \beta\left(r-r_{0}\right)}\right]
$$

```python
x, x0, epsilon, beta = symbols('x x0 epsilon beta')
V = epsilon * (1 - exp(-beta * (x-x0)))**2

# harmonic approximation (i.e., terms in 0, 1, 2)
V.series(x, x0, n=3)
```

Notice that the simplest approximation of the potential energy is a harmonic expression (e.g., like a spring):

$$
V(r) \approx k(r - r_0)^2
$$

Near the optimal bond length, $r_0$ this looks fairly good. The problems come with higher energy, if the bond is greatly stretched or compressed.

At 298K, thermal energy is ≈0.025 eV = 0.592 kcal/mol, while we can see *anharmonic* parts of the curve (e.g., stretching at longer bond lengths) this is usually not a large effect.

```python
# let's plot it!
x = np.linspace(0.5, 2.5, 100)
beta = 2.253
epsilon = 265.79
x0 = 0.9624 # Angstroms

plt.plot(x, epsilon * (1 - np.exp(-beta * (x-x0)))**2, color='red')
plt.plot(x, beta**2*epsilon*(x - x0)**2, color='green')

plt.xlabel('Bond Length (Å)')
plt.ylabel('Relative Energy (kcal/mol)')
plt.ylim(0, 500)
plt.xlim(0.5, 1.75)
plt.show()
```

We can go to a higher-order Taylor series expansion to gain accuracy across a larger region. Here's the expansion up to 4th order. (Remember that Python counts from 0, so this goes up to n-1 terms, and n=5 is the O() error term.)

```python
x, x0 = symbols('x x0')
V.series(x, x0, n=5)
```

```python
# let's plot the expansion terms
x = np.linspace(0.5, 2.5, 100)
beta = 2.253
epsilon = 265.79
x0 = 0.9624 # Angstroms

plt.plot(x, epsilon * (1 - np.exp(-beta * (x-x0)))**2, color='red', label='Morse function')
plt.plot(x, beta**2*epsilon*(x - x0)**2, color='green', label='2nd order')
plt.plot(x, beta**2*epsilon*(x - x0)**2 - beta**3*epsilon*(x-x0)**3, label='3rd order')
plt.plot(x, beta**2*epsilon*(x - x0)**2
         - beta**3*epsilon*(x-x0)**3
         + 7/12*beta**4*epsilon*(x-x0)**4, label='4th order')

plt.xlabel('Bond Length (Å)')
plt.ylabel('Relative Energy (kcal/mol)')
plt.ylim(0, 1000)
plt.xlim(0.5, 1.75)
plt.legend()
plt.show()
```

While the 3rd order term goes negative at large separation (uh oh!), the 4th order term looks pretty successful from ~0.6-1.3 Å. We can continue to expand the Taylor series if we wish to have higher accuracy. Notice that the odd orders have negative sign, so we'll need to be careful to pick even orders to avoid negative energies.

```python
x, x0 = symbols('x x0')
V.series(x, x0, n=9)
```

```python
# let's plot the expansion terms
x = np.linspace(0.5, 2.5, 100)
beta = 2.253
epsilon = 265.79
x0 = 0.9624 # Angstroms

plt.plot(x, epsilon * (1 - np.exp(-beta * (x-x0)))**2, color='red')
plt.plot(x, beta**2*epsilon*(x - x0)**2, color='green')
plt.plot(x, beta**2*epsilon*(x - x0)**2
         - beta**3*epsilon*(x-x0)**3
         + 7/12*beta**4*epsilon*(x-x0)**4, color='blue')
plt.plot(x, beta**2*epsilon*(x - x0)**2
         - beta**3*epsilon*(x-x0)**3
         + 7/12*beta**4*epsilon*(x-x0)**4
         - 0.25*beta**5*epsilon*(x-x0)**5
         + 31/360*beta**6*epsilon*(x-x0)**6, color='gold')
plt.plot(x, beta**2*epsilon*(x - x0)**2
         - beta**3*epsilon*(x-x0)**3
         + 7/12*beta**4*epsilon*(x-x0)**4
         - 1/4*beta**5*epsilon*(x-x0)**5
         + 31/360*beta**6*epsilon*(x-x0)**6
         - 1/40*beta**7*epsilon*(x-x0)**7
         + 127/20160*beta**8*epsilon*(x-x0)**8, color='purple')

plt.xlabel('Bond Length (Å)')
plt.ylabel('Relative Energy (kcal/mol)')
plt.ylim(0, 750)
plt.xlim(0.5, 1.75)
plt.show()
```

-------
This notebook is from Prof. Geoffrey Hutchison, University of Pittsburgh
https://github.com/ghutchis/chem1000


# NUME

Instructor: Keny **ORDAZ**

# Final Exam

$\newcommand{\vb}[1]{\boldsymbol{#1}}\newcommand{\RR}[1]{\mathbb{R}^{#1}}
\newcommand{\vt}[1]{\vb{#1}^\top}$

$\newcommand{\x}{\vb{x}}
\newcommand{\xT}{\vt{x}}
\newcommand{\b}{\vb{b}}
\newcommand{\A}{\vb{A}}
\newcommand{\bT}{\vt{b}}
\newcommand{\AT}{\vt{A}}$

## Instructions

- You have exactly 24 hours to complete and submit your exam
- The results and the solving process are to be evaluated
- Clarity of code and good programming practices must be observed
- It must be presented as a **self-contained Jupyter Python Notebook** by e-mail.
- Take all the appropriate precautions to deliver the exam on time.
- Out-of-time deliveries will not be graded.
- Python libraries accepted: `numpy`, `scipy`, `sympy`, `matplotlib`.
- You MUST prove your conclusions or show a counter-example for all problems unless otherwise noted.

- Submit solutions to four (and no more) of the following six problems.

```python
import numpy as np
import scipy
import sympy as sp
import matplotlib.pyplot as plt
from IPython.display import SVG
```

# Problema 1

Sea $\A \in \RR{n \times n}$ y $\x \in \RR{k}$.

1) Encontrar la primer columna de $\vb{M} = (\A - x_1 \vb{I})(\A - x_2 \vb{I}) \cdots (\A - x_k \vb{I})$ empleando una secuencia de operaciones `GAXPY` ($\vb{y} = \A\x + \vb{y}$).

2) Probar para $n = 6$ y $k = 5$.

3) ¿Se simplifica si: (a) $k < n$, (b) $k = n$, (c) $k > n$, (d) u otra condición?

4) Proveer 6 ejemplos numéricos y su verificación.

5) Reporte observaciones sobre errores numéricos en los ejemplos del inciso anterior

# Problema 2

Sean $\vb{p}, \vb{q} \in \RR{n}, n \in \{ 2, 3\}$, vértices de un triángulo en un espacio euclidiano, $\vec{\vb{r}}, \vec{\vb{s}} \in \mathcal{V}$ vectores al espacio vectorial asociado al espacio euclidiano, isomórfico a $\RR{n}$.

1) Encontrar 3 formas de obtener el tercer vértice $\vb{o}$ dadas las siguientes precondiciones:

  - Los dos vectores no son paralelos y su producto punto es $< 0$.
  - El vector $\vec{\vb{t}} = \vb{q} - \vb{p}$ y los vectores $\vec{\vb{r}}, \vec{\vb{s}}$ son coplanares.

2) Proveer funciones que realicen la determinación de $\vb{o}$ y otra función para la verificación de las precondiciones.

3) Revisar:

  - Ley de senos, ley de cosenos, producto punto, producto cruz, sistemas de ecuaciones lineales, triple producto escalar, determinante de $\vb{A} \in \RR{2 \times 2}$

4) Encontrar el método / algoritmo más eficiente (menor número de operaciones) y que produzca menor error numérico.

5) Proveer casos de prueba y casos aleatorios y sus correspondientes verificaciones.

6) Todos los incisos para $n=2$ y para $n=3$. Generalizar para $n$ de ser posible.

```python
display(SVG('p2.svg'))
```

# Problema 3

Considera el problema de mínimos cuadrados, con precisión de 16 bits (media precisión: `numpy.half`):

$$\min_{\x \in \RR{n}} \|\A\x - \b \|\quad \text{donde } \A = \begin{pmatrix}%
1 & -1 \\
0 & 10^{-5}\\
0 & 0
\end{pmatrix} \text{ y } \b\begin{pmatrix}%
0 \\
10^{-5}\\
1
\end{pmatrix}
$$

1) Proporciona la aproximación a la solución en términos de mínimos cuadrados, usando:

  - Cholesky
  - Factorización QR

2) Con respecto a este caso, menciona las diferencias entre Cholesky y QR en términos de la estabilidad numérica.

# Problema 4

Sean $\A \in \RR{n \times n}$ y $\b, \x \in \RR{n}$. $\A(t), t \in \RR{}, a_{0, n- 1}(t) = a_{n-1, 0}(t) = f(t)$.

$n = 20, \; t \in [0, 10], \epsilon = 2^{52}, \epsilon_\text{neg} = 2^{53}$

1) Provea un sistema con $f(t) = -1^{\lceil t \rceil}$, y encuentre el método con mayor eficiencia temporal para solucionar el sistema en todos los momentos $t$ indicados, con $\Delta t = 0.125$

2) Provea un sistema con $f(t) = 1 - \frac{\epsilon_\text{neg}}{2}$, y encuentre el método con mayor eficiencia temporal para solucionar el sistema en todos los momentos $t$ indicados, con $\Delta t = 0.125$

3) Provea un sistema con $f(t) = 1 + \frac{\epsilon}{2}$, y encuentre el método con mayor eficiencia temporal para solucionar el sistema en todos los momentos $t$ indicados, con $\Delta t = 0.125$

4) Provea un sistema con $f(t) = \sin^2(t) + 5t$, y encuentre el método con mayor eficiencia temporal para solucionar el sistema en todos los momentos $t$ indicados, con $\Delta t = 0.125$

```python

```