# **Regression:**
Here I am going to share with you everything I know about **regression**.
All the information I gathered in these files was from free **educational resources** that I collected and improved with the help of my own knowledge.
A link to all the **resources** I used can be seen at the **end of this file**.
This collection is completely **free** and you can freely **share** it with other friends. I hope you can help increase your knowledge in **machine learning**.

## Contents:
* [What is Regression?](#what_is_regression)
* [Simple Linear Regression](#s_linear_regression)
* [Model Evaluation](#model_evaluation)
* [Multiple Linear Regression](#m_linear_regression)
* [Polynomial Regression](#poly_regression)
* [Non Linear Regression](#non_linear_regression)
* [Resources](#resources)

## -What is Regression?

**Regression** analysis is a **fundamental** concept in the field of **machine learning**. It falls under **supervised learning** wherein the algorithm is trained with both input features and output labels. It helps in establishing a relationship among the variables by estimating how one variable **affects** the other.

Imagine you're car shopping and have decided that gas mileage is a deciding factor in your decision to buy. If you wanted to **predict** the miles per gallon of some promising rides, how would you do it? Well, since you know the different **features** of the car (weight, horsepower, displacement, etc.) one possible method is **regression**. By plotting the average MPG of each car given its features you can then use regression techniques to find the relationship of the MPG and the input features. The regression function here could be represented as $Y = f(X)$, where **Y** would be the MPG and **X** would be the input features like the weight, displacement, horsepower, etc. The target function is $f$ and this curve helps us predict whether it’s beneficial to buy or not buy. This mechanism is called **regression**.

**Regression** in **machine learning** consists of **mathematical** methods that allow data scientists to **predict** a continuous outcome **(y)** based on the value of one or more predictor variables **(x)**. Linear regression is probably the most popular form of regression analysis because of its ease-of-use in predicting and forecasting.

* [go to contents](#contents)

## -Simple Linear Regression

**Simple linear regression** allows us to predict the label using a **single** feature.

### Understanding the Data:

### `FuelConsumption.csv`:
We have downloaded a fuel consumption dataset, **`FuelConsumption.csv`**, which contains model-specific fuel consumption ratings and estimated carbon dioxide emissions for new light-duty vehicles for retail sale in Canada. [Dataset source](http://open.canada.ca/data/en/dataset/98f1a129-f628-4ce4-b24d-6f16bf24dd64?utm_medium=Exinfluencer\&utm_source=Exinfluencer\&utm_content=000026UJ\&utm_term=10006555\&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2021-01-01)

*   **MODELYEAR** e.g. 2014
*   **MAKE** e.g. Acura
*   **MODEL** e.g. ILX
*   **VEHICLE CLASS** e.g. SUV
*   **ENGINE SIZE** e.g. 4.7
*   **CYLINDERS** e.g 6
*   **TRANSMISSION** e.g. A6
*   **FUEL CONSUMPTION in CITY(L/100 km)** e.g. 9.9
*   **FUEL CONSUMPTION in HWY (L/100 km)** e.g. 8.9
*   **FUEL CONSUMPTION COMB (L/100 km)** e.g. 9.2
*   **CO2 EMISSIONS (g/km)** e.g. 182   --> low --> 0

### Importing Packages:

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import warnings
%matplotlib inline
warnings.filterwarnings('ignore')
plt.style.use("seaborn-whitegrid")
```

### Reading The Dataset:

```python
data = pd.read_csv("FuelConsumption.csv")
data.head()
```

      MODELYEAR
      MAKE
      MODEL
      VEHICLECLASS
      ENGINESIZE
      CYLINDERS
      TRANSMISSION
      FUELTYPE
      FUELCONSUMPTION_CITY
      FUELCONSUMPTION_HWY
      FUELCONSUMPTION_COMB
      FUELCONSUMPTION_COMB_MPG
      CO2EMISSIONS

      0
      2014
      ACURA
      ILX
      COMPACT
      2.0
      4
      AS5
      Z
      9.9
      6.7
      8.5
      33
      196

      1
      2014
      ACURA
      ILX
      COMPACT
      2.4
      4
      M6
      Z
      11.2
      7.7
      9.6
      29
      221

      2
      2014
      ACURA
      ILX HYBRID
      COMPACT
      1.5
      4
      AV7
      Z
      6.0
      5.8
      5.9
      48
      136

      3
      2014
      ACURA
      MDX 4WD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.7
      9.1
      11.1
      25
      255

      4
      2014
      ACURA
      RDX AWD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.1
      8.7
      10.6
      27
      244

### Data Exploration:
Let's first have a **descriptive** exploration on our data.

```python
data.describe()
```

      MODELYEAR
      ENGINESIZE
      CYLINDERS
      FUELCONSUMPTION_CITY
      FUELCONSUMPTION_HWY
      FUELCONSUMPTION_COMB
      FUELCONSUMPTION_COMB_MPG
      CO2EMISSIONS

      count
      1067.0
      1067.000000
      1067.000000
      1067.000000
      1067.000000
      1067.000000
      1067.000000
      1067.000000

      mean
      2014.0
      3.346298
      5.794752
      13.296532
      9.474602
      11.580881
      26.441425
      256.228679

      std
      0.0
      1.415895
      1.797447
      4.101253
      2.794510
      3.485595
      7.468702
      63.372304

      min
      2014.0
      1.000000
      3.000000
      4.600000
      4.900000
      4.700000
      11.000000
      108.000000

      25%
      2014.0
      2.000000
      4.000000
      10.250000
      7.500000
      9.000000
      21.000000
      207.000000

      50%
      2014.0
      3.400000
      6.000000
      12.600000
      8.800000
      10.900000
      26.000000
      251.000000

      75%
      2014.0
      4.300000
      8.000000
      15.550000
      10.850000
      13.350000
      31.000000
      294.000000

      max
      2014.0
      8.400000
      12.000000
      30.200000
      20.500000
      25.800000
      60.000000
      488.000000

Let's select some **features** to explore more.

```python
data = data[['CYLINDERS', 'ENGINESIZE', 'FUELCONSUMPTION_CITY', 'FUELCONSUMPTION_COMB', 'CO2EMISSIONS']]
data.head(9)
```

      CYLINDERS
      ENGINESIZE
      FUELCONSUMPTION_CITY
      FUELCONSUMPTION_COMB
      CO2EMISSIONS

      0
      4
      2.0
      9.9
      8.5
      196

      1
      4
      2.4
      11.2
      9.6
      221

      2
      4
      1.5
      6.0
      5.9
      136

      3
      6
      3.5
      12.7
      11.1
      255

      4
      6
      3.5
      12.1
      10.6
      244

      5
      6
      3.5
      11.9
      10.0
      230

      6
      6
      3.5
      11.8
      10.1
      232

      7
      6
      3.7
      12.8
      11.1
      255

      8
      6
      3.7
      13.4
      11.6
      267

We can **plot** each of these features:

```python
plt.rcParams["figure.figsize"] = (16, 9)
data[['CYLINDERS', 'ENGINESIZE', 'FUELCONSUMPTION_CITY', 'FUELCONSUMPTION_COMB']].hist();
plt.show();
```

Now, let's plot **each of these features against the Emission**, to see how linear their relationship is:

```python
fig, ([ax1,ax2],[ax3, ax4]) = plt.subplots(2, 2, figsize=(15, 9))

ax1.scatter(x=data.CYLINDERS, y=data.CO2EMISSIONS)
ax1.set_xlabel("CYLINDERS")
ax1.set_ylabel("Emission")
ax1.set_title("CYLINDERS Feature")

ax2.scatter(x=data.ENGINESIZE, y=data.CO2EMISSIONS)
ax2.set_xlabel("ENGINESIZE")
ax2.set_ylabel("Emission")
ax2.set_title("ENGINESIZE Feature")

ax3.scatter(x=data.FUELCONSUMPTION_CITY, y=data.CO2EMISSIONS)
ax3.set_xlabel("FUELCONSUMPTION_CITY")
ax3.set_ylabel("Emission")
ax3.set_title("FUELCONSUMPTION_CITY Feature")

ax4.scatter(x=data.FUELCONSUMPTION_COMB, y=data.CO2EMISSIONS)
ax4.set_xlabel("FUELCONSUMPTION_COMB")
ax4.set_ylabel("Emission")
ax4.set_title("FUELCONSUMPTION_COMB Feature")

plt.subplots_adjust(left=0.1, bottom=0.1, right=0.9,
                    top=0.9, wspace=0.2, hspace=0.4)

plt.show()
```

### Creating Train and Test Data:

```python
msk = np.random.rand(len(data)) < 0.8
train = data[msk]
test = data[~msk]

x_train = np.asanyarray(train[["ENGINESIZE"]])
y_train = np.asanyarray(train[["CO2EMISSIONS"]])
```

### Modeling:

```python
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(x_train, y_train)
print('Coefficients: ', model.coef_)
print('Intercept: ', model.intercept_)
```

    Coefficients:  [[38.64150654]]
    Intercept:  [126.78041814]

### Plotting The Model:

```python
plt.rcParams["figure.figsize"] = (15, 8)
plt.scatter(train.ENGINESIZE, train.CO2EMISSIONS)
plt.plot(x_train, model.coef_[0][0] * x_train + model.intercept_[0], '-r')
plt.xlabel("ENGINESIZE", fontsize=13)
plt.ylabel("Emission", fontsize=13)
plt.show();
```

* [go to contents](#contents)

## -Model Evaluation

We compare the actual values and **predicted** values to calculate the **accuracy of a regression model**. Evaluation metrics provide a key role in the development of a model, as it provides insight to areas that require improvement.

There are different model **evaluation metrics**, lets use MSE here to calculate the accuracy of our model based on the test set:

*   **Mean Absolute Error(MAE)**: It is the mean of the absolute value of the errors. This is the easiest of the metrics to understand since it’s just average error.

*   **Mean Squared Error(MSE)**: Mean Squared Error (MSE) is the mean of the squared error. It’s more popular than Mean Absolute Error because the focus is geared more towards large errors. This is due to the squared term exponentially increasing larger errors in comparison to smaller ones.

*   **Root Mean Squared Error(RMSE)**.

*   **R-squared** is not an error, but rather a popular metric to measure the performance of your regression model. It represents how close the data points are to the fitted regression line. The higher the R-squared value, the better the model fits your data. The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse).

```python
from sklearn.metrics import r2_score

test_x = np.asanyarray(test[['ENGINESIZE']])
test_y = np.asanyarray(test[['CO2EMISSIONS']])
y_prim = model.predict(test_x)

print("Mean absolute error: %.2f" % np.mean(np.absolute(y_prim - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_prim - test_y) ** 2))
print("R2-score: %.2f" % r2_score(test_y , y_prim) )
```

    Mean absolute error: 24.89
    Residual sum of squares (MSE): 1059.03
    R2-score: 0.76

* [go to contents](#contents)

## -Multiple Linear Regression
**Simple linear regression** allows us to predict the label using **multiple** features.

### Reading The Dataset:

```python
data = pd.read_csv("FuelConsumption.csv")
data.head()
```

      MODELYEAR
      MAKE
      MODEL
      VEHICLECLASS
      ENGINESIZE
      CYLINDERS
      TRANSMISSION
      FUELTYPE
      FUELCONSUMPTION_CITY
      FUELCONSUMPTION_HWY
      FUELCONSUMPTION_COMB
      FUELCONSUMPTION_COMB_MPG
      CO2EMISSIONS

      0
      2014
      ACURA
      ILX
      COMPACT
      2.0
      4
      AS5
      Z
      9.9
      6.7
      8.5
      33
      196

      1
      2014
      ACURA
      ILX
      COMPACT
      2.4
      4
      M6
      Z
      11.2
      7.7
      9.6
      29
      221

      2
      2014
      ACURA
      ILX HYBRID
      COMPACT
      1.5
      4
      AV7
      Z
      6.0
      5.8
      5.9
      48
      136

      3
      2014
      ACURA
      MDX 4WD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.7
      9.1
      11.1
      25
      255

      4
      2014
      ACURA
      RDX AWD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.1
      8.7
      10.6
      27
      244

### Creating Train and Test Data:

```python
msk = np.random.rand(len(data)) < 0.8
train = data[msk]
test = data[~msk]
```

### Modeling:

```python
from sklearn.linear_model import LinearRegression
model = LinearRegression()
x_train = np.asanyarray(train[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])
y_train = np.asanyarray(train[['CO2EMISSIONS']])
model.fit (x_train, y_train)
print('Coefficients: ', model.coef_)
print('Intercept: ', model.intercept_)
```

    Coefficients:  [[10.6228337   7.324226    9.94831239]]
    Intercept:  [63.39547609]

As mentioned before, **Coefficient** and **Intercept**  are the parameters of the fitted line.
Given that it is a multiple linear regression model with 3 parameters and that the parameters are the intercept and coefficients of the hyperplane, sklearn can estimate them from our data. Scikit-learn uses plain Ordinary Least Squares method to solve this problem.

#### Ordinary Least Squares (OLS)

OLS is a method for estimating the unknown parameters in a linear regression model. OLS chooses the parameters of a linear function of a set of explanatory variables by minimizing the sum of the squares of the differences between the target dependent variable and those predicted by the linear function. In other words, it tries to minimizes the sum of squared errors (SSE) or mean squared error (MSE) between the target variable (y) and our predicted output ($\hat{y}$) over all samples in the dataset.

OLS can find the best parameters using of the following methods:

*   Solving the model parameters analytically using closed-form equations
*   Using an optimization algorithm (Gradient Descent, Stochastic Gradient Descent, Newton’s Method, etc.)

### Prediction:

```python
y_prim = model.predict(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])
x_test = np.asanyarray(test[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB']])
y_test = np.asanyarray(test[['CO2EMISSIONS']])

print("Residual sum of squares: %.2f"
      % np.mean((y_prim - y_test) ** 2))

# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % model.score(x_test, y_test))
```

    Residual sum of squares: 588.66
    Variance score: 0.84

* [go to contents](#contents)

## -Polynomial Regression

Sometimes, the trend of data is not really linear, and looks curvy. In this case we can use Polynomial regression methods. In fact, many different regressions exist that can be used to fit whatever the dataset looks like, such as quadratic, cubic, and so on, and it can go on and on to infinite degrees.

In essence, we can call all of these, polynomial regression, where the relationship between the independent variable x and the dependent variable y is modeled as an nth degree polynomial in x. Lets say you want to have a polynomial regression (let's make 2 degree polynomial):

$$y = b + \theta\_1  x + \theta\_2 x^2$$

Now, the question is: how we can fit our data on this equation while we have only x values, such as **Engine Size**?
Well, we can create a few additional features: 1, $x$, and $x^2$.

**PolynomialFeatures()** function in Scikit-learn library, drives a new feature sets from the original feature set. That is, a matrix will be generated consisting of all polynomial combinations of the features with degree less than or equal to the specified degree. For example, lets say the original feature set has only one feature, *ENGINESIZE*. Now, if we select the degree of the polynomial to be 2, then it generates 3 features, degree=0, degree=1 and degree=2:

### Reading The Dataset:

```python
data = pd.read_csv("FuelConsumption.csv")
data.head()
```

      MODELYEAR
      MAKE
      MODEL
      VEHICLECLASS
      ENGINESIZE
      CYLINDERS
      TRANSMISSION
      FUELTYPE
      FUELCONSUMPTION_CITY
      FUELCONSUMPTION_HWY
      FUELCONSUMPTION_COMB
      FUELCONSUMPTION_COMB_MPG
      CO2EMISSIONS

      0
      2014
      ACURA
      ILX
      COMPACT
      2.0
      4
      AS5
      Z
      9.9
      6.7
      8.5
      33
      196

      1
      2014
      ACURA
      ILX
      COMPACT
      2.4
      4
      M6
      Z
      11.2
      7.7
      9.6
      29
      221

      2
      2014
      ACURA
      ILX HYBRID
      COMPACT
      1.5
      4
      AV7
      Z
      6.0
      5.8
      5.9
      48
      136

      3
      2014
      ACURA
      MDX 4WD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.7
      9.1
      11.1
      25
      255

      4
      2014
      ACURA
      RDX AWD
      SUV - SMALL
      3.5
      6
      AS6
      Z
      12.1
      8.7
      10.6
      27
      244

Let's select some **features** that we want to use for **regression**.

```python
data = data[['ENGINESIZE','CYLINDERS','FUELCONSUMPTION_COMB','CO2EMISSIONS']]
data.head(9)
```

      ENGINESIZE
      CYLINDERS
      FUELCONSUMPTION_COMB
      CO2EMISSIONS

      0
      2.0
      4
      8.5
      196

      1
      2.4
      4
      9.6
      221

      2
      1.5
      4
      5.9
      136

      3
      3.5
      6
      11.1
      255

      4
      3.5
      6
      10.6
      244

      5
      3.5
      6
      10.0
      230

      6
      3.5
      6
      10.1
      232

      7
      3.7
      6
      11.1
      255

      8
      3.7
      6
      11.6
      267

### Creating Train and Test Data:

```python
msk = np.random.rand(len(data)) < 0.8
train = data[msk]
test = data[~msk]
```

### Polynomial Features:

```python
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

x_train = np.asanyarray(train[['ENGINESIZE']])
y_train = np.asanyarray(train[['CO2EMISSIONS']])

x_test = np.asanyarray(test[['ENGINESIZE']])
y_test = np.asanyarray(test[['CO2EMISSIONS']])

poly = PolynomialFeatures(degree=2)

x_train_poly = poly.fit_transform(x_train)

for i in range(5):
    print(f'{x_train[i]}  --->   {x_train_poly[i]}')
```

    [2.]  --->   [1. 2. 4.]
    [2.4]  --->   [1.   2.4  5.76]
    [1.5]  --->   [1.   1.5  2.25]
    [3.5]  --->   [ 1.    3.5  12.25]
    [3.5]  --->   [ 1.    3.5  12.25]

**fit_transform** takes our x values, and output a list of our data raised from power of 0 to power of 2 (since we set the degree of our polynomial to 2).

The equation and the sample example is displayed below.

$$
\begin{bmatrix}
v\_1\\\\
v\_2\\\\
\vdots\\\\
v_n
\end{bmatrix}\longrightarrow \begin{bmatrix}
\[ 1 & v\_1 & v\_1^2]\\\\
\[ 1 & v\_2 & v\_2^2]\\\\
\vdots & \vdots & \vdots\\\\
\[ 1 & v_n & v_n^2]
\end{bmatrix}
$$

$$
\begin{bmatrix}
2.\\\\
2.4\\\\
1.5\\\\
\vdots
\end{bmatrix} \longrightarrow \begin{bmatrix}
\[ 1 & 2. & 4.]\\\\
\[ 1 & 2.4 & 5.76]\\\\
\[ 1 & 1.5 & 2.25]\\\\
\vdots & \vdots & \vdots\\\\
\end{bmatrix}
$$

It looks like feature sets for multiple linear regression analysis, right? Yes. It Does.
Indeed, Polynomial regression is a special case of linear regression, with the main idea of how do you select your features. Just consider replacing the  $x$ with $x\_1$, $x\_1^2$ with $x\_2$, and so on. Then the degree 2 equation would be turn into:

$$y = b + \theta\_1  x\_1 + \theta\_2 x\_2$$

Now, we can deal with it as 'linear regression' problem. Therefore, this polynomial regression is considered to be a special case of traditional multiple linear regression. So, you can use the same mechanism as linear regression to solve such a problems.

so we can use **LinearRegression()** function to solve it:

### Modeling:

```python
model = LinearRegression()
model.fit(x_train_poly, y_train)
# The coefficients
print ('Coefficients: ', model.coef_)
print ('Intercept: ', model.intercept_)
```

    Coefficients:  [[ 0.         50.21106618 -1.42028818]]
    Intercept:  [106.51040218]

As mentioned before, **Coefficient** and **Intercept** , are the parameters of the fit curvy line.
Given that it is a typical multiple linear regression, with 3 parameters, and knowing that the parameters are the intercept and coefficients of hyperplane, sklearn has estimated them from our new set of feature sets. Lets plot it:

```python
plt.scatter(train.ENGINESIZE, train.CO2EMISSIONS)
XX = np.arange(0.0, 10.0, 0.1)
yy = model.intercept_[0]+ model.coef_[0][1]*XX+ model.coef_[0][2]*np.power(XX, 2)
plt.plot(XX, yy, '-r' )
plt.xlabel("Engine size")
plt.ylabel("Emission")
plt.show();
```

### Evaluation:

```python
from sklearn.metrics import r2_score

x_test_poly = poly.fit_transform(x_test)
y_prim = model.predict(x_test_poly)

print("Mean absolute error: %.2f" % np.mean(np.absolute(y_prim - y_test)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_prim - y_test) ** 2))
print("R2-score: %.2f" % r2_score(y_test, y_prim))
```

    Mean absolute error: 24.27
    Residual sum of squares (MSE): 1007.16
    R2-score: 0.73

* [go to contents](#contents)

## -Non Linear Regression

If the data shows a **curvy trend**, then linear regression will not produce very **accurate** results when compared to a **non-linear** regression since linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example in python. In this notebook, we fit a non-linear model to the datapoints corrensponding to **China's GDP** from **1960 to 2014**.

Although linear regression can do a great job at modeling some datasets, it cannot be used for **all datasets**. First recall how linear regression, models a dataset. It models the linear relationship between a **dependent variable y** and the **independent variables x**. It has a simple equation, of degree 1, for example y = $2x$ + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise

#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

**Non-linear regression** is a method to model the non-linear relationship between the independent variables $x$ and the dependent variable $y$. Essentially any relationship that is **not linear** can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).  For example:

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like **exponentials**, **logarithms**, f**ractions**, and so on. For example: $$ y = \log(x)$$

We can have a function that's even more **complicated** such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic: $$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

### Exponential: $$ Y = a + b c^X$$
where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
Y= np.exp(X)
plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

### Logarithmic: $$ y = \log(x)$$

The response $y$ is a results of applying the logarithmic map from the input $x$ to the output $y$. It is one of the simplest form of **log()**: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be a polynomial representation of the $x$ values. In general form it would be written as\
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

### Sigmoidal/Logistic: $$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))
plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

For an example, we're going to try and fit a model to the datapoints corresponding to **China's GDP from 1960 to 2014**. We download a dataset with two columns, the first, a year between **1960 and 2014**, the second, **China's corresponding annual gross domestic income** in US dollars for that year.

### Reading The Dataset:

```python
data = pd.read_csv("china_gdp.csv")
data.head(10)
```

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

### Plotting The Dataset:
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerate slightly in the 2010s.

```python
x_data, y_data = (data["Year"].values, data["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show();
```

### Choosing a model:

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show();
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta\_1(X-\beta\_2)}}$$

$\beta\_1$: Controls the curve's steepness,

$\beta\_2$: Slides the curve on the x-axis.

### Building The Model:

Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
plt.show();
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata = x_data/max(x_data)
ydata = y_data/max(y_data)
```

#### How we find the best parameters for our fit line?

we can use **curve_fit** which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, \*popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

     beta_1 = 690.453018, beta_2 = 0.997207

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)

y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')

plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show();
```

* [go to contents](#contents)

## -Resources:
   *    Edx IBM Developer Skills Network, Machine Learning Course With Saaed Aghabozorgi & Joseph Santarcangelo.
   *    Maktabkhooneh Machine Learning With Jadi Course.
   *    https://builtin.com/data-science/regression-machine-learning

* [go to contents](#contents)


# Quantum Fourier Transforms

The **"QFT (Quantum Fourier Transform)"** quantum kata is a series of exercises designed
to teach you the basics of quantum Fourier transform (QFT). It covers implementing QFT and using
it to perform simple state transformations.

Each task is wrapped in one operation preceded by the description of the task.
Your goal is to fill in the blank (marked with the `// ...` comments)
with some Q# code that solves the task. To verify your answer, run the cell using Ctrl+Enter (⌘+Enter on macOS).

Within each section, tasks are given in approximate order of increasing difficulty;
harder ones are marked with asterisks.

## Part I. Implementing Quantum Fourier Transform

This sequence of tasks uses the implementation of QFT described in Nielsen & Chuang.
All numbers in this kata use big endian encoding: most significant bit of the number
 is stored in the first (leftmost) bit/qubit.

### Task 1.1. 1-qubit QFT

**Input:**

  A qubit in state $|\psi\rangle = x_0 |0\rangle + x_1 |1\rangle$.

**Goal:**

Apply QFT to this qubit, i.e., transform it to a state
$\frac{1}{\sqrt{2}} \big((x_0 + x_1) |0\rangle + (x_0 - x_1) |1\rangle\big)$.

In other words, transform a basis state $|j\rangle$ into a state $\frac{1}{\sqrt{2}} \big(|0\rangle + e^{2\pi i \cdot \frac{j}{2}}|1\rangle\big)$ .

```qsharp
%kata T11_OneQubitQFT

operation OneQubitQFT (q : Qubit) : Unit is Adj+Ctl {
    H(q);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.1.-1-qubit-QFT).*

### Task 1.2. Rotation gate

**Inputs:**

  1. A qubit in state $|\psi\rangle = \alpha |0\rangle + \beta |1\rangle$.

  2. An integer k $\geq$ 0.

**Goal:**

Change the state of the qubit to $\alpha |0\rangle + \beta \cdot e^{\frac{2\pi i}{2^{k}}} |1\rangle$.

> Be careful about not introducing an extra global phase!
This is going to be important in the later tasks.

```qsharp
%kata T12_Rotation

open Microsoft.Quantum.Convert;
open Microsoft.Quantum.Math;

operation Rotation (q : Qubit, k : Int) : Unit is Adj+Ctl {
    // The R1 gate leaves |0> untouched,
    // sends |1> to e^(i * theta)|1>
    let theta = (2.0 * PI())/(2.0 ^ IntAsDouble(k));
    R1(theta, q);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.2.-Rotation-gate).*

### Task 1.3. Prepare binary fraction exponent (classical input)

**Inputs:**

  1. A qubit in state $|\psi\rangle = \alpha |0\rangle + \beta |1\rangle$.

  2. An array of $n$ bits $[j_1, j_2, ..., j_n]$, stored as `Int[]` ($ j_k \in \{0,1\}$).

**Goal:**

Change the state of the qubit to $\alpha |0\rangle + \beta \cdot e^{2\pi i \cdot 0.j_1 j_2 ... j_n} |1\rangle$,
where $0.j_1 j_2 ... j_n$ is a binary fraction, similar to decimal fractions:

$$0.j_1 j_2 ... j_n = j_1 \cdot \frac{1}{2^1} + j_2 \cdot \frac{1}{2^2} + ... j_n \cdot \frac{1}{2^n}$$

```qsharp
%kata T13_BinaryFractionClassical

operation BinaryFractionClassical (q : Qubit, j : Int[]) : Unit is Adj+Ctl {
    // e^(2 * Pi * (j1 + j2 + j3 + .. j2)) =
    // e^(2 * PI) * e^j1 * ...
    for i in 0 .. Length(j) - 1 {
        if (j[i] == 1) {
            Rotation(q, i + 1);
        }
    }
}
```

    1-bit 0 = [0]
    1-bit 1 = [1]
    2-bit 0 = [0,0]
    2-bit 1 = [0,1]
    2-bit 2 = [1,0]
    2-bit 3 = [1,1]
    3-bit 0 = [0,0,0]
    3-bit 1 = [0,0,1]
    3-bit 2 = [0,1,0]
    3-bit 3 = [0,1,1]
    3-bit 4 = [1,0,0]
    3-bit 5 = [1,0,1]
    3-bit 6 = [1,1,0]
    3-bit 7 = [1,1,1]
    4-bit 0 = [0,0,0,0]
    4-bit 1 = [0,0,0,1]
    4-bit 2 = [0,0,1,0]
    4-bit 3 = [0,0,1,1]
    4-bit 4 = [0,1,0,0]
    4-bit 5 = [0,1,0,1]
    4-bit 6 = [0,1,1,0]
    4-bit 7 = [0,1,1,1]
    4-bit 8 = [1,0,0,0]
    4-bit 9 = [1,0,0,1]
    4-bit 10 = [1,0,1,0]
    4-bit 11 = [1,0,1,1]
    4-bit 12 = [1,1,0,0]
    4-bit 13 = [1,1,0,1]
    4-bit 14 = [1,1,1,0]
    4-bit 15 = [1,1,1,1]
    5-bit 0 = [0,0,0,0,0]
    5-bit 1 = [0,0,0,0,1]
    5-bit 2 = [0,0,0,1,0]
    5-bit 3 = [0,0,0,1,1]
    5-bit 4 = [0,0,1,0,0]
    5-bit 5 = [0,0,1,0,1]
    5-bit 6 = [0,0,1,1,0]
    5-bit 7 = [0,0,1,1,1]
    5-bit 8 = [0,1,0,0,0]
    5-bit 9 = [0,1,0,0,1]
    5-bit 10 = [0,1,0,1,0]
    5-bit 11 = [0,1,0,1,1]
    5-bit 12 = [0,1,1,0,0]
    5-bit 13 = [0,1,1,0,1]
    5-bit 14 = [0,1,1,1,0]
    5-bit 15 = [0,1,1,1,1]
    5-bit 16 = [1,0,0,0,0]
    5-bit 17 = [1,0,0,0,1]
    5-bit 18 = [1,0,0,1,0]
    5-bit 19 = [1,0,0,1,1]
    5-bit 20 = [1,0,1,0,0]
    5-bit 21 = [1,0,1,0,1]
    5-bit 22 = [1,0,1,1,0]
    5-bit 23 = [1,0,1,1,1]
    5-bit 24 = [1,1,0,0,0]
    5-bit 25 = [1,1,0,0,1]
    5-bit 26 = [1,1,0,1,0]
    5-bit 27 = [1,1,0,1,1]
    5-bit 28 = [1,1,1,0,0]
    5-bit 29 = [1,1,1,0,1]
    5-bit 30 = [1,1,1,1,0]
    5-bit 31 = [1,1,1,1,1]

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.3.-Prepare-binary-fraction-exponent-(classical-input)).*

### Task 1.4. Prepare binary fraction exponent (quantum input)

**Inputs:**

  1. A qubit in state $|\psi\rangle = \alpha |0\rangle + \beta |1\rangle$.

  2. A register of $n$ qubits in state $|j_1 j_2 ... j_n\rangle$.

**Goal:**

Change the state of the input
from $(\alpha |0\rangle + \beta |1\rangle) \otimes |j_1 j_2 ... j_n\rangle$
to $(\alpha |0\rangle + \beta \cdot e^{2\pi i \cdot 0.j_1 j_2 ... j_n} |1\rangle) \otimes |j_1 j_2 ... j_n\rangle$,

where $0.j_1 j_2 ... j_n$ is a binary fraction corresponding to the basis state $j_1 j_2 ... j_n$ of the register.

> The register of qubits can be in superposition as well;
the behavior in this case is defined by behavior on the basis states and the linearity of unitary transformations.

```qsharp
%kata T14_BinaryFractionQuantum

operation BinaryFractionQuantum (q : Qubit, jRegister : Qubit[]) : Unit is Adj+Ctl {
    for i in 0 .. Length(jRegister) - 1 {
        Controlled Rotation ([jRegister[i]], (q, i + 1));
    }
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.4.-Prepare-binary-fraction-exponent-(quantum-input)).*

### Task 1.5. Prepare binary fraction exponent in place (quantum input)

**Input:**

 A register of $n$ qubits in state $|j_1 j_2 ... j_n \rangle$.

**Goal:**

Change the state of the register
from $|j_1\rangle \otimes |j_2 ... j_n\rangle$
to $\frac{1}{\sqrt{2}} \big(|0\rangle + e^{2\pi i \cdot 0.j_1 j_2 ... j_n} |1\rangle \big) \otimes |j_2 ... j_n\rangle$.

> The register of qubits can be in superposition as well;
the behavior in this case is defined by behavior on the basis states and the linearity of unitary transformations.

  Need a hint? Click here
  This task is very similar to task 1.4, but the digit $j_1$ is encoded in-place, using task 1.1.

```qsharp
%kata T15_BinaryFractionQuantumInPlace

operation BinaryFractionQuantumInPlace (register : Qubit[]) : Unit is Adj+Ctl {
    OneQubitQFT(register[0]);

    for ind in 1 .. Length(register) - 1 {
        Controlled Rotation([register[ind]], (register[0] , ind + 1));
    }
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.5.-Prepare-binary-fraction-exponent-in-place-(quantum-input)).*

### Task 1.6. Reverse the order of qubits

**Input:**

 A register of $n$ qubits in state $|x_1 x_2 ... x_n \rangle$.

**Goal:**

Reverse the order of qubits, i.e., convert the state of the register to $|x_n ... x_2 x_1\rangle$.

```qsharp
%kata T16_ReverseRegister

operation ReverseRegister (register : Qubit[]) : Unit is Adj+Ctl {
    let N = Length(register);
    for ind in 0 .. N / 2 - 1 {
        SWAP(register[ind], register[N - 1 - ind]);
    }
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.6.-Reverse-the-order-of-qubits).*

### Task 1.7. Quantum Fourier transform

**Input:**

 A register of $n$ qubits in state $|j_1 j_2 ... j_n \rangle$.

**Goal:**

Apply quantum Fourier transform to the input register, i.e., transform it to a state

$$\frac{1}{\sqrt{2^{n}}} \sum_{k=0}^{2^n-1} e^{2\pi i \cdot \frac{jk}{2^{n}}} |k\rangle = $$
$$\begin{align}= &\frac{1}{\sqrt{2}} \big(|0\rangle + e^{2\pi i \cdot 0.j_n} |1\rangle\big) \otimes \\
\otimes &\frac{1}{\sqrt{2}} \big(|0\rangle + e^{2\pi i \cdot 0.j_{n-1} j_n} |1\rangle\big) \otimes ... \otimes \\
\otimes &\frac{1}{\sqrt{2}} \big(|0\rangle + e^{2\pi i \cdot 0.j_1 j_2 ... j_{n-1} j_n} |1\rangle\big)\end{align}$$

> The register of qubits can be in superposition as well;
the behavior in this case is defined by behavior on the basis states and the linearity of unitary transformations.

> You can do this with a library call, but we recommend
implementing the algorithm yourself for learning purposes, using the previous tasks.

  Need a hint? Click here
Consider preparing a different state first and transforming it to the goal state:

$\frac{1}{\sqrt{2}} \big(|0\rangle + exp(2\pi i \cdot 0.j_1 j_2 ... j_{n-1} j_n) |1\rangle\big) \otimes ...
\otimes \frac{1}{\sqrt{2}} \big(|0\rangle + exp(2\pi i \cdot 0.j_{n-1} j_n) |1\rangle\big)
\otimes \frac{1}{\sqrt{2}} \big(|0\rangle + exp(2\pi i \cdot 0.j_n) |1\rangle\big)$

```qsharp
%kata T17_QuantumFourierTransform

operation QuantumFourierTransform (register : Qubit[]) : Unit is Adj+Ctl {
    // If I apply BinaryFractionQuantumInPlace to the entire register, I get:
    //1/sqrt(2)(|0> + e^(2*pi*0.j1j2...jn)|1>) x |j2...jn>
    let N = Length(register);
    for i in 0 .. N - 1 {
        BinaryFractionQuantumInPlace (register[i..N-1]);
    }

    ReverseRegister(register);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.7.-Quantum-Fourier-transform).*

### Task 1.8. Inverse QFT

**Input:**

 A register of $n$ qubits in state $|j_1 j_2 ... j_n \rangle$.

**Goal:**

Apply inverse QFT to the input register, i.e., transform it to a state
$\frac{1}{\sqrt{2^{n}}} \sum_{k=0}^{2^n-1} e^{-2\pi i \cdot \frac{jk}{2^{n}}} |k\rangle$.

  Need a hint? Click here
Inverse QFT is literally the inverse transformation of QFT.
Do you know a quantum way to express this?

```qsharp
%kata T18_InverseQFT

operation InverseQFT (register : Qubit[]) : Unit is Adj+Ctl {
    Adjoint QuantumFourierTransform(register);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-1.8.-Inverse-QFT).*

## Part II. Using the Quantum Fourier Transform

This section offers you tasks on state preparation and state analysis
that can be solved using QFT (or inverse QFT). It is possible to solve them
without QFT, but we recommend that you to try and come up with a QFT-based solution.

### Task 2.1. Prepare an equal superposition of all basis states

**Input:**

A register of $n$ qubits in state $|0...0\rangle$.

**Goal:**

Prepare an equal superposition of all basis vectors from $|0...0\rangle$ to $|1...1\rangle$
(i.e., state $\frac{1}{\sqrt{2^{n}}} \big(|0...0\rangle + ... + |1...1\rangle\big)$.

```qsharp
%kata T21_PrepareEqualSuperposition

operation PrepareEqualSuperposition (register : Qubit[]) : Unit is Adj+Ctl {
    ApplyToEachCA(H, register);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.1.-Prepare-an-equal-superposition-of-all-basis-states).*

### Task 2.2. Prepare a periodic state

**Inputs:**

  1. A register of $n$ qubits in state $|0...0\rangle$.

  2. An integer frequency F ($0 \leq F \leq 2^{n}-1$).

**Goal:**

Prepare a periodic state with frequency F on this register:

$$\frac{1}{\sqrt{2^{n}}} \sum_k e^{2\pi i \cdot \frac{Fk}{2^{n}}} |k\rangle$$

> For example, for $n = 2$ and $F = 1$ the goal state is $\frac{1}{2}\big(|0\rangle + i|1\rangle - |2\rangle - i|3\rangle\big)$.

> If you're using `DumpMachine` to debug your solution,
remember that this kata uses big endian encoding of states,
while `DumpMachine` uses little endian encoding.
You can use [`%config` magic command](https://docs.microsoft.com/en-us/qsharp/api/iqsharp-magic/config)
to reconfigure `DumpMachine` to use big endian encoding or bit strings.

  Need a hint? Click here
Which basis state can be mapped to this state using QFT?

```qsharp
%kata T22_PreparePeriodicState

open Microsoft.Quantum.Arithmetic;
open Microsoft.Quantum.Arrays;
open Microsoft.Quantum.Convert;

operation PreparePeriodicState (register : Qubit[], F : Int) : Unit is Adj+Ctl {
    // F = 1 = 01 (or 10?)
    // e * (i * theta) = cos(theta) + i * sin(theta)
    // e * ((i * pi)/2) = i
    // so, 0, pi/2, pi, 3pi/2
    // e^((2pi*1*0)/4) + e^((2pi*1*1)/4) + e^((2pi*1*2)/4) + e^((2pi*1*3)/4)
    // e^0 + e^(pi/2) + e^pi + e^(3pi/2)
    let bitsBE = Reversed(IntAsBoolArray(F, Length(register)));
    ApplyPauliFromBitString(PauliX, true, bitsBE, register); // This converts the register to a representation of F.

    //QFT(BigEndian(register));
    QuantumFourierTransform(register);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.2.-Prepare-a-periodic-state).*

### Task 2.3. Prepare a periodic state with alternating $1$ and $-1$ amplitudes

**Input:**

A register of $n$ qubits in state $|0...0\rangle$.

**Goal:**

Prepare a periodic state with alternating $1$ and $-1$ amplitudes of basis states:

$$\frac{1}{\sqrt{2^{n}}} \big(|0\rangle - |1\rangle + |2\rangle - |3\rangle + ... - |2^{n}-1\rangle\big)$$

> For example, for $n = 2$ the goal state is $\frac{1}{2} \big(|0\rangle - |1\rangle + |2\rangle - |3\rangle\big)$.

  Need a hint? Click here
Which basis state can be mapped to this state using QFT?  Which frequency would allow you to use task 2.2 here?

```qsharp
%kata T23_PrepareAlternatingState

operation PrepareAlternatingState (register : Qubit[]) : Unit is Adj+Ctl {
    X(Head(register));

    QuantumFourierTransform(register);
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.3.-Prepare-a-periodic-state-with-alternating-$1$-and-$-1$-amplitudes).*

### Task 2.4. Prepare an equal superposition of all even basis states

**Input:**

A register of $n$ qubits in state $|0...0\rangle$.

**Goal:**

Prepare an equal superposition of all even basis vectors:
$\frac{1}{\sqrt{2^{n-1}}} \big(|0\rangle + |2\rangle + ... + |2^{n}-2\rangle\big)$.

  Need a hint? Click here
Which superposition of two basis states can be mapped to this state using QFT?
    Use the solutions to tasks 2.1 and 2.3 to figure out the answer.

```qsharp
%kata T24_PrepareEqualSuperpositionOfEvenStates

operation PrepareEqualSuperpositionOfEvenStates (register : Qubit[]) : Unit is Adj+Ctl {
    H(Head(register));
    QFT(BigEndian(register));
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.4.-Prepare-an-equal-superposition-of-all-even-basis-states).*

### Task 2.5. Prepare a square-wave signal

**Input:**

A register of $n\geq2$ qubits in state $|0...0\rangle$.

**Goal:**

Prepare a periodic state with alternating $1, 1, -1, -1$ amplitudes of basis states:
$$\frac{1}{\sqrt{2^{n}}} \big(|0\rangle + |1\rangle - |2\rangle - |3\rangle + ... - |2^{n}-2\rangle - |2^{n}-1\rangle\big)$$

  Need a hint? Click here
Which superposition of two basis states can be mapped to this state using QFT?
    Remember that sum of two complex amplitudes can be a real number if their imaginary parts cancel out.

```qsharp
%kata T25_PrepareSquareWaveSignal

operation PrepareSquareWaveSignal (register : Qubit[]) : Unit is Adj+Ctl {
    X(register[1]);
    // |010...0⟩
    H(register[0]);
    // |010...0⟩ + |110...0⟩
    T(register[0]);
    within { X(register[0]); }
    apply { Adjoint T(register[0]); }

    QFT(BigEndian(register));
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.5.-Prepare-a-square-wave-signal).*

### Task 2.6. Get the frequency of a signal

**Input:**

A register of $n\geq2$ qubits in state
$\frac{1}{\sqrt{2^{n}}} \sum_k e^{2\pi i \cdot \frac{Fk}{2^{n}}} |k\rangle$, $0\leq F\leq 2^{n}-1$.

**Goal:**

Return the frequency F of the "signal" encoded in this state.

```qsharp
%kata T26_Frequency

open Microsoft.Quantum.Arithmetic;
open Microsoft.Quantum.Arrays;
open Microsoft.Quantum.Measurement;

operation Frequency (register : Qubit[]) : Int {
    Adjoint QFT(BigEndian(register));
    let bitsBE = MultiM(register);
    return ResultArrayAsInt(Reversed(bitsBE));
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-2.6.-Get-the-frequency-of-a-signal).*

## Part III. Powers and roots of the QFT

### Task 3.1 Implement powers of the QFT

**Inputs:**

  1. A register of $n$ qubits in an arbitrary state.

  2. An integer $P$ ($ 0 \leq P \leq 2^{20} - 1 $).

**Goal:**

Transform state $|x\rangle$ into state $ QFT^{P} |x\rangle$, where $QFT$ is the quantum Fourier transform implemented in part I.

**Note:**

Your solution has to run fast for any $P$ in the given range!

```qsharp
%kata T31_QFTPower

operation QFTPower (P : Int, inputRegister : Qubit[]) : Unit is Adj+Ctl {
    for _ in 1 .. (P % 4) {
        QFT(BigEndian(inputRegister));
    }
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-3.1-Implement-powers-of-the-QFT).*

### Task 3.2. Implement roots of the QFT
**Inputs:**

  1. A register of $n$ qubits in an arbitrary state.

  2. An integer $P$ ($ 2 \leq P \leq 8 $).

**Goal:**

Transform state $|x\rangle$ into state $V |x\rangle$, where $V^{P} = QFT$. In other words, implement a $P^{th}$ root of the $QFT$. You can implement the required unitary up to a global phase.

```qsharp
operation Circ (qs : LittleEndian, alpha : Double) : Unit is Adj + Ctl {
    within {
        Adjoint QFTLE(qs);
    } apply {
        ApplyDiagonalUnitary(
            [0.0, -alpha, -2.0 * alpha, alpha],
            qs
        );
    }
}
```

Circ

```qsharp
%kata T32_QFTRoot

open Microsoft.Quantum.Arithmetic;
open Microsoft.Quantum.Convert;
open Microsoft.Quantum.Math;

operation QFTRoot (P : Int, inputRegister : Qubit[]) : Unit is Adj + Ctl {
    use aux = Qubit[2];
    let Q = QFT;
    let Q2 = OperationPowCA(Q, 2);
    within {
        ApplyToEachCA(H, aux);
        Controlled Adjoint Q([aux[0]], BigEndian(inputRegister));
        Controlled Adjoint Q2([aux[1]], BigEndian(inputRegister));
    } apply {
        Circ(LittleEndian(aux), PI() / (2.0 * IntAsDouble(P)));
    }
}
```

    Success!

*Can't come up with a solution? See the explained solution in the [QFT Workbook](./Workbook_QFT.ipynb#Task-3.2.-Implement-roots-of-the-QFT).*


# Interior Point Methods

## Introduction

In the 1980s it was discovered that many large linear programs could
be solved efficiently by using formulations and algorithms from nonlinear
programming. One characteristic of these methods was that they required
all iterates to satisfy the inequality constraints in the problem
strictly, so they became known as _interior-point methods_. In
the 1990s, a subclass of interior-point methods known as _primal-dual
methods_ had distinguished themselves as the most efficient practical
approaches, and proved to be strong competitors to the _simplex
method_ on large problems. Recently, it has been shown that
interior-point methods are as successful for nonlinear optimization
as for linear programming. General primal-dual interior point methods
are the focus of this chapter

## Interior-point Quadratic Programming

For simplicity, we restrict our attention to convex quadratic programs,
which we write as follows:
\begin{align*}
\min_{\vec{x}}\, & f\left(\vec{x}\right)\overset{\vartriangle}{=}\frac{1}{2} \vec{x}^\mathsf{T}Q\vec{x}- \vec{c}^\mathsf{T}\vec{x}\\
\textrm{subject to}\, & \begin{cases}
A_{\textrm{eq}}\vec{x}=\vec{b}_{\textrm{eq}}\,,\\
A_{\textrm{in}}\vec{x}\leq\vec{b}_{\textrm{in}}\,,
\end{cases}
\end{align*}
where $Q$ is a symmetric and positive semidefinite $n\times n$ matrix,
$\vec{c}\in\mathbb{R}^{n}$, $A_{\textrm{eq}}$ is a $m\times n$ matrix,
$\vec{b}_{\textrm{eq}}\in\mathbb{R}^{m}$, $A_{\textrm{in}}$ is a $p\times n$
matrix and $\vec{b}_{\textrm{in}}\in\mathbb{R}^{p}$. Rewriting the KKT
conditions in this notation, we obtain
\begin{align*}
\vec{\mu} & \geq\vec{0}\,,\\
Q\vec{x}+ A_{\textrm{eq}}^\mathsf{T}\vec{\lambda}+ A_{\textrm{in}}^\mathsf{T}\vec{\mu}-\vec{c} & =\vec{0}\,,\\
\left(A_{\textrm{in}}\vec{x}-\vec{b}_{\textrm{in}}\right)_{j}\mu_{j} & =0\,,\quad j=1,\dots,p\,,\\
A_{\textrm{eq}}\vec{x} & =\vec{b}_{\textrm{eq}}\,,\\
A_{\textrm{in}}\vec{x} & \leq\vec{b}_{\textrm{in}}\,.
\end{align*}
By introducing the slack vector $\vec{y}\geq\vec{0}$, we can rewrite
these conditions as
\begin{align*}
\left(\vec{\mu},\vec{y}\right) & \geq\vec{0}\,,\\
Q\vec{x}+ A_{\textrm{eq}}^\mathsf{T}\vec{\lambda}+ A_{\textrm{in}}^\mathsf{T}\vec{\mu}-\vec{c} & =\vec{0}\,,\\
y_{j}\mu_{j} & =0\,,\quad j=1,\dots,p\,,\\
A_{\textrm{eq}}\vec{x}-\vec{b}_{\textrm{eq}} & =\vec{0}\,,\\
A_{\textrm{in}}\vec{x}-\vec{b}_{\textrm{in}}+\vec{y} & =\vec{0}\,.
\end{align*}
Since we assume that $Q$ is positive semidefinite, these KKT conditions
are not only necessary but also sufficient, so we can solve the convex
quadratic program by finding solutions of this system.

Primal-dual methods generate iterates that satisfy the bounds strictly; that is, $\vec{y}>0$
and $\vec{\mu}>0$. This property is the origin of the term interior-point.
By respecting these bounds, the methods avoid spurious solutions,
points that satisfy the system but not the bounds. Spurious solutions
abound, and do not provide useful information about real solutions,
so it makes sense to exclude them altogether. Given a current iterate
$\left(\vec{x}^{\left(k\right)},\vec{y}^{\left(k\right)},\vec{\lambda}^{\left(k\right)},\vec{\mu}^{\left(k\right)}\right)$
that satisfies $\left(\vec{\mu}^{\left(k\right)},\vec{y}^{\left(k\right)}\right)>0$,
we can define a _complementary measure_
\begin{equation}
\nu_{k}=\frac{ \left(\vec{y}^{\left(k\right)}\right)^\mathsf{T}\vec{\mu}^{\left(k\right)}}{p}\,.
\end{equation}
This measure gives an indication of the desirability of the couple
$\left(\vec{\mu}^{\left(k\right)},\vec{y}^{\left(k\right)}\right)$.

We derive a path-following, primal-dual method by considering the
perturbed KKT conditions by
\begin{equation}
\vec{F}\left(\vec{x}^{\left(k+1\right)},\vec{y}^{\left(k+1\right)},\vec{\lambda}^{\left(k+1\right)},\vec{\mu}^{\left(k+1\right)},\sigma_{k}\nu_{k}\right)=\begin{pmatrix}Q\vec{x}^{\left(k+1\right)}+ A_{\textrm{eq}}^\mathsf{T}\vec{\lambda}^{\left(k+1\right)}+ A_{\textrm{in}}^\mathsf{T}\vec{\mu}^{\left(k+1\right)}-\vec{c}\\
Y_{k+1}M_{k+1}\vec{1}-\sigma_{k}\nu_{k}\vec{1}\\
A_{\textrm{eq}}\vec{x}^{\left(k+1\right)}-\vec{b}_{\textrm{eq}}\\
A_{\textrm{in}}\vec{x}^{\left(k+1\right)}-\vec{b}_{\textrm{in}}+\vec{y}^{\left(k+1\right)}
\end{pmatrix}=\vec{0}\,,
\end{equation}
where
\begin{equation}
Y_{k+1}=\begin{pmatrix}y_{1}^{\left(k+1\right)} & 0 & \cdots & 0\\
0 & y_{2}^{\left(k+1\right)} & \ddots & 0\\
\vdots & \ddots & \ddots & 0\\
0 & 0 & 0 & y_{p}^{\left(k+1\right)}
\end{pmatrix}\,,\quad M_{k+1}=\begin{pmatrix}\mu_{1}^{\left(k+1\right)} & 0 & \cdots & 0\\
0 & \mu_{2}^{\left(k+1\right)} & \ddots & 0\\
\vdots & \ddots & \ddots & 0\\
0 & 0 & 0 & \mu_{p}^{\left(k+1\right)}
\end{pmatrix}\,,
\end{equation}
and $\sigma\in\left[0,1\right]$ is the reduction factor that we wish
to achieve in the complementary measure on one step. We call $\sigma$
the \emph{centering parameter}. The solution of this system for all
positive values of $\sigma$ and $\nu$ define the \emph{central path},
which is a trajectory that leads to the solution of the quadratic
program as $\sigma\nu$ tends to zero.

By fixing $\sigma_{k}$ and applying Newton's method to the system,
we obtain the linear system
\begin{equation}
\begin{pmatrix}Q & 0 &  A_{\textrm{eq}}^\mathsf{T} &  A_{\textrm{in}}^\mathsf{T}\\
0 & M_{k} & 0 & Y_{k}\\
A_{\textrm{eq}} & 0 & 0 & 0\\
A_{\textrm{in}} & I & 0 & 0
\end{pmatrix}\begin{pmatrix}\vec{d}_{\vec{x}}^{\left(k\right)}\\
\vec{d}_{\vec{y}}^{\left(k\right)}\\
\vec{d}_{\vec{\lambda}}^{\left(k\right)}\\
\vec{d}_{\vec{\mu}}^{\left(k\right)}
\end{pmatrix}=-\begin{pmatrix}Q\vec{x}^{\left(k\right)}+ A_{\textrm{eq}}^\mathsf{T}\vec{\lambda}^{\left(k\right)}+ A_{\textrm{in}}^\mathsf{T}\vec{\mu}^{\left(k\right)}-\vec{c}\\
Y_{k}M_{k}\vec{1}-\sigma_{k}\nu_{k}\vec{1}\\
A_{\textrm{eq}}\vec{x}^{\left(k\right)}-\vec{b}_{\textrm{eq}}\\
A_{\textrm{in}}\vec{x}^{\left(k\right)}-\vec{b}_{\textrm{in}}+\vec{y}^{\left(k\right)}
\end{pmatrix}\,.
\end{equation}
We obtain the next iterate by setting
\begin{equation}
\begin{pmatrix}\vec{x}^{\left(k+1\right)}\\
\vec{y}^{\left(k+1\right)}\\
\vec{\lambda}^{\left(k+1\right)}\\
\vec{\mu}^{\left(k+1\right)}
\end{pmatrix}=\begin{pmatrix}\vec{x}^{\left(k\right)}\\
\vec{y}^{\left(k\right)}\\
\vec{\lambda}^{\left(k\right)}\\
\vec{\mu}^{\left(k\right)}
\end{pmatrix}+\alpha_{k}\begin{pmatrix}\vec{d}_{\vec{x}}^{\left(k\right)}\\
\vec{d}_{\vec{y}}^{\left(k\right)}\\
\vec{d}_{\vec{\lambda}}^{\left(k\right)}\\
\vec{d}_{\vec{\mu}}^{\left(k\right)}
\end{pmatrix}\,,
\end{equation}
where $\alpha_{k}$ is chosen to retain the bounds $\left(\vec{\mu}^{\left(k+1\right)},\vec{y}^{\left(k+1\right)}\right)>0$
and possibly to satisfy various other conditions.

The choices of centering parameter $\sigma_{k}$ and step-length $\alpha_{k}$
are crucial for the performance of the method. Techniques for controlling
these parameters, directly and indirectly, give rise to a wide variety
of methods with diverse properties. One option is to use equal step
length for the primal and dual updates, and to set $\alpha_{k}=\min\left\{ \alpha_{k}^{\textrm{pri}},\alpha_{k}^{\textrm{dual}}\right\} $,
where
\begin{align}
\alpha_{k}^{\textrm{pri}} & =\max\left\{ \alpha\in\left\{ 0,1\right\} :\vec{y}^{\left(k\right)}+\alpha\vec{d}_{\vec{y}}^{\left(k\right)}\geq\left(1-\tau\right)\vec{y}^{\left(k\right)}\right\} \,,\label{eq:alpha_pri_max}\\
\alpha_{k}^{\textrm{dual}} & =\max\left\{ \alpha\in\left\{ 0,1\right\} :\vec{\mu}^{\left(k\right)}+\alpha\vec{d}_{\vec{\mu}}^{\left(k\right)}\geq\left(1-\tau\right)\vec{\mu}^{\left(k\right)}\right\} \,,\label{eq:alpha_dual_max}
\end{align}
the parameter $\tau\in\left]0,1\right[$ controls how far we back
off from the maximum step for which the conditions $\vec{y}^{\left(k\right)}+\alpha\vec{d}_{\vec{y}}^{\left(k\right)}\geq\vec{0}$
and $\vec{\mu}^{\left(k\right)}+\alpha\vec{d}_{\vec{\mu}}^{\left(k\right)}\geq\vec{0}$
are satisfied. A typical value of $\tau=0.995$ and we can choose
$\tau_{k}$ to approach $1$ as the iterates approach the solution,
to accelerate the convergence.

The most popular interior-point method for convex QP is based on Mehrotra's
predictor-corrector. First we compute an affine scaling step $\left(\vec{d}_{\vec{x},\textrm{aff}},\vec{d}_{\vec{y},\textrm{aff}},\vec{d}_{\vec{\lambda},\textrm{aff}},\vec{d}_{\vec{\mu},\textrm{aff}}\right)$
by setting $\sigma_{k}=0$. We improve upon this step by computing
a corrector step. Next, we compute the centering parameter $\sigma_{k}$
using following heuristic
\begin{equation}
\sigma_{k}=\left(\frac{\nu_{\textrm{aff}}}{\nu_{k}}\right)^{3}\,,
\end{equation}
where $\nu_{\textrm{aff}}=\frac{ \left(\vec{y}_{\textrm{aff}}\right)^\mathsf{T}\left(\vec{\mu}_{\textrm{aff}}\right)}{p}$.
The total step is obtained by solving the following system
\begin{equation}
\begin{pmatrix}Q & 0 &  A_{\textrm{eq}}^\mathsf{T} &  A_{\textrm{in}}^\mathsf{T}\\
0 & M_{k} & 0 & Y_{k}\\
A_{\textrm{eq}} & 0 & 0 & 0\\
A_{\textrm{in}} & I & 0 & 0
\end{pmatrix}\begin{pmatrix}\vec{d}_{\vec{x}}^{\left(k\right)}\\
\vec{d}_{\vec{y}}^{\left(k\right)}\\
\vec{d}_{\vec{\lambda}}^{\left(k\right)}\\
\vec{d}_{\vec{\mu}}^{\left(k\right)}
\end{pmatrix}=-\begin{pmatrix}Q\vec{x}^{\left(k\right)}+ A_{\textrm{eq}}^\mathsf{T}\vec{\lambda}^{\left(k\right)}+ A_{\textrm{in}}^\mathsf{T}\vec{\mu}^{\left(k\right)}-\vec{c}\\
Y_{k}M_{k}\vec{1}+\Delta Y_{\textrm{aff}}\Delta M_{\textrm{aff}}\vec{1}-\sigma_{k}\nu_{k}\vec{1}\\
A_{\textrm{eq}}\vec{x}^{\left(k\right)}-\vec{b}_{\textrm{eq}}\\
A_{\textrm{in}}\vec{x}^{\left(k\right)}-\vec{b}_{\textrm{in}}+\vec{y}^{\left(k\right)}
\end{pmatrix}\,,
\end{equation}
where
\begin{equation}
\Delta Y_{\textrm{aff}}=Y_{\textrm{aff}}-Y_{k}\,,\quad\Delta M_{\textrm{aff}}=M_{\textrm{aff}}-M_{k}\,.
\end{equation}

## Julia

The package `Ipopt` integrated in `JuMP` solves a quadratic problem with an interior point method.

```julia
#using Pkg
#pkg"add Ipopt"
using Ipopt
using JuMP

model = Model(with_optimizer(Ipopt.Optimizer))
@variable(model, 0 <= x, start=2)
@variable(model, 0 <= y, start=0)
@NLobjective(model, Min, (x-1)^2+(y-2.5)^2)
@constraint(model, con1, -x+2y <= 2)
@constraint(model, con2,  x+2y <= 6)
@constraint(model, con3,  x-2y <= 2)
optimize!(model)
println("x = ", value(x), " y = ", value(y))
```

## General interior-point method

We consider the general constrained problem
\begin{align*}
\min\, & f\left(\vec{x}\right)\\
\textrm{subject to} & \begin{cases}
\vec{h}\left(\vec{x}\right)=\vec{0}\\
\vec{g}\left(\vec{x}\right)\leq\vec{0}
\end{cases}
\end{align*}
where $f: \mathbb{R}^{n}\rightarrow \mathbb{R}$, $\vec{h}: \mathbb{R}^{n}\rightarrow \mathbb{R}^{m}$,
$m\leq n$, and $\vec{g}: \mathbb{R}^{n}\rightarrow \mathbb{R}^{p}$. The general
interior-point method can be viewed as a direct extension of interior-point
methods for quadratic programming.

The KKT conditions for the nonlinear optimization problem can be written
as
\begin{align*}
\left(\vec{\mu},\vec{y}\right) & \geq\vec{0}\,,\\
\nabla f\left(\vec{x}\right)+ \mathsf{J}^\mathsf{T}\vec{h}\left(\vec{x}\right)\vec{\lambda}+ \mathsf{J}^\mathsf{T}\vec{g}\left(\vec{x}\right)\vec{\mu} & =\vec{0}\,,\\
YM\vec{1} & -\sigma\nu\vec{1}=0\,,\\
\vec{h}\left(\vec{x}\right) & =\vec{0}\,,\\
\vec{g}\left(\vec{x}\right)+\vec{y} & =\vec{0}\,,
\end{align*}
with $\nu=0$. This first and third equation from the system introduce
into the problem the combinatorial aspects of determining the optimal
active set. We circumvent this difficulty be letting $\nu$ be strict
positive, thus forcing the variables $\vec{\mu}$ and $\vec{y}$ to
take positive values. We solve approximately the perturbed KKT conditions
for a sequence of positive parameters $\left\{ \sigma_{k}\nu_{k}\right\} $
that converges to zero, while maintaining $\left(\vec{\mu}^{\left(k\right)},\vec{y}^{\left(k\right)}\right)\geq\vec{0}$.

Applying Newton's method to the nonlinear system, we obtain
\begin{equation}
\begin{bmatrix}\mathsf{H} \mathcal{L}\left(\vec{x}^{\left(k\right)},\vec{y}^{\left(k\right)},\vec{\lambda}^{\left(k\right)},\vec{\mu}^{\left(k\right)}\right) & 0 &  \mathsf{J}^\mathsf{T}\vec{h}\left(\vec{x}^{\left(k\right)}\right) &  \mathsf{J}^\mathsf{T}\vec{g}\left(\vec{x}^{\left(k\right)}\right)\\
0 & M_{k} & 0 & Y_{k}\\
\mathsf{J}\vec{h}\left(\vec{x}^{\left(k\right)}\right) & 0 & 0 & 0\\
\mathsf{J}\vec{g}\left(\vec{x}^{\left(k\right)}\right) & I & 0 & 0
\end{bmatrix}\begin{bmatrix}\vec{d}_{\vec{x}}^{\left(k\right)}\\
\vec{d}_{\vec{y}}^{\left(k\right)}\\
\vec{d}_{\vec{\lambda}}^{\left(k\right)}\\
\vec{d}_{\vec{\mu}}^{\left(k\right)}
\end{bmatrix}=-\begin{bmatrix}\nabla f\left(\vec{x}^{\left(k\right)}\right)+ \mathsf{J}^\mathsf{T}\vec{h}\left(\vec{x}^{\left(k\right)}\right)\vec{\lambda}^{\left(k\right)}+ \mathsf{J}^\mathsf{T}\vec{g}\left(\vec{x}^{\left(k\right)}\right)\vec{\mu}^{\left(k\right)}\\
Y_{k}M_{k}-\sigma_{k}\nu_{k}\vec{1}\\
\vec{h}\left(\vec{x}^{\left(k\right)}\right)\\
\vec{g}\left(\vec{x}^{\left(k\right)}\right)+\vec{y}^{\left(k\right)}
\end{bmatrix}\,.
 \end{equation}
Numerical experience has shown, however, that using different step-lengths
in the primal and dual variables often leads to better convergence.
After the step $k$ has been computed and the maximum step lengths have been
determined, we perform a backtracking line search that computes the
step-lengths.
 \begin{equation}
\alpha_{k,\textrm{back}}^{\textrm{pri}}\in\left]0,\alpha_{k}^{\textrm{pri}}\right]\,,\quad\alpha_{k,\textrm{back}}^{\textrm{dual}}\in\left]0,\alpha_{k}^{\textrm{dual}}\right]\,,
 \end{equation}
providing sufficient decrease.

The centering parameter $\sigma_{k}$ can be calculated by a predictor-corrector
strategy as in the quadratic programming case.

```julia
using Ipopt
using JuMP

m = Model(with_optimizer(Ipopt.Optimizer))

a1 = 2
b1 = 0
a2 = -1
b2 = 1

@variable(m, x1, start=1.234)
@variable(m, x2 >= 0, start=5.678)

@NLobjective(m, Min, sqrt(x2))
@NLconstraint(m, x2 >= (a1*x1+b1)^3)
@NLconstraint(m, x2 >= (a2*x1+b2)^3)

optimize!(m)
println("x1 = ", value(x1), " x2 = ", value(x2))
```

```julia

```


```python
%matplotlib inline
```

#  Doing Math with Python

 Amit Saha

May 29, PyCon US 2016 Education Summit

Portland, Oregon

## About me

- Software Engineer at [Freelancer.com](https://www.freelancer.com) HQ in Sydney, Australia

- Author of "Doing Math with Python" (No Starch Press, 2015)

- Writes for Linux Voice, Linux Journal, etc.

- [Blog](http://echorand.me), [GitHub](http://github.com/amitsaha)

#### Contact

- [@echorand](http://twitter.com/echorand)

- [Email](mailto:amitsaha.in@gmail.com)

### This talk - a proposal, a hypothesis, a statement

*Python can lead to a more enriching learning and teaching experience in the classroom*

```python
As I will attempt to describe in the next slides, Python is an amazing way to lead to a more fun learning and teaching
experience.

It can be a basic calculator, a fancy calculator and

Math, Science, Geography..

Tools that will help us in that quest are:
```

###  (Main) Tools

### Python - a scientific calculator

- Python 3 is my favorite calculator (not Python 2 because 1/2 = 0)

- `fabs()`, `abs()`, `sin()`, `cos()`, `gcd()`, `log()` (See [math](https://docs.python.org/3/library/math.html))

- Descriptive statistics (See [statistics](https://docs.python.org/3/library/statistics.html#module-statistics))

### Python - a scientific calculator

- Develop your own functions: unit conversion, finding correlation, .., anything really

- Use PYTHONSTARTUP to extend the battery of readily available mathematical functions

```
$ PYTHONSTARTUP=~/work/dmwp/pycon-us-2016/startup_math.py idle3 -s
```

### Unit conversion functions

```
>>> unit_conversion()
1. Kilometers to Miles
2. Miles to Kilometers
3. Kilograms to Pounds
4. Pounds to Kilograms
5. Celsius to Fahrenheit
6. Fahrenheit to Celsius
Which conversion would you like to do? 6
Enter temperature in fahrenheit: 98
Temperature in celsius: 36.66666666666667
>>>

```

### Finding linear correlation

```
>>>
>>> x = [1, 2, 3, 4]
>>> y = [2, 4, 6.1, 7.9]
>>> find_corr_x_y(x, y)
0.9995411791453812

```

### Python - a really fancy calculator

SymPy - a pure Python symbolic math library

*from sympy import awesomeness*  - don't try that :)

```python
When you bring in SymPy to the picture, things really  get awesome. You are suddenly writing computer
programs which are capable of speaking algebra. You are no more limited to numbers.

```

```python
# Create graphs from algebraic expressions

from sympy import Symbol, plot
x = Symbol('x')
p = plot(2*x**2 + 2*x + 2)

```

```python
# Solve equations

from sympy import solve, Symbol
x = Symbol('x')
solve(2*x + 1)

```

    [-1/2]

```python
# Limits

from sympy import Symbol, Limit, sin
x = Symbol('x')
Limit(sin(x)/x, x, 0).doit()

```

    1

```python
# Derivative

from sympy import Symbol, Derivative, sin, init_printing
x = Symbol('x')
init_printing()
Derivative(sin(x)**(2*x+1), x).doit()
```

```python
# Indefinite integral

from sympy import Symbol, Integral, sqrt, sin, init_printing
x = Symbol('x')
init_printing()
Integral(sqrt(x)).doit()
```

```python
# Definite integral

from sympy import Symbol, Integral, sqrt
x = Symbol('x')
Integral(sqrt(x), (x, 0, 2)).doit()
```

### Python - Making other subjects more lively

- matplotlib

- basemap

- Interactive Jupyter Notebooks

#### Bringing Science to life

*Animation of a Projectile motion*

#### Drawing fractals

*Interactively drawing a Barnsley Fern*

#### The world is your graph paper

*Showing places on a digital map*

### Great base for the future

*Statistics and Graphing data* -> *Data Science*

*Differential Calculus* -> *Machine learning*

### Application of differentiation

Use gradient descent to find a function's minimum value

### Predict the college admission score based on high school math score

Use gradient descent as the optimizer for single variable linear regression model

```python
### TODO: digit recognition using Neural networks

```

```python
### Scikitlearn, pandas, scipy, statsmodel

```

### Book: Doing Math With Python

Published by No Starch Press, out in 2015.

Early feedback very encouraging

#### Comments

> Saha does an excellent job providing a clear link between Python and upper-level math concepts, and demonstrates how Python can be transformed into a mathematical stage.

> This book is highly recommended for the high school or college student and anyone who is looking for a more natural way of programming math and scientific functions

> As a teacher I highly recommend this book as a way to work with someone in learning both math and programming

### Links

- [Doing Math with Python](http://nostarch.com/doingmathwithpython)

- [Doing Math with Python Blog](doingmathwithpython.github.io)

- [Upcoming O'Reilly Webcast](http://www.oreilly.com/pub/e/3712)

### PyCon Special!

*Use PYCONMATH code to get 30% off from No Starch Press*

(Valid from  May 26th - June 8th)

Book Signing - May 31st - 2.00 PM - No Starch Press booth

### Dialogue

Questions, Thoughts, comments, discussions?

Online: @echorand, amitsaha.in@gmail.com

### Acknowledgements

PyCon US Education Summit team for inviting me

Thanks to PyCon US for reduced registration rates

Massive thanks to my employer, Freelancer.com for sponsoring my travel and stay


```python
# Just importing stuff and setting-up matplolib
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import loadmat
import matplotlib as mpl
from scipy.optimize import minimize
mpl.rcParams["xtick.direction"] = "in"
mpl.rcParams["ytick.direction"] = "in"
mpl.rcParams["lines.markeredgecolor"] = "k"
mpl.rcParams["lines.markeredgewidth"] = 1.5
mpl.rcParams["figure.dpi"] = 200
from matplotlib import rc
rc('font', family='serif')
rc('text', usetex=True)
rc('xtick', labelsize='medium')
rc('ytick', labelsize='medium')
rc("axes", labelsize = "large")
def cm2inch(value):
    return value/2.54
def cm2inch(value):
    return value/2.54
def gauss_function(x, a, x0, sigma):
    return a*np.exp(-(x-x0)**2/(2*sigma**2))
def pdf(data, bins = 10, density = True):

    pdf, bins_edge = np.histogram(data, bins = bins, density = density)
    bins_center = (bins_edge[0:-1] + bins_edge[1:]) / 2

    return pdf, bins_center
```

# Experimental force measurement

In this notebook, we are gonna compute the force felt by our particle, experimentaly using the data of the preprint.

In the first part we will measure the force using the experimentaly measured potential.

On the second part we will use the local drifts and check if everything match.

```python
# Define logarythmic histrogram in order to have more points near the surface.
def logarithmic_hist(data,begin,stop,num = 50,base = 2):

    if begin == 0:
        beg = stop/num
        bins = np.logspace(np.log(beg)/np.log(base), np.log(stop)/np.log(base), num-1, base=base)
        widths = (bins[1:] - bins[:-1])
        bins = np.cumsum(widths[::-1])
        bins = np.concatenate(([0],bins))
        widths = (bins[1:] - bins[:-1])

    else:
        bins = np.logspace(np.log(begin)/np.log(base), np.log(stop)/np.log(base), num, base=base)
        widths = (bins[1:] - bins[:-1])

    hist,bins = np.histogram(data, bins=bins,density=True)
    # normalize by bin width
    bins_center = (bins[1:] + bins[:-1])/2

    return bins_center,widths, hist
```

Data loading

```python
raw_data = loadmat("data_graphs.mat")
z_exp = np.squeeze(raw_data["z"]) *1e-6
z_exp = z_exp - np.min(z_exp)
```

```python
# We compute the PDF
z_hist, _, P_z= logarithmic_hist(z_exp, 39e-9, 2e-6, num=50)
P_z = P_z / np.trapz(P_z, z_hist)
```

```python
# Just a plot of the PDF
plt.semilogy(z_hist, P_z, "o")
plt.xlabel("$z$ (m)")
plt.ylabel("$P_{eq}(z)$")
```

To compute de force we then avec to take the derivative of $\mathrm{log}(P_{eq})$

```python
F_eq =  np.gradient(np.log(P_z), z_hist) * 4e-21
#F_eq = (P_z[1:] - P_z[:-1]) / (z_hist[1:] - z_hist[:-1])
```

We want now to compare with the results of the force that can be computed from the result of the multifit, which gives us a good measurement of the following parameters.

B = 4.5
ld = 21 nm
lb = 532 nm

```python
B = 4.8
ld = 20.8e-9
lb = 532e-9

```

```python
z_F_th = np.linspace(20e-9,3e-6,10000)
# just a function to compute the theoritical force from the results of the multifit
f_th = lambda z:4e-21 * ( B/(ld) * np.exp(- z/(ld)) - 1/(lb))

F_th = f_th(z_F_th)

```

```python
plt.semilogx(z_hist, F_eq * 1e15, "s", label = "Force from E")
plt.plot(z_F_th, F_th * 1e15, color = "k", label = "multi-fit force")
plt.xlabel("$z$ (m)")
plt.ylabel("$F_{eq}$ (fN)")
plt.ylim((None,350))
plt.legend()
plt.show()
```

We see that there is an offset to take into account on $z$, to correctly take in into account we need to measure it correctly. in order to do it, i'll directly do a function that compute the $\chi ^2$ as a function of the offset that we apply on the trajectory.

As a reminder we do have this offset due to the way the holographic method we use. This method method mesure the distance between the imaging plane of the objective and the particle but not between the surface and the particle that we have to deduce form the data.

```python
def chi_2_force(offset):
    """ Function that permits to directly measure the chi 2 as a function as the offset on the trajectory"""
    offset = offset * 1e-9
    z = np.squeeze(raw_data["z"]) *1e-6
    z = z-np.min(z)
    z = z-offset
    z_hist, _, P_z= logarithmic_hist(z, 39e-9, 2e-6, num=20)
    F_eq =  np.gradient(np.log(P_z), z_hist) * 4e-21
    #return z_hist, P_z
    return np.mean((F_eq*1e15 - f_th(z_hist)*1e15) ** 2)

```

```python
z_chi2 = np.linspace(1, 200, 30)
chi_2 = [chi_2_force(i) for i in z_chi2]
```

```python
plt.plot(z_chi2, chi_2 / max(chi_2), "o")
plt.xlabel("$z$ (nm)")
plt.ylabel("$\chi ^2 / max(\chi^2) $")
```

Okay ! we see that we have a minimum around $62.5$ nanometers, let's have a better measurement with the minimize algorithm !

```python
minimize(chi_2_force, x0 = 62.5,  method='Nelder-Mead', tol=1e-40)
```

     final_simplex: (array([[61.95602417],
           [61.95602417]]), array([25.38935383, 25.38935383]))
               fun: 25.389353833953738
           message: 'Optimization terminated successfully.'
              nfev: 143
               nit: 52
            status: 0
           success: True
                 x: array([61.95602417])

we see that the offset sould be equal to $62 ~ \mathrm{nm}$. Giving the following force :

```python
offset = 62 * 1e-9
z = np.squeeze(raw_data["z"]) *1e-6
z = z-np.min(z)
z = z-offset
z_hist, _, P_z= logarithmic_hist(z, 39e-9, 2e-6, num=50)
F_eq =  np.gradient(np.log(P_z), z_hist) * 4e-21
```

```python
plt.semilogx(z_hist, F_eq * 1e15, "s", label = "$F_{exp}$ form E")
#plt.plot(z_F_th, F_th * 1e15, color = "k")
plt.plot(z_hist, f_th(z_hist) * 1e15, color = "k", label = "$F_{th}$")
plt.xlabel("$z$ (m)")
plt.ylabel("$F_{eq}$ (fN)")
plt.ylim((None,150))
plt.legend()
plt.show()
```

Now that we have verified that we corectly get the forces from the equilibrium probability density of height that permited us to measure the offset on the trajectory.

Now that everything is set up correctly we can try to measure the forces from the local drifts.

## Force measurement from the local drifts

In order to measure the forces from the local drifts we are gonna use some moving average magiK. This will permit us to have a quasi continuous measurement.

Bascicaly, the measurement is done in a few steps :

* Compute the volocity array `(x[1:] - x[0:-1])/dt `

* Sort the velocity and the height in the height order.

* Do a moving average on the velocity

* Compute the force : $F(z) = 6 \pi \eta(z) a \langle v \rangle $

* Apply the correction due to the Itö convention on the force $ \propto - \frac{dD}{D}$

```python
def movmean(x, w):
    """ Moving average function using convolution """
    return np.convolve(x, np.ones(w), 'same') / w

def movstd(x,w):
    mean = movmean(x,w)
    new_x = (x - mean)**2
    return np.sqrt(movmean(new_x, w))

def F_correction(alpha,z):
    """ Function to compute the correction on the force measurement due to the ito convention"""
    return - 4e-21 * alpha *(42*a*z*z + 24*a*a*z + 4*a*a*a)/(36*(z**4) + 66*a*(z**3) + 30*a*a*z*z + 4*(a**3)*z)
def eta(z):
    """visosity along the z axis -- utlity function to measure the forces"""
    result = ((6*z*z + 2*a*z) / (6*z*z + 9*a*z + 2*a*a))
    return 1/result
```

```python
z_drifts = np.squeeze(raw_data["z"]) *1e-6
z_drifts = z_drifts - np.min(z_drifts) - offset
```

```python
# On calcul les vitesses:
dt = 1/60 # 60 fps
v = (z_drifts[1:] - z_drifts[:-1]) / dt
z_drifts = z_drifts[:-1] # Convention Itö -> on attribue la vitesse a z_1
v = v[z_drifts.argsort()] # on tri les vitesse en fonction de la hauteur
z_drifts = np.sort(z_drifts) # on tri les hauteurs.

```

We can also compute the error on the force :

\begin{equation}
Err_f = \sqrt{24 \pi k_\mathrm{B} T \eta_z(z) a / \tau_{box} }
\end{equation}

with $\tau_{box} = w_{average} / \mathrm{fps}$, $w_{average}$ being the width of the moving average.

```python
# Let's do some moving average magic ! ;)
w = 1500
a = 1.516e-6
F_drifts = 6 * np.pi * 0.001 * eta(z_drifts) * a * movmean(v, w)
F_drifts = F_drifts + F_correction(1, z_drifts)
#F_err = np.sqrt(24 * np.pi * 4e-21 *0.001*  eta(z_drifts) * a / (w*dt))
F_exp_err = 6 * np.pi * 0.001 * eta(z_drifts) * a * movstd(v, w)
z_th = np.linspace(10e-9, 4e-6, 1000)

F_err_th = np.sqrt(24 * np.pi * 4e-21 *0.001*  eta(z_th) * a / (w*dt))
F_drift_th = f_th(z_th)
```

```python
plt.semilogx(z_hist, F_eq * 1e15, "s", label = "$F_{exp}$ form E", zorder = 10)
plt.plot(z_drifts, F_drifts * 1e15, color = "tab:cyan", label = "local drift force")
plt.semilogx(z_th, F_drift_th  * 1e15, color = "k", zorder = 11, label="multifit force")
#plt.fill_between(z_drifts, F_drifts * 1e15 - F_exp_err*1e15, F_drifts * 1e15 + F_exp_err*1e15, alpha = 0.5, color = "tab:purple", label="drift force error")
plt.ylim((-40, 300))
plt.xlim((10e-9,None))
plt.legend()
plt.xlabel("$z$ (m)")
plt.ylabel("$F_{eq}$ (fN)")
plt.show()
```

same plot but with fewer points

```python
plt.plot(z_hist, F_eq * 1e15, "s", label = "$local form U", zorder = 10)
plt.errorbar(z_drifts[::2000],F_drifts[::2000] * 1e15,linestyle="", marker="o", label = "local drift force")
plt.semilogx(z_th, F_drift_th  * 1e15, color = "k", zorder = 11, label="multifit force")
plt.fill_between(z_th, F_drift_th  * 1e15 -F_err_th*1e15, F_drift_th  * 1e15 + F_err_th*1e15, alpha = 0.2, color ="k")
plt.plot(z_th, F_drift_th  * 1e15 -F_err_th*1e15, "--" ,color ="k")
plt.plot(z_th, F_drift_th  * 1e15 +F_err_th*1e15, "--", color ="k")

plt.ylim((-40, 300))
plt.xlim((10e-9,None))
plt.xlabel("$z$ (m)")
plt.ylabel("$F$ (fN)")
plt.legend()
plt.show()
```

```python
offset*1e6
```

    0.062

Finaly, everything seems to work, we do measure a sligtly higher from the drifts but it could simply be explain from a error on the offset of 5 nanometers.

```python
z = np.squeeze(raw_data["z"]) *1e-6
z = z - np.min(z) - offset*1e6

bins_center_pdf_z, widths, hist = logarithmic_hist(z, 39e-3, 3, num = 100, base=2)
z_F_exp = (bins_center_pdf_z - widths/2) * 1e-6
z_F_exp = z_F_exp[:-1]
fps = 60

def Dz_z(z,off):
    off =(off * 1e-6)
    z = z - off
    result = ((6*z*z + 2*a*z) / (6*z*z + 9*a*z + 2*a*a))
    return result

def vzz(Y, z,times=[1], method = "anti-Ito"):

    # We frist get all the different variables of the problem
    Y = Y * 1e-6
    z_z = z * 1e-6

    # We choose over wich time we want to comput the diffusion coefficient
    # Theoritically it should be the same for each time.
    # We initialize the variable used to store the results.
    # For the diffusion coefficient itself.
    vz = np.zeros((len(times), len(Y[:-1])))

    # For the error
    errvz = np.zeros((len(times), len(Y[:-1])))

    for n, i in enumerate(times):

        # Conpute the Delta x = x(Dt + t) - x(t) for given Dt -- same over y
        if method == "Ito" or method == "anti-Ito":
            Dzs = z_z[i:] - z_z[0:-i]
            Dzs = Dzs + F_correction(1,z[0:-i])/(6*np.pi*0.001*a*Dz_z(z[0:1],0))

        if method == "Stratanovich":
            Dzs = z_z[2*i:] - z_z[0:-2*i]
        #Dzs = z_z[0:-i] - z_z[i:]
        # Now for each z-bin we are going to measure the diffusion coefficient.

        for m in range(len(Y)-1):
            #print(len(Dzs))
            #print(len(z[i:]))
            # We take the Dx corresponding the actual bin
            if method == "anti-Ito":
                dz = Dzs[(z_z[i:] > Y[m]) & (z_z[i:] < Y[m+1])]
            if method == "Ito":
                dz = Dzs[(z_z[:-i] > Y[m]) & (z_z[:-i] < Y[m+1])]
            if method == "Stratanovich":
                dz = Dzs[(z_z[i:-i] > Y[m]) & (z_z[i:-i] < Y[m+1])]

            if method == "Ito" or method == "anti-Ito":
                vz[n,m] = np.mean(dz)/(i/fps)
            if method == "Stratanovich":
                vz[n,m] = np.mean(dz)/(2*i/fps)
            #errvz[n,m] = np.std(dz)/(i/fps)
            errvz[n,m] = len(dz)
    err_vz = np.nanmean(errvz, axis=0)
    vz = np.nanmean(vz, axis=0)

    return vz, err_vz

def F_corr(alpha,z):
    return -alpha * D(z) * 4e-21 * D_1_prime(z)

def D_1_prime(z):
    return (2 * r * (2 * r**2 + 12 * r * z + 21 *  z**2))/(2 * r** + 9 * r * z + 6 * z**2)**2

def D(z):
    return (6*z**2 + 9*r*z + 2*r**2) / (6*z**2 + 2 * r * z)

def F_corr(alpha,z):
    return -alpha * D(z) * 4e-21 * D_1_prime(z)
def F_corr(alpha,z):
    return - 4e-21 * alpha *(42*a*z*z + 24*a*a*z + 4*a*a*a)/(36*(z**4) + 66*a*(z**3) + 30*a*a*z*z + 4*(a**3)*z)

```

    C:\Users\maxim\miniconda3\envs\cython\lib\site-packages\numpy\lib\histograms.py:905: RuntimeWarning: invalid value encountered in true_divide
      return n/db/n.sum(), bin_edges

```python
a = 1.51e-6
vz_ito, err_ito = vzz(np.array([1,2,3,4,5,6]),z, method = "Ito")
#z_F_exp = bins_center_pdf_z[:-1] * 1e-6
F_exp_ito = 6 * np.pi  * 0.001 * a * vz_ito/ Dz_z(z_F_exp, 0) + F_corr(1,z_F_exp)
F_err_ito = 6 * np.pi  * 0.001 * a * err_ito/ Dz_z(z_F_exp, 0) + F_corr(1,z_F_exp)
```

```python
F
```

```python
plt.errorbar(z_F_exp, F_exp_ito * 1e15,yerr=F_err_ito*1e15/2,xerr=widths[:-1]/2*1e-6, marker = "o", linestyle="")
plt.plot(z_hist, F_eq * 1e15, "s", label = "$F_{exp}$ form E", zorder = 10)
plt.semilogx(z_th, F_drift_th  * 1e15, color = "k", zorder = 11, label="multifit force")
#plt.fill_between(z_th, F_drift_th  * 1e15 -F_err_th*1e15, F_drift_th  * 1e15 + F_err_th*1e15, alpha = 0.2, color ="k")
#plt.plot(z_th, F_drift_th  * 1e15 -F_err_th*1e15, "--" ,color ="k")
#plt.plot(z_th, F_drift_th  * 1e15 +F_err_th*1e15, "--", color ="k")

plt.ylim((-40, 300))
plt.xlim((10e-9,None))
plt.xlabel("$z$ (m)")
plt.ylabel("$F$ (fN)")
plt.legend()
plt.show()
```

```python
len(bins_center_pdf_z)
```

    99

# Froce error

Now we would like to compute the stochastic error on the force.

```python
def random_on_force():
    z = np.squeeze(raw_data["z"])
    z = z - np.min(z) - offset*1e6
    bins_center_pdf_z, widths, hist = logarithmic_hist(z, 39e-3, 3, num = 45, base=2)
    z = z + np.random.normal(0, 1e-3, len(z))
    z_F_exp = (bins_center_pdf_z - widths/2) * 1e-6
    z_F_exp = z_F_exp[:-1]
    a = 1.51e-6
    vz_ito, err_ito = vzz(bins_center_pdf_z, z, [1,2], method = "Ito")
    #z_F_exp = bins_center_pdf_z[:-1] * 1e-6
    F_exp_ito = 6 * np.pi  * 0.001 * a * vz_ito/ Dz_z(z_F_exp, 0) + F_corr(1,z_F_exp)
    return F_exp_ito, err_ito

```

```python
len(z_F_exp)
```

    98

```python
len(random_on_force()[0])
```

    43

```python
bins_center_pdf_z, widths, hist = logarithmic_hist(z, 39e-3, 3, num = 45, base=2)
z = z + np.random.normal(0, 1e-3, len(z))
z_F_exp = (bins_center_pdf_z - 1*widths/2) * 1e-6
z_F_exp = z_F_exp[:-1]

N = 100
all_F = np.ones((len(random_on_force()[0]),N))
all_err_F = np.ones((len(random_on_force()[0]),N))

for i in range(N):
    F, err = random_on_force()
    all_F[:,i] = F
    all_err_F[:,i] = err

```

```python
F_meaned = np.mean(all_F, axis=1)
F_err_tot = np.std(all_F, axis=1)

```

```python
T_boxes = np.mean(all_err_F, axis=1)
```

```python
#plt.plot(z_F_exp, T_boxes)
```

```python
f_th2 = f_th(z_F_exp)
F_err_th = np.sqrt(24 * np.pi * 4e-21 *0.001*  eta(z_F_exp) * a / (T_boxes*dt))

```

```python
[3]*4
```

    [3, 3, 3, 3]

```python
f_th
```

    <function __main__.<lambda>(z)>

```python
plt.figure(figsize=( cm2inch(16),cm2inch(9)))

plt.errorbar(z_F_exp, F_meaned*1e15, yerr=F_err_tot*1e15,xerr = widths[:-1]*1e-6/2 ,marker = "o", linestyle="", ecolor="k", capsize = 2, label = "Eq. (5.2.19)")
#plt.plot(z_hist, F_eq * 1e15, "s", label = "$F_{exp}$ form E", zorder = 10)
plt.fill_between(z_F_exp, f_th2  * 1e15 -F_err_th*1e15, f_th2  * 1e15 + F_err_th*1e15, alpha = 0.2, color ="k", label = "thermal noise range")
plt.plot(z_F_exp, f_th2  * 1e15 - F_err_th*1e15, "--" ,color ="k", linewidth = 2)
plt.plot(z_F_exp, f_th2  * 1e15 + F_err_th*1e15, "--" ,color ="k", linewidth = 2)
plt.plot(z_hist, F_eq * 1e15, "s", label = "Eq. (5.2.18)", zorder = 1)

plt.plot(z_F_exp, [np.mean( F_meaned[5:]  * 1e15)]*len(z_F_exp), color = "tab:red", linestyle= "--", linewidth = 2)
plt.semilogx(z_F_exp, f_th2 * 1e15, color = "k", zorder = 11, label="Eq. (5.2.21)")

#plt.plot(z_th, F_drift_th  * 1e15 +F_err_th*1e15, "--", color ="k")
plt.text(5e-7, 20, "$F_\mathrm{g} = -7$ fN", color="tab:red", fontsize ="medium" )
plt.ylim((-40, 200))
plt.xlim((35e-9,2.5e-6))
plt.xlabel("$z$ (m)", fontsize="medium")
plt.ylabel("$F$ (fN)", fontsize="medium")
plt.legend()
plt.tight_layout(pad=0.2)
plt.savefig("figure_force_total.pdf")
plt.show()
```

```python
plt.figure(figsize=( cm2inch(6*1.68),cm2inch(6)))

plt.errorbar(z_F_exp, F_meaned*1e15, yerr=F_err_tot*1e15,xerr = widths[:-1]*1e-6/2 ,marker = "o", linestyle="", ecolor="k", capsize = 2, label = "Eq. (5.2.19)")
plt.plot(z_hist, F_eq * 1e15, "s", label = "$F_{exp}$ form E", zorder = 10, color = "tab:orange")
plt.fill_between(z_F_exp, f_th2  * 1e15 -F_err_th*1e15, f_th2  * 1e15 + F_err_th*1e15, alpha = 0.2, color ="k", label = "thermal noise range")
plt.plot(z_F_exp, f_th2  * 1e15 - F_err_th*1e15, "--" ,color ="k", linewidth = 2)
plt.plot(z_F_exp, f_th2  * 1e15 + F_err_th*1e15, "--" ,color ="k", linewidth = 2)
plt.plot(z_hist, F_eq * 1e15, "s", label = "Eq. (5.2.18)", zorder = 1)

plt.plot(z_F_exp, [np.mean( F_meaned[5:]  * 1e15)]*len(z_F_exp), color = "tab:red", linestyle= "--", linewidth = 2)
plt.semilogx(z_F_exp, f_th2 * 1e15, color = "k", zorder = 11)

plt.text(5e-7, 20, "$F_\mathrm{g} = -7$ fN", color="tab:red", fontsize ="medium" )
plt.ylim((-40, 200))
plt.xlim((35e-9,2.5e-6))
plt.xlabel("$z$ (m)", fontsize="medium")
plt.ylabel("$F$ (fN)", fontsize="medium")
#plt.legend()
plt.tight_layout(pad=0.2)
plt.savefig("figure_force_pdf.svg")

```

```python
dict_force = {}
dict_force["z_F_exp"] = z_F_exp
dict_force["F_meaned"] = F_meaned
dict_force["widths"] = widths
dict_force["z_hist"] = z_hist
dict_force["f_th2"] = f_th2
dict_force["F_err_th"] = F_err_th
dict_force["F_err_tot"] = F_err_tot

import scipy
scipy.io.savemat("dict_force.mat", dict_force)

```

```python
z_hist
```

    array([4.06315748e-08, 4.40312392e-08, 4.77153552e-08, 5.17077230e-08,
           5.60341343e-08, 6.07225386e-08, 6.58032240e-08, 7.13090130e-08,
           7.72754741e-08, 8.37411520e-08, 9.07478164e-08, 9.83407320e-08,
           1.06568951e-07, 1.15485629e-07, 1.25148369e-07, 1.35619596e-07,
           1.46966957e-07, 1.59263756e-07, 1.72589435e-07, 1.87030080e-07,
           2.02678981e-07, 2.19637234e-07, 2.38014392e-07, 2.57929176e-07,
           2.79510240e-07, 3.02897003e-07, 3.28240548e-07, 3.55704600e-07,
           3.85466584e-07, 4.17718767e-07, 4.52669507e-07, 4.90544593e-07,
           5.31588707e-07, 5.76067002e-07, 6.24266819e-07, 6.76499538e-07,
           7.33102596e-07, 7.94441660e-07, 8.60912994e-07, 9.32946019e-07,
           1.01100608e-06, 1.09559747e-06, 1.18726667e-06, 1.28660587e-06,
           1.39425683e-06, 1.51091500e-06, 1.63733402e-06, 1.77433058e-06,
           1.92278972e-06])

```python

```

```python

```

```python

```

```python

```


Text provided under Creative Commons Attribution License CC-BY.  All Code is made available
under the FSF-approved BSD-3 license (c) Lorena A. Barba, Gilbert F. Forsyth 2017. Thanks to
NSF for support via CAREER award #1149784.

 CFDPython_4.py

Source: https://github.com/barbagroup/CFDPython/blob/master/lessons/03_CFL_Condition.ipynb
This if from a class on learning to program CFD using Python by Dr. Lorena A. Barba

Step 4: 1-D Burgers' Equation - convection and diffusion

Also called viscous Burgers' equation for convection and diffusion in one dimension.  You can read about Burgers' Equation on its wikipedia page.
The equation looks like this:

As the equation shows, it is a combination of non-linear convection and diffusion equations seen separately in the previous lessons 1-3.  We can solve this equation by using a taylor series as in the previous lessons.  Using forward difference for time, backward difference for space and our 2nd-order method for the second derivitive yields:

We can solve for our only unknow , and step in time as follows:

Initial and Boundary Conditions

To investigate some interesting properties of Berger's equation, it is helpful to use different initial and boundary conditions than we used in our previous steps.  Our initial conditions are going to be

This has an analytical solution given by

 Our boundary condition will be:

This is called a periodic boundary condition.

Saving Time with SymPy

The initial condition we're using for Burgers' Equation can be a bit of a pain to evaluate by hand. The derivative $\frac{\partial \phi}{\partial x}$ isn't too terribly difficult, but it would be easy to drop a sign or forget a factor of $x$ somewhere, so we're going to use SymPy to help us out.

SymPy is the symbolic math library for Python. It has a lot of the same symbolic math functionality as Mathematica with the added benefit that we can easily translate its results back into our Python calculations (it is also free and open source).

Start by loading the SymPy library, together with our favorite library, NumPy.  Then we're going to tell SymPy that we want all of its output to be rendered using $\LaTeX$. This will make our Notebook beautiful!

```python
#Remember: Comments in python are denoted by the pound sign
import numpy                   # Load the numpy maxrix algebra library
import sympy                   # To simplify our mathematics use this symbolic math library.

#tell Sympy that we want our output rendered using LATEX.  This will make our notebook beautiful?!
from sympy import init_printing
init_printing(use_latex=True)
```

Now we set up the symbolic variable we will use in sympy to compute phi and output the equation for phi in LATEX format to verify we got what we thought we were programming.

```python
x, nu, t = sympy.symbols('x nu t')
phi = (sympy.exp(-(x - 4 * t)**2 / (4 * nu * (t + 1))) +
       sympy.exp(-(x - 4 * t - 2 * sympy.pi)**2 / (4 * nu * (t + 1))))
phi
```

The above output is small and hard to read, but corresponds to our equation for phi.  Now
set up our equation for the partial derivative of phi with respect to X.

```python
phiprime = phi.diff(x)
phiprime
```

To see the unrendered version use the Python print command

```python
print(phiprime)
```

    -(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 4*pi)*exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1)))/(4*nu*(t + 1))

Now What?

Now we have a Python version of phi and it's derivative.  Next we want to write out the full initial condition equation and then translate it into a usable python expression.  For this we will use the lambdify function from the SymPy.utilities library which takes a SymPy symbolic equation and turns it into a callable function.

"Numerical" VS "Analytical" solutions

"Numerical" usually indicates an approximate solution obtained by methods of numerical analysis.  "Analytical" solutions are exact and obtained by methods of symbolic manipulation, derived using analysis. Typically Analytical solutions are signifitantly more compute intensive than Numerical methods, but provide exact solutions as opposed to the numerical solutions that generally involve some approximation to simplify their calculations.  Below we are going to compare a Numberical and Analytical solution to see what the Numeric approximation error looks like.

```python
from sympy.utilities.lambdify import lambdify

#initial condition equation:
u = -2 * nu *(phiprime / phi) + 4
print (u)
```

    -2*nu*(-(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 4*pi)*exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)))/(exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1))) + exp(-(-4*t + x)**2/(4*nu*(t + 1)))) + 4

Lambdify:

The sympy.utilities.lambdify module provides convenient functions to transform sympy expressions to lambda functions which can be used to calculate numerical values very fast. In addition, it converts SymPy names to the names used in the given numerical library, usually NumPy. To lambdify an expression into a usable function, we tell lambdify which variable to request and the function we want to plug them into.

```python
ufunc = lambdify((t, x, nu), u)
print(ufunc(1,4,3))
```

    3.49170664206445

Back to Burgers' Equation

Now that we have the initial conditions set, we can finish setting up the problem. Once the setup is complete, we will plot the initial conditions using our lambdify-ed function.

```python
#load our plotting routine and tell it to output our plots inline with our
# Junyper Notebook, rather than in a separate window.
from matplotlib import pyplot
%matplotlib inline

#Now declare our variables
nx = 101
nt = 100
dx = 2 * numpy.pi / (nx-1)
nu = 0.07
dt = dx * nu

x = numpy.linspace(0, 2 * numpy.pi, nx)
un = numpy.empty(nx)
t = 0

u = numpy.asarray([ufunc(t, x0, nu) for x0 in x])
#Output the resulting numerical array.
u
```

    array([4.        , 4.06283185, 4.12566371, 4.18849556, 4.25132741,
           4.31415927, 4.37699112, 4.43982297, 4.50265482, 4.56548668,
           4.62831853, 4.69115038, 4.75398224, 4.81681409, 4.87964594,
           4.9424778 , 5.00530965, 5.0681415 , 5.13097336, 5.19380521,
           5.25663706, 5.31946891, 5.38230077, 5.44513262, 5.50796447,
           5.57079633, 5.63362818, 5.69646003, 5.75929189, 5.82212374,
           5.88495559, 5.94778745, 6.0106193 , 6.07345115, 6.136283  ,
           6.19911486, 6.26194671, 6.32477856, 6.38761042, 6.45044227,
           6.51327412, 6.57610598, 6.63893783, 6.70176967, 6.76460125,
           6.82742866, 6.89018589, 6.95176632, 6.99367964, 6.72527549,
           4.        , 1.27472451, 1.00632036, 1.04823368, 1.10981411,
           1.17257134, 1.23539875, 1.29823033, 1.36106217, 1.42389402,
           1.48672588, 1.54955773, 1.61238958, 1.67522144, 1.73805329,
           1.80088514, 1.863717  , 1.92654885, 1.9893807 , 2.05221255,
           2.11504441, 2.17787626, 2.24070811, 2.30353997, 2.36637182,
           2.42920367, 2.49203553, 2.55486738, 2.61769923, 2.68053109,
           2.74336294, 2.80619479, 2.86902664, 2.9318585 , 2.99469035,
           3.0575222 , 3.12035406, 3.18318591, 3.24601776, 3.30884962,
           3.37168147, 3.43451332, 3.49734518, 3.56017703, 3.62300888,
           3.68584073, 3.74867259, 3.81150444, 3.87433629, 3.93716815,
           4.        ])

```python
#plot the outline of our figure.
pyplot.figure(figsize=(11,7), dpi=100)
#set the x-axis to have markers
pyplot.plot(x, u, marker='o', lw=2)
#set the x-axis limits from 0 to 2*pi
pyplot.xlim({0, 2 * numpy.pi})
#set the y-axis limits
pyplot.ylim([0,10]);
```

This is a numeric solution to Burgers' equation.

 This is definitely not the hat function we were previously dealing with until now.  This function is called a "saw-tooth function".  Lets proceed and see what happens.

Periodic Boundary Conditions

One of the big differences between Step 3 and the previous lessons is the use of periodic boundary conditions. If you experimented with step 1 and 2 and make the simulation run long enough (by increasing nt) you will notice the wave will keep moving to the right until it no longer even shows up on the plot.

With periodic boundary conditions, when a point gets to the right-hand side of the frame, it wraps around back to the front of the frame.

Recall the discretization that we worked out at the beginning of this notebook:

"What does"mean " when "" is already at the end of the frame?"

Think about this for a minute before proceeding.

Below we are going to compute the analytical solution and plot the solutions (the Numerical or Computational and Analytical) together.

```python
for n in range(nt):
    un = u.copy()
    for i in range(1, nx-1):
        u[i] = un[i] - un[i] * dt / dx *(un[i] - un[i-1]) + nu * dt / dx**2 *\
                (un[i+1] - 2 * un[i] + un[i-1])
    u[0] = un[0] - un[0] * dt / dx * (un[0] - un[-2]) + nu * dt / dx**2 *\
                (un[1] - 2 * un[0] + un[-2])
    u[-1] = u[0]

u_analytical = numpy.asarray([ufunc(nt * dt, xi, nu) for xi in x])
```

```python
pyplot.figure(figsize=(11, 7), dpi=100)
pyplot.plot(x,u, marker='o', lw=2, label='Computational')
pyplot.plot(x, u_analytical, label='Analytical')
pyplot.xlim([0, 2 * numpy.pi])
pyplot.ylim([0, 10])
pyplot.legend();
```

What Next?

The subsequent steps from 4 to 11 will extend the 1D finite-difference formulas (diffusion, convection and burgers' equations) into two dimensions.  Just apply the definition -- a partial derivative with respect to x is the variation in the x direction while keeping y constant.

Before moving on, take a slight break and learn about array operations with mumpy.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```