_Lambda School Data Science_

# Polynomial & Log-Linear Regression

## "Linear" Regression?

Which of the following is a linear regression model?

**All** of these functional forms can be fit using Linear Regression. The "Linear" in linear regression refers to the linear form of the equation.

### Linear Combinations

Remember when we rewrote vectors as a **linear combination** of scalars and unit vectors?

\begin{align}
v = \begin{bmatrix}2 \\ 3\end{bmatrix} = 2 \begin{bmatrix} 1 \\ 0 \end{bmatrix} + 3 \begin{bmatrix} 0 \\ 1\end{bmatrix} = 2\hat{i} + 3\hat{j}
\end{align}

The syntax where we have a scalar (think coefficient) multiplying some vector (unit vector in this case) and all of them being added together is what makes this a linear combination.

### Linear Equations

A "Linear Equation" is any equation that takes the following form:

\begin{align}
a_1x_1 + \ldots + a_nx_n + b = 0
\end{align}

Does this look familiar? A linear equation is one where we have $x_1, \ldots, x_n$ unknowns and $b, a_1, \ldots, a_n$ coefficients which are considered parameters of the equation. "The solutions of such an equation are the values that, when substituted to the unknowns, make the equality true."

[Linear Equation Wikipedia](https://en.wikipedia.org/wiki/Linear_equation)

Linear Regression is **linear** not because it can only plot straight lines and fit straight-line patterns in data, but because the form of the equation used to represent our regression is in the form of a **Linear Equation**.

### _So how do we fit curved data?_

# Example #1: Moore's Law dataset

#### Background
- https://en.wikipedia.org/wiki/Moore%27s_law
- https://en.wikipedia.org/wiki/Transistor_count

#### Scrape HTML tables with Pandas!
- https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_html.html
- https://medium.com/@ageitgey/quick-tip-the-easiest-way-to-grab-data-out-of-a-web-page-in-python-7153cecfca58

#### More web scraping options
- https://automatetheboringstuff.com/chapter11/

### Imports

```python
%matplotlib inline
from ipywidgets import interact
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
import statsmodels.api as sm
```

### Get data

```python
tables = pd.read_html('https://en.wikipedia.org/wiki/Transistor_count', header=0)
print([table.shape for table in tables])
```

```python
moore = tables[0]
moore
```

### Clean data

```python
moore.info()
```

```python
feature = 'Date of introduction'
target  = 'Transistor count'

moore = moore.dropna(subset=[feature, target]).copy()

for column in [feature, target]:
    moore[column] = (moore[column]
                     .str.split('[').str[0]  # Remove citations
                     .str.replace(r'\D','')  # Remove non-digit characters
                     .astype(int))

moore = moore.sort_values(by=feature)
```

### Visualize

```python
sns.distplot(moore[target]);
```

```python
moore.plot(x=feature, y=target, kind='scatter', alpha=0.5);
```

```python
moore.plot(x=feature, y=target, kind='scatter', alpha=0.5, logy=True);
```

### Ordinary Least Squares Regression

```python
ax = moore.plot(x=feature, y=target, kind='scatter', alpha=0.5)

X = moore[[feature]]
y = moore[target]
model = LinearRegression()
model.fit(X, y)
ax.plot(X, model.predict(X))
print('R^2', model.score(X, y))
```

### Log-Linear Regression

```python
ax = moore.plot(x=feature, y=target, kind='scatter', alpha=0.5, logy=True)

X = moore[[feature]]
y = np.log(moore[target]) # Apply natural log function to the target
model = LinearRegression()
model.fit(X, y)
y_pred = np.exp(model.predict(X)) # Apply exponential function (inverse of natural log) to the predictions
ax.plot(X, y_pred)
print('R^2', model.score(X, y))
```

```python
ax = moore.plot(x=feature, y=target, kind='scatter', alpha=0.5)
ax.plot(X, y_pred)
print('R^2', model.score(X, y))
```

```python
sns.distplot(y);
```

### Polynomial Regression

```python
# Make polynomial feature
moore['Date of introduction ** 2'] = moore['Date of introduction'] ** 2

features = ['Date of introduction', 'Date of introduction ** 2']
X = moore[features]
y = moore[target]
model = LinearRegression()
model.fit(X, y)
ax = moore.plot(x='Date of introduction', y=target, kind='scatter', alpha=0.5)
ax.plot(X['Date of introduction'], model.predict(X))
print('R^2', model.score(X, y))
```

```python
# Generalize for higher degree polynomials, and print equation

def polynomial_regression(degrees=2):

    # Make polynomial features
    feature = 'Date of introduction'
    polynomial_features = []
    for degree in range(2, degrees+1):
        name = f'{feature} ** {degree}'
        moore[name] = moore[feature] ** degree
        polynomial_features.append(name)

    features = [feature] + polynomial_features
    target  = 'Transistor count'
    X = moore[features]
    y = moore[target]
    model = LinearRegression()
    model.fit(X, y)

    ax = moore.plot(x=feature, y=target, kind='scatter', alpha=0.5)
    ax.plot(moore[feature], model.predict(X))
    betas = [model.intercept_] + model.coef_.tolist()
    equation = ' + '.join(f'{beta}x**{i}' for i, beta in enumerate(betas))
    print(equation)
    print('R^2', model.score(X, y))

polynomial_regression(3)
```

```python
interact(polynomial_regression, degrees=(1,10));
```

# Polynomial Regression, explained

Just as multiple regression was an extension of the bivariate case, Polynomial Regression is an extention of multiple regression and can be used to fit data to any (curved) shape. This is one of the reasons why data exploration is so important. You won't know that you need to fit a polynomial function to a feature unless you have examined its distribution.

[Why is polynomial regression considered a special case of multiple linear regression?](https://stats.stackexchange.com/questions/92065/why-is-polynomial-regression-considered-a-special-case-of-multiple-linear-regres)

# Example #1: King County Housing Data

[from Kaggle](https://www.kaggle.com/harlfoxem/housesalesprediction)

```python
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/kc_house_data.csv')
print(df.shape)
df.head().T
```

## Find a "curved" feature in the dataset

```python
target = 'price'
numeric_columns = df.select_dtypes(include='number').columns
for feature in numeric_columns.drop(target):
    sns.scatterplot(x=feature, y=target, data=df, alpha=0.1)
    plt.show()
```

## Make or "engineer" a new grade_squared feature

```python
df['grade_squared'] = df['grade']**2
for feature in ['grade', 'grade_squared']:
    sns.scatterplot(x=feature, y=target, data=df, alpha=0.1)
    plt.show()
```

## Test the fit of a polynomial regression to that feature

First we'll fit a regular bivariate regression line and calculate its $R^2$ to get a baseline. Since we want to know if this generated feature is improving our model or not we'll first run our code without it so that we have something to compare to.

$price_i = \beta_0 + \beta_1grade_i + \epsilon_i$

```python
# Separate dependent and independent variables
target = 'price'
features = ['grade']
y = df[target]
X = df[features]

def run_linear_model(X, y):
    # Split into test and train data
    X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.80, test_size=0.20, random_state=42)

    # Fit model using train data
    model = LinearRegression()
    model.fit(X_train, y_train)

    # Make predictions using test features
    y_pred = model.predict(X_test)

    # Compare predictions to test target
    rmse = (np.sqrt(mean_squared_error(y_test, y_pred)))
    r2 = r2_score(y_test, y_pred)

    print('Root Mean Squared Error', rmse)
    print('R^2 Score', r2)
    print('Intercept', model.intercept_)
    coefficients = pd.Series(model.coef_, X_train.columns)
    print(coefficients.to_string())

run_linear_model(X, y)
```

## Lets try fitting grade_squared as a bivariate model

$price_i = \beta_0 + \beta_1grade^{2}_i + \epsilon_i$

```python
# Separate dependent and independent variables
target = 'price'
features = ['grade_squared']
y = df[target]
X = df[features]

run_linear_model(X, y)
```

## Multiple Regression using both grade and grade_squred

$price_i = \beta_0 + \beta_1 grade_i + \beta_2grade^{2}_i + \epsilon_i$

```python
# Separate dependent and independent variables
target = 'price'
features = ['grade', 'grade_squared']
y = df[target]
X = df[features]

run_linear_model(X, y)
```

# How to find non-linear features

## 1) Domain Knowledge (think about your variables and hypothesize)

This is why having domain knowledge about the problem that you're trying to solve is something that's so important. In the context of home prices, variables that have a curved structure often are that way due to some form of diminishing returns increases in certain amenities. Lets think about the following variables:

- Lot Size:

The more land you're willing to buy all at once the cheaper it will be on a per-acre basis (Saving money when you buy in bulk). This trend carries through to small to medium sized lots but but with a more shallow curve.

- Square Footage:

Square footage of a home sees a similar pattern. The value an additional 100 square feet in small homes (imagine the difference between say a 800 sq foot home and a 900 sq foot home) makes a big difference to buyers, where as an additional 100 square feet in a mansion probably isn't valued quite as highly.

- Age:

Just like how the prices of new cars drop steeply in the first few years, the value of homes due to age drop quickly in the first few years after its built and then less quickly as time goes on. This is not a linear pattern and needs to be fitted by a polynomial model.

## 2) Visual Inspection

We already talked about how generating scatterplots or other graphs is vital in our data exploration stage and can lead us to identify possible candidates for polynomials. Here I just wanted to share one more tip that can help you analyze scatterplots when you have a lot of data.

If you have so much data that it's hard to tell what's going on in your scatterplot, then sample your data and regenerate them to get a better idea.

```python
# Sample our dataframe to take 1/20th the values
sampled = df.sample(frac=0.05, replace=True, random_state=42)

target = 'price'
numeric_columns = df.select_dtypes(include='number').columns
for feature in numeric_columns.drop(target):
    sns.scatterplot(x=feature, y=target, data=sampled, alpha=0.2)
    plt.show()
```

## 3) Inspect the distribution of residuals

## An Aside: The "Hedonic Housing Model"

Using Linear Regression to model home prices is a very common use of predictive linear regression modeling. It's so common fact that it has its own name: The Hedonic Housing Model. In the Hedonic Housing model it is well understood and reiterated that certain features tend to be curved in nature and these polynomial features (like the ones mentioned above) have all been well explored in real estate prediction circles. This is another of how domain knowledge can give you an edge. The best way to gain domain knowledge is to dive in and try and solve one particular kind of problem, and pick up little tips and tidbits as time goes on.

# Log-Linear Regression

In a log-linear regression model, we take the natural log of all of our y variable and use that as our y vector instead of the raw y values. Why would we do that?

<https://www.kaggle.com/c/house-prices-advanced-regression-techniques/discussion/60861>

## 1) To reduce skew in y

Where we have variables with lots of relatively low values and few high values (like with home prices) we would expect to see our data more clustered on the left-hand side with a long tail extending to the right up into the expensive homes. The fact of the matter is that we will be able to make better predictions if we can normalize our data to some degree and one way to do this is by taking the natural log of it.

```python
df['ln_price'] = np.log(df['price'])
```

```python
sns.distplot(df['price']);
```

```python
sns.distplot(df['ln_price']);
```

## New distribution of variables

```python
target = 'ln_price'
numeric_columns = df.select_dtypes(include='number').columns
for feature in numeric_columns.drop(target):
    sns.scatterplot(x=feature, y=target, data=df, alpha=0.1)
    plt.show()
```

## 2) Make coefficients easier to interpret

Transforming our price values in this way won't change our model's ability to generate predictions, but what it **will** do is change the interpretation of all of our x-coefficients. This will change our x-coefficients from have an elasticity type interpretation (a raw dollar amount change if there is a 1 unit increase in x) to having a percentage-terms interpretation. Lets demonstrate and talk about this further.

## 3) Make our errors easier to interpret

Errors that have been calculated on variables that are in log form can also be interpreted roughly as percentage error. We've been using percentages all our lives and they have immediate meaning to us. This is why I prefer log-linear regression models when possible.

<https://people.duke.edu/~rnau/411log.htm>

Feature Engineering is a big topic in machine learning. We won't be able to cover every aspect of it today, but hopefully we can give you a strong idea of what it is and how to go about it.

[Understanding Feature Engineering Part 1](https://towardsdatascience.com/understanding-feature-engineering-part-1-continuous-numeric-data-da4e47099a7b)

Feature engineering is key to success in predictive modeling. It is the process by which we take existing features and combine them or alter them in ways that will expose additional signal to our model. Feature engineering is all about making the most of the data that we already had.

```python
# Log-Linear Regression
# Separate dependent and independent variables
target = 'ln_price'
features = ['grade']
y = df[target]
X = df[features]

run_linear_model(X, y)
```

This means that a one unit increase in the grade of a home increases its sale price by 31%. Often it is much easier to interpret coefficients in this manner than in the regular way.

```python
## Log-Linear Regression
# Separate dependent and independent variables
target = 'ln_price'
features = ['grade', 'grade_squared']
y = df[target]
X = df[features]

run_linear_model(X, y)
```

Our RMSE is really "small" now because it now represents error in percentage terms. We're on average about 37% off in our predictions of house prices.

Our coefficients can also be understood in percentage terms which makes the coefficients on our regression much more digestable at a glance.

## A note about $R^2$

$R^2$ If we add any feature to our model (even nearly meaningless ones) our $R^2$ will improve. For this reason raw $R^2$ is not the ultimate measure of goodness of fit. It is informative but completely secondary to our Root-Mean-Squared-Error (in predictive modeling). While a higher $R^2$ is generally better, this isn't the thing that we're trying to optimize. We care more about minimizing RMSE than maximizing $R^2$

### "Kitchen Sink" models

You may hear the term "kitchen sink" regression model used. This refers to a regression model that throws every available explanatory variable into the model in an attempt to improve it without much thought to whether those variables should really be considered as affecting y. Kitchen Sink models will have a higher $R^2$ than others but will have higher standard errors (estimates about particular coefficients may be less precise).

Therefore, you tend to see "Kitchen Sink" models when the only priority is predictive accuracy and not interpretability.

### Alternative measures of Goodness-of-fit

Efforts have been made to improve upon $R^2$ due to these limitations. A metric called "Adjusted $R^2$" seeks to account for the number of explanatory variables included in a model and adjust $R^2$ accordingly. This is something that you can look up if you're curious. I just wanted to make you aware of it more than anything.

## A note about dirty linear regression data

### Linear Regression models can only process numeric values

### Your data must be free of NaN values before being passed to the algorithm

(some data cleaning will be required in today's assignment)

# Feature Engineering

### Polynomial Regression requires feature engineering

You've already seen an example of feature engineering today when we created the $grade^2$ variable. We took an existing feature and used it to generate a new feature that exposed the data to the model in a slightly different way.

"Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data." - [Jason Brownlee](https://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/)

### What features could we engineer with the King County dataset?

```python
df.head().T
```

- **[date]** The date is in a format that is not super useful to us. If were to extract the year we could then take the difference between year and yr_built to find the age of the home. We could also include the squared term of the home age since we know (from our hedonic housing model domain knowledge) that home age typically is not linear.
- **[bedrooms]** & **[bathrooms]** We maybe use a combined measure of bedrooms and bathrooms, or find an average room square footage by taking total number of rooms and dividing by the square footage.
- **[sqft_living]** **[sqft_lot]** The difference between lot square footage and home square footage ought to also give us a rough measure of the size of the yard. Rough measures are fine as long as the engineered features expose some new shred of meaning to our model.
- **[floors]** We could calculate an average number of square feet per floor
- **[lat]** **[long]** There are all kinds of things that we could do with the latitude and longitude especially if we use some kind of outside API or external dataset to bring in new features associated with the location of the homes. This would take a lot of work but these could potentially be very powerful features.

### Kaggle is one of the best places to get feature engineering ideas

Look at Kernels: https://www.kaggle.com/c/house-prices-advanced-regression-techniques/kernels

Here's just one example: https://www.kaggle.com/thevachar/house-price-regression-and-feature-engineering

Kaggle has definitions of the columns too: https://www.kaggle.com/harlfoxem/housesalesprediction

### Start with baseline, before feature engineering

```python
df['ln_price'] = np.log(df['price'])
target = 'ln_price'
numeric_columns = df.select_dtypes(include='number').columns
features = numeric_columns.drop([target, 'price', 'id'])
features
```

```python
# Non-Feature Engineered Baseline Model
y = df[target]
X = df[features]
run_linear_model(X, y)
```

### Engineer a new feature

```python
df['age'] = 2015 - df['yr_built']
```

```python
numeric_columns = df.select_dtypes(include='number').columns
features = numeric_columns.drop([target, 'price', 'id'])
y = df[target]
X = df[features]
run_linear_model(X, y)
```

### Can you engineer more features?

## More topics

### Interaction Terms

An interaction occurs when an independent variable has a different effect on the outcome depending on the values of another independent variable.

Lets look at an example where we were trying to estimate the level of satisfaction that a person would have when eating some kind of food with a condiment (sauce) on it.

$satisfaction_i = \beta_0 + \beta_1 food_i + \beta_2condiment_i + \epsilon$

Imagine that we have two foods: Ice Cream and Hot Dogs, and we also have to condiments: hot fudge and mustard.

$\beta_1$ in this example is trying to capture the effect of on satisfaction between eating hot dogs vs eating ice cream, and $\beta_2$ is trying to capture the effect of putting hot fudge (chocolate sauce) vs mustard on your food.

$\beta_2$ is a little more problematic in this scenario. If someone were to come up to you and ask if you preferred hot fudge or mustard on your food, how would you answer?

You would probably say something like **"IT DEPENDS ON WHAT THE FOOD IS."** This means that the effect of our x variables on y (satisfaction) depends on the combination of food and condiment. I don't know about you guys, but I wouldn't be as satisfied if I had hot fudge on my hot dog or mustard on my ice cream.

An interaction terms is something that we add to our regression to account for these "It Depends" moments between two x variables. We do this by multiplying the two of them together or *interacting* them with each other to capture the implications of the different combinations taking place.

$satisfaction_i = \beta_0 + \beta_1 food_i + \beta_2condiment_i + \beta_3(food\times condiment_i) + \epsilon$

<http://statisticsbyjim.com/regression/interaction-effects/>

### Removing Outliers

To remove outliers via the 1.5*Interquartile-Range method. The first step is to calculate the IQR for each variable.

The IQR is the difference between the 25th and 75th percentiles of the feature.

Find the IQR and multiply it by 1.5

Then add the 1.5*IQR to the 3rd quartile (75th percentile). Anything above that range is an outlier.
Subtract 1.5*IQR from the 1st quartile (25th percentile). Anything below that value is also an outlier.

You want to minimize outliers in your dataset, so remove them by dropping observations that contain outliers in key features.

Typically you will wan to remove outliers before doing anything else with your dataset. We haven't focused on this strongly yet in the class, but coefficients get strongly biased by outliers so if you want to really have accurate predictions, remove outliers before you begin your feature engineering and modeling.

# Major Takeaways

- Polynomial Regression
  - Linear Regression can fit curved lines.
  - Including squared and cubed terms can improve fit and accuracy.

- Log-linear Regression
  - ln(y) helps normalize data with a skewed y variable.
  - ln(y) changes interpretation of coefficients and errors to be percentages.

- Feature Engineering
  - Generating Features improves model accuracy if done well.
  - This is where creativity and domain knowledge really pay off.
  - When you think that certain combinations of x variables might affect y differently than how they do separately. Include an interaction term.


**CAMB usage example for matter power spectrum**

Taken from https://camb.readthedocs.io/en/latest/CAMBdemo.html

```
!pip install camb
```

    Requirement already satisfied: camb in /usr/local/lib/python3.6/dist-packages (1.1.3)
    Requirement already satisfied: sympy>=1.0 in /usr/local/lib/python3.6/dist-packages (from camb) (1.1.1)
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.6/dist-packages (from camb) (1.4.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.0->camb) (1.1.0)
    Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.6/dist-packages (from scipy>=1.0->camb) (1.18.5)

```
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import camb
from camb import model, initialpower
```

```
# Set cosmological parameters
pars = camb.CAMBparams()
pars.set_cosmology(H0=67.70, ombh2=0.022447, omch2=0.11928, omk=0, tau=0.0568)
pars.InitPower.set_params(ns=0.965)
pars.InitPower.set_params(As=np.exp(3.0480) / 1e10, ns=0.9682, r=0)

# Linear matter power spectra
pars.set_matter_power(redshifts=[0., 0.8], kmax=2.0)
pars.NonLinear = model.NonLinear_none

# Run camb
results = camb.get_results(pars)
```

    Note: redshifts have been re-sorted (earliest first)

```
# Matter power spectrum
k_over_h, z, pk = results.get_matter_power_spectrum(minkh=1e-4, maxkh=1, npoints = 200)
for i, (redshift, line) in enumerate(zip(z,['-','--'])):
    plt.loglog(k_over_h, pk[i,:], color='k', ls = line)
plt.xlabel('k [h/Mpc]')
plt.ylabel('P(k) [(Mpc/h)$^3$]')
plt.title('Matter power at z=%s and z= %s'%tuple(z));
```

```
# CMB temperature perturbations.
powers =results.get_cmb_power_spectra(pars, CMB_unit='muK')
totCl=powers['total']
Cl_TT = totCl[:,0]
ls = np.arange(totCl.shape[0])
plt.plot(ls, Cl_TT)
plt.xlabel("$\ell$")
plt.ylabel("$\ell(\ell + 1) C_\ell/2\pi$   [$\mu$K$^2$]")
plt.xlim((5, 2500));
```

```

```


```python
import random
from sympy import *
import numpy as np
```

### Chapter 5

#### Prufer's Algorithm

```python
#T=[[6, 2], [6, 7], [5, 6], [3, 5], [5, 1], [1, 4]]
#V=[2, 6, 7, 3, 5, 1, 4]

## Ex 5.1, 31
T=[[1, 2], [2, 3], [5, 2], [5, 4], [6, 5], [7, 5]]
V=[1, 2, 3, 4, 5, 6, 7]

def PrA(T, V):
    a=[]
    while len(T)!=1:
        L=[i for i in V if (flatten(T)).count(i)==1]
        v1=min(L)
        for i in T:
            if v1 in i:
                e=i
                break

        ak=e[1] if e[0]==v1 else e[0]
        del V[V.index(v1)]
        a+=[ak, ]
        del T[T.index(e)]
    return a
    return a

PrA(T, V)
```

    [2, 2, 5, 5, 5]

#### Prim's Algorithm

```python
#Ed=[['A', 'B', 4], ['E', 'A', 3], ['B', 'E', 5], ['E', 'D', 2], ['E', 'C', 3], ['C', 'B', 5], ['C', 'D', 3]]

#Ed=[['A', 'B', 9], ['D', 'B', 11], ['D','A', 3], ['B', 'F', 8], ['F', 'D', 9], ['F', 'C', 5], ['C', 'B', 8], ['E', 'F', 6], ['C', 'E', 3], ['E', 'D', 11]]

## Ex 5.2, 19
#Ed=[['C', 'B', 4], ['A', 'C', 3], ['C', 'D', 5], ['B', 'A', 6], ['A', 'D', 4], ['D', 'E', 2], ['H', 'E', 2], ['E', 'F', 4], ['F', 'I', 3], ['I', 'H', 3], ['I', 'G', 2], ['G', 'H', 4], ['G', 'E', 1]]

## Ex 5.2, 18
Ed=[['A', 'B', 4], ['A', 'E', 3], ['E', 'F', 6], ['F', 'B', 5], ['F', 'I', 4], ['I', 'J', 3], ['J', 'G', 1], ['G', 'F', 1], ['A', 'F', 2],['I', 'G', 5], ['G', 'C', 2], ['G', 'H', 4], ['H', 'D', 2], ['D', 'C', 3]]

def PA(Ed, st):
    """Ed is the list of edges with weights and st is the starting vertex"""
    E1=[]
    E2=[]
    for i in Ed:
        E1+=[[i[0], i[1]], ]
        E2+=[i[2], ]

    X=flatten(E1)
    V=list(set(X))
    n=len(V)

    E3=E1[:]
    L=[st]
    T=[]
    w=0

    while True:
        I=[]
        temp=[]
        s=0
        for i in E1:
            if (i[0] in L and i[1] not in L) or (i[1] in L and i[0] not in L):
                I+=[E1.index(i),]
                temp+=[i,]

        M=[E2[i] for i in I]
        Min=min(M) # Max=max(M) for maximal spanning tree
        te=temp[M.index(Min)]
        T+=[te,]
        del E3[E3.index(te)]
        x, y=te[0], te[1]
        L+=[x if y in L else y, ]

        for i in E3:
            if (i[0] in L and i[1] not in L) or (i[1] in L and i[0] not in L):s+=1

        if s==0: break

    for i in T: w+=E2[E1.index(i)]

    return T, w if len(L)==n else "No minimal spanning tree"

PA(Ed, 'C')
```

    ([['G', 'C'],
      ['J', 'G'],
      ['G', 'F'],
      ['A', 'F'],
      ['A', 'E'],
      ['I', 'J'],
      ['D', 'C'],
      ['H', 'D'],
      ['A', 'B']],
     21)

#### Kruskal's Algorithm

```python
#Ed=[['A', 'B', 4], ['E', 'A', 3], ['B', 'E', 5], ['E', 'D', 2], ['E', 'C', 3], ['C', 'B', 5], ['C', 'D', 3]]

#Ed=[['C', 'B', 4], ['A', 'C', 3], ['C', 'D', 5], ['B', 'A', 6], ['A', 'D', 4], ['D', 'E', 2], ['H', 'E', 2], ['E', 'F', 4], ['F', 'I', 3], ['I', 'H', 3], ['I', 'G', 2], ['G', 'H', 4], ['G', 'E', 1]]

Ed=[['A', 'B', 4], ['A', 'E', 3], ['E', 'F', 6], ['F', 'B', 5], ['F', 'I', 4], ['I', 'J', 3], ['J', 'G', 1], ['G', 'F', 1], ['A', 'F', 2],['I', 'G', 5], ['G', 'C', 2], ['G', 'H', 4], ['H', 'D', 2], ['D', 'C', 3]]

def Kr_A(Ed):
    E1=[]
    E2=[]

    for i in Ed:
        E1+=[[i[0], i[1]], ]
        E2+=[i[2], ]

    X=flatten(E1)
    V=list(set(X))
    n=len(V)
    E3=E1[:]
    E4=E2[:]
    w=0
    M=min(E2)
    e=E1[E2.index(M)]
    cl=[e]
    T=[e, ]
    ind=E3.index(e)
    del E3[ind]
    del E4[ind]

    while len(T)<n-1 and len(E3)!=0:
        M=min(E4)
        e=E3[E4.index(M)]

        for i in cl:
            if e[0] in i and e[1] not in i:
                ind=cl.index(i)
                cl1=cl[:ind]+cl[ind+1:]
                s=0

                for j in cl1:
                    if e[1] in j:
                        ind1=cl1.index(j)
                        cl[ind]=cl[ind]+cl1[ind1]
                        J=j
                        T+=[e,]
                        break
                    else:
                        s+=1
                if s==len(cl1):
                    cl[ind]=cl[ind]+[e[1]]
                    T+=[e, ]
                    J=0
                break

            elif e[1] in i and e[0] not in i:
                ind=cl.index(i)
                cl1=cl[:ind]+cl[ind+1:]
                s=0

                for j in cl1:
                    if e[0] in j:
                        ind1=cl1.index(j)
                        cl[ind]=cl[ind]+cl1[ind1]
                        J=j
                        T+=[e,]
                        break
                    else:
                        s+=1

                if s==len(cl1):
                    cl[ind]=cl[ind]+[e[0]]
                    T+=[e, ]
                    J=0
                break
            elif e[0] not in i and e[1] not in i:
                cl+=[e,]
                T+=[e,]
                J=0
                break
            else:
                J=0
                break

        ind=E3.index(e)
        del E3[ind]
        del E4[ind]
        if J!=0: del cl[cl.index(j)]

    for i in T: w+=E2[E1.index(i)]

    return T, w
Kr_A(Ed)
```

    ([['J', 'G'],
      ['G', 'F'],
      ['A', 'F'],
      ['G', 'C'],
      ['H', 'D'],
      ['A', 'E'],
      ['I', 'J'],
      ['D', 'C'],
      ['A', 'B']],
     21)

#### Depth-First Search Algoritm

```python
Ed=[['A', 'B'], ['G', 'A'], ['G', 'F'], ['F', 'B'], ['F', 'C'], ['C', 'G'], ['C', 'D'], ['C', 'E'], ['F', 'H'], ['H', 'B'], ['B', 'J'], ['J', 'H'], ['H', 'I'], ['I', 'F']]

#Ed=[['A', 'C'], ['C', 'G'], ['G', 'H'], ['H', 'A'], ['G', 'F'], ['F', 'C'], ['H', 'F'], ['F', 'B'], ['B', 'D'], ['D', 'E'], ['E', 'F'], ['E', 'B']]

#Ed=[['A', 'B'], ['B', 'E'], ['E', 'A'], ['E', 'C'], ['C', 'D'], ['B', 'F'], ['F', 'G'], ['E', 'G'], ['D', 'H'], ['H', 'C'], ['C', 'I'], ['I', 'J'], ['J', 'D']]
st='A'

def DFA(Ed, st):
    V=list(set(flatten(Ed)))
    L={'A':[1, '-']}
    T=[]
    k=2
    while len(L)!=len(V):
        temp=[]
        t=[]
        for i in Ed:
            if st in i: temp+=[i,]
        for i in temp:
            if i[0]==st and i[1] not in L: t+=[i[1], ]
            elif i[1]==st and i[0] not in L: t+=[i[0], ]

        if len(t)!=0:
            v=min(t)
            e=[v, st] if [v, st] in temp else [st, v]

            L[v]=[k, st]
            T+=[e, ]
            del Ed[Ed.index(e)]
            st=v
            k+=1
        else:
            st=L[st][1]
            if st=='A': break

    return L, T if len(L)==len(V) else "No spanning tree"

DFA(Ed, st)
```

    ({'A': [1, '-'],
      'B': [2, 'A'],
      'F': [3, 'B'],
      'C': [4, 'F'],
      'D': [5, 'C'],
      'E': [6, 'C'],
      'G': [7, 'C'],
      'H': [8, 'F'],
      'I': [9, 'H'],
      'J': [10, 'H']},
     [['A', 'B'],
      ['F', 'B'],
      ['F', 'C'],
      ['C', 'D'],
      ['C', 'E'],
      ['C', 'G'],
      ['F', 'H'],
      ['H', 'I'],
      ['J', 'H']])

```python
#Ed=[['A', 'B'], ['A', 'C'], ['A', 'D'], ['D', 'C'], ['C', 'B']]
Ed=[['A', 'B'], ['B', 'D'], ['D', 'A'], ['C', 'F'], ['F', 'E']] # ['B', 'E'] is a bridge
st='A'
DFA(Ed, st)

```

    ({'A': [1, '-'], 'B': [2, 'A'], 'D': [3, 'B']}, 'No spanning tree')

#### Preorder Traversal Algorithm

```python
#Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'E', 'r'], ['B', 'D', 'l'], ['D', 'F', 'r'], ['E', 'G', 'l']]

#Ex 5.5, 17
Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'D', 'l'], ['B', 'E', 'r'], ['D', 'G', 'l'], ['G', 'L', 'l'], ['E', 'I', 'r'], ['E', 'H', 'l'], ['H', 'M', 'l'], ['I', 'N', 'l'], ['C', 'F', 'r'], ['F', 'K', 'r'], ['F', 'J', 'l'], ['K', 'Q', 'r'], ['J', 'O', 'l'], ['J', 'P', 'r']]
##'r' means right child, 'l' means left child and ['A', 'B', 'l'] means 'A' is the parent and 'B' is the left child of 'A'

def Preo_trav(Ed):
    E1=[]
    E2=[]
    for i in Ed:
        E1+=[i[:2], ]
        E2+=[i[2], ]

    V=list(set(flatten(E1)))

    # we need to determine the root, i.e, the node which has indegree=0
    In=0
    root=''
    for v in V:
        for i in E1:
            if i[1]==v:In+=1
        if In==0:
            root=v
            break
        else: In=0

    st=root
    L={st:[1, '-']}
    k=2

    while len(L)!=len(V):
        t1=[]
        t2=[]
        for i in E1:
            if i[0]==st and i[1] not in L:
                t1+=[i, ]
                t2+=[E2[E1.index(i)], ]

        if len(t1)==2:
            ind=t2.index('l')
            e=t1[ind]
            L[e[1]]=[k, e[0]]
            st=e[1]
            k+=1
            ind1=E1.index(e)
            del E1[ind1]
            del E2[ind1]

        elif len(t2)==1:
            e=t1[0]
            L[e[1]]=[k, e[0]]
            st=e[1]
            k+=1
            ind1=E1.index(e)
            del E1[ind1]
            del E2[ind1]

        else: st=L[st][1]

    l={}
    print(L)
    for i in L: l[L[i][0]]=i
    return l

Preo_trav(Ed)

```

    {'A': [1, '-'], 'B': [2, 'A'], 'D': [3, 'B'], 'G': [4, 'D'], 'L': [5, 'G'], 'E': [6, 'B'], 'H': [7, 'E'], 'M': [8, 'H'], 'I': [9, 'E'], 'N': [10, 'I'], 'C': [11, 'A'], 'F': [12, 'C'], 'J': [13, 'F'], 'O': [14, 'J'], 'P': [15, 'J'], 'K': [16, 'F'], 'Q': [17, 'K']}

    {1: 'A',
     2: 'B',
     3: 'D',
     4: 'G',
     5: 'L',
     6: 'E',
     7: 'H',
     8: 'M',
     9: 'I',
     10: 'N',
     11: 'C',
     12: 'F',
     13: 'J',
     14: 'O',
     15: 'P',
     16: 'K',
     17: 'Q'}

#### Postorder Traversal Algorithm

```python
#Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'E', 'r'], ['B', 'D', 'l'], ['D', 'F', 'r'], ['E', 'G', 'l']]

Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'D', 'l'], ['B', 'E', 'r'], ['D', 'G', 'l'], ['G', 'L', 'l'], ['E', 'I', 'r'], ['E', 'H', 'l'], ['H', 'M', 'l'], ['I', 'N', 'l'], ['C', 'F', 'r'], ['F', 'K', 'r'], ['F', 'J', 'l'], ['K', 'Q', 'r'], ['J', 'O', 'l'], ['J', 'P', 'r']]

def Posto_trav(Ed):
    E1=[]
    E2=[]
    for i in Ed:
        E1+=[i[:2], ]
        E2+=[i[2], ]

    V=list(set(flatten(E1)))

    # we need to determine the root, i.e, the node which has indegree=0
    In=0
    root=''
    for v in V:
        for i in E1:
            if i[1]==v:In+=1
        if In==0:
            root=v
            break
        else: In=0

    st=root
    L1={st:'-'}
    L2={}

    k=1
    while len(L2)!=len(V):
        t1=[]
        t2=[]
        for i in E1:
            if i[0]==st and i[1] not in L2:
                t1+=[i, ]
                t2+=[E2[E1.index(i)], ]

        if len(t2)==2:
            ind=t2.index('l')
            e=t1[ind]
            L1[e[1]]=st
            st=e[1]

        elif len(t2)==1:
            e=t1[0]
            L1[e[1]]=st
            st=e[1]

        else:
            L2[st]=k
            e=[L1[st], st]
            if e[0]=='-':
                L2[e[1]]=k
                break
            st=L1[st]
            ind1=E1.index(e)
            del E1[ind1]
            del E2[ind1]
            k+=1

    return(L2)

Posto_trav(Ed)
```

    {'L': 1,
     'G': 2,
     'D': 3,
     'M': 4,
     'H': 5,
     'N': 6,
     'I': 7,
     'E': 8,
     'B': 9,
     'O': 10,
     'P': 11,
     'J': 12,
     'Q': 13,
     'K': 14,
     'F': 15,
     'C': 16,
     'A': 17}

#### Inorder Traversal Algorithm

```python
#Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'E', 'r'], ['B', 'D', 'l'], ['D', 'F', 'r'], ['E', 'G', 'l']]

Ed=[['A', 'B', 'l'], ['A', 'C', 'r'], ['B', 'D', 'l'], ['B', 'E', 'r'], ['D', 'G', 'l'], ['G', 'L', 'l'], ['E', 'I', 'r'], ['E', 'H', 'l'], ['H', 'M', 'l'], ['I', 'N', 'l'], ['C', 'F', 'r'], ['F', 'K', 'r'], ['F', 'J', 'l'], ['K', 'Q', 'r'], ['J', 'O', 'l'], ['J', 'P', 'r']]

def Ino_trav(Ed):
    E1=[]
    E2=[]
    for i in Ed:
        E1+=[i[:2], ]
        E2+=[i[2], ]

    V=list(set(flatten(E1)))

    # we need to determine the root, i.e, the node which has indegree=0
    In=0
    root=''
    for v in V:
        for i in E1:
            if i[1]==v:In+=1
        if In==0:
            root=v
            break
        else: In=0

    st=root
    L1={st:'-'}
    L2={}
    k=1

    while len(L2)!=len(V):
        t1=[]
        t2=[]
        for i in E1:
            if i[0]==st and i[1] not in L2:
                t1+=[i, ]
                t2+=[E2[E1.index(i)], ]

        if len(t2)==2:
            ind=t2.index('l')
            e=t1[ind]
            L1[e[1]]=st
            st=e[1]

        elif len(t2)==1:
            if t2[0]=='l':
                e=t1[0]
                L1[e[1]]=st
                st=e[1]

            else:
                e=t1[0]
                L2[e[0]]=k
                L1[e[1]]=st
                st=e[1]
                k+=1

        else:
            if st in L2:
                st=L1[st]
            else:
                L2[st]=k
                e=[L1[st], st]
                if e[0]=='-':
                    L2[e[1]]=k
                    break
                st=L1[st]
                k+=1
    return(L2)

Ino_trav(Ed)
```

    {'L': 1,
     'G': 2,
     'D': 3,
     'B': 4,
     'M': 5,
     'H': 6,
     'E': 7,
     'N': 8,
     'I': 9,
     'A': 10,
     'C': 11,
     'O': 12,
     'J': 13,
     'P': 14,
     'F': 15,
     'K': 16,
     'Q': 17}

#### Huffman's Optimal Binary Tree Algorithm:
##### For nonnegative real numbers $w_1$, $w_2$, ..., $w_k$, where $k>=2$, this algorithm constructs an optimal binary tree for the weights $w_1$, $w_2$, ..., $w_k$. In the algorithm a vertex is referred to by its label

```python
#S=[2, 3, 4, 7, 8]
#S=[2, 4, 5, 6]
#S=[2, 3, 5, 5, 6]
#S=[10, 12, 13, 16, 17, 17]

#Ex 5.6, 33
S=[1, 4, 9, 16, 25, 36]

def HOBT(S):
    S.sort()
    S1=S[:]
    T=[]
    while len(S)!=1:
        m1=min(S1)
        del S1[S1.index(m1)]
        m2=min(S1)
        del S1[S1.index(m2)]
        root=m1+m2

        if m1!=m2:
            ind1=S.index(m1)
            ind2=S.index(m2)
        else:
            ind1=S.index(m1)
            S[ind1]+=1
            ind2=S.index(m2)
            S[ind1]-=1

        i1=ind1 if ind1<=ind2 else ind2
        i2=ind1 if i1==ind2 else ind2
        T+=[[root, S[i1], 'l'], ]
        T+=[[root, S[i2], 'r'], ]
        S[i1]=root
        del S[i2]
        S1=S[:]
        S.sort()
        S1.sort()
    return T

HOBT(S)
```

    [[5, 1, 'l'],
     [5, 4, 'r'],
     [14, 5, 'l'],
     [14, 9, 'r'],
     [30, 14, 'l'],
     [30, 16, 'r'],
     [55, 25, 'l'],
     [55, 30, 'r'],
     [91, 36, 'l'],
     [91, 55, 'r']]

```python
HOBT([32, 28, 20, 4, 1])
```

    [[5, 1, 'l'],
     [5, 4, 'r'],
     [25, 5, 'l'],
     [25, 20, 'r'],
     [53, 25, 'l'],
     [53, 28, 'r'],
     [85, 32, 'l'],
     [85, 53, 'r']]

#### Binary Search Tree Construction Algorithm

##### This algorithm constructs a binary search tree in which the vertices are labeled $a_1$, $a_2$, ..., $a_n$, where $a_1$, $a_2$, ..., $a_n$ are distinct and $n>=2$. In the algorithm, the vertex is referred to by its label

```python
#a=[5, 9, 8, 1, 2, 4, 10, 6]

#Ex. 5.6, 51
a=[14, 17, 3, 6, 15, 1, 20, 2, 5, 10, 18, 7, 16]

def BSTC(a):
    n=len(a)
    root=a[0]
    T=[]

    k=1
    while k<n:
        V=root
        while True:
            temp=[]
            l=0
            r=0
            for i in T:
                if i[0]==V:
                    if i[2]=='l':l+=1
                    else: r+=1
                    temp+=[i, ]

            if a[k]<V:
                if l==0:
                    T+=[[V, a[k], 'l'], ]
                    break
                else: V=temp[0][1] if temp[0][2]=='l' else temp[1][1]
            elif a[k]>V:
                if r==0:
                    T+=[[V, a[k], 'r'], ]
                    break
                else: V=temp[0][1] if temp[0][2]=='r' else temp[1][1]

        k+=1
    return (T)

BSTC(a)
```

    [[14, 17, 'r'],
     [14, 3, 'l'],
     [3, 6, 'r'],
     [17, 15, 'l'],
     [3, 1, 'l'],
     [17, 20, 'r'],
     [1, 2, 'r'],
     [6, 5, 'l'],
     [6, 10, 'r'],
     [20, 18, 'l'],
     [10, 7, 'l'],
     [15, 16, 'r']]

#### Binary Search Tree Search Algorithm

```python
T=[[5, 9, 'r'], [9, 8, 'l'], [5, 1, 'l'], [1, 2, 'r'], [2, 4, 'r'], [9, 10, 'r'], [8, 6, 'l']]
s=7

def BSTS(T, s):
    """T is the binary search tree, s is the element to be searched"""
    E1=[]
    E2=[]
    for i in T:
        E1+=[i[:2], ]
        E2+=[i[2], ]

    V=list(set(flatten(E1)))

    # we need to determine the root, i.e, the node which has indegree=0
    In=0
    root=0
    for v in V:
        for i in E1:
            if i[1]==v:In+=1
        if In==0:
            root=v
            break
        else: In=0

    v=root

    while True:
        if s==v: return True

        t1=[]
        t2=[]
        for i in range(len(E1)):
            if E1[i][0]==v:
                t1+=[E1[i], ]
                t2+=[E2[i], ]

        if len(t2)==2:
            if s>v:
                ind=t2.index('r')
                e=t1[ind]
                v=e[1]
            elif s<v:
                ind=t2.index('l')
                e=t1[ind]
                v=e[1]

        elif len(t2)==1:
            if s<v:
                if t2[0]=='l':
                    e=t1[0]
                    v=e[1]
                elif t2[0]=='r': return False
            elif s>v:
                if t2[0]=='l': return False
                elif t2[0]=='r':
                    e=t1[0]
                    v=e[1]
        else: return False

BSTS(T, s)

```

    False

### Chapter 6

#### Independent Set Algorithm (A Matching Algorithm)

```python
A=np.array((['1', 0, 1, 0, 0, 1, 0], [0, 0, '1', 1, 1, 0, 1], [1, 0, 1, 0, 0, 0, 0], [1, 0, 0, 0, 0, '1', 0], [0, '1', 0, 0, 1, 0, 1], [0, 0, 1, 0, 0, 1, 0]), dtype=object)

#Ex. 6.3, 5
#A=np.array(([0, '1', 0, 1], ['1', 1, 0, 0], [0, 0, '1', 1], [1, 1, 1, 0]), dtype=object)

#Ex. 6.3, 7
#A=np.array((['1', 1, 0, 1, 1], [1, 0, 0, 0, '1'], [0, '1', 0, 1, 0], [1, 1, 0, 0, 1]), dtype=object)

#A=np.array((['1', 0, 1, 1], [0, '1', 0, 0], [1, 1, 0, 0], [0, 1, 0, 0]), dtype=object)

#Ex. 6.3, 9
#A=np.array((['1', 1, 1, 1, 1], [1, 0, 0, 0, 0], [0, '1', 0, 0, 0], [1, 1, 0 ,0, 0], [1, 0, '1', 0, 1]), dtype=object)

def ISA(A):
    row=len(A)
    col=len(A[0])

    while True:
        breakage=0
        cln=np.array([[0]]*row)
        A=np.append(A, cln, axis=1)
        A=np.append(A, [[0]*(col+1)], axis=0)
        for i in range(col):
            x=A[:, i]
            if list(x).count(1)>0 and list(x).count('1')==0: A[row, i]='#'

        while True:
            for i in range(col):
                if A[row, i]!=0:
                    if '/' not in A[row, i]:
                        x=A[:row, i]
                        for j in range(row):
                            if x[j]==1 and A[j, col]==0: A[j, col]=str(i)
                        A[row, i]+='/'
            A1=np.copy(A)
            print("A1")
            print(A1)

            for i in range(row):
                if A[i, col]!=0:
                    if '/' not in A[i, col]:
                        x=A[i, :col]
                        if list(x).count('1')>0:
                            for j in range(col):
                                if x[j]=='1' and A[row, j]==0: A[row, j]=str(i)
                            A[i, col]+='/'
                        elif list(x).count('1')==0:
                            breakage=1
                            A[i, col]+='!'
                            break
            A2=np.copy(A)
            print("A2")
            print(A2)

            if np.array_equal(A1, A2): break

            if breakage==1:
                j=1
                R=i
                C=int(A[i, col][0])
                ind=[[R, C]]
                while True:
                    RC=ind[-1]
                    if j%2==1:
                        C=RC[1]
                        if '#' in A[row, C]: break
                        R=int(A[row, C][0]) if '/' in A[row, C] else int(A[row, C])
                        ind+=[[R, C], ]
                        j+=1
                    elif j%2==0:
                        R=RC[0]
                        C=int(A[R, col][0]) if '/' in A[R, col] else int(A[R, col])
                        ind+=[[R, C], ]
                        j+=1
                for i in ind: A[i[0], i[1]]='1' if A[i[0], i[1]]==1 else 1
                A=A[:row]
                A=np.delete(A, -1, axis=1)
                print(A)

                break
        if np.array_equal(A1, A2):
            A=A[:row]
            A=np.delete(A, -1, axis=1)
            print("a")
            print(A)
            i, j=np.where(A=='1')
            I=[str(k+1) for k in i]
            J=[chr(k+65) for k in j]
            break
    return [I[i]+J[i] for i in range(len(I))]

ISA(A)

```

    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 '1' 1 1 0 1 '3']
     [1 0 1 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4']
     [0 0 1 0 0 1 0 0]
     [0 0 0 '#/' '#/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 0]
     [0 '4' '1' '#/' '#/' 0 '#/' 0]]
    A1
    [['1' 0 1 0 0 1 0 '2']
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 '2']
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 '2']
     [0 '4/' '1/' '#/' '#/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 '2/']
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 '2!']
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 '2']
     ['0' '4/' '1/' '#/' '#/' 0 '#/' 0]]
    [['1' 0 1 0 0 1 0]
     [0 0 1 '1' 1 0 1]
     [1 0 '1' 0 0 0 0]
     [1 0 0 0 0 '1' 0]
     [0 '1' 0 0 1 0 1]
     [0 0 1 0 0 1 0]]
    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '4']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4']
     [0 0 1 0 0 1 0 0]
     [0 0 0 0 '#/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '4/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 0]
     [0 '4' 0 '1' '#/' 0 '#/' 0]]
    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '4/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 0]
     [0 '4/' 0 '1/' '#/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '4/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 '1' 0 0 1 0 1 '4/']
     [0 0 1 0 0 1 0 0]
     [0 '4/' 0 '1/' '#/' 0 '#/' 0]]
    a
    [['1' 0 1 0 0 1 0]
     [0 0 1 '1' 1 0 1]
     [1 0 '1' 0 0 0 0]
     [1 0 0 0 0 '1' 0]
     [0 '1' 0 0 1 0 1]
     [0 0 1 0 0 1 0]]

    ['1A', '2D', '3C', '4F', '5B']

#### The problem of the courses and professors, Pg 342

```python
A=np.array((['1', 0, 1, 0, 0, 1, 0], [0, 0, '1', 1, 1, 0, 1], [1, 0, 1, 0, 0, 0, 0], [1, 0, 0, 0, 0, '1', 0], [0, 1, 0, 0, '1', 0, 1], [0, 0, 1, 0, 0, 1, 0]), dtype=object)
ISA(A)
```

    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 '1' 1 1 0 1 '3']
     [1 0 1 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1']
     [0 0 1 0 0 1 0 0]
     [0 '#/' 0 '#/' 0 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 0]
     [0 '#/' '1' '#/' '4' 0 '#/' 0]]
    A1
    [['1' 0 1 0 0 1 0 '2']
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 '2']
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 '2']
     [0 '#/' '1/' '#/' '4/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 '2/']
     [0 0 '1' 1 1 0 1 '3/']
     [1 0 1 0 0 0 0 '2!']
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 '2']
     ['0' '#/' '1/' '#/' '4/' 0 '#/' 0]]
    [['1' 0 1 0 0 1 0]
     [0 0 1 '1' 1 0 1]
     [1 0 '1' 0 0 0 0]
     [1 0 0 0 0 '1' 0]
     [0 1 0 0 '1' 0 1]
     [0 0 1 0 0 1 0]]
    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '6']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1']
     [0 0 1 0 0 1 0 0]
     [0 '#/' 0 0 0 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '6/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 0]
     [0 '#/' 0 '1' '4' 0 '#/' 0]]
    A1
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '6/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 0]
     [0 '#/' 0 '1/' '4/' 0 '#/' 0]]
    A2
    [['1' 0 1 0 0 1 0 0]
     [0 0 1 '1' 1 0 1 '6/']
     [1 0 '1' 0 0 0 0 0]
     [1 0 0 0 0 '1' 0 0]
     [0 1 0 0 '1' 0 1 '1/']
     [0 0 1 0 0 1 0 0]
     [0 '#/' 0 '1/' '4/' 0 '#/' 0]]
    a
    [['1' 0 1 0 0 1 0]
     [0 0 1 '1' 1 0 1]
     [1 0 '1' 0 0 0 0]
     [1 0 0 0 0 '1' 0]
     [0 1 0 0 '1' 0 1]
     [0 0 1 0 0 1 0]]

    ['1A', '2D', '3C', '4F', '5E']

#### Hungarian Algorithm

```python
#Ex 6.5
#A=np.array(([3, 6, 3, 5, 3], [7, 3, 5, 8, 5], [5, 2, 8, 6, 2], [8, 3, 6, 4, 4], [0, 0, 0, 0, 0]), dtype=object)
#A=np.array(([6, 2, 5, 8], [6, 7, 1, 6], [6, 3, 4, 5], [5, 4, 3, 4]), dtype=object)
#A=np.array(([3, 6, 3, 5], [7, 3, 5, 8], [5, 2, 8, 6], [8, 3, 6, 4]), dtype=object)
#A=np.array(([3, 5, 5, 3, 8], [4, 6,4, 2, 6], [4, 6, 1, 3, 6], [3, 4, 4, 6, 5], [5, 7, 3, 5, 9]), dtype=object)
A=np.array(([5, 6, 2, 3, 4, 3], [6, 4, 4, 2, 0, 3], [5, 4, 5, 2, 6, 6], [5, 6, 1, 4, 7, 6]), dtype=object)

def reduce(A):
    row=len(A)
    col=len(A[0])
    ind=[]
    a=np.copy(A)

    for r in range(row):
        x=A[r, :]
        if 0 in x:
            c=list(x).index(0)
            ind+=[[r, c], ]
            A[r, :]='x'
            A[:, c]='x'
    for i in ind: a[i[0], i[1]]='0'
    return(a)

def ISA2(A):
    row=len(A)
    col=len(A[0])

    while True:
        breakage=0
        cln=np.array([[0]]*row)
        A=np.append(A, cln, axis=1)
        A=np.append(A, [[0]*(col+1)], axis=0)
        for i in range(col):
            x=A[:, i]
            if list(x).count(0)>0 and list(x).count('0')==0: A[row, i]='#'

        while True:
            for i in range(col):
                if A[row, i]!=0:
                    if '/' not in A[row, i]:
                        x=A[:row, i]
                        for j in range(row):
                            if x[j]==0 and A[j, col]==0: A[j, col]=str(i)
                        A[row, i]+='/'
            A1=np.copy(A)

            for i in range(row):
                if A[i, col]!=0:
                    if '/' not in A[i, col]:
                        x=A[i, :col]
                        if list(x).count('0')>0:
                            for j in range(col):
                                if x[j]=='0' and A[row, j]==0: A[row, j]=str(i)
                            A[i, col]+='/'
                        elif list(x).count('0')==0:
                            breakage=1
                            A[i, col]+='!'
                            break
            A2=np.copy(A)

            if np.array_equal(A1, A2): break

            if breakage==1:
                j=1
                R=i
                C=int(A[i, col][0])
                ind=[[R, C]]
                while True:
                    RC=ind[-1]
                    if j%2==1:
                        C=RC[1]
                        if '#' in A[row, C]: break
                        R=int(A[row, C][0]) if '/' in A[row, C] else int(A[row, C])
                        ind+=[[R, C], ]
                        j+=1
                    elif j%2==0:
                        R=RC[0]
                        C=int(A[R, col][0]) if '/' in A[R, col] else int(A[R, col])
                        ind+=[[R, C], ]
                        j+=1
                for i in ind: A[i[0], i[1]]='0' if A[i[0], i[1]]==0 else 0
                A=A[:row]
                A=np.delete(A, -1, axis=1)
                print(A)

                break
        if np.array_equal(A1, A2):
            break
    return A
def Hungarian(A):
    a=np.copy(A)
    row=len(A)
    col=len(A[0])

    if row<col:
        num=col-row
        for i in range(num):A=np.append(A, [[0]*col], axis=0)
        row=len(A)
        col=len(A[0])
        a=np.copy(A)
    print(A)

    for i in range(row): A[i, :]-=min(A[i, :])
    for j in range(col): A[:, j]-=min(A[:, j])

    while True:
        A=ISA2(reduce(A))
        print(A)

        R1, R2, C1, C2=[], [], [], []
        for i in range(row):
            if A[i, col]==0:R1+=[i, ]
            else:R2+=[i, ]

        for i in range(col):
            if A[row, i]!=0:C1+=[i, ]
            else: C2+=[i, ]

        X=[]
        Y=[]
        for i in R1:
            for j in C1:
                X+=[A[i, j], ]
                Y+=[[i, j], ]
        if len(X)==0: break
        M=min(X)
        print(M)

        Int=[]
        for i in R2:
            for j in C2: Int+=[[i, j], ]

        for i in Y: A[i[0], i[1]]-=M
        for i in Int: A[i[0], i[1]]+=M

        print("Y=",Y)
        print("Int=",Int)

        A=A[:row]
        A=np.delete(A, -1, axis=1)
        I, J=np.where(A=='0')
        ind=[]
        for i in range(len(I)):A[I[i], J[i]]=0
        print(A)

    I, J=np.where(A=='0')
    s=0
    for i in range(len(I)): s+=a[I[i], J[i]]

    return s

Hungarian(A)
```

    [[5 6 2 3 4 3]
     [6 4 4 2 0 3]
     [5 4 5 2 6 6]
     [5 6 1 4 7 6]
     [0 0 0 0 0 0]
     [0 0 0 0 0 0]]
    [[3 4 '0' 1 2 1 0]
     [6 4 4 2 '0' 3 0]
     [3 2 3 '0' 4 4 0]
     [4 5 0 3 6 5 0]
     ['0' 0 0 0 0 0 '5/']
     [0 '0' 0 0 0 0 '5/']
     ['4/' '5/' 0 0 0 '#/' 0]]
    1
    Y= [[0, 0], [0, 1], [0, 5], [1, 0], [1, 1], [1, 5], [2, 0], [2, 1], [2, 5], [3, 0], [3, 1], [3, 5]]
    Int= [[4, 2], [4, 3], [4, 4], [5, 2], [5, 3], [5, 4]]
    [[2 3 0 1 2 0]
     [5 3 4 2 0 2]
     [2 1 3 0 4 3]
     [3 4 0 3 6 4]
     [0 0 1 1 1 0]
     [0 0 1 1 1 0]]
    [[2 3 0 1 2 '0']
     [5 3 4 2 '0' 2]
     [2 1 3 '0' 4 3]
     [3 4 '0' 3 6 4]
     ['0' 0 1 1 1 0]
     [0 '0' 1 1 1 0]]
    [[2 3 0 1 2 '0' 0]
     [5 3 4 2 '0' 2 0]
     [2 1 3 '0' 4 3 0]
     [3 4 '0' 3 6 4 0]
     ['0' 0 1 1 1 0 0]
     [0 '0' 1 1 1 0 0]
     [0 0 0 0 0 0 0]]

    6

```python

```


# Aufgabe 3.1

```python
x = 60

erste = (15/2)*x**2 - (x**3)/12
x = 0
zweite = (15/2)*x**2 - (x**3)/12
resultat = erste - zweite
print(resultat)
```

    9000.0

```python
from sympy.solvers import solve
from sympy import Symbol
x = Symbol('x')
solve(8100+1/12*x**3-15/2*x**2, x)

# solve with sympy
```

    [-28.6249720273366 - 0.e-22*I,
     48.2519936604545 + 0.e-19*I,
     70.3729783668821 + 0.e-19*I]

```python
(((5*60**3)-(60**4)/16)-((5*(0**3))-(0**4)/16))/9000
```

    30.0

```python
(5/10-5**2/4000) - (1*0/10-0**2/400)
```

    0.49375

```python
15*(60**2)/2 - 60**3/12
```

    9000.0

```python
x = 10
(0.1*x)-x**2/400
```

    0.75

```python
1/(3/20)
```

    6.666666666666667

```python
20/3
```

    6.666666666666667

```python
import math

math.log(2)/0.04
```

    17.328679513998633

```python
1/0.04
```

    25.0

```python
import numpy as np
from pandas import Series

autoSeries = Series([11.96, 5.03, 67.40,16.07, 31.50,  7.73, 11.10, 22.38])
autoSeries.mean()
#autoSeries.median()
```

    21.64625

```python

```


```python
import numpy as np
import scipy

import qiskit
from qiskit import QuantumRegister, QuantumCircuit, ClassicalRegister, Aer
from qiskit.circuit import ParameterVector, Parameter
from qiskit import algorithms
from qiskit.ignis.verification.tomography import state_tomography_circuits, TomographyFitter, StateTomographyFitter
from qiskit.quantum_info import Statevector, state_fidelity
from qiskit.opflow import I, X, Y, Z
from qiskit.quantum_info.operators import Operator
from qiskit.algorithms.optimizers import SPSA

from qiskit import IBMQ, transpile
from qiskit.providers.aer import AerSimulator

from qiskit.test.mock import FakeGuadalupe

from qiskit_ionq import IonQProvider

import matplotlib.pyplot as plt
plt.style.use('ggplot')
import pandas as pd
```

```python
IBMQ.load_account() # Load account from disk
IBMQ.providers()    # List all available providers
```

    [<AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>,
     <AccountProvider for IBMQ(hub='ibm-q-community', group='qhack-hackathon', project='7-qubit')>,
     <AccountProvider for IBMQ(hub='ibm-q-community', group='qhack-hackathon', project='16-qubit')>]

## Table of contents
- [Experiment 1: Two qubits, one layer](#exp1)
- [Experiment 2: Three qubits, two layers](#exp2)
- [Experiment 3: Generalization](#exp3)
- [Experiment 4: Model capacity](#exp4)

# Simulating collective neutrino oscillation using QAOA algorithm

As stated in the readme, our project aim is to demonstrate the time and space evolution of the set of amplitudes from a Schrodinger equation:

\begin{equation}
|\phi(t)\rangle=e^{-iHt}|\phi_{0}\rangle
\end{equation}

The Hamiltonian from the equation is the Hamiltonian for neutrino flavor evolution in an environment with high denisty of neutrinos which inlcude vacuum and forward-scattering interaction contributions. Let have a closer look into this Hamiltonian

The Hamiltonian that characterizes the system of $N$ interacting neutrinos (each represented by a qubit) is given by
\begin{equation}
H = \sum_{k=1}^N \overrightarrow{b} \cdot \overrightarrow{\sigma_k} + \sum_{p<q}^N J_{pq} \overrightarrow{\sigma_p} \cdot \overrightarrow{\sigma_q}
\end{equation}
with the external field $\overrightarrow{b} = (b^x,b^y,b^z) = \left(\sqrt{1-0.925^2}, 0, -0.925\right)$ and the pair coupling matrix $J_{pq} = 1 - \cos(\theta_{pq})$, where $\theta_{pq} = \arccos(0.9) \frac{|p-q|}{N-1}$.

We choose to trotterize $H$ by viewing it in the form of $H = A_1 + A_2 + A_3 + B_1 + B_2 + B_3$, where
\begin{align}
A_1 &= \sum_{k=1}^N b^x \sigma_k^x,
&A_2 &= \sum_{k=1}^N b^y \sigma_k^y,
&A_3 &= \sum_{k=1}^N b^z \sigma_k^z, \\
B_1 &= \sum_{p<q}^N J_{pq} \sigma_p^x \sigma_q^x,
&B_2 &= \sum_{p<q}^N J_{pq} \sigma_p^y \sigma_q^y,
&B_3 &= \sum_{p<q}^N J_{pq} \sigma_p^z \sigma_q^z.
\end{align}

The time evolution can be approximated by
\begin{align}
e^{-iHt} &\approx (e^{-iA_1t/n}e^{-iA_2t/n}e^{-iA_3t/n}e^{-iB_1t/n}e^{-iB_2t/n}e^{-iB_3t/n})^n \\
&= \prod_{i=1}^n e^{-iA_1t/n}e^{-iA_2t/n}e^{-iA_3t/n}e^{-iB_1t/n}e^{-iB_2t/n}e^{-iB_3t/n}
\end{align}
We introduce parameters $\boldsymbol{\alpha},\boldsymbol{\beta},\boldsymbol{\gamma}, \boldsymbol{\delta}, \boldsymbol{\epsilon}, \boldsymbol{\kappa} \in \mathbb{R}^n$ and trotterize the unitary operator as follows
\begin{align}
e^{-iHt} &= \prod_{i=1}^n e^{-iA_1\alpha_i t}e^{-iA_2\beta_i t}e^{-iA_3\gamma_i t}e^{-iB_1 \delta_i t}e^{-iB_2 \epsilon_i t}e^{-iB_3 \kappa_i t} \end{align}
Let's see how to implement these components. We show that the first term can be written as a product of rotations about the $x$-axis.
\begin{align}
e^{-iA_1\alpha_i t} &= e^{-i\alpha_i t\sum_{k=1}^N b^x \sigma_k^x} \\
&\equiv e^{-i\alpha_i t\sum_{k=1}^N \sigma_k^x} \\
&= \prod_{k=1}^N e^{-i\alpha_i t \sigma_k^x} \\
&= \prod_{k=1}^N \text{RX}_k(2\alpha_i t)
\end{align}
In the second line, $b^x$ has been absorbed into the learnable parameters $\alpha_i$. Moreover, the term can be written as a product of smaller exponential terms as in the third line because $\sigma_x$'s commute with each other. This is due to the fact that $[A,B] \equiv AB-BA = 0$ implies $e^A = e^B$. Similarly,
\begin{align}
e^{-iA_2\beta_i t} &= \prod_{k=1}^N \text{RY}_k(2\beta_i t) \\
e^{-iA_3\gamma_i t} &= \prod_{k=1}^N \text{RZ}_k(2\gamma_i t)
\end{align}

The same derivation can be made for two-qubit interactions. In particular,
\begin{align}
e^{-iB_1\delta_i} &= \prod_{p<q}^N e^{-i\delta_i J_{pq} \sigma_p^x \sigma_q^x} \\
&= \prod_{p<q}^N \text{RXX}_k(2 J_{pq} \delta_i t) \\
e^{-iB_2\epsilon_i} &= \prod_{p<q}^N \text{RYY}_k(2 J_{pq} \epsilon_i t) \\
e^{-iB_3\kappa_i} &= \prod_{p<q}^N \text{RZZ}_k(2 J_{pq} \kappa_i t)
\end{align}

With the decomposition, the time evolution be can approximated by a sequence of single- and two-qubit rotation gates. The sequence of those gates contains $6n+1$ parameters, $6n$ out of which are learnables, and the other is the time elapsed of the evolution.

```python
def circuit(num_qubits:int, num_layers:int):
    """
    Construct the variational form
    """
    varform = QuantumCircuit(num_qubits)

    t = Parameter('t')
    alpha = ParameterVector('alpha', num_layers)
    beta = ParameterVector('beta', num_layers)
    gamma = ParameterVector('gamma', num_layers)

    delta = ParameterVector('delta', num_layers)
    eps = ParameterVector('eps', num_layers)
    kappa = ParameterVector('kappa', num_layers)

    const = np.arccos(0.9)
    angles = np.zeros((num_qubits,num_qubits))
    for p in range(num_qubits):
        for q in range(num_qubits):
            angles[p,q] = const * abs(p-q) / (num_qubits-1)
    J = 1 - np.cos(angles)

    for n in range(num_layers):
        for k in range(num_qubits):
            varform.rx(2*alpha[n]*t, k)
        for k in range(num_qubits):
            varform.ry(2*beta[n]*t, k)
        for k in range(num_qubits):
            varform.rz(2*gamma[n]*t, k)

        ## 2-body interactions
        for p in range(num_qubits):
            for q in range(num_qubits):
                if p < q:
                    varform.rxx(2*delta[n]*J[p,q]*t, p, q)

        for p in range(num_qubits):
            for q in range(num_qubits):
                if p < q:
                    varform.ryy(2*eps[n]*J[p,q]*t, p, q)

        for p in range(num_qubits):
            for q in range(num_qubits):
                if p < q:
                    varform.rzz(2*kappa[n]*J[p,q]*t, p, q)
        varform.barrier()

    return varform, t
```

```python
varform, time_param = circuit(num_qubits=2, num_layers=1)
```

```python
print(varform.num_parameters)
print(varform.parameters)
print(varform.draw())
```

    7
    ParameterView([ParameterVectorElement(alpha[0]), ParameterVectorElement(beta[0]), ParameterVectorElement(delta[0]), ParameterVectorElement(eps[0]), ParameterVectorElement(gamma[0]), ParameterVectorElement(kappa[0]), Parameter(t)])
         ┌──────────────────┐┌─────────────────┐┌──────────────────┐»
    q_0: ┤ Rx(2*alpha[0]*t) ├┤ Ry(2*beta[0]*t) ├┤ Rz(2*gamma[0]*t) ├»
         ├──────────────────┤├─────────────────┤├──────────────────┤»
    q_1: ┤ Rx(2*alpha[0]*t) ├┤ Ry(2*beta[0]*t) ├┤ Rz(2*gamma[0]*t) ├»
         └──────────────────┘└─────────────────┘└──────────────────┘»
    «     ┌──────────────────────┐┌────────────────────┐                      ░
    «q_0: ┤0                     ├┤0                   ├─■────────────────────░─
    «     │  Rxx(0.2*delta[0]*t) ││  Ryy(0.2*eps[0]*t) │ │ZZ(0.2*kappa[0]*t)  ░
    «q_1: ┤1                     ├┤1                   ├─■────────────────────░─
    «     └──────────────────────┘└────────────────────┘                      ░

To avoid confusion, denote the unitary obtained by the variational form is $U(\boldsymbol{\theta},t)$, where $\boldsymbol{\theta}$ represent all learnable parameters, in constrast to the true time evolution $e^{-iHt}$. Our goal is to simulate $e^{-iHt}$ over a period of time. We choose the following objective function to capture the consistency in the capability of $U(\boldsymbol{\theta},t)$ to approximate $e^{-iHt}$.
\begin{align}
\max_\boldsymbol{\theta} & f(\boldsymbol{\theta}), & \text{where } & f(\boldsymbol{\theta}) = \int_0^T \left| \langle 0|e^{iHt} U(\boldsymbol{\theta},t) |0\rangle \right|^2 dt
\end{align}
The function is averaged fidelity between the ideal final states and the final states obtain by $U(\boldsymbol{\theta},t)$. We observe through experiments that the variational form has relatively good generalization capability, thus a small number of time marks suffices. In practice, running quantum circuits on a real device outputs a statistic of measurement results. It is possible to reconstruct the density matrix $\rho(\boldsymbol{\theta},t)$ of the final state by apping state tomography. The optimization problem we actually solve is
\begin{align}
\max_\boldsymbol{\theta} & \frac{1}{S} \sum_{s=1}^S \left| \langle 0|e^{iHt_s} U(\boldsymbol{\theta},t_s) |0\rangle \right|^2, & \text{for statevector simulator, or} \\
\max_\boldsymbol{\theta} & \frac{1}{S} \sum_{s=1}^S \left| \langle 0|e^{iHt_s} \rho(\boldsymbol{\theta},t_s) e^{-iHt_s} |0\rangle \right|^2, & \text{for physical devices.}
\end{align}
for a set of evenly spaced time marks $\{t_s\}_{s=1}^S$ from $0$ to $T$.

```python
def exact_hamiltonian(num_qubits:int):
    """
    Compute the exact Hamiltonian
    """

    def pair_coupling_matrix():
        J = np.zeros(shape=(num_qubits, num_qubits))
        for p in range(num_qubits):
            for q in range(num_qubits):
                J[p,q] = 1 - np.cos(np.arccos(0.9)*(np.abs(p-q)/(num_qubits-1)))
        return J

    def sigma_k(k):
        sigma_X = ( (I^k) ^ X ^ (I^(num_qubits-k-1)) ).to_matrix()
        sigma_Y = ( (I^k) ^ Y ^ (I^(num_qubits-k-1)) ).to_matrix()
        sigma_Z = ( (I^k) ^ Z ^ (I^(num_qubits-k-1)) ).to_matrix()
        return np.array([sigma_X, sigma_Y, sigma_Z])

    dim = 2**num_qubits

    b_vector = np.array([np.sqrt(1.0-0.925**2), 0.0, -0.925])
    sigma_vectors = [sigma_k(qubit) for qubit in range(num_qubits)] # num_qubits x 3 x (dim x dim)
    J = pair_coupling_matrix()

    first_term = np.zeros(shape=(dim,dim), dtype=np.complex128)
    for i in range(num_qubits):
        sigma_vec_mul = (b_vector[:, None, None] * sigma_vectors[i])
        first_term += np.sum(sigma_vec_mul, axis=0)

    second_term = np.zeros(shape=(dim,dim), dtype=np.complex128)
    for p in range(num_qubits):
        for q in range(num_qubits):
            if p < q:
                for pauli_idx in range(3):
                    second_term += J[p,q] * (sigma_vectors[p][pauli_idx] @ sigma_vectors[q][pauli_idx])

    return first_term + second_term
```

```python
def exact_unitaries(num_qubits:int, ts:np.ndarray):
    """
    Compute the exact unitary operators exp(-iHt) for various t
    """
    H = exact_hamiltonian(num_qubits)
    operators = []
    for t in ts:
        U = scipy.linalg.expm(-1j * t * H)
        operators.append(Operator(U))
        #print(operators[-1].is_unitary())
    return operators

def exact_states(num_qubits:int, ts:np.ndarray , init_state:Statevector=None):
    """
    Compute the exact output states by applying exp(-iHt) on an initial state for various t
    """
    init_state = init_state or Statevector.from_label('0'*num_qubits)

    unitaries = exact_unitaries(num_qubits, ts)

    output_states = [init_state.evolve(U) for U in unitaries]
    return output_states
```

We perform training experiments with both noiseless (Qiskit Statevector) simulator and noisy simulators. Due to high traffic, it is not possible to use IBM and IonQ's hardwares and online simulators as the optimization routine requires many queries to the cost function which involves running quantum circuits. Instead, the use FakeGuadalupe as our noisy simulator. The optimization algorithm is BFGS for noiseless and SPSA for noisy simulations.

```python
def optimize_simulation(varform:QuantumCircuit, time_param:float, num_time_marks:int,
                        time_max:float=1., noisy_backend:bool=False, verbose=False):

    def cost_fn(x):
        full_circuits = [circ.bind_parameters(x) for circ in bind_circuits]

        if noisy_backend:
            tomo_states = []
            for circ in full_circuits:
                qst = state_tomography_circuits(circ, circ.qubits)
                job_res = qiskit.execute(qst, backend, shots=512).result()
                #job_res = [qiskit.execute(circ, backend, shots=512).result() for circ in qst]

                #job_res = meas_filter.apply(job_res) # apply measurement error mitigation
                tomo_fitter = StateTomographyFitter(job_res, qst)
                rho_fit = tomo_fitter.fit(method='lstsq')
                tomo_states.append(rho_fit)

            fids = []
            for i in range(len(tomo_states)):
                fids.append(state_fidelity(tomo_states[i], ideal_states[i]))
            sum_fid = sum(fids)

            if verbose:
                print("Fidelity sum: ", sum_fid)
            return -sum_fid / len(tomo_states)

        else: # noiseless backend
            circuit_states = [init_state.evolve(full_circ) for full_circ in full_circuits]
            fids = []
            for i in range(len(circuit_states)):
                fids.append(state_fidelity(circuit_states[i], ideal_states[i]))
            sum_fid = sum(fids)

            if verbose:
                print("Fidelity sum: ", sum_fid)

            return -sum_fid / len(circuit_states)

    if noisy_backend:
        backend = FakeGuadalupe()
        #backend = IonQProvider().get_backend("ionq_simulator")
#     else:
#         backend = Aer.get_backend('qasm_simulator')

    start = 0.
    end = time_max
    ts = np.linspace(end,start,num_time_marks, endpoint=False)[::-1]
    if verbose:
        print("Time marks used: ", ts)
    bind_circuits = [varform.bind_parameters({time_param: t}) for t in ts]
    init_state = Statevector.from_label('0'*varform.num_qubits)
    ideal_states = exact_states(varform.num_qubits, ts, init_state)

    if noisy_backend:
        init_params = np.random.rand(varform.num_parameters-1)
        opt = SPSA(maxiter=500)
        point, value, nfev = opt.optimize(num_vars=varform.num_parameters-1, objective_function=cost_fn,initial_point= init_params)
        return point, value

    else:
        init_params = np.random.rand(varform.num_parameters-1)
        options = {'disp':True}
        res = scipy.optimize.minimize(cost_fn, x0=init_params, method='BFGS', tol=10e-4, options=options)
        return res.x, res.fun
```

```python
def simulation(varform, param, ts):
    init_state = Statevector.from_label('0'*varform.num_qubits)
    ideal_states = exact_states(varform.num_qubits, ts, init_state)

    bind_circuits = [varform.bind_parameters({time_param: t}) for t in ts]
    full_circuits = [circ.bind_parameters(param) for circ in bind_circuits]

    circuit_states = [init_state.evolve(circ) for circ in full_circuits]

    backend = FakeGuadalupe()
    tomo_states = []
    for circ in full_circuits:
        qst = state_tomography_circuits(circ, circ.qubits)
        job_res = qiskit.execute(qst, backend, shots=512).result()
        tomo_fitter = StateTomographyFitter(job_res, qst)
        rho_fit = tomo_fitter.fit(method='lstsq')
        tomo_states.append(rho_fit)

    noiseless_fids = []
    noisy_fids = []
    device_fids = []
    for i in range(len(full_circuits)):
        noiseless_fids.append(state_fidelity(circuit_states[i], ideal_states[i]))
        device_fids.append(state_fidelity(circuit_states[i], tomo_states[i]))
        noisy_fids.append(state_fidelity(tomo_states[i], ideal_states[i]))

    return noiseless_fids, noisy_fids, device_fids

def plot(ts, nless_sim, nsy_sim, tomo_fid, min_y, title):
    plt.figure(figsize=(8, 6), dpi=80)
    plt.plot(ts, nless_sim, label='Noiseless simulation')
    plt.plot(ts, nsy_sim, label='Noisy simulation')
    plt.plot(ts, tomo_fid, label='Tomography fidelity', linestyle='dashdot')

    plt.xlabel('Time')
    plt.ylabel('Fidelity')
    plt.ylim(min_y)
    plt.title(title)
    plt.legend()
```

# Experiment 1: Two qubits, one layer
We train the circuit based on fidelity in two time marks $t \in \{0.5,1.0\}$. The time interval for simulation is set to $[0,1]$,

```python
varform, time_param = circuit(num_qubits=2, num_layers=1)
print("Number of learnable parameters: ", varform.num_parameters-1)
print(varform.parameters[:-1])
```

    Number of learnable parameters:  6
    [ParameterVectorElement(alpha[0]), ParameterVectorElement(beta[0]), ParameterVectorElement(delta[0]), ParameterVectorElement(eps[0]), ParameterVectorElement(gamma[0]), ParameterVectorElement(kappa[0])]

```python
# param_opt_noiseless, value_opt_noiseless = optimize_simulation(varform, time_param, 2, noisy_backend=False)
# param_opt_noisy, value_opt_noisy = optimize_simulation(varform, time_param, 2, noisy_backend=True)

# pd.DataFrame(param_opt_noiseless).to_csv("opt_2q_1l_noiseless.csv", header=None, index=None)
# pd.DataFrame(param_opt_noisy).to_csv("opt_2q_1l_noisy.csv", header=None, index=None)
```

```python
param_opt_noiseless = pd.read_csv("opt_2q_1l_noiseless.csv", header=None).to_numpy().flatten()
param_opt_noisy = pd.read_csv("opt_2q_1l_noisy.csv", header=None).to_numpy().flatten()
```

```python
ts = np.linspace(0,1,20)
nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid = simulation(varform, param_opt_noiseless, ts)
nsy_train_nless_sim, nsy_train_nsy_sim, nsy_train_tomo_fid = simulation(varform, param_opt_noisy, ts)
```

    /Users/erio/opt/anaconda3/envs/QML/lib/python3.8/site-packages/qiskit/ignis/verification/tomography/basis/circuits.py:468: DeprecationWarning: The QuantumCircuit.__iadd__() method is being deprecated. Use the compose() (potentially with the inplace=True argument) and tensor() methods which are more flexible w.r.t circuit register compatibility.
      prep += circuit
    /Users/erio/opt/anaconda3/envs/QML/lib/python3.8/site-packages/qiskit/circuit/quantumcircuit.py:942: DeprecationWarning: The QuantumCircuit.extend() method is being deprecated. Use the compose() (potentially with the inplace=True argument) and tensor() methods which are more flexible w.r.t circuit register compatibility.
      return self.extend(rhs)
    /Users/erio/opt/anaconda3/envs/QML/lib/python3.8/site-packages/qiskit/ignis/verification/tomography/basis/circuits.py:478: DeprecationWarning: The QuantumCircuit.__add__() method is being deprecated.Use the compose() method which is more flexible w.r.t circuit register compatibility.
      circ = prep + meas
    /Users/erio/opt/anaconda3/envs/QML/lib/python3.8/site-packages/qiskit/circuit/quantumcircuit.py:933: DeprecationWarning: The QuantumCircuit.combine() method is being deprecated. Use the compose() method which is more flexible w.r.t circuit register compatibility.
      return self.combine(rhs)

```python
plot(ts,nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid, 0.8, "Neutrino evolution simulation - Noiseless training")
plot(ts,nsy_train_nless_sim, nsy_train_nsy_sim, nsy_train_tomo_fid, 0.8, "Neutrino evolution simulation - Noisy training")
```

# Experiment 2: Three qubits, two layers
We train the circuit based on fidelity in four time marks $t \in \{0.5,1.0,1.5,2.0\}$. The time interval for simulation is set to $[0,2]$,

```python
varform, time_param = circuit(num_qubits=3, num_layers=2)
print("Number of learnable parameters: ", varform.num_parameters-1)
print(varform.parameters[:-1])
```

    Number of learnable parameters:  12
    [ParameterVectorElement(alpha[0]), ParameterVectorElement(alpha[1]), ParameterVectorElement(beta[0]), ParameterVectorElement(beta[1]), ParameterVectorElement(delta[0]), ParameterVectorElement(delta[1]), ParameterVectorElement(eps[0]), ParameterVectorElement(eps[1]), ParameterVectorElement(gamma[0]), ParameterVectorElement(gamma[1]), ParameterVectorElement(kappa[0]), ParameterVectorElement(kappa[1])]

```python
# param_opt_noiseless, value_opt_noiseless = optimize_simulation(varform, time_param, 4, time_max=2, noisy_backend=False)
# param_opt_noisy, value_opt_noisy = optimize_simulation(varform, time_param, 4, time_max=2, noisy_backend=True)

# pd.DataFrame(param_opt_noiseless).to_csv("opt_3q_2l_noiseless.csv", header=None, index=None)
# pd.DataFrame(param_opt_noisy).to_csv("opt_3q_2l_noisy.csv", header=None, index=None)
```

```python
param_opt_noiseless = pd.read_csv("opt_3q_2l_noiseless.csv", header=None).to_numpy().flatten()
param_opt_noisy = pd.read_csv("opt_3q_2l_noisy.csv", header=None).to_numpy().flatten()
```

```python
ts = np.linspace(0,2,20)
nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid = simulation(varform, param_opt_noiseless, ts)
nsy_train_nless_sim, nsy_train_nsy_sim, nsy_train_tomo_fid = simulation(varform, param_opt_noisy, ts)
```

```python
plot(ts,nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid, 0.4, "Neutrino evolution simulation - Noiseless training")
plot(ts,nsy_train_nless_sim, nsy_train_nsy_sim, nsy_train_tomo_fid, 0.4, "Neutrino evolution simulation - Noisy training")
```

# Experiment 3: Generalization
We show that getting the model to learn the optimal parameters for a few time marks suffices. In particular, we let it optimize the average fidelity at $2S$ time marks $t \in \{0.5,1.0,\dots,S-0.5,S\}$ in the time interval $[0,S]$.
The optimized model can simulate the exact time evolution over the entire period with high precision.

```python
varform, time_param = circuit(num_qubits=4, num_layers=4)
print("Number of learnable parameters: ", varform.num_parameters-1)
print(varform.parameters[:-1])
```

    Number of learnable parameters:  24
    [ParameterVectorElement(alpha[0]), ParameterVectorElement(alpha[1]), ParameterVectorElement(alpha[2]), ParameterVectorElement(alpha[3]), ParameterVectorElement(beta[0]), ParameterVectorElement(beta[1]), ParameterVectorElement(beta[2]), ParameterVectorElement(beta[3]), ParameterVectorElement(delta[0]), ParameterVectorElement(delta[1]), ParameterVectorElement(delta[2]), ParameterVectorElement(delta[3]), ParameterVectorElement(eps[0]), ParameterVectorElement(eps[1]), ParameterVectorElement(eps[2]), ParameterVectorElement(eps[3]), ParameterVectorElement(gamma[0]), ParameterVectorElement(gamma[1]), ParameterVectorElement(gamma[2]), ParameterVectorElement(gamma[3]), ParameterVectorElement(kappa[0]), ParameterVectorElement(kappa[1]), ParameterVectorElement(kappa[2]), ParameterVectorElement(kappa[3])]

```python
# param_opt_noiseless, value_opt_noiseless = optimize_simulation(varform, time_param, 6, time_max=3., noisy_backend=False, verbose=True)
# pd.DataFrame(param_opt_noiseless).to_csv("opt_4q_4l_noiseless.csv", header=None, index=None)
```

```python
param_opt_noiseless = pd.read_csv("opt_4q_4l_noiseless.csv", header=None).to_numpy().flatten() # avg fid = 0.9999949
ts = np.linspace(0,3.0,20)
nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid = simulation(varform, param_opt_noiseless, ts)
```

```python
plot(ts,nless_train_nless_sim, nless_train_nsy_sim, nless_train_tomo_fid, 0., "Neutrino evolution simulation - Noiseless training")
```

# Experiment 4: Model capacity
In this experiment, we investigate the variational form in its ability to approximate the exact time evolution over a long period. Only noiseless training is performed due to the time constraint of the hackathon.

We fix a configuration with $4$ qubits and $4$ layers. We want to find the longest time interval that the variational form still performs well. The number of time marks will be twice the length of the time interval, i.e. if the time interval is $[0,3]$, then there are 6 time marks $t \in \{0.5,1,\dots,2.5,3.0\}$.

```python
varform, time_param = circuit(num_qubits=4, num_layers=4)
print("Number of learnable parameters: ", varform.num_parameters-1)
print(varform.parameters[:-1])
```

    Number of learnable parameters:  24
    [ParameterVectorElement(alpha[0]), ParameterVectorElement(alpha[1]), ParameterVectorElement(alpha[2]), ParameterVectorElement(alpha[3]), ParameterVectorElement(beta[0]), ParameterVectorElement(beta[1]), ParameterVectorElement(beta[2]), ParameterVectorElement(beta[3]), ParameterVectorElement(delta[0]), ParameterVectorElement(delta[1]), ParameterVectorElement(delta[2]), ParameterVectorElement(delta[3]), ParameterVectorElement(eps[0]), ParameterVectorElement(eps[1]), ParameterVectorElement(eps[2]), ParameterVectorElement(eps[3]), ParameterVectorElement(gamma[0]), ParameterVectorElement(gamma[1]), ParameterVectorElement(gamma[2]), ParameterVectorElement(gamma[3]), ParameterVectorElement(kappa[0]), ParameterVectorElement(kappa[1]), ParameterVectorElement(kappa[2]), ParameterVectorElement(kappa[3])]

```python
# max_time_length = 6
# param_opt_list = []
# value_opt_list = []
# for n in range(1,max_time_length+1):
#     print(f'Time interval length = {n}, Num. time marks = {2*n}')

#     best_param_opt = 0
#     best_value_opt = 0
#     for _ in range(3):
#         param_opt_noiseless, value_opt_noiseless = optimize_simulation(varform, time_param, 2*n, time_max=n, noisy_backend=False)
#         if value_opt_noiseless < best_value_opt:
#             best_value_opt = value_opt_noiseless
#             best_param_opt = param_opt_noiseless
#     param_opt_list.append(best_param_opt)
#     value_opt_list.append(best_value_opt)
#     pd.DataFrame(np.array(value_opt_list)).to_csv("opt_val_time_length.csv", header=None, index=None)
```

```python
value_opt_list = pd.read_csv("opt_val_time_length.csv", header=None).to_numpy().flatten()

plt.figure(figsize=(8, 6), dpi=80)
plt.plot(list(range(1,len(value_opt_list)+1)), -np.around(value_opt_list,decimals=3))
plt.xlabel('Length of time interval')
plt.ylabel('Highest average fidelity (Best of 3)')
plt.ylim(0.6)
plt.title("Capacity of the variational model over long time lengths")
plt.show()

```


```python
from sympy import *
def bvp(x_0: float, y_0: float, x_n: float, y_n: float, F: function):
    x = Symbol('x')
    y = Function('y')(x)
    F = F(x)
    ode = Eq(y.diff(x, 2) + F * y, 0)
    ics = {
        y.subs(x, x_0): y_0,
        y.subs(x, x_n): y_n,
    }
    Y = dsolve(ode, y, ics=ics).rhs.evalf()
    return {'x': x, 'y': y}, {'Y': Y, 'F': F}
```

```python
import numpy as np
dtype = np.float32
def to_arrays(xlim: dict, x: Symbol, y: Function, Y: Function, F: Function):
    x_0, x_n = xlim['x_0'], xlim['x_n']
    n = (x_n - x_0) * 10
    aX = np.linspace(x_0, x_n, n, dtype=dtype)
    aY = np.array([Y.subs(x, i).evalf() for i in aX], dtype=dtype)
    aF = np.array([F.subs(x, i).evalf() for i in aX], dtype=dtype)
    arrays = {'Y(x)': np.array([aX, aY]), 'F(x)': np.array([aX, aF])}
    return arrays
```

```python
import matplotlib.pyplot as plt
def plot(xlim: dict, arrays: dict):
    fig = plt.figure(figsize=(17, 7))
    ax = fig.gca()
    ax.set_xlabel('X')
    ax.set_ylabel('Y')
    ax.set_xlim([xlim['x_0'], xlim['x_n']])
    ax.grid()
    for label, value in arrays.items():
        ax.plot(*value, label=label)
    ax.legend()
    return fig
```

```python
xlim = {'x_0': 0, 'x_n': 15}
conditions = {
    'y_0': 1, 'y_n': 1,
    'F': lambda x: -1 + 2 * x,
}
conditions.update(xlim)
variables, functions = bvp(**conditions)
functions['Y'].subs(variables['x'], 2).evalf()
```

$\displaystyle 1.97163924964119$

```python
arrays = to_arrays(xlim, **variables, **functions)
fig = plot(xlim, arrays)
```

```python
from multiprocessing import Pool
from tqdm import tqdm

def iter_samples():
    np.random.seed(0)
    n_samples = 1000
    B = np.linspace(-500 * np.pi, 500 * np.pi, n_samples)
    xlim = {'x_0': 0, 'x_n': 10}
    for i in tqdm(range(n_samples)):
        conditions = {
            'kind': 1,
            'y_0': 1, 'y_n': 2,
            'G': lambda x: 0 * x,
        }
        conditions.update(xlim)
        # A, B = koef[i]
        conditions['F'] = lambda x: B[i] * x
        yield i, conditions

def save_sample(args):
    i, conditions = args
    print(i)
    # variables, functions = bvp(**conditions)
    # print(f'{i} - solved:', functions['Y'].subs(variables['x'], 2).evalf())
    # arrays = to_arrays(xlim, **variables, **functions)
    # inputs = arrays['Y(x)'][1]
    # targets = arrays['F(x)'][1]
    # samples = np.vstack((inputs, targets))
    # fname = f'dataset/{i}_sample.'
    # np.save(fname + 'npy', samples)
    # fig = plot(xlim, arrays)
    # fig.savefig(fname + 'png')
    # print(f'{i} - saved!')

with Pool(3) as pool:
    pool.map(save_sample, iter_samples())
```

```python

```


```python
%reset -f
import numpy as np
import matplotlib.pyplot as plt

from ipywidgets import interact_manual
import ipywidgets as wd

from os import system

''' GLOBAL CONSTANTS '''
PI = 3.141592
BOLTZMANN = 1.380649E-23 #N·m/K
AVOGADROS = 6.02214076E+23 #mol⁻¹
TEMP = 25 + 273.15 #K
g = 9.81 #m/s²

DIELECTRIC_WATER = 80
PERMITIVITY_FREE_SPACE = 8.854E-12 #C/m·J
ELEMENTARY_CHARGE = 1.60219E-19 #C
```

# DLVO profiles

\begin{equation}
    \Phi_T(x) = \Phi_{\rm dl}(x) + \Phi_{\rm vdW}(x) + \Phi_{\rm Born}(x)
\end{equation}

- $\Phi_T(x)$ : total potential [J]
- $\Phi_{\rm dl}$ : double layer interaction potential [J]
- $\Phi_{\rm vdW}$ : van der Waals interaction [J]
- $\Phi_{\rm Born}$ : Born interaction [J]

```python
def DLVO_T(Phi_dl,Phi_vdw,Phi_born):
    return Phi_dl + Phi_vdw + Phi_born
```

## Double layer interaction $\Phi_{\rm dl}$

\begin{equation}
    \Phi_{\rm dl}(x) =
        \pi
        \epsilon
        \epsilon_0
        r_p
        \left[
            2
            \psi_p
            \psi_c
            \ln{\left(
                \dfrac{1+ \exp{\left(-\kappa x\right)}}
                    {1 - \exp{\left(-\kappa x\right)}}
                \right)
                }
            +
            \left( \psi_p^2 + \psi_c^2 \right)
            \ln{\left(1 - \exp{\left(-2 \kappa x\right)}\right)}
        \right]
\end{equation}

- $\epsilon$ : dielectric constant of water [-]
- $\epsilon_0$ : permitivity of free space [C/m·J]
- $r_p$ : radius of the colloid particle [m]
- $\psi_p$, $\psi_c$ : surface potential of the particle and the collector [V]
- $\kappa$ : inverse Debye-Huckel distance [m⁻¹]

```python
def DLVO_dl(rp,psip,psic,kappa,x):
    C1 = PI * DIELECTRIC_WATER * PERMITIVITY_FREE_SPACE * rp
    C20 = 2 * psip * psic
    C210= 1 + np.exp(-kappa*x)
    C211= 1 - np.exp(-kappa*x)
    C21 = np.log(C210/C211)
    C30 = psip**2 + psic**2
    C31 = np.log(1-np.exp(-2*kappa*x))
    return C1 *((C20*C21) + (C30*C31))
```

### Debye-Huckel lenght $1/\kappa$

\begin{equation}
    \kappa =
    \left(
    \dfrac{2000 I N_A e^2}
        {\epsilon \epsilon_0 K_B T}
    \right)^{1/2}
\end{equation}

- $N_A$ : Avogadro's number [mol⁻¹]
- $e$ : elementary (electron) charge [V]
- $I$ : ionic strenght [mol/L]
- $K_B$ : Boltzmann constant [N·m/K]
- $T$ : absolute temperature [K]

```python
def DebbyHuckel(I):
    return np.power(2000*I*AVOGADROS*ELEMENTARY_CHARGE**2/(DIELECTRIC_WATER*PERMITIVITY_FREE_SPACE*BOLTZMANN*TEMP),0.5)
```

### Ionic Strenght I

\begin{equation}
    I = \dfrac{1}{2}\sum_{i}{c_iz_i^2}
\end{equation}

- $c_i$ concentration of the ion $i$ in the solution [mol/L]
- $z_i$ is the valence of the ion $i$ [-]

```python
def IonicStrenght(c,z):
    return 0.5 * np.sum(np.multiply(c,np.power(z,2.0)))
```

## van der Waals interaction $\Phi_{\rm vdW}$

\begin{equation}
    \Phi_{\rm vdW}(x) = -\dfrac{Ar_p}{6x}\left[1 + \dfrac{14x}{\lambda}\right]^{-1}
\end{equation}

- $A$ : Hamaker constant \[J\]
- $\lambda$ : *characteristic* wavelenght = $10^{-7}$m

```python
CHARACTERISTIC_WAVELENGHT = 1.0E-7 #m
```

```python
def DLVO_vdW(A,rp,x):
    return -A*rp/(6*x)/(1+(14*x/CHARACTERISTIC_WAVELENGHT))
```

## Born interaction $\Phi_{\rm Born}$

\begin{equation}
    \Phi_{\rm Born}(x) = \dfrac{A\sigma^6}{7560}\left[
        \dfrac{8r_p + x}{\left(2r_p + x\right)^7} +
        \dfrac{6r_p - x}{x^7}
    \right]
\end{equation}

- $\sigma$ : Born collision parameter = 5Å [m]

```python
BORN_COLLISION_PARAM = 5E-10 #m
```

```python
def DLVO_Born(A,rp,x):
    return A*(BORN_COLLISION_PARAM**6)/7560 * (((8*rp + x)/(2*rp + x)**7)+((6*rp - x)/(x**7)))
```

## Calculate a potential profile

This is, the evolution of the potential between the particle and the collector.

```python
'''CASE CONSTANTS'''

# Collector-particle sizes [m]
dc,dp = 2.0E-3,100.0E-9
rc,rp = dc/2,dp/2

# Surface charges [V]
psip,psic = -25.0E-3,-25.0E-3

# Hamaker constant
#A = 5.0E-21 # J = N·m
A = 1.0E-20 # J = N·m

# Water chemistry
concentrations, valences = [0.1,0.1],[1,-1]
ionicStrenght = IonicStrenght(concentrations,valences)

# Debye Huckel inverse distance
kappa = DebbyHuckel(ionicStrenght)
```

```python
# Parameters summary
paramSummary = "\
    $d_p$ = {:.1E} m \n\
    $d_c$ = {:.1E} m \n\
    $A$ = {:.1E} J \n\
    $\psi_p$ = {:.1E} V \n\
    $\psi_c$ = {:.1E} V \n\
    $I$ = {} mol/L".format(dp,dc,A,psip,psic,ionicStrenght)
```

```python
x = np.power(10.,np.linspace(-10,-7,100))
```

```python
Phi_DoubleLayer = DLVO_dl(rp,psip,psic,kappa,x)
Phi_vanderWaals = DLVO_vdW(A,rp,x)
Phi_BornInterac = DLVO_Born(A,rp,x)
Phi_TotalIntera = DLVO_T(Phi_DoubleLayer,Phi_vanderWaals,Phi_BornInterac)
```

```python
plt.figure(figsize=(6,5),facecolor="white")
ax1 = plt.subplot(1,1,1)

ax1.plot(x,Phi_DoubleLayer/(BOLTZMANN*TEMP),\
         lw=1.5,c="blue",ls="dashed",\
         label="$\Phi_{\\rm dl}(x)$")

ax1.plot(x,Phi_vanderWaals/(BOLTZMANN*TEMP),\
         lw=1.5,c="red",ls="dashed",\
         label="$\Phi_{\\rm vdW}(x)$")

ax1.plot(x,Phi_BornInterac/(BOLTZMANN*TEMP),\
         lw=1.5,c="green",ls="dashed",\
         label="$\Phi_{\\rm Born}(x)$")

ax1.plot(x,Phi_TotalIntera/(BOLTZMANN*TEMP),\
         lw=2.5,c="gray",zorder=2,\
         label="$\Phi_{\\rm T}(x)$")

## Straight lines
ax1.axhline(y=0.0,\
            ls="dotted",lw=1,c="k")

## Straight spans

## Miscelaneous
ax1.set_yscale("linear")
ax1.set_xscale("log")
ax1.set_ylabel('Interaction energy \t $\dfrac{\Phi}{k_BT}$ [-]')
ax1.set_xlabel('Distance collector - particle [m]')
ax1.set_ylim(-50,50)

## Legend
ax1.legend(loc="best",\
           title="DLVO Interactions",\
           fontsize="large",title_fontsize="large")

ax1.text(1.0E-8,-15,paramSummary,\
        ha="left", va="top")

plt.tight_layout()
plt.show()
```

```python

```

### DLVO interactions function of $d_p$

```python
dpi = np.power(10.,np.array([-9,-8,-7,-6]))
rpi = dpi/2

plt.figure(figsize=(6,5),facecolor="white")
ax1 = plt.subplot(1,1,1)

for rp in rpi:
    Phi_DoubleLayer = DLVO_dl(rp,psip,psic,kappa,x)
    Phi_vanderWaals = DLVO_vdW(A,rp,x)
    Phi_BornInterac = DLVO_Born(A,rp,x)

    Phi_TotalIntera = DLVO_T(Phi_DoubleLayer,Phi_vanderWaals,Phi_BornInterac)

    ax1.plot(x,Phi_TotalIntera/(BOLTZMANN*TEMP),\
         lw=2,zorder=2,\
         label="$d_p = $" + "{:.1E} m".format(rp*2))

## Straight lines
ax1.axhline(y=0.0,ls="dotted",lw=1,c="k")

## Miscelaneous
ax1.set_yscale("linear")
ax1.set_xscale("log")
ax1.set_ylabel('Interaction energy \t $\dfrac{\Phi}{k_BT}$ [-]')
ax1.set_xlabel('Distance collector - particle [m]')
ax1.set_ylim(-50,50)

# Parameters summary
paramSummary = "\
    $d_c$ = {:.1E} m \n\
    $A$ = {:.1E} J \n\
    $\psi_p$ = {:.1E} V \n\
    $\psi_c$ = {:.1E} V \n\
    $I$ = {} mol/L".format(dc,A,psip,psic,ionicStrenght)

## Legend
ax1.legend(loc="best",\
           title="DLVO Interactions $\Phi_{\\rm T}(x)$",\
           fontsize="large",title_fontsize="large")

ax1.text(1.0E-8,-20,paramSummary,\
        ha="left", va="top")

plt.tight_layout()
plt.show()
```

```python
# Collector-particle sizes [m]
dc,dp = 2.0E-3,100.0E-9
rc,rp = dc/2,dp/2

plt.figure(figsize=(6,5),facecolor="white")
ax1 = plt.subplot(1,1,1)

Ai = [5.0E-21,7.5E-21,1.0E-20,1.5E-20]

for A in Ai:
    Phi_DoubleLayer = DLVO_dl(rp,psip,psic,kappa,x)
    Phi_vanderWaals = DLVO_vdW(A,rp,x)
    Phi_BornInterac = DLVO_Born(A,rp,x)

    Phi_TotalIntera = DLVO_T(Phi_DoubleLayer,Phi_vanderWaals,Phi_BornInterac)

    ax1.plot(x,Phi_TotalIntera/(BOLTZMANN*TEMP),\
         lw=2,zorder=2,\
         label="$A = $" + "{:.1E} J".format(A))

## Straight lines
ax1.axhline(y=0.0,ls="dotted",lw=1,c="k")

## Miscelaneous
ax1.set_yscale("linear")
ax1.set_xscale("log")
ax1.set_ylabel('Interaction energy \t $\dfrac{\Phi}{k_BT}$ [-]')
ax1.set_xlabel('Distance collector - particle [m]')
ax1.set_ylim(-50,50)

# Parameters summary
paramSummary = "\
    $d_p$ = {:.1E} m \n\
    $d_c$ = {:.1E} m \n\
    $\psi_p$ = {:.1E} V \n\
    $\psi_c$ = {:.1E} V \n\
    $I$ = {} mol/L".format(dp,dc,psip,psic,ionicStrenght)

## Legend
ax1.legend(loc="best",\
           title="DLVO Interactions $\Phi_{\\rm T}(x)$",\
           fontsize="large",title_fontsize="large")

ax1.text(1.0E-8,-20,paramSummary,\
        ha="left", va="top")

plt.tight_layout()
plt.show()
```

```python
A = 1.0E-20

plt.figure(figsize=(6,5),facecolor="white")
ax1 = plt.subplot(1,1,1)
psi_i = np.array([-50,-40,-30,-20])*1.0E-3

for psi in psi_i:
    psip = psi
    psic = psi
    Phi_DoubleLayer = DLVO_dl(rp,psip,psic,kappa,x)
    Phi_vanderWaals = DLVO_vdW(A,rp,x)
    Phi_BornInterac = DLVO_Born(A,rp,x)

    Phi_TotalIntera = DLVO_T(Phi_DoubleLayer,Phi_vanderWaals,Phi_BornInterac)

    ax1.plot(x,Phi_TotalIntera/(BOLTZMANN*TEMP),\
         lw=2,zorder=2,\
         label="$\psi_i = $" + "{:.1E} V".format(psi))

## Straight lines
ax1.axhline(y=0.0,ls="dotted",lw=1,c="k")

## Miscelaneous
ax1.set_yscale("linear")
ax1.set_xscale("log")
ax1.set_ylabel('Interaction energy \t $\dfrac{\Phi}{k_BT}$ [-]')
ax1.set_xlabel('Distance collector - particle [m]')
ax1.set_ylim(-50,50)

# Parameters summary
paramSummary = "\
    $d_p$ = {:.1E} m \n\
    $d_c$ = {:.1E} m \n\
    $A$ = {:.1E} J \n\
    $I$ = {} mol/L".format(dp,dc,A,ionicStrenght)

## Legend
ax1.legend(loc="best",\
           title="DLVO Interactions $\Phi_{\\rm T}(x)$",\
           fontsize="large",title_fontsize="large")

ax1.text(1.0E-8,-20,paramSummary,\
        ha="left", va="top")

plt.tight_layout()
plt.show()
```

```python
A = 1.0E-20
psip,psic = -25.0E-3,-25.0E-3

plt.figure(figsize=(6,5),facecolor="white")
ax1 = plt.subplot(1,1,1)

I_i = [0.001,0.01,0.1,1.0]

for I in I_i:
    kappa2 = DebbyHuckel(I)
    Phi_DoubleLayer = DLVO_dl(rp,psip,psic,kappa2,x)
    Phi_vanderWaals = DLVO_vdW(A,rp,x)
    Phi_BornInterac = DLVO_Born(A,rp,x)

    Phi_TotalIntera = DLVO_T(Phi_DoubleLayer,Phi_vanderWaals,Phi_BornInterac)

    ax1.plot(x,Phi_TotalIntera/(BOLTZMANN*TEMP),\
         lw=2,zorder=2,\
         label="$I = $" + "{:.1E} mol/L".format(I))

## Straight lines
ax1.axhline(y=0.0,ls="dotted",lw=1,c="k")

## Miscelaneous
ax1.set_yscale("linear")
ax1.set_xscale("log")
ax1.set_ylabel('Interaction energy \t $\dfrac{\Phi}{k_BT}$ [-]')
ax1.set_xlabel('Distance collector - particle [m]')
ax1.set_ylim(-50,50)

# Parameters summary
paramSummary = "\
    $d_p$ = {:.1E} m \n\
    $d_c$ = {:.1E} m \n\
    $A$ = {:.1E} J \n\
    $\psi_p$ = {:.1E} V \n\
    $\psi_c$ = {:.1E} V \n".format(dp,dc,A,psip,psic)

## Legend
ax1.legend(loc="upper right",\
           title="DLVO Interactions $\Phi_{\\rm T}(x)$",\
           fontsize="medium",title_fontsize="large")

ax1.text(1.0E-8,-20,paramSummary,\
        ha="left", va="top")

plt.tight_layout()
plt.show()
```

```python

```

notebooks/.ipynb_checkpoints/DLVO & collision-attachment efficiency-checkpoint.ipynb

notebooks/Concepts/DLVO & collision-attachment efficiency.ipynb