# Home Assignment -- 2

Please, write your solutions within the designated blocks:
```python
...
### BEGIN Solution

# >>> your solution here <<<

### END Solution
...
```

Import visualization and data processing libraries

```python
import numpy as np
import pandas as pd

%matplotlib inline
import matplotlib.pyplot as plt
```

# Part 1 (33 pt.): Convex Programs and Support Vector Models

In this **first two tasks** of this part of the homework you will implement
your own Support Vectors Regression and then test on a simple real dataset.

In the last task you are asked to solve a simple small NLP problem using
support vector classfiers from [scikit-learn](http://scikit-learn.org/stable/modules/svm.html#classification).
If you successfully complete the task, it will give you a ready-to-use and
straighforward baseline for any NLP classification task of moderate size.

### Support Vector Regression

$$\frac{1}{2}\|w\|^2 + C\sum_{i=1}^l(\xi_i + \xi_i') \to \min_{w, b, \xi_i, \xi_i'}\\
(w^Tx_i + b) - y_i \leq \epsilon - \xi_i\\
y_i - (w^Tx_i + b) \leq \epsilon - \xi_i'\\
\xi_i \geq 0\\
\xi_i' \geq 0$$

This problem can be written as empirical risk minimization problem with regularization

#### Equivalent
$$ \frac{1}{2}\|w\|^2 + C\sum_{i=1}^l h(\hat{y}_i, y_i)$$

Where
$$ h(\hat{y}_i, y_i) = \max(0, |\hat{y} - y| - \epsilon ) $$
is the $\epsilon$-insensitive loss, and
$$ \hat{y}_i = (w, x_i) + b $$
is the prediction.

For our experiments we will use "Cars" dataset, which contains information about braking distances for several cars from the 1920s.

```python
cars_info = pd.read_csv('data/cars.csv', index_col=0, dtype=np.float)

X = cars_info.speed.values.reshape(-1, 1)
```

Normalization leads to better convergence

```python
from sklearn.preprocessing import StandardScaler

X_scaler = StandardScaler()

X_train = X_scaler.fit_transform(X)
y_train = np.array(cars_info.dist)
```

```python
ax = plt.gca()
ax.plot(X, y_train, 'o')
ax.set_xlabel('Speed')
ax.set_ylabel('Distance')
```

## Task 1 (11 pt.)

Please implement a function that calculates the current loss value and its gradient
at the specified values of weights $w$, bias $b$, selected $C$ and $\epsilon$.

```python
def calculate_eps_insensetive_loss(X, y, w, b, C=1.0, eps=0.1):
    '''Calculate hinge loss function
    :param: X -- numpy array of shape l x n
    :param: y -- numpy array of shape l with values 1 and -1
    :param: w -- numpy array of shape n
    :param: b -- np.float64
    :param: C -- np.float64
    '''
    # Calculate Loss
    ### BEGIN Solution
    positives= np.abs(X.dot(w)+b-y)-eps >0 #+b-y - eps
    loss = np.linalg.norm(w,ord=2)**2/2+ C* np.sum((np.abs(X.dot(w)+b-y)-eps)[positives])
    ### END Solution

    # Calculate Gradient for hinge loss
    ### BEGIN Solution
    w_grad = w + C * (np.sign(X.dot(w)+b-y)[positives]).dot(X[positives,:])
    b_grad = C* np.sum (np.sign(X.dot(w)+b-y)[positives])
    ### END Solution

    return loss, w_grad, b_grad

#X= np.array([[0,1],[1,1],[1,0]])
#b=np.array([-1,-3,-9])
#y=np.array([-1,1,-1])
#w=np.array([1,2])
#calculate_eps_insensetive_loss(X,y, w, b)
```

Following code finds optimal values for your loss function with simple constant step gradient descend.

NOTE: In order to get better convergence, the modified descent is used.

```python
# Model's metaparameters
eps = 0.1
C = 100

# Gradient descent parameters
step = 1e-4
tolerance = 1e-4

# Optimization logging
converged = False
index = 0
training_loss_history = []
training_loss_iteration = []

# Parameters initialization
w = np.zeros(1)
b = 0
loss, w_grad, b_grad = calculate_eps_insensetive_loss(X_train, y_train, w, b, C, eps)
NN=0

while not converged:
    NN=NN+1
    # Update parameters
    new_w = w - step * w_grad
    new_b = b - step * b_grad
    # Calculate new loss and gradient
    new_loss, w_grad, b_grad = calculate_eps_insensetive_loss(X_train,
                                                    y_train,
                                                    new_w, new_b, C, eps)
    # Check convergence
    if np.abs(loss - new_loss) < tolerance:
        converged = True

    #if (np.abs(loss-new_loss)<1e-2):
    #    step=step*0.9

    b, w, loss = new_b, new_w, new_loss

    # Update history
    index += 1
    if index % 10 == 0:
        training_loss_history.append(new_loss)
        training_loss_iteration.append(index)
```

```python
axis = plt.gca()
axis.plot(training_loss_iteration, training_loss_history)
axis.set_yscale("log")
axis.set_xlabel('Iterations')
axis.set_ylabel('Loss value')
```

We are going calculate predictions for all speeds in range from $0$ to $30$.

```python
x_grid = np.arange(0., 30., 1).reshape(-1, 1)
x_test = X_scaler.transform(x_grid)
```

```python
predictions = x_test @ w + b
```

```python
ax = plt.gca()
ax.plot(x_grid, predictions, label='Predictions')
ax.plot(X, y_train, 'o')
ax.set_xlabel('Speed')
ax.set_ylabel('Distance')
ax.legend(loc='best')
```

## Task 2 (11 pt.)

In this task the goal is to implement the SVR algorithm based on its dual problem version.

### SVR Dual Problem

\begin{equation}
    \begin{aligned}
        & \underset{\alpha', \alpha}{\text{minimize}}
          & & \frac{1}{2}(\alpha' - \alpha)^T  K (\alpha' - \alpha)
              - (\alpha' - \alpha)^T Y
              + \epsilon (\alpha' + \alpha)^T \mathbf{1}
              \,, \\
        & \text{subject to}
          & & (\alpha' - \alpha)^T \mathbf{1} = 0
              \,, \\
        & & & \alpha', \alpha \in [0, C]
              \,.
    \end{aligned}
\end{equation}

### Quadtratic Optimization Problem

\begin{equation}
    \begin{aligned}
        & \underset{x}{\text{minimize}}
          & & \tfrac12 x^T P x + q^T x
              \,, \\
        & \text{subject to}
          & & G x \leq h
              \,, \\
        & & & Ax = b
              \,.
    \end{aligned}
\end{equation}

Please notice that instead of just $\alpha$, like in the classification case, we have
two vectors of coefficients: $\alpha$ and $\alpha'$. As an input space consider using
stacked column vectors
\begin{equation}
    \begin{pmatrix}
        \alpha \\
        \alpha'
    \end{pmatrix}\,.
\end{equation}

**Hint**: You may find function `np.block` very usefull in this task

Now we are going to use **cvxopt** library, which has solvers for different kind
of convex optimization problems. Please notice that installation in Windows can
be a little tricky and painful.

http://cvxopt.org/

For more information about different types of optimization problem you can check
this presentations. This material could be very helpful, however is not necessary
for understatnding this course in Machine Learning.

http://ee364a.stanford.edu/lectures/problems.pdf

### Reformulate the Dual Problem

```python
import numpy
from cvxopt import matrix

def formulate_problem(X, y, kernel, C, epsilon):
    """Formulate quadratic optimization
    problem by defining matrices
    :param: X np.array of size n_elem * n_dim with reaining values
    :param: y np.array of size n_elem with labels 1 and -1
    :kernel: kernel function wich accepts two matrices of objects
    """
    quadratic_problem = {}

    # REMEMBER: all values should be wrapped in `cvxopt.matrix`
    # datatype (you can just use matrix(a) where a is `np.array`).
    # Please be careful cvxopt accepts only FLOAT not INTEGER data

    ### BEGIN Solution
    quadratic_problem['P'] = matrix(np.block([[kernel(X,X),-kernel(X,X)],[-kernel(X,X),kernel(X,X)]]))
    quadratic_problem['q'] = matrix(np.block([[y.reshape((X.shape[0],1))+epsilon],[-y.reshape((X.shape[0],1))+epsilon]]))
    #print(np.block([[y.reshape((X.shape[0],1))+epsilon],[-y.reshape((X.shape[0],1))+epsilon]]))

    quadratic_problem['G'] = matrix(np.block([[np.eye(X.shape[0]*2)],[-np.eye(X.shape[0]*2)]]))
    quadratic_problem['h'] = matrix(np.block([[C*np.ones((X.shape[0]*2,1))],[np.zeros((X.shape[0]*2,1))]]))

    quadratic_problem['A'] = matrix(np.block([np.ones((1,X.shape[0])),-np.ones((1,X.shape[0]))]))
    #print(np.block([np.ones((1,X.shape[0])),-np.ones((1,X.shape[0]))]))
    quadratic_problem['b'] = matrix(np.zeros((1,1)))

    ### END Solution

    return quadratic_problem
```

### Check the Results

The module **metrics.pairwise** in **sklearn** has a very fast and well documented
implementation of kernel matrix calculator. We fix the width of the RBF kernel at
$1.0$ for simplicity.

```python
from sklearn.metrics.pairwise import rbf_kernel
from functools import partial

# the width of the RBF kernel is 1.0
kernel = partial(rbf_kernel, gamma=0.1)
```

**cvxopt** has powerful and versatile solvers for almost every convex
optimization problem: linear, conic, and **quadratic**. We are using
the latter.

```python
# Optimization routine
from cvxopt.solvers import qp

qudratic_problem = formulate_problem(X_train, y_train.astype(float), kernel, C, eps)

results = qp(**qudratic_problem)
```

         pcost       dcost       gap    pres   dres
     0: -1.2634e+04 -2.4630e+05  2e+05  2e-12  5e-15
     1: -3.6309e+04 -7.2365e+04  4e+04  9e-13  7e-15
     2: -5.2798e+04 -5.9730e+04  7e+03  7e-13  6e-15
     3: -5.5424e+04 -5.7411e+04  2e+03  2e-13  7e-15
     4: -5.6356e+04 -5.6677e+04  3e+02  4e-13  6e-15
     5: -5.6510e+04 -5.6553e+04  4e+01  1e-12  7e-15
     6: -5.6532e+04 -5.6534e+04  2e+00  2e-12  7e-15
     7: -5.6533e+04 -5.6533e+04  3e-02  4e-14  9e-15
    Optimal solution found.

Extract optimal point form the results and force **small values** of the coefficients to **zero**.

```python
optimal_stacked_alpha = np.array(results['x']).ravel()

zero_threshold = 1e-6
### BEGIN Solution
# force negligible values to be exactly zero

### END Solution
```

### Prediction

Having a kernel function $K$ and optimal $(\alpha, \alpha')$, implement a function
that computes the dot product $(w \cdot x_i)$:

$$(w \cdot x_{test}) = \sum_{i=1}^l(\alpha_i' - \alpha_i)K(x_i, x_{test}) \,. $$

```python
def kernel_dot_product(X_train, y_train, X_test, kernel, alpha):
    '''Calculate scalar product with vector w
    based on support vectors and dual coefficients
    :param: X_train np.array with train data
    :param: y_train np.array with train labels
    :param: X_test np.array with test data
    :parma: kernel the kernel function
    :param: alpha stacked dual coefficients
    '''
    output = np.zeros(len(X_test))

    ### BEGIN Solution
    output = ((alpha[len(alpha)//2:]-alpha[:len(alpha)//2]).reshape((1,len(alpha)//2))) #alpha' - alpha transposed
    output = output.dot(kernel(X_train,X_test))
    ### END Solution

    return output
```

### Constant Calculation

The optimal constant $b$ can be derived from the theoretical properties of
the support vectors:
$$ b = y_i + \epsilon - (w \cdot x_i) \, \text{ if }\, 0 < \alpha_i < C \,, $$
and
$$ b = y_i - \epsilon - (w \cdot x_i) \, \text{ if }\, 0 < \alpha_i' < C \,. $$

For example, on infinitely powerful hardware with infinite precision arithmetic
you could have used any single support vector to get $b$. But it won't work IRL,
where the most you can hope for is [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754)
quadruple precision [floating point numbers](https://en.wikipedia.org/wiki/Floating-point_arithmetic])
(setting aside special libraries or hardware). In the **numpy-scipy-sklearn** stack
on a typical x86-64 hardware your algorithms work in finite precision arithmetic, which
is subject to roundoff errors. The arithmetic results are pretty accurate, but still
**inexact**. Besides the numerical optimization algorithms (**cvxopt.qp**, **BFGS**,
**Your own SGD** and the like) themselves aren't exact, and produce just an **extremely
faithful** approximation of the optmial coefficients.

Your task here is to use as many support vectors as possible to get an estimate of $b$.

```python
def calculate_bias(X_train, y_train, kernel, alpha, eps=0.1):
    """Calculates the bias
    :param: X_train np.array with train data
    :param: y_train np.array with train labels
    :parma: kernel the kernel function
    :param: alpha stacked dual coefficients'''
    """
    b = 0.0

    ### BEGIN Solution
    suppIndex1 = np.where((alpha[:len(alpha)//2] < C) & (alpha[:len(alpha)//2]>0))[0]
    suppIndex2 = np.where((alpha[len(alpha)//2:] < C) & (alpha[len(alpha)//2:]>0))[0]

    bs1 = y_train[suppIndex1] - kernel_dot_product(X_train[suppIndex1,:], y_train, X_train[suppIndex1,:], kernel, alpha)+eps
    bs2 = y_train[suppIndex2] - kernel_dot_product(X_train[suppIndex2,:], y_train, X_train[suppIndex2,:], kernel, alpha)-eps

    #print(bs1)
    b=np.median(np.concatenate((bs1,bs2),axis=1))
    ### END Solution

    return b
```

```python

b = calculate_bias(X_train, y_train, kernel, optimal_stacked_alpha, eps)
print(b)
```

    48.6163827611

Now we use the optimal $b$ and $\alpha, \alpha'$ to get the predictions and finally plot
a nice picture of the SVR.

```python
predictions = kernel_dot_product(X_train, y_train, x_test, kernel, optimal_stacked_alpha) + b
```

```python
ax = plt.gca()
ax.plot(x_grid, predictions.T, label='Predictions')
ax.plot(X, y_train, 'o')
ax.set_xlabel('Speed')
ax.set_ylabel('Distance')
ax.legend(loc='best')
```

## Task 3 (11 pt.)

In this task we are going to use SVM for text classification. We will work with
dataset which  contains different tweets with the related conversation context.
Some of the tweets are generated by bots, others -- written by humans.

```python
train_data = pd.read_csv('data/bot_or_not_train.csv', sep='\t', index_col=0)

test_data = pd.read_csv('data/bot_or_not_test.csv', sep='\t', index_col=0)
```

```python
train_data.head()
```

      context
      response
      human-generated

      id

      0
      <first_speaker> 95 degrees with <number> % hum...
      <second_speaker> <at> i forgot that thanksabit...
      0

      1
      <first_speaker> <at> <at> yar ! oy just appear...
      <first_speaker> <at> lol b we 'll see . we hea...
      0

      2
      <first_speaker> ohh ! deu certo ! ! dddddddddd...
      <second_speaker> <at> acordei agora , qqqqq ? ...
      1

      3
      <first_speaker> ughhhh i wanted a pickle . non...
      <second_speaker> <at> lol gm
      0

      4
      <first_speaker> <at> <at> <at> need to know to...
      <first_speaker> <at> ok , will do - don 't be ...
      1

Please notice that context and response columns contains raw text, which is not
directly suitable for machine learning.

### Task 3.1 (6 pt.)

To extract features we are going to use the **bag of words** techniques. Please read
the documentation for `CountVectorizer` [in scikit](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html)
and do the following:

* For both the context and the response generate a set of features using
two separate count vectorizers.

* Join this sets of features into one feature matrix (please notice that
`CountVectorizer` returns Scipy [CSR sparse matrices](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_(CSR,_CRS_or_Yale_format)).

* Pick the optimal $C$ for the **linear kernel** using cross valudation
`GridSearchCV` and `ROC AUC` as the target metric.

* Choose the optimal `gamma` and $C$ for the RBF kernel using cross valudation
`GridSearchCV` targeting `ROC AUC` metric.

* Take the **best models** with linear and RBF kernels, plot **their ROC curves**
on the provided heldout test data on the same figure, and print their `ROC AUC`
score. Which models work better on the test dataset?

```python
from sklearn.feature_extraction.text import CountVectorizer
from scipy import sparse
from sklearn.metrics import roc_auc_score, roc_curve
from sklearn.svm import SVC
```

Generate feature matricies for the context and the response. Fill free to create additional
feature extraction objects if you like.

```python
y_train = train_data['human-generated']

### BEGIN Solution
contextVectorizer = CountVectorizer().fit(train_data['context'])
x_context = contextVectorizer.transform(train_data['context'])

responseVectorizer = CountVectorizer().fit(train_data['response'])
x_response = responseVectorizer.transform(train_data['response'])

# Join the feature matrix
x_train = sparse.hstack([x_context,x_response])

### END Solution
```

Generate the response and contet test sets with their **relevant** feature encodings.

```python
y_test = test_data['human-generated']

### BEGIN Solution
x_context_test = contextVectorizer.transform(test_data['context'])

x_response_test = responseVectorizer.transform(test_data['response'])

x_test = sparse.hstack([x_context_test,x_response_test])
### END Solution
```

Select the best linear model among $C \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$.

```python
### BEGIN Solution
# pick the best C

from sklearn.model_selection import GridSearchCV

parameters = { 'C':np.linspace(1e-3,1e3,5)}

clf = GridSearchCV(SVC(), parameters, scoring="roc_auc")
clf = clf.fit(x_train, y_train)

best_linear_model = clf

### END Solution
```

Select the best RBF kernel model among the hyperparameters
$C \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$ and $\gamma \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$.

```python
### BEGIN Solution
# pick the best C and gamma

parameters = {'gamma':np.linspace(1e-3,1e3,5),'C':np.linspace(1e-3,1e3,5)}

clf1 = GridSearchCV(SVC(kernel="rbf"), parameters,scoring="roc_auc")
clf1 = clf1.fit(x_train, y_train)

best_kernel_model = clf1

### END Solution
```

Plot the `ROC` curves of both models and their `ROC AUC` scores.

```python
### BEGIN Solution
#best_linear_model = clf
#best_kernel_model = clf1

yScore_linear = best_linear_model.decision_function(x_test)
yScore_kernel = best_kernel_model.decision_function(x_test)

#for each classifier
#linear
fpr_lin, tpr_lin, _ = roc_curve(y_test[:], yScore_linear[:], drop_intermediate = False)
print(fpr_lin)
roc_auc_lin = roc_auc_score(y_test, yScore_linear)

#kernel
fpr_ker, tpr_ker, _ = roc_curve(y_test[:], yScore_kernel[:], drop_intermediate = False)
roc_auc_ker = roc_auc_score(y_test, yScore_kernel)

f,ax = plt.subplots(figsize=(10,5))

ax.plot(fpr_lin, tpr_lin, color='darkorange',
         lw=1, label='LinearSVM (area = %0.2f)' % roc_auc_lin)

ax.plot(fpr_ker, tpr_ker, color='red',
         lw=1, label='KernelSVM (area = %0.2f)' % roc_auc_ker)

ax.plot([0, 1], [0, 1], color='navy', lw=1, linestyle='--')#coin flip
ax.set_xlim([0.0, 1.0])
ax.set_ylim([0.0, 1.05])

ax.set_xlabel('False Positive Rate')
ax.set_ylabel('True Positive Rate')
ax.set_title('Receiver operating characteristic')
ax.legend(loc="lower right")
### END Solution
```

... and the best model is...?
It seems that Linear SVM has larger auc-score.

```python
best_model = best_linear_model
test_predictions = best_model.decision_function(x_test)
test_score = roc_auc_score(y_test, test_predictions)
print("Test ROC AUC = {}".format(test_score))
```

    Test ROC AUC = 0.6495314721485077

### Task 3.2 (5 pt.)

Now you are asked to **redo the last task (3.1)**, bit this time with more advanced
features generated by [TfidfVectorizer](https://en.wikipedia.org/wiki/Tf%E2%80%93idf).
To do this you can use `TfidfVectorizer` from sklearn.

```python
from sklearn.feature_extraction.text import TfidfVectorizer
```

Generate feature matricies for context and response. Fill free to create
additional feature extraction object if necessary.

```python
y_train = train_data['human-generated']

### BEGIN Solution
contextVectorizerTF = TfidfVectorizer().fit(train_data['context'])
x_context = contextVectorizerTF.transform(train_data['context'])

responseVectorizerTF = TfidfVectorizer().fit(train_data['response'])
x_response = responseVectorizerTF.transform(train_data['response'])

# Join the feature matrix
x_train = sparse.hstack([x_context,x_response])

### END Solution
```

Generate the response and contet test sets with their **relevant** feature encodings.

```python
y_test = test_data['human-generated']

### BEGIN Solution
x_context_test = contextVectorizerTF.transform(test_data['context'])

x_response_test = responseVectorizerTF.transform(test_data['response'])

x_test = sparse.hstack([x_context_test,x_response_test])
### END Solution
```

Select the best linear model among $C \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$.

```python
### BEGIN Solution
# pick the best C

parameters = { 'C':np.linspace(1e-3,1e3,5)}

lm_tf = GridSearchCV(SVC(), parameters, scoring="roc_auc")
lm_tf = lm_tf.fit(x_train, y_train)

best_linear_model_tf = lm_tf

### END Solution
```

Select the best RBF kernel model among the hyperparameters
$C \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$ and $\gamma \in [\mathtt{1.e-3}, .., \mathtt{1.e3}]$.

```python
### BEGIN Solution
# pick the best C and gamma

parameters = {'gamma':np.linspace(1e-3,1e3,5),'C':np.linspace(1e-3,1e3,5)}

ker_tf = GridSearchCV(SVC(kernel="rbf"), parameters,scoring="roc_auc")
ker_tf = ker_tf.fit(x_train, y_train)

best_kernel_model_tf = ker_tf

### END Solution
```

Plot the `ROC` curves of both models and their `ROC AUC` scores.

```python
### BEGIN Solution

yScore_linear_tf = best_linear_model_tf.decision_function(x_test)
yScore_kernel_tf = best_kernel_model_tf.decision_function(x_test)

#for each classifier
#linear
fpr_lin, tpr_lin, _ = roc_curve(y_test[:], yScore_linear_tf[:], drop_intermediate = False)
print(fpr_lin)
roc_auc_lin_tf = roc_auc_score(y_test, yScore_linear_tf)

#kernel
fpr_ker, tpr_ker, _ = roc_curve(y_test[:], yScore_kernel_tf[:], drop_intermediate = False)
roc_auc_ker_tf = roc_auc_score(y_test, yScore_kernel_tf)

f,ax = plt.subplots(figsize=(10,5))

ax.plot(fpr_lin, tpr_lin, color='darkorange',
         lw=1, label='LinearSVM (area = %0.2f)' % roc_auc_lin_tf)

ax.plot(fpr_ker, tpr_ker, color='red',
         lw=1, label='KernelSVM (area = %0.2f)' % roc_auc_ker_tf)

ax.plot([0, 1], [0, 1], color='navy', lw=1, linestyle='--')#coin flip
ax.set_xlim([0.0, 1.0])
ax.set_ylim([0.0, 1.05])

ax.set_xlabel('False Positive Rate')
ax.set_ylabel('True Positive Rate')
ax.set_title('Receiver operating characteristic')
ax.legend(loc="lower right")
### END Solution
```

... and the best model is...? Still Linear SVM, but the Kernel version is very close to it.

```python
best_model = best_linear_model_tf
test_predictions = best_model.decision_function(x_test)
test_score = roc_auc_score(y_test, test_predictions)
print("Test ROC AUC = {}".format(test_score))
```

    Test ROC AUC = 0.6656884838453689


# Table of Contents
* [1a. Periodic spinodal decomposition on a square domain](#1a.-Periodic-spinodal-decomposition-on-a-square-domain)
	* [Use Binder For Live Examples](#Use-Binder-For-Live-Examples)
	* [Define $f_0$](#Define-$f_0$)
	* [Define the Equation](#Define-the-Equation)
	* [Solve the Equation](#Solve-the-Equation)
	* [Run the Example Locally](#Run-the-Example-Locally)
	* [Movie of Evolution](#Movie-of-Evolution)

# 1a. Periodic spinodal decomposition on a square domain

## Use Binder For Live Examples

[](http://mybinder.org/repo/wd15/fipy-hackathon1)

The free energy is given by,

$$ f_0\left[ c \left( \vec{r} \right) \right] =
   - \frac{A}{2} \left(c - c_m\right)^2
   + \frac{B}{4} \left(c - c_m\right)^4
   + \frac{c_{\alpha}}{4} \left(c - c_{\alpha} \right)^4
   + \frac{c_{\beta}}{4} \left(c - c_{\beta} \right)^4 $$

In FiPy we write the evolution equation as

$$ \frac{\partial c}{\partial t} = \nabla \cdot  \left[
       D \left( c \right) \left( \frac{ \partial^2 f_0 }{ \partial c^2} \nabla c - \kappa \nabla \nabla^2 c \right)
   \right] $$

Let's start by calculating $ \frac{ \partial^2 f_0 }{ \partial c^2} $ using sympy. It's easy for this case, but useful in the general case for taking care of difficult book keeping in phase field problems.

```python
%matplotlib inline

import sympy
import fipy as fp
import numpy as np
```

```python
A, c, c_m, B, c_alpha, c_beta = sympy.symbols("A c_var c_m B c_alpha c_beta")
```

```python
f_0 = - A / 2 * (c - c_m)**2 + B / 4 * (c - c_m)**4 + c_alpha / 4 * (c - c_alpha)**4 + c_beta / 4 * (c - c_beta)**4
```

```python
print f_0
```

    -A*(-c_m + c_var)**2/2 + B*(-c_m + c_var)**4/4 + c_alpha*(-c_alpha + c_var)**4/4 + c_beta*(-c_beta + c_var)**4/4

```python
sympy.diff(f_0, c, 2)
```

    -A + 3*B*(c_m - c_var)**2 + 3*c_alpha*(c_alpha - c_var)**2 + 3*c_beta*(c_beta - c_var)**2

The first step in implementing any problem in FiPy is to define the mesh. For [Problem 1a]({{ site.baseurl }}/hackathon1/#a.-Square-Periodic) the solution domain is just a square domain, but the boundary conditions are periodic, so a `PeriodicGrid2D` object is used. No other boundary conditions are required.

```python
mesh = fp.PeriodicGrid2D(nx=50, ny=50, dx=0.5, dy=0.5)
```

The next step is to define the parameters and create a solution variable.

```python
c_alpha = 0.05
c_beta = 0.95
A = 2.0
kappa = 2.0
c_m = (c_alpha + c_beta) / 2.
B = A / (c_alpha - c_m)**2
D = D_alpha = D_beta = 2. / (c_beta - c_alpha)
c_0 = 0.45
q = np.sqrt((2., 3.))
epsilon = 0.01

c_var = fp.CellVariable(mesh=mesh, name=r"$c$", hasOld=True)
```

Now we need to define the initial conditions given by,

Set $c\left(\vec{r}, t\right)$ such that

$$ c\left(\vec{r}, 0\right) = \bar{c}_0 + \epsilon \cos \left( \vec{q} \cdot \vec{r} \right) $$

```python
r = np.array((mesh.x, mesh.y))
c_var[:] = c_0 + epsilon * np.cos((q[:, None] * r).sum(0))
viewer = fp.Viewer(c_var)
```

## Define $f_0$

To define the equation with FiPy first define `f_0` in terms of FiPy. Recall `f_0` from above calculated using Sympy. Here we use the string representation and set it equal to `f_0_var` using the `exec` command.

```python
out = sympy.diff(f_0, c, 2)
```

```python
exec "f_0_var = " + repr(out)
```

```python
#f_0_var = -A + 3*B*(c_var - c_m)**2 + 3*c_alpha*(c_var - c_alpha)**2 + 3*c_beta*(c_var - c_beta)**2
f_0_var
```

    (((((pow((0.5 - $c$), 2)) * 29.6296296296) + -2.0) + ((pow((0.05 - $c$), 2)) * 0.15)) + ((pow((0.95 - $c$), 2)) * 2.85))

## Define the Equation

```python
eqn = fp.TransientTerm(coeff=1.) == fp.DiffusionTerm(D * f_0_var) - fp.DiffusionTerm((D, kappa))
eqn
```

    (TransientTerm(coeff=1.0) + (DiffusionTerm(coeff=[-(((((((pow((0.5 - $c$), 2)) * 29.6296296296) + -2.0) + ((pow((0.05 - $c$), 2)) * 0.15)) + ((pow((0.95 - $c$), 2)) * 2.85)) * 2.22222222222))]) + DiffusionTerm(coeff=[2.2222222222222223, 2.0])))

## Solve the Equation

To solve the equation a simple time stepping scheme is used which is decreased or increased based on whether the residual decreases or increases. A time step is recalculated if the required tolerance is not reached.

```python
elapsed = 0.0
steps = 0
dt = 0.01
total_sweeps = 2
tolerance = 1e-1
total_steps = 100
```

```python
c_var[:] = c_0 + epsilon * np.cos((q[:, None] * r).sum(0))
c_var.updateOld()
from fipy.solvers.pysparse import LinearLUSolver as Solver
solver = Solver()

while steps < total_steps:
    res0 = eqn.sweep(c_var, dt=dt, solver=solver)

    for sweeps in range(total_sweeps):
        res = eqn.sweep(c_var, dt=dt, solver=solver)

    if res < res0 * tolerance:
        steps += 1
        elapsed += dt
        dt *= 1.1
        c_var.updateOld()
    else:
        dt *= 0.8
        c_var[:] = c_var.old

viewer.plot()
print 'elapsed_time:',elapsed
```

## Run the Example Locally

The following cell will dumpy a file called `fipy_hackathon1a.py` to the local file system to be run. The images are saved out at each time step.

```python
%%writefile fipy_hackathon_1a.py

import fipy as fp
import numpy as np

mesh = fp.PeriodicGrid2D(nx=400, ny=400, dx=0.5, dy=0.5)

c_alpha = 0.05
c_beta = 0.95
A = 2.0
kappa = 2.0
c_m = (c_alpha + c_beta) / 2.
B = A / (c_alpha - c_m)**2
D = D_alpha = D_beta = 2. / (c_beta - c_alpha)
c_0 = 0.45
q = np.sqrt((2., 3.))
epsilon = 0.01

c_var = fp.CellVariable(mesh=mesh, name=r"$c$", hasOld=True)

r = np.array((mesh.x, mesh.y))
c_var[:] = c_0 + epsilon * np.cos((q[:, None] * r).sum(0))

f_0_var = -A + 3*B*(c_var - c_m)**2 + 3*c_alpha*(c_var - c_alpha)**2 + 3*c_beta*(c_var - c_beta)**2

eqn = fp.TransientTerm(coeff=1.) == fp.DiffusionTerm(D * f_0_var) - fp.DiffusionTerm((D, kappa))

elapsed = 0.0
steps = 0
dt = 0.01
total_sweeps = 2
tolerance = 1e-1
total_steps = 1000

c_var[:] = c_0 + epsilon * np.cos((q[:, None] * r).sum(0))

c_var.updateOld()

from fipy.solvers.pysparse import LinearLUSolver as Solver

solver = Solver()

viewer = fp.Viewer(c_var)
while steps < total_steps:
    res0 = eqn.sweep(c_var, dt=dt, solver=solver)

    for sweeps in range(total_sweeps):
        res = eqn.sweep(c_var, dt=dt, solver=solver)

        print ' '
        print 'steps',steps
        print 'res',res
        print 'sweeps',sweeps
        print 'dt',dt

    if res < res0 * tolerance:
        steps += 1
        elapsed += dt
        dt *= 1.1
        if steps % 1 == 0:
             viewer.plot('image{0}.png'.format(steps))
        c_var.updateOld()
    else:
        dt *= 0.8
        c_var[:] = c_var.old
```

    Writing fipy_hackathon_1a.py

## Movie of Evolution

The movie of the evolution for 900 steps.

The movie was generated with the output files of the form `image*.png` using the following commands,

    $ rename 's/\d+/sprintf("%05d",$&)/e' image*
    $ ffmpeg -f image2 -r 6 -i 'image%05d.png' output.mp4

```python
from IPython.display import YouTubeVideo
scale = 1.5
YouTubeVideo('t3tMYp806E4', width=420 * scale, height=315 * scale, rel=0)
```

```python

```