```python
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from IPython.display import display
```

## Logistic Regression
- Apply Sigmoid to Linear Regression

- Output range [0,1]

### Binary Classification Cost function(Entropy)
$$c_i = -(1-y_i)log(1-\widehat{y_i}) - y_ilog(\widehat{y_i})$$

\begin{equation}
  => c_i=\begin{cases}
    -log(1-\widehat{y_i}), & \text{when $y_i=0$}\\
    -log(\widehat{y_i}), & \text{when $y_i=1$}
  \end{cases}
\end{equation}

### Binary Log Loss(Average Entropy)
$$c = \frac{1}{N} \sum_{i}c_i$$

## Data Exploratory

```python
df = pd.read_csv('user_visit_duration.csv')

display(df.info())
display(df.head())
display(df.describe())

display(df['Buy'].value_counts())

df.plot(kind='scatter', x='Time (min)', y='Buy')
```

## Train/Test split

```python
X = df[['Time (min)']].values
y = df['Buy'].values

display(X.shape)
display(y.shape)
```

    (100, 1)

    (100,)

```python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2)
```

## Model

```python
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam, SGD

model = Sequential()

model.add(Dense(1, input_shape=(1,), activation='sigmoid'))

model.compile(SGD(lr=0.5), 'binary_crossentropy', metrics=['accuracy'])

model.summary()
```

    Using TensorFlow backend.

    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    dense_1 (Dense)              (None, 1)                 2
    =================================================================
    Total params: 2
    Trainable params: 2
    Non-trainable params: 0
    _________________________________________________________________

```python
# Train
history = model.fit(
    X_train, y_train,
    epochs=40, verbose=0)
```

```python
historydf = pd.DataFrame(history.history, index=history.epoch)
historydf.plot();
```

## Class Prediction
- Apply Step function to Logistic Regression
\begin{equation}
  => y\_pred = \begin{cases}
    1, & \text{if $\widehat{y} >= 0.5$}\\
    0, & \text{if $\widehat{y} < 0.5$}
  \end{cases}
\end{equation}

## Confusion matrix
- Binary Classification

- Multiclass Classification

### Accuracy
$$A = \frac{Corrects}{Total} = \frac{TN + TP}{TN + FP + FN + TP}$$

### Error Rate
$$ER = \frac{Wrongs}{Total} = \frac{FP + FN}{TN + FP + FN + TP}$$

### Precision
Test = positive, How often prediction correct?
$$P = \frac{True Positive}{Test Positive} = \frac{TP}{TP + FP}$$

- Example: Spam email filter model
    + Acceptable if spam detected as not spam
    + False Negative is OK, False Positive is not accepted
    + Need high Precision

### Recall
Actual value positive, How often prediction correct?
$$R = \frac{True Positive}{Actual Positive} = \frac{TP}{TP+FN}$$

- Example: Cancer diagnosing model
    + Acceptable if non-cancer detected as cancer
    + False Positive is OK, False Negative is not
    + Need high Recall

### F1 Score
High if both Precision and Recall High
$$F1 = \frac{2*P*R}{P+R}$$

### $F_\beta$ score

$$F_\beta = (1+\beta^2)\frac{P*R}{\beta^2*P+R}$$

- $\beta$ is in range $[0, \infty]$
    + If $\beta = 0$, $F_\beta = Precision$
    + If $\beta = 1$, $F_\beta = F1$
    + If $\beta = \infty$, $F_\beta = Recall$

## AUC-ROC Area Under the Curve of the Receiver Operating Characteristic
### Split
- Example: 1D Split
    + Perfect Split
    + Good Split
    + Random Split

### True positive rate (TPR) - aka: sensitivity, hit rate, recall
$$TPR = \frac{True\ Positive}{All\ Positive} = \frac{TP}{TP+FN}$$

### False positive rate (FPR) - aka: fall-out
$$FPR = \frac{False\ Positive}{All\ Negative} = \frac{FP}{FP+TN}$$

### AUC - ROC
- ROC = Curve
    + Find all possible splits on the dataset
    + Construct all pairs of (TPR, FPR)
    + Draw the Curve
- AUC = The Area under the Curve
    - AUC range [0, 1]
        - Random model AUC = 0.5
        - Perfect model AUC = 1.0
        - Good model AUC > 0.8
        - Bad model AUC < 0.5

```python
## Class Prediction
y_train_class_pred = model.predict(X_train) > 0.5 # y_train_class_pred = model.predict_classes(X_train)
y_test_class_pred = model.predict(X_test) > 0.5 # y_test_class_pred = model.predict_classes(X_test)
```

```python
## Confusion matrix
from sklearn.metrics import confusion_matrix

print("The train confusion matrix:")
print(confusion_matrix(y_train, y_train_class_pred))
print('')
print("The test confusion matrix:")
print(confusion_matrix(y_test, y_test_class_pred))
```

    The train confusion matrix:
    [[30  8]
     [ 7 35]]

    The test confusion matrix:
    [[10  2]
     [ 0  8]]

```python
## Accuracy
from sklearn.metrics import accuracy_score

print("The train Accuracy score is {:0.3f}".format(accuracy_score(y_train, y_train_class_pred)))
print("The test Accuracy score is {:0.3f}".format(accuracy_score(y_test, y_test_class_pred)))
```

    The train Accuracy score is 0.812
    The test Accuracy score is 0.900

```python
## Precision
from sklearn.metrics import precision_score

print("The train Precision score is {:0.3f}".format(precision_score(y_train, y_train_class_pred)))
print("The test Precision score is {:0.3f}".format(precision_score(y_test, y_test_class_pred)))
```

    The train Precision score is 0.814
    The test Precision score is 0.800

```python
## Recall
from sklearn.metrics import recall_score

print("The train Recall score is {:0.3f}".format(recall_score(y_train, y_train_class_pred)))
print("The test Recall score is {:0.3f}".format(recall_score(y_test, y_test_class_pred)))
```

    The train Recall score is 0.833
    The test Recall score is 1.000

```python
## F1 score
from sklearn.metrics import f1_score

print("The train F1 score is {:0.3f}".format(f1_score(y_train, y_train_class_pred)))
print("The test F1 score is {:0.3f}".format(f1_score(y_test, y_test_class_pred)))
```

    The train F1 score is 0.824
    The test F1 score is 0.889

```python
## Report
from sklearn.metrics import classification_report

print("The train classification report:")
print(classification_report(y_train, y_train_class_pred))
print('')
print("The test classification report:")
print(classification_report(y_test, y_test_class_pred))
```

    The train classification report:
                 precision    recall  f1-score   support

              0       0.81      0.79      0.80        38
              1       0.81      0.83      0.82        42

    avg / total       0.81      0.81      0.81        80

    The test classification report:
                 precision    recall  f1-score   support

              0       1.00      0.83      0.91        12
              1       0.80      1.00      0.89         8

    avg / total       0.92      0.90      0.90        20

## Cross Validation
- Tradeoff Computational power <-> Accuracy
- Can you distributed computing or parallelization to Train multiple Cross validation

```python
from keras.wrappers.scikit_learn import KerasClassifier

def build_logistic_regression_model():
    model = Sequential()
    model.add(Dense(1, input_shape=(1,), activation='sigmoid'))
    model.compile(SGD(lr=0.5),
                  'binary_crossentropy',
                  metrics=['accuracy'])
    return model
```

```python
model = KerasClassifier(
    build_fn=build_logistic_regression_model,
    epochs=25, verbose=0)
```

```python
from sklearn.model_selection import cross_val_score, KFold

scores = cross_val_score(
    model,
    X, y,
    cv=KFold(3, shuffle=True))

display(scores)
print("The cross validation accuracy is {:0.4f} ± {:0.4f}".format(scores.mean(), scores.std()))
```

    array([0.85294118, 0.84848485, 0.6969697 ])

    The cross validation accuracy is 0.7995 ± 0.0725


# Multivariate Kalman filter on linear manoeuvring data

# Purpose
* implementation of multivariate kalman filter inspired by: [https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/06-Multivariate-Kalman-Filters.ipynb](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/06-Multivariate-Kalman-Filters.ipynb)

# Methodology
* Implement a kalman filter to chase a ship at unsteady velocity

# Setup

```python
# %load imports.py
## Local packages:

%matplotlib inline
%load_ext autoreload
%autoreload 2
%config Completer.use_jedi = False  ## (To fix autocomplete)

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt
#if os.name == 'nt':
#    plt.style.use('presentation.mplstyle')  # Windows

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

from src.symbols import *
from src.parameters import *
import src.symbols as symbols
from src import prime_system
from src.models import regression
from src.visualization.regression import show_pred
from src.visualization.plot import track_plot

## Load models:
# (Uncomment these for faster loading):
import src.models.vmm_linear  as vmm
from src.data.case_0 import ship_parameters, df_parameters, ps, ship_parameters_prime
from src.data.transform import transform_to_ship

from scipy.stats import norm
import filterpy.stats as stats
from filterpy.common import Q_discrete_white_noise
from filterpy.kalman import KalmanFilter
np.set_printoptions(linewidth=150)

from numpy import zeros,dot
from numpy.linalg import inv
```

## Generate some data

```python
mask = df_parameters['prime'].notnull()
index = df_parameters.loc[mask,'prime'].index
coefficients=vmm.simulator.get_all_coefficients(sympy_symbols=False)
missing_coefficients = set(coefficients) - set(index)
missing_coefficients
```

```python
mask = df_parameters['prime'].notnull()
df_parameters.loc[mask,'prime']
```

## Simulate data

```python
parameters=df_parameters['prime'].copy()

t_ = np.arange(0,4,0.01)
df_ = pd.DataFrame(index=t_)

df_['u'] = 1
df_['v'] = 0
df_['r'] = 0
df_['x0'] = 0
df_['y0'] = 0
df_['psi'] = 0
df_['U'] = np.sqrt(df_['u']**2 + df_['v']**2)
df_['beta'] = -np.arctan2(df_['v'],df_['u'])
df_['delta'] = np.deg2rad(20)

## Simulation in prime system!
result = vmm.simulator.simulate(df_=df_, parameters=parameters, ship_parameters=ship_parameters_prime,
                                control_keys=['delta'], primed_parameters=False,
                                prime_system=ps, method='Radau')
df_result=result.result.copy()
```

```python
result.track_plot(compare=False)
result.plot(compare=False);
```

## Measurement noise

```python
df_measure = df_result.copy()
df_measure['u1d'] = np.NaN
df_measure['v1d'] = np.NaN
df_measure['r1d'] = np.NaN

R_u_var = ((df_measure['u'].abs().max())/30)**2
R_v_var = ((df_measure['v'].abs().max())/30)**2
R_r_var = ((df_measure['r'].abs().max())/30)**2

df_measure['R_u'] = np.random.normal(loc=0, scale=np.sqrt(R_u_var), size=len(df_measure))  # measurement noise
df_measure['R_v'] = np.random.normal(loc=0, scale=np.sqrt(R_v_var), size=len(df_measure))  # measurement noise
df_measure['R_r'] = np.random.normal(loc=0, scale=np.sqrt(R_r_var), size=len(df_measure))  # measurement noise

Q_var = 0.00001

df_measure['Q'] = np.random.normal(loc=0, scale=np.sqrt(Q_var), size=len(df_measure))  # process noise

df_measure['u']+=(df_measure['R_u']+df_measure['Q'])
df_measure['v']+=(df_measure['R_v']+df_measure['Q'])
df_measure['r']+=(df_measure['R_r']+df_measure['Q'])

```

```python
A = vmm.simulator.A
b = vmm.simulator.b
acceleration = sp.matrices.MutableDenseMatrix([u1d,v1d,r1d])
eq_simulator = sp.Eq(sp.UnevaluatedExpr(A)*sp.UnevaluatedExpr(acceleration),sp.UnevaluatedExpr(b))
Math(vlatex(eq_simulator))
```

```python
A_inv = A.inv()
S = sp.symbols('S')
eq_S=sp.Eq(S,-sp.fraction(A_inv[1,1])[1])

A_inv_S = A_inv.subs(eq_S.rhs,S)
eq_acceleration_matrix_clean = sp.Eq(sp.UnevaluatedExpr(acceleration),sp.UnevaluatedExpr(A_inv_S)*sp.UnevaluatedExpr(b))
Math(vlatex(eq_acceleration_matrix_clean))
```

```python
eq_S
```

```python
eq_acceleration_matrix_clean_specific = eq_acceleration_matrix_clean.subs([
    (X_qs, vmm.simulator.X_qs_eq.rhs),
    (Y_qs, vmm.simulator.Y_qs_eq.rhs),
    (N_qs, vmm.simulator.N_qs_eq.rhs),
])

Math(vlatex(eq_acceleration_matrix_clean_specific))
```

```python
u_,v_,r_,u1d,v1d,r1d,delta_ = sp.symbols('u v r u1d v1d r1d delta')

subs = [

    (u1d,u1d),
    (v1d,v1d),
    (r1d,r1d),

    (u,u_),
    (v,v_),
    (r,r_),
    (delta,0),

]

A,b = sp.linear_eq_to_matrix(eq_acceleration_matrix_clean_specific.doit().subs(subs).rhs,
                       [u_,v_,r_])
```

```python
A
```

```python
subs = [

    (u1d,u1d),
    (v1d,v1d),
    (r1d,r1d),

    (u,0),
    (v,0),
    (r,0),
    (delta,delta_),

]
B,b = sp.linear_eq_to_matrix(eq_acceleration_matrix_clean_specific.doit().subs(subs).rhs,
                       [delta_])
#B=sp.Matrix.vstack(B*symbols.dt,B)
B=sp.Matrix.vstack(sp.zeros(3,1),B)
```

```python
B
```

```python
F = sp.Matrix.vstack(
    sp.Matrix.hstack(sp.eye(3),symbols.dt*sp.eye(3)),
    sp.Matrix.hstack(A,sp.zeros(3))
)  # transition matrix
F
```

```python
subs = {value:key for key,value in p.items()}
A_lambda = lambdify(A.subs(subs))

F_lambda = lambdify(F.subs(subs))

subs = {value:key for key,value in p.items()}
B_lambda = lambdify(B.subs(subs))

S_lambda = lambdify(eq_S.rhs.subs(subs))
```

```python
df = df_measure
```

```python
parameters=df_parameters['prime'].copy()
mask = parameters.notnull()

values = parameters.loc[mask]
parameters.loc[mask]+=0.01*values*np.random.rand(len(values))  # add some noise to the parameters
mask=(parameters==0)
parameters.loc[mask]=0.00001
```

```python
mask = parameters.notnull()
parameters.loc[mask]
```

```python
U_ = float(df.iloc[0]['U'])
dt_ = df.index[1] - df.index[0]
S_ = run(S_lambda, inputs=parameters, **ship_parameters_prime)
F_ = run(F_lambda, inputs=parameters, **ship_parameters_prime, S=S_, U=U_, dt=dt_)
B_ = run(B_lambda, inputs=parameters, **ship_parameters_prime, S=S_, U=U_, dt=dt_)
A_ = run(A_lambda, inputs=parameters, **ship_parameters_prime, S=S_, U=U_, dt=dt_)
```

```python
F_
```

```python
B_
```

```python
np.eye(3,3) + A_*dt_ + (A_*dt_)**2/np.math.factorial(2) + (A_*dt_)**3/np.math.factorial(3)
```

## Kalman filtering

```python
from scipy.linalg import inv

def predict(F,x,B,u):
    x = F @ x + B.dot(u)
    return x

def filter2(zs, x,us,P,F,H,R,Q,B):

    xs, cov = [], []
    for z,u in zip(zs,us):

        z = z.reshape(len(z),1)

        # predict
        x = predict(F=F,x=x,B=B,u=u)
        P = F @ P @ F.T + Q

        #update
        S = H @ P @ H.T + R
        K = P @ H.T @ inv(S)
        y = z - H @ x
        x += K @ y
        P = P - K @ H @ P

        xs.append(x)
        cov.append(P)

    xs, cov = np.array(xs), np.array(cov)
    return xs, cov
```

```python
x=np.array([list(df.iloc[0][['u','v','r']].values) + [0,0,0]]).T  # state mean

state_keys = ['u','v','r','u1d','v1d','r1d']

est_cov = (df_result[state_keys].abs().max()/3)**2
mask = est_cov==0
est_cov[mask] = np.mean(est_cov)

P = np.diag(est_cov)         # state covariance

F = F_       # measurement function         # transition matrix

H = np.concatenate([np.eye(3),np.zeros((3,3))],axis=1)       # measurement function
R_var = 100
R = np.diag(((df[['u','v','r']].abs().max()/3)**2).values)        # measurement covariance
Q = 0  # process covariance
B = B_     # control transition matrix

zs=df[['u','v','r']].values

us= df['delta'].values
#us=np.zeros(len(df))
```

```python
np.linalg.inv(P)
```

```python
P.shape
```

```python
x
```

```python
zs=df[['u','v','r']].values
us= df['delta'].values

x=np.array([df_result.iloc[0][state_keys].values]).T  # state mean

xs, cov = [], []
x_prime = np.zeros(x.shape)
L_ = ship_parameters['L']

for z,u in zip(zs,us):

    z = z.reshape(len(z),1)

    x = predict(F=F,x=x,B=B,u=u)
    xs.append(x)

xs = np.array(xs)

df_pred = pd.DataFrame(data=xs.reshape((len(df),6)),
                       columns=['u','v','r','u1d','v1d','r1d'], index=df.index)
```

```python
x=np.array([list(df.iloc[0][['u','v','r']].values) + [0,0,0]]).T  # state mean
u_=us[0]
predict(F=F,x=x,B=B,u=u_)
```

```python
B.dot(u)
```

```python
B
```

```python
for key in ['u','v','r','u1d','v1d','r1d']:
    fig,ax=plt.subplots()
    fig.set_size_inches(10,2)
    df_result.plot(y=key, ax=ax, label='real')
    df_measure.plot(y=key, style='.', ax=ax, label='measurement')

    df_pred.plot(y=key, style='--', ax=ax, label='prediction')
    ax.set_ylabel(key)
```

```python
x
```

```python
#x=np.array([list(df.iloc[0][['u','v','r']].values) + [0,0,0]]).T  # state mean
x=np.array([list(df_result.iloc[0][['u','v','r','u1d','v1d','r1d']].values)]).T  # state mean

xs,cov=filter2(zs=zs, x=x, us=us, P=P, F=F, H=H, R=R, Q=Q, B=B)
df_pred = pd.DataFrame(data=xs.reshape((len(df),6)),
                       columns=['u','v','r','u1d','v1d','r1d'], index=df.index)
```

```python
for key in ['u','v','r','u1d','v1d','r1d']:
    fig,ax=plt.subplots()
    fig.set_size_inches(10,2)
    df_result.plot(y=key, ax=ax, label='real')
    df.plot(y=key, style='.', ax=ax, label='measure')

    df_pred.plot(y=key, style='--', ax=ax, label='filter')
    #df_smooth.plot(y=key, style='--', ax=ax, label='smoother')
    ax.set_ylabel(key)

```

```python
Xs = xs.reshape(len(zs),len(x))
Ps = cov
n = Xs.shape[0]
Fs = [F]*n
Qs = [Q]*n
```

```python
Xs.shape
```

```python
def rts_smoother(Xs, Ps, F, Q):
    n, dim_x = Xs.shape

    # smoother gain
    K = zeros((n,dim_x, dim_x))
    x, P, Pp = Xs.copy(), Ps.copy(), Ps.copy()

    for k in range(n-2,-1,-1):
        Pp[k] = F @ P[k] @ F.T + Q # predicted covariance

        try:
            K[k]  = P[k] @ F.T @inv(Pp[k])
        except:
            # I don't understand why this happens...
            break

        x[k] += K[k] @ (x[k+1] - (F @ x[k]))
        P[k] += K[k] @ (P[k+1] - Pp[k]) @ K[k].T

    return (x, P, K, Pp)
```

```python
(x, P, K, Pp) = rts_smoother(Xs=Xs, Ps=Ps, F=F, Q=Q)

df_smooth = pd.DataFrame(data=x.reshape((len(df),6)),
                       columns=['u','v','r','u1d','v1d','r1d'], index=df.index)
```

## PyKalman

```python
B
```

```python
F2 = np.concatenate([F,B], axis=1)
F2 = np.concatenate([F2,np.zeros((1,F2.shape[1]))], axis=0)
F2[F2.shape[0]-1,F2.shape[1]-1]=1
print(F2)
```

```python
H2 = np.concatenate([H,np.zeros((H.shape[0],1))], axis=1)
H2 = np.concatenate([H2,np.zeros((1,H2.shape[1]))], axis=0)
H2[H2.shape[0]-1,H2.shape[1]-1]=1
H2
```

```python
delta_ = df.iloc[0]['delta']

x=np.array([list(df.iloc[0][['u','v','r']].values) + [0,0,0] + [delta_]]).T  # state mean

P2 = np.diag(list((df_result[state_keys].abs().max()/3)**2) + [0])         # state covariance

R_var = 100
R2 = np.diag(list(((df[['u','v','r']].abs().max()/3)**2).values) + [0])        # measurement covariance

Q_var = 0.01
Q = 0  # process covariance
B = B_         # control transition matrix

zs=df[['u','v','r','delta']].values

kf = KalmanFilter(transition_matrices=F2,
                  #observation_matrices=H2,
                  #transition_covariance=Q,
                  #observation_covariance=R2,
                  initial_state_mean=x[:,0],
                  #initial_state_covariance=P2
             )
```

```python
F2.shape
```

```python
H2.shape
```

```python
Q
```

```python
R2.shape
```

```python
P2.shape
```

```python
x.shape
```

```python
kf.filter(zs)
```

```python

```


---
title: Facility Location
---

**Originally Contributed by**: Mathieu Tanneau and Alexis Montoison

```julia
using Random
using LinearAlgebra

using JuMP
import GLPK
using Plots
```

## Uncapacitated facility location

### Problem description

We are given
* $M=\{1, \dots, m\}$ clients
* $N=\{ 1, \dots, n\}$ sites where a facility can be built

**Decision variables**
Decision variables are split into two categories:
* Binary variable $y_{j}$ indicates whether facility $j$ is built or not
* Binary variable $x_{i, j}$ indicates whether client $i$ is assigned to facility $j$

**Objective**
The objective is to minimize the total cost of serving all clients.
This costs breaks down into two components:
* Fixed cost of building a facility.
In this example, this cost is $f_{j} = 1, \ \forall j$.
* Cost of serving clients from the assigned facility.
In this example, the cost $c_{i, j}$
of serving client $i$ from facility $j$
is the Euclidean distance between the two.

**Constraints**
* Each customer must be served by exactly one facility
* A facility cannot serve any client unless it is open

### MILP formulation

The problem can be formulated as the following MILP:

\begin{align}
\min_{x, y} \ \ \ &
\sum_{i, j} c_{i, j} x_{i, j} +
\sum_{j} f_{j} y_{j}\\
s.t. &
\sum_{j} x_{i, j} = 1, && \forall i \in M\\
& x_{i, j} \leq y_{j}, && \forall i \in M, j \in N\\
& x_{i, j}, y_{j} \in \{0, 1\}, && \forall i \in M, j \in N
\end{align}

where the first set of constraints ensures
that each client is served exactly once,
and the second set of constraints ensures
that no client is served from an unopened facility.

### Problem data

```julia
Random.seed!(314)

m = 12  # number of clients
n = 5  # number of facility locations

# Clients' locations
Xc = rand(m)
Yc = rand(m)

# Facilities' potential locations
Xf = rand(n)
Yf = rand(n)

# Fixed costs
f = ones(n);

# Distance
c = zeros(m, n)
for i in 1:m
    for j in 1:n
        c[i, j] = norm([Xc[i] - Xf[j], Yc[i] - Yf[j]], 2)
    end
end
```

```julia
# Display the data
scatter(Xc, Yc, label = "Clients", markershape=:circle, markercolor=:blue)
scatter!(Xf, Yf, label="Facility",
    markershape=:square, markercolor=:white, markersize=6,
    markerstrokecolor=:red, markerstrokewidth=2
)
```

### JuMP implementation

```julia
# Create a JuMP model
ufl = Model(GLPK.Optimizer)

# Variables
@variable(ufl, y[1:n], Bin);
@variable(ufl, x[1:m, 1:n], Bin);

# Each client is served exactly once
@constraint(ufl, client_service[i in 1:m],
    sum(x[i, j] for j in 1:n) == 1
);

# A facility must be open to serve a client
@constraint(ufl, open_facility[i in 1:m, j in 1:n],
    x[i, j] <= y[j]
)

# Objective
@objective(ufl, Min, f'y + sum(c .* x));
```

```julia
# Solve the uncapacitated facility location problem with GLPK
optimize!(ufl)
println("Optimal value: ", objective_value(ufl))
```

    Optimal value: 5.226417970467934

### Visualizing the solution

```julia
# The threshold 1e-5 ensure that edges between clients and facilities are drawn when x[i, j] ≈ 1.
x_ = value.(x) .> 1 - 1e-5
y_ = value.(y) .> 1 - 1e-5

# Display clients
p = scatter(Xc, Yc, markershape=:circle, markercolor=:blue, label=nothing)

# Show open facility
mc = [(y_[j] ? :red : :white) for j in 1:n]
scatter!(Xf, Yf,
    markershape=:square, markercolor=mc, markersize=6,
    markerstrokecolor=:red, markerstrokewidth=2,
    label=nothing
)

# Show client-facility assignment
for i in 1:m
    for j in 1:n
        if x_[i, j] == 1
           plot!([Xc[i], Xf[j]], [Yc[i], Yf[j]], color=:black, label=nothing)
        end
    end
end

display(p)
```

## Capacitated Facility location

### Problem formulation

The capacitated variant introduces a capacity constraint on each facility, i.e., clients have a certain level of demand to be served, while each facility only has finite capacity which cannot be exceeded.

Specifically, let
* $a_{i} \geq 0$ denote the demand of client $i$
* $q_{j} \geq 0$ denote the capacity of facility $j$

The capacity constraints then write
\begin{align}
\sum_{i} a_{i} x_{i, j} &\leq q_{j} y_{j} && \forall j \in N
\end{align}

Note that, if $y_{j}$ is set to $0$, the capacity constraint above automatically forces $x_{i, j}$ to $0$.

Thus, the capacitated facility location can be formulated as follows

\begin{align}
\min_{x, y} \ \ \ &
\sum_{i, j} c_{i, j} x_{i, j} +
\sum_{j} f_{j} y_{j}\\
s.t. &
\sum_{j} x_{i, j} = 1, && \forall i \in M\\
& \sum_{i} a_{i} x_{i, j} \leq q_{j} y_{j}, && \forall j \in N\\
& x_{i, j}, y_{j} \in \{0, 1\}, && \forall i \in M, j \in N
\end{align}

For simplicity, we will assume that there is enough capacity to serve the demand,
 i.e., there exists at least one feasible solution.

```julia
# Demands
a = rand(1:3, m);

# Capacities
q = rand(5:10, n);
```

```julia
# Display the data
scatter(Xc, Yc, label=nothing,
    markershape=:circle, markercolor=:blue, markersize= 2 .*(2 .+ a)
)

scatter!(Xf, Yf, label=nothing,
    markershape=:rect, markercolor=:white, markersize= q,
    markerstrokecolor=:red, markerstrokewidth=2
)
```

### JuMP implementation

```julia
# Create a JuMP model
cfl = Model(GLPK.Optimizer)

# Variables
@variable(cfl, y[1:n], Bin);
@variable(cfl, x[1:m, 1:n], Bin);

# Each client is served exactly once
@constraint(cfl, client_service[i in 1:m], sum(x[i, :]) == 1)

# Capacity constraint
@constraint(cfl, capacity, x'a .<= (q .* y))

# Objective
@objective(cfl, Min, f'y + sum(c .* x));
```

```julia
# Solve the problem
optimize!(cfl)
println("Optimal value: ", objective_value(cfl))
```

    Optimal value: 5.836938427763575

### Visualizing the solution

```julia
# The threshold 1e-5 ensure that edges between clients and facilities are drawn when x[i, j] ≈ 1.
x_ = value.(x) .> 1 - 1e-5
y_ = value.(y) .> 1 - 1e-5

# Display the solution
p = scatter(Xc, Yc, label=nothing,
    markershape=:circle, markercolor=:blue, markersize= 2 .*(2 .+ a)
)

mc = [(y_[j] ? :red : :white) for j in 1:n]
scatter!(Xf, Yf, label=nothing,
    markershape=:rect, markercolor=mc, markersize=q,
    markerstrokecolor=:red, markerstrokewidth=2
)

# Show client-facility assignment
for i in 1:m
    for j in 1:n
        if x_[i, j] == 1
            plot!([Xc[i], Xf[j]], [Yc[i], Yf[j]], color=:black, label=nothing)
            break
        end
    end
end

display(p)
```

## Further
* [Benders decomposition](https://github.com/JuliaOpt/JuMPTutorials.jl/blob/master/script/optimization_concepts/benders_decomposition.jl)
is a method of choice for solving facility location problems.
* Benchmark instances can be found
[here](https://resources.mpi-inf.mpg.de/departments/d1/projects/benchmarks/UflLib/).

```julia

```


```
"""Header cell, contains modules and functions to make the whole notebook experience better"""
%matplotlib inline
# plots graphs within the notebook

from IPython.display import display,Image, Latex
from sympy.interactive import printing
printing.init_printing(use_latex='mathjax')
from IPython.display import clear_output

import time

from IPython.display import display,Image, Latex

from IPython.display import clear_output

import matplotlib.pyplot as plt
import numpy as np
import math
import scipy.constants as sc

import sympy as sym

font = {'family' : 'serif',
        #'color'  : 'black',
        'weight' : 'normal',
        'size'   : 12,
        }
fontlabel = {'family' : 'serif',
        #'color'  : 'black',
        'weight' : 'normal',
        'size'   : 12,
        }

from matplotlib.ticker import FormatStrFormatter
plt.rc('font', **font)

class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return ''.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in
        IPython Notebook. """

    def _repr_html_(self):
        html = ["<table>"]
        for row in self:
            html.append("<tr>")

            for col in row:
                html.append("<td>{0}</td>".format(col))

            html.append("</tr>")
        html.append("</table>")
        return ''.join(html)

```

```
PDF('bot-sketch.pdf',size = (550,400))
```

# Differential Drive Robot - 1

Alphabot 2 is a differential drive robot: Two independent motors drive the left and right wheels and the differential speed between the two wheels control the speed and direction of the robot. The following code is copied from http://enesbot.me/kinematic-model-of-a-differential-drive-robot.html

## Objective
This notebook focuses on the control of a simulated differential drive robot for prescribed simple tasks.

## Theoretical Background

The robot parameters are:

 Wheel radius: $r$.

 Length between wheels: $L$

 Angular velocity of the left and right wheels: $\omega_L$, $\omega_R$, respectively.

 Angle from horizontal: $\alpha$.

 Position vector of the robot: $(x,y)$.

 Velocity vector of the robot: $V=(\dot{x},\dot{y})$.

Hereafter $\dot{a}$ refers to the time derivative of variable $a$:

$$
\dot{a}=\frac{da}{dt}
$$

The velocities of the wheels are therefore defined as:

$$
V_L=\omega_Lr\text{ and }V_R=\omega_Rr.
$$

The velocity of the robot, taken at the center of the wheels, is simply:

$$
\vec{V}=\frac{V_R+V_L}{2}(\cos(\alpha)\vec{e}_x+\sin(\alpha\vec{e}_y),
$$

yielding the following equations of motions:

$$
\dot{x}=r\frac{\omega_R+\omega_L}{2}\cos{\alpha}
$$

$$
\dot{y}=r\frac{\omega_R+\omega_L}{2}\sin{\alpha}
$$

and

$$
\dot{\alpha}=\frac{r}{L}(\omega_R-\omega_L)
$$

Numerically, these solutions of these equations requires their discretization. Remember that the derivative $da/dt$ of a variable $a=f(t)$ is the slope $s$ of the local tangent. Over a small enough time interval $\Delta t$, any continuous function $f$ can be approximated by a linear function based on the slope $a(t+\Delta t$)=a(t)+s(t)\Delta t$. The derivative is then estimated as:

$$
\frac{da}{dt}\approx\frac{\Delta a}{\Delta t}
$$

where $\Delta a=a(t+\Delta t$)-a(t)$. The discretized equations of motion may be recast as an integration of the trajectory of the robot:

$$
x(t+\Delta t) = x(t) +\Delta t\left(r\frac{\omega_R+\omega_L}{2}\cos{\alpha}\right),
$$

$$
y(t+\Delta t) = y(t) +\Delta t\left(r\frac{\omega_R+\omega_L}{2}\sin{\alpha}\right),
$$

$$
\alpha(t+\Delta t) = \alpha(t) +\Delta t\left(r\frac{\omega_R-\omega_L}{L}\right).
$$

As long as, the initial conditions are prescribed at $t=0$, the motion of the robot is predictable providing that $\Delta t$ is not too large, as we shall see later. The present discretization is the simplest one, and also the most rudimentary. Even though the above equations carry fairly large numerical errors, no robot is perfect and these numerical errors will not be discussed here.

## Code description

The code below defines a class, named Robot, which includes the robot dimensions, position variables ($x,y,\alpha$), the time step $\Delta t$, the left/right angular velocities and various functions necessary to integrate the robot motion. First execute the cell below.

```
import matplotlib.pyplot as plt
import numpy as np
import math
import scipy.constants as sc

import sympy as sym

from math import sin, cos

class Robot(object):
    """Defines basic mobile robot properties"""
    def __init__(self):
        self.pos_x  = 0.0
        self.pos_y  = 0.0
        self.angle  = 0.0
        self.plot   = False
        self._delta = 0.01
        self.step_plot = int(5)
        self.mag_plot = 1.

    # Movement
    def step(self):
        """ updates the x,y and angle """
        self.deltax()
        self.deltay()
        self.deltaa()

    def move(self, seconds):
        """ Moves the robot for an 's' amount of seconds"""
        for i in range(int(seconds/self._delta)):
            self.step()
            if i % self.step_plot == 0 and self.plot: # plot path every 3 steps
                self.plot_xya()

    # Printing-and-plotting:
    def print_xya(self):
        """ prints the x,y position and angle """
        print ("x = " + str(self.pos_x) +" "+ "y = " + str(self.pos_y))
        print ("a = " + str(self.angle))

    def plot_robot(self):
        """ plots a representation of the robot """
        plt.arrow(self.pos_x, self.pos_y, 0.001
                  * cos(self.angle), 0.001 * sin(self.angle),
                  head_width=self.mag_plot*self.length, head_length=self.mag_plot*self.length,
                  fc='k', ec='k')

    def plot_xya(self):
        """ plots a dot in the position of the robot """
        plt.scatter(self.pos_x, self.pos_y, c='r', edgecolors='r')

class DDRobot(Robot):
    """Defines a differential drive robot"""

    def __init__(self):
        Robot.__init__(self)
        self.radius = 0.1
        self.length = 0.4

        self.rt_spd_left = 0.0
        self.rt_spd_right = 0.0

    def deltax(self):
        """ update x depending on l and r angular speeds """
        self.pos_x += self._delta * (self.radius*0.5) \
        * (self.rt_spd_right + self.rt_spd_left)*cos(self.angle)

    def deltay(self):
        """ update y depending on l and r angular speeds """
        self.pos_y += self._delta * (self.radius*0.5) \
        * (self.rt_spd_right + self.rt_spd_left)*sin(self.angle)

    def deltaa(self):
        """ update z depending on l and r angular speeds """
        self.angle += self._delta * (self.radius/self.length) \
        * (self.rt_spd_right - self.rt_spd_left)
# function to convert degrees to radians
def D2R(a):
    return math.pi*a/180
def R2D(a):
    return 180*a/math.pi
# mybot = DDRobot()            # robot called 'enesbot'

# mybot.angle = 3.1416/4        # 45 degrees
# mybot.plot = True             # plot the robot!
# mybot.plot_robot()

# mybot.rt_spd_left = 10
# mybot.rt_spd_right = 10       # straight line
# mybot.move(2)                 # move for 2 seconds

# mybot.rt_spd_left = 12.5664
# mybot.rt_spd_right = 18.8496  # (2m diameter circle)
# mybot.move(1)                 # move for 1 second

# mybot.rt_spd_left = 18.8496
# mybot.rt_spd_right = 12.5664  # (2m diameter circle)
# mybot.move(2.5)               # move for 2.5 second

# mybot.rt_spd_left = 12.5664
# mybot.rt_spd_right = 18.8496  # (2m diameter circle)
# mybot.move(3.5)               # move for 2.5 second

# mybot.plot_robot()

# plt.xlim([-1, 6])               # axis limits
# plt.ylim([-1, 6])

# plt.show()
```

## Quick Tutorial

Now you can create your first robot.

```
mybot = DDRobot()
```

For simplicity, assume that all units are based on the metric system. Your new robot, mybot, is by default located at $(x=0,y=0)$, its angle from the horizontal axis $x$ is $0$ and the wheel radius and length between wheel are $r=0.1$m and $L=0.4$m (see class DDRobot). The default time step is $\Delta t=0.01$s. All these variables and more can be easily customized to your problem:

```
mybot.pos_x = 0 #x
mybot.pos_y = 0 #y
mybot.angle = D2R(45) #alpha
mybot.length =0.4 #L
mybot.radius = 0.1 #r
mybot.rt_spd_left = 10 # omega_L
mybot.rt_spd_right = 10 # omega_R

```

Now that our initial conditions and rotational speed is set, let's move our robot in a straight line and plot its trajectory.

```
mybot.plot = True #True if you want to plot the robot's trajectory
mybot.plot_robot() #draw an arrow for the location of the robot at t=0

mybot.move(10) #move the robot for 10s
mybot.plot_robot() #draw the new location

plt.xlim(-1,11)
plt.ylim(-1,11)
plt.xlabel(r"$x$")
plt.ylabel(r"$y$")
plt.show()
```

You may change the size of the arrow and the spacing of the dots making the trajectory using the following variables:

```
mybot.mag_plot = 2 #coefficient of magnification of the arrow
mybot.step_plot = 100 # plot location every 100 iterations

mybot.pos_x = 0 #x
mybot.pos_y = 0 #y
mybot.angle = D2R(45) #alpha
mybot.length =0.4 #L
mybot.radius = 0.1 #r
mybot.rt_spd_left = 10 # omega_L
mybot.rt_spd_right = 10 # omega_R

mybot.plot = True #True if you want to plot the robot's trajectory
mybot.plot_robot() #draw an arrow for the location of the robot at t=0

mybot.move(10) #move the robot for 10s
mybot.plot_robot() #draw the new location

plt.xlim(-1,11)
plt.ylim(-1,11)
plt.xlabel(r"$x$")
plt.ylabel(r"$y$")
plt.show()
```

You may print the location of your robot using:

```
mybot.print_xya()
```

    x = 7.071067811865343 y = 7.071067811865343
    a = 0.7853981633974483

which you can compare to the theoretical location of the robot. Since the robot's wheel radius is $0.1$m, the rotational speed is $\omega=10$ (1/s), therefore the robot speed is $1$m/s. Over $10$s of travel time, starting at an angle of $45^\circ$, the location of the robot should be $x=10\cos(\pi/4)$ and $y=10\sin(\pi/4)$.

```
print('absolute error x: %1.1e m' %(abs(mybot.pos_x - 10*math.cos(math.pi/4))))
print('absolute error y: %1.1e m' %(abs(mybot.pos_y - 10*math.sin(math.pi/4))))
print('absolute error alpha: %1.1e rad' %(abs(mybot.angle - math.pi/4)))
```

    absolute error x: 1.3e-13 m
    absolute error y: 1.3e-13 m
    absolute error alpha: 0.0e+00 m

To turn, the left and right angular velocities must be different, as shown below

```

mybot.mag_plot = 2 #coefficient of magnification of the arrow
mybot.step_plot = 10 # plot location every 100 iterations

mybot.pos_x = 0 #x
mybot.pos_y = 0 #y
mybot.angle = D2R(45) #alpha
mybot.length =0.4 #L
mybot.radius = 0.1 #r

mybot.plot = True #True if you want to plot the robot's trajectory
mybot.plot_robot() #draw an arrow for the location of the robot at t=0

# move straight for 10m
mybot.rt_spd_left = 10 # omega_L
mybot.rt_spd_right = 10 # omega_R
mybot.move(10) #move the robot for 10s

# 2m radius turn
mybot.rt_spd_left = 18.8496
mybot.rt_spd_right = 12.5664
mybot.move(2.5)

mybot.plot_robot() #draw the new location

plt.xlim(-1,11)
plt.ylim(-1,11)
plt.xlabel(r"$x$")
plt.ylabel(r"$y$")
plt.show()
```

## Assignment

### 1 Iterative turn
In this first assignment the robot must start at $(0,0)$, drive horizontally to $(10,0)$, then vertically to $(10,10)$, horizontally to $0,10$ and finally vertically to $(0,0)$. Write a code using a for loop to move your robot. Within the for loop, turn your robot using a while loop. Use the same robot dimensions as above and an absolute angular velocity $\omega = 10$.

```
mybot = DDRobot()            # robot called 'enesbot'
mybot.mag_plot = 2 #coefficient of magnification of the arrow
mybot.step_plot = 50 # plot location every 100 iterations

mybot.pos_x = 0 #x
mybot.pos_y = 0 #y
mybot.angle = 0 #alpha
mybot.length =0.4 #L
mybot.radius = 0.1 #r
omega = 10
rot_time_increment = 0.01 #time increment for rotation
rotation_angle = D2R(90)

mybot.plot = True #True if you want to plot the robot's trajectory
mybot.plot_robot() #draw an arrow for the location of the robot at t=0

for i in range(4):

    # move straight for 10m
    mybot.rt_spd_left = omega # omega_L
    mybot.rt_spd_right = omega # omega_R
    mybot.move(10) #move the robot for 10s
    # turn 90 degrees to the left
    mybot.rt_spd_left = -omega # omega_L
    mybot.rt_spd_right = omega # omega_R
    alpha_0 = mybot.angle
    while (mybot.angle - alpha_0) < rotation_angle:
        mybot.move(rot_time_increment)

    mybot.plot_robot() #draw the new location

plt.xlim(-1,11)
plt.ylim(-1,11)
plt.xlabel(r"$x$")
plt.ylabel(r"$y$")
plt.show()

```

As you can see, the robot does not return to its origin. Quantify the error.

```
print('absolute error x: %1.1e m' %(math.sqrt(mybot.pos_x**2 +mybot.pos_y**2)))
```

    absolute error x: 8.1e-01 m

### 2 Exact rotation

Derive and implement an equation to determine the exact time needed for the robot to achieve a rotation of a given angle around its center. Quantify the error

```
# your code here
```

### 3 Circular motion

Derive and implement an equation to determine the paramters (angular velocities and motion time) needed for the robot to achieve an arc of a given radius and angle (from origin to end).

```
# your code here
```


```python
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
%matplotlib widget
```

# Fourier Transforms
    - The discrete Fourier transform is defined as

$$ y[k] = \sum_{n=0}^{N-1} e^{-2{\pi}in(k/N)}\,x[n]$$

    - k / N represents a specific frequency (dimensionless)
    - can be converted to a frequency(Hz) if you know the spacing in x

```python
x = np.linspace(-2*np.pi, 2*np.pi, 100)
y = np.cos(x) + 1j*np.sin(x)
eix = np.exp(1j*x)
eix_ = np.exp(-1j*x)
#plt.plot(x,y)
#plt.plot(x,eix+1,label=r'$e^i$')
plt.plot(x,np.sin(x),label=r'$\sin(x)$')
plt.plot(x,np.sin(3*x),label=r'$\sin(3x)$')
plt.plot(x,np.sin(-3*x),label=r'$\sin(-3x)$')
#plt.plot(x,np.exp(-1j*x),label=r'$e^-i$')
plt.legend()
```

    <matplotlib.legend.Legend at 0x7feaace8bd90>

        Figure

```python
import sympy as sp
N = sp.symbols('N')
e = (1+1/N)**N
(e**2).apart()
```

$\displaystyle \left(1 + \frac{1}{N}\right)^{2 N}$

```python

```

```python

```