# BT2101 Introduction to Ensemble Learning

## 1 Goal

In this notebook, we will explore **Ensemble Learning** including:
* Bagging
* Random Forest
* AdaBoost

For the **Decision Tree** method, you will:
* Use open-source package to do ensemble learning
* Compare performances of different methods

```python
# -*- coding:utf-8 -*-
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from math import sqrt
from __future__ import division
%matplotlib inline
```

## 2 Ensemble Learning

### 2.1 Bias-Variance Decomposition

\begin{align}
E[(y-\hat{f}(x))^2] &= (E[\hat{f}(x)-f(x)])^2 + E[\hat{f}(x)-E[\hat{f}(x)]]^2 + \sigma^2  \\
&= \text{Bias}[\hat{f}(x)]^2 + \text{Var}[\hat{f}(x)] + \text{Irreducibel Error}
\end{align}

A figure to illustrate **Bias** and **Variance**:

A chart to understand the tradeoff of **Bias** and **Variance**:

An example to show **Overfit** problem:

Which line is more preferred? Black or Green?

### 2.2  Ensemble Learning
Ensemble learning methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone ([Wikipedia](https://en.wikipedia.org/wiki/Ensemble_learning)).

Ensemble learning is an appropriate way to somehow "relieve" overfitting problem (Remember that ensemble learning methods may still overfit). From the lecture class, we know that a basic workflow of ensemble method:

Examples of Ensemble Learning methods include:

Bootstrap Aggregating (aka.Bagging)

Random Forest

Boosting

## 3 Examples

### 3.1 Case on Handwritten Digit Recognition

#### Dataset:

The **Kaggle** competition dataset can be obtained from https://www.kaggle.com/c/digit-recognizer/data.

#### Overview:

MNIST ("Modified National Institute of Standards and Technology") is the de facto “hello world” dataset of computer vision. Since its release in 1999, this classic dataset of **handwritten images** has served as the basis for benchmarking classification algorithms. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.

In this competition, your goal is to **correctly identify digits** from a dataset of tens of thousands of handwritten images. We’ve curated a set of tutorial-style kernels which cover everything from regression to neural networks. We encourage you to experiment with different algorithms to learn first-hand what works well and how techniques compare.

#### Acknowlegements:

More details about the dataset, including algorithms that have been tried on it and their levels of success, can be found at http://yann.lecun.com/exdb/mnist/index.html. The dataset is made available under a Creative Commons Attribution-Share Alike 3.0 license.

#### Attributes:

The data files train.csv and test.csv contain gray-scale images of hand-drawn digits, from zero through nine.

Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.

The training data set, (train.csv), has 785 columns. The first column, called "label", is the digit that was drawn by the user. The rest of the columns contain the pixel-values of the associated image.

Each pixel column in the training set has a name like pixelx, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixelx is located on row i and column j of a 28 x 28 matrix, (indexing by zero).

The test data set, (test.csv), is the same as the training set, except that it does not contain the "label" column.

The evaluation metric for this contest is the categorization accuracy, or the proportion of test images that are correctly classified. For example, a categorization accuracy of 0.97 indicates that you have correctly classified all but 3% of the images.

```python
# Load dataset: You need to download dataset first
%pwd
train = pd.read_csv('./train.csv')
```

```python
# 42,000 pictures; Each picture is composed of 28*28 dimensional pixels
train.shape
```

    (42000, 785)

```python
# What does an image look like
plt.imshow(np.array(train.iloc[1,1:]).reshape((28, 28)), cmap="gray") #reshape into a 28 x 28 matrix with the remaining columns
plt.title("This digit is %d" % train.iloc[1,0]) # first column is the answer (index 0)
plt.show()
```

**Create a single decision tree model**

```python
# Import libraries
from sklearn import datasets
from sklearn.cross_validation import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix

# Tranform dataframe to array
train_feature = train.iloc[:,1:].values #all but the first column
train_target = train.iloc[:,0].values #the first column

# Split train data: 70% for model fit, 30% for validation
X_train, X_valid, y_train, y_valid = train_test_split(train_feature, train_target, test_size=0.3, random_state=12345)

# Single decision tree model
DT = DecisionTreeClassifier(criterion='entropy', random_state=12345)

# Fit decision tree model
DT_model = DT.fit(X_train, y_train)
DT_model.classes_

# Validation
y_pred_valid = DT_model.predict(X_valid)

# Performance of Random Forest model
cm = confusion_matrix(y_valid, y_pred_valid)
print(cm)
print("Accuracy: ", accuracy_score(y_valid, y_pred_valid))
```

    C:\Users\bensj\Anaconda3\lib\site-packages\sklearn\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
      "This module will be removed in 0.20.", DeprecationWarning)

    [[1162    0   16    8    9   15   22    0   12    4]
     [   0 1346   12    3    3    5    5    2   20    5]
     [  12   16 1045   40   17   18   22   22   35   12]
     [   5    5   43 1109    5   79    6   20   29   35]
     [   6    6   14    7 1001    9   19   11   26   83]
     [  21    7   12   56   14  926   38   11   44   26]
     [  27    6   19   12   34   34 1090    1   11   13]
     [   5   16   26    9   17    4    0 1170   10   44]
     [  12    8   25   46   28   45   23   14  998   29]
     [  15    7    8   31   72   27    5   47   31 1020]]
    Accuracy:  0.8624603174603175

### 3.2 Bagging

A basic procedure for **Bagging** is:

```python
# Bootstrap Aggregating Package
from sklearn.ensemble import BaggingClassifier
from sklearn.cross_validation import train_test_split

# Tranform dataframe to array
train_feature = train.iloc[:,1:].values
train_target = train.iloc[:,0].values

# Split train data: 70% for model fit, 30% for validation
X_train, X_valid, y_train, y_valid = train_test_split(train_feature, train_target, test_size=0.3, random_state=12345)
```

```python
# Fit Bagging Model; A bundle of decision trees
# Just wait a minute...It will take some time
BA = BaggingClassifier(n_estimators=100, random_state=12345)
BA_model = BA.fit(X_train, y_train)
BA_model.classes_
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)

```python
# Validation
y_pred_valid = BA_model.predict(X_valid)
```

```python
# Performance of Random Forest model
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix
cm = confusion_matrix(y_valid, y_pred_valid)
print(cm)
print("Accuracy: ", accuracy_score(y_valid, y_pred_valid))
```

    [[1215    0    6    0    3    5    6    1   11    1]
     [   0 1378   10    4    1    3    2    1    2    0]
     [  11    7 1160   12    6    1    7   15   16    4]
     [   5    3   27 1231    5   16    1   12   24   12]
     [   2    4    4    0 1111    2    6    5    7   41]
     [  15    3    2   20    4 1066   19    5   14    7]
     [   7    0    2    1    7   15 1201    2   12    0]
     [   1    6   17    1    3    0    1 1249    4   19]
     [   5    6    9   11    6   18   10    5 1135   23]
     [   7    4    8   18   26    8    1   11   12 1168]]
    Accuracy:  0.9455555555555556

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html

### 3.3 Random Forest

A basic procedure for **Random Forest** is:

```python
# Random Forest package
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split

# Tranform dataframe to array
train_feature = train.iloc[:,1:].values
train_target = train.iloc[:,0].values

# Split train data: 70% for model fit, 30% for validation
X_train, X_valid, y_train, y_valid = train_test_split(train_feature, train_target, test_size=0.3, random_state=12345)
```

```python
# Fit Random Forest Model; Binary Splitting using Entropy
RF = RandomForestClassifier(criterion='entropy', n_estimators=100, random_state=12345)
RF_model = RF.fit(X_train, y_train)
RF_model.classes_
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)

```python
# Validation
y_pred_valid = RF_model.predict(X_valid)
```

```python
# Performance of Random Forest model
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix
cm = confusion_matrix(y_valid, y_pred_valid)
print(cm)
print("Accuracy: ", accuracy_score(y_valid, y_pred_valid))
```

    [[1229    0    1    0    2    0    8    0    8    0]
     [   0 1385    5    4    3    1    1    0    2    0]
     [   6    3 1183    9    8    0   12   11    4    3]
     [   4    2   18 1260    1   18    0    9   15    9]
     [   2    4    2    0 1138    0    5    3    2   26]
     [   6    1    1   14    3 1104   12    1    9    4]
     [   6    1    0    0    5    9 1223    0    3    0]
     [   0    9   17    1    9    1    0 1244    4   16]
     [   3    4    7    8    6    7    9    1 1169   14]
     [   9    6    5   15   16    3    1   11    6 1191]]
    Accuracy:  0.9623809523809523

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

## 4 Boosting

A typical procedure for boosting method is:

Examples of boosting methods include:
1. Boosting based on weights: Adaptive boosting (Adaboost)
2. Boosting based on residuals: Gradient boosting decision trees (GBDT)

### 4.1 Adaboost

The algorithm for **Adaboost** is:

```python
# AdaBoost package
from sklearn.ensemble import AdaBoostClassifier
from sklearn.model_selection import cross_val_score
from sklearn.cross_validation import train_test_split

# Tranform dataframe to array
train_feature = train.iloc[:,1:].values
train_target = train.iloc[:,0].values

# Split train data: 70% for model fit, 30% for validation
X_train, X_valid, y_train, y_valid = train_test_split(train_feature, train_target, test_size=0.3, random_state=12345)
```

```python
# Fit Adaboosting Model
Ada = AdaBoostClassifier(n_estimators=100, random_state=12345)
Ada_model = Ada.fit(X_train, y_train)
Ada_model.classes_
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)

```python
# Validation
y_pred_valid = Ada_model.predict(X_valid)
```

```python
# Performance of Adaboosting model
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix
cm = confusion_matrix(y_valid, y_pred_valid)
print(cm)
print("Accuracy: ", accuracy_score(y_valid, y_pred_valid))
```

    [[1024    1   23    0    0   84   97    0   14    5]
     [   0 1337   10   11    2   15    3    8   15    0]
     [  25   68  530   57   24   26  386   21   86   16]
     [  33   44   21  828   12   93   55   31  177   42]
     [   3   11   41    7  913   22   22   38   21  104]
     [  32   24   15  149   15  659   55    2  152   52]
     [   5   13   45   11    9   39 1103    3   18    1]
     [  33   14   30   10   21   26    0  953    6  208]
     [   9   59   27   71   15   58   26    8  906   49]
     [   7   11   64   24  260    7    2  114   29  745]]
    Accuracy:  0.7141269841269842

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html

More about Ensemble Learning can be found at http://scikit-learn.org/stable/modules/ensemble.html.

## 5 Assignments

**Purpose: Comparing single decision tree model with ensemble methods (5 points)**

```python
# Import libraries
from sklearn import datasets
from sklearn.cross_validation import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix
from sklearn.model_selection import KFold, cross_val_score
from sklearn.ensemble import BaggingClassifier, RandomForestClassifier, AdaBoostClassifier
```

```python
# Load Dataset
# Tranform dataframe to array
train_feature = train.iloc[:,1:].values
train_target = train.iloc[:,0].values
```

```python
# 2-Fold Cross Validation
kf = KFold(n_splits=2, shuffle=True, random_state=12345)
```

**Create a single decision tree model**
* Set `criterion='entropy'`, and `random_state=12345`

```python
# Model 1: Single decision tree model
decision_tree = DecisionTreeClassifier(criterion='entropy', random_state=12345)

# Cross validation on Model 1
cv_model_1 = cross_val_score(decision_tree, # Cross-validation on Model 1
                             train.iloc[:,1:], # Feature matrix
                             train.iloc[:,0], # Output vector
                             cv=kf, # Cross-validation technique
                             scoring='accuracy' # Model performance metrics: accuracy
                            )

# Report performance of Model 1
print("Single decision tree model: %s" %(cv_model_1.mean()))
```

    Single decision tree model: 0.8485238095238095

**Create a bagging model**
* Set `n_estimators=100`, and `random_state=12345`
* Reference: http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html

```python
# Model 2: Bagging method
bagging = BaggingClassifier(n_estimators=100, random_state=12345)

# Cross validation on Model 2
cv_model_2 = cross_val_score(bagging, # Cross-validation on Model 2
                             train.iloc[:,1:], # Feature matrix
                             train.iloc[:,0], # Output vector
                             cv=kf, # Cross-validation technique
                             scoring='accuracy' # Model performance metrics: accuracy
                            )

# Report performance of Model 2
print("Bagging model: %s" %(cv_model_2.mean()))
```

    Bagging model: 0.9398333333333333

**Question 1. Create a random forest model (2 points)**
* Set `criterion='entropy'`, `n_estimators=100`, and `random_state=12345`
* Reference: http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

```python
# Model 3: Random Forest method
random_forest = RandomForestClassifier(criterion = 'entropy', n_estimators = 100, random_state = 12345)

# Cross validation on Model 3
cv_model_3 = cross_val_score(random_forest, # Cross-validation on Model 3
                             train.iloc[:,1:], # Feature matrix
                             train.iloc[:,0], # Output vector
                             cv=kf, # Cross-validation technique
                             scoring='accuracy' # Model performance metrics: accuracy
                            )

# Report performance of Model 3
print("Random forest model: %s" %(cv_model_3.mean()))
```

    Random forest model: 0.9576190476190476

**Question 2. Create an Adaboost model (2 points)**
* Set `n_estimators=100`, and `random_state=12345`
* Reference: http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html

```python
# Model 4: AdaBoosting method
Adaboost = AdaBoostClassifier(n_estimators = 100, random_state = 12345)

# Cross validation on Model 4
cv_model_4 = cross_val_score(Adaboost, # Cross-validation on Model 4
                             train.iloc[:,1:], # Feature matrix
                             train.iloc[:,0], # Output vector
                             cv=kf, # Cross-validation technique
                             scoring='accuracy' # Model performance metrics: accuracy
                            )

# Report performance of Model 4
print("Adaboost model: %s" %(cv_model_4.mean()))
```

    Adaboost model: 0.7053809523809524

**Question 3. Based on your results, what is your finding? (1 point)**

Hint: Just write your findings based on your model results.

```python
'''
The order of accuracy of models for this dataset, starting from the most accurate one is as follow:
1) Random forest (ensemble)
2) Bagging (ensemble)
3) Decison tree (single classification)
4) Adaboost (ensemble)
'''
```

    '\nThe order of accuracy of models for this dataset, starting from the most accurate one is as follow:\n1) Random forest (ensemble)\n2) Bagging (ensemble)\n3) Decison tree (single classification)\n4) Adaboost (ensemble)\n'

## 6 References

[1] Chris Albon. (2018). Machine Learning with Python Cookbook. O'Reilly.


# Laboratório 9: Populações de Urso

### Referente ao capítulo 15

O cenário neste laboratório é motivado pelo *Great Smoky Mountain National Park*. É considerad a população de ursos em um parque genérico em áreas populadas por humanos, tal que essas regiões apresentem bordas compartilhadas. A emigração e imigração de ursos é baseada na conectividade entre caminhos.

|Parâmetro|Significado|
|---------|-----------|
|$r$|taxa de crecimento de ursos|
|$K$|capacidade de carga|
|$m_p$|proporção da fronteira do parque conectada com a floresta|
|$m_f$|proporção da fronteira da floresta conectada com o parque|

Seja $P(t), F(t)$ e $O(t)$ a densidade de ursos no parque, na floresta e nas regiões de fora. Os controles serão $u_p$ e $u_f$, que são as taxas de caça de ursos no parque e na floresta, respectivamente.

Atualmente a caça em parques não é legalizada, mas o modelo considera que ocorrem também, além da caça na floresta. Queremos minimizar o número de ursos nas áreas populadas por humanos. Assim, o problema pode ser descrito pelo seguinte modelo.

$$ \min_{u_p, u_f} \int_0^T O(t) + c_pu_p(t)^2 + c_fu_f(t)^2 dt $$

$$\text{sujeito a  }P'(t) = rP(t) - \frac{r}{K}P(t)^2 + \frac{m_fr}{K}\left(1 - \frac{P(t)}{K}\right)F(t)^2 - u_p(t)P(t),$$
$$F'(t) = rF(t) - \frac{r}{K}F(t)^2 + \frac{m_pr}{K}\left(1 - \frac{F(t)}{K}\right)P(t)^2 - u_f(t)F(t),$$
$$O'(t) = r(1 - m_p)\frac{P(t)^2}{K} + r(1 - m_f)\frac{F(t)^2}{K} + \frac{m_fr}{K^2}P(t)F(t)^2 + \frac{m_pr}{K^2}P(t)^2F(t),$$

$$P(0) = P_0 \geq 0, F(0) = F_0 \geq 0, O(0) = O_0 \geq 0, $$
$$0 \leq u_p(t) \leq 1, 0 \leq u_f(t) \leq 1 $$

O termo $\frac{r}{K}P(t)^2$ representa a emigração densidade-depentente. O terceiro termo na primeira equação é a imigração da floresta densidade-dependente e $u_pP$ o nível de caça. O primeiro termo da terceira equação é a emigração do parque para regiões fora do parque, enquanto o segundo é a emigração da floresta para outras regiões.

### Importanto as bibliotecas

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
import sympy as sp

import sys
sys.path.insert(0, '../pyscripts/')

from optimal_control_class import OptimalControl
```

## Condições Necessárias

Vamos utilizar a bilbioteca `sympy` para nos ajudar a escrever as condições necessárias.

```python
P, F, O, uf, up, l1, l2, l3, cp, cf, r, K, mf, mp, t = sp.symbols('P F O uf up l1 l2 l3 cp cf r K mf mp t')
```

### O Hamiltoniano

\begin{multline}
H(t, \vec{x}, \vec{u}, \vec{\lambda}) = O(t) + c_pu_p(t)^2 + c_fu_f(t)^2 +  \\
\lambda_1(t)\left[rP(t) - \frac{r}{K}P(t)^2 + \frac{m_fr}{K}\left(1 - \frac{P(t)}{K}\right)F(t)^2 - u_p(t)P(t)\right] + \\
\lambda_2(t)\left[rF(t) - \frac{r}{K}F(t)^2 + \frac{m_pr}{K}\left(1 - \frac{F(t)}{K}\right)P(t)^2 - u_f(t)F(t)\right] + \\
\lambda_3(t)\left[r(1 - m_p)\frac{P(t)^2}{K} + r(1 - m_f)\frac{F(t)^2}{K} + \frac{m_fr}{K^2}P(t)F(t)^2 + \frac{m_pr}{K^2}P(t)^2F(t)\right]
\end{multline}

```python
H =  O + cp*up**2 + cf*uf**2 + l1*(r*P - r*P**2/K + mf*(r/K)*(1 - P/K)*F**2 - up*P) + \
                               l2*(r*F - r*F**2/K + mp*(r/K)*(1 - F/K)*P**2 - uf*F) + \
                               l3*(r*(1 - mp)*P**2/K + r*(1 - mf)*F**2/K + mf*r*P*F**2/K**2 + mp*r*P**2*F/K**2)
```

### Condições do Estado

```python
print(sp.diff(H,l1), '\n')
print(sp.diff(H,l2), '\n')
print(sp.diff(H,l3))
```

    F**2*mf*r*(1 - P/K)/K + P*r - P*up - P**2*r/K

    -F**2*r/K + F*r - F*uf + P**2*mp*r*(-F/K + 1)/K

    F**2*r*(1 - mf)/K + F**2*P*mf*r/K**2 + F*P**2*mp*r/K**2 + P**2*r*(1 - mp)/K

### Equações Adjuntas

```python
dl1 = (-1)*sp.diff(H,P)
dl2 = (-1)*sp.diff(H,F)
dl3 = (-1)*sp.diff(H,O)
print(dl1, '\n')
print(dl2, '\n')
print(dl3)
```

    -l1*(-F**2*mf*r/K**2 + r - up - 2*P*r/K) - l3*(F**2*mf*r/K**2 + 2*F*P*mp*r/K**2 + 2*P*r*(1 - mp)/K) - 2*P*l2*mp*r*(-F/K + 1)/K

    -2*F*l1*mf*r*(1 - P/K)/K - l2*(-2*F*r/K + r - uf - P**2*mp*r/K**2) - l3*(2*F*r*(1 - mf)/K + 2*F*P*mf*r/K**2 + P**2*mp*r/K**2)

    -1

### Condições de Otimalidade

```python
du1 = sp.diff(H,up)
du2 = sp.diff(H,uf)
print(du1, '\n')
print(du2)
```

    -P*l1 + 2*cp*up

    -F*l2 + 2*cf*uf

$$ \frac{\partial H}{\partial u_p} > 0 \implies u_p^*(t) = 0 \implies \frac{P\lambda_1}{2c_p} < 0$$
$$ \frac{\partial H}{\partial u_p} < 0 \implies u_p^*(t) = 1 \implies \frac{P\lambda_1}{2c_p} > 1$$
$$ \frac{\partial H}{\partial u_p} = 0 \implies 0 \leq u_p^*(t) = \frac{P\lambda_1}{2c_p} \leq 1$$

$$u_p^* = \max(0, \min(1, \frac{P\lambda_1}{2c_p}))$$

$$ \frac{\partial H}{\partial u_f} > 0 \implies u_f^*(t) = 0 \implies \frac{F\lambda_2}{2c_f} < 0$$
$$ \frac{\partial H}{\partial u_f} < 0 \implies u_f^*(t) = 1 \implies \frac{F\lambda_2}{2c_f} > 1$$
$$ \frac{\partial H}{\partial u_f} = 0 \implies 0 \leq u_f^*(t) = \frac{F\lambda_2}{2c_f} \leq 1$$

$$u_f^* = \max(0, \min(1, \frac{F\lambda_2}{2c_f}))$$

## Aplicando a classe ao exemplo

Nesse laboratório, o leitor é convidado a escrever as equações e realizar os testes por conta. Você deve mudar a célula de Raw para Code e escrever as funções como fizemos até então.
parameters = {'r': None, 'K': None, 'mp': None, 'mf': None, 'cp': None, 'cf': None}

# Lembre: P: x[0], F: x[1], O: x[2], up: u[0], uf: u[1]
diff_state = lambda t, x, u, par: np.array([
    x[1]**2*par['mf']*par['r']*(1 - x[0]/par['K'])/par['K'] + x[0]*par['r'] - \
        x[0]*u[0] - x[0]**2*par['r']/par['K'],
    -x[1]**2*par['r']/par['K'] + x[1]*par['r'] - x[1]*u[1] + \
        x[0]**2*par['mp']*par['r']*(-x[1]/par['K'] + 1)/par['K'],
    x[1]**2*par['r']*(1 - par['mf'])/par['K'] + x[1]**2*x[0]*par['mf']*par['r']/par['K']**2 + \
        x[1]*x[0]**2*par['mp']*par['r']/par['K']**2 + x[0]**2*par['r']*(1 - par['mp'])/par['K']
])

diff_lambda = lambda t, x, u, l, par: np.array([
    -l[0]*(-x[1]**2*par['mf']*par['r']/par['K']**2 + par['r'] - u[0] - 2*x[0]*par['r']/par['K']) - \
        l[2]*(x[1]**2*par['mf']*par['r']/par['K']**2 + 2*x[1]*x[0]*par['mp']*par['r']/par['K']**2 + \
        2*x[0]*par['r']*(1 - par['mp'])/par['K'])-2*x[0]*l[1]*par['mp']*par['r']*(1-x[1]/par['K'])/par['K'],
    -2*x[1]*l[0]*par['mf']*par['r']*(1 - x[0]/par['K'])/par['K'] - \
        l[1]*(-2*x[1]*par['r']/par['K'] + par['r'] - u[1] - x[0]**2*par['mp']*par['r']/par['K']**2) - \
        l[2]*(2*x[1]*par['r']*(1 - par['mf'])/par['K'] + 2*x[0]*x[1]*par['mf']*par['r']/par['K']**2 + \
        x[0]**2*par['mp']*par['r']/par['K']**2),
    -1
])

update_u = lambda t, x, lambda_, par: np.array([
    np.maximum(0.0, np.minimum(1.0, 0.5*x[0]*lambda_[0]/par['cp'])),
    np.maximum(0.0, np.minimum(1.0, 0.5*x[1]*lambda_[1]/par['cf']))
])problem = OptimalControl(diff_state, diff_lambda, update_u,
                         n_controls = 2, n_states = 3,
                         bounds = [(0,1), (0,1)])x0 = np.array([0.4, 0.2, 0])
T = 25
parameters['r'] = 0.1
parameters['K'] = 0.75
parameters['mp'] = 0.5
parameters['mf'] = 0.5
parameters['cp'] = 10000
parameters['cf'] = 10t,x,u,lambda_ = problem.solve(x0, T, parameters)
### Este é o final do notebook


# Introduction

This exercise solution is built on top of [this blog post by Hashir Ahmad](https://hash-ir.github.io/2020/05/17/bike-sharing-patterns/). This exercise will provide a step by step guide to building a multilayer perceptron to predict the count of rental bike usage given various features.

The python packages we will be using are:

[NumPy](https://numpy.org/): Powerful tool to work with N-dimensional arrays

[Pandas](https://pandas.pydata.org/): Data analysis and manipulation tool

[Matplotlib](https://matplotlib.org/): Visualization tool

```
import numpy as np
import pandas as pd
import time
import matplotlib.pyplot as plt
```

# Data Preprocessing

Real data is always dirty! Before we can do any machine learning, we need to process the data so that it is in good shape for training our model.

## Raw Data
Lets take a look at the data we will be working with, it is taken from the [University of California Irvine Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets/Bike+Sharing+Dataset#), it contains the count of rental bikes between years 2011 and 2012 in Capital bikeshare system with the corresponding weather and seasonal information. Each data entry contains information for one hour of data and there are 17379 entries in total.

```
# read the raw data into a pandas dataframe
data_path = "https://raw.githubusercontent.com/xuwil/99-ML-Learning-Projects/development/011/data/"

raw_df = pd.read_csv(data_path + 'hour.csv')

# take a look at the shape of our raw data, there are 17379 entries and each entry contains 17 fields
raw_df.shape
```

```
raw_df.head()
```

```
raw_df[:24*10].plot(x='dteday', y='cnt')
```

It has the following fields:
```
instant: record index
dteday : date
season : season (1:winter, 2:spring, 3:summer, 4:fall)
yr : year (0: 2011, 1:2012)
mnth : month ( 1 to 12)
hr : hour (0 to 23)
holiday : weather day is holiday or not
weekday : day of the week (0 to 6)
workingday : if day is neither weekend nor holiday is 1, otherwise is 0.
weathersit :
- 1: Clear, Few clouds, Partly cloudy, Partly cloudy
- 2: Mist + Cloudy, Mist + Broken clouds, Mist + Few clouds, Mist
- 3: Light Snow, Light Rain + Thunderstorm + Scattered clouds, Light Rain + Scattered clouds
- 4: Heavy Rain + Ice Pallets + Thunderstorm + Mist, Snow + Fog
temp : Normalized temperature in Celsius. The values are derived via (t-t_min)/(t_max-t_min), t_min=-8, t_max=+39 (only in hourly scale)
atemp: Normalized feeling temperature in Celsius. The values are derived via (t-t_min)/(t_max-t_min), t_min=-16, t_max=+50 (only in hourly scale)
hum: Normalized humidity. The values are divided to 100 (max)
windspeed: Normalized wind speed. The values are divided to 67 (max)
casual: count of casual users
registered: count of registered users
cnt: count of total rental bikes including both casual and registered
```
Our goal is to predict the count of bike rentals (ie. casual, registered, cnt) given the other fields (features).

## Unused Features

Is every feature useful in making this prediction? Consider:

**instant:** the record index does not provide any useful information about the count of bike rentals

**dteday:** the date does not contribute to the count of bike rentals, the information it offers is captured in the other features (if the date is a holiday,if the date is a weekday or a weekend)

**atemp:** the information this field offers is captured in the `temp` feature.

**workingday:** the information this field offers is captured int the `weekday` and `holiday` features.

Since these features offers no information (or duplicate information) for predicting the count of bike rentals, we first want to clean up the data by removing them.

```
unused_features = ['instant', 'dteday', 'atemp', 'workingday']
processed_df = raw_df.drop(unused_features, axis=1)  # drop the unused features
processed_df.head()
```

## Categorical Features
Is the data ready to be used now?

Consider the feature `mnth` which has values from 1 - 12 representing Janurary to December. The integer values have a natural ordered relationship between each other. However, should Feburary be ranked higher than January?

No, no such ordinal relationship exists between each month. Therefore,
we cannot use them as is, we need to turn them into one-hot encoded form.

Other features that needs to be one-hot encoded includes:
`season, weathersit, hr, weekday`

[Further Reading on Categorical Features](https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/)

```
# There are many ways to implement one hot encoding:
# 1. using numpy
def numpy_one_hot(df, features_to_encode):
  """
  transform categorical data from 0-n into one-hot encoded form
  https://stackoverflow.com/questions/38592324/one-hot-encoding-using-numpy

  uses https://numpy.org/doc/stable/reference/generated/numpy.eye.html
  np.eye(3) = [[1., 0., 0.],
               [0., 1., 0.],
               [0., 0., 1.]]
  np.eye(3)[[1,1,1,0]] = [[0., 1., 0.],
                          [0., 1., 0.],
                          [0., 1., 0.],
                          [1., 0., 0.]]
  """
  # this method only works when the categorical data is valued from 0 to n
  # we need to adjust season, weathersit, mnth
  adjust_categorical_features = ["season", "weathersit", "mnth"]

  for feature in features_to_encode:
    num_categories = len(processed_df[feature].unique())
    if feature in adjust_categorical_features:
      targets = processed_df[feature].values - 1
    else:
      targets = processed_df[feature].values
    print(feature)
    one_hot_targets = np.eye(num_categories)[targets]

    # add the one hot encoded feature back into the dataframe
    df[["{}_{}".format(feature,i) for i in range(num_categories)]] = pd.DataFrame(one_hot_targets, index=df.index)
  return df

# 2. using pandas
def pandas_one_hot(df, features_to_encode):
  """
  pandas function to transform categorical data to one-hot
  https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.get_dummies.html
  """
  for feature in categorical_features:
    dummies = pd.get_dummies(df[feature], prefix=feature, drop_first=False)
    df = pd.concat([df, dummies], axis=1)
  return df

# 3. using sklearn
# https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html
```

```
# identify the features that needs to be one-hot encoded
categorical_features = ['season', 'weathersit', 'mnth', 'hr', 'weekday']

# processed_df = numpy_one_hot(processed_df, categorical_features)
processed_df = pandas_one_hot(processed_df, categorical_features)

# we can now drop the categorical features since they are no longer needed
processed_df = processed_df.drop(categorical_features, axis=1)
processed_df.head()
```

## Continuous Features
Is the data ready to be used now? Consider:

`temp`: 30 degrees is larger than 20 degrees

`windspeed`: 15 m/s is larger than 10 m/s

However, the range of the two features are different. In extreme cases, one feature has range so large that it will dominate the other features. Therefore, we need to perform feature scaling. In our case, we will perform standardization  so that each feature has values with mean of 0 and standard deviation of 1.

[Further Reading on Feature Scaling](https://www.analyticsvidhya.com/blog/2020/04/feature-scaling-machine-learning-normalization-standardization/)

```
# identify contiunous fields that needs to be standardized
continuous_features = ['casual', 'registered', 'cnt', 'temp', 'hum', 'windspeed']

# create mapping to revert values in later steps
scaled_features = {}
for feature in continuous_features:
    mean, std = processed_df[feature].mean(), processed_df[feature].std() # calculate mean and standard deviation of the feature
    scaled_features[feature] = [mean, std]  # store the values
    processed_df.loc[:, feature] = (processed_df[feature] - mean)/std # standardized values

processed_df.head()
```

## Train, Validation, Test split

One crucial step remains before our data is ready to be used for machine learning.

We can not use the same data to train and to evaluate the model, since the model can simply remember the entire dataset and fail to [generalize](https://deepai.space/what-is-generalization-in-machine-learning/). Therefore, we need to split the data into three seperate sets.

**Training Set** will be used to train the model.

**Validation Set** will be used to tune the hyper parameters (Will be covered in a later section)

**Test Set** will be used to evaluate the model's true performance on data that it has not seen before.

We will split our data 70%|20%|10% for Train|Validation|Test

[Further Reading on Dataset Split](https://machinelearningmastery.com/difference-test-validation-datasets/)

```
# determine the number of entries in each set
total_entries = processed_df.shape[0]
train_entries = int(total_entries * 0.7)
val_entries = int(total_entries * 0.2)

# Split dataframe into train, validation and test
train_data = processed_df[:train_entries]
validation_data = processed_df[train_entries:train_entries+val_entries]
test_data = processed_df[train_entries+val_entries:]

# Separate the data into features and targets
label_fields = ['cnt', 'casual', 'registered']
train_features, train_labels = train_data.drop(label_fields, axis=1), train_data[label_fields]
validation_features, validation_labels = validation_data.drop(label_fields, axis=1), validation_data[label_fields]
test_features, test_targets = test_data.drop(label_fields, axis=1), test_data[label_fields]
```

```
train_features.head()
```

```
train_labels.head()
```

```
train_features.shape
```

In the end, we have 12165 entries to train our model with, each entry contains 56 features

# Multi Layer Perceptron

## Overview
Now that we have the data, how do we turn the features into a prediction?

We will be using a simple multilayer perceptron. It will take in the 56 features and output a predicted label `cnt`.

The simple MLP we will be constructing looks something like

The 56 features are fed into the MLP as inputs, they will be processed through a hidden layer (size to be determined) and a prediction will be produced through a output layer of size 1.

Let's take a closer look at one hidden layer node.

The output of this node is obtained by finding the dot product between the input vector and the weights vector, and then putting the results through an activation function $\phi$

\begin{equation*}
h = \phi \left( \sum_{i=1}^{56}x_i w_i \right)
\end{equation*}

[Further Reading on MLP](https://machinelearningmastery.com/neural-networks-crash-course/)

## Activation Functions

The activation function $\phi$ is a mathematical “gate” that transforms the dot product of input vector and weights vector into an output going to the next layer. Non-linear activation functions allows the model to learn much more complex data, since the hypothesis space for the model becomes non-linear.

A few of the most commonly used activation functions will be briefly introduced below.

[Further Reading on Activation Functions](https://medium.com/@himanshuxd/activation-functions-sigmoid-relu-leaky-relu-and-softmax-basics-for-neural-networks-and-deep-8d9c70eed91e)

```
def visualize(activation_function):
  # helper function to visualize activation functions
  # https://stackoverflow.com/a/42170061
  x = np.arange(-10, 10, 0.1)
  val = activation_function(x)
  fig = plt.figure()
  ax = fig.add_subplot(1, 1, 1)
  ax.spines['left'].set_position(('data', 0.0))
  ax.spines['bottom'].set_position(('data', 0.0))
  ax.spines['right'].set_color('none')
  ax.spines['top'].set_color('none')
  plt.plot(x,val)
  plt.title("{} activation function".format(activation_function.__name__))
  plt.show()
```

### 1.ReLU

Rectified Linear Unit(ReLU) has output 0 if the input is less than 0, and raw output otherwise.

\begin{equation*}
ReLU(x) = max(0, x)
\end{equation*}

```
def relu(x):
    return np.maximum(0,x)

visualize(relu)
```

### 2.Leaky ReLU

Leaky ReLU is a modified version of ReLU. It has a small slope for negative values defined by parameter $\alpha$, instead of zero. This gives an indication of how wrong the perdiction was which ReLu lacks.

\begin{equation}
  LeakyReLU(x) =
    \begin{cases}
      \alpha x & \text{if } x < 0\\
      x & \text{if } x > 0\\
      0 & \text{otherwise}
    \end{cases}
\end{equation}

[Further Reading on ReLU vs Leaky ReLU](https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7)

```
def leaky_relu(x):
  alpha = 0.1
  return np.where(x > 0, x, x * alpha)

visualize(leaky_relu)
```

### 3.Sigmoid
Sigmoid function output a number between 0 and 1.
If the input is very negative, the output is approximately 0. If the input is very positive, the output is approximately 1. If the input is 0, the output is 0.5

\begin{equation*}
Sigmoid(x) = \frac{1}{1 + e^{-x}}
\end{equation*}

```
def sigmoid(x):
    return 1/(1+np.exp(-x))

visualize(sigmoid)
```

### 4.Tanh

Tanh function is a scaled version of the sigmoid function, it outputs a number between -1 and 1 instead.
If the input is very negative, the output is approximately -1. If the input is very positive, the output is approximately 1. If the input is 0, the output is 0.

\begin{equation*}
Tanh(x) = \frac{2}{1 + e^{-2x}} -1
\end{equation*}

```
def tanh(x):
    return np.tanh(x)

visualize(tanh)
```

## Initialization

We will be initializing our weights using the Xavier Initialization:
\begin{equation*}
W \sim \mathcal{N}(\mu = 0,\,\sigma^{2} = \frac{1}{\text{previous layer size}})\,.
\end{equation*}

[Further Reading on Weight Initialization](https://www.deeplearning.ai/ai-notes/initialization/)

```
# Initialize our Multilayer Perceptron
class MLP(object):
    def __init__(self, input_size, hidden_size, output_size, learning_rate):
        # Set number of nodes in input, hidden and output layers.
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size

        # Initialize weights
        # np.random.normal takes the standard deviation as input, so we take the square root of layer size
        # https://numpy.org/doc/stable/reference/random/generated/numpy.random.normal.html
        self.weights_input_to_hidden = np.random.normal(0.0, self.input_size**-0.5,
                                       (self.input_size, self.hidden_size))

        self.weights_hidden_to_output = np.random.normal(0.0, self.hidden_size**-0.5,
                                       (self.hidden_size, self.output_size))

        # Initialize learning rate (see later section)
        self.learning_rate = learning_rate

        # Sigmoid activation function
        self.activation_function = sigmoid
```

# Learning Process

Now its time to start looking at how the the weights are optimized. A high level explanation of this that the training is fed into the initialized model and the output is compared to the known values. The comparing is down using what is known as a loss function, discussed later, this returns an indication of how wrong the prediction was. This loss is then used to adjust the weights of each neuron.

## Forward Pass

Forward pass is the process of feeding the data through the layers of neurons to generate a perdiction. As discussed above each neuron generates an output using \begin{equation*}
h = \phi \left( \sum_{i=1}^{N}x_i w_i \right)
\end{equation*}

You might recognize this is the same as a dot product of the x and w vectors.
\begin{equation*}
h = \phi \left( \vec{x} \cdot \vec{w_1} \right)
\end{equation*}

Here $\vec{x}$ is a vector of one row of input data, represented by the Input layer, and $\vec{w}$ is a vector of weights for one neutron. We will have to perform this operation for every neuron in the Hidden layer. Each neuron will have different values in their $\vec{w}$. The outputs of the Hidden layer will then be fed to the Output layer and be dotted with its $\vec{w}$. This will not be subject to an activation function as we want a real value of riders.
\begin{equation*}
o = \left( \vec{h} \cdot \vec{w_2} \right)
\end{equation*}

Let's now have a look at what the code for this looks like.

```
def forward_pass_train(model, X):
    ''' Forward pass through the network

        Arguments
        ---------
        X: features batch

    '''
    ### Forward pass ###
    # Please note that weights_input_to_hidden is actually a matrix of size num_inputs x num_hidden_layers
    # an example of np.dot dealing with a vector and an array is shown in the next cell
    hidden_inputs = # Your Code Here
    # The hidden_inputs variable is a vector of length num_hidden_layers
    hidden_outputs = # Your Code Here

    # Output layer
    final_outputs = # Your Code Here

    return final_outputs, hidden_outputs
```

```
"""
This is a mock example of np.dot acting on a vector and a matrix
"""
# We declare x as a vector and w as a matrix
x = np.array([1,2,3])
w = np.array([[2, 1, 3],[4,5,6],[1,1,1]])
print("x shape", x.shape)
print("w shape", w.shape)

dot = np.dot(x,w)

print("Dotting x and w resulted in", dot)

# Here we are demonstrating that np.dot acts along the first dimension of the matrix and iterates through the second dimension
dot2 = x[0]*w[0, 0] + x[1]*w[1, 0] + x[2]*w[2, 0]
print(dot2)
```

## Back Propagation

Now that we have found the output of the model we'll look at how we can use that to improve the perdiction the model makes. This will be done through Back Propagation which uses the error in the output to update the weights of the neurons.

The first step in Back Propogation is to get the gradients for each layer to find the error with respect to the different weights. To make determining what to take the gradients of easier let's first re-write the equations we use to get our lay outputs.
\begin{equation*}
h = \phi \left( \vec{x} \cdot \vec{w_1} \right)
\end{equation*}

\begin{equation*}
o = \left( \vec{h} \cdot \vec{w_2} \right) = \left( \phi \left( \vec{x} \cdot \vec{w_1} \right) \cdot \vec{w_2} \right)
\end{equation*}

Let us recall that a gradient is the rate of change of a function. From this we change realize that we are looking for the change in each of the functions above.
\begin{equation*}
\Delta o = \left( Y - o \right) \left( \vec{h} \cdot \vec{w_2} \right) = \left( Y - o \right)
\end{equation*}

Where Y is the expected output. The $ \left( \vec{h} \cdot \vec{w_2} \right) $ simplifies out as there is no activation function on the Output layer.

Since this is Back Propogation we take the gradient of the Output layer into account when finding the gradient of the Hidden layer:
\begin{equation*}
\Delta h = \left(\Delta o \cdot \vec{w_2}\right) * \phi' \left( \vec{x} \cdot \vec{w_1} \right) = \left(\left( Y - o \right) \cdot \vec{w_2}\right) * \phi \left( \vec{x} \cdot \vec{w_1} \right) \left(1 - \phi \left( \vec{x} \cdot \vec{w_1} \right)\right) = \left(\left( Y - o \right) \cdot \vec{w_2} \right) * h \left(1 - h\right)
\end{equation*}

After we have found these gradients there is one more step to Back Propagation, find the wieght step values. These are the amounts that each weight should be changed by proportional to the learning rate (we will discuss this later). To find the weight step values we simple multiply the gradients by the inputs to their respective neutrons and add the product to the current weight step values.

\begin{equation*}
\Delta_{w1}' = \Delta_{w1} + \Delta h * \vec{x}
\end{equation*}

\begin{equation*}
\Delta_{w2}' = \Delta_{w2} + \Delta o * \vec{h}
\end{equation*}

Let's now look at the code for this.

```
def backpropagation(model, final_outputs, hidden_outputs, X, y, delta_weights_i_h, delta_weights_h_o):
        ''' Backward pass through the network

            Arguments
            ---------
            final_outputs: output from forward pass
            y: target (i.e. label) batch
            delta_weights_i_h: change in weights from input to hidden layers
            delta_weights_h_o: change in weights from hidden to output layers

        '''
        ### Backward pass ###

        # Output error
        error = # Your Code Here Output layer error is the difference between desired target and actual output.

        # Hidden layer's contribution to the error
        hidden_error = # Your Code Here Is the error taking the weights into account

        # Backpropagated error terms
        hidden_error_term = # Your Code Here Hidden Error is the total error from the hidden layer

        # Weight step (input to hidden)
        delta_weights_i_h += X[:,None] * hidden_error_term
        # Weight step (hidden to output)
        delta_weights_h_o += hidden_outputs[:, None] * error

        return delta_weights_i_h, delta_weights_h_o
```

At this point you have seen learning rate mentioned a few times. Learning rate is the speed with which we change the weights. Initially it might seem that the higher the learning rate the sooner the model will be trained. Unfortunately this is not the case. A high learning rate can cause the model to overshoot the optimal weights. It might then jump back in the direct in came from and just keep jumping around never finding the solution. This will be demonstrated in the Hyperparameters Tuning section of this tutorial.

Now lets look at how we use the learing rate. We'll use $\eta$ to denote learning rate.

\begin{equation*}
w_1' = w_1 + \frac{\eta*\Delta_{w1}}{N}
\end{equation*}

\begin{equation*}
w_2' = w_2 + \frac{\eta*\Delta_{w2}}{N}
\end{equation*}

In the equations above w_1' and w_2' are the new values for the weights. The learning rate is multiplied by the weight step values to allow for control of the learning process. It is then divided by the batch size (batch size is the number of records grouped together it will be discussed in the Hyperparameter Tuning section) to normalize the weight step values.

```
def update_weights(model, delta_weights_i_h, delta_weights_h_o, n_records):
        ''' Update weights on gradient descent step

            Arguments
            ---------
            delta_weights_i_h: change in weights from input to hidden layers
            delta_weights_h_o: change in weights from hidden to output layers
            n_records: number of records (batch size)

        '''
        model.weights_hidden_to_output +=   # update hidden-to-output weights with gradient descent step
        model.weights_input_to_hidden +=  # update input-to-hidden weights with gradient descent step
```

# Training Loop

We have now assembled all the components that we need to train a model except the one that actually does it. The training loop is very simple all that is required is a loop in which the entire training set is run through the forward pass function with the perdicted output then fed to the back propagation function. After the loop the the weight step values are fed to the up date weights function.

Once these steps are due the model has been trained for one iteration. The code for the training function can be seen below.

```
    def train(model, features, targets):
        ''' Train the network on batch of features and targets.

            Arguments
            ---------

            features: 2D array, each row is one data record, each column is a feature
            targets: 1D array of target values

        '''
        n_records = features.shape[0]
        delta_weights_i_h = np.zeros(model.weights_input_to_hidden.shape)
        delta_weights_h_o = np.zeros(model.weights_hidden_to_output.shape)
        for X, y in zip(features, targets):
        # Your Training Function here. It should only go through the provided data once.

```

# Hyperparameters Tuning

**Convert train, validation and test datasets from Pandas to Numpy format**

In the beginning of this tutorial we loaded the data in using pandas as a DataFrame. Lets now convert our datasets to numpy arrays for simplicity.

```
"""
converting from Pandas to numpy
"""

np_train_features = train_features.to_numpy(dtype=float)
np_train_labels = train_labels.to_numpy(dtype=float)
print(np_train_features.shape)

np_validation_features = validation_features.to_numpy(dtype=float)
np_validation_labels = validation_labels.to_numpy(dtype=float)
print(np_validation_features.shape)

np_test_features = test_features.to_numpy(dtype=float)
np_test_targets = test_targets.to_numpy(dtype=float)
print(np_test_features.shape)
```

## Loss Function

Loss functions, also known as cost funcitons, are used while training the model to give an indication of how wrong the models perdiction is for the training dataset. For this demonstration we'll be using the mean-squared error loss function
\begin{equation*}
L = \frac{1}{N} \sum_{i=0}^{N} \left(y_i - o_i \right)^{2} = \frac{1}{N}\left(Y - o\right)^{T}\left(Y - o\right)
\end{equation*}

Where Y is the expected output and N is the total number of data points (rows)

You can read more about loss functions here
[Common Loss Functions in Machine Learning](https://towardsdatascience.com/common-loss-functions-in-machine-learning-46af0ffc4d23)

```
# Define the mean squared loss function
def MSE(y, Y):
    mean_squared_error = # Your Code Here. ## HINT ## There may be an easy way to do this with numpy
    return mean_squared_error
```

## Training the network

In principal, training the model is simply calling the train function in the model class defined above multiple times. Each time the train function is run the weights are updated and howfully improve the models prediction accuracy. There are, however, several factors which can affect how effect this training is:

Learning Rate - As previously discussed learning rate changes how quickly the weights are altered. Make it too high and you might keep overshooting the optimal solution, though low and the model will take a very long time to reach a good solution.

[More on Learning Rate Here](https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10)

Hidden Nodes - The number of Hidden Nodes can be a tricky hyperparameter to tune. Pick too few are there are not enough neurons to capture the complexity of the relation between the features and the target. Too many increases training time and can lead to the model essentially memorizing the correct output for the training dataset.

[More on Hidden Nodes Here](https://www.allaboutcircuits.com/technical-articles/how-many-hidden-layers-and-hidden-nodes-does-a-neural-network-need/)

Batch Size - The batch is a randomly selected subset of the training dataset which is selected each iteration and used to train the model for that iteration. Batching the data in this way can greatly speed up training speed. There is a caveat on that. The batch should be a decent representation of the dataset as a whole otherwise the model may adjust its weights in an undesirable direction. For this reason the batch size should not be too small. Making the batch size too large will result in a longer learning duration.

The code block below takes in the hyperparameters and trains the model based on them.

```
def train_model(learning_rate=.3, hidden_nodes=50, iterations=1000, output_nodes= 1, batch_size=128):

    # declare empty loss lists for visualization purposes
    training_loss = []
    validation_loss = []

    # Get the number of input features
    N_i = train_features.shape[1]

    # Initialize the model
    model = MLP(N_i, hidden_nodes, output_nodes, learning_rate)

    # Gets the time before training starts
    begin_time = time.time()

    # Loops the training for the desired number of iterations
    for i in range(iterations):
        # Go through a random batch from the training data set
        batch = np.random.choice(train_features.index, size=batch_size)
        X, y = train_features.loc[batch].values, train_labels.loc[batch]['cnt']

        # train the model for one iteration
        ## Your Code Here ##

        # compute losses
        train_output, _ = forward_pass_train(model, np_train_features)
        val_output, _ = forward_pass_train(model, np_validation_features)

        # Stores the loss values for graphing
        train_loss = MSE(train_output, np_train_labels[:,0])
        val_loss = MSE(val_output, np_validation_labels[:,0])

        #These functions are setting a ceiling on the loss values stored for the purposes of making the graph readable
        if(train_loss > 4):
            training_loss.append(4)
        else:
            training_loss.append(train_loss)

        if(val_loss > 4):
            validation_loss.append(4)
        else:
            validation_loss.append(val_loss)

        # Prints training information
        if(i%(iterations/5) == 0):
            print("Iteration:", i, "    Training Loss:", train_loss, "    Validation Loss:",  val_loss)

    total_time = time.time() - begin_time

    print("Total Training Time:", total_time)

    return model, training_loss, validation_loss

def graph_training(model, training_loss, validation_loss):

    # Graphes the loss per iteration
    plt.title("Train vs Validation Loss")
    plt.plot(training_loss, label="Train")
    plt.plot(validation_loss, label="Validation")
    plt.xlabel("Epoch")
    plt.ylabel("Loss")
    plt.legend(loc='best')
    plt.show()

    # Runs the datasets throught the trained model to see perdictions versus the actual numbers
    train_perd, _ = forward_pass_train(model, np_train_features)
    val_perd, _ = forward_pass_train(model, np_validation_features)

    #gets the feature scaling data for the number of riders so the original values are shown and the perdictions are adjusted to match them
    mean, std = scaled_features['cnt']

    # Plots the Perdictions versus the actual numbers
    plt.title("Perdictions VS Actual")
    plt.plot((train_perd[:20,0] * std) + mean, label="Training Perdiction")
    plt.plot((np_train_labels[:20,0] * std) + mean, label="Training Actual")
    plt.plot((val_perd[:20,0] * std) + mean, label="Validation Perdiction")
    plt.plot((np_validation_labels[:20,0] * std) + mean, label="Validation Actual")
    plt.xlabel("Record")
    plt.ylabel("Number of Riders")
    plt.legend(loc='best')
    plt.show()

```

```
model, training_loss, validation_loss = train_model()
graph_training(model, training_loss, validation_loss)
```

Now that you have seen some examples of the effects of tuning hyperparameters try some yourself!

You have also have a look at algorithms for tuning hyperparameters: like grid search see the code example below. Beware that grid searches can take a very long time.

```
"""
Example of Grid Search Code
"""
# Declare lists of hyperparameters to check
iteration = 100
hidden_node = (10,  50)
learning_rate = (.1, .5)
batch_size = (50, 1000)

# Initialize lists to store outputs
train_losses = []
val_losses = []
models = []
batchs = []
nodes = []
learning_rates = []
iterations = []

# Loop through Hyperparameter lists
for batch in batch_size:
    for hidden in hidden_node:
        for lr in learning_rate:
            model, train_loss, val_loss = train_model(learning_rate=lr, hidden_nodes=hidden, iterations=iteration, output_nodes= 1, batch_size=batch)

            train_losses.append(train_loss[-1])
            val_losses.append(val_loss[-1])
            models.append(model)
            batchs.append(batch)
            nodes.append(hidden)
            learning_rates.append(lr)
            iterations.append(iteration)

for i in range(len(train_losses)):
    print("Training Loss:", train_losses[i], "Validation Loss", val_losses[i], "Batch size:", batchs[i], "Nodes:", nodes[i], "Learning Rate:", learning_rates[i], "Iterations:", iterations[i])

```

## Show Trained Results

```
for i in range(len(models)):
    test_output, _ = forward_pass_train(models[i], np_test_features)

    mean, std = scaled_features['cnt']

    # Plots the Perdictions versus the actual numbers
    title = "Perdictions VS Actual Hidden Nodes:", nodes[i], "Learning Rate:", learning_rates[i], "Batch Size:", batchs[i]
    plt.title(title)
    plt.plot((test_output[:20,0] * std) + mean, label="Test Perdiction")
    plt.plot((np_test_targets[:20,0] * std) + mean, label="Test Actual")
    plt.xlabel("Record")
    plt.ylabel("Number of Riders")
    plt.legend(loc='best')
    plt.show()
```

# Conclusion

Now that you have seen how to go about writing and training a Multi-Layer Percptron try tuning the hyperparameters further to see how much you can improve the examples above.


## D3Factor
EldenRing!!!(bushi

```python
from Crypto.Util.number import bytes_to_long, getPrime
from secret import msg
from sympy import nextprime
from gmpy2 import invert
from hashlib import md5

flag = 'd3ctf{'+md5(msg).hexdigest()+'}'
p = getPrime(256)
q = getPrime(256)
assert p > q
n = p * q
e = 0x10001
m = bytes_to_long(msg)
c = pow(m, e, n)

N = pow(p, 7) * q
phi = pow(p, 6) * (p - 1) * (q - 1)
d1 = getPrime(2000)
d2 = nextprime(d1 + getPrime(1000))
e1 = invert(d1, phi)
e2 = invert(d2, phi)

print(f'c = {c}')
print(f'N = {N}')
print(f'e1 = {e1}')
print(f'e2 = {e2}')
'''
c = 2420624631315473673388732074340410215657378096737020976722603529598864338532404224879219059105950005655100728361198499550862405660043591919681568611707967
N = 1476751427633071977599571983301151063258376731102955975364111147037204614220376883752032253407881568290520059515340434632858734689439268479399482315506043425541162646523388437842149125178447800616137044219916586942207838674001004007237861470176454543718752182312318068466051713087927370670177514666860822341380494154077020472814706123209865769048722380888175401791873273850281384147394075054950169002165357490796510950852631287689747360436384163758289159710264469722036320819123313773301072777844457895388797742631541101152819089150281489897683508400098693808473542212963868834485233858128220055727804326451310080791
e1 = 425735006018518321920113858371691046233291394270779139216531379266829453665704656868245884309574741300746121946724344532456337490492263690989727904837374279175606623404025598533405400677329916633307585813849635071097268989906426771864410852556381279117588496262787146588414873723983855041415476840445850171457530977221981125006107741100779529209163446405585696682186452013669643507275620439492021019544922913941472624874102604249376990616323884331293660116156782891935217575308895791623826306100692059131945495084654854521834016181452508329430102813663713333608459898915361745215871305547069325129687311358338082029
e2 = 1004512650658647383814190582513307789549094672255033373245432814519573537648997991452158231923692387604945039180687417026069655569594454408690445879849410118502279459189421806132654131287284719070037134752526923855821229397612868419416851456578505341237256609343187666849045678291935806441844686439591365338539029504178066823886051731466788474438373839803448380498800384597878814991008672054436093542513518012957106825842251155935855375353004898840663429274565622024673235081082222394015174831078190299524112112571718817712276118850981261489528540025810396786605197437842655180663611669918785635193552649262904644919
'''

```

$e_1d_1=1(\bmod \varphi(n)), e_2d_2=1(\bmod \varphi(n))$

-> $e_1e_2(d_1-d_2)-(e_2-e_1)=0 (\bmod \varphi(n))$

得到模意义下的方程$e_1e_2x-(e_2-e_1)=0 (\bmod \varphi(n))$ 有一个解$d_1-d_2$，而$\varphi(n) = p^{r-1}(p-1)(q-1)$，因此$d_1-d_2$也是方程
$e_1e_2x-(e_2-e_1)=0 (\bmod p^{r-1})$

在本题$|d_1-d_2|$较小，所以可以对$e_1e_2x-(e_2-e_1)=0 (\bmod n)$使用Coppersmith方法求出$|d_1-d_2|$（？？？？）

再将解代入计算$\mathrm{gcd}(e_1e_2x_0-(e_2-e_1), n)$即可得到$p^{r-1}$
从而分解$n$

From r3kapig👇

```python
# s a g e
from Crypto.Util.number import *
from hashlib
import md5
import gmpy2
r = 7
N = 1476751427633071977599571983301151063258376731102955975364111147037204614220376883752032253407881568290520059515340434632858734689439268479399482315506043425541162646523388437842149125178447800616137044219916586942207838674001004007237861470176454543718752182312318068466051713087927370670177514666860822341380494154077020472814706123209865769048722380888175401791873273850281384147394075054950169002165357490796510950852631287689747360436384163758289159710264469722036320819123313773301072777844457895388797742631541101152819089150281489897683508400098693808473542212963868834485233858128220055727804326451310080791
e1 = 425735006018518321920113858371691046233291394270779139216531379266829453665704656868245884309574741300746121946724344532456337490492263690989727904837374279175606623404025598533405400677329916633307585813849635071097268989906426771864410852556381279117588496262787146588414873723983855041415476840445850171457530977221981125006107741100779529209163446405585696682186452013669643507275620439492021019544922913941472624874102604249376990616323884331293660116156782891935217575308895791623826306100692059131945495084654854521834016181452508329430102813663713333608459898915361745215871305547069325129687311358338082029
e2 = 1004512650658647383814190582513307789549094672255033373245432814519573537648997991452158231923692387604945039180687417026069655569594454408690445879849410118502279459189421806132654131287284719070037134752526923855821229397612868419416851456578505341237256609343187666849045678291935806441844686439591365338539029504178066823886051731466788474438373839803448380498800384597878814991008672054436093542513518012957106825842251155935855375353004898840663429274565622024673235081082222394015174831078190299524112112571718817712276118850981261489528540025810396786605197437842655180663611669918785635193552649262904644919
c = 2420624631315473673388732074340410215657378096737020976722603529598864338532404224879219059105950005655100728361198499550862405660043591919681568611707967

x = polygen(Zmod(N))
f = e1 * e2 * x - (e2 - e1)
f = f.monic()
roots = f.small_roots(beta=((r - 1) / (r + 1) - 0.01), epsilon=0.05)

if not roots:
    print("qwq")
    exit()

res = gmpy2.iroot(gmpy2.gcd(int(f(roots[0])), N), r - 1)
assert res[1]
print("p =", res[0])
p = res[0]

q = N // (p**7)
n = p * q
phi = (p - 1) * (q - 1)
d = inverse(0x10001, phi)
msg = long_to_bytes(power_mod(c, d, n))
print(msg)
flag = 'd3ctf{' + md5(msg).hexdigest() + '}'
print(flag)

```


# Quenching and drifting examples
In this notebook we apply the quenching and drifting stages to a track dataset

```python
# This is need so you can import larndsim without doing python setup.py install
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
```

```python
from larndsim import consts
consts.load_detector_properties("../larndsim/detector_properties/singlecube.yaml","../larndsim/pixel_layouts/layout-singlecube.yaml")

from larndsim import quenching, drifting
import importlib
importlib.reload(drifting)
importlib.reload(quenching)

from math import ceil

import matplotlib.pyplot as plt

import numpy as np
import h5py
```

```python
with h5py.File('edepsim_1M.h5', 'r') as f:
    tracks = np.array(f['segments'])

y_start = np.copy(tracks['y_start'] )
y_end = np.copy(tracks['y_end'])
y = np.copy(tracks['y'])

tracks['y_start'] = np.copy(tracks['z_start'])
tracks['y_end'] = np.copy(tracks['z_end'])
tracks['y'] = np.copy(tracks['z'])

tracks['z_start'] = y_start
tracks['z_end'] = y_end
tracks['z'] = y

tracks = tracks[:1000]
```

```python
tracks_box = np.copy(tracks)
tracks_birks = np.copy(tracks)
```

```python
threadsperblock = 256
blockspergrid = ceil(tracks.shape[0] / threadsperblock)

quenching.quench[blockspergrid,threadsperblock](tracks_box, consts.box)
quenching.quench[blockspergrid,threadsperblock](tracks_birks, consts.birks)
```

## Recombination

The number of ionized electrons must be corrected by the recombination effect. Two models have been implemented:

- Birks (Amoruso et al. NIM A 523 (2004) 275)
\begin{equation}
R_{Birks} = \frac{A_b}{1+k_b/\epsilon\cdot dE/dx }
\end{equation}

- Modified box (Baller, 2013 JINST 8 P08005)
\begin{equation}
R_{box} = \frac{\log(\alpha + \beta \frac{dE/dx}{E\rho})}{\beta/\epsilon\cdot dE/dx}
\end{equation}

where $\epsilon=E\rho_{\mathrm{LAr}}$.

```python
fig, ax = plt.subplots(1, 1)

recomb_birks = tracks_birks["n_electrons"] / (consts.MeVToElectrons * tracks_birks["dE"])
recomb_box = tracks_box["n_electrons"] / (consts.MeVToElectrons * tracks_box["dE"])

ax.plot(np.sort(tracks_box["dEdx"]), recomb_box[np.argsort(tracks_box["dEdx"])], label="Box model", lw=2)
ax.plot(np.sort(tracks_birks["dEdx"]), recomb_birks[np.argsort(tracks_birks["dEdx"])], label="Birks model", lw=2)
ax.set_xlabel("$dE/dx$ [MeV/cm]")
ax.set_ylabel("Recombination factor")
ax.set_xlim(0,10)
_ = ax.legend()
```

```python
tracks_quenched = np.copy(tracks)
quenching.quench[blockspergrid,threadsperblock](tracks_quenched, consts.box)
tracks_drifted = np.copy(tracks_quenched)
drifting.drift[blockspergrid,threadsperblock](tracks_drifted)
```

## Electron lifetime
The electrons traveling towards the anode are absorbed by impurities present in the liquid argon. The decrease follows an exponential curve $N_e^{\mathrm{anode}}/N_e=\exp(-t_{\mathrm{drift}}/e_{\mathrm{lifetime}})$.

```python
fig, ax = plt.subplots(1, 1)
drift_time = abs(tracks_quenched["z"] - consts.module_borders[0][2][0])/consts.vdrift
ax.plot(np.sort(tracks_quenched["z"]), np.exp(-drift_time / consts.lifetime)[np.argsort(tracks_quenched['z'])], lw=2)
ax.set_ylabel("$N_{e}^{\mathrm{anode}}/N_{e}$")
_ = ax.set_xlabel("$z$ coordinate [cm]")
```

## Diffusion
The diffusion coefficients (longitudinal and transverse) are proportional to $\sqrt{2t_{\mathrm{drift}}}$ (which in turn is given by $(z_{\mathrm{start}} - z_{\mathrm{anode}})/v_{\mathrm{drift}}$).

```python
fig, ax = plt.subplots(1, 1)
ax.plot(np.sort(tracks_quenched["z"]), tracks_drifted["long_diff"][np.argsort(tracks_quenched["z"])], label="Longitudinal diffusion", lw=2)
ax.plot(np.sort(tracks_quenched["z"]), tracks_drifted["tran_diff"][np.argsort(tracks_quenched["z"])], label="Transverse diffusion", lw=2)
ax.set_xlabel("$z$ start coordinate [cm]")
ax.set_ylabel("Diffusion coefficient")
_ = ax.legend()
```


# Bucle de control

## Objetivos
- Identificar los elementos típicos de los sistemas controlados.
- Identificar las tareas de cada elemento del bucle de control.

## Definición
Se conoce como bucle de control al conjunto de sistemas interactando entre sí para tener [control en lazo cerrado](https://www.electronics-tutorials.ws/systems/closed-loop-system.html). El objetivo de esta interacción es obtener comportamientos deseados como respuesta de una planta.

En este diagrama se observa un bucle (bucla o lazo) típico de control que sirve como orientación a la hora de automatizar procesos.

- La **planta** es el proceso que se debe controlar.
- El **actuador** cambia el comportamiento de la planta a partir de las ordenes del **controlador**.
- El **controlador** toma decisiones a partir del error del proceso para que el sistema controlado cumpla con el objetivo impuesto con la señal de **referencia**.
- El **sensor** se encarga de medir el comportamiento de la **planta** para dar esta información al **controlador**.

Con la tecnología del momento, los controladores son electrónicos. Por esta razón, el **actuador** recibe señales eléctricas y las lleva a la naturaleza propia de la planta, y el sensor transfiere la información del comportamiento de la planta a forma eléctrica.

Si se considera que:
- Los sistemas **actuador** y **planta** se ven como un solo sistema desde el punto de vista del controlador, podría adoptar el nombre de **proceso**.
- El sistema **sensor** brinda información sin error de forma muy rápida en comparación a la evolución del proceso.

## Reducción del bucle

El bucle de control puede reducirse a:

- $Y_{sp}$ es la señal de referencia (sp por SetPoint)
- $Y$ es la señal de respuesta del sistema controlado.
- $E = Y_{sp} - Y$ es la señal de error.
- $G_C$ es el **controlador**.
- $U$ es el decisión tomada por el **controlador** y la excitación del **proceso**.
- $G_P$ es el **proceso**.

Recuerde que las señales varían en el tiempo. Así, pueden definirse:

\begin{align}
E(t) &= Y_{sp}(t)-Y(t)\\
U(t) &=  \mathcal{G_C} \{E(t) \}  =  \mathcal{G_C} \{ Y_{sp}(t)-Y(t) \} \\
Y(t) &=   \mathcal{G_P} \{U(t) \} =   \mathcal{G_P} \{ \mathcal{G_C} \{ Y_{sp}(t)-Y(t) \} \}
\end{align}

## Bucle con sistemas LTI

Si los sistemas son **LTI**, se pueden denominar $g_{C}(t)$ a la respuesta impulsional del **controlador**, y $g_{P}(t)$ a la respuesta impulsional del **proceso**. Esto permite reescribir las expresiones anteriores como:

\begin{align}
E(t) &= Y_{sp}(t)-Y(t)\\
U(t) &=  g_C(t) * ( Y_{sp}(t)-Y(t) ) \\
Y(t) &=  g_P(t) * g_C(t) * ( Y_{sp}(t)-Y(t) )
\end{align}

Esto indica que la señal de respuesta depende de:
- el deseo $Y_{sp}$
- el proceso $G_{P}$
- el controlador $G_{C}$

Observe que para obtener un comportamiento deseado en $Y(t)=g_P(t)*g_C(t)*( Y_{sp}(t)-Y(t) )$ debe definirse $g_C(t)$ de manera que se corrijan los comportamientos del proceso. La labor de ingeniería de control es diseñar el controlador para cumplir especificaciones.

Para facilitar el análisis y trabajo de los sistemas controlados, se usará la **transformada de Laplace**.

## Juego de control

```python
import matplotlib.pyplot as plt
%matplotlib inline
from juego import ControlGame
game = ControlGame(runtime=45)  # segundos
```

Suponga que usted debe operar un sistema **SISO** (de una entrada y una salida) usando un botón deslizable y su percepción del funcionamiento del sistema.

- Ejecute la celda con `game.ui()`.
- Presione el botón `Ejecutar` y mueva el botón `U(t)` para que la señal `Salida` siga a la señal `Referencia`, que cambia de forma aleatoria después de cierta cantidad de segundos.
- Tenga en cuenta que el `Puntaje` crece más rápido mientras menor sea el error.
- Ejecute la celda varias veces para ver cómo usted aprende a controlar el sistema.
- Para visualizar su desempeño como controlador, ejecute la celda con `game.plot()`.

```python
game.ui()
```

    VBox(children=(HBox(children=(Button(description='Ejecutar', style=ButtonStyle()), Text(value='0', description…

```python
game.plot()
```

Los cambios que acaba de realizar manualmente deben ser ejercidos de manera automática por el **controlador**.

En el resto del curso se discutirá sobre las técnicas de análisis y diseño más usadas para sistemas análogos.

```python

```


This file is part of the pyMOR project (http://www.pymor.org).
Copyright 2013-2020 pyMOR developers and contributors. All rights reserved.
License: BSD 2-Clause License (http://opensource.org/licenses/BSD-2-Clause)
# Heat equation example

## Analytic problem formulation

We consider the heat equation on the segment $[0, 1]$, with dissipation on both sides, heating (input) $u$ on the left, and measurement (output) $\tilde{y}$ on the right:
$$
\begin{align*}
    \partial_t T(z, t) & = \partial_{zz} T(z, t), & 0 < z < 1,\ t > 0, \\
    \partial_z T(0, t) & = T(0, t) - u(t), & t > 0, \\
    \partial_z T(1, t) & = -T(1, t), & t > 0, \\
    \tilde{y}(t) & = T(1, t), & t > 0.
\end{align*}
$$

## Import modules

```python
import numpy as np
import scipy.linalg as spla
import scipy.integrate as spint
import matplotlib.pyplot as plt

from pymor.basic import *
from pymor.core.config import config
from pymor.reductors.h2 import OneSidedIRKAReductor

from pymor.core.logger import set_log_levels
set_log_levels({'pymor.algorithms.gram_schmidt.gram_schmidt': 'WARNING'})
```

## Assemble LTIModel

### Discretize problem

```python
p = InstationaryProblem(
    StationaryProblem(
        domain=LineDomain([0.,1.], left='robin', right='robin'),
        diffusion=ConstantFunction(1., 1),
        robin_data=(ConstantFunction(1., 1), ExpressionFunction('(x[...,0] < 1e-10) * 1.', 1)),
        outputs=(('l2_boundary', ExpressionFunction('(x[...,0] > (1 - 1e-10)) * 1.', 1)),)
    ),
    ConstantFunction(0., 1),
    T=3.
)

fom, _ = discretize_instationary_cg(p, diameter=1/100, nt=100)
print(fom)
```

### Visualize solution for constant input of 1

```python
fom.visualize(fom.solve())
```

### Convert to LTIModel

```python
lti = fom.to_lti()
print(lti)
```

## System analysis

```python
poles = lti.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('System poles')
plt.show()
```

```python
w = np.logspace(-2, 3, 100)
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the full model')
plt.show()
```

```python
hsv = lti.hsv()
fig, ax = plt.subplots()
ax.semilogy(range(1, len(hsv) + 1), hsv, '.-')
ax.set_title('Hankel singular values')
plt.show()
```

```python
print(f'FOM H_2-norm:    {lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'FOM H_inf-norm:  {lti.hinf_norm():e}')
print(f'FOM Hankel-norm: {lti.hankel_norm():e}')
```

## Balanced Truncation (BT)

```python
r = 5
bt_reductor = BTReductor(lti)
rom_bt = bt_reductor.reduce(r, tol=1e-5)
```

```python
err_bt = lti - rom_bt
print(f'BT relative H_2-error:    {err_bt.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'BT relative H_inf-error:  {err_bt.hinf_norm() / lti.hinf_norm():e}')
print(f'BT relative Hankel-error: {err_bt.hankel_norm() / lti.hankel_norm():e}')
```

```python
poles = rom_bt.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('Poles of the BT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_bt.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and BT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_bt.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the BT error system')
plt.show()
```

## LQG Balanced Truncation (LQGBT)

```python
r = 5
lqgbt_reductor = LQGBTReductor(lti)
rom_lqgbt = lqgbt_reductor.reduce(r, tol=1e-5)
```

```python
err_lqgbt = lti - rom_lqgbt
print(f'LQGBT relative H_2-error:    {err_lqgbt.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'LQGBT relative H_inf-error:  {err_lqgbt.hinf_norm() / lti.hinf_norm():e}')
print(f'LQGBT relative Hankel-error: {err_lqgbt.hankel_norm() / lti.hankel_norm():e}')
```

```python
poles = rom_lqgbt.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('Poles of the LQGBT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_lqgbt.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and LQGBT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_lqgbt.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the LGQBT error system')
plt.show()
```

## Bounded Real Balanced Truncation (BRBT)

```python
r = 5
brbt_reductor = BRBTReductor(lti, 0.34)
rom_brbt = brbt_reductor.reduce(r, tol=1e-5)
```

```python
err_brbt = lti - rom_brbt
print(f'BRBT relative H_2-error:    {err_brbt.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'BRBT relative H_inf-error:  {err_brbt.hinf_norm() / lti.hinf_norm():e}')
print(f'BRBT relative Hankel-error: {err_brbt.hankel_norm() / lti.hankel_norm():e}')
```

```python
poles = rom_brbt.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('Poles of the BRBT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_brbt.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and BRBT reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_brbt.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the BRBT error system')
plt.show()
```

## Iterative Rational Krylov Algorithm (IRKA)

```python
r = 5
irka_reductor = IRKAReductor(lti)
rom_irka = irka_reductor.reduce(r)
```

```python
fig, ax = plt.subplots()
ax.semilogy(irka_reductor.conv_crit, '.-')
ax.set_title('Distances between shifts in IRKA iterations')
plt.show()
```

```python
err_irka = lti - rom_irka
print(f'IRKA relative H_2-error:    {err_irka.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'IRKA relative H_inf-error:  {err_irka.hinf_norm() / lti.hinf_norm():e}')
print(f'IRKA relative Hankel-error: {err_irka.hankel_norm() / lti.hankel_norm():e}')
```

```python
poles = rom_irka.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('Poles of the IRKA reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_irka.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and IRKA reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_irka.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the IRKA error system')
plt.show()
```

## Two-Sided Iteration Algorithm (TSIA)

```python
r = 5
tsia_reductor = TSIAReductor(lti)
rom_tsia = tsia_reductor.reduce(r)
```

```python
fig, ax = plt.subplots()
ax.semilogy(tsia_reductor.conv_crit, '.-')
ax.set_title('Distances between shifts in TSIA iterations')
plt.show()
```

```python
err_tsia = lti - rom_tsia
print(f'TSIA relative H_2-error:    {err_tsia.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'TSIA relative H_inf-error:  {err_tsia.hinf_norm() / lti.hinf_norm():e}')
print(f'TSIA relative Hankel-error: {err_tsia.hankel_norm() / lti.hankel_norm():e}')
```

```python
poles = rom_tsia.poles()
fig, ax = plt.subplots()
ax.plot(poles.real, poles.imag, '.')
ax.set_title('Poles of the TSIA reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_tsia.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and TSIA reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_tsia.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the TSIA error system')
plt.show()
```

## One-Sided IRKA

```python
r = 5
one_sided_irka_reductor = OneSidedIRKAReductor(lti, 'V')
rom_one_sided_irka = one_sided_irka_reductor.reduce(r)
```

```python
fig, ax = plt.subplots()
ax.semilogy(one_sided_irka_reductor.conv_crit, '.-')
ax.set_title('Distances between shifts in one-sided IRKA iterations')
plt.show()
```

```python
fig, ax = plt.subplots()
osirka_poles = rom_one_sided_irka.poles()
ax.plot(osirka_poles.real, osirka_poles.imag, '.')
ax.set_title('Poles of the one-sided IRKA ROM')
plt.show()
```

```python
err_one_sided_irka = lti - rom_one_sided_irka
print(f'One-sided IRKA relative H_2-error:    {err_one_sided_irka.h2_norm() / lti.h2_norm():e}')
if config.HAVE_SLYCOT:
    print(f'One-sided IRKA relative H_inf-error:  {err_one_sided_irka.hinf_norm() / lti.hinf_norm():e}')
print(f'One-sided IRKA relative Hankel-error: {err_one_sided_irka.hankel_norm() / lti.hankel_norm():e}')
```

```python
fig, ax = plt.subplots()
lti.mag_plot(w, ax=ax)
rom_one_sided_irka.mag_plot(w, ax=ax, linestyle='dashed')
ax.set_title('Magnitude plot of the full and one-sided IRKA reduced model')
plt.show()
```

```python
fig, ax = plt.subplots()
err_one_sided_irka.mag_plot(w, ax=ax)
ax.set_title('Magnitude plot of the one-sided IRKA error system')
plt.show()
```

## Transfer Function IRKA (TF-IRKA)

Applying Laplace transformation to the original PDE formulation, we obtain a parametric boundary value problem
$$
\begin{align*}
    s \hat{T}(z, s) & = \partial_{zz} \hat{T}(z, s), \\
    \partial_z \hat{T}(0, s) & = \hat{T}(0, s) - \hat{u}(s), \\
    \partial_z \hat{T}(1, s) & = -\hat{T}(1, s), \\
    \hat{\tilde{y}}(s) & = \hat{T}(1, s),
\end{align*}
$$
where $\hat{T}$, $\hat{u}$, and $\hat{\tilde{y}}$ are respectively Laplace transforms of $T$, $u$, and $\tilde{y}$.
We assumed the initial condition to be zero ($T(z, 0) = 0$).
The parameter $s$ is any complex number in the region convergence of the Laplace tranformation.

Inserting $\hat{T}(z, s) = c_1 \exp\left(\sqrt{s} z\right) + c_2 \exp\left(-\sqrt{s} z\right)$, from the boundary conditions we get a system of equations
$$
\begin{align*}
    \left(\sqrt{s} - 1\right) c_1 - \left(\sqrt{s} + 1\right) c_2 + \hat{u}(s) & = 0, \\
    \left(\sqrt{s} + 1\right) \exp\left(\sqrt{s}\right) c_1 - \left(\sqrt{s} - 1\right) \exp\left(-\sqrt{s}\right) c_2 & = 0.
\end{align*}
$$
We can solve it using `sympy` and then find the transfer function ($\hat{\tilde{y}}(s) / \hat{u}(s)$).

```python
import sympy as sy
sy.init_printing()

sy_s, sy_u, sy_c1, sy_c2 = sy.symbols('s u c1 c2')

sol = sy.solve([(sy.sqrt(sy_s) - 1) * sy_c1 - (sy.sqrt(sy_s) + 1) * sy_c2 + sy_u,
                (sy.sqrt(sy_s) + 1) * sy.exp(sy.sqrt(sy_s)) * sy_c1 -
                (sy.sqrt(sy_s) - 1) * sy.exp(-sy.sqrt(sy_s)) * sy_c2],
               [sy_c1, sy_c2])

y = sol[sy_c1] * sy.exp(sy.sqrt(sy_s)) + sol[sy_c2] * sy.exp(-sy.sqrt(sy_s))

sy_tf = sy.simplify(y / sy_u)
sy_tf
```

Notice that for $s = 0$, the expression is of the form $0 / 0$.

```python
sy.limit(sy_tf, sy_s, 0)
```

```python
sy_dtf = sy_tf.diff(sy_s)
sy_dtf
```

```python
sy.limit(sy_dtf, sy_s, 0)
```

We can now form the transfer function system.

```python
def H(s):
    if s == 0:
        return np.array([[1 / 3]])
    else:
        return np.array([[complex(sy_tf.subs(sy_s, s))]])

def dH(s):
    if s == 0:
        return np.array([[-13 / 54]])
    else:
        return np.array([[complex(sy_dtf.subs(sy_s, s))]])

tf = TransferFunction(lti.input_space, lti.output_space, H, dH)
print(tf)
```

Here we compare it to the discretized system, by magnitude plot, $\mathcal{H}_2$-norm, and $\mathcal{H}_2$-distance.

```python
tf_lti_diff = tf - lti
fig, ax = plt.subplots()
tf_lti_diff.mag_plot(w, ax=ax)
ax.set_title('Distance between PDE and discretized transfer function')
plt.show()
```

```python
print(f'TF H_2-norm  = {tf.h2_norm():e}')
print(f'LTI H_2-norm = {lti.h2_norm():e}')
```

```python
print(f'TF-LTI relative H_2-distance = {tf_lti_diff.h2_norm() / tf.h2_norm():e}')
```

TF-IRKA finds a reduced model from the transfer function.

```python
tf_irka_reductor = TFIRKAReductor(tf)
rom_tf_irka = tf_irka_reductor.reduce(r)
```

```python
fig, ax = plt.subplots()
tfirka_poles = rom_tf_irka.poles()
ax.plot(tfirka_poles.real, tfirka_poles.imag, '.')
ax.set_title('Poles of the TF-IRKA ROM')
plt.show()
```

Here we compute the $\mathcal{H}_2$-distance from the original PDE model to the TF-IRKA's reduced model and to the IRKA's reduced model.

```python
err_tf_irka = tf - rom_tf_irka
print(f'TF-IRKA relative H_2-error = {err_tf_irka.h2_norm() / tf.h2_norm():e}')
```

```python
err_irka_tf = tf - rom_irka
print(f'IRKA relative H_2-error (from TF) = {err_irka_tf.h2_norm() / tf.h2_norm():e}')
```


# How does MICOM model communities

There are many frameworks for microbial modeling and all make their own specific assumptions in order to fit microbial communities into a mathematical representation. This is an overview about the assumptions `MICOM` makes and how it translates the community into mathematical terms. As it happens all formulations here are similar to formulation used in the [OptCom](https://doi.org/10.1371/journal.pcbi.1002363) and [SteadyCom](https://doi.org/10.1371/journal.pcbi.1005539) papers.

In general `MICOM` makes several assumptions to infer growth rates and fluxes in a microbial community. The most important ones being:

1. Growth rates and relative abundances in the community are in steady state.
2. Most taxa that are present in a biological sample should be able to grow (have a growth rate >0).
2. Relative DNA abundances can be used as a proxy for dry weight taxa abundances.

Different to SteadyCom, MICOM does not assume that all taxa have the same growth rate and allows for taxa-specific dilution rates.

## Exchanges and community growth rate

One of the things that can easily overlooked when using FBA for communities is that growth rates and fluxes are usually given in unit mass/(abundance * time), for instance mmmol/(gDW * h). Thus, all fluxes are relative to the abundance of the single bacteria they describe. However, in a community all bacteria might have different abundances so we have to take care to balance fluxes. For instance let us take a system containing only one bacteria $i$ which imports metabolite X from the external medium. In the medium metabolite X enters the system with an unscaled flux $v^m_x$ which has units mmmol/h. Within the bacteria the metabolite is consumed with the sclaed flux $v^i_x$ which has units mmol/(gDW * h). $v^i_x$ described the flux that can be realized by 1 gDW of the respective bacteria. However, the abundance for bacteria $i$, $b_i$ (in gDW), might be different than that. In order to have balanced fluxes we have to enforce that $v^m_x = b_i \cdot v^i_x$ (overall influx equals overall consumption). If many bacteria import the respective metabolite we thus need to enforce

$$
v^m_x = \sum_i b_i \cdot v^i_x
$$

The actual abundances $b_i$ are usually not known, but we can divide the equation by the total bacterial abundance $B = \sum_i b_i$ and obtain

$$
\tilde{v}^m_x = v^m_x/B = \sum_i b_i/B \cdot v^i_x
$$

$\tilde{v}^m_x$ is now a scaled flux in the medium relative to the overall bacterial biomass in the community. The relative abundances $\tilde{b}_i = b_i/B$ can be taken from metagenomic studies such as 16S rRNA quantities. This is how `MICOM` uses abundance data.

In a similar manner the unscaled community growth rate (total biomass production) is given by

$$
v_{biomass} = \sum_i b_i\cdot v^i_{biomass}
$$

and dividing by the total biomass $B$ yields

$$
\tilde{v}_{biomass} = \sum_i b_i/B v^i_{biomass}
$$

where $\tilde{v}_{biomass}$ now again is a scaled biomass flux relative to a total community biomass of 1 gDW. In order to adjust growth rate predictions from `MICOM` to your sample you would have to divide those growth rates
by the overall weight of microbiota in your sample. For instance [a recent publication](https://dx.doi.org/10.1371%2Fjournal.pbio.1002533) estimates that an average human has 200g of bacteria in the gut.

## Steady states in the community

`MICOM` has be designed with the gut microbiota in mind. One of the major problems when trying to apply flux balance analysis (FBA) to microbial community data especially metagenomic data are paradoxical assumptions about community growth. Flux balance analysis usually assumes a maximization of the growth rate, or at least the realization of one particular growth rate $\mu$, however metagenomic experiments usually only quantify the microbial compositions at one particular time point, assuming that the microbial abundance does not change. Those two assumptions are not compatible per sé. If members of the community grow with a constant rate they will accumulate exponentially over time, however we know that this is not the case in systems as the intestine (otherwise we would probably explode due to an overpopulation of bacteria and fungi). This disagreement can be aleviated by accounting for dilution of the microbiota. For instance in the gut bacteria are constantly removed in small amounts by death (when arriving at their specific life span) and in larger amounts by defecation. The sum of all processes removing bacteria from the system is what we call dilution here. In `MICOM` we assume the follwing about the dilution process:

1. It is relative to the bacterial abundance (the more you have in your system the more is removed by dilution)
2. It may be specific to the bacterial strain
3. It may specific to the sample

(1) is known to be true for the gut microbiome since we know that a higher concentration of a bacteria in the gut is usually associated with a higher concentration in stool samples (which is the major dilution contributor). This is similar to [SteadyCom](https://doi.org/10.1371/journal.pcbi.1005539.s016) but does not assume that the dilution rate is constant across all taxa. (2) is based on the observation that bacteria may have distinct spatial arrangements which make it easier or hader to be diluted. (3) is based on the assumption that the respective systems may be different (no gut is the same :D), however it is one of the assumptions we are currently trying to validate. As a consequence we assume that the abundance for bacteria $i$, $b_i$ occurs with a growth rate $\mu_i$ and balances with an arbitrary dilution process $d_i(t) = d_i(b_i)$ as

$$
\frac{d b_i(t)}{dt} = \mu_i b_i(t) - d_i(t)
$$

As we can see absolute bacterial abundance can only be in steady state if growth and dilution are balanced. Additionally one could also formulate the problem in terms of relative concentrations to the total community abundance $B = \sum_i b_i$ using the quotient rule, which yields:

$$
\frac{d \tilde{b}_i}{dt} = \frac{d b_i/B}{dt} = \mu_i\tilde{b}_i - \frac{d_i(t)}{B} - \cdot \tilde{b}_i\sum_k \left[\mu_k\tilde{b}_k - \frac{d_k(t)}{B}\right]
$$

As one can see, this equation has a steady state if for all i $\mu_i - \frac{d_i(t)}{B\cdot \tilde{b}_i} = C$ for any constant C:

\begin{align}
\frac{d \tilde{b}_i}{dt} &= C\tilde{b}_i - \tilde{b}_i\sum_k C \tilde{b}_k\\
& = \tilde{b}_i\left(C - C\cdot 1) \right) = 0
\end{align}

However, in this the total abundance $B$ would increase indefinitely in time. In particular it holds that $\frac{dB}{dt} = C$.

Okay, what can we conclude from that? Most importantly that any abundance $b_i$ or relative abundance $\tilde{b}_i$ can be a valid steady state abundance as long as the respective growth and dilution rates are balanced. This means there is an infinite amount of growth rates that yield a steady state (growth rates can not be uniquely identified from abundance alone).

## Cooperative tradeoff

Instead of solving a multi-objective problem MICOM achieves a global optimum by a two step approach called `cooperative tradeoff`. In simple terms:

1. obtain the maximum community growth rate $\mu_c^*$ by classic FBA
2. choose a tradeoff term $\alpha$ in $[0, 1]$ and constrain the community growth rate such that $\mu_c \geq \alpha\cdot\mu_c^*$
3. minimize the regularization term $\sum_i \mu_i^2$ which distributes growth across all individuals in the community

This method is much faster than alternative methods. Also it results in solution where each individual growth is simultaneously at its maximum rate without diminishing the growth rate of other individuals (thus the term "cooperative"). In consequence, cooperative
tradeoff results in an optimum which is conceptually very similar to what OptCom does but with a simpler method. In the absence of constraints MICOM yields growth rate estimates that are correlated with abundance *within a single sample*. This is not an assumption made *a priori* but a consequence of the optimization strategy.

## Numerical stability and solvers

MICOM support all solver implemented in optlang (GLPK, CPLEX and Gurobi). However, since MICOM models can become quite large (50k - 500k variables) many solvers have numerical issues when solving the corresponding LP and QP problems. For realistic communities modeled on the genus, species or strain level we thus recommend using a solver that implements numerically stable interior point methods, currently only Cplex and Gurobi. Both have academic licenses available. In our experience the simpex methods in GLPK still work well up to 10,000 variables, but get gradually slower towards 50,000 variables and being too slow to be usable beyond that.

Nevertheless, even the commercial solvers can usaully not control numerical issues in the quadratic interior point methods well. In fact for larger models they always never return an optimal solution. To avoid those numerical issues MICOM employs a crossover strategy where it will perform as many iterations in the quadratic interior point method as possible to get a candidate solution for the individual growth rates $\mu_i^{ca}$, followed by solving a linear programming problem to find a feasible solution close to the candidates:

$$
\begin{align}
\text{maximize } & \mu_c = \sum_i b_i/B \mu^i\\
& s.t. \mu_i \leq \mu_i^{ca}
\end{align}
$$

```python

```