```python
# This cell is for the Google Colaboratory
# https://stackoverflow.com/a/63519730
if 'google.colab' in str(get_ipython()):
  # https://colab.research.google.com/notebooks/io.ipynb
  import google.colab.drive as gcdrive
  # may need to visit a link for the Google Colab authorization code
  gcdrive.mount("/content/drive/")
  import sys
  sys.path.insert(0,"/content/drive/My Drive/Colab Notebooks/nmisp/60_linear_algebra_2")

```

```python
import numpy as np
import numpy.linalg as nl
import numpy.random as nr
import sympy as sym
import IPython.display as disp

sym.init_printing()

```

```python
a, b, c, d = sym.symbols('a b c d')

```

# 역행렬
Inverse matrix

## 2x2

다음과 같은 2x2 행렬의 역행렬을 구하는 법은 이미 알고 있을 것이다.

We know how we can find a 2x2 matrix as follows.

$$
A = \begin{bmatrix}
    a & b\\
    c & d \\
\end{bmatrix}
$$

```python
A = sym.Matrix([
    [a, b],
    [c, d]]
)
A

```

$$
A^{-1} = \frac {1}{ad-bc}\begin{bmatrix}
    d & -b\\
   -c & a \\
\end{bmatrix}
$$

여기서 $ad-bc$ 는 해당 행렬의 *행렬식* 이다.

Here $ad-bc$ is the *determinant* of the matrix.

```python
def det22(matA):
    assert (2, 2) == matA.shape
    return matA[0, 0] * matA[1, 1] - matA[0, 1] * matA[1, 0]

```

그런데 행렬 부분은 어떠한가?

What about the matrix part?

```python
adjA = sym.Matrix([
    [d, -b],
    [-c, a]]
)
adjA

```

원래의 행렬과 곱해보자.

Let's multiply with the original matrix.

```python
A * adjA

```

결과는 어떠한 형태인가?

What does the result look like?

임의의 크기의 행렬에 대해 그러한 행렬을 찾을 수 있을까?

Can we find such matrix for a matrix of an arbitrary size?

## 행렬식
Determinant

### 3x3

다음과 같은 3x3 행렬을 생각해 보자.

Let's think about a 3x3 matrix as follows.

```python
e, f, g, h, i = sym.symbols(list('efghi'))

```

```python
A = sym.Matrix([
    [a, b, c],
    [d, e, f,],
    [g, h, i,],
])
A

```

해당 행렬의 행렬식을 구해보자.

Let's find the determinant of the matrix.

#### 소행렬식
Minor

$i$ $j$ 번째 행을 삭제한 행렬을 생각해 보자.

Let's consider a matrix without $i$-th row and $j$-th column.

```python
def row_col_del(matA, row_i, col_j):
    result = sym.Matrix(matA)
    result.row_del(row_i)
    result.col_del(col_j)
    return result

```

```python
A00 = row_col_del(A, 0, 0)
A00

```

```python
A01 = row_col_del(A, 0, 1)
A01

```

```python
A02 = row_col_del(A, 0, 2)
A02

```

행렬 $A_{ij}$ 는 $A$ 행렬에서 $i$행 $j$열을 제외한 $\left(n-1\right) \times \left(n-1\right)$ 행렬이다 .

Matrix $A_{ij}$ is a $\left(n-1\right) \times \left(n-1\right)$ matrix excluding $i$-th row and $j$-th column of the matrix $A$.

행렬식은 다음과 같이 계산할 수 있다.  이를 *소행렬식* 이라 한다.

We can calculate the determinant as follows.  This is called the *Minor*.

```python
det1 = A[0, 0] * det22(A00) - A[0, 1] * det22(A01) + A[0, 2] * det22(A02)
det1

```

함수 형태로는 다음과 같다.

We may write in a function as follows.

```python
def det33(matA:np.ndarray) -> float:

    result = 0

    for j in range(matA.shape[1]):
        result += ((-1) ** j) * matA[0, j] * det22(row_col_del(matA, 0, j))

    return result

```

리스트 줄여쓰기로 다음과 같이 구현할 수도 있다.

We can implement as follows using the list comprehension.

```python
def det33_list_comprehension(matA:np.ndarray) -> float:
    return sum(
        [((-1) ** j) * matA[0, j] * det22(row_col_del(matA, 0, j)) for j in range(matA.shape[1])]
    )

```

아래 셀은 함수를 확인한다.

Following cell checks the function.

```python
assert 0 == sym.simplify(det33(A) - det1)
assert 0 == sym.simplify(det33_list_comprehension(A) - det1)

```

### n x n, n > 3

일반화 하면 다음과 같다.

We can generalize as follows.

```python
def det(matA:np.ndarray) -> float:
    if 2 == matA.shape[0]:
        result = det22(matA)
    elif 2 < matA.shape[0]:
        result = sum(
            [((-1) ** j) * matA[0, j] * det(row_col_del(matA, 0, j)) for j in range(matA.shape[1])]
        )
    return result

```

## 여인수행렬
Cofactor matrix

행렬 $A_{ij}$ 의 행렬식, *소행렬식* 에 번갈아 부호를 붙인 것을 *여인수* $C_{ij}$ 라고 한다.

The signed version of the *minor*, determinant of the matrix $A_{ij}$, is the *cofactor* $C_{ij}$.

```python
def cofactor(A, i, j):
    return det(row_col_del(A, i, j))

```

여인수로 이루어진 행렬을 생각해 보자.

Let's think about the matrix of cofactors.

```python
def cofactor_matrix(matA):
       return sym.Matrix([[cofactor(matA, i, j) * ((-1) ** (i+j)) for j in range(matA.shape[1])] for i in range(matA.shape[0])])

```

```python
cofactor_A = cofactor_matrix(A)
cofactor_A

```

*여인수행렬* 의 전치행렬을 *수반행렬* 이라고 한다.

The transpose of a *cofactor matrix* is the *adjugate matrix*.

원래 행렬 $A$와 *수반행렬* 을 곱해 보자.

Let's multiply the origitnal matrix $A$ and its *adjugate matrix*.

```python
sym.simplify(A * cofactor_A.T)

```

위 행렬을 $A$의 행렬식으로 나누어 보자.

Let's divide the matrix above with the determinant of $A$.

```python
sym.simplify(A * cofactor_A.T / det1)

```

위 결과는 어떠한가?

How is the result above?

## 2x2 : Gauss Jordan Method

다음 비디오는 역행열 찾는 가우스 조단법을 소개한다.

Following video introduces Gauss Jordan method finding the inverse matrix. (36:23 ~ 42:20)

[](https://www.youtube.com/watch?v=FX4C-JpTFgY&list=PL221E2BBF13BECF6C&index=9&start=2183&end=2540)

아래 2x2 행렬을 생각해 보자.

Let's think about the 2x2 matrix.

```python
A22 = np.array([
    [1, 3],
    [2, 7]
])

```

```python
A22

```

오른쪽에 같은 크기의 단위행렬을 붙여 보자.

Let's augment an identity matrix of the same size.

```python
I22 = np.identity(2)

```

```python
I22

```

```python
AX22 = np.hstack([A22, I22])

```

```python
AX22

```

이제 왼쪽 2x2 부분을 단위행렬로 만들어 보자.

Let's make the left 2x2 part an identity matrix.

첫 행에 2를 곱한 후 2행에서 빼 보자.

Let's multipy 2 to the first row and then subtract from the second row.

```python
AX22[1, :] -= 2 * AX22[0, :]

```

```python
AX22

```

이번에는 2번째 행에 3을 곱해서 첫 행에서 빼 보자.

Now let's multipy 3 to the second row and subtract from the first row.

```python
AX22[0, :] -= 3 * AX22[1, :]

```

```python
AX22

```

위 `AX22` 행렬에서 오른쪽 두 행을 따로 떼어 보자.

Let's separate the right two columns of the `AX22` matrix above.

```python
A22_inv = AX22[:, 2:]

```

```python
A22_inv

```

`A22` 행렬과 곱해보자.

Let's multipy with the A matrix.

```python
A22 @ A22_inv

```

## `numpy`

`numpy.matrix` 의 `.I` 속성을 이용할 수도 있다.

We can use the `.I` property of `numpy.matrix`.

```python
mat_A22_inv = np.matrix(A22).I

```

```python
mat_A22_inv

```

```python
A22 @ mat_A22_inv

```

또한 `numpy.linalg.inv()` 함수도 있다.

Also, `numpy.linalg.inv()` function is available.

```python
A22_inv = nl.inv(A22)

```

```python
A22_inv

```

```python
A22_inv @ A22

```

## 3x3

다음 비디오는 3x3 행렬에 가우스 조단법을 적용한다.

Following video applies the Gauss Jordan method to invert a 3x3 matrix.

[](https://www.youtube.com/watch?v=obts_JDS6_Q)

아래 행렬을 생각해 보자.

Let's think about the following matrix.

```python
A33_list = [
    [1, 0, 1],
    [0, 2, 1],
    [1, 1, 1],
]

```

```python
A33_list

```

```python
A33 = np.array(A33_list)

```

```python
A33

```

마찬가지로, 오른쪽에 같은 크기의 단위행렬을 붙여 보자.

Same as before, let's augment an identity matrix of the same size.

```python
I33 = np.identity(A33.shape[0])

```

```python
I33

```

```python
AX33 = np.hstack([A33, I33])

```

```python
AX33

```

이제 왼쪽 부분을 단위행렬로 만들어 보자.

Let's make the left part an identity matrix.

첫 행을 3행에서 빼 보자.

Let's subtract the first row from the third row.

```python
AX33[2, :] -= AX33[0, :]

```

```python
AX33

```

이번에는 2번째 행과 3번째 행을 바꾸자.

Now let's swap the second and the third rows. ([ref](https://stackoverflow.com/a/54069951))

```python
AX33[[1, 2]] = AX33[[2, 1]]

```

```python
AX33

```

두번째 행에 2를 곱해서 3행에서 빼 보자.

Let's multiply 2 to the second row and subtract from the third row.

```python
AX33[2, :] -= 2 * AX33[1, :]

```

```python
AX33

```

첫번째 행에서 3번째 행을 빼 보자.

Let's subtract the third row from the first row.

```python
AX33[0, :] -= AX33[2, :]

```

```python
AX33

```

위 `AX` 행렬에서 오른쪽 세 열을 따로 떼어 보자.

Let's separate the right three columns of the `AX` matrix above.

```python
A33_inv = AX33[:, 3:]

```

```python
A33_inv

```

`A33` 행렬과 곱해보자.

Let's multipy with the `A33` matrix.

```python
A33 @ A33_inv

```

## `numpy`

`numpy.maxti` 의 `.I` 속성을 이용할 수도 있다.

We can use the `.I` property of `numpy.matrix`.

```python
mat_A33_inv = np.matrix(A33).I

```

```python
mat_A33_inv

```

```python
A33 @ mat_A33_inv

```

## 표준 기능으로 구현한 가우스 조단법
Gauss Jordan method in Standard library

다음 셀은 가우스 조단법을 표준기능 만으로 구현한다.

Following cell implements the Gauss Jordan method with standard library only.

```python
import typing

Scalar = typing.Union[int, float]
Row = typing.Union[typing.List[Scalar], typing.Tuple[Scalar]]
Matrix = typing.Union[typing.List[Row], typing.Tuple[Row]]

def get_zero(n:int) -> Matrix:
    return [
        [0] * n for i in range(n)
    ]

def get_identity(n:int) -> Matrix:
    result = get_zero(n)
    for i in range(n):
        result[i][i] = 1

    return result

def augment_mats(A:Matrix, B:Matrix):
    assert len(A) == len(B)
    return [row_A + row_B for row_A, row_B in zip(A, B)]

def gauss_jordan(A:Matrix) -> Matrix:
    AX = augment_mats(A, get_identity(len(A)))

    # pivot loop
    for p in range(len(AX)):

        one_over_pivot = 1.0 / AX[p][p]

        # normalize a row with one_over_pivot
        for j in range(len(AX[p])):
            AX[p][j] *= one_over_pivot

        # row loop
        for i in range(len(AX)):
            if i != p:
                # row operation
                multiplier = - AX[i][p]

                # column loop
                for j in range(0, len(AX[p])):
                    AX[i][j] += multiplier * AX[p][j]

    return [row[len(A):] for row in AX]

```

위 행렬의 예로 확인해 보자.

Let's check with the matrix above.

```python
mat_A33_inv_GJ = gauss_jordan(A33_list)

```

```python
import pprint
pprint.pprint(mat_A33_inv_GJ, width=40)

```

```python
np.array(mat_A33_inv_GJ) @ A33

```

## 4x4

아래 행렬을 생각해 보자.

Let's think about the following matrix.

```python
A44_list = [
    [1, 0, 2, 0],
    [1, 1, 0, 0],
    [1, 2, 0, 1],
    [1, 1, 1, 1],
]

```

```python
A44 = np.array(A44_list)

```

```python
A44

```

다음 셀은 넘파이 다차원 배열 `numpy.ndarray` 을 위해 구현한 가우스 조르단 소거법 함수를 불러들인다.

Following cell imports an implementation of the Gauss Jordan Elimination for a `numpy.ndarray`.

```python
import gauss_jordan as gj

```

위 행렬에 적용해 보자.

Let's apply to the matrix above.

```python
A44_inv_array = gj.inv(A44)

```

```python
A44_inv_array

```

```python
A44_inv_array @ A44

```

```python
import numpy.testing as nt

nt.assert_array_almost_equal(A44_inv_array @ A44, np.array(get_identity(len(A44_list))))

```

### n x n

좀 더 큰 행렬의 경우, 여인수행렬과 비교해 보자.

Let's compare with the cofactor matrix.

```python
n = 7
Ann = nr.random((n, n))

```

```python
%time
inv1 = gj.inv(Ann, b_verbose=False)

```

```python
%time
inv2 = (1/det(Ann)) * cofactor_matrix(Ann).T

```

```python
import numpy.testing as nt

nt.assert_array_almost_equal(inv1, inv2)

```

## 연습 문제
Exercise

위 가우스 조단법에서 메모리를 더 절약하는 방안을 제안해 보시오

Regarding the Gauss Jordan implementation above, propose how we can save more memory.

## 소거행렬
Elimination Matrix

### 2x2

다시 한번 단위 행렬을 덧붙인 행렬을 생각해 보자.

Let's think about the augmented matrix again.

```python
AX22 = np.hstack([A22, I22])
AX22

```

(1, 0) 위치의 요소를 0으로 만드는 소거 행렬을 생각해 보자.

Let's think about the elimination matrix making element at (1, 0) location.

```python
E10 = np.identity(A22.shape[0])
E10[1, 0] = -AX22[1, 0] / AX22[0, 0]
E10

```

```python
AX22 = E10 @ AX22
AX22

```

이번에는, (0, 1) 위치의 요소를 0으로 만드는 소거 행렬을 생각해 보자.

Let's think about the elimination matrix making element at (0, 1) location.

```python
E01 = np.identity(A22.shape[0])
E01[0, 1] = -AX22[0, 1] / AX22[1, 1]
E01

```

```python
AX22 = E01 @ AX22
AX22

```

지금까지의 소거행렬을 순서 대로 곱해보자.

Let's multipy the elimination matrices in the order.

```python
E01 @ E10

```

위에서 구했던 역행렬도 다음과 같다.

The inverse matrix we found above were as follows, too.

```python
A22_inv

```

## 참고문헌
References

* Gilbert Strang. 18.06 Linear Algebra. Spring 2010. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA.
* Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong, Mathematics For Machine Learning, Cambridge University Press, 2020, ISBN 978-1108455145.
* Erwin Kreyszig, Advanced Engineering Mathematics, 5th Ed, John Wiley & Sons, 1983, ISBN 0-471-86251-7.

## Final Bell
마지막 종

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```


## 学习神经网络

* 查阅scikit-learn工具包中前馈神经网络学习器的相关说明，了解相关函数使用方法。
* 基于NumPy设计一个包含输入层、隐含层和多分类输出层的前馈神经网络，并以此为基础对手写数字图像进行分类实验。（交电子版实验报告）

有余力实现输入输出规模适应

# sklearn前馈神经网络

## 加载数据

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
pd.options.display.latex.repr = False
from sklearn.datasets import load_digits
digits = load_digits()
print(digits.data.shape,digits.target.shape)
```

    (1797, 64) (1797,)

## 训练集、验证集、测试集分割

使用`train_test_split`对训练集、测试集进行分割，使20%数据进入测试集。用Y值设定`stratify`参数，保证训练集、测试集类别分布一致。

```python
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size = 0.2, random_state = 512, stratify = digits.target)
```

使用K折交叉验证分割训练集和验证集。

```python
from sklearn.model_selection import StratifiedKFold
N=5
skf = StratifiedKFold(n_splits=N,random_state=256,shuffle=True)
```

## 模型加载

`sklearn`的`neural_network`包有多层感知机分类模型`MLPClassifier()`，搭建baseline：

```python
from sklearn.neural_network import MLPClassifier

clf = MLPClassifier()
clf.fit(X_train,y_train)
clf.score(X_test,y_test)
```

    0.9722222222222222

## 模型优化

由于选择的是小手写数据集，每条记录仅由64维0-1数据构成，默认参数下模型就能有较好表现。查阅`sklearn`参数后，发现多层感知机有多个参数可以调节：
```Python
class sklearn.neural_network.MLPClassifier(hidden_layer_sizes=100, activation='relu', *, solver='adam', alpha=0.0001, batch_size='auto', learning_rate='constant', learning_rate_init=0.001, power_t=0.5, max_iter=200, shuffle=True, random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08, n_iter_no_change=10, max_fun=15000)
```

* `hidden_layer_sizes`: 隐层规模，可以传入多维数组，第i维数值代表第i层隐层的神经元数量。这个参数可以直观调整网络结构。
* `activation`：激活函数，可取`identity`（$f(x)=x$）、`logistic`（$f(x)=\dfrac{1}{1+e^{-x}}$）、`tanh`（$f(x)=\tanh{(x)}$）、`relu`（$f(x)=\max(0,x)$）
* `solver`：权值优化器，默认`adam`在大数据集上表现很好，小数据集其实可以改为`lbfgs`提升速度。
* `learning_rate_init`：初始学习率，默认为0.001
* `learning_rate`：学习率更新方式：`constant`不变，`invscaling`使学习率按轮数t衰减，由参数`power_t`控制： $lr_{t}= \dfrac{lr_{\text{init}}}{t^\text{power_t}}$，`adaptive`若连续两轮损失函数减少值/验证集准确率增加值少于`tol`则当前准确率除以5，否则保持不变。

…………

因为数据集规模小，后续`solver`都采用`lbfgs`，调节其他参数。先尝试调节学习率：

```python
ts = 0; vs = 0;
for train_idx, val_idx in skf.split(X_train,y_train):
    clf = MLPClassifier(solver='lbfgs',learning_rate_init=0.01,learning_rate='adaptive')
    clf.fit(X_train[train_idx], y_train[train_idx])
    ts += clf.score(X_train[train_idx],y_train[train_idx])
    vs += clf.score(X_train[val_idx],y_train[val_idx])
print('Training Score: %.3f' % float(ts/N))
print('Validation Score: %.3f' % float(vs/N))
```

    Training Score: 1.000
    Validation Score: 0.974

设定初始参数为不同数量级（$10^{-1}$ ~ $10^{-7}$）,最终验证集准确率均在0.96~0.97水平之间。可能这是因为数据复杂度不高，学习率的影响不大。

尝试调节网络结构：

```python
ts = 0; vs = 0;
for train_idx, val_idx in skf.split(X_train,y_train):
    clf = MLPClassifier(solver='lbfgs',hidden_layer_sizes=(16,16))
    clf.fit(X_train[train_idx], y_train[train_idx])
    ts += clf.score(X_train[train_idx],y_train[train_idx])
    vs += clf.score(X_train[val_idx],y_train[val_idx])
print('Training Score: %.3f' % float(ts/N))
print('Validation Score: %.3f' % float(vs/N))
```

    Training Score: 1.000
    Validation Score: 0.945

当调整隐层神经元数量为较小值时，准确率会出现明显的下降，甚至有的迭代过程中会导致函数出现无法收敛的报错。

当调整隐层神经元数量小于分类数时，这时的模型已经非常不合理，通过少于分类数的运算，输出层无法得出所有可能输出，准确率出现明显的下降。

```python
ts = 0; vs = 0;
for train_idx, val_idx in skf.split(X_train,y_train):
    clf = MLPClassifier(solver='lbfgs',hidden_layer_sizes=(200,100))
    clf.fit(X_train[train_idx], y_train[train_idx])
    ts += clf.score(X_train[train_idx],y_train[train_idx])
    vs += clf.score(X_train[val_idx],y_train[val_idx])
print('Training Score: %.3f' % float(ts/N))
print('Validation Score: %.3f' % float(vs/N))
```

    Training Score: 1.000
    Validation Score: 0.973

当调整隐层神经元数量至较大值或者随意地增加隐层数目，会增加模型复杂度，导致运算时间增大，而准确率提升不明显。

所以，隐层参数应根据问题进行合理调整，并且一般后面的隐层神经元数量要小于等于上一隐层。

# 神经网络实现-基于Numpy

## 神经元结构确定

### 基本结构：权重、偏置、输入输出
首先，观察一个神经元的结构：

在一个前馈神经网络中，一个神经元需要接受上一层的所有神经元的输出$\boldsymbol{x}^{(l-1)}=(x_1^{(l-1)},x_2^{(l-1)},\cdots,x_n^{(l-1)})$，乘上权重$\boldsymbol{w}^{(l)}=(w_1^{(l-1)},w_2^{(l-1)},\cdots,w_n^{(l-1)})$加上偏置$b^{(l)}_i$作为输入$\alpha_i^{(l)}=\boldsymbol{w}_i^{(l)}\cdot \boldsymbol{x}_i^{(l-1)}+b^{(l)}_i$，而本层将输入用激活函数$\sigma(\alpha_i^{(l)})$进行计算，得到输出$x_i^{(l)}=\sigma(\boldsymbol{\alpha})$。

所以，对于一层的神经元而言，需要的基础变量是上一层的输入对应的权重$\boldsymbol{w}^{(l)}$、偏置$\boldsymbol{b}^{(l)}$，以及指定的激活函数。在训练过程中，需要对$\boldsymbol{w}^{(l)}$、$\boldsymbol{b}^{(l)}$进行更新:

$$
 w \leftarrow w-\eta \dfrac{\partial\mathcal{L}}{\partial w} \\
 b \leftarrow b-\eta \dfrac{\partial\mathcal{L}}{\partial b}
$$

其中$\mathcal{L}$是损失函数，$\eta$为学习率。

```python
class Layer:
    def __init__(self, size, prevsize):
        self.w = np.random.randn(prevsize,size)
        self.b = np.random.randn(size,1)
        self.alpha = np.zeros((size,1))
        self.x = np.zeros((size,1))
```

### 损失函数

由于参数的更新涉及到求导，损失函数可以选用易于求导的函数，对训练例$(\boldsymbol{x}_k,\boldsymbol{y}_k)$，设输出层输出为$\hat{y}_j^k(j=1,2,\cdots,l)$，目标输出为$\boldsymbol{y}_k=(y_1^k,y_2^k,\cdots,y_l^k)$，损失函数可采用交叉熵损失函数

$$
\mathcal{L}(\boldsymbol{y}_k,\boldsymbol{\hat{y}}_k)=\sum\limits_{j=1}^l y_j^k\log\hat{y}_j^k
$$

其偏微分是易求的，例如对输出层第m个输出$\hat{y}_m^k$的偏微分$\dfrac{\partial\mathcal{L}}{\partial \hat{y}_m^k}=\dfrac{y_m^k}{\hat{y}_m^k}$

```python
def crossEntropy(y,y_pred):
    return -np.sum(y.T*np.log(y_pred+1e-16)) #y_pred加上一个"无穷小量"，防止出现对数为0.
```

### 激活函数

由于参数的更新涉及到求导，为简化这一过程的实现，隐藏层的激活函数可以采用Logistic函数$\sigma(x) = \dfrac{1}{1+e^{-x}}$，因为这一函数的导数具有容易计算的优秀性质：$\sigma'(x)=\sigma(x)(1-\sigma(x))$。也可以采用Relu函数$\text{Relu}(x)=\max(0,x)$，导数为$\text{Relu}'(x)=\left\{ \begin{align}1,\ &x\ge 0\\ 0, \ &x\lt 0\end{align}\right.$

而输出层的激活函数可以根据任务而定，本任务为手写数字分类，为多分类任务，可以使用Softmax函数。

Softmax函数的求导较为复杂，通过阅读[@永远在你身后 的博文](https://zhuanlan.zhihu.com/p/67759205)，发现在本次应用中可以将其简化，这篇文章的推导最终指出，使用交叉熵作为多分类计算的损失函数时，
$$
\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(\text{output})}}=\text{Softmax}\left(\alpha_i^{(\text{output})}\right)-y_i
$$

其中，$y_i$为0或1，为1时为正确样本分类。

即不体现计算$\dfrac{\partial \mathcal{L}}{\partial x_i^{(\text{output})}}\dfrac{\partial x_i^{(\text{output})}}{\partial \alpha_i^{(\text{output})}}$ 而是直接计算 $\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(\text{output})}}$。

输入层不需要激活函数（identity即可）。

```python
class sigmoid:
    def __init__(self):
        pass
    def func(self,X):
        return 1.0/(1+np.exp(-X))
    def deriv(self,X):
        return self.func(X)*(1-self.func(X))

class relu:
    def __init__(self):
        pass
    def func(self,X):
        return np.maximum(0,X)
    def deriv(self,X):
        return 1*(X >= 0)

class softmax:
    def __init__(self):pass
    def func(self,X):
        return np.exp(X-X.max())/np.sum(np.exp(X-X.max()))
    # 统一减去最大值可以保证不出现溢出，同时因为存在归一化所以结果不变
    # 仅用于输出层，不可调整，故不定义导数

class identity:
    def __init__(self):pass
    def func(self,X): return X
    def deriv(self,X): return 1
```

### 参数更新：基于反向传播

损失函数我们只有在输出层通过计算得到，所以要想实现对参数的更新，要先通过参数计算前向传播到输出层得到损失，再反向传播求偏导更新各层参数。

由$\alpha_i^{(l)}=\sum\limits_{j=1}^{N_{l-1}} w_{ji}^{(l)}x_j^{(l-1)}+b_i^{(l)}$知$\dfrac{\partial \alpha_i^{(l)}}{\partial x_j^{(l-1)}}=w_{ji}^{(l)}$，$\dfrac{\partial \alpha_i^{(l)}}{\partial w_{ji}^{(l)}}=x_{j}^{(l-1)}$，$\dfrac{\partial \alpha_i^{(l)}}{\partial b}=1$。

由$x_i^{(l)}=\sigma(\alpha_i^{(l)})$知$\dfrac{\partial x_i^{(l)}}{\partial \alpha_i^{(l)}}=\sigma'(\alpha_i^{(l)})=\sigma(\alpha_i^{(l)})\times\sigma(1-\alpha_i^{(l)})$。

所以，结合上图，要实现对参数的更新，需要从输出层开始迭代计算才能得到。则更新参数：

$$
\begin{align}
w_{ij}^{(l)} &= w_{ij}^{(l)} - \eta\dfrac{\partial \mathcal{L}}{\partial w_{ij}^{(l)}}\\
&= w_{ij}^{(l)} - \eta\sum\limits_{m}^{N_L}\dfrac{\partial \mathcal{L}}{\partial \hat{y}_m}\dfrac{\partial \hat{y}_m}{\partial \alpha_m^{(L)}}\left(\sum\limits_{m}^{N_{L}}\sum\limits_{p}^{N_{L-1}}\dfrac{\partial \alpha_m^{(L)}}{\partial x_p^{(L-1)}}\dfrac{\partial x_p^{(L-1)}}{\partial \alpha_p^{(L-1)}}\cdots\sum\limits_{q}^{N_{l+1}}\sum\limits_{r}^{N_{l}}\dfrac{\partial \alpha_q^{(l+1)}}{\partial x_r^{(l)}}\dfrac{\partial x_r^{(l)}}{\partial \alpha_r^{(l)}}\cdot\dfrac{\partial \alpha_r^{(l)}}{\partial w_{ij}^{(l)}}\right)\\
b_{j}^{(l)} &= b_{j}^{(l)} - \eta\dfrac{\partial \mathcal{L}}{\partial b_{j}^{(l)}}\\
&= b_{j}^{(l)} - \eta\sum\limits_{m}^{N_L}\dfrac{\partial \mathcal{L}}{\partial \hat{y}_m}\dfrac{\partial \hat{y}_m}{\partial \alpha_m^{(L)}}\left(\sum\limits_{m}^{N_{L}}\sum\limits_{p}^{N_{L-1}}\dfrac{\partial \alpha_m^{(L)}}{\partial x_p^{(L-1)}}\dfrac{\partial x_p^{(L-1)}}{\partial \alpha_p^{(L-1)}}\cdots\sum\limits_{q}^{N_{l+1}}\sum\limits_{r}^{N_{l}}\dfrac{\partial \alpha_q^{(l+1)}}{\partial x_r^{(l)}}\dfrac{\partial x_r^{(l)}}{\partial \alpha_r^{(l)}}\cdot\dfrac{\partial \alpha_r^{(l)}}{\partial b_{j}^{(l)}}\right)
\end{align}
$$
实际上，要更新某一层网络的参数，需要首先算出的是损失函数对当前神经元输入的偏微分$\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}$，再分别求输入对$w$、$b$的偏微分。而算出$\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}$实际上依赖于下一层的梯度$\dfrac{\partial \mathcal{L}}{\partial \alpha_j^{(l+1)}}(j=1,2,\cdots.N_{l+1})$，以此类推，所以还需要一个属性用于存储该梯度，便于反向运算。

### 层结构确定

* 基本结构：
    * 权重矩阵`w`，规模为上层神经元数量×本层神经元数量。还要加上一个旧$w$的存储。
    * 偏置向量`b`，规模为本层神经元数量×1。
    * 本层输入向量`x`：规模为本层神经元数量×1。
    * 本层输出向量$\alpha$：规模为本层神经元数量×1。
    * 梯度向量：存储$\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}$值，规模为本层神经元数量×1。
    * 损失函数
    * 指向上一层（前向传播和反向传播求$w$导数时用到了上一层的$x$）、下一层（反向传播用到了后层的梯度和$w$）的指针。

* 方法：
    * 初始化：随机指定$w$、$b$
    * `forward()`实现前向传播
        * 根据上层输出计算本层输入$\alpha_i^{(l)}=\sum\limits_{j=1}^{N_{l-1}}w_{ji}^{(l)}x_j^{(l-1)} + b_i^{(l)}$
        * 使用激活函数算出本层输出$x_i^{(l)}=\sigma(\alpha_i^{(l)})$
        * 递归调用后一层的`forward()`
    * `backPropagation()`实现反向传播
        * 计算本层梯度$\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}=\sigma'(\alpha_i^{(l)})\cdot\left(\sum\limits_{j=1}^{N_{l-1}}\dfrac{\partial \mathcal{L}}{\partial \alpha_j^{(l+1)}}w_{ij}^{(l+1)}\right)$。
        * 更新参数$w_{ij}^{(l)}=w_{ij}^{(l)}-\eta\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}\cdot x_i^{(l)}$；$b_{i}^{(l)}=b_{i}^{(l)}-\eta\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(l)}}\cdot1$
        * 递归调用前一层的BP.

* 继承：
    * 设定隐层为基类`Layer`，输入层和输出层集成基类，重写方法。更改指定的激活函数。
    * 输出层`outputLayer`：
        * 追加属性`y`，存储目标值
        * 追加计算损失函数的方法`calcLoss()`;
        * 重写BP方法，修改梯度的计算方法为$\dfrac{\partial \mathcal{L}}{\partial \alpha_i^{(\text{output})}}=\text{Softmax}\left(\alpha_i^{(\text{output})}\right)-y_i$;
        * 重写forward方法，不再递归调用forward。
    * 输入层`inputLayer`：
        * 重写forward方法，要求传入x直接为输出，不计算w、b。
        * 重写BP方法，不进行计算，不递归调用。

```python
class Layer:
    lr = 0.001
    def __init__(self, size, prevsize, activation_func=sigmoid()):
        self.w = np.random.randn(prevsize,size)
        self.b = np.random.randn(size,1)
        self.alpha = np.zeros((size,1))
        self.x = np.zeros((size,1))
        self.w_old = self.w.copy()
        self.grad = np.zeros((size,1))
        self.act_func = activation_func
        self.prevLayer = None
        self.nextLayer = None

    def forward(self):
        self.alpha = np.sum(self.prevLayer.x* self.w.T ,axis=1)
        self.x = self.act_func.func(self.alpha)
        self.w_old = self.w.copy()
        self.nextLayer.forward()

    def backPropagation(self):
        self.grad = np.sum(self.nextLayer.grad*self.nextLayer.w_old,axis=1)*self.act_func.deriv(self.alpha)
        self.w = np.array(self.w - Layer.lr*np.mat(self.prevLayer.x).T*np.mat(self.grad))
        self.b = self.w - Layer.lr*self.grad
        self.prevLayer.backPropagation()
```

```python
class outputLayer(Layer):
    def __init__(self, size, prevsize, activation_func=softmax(), loss_func=crossEntropy):
        super(outputLayer, self).__init__(size, prevsize,activation_func)
        self.y = np.zeros((size,1))
        self.loss_func = crossEntropy

    def calcLoss(self):
        return self.loss_func(self.y,self.x)

    def forward(self):
        self.alpha = np.sum(self.prevLayer.x* self.w.T ,axis=1)
        self.x = self.act_func.func(self.alpha)
        self.w_old = self.w.copy()

    def backPropagation(self):
        self.grad = self.act_func.func(self.alpha) - self.y
        self.w = np.array(self.w - Layer.lr*np.mat(self.prevLayer.x).T*np.mat(self.grad))
        self.b = self.b - Layer.lr*self.grad
        self.prevLayer.backPropagation()
```

```python
class inputLayer(Layer):
    def __init__(self, size, activation_func=identity()):
        super(inputLayer, self).__init__(size, 1, activation_func)

    def forward(self,X):
        self.x = self.act_func.func(X)
        self.nextLayer.forward()

    def backPropagation(self):
        pass
```

## 神经网络实现

因为上一节中体验了`sklearn`的神经网络分类器，因此部分功能的实现和方法也尽量仿照其功能来写，沿用部分属性名。

### 初始化

注意到`sklearn`的初始化并未传入训练数据，且不指定输入输出层参数（实际上与数据有关），判定此时实际上没有初始化，仅进行参数存储，因此`__init__()`方法也只存参数。

### 训练

网络的初始化实际从这里开始，根据输入输出特征（自适应）调整输入层、输出层结构，根据参数调整隐层结构。

当网络各层构建完成时，更新网络属性中的前后层指针。

开始迭代训练（调用第一层forward和最后一层BP），设置终止条件：①迭代次数到限制；②参数变化小于阈值。

记录训练过程（损失值、准确率）

### 测试

仅调用第一层forward，输入数据，计算损失值、准确率并返回。

### 预测

由于原数据值不是onehot而是一列向量，所以返回的应该是经过argmax处理的结果。同样是调用forward计算到最后一层，再保存所有结果。

### 实现

```python
class BPNeuralNetwork:
    def __init__(self,hidden_layer_sizes=100,activation=sigmoid(),learning_rate= 0.001,tol=0.0001,max_iter=200,n_iter_no_change=5):
        self.hidden_layer_sizes = hidden_layer_sizes
        self.activation = activation
        self.learning_rate = learning_rate
        self.tol = tol
        self.max_iter = max_iter
        self.n_iter_no_change = n_iter_no_change
        self.layers = None

    def train(self,X,Y):
        # 初始化网络
        Layer.lr = self.learning_rate
        input_dim = X.shape[1]
        output_dim = np.unique(Y).shape[0]
        self.layers = []
        self.layers.append(inputLayer(input_dim))
        prev_size = input_dim
        if type(self.hidden_layer_sizes) == tuple:
            for size in self.hidden_layer_sizes:
                hidden_layer = Layer(size,prev_size,activation_func=self.activation)
                self.layers.append(hidden_layer)
                prev_size = size
        elif type(self.hidden_layer_sizes) == int:
            self.layers.append(Layer(self.hidden_layer_sizes,prev_size,activation_func=self.activation))
            prev_size = self.hidden_layer_sizes
        else: raise TypeError('Only int & tuple inputs supported for \'hidden_layer_sizes\'')
        self.layers.append(outputLayer(output_dim, prev_size))
        for i in range(len(self.layers)-1):
            self.layers[i].nextLayer = self.layers[i+1]
        for i in range(len(self.layers)-1,0,-1):
            self.layers[i].prevLayer = self.layers[i-1]

        Y_oh = np.eye(output_dim)[Y]

        # 训练
        i = 0; iter_no_change = 0
        accuracy = [0,]; loss = [0,]
        flag = False
        while(i < self.max_iter):
            i+=1
            accu_count = 0; iter_loss = 0
            print('\rTotal Iter: %d / %d' % (i,self.max_iter),end='')
            for j in range(X.shape[0]):
                self.layers[-1].y = Y_oh[j]
                self.layers[0].forward(X[j]) #前向传播
                self.layers[-1].backPropagation() # 反向传播

                iter_loss += self.layers[-1].calcLoss()
                accu_count += (np.argmax(self.layers[-1].x)==Y[j])

            iter_accu = accu_count/X.shape[0]
            iter_loss = iter_loss/X.shape[0]

            accuracy.append(iter_accu)
            loss.append(iter_loss)
            history = {'accuracy':accuracy,'loss':loss}
            if abs(accuracy[i]-accuracy[i-1]) <= self.tol or abs(loss[i]-loss[i-1]) <= self.tol:
                iter_no_change += 1
                if iter_no_change >= self.n_iter_no_change:
                    print('\nOptimization beneath tol.')
                    return history # 终止条件2：n次优化值小于tol
            else: iter_no_change = 0
        print()
        return history # 终止条件1：迭代次数达max

    def test(self,X,Y):
        output_dim = np.unique(Y).shape[0]
        Y_oh = np.eye(output_dim)[Y]
        accu_count = 0; loss = 0
        for j in range(X.shape[0]):
            self.layers[-1].y = Y_oh[j]
            self.layers[0].forward(X[j]) #前向传播

            loss += self.layers[-1].calcLoss()
            accu_count += (np.argmax(self.layers[-1].x)==Y[j])

        accu = accu_count/X.shape[0]
        loss = loss/X.shape[0]
        result = {'accuracy':accu,'loss':loss}
        return result

    def predict(self,X):
        result = []
        for j in range(X.shape[0]):
            self.layers[0].forward(X[j]) #前向传播
            result.append(np.argmax(self.layers[-1].x))
        return np.array(result)

    def display_model(self):
        if self.layers is None: return None
        else:
            layers_info = []
            for layer in self.layers:
                info = []
                info.append(layer.__class__.__name__)
                info.append(layer.x.shape[0])
                info.append(layer.act_func.__class__.__name__)
                layers_info.append(info)
            return pd.DataFrame(layers_info,columns=['Layer Type','Size','Activation Function'])
```

## 模型应用：手写数字分类

数据已经在第一节时加载，现在根据现有模型调整不同参数展开实验。

```python
def train_evaluate(model,history,X_test,y_test): #可视化训练过程
    print('Model Structure:')
    display(model.display_model())
    train_accu = history['accuracy']
    train_loss = history['loss']

    iters = range(1,len(train_accu))

    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.plot(iters,train_accu[1:],'-g',label='train_accu')
    ax2 = ax.twinx();ax2.set_ylabel('loss')
    ax2.plot(iters,train_loss[1:],'--r',label='train_loss')
    ax2.set_ylim(0,train_loss[1])
    ax.set_ylim(0,1);ax.set_ylabel('accuracy')
    ax.set_xlabel('iteration')
    ax.set_title('Training Process of Neural Network')
    fig.legend()
    result = model.test(X_test,y_test)
    print('Test Loss:\t %.5f' % result['loss'])
    print('Test Accuracy:\t %.5f' % result['accuracy'])
```

```python
model = BPNeuralNetwork(hidden_layer_sizes=(100,),activation=sigmoid(),tol=1e-8,max_iter=200)
history = model.train(X_train,y_train)
```

    Total Iter: 87 / 200
    Optimization beneath tol.

```python
train_evaluate(model,history,X_test,y_test)
```

```python
model = BPNeuralNetwork(hidden_layer_sizes=(100,),activation=relu(),tol=1e-8,max_iter=200)
history = model.train(X_train,y_train)
```

    Total Iter: 23 / 200
    Optimization beneath tol.

```python
train_evaluate(model,history,X_test,y_test)
```

本轮对比可以看出，网络结构相同的情况下，使用`relu`作为激活函数往往能够使模型加速收敛，且最终表现较好；`sigmoid`函数因为导函数值较小，容易出现梯度消失的情况，所以不仅模型收敛速度较慢，而且也有可能因为多次梯度消失导致提前收敛，最终表现不是特别好。

```python
model.predict(X_test[:10])
```

    array([6, 2, 5, 3, 7, 8, 6, 2, 1, 3])

```python
model = BPNeuralNetwork(hidden_layer_sizes=(32,),activation=relu(),tol=1e-8,max_iter=200)
history = model.train(X_train,y_train)

train_evaluate(model,history,X_test,y_test)
```

```python
model = BPNeuralNetwork(hidden_layer_sizes=(32,16),activation=relu(),tol=1e-8,max_iter=200)
history = model.train(X_train,y_train)

train_evaluate(model,history,X_test,y_test)
```

```python
model.layers[2].grad
```

    array([-0.00000000e+00,  0.00000000e+00, -0.00000000e+00,  0.00000000e+00,
           -2.53209703e-08, -0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
            0.00000000e+00,  0.00000000e+00, -0.00000000e+00,  0.00000000e+00,
            0.00000000e+00, -0.00000000e+00, -0.00000000e+00, -0.00000000e+00])

这一次实验隐藏层数增加，展现了`relu`函数的弱点，即**“死亡Relu”**。因为参数不恰当的更新（比如学习率设置的过大）导致出现负值，而relu函数、导函数在自变量为负值时均为0，会导致某一梯度变为0之后，永远无法更新，也就是这个隐层神经元“死亡”了。而同样的网络结构换用`sigmoid`作为激活函数并不会出现这样的问题。

```python
model = BPNeuralNetwork(hidden_layer_sizes=(32,16),activation=sigmoid(),tol=1e-8,max_iter=200)
history = model.train(X_train,y_train)

train_evaluate(model,history,X_test,y_test)
```

```python
model.layers[2].grad
```

    array([ 3.60639992e-03,  1.25682840e-04,  6.93461731e-03, -6.40269890e-04,
           -4.00750112e-02,  2.30742193e-02, -5.82608968e-02,  4.12186950e-06,
           -1.02405951e-03, -5.15133091e-04,  3.01855122e-04, -2.57302600e-03,
           -6.43587049e-04,  1.54500792e-02,  4.61627819e-02,  1.63981829e-02])

# 总结

1. 完成了实验的各项要求，通过使用sklearn的神经网络分类器和自己使用numpy实现神经网络分类器，增进了对前馈神经网络的理解，通过模型对手写数字数据集进行了分类，调优后的模型表现均尚可。
2. sklearn实验能够逐步理解参数，而复现神经网络感觉难度较大，大量的时间花费在对基本的神经网络正向传播和反向传播的理解中，自己也参考课本和网络上的一些资料，对于推导过程进行了推演和编程实现，通过广泛的学习和实验，对网络布局和激活函数的使用有了更加直观的理解。理论课程中虽说感觉把握了神经网络及其反向传播的基本思想，但是在复现的过程中还是感觉有些许困难，必须自己推演一遍才能掌握。另外，感觉自己对于numpy向量、矩阵运算不够熟练，导致完成时间较长。
3. 神经网络复现部分，参考对比了网络上的一些实现的文章，我实现的神经网络的优点主要在于：
    1. 能够自适应调整输入、输出层维度；
    2. 能够自定义添加多个隐层（很多实现只包含单个隐层），且有多个参数可调节，自定义程度更高；
    3. 网络、层、激活函数都使用了面向对象的思想，代码结构方面可能更清晰、易于理解。
    4. 能够展现网络结构，易于理解。
4. 本次实验的不足：
    1. 没有实现分批(batch)训练，整体训练速度一般。
    2. 虽然使用面向对象易于理解，但是代码整体较长，不如其他人的简洁凝练。
    3. 没有添加完备的异常处理，仅对神经网络进行了复现和基本实验，健壮性有待提升。


# Simplified Ikeda sensitivity

```python
#from jupyterthemes import jtplot
#jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)
```

```python
%matplotlib inline
%load_ext autoreload
%autoreload 2
```

```python
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
import numpy as np
import matplotlib.pyplot as plt
from pylab import rcParams
rcParams['figure.figsize'] = 15, 5

from rolldecayestimators.simplified_ikeda import calculate_roll_damping, SimplifiedIkedaInputError
from rolldecayestimators import equations
import sympy as sp
from rolldecayestimators import symbols
from rolldecayestimators.substitute_dynamic_symbols import lambdify
from mdldb.tables import Run
from rolldecayestimators.ikeda_estimator import IkedaEstimator
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from sklearn.pipeline import Pipeline

from rolldecay import database

import data
import copy
from rolldecay import database
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from rolldecay.paper_writing import save_fig
```

```python
from rolldecay.sensitivity import calculate
```

```python
db = database.get_db()
```

```python
sql =  """
SELECT * from
loading_conditions
INNER JOIN models
    ON loading_conditions.model_number==models.model_number
    INNER JOIN ships
        ON ships.name == models.ship_name
"""

df = pd.read_sql_query(sql=sql, con=db.engine)
```

```python
mask = df['Volume']>0
df=df.loc[mask].copy()
```

```python
df.describe()
```

```python
df['DRAFT']=(df.TA + df.TF)/2
df['CB'] = df.Volume/(df.lpp*df.beam*df.DRAFT)
mask = df['CB'].abs()!=np.inf
df=df.loc[mask]
```

```python
df['CB'].hist(bins=30)
```

```python
mask = ((0 < df['CB']) & (df['CB'] <=1))
df = df.loc[mask]
```

```python
mask = df['Volume'] == df['Volume'].median()
ship = df.loc[mask].iloc[0]
ship['phi_max'] = 5
ship['omega0'] = 0.5
```

```python
ship.Volume/(ship.lpp*ship.beam*(ship.TA+ship.TF)/2)
```

```python
from rolldecay.sensitivity import variate_ship
```

```python
N=30
changes = np.linspace(0.5,1.5,N)
df = variate_ship(ship=ship, key='lpp', changes=changes)
```

```python
result = df.apply(func=calculate, axis=1)
```

```python
result.plot()
```

```python
from rolldecay.sensitivity import plot_variation, calculate_variation
```

```python
plot_variation(ship=ship, key='lpp')
```

```python
variations = ['lpp','beam','DRAFT','phi_max','BKL','BKB','omega0','kg','CB','A0']
N=30
changes = np.linspace(0.85,1.32,N)

ylims=[]
axes=[]
for variation in variations:

    fig,ax=plt.subplots()
    plot_variation(ship=ship, key=variation, ax=ax, changes=changes, catch_error=True)
    ax.set_title('Variation of %s: %0.2f' % (variation, ship[variation]))

    ylims.append(list(ax.get_ylim()))
    axes.append(ax)

for ax in axes:
    ax.set_ylim(np.min(ylims),np.max(ylims))

```

```python

```

```python
plot_variation(ship=ship, key='lpp')
```

```python
ikeda_parameters = ['lpp','beam','DRAFT','phi_max','BKL','BKB','omega0','kg','CB','A0']
ship_=ship[ikeda_parameters]
ship_['phi_max']=8
```

```python
ship_
```

```python
calculate(ship_)
```

```python
ship2 = ship_.copy()
scale = 0.01
ship2['lpp']/=scale
ship2['beam']/=scale
ship2['DRAFT']/=scale
ship2['BKL']/=scale
ship2['BKB']/=scale
ship2['kg']/=scale
ship2['omega0']*=np.sqrt(scale)
calculate(ship2)
```

```python
ship2 = ship_.copy()
scale = ship_['lpp']
ship2['lpp']/=scale
ship2['beam']/=scale
ship2['DRAFT']/=scale
ship2['BKL']/=scale
ship2['BKB']/=scale
ship2['kg']/=scale
ship2['omega0']*=np.sqrt(scale)

#ship2['BKB']*=10
calculate(ship2)
```

```python
variations = list(set(ikeda_parameters) - set(['lpp']))

N=100
changes = np.linspace(0.85,1.43,N)

ylims=[]
axes=[]
for variation in variations:

    fig,ax=plt.subplots()
    plot_variation(ship=ship2, key=variation, ax=ax, changes=changes, catch_error=True,
                   plot_change_factor=False)

    ylims.append(list(ax.get_ylim()))
    axes.append(ax)
    ax.grid(True)

for ax in axes:
    ax.set_ylim(np.min(ylims),np.max(ylims))
```

```python

df = variate_ship(ship=ship2, key='beam', changes=changes)
result = calculate_variation(df=df, catch_error=True)
result = pd.concat([result,df], axis=1)
```

```python
result['Disp'] = result['CB']*result['lpp']*result['beam']*result['DRAFT']
```

```python
equations.B44_hat_equation
```

```python
B44_equation = sp.Eq(symbols.B_44,sp.solve(equations.B44_hat_equation,symbols.B_44)[0])
B44_equation
```

```python
B44_lambda = lambdify(sp.solve(B44_equation,symbols.B_44)[0])
```

```python
result['B44'] = B44_lambda(result['B44HAT'], result['Disp'], beam=result['beam'], g=9.81, rho=1000)
```

```python
result.plot(x='beam', y = 'B44')
```

```python
variations = list(set(ikeda_parameters) - set(['lpp']))

N=100
changes = np.linspace(0.7,1.32,N)

ylims=[]
axes=[]
for variation in variations:

    fig,ax=plt.subplots()
    df = variate_ship(ship=ship2, key=variation, changes=changes)
    result = calculate_variation(df=df, catch_error=True)
    #result.fillna(0,inplace=True)

    df['Disp'] = df['CB']*df['lpp']*df['beam']*df['DRAFT']

    dampings = ['B44','BW','BE','BBK']
    for key in dampings:
        result[key] = B44_lambda(np.array(result[key+'HAT']), np.array(df['Disp']), beam=df['beam'].astype(float),
                                 g=9.81, rho=1000)

    result[variation]=df[variation]
    result.plot(x=variation, y=dampings, ax=ax)
    ax.plot(ship2[variation],0,'ro', label='reference')

    ylims.append(list(ax.get_ylim()))
    axes.append(ax)

for ax in axes:
    ax.set_ylim(np.min(ylims),np.max(ylims))
```

```python
from rolldecayestimators.ikeda_speed import bilge_keel
variation='BKB'
N=100
changes = np.linspace(0.5,3,N)
df = variate_ship(ship=ship2, key=variation, changes=changes)
result = calculate_variation(df=df, catch_error=True)
```

```python
dddf = variate_ship(ship=ship2, key=variation, changes=changes)
ddresult = calculate_variation(df=df, catch_error=True)
```

```python
R=0.3/175
df=df.astype(float)

def calculate_bilge_keel(row,R):
    OG=-row.kg + row.DRAFT
    Ho=(row['beam']/(2*row['DRAFT'])  )
    A=(row['A0']*row['beam']*row['DRAFT'])

    Bp44BK_N0, Bp44BK_H0, B44BK_L, B44BKW0 = bilge_keel(w=row['omega0'],
               fi_a=np.deg2rad(row['phi_max']), V=0, B=row['beam'],
               d=row['DRAFT'],
               A=A, bBK=row['BKB'], R=R, g=9.81, OG=OG, Ho=Ho, ra=1000)

    s = pd.Series(name=row.name)
    s['Bp44BK_N0'] = Bp44BK_N0
    s['Bp44BK_H0'] = Bp44BK_H0
    s['B44BK_L'] = B44BK_L
    s['B44BKW0'] = B44BKW0

    LBK = row.BKL
    B44BK_N0 = Bp44BK_N0*LBK
    B44BK_H0 = Bp44BK_H0*LBK
    B44BK_L = B44BK_L
    # B44BKW0 = B44BKW0 * dim...
    B44_BK = B44BK_N0 + B44BK_H0 + B44BK_L
    s['B44_BK'] = B44_BK

    return s
```

```python
result2 = df.apply(func=calculate_bilge_keel, R=R, axis=1)
```

```python
key='BBK'
df['Disp'] = df['CB']*df['lpp']*df['beam']*df['DRAFT']
result[key] = B44_lambda(np.array(result[key+'HAT']), np.array(df['Disp']), beam=df['beam'].astype(float),
                                 g=9.81, rho=1000)
```

```python
fig,ax=plt.subplots()
result.plot(y='BBK', ax=ax)
result2.plot(y='B44_BK', ax=ax)
```

```python
df_rolldecay = database.load(rolldecay_table_name='rolldecay_quadratic_b', limit_score=0.90,
                             exclude_table_name='rolldecay_exclude')
db = database.get_db()
```

```python
loading_conditions=df_rolldecay.copy()
scale = np.array(loading_conditions['lpp'])
loading_conditions['lpp']/=scale
loading_conditions['beam']/=scale
loading_conditions['DRAFT']=(loading_conditions['TA']+loading_conditions['TF'])/2/scale
loading_conditions['BKL']/=scale
loading_conditions['BKB']/=scale
loading_conditions['kg']/=scale
volume = loading_conditions['Volume']/scale**3
lpp=loading_conditions['lpp']
T=loading_conditions['DRAFT']
beam=loading_conditions['beam']
loading_conditions['phi_max']=np.rad2deg(loading_conditions['phi_start'].abs())

loading_conditions['CB']=volume/(lpp*beam*T)
```

```python
variations = list(set(ikeda_parameters) - set(['lpp']))

N=100
ship2.omega0 = loading_conditions['omega0'].median()
ylimss=[]
axes=[]
fig,axes=plt.subplots(nrows=len(variations))

for variation,ax in zip(variations,axes):

    s = loading_conditions[variation]
    changes = np.linspace(s.min(),s.max(),N)/ship2[variation]

    df = variate_ship(ship=ship2, key=variation, changes=changes)
    result = calculate_variation(df=df, catch_error=True)
    #result.fillna(0,inplace=True)

    df['Disp'] = df['CB']*df['lpp']*df['beam']*df['DRAFT']

    dampings = ['B44','BW','BE','BBK']
    for key in dampings:
        result[key] = B44_lambda(np.array(result[key+'HAT']), np.array(df['Disp']), beam=df['beam'].astype(float),
                                 g=9.81, rho=1000)

    result[variation]=df[variation]
    result.plot(x=variation, y=dampings, ax=ax)
    ylims = ax.get_ylim()
    ax.plot([ship2[variation],ship2[variation]],ylims,'r--', label='median ship')
    ax.get_legend().remove()

axes[0].legend()
plt.tight_layout()

save_fig(fig=fig, name='ikeda_variation', full_page=True)
```

```python

```


```python
# Importing standard Qiskit libraries
from qiskit import QuantumCircuit, execute, Aer, IBMQ, QuantumRegister
from qiskit.compiler import transpile, assemble
from qiskit.tools.jupyter import *
from qiskit.visualization import *
from ibm_quantum_widgets import *

import numpy as np
import qiskit as qk
import matplotlib.pyplot as plt

from fractions import Fraction
from qiskit.providers.ibmq import least_busy
from qiskit.tools.monitor import job_monitor

# Loading your IBM Q account(s)
provider = IBMQ.load_account()
```

```python
simulator = qk.BasicAer.get_backend('qasm_simulator')
real = least_busy(provider.backends(filters=lambda x: x.configuration().n_qubits > 4,
                                            operational=True, simulator=False))
print(real)
```

    ibmq_belem

### A Bernstein-Vazarini Probléma

Adott egy feketedoboz függvény ami egy $\underline{x} = (x_1, x_2, ..., x_n)$ bitsorhoz rendel hozzá 0-t vagy 1-et:

\begin{equation}
f(x_1, x_2, ..., x_n) = 0 \text{ or } 1.
\end{equation}
(Minden $i$-re $x_i$ 0 vagy 1.) Tudjuk, hogy a függvény minden $\underline{x}$ bitsor esetén egy $\underline{s}$ bitsorral vett bitenkénti szorzatának 2 szerinti maradékát adja vissza. A bitenkénti szorzat alatt a következő műveletet értjük: $\underline{s}\cdot\underline{x} = x_1\cdot s_1 +x_2\cdot s_2 +...+x_n\cdot s_n$. Tehát a függvényt a következő alakban írható fel: $f(\underline{x}) = \underline{s}\cdot\underline{x} \text{ mod } 2$. Az a feladat, hogy találjuk ki, hogy mi az $\underline{s}$ bitsor.

**1. feladat**

Gondoljuk végig, hogy hogyan oldanánk meg A Berstein-Vazarini problémát egy klasszikus számítógéppel. Hányszor kéne ehhez elvégezni az $f$ függvényt megvalósító műveletet?

```python
n = 4
s = np.random.randint(0, 2, n)

def f(x, s): # x egy és s azonos hosszúságú numpy array-ok
    if len(x) != n:
        raise ValueError("x and s have to be of the same length")
    return np.dot(x, s)%2
```

**2. feladat**
Határozzuk meg a véletlenszerűen generált $s$ bitsort, anélkül, hogy kiírnánk az értékét. Használjuk az f(x, s) függvényt, ami az $\underline{s}\cdot\underline{x} \text{ mod } 2$ értéket adja vissza.

```python
# megoldás helye

guess = np.zeros(n, dtype=int)
for i in range(n):
    x = np.zeros(n)
    x[i] = 1
    guess[i] = f(x, s)
print(guess, s)
```

    [0 0 1 1] [0 0 1 1]

**Állítás**: Az alábbi ábrán látható kvantumáramkör elvégzése után pontosan az $s$ bitsort mérjük. (Az utolsó qubitet nem kell mérni, annak értéke nem érdekel minket.) Tehát elég egyszer elvégeznünk az $f$ függvényt megvalósító műveletet.

A $|-\rangle$ állapotot úgy állíthatjuk elő, hogy az $|{1}\rangle$ állapotra hatunk egy $H$ kapuval.

```python
def black_box(s): # s egy bitsor
    n = len(s)
    qc = QuantumCircuit(n+1)
    for i in range(len(s)):
        if s[n-i-1] == 1:
            qc.cx(i, n)
    qc.name = "f"
    return qc
```

```python
"""az i egész szám bináris alakját írja be a függvény n darab qubitbe"""
def encode(i, n):
    if 2**n <= i:
        raise ValueError("'i' is too big to be stored on n qubits")
    bits = np.array(list(format(i, "b")), dtype=int)
    while len(bits) < n:
        bits = np.insert(bits, 0, 0)
    qc = QuantumCircuit(n)
    for j in range(len(bits)):
        if bits[j]  == 1:
            qc.x(n-j-1)
    qc.name = "%i" %i
    return qc
```

**3. feladat (szorgalmi)**

Ellenőrizzük, hogy a black_box(s) kvantumkapu úgy működik-e ahogy azt elvárjuk tőle az $\underline{s}=(1, 0, 1, 1)$ bitsor esetén:
- Hozzunk létre egy 5 qubites kvantumáramkört.
- Írjunk bele egy $\underline{x}$ bitsort az első 4 qubitbe. Ehhez használhatjuk az encode($i$, $n$) függvényt, ami az $i$ egész szám bináris alakját írja bele $n$ darab qubitbe, de a függvény nélkül is könnyen megoldható a feladat.
- Hattassuk a black_box(s) kaput az 5 qubitre, majd mérjük meg az 5. qubitet.
- Ha a black_box(s) kvantumkapu jól működik az $\underline{x}_0 = (0, 0, 0, 1)$, $\underline{x}_1 = (0, 0, 1, 0)$, $\underline{x}_2 = (0, 1, 0, 0)$, $\underline{x}_3 = (1, 0, 0, 0)$ bemeneti bitsorokra, akkor minden bemeneti bitsorra jól működik.

```python
s = np.array([1, 0, 1, 1])

# megoldás helye

for i in range(4):
    x = np.zeros(4)
    x[i] = 1
    qc = QuantumCircuit(5, 1)
    qc.append(encode(2**i, 4), range(4))
    qc.append(black_box(s), range(5))
    qc.measure(4, 0)
    counts = execute(qc, simulator, shots=1).result().get_counts()
    print(counts)
```

    {'1': 1}
    {'1': 1}
    {'0': 1}
    {'1': 1}

**4.feladat**
Rakjuk össze a fenti ábrán látható áramkört. Az $f$-el jelölt kapu helyére rakjuk a black_box($\underline{s}$) kaput. Legyen $\underline{s} = (1, 0, 1, 1)$.
Ellenőrizzük, hogy a kvantumáramkör mérésekor tényleg visszakapjuk-e az $s$ bitsort. (Az áramkört futtathatjuk $\underline{s}$ más értékeire is.) Próbáljuk ki a kvantumáramkört szimulátoron is és igazi kvantumszámítógépen is.

```python
# megoldás helye

qc = QuantumCircuit(5, 4)
qc.x(4)
qc.h(list(range(5)))
qc.append(black_box(s), range(5))
qc.h(list(range(4)))
qc.measure(range(4), range(4))
job = execute(qc, real, shots=100)
job_monitor(job)
counts = job.result().get_counts()
print(counts)
qc.draw()
```

    Job Status: job is being validated

```python
plt.bar(counts.keys(), counts.values())
```

```python

```


# Double Machine Learning: Use Cases and Examples

Double Machine Learning (DML) is an algorithm that applies arbitrary machine learning methods
to fit the treatment and response, then uses a linear model to predict the response residuals
from the treatment residuals.

The EconML SDK implements the following DML classes:
* LinearDML: suitable for estimating heterogeneous treatment effects.
* SparseLinearDML: suitable for the case when $W$ is high dimensional vector and both the first stage and second stage estimate are linear.

In ths notebook, we show the performance of the DML on both synthetic data and observational data.

**Notebook contents:**

1. Example usage with single continuous treatment synthetic data
2. Example usage with single binary treatment synthetic data
3. Example usage with multiple continuous treatment synthetic data
4. Example usage with single continuous treatment observational data
5. Example usage with multiple continuous treatment, multiple outcome observational data

```python
import econml
```

```python
## Ignore warnings
import warnings
warnings.filterwarnings("ignore")
```

```python
# Main imports
from econml.dml import DML, LinearDML, SparseLinearDML, CausalForestDML

# Helper imports
import numpy as np
from itertools import product
from sklearn.linear_model import (Lasso, LassoCV, LogisticRegression,
                                  LogisticRegressionCV,LinearRegression,
                                  MultiTaskElasticNet,MultiTaskElasticNetCV)
from sklearn.ensemble import RandomForestRegressor,RandomForestClassifier
from sklearn.preprocessing import PolynomialFeatures
import matplotlib.pyplot as plt
import matplotlib
from sklearn.model_selection import train_test_split

%matplotlib inline
```

## 1. Example Usage with Single Continuous Treatment Synthetic Data and Model Selection

### 1.1. DGP
We use the data generating process (DGP) from [here](https://arxiv.org/abs/1806.03467). The DGP is described by the following equations:

\begin{align}
T =& \langle W, \beta\rangle + \eta, & \;\eta \sim \text{Uniform}(-1, 1)\\
Y =& T\cdot \theta(X) + \langle W, \gamma\rangle + \epsilon, &\; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim& \text{Normal}(0,\, I_{n_w})\\
X \sim& \text{Uniform}(0,1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders and $\beta, \gamma$ have high sparsity.

For this DGP,
\begin{align}
\theta(x) = \exp(2\cdot x_1).
\end{align}

```python
# Treatment effect function
def exp_te(x):
    return np.exp(2*x[0])
```

```python
# DGP constants
np.random.seed(123)
n = 2000
n_w = 30
support_size = 5
n_x = 1
# Outcome support
support_Y = np.random.choice(np.arange(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n: np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE = np.array([exp_te(x_i) for x_i in X])
T = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
Y = TE * T + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)

Y_train, Y_val, T_train, T_val, X_train, X_val, W_train, W_val = train_test_split(Y, T, X, W, test_size=.2)
# Generate test data
X_test = np.array(list(product(np.arange(0, 1, 0.01), repeat=n_x)))
```

### 1.2. Train Estimator
We train models in three different ways, and compare their performance.

#### 1.2.1. Default Setting

```python
est = LinearDML(model_y=RandomForestRegressor(),
                model_t=RandomForestRegressor(),
                random_state=123)
est.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred = est.effect(X_test)
```

#### 1.2.2. Polynomial Features for Heterogeneity

```python
est1 = SparseLinearDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestRegressor(),
                       featurizer=PolynomialFeatures(degree=3),
                       random_state=123)
est1.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred1 = est1.effect(X_test)
```

    The number of features in the final model (< 5) is too small for a sparse model. We recommend using the LinearDML estimator for this low-dimensional setting.

#### 1.2.3. Polynomial Features with regularization

```python
est2 = DML(model_y=RandomForestRegressor(),
           model_t=RandomForestRegressor(),
           model_final=Lasso(alpha=0.1, fit_intercept=False),
           featurizer=PolynomialFeatures(degree=10),
           random_state=123)
est2.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred2 = est2.effect(X_test)
```

#### 1.2.4 Non-Parametric Heterogeneity with Causal Forest

```python
est3 = CausalForestDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestRegressor(),
                       criterion='mse', n_estimators=1000,
                       min_impurity_decrease=0.001,
                       random_state=123)
est3.tune(Y_train, T_train, X=X_train, W=W_train)
est3.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred3 = est3.effect(X_test)
```

### 1.3. Performance Visualization

```python
plt.figure(figsize=(10,6))
plt.plot(X_test, te_pred, label='DML default')
plt.plot(X_test, te_pred1, label='DML polynomial degree=3')
plt.plot(X_test, te_pred2, label='DML polynomial degree=10 with Lasso')
plt.plot(X_test, te_pred3, label='ForestDML')
expected_te = np.array([exp_te(x_i) for x_i in X_test])
plt.plot(X_test, expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.show()
```

### 1.4. Model selection

For the three different models above, we can use score function to estimate the final model performance. The score is the MSE of the final stage Y residual, which can be seen as a proxy of the MSE of treatment effect.

```python
score={}
score["DML default"] = est.score(Y_val, T_val, X_val, W_val)
score["DML polynomial degree=3"] = est1.score(Y_val, T_val, X_val, W_val)
score["DML polynomial degree=10 with Lasso"] = est2.score(Y_val, T_val, X_val, W_val)
score["ForestDML"] = est3.score(Y_val, T_val, X_val, W_val)
score
```

    {'DML default': 1.815769478666336,
     'DML polynomial degree=3': 1.6913528143752934,
     'DML polynomial degree=10 with Lasso': 2.222578476206349,
     'ForestDML': 1.9002757666765648}

```python
print("best model selected by score: ",min(score,key=lambda x: score.get(x)))
```

    best model selected by score:  DML polynomial degree=3

```python
mse_te={}
mse_te["DML default"] = ((expected_te - te_pred)**2).mean()
mse_te["DML polynomial degree=3"] = ((expected_te - te_pred1)**2).mean()
mse_te["DML polynomial degree=10 with Lasso"] = ((expected_te - te_pred2)**2).mean()
mse_te["ForestDML"] = ((expected_te - te_pred3)**2).mean()
mse_te
```

    {'DML default': 0.3565984526892961,
     'DML polynomial degree=3': 0.257796113358552,
     'DML polynomial degree=10 with Lasso': 0.26248739267870214,
     'ForestDML': 0.42857317657348887}

```python
print("best model selected by MSE of TE: ", min(mse_te, key=lambda x: mse_te.get(x)))
```

    best model selected by MSE of TE:  DML polynomial degree=3

### 1.5 Changing only Final Stage Model Specification and Refitting

It is also feasible to change the parameters of the estimator and only fit the final stage model using the existing first stage residual calculation. To enable this feature, the `fit` method should be called with the flag `cache_values=True`, so that the original fit data and their corresponding first stage estimates be stored in the model. This can be done for any estimator in the dml module. We portray an example below, using the `DML` estimator.

```python
est = DML(model_y=RandomForestRegressor(),
          model_t=RandomForestRegressor(),
          model_final=Lasso(alpha=0.1, fit_intercept=False),
          featurizer=PolynomialFeatures(degree=1, include_bias=False),
          random_state=123)
est.fit(Y_train, T_train, X=X_train, W=W_train, cache_values=True)
est.summary()
```

    Final model doesn't have a `coef_stderr_` and `intercept_stderr_` attributes, only point estimates will be available.
    Final model doesn't have a `coef_stderr_` and `intercept_stderr_` attributes, only point estimates will be available.

Coefficient Results

   	  point_estimate

  X0	      5.953

CATE Intercept Results

         	        point_estimate

  cate_intercept	       0.5

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

```python
from econml.sklearn_extensions.linear_model import DebiasedLasso
est.featurizer = PolynomialFeatures(degree=2, include_bias=False)
est.model_final = DebiasedLasso(fit_intercept=False)
est.refit_final()
est.summary()
```

Coefficient Results

    	   point_estimate	 stderr	 zstat	 pvalue	 ci_lower	 ci_upper

  X0	       -0.054	      0.685	 -0.079	  0.937	   -1.18	    1.072

  X0^2	      7.253	      0.695	 10.437	   0.0	    6.11	     8.396

CATE Intercept Results

         	        point_estimate	 stderr	 zstat	 pvalue	 ci_lower	 ci_upper

  cate_intercept	      1.289	      0.154	 8.374	   0.0	    1.036	    1.542

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

```python
from econml.sklearn_extensions.linear_model import StatsModelsLinearRegression
est.model_final = StatsModelsLinearRegression(fit_intercept=False)
est.refit_final()
est.summary()
```

Coefficient Results

    	   point_estimate	 stderr	  zstat	 pvalue	 ci_lower	 ci_upper

  X0	       -0.068	      0.759	 -0.089	  0.929	  -1.316	    1.18

  X0^2	      7.26	       0.744	  9.761	   0.0	    6.037	    8.484

CATE Intercept Results

         	        point_estimate	 stderr	 zstat	 pvalue	 ci_lower	 ci_upper

  cate_intercept	      1.29	       0.159	 8.116	   0.0	    1.029	    1.552

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

```python
from econml.sklearn_extensions.linear_model import StatsModelsRLM
est.model_final = StatsModelsRLM(fit_intercept=False)
est.fit_cate_intercept = False
est.refit_final()
est.summary()
```

    CATE Intercept Results:  No intercept was fitted!

Coefficient Results

    	   point_estimate	 stderr	  zstat	 pvalue	 ci_lower	 ci_upper

  X0	        5.566	      0.347	 16.029	   0.0	    4.995	    6.137

  X0^2	      2.243	      0.451	  4.975	   0.0	    1.501	    2.984

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

```python
est.model_final = Lasso(fit_intercept=False)
est.refit_final()
est.effect_inference(X).population_summary()
```

    Final model doesn't have a `prediction_stderr` method, only point estimates will be returned.

Uncertainty of Mean Point Estimate

  mean_point

     0.697

Distribution of Point Estimate

  std_point	 pct_point_lower	 pct_point_upper

    0.398	        0.075	           1.319

## 2. Example Usage with Single Binary Treatment Synthetic Data and Confidence Intervals

### 2.1. DGP
We use the following DGP:

\begin{align}
T \sim & \text{Bernoulli}\left(f(W)\right), &\; f(W)=\sigma(\langle W, \beta\rangle + \eta), \;\eta \sim \text{Uniform}(-1, 1)\\
Y = & T\cdot \theta(X) + \langle W, \gamma\rangle + \epsilon, & \; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim & \text{Normal}(0,\, I_{n_w}) & \\
X \sim & \text{Uniform}(0,\, 1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders, $\beta, \gamma$ have high sparsity and $\sigma$ is the sigmoid function.

For this DGP,
\begin{align}
\theta(x) = \exp( 2\cdot x_1 ).
\end{align}

```python
# Treatment effect function
def exp_te(x):
    return np.exp(2 * x[0])# DGP constants

np.random.seed(123)
n = 1000
n_w = 30
support_size = 5
n_x = 4
# Outcome support
support_Y = np.random.choice(range(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n:np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE = np.array([exp_te(x_i) for x_i in X])
# Define treatment
log_odds = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
T_sigmoid = 1/(1 + np.exp(-log_odds))
T = np.array([np.random.binomial(1, p) for p in T_sigmoid])
# Define the outcome
Y = TE * T + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)

# get testing data
X_test = np.random.uniform(0, 1, size=(n, n_x))
X_test[:, 0] = np.linspace(0, 1, n)
```

### 2.2. Train Estimator

```python
est = LinearDML(model_y=RandomForestRegressor(),
                model_t=RandomForestClassifier(min_samples_leaf=10),
                discrete_treatment=True,
                linear_first_stages=False,
                cv=6)
est.fit(Y, T, X=X, W=W)
te_pred = est.effect(X_test)
lb, ub = est.effect_interval(X_test, alpha=0.01)
```

```python
est2 = SparseLinearDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestClassifier(min_samples_leaf=10),
                       discrete_treatment=True,
                       featurizer=PolynomialFeatures(degree=2),
                       linear_first_stages=False,
                       cv=6)
est2.fit(Y, T, X=X, W=W)
te_pred2 = est2.effect(X_test)
lb2, ub2 = est2.effect_interval(X_test, alpha=0.01)
```

```python
est3 = CausalForestDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestClassifier(min_samples_leaf=10),
                       discrete_treatment=True,
                       n_estimators=1000,
                       min_impurity_decrease=0.001,
                       verbose=0,
                       cv=6)
est3.tune(Y, T, X=X, W=W)
est3.fit(Y, T, X=X, W=W)
te_pred3 = est3.effect(X_test)
lb3, ub3 = est3.effect_interval(X_test, alpha=0.01)
```

```python
est3.feature_importances_
```

    array([0.99117512, 0.00264324, 0.00244716, 0.00373448])

### 2.3. Performance Visualization

```python
expected_te=np.array([exp_te(x_i) for x_i in X_test])
plt.figure(figsize=(16,6))
plt.subplot(1, 3, 1)
plt.plot(X_test[:, 0], te_pred, label='LinearDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb, ub, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.subplot(1, 3, 2)
plt.plot(X_test[:, 0], te_pred2, label='SparseLinearDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb2, ub2, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.subplot(1, 3, 3)
plt.plot(X_test[:, 0], te_pred3, label='ForestDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb3, ub3, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.show()
```

### 2.4 Interpretability with SHAP Values

Explain the hetergoeneity model for the constant marginal effect of the treatment using SHAP values.

```python
import shap
```

```python
shap_values = est.shap_values(X)
shap.plots.beeswarm(shap_values['Y0']['T0_1'])
```

```python
shap_values2 = est2.shap_values(X)
shap.plots.beeswarm(shap_values2['Y0']['T0_1'])
```

```python
shap_values3 = est3.shap_values(X)
shap.plots.beeswarm(shap_values3['Y0']['T0_1'])
plt.show()
```

### 2.5. Other Inferences
#### 2.5.1 Effect Inferences
Other than confidence interval, we could also output other statistical inferences of the effect include standard error, z-test score and p value given each sample $X[i]$.

```python
est.effect_inference(X_test[:10,]).summary_frame(alpha=0.1, value=0, decimals=3)
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      X

      0
      0.372
      0.146
      2.540
      0.011
      0.131
      0.613

      1
      0.416
      0.163
      2.553
      0.011
      0.148
      0.683

      2
      0.291
      0.170
      1.709
      0.087
      0.011
      0.571

      3
      0.410
      0.150
      2.733
      0.006
      0.163
      0.656

      4
      0.584
      0.152
      3.834
      0.000
      0.333
      0.834

      5
      0.553
      0.139
      3.974
      0.000
      0.324
      0.782

      6
      0.430
      0.124
      3.472
      0.001
      0.226
      0.634

      7
      0.605
      0.154
      3.921
      0.000
      0.351
      0.859

      8
      0.439
      0.127
      3.471
      0.001
      0.231
      0.647

      9
      0.494
      0.162
      3.048
      0.002
      0.227
      0.760

We could also get the population inferences given sample $X$.

```python
est.effect_inference(X_test).population_summary(alpha=0.1, value=0, decimals=3, tol=0.001)
```

Uncertainty of Mean Point Estimate

  mean_point	 stderr_mean	  zstat	 pvalue	 ci_mean_lower	 ci_mean_upper

     3.368	      0.061	    54.777	   0.0	      3.267	         3.469

Distribution of Point Estimate

  std_point	 pct_point_lower	 pct_point_upper

    1.729	        0.683	           6.074

Total Variance of Point Estimate

  stderr_point	 ci_point_lower	 ci_point_upper

      1.73	          0.675	          6.067

#### 2.5.2 Coefficient and Intercept Inferences
We could also get the coefficient and intercept inference for the final model when it's linear.

```python
est.coef__inference().summary_frame()
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      X

      X0
      5.969
      0.229
      26.053
      0.000
      5.592
      6.346

      X1
      -0.060
      0.216
      -0.277
      0.782
      -0.415
      0.295

      X2
      -0.363
      0.219
      -1.660
      0.097
      -0.722
      -0.003

      X3
      0.271
      0.212
      1.280
      0.201
      -0.077
      0.620

```python
est.intercept__inference().summary_frame()
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      X

      cate_intercept
      0.456
      0.218
      2.09
      0.037
      0.097
      0.815

```python
est.summary()
```

Coefficient Results

   	  point_estimate	 stderr	  zstat	 pvalue	 ci_lower	 ci_upper

  X0	      5.969	      0.229	 26.053	   0.0	    5.592	    6.346

  X1	      -0.06	      0.216	 -0.277	  0.782	  -0.415	    0.295

  X2	     -0.363	      0.219	  -1.66	  0.097	  -0.722	   -0.003

  X3	      0.271	      0.212	  1.28	   0.201	  -0.077	    0.62

CATE Intercept Results

         	        point_estimate	 stderr	 zstat	 pvalue	 ci_lower	 ci_upper

  cate_intercept	      0.456	      0.218	 2.09	   0.037	   0.097	    0.815

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

#### 2.5.3 Doubly Robust Average Treatment Effect Inference

For the case of `CausalForestDML`, the estimator also fits a doubly robust average treatment effect at fit time. This inference result can be accessed as follows:

```python
est3.ate_
```

    array([3.53511397])

```python
est3.ate__inference().summary_frame()
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      ATE
      3.535
      0.079
      44.671
      0.0
      3.405
      3.665

## 3. Example Usage with Multiple Continuous Treatment Synthetic Data

### 3.1. DGP
We use the data generating process (DGP) from [here](https://arxiv.org/abs/1806.03467), and modify the treatment to generate multiple treatments. The DGP is described by the following equations:

\begin{align}
T =& \langle W, \beta\rangle + \eta, & \;\eta \sim \text{Uniform}(-1, 1)\\
Y =& T\cdot \theta_{1}(X) + T^{2}\cdot \theta_{2}(X) + \langle W, \gamma\rangle + \epsilon, &\; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim& \text{Normal}(0,\, I_{n_w})\\
X \sim& \text{Uniform}(0,1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders and $\beta, \gamma$ have high sparsity.

For this DGP,
\begin{align}
\theta_{1}(x) = \exp(2\cdot x_1)\\
\theta_{2}(x) = x_1^{2}\\
\end{align}

```python
# DGP constants
np.random.seed(123)
n = 6000
n_w = 30
support_size = 5
n_x = 5
# Outcome support
support_Y = np.random.choice(np.arange(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n: np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE1 = np.array([x_i[0] for x_i in X])
TE2 = np.array([x_i[0]**2 for x_i in X]).flatten()
T = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
Y = TE1 * T + TE2 * T**2 + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)
# Generate test data
X_test = np.random.uniform(0, 1, size=(100, n_x))
X_test[:, 0] = np.linspace(0, 1, 100)
```

### 3.2. Train Estimator

```python
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor
est = LinearDML(model_y=GradientBoostingRegressor(n_estimators=100, max_depth=3, min_samples_leaf=20),
                model_t=MultiOutputRegressor(GradientBoostingRegressor(n_estimators=100,
                                                                       max_depth=3,
                                                                       min_samples_leaf=20)),
                featurizer=PolynomialFeatures(degree=2, include_bias=False),
                linear_first_stages=False,
                cv=5)
```

```python
T = T.reshape(-1,1)
est.fit(Y, np.concatenate((T, T**2), axis=1), X=X, W=W)
```

    <econml.dml.dml.LinearDML at 0x1377f9aa898>

```python
te_pred = est.const_marginal_effect(X_test)
```

```python
lb, ub = est.const_marginal_effect_interval(X_test, alpha=0.01)
```

```python
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor
est2 = CausalForestDML(model_y=GradientBoostingRegressor(n_estimators=100, max_depth=3, min_samples_leaf=20),
                       model_t=MultiOutputRegressor(GradientBoostingRegressor(n_estimators=100,
                                                                              max_depth=3,
                                                                              min_samples_leaf=20)),
                       cv=5,
                       criterion='mse', n_estimators=1000,
                       min_samples_leaf=10,
                       min_impurity_decrease=0.001,
                       random_state=123)
T = T.reshape(-1,1)
est2.tune(Y, np.concatenate((T, T**2), axis=1), X=X, W=W)
est2.fit(Y, np.concatenate((T, T**2), axis=1), X=X, W=W)
```

    <econml.dml.causal_forest.CausalForestDML at 0x137061c6cc0>

```python
te_pred2 = est2.const_marginal_effect(X_test)
```

```python
lb2, ub2 = est2.const_marginal_effect_interval(X_test, alpha=0.01)
```

### 3.3. Performance Visualization

```python
plt.figure(figsize=(15, 6))
plt.subplot(1, 2, 1)
plt.title("LinearDML with 2d-Poly Features")
plt.plot(X_test[:, 0], te_pred[:, 0], label='DML estimate1')
plt.fill_between(X_test[:, 0], lb[:, 0], ub[:, 0], alpha=.4)
plt.plot(X_test[:, 0], te_pred[:, 1], label='DML estimate2')
plt.fill_between(X_test[:, 0], lb[:, 1], ub[:, 1], alpha=.4)
expected_te1 = np.array([x_i[0] for x_i in X_test])
expected_te2=np.array([x_i[0]**2 for x_i in X_test]).flatten()
plt.plot(X_test[:, 0], expected_te1, '--', label='True effect1')
plt.plot(X_test[:, 0], expected_te2, '--', label='True effect2')
plt.ylabel("Treatment Effect")
plt.xlabel("x")
plt.legend()
plt.subplot(1, 2, 2)
plt.title("CausalForestDML")
plt.plot(X_test[:, 0], te_pred2[:, 0], label='DML estimate1')
plt.fill_between(X_test[:, 0], lb2[:, 0], ub2[:, 0], alpha=.4)
plt.plot(X_test[:, 0], te_pred2[:, 1], label='DML estimate2')
plt.fill_between(X_test[:, 0], lb2[:, 1], ub2[:, 1], alpha=.4)
expected_te1 = np.array([x_i[0] for x_i in X_test])
expected_te2=np.array([x_i[0]**2 for x_i in X_test]).flatten()
plt.plot(X_test[:, 0], expected_te1, '--', label='True effect1')
plt.plot(X_test[:, 0], expected_te2, '--', label='True effect2')
plt.ylabel("Treatment Effect")
plt.xlabel("x")
plt.legend()
plt.show()
```

### 3.4 Tree Interpreter

Interpreting heterogeneity via a tree based rule.

```python
from econml.cate_interpreter import SingleTreeCateInterpreter

intrp = SingleTreeCateInterpreter(include_model_uncertainty=True, max_depth=2)
intrp.interpret(est, X)
```

    <econml.cate_interpreter._interpreters.SingleTreeCateInterpreter at 0x13755a2a400>

```python
plt.figure(figsize=(15, 5))
intrp.plot()
plt.show()
```

## 4. Example Usage with Single Continuous Treatment Observational Data

We applied our technique to Dominick’s dataset, a popular historical dataset of store-level orange juice prices and sales provided by University of Chicago Booth School of Business.

The dataset is comprised of a large number of covariates $W$, but researchers might only be interested in learning the elasticity of demand as a function of a few variables $x$ such
as income or education.

We applied the `LinearDML` to estimate orange juice price elasticity
as a function of income, and our results, unveil the natural phenomenon that lower income consumers are more price-sensitive.

### 4.1. Data

```python
# A few more imports
import os
import pandas as pd
import urllib.request
from sklearn.preprocessing import StandardScaler
```

```python
# Import the data
file_name = "oj_large.csv"

if not os.path.isfile(file_name):
    print("Downloading file (this might take a few seconds)...")
    urllib.request.urlretrieve("https://msalicedatapublic.blob.core.windows.net/datasets/OrangeJuice/oj_large.csv", file_name)
oj_data = pd.read_csv(file_name)
```

```python
oj_data.head()
```

      store
      brand
      week
      logmove
      feat
      price
      AGE60
      EDUC
      ETHNIC
      INCOME
      HHLARGE
      WORKWOM
      HVAL150
      SSTRDIST
      SSTRVOL
      CPDIST5
      CPWVOL5

      0
      2
      tropicana
      40
      9.018695
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      1
      2
      tropicana
      46
      8.723231
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      2
      2
      tropicana
      47
      8.253228
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      3
      2
      tropicana
      48
      8.987197
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      4
      2
      tropicana
      50
      9.093357
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

```python
# Prepare data
Y = oj_data['logmove'].values
T = np.log(oj_data["price"]).values
scaler = StandardScaler()
W1 = scaler.fit_transform(oj_data[[c for c in oj_data.columns if c not in ['price', 'logmove', 'brand', 'week', 'store','INCOME']]].values)
W2 = pd.get_dummies(oj_data[['brand']]).values
W = np.concatenate([W1, W2], axis=1)
X=scaler.fit_transform(oj_data[['INCOME']].values)
```

```python
## Generate test data
min_income = -1
max_income = 1
delta = (1 - (-1)) / 100
X_test = np.arange(min_income, max_income + delta - 0.001, delta).reshape(-1,1)
```

### 4.2. Train Estimator

```python
est = LinearDML(model_y=RandomForestRegressor(),model_t=RandomForestRegressor())
est.fit(Y, T, X=X, W=W)
te_pred=est.effect(X_test)
```

### 4.3. Performance Visualization

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(10,6))
plt.plot(X_test, te_pred, label="OJ Elasticity")
plt.xlabel(r'Scale(Income)')
plt.ylabel('Orange Juice Elasticity')
plt.legend()
plt.title("Orange Juice Elasticity vs Income")
plt.show()
```

### 4.4. Confidence Intervals

We can also get confidence intervals around our predictions by passing an additional `inference` argument to `fit`.  All estimators support bootstrap intervals, which involves refitting the same estimator repeatedly on subsamples of the original data, but `LinearDML` also supports a more efficient approach which can be achieved by leaving inference set to the default of `'auto'` or by explicitly passing `inference='statsmodels'`.

```python
est.fit(Y, T, X=X, W=W)
te_pred=est.effect(X_test)
te_pred_interval = est.const_marginal_effect_interval(X_test, alpha=0.02)
```

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(10,6))
plt.plot(X_test.flatten(), te_pred, label="OJ Elasticity")
plt.fill_between(X_test.flatten(), te_pred_interval[0], te_pred_interval[1], alpha=.5, label="1-99% CI")
plt.xlabel(r'Scale(Income)')
plt.ylabel('Orange Juice Elasticity')
plt.title("Orange Juice Elasticity vs Income")
plt.legend()
plt.show()
```

## 5. Example Usage with Multiple Continuous Treatment, Multiple Outcome Observational Data

We use the same data, but in this case, we want to fit the demand of multiple brand as a function of the price of each one of them, i.e. fit the matrix of cross price elasticities. It can be done, by simply setting as $Y$ to be the vector of demands and $T$ to be the vector of prices. Then we can obtain the matrix of cross price elasticities.

\begin{align}
Y=[Logmove_{tropicana},Logmove_{minute.maid},Logmove_{dominicks}] \\
T=[Logprice_{tropicana},Logprice_{minute.maid},Logprice_{dominicks}] \\
\end{align}

### 5.1. Data

```python
# Import the data
oj_data = pd.read_csv(file_name)
```

```python
# Prepare data
oj_data['price'] = np.log(oj_data["price"])
# Transform dataset.
# For each store in each week, get a vector of logmove and a vector of logprice for each brand.
# Other features are store specific, will be the same for all brands.
groupbylist = ["store", "week", "AGE60", "EDUC", "ETHNIC", "INCOME",
               "HHLARGE", "WORKWOM", "HVAL150",
               "SSTRDIST", "SSTRVOL", "CPDIST5", "CPWVOL5"]
oj_data1 = pd.pivot_table(oj_data,index=groupbylist,
                          columns=oj_data.groupby(groupbylist).cumcount(),
                          values=['logmove', 'price'],
                          aggfunc='sum').reset_index()
oj_data1.columns = oj_data1.columns.map('{0[0]}{0[1]}'.format)
oj_data1 = oj_data1.rename(index=str,
                           columns={"logmove0": "logmove_T",
                                    "logmove1": "logmove_M",
                                    "logmove2":"logmove_D",
                                    "price0":"price_T",
                                    "price1":"price_M",
                                    "price2":"price_D"})

# Define Y,T,X,W
Y = oj_data1[['logmove_T', "logmove_M", "logmove_D"]].values
T = oj_data1[['price_T', "price_M", "price_D"]].values
scaler = StandardScaler()
W=scaler.fit_transform(oj_data1[[c for c in groupbylist if c not in ['week', 'store', 'INCOME']]].values)
X=scaler.fit_transform(oj_data1[['INCOME']].values)
```

```python
## Generate test data
min_income = -1
max_income = 1
delta = (1 - (-1)) / 100
X_test = np.arange(min_income, max_income + delta - 0.001, delta).reshape(-1, 1)
```

### 5.2. Train Estimator

```python
est = LinearDML(model_y=MultiTaskElasticNetCV(cv=3, tol=1, selection='random'),
                model_t=MultiTaskElasticNetCV(cv=3),
                featurizer=PolynomialFeatures(1),
                linear_first_stages=True)
est.fit(Y, T, X=X, W=W)
te_pred = est.const_marginal_effect(X_test)
```

### 5.3. Performance Visualization

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(18, 10))
dic={0:"Tropicana", 1:"Minute.maid", 2:"Dominicks"}
for i in range(3):
    for j in range(3):
        plt.subplot(3, 3, 3 * i + j + 1)
        plt.plot(X_test, te_pred[:, i, j],
                 color="C{}".format(str(3 * i + j)),
                 label="OJ Elasticity {} to {}".format(dic[j], dic[i]))
        plt.xlabel(r'Scale(Income)')
        plt.ylabel('Orange Juice Elasticity')
        plt.legend()
plt.suptitle("Orange Juice Elasticity vs Income", fontsize=16)
plt.show()
```

**Findings**: Look at the diagonal of the matrix, the TE of OJ prices are always negative to the sales across all the brand, but people with higher income are less price-sensitive. By contrast, for the non-diagonal of the matrix, the TE of prices for other brands are always positive to the sales for that brand, the TE is affected by income in different ways for different competitors. In addition, compare to previous plot, the negative TE of OJ prices for each brand are all larger than the TE considering all brand together, which means we would have underestimated the effect of price changes on demand.

### 5.4. Confidence Intervals

```python
te_pred_interval = est.const_marginal_effect_interval(X_test, alpha=0.02)
```

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(18, 10))
dic={0:"Tropicana", 1:"Minute.maid", 2:"Dominicks"}
for i in range(3):
    for j in range(3):
        plt.subplot(3, 3, 3 * i + j + 1)
        plt.plot(X_test, te_pred[:, i, j],
                 color="C{}".format(str(3 * i + j)),
                 label="OJ Elasticity {} to {}".format(dic[j], dic[i]))
        plt.fill_between(X_test.flatten(), te_pred_interval[0][:, i, j],te_pred_interval[1][:, i,j], color="C{}".format(str(3*i+j)),alpha=.5, label="1-99% CI")
        plt.xlabel(r'Scale(Income)')
        plt.ylabel('Orange Juice Elasticity')
        plt.legend()
plt.suptitle("Orange Juice Elasticity vs Income",fontsize=16)
plt.show()
```

notebooks/Double Machine Learning Examples.ipynb


```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import control
from ipywidgets import interact

```

# Error de Estado Estacionario

En el bucle de control, el error se calcula como la diferencia entre la referencia y la variable controlada.

Los cambios en la señal de referencia provocan errores durante los transitorios y pueden generar un error estacionario.

Considerando un sistema de control con realimentación unitaria y función de transferencia de lazo abierto $G(s) = G_c(s)G_a(s)G_p(s)$:

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^N(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)}$$

Este sistema tiene:
- $c$ ceros definidos por $T_{c1}$, $T_{c2}$, $T_{c3}$, ..., $T_{cc}$
- $N$ polos en $0$
- $p$ polos definidos por $T_{p1}$, $T_{p2}$, $T_{p3}$, ..., $T_{pp}$

La función de transferencia de lazo cerrado desde la referencia hacia la variable controlada es:

$$\frac{C(s)}{R(s)} = \frac{G(s)}{1 + G_(s)}$$

La función de transferencia de lazo cerrado desde la referencia hacia el error es:

$$\frac{E(s)}{R(s)} = \frac{R(s)-C(s)}{R(s)} = 1- \frac{C(s)}{R(s)} = 1 - \frac{G(s)}{1 + G(s)} = \frac{1}{1+G(s)}$$

El **error de estado estacionario** puede calcularse como:
$$E(s) = \int_0^\infty e(t)e^{-st}dt$$

$$E_{ee} = \lim_{t \rightarrow \infty} e(t) = \lim_{s \rightarrow 0} sE(s) = \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)}$$

## Error de estado estacionario ante una entrada escalón (posición)

Dada esta señal como referencia:

$$R(s) = \frac{1}{s}$$

y

$$E_{ee} = \lim_{s \rightarrow 0}\frac{s \frac{1}{s} }{1+G(s)} = \frac{1}{1+ \lim_{s \rightarrow 0} G(s) }$$

**Ejemplo**

Suponga:

$$G(s) = \frac{1}{5s + 1}$$

Entonces:

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{1}{5s + 1}}_{1} }$$

Si se introduce una ganancia $K_p$ (controlador proporcional):

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{K_p}{5s + 1}}_{K_p} }$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1])
G
```

$$\frac{1}{5 s + 1}$$

Análisis $E_{ee}$

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{K_p}{5s + 1}}_{K_p} } = \frac{1}{1+K_p}$$

```python
ts = np.linspace(0, 10, 2000)

Kp = [0.5,0.75,1,2,3,5,10,100]
etiq = []#lista para etiquetas
e = []#lista para errores
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(GLC, ts)
    e.append(1/(1+k))
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [1,1])
plt.legend(etiq)
plt.grid(True)
print(e)
```

¿Qué pasa si el sistema tiene una dinámica integradora?

$$G(s) = \frac{1}{s(5s + 1)}$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1,0])
G
```

$$\frac{1}{5 s^2 + s}$$

Análisis $E_{ee}$

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{1}{s(5s + 1)}}_{\infty} } = 0$$

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.5,1,1.5,2.0,3,3.1]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(GLC, ts)
    e.append(0)
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [1,1])
plt.legend(etiq)
plt.grid(True)
print(e)
```

Observe que la presencia de una dinámica integradora hace que el sistema en lazo cerrado tenga error de estado estable nulo. La cantidad de polos integradores determina el **tipo** del sistema:

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = \frac{1}{1+ K }
\end{align}

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = 0
\end{align}

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = 0
\end{align}

## Error de estado estacionario ante una entrada rampa (velocidad)
$$R(s) = \frac{1}{s^2} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{s\frac{1}{s^2}}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{1}{s+sG(s)} \\
&= \lim_{s \rightarrow 0} \frac{1}{sG(s)}
\end{align}

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \infty
\end{align}

**Ejemplo**

Suponga:

$$G(s) = \frac{1}{5s + 1}$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1])
G
```

$$\frac{1}{5 s + 1}$$

```python
ts = np.linspace(0, 100, 2000)

Kp = [0.5,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0])*GLC, ts)
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.legend(etiq)
plt.grid(True)
print(e)
```

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{p3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{1}{K}
\end{align}

```python
# Se define la función de transferencia
G = control.tf(1, [5,1,0])
G
```

$$\frac{1}{5 s^2 + s}$$

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.1,1,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0])*GLC, ts)
    e.append(1/(k))
    plt.plot(ts, ts-yLC)
    etiq.append('k = '+str(k))
# etiq.append('referencia')
# plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.legend(etiq)
plt.grid(True)
print(e)
```

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{s}{K} = 0
\end{align}

```python
## Desarrolle su propio ejemplo
G = control.tf([10,1], [5,1,0,0])
G

```

$$\frac{10 s + 1}{5 s^3 + s^2}$$

```python
rlist,klist = control.root_locus(G,plot = True)
```

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.1,1,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0])*GLC, ts)
    plt.plot(ts,yLC)
    etiq.append('k = '+str(k))
# etiq.append('referencia')
# plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.legend(etiq)
plt.grid(True)
print(e)
```

## Error de estado estacionario ante una entrada parábola (aceleración)

$$R(s) = \frac{1}{s^3} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{s\frac{1}{s^3}}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{1}{s^2+s^2G(s)} \\
&= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)}
\end{align}

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \infty
\end{align}

```python
## Desarrolle su propio ejemplo
# Se define la función de transferencia
G = control.tf(1, [5,1])
G
```

$$\frac{1}{5 s + 1}$$

```python
ts = np.linspace(0, 100, 2000)

Kp = [0.1,1,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0,0])*GLC, ts)
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot(ts, ts*ts/2,'k')
plt.legend(etiq)
plt.grid(True)
print(e)
```

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{p3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& =  \infty
\end{align}

```python
## Desarrolle su propio ejemplo

```

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{1}{K}
\end{align}

```python
## Desarrolle su propio ejemplo

```

## Tabla

|error   | $$\frac{1}{s}$$|$$\frac{1}{s^2}$$|$$\frac{1}{s^3}$$|
|--------|----------------|-----------------|-----------------|
| tipo 0 |$$\frac{1}{1+K}$$ |$$\infty$$     | $$\infty$$      |
| tipo 1 |$$0$$           |$$ \frac{1}{K}$$ | $$\infty$$      |
| tipo 2 |$$0$$           |$$ 0 $$          | $$ \frac{1}{K}$$|

## Ejemplo

Suponga un proceso con el siguiente modelo:

$$G_p(s) = \frac{2.5}{s^2+6s+5}$$

Analice el comportamiento del sistema

```python
Gp = control.tf(2.5,[1,6,5])
Gp
```

$$\frac{2.5}{s^2 + 6 s + 5}$$

```python
Gp.pole()
```

    array([-5., -1.])

```python
Gp.zero()
```

    array([], dtype=float64)

```python
rlist,klist = control.root_locus(Gp,plot = True)
```

```python
np.exp(-5)
```

    0.006737946999085467

```python
ts = np.linspace(0, 10, 2000)
e = 0.05

_, y = control.step_response(Gp, ts)
plt.plot(ts, y)
plt.plot([ts[0],ts[-1]], [y[-1],y[-1]],':k',
         [ts[0],ts[-1]], [y[-1]*(1-e),y[-1]*(1-e)],':k',
         [ts[0],ts[-1]], [y[-1]*(1+e),y[-1]*(1+e)],':k')
# etiq.append('referencia')
# plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.grid(True)

```

Observaciones:
- Sistema estable (polos con parte real negativa).
- El polo más lento está en $s=-1$, su constante de tiempo asociada es $\tau = -1/\sigma = 1s$, por lo que el tiempo de estabilización es aproximadamente $3\tau=3s$.
- El error de estado estable ante una entrada escalón es $0.5$.

A partir de las observaciones en lazo abierto, se plantean los requerimientos para el sistema controlado:

- Que el sistema en lazo cerrado reduzca el error de estado estable a $0.1$
- Que el sistema en lazo cerrado reduzca el tiempo de estabilización.

Se plantea una forma para el controlador $G_c(s)$ de manera que sea capaz de eliminar a la dinámica más lenta del proceso.
$$G_c(s) = k_c\frac{s+a}{s+b}$$

El diseño de este controlador consiste en encontrar los valores apropiados para $k_c,a$ y $b$.

Teniendo en cuenta el bucle de control

se obtiene la función de transferencia de lazo directo:

\begin{align}
G(s) &= G_c(s)G_p(s)\\
G(s) &= k_c\frac{s+a}{s+b} \frac{2.5}{s^2+6s+5}\\
G(s) &= k_c\frac{s+a}{s+b} \frac{2.5}{(s+1)(s+5)}
\end{align}

Como la dinámica más lenta del proceso es el polo en $s=-1$, puede definirse que $a=1$.

\begin{align}
G(s) &= k_c\frac{s+1}{s+b} \frac{2.5}{(s+1)(s+5)}\\
G(s) &= k_c\frac{1}{s+b} \frac{2.5}{s+5}\\
G(s) &= k_c\frac{2.5}{(s+b)(s+5)}
\end{align}

Suponga que $b=8$ para introducir una dinámica más rápida en el comportamiento.

$$G_p(s) = \frac{2.5}{(s+1)(s+5)}$$
$$G_c(s) = k_c\frac{s+1}{s+8}$$
$$G(s) = \frac{2.5k_c}{(s+8)(s+5)}$$

```python
Gc = control.tf([1,1],[1,8])
Gc
```

$$\frac{s + 1}{s + 8}$$

```python
_,_ = control.root_locus(Gp,plot = True)
```

```python
_,_ = control.root_locus(Gc*Gp,plot = True)
```

El sistema es tipo 0 por no tener polos integradores, el error de estado estacionario ante una señal escalón es:

\begin{align}
E_{ee} &= \frac{1}{1+\lim_{s \rightarrow 0}G(s)}\\
E_{ee} &= \frac{1}{1+\lim_{s \rightarrow 0}k_c\frac{5}{2(s+8)(s+5)}}\\
E_{ee} &= \frac{1}{1+k_c\frac{5}{80}}\\
E_{ee} &= \frac{80}{80+5k_c}\\
\end{align}

Se desea $E_{ee} = 0.1$

$$E_{ee} =0.1= \frac{80}{80+5k_c}$$
$$k_c= \frac{720}{5}=144$$

El controlador es:

$$G_c(s) = 144 \frac{s+1}{s+8}$$

```python
Gc = 144*control.tf([1,1],[1,8])
Gc
```

$$\frac{144 s + 144}{s + 8}$$

```python
G = Gc*Gp
G
```

$$\frac{360 s + 360}{s^3 + 14 s^2 + 53 s + 40}$$

```python
GLC = control.feedback(G,1)
GLC
```

$$\frac{360 s + 360}{s^3 + 14 s^2 + 413 s + 400}$$

```python
ts = np.linspace(0, 4, 2000)
e = 0.05

_, y = control.step_response(Gp, ts)
_, yLC = control.step_response(GLC, ts)

plt.plot(ts, y,ts,yLC)
plt.plot([ts[0],ts[-1]], [y[-1],y[-1]],':k',
         [ts[0],ts[-1]], [y[-1]*(1-e),y[-1]*(1-e)],':k',
         [ts[0],ts[-1]], [y[-1]*(1+e),y[-1]*(1+e)],':k')
plt.plot([ts[0],ts[-1]], [yLC[-1],yLC[-1]],':k',
         [ts[0],ts[-1]], [yLC[-1]*(1-e),yLC[-1]*(1-e)],':k',
         [ts[0],ts[-1]], [yLC[-1]*(1+e),yLC[-1]*(1+e)],':k')
plt.grid(True)
```

- ¿Es satisfactorio el comportamiento obtenido para el sistema en lazo cerrado?
- ¿Qué propone hacer para mejorar el comportamiento?
- ¿Qué propone hacer para eliminar el error de estado estable?

```python

```