## EM算法
与高斯判别模型相比,求解高斯混合模型的时候是不知道每条训练数据的类别标签的,在高斯混合模型中,数据的类别标签是一个隐变量.相对于求解$\theta^{MLE} = argmax_{\theta}(log(P(X|\theta)))$,求解GMM将使用EM算法:
1. 假设模型中存在隐变量Z
2. 将生成一系列$\Theta$值,$\Theta = \{ \Theta^1,\Theta^2, \dots, \Theta^t\}$

在EM算法的每一步,皆要执行:
\begin{equation}
\Theta^{(g+1)} = argmax_{\Theta}(\int_Z log(P(X,Z|\Theta))P(Z|X,\Theta^{(g)})dz)
\end{equation}

为了证明这个迭代是收敛的,也就是需要证明:
\begin{equation}
log(P(X|\Theta^{(g+1)})) = L(\Theta^{(g+1)}) \ge L(\Theta^{(g)})
\end{equation}

$L(\Theta)等于:$
\begin{align*}
L(\Theta) &= In(P(X|\Theta)) \\
          &= In(P(X|\Theta)) \\
          &= In \left(\frac{P(X,Z|\Theta)}{P(Z|X,\Theta)}\right) \\
          &= In \left(\frac{\frac{P(X,Z|\Theta)}{Q(Z)}}{\frac{P(Z|X,\Theta)}{Q(Z)}}\right) \\
          &= In \left(\frac{P(X,Z|\Theta)}{Q(Z)} \times \frac{Q(Z)}{P(Z|X,\Theta)}\right) \\
          &= In \left(\frac{P(X,Z|\Theta)}{Q(Z)}\right) + In \left(\frac{Q(Z)}{P(Z|Z,\Theta)}\right) \\
          &= \int_Z In \left(\frac{P(X,Z|\Theta)}{Q(Z)}\right)Q(Z) + KL \left( Q(Z)||P(Z|X,\Theta)\right) \\
          &= 1 + 2 \\
\end{align*}

$KL(Q(Z)||P(Z|X,\Theta))$表示概率分布$Q(Z)$和$P(Z|X,\Theta)$之间的距离,KL散度是一个$\ge 0$的值.
因此可得到：
\begin{equation}
L(\Theta) \ge \int_Z In \left(\frac{P(X,Z|\Theta)}{Q(Z)}\right)Q(Z)
\end{equation}

由Jensen不等式也可以得到:
\begin{align*}
L(\theta) &= In \left( P(X|\Theta) \right) \\
          &= In \left( \int_Z \frac{P(X,Z|\Theta)}{Q(Z)} Q(Z) \right) \\
          &\ge \int_Z In \left( \frac{P(X,Z|\Theta)}{Q(Z)} Q(Z) \right) \\
\end{align*}

### E-M算法2步流程:求两个最大化.

**Step 1:** 固定$\Theta$,令$\Theta = \Theta^{(g)}$,最大化$Q(Z)$.
为了使$KL(.) = 0$,也就是要让$L(\Theta)$等于第1项.
而当$Q(Z)=P(Z|X,\Theta^{(g)})$,$KL(.) = 0$,得到:
\begin{equation}
L(\Theta) = \int_Z In \left( \frac{P(X,Z|\Theta)}{P(Z|X,\Theta^{(g)})} \right) P(Z|X,\Theta^{(g)})
\end{equation}

**Step 2:** 固定$Q(Z)$,求参数最大化:
\begin{equation}
\Theta^{(g+1)} = argmax_{\Theta} \left( \int_Z log(P(X,Z|\Theta))P(Z|X,\Theta^{(g)}) \right)dz
\end{equation}

### E-M算法收敛性证明.
\begin{align*}
L(\Theta) &= In \left( P(X|\Theta) \right) \\
          &= \int_Z In \left( P(X,Z|\Theta)\right) P(Z|X,\Theta^{(g)})dz - \int_Z In \left( P(Z|X,\theta)\right)P(Z|X,\Theta^{(g)})dz \\
          &= F(\Theta, \Theta^{(g)}) - H(\Theta, \Theta^{(g)})\\
\end{align*}

也就是需要证明:
\begin{align*}
\Theta^{(g)} &= argmax_{\Theta} \left( \int_Z In(P(Z|X,\Theta))P(Z|X,\Theta^{(g)}) \right) \\
             &\Longrightarrow H(\Theta^{(g+1)},\Theta^{(g)}) \le H(\Theta^{(g)}, \Theta^{(g)})
\end{align*}

然后证明:
\begin{align*}
L(\Theta^{(g+1)}) &= F(\Theta^{(g+1)}, \Theta^{(g)}) - H(\Theta^{(g+1)}, \Theta^{(g)})  \\
                  &\ge F(\Theta^{(g)}, \Theta^{(g)}) - H(\Theta^{(g)}, \Theta^{(g)})  \\
                  &= L(\Theta^{(g)})
\end{align*}

**证明**$H(\Theta^{(g+1)}, \Theta^{(g)}) \le H(\Theta^{(g)}, \Theta^{(g)})$:
\begin{align*}
H(\Theta^{(g)}, \Theta^{(g)}) - H(\Theta, \Theta^{(g)}) &= \int_Z In(P(Z|X,\Theta^{(g)}))P(Z|X,\Theta^{(g)})dz - \int_Z In(P(Z|X,\Theta))P(Z|X,\Theta^{(g)})dz \\
&= \int_Z In \left( \frac{P(Z|X,\Theta^{(g)})}{P(Z|X,\Theta)} \right) P(Z|X,\Theta^{(g)}) dz \\
&= \int_Z -In \left( \frac{P(Z|X,\Theta)}{P(Z|X,\Theta^{(g)})} \right)P(Z|X,\Theta^{(g)}) dz \\
&\ge -In \left( \int_Z \frac{P(Z|X,\Theta)}{P(Z|X,\Theta^{(g)})} P(Z|X,\Theta^{(g)}) dz \right)  \\
&= 0
\end{align*}

**证明**$F(\Theta^{(g+1)}, \Theta^{(g)}) \ge F(\Theta^{(g)}, \Theta^{(g)})$:

由上可知,$H(\Theta, \Theta^{(g)})$在参数为$\Theta^{(g)}$处取得最大值,随着$\Theta$的不断迭代进化,$H(\Theta, \Theta^{(g)})$将越来越小. 因而,求$L(\Theta)$的参数的最大似然估计,就转变为求$F(\Theta, \Theta^{(g)})$的参数最大化.

## 高斯混合模型(GMM)

\begin{equation}
P(x|\Theta) = \sum_{l=1}^k \alpha_l N(x|\mu_l, \Sigma_l) \\
\sum_{l=1}^k \alpha_l = 1 \\
\Theta = \left\{ \alpha_1,\ldots,\alpha_k, \mu_1,\ldots,\mu_k, \Sigma_1,\ldots,\Sigma_k \right\}
\end{equation}

数据集$X = \{x_1,\ldots, x_m\}$,大小为m,每一条数据$x_i$对应隐变量集合$Z = \{z_1, \ldots, z_m\}$中的某个$z_i$,它指示$x_i$属于混合高斯中的那个部分的高斯分布.
$z_i \in \{1, \ldots, k\}$,总共有k个不同的分类.

**EM算法**的参数最大化过程为:
\begin{align*}
\Theta^{(g+1)} &= argmax_{\Theta} \left( F(\theta, \Theta^{(g)})\right) \\
               &= argmax_{\Theta} \left( \int_Z In(P(X,Z|\theta))P(Z|X,\Theta^{(g)}) dz\right) \\
\end{align*}

**定义**$P(X,Z|\Theta)$
\begin{equation}
P(X|\Theta) = \prod_{i=1}^m p(x_i|\Theta) = \prod_{i=1}^m \sum_{l=1}^k \alpha_l N(x_i|\mu_l, \Sigma_l) \\
P(X,Z|\Theta) = \prod_{i=1}^m p(x_i, z_i|\Theta) = \prod_{i=1}^m p(x_i|z_i,\Theta)p(z_i|\Theta)=\prod_{i=1}^m \alpha_{zi} N(x_i|\mu_{z_i}, \Sigma_{z_i})
\end{equation}

**定义**$P(Z|X,\Theta)$
\begin{equation}
P(Z|X,\Theta) = \prod_{i=1}^m p(z_i|x_i, \Theta) = \prod_{i=1}^m \frac{\alpha_{zi} N(\mu_{zi}, \Sigma_{zi})}{\sum_{l=1}^k \alpha_l N(\mu_l, \Sigma_l)}
\end{equation}

### E-step:
\begin{align*}
F(\Theta, \Theta^{(g)}) &= \int_Z In(P(X,Z|\Theta))P(Z|X,\Theta^{(g)}) dz \\
                        &= \int_{z_1} \dots \int_{z_m} \left( \sum_{i=1}^m In(p(x_i, z_i|\Theta)) \prod_{i=1}^m p(z_i|x_i, \Theta^{(g)})\right) dz_1,\dots,dz_m \\
                        &= \int_{z_1} \dots \int_{z_m} \left( In(p(x_i, z_i|\Theta)) \prod_{i=1}^m p(z_i|x_i, \Theta^{(g)})\right) \prod_{i=1}^m dz_i + \ldots \\
                        &= \int_{z_1} \left( \int_{z_2} \dots \int_{z_m} In(p(x_i, z_i|\Theta)) \prod_{i=1}^m p(z_i|x_i, \Theta^{(g)}) \prod_{i=2}^m dz_i \right) dz_1  + \ldots \\
                        &= \int_{z_1} In(p(x_i, z_i|\Theta)) \left( \int_{z_2} \dots \int_{z_m} \prod_{i=1}^m p(z_i|x_i, \Theta^{(g)}) \prod_{i=2}^m dz_i \right) dz_1 + \ldots \\
                        &= \sum_{i=1}^m \left( \int_{z_i} In(p(z_i,x_i|\Theta))p(z_i|x_i,\Theta^{(g)})dz_i \right) \\
                        &= \sum_{i=1}^m \sum_{z_i=1}^k In(p(z_i,x_i|\Theta))p(z_i|x_i,\Theta^{(g)}) \\
                        &= \sum_{z_i=1}^k \sum_{i=1}^m In(p(z_i,x_i|\Theta))p(z_i|x_i,\Theta^{(g)}) \\
                        &= \sum_{l=1}^k \sum_{i=1}^m In(\alpha_l N(x_i|\mu_l,\Sigma_l))p(l|x_i, \Theta^{(g)}) \\
                        &= \sum_{l=1}^k \sum_{i=1}^m In(\alpha_l)p(l|x_i, \Theta^{(g)}) + \sum_{l=1}^k \sum_{i=1}^m In(N(x_i|\mu_l,\Sigma_l))p(l|x_i, \Theta^{(g)}) \\
                        &= 3 + 4
\end{align*}

### M-step: maximizing $\alpha$
如前,$F(\Theta,\Theta^{(g)})$只有3式包含$\alpha$参数,对$F$求$\alpha$的偏导数,后一项等于0,因而只取3式来求$\alpha$参数.

\begin{equation}
\frac{\partial F(\Theta,\Theta^{(g)})}{\partial \alpha_1, \ldots, \partial \alpha_k} = [0, \ldots, 0 ] \ subject\ to\ \sum_{l=1}^k \alpha_l = 1
\end{equation}

使用拉格朗日乘数法:
\begin{align*}
LM(\alpha_1,\ldots,\alpha_k, \lambda) &= \sum_{l=1}^k In(\alpha_l) \left( \sum_{i=1}^m p(l|x_i, \Theta^{(g)}) \right) - \lambda \left(\sum_{l=1}^k \alpha_l -1 \right) \\
&\Longrightarrow \frac{\partial LM}{\partial \alpha_l} = \frac{1}{\alpha_l} \left( \sum_{i=1}^m p(l|x_i, \Theta^{(g)}) \right) - \lambda = 0 \\
&\Longrightarrow \alpha_l = \frac{1}{\lambda} \sum_{i=1}^m  p(l|x_i, \Theta^{(g)}) \\
&\Longrightarrow \sum_{l=1}^k \alpha_l = \frac{1}{\lambda} \sum_{l=1}^k \sum_{i=1}^m p(l|x_i, \Theta) = 1 \\
&\Longrightarrow \frac{1}{\lambda} \sum_{i=1}^m \sum_{l=1}^k  p(l|x_i, \Theta) = 1 \\
&\Longrightarrow \lambda = m \\
&\Longrightarrow \alpha_l = \frac{1}{m} \sum_{i=1}^m  p(l|x_i, \Theta^{(g)}) \\
\end{align*}

### 线性代数的一些求导公式:
**行列式X对数的导数(带行列式):**
\begin{equation}
\frac{\partial In(|X|)}{\partial X} = (X^{-1})^T
\end{equation}

**矩阵X迹的求导,$f(.)$是$F(.)$的导数:**
\begin{equation}
\frac{\partial tr(F(X))}{\partial X} = (f(X))^T
\end{equation}

**矩阵X作为迹运算因子的矩阵求导1:**
\begin{equation}
\frac{\partial tr(A X B)}{\partial X} = A^T B^T
\end{equation}

**矩阵X作为迹运算因子的矩阵求导2:**
\begin{equation}
\frac{\partial tr(X + A)^{-1}}{\partial X} = -((X+A)^{-1}(X+A)^{-1})^T
\end{equation}

**矩阵X作为迹运算因子的矩阵求导3:**
\begin{equation}
\frac{\partial tr(AX^{-1}B)}{\partial X} = -(X^{-1}BAX^{-1})^T
\end{equation}

### 关于迹的一些运算
\begin{equation}
tr(a) = a,\ a\ is\ constant \\
tr(AB) = tr(BA) \\
tr(ABC) = tr(CAB) = tr(BCA)
\end{equation}

### M-step: maximizing $\mu$
求解$\mu, \Sigma$只和4式有关,因为3式不包含参数$\mu, \Sigma$,对其求偏导数为0.

\begin{align*}
4 &= \sum_{l=1}^k \sum_{i=1}^m In(N(x_i|\mu_l,\Sigma_l))p(l|x_i, \Theta^{(g)}) \\
&= \sum_{l=1}^k \sum_{i=1}^m In\left(\frac{1}{\sqrt{(2\pi)^d|\Sigma_l|}}exp\left( -\frac{1}{2} (x_i - \mu_l)^T\Sigma^{-1}(x_i-\mu_l)\right)\right) p(l|x_i, \Theta^{(g)})\\
&\Rightarrow \sum_{l=1}^k \left( \frac{d}{2}In(2\pi) - \frac{1}{2}In(|\Sigma|) - \frac{1}{2}tr(\Sigma^{-1}YY^T) \right) p(l|x_i, \Theta^{(g)})\\
\end{align*}

这里,$Y = X-\mu_l$,而且$(x_i - \mu_l)^T\Sigma^{-1}(x_i-\mu_l)$是一个常数,常数的迹为其本身.包含$\mu_l,\Sigma_l^{-1}$的项为$S(\mu_l,\Sigma_l^{-1})$.
\begin{align*}
S(\mu_l,\Sigma_l^{-1}) &= -Tr\left( \frac{\Sigma_l^{-1}}{2} \sum_{i=1}^m (x_i - \mu_l)(x_i - \mu_l)^T p(l|x_i,\Theta^{(g)})\right) \\
\frac{\partial S(\mu_l,\Sigma_l^{-1})}{\partial \mu_l} &= \frac{\Sigma_l^{-1}}{2} \sum_{i=1}^m 2(x_i - \mu_l)p(l|x_i,\Theta^{(g)}) = 0 \\
&\Rightarrow \mu_l = \frac{\sum_{i=1}^m x_i p(l|x_i, \Theta^{(g)})}{\sum_{i=1}^m p(l|x_i, \Theta^{(g)})}
\end{align*}

### M-step: maximizing $\Sigma$
求解$\mu, \Sigma$只和4式有关,因为3式不包含参数$\mu, \Sigma$,对其求偏导数为0.
定义,$Y = X-\mu_l$,$P$为一个对角矩阵,概率分布$L$,则有:
\begin{equation}
L \equiv L(p(Y|\mu_l,\Sigma_l)) = -\frac{d \times tr(P)}{2}In(2\pi) - \frac{tr(P)}{2}In(|\Sigma_l|) - \frac{1}{2}tr(\Sigma_l^{-1}YPY^T)
\end{equation}

\begin{align*}
\frac{\partial L}{\partial \Sigma_l} &= \Sigma_l^{-1}YPY^T\Sigma_l^{-1} - tr(P)\Sigma_l^{-1} = 0 \\
&\Rightarrow  \Sigma_l^{-1}YPY^T\Sigma_l^{-1} = tr(P)\Sigma_l^{-1} \\
&\Rightarrow YPY^T\Sigma_l^{-1} = tr(P) \\
&\Rightarrow \Sigma_l^{-1} = tr(P)(YPY^T)^{-1} \\
&\Rightarrow \Sigma_l = tr(P)^{-1}(YPY^T) = \frac{(YPY^T)}{tr(P)} \\
&= \frac{\sum_{i=1}^m (x_i - \mu_l)(x_i - \mu_l)^T p(l|x_i, \Theta^{(g)})}{\sum_{i=1}^m p(l|x_i, \Theta^{(g)})}
\end{align*}

## 总结
\begin{equation}
\alpha_l^{(g+1)} = \frac{1}{m} \sum_{i=1}^m p(l|x_i, \Theta^{(g)}) \\
\mu_l^{(g+1)} = \frac{\sum_{i=1}^m x_i p(l|x_i, \Theta^{(g)})}{\sum_{i=1}^m p(l|x_i, \Theta^{(g)})} \\
\Sigma_l^{(g+1)} = \frac{\sum_{i=1}^m (x_i - \mu_l)(x_i - \mu_l)^T p(l|x_i, \Theta^{(g)})}{\sum_{i=1}^m p(l|x_i, \Theta^{(g)})} \\
p(l|x_i, \Theta^{(g)}) = \frac{ N(x_i|\mu_l, \Sigma_l)}{\sum_{s=1}^k  N(x_i|\mu_s, \Sigma_s)}
\end{equation}

```python
import numpy as np
import matplotlib.pyplot as plt

m_samples = 1000
t_samples = 400
np.random.seed(0)

mu_1 = np.array([7, 15])
sigma_1 = np.array([[3, 0], [0, 3]])
train_sample_1 = np.random.randn(m_samples, 2) + mu_1
train_sample_1 = np.dot(train_sample_1, sigma_1)
test_sample_1 = np.random.randn(t_samples, 2) + mu_1
test_sample_1 = np.dot(test_sample_1, sigma_1)
test_label_1 = np.zeros(t_samples)

mu_2 = np.array([-3, 21])
sigma_2 = np.array([[5, 0], [0.3, 2]])
train_sample_2 = np.random.randn(m_samples * 2, 2) + mu_2
train_sample_2 = np.dot(train_sample_2, sigma_2)
test_sample_2 = np.random.randn(t_samples, 2) + mu_2
test_sample_2 = np.dot(test_sample_2, sigma_2)
test_label_2 = np.ones(t_samples)

X_train = np.vstack([train_sample_1, train_sample_2])
idxs = xrange(X_train.shape[0])
idxs = np.random.choice(idxs, m_samples, replace=False)
X_train = X_train[idxs]

X_test = np.vstack([test_sample_1, test_sample_2])
X_test_label = np.concatenate([test_label_1, test_label_2])

print(X_train.shape)
print(X_test.shape)
print(X_test_label.shape)
plt.figure(1)
plt.scatter(X_train[:,0],X_train[:,1])
plt.title('X_train')
plt.figure(2)
plt.scatter(X_test[:,0], X_test[:,1])
plt.title('X_test')
```

### 参数$\alpha$
隐变量$Z=\{ z_1,\ldots,z_m\}$,$Z \sim Multinomial(\alpha)$,即Z服从于一个以$\alpha$为参数的多项式分布,这里$\alpha = \{ \alpha_1, \ldots, \alpha_k\}$,表示k个分类中的每个分类的概率,$\sum_{l=1}^k \alpha_l = 1$.而参数$\alpha$服从于一个狄利克雷分布,即$\alpha \sim Dirichlet([a_1,a_2,\ldots,a_k])$.

对于一个只有两个类的二分类问题而言,$Z=\{ z_1,z_2\}$,$Z \sim Binomial(\phi)$,这里的$\phi = \alpha$, $\phi \sim Beta(a,b)$,因此可以从一个beta分布中采样生成$\alpha$的值.

```python
def N(x, mu, sigma):
    sig_det = np.linalg.det(sigma)
    sig_inv = np.linalg.inv(sigma)
    div_val = 2 * np.pi * np.sqrt(sig_det)
    # x=[1x2]
    tx = np.dot(x - mu, sig_inv)
    tx = np.dot(tx, (x - mu).T)
    px = (np.exp(-0.5 * tx) / div_val).flatten()[0]

    return px

def trainGMM(X_train, epoch):
    np.random.seed(6)
    m,D = X_train.shape

    # 随机生成\Theta_0
    # 从beta分布采样一个二项分布的参数
    # 1类的概率
    a_1 = np.random.beta(3,4,size=1)
    a_1 = np.around(a_1, decimals=2)[0] # 只取2位小数
    # 2类的概率
    a_2 = 1 - a_1
    mu_1 = np.random.randn(1,2)
    mu_2 = np.random.randn(1,2)
    sigma_1 = np.random.randn(2,2)
    sigma_2 = np.random.randn(2,2)

    # 迭代优化
    for e in range(epoch):
        psum_1 = 0
        psum_2 = 0
        mu_sum_1 = np.zeros((1,2))
        mu_sum_2 = np.zeros((1,2))
        sig_sum_1 = np.zeros((2,2))
        sig_sum_2 = np.zeros((2,2))
        for i in range(m):
            x = X_train[i].reshape(1,2)
            # 在计算样本在各高斯分类下的概率总和时
            # 在训练过程中，加入各分类的权重影响会过早的陷入过拟合的状态
            # 即放大权重高的分类的影响,使得权重低的分类太早的收敛到0
            # 使得训练过程中出现权重低的部分的sigma过早为0,使得训练出错
            #c1 = a_1 * N(x, mu_1, sigma_1)
            #c2 = a_2 * N(x, mu_2, sigma_2)
            c1 = N(x, mu_1, sigma_1)
            c2 = N(x, mu_2, sigma_2)
            p1 = c1 / (c1 + c2)
            p2 = c2 / (c1 + c2)
            psum_1 += p1
            psum_2 += p2
            mu_sum_1 += x * p1
            mu_sum_2 += x * p2
            tx_1 = np.dot((x - mu_1).T, x - mu_1)
            tx_2 = np.dot((x - mu_2).T, x - mu_2)
            sig_sum_1 += tx_1 * p1
            sig_sum_2 += tx_2 * p2
        # 更新参数
        a_1 = psum_1 / float(m)
        a_2 = psum_2 / float(m)
        mu_1 = mu_sum_1 / psum_1
        mu_2 = mu_sum_2 / psum_2
        sigma_1 = sig_sum_1 / psum_1
        sigma_2 = sig_sum_2 / psum_2

    theta = {'a1':a_1, 'a2':a_2, 'mu1':mu_1, 'mu2':mu_2, 'sig1':sigma_1, 'sig2':sigma_2}
    return theta

theta = trainGMM(X_train, 100)
print(theta['a1'])
print(theta['a2'])
print(theta['mu1'])
print(theta['mu2'])
print(theta['sig1'])
print(theta['sig2'])
```

    0.33798722568970563
    0.6620127743102943
    [[20.94006314 45.15960195]]
    [[-8.70950345 41.99855927]]
    [[ 8.99880425 -0.21349852]
     [-0.21349852  8.37497791]]
    [[24.01209826  0.60455193]
     [ 0.60455193  3.97427246]]

```python
def prediction(X_test, theta):
    m = X_test.shape[0]
    a1 = theta['a1']
    a2 = theta['a2']
    mu1 = theta['mu1']
    mu2 = theta['mu2']
    sig1 = theta['sig1']
    sig2 = theta['sig2']

    pre_label = []
    for i in range(m):
        x = X_test[i].reshape(1,2)
        # 计算总概率的时候,需要加入各个高斯分布的权重影响
        p1 = a1 * N(x, mu1, sig1)
        p2 = a2 * N(x, mu2, sig2)
        #p1 = N(x, mu1, sig1)
        #p2 = N(x, mu2, sig2)
        if p1 > p2:
            pre_label.append(0)
        else:
            pre_label.append(1)

    return pre_label

pre_label = prediction(X_test,theta)
total_num = len(pre_label)
hit_num = X_test_label[X_test_label == pre_label].shape[0]
accuracy = float(hit_num) / float(total_num)
print('accuracy is %d %%' % (accuracy * 100))
plt.scatter(X_test[:, 0], X_test[:, 1], c=pre_label, s=40, cmap='viridis');
```

```python
from sklearn import mixture

# GMM的分类器，n_components 代表有几个高斯分量，
# covariance_type 代表协方差矩阵的类型，
# 'full'代表每个高斯分量都有自己的协方差矩阵，
# 另外比如还有'tied',代表每个分量的协方差矩阵都是相同的
gmm = mixture.GaussianMixture(n_components = 2, covariance_type = 'full')
gmm.fit(X_train)

#score = clf.score_samples(X_test)

labels = gmm.predict(X_test)
plt.scatter(X_test[:, 0], X_test[:, 1], c=labels, s=40, cmap='viridis');
```

```python
%pylab
```

    Using matplotlib backend: Qt5Agg
    Populating the interactive namespace from numpy and matplotlib


# Spam classification using logistic regression

We have 4601 email addresses with 57 features. The data can be found on [GitHub](https://github.com/probml/pmtk3/tree/master/data/spamData). The column names can be found [here](ftp://ftp.ics.uci.edu/pub/machine-learning-databases/spambase/spambase.names). We want to predict whether the email is spam or not, so we have a binary response, where 1 indicates spam and 0 indicates non-spam.

You can find some utility functions on [GitHub](https://github.com/ppham27/MLaPP-solutions/blob/master/chap08/classifiers.py).

```python
%matplotlib inline

import numpy as np
import pandas as pd
from classifiers import read_spam_data, transform_log, transform_binary
#from sklearn.linear_model import LogisticRegression # reference sklearn implementation
from classifiers import LogisticRegression

train_data, test_data = read_spam_data()
train_data.head()
```

      spam
      word_freq_make
      word_freq_address
      word_freq_all
      word_freq_3d
      word_freq_our
      word_freq_over
      word_freq_remove
      word_freq_internet
      word_freq_order
      ...
      word_freq_conference
      char_freq_;
      char_freq_(
      char_freq_[
      char_freq_!
      char_freq_$
      char_freq_#
      capital_run_length_average
      capital_run_length_longest
      capital_run_length_total

      0
      1
      0.21
      0.28
      0.5
      0.0
      0.14
      0.28
      0.21
      0.07
      0.00
      ...
      0.0
      0.0
      0.132
      0.0
      0.372
      0.180
      0.048
      5.114
      101.0
      1028.0

      1
      1
      0.00
      0.00
      0.0
      0.0
      0.63
      0.00
      0.31
      0.63
      0.31
      ...
      0.0
      0.0
      0.137
      0.0
      0.137
      0.000
      0.000
      3.537
      40.0
      191.0

      2
      1
      0.00
      0.00
      0.0
      0.0
      0.63
      0.00
      0.31
      0.63
      0.31
      ...
      0.0
      0.0
      0.135
      0.0
      0.135
      0.000
      0.000
      3.537
      40.0
      191.0

      3
      1
      0.00
      0.00
      0.0
      0.0
      1.85
      0.00
      0.00
      1.85
      0.00
      ...
      0.0
      0.0
      0.223
      0.0
      0.000
      0.000
      0.000
      3.000
      15.0
      54.0

      4
      1
      0.00
      0.00
      0.0
      0.0
      1.92
      0.00
      0.00
      0.00
      0.00
      ...
      0.0
      0.0
      0.054
      0.0
      0.164
      0.054
      0.000
      1.671
      4.0
      112.0

5 rows × 58 columns

In logistic regression, we let

\begin{equation}
y \mid \mathbf{x} \sim \operatorname{Bernoulli}\left(\frac{1}{1 + \exp\left(-\left(w_0 + \mathbf{w}^\intercal \mathbf{x}\right)\right)}\right),
\end{equation}

so if we make $N$ observations the likelihood is

\begin{equation}
L\left(w_0, \mathbf{w}\right) = \prod_{i=1}^N\left(\frac{1}{1 + \exp\left(-\left(w_0 + \mathbf{w}^\intercal \mathbf{x}\right)\right)}\right)^{y_i}\left(\frac{1}{1 + \exp\left(w_0 + \mathbf{w}^\intercal \mathbf{x}\right)}\right)^{1-y_i}.
\end{equation}

Define $\tilde{y}_i = 2y_i - 1.$ We want to minimize the negative log-likelihood, which is

\begin{equation}
J\left(w_0, \mathbf{w}\right) = \sum_{i=1}^N\log\left(1 + \exp\left(-\tilde{y}_i\left(w_0 + \mathbf{w}^\intercal \mathbf{x}\right)\right)\right).
\end{equation}

In practice, this function is not always convex, so we add some regularization. Now, we minimize

\begin{equation}
J_\lambda\left(w_0, \mathbf{w}\right) = \sum_{i=1}^N\log\left(1 + \exp\left(-\tilde{y}_i\left(w_0 + \mathbf{w}^\intercal \mathbf{x}\right)\right)\right) + \frac{\lambda}{2}\mathbf{w}^\intercal\mathbf{w}.
\end{equation}

Note that there is no penalty for $w_0$. One way to minimize this is with Newton's algorithm. To do so, we need to calculate the gradient $\mathbf{g}$ and Hessian $\mathbf{H}$. In exercise 3 (not shown), I show that

\begin{align}
\mathbf{g}\left(w_0, \mathbf{w}\right) &= \tilde{\mathbf{X}}^\intercal\left(\boldsymbol\mu - \mathbf{y}\right) + \lambda\begin{pmatrix}0 \\ \mathbf{w}\end{pmatrix} \\
\mathbf{H}\left(w_0, \mathbf{w}\right) &= \tilde{\mathbf{X}}^\intercal \mathbf{M} \tilde{\mathbf{X}}
+ \lambda
\begin{pmatrix}
0 & \mathbf{0} \\
\mathbf{0} & I
\end{pmatrix},
\end{align}

where $\tilde{\mathbf{X}}$ is the data matrix with a column of 1s inserted as the first column. Also, we have defined

\begin{equation}
\boldsymbol\mu = \begin{pmatrix}
\left(1 + \exp\left(-\left(w_0 + \mathbf{w}^\intercal\mathbf{x}_1\right)\right)\right)^{-1}\\
\left(1 + \exp\left(-\left(w_0 + \mathbf{w}^\intercal\mathbf{x}_2\right)\right)\right)^{-1}\\
\vdots \\
\left(1 + \exp\left(-\left(w_0 + \mathbf{w}^\intercal\mathbf{x}_N\right)\right)\right)^{-1}\\
\end{pmatrix},
~\text{and}~
\mathbf{M} = \operatorname{diag}\left(
\mu_1(1-\mu_1), \mu_2(1-\mu_2), \ldots,\mu_N(1-\mu_N)
\right).
\end{equation}

Let $\boldsymbol\theta = (w_0,\mathbf{w})^\intercal$. Choose some initial estimate for $\boldsymbol\theta_0$. Then, by Newton's method,

\begin{equation}
\mathbf{H}\left(\boldsymbol\theta_k\right)\left(\boldsymbol\theta_{k+1} - \boldsymbol\theta_{k}\right) = -\mathbf{g}\left(\boldsymbol\theta_k\right).
\end{equation}

```python
#logReg = LogisticRegression(solver='newton-cg', C=1) # sklearn implementation, only newton-cg converges
logReg = LogisticRegression(regularization = 1)
logReg.fit(train_data.drop('spam', axis = 1).as_matrix(), train_data['spam'].as_matrix())
print(logReg.intercept_)
print(logReg.coef_)
print(1-logReg.score(train_data.drop('spam', axis = 1).as_matrix(), train_data['spam'].as_matrix()))
print(1-logReg.score(test_data.drop('spam', axis = 1).as_matrix(), test_data['spam'].as_matrix()))
```

    -1.49981367068
    [ -1.26600248e-01  -1.56094390e-01   2.54708222e-01   8.58581226e-01
       4.09630906e-01   8.96135831e-01   2.19302802e+00   7.60903407e-01
       6.71931927e-01   1.01103996e-01  -2.82796444e-01  -1.83924290e-01
      -1.11320045e-01   1.03485121e-01   5.54364725e-01   1.18430618e+00
       1.06659155e+00   1.31293171e-01   1.43401894e-01   7.11353892e-01
       2.31623776e-01   3.36640589e-01   1.87194517e+00   6.08773300e-01
      -1.82320650e+00  -5.14474564e-01  -4.14759176e+00  -1.98631296e-01
      -1.18020809e+00  -1.99879005e-01  -1.77496082e-01   1.48991710e-02
      -7.45306456e-01  -1.68448134e-01  -5.55552499e-01   8.47931331e-01
       3.72525690e-02  -6.12885760e-01  -1.00486600e+00  -1.36561188e-01
      -1.16561300e+00  -1.59350006e+00  -9.86968673e-01  -1.38680860e+00
      -7.55505052e-01  -1.82108130e+00  -6.55434252e-01  -1.61610798e+00
      -1.18107595e+00  -3.44812567e-01  -3.76344925e-01   2.25717319e-01
       4.05625095e+00   1.06168683e+00  -1.38267945e-02   8.02586523e-03
       4.62754722e-04]
    0.0698205546493
    0.0703125

The default settings work pretty well and give us a 7% misclassification rate. Let us try various transformations.

- Standardize, that is, $\displaystyle\frac{\mathbf{x}_j - \hat{\mu}_j}{\hat{\sigma}_j}$
- log, $\log\left(\mathbf{x}_j + 0.1\right)$
- Binarize, $\mathbb{I}\left(\mathbf{x}_j > 0\right)$

You can see the transformations and the logistic regression class that I wrote by hand on [GitHub](https://github.com/ppham27/MLaPP-solutions/blob/master/chap08/classifiers.py).

```python
from sklearn import preprocessing

# transform the data
ytrain = train_data['spam'].as_matrix()
ytest = test_data['spam'].as_matrix()
Xtrain_raw = train_data.drop('spam', axis = 1).as_matrix()
Xtest_raw = test_data.drop('spam', axis = 1).as_matrix()
Xtrain_standard = preprocessing.scale(Xtrain_raw, axis=0)
Xtest_standard = preprocessing.scale(Xtest_raw, axis=0)
Xtrain_log = np.apply_along_axis(transform_log, axis = 0, arr=Xtrain_raw)
Xtest_log = np.apply_along_axis(transform_log, axis = 0, arr=Xtest_raw)
Xtrain_binary = np.apply_along_axis(transform_binary, axis = 0, arr=Xtrain_raw)
Xtest_binary = np.apply_along_axis(transform_binary, axis = 0, arr=Xtest_raw)

data_transform = ['Raw', 'Standard', 'Log', 'Binary']
Xtrain = [Xtrain_raw, Xtrain_standard, Xtrain_log, Xtrain_binary]
Xtest = [Xtest_raw, Xtest_standard, Xtest_log, Xtest_binary]

## now run lots of models to find regularization parameter
regularization = np.linspace(0, 20, num=41)
misclassification_rates = pd.DataFrame(dtype=np.float64,
                                       index = np.arange(len(regularization)),
                                       columns = ['Regularization'] +
                                       list(map(lambda x : x + ' Train',  data_transform)) +
                                       list(map(lambda x : x + ' Test',  data_transform)))
for i in range(len(regularization)):
    misclassification_rates.iloc[i]['Regularization'] = regularization[i]
    if regularization[i] == 0:
        regularization[i] += 0.01 # hack when there's no convergence.
    logReg = LogisticRegression(regularization = regularization[i])
    for j in range(len(data_transform)):
        logReg.fit(Xtrain[j], ytrain)
        misclassification_rates.iloc[i][data_transform[j] + ' Train'] = 1 - logReg.score(Xtrain[j], ytrain)
        misclassification_rates.iloc[i][data_transform[j] + ' Test'] = 1 - logReg.score(Xtest[j], ytest)
misclassification_rates
```

      Regularization
      Raw Train
      Standard Train
      Log Train
      Binary Train
      Raw Test
      Standard Test
      Log Test
      Binary Test

      0
      0.0
      0.070799
      0.070147
      0.051876
      0.064600
      0.076172
      0.086589
      0.058594
      0.074219

      1
      0.5
      0.069494
      0.072431
      0.051223
      0.064274
      0.070964
      0.087891
      0.059245
      0.072266

      2
      1.0
      0.069821
      0.074062
      0.050571
      0.063295
      0.070312
      0.087240
      0.059896
      0.072266

      3
      1.5
      0.071126
      0.075367
      0.049918
      0.063295
      0.070312
      0.085938
      0.060547
      0.072917

      4
      2.0
      0.072757
      0.076020
      0.050245
      0.063295
      0.069661
      0.084635
      0.061198
      0.073568

      5
      2.5
      0.073409
      0.075367
      0.050571
      0.063295
      0.071615
      0.085286
      0.059896
      0.074219

      6
      3.0
      0.072757
      0.075693
      0.051876
      0.063948
      0.072266
      0.084635
      0.059896
      0.074219

      7
      3.5
      0.074715
      0.075367
      0.051223
      0.063295
      0.075521
      0.085286
      0.059896
      0.074219

      8
      4.0
      0.075693
      0.076020
      0.051223
      0.063622
      0.075521
      0.083984
      0.059896
      0.074219

      9
      4.5
      0.076020
      0.076020
      0.051223
      0.063295
      0.076823
      0.084635
      0.059896
      0.074219

      10
      5.0
      0.075041
      0.076020
      0.050897
      0.063622
      0.076823
      0.085286
      0.059896
      0.074870

      11
      5.5
      0.074062
      0.076346
      0.050897
      0.063622
      0.077474
      0.085938
      0.059896
      0.075521

      12
      6.0
      0.074388
      0.076998
      0.050571
      0.063948
      0.076823
      0.085286
      0.059896
      0.076172

      13
      6.5
      0.074388
      0.076998
      0.050245
      0.064600
      0.075521
      0.083984
      0.058594
      0.076172

      14
      7.0
      0.074388
      0.077651
      0.050245
      0.064600
      0.076172
      0.084635
      0.058594
      0.076172

      15
      7.5
      0.074388
      0.078303
      0.050897
      0.064600
      0.076172
      0.084635
      0.057943
      0.076172

      16
      8.0
      0.074388
      0.078303
      0.051550
      0.064927
      0.076823
      0.083984
      0.057943
      0.076172

      17
      8.5
      0.074715
      0.078630
      0.051550
      0.064927
      0.077474
      0.083984
      0.059245
      0.075521

      18
      9.0
      0.075367
      0.078630
      0.051876
      0.064600
      0.076823
      0.083984
      0.059245
      0.075521

      19
      9.5
      0.074715
      0.079282
      0.051876
      0.065253
      0.076823
      0.083984
      0.059245
      0.075521

      20
      10.0
      0.074715
      0.078956
      0.051876
      0.064600
      0.076823
      0.085286
      0.059245
      0.075521

      21
      10.5
      0.074715
      0.078956
      0.052202
      0.064600
      0.076823
      0.084635
      0.059245
      0.075521

      22
      11.0
      0.074388
      0.079282
      0.051550
      0.065253
      0.077474
      0.084635
      0.059245
      0.074219

      23
      11.5
      0.075367
      0.079282
      0.051223
      0.066232
      0.077474
      0.084635
      0.059245
      0.074870

      24
      12.0
      0.075367
      0.079608
      0.051550
      0.069168
      0.077474
      0.084635
      0.059245
      0.075521

      25
      12.5
      0.075693
      0.079608
      0.051550
      0.069821
      0.077474
      0.085286
      0.059245
      0.074219

      26
      13.0
      0.075693
      0.079608
      0.051550
      0.070147
      0.076823
      0.085286
      0.059245
      0.074219

      27
      13.5
      0.075693
      0.079608
      0.051550
      0.070147
      0.076823
      0.085286
      0.059896
      0.074219

      28
      14.0
      0.076020
      0.080587
      0.052202
      0.070147
      0.076823
      0.085286
      0.059896
      0.075521

      29
      14.5
      0.076346
      0.080587
      0.052202
      0.069821
      0.077474
      0.085286
      0.060547
      0.075521

      30
      15.0
      0.076346
      0.080587
      0.051876
      0.069821
      0.079427
      0.085286
      0.060547
      0.075521

      31
      15.5
      0.076346
      0.080914
      0.051876
      0.070473
      0.079427
      0.085286
      0.060547
      0.078125

      32
      16.0
      0.076346
      0.080914
      0.052202
      0.071778
      0.080078
      0.085286
      0.061849
      0.078776

      33
      16.5
      0.076346
      0.081240
      0.052202
      0.071778
      0.080078
      0.085286
      0.061849
      0.078776

      34
      17.0
      0.077325
      0.080914
      0.051876
      0.071452
      0.080078
      0.085938
      0.061849
      0.078776

      35
      17.5
      0.077651
      0.080914
      0.051876
      0.071452
      0.080729
      0.085938
      0.062500
      0.078776

      36
      18.0
      0.077651
      0.080914
      0.051550
      0.071452
      0.080729
      0.085938
      0.062500
      0.078776

      37
      18.5
      0.077651
      0.081240
      0.051550
      0.071452
      0.080729
      0.085938
      0.062500
      0.079427

      38
      19.0
      0.077651
      0.081566
      0.051876
      0.071452
      0.081380
      0.086589
      0.062500
      0.079427

      39
      19.5
      0.077651
      0.081566
      0.052529
      0.071778
      0.081380
      0.086589
      0.062500
      0.080078

      40
      20.0
      0.077651
      0.081566
      0.052529
      0.072431
      0.081380
      0.086589
      0.062500
      0.080078

Let's plot these results.

```python
import matplotlib.pyplot as plt
colors = ['#e41a1c', '#377eb8', '#4daf4a', '#984ea3']
plt.figure(figsize=(12,8))
for i in range(len(data_transform)):
    plt.plot(misclassification_rates['Regularization'], misclassification_rates[data_transform[i] + ' Train'],
             color=colors[i], linestyle='--', linewidth=2, marker='.')
    plt.plot(misclassification_rates['Regularization'], misclassification_rates[data_transform[i] + ' Test'],
             color=colors[i], linestyle='-', linewidth=2, marker='.')
plt.grid(True)
plt.legend(loc='center left', bbox_to_anchor=(1,0.5), title="Dataset")
plt.ylabel("Misclassification Rate")
plt.xlabel("Regularization Parameter ($\lambda$)")
plt.title("Misclassification Rates for Various Transforms and Regularizations")
plt.show()
```

The problem tells us to select the regularization paramter with cross validation, but that seems pointless. We see that the log transform performs best with a misclassification rate of 5.8% when $\lambda = 8$.


# Circuitos CA

Jupyter Notebook desenvolvido por [Gustavo S.S.](https://github.com/GSimas)

## Senoides e Fasores

A análise em Corrente Alternada (CA) é a análise de circuitos nos quais a fonte de tensão ou
de corrente varia com o tempo. Os circuitos acionados
por fontes de tensão ou de corrente senoidais são chamados circuitos CA.

**Senoide é um sinal que possui a forma da função seno ou cosseno.**

Consideremos a tensão senoidal:

\begin{align}
{\Large v(t) = V_m sen(\omega t)}
\end{align}

onde

Vm = amplitude da senoide (V)

ω = frequência angular (rad/s)

ωt = argumento da senoide (rad)

\begin{align}
{\Large T = \frac{2 \pi}{\omega}}
\end{align}

A senoide é mostrada na Figura 9.1a em função de seu argumento e na Figura
9.1b em função do tempo.

O fato de v(t) repetir-se a cada T segundos é demonstrado substituindo-se t por
t + T. Portanto:

\begin{align}
{\Large v(t + T) = v(t)}
\end{align}

Função periódica é aquela que satisfaz f(t) = f(t + nT), para todo t e para
todos os inteiros n.

Conforme mencionado anteriormente, o período T da função periódica é o
tempo de um ciclo completo ou o número de segundos por ciclo. O inverso
desse valor é o número de ciclos por segundo, conhecido como frequência
cíclica f da senoide. Consequentemente,

\begin{align}
{\Large f = \frac{1}{T}}
\\{\Large \omega = 2 \pi f}
\end{align}

Uma expressão mais genérica para a senoide:

\begin{align}
{\Large v(t) = V_m sen(\omega t + \phi)}
\end{align}

onde ϕ é a fase.

Consideremos duas senoides, sendo:

\begin{align}
{\Large v_1(t) = V_m sen(\omega t)}
\\{\Large v_2(t) = V_m sen(\omega t + \phi)}
\end{align}

Uma senoide pode ser expressa em termos de seno ou de cosseno. Isso pode ser conseguido
usando-se as seguintes identidades trigonométricas:

\begin{align}
{\Large sen(A \pm B) = sen(A)cos(B) \pm sen(B)cos(A)}
\\{\Large cos(A \pm B) = cos(A)cos(B) \mp sen(A)sen(B)}
\end{align}

Com essas identidades, fica fácil demonstrar que:

\begin{align}
{\Large sen(\omega t \pm 180º) = -sen(\omega t)}
\\{\Large cos(\omega t \pm 180º) = -cos(\omega t)}
\\{\Large sen(\omega t \pm 90º) = \pm cos(\omega t)}
\\{\Large cos(\omega t \pm 90º) = \mp sen(\omega t)}
\end{align}

Usando essas relações, podemos transformar uma senoide na forma de seno
para uma na forma de cosseno, ou vice-versa.

A magnitude e o argumento da senoide resultante na forma de cosseno
são imediatamente obtidos do triângulo. Portanto:

\begin{align}
{\Large A cos(\omega t) + B sen(\omega t) = r cos(\omega t - \theta)}
\\{\Large r = \sqrt{A^2 + B^2}}
\\{\Large \theta = arctg \frac{B}{A}}
\end{align}

**Exemplo 9.1**

Determine a amplitude, a fase, o período e a frequência da senoide

v(t) 12 cos(50 t + 10º)

```python
print("Exemplo 9.1")

import numpy as np

Vm = 12
phi = 10
omega = 50
T = 2*np.pi/omega
f = 1/T

print("Amplitude:",Vm,"V")
print("Fase:",phi,"º")
print("Frequência angular:",omega,"rad/s")
print("Período:",T,"s")
print("Frequência:",f,"Hz")
```

    Exemplo 9.1
    Amplitude: 12 V
    Fase: 10 º
    Frequência angular: 50 rad/s
    Período: 0.12566370614359174 s
    Frequência: 7.957747154594767 Hz

**Problema Prático 9.1**

Dada a senoide 30 sen(4pit – 75°), calcule sua amplitude, fase, frequência angular, período e frequência

```python
print("Problema Prático 9.1")

Vm = 30
#30sin(4*pi*t - 75º) = 30cos(4*pi*t + 165º)
phi = -75
omega = 4*np.pi
T = 2*np.pi/omega
f = 1/T

print("Amplitude:",Vm,"V")
print("Fase:",phi,"º")
print("Frequência angular:",omega,"rad/s")
print("Período:",T,"s")
print("Frequência:",f,"Hz")
```

    Problema Prático 9.1
    Amplitude: 30 V
    Fase: -75 º
    Frequência angular: 12.566370614359172 rad/s
    Período: 0.5 s
    Frequência: 2.0 Hz

**Exemplo 9.2**

Calcule o ângulo de fase entre v1 = –10 cos(wt + 50°) e v2 = 12 sen(wt – 10°). Indique
qual senoide está avançada.

```python
print("Exemplo 9.2")

#v1 = -10cos(wt + 50º) = 10cos(wt + 50 - 180) = 10cos(wt - 130º)
#v2 = 12sen(wt - 10º) = 12cos(wt - 100º)
#-130 - (-100) = -30

phi = 30

print("v2 esta avancada em {}º em relação a v1".format(phi))
```

    Exemplo 9.2
    v2 esta avancada em 30º em relação a v1

**Problema Prático 9.2**

Determine o ângulo de fase entre

i1(t) = -4sen(377t + 55º)

e

i2(t) =  5cos(377t - 65º)

```python
print("Problema Prático 9.2")

#i1 = -4sen(377t + 55) = 4sen(377t + 55 + 180) = 4sen(377t + 235) = 4cos(377t + 145)
#i2 = 5cos(377t - 65)
phi = 145 - (-65)

print("i1 esta avancada em {}º em relação a i2".format(phi))
```

    Problema Prático 9.2
    i1 esta avancada em 210º em relação a i2

# Fasores

**Fasor é um número complexo que representa a amplitude e a fase de uma
senoide**

Os fasores se constituem de maneira simples para analisar circuitos lineares
excitados por fontes senoidais; encontrar a solução para circuitos desse tipo
seria impraticável de outro modo. A noção de resolução de circuitos CA usando
fasores foi introduzida inicialmente por Charles Steinmetz em 1893.

Um número complexo z pode ser escrito na forma retangular como

\begin{align}
{\Large z = x + jy}
\\{\Large j = \sqrt{-1}}
\end{align}

O número complexo z também pode ser escrito na forma polar ou exponencial:

\begin{align}
{\Large z = r \angle \phi = re^{j \phi}}
\\{\Large r = \sqrt{x^2 + y^2}}
\\{\Large \phi = arctg(\frac{y}{x})}
\end{align}

Por outro lado, se conhecermos r e f, podemos obter x e y como:

\begin{align}
{\Large x = rcos(\phi)}
\\{\Large y = rsen(\phi)}
\end{align}

## Operações com Complexos

As seguintes operações são importantes.

**Adição**

\begin{align}
{\Large z_1 + z_2 = (x_1 + x_2) + j(y_1 + y_2)}
\end{align}

**Subtração**

\begin{align}
{\Large z_1 + z_2 = (x_1 - x_2) + j(y_1 - y_2)}
\end{align}

**Multiplicação**

\begin{align}
{\Large z_1 z_2 = r_1 r_2 \angle \phi_1 + \phi_2}
\end{align}

**Divisão**

\begin{align}
{\Large \frac{z_1}{z_2} = \frac{r_1}{r_2} \angle \phi_1 - \phi_2}
\end{align}

**Inverso**

\begin{align}
{\Large \frac{1}{z} = \frac{1}{r} \angle - \phi}
\end{align}

**Raiz Quadrada**

\begin{align}
{\Large \sqrt{z} = \sqrt{r} \angle \phi / 2}
\end{align}

**Conjugado Complexo**

\begin{align}
{\Large z* = x - jy = r \angle -\phi = re^{-j \phi}}
\end{align}

A ideia da representação de fasor se baseia na identidade de Euler. Em
geral:

\begin{align}
{\Large e^{\pm j \phi} = cos(\phi) \pm jsen(\phi)}
\end{align}

Assim, podemos escrever:

\begin{align}
{\Large cos(\phi) = Re(e^{j \phi})}
\\{\Large sen(\phi) = Im(e^{j \phi})}
\end{align}

Dada a senoide v(t) = Vm cos(vt + Φ), podemos representar como:

\begin{align}
{\Large v(t) = Re(V e^{j \omega t})}
\\{\Large V = V_m e^{j \phi} = V_m \angle \phi}
\end{align}

**V é, portanto, a representação fasorial da senoide v(t)**

# Derivada e Integral de Fasores

A derivada de v(t) é transformada para o domínio dos fasores
como jwV:

\begin{align}
{\Large \frac{dv}{dt} = - \omega V_m sen(\omega t + \phi) = \omega V_m cos(\omega t + \phi + 90º)}
\\{\Large = Re(\omega V_m e^{j \omega t} e^{j \omega} e^{j 90º} = Re(j \omega V e^{j \omega t})}
\end{align}

Assim:

\begin{align}
{\Large \frac{dv}{dt} = j\omega V}
\\{\Large \int v dt = \frac{V}{j\omega}}
\end{align}

As diferenças entre v(t) e V devem ser enfatizadas:

1. v(t) é a representação instantânea ou no domínio do tempo, enquanto **V** é a representação em termos de frequência ou no domínio dos fasores.

2. v(t) é dependente do tempo, enquanto V não é.

3. v(t) é sempre real sem nenhum termo complexo, enquanto **V** geralmente é complexo.

**Exemplo 9.4**

Transforme as senoides seguintes em fasores:

(a) i = 6cos(50t - 40º) A

(b) v = -4sen(30t + 50º) V

```python
print("Exemplo 9.4")

#6cos(50t - 40)
#r = 6
#phi = -40

#-4sen(30t + 50) = 4sen(30t + 50 + 180) = 4cos(30t + 140)
#r = 4
#phi = 140

print("I: 6[-40º]")
print("V: 4[140º]")
```

    Exemplo 9.4
    I: 6[-40º]
    V: 4[140º]

**Problema Prático 9.4**

Expresse as senoides seguintes na forma de fasores:

(a) v = 7cos(2t + 40º) V

(b) i = -4sen(10t + 10º) A

```python
print("Problema Prático 9.4")

#7cos(2t + 40)
#r = 7
#phi = 40

#-4sen(10t + 10) = 4sen(10t + 10 + 180) = 4cos(10t + 100)
#r = 4
#phi = 100

print("V: 7[40º]")
print("I: 4[100º]")
```

    Problema Prático 9.4
    V: 7[40º]
    I: 4[100º]

**Exemplo 9.5**

Determine as senoides representadas pelos fasores seguintes:

(a) I = -3 + j4

(b) V = j8e^(-j20º)

```python
print("Exemplo 9.5")

import numpy as np

r = np.sqrt((-3)**2 + 4**2)
phi = np.arctan(4/(-3))*180/np.pi + 180

print("I: {}[{}º]".format(r,phi))

#j = 1[90º]
#V = 8e^(-j20) = 8[-20º]
#jV = 1*8 [90 -20] = 8[70º]

print("V: 8[70º]")

```

    Exemplo 9.5
    I: 5.0[126.86989764584402º]
    V: 8[70º]

**Problema Prático 9.5**

Determine as senoides correspondentes aos fasores seguintes:

(a) V = -25[40º]

(b) I = j(12 - j5)

```python
print("Problema Prático 9.5")

print("v(t) = 25cos(wt + 220)")

#j(12 - j5) = 5 + 12j

r = np.sqrt(5**2 + 12**2)
phi = np.arctan(12/5)*180/np.pi

print("I: {}[{}º]".format(r,phi))
```

    Problema Prático 9.5
    v(t) = 25cos(wt + 220)
    I: 13.0[67.38013505195958º]

**Exemplo 9.6**

Dados i1(t) = 4 cos(wt + 30°) A e i2(t) = 5 sen(wt + 20°) A, determine sua soma.

```python
print("Exemplo 9.6")

#4cos(wt + 30) = 4[30]
#5sen(wt + 20) = 5cos(wt + 70) = 5[-110]

x = 4*np.cos(30*np.pi/180) + 5*np.cos(-110*np.pi/180)
y = 4*np.sin(30*np.pi/180) + 5*np.sin(-110*np.pi/180)

print("i1 + i2: {} + j{}".format(x,y))

r = np.sqrt(x**2 + y**2)
phi = np.arctan(y/x)*180/np.pi

print("I: {}[{}]".format(r,phi))
print("i(t): {}cos(wt + {})".format(r,phi))
```

    Exemplo 9.6
    i1 + i2: 1.7540008985094113 + j-2.6984631039295426
    I: 3.218419219934048[-56.976134221108644]
    i(t): 3.218419219934048cos(wt + -56.976134221108644)

**Problema Prático 9.6**

Se v1(t) = –10 sen(vt – 30°) V e v2(t) = 20 cos(vt + 45°), determine v = v1 + v2.

```python
print("Problema Prático 9.6")

#-10sen(wt - 30) = 10sen(wt + 150) = 10sen(wt + 60) = 10[60]
#20cos(wt + 45) = 20[45]

x = 10*np.cos(60*np.pi/180) + 20*np.cos(45*np.pi/180)
y = 10*np.sin(60*np.pi/180) + 20*np.sin(45*np.pi/180)

print("v1 + v2: {} + j{}".format(x,y))

r = np.sqrt(x**2 + y**2)
phi = np.arctan(y/x)*180/np.pi

print("V: {}[{}]".format(r,phi))
print("v(t): {}cos(wt + {})".format(r,phi))
```

    Problema Prático 9.6
    v1 + v2: 19.14213562373095 + j22.802389661575337
    V: 29.771972230868872[49.98723472563248]
    v(t): 29.771972230868872cos(wt + 49.98723472563248)

**Exemplo 9.7**

Usando o método de fasores, determine a corrente i(t) em um circuito descrito pela
equação diferencial

\begin{align}
{\Large 4i + 8 \int idt - 3 \frac{di}{dt} = 50cos(2t + 75)}
\end{align}

```python
print("Exemplo 9.7")

#4I + 8I/jw - 3jwI = 50[75]
#4I -4jI - 6jI = 50[75]
#I = 50[75] / (4 - j10)

r = np.sqrt(4**2 + (-10)**2)
phi = np.arctan((-10)/4)*180/np.pi

R = 50/r
Phi = 75 - phi

print("Fasor I: {}[{}]".format(R,Phi))
print("i(t) = {}cos(wt + {}º)".format(R,Phi))
```

    Exemplo 9.7
    Fasor I: 4.642383454426297[143.19859051364818]
    i(t) = 4.642383454426297cos(wt + 143.19859051364818º)

**Problema prático 9.7**

Determine a tensão v(t) em um circuito descrito pela equação integro-diferencial a seguir:

\begin{align}
{\Large 2\frac{dv}{dt} + 5v + 10 \int vdt = 50cos(5t - 30º)}
\end{align}

```python
print("Problema Prático 9.7")

#2Vjw + 5V + 10v/jw = 50[-30]
#5V -2jV + 10jV = 50[-30]
#V = 50[-30] / (5 + j8)

r = np.sqrt(5**2 + 8**2)
phi = np.arctan(8/5)*180/np.pi

R = 50/r
Phi = -30 - phi

print("Fasor I: {}[{}]".format(R,Phi))
print("i(t) = {}cos(wt + {}º)".format(R,Phi))
```

    Problema Prático 9.7
    Fasor I: 5.299989400031801[-87.9946167919165]
    i(t) = 5.299989400031801cos(wt + -87.9946167919165º)


# Exercise 11) Eligibility Traces

In this exercise we will explore whether we can enhance the learning behavior within the MountainCar environment when using eligibility traces. The basic idea behind eligibility traces is that - much like in n-step learning - past actions contributed to the current situation. Contrary to n-step learning, however, intuition tells us that more recent decisions had a more severe impact on the present situation than decisions that were made a long time ago. Thus, it may be helpful to integrate a forgetting factor $\lambda$ which decreases the assumed influence of actions over time.

Once again we will be looking at the MountainCar with discrete action space.

```python
import numpy as np
import gym
import random
import time
from tqdm.notebook import tqdm
import matplotlib.pyplot as plt
plt.style.use('seaborn-talk')
from mpl_toolkits.mplot3d import Axes3D
from collections import deque
from sklearn.kernel_approximation import RBFSampler
import sklearn.pipeline
import sklearn.preprocessing

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import SGD
```

## 0) Preprocessing Gradients

For many applications we are not satisfied with directly using the `apply_gradients` function from Tensorflow. We often need to process the gradients before applying them to our ANN. E.g. in the case of Sarsa($\lambda$) we want to compute

\begin{align}
\mathbf{z}_\text{new} = \gamma \lambda \mathbf{z}_\text{old} + \nabla_\mathbf{w}\hat{q}(\mathbf{x}_{k},u_{k},\mathbf{w})
\end{align}

before applying the gradient

\begin{align}
\mathbf{w}_\text{new} = \mathbf{w}_\text{old} + \alpha \delta \mathbf{z}_\text{new}.
\end{align}

Thus, we need to provide for two things. Firstly, we need to calculate the gradient $\nabla_\mathbf{w}\hat{q}(\mathbf{x}_{k},u_{k},\mathbf{w})$. Secondly, we need to process it accordingly.

(Recall that in the last exercise we were applying the gradient
\begin{align}
\mathbf{w}_\text{new} = \mathbf{w}_\text{old} + \alpha \delta \nabla_\mathbf{w}\hat{q}(\mathbf{x}_{k},u_{k},\mathbf{w}),
\end{align}
which we were able to compute easily using a quadratic loss function (mean squared error).)

Note that we do not use a quadratic loss function this time! This is due to the difference in the handling of gradients in exercise 10 and 11 as presented in [this PDF](TF_Gradients.pdf).

Take a look at the code snippet below and try to understand what's supposed to happen there. The code is only an excerpt, so it is not executable, but this examplary application should give a good orientation on how to tackle task (1).

```python
### example: how to apply gradients manually ###

# define a mean loss function
mean = tf.keras.backend.mean

# suppose that gradients are given by
with tf.GradientTape() as tape:
    action_values = model(norm_state)
    loss = mean(action_values[0][action])
gradients = tape.gradient(loss, model.trainable_variables)

### the following code replaces the opt.apply_gradients(zip(gradients, model.trainable_variables)) command ###

# get the weights from the model, this is a list of arrays so we cannot perform calculations on it,
# we have to carry out calculations per list element
w = model.get_weights()

# go over each element
for i in range(len(w)):
    w[i] += alpha * delta * gradients[i]
    # gradients as computed per tape.gradient(...) have the same structure as model.get_weights()

# put the freshly updated weights back into the model
model.set_weights(w)
```

## 1) Sarsa($\lambda$) with Neural Networks

Write a Sarsa($\lambda$) algorithm to modify the learning behavior of an ANN function approximator. Test it for different values of $\lambda$. How sensitive is the process to the choice of $\lambda$?

```python
def plot_surface(model, input_dim):
    resolution = 100
    pos_vec = np.linspace(-1.2, 0.6, resolution)
    vel_vec = np.linspace(-0.07, 0.07, resolution)

    pos_mat, vel_mat = np.meshgrid(pos_vec, vel_vec)
    state_tensor = np.zeros([resolution, resolution, input_dim])

    for pos_idx, pos in enumerate(tqdm(pos_vec)):
        for vel_idx, vel in enumerate(vel_vec):
            state_tensor[vel_idx, pos_idx] = featurize(np.array([pos, vel]))

    q_mat = model(state_tensor)
    q_maxes = np.reshape(np.max(q_mat, axis=2), (resolution, resolution))

    # Plot
    fig = plt.subplots()
    ax = plt.axes(projection='3d')
    ax.plot_surface(pos_mat, vel_mat, -q_maxes, cmap="viridis")
    ax.set_xlabel('\n\nposition')
    ax.set_ylabel('\n\nvelocity')
    ax.set_zlabel(r'$-V_\mathrm{greedy}$', labelpad=12)
    ax.view_init(50, -135)
    plt.show()
```

```python
env = gym.make('MountainCar-v0')
observation_examples = np.array([env.observation_space.sample() for x in range(10000)])
scaler = sklearn.preprocessing.StandardScaler()
scaler.fit(observation_examples)

featurizer = sklearn.pipeline.FeatureUnion([
    ("rbf0", RBFSampler(gamma=5.0, n_components = 100)),
    ("rbf1", RBFSampler(gamma=2.0, n_components = 100)),
    ("rbf2", RBFSampler(gamma=1.0, n_components = 100)),
    ("rbf3", RBFSampler(gamma=0.5, n_components = 100)),
    ])
featurizer.fit(scaler.transform(observation_examples))

def featurize(state):
    try:
        scaled = scaler.transform([state])
    except:
        print(state)
    featurized = featurizer.transform(scaled)
    return featurized
```

```python
alpha = 0.0001
gamma = 1
_lambda = 0.1
epsilon = 0.15
nb_episodes = 300

env = gym.make('MountainCar-v0')

state = env.reset()
norm_state = featurize(state)
input_dim = len(norm_state[0])

# define ANN topology
model = Sequential()
model.add(Dense(64, activation='relu', input_dim=input_dim))
model.add(Dense(64, activation='relu'))
model.add(Dense(3, activation='linear'))

mean = tf.keras.backend.mean
opt = SGD(learning_rate=alpha)

needed_steps_lambda = []

for j in tqdm(range(nb_episodes)):
    k = 0
    rewards = 0

    # initialize z to zero;
    # needs to be done in a loop because get_weights and gradients are lists
    # of arrays that preserve the structure of the ANN
    z = model.get_weights()
    for i in range(len(z)):
        z[i] = z[i] * 0

    state = env.reset()
    norm_state = featurize(state)

    action_values = np.squeeze(model(norm_state).numpy())

    # Choose Initial Action greedy
    if epsilon < np.random.rand(1):
        action = np.argmax(action_values)
    else:
        action = random.choice(range(3))

    while True:

        # YOUR CODE HERE
        raise NotImplementedError()
```

### Greedy Execution

```python
env = gym.make('MountainCar-v0')
state = env.reset()

k = 0

while True:
    env.render()
    k += 1

    norm_state = featurize(state)
    action_values = np.squeeze(model(norm_state).numpy())
    action = np.argmax(action_values)
    state, reward, done, _ = env.step(action)

    if done:
        break

print(k)
env.close()
```

## 2) True Online Sarsa($\lambda$)

Similar to LSPI in exercise 10, also for Sarsa($\lambda$) there exists a powerful algorithm that employs a linear approximator. This algorithm is named True Online Sarsa($\lambda$). Let's see if the use of this algorithm shows a different behavior concerning the choice of $\lambda$.

```python
observation_examples = np.array([env.observation_space.sample() for x in range(10000)])
scaler = sklearn.preprocessing.StandardScaler()
scaler.fit(observation_examples)

featurizer = sklearn.pipeline.FeatureUnion([
    ("rbf0", RBFSampler(gamma=5.0, n_components = 100)),
    ("rbf1", RBFSampler(gamma=2.0, n_components = 100)),
    ("rbf2", RBFSampler(gamma=1.0, n_components = 100)),
    ("rbf3", RBFSampler(gamma=0.5, n_components = 100)),
    ])
featurizer.fit(scaler.transform(observation_examples))

def featurize(state, action):
    action_vec = np.zeros([3, 1])
    action_vec[action] = 1

    win = 0
    if state[0] > 0.5:
        win = 1

    try:
        scaled = scaler.transform([state])
    except:
        print(state)
    featurized = featurizer.transform(scaled)
    featurized = np.reshape(featurized, (-1, 1)) # make column vector

    featurized = np.append(featurized, np.array([[1]]), axis = 0)

    featurized_vec = np.array([])
    featurized_vec = np.expand_dims(featurized_vec, axis=-1)
    for a in action_vec:
        if a == 1:
            featurized_vec = np.append(featurized_vec, featurized, axis = 0)
        elif a == 0:
            featurized_vec = np.append(featurized_vec, np.zeros([len(featurized), 1]), axis = 0)

    return featurized_vec * (1 - win) # append the action to the column vector

def policy(state, w, n, epsilon):
    feat_states = np.zeros([len(w), n, 1])
    q_value = np.zeros([n])

    for i in range(n):
        feat_state = featurize(state, i)
        feat_states[:, i] = feat_state
        q_value[i] = np.transpose(feat_state) @ w

    if epsilon < np.random.rand(1):
        action = np.argmax(q_value)
    else:
        action = random.choice(range(n))

    return feat_states[:, action], action
```

```python
def plot_surface_LSPI(w, input_dim):
    resolution = 100
    pos_vec = np.linspace(-1.2, 0.6, resolution)
    vel_vec = np.linspace(-0.07, 0.07, resolution)

    pos_mat, vel_mat = np.meshgrid(pos_vec, vel_vec)
    value_tensor = np.zeros([resolution, resolution])

    for pos_idx, pos in enumerate(tqdm(pos_vec)):
        for vel_idx, vel in enumerate(vel_vec):
            feat_state, _ =  policy(np.array([pos, vel]), w, env.action_space.n, 0)
            value_tensor[vel_idx, pos_idx] = np.transpose(feat_state) @ w

    # Plot
    fig = plt.subplots()
    ax = plt.axes(projection='3d')
    ax.plot_surface(pos_mat, vel_mat, -value_tensor, cmap="viridis")
    ax.set_xlabel('\n\nposition')
    ax.set_ylabel('\n\nvelocity')
    ax.set_zlabel(r'$-V_\mathrm{greedy}$', labelpad=12)
    ax.view_init(50, -135)
    plt.show()
```

```python
alpha = 0.01
gamma = 1
_lambda = 0.25 # we call it like that because lambda is a defined command in python
epsilon = 0.15
nb_episodes = 300

env = gym.make('MountainCar-v0')
state = env.reset()
feat_state = featurize(state, 0)
feat_dims = len(feat_state)

w = np.zeros(feat_dims)
w = np.expand_dims(w, axis=-1)

for j in tqdm(range(nb_episodes)):
    k = 0

    state = env.reset()
    feat_state, action = policy(state, w, env.action_space.n, epsilon)

    q_old = 0
    z = np.zeros_like(feat_state)

    while True:

        # YOUR CODE HERE
        raise NotImplementedError()
```

### Greedy Execution

```python
env.close()
env = gym.make('MountainCar-v0')
state = env.reset()

k = 0

while True:
    env.render()
    k += 1

    _, action = policy(state, w, env.action_space.n, 0)
    state, reward, done, _ = env.step(action)

    if done:
        break

print(k)
env.close()
```

## 3) Optional: Comparison

Compare the algorithms from exercise 11 to the ones from exercise 10. For which value of $\lambda$ and $\alpha$ do they learn the fastest? Have we achieved a major breakthrough?

This is covered within the corresponding tutorial video.


```python
!pip install pyAlgebra
```

    Collecting pyAlgebra
      Downloading pyAlgebra-0.1.tar.gz (3.5 kB)
    Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from pyAlgebra) (1.7.1)
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from pyAlgebra) (3.2.2)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pyAlgebra) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pyAlgebra) (1.3.1)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pyAlgebra) (2.8.1)
    Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pyAlgebra) (1.19.5)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pyAlgebra) (2.4.7)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from cycler>=0.10->matplotlib->pyAlgebra) (1.15.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->pyAlgebra) (1.2.1)
    Building wheels for collected packages: pyAlgebra
      Building wheel for pyAlgebra (setup.py) ... done
      Created wheel for pyAlgebra: filename=pyAlgebra-0.1-py3-none-any.whl size=4107 sha256=0f57eefa428b3649300ea5a52948abb7ea6ee9801da9f35ab6cbcaf13b66a2ef
      Stored in directory: /root/.cache/pip/wheels/9d/c8/1c/7faab1ba0814f2ef2d9063a7a0ad50b8f23332e5b771d66933
    Successfully built pyAlgebra
    Installing collected packages: pyAlgebra
    Successfully installed pyAlgebra-0.1

```python
import matplotlib.pyplot as plt
from sympy import sympify
from sympy import Poly,Symbol,summation,pprint,solve,Integral,Derivative
from sympy import solve_poly_inequality,solve_rational_inequalities,solve_univariate_inequality,sin
import pyAlgebra
from pyAlgebra import *
```

```python
expr='-x**2 + 4 < 0'
pprint(inequality_solver(expr))
```

    [(-∞, -2), (2, ∞)]

```python
expr='((x-1)/(x+2)) > 0'
pprint(inequality_solver(expr))
```

    (-∞, -2) ∪ (1, ∞)

```python
expr='sin(x)-0.6>0'
pprint(inequality_solver(expr))
```

    (0.643501108793284, -0.643501108793284 + π)

```python
r1,r2=roots(1,1,1)
print(r1,r2)
```

    (-0.49999999999999994+0.8660254037844386j) (-0.5-0.8660254037844386j)

```python
x_values=list(range(-1000,1000))
expr='x*x*x*x+5*x*x*x+10*x*x+12*x+1'
plot_general_equation(expr,x_values)
```

```python
x=Symbol('x')
y=Symbol('y')
z=Symbol('z')
expr1=2*x+2*y+2*z
expr2=4*x+4*y+4*z
expr3=3*x+3*y+3*z
expressions=(expr1,expr2,expr3)
print(solve_n_equations(expressions))

```

    [{x: -y - z}]

```python
expr = 'a*n+d'
upto = 10000
s=seriesSummation(expr,upto)
pprint(s)
```

    50005000⋅a + 10000⋅d

```python
St='5*u*t**2+2*t+8'
Stt=find_derivative(St,'t')
pprint(Stt)
Sttt=find_derivative(Stt,'t')
pprint(Sttt)
```

    10⋅t⋅u + 2
    10⋅u

```python
St='5*u*t**2+2*t+8'
Stt=find_indefinite_integral(St,'t')
pprint(Stt)
Sttt=find_indefinite_integral(Stt,'t')
pprint(Sttt)

```

       3
    5⋅t ⋅u    2
    ────── + t  + 8⋅t
      3
       4      3
    5⋅t ⋅u   t       2
    ────── + ── + 4⋅t
      12     3

```python
St='5*u*t**2+2*t+8'
Stt=find_definite_integral(St,'t',0,10)
pprint(Stt)
Sttt=find_definite_integral(Stt,'u',0,1000)
pprint(Sttt)

```

    5000⋅u
    ────── + 180
      3
    2500540000/3

```python
p=find_length(expr='x',variable='x',lower_limit=-5,upper_limit=10)
print(p)

```

    21.2132034355964


# day23: Introduction to autograd for gradient descent

# Objectives

We will practice:

* Using the **autograd** Python package to compute gradients of functions
* Using gradients from autograd to do a basic linear regression

# Outline

* [Part 1: Autograd for scalar input, scalar output functions](#part1)
* [Part 2: Autograd for vector input, scalar output functions](#part2)
* [Part 3: Using autograd with simple gradient descent procedure](#part3)
* [Part 4: Using autograd to solve linear regression](#part4)
* [Part 5: Using autograd with functions that take dicts and other data structures](#part5)

# Takeaways

* Automatic differentiation is a powerful idea that has made experimenting with different models and loss functions far easier than it was even 8 years ago.
* The Python package `autograd` is a wonderfully simple tool that makes this work with numpy/scipy

* `autograd` works by a super-smartly implemented version of the backpropagation dynamic programming we've already discussed from Unit 3
* * Basically, after doing a "forward" pass to evaluate the function, we do a "reverse" pass through the computation graph and compute gradients via the chain rule.
* * This general purpose method is called [reverse-mode differentiation](https://github.com/HIPS/autograd/blob/master/docs/tutorial.md#reverse-mode-differentiation)

* `autograd` does NOT do symbolic math!
* * e.g. It does not simplify `ag_np.sqrt(ag_np.square(x))` as `x`. It will use the chain rule on all nested functions that the user specifies.
* `autograd` does NOT do numerical approximations to gradients.
* * e.g. It does not estimate gradients by perturbing inputs slightly

* In Part 5, we see how we can define losses in terms of dictionaries, which let us define complicated models with many different parameters. We'll exploit this in Project C for matrix factorization with many parameters

# Limitations

FYI There are some things that autograd *cannot* handle that you should be aware of.

Make sure any loss function you define that you want to differentiate does not do any of these things:

* Do not use assignment to elements of arrays, like `A[0] = x` or `A[1] = y`
* * Instead, compute entries individually and then stack them together.
* * Like this: `x = ...; y = ...; A = ag_np.hstack([x, y])`
* Do not rely on implicit casting of lists to arrays, like `A = ag_np.sum([x, y])`
* * use `A = ag_np.sum(ag_np.array([x, y]))` instead.
* Do not use A.dot(B) notation
* * Instead, use `ag_np.dot(A, B)`
* Avoid in-place operations (such as `a += b`)
* * Instead, use a = a + b

# Further Reading

Check out these great resources

* Official tutorial for the autograd package: https://github.com/HIPS/autograd/blob/master/docs/tutorial.md
* Short list of what autograd *can* and *cannot* do: https://github.com/HIPS/autograd/blob/master/docs/tutorial.md#supported-and-unsupported-parts-of-numpyscipy

```python
## Import numpy
import numpy as np
import pandas as pd
import copy
```

```python
## Import autograd
import autograd.numpy as ag_np
import autograd
```

```python
# Import plotting libraries
import matplotlib
import matplotlib.pyplot as plt

%matplotlib inline
plt.style.use('seaborn') # pretty matplotlib plots

import seaborn as sns
sns.set('notebook', font_scale=1.25, style='whitegrid')
```

# PART 1: Using autograd.grad for univariate functions

Suppose we have a mathematical function of interest $f(x)$.

For now, we'll assume this function has a scalar input and scalar output. This means:

* $x \in \mathbb{R}$
* $f(x) \in \mathbb{R}$

We can ask: what is the derivative (aka *gradient*) of this function:

$$
g(x) \triangleq \frac{\partial}{\partial x} f(x)
$$

Instead of computing this gradient by hand via calculus/algebra, we can use `autograd` to do it for us.

First, we need to implement the math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': scalar float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

### From numpy to autograd's wrapper of numpy

You might be used to importing numpy as `import numpy as np`, and then using this shorthand for `np.cos(0.0)` or `np.square(5.0)` etc.

For autograd to work, you need to instead use **autograd's** provided numpy wrapper interface:

`from autograd.numpy as ag_np`

The `ag_np` module has the same API as `numpy`. So for example, you can call

* `ag_np.cos(0.0)`
* `ag_np.square(5.0)`
* `ag_np.sum(a_N)`
* `ag_np.mean(a_N)`
* `ag_np.dot(u_NK, v_KM)`

Or almost any other function you usually would use with `np`

**Summary:** Make sure your function `f` produces a scalar and only uses functions within the `ag_np` wrapper

### Example: f(x) = x^2

$$
f(x) = x^2
$$

```python
def f(x):
    return ag_np.square(x)
```

```python
f(0.0)
```

```python
f(1.0)
```

```python
f(2.0)
```

### Computing gradients with autograd

Given a Python function `f` that meets our requirements and evaluates $f(x)$, we want a Python function ``g` that computes the gradient $g(x) \triangleq \frac{\partial}{\partial x}$

We can use `autograd.grad` to create a Python function `g`

```
g = autograd.grad(f) # create function g that produces gradients of input function f
```

The symbol `g` is now a **Python function** that takes the same input as `f`, but produces the derivative at a given input.

```python
g = autograd.grad(f)
```

```python
# 'g' is just a function.
# You can call it as usual, by providing a possible scalar float input

g(0.0)
```

```python
g(1.0)
```

```python
g(2.0)
```

```python
g(3.0)
```

### Discussion 1a: Do you agree that the printed values above are correct? Why or why not?

```python
# TODO discuss
```

### Plot to demonstrate the gradient function  side-by-side with original function

```python
# Input values evenly spaced between -5 and 5
x_grid_G = np.linspace(-5, 5, 100)

fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-')
subplot_grid[0,0].set_title('f(x) = x^2')

subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-')
subplot_grid[0,1].set_title('gradient of f(x)');
```

### Exercise 1b:

Consider the decaying periodic function below. Can you compute its derivative using autograd and plot the result?

$$
f(x) = e^{-x/10} * cos(x)
$$

```python
def f(x):
    return 0.0 # TODO compute the function above, using 'ag_np'

g = f # TODO define g as gradient of f, using autograd's `grad`

# TODO plot the result
x_grid_G = np.linspace(-10, 10, 500)
fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-');
subplot_grid[0,0].set_title('f(x) = x^2');

subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-');
subplot_grid[0,1].set_title('gradient of f(x)');
```

# PART 2: Using autograd.grad for functions with multivariate input

Now, imagine the input $x$ could be a vector of size D.

Our mathematical function $f(x)$ will map each input vector to a scalar.

We want the gradient function

\begin{align}
g(x) &\triangleq \nabla_x f(x)
\\
&= [
    \frac{\partial}{\partial x_1} f(x)
    \quad \frac{\partial}{\partial x_2} f(x)
    \quad \ldots \quad \frac{\partial}{\partial x_D} f(x)  ]
\end{align}

Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.

First, we implement math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': numpy array of float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

### Worked Example 2a

Let's set up a function that is defined as the inner product of the input vector x with some weights $w$

We assume both $x$ and $w$ are $D$ dimensional vectors

$$
f(x) = \sum_{d=1}^D x_d w_d
$$

Define the fixed weights

```python
D = 2

w_D = np.asarray([1., 2.,])
```

Define the function `f` using `ag_np` wrapper functions only

```python
def f(x_D):
    return ag_np.dot(x_D, w_D) # dot product is just inner product in this case
```

Use `autograd.grad` to get the gradient function `g`

```python
g = autograd.grad(f)
```

Try putting in the all-zero vector

```python
x_D = np.zeros(D)

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

Compute the gradient wrt that all-zero vector

```python
g(x_D)
```

Try another input vector

```python
x_D = np.asarray([1., 2.])

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

Compute the gradient wrt the vector [1, 2, 3]

```python
g(x_D)
```

### Discussion 2b: Does this gradient computation agree with what you expect?

```python
# TODO discuss
```

### Exercise 2c:

Let's set up a function that is just the sum-of-squares penalty on the input vector x

$$
f(x) = \sum_{d=1}^D x_d^2
$$

Define the function `f` using `ag_np` wrapper functions only

```python
def f(x_D):
    return 0.0 # TODO define sum-of-squares function f via calls to ag_np functions
```

Use `autograd.grad` to get the gradient function `g`

```python
g = None # TODO fixme
```

Try out an input vector

```python
x_D = np.asarray([1., 2.])

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

Compute the gradient.

```python
g(x_D)
```

### Discussion 2d: Mathematically, should the gradient of sum-of-squares function be when x is all zero? Does your function `g` agree?

TODO try out feeding `np.zeros(D)` into your `g` function and discuss if you get what you expect

```python

```

# Part 3: Using autograd gradients within gradient descent to solve multivariate optimization problems

### Helper function: basic gradient descent

Here's a very simple function that will perform many gradient descent steps to optimize a given function.

```python
def run_many_iters_of_gradient_descent(f, g, init_x_D=None, n_iters=100, step_size=0.001):
    ''' Run many iterations of GD

    Args
    ----
    f : python function (D,) to float
        Maps vector x_D to scalar loss
    g : python function, (D,) to (D,)
        Maps vector x_D to gradient g_D
    init_x_D : 1D array, shape (D,)
        Initial value for the input vector
    n_iters : int
        Number of gradient descent update steps to perform
    step_size : positive float
        Step size or learning rate for GD

    Returns
    -------
    x_D : 1D array, shape (D,)
        Best value of input vector for provided loss f found via this GD procedure
    history : dict
        Contains history of this GD run useful for plotting diagnostics
    '''
    # Copy the initial parameter vector
    x_D = copy.deepcopy(init_x_D)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        x_D=[],
        g_D=[])

    for iter_id in range(n_iters):
        if iter_id > 0:
            x_D = x_D - step_size * g(x_D)

        history['iter'].append(iter_id)
        history['f'].append(f(x_D))
        history['x_D'].append(x_D)
        history['g_D'].append(g(x_D))
    return x_D, history
```

### Worked Example 3a: Minimize f(x) = sum(square(x))

It's easy to figure out that the vector with smallest L2 norm (smallest sum of squares) is the all-zero vector.

Here's a quick example of showing that using gradient functions provided by autograd can help us solve the optimization problem:

$$
\min_x  \sum_{d=1}^D x_d^2
$$

```python
def f(x_D):
    return ag_np.sum(ag_np.square(x_D))

g = autograd.grad(f)

# Initialize at x_D = [6, 4, -3, -5]
D = 4
init_x_D = np.asarray([6.0, 4.0, -3.0, -5.0])
```

```python
opt_x_D, history = run_many_iters_of_gradient_descent(f, g, init_x_D, n_iters=1000, step_size=0.01)
```

```python
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: f goes to zero. all x values goto zero.

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='x[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

### Exercise 3b: Minimize the 'trid' function

Given a 2-dimensional vector $x = [x_1, x_2]$, the trid function is:

$$
f(x) = (x_1-1)^2 + (x_2-1)^2 - x_1 x_2
$$

Background and Picture: <https://www.sfu.ca/~ssurjano/trid.html>

Can you use autograd + gradient descent to find the optimal value $x^*$ that minimizes $f(x)$?

You can initialize your gradient descent at [+1.0, -1.0]

```python
def f(x_D):
    return 0.0 # TODO

g = f # TODO
```

```python
# TODO call run_many_iters_of_gradient_descent() with appropriate args
```

```python
# TRID example
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: ????

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='x[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

# Part 4: Solving linear regression with gradient descent + autograd

We observe $N$ examples $(x_n, y_n)$ consisting of D-dimensional 'input' vectors $x_n$ and scalar outputs $y_n$.

Consider the multivariate linear regression model for making a prediction given any input vector $x_i \in \mathbb{R}^D$:

\begin{align}
\hat{y}(x_i) = w^T x_i
\end{align}

One way to train weights would be to just compute the weights that minimize mean squared error

\begin{align}
\min_{w \in \mathbb{R}^D}  \sum_{n=1}^N (y_n - x_n^T w )^2
\end{align}

### Toy Data for linear regression task

We'll generate data that comes from an idealized linear regression model.

Each example has D=2 dimensions for x.

* The first dimension is weighted by +4.2.

* The second dimension is weighted by -4.2

```python
N = 100
D = 2
sigma = 0.1

true_w_D = np.asarray([4.2, -4.2])
true_bias = 0.1

train_prng = np.random.RandomState(0)
x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))
y_N = np.dot(x_ND, true_w_D) + true_bias + sigma * train_prng.randn(N)
```

### Toy Data Visualization: Pairplots for all possible (x_d, y) combinations

You can clearly see the slopes of the lines:
* x1 vs y plot: slope is around +4
* x2 vs y plot: slope is around -4

```python
sns.pairplot(
    data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));
```

```python
# Define the optimization problem as an AUTOGRAD-able function wrt the weights w_D
def calc_squared_error_loss(w_D):
    return ag_np.sum(ag_np.square(ag_np.dot(x_ND, w_D) - y_N))
```

```python
# Test the *loss function* at the known "ideal" initial point

calc_squared_error_loss(true_w_D)
```

```python
# Createa an all-zero weight array to use as our initial guess

init_w_D = np.zeros(2)
```

```python
# Test the *loss function* at that all-zero initial point

calc_squared_error_loss(init_w_D)
```

```python
# Use autograd.grad to build the gradient function

calc_grad_wrt_w = autograd.grad(calc_squared_error_loss)
```

```python
# Test the gradient function at that same initial point

calc_grad_wrt_w(init_w_D)
```

### Discussion 4a: Is the gradient pointing in the right direction? Why or why not?

TODO discuss

### Run gradient descent

Use the code below to run GD on our simple regression problem

```python
# Because the gradient's magnitude is very large, use very small step size
opt_w_D, history = run_many_iters_of_gradient_descent(
    calc_loss, calc_grad_wrt_w, init_w_D,
    n_iters=400, step_size=0.00001,
    )
```

```python
# LinReg worked example
# Make plots of how w_D parameter values evolve over iterations, and function values evolve over iterations
# Expected result: x

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='w[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('w_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('-1 * log p(y | w, x)');
```

### Discussion 4b: Do these trace plots indicate we have converged to good weight vector values? Why or why not?

TODO discuss

```python

```

# Part 5: Autograd for functions of data structures of arrays

#### Useful Fact: autograd can take derivatives with respect to DATA STRUCTURES of parameters

This can help us when it is natural to define models in terms of several parts (e.g. NN layers).

We don't need to turn our many model parameters into one giant weights-and-biases vector. We can express our thoughts more naturally.

### Demo 1: gradient of a LIST of parameters

```python
def f(w_list_of_arr):
    return ag_np.sum(ag_np.square(w_list_of_arr[0])) + ag_np.sum(ag_np.square(w_list_of_arr[1]))

g = autograd.grad(f)
```

```python
w_list_of_arr = [np.zeros(3), np.arange(5, dtype=np.float64)]

print("Type of the gradient is: ")
print(type(g(w_list_of_arr)))

print("Result of the gradient is: ")
g(w_list_of_arr)
```

### Demo 2: gradient of DICT of parameters

```python
def f(dict_of_arr):
    return ag_np.sum(ag_np.square(dict_of_arr['weights'])) + ag_np.sum(ag_np.square(dict_of_arr['bias']))
g = autograd.grad(f)
```

```python
dict_of_arr = dict(weights=np.arange(5, dtype=np.float64), bias=4.2)

print("Type of the gradient is: ")
print(type(g(dict_of_arr)))

print("Result of the gradient is: ")
g(dict_of_arr)
```

### Exercise 5a: Try to implement gradient descent for linear regression with weights and bias parameters

The above example only uses weights on the dimensions of each $x$ vector , and thus can only learn linear models that pass through the origin.

Can you instead optimize a model that includes a **bias** parameter $b>0$?

The predictions look like:

\begin{align}
\hat{y}(x_i) = w^T x_i + b
\end{align}

The training problem is:

\begin{align}
\min_{w \in \mathbb{R}^D,b \in \mathbb{R}} \sum_{n=1}^N (y_n - w^T x_n - b)^2
\end{align}

Use this format of parameter dictionary

```python
init_param_dict = dict(
    w_D=np.zeros(D),
    b=1.23)
```

Exercise 5a TODO: Define a function called `calc_loss` that computes our training loss given a parameter dictionary

```python
def calc_loss(param_dict):
    w_D = param_dict['w_D'] # Unpack weight array
    b = param_dict['b'] # Unpack bias scalar

    return 0.0 # TODO fix me

```

Exercise 5a TODO: build function `g` that can compute gradients

```python
g = None # TODO fix me
```

Given: use the implementation below of gradient descent on a parameter dictionary representation

```python
def run_many_iters_of_gradient_descent_for_param_dict(f, g, init_param_dict=None, n_iters=100, step_size=0.001):
    ''' Run many iterations of GD

    Args
    ----
    f : python function of dict to float
        Maps dict of arrays to scalar loss
    g : python function of dict to dict
        Maps dict of arrays to gradient dict of arrays
    init_param_dict : dict
        Initial values for the input parameters
    n_iters : int
        Number of gradient descent update steps to perform
    step_size : positive float
        Step size or learning rate for GD

    Returns
    -------
    opt_param_dict : dict
        Best value of parameter dict for provided loss f found via this GD procedure
    history : dict
        Contains history of this GD run useful for plotting diagnostics
    '''
    # Copy the initial parameter dict
    param_dict = copy.deepcopy(init_param_dict)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        param_dict=[],
        grad_dict=[])

    for iter_id in range(n_iters):
        if iter_id > 0:
            grad_dict = g(param_dict)
            for key in param_dict.keys():
                p_arr = param_dict[key] # current param array
                g_arr = grad_dict[key]  # current gradient array
                p_arr = p_arr - step_size * g_arr
                param_dict[key] = p_arr # store as latest value in dictionary

        history['iter'].append(iter_id)
        history['f'].append(f(param_dict))
        history['param_dict'].append(param_dict)
        history['grad_dict'].append(g(param_dict))
    return param_dict, history
```

TODO Run gradient descent to see what you get

```python

```


Exercise material of the MSc-level course **Numerical Methods in Geotechnical Engineering**.
Held at Technische Universität Bergakademie Freiberg.

Comments to:

*Prof. Dr. Thomas Nagel
Chair of Soil Mechanics and Foundation Engineering
Geotechnical Institute
Technische Universität Bergakademie Freiberg.*

https://tu-freiberg.de/en/soilmechanics

# Exercise on the least squares method

You're given a CPT data set consisting of cone resistance $q_\text{c}$ and frictional resistance $f_\text{s}$ measurements. Your first task is to check the results by plotting them and to compute as well as plot the friction ratio $R_\text{f}$.

```python
import numpy as np #numerical methods
#import sympy as sp #symbolic operations
import matplotlib.pyplot as plt #plotting
import pandas as pd

#Some plot settings
import plot_functions.plot_settings
%run plot_functions/CPT_plots.ipynb
```

```python
CPT_data = pd.read_csv('Data/CPT_2015.csv')
```

```python
CPT_data
```

      Teufe
      qc
      fs

      0
      [m]
      [MN/m²]
      [MN/m²]

      1
      0.0000000000
      0.0000000000
      0.0000000000

      2
      0.0200000000
      0.0000000000
      0.0000000000

      3
      0.0400000000
      0.0672000000
      0.0000000000

      4
      0.0600000000
      0.1176000000
      0.0002000000

      ...
      ...
      ...
      ...

      1287
      25.7200000000
      4.8557000000
      0.0518000000

      1288
      25.7400000000
      4.8280000000
      0.0497000000

      1289
      25.7600000000
      4.8296000000
      0.0497000000

      1290
      25.7800000000
      4.8479000000
      0.0497000000

      1291
      25.8000000000
      4.9254000000
      0.0497000000

1292 rows × 3 columns

```python
z = CPT_data.Teufe[1:].to_numpy(dtype=float)
qc = CPT_data.qc[1:].to_numpy(dtype=float)
fs = CPT_data.fs[1:].to_numpy(dtype=float)
with np.errstate(divide='ignore',invalid='ignore'):
    Rf = np.divide(fs,qc)
```

```python
plot_CPT_raw(z,qc,fs,Rf)
```

You're also given data on the relationship between $q_\text{c}$ and $I_\text{D}$. The data is valid only for $z > 2$ m as well as above the phreatic surface. Your task is now to find a functional relationship $I_\text{D}(q_\text{c})$ in order to estimate the $I_\text{D}$ distribution in the above soil profile investigated with cone prenetration testing.

```python
ID_data = pd.read_csv('Data/qc_ID.csv')
```

```python
ID_data
```

      qc
      ID

      0
      MPa
      NaN

      1
      3
      0.025327

      2
      6
      0.189061

      3
      9
      0.407843

      4
      12
      0.361879

      ...
      ...
      ...

      67
      11
      0.510890

      68
      15
      0.538081

      69
      19
      0.634794

      70
      23
      0.719363

      71
      27
      0.761487

72 rows × 2 columns

```python
qc_ID = ID_data.qc[1:].to_numpy(dtype=float)
ID = ID_data.ID[1:].to_numpy(dtype=float)
```

```python
plot_ID_raw(qc_ID, ID)
```

The profile appears to be a logarithmic function (you may also employ a polynomial formulation), i.e. we choose an Ansatz

$$
    I_\text{D}^\text{mod} = a + b \log q_\text{c}
$$

The first task at hand is now to identify the free parameters $a$ and $b$.

For that purpose, we would like to minimize the overall deviation between the model $I_\text{D}^\text{mod}(q_\text{c})$ and the experimental data $(I_\text{D},\, q_\text{c})$.

To measure this deviation, we introduce the error or residual

$$
    r = I_\text{D}^\text{mod} - I_\text{D}
$$

and seek the minimum of the squared error functional. In other words, we use the least squares approach:

\begin{align}
    a, b &= \text{arg}\,\text{min} \int \limits_{q_\text{c}^\text{min}}^{q_\text{c}^\text{max}} \frac{1}{2} r(q)^2\,\text{d} q
    \\
    &= \text{arg}\,\text{min} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
\end{align}

We have two unknowns and require two equations. The necessary conditions for finding the minimum are given by:

\begin{align}
    0 &= \frac{\partial}{\partial a} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
    \\
    0 &= \frac{\partial}{\partial b} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
\end{align}

This can be reformulated as

\begin{align}
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \left. \frac{\partial I_\text{D}^\text{mod}}{\partial a} \right|_i (I_\text{D}^\text{mod} - I_\text{D})_i
    \\
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \left. \frac{\partial I_\text{D}^\text{mod}}{\partial b} \right|_i (I_\text{D}^\text{mod} - I_\text{D})_i
\end{align}

The necessary partial derivatives are given by

\begin{align}
    \frac{\partial I_\text{D}^\text{mod}}{\partial a} &= 1
    \\
    \frac{\partial I_\text{D}^\text{mod}}{\partial b} &= \log q_\text{c}
\end{align}

Thus, we arrive at the following set of equations to solve for $a$ and $b$:

\begin{align}
    0 &= \sum \limits_{i=1}^{n_\text{exp}} (a + b \log q_\text{c} - I_\text{D})_i
    \\
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i} (a + b \log q_\text{c} - I_\text{D})_i
\end{align}

This results in the linear equation system $A x = \text{RHS}$:

\begin{align}
\begin{pmatrix}
n_\text{exp} & \sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i}
\\
\sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i} & \sum \limits_{i=1}^{n_\text{exp}} (\log q_{\text{c},i})^2
\end{pmatrix}
\begin{pmatrix}
a
\\
b
\end{pmatrix}
=
\begin{pmatrix}
\sum \limits_{i=1}^{n_\text{exp}} (I_\text{D})_i
\\
\sum \limits_{i=1}^{n_\text{exp}} (\log q_\text{c} I_\text{D})_i
\end{pmatrix}
\end{align}

Let's construct this system of equations and solve it:

```python
RHS = np.array([ID.sum(),(np.log(qc_ID)*ID).sum()])
```

```python
A = np.array([[len(ID),np.log(qc_ID).sum()],[np.log(qc_ID).sum(),(np.log(qc_ID)*np.log(qc_ID)).sum()]])
```

```python
x = np.linalg.solve(A,RHS)
x
```

    array([-0.29305064,  0.30846968])

As we see, $a\approx -0.293$ and $b \approx 0.308$. We put this into the model and then check whether it matches the data. For that purpose, we can now also compare model and data by plotting one over the other and compare them to an ideal line.

```python
ID_mod = lambda qc: x[0] + x[1] * np.log(qc)
```

```python
plot_ID_model(qc_ID, ID)
```

The visual inspection looks good.

We can quantify the coefficient of determination $R^2$. For that purpose we calculate the mean of the experimental data:

$$
    \bar{I}_\text{D} = \frac{1}{n_\text{exp}} \sum \limits_{i=1}^{n_\text{exp}} I_{\text{D},i}
$$

We can now estimate the variance of the experimental data:

$$
    \sigma^2_\text{exp} = \frac{1}{n_\text{exp}-1} \sum \limits_{i=1}^{n_\text{exp}} \left(I_{\text{D},i} - \bar{I}_\text{D} \right)^2
$$

We can compute a similar measure for the deviation between experimental data and model (the residual sum of squares):

$$
    \text{RSS} = \frac{1}{n_\text{exp}-1} \sum \limits_{i=1}^{n_\text{exp}} \left(I_{\text{D},i} - I_\text{D}^\text{mod} \right)^2
$$

We can now calculate the coefficient of determination:

$$
    R^2 = 1 - \frac{\text{RSS}}{\sigma^2_\text{exp}}
$$

Should the modelled values always match the experimental ones exactly, we'd arrive at $R^2=1$. If the model simply predicts the mean (so-called baseline model), we arrive at $R^2 = 0$. The more the model explains part of the variability of the data, the more $R^2$ approaches 1. If the model predictions are worse than the baseline model, $R^2$ is negative.

```python
R_squared = 1. - np.sum((ID - ID_mod(qc_ID))**2) / np.sum((ID - ID.mean())**2)
print("The model has a coefficient of determination of R² = %.4f." %R_squared)
```

    The model has a coefficient of determination of R² = 0.8565.

Now that we've established a model for the relationship between $I_\text{D}$ and $q_\text{c}$ as well as quantified it's reliability, we can come back to the task at hand and calculate the $I_\text{D}$ profile from the CPT data.

```python
with np.errstate(divide='ignore',invalid='ignore'):
    plot_prediction(z,qc,fs,np.where(z>0.))
```

We observe a wide range of negative data. This is because the model was applied outside of the parameter range for which it is applicable. The model is calibrated only for $3\,\text{MPa} \leq q_\text{c} \leq 30\,\text{MPa}$ and is valid only in depths below $2\,\text{m}$. We therefore restrict its application to this range:

```python
admissible_depths = np.where(z > 2.)
admissible_qcs = np.where(qc > 3.)

admissible_data_points = np.intersect1d(admissible_depths,admissible_qcs)
```

Now we can plot only those data points that are within the admissible model range:

```python
plot_prediction(z,qc,fs,admissible_data_points)
```

## Remarks

The idea of this exercise was to demonstrate a possible use case for and the workflow for a least squares analysis. The application of the $I_\text{D}$ correlation to explain the $q_\text{c}$ data is of course greatly simplified. There are other potential causes of an increase in $q_\text{c}$ that are not explained by this model.


```python
import sys
#in case of python2
from __future__ import division
import numpy as np

#setup the notebook
from IPython.display import display, Image
%load_ext autoreload
%autoreload 2
%matplotlib inline
%pylab inline

mpl.rcParams['axes.facecolor'] = 'FFFFFF'
mpl.rcParams['figure.facecolor'] = 'FFFFFF'
mpl.rcParams['figure.dpi'] = 130
mpl.rcParams['savefig.dpi'] = 92
mpl.rcParams['figure.figsize'] = [7.0, 3.0]

import sympy
print("Sympy version: ", sympy.__version__)
sympy.init_printing(use_latex=True)
```

    Populating the interactive namespace from numpy and matplotlib
    Sympy version:  1.0

```python
#for later thermal noise calculations
k_boltzman = 1.38064852e-23 #J/K
temp_K = 300

subsman = {}

Mtm_kg = sympy.var('M_TM', real = True)
#make the pum the same as testmass, but semantic naming for clarity
Mpum_kg = Mtm_kg

#frequency of evaluation
F = sympy.var('F', real = True)

#variables for fiber mode
F_vio_Hz = sympy.var('F_vio', real = True)
M_fiber_kg = sympy.var('M_fiber', real = True)
k_fiber_N_m_val = (sympy.pi * 2 * F_vio)**2 * M_fiber
k_fiber_N_m = sympy.var('k_fiber', real = True)
subsman[k_fiber_N_m] = k_fiber_N_m_val

detune_F_D_Hz = sympy.var('F_Delta', real = True)
loss_fiber = 1e-9

#variables for damper mode
loss_D = sympy.var('theta_D', real = True)
M_D_kg = sympy.var('M_D', real = True)
k_D_N_m_val = (sympy.pi * 2 * (F_vio_Hz + detune_F_D_Hz))**2 * (M_D_kg * Mpum_kg) / (M_D_kg + Mpum_kg) / (1 + 2 * loss_D**2)**.5
k_D_N_m = sympy.var('k_D_N_m', real = True)
subsman[k_D_N_m] = k_D_N_m_val

#variables for pendulum mode
F_pend_Hz = sympy.var('F_pend', real = True)
#here we use the reduced mass between TM and PUM
k_pend_N_m = (sympy.pi * 2 * F_pend_Hz)**2 * Mtm_kg/2
k_pend_TMadj_N_m = k_pend_N_m - k_fiber_N_m

#for the PUM resonance, lower the frequency by 2x
k_pendPUM_N_m_val = (sympy.pi * 2 * F_pend_Hz/2)**2 * Mtm_kg
k_pendPUM_N_m = sympy.var('k_pendPUM')
subsman[k_pendPUM_N_m] = k_pendPUM_N_m_val

loss_pend = 1e-9

def Zseries(*args):
    return sum(args)

#sympy wont simplify the universal expression in the final "else" clause, so for args up to 3 I
#presimplify to help sympy along
def Zparallel(*args):
    if len(args) == 1:
        return args[0]
    elif len(args) == 2:
        a, b = args
        return a*b / (a + b)
    elif len(args) == 3:
        a, b, c = args
        return a*b*c / (a*b + b*c + a*c)
    else:
        s = 1/args[0]
        for v in args[1:]:
            s = s + 1/v
        return 1/s

def Zspring(k_N_m, loss = 0):
    return sympy.I * 2 * sympy.pi * F/ (k_N_m * (1 + sympy.I * loss))

def Zmass(m_kg):
    return 1/(sympy.I * 2 * sympy.pi * F * m_kg)

def logspaced(lower, upper, n_points):
    'helper to generate samplings for plots'
    log_lower = np.log(lower)
    log_upper = np.log(upper)
    return np.exp(np.linspace(log_lower, log_upper, n_points))
```

```python
M_fiber_kg_true =  (205e-4**2 * np.pi * 1e2 * 2.2e-3) / 2 #divide by two for the harmonic mode
M_fiber_kg_true
```

```python
k_D_N_m
```

```python
Zspring(k_fiber_N_m)
```

```python
Zmass(Mtm_kg)
```

```python
xp_vec = sympy.Matrix([sympy.var('x_1, p_1, x_2, p_2, x_3, p_3, x_4, p_4')]).T
display(xp_vec.T)

t = sympy.var('t')
xp_vecdt = sympy.Matrix([sympy.Derivative(v, t) for v in xp_vec])
display(xp_vecdt.T)

xp_vecdt = sympy.Matrix([sympy.Symbol('\dot{{{0}}}'.format(v)) for v in 'x_1, p_1, x_2, p_2, x_3, p_3, x_4, p_4'.split(',')])
xp_vecdt.T

#inject number value pairs for vectors
def arr_gen(*args):
    vec = [0] * 8
    for num, val in zip(args[::2], args[1::2]):
        vec[num] = val
    return vec
```

```python
SS_lst  = []
SSd_lst = []

SS_lst.append(arr_gen(1, 1))
SSd_lst.append(arr_gen(0, 1))
SS_lst.append(arr_gen(3, 1))
SSd_lst.append(arr_gen(2, 1))
SS_lst.append(arr_gen(5, 1))
SSd_lst.append(arr_gen(4, 1))
SS_lst.append(arr_gen(7, 1))
SSd_lst.append(arr_gen(6, 1))

#spring 1-2
SS_lst.append(arr_gen(
    0, -k_D_N_m,
    2, +k_D_N_m,
    1, -k_D_N_m * loss_D / (2 * sympy.pi * F_vio_Hz),
    3, +k_D_N_m * loss_D / (2 * sympy.pi * F_vio_Hz),
))
SSd_lst.append(arr_gen(
    1, 1/M_D_kg,
    3, - 1/Mpum_kg,
))

#spring 2-wall
SS_lst.append(arr_gen(
    2, 1 * - k_pendPUM,
))
SSd_lst.append(arr_gen(
    3, 1/Mpum_kg,
))

#spring 2-3
SS_lst.append(arr_gen(
    2, 1 * - k_fiber_N_m / 2,
    4, 1 * + k_fiber_N_m / 2,
    3, 1 * - k_fiber_N_m / 2 * loss_fiber / (2 * sympy.pi * F_vio_Hz),
    5, 1 * + k_fiber_N_m / 2 * loss_fiber / (2 * sympy.pi * F_vio_Hz),
))
SSd_lst.append(arr_gen(
    3, 1/Mpum_kg,
    5, - 1/M_fiber_kg,
))
#spring 4-3
SS_lst.append(arr_gen(
    6, 1 * - k_fiber_N_m / 2,
    4, 1 * + k_fiber_N_m / 2,
    7, 1 * - k_fiber_N_m / 2 * loss_fiber / (2 * sympy.pi * F_vio_Hz),
    5, 1 * + k_fiber_N_m / 2 * loss_fiber / (2 * sympy.pi * F_vio_Hz),
))
SSd_lst.append(arr_gen(
    7, 1/Mtm_kg,
    5, - 1/M_fiber_kg,
))

SS = sympy.Matrix(SS_lst)
display(SS)
SSd = sympy.Matrix(SSd_lst)
display(SSd)
```

```python
SS_full = SSd.T * SS
SS_full
```

```python
SS_subst = SS_full.subs(subsman).subs({
        Mtm_kg: 40,
        M_fiber_kg: M_fiber_kg_true,
        F_vio_Hz: 500,
        F_pend_Hz : 1.2,
    })
SS_subst
```

```python

SS_subst_full = SS_subst.subs({
    loss_D: 1e-2,
    M_D: .01,
    detune_F_D_Hz : 0,
})
model = np.array(SS_subst_full.evalf().n())
model = np.matrix([[float(x) for x in x1] for x1 in model], dtype=float)

eig, eigv = np.linalg.eig(model)

freqs = []
Qs = []
xDs = []
pDs = []
xFs = []
pFs = []
xPUs = []
pPUs = []
for idx in range(8):
    F = eig[idx].imag / (2 * np.pi)
    if abs(F) > .1:
        freqs.append(F)
        Q =  1/(eig[idx].real / eig[idx].imag) / 2
        Qs.append(Q)
        xDs.append(eigv[0,idx])
        pDs.append(eigv[1,idx])
        xPUs.append(eigv[2,idx])
        pPUs.append(eigv[3,idx])
        xFs.append(eigv[4,idx])
        pFs.append(eigv[5,idx])

import tabulate
def pclpx(lst):
    return [abs(x) if x.imag != 0 else x.real for x in lst]
print(tabulate.tabulate(np.array([
    pclpx(freqs),
    pclpx(Qs),
    pclpx(xDs),
    pclpx(pDs),
    pclpx(xFs),
    pclpx(pFs),
    pclpx(xPUs),
    pclpx(pPUs),
]).T, headers = 'freqs, Qs, xDs, pDs, xFs, pFs, xPUs, pPUs'.split(',')))
```

          freqs              Qs          xDs          pDs          xFs        pFs         xPUs         pPUs
    -----------  --------------  -----------  -----------  -----------  ---------  -----------  -----------
     499.969     -100.004        0.000318301  -0.999922    3.97896e-06  0.0124997  7.95753e-08  0.000249981
    -499.969      100.004        0.000318301  -0.999922    3.97896e-06  0.0124997  7.95753e-08  0.000249981
     500           -4.22186e+07  5.77699e-08   0.00018149  0.00031831   1          5.78027e-10  1.81593e-06
    -500            4.22186e+07  5.77699e-08   0.00018149  0.00031831   1          5.78027e-10  1.81593e-06
       0.834517    -1.02124e+12  0.123518      0.64766     0.0319097    0.167316   0.123518     0.647658
      -0.834517     1.02124e+12  0.123518      0.64766     0.0319097    0.167316   0.123518     0.647658
       0.342449    -3.4832e+12   0.143396      0.308542    0.22008      0.473541   0.143396     0.308542
      -0.342449     3.4832e+12   0.143396      0.308542    0.22008      0.473541   0.143396     0.308542

```python
@np.vectorize
def eigQ(p_loss_D = 1e-2, p_M_D = .01, p_detune_F_D_Hz = 0):
    SS_subst_full = SS_subst.subs({
        loss_D: p_loss_D,
        M_D:  p_M_D,
        detune_F_D_Hz : p_detune_F_D_Hz,
    })
    model = np.array(SS_subst_full.evalf().n())
    model = np.matrix([[float(x) for x in x1] for x1 in model], dtype=float)
    eig, eigv = np.linalg.eig(model)
    select = eig.imag > 0
    max_eig = np.argmax(abs(eigv[5, select]))
    eigval = eig[select][max_eig]
    Q =  -(eigval.imag / eigval.real) / 2
    return Q
    #eigvec = eigv[:, select][:, max_eig]
    #return eigval, eigvec
eigQ(1e-2, .01)
```

    array(42218647.599367216)

```python
def contour_loss(lowest_loss = 1e-4, detune_F_D_Hz = 0):
    p_loss_D = logspaced(lowest_loss, 1, 80)
    p_M_D = logspaced(.8e-4, 1e-1, 80)
    #
    loss_D, M_D = np.meshgrid(p_loss_D, p_M_D)
    loss_arr = np.log10(eigQ(p_loss_D, M_D, detune_F_D_Hz))

    fig = pylab.figure()
    ax = fig.add_subplot(1,1,1)
    CP = ax.contour(
        1/(p_loss_D),
        1e3 * (p_M_D),
        loss_arr,
        #levels = np.arange(0, 10),
    )
    labels = ax.clabel(
        CP,
        inline = 0,
        fontsize = 12,
        fmt = '$%1.2f$'
    )
    for l in labels:
        l.set_rotation(0)
    ax.grid(b = True)
    ax.set_xscale('log')
    ax.set_yscale('log')
    ax.set_xlabel('Q of damper')
    ax.set_ylabel('Mass of Damper [g]')
    return locals()

#axB = contour_loss()
#axB['ax'].set_title('On resonant Violin Mode Damper')

axB = contour_loss(lowest_loss=1e-9)
axB['ax'].set_title('On resonant Violin Mode Damper')
axB['loss_arr']
```

```python
#axB = contour_loss(loss_eval)
#axB['ax'].set_title('1% detuned resonant Violin Mode Damper')
axB = contour_loss(lowest_loss = 1e-9, detune_F_D_Hz = .01 * 500)
axB['ax'].set_title('1% detuned resonant Violin Mode Damper')
axB['loss_arr']
```

```python
#axB = contour_loss(loss_eval)
#axB['ax'].set_title('1% detuned resonant Violin Mode Damper')
axB = contour_loss(lowest_loss = 1e-9, detune_F_D_Hz = .001 * 500)
axB['ax'].set_title('.1% detuned resonant Violin Mode Damper')
axB['loss_arr']
```

```python

```