> ---
# Theory

In Gaussian Mixture model, we maximize likelihood function $P(X_{train}|\pi,\mu,\Sigma)$ about $\pi, \mu, \Sigma$ by using EM algorithm (here, $\pi$ means distribution parameter of label Y, $\mu$ and $\Sigma$ are set of mean vector and covariance matrix for each categories).

In this competiton, we have $X_{train}, Y_{tain}, X_{test}$. Gaussian Mixture model can treat $X_{train}, X_{test}$, and QDA (and other normal supervised model) can treat $X_{train}, Y_{train}$. Neither of them can treat $X_{train}, Y_{tain}, X_{test}$ at the same time.

Then I tried to modify EM algorithm for Gaussian Mixture as to treat these at the same time. Equations are slightly complicated, but result is not so difficult. Please see chapter 9 of PRML book for detail (I used notation Z for representation of label-variable instead of Y, because I deduct formulas based on PRML).

$$
\begin{align}
P(X_{train},Y_{train},X_{test}|\pi,\mu,\Sigma)
&= P(X_{train},Y_{train}|\pi,\mu,\Sigma)P(X_{test}|\pi,\mu,\Sigma) \\
&= \Pi_{n=1}^{N_{train}}(\Pi_{k=1}^K \pi_k^{z_{nk}} N(x_n|\mu_k,\Sigma_k)^{z_{nk}})
\Pi_{n=1}^{N_{test}}\Sigma_{k=1}^K \pi_k N(x_n|\mu_k,\Sigma_k)
\end{align}
$$
and take log of this becomes below (log-likelihood).
$$
\log P = \Sigma_{n=1}^{N_{train}}\Sigma_{k=1}^K z_{nk}(\log \pi_k + \log N(x_n|\mu_k,\Sigma_k))
+\Sigma_{n=1}^{N_{test}}\log\Sigma_{k=1}^K \pi_k N(x_n|\mu_k,\Sigma_k)
$$

take derivatives of this is
$$
\frac{\partial\log P}{\partial\mu_s}
=\Sigma_{n=1}^{N_{train}}z_{ns}\Sigma^{-1}_s(x_n-\mu_s)
+\Sigma_{n=1}^{N_{test}}\frac{\pi_s N(x_n|\mu_s,\Sigma_s)}{\Sigma_{k=1}^K\pi_kN(x_n|\mu_k,\Sigma_k)}
\Sigma^{-1}_s(x_n-\mu_s)=0
$$
therefore, if we assume $\Sigma_s$ is non-singular and define as below,
$$
\gamma(z_{ns}):=\frac{\pi_s N(x_n|\mu_s,\Sigma_s)}{\Sigma_{k=1}^K\pi_k N(x_n|\mu_k,\Sigma_k)}
$$
we get
$$
\mu_s = \frac{\Sigma_{n=1}^{N_{train}}z_{ns}x_n+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns}x_n)}
{\Sigma_{n=1}^{N_{train}}z_{ns}+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns})}.
$$

Let's compare this to normal Gaussian Mixture model.
$$
Normal Gaussian Mixture \\
\mu_s = \frac{\Sigma_{n=1}^{N_{test}}\gamma(z_{ns}x_n)}
{\Sigma_{n=1}^{N_{test}}\gamma(z_{ns})}.
$$
The meaning is very simple. We just use actual $z_{nk}$ instead of $\gamma(z_{ns})$ for taking weighted average step.

Then I finally get
$$
\begin{align}
\mu_s &=\frac{\Sigma_{n=1}^{N_{train}}z_{ns}x_n+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns}x_n)}
{\Sigma_{n=1}^{N_{train}}z_{ns}+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns})} \\
\Sigma_s &=\frac{\Sigma_{n=1}^{N_{train}}z_{ns}(x_n-\mu_s)(x_n-\mu_s)^T+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns}(x_n-\mu_s)(x_n-\mu_s)^T)}
{\Sigma_{n=1}^{N_{train}}z_{ns}+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns})} \\
\pi_s &=\frac{\Sigma_{n=1}^{N_{train}}z_{ns}+\Sigma_{n=1}^{N_{test}}\gamma(z_{ns})}
{\Sigma_t(\Sigma_{n=1}^{N_{train}}z_{nt}+\Sigma_{n=1}^{N_{test}}\gamma(z_{nt}))}
\end{align}
$$
(Sorry, I deduct other than $\mu_s$ intuitively, then they may be wrong).

---
# Load Data

```python
import numpy as np
import pandas as pd

import warnings
warnings.filterwarnings('ignore')

df_train = pd.read_csv('../input/train.csv')
df_test = pd.read_csv('../input/test.csv')

wcrms = df_train['wheezy-copper-turtle-magic'].unique()
```

---
# Modeling

```python
import numpy as np
from scipy import stats

class SSGaussianMixture(object):
    def __init__(self, n_features, n_categories):
        self.n_features = n_features
        self.n_categories = n_categories

        self.mus = np.array([np.random.randn(n_features)]*n_categories)
        self.sigmas = np.array([np.eye(n_features)]*n_categories)
        self.pis = np.array([1/n_categories]*n_categories)

    def fit(self, X_train, y_train, X_test, threshold=0.00001, max_iter=100):
        Z_train = np.eye(self.n_categories)[y_train]

        for i in range(max_iter):
        # EM algorithm
            # M step
            Z_test = np.array([self.gamma(X_test, k) for k in range(self.n_categories)]).T
            Z_test /= Z_test.sum(axis=1, keepdims=True)

            # E step
            datas = [X_train, Z_train, X_test, Z_test]
            mus = np.array([self._est_mu(k, *datas) for k in range(self.n_categories)])
            sigmas = np.array([self._est_sigma(k, *datas) for k in range(self.n_categories)])
            pis = np.array([self._est_pi(k, *datas) for k in range(self.n_categories)])

            diff = max(np.max(np.abs(mus-self.mus)),
                       np.max(np.abs(sigmas-self.sigmas)),
                       np.max(np.abs(pis-self.pis)))
            #print(diff)
            self.mus = mus
            self.sigmas = sigmas
            self.pis = pis
            if diff<threshold:
                break

    def predict_proba(self, X):
        Z_pred = np.array([self.gamma(X, k) for k in range(self.n_categories)]).T
        Z_pred /= Z_pred.sum(axis=1, keepdims=True)
        return Z_pred

    def gamma(self, X, k):
        # X is input vectors, k is feature index
        return stats.multivariate_normal.pdf(X, mean=self.mus[k], cov=self.sigmas[k])

    def _est_mu(self, k, X_train, Z_train, X_test, Z_test):
        mu = (Z_train[:,k]@X_train + Z_test[:,k]@X_test).T / \
                 (Z_train[:,k].sum() + Z_test[:,k].sum())
        return mu

    def _est_sigma(self, k, X_train, Z_train, X_test, Z_test):
        cmp1 = (X_train-self.mus[k]).T@np.diag(Z_train[:,k])@(X_train-self.mus[k])
        cmp2 = (X_test-self.mus[k]).T@np.diag(Z_test[:,k])@(X_test-self.mus[k])
        sigma = (cmp1+cmp2) / (Z_train[:,k].sum() + Z_test[:k].sum())
        return sigma

    def _est_pi(self, k, X_train, Z_train, X_test, Z_test):
        pi = (Z_train[:,k].sum() + Z_test[:,k].sum()) / \
                 (Z_train.sum() + Z_test.sum())
        return pi

```

```python
# Below is just a lapper object.

from sklearn.feature_selection import VarianceThreshold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

from sklearn.metrics import roc_auc_score
from sklearn.model_selection import KFold

class BaseClassifier(object):
    def __init__(self):
        self.preprocess = Pipeline([('vt', VarianceThreshold(threshold=2)), ('scaler', StandardScaler())])

    def fit(self, X_train, y_train, X_test, cv_qda=2, cv_meta=2):
        X_train_org = X_train
        self.preprocess_tune(np.vstack([X_train, X_test]))
        X_train = self.preprocess.transform(X_train)
        X_test = self.preprocess.transform(X_test)

        self.cgm = SSGaussianMixture(n_features=X_train.shape[1], n_categories=2)
        self.validation(X_train_org, y_train)
        self.cgm.fit(X_train, y_train, X_test)

    def predict(self, X):
        X = self.preprocess.transform(X)
        return self.cgm.predict_proba(X)[:,1]

    def preprocess_tune(self, X):
        self.preprocess.fit(X)

    def validation(self, X, y):
        X = self.preprocess.transform(X)
        kf = KFold(n_splits=3, shuffle=True)
        scores = []
        for train_index, test_index in kf.split(X):
            X_train, X_test = X[train_index], X[test_index]
            y_train, y_test = y[train_index], y[test_index]
            self.cgm.fit(X_train, y_train, X_test)
            y_pred = self.cgm.predict_proba(X_test)[:,1]
            scores.append(roc_auc_score(y_test, y_pred))
        self.score = np.array(scores).mean()
        print('validation score = ', self.score)
```

```python
df_train_sample = df_train[df_train['wheezy-copper-turtle-magic']==wcrms[3]]
X_train_sample = df_train_sample.drop(['id', 'target', 'wheezy-copper-turtle-magic'], axis=1).values
y_train_sample = df_train_sample['target'].values

df_test_sample = df_test[df_test['wheezy-copper-turtle-magic']==wcrms[3]]
X_test_sample = df_test_sample.drop(['id', 'wheezy-copper-turtle-magic'], axis=1).values
```

```python
bc = BaseClassifier()
bc.fit(X_train_sample, y_train_sample, X_test_sample)
```

    validation score =  0.9711470469469882

```python
class ConsolEstimator(object):
    def __init__(self, ids):
        self.clfs = {}
        self.id_column = 'wheezy-copper-turtle-magic'
        self.ids = ids

    def predict(self, df_X):
        y_pred = np.zeros(shape=(len(df_X)))
        for id in df_X[self.id_column].unique():
            id_rows = (df_X[self.id_column]==id)
            X = df_X.drop(['id', self.id_column], axis=1).values[id_rows]
            y_pred[id_rows] = self.clfs[id].predict(X)
        return y_pred

    def fit(self, df_train, df_test):
        for i, id in enumerate(self.ids):
            print(i, 'th training...')
            df_train_id = df_train[df_train[self.id_column]==id]
            df_test_id = df_test[df_test[self.id_column]==id]
            if len(df_train_id)==0 or len(df_test_id)==0:
                continue

            X_train = df_train_id.drop(['id', 'target', self.id_column], axis=1).values
            y_train = df_train_id['target'].values
            X_test = df_test_id.drop(['id', self.id_column], axis=1).values

            self.clfs[id] = BaseClassifier()
            self.clfs[id].fit(X_train, y_train, X_test)

        print('mean score = ', np.array([clf.score for clf in self.clfs.values()]).mean())
```

```python
ce = ConsolEstimator(ids=wcrms)
ce.fit(df_train, df_test)
```

    0 th training...
    validation score =  0.9478242204745962
    1 th training...
    validation score =  0.9591406507817898
    2 th training...
    validation score =  0.9619554350593779
    3 th training...
    validation score =  0.9700546301092244
    4 th training...
    validation score =  0.9592800733636414
    5 th training...
    validation score =  0.9452161137149783
    6 th training...
    validation score =  0.969349211535541
    7 th training...
    validation score =  0.9630749669449102
    8 th training...
    validation score =  0.9638470703672554
    9 th training...
    validation score =  0.9637347474197314
    10 th training...
    validation score =  0.9802458847047263
    11 th training...
    validation score =  0.9644841587646885
    12 th training...
    validation score =  0.9630157397117017
    13 th training...
    validation score =  0.9511988940820197
    14 th training...
    validation score =  0.9744691627452476
    15 th training...
    validation score =  0.9609040187145785
    16 th training...
    validation score =  0.9697937734504604
    17 th training...
    validation score =  0.9659163004203205
    18 th training...
    validation score =  0.9525325378038555
    19 th training...
    validation score =  0.9787886424292714
    20 th training...
    validation score =  0.956662739546104
    21 th training...
    validation score =  0.9632907386069359
    22 th training...
    validation score =  0.9794526086899727
    23 th training...
    validation score =  0.9645183816759945
    24 th training...
    validation score =  0.9614860350773812
    25 th training...
    validation score =  0.9574288314729062
    26 th training...
    validation score =  0.9597165760398396
    27 th training...
    validation score =  0.9747739346100337
    28 th training...
    validation score =  0.9712536967959128
    29 th training...
    validation score =  0.9576431778629764
    30 th training...
    validation score =  0.9678323568062179
    31 th training...
    validation score =  0.9730145225214512
    32 th training...
    validation score =  0.9758873998826928
    33 th training...
    validation score =  0.944196880366484
    34 th training...
    validation score =  0.9516411443494777
    35 th training...
    validation score =  0.9673754482141499
    36 th training...
    validation score =  0.9676628074185109
    37 th training...
    validation score =  0.9543428855933906
    38 th training...
    validation score =  0.9682517159127849
    39 th training...
    validation score =  0.9627549442399549
    40 th training...
    validation score =  0.9706285986247377
    41 th training...
    validation score =  0.9642791772480636
    42 th training...
    validation score =  0.964911074124266
    43 th training...
    validation score =  0.9692228867631353
    44 th training...
    validation score =  0.9699072059187924
    45 th training...
    validation score =  0.9612767608031207
    46 th training...
    validation score =  0.9739711901002224
    47 th training...
    validation score =  0.9783881244595771
    48 th training...
    validation score =  0.9811234387451182
    49 th training...
    validation score =  0.947622303755084
    50 th training...
    validation score =  0.936548435787253
    51 th training...
    validation score =  0.9596742021276596
    52 th training...
    validation score =  0.9608738544265552
    53 th training...
    validation score =  0.9694155282335456
    54 th training...
    validation score =  0.9559471861205578
    55 th training...
    validation score =  0.9687710418544572
    56 th training...
    validation score =  0.9601431430235778
    57 th training...
    validation score =  0.9678672256221118
    58 th training...
    validation score =  0.9832925666704044
    59 th training...
    validation score =  0.9594099081346457
    60 th training...
    validation score =  0.9716286255237653
    61 th training...
    validation score =  0.9673382447125327
    62 th training...
    validation score =  0.9615522232131042
    63 th training...
    validation score =  0.9637032577062562
    64 th training...
    validation score =  0.9751262505321413
    65 th training...
    validation score =  0.9462954848325649
    66 th training...
    validation score =  0.9734078440000943
    67 th training...
    validation score =  0.9496625034163623
    68 th training...
    validation score =  0.9646340897658194
    69 th training...
    validation score =  0.9570533295420806
    70 th training...
    validation score =  0.9673555264730845
    71 th training...
    validation score =  0.9523382643306061
    72 th training...
    validation score =  0.940336151387565
    73 th training...
    validation score =  0.9683651446778784
    74 th training...
    validation score =  0.9841312311416228
    75 th training...
    validation score =  0.9707257675061051
    76 th training...
    validation score =  0.9594545225257374
    77 th training...
    validation score =  0.9567206307709109
    78 th training...
    validation score =  0.9628438819878999
    79 th training...
    validation score =  0.9636227627973616
    80 th training...
    validation score =  0.9888587278059201
    81 th training...
    validation score =  0.977171237759694
    82 th training...
    validation score =  0.967968398428388
    83 th training...
    validation score =  0.9706467802473818
    84 th training...
    validation score =  0.9475300191269526
    85 th training...
    validation score =  0.9570874739763525
    86 th training...
    validation score =  0.969395457374484
    87 th training...
    validation score =  0.9636732769746446
    88 th training...
    validation score =  0.9721263493092659
    89 th training...
    validation score =  0.9564786439602778
    90 th training...
    validation score =  0.9682945850905863
    91 th training...
    validation score =  0.9709710443508609
    92 th training...
    validation score =  0.9657445607735591
    93 th training...
    validation score =  0.9692541502748085
    94 th training...
    validation score =  0.9713684954158904
    95 th training...
    validation score =  0.9711486637012269
    96 th training...
    validation score =  0.9685974838862487
    97 th training...
    validation score =  0.9718497347596572
    98 th training...
    validation score =  0.9837915744611957
    99 th training...
    validation score =  0.9645974803168813
    100 th training...
    validation score =  0.9829724532365822
    101 th training...
    validation score =  0.977085765465218
    102 th training...
    validation score =  0.9862052429667519
    103 th training...
    validation score =  0.9652438739638732
    104 th training...
    validation score =  0.9560347281756857
    105 th training...
    validation score =  0.968809809722603
    106 th training...
    validation score =  0.9674604004567205
    107 th training...
    validation score =  0.9710192021056997
    108 th training...
    validation score =  0.9724253276082545
    109 th training...
    validation score =  0.9850669864270717
    110 th training...
    validation score =  0.9651700601347134
    111 th training...
    validation score =  0.9552919026304618
    112 th training...
    validation score =  0.9525015741573322
    113 th training...
    validation score =  0.9650953076095754
    114 th training...
    validation score =  0.9364818180161962
    115 th training...
    validation score =  0.9685686866903523
    116 th training...
    validation score =  0.9679511601829458
    117 th training...
    validation score =  0.9550786844139146
    118 th training...
    validation score =  0.9829245034741794
    119 th training...
    validation score =  0.9702954825695828
    120 th training...
    validation score =  0.9626001596994399
    121 th training...
    validation score =  0.9704277904734058
    122 th training...
    validation score =  0.975094475368793
    123 th training...
    validation score =  0.9579295449706237
    124 th training...
    validation score =  0.9566518168751547
    125 th training...
    validation score =  0.9778755449558251
    126 th training...
    validation score =  0.9687791446122417
    127 th training...
    validation score =  0.9690414488071655
    128 th training...
    validation score =  0.9566566047082142
    129 th training...
    validation score =  0.9641866707578544
    130 th training...
    validation score =  0.9667181512646708
    131 th training...
    validation score =  0.9569413099533804
    132 th training...
    validation score =  0.9683754156781698
    133 th training...
    validation score =  0.9663048272152973
    134 th training...
    validation score =  0.9586757838168989
    135 th training...
    validation score =  0.9699533980798541
    136 th training...
    validation score =  0.9738493247353034
    137 th training...
    validation score =  0.9557322190338491
    138 th training...
    validation score =  0.9662484862234851
    139 th training...
    validation score =  0.9649763913306755
    140 th training...
    validation score =  0.9672205097397292
    141 th training...
    validation score =  0.9463979385166651
    142 th training...
    validation score =  0.9645178239188427
    143 th training...
    validation score =  0.9567987132476632
    144 th training...
    validation score =  0.9524194622544075
    145 th training...
    validation score =  0.9687042040971376
    146 th training...
    validation score =  0.962121212121212
    147 th training...
    validation score =  0.9659904560969282
    148 th training...
    validation score =  0.9697845167172567
    149 th training...
    validation score =  0.9573295307341536
    150 th training...
    validation score =  0.9462819149659705
    151 th training...
    validation score =  0.9754776742238036
    152 th training...
    validation score =  0.9708730879813258
    153 th training...
    validation score =  0.9686875847832553
    154 th training...
    validation score =  0.9677878832118778
    155 th training...
    validation score =  0.9385863496835455
    156 th training...
    validation score =  0.9314713684985306
    157 th training...
    validation score =  0.9662989805733712
    158 th training...
    validation score =  0.9857674975435032
    159 th training...
    validation score =  0.9561000010631159
    160 th training...
    validation score =  0.9582539208396845
    161 th training...
    validation score =  0.9717102033846983
    162 th training...
    validation score =  0.9634566257024204
    163 th training...
    validation score =  0.9611654052419755
    164 th training...
    validation score =  0.9694653882241452
    165 th training...
    validation score =  0.9573781463626672
    166 th training...
    validation score =  0.9644210696446122
    167 th training...
    validation score =  0.9466807187494867
    168 th training...
    validation score =  0.9665967485116959
    169 th training...
    validation score =  0.9694117230985126
    170 th training...
    validation score =  0.9811174629094683
    171 th training...
    validation score =  0.9551321082564922
    172 th training...
    validation score =  0.9589394185453299
    173 th training...
    validation score =  0.9646728849575683
    174 th training...
    validation score =  0.9804860115683437
    175 th training...
    validation score =  0.955501384653684
    176 th training...
    validation score =  0.95403218225906
    177 th training...
    validation score =  0.9644939725329237
    178 th training...
    validation score =  0.9656616868527147
    179 th training...
    validation score =  0.9704646600265008
    180 th training...
    validation score =  0.9705905318520965
    181 th training...
    validation score =  0.9625557529085259
    182 th training...
    validation score =  0.9625961880743911
    183 th training...
    validation score =  0.9677963852736443
    184 th training...
    validation score =  0.9591234323035677
    185 th training...
    validation score =  0.9690600230966876
    186 th training...
    validation score =  0.9637184622953123
    187 th training...
    validation score =  0.9627092204792964
    188 th training...
    validation score =  0.9641779855720246
    189 th training...
    validation score =  0.9599011016870964
    190 th training...
    validation score =  0.9734119032364646
    191 th training...
    validation score =  0.9497360393513748
    192 th training...
    validation score =  0.9804277604186377
    193 th training...
    validation score =  0.9654990658922333
    194 th training...
    validation score =  0.9635461267335211
    195 th training...
    validation score =  0.9600012496581124
    196 th training...
    validation score =  0.9679496301646271
    197 th training...
    validation score =  0.9629218151600921
    198 th training...
    validation score =  0.9770768151134387
    199 th training...
    validation score =  0.9719671163548692
    200 th training...
    validation score =  0.9720962108560428
    201 th training...
    validation score =  0.9442675685317138
    202 th training...
    validation score =  0.9612049046940153
    203 th training...
    validation score =  0.9389899924848305
    204 th training...
    validation score =  0.975855855571706
    205 th training...
    validation score =  0.9872290628770175
    206 th training...
    validation score =  0.9745721352668655
    207 th training...
    validation score =  0.9676470230702284
    208 th training...
    validation score =  0.9827954292522655
    209 th training...
    validation score =  0.9670454104279633
    210 th training...
    validation score =  0.9509242840583362
    211 th training...
    validation score =  0.950827798144962
    212 th training...
    validation score =  0.9591670016005743
    213 th training...
    validation score =  0.961981066686123
    214 th training...
    validation score =  0.966545998090285
    215 th training...
    validation score =  0.964733700323752
    216 th training...
    validation score =  0.985451624205529
    217 th training...
    validation score =  0.9433831292092911
    218 th training...
    validation score =  0.9748695915363305
    219 th training...
    validation score =  0.9693575106347948
    220 th training...
    validation score =  0.963597161748634
    221 th training...
    validation score =  0.955675959501742
    222 th training...
    validation score =  0.9572240709821989
    223 th training...
    validation score =  0.9436648268512874
    224 th training...
    validation score =  0.9699283421915976
    225 th training...
    validation score =  0.9584658678286129
    226 th training...
    validation score =  0.9723711598746082
    227 th training...
    validation score =  0.9649564467852073
    228 th training...
    validation score =  0.9602945042162586
    229 th training...
    validation score =  0.9638186604229592
    230 th training...
    validation score =  0.9702728750938627
    231 th training...
    validation score =  0.9561936873851553
    232 th training...
    validation score =  0.9482679936569499
    233 th training...
    validation score =  0.9729643826191955
    234 th training...
    validation score =  0.9669807489700344
    235 th training...
    validation score =  0.9626428430858626
    236 th training...
    validation score =  0.9654046262268768
    237 th training...
    validation score =  0.9739597829256273
    238 th training...
    validation score =  0.9722030444517848
    239 th training...
    validation score =  0.9730931911453359
    240 th training...
    validation score =  0.9758627041442005
    241 th training...
    validation score =  0.978965827686706
    242 th training...
    validation score =  0.9621982604822037
    243 th training...
    validation score =  0.9787906792631965
    244 th training...
    validation score =  0.9553523194263417
    245 th training...
    validation score =  0.9654542833705779
    246 th training...
    validation score =  0.9654071007480436
    247 th training...
    validation score =  0.9631375777815343
    248 th training...
    validation score =  0.9630068944849444
    249 th training...
    validation score =  0.9661556739916004
    250 th training...
    validation score =  0.9650309890926964
    251 th training...
    validation score =  0.9729419796553028
    252 th training...
    validation score =  0.9690587262518048
    253 th training...
    validation score =  0.9533262592435556
    254 th training...
    validation score =  0.9395347171767973
    255 th training...
    validation score =  0.9554624684887653
    256 th training...
    validation score =  0.9745541768973854
    257 th training...
    validation score =  0.9404544755198136
    258 th training...
    validation score =  0.9645988363686341
    259 th training...
    validation score =  0.9774179374905105
    260 th training...
    validation score =  0.9516703215280754
    261 th training...
    validation score =  0.9618746863700377
    262 th training...
    validation score =  0.9660593593887746
    263 th training...
    validation score =  0.9674448354144017
    264 th training...
    validation score =  0.9784658143134681
    265 th training...
    validation score =  0.9724188418949579
    266 th training...
    validation score =  0.9596368993171206
    267 th training...
    validation score =  0.9759849519725469
    268 th training...
    validation score =  0.9681291624300196
    269 th training...
    validation score =  0.9608782293556999
    270 th training...
    validation score =  0.964616741360984
    271 th training...
    validation score =  0.9547808543540558
    272 th training...
    validation score =  0.9725587106009798
    273 th training...
    validation score =  0.9691933560893107
    274 th training...
    validation score =  0.9715733637895744
    275 th training...
    validation score =  0.9519221786907944
    276 th training...
    validation score =  0.9656040386688716
    277 th training...
    validation score =  0.937483067774715
    278 th training...
    validation score =  0.963214049180242
    279 th training...
    validation score =  0.9746444155653101
    280 th training...
    validation score =  0.9615902865814493
    281 th training...
    validation score =  0.9663646748286118
    282 th training...
    validation score =  0.949520747138506
    283 th training...
    validation score =  0.9636386306888852
    284 th training...
    validation score =  0.9631467474763467
    285 th training...
    validation score =  0.9654583869323307
    286 th training...
    validation score =  0.9529360169541459
    287 th training...
    validation score =  0.9558863350212025
    288 th training...
    validation score =  0.9740135154746161
    289 th training...
    validation score =  0.9661729996453069
    290 th training...
    validation score =  0.9663739471975598
    291 th training...
    validation score =  0.9783953037494504
    292 th training...
    validation score =  0.9640458778225781
    293 th training...
    validation score =  0.9457997817287325
    294 th training...
    validation score =  0.9710821531232217
    295 th training...
    validation score =  0.9806307747236244
    296 th training...
    validation score =  0.9538837451995944
    297 th training...
    validation score =  0.962526862678482
    298 th training...
    validation score =  0.9623950744984809
    299 th training...
    validation score =  0.9471578515135683
    300 th training...
    validation score =  0.9614606695178983
    301 th training...
    validation score =  0.9597549042532713
    302 th training...
    validation score =  0.9650513439475384
    303 th training...
    validation score =  0.9727388570908588
    304 th training...
    validation score =  0.9579726237077185
    305 th training...
    validation score =  0.9713536867435636
    306 th training...
    validation score =  0.9660534481949169
    307 th training...
    validation score =  0.9444446298882953
    308 th training...
    validation score =  0.9544741404263887
    309 th training...
    validation score =  0.9686191605895544
    310 th training...
    validation score =  0.9582931424710232
    311 th training...
    validation score =  0.9661803513617869
    312 th training...
    validation score =  0.9783595929507407
    313 th training...
    validation score =  0.9861668102504098
    314 th training...
    validation score =  0.9775839801749795
    315 th training...
    validation score =  0.9574492609082467
    316 th training...
    validation score =  0.9629735836959963
    317 th training...
    validation score =  0.955352170330475
    318 th training...
    validation score =  0.977686209991521
    319 th training...
    validation score =  0.975467894413247
    320 th training...
    validation score =  0.9558709305502759
    321 th training...
    validation score =  0.9655916764869029
    322 th training...
    validation score =  0.9655045860610728
    323 th training...
    validation score =  0.9516719236573726
    324 th training...
    validation score =  0.9502046687756524
    325 th training...
    validation score =  0.9728870467301726
    326 th training...
    validation score =  0.9709859653608813
    327 th training...
    validation score =  0.9685627565858289
    328 th training...
    validation score =  0.9715255325627798
    329 th training...
    validation score =  0.9580486839519361
    330 th training...
    validation score =  0.9593152672165567
    331 th training...
    validation score =  0.9598028387594715
    332 th training...
    validation score =  0.9687559415780832
    333 th training...
    validation score =  0.982424830787445
    334 th training...
    validation score =  0.9705090982223004
    335 th training...
    validation score =  0.9861906850114065
    336 th training...
    validation score =  0.9538877210288597
    337 th training...
    validation score =  0.9726902831252392
    338 th training...
    validation score =  0.9364806028994671
    339 th training...
    validation score =  0.9582826002396828
    340 th training...
    validation score =  0.9620699537812326
    341 th training...
    validation score =  0.9724093615746892
    342 th training...
    validation score =  0.9714375572387507
    343 th training...
    validation score =  0.9652932917595138
    344 th training...
    validation score =  0.9772534823546666
    345 th training...
    validation score =  0.9737965597631458
    346 th training...
    validation score =  0.9565530064413972
    347 th training...
    validation score =  0.9631294448764486
    348 th training...
    validation score =  0.9441986259218239
    349 th training...
    validation score =  0.9705444370461738
    350 th training...
    validation score =  0.9639910094104823
    351 th training...
    validation score =  0.9640291621841103
    352 th training...
    validation score =  0.9599062835174577
    353 th training...
    validation score =  0.9732503759658432
    354 th training...
    validation score =  0.9871848714128366
    355 th training...
    validation score =  0.9723688043769328
    356 th training...
    validation score =  0.9462288487297789
    357 th training...
    validation score =  0.9605185092524229
    358 th training...
    validation score =  0.9619060057990253
    359 th training...
    validation score =  0.9773869367587711
    360 th training...
    validation score =  0.9477912291682178
    361 th training...
    validation score =  0.9585114034013197
    362 th training...
    validation score =  0.9479955360081473
    363 th training...
    validation score =  0.951378891118484
    364 th training...
    validation score =  0.9763422279568569
    365 th training...
    validation score =  0.9745255654859486
    366 th training...
    validation score =  0.968220939375125
    367 th training...
    validation score =  0.9680960003592328
    368 th training...
    validation score =  0.9632270599728504
    369 th training...
    validation score =  0.9619790335093409
    370 th training...
    validation score =  0.9749495224317214
    371 th training...
    validation score =  0.962653157880093
    372 th training...
    validation score =  0.9677305291099619
    373 th training...
    validation score =  0.9696064746222102
    374 th training...
    validation score =  0.9581335953734974
    375 th training...
    validation score =  0.9571281422492741
    376 th training...
    validation score =  0.956469061487924
    377 th training...
    validation score =  0.9718049533468284
    378 th training...
    validation score =  0.9674387432145508
    379 th training...
    validation score =  0.9538422943799771
    380 th training...
    validation score =  0.9696744044513997
    381 th training...
    validation score =  0.9594522108814667
    382 th training...
    validation score =  0.9633739598095746
    383 th training...
    validation score =  0.9583808631491261
    384 th training...
    validation score =  0.973362648221344
    385 th training...
    validation score =  0.9617211199286239
    386 th training...
    validation score =  0.9740344930242628
    387 th training...
    validation score =  0.9700393184600079
    388 th training...
    validation score =  0.9576239084543893
    389 th training...
    validation score =  0.9671341252666554
    390 th training...
    validation score =  0.9567894750532067
    391 th training...
    validation score =  0.9742805937066613
    392 th training...
    validation score =  0.979996634240531
    393 th training...
    validation score =  0.970785031206606
    394 th training...
    validation score =  0.9462331249395864
    395 th training...
    validation score =  0.9696956963649779
    396 th training...
    validation score =  0.9658736095945398
    397 th training...
    validation score =  0.9624733513880699
    398 th training...
    validation score =  0.9593553747382288
    399 th training...
    validation score =  0.9670140486311326
    400 th training...
    validation score =  0.9689746515071774
    401 th training...
    validation score =  0.9748972545667347
    402 th training...
    validation score =  0.970900325430558
    403 th training...
    validation score =  0.9552124212530564
    404 th training...
    validation score =  0.9651577180988946
    405 th training...
    validation score =  0.9731698508744729
    406 th training...
    validation score =  0.9561347602243515
    407 th training...
    validation score =  0.9624075839905597
    408 th training...
    validation score =  0.9660147229225471
    409 th training...
    validation score =  0.9551550898312454
    410 th training...
    validation score =  0.9492856715798111
    411 th training...
    validation score =  0.9720612088953472
    412 th training...
    validation score =  0.9592934239354703
    413 th training...
    validation score =  0.9740200062691584
    414 th training...
    validation score =  0.9659993681788771
    415 th training...
    validation score =  0.9503655564128427
    416 th training...
    validation score =  0.9637715918900166
    417 th training...
    validation score =  0.9681620899716262
    418 th training...
    validation score =  0.9738715848889061
    419 th training...
    validation score =  0.9667561460184185
    420 th training...
    validation score =  0.9570856017477388
    421 th training...
    validation score =  0.96726686710275
    422 th training...
    validation score =  0.9729568884601276
    423 th training...
    validation score =  0.9788289356907028
    424 th training...
    validation score =  0.9814815145613109
    425 th training...
    validation score =  0.9712874358509042
    426 th training...
    validation score =  0.9768915831093633
    427 th training...
    validation score =  0.9775678426760958
    428 th training...
    validation score =  0.9779730781582384
    429 th training...
    validation score =  0.9652971007318834
    430 th training...
    validation score =  0.9753360807308438
    431 th training...
    validation score =  0.9561107772968126
    432 th training...
    validation score =  0.9792419126164232
    433 th training...
    validation score =  0.964748738054277
    434 th training...
    validation score =  0.9751960015780549
    435 th training...
    validation score =  0.9553977690244206
    436 th training...
    validation score =  0.9654448914585386
    437 th training...
    validation score =  0.9702747535355605
    438 th training...
    validation score =  0.9694748254132882
    439 th training...
    validation score =  0.9635892381644015
    440 th training...
    validation score =  0.9773615204367765
    441 th training...
    validation score =  0.9488984829775206
    442 th training...
    validation score =  0.972981466213669
    443 th training...
    validation score =  0.977678434671247
    444 th training...
    validation score =  0.954399766041468
    445 th training...
    validation score =  0.972155451327818
    446 th training...
    validation score =  0.9658976984647483
    447 th training...
    validation score =  0.9453399304480029
    448 th training...
    validation score =  0.9466809995112451
    449 th training...
    validation score =  0.9687569080617333
    450 th training...
    validation score =  0.9725301852953706
    451 th training...
    validation score =  0.9680523838465064
    452 th training...
    validation score =  0.966478287092205
    453 th training...
    validation score =  0.9704534367216988
    454 th training...
    validation score =  0.9742013905230866
    455 th training...
    validation score =  0.9697518005394675
    456 th training...
    validation score =  0.9547656639487311
    457 th training...
    validation score =  0.9758555433267443
    458 th training...
    validation score =  0.9724600712013626
    459 th training...
    validation score =  0.9733174040687502
    460 th training...
    validation score =  0.9562587164220702
    461 th training...
    validation score =  0.9575277617107858
    462 th training...
    validation score =  0.9609551258232291
    463 th training...
    validation score =  0.968867709246709
    464 th training...
    validation score =  0.9531257907332584
    465 th training...
    validation score =  0.9721192520843251
    466 th training...
    validation score =  0.9714357726439875
    467 th training...
    validation score =  0.9652104921623818
    468 th training...
    validation score =  0.9519842181060619
    469 th training...
    validation score =  0.9341674131209015
    470 th training...
    validation score =  0.9700209053073316
    471 th training...
    validation score =  0.9513728488999522
    472 th training...
    validation score =  0.9798117832611933
    473 th training...
    validation score =  0.9592700265215374
    474 th training...
    validation score =  0.963089179316357
    475 th training...
    validation score =  0.9766632794693951
    476 th training...
    validation score =  0.9720239555013331
    477 th training...
    validation score =  0.9764579204979272
    478 th training...
    validation score =  0.9496382943416383
    479 th training...
    validation score =  0.9762300971044038
    480 th training...
    validation score =  0.9753860224019933
    481 th training...
    validation score =  0.9607134680995046
    482 th training...
    validation score =  0.9622216975172403
    483 th training...
    validation score =  0.956193676792219
    484 th training...
    validation score =  0.9632995914434148
    485 th training...
    validation score =  0.9649750862731956
    486 th training...
    validation score =  0.9705539857778679
    487 th training...
    validation score =  0.9499683444127888
    488 th training...
    validation score =  0.9658441238688152
    489 th training...
    validation score =  0.976416618913284
    490 th training...
    validation score =  0.9854711794568268
    491 th training...
    validation score =  0.9651414627111041
    492 th training...
    validation score =  0.9585068527922527
    493 th training...
    validation score =  0.9626638286788932
    494 th training...
    validation score =  0.9734313882547694
    495 th training...
    validation score =  0.9677428767088921
    496 th training...
    validation score =  0.9747147366966945
    497 th training...
    validation score =  0.972518430383599
    498 th training...
    validation score =  0.9628694396825815
    499 th training...
    validation score =  0.9751837365246679
    500 th training...
    validation score =  0.9612539937018157
    501 th training...
    validation score =  0.9568603728367089
    502 th training...
    validation score =  0.9494535907540219
    503 th training...
    validation score =  0.9594794974915208
    504 th training...
    validation score =  0.9728658300486384
    505 th training...
    validation score =  0.972155039672547
    506 th training...
    validation score =  0.9798740337061179
    507 th training...
    validation score =  0.9723374344269736
    508 th training...
    validation score =  0.9593285835141471
    509 th training...
    validation score =  0.963550557629505
    510 th training...
    validation score =  0.9559764072082598
    511 th training...
    validation score =  0.9685118193562353
    mean score =  0.9649141663521423

---
# Submission

```python
y_pred = ce.predict(df_test)
```

```python
df_submission = pd.concat([df_test['id'], pd.Series(y_pred, name='target')], axis=1)
df_submission.to_csv('submission.csv', index=False)
```

I tried hyper parameter tuning and bugging, but LB score didn't improve.
When I used BayesianGaussianMixture instead of GaussianMixture in stacking, my CV score improved, then if we can treat above model in Bayesian, score may improve.
But for me, treating Gauss-Wishart distribution is hard (sklean implementation of GayesianGaussianMixture can be used as a refference).


```python
from sympy.physics.units import *
from sympy import *

import decimal
from decimal import Decimal as DX
def iso_round(obj, pv, rounding=decimal.ROUND_HALF_EVEN):
    import sympy
    """
    Rounding according to: DIN EN ISO 80000-1:2013-08
    pv: place value = Rundestellenwert
    """
    # pv     :  Rounded digit
    # 1      :  last digit before decimal
    # 0.1    :  1st digit after decimal
    # 0.01   :  2nd digit
    # 0.001  :  3rd digit
    assert pv in set([ # round to:
        100,           #  3rd last digit before decimal
        10,            #  2nd last
        1,             #  last
        0.1,           #  1st digit after decimal
        0.01,          #  2nd
        0.001,         #  3rd
        0.0001,        #  4th
        0.00001,       #  5th
        0.000001,      #  6th
        0.0000001,     #  7th
        0.00000001,    #  8th
        0.000000001,   #  9th
        0.0000000001,  # 10th
        ])
    try:
        tmp = float(obj)
        tmp = DX(str(tmp))
        obj = tmp.quantize(DX(str(pv)), rounding=rounding)
    except:
        for i in range(len(obj)):
            tmp = float(obj[i])
            tmp = DX(str(tmp))
            obj[i] = tmp.quantize(DX(str(pv)), rounding=rounding)
    return obj

# LateX:
kwargs = {"mat_str": "bmatrix",  "mat_delim": ""}

kilo = 1000
mega = 1000*1000
giga = 1000*1000*1000
(mm, cm) = (m/1000, m/100)

deg = pi/180
Newton = kg*m/s**2
Pa = Newton/m**2
MPa = mega*Pa
GPa = giga*Pa
kN = kilo*Newton

###

half = S(1)/2

l, EI, q = var("l, EI, q")
a1, a2 = var("a1, a2")

x = var("x")

# deflection:
l2 = l*l
x2 = x**2
x3 = x**3

t1 = a1*(x2 - l*x)
t2 = a2*(x3 - l2*x)

w = t1 + t2
w1 = diff(w,  x)
w2 = diff(w1, x)

pprint("\nUi:")
Ui = half * EI * integrate(w2*w2, (x, 0, l))
pprint(Ui)

pprint("\nUa:")
Ua =  integrate(q*w, (x, 0, l))
pprint(Ua)

U = Ui - Ua

dUda1 = diff(U,a1)
dUda2 = diff(U,a2)

eq1 = Eq(dUda1,0)
eq2 = Eq(dUda2,0)

pprint("\nEq 1:")
pprint(eq1)
pprint("\nEq 2:")
pprint(eq2)
sol = solve([eq1,eq2], [a1,a2])

a1s = sol[a1]
a2s = sol[a2]
pprint("\na1:")
pprint(a1s)

pprint("\na2:")
pprint(a2s)

w = w.subs({a1: a1s, a2:a2s})
w = w.simplify()
pprint("\nw(x):")
pprint(w)
pprint("\nw(x=L/2):")
pprint(w.subs(x,half*l))

# Ui:
#    ⎛    2               2        2  3⎞
# EI⋅⎝4⋅a₁ ⋅l + 12⋅a₁⋅a₂⋅l  + 12⋅a₂ ⋅l ⎠
# ──────────────────────────────────────
#                   2
#
# Ua:
#     3         4        ⎛               2  ⎞
# a₁⋅l ⋅q   a₂⋅l ⋅q    2 ⎜  a₁⋅l⋅q   a₂⋅l ⋅q⎟
# ─────── + ─────── + l ⋅⎜- ────── - ───────⎟
#    3         4         ⎝    2         2   ⎠
#
# Eq 1:
#    ⎛                2⎞    3
# EI⋅⎝8⋅a₁⋅l + 12⋅a₂⋅l ⎠   l ⋅q
# ────────────────────── + ──── = 0
#           2               6
#
# Eq 2:
#    ⎛       2          3⎞    4
# EI⋅⎝12⋅a₁⋅l  + 24⋅a₂⋅l ⎠   l ⋅q
# ──────────────────────── + ──── = 0
#            2                4
#
# a1:
#   2
# -l ⋅q
# ──────
# 24⋅EI
#
# a2:
# 0
#
# w(x):
#  2
# l ⋅q⋅x⋅(l - x)
# ──────────────
#     24⋅EI
#
# w(x=L/2):
#   4
#  l ⋅q
# ─────
# 96⋅EI

```


Testing derivaties for 1D MT problem.

Especially the rx.projectFieldsDeriv

```
import SimPEG as simpeg
import simpegEM as simpegem, simpegMT as simpegmt
from SimPEG.Utils import meshTensor
import numpy as np
```

```
simpegmt.FieldsMT.FieldsMT_1D
```

    simpegMT.FieldsMT.FieldsMT_1D

```
# Setup the problem
sigmaHalf = 1e-2
# Frequency
nFreq = 33
# freqs = np.logspace(3,-3,nFreq)
freqs = np.array([100])
# Make the mesh
ct = 5
air = meshTensor([(ct,25,1.3)])
# coreT0 = meshTensor([(ct,15,1.2)])
# coreT1 = np.kron(meshTensor([(coreT0[-1],15,1.3)]),np.ones((7,)))
core = np.concatenate( (  np.kron(meshTensor([(ct,15,-1.2)]),np.ones((10,))) , meshTensor([(ct,20)]) ) )
bot = meshTensor([(core[0],10,-1.3)])
x0 = -np.array([np.sum(np.concatenate((core,bot)))])
# Change to use no air
m1d = simpeg.Mesh.TensorMesh([np.concatenate((bot,core))], x0=x0)
# Make the model
sigma = np.zeros(m1d.nC) + sigmaHalf
sigma[ m1d.gridCC > 0 ] = 1e-8

rxList = []
for rxType in ['z1dr','z1di']:
    rxList.append(simpegmt.SurveyMT.RxMT(simpeg.mkvc(np.array([0.0]),2).T,rxType))
# Source list
srcList =[]
tD = False
if tD:
    for freq in freqs:
        srcList.append(simpegmt.SurveyMT.srcMT_polxy_1DhomotD(rxList,freq))
else:
    for freq in freqs:
        srcList.append(simpegmt.SurveyMT.srcMT_polxy_1Dprimary(rxList,freq))
# Make the survey
survey = simpegmt.SurveyMT.SurveyMT(srcList)

# Set the problem
problem = simpegmt.ProblemMT1D.eForm_psField(m1d)
problem.sigmaPrimary = sigma
problem.pair(survey)

# Get the fields
fields = problem.fields(sigma)

# Project the data
data = survey.projectFields(fields)

```

```
%debug

```

    ERROR: No traceback has been produced, nothing to debug.

We need calculate this derivative.
\begin{align}
\underbrace{\frac{\partial P(f(u(m)),m^{fix})}{\partial f}}_{Rx}
\end{align}

Use the rule
\begin{align}
\frac{d}{dx}\left( \frac{a(x)}{b(x)} \right)  = \frac{\frac{d }{dx} a(x)  b(x) - a(x)\frac{d }{dx} b(x)  }{ b(x)^2 }
\end{align}

In the case of the 1D MT problem the data is calculated as
\begin{align}
MT1Ddata = P(f(m)) &= \frac{P_{ex} f_e(src,m)}{P_{bx} f_b(src,m) \frac{1}{\mu_0}} = \frac{P_e u}{P_b f_b(u)} \\
\frac{\partial P(f(m))}{\partial u} v &=  \frac{P_e}{P_b \frac{1}{mu_0} f_b(u)}v - \frac{P_e u}{\left(P_b \frac{1}{mu_0} f_b(u)\right)^2} P_b \frac{1}{mu_0} \frac{d f_b}{du} v
\end{align}
where u is the fields that we solve for.
\begin{align}
\frac{d f_b}{du} = - \frac{1}{i \omega} \nabla
\end{align}

```
# Unused code &= \frac{ P_{ex} P_{bx} \frac{1}{\mu_0} \left( f_b(src,m) - f_e(src,m) \right) } { \left(P_{bx}f_b(src,m) \frac{1}{\mu_0} \right)^2 }
```

As matrices the formulas above can be written as
\begin{align}
\left[ \frac{\partial P(f(m))}{\partial u} v \right] = \left[ diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right] [P_e v] , diag[P_e u] diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right]^T diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right] \left[ P_b  \frac{1}{mu_0} \frac{d f_b}{du}(v)  \right] \right]
\end{align}

The adjoint problem is done simliarly
\begin{align}
\left[ \frac{\partial P(f(m))}{\partial u} v \right]^T = [P_e]^T diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right]^T v - \left[ P_b   \frac{d f_b}{du} \frac{1}{mu_0} \right]^T diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right] diag \left[ \frac{1}{\left(P_b \frac{1}{mu_0} f_b(u)\right)} \right]^T diag \left[ P_e u  \right]^T v
\end{align}

```
# def projectFields(self, src, mesh, u):
#         '''
#         Project the fields and return the
#         '''

#         if self.projType is 'Z1D':
#             Pex = mesh.getInterpolationMat(self.locs,'Fx')
#             Pbx = mesh.getInterpolationMat(self.locs,'Ex')
#             ex = Pex*mkvc(u[src,'e_1d'],2)
#             bx = Pbx*mkvc(u[src,'b_1d'],2)/mu_0
#             f_part_complex = ex/bx
#         real_or_imag = self.projComp
#         f_part = getattr(f_part_complex, real_or_imag)
#         return f_part
```

```
# Initate things for the derivs Test
src = survey.srcList[0]
rx = src.rxList[0]
v = np.random.randn(m1d.nN)
v0 = np.random.randn(m1d.nF+m1d.nE)
u0 = np.random.randn(m1d.nN)+np.random.randn(m1d.nN)*1j
f0 = problem.fieldsPair(m1d,survey)
f0[src,'e_1dSolution'] = u0
# f0[src,'b_1d'] = -1/(1j*simpegem.Utils.EMUtils.omega(src.freq))*m1d.nodalGrad*u0
```

```
# Run a test
def fun(u):
    f = problem.fieldsPair(m1d,survey)
    f[src,'e_1dSolution'] = u
    return rx.projectFields(src,m1d,f), lambda t: rx.projectFieldsDeriv(src,m1d,f0,t)
simpeg.Tests.checkDerivative(fun,u0,num=5,plotIt=False)
```

    ==================== checkDerivative ====================
    iter    h         |ft-f0|   |ft-f0-h*J0*dx|  Order
    ---------------------------------------------------------
     0   1.00e-01    3.089e-04     2.730e-05      nan
     1   1.00e-02    2.841e-05     2.582e-07      2.024
     2   1.00e-03    2.818e-06     2.568e-09      2.002
     3   1.00e-04    2.816e-07     2.567e-11      2.000
     4   1.00e-05    2.816e-08     2.567e-13      2.000
    ========================= PASS! =========================
    The test be workin!

    True

```
rx.projectFieldsDeriv(src,m1d,f0,u0)
```

    array([-0.00285083])

```
rx.projectFields(src,m1d,f0)
```

    array([[-0.00130618]])

```
# Test the Jvec derivative.
```

```
# print '%s formulation - %s' % (fdemType, comp)
CONDUCTIVITY = 0.01
m0 = np.log(np.ones(problem.mesh.nC)*CONDUCTIVITY)
# mu = np.log(np.ones(problem.mesh.nC)*MU)

if True:
    m0  = m0 + np.random.randn(problem.mesh.nC)*CONDUCTIVITY*1e-1
#     mu = mu + np.random.randn(prb.mesh.nC)*MU*1e-1

# prb.mu = mu
# survey = prb.survey
def fun(x):

    return survey.dpred(x), lambda x: problem.Jvec(m0, x)
simpeg.Tests.checkDerivative(fun, m0, num=4, plotIt=False)
```

    ==================== checkDerivative ====================
    iter    h         |ft-f0|   |ft-f0-h*J0*dx|  Order
    ---------------------------------------------------------
     0   1.00e-01    1.537e-05     1.005e-07      nan
     1   1.00e-02    1.541e-06     1.008e-09      1.999
     2   1.00e-03    1.541e-07     1.008e-11      2.000
     3   1.00e-04    1.541e-08     1.008e-13      2.000
    ========================= PASS! =========================
    The test be workin!

    True

```
%debug
```

    ERROR: No traceback has been produced, nothing to debug.

```
### Adjoint test
```

We have
\begin{align}
Jvec =&

```
# Run a test
TOL = 1e-4
FLR = 1e-20

def projectFieldsAdjointTest(fdemType, comp):
    print 'Adjoint %s formulation - %s' % (fdemType, comp)

    m  = np.log(np.ones(problem.mesh.nC)*0.01)
    if True:
        m  = m + np.random.randn(problem.mesh.nC)*0.01*1e-1

    u = problem.fields(m)
    v = np.random.randn(1)#+np.random.randn(1)*1j
    # print prb.PropMap.PropModel.nP
    w = np.random.randn(m1d.nN)+np.random.randn(m1d.nN)*1j

    vJw = v.dot(rx.projectFieldsDeriv(src,m1d,f0,w))
    wJtv = w.dot(rx.projectFieldsDeriv(src,m1d,f0,v,adjoint=True)).real
    tol = np.max([TOL*(10**int(np.log10(np.abs(vJw)))),FLR])
    print vJw, wJtv, vJw - wJtv, tol, np.abs(vJw - wJtv) < tol
    return np.abs(vJw - wJtv) < tol
projectFieldsAdjointTest('e','projectFieldsDeriv')
```

    Adjoint e formulation - projectFieldsDeriv
    0.000177264079929 0.000177264079929 3.52365706058e-19 1e-07 True

    True

```
# Run a test
TOL = 1e-4
FLR = 1e-20

def getADeriv_mAdjointTest():
    print 'Adjoint test e formulation - getADeriv_m'

    m  = np.log(np.ones(problem.mesh.nC)*0.01)
    if True:
        m  = m + np.random.randn(problem.mesh.nC)*0.01*1e-1

    u = problem.fields(m)
    v = np.random.randn(m1d.nN)#+np.random.randn(1)*1j
    # print prb.PropMap.PropModel.nP
    w = np.random.randn(m1d.nC)#+np.random.randn(m1d.nN)*1j

    vJw = v.dot(problem.getADeriv_m(freq,f0,w))
    wJtv = w.dot(problem.getADeriv_m(freq,f0,v,adjoint=True))
    tol = np.max([TOL*(10**int(np.log10(np.abs(vJw)))),FLR])
    print vJw, wJtv, vJw - wJtv, tol, np.abs(vJw - wJtv) < tol
    return np.abs(vJw - wJtv) < tol
getADeriv_mAdjointTest()
```

```
%debug
```

    > /media/gudni/ExtraDrive1/Codes/python/simpegmt/simpegMT/ProblemMT1D/Problems.py(69)getADeriv_m()
         68         #
    ---> 69         u_src = u['e_1dSolution']
         70         dMf_dsig = self.mesh.getFaceInnerProductDeriv(self.curModel.sigma)(u_src) * self.curModel.sigmaDeriv

    ipdb> u
    > <ipython-input-16-0d25a7e16094>(17)getADeriv_mAdjointTest()
         16
    ---> 17     vJw = v.dot(problem.getADeriv_m(freq,u0,w))
         18     wJtv = w.dot(problem.getADeriv_m(freq,u0,v,adjoint=True))

    ipdb> d
    > /media/gudni/ExtraDrive1/Codes/python/simpegmt/simpegMT/ProblemMT1D/Problems.py(69)getADeriv_m()
         68         #
    ---> 69         u_src = u['e_1dSolution']
         70         dMf_dsig = self.mesh.getFaceInnerProductDeriv(self.curModel.sigma)(u_src) * self.curModel.sigmaDeriv

    ipdb> p u
    array([  7.09826705e-01 -3.42993400e-01j,
             2.06495724e-02 -1.13236591e+00j,
             7.19940065e-01 +6.18299116e-01j,
            -3.76573540e-01 +9.09687373e-01j,
            -5.96166731e-01 +2.23309643e-01j,
             3.30533731e-01 +7.98068064e-01j,
            -1.41356795e+00 -1.74137927e-01j,
             1.51554175e+00 -2.16363375e-01j,
             1.59469627e+00 -1.87971566e+00j,
            -5.54637698e-01 -1.01936458e+00j,
             1.01869865e+00 +8.33443055e-01j,
            -4.92616671e-01 -1.97036224e+00j,
             1.00389102e+00 -1.52567261e-01j,
             7.90254779e-01 -5.68065046e-01j,
             9.17329964e-01 -1.18879097e+00j,
            -1.38317362e+00 -2.17789273e+00j,
             2.58324368e-01 -1.51003598e+00j,
            -2.04803634e+00 -1.44299943e-01j,
             3.33011285e+00 +1.52905384e+00j,
             2.91429972e-01 -1.91849967e+00j,
            -1.10273784e+00 -8.65794290e-01j,
            -2.80103156e-01 -1.42653192e+00j,
            -9.81404885e-01 +9.06845144e-02j,
             6.37579045e-01 +3.84148006e-01j,
            -6.83597988e-01 +6.91187965e-01j,
            -8.98575981e-01 +3.80347476e-02j,
             1.02647319e+00 -1.13876879e-01j,
             8.28006255e-01 -9.95903406e-01j,
             3.53931083e-01 -2.80188697e-01j,
            -1.09339690e+00 +1.43646692e+00j,
             2.59276626e+00 +4.09662561e-01j,
            -5.97873868e-01 -5.23852065e-01j,
             8.87054287e-01 -1.69872744e+00j,
             2.29767481e-01 -5.88143928e-01j,
            -6.97610315e-01 +5.68810360e-01j,
            -4.47049016e-01 +3.29354463e-01j,
            -3.31498311e-01 -8.27120276e-01j,
             1.57689998e-01 +6.68378872e-01j,
             9.47018631e-02 +2.60180446e-01j,
            -4.59177257e-01 -7.66896049e-01j,
            -1.16791204e+00 +4.20585804e-01j,
             2.72405147e-01 +3.48079787e-01j,
            -2.45880514e-02 +1.90226778e-01j,
            -7.03428363e-01 +1.20531568e+00j,
             1.60362217e+00 +6.88472259e-01j,
            -5.49144335e-01 +6.96208292e-01j,
            -9.83200346e-01 -1.15726575e+00j,
             1.22826524e+00 -8.58070855e-01j,
            -5.99020439e-01 -2.19182124e+00j,
            -5.69992871e-01 +1.68586900e+00j,
             9.26752644e-01 +1.05767829e+00j,
             8.64247444e-01 +1.51282356e-01j,
            -6.24547862e-01 -1.54517937e+00j,
            -1.72043526e+00 +1.91659834e+00j,
            -1.84501951e+00 -2.40165623e-01j,
             1.15210121e+00 -1.19627222e+00j,
            -3.93036395e-01 -1.94918396e+00j,
            -1.18266273e+00 -5.24652722e-01j,
             2.67461628e-01 -8.07279138e-01j,
            -1.56363641e+00 +1.59257615e-03j,
            -3.17211883e-01 -3.23379690e+00j,
            -1.08658939e-01 -8.66079039e-01j,
            -9.30350013e-01 -1.22668219e+00j,
            -1.07263137e+00 -5.56847000e-02j,
             1.27878013e+00 -1.34173537e+00j,
             7.91441492e-01 +6.42111414e-01j,
            -2.60121575e+00 -1.08140886e+00j,
            -1.15027808e+00 -4.47110142e-01j,
            -5.69747800e-01 +2.34065916e-01j,
             1.52155089e+00 -5.80340775e-01j,
             2.50052750e-01 -6.58793179e-01j,
             2.96944135e-01 +1.27455206e+00j,
             9.18939766e-01 -4.58122761e-01j,
             4.90015531e-01 -8.86288179e-01j,
            -1.65155083e+00 -6.57733967e-01j,
             9.72148169e-01 -1.19446820e+00j,
             2.08365350e+00 -1.53585595e-01j,
            -4.11101072e-01 +1.01418536e+00j,
             9.72580562e-01 -2.20470840e-01j,
             1.72922147e-01 -6.86360934e-01j,
             3.80255034e-01 +1.96862619e-01j,
            -1.53628572e+00 +5.31946011e-01j,
            -4.65939167e-01 +9.11725611e-01j,
            -1.06390365e+00 -3.07658816e-02j,
            -2.40734296e+00 -7.60530882e-01j,
             1.28242272e+00 +3.32085973e+00j,
             3.17004751e+00 +1.62654811e+00j,
            -8.49758476e-01 +9.86658359e-01j,
             4.99578557e-01 +4.01392488e-01j,
            -9.48695998e-01 +4.04322381e-01j,
             1.75371116e-01 +7.85458813e-01j,
             4.90232886e-01 -1.39280042e-01j,
            -2.31430252e-01 -8.44065839e-01j,
            -1.02319133e+00 +1.90560784e+00j,
             1.08940667e+00 +1.63046186e+00j,
            -4.34233324e-01 +2.90439312e-01j,
             4.47622371e-01 +1.26507976e+00j,
            -2.07317470e+00 -2.21558532e+00j,
            -3.19553487e-01 -9.19210427e-01j,
             6.09870390e-01 +3.05692898e-01j,
             4.92081860e-01 -2.18326059e-01j,
             1.48544627e+00 -2.02054490e-01j,
            -1.59612367e-01 +7.90853777e-02j,
             2.85854411e-01 +2.16567814e-01j,
             5.28063071e-01 +2.08838949e+00j,
             6.17580558e-01 +9.90748539e-01j,
            -2.64539211e-01 +1.16598665e+00j,
            -1.07357542e+00 -7.58002064e-01j,
             6.13399713e-02 +2.52338250e+00j,
            -4.84180135e-01 -1.45046020e+00j,
            -6.57519374e-01 -1.52719059e+00j,
            -1.54949084e-01 -7.64796563e-01j,
            -2.01172915e-02 +1.07377565e+00j,
             2.19840636e-01 +6.89622678e-01j,
            -2.64578500e+00 +1.46933966e+00j,
             1.22168696e+00 +2.33458454e-01j,
             2.71303177e+00 +2.77858685e-01j,
             4.39426665e-01 +1.79924695e-01j,
            -1.73858667e+00 -3.03473636e-01j,
             2.68736300e-01 +8.32359634e-01j,
             2.57517784e+00 +1.88097576e-01j,
             2.76775469e-01 -6.49891901e-01j,
            -9.87790301e-01 +4.50328269e-01j,
             9.19033669e-01 +1.01453748e+00j,
            -1.35706747e+00 +5.90240590e-01j,
             6.60118738e-01 +5.10361208e-01j,
            -2.17652642e-01 -4.26632365e-01j,
            -1.39163540e+00 +9.12739752e-01j,
            -1.76241829e+00 -1.04835908e+00j,
            -7.28279529e-01 +2.75961750e-01j,
            -1.32043513e+00 -3.28790183e-01j,
             5.88227093e-01 +9.41865970e-01j,
             1.97695607e+00 +1.04535450e+00j,
            -3.02599128e+00 +1.02031744e+00j,
            -1.34308763e+00 -4.48222977e-01j,
            -1.86687861e+00 +2.26767568e+00j,
             5.61472023e-01 -5.28652462e-01j,
            -5.26747946e-01 +2.75545054e-01j,
             6.16116514e-01 +3.41765793e-01j,
             1.38709750e+00 +5.87537003e-01j,
            -2.27923203e-01 +7.19317677e-01j,
            -8.66404713e-01 -1.80932081e-01j,
            -4.58146668e-01 +1.75955171e+00j,
            -5.12526647e-01 -1.91939928e-01j,
            -6.26532699e-01 -5.71301355e-01j,
            -3.35566040e-01 -5.06173544e-01j,
            -3.86124497e-01 -8.00560567e-01j,
             2.47252265e+00 +3.83928843e-01j,
             3.35561169e-01 +1.19309445e+00j,
            -6.19075444e-01 -1.19021628e+00j,
            -1.17335297e+00 +5.80500783e-01j,
            -2.35111248e-01 -1.59785495e+00j,
            -1.24819583e-01 -8.05281256e-01j,
            -9.18606131e-01 -1.25312332e-01j,
             3.14400972e-01 +1.18650079e+00j,
             6.50630049e-01 -6.99853497e-01j,
             1.28269722e+00 -1.29832850e+00j,
             2.07880202e-01 +2.56385692e-02j,
            -1.45089461e+00 -9.35093103e-01j,
             6.84146951e-01 +7.74422758e-01j,
             9.45439196e-01 -3.50695577e-01j,
             1.00057580e+00 -1.27412936e+00j,
            -2.01665427e-01 +1.73591867e+00j,
             1.76623768e+00 -1.42964638e+00j,
             1.12282181e+00 +9.02767375e-01j,
            -2.22663798e+00 +7.34672692e-02j,
             8.26230180e-02 -7.46601441e-01j,
            -4.92810641e-01 -5.49544995e-01j,
             1.06703148e-01 +1.70461156e+00j,
            -6.84949746e-01 -1.46950185e-01j,
             1.47509281e+00 +8.77421590e-01j,
            -9.83722735e-03 -6.30597166e-01j,
             2.36006569e-03 +2.12883413e-01j,
            -3.67899785e-01 -9.95508241e-02j,
             1.26752410e+00 +2.49370098e-01j,
            -1.19439605e+00 +2.94935402e-01j,
            -2.28617888e+00 +5.70564449e-02j,
             3.95617530e-02 -2.80995512e-01j,
            -7.51027805e-01 -8.14567233e-01j,
            -1.24160227e+00 -4.13553574e-01j,  -4.76626111e-01 +4.16480574e-01j])
    ipdb> u_src
    *** NameError: name 'u_src' is not defined
    ipdb> c

```
# Run a test
TOL = 1e-4
FLR = 1e-20

def getRHSDeriv_mAdjointTest():
    print 'Adjoint test e formulation - getRHSDeriv_m'

    m  = np.log(np.ones(problem.mesh.nC)*0.01)
    if True:
        m  = m + np.random.randn(problem.mesh.nC)*0.01*1e-1

    u = problem.fields(m)
    v = np.random.randn(m1d.nN)#+np.random.randn(1)*1j
    # print prb.PropMap.PropModel.nP
    w = np.random.randn(m1d.nC)#+np.random.randn(m1d.nN)*1j

    vJw = v.dot(problem.getRHSDeriv_m(freq,w))
    wJtv = w.dot(problem.getRHSDeriv_m(freq,v,adjoint=True))
    tol = np.max([TOL*(10**int(np.log10(np.abs(vJw)))),FLR])
    print vJw, wJtv, vJw - wJtv, tol, np.abs(vJw - wJtv) < tol
    return np.abs(vJw - wJtv) < tol
getRHSDeriv_mAdjointTest( )
```

    Adjoint test e formulation - getRHSDeriv_m
    (-10077.7224119-28916.4723594j) (-10077.7224119-28916.4723594j) (1.81898940355e-12-3.63797880709e-12j) 1.0 True

    True

```
simpeg.mkvc(np.random.randn(survey.nD)+np.random.randn(survey.nD)*1j,2)

print survey.nD
```

```
TOL = 1e-4
FLR = 1e-20

def JvecAdjointTest():
    print 'Adjoint e formulation - Jvec'

    m  = np.log(np.ones(problem.mesh.nC)*0.01)
    if True:
        m  = m + np.random.randn(problem.mesh.nC)*0.01*1e-1

    u = problem.fields(m)

    v = np.random.rand(survey.nD)
    # print prb.PropMap.PropModel.nP
    w = np.random.rand(problem.mesh.nC)

    vJw = v.dot(problem.Jvec(m, w, u))
    wJtv = w.dot(problem.Jtvec(m, v, u))
    tol = np.max([TOL*(10**int(np.log10(np.abs(vJw)))),FLR])
    print vJw, wJtv, vJw - wJtv, tol, np.abs(vJw - wJtv) < tol
    return np.abs(vJw - wJtv) < tol
```

```
JvecAdjointTest()
```

    Adjoint e formulation - Jvec
    1.09508355274e-05 1.09508355274e-05 -1.01643953671e-20 1e-08 True

    True

```

```


# 1.9.3. Complemento Naive Bayes

## O ComplementNB implementa o algoritmo Bayes naive do complemento (CNB). CNB é uma adaptação do algoritmo padrão multinomial ingênuo Bayes (MNB) que é particularmente adequado para conjuntos de dados desequilibrados. Especificamente, o CNB usa estatísticas do complemento de cada classe para calcular os pesos do modelo. Os inventores do CNB mostram empiricamente que as estimativas dos parâmetros para o CNB são mais estáveis do que para o MNB. Além disso, o CNB regularmente supera o MNB (muitas vezes por uma margem considerável) em tarefas de classificação de texto. O procedimento para calcular os pesos é o seguinte:

        ### \begin{align}\begin{aligned}\hat{\theta}_{ci} = \frac{\alpha_i + \sum_{j:y_j \neq c} d_{ij}}
                         {\alpha + \sum_{j:y_j \neq c} \sum_{k} d_{kj}}\\w_{ci} = \log \hat{\theta}_{ci}\\w_{ci} = \frac{w_{ci}}{\sum_{j} |w_{cj}|}\end{aligned}\end{align}

## onde as somas são sobre todos os documentos j não na classe c, d_ {ij} é a contagem ou o valor tf-idf do termo no documento j, \ alpha_i é um hiperparâmetro de suavização como aquele encontrado no MNB, e \ alpha = \ sum_ {i} \ alpha_i. A segunda normalização aborda a tendência de documentos mais longos dominarem as estimativas de parâmetros no MNB. A regra de classificação é:

        ### \ hat {c} = \ arg \ min_c \ sum_ {i} t_i w_ {ci}

## ou seja, um documento é atribuído à classe que tem a menor correspondência de complemento.

## Referências:

## Rennie, J. D., Shih, L., Teevan, J., & Karger, D. R. (2003). Tackling the poor assumptions of naive bayes text classifiers. In ICML (Vol. 3, pp. 616-623). (https://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf)

```python

```


# Sitzung 5

Diese Skripte sind ausschließlich als Zusatz-Material gedacht. Speziell für diejenigen unter Euch, die einen Einblick in das Programmieren gewinnen wollen. Wenn Du es also leid bist repetitive Tätigkeiten auszuführen und das lieber einer Maschine überlassen willst, bist Du hier genau richtig.

<span style="color:red"> Die Codes sind nicht für die Klausur relevant, genau genommen haben sie mit dem Lehrstuhl für Statistik __rein gar nichts__ zu tun. </span>

---

Dieses Mal modellieren wir die Wahrscheinlichkeit für die Anzahl matches, die man beim Tindern erwarten, kann (andere Dating-Apps sind erhältlich). Sag einer Statistik hätte nichts mit dem echten Leben zu tun...

Falls Dir ein seriöseres Beispiel lieber ist, gibt es eine ausgezeichnete Motivation desselben mathematischen Problems [hier](https://mourafiq.com/2016/02/05/intuition-behind-poisson-distribution.html). Die Herleitung am Ende ist aber ein wenig kurz gehalten, darum gibt es eine bessere Antwort auf [stackoverflow](https://math.stackexchange.com/a/836585). Eine ernstere Version der Exponentialverteilung gibt es [hier](http://www.milefoot.com/math/stat/pdfc-exponential.htm).

---

## Tinder und die Poisson-Verteilung

Stell Dir vor, Du wüsstest bereits aufgrund Deiner langjährigen Erfahrung mit Tinder, wie viele matches Du pro Tag erwarten kannst. Wir nennen diesen Wert $\lambda$. Für die einen kann $\lambda$ ziemlich hoch sein, wer weiß, vielleicht $\lambda=12$? Für jemand anderen kann es eventuell etwas schwieriger sein ein Tinder-match abzukriegen, dann ist $\lambda$ natülich kleiner.

Um unsere Rechnung zu vereinfachen gehen wir weiter davon aus, dass wir überhaupt nicht wählerisch sind - wenn wir einen Vorschlag kriegen, akzeptieren wir die Kandidatin/den Kandidaten ohne Umschweife.

### Anzahl matches pro Stunde

Wenn wir die Wahrscheinlichkeit für die Zufallsvariable

$$X: \text{Anzahl matches auf Tinder innerhalb einer Stunde}$$

modellieren wollen, ist eine erste Idee, die Binomialverteilung zu rate zu ziehen, schließlich besteht ein Tag aus 24 Stunden und es gibt, auf den ersten Blick, keinen guten Grund anzunehmen, dass sich die Wahrscheinlichkeit zwischen den Stunden ändert.

Wir wissen, dass die Binomialverteilung entsteht wenn wir:
* n Versuche durchführen und die Summe der Erfolge: $\sum\limits_{i=1}^n X_i$ betrachten,
* wobei $X_i \in \{0, 1\}$ ein Bernoulli-Experiment mit Erfolgswahrscheinlichkeit $p$ beschreibt.

Wir wollen also sagen $X \sim Bin(n; p)$ und wir wissen, dass $n=24$ ist, wir wissen aber noch nicht wie hoch die Wahrscheinlichkeit des einzelnen Zufallsexperiments ist, wir kennen nur den Erwartungswert $\lambda$. Wir wissen aber, wie man den Erwartungswert einer binomial verteilten Zufallsvariable allgemein berechnet. Der Erwartungswert einer binomial verteilten Zufallsvariable ist gegeben als

\begin{align}
E\left(X\right) &= n \cdot p \stackrel{!}{=} \lambda \\
          \Leftrightarrow p &= \frac{\lambda}{n}
\end{align}

Damit können wir sagen, dass

$$P(X=k) = {24 \choose k}\left(\frac{\lambda}{24}\right)^k \left(1-\frac{\lambda}{24}\right)^{24-k}$$

Irgend etwas stimmt aber noch nicht. Wie haben wir denn die zugrunde liegenden Experimente $X_i$ definiert?

$$X_i = \begin{cases}
0 \text{ für kein einziges Tinder match in Stunde } \textit{i}\\
1 \text{ sonst}
\end{cases}
$$

Der Knackpunkt liegt in dem Wörtchen *sonst*. In diesem Fall heißt das nämlich **ein oder mehr** Tinder matches in Stunde $i$. Wir ignorieren völlig, dass wir in einer Stunde zum Beispiel $2$ Tinder matches kriegen könnten. Dieses Problem können wir lösen, indem wir $n$ größer machen. Zurzeit ist es so, als würden wir an einem Tag nur einmal pro Stunde nachsehen, ob wir ein match erzielen konnten - und das ist ja natürlich Quatsch. Wir können ja auch jede halbe Stunde, nachsehen. Oder jede Viertel Stunde, oder jede Minute oder jede Sekunde. Eigentlich können wir ständig nachsehen. Mathematisch gesehen, ist dieses leicht verstörende Verhalten, das wir beschrieben haben das Gleiche, wie den Grenzwert zu bilden für $n \rightarrow \infty$.

\begin{align}
&\lim \color{red}{{n \choose k}} \cdot \left(\frac{\lambda}{n}\right)^{\color{blue}k} \cdot \left(1-\frac{\lambda}{n}\right)^{\color{orange}{n-k}}\\
&= \lim \color{red}{\frac{n!}{k!(n-k)!}} \cdot \frac{\lambda^{\color{blue}k}}{n^{\color{blue}k}} \cdot \left(1-\frac{\lambda}{n}\right)^{\color{orange}{n}} \cdot \left(1-\frac{\lambda}{n}\right)^{\color{orange}{-k}} \\
&= \lim \frac{\lambda^k}{k!} \cdot \underbrace{\left(1-\frac{\lambda}{n}\right)^{n}}_{\color{purple}{\rightarrow exp(-\lambda)}} \cdot \left(1-\frac{\lambda}{n}\right)^{-k} \cdot \overbrace{\frac{n!}{(n-k)!}}^{\text{hat k Faktoren}} \cdot \underbrace{\frac{1}{n^k}}_{\substack{\text{hat auch} \\ \text{k Faktoren}}} \\
&= \lim \frac{\lambda^k}{k!} \cdot \left(1-\frac{\lambda}{n}\right)^{n} \cdot \underbrace{\left(1-\frac{\lambda}{n}\right)^{-k}}_{\rightarrow 1} \cdot \overbrace{\frac{n}{n}}^{\rightarrow 1} \cdot \overbrace{\frac{n-1}{n}}^{\rightarrow 1} \cdot \overbrace{\frac{n-2}{n}}^{\rightarrow 1} \cdot \dots \cdot \overbrace{\frac{n-k+1}{n}}^{\rightarrow 1} \\
&= \frac{\lambda^k}{k!} \exp\left(-\lambda\right)
\end{align}

Damit haben wir soeben die Poisson Verteilung hergeleitet. Damit können wir sehen wie wahrscheinlich unsere verschiedenen Wunsch-Ereignisse auf Tinder ausfallen.

---

```python
import numpy as np
from matplotlib import pyplot as plt
from ipywidgets import interact, FloatSlider, Layout
from textwrap import dedent

def tinder_dist(l=1.5):
    # Calculate probabilities
    K = np.arange(0, 100)
    y = np.array([(l**k/np.math.factorial(k))*np.exp(-l) for k in K])
    cum_y = np.cumsum(y)
    # Create plots
    fig, ax = plt.subplots(1, 2, figsize=(13, 4))
    ax[0].bar(K, y, color="red", alpha=0.5)
    ax[1].step(K, cum_y, where="post", c="red")
    # Fix limits
    xlim = np.argwhere(y > 0.01)
    ax[0].set_xlim([np.min(xlim)-0.5, np.max(xlim)+0.5])
    ax[1].set_xlim([np.min(xlim)-0.5, np.max(xlim)+0.5])
    ax[1].set_ylim([0, 1])
    # Set titles
    ax[0].set_title("Wahrscheinlichkeitsfunktion")
    ax[1].set_title("Verteilungsfunktion")
    plt.suptitle("$X \sim Pois({})$".format(l))
    plt.show()
```

```python
lambda_widget = FloatSlider(min=0.1, max=50, value=2.2, step=0.01,
                            description="Erwartete Matches pro Stunde",
                            layout = Layout(width='50%'),
                            style={'description_width': '200px'})
x = interact(tinder_dist, l=lambda_widget);
```

    interactive(children=(FloatSlider(value=2.2, description='Erwartete Matches pro Stunde', layout=Layout(width='…

---
## Tinder und die Exponential-Verteilung

Was ist jetzt, wenn wir uns gar nicht dafür interessieren, wie viele matches wir bekommen können. Vielleicht reicht uns ja genau eins - das erste. Wie lange dauert es also bis zu unserem ersten Tinder-match?

Oder anders ausgedrückt: wie lange müssen wir warten, bis sich auf Tinder jemand findet, der bereit ist, mit uns zu chatten. Dazu muss uns erst mal klar werden, was "warten" eigentlich bedeutet: Warten heißt, dass bisher nichts passiert ist. Die absolute Anzahl unserer Tinder-matches bis zum jetzigen Zeitpunkt ist $X = 0$. Und die Wahrscheinlichkeit für diesen Zustand lässt sich mit der Poisson-Verteilung berechnen:

\begin{equation}
P(\{\text{kein Tinder match in einer Zeiteinheit}\}) = P(X=0) = \frac{\lambda^0}{0!}\exp(-\lambda) = \exp(-\lambda)
\end{equation}

Jetzt wollen wir wissen wie wahrscheinlich es ist, innerhalb von zwei Zeiteinheiten auf Tinder leer auszugehen. Dazu müssen wir ein klein wenig mentale Akrobatik vollführen. Wir müssen unsere Frage so verpacken, dass sie sich in einen mathematischen Ausdrück überführen lässt. Wenn wir zwei Zeiteinheiten lang warten müssen, heißt das ja, dass in der ersten Zeiteinheit kein Ereignis aufgetreten ist - **und** in der zweiten Zeiteinheit ist auch nichts passiert. Mathematisch gesehen, ist das eine **gemeinsame** Wahrscheinlichkeit:

\begin{equation}
P(\{\text{kein Tinder match in zwei Zeiteinheiten}\}) =\\
P(\{\text{kein Tinder match in der ersten Zeiteinheit}\}, \{ \text{kein Tinder match in der zweiten Zeiteinheit} \})
\end{equation}

Der nächste Schritt, ist festzustellen, dass die beiden Zeiteinheiten **unabhängig** voneinander sind. Informationen über die Anzahl matches in der ersten Zeiteinheit helfen uns nicht, die Anzahl in der zweiten Zeiteinheit vorherzusagen. Die gemeinsame Wahrscheinlichkeit von unabhängigen Zufallsvariablen, kann man schreiben als das Produkt der beiden Einzelwahrscheinlichkeiten

\begin{align}
P(\{\text{kein match in der ersten Zeiteinheit}\} &, \{ \text{kein match in der zweiten Zeiteinheit} \}) \\
= P(\{\text{kein match in der ersten Zeiteinheit}\}) &\cdot P(\{\text{kein match in der zweiten Zeiteinheit}\}) \\
= \exp(-\lambda) &\cdot \exp(-\lambda) = \exp(-2\lambda)
\end{align}

**Allgemein**, für t Zeiteinheiten können wir sagen: $$P(\{\text{kein match in t Zeiteinheiten}\}) = \exp(-\lambda t)$$

Und jetzt sind wir eigentlich schon fast fertig, denn die Wahrscheinlichkeit in $t$ Zeiteinheiten kein einziges match zu erzielen ist, logisch gesehen, das Gleiche wie länger als $t$ Zeiteinheiten auf das erste match zu warten. Damit definieren wir die Zufallsvariable

$$T: \text{ Wartezeit auf das erste Tinder match}.$$

Und wir kennen auch schon die Verteilung von $T$:

\begin{equation}
P(T > t) = P(\{\text{kein match in t Zeiteinheiten}\}) = \exp(-\lambda t).
\end{equation}

Jetzt ist das aber nicht die Art, wie wir Verteilungsfunktionen definieren wollen. Normalerweise wollen wir, dass eine Verteilungsfunktion uns angibt, wie groß die Wahrscheinlichkeit ist, dass eine Zufallsvariable einen Wert annimmt, der **kleiner** ist als eine bestimmte Grenze. Unsere Formulierung macht genau das Gegenteil - wir können es aber reparieren:

\begin{equation}
P(T \leq t) = 1 - P(T > 1) =1 - \exp(-\lambda t).
\end{equation}

Und so gelangen wir zur Verteilungsfunktion der Exponential-Verteilung und können schreiben:
$$T \sim Exp(\lambda)$$

---

```python
def tinder_wait_dist(l=1.5):
    # Calculate probabilities
    upper = np.max([6/l, 1])
    T = np.linspace(0, upper, 100)
    pdf_y = np.array([l*np.exp(-l*t) for t in T])
    cdf_y = np.array([1-np.exp(-l*t) for t in T])

    # Calculate 95% waiting time
    q95 = -1/l * np.log(0.05)

    # Create plots
    fig, ax = plt.subplots(1, 2, figsize=(13, 4))
    ax[0].plot(T, pdf_y, color="red")
    ax[1].plot(T, cdf_y, color="red")
    ax[1].plot([-1, q95, q95], [0.95, 0.95, 0], "--", c="grey")
    # Correct axis and add annotations
    ax[1].set_ylim([0, 1])
    ax[1].set_xlim([0, upper])
    ax[1].text(q95 + 0.1, 0.5,
              ("Mit 95%-iger"
               "\nWahrscheinlichkeit\n"
               "musst Du weniger als\n"
               "{:3.3} Stunden warten".format(q95)))
    # Set titles
    ax[0].set_title("Dichtefunktion")
    ax[1].set_title("Verteilungsfunktion")
    plt.suptitle("$X \sim Exp({})$".format(l))
    plt.show()
```

Wie lange musst Du jetzt auf dein erstes Tinder match warten? Finde es einfach raus, indem Du deinen Erwartungswert vorgibst.

```python
interact(tinder_wait_dist, l=lambda_widget);
```

    interactive(children=(FloatSlider(value=2.2, description='Erwartete Matches pro Stunde', layout=Layout(width='…

```python

```


```python
import open3d as o3d
import numpy as np
import copy
import os
import sys

# monkey patches visualization and provides helpers to load geometries
sys.path.append('..')
import open3d_tutorial as o3dtut
# change to True if you want to interact with the visualization windows
o3dtut.interactive = not "CI" in os.environ
```

# ICP registration
This tutorial demonstrates the ICP (Iterative Closest Point) registration algorithm. It has been a mainstay of geometric registration in both research and industry for many years. The input are two point clouds and an initial transformation that roughly aligns the source point cloud to the target point cloud. The output is a refined transformation that tightly aligns the two point clouds. A helper function `draw_registration_result` visualizes the alignment during the registration process. In this tutorial, we show two ICP variants, the point-to-point ICP and the point-to-plane ICP [\[Rusinkiewicz2001\]](../reference.html#rusinkiewicz2001).

## Helper visualization function
The function below visualizes a target point cloud and a source point cloud transformed with an alignment transformation. The target point cloud and the source point cloud are painted with cyan and yellow colors respectively. The more and tighter the two point clouds overlap with each other, the better the alignment result.

```python
def draw_registration_result(source, target, transformation):
    source_temp = copy.deepcopy(source)
    target_temp = copy.deepcopy(target)
    source_temp.paint_uniform_color([1, 0.706, 0])
    target_temp.paint_uniform_color([0, 0.651, 0.929])
    source_temp.transform(transformation)
    o3d.visualization.draw_geometries([source_temp, target_temp],
                                      zoom=0.4459,
                                      front=[0.9288, -0.2951, -0.2242],
                                      lookat=[1.6784, 2.0612, 1.4451],
                                      up=[-0.3402, -0.9189, -0.1996])
```

**Note:**

Since the functions `transform` and `paint_uniform_color` change the point cloud, we call `copy.deepcopy` to make copies and protect the original point clouds.

## Input
The code below reads a source point cloud and a target point cloud from two files. A rough transformation is given.

**Note:**

The initial alignment is usually obtained by a global registration algorithm. See [Global registration](../pipelines/global_registration.rst) for examples.

```python
demo_icp_pcds = o3d.data.DemoICPPointClouds()
source = o3d.io.read_point_cloud(demo_icp_pcds.paths[0])
target = o3d.io.read_point_cloud(demo_icp_pcds.paths[1])
threshold = 0.02
trans_init = np.asarray([[0.862, 0.011, -0.507, 0.5],
                         [-0.139, 0.967, -0.215, 0.7],
                         [0.487, 0.255, 0.835, -1.4], [0.0, 0.0, 0.0, 1.0]])
draw_registration_result(source, target, trans_init)
```

The function `evaluate_registration` calculates two main metrics:

- `fitness`, which measures the overlapping area (# of inlier correspondences / # of points in target). The higher the better.
- `inlier_rmse`, which measures the RMSE of all inlier correspondences. The lower the better.

```python
print("Initial alignment")
evaluation = o3d.pipelines.registration.evaluate_registration(
    source, target, threshold, trans_init)
print(evaluation)
```

## Point-to-point ICP
In general, the ICP algorithm iterates over two steps:

1. Find correspondence set $\mathcal{K}=\{(\mathbf{p}, \mathbf{q})\}$ from target point cloud $\mathbf{P}$, and source point cloud $\mathbf{Q}$ transformed with current transformation matrix $\mathbf{T}$.
2. Update the transformation $\mathbf{T}$ by minimizing an objective function $E(\mathbf{T})$ defined over the correspondence set $\mathcal{K}$.

Different variants of ICP use different objective functions $E(\mathbf{T})$ [\[BeslAndMcKay1992\]](../reference.html#beslandmckay1992) [\[ChenAndMedioni1992\]](../reference.html#chenandmedioni1992) [\[Park2017\]](../reference.html#park2017).

We first show a point-to-point ICP algorithm [\[BeslAndMcKay1992\]](../reference.html#beslandmckay1992) using the objective

\begin{equation}
E(\mathbf{T}) = \sum_{(\mathbf{p},\mathbf{q})\in\mathcal{K}}\|\mathbf{p} - \mathbf{T}\mathbf{q}\|^{2}
\end{equation}

The class `TransformationEstimationPointToPoint` provides functions to compute the residuals and Jacobian matrices of the point-to-point ICP objective. The function `registration_icp` takes it as a parameter and runs point-to-point ICP to obtain the results.

```python
print("Apply point-to-point ICP")
reg_p2p = o3d.pipelines.registration.registration_icp(
    source, target, threshold, trans_init,
    o3d.pipelines.registration.TransformationEstimationPointToPoint())
print(reg_p2p)
print("Transformation is:")
print(reg_p2p.transformation)
draw_registration_result(source, target, reg_p2p.transformation)
```

The `fitness` score increases from 0.174723 to 0.372450. The `inlier_rmse` reduces from 0.011771 to 0.007760. By default, `registration_icp` runs until convergence or reaches a maximum number of iterations (30 by default). It can be changed to allow more computation time and to improve the results further.

```python
reg_p2p = o3d.pipelines.registration.registration_icp(
    source, target, threshold, trans_init,
    o3d.pipelines.registration.TransformationEstimationPointToPoint(),
    o3d.pipelines.registration.ICPConvergenceCriteria(max_iteration=2000))
print(reg_p2p)
print("Transformation is:")
print(reg_p2p.transformation)
draw_registration_result(source, target, reg_p2p.transformation)
```

The final alignment is tight. The `fitness` score improves to 0.621123. The `inlier_rmse` reduces to 0.006583.

## Point-to-plane ICP
The point-to-plane ICP algorithm [\[ChenAndMedioni1992\]](../reference.html#chenandmedioni1992) uses a different objective function

\begin{equation}
E(\mathbf{T}) = \sum_{(\mathbf{p},\mathbf{q})\in\mathcal{K}}\big((\mathbf{p} - \mathbf{T}\mathbf{q})\cdot\mathbf{n}_{\mathbf{p}}\big)^{2},
\end{equation}

where $\mathbf{n}_{\mathbf{p}}$ is the normal of point $\mathbf{p}$. [\[Rusinkiewicz2001\]](../reference.html#rusinkiewicz2001) has shown that the point-to-plane ICP algorithm has a faster convergence speed than the point-to-point ICP algorithm.

`registration_icp` is called with a different parameter `TransformationEstimationPointToPlane`. Internally, this class implements functions to compute the residuals and Jacobian matrices of the point-to-plane ICP objective.

```python
print("Apply point-to-plane ICP")
reg_p2l = o3d.pipelines.registration.registration_icp(
    source, target, threshold, trans_init,
    o3d.pipelines.registration.TransformationEstimationPointToPlane())
print(reg_p2l)
print("Transformation is:")
print(reg_p2l.transformation)
draw_registration_result(source, target, reg_p2l.transformation)
```

The point-to-plane ICP reaches tight alignment within 30 iterations (a `fitness` score of 0.620972 and an `inlier_rmse` score of 0.006581).

The point-to-plane ICP algorithm uses point normals. In this tutorial, we load normals from files. If normals are not given, they can be computed with [Vertex normal estimation](pointcloud.ipynb#vertex-normal-estimation).


Simulação de Canal de Comunicação segundo Modelo Erceg

```python
import numpy as np
import random
import matplotlib.pyplot as plt
from PIL import Image
import math
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

#%%latex
```

O modelo Erceg foi construído e estudado de acordo com o dado experimentais coletados pela AT&T Wireless Service em 95 estações dentro dos EUA operando a 1,9GHz. O modelo enumera três categorias de terrenos que provocam perdas no sinal e eles são divididos da seguinte maneira; a categoria A representa terrenos com morros e colinas e com alta de densidade de obejetos no caminho do sinal, o que representa uma alta perda de percurso. Categoria C representa um terreno plano com baixa densidade de objetos no caminho do sinal, representando uma baixa perda do sinal, já a categoria B é constituída por terrenos montanhososo com baixa densidade de objetos ou terrenos plano com densidade consideravel de objetos no caminho do sinal, basicamente a categoria B representa um meio termo entre as categorias A e C, com perda de sinal mediana quando compara com as outras.

Para todas as três categorias o caminho de perda mediana é representada pela mesma equação com a condição de $d > d_0$, no caso a seguinte equação.$$
\begin{equation}\label{eq:erceg}
    P_L(dB) = 20\log_{10}(4\pi d_0/\lambda)+10\gamma \log_{10}(d/d_0)+s
\end{equation}
$$Onde $\lambda$ é o comprimento de onda do sinal, s é o efeito de sombreamento do sinal, $\gamma$ representa o caminho de menor perda considerando as três categorias do modelo, representado pela seguinte equação
$$
\gamma = a - bh_b + c/h_b
$$
$h_b$ representa a altura da estação base em metros, normalmente entre 10 e 80m, $d_0=100m$ e os valores nominais de a, b e c variam de acordo com a catgoria do terrenos, seus valores variam de acordo a seguinte tabela:

|Parametro|Categoria A|Categoria B|Categoria C|
|---------|-----------|-----------|-----------|
|a|4,6|3|3,6|
|b|0,0076|0,0065|0,005|
|c|12,6|17,1|20|

Definiçao das variáveis $d_0$, $h_b$ e $s$

```python
d0 = 100
hb = 50
s = 6
f = 1900000000
```

Definição das distâncias a serem simuladas

```python
d = np.arange(1, 1000, 1)
```

Construção dos arrays das categorias

```python
a = np.array([4.6, 3, 3.6])
b = np.array([0.0076, 0.0065, 0.005])
c = np.array([12.6, 17.1, 20])
```

Array com os valores de gamma

```python
gamma = [a[i]-b[i]*hb+c[i]/hb for i in range(3)]
```

Com os valores instânciados podemos analisar a resposta para cada uma das três situações

```python
v1 = 20*np.log10(4*math.pi*d0*f/300000000)+10*gamma[0]*np.log10(d/d0)+s
v2 = 20*np.log10(4*math.pi*d0*f/300000000)+10*gamma[1]*np.log10(d/d0)+s
v3 = 20*np.log10(4*math.pi*d0*f/300000000)+10*gamma[2]*np.log10(d/d0)+s
```

Plotando os resultados obtidos

```python
plt.plot(d, v1, label='Categoria A')
plt.plot(d, v2, label='Categoria B')
plt.plot(d, v3, label='Categoria C')

plt.xlabel('Distancia entre estacao movel e a base [m]')
plt.ylabel('Perda de potencia incidente na estacao movel [db]')
plt.legend()
# plt.title('Perda de Potencia em funcao da distancia entre estacao movel e base')
```

Analisando as curva obtidas em função da distância entre a estação rádio base e o dispositivo móvel podemos verificar para distâncias $d<d_0$ as curvas apresentam comportamento similar, apresentado perdas de potência próximas, um forte indicativo de que os fatores usados para compor o modelo Erceg entregam praticamente os mesmos valores independente da categoria do terreno por onde sinal se propaga. É interessante notar que as divergência entre as curvas para cada categoria se tornam evidentes quando $d>2d_0$, fazendo as curvas divergirem.

Um ponto interessante a se simular é fixar a distância entre a estação rádio móvel, no caso para as três situações possíveis, $d<d_0$, $d=d_0$ e $d>d_0$ e verificar o comportamento de uma frequencia variavel, alguns limites para esse caso são importantes pois o modelo Erceg foi projetado para trabalhar com frequências próximas de 2GHz, logo não faz sentido simular algo para todo o espectro, logo a simulação foi centrada em 2GHz com uma faixa de $+/-$300MHz, ou seja, foi simulado o comportamento da perda de potência para uma distância fixa para a faixa de frequência iniciando em 1,7GHz e finalizando em 2,3GHz no passo de 100KHz.

```python
fval = np.arange(1700000000, 2300000000, 100000)
fv1 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[0]*np.log10(100/d0)+s
fv2 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[1]*np.log10(100/d0)+s
fv3 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[2]*np.log10(100/d0)+s

fv4 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[0]*np.log10(50/d0)+s
fv5 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[1]*np.log10(50/d0)+s
fv6 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[2]*np.log10(50/d0)+s

fv7 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[0]*np.log10(200/d0)+s
fv8 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[1]*np.log10(200/d0)+s
fv9 = 20*np.log10(4*math.pi*d0*fval/300000000)+10*gamma[2]*np.log10(200/d0)+s
```

```python
plt.plot(fval, fv1, label='Categoria A d=d0')
plt.plot(fval, fv2, label='Categoria B d=d0')
plt.plot(fval, fv3, label='Categoria C d=d0')

plt.plot(fval, fv4, label='Categoria A d<d0')
plt.plot(fval, fv5, label='Categoria B d<d0')
plt.plot(fval, fv6, label='Categoria C d<d0')

plt.plot(fval, fv7, label='Categoria A d>d0')
plt.plot(fval, fv8, label='Categoria B d>d0')
plt.plot(fval, fv9, label='Categoria C d>d0')

plt.xlabel('Frequencia do sinal')
plt.ylabel('Perda de potencia incidente na estacao movel [db]')
plt.legend()
```

Analisando as curvas geradas, chama atenção que quando $d=d_0$ as curvas são iguais, ou seja, independente da categoria a variação da frequência produz o mesmo resultado, contudo este comportamente era previsto pelo modelo pois com $d=d_0$ o termo que depende das categorias de terreno é anulado do modelo, fazendo com que ele dependesse apenas da frequência.

Para distâncias $d<d_0$ o modelo apresenta baixa perdas quando comparado com os outros casos simulados, configurando este como o caso ideal de uso. Para distâncias $d>d_0$ o modelo já apresenta perdas maiores quando colocado em comparação com as outras curvas geradas.

Outra simulacao possivel seria variar a altura da torre fixando a uma frequencia e um $d$ especifico assim seria possivel enteder qual a relacao da alura da torre

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.210.3876&rep=rep1&type=pdf
http://morse.colorado.edu/~tlen5510/text/classwebch3.html
https://www.mathworks.com/matlabcentral/fileexchange/39322-erceg-model
https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Working%20With%20Markdown%20Cells.html

Simulação de Propagacao em Multipercurso

$$X(t)=X_c(t)+jX_s(t)$$
$$X_c(t)=\dfrac{2}{\sqrt M}\sum_{n=1}^{M}\cos (\Psi_n)\cos (\omega_d t \cos a_n + \phi)$$
$$X_s(t)=\dfrac{2}{\sqrt M}\sum_{n=1}^{M}\sin (\Psi_n)\cos (\omega_d t \cos a_n + \phi)$$
$$a_n = \dfrac{2\pi n - \pi + \theta}{4M}$$

Onde $\theta$, $\phi$ e $\Psi_n$ sao estatisticamente independente, variando entre $[-\pi,\pi)$ para todo $n$. A simulacao do artigo foi testada com $M = 8$

A expressão pode ser manilpulada de forma a reunir todos os fatores em uma única expressão.
$$
X_k(t)= \sqrt{\dfrac{2}{M}} \left\{ \sum_{n=1}^{M}\cos (\Psi_{n,k})\cos \left[\omega_d t \cos \left(\dfrac{2\pi n - \pi + \theta}{4M}\right) + \phi_k \right] \right\} + j\left\{ \sum_{n=1}^{M}\sin (\Psi_{n,k})\cos \left[\omega_d t \cos \left(\dfrac{2\pi n - \pi + \theta}{4M}\right) + \phi_k \right] \right\}
$$

Primeiro é preciso instanciar as bibliotecas para gerar um distribuição uniforme para $\theta$, $\phi$ e $\Psi_n$ entre $[-\pi,\pi)$

```python
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

fig, ax = plt.subplots(figsize=(25, 15))
M = 8
const = np.sqrt(2/M)
ts = np.arange(0, 100, step=0.025)
Nstats = [10, 50, 100]
colors = ['r', 'g', 'b']

trials = {}
for i, Nstat in enumerate(Nstats):
    for N in range(Nstat):
        Xc_t = []
        theta_multi = np.random.uniform(-np.pi, np.pi)
        phi_multi = np.random.uniform(-np.pi, np.pi)
        psi_multi = np.random.uniform(-np.pi, np.pi, size=M)

        for t in ts:
            def return_an(n):
                return (2*np.pi*n-np.pi+theta_multi)/(4*M)

            Xc = np.sum(np.array([np.cos(psi_multi[m-1])*np.cos(2*np.pi*t*np.cos(return_an(m)+phi_multi))
                                  for m in range(1, M+1)]))
            Xc_t.append(const*Xc)

        Xc_t = np.array(Xc_t)
        trials[N] = Xc_t
    df = pd.DataFrame(trials)
    averages = df.mean(axis=1)

    c = np.correlate(averages.values, averages.values, mode='same')
    sns.lineplot(ts, c / c.max(), color=colors[i], label='Nstat ' + str(Nstat), ax=ax)

    ax.set_xlim(50, 65)

ax.legend()

ax.set_xticklabels(list(range(0, 15, 2)))
ax.set_xlabel('Tempo Normalizado')
ax.set_ylabel('Autocorrelação X_c(t)')
```

```python

```

```python

```

```python

```

```python
E_0 = np.sqrt(2)
C_n = 1/np.sqrt(4*M+2)
phi_multi = np.random.uniform(-np.pi, np.pi)
gc_t = []
ts = np.arange(0, 30, step=0.025)
M = 8

phi_n = np.random.uniform(-np.pi, np.pi)
a_n = np.random.uniform(-np.pi, np.pi)

for t in ts:
    def return_an(n):
        return p.random.uniform(-np.pi, np.pi)(2*np.pi*n/(4*M+2))

    gc = E_0*np.sum(np.array([C_n*np.cos(2*np.pi*t*np.cos(a_n))
                          for m in range(1, M+1)]))
    gc_t.append(gc)

fig, ax = plt.subplots(figsize=(25, 15))
c = np.correlate(np.array(gc_t), np.array(gc_t), mode='same')
sns.lineplot(ts, c / c.max(), ax=ax)
ax.set_xlim(15, 30)
```

```python

```

```python

```