```python
%matplotlib inline

import numpy as np
import pandas as pd

import pymc3 as pm

import matplotlib.pyplot as plt
import seaborn as sns
import arviz as az

import warnings
warnings.filterwarnings('ignore')
```

    WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

## Archimedes procedure for porous material density determination

Experimental procedure is as follow, for further detail refer to standard ASTM B962.

1. Measure temperature of liquid to determine its ($\rho_{\text{water}}$) and find object theoretical full density ( $\rho_{\text{solid}}$ ) from datasheet
2. Measure mass of sample(s) in air(A)
3. Impregnate the sample(s) with oil
4. Measure mass of sample(s) saturated with oil in air (B)
5. Measure mass of sample(s) saturated with oil in water (F)

\begin{equation}
    \text{Green Density}, D_g = \frac{A \rho_{water}}{B-F}
\end{equation}

\begin{equation}
    \text{true density}, \pi_d = \frac{ D_g}{\rho_{\text{solid}}} * 100
\end{equation}

Parts using two powder type are made. Parts labeled bimodal should have a higher density then parts labelled unimodal.

## Load Data from excel sheet

```python
data = pd.read_excel('./data/green_sample_benchmark.xlsx', sheet_name = 'oil', index_col= [0,1])
oil_density = pd.read_excel('./data/green_sample_benchmark.xlsx', sheet_name = 'oil_param', index_col= 0)
```

## Display first few row of excel sheet

```python
data.head()
```

      powder type
      dry weight
      coated weight
      wet coated
      exp
      operator

      batch
      ID

      2
      1
      Bimodal
      15.4440
      16.6229
      13.1582
      1
      1

      5
      Bimodal
      15.8476
      17.0214
      13.5416
      1
      1

      1
      2
      Unimodal
      15.2508
      16.6189
      13.0623
      1
      1

      4
      Unimodal
      15.3899
      16.6853
      13.1755
      1
      1

      4
      1
      Unimodal
      14.8467
      16.4155
      12.7117
      2
      1

## Carrier fluid density

Density of the carrier fluid ($\rho_{\text{water}}$) is necessary for density calculation. Since the density of the fluid fluctuate depending on temperature and temperature varies from experiment to experiment, it is recorded for each experiment. For experiment 2 it was recorded at the beginning and at the end.

```python
oil_density.head()
```

      oil density
      water density

      exp

      1
      0.914
      0.99730

      2
      0.914
      0.99780

      3
      0.914
      0.99798

      3
      0.914
      0.99787

```python
def plot_hist(data, b):
    fig, axes = plt.subplots(1,3, figsize = (15,5))
    axes[0].hist(data['dry weight'], bins = b, density = True)
    axes[0].set_title('dry weight')
    axes[0].set_xlabel('weight (g)')
    axes[0].set_ylabel('%')

    axes[1].hist(data['wet coated'], bins = b, density = True)
    axes[1].set_title('wet weight')
    axes[1].set_xlabel('weight (g)')
    axes[1].set_ylabel('%')

    axes[2].hist(data['coated weight'], bins = b, density = True)
    axes[2].set_title('coated weight')
    axes[2].set_xlabel('weight (g)')
    axes[2].set_ylabel('%')

    plt.tight_layout()
    plt.show()

plot_hist(data, 20)
```

## Compute density
Each sample was measured 3 times at each stage. Since we are only interested in the uncertainty in the true density ($\pi_d$), a cartesian product of each measured weights (dry, wet and coated) and liquid density is done for each sample in order to compute true density.

```python
from itertools import product
from sklearn import preprocessing

density_df = pd.DataFrame(columns = ['batch', 'id', 'density', 'operator', 'powder'])

# density of stainless steel 316L (g/cc) taken from data sheet
den_ss316 = 8.0

unique_id = 0

# iterate over each sample to do cartesian product
for idx in data.index.unique():
    sample = data.loc[idx]

    op = int(np.unique(sample['operator']).squeeze())
    powder = str(np.unique(sample['powder type']).squeeze())
    batch = idx[0]

    d_weight = sample['dry weight'].to_numpy()
    wc_weight = sample['wet coated'].to_numpy()
    c_weight = sample['coated weight'].to_numpy()

    exp_num = sample['exp']
    # density of liquid taken from datasheet
    water_den = np.unique(oil_density.loc[exp_num]['water density'].to_numpy())

    prod = product(d_weight, wc_weight, c_weight, water_den)

    # cartesian product
    for d_w, wc_w, c_w, water_d in prod:
        #compute true density refer to equation on top
        den = d_w/(c_w - wc_w)*water_d/den_ss316*100
        new_entry = {
            'batch' : batch,
            'id' : unique_id,
            'density' : float(den.squeeze()),
            'operator' : op,
            'powder' : powder
        }

        density_df = density_df.append(new_entry, ignore_index = True)

    unique_id += 1
```

    <ipython-input-6-62aede59eea2>:11: PerformanceWarning: indexing past lexsort depth may impact performance.
      sample = data.loc[idx]

```python
ax = sns.histplot(density_df['density'])
```

```python
density_df.to_csv('./data/density_oil.csv', index=False)
```

## Setup Model

Fitted a nested model, since samples are not reused across experiment.

\begin{equation}
    y_{ijk}  = \mu_{i} + Part_{j(i)} + \epsilon_{k(ij)}
    \label{eq:Model}
\end{equation}

Each powder type follows a global mean density $\mu_{i}$. Since the measurement is destructive, therefore the same parts cannot be measured using different methods. Consequently, $Part_{j(i)}$ represent the nesting relationship between the Fixed effect of the powder and the measurement of each part. Finally, $\epsilon_{k(ij)}$ represents the measurement error ($\sigma_{gage}$). An operator and a batch random effect were previously included in the model. Including the two effects caused the sampler to diverge. The cause of the divergence was likely due to the low number of replicates for the operator(2). As for the batch effect, the removal did not change the estimated value for $\sigma_{gage}$. Therefore, it is likely that the variance due to the batch is very low. The model is set up as follows:

\begin{align*}
    \{\mu_{i} \sim Uniform(20.0, 60.0)&: i = \{\text{Unimodal},\text{Bimodal}\}\} \\
    \text{Part}_{j(i)} &\sim N(0, \sigma_{\text{part}}) \\
    \mu &= \mu_{i} + \text{Part}_{j(i)} \\
    y_{ijk} &\sim  N(\mu, \sigma_{\text{gage}})
\end{align*}
\begin{equation}
    \sigma_{\text{part}} \sim \text{HalfCauchy}(25)
    \label{eq:model_priors_p}
\end{equation}
\begin{equation}
    \sigma_{\text{gage}} \sim \text{HalfCauchy}(25)
    \label{eq:model_priors_g}
\end{equation}

```python
basic_model = pm.Model()

n_part = unique_id

powder_mean = density_df.groupby('powder').mean()['density']

part_id = density_df['id'].to_numpy().astype(np.int32)
obs_pores = density_df['density'].to_numpy().astype(np.int32)

# Relabel data to integers
powder_enc = preprocessing.LabelEncoder()
powder_enc.fit(['Unimodal','Bimodal'])
powder_data = powder_enc.transform(density_df['powder'])

#compute starting point for mean inference
test_val_mu = [powder_mean[c] for c in powder_enc.classes_]

A = 10000
with basic_model:

    # setup independent mu_i
    mu_p = pm.Uniform('muP', 20.0, 60.0, shape = 2, testval = test_val_mu)
    mu_real = pm.math.switch(powder_data, mu_p[1], mu_p[0])

    # setup prior for part variance and error variance
    sig_repeat = pm.HalfCauchy('sig_repeat', 25)
    sig_part = pm.HalfCauchy('sig_part', 25)

    # each part will vary independently (nesting relationship)
    P_t = pm.Normal('P_t', mu = 0, sigma = 1, shape = n_part)

    # setup mu
    mu = pm.Deterministic('mu', mu_real + P_t[part_id]*sig_part)

    # add error and data to model
    porosity = pm.Normal('density', mu = mu, sigma = sig_repeat, observed = obs_pores)

```

## Run Inference

```python
with basic_model:
    trace = pm.sample(A, chains = 4, tune = 1000, target_accept=0.9)
    prior = pm.sample_prior_predictive()
    posterior_predictive = pm.sample_posterior_predictive(trace)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [P_t, sig_part, sig_repeat, muP]
    Sampling 4 chains, 0 divergences: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 44000/44000 [02:04<00:00, 354.41draws/s]
    The number of effective samples is smaller than 25% for some parameters.
    100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40000/40000 [00:35<00:00, 1135.83it/s]

Convert trace to arviz data format for diagnostic

```python
trace_data_az = az.from_pymc3(
    trace=trace,
    prior=prior,
    posterior_predictive=posterior_predictive,
    model=basic_model
)
```

```python
trace_data_az.to_netcdf('./data/oil_trace.nc')
```

    './data/oil_trace.nc'

## Trace plot
The sampler is exploring the probability space by sampling different values of the model input parameters ($\mu_i, \sigma_{\text{part}}, \sigma_{\text{gage}}$). The left graph shows the probability density of the sampled values. The right graph shows the sampled with respect to sample number. The left graph is computed by summing up the right graph. When the left graph looks skewed towards zero or the right graph does not resemble a thick caterpillar, this indicates a biased sampling and a lack of convergence.

```python
pm.traceplot(trace, var_names=['muP','sig_repeat','sig_part'])
plt.show()
```

## Predictive check plot

Predictive posterior plot overlays data generated by the model's posterior over the real data. This is useful to check the fitness of the model.

```python
az.plot_ppc(trace_data_az, data_pairs={"density":"density"}, kind = 'scatter')
plt.show()
```

## Forest plot

Forest plot on estimated real mean $\mu_{i}$ to check if the values make sense

```python
pm.plots.forestplot(trace, var_names=['muP'])
plt.show()
```

## Compute split $\hat{R}$

In the Markov-Chain Monte Carlo (MCMC) sampler, the sampling is done simultaneously in multiple processes (chain). The rank normalized $\hat{R}$ checks for convergence by comparing the variance between chain with the variance within chain. If convergence has been achieved then the variance should be the same ($\hat{R}=1$) (Vehtari et al). [Vehtari et al.](https://arxiv.org/pdf/1903.08008v1.pdf "vehtari") recommends a $\hat{R} < 1.01$.

```python
az.rhat(trace_data_az)
```

<svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>

<xarray.Dataset>
Dimensions:     (P_t_dim_0: 35, muP_dim_0: 2, mu_dim_0: 372)
Coordinates:
  * P_t_dim_0   (P_t_dim_0) int32 0 1 2 3 4 5 6 7 8 ... 27 28 29 30 31 32 33 34
  * muP_dim_0   (muP_dim_0) int32 0 1
  * mu_dim_0    (mu_dim_0) int32 0 1 2 3 4 5 6 7 ... 365 366 367 368 369 370 371
Data variables:
    P_t         (P_t_dim_0) float64 1.0 1.0 1.001 1.001 ... 1.001 1.001 1.001
    muP         (muP_dim_0) float64 1.001 1.001
    sig_repeat  float64 1.0
    sig_part    float64 1.001
    mu          (mu_dim_0) float64 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0 1.0

xarray.Dataset

<input id='section-5353a7e8-1e1b-446b-ae6f-584264554b2d' class='xr-section-summary-in' type='checkbox' disabled >Dimensions:

P_t_dim_0: 35

muP_dim_0: 2

mu_dim_0: 372

<input id='section-98590606-ba90-43c9-882b-a4191a94b4d8' class='xr-section-summary-in' type='checkbox'  checked>Coordinates: (3)

P_t_dim_0

(P_t_dim_0)

int32

0 1 2 3 4 5 6 ... 29 30 31 32 33 34
<input id='attrs-b15a0352-c5cf-4ef8-a5c1-efc13f1e0dad' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-4111e386-6ff3-4b30-ba67-3eb9144e5dbc' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
       18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34])

muP_dim_0

(muP_dim_0)

int32

0 1
<input id='attrs-e6db880f-476e-4216-a7a4-a3834efb9c93' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-8f51d16a-f4d5-4954-93e6-e7195f04ab60' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([0, 1])

mu_dim_0

(mu_dim_0)

int32

0 1 2 3 4 5 ... 367 368 369 370 371
<input id='attrs-a1dbf18f-71ef-4247-a3d8-7d934da77364' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-91c45034-9c0a-48d0-b329-fe87415677ae' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([  0,   1,   2, ..., 369, 370, 371])

<input id='section-0c29a465-dfdf-4d6a-a877-9016a5cb0c4c' class='xr-section-summary-in' type='checkbox'  checked>Data variables: (5)

P_t

(P_t_dim_0)

float64

1.0 1.0 1.001 ... 1.001 1.001 1.001
<input id='attrs-a954184d-14f3-4c78-b8dd-2202b627b67b' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-c068a09f-8244-41e5-8ab3-45c651abdad7' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([1.0002049 , 1.0004686 , 1.00095466, 1.00101573, 1.00072378,
       1.00092097, 1.00135362, 1.00121031, 1.00065131, 1.00083459,
       1.00067765, 1.00062369, 1.00074333, 1.00070743, 1.00064432,
       1.00077924, 1.00112069, 1.00069368, 1.00125097, 1.00119211,
       1.00058562, 1.00050884, 1.00077031, 1.00073721, 1.00049027,
       1.00038773, 1.00038226, 1.0009259 , 1.00094602, 1.00049631,
       1.00138614, 1.00124608, 1.00141291, 1.00135585, 1.00112458])

muP

(muP_dim_0)

float64

1.001 1.001
<input id='attrs-b48e01b2-05ae-4e35-a6b6-ec1be9407229' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-5bb5196d-9bcb-4aa5-8c7f-94a00dcf4fa6' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([1.00082541, 1.00129751])

sig_repeat

()

float64

1.0
<input id='attrs-bc4071f1-3908-475e-897e-3829c55cfc10' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-1973cf13-fd79-4abe-bbbf-025627cd8649' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array(1.0000968)

sig_part

()

float64

1.001
<input id='attrs-65f912a8-cfc5-4c33-90e2-c9cc8f4d8db7' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-33ae0cad-7c94-4a44-a63c-56bd8464b1e5' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array(1.00056165)

mu

(mu_dim_0)

float64

1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0
<input id='attrs-073ce6d5-92f4-4e7f-9491-00e7eed37ea6' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-b23e3ac1-0368-4587-8c96-232595bc1c92' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([1.00008943, 1.00011422, 1.00014336, 1.00008789, 0.99998241,
       0.99998241, 0.99998241, 0.99998241, 0.99998241, 0.99998241,
       0.99998241, 0.99998241, 1.00015302, 1.00015302, 1.00015302,
       1.00015302, 1.00015302, 1.00015302, 1.00015302, 1.00015302,
       1.00012718, 1.00012718, 1.00012718, 1.00012718, 1.00012718,
       1.00012718, 1.00012718, 1.00012718, 0.99996314, 0.99996314,
       0.99996314, 0.99996314, 0.99996314, 0.99996314, 0.99996314,
       0.99996314, 0.9999983 , 0.9999983 , 0.9999983 , 0.9999983 ,
       0.9999983 , 0.9999983 , 0.9999983 , 0.9999983 , 1.0000605 ,
       1.0000605 , 1.0000605 , 1.0000605 , 1.0000605 , 1.0000605 ,
       1.0000605 , 1.0000605 , 1.00017523, 1.00017523, 1.00017523,
       1.00017523, 1.00017523, 1.00017523, 1.00017523, 1.00017523,
       1.000311  , 1.000311  , 1.000311  , 1.000311  , 1.000311  ,
       1.000311  , 1.000311  , 1.000311  , 1.00010119, 1.00010119,
       1.00010119, 1.00010119, 1.00010119, 1.00010119, 1.00010119,
       1.00010119, 1.0000028 , 1.0000028 , 1.0000028 , 1.0000028 ,
       1.0000028 , 1.0000028 , 1.0000028 , 1.0000028 , 1.00014934,
       1.00014934, 1.00014934, 1.00014934, 1.00014934, 1.00014934,
       1.00014934, 1.00014934, 0.99997986, 0.99997986, 0.99997986,
       0.99997986, 0.99997986, 0.99997986, 0.99997986, 0.99997986,
...
       1.00012857, 0.99998139, 0.99998139, 0.99998139, 0.99998139,
       0.99998139, 0.99998139, 0.99998139, 0.99998139, 0.99998139,
       0.99998139, 0.99998139, 0.99998139, 0.99998139, 0.99998139,
       0.99998139, 0.99998139, 0.99995942, 0.99995942, 0.99995942,
       0.99995942, 0.99995942, 0.99995942, 0.99995942, 0.99995942,
       0.99995942, 0.99995942, 0.99995942, 0.99995942, 0.99995942,
       0.99995942, 0.99995942, 0.99995942, 1.00034156, 1.00034156,
       1.00034156, 1.00034156, 1.00034156, 1.00034156, 1.00034156,
       1.00034156, 1.00034156, 1.00034156, 1.00034156, 1.00034156,
       1.00034156, 1.00034156, 1.00034156, 1.00034156, 1.00029664,
       1.00029664, 1.00029664, 1.00029664, 1.00029664, 1.00029664,
       1.00029664, 1.00029664, 1.00029664, 1.00029664, 1.00029664,
       1.00029664, 1.00029664, 1.00029664, 1.00029664, 1.00029664,
       0.99997716, 0.99997716, 0.99997716, 0.99997716, 0.99997716,
       0.99997716, 0.99997716, 0.99997716, 0.99997716, 0.99997716,
       0.99997716, 0.99997716, 0.99997716, 0.99997716, 0.99997716,
       0.99997716, 1.00003034, 1.00003034, 1.00003034, 1.00003034,
       1.00003034, 1.00003034, 1.00003034, 1.00003034, 1.00003034,
       1.00003034, 1.00003034, 1.00003034, 1.00003034, 1.00003034,
       1.00003034, 1.00003034])

<input id='section-eee6565f-6cba-4c09-a0de-640e4274a6a6' class='xr-section-summary-in' type='checkbox' disabled >Attributes: (0)

## Calculate highest density interval (HDI)

Taken from [arviz doc](https://arviz-devs.github.io/arviz/api/generated/arviz.hdi.html#arviz.hdi "arviz") : The HDI is the minimum width Bayesian credible interval (BCI).

```python
hdi_data = az.hdi(trace_data_az)
hdi_data.to_netcdf('./data/oil_inference.nc')
hdi_data
```

<svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>

<xarray.Dataset>
Dimensions:     (P_t_dim_0: 35, hdi: 2, muP_dim_0: 2, mu_dim_0: 372)
Coordinates:
  * P_t_dim_0   (P_t_dim_0) int32 0 1 2 3 4 5 6 7 8 ... 27 28 29 30 31 32 33 34
  * muP_dim_0   (muP_dim_0) int32 0 1
  * mu_dim_0    (mu_dim_0) int32 0 1 2 3 4 5 6 7 ... 365 366 367 368 369 370 371
  * hdi         (hdi) <U6 'lower' 'higher'
Data variables:
    P_t         (P_t_dim_0, hdi) float64 0.0504 1.327 0.5333 ... -0.5963 0.3443
    muP         (muP_dim_0, hdi) float64 52.89 54.52 50.89 52.58
    sig_repeat  (hdi) float64 0.4083 0.4722
    sig_part    (hdi) float64 1.392 2.258
    mu          (mu_dim_0, hdi) float64 54.11 55.73 55.05 ... 51.71 51.3 51.71

xarray.Dataset

<input id='section-e4f3ae9d-e80e-4203-855d-39b5767a3268' class='xr-section-summary-in' type='checkbox' disabled >Dimensions:

P_t_dim_0: 35

hdi: 2

muP_dim_0: 2

mu_dim_0: 372

<input id='section-f3cc88dc-3c42-488e-87ed-afc4abdfbe7d' class='xr-section-summary-in' type='checkbox'  checked>Coordinates: (4)

P_t_dim_0

(P_t_dim_0)

int32

0 1 2 3 4 5 6 ... 29 30 31 32 33 34
<input id='attrs-0328c5ab-2a9a-4de7-ad82-925774023f7b' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-b6d1cf2f-80fe-4118-aac9-28494ea09c8c' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
       18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34])

muP_dim_0

(muP_dim_0)

int32

0 1
<input id='attrs-d9e652f5-dd45-46d9-a266-ebf859e32930' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-2d2d6a85-0b18-4d47-8f43-9e65d064520c' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([0, 1])

mu_dim_0

(mu_dim_0)

int32

0 1 2 3 4 5 ... 367 368 369 370 371
<input id='attrs-267f65b1-c91f-47d2-8b55-51563386fea9' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-ce53b1d4-2030-46ec-addb-2e750bce9210' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([  0,   1,   2, ..., 369, 370, 371])

hdi

(hdi)

<U6

'lower' 'higher'
<input id='attrs-1f6c53ac-585e-4a22-b7fe-bd8a6ba25e53' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-2be6e831-1c61-4a68-bf97-88caf6cdc0d2' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

hdi_prob :

0.94

array(['lower', 'higher'], dtype='<U6')

<input id='section-c31dd39e-c220-4997-a2f7-196c6a2d2b0c' class='xr-section-summary-in' type='checkbox'  checked>Data variables: (5)

P_t

(P_t_dim_0, hdi)

float64

0.0504 1.327 ... -0.5963 0.3443
<input id='attrs-df5d3a0b-48fb-49e0-86c1-ef7dbe42df39' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-a244070c-a58a-42fd-a0b5-c4766d40a246' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([[ 0.05040454,  1.3265142 ],
       [ 0.53327834,  1.86816929],
       [ 0.01482925,  1.29521847],
       [ 0.51843802,  1.8618607 ],
       [-1.79185197, -0.67161134],
       [-0.90661369,  0.07089492],
       [ 0.1833449 ,  1.20775401],
       [-0.6175607 ,  0.34273097],
       [-1.45708913, -0.42550164],
       [-1.32303253, -0.31471766],
       [-0.86721787,  0.09263776],
       [-1.30992114, -0.29903838],
       [-1.46028691, -0.42457789],
       [-1.17548948, -0.18484207],
       [-0.6560623 ,  0.28705155],
       [-1.16186996, -0.18106307],
       [-0.6042446 ,  0.35938508],
       [-1.56164202, -0.49317852],
       [-0.32584639,  0.65041043],
       [-0.06482608,  0.92331093],
       [ 0.24711599,  1.22131828],
       [ 0.49664023,  1.52624722],
       [-1.45458396, -0.43668998],
       [-0.85846451,  0.07204972],
       [ 0.74856018,  1.8451485 ],
       [ 1.20998091,  2.47500504],
       [ 0.7347091 ,  1.82408721],
       [-1.77238031, -0.69448916],
       [-0.87922244,  0.07871655],
       [-3.24595933, -1.75259856],
       [ 0.45738823,  1.50938758],
       [-0.31946214,  0.62731909],
       [ 0.20718706,  1.20856983],
       [ 0.45394116,  1.50414406],
       [-0.59634912,  0.34431323]])

muP

(muP_dim_0, hdi)

float64

52.89 54.52 50.89 52.58
<input id='attrs-761c2265-2f7f-41ea-a913-7be38572fd34' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-ac7fc6b7-8896-4476-82f3-d6856cd80027' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([[52.89194285, 54.5183393 ],
       [50.88789518, 52.57556468]])

sig_repeat

(hdi)

float64

0.4083 0.4722
<input id='attrs-3693b959-60bf-4b56-a95e-e1cd6e95db16' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-c104bd0f-02b3-413e-a629-b58cc6f5bb5c' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([0.40831727, 0.47223401])

sig_part

(hdi)

float64

1.392 2.258
<input id='attrs-30612f66-63a0-45a5-b7be-89801bce9a18' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-52fbb7b4-8131-42af-890c-268a8776a094' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([1.3924772 , 2.25828961])

mu

(mu_dim_0, hdi)

float64

54.11 55.73 55.05 ... 51.3 51.71
<input id='attrs-9d039004-7c0c-493b-bdc6-18ea07de2d9b' class='xr-var-attrs-in' type='checkbox' disabled><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-4aab98ab-e19e-4230-8e10-b93983ecb695' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

array([[54.11365302, 55.72556442],
       [55.05415815, 56.68806652],
       [52.1290023 , 53.73764389],
       [53.0614181 , 54.6725131 ],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [49.2242649 , 49.80800232],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
       [50.71126332, 51.29733727],
...
       [53.29068453, 53.70281272],
       [53.29068453, 53.70281272],
       [53.29068453, 53.70281272],
       [53.29068453, 53.70281272],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639],
       [51.29913971, 51.71047639]])

<input id='section-d37c91b6-c365-4339-8a84-af4cdce88d49' class='xr-section-summary-in' type='checkbox' disabled >Attributes: (0)


```python
%load_ext autoreload
%autoreload 2
%matplotlib inline

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import sys
import os

# Set path to use 'src' subdir
sys.path.append(os.path.join(os.getcwd(), 'src'))
```

# Classification model

This is a toy exampe using a simple classification model. The data is created using three numerical variables.

\begin{align}
z & = 3 x_1 - x_2 + \frac{x_3}{2} + 0.1 n\\
y & = \left[
\begin{array}{ll}
'low' & z < -3 \\
'mid' & z < 3 \\
'hi'  & ! \\
\end{array}
\right. \\
\end{align}

where:
- x1 : Normally distributted N(0, 1)
- x2 : Normally distributted N(2, 2)
- x3 : Normally distributted N(-3, 5)
- n : Random noise, Normally distributted N(0, 1)

Here is the code for creating the dataset

```python
def to_class(c):
    if c < -3:
        return 'low'
    if c < 3:
        return 'mid'
    return 'high'

# Create dataset
def create_dataset():
    # Number of samples
    num = 2000
    # Inputs: x1, x2, x3
    x1 = np.random.normal(0, 1, num)
    x2 = np.random.normal(2, 3, num)
    x3 = np.random.normal(-3, 5, num)
    # Noise
    n = np.random.normal(0, 1, num)
    # Output
    y = 3. * x1 - 1. * x2 + 0.5 * x3 + 0.1 * n
    # Categorical output
    y_str = np.array([to_class(c) for c in y])
    # Add missing data
    x1_na = (np.random.rand(num) < 0.01)
    x2_na = (np.random.rand(num) < 0.01)
    x3_na = (np.random.rand(num) < 0.01)
    x1[x1_na] = np.nan
    x2[x2_na] = np.nan
    x3[x3_na] = np.nan
    y_na = (np.random.rand(num) < 0.01)
    y_str[y_na] = ''
    # Create dataFrame
    df = pd.DataFrame({'x1': x1, 'x2': x2, 'x3': x3, 'y': y_str})
    df.to_csv('class3.csv', index=False)
```

# LogMl

We create a LogMl object, and then run it

```python
!rm -rvf data/class3/class3.pkl logml_plots/class3
```

```python
from logml import *

ml = LogMl('config/class3.yaml', verbose=True)
ml()
```

```python

```