# Multi-objective robust decision making (MORDM)

This exercise demostrates the application of MORDM on the lake model, which was used in earlier exercises.

MORDM has four main steps:

(i)	    **problem formulation** based on a systems analytical problem definition framework

(ii)	**searching** for candidate solutions that optimize multiple objectives by using multi-objective evolutionary algorithms

(iii)	generating an ensemble of scenarios to **explore** the effects of uncertainties

(iv)	using **scenario discovery** to detect the vulnerabilities of candidate solutions and improving thecandidate solutions

## Step 1: Problem formulation
### Lake Model

Remember the lake problem used in the assignments in previous weeks. The lake problem is a hypothetical case where the inhabitants of a lake town decide on the amount of annual pollution they release into a lake. It the pollution in the lake passes a threshold, it will suffer irreversible eutrophication.

The lake problem has 4 **outcome indicators**:
   - **max_P**: maximum pollution over time, to be minimized
   - **utility**: economic benefits obtained from polluting the lake, to be maximized
   - **inertia**: the percentage of significant annual changes in the anthropogenic pollution rate, to be maximized
   - **reliability**: the percentage of years where the pollution level is below the critical threshold, to be maximized

See the lake model exercise for the formulation of these outcome variables.

The lake problem is characterized by both stochastic uncertainty and **deep uncertainty**. The stochastic uncertainty arises from the natural inflow. To reduce this stochastic uncertainty, multiple replications are performed and the average over the replication is taken. Deep uncertainty is presented by uncertainty about the mean $\mu$ and standard deviation $sigma$ of the lognormal distribution characterizing the natural inflow, the natural removal rate of the lake $\beta$, the natural recycling rate of the lake $q$, and the discount rate $\delta$. The table below specifies the ranges for the deeply uncertain factors, as well as their best estimate or default values.

|Parameter	|Range	        |Default value|
|-----------|--------------:|------------:|
|$\mu$    	|0.01 – 0.05	|0.02         |
|$\sigma$	|0.001 – 0.005 	|0.0017       |
|$b$      	|0.1 – 0.45	    |0.42         |
|$q$	    |2 – 4.5	    |2            |
|$\delta$	|0.93 – 0.99	|0.98         |

The lake problem in previous assignments had 100 decision **levers**, meaning that the decision makers independently decide on the amount of anthropogenic pollution at every time step (100). Then a 'policy' was a set of values for these 100 levers, which you composed by sampling from the range [0, 0.1].

In this exercise, we will use a more advanced way of deciding on the amout of anhtropogenic polution. We will use a **closed loop** version of the lake model, meaning that $a_t$ (anthropogenic pollution) is dependent on $X_t$ (the pollution level at time t). For instance, the rate of anthropogenic pollutions is lowered if the pollution level is approaching a critical threshold. Here, we use "cubic radial basis functions" following [Quinn et al. 2017](http://www.sciencedirect.com/science/article/pii/S1364815216302250) and formulate $a_t$ as follows:

\begin{equation}
    a_{t} =  min\Bigg(max\bigg(\sum\limits_{j=1}^{n} w_{j}\left\vert{\frac{X_{t,i}-c_{j}}{r_{j}}}\right\vert^3, 0.01\bigg), 0.1\Bigg) \\
    s.t. \\
    -2 \leq c_{j} \leq 2 \\
    0 \leq r_{j} \leq 2 \\
    0 \leq w_{j} \leq 1 \\
    \sum\limits_{j=1}^{n} w_{j} = 1
\end{equation}

The parameters that define this function also define the pollution strategy over time. Hence, the decision **levers** are the five parameters of this functions, namely $c_1$, $c_2$, $r_1$, $r_2$ and $w_1$. ($w_2$ = 1 - $w_1$).

Note:: i is index for the realization, given m realizations; j is the index for the radial basis function, given 2 radial basis functions.

**To formulate this problem, do the following:**

**1) Import the lake model function from dps_lake_model.py**

**2) Create an ema_workbench interface for this problem, with corresponding uncertainties, levers and outcomes as specified above**

```python
from dps_lake_model import lake_model
from ema_workbench import (Model, RealParameter, ScalarOutcome,
                           MultiprocessingEvaluator, ema_logging,
                           Constant)

from ema_workbench.em_framework.optimization import (HyperVolume,
                                                     EpsilonProgress)
from ema_workbench import Constraint
from ema_workbench.analysis import parcoords
from ema_workbench import save_results

import pandas as pd
import matplotlib.pyplot as plt
import os
```

    C:\Users\jryap\anaconda3\envs\mbdm\lib\site-packages\ema_workbench\analysis\prim.py:31: ImportWarning: altair based interactive inspection not available
      "inspection not available"), ImportWarning)

```python
ema_logging.log_to_stderr(ema_logging.INFO)

#instantiate the model
lake_m = Model('lakemodel', function=lake_model)
lake_m.time_horizon = 100 # used to specify the number of timesteps

#specify uncertainties
lake_m.uncertainties = [RealParameter('mean', 0.01, 0.05),
                            RealParameter('stdev', 0.001, 0.005),
                            RealParameter('b', 0.1, 0.45),
                            RealParameter('q', 2.0, 4.5),
                            RealParameter('delta', 0.93, 0.99)]

# set levers, one for each time step
lake_m.levers = [RealParameter('c1', -2, 2),
                     RealParameter('c2', -2, 2),
                     RealParameter('r1', 0, 2),
                     RealParameter('r2', 0, 2),
                     RealParameter('w1', 0, 1)]

#specify outcomes
lake_m.outcomes = [ScalarOutcome('max_P', kind=ScalarOutcome.MINIMIZE,
                                     expected_range=(0,5)),
                       ScalarOutcome('utility', kind=ScalarOutcome.MAXIMIZE,
                                     expected_range=(0,2)),
                       ScalarOutcome('inertia', kind=ScalarOutcome.MAXIMIZE,
                                    expected_range=(0,1)),
                       ScalarOutcome('reliability', kind=ScalarOutcome.MAXIMIZE,
                                     expected_range=(0,1))]

# lake_model.constantcs = [Constant('alpha', 0.41),
#                          Constant('reps', 150)],
```

## Step 2: Searching for candidate solutions

In the second step of MORDM, candidate strategies are identified which are pareto optimal conditional on a reference scenario. These candiate strategies are identified through search with multi-objective evolutionary algorithms, that iteratively evaluate a large number of alternatives on multiple objectives until they find the best candidates. For instance, when we optimize the lake model levers, the lake model function will be called for each candidate evaluation, and the corresponding four objective values will be generated.

Take the model interface developed in the previous step and use the optimization functionality of the workbench to identify the pareto approximate set of solutions. Try the following:
* change the epsilon values between 0.01 and 0.1, what changes, why?
* change the number of function evaluations from 1000 to 10.000 (this requires using multiprocessing unless you are very patient). What is the difference? You can use  convergence as explained in assignment 7 for this

```python
convergence_metrics = [HyperVolume.from_outcomes(lake_m.outcomes),
                       EpsilonProgress()]

constraints = [Constraint("max pollution", outcome_names="max_P",
                          function=lambda x:max(0, x-5))]
```

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    results_eps1, convergence_eps1 = evaluator.optimize(nfe=5000, searchover='levers',
                                    epsilons=[0.1, 0.1, 0.1, 0.1],
                                    convergence=convergence_metrics,
                                    constraints=constraints)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] generation 0: 0/5000 nfe
    [MainProcess/INFO] generation 5: 498/5000 nfe
    [MainProcess/INFO] generation 10: 991/5000 nfe
    [MainProcess/INFO] generation 15: 1486/5000 nfe
    [MainProcess/INFO] generation 20: 1984/5000 nfe
    [MainProcess/INFO] generation 25: 2483/5000 nfe
    [MainProcess/INFO] generation 30: 2978/5000 nfe
    [MainProcess/INFO] generation 35: 3469/5000 nfe
    [MainProcess/INFO] generation 40: 3954/5000 nfe
    [MainProcess/INFO] generation 45: 4451/5000 nfe
    [MainProcess/INFO] generation 50: 4947/5000 nfe
    [MainProcess/INFO] optimization completed, found 16 solutions
    [MainProcess/INFO] terminating pool

    Wall time: 2min 7s

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    results_eps2, convergence_eps2 = evaluator.optimize(nfe=5000, searchover='levers',
                                    epsilons=[0.01, 0.01, 0.01, 0.01],
                                    convergence=convergence_metrics,
                                    constraints=constraints)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] generation 0: 0/5000 nfe
    [MainProcess/INFO] generation 5: 497/5000 nfe
    [MainProcess/INFO] generation 10: 992/5000 nfe
    [MainProcess/INFO] generation 15: 1486/5000 nfe
    [MainProcess/INFO] generation 20: 1981/5000 nfe
    [MainProcess/INFO] generation 25: 2476/5000 nfe
    [MainProcess/INFO] generation 30: 2970/5000 nfe
    [MainProcess/INFO] generation 35: 3464/5000 nfe
    [MainProcess/INFO] generation 40: 3960/5000 nfe
    [MainProcess/INFO] generation 45: 4455/5000 nfe
    [MainProcess/INFO] generation 50: 4950/5000 nfe
    [MainProcess/INFO] optimization completed, found 178 solutions
    [MainProcess/INFO] terminating pool

    Wall time: 3min 36s

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    results_nfe1, convergence_nfe1 = evaluator.optimize(nfe=1000, searchover='levers',
                                    epsilons=[0.05, 0.05, 0.05, 0.05],
                                    convergence=convergence_metrics,
                                    constraints=constraints)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] generation 0: 0/1000 nfe
    [MainProcess/INFO] generation 5: 494/1000 nfe
    [MainProcess/INFO] generation 10: 988/1000 nfe
    [MainProcess/INFO] optimization completed, found 20 solutions
    [MainProcess/INFO] terminating pool

    Wall time: 42.9 s

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    results_nfe2, convergence_nfe2 = evaluator.optimize(nfe=10000, searchover='levers',
                                    epsilons=[0.05, 0.05, 0.05, 0.05],
                                    convergence=convergence_metrics,
                                    constraints=constraints)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] generation 0: 0/10000 nfe
    [MainProcess/INFO] generation 5: 497/10000 nfe
    [MainProcess/INFO] generation 10: 995/10000 nfe
    [MainProcess/INFO] generation 15: 1492/10000 nfe
    [MainProcess/INFO] generation 20: 1990/10000 nfe
    [MainProcess/INFO] generation 25: 2478/10000 nfe
    [MainProcess/INFO] generation 30: 2975/10000 nfe
    [MainProcess/INFO] generation 35: 3471/10000 nfe
    [MainProcess/INFO] generation 40: 3961/10000 nfe
    [MainProcess/INFO] generation 45: 4451/10000 nfe
    [MainProcess/INFO] generation 50: 4948/10000 nfe
    [MainProcess/INFO] generation 55: 5440/10000 nfe
    [MainProcess/INFO] generation 60: 5934/10000 nfe
    [MainProcess/INFO] generation 65: 6428/10000 nfe
    [MainProcess/INFO] generation 70: 6920/10000 nfe
    [MainProcess/INFO] generation 75: 7413/10000 nfe
    [MainProcess/INFO] generation 80: 7908/10000 nfe
    [MainProcess/INFO] generation 85: 8407/10000 nfe
    [MainProcess/INFO] generation 90: 8904/10000 nfe
    [MainProcess/INFO] generation 95: 9402/10000 nfe
    [MainProcess/INFO] generation 100: 9897/10000 nfe
    [MainProcess/INFO] optimization completed, found 39 solutions
    [MainProcess/INFO] terminating pool

    Wall time: 5min 40s

Observations:
- By decreasing the epsilons, we see that the number of solutions increases significantly (from 16 with 0.1 to 178 with 0.01).
- Increasing the number of function evaluations produces a similar effect on the experiment results, the number of solutions increases significantly. The reason why number of solutions increased with more NFEs is that the algorithm had not converged yet.

**plot the tradeoffs you have found using a parallel axis plot**

We can visualize these tradeoffs on **parallel axis plots**. In these plots, each dimension is shown as a vertical axis. Each solution is represented by a line on this plot, which crosses the objective axes at the corresponsing value. You can use the [parcoords functionality](https://emaworkbench.readthedocs.io/en/latest/ema_documentation/analysis/parcoords.html) for this that comes with the ema_workbench. Ensure that the direction of desirability is the same for the four objectives.|

```python
# plt.subplots(nrows = 4)

outcomes1 = results_eps1.loc[:, ['max_P', 'utility', 'reliability', 'inertia']]
outcomes2 = results_eps2.loc[:, ['max_P', 'utility', 'reliability', 'inertia']]
outcomes3 = results_nfe1.loc[:, ['max_P', 'utility', 'reliability', 'inertia']]
outcomes4 = results_nfe2.loc[:, ['max_P', 'utility', 'reliability', 'inertia']]

limits1 = parcoords.get_limits(outcomes1)
ax[0] = parcoords.ParallelAxes(limits1)
ax[0].invert_axis('max_P')
ax[0].plot(outcomes1)

limits2 = parcoords.get_limits(outcomes2)
ax[1] = parcoords.ParallelAxes(limits2)
ax[1].invert_axis('max_P')
ax[1].plot(outcomes2)

limits3 = parcoords.get_limits(outcomes3)
ax[2] = parcoords.ParallelAxes(limits3)
ax[2].invert_axis('max_P')
ax[2].plot(outcomes3)

limits4 = parcoords.get_limits(outcomes4)
ax[3] = parcoords.ParallelAxes(limits4)
ax[3].invert_axis('max_P')
ax[3].plot(outcomes4)

# we invert this axis so direction of desirability is the same

plt.show()
```

**What does this plot tell us about the tradeoffs and conflicting objectives?**

Observations:
- The tradeoffs and conflicting objectives is that lowering pollution and increasing reliability (positive outcomes) will decrease utility (undesirable) and can sometimes reduce inertia (also undesirable).

- Inertia does not always get reduced with minimized pollution however. There is a scenario outcome where minimized pollution and a high level of reliability and less utility (but not the minimum level) of utility still leads to maximum inertia.

- In scenarios in which pollution is allowed to rise above desirable levels, we observe that there is still a tradeoff between reliability, utility and inertia: the model cannot achieve high reliability, high utility and high inertia in a single scenario. Thus, it is always better to minimizae pollution, as only one outcome will have to be traded off (utility).

**Evaluating convergence metrics**

```python
fig, ax = plt.subplots(nrows=4, ncols=2, sharex=True, figsize=(12,8))
ax[0,0].plot(convergence_eps1.nfe, convergence_eps1.epsilon_progress)
ax[0,0].set_ylabel('$\epsilon$-progress')

ax[0,1].plot(convergence_eps1.nfe, convergence_eps1.hypervolume)
ax[0,1].set_ylabel('hypervolume')

ax[1,0].plot(convergence_eps2.nfe, convergence_eps2.epsilon_progress)
ax[1,0].set_ylabel('$\epsilon$-progress')

ax[1,1].plot(convergence_eps2.nfe, convergence_eps2.hypervolume)
ax[1,1].set_ylabel('hypervolume')

ax[2,0].plot(convergence_nfe1.nfe, convergence_nfe1.epsilon_progress)
ax[2,0].set_ylabel('$\epsilon$-progress')

ax[2,1].plot(convergence_nfe1.nfe, convergence_nfe1.hypervolume)
ax[2,1].set_ylabel('hypervolume')

ax[3,0].plot(convergence_nfe2.nfe, convergence_nfe2.epsilon_progress)
ax[3,0].set_ylabel('$\epsilon$-progress')

ax[3,1].plot(convergence_nfe2.nfe, convergence_nfe2.hypervolume)
ax[3,1].set_ylabel('hypervolume')

ax[0,0].set_xlabel('number of function evaluations')
ax[0,1].set_xlabel('number of function evaluations')
ax[1,0].set_xlabel('number of function evaluations')
ax[1,1].set_xlabel('number of function evaluations')
ax[2,0].set_xlabel('number of function evaluations')
ax[2,1].set_xlabel('number of function evaluations')
ax[3,0].set_xlabel('number of function evaluations')
ax[3,1].set_xlabel('number of function evaluations')
plt.show()
```

Observations:

- It seems like the hypervolume for the NFE10000 function plateau-ed after about 8000 function evaluations, indicating convergence. It does not seem like the epsilon-progress indicator converged though, meaning that a new solution is still likely to be found after 10000 function evalutions. This may indicate that more function evaluations are likely to be necessary.

## Step 3: Re-evaluate candidate solutions under uncertainty

We now have a large number of candidate solutions (policies), we can re-evaluate them over the various deeply uncertain factors to assess their robustness against uncertainties.

For this robustness evaluation, we need to explore the scenarios for each solution. It means that, if we would like to run for instance 1000 scenarios for each solution, we might have to execute a very large number of runs.

Here, to simplify the case, let's suppose that decision makers have a hard constrain on *reliability*. No solution with less than 90% reliability is acceptable for them. Therefore, we can reduce the size of the solution set according to this constraint.

**Apply this constraint of reliability on the results, and create a new dataframe named new_reults**

```python
constraints = [Constraint("reliability", outcome_names="reliability",
                          function=lambda x:max(0,0.9-x))]
```

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    new_results = evaluator.optimize(nfe=10000, searchover='levers',
                                    epsilons=[0.05, 0.05, 0.05, 0.05],
                                    convergence=convergence_metrics,
                                    constraints=constraints)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] generation 0: 0/10000 nfe
    [MainProcess/INFO] generation 5: 495/10000 nfe
    [MainProcess/INFO] generation 10: 992/10000 nfe
    [MainProcess/INFO] generation 15: 1481/10000 nfe
    [MainProcess/INFO] generation 20: 1974/10000 nfe
    [MainProcess/INFO] generation 25: 2469/10000 nfe
    [MainProcess/INFO] generation 30: 2967/10000 nfe
    [MainProcess/INFO] generation 35: 3459/10000 nfe
    [MainProcess/INFO] generation 40: 3954/10000 nfe
    [MainProcess/INFO] generation 45: 4451/10000 nfe
    [MainProcess/INFO] generation 50: 4951/10000 nfe
    [MainProcess/INFO] generation 55: 5443/10000 nfe
    [MainProcess/INFO] generation 60: 5936/10000 nfe
    [MainProcess/INFO] generation 65: 6430/10000 nfe
    [MainProcess/INFO] generation 70: 6926/10000 nfe
    [MainProcess/INFO] generation 75: 7420/10000 nfe
    [MainProcess/INFO] generation 80: 7918/10000 nfe
    [MainProcess/INFO] generation 85: 8413/10000 nfe
    [MainProcess/INFO] generation 90: 8909/10000 nfe
    [MainProcess/INFO] generation 95: 9404/10000 nfe
    [MainProcess/INFO] generation 100: 9898/10000 nfe
    [MainProcess/INFO] optimization completed, found 6 solutions
    [MainProcess/INFO] terminating pool

    Wall time: 6min 46s

```python
outcomes = new_results[0].loc[:, ['max_P', 'utility', 'reliability', 'inertia']]

limits = parcoords.get_limits(outcomes)
axes = parcoords.ParallelAxes(limits)
axes.plot(outcomes)

# we invert this axis so direction of desirability is the same
axes.invert_axis('max_P')
plt.show()
```

Observations:
- With the new constraint on reliability, it is easier to identify policies of interest, which come with the smallest tradeoff. We can see that the policy in the green line satisfies low pollution high reliability and high inertia while coming with the smallest tradeoff for utility.

**From new_results, which is the reduced dataframe of candidate solutions, make a list of policies in a format that can be inputed to the *perform_experiments* function of the EMA workbench.**

*hint: you need to transform each policy to a dict, and then use this dict as input for the Policy class that comes with the workbench*

```python
from ema_workbench import Policy
```

```python
# policies = pd.DataFrame.to_dict(new_results[0].iloc[:,:5], orient='records')

policy = new_results[0].iloc[:,:5]
```

```python
policy.head()
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>c1</th>
      <th>c2</th>
      <th>r1</th>
      <th>r2</th>
      <th>w1</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.731110</td>
      <td>0.259937</td>
      <td>1.884444</td>
      <td>0.411510</td>
      <td>0.039458</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.268591</td>
      <td>0.334806</td>
      <td>0.468914</td>
      <td>0.460374</td>
      <td>0.208497</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.208356</td>
      <td>0.342701</td>
      <td>0.562405</td>
      <td>0.338575</td>
      <td>0.309206</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0.208209</td>
      <td>0.250303</td>
      <td>0.372215</td>
      <td>0.827863</td>
      <td>0.319882</td>
    </tr>
    <tr>
      <th>4</th>
      <td>-0.012981</td>
      <td>0.335389</td>
      <td>0.557248</td>
      <td>0.411101</td>
      <td>0.181341</td>
    </tr>
  </tbody>
</table>
</div>

```python
policies = []

for i,policy in policy.iterrows():
    policies.append(Policy(str(i),**policy.to_dict()))
```

**Perform 1000 scenarios for each of the policy options. Depending on how many solutions are left after implementing the constraint, consider using multiprocessing or ipyparallel to speed up calculations.**

If you want to use ipyparallel, don't forget to start ipcluster.

```python
%%time
with MultiprocessingEvaluator(lake_m) as evaluator:
    experiments_1, outcomes_1 = evaluator.perform_experiments(scenarios=1000, policies=policies)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] performing 1000 scenarios * 6 policies * 1 model(s) = 6000 experiments
    [MainProcess/INFO] 600 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1800 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 3000 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4200 cases completed
    [MainProcess/INFO] 4800 cases completed
    [MainProcess/INFO] 5400 cases completed
    [MainProcess/INFO] 6000 cases completed
    [MainProcess/INFO] experiments finished
    [MainProcess/INFO] terminating pool

    Wall time: 4min 14s

We can now evaluate the **robustness** of each of the policy options based on these scenario results. We can calculate the robustness of a policy option in terms of its performance on an outcome indicator across the 1000 scenarios. In other words, we can identify how robust a policy is in terms of each outcome indicator, and investigate the robustness tradeoffs.

There are multiple metrics to quantify robustness. On of them is the *signal to noise ratio*, which is simply the mean of a dataset divided by its standard deviation. For instance, for an outcome indicator to be maximized, we prefer a high average value across the scenarios, and a low standard deviation, implying a narrow uncertainty range. Therefore, we want to maximize the signal-to-noise ratio. For an outcome indicator to be minimized, a lower mean and a lower standard deviation is preferred. Therefore the formulation is different.

**Write a function to calculate the signal-to-noise ratio for both kinds of outcome indicators. Calculate the signal-to-noise ratios for each outcome and each policy option. Plot the tradeoffs on a parallel axis plot. Which solutions look like a good compromise policy?**

```python
outcomess = pd.concat([pd.DataFrame(outcomes_1), experiments_1['policy']], axis=1)
```

__Method 1 for calculating signal-to-noise.__

```python
def signaltonoise_max(a, axis=0, ddof=0):
    a = np.asanyarray(a)
    m = a.mean(axis)
    sd = a.std(axis=axis, ddof=ddof)
    return np.where(sd == 0, 0, m/sd)

def signaltonoise_min(a, axis=0, ddof=0):
    a = np.asanyarray(a)
    m = a.mean(axis)
    sd = a.std(axis=axis, ddof=ddof)
    return np.where(sd == 0, 0, m*sd)
```

```python
import numpy as np
policies_stn = outcomess.groupby('policy').agg({'max_P': lambda x: signaltonoise_min(x),
                                                       'utility': lambda x: signaltonoise_max(x),
                                                       'inertia': lambda x: signaltonoise_max(x),
                                                       'reliability': lambda x: signaltonoise_max(x)})
```

__Method 2 for calculating signal-to-noise.__

```python
outcomes_df = pd.DataFrame.from_dict(outcomes_1)
experiments_1[["policy", "scenario"]] = experiments_1[["policy", "scenario"]].astype(int)
```

```python
signal_to_noise = pd.DataFrame()
maximize = ['utility', 'inertia', 'reliability']
minimize= ["max_P"]

for i in range(len(policies)):
    outcomes_ = outcomes_df.iloc[experiments_1[experiments_1.policy == i].index]
    a = pd.DataFrame(outcomes_[maximize].mean()/outcomes_[maximize].std())
    b = pd.DataFrame(outcomes_[minimize].mean()*outcomes_[minimize].std())
    c = pd.concat([a,b])
    signal_to_noise = pd.concat([signal_to_noise,c], axis=1)

signal_to_noise.columns = list(range(len(policies)))
signal_to_noise = signal_to_noise.T
# signal_to_noise
```

```python
signal_to_noise = signal_to_noise[["max_P", "utility", "inertia", "reliability"]]
```

```python
signal_to_noise
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>max_P</th>
      <th>utility</th>
      <th>inertia</th>
      <th>reliability</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>8.042885</td>
      <td>1.180603</td>
      <td>121.425651</td>
      <td>1.789205</td>
    </tr>
    <tr>
      <th>1</th>
      <td>8.046091</td>
      <td>1.366345</td>
      <td>123.834189</td>
      <td>1.730212</td>
    </tr>
    <tr>
      <th>2</th>
      <td>8.159789</td>
      <td>1.487764</td>
      <td>100.668918</td>
      <td>1.653959</td>
    </tr>
    <tr>
      <th>3</th>
      <td>8.068251</td>
      <td>0.986622</td>
      <td>118.922502</td>
      <td>1.839611</td>
    </tr>
    <tr>
      <th>4</th>
      <td>11.285717</td>
      <td>1.629839</td>
      <td>133.638665</td>
      <td>1.162730</td>
    </tr>
    <tr>
      <th>5</th>
      <td>9.142396</td>
      <td>1.723601</td>
      <td>4.108406</td>
      <td>1.369857</td>
    </tr>
  </tbody>
</table>
</div>

```python
policies_stn
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>max_P</th>
      <th>utility</th>
      <th>inertia</th>
      <th>reliability</th>
    </tr>
    <tr>
      <th>policy</th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>8.038862</td>
      <td>1.181193</td>
      <td>121.486409</td>
      <td>1.790100</td>
    </tr>
    <tr>
      <th>1</th>
      <td>8.042067</td>
      <td>1.367029</td>
      <td>123.896153</td>
      <td>1.731078</td>
    </tr>
    <tr>
      <th>2</th>
      <td>8.155708</td>
      <td>1.488509</td>
      <td>100.719290</td>
      <td>1.654787</td>
    </tr>
    <tr>
      <th>3</th>
      <td>8.064216</td>
      <td>0.987116</td>
      <td>118.982007</td>
      <td>1.840531</td>
    </tr>
    <tr>
      <th>4</th>
      <td>11.280073</td>
      <td>1.630654</td>
      <td>133.705535</td>
      <td>1.163312</td>
    </tr>
    <tr>
      <th>5</th>
      <td>9.137824</td>
      <td>1.724464</td>
      <td>4.110462</td>
      <td>1.370542</td>
    </tr>
  </tbody>
</table>
</div>

```python
limits = parcoords.get_limits(signal_to_noise)
axes = parcoords.ParallelAxes(limits)
axes.plot(signal_to_noise)

# we invert this axis so direction of desirability is the same
axes.invert_axis('max_P')
axes.legend()
plt.show()
```

Observations:
- From the signal to noise test we can see that the policies in the yellow and green lines are more optimal. This is because with both policies the signal to noise remains relatively high for utlity, inertia and reliability, and is minimized for pollution.

Another robustness metric is **maximum regret**, calculated again for each policy and for each outcome indicator. *Regret* is defined for each policy under each scenario, as the difference between the performance of the policy in a specific scenario and the berformance of a no-regret (i.e. best possible result in that scenario) policy. The *maximum regret*  is then the maximum of such regret values across all scenarios. We of course favor policy options with low *maximum regret* values.

**Write a function to calculate the maximum regret. Calculate the maximum regret values for each outcome and each policy option. Plot the tradeoffs on a parallel plot. Which solutions look like a good compromise policy?**

```python
policies_max_regret = outcomess.copy()
policies_max_regret['max_P'] = policies_max_regret['max_P'].apply(lambda x: x - min(policies_max_regret['max_P']))
policies_max_regret['utility'] = policies_max_regret['utility'].apply(lambda x: x - max(policies_max_regret['utility']))
policies_max_regret['inertia'] = policies_max_regret['inertia'].apply(lambda x: x - max(policies_max_regret['inertia']))
policies_max_regret['reliability'] = policies_max_regret['reliability'].apply(lambda x: x - max(policies_max_regret['reliability']))

policies_max_regret = policies_max_regret.groupby('policy').agg({'max_P': 'max',
                                                              'utility': 'max',
                                                              'inertia': 'max',
                                                              'reliability': 'max'})
policies_max_regret
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>max_P</th>
      <th>utility</th>
      <th>inertia</th>
      <th>reliability</th>
    </tr>
    <tr>
      <th>policy</th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>11.311056</td>
      <td>-0.184061</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>11.311151</td>
      <td>-0.165531</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>11.311113</td>
      <td>-0.120586</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>11.310864</td>
      <td>-0.230452</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>11.311440</td>
      <td>-0.068528</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>5</th>
      <td>11.311786</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>0.0</td>
    </tr>
  </tbody>
</table>
</div>

```python
limits = parcoords.get_limits(policies_max_regret)
axes = parcoords.ParallelAxes(limits)
axes.plot(policies_max_regret)
axes.legend()

# we invert this axis so direction of desirability is the same
axes.invert_axis('utility')
axes.invert_axis('inertia')
axes.invert_axis('reliability')
axes.invert_axis('max_P')
plt.show()
```

Observations:
- from the calculations of maximum regret it is unclear which polcies are optimal, as maximum regret values do not vary.

We now have an understanding of which solutions have decent robustness using 2 different robustness metrics. A related but different question is to assess the uncertain conditions under which we get poor performance. For this, we can use scenario discovery. Since we want to identify the uncertainties only, we can remove the policy column from the experiments DataFrame.

**Perform Scenario Discovery, focussed on understanding the conditions under which utility is lower than 0.25**

from the trade off curve between coverage, density and number of restricted dimensions, select a point which balances them. Next, using the `yi` attribute, select from the experiments data frame the rows which are within the box as well as the outcomes associated with these experiments.Save these results. They are the starting point for the next assignment. In pseudo code:

```python
from ema_workbench import save_results

selected_experiments = experiments.iloc[box.yi]
selected_outcomes = {k:v[box.yi] for k,v in outcomes.items()}

save_results((selected_experiments, selected_outcomes), './results/selected_results.tar.gz')
```

```python
from ema_workbench.analysis import prim
```

```python
ema_logging.log_to_stderr(ema_logging.INFO)

x = experiments_1.iloc[:, :5]
y = outcomes_df["utility"] > 0.25

prim_alg = prim.Prim(x, y, threshold=0.7, peel_alpha=0.05)
box = prim_alg.find_box()
```

    [MainProcess/INFO] 6000 points remaining, containing 4745 cases of interest
    [MainProcess/INFO] mean: 1.0, mass: 0.121, coverage: 0.1530031612223393, density: 1.0 restricted_dimensions: 3

```python
box.show_tradeoff()
plt.show()
```

```python
selected_experiments = experiments_1.iloc[box.yi]
selected_outcomes = {k:v[box.yi] for k,v in outcomes_df.items()}

directory=os.getcwd()+ "/results/"

if not os.path.exists(directory):
    print(f'Succefully created new directory {directory}')
    os.makedirs(directory)

save_results((selected_experiments, selected_outcomes), directory + 'selected_results.tar.gz')
```

    [MainProcess/INFO] results saved successfully to C:\Users\jryap\Desktop\Model-based Decision-making\epa1361-g21\epa1361_open\Week 5-6 - robustness and direct search\results\selected_results.tar.gz

epa1361_open/Week 5-6 - robustness and direct search/assignment 8 - MORDM.ipynb


# Derivation and Conversion of Planck-Einstein Terms

Start with a generalized Helmholtz energy from
$$\alpha^0 = a_k\log\left[c_k + d_k\exp\left(\frac{b_k\tau}{T_c}\right)\right]$$

The ideal-gas specific heat can be obtained from
$$\frac{c_p^0}{R} = -\tau^2\alpha^0_{\tau\tau}$$

After some math shown below (and some manual grouping):
$$\frac{c_p^0}{R} = - \frac{a_{k} (b_{k}\tau/T_c)^{2} c_k d_{k} e^{\frac{b_{k} \tau}{T_{c}}}}{ \left(c_{k} + d_{k} e^{\frac{b_{k} \tau}{T_{c}}}\right)^2} $$

Two important identities:
$$ \left(\frac{1}{\cosh x}\right) = \frac{2e^{-x}}{1+e^{-2x}}$$
$$ \left(\frac{1}{\sinh x}\right) = \frac{2e^{-x}}{1-e^{-2x}}$$
which yield
$$ \left(\frac{x}{\cosh x}\right)^2 = \frac{4x^2e^{-2x}}{(1+e^{-2x})^2} = \frac{(-2x)^2e^{-2x}}{(1+e^{-2x})^2}$$
$$ \left(\frac{x}{\sinh x}\right)^2 = \frac{4x^2e^{-2x}}{(1-e^{-2x})^2} = \frac{(-2x)^2e^{-2x}}{(1-e^{-2x})^2}$$

```
from __future__ import division
from sympy import *
init_session(quiet=True)
init_printing()
```

    IPython console for SymPy 0.7.5 (Python 2.7.2-32-bit) (ground types: python)

```
a_k,b_k,c_k,d_k,tau,T_c = symbols('a_k,b_k,c_k,d_k,tau,T_c',real=True)
alpha0 = a_k*log(c_k + d_k*exp(b_k*tau/T_c))
print 'No derivatives'
display(alpha0)
print 'First partial w.r.t. tau'
display(diff(alpha0, tau, 1).simplify())
print 'Second partial w.r.t. tau'
display(diff(alpha0, tau, 2).simplify())
print latex(diff(alpha0, tau, 2).simplify()*(-tau**2))
print 'Third partial w.r.t. tau'
display(diff(alpha0, tau, 3).simplify())
```

Aly-Lee starts as
$$\frac{c_p^0}{R_u} = A + B\left(\frac{C/T}{\sinh(C/T)}\right)^2 + D\left(\frac{E/T}{\cosh(E/T)}\right)^2$$
and generalized Plank-Einstein term is given by
$$\frac{c_p^0}{R_u} = - \frac{a_{k} b_{k}^{2} c_{k} d_{k} \tau^{2} e^{\frac{b_{k} \tau}{T_{c}}}}{T_{c}^{2} \left(c_{k} + d_{k} e^{\frac{b_{k} \tau}{T_{c}}}\right)^{2}}$$

Constant is separated out, and handled separately.  sinh part can be expanded as
$$B\left(\frac{C/T}{\sinh(C/T)}\right)^2 = \frac{B(-2C/T)^2\exp(-2C/T)}{(1-\exp(-2C/T))^2}$$
where
$$a_k = B$$
$$b_k = -2C$$
$$c_k = 1$$
$$d_k = -1$$

cosh part can be expanded as
$$D\left(\frac{E/T}{\cosh(E/T)}\right)^2 = \frac{D(-2E/T)^2\exp(-2E/T)}{(1+\exp(-2E/T))^2}$$
where
$$a_k = -D$$
$$b_k = -2E$$
$$c_k = 1$$
$$d_k = 1$$

```
B, D, T = symbols('B, D, T', real = True, positive = True)
# From Aly-Lee:
term1 = ((D/T)/sinh(D/T))**2
# Planck Einstein
term2 = (B/T)**2*exp(B/T)/(exp(B/T)-1)**2
diff1 = term2 - term1
display(diff1.rewrite(exp).subs(B, 2*D).simplify())
term2 = (2*D/T)**2*exp(2*D/T)/(exp(2*D/T)-1)**2
display(term2)
print 'which is the same as the Aly-Lee sinh term with a_k = C, b_k = 2*D, c_k = -1, d_k = 1'

B, F, T = symbols('B, F, T', real = True, positive = True)
# From Aly-Lee:
term1 = ((F/T)/cosh(F/T))**2
# Planck Einstein
term2 = (B/T)**2*exp(-B/T)/(exp(-B/T)+1)**2
diff2 = term2 - term1
display(diff2.rewrite(exp).subs(B, 2*F).simplify())
# So Aly-Lee cosh term can be directly converted to Plank-Einstein2 term with b_k = 2*F, and c=1
```