## Introduction

This notebook will go over optimization of a single response variable using the MRUplift Framework. It will go over:

1. The Business Problem and Data Generating Process

2. Building / Gridsearching an uplift model

3. Evaluating Model with out-of-sample ERUPT metric

4. Assigning Optimal Treatments for new observations

```python
import numpy as np
import pandas as pd

from mr_uplift.dataset.data_simulation import get_simple_uplift_data
from mr_uplift.mr_uplift import MRUplift
from ggplot import *
```

    /Users/samweiss/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters
    Using TensorFlow backend.
    /Users/samweiss/anaconda3/lib/python3.6/site-packages/ggplot/utils.py:81: FutureWarning: pandas.tslib is deprecated and will be removed in a future version.
    You can access Timestamp as pandas.Timestamp
      pd.tslib.Timestamp,
    /Users/samweiss/anaconda3/lib/python3.6/site-packages/ggplot/stats/smoothers.py:4: FutureWarning: The pandas.lib module is deprecated and will be removed in a future version. These are private functions and can be accessed from pandas._libs.lib instead
      from pandas.lib import Timestamp
    /Users/samweiss/anaconda3/lib/python3.6/site-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
      from pandas.core import datetools

### Business Problem

Imagine we are data scientists working for a startup that would like to be more profitibile. As a tactic to increase user activity the company gives all users an expensive bonus (referred to as the treatment). In order to reduce costs we were assigned the task of using data to find a subset of users that should continue receiving the costly treatment.

We are given explanatory variables for users $X$, a random treatment of whether a users recieved the treatment or not $T$, and response variable of profitibility $y$.

We can use uplift models and the IbottaUplift package specifically to find users who should receive the treatment.

### Uplift Problem Setup
The general setup for a lift model is:

$y$: Response variable of interest you’d like to maximize. Here it is profitibility.

$X$: User level covariates. Includes things like previous activity per user.

$T$: The randomly assigned treatment. In this case it is whether or not to give a bonus to a particular user and is binary. Assume that the distribution and assignment of a treatment is uniform and random.

With the data $(y, X, T)$ the goal is to build a treatment assignment policy 𝜋(x) that will use $X$ to assign $T$ that maximizes the value of $y$. Or in this case we want to use user history to assign whether to give a bonus to a user in order to maximize profit.

A frequent practice is to model the expected outcome $y_i$ under different treatments and choose the treatment $T$ that maximizes $y_i$ for each user.

\begin{equation}
 \pi(x_i) =argmax \:_{t \in T} E[y_i | X=x_i, T=t]
\end{equation}

There are several approaches to do this and can be done with a run of the mill ML algorithm that incorporates interactions. IbottaUplift uses a neural network.

To get the counterfactual for each treatment one needs to predict with different values of $t$. This calculation is closely related to to creating an [ICE](https://arxiv.org/pdf/1309.6392.pdf) plot with the treatment variable.

### Data Generating Process

Below is the data generating process of the data we are given.

\begin{equation}
x_1  \sim runif(0,1)
\end{equation}
\begin{equation}
x_2 \sim runif(0,1)
\end{equation}
\begin{equation}
e_1 \sim rnorm(0,1)
\end{equation}
\begin{equation}
e_2 \sim rnorm(0,1)
\end{equation}
\begin{equation}
t \sim rbinom(.5)
\end{equation}

\begin{equation}
revenue = x_1*t + e_1
\end{equation}
\begin{equation}
costs = x_2*t + e_2
\end{equation}
\begin{equation}
profit = revenue - costs
\end{equation}

(In this problem we are interested in only the response variable $profit$)

```python
y, x, t = get_simple_uplift_data(10000)

y = pd.DataFrame(y)
y.columns = ['revenue','cost', 'noise']
y['profit'] = y['revenue'] - y['cost']
```

### Model Building / Gridsearch
After instantiating the MRUplift class the `.fit` function will build the model. It first seperates the data into a train / test split and builds standard scaler transformerd on all variables $x, y, t$.

Then it builds and runs grisdesarch using neural network model that minimizes the mean squared error of the form $y = f(t,x)$. The user can input a custom parameter grid.

```python
uplift_model = MRUplift()
param_grid = dict(num_nodes=[8], dropout=[.1, .5], activation=[
                          'relu'], num_layers=[1, 2], epochs=[25], batch_size=[30])

uplift_model.fit(x, y[['profit']], t.reshape(-1,1), param_grid = param_grid, n_jobs = 1)
```

    /Users/samweiss/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.
      warnings.warn(msg, DataConversionWarning)

### Expected Response Under Proposed Treatments (ERUPT) Metric

After gridsearching we want to know how much better the model is than the current state. Evaluating the out-of-sample importance is key to ensure the model will perform as intended in production. While there are other metrics such as the Qini metric they are usually limited to single treatment case. ERUPT is the only metric I'm aware of that can be applied to multiple treatments and provides an unbiased estimate what would happen if the model were applied.

#### ERUPT
Suppose you have an observation where 𝜋(x) proposes a treatment of not giving bonus and the randomly assigned treatment was given a bonus. Since these do not align it’s not clear we can say anything about it.

However, if the optimal treatment for a model is equal to the assigned treatment we can include that observation in our proposed treatment examples. We go through this exercise for all observations and calculate the response mean for only those where the 𝜋(x) = assigned treatment. This is our estimated value of y under the model! Mathematically it is:

$$\frac{\sum_i y_i I(\pi(x_i) = t_i)} {\sum_i I(\pi(x_i)=t_i)}$$

Note that this formula assumes the treatments distirbution is uniform (same number for each treatment) and randomly assigned. The functionality in this package does not require uniform treatments but does require them to be randomly assigned.

For further information please my blog post [here](https://medium.com/building-ibotta/erupt-expected-response-under-proposed-treatments-ff7dd45c84b4).

### Evaluating Model with out-of-sample ERUPT metric
Using the test dataset MRUplift will then evaluate the model using the ERUPT metric. This functionality gives the model builder insight into whether or not the model performs well out of sample.

It outputs two dataframes:

1) The first dataframe shows the ERUPT metric and standard deviation for the model assignment. In this example it tells us the expected profit if we were to use this model. In addition we can also see a 'random' row under the assignment column. This uses the same distribution as $\pi(x)$ but shuffles the treatments so as to make it a random assignment. Looking at the difference between the Model and Random assignments should tell us if the model is learning the individual treatment effects well.

Below we can see that the model performs much better than the randomized treatments suggesting the model learned the heterogenity of the treatment effects well. If we deployed the model we expect to see profit to be ~ 0.16.

2) The dataframe shows the distribution of treatments under the optimal assignment. In this example we can see about half are assigned the treatment and half are not.

```python
erupt_curves, dists = uplift_model.get_erupt_curves()
erupt_curves
```

    /Users/samweiss/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.
      warnings.warn(msg, DataConversionWarning)

      mean
      std
      response_var_names
      weights
      assignment

      0
      0.161856
      0.004738
      profit
      1
      model

      0
      -0.006752
      0.005352
      profit
      1
      random

```python
dists
```

      num_observations
      tmt
      weights
      percent_tmt

      0
      3646
      0
      1
      0.520857

      1
      3354
      1
      1
      0.479143

### Assigning Optimal Treatments for New Observations
After building and evaluating an uplift model the modeler may deem it worthy of production. To assign new users the optimal treatment one can use the `predict_optimal_treatments` function as shown below.

```python
#generate 5 new observation
_, x_new ,_  = get_simple_uplift_data(5)
uplift_model.predict_optimal_treatments(x_new)
```

    /Users/samweiss/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.
      warnings.warn(msg, DataConversionWarning)

    array([[0],
           [1],
           [0],
           [1],
           [0]])


# メモ

# networkx と matplotlib

スタブのみ。

# サイン・カーブ

```python
# sinカーブ
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
x = np.arange(0,10,0.2)
y = np.sin(x)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x,y)
plt.show

```

```python
# sinカーブ # need kernel restart
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
x = np.arange(0,10,0.2)
y = np.sin(x)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x,y)
plt.show
```

    <IPython.core.display.Javascript object>

    <function matplotlib.pyplot.show>

    <IPython.core.display.Javascript object>

# nbviewer.jupyter.org
githubでは表示されないグラフが表示される
とか
bokehはjavascriptに依存していて、githubはjavascriptは動作しない。
なるほど

#  pandas
git clone https://github.com/practical-jupyter/sample-data.git

```python
import os
base_url = 'https://raw.githubusercontent.com/practical-jupyter/sample-data/master/anime/'
anime_csv = os.path.join(base_url, 'anime.csv')
anime_csv
```

    'https://raw.githubusercontent.com/practical-jupyter/sample-data/master/anime/anime.csv'

```python
import pandas as pd
anime_csv = os.path.join(base_url, 'anime.csv')
pd.read_csv(anime_csv).head()

```

      anime_id
      name
      genre
      type
      episodes
      rating
      members

      0
      32281
      Kimi no Na wa.
      Drama, Romance, School, Supernatural
      Movie
      1
      9.37
      200630

      1
      5114
      Fullmetal Alchemist: Brotherhood
      Action, Adventure, Drama, Fantasy, Magic, Mili...
      TV
      64
      9.26
      793665

      2
      28977
      Gintama°
      Action, Comedy, Historical, Parody, Samurai, S...
      TV
      51
      9.25
      114262

      3
      9253
      Steins;Gate
      Sci-Fi, Thriller
      TV
      24
      9.17
      673572

      4
      9969
      Gintama&#039;
      Action, Comedy, Historical, Parody, Samurai, S...
      TV
      51
      9.16
      151266

```python
# 前処理の結果
anime_master_csv = os.path.join(base_url, 'anime_master.csv')
pd.read_csv(anime_master_csv).head()
```

      anime_id
      name
      genre
      type
      episodes
      rating
      members

      0
      32281
      Kimi no Na wa.
      Drama, Romance, School, Supernatural
      Movie
      1
      9.37
      200630

      1
      5114
      Fullmetal Alchemist: Brotherhood
      Action, Adventure, Drama, Fantasy, Magic, Mili...
      TV
      64
      9.26
      793665

      2
      28977
      Gintama°
      Action, Comedy, Historical, Parody, Samurai, S...
      TV
      51
      9.25
      114262

      3
      9253
      Steins;Gate
      Sci-Fi, Thriller
      TV
      24
      9.17
      673572

      4
      9969
      Gintama'
      Action, Comedy, Historical, Parody, Samurai, S...
      TV
      51
      9.16
      151266

```python
# ジャンル別に加工
anime_split_genre_csv = os.path.join(base_url, 'anime_split_genre.csv')
pd.read_csv(anime_split_genre_csv).head()
```

      anime_id
      name
      genre
      type
      episodes
      rating
      members

      0
      20707
      "0"
      Music
      Music
      1
      5.06
      1170

      1
      25627
      "Aesop" no Ohanashi yori: Ushi to Kaeru, Yokub...
      Kids
      Movie
      1
      5.00
      113

      2
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      Comedy
      OVA
      1
      7.06
      14351

      3
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      School
      OVA
      1
      7.06
      14351

      4
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      Fantasy
      OVA
      1
      7.06
      14351

```python
# メンバ数上位10
anime_genre_top10_csv = os.path.join(base_url, 'anime_genre_top10.csv')
pd.read_csv(anime_genre_top10_csv).head()
```

      anime_id
      name
      genre
      type
      episodes
      rating
      members

      0
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      Comedy
      OVA
      1
      7.06
      14351

      1
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      School
      OVA
      1
      7.06
      14351

      2
      7669
      "Bungaku Shoujo" Kyou no Oyatsu: Hatsukoi
      Fantasy
      OVA
      1
      7.06
      14351

      3
      8481
      "Bungaku Shoujo" Memoire
      School
      OVA
      3
      7.54
      18013

      4
      8481
      "Bungaku Shoujo" Memoire
      Drama
      OVA
      3
      7.54
      18013

```python
# クロス集計
anime_genre_top10_pivoted_csv = os.path.join(base_url, 'anime_genre_top10_pivoted.csv')
pd.read_csv(anime_genre_top10_pivoted_csv).head()

```

      genre
      Movie
      Music
      ONA
      OVA
      Special
      TV

      0
      Comedy
      7293127.0
      20860.0
      1477266.0
      5614758.0
      6659293.0
      65420862.0

      1
      Action
      10224960.0
      77054.0
      524907.0
      5793680.0
      3412689.0
      63364032.0

      2
      Drama
      9034099.0
      100734.0
      188427.0
      3043374.0
      1915578.0
      41011557.0

      3
      Romance
      5245386.0
      42811.0
      411331.0
      3143167.0
      2015820.0
      40703388.0

      4
      Supernatural
      5452779.0
      9189.0
      192989.0
      2696715.0
      2336723.0
      38956520.0

```python
# 株価データ
anime_stock_price_csv = os.path.join(base_url, 'anime_stock_price.csv')
pd.read_csv(anime_stock_price_csv, index_col=0, parse_dates=['Date']).head()
```

      TOEI ANIMATION
      IG Port

      Date

      2015-01-01
      3356.86
      1201.51

      2015-01-02
      3356.86
      1201.51

      2015-01-05
      3396.12
      1218.44

      2015-01-06
      3361.77
      1201.51

      2015-01-07
      3297.97
      1202.51

```python
# 騰落率
anime_stock_returns_csv = os.path.join(base_url, 'anime_stock_returns.csv')
pd.read_csv(anime_stock_returns_csv, index_col=0, parse_dates=['Date']).head()
```

      TOEI ANIMATION
      IG Port

      Date

      2015-01-01
      1.000000
      1.000000

      2015-01-02
      1.000000
      1.000000

      2015-01-05
      1.011695
      1.014082

      2015-01-06
      1.001463
      1.000000

      2015-01-07
      0.982457
      1.000824

```python
# 4本値
t4816_csv = os.path.join(base_url, '4816.csv')
pd.read_csv(t4816_csv, index_col=0, parse_dates=['Date']).head()
```

      Open
      High
      Low
      Close
      Volume

      Date

      2010-01-04
      1600.0
      1600.0
      1580.0
      1597.0
      5600.0

      2010-01-05
      1597.0
      1605.0
      1590.0
      1600.0
      14800.0

      2010-01-06
      1600.0
      1602.0
      1579.0
      1601.0
      8300.0

      2010-01-07
      1600.0
      1600.0
      1590.0
      1595.0
      3700.0

      2010-01-08
      1599.0
      1601.0
      1595.0
      1600.0
      32300.0

# 数列 series

```python
x = symbols('x')
a = x**3
a
```

```python
from sympy import *
degree(a,x)
```


# Introduction to RLlib

RLlib is an open-source library for reinforcement learning that offers both high scalability and a unified API for a variety of applications.
The following introduction was adapted from [rllib_exercises](https://github.com/ray-project/tutorial/blob/master/rllib_exercises/rllib_colab.ipynb).

For more information about RLlib and its open source community:

  - [documentation](https://ray.readthedocs.io/en/latest/rllib.html)
  - [GitHub repo](https://github.com/ray-project/ray/tree/master/rllib#rllib-scalable-reinforcement-learning)
  - [project board](https://github.com/ray-project/ray/projects/6)
  - [Slack sign-up](https://forms.gle/9TSdDYUgxYs8SA9e8)
  - [Twitter](https://twitter.com/raydistributed)

## Install Dependencies

First, install the necessary dependencies before beginning the exercises:

```python
!pip install ray[rllib]
!pip install ray[debug]
!pip install ray[tune]
!pip install pandas
!pip install requests
!pip install tensorflow
```

## RLlib: Markov Decision Processes

**GOAL:** The goal of the exercise is to introduce the Markov Decision Process abstraction and to show its use in Python.

**The key abstraction in reinforcement learning is the Markov Decision Process (MDP).**
An MDP models sequential interactions with an external environment. It consists of the following:

- a **state space**
- a set of **actions**
- a **transition function** which describes the probability of being in a state $s'$ at time $t+1$ given that the MDP was in state $s$ at time $t$ and action $a$ was taken
- a **reward function**, which determines the reward received at time $t$
- a **discount factor** $\gamma$

More details are available [here](https://en.wikipedia.org/wiki/Markov_decision_process).

**NOTE:** Reinforcement learning algorithms are often applied to problems that don't strictly fit into the MDP framework. In particular, situations in which the state of the environment is not fully observed lead to violations of the MDP assumption. Nevertheless, RL algorithms can be applied anyway.

### Policies

A **policy** is a function that takes in a **state** and returns an **action**. A policy may be stochastic (i.e., it may sample from a probability distribution) or it can be deterministic.

The **goal of reinforcement learning** is to learn a **policy** for maximizing the cumulative reward in an MDP. That is, we wish to find a policy $\pi$ which solves the following optimization problem

\begin{equation}
\arg\max_{\pi} \sum_{t=1}^T \gamma^t R_t(\pi),
\end{equation}

where $T$ is the number of steps taken in the MDP (this is a random variable and may depend on $\pi$) and $R_t$ is the reward received at time $t$ (also a random variable which depends on $\pi$).

A number of algorithms are available for solving reinforcement learning problems. Several of the most widely known are [value iteration](https://en.wikipedia.org/wiki/Markov_decision_process#Value_iteration), [policy iteration](https://en.wikipedia.org/wiki/Markov_decision_process#Policy_iteration), and [Q learning](https://en.wikipedia.org/wiki/Q-learning).

### RL in Python

The `gym` Python module provides MDP interfaces to a variety of simulators. For example, the CartPole environment interfaces with a simple simulator which simulates the physics of balancing a pole on a cart. The CartPole problem is described at https://gym.openai.com/envs/CartPole-v0. This example fits into the MDP framework as follows.
- The **state** consists of the position and velocity of the cart as well as the angle and angular velocity of the pole that is balancing on the cart.
- The **actions** are to decrease or increase the cart's velocity by one unit.
- The **transition function** is deterministic and is determined by simulating physical laws.
- The **reward function** is a constant 1 as long as the pole is upright, and 0 once the pole has fallen over. Therefore, maximizing the reward means balancing the pole for as long as possible.
- The **discount factor** in this case can be taken to be 1.

More information about the `gym` Python module is available at https://gym.openai.com/.

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import gym
import numpy as np
```

The code below illustrates how to create and manipulate MDPs in Python. An MDP can be created by calling `gym.make`. Gym environments are identified by names like `CartPole-v0`. A **catalog of built-in environments** can be found at https://gym.openai.com/envs.

```python
env = gym.make('CartPole-v0')
print('Created env:', env)
```

Reset the state of the MDP by calling `env.reset()`. This call returns the initial state of the MDP.

```python
state = env.reset()
print('The starting state is:', state)
```

The `env.step` method takes an action (in the case of the CartPole environment, the appropriate actions are 0 or 1, for moving left or right). It returns a tuple of four things:
1. the new state of the environment
2. a reward
3. a boolean indicating whether the simulation has finished
4. a dictionary of miscellaneous extra information

```python
# Simulate taking an action in the environment. Appropriate actions for
# the CartPole environment are 0 and 1 (for moving left and right).
action = 0
state, reward, done, info = env.step(action)
print(state, reward, done, info)
```

A **rollout** is a simulation of a policy in an environment. It alternates between choosing actions based (using some policy) and taking those actions in the environment.

The code below performs a rollout in a given environment. It takes **random actions** until the simulation has finished and returns the cumulative reward.

```python
def random_rollout(env):
    state = env.reset()

    done = False
    cumulative_reward = 0

    # Keep looping as long as the simulation has not finished.
    while not done:
        # Choose a random action (either 0 or 1).
        action = np.random.choice([0, 1])

        # Take the action in the environment.
        state, reward, done, _ = env.step(action)

        # Update the cumulative reward.
        cumulative_reward += reward

    # Return the cumulative reward.
    return cumulative_reward

reward = random_rollout(env)
print(reward)
reward = random_rollout(env)
print(reward)
```

**EXERCISE:** Finish implementing the `rollout_policy` function below, which should take an environment *and* a policy. The *policy* is a function that takes in a *state* and returns an *action*. The main difference is that instead of choosing a **random action**, the action should be chosen **with the policy** (as a function of the state).

```python
def rollout_policy(env, policy):
    state = env.reset()

    done = False
    cumulative_reward = 0

    # EXERCISE: Fill out this function by copying the 'random_rollout' function
    # and then modifying it to choose the action using the policy.
    raise NotImplementedError

    # Return the cumulative reward.
    return cumulative_reward

def sample_policy1(state):
    return 0 if state[0] < 0 else 1

def sample_policy2(state):
    return 1 if state[0] < 0 else 0

reward1 = np.mean([rollout_policy(env, sample_policy1) for _ in range(100)])
reward2 = np.mean([rollout_policy(env, sample_policy2) for _ in range(100)])

print('The first sample policy got an average reward of {}.'.format(reward1))
print('The second sample policy got an average reward of {}.'.format(reward2))

assert 5 < reward1 < 15, ('Make sure that rollout_policy computes the action '
                          'by applying the policy to the state.')
assert 25 < reward2 < 35, ('Make sure that rollout_policy computes the action '
                           'by applying the policy to the state.')
```

## RLlib: Proximal Policy Optimization

**GOAL:** The goal of this exercise is to demonstrate how to use the proximal policy optimization (PPO) algorithm.

To understand how to use **RLlib**, see the documentation at http://rllib.io.

PPO is described in detail in https://arxiv.org/abs/1707.06347. It is a variant of Trust Region Policy Optimization (TRPO) described in https://arxiv.org/abs/1502.05477

PPO works in two phases. In one phase, a large number of rollouts are performed (in parallel). The rollouts are then aggregated on the driver and a surrogate optimization objective is defined based on those rollouts. We then use SGD to find the policy that maximizes that objective with a penalty term for diverging too much from the current policy.

**NOTE:** The SGD optimization step is best performed in a data-parallel manner over multiple GPUs. This is exposed through the `num_gpus` field of the `config` dictionary (for this to work, you must be using a machine that has GPUs).

```python
# Be sure to install the latest version of RLlib.
! pip install -U ray[rllib]
```

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import gym
import ray
from ray.rllib.agents.ppo import PPOTrainer, DEFAULT_CONFIG
from ray.tune.logger import pretty_print
```

```python
# Start up Ray. This must be done before we instantiate any RL agents.
ray.init(num_cpus=3, ignore_reinit_error=True, log_to_driver=False)
```

Instantiate a PPOTrainer object. We pass in a config object that specifies how the network and training procedure should be configured. Some of the parameters are the following.

- `num_workers` is the number of actors that the agent will create. This determines the degree of parallelism that will be used.
- `num_sgd_iter` is the number of epochs of SGD (passes through the data) that will be used to optimize the PPO surrogate objective at each iteration of PPO.
- `sgd_minibatch_size` is the SGD batch size that will be used to optimize the PPO surrogate objective.
- `model` contains a dictionary of parameters describing the neural net used to parameterize the policy. The `fcnet_hiddens` parameter is a list of the sizes of the hidden layers.

```python
config = DEFAULT_CONFIG.copy()
config['num_workers'] = 1
config['num_sgd_iter'] = 30
config['sgd_minibatch_size'] = 128
config['model']['fcnet_hiddens'] = [100, 100]
config['num_cpus_per_worker'] = 0  # This avoids running out of resources in the notebook environment when this cell is re-executed

agent = PPOTrainer(config, 'CartPole-v0')
```

Train the policy on the `CartPole-v0` environment for 2 steps. The CartPole problem is described at https://gym.openai.com/envs/CartPole-v0.

**EXERCISE:** Inspect how well the policy is doing by looking for the lines that say something like

```
episode_len_mean: 22.262569832402235
episode_reward_mean: 22.262569832402235
```

This indicates how much reward the policy is receiving and how many time steps of the environment the policy ran. The maximum possible reward for this problem is 200. The reward and trajectory length are very close because the agent receives a reward of one for every time step that it survives (however, that is specific to this environment).

```python
for i in range(2):
    result = agent.train()
    print(pretty_print(result))
```

**EXERCISE:** The current network and training configuration are too large and heavy-duty for a simple problem like CartPole. Modify the configuration to use a smaller network and to speed up the optimization of the surrogate objective (fewer SGD iterations and a larger batch size should help).

```python
config = DEFAULT_CONFIG.copy()
config['num_workers'] = 3
config['num_sgd_iter'] = 30
config['sgd_minibatch_size'] = 128
config['model']['fcnet_hiddens'] = [100, 100]
config['num_cpus_per_worker'] = 0

agent = PPOTrainer(config, 'CartPole-v0')
```

**EXERCISE:** Train the agent and try to get a reward of 200. If it's training too slowly you may need to modify the config above to use fewer hidden units, a larger `sgd_minibatch_size`, a smaller `num_sgd_iter`, or a larger `num_workers`.

This should take around 20 or 30 training iterations.

```python
for i in range(2):
    result = agent.train()
    print(pretty_print(result))
```

Checkpoint the current model. The call to `agent.save()` returns the path to the checkpointed model and can be used later to restore the model.

```python
checkpoint_path = agent.save()
print(checkpoint_path)
```

Now let's use the trained policy to make predictions.

**NOTE:** Here we are loading the trained policy in the same process, but in practice, this would often be done in a different process (probably on a different machine).

```python
trained_config = config.copy()

test_agent = PPOTrainer(trained_config, 'CartPole-v0')
test_agent.restore(checkpoint_path)
```

Now use the trained policy to act in an environment. The key line is the call to `test_agent.compute_action(state)` which uses the trained policy to choose an action.

**EXERCISE:** Verify that the reward received roughly matches up with the reward printed in the training logs.

```python
env = gym.make('CartPole-v0')
state = env.reset()
done = False
cumulative_reward = 0

while not done:
    action = test_agent.compute_action(state)
    state, reward, done, _ = env.step(action)
    cumulative_reward += reward

print(cumulative_reward)
```


```python
# Author: Ciarán O'Brien & Chloe Doyle
# Lecture: Svetlana Hensman
# Submitted: 07/12/18
# This file is in response to the second assignment as set out per the classification goal

```

```python
# Boiler plate imports
import matplotlib.pyplot as plt
import seaborn as sn
import numpy as np
import pandas as pd
import re
import time
import cardinality
import statistics
from math import floor
from statistics import mean, median
from collections import Counter
from sympy import pretty_print as pp,latex
import seaborn as sn
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.model_selection import train_test_split

from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn import tree
from sklearn.neural_network import MLPClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.gaussian_process.kernels import RBF
from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import GaussianNB
```

```python
data = pd.read_csv("data/trainingset.txt",sep='\,',header=None, encoding='utf-8')
```

    c:\users\ciaran\appdata\local\programs\python\python35\lib\site-packages\ipykernel_launcher.py:1: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
      """Entry point for launching an IPython kernel.

```python
data.head()
```

      id
      age
      job
      marital
      education
      default
      balance
      housing
      loan
      contact
      day
      month
      duration
      campaign
      pdays
      previous
      poutcome
      y

      0
      TR1
      44
      "JobCat9"
      "single"
      "secondary"
      "no"
      29
      "yes"
      "no"
      "unknown"
      5
      "may"
      0
      1
      -1
      0
      "unknown"
      "TypeA"

      1
      TR2
      31
      "JobCat4"
      "married"
      "secondary"
      "no"
      2
      "yes"
      "yes"
      "unknown"
      5
      "may"
      0
      1
      -1
      0
      "unknown"
      "TypeA"

      2
      TR3
      42
      "JobCat4"
      "divorced"
      "tertiary"
      "yes"
      2
      "yes"
      "no"
      "unknown"
      5
      "may"
      0
      1
      -1
      0
      "unknown"
      "TypeA"

      3
      TR4
      58
      "JobCat2"
      "married"
      "primary"
      "no"
      121
      "yes"
      "no"
      "unknown"
      5
      "may"
      0
      1
      -1
      0
      "unknown"
      "TypeA"

      4
      TR5
      43
      "JobCat9"
      "single"
      "secondary"
      "no"
      593
      "yes"
      "no"
      "unknown"
      5
      "may"
      0
      1
      -1
      0
      "unknown"
      "TypeA"

```python
continuous_data = data.select_dtypes(include = ['int64','float64'])
categorical_data = data.select_dtypes(include = ['object'])
```

```python
categorical_data.drop('id', axis=1, inplace=True)
categorical_data.head()
```

    c:\users\ciaran\appdata\local\programs\python\python35\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
      """Entry point for launching an IPython kernel.

      job
      marital
      education
      default
      housing
      loan
      contact
      month
      poutcome
      y

      0
      "JobCat9"
      "single"
      "secondary"
      "no"
      "yes"
      "no"
      "unknown"
      "may"
      "unknown"
      "TypeA"

      1
      "JobCat4"
      "married"
      "secondary"
      "no"
      "yes"
      "yes"
      "unknown"
      "may"
      "unknown"
      "TypeA"

      2
      "JobCat4"
      "divorced"
      "tertiary"
      "yes"
      "yes"
      "no"
      "unknown"
      "may"
      "unknown"
      "TypeA"

      3
      "JobCat2"
      "married"
      "primary"
      "no"
      "yes"
      "no"
      "unknown"
      "may"
      "unknown"
      "TypeA"

      4
      "JobCat9"
      "single"
      "secondary"
      "no"
      "yes"
      "no"
      "unknown"
      "may"
      "unknown"
      "TypeA"

# Data exploration

Viewing all the unique values from each coloumn

## Results

Column 12 only contain 1 unique value, it's no use to us.
Columns 5,7,8 are only binary

```python
for col in data.columns.values:
    print(col, data[col].unique())
```

    0 ['TR1' 'TR2' 'TR3' ... 'TR24316' 'TR24317' 'TR24318']
    1 [44 31 42 58 43 57 51 45 40 37 52 46 34 49 50 60 54 30 53 33 55 47 59 41
     27 56 26 48 24 32 29 36 28 35 22 21 23 25 61 20 19 18 66 83 70 68 65 64
     69 62 75 71 67 76 85 63 90 73 78 80 94 72 17 74 86 79 95 82 81 77 16 84
     87 93 88]
    2 ['"JobCat9"' '"JobCat4"' '"JobCat2"' '"JobCat7"' '"JobCat11"' '"JobCat6"'
     '"JobCat3"' '"JobCat8"' '"JobCat10"' '"JobCat1"' '"JobCat5"' '"unknown"']
    3 ['"single"' '"married"' '"divorced"']
    4 ['"secondary"' '"tertiary"' '"primary"' '"unknown"']
    5 ['"no"' '"yes"']
    6 [   29     2   121 ... 16353  5083  4655]
    7 ['"yes"' '"no"']
    8 ['"no"' '"yes"']
    9 ['"unknown"' '"cellular"' '"telephone"']
    10 [ 5  6  7  8  9 12 13 14 15 16 19 20 21 23 26 27 28 29 30  2  3  4 11 17
     18 24 25  1 10 22 31]
    11 ['"may"' '"jun"' '"jul"' '"aug"' '"oct"' '"nov"' '"dec"' '"jan"' '"feb"'
     '"mar"' '"apr"' '"sep"']
    12 [0]
    13 [ 1  2  3  5  4  6  8  7  9 10 13 11 12 14 32 18 22 15 17 25 21 19 63 26
     28 16 50 38 23 24 37 27 29 30 41 20 31 33 35 34 39 36]
    14 [ -1 151  86 147 176 174 170 195 188 196 172 118 119 171 131 123 159 186
     111 115 116 173 166 164 110  96 103 150 175 104 193 181 185 154 138 145
     132 126 180 129 101 167 117 109  97 130 168 125 105 102  26 179 182  28
     183 165 127 112 124 187 190 113 162 152 134 169 189   8 120 144 191 184
     177  99 133  93  92 155  91 100 156 106 198 128 153 160 107  90 197 136
     139 157 122 178 135  30  98 163 121  10 141 158 192  31  94 199 137 108
     268 247 253 226 244 245 231 258 223 265 246 250 240 204 205 254 266 259
     241 261 239 251 225 161 237 248 255 262 227 234 224 206 238 260   2 270
     232 252 228 235 256   5 273 242 214 249 208 272 271 207 202 269 201 229
     210 264 216 217 230  76 200 263  73 203 221 267 222  77  82   6 209 257
     274 243   7 212 215 233 213 276   9 275 211   1 279  12 280  88 194 277
     236  84  85 219  41 294 329 307 331  64 314 323 332 333 326 335 313 312
     325 327 328 140 330 316 310 289  57 295 336 339 301 315 337 300 143 334
     340 319 146  17  74 322 148 299 344 320 149 321 342 324 318 317 346 345
     282 281 343 305 338  14 303  15 347 304 302 341 348 306 349 285 287 350
      25 278  81  87  79  70  13  83  37  80  63  22 355  19   4  89  35 351
     362 365 309 358 366 363 356 364 288 357 293 360 297 359 367 353 352 296
     361 368 290 283 298 308  66 371 370 284 369 286 292 354  50 373 374 372
     291 311  59  95  40  29  43 114  20  69  60  56  55  78 391  34  27  44
      67  32 393  65 395 388 399  49 389 412 385 434 394 440  68 461 462 422
     430 442 403 457 459 454 379 428 392 410 475 477 478  54 474 463 479  45
      46 142 495  58  48 518  75 378 218  33 544 435 433 436 558 469 616 561
     553 555 585 480 667 626 449 633 426  61  53 460  71 670 551 404 651 686
     384 425 376 504 578 674 386 450 745 514 417 424 776 381 396 439 415 456
     458 481 791 531 792 413 535 784 455 491 431 446 472 437 782 414 828 524
     562 761 492 775 493 760 432 655 427 749 779 842 484 489  62 409 444 485
     503 772 526 528 826 508 547 541  38 550]
    15 [ 0  3  4  2  1  6  5 10  7  8 14 11 15  9 12 37 16 27 17 38 18 24 51 23
     20 13 29 19 21 30 22 58 25 28 26 40 55]
    16 ['"unknown"' '"failure"' '"other"' '"success"']
    17 ['"TypeA"' '"TypeB"']

```python

```

```python

```

# TODO
Look at removing all the "" from the values entries

## Optional
set the columns from a regex apllied to the text file provided by Svetlana, datadescription.txt

```python

# with open('data/datadescription.txt', 'r') as myfile:
#     filedata=myfile.read().replace('\n', '')
```

```python
# expression = re.compile("((100)|[1-9]\d? - \b[^\d\W]+\b)")
# regx = '(100)|[1-9]\d? - \b[^\d\W]+\b'

# match = re.findall(regx,filedata)
# print(match)
#     #print(re.search(expression,str(dataFeatures[rows])))
```

```python
data.columns = ["id","age","job","marital","education","default","balance","housing","loan","contact","day","month","duration","campaign","pdays","previous","poutcome","y"]
```

```python

```

```python
def label_encode(df, columns):
    for col in columns:
        le = LabelEncoder()
        col_values_unique = list(df[col].unique())
        le_fitted = le.fit(col_values_unique)

        col_values = list(df[col].values)
        le.classes_
        col_values_transformed = le.transform(col_values)
        df[col] = col_values_transformed

# Encoding a unique label to each catergorical data entry
df_categorical_labled = categorical_data.copy(deep=True)
to_be_encoded_cols = df_categorical_labled.columns.values
label_encode(df_categorical_labled, to_be_encoded_cols)
display(df_categorical_labled.head())

```

      job
      marital
      education
      default
      housing
      loan
      contact
      month
      poutcome
      y

      0
      10
      2
      1
      0
      1
      0
      2
      8
      3
      0

      1
      5
      1
      1
      0
      1
      1
      2
      8
      3
      0

      2
      5
      0
      2
      1
      1
      0
      2
      8
      3
      0

      3
      3
      1
      0
      0
      1
      0
      2
      8
      3
      0

      4
      10
      2
      1
      0
      1
      0
      2
      8
      3
      0

```python
# Heatmap of the coeffecients
# Shows how dependant each column is on each other
# Needs the encoding above to work
my_cmap = sn.light_palette("Navy", as_cmap=True)
corrMatt = df_categorical_labled.corr()
mask = np.array(corrMatt)
mask[np.tril_indices_from(mask)] = False
fig,ax= plt.subplots()
fig.set_size_inches(20,10)
sn.heatmap(corrMatt, mask=mask,vmax=.8, square=True,annot=True,cmap=my_cmap)

```

```python
# Heatmap of the coeffecients
# Shows how dependant each column is on each other
# Needs the encoding above to work
my_cmap = sn.light_palette("Navy", as_cmap=True)
corrMatt = data.corr()
mask = np.array(corrMatt)
mask[np.tril_indices_from(mask)] = False
fig,ax= plt.subplots()
fig.set_size_inches(20,10)
sn.heatmap(corrMatt, mask=mask,vmax=.8, square=True,annot=True,cmap=my_cmap)

```

# Results

Further away from 1, the higher the correlation

```python
#Here we'll split the data into our training and validation sets
#X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
```

# Classification
She said we can build our classifier from sklearn librayr, they have a load of classifiers compared [here.](http://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html)

Each classifier take various different values, each are outlined in the documentation

```python
def get_train_test(df, y_col, x_cols, ratio):
    """
    This method transforms a dataframe into a train and test set, for this you need to specify:
    1. the ratio train : test (usually 0.7)
    2. the column with the Y_values
    """
    mask = np.random.rand(len(df)) < ratio
    df_train = df[mask]
    df_test = df[~mask]

    Y_train = df_train[y_col].values
    Y_test = df_test[y_col].values
    X_train = df_train[x_cols].values
    X_test = df_test[x_cols].values
    return df_train, df_test, X_train, Y_train, X_test, Y_test
```

```python
dict_classifiers = {
    "Logistic Regression": LogisticRegression(),
    "Nearest Neighbors": KNeighborsClassifier(),
    "Linear SVM": SVC(),
#     "Gradient Boosting Classifier": GradientBoostingClassifier(n_estimators=1000),
    "Decision Tree": tree.DecisionTreeClassifier(),
#     "Random Forest": RandomForestClassifier(n_estimators=1000),
#     "Neural Net": MLPClassifier(alpha = 1),
    "Naive Bayes": GaussianNB()
}
```

```python
def batch_classify(X_train, Y_train, X_test, Y_test, no_classifiers = 5, verbose = True):
    """
    This method, takes as input the X, Y matrices of the Train and Test set.
    And fits them on all of the Classifiers specified in the dict_classifier.
    The trained models, and accuracies are saved in a dictionary. The reason to use a dictionary
    is because it is very easy to save the whole dictionary with the pickle module.

    Usually, the SVM, Random Forest and Gradient Boosting Classifier take quiet some time to train.
    So it is best to train them on a smaller dataset first and
    decide whether you want to comment them out or not based on the test accuracy score.
    """

    dict_models = {}
    for classifier_name, classifier in list(dict_classifiers.items())[:no_classifiers]:
        t_start = time.clock()
        classifier.fit(X_train, Y_train)
        t_end = time.clock()

        t_diff = t_end - t_start
        train_score = classifier.score(X_train, Y_train)
        test_score = classifier.score(X_test, Y_test)

        dict_models[classifier_name] = {'model': classifier, 'train_score': train_score, 'test_score': test_score, 'train_time': t_diff}
        if verbose:
            print("trained {c} in {f:.2f} s".format(c=classifier_name, f=t_diff))
    return dict_models
```

```python
def display_dict_models(dict_models, sort_by='test_score'):
    cls = [key for key in dict_models.keys()]
    test_s = [dict_models[key]['test_score'] for key in cls]
    training_s = [dict_models[key]['train_score'] for key in cls]
    training_t = [dict_models[key]['train_time'] for key in cls]

```

```python
y_col_bank = 'job'
x_cols_glass = list(df_categorical_labled.columns.values)
train_test_ratio = 0.7
df_train, df_test, X_train, Y_train, X_test, Y_test = get_train_test(df_categorical_labled, y_col_bank, x_cols_glass, train_test_ratio)

dict_models = batch_classify(X_train, Y_train, X_test, Y_test, no_classifiers = 8)
display_dict_models(dict_models)
```

    trained Logistic Regression in 1.98 s
    trained Nearest Neighbors in 0.09 s
    trained Decision Tree in 0.02 s
    trained Linear SVM in 3.73 s
    trained Naive Bayes in 0.06 s

```python

```

ML CA2 - Classifier - C14385336- C15765215.ipynb


```python
#we may need some code in the ../python directory and/or matplotlib styles
import sys
import os
sys.path.append('../python/')

#set up matplotlib
os.environ['MPLCONFIGDIR'] = '../mplstyles'
print(os.environ['MPLCONFIGDIR'])
import matplotlib as mpl
from matplotlib import pyplot as plt
#got smarter about the mpl config: see mplstyles/ directory
plt.style.use('standard')
print(mpl.__version__)
print(mpl.get_configdir())

#fonts
# Set the font dictionaries (for plot title and axis titles)
title_font = {'fontname':'Arial', 'size':'16', 'color':'black', 'weight':'normal',
              'verticalalignment':'bottom'} # Bottom vertical alignment for more space
axis_font = {'fontname':'Arial', 'size':'32'}
legend_font = {'fontname':'Arial', 'size':'22'}

#fonts global settings
mpl.rc('font',family=legend_font['fontname'])

#set up numpy
import numpy as np
```

    ../mplstyles
    3.0.3
    /home/phys/villaa/analysis/misc/nrFano_paper2019/mplstyles

# ER/NR Band Calculation

In a previous notebook `QEr_2D_joint.ipynb` we calculated the correct two-dimensional probability distribution for energy recoils of arbitrary average ionization yield, given the resolutions ($\sigma_H$,$\sigma_I$, and $\sigma_N$) as functions of the true recoil energy. The variance on the number of e/h pairs created, $\sigma_N$ included the effective Fano factor, $F$.

This was all done starting from the fundamental variables $\delta H$, $\delta I$, and N (see `QEr_2D_joint.ipynb` for all variable definitions). The final result was the probability distribution in the Q,$\tilde{E}_r$ plane of measured yield and recoil energy, given a true recoil distribution $P(E_r)$. $P(E_r)$ was taken to be:

\begin{equation}
P(E_r) = \frac{1}{\alpha}e^{-\alpha E_r},
\end{equation}

where $\alpha$ is the decay constant and we take to be equal to about 1/100 keV$^{-1}$ for nuclear recoils.

In this note we will use this model to construct the variance of constant-$\tilde{E}_r$ slices of this distribution. The motivation for this calculation is that these variances are often the ones measured in experiments like Edelweiss [REF] and SuperCDMS.

## Ionization Yield Distributions at Fixed $\tilde{E}_r$

First, we want to construct bands for electron recoils with the parameters $F^{\prime}$=0 (Fano factor) and $\alpha^{\prime}$ = 1/100000 keV$^{-1}$ (essentially flat true recoil distribution).

**NOTE: The value of $F^{\prime}$ is choosen as such because the measured resolutions are done with electron recoils and implicitly include the true Fano factor. Although this Fano factor (typically in germanium of order 0.13 with large variability in the measurements [REF]) is included note that it is not included correctly because it is not separated from the variance of $\delta I$. Nevertheless, since the Fano contribution for electron recoils is small compared to the ionization resolutions in most published work, it is safely parameterized inside the $\delta I$ resolution for electron recoils.**

Next we will construct a nuclear recoil band with the parameters $F$=15 (this appears to be a plausible lower limit for measurements above 10 keV, see [REF] Dougherty, and the notebook constructing the Fano factor from that) and $\alpha$=1/100 keV$^{-1}$.

We are constructing the distribution that results from the following:

\begin{equation}
P(Q,\tilde{E}_r=E) = \int_0^{\infty} dE_r P(Q,\tilde{E}_r=E|E_r)P(E_r)
\end{equation}

the following two cells set all the parameters and get the correct resolutions. I am using detector GGA1 here to compare with because in Fig. 5 of the Edelweiss paper [REF] those bands are actually plotted and we have digitized versions of the 90% (1.645$\sigma$) containment bands.

```python
#constants
V=4.0 #volts
eps = 3.0/1000 #keV per pair, I usually use 3.3 for the numerator, but Edw. uses 3.
FWHM_to_SIG = 1 / (2*np.sqrt(2*np.log(2)))

#yield models
a=0.16
b=0.18
Qbar = lambda Er: a*Er**b
Qer = lambda Er: 1
```

```python
#getting the resolutions
import EdwRes as er

#these happen to be the parameters for GGA1 detector (Edelweiss)
aH=0.02
fh2 = er.get_heatRes_func(1.3, 3.5,aH*FWHM_to_SIG)
sigH = lambda x:fh2(x)

fi2 = er.get_ionRes_func(1.3, 1.3, 2.8)
sigI = lambda x:fi2(x)

#new resolution functions
Ehee = lambda Er: ((1+(V/(1000*eps))*Qbar(Er))*Er)/(1+(V/(1000*eps)))
EIee = lambda Er: Qbar(Er)*Er

sigH_NR = lambda Er: sigH(Ehee(Er))

sigI_NR = lambda Er: sigI(EIee(Er))

#check sigI @ 122keV
print('FWHM @ 122 keV: {}'.format(2.355*sigI(122)))
```

    FWHM @ 122 keV: 2.8002139755495987

In the next two cells, I set up the proability distributions for the afromentioned parameters with the functions developed as a result of the `QEr_2D.ipynb` notebook.

As a test energy I choose 10 keV because the distributions begin to look wide and clearly non-Gaussian there. At that energy we take a slice of the $P(Q,\tilde{E}_r)$ distribution, and calculate the integral so that we can properly normalize it. This is the ionization yield distribution that would be observed at a single measured recoil energy, $\tilde{E}_r$.

```python
import prob_dist as pd

#set up ER distribution
Fp = 0.0001
alphap=1/100000
Per = pd.QEr_v2_2D_fast(sigH,sigI,V,eps,Fp,Qer)
er_band = pd.expband_2D(Per,alphap,3)

#set up NR distribution
F = 15
alpha=1/100.0
Pnr = pd.QEr_v2_2D_fast(sigH_NR,sigI_NR,V,eps,F,Qbar)
nr_band = pd.expband_2D(Pnr,alpha,1.5)
```

```python
#test the dists at a certain energy
Etest=10

from scipy import integrate
ernorm = integrate.quad(er_band,0,4,args=(Etest,))[0]
nrnorm = integrate.quad(nr_band,0,4,args=(Etest,))[0]
print(ernorm)
print(nrnorm)

X=np.arange(0,1.5,0.01)
Qdist_er = lambda Q: (1/ernorm)*er_band(Q,Etest)
Qdist_nr = lambda Q: (1/nrnorm)*nr_band(Q,Etest)
Qerv = np.vectorize(Qdist_er)
Qnrv = np.vectorize(Qdist_nr)
Yer = Qerv(X)
Ynr = Qnrv(X)
```

    100038.55775424647
    88.38426954941066

The next two cells show the ER and NR distributions for the above parameters on a logarithmic and linear vertical scale. It can be seen on the logarithmic plots that the distributions are not Gaussian.

```python
#set up a 1d plot
fig,axes = plt.subplots(1,1,figsize=(9.0,8.0),sharex=True)
ax1 = axes

ax1.plot(X,Yer,color='orange',linestyle='-',label='{} keV ER band'.format(Etest))
ax1.plot(X,Ynr,color='steelblue',linestyle='-',label='{} keV NR band'.format(Etest))

ax1.axvline(Qbar(Etest), color='r', linestyle='--', lw=2, alpha=0.8,label='predicted centers')
ax1.axvline(1, color='r', linestyle='--', lw=2, alpha=0.8)

ymin = 1e-4
max1 = np.max(Yer)
max2 = np.max(Ynr)
ymax = 1.01*np.max(np.asarray([max1,max2]))

ax1.set_yscale('linear')
ax1.set_yscale('log')
ax1.set_xlim(0, 1.5)
ax1.set_ylim(ymin,ymax)
ax1.set_xlabel(r'ionization yield',**axis_font)
ax1.set_ylabel('PDF',**axis_font)
ax1.grid(True)
#ax1.yaxis.grid(True,which='minor',linestyle='--')
ax1.legend(loc=4,prop={'size':22})

for axis in ['top','bottom','left','right']:
  ax1.spines[axis].set_linewidth(2)

plt.tight_layout()
#plt.savefig('figures/figure.png')
plt.show()
```

```python
#set up a 1d plot
fig,axes = plt.subplots(1,1,figsize=(9.0,8.0),sharex=True)
ax1 = axes

ax1.plot(X,Yer,color='orange',linestyle='-',label='{} keV ER band'.format(Etest))
ax1.plot(X,Ynr,color='steelblue',linestyle='-',label='{} keV NR band'.format(Etest))

ax1.axvline(Qbar(Etest), color='r', linestyle='--', lw=2, alpha=0.8,label='predicted centers')
ax1.axvline(1, color='r', linestyle='--', lw=2, alpha=0.8)

ymin = 1e-4
max1 = np.max(Yer)
max2 = np.max(Ynr)
ymax = 1.01*np.max(np.asarray([max1,max2]))

ax1.set_yscale('linear')
ax1.set_yscale('linear')
ax1.set_xlim(0, 1.5)
ax1.set_ylim(ymin,ymax)
ax1.set_xlabel(r'ionization yield',**axis_font)
ax1.set_ylabel('PDF',**axis_font)
ax1.grid(True)
#ax1.yaxis.grid(True,which='minor',linestyle='--')
ax1.legend(loc=1,prop={'size':22})

for axis in ['top','bottom','left','right']:
  ax1.spines[axis].set_linewidth(2)

plt.tight_layout()
#plt.savefig('figures/figure.png')
plt.show()
```

## The 68.27% Symmetric Containment Region

Since these distributions are not quite Gaussian for all combinations of parameters, we must approximate the widths, $\sigma_Q(\tilde{E}_r)$. We define these to be the symmetric regions about the expected yield centers (note, it may be true that in some cases the distribution can be skewed so that neither the mean nor the mode are at the expected yield centers) that contain 68.27% of the distribution.

For this we can use the optimization libraries in `scipy.optimize`. We define a numerical function that will be zero when the size of the aformentioned region is supplied as an argument.

```python
import scipy.optimize as so

Qint_er = lambda a: integrate.quad(Qerv,Qer(Etest)-a,Qer(Etest)+a,limit=100)[0]
Qint_nr = lambda a: integrate.quad(Qnrv,Qbar(Etest)-a,Qbar(Etest)+a,limit=100)[0]

siger_zero = lambda a: Qint_er(a) - 0.6827 #one sigma
signr_zero = lambda a: Qint_nr(a) - 0.6827 #one sigma

siger_90 = lambda a: Qint_er(a) - 0.9 #90%
signr_90 = lambda a: Qint_nr(a) - 0.9 #90%

sigQer = so.brentq(siger_zero,0,1,rtol=0.001,maxiter=100)
print(sigQer)
sigQnr = so.brentq(signr_zero,0,1,rtol=0.001,maxiter=100)
print(sigQnr)

sigQer90 = so.brentq(siger_90,0,1,rtol=0.001,maxiter=100)
print('1.645xone sigma: {}; 90\%:{}'.format(1.645*sigQer,sigQer90))
sigQnr90 = so.brentq(signr_90,0,1,rtol=0.001,maxiter=100)
print('1.645xone sigma: {}; 90\%:{}'.format(1.645*sigQnr,sigQnr90))
```

    0.22478180172240245
    0.11474735445072594
    1.645xone sigma: 0.36976606383335203; 90\%:0.3698908536745943
    1.645xone sigma: 0.18875939807144418; 90\%:0.1840166980484885

```python
import time
start = time.time()
#root = pd.sigrootEdw(10,10,V,eps,(1/100),Qbar)
root = pd.sigrootEdw(Fp,7,V,eps,alphap,Qer)
end = time.time()
print('sigma: {}'.format(root))
print('{} s'.format(end - start))
```

    sigma: 0.3220219209327321
    9.04426646232605 s

```python
import fano_calc as fc

(nrsigma,nrE) = fc.calcQWidth(10,F,V,eps,alpha,Qbar,aH)
(ersigma,erE) = fc.calcQWidth(10,Fp,V,eps,alphap,Qer,aH)
```

In the Edelweiss paper [REF], a Gaussian approximation was used wherein the width in ionization yield could be written as:

\begin{equation}
\sigma_Q(\tilde{E}_r) \simeq \frac{1}{\tilde{E}_r} \sqrt{\left(1+\frac{V}{\epsilon}\bar{Q}\right)^2\sigma_I^2 + \left(1+\frac{V}{\epsilon}\right)^2\bar{Q}^2\sigma_H^2}.
\end{equation}

It is therefore useful to compare this function to the values obtained from the more fundamental derivation that we are seeking.

```python
#make functions for analytical bands
sigQer = lambda Etr: (1/Etr)*np.sqrt((1+(V/(1000*eps))*Qer(Etr))**2*sigI(Etr)**2 + (1+(V/(1000*eps)))**2*Qer(Etr)**2 \
                                     *sigH(Etr)**2)

sigQnr = lambda Etr: (1/Etr)*np.sqrt((1+(V/(1000*eps))*Qbar(Etr))**2*sigI_NR(Etr)**2 + (1+(V/(1000*eps)))**2*Qbar(Etr)**2 \
                                     *sigH_NR(Etr)**2)

print(sigQer(10))
print(sigQnr(10))
sigQerv = np.vectorize(sigQer)
sigQnrv = np.vectorize(sigQnr)
```

    0.2246462352359262
    0.10795032751310846

## Comparing Modeled and Edelweiss Published Bands

Since we can now calculate the 68.27% containments at a given recoil energy, we can find what they are for all recoil energy and compare to the bands published in the Edelweiss paper [REF]. We compare for detector GGA1, because that is the detector that the Edw. paper shows the bands from.

```python
import pandas as pd
band_data = pd.read_csv("data/edelweiss_bands_GGA1.csv")
print (band_data.head(10))
```

           x   Curve1   curve2   curve3   curve4
    0  1.667 -6.44581 -0.08957  1.49300 -0.64478
    1  1.802 -6.06069 -0.08789  1.44296 -0.62392
    2  1.804 -6.05415 -0.08786  1.46798 -0.62356
    3  1.938 -5.68168 -0.08613  1.41795 -0.60321
    4  2.075 -5.31531 -0.08430  1.39293 -0.58303
    5  2.212 -4.96139 -0.08240  1.36791 -0.56335
    6  2.883 -3.39678 -0.07177  1.11174 -0.47381
    7  3.158 -2.83384 -0.06656  1.10052 -0.44031
    8  3.295 -2.57110 -0.06372  1.06171 -0.42438
    9  4.518 -0.66571 -0.01776  0.75550 -0.30031

```python
# path 0: 68.75 keV inelastic scattering
# path 1: average NR line
# path 2: 13.26 keV inelastic
# path 3: ionization threshold
# path 4: lower nuclear recoil band
# path 5: upper nuclear recoil band
# path 6: upper and lower electron recoil band
import pandas as pd
band_data_svg = pd.read_csv("data/edelweiss_band_GGA1_allCurveData.txt", skiprows=1, header=None, delim_whitespace=True)
print (band_data_svg.head(5))
list(band_data_svg.columns.values)
```

            0         1        2         3        4         5        6        7   \
    0  69.4986  0.991839  1.02904  0.159922  14.2519  0.944224  2.10350  1.49896
    1  69.8626  0.987244  1.31515  0.167136  14.3933  0.936084  2.11548  1.48548
    2  70.2266  0.982648  1.60126  0.174349  14.5347  0.927944  2.12746  1.47200
    3  70.5907  0.978053  2.05730  0.181087  14.6761  0.919803  2.13943  1.45851
    4  70.9944  0.973552  2.45114  0.187750  14.8725  0.911724  2.15141  1.44503

            8         9        10       11        12       13
    0  11.0197 -0.002028  1.89725  1.50072   9.99176  1.50730
    1  11.3235  0.005082  1.92804  1.48860  10.39770  1.48881
    2  11.6309  0.012183  1.95883  1.47648  10.78450  1.47030
    3  11.9934  0.019148  1.98962  1.46436  11.35520  1.44694
    4  12.3254  0.026192  2.02041  1.45224  12.01670  1.42316

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]

```python
#set up a 1d plot
fig,axes = plt.subplots(1,1,figsize=(9.0,8.0),sharex=True)
ax1 = axes

#X = np.arange(0.1,100,0.1)

erQ = Qer(erE)
nrQ = Qbar(nrE)

# plot the data-theifed bands
#ax1.plot(band_data.x[band_data.x>7.82], band_data.Curve1[band_data.x>7.82], 'b-',label='')
#ax1.plot(band_data.x, band_data.curve2, 'b-',label='')
#ax1.plot(band_data.x, band_data.curve3, 'r-',label='')
#ax1.plot(band_data.x, band_data.curve4, 'r-',label='')
ax1.plot(nrE,nrQ+1.645*nrsigma,'r-',linewidth=2,label='NR band model (F={:2.1f}; a$_H$={})'.format(F,aH))
ax1.plot(erE,erQ+1.645*ersigma,'b-',linewidth=2,label='ER band model (F={:2.1f}; a$_H$={})'.format(Fp,aH))
ax1.plot(nrE,nrQ-1.645*nrsigma,'r-',linewidth=2)
ax1.plot(erE,erQ-1.645*ersigma,'b-',linewidth=2)

ax1.plot(nrE,nrQ+1.645*sigQnrv(nrE),color='orange',linestyle='--',linewidth=3,label='NR model Edw. (a$_H$={})'.format(aH))
ax1.plot(erE,erQ+1.645*sigQerv(erE),color='steelblue',linestyle='--',linewidth=3,label='ER model Edw. (a$_H$={})'.format(aH))
ax1.plot(nrE,nrQ-1.645*sigQnrv(nrE),color='orange',linestyle='--',linewidth=3)
ax1.plot(erE,erQ-1.645*sigQerv(erE),color='steelblue',linestyle='--',linewidth=3)

#svg-extracted
# plot the svg? electron-recoil bands
ax1.plot(band_data_svg[12][band_data_svg[13]>1], band_data_svg[13][band_data_svg[13]>1], 'b:', linewidth=2, \
        label='Edw. paper ER')
ax1.plot(band_data_svg[12][band_data_svg[13]<1], band_data_svg[13][band_data_svg[13]<1], 'b:', linewidth=2,label='')

# nuclear recoil bands
ax1.plot(band_data_svg[8], band_data_svg[9], 'r:', linewidth=2, label="Edw. paper NR")
ax1.plot(band_data_svg[10], band_data_svg[11], 'r:', linewidth=2, label='')

#ax1.plot(X,ynr_muv(X),'r--',label='NR mu')
#ax1.plot(X,ynr_muv(X)+3*ynr_sigv(X),'r-',label='NR 3$\sigma$')
#ax1.plot(X,ynr_muv(X)-3*ynr_sigv(X),'r-',label=None)

#ax1.plot(X,yer_muv(X),color='orange',linestyle='--',label='ER mu')
#ax1.plot(X,yer_muv(X)+3*yer_sigv(X),color='orange',linestyle='-',label='ER 3$\sigma$')
#ax1.plot(X,yer_muv(X)-3*yer_sigv(X),color='orange',linestyle='-',label=None)

#ax1.axvline(t(t_test[idx]), color='k', linestyle='-', lw=2, alpha=0.8,label=None)

ymin = 0
ymax = 1.7

ax1.set_yscale('linear')
#ax1.set_yscale('linear')
ax1.set_xlim(0, 100)
ax1.set_ylim(ymin,ymax)
ax1.set_xlabel(r'recoil energy [keV]',**axis_font)
ax1.set_ylabel('ionization yield',**axis_font)
ax1.grid(True)
ax1.yaxis.grid(True,which='minor',linestyle='--')
#ax1.legend(loc=1,prop={'size':22})
lgd = ax1.legend(bbox_to_anchor=(1.04,1),loc='upper left',borderaxespad=0,prop={'size':22})
#ax1.legend(bbox_to_anchor=(1.04,1),borderaxespad=0,prop={'size':22})

for axis in ['top','bottom','left','right']:
  ax1.spines[axis].set_linewidth(2)

#plt.tight_layout()
#plt.savefig('figures/GGA1_bandModel.png')
plt.savefig('figures/GGA1_bandModel.png', bbox_extra_artists=(lgd,), bbox_inches='tight')
plt.show()
```

# Matching Published and Calculated Widths

It is obvious from the plot above that, while our calculation using the fully correct distribution in the ionization-recoil plane matches very well with the Edw. paper model, it does not match with the bands extracted from Fig. 5 of the Edw. paper. I should plot these more carefully and look for discrepancies.

```python
#set up a 1d plot
fig,axes = plt.subplots(1,1,figsize=(9.0,8.0),sharex=True)
ax1 = axes

X=np.arange(0.1,160,0.1)

#ax1.plot(X,sigQnrv(X),color='r',linestyle="--",linewidth=2,label='single-scatter yield model (aH={:1.3})'.format(aH))
#ax1.plot(X,1.645*sigQerv(X),color='b',linestyle="--",linewidth=2,label='ER model Edw. (a$_H$={})'.format(aH))
ax1.plot(X,2.2*sigQerv(X),color='b',linestyle="--",linewidth=2,label='ER model Edw. (a$_H$={})'.format(aH))
ax1.plot(band_data_svg[12][band_data_svg[13]>1], band_data_svg[13][band_data_svg[13]>1]-1.0, 'b:', linewidth=2, \
        label='Edw. paper ER band plot')

ymin = 0.04
ymax = 0.2

ax1.set_yscale('linear')
#ax1.set_yscale('log')
ax1.set_xlim(0, 160)
ax1.set_ylim(ymin,ymax)
ax1.set_xlabel(r'recoil energy [keV]',**axis_font)
ax1.set_ylabel('ionization yield width',**axis_font)
ax1.grid(True)
ax1.yaxis.grid(True,which='minor',linestyle='--')
#ax1.legend(loc=1,prop={'size':22})
#ax1.legend(loc='upper right', bbox_to_anchor=(1, 0.5),prop={'size':22})
lgd = ax1.legend(bbox_to_anchor=(1.04,1),loc="upper left",borderaxespad=0,prop={'size':22})

for axis in ['top','bottom','left','right']:
  ax1.spines[axis].set_linewidth(2)

#plt.tight_layout()
#plt.savefig('figures/figure.png')
#plt.savefig('figures/figure.png', bbox_extra_artists=(lgd,), bbox_inches='tight')
plt.show()
```

So, by adjusting $a_H$ to 0.02 **_and_** increasing the ER model do report 2.2$\sigma$ we get close. I can keep playing this game until I get the curve right, but why in the world would Edw. have plotted some arbitrary value like 2.2$\sigma$?

```python

```