# Customer Segmentation: Estimate Individualized Responses to Incentives

Nowadays, business decision makers rely on estimating the causal effect of interventions to answer what-if questions about shifts in strategy, such as promoting specific product with discount, adding new features to a website or increasing investment from a sales team. However, rather than learning whether to take action for a specific intervention for all users, people are increasingly interested in understanding the different responses from different users to the two alternatives. Identifying the characteristics of users having the strongest response for the intervention could help make rules to segment the future users into different groups. This can help optimize the policy to use the least resources and get the most profit.

In this case study, we will use a personalized pricing example to explain how the [EconML](https://aka.ms/econml) and [DoWhy](https://github.com/microsoft/dowhy) libraries could fit into this problem and provide robust and reliable causal solutions.

### Summary

1. [Background](#background)
2. [Data](#data)
3. [Create Causal Model and Identify Causal Effect with DoWhy](#identify)
4. [Get Causal Effects with EconML](#estimate)
5. [Test Estimate Robustness with DoWhy](#robustness)
    1. [Add Random Common Cause](#random-common-cause)
    2. [Add Unobserved Common Cause](#unobserved-common-cause)
    3. [Replace Treatment with a Random (Placebo) Variable](#placebo-variable)
    4. [Remove a Random Subset of the Data](#subset)
6. [Understand Treatment Effects with EconML](#interpret)
7. [Make Policy Decisions with EconML](#policy)
8. [Conclusions](#conclusion)

# Background

The global online media market is growing fast over the years. Media companies are always interested in attracting more users into the market and encouraging them to buy more songs or become members. In this example, we'll consider a scenario where one experiment a media company is running is to give small discount (10%, 20% or 0) to their current users based on their income level in order to boost the likelihood of their purchase. The goal is to understand the **heterogeneous price elasticity of demand** for people with different income level, learning which users would respond most strongly to a small discount. Furthermore, their end goal is to make sure that despite decreasing the price for some consumers, the demand is raised enough to boost the overall revenue.

The EconML and DoWhy libraries complement each other in implementing this solution. On one hand, the DoWhy library can help [build a causal model, indentify the causal effect](#identify) and [test causal assumptions](#robustness). On the other hand, EconML’s `DML` based estimators can be used to take the discount variation in existing data, along with a rich set of user features, to [estimate heterogeneous price sensitivities](#estimate) that vary with multiple customer features. Then, the `SingleTreeCateInterpreter` provides a [presentation-ready summary](#interpret) of the key features that explain the biggest differences in responsiveness to a discount, and the `SingleTreePolicyInterpreter` recommends a [policy](#policy) on who should receive a discount in order to increase revenue (not only demand), which could help the company to set an optimal price for those users in the future.

```python
# Some imports to get us started
import warnings
warnings.simplefilter('ignore')

# Utilities
import os
import urllib.request
import numpy as np
import pandas as pd
from networkx.drawing.nx_pydot import to_pydot
from IPython.display import Image, display

# Generic ML imports
from sklearn.preprocessing import PolynomialFeatures
from sklearn.ensemble import GradientBoostingRegressor

# EconML imports
from econml.dml import LinearDML, CausalForestDML
from econml.cate_interpreter import SingleTreeCateInterpreter, SingleTreePolicyInterpreter

import matplotlib.pyplot as plt

%matplotlib inline
```

# Data

The dataset* has ~10,000 observations and includes 9 continuous and categorical variables that represent user's characteristics and online behaviour history such as age, log income, previous purchase, previous online time per week, etc.

We define the following variables:

Feature Name|Type|Details
:--- |:---|:---
**account_age** |W| user's account age
**age** |W|user's age
**avg_hours** |W| the average hours user was online per week in the past
**days_visited** |W| the average number of days user visited the website per week in the past
**friend_count** |W| number of friends user connected in the account
**has_membership** |W| whether the user had membership
**is_US** |W| whether the user accesses the website from the US
**songs_purchased** |W| the average songs user purchased per week in the past
**income** |X| user's income
**price** |T| the price user was exposed during the discount season (baseline price * small discount)
**demand** |Y| songs user purchased during the discount season

**To protect the privacy of the company, we use the simulated data as an example here. The data is synthetically generated and the feature distributions don't correspond to real distributions. However, the feature names have preserved their names and meaning.*

The treatment and outcome are generated using the following functions:
$$
T =
\begin{cases}
  1 & \text{with } p=0.2,  \\
  0.9 & \text{with }p=0.3, & \text{if income}<1 \\
  0.8 & \text{with }p=0.5, \\
  \\
    1 & \text{with }p=0.7, \\
  0.9 & \text{with }p=0.2, & \text{if income}\ge1 \\
  0.8 & \text{with }p=0.1, \\
\end{cases}
$$

\begin{align}
\gamma(X) & = -3 - 14 \cdot \{\text{income}<1\} \\
\beta(X,W) & = 20 + 0.5 \cdot \text{avg_hours} + 5 \cdot \{\text{days_visited}>4\} \\
Y &= \gamma(X) \cdot T + \beta(X,W)
\end{align}

```python
# Import the sample pricing data
file_url = "https://msalicedatapublic.blob.core.windows.net/datasets/Pricing/pricing_sample.csv"
train_data = pd.read_csv(file_url)
```

```python
# Data sample
train_data.head()
```

      account_age
      age
      avg_hours
      days_visited
      friends_count
      has_membership
      is_US
      songs_purchased
      income
      price
      demand

      0
      3
      53
      1.834234
      2
      8
      1
      1
      4.903237
      0.960863
      1.0
      3.917117

      1
      5
      54
      7.171411
      7
      9
      0
      1
      3.330161
      0.732487
      1.0
      11.585706

      2
      3
      33
      5.351920
      6
      9
      0
      1
      3.036203
      1.130937
      1.0
      24.675960

      3
      2
      34
      6.723551
      0
      8
      0
      1
      7.911926
      0.929197
      1.0
      6.361776

      4
      4
      30
      2.448247
      5
      8
      1
      0
      7.148967
      0.533527
      0.8
      12.624123

```python
# Define estimator inputs
train_data["log_demand"] = np.log(train_data["demand"])
train_data["log_price"] = np.log(train_data["price"])

Y = train_data["log_demand"].values
T = train_data["log_price"].values
X = train_data[["income"]].values  # features
confounder_names = ["account_age", "age", "avg_hours", "days_visited", "friends_count", "has_membership", "is_US", "songs_purchased"]
W = train_data[confounder_names].values
```

```python
# Get test data
X_test = np.linspace(0, 5, 100).reshape(-1, 1)
X_test_data = pd.DataFrame(X_test, columns=["income"])
```

# Create Causal Model and Identify Causal Effect with DoWhy

We define the causal assumptions with DoWhy. For example, we can include features we believe as confounders and features we think will influence the heterogeneity of the effect. With these assumptions defined, DoWhy can generate a causal graph for us, and use that graph to first identify the causal effect.

```python
# initiate an EconML cate estimator
est = LinearDML(model_y=GradientBoostingRegressor(), model_t=GradientBoostingRegressor(),
              featurizer=PolynomialFeatures(degree=2, include_bias=False))
```

```python
# fit through dowhy
est_dw = est.dowhy.fit(Y, T, X=X, W=W, outcome_names=["log_demand"], treatment_names=["log_price"], feature_names=["income"],
               confounder_names=confounder_names, inference="statsmodels")
```

    WARNING:dowhy.causal_model:Causal Graph not provided. DoWhy will construct a graph based on data inputs.
    INFO:dowhy.causal_graph:If this is observed data (not from a randomized experiment), there might always be missing confounders. Adding a node named "Unobserved Confounders" to reflect this.
    INFO:dowhy.causal_model:Model to find the causal effect of treatment ['log_price'] on outcome ['log_demand']
    WARNING:dowhy.causal_identifier:If this is observed data (not from a randomized experiment), there might always be missing confounders. Causal effect cannot be identified perfectly.
    INFO:dowhy.causal_identifier:Continuing by ignoring these unobserved confounders because proceed_when_unidentifiable flag is True.
    INFO:dowhy.causal_identifier:Instrumental variables for treatment and outcome:[]
    INFO:dowhy.causal_identifier:Frontdoor variables for treatment and outcome:[]
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income

```python
# Visualize causal graph
try:
    # Try pretty printing the graph. Requires pydot and pygraphviz
    display(
        Image(to_pydot(est_dw._graph._graph).create_png())
    )
except:
    # Fall back on default graph view
    est_dw.view_model()
```

```python
identified_estimand = est_dw.identified_estimand_
print(identified_estimand)
```

    Estimand type: nonparametric-ate

    ### Estimand : 1
    Estimand name: backdoor1 (Default)
    Estimand expression:
         d
    ────────────(Expectation(log_demand|has_membership,age,songs_purchased,friends
    d[log_price]

    _count,is_US,income,avg_hours,account_age,days_visited))

    Estimand assumption 1, Unconfoundedness: If U→{log_price} and U→log_demand then P(log_demand|log_price,has_membership,age,songs_purchased,friends_count,is_US,income,avg_hours,account_age,days_visited,U) = P(log_demand|log_price,has_membership,age,songs_purchased,friends_count,is_US,income,avg_hours,account_age,days_visited)

    ### Estimand : 2
    Estimand name: iv
    No such variable found!

    ### Estimand : 3
    Estimand name: frontdoor
    No such variable found!

# Get Causal Effects with EconML

Based on the identified causal effect above, we fit the model as follows using EconML:

\begin{align}
log(Y) & = \theta(X) \cdot log(T) + f(X,W) + \epsilon \\
log(T) & = g(X,W) + \eta
\end{align}

where $\epsilon, \eta$ are uncorrelated error terms.

The models we fit here aren't an exact match for the data generation function above, but if they are a good approximation, they will allow us to create a good discount policy.  Although the model is misspecified, we hope to see that our `DML` based estimators can still capture the right trend of $\theta(X)$ and that the recommended policy beats other baseline policies (such as always giving a discount) on revenue.  Because of the mismatch between the data generating process and the model we're fitting, there isn't a single true $\theta(X)$ (the true elasticity varies with not only X but also T and W), but given how we generate the data above, we can still calculate the range of true $\theta(X)$ to compare against.

```python
# Define underlying treatment effect function given DGP
def gamma_fn(X):
    return -3 - 14 * (X["income"] < 1)

def beta_fn(X):
    return 20 + 0.5 * (X["avg_hours"]) + 5 * (X["days_visited"] > 4)

def demand_fn(data, T):
    Y = gamma_fn(data) * T + beta_fn(data)
    return Y

def true_te(x, n, stats):
    if x < 1:
        subdata = train_data[train_data["income"] < 1].sample(n=n, replace=True)
    else:
        subdata = train_data[train_data["income"] >= 1].sample(n=n, replace=True)
    te_array = subdata["price"] * gamma_fn(subdata) / (subdata["demand"])
    if stats == "mean":
        return np.mean(te_array)
    elif stats == "median":
        return np.median(te_array)
    elif isinstance(stats, int):
        return np.percentile(te_array, stats)
```

```python
# Get the estimate and range of true treatment effect
truth_te_estimate = np.apply_along_axis(true_te, 1, X_test, 1000, "mean")  # estimate
truth_te_upper = np.apply_along_axis(true_te, 1, X_test, 1000, 95)  # upper level
truth_te_lower = np.apply_along_axis(true_te, 1, X_test, 1000, 5)  # lower level
```

    INFO:numexpr.utils:NumExpr defaulting to 8 threads.

## Parametric heterogeneity
First of all, we can try to learn a **linear projection of the treatment effect** assuming a polynomial form of $\theta(X)$. We use the `LinearDML` estimator. Since we don't have any priors on these models, we use a generic gradient boosting tree estimators to learn the expected price and demand from the data.

```python
lineardml_estimate = est_dw.estimate_
print(lineardml_estimate)
```

    *** Causal Estimate ***

    ## Identified estimand
    Estimand type: nonparametric-ate

    ## Realized estimand
    b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    Target units: ate

    ## Estimate
    Mean value: -0.995160166332067
    Effect estimates: [-1.10245137 -1.48197402 -0.84719356 ... -1.33838032 -1.90324458
     -0.4240255 ]

```python
# Get treatment effect and its confidence interval
te_pred = est_dw.effect(X_test).flatten()
te_pred_interval = est_dw.effect_interval(X_test)
```

```python
# Compare the estimate and the truth
plt.figure(figsize=(10, 6))
plt.plot(X_test.flatten(), te_pred, label="Sales Elasticity Prediction")
plt.plot(X_test.flatten(), truth_te_estimate, "--", label="True Elasticity")
plt.fill_between(
    X_test.flatten(),
    te_pred_interval[0].flatten(),
    te_pred_interval[1].flatten(),
    alpha=0.2,
    label="90% Confidence Interval",
)
plt.fill_between(
    X_test.flatten(),
    truth_te_lower,
    truth_te_upper,
    alpha=0.2,
    label="True Elasticity Range",
)
plt.xlabel("Income")
plt.ylabel("Songs Sales Elasticity")
plt.title("Songs Sales Elasticity vs Income")
plt.legend(loc="lower right")
```

From the plot above, it's clear to see that the true treatment effect is a **nonlinear** function of income, with elasticity around -1.75 when income is smaller than 1 and a small negative value when income is larger than 1. The model fits a quadratic treatment effect, which is not a great fit. But it still captures the overall trend: the elasticity is negative and people are less sensitive to the price change if they have higher income.

```python
# Get the final coefficient and intercept summary
est_dw.summary(feature_names=["income"])
```

Coefficient Results

      	     point_estimate	 stderr	  zstat	  pvalue	 ci_lower	 ci_upper

  income	        2.346	      0.082	 28.606	    0.0	    2.211	    2.481

  income^2	     -0.404	      0.028	 -14.598	   0.0	    -0.45	   -0.358

CATE Intercept Results

         	        point_estimate	 stderr	  zstat	  pvalue	 ci_lower	 ci_upper

  cate_intercept	     -2.984	      0.051	 -58.786	   0.0	   -3.067	    -2.9

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

`LinearDML` estimator can also return the summary of the coefficients and intercept for the final model, including point estimates, p-values and confidence intervals. From the table above, we notice that $income$ has positive effect and ${income}^2$ has negative effect, and both of them are statistically significant.

## Nonparametric Heterogeneity
Since we already know the true treatment effect function is nonlinear, let us fit another model using `CausalForestDML`, which assumes a fully **nonparametric estimation of the treatment effect**.

```python
# initiate an EconML cate estimator
est_nonparam = CausalForestDML(model_y=GradientBoostingRegressor(), model_t=GradientBoostingRegressor())
# fit through dowhy
est_nonparam_dw = est_nonparam.dowhy.fit(Y, T, X=X, W=W, outcome_names=["log_demand"], treatment_names=["log_price"],
                                         feature_names=["income"], confounder_names=confounder_names, inference="blb")
```

    WARNING:dowhy.causal_model:Causal Graph not provided. DoWhy will construct a graph based on data inputs.
    INFO:dowhy.causal_graph:If this is observed data (not from a randomized experiment), there might always be missing confounders. Adding a node named "Unobserved Confounders" to reflect this.
    INFO:dowhy.causal_model:Model to find the causal effect of treatment ['log_price'] on outcome ['log_demand']
    WARNING:dowhy.causal_identifier:If this is observed data (not from a randomized experiment), there might always be missing confounders. Causal effect cannot be identified perfectly.
    INFO:dowhy.causal_identifier:Continuing by ignoring these unobserved confounders because proceed_when_unidentifiable flag is True.
    INFO:dowhy.causal_identifier:Instrumental variables for treatment and outcome:[]
    INFO:dowhy.causal_identifier:Frontdoor variables for treatment and outcome:[]
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income

```python
# Get treatment effect and its confidence interval
te_pred = est_nonparam_dw.effect(X_test).flatten()
te_pred_interval = est_nonparam_dw.effect_interval(X_test)
```

```python
# Compare the estimate and the truth
plt.figure(figsize=(10, 6))
plt.plot(X_test.flatten(), te_pred, label="Sales Elasticity Prediction")
plt.plot(X_test.flatten(), truth_te_estimate, "--", label="True Elasticity")
plt.fill_between(
    X_test.flatten(),
    te_pred_interval[0].flatten(),
    te_pred_interval[1].flatten(),
    alpha=0.2,
    label="90% Confidence Interval",
)
plt.fill_between(
    X_test.flatten(),
    truth_te_lower,
    truth_te_upper,
    alpha=0.2,
    label="True Elasticity Range",
)
plt.xlabel("Income")
plt.ylabel("Songs Sales Elasticity")
plt.title("Songs Sales Elasticity vs Income")
plt.legend(loc="lower right")
```

We notice that this model fits much better than the `LinearDML`, the 90% confidence interval correctly covers the true treatment effect estimate and captures the variation when income is around 1. Overall, the model shows that people with low income are much more sensitive to the price changes than higher income people.

# Test Estimate Robustness with DoWhy

### Add Random Common Cause

How robust are our estimates to adding another confounder? We use DoWhy to test this!

```python
res_random = est_nonparam_dw.refute_estimate(method_name="random_common_cause")
print(res_random)
```

    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited+w_random | income

    Refute: Add a Random Common Cause
    Estimated effect:-0.9541062505955941
    New effect:-0.9567991329993085

### Add Unobserved Common Cause

How robust are our estimates to unobserved confounders? Since we assume the model is under unconfoundedness, adding an unobserved confounder might bias the estimates. We use DoWhy to test this!

```python
res_unobserved = est_nonparam_dw.refute_estimate(
    method_name="add_unobserved_common_cause",
    confounders_effect_on_treatment="linear",
    confounders_effect_on_outcome="linear",
    effect_strength_on_treatment=0.1,
    effect_strength_on_outcome=0.1,
)
print(res_unobserved)
```

    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income

    Refute: Add an Unobserved Common Cause
    Estimated effect:-0.9541062505955941
    New effect:0.18928228066056693

### Replace Treatment with a Random (Placebo) Variable

What happens our estimates if we replace the treatment variable with noise? Ideally, the average effect would be wildly different than our original estimate. We use DoWhy to investigate!

```python
res_placebo = est_nonparam_dw.refute_estimate(
    method_name="placebo_treatment_refuter", placebo_type="permute",
    num_simulations=3
)
print(res_placebo)
```

    INFO:dowhy.causal_refuters.placebo_treatment_refuter:Refutation over 3 simulated datasets of permute treatment
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~placebo+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~placebo+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~placebo+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    WARNING:dowhy.causal_refuters.placebo_treatment_refuter:We assume a Normal Distribution as the sample has less than 100 examples.
                     Note: The underlying distribution may not be Normal. We assume that it approaches normal with the increase in sample size.

    Refute: Use a Placebo Treatment
    Estimated effect:-0.9541062505955941
    New effect:-0.0007540292387552173
    p value:0.46950460937416916

### Remove a Random Subset of the Data

Do we recover similar estimates on subsets of the data? This speaks to the ability of our chosen estimator to generalize well. We use DoWhy to investigate this!

```python
res_subset = est_nonparam_dw.refute_estimate(
    method_name="data_subset_refuter", subset_fraction=0.8,
    num_simulations=3)
print(res_subset)
```

    INFO:dowhy.causal_refuters.data_subset_refuter:Refutation over 0.8 simulated datasets of size 8000.0 each
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    INFO:dowhy.causal_estimator:INFO: Using EconML Estimator
    INFO:dowhy.causal_estimator:b: log_demand~log_price+has_membership+age+songs_purchased+friends_count+is_US+income+avg_hours+account_age+days_visited | income
    WARNING:dowhy.causal_refuters.data_subset_refuter:We assume a Normal Distribution as the sample has less than 100 examples.
                     Note: The underlying distribution may not be Normal. We assume that it approaches normal with the increase in sample size.

    Refute: Use a subset of data
    Estimated effect:-0.9541062505955941
    New effect:-0.955504419131402
    p value:0.3540851081425652

# Understand Treatment Effects with EconML
EconML includes interpretability tools to better understand treatment effects. Treatment effects can be complex, but oftentimes we are interested in simple rules that can differentiate between users who respond positively, users who remain neutral and users who respond negatively to the proposed changes.

The EconML `SingleTreeCateInterpreter` provides interperetability by training a single decision tree on the treatment effects outputted by the any of the EconML estimators. In the figure below we can see in dark red users respond strongly to the discount and the in white users respond lightly to the discount.

```python
intrp = SingleTreeCateInterpreter(include_model_uncertainty=True, max_depth=2, min_samples_leaf=10)
intrp.interpret(est_nonparam_dw, X_test)
plt.figure(figsize=(25, 5))
intrp.plot(feature_names=["income"], fontsize=12)
```

# Make Policy Decision with EconML
We want to make policy decisions to maximum the **revenue** instead of the demand. In this scenario,

\begin{align}
Rev & = Y \cdot T \\
    & = \exp^{log(Y)} \cdot T\\
    & = \exp^{(\theta(X) \cdot log(T) + f(X,W) + \epsilon)} \cdot T \\
    & = \exp^{(f(X,W) + \epsilon)} \cdot T^{(\theta(X)+1)}
\end{align}

With the decrease of price, revenue will increase only if $\theta(X)+1<0$. Thus, we set `sample_treatment_cast=-1` here to learn **what kinds of customers we should give a small discount to maximum the revenue**.

The EconML library includes policy interpretability tools such as `SingleTreePolicyInterpreter` that take in a treatment cost and the treatment effects to learn simple rules about which customers to target profitably. In the figure below we can see the model recommends to give discount for people with income less than $0.985$ and give original price for the others.

```python
intrp = SingleTreePolicyInterpreter(risk_level=0.05, max_depth=2, min_samples_leaf=1, min_impurity_decrease=0.001)
intrp.interpret(est_nonparam_dw, X_test, sample_treatment_costs=-1)
plt.figure(figsize=(25, 5))
intrp.plot(feature_names=["income"], treatment_names=["Discount", "No-Discount"], fontsize=12)
```

Now, let us compare our policy with other baseline policies! Our model says which customers to give a small discount to, and for this experiment, we will set a discount level of 10% for those users. Because the model is misspecified we would not expect good results with large discounts. Here, because we know the ground truth, we can evaluate the value of this policy.

```python
# define function to compute revenue
def revenue_fn(data, discount_level1, discount_level2, baseline_T, policy):
    policy_price = baseline_T * (1 - discount_level1) * policy + baseline_T * (1 - discount_level2) * (1 - policy)
    demand = demand_fn(data, policy_price)
    rev = demand * policy_price
    return rev
```

```python
policy_dic = {}
# our policy above
policy = intrp.treat(X)
policy_dic["Our Policy"] = np.mean(revenue_fn(train_data, 0, 0.1, 1, policy))

## previous strategy
policy_dic["Previous Strategy"] = np.mean(train_data["price"] * train_data["demand"])

## give everyone discount
policy_dic["Give Everyone Discount"] = np.mean(revenue_fn(train_data, 0.1, 0, 1, np.ones(len(X))))

## don't give discount
policy_dic["Give No One Discount"] = np.mean(revenue_fn(train_data, 0, 0.1, 1, np.ones(len(X))))

## follow our policy, but give -10% discount for the group doesn't recommend to give discount
policy_dic["Our Policy + Give Negative Discount for No-Discount Group"] = np.mean(revenue_fn(train_data, -0.1, 0.1, 1, policy))

## give everyone -10% discount
policy_dic["Give Everyone Negative Discount"] = np.mean(revenue_fn(train_data, -0.1, 0, 1, np.ones(len(X))))
```

```python
# get policy summary table
res = pd.DataFrame.from_dict(policy_dic, orient="index", columns=["Revenue"])
res["Rank"] = res["Revenue"].rank(ascending=False)
res
```

      Revenue
      Rank

      Our Policy
      14.686241
      2.0

      Previous Strategy
      14.349342
      4.0

      Give Everyone Discount
      13.774469
      6.0

      Give No One Discount
      14.294606
      5.0

      Our Policy + Give Negative Discount for No-Discount Group
      15.564411
      1.0

      Give Everyone Negative Discount
      14.612670
      3.0

**We beat the baseline policies!** Our policy gets the highest revenue except for the one raising the price for the No-Discount group. That means our currently baseline price is low, but the way we segment the user does help increase the revenue!

# Conclusions

In this notebook, we have demonstrated the power of using EconML and DoWhy to:

* Estimate the treatment effect correctly even the model is misspecified
* Test causal assumptions and investigate the robustness of the resulting estimates
* Interpret the resulting individual-level treatment effects
* Make the policy decision beats the previous and baseline policies

To learn more about what EconML can do for you, visit our [website](https://aka.ms/econml), our [GitHub page](https://github.com/microsoft/EconML) or our [docummentation](https://econml.azurewebsites.net/).

To learn more about what DoWhy can do for you, visit the [GitHub page](https://github.com/microsoft/dowhy) or [documentation](https://microsoft.github.io/dowhy/index.html).

notebooks/CustomerScenarios/Case Study - Customer Segmentation at An Online Media Company - EconML + DoWhy.ipynb


```python
import numpy as np
import math
import sympy as sy
import csv

#opening csv file (Sheets or Excel) as np in an array format
my_data = np.genfromtxt('physic_data.csv', delimiter=',')

#creating files to save the new datas
arq = open('medias.csv', "w+")
arq2 = open( 'alpha.csv', "w+")
arq3 = open( 'delta_T.csv', "w+")

```

```python
# array to save the new data
medias = []

#numero de amostras coletadas para cada um dos 11 experimentos
amostras = 5

print('=' * 40)
print(f'{"FORMULAS":^40}')
print('soma = SOMAR OS 5 VALORES')
print('tempo_Medio = soma / 5')
print('=' * 40)

for data_column in range(0, len(my_data[0])):
  print('------------------>> COLUNA {} <<------------------  '.format(data_column+1))
  soma = 0

  #sum all the 5 values
  for data_row in range(1, 6):
    soma += (my_data[data_row][data_column])

  print(f'{"Soma das"} {amostras} {"amostras":.<30} {soma:>7.4f}')
  #printin data
  print(f'{"tempo_Medio":.<30} {((soma)/5):>7.4f}')
  #adding data to array
  medias.append(soma/5)

print(medias)

# writing data to file
for element in medias:
    arq.write("%s\n" % element)
arq.close()

```

    ========================================
                    FORMULAS
    soma = SOMAR OS 5 VALORES
    tempo_Medio = soma / 5
    ========================================
    ------------------>> COLUNA 1 <<------------------
    Soma das 5 amostras......................  0.3092
    tempo_Medio...................  0.0618
    ------------------>> COLUNA 2 <<------------------
    Soma das 5 amostras......................  0.6159
    tempo_Medio...................  0.1232
    ------------------>> COLUNA 3 <<------------------
    Soma das 5 amostras......................  0.9022
    tempo_Medio...................  0.1804
    ------------------>> COLUNA 4 <<------------------
    Soma das 5 amostras......................  1.2523
    tempo_Medio...................  0.2505
    ------------------>> COLUNA 5 <<------------------
    Soma das 5 amostras......................  1.5438
    tempo_Medio...................  0.3088
    ------------------>> COLUNA 6 <<------------------
    Soma das 5 amostras......................  1.8513
    tempo_Medio...................  0.3703
    ------------------>> COLUNA 7 <<------------------
    Soma das 5 amostras......................  2.1723
    tempo_Medio...................  0.4345
    ------------------>> COLUNA 8 <<------------------
    Soma das 5 amostras......................  2.4955
    tempo_Medio...................  0.4991
    ------------------>> COLUNA 9 <<------------------
    Soma das 5 amostras......................  2.7763
    tempo_Medio...................  0.5553
    ------------------>> COLUNA 10 <<------------------
    Soma das 5 amostras......................  3.1556
    tempo_Medio...................  0.6311
    ------------------>> COLUNA 11 <<------------------
    Soma das 5 amostras......................  3.4579
    tempo_Medio...................  0.6916
    [0.06183999999999999, 0.12318, 0.18044, 0.25046, 0.30876000000000003, 0.37026, 0.43446000000000007, 0.4991, 0.55526, 0.63112, 0.69158]

```python
import math
import sympy as sy

#numero de amostras coletadas para cada um dos 11 experimentos
amostras = 5

for data_column in range(0, len(my_data[0])):
  print('\n------------------>> COLUNA {} <<------------------  \n'.format(data_column+1))

  for data_row in range(1, 6):
    # formular to calculete the alpha
    alpha = math.sqrt(
        ((my_data[data_row][data_column] - medias[data_column])**2) / (amostras - 1))
    # Formular to calculete the delta_T
    delta_T = (alpha / math.sqrt(amostras))

    #printing the data
    print('=' * 40)
    print(f'{"FORMULAS {}":^40}'.format(data_column+1))
    print(
        'alpha = sqrt( (({} - {})**2) / (4)) '.format(my_data[data_row][data_column], medias[data_column]))
    print(
        'delta_T = ({} / sqrt({}))'.format(alpha, amostras))
    print('=' * 40)

    # printing and writing the data in file
    print(f'{"alpha":.<30} {alpha:>7.30f}')
    arq2.write("%s\n" % alpha)

    # printing and writing the data in file
    print(f'{"delta_T":.<30} {delta_T:>7.30f}')
    arq3.write("%f\n" % delta_T)

    print('=' * 40)

arq2.close()
arq3.close()

```

    ------------------>> COLUNA 1 <<------------------

    ========================================
                  FORMULAS 1
    alpha = sqrt( ((0.0616 - 0.06183999999999999)**2) / (4))
    delta_T = (0.00011999999999999511 / sqrt(5))
    ========================================
    alpha......................... 0.000119999999999995110577799551
    delta_T....................... 0.000053665631459992763107377106
    ========================================
    ========================================
                  FORMULAS 1
    alpha = sqrt( ((0.0615 - 0.06183999999999999)**2) / (4))
    delta_T = (0.00016999999999999654 / sqrt(5))
    ========================================
    alpha......................... 0.000169999999999996542765501317
    delta_T....................... 0.000076026311234991304582007954
    ========================================
    ========================================
                  FORMULAS 1
    alpha = sqrt( ((0.0636 - 0.06183999999999999)**2) / (4))
    delta_T = (0.0008800000000000058 / sqrt(5))
    ========================================
    alpha......................... 0.000880000000000005777600620149
    delta_T....................... 0.000393547964039965537073884061
    ========================================
    ========================================
                  FORMULAS 1
    alpha = sqrt( ((0.0613 - 0.06183999999999999)**2) / (4))
    delta_T = (0.00026999999999999594 / sqrt(5))
    ========================================
    alpha......................... 0.000269999999999995937693952897
    delta_T....................... 0.000120747670784986815438119545
    ========================================
    ========================================
                  FORMULAS 1
    alpha = sqrt( ((0.0612 - 0.06183999999999999)**2) / (4))
    delta_T = (0.00031999999999999737 / sqrt(5))
    ========================================
    alpha......................... 0.000319999999999997369881654663
    delta_T....................... 0.000143108350559985363689013971
    ========================================

    ------------------>> COLUNA 2 <<------------------

    ========================================
                  FORMULAS 2
    alpha = sqrt( ((0.1208 - 0.12318)**2) / (4))
    delta_T = (0.0011899999999999966 / sqrt(5))
    ========================================
    alpha......................... 0.001189999999999996616040220943
    delta_T....................... 0.000532184178644948442660211896
    ========================================
    ========================================
                  FORMULAS 2
    alpha = sqrt( ((0.128 - 0.12318)**2) / (4))
    delta_T = (0.0024100000000000024 / sqrt(5))
    ========================================
    alpha......................... 0.002410000000000002418065747634
    delta_T....................... 0.001077784765154899760189466384
    ========================================
    ========================================
                  FORMULAS 2
    alpha = sqrt( ((0.1176 - 0.12318)**2) / (4))
    delta_T = (0.002790000000000001 / sqrt(5))
    ========================================
    alpha......................... 0.002790000000000000812683254026
    delta_T....................... 0.001247725931444882904730597772
    ========================================
    ========================================
                  FORMULAS 2
    alpha = sqrt( ((0.1235 - 0.12318)**2) / (4))
    delta_T = (0.00016000000000000042 / sqrt(5))
    ========================================
    alpha......................... 0.000160000000000000419664303308
    delta_T....................... 0.000071554175279993454338554881
    ========================================
    ========================================
                  FORMULAS 2
    alpha = sqrt( ((0.126 - 0.12318)**2) / (4))
    delta_T = (0.0014100000000000015 / sqrt(5))
    ========================================
    alpha......................... 0.001410000000000001529887327933
    delta_T....................... 0.000630571169654941371916778703
    ========================================

    ------------------>> COLUNA 3 <<------------------

    ========================================
                  FORMULAS 3
    alpha = sqrt( ((0.1834 - 0.18044)**2) / (4))
    delta_T = (0.001480000000000009 / sqrt(5))
    ========================================
    alpha......................... 0.001480000000000009086065233532
    delta_T....................... 0.000661876121339941827712016753
    ========================================
    ========================================
                  FORMULAS 3
    alpha = sqrt( ((0.1827 - 0.18044)**2) / (4))
    delta_T = (0.001130000000000006 / sqrt(5))
    ========================================
    alpha......................... 0.001130000000000005999645225074
    delta_T....................... 0.000505351362914955161247110293
    ========================================
    ========================================
                  FORMULAS 3
    alpha = sqrt( ((0.1774 - 0.18044)**2) / (4))
    delta_T = (0.0015199999999999936 / sqrt(5))
    ========================================
    alpha......................... 0.001519999999999993578470025568
    delta_T....................... 0.000679764665159933120265611795
    ========================================
    ========================================
                  FORMULAS 3
    alpha = sqrt( ((0.1772 - 0.18044)**2) / (4))
    delta_T = (0.0016199999999999964 / sqrt(5))
    ========================================
    alpha......................... 0.001619999999999996442845429101
    delta_T....................... 0.000724486024709930270977509270
    ========================================
    ========================================
                  FORMULAS 3
    alpha = sqrt( ((0.1815 - 0.18044)**2) / (4))
    delta_T = (0.0005300000000000027 / sqrt(5))
    ========================================
    alpha......................... 0.000530000000000002691180611691
    delta_T....................... 0.000237023205614978897714031914
    ========================================

    ------------------>> COLUNA 4 <<------------------

    ========================================
                  FORMULAS 4
    alpha = sqrt( ((0.2407 - 0.25046)**2) / (4))
    delta_T = (0.004880000000000009 / sqrt(5))
    ========================================
    alpha......................... 0.004880000000000009330314298950
    delta_T....................... 0.002182402346039798764903983042
    ========================================
    ========================================
                  FORMULAS 4
    alpha = sqrt( ((0.2996 - 0.25046)**2) / (4))
    delta_T = (0.02456999999999998 / sqrt(5))
    ========================================
    alpha......................... 0.024569999999999980744291860901
    delta_T....................... 0.010988038041433957278658617440
    ========================================
    ========================================
                  FORMULAS 4
    alpha = sqrt( ((0.238 - 0.25046)**2) / (4))
    delta_T = (0.006230000000000013 / sqrt(5))
    ========================================
    alpha......................... 0.006230000000000013304912727108
    delta_T....................... 0.002786140699964743602801142686
    ========================================
    ========================================
                  FORMULAS 4
    alpha = sqrt( ((0.2388 - 0.25046)**2) / (4))
    delta_T = (0.005830000000000002 / sqrt(5))
    ========================================
    alpha......................... 0.005830000000000001847411112976
    delta_T....................... 0.002607255261764755433634421777
    ========================================
    ========================================
                  FORMULAS 4
    alpha = sqrt( ((0.2352 - 0.25046)**2) / (4))
    delta_T = (0.007630000000000012 / sqrt(5))
    ========================================
    alpha......................... 0.007630000000000011772804953125
    delta_T....................... 0.003412239733664684197128602605
    ========================================

    ------------------>> COLUNA 5 <<------------------

    ========================================
                  FORMULAS 5
    alpha = sqrt( ((0.3063 - 0.30876000000000003)**2) / (4))
    delta_T = (0.0012300000000000089 / sqrt(5))
    ========================================
    alpha......................... 0.001230000000000008864020628607
    delta_T....................... 0.000550072722464952203538790521
    ========================================
    ========================================
                  FORMULAS 5
    alpha = sqrt( ((0.3009 - 0.30876000000000003)**2) / (4))
    delta_T = (0.003930000000000017 / sqrt(5))
    ========================================
    alpha......................... 0.003930000000000016813217484923
    delta_T....................... 0.001757549430314842096173544306
    ========================================
    ========================================
                  FORMULAS 5
    alpha = sqrt( ((0.3053 - 0.30876000000000003)**2) / (4))
    delta_T = (0.0017300000000000093 / sqrt(5))
    ========================================
    alpha......................... 0.001730000000000009308109838457
    delta_T....................... 0.000773679520214931343465025737
    ========================================
    ========================================
                  FORMULAS 5
    alpha = sqrt( ((0.3219 - 0.30876000000000003)**2) / (4))
    delta_T = (0.0065699999999999925 / sqrt(5))
    ========================================
    alpha......................... 0.006569999999999992512655921928
    delta_T....................... 0.002938193322434720384378481484
    ========================================
    ========================================
                  FORMULAS 5
    alpha = sqrt( ((0.3094 - 0.30876000000000003)**2) / (4))
    delta_T = (0.00031999999999998696 / sqrt(5))
    ========================================
    alpha......................... 0.000319999999999986961540798802
    delta_T....................... 0.000143108350559980701619672283
    ========================================

    ------------------>> COLUNA 6 <<------------------

    ========================================
                  FORMULAS 6
    alpha = sqrt( ((0.3673 - 0.37026)**2) / (4))
    delta_T = (0.0014799999999999813 / sqrt(5))
    ========================================
    alpha......................... 0.001479999999999981330489617903
    delta_T....................... 0.000661876121339929359387033170
    ========================================
    ========================================
                  FORMULAS 6
    alpha = sqrt( ((0.3791 - 0.37026)**2) / (4))
    delta_T = (0.004420000000000007 / sqrt(5))
    ========================================
    alpha......................... 0.004420000000000007256417688950
    delta_T....................... 0.001976684092109817422744377780
    ========================================
    ========================================
                  FORMULAS 6
    alpha = sqrt( ((0.3666 - 0.37026)**2) / (4))
    delta_T = (0.0018299999999999983 / sqrt(5))
    ========================================
    alpha......................... 0.001829999999999998294697434176
    delta_T....................... 0.000818400879764922205804322797
    ========================================
    ========================================
                  FORMULAS 6
    alpha = sqrt( ((0.3717 - 0.37026)**2) / (4))
    delta_T = (0.0007199999999999984 / sqrt(5))
    ========================================
    alpha......................... 0.000719999999999998419042412934
    delta_T....................... 0.000321993788759968979206610440
    ========================================
    ========================================
                  FORMULAS 6
    alpha = sqrt( ((0.3666 - 0.37026)**2) / (4))
    delta_T = (0.0018299999999999983 / sqrt(5))
    ========================================
    alpha......................... 0.001829999999999998294697434176
    delta_T....................... 0.000818400879764922205804322797
    ========================================

    ------------------>> COLUNA 7 <<------------------

    ========================================
                  FORMULAS 7
    alpha = sqrt( ((0.4398 - 0.43446000000000007)**2) / (4))
    delta_T = (0.002669999999999978 / sqrt(5))
    ========================================
    alpha......................... 0.002669999999999977946529838846
    delta_T....................... 0.001194060299984877693627027817
    ========================================
    ========================================
                  FORMULAS 7
    alpha = sqrt( ((0.4226 - 0.43446000000000007)**2) / (4))
    delta_T = (0.005930000000000046 / sqrt(5))
    ========================================
    alpha......................... 0.005930000000000046345149939953
    delta_T....................... 0.002651976621314771232623686004
    ========================================
    ========================================
                  FORMULAS 7
    alpha = sqrt( ((0.4306 - 0.43446000000000007)**2) / (4))
    delta_T = (0.0019300000000000428 / sqrt(5))
    ========================================
    alpha......................... 0.001930000000000042792436261152
    delta_T....................... 0.000863122239314937896373369774
    ========================================
    ========================================
                  FORMULAS 7
    alpha = sqrt( ((0.4302 - 0.43446000000000007)**2) / (4))
    delta_T = (0.0021300000000000208 / sqrt(5))
    ========================================
    alpha......................... 0.002130000000000020765611452589
    delta_T....................... 0.000952564958414919621051963894
    ========================================
    ========================================
                  FORMULAS 7
    alpha = sqrt( ((0.4491 - 0.43446000000000007)**2) / (4))
    delta_T = (0.007319999999999965 / sqrt(5))
    ========================================
    alpha......................... 0.007319999999999965423214121074
    delta_T....................... 0.003273603519059676680152959349
    ========================================

    ------------------>> COLUNA 8 <<------------------

    ========================================
                  FORMULAS 8
    alpha = sqrt( ((0.5004 - 0.4991)**2) / (4))
    delta_T = (0.0006499999999999839 / sqrt(5))
    ========================================
    alpha......................... 0.000649999999999983923970603428
    delta_T....................... 0.000290688837074965433435180806
    ========================================
    ========================================
                  FORMULAS 8
    alpha = sqrt( ((0.4869 - 0.4991)**2) / (4))
    delta_T = (0.006099999999999994 / sqrt(5))
    ========================================
    alpha......................... 0.006099999999999994315658113919
    delta_T....................... 0.002728002932549740758294554155
    ========================================
    ========================================
                  FORMULAS 8
    alpha = sqrt( ((0.4888 - 0.4991)**2) / (4))
    delta_T = (0.005149999999999988 / sqrt(5))
    ========================================
    alpha......................... 0.005149999999999987920773492078
    delta_T....................... 0.002303150016824778018031949500
    ========================================
    ========================================
                  FORMULAS 8
    alpha = sqrt( ((0.5071 - 0.4991)**2) / (4))
    delta_T = (0.0040000000000000036 / sqrt(5))
    ========================================
    alpha......................... 0.004000000000000003552713678801
    delta_T....................... 0.001788854381999833336250316229
    ========================================
    ========================================
                  FORMULAS 8
    alpha = sqrt( ((0.5123 - 0.4991)**2) / (4))
    delta_T = (0.006599999999999995 / sqrt(5))
    ========================================
    alpha......................... 0.006599999999999994759747323769
    delta_T....................... 0.002951609730299720006641006620
    ========================================

    ------------------>> COLUNA 9 <<------------------

    ========================================
                  FORMULAS 9
    alpha = sqrt( ((0.5479 - 0.55526)**2) / (4))
    delta_T = (0.003679999999999961 / sqrt(5))
    ========================================
    alpha......................... 0.003679999999999961080021648741
    delta_T....................... 0.001645746031439827752190785404
    ========================================
    ========================================
                  FORMULAS 9
    alpha = sqrt( ((0.5535 - 0.55526)**2) / (4))
    delta_T = (0.0008799999999999919 / sqrt(5))
    ========================================
    alpha......................... 0.000879999999999991899812812335
    delta_T....................... 0.000393547964039959357121500894
    ========================================
    ========================================
                  FORMULAS 9
    alpha = sqrt( ((0.5579 - 0.55526)**2) / (4))
    delta_T = (0.0013199999999999878 / sqrt(5))
    ========================================
    alpha......................... 0.001319999999999987849719218502
    delta_T....................... 0.000590321946059939035682251340
    ========================================
    ========================================
                  FORMULAS 9
    alpha = sqrt( ((0.5592 - 0.55526)**2) / (4))
    delta_T = (0.0019700000000000273 / sqrt(5))
    ========================================
    alpha......................... 0.001970000000000027284841053188
    delta_T....................... 0.000881010783134929297347182064
    ========================================
    ========================================
                  FORMULAS 9
    alpha = sqrt( ((0.5578 - 0.55526)**2) / (4))
    delta_T = (0.0012699999999999934 / sqrt(5))
    ========================================
    alpha......................... 0.001269999999999993356425420643
    delta_T....................... 0.000567961266284943604512602811
    ========================================

    ------------------>> COLUNA 10 <<------------------

    ========================================
                  FORMULAS 10
    alpha = sqrt( ((0.6198 - 0.63112)**2) / (4))
    delta_T = (0.005659999999999998 / sqrt(5))
    ========================================
    alpha......................... 0.005659999999999998365751707752
    delta_T....................... 0.002531228950529761188154020957
    ========================================
    ========================================
                  FORMULAS 10
    alpha = sqrt( ((0.6259 - 0.63112)**2) / (4))
    delta_T = (0.002610000000000001 / sqrt(5))
    ========================================
    alpha......................... 0.002610000000000001207922650792
    delta_T....................... 0.001167227484254890809006743879
    ========================================
    ========================================
                  FORMULAS 10
    alpha = sqrt( ((0.6037 - 0.63112)**2) / (4))
    delta_T = (0.01371 / sqrt(5))
    ========================================
    alpha......................... 0.013709999999999999964472863212
    delta_T....................... 0.006131298394304422749723215702
    ========================================
    ========================================
                  FORMULAS 10
    alpha = sqrt( ((0.6504 - 0.63112)**2) / (4))
    delta_T = (0.009639999999999982 / sqrt(5))
    ========================================
    alpha......................... 0.009639999999999981916687374905
    delta_T....................... 0.004311139060619586030331795712
    ========================================
    ========================================
                  FORMULAS 10
    alpha = sqrt( ((0.6558 - 0.63112)**2) / (4))
    delta_T = (0.012340000000000018 / sqrt(5))
    ========================================
    alpha......................... 0.012340000000000017621459846850
    delta_T....................... 0.005518615768469488716552184826
    ========================================

    ------------------>> COLUNA 11 <<------------------

    ========================================
                  FORMULAS 11
    alpha = sqrt( ((0.708 - 0.69158)**2) / (4))
    delta_T = (0.008209999999999995 / sqrt(5))
    ========================================
    alpha......................... 0.008209999999999995079491554861
    delta_T....................... 0.003671623619054652318954845569
    ========================================
    ========================================
                  FORMULAS 11
    alpha = sqrt( ((0.6628 - 0.69158)**2) / (4))
    delta_T = (0.014390000000000014 / sqrt(5))
    ========================================
    alpha......................... 0.014390000000000013891110484110
    delta_T....................... 0.006435403639244400599006556973
    ========================================
    ========================================
                  FORMULAS 11
    alpha = sqrt( ((0.7292 - 0.69158)**2) / (4))
    delta_T = (0.018809999999999993 / sqrt(5))
    ========================================
    alpha......................... 0.018809999999999993391952557431
    delta_T....................... 0.008412087731354205011324864927
    ========================================
    ========================================
                  FORMULAS 11
    alpha = sqrt( ((0.6875 - 0.69158)**2) / (4))
    delta_T = (0.0020399999999999863 / sqrt(5))
    ========================================
    alpha......................... 0.002039999999999986268761631436
    delta_T....................... 0.000912315734819907960678753156
    ========================================
    ========================================
                  FORMULAS 11
    alpha = sqrt( ((0.6704 - 0.69158)**2) / (4))
    delta_T = (0.010589999999999988 / sqrt(5))
    ========================================
    alpha......................... 0.010589999999999988311571996746
    delta_T....................... 0.004735991976344548770594400366
    ========================================

-------------TEST BELOW----------------

```python
column1 = []
column2 = []

for column in my_data:
    column1.append([column[0]])
    column2.append([column[1]])
print('-----------')

print(column1)
print(column2)

```

    -----------
    [[nan], [0.0616], [0.0615], [0.0636], [0.0613], [0.0612]]
    [[nan], [0.1208], [0.128], [0.1176], [0.1235], [0.126]]

```python
print(my_data[5][0])
print(len(my_data[0]))

print(my_data)
```

    0.0612
    11
    [[   nan    nan    nan    nan    nan    nan    nan    nan    nan    nan
         nan]
     [0.0616 0.1208 0.1834 0.2407 0.3063 0.3673 0.4398 0.5004 0.5479 0.6198
      0.708 ]
     [0.0615 0.128  0.1827 0.2996 0.3009 0.3791 0.4226 0.4869 0.5535 0.6259
      0.6628]
     [0.0636 0.1176 0.1774 0.238  0.3053 0.3666 0.4306 0.4888 0.5579 0.6037
      0.7292]
     [0.0613 0.1235 0.1772 0.2388 0.3219 0.3717 0.4302 0.5071 0.5592 0.6504
      0.6875]
     [0.0612 0.126  0.1815 0.2352 0.3094 0.3666 0.4491 0.5123 0.5578 0.6558
      0.6704]]

```python
i = sy.Symbol('i')
```

```python
con = np.concatenate((column1, column2), axis=1)
# https://numpy.org/doc/stable/reference/generated/numpy.concatenate.html
print(con)

```

    [[   nan    nan]
     [0.0616 0.1208]
     [0.0615 0.128 ]
     [0.0636 0.1176]
     [0.0613 0.1235]
     [0.0612 0.126 ]]

```python

```


# Learning in LTN

This tutorial explains how to learn some language symbols (predicates, functions, constants) using the satisfaction of a knowledgebase as an objective. It expects basic familiarity of the first two turoials on LTN (grounding symbols and connectives).

```python
import logictensornetworks as ltn
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
```

    Init Plugin
    Init Graph Optimizer
    Init Kernel

We use the following simple example to illustrate learning in LTN.

The domain is the square $[0,4] \times [0,4]$. We have one example of the class $A$ and one example of the class $B$. The rest of the individuals are not labelled, but there are two assumptions:
- $A$ and $B$ are mutually exclusive,
- any two close points should share the same label.

```python
points = np.array(
        [[0.4,0.3],[1.2,0.3],[2.2,1.3],[1.7,1.0],[0.5,0.5],[0.3, 1.5],[1.3, 1.1],[0.9, 1.7],
        [3.4,3.3],[3.2,3.3],[3.2,2.3],[2.7,2.0],[3.5,3.5],[3.3, 2.5],[3.3, 1.1],[1.9, 3.7],[1.3, 3.5],[3.3, 1.1],[3.9, 3.7]])
point_a = [3.3,2.5]
point_b = [1.3,1.1]
fig, ax = plt.subplots()
ax.set_xlim(0,4)
ax.set_ylim(0,4)
ax.scatter(points[:,0],points[:,1],color="black",label="unknown")
ax.scatter(point_a[0],point_a[1],color="blue",label="a")
ax.scatter(point_b[0],point_b[1],color="red",label="b")
ax.set_title("Dataset of individuals")
plt.legend();
```

We define the membership predicate $C(x,l)$, where $x$ is an individual and $l$ is a onehot label to denote the two classes. $C$ is approximated by a simple MLP. The last layer, that computes probabilities per class, uses a `softmax` activation, ensuring that the classes are mutually-exclusive.

We define the knowledgebase $\mathcal{K}$ composed of the following rules:
\begin{align}
& C(a,l_a)\\
& C(b,l_b)\\
\forall x_1,x_2,l\ \big(\mathrm{Sim}(x_1,x_2) & \rightarrow \big(C(x_1,l)\leftrightarrow C(x_2,l)\big)\big)
\end{align}
where $a$ and $b$ the two individuals already classified; $x_1$,$x_2$ are variables ranging over all individuals; $l_a$, $l_b$ are the one-hot labels for $A$ and $B$; $l$ is a variable ranging over the labels. $\mathrm{Sim}$ is a predicate measuring similarity between two points. $\mathcal{G}(\mathrm{Sim}):\vec{u},\vec{v}\mapsto \exp(-\|\vec{u}-\vec{v} \|^2)$.

The objective is to learn the predicate $C$ to maximize the satisfaction of $\mathcal{K}$. If $\theta$ denotes the set of trainable parameters, the task is :
$$
\theta^\ast = \mathrm{argmax}_{\theta\in\Theta}\ \mathrm{SatAgg}_{\phi\in\mathcal{K}}\mathcal{G}_{\theta}(\phi)
$$
where $\mathrm{SatAgg}$ is an operator that aggregates the truth values of the formulas in $\mathcal{K}$ (if there are more than one formula).

To evaluate the grounding of each formula, one has to define the grounding of the non-logical symbols and of the operators.

```python
class ModelC(tf.keras.Model):
    def __init__(self):
        super(ModelC, self).__init__()
        self.dense1 = tf.keras.layers.Dense(5, activation=tf.nn.elu)
        self.dense2 = tf.keras.layers.Dense(5, activation=tf.nn.elu)
        self.dense3 = tf.keras.layers.Dense(2, activation=tf.nn.softmax)

    def call(self, inputs):
        """inputs[0]: point, inputs[1]: onehot label"""
        x, label = inputs[0], inputs[1]
        x = self.dense1(x)
        x = self.dense2(x)
        prob = self.dense3(x)
        return tf.math.reduce_sum(prob*label,axis=1)

C = ltn.Predicate(ModelC())
```

    Metal device set to: Apple M1

    systemMemory: 16.00 GB
    maxCacheSize: 5.33 GB

    2021-09-24 17:12:13.181397: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
    2021-09-24 17:12:13.181501: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)

```python
x1 = ltn.Variable("x1",points)
x2 = ltn.Variable("x2",points)
a = ltn.Constant([3.3,2.5], trainable=False)
b = ltn.Constant([1.3,1.1], trainable=False)
l_a = ltn.Constant([1,0], trainable=False)
l_b = ltn.Constant([0,1], trainable=False)
l = ltn.Variable("l",[[1,0],[0,1]])

Sim = ltn.Predicate.Lambda(
    lambda args: tf.exp(-1.*tf.sqrt(tf.reduce_sum(tf.square(args[0]-args[1]),axis=1)))
)
```

```python
similarities_to_a = Sim([x1,a]).tensor
fig, ax = plt.subplots()
ax.set_xlim(0,4)
ax.set_ylim(0,4)
ax.scatter(points[:,0],points[:,1],color="black")
ax.scatter(a.tensor[0],a.tensor[1],color="blue")
ax.set_title("Illustrating the similarities of each point to a")
for i, sim_to_a in enumerate(similarities_to_a):
    plt.plot([points[i,0],a.tensor[0]],[points[i,1],a.tensor[1]], alpha=sim_to_a.numpy(),color="blue")
```

Notice the operator for equivalence $p \leftrightarrow q$; in LTN, it is simply implemented as $(p \rightarrow q)\land(p \leftarrow q)$ using one operator for conjunction and one operator for implication.

```python
Not = ltn.Wrapper_Connective(ltn.fuzzy_ops.Not_Std())
And = ltn.Wrapper_Connective(ltn.fuzzy_ops.And_Prod())
Or = ltn.Wrapper_Connective(ltn.fuzzy_ops.Or_ProbSum())
Implies = ltn.Wrapper_Connective(ltn.fuzzy_ops.Implies_Reichenbach())
Equiv = ltn.Wrapper_Connective(ltn.fuzzy_ops.Equiv(ltn.fuzzy_ops.And_Prod(),ltn.fuzzy_ops.Implies_Reichenbach()))
Forall = ltn.Wrapper_Quantifier(ltn.fuzzy_ops.Aggreg_pMeanError(p=2),semantics="forall")
Exists = ltn.Wrapper_Quantifier(ltn.fuzzy_ops.Aggreg_pMean(p=6),semantics="exists")
```

If there are several closed formulas in $\mathcal{K}$, their truth values need to be aggregated.
We recommend to use the generalized mean inspired operator `pMeanError`, already used to implement $\forall$.
The hyperparameter again allows flexibility in how strict the formula aggregation is ($p = 1$ corresponds to `mean`; $p \to +\inf$ corresponds to `min`).

The knowledgebase should be written inside of a function that is decorated with `tf.function`. This Tensorflow decorator compiles the function into a callable TensorFlow graph (static).

```python
formula_aggregator = ltn.Wrapper_Formula_Aggregator(ltn.fuzzy_ops.Aggreg_pMeanError(p=2))

@tf.function
def axioms():
    axioms = [
        C([a,l_a]),
        C([b,l_b]),
        Forall(
            [x1,x2,l],
            Implies( Sim([x1,x2]),
                    Equiv(C([x1,l]),C([x2,l]))
                   )
        )
    ]
    kb = formula_aggregator(axioms)
    sat = kb.tensor
    return sat
```

**It is important to always run (forward pass) the knowledgebase once before training, as Tensorflow initializes weights and compiles the graph during the first call.**

```python
axioms()
```

    2021-09-24 17:15:46.561522: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
    2021-09-24 17:15:46.566398: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
    2021-09-24 17:15:46.567351: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.

    <tf.Tensor: shape=(), dtype=float32, numpy=0.4704498>

Eventually, one can write a custom training loop in Tensorflow.

```python
trainable_variables = C.trainable_variables
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)

for epoch in range(2000):
    with tf.GradientTape() as tape:
        loss = 1. - axioms()
    grads = tape.gradient(loss, trainable_variables)
    optimizer.apply_gradients(zip(grads, trainable_variables))
    if epoch%200 == 0:
        print("Epoch %d: Sat Level %.3f"%(epoch, axioms()))
print("Training finished at Epoch %d with Sat Level %.3f"%(epoch, axioms()))
```

    2021-09-24 17:16:07.536407: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
    2021-09-24 17:16:07.619984: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.

    Epoch 0: Sat Level 0.471
    Epoch 200: Sat Level 0.678
    Epoch 400: Sat Level 0.931
    Epoch 600: Sat Level 0.954
    Epoch 800: Sat Level 0.958
    Epoch 1000: Sat Level 0.959
    Epoch 1200: Sat Level 0.959
    Epoch 1400: Sat Level 0.959
    Epoch 1600: Sat Level 0.959
    Epoch 1800: Sat Level 0.959
    Training finished at Epoch 1999 with Sat Level 0.959

After a few epochs, the system has learned to identify samples close to the point $a$ (resp. $b$) as belonging to class $A$ (resp. $B$) based on the rules of the knowledgebase.

```python
fig = plt.figure(figsize=(10,3))
fig.add_subplot(1,2,1)
is_a = C([x1,l_a])
plt.scatter(x1.tensor[:,0],x1.tensor[:,1],c=is_a.tensor.numpy(),vmin=0,vmax=1)
plt.title("C(x,l_a)")
plt.colorbar()
fig.add_subplot(1,2,2)
is_b = C([x1,l_b])
plt.scatter(x1.tensor[:,0],x1.tensor[:,1],c=is_b.tensor.numpy(),vmin=0,vmax=1)
plt.title("C(x,l_b)")
plt.colorbar()
plt.show();
```

## Special Cases

### Variables grounded by batch

When working with batches of data, grounding the variables with different values at each step:
1. Pass the values in arguments to the knowledgebase function,
2. Create the ltn variables within the function.

```python
@tf.function
def axioms(data_x, data_y):
    x = ltn.Variable("x", data_x)
    y = ltn.Variable("y", data_y)
    return Forall([x,y],P([x,y]))

...
for epoch in range(epochs):
    for batch_x, batch_y in dataset:
        with tf.GradientTape() as tape:
            loss_value = 1. - axioms(batch_x, batch_y)
        grads = tape.gradient(loss_value, trainable_variables)
        optimizer.apply_gradients(zip(grads, trainable_variables))
```

### Variables denoting a sequence of trainable constants

When a variable denotes a sequence of trainable constants (embeddings):
1. Do not create the variable outside the scope of `tf.GradientTape()`,
2. Create the variable within the training step function.

```python
c1 = ltn.Constant([2.1,3], trainable=True)
c2 = ltn.Constant([4.5,0.8], trainable=True)
# Do not assign the variable here. Tensorflow would not keep track of the
# gradients between c1/c2 and x during training.
...
@tf.function
def axioms():
    # The assignation must be done within the tf.GradientTape,
    # inside of the training step function.
    x = ltn.Variable.from_constants("x", [c1,c2])
    return Forall(x,P(x))
...
for epoch in range(epochs):
    with tf.GradientTape() as tape:
        loss_value = 1. - axioms() # the training step function is called within the tape.
    grads = tape.gradient(loss_value, trainable_variables)
    optimizer.apply_gradients(zip(grads, trainable_variables))
```


#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "//player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1yV41167Di', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1yV41167Di

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='red')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, n_bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
  plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ################################################################################
  # Students: compute dv, then comment out or remove the next line
  raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = ...

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
# v, spike_times = lif_neuron()
# plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    dv = alpha * exc[i]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

v, spike_times = lif_neuron()
with plt.xkcd():
  plot_neuron_stats(v, spike_times)
```

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
#@title

#@markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
  rate=widgets.IntSlider(10, min=5, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 2: Linear-IF models
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "//player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1iZ4y1u7en', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1iZ4y1u7en

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """

  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ###############################################################################
  # Students: compute dv, then comment out or remove the next line
  raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = ...

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
#v, spike_times = lif_neuron_inh()
#plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

v, spike_times = lif_neuron_inh()

with plt.xkcd():
  plot_neuron_stats(v, spike_times)
```

## Interactive Demo: LIF + inhibition neuron

```python
#@title
#@markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 3: LIF + inhibition
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "//player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1nV41167mS', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1nV41167mS

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).