# Double Machine Learning: Use Cases and Examples

Double Machine Learning (DML) is an algorithm that applies arbitrary machine learning methods
to fit the treatment and response, then uses a linear model to predict the response residuals
from the treatment residuals.

The EconML SDK implements the following DML classes:
* LinearDML: suitable for estimating heterogeneous treatment effects.
* SparseLinearDML: suitable for the case when $W$ is high dimensional vector and both the first stage and second stage estimate are linear.

In ths notebook, we show the performance of the DML on both synthetic data and observational data.

**Notebook contents:**

1. Example usage with single continuous treatment synthetic data
2. Example usage with single binary treatment synthetic data
3. Example usage with multiple continuous treatment synthetic data
4. Example usage with single continuous treatment observational data
5. Example usage with multiple continuous treatment, multiple outcome observational data

```python
import econml
```

```python
## Ignore warnings
import warnings
warnings.filterwarnings('ignore')
```

```python
# Main imports
from econml.dml import DML, LinearDML,SparseLinearDML

# Helper imports
import numpy as np
from itertools import product
from sklearn.linear_model import Lasso, LassoCV, LogisticRegression, LogisticRegressionCV,LinearRegression,MultiTaskElasticNet,MultiTaskElasticNetCV
from sklearn.ensemble import RandomForestRegressor,RandomForestClassifier
from sklearn.preprocessing import PolynomialFeatures
import matplotlib.pyplot as plt
import matplotlib
from sklearn.model_selection import train_test_split

%matplotlib inline
```

## 1. Example Usage with Single Continuous Treatment Synthetic Data and Model Selection

### 1.1. DGP
We use the data generating process (DGP) from [here](https://arxiv.org/abs/1806.03467). The DGP is described by the following equations:

\begin{align}
T =& \langle W, \beta\rangle + \eta, & \;\eta \sim \text{Uniform}(-1, 1)\\
Y =& T\cdot \theta(X) + \langle W, \gamma\rangle + \epsilon, &\; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim& \text{Normal}(0,\, I_{n_w})\\
X \sim& \text{Uniform}(0,1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders and $\beta, \gamma$ have high sparsity.

For this DGP,
\begin{align}
\theta(x) = \exp(2\cdot x_1).
\end{align}

```python
# Treatment effect function
def exp_te(x):
    return np.exp(2*x[0])
```

```python
# DGP constants
np.random.seed(123)
n = 2000
n_w = 30
support_size = 5
n_x = 1
# Outcome support
support_Y = np.random.choice(np.arange(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n: np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE = np.array([exp_te(x_i) for x_i in X])
T = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
Y = TE * T + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)

Y_train, Y_val, T_train, T_val, X_train, X_val, W_train, W_val = train_test_split(Y, T, X, W, test_size=.2)
# Generate test data
X_test = np.array(list(product(np.arange(0, 1, 0.01), repeat=n_x)))
```

### 1.2. Train Estimator
We train models in three different ways, and compare their performance.

#### 1.2.1. Default Setting

```python
est = LinearDML(model_y=RandomForestRegressor(),
                model_t=RandomForestRegressor(),
                random_state=123)
est.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred = est.effect(X_test)
```

#### 1.2.2. Polynomial Features for Heterogeneity

```python
est1 = SparseLinearDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestRegressor(),
                       featurizer=PolynomialFeatures(degree=3),
                       random_state=123)
est1.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred1=est1.effect(X_test)
```

#### 1.2.3. Polynomial Features with regularization

```python
est2 = DML(model_y=RandomForestRegressor(),
           model_t=RandomForestRegressor(),
           model_final=Lasso(alpha=0.1, fit_intercept=False),
           featurizer=PolynomialFeatures(degree=10),
           random_state=123)
est2.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred2=est2.effect(X_test)
```

#### 1.2.4 Random Forest Final Stage

```python
from econml.dml import ForestDML

# One can replace model_y and model_t with any scikit-learn regressor and classifier correspondingly
# as long as it accepts the sample_weight keyword argument at fit time.
est3 = ForestDML(model_y=RandomForestRegressor(),
                 model_t=RandomForestRegressor(),
                 discrete_treatment=False,
                 n_estimators=1000,
                 subsample_fr=.8,
                 min_samples_leaf=10,
                 min_impurity_decrease=0.001,
                 verbose=0, min_weight_fraction_leaf=.01)
est3.fit(Y_train, T_train, X=X_train, W=W_train)
te_pred3 = est3.effect(X_test)
```

```python
est3.feature_importances_
```

    array([1.])

### 1.3. Performance Visualization

```python
plt.figure(figsize=(10,6))
plt.plot(X_test, te_pred, label='DML default')
plt.plot(X_test, te_pred1, label='DML polynomial degree=3')
plt.plot(X_test, te_pred2, label='DML polynomial degree=10 with Lasso')
plt.plot(X_test, te_pred3, label='ForestDML')
expected_te = np.array([exp_te(x_i) for x_i in X_test])
plt.plot(X_test, expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.show()
```

### 1.4. Model selection

For the three different models above, we can use score function to estimate the final model performance. The score is the MSE of the final stage Y residual, which can be seen as a proxy of the MSE of treatment effect.

```python
score={}
score["DML default"] = est.score(Y_val, T_val, X_val, W_val)
score["DML polynomial degree=2"] = est1.score(Y_val, T_val, X_val, W_val)
score["DML polynomial degree=10 with Lasso"] = est2.score(Y_val, T_val, X_val, W_val)
score["ForestDML"] = est3.score(Y_val, T_val, X_val, W_val)
score
```

    {'DML default': 1.815769478666336,
     'DML polynomial degree=2': 1.6951945911574153,
     'DML polynomial degree=10 with Lasso': 2.148988483212333,
     'ForestDML': 1.74374363161912}

```python
print("best model selected by score: ",min(score,key=lambda x: score.get(x)))
```

    best model selected by score:  DML polynomial degree=2

```python
mse_te={}
mse_te["DML default"] = ((expected_te - te_pred)**2).mean()
mse_te["DML polynomial degree=2"] = ((expected_te - te_pred1)**2).mean()
mse_te["DML polynomial degree=10 with Lasso"] = ((expected_te - te_pred2)**2).mean()
mse_te["ForestDML"] = ((expected_te - te_pred3)**2).mean()
mse_te
```

    {'DML default': 0.3565984526892961,
     'DML polynomial degree=2': 0.2153049849895232,
     'DML polynomial degree=10 with Lasso': 0.1966157424558891,
     'ForestDML': 0.11973811235711414}

```python
print("best model selected by MSE of TE: ", min(mse_te, key=lambda x: mse_te.get(x)))
```

    best model selected by MSE of TE:  ForestDML

## 2. Example Usage with Single Binary Treatment Synthetic Data and Confidence Intervals

### 2.1. DGP
We use the following DGP:

\begin{align}
T \sim & \text{Bernoulli}\left(f(W)\right), &\; f(W)=\sigma(\langle W, \beta\rangle + \eta), \;\eta \sim \text{Uniform}(-1, 1)\\
Y = & T\cdot \theta(X) + \langle W, \gamma\rangle + \epsilon, & \; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim & \text{Normal}(0,\, I_{n_w}) & \\
X \sim & \text{Uniform}(0,\, 1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders, $\beta, \gamma$ have high sparsity and $\sigma$ is the sigmoid function.

For this DGP,
\begin{align}
\theta(x) = \exp( 2\cdot x_1 ).
\end{align}

```python
# Treatment effect function
def exp_te(x):
    return np.exp(2 * x[0])# DGP constants

np.random.seed(123)
n = 1000
n_w = 30
support_size = 5
n_x = 4
# Outcome support
support_Y = np.random.choice(range(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n:np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE = np.array([exp_te(x_i) for x_i in X])
# Define treatment
log_odds = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
T_sigmoid = 1/(1 + np.exp(-log_odds))
T = np.array([np.random.binomial(1, p) for p in T_sigmoid])
# Define the outcome
Y = TE * T + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)

# get testing data
X_test = np.random.uniform(0, 1, size=(n, n_x))
X_test[:, 0] = np.linspace(0, 1, n)
```

### 2.2. Train Estimator

```python
est = LinearDML(model_y=RandomForestRegressor(),
                model_t=RandomForestClassifier(min_samples_leaf=10),
                discrete_treatment=True,
                linear_first_stages=False,
                n_splits=6)
est.fit(Y, T, X=X, W=W)
te_pred = est.effect(X_test)
lb, ub = est.effect_interval(X_test, alpha=0.01)
```

```python
est2 = SparseLinearDML(model_y=RandomForestRegressor(),
                       model_t=RandomForestClassifier(min_samples_leaf=10),
                       discrete_treatment=True,
                       featurizer=PolynomialFeatures(degree=2),
                       linear_first_stages=False,
                       n_splits=6)
est2.fit(Y, T, X=X, W=W)
te_pred2 = est2.effect(X_test)
lb2, ub2 = est2.effect_interval(X_test, alpha=0.01)
```

```python
est3 = ForestDML(model_y=RandomForestRegressor(),
                 model_t=RandomForestClassifier(min_samples_leaf=10),
                 discrete_treatment=True,
                 n_estimators=1000,
                 subsample_fr=.8,
                 min_samples_leaf=10,
                 min_impurity_decrease=0.001,
                 verbose=0, min_weight_fraction_leaf=.01,
                 n_crossfit_splits=6)
est3.fit(Y, T, X=X, W=W)
te_pred3 = est3.effect(X_test)
lb3, ub3 = est3.effect_interval(X_test, alpha=0.01)
```

```python
est3.feature_importances_
```

    array([0.89352545, 0.03341642, 0.03505072, 0.0380074 ])

### 2.3. Performance Visualization

```python
expected_te=np.array([exp_te(x_i) for x_i in X_test])
plt.figure(figsize=(16,6))
plt.subplot(1, 3, 1)
plt.plot(X_test[:, 0], te_pred, label='LinearDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb, ub, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.subplot(1, 3, 2)
plt.plot(X_test[:, 0], te_pred2, label='SparseLinearDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb2, ub2, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.subplot(1, 3, 3)
plt.plot(X_test[:, 0], te_pred3, label='ForestDML', alpha=.6)
plt.fill_between(X_test[:, 0], lb3, ub3, alpha=.4)
plt.plot(X_test[:, 0], expected_te, 'b--', label='True effect')
plt.ylabel('Treatment Effect')
plt.xlabel('x')
plt.legend()
plt.show()
```

### 2.4. Other Inferences
#### 2.4.1 Effect Inferences
Other than confidence interval, we could also output other statistical inferences of the effect include standard error, z-test score and p value given each sample $X[i]$.

```python
est.effect_inference(X_test[:10,]).summary_frame(alpha=0.1, value=0, decimals=3)
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      0
      0.381
      0.146
      2.605
      0.009
      0.140
      0.621

      1
      0.409
      0.163
      2.512
      0.012
      0.141
      0.677

      2
      0.308
      0.170
      1.811
      0.070
      0.028
      0.588

      3
      0.416
      0.150
      2.780
      0.005
      0.170
      0.662

      4
      0.562
      0.152
      3.700
      0.000
      0.312
      0.811

      5
      0.538
      0.139
      3.879
      0.000
      0.310
      0.766

      6
      0.436
      0.124
      3.531
      0.000
      0.233
      0.639

      7
      0.589
      0.154
      3.825
      0.000
      0.336
      0.842

      8
      0.446
      0.126
      3.530
      0.000
      0.238
      0.653

      9
      0.487
      0.162
      3.011
      0.003
      0.221
      0.753

We could also get the population inferences given sample $X$.

```python
est.effect_inference(X_test).population_summary(alpha=0.1, value=0, decimals=3, tol=0.001)
```

Uncertainty of Mean Point Estimate

  mean_point	 stderr_mean	  zstat	 pvalue	 ci_mean_lower	 ci_mean_upper

     3.369	      0.14	     24.056	   0.0	      3.138	         3.599

Distribution of Point Estimate

  std_point	 pct_point_lower	 pct_point_upper

    1.724	        0.691	           6.067

Total Variance of Point Estimate

  stderr_point	 ci_point_lower	 ci_point_upper

      1.73	          0.682	          6.065

Note: The stderr_mean is a conservative upper bound.

#### 2.4.2 Coefficient and Intercept Inferences
We could also get the coefficient and intercept inference for the final model when it's linear.

```python
est.coef__inference().summary_frame()
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      X0
      5.958
      0.227
      26.211
      0.000
      5.584
      6.332

      X1
      -0.058
      0.216
      -0.267
      0.789
      -0.413
      0.297

      X2
      -0.326
      0.218
      -1.491
      0.136
      -0.685
      0.034

      X3
      0.217
      0.211
      1.029
      0.304
      -0.130
      0.564

```python
est.intercept__inference().summary_frame()
```

      point_estimate
      stderr
      zstat
      pvalue
      ci_lower
      ci_upper

      intercept
      0.47
      0.218
      2.154
      0.031
      0.111
      0.829

```python
est.summary()
```

Coefficient Results

   	  point_estimate	 stderr	  zstat	 pvalue	 ci_lower	 ci_upper

  X0	      5.958	      0.227	 26.211	   0.0	    5.584	    6.332

  X1	     -0.058	      0.216	 -0.267	  0.789	  -0.413	    0.297

  X2	     -0.326	      0.218	 -1.491	  0.136	  -0.685	    0.034

  X3	      0.217	      0.211	  1.029	  0.304	   -0.13	    0.564

Intercept Results

      	      point_estimate	 stderr	 zstat	 pvalue	 ci_lower	 ci_upper

  intercept	      0.47	       0.218	 2.154	  0.031	   0.111	    0.829

## 3. Example Usage with Multiple Continuous Treatment Synthetic Data

### 3.1. DGP
We use the data generating process (DGP) from [here](https://arxiv.org/abs/1806.03467), and modify the treatment to generate multiple treatments. The DGP is described by the following equations:

\begin{align}
T =& \langle W, \beta\rangle + \eta, & \;\eta \sim \text{Uniform}(-1, 1)\\
Y =& T\cdot \theta_{1}(X) + T^{2}\cdot \theta_{2}(X) + \langle W, \gamma\rangle + \epsilon, &\; \epsilon \sim \text{Uniform}(-1, 1)\\
W \sim& \text{Normal}(0,\, I_{n_w})\\
X \sim& \text{Uniform}(0,1)^{n_x}
\end{align}

where $W$ is a matrix of high-dimensional confounders and $\beta, \gamma$ have high sparsity.

For this DGP,
\begin{align}
\theta_{1}(x) = \exp(2\cdot x_1)\\
\theta_{2}(x) = x_1^{2}\\
\end{align}

```python
# DGP constants
np.random.seed(123)
n = 6000
n_w = 30
support_size = 5
n_x = 5
# Outcome support
support_Y = np.random.choice(np.arange(n_w), size=support_size, replace=False)
coefs_Y = np.random.uniform(0, 1, size=support_size)
epsilon_sample = lambda n: np.random.uniform(-1, 1, size=n)
# Treatment support
support_T = support_Y
coefs_T = np.random.uniform(0, 1, size=support_size)
eta_sample = lambda n: np.random.uniform(-1, 1, size=n)

# Generate controls, covariates, treatments and outcomes
W = np.random.normal(0, 1, size=(n, n_w))
X = np.random.uniform(0, 1, size=(n, n_x))
# Heterogeneous treatment effects
TE1 = np.array([x_i[0] for x_i in X])
TE2 = np.array([x_i[0]**2 for x_i in X]).flatten()
T = np.dot(W[:, support_T], coefs_T) + eta_sample(n)
Y = TE1 * T + TE2 * T**2 + np.dot(W[:, support_Y], coefs_Y) + epsilon_sample(n)
# Generate test data
X_test = np.random.uniform(0, 1, size=(100, n_x))
X_test[:, 0] = np.linspace(0, 1, 100)
```

### 3.2. Train Estimator

```python
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor
from sklearn.linear_model import ElasticNetCV
est = LinearDML(model_y=GradientBoostingRegressor(n_estimators=100, max_depth=3, min_samples_leaf=20),
                model_t=MultiOutputRegressor(GradientBoostingRegressor(n_estimators=100,
                                                                       max_depth=3,
                                                                       min_samples_leaf=20)),
                featurizer=PolynomialFeatures(degree=2, include_bias=False),
                linear_first_stages=False,
                n_splits=5)
```

```python
T = T.reshape(-1,1)
est.fit(Y, np.concatenate((T, T**2), axis=1), X=X, W=W)
```

    <econml.dml.LinearDML at 0x1da9d64bf08>

```python
te_pred = est.const_marginal_effect(X_test)
```

```python
lb, ub = est.const_marginal_effect_interval(X_test, alpha=0.01)
```

### 3.3. Performance Visualization

```python
plt.figure(figsize=(10,6))
plt.plot(X_test[:, 0], te_pred[:, 0], label='DML estimate1')
plt.fill_between(X_test[:, 0], lb[:, 0], ub[:, 0], alpha=.4)
plt.plot(X_test[:, 0], te_pred[:, 1], label='DML estimate2')
plt.fill_between(X_test[:, 0], lb[:, 1], ub[:, 1], alpha=.4)
expected_te1 = np.array([x_i[0] for x_i in X_test])
expected_te2=np.array([x_i[0]**2 for x_i in X_test]).flatten()
plt.plot(X_test[:, 0], expected_te1, '--', label='True effect1')
plt.plot(X_test[:, 0], expected_te2, '--', label='True effect2')
plt.ylabel("Treatment Effect")
plt.xlabel("x")
plt.legend()
plt.show()
```

## 4. Example Usage with Single Continuous Treatment Observational Data

We applied our technique to Dominick’s dataset, a popular historical dataset of store-level orange juice prices and sales provided by University of Chicago Booth School of Business.

The dataset is comprised of a large number of covariates $W$, but researchers might only be interested in learning the elasticity of demand as a function of a few variables $x$ such
as income or education.

We applied the `LinearDML` to estimate orange juice price elasticity
as a function of income, and our results, unveil the natural phenomenon that lower income consumers are more price-sensitive.

### 4.1. Data

```python
# A few more imports
import os
import pandas as pd
import urllib.request
from sklearn.preprocessing import StandardScaler
```

```python
# Import the data
file_name = "oj_large.csv"

if not os.path.isfile(file_name):
    print("Downloading file (this might take a few seconds)...")
    urllib.request.urlretrieve("https://msalicedatapublic.blob.core.windows.net/datasets/OrangeJuice/oj_large.csv", file_name)
oj_data = pd.read_csv(file_name)
```

```python
oj_data.head()
```

      store
      brand
      week
      logmove
      feat
      price
      AGE60
      EDUC
      ETHNIC
      INCOME
      HHLARGE
      WORKWOM
      HVAL150
      SSTRDIST
      SSTRVOL
      CPDIST5
      CPWVOL5

      0
      2
      tropicana
      40
      9.018695
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      1
      2
      tropicana
      46
      8.723231
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      2
      2
      tropicana
      47
      8.253228
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      3
      2
      tropicana
      48
      8.987197
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

      4
      2
      tropicana
      50
      9.093357
      0
      3.87
      0.232865
      0.248935
      0.11428
      10.553205
      0.103953
      0.303585
      0.463887
      2.110122
      1.142857
      1.92728
      0.376927

```python
# Prepare data
Y = oj_data['logmove'].values
T = np.log(oj_data["price"]).values
scaler = StandardScaler()
W1 = scaler.fit_transform(oj_data[[c for c in oj_data.columns if c not in ['price', 'logmove', 'brand', 'week', 'store','INCOME']]].values)
W2 = pd.get_dummies(oj_data[['brand']]).values
W = np.concatenate([W1, W2], axis=1)
X=scaler.fit_transform(oj_data[['INCOME']].values)
```

```python
## Generate test data
min_income = -1
max_income = 1
delta = (1 - (-1)) / 100
X_test = np.arange(min_income, max_income + delta - 0.001, delta).reshape(-1,1)
```

### 4.2. Train Estimator

```python
est = LinearDML(model_y=RandomForestRegressor(),model_t=RandomForestRegressor())
est.fit(Y, T, X=X, W=W)
te_pred=est.effect(X_test)
```

### 4.3. Performance Visualization

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(10,6))
plt.plot(X_test, te_pred, label="OJ Elasticity")
plt.xlabel(r'Scale(Income)')
plt.ylabel('Orange Juice Elasticity')
plt.legend()
plt.title("Orange Juice Elasticity vs Income")
plt.show()
```

### 4.4. Confidence Intervals

We can also get confidence intervals around our predictions by passing an additional `inference` argument to `fit`.  All estimators support bootstrap intervals, which involves refitting the same estimator repeatedly on subsamples of the original data, but `LinearDML` also supports a more efficient approach which can be achieved by leaving inference set to the default of `'auto'` or by explicitly passing `inference='statsmodels'`.

```python
est.fit(Y, T, X=X, W=W)
te_pred=est.effect(X_test)
te_pred_interval = est.const_marginal_effect_interval(X_test, alpha=0.02)
```

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(10,6))
plt.plot(X_test.flatten(), te_pred, label="OJ Elasticity")
plt.fill_between(X_test.flatten(), te_pred_interval[0], te_pred_interval[1], alpha=.5, label="1-99% CI")
plt.xlabel(r'Scale(Income)')
plt.ylabel('Orange Juice Elasticity')
plt.title("Orange Juice Elasticity vs Income")
plt.legend()
plt.show()
```

## 5. Example Usage with Multiple Continuous Treatment, Multiple Outcome Observational Data

We use the same data, but in this case, we want to fit the demand of multiple brand as a function of the price of each one of them, i.e. fit the matrix of cross price elasticities. It can be done, by simply setting as $Y$ to be the vector of demands and $T$ to be the vector of prices. Then we can obtain the matrix of cross price elasticities.

\begin{align}
Y=[Logmove_{tropicana},Logmove_{minute.maid},Logmove_{dominicks}] \\
T=[Logprice_{tropicana},Logprice_{minute.maid},Logprice_{dominicks}] \\
\end{align}

### 5.1. Data

```python
# Import the data
oj_data = pd.read_csv(file_name)
```

```python
# Prepare data
oj_data['price'] = np.log(oj_data["price"])
# Transform dataset.
# For each store in each week, get a vector of logmove and a vector of logprice for each brand.
# Other features are store specific, will be the same for all brands.
groupbylist = ["store", "week", "AGE60", "EDUC", "ETHNIC", "INCOME",
               "HHLARGE", "WORKWOM", "HVAL150",
               "SSTRDIST", "SSTRVOL", "CPDIST5", "CPWVOL5"]
oj_data1 = pd.pivot_table(oj_data,index=groupbylist,
                          columns=oj_data.groupby(groupbylist).cumcount(),
                          values=['logmove', 'price'],
                          aggfunc='sum').reset_index()
oj_data1.columns = oj_data1.columns.map('{0[0]}{0[1]}'.format)
oj_data1 = oj_data1.rename(index=str,
                           columns={"logmove0": "logmove_T",
                                    "logmove1": "logmove_M",
                                    "logmove2":"logmove_D",
                                    "price0":"price_T",
                                    "price1":"price_M",
                                    "price2":"price_D"})

# Define Y,T,X,W
Y = oj_data1[['logmove_T', "logmove_M", "logmove_D"]].values
T = oj_data1[['price_T', "price_M", "price_D"]].values
scaler = StandardScaler()
W=scaler.fit_transform(oj_data1[[c for c in groupbylist if c not in ['week', 'store', 'INCOME']]].values)
X=scaler.fit_transform(oj_data1[['INCOME']].values)
```

```python
## Generate test data
min_income = -1
max_income = 1
delta = (1 - (-1)) / 100
X_test = np.arange(min_income, max_income + delta - 0.001, delta).reshape(-1, 1)
```

### 5.2. Train Estimator

```python
est = LinearDML(model_y=MultiTaskElasticNetCV(cv=3, tol=1, selection='random'),
                model_t=MultiTaskElasticNetCV(cv=3),
                featurizer=PolynomialFeatures(1),
                linear_first_stages=True)
est.fit(Y, T, X=X, W=W)
te_pred = est.const_marginal_effect(X_test)
```

### 5.3. Performance Visualization

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(18, 10))
dic={0:"Tropicana", 1:"Minute.maid", 2:"Dominicks"}
for i in range(3):
    for j in range(3):
        plt.subplot(3, 3, 3 * i + j + 1)
        plt.plot(X_test, te_pred[:, i, j],
                 color="C{}".format(str(3 * i + j)),
                 label="OJ Elasticity {} to {}".format(dic[j], dic[i]))
        plt.xlabel(r'Scale(Income)')
        plt.ylabel('Orange Juice Elasticity')
        plt.legend()
plt.suptitle("Orange Juice Elasticity vs Income", fontsize=16)
plt.show()
```

**Findings**: Look at the diagonal of the matrix, the TE of OJ prices are always negative to the sales across all the brand, but people with higher income are less price-sensitive. By contrast, for the non-diagonal of the matrix, the TE of prices for other brands are always positive to the sales for that brand, the TE is affected by income in different ways for different competitors. In addition, compare to previous plot, the negative TE of OJ prices for each brand are all larger than the TE considering all brand together, which means we would have underestimated the effect of price changes on demand.

### 5.4. Confidence Intervals

```python
est.fit(Y, T, X=X, W=W)
te_pred = est.const_marginal_effect(X_test)
te_pred_interval = est.const_marginal_effect_interval(X_test, alpha=0.02)
```

```python
# Plot Oranje Juice elasticity as a function of income
plt.figure(figsize=(18, 10))
dic={0:"Tropicana", 1:"Minute.maid", 2:"Dominicks"}
for i in range(3):
    for j in range(3):
        plt.subplot(3, 3, 3 * i + j + 1)
        plt.plot(X_test, te_pred[:, i, j],
                 color="C{}".format(str(3 * i + j)),
                 label="OJ Elasticity {} to {}".format(dic[j], dic[i]))
        plt.fill_between(X_test.flatten(), te_pred_interval[0][:, i, j],te_pred_interval[1][:, i,j], color="C{}".format(str(3*i+j)),alpha=.5, label="1-99% CI")
        plt.xlabel(r'Scale(Income)')
        plt.ylabel('Orange Juice Elasticity')
        plt.legend()
plt.suptitle("Orange Juice Elasticity vs Income",fontsize=16)
plt.show()
```

notebooks/Double Machine Learning Examples.ipynb


# Plotting with Matplotlib

## Prepare for action

```
import numpy as np
import scipy as sp
import sympy

# Pylab combines the pyplot functionality (for plotting) with the numpy
# functionality (for mathematics and for working with arrays) in a single namespace
# aims to provide a closer MATLAB feel (the easy way). Note that this approach
# should only be used when doing some interactive quick and dirty data inspection.
# DO NOT USE THIS FOR SCRIPTS
#from pylab import *

# the convienient Matplotlib plotting interface pyplot (the tidy/right way)
# use this for building scripts. The examples here will all use pyplot.
import matplotlib.pyplot as plt

# for using the matplotlib API directly (the hard and verbose way)
# use this when building applications, and/or backends
import matplotlib as mpl
```

How would you like the IPython notebook show your plots? In order to use the
matplotlib IPython magic youre IPython notebook should be launched as

    ipython notebook --matplotlib=inline

Make plots appear as a pop up window, chose the backend: 'gtk', 'inline', 'osx', 'qt', 'qt4', 'tk', 'wx'

    %matplotlib qt

or inline the notebook (no panning, zooming through the plot). Not working in IPython 0.x

    %matplotib inline

```
# activate pop up plots
#%matplotlib qt
# or change to inline plots
%matplotlib inline
```

    ERROR: Line magic function `%matplotlib` not found.

### Matplotlib documentation

Finding your own way (aka RTFM). Hint: there is search box available!

* http://matplotlib.org/contents.html

The Matplotlib API docs:

* http://matplotlib.org/api/index.html

Pyplot, object oriented plotting:

* http://matplotlib.org/api/pyplot_api.html
* http://matplotlib.org/api/pyplot_summary.html

Extensive gallery with examples:

* http://matplotlib.org/gallery.html

### Tutorials for those who want to start playing

If reading manuals is too much for you, there is a very good tutorial available here:

* http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-4-Matplotlib.ipynb

Note that this tutorial uses

    from pylab import *

which is usually not adviced in more advanced script environments. When using

    import matplotlib.pyplot as plt

you need to preceed all plotting commands as used in the above tutorial with

    plt.

Give me more!

[EuroScipy 2012 Matlotlib tutorial](http://www.loria.fr/~rougier/teaching/matplotlib/). Note that here the author uses ```from pylab import * ```. When using ```import matplotliblib.pyplot as plt``` the plotting commands need to be proceeded with ```plt.```

## Plotting template starting point

```
# some sample data
x = np.arange(-10,10,0.1)
```

To change the default plot configuration values.

```
page_width_cm = 13
dpi = 200
inch = 2.54 # inch in cm
# setting global plot configuration using the RC configuration style
plt.rc('font', family='serif')
plt.rc('xtick', labelsize=12) # tick labels
plt.rc('ytick', labelsize=20) # tick labels
plt.rc('axes', labelsize=20)  # axes labels
# If you don’t need LaTeX, don’t use it. It is slower to plot, and text
# looks just fine without. If you need it, e.g. for symbols, then use it.
#plt.rc('text', usetex=True) #<- P-E: Doesn't work on my Mac
```

```
# create a figure instance, note that figure size is given in inches!
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8,6))
# set the big title (note aligment relative to figure)
fig.suptitle("suptitle 16, figure alignment", fontsize=16)

# actual plotting
ax.plot(x, x**2, label="label 12")

# set axes title (note aligment relative to axes)
ax.set_title("title 14, axes alignment", fontsize=14)

# axes labels
ax.set_xlabel('xlabel 12')
ax.set_ylabel(r'$y_{\alpha}$ 12', fontsize=22)

# legend
ax.legend(fontsize=12, loc="best")

# saving the figure in different formats
fig.savefig('figure-%03i.png' % dpi, dpi=dpi)
fig.savefig('figure.svg')
fig.savefig('figure.eps')
```

```
# following steps are only relevant when using figures as pop up windows (with %matplotlib qt)
# to update a figure with has been modified
fig.canvas.draw()
# show a figure
fig.show()
```

## Exercise

The current section is about you trying to figure out how to do several plotting features. You should use the previously mentioned resources to find how to do that. In many cases, google is your friend!

* add a grid to the plot

```
plt.plot(x,x**2)
#Write code to show grid in plot here
plt.grid()
```

* change the location of the legend to different places

```
plt.plot(x,x**2, label="label 12")
plt.legend(fontsize=12, loc="upper center")
plt.grid()
```

* find a way to control the line type and color, marker type and color, control the frequency of the marks (`markevery`). See plot options at: http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot

```
plt.plot(x,x**2,'--',color="black",marker=">",markerfacecolor="red",markeredgecolor="black",markevery=5,markersize=10,label="F(x)=x^2")
plt.legend(fontsize=12, loc="upper center")
plt.grid()
```

* add different sub-plots

```
x = np.linspace(0, 1 * np.pi, 120)
y = np.tanh(x ** 1.5)

f, axarr = plt.subplots(2, 2, figsize=(8,6))
axarr[0, 0].plot(x, y,"--",color="red")
axarr[0, 0].set_title('Axis [0,0]')
axarr[0, 1].plot(x, y,"o",color="red")
axarr[0, 1].set_title('Axis [0,1]')
axarr[1, 0].plot(x, y ** 2)
axarr[1, 0].set_title('Axis [1,0]')
axarr[1, 1].scatter(x, y ** 2)
axarr[1, 1].set_title('Axis [1,1]')

#Hides x ticks for top plots and y ticks for right plots
plt.setp([a.get_xticklabels() for a in axarr[0, :]], visible=False);
plt.setp([a.get_yticklabels() for a in axarr[:, 1]], visible=False);

```

* size the figure such that when included on an A4 page the fonts are given in their true size

```
figure(num=None, figsize=(12, 10), dpi=80, facecolor='w', edgecolor='k')
plt.plot(x,exp(x)*cos(x),'-',color="red",marker=">",markerfacecolor="black",markeredgecolor="black",markevery=4,markersize=14,label="F(x)=cos(x)e^x")
plt.legend(fontsize=16, loc="upper right")
plt.grid()
```

* make a contour plot

```
X, Y = np.meshgrid(x,x)
Z=np.cos(X+Y)*np.exp(Y);

plt.contourf(X,Y,Z)
plt.colorbar()
plt.title('F(x,y)=cos(x+y)e^(y)')
plt.grid()
```

* use twinx() to create a second axis on the right for the second plot

```
plt.plot(x,x**2)
plt.plot(x,x**4, 'r')
plt.grid()
ax=twinx()
ax.set_ylim(0,50)

```

* add horizontal and vertical lines using axvline(), axhline()

```
plt.plot(x,x**2)
axvline(x=0.25,ymin=0,ymax=10,color="black")
axhline(y=5,xmin=0,xmax=3.5,color="black")
plt.grid()
```

* autoformat dates for nice printing on the x-axis using fig.autofmt_xdate()

```
import datetime
dates = np.array([datetime.datetime.now() + datetime.timedelta(days=i) for i in xrange(24)])
fig, ax = plt.subplots(nrows=1, ncols=1)
plt.plot(x,x**2)
plt.grid()
fig.autofmt_xdate(bottom=0.1, rotation=90, ha="left")
```

## Advanced exercises

We are going to play a bit with regression

* Create a vector x of equally spaced number between $x \in [0, 5\pi]$ of 1000 points (keyword: linspace)

```
x=linspace(0,5*pi,1000)
```

* create a vector y, so that y=sin(x) with some random noise

```
y=sin(x)+np.random.normal(-0.2,0.2,1000)
```

* plot it like this:

```
plt.plot(x,y,'o',color="black",label="F(x)=sin(x)+E")
plt.legend(fontsize=12, loc="upper right")
plt.grid()
```

Try to do a polynomial fit on y(x) with different polynomial degree (Use numpy.polyfit to obtain coefficients)

Plot it like this (use np.poly1d(coef)(x) to plot polynomials)

```
#Polynomial fits:
PF0 = np.polyfit(x,y,0)
PF1 = np.polyfit(x,y,1)
PF2 = np.polyfit(x,y,2)
PF3 = np.polyfit(x,y,3)
PF4 = np.polyfit(x,y,4)
PF5 = np.polyfit(x,y,5)
PF6 = np.polyfit(x,y,6)
PF7 = np.polyfit(x,y,7)
PF8 = np.polyfit(x,y,8)
PF9 = np.polyfit(x,y,9)

#Polynomial values in x∈[0,5π]
PV0 = np.poly1d(PF0)
PV1 = np.poly1d(PF1)
PV2 = np.poly1d(PF2)
PV3 = np.poly1d(PF3)
PV4 = np.poly1d(PF4)
PV5 = np.poly1d(PF5)
PV6 = np.poly1d(PF6)
PV7 = np.poly1d(PF7)
PV8 = np.poly1d(PF8)
PV9 = np.poly1d(PF9)
```

```
#Original plot:
figure(num=None, figsize=(16, 14), dpi=80, facecolor='w', edgecolor='k')
plt.plot(x,y,'o',color="black",label="F(x)=sin(x)+E")
plt.legend(fontsize=12, loc="upper right")
plt.grid()

#Polynomial fits:
plt.plot(x,PV0(x), linewidth=3,label="deg=0")
plt.plot(x,PV1(x), linewidth=3,label="deg=1")
plt.plot(x,PV2(x), linewidth=3,label="deg=2")
plt.plot(x,PV3(x), linewidth=3,label="deg=3")
plt.plot(x,PV4(x), linewidth=3,label="deg=4")
plt.plot(x,PV5(x), linewidth=3,label="deg=5")
plt.plot(x,PV6(x), linewidth=3,label="deg=6")
plt.plot(x,PV7(x), linewidth=3,label="deg=7")
plt.plot(x,PV8(x), linewidth=3,label="deg=8")
plt.plot(x,PV9(x), linewidth=3,label="deg=9")
plt.legend(fontsize=12,loc="best")

title("Polynomial Fit", fontsize=25)

```

```

```


```
from __future__ import division
```

```
from IPython.core.display import HTML
def css_styling():
    styles = open("styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Cambria,Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family:"Aurulent Sans Mono","Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }

    BODY {background-color:#FBF5E6}

</style>

```
import json
s = json.load( open("styles/bmh_matplotlibrc.json") )
matplotlib.rcParams.update(s)
```

```
%qtconsole
```

## Introduction

We want to generate samples of a given density, $f(x)$. In this case, we can assume we already have a reliable way to generate samples from a uniform distribution, $\mathcal{U}[0,1]$. How do we know a random sample ($v$) comes from the $f(x)$ distribution? One way to think about it is that a histogram of samples must approximate $f(x)$. This means that

$$ \mathbb{P}( v \in N_{\Delta}(x) )  = f(x) \Delta x$$

which says that the probability that a sample is in some $\Delta$ neighborhood of x is approximately $f(x)\Delta x$.

Let's consider how to create these samples for both discrete and continuous random variables.

## Inverse CDF Method for Discrete Variables

Suppose we have a probability mass function,

$$ f(x) = \sum p_i \delta(x-x_i) $$

where $p_i$ is the probability *mass* at the point $x_i$. For example, for a fair six-sided die, we have

$$ f(x) = \frac{1}{6} \sum_{i=1}^6 \delta(x-i) $$

This generates the corresponding cumulative mass function,

$$ F(x) = \frac{1}{6} \sum_{i=1}^6 U(x-i) $$

where $U$ is the unit-step function. The next block of code develops this using `sympy`.

```
import sympy as S
x=S.symbols('x')
F = sum([S.Heaviside(x-i+1) for i in range(1,7)])/6 # offset to satisfy sympy definition
S.plot(F,(x,0,6),ylabel='CDF(x)',xlabel='x');
```

Now, we want a random number generator that outputs an element of $\left \{  1,2,3,..,6\right \} $ with equal probability. We can generate a uniform random variable and think of it as picking a point on the y-axis of the plot above. Then, all we do is pick the corresponding x-axis value as the output. Let's do this in the next code block

```
invF=S.Piecewise((1,0<x<=1/6),    # if uniform sample between 0 and 1/6, choose die-side labeled 1
                 (2,1/6<x<=2/6),
                 (3,2/6<x<=3/6),
                 (4,3/6<x<=4/6),
                 (5,4/6<x<=5/6),
                 (6,5/6<x<=1))

samples=array([invF.subs(x,i) for i in  rand(500)])
hist(samples,bins=[1,2,3,4,5,6,7],normed=1,align='left');
axis(ymax=2/6.)
title('Estimated PMF of Fair Six-Sided Die');
```

For comparison, here is the estimated CDF compared with $F$. You can trying using more or fewer samples in `rand()` to see how this changes.

```
bar(-.5+arange(1,7),cumsum([mean(samples==i) for i in range(1,7)]),alpha=.3,label='ECDF')   # estimated CDF
bar(-.25+arange(1,7),[F.subs(x,i).evalf() for i in range(1,7)],fc='g',alpha=.3,width=.5,label='CDF') # target CDF
legend(loc=0)
title('Estimated CDF for fair six-sided die',fontsize=18)
ylabel('Frequency',fontsize=18)
xlabel('Die face index',fontsize=18)
axis(ymax=1.1)
```

By repeating the same argument, we can construct an *unfair* six-sided die as

```
p = array([1,2,9,2,1,1]) # the faces 2,4 and more probable than 1,5,6 and 3 is the most probable
p = p/sum(p) # normalize to 1
Fu = sum([S.Heaviside(x-i+1)*p[i-1] for i in range(1,7)])
S.plot(Fu,(x,0,6),ylabel='CDF(x)',xlabel='x',title='Unfair 6-sided Die');
```

In the next block, we automate computing the inverse.

```
cp=cumsum([0]+p.tolist()) # need to find edges on vertical axis, add [0] to get left edge
invFu=S.Piecewise(*[(i,j<x<=k) for i,j,k in zip(range(1,7),cp[:-1],cp[1:])])
hist([invFu.subs(x,i) for i in  rand(1000)],bins=[1,2,3,4,5,6,7],normed=1,align='left');
axis(ymax=1.)
title('Estimated PMF of Unfair Six-Sided Die');
```

## Inverse CDF Method for Continuous Variables

The same idea applies to continuous random variables, but now we have to use squeeze the  intervals down to individual points. In the example above, our inverse function was a piecewise function that operated on uniform random samples. In this case, the piecewise function collapses to a continuous inverse function.

We want to generate random samples for a CDF $F$ that is invertible. The criterion for generating an appropriate sample is the following,

$$ \mathbb{P}(F(x) < v < F(x+\Delta x)) =  F(x+\Delta x) - F(x) = \int_x^{x+\Delta x} f(u) du \approx  f(x) \Delta x$$

which is saying that the probability that the sample $v$ is contained in a $\Delta x$ interval is approximately equal to the density function, $f(x) \Delta x$ at that point. The trick is to use a uniform random sample ($u$) and an invertible CDF $F(x)$ to construct these samples.

Note that for a uniform random variable $u \sim \mathcal{U}[0,1]$, we have,

$$ \mathbb{P}(x < F^{-1}(u) < x+\Delta x) =\mathbb{P}(F(x) < u < F(x+\Delta x)) =  F(x+\Delta x) - F(x) = \int_x^{x+\Delta x} f(p) dp \approx  f(x) \Delta x$$

This means that $ v=F^{-1}(u) \sim f $, which is what we were after. Let's try this with the exponential distribution,

$$ f_{\alpha}(x) = \alpha\exp(-\alpha x) $$

with the following CDF,

$$ F(x) = 1-\exp(-\alpha x )$$

and corresponding inverse,

$$ F^{-1}(u)  = \frac{1}{\alpha}\ln \frac{1}{(1-u)}$$

```
import scipy.stats

alpha = 1.
nsamp = 1000

u=scipy.stats.uniform(0,1)
x=scipy.stats.expon(alpha)

Finv=lambda u: 1/alpha*log(1/(1-u))
#Finv=lambda u: 1+1/alpha*log(1/(1-u)) # shift over to correct

fig,ax = subplots()

xhat = array(map(Finv,u.rvs(nsamp)))
# exponential distrib samples by inverse method
ax.hist(xhat,normed=1,bins=30,alpha=0.3,label='inverse method')
# exponential distrib samples by scipy.stats
xrvs = x.rvs(nsamp)
xe = linspace(0,xrvs.max(),100)
ax.hist(xrvs,bins=30,normed=1,alpha=0.3,label='scipy.stats')
# exponential theoretical density function
ax.plot(xe,x.pdf(xe),'r-',label='theoretical')
ax.set_xticks(range(int(xrvs.max())))
ax.axis(ymax=1.05)
ax.set_title('Inverse CDF for Exponential Distribution')
ax.set_ylabel('pdf(x)',fontsize=18)
ax.set_xlabel('x',fontsize=18)
ax.legend();
```

The reason the histogram is shifted over is because we lose knowledge of a constant shift in $v$ because $1-v$ is also uniformly distributed between $\mathcal{U}[0,1]$. This means we have to change the inverse function to the following:

$$ F^{-1}(u)  = 1+\frac{1}{\alpha}\ln \frac{1}{(1-u)}$$

## Rejection Method

In some cases, you may not be able to invert for the CDF. The `rejection` method can handle this situation. The idea is to pick a uniform ($\mathcal{U}[a,b]$) random variable $u_1$ and $u_2$ so that

$$ \mathbb{P}\left( u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{f(u_1)}{M} \right) \hspace{0.5em} \approx \frac{\Delta x}{b-a} \frac{f(u_1)}{M} $$

where we take $u_1=x$ and $f(x) < M $. The only job of the $M$ variable is to scale down the $f(x)$ so that the $u_2$ variable can span the range.  The *efficiency* of this method is the probability of not rejecting $u_1$ which comes from integrating out the above approximation.

$$ \int dx \frac{f(x)}{M} \hspace{0.5em} = \frac{1}{M(b-a)}$$

This means that we don't want an unecessarily large $M$ because that makes it more likely that samples will be discarded. Let's use a density that does not have a continuous inverse.

$$ f(x) = \exp\left(-\frac{(x-1)^2}{2x} \right) \hspace{1em}  (x+1)/12 $$

where $x>0$. Note that this does not *exactly* integrate out to one like a good probability density function should, but I probably need to think of a better function because the normalization constant for this is pretty convoluted.

Nevertheless, here it is with its corresponding CDF.

```
x = linspace(0.001,15,100)
fig,ax=subplots()
f= lambda x: exp(-(x-1)**2/2./x)*(x+1)/12.
fx = f(x)
ax.plot(x,fx,label='$f(x)$')
ax.plot(x,cumsum(fx)*diff(x)[0],'g',label='$F(x)$')
ax.legend(loc=0,fontsize=16);
```

And, following our rejection plan, the following are the simulated random samples of $f$.

```
M=.3 # scale factor
u1 = rand(10000)*15 # uniform random samples scaled out
u2 = rand(10000)    # uniform random samples
idx=where(u2<=f(u1)/M)[0] # rejection criterion
v = u1[idx]

fig,ax=subplots()
ax.hist(v,normed=1,bins=40,alpha=.3)
ax.plot(x,fx,'r',lw=3.,label='$f(x)$')
ax.set_title('Estimated Efficency=%3.1f%%'%(100*len(v)/len(u1)))
ax.legend(fontsize=18)
```

The plot above shows a pretty good match between our target density function, $f$, and our histogram of the samples we just generated using the rejection method.

Conceptually, there's nothing wrong with this result. The problem is the efficiency is low -- we are throwing away too many samples, as shown in the figure below. We need to somehow do better.

```
fig,ax=subplots()
ax.plot(u1,u2,'.',label='rejected',alpha=.3)
ax.plot(u1[idx],u2[idx],'g.',label='accepted',alpha=.3)
ax.legend(fontsize=16)
```

The above shows the samples that were accepted and rejected by this method. The fact that we threw out so many is very inefficient, and we want to do better.

The above argument uses $u_1$ to select along the domain of $f(x)$ and the other $u_2$ uniform random variable  decides whether to accept or not. One idea would be to choose $u_1$ so that $x$ values are coincidentally those that are near the peak of $f(x)$, instead of uniformly anywhere in the domain, especially near the tails, which are low probability anyway. Now, the trick is to find a new density function $g(x)$ to sample from that has a similiar concentration of probability density. One way it to familiarize oneself with the popular density functions that have adjustable parameters and fast random sample generators already. There are lots of places to look and, chances are, there is likely already such a generator for your problem. Otherwise, the [family of $\beta$ densities](http://en.wikipedia.org/wiki/Beta_distribution) is a good place to start looking.

To be explicit, what we want is $u_1 \sim g(x) $ so that, returning to our earlier argument,

$$ \mathbb{P}\left( u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{f(u_1)}{M} \right) \approx g(x) \Delta x \frac{f(u_1)}{M} $$

but this is *not* what we need here. The problem is with the second part of the $\bigwedge$ clause. We need to put something there that will give us something proportional to $f(x)$. Define the following:

$$ h(x) = \frac{f(x)}{g(x)}$$

with corresponding maximum on the domain as $h_{\max}$ and then go back and construct the second part of the clause as

$$ \mathbb{P}\left( u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{h(u_1)}{h_{\max}} \right) \approx g(x) \Delta x \frac{h(u_1)}{h_{\max}} = f(x)/h_{\max} $$

Recall that satisfying this criterion means that $u_1=x$. As before, we can estimate the probability of acceptance of the $u_1$ as $ 1/h_{\max}$.

Now, to construct such a $g(x)$ function. Here,we choose the chi-squared distribution. The following plots the $g(x)$ and $f(x)$ (left plot) and the corresponding $h(x)=f(x)/g(x)$ (right plot). Note that $g(x)$ and $f(x)$ have peaks that almost coincide, which is what we are looking for.

```
ch=scipy.stats.chi2(4) # chi-squared
h = lambda x: f(x)/ch.pdf(x) # h-function

fig,axs=subplots(1,2)
fig.set_size_inches(12,4)
axs[0].plot(x,fx,label='$f(x)$')
axs[0].plot(x,ch.pdf(x),label='$g(x)$')
axs[0].legend(loc=0,fontsize=18)
axs[1].plot(x,h(x))
axs[1].set_title('$h(x)=f(x)/g(x)$',fontsize=18)
```

Now, let's generate  some samples from this $\chi^2$ distribution with the rejection method.

```
hmax=h(x).max()
u1 = ch.rvs(5000) # samples from chi-square distribution
u2 = rand(5000)   # uniform random samples
idx = (u2 <= h(u1)/hmax)  # Rejection criterion

v = u1[idx]  # keep these only

fig,ax=subplots()
ax.hist(v,normed=1,bins=40,alpha=.3)
ax.plot(x,fx,'r',lw=3.,label='$f(x)$')
ax.set_title('Estimated Efficency=%3.1f%%'%(100*len(v)/len(u1)))
ax.legend(fontsize=18)
```

Using the $\chi^2$ distribution with the rejection method results in throwing away less than 10% of the generated samples compared with our prior example where we threw out at least 80% of the generated samples. Obviously, this way is much more computationally efficient and this is the kind of thing we are always looking for.

For completeness, here is the corresponding plot that shows the samples with the corresponding threshold $h(x)/h_{\max}$ that was used to select them.

```
fig,ax=subplots()
ax.plot(u1,u2,'.',label='rejected',alpha=.3)
ax.plot(u1[idx],u2[idx],'g.',label='accepted',alpha=.3)
ax.plot(x,h(x)/hmax,'r',lw=3.,label='$h(x)$')
ax.legend(fontsize=16)
```

## Summary

In this section, we investigated how to generate random samples from a given distribution, beit discrete or continuous. For the continuouse case, the key issue was whether or not the cumulative density function had a continuous inverse. If not, we had to turn to the rejection method, and find an appropriate related density that we could easily sample from to use as part of a rejection threshold. Finding such a function is an art, but many families of probability densities have been studied over the years that already have fast sample-generators.

It is possible to go much deeper with the rejection method, but these involve careful partitioning of the domains and lots of special methoods for separate domains and corner cases.  Nonetheless, all of these advanced techniques are still variations on the same fundamental theme we illustrated here.

## References

[Exploring Monte Carlo Methods by Dunn, 2011](http://books.google.com/books?id=ACTrrQk1UgoC&dq=Exploring+Monte+Carlo+Methods&source=gbs_navlinks_s)

[Continuous univariate distributions by Balakrishnan, 1995](http://books.google.com/books?id=0QzvAAAAMAAJ&dq=continuous+balakrishnan&hl=en&sa=X&ei=vzcuUtqvOYOKjAKr14BI&ved=0CC8Q6AEwAA)


\title{Combinational-Circuit Building Blocks aka medium scale integrated circuit (MSI) in myHDL}
\author{Steven K Armour}
\maketitle

Table of Contents

1  Refs

2  Python Libraries Utilized

3  Multiplexers (mux)

3.1  Shannon’s Expansion Theorem

3.2  2:1 MultiPlexer

3.2.1  myHDL 2:1 MUX Gate Level and Testing

3.2.2  myHDL 2:1 MUX Gate Level HDL Synthesis

3.3  2:1 Multiplexer Behavioral

3.3.1  myHDL 2:1 MUX Behavioral Level and Testing

3.3.2  myHDL 2:1 MUX Behavioral Level HDL Synthesis

3.4  4:1 MUX

3.4.1  !? Insert Digram below

3.4.2  myHDL 4:1 MUX Gate Level and Testing

3.4.3  myHDL 4:1 MUX Gate Level HDL Synthesis

3.5  4:1 Multiplexer Behavioral

3.5.1  myHDL 4:1 MUX Behavioral Level and Testing

3.5.2  myHDL 4:1 MUX Behavioral Level HDL Synthesis

3.6  4:1 Multiplexer Behavioral with bitvectors

3.6.1  How bit vectors work in myHDL and in Verilog/VHDL

3.6.2  Understanding BitVector bit selection in myHDL

3.6.3  myHDL 4:1 MUX Behavioral with BitVecters and Testing

3.6.3.1  !? This needs to be checked

3.6.4  myHDL 4:1 MUX Behavioral with BitVecters HDL Synthesis

3.7  Generic Expressions via MUXs

3.7.1  myHDL Generic Expression via MUXs and Testing

3.7.2  myHDL Generic Expression via MUXs HDL Synthesis

4  Demultiplexers

5  Encoders

6  Decoders

# Refs
@book{brown_vranesic_2014,
place={New York, NY},
edition={3},
title={Fundamentals of digital logic with Verilog design},
publisher={McGraw-Hill},
author={Brown, Stephen and Vranesic, Zvonko G},
year={2014}
},

@book{lameres_2017,
title={Introduction to logic circuits & logic design with Verilog},
publisher={springer},
author={LaMeres, Brock J},
year={2017}
},

@misc{peeker_simple_mux,
url={http://www.xess.com/static/media/pages/peeker_simple_mux.html},
journal={Xess.com},
year={2017}
},

# Python Libraries Utilized

```python
import numpy as np
import pandas as pd
from sympy import *
init_printing()
from myhdl import *
from myhdlpeek import *
import random
from sympy_myhdl_tools import *
pass
#https://github.com/jrjohansson/version_information
%load_ext version_information
%version_information myhdl, myhdlpeek, numpy, pandas, matplotlib, sympy, random
```

Software	Version

Python	3.6.4 64bit [GCC 7.2.0]

IPython	6.2.1

OS	Linux 4.13.0 45 generic x86_64 with debian stretch sid

myhdl	0.10

myhdlpeek	0.0.6

numpy	1.13.3

pandas	0.21.1

matplotlib	2.1.1

sympy	1.1.1

random	The 'random' distribution was not found and is required by the application

Sat Jul 07 01:20:05 2018 MDT

# Multiplexers (mux)
a junction switch between one of n inputs to a single output; equivalent to a "if" or "case" statement

let $Z$ be its output $m_k$ the minterms of the controls to the mux and $I_k$ be the input feeds to the mux; then the expression for the mux in terms of boolean algebra becomes
$$Z=\sum^{2^k-1}_{k=0} m_k \cdot I_k= \text{OR}(m_k \& I_k) $$

## Shannon’s Expansion Theorem
The above is Shannon's theorem

it can be written more sincintly as:
$$f(x_1, x_2, ..., x_n)=\bar{x_1}f(0, x_2, ..., x_n)+x_1 f(x_1, x_2, ..., x_n)$$
and then each $f(0, x_2, ..., x_n)$ \& $f(x_1, x_2, ..., x_n)$ is broken down as the above till the maximum  number of control statement and minim inputs are needed

```python
def shannon_exspanson(f, term):
    """
    f is not a full equation
    """
    cof0=simplify(f.subs(term, 0)); cof1=simplify(f.subs(term, 1))
    return ((~term & cof0 | (term & cof1))), cof0, cof1
```

 ## 2:1 MultiPlexer

```python
sel, x_1in, x_2in=symbols('sel, x_1in, x_2in')
```

let $f(m_1, m_2, m_3)$ be the total set of minterms for a 3-bit then let $m_1$ be designated the select terms then by shannon's theorem states
$$f(m_1, m_2, m_3)=\bar{m_1} \cdot f_1'(0, m_2, m_3)+m_1 \cdot f_1(1, m_2, m_3)$$
in other words we want select the two subset of the f where $m_1$ is 1 or 0 and call thouse two subsets $f_1'$, $f_1$

```python
x_1in, x_2in, sel=symbols('x_1in, x_2in, sel')
```

$$f(m_1, m_2, m_3)$$

```python
ConversionTable=pd.DataFrame()
Terms=[bin(i, 3) for i in np.arange(0, 2**3)]
ConversionTable['sel']=[int(j[0]) for j in Terms]
ConversionTable['x_1in']=[int(j[1]) for j in Terms]
ConversionTable['x_2in']=[int(j[2]) for j in Terms]
#this is shannos theorm
ConversionTable['f']=list(ConversionTable.loc[ConversionTable['sel'] == 0]['x_1in'])+list(ConversionTable.loc[ConversionTable['sel'] == 1]['x_2in'])

ConversionTable.index.name='MinMaxTerm'
ConversionTable
```

      sel
      x_1in
      x_2in
      f

      MinMaxTerm

      0
      0
      0
      0
      0

      1
      0
      0
      1
      0

      2
      0
      1
      0
      1

      3
      0
      1
      1
      1

      4
      1
      0
      0
      0

      5
      1
      0
      1
      1

      6
      1
      1
      0
      0

      7
      1
      1
      1
      1

```python
POS=list(ConversionTable.loc[ConversionTable['f'] == 0].index)
SOP=list(ConversionTable.loc[ConversionTable['f'] == 1].index)
f"POS: {POS}, SOP:{SOP}"
```

    'POS: [0, 1, 4, 6], SOP:[2, 3, 5, 7]'

```python
f, _=POS_SOPformCalcater([sel, x_1in, x_2in], SOP, POS)
f
```

```python
a, b, c=shannon_exspanson(f, sel)
f,'= via shannaon', a
```

    ((sel & x_2in) | (x_1in & ~sel),
     '= via shannaon',
     (sel & x_2in) | (x_1in & ~sel))

$$\bar{m_1} \cdot f_1'(0, m_2, m_3)$$

```python
m1bar_f0=~sel&x_1in; m1bar_f0
```

```python
f0Table=ConversionTable.loc[ConversionTable['sel'] == 0].copy()
f0Table['f0']=[m1bar_f0.subs({sel:i, x_1in:j}) for i, j in zip(f0Table['sel'], f0Table['x_1in'])]
f0Table
```

      sel
      x_1in
      x_2in
      f
      f0

      MinMaxTerm

      0
      0
      0
      0
      0
      0

      1
      0
      0
      1
      0
      0

      2
      0
      1
      0
      1
      1

      3
      0
      1
      1
      1
      1

$$m_1 \cdot f_1(1, m_2, m_3)$$

```python
m1_f1=sel&x_2in; m1_f1
```

```python
f1Table=ConversionTable.loc[ConversionTable['sel'] == 1].copy()
f1Table['f1']=[m1_f1.subs({sel:i, x_2in:j}) for i, j in zip(f1Table['sel'], f1Table['x_2in'])]
f1Table
```

      sel
      x_1in
      x_2in
      f
      f1

      MinMaxTerm

      4
      1
      0
      0
      0
      0

      5
      1
      0
      1
      1
      1

      6
      1
      1
      0
      0
      0

      7
      1
      1
      1
      1
      1

and since this is the lowest order mux this case use of shannon's theorem is kind of trivial

### myHDL 2:1 MUX Gate Level and Testing

```python
@block
def mux21_gates(sel, x_1in, x_2in,  f_out):
    @always_comb
    def logic():
        f_out.next=(sel and x_2in) or (x_1in and not sel)
    return logic
```

```python
Peeker.clear()

sel, x_1in, x_2in,  f_out=[Signal(bool(0)) for _ in range(4)]
Peeker(sel, 'sel'); Peeker(x_1in, 'x_1in'); Peeker(x_2in, 'x_2in')
Peeker(f_out, 'f_out')

DUT=mux21_gates(sel, x_1in, x_2in,  f_out)

inputs=[sel, x_1in, x_2in]
sim=Simulation(DUT, Combo_TB(inputs), *Peeker.instances()).run()
Peeker.to_wavedrom(start_time=0, stop_time=2*2**len(inputs), tock=True,
                  title='MUX 2:1 gate type simulation',
                  caption=f'after clock cycle {2**len(inputs)-1} ->random input')

```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      sel
      x_1in
      x_2in

      0
      0
      0
      0
      0

      1
      0
      0
      0
      1

      2
      1
      0
      1
      0

      3
      1
      0
      1
      1

      4
      0
      1
      0
      0

      5
      1
      1
      0
      1

      6
      0
      1
      1
      0

      7
      1
      1
      1
      1

      8
      1
      0
      1
      0

      9
      0
      1
      1
      0

      11
      1
      1
      0
      1

      12
      0
      1
      1
      0

      13
      0
      0
      0
      1

      14
      1
      1
      1
      1

### myHDL 2:1 MUX Gate Level HDL Synthesis

```python
DUT.convert()
_=VerilogTextReader('mux21_gates')
```

    ***Verilog modual from mux21_gates.v***

     // File: mux21_gates.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:23:00 2018

    `timescale 1ns/10ps

    module mux21_gates (
        sel,
        x_1in,
        x_2in,
        f_out
    );

    input sel;
    input x_1in;
    input x_2in;
    output f_out;
    wire f_out;

    assign f_out = ((sel && x_2in) || (x_1in && (!sel)));

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL 2:1 MUX Gate level verilog code

however as will be shown doing gate implementation of MUXs is not sustainable in HDL code and this we will have to implement behavioral syntax as follows, thouse the caveat is that this only works for standard MUXs

## 2:1 Multiplexer Behavioral

### myHDL 2:1 MUX Behavioral Level and Testing

```python
@block
def mux21_behavioral(sel, x_1in, x_2in,  f_out):
    @always_comb
    def logic():
            if sel:
                f_out.next=x_1in
            else:
                f_out.next=x_2in
    return logic
```

```python
Peeker.clear()

sel, x_1in, x_2in,  f_out=[Signal(bool(0)) for _ in range(4)]
Peeker(sel, 'sel'); Peeker(x_1in, 'x_1in'); Peeker(x_2in, 'x_2in')
Peeker(f_out, 'f_out')

DUT=mux21_behavioral(sel, x_1in, x_2in,  f_out)

inputs=[sel, x_1in, x_2in]
sim=Simulation(DUT, Combo_TB(inputs), *Peeker.instances()).run()
Peeker.to_wavedrom(start_time=0, stop_time=2*2**len(inputs), tock=True,
                  title='MUX 2:1 behaviroal type simulation',
                  caption=f'after clock cycle {2**len(inputs)-1} ->random input')

```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      sel
      x_1in
      x_2in

      0
      0
      0
      0
      0

      1
      1
      0
      0
      1

      2
      0
      0
      1
      0

      3
      1
      0
      1
      1

      4
      0
      1
      0
      0

      5
      0
      1
      0
      1

      6
      1
      1
      1
      0

      7
      1
      1
      1
      1

      8
      1
      0
      1
      1

      9
      0
      1
      0
      0

      11
      0
      0
      1
      0

      12
      0
      0
      0
      0

      13
      1
      1
      1
      0

      14
      1
      0
      1
      1

      15
      1
      1
      1
      0

### myHDL 2:1 MUX Behavioral Level HDL Synthesis

```python
DUT.convert()
VerilogTextReader('mux21_behavioral');
```

    ***Verilog modual from mux21_behavioral.v***

     // File: mux21_behavioral.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:24:44 2018

    `timescale 1ns/10ps

    module mux21_behavioral (
        sel,
        x_1in,
        x_2in,
        f_out
    );

    input sel;
    input x_1in;
    input x_2in;
    output f_out;
    reg f_out;

    always @(sel, x_1in, x_2in) begin: MUX21_BEHAVIORAL_LOGIC
        if (sel) begin
            f_out = x_1in;
        end
        else begin
            f_out = x_2in;
        end
    end

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL behavioral level 2:1 MUX's verilog code

## 4:1 MUX

If you try to repeat the above using a 4:1 which has four input lines and needs two select lines you can become overwhelmed quickly instead it is easier to use the following diagram to than synthesis the gate level architecture

### !? Insert Digram below

### myHDL 4:1 MUX Gate Level and Testing

```python
@block
def MUX41_gates(sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out):
    @always_comb
    def logic():
        f_out.next=((not sel_1) and (not sel_2) and x_1in) or ((not sel_1) and ( sel_2) and x_2in) or (( sel_1) and (not sel_2) and x_3in) or (( sel_1) and ( sel_2) and x_4in)
    return logic
```

```python
Peeker.clear()

sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out=[Signal(bool(0)) for _ in range(7)]
Peeker(sel_1, 'sel_1'); Peeker(sel_2, 'sel_2');
Peeker(x_1in, 'x_1in'); Peeker(x_2in, 'x_2in'); Peeker(x_3in, 'x_3in'); Peeker(x_4in, 'x_4in')
Peeker(f_out, 'f_out')

DUT=MUX41_gates(sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out)

inputs=[sel_1, sel_2, x_1in, x_2in, x_3in, x_4in]
sim=Simulation(DUT, Combo_TB(inputs), *Peeker.instances()).run()
Peeker.to_wavedrom(start_time=0, stop_time=2*2**len(inputs), tock=True,
                  title='MUX 4:1 gate type simulation',
                  caption=f'after clock cycle {2**len(inputs)-1} ->random input')

```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      sel_1
      sel_2
      x_1in
      x_2in
      x_3in
      x_4in

      0
      0
      0
      0
      0
      0
      0
      0

      1
      0
      0
      0
      0
      0
      0
      1

      2
      0
      0
      0
      0
      0
      1
      0

      3
      0
      0
      0
      0
      0
      1
      1

      4
      0
      0
      0
      0
      1
      0
      0

      5
      0
      0
      0
      0
      1
      0
      1

      6
      0
      0
      0
      0
      1
      1
      0

      7
      0
      0
      0
      0
      1
      1
      1

      8
      1
      0
      0
      1
      0
      0
      0

      9
      1
      0
      0
      1
      0
      0
      1

      10
      1
      0
      0
      1
      0
      1
      0

      11
      1
      0
      0
      1
      0
      1
      1

      12
      1
      0
      0
      1
      1
      0
      0

      13
      1
      0
      0
      1
      1
      0
      1

      14
      1
      0
      0
      1
      1
      1
      0

      15
      1
      0
      0
      1
      1
      1
      1

      16
      0
      0
      1
      0
      0
      0
      0

      17
      0
      0
      1
      0
      0
      0
      1

      18
      0
      0
      1
      0
      0
      1
      0

      19
      0
      0
      1
      0
      0
      1
      1

      20
      1
      0
      1
      0
      1
      0
      0

      21
      1
      0
      1
      0
      1
      0
      1

      22
      1
      0
      1
      0
      1
      1
      0

      23
      1
      0
      1
      0
      1
      1
      1

      24
      0
      0
      1
      1
      0
      0
      0

      25
      0
      0
      1
      1
      0
      0
      1

      26
      0
      0
      1
      1
      0
      1
      0

      27
      0
      0
      1
      1
      0
      1
      1

      28
      1
      0
      1
      1
      1
      0
      0

      29
      1
      0
      1
      1
      1
      0
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      97
      1
      1
      1
      1
      1
      0
      1

      98
      0
      0
      1
      0
      0
      1
      1

      99
      1
      1
      0
      0
      0
      1
      1

      100
      1
      1
      1
      0
      0
      1
      1

      101
      0
      1
      0
      1
      1
      0
      0

      102
      0
      0
      0
      0
      1
      1
      1

      103
      1
      1
      1
      0
      0
      1
      1

      104
      0
      1
      0
      0
      0
      0
      0

      105
      1
      0
      1
      0
      1
      0
      0

      106
      1
      0
      0
      1
      1
      0
      0

      107
      0
      0
      1
      1
      0
      1
      1

      108
      0
      1
      0
      0
      1
      0
      1

      109
      1
      0
      0
      1
      0
      0
      1

      111
      0
      0
      1
      1
      0
      1
      0

      112
      1
      1
      0
      0
      1
      1
      1

      113
      1
      0
      0
      1
      0
      0
      0

      114
      0
      0
      0
      0
      0
      0
      0

      115
      1
      0
      1
      1
      1
      1
      1

      116
      1
      1
      1
      1
      0
      0
      1

      117
      0
      1
      0
      0
      1
      0
      0

      118
      0
      0
      0
      0
      0
      0
      1

      119
      1
      0
      0
      1
      0
      1
      0

      120
      0
      1
      1
      1
      1
      0
      0

      121
      0
      1
      1
      1
      1
      1
      0

      122
      1
      1
      1
      0
      0
      1
      1

      123
      1
      0
      1
      1
      1
      1
      0

      124
      1
      1
      0
      0
      0
      1
      0

      125
      1
      1
      1
      0
      0
      1
      1

      126
      0
      1
      1
      0
      1
      0
      0

      127
      0
      0
      1
      1
      0
      0
      1

123 rows × 7 columns

### myHDL 4:1 MUX Gate Level HDL Synthesis

```python
DUT.convert()
VerilogTextReader('MUX41_gates');
```

    ***Verilog modual from MUX41_gates.v***

     // File: MUX41_gates.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:25:39 2018

    `timescale 1ns/10ps

    module MUX41_gates (
        sel_1,
        sel_2,
        x_1in,
        x_2in,
        x_3in,
        x_4in,
        f_out
    );

    input sel_1;
    input sel_2;
    input x_1in;
    input x_2in;
    input x_3in;
    input x_4in;
    output f_out;
    wire f_out;

    assign f_out = (((!sel_1) && (!sel_2) && x_1in) || ((!sel_1) && sel_2 && x_2in) || (sel_1 && (!sel_2) && x_3in) || (sel_1 && sel_2 && x_4in));

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL 4:1 MUX Gate level verilog code

## 4:1 Multiplexer Behavioral

As one can clearly see this is not sustainable and thus 'if' Statements need to be used via behavioral logic modeling

### myHDL 4:1 MUX Behavioral Level and Testing

```python
@block
def MUX41_behavioral(sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out):
    @always_comb
    def logic():
        if (not sel_1) and (not sel_2):
            f_out.next=x_1in

        elif (not sel_1) and sel_2:
            f_out.next=x_2in

        elif sel_1 and (not sel_2):
            f_out.next=x_3in

        else:
            f_out.next=x_4in

    return logic
```

```python
Peeker.clear()

sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out=[Signal(bool(0)) for _ in range(7)]
Peeker(sel_1, 'sel_1'); Peeker(sel_2, 'sel_2');
Peeker(x_1in, 'x_1in'); Peeker(x_2in, 'x_2in'); Peeker(x_3in, 'x_3in'); Peeker(x_4in, 'x_4in')
Peeker(f_out, 'f_out')

DUT=MUX41_behavioral(sel_1, sel_2, x_1in, x_2in, x_3in, x_4in,  f_out)

inputs=[sel_1, sel_2, x_1in, x_2in, x_3in, x_4in]
sim=Simulation(DUT, Combo_TB(inputs), *Peeker.instances()).run()
Peeker.to_wavedrom(start_time=0, stop_time=2*2**len(inputs), tock=True,
                  title='MUX 4:1 behavioral type simulation',
                  caption=f'after clock cycle {2**len(inputs)-1} ->random input')

```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      sel_1
      sel_2
      x_1in
      x_2in
      x_3in
      x_4in

      0
      0
      0
      0
      0
      0
      0
      0

      1
      0
      0
      0
      0
      0
      0
      1

      2
      0
      0
      0
      0
      0
      1
      0

      3
      0
      0
      0
      0
      0
      1
      1

      4
      0
      0
      0
      0
      1
      0
      0

      5
      0
      0
      0
      0
      1
      0
      1

      6
      0
      0
      0
      0
      1
      1
      0

      7
      0
      0
      0
      0
      1
      1
      1

      8
      1
      0
      0
      1
      0
      0
      0

      9
      1
      0
      0
      1
      0
      0
      1

      10
      1
      0
      0
      1
      0
      1
      0

      11
      1
      0
      0
      1
      0
      1
      1

      12
      1
      0
      0
      1
      1
      0
      0

      13
      1
      0
      0
      1
      1
      0
      1

      14
      1
      0
      0
      1
      1
      1
      0

      15
      1
      0
      0
      1
      1
      1
      1

      16
      0
      0
      1
      0
      0
      0
      0

      17
      0
      0
      1
      0
      0
      0
      1

      18
      0
      0
      1
      0
      0
      1
      0

      19
      0
      0
      1
      0
      0
      1
      1

      20
      1
      0
      1
      0
      1
      0
      0

      21
      1
      0
      1
      0
      1
      0
      1

      22
      1
      0
      1
      0
      1
      1
      0

      23
      1
      0
      1
      0
      1
      1
      1

      24
      0
      0
      1
      1
      0
      0
      0

      25
      0
      0
      1
      1
      0
      0
      1

      26
      0
      0
      1
      1
      0
      1
      0

      27
      0
      0
      1
      1
      0
      1
      1

      28
      1
      0
      1
      1
      1
      0
      0

      29
      1
      0
      1
      1
      1
      0
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      97
      0
      0
      0
      0
      0
      1
      0

      98
      0
      0
      1
      1
      0
      1
      1

      99
      0
      1
      0
      0
      0
      0
      1

      100
      1
      1
      1
      0
      0
      1
      1

      101
      1
      1
      0
      0
      1
      1
      1

      102
      0
      1
      0
      1
      0
      0
      0

      103
      1
      1
      0
      1
      1
      1
      1

      104
      0
      1
      0
      1
      0
      0
      0

      105
      1
      1
      1
      1
      0
      0
      1

      106
      0
      0
      1
      1
      0
      1
      1

      107
      1
      0
      0
      1
      0
      1
      0

      108
      1
      0
      1
      1
      1
      1
      1

      109
      0
      0
      1
      0
      0
      1
      0

      110
      0
      0
      0
      0
      1
      0
      0

      111
      1
      1
      1
      1
      0
      1
      1

      113
      0
      1
      1
      1
      0
      1
      0

      114
      1
      1
      0
      1
      1
      1
      1

      115
      0
      1
      0
      0
      0
      0
      1

      116
      1
      0
      1
      0
      1
      0
      1

      117
      0
      0
      0
      0
      1
      1
      1

      118
      0
      1
      1
      0
      1
      0
      0

      119
      1
      1
      0
      0
      0
      1
      0

      120
      1
      0
      1
      0
      1
      0
      0

      121
      1
      0
      0
      1
      1
      1
      1

      122
      1
      1
      1
      1
      1
      1
      1

      123
      0
      1
      0
      0
      0
      0
      1

      124
      0
      1
      0
      1
      1
      0
      1

      125
      1
      1
      1
      1
      0
      1
      1

      126
      0
      1
      0
      1
      1
      0
      1

      127
      0
      0
      0
      0
      0
      0
      0

126 rows × 7 columns

### myHDL 4:1 MUX Behavioral Level HDL Synthesis

```python
DUT.convert()
VerilogTextReader('MUX41_behavioral');
```

    ***Verilog modual from MUX41_behavioral.v***

     // File: MUX41_behavioral.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:26:38 2018

    `timescale 1ns/10ps

    module MUX41_behavioral (
        sel_1,
        sel_2,
        x_1in,
        x_2in,
        x_3in,
        x_4in,
        f_out
    );

    input sel_1;
    input sel_2;
    input x_1in;
    input x_2in;
    input x_3in;
    input x_4in;
    output f_out;
    reg f_out;

    always @(sel_2, x_1in, sel_1, x_3in, x_4in, x_2in) begin: MUX41_BEHAVIORAL_LOGIC
        if (((!sel_1) && (!sel_2))) begin
            f_out = x_1in;
        end
        else if (((!sel_1) && sel_2)) begin
            f_out = x_2in;
        end
        else if ((sel_1 && (!sel_2))) begin
            f_out = x_3in;
        end
        else begin
            f_out = x_4in;
        end
    end

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL behavioral level 4:1 MUX's verilog code

## 4:1 Multiplexer Behavioral with bitvectors

taking this a step further using bytes we can implement the behavioral using vector inputs instead of single bit inputs as follows

### How bit vectors work in myHDL and in Verilog/VHDL

###  Understanding BitVector bit selection in myHDL

```python
sel=intbv(1)[2:]; x_in=intbv(7)[4:]; f_out=bool(0)
```

```python
for i in x_in:
    print(i)
```

    False
    True
    True
    True

```python
for i in range(4):
    print(x_in[i])
```

    True
    True
    True
    False

### myHDL 4:1 MUX Behavioral with BitVecters and Testing
#### !? This needs to be checked

```python
@block
def MUX41_behavioralVec(sel, x_in, f_out):
    @always_comb
    def logic():
        f_out.next=x_in[sel]
    return logic
```

```python
Peeker.clear()
sel=Signal(intbv(0)[2:]); Peeker(sel, 'sel')
x_in=Signal(intbv(0)[4:]); Peeker(x_in, 'x_in')
f_out=Signal(bool(0)); Peeker(f_out, 'f_out')

DUT=MUX41_behavioralVec(sel, x_in, f_out)

def MUX41_behavioralVec_TB(sel, x_in):

    selLen=len(sel); x_inLen=len(x_in)

    for i in range(2**x_inLen):
        x_in.next=i
        for j in range(selLen):
            sel.next=j

            yield delay(1)
            now()

```

```python
im=Simulation(DUT, MUX41_behavioralVec_TB(sel, x_in), *Peeker.instances()).run()
Peeker.to_wavedrom(tock=True,
                  title='MUX 4:1 behavioral vectype simulation')
```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      sel
      x_in

      0
      0
      0
      0

      1
      0
      1
      0

      2
      1
      0
      1

      3
      0
      1
      1

      4
      0
      0
      2

      5
      1
      1
      2

      6
      1
      0
      3

      7
      1
      1
      3

      8
      0
      0
      4

      9
      0
      1
      4

      10
      1
      0
      5

      11
      0
      1
      5

      12
      0
      0
      6

      13
      1
      1
      6

      14
      1
      0
      7

      15
      1
      1
      7

      16
      0
      0
      8

      17
      0
      1
      8

      18
      1
      0
      9

      19
      0
      1
      9

      20
      0
      0
      10

      21
      1
      1
      10

      22
      1
      0
      11

      23
      1
      1
      11

      24
      0
      0
      12

      25
      0
      1
      12

      26
      1
      0
      13

      27
      0
      1
      13

      28
      0
      0
      14

      29
      1
      1
      14

      30
      1
      0
      15

      31
      1
      1
      15

### myHDL 4:1 MUX Behavioral with BitVecters HDL Synthesis

```python
DUT.convert()
VerilogTextReader('MUX41_behavioralVec');
```

    ***Verilog modual from MUX41_behavioralVec.v***

     // File: MUX41_behavioralVec.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:27:41 2018

    `timescale 1ns/10ps

    module MUX41_behavioralVec (
        sel,
        x_in,
        f_out
    );

    input [1:0] sel;
    input [3:0] x_in;
    output f_out;
    wire f_out;

    assign f_out = x_in[sel];

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL behavioral level 4:1 MUX using Bitvecters verilog code

## Generic Expressions via MUXs
(clean this and find harder exsample)

while shannon's theorem did not prove very much  useful in designing a 4:1 MUX it's true power lies converting boolean logic expression from and or gates to MUX's

using example 4.5 from Brown & Vranesic 3rd Ed

```python
w1, w2, w3=symbols('w_1, w_2, w_3')
f=(~w1&~w3)|(w1&w2)|(w1&w3)
f
```

```python
s1=w1
fp, fp0, fp1=shannon_exspanson(f, s1)
fp, fp0, fp1
```

```python
s2=w2
fpp0, fpp00, fpp01=shannon_exspanson(fp0, s2)
fpp1, fpp10, fpp11=shannon_exspanson(fp1, s2)

fpp0, fpp00, fpp01, fpp1, fpp10, fpp11
```

### myHDL Generic Expression via MUXs and Testing

```python
@block
def Shannon21MUX(s1, s2, w_3in, f_out):
    @always_comb
    def logic():
        if (not s1) and (not s2):
            f_out.next=not w_3in
        elif (not s1) and ( s2):
            f_out.next=not w_3in
        elif ( s1) and (not s2):
            f_out.next= w_3in
        else:
            f_out.next=1
    return logic

```

```python
Peeker.clear()

s1, s2, w_3in,  f_out=[Signal(bool(0)) for _ in range(4)]
Peeker(s1, 's1'); Peeker(s2, 's2');
Peeker(w_3in, 'w_3in')
Peeker(f_out, 'f_out')

DUT=Shannon21MUX(s1, s2, w_3in, f_out)

inputs=[s1, s2, w_3in, f_out]
sim=Simulation(DUT, Combo_TB(inputs), *Peeker.instances()).run()
Peeker.to_wavedrom(start_time=0, stop_time=2*2**len(inputs), tock=True,
                  title='Shannon 2:1 MUX gate type simulation',
                  caption=f'after clock cycle {2**len(inputs)-1} ->random input')

```

    <class 'myhdl.StopSimulation'>: No more events

```python
Peeker.to_dataframe()
```

      f_out
      s1
      s2
      w_3in

      0
      1
      0
      0
      0

      2
      0
      0
      0
      1

      3
      1
      0
      0
      1

      4
      1
      0
      1
      0

      6
      0
      0
      1
      1

      7
      1
      0
      1
      1

      8
      0
      1
      0
      0

      9
      1
      1
      0
      0

      10
      1
      1
      0
      1

      12
      1
      1
      1
      0

      14
      1
      1
      1
      1

      16
      0
      0
      1
      1

      17
      1
      1
      0
      1

      18
      1
      0
      1
      0

      19
      1
      1
      0
      1

      20
      0
      0
      0
      1

      21
      0
      1
      0
      0

      22
      0
      0
      0
      1

      23
      1
      0
      0
      1

      24
      1
      0
      1
      0

      25
      1
      1
      0
      1

      26
      1
      0
      0
      0

      27
      1
      1
      1
      0

      28
      1
      0
      1
      0

      29
      0
      0
      1
      1

      30
      1
      0
      0
      0

### myHDL Generic Expression via MUXs HDL Synthesis

```python
DUT.convert()
VerilogTextReader('Shannon21MUX');
```

    ***Verilog modual from Shannon21MUX.v***

     // File: Shannon21MUX.v
    // Generated by MyHDL 0.10
    // Date: Sat Jul  7 01:28:46 2018

    `timescale 1ns/10ps

    module Shannon21MUX (
        s1,
        s2,
        w_3in,
        f_out
    );

    input s1;
    input s2;
    input w_3in;
    output f_out;
    reg f_out;

    always @(s2, w_3in, s1) begin: SHANNON21MUX_LOGIC
        if (((!s1) && (!s2))) begin
            f_out = (!w_3in);
        end
        else if (((!s1) && s2)) begin
            f_out = (!w_3in);
        end
        else if ((s1 && (!s2))) begin
            f_out = w_3in;
        end
        else begin
            f_out = 1;
        end
    end

    endmodule

The following shows the **Xilinx**'s _Vivado 2016.1_ RTL generated schematic of our myHDL 2:1 Mux expansion of $f$ using Shannon's Expansion Theorom

# Demultiplexers

# Encoders

# Decoders


```python
from decodes.core import *
from decodes.io.jupyter_out import JupyterOut
import math

out = JupyterOut.unit_square( )
```

# Intersections

## Intersector Objects in Decod.es

```python
"""
Intersector Initialization
Two primary members are defined: _geom that stores the geometric results of an
intersection, and log, a String that gives feedback to the calling context. The
geometric results are also made available by the property results.
"""
class Intersector(object):
    def __init__(self):
        self._geom = []
        self.log = None
        self.tol = EPSILON

    @property
    def results(self):
        return self._geom
```

```python
"""
Intersect Method
Given two pieces of Decod.es geometry, a and b, this method attempts to
intersect them by calling upon the appropriate pairwise intersection method,
and returns a success boolean.
"""
def of(self,a,b,**kargs):
    # clear re-initializes the _geom collection
     self.clear()

    # white-list of types supported by intersection methods
    supported_types = [Plane,Circle,PGon,RGon,Line,Ray,Segment,PLine,Arc]
    # ensures the ordering of a and b by type
    a,b = sorted( [a,b], key = lambda obj: supported_types.index(type(obj)) )

    if type(a) == Plane:
        if type(b) == Line : return self._line_plane(b,a,**kargs)
        if type(b) == Ray : return self._ray_plane(b,a,**kargs)
        if type(b) == Segment : return self._seg_plane(b,a)
        if type(b) == PLine : return self._pline_plane(b,a)
        if type(b) == Circle : return self._circle_plane(b,a)
        if type(b) == Arc : return self._arc_plane(b,a)
        if type(b) == Plane : return self._plane_plane(b,a)

    if type(a) == Circle:
        if type(b) == Circle : return self._circle_circle(b,a)
        if isinstance(b, LinearEntity) : return self._line_circle(b,a)

    if type(a) == RGon or type(a) == PGon:
        if isinstance(b,LinearEntity) : return self._line_pgon(b,a,**kargs)

    if isinstance(a,LinearEntity) and isinstance(b,LinearEntity):
        return self._line_line(a,b)

     raise NotImplementedError("I don't know how to intersect these objects.")
```

```python
"""
Intersecting Two Segments
The xsec.of() method attempts to intersect any two given objects, and returns a
Boolean indicating success. If any geometry results from the intersection
operation, it is accessible via the xsec.results property, or by square-bracket
index as is in xsec[0].
"""
xsec = Intersector()
if xsec.of(seg_a, seg_b):
    print "segs intersect at ", xsec.results[0]
```

```python
"""
Trimming Two Segments
Any useful auxiliary properties that are generated as a by-product of an
intersection operation are dynamically assigned as members of an Intersector
object. Here, the auxiliary properties of a Line-Line intersection operation
are used to rescale the two given Segments in something like a trim operation.
"""
xsec = Intersector()
if xsec.of(seg_a, seg_b):
    print "segs intersect at ", xsec[0]
    seg_a.vec *= xsec.ta
    seg_b.vec *= xsec.tb
```

## Elemental Intersections

\begin{align}
t^{*} = \frac{-\vec{n} \bullet (\vec{p_{0}} - \vec{q_{0}} ) }{\vec{n} \bullet \vec{v}}
\end{align}

```python
"""
Line-Line Intersection
Finds intersection of two given LinearEntitites, returning False if the two
are parallel, or if no intersection is found. Collinear lines are handled
separately.
"""
def _line_line(self,ln_a,ln_b):
    # collinear line-line intersections handled separately
    if ln_a.is_collinear(ln_b): return self._line_line_collinear(ln_a,ln_b)
    # parallel lines do not intersect
    if ln_a.is_parallel(ln_b, self.tol): return False

    va, vb = ln_a.vec, ln_b.vec
    vqp = Vec(ln_a.spt-ln_b.spt)
    dp = va.dot(vb)
    denom = -va.length2*vb.length2 + dp*dp

    self.ta = (vb.length2*(va.dot(vqp)) - dp*(vb.dot(vqp)))/denom
    self.tb = (dp*(va.dot(vqp)) - va.length2*(vb.dot(vqp)))/denom
    pa, pb = ln_a.eval(self.ta), ln_b.eval(self.tb)
    #if there is an intersection point
    if pa.is_equal(pb, self.tol) :
        #check parameter interval for separate line types
        if type(ln_a) == Ray and self.ta < 0.0 : return False
        if type(ln_b) == Ray and self.tb < 0.0 : return False
        if type(ln_a) == Segment and (self.ta< 0 or self.ta> 1) : return False
        if type(ln_b) == Segment and (self.tb< 0 or self.tb> 1) : return False
        self._geom.append(pa)
        return True
    else:
        self.log = "No intersection found, recording shortest Segment"
        self._geom.append(Segment(pa,pb))
        return False
```

```python
"""
Line-Plane Intersection Method
"""
def _line_plane(self,line,plane):
    # Line lies on the Plane
    if plane.contains(line.spt) and plane.contains(line.spt+line.vec):
        # store resulting Line intersection
        self._geom.append(line)
        return True

    denom = plane.normal.dot(line.vec)
    #Line lies in a parallel Plane
    if denom == 0 : return False
    t = plane.normal.dot(plane.origin-line.spt) / denom
    # {a} dynamically create a new member
    self.dist = t

    #check inclusion of parameter in interval of line type
    if type(line) == Ray and (t < 0.0): return False
    if type(line) == Segment and ((t < 0.0) or t > 1.0): return False
    # store resulting Point intersection
    self._geom.append(line.eval(t))
    return True
```

```python
"""
Plane-Plane Intersection Method
Finds the intersection of two given Planes, returning False if the two are
parallel and do not intersect.
"""
def _plane_plane(self,pln_a,pln_b):
    # parallel planes do not intersect
    if pln_a.normal.is_parallel(pln_b.normal) : return False
    n1, n2 = pln_a.normal, pln_b.normal
    dot, r1, r2 = n1.dot(n2), n1.dot(pln_a.origin), n2.dot(pln_b.origin)
    vec = n1.cross(pln_b.normal)
    denom = 1-dot*dot
    c1, c2 = (r1 - dot*r2)/denom, (r2 - dot*r1)/denom
    p0 = n1*c1 + n2*c2
    self._geom.append(Line(p0, vec))
    return True
```

```python
"""
Circle-Plane Intersection Method
Finds the intersection of a Plane and a Circle. First, the intersection of the
given Plane and the Circle Plane is found. If successful, the problem reduces
to a Line-Circle intersection problem.
"""
def _circle_plane(self,circ,plane):
    # intersection of the circle plane and the given plane
    xsec = Intersector()
    plane_success = xsec._plane_plane(circ,plane)
    # if planes don't intersect, neither does the circle
    if not plane_success :  return False
    # record the line of plane-plane intersection
    self.line = xsec[0]
    # nearest point projection
    npt, t, dist = self.line.near(circ.origin)
    # nearest point lies outside circle, no intersection
    if dist > circ.rad: return False
    # nearest point is on circle, one point of intersection
    if (abs(dist-circ.rad) < self.tol):
        self._geom.append(npt)
        return True
    # nearest point is in circle, two points of intersection
    if dist < circ.rad:
        factor = math.sqrt(circ.rad**2-dist**2)/self.line.vec.length
        self._geom.append(npt - self.line.vec*factor)
        self._geom.append(npt + self.line.vec*factor)
        return True
```

```python

```

107 - Transformations and Intersections/264 - Intersections.ipynb