# Understanding Anomalies in Time-Series - Is something going on?

```python
!pip install burst_detection
!pip install torch
!pip install torchvision
!apt-get install subversion
!svn export https://github.com/FreelancerFel/Anomalies_Detection/trunk/datasets

# Source for burst_detection implementation
# https://github.com/nmarinsek/burst_detection
# Source for Numenta Anomaly Benchmark
# https://github.com/numenta/NAB
```

    Requirement already satisfied: burst_detection in /usr/local/lib/python3.6/dist-packages (0.1.3)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from burst_detection) (1.18.5)
    Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from burst_detection) (1.1.4)
    Requirement already satisfied: mpmath in /usr/local/lib/python3.6/dist-packages (from burst_detection) (1.1.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->burst_detection) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas->burst_detection) (2.8.1)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.7.3->pandas->burst_detection) (1.15.0)
    Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.7.0+cu101)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.6/dist-packages (from torch) (3.7.4.3)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch) (0.16.0)
    Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from torch) (0.7)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torch) (1.18.5)
    Requirement already satisfied: torchvision in /usr/local/lib/python3.6/dist-packages (0.8.1+cu101)
    Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision) (7.0.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.18.5)
    Requirement already satisfied: torch==1.7.0 in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.7.0+cu101)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (3.7.4.3)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (0.16.0)
    Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (0.7)
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    subversion is already the newest version (1.9.7-4ubuntu1).
    0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
    svn: E155000: Destination directory exists; please remove the directory or use --force to overwrite
    svn: E155000: 'datasets' already exists

## Objective

* Understand anomalies in time-series
* Ability to implement anomalies detection using forecasting with ARIMA, Kleinburg's Algorithm for Burst Detection, and Pytorch implementation of Autoencoder.

```python
import warnings
warnings.filterwarnings('ignore')
```

```python
# load packages

import pandas as pd
import numpy as np
%pylab inline
### Dataset to play with?
from itertools import accumulate
from plotnine import *
import burst_detection as bd

from sklearn import preprocessing

import math
import statsmodels.api as sm
import statsmodels.tsa.api as smt
from sklearn.metrics import mean_squared_error
from matplotlib import pyplot
import matplotlib.pyplot as plt
```

    Populating the interactive namespace from numpy and matplotlib

## Data

We use twitter data from the Numenta Anomaly Benchmark (NAB) v1.1. NAB is a novel benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. The twitter data contain number of tweets related to ten company ranging from Apple, Amazon to Google between the period of Feb 2015 to April 2015

```python
### Load All twitter data, identify where is the 'unusual traffic'
tweets = ['AAPL','AMZN','CRM','CVS','FB','GOOG','IBM','KO','PFE','UPS']
df = pd.read_csv('./datasets/Twitter_volume_AAPL.csv', parse_dates=True)
df['key'] = 'AAPL'
for i in tweets[1:]:
    filename = './datasets/Twitter_volume_'+str(i)+'.csv'
    file = pd.read_csv(filename, parse_dates=True)
    file['key'] = i
    df = pd.concat([df,file], sort=False)
df
```

      timestamp
      value
      key

      0
      2015-02-26 21:42:53
      104
      AAPL

      1
      2015-02-26 21:47:53
      100
      AAPL

      2
      2015-02-26 21:52:53
      99
      AAPL

      3
      2015-02-26 21:57:53
      154
      AAPL

      4
      2015-02-26 22:02:53
      120
      AAPL

      ...
      ...
      ...
      ...

      15861
      2015-04-22 23:27:53
      3
      UPS

      15862
      2015-04-22 23:32:53
      2
      UPS

      15863
      2015-04-22 23:37:53
      2
      UPS

      15864
      2015-04-22 23:42:53
      3
      UPS

      15865
      2015-04-22 23:47:53
      3
      UPS

158631 rows × 3 columns

The dataset is collected every five minutes, but we are not interested in that small of an inteval for time period so it is sum at hour level.

Additionally, we also create a column for aggregate tweets for that company up until that moment and total number of tweets (for all 10 companies) for that hours. The reasoning for the total is to create a 'normal' behaviour for benchmarking

```python
df['timestamp'] =  pd.to_datetime(df['timestamp'], format='%Y-%m-%d %H:%M:%S')
df['timestamp'] = df['timestamp'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,dt.minute//60))
df_alt = df.groupby(['timestamp'])['value'].agg('sum').reset_index(inplace = False)
df = df.groupby(['timestamp','key'])['value'].agg('sum').reset_index(inplace = False)
df = df.sort_values(by=['key','timestamp'])
original = []
for i in tweets:
    output = [*accumulate(list(df[df['key']==i]['value']), lambda a, b: a+b)]
    original= original + output
df['aggregate'] = original
df = df[df['value']!=0]
df = df.merge(df_alt, on=['timestamp'], how='left')
df.columns = ['timestamp','key','value','aggregate','total']
```

```python
df.head(n=20)
```

      timestamp
      key
      value
      aggregate
      total

      0
      2015-02-26 21:00:00
      AAPL
      457
      457
      1181

      1
      2015-02-26 22:00:00
      AAPL
      1906
      2363
      3961

      2
      2015-02-26 23:00:00
      AAPL
      973
      3336
      2520

      3
      2015-02-27 00:00:00
      AAPL
      707
      4043
      2275

      4
      2015-02-27 01:00:00
      AAPL
      527
      4570
      1977

      5
      2015-02-27 02:00:00
      AAPL
      782
      5352
      2752

      6
      2015-02-27 03:00:00
      AAPL
      678
      6030
      3311

      7
      2015-02-27 04:00:00
      AAPL
      492
      6522
      1692

      8
      2015-02-27 05:00:00
      AAPL
      701
      7223
      2163

      9
      2015-02-27 06:00:00
      AAPL
      393
      7616
      1411

      10
      2015-02-27 07:00:00
      AAPL
      248
      7864
      1100

      11
      2015-02-27 08:00:00
      AAPL
      255
      8119
      1108

      12
      2015-02-27 09:00:00
      AAPL
      202
      8321
      1156

      13
      2015-02-27 10:00:00
      AAPL
      283
      8604
      1423

      14
      2015-02-27 11:00:00
      AAPL
      332
      8936
      1517

      15
      2015-02-27 12:00:00
      AAPL
      404
      9340
      1598

      16
      2015-02-27 13:00:00
      AAPL
      639
      9979
      2156

      17
      2015-02-27 14:00:00
      AAPL
      680
      10659
      2495

      18
      2015-02-27 15:00:00
      AAPL
      764
      11423
      2515

      19
      2015-02-27 16:00:00
      AAPL
      898
      12321
      2838

In NAB, we also have the answer. lA json file labelling all 'anomalies' datapoint which is given to us to help with validation.

```python
### Read Answer Data
import json
# read file
with open('./datasets/combined_labels.json', 'r') as myfile:
    data=myfile.read()
# parse file
obj = json.loads(data)
```

```python
datetimenew = []
keyword = []
for i in tweets:
    filename = 'realTweets/Twitter_volume_'+str(i)+'.csv'
    size = len(obj[filename])
    datetimenew += obj[filename]
    keyword += [i]*size
target = pd.DataFrame({'key':keyword,'timestamp':datetimenew,'outlier':1})
target['timestamp'] =  pd.to_datetime(target['timestamp'], format='%Y-%m-%d %H:%M:%S')
target['timestamp'] = target['timestamp'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,dt.minute//60))
```

```python
target.head()
```

      key
      timestamp
      outlier

      0
      AAPL
      2015-03-03 21:00:00
      1

      1
      AAPL
      2015-03-09 17:00:00
      1

      2
      AAPL
      2015-03-16 02:00:00
      1

      3
      AAPL
      2015-03-31 03:00:00
      1

      4
      AMZN
      2015-03-05 19:00:00
      1

```python
df = df.merge(target, on=['key','timestamp'], how='left')
df = df.fillna(0)
```

We now have a dataset with six columns containing everything we need to run some basic anomaly/outlier detection.

```python
df.head()
```

      timestamp
      key
      value
      aggregate
      total
      outlier

      0
      2015-02-26 21:00:00
      AAPL
      457
      457
      1181
      0.0

      1
      2015-02-26 22:00:00
      AAPL
      1906
      2363
      3961
      0.0

      2
      2015-02-26 23:00:00
      AAPL
      973
      3336
      2520
      0.0

      3
      2015-02-27 00:00:00
      AAPL
      707
      4043
      2275
      0.0

      4
      2015-02-27 01:00:00
      AAPL
      527
      4570
      1977
      0.0

## Visualization

Before we get into the detail, let's see what is anomalies in NAB dataset and how does it look like. Some basic plot of each time-series and anomalies are shown below.

```python
ggplot(df, aes(x='timestamp',y='value', group='key', color='key'))+ geom_line()+ theme(
  axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400)
)+geom_point(df[df['outlier']==1],aes(x='timestamp',y='value', group='key'), color='black')+facet_wrap('~key', scales='free')
```

What you may also noticed is that anomalies point that were given by NAB benchmark wasn't entirely accurate in all case or at the very least up for interpretation.

```python
obj['realTweets/Twitter_volume_AAPL.csv']
```

    ['2015-03-03 21:07:53',
     '2015-03-09 17:32:53',
     '2015-03-16 02:57:53',
     '2015-03-31 03:27:53']

```python
ggplot(df[df['key']=='AAPL'], aes(x='timestamp',y='value', group='key', color='key'))+ geom_line()+ theme(
  axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400)
)+geom_point(df[(df['outlier']==1)&(df['key']=='AAPL')],aes(x='timestamp',y='value', group='key'), color='black')

```

We also run visualization by 'aggregate data', this is less clear.

```python
ggplot(df, aes(x='timestamp',y='aggregate', group='key', color='key'))+ geom_line()+ theme(
  axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400)
)+geom_point(df[df['outlier']==1],aes(x='timestamp',y='aggregate', group='key'), color='black')+facet_wrap('~key', scales='free')
```

```python
ggplot(df[df['key']=='AAPL'], aes(x='timestamp',y='aggregate', group='key', color='key'))+ geom_line()+ theme(
  axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400)
)+geom_point(df[(df['outlier']==1)&(df['key']=='AAPL')],aes(x='timestamp',y='aggregate', group='key'), color='black')
```

## Anomalies

After some basic visualization to understand what kind of anomalies existed in this example dataset., we now just need to give a brief explanation of an anomaly (or outlier). It is the deviation in a quantity from its expected value. Anomaly detection is the search of that which ‘does not conform’ with expected value.

* Point Anomaly

* Contextual Anomaly

* Collective Anomaly

# Looking into the future - Using Forecasting to detect anomalies

One of the most simple way of detecting anomalies is to use time-series prediction. Essentially, anomalies is about finding datapoint that is out of ordinary. Time series forecasting helps us estimate that ordinary which we can then use to detect anomalies by comparing them with actual data.

**It does requires you to set a threshold, but this can be navigate with multiple threshold if you are unsure.**

This obviously relies on good forecasting model, but it also means that any forecasting model is compatible with this idea.

* Exponential Smoothing
* ARIMA
* XGBoost - Lag Feature

In this instance, we will simply use ARIMA which is essentially lag feature regression.

```python
AnomaliesData = df[df['key']=='AAPL']
train, test = (AnomaliesData.value[0:-48]), (AnomaliesData.value[-48:])
train_log, test_log = np.log10(train), np.log10(test) # take log to scale
AnomaliesData.head()
```

      timestamp
      key
      value
      aggregate
      total
      outlier

      0
      2015-02-26 21:00:00
      AAPL
      457
      457
      1181
      0.0

      1
      2015-02-26 22:00:00
      AAPL
      1906
      2363
      3961
      0.0

      2
      2015-02-26 23:00:00
      AAPL
      973
      3336
      2520
      0.0

      3
      2015-02-27 00:00:00
      AAPL
      707
      4043
      2275
      0.0

      4
      2015-02-27 01:00:00
      AAPL
      527
      4570
      1977
      0.0

Determine P,D,Q through ACF and PACF table, alternatively use grid search according to either AIC, AICc or BIC value

```python
fig, axes = plt.subplots(1, 2, figsize=(15,4))
fig = sm.graphics.tsa.plot_acf(train, lags=40, ax=axes[0])
fig = sm.graphics.tsa.plot_pacf(train, lags=40, ax=axes[1])
```

```python
order_pdq = (1, 1, 1)
seasonal_order_pdq = (0, 1, 1, 6)
datapoint = [x for x in train_log]
model = sm.tsa.SARIMAX(datapoint, order=order_pdq, seasonal_order=seasonal_order_pdq,enforce_stationarity=False,enforce_invertibility=False)
res = model.fit(disp=False)
print(res.summary())
```

                                     Statespace Model Results
    =========================================================================================
    Dep. Variable:                                 y   No. Observations:                 1276
    Model:             SARIMAX(1, 1, 1)x(0, 1, 1, 6)   Log Likelihood                 181.208
    Date:                           Sun, 08 Nov 2020   AIC                           -354.416
    Time:                                   23:36:19   BIC                           -333.857
    Sample:                                        0   HQIC                          -346.691
                                              - 1276
    Covariance Type:                             opg
    ==============================================================================
                     coef    std err          z      P>|z|      [0.025      0.975]
    ------------------------------------------------------------------------------
    ar.L1          0.7932      0.012     66.069      0.000       0.770       0.817
    ma.L1         -1.0000      2.761     -0.362      0.717      -6.412       4.412
    ma.S.L6       -1.0133      0.012    -87.482      0.000      -1.036      -0.991
    sigma2         0.0418      0.115      0.363      0.717      -0.184       0.268
    ===================================================================================
    Ljung-Box (Q):                      133.56   Jarque-Bera (JB):              5984.54
    Prob(Q):                              0.00   Prob(JB):                         0.00
    Heteroskedasticity (H):               1.14   Skew:                             1.82
    Prob(H) (two-sided):                  0.18   Kurtosis:                        13.03
    ===================================================================================

    Warnings:
    [1] Covariance matrix calculated using the outer product of gradients (complex-step).

We are not trying to predict everything from scratch, but rather use information at each stage to predict what is likely to be the next stage. So instead of train/test split that are norm in forecasting, we reuse the data at every stage and make prediction only for the next datapoint.

```python
predictions = list()
predict_log = list()
for t in range(len(test_log)):
    model = sm.tsa.SARIMAX(datapoint, order=order_pdq, seasonal_order=seasonal_order_pdq,enforce_stationarity=False,enforce_invertibility=False)
    model_fit = model.fit(disp=0)
    output = model_fit.forecast()
    predict_log.append(output[0])
    yhat = 10**output[0]
    predictions.append(yhat)
    obs = test_log.iloc[t]
    datapoint.append(obs)
predictions = pd.Series(predictions)
predictions.index = test.index
```

    /usr/local/lib/python3.6/dist-packages/statsmodels/base/model.py:512: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
      "Check mle_retvals", ConvergenceWarning)

```python
figsize=(15, 7)
plt.figure(figsize=figsize)
pyplot.plot(train,label='train', color='blue')
pyplot.plot(test,label='test', color='purple')
pyplot.plot(predictions, color='green',label='Predicted')
pyplot.legend(loc='upper right')
pyplot.show()
```

Obviously, if you want to look at anomalies in a certain area, you shouldn't train your forecast model using that data. So we reserved some data at the end for some test.

```python
figsize=(15, 7)
plt.figure(figsize=figsize)
pyplot.plot(test,label='test', color='purple')
pyplot.plot(predictions, color='green',label='Predicted')
pyplot.legend(loc='upper right')
pyplot.show()
```

```python
predicted_df=pd.DataFrame()
predicted_df['load_date']=AnomaliesData['timestamp'][-48:]
predicted_df['actuals']=test
predicted_df['predicted']=predictions
predicted_df.reset_index(inplace=True)
del predicted_df['index']
predicted_df.head()
```

      load_date
      actuals
      predicted

      0
      2015-04-21 03:00:00
      1334
      1102.542009

      1
      2015-04-21 04:00:00
      847
      1092.374260

      2
      2015-04-21 05:00:00
      675
      856.415241

      3
      2015-04-21 06:00:00
      502
      651.847114

      4
      2015-04-21 07:00:00
      389
      504.849428

What we have now is the real and predicted data, we just need to look at the average and distribution of the 'error' of our model to create potential approximate 'normal' pattern.

Subsequently, anything outside of that pattern is a potential anomalies.

```python
import numpy as np
def anomalies(df,rollingwindows):
    df['error']=df['actuals']-df['predicted']
    df['meanerror'] = df['error'].rolling(window=rollingwindows).mean()
    df['deviationerror'] = df['error'].rolling(window=rollingwindows).std()
    df['-3sderror'] = df['meanerror'] - (3 * df['deviationerror'])
    df['3sderror'] = df['meanerror'] + (3 * df['deviationerror'])
    df['-2sderror'] = df['meanerror'] - (2 * df['deviationerror'])
    df['2sderror'] = df['meanerror'] + (2 * df['deviationerror'])
    df['-1sderror'] = df['meanerror'] - (1 * df['deviationerror'])
    df['1sderror'] = df['meanerror'] + (1 * df['deviationerror'])
    cut_list = df[['error', '-3sderror', '-2sderror', '-1sderror', 'meanerror', '1sderror', '2sderror', '3sderror']]
    cut_sort = np.sort(cut_list.values)
    df['Anomalies_Strength'] = [(lambda number: np.where(cut_sort == df['error'][number])[1][0])(number)\
                                for number in range(len(df['error']))]
    severity = {0: 'Severe Anomaly', 1: 'Anomaly', 2: 'Unusual', 3: 'Normal', 4: 'Normal',
                5: 'Unusual', 6: 'Anomaly', 7: 'Severe Anomaly'}
    df['Anomalies_Strength'] =  df['Anomalies_Strength'].map(severity)
    df['anomaly_points'] = np.where(df['Anomalies_Strength'] == 'Severe Anomaly', df['error'], np.nan)
    df = df.sort_values(by='load_date', ascending=False)
    df.load_date = pd.to_datetime(df['load_date'].astype(str), format="%Y-%m-%d")
    df[-rollingwindows:]['anomaly_points'] = np.NaN
    df[-rollingwindows:]['Anomalies_Strength'] = 'Normal'
    df['upper_boundary'] = df['predicted']+df['1sderror']
    df['lower_boundary'] = df['predicted']+df['-1sderror']
    df['upper_boundary_high'] = df['predicted']+df['3sderror']
    df['lower_boundary_high'] = df['predicted']+df['-3sderror']
    df = df.drop(columns=['1sderror', '-1sderror','2sderror','-2sderror','-3sderror','3sderror'])
    return df
```

```python
classify_df=anomalies(predicted_df,12)
```

```python
ggplot(classify_df)+ geom_line(aes(x='load_date',y='actuals')) \
+ geom_point(aes(x='load_date',y='actuals',color='Anomalies_Strength'))+\
  scale_color_manual(values = {'Normal':"black",'Anomaly':"orange",'Severe Anomaly':"red",'Unusual':'yellow'})\
+theme(axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400))\
+geom_line(aes(x='load_date',y='upper_boundary'),color='orange',linetype="dotted")\
+geom_line(aes(x='load_date',y='lower_boundary'),color='orange',linetype="dotted")\
+geom_line(aes(x='load_date',y='upper_boundary_high'),color='red',linetype="dotted")\
+geom_line(aes(x='load_date',y='lower_boundary_high'),color='red',linetype="dotted")
```

```python
classify_df.head()
```

      load_date
      actuals
      predicted
      error
      meanerror
      deviationerror
      Anomalies_Strength
      anomaly_points
      upper_boundary
      lower_boundary
      upper_boundary_high
      lower_boundary_high

      47
      2015-04-23 02:00:00
      445
      781.473252
      -336.473252
      37.698309
      543.889056
      Normal
      NaN
      1363.060618
      275.282505
      2450.838730
      -812.495608

      46
      2015-04-23 01:00:00
      717
      685.537941
      31.462059
      69.818659
      531.011939
      Normal
      NaN
      1286.368539
      224.344661
      2348.392417
      -837.679218

      45
      2015-04-23 00:00:00
      718
      662.786431
      55.213569
      66.042498
      531.470421
      Normal
      NaN
      1260.299349
      197.358508
      2323.240191
      -865.582334

      44
      2015-04-22 23:00:00
      670
      833.300994
      -163.300994
      58.816624
      532.219965
      Normal
      NaN
      1424.337583
      359.897653
      2488.777513
      -704.542277

      43
      2015-04-22 22:00:00
      843
      778.259680
      64.740320
      80.837811
      527.641327
      Normal
      NaN
      1386.738818
      331.456164
      2442.021472
      -723.826490

# Looking into the past - Kleinberg's Burst Detection

http://www.cs.cornell.edu/home/kleinber/bhs.pdf

Kleinberg’s algorithm identifies periods of time in which an event of particular interest is uncharacteristically frequent, or “bursty.”  It can be used to detect bursts in a continuous stream or batches of events by assumeing that there are only two possible states, the baseline state and the 'bursty' state.

The probability of event of interest happening in baseline state is equal to the total number of event of interest divided by total number of events. The 'bursty' state probability is constant multiplied by baseline probability.

* pybursts (Continuous stream) = https://github.com/romain-fontugne/pybursts (Forked version which fix some problem)
* burst_detection (Batch event) = https://github.com/nmarinsek/burst_detection

The total cost of transitioning from normal state to burst is equal to the sum of the two functions below. With the cost found, we can then find the optimal state sequence that tell us what is the state at each timepoint by measuring if the new state has been reached or not.

The goodness of fit between the observed proportion and the expected probability of each state.

$\sigma= - ln[\binom{d_t}{r_t}p_i^{rt}(1-p_i)^{d_t-r_t}]$

```python
import sympy

def fitcost(normal,interest,p):
    return -np.log(np.float(sympy.binomial(normal,interest))*(p**interest)*((1-p)**(normal-interest)))
```

The difficulty of transitioning from the previous state to the next state.

$\tau= - (i_{next}-i_{prev})*\gamma*ln(n)$

```python
def transitioncost(previousstate,nextstate,gamma,n):
    if previousstate>=nextstate:
        return 0
    else:
        return (nextstate-previousstate) * gamma * np.log(n)
```

```python
def burstdetection(interest,normal,totaltimepoint,activestatecost,gamma,smooth):

    k = 2
    # smooth time_series, can be ignored if not needed.
    if smooth > 1:
        temp_p = interest/normal
        temp_p = temp_p.rolling(window=smooth, center=True).mean()
        interest = temp_p*normal
        real_n = sum(~np.isnan(interest))
    else:
        real_n = totaltimepoint

    #calculate the expected proportions for states 0 and 1
    p = {}
    p[0] = np.nansum(interest) / float(np.nansum(normal)) #baseline state
    p[1] = p[0] * activestatecost  #cost to move from normal state to a burst state.
    if p[1] > 1: #active state can't be more than maximum of proportuib
      p[1] = 0.99999

    #initialize matrices to hold the costs and optimal state sequence
    cost = np.zeros([totaltimepoint,k])
    q = np.zeros([totaltimepoint,1])

    for t in range(int((smooth-1)/2),(int((smooth-1)/2))+real_n):

        for j in range(k): # There are two states, which one has less cost: 'Active' or Normal
            if t == int((smooth-1)/2):
                cost[t,j] = fitcost(normal[t],interest[t],p[j])
            else:
                cost[t,j] = transitioncost(q[t-1],j,gamma,real_n) + fitcost(normal[t],interest[t],p[j])

        #add the state with the minimum cost to the optimal state sequence
        q[t] = np.where(cost[t,:] == min(cost[t,:]))

    return q,p
```

```python
AnomaliesData_Start = df[df['key']=='AAPL']
min_max_scaler = preprocessing.MinMaxScaler()
np_scaled = min_max_scaler.fit_transform(AnomaliesData_Start[['value','total']])
AnomaliesData = pd.DataFrame(np_scaled*100, columns = ['value','total'])
#Scaled the data because of issue with binomial coefficient
r = pd.Series(AnomaliesData['value']).astype('float')
d = pd.Series(AnomaliesData['total']).astype('float')
```

Burst Detection Function: burst_detection(r,d,len(r),s,d,smooth_win)

* r = number of events of interested
* d = number of all events
* s = distance between normal state and a burst state
* gamma = cost to move from normal state to a burst state.
* smooth = smoothing windows

```python
[q,p] = burstdetection(r,d,len(r),2,1,smooth=3)
bursts = bd.enumerate_bursts(q, 's='+str(3)+', g='+str(1))
bursts = bd.burst_weights(bursts, r, d, p)
```

```python
data_r = pd.DataFrame({'day':range(len(r)),'data':r,'total':d,'outlier':AnomaliesData_Start['outlier']})
data_r['Prediction'] = 0
for index, row in bursts.iterrows():
    data_r['Prediction'] = np.where(data_r['day']>=row.begin,np.where(data_r['day']<=row.end,1,data_r['Prediction']), data_r['Prediction'])
```

```python
ggplot(data_r, aes(x='day',y='data'))+ geom_line()+ theme(
  axis_text_x = element_text(size = 5.5,angle = 90, hjust = 20, vjust = -400)
)+geom_point(data_r[(data_r['outlier']==1)],aes(x='day',y='data'), color='red',shape = 7)+geom_point(data_r[(data_r['Prediction']==1)],aes(x='day',y='data'), color='blue',shape = 6)
```

The strength, or weight, of a burst (that begins at time point t1 and ends at time point t2) can be estimated with the following function:

```python
def plot_burst_timeline(bursts, timepoints):

    f, ax = plt.subplots(figsize=(8, 0.5))
    ax.set(xlim=(0, timepoints), ylabel="", xlabel="")

    #create boxes around bursting periods
    for index, burst in bursts.iterrows():

        #define outline positions
        y = 0.25
        xstart = burst['begin'] -1
        width = burst['end'] - burst['begin'] +1

        ax.add_patch(Rectangle((xstart,y), width, height=0.5,
                               facecolor='#00bbcc', edgecolor='none', linewidth=1))

    ax.xaxis.set_visible(False)

    plt.axhline(0.5, linewidth=1, color='k', alpha=1, zorder=0.5)
    plt.show()
```

```python
plot_burst_timeline(bursts, len(r))
```

# Finding the Pattern - CNN Autoencoders

Last, but by no mean least. There might be some situation where you lack understanding of the data to be able to correctly estimated boundary or level of energy for either forecasting or burst detection.

Essentially, Autoencoder try to reconstruct the data based on what is learn before. If the autoencoder is good enough, it should be able to generalized to most situation and retain most of the standard information of time-series.

```python
import time
import numpy as np
import torch
import torch.nn.functional as F
from torchvision import datasets
from torchvision import transforms
from torch.utils.data import DataLoader
from sklearn.model_selection import train_test_split

if torch.cuda.is_available():
    torch.backends.cudnn.deterministic = True

def get_value_from_df(df):
    return df.value.to_list()

def normalize(values,mean=None,std=None):
    if mean is None:
        mean = np.mean(values)
    values -= mean
    if std is None:
        std = np.std(values)
    values /= std
    return values, mean, std

def create_sequences(values, time_steps=24):
    output = []
    for i in range(len(values) - time_steps):
        output.append(values[i : (i + time_steps)])
    # Convert 2D sequences into 3D as we will be feeding this into
    # a convolutional layer.
    return np.expand_dims(output, axis=2)
```

Using the same data and splitting into test and train, we then decided to use 24 hours of data for each autoencoders. Hence, 24 timesteps.

```python
Autoencoder_Data = df[df['key']=='AAPL']
```

```python
# Get the `value` column from the training dataframe.
training_value = get_value_from_df(Autoencoder_Data)
# Normalize `value` and save the mean and std we get
training_value, training_mean, training_std = normalize(training_value)
TIME_STEPS = 24
batch_size = 128
train = create_sequences(training_value,24)
```

```python
train, test = (train[0:1000,:,:], train[1000:,:,:])
```

```python
X_train = torch.FloatTensor([train]).cuda().reshape(-1, 24, 1)
X_test = torch.FloatTensor([test]).cuda().reshape(-1, 24, 1)
print("training input shape: ", X_train.shape)
print("testing input shape: ", X_test.shape)
```

    training input shape:  torch.Size([1000, 24, 1])
    testing input shape:  torch.Size([300, 24, 1])

```python
train_loader = DataLoader(dataset=X_train,
                          batch_size=batch_size,
                          shuffle=True)

test_loader = DataLoader(dataset=X_test,
                         batch_size=batch_size,
                         shuffle=False)
```

```python
##########################
### SETTINGS
##########################

# Device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Device:', device)

# Hyperparameters
learning_rate = 0.001
num_epochs = 100
```

    Device: cuda:0

Autoencoder model is created here. The model will take input and return output of the same shape.

```python
class ConvolutionalAutoencoder(torch.nn.Module):

    def __init__(self):
        super(ConvolutionalAutoencoder, self).__init__()

        ### ENCODER
        self.conv_1 = torch.nn.Conv1d(in_channels=1,
                                      out_channels=32,
                                      kernel_size=7,
                                      stride=2,
                                      padding=1)
        self.dropout_1 = torch.nn.Dropout(p=0.2)
        self.conv_2 = torch.nn.Conv1d(in_channels=32,
                                      out_channels=16,
                                      kernel_size=7,
                                      stride=2,
                                      padding=1)
        self.dropout_2 = torch.nn.Dropout(p=0.2)

        ### DECODER
        self.deconv_1 = torch.nn.ConvTranspose1d(in_channels=16,
                                                 out_channels=32,
                                                 kernel_size=7,
                                                 stride=2,
                                                 padding=1)
        self.dropout_3 = torch.nn.Dropout(p=0.2)
        self.deconv_2 = torch.nn.ConvTranspose1d(in_channels=32,
                                                 out_channels=32,
                                                 kernel_size=7,
                                                 stride=2,
                                                 padding=1)
        self.deconv_3 = torch.nn.ConvTranspose1d(in_channels=32,
                                                 out_channels=1,
                                                 kernel_size=7,
                                                 stride=1,
                                                 padding=1)

    def forward(self, x):
        ### Input: (batch_size, seq_len, input_size)
        ### ENCODER
        batch_size = x.size()[0]
        seq_length = x.size()[1]
        x = x.view(batch_size,1,seq_length)
        x = self.conv_1(x)
        x = F.relu(x)
        x = self.dropout_1(x)
        x = self.conv_2(x)
        x = F.relu(x)
        x = self.dropout_2(x)

        ### DECODER
        x = self.deconv_1(x)
        x = F.relu(x)
        x = self.dropout_3(x)
        x = self.deconv_2(x)
        x = F.relu(x)
        x = self.deconv_3(x)
        x = x[:,:,:seq_length]
        x = x.view(batch_size,seq_length,1)
        return x

# torch.manual_seed(random_seed)
model = ConvolutionalAutoencoder()
model = model.to(device)

optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
```

```python
model
```

    ConvolutionalAutoencoder(
      (conv_1): Conv1d(1, 32, kernel_size=(7,), stride=(2,), padding=(1,))
      (dropout_1): Dropout(p=0.2, inplace=False)
      (conv_2): Conv1d(32, 16, kernel_size=(7,), stride=(2,), padding=(1,))
      (dropout_2): Dropout(p=0.2, inplace=False)
      (deconv_1): ConvTranspose1d(16, 32, kernel_size=(7,), stride=(2,), padding=(1,))
      (dropout_3): Dropout(p=0.2, inplace=False)
      (deconv_2): ConvTranspose1d(32, 32, kernel_size=(7,), stride=(2,), padding=(1,))
      (deconv_3): ConvTranspose1d(32, 1, kernel_size=(7,), stride=(1,), padding=(1,))
    )

Train the model.

```python
start_time = time.time()
for epoch in range(num_epochs):
    for batch_idx, (features) in enumerate(train_loader):

        # don't need labels, only the images (features)
        features = features.to(device)

        ### FORWARD AND BACK PROP
        logits = model(features)
        cost = F.mse_loss(logits, features)
        optimizer.zero_grad()

        cost.backward()

        ### UPDATE MODEL PARAMETERS
        optimizer.step()

        ### LOGGING
        if not batch_idx % 5 and not epoch % 5:
            print ('Epoch: %03d/%03d | Batch %03d/%03d | Cost: %.4f'
                   %(epoch+1, num_epochs, batch_idx,
                     len(X_train)//batch_size, cost))

    print('Time elapsed: %.2f min' % ((time.time() - start_time)/60))

# print('Total Training Time: %.2f min' % ((time.time() - start_time)/60))
```

    Epoch: 001/100 | Batch 000/007 | Cost: 0.7682
    Epoch: 001/100 | Batch 005/007 | Cost: 0.8018
    Time elapsed: 0.00 min
    Time elapsed: 0.00 min
    Time elapsed: 0.00 min
    Time elapsed: 0.00 min
    Time elapsed: 0.00 min
    Epoch: 006/100 | Batch 000/007 | Cost: 0.3898
    Epoch: 006/100 | Batch 005/007 | Cost: 0.2225
    Time elapsed: 0.00 min
    Time elapsed: 0.00 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Epoch: 011/100 | Batch 000/007 | Cost: 0.3602
    Epoch: 011/100 | Batch 005/007 | Cost: 0.1763
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Epoch: 016/100 | Batch 000/007 | Cost: 0.3334
    Epoch: 016/100 | Batch 005/007 | Cost: 0.3780
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Epoch: 021/100 | Batch 000/007 | Cost: 0.2987
    Epoch: 021/100 | Batch 005/007 | Cost: 0.1593
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.01 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Epoch: 026/100 | Batch 000/007 | Cost: 0.0993
    Epoch: 026/100 | Batch 005/007 | Cost: 0.2322
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Epoch: 031/100 | Batch 000/007 | Cost: 0.2238
    Epoch: 031/100 | Batch 005/007 | Cost: 0.0400
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Epoch: 036/100 | Batch 000/007 | Cost: 0.2240
    Epoch: 036/100 | Batch 005/007 | Cost: 0.1754
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.02 min
    Time elapsed: 0.03 min
    Epoch: 041/100 | Batch 000/007 | Cost: 0.1168
    Epoch: 041/100 | Batch 005/007 | Cost: 0.0642
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Epoch: 046/100 | Batch 000/007 | Cost: 0.1586
    Epoch: 046/100 | Batch 005/007 | Cost: 0.2027
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Epoch: 051/100 | Batch 000/007 | Cost: 0.0705
    Epoch: 051/100 | Batch 005/007 | Cost: 0.0530
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Time elapsed: 0.03 min
    Epoch: 056/100 | Batch 000/007 | Cost: 0.0791
    Epoch: 056/100 | Batch 005/007 | Cost: 0.0858
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Epoch: 061/100 | Batch 000/007 | Cost: 0.1232
    Epoch: 061/100 | Batch 005/007 | Cost: 0.1684
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Epoch: 066/100 | Batch 000/007 | Cost: 0.1947
    Epoch: 066/100 | Batch 005/007 | Cost: 0.0378
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Epoch: 071/100 | Batch 000/007 | Cost: 0.1217
    Epoch: 071/100 | Batch 005/007 | Cost: 0.0398
    Time elapsed: 0.04 min
    Time elapsed: 0.04 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Epoch: 076/100 | Batch 000/007 | Cost: 0.0932
    Epoch: 076/100 | Batch 005/007 | Cost: 0.1349
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Epoch: 081/100 | Batch 000/007 | Cost: 0.1201
    Epoch: 081/100 | Batch 005/007 | Cost: 0.0889
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Epoch: 086/100 | Batch 000/007 | Cost: 0.0837
    Epoch: 086/100 | Batch 005/007 | Cost: 0.0510
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.05 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Epoch: 091/100 | Batch 000/007 | Cost: 0.0655
    Epoch: 091/100 | Batch 005/007 | Cost: 0.0918
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Epoch: 096/100 | Batch 000/007 | Cost: 0.0445
    Epoch: 096/100 | Batch 005/007 | Cost: 0.1958
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min
    Time elapsed: 0.06 min

### Detecting anomalies
Anomalies is detected by comparing input and output and determine those with mean absolute error that is higher than threshold.

If the MEA loss is greater than  threshold value then it can be infer that the model is seeing a pattern that it isn't familiar with which is a potential anomaly. In this case, we used maximum error from training set.

```python
x_train_pred = model(X_train)
x_train_pred = x_train_pred.detach().cpu().numpy()
```

```python
train_mae_loss = np.mean(np.abs(x_train_pred - train), axis=1)

plt.hist(train_mae_loss, bins=50)
plt.xlabel("Train MAE loss")
plt.ylabel("No of samples")
plt.show()

# Get reconstruction loss threshold.
threshold = np.max(train_mae_loss)
print("Reconstruction error threshold: ", threshold)
```

We now validate it by looking at test set for index of time that has high MAE.

```python
x_test_pred = model(X_test)
x_test_pred = x_test_pred.detach().cpu().numpy()
test_mae_loss = np.mean(np.abs(x_test_pred - test), axis=1)
test_mae_loss = test_mae_loss.reshape((-1))

plt.hist(test_mae_loss, bins=50)
plt.xlabel("test MAE loss")
plt.ylabel("No of samples")
plt.show()

# Detect all the samples which are anomalies.
anomalies = (test_mae_loss > threshold).tolist()
index = np.where(anomalies)
print("Number of anomaly samples: ", np.sum(anomalies))
print("Indices of anomaly samples: ", np.where(anomalies))
```

```python
Test_Autoencoder = Autoencoder_Data[1000:]
```

```python
anomalous_data_indices = []
for data_idx in range(24 - 1, len(Test_Autoencoder) - 24 + 1):
    time_series = range(data_idx - 24 + 1, data_idx)
    if any([anomalies[j] for j in time_series]):
        Test_Autoencoder['outlier'].iloc[data_idx,:] = 1
        anomalous_data_indices.append(data_idx)
```

```python
Test_Autoencoder[Test_Autoencoder['outlier']==1]
```

      timestamp
      key
      value
      aggregate
      total
      outlier

      1106
      2015-04-14 01:00:00
      AAPL
      634
      1052364
      1721
      1.0

      1107
      2015-04-14 02:00:00
      AAPL
      1409
      1053773
      2658
      1.0

      1108
      2015-04-14 03:00:00
      AAPL
      1001
      1054774
      2054
      1.0

      1109
      2015-04-14 04:00:00
      AAPL
      1116
      1055890
      2104
      1.0

      1110
      2015-04-14 05:00:00
      AAPL
      539
      1056429
      1312
      1.0

      1111
      2015-04-14 06:00:00
      AAPL
      548
      1056977
      1279
      1.0

      1112
      2015-04-14 07:00:00
      AAPL
      391
      1057368
      1251
      1.0

      1113
      2015-04-14 08:00:00
      AAPL
      274
      1057642
      1213
      1.0

      1114
      2015-04-14 09:00:00
      AAPL
      262
      1057904
      1173
      1.0

      1115
      2015-04-14 10:00:00
      AAPL
      380
      1058284
      1307
      1.0

      1116
      2015-04-14 11:00:00
      AAPL
      431
      1058715
      1537
      1.0

      1117
      2015-04-14 12:00:00
      AAPL
      546
      1059261
      2621
      1.0

      1118
      2015-04-14 13:00:00
      AAPL
      690
      1059951
      2938
      1.0

      1119
      2015-04-14 14:00:00
      AAPL
      9164
      1069115
      13646
      1.0

      1120
      2015-04-14 15:00:00
      AAPL
      2203
      1071318
      4203
      1.0

      1121
      2015-04-14 16:00:00
      AAPL
      2180
      1073498
      4041
      1.0

      1122
      2015-04-14 17:00:00
      AAPL
      1731
      1075229
      3593
      1.0

      1123
      2015-04-14 18:00:00
      AAPL
      1359
      1076588
      3448
      1.0

      1124
      2015-04-14 19:00:00
      AAPL
      1311
      1077899
      3254
      1.0

      1125
      2015-04-14 20:00:00
      AAPL
      1415
      1079314
      3565
      1.0

      1126
      2015-04-14 21:00:00
      AAPL
      2964
      1082278
      4722
      1.0

      1127
      2015-04-14 22:00:00
      AAPL
      16628
      1098906
      18234
      1.0

      1128
      2015-04-14 23:00:00
      AAPL
      68745
      1167651
      70052
      1.0

      1129
      2015-04-15 00:00:00
      AAPL
      3857
      1171508
      4996
      1.0

      1130
      2015-04-15 01:00:00
      AAPL
      1879
      1173387
      3075
      1.0

      1131
      2015-04-15 02:00:00
      AAPL
      2008
      1175395
      3310
      1.0

      1132
      2015-04-15 03:00:00
      AAPL
      1294
      1176689
      2401
      1.0

      1133
      2015-04-15 04:00:00
      AAPL
      1227
      1177916
      1994
      1.0

      1134
      2015-04-15 05:00:00
      AAPL
      1522
      1179438
      2457
      1.0

      1135
      2015-04-15 06:00:00
      AAPL
      1282
      1180720
      2314
      1.0

      1136
      2015-04-15 07:00:00
      AAPL
      521
      1181241
      1357
      1.0

      1137
      2015-04-15 08:00:00
      AAPL
      488
      1181729
      1527
      1.0

      1138
      2015-04-15 09:00:00
      AAPL
      428
      1182157
      1334
      1.0

      1139
      2015-04-15 10:00:00
      AAPL
      492
      1182649
      2062
      1.0

      1140
      2015-04-15 11:00:00
      AAPL
      619
      1183268
      2360
      1.0

      1141
      2015-04-15 12:00:00
      AAPL
      708
      1183976
      2504
      1.0

      1142
      2015-04-15 13:00:00
      AAPL
      733
      1184709
      3149
      1.0

      1143
      2015-04-15 14:00:00
      AAPL
      986
      1185695
      3319
      1.0

      1144
      2015-04-15 15:00:00
      AAPL
      1154
      1186849
      3365
      1.0

      1145
      2015-04-15 16:00:00
      AAPL
      1266
      1188115
      4144
      1.0

      1146
      2015-04-15 17:00:00
      AAPL
      755
      1188870
      2716
      1.0

      1147
      2015-04-15 18:00:00
      AAPL
      952
      1189822
      2860
      1.0

```python
df_subset = Test_Autoencoder.iloc[anomalous_data_indices, :]
plt.subplots_adjust(bottom=0.2)
plt.xticks(rotation=25)
ax = plt.gca()

values = pd.Series(Test_Autoencoder["value"])
plt.plot(values, label="test data")
values = pd.Series(df_subset["value"])
plt.plot(values, label="anomalies", color="r")

plt.legend()
plt.show()
```


# Modeling non-scattering radiative transfer

This notebook is part of [The Climate Laboratory](https://brian-rose.github.io/ClimateLaboratoryBook) by [Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany.

____________

## 1. The two-stream Schwarschild equations
____________

Here we are going to introduce the governing equations of radiative transfer to put what we've been doing on a more solid theoretical footing.

Our derivations here will also serve as a coherent documentation for how some of the radiation solvers are implemented in `climlab`.

### Optical thickness

The **optical thickness** of a layer of absorbers is $\Delta \tau_\nu$. It is the emissivity and absorptivity of a layer of atmosphere.

Passing to the limit of very thin layers, we define $\tau_\nu$ through

$$ \frac{d \tau_\nu}{dp} = -\frac{1}{g} \kappa_\nu $$

where **$\kappa_\nu$ is an absorption cross-section** per unit mass at frequency $\nu$. It has units m$^2$ kg$^{-1}$. $\kappa$ is a measure of the area taken out of the incident beam by absorbers in a unit mass of atmosphere.

In general **$\tau_\nu$ depends on the frequency of the radiation**, as indicated here by the subscript $\nu$.

### Using optical depth as vertical coordinate

Since pressure decreases with altitude, $\tau_\nu$ increases with altitude.

The equations of radiative transfer can be simplified by using $\tau_\nu$ as vertical coordinate instead of pressure.

### Absorption cross-section

The specific absorption cross section $\kappa$ depends on the number of molecules of each greenhouse gas encountered by the beam and the absorption properties characteristic to each kind of greenhouse gas molecule. Letting $q_k$ be the mass-specific concentration of greenhouse gas $k$, we may write in general

$$ \kappa(\nu, p, T) = \sum_{k=1}^n \kappa_k\big(\nu, p, T \big) q_k(p) $$

For a well-mixed greenhouse gas, $q_k$ is a constant; for a non-well-mixed gas like water vapor we need to account for the vertical distribution of the gas through $q_k(p)$.

The dependence of $\kappa_k$ on temperature and pressure arises from certain aspects of the physics of molecular absorption.

### Two-stream Schwarzschild equations

For climate modeling we almost always seperate the total flux into two beams: **upward** and **downward**.

This involves taking integrals of the full angular dependence of the flux. We'll skip the details here.

Let $U_\nu$ be the upward beam, and $D_\nu$ be the downward beam. The governing equations for these beams are the *Schwarzschild equations*:

\begin{align}
\frac{d U_\nu}{d \tau_\nu} &= -U_\nu + E\big( \nu, T(\tau_\nu) \big) \\
\frac{d D_\nu}{d \tau_\nu} &= D_\nu - E\big( \nu, T(\tau_\nu) \big)
\end{align}

where $E$ is the **blackbody emission** (both up and down), which in general depends on both **frequency** and **temperature**. We have written temperature as a function of the vertical coordinate (optical depth).

The emissions are governed by the **Planck function**:
\begin{align}
E &= \pi~ B\big( \nu, T \big) \\
B\big( \nu, T \big) &= \frac{2 h \nu^3}{c^2} \frac{1}{\exp \left( \frac{h \nu}{k T} \right) -1}
\end{align}

with these fundamental physical constants:

- $h = 6.626 \times 10^{-34} ~\text{J s} $ is Planck's constant
- $c = 3.00 \times 10^8 ~\text{m s}^{-1} $ is the speed of light
- $k = 1.38 \times 10^{-23} ~\text{J K}^{-1} $ is the Boltzmann Thermodynamic Constant

The two-stream equations basically say the beam is **attenuated by absorption** (first term) and **augmented by emission** (second term) in each thin layer of gas.

These equations are valid for beam that are **not affected by scattering**. We may come back to that later.

____________

## 2. The Grey Gas Model
____________

The absorption properties $\kappa_i$ of most atmospheric gases varies enormously with the frequency of the radiation. Hence the optical thickness $\tau_\nu$ also has an intricate dependence on wavenumber or frequency.

Our job as climate modelers is to calculate and understand the net atmospheric absorption and transmission of radiation. To do this thoroughly and accurately, the fluxes must be solved for individually on a very dense grid of wavenumbers, and then the results integrated over all wavenumbers.

Actual radiative transfer codes (as used in GCMs) apply a lot of tricks and shortcuts to simplify this brute-force approach, but lead to sets of equations that are difficult to understand.

However, there is a lot we can understand about the basics of radiative transfer and the greenhouse effect by ignoring the spectral dependence of the flux.

Specifically, we make the approximation

$$ \kappa(\nu, p, T) = \kappa(p) $$

so that the optical depth $\tau$ is now **independent of frequency**.

This is known as the **grey gas approximation**.

## Grey gas versions of the Schwarzschild equations

### Integrating the Planck function

If we assume $\tau_\nu$ is independent of frequency, we can then integrate the two-stream equations over all frequencies.

The integral of the Planck function gives our familiar Stefan-Boltzmann blackbody radiation law

$$ E(T) = \int_0^{\infty} \pi B\big( \nu, T \big) d\nu = \sigma T^4  $$

with

$$ \sigma = \frac{2 \pi^5 k^4}{15 c^2 h^3} = 5.67 \times 10^{-8} ~\text{W m}^{-2}~\text{K}^{-4}$$

```python
# climlab has these constants available, and actually calculates sigma from the above formula
import numpy as np
import climlab

sigma = ((2*np.pi**5 * climlab.constants.kBoltzmann**4) /
         (15 * climlab.constants.c_light**2 * climlab.constants.hPlanck**3) )
print( sigma)
sigma == climlab.constants.sigma
```

    5.6703726225913323e-08

    True

### The grey gas equations

This gives the governing equations for the grey gas model:

\begin{align}
\frac{d U}{d \tau} &= -U + \sigma T(\tau)^4 \\
\frac{d D}{d \tau} &= D - \sigma T(\tau)^4
\end{align}

These equations now say that the beam is diminished by absorption in a thin layer (first term) and augmented by blackbody emission, where both processes are assumed to be independent of frequency.

### How do the fluxes change across a finite layer of absorbers?

The above equations are **linear, first order ODEs**, and they are uncoupled from each other (because we neglected scattering).

**Consider a layer of atmosphere** from optical level $\tau_{0}$ to $\tau_{1}$. The optical thickness of the layer is $\Delta \tau = \tau_{1} - \tau_{0}$.

The incident upwelling beam from below is denoted $U_{0}$; the upwelling beam that leaves the top of our layer is denoted $U_{1}$. We want to calculate $U_{1}$, which we can get by integrating the equation for $dU/d\tau$ over our layer.

The result is

$$ U_{1} = U_{0} \exp(-\Delta \tau) + \int_{\tau_{0}}^{\tau_{1}} E \exp \big( -(\tau_{1} - \tau) \big) d\tau $$

where $\tau$ is now a dummy variable of integration, and we've written the blackbody emissions as $E = \sigma T^4$.

The change in the downwelling beam is similar:

$$ D_{0} = D_{1} \exp(-\Delta \tau ) + \int_{\tau_0}^{\tau_1} E \exp\big( -(\tau - \tau_{0}) \big) d\tau $$

____________

## 3. Discretizing the Grey-gas equations on a finite grid
____________

One particularly important reason to think about these changes over finite layers is that we will typically solve the radiative transfer equations in a numerical model where the temperature is represented on a discrete grid (often using pressure coordinates).

If we assume that **temperature is uniform everywhere in our layer** then the blackbody emission $E$ is also a constant across the layer, which we will denote $E_{0}$.

It can come out the integral and the expressions simplify to

\begin{align}
U_1 &= U_0 ~ \exp(-\Delta \tau )  + E_{0} ~\Big( 1 - \exp(-\Delta \tau) \Big)  \\
D_0 &= D_1 ~ \exp(-\Delta \tau )  + E_{0} ~\Big( 1 - \exp(-\Delta \tau) \Big)
\end{align}

### Transmissitivity
The first term is the **transmission** of radition from the bottom to the top of the layer (or vice-versa). We can define the **transmissivity** of the layer (denoted $t_{0}$) as the fraction of the incident beam that is passed on to the next layer:

\begin{align}
t_{0} &= \frac{U_{0} \exp(-\Delta \tau)}{U_{0}}  \\
 &= \exp(-\Delta \tau)\\
\end{align}

### Emissivity
The second term is the net change in the beam due to **emissions** in the layer.

We can the define the **emissivity** $\epsilon_0$ of the layer analogously

\begin{align}
\epsilon_{0} &=  1 - \exp(-\Delta \tau) \\
  &= 1 - t_{0}
\end{align}

### The discrete two-stream equations on a finite pressure grid

Putting this all together gives us two simple equations that govern changes in the upwelling and downwelling beams across a discrete layer of optical depth $\Delta \tau$:

\begin{align}
U_{1} &= t_0 ~ U_{0} + \epsilon_{0} ~ E_{0}  \\
D_{0} &= t_0 ~ D_{1} + \epsilon_{0} ~ E_{0}  \\
\end{align}

Our model will typically be discretized in pressure coordinates. Suppose the thickness of the layer is $\Delta p$, then the optical depth is

$$ \Delta \tau  = -\frac{\kappa}{g} \Delta p$$
(the minus sign accounts for the opposite sign conventions of the two coordinates).

Thus the emissivity of the layer is

$$  \epsilon_{0} = 1 - \exp\big( \frac{\kappa}{g} \Delta p \big)  $$

In the grey gas approximation we ignore all spectral dependence of the flux, so $\kappa$ is a constant (we could let it vary in the vertical to represent non-well-mixed greenhouse gases like water vapor), and the blackbody emissions are simply

$$ E_0 = \sigma T_0^4 $$

____________

## 4. Matrix equations for the grey gas model
____________

`climlab` implements the discretized two-stream equations as written above.

We will now write out the equations on a discretized pressure grid with $N$ layers.

Let the **upwelling flux** be a column vector

$${\bf{U}} = [U_0, U_1, ..., U_{N-1}, U_N]^T$$

If there are $N$ levels then $\bf{U}$ has $N+1$ elements (i.e. the fluxes are defined at the boundaries between levels). We will number the layers starting from 0 following `numpy` index conventions.

- $U_0$ is the upwelling flux from surface to layer 0.
- $U_1$ is the upwelling flux layer 0 to layer 1, etc.
- $U_N$ is the upwelling flux from layer N-1 (the top level) to space.

Same for the **downwelling flux**

$${\bf{D}} = [D_0, D_1, ..., D_N]^T$$

So $D_N$ is the flux down from space and $D_0$ is the back-radiation to the surface.

The **temperature** and **blackbody emissions** are defined for each $N$ pressure level, and are related by

$$E_i = \sigma T_i^4$$

### Emissivity and Transmissivity vectors

Let the vector of absorptivity / emissivity be

$$ {\bf{\epsilon}} = [\epsilon_0, \epsilon_1, ..., \epsilon_{N-1}] $$

where each element is determined by the thickness of the layer and the absorption cross-section for this particular spectral band:

$$ \epsilon_{i} = 1 - \exp\big( \frac{\kappa}{g} \Delta p_i \big) $$

And the transmissivity of individual layers is

$$ t_{i} = 1 - \epsilon_{i} $$

It is convenient to define a vector of transmissivities ${\bf{t}}$ with $N+1$ elements:

$$ {\bf{t}} = [1, t_0, t_1, ..., t_{N-1}] $$

### The downwelling beam

For the downwelling beam, we define a column vector of emissions with $N+1$ elements:

$$ {\bf{E_{down}}} = [E_0, E_1, ..., E_{N-1}, E_N]^{T} $$

where we define the last element $E_N$ as the **incident flux at the TOA**.

For a longwave model, we would usually set $E_N = 0$. For a shortwave model, this would be the incident solar radiation.

We want a matrix ${\bf{T_{down}}}$ that, when multiplied by $E_N$, gives the downwelling beam at each of the $N+1$ layer interfaces:

$${\bf{D}} = {\bf{T_{down}}} ~ {\bf{E_{down}}} $$

### The upwelling beam

Define a vector of emissions for the upwelling beam thus:

$$ {\bf{E_{up}}} = [up_{sfc}, E_0, E_1, ..., E_{N-1}] $$

We need to add the reflected part of the downwelling beam at the surface to any emissions from the surface:

$$ up_{sfc} = E_{sfc} + \alpha_{sfc} ~ D[0] $$

Now we want a matrix ${\bf{T_{up}}}$ such that the upwelling beam is

$${\bf{U}} = {\bf{T_{up}}} ~ {\bf{E_{up}}} $$

### The transmissivity matrices

$$ {\bf{T_{up}}} = \left[ \begin{array}{ccccc} 1 & 0 & 0 &... & 0 & 0 \\
                                             t_0 & 1 & 0 & ... & 0 & 0 \\
                                         t_1 t_0 & t_1 & 1 & ... & 0 & 0 \\
                                     t_2 t_1 t_0 & t_2 t_1 & t_2 & ... & 0 & 0 \\
                                     ... & ... & ... & ... & ... & ... \\
 \prod_0^{N-1} t_i & \prod_1^{N-1} t_i & \prod_2^{N-1} t_i & ... & t_{N-1} & 1
   \end{array} \right] $$

 and

$$ {\bf{T_{down}}} = {\bf{T_{up}}}^T  $$

These formulas have been implemented in `climlab.radiation.transmissivity.Transmissivity()` using vectorized `numpy` array operations.

```python
# example with N=2 layers and constant absorptivity
#  we construct an array of absorptivities
eps = np.array([0.58, 0.58])
#  and pass these as argument to the Transmissivity class
trans = climlab.radiation.transmissivity.Transmissivity(eps)
print( 'Matrix Tup is ')
print( trans.Tup)
print( 'Matrix Tdown is ')
print( trans.Tdown)
```

    Matrix Tup is
    [[1.     0.42   0.1764]
     [0.     1.     0.42  ]
     [0.     0.     1.    ]]
    Matrix Tdown is
    [[1.     0.     0.    ]
     [0.42   1.     0.    ]
     [0.1764 0.42   1.    ]]

____________

## 5. Band-averaged radiation models in `climlab`
____________

The Grey Gas model is a useful first step in understanding how radiation shapes the global energy balance. But we quickly run up against its limitations when trying to understand what really determines climate sensitivity.

What's the next step in the model hierarchy?

Suppose we break up the spectrum into a discrete number $M$ of **spectral bands**. The idea is that we find parts of the spectrum in which the absorption characteristics of the important gases are relatively uniform. We then write the **band-averaged absorption cross section** for gas $k$ and band $j$ as

$$ \kappa_{kj} \big(p, T \big) = \frac{\int_{\nu_j} \kappa_j \big(\nu, p, T \big) d \nu }{  \int_{\nu_j} d \nu } $$

where we integrate over whatever part of the spectrum we have chosen to define band $j$.

In our band models we will typically ignore any dependence of $\kappa$ on temperature. The total absorption cross section for our band is thus (summing over all absorbing gases):

$$ \kappa_j(p) = \sum_{k=1}^n \kappa_{kj}(p) q_k(p) $$

Notice that once we make this defintion, all of the formulas we wrote down above for the grey gas model can be written nearly identically for the fluxes in each band.

The optical depth in band $j$ is

$$ \Delta \tau_j  = -\frac{\kappa_j}{g} \Delta p$$

from which we can define emissivity and transmissivity for band $j$ just as above.

The only difference from the Grey Gas formulas is that the **blackbody emission** in band $j$ (denoted $E_j$) is now only a fraction of $\sigma T^4$.

We will denote this fraction as $b_j$.

The fraction $b_j$ is temperature-dependent, and can be solved by integrating the Planck function:

$$ E_j(T) = \int_{\nu_j} \pi B\big( \nu, T \big) d\nu $$

To simplify our band models, we might choose to fix $b_j$ in advance and just assume

$$ E_j(T) = b_j ~ \sigma ~ T^4 $$

which is sensible if the temperatures don't vary too much.

Regardless of how we calculate $b_j$, they must add up to one over all the bands in our model:

$$ \sum_0^{M-1} b_j = 1 $$

Once we've figured out this division of the total flux into multiple bands, and we know the absorption cross-sections of each band, we can calculate the upwelling and downwelling fluxes independently for each band, **using the same formulas (same code!) as we use in the grey gas model**.

To get the total flux, we just need to sum the beams over all bands:

\begin{align}
U &= \sum_0^{M-1} U_j  \\
D &= \sum_0^{M-1} D_j
\end{align}

____________

## References
____________

For many more details about radiative transfer and a more careful derivation of the two-stream equations, see

> Pierrehumbert, R. T. (2010). Principles of Planetary Climate. Cambridge University Press.

____________

## Credits

This notebook is part of [The Climate Laboratory](https://brian-rose.github.io/ClimateLaboratoryBook), an open-source textbook developed and maintained by [Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany.

It is licensed for free and open consumption under the
[Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) license.

Development of these notes and the [climlab software](https://github.com/brian-rose/climlab) is partially supported by the National Science Foundation under award AGS-1455071 to Brian Rose. Any opinions, findings, conclusions or recommendations expressed here are mine and do not necessarily reflect the views of the National Science Foundation.
____________

```python

```


```python
%matplotlib inline
```

`파이토치(PyTorch) 기본 익히기 <intro.html>`_ ||
`빠른 시작 <quickstart_tutorial.html>`_ ||
`텐서(Tensor) <tensorqs_tutorial.html>`_ ||
`Dataset과 Dataloader <data_tutorial.html>`_ ||
`변형(Transform) <transforms_tutorial.html>`_ ||
`신경망 모델 구성하기 <buildmodel_tutorial.html>`_ ||
**Autograd** ||
`최적화(Optimization) <optimization_tutorial.html>`_ ||
`모델 저장하고 불러오기 <saveloadrun_tutorial.html>`_

``torch.autograd``\ 를 사용한 자동 미분
==========================================================================

신경망을 학습할 때 가장 자주 사용되는 알고리즘은 **역전파**\ 입니다. 이 알고리즘에서,
매개변수(모델 가중치)는 주어진 매개변수에 대한 손실 함수의 **변화도(gradient)**\ 에
따라 조정됩니다.

이러한 변화도를 계산하기 위해 PyTorch에는 ``torch.autograd``\ 라고 불리는 자동 미분 엔진이
내장되어 있습니다. 이는 모든 계산 그래프에 대한 변화도의 자동 계산을 지원합니다.

입력 ``x``, 매개변수 ``w``\ 와 ``b`` , 그리고 일부 손실 함수가 있는 가장 간단한 단일 계층
신경망을 가정하겠습니다. PyTorch에서는 다음과 같이 정의할 수 있습니다:

```python
import torch

x = torch.ones(5)  # input tensor
y = torch.zeros(3)  # expected output
w = torch.randn(5, 3, requires_grad=True)
b = torch.randn(3, requires_grad=True)
z = torch.matmul(x, w)+b
loss = torch.nn.functional.binary_cross_entropy_with_logits(z, y)
```

Tensor, Function과 연산그래프(Computational graph)
------------------------------------------------------------------------------------------

이 코드는 다음의 **연산 그래프** 를 정의합니다:

.. figure:: /_static/img/basics/comp-graph.png
   :alt:

이 신경망에서, ``w``\ 와 ``b``\ 는 최적화를 해야 하는 **매개변수**\ 입니다. 따라서
이러한 변수들에 대한 손실 함수의 변화도를 계산할 수 있어야 합니다. 이를 위해서 해당 텐서에
``requires_grad`` 속성을 설정합니다.

Note

``requires_grad``\ 의 값은 텐서를 생성할 때 설정하거나, 나중에
          ``x.requires_grad_(True)`` 메소드를 사용하여 나중에 설정할 수도 있습니다.

연산 그래프를 구성하기 위해 텐서에 적용하는 함수는 사실 ``Function`` 클래스의 객체입니다.
이 객체는 *순전파* 방향으로 함수를 계산하는 방법과, *역방향 전파* 단계에서 도함수(derivative)를
계산하는 방법을 알고 있습니다. 역방향 전파 함수에 대한 참조(reference)는 텐서의 ``grad_fn``
속성에 저장됩니다. ``Function``\ 에 대한 자세한 정보는
`이 문서 <https://pytorch.org/docs/stable/autograd.html#function>`__
에서 찾아볼 수 있습니다.

```python
print('Gradient function for z =', z.grad_fn)
print('Gradient function for loss =', loss.grad_fn)
```

변화도(Gradient) 계산하기
-------------------------

신경망에서 매개변수의 가중치를 최적화하려면 매개변수에 대한 손실함수의 도함수(derivative)를
계산해야 합니다. 즉, ``x``\ 와 ``y``\ 의 일부 고정값에서 $\frac{\partial loss}{\partial w}$\ 와
$\frac{\partial loss}{\partial b}$ 가 필요합니다.
이러한 도함수를 계산하기 위해, ``loss.backward()`` 를 호출한 다음 ``w.grad``\ 와
``b.grad``\ 에서 값을 가져옵니다:

```python
loss.backward()
print(w.grad)
print(b.grad)
```

Note

- 연산 그래프의 잎(leaf) 노드들 중 ``requires_grad`` 속성이 ``True``\ 로 설정된
    노드들의 ``grad`` 속성만 구할 수 있습니다. 그래프의 다른 모든 노드에서는 변화도가
    유효하지 않습니다.
  - 성능 상의 이유로, 주어진 그래프에서의 ``backward``\ 를 사용한 변화도 계산은 한 번만
    수행할 수 있습니다. 만약 동일한 그래프에서 여러번의 ``backward`` 호출이 필요하면,
    ``backward`` 호출 시에 ``retrain_graph=True``\ 를 전달해야 합니다.

변화도 추적 멈추기
------------------------------------------------------------------------------------------

기본적으로, ``requires_grad=True``\ 인 모든 텐서들은 연산 기록을 추적하고 변화도 계산을
지원합니다. 그러나 모델을 학습한 뒤 입력 데이터를 단순히 적용하기만 하는 경우와 같이 *순전파*
연산만 필요한 경우에는, 이러한 추적이나 지원이 필요없을 수 있습니다.
연산 코드를 ``torch.no_grad()`` 블록으로 둘러싸서 연산 추적을 멈출 수 있습니다:

```python
z = torch.matmul(x, w)+b
print(z.requires_grad)

with torch.no_grad():
    z = torch.matmul(x, w)+b
print(z.requires_grad)
```

동일한 결과를 얻는 다른 방법은 텐서에 ``detach()`` 메소드를 사용하는 것입니다:

```python
z = torch.matmul(x, w)+b
z_det = z.detach()
print(z_det.requires_grad)
```

변화도 추적을 멈춰야 하는 이유들은 다음과 같습니다:
  - 신경망의 일부 매개변수를 **고정된 매개변수(frozen parameter)**\ 로 표시합니다. 이는
    `사전 학습된 신경망을 미세조정 <https://tutorials.pytorch.kr/beginner/finetuning_torchvision_models_tutorial.html>`__
    할 때 매우 일반적인 시나리오입니다.
  - 변화도를 추적하지 않는 텐서의 연산이 더 효율적이기 때문에, 순전파 단계만 수행할 때
    **연산 속도가 향상됩니다.**

연산 그래프에 대한 추가 정보
------------------------------------------------------------------------------------------

개념적으로, autograd는 데이터(텐서)의 및 실행된 모든 연산들(및 연산 결과가 새로운 텐서인 경우도 포함하여)의
기록을 `Function <https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function>`__ 객체로
구성된 방향성 비순환 그래프(DAG; Directed Acyclic Graph)에 저장(keep)합니다.
이 방향성 비순환 그래프(DAG)의 잎(leave)은 입력 텐서이고, 뿌리(root)는 결과 텐서입니다.
이 그래프를 뿌리에서부터 잎까지 추적하면 연쇄 법칙(chain rule)에 따라 변화도를 자동으로 계산할 수 있습니다.

순전파 단계에서, autograd는 다음 두 가지 작업을 동시에 수행합니다:

- 요청된 연산을 수행하여 결과 텐서를 계산하고,
- DAG에 연산의 *변화도 기능(gradient function)* 를 유지(maintain)합니다.

역전파 단계는 DAG 뿌리(root)에서 ``.backward()`` 가 호출될 때 시작됩니다. ``autograd``\ 는 이 때:

- 각 ``.grad_fn`` 으로부터 변화도를 계산하고,
- 각 텐서의 ``.grad`` 속성에 계산 결과를 쌓고(accumulate),
- 연쇄 법칙을 사용하여, 모든 잎(leaf) 텐서들까지 전파(propagate)합니다.

Note

**PyTorch에서 DAG들은 동적(dynamic)입니다.**
  주목해야 할 중요한 점은 그래프가 처음부터(from scratch) 다시 생성된다는 것입니다; 매번 ``.bachward()`` 가
  호출되고 나면, autograd는 새로운 그래프를 채우기(populate) 시작합니다. 이러한 점 덕분에 모델에서
  흐름 제어(control flow) 구문들을 사용할 수 있게 되는 것입니다; 매번 반복(iteration)할 때마다 필요하면
  모양(shape)이나 크기(size), 연산(operation)을 바꿀 수 있습니다.

선택적으로 읽기(Optional Reading): 텐서 변화도와 야코비안 곱 (Jacobian Product)
------------------------------------------------------------------------------------------

대부분의 경우, 스칼라 손실 함수를 가지고 일부 매개변수와 관련한 변화도를 계산해야 합니다.
그러나 출력 함수가 임의의 텐서인 경우가 있습니다. 이럴 때, PyTorch는 실제 변화도가 아닌
**야코비안 곱(Jacobian product)**\ 을 계산합니다.

$\vec{x}=\langle x_1,\dots,x_n\rangle$\ 이고,
$\vec{y}=\langle y_1,\dots,y_m\rangle$\ 일 때
벡터 함수 $\vec{y}=f(\vec{x})$\ 에서 $\vec{x}$\ 에 대한
$\vec{y}$ 의 변화도는 **야코비안 행렬(Jacobian matrix)**\ 로 주어집니다:

\begin{align}J=\left(\begin{array}{ccc}
      \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial y_{1}}{\partial x_{n}}\\
      \vdots & \ddots & \vdots\\
      \frac{\partial y_{m}}{\partial x_{1}} & \cdots & \frac{\partial y_{m}}{\partial x_{n}}
      \end{array}\right)\end{align}

야코비안 행렬 자체를 계산하는 대신, PyTorch는 주어진 입력 벡터 $v=(v_1 \dots v_m)$\ 에 대한
**야코비안 곱(Jacobian Product)**  $v^T\cdot J$\ 을 계산합니다.
이 과정은 $v$\ 를 인자로 ``backward``\ 를 호출하면 이뤄집니다. $v$\ 의 크기는
곱(product)을 계산하려고 하는 원래 텐서의 크기와 같아야 합니다.

```python
inp = torch.eye(5, requires_grad=True)
out = (inp+1).pow(2)
out.backward(torch.ones_like(inp), retain_graph=True)
print("First call\n", inp.grad)
out.backward(torch.ones_like(inp), retain_graph=True)
print("\nSecond call\n", inp.grad)
inp.grad.zero_()
out.backward(torch.ones_like(inp), retain_graph=True)
print("\nCall after zeroing gradients\n", inp.grad)
```

동일한 인자로 ``backward``\ 를 두차례 호출하면 변화도 값이 달라집니다.
이는 ``역방향`` 전파를 수행할 때, PyTorch가 **변화도를 누적(accumulate)해두기 때문**\
입니다. 즉, 계산된 변화도의 값이 연산 그래프의 모든 잎(leaf) 노드의 ``grad`` 속성에
추가됩니다. 따라서 제대로된 변화도를 계산하기 위해서는 ``grad`` 속성을 먼저 0으로 만들어야
합니다. 실제 학습 과정에서는 *옵티마이저(optimizer)*\ 가 이 과정을 도와줍니다.

Note

이전에는 매개변수 없이 ``backward()`` 함수를 호출했습니다. 이는 본질적으로
          ``backward(torch.tensor(1.0))`` 을 호출하는 것과 동일하며,
          신경망 훈련 중의 손실과 같은 스칼라-값 함수의 변화도를 계산하는 유용한 방법입니다.

--------------

더 읽어보기
~~~~~~~~~~~~~~~~~
- `Autograd Mechanics <https://pytorch.org/docs/stable/notes/autograd.html>`_


# Customer Segmentation: Estimate Individualized Responses to Incentives

Nowadays, business decision makers rely on estimating the causal effect of interventions to answer what-if questions about shifts in strategy, such as promoting specific product with discount, adding new features to a website or increasing investment from a sales team. However, rather than learning whether to take action for a specific intervention for all users, people are increasingly interested in understanding the different responses from different users to the two alternatives. Identifying the characteristics of users having the strongest response for the intervention could help make rules to segment the future users into different groups. This can help optimize the policy to use the least resources and get the most profit.

In this case study, we will use a personalized pricing example to explain how the [EconML](https://aka.ms/econml) library could fit into this problem and provide robust and reliable causal solutions.

### Summary

1. [Background](#background)
2. [Data](#data)
3. [Get Causal Effects with EconML](#estimate)
4. [Understand Treatment Effects with EconML](#interpret)
5. [Make Policy Decisions with EconML](#policy)
6. [Conclusions](#conclusion)

# Background

The global online media market is growing fast over the years. Media companies are always interested in attracting more users into the market and encouraging them to buy more songs or become members. In this example, we'll consider a scenario where one experiment a media company is running is to give small discount (10%, 20% or 0) to their current users based on their income level in order to boost the likelihood of their purchase. The goal is to understand the **heterogeneous price elasticity of demand** for people with different income level, learning which users would respond most strongly to a small discount. Furthermore, their end goal is to make sure that despite decreasing the price for some consumers, the demand is raised enough to boost the overall revenue.

EconML’s `DML` based estimators can be used to take the discount variation in existing data, along with a rich set of user features, to estimate heterogeneous price sensitivities that vary with multiple customer features. Then, the `SingleTreeCateInterpreter` provides a presentation-ready summary of the key features that explain the biggest differences in responsiveness to a discount, and the `SingleTreePolicyInterpreter` recommends a policy on who should receive a discount in order to increase revenue (not only demand), which could help the company to set an optimal price for those users in the future.

```python
# Some imports to get us started
# Utilities
import os
import urllib.request
import numpy as np
import pandas as pd

# Generic ML imports
from sklearn.preprocessing import PolynomialFeatures
from sklearn.ensemble import GradientBoostingRegressor

# EconML imports
from econml.dml import LinearDML, ForestDML
from econml.cate_interpreter import SingleTreeCateInterpreter, SingleTreePolicyInterpreter

import matplotlib.pyplot as plt

%matplotlib inline
```

# Data

The dataset* has ~10,000 observations and includes 9 continuous and categorical variables that represent user's characteristics and online behaviour history such as age, log income, previous purchase, previous online time per week, etc.

We define the following variables:

Feature Name|Type|Details
:--- |:---|:---
**account_age** |W| user's account age
**age** |W|user's age
**avg_hours** |W| the average hours user was online per week in the past
**days_visited** |W| the average number of days user visited the website per week in the past
**friend_count** |W| number of friends user connected in the account
**has_membership** |W| whether the user had membership
**is_US** |W| whether the user accesses the website from the US
**songs_purchased** |W| the average songs user purchased per week in the past
**income** |X| user's income
**price** |T| the price user was exposed during the discount season (baseline price * small discount)
**demand** |Y| songs user purchased during the discount season

**To protect the privacy of the company, we use the simulated data as an example here. The data is synthetically generated and the feature distributions don't correspond to real distributions. However, the feature names have preserved their names and meaning.*

The treatment and outcome are generated using the following functions:
$$
T =
\begin{cases}
  1 & \text{with } p=0.2,  \\
  0.9 & \text{with }p=0.3, & \text{if income}<1 \\
  0.8 & \text{with }p=0.5, \\
  \\
    1 & \text{with }p=0.7, \\
  0.9 & \text{with }p=0.2, & \text{if income}\ge1 \\
  0.8 & \text{with }p=0.1, \\
\end{cases}
$$

\begin{align}
\gamma(X) & = -3 - 14 \cdot \{\text{income}<1\} \\
\beta(X,W) & = 20 + 0.5 \cdot \text{avg_hours} + 5 \cdot \{\text{days_visited}>4\} \\
Y &= \gamma(X) \cdot T + \beta(X,W)
\end{align}

```python
# Import the sample pricing data
file_url = "https://msalicedatapublic.blob.core.windows.net/datasets/Pricing/pricing_sample.csv"
train_data = pd.read_csv(file_url)
```

```python
# Data sample
train_data.head()
```

      account_age
      age
      avg_hours
      days_visited
      friends_count
      has_membership
      is_US
      songs_purchased
      income
      price
      demand

      0
      3
      53
      1.834234
      2
      8
      1
      1
      4.903237
      0.960863
      1.0
      3.917117

      1
      5
      54
      7.171411
      7
      9
      0
      1
      3.330161
      0.732487
      1.0
      11.585706

      2
      3
      33
      5.351920
      6
      9
      0
      1
      3.036203
      1.130937
      1.0
      24.675960

      3
      2
      34
      6.723551
      0
      8
      0
      1
      7.911926
      0.929197
      1.0
      6.361776

      4
      4
      30
      2.448247
      5
      8
      1
      0
      7.148967
      0.533527
      0.8
      12.624123

```python
# Define estimator inputs
Y = train_data["demand"]  # outcome of interest
T = train_data["price"]  # intervention, or treatment
X = train_data[["income"]]  # features
W = train_data.drop(columns=["demand", "price", "income"])  # confounders
```

```python
# Get test data
X_test = np.linspace(0, 5, 100).reshape(-1, 1)
X_test_data = pd.DataFrame(X_test, columns=["income"])
```

# Get Causal Effects with EconML
To learn the price elasticity on demand as a function of income, we fit the model as follows:

\begin{align}
log(Y) & = \theta(X) \cdot log(T) + f(X,W) + \epsilon \\
log(T) & = g(X,W) + \eta
\end{align}

where $\epsilon, \eta$ are uncorrelated error terms.

The models we fit here aren't an exact match for the data generation function above, but if they are a good approximation, they will allow us to create a good discount policy.  Although the model is misspecified, we hope to see that our `DML` based estimators can still capture the right trend of $\theta(X)$ and that the recommended policy beats other baseline policies (such as always giving a discount) on revenue.  Because of the mismatch between the data generating process and the model we're fitting, there isn't a single true $\theta(X)$ (the true elasticity varies with not only X but also T and W), but given how we generate the data above, we can still calculate the range of true $\theta(X)$ to compare against.

```python
# Define underlying treatment effect function given DGP
def gamma_fn(X):
    return -3 - 14 * (X["income"] < 1)

def beta_fn(X):
    return 20 + 0.5 * (X["avg_hours"]) + 5 * (X["days_visited"] > 4)

def demand_fn(data, T):
    Y = gamma_fn(data) * T + beta_fn(data)
    return Y

def true_te(x, n, stats):
    if x < 1:
        subdata = train_data[train_data["income"] < 1].sample(n=n, replace=True)
    else:
        subdata = train_data[train_data["income"] >= 1].sample(n=n, replace=True)
    te_array = subdata["price"] * gamma_fn(subdata) / (subdata["demand"])
    if stats == "mean":
        return np.mean(te_array)
    elif stats == "median":
        return np.median(te_array)
    elif isinstance(stats, int):
        return np.percentile(te_array, stats)
```

```python
# Get the estimate and range of true treatment effect
truth_te_estimate = np.apply_along_axis(true_te, 1, X_test, 1000, "mean")  # estimate
truth_te_upper = np.apply_along_axis(true_te, 1, X_test, 1000, 95)  # upper level
truth_te_lower = np.apply_along_axis(true_te, 1, X_test, 1000, 5)  # lower level
```

## Parametric heterogeneity
First of all, we can try to learn a **linear projection of the treatment effect** assuming a polynomial form of $\theta(X)$. We use the `LinearDML` estimator. Since we don't have any priors on these models, we use a generic gradient boosting tree estimators to learn the expected price and demand from the data.

```python
# Get log_T and log_Y
log_T = np.log(T)
log_Y = np.log(Y)
```

```python
# Train EconML model
est = LinearDML(
    model_y=GradientBoostingRegressor(),
    model_t=GradientBoostingRegressor(),
    featurizer=PolynomialFeatures(degree=2, include_bias=False),
)
est.fit(log_Y, log_T, X=X, W=W, inference="statsmodels")
# Get treatment effect and its confidence interval
te_pred = est.effect(X_test)
te_pred_interval = est.effect_interval(X_test)
```

```python
# Compare the estimate and the truth
plt.figure(figsize=(10, 6))
plt.plot(X_test.flatten(), te_pred, label="Sales Elasticity Prediction")
plt.plot(X_test.flatten(), truth_te_estimate, "--", label="True Elasticity")
plt.fill_between(
    X_test.flatten(),
    te_pred_interval[0],
    te_pred_interval[1],
    alpha=0.2,
    label="90% Confidence Interval",
)
plt.fill_between(
    X_test.flatten(),
    truth_te_lower,
    truth_te_upper,
    alpha=0.2,
    label="True Elasticity Range",
)
plt.xlabel("Income")
plt.ylabel("Songs Sales Elasticity")
plt.title("Songs Sales Elasticity vs Income")
plt.legend(loc="lower right")
```

From the plot above, it's clear to see that the true treatment effect is a **nonlinear** function of income, with elasticity around -1.75 when income is smaller than 1 and a small negative value when income is larger than 1. The model fits a quadratic treatment effect, which is not a great fit. But it still captures the overall trend: the elasticity is negative and people are less sensitive to the price change if they have higher income.

```python
# Get the final coefficient and intercept summary
est.summary()
```

Coefficient Results

      	     point_estimate	 stderr	  zstat	  pvalue	 ci_lower	 ci_upper

  income	        2.414	      0.079	 30.372	    0.0	    2.283	    2.544

  income^2	     -0.435	      0.027	 -15.929	   0.0	    -0.48	    -0.39

CATE Intercept Results

         	        point_estimate	 stderr	  zstat	  pvalue	 ci_lower	 ci_upper

  cate_intercept	     -3.005	      0.049	 -61.397	   0.0	   -3.085	   -2.924

A linear parametric conditional average treatment effect (CATE) model was fitted:
$Y = \Theta(X)\cdot T + g(X, W) + \epsilon$
where for every outcome $i$ and treatment $j$ the CATE $\Theta_{ij}(X)$ has the form:
$\Theta_{ij}(X) = \phi(X)' coef_{ij} + cate\_intercept_{ij}$
where $\phi(X)$ is the output of the `featurizer` or $X$ if `featurizer`=None. Coefficient Results table portrays the $coef_{ij}$ parameter vector for each outcome $i$ and treatment $j$. Intercept Results table portrays the $cate\_intercept_{ij}$ parameter.

`LinearDML` estimator can also return the summary of the coefficients and intercept for the final model, including point estimates, p-values and confidence intervals. From the table above, we notice that $income$ has positive effect and ${income}^2$ has negative effect, and both of them are statistically significant.

## Nonparametric Heterogeneity
Since we already know the true treatment effect function is nonlinear, let us fit another model using `ForestDML`, which assumes a fully **nonparametric estimation of the treatment effect**.

```python
# Train EconML model
est = ForestDML(
    model_y=GradientBoostingRegressor(), model_t=GradientBoostingRegressor()
)
est.fit(log_Y, log_T, X=X, W=W, inference="blb")
# Get treatment effect and its confidence interval
te_pred = est.effect(X_test)
te_pred_interval = est.effect_interval(X_test)
```

```python
# Compare the estimate and the truth
plt.figure(figsize=(10, 6))
plt.plot(X_test.flatten(), te_pred, label="Sales Elasticity Prediction")
plt.plot(X_test.flatten(), truth_te_estimate, "--", label="True Elasticity")
plt.fill_between(
    X_test.flatten(),
    te_pred_interval[0],
    te_pred_interval[1],
    alpha=0.2,
    label="90% Confidence Interval",
)
plt.fill_between(
    X_test.flatten(),
    truth_te_lower,
    truth_te_upper,
    alpha=0.2,
    label="True Elasticity Range",
)
plt.xlabel("Income")
plt.ylabel("Songs Sales Elasticity")
plt.title("Songs Sales Elasticity vs Income")
plt.legend(loc="lower right")
```

We notice that this model fits much better than the `LinearDML`, the 90% confidence interval correctly covers the true treatment effect estimate and captures the variation when income is around 1. Overall, the model shows that people with low income are much more sensitive to the price changes than higher income people.

# Understand Treatment Effects with EconML
EconML includes interpretability tools to better understand treatment effects. Treatment effects can be complex, but oftentimes we are interested in simple rules that can differentiate between users who respond positively, users who remain neutral and users who respond negatively to the proposed changes.

The EconML `SingleTreeCateInterpreter` provides interperetability by training a single decision tree on the treatment effects outputted by the any of the EconML estimators. In the figure below we can see in dark red users respond strongly to the discount and the in white users respond lightly to the discount.

```python
intrp = SingleTreeCateInterpreter(include_model_uncertainty=True, max_depth=2, min_samples_leaf=10)
intrp.interpret(est, X_test)
plt.figure(figsize=(25, 5))
intrp.plot(feature_names=X.columns, fontsize=12)
```

# Make Policy Decision with EconML
We want to make policy decisions to maximum the **revenue** instead of the demand. In this scenario,

\begin{align}
Rev & = Y \cdot T \\
    & = \exp^{log(Y)} \cdot T\\
    & = \exp^{(\theta(X) \cdot log(T) + f(X,W) + \epsilon)} \cdot T \\
    & = \exp^{(f(X,W) + \epsilon)} \cdot T^{(\theta(X)+1)}
\end{align}

With the decrease of price, revenue will increase only if $\theta(X)+1<0$. Thus, we set `sample_treatment_cast=-1` here to learn **what kinds of customers we should give a small discount to maximum the revenue**.

The EconML library includes policy interpretability tools such as `SingleTreePolicyInterpreter` that take in a treatment cost and the treatment effects to learn simple rules about which customers to target profitably. In the figure below we can see the model recommends to give discount for people with income less than $0.985$ and give original price for the others.

```python
intrp = SingleTreePolicyInterpreter(risk_level=0.05, max_depth=2, min_samples_leaf=1, min_impurity_decrease=0.001)
intrp.interpret(est, X_test, sample_treatment_costs=-1, treatment_names=["Discount", "No-Discount"])
plt.figure(figsize=(25, 5))
intrp.plot(feature_names=X.columns, fontsize=12)
```

Now, let us compare our policy with other baseline policies! Our model says which customers to give a small discount to, and for this experiment, we will set a discount level of 10% for those users. Because the model is misspecified we would not expect good results with large discounts. Here, because we know the ground truth, we can evaluate the value of this policy.

```python
# define function to compute revenue
def revenue_fn(data, discount_level1, discount_level2, baseline_T, policy):
    policy_price = baseline_T * (1 - discount_level1) * policy + baseline_T * (1 - discount_level2) * (1 - policy)
    demand = demand_fn(data, policy_price)
    rev = demand * policy_price
    return rev
```

```python
policy_dic = {}
# our policy above
policy = intrp.treat(X)
policy_dic["Our Policy"] = np.mean(revenue_fn(train_data, 0, 0.1, 1, policy))

## previous strategy
policy_dic["Previous Strategy"] = np.mean(train_data["price"] * train_data["demand"])

## give everyone discount
policy_dic["Give Everyone Discount"] = np.mean(revenue_fn(train_data, 0.1, 0, 1, np.ones(len(X))))

## don't give discount
policy_dic["Give No One Discount"] = np.mean(revenue_fn(train_data, 0, 0.1, 1, np.ones(len(X))))

## follow our policy, but give -10% discount for the group doesn't recommend to give discount
policy_dic["Our Policy + Give Negative Discount for No-Discount Group"] = np.mean(revenue_fn(train_data, -0.1, 0.1, 1, policy))

## give everyone -10% discount
policy_dic["Give Everyone Negative Discount"] = np.mean(revenue_fn(train_data, -0.1, 0, 1, np.ones(len(X))))
```

```python
# get policy summary table
res = pd.DataFrame.from_dict(policy_dic, orient="index", columns=["Revenue"])
res["Rank"] = res["Revenue"].rank(ascending=False)
res
```

      Revenue
      Rank

      Our Policy
      14.686241
      2.0

      Previous Strategy
      14.349342
      4.0

      Give Everyone Discount
      13.774469
      6.0

      Give No One Discount
      14.294606
      5.0

      Our Policy + Give Negative Discount for No-Discount Group
      15.564411
      1.0

      Give Everyone Negative Discount
      14.612670
      3.0

**We beat the baseline policies!** Our policy gets the highest revenue except for the one raising the price for the No-Discount group. That means our currently baseline price is low, but the way we segment the user does help increase the revenue!

# Conclusions

In this notebook, we have demonstrated the power of using EconML to:

* Estimate the treatment effect correctly even the model is misspecified
* Interpret the resulting individual-level treatment effects
* Make the policy decision beats the previous and baseline policies

To learn more about what EconML can do for you, visit our [website](https://aka.ms/econml), our [GitHub page](https://github.com/microsoft/EconML) or our [documentation](https://econml.azurewebsites.net/).

notebooks/CustomerScenarios/Case Study - Customer Segmentation at An Online Media Company.ipynb