# Setup

```python
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
```

```python
date_filename = "2018-02-01_2018-02-28"
```

```python
data = pd.read_csv("articles_" + date_filename + ".csv", index_col="id", \
                   parse_dates=["published", "discovered"])
data.head()
```

      url
      headline
      discovered
      published
      fb_engagements
      fb_max_engagements_per_min
      fb_max_engagements_per_min_time
      fb_comments
      fb_reactions
      fb_shares
      publisher_name
      publisher_id
      mins_as_lead
      mins_on_front
      num_articles_on_front
      fb_brand_page
      fb_brand_page_likes
      fb_brand_page_time
      alexa_rank
      word_count

      id

      402fd72431c6a0e29e76fc9abaa6bc0d42f2ea81
      https://www.thetimes.co.uk/article/mistakes-pr...
      ‘Mistakes’ prove costly for a chastened H&M
      2018-02-01 00:06:11.391
      2018-02-01
      0
      0.000000
      2018-02-01T01:19:11.780Z
      0
      0
      0
      The Times
      thetimes_co_uk
      0
      1440
      283.0
      False
      NaN
      NaN
      6435
      125.0

      956698308752aa83f6ad988aa05a4f26f5514f9e
      https://www.thetimes.co.uk/article/met-office-...
      Met Office puts accent on the regions
      2018-02-01 00:05:43.754
      2018-02-01
      2
      0.016407
      2018-02-01T09:25:15.520Z
      1
      0
      1
      The Times
      thetimes_co_uk
      0
      1440
      283.0
      False
      NaN
      NaN
      6435
      125.0

      1524679962f3bef642390e1955d470fe0acc2aff
      https://www.thetimes.co.uk/article/world-s-ric...
      World’s richest offer £100k salaries for Briti...
      2018-02-01 00:05:46.313
      2018-02-01
      9
      0.032886
      2018-02-01T18:33:06.181Z
      1
      2
      6
      The Times
      thetimes_co_uk
      0
      1440
      283.0
      False
      NaN
      NaN
      6435
      125.0

      13646fa08dee5c8b5bdd718ab882cb988a1eb85f
      https://www.thetimes.co.uk/article/mother-give...
      Mother gives birth in jail as vital evidence i...
      2018-02-01 00:05:37.255
      2018-02-01
      5
      0.033241
      2018-02-01T16:31:16.418Z
      1
      2
      2
      The Times
      thetimes_co_uk
      0
      1440
      283.0
      False
      NaN
      NaN
      6435
      125.0

      44dccc673596c768f1129a020c1b6b0c6ff8ab66
      https://www.thetimes.co.uk/article/apple-inves...
      Apple investigated over phone slowdown
      2018-02-01 00:05:38.393
      2018-02-01
      12
      0.065574
      2018-02-01T11:27:12.692Z
      5
      4
      3
      The Times
      thetimes_co_uk
      0
      1440
      283.0
      False
      NaN
      NaN
      6435
      125.0

# Response Score

The response score is a number between 0 and 50 that indicates the level of response to an article.

Perhaps in the future we may choose to include other factors, but for now we just include engagements on Facebook. The maximum score of 50 should be achieved by an article that does really well compared with others.

```python
pd.options.display.float_format = '{:.2f}'.format
data.fb_engagements.describe([0.5, 0.75, 0.9, 0.95, 0.99, 0.995, 0.999])
```

    count    136062.00
    mean       1237.17
    std       11019.23
    min           0.00
    50%          25.00
    75%         241.00
    90%        1453.00
    95%        3841.85
    99%       23172.80
    99.5%     38692.09
    99.9%    116762.69
    max     1733438.00
    Name: fb_engagements, dtype: float64

There's 2 articles with more than 1 million engagements this month.

```python
data[data.fb_engagements > 1000000]
```

      url
      headline
      discovered
      published
      fb_engagements
      fb_max_engagements_per_min
      fb_max_engagements_per_min_time
      fb_comments
      fb_reactions
      fb_shares
      publisher_name
      publisher_id
      mins_as_lead
      mins_on_front
      num_articles_on_front
      fb_brand_page
      fb_brand_page_likes
      fb_brand_page_time
      alexa_rank
      word_count

      id

      fb1944a99013e1b35fd6a91c48804999102e083e
      https://www.nbcnews.com/politics/donald-trump/...
      Trump Signs Bill Revoking Obama-Era Gun Checks...
      2018-02-15 21:25:15.475
      2018-02-15 21:23:07
      1733438
      4222.05
      2018-02-17T21:37:14.123Z
      579201
      879715
      274522
      NBC News
      nbcnews_com
      0
      230
      123.00
      False
      nan
      NaN
      826
      410.00

      eb66c2d3e4862b7dd4b1321a8636fd9ce19a9454
      http://www.foxnews.com/us/2018/02/16/florida-t...
      Florida Teacher of the Year's gun violence pos...
      2018-02-16 20:39:09.526
      2018-02-16 20:01:41
      1005295
      2850.50
      2018-02-17T06:02:17.750Z
      119159
      699578
      186558
      Fox News
      foxnews_com
      0
      1168
      170.00
      True
      16202655.00
      2018-02-16T23:46:04.000Z
      285
      660.00

```python
data.fb_engagements.mode()
```

    0    0
    dtype: int64

*november* Going back to the enagement counts, we see the mean is 1,117, mode is zero, median is 24, 90th percentile is 1,453, 99th percentile is 21,166, 99.5th percentile is 33,982. The standard deviation is 8,083, significantly higher than the mean, so this is not a normal distribution.

*december* Going back to the enagement counts, we see the mean is 1,106, mode is zero, median is 24, 90th percentile is 1,545, 99th percentile is 20,228, 99.5th percentile is 32,446. The standard deviation is 9,852, significantly higher than the mean, so this is not a normal distribution.

*january 2018* Going back to the enagement counts, we see the mean is 1,108, mode is zero, median is 26, 90th percentile is 1,621, 99th percentile is 19,918, 99.5th percentile is 32,935. The standard deviation is 8,278, significantly higher than the mean, so this is not a normal distribution.

*febrary 2018* Going back to the enagement counts, we see the mean is 1,237, mode is zero, median is 25, 90th percentile is 1,453, 99th percentile is 23,172, 99.5th percentile is 38,692. The standard deviation is 11,019, significantly higher than the mean, so this is not a normal distribution.

Key publishers stats

```python
data.groupby("publisher_id").agg({'url': 'count', 'fb_engagements': ['sum', 'median', 'mean']})
```

      url
      fb_engagements

      count
      sum
      median
      mean

      publisher_id

      anotherangryvoice_blogspot_co_uk
      28
      94804
      1286.50
      3385.86

      bbc_co_uk
      11509
      8867217
      29.00
      770.46

      breitbart_com
      2565
      9295242
      182.00
      3623.88

      brexitcentral_com
      59
      39801
      230.00
      674.59

      buzzfeed_com
      1323
      4651835
      218.00
      3516.13

      cnn_com
      3898
      23528295
      565.00
      6035.99

      dailymail_co_uk
      23015
      13477979
      21.00
      585.62

      economist_com
      501
      165074
      36.00
      329.49

      evolvepolitics_com
      63
      188901
      1697.00
      2998.43

      foxnews_com
      5592
      16520792
      66.00
      2954.36

      ft_com
      4368
      322462
      4.00
      73.82

      huffingtonpost_com
      2498
      12873544
      710.00
      5153.54

      independent_co_uk
      5246
      3799617
      33.00
      724.29

      indy100_com
      434
      487881
      90.50
      1124.15

      lemonde_fr
      3765
      1695720
      63.00
      450.39

      libdemvoice_org
      149
      2012
      7.00
      13.50

      mirror_co_uk
      9642
      5307000
      30.00
      550.40

      nbcnews_com
      1685
      8958300
      458.00
      5316.50

      newstatesman_com
      418
      85245
      30.00
      203.94

      npr_org
      1952
      9781344
      255.50
      5010.93

      nytimes_com
      4352
      17345404
      215.00
      3985.62

      order-order_com
      240
      73039
      169.00
      304.33

      propublica_org
      41
      61980
      570.00
      1511.71

      reuters_com
      5653
      2095413
      18.00
      370.67

      rt_com
      2199
      1881653
      238.00
      855.69

      skwawkbox_org
      102
      47299
      244.50
      463.72

      telegraph_co_uk
      6223
      1530290
      16.00
      245.91

      thecanary_co
      182
      179683
      493.50
      987.27

      theguardian_com
      7750
      8451675
      117.00
      1090.54

      thetimes_co_uk
      8336
      307256
      1.00
      36.86

      washingtonpost_com
      21670
      14148474
      0.00
      652.91

      westmonster_com
      206
      239325
      49.50
      1161.77

      yournewswire_com
      398
      1826893
      123.50
      4590.18

```python
mean = data.fb_engagements.mean()
median = data.fb_engagements.median()

non_zero_fb_enagagements = data.fb_engagements[data.fb_engagements > 0]
```

That's a bit better, but still way too clustered at the low end. Let's look at a log normal distribution.

```python
mean = data.fb_engagements.mean()
median = data.fb_engagements.median()
ninety = data.fb_engagements.quantile(.90)
ninetyfive = data.fb_engagements.quantile(.95)
ninetynine = data.fb_engagements.quantile(.99)

plt.figure(figsize=(12,4.5))
plt.hist(np.log(non_zero_fb_enagagements + median), bins=50)
plt.axvline(np.log(mean), linestyle=':', label=f'Mean ({mean:,.0f})', color='green')
plt.axvline(np.log(median), label=f'Median ({median:,.0f})', color='green')
plt.axvline(np.log(ninety), linestyle='--', label=f'90% percentile ({ninety:,.0f})', color='red')
plt.axvline(np.log(ninetyfive), linestyle='-.', label=f'95% percentile ({ninetyfive:,.0f})', color='red')
plt.axvline(np.log(ninetynine), linestyle=':', label=f'99% percentile ({ninetynine:,.0f})', color='red')
leg = plt.legend()

```

```python
eng = data.fb_engagements[(data.fb_engagements < 5000)]
mean = data.fb_engagements.mean()
median = data.fb_engagements.median()
ninety = data.fb_engagements.quantile(.90)
ninetyfive = data.fb_engagements.quantile(.95)
ninetynine = data.fb_engagements.quantile(.99)

plt.figure(figsize=(15,7))
plt.hist(eng, bins=50)
plt.title("Article count by engagements")
plt.axvline(median, label=f'Median ({median:,.0f})', color='green')
plt.axvline(mean, linestyle=':', label=f'Mean ({mean:,.0f})', color='green')
plt.axvline(ninety, linestyle='--', label=f'90% percentile ({ninety:,.0f})', color='red')
plt.axvline(ninetyfive, linestyle='-.', label=f'95% percentile ({ninetyfive:,.0f})', color='red')
# plt.axvline(ninetynine, linestyle=':', label=f'99% percentile ({ninetynine:,.0f})', color='red')
leg = plt.legend()

```

```python
log_engagements = (non_zero_fb_enagagements
                   .clip_upper(data.fb_engagements.quantile(.999))
                   .apply(lambda x: np.log(x + median))
                  )
log_engagements.describe()
```

    count   110738.00
    mean         4.99
    std          1.75
    min          3.26
    25%          3.53
    50%          4.41
    75%          6.03
    max         11.67
    Name: fb_engagements, dtype: float64

Use standard feature scaling to bring that to a 1 to 50 range

```python
def scale_log_engagements(engagements_logged):
    return np.ceil(
        50 * (engagements_logged - log_engagements.min()) / (log_engagements.max() - log_engagements.min())
    )

def scale_engagements(engagements):
    return scale_log_engagements(np.log(engagements + median))

scaled_non_zero_engagements = scale_log_engagements(log_engagements)
scaled_non_zero_engagements.describe()
```

    count   110738.00
    mean        10.78
    std         10.43
    min          0.00
    25%          2.00
    50%          7.00
    75%         17.00
    max         50.00
    Name: fb_engagements, dtype: float64

```python
# add in the zeros, as zero
scaled_engagements = pd.concat([scaled_non_zero_engagements, data.fb_engagements[data.fb_engagements == 0]])
```

```python
proposed = pd.DataFrame({"fb_engagements": data.fb_engagements, "response_score": scaled_engagements})
proposed.response_score.plot.hist(bins=50)
```

Looks good to me, lets save that.

```python
data["response_score"] = proposed.response_score
```

### Proposal

The maximum of 50 points is awarded when the engagements are greater than the 99.9th percentile, rolling over the last month.

i.e. where $limit$ is the 99.5th percentile of engagements calculated over the previous month, the response score for article $a$ is:

\begin{align}
basicScore_a & =
\begin{cases}
   0 & \text{if } engagements_a = 0 \\
   \log(\min(engagements_a,limit) + median(engagements))      & \text{if } engagements_a > 0
\end{cases} \\
responseScore_a & =
\begin{cases}
   0 & \text{if } engagements_a = 0 \\
   50 \cdot \frac{basicScore_a - \min(basicScore)}{\max(basicScore) - \min(basicScore)}       & \text{if } engagements_a > 0
\end{cases} \\
\\
\text{The latter equation can be expanded to:} \\
responseScore_a & =
\begin{cases}
   0 & \text{if } engagements_a = 0 \\
   50 \cdot
   \frac{\log(\min(engagements_a,limit) + median(engagements)) - \log(1 + median(engagements))}
   {\log(limit + median(engagements)) - \log(1 + median(engagements))}       & \text{if } engagements_a > 0
\end{cases} \\
\end{align}

# Promotion Score

The aim of the promotion score is to indicate how important the article was to the publisher, by tracking where they chose to promote it. This is a number between 0 and 50 comprised of:

- 20 points based on whether the article was promoted as the "lead" story on the publisher's home page
- 15 points based on how long the article was promoted anywhere on the publisher's home page
- 15 points based on whether the article was promoted on the publisher's main facebook brand page

The first two should be scaled by the popularity/reach of the home page, for which we use the alexa page rank as a proxy.

The last should be scaled by the popularity/reach of the brand page, for which we use the number of likes the brand page has.

### Lead story (20 points)

```python
data.mins_as_lead.describe([0.5, 0.75, 0.9, 0.95, 0.99, 0.995, 0.999])
```

    count   136062.00
    mean         9.28
    std        124.25
    min          0.00
    50%          0.00
    75%          0.00
    90%          0.00
    95%          0.00
    99%        270.00
    99.5%      557.78
    99.9%     1159.94
    max      30386.00
    Name: mins_as_lead, dtype: float64

As expected, the vast majority of articles don't make it as lead. Let's explore how long typically publishers put something as lead for.

```python
lead_articles = data[data.mins_as_lead > 0]
```

```python
lead_articles.mins_as_lead.describe([0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.995, 0.999])
```

    count    3769.00
    mean      335.12
    std       669.51
    min         4.00
    25%        89.00
    50%       180.00
    75%       415.00
    90%       818.20
    95%      1071.80
    99%      1575.92
    99.5%    2007.16
    99.9%    7185.27
    max     30386.00
    Name: mins_as_lead, dtype: float64

```python
lead_articles.mins_as_lead.plot.hist(bins=50)
```

For lead, it's a  significant thing for an article to be lead at all, so although we want to penalise articles that were lead for a very short time, mostly we want to score the maximum even if it wasn't lead for ages. So we'll give maximum points when something has been lead for an hour.

```python
lead_articles.mins_as_lead.clip_upper(60).plot.hist(bins=50)
```

We also want to scale this by the alexa page rank, such that the maximum score of 20 points is for an article that was on the front for 4 hours for the most popular site.

So lets explore the alexa nunbers.

```python
alexa_ranks = data.groupby(by="publisher_id").alexa_rank.mean().sort_values()
alexa_ranks
```

    publisher_id
    bbc_co_uk                               96
    cnn_com                                105
    nytimes_com                            120
    theguardian_com                        142
    buzzfeed_com                           147
    dailymail_co_uk                        158
    washingtonpost_com                     191
    huffingtonpost_com                     215
    foxnews_com                            285
    rt_com                                 365
    telegraph_co_uk                        370
    independent_co_uk                      386
    reuters_com                            497
    npr_org                                594
    lemonde_fr                             618
    mirror_co_uk                           706
    nbcnews_com                            826
    breitbart_com                          994
    ft_com                                1596
    economist_com                         1825
    indy100_com                           5014
    thetimes_co_uk                        6435
    newstatesman_com                     12769
    thecanary_co                         15686
    propublica_org                       16066
    yournewswire_com                     22568
    order-order_com                      32515
    anotherangryvoice_blogspot_co_uk     77827
    westmonster_com                      97775
    evolvepolitics_com                  119412
    skwawkbox_org                       152475
    libdemvoice_org                     344992
    brexitcentral_com                   469149
    Name: alexa_rank, dtype: int64

```python
alexa_ranks.plot.bar(figsize=[10,5])
```

Let's try the simple option first: just divide the number of minutes as lead by the alexa rank. What's the scale of numbers we get then.

```python
lead_proposal_1 = lead_articles.mins_as_lead.clip_upper(60) / lead_articles.alexa_rank
lead_proposal_1.plot.hist()
```

Looks like there's too much of a cluster around 0. Have we massively over penalised the publishers with a high alexa rank?

```python
lead_proposal_1.groupby(data.publisher_id).mean().plot.bar(figsize=[10,5])
```

Yes. Let's try taking the log of the alexa rank and see if that looks better.

```python
lead_proposal_2 = (lead_articles.mins_as_lead.clip_upper(60) / np.log(lead_articles.alexa_rank))
lead_proposal_2.plot.hist()
```

```python
lead_proposal_2.groupby(data.publisher_id).describe()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      publisher_id

      anotherangryvoice_blogspot_co_uk
      28.00
      5.33
      0.00
      5.33
      5.33
      5.33
      5.33
      5.33

      bbc_co_uk
      103.00
      12.80
      1.68
      2.19
      13.15
      13.15
      13.15
      13.15

      breitbart_com
      194.00
      8.32
      1.43
      0.72
      8.69
      8.69
      8.69
      8.69

      brexitcentral_com
      51.00
      4.16
      1.11
      0.77
      4.59
      4.59
      4.59
      4.59

      buzzfeed_com
      290.00
      11.73
      1.21
      2.00
      12.02
      12.02
      12.02
      12.02

      cnn_com
      190.00
      12.36
      1.81
      2.15
      12.89
      12.89
      12.89
      12.89

      dailymail_co_uk
      158.00
      11.39
      1.70
      1.78
      11.85
      11.85
      11.85
      11.85

      economist_com
      65.00
      7.79
      0.98
      2.40
      7.99
      7.99
      7.99
      7.99

      foxnews_com
      95.00
      10.45
      1.05
      0.88
      10.61
      10.61
      10.61
      10.61

      ft_com
      77.00
      7.53
      1.66
      0.68
      8.14
      8.14
      8.14
      8.14

      huffingtonpost_com
      156.00
      11.03
      0.96
      2.79
      11.17
      11.17
      11.17
      11.17

      independent_co_uk
      134.00
      9.50
      1.89
      0.67
      10.07
      10.07
      10.07
      10.07

      indy100_com
      120.00
      6.17
      1.53
      1.64
      5.75
      7.04
      7.04
      7.04

      lemonde_fr
      251.00
      7.95
      2.63
      0.62
      7.78
      9.34
      9.34
      9.34

      libdemvoice_org
      142.00
      4.61
      0.48
      1.10
      4.71
      4.71
      4.71
      4.71

      nbcnews_com
      114.00
      8.68
      1.22
      0.74
      8.93
      8.93
      8.93
      8.93

      newstatesman_com
      58.00
      6.12
      0.88
      1.59
      6.35
      6.35
      6.35
      6.35

      npr_org
      143.00
      8.95
      1.59
      1.41
      9.39
      9.39
      9.39
      9.39

      nytimes_com
      51.00
      12.45
      0.61
      8.15
      12.53
      12.53
      12.53
      12.53

      order-order_com
      237.00
      4.48
      1.67
      0.48
      3.27
      5.78
      5.78
      5.78

      propublica_org
      16.00
      6.20
      0.00
      6.20
      6.20
      6.20
      6.20
      6.20

      reuters_com
      94.00
      9.26
      1.71
      0.64
      9.66
      9.66
      9.66
      9.66

      rt_com
      135.00
      9.60
      1.80
      0.85
      10.17
      10.17
      10.17
      10.17

      skwawkbox_org
      102.00
      4.71
      0.86
      0.42
      5.03
      5.03
      5.03
      5.03

      telegraph_co_uk
      104.00
      9.76
      1.58
      1.69
      10.15
      10.15
      10.15
      10.15

      thecanary_co
      172.00
      4.95
      1.66
      0.93
      3.93
      6.21
      6.21
      6.21

      theguardian_com
      161.00
      11.50
      2.02
      2.82
      12.11
      12.11
      12.11
      12.11

      thetimes_co_uk
      63.00
      6.65
      1.01
      1.14
      6.84
      6.84
      6.84
      6.84

      washingtonpost_com
      80.00
      10.96
      1.96
      0.95
      11.42
      11.42
      11.42
      11.42

      westmonster_com
      36.00
      5.01
      0.78
      0.87
      5.22
      5.22
      5.22
      5.22

      yournewswire_com
      149.00
      5.69
      0.66
      3.19
      5.99
      5.99
      5.99
      5.99

```python
lead_proposal_2.groupby(data.publisher_id).min().plot.bar(figsize=[10,5])
```

That looks about right, as long as the smaller publishers were closer to zero. So let's apply feature scaling to this, to give a number between 1 and 20. (Anything not as lead will pass though as zero.)

```python
def rescale(series):
    return (series - series.min()) / (series.max() - series.min())

lead_proposal_3 = np.ceil(20 * rescale(lead_proposal_2))
```

```python
lead_proposal_2.min(), lead_proposal_2.max()
```

    (0.41894447029448817, 13.145359968846892)

```python
lead_proposal_3.plot.hist()
```

```python
lead_proposal_3.groupby(data.publisher_id).median().plot.bar(figsize=[10,5])
```

```python
data["lead_score"] = pd.concat([lead_proposal_3, data.mins_as_lead[data.mins_as_lead==0]])
```

```python
data.lead_score.value_counts().sort_index()
```

    0.00     132294
    1.00         41
    2.00         49
    3.00         45
    4.00         44
    5.00         60
    6.00         63
    7.00        244
    8.00        194
    9.00        273
    10.00       179
    11.00       159
    12.00        93
    13.00        77
    14.00       293
    15.00       407
    16.00       343
    17.00       251
    18.00       226
    19.00       407
    20.00       320
    Name: lead_score, dtype: int64

```python
data.lead_score.groupby(data.publisher_id).max()
```

    publisher_id
    anotherangryvoice_blogspot_co_uk    8.00
    bbc_co_uk                          20.00
    breitbart_com                      14.00
    brexitcentral_com                   7.00
    buzzfeed_com                       19.00
    cnn_com                            20.00
    dailymail_co_uk                    18.00
    economist_com                      12.00
    evolvepolitics_com                  0.00
    foxnews_com                        17.00
    ft_com                             13.00
    huffingtonpost_com                 17.00
    independent_co_uk                  16.00
    indy100_com                        11.00
    lemonde_fr                         15.00
    libdemvoice_org                     7.00
    mirror_co_uk                        0.00
    nbcnews_com                        14.00
    newstatesman_com                   10.00
    npr_org                            15.00
    nytimes_com                        20.00
    order-order_com                     9.00
    propublica_org                     10.00
    reuters_com                        15.00
    rt_com                             16.00
    skwawkbox_org                       8.00
    telegraph_co_uk                    16.00
    thecanary_co                       10.00
    theguardian_com                    19.00
    thetimes_co_uk                     11.00
    washingtonpost_com                 18.00
    westmonster_com                     8.00
    yournewswire_com                    9.00
    Name: lead_score, dtype: float64

In summary then, score for article $a$ is:

$$
unscaledLeadScore_a = \frac{\min(minsAsLead_a, 60)}{\log(alexaRank_a)}\\
leadScore_a = 19 \cdot
\frac{unscaledLeadScore_a - \min(unscaledLeadScore)}
{\max(unscaledLeadScore) - \min(unscaledLeadScore)}
+ 1
$$

Since the minium value of $minsAsLead$ is 1, $\min(unscaledLeadScore)$ is pretty insignificant. So we can simplify this to:

$$
leadScore_a = 20 \cdot
\frac{unscaledLeadScore_a }
     {\max(unscaledLeadScore)}
$$

or:

$$
leadScore_a = 20 \cdot
\frac{\frac{\min(minsAsLead_a, 60)}{\log(alexaRank_a)} }
     {\frac{60}{\log(\max(alexaRank))}}
$$

$$
leadScore_a = \left( 20 \cdot
\frac{\min(minsAsLead_a, 60)}{\log(alexaRank_a)} \cdot
     {\frac{\log(\max(alexaRank))}{60}} \right)
$$

## Time on front score (15 points)

This is similar to time as lead, so lets try doing the same calculation, except we also want to factor in the number of slots on the front:

$$frontScore_a =
15
\left(\frac{\min(minsOnFront_a, 1440)}{alexaRank_a \cdot numArticlesOnFront_a}\right)
\left( \frac{\min(alexaRank \cdot numArticlesOnFront)}{1440} \right)$$

```python
(data.alexa_rank * data.num_articles_on_front).min() / 1440
```

    2.4500000000000002

```python
time_on_front_proposal_1 = np.ceil(data.mins_on_front.clip_upper(1440) / (data.alexa_rank * data.num_articles_on_front) * (2.45) * 15)
```

```python
time_on_front_proposal_1.plot.hist(figsize=(15, 7), bins=15)
```

```python
time_on_front_proposal_1.value_counts().sort_index()
```

    1.00     68084
    2.00      6873
    3.00      4369
    4.00      4066
    5.00       769
    6.00       643
    7.00       543
    8.00       522
    9.00       774
    10.00      359
    11.00      135
    12.00      135
    13.00       92
    14.00       49
    15.00       20
    dtype: int64

```python
time_on_front_proposal_1.groupby(data.publisher_id).sum()
```

    publisher_id
    anotherangryvoice_blogspot_co_uk      28.00
    bbc_co_uk                          13177.00
    breitbart_com                       2469.00
    brexitcentral_com                     59.00
    buzzfeed_com                        9419.00
    cnn_com                            11915.00
    dailymail_co_uk                    13020.00
    economist_com                        316.00
    evolvepolitics_com                    59.00
    foxnews_com                         6728.00
    ft_com                              2871.00
    huffingtonpost_com                  7483.00
    independent_co_uk                   3708.00
    indy100_com                          434.00
    lemonde_fr                          3752.00
    libdemvoice_org                      148.00
    mirror_co_uk                        9134.00
    nbcnews_com                         1633.00
    newstatesman_com                     413.00
    npr_org                             2586.00
    nytimes_com                         8873.00
    order-order_com                      240.00
    propublica_org                        41.00
    reuters_com                         6466.00
    rt_com                              4351.00
    skwawkbox_org                        102.00
    telegraph_co_uk                     4496.00
    thecanary_co                         182.00
    theguardian_com                    11165.00
    thetimes_co_uk                      8325.00
    washingtonpost_com                  8565.00
    westmonster_com                      168.00
    yournewswire_com                     398.00
    dtype: float64

That looks good to me.

```python
data["front_score"] = np.ceil(data.mins_on_front.clip_upper(1440) / (data.alexa_rank * data.num_articles_on_front) * (2.45) * 15).fillna(0)
```

```python
data.front_score
```

    id
    402fd72431c6a0e29e76fc9abaa6bc0d42f2ea81   1.00
    956698308752aa83f6ad988aa05a4f26f5514f9e   1.00
    1524679962f3bef642390e1955d470fe0acc2aff   1.00
    13646fa08dee5c8b5bdd718ab882cb988a1eb85f   1.00
    44dccc673596c768f1129a020c1b6b0c6ff8ab66   1.00
    c79fa786172cf75bc8c7df55261f88802762dfce   1.00
    2ea986b1e9669911a984b966ceca1ce95fbfe119   1.00
    c261550cf016a29150e2d0648916160178e69eaa   1.00
    6faa4d962b6436f2db4a469d6d75b6f85e46eace   1.00
    cfae32d025a1bfe4e55b5fc1e31f8822d54b1301   1.00
    d91572cf3c19bfcb811f2ad3d76ba91c64a16dba   1.00
    22064c30c58acf5f08ee572dbe05768d1342aac2   1.00
    5dee083190bd81f2fa8eb58e584ece3fbfe81d81   1.00
    da47e9b97d048bbd6416d5e6bd9819e05622bb20   1.00
    9445e067eb1cfbae505a28911692b72f627a0302   1.00
    9c1c83e1d94d6fab16933fbd6d82f4a17472214c   1.00
    fed2156894429a97eaad8d263af59676cc1b0d4e   1.00
    9d06a956e45f3d75253f25f142dcb008fbe91ec0   1.00
    cba30053abc9c1e1bdc4400e87f0ae8b27b6dde1   1.00
    ae8d3ed982b149d3a9e69abef81c06840b100954   1.00
    65671059afa84f9546e0459a8316bebea025a34c   1.00
    cfea31674c56ecf591faf746d182806b4ae94146   1.00
    62d11d26e6dd88b2de2b0308b5c2f9f3baf35eea   1.00
    7e16c7440040514351056d45e2347d47096299fb   1.00
    62c27fe7eb7da3e90404789704e49aa8bfc00bb3   1.00
    e056daeaabff5dc242f9d3d00498f94c09add6f6   1.00
    698e07a38c6792e0a101c715dc3d83d864687c4f   1.00
    891ace61198aa413a27f39bd6776018decdf7a49   1.00
    cadaed9e9c0f564e850c71893e1b4749b26c1d4e   1.00
    04f3a557366bbf9d5d74cf6e4857c31db80991f4   1.00
                                               ...
    ecf84098352929c7b4bb4602dcf88a408aec9709   0.00
    eed8ab55d6ac640cf925fb53dc0bddbe2208d61b   2.00
    24cd4487ce3ff489cab5d9d48344ef010a78d133   0.00
    a5acdee911bf4e90ef35e8bbcd2b5bbbe3bf50e9   6.00
    23bbbaa3b0ba8d926b36eebdced1e06b5aad2ba7   0.00
    fa3c0e904506607f7192a1d448f5ae78f57a7523   0.00
    8a6d59b792cb25798317b5b4e2e7a16589e873b1   1.00
    39c11ab924bbcea0498cb0a2dfe80a1ff49dff63   3.00
    aaeff59f952be5f94c939fc21d9441444a30a7b6   0.00
    91b483f19852504d3e72426b1fdf91a9ec17215a   7.00
    55140aa37babfb520483bdcacd4eeb7ec759ec95   0.00
    12b69ae6e35fee649c725be270c380392eac534b   0.00
    816f08291e3d28f5d10241a45e2a0e960085db72   1.00
    f987dc2c2f9836b66b8d22432ec3638ae960831b   0.00
    10e9d6511dc5bb2e250da691c6726d2e19b15e4c   1.00
    87525ea811f9d3d55c9e4ee54dad8648b46d89b6   5.00
    24905b5f9aa051b4180eb5d0338c5727eccb1191   1.00
    1e276f81a9fa6d403599cf51acfc8c64460df980   0.00
    776e1c0fc1f4ca712b30078f9a26d14869facabb   9.00
    d15d5be98c6f5b9953ae939b66eea49de152abd8   1.00
    62a87f7a2f96166215ad6cae2b3e7c869f184e9d   1.00
    1e1b62e21309c4408a96b1065a5154654d758ea0   1.00
    e1bf734bd951bc70527256e29daf454c46ed5fe0   2.00
    aa382546bf93e906129cdb467711efd29ea25ec7   1.00
    a8158261b9c5fa7b92de97b726f07d1d739840e7   1.00
    bd8043b89ee9f13739ff944c7e4efc153adfac1b   0.00
    eca5955549302beee4edff3166d494e431c8a4a8   5.00
    54254be01a6d115ebbc865dffa043987950e6e48   1.00
    78e4a922fcc9bd0617d0af9cd44be86ac2431129   1.00
    c92ad0932df8f0d74dde5dff449c9abce762c511   5.00
    Name: front_score, Length: 136062, dtype: float64

## Facebook brand page promotion (15 points)

One way a publisher has of promoting content is to post to their brand page. The significance of doing so is stronger when the brand page has more followers (likes).

$$ facebookPromotionProposed1_a = 15 \left( \frac {brandPageLikes_a} {\max(brandPageLikes)} \right) $$

Now lets explore the data to see if that makes sense. **tr;dr the formula above is incorrect**

```python
data.fb_brand_page_likes.max()
```

    45953133.0

```python
facebook_promotion_proposed_1 = np.ceil((15 * (data.fb_brand_page_likes / data.fb_brand_page_likes.max())).fillna(0))
```

```python
facebook_promotion_proposed_1.value_counts().sort_index().plot.bar()
```

```python
facebook_promotion_proposed_1.groupby(data.publisher_id).describe()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      publisher_id

      anotherangryvoice_blogspot_co_uk
      28.00
      1.00
      0.00
      1.00
      1.00
      1.00
      1.00
      1.00

      bbc_co_uk
      11509.00
      0.55
      2.81
      0.00
      0.00
      0.00
      0.00
      15.00

      breitbart_com
      2565.00
      0.78
      0.98
      0.00
      0.00
      0.00
      2.00
      2.00

      brexitcentral_com
      59.00
      0.86
      0.35
      0.00
      1.00
      1.00
      1.00
      1.00

      buzzfeed_com
      1323.00
      0.41
      0.49
      0.00
      0.00
      0.00
      1.00
      1.00

      cnn_com
      3898.00
      2.48
      4.32
      0.00
      0.00
      0.00
      0.00
      10.00

      dailymail_co_uk
      23015.00
      0.61
      1.64
      0.00
      0.00
      0.00
      0.00
      5.00

      economist_com
      501.00
      2.46
      1.16
      0.00
      3.00
      3.00
      3.00
      3.00

      evolvepolitics_com
      63.00
      0.95
      0.21
      0.00
      1.00
      1.00
      1.00
      1.00

      foxnews_com
      5592.00
      0.63
      1.84
      0.00
      0.00
      0.00
      0.00
      6.00

      ft_com
      4368.00
      0.49
      0.86
      0.00
      0.00
      0.00
      0.00
      2.00

      huffingtonpost_com
      2498.00
      1.91
      2.00
      0.00
      0.00
      0.00
      4.00
      4.00

      independent_co_uk
      5246.00
      0.63
      1.22
      0.00
      0.00
      0.00
      0.00
      3.00

      indy100_com
      434.00
      0.71
      0.46
      0.00
      0.00
      1.00
      1.00
      1.00

      lemonde_fr
      3765.00
      0.61
      0.92
      0.00
      0.00
      0.00
      2.00
      2.00

      libdemvoice_org
      149.00
      0.95
      0.21
      0.00
      1.00
      1.00
      1.00
      1.00

      mirror_co_uk
      9642.00
      0.23
      0.42
      0.00
      0.00
      0.00
      0.00
      1.00

      nbcnews_com
      1685.00
      2.61
      1.91
      0.00
      0.00
      4.00
      4.00
      4.00

      newstatesman_com
      418.00
      0.81
      0.39
      0.00
      1.00
      1.00
      1.00
      1.00

      npr_org
      1952.00
      1.44
      1.50
      0.00
      0.00
      0.00
      3.00
      3.00

      nytimes_com
      4352.00
      1.74
      2.53
      0.00
      0.00
      0.00
      5.00
      6.00

      order-order_com
      240.00
      0.75
      0.43
      0.00
      0.75
      1.00
      1.00
      1.00

      propublica_org
      41.00
      0.80
      0.40
      0.00
      1.00
      1.00
      1.00
      1.00

      reuters_com
      5653.00
      0.63
      0.93
      0.00
      0.00
      0.00
      2.00
      2.00

      rt_com
      2199.00
      1.08
      1.00
      0.00
      0.00
      2.00
      2.00
      2.00

      skwawkbox_org
      102.00
      1.00
      0.00
      1.00
      1.00
      1.00
      1.00
      1.00

      telegraph_co_uk
      6223.00
      0.40
      0.80
      0.00
      0.00
      0.00
      0.00
      2.00

      thecanary_co
      182.00
      0.98
      0.13
      0.00
      1.00
      1.00
      1.00
      1.00

      theguardian_com
      7750.00
      0.50
      1.12
      0.00
      0.00
      0.00
      0.00
      3.00

      thetimes_co_uk
      8336.00
      0.05
      0.22
      0.00
      0.00
      0.00
      0.00
      1.00

      washingtonpost_com
      21670.00
      0.16
      0.68
      0.00
      0.00
      0.00
      0.00
      3.00

      westmonster_com
      206.00
      0.40
      0.49
      0.00
      0.00
      0.00
      1.00
      1.00

      yournewswire_com
      398.00
      0.32
      0.47
      0.00
      0.00
      0.00
      1.00
      1.00

That's too much variation: sites like the Guardian, which have a respectable 7.5m likes, should not be scoring a 3. Lets try applying a log to it, and then standard feature scaling again.

```python
data.fb_brand_page_likes.groupby(data.publisher_id).max()
```

    publisher_id
    anotherangryvoice_blogspot_co_uk     330207.00
    bbc_co_uk                          45953133.00
    breitbart_com                       3827443.00
    brexitcentral_com                     15654.00
    buzzfeed_com                        2929729.00
    cnn_com                            29784431.00
    dailymail_co_uk                    14362724.00
    economist_com                       8474471.00
    evolvepolitics_com                   127618.00
    foxnews_com                        16239060.00
    ft_com                              3755253.00
    huffingtonpost_com                  9856230.00
    independent_co_uk                   8077017.00
    indy100_com                          239388.00
    lemonde_fr                          4058907.00
    libdemvoice_org                        8629.00
    mirror_co_uk                        2982278.00
    nbcnews_com                         9578008.00
    newstatesman_com                     154662.00
    npr_org                             6299668.00
    nytimes_com                        15379547.00
    order-order_com                       45610.00
    propublica_org                       379167.00
    reuters_com                         3978791.00
    rt_com                              4978221.00
    skwawkbox_org                          7280.00
    telegraph_co_uk                     4425050.00
    thecanary_co                         158031.00
    theguardian_com                     7891342.00
    thetimes_co_uk                       750928.00
    washingtonpost_com                  6147257.00
    westmonster_com                       18799.00
    yournewswire_com                      28301.00
    Name: fb_brand_page_likes, dtype: float64

```python
np.log(2149)
```

    7.6727578966425103

```python
np.log(data.fb_brand_page_likes.groupby(data.publisher_id).max())
```

    publisher_id
    anotherangryvoice_blogspot_co_uk   12.71
    bbc_co_uk                          17.64
    breitbart_com                      15.16
    brexitcentral_com                   9.66
    buzzfeed_com                       14.89
    cnn_com                            17.21
    dailymail_co_uk                    16.48
    economist_com                      15.95
    evolvepolitics_com                 11.76
    foxnews_com                        16.60
    ft_com                             15.14
    huffingtonpost_com                 16.10
    independent_co_uk                  15.90
    indy100_com                        12.39
    lemonde_fr                         15.22
    libdemvoice_org                     9.06
    mirror_co_uk                       14.91
    nbcnews_com                        16.07
    newstatesman_com                   11.95
    npr_org                            15.66
    nytimes_com                        16.55
    order-order_com                    10.73
    propublica_org                     12.85
    reuters_com                        15.20
    rt_com                             15.42
    skwawkbox_org                       8.89
    telegraph_co_uk                    15.30
    thecanary_co                       11.97
    theguardian_com                    15.88
    thetimes_co_uk                     13.53
    washingtonpost_com                 15.63
    westmonster_com                     9.84
    yournewswire_com                   10.25
    Name: fb_brand_page_likes, dtype: float64

That's more like it, but the lower numbers should be smaller.

```python
np.log(data.fb_brand_page_likes.groupby(data.publisher_id).max() / 1000)
```

    publisher_id
    anotherangryvoice_blogspot_co_uk    5.80
    bbc_co_uk                          10.74
    breitbart_com                       8.25
    brexitcentral_com                   2.75
    buzzfeed_com                        7.98
    cnn_com                            10.30
    dailymail_co_uk                     9.57
    economist_com                       9.04
    evolvepolitics_com                  4.85
    foxnews_com                         9.70
    ft_com                              8.23
    huffingtonpost_com                  9.20
    independent_co_uk                   9.00
    indy100_com                         5.48
    lemonde_fr                          8.31
    libdemvoice_org                     2.16
    mirror_co_uk                        8.00
    nbcnews_com                         9.17
    newstatesman_com                    5.04
    npr_org                             8.75
    nytimes_com                         9.64
    order-order_com                     3.82
    propublica_org                      5.94
    reuters_com                         8.29
    rt_com                              8.51
    skwawkbox_org                       1.99
    telegraph_co_uk                     8.40
    thecanary_co                        5.06
    theguardian_com                     8.97
    thetimes_co_uk                      6.62
    washingtonpost_com                  8.72
    westmonster_com                     2.93
    yournewswire_com                    3.34
    Name: fb_brand_page_likes, dtype: float64

```python
scaled_fb_brand_page_likes = (data.fb_brand_page_likes / 1000)
facebook_promotion_proposed_2 = np.ceil(\
    (15 * \
     (np.log(scaled_fb_brand_page_likes) / np.log(scaled_fb_brand_page_likes.max()))\
    )\
                                       ).fillna(0)
```

```python
facebook_promotion_proposed_2.groupby(data.publisher_id).max()
```

    publisher_id
    anotherangryvoice_blogspot_co_uk    9.00
    bbc_co_uk                          15.00
    breitbart_com                      12.00
    brexitcentral_com                   4.00
    buzzfeed_com                       12.00
    cnn_com                            15.00
    dailymail_co_uk                    14.00
    economist_com                      13.00
    evolvepolitics_com                  7.00
    foxnews_com                        14.00
    ft_com                             12.00
    huffingtonpost_com                 13.00
    independent_co_uk                  13.00
    indy100_com                         8.00
    lemonde_fr                         12.00
    libdemvoice_org                     4.00
    mirror_co_uk                       12.00
    nbcnews_com                        13.00
    newstatesman_com                    8.00
    npr_org                            13.00
    nytimes_com                        14.00
    order-order_com                     6.00
    propublica_org                      9.00
    reuters_com                        12.00
    rt_com                             12.00
    skwawkbox_org                       3.00
    telegraph_co_uk                    12.00
    thecanary_co                        8.00
    theguardian_com                    13.00
    thetimes_co_uk                     10.00
    washingtonpost_com                 13.00
    westmonster_com                     5.00
    yournewswire_com                    5.00
    Name: fb_brand_page_likes, dtype: float64

LGTM. So the equation is

$$ facebookPromotion_a = 15 \left(
\frac {\log(\frac {brandPageLikes_a}{1000})}
{\log(\frac {\max(brandPageLikes)}{1000}))} \right) $$

Now, let's try applying standard feature scaling approch to this, rather than using a magic number of 1,000. That equation would be:

\begin{align}
unscaledFacebookPromotion_a &=
   \log(brandPageLikes_a) \\
facebookPromotion_a &=
   15 \cdot \frac{unscaledFacebookPromotion_a - \min(unscaledFacebookPromotion)}{\max(unscaledFacebookPromotion) - \min(unscaledFacebookPromotion)}  \\
\\
\text{The scaling can be simplified to:} \\
facebookPromotion_a &=
   15 \cdot \frac{unscaledFacebookPromotion_a - \log(\min(brandPageLikes))}{\log(\max(brandPageLikes)) - \log(\min(brandPageLikes))}  \\
\\
\text{Meaning the overall equation becomes:} \\
facebookPromotion_a &=
   15 \cdot \frac{\log(brandPageLikes_a) - \log(\min(brandPageLikes))}{\log(\max(brandPageLikes)) - \log(\min(brandPageLikes))}
   \end{align}

```python
facebook_promotion_proposed_3 = np.ceil(
    (14 *
     (
         (np.log(data.fb_brand_page_likes) - np.log(data.fb_brand_page_likes.min()) ) /
         (np.log(data.fb_brand_page_likes.max()) - np.log(data.fb_brand_page_likes.min()))
     )
    ) + 1
                                       )
```

```python
facebook_promotion_proposed_3.groupby(data.publisher_id).max()
```

    publisher_id
    anotherangryvoice_blogspot_co_uk    8.00
    bbc_co_uk                          15.00
    breitbart_com                      12.00
    brexitcentral_com                   3.00
    buzzfeed_com                       11.00
    cnn_com                            15.00
    dailymail_co_uk                    14.00
    economist_com                      13.00
    evolvepolitics_com                  6.00
    foxnews_com                        14.00
    ft_com                             12.00
    huffingtonpost_com                 13.00
    independent_co_uk                  13.00
    indy100_com                         7.00
    lemonde_fr                         12.00
    libdemvoice_org                     2.00
    mirror_co_uk                       11.00
    nbcnews_com                        13.00
    newstatesman_com                    6.00
    npr_org                            12.00
    nytimes_com                        14.00
    order-order_com                     4.00
    propublica_org                      8.00
    reuters_com                        12.00
    rt_com                             12.00
    skwawkbox_org                       2.00
    telegraph_co_uk                    12.00
    thecanary_co                        6.00
    theguardian_com                    13.00
    thetimes_co_uk                      9.00
    washingtonpost_com                 12.00
    westmonster_com                     3.00
    yournewswire_com                    4.00
    Name: fb_brand_page_likes, dtype: float64

```python
data["facebook_promotion_score"] = facebook_promotion_proposed_3.fillna(0.0)
```

# Review

```python
data["promotion_score"] = (data.lead_score + data.front_score + data.facebook_promotion_score)
data["attention_index"] = (data.promotion_score + data.response_score)
```

```python
data.promotion_score.plot.hist(bins=np.arange(50), figsize=(15,6))
```

```python
data.attention_index.plot.hist(bins=np.arange(100), figsize=(15,6))
```

```python
data.attention_index.value_counts().sort_index()
```

    0.00     20300
    1.00     17446
    2.00     11190
    3.00      8050
    4.00      5819
    5.00      4612
    6.00      4321
    7.00      3502
    8.00      3230
    9.00      2833
    10.00     2778
    11.00     2473
    12.00     2249
    13.00     2144
    14.00     2048
    15.00     2005
    16.00     1848
    17.00     1782
    18.00     1652
    19.00     1634
    20.00     1623
    21.00     1545
    22.00     1478
    23.00     1479
    24.00     1475
    25.00     1439
    26.00     1365
    27.00     1230
    28.00     1293
    29.00     1178
             ...
    65.00      129
    66.00       99
    67.00       72
    68.00       64
    69.00       64
    70.00       59
    71.00       46
    72.00       34
    73.00       47
    74.00       41
    75.00       36
    76.00       24
    77.00       24
    78.00       34
    79.00       23
    80.00       25
    81.00       21
    82.00       20
    83.00       20
    84.00       19
    85.00       10
    86.00        9
    87.00       12
    88.00       15
    89.00        8
    90.00        2
    91.00        6
    92.00        7
    93.00        2
    94.00        5
    Name: attention_index, Length: 95, dtype: int64

```python
# and lets see the articles with the biggest attention index
data.sort_values("attention_index", ascending=False)
```

      url
      headline
      discovered
      published
      fb_engagements
      fb_max_engagements_per_min
      fb_max_engagements_per_min_time
      fb_comments
      fb_reactions
      fb_shares
      ...
      fb_brand_page_likes
      fb_brand_page_time
      alexa_rank
      word_count
      response_score
      lead_score
      front_score
      facebook_promotion_score
      promotion_score
      attention_index

      id

      50f3ffd91e936ee865166c1ba7ee6cd494cef168
      https://www.cnn.com/2018/02/11/health/aetna-ca...
      State launches Aetna probe after stunning admi...
      2018-02-11 21:16:21.717
      2018-02-11 21:11:31
      188224
      193.70
      2018-02-14T00:19:21.586Z
      44043
      109177
      35004
      ...
      29612675.00
      2018-02-11T22:30:02.000Z
      105
      1758.00
      50.00
      20.00
      9.00
      15.00
      44.00
      94.00

      bcde74d11c6932b426cd8629622a7a144de96318
      https://www.buzzfeed.com/briannasacks/the-fbi-...
      The FBI Was Warned About A School Shooting Thr...
      2018-02-15 09:25:25.051
      2018-02-15 07:38:51
      105241
      194.90
      2018-02-15T16:26:23.527Z
      29924
      54373
      20944
      ...
      2919016.00
      2018-02-15T09:43:35.000Z
      147
      502.00
      50.00
      19.00
      14.00
      11.00
      44.00
      94.00

      7fa31adc8e694a8f269a2d264135cbac1b284b30
      https://www.cnn.com/2018/02/16/politics/muelle...
      Grand jury returns indictment against 13 Russi...
      2018-02-16 18:13:14.937
      2018-02-16 18:04:10
      169116
      598.03
      2018-02-16T18:35:18.373Z
      51967
      93366
      23783
      ...
      29638206.00
      2018-02-16T18:14:03.000Z
      105
      66.00
      50.00
      20.00
      9.00
      15.00
      44.00
      94.00

      99ac15ddfc4a2219d9038bdac94f963bb9ee80a2
      http://money.cnn.com/2018/02/23/news/companies...
      Enterprise car rental ends NRA partnership
      2018-02-23 16:44:14.552
      2018-02-23 16:25:22
      371853
      1020.00
      2018-02-24T16:49:12.315Z
      47062
      295122
      29669
      ...
      29731553.00
      2018-02-24T16:11:42.000Z
      105
      nan
      50.00
      20.00
      9.00
      15.00
      44.00
      94.00

      3690e4bb3501e7ea4987e134a0ab4197b04eb22a
      https://www.cnn.com/2018/02/15/us/florida-shoo...
      These are the victims of the Florida school sh...
      2018-02-15 17:06:22.819
      2018-02-15 16:59:11
      148598
      407.25
      2018-02-16T10:03:11.380Z
      18324
      113279
      16995
      ...
      29660971.00
      2018-02-18T14:30:17.000Z
      105
      280.00
      50.00
      20.00
      9.00
      15.00
      44.00
      94.00

      fed2f0bddae438ac1ed84db4a496d38a55ee1b8e
      https://www.cnn.com/2018/02/03/politics/paul-r...
      Paul Ryan deletes tweet about tax cuts after T...
      2018-02-03 21:07:09.817
      2018-02-03 21:03:15
      105556
      190.11
      2018-02-05T00:07:11.101Z
      25550
      64114
      15892
      ...
      29587000.00
      2018-02-04T02:30:02.000Z
      105
      411.00
      50.00
      20.00
      8.00
      15.00
      43.00
      93.00

      ef30eb097153cc6c4ae903708b1af28058f8c30c
      https://www.cnn.com/2018/02/22/politics/donald...
      Trump proposes bonuses for teachers who get gu...
      2018-02-22 18:43:22.074
      2018-02-22 18:39:10
      239328
      556.07
      2018-02-22T20:21:18.030Z
      102984
      120967
      15377
      ...
      29720711.00
      2018-02-22T19:45:06.000Z
      105
      455.00
      50.00
      20.00
      8.00
      15.00
      43.00
      93.00

      595d15633260734dcd8d7d96b1f1995a38adc53d
      https://www.buzzfeed.com/remysmidt/heres-what-...
      Here’s What It’s Like At The Headquarters Of T...
      2018-02-20 05:25:13.116
      2018-02-20 05:08:34
      272546
      290.43
      2018-02-20T17:51:13.097Z
      18984
      224646
      28916
      ...
      2923003.00
      2018-02-20T15:02:00.000Z
      147
      1107.00
      50.00
      19.00
      12.00
      11.00
      42.00
      92.00

      3bd8bbc894078c77eec7ec36ec28065981d395dc
      https://www.cnn.com/2018/02/23/politics/parkla...
      Sources: Coral Springs police upset at some Br...
      2018-02-23 21:40:15.835
      2018-02-23 21:35:48
      79066
      111.20
      2018-02-23T22:14:03.424Z
      29161
      36820
      13085
      ...
      29727467.00
      2018-02-23T21:47:07.000Z
      105
      799.00
      48.00
      20.00
      9.00
      15.00
      44.00
      92.00

      732f13426df8d1ad17644d72ae3657f452da4477
      http://money.cnn.com/2018/02/12/news/economy/f...
      White House wants to deliver food to the poor,...
      2018-02-13 01:24:14.852
      2018-02-13 00:53:36
      108874
      252.67
      2018-02-13T22:04:09.451Z
      59616
      38976
      10282
      ...
      29645980.00
      2018-02-17T14:00:29.000Z
      105
      nan
      50.00
      20.00
      7.00
      15.00
      42.00
      92.00

      49533f2f5395c1a675fc03445724096204f87734
      https://www.buzzfeed.com/verabergengruen/comba...
      Combat Veterans Think Arming Teachers Is A Rea...
      2018-02-23 00:01:22.203
      2018-02-22 23:43:41
      120715
      62.73
      2018-02-24T16:30:08.356Z
      21933
      81291
      17491
      ...
      2925048.00
      2018-02-23T15:03:58.000Z
      147
      1188.00
      50.00
      19.00
      12.00
      11.00
      42.00
      92.00

      7384cb9d557758572326b7195b1d69c356c46f55
      http://www.bbc.co.uk/news/world-us-canada-4314...
      Trump backs guns for teachers
      2018-02-21 23:09:09.136
      2018-02-21 23:07:36
      174798
      327.68
      2018-02-22T20:26:11.741Z
      83346
      77047
      14405
      ...
      45877722.00
      2018-02-22T20:15:05.000Z
      96
      360.00
      50.00
      20.00
      7.00
      15.00
      42.00
      92.00

      7ef7490379626b4a008728ad8ae9b07ed7744828
      http://www.bbc.co.uk/news/world-us-canada-4306...
      Shooting at Florida school - reports
      2018-02-14 20:07:16.659
      2018-02-14 20:05:41
      96370
      337.93
      2018-02-14T20:30:04.815Z
      31532
      53174
      11664
      ...
      45794144.00
      2018-02-15T02:30:15.000Z
      96
      61.00
      49.00
      20.00
      8.00
      15.00
      43.00
      92.00

      43550ca5cdda76dd1e905ea2e7b1dcd9239587aa
      https://www.cnn.com/2018/02/22/politics/wayne-...
      The NRA's Wayne LaPierre to speak at CPAC on T...
      2018-02-22 14:19:26.847
      2018-02-22 14:15:30
      71741
      132.33
      2018-02-23T01:13:15.202Z
      29125
      33007
      9609
      ...
      29722659.00
      2018-02-23T01:00:19.000Z
      105
      363.00
      48.00
      20.00
      9.00
      15.00
      44.00
      92.00

      4d855aa92f114083340d536db271bdae9487e182
      https://www.buzzfeed.com/dominicholden/a-feder...
      A Federal Court Just Ruled For Gay Rights In A...
      2018-02-26 15:01:11.437
      2018-02-26 14:58:46
      75023
      121.00
      2018-02-26T20:55:09.383Z
      2529
      66121
      6373
      ...
      2926728.00
      2018-02-26T16:39:37.000Z
      147
      706.00
      48.00
      19.00
      13.00
      11.00
      43.00
      91.00

      d504855fd090afa4745d80839dfb1a2af86da1b5
      https://www.cnn.com/2018/02/26/politics/daca-s...
      Supreme Court won't hear Trump bid to end DACA...
      2018-02-26 14:49:24.336
      2018-02-26 14:44:00
      74782
      488.34
      2018-02-26T15:12:06.328Z
      6605
      59632
      8545
      ...
      29739965.00
      2018-02-26T14:59:05.000Z
      105
      92.00
      48.00
      20.00
      8.00
      15.00
      43.00
      91.00

      fcaae011c5530fa995d63ff29817f42a5ba0b7b1
      http://www.bbc.co.uk/news/world-us-canada-4314...
      US preacher Billy Graham dies
      2018-02-21 13:25:17.564
      2018-02-21 13:24:08
      81703
      382.35
      2018-02-21T14:20:13.936Z
      14894
      55099
      11710
      ...
      45865499.00
      2018-02-21T13:25:41.000Z
      96
      90.00
      48.00
      20.00
      8.00
      15.00
      43.00
      91.00

      44a85408d42ef8368e95485b0c3436490453e558
      https://www.cnn.com/2018/02/06/politics/milita...
      Washington Post: Trump tells Pentagon to plan ...
      2018-02-06 23:37:22.096
      2018-02-06 23:29:20
      133249
      189.77
      2018-02-07T06:04:09.090Z
      66239
      56156
      10854
      ...
      29599493.00
      2018-02-07T13:00:13.000Z
      105
      160.00
      50.00
      20.00
      6.00
      15.00
      41.00
      91.00

      bdec0fff8f7508de17de5eaa899cf4272dd9fc72
      https://www.cnn.com/2018/02/22/us/florida-scho...
      Security footage from school massacre scene wa...
      2018-02-22 16:19:30.029
      2018-02-22 16:13:27
      94072
      140.82
      2018-02-23T01:25:16.843Z
      48710
      34633
      10729
      ...
      29721683.00
      2018-02-22T22:02:04.000Z
      105
      148.00
      49.00
      20.00
      7.00
      15.00
      42.00
      91.00

      030598f1d5c669d3e3adef3be9ededb9bfe98e2f
      https://www.cnn.com/2018/02/20/us/florida-legi...
      Florida House votes down weapons ban considera...
      2018-02-20 22:52:18.624
      2018-02-20 22:50:21
      136107
      517.77
      2018-02-21T01:32:08.912Z
      36663
      83885
      15559
      ...
      29683383.00
      2018-02-21T01:06:49.000Z
      105
      61.00
      50.00
      20.00
      6.00
      15.00
      41.00
      91.00

      3e5b38ac8f7dd73b8f3b3d6037d14415b88372ed
      https://www.cnn.com/2018/02/23/us/florida-scho...
      Armed deputy stayed outside during Florida sch...
      2018-02-23 09:52:18.646
      2018-02-23 09:45:08
      94821
      138.37
      2018-02-23T16:03:12.436Z
      44086
      39807
      10928
      ...
      29725457.00
      2018-02-23T11:37:03.000Z
      105
      1226.00
      49.00
      20.00
      6.00
      15.00
      41.00
      90.00

      0ad0a1c1c17aa19b40632bfd7c7f2a3a6dcb4f30
      https://www.cnn.com/2018/02/09/politics/democr...
      Trump won't declassify Democratic memo, sends ...
      2018-02-10 01:07:22.787
      2018-02-10 01:01:27
      78340
      326.98
      2018-02-10T01:29:16.517Z
      33926
      34276
      10138
      ...
      29607746.00
      2018-02-10T01:10:22.000Z
      105
      245.00
      48.00
      20.00
      7.00
      15.00
      42.00
      90.00

      81832cf7eda554c7263210894d3e33032d71906f
      https://www.cnn.com/2018/02/25/politics/cnn-po...
      CNN Poll: Trump approval slides, matches lowes...
      2018-02-25 14:07:19.613
      2018-02-25 14:00:15
      46003
      136.21
      2018-02-25T16:53:10.637Z
      16092
      24226
      5685
      ...
      29736526.00
      2018-02-25T16:35:35.000Z
      105
      485.00
      45.00
      20.00
      9.00
      15.00
      44.00
      89.00

      9f5cd7c4119f7da78c9c94708ff4a20d121afc9e
      https://www.cnn.com/2018/02/24/politics/mexica...
      WH official: Peña Nieto calls off visit to Whi...
      2018-02-25 01:31:13.954
      2018-02-25 01:24:45
      43112
      133.80
      2018-02-25T19:16:09.811Z
      10601
      26121
      6390
      ...
      29736950.00
      2018-02-25T19:08:24.000Z
      105
      277.00
      45.00
      20.00
      9.00
      15.00
      44.00
      89.00

      fbb9012b3f033f1ce41927a700146d6d56c37f96
      https://www.cnn.com/2018/02/16/politics/muelle...
      Robert Mueller just made it impossible for Tru...
      2018-02-16 20:10:16.588
      2018-02-16 20:05:10
      47510
      135.14
      2018-02-16T23:43:15.792Z
      10099
      32107
      5304
      ...
      29639980.00
      2018-02-16T23:28:01.000Z
      105
      919.00
      45.00
      20.00
      9.00
      15.00
      44.00
      89.00

      090aad188705194f32d7cbf2adf6cf7b171ca6f0
      https://www.cnn.com/2018/02/03/politics/donald...
      Trump tweets on Nunes memo
      2018-02-03 15:10:18.821
      2018-02-03 15:05:03
      54179
      423.11
      2018-02-03T16:14:09.449Z
      20525
      30935
      2719
      ...
      29585302.00
      2018-02-03T16:00:07.000Z
      105
      145.00
      46.00
      20.00
      8.00
      15.00
      43.00
      89.00

      7a658ecfdc344e3796e0935b6cd913534c0fa2a4
      https://www.cnn.com/2018/02/19/politics/muelle...
      Mueller's interest in Kushner grows to include...
      2018-02-19 21:07:18.742
      2018-02-19 21:00:59
      46157
      231.53
      2018-02-19T21:30:07.494Z
      8855
      30099
      7203
      ...
      29674422.00
      2018-02-19T21:16:05.000Z
      105
      806.00
      45.00
      20.00
      9.00
      15.00
      44.00
      89.00

      d41cdce11a5fda07196fbab9e1f327219ae2a2b0
      https://www.cnn.com/2018/02/14/us/florida-high...
      Shooting reported at Florida high school
      2018-02-14 20:07:20.293
      2018-02-14 20:02:51
      508808
      1012.36
      2018-02-15T00:16:13.133Z
      161675
      286961
      60172
      ...
      29622210.00
      2018-02-14T20:08:06.000Z
      105
      32.00
      50.00
      20.00
      4.00
      15.00
      39.00
      89.00

      6b77e8af4ad35821df4e7a4613306dbb92629308
      https://www.buzzfeed.com/adambvary/black-panth...
      "Black Panther" Is An Unprecedented Box Office...
      2018-02-18 16:34:20.965
      2018-02-18 16:26:53
      68194
      84.76
      2018-02-18T18:48:11.622Z
      7158
      54349
      6687
      ...
      2922134.00
      2018-02-18T17:50:10.000Z
      147
      1976.00
      47.00
      19.00
      12.00
      11.00
      42.00
      89.00

      76d1c6e03c5d5273ee6a33d73da8a50b25b431a5
      https://www.cnn.com/2018/02/20/politics/robert...
      Mueller charges lawyer with lying about intera...
      2018-02-20 14:40:11.669
      2018-02-20 14:33:43
      45215
      247.18
      2018-02-20T16:18:11.270Z
      8388
      30749
      6078
      ...
      29679686.00
      2018-02-20T14:51:32.000Z
      105
      111.00
      45.00
      20.00
      9.00
      15.00
      44.00
      89.00

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      711226f4cc221161fc34c3cd126c1f20c2c1c05d
      http://www.bbc.co.uk/sport/scotland/43140856
      Gold Coast 2018: Scots kit up for Commonwealth...
      2018-02-21 09:46:04.217
      2018-02-21 09:44:08
      0
      0.00
      2018-02-22T00:10:05.823Z
      0
      0
      0
      ...
      nan
      NaN
      96
      nan
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      50a513e1ef2b483ff3f40373dcf62a3f1c40cbc0
      http://www.dailymail.co.uk/home/event/article-...
      Chris Evans takes a spin in the Seat Arona FR ...
      2018-02-17 22:04:45.718
      2018-02-17 22:01:01
      0
      0.00
      2018-02-18T05:23:07.502Z
      0
      0
      0
      ...
      nan
      NaN
      158
      1082.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      6b95aa067d195e0a470d2c6ab8222a13d5a0f5cf
      http://www.dailymail.co.uk/home/event/article-...
      Rag'n'Bone Man review: Sings the blues superbly
      2018-02-17 22:04:23.862
      2018-02-17 22:01:05
      1
      0.02
      2018-02-18T08:25:10.269Z
      0
      0
      1
      ...
      nan
      NaN
      158
      773.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      11a20b75e5c28495503c361837bdc3b4a3d7187d
      http://www.dailymail.co.uk/home/event/article-...
      lolanthe review: All the fun of the fairytale
      2018-02-17 22:04:39.971
      2018-02-17 22:01:19
      1
      0.02
      2018-02-18T08:26:07.367Z
      0
      0
      1
      ...
      nan
      NaN
      158
      719.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      d5a14386776ba2ca4de4c52e28223148d8e37351
      http://www.dailymail.co.uk/home/event/article-...
      The Winslow Boy review: Timely in its feminism
      2018-02-17 22:04:39.159
      2018-02-17 22:01:27
      1
      0.02
      2018-02-18T08:26:07.346Z
      0
      0
      1
      ...
      nan
      NaN
      158
      760.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      f911c1f884be46a59314570c419fcacc5f15e1e5
      http://www.dailymail.co.uk/travel/article-5403...
      1930s New York? Just turn right at London's Hy...
      2018-02-17 22:04:44.779
      2018-02-17 22:01:51
      0
      0.00
      2018-02-18T12:29:11.133Z
      0
      0
      0
      ...
      nan
      NaN
      158
      318.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      1f44623aa1b251473b4560d597d14a2dd7abc2d1
      https://www.washingtonpost.com/national/author...
      Authorities say one officer has died, two depu...
      2018-02-09 19:16:19.407
      2018-02-09 19:13:11
      0
      0.00
      2018-02-10T06:38:09.956Z
      0
      0
      0
      ...
      nan
      NaN
      191
      37.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      85b6b4db2f9c797adcd982653fc066ac716e1f42
      http://www.dailymail.co.uk/travel/article-5403...
      Lilies and the Louvre, you can find all of the...
      2018-02-17 22:04:27.109
      2018-02-17 22:02:04
      0
      0.00
      2018-02-18T12:28:23.715Z
      0
      0
      0
      ...
      nan
      NaN
      158
      919.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      a595eeda63180aebd0f6ee93032dd70ee03fb4cd
      http://www.dailymail.co.uk/tvshowbiz/article-5...
      Roxy Jacenko flashes her fading bruises from b...
      2018-02-23 12:10:29.917
      2018-02-23 12:06:46
      0
      0.00
      2018-02-23T12:22:14.299Z
      0
      0
      0
      ...
      nan
      NaN
      158
      539.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      24f65f6b2ea7f757ab7be360496a3afdb1b7e36b
      https://www.theguardian.com/sport/2018/feb/17/...
      Wales’ Ken Owens says Warren Gatland deserves ...
      2018-02-17 22:01:10.159
      2018-02-17 21:59:26
      0
      0.00
      2018-02-18T01:15:07.005Z
      0
      0
      0
      ...
      nan
      NaN
      142
      598.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      ba7b0867b6a77b0d78940111baa895ea8885374e
      http://www.dailymail.co.uk/tvshowbiz/article-5...
      Australian Spartan contestants told to cover u...
      2018-02-26 08:34:10.346
      2018-02-26 08:30:26
      0
      0.00
      2018-02-26T09:46:12.872Z
      0
      0
      0
      ...
      nan
      NaN
      158
      376.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      1eb3f2a266ee8bb74f625c0ed78b628538ad55e1
      http://www.bbc.co.uk/news/world-europe-43196047
      Bank of Ireland to pay first dividend since cr...
      2018-02-26 08:34:10.155
      2018-02-26 08:30:34
      1
      0.02
      2018-02-26T09:46:12.852Z
      0
      0
      1
      ...
      nan
      NaN
      96
      101.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      f96f3971ba36b30240fbf6530485b0cf6f6bc9b3
      https://www.washingtonpost.com/sports/colleges...
      Persons, El-Amin help Ball State sweep Toledo ...
      2018-02-17 22:01:14.952
      2018-02-17 21:52:42
      0
      0.00
      2018-02-18T07:21:06.620Z
      0
      0
      0
      ...
      nan
      NaN
      191
      203.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      c33b12cd48e84e4af4c54549cd12d0f84167d3e0
      https://www.ft.com/content/e732c9f2-0da5-11e8-...
      Direct Line profits surge thanks to own-brand ...
      2018-02-09 19:22:20.171
      2018-02-09 19:20:27
      0
      0.00
      2018-02-10T15:52:02.478Z
      0
      0
      0
      ...
      nan
      NaN
      1596
      nan
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      cb02a50107e39382d8a5eebef27aa17930fa1433
      https://www.washingtonpost.com/sports/colleges...
      Waters’ field goal gives LSU 64-63 win over Mi...
      2018-02-17 21:52:15.082
      2018-02-17 21:47:19
      0
      0.00
      2018-02-18T06:11:03.657Z
      0
      0
      0
      ...
      nan
      NaN
      191
      275.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      2ad4f536947f32f136d4129e9a47a2ca157ddb48
      http://www.independent.co.uk/news/business/new...
      Primark: Unseasonably warm Autumn weather expe...
      2018-02-26 08:43:15.140
      2018-02-26 08:38:32
      0
      0.00
      2018-02-26T08:54:16.338Z
      0
      0
      0
      ...
      nan
      NaN
      386
      227.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      171af42d03b106505bef6807435978615b4ad026
      https://www.washingtonpost.com/politics/ap-cou...
      AP count: Nearly 11.8M enroll for Obama health...
      2018-02-07 17:13:12.806
      2018-02-07 17:00:11
      0
      0.00
      2018-02-07T22:29:09.796Z
      0
      0
      0
      ...
      nan
      NaN
      191
      148.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      e0e74d851ce9fa2338acd0458feb5f78ca10a915
      https://www.washingtonpost.com/sports/olympics...
      Goggia gets gold; Russian women close in on en...
      2018-02-21 09:58:11.501
      2018-02-21 09:47:25
      0
      0.00
      2018-02-22T01:23:23.118Z
      0
      0
      0
      ...
      nan
      NaN
      191
      779.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      35cc3c9fae6aba00a1be05234b8a956ae7bfa0a3
      https://www.washingtonpost.com/sports/colleges...
      Walker, Avery lead Kent State past Akron 78-68
      2018-02-17 22:01:17.209
      2018-02-17 21:51:14
      0
      0.00
      2018-02-18T07:21:06.635Z
      0
      0
      0
      ...
      nan
      NaN
      191
      192.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      8cd1d739c76b84a71c3ae4b21403212e2d7df2c5
      https://www.washingtonpost.com/business/greece...
      Greece: Parliament debates investigation over ...
      2018-02-21 09:58:14.030
      2018-02-21 09:46:15
      0
      0.00
      2018-02-22T01:23:23.123Z
      0
      0
      0
      ...
      nan
      NaN
      191
      147.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      40df170ea6c8c2eecf343fafea8038a8d588e094
      https://www.washingtonpost.com/local/winning-n...
      Winning numbers drawn in ‘Play 3 Day’ game
      2018-02-09 19:25:22.200
      2018-02-09 19:19:09
      0
      0.00
      2018-02-10T13:53:25.608Z
      0
      0
      0
      ...
      nan
      NaN
      191
      41.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      3cbc373ce713b17ee5344c5bec0b09febac97b7f
      https://www.washingtonpost.com/local/winning-n...
      Winning numbers drawn in ‘DC 4 Midday’ game
      2018-02-09 19:25:16.895
      2018-02-09 19:18:43
      0
      0.00
      2018-02-09T19:37:19.606Z
      0
      0
      0
      ...
      nan
      NaN
      191
      43.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      379297e818cbd90b451670440e6ebf0fe2b20595
      https://www.washingtonpost.com/sports/colleges...
      Duke’s Bagley to miss 3rd game with sprained knee
      2018-02-17 22:01:14.885
      2018-02-17 21:53:27
      0
      0.00
      2018-02-18T01:16:02.583Z
      0
      0
      0
      ...
      nan
      NaN
      191
      114.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      915ce8988a6bcf9af2c8888e99e7fc06d0f631be
      https://www.washingtonpost.com/local/winning-n...
      Winning numbers drawn in ‘Pick 3 Day’ game
      2018-02-09 19:25:15.729
      2018-02-09 19:19:07
      0
      0.00
      2018-02-10T16:56:10.146Z
      0
      0
      0
      ...
      nan
      NaN
      191
      41.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      65f713ec044f55558fb6ecc552d6d8e99c6fea03
      https://www.washingtonpost.com/sports/colleges...
      Auburn starter McLemore taken off court on str...
      2018-02-17 22:13:20.401
      2018-02-17 21:58:13
      0
      0.00
      2018-02-18T09:35:08.984Z
      0
      0
      0
      ...
      nan
      NaN
      191
      174.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      f1f614f19c33ea71f7aee8681849f6ee434345eb
      https://www.washingtonpost.com/local/winning-n...
      Winning numbers drawn in ‘Play 4 Day’ game
      2018-02-09 19:25:18.532
      2018-02-09 19:19:02
      0
      0.00
      2018-02-09T19:37:19.621Z
      0
      0
      0
      ...
      nan
      NaN
      191
      42.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      78425473e026b8bcc3cf28d82405fd2b7dfc846b
      https://www.washingtonpost.com/local/winning-n...
      Winning numbers drawn in ‘DC 5 Midday’ game
      2018-02-09 19:25:25.498
      2018-02-09 19:18:55
      0
      0.00
      2018-02-10T13:53:25.644Z
      0
      0
      0
      ...
      nan
      NaN
      191
      44.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      b689f14b3ba9d55680eb1ab90725ba5458765cb6
      https://www.washingtonpost.com/sports/national...
      Jays hope healthy Sanchez can stabilize starti...
      2018-02-09 19:31:16.532
      2018-02-09 19:18:53
      0
      0.00
      2018-02-10T13:59:10.384Z
      0
      0
      0
      ...
      nan
      NaN
      191
      543.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      5deb3bc1272e04f98cffa28ee49abb2bb27358b1
      http://www.bbc.co.uk/sport/wales/43193037
      Julie Hoornweg: Wales netball appoint Australi...
      2018-02-26 08:34:12.629
      2018-02-26 08:30:52
      0
      0.00
      2018-02-26T09:46:12.906Z
      0
      0
      0
      ...
      nan
      NaN
      96
      nan
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

      f6635f20d7e72231a7904999f56b057e5ff967c1
      https://www.washingtonpost.com/national/veteri...
      Veterinarian accused of fatally shooting neigh...
      2018-02-22 15:01:09.063
      2018-02-22 14:57:17
      0
      0.00
      2018-02-23T07:27:07.458Z
      0
      0
      0
      ...
      nan
      NaN
      191
      141.00
      0.00
      0.00
      0.00
      0.00
      0.00
      0.00

136062 rows × 26 columns

```python
data["score_diff"] = data.promotion_score - data.response_score
```

```python
# promoted but low response
data.sort_values("score_diff", ascending=False).head(25)
```

      url
      headline
      discovered
      published
      fb_engagements
      fb_max_engagements_per_min
      fb_max_engagements_per_min_time
      fb_comments
      fb_reactions
      fb_shares
      ...
      fb_brand_page_time
      alexa_rank
      word_count
      response_score
      lead_score
      front_score
      facebook_promotion_score
      promotion_score
      attention_index
      score_diff

      id

      2dd7184cf1962f76dc17a9d4b910c9c961c4a639
      https://www.buzzfeed.com/stephaniemcneal/monat
      This Trendy Hair Company Is Suing A Woman Who ...
      2018-02-07 19:37:11.282
      2018-02-01 15:42:38
      9
      0.82
      2018-02-07T19:49:11.063Z
      0
      0
      9
      ...
      2018-02-07T20:39:47.000Z
      147
      1673.00
      2.00
      19.00
      15.00
      11.00
      45.00
      47.00
      43.00

      ccef80e232f379f5df12683200476c17734b8d49
      https://www.buzzfeed.com/patrickstrudwick/this...
      This Man Is Planning To Open The World's Large...
      2018-02-27 08:09:13.612
      2018-02-23 12:25:59
      4
      0.09
      2018-02-27T08:21:08.839Z
      0
      2
      2
      ...
      2018-02-27T14:54:08.000Z
      147
      1530.00
      1.00
      19.00
      14.00
      11.00
      44.00
      45.00
      43.00

      77b260359cdb03b2a160441e435208aec0755826
      https://www.buzzfeed.com/katebubacz/what-happe...
      What Happened After Standing Rock?
      2018-02-28 22:04:16.683
      2018-02-22 23:43:47
      0
      0.00
      2018-02-28T22:15:21.682Z
      0
      0
      0
      ...
      2018-03-01T00:52:51.000Z
      147
      4975.00
      0.00
      19.00
      13.00
      11.00
      43.00
      43.00
      43.00

      872a77a39949fce75a819caa9b57ff25eda8d1d7
      https://www.buzzfeed.com/hannahallam/inside-th...
      Inside The Uneasy Fight To End Female Genital ...
      2018-02-05 14:04:16.865
      2018-02-01 21:06:42
      1
      0.09
      2018-02-05T14:16:13.088Z
      0
      0
      1
      ...
      2018-02-05T16:14:53.000Z
      147
      2097.00
      0.00
      19.00
      12.00
      11.00
      42.00
      42.00
      42.00

      dd3b404c6f7743717957de0cb9db0a63a86bc385
      https://www.buzzfeed.com/buzzfeednews/heres-ho...
      Here's How Many Medals The US Has Won So Far A...
      2018-02-11 02:44:12.255
      2018-02-09 18:28:27
      1
      0.10
      2018-02-11T02:55:18.851Z
      0
      0
      1
      ...
      2018-02-14T03:18:57.000Z
      147
      nan
      0.00
      19.00
      12.00
      11.00
      42.00
      42.00
      42.00

      5a88c2c270cd23fbb4caa33f9f592bd2eb8b7979
      https://www.buzzfeed.com/emaoconnor/columbia-t...
      She “Provoked” Him: Columbia’s Old Policies Le...
      2018-03-01 16:49:33.800
      2018-02-26 23:23:53
      2
      0.18
      2018-03-01T17:01:22.335Z
      0
      0
      2
      ...
      2018-03-01T17:11:17.000Z
      147
      3974.00
      1.00
      19.00
      12.00
      11.00
      42.00
      43.00
      41.00

      367ce38e6802afe4b09fa8562631c7abedebbe08
      https://www.buzzfeed.com/carolinekee/juul-ecig...
      The JUUL Is An Insanely Popular E-Cigarette Bu...
      2018-02-05 22:52:21.271
      2018-02-02 18:38:13
      2
      0.18
      2018-02-05T23:04:10.998Z
      0
      0
      2
      ...
      2018-02-06T01:14:32.000Z
      147
      83.00
      1.00
      19.00
      10.00
      11.00
      40.00
      41.00
      39.00

      08bd4bc0640627333b78efecb6c02c69f2eb2bdc
      https://www.buzzfeed.com/mitchprothero/their-p...
      Their Parents Fought With ISIS. Now France Is ...
      2018-02-19 14:04:23.387
      2018-02-18 20:55:53
      23
      0.10
      2018-02-19T15:17:08.351Z
      2
      4
      17
      ...
      2018-02-19T21:49:19.000Z
      147
      688.00
      4.00
      19.00
      13.00
      11.00
      43.00
      47.00
      39.00

      77e20f89fc8c29a4e763cc24cd844a37e40b378f
      https://www.buzzfeed.com/alisonwillmore/fifty-...
      Difficult Men Meet Their Match In “Fifty Shade...
      2018-02-15 18:04:25.149
      2018-02-14 17:26:42
      2
      0.18
      2018-02-15T18:16:11.035Z
      0
      0
      2
      ...
      2018-02-19T03:01:00.000Z
      147
      2552.00
      1.00
      17.00
      12.00
      11.00
      40.00
      41.00
      39.00

      67e78216ef055244f9675cb06748fe507a200e7a
      https://www.buzzfeed.com/tamerragriffin/this-i...
      This Is What It's Like When Your Country Runs ...
      2018-02-03 15:49:21.870
      2018-02-02 16:31:03
      24
      0.36
      2018-02-03T17:02:06.588Z
      0
      3
      21
      ...
      2018-02-03T20:14:41.000Z
      147
      1718.00
      4.00
      19.00
      12.00
      11.00
      42.00
      46.00
      38.00

      532f617cc49df5e98cfc7e3735f90e5173fd45f0
      https://www.buzzfeed.com/albertsamaha/this-tee...
      This Teenager Accused Two On-Duty Cops Of Rape...
      2018-02-07 10:31:16.262
      2018-02-05 18:14:44
      0
      0.00
      2018-02-07T10:43:11.846Z
      0
      0
      0
      ...
      2018-02-07T12:10:33.000Z
      147
      3359.00
      0.00
      19.00
      8.00
      11.00
      38.00
      38.00
      38.00

      318ecacd91e6c5e9d74e394f6b74c4402d90924c
      https://www.buzzfeed.com/talalansari/porter
      Trump's Spokesman Says The White House "Could ...
      2018-02-08 21:04:23.795
      2018-02-08 21:01:47
      73
      0.51
      2018-02-08T22:17:13.692Z
      11
      38
      24
      ...
      2018-02-08T21:21:04.000Z
      147
      267.00
      8.00
      19.00
      15.00
      11.00
      45.00
      53.00
      37.00

      d4b243e11f6e9b4ffdd49fa4aed40701f07bfa3c
      https://www.buzzfeed.com/alexkantrowitz/facebo...
      Facebook Won't Release Its “5% Of News Feed Co...
      2018-02-18 17:55:16.027
      2018-02-17 20:04:45
      13
      0.11
      2018-02-18T20:09:10.239Z
      1
      3
      9
      ...
      2018-02-19T20:22:13.000Z
      147
      1056.00
      3.00
      19.00
      10.00
      11.00
      40.00
      43.00
      37.00

      5d57a4e0bd91a2cb50cdba0b5fee6c928f18fc68
      https://www.buzzfeed.com/louispeitzman/2018-ac...
      Here Are The Winners At The 2018 Academy Awards
      2018-03-05 01:29:15.278
      2018-02-28 21:19:39
      7
      0.30
      2018-03-05T01:40:16.367Z
      0
      0
      7
      ...
      2018-03-05T05:55:00.000Z
      147
      nan
      2.00
      19.00
      8.00
      11.00
      38.00
      40.00
      36.00

      61c98348add4be38653ef9956749854447f62362
      https://www.buzzfeed.com/tamerragriffin/south-...
      South Africa’s “Teflon President” Might Be Fin...
      2018-02-13 18:07:21.156
      2018-02-13 17:47:23
      38
      0.11
      2018-02-14T05:29:09.692Z
      10
      17
      11
      ...
      2018-02-14T21:17:09.000Z
      147
      nan
      6.00
      19.00
      12.00
      11.00
      42.00
      48.00
      36.00

      984ae7c0ec9960f8cc82916a7df12284d388d333
      https://www.buzzfeed.com/nicolenguyen/homepod-...
      HomePod Review: A Smart Speaker For iPhone Use...
      2018-02-06 13:31:31.366
      2018-02-06 02:47:46
      48
      0.28
      2018-02-06T13:43:07.624Z
      3
      25
      20
      ...
      2018-02-06T16:45:04.000Z
      147
      2652.00
      7.00
      17.00
      14.00
      11.00
      42.00
      49.00
      35.00

      4baa6af1c4de278dd08b7edd01e038af97ba3a00
      https://www.buzzfeed.com/michaelblackmon/taran...
      Quentin Tarantino Gave His Side Of The Story A...
      2018-02-06 17:49:21.235
      2018-02-06 17:18:57
      126
      0.48
      2018-02-06T23:05:08.798Z
      5
      105
      16
      ...
      2018-02-06T20:05:27.000Z
      147
      493.00
      11.00
      19.00
      15.00
      11.00
      45.00
      56.00
      34.00

      5a00b0b5556c3808f5b79ed69b6cd76daff85f03
      https://www.buzzfeed.com/venessawong/mcdonalds...
      McDonald’s And Uber Eats Need Each Now Other M...
      2018-02-15 18:25:18.096
      2018-02-15 18:24:58
      32
      0.09
      2018-02-15T18:37:09.793Z
      4
      14
      14
      ...
      2018-02-16T02:44:41.000Z
      147
      1064.00
      5.00
      19.00
      9.00
      11.00
      39.00
      44.00
      34.00

      4588bd12d8ef2558abb2281970a2cf997a2ca5ad
      https://www.buzzfeed.com/lesterfeder/italian-e...
      Meet The Politician Getting Death Threats For ...
      2018-02-24 15:31:11.736
      2018-02-23 11:45:54
      1
      0.09
      2018-02-24T15:43:07.081Z
      0
      0
      1
      ...
      NaN
      147
      3405.00
      0.00
      19.00
      14.00
      0.00
      33.00
      33.00
      33.00

      0cc686fee28a082c58f5857f8405aad3cd473126
      https://www.buzzfeed.com/borzoudaragahi/unfair...
      This Man Was Arrested 10 Times For Being An IS...
      2018-02-25 14:01:19.862
      2018-02-22 09:37:12
      0
      0.00
      2018-02-25T14:12:16.332Z
      0
      0
      0
      ...
      NaN
      147
      3921.00
      0.00
      19.00
      14.00
      0.00
      33.00
      33.00
      33.00

      4cd1abee9ee0a70a28db7692076cdb64290d2558
      https://www.buzzfeed.com/paulmcleod/the-govern...
      The Government Could Shut Down Again Because C...
      2018-02-01 22:04:44.109
      2018-02-01 22:02:51
      154
      0.49
      2018-02-02T04:21:16.604Z
      49
      75
      30
      ...
      2018-02-02T03:54:46.000Z
      147
      921.00
      12.00
      19.00
      15.00
      11.00
      45.00
      57.00
      33.00

      18207f66ea49fc0711d7b656f361bf36c280696e
      https://www.buzzfeed.com/pdominguez/how-omaros...
      How Omarosa Really Won Big Brother
      2018-02-28 21:13:17.272
      2018-02-28 19:49:55
      94
      0.82
      2018-02-28T21:25:18.654Z
      46
      18
      30
      ...
      2018-02-28T21:10:28.000Z
      147
      2199.00
      10.00
      19.00
      13.00
      11.00
      43.00
      53.00
      33.00

      88fbb92d88927378ecc08cb51f0526e5b1311156
      https://www.buzzfeed.com/ryanmac/this-is-the-m...
      This Man Helped Peter Thiel Demolish Gawker
      2018-02-23 19:10:19.656
      2018-02-23 17:38:58
      140
      0.55
      2018-02-23T19:22:07.651Z
      3
      60
      77
      ...
      2018-02-23T23:24:56.000Z
      147
      2203.00
      11.00
      19.00
      13.00
      11.00
      43.00
      54.00
      32.00

      51fc22587b0f28c4f7777610bdce52994171f9b8
      https://www.buzzfeed.com/borzoudaragahi/what-h...
      This Woman Spoke Out Against A War And Then Po...
      2018-02-11 15:34:18.446
      2018-02-09 20:32:21
      2
      0.18
      2018-02-11T15:46:13.755Z
      0
      0
      2
      ...
      NaN
      147
      1347.00
      1.00
      19.00
      14.00
      0.00
      33.00
      34.00
      32.00

      b34b52dcdcedb91fdf22c99651187e527adeecd4
      https://www.buzzfeed.com/mbvd/a-new-report-on-...
      A New Report On The Hawaii Missile Alert Says ...
      2018-02-21 01:31:19.349
      2018-02-21 00:54:53
      64
      0.28
      2018-02-21T04:46:10.051Z
      12
      33
      19
      ...
      2018-02-21T03:22:00.000Z
      147
      598.00
      8.00
      19.00
      10.00
      11.00
      40.00
      48.00
      32.00

25 rows × 27 columns

```python
# high response but not promoted
data.sort_values("score_diff", ascending=True).head(25)
```

      url
      headline
      discovered
      published
      fb_engagements
      fb_max_engagements_per_min
      fb_max_engagements_per_min_time
      fb_comments
      fb_reactions
      fb_shares
      ...
      fb_brand_page_time
      alexa_rank
      word_count
      response_score
      lead_score
      front_score
      facebook_promotion_score
      promotion_score
      attention_index
      score_diff

      id

      5e21237490a319711d22eea26dc26a4c3682dea0
      https://www.washingtonpost.com/news/reliable-s...
      Dolly Parton likes to give away books. She jus...
      2018-02-27 19:13:20.537
      2018-02-27 19:02:15.000
      114572
      448.59
      2018-02-28T06:02:20.498Z
      7737
      96894
      9941
      ...
      NaN
      191
      322.00
      50.00
      0.00
      0.00
      0.00
      0.00
      50.00
      -50.00

      9e82991a7f4b60377f15ada5d5bb9f02c57a892f
      https://www.washingtonpost.com/news/national/w...
      Armed deputy on duty at Fla. school waited out...
      2018-02-22 23:14:14.096
      2018-02-22 23:10:00.000
      201911
      385.05
      2018-02-23T00:09:16.414Z
      93017
      86962
      21932
      ...
      NaN
      191
      102.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      0663587e18695b514a676ef31678320225937272
      http://www.foxnews.com/us/2018/02/13/delaware-...
      Delaware may let students ‘choose’ their race,...
      2018-02-20 20:14:20.100
      2018-02-20 20:01:33.000
      126486
      170.66
      2018-02-21T05:54:20.144Z
      44600
      69253
      12633
      ...
      NaN
      285
      827.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      fb1944a99013e1b35fd6a91c48804999102e083e
      https://www.nbcnews.com/politics/donald-trump/...
      Trump Signs Bill Revoking Obama-Era Gun Checks...
      2018-02-15 21:25:15.475
      2018-02-15 21:23:07.000
      1733438
      4222.05
      2018-02-17T21:37:14.123Z
      579201
      879715
      274522
      ...
      NaN
      826
      410.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      d700e2555219b689fa3e9165aea1ed1817931a69
      http://www.foxnews.com/us/2018/02/27/texas-gir...
      Texas girl, 17, stabs newborn to death minutes...
      2018-02-28 00:39:13.863
      2018-02-28 00:39:13.863
      113688
      70.98
      2018-03-01T02:58:10.218Z
      25583
      68921
      19184
      ...
      NaN
      285
      344.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      5c33cf13f73345e32664ed8be841dc56b86e107d
      https://www.indy100.com/article/emma-gonzalez-...
      The Florida student spearheading the gun debat...
      2018-02-25 11:24:17.265
      2018-02-25 11:16:22.000
      159836
      126.53
      2018-02-25T22:14:10.236Z
      10622
      138519
      10695
      ...
      NaN
      5014
      258.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      073db9d2d004bb61e3f507a4670347f1142b1819
      http://www.foxnews.com/us/2018/02/20/florida-h...
      Florida House aide fired after claiming Florid...
      2018-02-21 00:49:14.722
      2018-02-21 00:49:14.722
      99873
      103.73
      2018-02-21T14:02:13.113Z
      42670
      42234
      14969
      ...
      NaN
      285
      590.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      8d3e2975bed53d3885889ca47afdc8a309d74370
      https://www.washingtonpost.com/opinions/beat-y...
      Beat your wife, lose your gun. That’s how it s...
      2018-02-28 00:16:12.833
      2018-02-28 00:09:50.000
      84513
      307.37
      2018-03-02T16:10:07.064Z
      12548
      66034
      5931
      ...
      NaN
      191
      501.00
      49.00
      0.00
      0.00
      0.00
      0.00
      49.00
      -49.00

      6864f9a08eee3a16c208675d81ea26c6dba34608
      http://www.dailymail.co.uk/tvshowbiz/article-5...
      MAFS stars Troy and Carly engage in steamy kis...
      2018-02-12 01:10:20.976
      2018-02-12 01:07:48.000
      112395
      245.34
      2018-02-12T05:35:12.625Z
      99171
      11772
      1452
      ...
      NaN
      158
      438.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      e4af688c612406377380281ed31844390fa0e11b
      http://www.dailymail.co.uk/news/article-542452...
      Now plastic straws are banned
      2018-02-22 23:10:21.766
      2018-02-22 23:07:43.000
      107178
      80.52
      2018-02-23T13:06:12.913Z
      21769
      73417
      11992
      ...
      NaN
      158
      922.00
      50.00
      0.00
      1.00
      0.00
      1.00
      51.00
      -49.00

      c3fa7bc71b67c98cbb1e9f209ed0a719adc0aace
      https://www.cnn.com/videos/us/2018/02/15/victi...
      Victim's mom begs Trump: Do something! - CNN V...
      2018-02-15 20:59:11.852
      2018-02-15 20:49:32.000
      85974
      993.42
      2018-02-16T04:03:07.087Z
      23979
      46502
      15493
      ...
      NaN
      105
      nan
      49.00
      0.00
      1.00
      0.00
      1.00
      50.00
      -48.00

      8a6dd31f5ad1a2dc9216017bf763e41de97981b0
      https://www.washingtonpost.com/news/inspired-l...
      Lucas was just named 2018 Gerber baby. He has ...
      2018-02-07 17:22:16.077
      2018-02-07 17:10:53.000
      114315
      116.45
      2018-02-08T01:57:11.230Z
      6202
      101278
      6835
      ...
      NaN
      191
      700.00
      50.00
      0.00
      2.00
      0.00
      2.00
      52.00
      -48.00

      14d31a214e4facda0c86d87707a65d161d7e9eaa
      http://www.foxnews.com/us/2018/02/28/georgia-t...
      Georgia teacher in custody after reports of sh...
      2018-02-28 18:19:21.900
      2018-02-28 18:03:42.000
      178634
      363.57
      2018-02-28T20:15:08.070Z
      68963
      71257
      38414
      ...
      NaN
      285
      220.00
      50.00
      0.00
      2.00
      0.00
      2.00
      52.00
      -48.00

      da38b48e9d96a8525f8c805e78e9f9443d3e882d
      http://www.foxnews.com/us/2018/02/09/tim-tebow...
      Tim Tebow's 'Night to Shine' celebrates 90,000...
      2018-02-09 15:04:39.518
      2018-02-09 14:56:51.000
      112638
      203.83
      2018-02-09T20:21:08.978Z
      5261
      99225
      8152
      ...
      NaN
      285
      458.00
      50.00
      0.00
      2.00
      0.00
      2.00
      52.00
      -48.00

      670255849c49056b0e26784b1c6d657dbf5a1ddd
      https://www.huffingtonpost.com/entry/trump-sup...
      Trump Supporters Consume And Share The Most Fa...
      2018-02-07 01:19:21.756
      2018-02-07 01:06:18.034
      60118
      347.80
      2018-02-07T20:03:08.685Z
      15620
      34706
      9792
      ...
      NaN
      215
      364.00
      47.00
      0.00
      0.00
      0.00
      0.00
      47.00
      -47.00

      240e377bd6e8d17dc5b3785a0be38213aa927e49
      http://www.foxnews.com/us/2018/02/06/high-scho...
      High school cancels musical after white studen...
      2018-02-06 20:29:14.140
      2018-02-06 20:06:25.000
      76566
      217.97
      2018-02-06T22:15:11.567Z
      28817
      38169
      9580
      ...
      NaN
      285
      542.00
      48.00
      0.00
      1.00
      0.00
      1.00
      49.00
      -47.00

      91b6442a3738ab3d7ac035ddd9db8efbfc7a5535
      https://www.washingtonpost.com/news/grade-poin...
      Trump and DeVos call for massive cuts to colle...
      2018-02-13 21:58:12.640
      2018-02-13 21:45:58.000
      64638
      24.41
      2018-02-15T02:10:09.694Z
      16484
      36782
      11372
      ...
      NaN
      191
      1122.00
      47.00
      0.00
      0.00
      0.00
      0.00
      47.00
      -47.00

      8985d303c1803ccb878bbce2d18fab0270402683
      https://www.washingtonpost.com/news/answer-she...
      If Americans really cared about students’ ment...
      2018-02-16 00:28:17.677
      2018-02-16 00:20:52.000
      72193
      47.30
      2018-02-17T16:41:03.436Z
      9465
      53129
      9599
      ...
      NaN
      191
      881.00
      48.00
      0.00
      2.00
      0.00
      2.00
      50.00
      -46.00

      8467fa14011d0affe30554e7a793ee799bcc7b50
      https://www.theguardian.com/us-news/2018/feb/2...
      Spokesman falsely denied expensive table had b...
      2018-02-28 00:07:04.792
      2018-02-28 00:02:53.000
      52895
      113.37
      2018-02-28T16:32:21.970Z
      7703
      39002
      6190
      ...
      NaN
      142
      408.00
      46.00
      0.00
      0.00
      0.00
      0.00
      46.00
      -46.00

      a7eafd17d0a4717e181718210ed23a452097d37e
      http://yournewswire.com/cdc-doctor-flu-shot-dead/
      CDC Doctor, Who Claimed Flu Shot Caused Outbre...
      2018-02-22 19:34:37.873
      2018-02-22 18:12:03.000
      179799
      71.96
      2018-02-24T02:51:18.102Z
      38509
      95811
      45479
      ...
      2018-02-22T20:05:04.000Z
      22568
      613.00
      50.00
      0.00
      1.00
      4.00
      5.00
      55.00
      -45.00

      6594c8cd8e1992246dedb7d88c6b668b9cb40dc7
      http://yournewswire.com/pedophile-dead-penis-s...
      Pedophile Who Raped 3-Month-Old Baby Found Dea...
      2018-02-17 17:59:28.027
      2018-02-17 17:23:40.000
      102627
      32.73
      2018-02-21T19:53:13.378Z
      24098
      67729
      10800
      ...
      2018-02-17T20:31:35.000Z
      22568
      360.00
      50.00
      0.00
      1.00
      4.00
      5.00
      55.00
      -45.00

      fb223beac020beeada1600f213901cec0b9853b8
      http://yournewswire.com/judge-pedophile-prison...
      Judge Cuts Pedophile’s Prison Term Claiming 3-...
      2018-02-10 16:59:30.838
      2018-02-10 16:43:30.000
      314525
      363.43
      2018-02-14T20:05:21.412Z
      104622
      164861
      45042
      ...
      2018-02-10T19:13:53.000Z
      22568
      403.00
      50.00
      0.00
      1.00
      4.00
      5.00
      55.00
      -45.00

      ce9daaa17cbab41fef36b8ddb4c8ec83f4560f0a
      https://www.washingtonpost.com/powerpost/house...
      House passes changes to Americans With Disabil...
      2018-02-15 17:54:14.705
      2018-02-15 17:35:00.000
      51851
      56.45
      2018-02-16T02:34:13.544Z
      8709
      31582
      11560
      ...
      NaN
      191
      517.00
      46.00
      0.00
      1.00
      0.00
      1.00
      47.00
      -45.00

      b1b8b57a292c94f4259fde4bab63f84a9c8ca810
      http://yournewswire.com/catholic-archbishop-pe...
      Catholic Archbishop Says Pedophilia Is ‘Spirit...
      2018-02-12 17:54:31.976
      2018-02-12 17:23:02.000
      263961
      201.12
      2018-02-18T04:08:02.439Z
      108399
      126464
      29098
      ...
      2018-02-14T13:20:00.000Z
      22568
      427.00
      50.00
      0.00
      1.00
      4.00
      5.00
      55.00
      -45.00

      916a275eedcdb097b624d00a0c14dfb8ca53905c
      https://www.washingtonpost.com/news/wonk/wp/20...
      Report: No progress for African Americans on h...
      2018-02-26 20:52:11.920
      2018-02-26 20:45:06.000
      65459
      31.23
      2018-02-27T17:04:12.592Z
      11981
      38723
      14755
      ...
      NaN
      191
      743.00
      47.00
      0.00
      2.00
      0.00
      2.00
      49.00
      -45.00

25 rows × 27 columns

Write that data to a file. Note that the scores here are provisional for two reasons:
1. they should be using a rolling-month based on the article publication date to calculate medians/min/max etc, whereas in this workbook we as just using values for the month of May
2. for analysis, we've rounded the numbers; we don't expect to do that for the actual scores

```python
data.to_csv("articles_with_provisional_scores_" + date_filename + ".csv")
```

# Summary

The attention index of an article is comprised of four components:

- *lead score* (max 20 points) based on how long an article was the lead story on the publisher's home page, scaled by the traffic to that publisher
- *front score* (max 15 points) based on how long an article was present on the publisher's home page, scaled by traffic to that publisher
- *Facebook promotion score* (max 15 points) based on whether the article was promoted to the publisher's Facebook brand page, scaled by the reach of that brand page
- *response score* (max 50 points) based on the number of Facebook engagements the article received, relative to other articles

Or, in other words:

\begin{align}
attentionIndex_a &= leadScore_a + frontScore_a + facebookPromotionScore_a + responseScore_a \\
leadScore_a &= 20 \cdot \left(\frac{\min(minsAsLead_a, 60)}{alexaRank_a}\right) \cdot \left( \frac{\min(alexaRank)}{60} \right) \\
frontScore_a &=
15 \cdot
\left(\frac{\min(minsOnFront_a, 1440)}{alexaRank_a \cdot numArticlesOnFront_a}\right) \cdot
\left( \frac{\min(alexaRank \cdot numArticlesOnFront)}{1440} \right) \\
facebookPromotion_a &=
\begin{cases}
 0 \text{ if not shared on brand page }\\
   15 \cdot \frac{\log(brandPageLikes_a) - \log(\min(brandPageLikes))}{\log(\max(brandPageLikes)) - \log(\min(brandPageLikes))}  \text{ otherwise }
\end{cases}
\\
responseScore_a &=
\begin{cases}
   0 \text{ if } engagements_a = 0 \\
   50 \cdot
   \frac{\log(\min(engagements_a,limit) + median(engagements)) - \log(1 + median(engagements))}
   {\log(limit + median(engagements)) - \log(1 + median(engagements))} \text{ if } engagements_a > 0
\end{cases} \\
\end{align}

```python

```

```python

```