# California wildfires 2017 - Thomas Fire analysis

The Thomas Fire is a massive wildfire burning in Ventura and Santa Barbara Counties, and one of multiple wildfires that started in early December 2017 in Southern California. It has burned approximately 270,500 acres (423 sq mi; 1,095 km2), becoming the largest wildfire of the 2017 California wildfire season, and the third-largest wildfire in modern California history.

```python
from arcgis.gis import *
import pandas as pd
```

```python
gis = GIS("https://python.playground.esri.com/portal", "arcgis_python", "amazing_arcgis_123")
```

```python
landsat_item = gis.content.search('title:Multispectral Landsat', 'Imagery Layer', outside_org=True)[0]
landsat = landsat_item.layers[0]
landsat_item
```

                        Multispectral Landsat

Landsat 8 OLI, 30m Multispectral 8 band scenes with visual renderings and indices. Updated daily. Based on the Landsat on AWS collections.Imagery Layer by esri_livingatlas

Last Modified: October 06, 2016

0 comments, 0 views

### Select before and after rasters

```python
aoi = {'spatialReference': {'latestWkid': 3857, 'wkid': 102100}, 'type': 'extent',
       'xmax': -13305000, 'xmin': -13315000, 'ymax': 4106000, 'ymin': 4052000}
landsat.extent = aoi
```

```python
selected = landsat.filter_by(where="(Category = 1)",
                             geometry=arcgis.geometry.filters.intersects(aoi))

df = selected.query(out_fields="AcquisitionDate, GroupName, CloudCover, DayOfYear", order_by_fields="AcquisitionDate").df
df['AcquisitionDate'] = pd.to_datetime(df['AcquisitionDate'], unit='ms')
df.tail()
```

      AcquisitionDate
      CloudCover
      DayOfYear
      GroupName
      OBJECTID
      Shape_Area
      Shape_Length
      SHAPE

      74
      2017-10-22 18:34:49.100
      0.0004
      295
      LC80420362017295LGN00_MTL
      580864
      5.128295e+10
      906224.695944
      {'rings': [[[-13210242.4169, 4199065.126800001...

      75
      2017-11-07 18:34:47.721
      0.2025
      311
      LC80420362017311LGN00_MTL
      584295
      5.129209e+10
      906301.639126
      {'rings': [[[-13210307.907200001, 4199065.8737...

      76
      2017-11-23 18:34:42.440
      0.0441
      327
      LC80420362017327LGN00_MTL
      591810
      5.131484e+10
      906509.716682
      {'rings': [[[-13208036.3849, 4199102.123899996...

      77
      2017-12-09 18:34:39.504
      0.2136
      343
      LC80420362017343LGN00_MTL
      599875
      5.127378e+10
      906136.246386
      {'rings': [[[-13208251.4004, 4199098.6241], [-...

      78
      2017-12-25 18:34:42.003
      0.6529
      359
      LC80420362017359LGN00_MTL
      608116
      5.129089e+10
      906293.823931
      {'rings': [[[-13210387.2213, 4199063.157099999...

```python
prefire = landsat.filter_by('OBJECTID=591810') # 2017-11-23
midfire = landsat.filter_by('OBJECTID=599875') # 2017-12-09
```

## Visual Assessment

```python
from arcgis.raster.functions import *

apply(midfire, 'Natural Color with DRA')
```

### Visualize Burn Scars

Extract the [6, 4, 1] bands to improve visibility of fire and burn scars. This band combination pushes further into the SWIR range of the electromagnetic spectrum, where there is less susceptibility to smoke and haze generated by a burning fire.

```python
extract_band(midfire, [6,4,1])
```

For comparison, the same area before the fire started shows no burn scar:

```python
extract_band(prefire, [6,4,1])
```

## Quantitative Assessment

The **Normalized Burn Ratio (NBR)** was defined to highlight areas that have burned and to index the severity of a burn using Landsat imagery.

The formula for the NBR is very similar to that of NDVI except that it uses near-infrared band 5 and the short-wave infrared band 7:
\begin{align}
{\mathbf{NBR}} = \frac{\mathbf{B_5} - \mathbf{B_7}}{\mathbf{B_5} + \mathbf{B_7}} \\
\end{align}

The NBR equation was designed to be calcualted from reflectance, but it can be calculated from radiance and digital_number_(dn) with changes to the burn severity table below.

For a given area, NBR is calculated from an image just prior to the burn and a second NBR is calculated for an image immediately following the burn. Burn extent and severity is judged by taking the difference between these two index layers:

\begin{align}
{\Delta \mathbf{NBR}} = \mathbf{NBR_{prefire}} - \mathbf{NBR_{postfire}} \\
\end{align}

The meaning of the ∆NBR values can vary by scene, and interpretation in specific instances should always be based on some field assessment. However, the following table from the USGS FireMon program can be useful as a first approximation for interpreting the NBR difference:

| \begin{align}{\Delta \mathbf{NBR}}  \end{align}      | Burn Severity |
| ------------- |:-------------:|
| 0.1 to 0.27   | Low-severity burn |
| 0.27 to 0.44  | Moderate-low severity burn |
| 0.44 to 0.66 | Moderate-high severity burn |
| > 0.66 | High severity burn |

[Source: http://wiki.landscapetoolbox.org/doku.php/remote_sensing_methods:normalized_burn_ratio]

```python
nbr_prefire  = band_arithmetic(prefire, "(b5 - b7) / (b5 + b7)")
nbr_postfire = band_arithmetic(midfire, "(b5 - b7) / (b5 + b7)")

nbr_diff = nbr_prefire - nbr_postfire
```

```python
mask = remap(nbr_diff,
             input_ranges=[0.1, 0.27, 0.27, 0.44, 0.44, 0.66, 0.66, 1],  # change in NBR
             output_values=[1, 2, 3, 4],                                 # burn severity
             no_data_ranges=[-1, 0.1], astype='u8')
```

### Compute burnt area

```python
ext = {"xmax": -13246079.10806628, "ymin": 4035733.9433013694, "xmin": -13438700.419344831, "ymax": 4158033.188557592,
       "spatialReference": {"wkid": 102100, "latestWkid": 3857}, "type": "extent"}
pixx = (ext['xmax'] - ext['xmin']) / 1200.0
pixy = (ext['ymax'] - ext['ymin']) / 450.0
```

```python
res = mask.compute_histograms(ext, pixel_size={'x':pixx, 'y':pixy})
numpix = 0
for i in res['histograms'][0]['counts']:
    numpix += i

sqmarea = numpix * pixx * pixy # in sq. m
acres = 0.00024711 * sqmarea   # in acres

print('Thomas fire had consumed {:,} acres till 09 December 2017.'.format(int(acres)))
```

    Thomas fire had consumed 229,304 acres till 09 December 2017.

### Visualize burnt areas

```python
burnt_areas = colormap(mask, colormap=[[4, 0xFF, 0xC3, 0],
                            [3, 0xFA, 0x8E, 0],
                            [2, 0xF2, 0x55, 0],
                            [1, 0xE6, 0,    0]])
```

```python
m = gis.map('Carpinteria, CA', zoomlevel=10)
m
```

Failed to display Jupyter Widget of type MapView.

  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean
  that the widgets JavaScript is still loading. If this message persists, it
  likely means that the widgets JavaScript library is either not installed or
  not enabled. See the Jupyter
  Widgets Documentation for setup instructions.

  If you're reading this message in another notebook frontend (for example, a static
  rendering on GitHub or NBViewer),
  it may mean that your frontend doesn't currently support widgets.

```python
m.add_layer([midfire, burnt_areas])
```

Compare with final extent of Thomas Fire.

```python
from arcgis.features import FeatureLayer
live_fires = FeatureLayer('https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/Thomas_Fire/FeatureServer/1')

live_fires_df = live_fires.query().df
```

```python
m.add_layer(live_fires, {'opacity': 0.5})
```

## Impact Assessment

```python
from arcgis.geoenrichment import *
```

```python
thomas_pop = enrich(live_fires_df, 'Age')
```

### Visualize using charts

```python
import warnings
import seaborn as sns
import matplotlib.pyplot as plt

%matplotlib inline
warnings.simplefilter(action='ignore', category=FutureWarning)
pd.options.mode.chained_assignment = None
plt.style.use('ggplot')

sf = thomas_pop.T
sf.reset_index(inplace=True)
sf['age'] = sf['index'].str.extract('(\d+)')

fem = sf[sf['index'].str.startswith('FEM')]
fem.rename(columns={0:'female'}, inplace=True)
fem['age'] = pd.to_numeric(fem['age'])
fem.sort_values(by='age', ascending=False, inplace=True)

male = sf[sf['index'].str.startswith('MALE')]
male.rename(columns={0:'male'}, inplace=True)
male['age'] = pd.to_numeric(male['age'])
male.sort_values(by='age', ascending=False, inplace=True)

popdf = pd.merge(male, fem, on='age')
popdf['agelabel'] = popdf['age'].map(str) + ' - ' + (popdf['age']+4).map(str)
popdf['male'] = -popdf['male']
```

### Age Pyramid of affected population

```python
print('Number of people in fire perimeter: ' + str(popdf['female'].sum() - popdf['male'].sum()))
```

    Number of people in fire perimeter: 11228

```python
sns.barplot(x="female", y="agelabel", color="#CC6699", label="Female", data=popdf, edgecolor='none')
sns.barplot(x="male",   y="agelabel", color="#008AB8", label="Male",   data=popdf,  edgecolor='none')
plt.ylabel('Age group')
plt.xlabel('Number of people');
```


```
print "Hallo Welt!"
```

    Hallo Welt!

```
2**200
```

    1606938044258990275541962092341162602522202993782792835301376L

```
# Multi-line editing. Syntax-coloring.
for num in ['eins', 'zwei', 'drei']:
    print "Line %s" % num
```

    Line eins
    Line zwei
    Line drei

# Überschriften

Normaler Text in ganzen Sätzen. Formatiert als Markdown: *hervorgehoben* **fett**

## Hilfe und Tab-Completion

```
s="hallo welt"
#s?
#s.encode(
s.encode("base64")
```

    'aGFsbG8gd2VsdA==\n'

## Dateiformat: JSON

Abspeichern als JSON, wieder einladen

# Shell-Kommandos

```
!ls -l
```

    insgesamt 10272
    -rw-rw-rw-  1 dloss dloss  302868 Mai  2 14:41 Bitcoin.ipynb
    -rw-rw-rw-  1 dloss dloss    1147 Mai  2 14:37 bitcoin_weighted_prices.json
    drwxrwxr-x  2 dloss dloss    4096 Apr 29 18:42 bundesliga
    -rw-rw-rw-  1 dloss dloss  119335 Apr  7 16:37 bundesliga.ipynb
    -rw-rw-r--  1 dloss dloss  408138 Mai  2 16:28 Chapter1_Introduction.ipynb
    drwxrwxr-x  7 dloss dloss   12288 Apr 29 18:42 crosscompute-tutorials
    drwxrwxrwx  3 dloss dloss    4096 Mai  1 17:46 datasets
    -rw-rw-rw-  1 dloss dloss 2353817 Mai  3 09:54 datenanalyse-ipython-pandas.pdf
    -rw-rw-rw-  1 dloss dloss 2727936 Mai  2 15:30 datenanalyse-ipython-pandas.ppt
    -rw-rw-rw-  1 dloss dloss  792146 Apr  7 17:00 E1_BEZIRK_201112.csv
    -rw-rw-rw-  1 dloss dloss  225923 Nov 24 20:17 ergebnisse-1963-2010.zip
    -rw-r--r--  1 dloss dloss    2048 Mai  2 15:38 foo.db
    -rw-rw-rw-  1 dloss dloss    2037 Apr 20 12:59 genua-vortrag.txt
    drwxrwxrwx  4 dloss dloss    4096 Mai  1 17:03 hpf
    -rw-rw-rw-  1 dloss dloss  326577 Mai  2 13:58 hpf-admin.ipynb
    -rw-rw-rw-  1 dloss dloss  148820 Mai  2 15:41 IPython-Demo.ipynb
    -rw-rw-r--  1 dloss dloss   27092 Mai  1 16:35 ipy.vim
    -rw-rw-r--  1 dloss dloss     217 Mai  3 10:14 links.txt
    -rw-rw-rw-  1 dloss dloss  861410 Apr 27 21:29 mtgox.csv
    -rw-rw-rw-  1 dloss dloss  861218 Mai  2 16:33 mtgoxUSD.csv
    -rw-rw-rw-  1 dloss dloss  167366 Apr  7 18:28 Muenchen.ipynb
    -rw-rw-r--  1 dloss dloss   21907 Mai  2 16:23 mypicture.pdf
    drwxrwxr-x  6 dloss dloss    4096 Mai  1 13:25 notebooks
    -rw-rw-rw-  1 dloss dloss  355731 Mai  2 15:37 OpenBSD.ipynb
    drwxrwxr-x 11 dloss dloss    4096 Mai  2 10:59 other
    -rw-rw-rw-  1 dloss dloss   54449 Mär 28 21:00 pandas_basics.ipynb
    -rw-rw-rw-  1 dloss dloss   84604 Mär 30 21:11 Pandas Cookbook.ipynb
    drwxrwxr-x  3 dloss dloss    4096 Apr 29 18:49 pandas-demo
    drwxrwxr-x  3 dloss dloss    4096 Apr 29 18:53 pcap
    -rw-rw-rw-  1 dloss dloss   31161 Mai  2 13:02 pcap.ipynb
    -rw-rw-rw-  1 dloss dloss   26785 Mai  1 13:01 Pi.ipynb
    -rw-rw-rw-  1 dloss dloss  191731 Apr 28 22:17 python-slices.graffle
    -rw-rw-r--  1 dloss dloss   76393 Mai  1 16:30 stock_data.csv
    drwxrwxr-x  3 dloss dloss   12288 Apr 29 18:42 tippkick
    -rw-rw-r--  1 dloss dloss    7943 Mai  1 16:30 tips.csv
    drwxr-xr-x  7 dloss dloss    4096 Apr 30 23:08 tutorials
    -rw-rw-r--  1 dloss dloss    2267 Mai  2 16:12 Vortrag.ipynb

```
!head -20 IPython-Demo.ipynb
```

    {
     "metadata": {
      "name": "IPython-Demo"
     },
     "nbformat": 3,
     "nbformat_minor": 0,
     "worksheets": [
      {
       "cells": [
        {
         "cell_type": "code",
         "collapsed": false,
         "input": [
          "print \"Hallo Welt!\""
         ],
         "language": "python",
         "metadata": {},
         "outputs": [
          {
           "output_type": "stream",

Python-Variablen übergeben

```
interface="eth0"
data = !ifconfig $interface
data.fields()
```

    [['/bin/sh:', '1:', 'ifconfig:', 'not', 'found']]

```
# Interrupt with CTRL-m i
!ping -c 6 8.8.8.8
```

    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    From 192.168.253.1 icmp_seq=1 Destination Port Unreachable
    From 192.168.253.1 icmp_seq=2 Destination Port Unreachable
    From 192.168.253.1 icmp_seq=3 Destination Port Unreachable
    From 192.168.253.1 icmp_seq=4 Destination Port Unreachable
    From 192.168.253.1 icmp_seq=5 Destination Port Unreachable
    From 192.168.253.1 icmp_seq=6 Destination Port Unreachable

    --- 8.8.8.8 ping statistics ---
    6 packets transmitted, 0 received, +6 errors, 100% packet loss, time 5001ms

### Andere Programmiersprachen

```ruby
%%ruby
name = "Ruby"
3.times { puts "Hallo #{ name }" }
```

    Hallo Ruby
    Hallo Ruby
    Hallo Ruby

```perl
%%perl
# https://github.com/peelle/Perl-Euler-Solutions/blob/master/1/solution.pl
# Straight forward iteration over each number
my $sum_of_multiples = 0;

foreach my $number ( 1..999 ) {
# using modulus to filter
next if ( $number % 3 and $number % 5 );
$sum_of_multiples += $number;
}

print $sum_of_multiples;
```

    233168

```bash
%%bash --out output --err error
echo "Zeilen:"
for i in a b c d; do echo $i; done
```

Stdout wurde an *output* übergeben:

```
print output
```

    Zeilen:
    a
    b
    c
    d

```
%%script lua
function allwords ()
  local line = "Hallo Lua program"
  local pos = 1           -- current position in the line
  return function ()      -- iterator function
    while line do         -- repeat while there are lines
      local s, e = string.find(line, "%w+", pos)
      if s then           -- found a word?
        pos = e + 1       -- next position is after this word
        return string.sub(line, s, e)     -- return the word
      else
        line = io.read()  -- word not found; try next line
        pos = 1           -- restart from first position
      end
    end
    return nil            -- no more lines: end of traversal
  end
end

-- use example
for word in allwords() do
  print(word)
end
```

    Hallo
    Lua
    program

# Plotting

```
%pylab inline
```

    Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.zmq.pylab.backend_inline].
    For more information, type 'help(pylab)'.

```
x = np.random.normal(size=1000)
y = np.random.normal(size=1000)
scatter(x, y, alpha=0.2)
savefig("mypicture.pdf")
```

```
x = np.linspace(1, 18, 100)
y = x[:, np.newaxis]
im = y * np.sin(x)
plt.imshow(im, cmap=matplotlib.cm.gray)
```

### Matplotlib Gallery

```
%load http://matplotlib.org/mpl_examples/api/radar_chart.py
```

```
"""
Example of creating a radar chart (a.k.a. a spider or star chart) [1]_.

Although this example allows a frame of either 'circle' or 'polygon', polygon
frames don't have proper gridlines (the lines are circles instead of polygons).
It's possible to get a polygon grid by setting GRIDLINE_INTERPOLATION_STEPS in
matplotlib.axis to the desired number of vertices, but the orientation of the
polygon is not aligned with the radial axes.

.. [1] http://en.wikipedia.org/wiki/Radar_chart
"""
import numpy as np

import matplotlib.pyplot as plt
from matplotlib.path import Path
from matplotlib.spines import Spine
from matplotlib.projections.polar import PolarAxes
from matplotlib.projections import register_projection

def radar_factory(num_vars, frame='circle'):
    """Create a radar chart with `num_vars` axes.

    This function creates a RadarAxes projection and registers it.

    Parameters
    ----------
    num_vars : int
        Number of variables for radar chart.
    frame : {'circle' | 'polygon'}
        Shape of frame surrounding axes.

    """
    # calculate evenly-spaced axis angles
    theta = 2*np.pi * np.linspace(0, 1-1./num_vars, num_vars)
    # rotate theta such that the first axis is at the top
    theta += np.pi/2

    def draw_poly_patch(self):
        verts = unit_poly_verts(theta)
        return plt.Polygon(verts, closed=True, edgecolor='k')

    def draw_circle_patch(self):
        # unit circle centered on (0.5, 0.5)
        return plt.Circle((0.5, 0.5), 0.5)

    patch_dict = {'polygon': draw_poly_patch, 'circle': draw_circle_patch}
    if frame not in patch_dict:
        raise ValueError('unknown value for `frame`: %s' % frame)

    class RadarAxes(PolarAxes):

        name = 'radar'
        # use 1 line segment to connect specified points
        RESOLUTION = 1
        # define draw_frame method
        draw_patch = patch_dict[frame]

        def fill(self, *args, **kwargs):
            """Override fill so that line is closed by default"""
            closed = kwargs.pop('closed', True)
            return super(RadarAxes, self).fill(closed=closed, *args, **kwargs)

        def plot(self, *args, **kwargs):
            """Override plot so that line is closed by default"""
            lines = super(RadarAxes, self).plot(*args, **kwargs)
            for line in lines:
                self._close_line(line)

        def _close_line(self, line):
            x, y = line.get_data()
            # FIXME: markers at x[0], y[0] get doubled-up
            if x[0] != x[-1]:
                x = np.concatenate((x, [x[0]]))
                y = np.concatenate((y, [y[0]]))
                line.set_data(x, y)

        def set_varlabels(self, labels):
            self.set_thetagrids(theta * 180/np.pi, labels)

        def _gen_axes_patch(self):
            return self.draw_patch()

        def _gen_axes_spines(self):
            if frame == 'circle':
                return PolarAxes._gen_axes_spines(self)
            # The following is a hack to get the spines (i.e. the axes frame)
            # to draw correctly for a polygon frame.

            # spine_type must be 'left', 'right', 'top', 'bottom', or `circle`.
            spine_type = 'circle'
            verts = unit_poly_verts(theta)
            # close off polygon by repeating first vertex
            verts.append(verts[0])
            path = Path(verts)

            spine = Spine(self, spine_type, path)
            spine.set_transform(self.transAxes)
            return {'polar': spine}

    register_projection(RadarAxes)
    return theta

def unit_poly_verts(theta):
    """Return vertices of polygon for subplot axes.

    This polygon is circumscribed by a unit circle centered at (0.5, 0.5)
    """
    x0, y0, r = [0.5] * 3
    verts = [(r*np.cos(t) + x0, r*np.sin(t) + y0) for t in theta]
    return verts

def example_data():
    #The following data is from the Denver Aerosol Sources and Health study.
    #See  doi:10.1016/j.atmosenv.2008.12.017
    #
    #The data are pollution source profile estimates for five modeled pollution
    #sources (e.g., cars, wood-burning, etc) that emit 7-9 chemical species.
    #The radar charts are experimented with here to see if we can nicely
    #visualize how the modeled source profiles change across four scenarios:
    #  1) No gas-phase species present, just seven particulate counts on
    #     Sulfate
    #     Nitrate
    #     Elemental Carbon (EC)
    #     Organic Carbon fraction 1 (OC)
    #     Organic Carbon fraction 2 (OC2)
    #     Organic Carbon fraction 3 (OC3)
    #     Pyrolized Organic Carbon (OP)
    #  2)Inclusion of gas-phase specie carbon monoxide (CO)
    #  3)Inclusion of gas-phase specie ozone (O3).
    #  4)Inclusion of both gas-phase speciesis present...
    data = {
        'column names':
            ['Sulfate', 'Nitrate', 'EC', 'OC1', 'OC2', 'OC3', 'OP', 'CO',
             'O3'],
        'Basecase':
            [[0.88, 0.01, 0.03, 0.03, 0.00, 0.06, 0.01, 0.00, 0.00],
             [0.07, 0.95, 0.04, 0.05, 0.00, 0.02, 0.01, 0.00, 0.00],
             [0.01, 0.02, 0.85, 0.19, 0.05, 0.10, 0.00, 0.00, 0.00],
             [0.02, 0.01, 0.07, 0.01, 0.21, 0.12, 0.98, 0.00, 0.00],
             [0.01, 0.01, 0.02, 0.71, 0.74, 0.70, 0.00, 0.00, 0.00]],
        'With CO':
            [[0.88, 0.02, 0.02, 0.02, 0.00, 0.05, 0.00, 0.05, 0.00],
             [0.08, 0.94, 0.04, 0.02, 0.00, 0.01, 0.12, 0.04, 0.00],
             [0.01, 0.01, 0.79, 0.10, 0.00, 0.05, 0.00, 0.31, 0.00],
             [0.00, 0.02, 0.03, 0.38, 0.31, 0.31, 0.00, 0.59, 0.00],
             [0.02, 0.02, 0.11, 0.47, 0.69, 0.58, 0.88, 0.00, 0.00]],
        'With O3':
            [[0.89, 0.01, 0.07, 0.00, 0.00, 0.05, 0.00, 0.00, 0.03],
             [0.07, 0.95, 0.05, 0.04, 0.00, 0.02, 0.12, 0.00, 0.00],
             [0.01, 0.02, 0.86, 0.27, 0.16, 0.19, 0.00, 0.00, 0.00],
             [0.01, 0.03, 0.00, 0.32, 0.29, 0.27, 0.00, 0.00, 0.95],
             [0.02, 0.00, 0.03, 0.37, 0.56, 0.47, 0.87, 0.00, 0.00]],
        'CO & O3':
            [[0.87, 0.01, 0.08, 0.00, 0.00, 0.04, 0.00, 0.00, 0.01],
             [0.09, 0.95, 0.02, 0.03, 0.00, 0.01, 0.13, 0.06, 0.00],
             [0.01, 0.02, 0.71, 0.24, 0.13, 0.16, 0.00, 0.50, 0.00],
             [0.01, 0.03, 0.00, 0.28, 0.24, 0.23, 0.00, 0.44, 0.88],
             [0.02, 0.00, 0.18, 0.45, 0.64, 0.55, 0.86, 0.00, 0.16]]}
    return data

if __name__ == '__main__':
    N = 9
    theta = radar_factory(N, frame='polygon')

    data = example_data()
    spoke_labels = data.pop('column names')

    fig = plt.figure(figsize=(9, 9))
    fig.subplots_adjust(wspace=0.25, hspace=0.20, top=0.85, bottom=0.05)

    colors = ['b', 'r', 'g', 'm', 'y']
    # Plot the four cases from the example data on separate axes
    for n, title in enumerate(data.keys()):
        ax = fig.add_subplot(2, 2, n+1, projection='radar')
        plt.rgrids([0.2, 0.4, 0.6, 0.8])
        ax.set_title(title, weight='bold', size='medium', position=(0.5, 1.1),
                     horizontalalignment='center', verticalalignment='center')
        for d, color in zip(data[title], colors):
            ax.plot(theta, d, color=color)
            ax.fill(theta, d, facecolor=color, alpha=0.25)
        ax.set_varlabels(spoke_labels)

    # add legend relative to top-left plot
    plt.subplot(2, 2, 1)
    labels = ('Factor 1', 'Factor 2', 'Factor 3', 'Factor 4', 'Factor 5')
    legend = plt.legend(labels, loc=(0.9, .95), labelspacing=0.1)
    plt.setp(legend.get_texts(), fontsize='small')

    plt.figtext(0.5, 0.965, '5-Factor Solution Profiles Across Four Scenarios',
                ha='center', color='black', weight='bold', size='large')
    plt.show()

```

# Rich display

```
from IPython.core.display import Image, HTML, SVG
```

```
Image(filename="/usr/share/doc/texlive-doc/latex/ctable/penguin.jpg")
```

```
SVG(filename="/usr/share/doc/ipython/examples/notebooks/python-logo.svg")
```

```
from IPython.display import HTML
HTML('')
```

```
from IPython.display import YouTubeVideo
YouTubeVideo("2G5YTlheCbw")
```

```
import IPython
IPython.display.Math(r"\sum_{n=1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6}")
```

$$\sum_{n=1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6}$$

LaTeX innerhalb von Markdown geht auch: $\frac{1}{\sqrt{ 2 \pi \sigma^2 }} e^{ - \frac{ (x - \mu)^2 } {2 \sigma^2} }$

### Darstellungen für eigene Objekte:

```
class Disc(object):
    def __init__(self, size, color='red'):
        self.size = size
        self.color = color

    def _repr_svg_(self):
        return """<svg xmlns="http://www.w3.org/2000/svg" version="1.1">
                  <circle cx="{0:d}" cy="{0:d}" r="{0:d}" fill="{1:s}" />
                  </svg>""".format(self.size, self.color)
```

```
Disc(65, "blue")
```

# SymPy

```
from sympy import *
%load_ext sympy.interactive.ipythonprinting
x = Symbol('x')
```

    /usr/lib/pymodules/python2.7/mpl_toolkits/__init__.py:2: UserWarning: Module mpl_toolkits was already imported from /usr/lib/pymodules/python2.7/mpl_toolkits/__init__.pyc, but /usr/local/lib/python2.7/dist-packages is being added to sys.path
      __import__('pkg_resources').declare_namespace(__name__)

```
pi.evalf(100)
```

```
(pi+x*2)**3
```

```
((pi+x*2)**3).expand()
```

# SQL Extension

```
%load_ext sql
```

```
rm -rf foo.db
```

```
%sql sqlite:///foo.db
```

    u'Connected: None@foo.db'

```sql
%%sql sqlite:///foo.db
CREATE TABLE writer (first_name, last_name, year_of_death);
INSERT INTO writer VALUES ('William', 'Shakespeare', 1616);
INSERT INTO writer VALUES ('Bertold', 'Brecht', 1956);
```

$$\begin{bmatrix}\end{bmatrix}$$

```sql
%%sql select * from writer
```

        first_name
        last_name
        year_of_death

        William
        Shakespeare
        1616

        Bertold
        Brecht
        1956


# План :
## Задача
https://www.kaggle.com/c/mlcourse-dota2-win-prediction
## Код
## Бонус GAFAROV10

```python
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
```

```python
pseudo_train = pd.read_csv('pseudo_train_df.csv')
pseudo_test = pd.read_csv('pseudo_test_df.csv')
```

```python
pseudo_train.shape
```

    (29999, 247)

```python
pseudo_test.shape
```

    (654, 280)

```python
pseudo_train.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 29999 entries, 0 to 29998
    Columns: 247 entries, match_id_hash to radiant_win
    dtypes: float64(15), int64(231), object(1)
    memory usage: 56.5+ MB

```python
pseudo_test
```

      match_id_hash
      game_time
      game_mode
      lobby_type
      objectives_len
      chat_len
      r1_hero_id
      r1_kills
      r1_deaths
      r1_assists
      ...
      d5_creeps_stacked
      d5_camps_stacked
      d5_rune_pickups
      d5_firstblood_claimed
      d5_teamfight_participation
      d5_towers_killed
      d5_roshans_killed
      d5_obs_placed
      d5_sen_placed
      radiant_win

      0
      7b806ddf6d6d616c10ae2089ffb8b5be
      887
      22
      7
      1
      9
      70
      0
      3
      1
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1

      1
      725ec4b525a20f3088842bea2a22132c
      15
      22
      7
      0
      0
      102
      0
      0
      0
      ...
      0
      0
      1
      0
      0
      0
      0
      0
      0
      0

      2
      e36041c6f536ed0af8b3f9396a960f91
      282
      22
      7
      1
      0
      98
      1
      0
      0
      ...
      0
      0
      2
      0
      0
      0
      0
      0
      0
      0

      3
      dcdb0cf951ca9601e3bfaca154e95e41
      2573
      5
      0
      22
      2
      107
      3
      17
      13
      ...
      0
      0
      9
      0
      0
      3
      0
      0
      0
      0

      4
      a079655f92f64076c93876a36874eff0
      443
      4
      0
      3
      5
      10
      1
      0
      1
      ...
      0
      0
      3
      0
      0
      0
      0
      1
      1
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      649
      2eeebcc853963794b01053ebe8bcb988
      1213
      4
      0
      4
      66
      48
      1
      4
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      3
      0

      650
      1f1004b872984704bb0d7018d06bebb3
      3415
      22
      0
      23
      10
      86
      7
      8
      17
      ...
      0
      0
      14
      0
      0
      2
      1
      1
      0
      1

      651
      87169e0052a98e224c6876840fe3c11e
      2308
      22
      0
      9
      18
      8
      7
      2
      4
      ...
      2
      1
      12
      0
      0
      0
      0
      9
      1
      0

      652
      39a676bd4fd4753baeeccf5d18dee1ce
      2576
      22
      7
      18
      0
      111
      3
      6
      7
      ...
      5
      2
      11
      0
      0
      2
      0
      4
      9
      1

      653
      95eb72b82aee1d0d8b6f2b2dc11e9d0d
      1392
      22
      7
      5
      1
      50
      4
      5
      8
      ...
      0
      0
      4
      1
      0
      0
      0
      1
      1
      1

654 rows × 247 columns

```python
pseudo_train.describe()
```

      game_time
      game_mode
      lobby_type
      objectives_len
      chat_len
      r1_hero_id
      r1_kills
      r1_deaths
      r1_assists
      r1_denies
      ...
      d5_creeps_stacked
      d5_camps_stacked
      d5_rune_pickups
      d5_firstblood_claimed
      d5_teamfight_participation
      d5_towers_killed
      d5_roshans_killed
      d5_obs_placed
      d5_sen_placed
      radiant_win

      count
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.00000
      ...
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000
      29999.000000

      mean
      1144.037868
      19.634554
      4.778826
      6.515584
      7.364345
      50.963299
      3.111670
      3.249308
      4.639855
      6.29611
      ...
      1.025534
      0.339811
      4.657489
      0.091670
      0.041368
      0.298910
      0.024834
      1.281276
      0.798360
      0.526018

      std
      767.019605
      6.249887
      3.258060
      6.493429
      13.440356
      34.558785
      3.677295
      3.267914
      5.183135
      8.19352
      ...
      3.419258
      0.948468
      4.616371
      0.288564
      0.199143
      0.730656
      0.169170
      2.621312
      2.465113
      0.499331

      min
      0.000000
      2.000000
      0.000000
      0.000000
      0.000000
      1.000000
      0.000000
      0.000000
      0.000000
      0.00000
      ...
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000

      25%
      519.000000
      22.000000
      0.000000
      1.000000
      0.000000
      20.000000
      0.000000
      1.000000
      1.000000
      1.00000
      ...
      0.000000
      0.000000
      1.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000

      50%
      1040.000000
      22.000000
      7.000000
      4.000000
      3.000000
      44.000000
      2.000000
      2.000000
      3.000000
      3.00000
      ...
      0.000000
      0.000000
      3.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      1.000000

      75%
      1651.000000
      22.000000
      7.000000
      10.000000
      9.000000
      81.000000
      5.000000
      5.000000
      7.000000
      9.00000
      ...
      0.000000
      0.000000
      7.000000
      0.000000
      0.000000
      0.000000
      0.000000
      1.000000
      0.000000
      1.000000

      max
      4574.000000
      23.000000
      7.000000
      43.000000
      272.000000
      120.000000
      31.000000
      25.000000
      40.000000
      84.00000
      ...
      107.000000
      29.000000
      52.000000
      1.000000
      1.000000
      8.000000
      4.000000
      26.000000
      47.000000
      1.000000

8 rows × 246 columns

```python
pseudo_valid = pseudo_train.sample(999).reset_index(drop=True)
pseudo_train = pseudo_train[~pseudo_train.match_id_hash.isin(pseudo_valid.match_id_hash.unique())].reset_index(drop=True)
```

```python

```

```python
X = pseudo_train.loc[:, (pseudo_train.columns != 'radiant_win') & (pseudo_train.columns != 'match_id_hash')]
y = pseudo_train.radiant_win
clf = LogisticRegressionCV(cv=5, random_state=42, n_jobs =-1 ,scoring = 'roc_auc').fit(X, y)
```

    /home/s.gafarov/.local/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      n_iter_i = _check_optimize_result(

```python
clf.score(X, y)
```

    0.813366406926179

```python
X_valid = pseudo_valid.loc[:, (pseudo_valid.columns != 'radiant_win') & (pseudo_valid.columns != 'match_id_hash')]
y_valid_pred = clf.predict_proba(X_valid)

X_test = pseudo_test.loc[:, (pseudo_test.columns != 'radiant_win') & (pseudo_test.columns != 'match_id_hash')]
y_test_pred = clf.predict_proba(X_test)
```

```python
print('Validation score: ',roc_auc_score(pseudo_valid['radiant_win'], y_valid_pred[:,1]))
print('Test score: ',roc_auc_score(pseudo_test['radiant_win'], y_test_pred[:,1]))
```

    Validation score:  0.8064973453175577
    Test score:  0.7990269461077844

```python

```

```python
# Работа с категориальными переменными
```

```python
pseudo_train.lobby_type.unique()
```

    array([7, 0])

```python
categorical_features_list = [ x for x in pseudo_train.columns if x.endswith('hero_id')] + ['lobby_type']
```

```python
pseudo_train[categorical_features_list]
```

      r1_hero_id
      r2_hero_id
      r3_hero_id
      r4_hero_id
      r5_hero_id
      d1_hero_id
      d2_hero_id
      d3_hero_id
      d4_hero_id
      d5_hero_id
      lobby_type

      0
      11
      78
      14
      59
      77
      12
      21
      60
      84
      34
      7

      1
      15
      96
      27
      63
      89
      58
      14
      1
      56
      92
      0

      2
      14
      99
      101
      26
      41
      18
      98
      8
      69
      86
      7

      3
      42
      69
      27
      104
      65
      23
      22
      35
      72
      1
      7

      4
      57
      10
      81
      98
      119
      48
      68
      60
      102
      8
      7

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      21
      119
      17
      84
      94
      72
      23
      27
      74
      29
      7

      28996
      101
      49
      45
      40
      35
      44
      32
      99
      91
      41
      7

      28997
      106
      41
      35
      14
      17
      22
      26
      4
      43
      44
      0

      28998
      32
      120
      30
      48
      74
      14
      21
      8
      11
      16
      7

      28999
      5
      100
      10
      22
      71
      78
      14
      50
      41
      17
      7

29000 rows × 11 columns

```python
# One hot encoding
pd.get_dummies(pseudo_train['r1_hero_id'], drop_first=True)
```

      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      ...
      107
      108
      109
      110
      111
      112
      113
      114
      119
      120

      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      1
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      1
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      2
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      3
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      4
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      1
      0

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      28996
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      28997
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      28998
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      28999
      0
      0
      0
      1
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

29000 rows × 114 columns

```python
# Mean Encoding
for col in categorical_features_list:
    means_map = pseudo_train.groupby(col).radiant_win.mean()
    pseudo_train[col+'_mean_enc'] = pseudo_train[col].map(means_map)
    # Don't forget about validation dataset
    pseudo_valid[col+'_mean_enc'] =  pseudo_valid[col].map(means_map)
    # and test dataset
    pseudo_test[col+'_mean_enc'] =  pseudo_test[col].map(means_map)
```

```python
me_list = [ x for x in pseudo_train.columns if x.endswith('_mean_enc')]
```

```python
pseudo_train[me_list]
```

      r1_hero_id_mean_enc
      r2_hero_id_mean_enc
      r3_hero_id_mean_enc
      r4_hero_id_mean_enc
      r5_hero_id_mean_enc
      d1_hero_id_mean_enc
      d2_hero_id_mean_enc
      d3_hero_id_mean_enc
      d4_hero_id_mean_enc
      d5_hero_id_mean_enc
      lobby_type_mean_enc

      0
      0.475108
      0.585106
      0.536323
      0.519737
      0.544118
      0.552846
      0.517564
      0.508772
      0.511719
      0.551181
      0.526813

      1
      0.382609
      0.537815
      0.560096
      0.527919
      0.527778
      0.584906
      0.528090
      0.554717
      0.505455
      0.472727
      0.524688

      2
      0.526758
      0.516129
      0.606272
      0.480000
      0.562887
      0.542169
      0.594595
      0.500768
      0.594340
      0.559653
      0.526813

      3
      0.583153
      0.521127
      0.560096
      0.544413
      0.439394
      0.544271
      0.402299
      0.541586
      0.542056
      0.596296
      0.526813

      4
      0.554054
      0.489209
      0.586854
      0.424051
      0.579439
      0.479866
      0.502041
      0.508772
      0.426829
      0.540193
      0.526813

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      0.492574
      0.507082
      0.538168
      0.522556
      0.433333
      0.554167
      0.551724
      0.498856
      0.537068
      0.515924
      0.526813

      28996
      0.564356
      0.558065
      0.538462
      0.530405
      0.510092
      0.544379
      0.436856
      0.463768
      0.600000
      0.542857
      0.526813

      28997
      0.451282
      0.491870
      0.517647
      0.535144
      0.478261
      0.477435
      0.530612
      0.521531
      0.539683
      0.553030
      0.524688

      28998
      0.569817
      0.493056
      0.550000
      0.545455
      0.514440
      0.540765
      0.517564
      0.500768
      0.560714
      0.567568
      0.526813

      28999
      0.533898
      0.504587
      0.503788
      0.588235
      0.496212
      0.430108
      0.528090
      0.588571
      0.540161
      0.570342
      0.526813

29000 rows × 11 columns

```python
X_mean = pseudo_train.loc[:,
                     ~pseudo_train.columns.isin(categorical_features_list +['radiant_win','match_id_hash'])]
y_mean = pseudo_train.radiant_win
clf_mean = LogisticRegressionCV(cv=5, random_state=42, n_jobs =-1, scoring = 'roc_auc').fit(X_mean, y_mean)
```

    /home/s.gafarov/.local/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      n_iter_i = _check_optimize_result(

```python
clf_mean.score(X_mean,y_mean)
```

    0.813015259450802

```python
X_valid_mean = pseudo_valid.loc[:,
                     ~pseudo_valid.columns.isin(categorical_features_list +['radiant_win','match_id_hash'])]
y_valid_pred_mean = clf_mean.predict_proba(X_valid_mean)

X_test_mean = pseudo_test.loc[:,
                     ~pseudo_test.columns.isin(categorical_features_list +['radiant_win','match_id_hash'])]
y_test_pred_mean = clf_mean.predict_proba(X_test_mean)
```

```python
print('Validation score: ',roc_auc_score(pseudo_valid['radiant_win'], y_valid_pred_mean[:,1]))
print('Test score: ',roc_auc_score(pseudo_test['radiant_win'], y_test_pred_mean[:,1]))
```

    Validation score:  0.8064732483513659
    Test score:  0.7992889221556886

```python

```

```python

```

```python
# Smoothed means
```

```python
for col in categorical_features_list:
    agg = pseudo_train.groupby(col).radiant_win.agg(['count','mean'])
    counts = agg['count']
    means = agg['mean']
    weight = 100
    smooth = (counts * means + weight * means) / (counts + weight)
    smooth_s = pseudo_train[col].map(smooth)
    pseudo_train[col+'_s_mean_enc'] = smooth_s
    # Don't forget about validation dataset
    pseudo_valid[col+'_s_mean_enc'] =  pseudo_valid[col].map(smooth)
    # and test dataset
    pseudo_test[col+'_s_mean_enc'] =  pseudo_test[col].map(smooth)
```

```python
sme_list = [ x for x in pseudo_train.columns if x.endswith('_s_mean_enc')]
```

```python
pseudo_train[sme_list]
```

      r1_hero_id_s_mean_enc
      r2_hero_id_s_mean_enc
      r3_hero_id_s_mean_enc
      r4_hero_id_s_mean_enc
      r5_hero_id_s_mean_enc
      d1_hero_id_s_mean_enc
      d2_hero_id_s_mean_enc
      d3_hero_id_s_mean_enc
      d4_hero_id_s_mean_enc
      d5_hero_id_s_mean_enc
      lobby_type_s_mean_enc

      0
      0.475108
      0.585106
      0.536323
      0.519737
      0.544118
      0.552846
      0.517564
      0.508772
      0.511719
      0.551181
      0.526813

      1
      0.382609
      0.537815
      0.560096
      0.527919
      0.527778
      0.584906
      0.528090
      0.554717
      0.505455
      0.472727
      0.524688

      2
      0.526758
      0.516129
      0.606272
      0.480000
      0.562887
      0.542169
      0.594595
      0.500768
      0.594340
      0.559653
      0.526813

      3
      0.583153
      0.521127
      0.560096
      0.544413
      0.439394
      0.544271
      0.402299
      0.541586
      0.542056
      0.596296
      0.526813

      4
      0.554054
      0.489209
      0.586854
      0.424051
      0.579439
      0.479866
      0.502041
      0.508772
      0.426829
      0.540193
      0.526813

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      0.492574
      0.507082
      0.538168
      0.522556
      0.433333
      0.554167
      0.551724
      0.498856
      0.537068
      0.515924
      0.526813

      28996
      0.564356
      0.558065
      0.538462
      0.530405
      0.510092
      0.544379
      0.436856
      0.463768
      0.600000
      0.542857
      0.526813

      28997
      0.451282
      0.491870
      0.517647
      0.535144
      0.478261
      0.477435
      0.530612
      0.521531
      0.539683
      0.553030
      0.524688

      28998
      0.569817
      0.493056
      0.550000
      0.545455
      0.514440
      0.540765
      0.517564
      0.500768
      0.560714
      0.567568
      0.526813

      28999
      0.533898
      0.504587
      0.503788
      0.588235
      0.496212
      0.430108
      0.528090
      0.588571
      0.540161
      0.570342
      0.526813

29000 rows × 11 columns

```python
X_sm = pseudo_train.loc[:,
                     ~pseudo_train.columns.isin(categorical_features_list +
                                                ['radiant_win','match_id_hash'] + me_list)]
y_sm = pseudo_train.radiant_win
clf_sm = LogisticRegressionCV(cv=5, random_state=42, n_jobs =-1, scoring = 'roc_auc').fit(X_sm, y_sm)
```

    /home/s.gafarov/.local/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      n_iter_i = _check_optimize_result(

```python
clf_sm.score(X_sm,y_sm)
```

    0.8133197078071

```python
X_valid_sm = pseudo_valid.loc[:,
                     ~pseudo_valid.columns.isin(categorical_features_list +['radiant_win','match_id_hash']
                                               + me_list)]
y_valid_pred_sm = clf_sm.predict_proba(X_valid_sm)

X_test_sm = pseudo_test.loc[:,
                     ~pseudo_test.columns.isin(categorical_features_list +['radiant_win','match_id_hash']
                                              + me_list)]
y_test_pred_sm = clf_sm.predict_proba(X_test_sm)
```

```python
print('Validation score: ',roc_auc_score(pseudo_valid['radiant_win'], y_valid_pred_sm[:,1]))
print('Test score: ',roc_auc_score(pseudo_test['radiant_win'], y_test_pred_sm[:,1]))
```

    Validation score:  0.8067182341743174
    Test score:  0.801375374251497

```python

```

```python

```

```python
# WOE
```

\begin{equation}
WOE =(ln(\frac{GoodDistr}{BadDistr}))* 100
\end{equation}

```python
for col in categorical_features_list:
    agg = pseudo_train.groupby(col).radiant_win.mean()
    woe_df = pd.DataFrame(agg).rename(columns={'radiant_win':"WOE_Good"})
    woe_df['WOE_Bad'] = 1 - woe_df['WOE_Good']
    woe_df['WOE_Bad'] = np.where(woe_df['WOE_Bad'] == 0, 0.00001, woe_df['WOE_Bad'])
    woe_df['woe'] = np.log(woe_df.WOE_Good/woe_df.WOE_Bad)

    woe = pseudo_train[col].map(woe_df['woe'])
    pseudo_train[col+'_woe'] = woe

    # Don't forget about validation dataset
    pseudo_valid[col+'_woe'] =  pseudo_valid[col].map(woe_df['woe'])
    # and test dataset
    pseudo_test[col+'_woe'] =  pseudo_test[col].map(woe_df['woe'])
```

```python
woe_list = [ x for x in pseudo_train.columns if x.endswith('_woe')]
```

```python
pseudo_train[woe_list]
```

      r1_hero_id_woe
      r2_hero_id_woe
      r3_hero_id_woe
      r4_hero_id_woe
      r5_hero_id_woe
      d1_hero_id_woe
      d2_hero_id_woe
      d3_hero_id_woe
      d4_hero_id_woe
      d5_hero_id_woe
      lobby_type_woe

      0
      -0.099649
      0.343772
      0.145548
      0.078988
      0.176931
      0.212175
      0.070287
      0.035091
      0.046884
      0.205444
      0.107356

      1
      -0.478490
      0.151550
      0.241552
      0.111791
      0.111226
      0.342945
      0.112478
      0.219748
      0.021819
      -0.109199
      0.098832

      2
      0.107136
      0.064539
      0.431667
      -0.080043
      0.252886
      0.169076
      0.382992
      0.003072
      0.381935
      0.239754
      0.107356

      3
      0.335732
      0.084557
      0.241552
      0.178120
      -0.243622
      0.177548
      -0.395896
      0.166729
      0.168623
      0.390056
      0.107356

      4
      0.217065
      -0.043172
      0.350977
      -0.306167
      0.320472
      -0.080580
      0.008163
      0.035091
      -0.294800
      0.161119
      0.107356

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      -0.029705
      0.028331
      0.152969
      0.090287
      -0.268264
      0.217520
      0.207639
      -0.004577
      0.148543
      0.063716
      0.107356

      28996
      0.258862
      0.233311
      0.154151
      0.121772
      0.040372
      0.177983
      -0.253933
      -0.145182
      0.405465
      0.171850
      0.107356

      28997
      -0.195492
      -0.032523
      0.070618
      0.140807
      -0.087011
      -0.090323
      0.122602
      0.086178
      0.159065
      0.212922
      0.098832

      28998
      0.281103
      -0.027780
      0.200671
      0.182322
      0.057778
      0.163424
      0.070287
      0.003072
      0.244061
      0.271934
      0.107356

      28999
      0.135802
      0.018349
      0.015152
      0.356675
      -0.015152
      -0.281412
      0.112478
      0.358063
      0.160989
      0.283247
      0.107356

29000 rows × 11 columns

```python
X_woe = pseudo_train.loc[:,
                     ~pseudo_train.columns.isin(categorical_features_list +
                                                ['radiant_win','match_id_hash'] + me_list + sme_list)]
y_woe = pseudo_train.radiant_win
clf_woe = LogisticRegressionCV(cv=5, random_state=42, n_jobs =-1, scoring = 'roc_auc').fit(X_woe, y_woe)
```

    /home/s.gafarov/.local/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      n_iter_i = _check_optimize_result(

```python
clf_woe.score(X_woe,y_woe)
```

    0.8138006256488719

```python
X_valid_woe = pseudo_valid.loc[:,
                     ~pseudo_valid.columns.isin(categorical_features_list +['radiant_win','match_id_hash']
                                               + me_list + sme_list)]
y_valid_pred_woe = clf_woe.predict_proba(X_valid_woe)

X_test_woe = pseudo_test.loc[:,
                     ~pseudo_test.columns.isin(categorical_features_list +['radiant_win','match_id_hash']
                                              + me_list + sme_list)]
y_test_pred_woe = clf_woe.predict_proba(X_test_woe)
```

```python
print('Validation score: ',roc_auc_score(pseudo_valid['radiant_win'], y_valid_pred_woe[:,1]))
print('Test score: ',roc_auc_score(pseudo_test['radiant_win'], y_test_pred_woe[:,1]))
```

    Validation score:  0.8079953733824911
    Test score:  0.8012818113772455

```python

```

```python

```

```python
# Expanding mean
```

```python
for col in categorical_features_list:
    cumsum = pseudo_train.groupby(col).radiant_win.cumsum() - pseudo_train['radiant_win']
    cumcnt = pseudo_train.groupby(col).cumcount() + 1
    exp_mean = cumsum/cumcnt
    pseudo_train[col+ '_exp_mean'] = exp_mean
```

```python
exp_mean_list = [ x for x in pseudo_train.columns if x.endswith('_exp_mean')]
```

```python
pseudo_train[exp_mean_list]
```

      r1_hero_id_exp_mean
      r2_hero_id_exp_mean
      r3_hero_id_exp_mean
      r4_hero_id_exp_mean
      r5_hero_id_exp_mean
      d1_hero_id_exp_mean
      d2_hero_id_exp_mean
      d3_hero_id_exp_mean
      d4_hero_id_exp_mean
      d5_hero_id_exp_mean
      lobby_type_exp_mean

      0
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000

      1
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000

      2
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000

      3
      0.000000
      0.000000
      0.500000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.333333

      4
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.250000

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      28995
      0.520661
      0.500000
      0.540670
      0.492135
      0.578947
      0.474063
      0.518095
      0.518519
      0.523238
      0.522523
      0.527013

      28996
      0.495074
      0.511429
      0.540541
      0.535316
      0.430464
      0.559322
      0.562025
      0.490698
      0.534653
      0.516129
      0.527037

      28997
      0.448980
      0.491903
      0.512922
      0.535200
      0.472656
      0.472019
      0.532164
      0.509524
      0.535433
      0.550864
      0.523779

      28998
      0.569848
      0.486395
      0.555819
      0.550523
      0.518116
      0.542714
      0.516129
      0.493808
      0.557945
      0.560000
      0.527010

      28999
      0.529536
      0.501558
      0.500000
      0.580563
      0.496241
      0.457447
      0.524476
      0.578652
      0.534274
      0.570342
      0.527034

29000 rows × 11 columns

```python

```

```python

```

```python

```

```python

```

```python
# Simple stacking
new_validation_df = pseudo_valid[['match_id_hash','radiant_win']]
new_validation_df.loc[:,'le_pred'] = y_valid_pred[:, 1]
new_validation_df.loc[:,'me_pred'] = y_valid_pred_mean[:, 1]
new_validation_df.loc[:,'sme_pred'] = y_valid_pred_sm[:, 1]
new_validation_df.loc[:,'woe_pred'] = y_valid_pred_woe[:, 1]

new_test_df = pseudo_test[['match_id_hash','radiant_win']]
new_test_df.loc[:,'le_pred'] = y_test_pred[:, 1]
new_test_df.loc[:,'me_pred'] = y_test_pred_mean[:, 1]
new_test_df.loc[:,'sme_pred'] = y_test_pred_sm[:, 1]
new_test_df.loc[:,'woe_pred'] = y_test_pred_woe[:, 1]
```

    /home/s.gafarov/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1597: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
      self.obj[key] = value
    /home/s.gafarov/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1676: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
      self._setitem_single_column(ilocs[0], value, pi)

```python
X_meta = new_validation_df.loc[:,
                     ~new_validation_df.columns.isin(['radiant_win', 'match_id_hash'])]
y_meta = new_validation_df.radiant_win
clf_meta = LogisticRegressionCV(cv=5, random_state=42, n_jobs =-1, scoring = 'roc_auc').fit(X_meta, y_meta)
```

```python
clf_meta.score(X_meta,y_meta)
```

    0.8082885531378265

```python
X_test_meta = new_test_df.loc[:,
                     ~new_test_df.columns.isin(['radiant_win','match_id_hash'])]
y_test_meta = clf_meta.predict_proba(X_test_meta)
```

```python
print('Test score: ',roc_auc_score(pseudo_test['radiant_win'], y_test_meta[:,1]))
```

    Test score:  0.8027039670658682

```python

```

```python

```

```python

```

```python

```

```python

```

```python
from sklearn.model_selection import StratifiedKFold
```

```python
train = pd.read_csv('pseudo_train_df.csv')
test = pd.read_csv('pseudo_test_df.csv')
```

```python
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=101)
train_new = pd.DataFrame()

X_ = train.loc[:,
               ~train.columns.isin(['radiant_win','match_id_hash'])]

y_ = train.radiant_win

for tr_ind, val_ind in skf.split(X_,y_):
    cv_trian , cv_valid = train.loc[tr_ind], train.loc[val_ind]
    X_test = test.copy()
    # Mean Encoding
    for col in categorical_features_list:
        means_map = cv_trian.groupby(col).radiant_win.mean()
        # Don't forget about validation dataset
        cv_valid[col + '_mean_enc'] =  cv_valid[col].map(means_map)
        # and test dataset
        X_test[col+'_mean_enc'] =  X_test[col].map(means_map)
    train_new = pd.concat([train_new,cv_valid])

    #your code here

```

```python
train.shape
```

    (29999, 247)

```python
train_new.shape
```

    (29999, 258)

```python
train_new
```

      match_id_hash
      game_time
      game_mode
      lobby_type
      objectives_len
      chat_len
      r1_hero_id
      r1_kills
      r1_deaths
      r1_assists
      ...
      r2_hero_id_mean_enc
      r3_hero_id_mean_enc
      r4_hero_id_mean_enc
      r5_hero_id_mean_enc
      d1_hero_id_mean_enc
      d2_hero_id_mean_enc
      d3_hero_id_mean_enc
      d4_hero_id_mean_enc
      d5_hero_id_mean_enc
      lobby_type_mean_enc

      3
      b1b35ff97723d9b7ade1c9c3cf48f770
      453
      22
      7
      1
      3
      42
      0
      1
      1
      ...
      0.497207
      0.587021
      0.535461
      0.442308
      0.528302
      0.396610
      0.548694
      0.558140
      0.621622
      0.527060

      4
      19c39fe2af2b547e48708ca005c6ae74
      160
      22
      7
      0
      0
      57
      0
      0
      0
      ...
      0.500000
      0.588889
      0.458647
      0.589431
      0.468504
      0.523364
      0.529412
      0.427481
      0.552326
      0.527060

      13
      ab3cc6ccac661a1385e73a2e9f21313a
      721
      4
      0
      2
      1
      30
      2
      2
      1
      ...
      0.548837
      0.488152
      0.498645
      0.482540
      0.568182
      0.465217
      0.565217
      0.474227
      0.527157
      0.523810

      15
      34c81a8faede0d8f1f87dcc6ee824658
      1362
      4
      0
      6
      4
      39
      1
      1
      4
      ...
      0.479730
      0.566667
      0.574879
      0.505618
      0.492308
      0.542986
      0.556452
      0.578035
      0.553191
      0.523810

      16
      5feece770ca79e5e8cd8052198b3f533
      2388
      4
      0
      16
      10
      103
      9
      8
      18
      ...
      0.492268
      0.513812
      0.574074
      0.554386
      0.524510
      0.530387
      0.514286
      0.471390
      0.518182
      0.523810

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      29973
      b3816a237a4ab69585cfc464194d0a39
      1683
      22
      0
      7
      0
      22
      9
      3
      7
      ...
      0.452489
      0.517094
      0.547151
      0.564220
      0.591667
      0.583333
      0.644737
      0.525847
      0.522059
      0.525671

      29982
      676f0d3a2e2724336dabc96c78324a2f
      2263
      22
      7
      21
      0
      119
      6
      9
      18
      ...
      0.517007
      0.486034
      0.519841
      0.560748
      0.474104
      0.523810
      0.562500
      0.606618
      0.508091
      0.526152

      29985
      1d6fc47e76dedb38169282edfe96b20b
      518
      2
      7
      1
      0
      95
      0
      0
      0
      ...
      0.502075
      0.537037
      0.555556
      0.497297
      0.519553
      0.636364
      0.436937
      0.471154
      0.627530
      0.526152

      29997
      af3bd887e502915daaab9545f8294af3
      62
      22
      7
      0
      4
      32
      0
      0
      0
      ...
      0.486957
      0.568966
      0.530612
      0.509890
      0.543347
      0.513661
      0.491396
      0.557664
      0.570370
      0.526152

      29998
      190d0fc8a7eddcae6ca11885ac3574db
      398
      22
      7
      1
      0
      5
      0
      1
      0
      ...
      0.513208
      0.525114
      0.594595
      0.533040
      0.460317
      0.523810
      0.582734
      0.537129
      0.561905
      0.526152

29999 rows × 258 columns

```python

```

## Stacking здорово человека

```python
#from https://machinelearningmastery.com/stacking-ensemble-machine-learning-with-python/
from numpy import mean
from numpy import std
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import RepeatedStratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.ensemble import StackingClassifier
from matplotlib import pyplot

# get the dataset
def get_dataset():
	X, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)
	return X, y

# get a stacking ensemble of models
def get_stacking():
	# define the base models
	level0 = list()
	level0.append(('lr', LogisticRegression()))
	level0.append(('knn', KNeighborsClassifier()))
	level0.append(('cart', DecisionTreeClassifier()))
	level0.append(('svm', SVC()))
	level0.append(('bayes', GaussianNB()))
	# define meta learner model
	level1 = LogisticRegression()
	# define the stacking ensemble
	model = StackingClassifier(estimators=level0, final_estimator=level1, cv=5)
	return model

# get a list of models to evaluate
def get_models():
	models = dict()
	models['lr'] = LogisticRegression()
	models['knn'] = KNeighborsClassifier()
	models['cart'] = DecisionTreeClassifier()
	models['svm'] = SVC()
	models['bayes'] = GaussianNB()
	models['stacking'] = get_stacking()
	return models

# evaluate a give model using cross-validation
def evaluate_model(model, X, y):
	cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)
	scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
	return scores

# define dataset
X, y = get_dataset()
# get the models to evaluate
models = get_models()
# evaluate the models and store results
results, names = list(), list()
for name, model in models.items():
	scores = evaluate_model(model, X, y)
	results.append(scores)
	names.append(name)
	print('>%s %.3f (%.3f)' % (name, mean(scores), std(scores)))
# plot model performance for comparison
pyplot.boxplot(results, labels=names, showmeans=True)
pyplot.show()
```

```python

```

```python

```

### Полезные ссылки
https://www.coursera.org/lecture/competitive-data-science/regularization-LGYQ2
https://www.coursera.org/specializations/aml
https://towardsdatascience.com/all-about-categorical-variable-encoding-305f3361fd02
https://machinelearningmastery.com/stacking-ensemble-machine-learning-with-python/
https://www.coursera.org/lecture/competitive-data-science/stacknet-s8RLi

```python

```

```python

```

```python

```

```python

```


```python
# Change directory to VSCode workspace root so that relative path loads work correctly. Turn this addition off with the DataScience.changeDirOnImportExport setting
import os
try:
	os.chdir(os.path.join(os.getcwd(), '..'))
	print(os.getcwd())
except:
	pass

```

```python
import os
try:
    os.chdir(os.path.join(os.getcwd(), 'course_berkeley/homework/hw2'))
    print(os.getcwd())
except:
    pass
```

 # Problem 1
 We want to show that
 \begin{equation}
 \sum_{t'=1}^{T}\mathbb{E}_{\tau \sim p_{\theta}(\tau)}[\nabla_{\theta}log \pi_{\theta}(a_{t'}|s_{t'})(b(s_{t'}))]=0
 \end{equation}
 \begin{align}
 \sum_{t=1}^{T}\mathbb{E}_{\tau \sim p_{\theta}(\tau)}[\nabla_{\theta}log \pi_{\theta}(a_t|s_t)(b(s_t))]
 &=\sum_{t=1}^{T}\int p_{\theta}(\tau)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)(b(s_t))d\tau\\
 %&=\sum_{t=1}^{T}\int p_{\theta}(s_t,a_t)p_{\theta}(\tau / s_t,a_t\mid s_t,a_t)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)(b(s_t))d\tau\\
 &=\sum_{t=1}^{T}\int \int p_{\theta}(s_t,a_t)(\int p_{\theta}(\tau / s_t,a_t\mid s_t,a_t)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)(b(s_t))d\tau )d s_t da_t)\\
 &=\sum_{t=1}^{T}\int \int p_{\theta}(s_t,a_t)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)b(s_t)\int p_{\theta}(\tau / s_t,a_t\mid s_t,a_t)d\tau d s_t d a_t\\
 &=\sum_{t=1}^{T}\int \int p_{\theta}(s_t,a_t)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)b(s_t) d s_t d a_t\\
 &=\sum_{t=1}^{T}\int \int p(s_{t+1}|a_t,s_t)\pi_{\theta}(a_t|s_t)\nabla_{\theta}log \pi_{\theta}(a_t|s_t)b(s_t) d s_t d a_t\\
 &=\sum_{t=1}^{T}\int \int p(s_{t+1}|a_t,s_t)b(s_t)\nabla_{\theta}\pi_{\theta}(a_t|s_t) d s_t d a_t\\
 &=\sum_{t=1}^{T} \nabla_{\theta} \int \int p(s_{t+1}|a_t,s_t)b(s_t)\pi_{\theta}(a_t|s_t) d s_t d a_t \\
 &=\sum_{t=1}^{T} \nabla_{\theta} \int \int p_{\theta}(a_t,s_t)b(s_t) d s_t d a_t \\
 \end{align}
Where $\nabla_{\theta}log \pi_{\theta}(A_t|S_t)b(S_t)=b(S_t)\nabla_{\theta}log \pi_{\theta}(A_t|S_t)$, since $b$ is assumed not to depend on $\theta$ (in practice it could be estimated using a different neural network with parameters $\phi$).

```python
  &=\mathbb{E}_{(A_t,S_t)}[\mathbb{E}_{\tau / (A_t,S_t) \sim p_{\theta}(\tau / S_t,A_t\mid S_t,A_t)}[0\mid A_t , S_t]]\\
  &= \mathbf{0}
 Where $\nabla_{\theta}log \pi_{\theta}(A_t|S_t)(b(S_t)) = 0$ given $A_t$ and $S_t$. Therefore
```

```python
from train_pg_f18 import train_PG
seed = 1
train_PG(
                exp_name="sb_no_rtg_dna",
                env_name="CartPole-v0",
                n_iter=100,
                gamma=0.99,
                min_timesteps_per_batch=1000,
				learning_rate=0.001,
				reward_to_go=None,
                max_path_length=99999999999,
                animate=False,
                logdir= None , #os.path.join("data",'%d'%seed),
				normalize_advantages=False,
                nn_baseline=None,
                seed=seed,
                n_layers=2,
                size=100
                )
```

```python
%load_ext autoreload

%autoreload 2
```

## Test continous  actions
We use the logprob expression
```
sy_logprob_n =  -1/2 * tf.reduce_sum(inverse_variance *(diff*diff),axis=1)
```
Even though
```
sy_logprob_n = - 1/2*(tf.log(variance)  + tf.reduce_sum(inverse_variance *(diff*diff),axis=1))
```
would have been more correct as to follow the logprob. For this small example this makes the variance be maximized after, and thus gives more incorrect results, however for the real examples it performes the same, probably due to maximization errors when doing the other thing. It is also what is suggested in Levines slides.

```python
from train_pg_f18 import train_PG, Agent
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
computation_graph_args = {
    'n_layers': 1,
    'ob_dim': 3,
    'ac_dim': 1,
    'discrete': False,
    'size': 2,
    'learning_rate': 0.005,
}

sample_trajectory_args = {
    'animate': False,
    'max_path_length': 1000,
    'min_timesteps_per_batch': 1000,
}

estimate_return_args = {
    'gamma': 0.99,
    'reward_to_go': True,
    'nn_baseline': False,
    'normalize_advantages': False,
}

with tf.Graph().as_default() as graph:
    agent = Agent(computation_graph_args, sample_trajectory_args, estimate_return_args)
    agent.build_computation_graph()
    tf_config = tf.ConfigProto(inter_op_parallelism_threads=1,
        intra_op_parallelism_threads=1,
        gpu_options = tf.GPUOptions(
            #per_process_gpu_memory_fraction=1./16. # 1gb
            allow_growth=True
        )
    )
    with tf.Session(config=tf_config).as_default() as sess:

        sess.__enter__() # equivalent to `with self.sess:`
        tf.global_variables_initializer().run() #pylint: disable=E1101

        # check that the loss is computed correct
        print(sess.run([agent.policy_parameters, agent.sy_logprob_n, agent.loss], {
            agent.sy_ob_no: [[0.5,0.5,0.5],
                            [0.5,0.5,0.5],
                            [0.5,0.5,0.5]],
            agent.sy_ac_na: [[2.0], [2.0], [2.0]],
            agent.sy_adv_n: [1,1,1]

        })
        )
        # check that the actions look correct
        n_samples = 100
        sampled_actions = sess.run(agent.sy_sampled_ac, {
            agent.sy_ob_no: [[0.5,0.5,0.5]]*n_samples,
            #agent.sy_ac_na: [[2.0]]*n_samples,
            #agent.sy_adv_n: [1]*n_samples

        })
        figure =  plt.figure()
        plot = plt.hist(sampled_actions)
        figure.show()
        print("std: {}, mean: {}".format(np.std(sampled_actions),np.mean(sampled_actions)))
        n_training = 4000
        for i in range(n_training):
            sy_ac_na = np.random.normal(loc=5.0 , scale=0.1,size=n_samples)
            sess.run(agent.update_op, {
                agent.sy_ob_no: [[0.5,0.5,0.5]]*n_samples,
                agent.sy_ac_na: sy_ac_na.reshape(-1,1),
                agent.sy_adv_n: [1]*n_samples

            })
            sampled_actions = sess.run(agent.sy_sampled_ac, {
                agent.sy_ob_no: [[0.5,0.5,0.5]]*n_samples,
                #agent.sy_ac_na: [[2.0]]*n_samples,
                #agent.sy_adv_n: [1]*n_samples
            })
            if i %100 == 99:
                print("it: {}, std: {}, mean: {}".format(i, np.std(sampled_actions),np.mean(sampled_actions)))

        figure =  plt.figure()
        plot = plt.hist(sampled_actions)
        figure.show()
```

    [(array([[0.],
           [0.],
           [0.]], dtype=float32), array([0.], dtype=float32)), array([-2., -2., -2.], dtype=float32), 2.0]
    std: 0.9857619404792786, mean: 0.053708259016275406

    /zhome/30/0/70339/.local/lib/python3.6/site-packages/matplotlib/figure.py:445: UserWarning: Matplotlib is currently using module://ipykernel.pylab.backend_inline, which is a non-GUI backend, so cannot show the figure.
      % get_backend())

    it: 99, std: 4.66080379486084, mean: 5.440565586090088
    it: 199, std: 6.097592830657959, mean: 4.187251091003418
    it: 299, std: 44.20418167114258, mean: 9.711440086364746
    it: 399, std: 111.15580749511719, mean: -4.385540962219238
    it: 499, std: 151.47842407226562, mean: 27.827970504760742
    it: 599, std: 186.75601196289062, mean: -14.619833946228027
    it: 699, std: 214.3723907470703, mean: 23.425809860229492
    it: 799, std: 224.56509399414062, mean: -21.883249282836914
    it: 899, std: 266.2421875, mean: -15.566503524780273
    it: 999, std: 271.6304626464844, mean: 19.704458236694336
    it: 1099, std: 304.3190002441406, mean: 26.475332260131836
    it: 1199, std: 300.62518310546875, mean: 10.948735237121582
    it: 1299, std: 336.2844543457031, mean: 23.704082489013672
    it: 1399, std: 375.9537353515625, mean: 68.61624145507812
    it: 1499, std: 345.3172302246094, mean: 39.9020881652832
    it: 1599, std: 362.7534484863281, mean: 26.962749481201172
    it: 1699, std: 387.88238525390625, mean: -39.475982666015625
    it: 1799, std: 368.72686767578125, mean: 40.964393615722656
    it: 1899, std: 415.8846130371094, mean: -0.6535653471946716
    it: 1999, std: 399.17840576171875, mean: -9.46063232421875
    it: 2099, std: 402.3943176269531, mean: 56.75299835205078
    it: 2199, std: 426.382568359375, mean: -9.584866523742676
    it: 2299, std: 397.1225280761719, mean: 32.09247589111328
    it: 2399, std: 440.0720520019531, mean: 11.018421173095703
    it: 2499, std: 448.4039306640625, mean: -24.780982971191406
    it: 2599, std: 431.1240539550781, mean: -5.091939926147461
    it: 2699, std: 454.5281066894531, mean: 127.94454193115234
    it: 2799, std: 522.6281127929688, mean: -42.318817138671875
    it: 2899, std: 539.570068359375, mean: 48.2517204284668
    it: 2999, std: 541.5744018554688, mean: -17.586069107055664
    it: 3099, std: 556.2981567382812, mean: 21.580507278442383
    it: 3199, std: 577.2489013671875, mean: 22.744112014770508
    it: 3299, std: 580.4326171875, mean: -20.103782653808594
    it: 3399, std: 575.3218994140625, mean: 100.09042358398438
    it: 3499, std: 584.818603515625, mean: 65.43984985351562
    it: 3599, std: 532.0647583007812, mean: 60.5954704284668
    it: 3699, std: 582.7791748046875, mean: -98.67958068847656
    it: 3799, std: 574.6396484375, mean: -18.537797927856445
    it: 3899, std: 621.84228515625, mean: 28.96257781982422
    it: 3999, std: 598.8465576171875, mean: -39.415287017822266

## Test baseline

```python
from train_pg_f18 import train_PG, Agent
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
computation_graph_args = {
    'n_layers': 1,
    'ob_dim': 3,
    'ac_dim': 1,
    'discrete': False,
    'size': 2,
    'learning_rate': 0.005,
}

sample_trajectory_args = {
    'animate': False,
    'max_path_length': 1000,
    'min_timesteps_per_batch': 1000,
}

estimate_return_args = {
    'gamma': 0.99,
    'reward_to_go': True,
    'nn_baseline': True,
    'normalize_advantages': False,
}

with tf.Graph().as_default() as graph:
    agent = Agent(computation_graph_args, sample_trajectory_args, estimate_return_args)
    agent.build_computation_graph()
    tf_config = tf.ConfigProto(inter_op_parallelism_threads=1,
        intra_op_parallelism_threads=1,
        gpu_options = tf.GPUOptions(
            #per_process_gpu_memory_fraction=1./16. # 1gb
            allow_growth=True
        )
    )
    with tf.Session(config=tf_config).as_default() as sess:

        sess.__enter__() # equivalent to `with sess:`
        tf.global_variables_initializer().run() #pylint: disable=E1101

        # check that the actions look correct
        n_samples = 1
        ob_no =  [[0.0,0.0,0.0],[-1.,-1.,-1.]]*n_samples
        q_n = [1,-1]*n_samples
        value = sess.run(agent.baseline_prediction, {
                agent.sy_ob_no: ob_no
            })
        print("Value of state before {}".format(value))
        target_n = (q_n - np.mean(q_n))/np.std(q_n)
        print("Target values {}".format(target_n))
        n_training = 1000
        print("Number of training baseline: {}".format(n_training))
        for _ in range(n_training):
            sess.run(agent.baseline_update_op, {
                agent.sy_ob_no: ob_no,
                agent.sy_target_n: target_n
            })
            value = sess.run(agent.baseline_prediction, {
                    agent.sy_ob_no: ob_no
                })
        print("Value of state after {}".format(value))
```

    Value of state before [0. 0.]
    Target values [ 1. -1.]
    Number of training baseline: 1000
    Value of state after [ 0.9999992 -0.9999991]

```python
from train_pg_f18 import train_PG, Agent
import tensorflow as tf
with tf.Graph().as_default() as graph:

    with tf.Session(config=tf_config).as_default() as sess:

        sess.__enter__() # equivalent to `with self.sess:`
        tf.global_variables_initializer().run() #pylint: disable=E1101
        batch  = tf.constant([[1.,2.,3.,4.],[5.,6.,7.,8.]])

        #indices =  tf.expand_dims(tf.constant([1,2]),axis=1)
        indices =  tf.expand_dims(tf.constant([1,2]),axis=1)
        print(tf.squeeze(tf.batch_gather(batch,indices)).eval())

```

    [2. 7.]