# SMART LASER NLP

## INDEX

1.2 IMPORTS

1.3 DATASET GATHERING AND CLEANING

1.4 DATASET ANALYSIS

1.5 VOCABULARY

1.6 RNN MODEL

1.7 TRAINING

1.8 MANUAL TESTING

2 Create my own dataset? I would like to try this. Creating my own template equations and text.

## IMPORTS

```python
#Data management
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import collections
import itertools

pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
#System
import sys
import json
import unicodedata
import os
import time

#Deep Learning
import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Embedding, Input, GRU, LSTM, Dense, Dropout
from tensorflow.python.client import device_lib  #print(device_lib.list_local_devices())

#Algebra
import sympy
#import scipy
```

## DATASET GATHERING AND CLEANING

```python
#Read the datasets. We will first use this subset instead of the bigger one (Dolphin18k) because this one is cleaner (only linear) and
#might be enough for our
def load_data():
    math_dataset = pd.read_excel('arithmeticquestionsDataset/arithmeticquestions.xlsx')
    math_dataset = math_dataset[['Question', 'Equation', 'Answer']]

    math_dataset['Question'] = math_dataset['Question'].apply(round_if_str2float)
    math_dataset['Equation'] = math_dataset['Equation'].apply(round_if_str2float)
    math_dataset['Answer'] = math_dataset['Answer'].apply(lambda x: round(x, 2))

    math_dataset.Question, math_dataset.Equation = numberMapping(math_dataset.Question, math_dataset.Equation)

    math_dataset = math_dataset.sample(frac=1).reset_index(drop=True)

    return math_dataset

#Round numbers on questions and equations to 3 decimal numbers.
def round_if_str2float(row):
    newStr = ''
    for el in row.split(' '):
        try:
            el = float(el)
            el = round(el, 2)
            el = str(el)
            newStr = newStr + ' ' + el
        except (ValueError, TypeError) as er:
            newStr = newStr + ' ' + el
    return newStr

def unicodeToAscii(e):
    return ''.join(
        c for c in unicodedata.normalize('NFD', e)
        if unicodedata.category(c) != 'Mn'
    )

def normalizeEquations(e):#lo usaba para el anterior dataset, no para este
    e = unicodeToAscii(e)
    # ensure each math symbol is it's own token
    e = "".join([ c if c.isalnum() else " {} ".format(c) for c in e])
    #Remove the unknowns part from the equation
    e = e[e.index(';') + 1:]
    #All characters in equation separated by a whitespace
    e = " ".join(e.split())

    return e

def numberMapping(input_tensor, target_tensor):#Esto será para sustituir los números del texto de los problemas por incógnitas n1, n2, n3, ... , nx. El modelo aprenderá mejor así.
    variables = ['n1', 'n2', 'n3', 'n4', 'n5']
    numberMapping = {}

    for i, (inp, targ) in enumerate(zip(input_tensor, target_tensor)):
        numberMapping[i] = []
        newStr = ''
        n = 0
        for el in targ.split(' '):
            try:
                el = str(float(el))# para que de error si no es convertible a float
                numberMapping[i].append(el)
                el = variables[n]
                n += 1

                newStr = newStr + ' ' + el
            except (ValueError, TypeError) as er:
                newStr = newStr + ' ' + el

        target_tensor[i] = newStr

        newStr = ''
        n = 0
        for el in inp.split(' '):
            try:
                el = str(float(el))# para que de error si no es convertible a float
                for n in range(len(numberMapping[i])):
                    if el == numberMapping[i][n]:
                        el = variables[n]
                        n += 1
                        break

                newStr = newStr + ' ' + el
            except (ValueError, TypeError) as er:
                newStr = newStr + ' ' + el

        input_tensor[i] = newStr

    return input_tensor, target_tensor

    pass

def data_splitter(input_tensor, target_tensor, percentageToTrain):
    rowsToTrain = int(percentageToTrain*len(input_tensor))
    input_tensor_train = input_tensor[:rowsToTrain]
    input_tensor_test = input_tensor[rowsToTrain:]

    target_tensor_train = target_tensor[:rowsToTrain]
    target_tensor_test = target_tensor[rowsToTrain:]
    return input_tensor_train, input_tensor_test, target_tensor_train, target_tensor_test

#Dataset description
math_dataset = load_data()
math_dataset
```

    C:\Users\Hermanos\Anaconda3\envs\tf-gpu-cuda8\lib\site-packages\ipykernel_launcher.py:67: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
    C:\Users\Hermanos\Anaconda3\envs\tf-gpu-cuda8\lib\site-packages\ipykernel_launcher.py:84: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

      Question
      Equation
      Answer

      0
      There are n1 oak trees currently in the park...
      X = n1 - n2
      15.00

      1
      Dan had n1 peaches and 10.0 pears at his roa...
      n1 + X = n2
      71.00

      2
      Sandy had 36.0 pennies and n1 nickels in her...
      X = n1 - n2
      11.00

      3
      A construction company bought n1 tons of gra...
      X = n1 + n2
      14.02

      4
      Jason has n1 Pokemon cards . Alyssa bought n...
      X = n1 - n2
      452.00

      5
      There are n1 walnut trees and 12.0 orange tr...
      X = n1 - n2
      29.00

      6
      There are n1 pencils and n2 rulers in the dr...
      X = n1 - n2
      5.00

      7
      Benny received n1 dollars and 9.0 movie tick...
      X = n1 - n2
      47.00

      8
      Alyssa 's dog had puppies . She gave n1 to h...
      X = n1 + n2
      12.00

      9
      Greg and Sharon own neighboring cornfields ....
      X = n1 - n2
      0.30

      10
      Paco 's Countertop Company purchased pieces ...
      X = n1 + n2 + n3
      0.75

      11
      Sally had n1 baseball cards , and 9.0 were t...
      X = n1 - n2
      15.00

      12
      Mike went to n1 basketball games this year ,...
      X = n1 + n2
      54.00

      13
      A farmer started the day with n1 buckets of ...
      X = n1 - n2
      2.75

      14
      Jason has n1 books and he has read 9.0 of th...
      X = n1 + n2
      60.00

      15
      An oil pipe in the sea broke . Before engine...
      X = n1 + n2
      11687.00

      16
      Gordon bought n1 pounds of fruit for a class...
      X = n1 - n2
      1.22

      17
      Joan picked n1 apples from the orchard , and...
      X = n1 - n2
      16.00

      18
      Karin 's science class weighed plastic rings...
      X = n1 + n2 + n3
      0.83

      19
      There are n1 walnut trees currently in the p...
      X = n1 + n2
      211.00

      20
      Sam had 7.0 pennies and n1 dimes in his bank...
      X = n1 - n2
      4.00

      21
      Jason went to n1 football games this month ....
      X = n1 + n2 + n3
      44.00

      22
      Students at Arcadia schools are participatin...
      X = n1 - n2
      2515.00

      23
      It rained n1 inches on Monday . On Tuesday ,...
      X = n1 - n2
      0.20

      24
      There are n1 maple trees currently in the pa...
      n1 + X = n2
      11.00

      25
      A carpenter bought a piece of wood that was ...
      X = n1 - n2
      0.08

      26
      Molly owns the Wafting Pie Company . This mo...
      X = n1 - n2
      523.00

      27
      Sam found n1 seashells on the beach , he gav...
      X = n1 - n2
      17.00

      28
      Melanie picked n1 plums and 4.0 oranges from...
      X = n1 - n2
      4.00

      29
      Sara grew n1 pumpkins , but the rabbits ate ...
      X = n1 - n2
      20.00

      30
      Alyssa went to n1 soccer games this year , b...
      X = n1 + n2 + n3
      39.00

      31
      A restaurant made n1 hamburgers to serve dur...
      X = n1 - n2
      6.00

      32
      Fred has n1 green balloons , he gave Sandy n...
      X = n1 - n2
      488.00

      33
      Mary went to the mall . She spent $ n1 on a ...
      X = n1 + n2
      25.31

      34
      After paying n1 dollars for the pie , Mary h...
      X - n1 = n2
      58.00

      35
      Jason has n1 blue and 16.0 red marbles . Tom...
      X = n1 + n2
      67.00

      36
      Melanie grew n1 turnips . Benny grew n2 turn...
      X = n1 + n2
      252.00

      37
      Dan 's cat had kittens and n3 had spots . He...
      X - n1 - n2 = n3
      16.00

      38
      Mike picked n1 pears and Jason picked n2 pea...
      X = n1 + n2
      15.00

      39
      Dan has n1 green and 38.0 violet marbles . M...
      X = n1 - n2
      9.00

      40
      Jason had Pokemon cards . He gave n1 to his ...
      X - n1 = n2
      13.00

      41
      Kenji and his classmates placed colored bloc...
      X = n1 - n2
      0.20

      42
      Jason grew n1 watermelons and 18.0 turnips ....
      X = n1 + n2
      51.00

      43
      Sally had n1 Pokemon cards . Dan gave her n2...
      X = n1 + n2 + n3
      88.00

      44
      n1 the students in the band are in the trump...
      X = n1 + n2
      0.62

      45
      A restaurant served n1 hot dogs during lunch...
      X = n1 + n2
      11.00

      46
      A cell phone company has a total of n1 custo...
      X = n1 - n2
      6699.00

      47
      At a pie-eating contest , Erik got through n...
      X = n1 - n2
      0.33

      48
      There are n1 maple trees and 5.0 popular tre...
      X = n1 + n2
      11.00

      49
      A chef bought n1 kilograms of almonds and n2...
      X = n1 + n2
      0.52

      50
      Nancy grew n1 onions , Dan grew n2 onions , ...
      X = n1 + n2 + n3
      15.00

      51
      There are n1 oak trees currently in the park...
      X = n1 + n2
      9.00

      52
      There are n1 books in Oak Grove 's public li...
      X = n1 + n2
      7092.00

      53
      Benny picked n1 apples and Dan picked n2 app...
      X = n1 + n2
      11.00

      54
      There are n2 walnut trees currently in the p...
      X = n1 - n2
      33.00

      55
      Joan found n1 seashells and 12.0 starfishes ...
      n1 - X = n2
      44.00

      56
      An oil pipe in the sea broke . Before engine...
      X = n1 - n2
      3731.00

      57
      Fred has n1 yellow balloons , Sam has n2 yel...
      X = n1 + n2 + n3
      18.00

      58
      Sandy grew n1 carrots and 7.0 turnips . Mary...
      X = n1 + n2
      14.00

      59
      There are n1 rulers and 34.0 crayons in the ...
      X = n1 + n2
      25.00

      60
      Sandy 's dog had n1 puppies and n2 had spots...
      X = n1 - n2
      4.00

      61
      Professor Ellison weighed 2.0 pieces of meta...
      X = n1 - n2
      10.33

      62
      Sam has n1 blue and 25.0 green balloons . Al...
      X = n1 + n2
      37.00

      63
      Tom went to n1 hockey games this year , but ...
      X = n1 + n2
      13.00

      64
      Ron weighed 2.0 colored metal balls during a...
      X = n1 + n2
      9.12

      65
      A restaurant made n1 hamburgers and 4.0 hot ...
      X = n1 - n2
      6.00

      66
      There are n1 walnut trees currently in the p...
      X = n1 + n2
      77.00

      67
      Mary loves eating fruits . Mary paid $ n1 fo...
      X = n1 + n2 + n3
      34.72

      68
      A restaurant served n1 pies during lunch and...
      X = n1 + n2 + n3
      20.00

      69
      There are n1 pencils in the drawer and n2 pe...
      X = n1 + n2 + n3
      78.00

      70
      There were originally n1 houses in Lincoln C...
      X = n1 + n2
      118558.00

      71
      Kelly 's chemistry textbook weighs n1 pounds...
      X = n1 - n2
      6.50

      72
      Alyssa picked n1 limes and Mike picked n2 li...
      X = n1 + n2
      57.00

      73
      Mary found n1 seashells and Keith found n2 s...
      X = n1 + n2
      7.00

      74
      Joan has n1 orange balloons but lost n2 of t...
      X = n1 - n2
      6.00

      75
      There are n1 popular trees currently in the ...
      X = n1 + n2
      98.00

      76
      Sally had n1 quarters in her bank . She spen...
      X = n1 - n2
      342.00

      77
      There are n1 pencils in the drawer . Sara pl...
      X = n1 + n2
      215.00

      78
      Jessica is baking a cake . The recipe calls ...
      X = n1 - n2
      4.00

      79
      There are n1 orchid bushes currently in the ...
      X = n1 + n2
      6.00

      80
      Sandy grew n1 pumpkins . Mike grew n2 pumpki...
      X = n1 + n2
      74.00

      81
      Carefully following a recipe , Kenny used ex...
      X = n1
      1.33

      82
      Last week Fred had n1 dollars and Jason had ...
      n1 + X = n2
      63.00

      83
      Jason picked n1 pears , Keith picked n2 pear...
      X = n1 + n2 + n3
      105.00

      84
      Jason has n1 Pokemon cards . Benny bought n2...
      X = n1 - n2
      1.00

      85
      Sally grew n1 carrots . Fred grew n2 carrots...
      X = n1 + n2
      10.00

      86
      Joan joined her school 's band . She bought ...
      X = n1 + n2 + n3
      163.28

      87
      There are n1 short trees and 32.0 tall trees...
      n1 + X = n2
      64.00

      88
      Jason went to the mall on Saturday to buy cl...
      X = n1 + n2
      19.02

      89
      Fred has n1 baseball cards . Keith bought n2...
      X = n1 - n2
      18.00

      90
      Fred had n1 dimes in his bank . His sister b...
      X = n1 - n2
      4.00

      91
      There are n1 short trees and 44.0 tall trees...
      X = n1 + n2
      98.00

      92
      A tailor cut n1 inch off a skirt and n2 inch...
      X = n1 - n2
      0.25

      93
      Kelly bought n1 pounds of peanuts and n2 pou...
      X = n1 + n2
      0.50

      94
      Tom purchased a football game for $ n1 , a s...
      X = n1 + n2 + n3
      35.52

      95
      Joan purchased a basketball game for $ n1 , ...
      X = n1 + n2
      9.43

      96
      Sally paid $ n1 total for peaches , after a ...
      X = n1 + n2
      23.86

      97
      There are n1 scissors in the drawer . Keith ...
      X = n1 + n2
      76.00

      98
      Mary picked n1 oranges and Jason picked n2 o...
      X = n1 + n2
      227.00

      99
      In 1.0 week , Mitch 's family drank n1 carto...
      X = n1 + n2
      0.60

      100
      For his car , Mike spent $ n1 on speakers an...
      X = n1 + n2
      224.87

      101
      Alyssa picked n1 plums and Jason picked n2 p...
      X = n1 + n2
      27.00

      102
      Sam had 49.0 pennies and n1 nickels in his b...
      X = n1 + n2
      63.00

      103
      Sara has n1 green and 5.0 red marbles . Tom ...
      X = n1 + n2
      7.00

      104
      Tim has n1 books . Sam has n2 books . How ma...
      X = n1 + n2
      96.00

      105
      Mike had 33.0 quarters and n1 nickels in his...
      X = n1 - n2
      12.00

      106
      On her vacation last summer , Trisha walked ...
      X = n1 + n2 + n3
      0.89

      107
      Tim 's cat had kittens . He gave n1 to Jessi...
      X = n1 + n2 + n3
      18.00

      108
      During a school play , Jonah staffed the sna...
      X = n1 + n2 + n3
      0.92

      109
      Milford Lake was originally blue because it ...
      X = n1 - n2
      2454.00

      110
      Joan has n1 books . Tom has n2 books . How m...
      X = n1 + n2
      48.00

      111
      There were n2 bales of hay in the barn . Tim...
      X = n1 - n2
      26.00

      112
      There are n1 pencils and 49.0 crayons in the...
      X = n1 - n2
      12.00

      113
      Joan has n1 blue balloons Melanie has n2 blu...
      X = n1 + n2
      81.00

      114
      On a hot day , Sam poured n1 bucket of water...
      X = n1 + n2
      9.80

      115
      Keith picked n1 pears and Jason picked n2 pe...
      X = n1 + n2
      5.00

      116
      Alyssa has n1 blue balloons , Sandy has n2 b...
      X = n1 + n2 + n3
      104.00

      117
      A ship full of grain crashes into a coral re...
      X = n1 + n2
      50870.00

      118
      Mary had 21.0 dimes and n1 pennies in her ba...
      X = n1 - n2
      20.00

      119
      There are n1 erasers and 118.0 scissors in t...
      X = n1 + n2
      270.00

      120
      A construction company ordered n1 ton of con...
      X = n1 + n2 + n3
      0.83

      121
      A farmer estimates that he will harvest n1 b...
      X = n1 + n2
      48781.00

      122
      Ezra drew a white line that was n1 inches lo...
      X = n1 - n2
      4.33

      123
      Oscar 's bus ride to school is n1 mile and C...
      X = n1 - n2
      0.50

      124
      Nina did a running drill to get in shape for...
      X = n1 + n2 + n3
      0.83

      125
      There are n1 scissors and 22.0 pencils in th...
      X = n1 + n2
      52.00

      126
      Mike had n2 peaches at his roadside fruit di...
      X = n1 - n2
      52.00

      127
      At the beach , Miki and her sister both buil...
      X = n1 - n2
      0.33

      128
      Alyssa loves eating fruits . Alyssa paid $ n...
      X = n1 + n2
      21.93

      129
      Mike has n1 books in his library . He bought...
      n1 + X = n2
      21.00

      130
      Jessica grew n1 watermelons and 30.0 carrots...
      X = n1 - n2
      8.00

      131
      There are n1 pencils in the drawer . Tim pla...
      X = n1 + n2
      5.00

      132
      Sara picked n1 pears and Sally picked n2 pea...
      X = n1 + n2
      56.00

      133
      Melanie had n1 baseball cards , and 9.0 were...
      X = n1 - n2
      12.00

      134
      Mandy made an apple pie . She used n1 tables...
      X = n1 - n2
      0.17

      135
      Sam had n1 pennies in his bank . He spent n2...
      X = n1 - n2
      5.00

      136
      Allie counted the leftover ice cream after a...
      X = n1 + n2 + n3
      0.83

      137
      Jessica had n1 quarters in her bank . Her si...
      X = n1 - n2
      5.00

      138
      Last year at Newberg 's airport , n1 passeng...
      X = n1 + n2
      14720.00

      139
      Joan has n1 blue balloons but lost n2 of the...
      X = n1 - n2
      7.00

      140
      A company painted some houses in Hancock Cou...
      X = n1 - n2
      6029.00

      141
      Michelle began her pizza delivery route with...
      X = n1 - n2
      0.33

      142
      Sally had n1 peaches at her roadside fruit d...
      n1 + X = n2
      42.00

      143
      Alyssa bought some toys . She bought a footb...
      X = n1 + n2
      12.30

      144
      Dan grew n1 turnips and 38.0 cantelopes . Je...
      X = n1 + n2
      89.00

      145
      Tori was n1 feet tall . Then she grew n2 fee...
      X = n1 + n2
      7.26

      146
      Joan spent $ n1 on shorts and $ n2 on a jack...
      X = n1 + n2 + n3
      42.33

      147
      There are n1 pencils in the drawer . Nancy p...
      X = n1 + n2
      72.00

      148
      Mary had n1 baseball cards , and 8.0 were to...
      X = n1 + n2 + n3
      84.00

      149
      A restaurant served n1 cakes during lunch an...
      X = n1 + n2
      15.00

      150
      Jason found n1 seashells and 48.0 starfish o...
      X = n1 - n2
      36.00

      151
      There are n1 pencils and 40.0 scissors in th...
      X = n1 + n2
      77.00

      152
      A multi-national corporation has n1 part-tim...
      X = n1 + n2
      65134.00

      153
      There is n1 cup of oil in Scarlett 's measur...
      X = n1 + n2
      0.83

      154
      Each year , salmon travel upstream , going f...
      X = n1 + n2
      971639.00

      155
      Melanie had n1 dimes in her bank . Her dad g...
      X = n1 + n2 + n3
      83.00

      156
      Melanie had 10.0 quarters and n1 pennies in ...
      X = n1 + n2 + n3
      63.00

      157
      Sam had n1 dimes in his bank . His dad gave ...
      X = n1 + n2
      16.00

      158
      Joan bought toy cars for $ n1 , a skateboard...
      X = n1 + n2 + n3
      25.62

      159
      Sara has n1 red and 15.0 green balloons . Sa...
      X = n1 + n2
      55.00

      160
      Abe 's family moved from the Bahamas to Japa...
      X = n1 + n2
      9844.00

      161
      Eve ran n1 mile and walked n2 mile . How muc...
      X = n1 - n2
      0.10

      162
      Joan found n1 seashells and Jessica found n2...
      X = n1 + n2
      14.00

      163
      Melanie had n1 dimes in her bank . Her dad g...
      X = n1 + n2 + n3
      19.00

      164
      Joan had n1 Pokemon cards , and 6.0 were tor...
      X = n1 - n2
      562.00

      165
      Nancy went to n1 football games this month ....
      X = n1 + n2 + n3
      24.00

      166
      Keith grew n1 turnips . Alyssa grew n2 turni...
      X = n1 + n2
      15.00

      167
      In Shannon 's apartment complex , n1 the apa...
      X = n1 + n2
      0.50

      168
      There are n1 short trees and 6.0 tall trees ...
      X = n1 + n2
      12.00

      169
      Kaleen filled a bucket with n1 gallon of wat...
      X = n1 - n2
      0.25

      170
      Suzie found 2.0 worms in the yard and measur...
      X = n1 - n2
      0.70

      171
      Marta picked 2.0 pumpkins . The first pumpki...
      X = n1 + n2
      12.70

      172
      So far , an orchard has sold a combined tota...
      X = n1 - n2
      6279.00

      173
      Mary picked n1 oranges and Jason picked n2 o...
      X = n1 + n2
      55.00

      174
      Mike has n1 baseball cards . Sam bought n2 o...
      X = n1 - n2
      74.00

      175
      Sandy grew n1 carrots . Sam grew n2 carrots ...
      X = n1 + n2
      9.00

      176
      Brennan had n1 grams of pepper . Then he use...
      X = n1 - n2
      0.09

      177
      There are n1 crayons in the drawer . Mary to...
      X = n1 - n2
      4.00

      178
      Brandy made trail mix for a backpacking trip...
      X = n1 + n2 + n3
      0.42

      179
      A carpenter bought a piece of wood that was ...
      X = n1 - n2
      6.60

      180
      There are n1 gallons of water in Becky 's fi...
      X = n1 + n2
      14.75

      181
      Sara had n1 peaches and 37.0 pears at her fr...
      n1 + X = n2
      37.00

      182
      Sally found n1 seashells , Tom found n2 seas...
      X = n1 + n2 + n3
      21.00

      183
      There are n1 pencils and 44.0 crayons in the...
      X = n1 + n2
      60.00

      184
      Mary had n1 nickels in her bank . Her dad ga...
      X = n1 + n2
      12.00

      185
      Sara 's high school played n1 basketball gam...
      X = n1 - n2
      8.00

      186
      Last Saturday , Spencer walked all over town...
      X = n1 + n2 + n3
      0.80

      187
      There are n1 pencils and 7.0 rulers in the d...
      X = n1 + n2
      9.00

      188
      Jason has n1 violet balloons and 4.0 red bal...
      X = n1 - n2
      4.00

      189
      Wendy ran n1 miles and walked n2 miles . How...
      X = n1 - n2
      10.67

      190
      During a visit to an orchard , Charlie picke...
      X = n1 + n2 + n3
      0.67

      191
      Alyssa picked n1 pears and Nancy picked n2 p...
      X = n1 + n2
      59.00

      192
      Sara picked n1 pears and 27.0 apples from th...
      X = n1 - n2
      7.00

      193
      Kendall is learning to drive , so this weeke...
      X = n1 + n2
      0.67

      194
      Jason joined his school 's band . He bought ...
      X = n1 + n2 + n3
      158.35

      195
      Jason grew n1 watermelons and 30.0 pumpkins ...
      X = n1 + n2
      48.00

      196
      Karen added n1 cup of walnuts to a batch of ...
      X = n1 + n2
      0.50

      197
      There were a total of n1 football games this...
      X = n1 - n2
      4.00

      198
      A truck carrying n1 pounds of sand travels t...
      X = n1 - n2
      1.70

      199
      Mary is baking a cake . The recipe calls for...
      X = n1 - n2
      5.00

      200
      Sally has n1 orange balloons and 4.0 blue ba...
      X = n1 - n2
      7.00

      201
      Sam has n1 books . Joan has n2 books . How m...
      X = n1 + n2
      212.00

      202
      There are n1 orchid bushes currently in the ...
      X = n1 + n2 + n3
      109.00

      203
      Mike joined his school 's band . He bought a...
      X = n1 + n2
      151.00

      204
      Tom purchased a Batman game for $ n1 , and a...
      X = n1 + n2
      18.66

      205
      Fred has n1 red balloons , Sam has n2 red ba...
      X = n1 + n2 + n3
      72.00

      206
      Mike bought some toys . He bought marbles fo...
      X = n1 + n2 + n3
      20.52

      207
      Diane is a beekeeper . Last year , she harve...
      X = n1 + n2
      8564.00

      208
      There are n1 short bushes and 4.0 tall trees...
      X = n1 - n2
      4.00

      209
      Joan decided to sell all of her old books . ...
      X = n1 - n2
      7.00

      210
      Mary has n1 yellow marbles Joan has n2 yello...
      X = n1 + n2
      12.00

      211
      Alyssa 's cat had n1 kittens and n1 had spot...
      X = n1 - n2
      4.00

      212
      Tim had 7.0 quarters and n1 nickels in his b...
      X = n1 + n2
      12.00

      213
      There are n1 dogwood trees currently in the ...
      X = n1 + n2
      83.00

      214
      Joan grew n1 carrots and 14.0 watermelons . ...
      X = n1 + n2
      40.00

      215
      Fred picked n1 limes , Alyssa picked n2 lime...
      X = n1 + n2 + n3
      103.00

      216
      A construction company is repaving a damaged...
      X = n1 - n2
      4133.00

      217
      There were n1 roses in the vase . Alyssa cut...
      n1 + X = n2
      11.00

      218
      Sara had n2 quarters and n1 dimes in her ban...
      X = n1 - n2
      4.00

      219
      Before starting her shift , a waitress check...
      X = n1 + n2 + n3
      0.88

      220
      Each day , the polar bear at Richmond 's zoo...
      X = n1 + n2
      0.60

      221
      Bonnie 's science class recorded the rainfal...
      X = n1 + n2 + n3
      0.67

      222
      A restaurant served n1 cakes during lunch an...
      X = n1 + n2 + n3
      14.00

      223
      To fill an order , the factory dyed n1 yards...
      X = n1 + n2
      111421.00

      224
      Jason grew n1 watermelons and 22.0 cantelope...
      X = n1 + n2
      63.00

      225
      Keith found n1 seashells , Jessica found n2 ...
      X = n1 + n2 + n3
      21.00

      226
      A waitress put leftover tarts into the fridg...
      X = n1 + n2 + n3
      0.92

      227
      There were n1 bales of hay in the barn and 3...
      n1 + X = n2
      14.00

      228
      Keith has n1 books . Jason has n2 books . Ho...
      X = n1 + n2
      41.00

      229
      A car company produced n1 cars in North Amer...
      X = n1 + n2
      6755.00

      230
      Mike has n1 orange marbles , he gave Sam n2 ...
      X = n1 - n2
      4.00

      231
      Mary is baking a cake . The recipe wants n1 ...
      X = n1 - n2
      6.00

      232
      A renovation project required n1 truck-load ...
      X = n1 + n2 + n3
      0.67

      233
      Christina just transferred $ n1 out of her b...
      X = n1 + n2
      27004.00

      234
      Mary found n1 seashells and Jessica found n2...
      X = n1 + n2
      59.00

      235
      Kyle jogged n1 laps in P.E. class and n2 lap...
      X = n1 + n2
      3.25

      236
      Fred has n1 baseball cards . Melanie bought ...
      X = n1 - n2
      2.00

      237
      Dan spent $ n1 on a snake toy , and a cage c...
      X = n1 + n2
      26.30

      238
      Kendra made punch for her friend 's birthday...
      X = n1 + n2 + n3
      0.75

      239
      Vince 's bus ride to school is n1 mile and Z...
      X = n1 - n2
      0.12

      240
      There are n1 dogwood trees currently in the ...
      X = n1 + n2 + n3
      12.00

      241
      Dan has n1 violet marbles , he gave Mary n2 ...
      X = n1 - n2
      50.00

      242
      Keith spent $ n1 on speakers , $ n2 on a CD ...
      X = n1 + n2 + n3
      387.85

      243
      A marine biologist measured 1.0 fish that wa...
      X = n1 - n2
      0.10

      244
      A worker at a medical lab is studying blood ...
      X = n1 - n2
      3120.00

      245
      Keith spent $ n1 on a rabbit toy , $ n2 on p...
      X = n1 + n2 + n3
      24.81

      246
      A petri dish originally contained n2 bacteri...
      X = n1 - n2
      8317.00

      247
      Scarlett made a fruit salad with n1 pound of...
      X = n1 + n2
      0.62

      248
      Sara got fast food for lunch . Sara spent $ ...
      X = n1 + n2
      10.46

      249
      At a pizza party , Mason and his friends dra...
      X = n1 + n2
      5.33

      250
      Benny found n1 seashells and 49.0 starfish o...
      X = n1 - n2
      14.00

      251
      Tom has n1 yellow balloons Sara has n2 yello...
      X = n1 + n2
      17.00

      252
      There are n1 dogwood trees currently in the ...
      X = n1 + n2 + n3
      100.00

      253
      While making desserts for a bake sale , Vict...
      X = n1 - n2
      0.38

      254
      Nicole found an orange caterpillar and a gre...
      X = n1 - n2
      1.83

      255
      There are n1 scissors and 3.0 pencils in the...
      X = n1 + n2
      9.00

      256
      Terrell hiked n1 miles on Saturday . Then , ...
      X = n1 + n2
      9.80

      257
      There are n1 erasers in the drawer and n2 er...
      X = n1 + n2 + n3
      117.00

      258
      In March it rained n1 inches . It rained n2 ...
      X = n1 - n2
      0.46

      259
      Pamela bought n1 ounces of sugar , and she s...
      X = n1 - n2
      4.60

      260
      Dale 's Vegetarian Restaurant bought n1 poun...
      X = n1 + n2
      5.67

      261
      Benny received n1 dollars for his birthday ....
      n1 - X = n2
      34.00

      262
      Tim has n1 books . Mike has n2 books . How m...
      X = n1 + n2
      42.00

      263
      There were n1 bales of hay in the barn and 2...
      n1 + X = n2
      66.00

      264
      Joan found n1 seashells on the beach . she g...
      X = n1 - n2
      43.00

      265
      It snowed n1 inches on Monday and n2 inches ...
      X = n1 + n2
      0.53

      266
      Some insects called aphids attack a large fa...
      X = n1 + n2
      67082.00

      267
      Sally picked n1 lemons and Mary picked n2 le...
      X = n1 + n2
      16.00

      268
      A bucket contains n1 gallons of water . If D...
      X = n1 + n2
      9.80

      269
      Sara has n1 black and 122.0 red marbles . Fr...
      X = n1 - n2
      559.00

      270
      There are n1 short trees and 119.0 tall tree...
      X = n1 + n2
      217.00

      271
      Elizabeth went to the salon and had n1 inch ...
      X = n1 + n2
      0.88

      272
      There are n1 walnut trees currently in the p...
      X = n1 + n2
      10.00

      273
      There were n1 bales of hay in the barn . Jas...
      n1 + X = n2
      23.00

      274
      Recently , the value of Kate 's retirement f...
      X = n1 - n2
      1460.00

      275
      Joan has n1 green and 48.0 red marbles . Fre...
      X = n1 - n2
      23.00

      276
      When Jake had 1.0 cat , he needed to serve n...
      X = n1 - n2
      0.40

      277
      There are n1 oak trees currently in the park...
      X = n1 - n2
      7.00

      278
      1.0 evening , a restaurant served a total of...
      X = n1 + n2
      0.60

      279
      Heather went to the county fair last weekend...
      X = n1 +  n2 + n3
      0.75

      280
      Tom found n1 seashells on the beach . he gav...
      X = n1 - n2
      3.00

      281
      Tom found n1 seashells but n2 were broken . ...
      X = n1 - n2
      3.00

      282
      Keith grew n1 cantelopes , Fred grew n2 cant...
      X = n1 + n2 + n3
      65.00

      283
      Sandy has n1 books , Benny has n2 books , an...
      X = n1 + n2 + n3
      67.00

      284
      Joan had n1 dimes in her bank . She spent n2...
      X = n1 - n2
      3.00

      285
      There are n1 dogwood trees currently in the ...
      X = n1 + n2 + n3
      16.00

      286
      Melanie picked n1 plums , Dan picked n2 plum...
      X = n1 + n2 + n3
      16.00

      287
      Joan grew n1 pumpkins , Keith grew n2 pumpki...
      X = n1 + n2 + n3
      79.00

      288
      Fred grew n1 cantelopes . Tim grew n2 cantel...
      X = n1 + n2
      82.00

      289
      Sara grew n1 onions , Sally grew n2 onions ,...
      X = n1 + n2 + n3
      18.00

      290
      Joan picked n1 oranges and Sara picked n2 or...
      X = n1 + n2
      47.00

      291
      There are n1 short bushes and 30.0 tall tree...
      X = n1 + n2
      57.00

      292
      Hoping to be named Salesperson of the Month ...
      X = n1 + n2
      18.80

      293
      Melanie has n1 books and 31.0 magazines in h...
      n1 + X = n2
      46.00

      294
      A bee colony produced n1 pounds of honey , b...
      X = n1 - n2
      0.31

      295
      Jessica spent $ n1 on a cat toy , and a cage...
      X = n1 + n2
      21.95

      296
      Sally had 8.0 pennies and n1 nickels in her ...
      X = n1 + n2 + n3
      18.00

      297
      Sara had n1 quarters in her bank . Her dad g...
      X = n1 + n2
      70.00

      298
      Dan picked n1 limes and gave Sara n2 of the ...
      X = n1 - n2
      5.00

      299
      Tom bought a skateboard for $ n1 , and spent...
      X = n1 + n2
      19.02

      300
      During a canned food drive , items were sort...
      X = n1 + n2 + n3
      0.75

      301
      A dust storm sweeps across the prairie . It ...
      X = n1 - n2
      64013.00

      302
      Ella owns 2.0 dogs . Each day , 1.0 dog eats...
      X = n1 + n2
      0.25

      303
      Tom had 27.0 pennies and n1 dimes in his ban...
      X = n1 + n2
      48.00

      304
      A bathing suit manufacturer has a supply of ...
      X = n1 + n2
      19766.00

      305
      There are n1 pencils in the drawer . Mike pl...
      X = n1 + n2
      71.00

      306
      Before the recent housing boom , there were ...
      X = n1 - n2
      574.00

      307
      Kevin bought 2.0 watermelons . The first wat...
      X = n1 + n2
      14.02

      308
      There were n2 roses in the vase . Mary cut s...
      X = n1 - n2
      10.00

      309
      The Richmond Tigers sold a total of n1 ticke...
      X = n1 - n2
      5703.00

      310
      Sam went to n1 football games this year . He...
      X = n1 + n2
      43.00

      311
      A treasure hunter discovered a buried treasu...
      X = n1 - n2
      5110.00

      312
      Each of farmer Cunningham 's n1 lambs is eit...
      X = n1 - n2
      5855.00

      313
      Dina made cookies . She used n1 cup of flour...
      X = n1 - n2
      0.38

      314
      While making pastries , a bakery used n1 bag...
      X = n1 + n2
      0.30

      315
      Sam has n1 yellow and 20.0 green marbles . J...
      X = n1 - n2
      61.00

      316
      A ship is filled with n1 tons of cargo . It ...
      X = n1 + n2
      14696.00

      317
      While playing a video game , Paul scored n2 ...
      X = n1 - n2
      2713.00

      318
      Nancy grew n1 potatoes . Sandy grew n2 potat...
      X = n1 + n2
      13.00

      319
      It rained n1 inches on Saturday and n2 inche...
      X = n1 + n2
      0.60

      320
      There were n1 red orchids and 4.0 white orch...
      n1 + X = n2
      16.00

      321
      Mike found n1 seashells and n2 starfish , bu...
      X = n1 - n2
      2.00

      322
      Jenny ran n1 mile and walked n2 mile . How m...
      X = n1 - n2
      0.20

      323
      Melanie bought a Batman game for $ n1 , a st...
      X = n1 + n2 + n3
      22.58

      324
      There are n1 orchid bushes and 40.0 orange t...
      X = n1 + n2
      35.00

      325
      Logan recorded the snowfall every day during...
      X = n1 + n2 + n3
      0.89

      326
      There are n1 maple trees and 22.0 orange tre...
      n1 + X = n2
      11.00

      327
      Joan went to n1 football games this year . S...
      X = n1 + n2
      13.00

      328
      Dan found n1 seashells on the beach , he gav...
      n1 - X = n2
      34.00

      329
      Dan had n1 baseball cards , and 8.0 were tor...
      X = n1 - n2
      82.00

      330
      Mike picked n1 oranges and Melanie picked n2...
      X = n1 + n2
      227.00

      331
      Sam grew n1 watermelons , but the rabbits at...
      X = n1 - n2
      1.00

      332
      Joan grew n1 watermelons and 4.0 turnips . T...
      X = n1 + n2
      17.00

      333
      Roadster 's Paving Company used n1 tons of c...
      X = n1 + n2
      15.10

      334
      At Lindsey 's Vacation Wear , n1 the garment...
      X = n1 + n2
      0.62

      335
      Sally grew n1 turnips and 118.0 pumpkins . M...
      X = n1 + n2
      242.00

      336
      Mary found n1 seashells , Melanie found n2 s...
      X = n1 + n2 + n3
      100.00

      337
      Last week Tom had n1 dollars . He washed car...
      n1 + X = n2
      12.00

      338
      Mary had n1 Pokemon cards , and 6.0 were tor...
      X = n1 + n2
      56.00

      339
      The Silvergrove Public Library used a grant ...
      X = n1 - n2
      5935.00

      340
      Blake filled a bucket with n1 gallon of wate...
      X = n1 - n2
      0.60

      341
      There are n1 crayons in the drawer . Benny p...
      X = n1 + n2
      12.00

      342
      Hannah 's Vegetarian Restaurant bought n1 po...
      X = n1 + n2
      0.67

      343
      Irene just bought a new lamp for her bedside...
      X = n1 - n2
      1.33

      344
      Sam found n1 seashells and Mary found n2 sea...
      X = n1 + n2
      65.00

      345
      Last year , egg producers in Douglas County ...
      X = n1 - n2
      3220.00

      346
      Fred went to n1 basketball games this year ,...
      X = n1 + n2
      47.00

      347
      In Mr. Olsen 's mathematics class , n1 the s...
      X = n1 + n2
      0.90

      348
      As part of a lesson on earthquakes , a scien...
      X = n1 + n2
      6.50

      349
      Fred found n1 seashells on the beach , he ga...
      X = n1 - n2
      22.00

      350
      The Montoya family spends n1 their budget on...
      X = n1 + n2
      0.80

      351
      Joan has n1 blue balloons , Sally has n2 blu...
      X = n1 + n2 + n3
      16.00

      352
      Dan had n1 potatoes and n2 cantelopes in the...
      X = n1 - n2
      3.00

      353
      Joan 's cat had n1 kittens . She gave n2 to ...
      X = n1 - n2
      6.00

      354
      Sara had 100.0 pennies and n1 quarters in he...
      X = n1 - n2
      512.00

      355
      Alec and his roommates ate n1 pints of ice c...
      X = n1 + n2
      3.50

      356
      At the hardware store , n1 the nails are siz...
      X = n1 + n2
      0.75

      357
      Shannon and her family use up a lot of straw...
      X = n1 - n2
      1792.00

      358
      Jason had n1 quarters in his bank . His dad ...
      X = n1 + n2
      74.00

      359
      Tim found n1 seashells and Sally found n2 se...
      X = n1 + n2
      50.00

      360
      Joan found n1 seashells and 14.0 starfishes ...
      n1 - X = n2
      13.00

      361
      Sandy went to the mall to buy clothes . She ...
      X = n1 + n2 + n3
      33.56

      362
      Tori is a school janitor . Last week , she p...
      X = n1 - n2
      1232.00

      363
      Darnel sprinted n1 lap and then took a break...
      X = n1 - n2
      0.12

      364
      Nancy found n1 seashells and 25.0 starfish o...
      X = n1 - n2
      18.00

      365
      There are n1 crayons in the drawer and n2 cr...
      X = n1 + n2 + n3
      17.00

      366
      There are n1 crayons and 26.0 pencils in the...
      X = n1 + n2
      53.00

      367
      Dan joined his school 's band . He bought a ...
      X = n1 + n2
      141.54

      368
      A spaceship traveled n1 light-year from Eart...
      X = n1 + n2 + n3
      0.70

      369
      Last week Tim had n1 dollars and Keith had 3...
      n1 + X = n2
      63.00

      370
      There are n1 rulers in the drawer . Tim took...
      X = n1 - n2
      21.00

      371
      Joan 's high school played n1 baseball games...
      X = n1 - n2
      469.00

      372
      At the beach , Janet and her sister both bui...
      X = n1 - n2
      1.33

      373
      Tom has n1 violet balloons , he gave Fred n2...
      X = n1 - n2
      14.00

      374
      Joan found n1 seashells on the beach , she g...
      X = n1 - n2
      16.00

      375
      There were n1 red orchids and 3.0 white orch...
      n1 + X = n2
      6.00

      376
      Last year , n1 people were born in a country...
      X = n1 + n2
      106491.00

      377
      While taking inventory at her pastry shop , ...
      X = n1 - n2
      0.10

      378
      Tom found n1 seashells and Fred found n2 sea...
      X = n1 + n2
      58.00

      379
      Irwin 's family went on a camping trip in th...
      X = n1 + n2 + n3
      0.70

      380
      Tim found n1 seashells and 110.0 starfish on...
      X = n1 - n2
      507.00

      381
      Before December , customers buy n1 ear muffs...
      X = n1 + n2
      7790.00

      382
      Mike picked n1 apples , Nancy picked n2 appl...
      X = n1 + n2 + n3
      16.00

      383
      Stanley ran n1 mile and walked n2 mile . How...
      X = n1 - n2
      0.20

      384
      Mary had n1 potatoes in the garden . The rab...
      X = n1 - n2
      5.00

      385
      Craig walked n1 mile from school to David 's...
      X = n1 + n2
      0.90

      386
      In Yardley it snowed n1 inch in the morning ...
      X = n1 + n2
      0.62

      387
      A restaurant served n1 slices of pie during ...
      X = n1 + n2
      12.00

      388
      Mary got fast food for lunch . Mary spent $ ...
      X = n1 + n2
      5.88

      389
      Jonah added n1 cup of yellow raisins and n2 ...
      X = n1 + n2
      0.70

      390
      A restaurant served n1 pizzas during lunch a...
      X = n1 + n2
      15.00

      391
      Sam 's dog had puppies and 8.0 had spots . H...
      X - n1 = n2
      8.00

      392
      Sara picked n1 pears and Tim picked n2 pears...
      X = n1 + n2
      11.00

      393
      Ellen made smoothies in the blender . She us...
      X = n1 + n2 + n3
      0.50

      394
      This afternoon Craig left school , rode the ...
      X = n1 - n2
      3.67

## DATASET ANALYSIS

```python
#Understanding the dataset through visualization

plt.figure(1, figsize=(15,5))
plt.title('Histogram of the length of the questions')
plt.xlabel('# of words in a problem')
plt.ylabel('Frequency (#)')
math_dataset['Question'].str.split().apply(len).value_counts().sort_index().plot(kind='bar', color='#7f8fa1')
plt.grid(color='black', linestyle='-', linewidth=0.1, axis='y')
```

## Vocabulary

```python
SOS = 0 #Start Of Sentence
EOS = 1 #End Of Sentence

class Lang:
    def __init__(self, name):
        self.name = name
        self.word2index = {} #Para pasar de palabra al indice donde se encuentra la palabra en el vocabulario
        self.word2count = {} #La cantidad de veces que aparece esa palabra
        self.index2word = {0: "SOS", 1: "EOS"} #Aqui se asigna a un indice una palabra única
        self.n_words = 2  #Número de palabras en el vocabulario. Se incluye SOS y EOS.

    def addSentence(self, sentence):
        for word in sentence.split(' '):
            self.addWord(word)

    def addWord(self, word):
        if word not in self.word2index:
            self.word2index[word] = self.n_words
            self.word2count[word] = 1
            self.index2word[self.n_words] = word
            self.n_words += 1
        else:
            self.word2count[word] += 1
```

```python
math_dataset = load_data()

#input_lang = Lang('text')
#target_lang = Lang('equations')
input_tensor = ['<start> ' + problem + ' <end>' for problem in math_dataset['Question']]
target_tensor = ['<start> ' + equation + ' <end>' for equation in math_dataset['Equation']]

#for inp, targ in zip(input_tensor, target_tensor):
#    input_lang.addSentence(inp)
#   target_lang.addSentence(targ)

#print(input_lang.n_words)
#print(target_lang.n_words)

```

    C:\Users\Hermanos\Anaconda3\envs\tf-gpu-cuda8\lib\site-packages\ipykernel_launcher.py:67: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
    C:\Users\Hermanos\Anaconda3\envs\tf-gpu-cuda8\lib\site-packages\ipykernel_launcher.py:84: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

```python
def tokenize(lang):
    # Creating a tokenizer
    lang_tokenizer = tf.keras.preprocessing.text.Tokenizer(
      filters='')

    # Building word indices
    lang_tokenizer.fit_on_texts(lang)

    # Tokenizing sentences
    tensor = lang_tokenizer.texts_to_sequences(lang)

    tensor = tf.keras.preprocessing.sequence.pad_sequences(tensor,
                                                         padding='post')

    # Creating a reverse dictionary
    index2word = dict(map(reversed, lang_tokenizer.word_index.items()))

    return tensor, lang_tokenizer, index2word

# Function takes a tokenized sentence and returns the words
def seq2text(list_of_indices, index2word):
    # Looking up words in dictionary
    text = [index2word.get(word) for word in list_of_indices if index2word.get(word) != None]
    return text

input_tensor, inp_lang, inp_lang_index2word = tokenize(input_tensor)
target_tensor, targ_lang, targ_lang_index2word = tokenize(target_tensor)

input_tensor_train, input_tensor_test, target_tensor_train, target_tensor_test = data_splitter(input_tensor, target_tensor, percentageToTrain=0.9)

BUFFER_SIZE = len(input_tensor_train)
vocab_inp_size = len(inp_lang.word_index)+1
vocab_tar_size = len(targ_lang.word_index)+1
```

## RNN Model

Usaremos un RecurrentNeuralNetwork-based seq2seq model. Con un encoder (Gated Recurrent Units) y decoder (Long Short-Term Memory)

-GRU:

    1) Update gate
    2) Reset gate
    3) New memory
    4) Hidden state

-LSTM:

    1) Input gate
    2) Forget gate
    3) Output gate
    4) New memory
    5) Final memory
    6) Hidden state

```python
class GRUEncoder(tf.keras.Model):
    def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz, dropout):
        super(GRUEncoder, self).__init__()
        self.batch_sz = batch_sz
        self.enc_units = enc_units
        self.embedding = Embedding(vocab_size, embedding_dim)
        self.gru = GRU(self.enc_units, return_sequences=True, return_state=True, recurrent_initializer='glorot_uniform')

        #self.dropout = Dropout(dropout)

    def call(self, x, hidden):
        #print('x preembed:',x)
        x = self.embedding(x)
        #print('x posembed:',x)
        output, state = self.gru(x, initial_state = hidden)

        #output = self.dropout(output)

        return output, state

    def initialize_hidden_state(self):
        return tf.zeros((self.batch_sz, self.enc_units))

class BahdanauAttention(tf.keras.layers.Layer):
    def __init__(self, units):
        super(BahdanauAttention, self).__init__()
        self.W1 = tf.keras.layers.Dense(units)
        self.W2 = tf.keras.layers.Dense(units)
        self.V = tf.keras.layers.Dense(1)

    def call(self, query, values):
        # query hidden state shape == (batch_size, hidden size)
        # query_with_time_axis shape == (batch_size, 1, hidden size)
        # values shape == (batch_size, max_len, hidden size)
        # we are doing this to broadcast addition along the time axis to calculate the score
        query_with_time_axis = tf.expand_dims(query, 1)

        # score shape == (batch_size, max_length, 1)
        # we get 1 at the last axis because we are applying score to self.V
        # the shape of the tensor before applying self.V is (batch_size, max_length, units)
        score = self.V(tf.nn.tanh(
            self.W1(query_with_time_axis) + self.W2(values)))

        # attention_weights shape == (batch_size, max_length, 1)
        attention_weights = tf.nn.softmax(score, axis=1)

        # context_vector shape after sum == (batch_size, hidden_size)
        context_vector = attention_weights * values
        context_vector = tf.reduce_sum(context_vector, axis=1)

        return context_vector, attention_weights

class GRUDecoder(tf.keras.Model):
    def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz, dropout):
        super(GRUDecoder, self).__init__()
        self.batch_sz = batch_sz
        self.dec_units = dec_units
        self.embedding = Embedding(vocab_size, embedding_dim)
        self.gru = GRU(self.dec_units, return_sequences=True, return_state=True, recurrent_initializer='glorot_uniform', dropout=0.5)
        #self.dropout = Dropout(dropout)
        self.fc = Dense(vocab_size)

        self.attention = BahdanauAttention(self.dec_units)

    def call(self, x, hidden, enc_output):
        context_vector, attention_weights = self.attention(hidden, enc_output)

        x = self.embedding(x)

        x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1) #a ver

        output, state = self.gru(x)

        output = tf.reshape(output, (-1, output.shape[2])) #no se

        #output = self.dropout(output)

        output = self.fc(output)

        return output, state

class LSTMDecoder(tf.keras.Model):
    def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz, dropout):
        super(LSTMDecoder, self).__init__()
        self.batch_sz = batch_sz
        self.dec_units = dec_units
        self.embedding = Embedding(vocab_size, embedding_dim)
        self.lstm = LSTM(self.dec_units, return_sequences=True, return_state=True, recurrent_initializer='glorot_uniform')
        #self.dropout = Dropout(dropout)
        self.fc = Dense(vocab_size)

        self.attention = BahdanauAttention(self.dec_units)

    def call(self, x, hidden, enc_output):
        context_vector, attention_weights = self.attention(hidden, enc_output)
        x = self.embedding(x)

        x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1) #a ver

        output, state, _ = self.lstm(x)

        output = tf.reshape(output, (-1, output.shape[2])) #no se

        #output = self.dropout(output)

        output = self.fc(output)

        return output, state

def setOptimizer(lr):
    optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
    loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True, reduction='none')
    return optimizer, loss_object

def loss_function(real, pred):
    mask = tf.math.logical_not(tf.math.equal(real, 0))
    loss_ = loss_object(real, pred)

    mask = tf.cast(mask, dtype=loss_.dtype)
    loss_ *= mask

    return tf.reduce_mean(loss_)

```

## Training and Validation

```python
def _train(inp, targ, enc_hidden):
    loss = 0

    with tf.GradientTape() as tape:
        enc_output, enc_hidden = encoder(inp, enc_hidden)

        dec_hidden = enc_hidden

        dec_input = tf.expand_dims([targ_lang.word_index['<start>']] * BATCH_SIZE, 1)

        # Teacher forcing - feeding the target as the next input
        for t in range(1, targ.shape[1]):
            # passing enc_output to the decoder
            predictions, dec_hidden = decoder(dec_input, dec_hidden, enc_output)

            loss += loss_function(targ[:, t], predictions)

            # using teacher forcing
            dec_input = tf.expand_dims(targ[:, t], 1)

    batch_loss = (loss / int(targ.shape[1]))

    variables = encoder.trainable_variables + decoder.trainable_variables

    gradients = tape.gradient(loss, variables)

    optimizer.apply_gradients(zip(gradients, variables))

    return batch_loss

train = tf.function(_train)

def val(sentence):

    inputs = [inp_lang.word_index[i] if i in inp_lang.word_index else inp_lang.word_index['what'] for i in sentence]##################################
    inputs = tf.keras.preprocessing.sequence.pad_sequences([inputs],
                                                     maxlen=input_tensor_test.shape[1],
                                                     padding='post')
    inputs = tf.convert_to_tensor(inputs)
    hidden = [tf.zeros((1, units))]
    enc_out, enc_hidden = encoder(inputs, hidden)

    dec_hidden = enc_hidden
    dec_input = tf.expand_dims([targ_lang.word_index['<start>']], 0)

    result = []
    for t in range(target_tensor_test.shape[1]):
        predictions, dec_hidden = decoder(dec_input, dec_hidden, enc_out)

        predicted_id = tf.argmax(predictions[0]).numpy()

        if predicted_id != None:
            result.append(predicted_id)

        if targ_lang.index_word[predicted_id] == '<end>':
            return result, sentence

        # the predicted ID is fed back into the model
        dec_input = tf.expand_dims([predicted_id], 0)

    return result, sentence

def tryTheModel(units, learning_rate, BATCH_SIZE, EPOCHS, embedding_dim):

    dataset = tf.data.Dataset.from_tensor_slices((input_tensor_train, target_tensor_train))#.shuffle(BUFFER_SIZE)
    dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)

    checkpoint_dir = './training_checkpoints'
    checkpoint_prefix = os.path.join(checkpoint_dir, "ckpt")
    checkpoint = tf.train.Checkpoint(optimizer=optimizer, encoder=encoder, decoder=decoder)

    ### TRAINING ###
    start = time.time()
    model_loss_train=[]
    for epoch in range(EPOCHS):

        enc_hidden = encoder.initialize_hidden_state()
        total_loss = 0
        for (batch, (inp, targ)) in enumerate(dataset.take(BATCH_SIZE)):
            try:
                batch_loss = train(inp, targ, enc_hidden)
            except (UnboundLocalError, ValueError):
                train = tf.function(_train)
                batch_loss = train(inp, targ, enc_hidden)

            total_loss += batch_loss

            #print('Training Epoch {} Batch {} Loss {:.4f}'.format(epoch + 1, batch, batch_loss.numpy()))

        model_loss_train.append(total_loss / (len(input_tensor_train)/BATCH_SIZE))
        # saving (checkpoint) the model
        checkpoint.save(file_prefix = checkpoint_prefix)

        print('Training Epoch {} Loss {:.8f}'.format(epoch + 1,
                                          total_loss / (len(input_tensor_train)/BATCH_SIZE)))

    plt.figure()
    plt.title("loss")
    plt.plot(model_loss_train, 'b-')
    plt.xlabel('epochs')
    plt.ylabel('loss')
    minTrainLoss = min(model_loss_train)
    print("Min training loss : ", minTrainLoss)
    print('Neural Units:', units)
    print('Learning rate:', learning_rate)
    print('Batch size:', BATCH_SIZE)
    print('Epochs:', EPOCHS)
    print('Embedding dim:', embedding_dim)
    #print('Momentum:', momentum)
    #print('Weight decay:', weight_decay)
    #print('Dropout:', dropout)
    #print("Nesterov:",l_nesterov)
    print('Time taken for training: {} sec\n'.format(time.time() - start))
    plt.show()

    ### VALIDATION ###
    checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))
    accuracy=0
    for sequence, equation in zip(input_tensor_test, target_tensor_test):
        sentence = seq2text(sequence, inp_lang_index2word)
        prediction, sentence = val(sentence)

        sentence = ' '.join(sentence[1:-1])
        equation = ''.join(seq2text(equation, targ_lang_index2word)[1:-1])
        prediction = ''.join(seq2text(prediction, targ_lang_index2word)[:-1])

        print('Sentence: ', sentence)
        print('Equation: ', equation)
        print('Prediction: ', prediction)
        print('')

        if prediction == equation:
            accuracy += 1

    accuracy = accuracy / len(target_tensor_test)
    print('Validation Accuracy = {:.8f}'.format(accuracy))

    return checkpoint

```

## Grid Search for Hyperparameter Tuning

```python
#HyperParemeters
units = 256#[32, 64, 128, 256, 512, 1024, 2048]
learning_rate = 0.01 #[0.01, 0.03, 0.05, 0.07]
BATCH_SIZE = 4 #[1, 2, 3, 4]
EPOCHS = 100
embedding_dim = 2048 #[512, 1024, 2048, 4096, 8192, 16384]
dropout = 1
#min train loss 0.00000097 con units256, lr0.01, bs4, ed2048, epochs100, dropoutEnDec0.5
best = [[256], [0.01], [4]]

for (units, learning_rate, BATCH_SIZE) in list(itertools.product(*best)):
    encoder = GRUEncoder(vocab_inp_size, embedding_dim, units, BATCH_SIZE, dropout)
    decoder = GRUDecoder(vocab_tar_size, embedding_dim, units, BATCH_SIZE, dropout)
    optimizer, loss_object = setOptimizer(learning_rate)

    checkpoint = tryTheModel(units, learning_rate, BATCH_SIZE, EPOCHS, embedding_dim)
```

## Manual testing

```python
sentence = 'sara had 100.0 pennies and n1 quarters in her bank . her dad borrowed n2 quarters from sara . how many quarters does she have now ?'
#sentence = 'what is 100 plus 100'
checkpoint.restore(tf.train.latest_checkpoint('./training_checkpoints'))
result, sentence = val(sentence.split(' '))
print(' '.join(sentence))
print('Equation predicted: ', ''.join(seq2text(result, targ_lang_index2word)[:-1]))
```

    sara had 100.0 pennies and n1 quarters in her bank . her dad borrowed n2 quarters from sara . how many quarters does she have now ?
    Equation predicted:  x=n1-n2

```python

```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2019 Daniel Koehn based on (c)2014 L.A. Barba, G.F. Forsyth, C. Cooper [CFDPython](https://github.com/barbagroup/CFDPython), (c)2013 L.A. Barba, also under CC-BY license.

```python
from IPython.core.display import HTML
css_file = '../style/custom.css'
HTML(open(css_file, 'r').read())
```

<link href="https://fonts.googleapis.com/css?family=Merriweather:300,300i,400,400i,700,700i,900,900i" rel='stylesheet' >
<link href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:300,300i,400,400i,700,700i" rel='stylesheet' >
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' >
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 800px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.5em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Source Sans Pro', sans-serif;
    line-height: 140%;
    font-size: 110%;
    width:680px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Merriweather', serif;
    font-style:regular;
    font-weight: bold;
    font-size: 250%;
    line-height: 100%;
    color: #004065;
    margin-bottom: 1em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Merriweather', serif;
    font-weight: bold;
    font-size: 180%;
    line-height: 100%;
    color: #0096d6;
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Merriweather', serif;
	font-size: 150%;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: regular;
    color: #008367;
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Merriweather', serif;
    font-weight: 300;
    font-size: 100%;
    line-height: 120%;
    text-align: left;
    width:500px;
    margin-top: 1em;
    margin-bottom: 2em;
    margin-left: 80pt;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Source Sans Pro', sans-serif;
    font-weight: regular;
    font-size: 130%;
    color: #e31937;
    font-style: italic;
    margin-bottom: .5em;
    margin-top: 1em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'Source Code Pro', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

# 1D Linear Advection Problem: Stability and CFL condition

In the first lesson of this lecture, we studied the numerical solution of the linear and non-linear advection equations, using the finite-difference method. Did you experiment there using different parameter choices? If you did, you probably ran into some unexpected behavior. Did your solution ever blow up (sometimes in a cool way!)?

In this Jupyter Notebook, we will explore why changing the discretization parameters can affect your solution in such a drastic way.

With the solution parameters we initially suggested, the spatial grid had 41 points and the time-step size was 0.25.  Now, we're going to experiment with the number of points in the grid.  The code below corresponds to the linear advection case, but written into a function so that we can easily examine what happens as we adjust just one variable: **the grid size**.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
```

```python
# Set the font family and size to use for Matplotlib figures.
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
```

```python
def linear_advection(nx, L=2.0, c=1.0, dt=0.025, nt=20):
    """
    Solves the 1D linear convection equation
    with constant speed c in the domain [0, L]
    and plots the solution (along with the initial conditions).

    Parameters
    ----------
    nx : integer
        Number of grid points to discretize the domain.
    L : float, optional
        Length of the domain; default: 2.0.
    c : float, optional
        advection speed; default: 1.0.
    dt : float, optional
        Time-step size; default: 0.025.
    nt : integer, optional
        Number of time steps to compute; default: 20.
    """

    # Discretize spatial grid.
    dx = L / (nx - 1)
    x = numpy.linspace(0.0, L, num=nx)

    # Set initial conditions.
    u0 = numpy.ones(nx)
    mask = numpy.where(numpy.logical_and(x >= 0.5, x <= 1.0))
    u0[mask] = 2.0

    # Integrate the solution in time.
    u = u0.copy()
    for n in range(1, nt):
        #u[1:] = u[1:] - c * dt / dx * (u[1:] - u[:-1])
        un = u.copy()
        for i in range(1, nx):
            u[i] = un[i] - c * dt / dx * (un[i] - un[i - 1])

    # Plot the solution along with the initial conditions.
    pyplot.figure(figsize=(4.0, 4.0))
    pyplot.xlabel('x')
    pyplot.ylabel('u')
    pyplot.grid()
    pyplot.plot(x, u0, label='Initial',
                color='C0', linestyle='--', linewidth=2)
    pyplot.plot(x, u, label='nt = {}'.format(nt),
                color='C1', linestyle='-', linewidth=2)
    pyplot.legend()
    pyplot.xlim(0.0, L)
    pyplot.ylim(0.0, 2.5);
```

Now let's examine the results of the linear advection problem with an increasingly fine mesh.  We'll try 41, 61 and 71 points ... then we'll shoot for 85. See what happens:

```python
linear_advection(41)  # solve using 41 spatial grid points
```

```python
linear_advection(61)
```

```python
linear_advection(71)
```

So far so good—as we refine the spatial grid, the wave is more square, indicating that the discretization error is getting smaller. But what happens when we refine the grid even further? Let's try 85 grid points.

```python
linear_advection(85)
```

Oops. This doesn't look anything like our original hat function. Something has gone awry. It's the same code that we ran each time, so it's not a bug!

### What happened?

To answer that question, we have to think a little bit about what we're actually implementing in the code when we solve the linear convection equation with the forward-time/backward-space method.

In each iteration of the time loop, we use the existing data about the solution at time $n$ to compute the solution in the subsequent time step, $n+1$.  In the first few cases, the increase in the number of grid points returned more accurate results.  There was less discretization error and the translating wave looked more like a square wave than it did in our first example.

Each iteration of the time loop advances the solution by a time-step of length $\Delta t$, which had the value 0.025 in the examples above. During this iteration, we evaluate the solution $u$ at each of the $x_i$ points on the grid.  But in the last plot, something has clearly gone wrong.

What has happened is that over the time period $\Delta t$, the wave is travelling a distance which is greater than `dx`, and we say that the solution becomes *unstable* in this situation (this statement can be proven formally, see below).  The length `dx` of grid spacing is inversely proportional to the number of total points `nx`: we asked for more grid points, so `dx` got smaller. Once `dx` got smaller than the $c\Delta t$—the distance travelled by the numerical solution in one time step—it's no longer possible for the numerical scheme to solve the equation correctly!

#### Graphical interpretation of the CFL condition.

Consider the illustration above. The green triangle represents the _domain of dependence_ of the numerical scheme. Indeed, for each time step, the variable $u_i^{n+1}$ only depends on the values $u_i^{n}$ and $u_{i-1}^{n}$.

When the distance $c\Delta t$ is smaller than $\Delta x$, the characteristic line traced from the grid coordinate $i, n+1$ lands _between_ the points $i-1,n$ and $i,n$ on the grid. We then say that the _mathematical domain of dependence_ of the solution of the original PDE is contained in the _domain of dependence_ of the numerical scheme.

On the contrary, if $\Delta x$ is smaller than $c\Delta t$, then the information about the solution needed for $u_i^{n+1}$ is not available in the _domain of dependence_ of the numerical scheme, because the characteristic line traced from the grid coordinate $i, n+1$ lands _behind_ the point $i-1,n$ on the grid.

The following condition thus ensures that the domain of dependence of the differential equation is contained in the _numerical_ domain of dependence:

$$
\begin{equation}
\sigma = \frac{c \Delta t}{\Delta x} \leq 1
\end{equation}
$$

As can be proven formally, for example using the [von Neumann analysis](https://de.wikipedia.org/wiki/Von-Neumann-Stabilit%C3%A4tsanalyse), stability of the numerical solution requires that step size `dt` is calculated with respect to the size of `dx` to satisfy the condition above.

The value of $c\Delta t/\Delta x$ is called the [**Courant-Friedrichs-Lewy number**](https://gdz.sub.uni-goettingen.de/id/PPN235181684_0100?tify={%22pages%22:[36],%22panX%22:0.519,%22panY%22:0.646,%22view%22:%22info%22,%22zoom%22:0.44}) (CFL number), often denoted by $\sigma$. The value $\sigma_{\text{max}}$ that will ensure stability depends on the discretization used; for the forward-time/backward-space scheme, the condition for stability is $\sigma<1$.

In a new version of our code—written _defensively_—, we'll use the CFL number to calculate the appropriate time-step `dt` depending on the size of `dx`. Furthermore, we also define a maximum modelling time `Tmax`

```python
def linear_advection_cfl(nx, Tmax, L=2.0, c=1.0, sigma=0.5):
    """
    Solves the 1D linear advection equation
    with constant speed c in the domain [0, L]
    and plots the solution (along with the initial conditions).
    Here, the time-step size is calculated based on a CFL constraint.

    Parameters
    ----------
    nx : integer
        Number of grid points to discretize the domain.
    Tmax : float
        Maximum integration time
    L : float, optional
        Length of the domain; default: 2.0.
    c : float, optional
        Advection speed; default: 1.0.
    sigma : float, optional
        CFL constraint; default: 0.5.
    """

    # Discretize spatial grid.
    dx = L / (nx - 1)
    x = numpy.linspace(0.0, L, num=nx)

    # Compute the time-step size based on the CFL constraint.
    dt = sigma * dx / c

    # Compute number of time steps based on Tmax and dt
    nt = (int)(Tmax/dt)

    # Set initial conditions.
    u0 = numpy.ones(nx)
    mask = numpy.where(numpy.logical_and(x >= 0.5, x <= 1.0))
    u0[mask] = 2.0

    # Integrate the solution in time.
    u = u0.copy()
    for n in range(1, nt):
        un = u.copy()
        for i in range(1, nx):
            u[i] = un[i] - c * dt / dx * (un[i] - un[i - 1])

    # Plot the solution along with the initial conditions.
    pyplot.figure(figsize=(4.0, 4.0))
    pyplot.xlabel('x')
    pyplot.ylabel('u')
    pyplot.grid()
    pyplot.plot(x, u0, label='Initial',
                color='C0', linestyle='--', linewidth=2)
    pyplot.plot(x, u, label='nt = {}'.format(nt),
                color='C1', linestyle='-', linewidth=2)
    pyplot.legend()
    pyplot.xlim(0.0, L)
    pyplot.ylim(0.0, 2.5);
```

Now, it doesn't matter how many points we use for the spatial grid: the solution will always be stable!

```python
# Define maximum time Tmax
Tmax = 0.5
```

```python
linear_advection_cfl(85,Tmax)
```

```python
linear_advection_cfl(441,Tmax)
```

```python
linear_advection_cfl(2000,Tmax)
```

Notice that as the number of points `nx` increases, the solution of the linear advection problem becomes more accurate at the cost that more time steps `nt` are required resulting in a longer computation time.

## What we learned:

- The numerical solution of 1D linear advection equation becomes unstable, if we choose a time step $\Delta t$ violating the Courant-Friedrichs-Lewy (CFL) criterion:

\begin{equation}
\Delta t \le \frac{\sigma \Delta x}{c} \notag
\end{equation}

depending on the spatial gridpoint distance $\Delta x$, advection velocity $c$ and a factor $\sigma$

- The factor $\sigma$ depends on the used FD-scheme used to solve the 1D linear advection problem or any PDE in general


## Data Analytics

### Distribution Transformations in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### Data Analytics: Distribution Transformations

Here's a demonstration of making and general use of distribution transformations in Python. This demonstration is part of the resources that I include for my courses in Spatial / Subsurface Data Analytics at the Cockrell School of Engineering at the University of Texas at Austin.

#### Distribution Transformations

Why do we do this?

* **Inference**: variable has expected shape

* **Data Preparation / Cleaning**: correcting for too few data and outliers

* **Theory**: a specific distribution assumption required for a method

How do we do it?

We apply this to all sample data, $x_{\alpha}$ $\forall$ $\alpha = 1,\ldots,n$.

\begin{equation}
y_{\alpha} = G^{-1}_Y\left(F_X(x_{\alpha})\right)
\end{equation}

were $X$ is the original feature with a $F_X$ original cumulative distribution function and $Y$ is transformed feature with a $G_Y$ transformed cumulative distribution function.

* Mapping from one distribution to another through percentiles

* This may be applied to any parametric or nonparametric distributions

* This is a rank preserving transform, e.g. P50 of 𝑋 is P50 of 𝑌

I have a lecture on distribution transformations available on [YouTube](https://www.youtube.com/watch?v=ZDIpE3OkAIU&list=PLG19vXLQHvSB-D4XKYieEku9GQMQyAzjJ&index=14).

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - [sample_data.csv](https://github.com/GeostatsGuy/GeoDataSets/blob/master/sample_data.csv).

#### Importing Packages

We will need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                        # ndarrys for gridded data
import pandas as pd                       # DataFrames for tabular data
import os                                 # set working directory, run executables
import matplotlib.pyplot as plt           # plotting
from scipy import stats                   # summary statistics
import math                               # trigonometry etc.
import scipy.signal as signal             # kernel for moving window calculation
import random                             # randon numbers
import seaborn as sns                     # matrix scatter plots
from scipy.stats import norm              # Gaussian parametric distribution
from sklearn import preprocessing
import geostatspy.GSLIB as GSLIB
from ipywidgets import interactive                      # widgets and interactivity
from ipywidgets import widgets
from ipywidgets import Layout
from ipywidgets import Label
from ipywidgets import VBox, HBox
```

#### Set the Random Number Seed

Set the random number seed so that we have a repeatable workflow

```python
seed = 73073
```

#### Set the Working Directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")                     # set the working directory
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.  For fun try misspelling the name. You will get an ugly, long error.

```python
df = pd.read_csv('sample_data.csv')     # load our data table (wrong name!)
```

It worked, we loaded our file into our DataFrame called 'df'. But how do you really know that it worked? Visualizing the DataFrame would be useful and we already leard about these methods in this demo (https://git.io/fNgRW).

We can preview the DataFrame by printing a slice or by utilizing the 'head' DataFrame member function (with a nice and clean format, see below). With the slice we could look at any subset of the data table and with the head command, add parameter 'n=13' to see the first 13 rows of the dataset.

```python
df.head(n=6)                           # we could also use this command for a table preview
```

      X
      Y
      Facies
      Porosity
      Perm
      AI

      0
      100.0
      900.0
      1.0
      0.100187
      1.363890
      5110.699751

      1
      100.0
      800.0
      0.0
      0.107947
      12.576845
      4671.458560

      2
      100.0
      700.0
      0.0
      0.085357
      5.984520
      6127.548006

      3
      100.0
      600.0
      0.0
      0.108460
      2.446678
      5201.637996

      4
      100.0
      500.0
      0.0
      0.102468
      1.952264
      3835.270322

      5
      100.0
      400.0
      0.0
      0.110579
      3.691908
      5295.267191

#### Calculating and Plotting a CDF by Hand

Let's demonstrate the calculation and plotting of a non-parametric CDF by hand

1. make a copy of the feature as a 1D array (ndarray from NumPy)
2. sort the data in ascending order
3. assign cumulative probabilities based on the tail assumptions
4. plot cumuative probability vs. value

```python
por = df['Porosity'].copy(deep = True).values # make a deepcopy of the feature from the DataFrame
print('The ndarray has a shape of ' + str(por.shape) + '.')

por = np.sort(por)                           # sort the data in ascending order
n = por.shape[0]                             # get the number of data samples

cprob = np.zeros(n)
for i in range(0,n):
    index = i + 1
    cprob[i] = index / n                     # known upper tail
    # cprob[i] = (index - 1)/n               # known lower tail
    # cprob[i] = (index - 1)/(n - 1)         # known upper and lower tails
    # cprob[i] = index/(n+1)                 # unknown tails

plt.subplot(111)
plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Non-parametric Porosity Cumulative Distribution Function")

plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2)
plt.show()
```

#### Transformation to a Parametric Distribution

We can transform our data feature distribution to any parametric distribution with this workflow.

1. Calculate the cumulative probability value of each of our data values, $p_{\alpha} = F_x(x_\alpha)$, $\forall$ $\alpha = 1,\ldots, n$.

2. Apply the inverse of the target parametric cumulative distribution function (CDF) to calculate the transformed values. $y_{\alpha} = G_y^{-1}\left(F_x(x_\alpha)\right)$, $\forall$ $\alpha = 1,\ldots, n$.

```python
y = np.zeros(n)

for i in range(0,n):
    y[i] = norm.ppf(cprob[i],loc=0.0,scale=1.0)

plt.subplot(121)
plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Non-parametric Porosity Cumulative Distribution Function")

plt.subplot(122)
plt.plot(y,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(y,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
plt.grid(); plt.xlim([-3.0,3.0]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("After Distribution Transformation to Gaussian")

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()

```

Let's make an interactive version of this plot to visualize the transformation.

```python
# widgets and dashboard
l = widgets.Text(value='                                                  Data Analytics, Distribution Transformation, Prof. Michael Pyrcz, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))

data_index = widgets.IntSlider(min=1, max = n-1, value=1.0, step = 10.0, description = 'Data Index, $\\alpha$',orientation='horizontal', style = {'description_width': 'initial'}, continuous_update=False)

ui = widgets.VBox([l,data_index],)

def run_plot(data_index):                       # make data, fit models and plot
    plt.subplot(131)
    plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.scatter(por,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
    plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
    plt.xlabel("Original Feature, $x$"); plt.ylabel("Cumulative Probability"); plt.title("Original Cumulative Distribution Function")
    plt.plot([por[data_index-1],por[data_index-1]],[0.0,cprob[data_index-1]],color = 'red',linestyle='dashed')
    plt.plot([por[data_index-1],3.0],[cprob[data_index-1],cprob[data_index-1]],color = 'red',linestyle='dashed')
    plt.annotate('x = ' + str(round(por[data_index-1],2)), xy=(por[data_index-1]+0.003, 0.01))
    plt.annotate('p = ' + str(round(cprob[data_index-1],2)), xy=(0.225, cprob[data_index-1]+0.02))

    plt.subplot(132)
    plt.plot(y,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.scatter(y,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
    plt.grid(); plt.xlim([-3.0,3.0]); plt.ylim([0.0,1.0])
    plt.xlabel("Gaussian Transformed Feature, $y$"); plt.ylabel("Cumulative Probability"); plt.title("After Distribution Transformation to Gaussian")
    plt.plot([-3.0,y[data_index-1]],[cprob[data_index-1],cprob[data_index-1]],color = 'red',linestyle='dashed')
    plt.plot([y[data_index-1],y[data_index-1]],[0.0,cprob[data_index-1]],color = 'red',linestyle='dashed')
    #plt.arrow(y[data_index-1],cprob[data_index-1],0.0,-1.0*(cprob[data_index-1]-0.01),color = 'red',width = 0.02, head_width = 0.1, linestyle='dashed', head_length = 0.01)
    plt.annotate('p = ' + str(round(cprob[data_index-1],2)), xy=(-2.90, cprob[data_index-1]+0.02))
    plt.annotate('y = ' + str(round(y[data_index-1],2)), xy=(y[data_index-1]+0.1, 0.01))

    plt.subplot(133)
    plt.plot(por,y, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([-3.0,3.0])
    plt.xlabel("Original Porosity (fraction)"); plt.ylabel("Gaussian Transformed Porosity (N[fraction])"); plt.title("Parametric Distribution Transformation, Q-Q Plot")
    #plt.plot([0.05,0.25],[0.05,0.25],color = 'red',linestyle='dashed', alpha = 0.4)
    plt.scatter(por[data_index-1],y[data_index-1],s = 50, c = 'red', edgecolor = 'black', alpha = 1.0, zorder=200) # plot the CDF points
    plt.scatter(por,y,s = 20, c = 'red', edgecolor = 'black', alpha = 0.1, zorder=100) # plot the CDF points

    plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=1.2, wspace=0.2, hspace=0.2)
    plt.show()

# connect the function to make the samples and plot to the widgets
interactive_plot = widgets.interactive_output(run_plot, {'data_index':data_index})
interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

### Interactive Data Analytics Distribution Transformation Demonstration

#### Michael Pyrcz, Associate Professor, The University of Texas at Austin

Select any data value and observe the distribution transform by mapping through cumulative probability.

### The Inputs

* **data_index** - the data index from 1 to n in the sorted ascending order

```python
display(ui, interactive_plot)                           # display the interactive plot
```

    VBox(children=(Text(value='                                                  Data Analytics, Distribution Tran…

    Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '
', 'i…

#### Distribution Transform to a Non-Parametric Distribution

We can apply the mapping through cumulative probabilities to transform from any distribution to any other distribution.

* let's make a new data set by randomly sampling from the previous one and adding error

Then we can demonstrate transforming this dataset to match the original distribution

* this is mimicking the situation where we transform a dataset to match the distribution of a better sampled analog distribution

```python
n_sample = 30
df_sample = df.sample(n_sample,random_state = seed)

df_sample = df_sample.copy(deep = True) # make a deepcopy of the feature from the DataFrame

df_sample['Porosity'] = df_sample['Porosity'].values + np.random.normal(loc = 0.0, scale = 0.01, size = n_sample)

df_sample = df_sample.sort_values(by = 'Porosity')                # sort the DataFrame
por_sample = df_sample['Porosity'].values
print('The sample ndarray has a shape of ' + str(por_sample.shape) + '.')

cprob_sample = np.zeros(n_sample)
for i in range(0,n_sample):
    index = i + 1
    cprob_sample[i] = index / n_sample       # known upper tail
    # cprob[i] = (index - 1)/n               # known lower tail
    # cprob[i] = (index - 1)/(n - 1)         # known upper and lower tails
    # cprob[i] = index/(n+1)                 # unknown tails

plt.subplot(121)
plt.plot(por_sample,cprob_sample, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por_sample,cprob_sample,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Sparse Sample with Noise Cumulative Distribution Function")

plt.subplot(122)
plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por,cprob,s = 10, alpha = 1.0, c = 'red', edgecolor = 'black') # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Non-parametric Porosity Cumulative Distribution Function")

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

Let's transform the values and show them on the target distribution.

```python
y_sample = np.zeros(n_sample)

for i in range(0,n_sample):
    y_sample[i] = np.percentile(por,cprob_sample[i]*100, interpolation = 'linear') # piecewise linear interpolation of inverse of target CDF

plt.subplot(121)
plt.plot(por_sample,cprob_sample, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por_sample,cprob_sample,s = 30, alpha = 1.0, c = 'green', edgecolor = 'black', zorder = 100) # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Sparse Sample with Noise Cumulative Distribution Function")

plt.subplot(122)
plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
plt.scatter(por,cprob,s = 10, c = 'red', edgecolor = 'black', alpha = 0.3) # plot the CDF points
plt.scatter(y_sample,cprob_sample,s = 30, c = 'green', edgecolor = 'black', alpha = 1.0, zorder = 100) # plot the CDF points
plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Non-parametric Porosity Cumulative Distribution Function")

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

Let's make an interactive version of this plot to visualize the transformation.

```python
# widgets and dashboard
l_sample = widgets.Text(value='                                                  Data Analytics, Distribution Transformation, Prof. Michael Pyrcz, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))

data_index_sample = widgets.IntSlider(min=1, max = n_sample, value=1.0, step = 1.0, description = 'Data Sample Index, $\\beta$',orientation='horizontal', style = {'description_width': 'initial'}, continuous_update=False)

ui_sample = widgets.VBox([l_sample,data_index_sample],)

def run_plot_sample(data_index_sample):                       # make data, fit models and plot
    plt.subplot(131)
    plt.plot(por_sample,cprob_sample, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.scatter(por_sample,cprob_sample,s = 30, alpha = 1.0, c = 'green', edgecolor = 'black',zorder = 100) # plot the CDF points
    plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
    plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Original Sparse Sample with Noise, Cumulative Distribution Function")
    plt.plot([por_sample[data_index_sample-1],por_sample[data_index_sample-1]],[0.0,cprob_sample[data_index_sample-1]],color = 'red',linestyle='dashed')
    plt.plot([por_sample[data_index_sample-1],3.0],[cprob_sample[data_index_sample-1],cprob_sample[data_index_sample-1]],color = 'red',linestyle='dashed')
    plt.annotate('x = ' + str(round(por_sample[data_index_sample-1],2)), xy=(por_sample[data_index_sample-1]+0.003, 0.01))
    plt.annotate('p = ' + str(round(cprob_sample[data_index_sample-1],2)), xy=(0.225, cprob_sample[data_index_sample-1]+0.02))

    plt.subplot(132)
    plt.plot(por,cprob, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.scatter(por,cprob,s = 10, c = 'red', edgecolor = 'black', alpha = 1.0) # plot the CDF points
    plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.0,1.0])
    plt.xlabel("Porosity (fraction)"); plt.ylabel("Cumulative Probability"); plt.title("Non-parametric Target Porosity Cumulative Distribution Function")
    plt.plot([0.0,y_sample[data_index_sample-1]],[cprob_sample[data_index_sample-1],cprob_sample[data_index_sample-1]],color = 'red',linestyle='dashed')
    plt.plot([y_sample[data_index_sample-1],y_sample[data_index_sample-1]],[0.0,cprob_sample[data_index_sample-1]],color = 'red',linestyle='dashed')
    plt.annotate('p = ' + str(round(cprob_sample[data_index_sample-1],2)), xy=(0.053, cprob_sample[data_index_sample-1]+0.02))
    plt.annotate('y = ' + str(round(y_sample[data_index_sample-1],2)), xy=(y_sample[data_index_sample-1]+0.003, 0.01))
    plt.scatter(y_sample[data_index_sample-1],cprob_sample[data_index_sample-1],s = 50, c = 'green', edgecolor = 'black', alpha = 1.0, zorder=100) # plot the CDF points

    plt.subplot(133)
    plt.plot(por_sample,y_sample, alpha = 0.2, c = 'black') # plot piecewise linear interpolation
    plt.grid(); plt.xlim([0.05,0.25]); plt.ylim([0.05,0.25])
    plt.xlabel("Original Porosity (fraction)"); plt.ylabel("Transformed Porosity (fraction)"); plt.title("Non-parametric Distribution Transformation, Q-Q Plot")
    plt.plot([0.05,0.25],[0.05,0.25],color = 'red',linestyle='dashed', alpha = 0.4)
    plt.scatter(por_sample[data_index_sample-1],y_sample[data_index_sample-1],s = 50, c = 'green', edgecolor = 'black', alpha = 1.0, zorder=200) # plot the CDF points
    plt.scatter(por_sample,y_sample,s = 20, c = 'green', edgecolor = 'black', alpha = 0.3, zorder=100) # plot the CDF points

    plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=1.2, wspace=0.2, hspace=0.2)
    plt.show()

# connect the function to make the samples and plot to the widgets
interactive_plot_s = widgets.interactive_output(run_plot_sample, {'data_index_sample':data_index_sample})
#interactive_plot_sample.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

### Interactive Data Analytics Distribution Transformation Demonstration

#### Michael Pyrcz, Associate Professor, The University of Texas at Austin

Select any data value and observe the distribution transform by mapping through cumulative probability.

#### The Inputs

* **data_index** - the data index from 1 to n in the sorted ascending order

```python
display(ui_sample, interactive_plot_s)                           # display the interactive plot
```

    VBox(children=(Text(value='                                                  Data Analytics, Distribution Tran…

    Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '
', 'i…

To summarize let's look at a DataFrame with the original noisey sample and the transformed to match the original distribution.

* we're making and showing a table of original values, $x_{\beta}$ $\forall$ $\beta = 1, \ldots, n_{sample}$, and the transformed values, $y_{\beta}$ $\forall$ $\beta = 1, \ldots, n_{sample}$.

```python
df_sample['Transformed_Por'] = y_sample
df_sample.head(n=n_sample)
```

      X
      Y
      Facies
      Porosity
      Perm
      AI
      Transformed_Por

      80
      900.0
      100.0
      0.0
      0.077913
      1.280257
      4573.656072
      0.081044

      189
      201.0
      456.0
      0.0
      0.091533
      0.546396
      5018.355476
      0.085867

      218
      251.0
      416.0
      0.0
      0.093602
      1.003374
      5822.467914
      0.091834

      207
      201.0
      426.0
      0.0
      0.101475
      0.400658
      5263.542112
      0.095628

      5
      100.0
      400.0
      0.0
      0.108655
      3.691908
      5295.267191
      0.099378

      41
      500.0
      400.0
      0.0
      0.109365
      6.312198
      5515.918646
      0.101987

      3
      100.0
      600.0
      0.0
      0.112756
      2.446678
      5201.637996
      0.103970

      72
      900.0
      900.0
      0.0
      0.118671
      12.433996
      6242.704810
      0.106704

      47
      600.0
      700.0
      0.0
      0.120329
      12.384496
      3595.586977
      0.108460

      71
      800.0
      100.0
      0.0
      0.120578
      7.739105
      5274.532660
      0.111491

      210
      231.0
      426.0
      0.0
      0.125237
      5.584040
      4919.074871
      0.113887

      53
      600.0
      100.0
      1.0
      0.126804
      42.396044
      4204.150893
      0.117984

      226
      211.0
      396.0
      0.0
      0.129894
      6.368529
      5725.334803
      0.121592

      84
      955.0
      559.0
      1.0
      0.184568
      74.215058
      3386.182722
      0.127131

      111
      955.0
      529.0
      1.0
      0.187061
      1113.971076
      3177.635737
      0.137062

      150
      985.0
      489.0
      1.0
      0.187291
      73.133040
      2672.294567
      0.153759

      93
      955.0
      549.0
      1.0
      0.190464
      374.298925
      3181.557281
      0.176536

      174
      955.0
      459.0
      1.0
      0.192032
      45.002088
      3394.563038
      0.185270

      245
      690.0
      529.0
      1.0
      0.194038
      316.905689
      4271.013148
      0.188136

      181
      935.0
      449.0
      1.0
      0.194922
      368.507601
      4249.477923
      0.191655

      151
      995.0
      489.0
      1.0
      0.195531
      460.494986
      2792.804322
      0.195998

      165
      955.0
      469.0
      1.0
      0.200185
      26.197239
      2889.196647
      0.198182

      158
      975.0
      479.0
      1.0
      0.201035
      69.336576
      2493.128177
      0.199315

      125
      1005.0
      519.0
      1.0
      0.204180
      54.667195
      2577.714678
      0.201943

      99
      925.0
      539.0
      1.0
      0.206040
      211.163296
      3442.885245
      0.206934

      141
      985.0
      499.0
      1.0
      0.211892
      68.276148
      2547.526113
      0.209051

      113
      975.0
      529.0
      1.0
      0.215852
      1548.094062
      3167.185377
      0.211638

      136
      935.0
      499.0
      1.0
      0.223535
      523.287810
      2579.032897
      0.215686

      129
      955.0
      509.0
      1.0
      0.225374
      1525.247066
      2512.061434
      0.224214

      149
      975.0
      489.0
      1.0
      0.243111
      21.109085
      2412.875330
      0.242298

It would be straitforward to modify the code above to perform distribution transformations:

* to a parametric distribution like Gaussian

* to a non-parametric distribution from actual data (build a CDF and interpolate between the data samples)

#### Comments

This was a basic demonstration of distribution transformations.

I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at [Python Demos](https://github.com/GeostatsGuy/PythonNumericalDemos) and a Python package for data analytics and geostatistics at [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy).

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```