```python
!pip install pennylane StrawberryFields tensorflow-hub tensorflow_text emoji pythainlp pennylane-sf  plotnine
```

    Requirement already satisfied: pennylane in /usr/local/lib/python3.6/dist-packages (0.9.0)
    Requirement already satisfied: StrawberryFields in /usr/local/lib/python3.6/dist-packages (0.14.0)
    Requirement already satisfied: tensorflow-hub in /usr/local/lib/python3.6/dist-packages (0.8.0)
    Requirement already satisfied: tensorflow_text in /usr/local/lib/python3.6/dist-packages (2.2.1)
    Requirement already satisfied: emoji in /usr/local/lib/python3.6/dist-packages (0.5.4)
    Requirement already satisfied: pythainlp in /usr/local/lib/python3.6/dist-packages (2.1.4)
    Requirement already satisfied: pennylane-sf in /usr/local/lib/python3.6/dist-packages (0.9.0)
    Collecting plotnine
      Downloading https://files.pythonhosted.org/packages/aa/ee/9f343ef0719541e02950384978722ea84cc433264cece09a11a0cccb2bf9/plotnine-0.7.0-py3-none-any.whl (4.4MB)
         |████████████████████████████████| 4.4MB 977kB/s eta 0:00:01
    Requirement already satisfied: autograd in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.3)
    Requirement already satisfied: networkx in /usr/local/lib/python3.6/dist-packages (from pennylane) (2.4)
    Requirement already satisfied: toml in /usr/local/lib/python3.6/dist-packages (from pennylane) (0.10.1)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.3.1)
    Requirement already satisfied: semantic-version==2.6 in /usr/local/lib/python3.6/dist-packages (from pennylane) (2.6.0)
    Requirement already satisfied: appdirs in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.4.4)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.17.4)
    Requirement already satisfied: quantum-blackbird>=0.2.3 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (0.2.4)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (2.8.1)
    Requirement already satisfied: requests>=2.22.0 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (2.22.0)
    Requirement already satisfied: thewalrus>=0.12 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (0.12.0)
    Requirement already satisfied: urllib3>=1.25.3 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (1.25.7)
    Requirement already satisfied: numba in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (0.49.1)
    Requirement already satisfied: sympy>=1.5 in /usr/local/lib/python3.6/dist-packages (from StrawberryFields) (1.6)
    Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-hub) (1.13.0)
    Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-hub) (3.10.0)
    Requirement already satisfied: tensorflow<2.3,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow_text) (2.2.0)
    Requirement already satisfied: nltk>=3.3 in /usr/local/lib/python3.6/dist-packages (from pythainlp) (3.4.5)
    Requirement already satisfied: dill>=0.3.0 in /usr/local/lib/python3.6/dist-packages (from pythainlp) (0.3.1.1)
    Requirement already satisfied: tqdm>=4.1 in /usr/local/lib/python3.6/dist-packages (from pythainlp) (4.38.0)
    Requirement already satisfied: tinydb>=3.0 in /usr/local/lib/python3.6/dist-packages (from pythainlp) (3.15.2)
    Collecting mizani>=0.7.1
      Downloading https://files.pythonhosted.org/packages/57/29/72ed07818b1cd7f79340d55c7230748238e09c35fc8461c545a5d350006d/mizani-0.7.1-py3-none-any.whl (62kB)
         |████████████████████████████████| 71kB 11.1MB/s eta 0:00:01
    Collecting statsmodels>=0.11.1
      Downloading https://files.pythonhosted.org/packages/cb/83/540fd83238a18abe6c2d280fa8e489ac5fcefa1f370f0ca1acd16ae1b860/statsmodels-0.11.1-cp36-cp36m-manylinux1_x86_64.whl (8.7MB)
         |████████████████████████████████| 8.7MB 11.3MB/s eta 0:00:01
    Requirement already satisfied: matplotlib>=3.1.1 in /usr/local/lib/python3.6/dist-packages (from plotnine) (3.2.1)
    Collecting patsy>=0.5.1
      Downloading https://files.pythonhosted.org/packages/ea/0c/5f61f1a3d4385d6bf83b83ea495068857ff8dfb89e74824c6e9eb63286d8/patsy-0.5.1-py2.py3-none-any.whl (231kB)
         |████████████████████████████████| 235kB 29.0MB/s eta 0:00:01
    Collecting descartes>=1.1.0
      Downloading https://files.pythonhosted.org/packages/e5/b6/1ed2eb03989ae574584664985367ba70cd9cf8b32ee8cad0e8aaeac819f3/descartes-1.1.0-py3-none-any.whl
    Requirement already satisfied: pandas>=1.0.3 in /usr/local/lib/python3.6/dist-packages (from plotnine) (1.0.3)
    Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from autograd->pennylane) (0.17.1)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx->pennylane) (4.4.1)
    Requirement already satisfied: antlr4-python3-runtime==4.7.2 in /usr/local/lib/python3.6/dist-packages (from quantum-blackbird>=0.2.3->StrawberryFields) (4.7.2)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.22.0->StrawberryFields) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.22.0->StrawberryFields) (2019.9.11)
    Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.22.0->StrawberryFields) (2.8)
    Requirement already satisfied: dask[delayed] in /usr/local/lib/python3.6/dist-packages (from thewalrus>=0.12->StrawberryFields) (2.18.1)
    Requirement already satisfied: llvmlite<=0.33.0.dev0,>=0.31.0.dev0 in /usr/local/lib/python3.6/dist-packages (from numba->StrawberryFields) (0.32.1)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from numba->StrawberryFields) (41.6.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.5->StrawberryFields) (1.1.0)
    Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (0.8.1)
    Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (1.25.0)
    Requirement already satisfied: tensorflow-estimator<2.3.0,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (2.2.0)
    Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (1.1.0)
    Requirement already satisfied: keras-preprocessing>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (1.1.2)
    Requirement already satisfied: google-pasta>=0.1.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (0.1.8)
    Requirement already satisfied: astunparse==1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (1.6.3)
    Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (1.11.2)
    Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (2.10.0)
    Requirement already satisfied: gast==0.3.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (0.3.3)
    Requirement already satisfied: tensorboard<2.3.0,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (2.2.2)
    Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (3.1.0)
    Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from tensorflow<2.3,>=2.2.0->tensorflow_text) (0.33.6)
    Collecting palettable
      Downloading https://files.pythonhosted.org/packages/ca/46/5198aa24e61bb7eef28d06cb69e56bfa1942f4b6807d95a0b5ce361fe09b/palettable-3.3.0-py2.py3-none-any.whl (111kB)
         |████████████████████████████████| 112kB 23.6MB/s eta 0:00:01
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=3.1.1->plotnine) (1.2.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=3.1.1->plotnine) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=3.1.1->plotnine) (0.10.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=1.0.3->plotnine) (2019.3)
    Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from dask[delayed]->thewalrus>=0.12->StrawberryFields) (5.1.2)
    Requirement already satisfied: toolz>=0.8.2; extra == "delayed" in /usr/local/lib/python3.6/dist-packages (from dask[delayed]->thewalrus>=0.12->StrawberryFields) (0.10.0)
    Requirement already satisfied: cloudpickle>=0.2.2; extra == "delayed" in /opt/tensorflow/horovod-source/.eggs/cloudpickle-1.2.2-py3.6.egg (from dask[delayed]->thewalrus>=0.12->StrawberryFields) (1.2.2)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (0.4.1)
    Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (1.7.0)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (1.6.0.post3)
    Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (3.1.1)
    Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (0.16.0)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (1.3.0)
    Requirement already satisfied: cachetools<3.2,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (3.1.1)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (0.2.7)
    Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (4.0)
    Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (3.1.0)
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow<2.3,>=2.2.0->tensorflow_text) (0.4.7)
    Installing collected packages: palettable, mizani, patsy, statsmodels, descartes, plotnine
    Successfully installed descartes-1.1.0 mizani-0.7.1 palettable-3.3.0 patsy-0.5.1 plotnine-0.7.0 statsmodels-0.11.1
    WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.

```python
!wget https://github.com/PyThaiNLP/truevoice-intent/archive/master.zip; unzip master.zip; cd truevoice-intent-master; unzip mari-intent.zip; cp *.* ../;
```

    --2020-06-11 03:12:29--  https://github.com/PyThaiNLP/truevoice-intent/archive/master.zip
    Resolving github.com (github.com)... 52.74.223.119
    Connecting to github.com (github.com)|52.74.223.119|:443... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: https://codeload.github.com/PyThaiNLP/truevoice-intent/zip/master [following]
    --2020-06-11 03:12:30--  https://codeload.github.com/PyThaiNLP/truevoice-intent/zip/master
    Resolving codeload.github.com (codeload.github.com)... 13.229.189.0
    Connecting to codeload.github.com (codeload.github.com)|13.229.189.0|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [application/zip]
    Saving to: ‘master.zip’

    master.zip              [    <=>             ]   1.29M  2.07MB/s    in 0.6s

    2020-06-11 03:12:31 (2.07 MB/s) - ‘master.zip’ saved [1350596]

    Archive:  master.zip
    7ce601a547a4bac938b3bb0640a6036f7b89266c
       creating: truevoice-intent-master/
      inflating: truevoice-intent-master/.gitignore
      inflating: truevoice-intent-master/LICENSE
      inflating: truevoice-intent-master/README.md
      inflating: truevoice-intent-master/class_features.py
      inflating: truevoice-intent-master/classification.ipynb
      inflating: truevoice-intent-master/exploration.ipynb
     extracting: truevoice-intent-master/mari-intent.zip
      inflating: truevoice-intent-master/remove-dup-split.ipynb
      inflating: truevoice-intent-master/thsarabunnew-webfont.ttf
    Archive:  mari-intent.zip
      inflating: mari-train-balanced.csv
      inflating: mari-test-balanced.csv

```python
!mkdir ft_data
!mkdir truevoice_data
```

```python
!wget https://github.com/rdisipio/qnlp/raw/master/models.py
```

    --2020-06-11 03:12:33--  https://github.com/rdisipio/qnlp/raw/master/models.py
    Resolving github.com (github.com)... 52.74.223.119
    Connecting to github.com (github.com)|52.74.223.119|:443... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: https://raw.githubusercontent.com/rdisipio/qnlp/master/models.py [following]
    --2020-06-11 03:12:34--  https://raw.githubusercontent.com/rdisipio/qnlp/master/models.py
    Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.8.133
    Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.8.133|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 1212 (1.2K) [text/plain]
    Saving to: ‘models.py’

    models.py           100%[===================>]   1.18K  --.-KB/s    in 0s

    2020-06-11 03:12:34 (71.3 MB/s) - ‘models.py’ saved [1212/1212]

```python
!wget https://github.com/rdisipio/qnlp/raw/master/models_quantum.py
```

    --2020-06-11 03:12:35--  https://github.com/rdisipio/qnlp/raw/master/models_quantum.py
    Resolving github.com (github.com)... 52.74.223.119
    Connecting to github.com (github.com)|52.74.223.119|:443... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: https://raw.githubusercontent.com/rdisipio/qnlp/master/models_quantum.py [following]
    --2020-06-11 03:12:35--  https://raw.githubusercontent.com/rdisipio/qnlp/master/models_quantum.py
    Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.8.133
    Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.8.133|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 8498 (8.3K) [text/plain]
    Saving to: ‘models_quantum.py’

    models_quantum.py   100%[===================>]   8.30K  --.-KB/s    in 0s

    2020-06-11 03:12:36 (41.7 MB/s) - ‘models_quantum.py’ saved [8498/8498]

```python
import pandas as pd
import numpy as np
from pythainlp import word_tokenize
from tqdm import tqdm_notebook
from collections import Counter
import re
from sklearn.metrics import f1_score,precision_score,recall_score

#viz
from plotnine import *
import matplotlib.pyplot as plt
import seaborn as sns

def replace_newline(t):
    return re.sub('[\n]{1,}', ' ', t)

ft_data = 'ft_data/'

y = 'destination'
nb_class = 7
```

```python
import string
import emoji
def replace_url(text):
    URL_PATTERN = r"""(?i)\b((?:https?:(?:/{1,3}|[a-z0-9%])|[a-z0-9.\-]+[.](?:com|net|org|edu|gov|mil|aero|asia|biz|cat|coop|info|int|jobs|mobi|museum|name|post|pro|tel|travel|xxx|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|ax|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|cr|cs|cu|cv|cx|cy|cz|dd|de|dj|dk|dm|do|dz|ec|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gg|gh|gi|gl|gm|gn|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|im|in|io|iq|ir|is|it|je|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mg|mh|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|ps|pt|pw|py|qa|re|ro|rs|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|Ja|sk|sl|sm|sn|so|sr|ss|st|su|sv|sx|sy|sz|tc|td|tf|tg|th|tj|tk|tl|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)/)(?:[^\s()<>{}\[\]]+|\([^\s()]*?\([^\s()]+\)[^\s()]*?\)|\([^\s]+?\))+(?:\([^\s()]*?\([^\s()]+\)[^\s()]*?\)|\([^\s]+?\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’])|(?:(?<!@)[a-z0-9]+(?:[.\-][a-z0-9]+)*[.](?:com|net|org|edu|gov|mil|aero|asia|biz|cat|coop|info|int|jobs|mobi|museum|name|post|pro|tel|travel|xxx|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|ax|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|cr|cs|cu|cv|cx|cy|cz|dd|de|dj|dk|dm|do|dz|ec|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gg|gh|gi|gl|gm|gn|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|im|in|io|iq|ir|is|it|je|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mg|mh|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|ps|pt|pw|py|qa|re|ro|rs|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|Ja|sk|sl|sm|sn|so|sr|ss|st|su|sv|sx|sy|sz|tc|td|tf|tg|th|tj|tk|tl|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)\b/?(?!@)))"""
    return re.sub(URL_PATTERN, 'xxurl', text)

def replace_rep(text):
    def _replace_rep(m):
        c,cc = m.groups()
        return f'{c}xxrep'
    re_rep = re.compile(r'(\S)(\1{2,})')
    return re_rep.sub(_replace_rep, text)

def ungroup_emoji(toks):
    res = []
    for tok in toks:
        if emoji.emoji_count(tok) == len(tok):
            for char in tok:
                res.append(char)
        else:
            res.append(tok)
    return res

def process_text(text):
    #pre rules
    res = text.lower().strip()
    res = replace_url(res)
    res = replace_rep(res)

    #tokenize
    res = [word for word in res.split('|') if word and not re.search(pattern=r"\s+", string=word)]

    #post rules
    res = ungroup_emoji(res)

    return res
```

#Train-validation-test Split

We perform 85/15 train-validation split in addition to the test split by TrueVoice.

```python
from sklearn.model_selection import train_test_split
all_df = pd.read_csv(f'mari-train-balanced.csv')
all_df['destination'] = all_df.destination.map(lambda x: x.replace(' ','_'))
#train_df, valid_df = train_test_split(all_df, test_size=0.15, random_state=1412)
train_df = all_df.reset_index(drop=True)
#valid_df = valid_df.reset_index(drop=True)
test_df = pd.read_csv(f'mari-test-balanced.csv')
test_df['destination'] = test_df.destination.map(lambda x: x.replace(' ','_'))
print(train_df.shape, test_df.shape)
```

    (10303, 5) (2607, 5)

```python
#test set prevalence
test_df['destination'].value_counts() / test_df.shape[0]
```

    billing_and_payment      0.376678
    promotions               0.220560
    other_queries            0.176064
    internet                 0.154200
    international_dialing    0.034522
    true_money               0.019946
    lost_and_stolen          0.018028
    Name: destination, dtype: float64

```python
#!pip install pennylane pennylane-sf

import pennylane
pennylane.device("strawberryfields.gaussian", wires=2) # ถ้า error ให้ restart runtime
```

    <StrawberryFieldsGaussian device (wires=2, shots=1000) at 0x7fdec808beb8>

```python
import tensorflow_hub as hub
import tensorflow_text
import tensorflow as tf #tensorflow 2.1.0
from tensorflow.keras.callbacks import EarlyStopping

from models_quantum import make_model_quantum
enc = hub.load('https://tfhub.dev/google/universal-sentence-encoder-multilingual/3')
```

```python
tf.keras.backend.set_floatx('float64')
```

```python
train_df.shape, test_df.shape
```

    ((10303, 5), (2607, 5))

```python

def embed_text(X_txt):
    print("Embedding input text...")

    X = np.array([np.array(x) for x in enc(X_txt)])

    return X

```

```python
X_train = embed_text(train_df.texts)
```

    Embedding input text...

```python
X_test = embed_text(test_df.texts)
```

    Embedding input text...

```python
embedding_dim =X_train.shape[-1]
```

```python
embedding_dim
```

    512

```python
index = {j:i for i,j in enumerate(list(set(train_df[y].values)))}
```

```python
index
```

    {'true_money': 0,
     'promotions': 1,
     'other_queries': 2,
     'internet': 3,
     'lost_and_stolen': 4,
     'billing_and_payment': 5,
     'international_dialing': 6}

```python
from tensorflow.keras.utils import to_categorical
```

```python
y_train = [index[i] for i in train_df[y].values]
```

```python
y_train = to_categorical(y_train)
```

```python
y_test  = [index[i] for i in test_df[y].values]
```

```python
y_test = to_categorical(y_test)
```

```python
N_EPOCHS = 20
BATCH_SIZE = 32
n_categories = 7
```

```python
embedding_dim = X_train.shape[-1]
```

```python
import pennylane as qml
qml.about()
```

    Name: PennyLane
    Version: 0.9.0
    Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
    Home-page: https://github.com/XanaduAI/pennylane
    Author: None
    Author-email: None
    License: Apache License 2.0
    Location: /usr/local/lib/python3.6/dist-packages
    Requires: semantic-version, scipy, appdirs, autograd, networkx, numpy, toml
    Required-by: PennyLane-SF
    Platform info:           Linux-4.15.0-72-generic-x86_64-with-Ubuntu-18.04-bionic
    Python version:          3.6.8
    Numpy version:           1.17.4
    Scipy version:           1.3.1
    Installed devices:
    - default.gaussian (PennyLane-0.9.0)
    - default.qubit (PennyLane-0.9.0)
    - default.tensor (PennyLane-0.9.0)
    - default.tensor.tf (PennyLane-0.9.0)
    - strawberryfields.fock (PennyLane-SF-0.9.0)
    - strawberryfields.gaussian (PennyLane-SF-0.9.0)

```python
model = make_model_quantum(n_categories=n_categories,  n_qubits=4, n_layers=1, embedding_dim=embedding_dim)
```

```python
model.summary()
```

    Model: "QuantumPreprintClassifier"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    text_in (InputLayer)         [(None, 512)]             0
    _________________________________________________________________
    dense_4 (Dense)              (None, 4)                 2052
    _________________________________________________________________
    variational_quantum_circuit_ (None, 4)                 12
    _________________________________________________________________
    dense_7 (Dense)              (None, 7)                 35
    =================================================================
    Total params: 2,099
    Trainable params: 2,099
    Non-trainable params: 0
    _________________________________________________________________

```python
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
```

```python
model.compile(optimizer, loss='categorical_crossentropy')
```

```python
early_stopping_callback = EarlyStopping(monitor='val_loss', patience=3, min_delta=0.01,restore_best_weights=True)
```

```python
from matplotlib import pyplot as plt
from IPython.display import clear_output
import numpy as np
```

```python
class PlotLosses(tf.keras.callbacks.Callback): # code from https://gist.github.com/stared/dfb4dfaf6d9a8501cd1cc8b8cb806d2e
    def on_train_begin(self, logs={}):
        self.i = 0
        self.x = []
        self.losses = []
        self.val_losses = []

        self.fig = plt.figure()

        self.logs = []

    def on_epoch_end(self, epoch, logs={}):

        self.logs.append(logs)
        self.x.append(self.i)
        self.losses.append(logs.get('loss'))
        self.val_losses.append(logs.get('val_loss'))
        self.i += 1

        clear_output(wait=True)
        plt.plot(self.x, self.losses, label="loss")
        plt.plot(self.x, self.val_losses, label="val_loss")
        plt.legend()
        plt.show();

plot_losses = PlotLosses()
```

```python
callbacks = [early_stopping_callback,plot_losses]
```

```python
history = model.fit(X_train, y_train,epochs=N_EPOCHS, batch_size=BATCH_SIZE,validation_split=0.15,callbacks =callbacks)
```

```python
dir(model)
```

    ['_TF_MODULE_IGNORED_PROPERTIES',
     '__call__',
     '__class__',
     '__delattr__',
     '__dict__',
     '__dir__',
     '__doc__',
     '__eq__',
     '__format__',
     '__ge__',
     '__getattribute__',
     '__getstate__',
     '__gt__',
     '__hash__',
     '__init__',
     '__init_subclass__',
     '__le__',
     '__lt__',
     '__module__',
     '__ne__',
     '__new__',
     '__reduce__',
     '__reduce_ex__',
     '__repr__',
     '__setattr__',
     '__setstate__',
     '__sizeof__',
     '__str__',
     '__subclasshook__',
     '__weakref__',
     '_activity_regularizer',
     '_add_inbound_node',
     '_add_trackable',
     '_add_variable_with_custom_getter',
     '_assert_compile_was_called',
     '_assert_weights_created',
     '_attribute_sentinel',
     '_auto_track_sub_layers',
     '_autocast',
     '_base_init',
     '_build_input_shape',
     '_call_accepts_kwargs',
     '_call_arg_was_passed',
     '_call_fn_args',
     '_call_full_argspec',
     '_callable_losses',
     '_check_call_args',
     '_checkpoint_dependencies',
     '_clear_losses',
     '_collect_input_masks',
     '_compile_was_called',
     '_compiled_trainable_state',
     '_compute_dtype',
     '_compute_output_and_mask_jointly',
     '_compute_tensor_usage_count',
     '_conform_to_reference_input',
     '_dedup_weights',
     '_deferred_dependencies',
     '_distribution_strategy',
     '_dtype',
     '_dtype_defaulted_to_floatx',
     '_dtype_policy',
     '_dynamic',
     '_eager_add_metric',
     '_eager_losses',
     '_enable_dict_to_input_mapping',
     '_expects_mask_arg',
     '_expects_training_arg',
     '_feed_input_names',
     '_feed_input_shapes',
     '_feed_inputs',
     '_flatten',
     '_flatten_to_reference_inputs',
     '_gather_children_attribute',
     '_gather_layers',
     '_gather_saveables_for_checkpoint',
     '_gather_unique_layers',
     '_get_call_arg_value',
     '_get_callback_model',
     '_get_compile_args',
     '_get_distribution_strategy',
     '_get_existing_metric',
     '_get_node_attribute_at_index',
     '_get_optimizer',
     '_get_save_spec',
     '_get_trainable_state',
     '_graph',
     '_graph_network_add_loss',
     '_graph_network_add_metric',
     '_handle_activity_regularization',
     '_handle_deferred_dependencies',
     '_handle_deferred_layer_dependencies',
     '_handle_weight_regularization',
     '_in_multi_worker_mode',
     '_inbound_nodes',
     '_init_call_fn_args',
     '_init_graph_network',
     '_init_set_name',
     '_init_subclassed_network',
     '_initial_weights',
     '_input_coordinates',
     '_input_layers',
     '_input_spec',
     '_insert_layers',
     '_is_compiled',
     '_is_graph_network',
     '_is_layer',
     '_keras_api_names',
     '_keras_api_names_v1',
     '_layer_call_argspecs',
     '_layer_checkpoint_dependencies',
     '_layers',
     '_list_extra_dependencies_for_serialization',
     '_list_functions_for_serialization',
     '_lookup_dependency',
     '_losses',
     '_maybe_build',
     '_maybe_cast_inputs',
     '_maybe_create_attribute',
     '_maybe_initialize_trackable',
     '_maybe_load_initial_epoch_from_ckpt',
     '_metrics',
     '_metrics_lock',
     '_name',
     '_name_based_attribute_restore',
     '_name_based_restores',
     '_name_scope',
     '_nested_inputs',
     '_nested_outputs',
     '_network_nodes',
     '_no_dependency',
     '_nodes_by_depth',
     '_non_trainable_weights',
     '_obj_reference_counts',
     '_obj_reference_counts_dict',
     '_object_identifier',
     '_outbound_nodes',
     '_output_coordinates',
     '_output_layers',
     '_output_mask_cache',
     '_output_shape_cache',
     '_output_tensor_cache',
     '_preload_simple_restoration',
     '_reset_compile_cache',
     '_restore_from_checkpoint_position',
     '_reuse',
     '_run_eagerly',
     '_run_internal_graph',
     '_saved_model_inputs_spec',
     '_scope',
     '_self_name_based_restores',
     '_self_setattr_tracking',
     '_self_unconditional_checkpoint_dependencies',
     '_self_unconditional_deferred_dependencies',
     '_self_unconditional_dependency_names',
     '_self_update_uid',
     '_set_connectivity_metadata_',
     '_set_dtype_policy',
     '_set_inputs',
     '_set_mask_metadata',
     '_set_output_names',
     '_set_save_spec',
     '_set_trainable_state',
     '_setattr_tracking',
     '_should_compute_mask',
     '_should_eval',
     '_single_restoration_from_checkpoint_position',
     '_stateful',
     '_supports_ragged_inputs',
     '_symbolic_add_metric',
     '_symbolic_call',
     '_tensor_usage_count',
     '_tf_api_names',
     '_tf_api_names_v1',
     '_thread_local',
     '_track_trackable',
     '_trackable_saved_model_saver',
     '_trackable_saver',
     '_tracking_metadata',
     '_trainable',
     '_trainable_weights',
     '_training_state',
     '_unconditional_checkpoint_dependencies',
     '_unconditional_dependency_names',
     '_undeduplicated_weights',
     '_update_uid',
     '_updated_config',
     '_updates',
     '_validate_compile',
     '_validate_graph_inputs_and_outputs',
     '_warn_about_input_casting',
     'activity_regularizer',
     'add_loss',
     'add_metric',
     'add_update',
     'add_variable',
     'add_weight',
     'apply',
     'build',
     'built',
     'call',
     'compile',
     'compiled_loss',
     'compiled_metrics',
     'compute_mask',
     'compute_output_shape',
     'compute_output_signature',
     'count_params',
     'distribute_strategy',
     'dtype',
     'dynamic',
     'evaluate',
     'evaluate_generator',
     'fit',
     'fit_generator',
     'from_config',
     'get_config',
     'get_input_at',
     'get_input_mask_at',
     'get_input_shape_at',
     'get_layer',
     'get_losses_for',
     'get_output_at',
     'get_output_mask_at',
     'get_output_shape_at',
     'get_updates_for',
     'get_weights',
     'history',
     'inbound_nodes',
     'input',
     'input_mask',
     'input_names',
     'input_shape',
     'input_spec',
     'inputs',
     'layers',
     'load_weights',
     'loss',
     'losses',
     'make_predict_function',
     'make_test_function',
     'make_train_function',
     'metrics',
     'metrics_names',
     'name',
     'name_scope',
     'non_trainable_variables',
     'non_trainable_weights',
     'optimizer',
     'outbound_nodes',
     'output',
     'output_mask',
     'output_names',
     'output_shape',
     'outputs',
     'predict',
     'predict_function',
     'predict_generator',
     'predict_on_batch',
     'predict_step',
     'reset_metrics',
     'reset_states',
     'run_eagerly',
     'save',
     'save_weights',
     'set_weights',
     'state_updates',
     'stateful',
     'stop_training',
     'submodules',
     'summary',
     'supports_masking',
     'test_function',
     'test_on_batch',
     'test_step',
     'to_json',
     'to_yaml',
     'train_function',
     'train_on_batch',
     'train_step',
     'trainable',
     'trainable_variables',
     'trainable_weights',
     'updates',
     'variables',
     'weights',
     'with_name_scope']

```python
model.predict(X_test)
```

    array([[1.00877702e-02, 2.80440106e-01, 1.22222863e-01, ...,
            1.36565214e-03, 5.59027094e-01, 1.03205016e-03],
           [2.23888947e-03, 7.68675006e-01, 4.06706911e-02, ...,
            2.29768297e-04, 1.65777818e-01, 4.97024380e-04],
           [8.44313633e-03, 3.29500807e-04, 2.59435960e-02, ...,
            2.01736112e-03, 9.63111778e-01, 3.68307758e-05],
           ...,
           [1.82711095e-02, 1.56434325e-01, 2.59251361e-01, ...,
            1.92394318e-03, 1.67550606e-01, 2.42491273e-03],
           [6.87279117e-03, 3.57861570e-01, 8.45242885e-02, ...,
            1.22508594e-03, 5.40770469e-01, 1.13316320e-03],
           [5.10181039e-03, 2.02961969e-02, 7.27803251e-02, ...,
            1.83616398e-04, 1.84993988e-02, 2.56527250e-04]])

```python
test_score = model.evaluate(X_test, y_test, verbose=2)
```

    82/82 - 34s - loss: 0.7899

```python
print(test_score)
```

    0.789912723104845

```python
t=model.predict(X_test)
```

```python
val2tag = {i:j for i,j in zip(index.values(),index.keys())}
```

```python
y_true = list(test_df[y].values)
```

```python
y_pred = [val2tag[list(i).index(i.max())] for i in t]
```

```python
from sklearn.metrics import classification_report
print(classification_report(y_true, y_pred))
```

                           precision    recall  f1-score   support

      billing_and_payment       0.80      0.83      0.82       982
    international_dialing       0.70      0.77      0.73        90
                 internet       0.70      0.67      0.68       402
          lost_and_stolen       0.00      0.00      0.00        47
            other_queries       0.61      0.68      0.64       459
               promotions       0.77      0.81      0.79       575
               true_money       0.00      0.00      0.00        52

                 accuracy                           0.74      2607
                macro avg       0.51      0.54      0.52      2607
             weighted avg       0.71      0.74      0.72      2607

    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.

```python

```