# Novel feedback experiences for enhanced mental training

Contributors: @cwbeitel

# Overview

Enhanced-feedback mental training experiences (FX) are the primary product of Project Clarify. Two categories of these are described in the following which in turn reference component issues:

- https://github.com/projectclarify/clarify/issues/84
- https://github.com/projectclarify/clarify/issues/83

This notebook is a central place for design docs (related to https://github.com/projectclarify/clarify/issues/147), interactive prototypes, and lightweight prototype development towards graduated prototypes later being integrated into our core front-end application.

# Setup

Please run the relevant sections here within and *remember* before newly-installed python dependencies become available the notebook runtime/kernel must be restarted (via "Runtime" > "Restart Runtime" above).

##### Clarify codebase and dependencies

```
# TODO: Check whether it's already present and if so skip
!git clone https://github.com/projectclarify/clarify.git
```

    Cloning into 'clarify'...
    remote: Enumerating objects: 72, done.
    remote: Counting objects: 100% (72/72), done.
    remote: Compressing objects: 100% (62/62), done.
    remote: Total 10244 (delta 30), reused 25 (delta 10), pack-reused 10172
    Receiving objects: 100% (10244/10244), 32.79 MiB | 10.61 MiB/s, done.
    Resolving deltas: 100% (2503/2503), done.

```
# Check out a fairly old version of the codebase that can be used to restore the
# image FEC model we trained in tensorflow.
!cd clarify && git checkout -b demo e5252a37fcdd4ccabdd9ef564ece0385365b577d && pip install -e .[tensorflow,tests]
```

    Checking out files: 100% (9719/9719), done.
    Switched to a new branch 'demo'
    Obtaining file:///content/clarify
    Collecting numpy==1.16.2
      Downloading https://files.pythonhosted.org/packages/35/d5/4f8410ac303e690144f0a0603c4b8fd3b986feb2749c435f7cdbb288f17e/numpy-1.16.2-cp36-cp36m-manylinux1_x86_64.whl (17.3MB)
         |████████████████████████████████| 17.3MB 207kB/s
    Requirement already satisfied: requests==2.21.0 in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (2.21.0)
    Collecting scipy==1.2.1
      Downloading https://files.pythonhosted.org/packages/7f/5f/c48860704092933bf1c4c1574a8de1ffd16bf4fde8bab190d747598844b2/scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8MB)
         |████████████████████████████████| 24.8MB 113kB/s
    Requirement already satisfied: six==1.12.0 in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (1.12.0)
    Requirement already satisfied: seaborn==0.9.0 in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (0.9.0)
    Collecting mne==0.17.1
      Downloading https://files.pythonhosted.org/packages/a1/e5/f60f23e3ef10f5b1efa1aabb407f0b2fc64ed12ac166a6e1c16dd7db6e77/mne-0.17.1.tar.gz (6.2MB)
         |████████████████████████████████| 6.2MB 18.5MB/s
    Collecting moviepy==1.0.0
      Downloading https://files.pythonhosted.org/packages/fb/32/a93f4af8b88985304a748ca0a66a64eb9fac53d0a9355ec33e713c4a3bf5/moviepy-1.0.0.tar.gz (398kB)
         |████████████████████████████████| 399kB 42.0MB/s
    Collecting kubernetes==9.0.0
      Downloading https://files.pythonhosted.org/packages/00/f7/4f196c55f1c2713d3edc8252c4b45326306eef4dc10048f13916fe446e2b/kubernetes-9.0.0-py2.py3-none-any.whl (1.4MB)
         |████████████████████████████████| 1.4MB 46.1MB/s
    Collecting rfc3339==6.0
      Downloading https://files.pythonhosted.org/packages/04/0b/8727b5496abde4622dc0e2ec2ea082e11d14ef70cfc73118b480447456ca/rfc3339-6.0.tar.gz
    Collecting dill==0.2.9
      Downloading https://files.pythonhosted.org/packages/fe/42/bfe2e0857bc284cbe6a011d93f2a9ad58a22cb894461b199ae72cfef0f29/dill-0.2.9.tar.gz (150kB)
         |████████████████████████████████| 153kB 47.1MB/s
    Requirement already satisfied: tensorflow_hub in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (0.7.0)
    Collecting google-cloud-bigtable
      Downloading https://files.pythonhosted.org/packages/70/de/e2023c20c39717741e0ef8eead585055ef6ea0cc1f9117e8aa18065ea9f3/google_cloud_bigtable-1.2.1-py2.py3-none-any.whl (234kB)
         |████████████████████████████████| 235kB 38.7MB/s
    Collecting google-cloud-pubsub
      Downloading https://files.pythonhosted.org/packages/b5/11/e1bfca8dd2f3407b55fa67e733fe1b38a6e0756750fdd1fbffda5ffbc526/google_cloud_pubsub-1.1.0-py2.py3-none-any.whl (120kB)
         |████████████████████████████████| 122kB 35.3MB/s
    Collecting firebase-admin
      Downloading https://files.pythonhosted.org/packages/49/b1/ba41c23eb0f7895a4db5a03255bfeec0f54ee8b70374dc68ff2586cafaa5/firebase_admin-3.2.1-py2.py3-none-any.whl (82kB)
         |████████████████████████████████| 92kB 10.1MB/s
    Collecting google-cloud-firestore==0.31.0
      Downloading https://files.pythonhosted.org/packages/58/0d/60b5f96aca1b88a7555f2758cae3a19956b9cdb65e562f5313224642783e/google_cloud_firestore-0.31.0-py2.py3-none-any.whl (125kB)
         |████████████████████████████████| 133kB 53.4MB/s
    Requirement already satisfied: google-auth in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (1.4.2)
    Collecting requests-toolbelt
      Downloading https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl (54kB)
         |████████████████████████████████| 61kB 6.3MB/s
    Requirement already satisfied: scikit-learn>=0.21.3 in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (0.22.1)
    Collecting faced@ https://github.com/iitzco/faced/archive/31ef0d30e1567a06113f49ff4a1202760d952df2.zip
      Downloading https://github.com/iitzco/faced/archive/31ef0d30e1567a06113f49ff4a1202760d952df2.zip
         / 81.2MB 3.9MB/s
    Collecting tensor2tensor==1.14.0
      Downloading https://files.pythonhosted.org/packages/64/75/e8fab6e46fcfaf278998b9d0a182361eaa1a9b5a9a7ecb58a0796d9e5229/tensor2tensor-1.14.0-py2.py3-none-any.whl (1.6MB)
         |████████████████████████████████| 1.6MB 31.8MB/s
    Collecting google-cloud-logging
      Downloading https://files.pythonhosted.org/packages/b9/06/3a89173f80d02d275ab4fdca5283a7fdb84845e214f3383b5d52c29f4b5f/google_cloud_logging-1.14.0-py2.py3-none-any.whl (134kB)
         |████████████████████████████████| 143kB 35.2MB/s
    Collecting pathos
      Downloading https://files.pythonhosted.org/packages/c6/a2/cd59f73d5ede4f122687bf8f63de5d671c9561e493ca699241f05b038278/pathos-0.2.5.tar.gz (162kB)
         |████████████████████████████████| 163kB 39.2MB/s
    Requirement already satisfied: opencv-python in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (4.1.2.30)
    Requirement already satisfied: tensorflow-probability==0.7.0 in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (0.7.0)
    Collecting gTTS
      Downloading https://files.pythonhosted.org/packages/a1/0c/4ca77eca3b739a4a08360930643f58d714e302fee0d2f8c654e67d9af8e7/gTTS-2.1.1-py3-none-any.whl
    Collecting tensorflow==1.14
      Downloading https://files.pythonhosted.org/packages/de/f0/96fb2e0412ae9692dbf400e5b04432885f677ad6241c088ccc5fe7724d69/tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2MB)
         |████████████████████████████████| 109.2MB 34kB/s
    Collecting pylint
      Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)
         |████████████████████████████████| 307kB 38.7MB/s
    Requirement already satisfied: pytest in /usr/local/lib/python3.6/dist-packages (from pcml==0.0.1) (3.6.4)
    Collecting pytest-cache
      Downloading https://files.pythonhosted.org/packages/d1/15/082fd0428aab33d2bafa014f3beb241830427ba803a8912a5aaeaf3a5663/pytest-cache-1.0.tar.gz
    Collecting tensorflow-serving-api==1.12.0
      Downloading https://files.pythonhosted.org/packages/79/69/1e724c0d98f12b12f9ad583a3df7750e14ec5f06069aa4be8d75a2ab9bb8/tensorflow_serving_api-1.12.0-py2.py3-none-any.whl
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests==2.21.0->pcml==0.0.1) (2019.11.28)
    Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests==2.21.0->pcml==0.0.1) (2.8)
    Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests==2.21.0->pcml==0.0.1) (1.24.3)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests==2.21.0->pcml==0.0.1) (3.0.4)
    Requirement already satisfied: pandas>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from seaborn==0.9.0->pcml==0.0.1) (0.25.3)
    Requirement already satisfied: matplotlib>=1.4.3 in /usr/local/lib/python3.6/dist-packages (from seaborn==0.9.0->pcml==0.0.1) (3.1.2)
    Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.6/dist-packages (from moviepy==1.0.0->pcml==0.0.1) (4.4.1)
    Requirement already satisfied: tqdm<5.0,>=4.11.2 in /usr/local/lib/python3.6/dist-packages (from moviepy==1.0.0->pcml==0.0.1) (4.28.1)
    Collecting proglog<=1.0.0
      Downloading https://files.pythonhosted.org/packages/fe/ab/4cb19b578e1364c0b2d6efd6521a8b4b4e5c4ae6528041d31a2a951dd991/proglog-0.1.9.tar.gz
    Collecting imageio<3.0,>=2.5
      Downloading https://files.pythonhosted.org/packages/1a/de/f7f985018f462ceeffada7f6e609919fbcc934acd9301929cba14bc2c24a/imageio-2.6.1-py3-none-any.whl (3.3MB)
         |████████████████████████████████| 3.3MB 37.2MB/s
    Collecting imageio_ffmpeg>=0.2.0
      Downloading https://files.pythonhosted.org/packages/1b/12/01126a2fb737b23461d7dadad3b8abd51ad6210f979ff05c6fa9812dfbbe/imageio_ffmpeg-0.3.0-py3-none-manylinux2010_x86_64.whl (22.2MB)
         |████████████████████████████████| 22.2MB 183kB/s
    Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.6/dist-packages (from kubernetes==9.0.0->pcml==0.0.1) (42.0.2)
    Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.6/dist-packages (from kubernetes==9.0.0->pcml==0.0.1) (2.6.1)
    Requirement already satisfied: pyyaml>=3.12 in /usr/local/lib/python3.6/dist-packages (from kubernetes==9.0.0->pcml==0.0.1) (3.13)
    Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0
      Downloading https://files.pythonhosted.org/packages/4c/5f/f61b420143ed1c8dc69f9eaec5ff1ac36109d52c80de49d66e0c36c3dfdf/websocket_client-0.57.0-py2.py3-none-any.whl (200kB)
         |████████████████████████████████| 204kB 52.8MB/s
    Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.6/dist-packages (from kubernetes==9.0.0->pcml==0.0.1) (1.3.0)
    Requirement already satisfied: protobuf>=3.4.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow_hub->pcml==0.0.1) (3.10.0)
    Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
      Downloading https://files.pythonhosted.org/packages/65/19/2060c8faa325fddc09aa67af98ffcb6813f39a0ad805679fa64815362b3a/grpc-google-iam-v1-0.12.3.tar.gz
    Requirement already satisfied: google-cloud-core<2.0dev,>=1.0.3 in /usr/local/lib/python3.6/dist-packages (from google-cloud-bigtable->pcml==0.0.1) (1.0.3)
    Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from google-cloud-bigtable->pcml==0.0.1) (1.15.0)
    Collecting cachecontrol>=0.12.4
      Downloading https://files.pythonhosted.org/packages/18/71/0a9df4206a5dc5ae7609c41efddab2270a2c1ff61d39de7591dc7302ef89/CacheControl-0.12.6-py2.py3-none-any.whl
    Requirement already satisfied: google-api-python-client>=1.7.8 in /usr/local/lib/python3.6/dist-packages (from firebase-admin->pcml==0.0.1) (1.7.11)
    Collecting google-cloud-storage>=1.18.0
      Downloading https://files.pythonhosted.org/packages/cd/6d/75c2a47af99d15aa8b4de4e66226c128e623f8c9d3e27a8588368ccc38fc/google_cloud_storage-1.25.0-py2.py3-none-any.whl (73kB)
         |████████████████████████████████| 81kB 8.2MB/s
    Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from google-cloud-firestore==0.31.0->pcml==0.0.1) (2018.9)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth->pcml==0.0.1) (0.2.7)
    Requirement already satisfied: cachetools>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth->pcml==0.0.1) (4.0.0)
    Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth->pcml==0.0.1) (4.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.21.3->pcml==0.0.1) (0.14.1)
    Requirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.3.2)
    Requirement already satisfied: bz2file in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.98)
    Requirement already satisfied: pypng in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.0.20)
    Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.1.1)
    Requirement already satisfied: gym in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.15.4)
    Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (6.2.2)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.16.0)
    Requirement already satisfied: flask in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.1.1)
    Requirement already satisfied: gunicorn in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (20.0.4)
    Requirement already satisfied: oauth2client in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (4.1.3)
    Requirement already satisfied: kfac in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.2.0)
    Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (2.8.0)
    Requirement already satisfied: mesh-tensorflow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.1.8)
    Requirement already satisfied: dopamine-rl in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.0.5)
    Requirement already satisfied: gin-config in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (0.3.0)
    Requirement already satisfied: tensorflow-gan in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (2.0.0)
    Requirement already satisfied: gevent in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.4.0)
    Requirement already satisfied: grpcio in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.14.0->pcml==0.0.1) (1.15.0)
    Collecting ppft>=1.6.6.1
      Downloading https://files.pythonhosted.org/packages/5b/16/9e83c2aa45949ee9cd6e8731275acdaeb6c624b8728d6598196c65074f3e/ppft-1.6.6.1.tar.gz (63kB)
         |████████████████████████████████| 71kB 6.5MB/s
    Collecting pox>=0.2.7
      Downloading https://files.pythonhosted.org/packages/6c/9a/957818485aa165ce93b646aeb20181215bb415c9dca8345bdbe23b08ae67/pox-0.2.7.tar.gz (110kB)
         |████████████████████████████████| 112kB 49.0MB/s
    Requirement already satisfied: multiprocess>=0.70.9 in /usr/local/lib/python3.6/dist-packages (from pathos->pcml==0.0.1) (0.70.9)
    Requirement already satisfied: cloudpickle>=0.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-probability==0.7.0->pcml==0.0.1) (1.2.2)
    Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.6/dist-packages (from gTTS->pcml==0.0.1) (4.6.3)
    Collecting gtts-token>=1.1.3
      Downloading https://files.pythonhosted.org/packages/e7/25/ca6e9cd3275bfc3097fe6b06cc31db6d3dfaf32e032e0f73fead9c9a03ce/gTTS-token-1.1.3.tar.gz
    Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from gTTS->pcml==0.0.1) (7.0)
    Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (1.11.2)
    Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (1.1.0)
    Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (1.0.8)
    Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (1.1.0)
    Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (0.1.8)
    Collecting tensorboard<1.15.0,>=1.14.0
      Downloading https://files.pythonhosted.org/packages/91/2d/2ed263449a078cd9c8a9ba50ebd50123adf1f8cfbea1492f9084169b89d9/tensorboard-1.14.0-py3-none-any.whl (3.1MB)
         |████████████████████████████████| 3.2MB 45.7MB/s
    Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (0.33.6)
    Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (0.2.2)
    Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0
      Downloading https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488kB)
         |████████████████████████████████| 491kB 40.9MB/s
    Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (0.8.1)
    Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14->pcml==0.0.1) (0.9.0)
    Collecting astroid<2.4,>=2.3.0
      Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)
         |████████████████████████████████| 215kB 37.1MB/s
    Collecting isort<5,>=4.2.5
      Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)
         |████████████████████████████████| 51kB 5.3MB/s
    Collecting mccabe<0.7,>=0.6
      Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
    Requirement already satisfied: more-itertools>=4.0.0 in /usr/local/lib/python3.6/dist-packages (from pytest->pcml==0.0.1) (8.0.2)
    Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from pytest->pcml==0.0.1) (1.8.1)
    Requirement already satisfied: pluggy<0.8,>=0.5 in /usr/local/lib/python3.6/dist-packages (from pytest->pcml==0.0.1) (0.7.1)
    Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.6/dist-packages (from pytest->pcml==0.0.1) (1.3.0)
    Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.6/dist-packages (from pytest->pcml==0.0.1) (19.3.0)
    Collecting execnet>=1.1.dev1
      Downloading https://files.pythonhosted.org/packages/d3/2e/c63af07fa471e0a02d05793c7a56a9f7d274a8489442a5dc4fb3b2b3c705/execnet-1.7.1-py2.py3-none-any.whl
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=1.4.3->seaborn==0.9.0->pcml==0.0.1) (2.4.6)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=1.4.3->seaborn==0.9.0->pcml==0.0.1) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=1.4.3->seaborn==0.9.0->pcml==0.0.1) (1.1.0)
    Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib->kubernetes==9.0.0->pcml==0.0.1) (3.1.0)
    Requirement already satisfied: googleapis-common-protos[grpc]<2.0.0dev,>=1.5.2 in /usr/local/lib/python3.6/dist-packages (from grpc-google-iam-v1<0.13dev,>=0.12.3->google-cloud-bigtable->pcml==0.0.1) (1.6.0)
    Requirement already satisfied: msgpack>=0.5.2 in /usr/local/lib/python3.6/dist-packages (from cachecontrol>=0.12.4->firebase-admin->pcml==0.0.1) (0.5.6)
    Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client>=1.7.8->firebase-admin->pcml==0.0.1) (3.0.1)
    Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client>=1.7.8->firebase-admin->pcml==0.0.1) (0.0.3)
    Requirement already satisfied: httplib2<1dev,>=0.9.2 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client>=1.7.8->firebase-admin->pcml==0.0.1) (0.11.3)
    Collecting google-resumable-media<0.6dev,>=0.5.0
      Downloading https://files.pythonhosted.org/packages/35/9e/f73325d0466ce5bdc36333f1aeb2892ead7b76e79bdb5c8b0493961fa098/google_resumable_media-0.5.0-py2.py3-none-any.whl
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth->pcml==0.0.1) (0.4.8)
    Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.6/dist-packages (from tensorflow-datasets->tensor2tensor==1.14.0->pcml==0.0.1) (0.15.2)
    Requirement already satisfied: promise in /usr/local/lib/python3.6/dist-packages (from tensorflow-datasets->tensor2tensor==1.14.0->pcml==0.0.1) (2.3)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy->tensor2tensor==1.14.0->pcml==0.0.1) (1.1.0)
    Requirement already satisfied: pyglet<=1.3.2,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from gym->tensor2tensor==1.14.0->pcml==0.0.1) (1.3.2)
    Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.14.0->pcml==0.0.1) (1.1.0)
    Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.14.0->pcml==0.0.1) (0.16.0)
    Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.14.0->pcml==0.0.1) (2.10.3)
    Requirement already satisfied: greenlet>=0.4.14; platform_python_implementation == "CPython" in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.14.0->pcml==0.0.1) (0.4.15)
    Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14->pcml==0.0.1) (3.1.1)
    Collecting lazy-object-proxy==1.4.*
      Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)
         |████████████████████████████████| 61kB 5.9MB/s
    Collecting typed-ast<1.5,>=1.4.0; implementation_name == "cpython" and python_version < "3.8"
      Downloading https://files.pythonhosted.org/packages/90/ed/5459080d95eb87a02fe860d447197be63b6e2b5e9ff73c2b0a85622994f4/typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (737kB)
         |████████████████████████████████| 747kB 34.5MB/s
    Collecting apipkg>=1.4
      Downloading https://files.pythonhosted.org/packages/67/08/4815a09603fc800209431bec5b8bd2acf2f95abdfb558a44a42507fb94da/apipkg-1.5-py2.py3-none-any.whl
    Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->flask->tensor2tensor==1.14.0->pcml==0.0.1) (1.1.1)
    Building wheels for collected packages: mne, moviepy, rfc3339, dill, faced, pathos, pytest-cache, proglog, grpc-google-iam-v1, ppft, pox, gtts-token
      Building wheel for mne (setup.py) ... done
      Created wheel for mne: filename=mne-0.17.1-cp36-none-any.whl size=6194555 sha256=a68ecf214aa8252952e86e5d5ccc5024d0816057382437c8285b2ffaa01a8128
      Stored in directory: /root/.cache/pip/wheels/39/f9/f5/3e2fdef321f8e1f64061730c8ba2c81ad55c4b1b860b29d2dd
      Building wheel for moviepy (setup.py) ... done
      Created wheel for moviepy: filename=moviepy-1.0.0-cp36-none-any.whl size=131367 sha256=2dea9fcb335f2641142d820c9a0b6350716444e31b28c049b257a9ce76025223
      Stored in directory: /root/.cache/pip/wheels/52/e2/4c/f594a5945bc98e052ef248b46a0f1f7ea838b0b2a5f8895651
      Building wheel for rfc3339 (setup.py) ... done
      Created wheel for rfc3339: filename=rfc3339-6.0-cp36-none-any.whl size=4502 sha256=45fa5d254302de4b8cc68a457cac4c0130b29ad6f2d4ef82d9069eaddd5b04ee
      Stored in directory: /root/.cache/pip/wheels/40/70/ac/f506969ec90fcf86c2065f6363bc9a1cdcc7d10190b421a9d5
      Building wheel for dill (setup.py) ... done
      Created wheel for dill: filename=dill-0.2.9-cp36-none-any.whl size=77403 sha256=684a102db18bbf6a1503f2b9eba8452188aebd0b432d5b2de0afca1b27e35083
      Stored in directory: /root/.cache/pip/wheels/5b/d7/0f/e58eae695403de585269f4e4a94e0cd6ca60ec0c202936fa4a
      Building wheel for faced (setup.py) ... done
      Created wheel for faced: filename=faced-0.1-cp36-none-any.whl size=27342262 sha256=05705ba11f348cbe6208adc1a5e8312b5f0fa7bc44ded342fe11b3b2abf37aa6
      Stored in directory: /tmp/pip-ephem-wheel-cache-0nxm84d5/wheels/5e/60/d1/7bb207222afc0b7b65100b4b748245d9814567629fb4d80020
      Building wheel for pathos (setup.py) ... done
      Created wheel for pathos: filename=pathos-0.2.5-cp36-none-any.whl size=77575 sha256=3a1e09021c9ea6c9dc7073bb28b0b315900640a77cfccfdc4d4735ced3a52a52
      Stored in directory: /root/.cache/pip/wheels/61/6d/83/90b0c3d2c271da2c4850731e894798c98f8dbedbac74e8eff0
      Building wheel for pytest-cache (setup.py) ... done
      Created wheel for pytest-cache: filename=pytest_cache-1.0-cp36-none-any.whl size=6931 sha256=8f1953543f35ec5cbfb5336c705abfde68e8542b8a11e2d5007ef084c3d6f98e
      Stored in directory: /root/.cache/pip/wheels/e4/9e/28/59d0a23800e49808f17920c5922252c661966ca4a36db5ee38
      Building wheel for proglog (setup.py) ... done
      Created wheel for proglog: filename=proglog-0.1.9-cp36-none-any.whl size=6148 sha256=58d3d828fbef512805d4fb59bc9764ea02fe51561cd357b43fcf8842aff7f5ea
      Stored in directory: /root/.cache/pip/wheels/65/56/60/1d0306a8d90b188af393c1812ddb502a8821b70917f82dcc00
      Building wheel for grpc-google-iam-v1 (setup.py) ... done
      Created wheel for grpc-google-iam-v1: filename=grpc_google_iam_v1-0.12.3-cp36-none-any.whl size=18499 sha256=9f548e176bc1a09c65b111e8a5fe6360bc135046f0b324c20b57586c1a6b8652
      Stored in directory: /root/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b
      Building wheel for ppft (setup.py) ... done
      Created wheel for ppft: filename=ppft-1.6.6.1-cp36-none-any.whl size=64710 sha256=74fae8c419e5f895b44403da742ded7ee5d478e65563ef4ec47fa35248cdeba2
      Stored in directory: /root/.cache/pip/wheels/6a/0c/53/ea8dd63608f75c1e7a64d5c5ce5d1e6d04f15ae8a6fce4c2a0
      Building wheel for pox (setup.py) ... done
      Created wheel for pox: filename=pox-0.2.7-cp36-none-any.whl size=28303 sha256=865daa930ff706376d5f40d95b3f23c9ed11e447c80fcee59fe019d64b17cd8d
      Stored in directory: /root/.cache/pip/wheels/47/10/7b/0e916f6246fe7cf7d3acd25a6d273ecf3f97888cea073a8ac2
      Building wheel for gtts-token (setup.py) ... done
      Created wheel for gtts-token: filename=gTTS_token-1.1.3-cp36-none-any.whl size=4097 sha256=60ca3d94dda4386d523c0e1e96a78435d627ee2f16b32c186099e4714cf26c9f
      Stored in directory: /root/.cache/pip/wheels/dd/11/61/33f7e51bf545e910552b2255eead2a7cd8ef54064b46dceb34
    Successfully built mne moviepy rfc3339 dill faced pathos pytest-cache proglog grpc-google-iam-v1 ppft pox gtts-token
    ERROR: multiprocess 0.70.9 has requirement dill>=0.3.1, but you'll have dill 0.2.9 which is incompatible.
    ERROR: google-cloud-storage 1.25.0 has requirement google-auth<2.0dev,>=1.9.0, but you'll have google-auth 1.4.2 which is incompatible.
    ERROR: google-cloud-storage 1.25.0 has requirement google-cloud-core<2.0dev,>=1.2.0, but you'll have google-cloud-core 1.0.3 which is incompatible.
    ERROR: google-cloud-bigquery 1.21.0 has requirement google-resumable-media!=0.4.0,<0.5.0dev,>=0.3.1, but you'll have google-resumable-media 0.5.0 which is incompatible.
    ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
    ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.
    ERROR: google-cloud-firestore 0.31.0 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 1.0.3 which is incompatible.
    ERROR: firebase-admin 3.2.1 has requirement google-cloud-firestore>=1.4.0; platform_python_implementation != "PyPy", but you'll have google-cloud-firestore 0.31.0 which is incompatible.
    ERROR: pathos 0.2.5 has requirement dill>=0.3.1, but you'll have dill 0.2.9 which is incompatible.
    Installing collected packages: numpy, scipy, mne, proglog, imageio, imageio-ffmpeg, moviepy, websocket-client, kubernetes, rfc3339, dill, grpc-google-iam-v1, google-cloud-bigtable, google-cloud-pubsub, cachecontrol, google-cloud-firestore, google-resumable-media, google-cloud-storage, firebase-admin, requests-toolbelt, tensorboard, tensorflow-estimator, tensorflow, faced, tensor2tensor, google-cloud-logging, ppft, pox, pathos, gtts-token, gTTS, lazy-object-proxy, typed-ast, astroid, isort, mccabe, pylint, apipkg, execnet, pytest-cache, tensorflow-serving-api, pcml
      Found existing installation: numpy 1.17.5
        Uninstalling numpy-1.17.5:
          Successfully uninstalled numpy-1.17.5
      Found existing installation: scipy 1.4.1
        Uninstalling scipy-1.4.1:
          Successfully uninstalled scipy-1.4.1
      Found existing installation: imageio 2.4.1
        Uninstalling imageio-2.4.1:
          Successfully uninstalled imageio-2.4.1
      Found existing installation: moviepy 0.2.3.5
        Uninstalling moviepy-0.2.3.5:
          Successfully uninstalled moviepy-0.2.3.5
      Found existing installation: dill 0.3.1.1
        Uninstalling dill-0.3.1.1:
          Successfully uninstalled dill-0.3.1.1
      Found existing installation: google-resumable-media 0.4.1
        Uninstalling google-resumable-media-0.4.1:
          Successfully uninstalled google-resumable-media-0.4.1
      Found existing installation: google-cloud-storage 1.16.2
        Uninstalling google-cloud-storage-1.16.2:
          Successfully uninstalled google-cloud-storage-1.16.2
      Found existing installation: tensorboard 1.15.0
        Uninstalling tensorboard-1.15.0:
          Successfully uninstalled tensorboard-1.15.0
      Found existing installation: tensorflow-estimator 1.15.1
        Uninstalling tensorflow-estimator-1.15.1:
          Successfully uninstalled tensorflow-estimator-1.15.1
      Found existing installation: tensorflow 1.15.0
        Uninstalling tensorflow-1.15.0:
          Successfully uninstalled tensorflow-1.15.0
      Found existing installation: tensor2tensor 1.14.1
        Uninstalling tensor2tensor-1.14.1:
          Successfully uninstalled tensor2tensor-1.14.1
      Running setup.py develop for pcml
    Successfully installed apipkg-1.5 astroid-2.3.3 cachecontrol-0.12.6 dill-0.2.9 execnet-1.7.1 faced-0.1 firebase-admin-3.2.1 gTTS-2.1.1 google-cloud-bigtable-1.2.1 google-cloud-firestore-0.31.0 google-cloud-logging-1.14.0 google-cloud-pubsub-1.1.0 google-cloud-storage-1.25.0 google-resumable-media-0.5.0 grpc-google-iam-v1-0.12.3 gtts-token-1.1.3 imageio-2.6.1 imageio-ffmpeg-0.3.0 isort-4.3.21 kubernetes-9.0.0 lazy-object-proxy-1.4.3 mccabe-0.6.1 mne-0.17.1 moviepy-1.0.0 numpy-1.16.2 pathos-0.2.5 pcml pox-0.2.7 ppft-1.6.6.1 proglog-0.1.9 pylint-2.4.4 pytest-cache-1.0 requests-toolbelt-0.9.1 rfc3339-6.0 scipy-1.2.1 tensor2tensor-1.14.0 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-estimator-1.14.0 tensorflow-serving-api-1.12.0 typed-ast-1.4.1 websocket-client-0.57.0

```
# Dependency needed for FX prototypes that involved synthesized speech.
!pip install gtts
```

    Requirement already satisfied: gtts in /usr/local/lib/python3.6/dist-packages (2.1.1)
    Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.6/dist-packages (from gtts) (4.6.3)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from gtts) (1.12.0)
    Requirement already satisfied: gtts-token>=1.1.3 in /usr/local/lib/python3.6/dist-packages (from gtts) (1.1.3)
    Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from gtts) (7.0)
    Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from gtts) (2.21.0)
    Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->gtts) (1.24.3)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->gtts) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->gtts) (2019.11.28)
    Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->gtts) (2.8)

##### TFServing dependencies

This is only relevant for the prototypes that query a model served locally with TFServing.

```
!sh /content/clarify/tools/serving/install_tf_serving.sh
```

##### Model checkpoint

TODO: Currently this needs to be obtained from cloud storage following authentication because for whatever reason there is an auth error when the GCS checkpoint location is referenced directly despite that being publicly-accessible

```
# 1. Log into your GCloud account by running the following, clicking the
# link, authenticating, and providing the resulting code in the box below.
!gcloud auth login
```

```
# 2. Next, download the model parameters locally (instead of relying on
# tensorflow to propperly manage credentials to fetch these itself).
!gsutil -m cp -r gs://clarify-public/models/fec-train-j1030-0136-3a8f/output /tmp/
```

    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572401042.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572400837.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572401247.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572400018.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/checkpoint...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572400222.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572401450.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572400635.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572400430.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572401655.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572401860.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572402065.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572402265.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572402469.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572403083.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572402875.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572402672.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572403291.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572403498.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572403706.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572403908.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572404317.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572404113.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572404521.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572404932.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572404727.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572405138.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572405344.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572405552.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572405758.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572405963.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572406168.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572406372.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572406576.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572406782.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572406989.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572407197.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572407404.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572407607.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572407808.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572408012.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572408215.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572408422.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572408629.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572408835.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572409043.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572409249.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572409457.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572409666.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572409874.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572410084.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572410294.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572410500.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572410705.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572410910.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572411112.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572411316.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572411521.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572411723.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572411926.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572412130.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572412334.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572412539.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572412744.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572412947.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572413150.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572413357.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572413559.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572413764.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572413968.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572414173.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572414378.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572414581.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572414788.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572414992.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572415195.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572415399.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572415604.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572415807.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572416014.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572416220.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572416422.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572416627.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572416834.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572417035.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572417236.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572417442.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572417645.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572417846.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572418053.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572418253.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572418461.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572418665.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572418868.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572419070.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572419273.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572419476.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572419681.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572419887.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572420092.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572420296.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572420502.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572420707.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572420912.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572421116.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572421322.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572421530.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572421734.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572421939.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572422142.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572422346.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572422550.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572422756.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572422961.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572423164.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572423367.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572423570.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572423772.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572423976.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572424180.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572424384.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572424591.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572424794.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572424999.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572425205.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572425410.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572425614.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572425820.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572426025.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572426232.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572426441.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572426648.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572426854.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572427060.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572427265.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572427470.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572427676.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572427878.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572428083.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572428290.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572428496.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572428905.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572428699.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572429106.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572429312.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572429518.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572429725.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572429930.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572430138.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572430343.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572430550.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572430758.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572430963.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572431168.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572431373.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572431577.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572431780.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572431983.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572432188.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572432394.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572432598.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572432804.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572433008.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572433211.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572433415.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572433619.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572434025.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572433821.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572434230.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572434435.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572434638.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572434843.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572435049.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572435255.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572435459.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572435663.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572435865.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572436069.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572436272.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572436477.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572436682.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572436885.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572437089.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572437294.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572437498.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572437703.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572437909.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572438116.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572438322.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572438527.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572438732.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572438937.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572439141.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572439345.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572439547.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572439753.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572439959.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572440166.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572440372.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572440578.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572440784.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572440988.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572441196.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572441400.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572441609.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572441814.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572442022.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572442228.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572442434.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572442639.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572442846.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572443050.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572443256.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572443461.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572443666.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572443872.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572444077.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572444285.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572444493.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572444696.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572444905.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572445113.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572445319.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572445526.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572445735.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572445942.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572446150.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572446359.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572446567.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572446773.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572446981.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572447184.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572447393.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572447598.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572447803.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572448011.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572448219.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572448425.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572448630.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572448836.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572449040.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572449247.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572449453.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572449668.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572449873.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572450078.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572450284.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572450490.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572450696.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572450900.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572451106.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572451313.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572451522.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572451730.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572451936.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572452144.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572452352.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572452559.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572452764.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572452973.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572453181.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572453387.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572453596.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572453803.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572454010.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572454424.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572454219.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572454630.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572454837.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572455045.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572455249.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572455455.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572455660.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572455867.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572456073.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572456279.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572456485.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572456693.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572456898.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572457102.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572457310.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572457517.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572457723.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572457930.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572458137.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572458343.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572458547.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572458752.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572458955.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572459162.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572459368.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572459574.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572459779.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572459984.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572460189.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572460392.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572460598.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572460802.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572461006.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572461211.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572461417.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572461622.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572461826.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572462030.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572462235.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572462440.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572462646.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572462853.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572463059.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572463263.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572463466.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572463672.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572463878.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572464082.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572464285.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572464489.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572464694.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572464899.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572465105.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572465306.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572465513.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572465716.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572465919.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572466125.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572466536.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572466330.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572466742.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572466946.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572467151.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572467356.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572467562.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572467769.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572467977.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572468182.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572468387.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572468594.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572468803.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572469008.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572469217.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572469424.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572469631.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572469838.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572470043.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572470249.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572470456.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572470660.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572470864.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572471071.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572471277.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572471481.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572471688.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572471895.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572472099.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572472307.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572472512.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572472718.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572472927.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572473135.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572473340.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572473547.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572473758.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572473974.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572474191.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572474410.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572474628.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572474843.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572475063.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572475279.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572475498.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572475713.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572475932.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572476146.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572476365.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572476582.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572476797.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572477017.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572477233.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572477448.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572477662.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572477878.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572478095.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572478311.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572478528.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572478745.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572478962.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572479178.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572479397.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572479614.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572479829.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572480047.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572480261.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572480474.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572480690.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572480904.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572481120.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572481333.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572481548.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572481761.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572481974.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572482188.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572482401.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572482615.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572482832.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572483046.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572483262.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572483475.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572483690.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572483904.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572484119.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572484333.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572484547.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572484763.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572485186.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572484974.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572485400.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572485615.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572485820.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572486032.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/eval/events.out.tfevents.1572486248.fec-train-j1030-0136-3a8f-master-0.v2...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/events.out.tfevents.1572399668.fec-train-j1030-0136-3a8f-master-0...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1574360147/saved_model.pbtxt...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1574360147/variables/variables.data-00000-of-00002...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1574360147/variables/variables.data-00001-of-00002...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1574360147/variables/variables.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1575913483/saved_model.pbtxt...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1575913483/variables/variables.data-00000-of-00002...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1575913483/variables/variables.data-00001-of-00002...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/export/1575913483/variables/variables.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/flags.txt...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/flags_t2t.txt...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/graph.pbtxt...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/hparams.json...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-172400.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-172400.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-172400.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-173200.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-173200.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-173200.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174000.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174000.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174000.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174800.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174800.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-174800.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-175600.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-175600.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-175600.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-176300.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-176300.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-176300.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177100.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177100.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177100.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177900.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177900.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-177900.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-178700.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-178700.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-178700.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-179500.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-179500.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-179500.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-180300.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-180300.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-180300.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181100.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181100.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181100.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181900.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181900.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-181900.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-182700.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-182700.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-182700.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-183400.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-183400.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-183400.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-184200.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-184200.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-184200.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185000.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185000.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185000.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185800.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185800.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-185800.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-186600.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-186600.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-186600.meta...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-187300.data-00000-of-00001...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-187300.index...
    Copying gs://clarify-public/models/fec-train-j1030-0136-3a8f/output/model.ckpt-187300.meta...
    / [492/492 files][  2.3 GiB/  2.3 GiB] 100% Done  94.5 MiB/s ETA 00:00:00
    Operation completed over 492 objects/2.3 GiB.

##### FX-specific dependencies

```
# TODO: We can install it but having trouble importing it
!npm install -g lit-element
```

    + lit-element@2.2.1
    added 2 packages from 1 contributor in 0.551s

##### Eager mode

Inference in tf.Eager mode is being phased out but for now it's important to initialize eager mode at startup if it's going to be used.

```
%tensorflow_version 1.x
import tensorflow as tf

tfe = tf.contrib.eager
tfe.enable_eager_execution()
Modes = tf.estimator.ModeKeys

```

##### Shared functions for image capture

```
from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode
import base64

import io
import numpy as np

from PIL import Image

from faced import FaceDetector
from faced.utils import annotate_image
import copy

import cv2

from pcml.datasets import image_aug

def take_photo(filename='photo.jpg', quality=0.8):
  js = Javascript('''
    async function takePhoto(quality) {
      const div = document.createElement('div');

      const video = document.createElement('video');
      video.style.display = 'block';
      const stream = await navigator.mediaDevices.getUserMedia({
          video: { width: 256, height: 128 }
      });

      document.body.appendChild(div);
      div.appendChild(video);
      video.srcObject = stream;
      video.style.visibility = "hidden";
      await video.play();

      // Resize the output to fit the video element.
      google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);

      const canvas = document.createElement('canvas');
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
      canvas.getContext('2d').drawImage(video, 0, 0);
      stream.getVideoTracks()[0].stop();

      data = canvas.toDataURL('image/jpeg', quality);

      div.remove();
      canvas.remove();
      video.remove();

      return data
    }
    ''')
  display(js)
  return eval_js('takePhoto({})'.format(quality))

def load_image(img_string):
  binary = b64decode(img_string.split(',')[1])
  filename = "/tmp/stream.jpeg"
  with open(filename, 'wb') as f:
    f.write(binary)
  img = Image.open(filename)
  return np.array(img)

def _random_crop_square(image):

  x,y,c = image.shape

  x_crop_before = 0
  x_crop_after = 0
  y_crop_before = 0
  y_crop_after = 0

  if x > y:
    x_crop = x - y
    x_crop_before = np.random.randint(0, x_crop)
    x_crop_after = x_crop - x_crop_before
  elif y > x:
    y_crop = y - x
    y_crop_before = np.random.randint(0, y_crop)
    y_crop_after = y_crop - y_crop_before

  x_start = x_crop_before
  x_end = x - x_crop_after
  y_start = y_crop_before
  y_end = y - y_crop_after

  return image[x_start:x_end, y_start:y_end, :]

def _normalize_dimensions(image, target_shape):

  image = _random_crop_square(image)

  mn, mx = np.amin(image), np.amax(image)
  if mn >=0 and mx <= 255:
    image = image / 255.0

  source_shape = image.shape
  scale_x_factor = target_shape[0]/source_shape[0]
  scale_y_factor = target_shape[1]/source_shape[1]
  scale_x_first = (scale_x_factor <= scale_y_factor)

  if scale_x_first:

    new_x = target_shape[0]
    new_y = int(source_shape[1]*scale_x_factor)
    resize_dim = (new_x, new_y)
    newimg = cv2.resize(image, resize_dim)
    pad_width = target_shape[1] - new_y
    if pad_width > 0:
      # Pad in Y direction
      newimg = np.pad(newimg, [(0,pad_width),(0,0),(0,0)], mode="mean")

  else:

    new_y = target_shape[1]
    new_x = int(source_shape[0]*scale_y_factor)
    resize_dim = (new_x, new_y)
    newimg = cv2.resize(image, resize_dim)
    pad_width = target_shape[0] - new_x
    if pad_width > 0:
      # Pad in X direction
      newimg = np.pad(newimg, [(0,0),(0,pad_width),(0,0)], mode="mean")

  newimg = (newimg*255.0).astype(np.int64)

  return newimg

def detect_and_preprocess(image):

  detector = FaceDetector()
  detect_threshold = 0.5
  predictions = detector.predict(image, detect_threshold)

  xcenter = predictions[0][0]
  ycenter = predictions[0][1]
  width = predictions[0][2]*1.80
  height = predictions[0][3]*1.80

  xmax = image.shape[1]
  ymax = image.shape[0]

  ystart = max(0,int(ycenter-height/2))
  yend = min(ymax,int(ycenter+height/2))
  xstart = max(0,int(xcenter-width/2))
  xend = min(xmax,int(xcenter+width/2))

  img_with_face = image[ystart:yend,xstart:xend,:]

  image_shape = (128,128,3)
  img_post = _normalize_dimensions(img_with_face, target_shape=image_shape)

  return img_post

```

The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.

We recommend you upgrade now
or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic:
more info.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/expert_utils.py:68: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

    WARNING:tensorflow:
    The TensorFlow contrib module will not be included in TensorFlow 2.0.
    For more information, please see:
      * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
      * https://github.com/tensorflow/addons
      * https://github.com/tensorflow/io (for I/O related ops)
    If you depend on functionality not listed there, please file an issue.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/adafactor.py:27: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/multistep_optimizer.py:32: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/glow_init_hook.py:25: The name tf.train.SessionRunHook is deprecated. Please use tf.estimator.SessionRunHook instead.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/neural_stack.py:38: The name tf.nn.rnn_cell.RNNCell is deprecated. Please use tf.compat.v1.nn.rnn_cell.RNNCell instead.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/gym_utils.py:235: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

    INFO:tensorflow:Entry Point [tensor2tensor.envs.tic_tac_toe_env:TicTacToeEnv] registered with id [T2TEnv-TicTacToeEnv-v0]
    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/trainer_lib.py:111: The name tf.OptimizerOptions is deprecated. Please use tf.compat.v1.OptimizerOptions instead.

    WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_gan/python/estimator/tpu_gan_estimator.py:42: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead.

    WARNING:tensorflow:From /content/clarify/pcml/models/dev.py:35: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

    WARNING:tensorflow:From /content/clarify/pcml/models/dev.py:35: The name tf.logging.DEBUG is deprecated. Please use tf.compat.v1.logging.DEBUG instead.

# Prototypes

##### Basic JS in colab

```javascript
%%javascript
function dev() {
  return 1
}
function dev2() {
  return dev()
}
console.log(dev2())
```

    <IPython.core.display.Javascript object>

##### Basic HTMLElement subclass, HTTP request

```
# Display a basic HTMLElement subclass that makes an HTTP request

import IPython
display(IPython.display.HTML('''

  <head>

  </head>

  <body>
    <fx-component></fx-component>
  </body>

'''))
```

  <fx-component></fx-component>

###### LitElement in Colab

Here are some docs to get started:
- https://lit-element.polymer-project.org/
- https://lit-element.polymer-project.org/try

```
display(IPython.display.HTML('''

  <head>

  </head>

  <body>
    <fx-component></fx-component>
  </body>

'''))
```

##### Polymer component in colab

For debugging tried Polymer < 3.0. This does not currently work.

```
import IPython
display(IPython.display.HTML('''

  <head>

  <link rel="import"  href="https://polygit.org/polymer+^1.9.1/webcomponentsjs+^0.7.0/components/polymer/polymer.html">

  </head>

  <body>
    <proto-element></proto-element>
  </body>

'''))

```

##### Call python model from JS

Towards taking the place of a remote HTTP query in the interest of simplicity while developing new FX components.

```
import IPython
from google.colab import output

def infer_state_from_image(input_tensor):
  # Use display.JSON to transfer a structured result.
  return IPython.display.JSON({'result': 1234})

output.register_callback('notebook.infer_state_from_image', infer_state_from_image)

```

```javascript

%%javascript
(async function() {

  const result = await google.colab.kernel.invokeFunction(
    'notebook.infer_state_from_image', // The callback name.
    ['image_data'], // The arguments.
    {}); // kwargs

  const text = result.data['application/json'];

  document.querySelector("#output-area").appendChild(document.createTextNode(text.result));

})();

```

    <IPython.core.display.Javascript object>

##### Faster js -> python image capture

This was the method used in the first notebook that was built and was somewhat slow. It could be sped up, perhaps by not removing and recreating media elements or we could opt for a different way of interoperating between python and js.

```
from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode
import cv2
import numpy as np

import matplotlib
%matplotlib inline

from matplotlib.pyplot import imshow

def get_frame(filename='photo.jpg', quality=0.8):
  """Some description of your FX demo."""

  js = Javascript('''

    async function takePhoto(quality=1234) {

      const div = document.createElement('div');

      const video = document.createElement('video');
      video.style.display = 'block';
      const stream = await navigator.mediaDevices.getUserMedia({video: true});

      document.body.appendChild(div);
      div.appendChild(video);
      video.srcObject = stream;
      await video.play();

      // Resize the output to fit the video element.
      google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);

      const canvas = document.createElement('canvas');
      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;
      canvas.getContext('2d').drawImage(video, 0, 0);
      stream.getVideoTracks()[0].stop();
      div.remove();
      return canvas.toDataURL('image/jpeg', quality);
    }
    ''')

  display(js)

  data = eval_js('takePhoto({})'.format(1234))

  data = b64decode(data.split(',')[1])

  img = cv2.imdecode(np.frombuffer(data, dtype=np.uint8),
                     cv2.IMREAD_COLOR)
  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

  return img

data = get_frame()

imshow(data)

```

##### Eager-mode inference

Code for the previous demo that did inference in TensorFlow Eager mode. Requires the runtime to be re-set each time the model is re-stored from a checkpoint.

```
%tensorflow_version 1.x
import tensorflow as tf

tfe = tf.contrib.eager
tfe.enable_eager_execution()
Modes = tf.estimator.ModeKeys

from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode
import base64

import io
import numpy as np

from PIL import Image

from faced import FaceDetector
from faced.utils import annotate_image
import copy

import cv2

import matplotlib
%matplotlib inline

from matplotlib import pyplot as plt

from pcml.models import percep_similarity_emb

from tensor2tensor.utils import registry

from pcml.datasets import image_aug

from tensor2tensor.serving import serving_utils

from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc
from pcml.utils.dev_utils import run_server, wait_for_server_ready

from gtts import gTTS
from IPython.display import (
    Audio, display, clear_output)
import time

problem_name = "facial_expression_correspondence"
model_name = "percep_similarity_triplet_emb"
hparams_set_name = "percep_similarity_triplet_emb"
mode = "predict"
ckpt_dir = "/tmp/output"
data_dir = ckpt_dir
export_dir = "/tmp/output/export"

hparams = registry.hparams(hparams_set_name)
hparams.data_dir = data_dir

problem_obj = registry.problem(problem_name)

p_hparams = problem_obj.get_hparams(hparams)

model_obj = registry.model(model_name)

def get_example():

  image_stats = {"mean": [0.330, 0.537, -0.242], "sd": [0.220, 0.169, 1.156]}
  shape = (64,64,3)
  mode = "eval"

  img = np.asarray((load_image(take_photo())))

  img = detect_and_preprocess(img)

  # Convert to int32

  example = {
    "image/a": img,
    "image/b": img,
    "image/c": img,
    "image/a/noaug": img,
    "image/b/noaug": img,
    "image/c/noaug": img,
    "triplet_code": [0],
    "type": [1],
    "targets": img
  }

  def _preproc(image):

    image = image_aug.preprocess_image(
      image, mode,
      resize_size=shape,
      normalize=True,
      image_statistics=image_stats,
      crop_area_min=1,
      contrast_lower=0.45,
      contrast_upper=0.55,
      brightness_delta_min=-0.01,
      brightness_delta_max=0.01)

    image.set_shape(shape)

    return image

  example["image/a"] = tf.expand_dims(_preproc(example["image/a"]),0)
  example["image/b"] = tf.expand_dims(_preproc(example["image/b"]),0)
  example["image/c"] = tf.expand_dims(_preproc(example["image/c"]),0)
  example["triplet_code"] = tf.expand_dims(tf.cast(example["triplet_code"], tf.int64),0)

  return example

```

```

def _goal_from_goals(goals):
  return goals[0]

def _tts(msg):
  filename = "/tmp/clarify-tmp.mp3"
  tts = gTTS(msg)
  tts.save(filename)
  display(Audio(filename, autoplay=True))

def play_instructions():
  message = "Goal definition will begin in twenty seconds following a tone and will "
  message += "last for ten seconds. Please assume a mental state that you want to "
  message += "be your goal. The definition period will begin in seven seconds."
  _tts(message)
  time.sleep(20)
  synth(600)

def play_beginning_session():
  _tts("Goal definition complete. Beginning session.")
  time.sleep(3)

def play_establishing_baseline():
  _tts("Establishing baseline. Please demonstrate a variety of expressions and poses.")
  time.sleep(3)

def play_baseline_complete():
  _tts("Finished establishing baseline.")
  time.sleep(3)

def synth(f):
  rate = 16000.
  duration = .25
  t = np.linspace(
    0., duration, int(rate * duration))

  x = np.sin(f * 2. * np.pi * t)
  display(Audio(x, rate=rate, autoplay=True))

def optimize():

  play_instructions()

  query_data = {}

  synth_min = 200
  synth_max = 600
  synth_range = synth_max - synth_min

  mn = None
  mx = None

  goals = []
  goal = None

  distance_threshold = 1.3

  num_sampled = 0

  baseline = True

  distances = []
  examples = []
  num_baseline_steps = 0

  with tfe.restore_variables_on_create(tf.train.latest_checkpoint(ckpt_dir)):

    model = model_obj(hparams, mode, p_hparams)

    while True:

      print("Doing step...")

      try:

        example = get_example()
        current, _ = model(example)

        num_sampled += 1

        if num_sampled <= 3:

          goals.append(current)
          distances.append(0)
          examples.append(example)

        else:

          if goal is None:
            goal = _goal_from_goals(goals)
            play_beginning_session()

          dist = np.linalg.norm(current - goal)
          print(dist)

          distances.append(dist)
          examples.append(example)

          if not mn:
            mn = dist
          if not mx:
            mx = dist

          if dist < mn:
            mn = dist

          if dist > mx:
            mx = dist

          if num_sampled == 4:
            play_establishing_baseline()

          synth_level = synth_min + synth_range*((dist-mn)/(mx-mn))

          #if distances:
          #  distance_threshold = np.mean(distances)

          print(synth_level)
          if dist > distance_threshold:
            synth(synth_level)

          if num_sampled == 20:
            play_baseline_complete()
            baseline = False
            num_baseline_steps = num_sampled

      except (KeyboardInterrupt, SystemExit):
        break

      except:
        raise
        print("there was an exception but we're not worried ;D")

  return locals()
```

```
optimize()
```

Critical assessment:

* It seems useful representations of goal vs. non-goal states are being made and there is a strong technical foundation for building state-based feedback experiences.
* This particular form of feedback experience can be improved in various ways. For one, it's distracting. Tones are uncomfortable and irregular (perhaps better to have these be continuous). Also, when a tone is played it interrupts music playing by temporarily turning down its volume - instead, regulation of system volume could itself be the feedback cue (i.e. on-target is desired volume).

##### Query locally served model

TODO: This is just temporary and to avoid having to restart the runtime for each run of the demo.

###### Run the server

Needs load_image and take_photo from eager section above

```
# Make sure you have actually obtained the checkpoint
!ls /tmp/output | wc -l
```

    68

```

import numpy as np

from pcml.utils.dev_utils import run_server, wait_for_server_ready

from pcml.models import percep_similarity_emb
from tensor2tensor.serving import serving_utils

from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc

problem_name = "facial_expression_correspondence"
model_name = "percep_similarity_triplet_emb"
hparams_set_name = "percep_similarity_triplet_emb"
mode = "predict"
ckpt_dir = "/tmp/output"
data_dir = ckpt_dir
export_dir = "/tmp/output/export"

tfms_path = "/usr/bin/tensorflow_model_server"
!ls {tfms_path}

_, server, _ = run_server(
    model_name,
    export_dir,
    tfms_path
)

wait_for_server_ready(server)

def prepare_request():

  img = np.asarray((load_image(take_photo())))

  img = detect_and_preprocess(img)

  image_feature = tf.train.Feature(int64_list=tf.train.Int64List(value=img.flatten().tolist()))

  ex = tf.train.Example(features=tf.train.Features(feature={
    "image/a": image_feature,
    "image/b": image_feature,
    "image/c": image_feature,
    "triplet_code": tf.train.Feature(int64_list=tf.train.Int64List(value=[0] * 1)),
    "type": tf.train.Feature(int64_list=tf.train.Int64List(value=[0] * 1)),
    "rating/mode": tf.train.Feature(int64_list=tf.train.Int64List(value=[0] * 1)),
    "targets": tf.train.Feature(int64_list=tf.train.Int64List(value=[0] * 1))
  }))

  return ex

def single_served_model_request(ex, servable_name):

  proto = tf.make_tensor_proto([ex.SerializeToString()], shape=[1])

  request = predict_pb2.PredictRequest()
  request.model_spec.name = model_name
  request.inputs["input"].CopyFrom(proto)

  response = stub.Predict(request, timeout_secs)
  outputs = tf.make_ndarray(response.outputs["outputs"])
  return outputs[0]

timeout_secs = 5
stub = serving_utils._create_stub(server)

```

###### Make requests

```
# Make an expression that will be the goal
goal = single_served_model_request(prepare_request(), model_name)
```

```
# Collect a series of on-target images
on_target = []
for i in range(20):
  on_target.append(single_served_model_request(prepare_request(), model_name))
```

```
# Collect a series of off-target images
off_target = []
for i in range(20):
  off_target.append(single_served_model_request(prepare_request(), model_name))
```

```
import numpy as np

# The distances off-target should be larger than on-target

print("on-target")
tot_on = 0
for emb in on_target:
  d = np.linalg.norm(goal-emb)
  tot_on += d
  print(d)
print("on target mean distance: {}".format(tot_on/len(on_target)))

print("\n")

print("off-target")
tot_off = 0
for emb in off_target:
  d = np.linalg.norm(goal-emb)
  tot_off += d
  print(d)
print("on target mean distance: {}".format(tot_off/len(off_target)))

```