## Install lib

```bash
%%bash
rm -rf /usr/local/cuda
ln -s /usr/local/cuda-10.1 /usr/local/cuda
```

```python
!nvcc --version
```

    nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2019 NVIDIA Corporation
    Built on Sun_Jul_28_19:07:16_PDT_2019
    Cuda compilation tools, release 10.1, V10.1.243

```python
!stat /usr/local/cuda
```

      File: /usr/local/cuda -> /usr/local/cuda-10.1
      Size: 20        	Blocks: 0          IO Block: 4096   symbolic link
    Device: 31h/49d	Inode: 3154263     Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2021-11-25 15:11:38.100786781 +0000
    Modify: 2021-11-25 15:11:37.221787620 +0000
    Change: 2021-11-25 15:11:37.221787620 +0000
     Birth: -

```python
!pip uninstall -y triton
```

    Found existing installation: triton 0.2.3
    Uninstalling triton-0.2.3:
      Successfully uninstalled triton-0.2.3

```python
!pip uninstall -y torch
```

    Found existing installation: torch 1.6.0+cu101
    Uninstalling torch-1.6.0+cu101:
      Successfully uninstalled torch-1.6.0+cu101

```bash
%%bash
export LD_LIBRARY_PATH=/usr/lib/
```

```python
!apt-get install clang-9 llvm-9 llvm-9-dev llvm-9-tools
```

    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    clang-9 is already the newest version (1:9-2~ubuntu18.04.2).
    llvm-9 is already the newest version (1:9-2~ubuntu18.04.2).
    llvm-9-dev is already the newest version (1:9-2~ubuntu18.04.2).
    llvm-9-tools is already the newest version (1:9-2~ubuntu18.04.2).
    0 upgraded, 0 newly installed, 0 to remove and 37 not upgraded.

```python
!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
```

    Looking in links: https://download.pytorch.org/whl/torch_stable.html
    Collecting torch==1.6.0+cu101
      Using cached https://download.pytorch.org/whl/cu101/torch-1.6.0%2Bcu101-cp37-cp37m-linux_x86_64.whl (708.0 MB)
    Requirement already satisfied: torchvision==0.7.0+cu101 in /usr/local/lib/python3.7/dist-packages (0.7.0+cu101)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torch==1.6.0+cu101) (1.18.2)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from torch==1.6.0+cu101) (0.16.0)
    Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.7/dist-packages (from torchvision==0.7.0+cu101) (7.1.2)
    Installing collected packages: torch
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    deepspeed 0.3.7 requires triton>=0.2.2, which is not installed.
    torchtext 0.11.0 requires torch==1.10.0, but you have torch 1.6.0+cu101 which is incompatible.
    torchaudio 0.10.0+cu111 requires torch==1.10.0, but you have torch 1.6.0+cu101 which is incompatible.
    Successfully installed torch-1.6.0+cu101

```python
%%writefile setup.sh

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
```

    Overwriting setup.sh

```python
!sh setup.sh
```

    fatal: destination path 'apex' already exists and is not an empty directory.
    /usr/local/lib/python3.7/dist-packages/pip/_internal/commands/install.py:232: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
      cmdoptions.check_install_build_global(options)
    Using pip 21.1.3 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)
    Value for scheme.platlib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/lib/python3.7/dist-packages
    sysconfig: /usr/lib/python3.7/site-packages
    Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/lib/python3.7/dist-packages
    sysconfig: /usr/lib/python3.7/site-packages
    Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/include/python3.7/UNKNOWN
    sysconfig: /usr/include/python3.7m/UNKNOWN
    Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/bin
    sysconfig: /usr/bin
    Value for scheme.data does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local
    sysconfig: /usr
    Additional context:
    user = False
    home = None
    root = None
    prefix = None
    Non-user install because site-packages writeable
    Created temporary directory: /tmp/pip-ephem-wheel-cache-hvi085dy
    Created temporary directory: /tmp/pip-req-tracker-pgfywie_
    Initialized build tracking at /tmp/pip-req-tracker-pgfywie_
    Created build tracker: /tmp/pip-req-tracker-pgfywie_
    Entered build tracker: /tmp/pip-req-tracker-pgfywie_
    Created temporary directory: /tmp/pip-install-_5lnz86r
    Processing /content/apex
      Created temporary directory: /tmp/pip-req-build-41vu3jcb
      DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
       pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
      Added file:///content/apex to build tracker '/tmp/pip-req-tracker-pgfywie_'
        Running setup.py (path:/tmp/pip-req-build-41vu3jcb/setup.py) egg_info for package from file:///content/apex
        Created temporary directory: /tmp/pip-pip-egg-info-cyzipaqb
        Running command python setup.py egg_info

        torch.__version__  = 1.6.0+cu101

        running egg_info
        creating /tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info
        writing /tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info/PKG-INFO
        writing dependency_links to /tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info/dependency_links.txt
        writing top-level names to /tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info/top_level.txt
        writing manifest file '/tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info/SOURCES.txt'
        adding license file 'LICENSE'
        writing manifest file '/tmp/pip-pip-egg-info-cyzipaqb/apex.egg-info/SOURCES.txt'
        /tmp/pip-req-build-41vu3jcb/setup.py:67: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies!
          warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")
      Source in /tmp/pip-req-build-41vu3jcb has version 0.1, which satisfies requirement apex==0.1 from file:///content/apex
      Removed apex==0.1 from file:///content/apex from build tracker '/tmp/pip-req-tracker-pgfywie_'
    Created temporary directory: /tmp/pip-unpack-77j1xid4
    Skipping wheel build for apex, due to binaries being disabled for it.
    Installing collected packages: apex
      Attempting uninstall: apex
        Found existing installation: apex 0.1
        Uninstalling apex-0.1:
          Created temporary directory: /tmp/pip-uninstall-1fd81op5
          Removing file or directory /usr/local/lib/python3.7/dist-packages/amp_C.cpython-37m-x86_64-linux-gnu.so
          Created temporary directory: /usr/local/lib/python3.7/dist-packages/~pex-0.1-py3.7.egg-info
          Removing file or directory /usr/local/lib/python3.7/dist-packages/apex-0.1-py3.7.egg-info
          Created temporary directory: /usr/local/lib/python3.7/dist-packages/~pex
          Removing file or directory /usr/local/lib/python3.7/dist-packages/apex/
          Removing file or directory /usr/local/lib/python3.7/dist-packages/apex_C.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/fused_dense_cuda.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/fused_layer_norm_cuda.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/mlp_cuda.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/scaled_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/scaled_upper_triang_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so
          Removing file or directory /usr/local/lib/python3.7/dist-packages/syncbn.cpython-37m-x86_64-linux-gnu.so
          Successfully uninstalled apex-0.1
      Value for scheme.platlib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
      distutils: /usr/local/lib/python3.7/dist-packages
      sysconfig: /usr/lib/python3.7/site-packages
      Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
      distutils: /usr/local/lib/python3.7/dist-packages
      sysconfig: /usr/lib/python3.7/site-packages
      Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
      distutils: /usr/local/include/python3.7/apex
      sysconfig: /usr/include/python3.7m/apex
      Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
      distutils: /usr/local/bin
      sysconfig: /usr/bin
      Value for scheme.data does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
      distutils: /usr/local
      sysconfig: /usr
      Additional context:
      user = False
      home = None
      root = None
      prefix = None
      Created temporary directory: /tmp/pip-record-xcqqv22a
        Running command /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-41vu3jcb/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-41vu3jcb/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --cpp_ext --cuda_ext install --record /tmp/pip-record-xcqqv22a/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7/apex

        torch.__version__  = 1.6.0+cu101

        /tmp/pip-req-build-41vu3jcb/setup.py:67: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies!
          warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")

        Compiling cuda extensions with
        nvcc: NVIDIA (R) Cuda compiler driver
        Copyright (c) 2005-2019 NVIDIA Corporation
        Built on Sun_Jul_28_19:07:16_PDT_2019
        Cuda compilation tools, release 10.1, V10.1.243
        from /usr/local/cuda/bin

        running install
        running build
        running build_py
        creating build
        creating build/lib.linux-x86_64-3.7
        creating build/lib.linux-x86_64-3.7/apex
        copying apex/__init__.py -> build/lib.linux-x86_64-3.7/apex
        copying apex/_autocast_utils.py -> build/lib.linux-x86_64-3.7/apex
        creating build/lib.linux-x86_64-3.7/apex/multi_tensor_apply
        copying apex/multi_tensor_apply/__init__.py -> build/lib.linux-x86_64-3.7/apex/multi_tensor_apply
        copying apex/multi_tensor_apply/multi_tensor_apply.py -> build/lib.linux-x86_64-3.7/apex/multi_tensor_apply
        creating build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/__init__.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/__version__.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/compat.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/_initialize.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/handle.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/wrap.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/rnn_compat.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/_process_optimizer.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/frontend.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/opt.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/scaler.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/_amp_state.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/utils.py -> build/lib.linux-x86_64-3.7/apex/amp
        copying apex/amp/amp.py -> build/lib.linux-x86_64-3.7/apex/amp
        creating build/lib.linux-x86_64-3.7/apex/contrib
        copying apex/contrib/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib
        creating build/lib.linux-x86_64-3.7/apex/mlp
        copying apex/mlp/__init__.py -> build/lib.linux-x86_64-3.7/apex/mlp
        copying apex/mlp/mlp.py -> build/lib.linux-x86_64-3.7/apex/mlp
        creating build/lib.linux-x86_64-3.7/apex/normalization
        copying apex/normalization/__init__.py -> build/lib.linux-x86_64-3.7/apex/normalization
        copying apex/normalization/fused_layer_norm.py -> build/lib.linux-x86_64-3.7/apex/normalization
        creating build/lib.linux-x86_64-3.7/apex/pyprof
        copying apex/pyprof/__init__.py -> build/lib.linux-x86_64-3.7/apex/pyprof
        creating build/lib.linux-x86_64-3.7/apex/RNN
        copying apex/RNN/__init__.py -> build/lib.linux-x86_64-3.7/apex/RNN
        copying apex/RNN/models.py -> build/lib.linux-x86_64-3.7/apex/RNN
        copying apex/RNN/cells.py -> build/lib.linux-x86_64-3.7/apex/RNN
        copying apex/RNN/RNNBackend.py -> build/lib.linux-x86_64-3.7/apex/RNN
        creating build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/microbatches.py -> build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/enums.py -> build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/log_util.py -> build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/parallel_state.py -> build/lib.linux-x86_64-3.7/apex/transformer
        copying apex/transformer/utils.py -> build/lib.linux-x86_64-3.7/apex/transformer
        creating build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/sync_batchnorm.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/__init__.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/optimized_sync_batchnorm.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/distributed.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/multiproc.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/sync_batchnorm_kernel.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/LARC.py -> build/lib.linux-x86_64-3.7/apex/parallel
        copying apex/parallel/optimized_sync_batchnorm_kernel.py -> build/lib.linux-x86_64-3.7/apex/parallel
        creating build/lib.linux-x86_64-3.7/apex/fp16_utils
        copying apex/fp16_utils/__init__.py -> build/lib.linux-x86_64-3.7/apex/fp16_utils
        copying apex/fp16_utils/fp16util.py -> build/lib.linux-x86_64-3.7/apex/fp16_utils
        copying apex/fp16_utils/fp16_optimizer.py -> build/lib.linux-x86_64-3.7/apex/fp16_utils
        copying apex/fp16_utils/loss_scaler.py -> build/lib.linux-x86_64-3.7/apex/fp16_utils
        creating build/lib.linux-x86_64-3.7/apex/fused_dense
        copying apex/fused_dense/__init__.py -> build/lib.linux-x86_64-3.7/apex/fused_dense
        copying apex/fused_dense/fused_dense.py -> build/lib.linux-x86_64-3.7/apex/fused_dense
        creating build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/__init__.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/fused_novograd.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/fused_lamb.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/fused_adagrad.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/fused_adam.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        copying apex/optimizers/fused_sgd.py -> build/lib.linux-x86_64-3.7/apex/optimizers
        creating build/lib.linux-x86_64-3.7/apex/reparameterization
        copying apex/reparameterization/__init__.py -> build/lib.linux-x86_64-3.7/apex/reparameterization
        copying apex/reparameterization/reparameterization.py -> build/lib.linux-x86_64-3.7/apex/reparameterization
        copying apex/reparameterization/weight_norm.py -> build/lib.linux-x86_64-3.7/apex/reparameterization
        creating build/lib.linux-x86_64-3.7/apex/amp/lists
        copying apex/amp/lists/__init__.py -> build/lib.linux-x86_64-3.7/apex/amp/lists
        copying apex/amp/lists/torch_overrides.py -> build/lib.linux-x86_64-3.7/apex/amp/lists
        copying apex/amp/lists/functional_overrides.py -> build/lib.linux-x86_64-3.7/apex/amp/lists
        copying apex/amp/lists/tensor_overrides.py -> build/lib.linux-x86_64-3.7/apex/amp/lists
        creating build/lib.linux-x86_64-3.7/apex/contrib/groupbn
        copying apex/contrib/groupbn/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/groupbn
        copying apex/contrib/groupbn/batch_norm.py -> build/lib.linux-x86_64-3.7/apex/contrib/groupbn
        creating build/lib.linux-x86_64-3.7/apex/contrib/xentropy
        copying apex/contrib/xentropy/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/xentropy
        copying apex/contrib/xentropy/softmax_xentropy.py -> build/lib.linux-x86_64-3.7/apex/contrib/xentropy
        creating build/lib.linux-x86_64-3.7/apex/contrib/layer_norm
        copying apex/contrib/layer_norm/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/layer_norm
        copying apex/contrib/layer_norm/layer_norm.py -> build/lib.linux-x86_64-3.7/apex/contrib/layer_norm
        creating build/lib.linux-x86_64-3.7/apex/contrib/bottleneck
        copying apex/contrib/bottleneck/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/bottleneck
        copying apex/contrib/bottleneck/bottleneck.py -> build/lib.linux-x86_64-3.7/apex/contrib/bottleneck
        copying apex/contrib/bottleneck/bottleneck_module_test.py -> build/lib.linux-x86_64-3.7/apex/contrib/bottleneck
        copying apex/contrib/bottleneck/test.py -> build/lib.linux-x86_64-3.7/apex/contrib/bottleneck
        creating build/lib.linux-x86_64-3.7/apex/contrib/sparsity
        copying apex/contrib/sparsity/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/sparsity
        copying apex/contrib/sparsity/asp.py -> build/lib.linux-x86_64-3.7/apex/contrib/sparsity
        copying apex/contrib/sparsity/sparse_masklib.py -> build/lib.linux-x86_64-3.7/apex/contrib/sparsity
        creating build/lib.linux-x86_64-3.7/apex/contrib/transducer
        copying apex/contrib/transducer/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/transducer
        copying apex/contrib/transducer/transducer.py -> build/lib.linux-x86_64-3.7/apex/contrib/transducer
        creating build/lib.linux-x86_64-3.7/apex/contrib/fmha
        copying apex/contrib/fmha/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/fmha
        copying apex/contrib/fmha/fmha.py -> build/lib.linux-x86_64-3.7/apex/contrib/fmha
        creating build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/self_multihead_attn.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        copying apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn
        creating build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/__init__.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/distributed_fused_lamb.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/fp16_optimizer.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/fused_lamb.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/distributed_fused_adam_v3.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/fused_adam.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/fused_sgd.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/distributed_fused_adam_v2.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        copying apex/contrib/optimizers/distributed_fused_adam.py -> build/lib.linux-x86_64-3.7/apex/contrib/optimizers
        creating build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/__init__.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/optim.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/misc.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/output.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/pooling.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/linear.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/blas.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/softmax.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/base.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/pointwise.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/index_slice_join_mutate.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/randomSample.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/dropout.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/__main__.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/activation.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/convert.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/utility.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/conv.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/normalization.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/data.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/usage.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/loss.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/prof.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/recurrentCell.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/embedding.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        copying apex/pyprof/prof/reduction.py -> build/lib.linux-x86_64-3.7/apex/pyprof/prof
        creating build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/__init__.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/nvvp.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/kernel.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/__main__.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/parse.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        copying apex/pyprof/parse/db.py -> build/lib.linux-x86_64-3.7/apex/pyprof/parse
        creating build/lib.linux-x86_64-3.7/apex/pyprof/nvtx
        copying apex/pyprof/nvtx/__init__.py -> build/lib.linux-x86_64-3.7/apex/pyprof/nvtx
        copying apex/pyprof/nvtx/nvmarker.py -> build/lib.linux-x86_64-3.7/apex/pyprof/nvtx
        creating build/lib.linux-x86_64-3.7/apex/transformer/_data
        copying apex/transformer/_data/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/_data
        copying apex/transformer/_data/_batchsampler.py -> build/lib.linux-x86_64-3.7/apex/transformer/_data
        creating build/lib.linux-x86_64-3.7/apex/transformer/amp
        copying apex/transformer/amp/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/amp
        copying apex/transformer/amp/grad_scaler.py -> build/lib.linux-x86_64-3.7/apex/transformer/amp
        creating build/lib.linux-x86_64-3.7/apex/transformer/functional
        copying apex/transformer/functional/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/functional
        copying apex/transformer/functional/fused_softmax.py -> build/lib.linux-x86_64-3.7/apex/transformer/functional
        creating build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel
        copying apex/transformer/pipeline_parallel/_timers.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel
        copying apex/transformer/pipeline_parallel/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel
        copying apex/transformer/pipeline_parallel/p2p_communication.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel
        copying apex/transformer/pipeline_parallel/utils.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel
        creating build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/cross_entropy.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/mappings.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/data.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/layers.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/random.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/memory.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        copying apex/transformer/tensor_parallel/utils.py -> build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel
        creating build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/standalone_gpt.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/arguments.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/standalone_bert.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/commons.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        copying apex/transformer/testing/global_vars.py -> build/lib.linux-x86_64-3.7/apex/transformer/testing
        creating build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        copying apex/transformer/pipeline_parallel/schedules/__init__.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        copying apex/transformer/pipeline_parallel/schedules/common.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules
        running build_ext
        building 'apex_C' extension
        creating /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7
        creating /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/1] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/flatten_unflatten.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/flatten_unflatten.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/flatten_unflatten.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=apex_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/flatten_unflatten.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        In file included from /tmp/pip-req-build-41vu3jcb/csrc/flatten_unflatten.cpp:2:0:
        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/utils/tensor_flatten.h: In member function ‘at::DeprecatedTypeProperties& torch::utils::TensorGroup::type()’:
        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/utils/tensor_flatten.h:36:28: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
             return tensors[0].type();
                                    ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/flatten_unflatten.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/flatten_unflatten.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.7/apex_C.cpython-37m-x86_64-linux-gnu.so
        building 'amp_C' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_scale_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_scale_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [2/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_sgd_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_sgd_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [3/12] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/amp_C_frontend.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/amp_C_frontend.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/amp_C_frontend.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/amp_C_frontend.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        [4/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_axpby_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_axpby_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [5/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_l2norm_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_l2norm_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [6/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_l2norm_scale_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_l2norm_scale_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [7/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_lamb_stage_2.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb_stage_2.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [8/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_lamb_stage_1.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb_stage_1.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [9/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_adam.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_adam.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [10/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_adagrad.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_adagrad.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [11/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_novograd.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_novograd.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [12/12] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/multi_tensor_lamb.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/amp_C_frontend.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_sgd_kernel.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_scale_kernel.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_axpby_kernel.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_l2norm_kernel.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_l2norm_scale_kernel.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb_stage_1.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb_stage_2.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_adam.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_adagrad.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_novograd.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/multi_tensor_lamb.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/amp_C.cpython-37m-x86_64-linux-gnu.so
        building 'syncbn' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/welford.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/welford.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=syncbn -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [2/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/syncbn.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/syncbn.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/syncbn.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=syncbn -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/syncbn.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/syncbn.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/welford.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/syncbn.cpython-37m-x86_64-linux-gnu.so
        building 'fused_layer_norm_cuda' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda_kernel.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/layer_norm_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -maxrregcount=50 -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=fused_layer_norm_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        [2/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/layer_norm_cuda.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/layer_norm_cuda.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=fused_layer_norm_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp: In function ‘std::vector<at::Tensor> layer_norm(at::Tensor, c10::IntArrayRef, double)’:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:129:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(input);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp: In function ‘std::vector<at::Tensor> layer_norm_affine(at::Tensor, c10::IntArrayRef, at::Tensor, at::Tensor, double)’:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:150:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(input);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:151:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(gamma);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:152:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(beta);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp: In function ‘std::vector<at::Tensor> layer_norm_affine_mixed_dtypes(at::Tensor, c10::IntArrayRef, at::Tensor, at::Tensor, double)’:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:174:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(input);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp: In function ‘at::Tensor layer_norm_gradient(at::Tensor, at::Tensor, at::Tensor, at::Tensor, c10::IntArrayRef, double)’:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:216:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(dout);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:217:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(mean);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:218:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(invvar);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:219:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(input);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp: In function ‘std::vector<at::Tensor> layer_norm_gradient_affine(at::Tensor, at::Tensor, at::Tensor, at::Tensor, c10::IntArrayRef, at::Tensor, at::Tensor, double)’:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:242:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(dout);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:243:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(mean);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:244:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(invvar);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:245:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(input);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:246:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(gamma);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/DeviceType.h:8:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Device.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/c10/core/Allocator.h:6,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                                                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/macros/Macros.h:146:65: note: in definition of macro ‘C10_UNLIKELY’
         #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                         ^~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:330:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
           if (C10_UNLIKELY_OR_CONST(!(cond))) {                               \
               ^~~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:318:3: note: in expansion of macro ‘TORCH_CHECK_WITH_MSG’
           TORCH_CHECK_WITH_MSG(error_t, cond, "", __VA_ARGS__)
           ^~~~~~~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/c10/util/Exception.h:341:32: note: in expansion of macro ‘TORCH_CHECK_WITH’
         #define TORCH_CHECK(cond, ...) TORCH_CHECK_WITH(Error, cond, __VA_ARGS__)
                                        ^~~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:117:23: note: in expansion of macro ‘TORCH_CHECK’
         #define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:119:24: note: in expansion of macro ‘CHECK_CUDA’
         #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
                                ^~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:247:3: note: in expansion of macro ‘CHECK_INPUT’
           CHECK_INPUT(beta);
           ^~~~~~~~~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/layer_norm_cuda.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/layer_norm_cuda.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/layer_norm_cuda_kernel.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/fused_layer_norm_cuda.cpython-37m-x86_64-linux-gnu.so
        building 'mlp_cuda' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/mlp.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/mlp.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=mlp_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In function ‘std::vector<at::Tensor> mlp_forward(int, int, std::vector<at::Tensor>)’:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:57:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
           for (int i = 0; i < num_layers; i++) {
                           ~~^~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:64:77: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto out = at::empty({batch_size, output_features.back()}, inputs[0].type());
                                                                                ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:65:67: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto reserved_space = at::empty({reserved_size}, inputs[0].type());
                                                                           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:65:68: warning: narrowing conversion of ‘reserved_size’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
           auto reserved_space = at::empty({reserved_size}, inputs[0].type());
                                                                            ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:65:68: warning: narrowing conversion of ‘reserved_size’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:67:59: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto lt_workspace = at::empty({1 << 22}, inputs[0].type());
                                                                   ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:54: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
                                                              ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:72:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_fp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:72:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_fp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:72:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_fp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:69:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In function ‘std::vector<at::Tensor> mlp_backward(int, int, at::Tensor, std::vector<at::Tensor>, std::vector<at::Tensor>)’:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:115:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
           for (int i = 0; i < num_layers; i++) {
                           ~~^~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:120:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
           for (int i = 0; i < inputs.size(); i++) {
                           ~~^~~~~~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:121:67: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
             outputs.push_back(at::empty(inputs[i].sizes(), inputs[i].type()));  // clone for testing now
                                                                           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:54: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
                                                              ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:126:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:130:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < inputs.size(); i++) {
                             ~~^~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:80: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                                                                ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:140:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_bp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:126:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:130:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < inputs.size(); i++) {
                             ~~^~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:80: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                                                                ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:140:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_bp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:126:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < num_layers; i++) {
                             ~~^~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:130:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
             for (int i = 0; i < inputs.size(); i++) {
                             ~~^~~~~~~~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:80: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                                                                ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:138:44: warning: narrowing conversion of ‘(work_size / sizeof (scalar_t))’ from ‘long unsigned int’ to ‘long int’ inside { } [-Wnarrowing]
             auto work_space = at::empty({work_size / sizeof(scalar_t)}, inputs[0].type());
                                          ~~~~~~~~~~^~~~~~~~~
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:140:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = mlp_bp<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/mlp.cpp:124:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
           ^
        [2/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/mlp_cuda.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/mlp_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=mlp_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/mlp.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/mlp_cuda.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/mlp_cuda.cpython-37m-x86_64-linux-gnu.so
        building 'fused_dense_cuda' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/fused_dense.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/fused_dense.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=fused_dense_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In function ‘at::Tensor linear_bias_forward(at::Tensor, at::Tensor, at::Tensor)’:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:30:63: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto out = at::empty({batch_size, out_features}, input.type());
                                                                       ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:33:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto lt_workspace = at::empty({1 << 22}, input.type());
                                                               ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:50: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
                                                          ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:37:15: warning: unused variable ‘b_ptr’ [-Wunused-variable]
             scalar_t* b_ptr = bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:38:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:37:15: warning: unused variable ‘b_ptr’ [-Wunused-variable]
             scalar_t* b_ptr = bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:38:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:37:15: warning: unused variable ‘b_ptr’ [-Wunused-variable]
             scalar_t* b_ptr = bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:38:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:35:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In function ‘std::vector<at::Tensor> linear_bias_backward(at::Tensor, at::Tensor, at::Tensor)’:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:64:69: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_weight = at::empty({out_features, in_features}, input.type());
                                                                             ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:68:54: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_bias = at::empty({out_features}, input.type());
                                                              ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:70:66: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_input = at::empty({batch_size, in_features}, input.type());
                                                                          ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:73:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto lt_workspace = at::empty({1 << 22}, input.type());
                                                               ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:50: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
                                                          ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:77:15: warning: unused variable ‘d_b_ptr’ [-Wunused-variable]
             scalar_t* d_b_ptr = d_bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:77:15: warning: unused variable ‘d_b_ptr’ [-Wunused-variable]
             scalar_t* d_b_ptr = d_bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:77:15: warning: unused variable ‘d_b_ptr’ [-Wunused-variable]
             scalar_t* d_b_ptr = d_bias.data_ptr<scalar_t>();
                       ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:78:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_bias_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:75:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In function ‘std::vector<at::Tensor> linear_gelu_linear_forward(at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor)’:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:106:70: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto output1 = at::empty({batch_size, hidden_features}, input.type());
                                                                              ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:107:70: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto gelu_in = at::empty({batch_size, hidden_features}, input.type());
                                                                              ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:108:67: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto output2 = at::empty({batch_size, out_features}, input.type());
                                                                           ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:111:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto lt_workspace = at::empty({1 << 22}, input.type());
                                                               ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:113:50: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_gelu_linear_forward", [&] {
                                                          ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:113:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_gelu_linear_forward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:118:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:113:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_gelu_linear_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:118:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:113:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_gelu_linear_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:118:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_forward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:113:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_gelu_linear_forward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In function ‘std::vector<at::Tensor> linear_gelu_linear_backward(at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor)’:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:149:73: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_weight1 = at::empty({hidden_features, in_features}, input.type());
                                                                                ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:150:74: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_weight2 = at::empty({out_features, hidden_features}, input.type());
                                                                                ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:151:58: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_bias1 = at::empty({hidden_features}, input.type());
                                                                  ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:152:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_bias2 = at::empty({out_features}, input.type());
                                                               ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:153:66: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_input = at::empty({batch_size, in_features}, input.type());
                                                                          ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:154:72: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto d_output1 = at::empty({batch_size, hidden_features}, input.type());
                                                                                ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:157:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           auto lt_workspace = at::empty({1 << 22}, input.type());
                                                               ^
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:159:50: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
                                                          ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:150:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
             const auto& the_type = TYPE;                                            \
                                    ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Tensor.h:3:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Context.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/core/TensorBody.h:268:30: note: declared here
           DeprecatedTypeProperties & type() const {
                                      ^~~~
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ATen.h:9:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:4,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:1:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:152:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
             at::ScalarType _st = ::detail::scalar_type(the_type);                   \
                                                                ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:66:23: note: declared here
         inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
                               ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:162:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:162:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp: In lambda function:
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:162:10: warning: unused variable ‘result’ [-Wunused-variable]
             auto result = linear_gelu_linear_backward_cuda<scalar_t>(
                  ^
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Dispatch.h:13:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
             return __VA_ARGS__();                          \
                    ^~~~~~~~~~~
        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
           AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "linear_bias_backward", [&] {
           ^
        [2/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/fused_dense_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=fused_dense_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1148): warning: variable "beta_zero" was declared but never referenced

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1272): warning: variable "alpha" was declared but never referenced

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1273): warning: variable "beta_zero" was declared but never referenced

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1274): warning: variable "status" was declared but never referenced

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1329): warning: variable "alpha" was declared but never referenced

        /tmp/pip-req-build-41vu3jcb/csrc/fused_dense_cuda.cu(1330): warning: variable "beta_zero" was declared but never referenced

        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/fused_dense.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/fused_dense_cuda.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/fused_dense_cuda.cpython-37m-x86_64-linux-gnu.so
        building 'scaled_upper_triang_masked_softmax_cuda' extension
        creating /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_upper_triang_masked_softmax.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/pip-req-build-41vu3jcb/csrc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_upper_triang_masked_softmax.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_upper_triang_masked_softmax.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scaled_upper_triang_masked_softmax_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_upper_triang_masked_softmax.cpp:18:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        [2/2] /usr/local/cuda/bin/nvcc -I/tmp/pip-req-build-41vu3jcb/csrc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_upper_triang_masked_softmax_cuda.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_upper_triang_masked_softmax_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scaled_upper_triang_masked_softmax_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_upper_triang_masked_softmax.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_upper_triang_masked_softmax_cuda.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/scaled_upper_triang_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so
        building 'scaled_masked_softmax_cuda' extension
        Emitting ninja build file /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/build.ninja...
        Compiling objects...
        Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
        [1/2] c++ -MMD -MF /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_masked_softmax.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/pip-req-build-41vu3jcb/csrc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_masked_softmax.cpp -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_masked_softmax.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scaled_masked_softmax_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
        In file included from /usr/local/lib/python3.7/dist-packages/torch/include/ATen/Parallel.h:149:0,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                         from /usr/local/lib/python3.7/dist-packages/torch/include/torch/extension.h:4,
                         from /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_masked_softmax.cpp:18:
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
         #pragma omp parallel for if ((end - begin) >= grain_size)

        [2/2] /usr/local/cuda/bin/nvcc -I/tmp/pip-req-build-41vu3jcb/csrc -I/usr/local/lib/python3.7/dist-packages/torch/include -I/usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/dist-packages/torch/include/TH -I/usr/local/lib/python3.7/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.7m -c -c /tmp/pip-req-build-41vu3jcb/csrc/megatron/scaled_masked_softmax_cuda.cu -o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_masked_softmax_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scaled_masked_softmax_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_37,code=sm_37 -std=c++14
        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/ATen/record_function.h(18): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(97): warning: attribute "__visibility__" does not apply here

        /usr/local/lib/python3.7/dist-packages/torch/include/torch/csrc/autograd/profiler.h(126): warning: attribute "__visibility__" does not apply here

        x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fdebug-prefix-map=/build/python3.7-Y7dWVB/python3.7-3.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_masked_softmax.o /tmp/pip-req-build-41vu3jcb/build/temp.linux-x86_64-3.7/csrc/megatron/scaled_masked_softmax_cuda.o -L/usr/local/lib/python3.7/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/scaled_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so
        running install_lib
        copying build/lib.linux-x86_64-3.7/fused_layer_norm_cuda.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/scaled_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/mlp_cuda.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        creating /usr/local/lib/python3.7/dist-packages/apex
        copying build/lib.linux-x86_64-3.7/apex/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex
        creating /usr/local/lib/python3.7/dist-packages/apex/multi_tensor_apply
        copying build/lib.linux-x86_64-3.7/apex/multi_tensor_apply/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/multi_tensor_apply
        copying build/lib.linux-x86_64-3.7/apex/multi_tensor_apply/multi_tensor_apply.py -> /usr/local/lib/python3.7/dist-packages/apex/multi_tensor_apply
        creating /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/__version__.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/compat.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/_initialize.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        creating /usr/local/lib/python3.7/dist-packages/apex/amp/lists
        copying build/lib.linux-x86_64-3.7/apex/amp/lists/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/amp/lists
        copying build/lib.linux-x86_64-3.7/apex/amp/lists/torch_overrides.py -> /usr/local/lib/python3.7/dist-packages/apex/amp/lists
        copying build/lib.linux-x86_64-3.7/apex/amp/lists/functional_overrides.py -> /usr/local/lib/python3.7/dist-packages/apex/amp/lists
        copying build/lib.linux-x86_64-3.7/apex/amp/lists/tensor_overrides.py -> /usr/local/lib/python3.7/dist-packages/apex/amp/lists
        copying build/lib.linux-x86_64-3.7/apex/amp/handle.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/wrap.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/rnn_compat.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/_process_optimizer.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/frontend.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/opt.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/scaler.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/_amp_state.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/utils.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        copying build/lib.linux-x86_64-3.7/apex/amp/amp.py -> /usr/local/lib/python3.7/dist-packages/apex/amp
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib
        copying build/lib.linux-x86_64-3.7/apex/contrib/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/groupbn
        copying build/lib.linux-x86_64-3.7/apex/contrib/groupbn/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/groupbn
        copying build/lib.linux-x86_64-3.7/apex/contrib/groupbn/batch_norm.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/groupbn
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/xentropy
        copying build/lib.linux-x86_64-3.7/apex/contrib/xentropy/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/xentropy
        copying build/lib.linux-x86_64-3.7/apex/contrib/xentropy/softmax_xentropy.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/xentropy
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/layer_norm
        copying build/lib.linux-x86_64-3.7/apex/contrib/layer_norm/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/layer_norm
        copying build/lib.linux-x86_64-3.7/apex/contrib/layer_norm/layer_norm.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/layer_norm
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck
        copying build/lib.linux-x86_64-3.7/apex/contrib/bottleneck/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck
        copying build/lib.linux-x86_64-3.7/apex/contrib/bottleneck/bottleneck.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck
        copying build/lib.linux-x86_64-3.7/apex/contrib/bottleneck/bottleneck_module_test.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck
        copying build/lib.linux-x86_64-3.7/apex/contrib/bottleneck/test.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity
        copying build/lib.linux-x86_64-3.7/apex/contrib/sparsity/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity
        copying build/lib.linux-x86_64-3.7/apex/contrib/sparsity/asp.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity
        copying build/lib.linux-x86_64-3.7/apex/contrib/sparsity/sparse_masklib.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/transducer
        copying build/lib.linux-x86_64-3.7/apex/contrib/transducer/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/transducer
        copying build/lib.linux-x86_64-3.7/apex/contrib/transducer/transducer.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/transducer
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/fmha
        copying build/lib.linux-x86_64-3.7/apex/contrib/fmha/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/fmha
        copying build/lib.linux-x86_64-3.7/apex/contrib/fmha/fmha.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/fmha
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/self_multihead_attn.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/encdec_multihead_attn.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        copying build/lib.linux-x86_64-3.7/apex/contrib/multihead_attn/self_multihead_attn_func.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn
        creating /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/distributed_fused_lamb.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/fp16_optimizer.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/fused_lamb.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/distributed_fused_adam_v3.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/fused_adam.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/fused_sgd.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/distributed_fused_adam_v2.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        copying build/lib.linux-x86_64-3.7/apex/contrib/optimizers/distributed_fused_adam.py -> /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers
        creating /usr/local/lib/python3.7/dist-packages/apex/mlp
        copying build/lib.linux-x86_64-3.7/apex/mlp/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/mlp
        copying build/lib.linux-x86_64-3.7/apex/mlp/mlp.py -> /usr/local/lib/python3.7/dist-packages/apex/mlp
        creating /usr/local/lib/python3.7/dist-packages/apex/normalization
        copying build/lib.linux-x86_64-3.7/apex/normalization/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/normalization
        copying build/lib.linux-x86_64-3.7/apex/normalization/fused_layer_norm.py -> /usr/local/lib/python3.7/dist-packages/apex/normalization
        copying build/lib.linux-x86_64-3.7/apex/_autocast_utils.py -> /usr/local/lib/python3.7/dist-packages/apex
        creating /usr/local/lib/python3.7/dist-packages/apex/pyprof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof
        creating /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/optim.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/misc.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/output.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/pooling.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/linear.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/blas.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/softmax.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/base.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/pointwise.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/index_slice_join_mutate.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/randomSample.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/dropout.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/__main__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/activation.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/convert.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/utility.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/conv.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/normalization.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/data.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/usage.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/loss.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/prof.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/recurrentCell.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/embedding.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        copying build/lib.linux-x86_64-3.7/apex/pyprof/prof/reduction.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof
        creating /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/nvvp.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/kernel.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/__main__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/parse.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        copying build/lib.linux-x86_64-3.7/apex/pyprof/parse/db.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse
        creating /usr/local/lib/python3.7/dist-packages/apex/pyprof/nvtx
        copying build/lib.linux-x86_64-3.7/apex/pyprof/nvtx/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/nvtx
        copying build/lib.linux-x86_64-3.7/apex/pyprof/nvtx/nvmarker.py -> /usr/local/lib/python3.7/dist-packages/apex/pyprof/nvtx
        creating /usr/local/lib/python3.7/dist-packages/apex/RNN
        copying build/lib.linux-x86_64-3.7/apex/RNN/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/RNN
        copying build/lib.linux-x86_64-3.7/apex/RNN/models.py -> /usr/local/lib/python3.7/dist-packages/apex/RNN
        copying build/lib.linux-x86_64-3.7/apex/RNN/cells.py -> /usr/local/lib/python3.7/dist-packages/apex/RNN
        copying build/lib.linux-x86_64-3.7/apex/RNN/RNNBackend.py -> /usr/local/lib/python3.7/dist-packages/apex/RNN
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer
        copying build/lib.linux-x86_64-3.7/apex/transformer/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        copying build/lib.linux-x86_64-3.7/apex/transformer/microbatches.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/_data
        copying build/lib.linux-x86_64-3.7/apex/transformer/_data/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/_data
        copying build/lib.linux-x86_64-3.7/apex/transformer/_data/_batchsampler.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/_data
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/amp
        copying build/lib.linux-x86_64-3.7/apex/transformer/amp/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/amp
        copying build/lib.linux-x86_64-3.7/apex/transformer/amp/grad_scaler.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/amp
        copying build/lib.linux-x86_64-3.7/apex/transformer/enums.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/functional
        copying build/lib.linux-x86_64-3.7/apex/transformer/functional/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/functional
        copying build/lib.linux-x86_64-3.7/apex/transformer/functional/fused_softmax.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/functional
        copying build/lib.linux-x86_64-3.7/apex/transformer/log_util.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/_timers.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules/common.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/p2p_communication.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/pipeline_parallel/utils.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/parallel_state.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/cross_entropy.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/mappings.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/data.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/layers.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/random.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/memory.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        copying build/lib.linux-x86_64-3.7/apex/transformer/tensor_parallel/utils.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel
        creating /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/standalone_gpt.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/arguments.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/standalone_bert.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/commons.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/testing/global_vars.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer/testing
        copying build/lib.linux-x86_64-3.7/apex/transformer/utils.py -> /usr/local/lib/python3.7/dist-packages/apex/transformer
        creating /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/sync_batchnorm.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/optimized_sync_batchnorm.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/distributed.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/multiproc.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/sync_batchnorm_kernel.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/LARC.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        copying build/lib.linux-x86_64-3.7/apex/parallel/optimized_sync_batchnorm_kernel.py -> /usr/local/lib/python3.7/dist-packages/apex/parallel
        creating /usr/local/lib/python3.7/dist-packages/apex/fp16_utils
        copying build/lib.linux-x86_64-3.7/apex/fp16_utils/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/fp16_utils
        copying build/lib.linux-x86_64-3.7/apex/fp16_utils/fp16util.py -> /usr/local/lib/python3.7/dist-packages/apex/fp16_utils
        copying build/lib.linux-x86_64-3.7/apex/fp16_utils/fp16_optimizer.py -> /usr/local/lib/python3.7/dist-packages/apex/fp16_utils
        copying build/lib.linux-x86_64-3.7/apex/fp16_utils/loss_scaler.py -> /usr/local/lib/python3.7/dist-packages/apex/fp16_utils
        creating /usr/local/lib/python3.7/dist-packages/apex/fused_dense
        copying build/lib.linux-x86_64-3.7/apex/fused_dense/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/fused_dense
        copying build/lib.linux-x86_64-3.7/apex/fused_dense/fused_dense.py -> /usr/local/lib/python3.7/dist-packages/apex/fused_dense
        creating /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/fused_novograd.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/fused_lamb.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/fused_adagrad.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/fused_adam.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        copying build/lib.linux-x86_64-3.7/apex/optimizers/fused_sgd.py -> /usr/local/lib/python3.7/dist-packages/apex/optimizers
        creating /usr/local/lib/python3.7/dist-packages/apex/reparameterization
        copying build/lib.linux-x86_64-3.7/apex/reparameterization/__init__.py -> /usr/local/lib/python3.7/dist-packages/apex/reparameterization
        copying build/lib.linux-x86_64-3.7/apex/reparameterization/reparameterization.py -> /usr/local/lib/python3.7/dist-packages/apex/reparameterization
        copying build/lib.linux-x86_64-3.7/apex/reparameterization/weight_norm.py -> /usr/local/lib/python3.7/dist-packages/apex/reparameterization
        copying build/lib.linux-x86_64-3.7/apex_C.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/scaled_upper_triang_masked_softmax_cuda.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/syncbn.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/fused_dense_cuda.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        copying build/lib.linux-x86_64-3.7/amp_C.cpython-37m-x86_64-linux-gnu.so -> /usr/local/lib/python3.7/dist-packages
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/multi_tensor_apply/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/multi_tensor_apply/multi_tensor_apply.py to multi_tensor_apply.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/__version__.py to __version__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/compat.py to compat.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/_initialize.py to _initialize.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/lists/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/lists/torch_overrides.py to torch_overrides.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/lists/functional_overrides.py to functional_overrides.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/lists/tensor_overrides.py to tensor_overrides.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/handle.py to handle.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/wrap.py to wrap.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/rnn_compat.py to rnn_compat.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/_process_optimizer.py to _process_optimizer.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/frontend.py to frontend.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/opt.py to opt.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/scaler.py to scaler.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/_amp_state.py to _amp_state.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/utils.py to utils.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/amp/amp.py to amp.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/groupbn/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/groupbn/batch_norm.py to batch_norm.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/xentropy/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/xentropy/softmax_xentropy.py to softmax_xentropy.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/layer_norm/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/layer_norm/layer_norm.py to layer_norm.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck/bottleneck.py to bottleneck.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck/bottleneck_module_test.py to bottleneck_module_test.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/bottleneck/test.py to test.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity/asp.py to asp.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/sparsity/sparse_masklib.py to sparse_masklib.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/transducer/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/transducer/transducer.py to transducer.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/fmha/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/fmha/fmha.py to fmha.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py to fast_self_multihead_attn_norm_add_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/self_multihead_attn.py to self_multihead_attn.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py to fast_self_multihead_attn_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py to fast_encdec_multihead_attn_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py to fast_encdec_multihead_attn_norm_add_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/encdec_multihead_attn.py to encdec_multihead_attn.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/encdec_multihead_attn_func.py to encdec_multihead_attn_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/mask_softmax_dropout_func.py to mask_softmax_dropout_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/multihead_attn/self_multihead_attn_func.py to self_multihead_attn_func.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/distributed_fused_lamb.py to distributed_fused_lamb.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/fp16_optimizer.py to fp16_optimizer.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/fused_lamb.py to fused_lamb.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/distributed_fused_adam_v3.py to distributed_fused_adam_v3.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/fused_adam.py to fused_adam.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/fused_sgd.py to fused_sgd.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/distributed_fused_adam_v2.py to distributed_fused_adam_v2.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/contrib/optimizers/distributed_fused_adam.py to distributed_fused_adam.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/mlp/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/mlp/mlp.py to mlp.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/normalization/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/normalization/fused_layer_norm.py to fused_layer_norm.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/_autocast_utils.py to _autocast_utils.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/optim.py to optim.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/misc.py to misc.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/output.py to output.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/pooling.py to pooling.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/linear.py to linear.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/blas.py to blas.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/softmax.py to softmax.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/base.py to base.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/pointwise.py to pointwise.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/index_slice_join_mutate.py to index_slice_join_mutate.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/randomSample.py to randomSample.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/dropout.py to dropout.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/__main__.py to __main__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/activation.py to activation.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/convert.py to convert.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/utility.py to utility.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/conv.py to conv.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/normalization.py to normalization.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/data.py to data.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/usage.py to usage.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/loss.py to loss.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/prof.py to prof.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/recurrentCell.py to recurrentCell.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/embedding.py to embedding.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/prof/reduction.py to reduction.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/nvvp.py to nvvp.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/kernel.py to kernel.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/__main__.py to __main__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/parse.py to parse.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/parse/db.py to db.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/nvtx/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/pyprof/nvtx/nvmarker.py to nvmarker.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/RNN/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/RNN/models.py to models.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/RNN/cells.py to cells.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/RNN/RNNBackend.py to RNNBackend.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/microbatches.py to microbatches.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/_data/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/_data/_batchsampler.py to _batchsampler.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/amp/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/amp/grad_scaler.py to grad_scaler.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/enums.py to enums.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/functional/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/functional/fused_softmax.py to fused_softmax.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/log_util.py to log_util.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/_timers.py to _timers.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py to fwd_bwd_pipelining_without_interleaving.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py to fwd_bwd_pipelining_with_interleaving.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules/common.py to common.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py to fwd_bwd_no_pipelining.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/p2p_communication.py to p2p_communication.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/pipeline_parallel/utils.py to utils.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/parallel_state.py to parallel_state.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/cross_entropy.py to cross_entropy.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/mappings.py to mappings.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/data.py to data.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/layers.py to layers.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/random.py to random.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/memory.py to memory.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/tensor_parallel/utils.py to utils.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/standalone_gpt.py to standalone_gpt.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/arguments.py to arguments.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/standalone_bert.py to standalone_bert.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/commons.py to commons.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/testing/global_vars.py to global_vars.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/transformer/utils.py to utils.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/sync_batchnorm.py to sync_batchnorm.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/optimized_sync_batchnorm.py to optimized_sync_batchnorm.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/distributed.py to distributed.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/multiproc.py to multiproc.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/sync_batchnorm_kernel.py to sync_batchnorm_kernel.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/LARC.py to LARC.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/parallel/optimized_sync_batchnorm_kernel.py to optimized_sync_batchnorm_kernel.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fp16_utils/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fp16_utils/fp16util.py to fp16util.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fp16_utils/fp16_optimizer.py to fp16_optimizer.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fp16_utils/loss_scaler.py to loss_scaler.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fused_dense/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/fused_dense/fused_dense.py to fused_dense.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/fused_novograd.py to fused_novograd.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/fused_lamb.py to fused_lamb.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/fused_adagrad.py to fused_adagrad.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/fused_adam.py to fused_adam.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/optimizers/fused_sgd.py to fused_sgd.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/reparameterization/__init__.py to __init__.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/reparameterization/reparameterization.py to reparameterization.cpython-37.pyc
        byte-compiling /usr/local/lib/python3.7/dist-packages/apex/reparameterization/weight_norm.py to weight_norm.cpython-37.pyc
        running install_egg_info
        running egg_info
        creating apex.egg-info
        writing apex.egg-info/PKG-INFO
        writing dependency_links to apex.egg-info/dependency_links.txt
        writing top-level names to apex.egg-info/top_level.txt
        writing manifest file 'apex.egg-info/SOURCES.txt'
        adding license file 'LICENSE'
        writing manifest file 'apex.egg-info/SOURCES.txt'
        Copying apex.egg-info to /usr/local/lib/python3.7/dist-packages/apex-0.1-py3.7.egg-info
        running install_scripts
        writing list of installed files to '/tmp/pip-record-xcqqv22a/install-record.txt'
        Running setup.py install for apex ... done
    Value for scheme.platlib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/lib/python3.7/dist-packages
    sysconfig: /usr/lib/python3.7/site-packages
    Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/lib/python3.7/dist-packages
    sysconfig: /usr/lib/python3.7/site-packages
    Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/include/python3.7/UNKNOWN
    sysconfig: /usr/include/python3.7m/UNKNOWN
    Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local/bin
    sysconfig: /usr/bin
    Value for scheme.data does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: /usr/local
    sysconfig: /usr
    Additional context:
    user = False
    home = None
    root = None
    prefix = None
    Successfully installed apex-0.1
    Removed build tracker: '/tmp/pip-req-tracker-pgfywie_'

```python
!pip install triton==0.2.3
```

    Collecting triton==0.2.3
      Using cached triton-0.2.3-cp37-cp37m-linux_x86_64.whl
    Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from triton==0.2.3) (1.7.1)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from triton==0.2.3) (1.18.2)
    Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from triton==0.2.3) (1.6.0+cu101)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->triton==0.2.3) (1.2.1)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from torch->triton==0.2.3) (0.16.0)
    Installing collected packages: triton
    Successfully installed triton-0.2.3

```python
# !pip uninstall -y typing
```

```python
!pip install cpufeature
```

    Requirement already satisfied: cpufeature in /usr/local/lib/python3.7/dist-packages (0.2.0)

```python
!DS_BUILD_CPU_ADAM=1 DS_BUILD_SPARSE_ATTN=1 pip install deepspeed==0.3.7
```

    Requirement already satisfied: deepspeed==0.3.7 in /usr/local/lib/python3.7/dist-packages (0.3.7)
    Requirement already satisfied: torch>=1.2 in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (1.6.0+cu101)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (4.46.0)
    Requirement already satisfied: tensorboardX==1.8 in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (1.8)
    Requirement already satisfied: ninja in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (1.10.2.3)
    Requirement already satisfied: torchvision>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (0.7.0+cu101)
    Requirement already satisfied: triton>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from deepspeed==0.3.7) (0.2.3)
    Requirement already satisfied: protobuf>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from tensorboardX==1.8->deepspeed==0.3.7) (3.17.3)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tensorboardX==1.8->deepspeed==0.3.7) (1.15.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from tensorboardX==1.8->deepspeed==0.3.7) (1.18.2)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from torch>=1.2->deepspeed==0.3.7) (0.16.0)
    Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.7/dist-packages (from torchvision>=0.4.0->deepspeed==0.3.7) (7.1.2)
    Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from triton>=0.2.2->deepspeed==0.3.7) (1.7.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->triton>=0.2.2->deepspeed==0.3.7) (1.2.1)

```python
!ds_report
```

    --------------------------------------------------
    DeepSpeed C++/CUDA extension op report
    --------------------------------------------------
    NOTE: Ops not installed will be just-in-time (JIT) compiled at
          runtime if needed. Op compatibility means that your system
          meet the required dependencies to JIT install the op.
    --------------------------------------------------
    JIT compiled ops requires ninja
    ninja .................. [OKAY]
    --------------------------------------------------
    op name ................ installed .. compatible
    --------------------------------------------------
    cpu_adam ............... [YES] ...... [OKAY]
    fused_adam ............. [NO] ....... [OKAY]
    fused_lamb ............. [NO] ....... [OKAY]
    sparse_attn ............ [YES] ...... [OKAY]
    transformer ............ [NO] ....... [OKAY]
    stochastic_transformer . [NO] ....... [OKAY]
    utils .................. [NO] ....... [OKAY]
    --------------------------------------------------
    DeepSpeed general environment info:
    torch install path ............... ['/usr/local/lib/python3.7/dist-packages/torch']
    torch version .................... 1.6.0+cu101
    torch cuda version ............... 10.1
    nvcc version ..................... 10.1
    deepspeed install path ........... ['/usr/local/lib/python3.7/dist-packages/deepspeed']
    deepspeed info ................... 0.3.7, unknown, unknown
    deepspeed wheel compiled w. ...... torch 1.6, cuda 10.1

```python
import deepspeed.ops.sparse_attention.sparse_attn_op
```

```python
%cd /content/

!rm -rf rugpt3xl_ilm
!cat /content/rugpt3xl_ilm/requirements.txt
```

    /content
    cat: /content/rugpt3xl_ilm/requirements.txt: No such file or directory

```python
!git clone https://github.com/lauberto/rugpt3xl_ilm.git
 # TODO: add path to requirements.txt and install requirements (make sure to use the requirements necessary both for ILM and RUGP3XL)
```

    Cloning into 'rugpt3xl_ilm'...
    remote: Enumerating objects: 752, done.
    remote: Counting objects: 100% (247/247), done.
    remote: Compressing objects: 100% (149/149), done.
    remote: Total 752 (delta 135), reused 194 (delta 94), pack-reused 505
    Receiving objects: 100% (752/752), 1.66 MiB | 7.84 MiB/s, done.
    Resolving deltas: 100% (435/435), done.

```python
%%writefile ilm_requirements.txt

boto3==1.13.16
botocore==1.16.16
certifi==2020.4.5.1
chardet==3.0.4
click==7.1.2
docutils==0.15.2
filelock==3.0.12
idna==2.9
jmespath==0.10.0
joblib==0.15.1
nltk==3.4.5
numpy==1.18.2
python-dateutil==2.8.1
regex==2020.5.14
requests==2.23.0
s3transfer==0.3.3
sacremoses==0.0.43
sentencepiece==0.1.91
six==1.15.0
tokenizers==0.5.2
tqdm==4.46.0
urllib3==1.25.9
```

    Overwriting ilm_requirements.txt

```python
# %cd /content/rugpt3xl_ilm

! pip install -r ilm_requirements.txt
! python -c "import nltk; nltk.download('punkt')"
```

    Requirement already satisfied: boto3==1.13.16 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 2)) (1.13.16)
    Requirement already satisfied: botocore==1.16.16 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 3)) (1.16.16)
    Requirement already satisfied: certifi==2020.4.5.1 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 4)) (2020.4.5.1)
    Requirement already satisfied: chardet==3.0.4 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 5)) (3.0.4)
    Requirement already satisfied: click==7.1.2 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 6)) (7.1.2)
    Requirement already satisfied: docutils==0.15.2 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 7)) (0.15.2)
    Requirement already satisfied: filelock==3.0.12 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 8)) (3.0.12)
    Requirement already satisfied: idna==2.9 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 9)) (2.9)
    Requirement already satisfied: jmespath==0.10.0 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 10)) (0.10.0)
    Requirement already satisfied: joblib==0.15.1 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 11)) (0.15.1)
    Requirement already satisfied: nltk==3.4.5 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 12)) (3.4.5)
    Requirement already satisfied: numpy==1.18.2 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 13)) (1.18.2)
    Requirement already satisfied: python-dateutil==2.8.1 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 14)) (2.8.1)
    Requirement already satisfied: regex==2020.5.14 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 15)) (2020.5.14)
    Requirement already satisfied: requests==2.23.0 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 16)) (2.23.0)
    Requirement already satisfied: s3transfer==0.3.3 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 17)) (0.3.3)
    Requirement already satisfied: sacremoses==0.0.43 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 18)) (0.0.43)
    Requirement already satisfied: sentencepiece==0.1.91 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 19)) (0.1.91)
    Requirement already satisfied: six==1.15.0 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 20)) (1.15.0)
    Requirement already satisfied: tokenizers==0.5.2 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 21)) (0.5.2)
    Requirement already satisfied: tqdm==4.46.0 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 22)) (4.46.0)
    Requirement already satisfied: transformers==2.7.0 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 23)) (2.7.0)
    Requirement already satisfied: urllib3==1.25.9 in /usr/local/lib/python3.7/dist-packages (from -r ilm_requirements.txt (line 24)) (1.25.9)
    [nltk_data] Downloading package punkt to /root/nltk_data...
    [nltk_data]   Package punkt is already up-to-date!

```python
!pip install transformers==3.5.1
```

    Collecting transformers==3.5.1
      Downloading transformers-3.5.1-py3-none-any.whl (1.3 MB)
         |████████████████████████████████| 1.3 MB 4.3 MB/s
    Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (3.0.12)
    Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (0.0.43)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (2.23.0)
    Requirement already satisfied: sentencepiece==0.1.91 in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (0.1.91)
    Collecting tokenizers==0.9.3
      Downloading tokenizers-0.9.3-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB)
         |████████████████████████████████| 2.9 MB 27.2 MB/s
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (1.18.2)
    Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (4.46.0)
    Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (21.3)
    Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (2020.5.14)
    Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from transformers==3.5.1) (3.17.3)
    Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->transformers==3.5.1) (3.0.6)
    Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.7/dist-packages (from protobuf->transformers==3.5.1) (1.15.0)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==3.5.1) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==3.5.1) (2020.4.5.1)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==3.5.1) (1.25.9)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==3.5.1) (2.9)
    Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers==3.5.1) (7.1.2)
    Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers==3.5.1) (0.15.1)
    Installing collected packages: tokenizers, transformers
      Attempting uninstall: tokenizers
        Found existing installation: tokenizers 0.5.2
        Uninstalling tokenizers-0.5.2:
          Successfully uninstalled tokenizers-0.5.2
      Attempting uninstall: transformers
        Found existing installation: transformers 2.7.0
        Uninstalling transformers-2.7.0:
          Successfully uninstalled transformers-2.7.0
    Successfully installed tokenizers-0.9.3 transformers-3.5.1

```python
!cp /content/rugpt3xl_ilm/src_utils/trainer_pt_utils.py /usr/local/lib/python3.7/dist-packages/transformers/trainer_pt_utils.py
```

# Test model

### Load model

```python
import warnings
warnings.filterwarnings("ignore")
```

```python
import sys
sys.path.append("rugpt3xl_ilm/")
```

```python
import os
os.environ["USE_DEEPSPEED"] = "1"
```

## ILM

```python
from google.colab import drive
drive.mount('/content/gdrive')

%cd gdrive/My Drive/RUGPT3XL
```

    Mounted at /content/gdrive
    /content/gdrive/My Drive/RUGPT3XL

```python
# TODO: upload data to gdrive
# TODO: use the right path for the script below
!! python3 /content/rugpt3xl_ilm/train_ilm.py \
	experiment \
  /content/gdrive/RUGPT3XL/model_Economics \
	/content/gdrive/RUGPT3XL/data/Economics_char_masks/ \
	--seed 0 \
	--train_examples_tag train \
  --data_loader_num_workers 8 \
  --eval_examples_tag valid \
  --eval_max_num_examples 512 \
  --tokenizer_name RUGPT3XL \
  --model_name sberbank-ai/rugpt3xl 2>&1 | tee rugpt3xl_TrainerLog.txt
```

    ['Traceback (most recent call last):',
     '  File "/content/rugpt3xl_ilm/train_ilm.py", line 23, in <module>',
     '    import ilm.mask.util',
     '  File "/content/rugpt3xl_ilm/ilm/mask/util.py", line 3, in <module>',
     '    from ..tokenize_util import tokens_offsets, align_charspan_to_tokenspan',
     '  File "/content/rugpt3xl_ilm/ilm/tokenize_util.py", line 10, in <module>',
     '    from .official_gpt2_encoder.encoder import Encoder as OfficialEncoder',
     '  File "/content/rugpt3xl_ilm/ilm/official_gpt2_encoder/encoder.py", line 8, in <module>',
     '    from ilm.tokenize_util import Tokenizer',
     "ImportError: cannot import name 'Tokenizer' from 'ilm.tokenize_util' (/content/rugpt3xl_ilm/ilm/tokenize_util.py)"]

```python
!cat /content/rugpt3xl_ilm/ilm/official_gpt2_encoder/encoder.py
```

    """Byte pair encoding utilities"""

    import os
    import json
    import regex as re
    from functools import lru_cache

    from ilm.tokenize_util import Tokenizer
    from ilm.paths import OFFICIAL_GPT2_ENCODER_DIR, OFFICIAL_RUGPT3_ENCODER_DIR

    @lru_cache()
    def bytes_to_unicode():
        """
        Returns list of utf-8 byte and a corresponding list of unicode strings.
        The reversible bpe codes work on unicode strings.
        This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.
        When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.
        This is a signficant percentage of your normal, say, 32K bpe vocab.
        To avoid that, we want lookup tables between utf-8 bytes and unicode strings.
        And avoids mapping to whitespace/control characters the bpe code barfs on.
        """
        bs = list(range(ord("!"), ord("~")+1))+list(range(ord("¡"), ord("¬")+1))+list(range(ord("®"), ord("ÿ")+1))
        cs = bs[:]
        n = 0
        for b in range(2**8):
            if b not in bs:
                bs.append(b)
                cs.append(2**8+n)
                n += 1
        cs = [chr(n) for n in cs]
        return dict(zip(bs, cs))

    def get_pairs(word):
        """Return set of symbol pairs in a word.

        Word is represented as tuple of symbols (symbols being variable-length strings).
        """
        pairs = set()
        prev_char = word[0]
        for char in word[1:]:
            pairs.add((prev_char, char))
            prev_char = char
        return pairs

    class Encoder:
        def __init__(self, encoder, bpe_merges, errors='replace'):
            self.encoder = encoder
            self.decoder = {v:k for k,v in self.encoder.items()}
            self.errors = errors # how to handle errors in decoding
            self.byte_encoder = bytes_to_unicode()
            self.byte_decoder = {v:k for k, v in self.byte_encoder.items()}
            self.bpe_ranks = dict(zip(bpe_merges, range(len(bpe_merges))))
            self.cache = {}

            # Should haved added re.IGNORECASE so BPE merges can happen for capitalized versions of contractions
            self.pat = re.compile(r"""'s|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+""")

        def bpe(self, token):
            if token in self.cache:
                return self.cache[token]
            word = tuple(token)
            pairs = get_pairs(word)

            if not pairs:
                return token

            while True:
                bigram = min(pairs, key = lambda pair: self.bpe_ranks.get(pair, float('inf')))
                if bigram not in self.bpe_ranks:
                    break
                first, second = bigram
                new_word = []
                i = 0
                while i < len(word):
                    try:
                        j = word.index(first, i)
                        new_word.extend(word[i:j])
                        i = j
                    except:
                        new_word.extend(word[i:])
                        break

                    if word[i] == first and i < len(word)-1 and word[i+1] == second:
                        new_word.append(first+second)
                        i += 2
                    else:
                        new_word.append(word[i])
                        i += 1
                new_word = tuple(new_word)
                word = new_word
                if len(word) == 1:
                    break
                else:
                    pairs = get_pairs(word)
            word = ' '.join(word)
            self.cache[token] = word
            return word

        def encode(self, text):
            bpe_tokens = []
            for token in re.findall(self.pat, text):
                token = ''.join(self.byte_encoder[b] for b in token.encode('utf-8'))
                bpe_tokens.extend(self.encoder[bpe_token] for bpe_token in self.bpe(token).split(' '))
            return bpe_tokens

        def decode(self, tokens):
            text = ''.join([self.decoder[token] for token in tokens])
            text = bytearray([self.byte_decoder[c] for c in text]).decode('utf-8', errors=self.errors)
            return text

    def get_encoder(tokenizer):
        if tokenizer == Tokenizer.RUGPT3XL:
            models_dir = OFFICIAL_RUGPT3_ENCODER_DIR
            with open(os.path.join(models_dir, 'vocab.json'), 'r') as f:
                encoder = json.load(f)
            with open(os.path.join(models_dir, 'merges.txt'), 'r', encoding="utf-8") as f:
                bpe_data = f.read()
            bpe_merges = [tuple(merge_str.split()) for merge_str in bpe_data.split('\n')[1:-1]]
            return Encoder(
                encoder=encoder,
                bpe_merges=bpe_merges,
            )
        elif tokenizer == Tokenizer.GPT2:
            models_dir = OFFICIAL_GPT2_ENCODER_DIR
            with open(os.path.join(models_dir, 'encoder.json'), 'r') as f:
                encoder = json.load(f)
            with open(os.path.join(models_dir, 'vocab.bpe'), 'r', encoding="utf-8") as f:
                bpe_data = f.read()
            bpe_merges = [tuple(merge_str.split()) for merge_str in bpe_data.split('\n')[1:-1]]
            return Encoder(
                encoder=encoder,
                bpe_merges=bpe_merges,
            )
        else:
            raise NotImplementedError("Only GPT2 and RUGPT3XL tokenizer-wrapping is supported.")

```python

```