# Project 2: Multi-location Transshipment Problem

In this project, we compare Stochastic Gradient Descent Vs. Bender's Decomposition in optimizing a Two-Stage Stochastic LP.

```python
!pip install autotime
!pip install nb-black
!pip install cvxpy
```

    Collecting autotime
      Using cached autotime-0.1.5-py3-none-any.whl (46 kB)
    Requirement already satisfied: statsmodels>=0.11.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (0.12.2)
    Requirement already satisfied: plotly>=4.5.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (4.14.3)
    Requirement already satisfied: scikit-learn>=0.22.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (0.24.2)
    Requirement already satisfied: seaborn>=0.9.0 in /home/fpga/.local/lib/python3.7/site-packages (from autotime) (0.11.0)
    Requirement already satisfied: pandas==0.25.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (0.25.3)
    Collecting dnntime>=0.3.4
      Using cached dnntime-0.4.1-py3-none-any.whl (47 kB)
    Requirement already satisfied: numpy>=1.18 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (1.20.2)
    Requirement already satisfied: art>=4.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (5.1)
    Collecting fbprophet>=0.5
      Using cached fbprophet-0.7.1.tar.gz (64 kB)
    Requirement already satisfied: tscv>=0.0.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (0.1.2)
    Requirement already satisfied: matplotlib>=3.1.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from autotime) (3.4.1)
    Collecting pyflux>=0.4.0
      Using cached pyflux-0.4.15.tar.gz (1.3 MB)
    Requirement already satisfied: scipy>=1.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from statsmodels>=0.11.0->autotime) (1.6.3)
    Requirement already satisfied: patsy>=0.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from statsmodels>=0.11.0->autotime) (0.5.1)
    Requirement already satisfied: six in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from plotly>=4.5.0->autotime) (1.15.0)
    Requirement already satisfied: retrying>=1.3.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from plotly>=4.5.0->autotime) (1.3.3)
    Requirement already satisfied: joblib>=0.11 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from scikit-learn>=0.22.0->autotime) (1.0.1)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from scikit-learn>=0.22.0->autotime) (2.1.0)
    Requirement already satisfied: python-dateutil>=2.6.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pandas==0.25.3->autotime) (2.8.1)
    Requirement already satisfied: pytz>=2017.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pandas==0.25.3->autotime) (2021.1)
    Requirement already satisfied: nbformat>=5.0.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from dnntime>=0.3.4->autotime) (5.1.3)
    Requirement already satisfied: tensorflow>=2.1.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from dnntime>=0.3.4->autotime) (2.4.1)
    Requirement already satisfied: pyyaml>=5.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from dnntime>=0.3.4->autotime) (5.4.1)
    Requirement already satisfied: Cython>=0.22 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (0.29.23)
    Requirement already satisfied: cmdstanpy==0.9.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (0.9.5)
    Requirement already satisfied: pystan>=2.14 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (3.0.2)
    Requirement already satisfied: LunarCalendar>=0.0.9 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (0.0.9)
    Requirement already satisfied: convertdate>=2.1.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (2.3.2)
    Requirement already satisfied: holidays>=0.10.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (0.11.1)
    Requirement already satisfied: setuptools-git>=1.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (1.2)
    Requirement already satisfied: tqdm>=4.36.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from fbprophet>=0.5->autotime) (4.60.0)
    Requirement already satisfied: cycler>=0.10 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from matplotlib>=3.1.3->autotime) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from matplotlib>=3.1.3->autotime) (1.3.1)
    Requirement already satisfied: pillow>=6.2.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from matplotlib>=3.1.3->autotime) (8.2.0)
    Requirement already satisfied: pyparsing>=2.2.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from matplotlib>=3.1.3->autotime) (2.4.7)
    Requirement already satisfied: numdifftools in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pyflux>=0.4.0->autotime) (0.9.39)
    Requirement already satisfied: ipython-genutils in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from nbformat>=5.0.4->dnntime>=0.3.4->autotime) (0.2.0)
    Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from nbformat>=5.0.4->dnntime>=0.3.4->autotime) (3.2.0)
    Requirement already satisfied: traitlets>=4.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from nbformat>=5.0.4->dnntime>=0.3.4->autotime) (5.0.5)
    Requirement already satisfied: jupyter-core in /home/fpga/.local/lib/python3.7/site-packages (from nbformat>=5.0.4->dnntime>=0.3.4->autotime) (4.7.0)
    Requirement already satisfied: typing-extensions~=3.7.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (3.7.4.3)
    Requirement already satisfied: wheel~=0.35 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.36.2)
    Requirement already satisfied: astunparse~=1.6.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.6.3)
    Requirement already satisfied: tensorflow-estimator<2.5.0,>=2.4.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2.4.0)
    Requirement already satisfied: wrapt~=1.12.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.12.1)
    Requirement already satisfied: tensorboard~=2.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2.5.0)
    Requirement already satisfied: protobuf>=3.9.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (3.15.8)
    Requirement already satisfied: h5py~=2.10.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2.10.0)
    Requirement already satisfied: opt-einsum~=3.3.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (3.3.0)
    Requirement already satisfied: termcolor~=1.1.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.1.0)
    Requirement already satisfied: absl-py~=0.10 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.12.0)
    Requirement already satisfied: grpcio~=1.32.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.32.0)
    Requirement already satisfied: google-pasta~=0.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.2.0)
    Requirement already satisfied: flatbuffers~=1.12.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.12)
    Requirement already satisfied: gast==0.3.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.3.3)
    Requirement already satisfied: keras-preprocessing~=1.1.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.1.2)
    Requirement already satisfied: pysimdjson<4.0,>=3.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pystan>=2.14->fbprophet>=0.5->autotime) (3.2.0)
    Requirement already satisfied: httpstan<4.5,>=4.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pystan>=2.14->fbprophet>=0.5->autotime) (4.4.2)
    Requirement already satisfied: clikit<0.7,>=0.6 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pystan>=2.14->fbprophet>=0.5->autotime) (0.6.2)
    Requirement already satisfied: aiohttp<4.0,>=3.6 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pystan>=2.14->fbprophet>=0.5->autotime) (3.7.4.post0)
    Requirement already satisfied: ephem>=3.7.5.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from LunarCalendar>=0.0.9->fbprophet>=0.5->autotime) (3.7.7.1)
    Requirement already satisfied: pymeeus<=1,>=0.3.13 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from convertdate>=2.1.2->fbprophet>=0.5->autotime) (0.5.11)
    Requirement already satisfied: hijri-converter in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from holidays>=0.10.2->fbprophet>=0.5->autotime) (2.1.1)
    Requirement already satisfied: korean-lunar-calendar in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from holidays>=0.10.2->fbprophet>=0.5->autotime) (0.2.1)
    Requirement already satisfied: pyrsistent>=0.14.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=5.0.4->dnntime>=0.3.4->autotime) (0.17.3)
    Requirement already satisfied: attrs>=17.4.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=5.0.4->dnntime>=0.3.4->autotime) (20.3.0)
    Requirement already satisfied: importlib-metadata; python_version < "3.8" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=5.0.4->dnntime>=0.3.4->autotime) (3.10.0)
    Requirement already satisfied: setuptools in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=5.0.4->dnntime>=0.3.4->autotime) (52.0.0.post20210125)
    Requirement already satisfied: werkzeug>=0.11.15 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.0.1)
    Requirement already satisfied: google-auth<2,>=1.6.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.30.0)
    Requirement already satisfied: markdown>=2.6.8 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (3.3.4)
    Requirement already satisfied: requests<3,>=2.21.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2.25.1)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.8.0)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.6.0)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.4.4)
    Requirement already satisfied: webargs<8.0,>=7.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from httpstan<4.5,>=4.4->pystan>=2.14->fbprophet>=0.5->autotime) (7.0.1)
    Requirement already satisfied: appdirs<2.0,>=1.4 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from httpstan<4.5,>=4.4->pystan>=2.14->fbprophet>=0.5->autotime) (1.4.4)
    Requirement already satisfied: marshmallow<4.0,>=3.10 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from httpstan<4.5,>=4.4->pystan>=2.14->fbprophet>=0.5->autotime) (3.11.1)
    Requirement already satisfied: lz4<4.0,>=3.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from httpstan<4.5,>=4.4->pystan>=2.14->fbprophet>=0.5->autotime) (3.1.3)
    Requirement already satisfied: pastel<0.3.0,>=0.2.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from clikit<0.7,>=0.6->pystan>=2.14->fbprophet>=0.5->autotime) (0.2.1)
    Requirement already satisfied: crashtest<0.4.0,>=0.3.0; python_version >= "3.6" and python_version < "4.0" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from clikit<0.7,>=0.6->pystan>=2.14->fbprophet>=0.5->autotime) (0.3.1)
    Requirement already satisfied: pylev<2.0,>=1.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from clikit<0.7,>=0.6->pystan>=2.14->fbprophet>=0.5->autotime) (1.3.0)
    Requirement already satisfied: yarl<2.0,>=1.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from aiohttp<4.0,>=3.6->pystan>=2.14->fbprophet>=0.5->autotime) (1.6.3)
    Requirement already satisfied: multidict<7.0,>=4.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from aiohttp<4.0,>=3.6->pystan>=2.14->fbprophet>=0.5->autotime) (5.1.0)
    Requirement already satisfied: async-timeout<4.0,>=3.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from aiohttp<4.0,>=3.6->pystan>=2.14->fbprophet>=0.5->autotime) (3.0.1)
    Requirement already satisfied: chardet<5.0,>=2.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from aiohttp<4.0,>=3.6->pystan>=2.14->fbprophet>=0.5->autotime) (4.0.0)
    Requirement already satisfied: zipp>=0.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema!=2.5.0,>=2.4->nbformat>=5.0.4->dnntime>=0.3.4->autotime) (3.4.1)
    Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (4.7.2)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (4.2.2)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.2.8)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.26.4)
    Requirement already satisfied: idna<3,>=2.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2.10)
    Requirement already satisfied: certifi>=2017.4.17 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (2020.12.5)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (1.3.0)
    Requirement already satisfied: pyasn1>=0.1.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from rsa<5,>=3.1.4; python_version >= "3.6"->google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (0.4.8)
    Requirement already satisfied: oauthlib>=3.0.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow>=2.1.0->dnntime>=0.3.4->autotime) (3.1.0)
    Building wheels for collected packages: fbprophet, pyflux
      Building wheel for fbprophet (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: /opt/anaconda3/envs/jeff/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-46ektaws
           cwd: /tmp/pip-install-cyipebue/fbprophet/
      Complete output (38 lines):
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib
      creating build/lib/fbprophet
      creating build/lib/fbprophet/stan_model
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 149, in <module>
          long_description_content_type='text/markdown',
        File "/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 48, in run
          build_models(target_dir)
        File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 38, in build_models
          StanBackendEnum.get_backend_class(backend).build_model(target_dir, MODEL_DIR)
        File "/tmp/pip-install-cyipebue/fbprophet/fbprophet/models.py", line 209, in build_model
          import pystan
      ModuleNotFoundError: No module named 'pystan'
      ----------------------------------------
      ERROR: Failed building wheel for fbprophet
      Running setup.py clean for fbprophet
      Building wheel for pyflux (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: /opt/anaconda3/envs/jeff/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cyipebue/pyflux/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cyipebue/pyflux/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-qzqjec6v
           cwd: /tmp/pip-install-cyipebue/pyflux/
      Complete output (628 lines):
      running bdist_wheel
      running build
      running config_cc
      unifing config_cc, config, build_clib, build_ext, build commands --compiler options
      running config_fc
      unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
      running build_src
      build_src
      building extension "pyflux.__check_build._check_build" sources
      building extension "pyflux.arma.arma_recursions" sources
      building extension "pyflux.arma.nn_architecture" sources
      building extension "pyflux.families.gas_recursions" sources
      building extension "pyflux.families.poisson_kalman_recursions" sources
      building extension "pyflux.garch.garch_recursions" sources
      building extension "pyflux.gas.gas_core_recursions" sources
      building extension "pyflux.gpnarx.kernel_routines" sources
      building extension "pyflux.inference.metropolis_sampler" sources
      building extension "pyflux.inference.bbvi_routines" sources
      building extension "pyflux.ssm.kalman" sources
      building extension "pyflux.var.var_recursions" sources
      build_src: building npy-pkg config files
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/results.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/latent_variables.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/setup.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/data_check.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/__init__.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/tsm.py -> build/lib.linux-x86_64-3.7/pyflux
      copying pyflux/covariances.py -> build/lib.linux-x86_64-3.7/pyflux
      creating build/lib.linux-x86_64-3.7/pyflux/__check_build
      copying pyflux/__check_build/setup.py -> build/lib.linux-x86_64-3.7/pyflux/__check_build
      copying pyflux/__check_build/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/__check_build
      creating build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/nnar.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/nnarx.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/arimax.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/setup.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      copying pyflux/arma/arma.py -> build/lib.linux-x86_64-3.7/pyflux/arma
      creating build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_t.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_poisson.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_cauchy.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_inference.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_normal.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_skewt.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_cauchy.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_exponential.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_laplace.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_poisson.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_laplace.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_t.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_exponential.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_inference.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arimax_normal.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      copying pyflux/arma/tests/test_arima_skewt.py -> build/lib.linux-x86_64-3.7/pyflux/arma/tests
      creating build/lib.linux-x86_64-3.7/pyflux/ensembles
      copying pyflux/ensembles/mixture_of_experts.py -> build/lib.linux-x86_64-3.7/pyflux/ensembles
      copying pyflux/ensembles/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/ensembles
      creating build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/t.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/exponential.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/family.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/laplace.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/inverse_gamma.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/cauchy.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/inverse_wishart.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/normal.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/poisson.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/setup.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/flat.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/truncated_normal.py -> build/lib.linux-x86_64-3.7/pyflux/families
      copying pyflux/families/skewt.py -> build/lib.linux-x86_64-3.7/pyflux/families
      creating build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/egarch.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/segarchm.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/garch.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/egarchmreg.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/setup.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/lmegarch.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/egarchm.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      copying pyflux/garch/segarch.py -> build/lib.linux-x86_64-3.7/pyflux/garch
      creating build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/scores.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gas.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gasllt.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gasx.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/setup.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gasllm.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gasreg.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      copying pyflux/gas/gasrank.py -> build/lib.linux-x86_64-3.7/pyflux/gas
      creating build/lib.linux-x86_64-3.7/pyflux/gpnarx
      copying pyflux/gpnarx/gpnarx.py -> build/lib.linux-x86_64-3.7/pyflux/gpnarx
      copying pyflux/gpnarx/kernels.py -> build/lib.linux-x86_64-3.7/pyflux/gpnarx
      copying pyflux/gpnarx/setup.py -> build/lib.linux-x86_64-3.7/pyflux/gpnarx
      copying pyflux/gpnarx/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/gpnarx
      creating build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/norm_post_sim.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/stoch_optim.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/metropolis_hastings.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/bbvi.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/setup.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      copying pyflux/inference/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/inference
      creating build/lib.linux-x86_64-3.7/pyflux/output
      copying pyflux/output/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/output
      copying pyflux/output/tableprinter.py -> build/lib.linux-x86_64-3.7/pyflux/output
      creating build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/dynlin.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/nllt.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/dynamic_glm.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/local_level.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/local_trend.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/llt.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/setup.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/ndynlin.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/nllm.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/dar.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      copying pyflux/ssm/llm.py -> build/lib.linux-x86_64-3.7/pyflux/ssm
      creating build/lib.linux-x86_64-3.7/pyflux/tests
      copying pyflux/tests/nhst.py -> build/lib.linux-x86_64-3.7/pyflux/tests
      copying pyflux/tests/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/tests
      creating build/lib.linux-x86_64-3.7/pyflux/var
      copying pyflux/var/var.py -> build/lib.linux-x86_64-3.7/pyflux/var
      copying pyflux/var/setup.py -> build/lib.linux-x86_64-3.7/pyflux/var
      copying pyflux/var/__init__.py -> build/lib.linux-x86_64-3.7/pyflux/var
      running build_ext
      customize UnixCCompiler
      customize UnixCCompiler using build_ext
      CCompilerOpt.cc_test_flags[999] : testing flags (-march=native)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      creating /tmp/tmp0x9e4tz_/opt
      creating /tmp/tmp0x9e4tz_/opt/anaconda3
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils
      creating /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks
      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-march=native'
      CCompilerOpt.cc_test_flags[999] : testing flags (-O3)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-O3'
      CCompilerOpt.cc_test_flags[999] : testing flags (-Werror)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-Werror'
      CCompilerOpt.__init__[1674] : check requested baseline
      CCompilerOpt.cc_test_flags[999] : testing flags (-msse)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse'
      CCompilerOpt.cc_test_flags[999] : testing flags (-msse2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse2'
      CCompilerOpt.feature_test[1446] : testing feature 'SSE' with flags (-msse -msse2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'SSE2' with flags (-msse -msse2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-msse3)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse3'
      CCompilerOpt.feature_test[1446] : testing feature 'SSE3' with flags (-msse -msse2 -msse3)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -Werror'
      CCompilerOpt.__init__[1683] : check requested dispatch-able features
      CCompilerOpt.cc_test_flags[999] : testing flags (-mssse3)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mssse3'
      CCompilerOpt.cc_test_flags[999] : testing flags (-msse4.1)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse4.1'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mpopcnt)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mpopcnt'
      CCompilerOpt.cc_test_flags[999] : testing flags (-msse4.2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse4.2'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'SSSE3' with flags (-msse -msse2 -msse3 -mssse3)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'POPCNT' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'SSE41' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mf16c)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mf16c'
      CCompilerOpt.feature_test[1446] : testing feature 'F16C' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'SSE42' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx2'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX2' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mavx2)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mavx2 -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mfma)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mfma'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512f)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512f'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512cd)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512cd'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512vl -mavx512bw -mavx512dq)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512vl -mavx512bw -mavx512dq'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512ifma -mavx512vbmi)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512ifma -mavx512vbmi'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512_CNL' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'FMA3' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512_SKX' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512CD' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -Werror'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512F' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512vnni)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512vnni'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.o.d -mavx512vnni) failed with exit status 1 output ->
      gcc: error: unrecognized command line option ‘-mavx512vnni’

      CCompilerOpt.cc_test_flags[1003] : testing failed
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512_CLX' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.o.d -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror) failed with exit status 1 output ->
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.c: In function ‘main’:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.c:6:17: error: implicit declaration of function ‘_mm512_dpbusd_epi32’ [-Werror=implicit-function-declaration]
           __m512i a = _mm512_dpbusd_epi32(_mm512_setzero_si512(), _mm512_setzero_si512(), _mm512_setzero_si512());
                       ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_clx.c:6:17: error: incompatible types when initializing type ‘__m512i {aka __vector(8) long long int}’ using type ‘int’
      cc1: all warnings being treated as errors

      CCompilerOpt.feature_test[1458] : testing failed
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx512er -mavx512pf)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx512er -mavx512pf'
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512_KNL' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512er -mavx512pf)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512er -mavx512pf -Werror'
      CCompilerOpt.cc_test_flags[999] : testing flags (-mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/test_flags.o.d -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq) failed with exit status 1 output ->
      gcc: error: unrecognized command line option ‘-mavx5124fmaps’
      gcc: error: unrecognized command line option ‘-mavx5124vnniw’
      gcc: error: unrecognized command line option ‘-mavx512vpopcntdq’

      CCompilerOpt.cc_test_flags[1003] : testing failed
      CCompilerOpt.feature_test[1446] : testing feature 'AVX512_KNM' with flags (-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512er -mavx512pf)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512er -mavx512pf -Werror'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.o.d -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512er -mavx512pf -Werror) failed with exit status 1 output ->
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c: In function ‘main’:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:9:9: error: implicit declaration of function ‘_mm512_4fmadd_ps’ [-Werror=implicit-function-declaration]
           b = _mm512_4fmadd_ps(b, b, b, b, b, NULL);
               ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:9:7: error: incompatible types when assigning to type ‘__m512 {aka __vector(16) float}’ from type ‘int’
           b = _mm512_4fmadd_ps(b, b, b, b, b, NULL);
             ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:11:9: error: implicit declaration of function ‘_mm512_4dpwssd_epi32’ [-Werror=implicit-function-declaration]
           a = _mm512_4dpwssd_epi32(a, a, a, a, a, NULL);
               ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:11:7: error: incompatible types when assigning to type ‘__m512i {aka __vector(8) long long int}’ from type ‘int’
           a = _mm512_4dpwssd_epi32(a, a, a, a, a, NULL);
             ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:13:9: error: implicit declaration of function ‘_mm512_popcnt_epi64’ [-Werror=implicit-function-declaration]
           a = _mm512_popcnt_epi64(a);
               ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/cpu_avx512_knm.c:13:7: error: incompatible types when assigning to type ‘__m512i {aka __vector(8) long long int}’ from type ‘int’
           a = _mm512_popcnt_epi64(a);
             ^
      cc1: all warnings being treated as errors

      CCompilerOpt.feature_test[1458] : testing failed
      CCompilerOpt.__init__[1696] : skip features (SSE3 SSE SSE2) since its part of baseline
      CCompilerOpt.__init__[1699] : initialize targets groups
      CCompilerOpt.__init__[1701] : parse target group simd_test
      CCompilerOpt._parse_target_tokens[1912] : skip targets (VSX3 ASIMD VSX FMA4 VSX2 XOP NEON) not part of baseline or dispatch-able features
      CCompilerOpt._parse_policy_not_keepbase[2022] : skip baseline features (SSE2)
      CCompilerOpt.generate_dispatch_header[2236] : generate CPU dispatch header: (build/src.linux-x86_64-3.7/numpy/distutils/include/npy_cpu_dispatch_config.h)
      CCompilerOpt.generate_dispatch_header[2247] : dispatch header dir build/src.linux-x86_64-3.7/numpy/distutils/include does not exist, creating it
      CCompilerOpt.feature_extra_checks[1519] : Testing extra checks for feature 'AVX512F' (AVX512F_REDUCE)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -Werror'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.o.d -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -Werror) failed with exit status 1 output ->
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c: In function ‘main’:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:13:21: error: implicit declaration of function ‘_mm512_reduce_add_ps’ [-Werror=implicit-function-declaration]
           float sum_ps  = _mm512_reduce_add_ps(one_ps);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:14:21: error: implicit declaration of function ‘_mm512_reduce_add_pd’ [-Werror=implicit-function-declaration]
           double sum_pd = _mm512_reduce_add_pd(one_pd);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:15:26: error: implicit declaration of function ‘_mm512_reduce_add_epi64’ [-Werror=implicit-function-declaration]
           int sum_int   = (int)_mm512_reduce_add_epi64(one_i64);
                                ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:16:26: error: implicit declaration of function ‘_mm512_reduce_add_epi32’ [-Werror=implicit-function-declaration]
               sum_int  += (int)_mm512_reduce_add_epi32(one_i64);
                                ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:18:16: error: implicit declaration of function ‘_mm512_reduce_mul_ps’ [-Werror=implicit-function-declaration]
           sum_ps  += _mm512_reduce_mul_ps(one_ps);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:19:16: error: implicit declaration of function ‘_mm512_reduce_mul_pd’ [-Werror=implicit-function-declaration]
           sum_pd  += _mm512_reduce_mul_pd(one_pd);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:20:21: error: implicit declaration of function ‘_mm512_reduce_mul_epi64’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_mul_epi64(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:21:21: error: implicit declaration of function ‘_mm512_reduce_mul_epi32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_mul_epi32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:23:16: error: implicit declaration of function ‘_mm512_reduce_min_ps’ [-Werror=implicit-function-declaration]
           sum_ps  += _mm512_reduce_min_ps(one_ps);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:24:16: error: implicit declaration of function ‘_mm512_reduce_min_pd’ [-Werror=implicit-function-declaration]
           sum_pd  += _mm512_reduce_min_pd(one_pd);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:25:21: error: implicit declaration of function ‘_mm512_reduce_min_epi32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_min_epi32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:26:21: error: implicit declaration of function ‘_mm512_reduce_min_epu32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_min_epu32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:27:21: error: implicit declaration of function ‘_mm512_reduce_min_epi64’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_min_epi64(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:29:16: error: implicit declaration of function ‘_mm512_reduce_max_ps’ [-Werror=implicit-function-declaration]
           sum_ps  += _mm512_reduce_max_ps(one_ps);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:30:16: error: implicit declaration of function ‘_mm512_reduce_max_pd’ [-Werror=implicit-function-declaration]
           sum_pd  += _mm512_reduce_max_pd(one_pd);
                      ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:31:21: error: implicit declaration of function ‘_mm512_reduce_max_epi32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_max_epi32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:32:21: error: implicit declaration of function ‘_mm512_reduce_max_epu32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_max_epu32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:33:21: error: implicit declaration of function ‘_mm512_reduce_max_epi64’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_max_epi64(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:35:21: error: implicit declaration of function ‘_mm512_reduce_and_epi32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_and_epi32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:36:21: error: implicit declaration of function ‘_mm512_reduce_and_epi64’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_and_epi64(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:38:21: error: implicit declaration of function ‘_mm512_reduce_or_epi32’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_or_epi32(one_i64);
                           ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512f_reduce.c:39:21: error: implicit declaration of function ‘_mm512_reduce_or_epi64’ [-Werror=implicit-function-declaration]
           sum_int += (int)_mm512_reduce_or_epi64(one_i64);
                           ^
      cc1: all warnings being treated as errors

      CCompilerOpt.feature_extra_checks[1537] : testing failed for checks (AVX512F_REDUCE)
      CCompilerOpt.feature_extra_checks[1519] : Testing extra checks for feature 'AVX512_SKX' (AVX512BW_MASK AVX512DQ_MASK)
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.o.d -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror) failed with exit status 1 output ->
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c: In function ‘main’:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c:12:11: error: implicit declaration of function ‘_kor_mask64’ [-Werror=implicit-function-declaration]
           m64 = _kor_mask64(m64, m64);
                 ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c:13:11: error: implicit declaration of function ‘_kxor_mask64’ [-Werror=implicit-function-declaration]
           m64 = _kxor_mask64(m64, m64);
                 ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c:14:11: error: implicit declaration of function ‘_cvtu64_mask64’ [-Werror=implicit-function-declaration]
           m64 = _cvtu64_mask64(_cvtmask64_u64(m64));
                 ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512bw_mask.c:14:26: error: implicit declaration of function ‘_cvtmask64_u64’ [-Werror=implicit-function-declaration]
           m64 = _cvtu64_mask64(_cvtmask64_u64(m64));
                                ^
      cc1: all warnings being treated as errors

      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      compile options: '-I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror'
      CCompilerOpt.dist_test[576] : CCompilerOpt._dist_test_spawn[713] : Command (gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/include/python3.7m -c /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c -o /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.o -MMD -MF /tmp/tmp0x9e4tz_/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.o.d -msse -msse2 -msse3 -mssse3 -msse4.1 -mpopcnt -msse4.2 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -Werror) failed with exit status 1 output ->
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c: In function ‘main’:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c:12:10: error: implicit declaration of function ‘_kor_mask8’ [-Werror=implicit-function-declaration]
           m8 = _kor_mask8(m8, m8);
                ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c:13:10: error: implicit declaration of function ‘_kxor_mask8’ [-Werror=implicit-function-declaration]
           m8 = _kxor_mask8(m8, m8);
                ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c:14:10: error: implicit declaration of function ‘_cvtu32_mask8’ [-Werror=implicit-function-declaration]
           m8 = _cvtu32_mask8(_cvtmask8_u32(m8));
                ^
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/distutils/checks/extra_avx512dq_mask.c:14:24: error: implicit declaration of function ‘_cvtmask8_u32’ [-Werror=implicit-function-declaration]
           m8 = _cvtu32_mask8(_cvtmask8_u32(m8));
                              ^
      cc1: all warnings being treated as errors

      CCompilerOpt.feature_extra_checks[1537] : testing failed for checks (AVX512BW_MASK AVX512DQ_MASK)
      Detected changes on compiler optimizations, force rebuilding
      building 'pyflux.__check_build._check_build' extension
      compiling C sources
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      creating build/temp.linux-x86_64-3.7/pyflux
      creating build/temp.linux-x86_64-3.7/pyflux/__check_build
      compile options: '-I/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/distutils/include -I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3'
      gcc: pyflux/__check_build/_check_build.c
      gcc -pthread -shared -B /opt/anaconda3/envs/jeff/compiler_compat -L/opt/anaconda3/envs/jeff/lib -Wl,-rpath=/opt/anaconda3/envs/jeff/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/pyflux/__check_build/_check_build.o -o build/lib.linux-x86_64-3.7/pyflux/__check_build/_check_build.cpython-37m-x86_64-linux-gnu.so
      building 'pyflux.arma.arma_recursions' extension
      compiling C sources
      C compiler: gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

      creating build/temp.linux-x86_64-3.7/pyflux/arma
      compile options: '-I/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/distutils/include -I/opt/anaconda3/envs/jeff/include/python3.7m -c'
      extra options: '-msse -msse2 -msse3'
      gcc: pyflux/arma/arma_recursions.c
      In file included from /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944:0,
                       from /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                       from /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                       from pyflux/arma/arma_recursions.c:274:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
       #warning "Using deprecated NumPy API, disable it with " \
        ^
      pyflux/arma/arma_recursions.c: In function ‘__Pyx__ExceptionSave’:
      pyflux/arma/arma_recursions.c:19331:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           *type = tstate->exc_type;
                         ^
      pyflux/arma/arma_recursions.c:19332:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           *value = tstate->exc_value;
                          ^
      pyflux/arma/arma_recursions.c:19333:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           *tb = tstate->exc_traceback;
                       ^
      pyflux/arma/arma_recursions.c: In function ‘__Pyx__ExceptionReset’:
      pyflux/arma/arma_recursions.c:19340:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tmp_type = tstate->exc_type;
                            ^
      pyflux/arma/arma_recursions.c:19341:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tmp_value = tstate->exc_value;
                             ^
      pyflux/arma/arma_recursions.c:19342:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tmp_tb = tstate->exc_traceback;
                          ^
      pyflux/arma/arma_recursions.c:19343:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tstate->exc_type = type;
                 ^
      pyflux/arma/arma_recursions.c:19344:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tstate->exc_value = value;
                 ^
      pyflux/arma/arma_recursions.c:19345:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tstate->exc_traceback = tb;
                 ^
      pyflux/arma/arma_recursions.c: In function ‘__Pyx__GetException’:
      pyflux/arma/arma_recursions.c:19400:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tmp_type = tstate->exc_type;
                            ^
      pyflux/arma/arma_recursions.c:19401:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tmp_value = tstate->exc_value;
                             ^
      pyflux/arma/arma_recursions.c:19402:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tmp_tb = tstate->exc_traceback;
                          ^
      pyflux/arma/arma_recursions.c:19403:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tstate->exc_type = local_type;
                 ^
      pyflux/arma/arma_recursions.c:19404:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tstate->exc_value = local_value;
                 ^
      pyflux/arma/arma_recursions.c:19405:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tstate->exc_traceback = local_tb;
                 ^
      pyflux/arma/arma_recursions.c: In function ‘__Pyx__ExceptionSwap’:
      pyflux/arma/arma_recursions.c:19427:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tmp_type = tstate->exc_type;
                            ^
      pyflux/arma/arma_recursions.c:19428:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tmp_value = tstate->exc_value;
                             ^
      pyflux/arma/arma_recursions.c:19429:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tmp_tb = tstate->exc_traceback;
                          ^
      pyflux/arma/arma_recursions.c:19430:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
           tstate->exc_type = *type;
                 ^
      pyflux/arma/arma_recursions.c:19431:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
           tstate->exc_value = *value;
                 ^
      pyflux/arma/arma_recursions.c:19432:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
           tstate->exc_traceback = *tb;
                 ^
      In file included from /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:21:0,
                       from /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                       from pyflux/arma/arma_recursions.c:274:
      pyflux/arma/arma_recursions.c: At top level:
      /opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include/numpy/__multiarray_api.h:1464:1: warning: ‘_import_array’ defined but not used [-Wunused-function]
       _import_array(void)
       ^
      error: Command "gcc -pthread -B /opt/anaconda3/envs/jeff/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/distutils/include -I/opt/anaconda3/envs/jeff/include/python3.7m -c pyflux/arma/arma_recursions.c -o build/temp.linux-x86_64-3.7/pyflux/arma/arma_recursions.o -MMD -MF build/temp.linux-x86_64-3.7/pyflux/arma/arma_recursions.o.d -msse -msse2 -msse3" failed with exit status 1

      ########### EXT COMPILER OPTIMIZATION ###########
      Platform      :
        Architecture: x64
        Compiler    : gcc

      CPU baseline  :
        Requested   : 'min'
        Enabled     : SSE SSE2 SSE3
        Flags       : -msse -msse2 -msse3
        Extra checks: none

      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CNL
        Generated   : none
      CCompilerOpt._cache_write[796] : write cache to path -> /tmp/pip-install-cyipebue/pyflux/build/temp.linux-x86_64-3.7/ccompiler_opt_cache_ext.py
      ----------------------------------------
      ERROR: Failed building wheel for pyflux
      Running setup.py clean for pyflux
    Failed to build fbprophet pyflux
    Installing collected packages: fbprophet, dnntime, pyflux, autotime
        Running setup.py install for fbprophet ... error
        ERROR: Command errored out with exit status 1:
         command: /opt/anaconda3/envs/jeff/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-7vad5ilk/install-record.txt --single-version-externally-managed --compile --install-headers /opt/anaconda3/envs/jeff/include/python3.7m/fbprophet
             cwd: /tmp/pip-install-cyipebue/fbprophet/
        Complete output (40 lines):
        running install
        running build
        running build_py
        creating build
        creating build/lib
        creating build/lib/fbprophet
        creating build/lib/fbprophet/stan_model
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 149, in <module>
            long_description_content_type='text/markdown',
          File "/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
            return distutils.core.setup(**attrs)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/core.py", line 148, in setup
            dist.run_commands()
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 966, in run_commands
            self.run_command(cmd)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/opt/anaconda3/envs/jeff/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
            return orig.install.run(self)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/command/install.py", line 545, in run
            self.run_command('build')
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/command/build.py", line 135, in run
            self.run_command(cmd_name)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/opt/anaconda3/envs/jeff/lib/python3.7/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 48, in run
            build_models(target_dir)
          File "/tmp/pip-install-cyipebue/fbprophet/setup.py", line 38, in build_models
            StanBackendEnum.get_backend_class(backend).build_model(target_dir, MODEL_DIR)
          File "/tmp/pip-install-cyipebue/fbprophet/fbprophet/models.py", line 209, in build_model
            import pystan
        ModuleNotFoundError: No module named 'pystan'
        ----------------------------------------
    ERROR: Command errored out with exit status 1: /opt/anaconda3/envs/jeff/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cyipebue/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-7vad5ilk/install-record.txt --single-version-externally-managed --compile --install-headers /opt/anaconda3/envs/jeff/include/python3.7m/fbprophet Check the logs for full command output.
    WARNING: You are using pip version 20.2.4; however, version 21.1 is available.
    You should consider upgrading via the '/opt/anaconda3/envs/jeff/bin/python -m pip install --upgrade pip' command.
    Requirement already satisfied: nb-black in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (1.0.7)
    Requirement already satisfied: black>='19.3'; python_version >= "3.6" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from nb-black) (21.4b2)
    Requirement already satisfied: ipython in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from nb-black) (7.22.0)
    Requirement already satisfied: regex>=2020.1.8 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (2021.4.4)
    Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (3.7.4.3)
    Requirement already satisfied: click>=7.1.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (7.1.2)
    Requirement already satisfied: pathspec<1,>=0.8.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (0.8.1)
    Requirement already satisfied: mypy-extensions>=0.4.3 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (0.4.3)
    Requirement already satisfied: toml>=0.10.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (0.10.2)
    Requirement already satisfied: appdirs in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (1.4.4)
    Requirement already satisfied: typed-ast>=1.4.2; python_version < "3.8" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from black>='19.3'; python_version >= "3.6"->nb-black) (1.4.3)
    Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (3.0.18)
    Requirement already satisfied: traitlets>=4.2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (5.0.5)
    Requirement already satisfied: jedi>=0.16 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (0.18.0)
    Requirement already satisfied: backcall in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (0.2.0)
    Requirement already satisfied: pygments in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (2.8.1)
    Requirement already satisfied: pickleshare in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (0.7.5)
    Requirement already satisfied: decorator in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (5.0.7)
    Requirement already satisfied: pexpect>4.3; sys_platform != "win32" in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (4.8.0)
    Requirement already satisfied: setuptools>=18.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from ipython->nb-black) (52.0.0.post20210125)
    Requirement already satisfied: wcwidth in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython->nb-black) (0.2.5)
    Requirement already satisfied: ipython-genutils in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from traitlets>=4.2->ipython->nb-black) (0.2.0)
    Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from jedi>=0.16->ipython->nb-black) (0.8.2)
    Requirement already satisfied: ptyprocess>=0.5 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from pexpect>4.3; sys_platform != "win32"->ipython->nb-black) (0.7.0)
    WARNING: You are using pip version 20.2.4; however, version 21.1 is available.
    You should consider upgrading via the '/opt/anaconda3/envs/jeff/bin/python -m pip install --upgrade pip' command.
    Requirement already satisfied: cvxpy in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (1.1.12)
    Requirement already satisfied: numpy>=1.15 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from cvxpy) (1.20.2)
    Requirement already satisfied: scipy>=1.1.0 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from cvxpy) (1.6.3)
    Requirement already satisfied: scs>=1.1.6 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from cvxpy) (2.1.3)
    Requirement already satisfied: ecos>=2 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from cvxpy) (2.0.7.post1)
    Requirement already satisfied: osqp>=0.4.1 in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from cvxpy) (0.6.2.post0)
    Requirement already satisfied: qdldl in /opt/anaconda3/envs/jeff/lib/python3.7/site-packages (from osqp>=0.4.1->cvxpy) (0.1.5.post0)
    WARNING: You are using pip version 20.2.4; however, version 21.1 is available.
    You should consider upgrading via the '/opt/anaconda3/envs/jeff/bin/python -m pip install --upgrade pip' command.

```python
# %load_ext autotime
# %load_ext nb_black

# General
import time
import numpy as np
import cvxpy as cp
import scipy as sp
import pandas as pd
import itertools

# Ignore warnings LOL
import warnings

warnings.simplefilter("ignore")

# SEED FOR RANDOMNESS
np.random.seed(420)
```

---
## 1. Preface

The type of problem we care about in this project are known as [two-stage stochastic programming](https://en.wikipedia.org/wiki/Stochastic_programming#Two-stage_problems). Let The general formulation of it is as follows (Adapted from Wikipedia):

### 1.1. General Two-Stage Stochastic Program

#### 1.1.1 First Stage Stochastic Program

\begin{align}
    \min_{x\in X} & \{ g(x)= f(x) + E_{\tilde{s}\sim S}[h(x, s=\tilde{s})]\}
\end{align}

, where $h(x, s=\tilde{s})$ is the optimal value of the second-stage problem given a specific scenario $\tilde{s}$. $x \in X$ is our first-stage decision variable vector, $S$ is a probability distribution of the scenario variables that we're sampling from.

#### 1.1.2 Second Stage Stochastic Program

For a specific $\tilde{x}$ and $\tilde{s}$,

\begin{align}
    h(x=\tilde{x}, s=\tilde{s}) = \min_{y} & \{ d(s, y) \,\vert\,C(s, x) +D(s, y) = \xi(s)\}  \\
\end{align}

, where $C, D, \xi$ are functions given a specific scenario $\tilde{s}$. $y$ is our second-stage decision variable vector.

At the first stage we optimize (minimize in the above formulation) the cost of the first-stage decision plus the expected cost of the (optimal) second-stage decision. We can view the second-stage problem simply as an optimization problem which describes our supposedly optimal behavior when the uncertain data is revealed (specific scenario $\tilde{s}$), or we can consider its solution as a recourse action where the term $D(s, y)$ compensates for a possible inconsistency of the system $C(s, x) \leq \xi(s)$ and $d(s, y)$ is the cost of this recourse action.

### 1.2. Two-Stage Stochastic Linear Program

The stochastic linear program below is just a specific instance of the general program above, with very minor changes - e.g. functions of first-stage / second-stage decision variables are no longer general functions, but rather vectors / matrices because of constraints must be linear for a linear program.

#### 1.2.1 First Stage Stochastic Linear Program

\begin{align}
    \min\limits_{x\in \mathbb{R}^n} & f(x)= c^\top x + E_{\tilde{s}\sim S}[h(x, s=\tilde{s})] & \\
    \text{subject to}
    & Ax = b &\\
    & x \geq 0
\end{align}

, where $h(x, s=\tilde{s})$ is the optimal value of the second-stage problem given a specific scenario $\tilde{s}$. $x \in \mathbb{R}^n$ is our first-stage decision variable vector, $S$ is a probability distribution of the scenario variables that we're sampling from., where $h(x, s)$ is the optimal value of the second-stage problem

#### 1.2.2 Second Stage Stochastic Linear Program

For a specific $\tilde{x}$ and $\tilde{s}$,

\begin{align}
    h(x=\tilde{x}, s=\tilde{s}) = \min\limits_{y\in \mathbb{R}^m}\,& d(s)^T y \\
    \text{subject to }\,& C(s)x + D(s)y = \xi(s) \\
    &y \geq 0
\end{align}

, where $C, D, \xi$ are matrices given a specific scenario $\tilde{s}$. $y$ is our second-stage decision variable vector.

At the first stage we optimize (minimize in the above formulation) the cost of the first-stage decision plus the expected cost of the (optimal) second-stage decision. We can view the second-stage problem simply as an optimization problem which describes our supposedly optimal behavior when the uncertain data is revealed (specific scenario $\tilde{s}$), or we can consider its solution as a recourse action where the term $D(s)y$ compensates for a possible inconsistency of the system $C(s)x \leq \xi(s)$ and $d(s)^\top y$ is the cost of this recourse action.

### 1.3. Two-Stage Stochastic Linear Program with Discretized Scenario Distribution

What happens if we can't get / don't want to sample from a continuous distribution for the scenarios? (What if $S$ is discrete instead of continuous?). Nothing much changes other than how we calculate the expectation of the second-stage objective values.

#### 1.3.1 First Stage Stochastic Linear Program with Discretized Scenario Distribution

\begin{align}
    \min\limits_{x\in \mathbb{R}^n}\,& f(x)= c^\top x + \underbrace{\sum_{s = \tilde{s}} p(s = \tilde{s}) h(x, s = \tilde{s})}_{E_{\tilde{s}\sim S}[h(x, s=\tilde{s})]} \\
    \text{subject to }\,
    & Ax = b \\
    & x \geq 0
\end{align}

, where $h(x, s)$ is the optimal value of the second-stage problem, and $p(s = \tilde{s})$ is the probability of the scenario $\tilde{s}$ occuring.

#### 1.3.2 Second Stage Stochastic Linear Program with Discretized Scenario Distribution

For a specific $\tilde{x}$ and $\tilde{s}$,

\begin{align}
    h(x=\tilde{x}, s=\tilde{s}) = \min\limits_{y\in \mathbb{R}^m}\,& d(s)^T y \\
    \text{subject to} & \, C(s)x + D(s)y = \xi(s) \text{, where }C(s), D(s), \xi(s) \\
    & y \geq 0 \\
\end{align}

However, how do we go about solving this two-stage stochastic programs? The problem we face here is We will present two algorithms: Stochastic Gradient Descent and L-Shaped / Bender's Method.

---
## 2. Method: Stochastic Gradient Descent

To keep things simple, we will work on using stochastic gradient descent on a stochastic linear program with s discretized scenario distribution.

First, we recall the general key update formula in gradient descent (adjusting parameter $w_i$ in negative direction of gradient of objective $f$):

\begin{align}
    & w_i \leftarrow w_i + \alpha \frac{\partial f}{\partial w_i}\,\forall i = 0, \cdots , \text{Number of parameters} \\
\end{align}

Let's recall the Lagrangian for the first stage problem :

\begin{align}
    \mathcal{L}(x) = c^\top x + E_{\tilde{s}\sim S}[h(x, s=\tilde{s})] - \mu
\end{align}

Assume we have solved the second stage problem, and have retrieved optimal primal variables $y^*$ and dual variables $\pi^*$ (equality constraints) and $\lambda^*$ (inequality constraints), then recall the Lagrangian of $h(x=\tilde{x}, s=\tilde{s})$:

\begin{align}
  \mathcal{L}(x) &= d(s)^\top y^* - \pi^* \left(C(s)x + D(s)y^* - \xi(s) \right) - \lambda^* y^* \\
\end{align}

To perform gradient descent on our obejctive function $f(x)$, we need to find its gradient / subgradient w.r.t. $x$:

\begin{align}
  f(x) &= c^\top x + \sum_s p_s h_s(x) \\
  \frac{df}{dx} &= c + \frac{d \left(\sum_s p_s h_s(x)\right)}{dx} \\
  &= c + \sum_s p_s\frac{d\left(h_s(x)\right)}{dx} \\
  &= c + \sum_s p_s\frac{d\left(\mathcal{L}(x)\right)}{dx} \\
  &= c + \sum_s p_s\frac{d\left(d^\top y_s - \pi_s \left( Dy_s - \xi_s + C_s x \right) - \lambda_s y_s \right)}{dx} \\
  &= c - \sum_s p_s (\pi^\top_s C_s) \\
\end{align}

Stochastic Gradient Descent:

\begin{align}
  x &= x + \alpha \frac{df}{dx} \\
  &= x + \alpha \left( c - \sum_s p_s (\pi^\top_s C_s) \right) \\
\end{align}

Orthogonal Projection onto [intersection of half-spaces (Convex Polytope)](https://en.wikipedia.org/wiki/Convex_polytope)

\begin{align}
    \arg \min_{x \geq 0} & \frac{1}{2} {\lVert x - y \rVert}_{2}^{2} \\
    \text{subject to } & Ax \leq b
\end{align}

Lagrangian

\begin{align}
    \mathcal{L}(x, \lambda) &= \frac{1}{2} {\lVert x - y \rVert}_{2}^{2} + \lambda \left(Ax  - b \right) \\
\end{align}

Lagrange Dual

\begin{align}
    \mathcal{L}(\lambda)
    &= \inf_{x \geq 0} \mathcal{L}(x, \lambda) \\
    &= \inf_{x \geq 0} \frac{1}{2} {\lVert x - y \rVert}_{2}^{2} + \lambda \left(Ax  - b \right) \\
\end{align}

KKT Conditions

Stationarity:

\begin{align}
    \nabla_x \mathcal{L}(x, \lambda) &= 0 \\
    \nabla_x \frac{1}{2} {\left( x - y \right) }^\top \left( x - y \right) + \lambda \left(Ax  - b \right) &= 0 \\
    \left( x - y \right) + \lambda A &= 0 \\
    \lambda &= A^{-1}\left(y - x\right) \\
\end{align}

```python
class StochasticGradientDescent:
    @staticmethod
    def projection(vector, constraints=None):
        """Find the projection of y onto the C(A) that minimizes
        the euclidean norm between y and the projected vector

        .. math::
            \begin{align}
                \arg \min_{x \geq 0} & \frac{1}{2} {\lVert x - y \rVert}_{2}^{2} \\
                \text{subject to } & x \in X, \, \text{where }X\text{ is a convex set}
            \end{align}

        Args:
            vector (np.array): Vector to be projected

        Returns:
            np.array: Projected vector
        """
        P_positive = lambda vector: np.maximum(
            vector, np.zeros(vector.shape)
        )  # Positive projection operator on a point
        if constraints is None:
            return P_positive(
                vector
            )  # Return positive projection if there isnt a convex polytope / set constraint

        proj = cp.Variable(vector.shape, nonneg=True)  # Projected Vector
        obj = cp.Minimize(
            cp.norm2(proj - vector)
        )  # Minimize euclidean norm of difference between y and projection
        prob = cp.Problem(obj, constraints=constraints)
        prob.solve()
        return proj.value

    @staticmethod
    def projection_dykstra(projections, x0, max_iter=1000, tol=1e-6):
        """Dykstra's Projection algorithm to find the orthogonal
        projection of a point onto a convex polytope (intersection
        of affine halfspaces). The algorithm below is adapted from
        https://github.com/mjhough/Dykstra/blob/main/dykstra/Dykstra.py

        Args:
            projections

        """
        assert len(x0.shape) == 1, "x0 must be a vector"

        x = x0.copy()
        y = np.zeros((projections.shape[0], x0.shape[0]))

        n = 0
        cI = np.inf
        while n < max_iter and cI >= tol:
            cI = 0
            for i in range(projections.shape[0]):

                # Update iterate
                prev_x = x.copy()
                x = projections[i](prev_x - y[i, :])

                # Update increment
                prev_y = y[i, :].copy()
                y[i, :] = x - (prev_x - prev_y)

                # Stop condition
                cI += np.linalg.norm(prev_y - y[i, :]) ** 2

                n += 1
        return x

    def __init__(
        self, c, x0, first_stage_constraints, s_gen, n, n_iter=None, α=1e0, ε=1e-04, log=True
    ):
        """The Stochastic Gradient Descent method implemented here
        is used to optimize the Two-Stage Stochastic Linear Program
        with second-stage variables following a discrete distribution

        """
        self.c = c  # First-stage decision coefficient
        self.x0 = x0  # Initial first-stage decision variable values
        self.first_stage_constraints = first_stage_constraints  # Function that gives a list of the constraints for first-stage problem
        self.s_gen = s_gen  # Function to generate scenario specific entities
        self.n = n  # Number of scenarios to generate using s_gen
        self.n_iter = n_iter  # Number of iterations of sgd
        self.α = α  # Learning rate for gradient descent
        self.ε = ε # Tolerance for how close subgradient is to 0
        self.log = log  # Whether or not to log progress

    def solve(self):
        """Solves the Two-stage Stochastic Optimization"""
        f_primal_optimal, x_star, k = self.first_stage()

        if self.log:
            print(
                f"Optimal primal objective of first stage problem: {np.round(f_primal_optimal, 3)}"
            )
            print(f"Optimal first-stage decision variables: {np.round(x_star, 3)}")
            print(f"Number of iterations for convergence: {k}")

        return f_primal_optimal, x_star, k

    def first_stage(self):
        """First-stage Optimization"""
        f_primal_optimal = np.inf  # First-stage primal optimal objective value
        x = (
            np.array([0.0] * self.c.shape[0]) if self.x0 is None else self.x0.copy()
        )  # First-stage decision variables

#         # Fix number of iterations
#         for k in range(self.n_iter):
        k = 0
        while True:

            data = list(self.s_gen(self.n))  # Random Scenarios and associated variables

            # Initialize subgradient of f
            ν = self.c.copy()

            # Initialize expectation of h
            expected_h = 0

            # Accumulating subgradients
            for p_s, d_s, C_s, D_s, ξ_s in data:

                π_s, h_primal_optimal = self.second_stage(
                    x=x, d=d_s, C=C_s, D=D_s, ξ=ξ_s
                )

                ν -= p_s * π_s.T @ C_s  # Update subgradient
                expected_h += (
                    p_s * h_primal_optimal
                )  # Update expected second-stage optimal value

            # Early stopping condition
            if np.allclose(ν, np.zeros(self.c.shape[0]), atol=self.ε):
                print("Early Stopping condition satisfied.")
                break

            # The current f primal optimal found
            current_f_primal_optimal = self.c.T @ x + expected_h

            # Updated x value [WHY TF is it a + and not a -]
            x = StochasticGradientDescent.projection(
                vector=x + (self.α * ν), constraints=self.first_stage_constraints
            )

            if self.log and k % 50 == 0:
                print("=" * 20 + f" Iteration {k}" + "=" * 20)
                print(
                    f"Current Primal Objective Found:",
                    np.round(current_f_primal_optimal, 3),
                    "|",
                    f"Best Primal Objective Found:",
                    np.round(f_primal_optimal, 3),
                )

            # If minimized value is better, update
            if f_primal_optimal > current_f_primal_optimal:
                f_primal_optimal = current_f_primal_optimal

            # Update k
            k += 1

        return f_primal_optimal, x, k

    def second_stage(self, x, d, C, D, ξ):
        """Second-stage Optimization"""
        y = cp.Variable(d.shape[0], nonneg=True)  # Second-stage decision variables
        constraints = [C @ x + D @ y == ξ]  # Second-stage constraints
        obj = cp.Minimize(d @ y)
        prob = cp.Problem(obj, constraints=constraints)
        prob.solve()

        return constraints[0].dual_value, prob.value
```

---
## 3. L-Shaped / Bender's Method

Master Problem:

\begin{align}
  \underset{S \geq 0}{\min} \mu & \\
\end{align}

Sub Problem:

\begin{align}
  \underset{F}{\min} h(S=S^* = \underset{S \geq 0}{\arg\min}\,\mu, d=\tilde{d}) &= \underset{F}{\min} \sum^N_{i=1} h_i F_{B_i E_i} + \sum^N_{i=1} \sum^N_{j=1} c_{ij} F_{B_i M_j} + \sum^N_i p_i F_{RM_i} \\
  \text{subject to } &S_i = F_{B_i M_i} + \sum^N_{j =1 \\ i \not= 1} F_{B_i M_j} + F_{B_i E_i}\qquad i = 1, \cdots , N, \\
  &F_{B_i M_i} + \sum^N_{j = 1 \\ j \not= i} F_{B_j M_i} + F_{RM_i} = {d}_i\qquad i = 1, \cdots , N, \\
  &\sum^N_{i=1} {d}_i = \sum^N_{i=1} F_{RM_i} + \sum^N_{i = 1} F_{RE_i}, \\
  &F_{B_i E_i} + F_{RE_i} = S_i\qquad i = 1, \cdots , N, \\
  &F_{B_i E_i}, F_{B_i M_j}, F_{RM_i}, F_{RE_i} \geq 0\qquad i = 1, \cdots , N, \\
  &j = 1, \cdots , N.
\end{align}

Sp:

\begin{align}
  h_s(x) &= \underset{y_s}{\min} d^\top y_s \\
  \text{subject to } Dy_s &= \xi_s - C_s x \\
  y_s &\geq 0 \\
\end{align}

Dual Sp:

\begin{align}
  \underset{\pi_s, \lambda_s}{\max} & \left[\xi_s - C_s x\right]^\top \left(\pi_s, \lambda_s\right) \\
  \text{subject to } & \pi_s \text{ free} \\
  & \lambda_s \geq 0 \\
\end{align}

```python
class Benders:
    def __init__(self, c, first_stage_constraints, s_gen, ε=1e-5, log=True):
        """The Bender's method implemented here
        is used to optimize the Two-Stage Stochastic Linear Program
        with second-stage variables following a discrete distribution

        """
        self.c = c  # First-stage decision coefficient
        self.first_stage_constraints = first_stage_constraints  # Function that gives a list of the constraints for first-stage problem
        self.s_gen = s_gen  # Function to generate scenario specific entities
        self.ε = ε  # Tolerance between Upper Bound and Lower Bound
        self.log = log  # Whether or not to log progress

    def solve(self):
        """Solves the Two-stage Stochastic Optimization"""
        # 1. Initialize UB = infinity, LB = -infinity
        UB, LB = np.inf, -np.inf
        x = cp.Variable(self.c.shape, nonneg=True)  # First-stage decision variables

        # Master Problem
        z = cp.Variable(1, nonneg=True)
        MP_obj = cp.Minimize(z)

        # Bender cuts that we will be adding to the master problem
        cuts = (
            self.first_stage_constraints
            if self.first_stage_constraints is not None
            else []
        )

        k = 0

        # 2. While UB - LB > ε, we will continue adding cuts
        while UB - LB > self.ε:

            expected_h = 0
            α, β = 0, 0  # Hyperplane of cut coefficients

            # Go through all the scenarios possible
            for p_s, d_s, C_s, D_s, ξ_s in self.s_gen():

                # 3a. Solve the Dual of the Sub-problem for all combinations of demand \tilde{d}
                # [Equivalent to solving primal since LPs have 0 duality gap]
                π_optimal, h_dual_optimal, status = self.second_stage(
                    x=x, d=d_s, C=C_s, D=D_s, ξ=ξ_s
                )

                #                 # 4a. If dual variables are unbounded, add cut
                #                 if status == "infeasible":
                #                     cuts += [(ξ_s - C_s @ x).T @ π_optimal <= 0]

                #                 # 4b. Else set UB and add cut on first stage primal optimal value
                #                 else:
                #                     expected_h += p_s * h_dual_optimal
                #                     cuts += [z >= (ξ_s - C_s @ x).T @ π_optimal]

                expected_h += p_s * h_dual_optimal
                α += p_s * ξ_s.T @ π_optimal
                β += -p_s * C_s.T @ π_optimal

            # 3b. Add a new halfspace constraint
            cuts += [z >= α + β.T @ x]

            # 4. Update UB
            UB = min(
                UB,
                self.c.T @ x.value + expected_h if x.value is not None else expected_h,
            )

            # 5. Solve Master Problem
            MP_prob = cp.Problem(MP_obj, constraints=cuts)
            MP_prob.solve()

            # 6. Update LB
            LB = MP_prob.value

            if self.log:
                print("=" * 20 + f" Iteration {k}" + "=" * 20)
                print(
                    f"Lower Bound:",
                    np.round(LB, 3),
                    "|",
                    f"Upper Bound:",
                    np.round(UB, 3),
                )
            k += 1

        if self.log:
            print(f"Optimal primal objective of first stage problem: {np.round(LB, 3)}")
            print(f"Optimal first-stage decision variables: {np.round(x.value, 3)}")
            print(f"Number of iterations for convergence: {k}")

        return LB, np.array(x.value), k

    def second_stage(self, x, d, C, D, ξ):
        """Dual Form of Second-stage Optimization"""

        π = cp.Variable(
            ξ.shape, nonneg=False
        )  # Second-stage dual variables are Free since they are dual of the equality constraints
        constraints = [D.T @ π <= d]  # Second-stage dual constraints
        obj = cp.Maximize(
            (ξ - C @ (np.zeros(C.shape[1]) if x.value is None else x.value)).T @ π
        )
        prob = cp.Problem(obj, constraints=constraints)
        prob.solve()

        return π.value, prob.value, prob.status
```

---
## 4. Two-stage Multi-location Transshipment Problem

### 4.1. Problem Setup

#### 4.1.1 First Stage Stochastic Linear Program

In the first stage problem, we want to figure out what the optimal order-up-to quantity $S$ is. We formulate the first stage LP as follows, minimizing the expected (stochastic demand) transshipment cost with the order-up-to quantities $S$ as our primal variables:

\begin{align}
\underset{S \geq 0}{\min}\,& \mathbb{E}_{\tilde{d} \sim D}\left[h(S, d = \tilde{d})\right] \qquad \text{, where }D\text{ is the demand distribution} \\
\end{align}

#### 4.1.2 Second Stage Stochastic Linear Program

In our second stage problem, given a specific order-up-to quantity $\tilde{S}$, and a fixed demand $\tilde{d}$, we want to solve the following constrained linear program over the primal variables $F$ (flow to and from each node), minimziing the transshipment cost for the specific instance of $\tilde{S}$ and $\tilde{d}$:

\begin{align}
    h(S = \tilde{S}, d=\tilde{d}) = \underset{F}{\min} &\sum^N_{i=1} h_i F_{B_i E_i} + \sum^N_{i=1} \sum^N_{j=1} c_{ij} F_{B_i M_j} + \sum^N_i p_i F_{RM_i} \\
  \text{subject to } & S_i = F_{B_i M_i} + \sum^N_{j =1 \\ i \not= 1} F_{B_i M_j} + F_{B_i E_i} \,\,\,\, i = 1, \cdots , N, \\
  &F_{B_i M_i} + \sum^N_{j = 1 \\ j \not= i} F_{B_j M_i} + F_{RM_i} = {d}_i \,\,\,\, i = 1, \cdots , N, \\
  &\sum^N_{i=1} {d}_i = \sum^N_{i=1} F_{RM_i} + \sum^N_{i = 1} F_{RE_i}, \\
  &F_{B_i E_i} + F_{RE_i} = S_i \,\,\,\, i = 1, \cdots , N, \\
  &F_{B_i E_i}, F_{B_i M_j}, F_{RM_i}, F_{RE_i} \geq 0 \,\,\,\, i = 1, \cdots , N, \\
  &j = 1, \cdots , N.
\end{align}

Reformulate objective and constraints into standard form:

```python
def get_transshipment_variables():

    # Number of retailers
    N = 4

    # Holding cost Vector
    h = np.array([1] * N)

    # Shortage cost Vector
    p = np.array([4] * N)

    # Cost of transshipment from retailer i to retailer j Matrix
    C = np.full((N, N), 0.5)
    np.fill_diagonal(C, 0)

    # PMF of Retailer Demand distribution
    P = np.array([0.020, 0.14, 0.68, 0.14, 0.020])

    # Each Retailer's Demand distribution associated with probabilities above
    RETAILER_DEMANDS = np.array(
        [
            [60, 80, 100, 120, 140],
            [100, 150, 200, 250, 300],
            [90, 120, 150, 180, 210],
            [70, 120, 170, 220, 270],
        ]
    )

    return N, h, p, C, P, RETAILER_DEMANDS

def random_demand(P, retailer_demands):
    """Samples a random demand vector according to demand pmf
    parameters given

    Args:
        P (numpy.array): Array of probabilities for each demand, shape: (Number of probabilities)
        retailer_demands (numpy.array): Matrix of retailer demands, shape: (Number of retailers, Number of discrete demands)

    Returns:
        Tuple[np.array, float]: A tuple of the (sampled Demand Vector, probability of the Sampled Demand Vector)
    """
    assert np.isclose(
        np.sum(P), 1
    ), "p must sum to 1 to be a valid probability distribution."
    sampled_demand = np.array(
        [np.random.choice(retailer_demand, p=P) for retailer_demand in retailer_demands]
    )
    p_s = np.prod(
        [
            P[np.argwhere(retailer_demand == D_i)[0][0]]
            for retailer_demand, D_i in zip(retailer_demands, sampled_demand)
        ]
    )
    return sampled_demand, p_s

def get_second_stage_variables(s):
    """"""

    N, h, p, C, P, RETAILER_DEMANDS = get_transshipment_variables()

    # Coefficient of Second-stage decision variables
    d_s = np.concatenate(
        (
            h,  # Coefficient of Amount of inventory that moves from begining to ending (FBE)
            C.flatten(),  # Coefficient of Amount of inventory to ship from current retailer to other retailers (FBM)
            p,  # Coefficient of Amount to ship from supplier to retailer's middle inventory (FRM)
            np.array(
                [0] * N
            ),  # Coefficient of Amount to ship from supplier to retailer's ending inventory (FRE)
        )
    )

    # Second-stage C(s) matrix
    C_s = np.concatenate(
        (
            np.eye(N),
            np.zeros(shape=(N, N)),
            np.zeros(shape=(1, N)),
            np.eye(N),
        ),
        axis=0,
    )

    # Second-stage D(s) matrix
    D_s = np.concatenate(
        (
            np.concatenate(
                (
                    -np.eye(N),
                    np.array(
                        [
                            ([-1] * N + [0] * (N * (N - 1)))[-N * idx :]
                            + ([-1] * N + [0] * (N * (N - 1)))[: -N * idx]
                            for idx in range(N)
                        ]
                    ),
                    np.zeros(shape=(N, N)),
                    np.zeros(shape=(N, N)),
                ),
                axis=1,
            ),
            np.concatenate(
                (
                    np.zeros(shape=(N, N)),
                    np.array(
                        [
                            np.array(
                                [
                                    ([1] + [0] * (N - 1))[-idx:]
                                    + ([1] + [0] * (N - 1))[:-idx]
                                ]
                                * N
                            ).flatten()
                            for idx in range(N)
                        ]
                    ),
                    np.eye(N),
                    np.zeros(shape=(N, N)),
                ),
                axis=1,
            ),
            np.array([[0] * N + [0] * (N * N) + [1] * N + [1] * N]),
            np.concatenate(
                (
                    -np.eye(N),
                    np.zeros(shape=(N, N * N)),
                    np.zeros(shape=(N, N)),
                    -np.eye(N),
                ),
                axis=1,
            ),
        ),
        axis=0,
    )

    # ξ(s) vector
    ξ_s = np.concatenate(
        (
            np.zeros(N),
            s,
            np.array([np.sum(s)]),
            np.zeros(N),
        ),
        axis=0,
    )

    return d_s, C_s, D_s, ξ_s

def s_gen_sgd(n=50):
    """Function that generates the scenario dependent
    second-stage variables by sampling demand

    """

    N, h, p, C, P, RETAILER_DEMANDS = get_transshipment_variables()

    for _ in range(n):

        # Get Second-stage variables
        s, p_s = random_demand(P=P, retailer_demands=RETAILER_DEMANDS)

        # Get scenario-dependent variables
        d_s, C_s, D_s, ξ_s = get_second_stage_variables(s)

        yield p_s, d_s, C_s, D_s, ξ_s

def s_gen_benders():
    """Function that generates the scenario dependent
    second-stage variables by enumerating all combinations of demand

    """

    N, h, p, C, P, RETAILER_DEMANDS = get_transshipment_variables()

    # Get Second-stage variables
    for s in itertools.product(*RETAILER_DEMANDS):

        p_s = np.prod(
            [
                P[np.argwhere(retailer_demand == D_i)[0][0]]
                for retailer_demand, D_i in zip(RETAILER_DEMANDS, s)
            ]
        )

        # Get scenario-dependent variables
        d_s, C_s, D_s, ξ_s = get_second_stage_variables(s)

        yield p_s, d_s, C_s, D_s, ξ_s
```

### 4.2. Stochastic Gradient Descent

```python
sgd = StochasticGradientDescent(
    c=np.array([0.0] * 4),
    x0=np.array([0.0] * 4),
    first_stage_constraints=None,
    s_gen=s_gen_sgd,
    n=100,
    α=1e-01,
    ε=1e-01
)

sgd.solve()
```

    ==================== Iteration 0====================
    Current Primal Objective Found: 15740.704 | Best Primal Objective Found: inf
    ==================== Iteration 50====================
    Current Primal Objective Found: 2404.495 | Best Primal Objective Found: 2729.085
    ==================== Iteration 100====================
    Current Primal Objective Found: 289.641 | Best Primal Objective Found: 264.472
    ==================== Iteration 150====================
    Current Primal Objective Found: 262.191 | Best Primal Objective Found: 229.16
    ==================== Iteration 200====================
    Current Primal Objective Found: 292.347 | Best Primal Objective Found: 197.497
    ==================== Iteration 250====================
    Current Primal Objective Found: 290.276 | Best Primal Objective Found: 197.497
    ==================== Iteration 300====================
    Current Primal Objective Found: 177.405 | Best Primal Objective Found: 167.574
    ==================== Iteration 350====================
    Current Primal Objective Found: 165.722 | Best Primal Objective Found: 156.476
    ==================== Iteration 400====================
    Current Primal Objective Found: 182.0 | Best Primal Objective Found: 130.407
    ==================== Iteration 450====================
    Current Primal Objective Found: 199.385 | Best Primal Objective Found: 130.407
    Early Stopping condition satisfied.
    Optimal primal objective of first stage problem: 130.407
    Optimal first-stage decision variables: [100.661 200.54  150.724 170.486]
    Number of iterations for convergence: 499

    (130.407359788351,
     array([100.66088181, 200.53971283, 150.72409382, 170.48564625]),
     499)

### 4.3. Benders

```python
benders = Benders(
    c=np.array([0] * 4),
    first_stage_constraints=None,
    s_gen=s_gen_benders,
)

benders.solve()
```

    ==================== Iteration 0====================
    Lower Bound: 0.0 | Upper Bound: 2480.0
    ==================== Iteration 1====================
    Lower Bound: 0.0 | Upper Bound: 1041.897
    ==================== Iteration 2====================
    Lower Bound: 0.0 | Upper Bound: 115.972
    ==================== Iteration 3====================
    Lower Bound: 0.0 | Upper Bound: 115.972
    ==================== Iteration 4====================
    Lower Bound: 0.0 | Upper Bound: 115.972
    ==================== Iteration 5====================
    Lower Bound: 48.645 | Upper Bound: 115.972
    ==================== Iteration 6====================
    Lower Bound: 49.022 | Upper Bound: 115.972
    ==================== Iteration 7====================
    Lower Bound: 51.52 | Upper Bound: 115.972
    ==================== Iteration 8====================
    Lower Bound: 66.869 | Upper Bound: 92.555
    ==================== Iteration 9====================
    Lower Bound: 67.483 | Upper Bound: 92.555
    ==================== Iteration 10====================
    Lower Bound: 70.326 | Upper Bound: 92.555
    ==================== Iteration 11====================
    Lower Bound: 71.874 | Upper Bound: 82.007
    ==================== Iteration 12====================
    Lower Bound: 73.429 | Upper Bound: 82.007
    ==================== Iteration 13====================
    Lower Bound: 79.194 | Upper Bound: 80.918
    ==================== Iteration 14====================
    Lower Bound: 79.771 | Upper Bound: 80.918
    ==================== Iteration 15====================
    Lower Bound: 80.473 | Upper Bound: 80.918
    ==================== Iteration 16====================
    Lower Bound: 80.523 | Upper Bound: 80.918
    ==================== Iteration 17====================
    Lower Bound: 80.523 | Upper Bound: 80.523
    Optimal primal objective of first stage problem: 80.523
    Optimal first-stage decision variables: [108.508 214.065 159.724 187.702]
    Number of iterations for convergence: 18

    (80.52286246994309,
     array([108.50808924, 214.06507897, 159.7243887 , 187.70244296]),
     18)

### 4.4. Stochastic Gradient Descent Vs. Benders

```python
def run_experiment_sgd(n_scenario, α, ε):

    optimal_obj_val, optimal_order_up_to_quantity, k = StochasticGradientDescent(
        c=np.array([0.0] * 4),
        x0=np.array([0.0] * 4),
        first_stage_constraints=None,
        s_gen=s_gen_sgd,
        n=n_scenario,
        α=α,
        ε=ε,
        log=False,
    ).solve()

    return optimal_obj_val, optimal_order_up_to_quantity, k

def run_experiment_grid_sgd(
    n_experiments=100, n_scenarios=[50, 100, 1000], αs=[1e0, 1e-01, 1e-02], εs=[1e-03, 1e-04, 1e-05]
):

    results = []

    for n_scenario in n_scenarios:

        for α in αs:

            for ε in εs:

                (
                    optimal_obj_vals_sgd,
                    optimal_order_up_to_quantities_sgd,
                    n_iterations_sgd,
                    experiment_times,
                ) = ([], [], [], [])

                # Run Experiments
                for k in range(n_experiments):

                    if k % 10 == 0:
                        print(
                            f"Experimenting with n_scenario={n_scenario} | α={α} | ε={ε} | k={k}..."
                        )

                    start = time.time()
                    optimal_obj_val, optimal_order_up_to_quantity, n_iterations = run_experiment_sgd(
                        n_scenario, α, ε
                    )
                    end = time.time()

                    optimal_obj_vals_sgd.append(optimal_obj_val)
                    optimal_order_up_to_quantities_sgd.append(optimal_order_up_to_quantity)
                    n_iterations_sgd.append(n_iterations)
                    experiment_times.append(end - start)

                optimal_obj_vals_sgd = np.round(np.array(optimal_obj_vals_sgd), 2)
                optimal_order_up_to_quantities_sgd = np.round(np.array(
                    optimal_order_up_to_quantities_sgd
                ), 2)
                n_iterations_sgd = np.array(n_iterations_sgd)
                experiment_times = np.round(np.array(experiment_times), 2)

                results.append(
                    (
                        n_scenario,
                        α,
                        ε,
                        optimal_obj_vals_sgd.mean(axis=0),
                        optimal_obj_vals_sgd.std(axis=0),
                        optimal_obj_vals_sgd.max(axis=0),
                        optimal_obj_vals_sgd.min(axis=0),
                        optimal_order_up_to_quantities_sgd.mean(axis=0),
                        optimal_order_up_to_quantities_sgd.std(axis=0),
                        optimal_order_up_to_quantities_sgd.max(axis=0),
                        optimal_order_up_to_quantities_sgd.min(axis=0),
                        n_iterations_sgd.mean(axis=0),
                        n_iterations_sgd.std(axis=0),
                        n_iterations_sgd.max(axis=0),
                        n_iterations_sgd.min(axis=0),
                        experiment_times.mean(axis=0),
                        experiment_times.std(axis=0),
                        experiment_times.max(axis=0),
                        experiment_times.min(axis=0),
                    )
                )

                pd.DataFrame(
                    results,
                    columns=[
                        "Number of Scenarios",
                        "Learning rate α",
                        "Tolerance for Subgradient Early Stopping ε",
                        "Mean(Objective Value)",
                        "Std(Objective Value)",
                        "Max(Objective Value)",
                        "Min(Objective Value)",
                        "Mean(Optimal Order-Up-To Quantitites)",
                        "Std(Optimal Order-Up-To Quantitites)",
                        "Max(Optimal Order-Up-To Quantitites)",
                        "Min(Optimal Order-Up-To Quantitites)",
                        "Mean(Number of Iterations)",
                        "Std(Number of Iterations)",
                        "Max(Number of Iterations)",
                        "Min(Number of Iterations)",
                        "Mean(Experiment Times)",
                        "Std(Experiment Times)",
                        "Max(Experiment Times)",
                        "Min(Experiment Times)",
                    ],
                ).to_csv("sgd_results.csv")

    return pd.DataFrame(
        results,
        columns=[
            "Number of Scenarios",
            "Learning rate α",
            "Tolerance for Subgradient Early Stopping ε",
            "Mean(Objective Value)",
            "Std(Objective Value)",
            "Max(Objective Value)",
            "Min(Objective Value)",
            "Mean(Optimal Order-Up-To Quantitites)",
            "Std(Optimal Order-Up-To Quantitites)",
            "Max(Optimal Order-Up-To Quantitites)",
            "Min(Optimal Order-Up-To Quantitites)",
            "Mean(Number of Iterations)",
            "Std(Number of Iterations)",
            "Max(Number of Iterations)",
            "Min(Number of Iterations)",
            "Mean(Experiment Times)",
            "Std(Experiment Times)",
            "Max(Experiment Times)",
            "Min(Experiment Times)",
        ],
    )

sgd_results = run_experiment_grid_sgd(
    n_experiments=100, n_scenarios=[10, 50, 100, 200], αs=[1e-01], εs=[1e-01]
)
sgd_results
```

    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=0...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=10...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=20...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=30...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=40...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=50...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=60...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=70...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=80...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=10 | α=0.1 | ε=0.1 | k=90...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Experimenting with n_scenario=50 | α=0.1 | ε=0.1 | k=0...
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.
    Early Stopping condition satisfied.

```python
pd.read_csv("./sgd_results.csv", index_col=[0])
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Number of Scenarios</th>
      <th>Learning rate α</th>
      <th>Tolerance for Subgradient Early Stopping ε</th>
      <th>Mean(Objective Value)</th>
      <th>Std(Objective Value)</th>
      <th>Max(Objective Value)</th>
      <th>Min(Objective Value)</th>
      <th>Mean(Optimal Order-Up-To Quantitites)</th>
      <th>Std(Optimal Order-Up-To Quantitites)</th>
      <th>Max(Optimal Order-Up-To Quantitites)</th>
      <th>Min(Optimal Order-Up-To Quantitites)</th>
      <th>Mean(Number of Iterations)</th>
      <th>Std(Number of Iterations)</th>
      <th>Max(Number of Iterations)</th>
      <th>Min(Number of Iterations)</th>
      <th>Mean(Experiment Times)</th>
      <th>Std(Experiment Times)</th>
      <th>Max(Experiment Times)</th>
      <th>Min(Experiment Times)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>10</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>16.5198</td>
      <td>6.759651</td>
      <td>55.55</td>
      <td>8.53</td>
      <td>[148.4059 157.5988 154.6814 157.4469]</td>
      <td>[1.96487307 2.73314115 2.02832641 2.73398928]</td>
      <td>[150.08 162.47 156.3  162.31]</td>
      <td>[138.58 144.07 143.93 143.92]</td>
      <td>684.31</td>
      <td>78.390267</td>
      <td>980</td>
      <td>560</td>
      <td>28.8340</td>
      <td>3.309319</td>
      <td>41.45</td>
      <td>23.71</td>
    </tr>
    <tr>
      <th>1</th>
      <td>50</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>53.6970</td>
      <td>4.858717</td>
      <td>64.30</td>
      <td>43.98</td>
      <td>[100.6798 200.597  150.6084 170.606 ]</td>
      <td>[0.37629504 0.35315011 0.34831227 0.34898424]</td>
      <td>[101.65 201.48 151.52 171.53]</td>
      <td>[100.02 200.01 150.02 170.03]</td>
      <td>786.58</td>
      <td>40.985163</td>
      <td>915</td>
      <td>729</td>
      <td>166.0559</td>
      <td>8.613365</td>
      <td>193.08</td>
      <td>153.22</td>
    </tr>
    <tr>
      <th>2</th>
      <td>100</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>127.4597</td>
      <td>9.509603</td>
      <td>152.36</td>
      <td>103.51</td>
      <td>[101.2032 201.0891 151.1315 171.1326]</td>
      <td>[0.74875481 0.63293459 0.63881668 0.62607926]</td>
      <td>[104.29 202.48 152.68 172.56]</td>
      <td>[100.04 200.01 150.03 170.04]</td>
      <td>598.64</td>
      <td>205.785933</td>
      <td>1469</td>
      <td>360</td>
      <td>253.2370</td>
      <td>87.046833</td>
      <td>620.72</td>
      <td>152.10</td>
    </tr>
  </tbody>
</table>
</div>

```python
def run_experiment_grid_benders(n_experiments=100):

    results = []

    (
        optimal_obj_vals_benders,
        optimal_order_up_to_quantities_benders,
        n_iterations_benders,
        experiment_times,
    ) = ([], [], [], [])

    # Run Experiments
    for k in range(n_experiments):

        if k % 10 == 0:
            print(f"Experimenting with k={k}...")

        start = time.time()
        optimal_obj_val, optimal_order_up_to_quantity, n_iterations = Benders(
            c=np.array([0] * 4),
            first_stage_constraints=None,
            s_gen=s_gen_benders,
            log=False,
        ).solve()
        end = time.time()

        optimal_obj_vals_benders.append(optimal_obj_val)
        optimal_order_up_to_quantities_benders.append(optimal_order_up_to_quantity)
        n_iterations_benders.append(n_iterations)
        experiment_times.append(end - start)

    optimal_obj_vals_benders = np.round(np.array(optimal_obj_vals_benders), 2)
    optimal_order_up_to_quantities_benders = np.round(np.array(
        optimal_order_up_to_quantities_benders
    ), 2)
    n_iterations_benders = np.array(n_iterations_benders)
    experiment_times = np.round(np.array(experiment_times), 2)

    results.append(
        (
            5 ** 4,
            1,
            optimal_obj_vals_benders.mean(axis=0),
            optimal_obj_vals_benders.std(axis=0),
            optimal_obj_vals_benders.max(axis=0),
            optimal_obj_vals_benders.min(axis=0),
            optimal_order_up_to_quantities_benders.mean(axis=0),
            optimal_order_up_to_quantities_benders.std(axis=0),
            optimal_order_up_to_quantities_benders.max(axis=0),
            optimal_order_up_to_quantities_benders.min(axis=0),
            n_iterations_benders.mean(axis=0),
            n_iterations_benders.std(axis=0),
            n_iterations_benders.max(axis=0),
            n_iterations_benders.min(axis=0),
            experiment_times.mean(axis=0),
            experiment_times.std(axis=0),
            experiment_times.max(axis=0),
            experiment_times.min(axis=0),
        )
    )

    return pd.DataFrame(
        results,
        columns=[
            "Number of Scenarios",
            "Number of Iterations",
            "Mean(Objective Value)",
            "Std(Objective Value)",
            "Max(Objective Value)",
            "Min(Objective Value)",
            "Mean(Optimal Order-Up-To Quantitites)",
            "Std(Optimal Order-Up-To Quantitites)",
            "Max(Optimal Order-Up-To Quantitites)",
            "Min(Optimal Order-Up-To Quantitites)",
            "Mean(Number of Iterations)",
            "Std(Number of Iterations)",
            "Max(Number of Iterations)",
            "Min(Number of Iterations)",
            "Mean(Experiment Times)",
            "Std(Experiment Times)",
            "Max(Experiment Times)",
            "Min(Experiment Times)",
        ],
    )

benders_results = run_experiment_grid_benders(n_experiments=100)
benders_results.to_csv("benders_results.csv")
benders_results
```

    Experimenting with k=0...
    Experimenting with k=10...
    Experimenting with k=20...
    Experimenting with k=30...
    Experimenting with k=40...
    Experimenting with k=50...
    Experimenting with k=60...
    Experimenting with k=70...
    Experimenting with k=80...
    Experimenting with k=90...

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Number of Scenarios</th>
      <th>Number of Iterations</th>
      <th>Mean(Objective Value)</th>
      <th>Std(Objective Value)</th>
      <th>Max(Objective Value)</th>
      <th>Min(Objective Value)</th>
      <th>Mean(Optimal Order-Up-To Quantitites)</th>
      <th>Std(Optimal Order-Up-To Quantitites)</th>
      <th>Max(Optimal Order-Up-To Quantitites)</th>
      <th>Min(Optimal Order-Up-To Quantitites)</th>
      <th>Mean(Number of Iterations)</th>
      <th>Std(Number of Iterations)</th>
      <th>Max(Number of Iterations)</th>
      <th>Min(Number of Iterations)</th>
      <th>Mean(Experiment Times)</th>
      <th>Std(Experiment Times)</th>
      <th>Max(Experiment Times)</th>
      <th>Min(Experiment Times)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>625</td>
      <td>1</td>
      <td>80.52</td>
      <td>1.421085e-14</td>
      <td>80.52</td>
      <td>80.52</td>
      <td>[108.51000000000016, 214.0699999999998, 159.71...</td>
      <td>[1.5631940186722204e-13, 1.9895196601282805e-1...</td>
      <td>[108.51, 214.07, 159.72, 187.7]</td>
      <td>[108.51, 214.07, 159.72, 187.7]</td>
      <td>18.0</td>
      <td>0.0</td>
      <td>18</td>
      <td>18</td>
      <td>38.7504</td>
      <td>0.110408</td>
      <td>39.2</td>
      <td>38.53</td>
    </tr>
  </tbody>
</table>
</div>

#### 4.4.1 Comparing Mean(Optimal Order-Up-To Quantities)

```python
pd.options.display.max_colwidth = 100
```

```python
import pandas as pd
```

```python
pd.read_csv("./sgd_results.csv", index_col=[0])
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Number of Scenarios</th>
      <th>Learning rate α</th>
      <th>Tolerance for Subgradient Early Stopping ε</th>
      <th>Mean(Objective Value)</th>
      <th>Std(Objective Value)</th>
      <th>Max(Objective Value)</th>
      <th>Min(Objective Value)</th>
      <th>Mean(Optimal Order-Up-To Quantitites)</th>
      <th>Std(Optimal Order-Up-To Quantitites)</th>
      <th>Max(Optimal Order-Up-To Quantitites)</th>
      <th>Min(Optimal Order-Up-To Quantitites)</th>
      <th>Mean(Number of Iterations)</th>
      <th>Std(Number of Iterations)</th>
      <th>Max(Number of Iterations)</th>
      <th>Min(Number of Iterations)</th>
      <th>Mean(Experiment Times)</th>
      <th>Std(Experiment Times)</th>
      <th>Max(Experiment Times)</th>
      <th>Min(Experiment Times)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>10</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>16.5198</td>
      <td>6.759651</td>
      <td>55.55</td>
      <td>8.53</td>
      <td>[148.4059 157.5988 154.6814 157.4469]</td>
      <td>[1.96487307 2.73314115 2.02832641 2.73398928]</td>
      <td>[150.08 162.47 156.3  162.31]</td>
      <td>[138.58 144.07 143.93 143.92]</td>
      <td>684.31</td>
      <td>78.390267</td>
      <td>980</td>
      <td>560</td>
      <td>28.8340</td>
      <td>3.309319</td>
      <td>41.45</td>
      <td>23.71</td>
    </tr>
    <tr>
      <th>1</th>
      <td>50</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>53.6970</td>
      <td>4.858717</td>
      <td>64.30</td>
      <td>43.98</td>
      <td>[100.6798 200.597  150.6084 170.606 ]</td>
      <td>[0.37629504 0.35315011 0.34831227 0.34898424]</td>
      <td>[101.65 201.48 151.52 171.53]</td>
      <td>[100.02 200.01 150.02 170.03]</td>
      <td>786.58</td>
      <td>40.985163</td>
      <td>915</td>
      <td>729</td>
      <td>166.0559</td>
      <td>8.613365</td>
      <td>193.08</td>
      <td>153.22</td>
    </tr>
    <tr>
      <th>2</th>
      <td>100</td>
      <td>0.1</td>
      <td>0.1</td>
      <td>127.4597</td>
      <td>9.509603</td>
      <td>152.36</td>
      <td>103.51</td>
      <td>[101.2032 201.0891 151.1315 171.1326]</td>
      <td>[0.74875481 0.63293459 0.63881668 0.62607926]</td>
      <td>[104.29 202.48 152.68 172.56]</td>
      <td>[100.04 200.01 150.03 170.04]</td>
      <td>598.64</td>
      <td>205.785933</td>
      <td>1469</td>
      <td>360</td>
      <td>253.2370</td>
      <td>87.046833</td>
      <td>620.72</td>
      <td>152.10</td>
    </tr>
  </tbody>
</table>
</div>

```python
pd.read_csv("./benders_results.csv", index_col=[0])
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Number of Scenarios</th>
      <th>Number of Iterations</th>
      <th>Mean(Objective Value)</th>
      <th>Std(Objective Value)</th>
      <th>Max(Objective Value)</th>
      <th>Min(Objective Value)</th>
      <th>Mean(Optimal Order-Up-To Quantitites)</th>
      <th>Std(Optimal Order-Up-To Quantitites)</th>
      <th>Max(Optimal Order-Up-To Quantitites)</th>
      <th>Min(Optimal Order-Up-To Quantitites)</th>
      <th>Mean(Number of Iterations)</th>
      <th>Std(Number of Iterations)</th>
      <th>Max(Number of Iterations)</th>
      <th>Min(Number of Iterations)</th>
      <th>Mean(Experiment Times)</th>
      <th>Std(Experiment Times)</th>
      <th>Max(Experiment Times)</th>
      <th>Min(Experiment Times)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>625</td>
      <td>1</td>
      <td>80.52</td>
      <td>1.421085e-14</td>
      <td>80.52</td>
      <td>80.52</td>
      <td>[108.51 214.07 159.72 187.7 ]</td>
      <td>[1.56319402e-13 1.98951966e-13 2.84217094e-13 ...</td>
      <td>[108.51 214.07 159.72 187.7 ]</td>
      <td>[108.51 214.07 159.72 187.7 ]</td>
      <td>18.0</td>
      <td>0.0</td>
      <td>18</td>
      <td>18</td>
      <td>38.7504</td>
      <td>0.110408</td>
      <td>39.2</td>
      <td>38.53</td>
    </tr>
  </tbody>
</table>
</div>

#### 4.4.2 Conclusion

We observe that benders goes through fewer iterations, and overall has a more accurate result.