# Density Estimation with Adaptative RFF and DMKDE quantum circuit in quantum simulator

## Libraries

```python
!pip install qiskit
!pip install pylatexenc
```

    Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
    Collecting qiskit
      Downloading qiskit-0.36.2.tar.gz (13 kB)
    Collecting qiskit-terra==0.20.2
      Downloading qiskit_terra-0.20.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.5 MB)
         |████████████████████████████████| 6.5 MB 6.1 MB/s
    Collecting qiskit-aer==0.10.4
      Downloading qiskit_aer-0.10.4-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (18.0 MB)
         |████████████████████████████████| 18.0 MB 161 kB/s
    Collecting qiskit-ibmq-provider==0.19.1
      Downloading qiskit_ibmq_provider-0.19.1-py3-none-any.whl (240 kB)
         |████████████████████████████████| 240 kB 18.0 MB/s
    Collecting qiskit-ignis==0.7.1
      Downloading qiskit_ignis-0.7.1-py3-none-any.whl (198 kB)
         |████████████████████████████████| 198 kB 46.5 MB/s
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.10.4->qiskit) (1.4.1)
    Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.10.4->qiskit) (1.21.6)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.19.1->qiskit) (1.24.3)
    Collecting websocket-client>=1.0.1
      Downloading websocket_client-1.3.2-py3-none-any.whl (54 kB)
         |████████████████████████████████| 54 kB 2.1 MB/s
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.19.1->qiskit) (2.23.0)
    Collecting requests-ntlm>=1.1.0
      Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.19.1->qiskit) (2.8.2)
    Collecting websockets>=10.0
      Downloading websockets-10.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (112 kB)
         |████████████████████████████████| 112 kB 49.8 MB/s
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ignis==0.7.1->qiskit) (57.4.0)
    Collecting retworkx>=0.8.0
      Downloading retworkx-0.11.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
         |████████████████████████████████| 1.6 MB 38.5 MB/s
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.20.2->qiskit) (0.3.5.1)
    Collecting symengine>=0.9
      Downloading symengine-0.9.2-cp37-cp37m-manylinux2010_x86_64.whl (37.5 MB)
         |████████████████████████████████| 37.5 MB 4.5 MB/s
    Collecting python-constraint>=1.4
      Downloading python-constraint-1.4.0.tar.bz2 (18 kB)
    Collecting tweedledum<2.0,>=1.1
      Downloading tweedledum-1.1.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (943 kB)
         |████████████████████████████████| 943 kB 19.3 MB/s
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.20.2->qiskit) (5.4.8)
    Collecting ply>=3.10
      Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 3.9 MB/s
    Collecting stevedore>=3.0.0
      Downloading stevedore-3.5.0-py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 2.9 MB/s
    Collecting scipy>=1.0
      Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
         |████████████████████████████████| 38.1 MB 62.2 MB/s
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.20.2->qiskit) (1.7.1)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-ibmq-provider==0.19.1->qiskit) (1.15.0)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.19.1->qiskit) (2022.5.18.1)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.19.1->qiskit) (3.0.4)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.19.1->qiskit) (2.10)
    Collecting ntlm-auth>=1.0.2
      Downloading ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB)
    Collecting cryptography>=1.3
      Downloading cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
         |████████████████████████████████| 4.0 MB 8.9 MB/s
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.19.1->qiskit) (1.15.0)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.19.1->qiskit) (2.21)
    Requirement already satisfied: importlib-metadata>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from stevedore>=3.0.0->qiskit-terra==0.20.2->qiskit) (4.11.4)
    Collecting pbr!=2.1.0,>=2.0.0
      Downloading pbr-5.9.0-py2.py3-none-any.whl (112 kB)
         |████████████████████████████████| 112 kB 52.5 MB/s
    Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra==0.20.2->qiskit) (4.2.0)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra==0.20.2->qiskit) (3.8.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra==0.20.2->qiskit) (1.2.1)
    Building wheels for collected packages: qiskit, python-constraint
      Building wheel for qiskit (setup.py) ... done
      Created wheel for qiskit: filename=qiskit-0.36.2-py3-none-any.whl size=11933 sha256=f047dcecd5b6fdc6c30f13b5a79270dbbffd6df2ee10a1290c7d5612e9b0798a
      Stored in directory: /root/.cache/pip/wheels/36/f7/83/e2755ad17aa35bc145fce34e184aaf394a265a978d95caaabf
      Building wheel for python-constraint (setup.py) ... done
      Created wheel for python-constraint: filename=python_constraint-1.4.0-py2.py3-none-any.whl size=24081 sha256=07d2193ac69c8759d69c542c836a0b002380c8bbcbbb9c4906566a73156e89dc
      Stored in directory: /root/.cache/pip/wheels/07/27/db/1222c80eb1e431f3d2199c12569cb1cac60f562a451fe30479
    Successfully built qiskit python-constraint
    Installing collected packages: pbr, tweedledum, symengine, stevedore, scipy, retworkx, python-constraint, ply, ntlm-auth, cryptography, websockets, websocket-client, requests-ntlm, qiskit-terra, qiskit-ignis, qiskit-ibmq-provider, qiskit-aer, qiskit
      Attempting uninstall: scipy
        Found existing installation: scipy 1.4.1
        Uninstalling scipy-1.4.1:
          Successfully uninstalled scipy-1.4.1
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
    Successfully installed cryptography-37.0.2 ntlm-auth-1.5.0 pbr-5.9.0 ply-3.11 python-constraint-1.4.0 qiskit-0.36.2 qiskit-aer-0.10.4 qiskit-ibmq-provider-0.19.1 qiskit-ignis-0.7.1 qiskit-terra-0.20.2 requests-ntlm-1.1.0 retworkx-0.11.0 scipy-1.7.3 stevedore-3.5.0 symengine-0.9.2 tweedledum-1.1.1 websocket-client-1.3.2 websockets-10.3
    Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
    Collecting pylatexenc
      Downloading pylatexenc-2.10.tar.gz (162 kB)
         |████████████████████████████████| 162 kB 8.1 MB/s
    Building wheels for collected packages: pylatexenc
      Building wheel for pylatexenc (setup.py) ... done
      Created wheel for pylatexenc: filename=pylatexenc-2.10-py3-none-any.whl size=136835 sha256=50d5aa77775511dc4b8f724a76061588dfdd5dc7b2cae743cd16950b6c9e3344
      Stored in directory: /root/.cache/pip/wheels/f1/8a/f5/33ee79d4473eb201b519fa40f989b842e373237395a3421f52
    Successfully built pylatexenc
    Installing collected packages: pylatexenc
    Successfully installed pylatexenc-2.10

```python
from qiskit import QuantumCircuit, transpile
from qiskit.quantum_info import Statevector
from qiskit.visualization import plot_bloch_multivector
from qiskit import QuantumCircuit, Aer, execute
from qiskit.extensions import Initialize
from qiskit.visualization import plot_histogram
from qiskit.providers.aer import StatevectorSimulator
import tensorflow as tf

import numpy as np
backend = Aer.get_backend('qasm_simulator')
```

## Clone the QMC from GitHUB

```python
!pip install git+https://github.com/fagonzalezo/qmc.git
```

    Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
    Collecting git+https://github.com/fagonzalezo/qmc.git
      Cloning https://github.com/fagonzalezo/qmc.git to /tmp/pip-req-build-718702nv
      Running command git clone -q https://github.com/fagonzalezo/qmc.git /tmp/pip-req-build-718702nv
    Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from qmc==0.0.1) (1.7.3)
    Requirement already satisfied: numpy>=1.19.2 in /usr/local/lib/python3.7/dist-packages (from qmc==0.0.1) (1.21.6)
    Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from qmc==0.0.1) (1.0.2)
    Requirement already satisfied: tensorflow>=2.2.0 in /usr/local/lib/python3.7/dist-packages (from qmc==0.0.1) (2.8.2+zzzcolab20220527125636)
    Requirement already satisfied: typeguard in /usr/local/lib/python3.7/dist-packages (from qmc==0.0.1) (2.7.1)
    Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.15.0)
    Requirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (14.0.1)
    Requirement already satisfied: tensorboard<2.9,>=2.8 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (2.8.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (57.4.0)
    Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.6.3)
    Requirement already satisfied: tensorflow-estimator<2.9,>=2.8 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (2.8.0)
    Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (3.3.0)
    Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.1.0)
    Requirement already satisfied: absl-py>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.0.0)
    Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.1.2)
    Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (3.1.0)
    Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (4.2.0)
    Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.14.1)
    Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (0.2.0)
    Requirement already satisfied: keras<2.9,>=2.8.0rc0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (2.8.0)
    Requirement already satisfied: gast>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (0.5.3)
    Requirement already satisfied: protobuf<3.20,>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (3.17.3)
    Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (0.26.0)
    Requirement already satisfied: flatbuffers>=1.12 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (2.0)
    Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2.2.0->qmc==0.0.1) (1.46.3)
    Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.7/dist-packages (from astunparse>=1.6.0->tensorflow>=2.2.0->qmc==0.0.1) (0.37.1)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py>=2.9.0->tensorflow>=2.2.0->qmc==0.0.1) (1.5.2)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (0.4.6)
    Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (1.0.1)
    Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (1.35.0)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (0.6.1)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (1.8.1)
    Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (3.3.7)
    Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (2.23.0)
    Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (4.8)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (4.2.4)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (0.2.8)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (1.3.1)
    Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (4.11.4)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (3.8.0)
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (0.4.8)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (2.10)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (2022.5.18.1)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (3.0.4)
    Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow>=2.2.0->qmc==0.0.1) (3.2.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->qmc==0.0.1) (3.1.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->qmc==0.0.1) (1.1.0)
    Building wheels for collected packages: qmc
      Building wheel for qmc (setup.py) ... done
      Created wheel for qmc: filename=qmc-0.0.1-py3-none-any.whl size=12757 sha256=bb8286974afae88a2c8bcaea7fb3d5c8c284ffd0fb4226a7fe007d1771466606
      Stored in directory: /tmp/pip-ephem-wheel-cache-1qpfxf24/wheels/b2/d2/8d/5870208920445c46dfe694f549251e5f63d7afbee56c01f720
    Successfully built qmc
    Installing collected packages: qmc
    Successfully installed qmc-0.0.1

```python
import tensorflow as tf
import numpy as np
import qmc.tf.layers as layers
import qmc.tf.models as models
import matplotlib.pyplot as plt
from time import time

from scipy.stats import norm

```

## Mount Google Drive

```python
from google.colab import drive
drive.mount('/content/drive')
```

    Mounted at /content/drive

## One dimensional approximation

```python
import pylab as pl
from sklearn.kernel_approximation import RBFSampler

def rff_map(x, gamma, n_components, random_state=0):
    rbf_feature = RBFSampler(gamma=gamma, n_components=n_components, random_state=random_state)
    return rbf_feature.fit_transform(x)

def gauss_kernel(x, y, gamma):
    return np.exp(-gamma * (x - y) ** 2)

mean = np.array([[2.]])
sigma = 0.5
gamma = 1/(2 * sigma ** 2)
print(gamma)

n_rffs = 16
x = np.linspace(-5.,10.,100)[:, np.newaxis]
x_rff = rff_map(x, gamma, n_rffs)
mean_rff = rff_map(mean, gamma, n_rffs)
gauss_pdf = gauss_kernel(x, mean[0, 0], gamma)
rff_pdf = x_rff @ mean_rff.T
pl.plot(x, gauss_pdf, 'r-',  alpha=0.6, label='Gaussian kernel')
pl.plot(x, rff_pdf, 'b-',  alpha=0.6, label='rff kernel')
pl.title("$dim = "+str(n_rffs)+"$")
pl.legend()
```

```python
import tensorflow as tf

class QFeatureMapAdaptRFF(layers.QFeatureMapRFF):
    def __init__(
                 self,
                 gamma_trainable=True,
                 weights_trainable=True,
                 **kwargs
                 ):
        self.g_trainable = gamma_trainable
        self.w_trainable = weights_trainable
        super().__init__(**kwargs)

    def build(self, input_shape):
        rbf_sampler = RBFSampler(
            gamma=0.5,
            n_components=self.dim,
            random_state=self.random_state)
        x = np.zeros(shape=(1, self.input_dim))
        rbf_sampler.fit(x)
        self.gamma_val = tf.Variable(
            initial_value=self.gamma,
            dtype=tf.float32,
            trainable=self.g_trainable,
            name="rff_gamma")
        self.rff_weights = tf.Variable(
            initial_value=rbf_sampler.random_weights_,
            dtype=tf.float32,
            trainable=self.w_trainable,
            name="rff_weights")
        self.offset = tf.Variable(
            initial_value=rbf_sampler.random_offset_,
            dtype=tf.float32,
            trainable=self.w_trainable,
            name="offset")
        self.built = True

    def call(self, inputs):
        vals = tf.sqrt(2 * self.gamma_val) * tf.matmul(inputs, self.rff_weights) + self.offset # original code
        vals = tf.cos(vals)
        vals = vals * tf.sqrt(2. / self.dim)
        norms = tf.linalg.norm(vals, axis=-1)
        psi = vals / tf.expand_dims(norms, axis=-1)
        return psi

class DMRFF(tf.keras.Model):
    def __init__(self,
                 dim_x,
                 num_rff,
                 gamma=1,
                 random_state=None):
        super().__init__()
        self.rff_layer = QFeatureMapAdaptRFF(input_dim=dim_x, dim=num_rff, gamma=gamma, random_state=random_state, gamma_trainable=False) #old framework

    def call(self, inputs):
        x1 = inputs[:, 0]
        x2 = inputs[:, 1]
        phi1 = self.rff_layer(x1)
        phi2 = self.rff_layer(x2)
        dot = tf.einsum('...i,...i->...', phi1, phi2) ** 2
        return dot

def calc_rbf(dmrff, x1, x2):
    return dmrff.predict(np.concatenate([x1[:, np.newaxis, ...],
                                         x2[:, np.newaxis, ...]],
                                        axis=1),
                         batch_size=256)

dmrff = DMRFF(dim_x=1, num_rff=n_rffs, gamma=gamma / 2, random_state=np.random.randint(10000))
dm_rff_pdf = calc_rbf(dmrff, np.broadcast_to(mean, x.shape), x)
pl.plot(x, gauss_pdf, 'r-',  alpha=0.6, label='Gaussian kernel')
pl.plot(x, dm_rff_pdf, 'b-',  alpha=0.6, label='dmrff kernel')
pl.title("$dim = "+str(n_rffs)+"$")
pl.legend()
```

```python
print(f'Mean: {np.mean(dmrff.rff_layer.rff_weights, axis=1)}')
print(f'Std: {np.std(dmrff.rff_layer.rff_weights, axis=1)}')
print(f'Gamma: {dmrff.rff_layer.gamma_val.numpy()}')
```

    Mean: [-0.3307618]
    Std: [1.0826544]
    Gamma: 1.0

```python
num_samples = 2000
rnd_idx1 = np.random.randint(x.shape[0],size=(num_samples, ))
rnd_idx2 = np.random.randint(x.shape[0],size=(num_samples, ))
x_train = np.concatenate([x[rnd_idx1][:, np.newaxis, ...],
                          x[rnd_idx2][:, np.newaxis, ...]],
                         axis=1)

print(gamma)
y_train = gauss_kernel(x_train[:, 0, ...], x_train[:, 1, ...], gamma=gamma)
```

    2.0

```python
ephocs = 20 # original ephocs = 20
dmrff.compile(optimizer="adam", loss='mse')
dmrff.fit(x_train, y_train, epochs=100)
```

    Epoch 1/100
    63/63 [==============================] - 1s 1ms/step - loss: 0.0062
    Epoch 2/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0037
    Epoch 3/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0028
    Epoch 4/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0024
    Epoch 5/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0022
    Epoch 6/100
    63/63 [==============================] - 0s 2ms/step - loss: 0.0021
    Epoch 7/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0020
    Epoch 8/100
    63/63 [==============================] - 0s 2ms/step - loss: 0.0019
    Epoch 9/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0018
    Epoch 10/100
    63/63 [==============================] - 0s 2ms/step - loss: 0.0017
    Epoch 11/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0016
    Epoch 12/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0015
    Epoch 13/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0013
    Epoch 14/100
    63/63 [==============================] - 0s 2ms/step - loss: 0.0012
    Epoch 15/100
    63/63 [==============================] - 0s 1ms/step - loss: 0.0011
    Epoch 16/100
    63/63 [==============================] - 0s 2ms/step - loss: 9.7958e-04
    Epoch 17/100
    63/63 [==============================] - 0s 1ms/step - loss: 8.8667e-04
    Epoch 18/100
    63/63 [==============================] - 0s 1ms/step - loss: 8.2074e-04
    Epoch 19/100
    63/63 [==============================] - 0s 1ms/step - loss: 7.7381e-04
    Epoch 20/100
    63/63 [==============================] - 0s 1ms/step - loss: 7.3749e-04
    Epoch 21/100
    63/63 [==============================] - 0s 1ms/step - loss: 7.0465e-04
    Epoch 22/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.8058e-04
    Epoch 23/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.6457e-04
    Epoch 24/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.4814e-04
    Epoch 25/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.3599e-04
    Epoch 26/100
    63/63 [==============================] - 0s 2ms/step - loss: 6.2530e-04
    Epoch 27/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.1650e-04
    Epoch 28/100
    63/63 [==============================] - 0s 1ms/step - loss: 6.0768e-04
    Epoch 29/100
    63/63 [==============================] - 0s 2ms/step - loss: 6.0238e-04
    Epoch 30/100
    63/63 [==============================] - 0s 1ms/step - loss: 5.9527e-04
    Epoch 31/100
    63/63 [==============================] - 0s 1ms/step - loss: 5.8674e-04
    Epoch 32/100
    63/63 [==============================] - 0s 1ms/step - loss: 5.8191e-04
    Epoch 33/100
    63/63 [==============================] - 0s 2ms/step - loss: 5.7402e-04
    Epoch 34/100
    63/63 [==============================] - 0s 3ms/step - loss: 5.6547e-04
    Epoch 35/100
    63/63 [==============================] - 0s 2ms/step - loss: 5.5832e-04
    Epoch 36/100
    63/63 [==============================] - 0s 2ms/step - loss: 5.5377e-04
    Epoch 37/100
    63/63 [==============================] - 0s 2ms/step - loss: 5.3941e-04
    Epoch 38/100
    63/63 [==============================] - 0s 3ms/step - loss: 5.2479e-04
    Epoch 39/100
    63/63 [==============================] - 0s 2ms/step - loss: 5.0666e-04
    Epoch 40/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.8322e-04
    Epoch 41/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.6961e-04
    Epoch 42/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.5749e-04
    Epoch 43/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.5103e-04
    Epoch 44/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.4861e-04
    Epoch 45/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.4401e-04
    Epoch 46/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.4642e-04
    Epoch 47/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.4388e-04
    Epoch 48/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.4393e-04
    Epoch 49/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3903e-04
    Epoch 50/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3873e-04
    Epoch 51/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3836e-04
    Epoch 52/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3532e-04
    Epoch 53/100
    63/63 [==============================] - 0s 4ms/step - loss: 4.3723e-04
    Epoch 54/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3398e-04
    Epoch 55/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3512e-04
    Epoch 56/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3301e-04
    Epoch 57/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3527e-04
    Epoch 58/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3440e-04
    Epoch 59/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3230e-04
    Epoch 60/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3183e-04
    Epoch 61/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3705e-04
    Epoch 62/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3521e-04
    Epoch 63/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3061e-04
    Epoch 64/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3210e-04
    Epoch 65/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3261e-04
    Epoch 66/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3153e-04
    Epoch 67/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2933e-04
    Epoch 68/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2911e-04
    Epoch 69/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3176e-04
    Epoch 70/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.3173e-04
    Epoch 71/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.2840e-04
    Epoch 72/100
    63/63 [==============================] - 0s 4ms/step - loss: 4.2867e-04
    Epoch 73/100
    63/63 [==============================] - 0s 4ms/step - loss: 4.3041e-04
    Epoch 74/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2801e-04
    Epoch 75/100
    63/63 [==============================] - 0s 4ms/step - loss: 4.2928e-04
    Epoch 76/100
    63/63 [==============================] - 0s 4ms/step - loss: 4.3024e-04
    Epoch 77/100
    63/63 [==============================] - 0s 3ms/step - loss: 4.2569e-04
    Epoch 78/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2779e-04
    Epoch 79/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.3306e-04
    Epoch 80/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2662e-04
    Epoch 81/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3114e-04
    Epoch 82/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.3206e-04
    Epoch 83/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2791e-04
    Epoch 84/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2663e-04
    Epoch 85/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2859e-04
    Epoch 86/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2976e-04
    Epoch 87/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2584e-04
    Epoch 88/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2347e-04
    Epoch 89/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2853e-04
    Epoch 90/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2702e-04
    Epoch 91/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2463e-04
    Epoch 92/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2412e-04
    Epoch 93/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2351e-04
    Epoch 94/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2610e-04
    Epoch 95/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2809e-04
    Epoch 96/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2522e-04
    Epoch 97/100
    63/63 [==============================] - 0s 2ms/step - loss: 4.2314e-04
    Epoch 98/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2730e-04
    Epoch 99/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2173e-04
    Epoch 100/100
    63/63 [==============================] - 0s 1ms/step - loss: 4.2623e-04

    <keras.callbacks.History at 0x7fcffa78e5d0>

```python
dm_rff_pdf = calc_rbf(dmrff, np.broadcast_to(mean, x.shape), x)
pl.plot(x, gauss_pdf, 'r-',  alpha=0.6, label='Gaussian kernel')
pl.plot(x, dm_rff_pdf, 'b-',  alpha=0.6, label='dmrff kernel')
pl.title("$dim = "+str(n_rffs)+"$")
pl.legend()
```

```python
print(f'Mean: {np.mean(dmrff.rff_layer.rff_weights, axis=1)}')
print(f'Std: {np.std(dmrff.rff_layer.rff_weights, axis=1)}')
print(f'Gamma: {dmrff.rff_layer.gamma_val.numpy()}')
```

    Mean: [-0.29038265]
    Std: [0.96020234]
    Gamma: 1.0

```python
dmrff.predict(x_train), x_train.shape
```

    (array([0.00890563, 0.0021974 , 0.0088729 , ..., 0.00714422, 0.00501961,
            0.01040566], dtype=float32), (2000, 2, 1))

## DMKDE for pdf Estimation

### (16x16) DMKDE Mixed State

```python
def calculate_constant_qmkde(gamma=1):
  constant_kernel = (2*np.pi)**(-1/2)
  sigma = (4*gamma)**(-1/2)
  constant_outside = 1/(sigma)
  return constant_kernel * constant_outside
```

```python
# ----------------------------------------------------------------------
# Plot a 1D density example
N = 1000
np.random.seed(0)
X = np.concatenate((np.random.normal(-1, 1, int(0.65 * N)),
                    np.random.normal(3, 1, int(0.35 * N))))[:, np.newaxis]

X_plot = np.linspace(-7, 7, 250)[:, np.newaxis]

true_dens = (0.65 * norm(-1, 1).pdf(X_plot[:, 0])
             + 0.35 * norm(3, 1).pdf(X_plot[:, 0]))
```

### DMKDE Tensorflow

```python
qmkdc_dig = models.QMDensity(fm_x = dmrff.rff_layer, dim_x=n_rffs)
qmkdc_dig.compile()
qmkdc_dig.fit(X, epochs=1)
out = qmkdc_dig.predict(X_plot)
```

    32/32 [==============================] - 0s 2ms/step

```python
prob_expected_mixed = calculate_constant_qmkde(dmrff.rff_layer.gamma_val.numpy()) * out
```

```python
params = {
   'axes.labelsize': 8,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
plt.plot(X_plot[:, 0], prob_expected_mixed, color='#006BB2', lw=lw,
            linestyle='-', label="DMKDE Mixed AdpRFF")
plt.text(4, 0.25, f"DM {n_rffs}x{n_rffs}")
plt.legend(loc='upper left')
plt.xlabel('')
plt.ylabel('Probability')
```

### DMKDE Numpy

```python
X_feat_train = dmrff.rff_layer.call(tf.cast(X, tf.float32))
X_feat_test = dmrff.rff_layer.call(tf.cast(X_plot, tf.float32))

X_feat_train = np.float64((X_feat_train).numpy())
X_feat_test = np.float64((X_feat_test).numpy())

X_feat_train = X_feat_train / np.linalg.norm(X_feat_train, axis = 1).reshape(-1, 1)
X_feat_test = X_feat_test / np.linalg.norm(X_feat_test, axis = 1).reshape(-1, 1)

X_feat_train.shape, X_feat_test.shape
```

    ((1000, 16), (250, 16))

```python
n_rffs
```

    16

```python
## Training

rho_train = np.zeros((n_rffs, n_rffs))

for i in range(len(X_feat_train)):
  rho_train += np.outer(X_feat_train[i], X_feat_train[i])

rho_train = rho_train / len(X_feat_train)

(X_feat_train[0]**2).sum()
```

    1.0000000000000002

```python
## Expected result

prob_expected_mixed = np.zeros(250)

for i in range(len(prob_expected_mixed)):
  prob_expected_mixed[i] = X_feat_test[i].T @ rho_train @ X_feat_test[i]

prob_expected_mixed = calculate_constant_qmkde(dmrff.rff_layer.gamma_val.numpy()) * prob_expected_mixed

prob_expected_mixed.shape
```

    (250,)

```python
## Expected result

prob_expected_mixed = np.zeros(len(X_plot))

for i in range(len(prob_expected_mixed)):
  prob_expected_mixed[i] = X_feat_test[i].T @ rho_train @ X_feat_test[i]

prob_expected_mixed = calculate_constant_qmkde(2**(0)) * prob_expected_mixed

prob_expected_mixed.shape
```

    (250,)

```python
params = {
   'axes.labelsize': 8,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
plt.plot(X_plot[:, 0], prob_expected_mixed, color='#006BB2', lw=lw,
            linestyle='-', label="DMKDE Mixed AdpRFF")
plt.text(4, 0.25, f"DM 16x16")
plt.legend(loc='upper left')
plt.xlabel('')
plt.ylabel('Probability')
```

### DMKDE Mixed State Qiskit

```python
# Find eigenvectors and eigenvalues
lambda_train, U_train = np.linalg.eigh(rho_train)

lambda_train, U_train
```

    (array([6.80651707e-16, 3.25733538e-11, 3.43821700e-10, 4.68417752e-07,
            1.88598303e-06, 3.55037925e-04, 4.89704657e-04, 9.11103166e-03,
            1.47662353e-02, 4.37572433e-02, 6.89238924e-02, 8.98806843e-02,
            1.04689434e-01, 1.64473339e-01, 1.80093822e-01, 3.23457221e-01]),
     array([[ 7.60689595e-04, -2.36704122e-03, -9.92419295e-02,
             -4.83644199e-02,  4.85418033e-01,  3.06581543e-01,
             -1.60602383e-01,  4.22217017e-01, -1.25971723e-01,
              6.62341560e-02, -1.24919210e-01,  3.69976208e-01,
             -2.50184499e-01,  4.51692558e-01,  8.60622389e-02,
             -8.57571875e-02],
            [ 4.30658093e-02,  6.52261138e-01, -1.52123887e-01,
             -4.34176532e-02, -7.98206397e-02, -2.28285461e-01,
             -2.00585258e-01, -2.65206960e-02, -4.06462886e-01,
             -5.97904157e-02, -1.29643010e-01,  3.80663078e-01,
              9.35867104e-02, -2.79557932e-01,  3.03555501e-02,
             -1.72972151e-01],
            [ 6.13271762e-01, -1.78804671e-01, -1.55021417e-01,
             -1.22514171e-01, -2.51621919e-01,  9.24994065e-02,
             -2.30921841e-01,  2.98940732e-01, -9.21425233e-03,
             -3.37839720e-01,  1.08451995e-01, -3.02538979e-02,
             -1.08063508e-01, -1.96757019e-01,  3.61132459e-01,
              1.74984782e-01],
            [ 1.34583157e-05, -1.48330284e-03, -6.31897462e-03,
              1.80098465e-02,  1.11813902e-01,  2.02138049e-01,
             -4.50315442e-01, -4.86982117e-01,  9.42951509e-02,
             -3.64750309e-02,  3.62483319e-01,  6.20062554e-02,
             -5.34123240e-01, -1.65404175e-01, -1.81419104e-01,
             -1.23756993e-01],
            [ 4.93143658e-01,  4.35527058e-01,  3.71350412e-03,
              4.44168540e-01,  3.60503675e-02,  1.79520457e-01,
              1.56149264e-01, -2.30879620e-02,  1.13836404e-01,
              9.20335429e-02, -1.36291273e-01, -1.69236958e-01,
             -1.12906336e-01,  1.45396296e-01, -3.51669376e-01,
              2.91853610e-01],
            [ 5.25048339e-05,  2.51392389e-03, -9.31517163e-03,
             -4.72663918e-02,  8.41878072e-02,  3.51974053e-01,
              5.82928260e-03, -1.82437297e-01, -5.58102330e-01,
             -2.46706982e-01,  4.17216753e-01, -1.41305264e-01,
              4.22349620e-01,  2.31066824e-01, -1.01899081e-01,
              1.57632742e-01],
            [ 6.13035505e-01, -1.98792466e-01,  2.16663360e-01,
             -1.97129341e-01,  2.39211388e-01, -2.04367333e-01,
              1.29870148e-01, -2.72166491e-01, -4.00238624e-02,
              2.54107361e-01,  3.12229582e-02,  1.48721703e-01,
              1.99847252e-01,  8.48264747e-02, -3.98533612e-02,
             -4.13459628e-01],
            [-3.94133597e-03, -8.48089569e-02,  2.03824689e-01,
              3.74163991e-01, -3.83010540e-01, -2.31727444e-01,
             -2.84908346e-01, -9.71730588e-02, -3.39686914e-01,
              2.61151418e-01, -3.19331137e-02, -1.21887825e-01,
             -1.73086279e-01,  4.36411763e-01,  3.13376017e-01,
             -4.64372358e-02],
            [-4.94504012e-03, -4.20593547e-01, -1.71885004e-01,
              5.33328008e-01,  9.55192889e-02,  6.70299660e-02,
              1.99603568e-01, -1.85036230e-01, -1.96177151e-01,
              3.33680183e-02, -1.05823570e-01,  4.70966481e-01,
              3.79884093e-02, -3.16162291e-01,  7.50309729e-02,
              2.04348086e-01],
            [ 9.86129113e-05, -4.28234603e-03, -2.58562716e-02,
              2.53011979e-02,  2.56102701e-01,  2.34075402e-01,
             -4.97540822e-01,  3.82296927e-03,  3.95746455e-02,
              4.81982668e-01, -2.07620028e-01, -2.91045442e-01,
              3.66893928e-01, -3.02101636e-01,  1.68615911e-01,
              1.21394954e-01],
            [ 2.36134578e-05,  2.75064748e-02,  1.88236938e-02,
             -2.04084185e-01, -1.08386117e-01,  4.64155871e-01,
              3.59381037e-01, -2.07080692e-01, -3.05541926e-01,
              3.14775575e-02, -4.21614629e-01, -2.53035546e-01,
             -3.14352481e-01, -1.52968291e-01,  2.19052063e-01,
             -2.27332724e-01],
            [ 3.85028044e-02, -2.60190602e-01,  2.98780766e-02,
             -3.73538574e-01,  1.40617524e-03, -2.65736212e-01,
             -1.70012115e-01, -2.67629647e-02, -3.31628211e-01,
              3.32889429e-02, -3.49532408e-01, -2.03082303e-02,
             -1.88628914e-01,  1.12749535e-03, -4.71457192e-01,
              4.45681228e-01],
            [-3.28621154e-02,  1.88798905e-01,  6.45308365e-01,
             -3.41690293e-02,  3.85308829e-01, -1.50000243e-01,
              1.14822583e-01, -5.34756872e-02, -4.05045908e-02,
             -1.35411337e-01,  8.26151389e-02,  1.61145931e-02,
             -1.45755596e-01, -1.45571370e-01,  3.81574160e-01,
              3.78921436e-01],
            [-1.87914143e-03,  9.50423957e-02,  2.06560336e-01,
             -3.01721898e-01, -4.50833216e-01,  3.52688738e-01,
              3.48806929e-02, -1.09440730e-01,  1.97009903e-01,
              3.46239909e-01,  8.05437575e-02,  4.89518280e-01,
              1.05144170e-01,  1.06439027e-01,  3.65546363e-02,
              2.96470980e-01],
            [ 1.20809523e-06, -4.26428697e-04, -1.05075367e-03,
              1.03935686e-02,  3.02443812e-02,  4.37403085e-02,
             -2.40043884e-01, -4.32891296e-01,  2.85426205e-01,
             -5.01293991e-01, -4.92549191e-01,  1.01242546e-01,
              2.36676737e-01,  3.02500162e-01,  1.34800925e-01,
              2.71063264e-02],
            [ 2.08180026e-02,  1.26980552e-01, -6.03157127e-01,
             -2.08545566e-01,  1.75457750e-01, -2.42399013e-01,
              1.94778552e-01, -3.11601010e-01,  5.66578737e-02,
              2.22335871e-01,  1.40137057e-01, -8.23156909e-02,
             -1.09043891e-01,  2.02275586e-01,  3.53654724e-01,
              3.07637662e-01]]))

```python
# Predict with the quantum circuit
backend = Aer.get_backend('qasm_simulator')
prob_test_mixed = np.zeros(250)

exp_time = time()
for i in range(len(prob_test_mixed)):
  qc = QuantumCircuit(8, 4)
  qc.initialize(X_feat_test[i], [0, 1, 2, 3])
  qc.initialize(np.sqrt(lambda_train, dtype = np.float64), [4, 5, 6, 7])
  qc.isometry(U_train.T, [], [0, 1, 2, 3]) # ArbRot as a isometry
  qc.cnot(7, 3)
  qc.cnot(6, 2)
  qc.cnot(5, 1)
  qc.cnot(4, 0)
  qc.measure(0, 0)
  qc.measure(1, 1)
  qc.measure(2, 2)
  qc.measure(3, 3)
  counts = execute(qc, backend, shots=10000).result().get_counts()
  prob_zero = (counts["0000"]/10000)
  prob_test_mixed[i] = prob_zero
  if i % 10 == 0:
    print(i)
    print(time() - exp_time)
    exp_time = time()
```

    0
    10.688226222991943
    10
    103.42297601699829
    20
    109.86266803741455
    30
    96.10453081130981
    40
    96.6865611076355
    50
    96.590580701828
    60
    96.68273520469666
    70
    96.83170461654663
    80
    97.2377872467041
    90
    97.10063123703003
    100
    97.02487874031067
    110
    97.39143204689026
    120
    97.23411917686462
    130
    96.87007737159729
    140
    97.3459005355835
    150
    96.93035173416138
    160
    100.31564593315125
    170
    100.93188953399658
    180
    111.45109796524048
    190
    126.73202610015869
    200
    99.8668897151947
    210
    98.30355381965637
    220
    97.12805819511414
    230
    97.61074328422546
    240
    96.87469434738159

```python
prob_test_mixed_2 = calculate_constant_qmkde(dmrff.rff_layer.gamma_val.numpy()) * prob_test_mixed
```

```python
params = {
   'axes.labelsize': 8,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
plt.plot(X_plot[:, 0], prob_test_mixed_2, color='#006BB2', lw=lw,
            linestyle='-', label="DMKDE Mixed \n AFF")
plt.text(4, 0.25, f"DM 16x16")
plt.legend(loc='upper left')
plt.xlabel('X', fontsize=12)
plt.ylabel('Probability Density', fontsize=10)
```

```python
#np.save("/content/drive/MyDrive/TesisMaestria/ResearchData/mixedstate_qubits_16comp_adpRFF_gamma1fixed.npy", prob_test_mixed_2)
prob_test_mixed_3 = np.load("/content/drive/MyDrive/TesisMaestria/ResearchData/mixedstate_qubits_16comp_adpRFF_gamma1fixed.npy")
```

### DMKDE Pure State Qiskit

```python
def calculate_constant_qmkde_pure(gamma=1):
  constant_kernel = (2*np.pi)**(-1/2)
  sigma = (2*gamma)**(-1/2)
  constant_outside = 1/(sigma)
  return constant_kernel * constant_outside
```

```python
# Given the eigenvalues this function return a Unitary gate which converts the |0> -> |lambda>
def create_U_train(x_train_param, seed=0):
  #np.random.seed(seed)
  size_U = len(x_train_param)
  U_train = np.zeros((size_U, size_U))
  x_1 = x_train_param
  U_train[:, 0] = x_1
  for i in range(1, size_U):
    x_i = np.random.randn(size_U)
    for j in range(0, i):
      x_i -= x_i.dot(U_train[:, j]) * U_train[:, j]
    x_i = x_i/np.linalg.norm(x_i)
    U_train[:, i] = x_i

  return U_train
```

```python
## Training pure state and create the Unitary matrix to initialize such state

psi_train = X_feat_train.sum(axis = 0)
psi_train = psi_train / np.linalg.norm(psi_train)

U_train = create_U_train(psi_train)

(psi_train**2).sum()
```

    1.0

```python
## Expected result
psi_train_total = (X_feat_train.sum(axis = 0) / np.linalg.norm(X_feat_train.sum(axis = 0))).reshape(-1, 1)
prob_test_16_expected = X_feat_test @ psi_train_total
prob_test_16_expected = calculate_constant_qmkde_pure(dmrff.rff_layer.gamma_val.numpy()) * np.sqrt(prob_test_16_expected**2) * np.linalg.norm(X_feat_train.sum(axis = 0)) / 1000
psi_train_total.shape, X_feat_test.shape, prob_test_16_expected.shape
```

    ((16, 1), (250, 16), (250, 1))

```python
# Expected result in a classical computer
params = {
   'axes.labelsize': 8,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
#plt.plot(X_plot[:, 0], prob_test_16_expected.ravel(), color='green', lw=lw,
            # linestyle='-', label="DMKDE Pure")
plt.plot(X_plot[:, 0], prob_test_16_expected.ravel(), label="DMKDE Pure AdpRFF")
plt.text(4, 0.25, f"DM 16x16")
plt.legend(loc='upper left')
plt.xlabel('X', fontsize=12)
plt.ylabel('Probability Density', fontsize=10)
```

```python

```

```python
# Predict with the pure state quantum circuit
prob_test = np.zeros(250)
backend = Aer.get_backend('qasm_simulator')
exp_time = time()

for i in range(len(prob_test)):
  #"""
  qc = QuantumCircuit(4, 4)
  qc.initialize(X_feat_test[i], [0, 1, 2, 3])
  qc.isometry(U_train.T, [], [0, 1, 2, 3]) # ArbRot as a isometry
  qc.measure(0, 0)
  qc.measure(1, 1)
  qc.measure(2, 2)
  qc.measure(3, 3)
  counts = execute(qc, backend, shots=10000).result().get_counts()
  try:
    prob_zero = (counts["0000"]/10000)
  except:
    prob_zero = 0
  prob_test[i] = prob_zero
  if i % 10 == 0:
    print(i)
    print(time() - exp_time)
    exp_time = time()
```

    0
    2.1645090579986572
    10
    19.241626977920532
    20
    24.76967477798462
    30
    18.77110481262207
    40
    18.345872402191162
    50
    18.968117713928223
    60
    20.73809838294983
    70
    18.633424282073975
    80
    18.55515480041504
    90
    18.838902950286865
    100
    18.662621021270752
    110
    17.629656076431274
    120
    17.81453800201416
    130
    18.075623750686646
    140
    17.940595626831055
    150
    17.309056520462036
    160
    18.008583545684814
    170
    18.015706300735474
    180
    18.119516372680664
    190
    17.9002902507782
    200
    18.359500408172607
    210
    18.22488260269165
    220
    17.942532539367676
    230
    17.79218864440918
    240
    17.64289903640747

```python
prob_test_pure = calculate_constant_qmkde_pure(dmrff.rff_layer.gamma_val.numpy()) * np.sqrt(prob_test) * (1/1000) * np.linalg.norm(X_feat_train.sum(axis = 0))
```

```python
params = {
   'axes.labelsize': 8,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
plt.plot(X_plot[:, 0], prob_test_pure_3, color='green', lw=lw,
            linestyle='-', label="DMKDE Pure")
#plt.text(4, 0.25, f"DM 16x16")
plt.legend(loc='upper left')
plt.xlabel('X', fontsize=12)
plt.ylabel('Probability Density', fontsize=10)
```

```python
#np.save("/content/drive/MyDrive/TesisMaestria/ResearchData/purestate_qubits_16comp_adpRFF_gamma1fixed.npy", prob_test_pure)
prob_test_pure_3 = np.load("/content/drive/MyDrive/TesisMaestria/ResearchData/purestate_qubits_16comp_adpRFF_gamma1fixed.npy")
```

### Summary of Mixed and Pure Qiskit

```python
prob_test_pure_3 = np.load("/content/drive/MyDrive/TesisMaestria/ResearchData/purestate_qubits_16comp_adpRFF_gamma1fixed.npy")
prob_test_mixed_3 = np.load("/content/drive/MyDrive/TesisMaestria/ResearchData/mixedstate_qubits_16comp_adpRFF_gamma1fixed.npy")
```

```python
params = {
   'axes.labelsize': 10,
   'legend.fontsize': 10,
   'xtick.labelsize': 10,
   'ytick.labelsize': 10,
   'text.usetex': False,
   'figure.figsize': [5.5, 4.5]
   }
plt.rcParams.update(params)

plt.plot(X_plot[:, 0], true_dens, "maroon", label='True pdf')
colors = ['navy']
kernels = ['gaussian']
lw = 2
plt.plot(X_plot[:, 0], prob_test_pure_3, color='green', lw=lw,
            linestyle='-', label="DMKDE Pure \n AFF")
plt.plot(X_plot[:, 0], prob_test_mixed_3, color='#006BB2', lw=lw,
            linestyle='-', label="DMKDE Mixed \n AFF")
#plt.text(4, 0.25, f"DM 16x16")
plt.legend(loc='upper left')
plt.xlabel('X', fontsize=12)
plt.ylabel('Probability Density', fontsize=10)
```

```python

```


[@ggruszczynski](https://github.com/ggruszczynski)

# LBM - some theory

In this tutorial you will get the idea of:

* symbolic code  generation
* a bit more advanced LBM models

## Moments of Equilibrium Distribution Function

We are going to show how the moments of equilibrium distribution function can be calculated.

The formulas for the discrete equilibrium distribution function $ f^{eq}_i $
comes from a discretization of the continous Maxwell-Boltzmann distribution function.
The Maxwell-Boltzmann equilibrium distribution function in a continuous, velocity space is known as:

$$
\Psi^{\textit{M-B, eq}} =
\Psi^{\textit{M-B, eq}}(\psi, \boldsymbol{\xi}, \boldsymbol{u}) =
\dfrac{\psi}{(2 \pi c_s^2)^{D/2}}
exp \left[
-\frac{(\boldsymbol{\xi}-\boldsymbol{u})^2}{2 c_s^2}
\right]
$$

Where $ \psi $  is the quantity of interest (like fluid density or enthalpy), $c_s^2$ is the lattice speed of sound (aka variance of the distribution) and $ D $ is the number of dimensions.
The continuous definition of the central moments is:

$$
\tilde{\kappa}_{mn} = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty}
(\xi_x - u_x)^m (\xi_y -u_y)^n
\Psi(\psi, \boldsymbol{\xi}, \boldsymbol{u})
d \xi_x d \xi_y
$$

```python
from sympy import Symbol, exp, pi, integrate, oo
from sympy import simplify, Float, preorder_traversal
from sympy.matrices import Matrix, eye, diag
from sympy.interactive.printing import init_printing
from sympy import ccode
import sympy as sp
import numpy as np

# init_printing()
```

```python
ex_D2Q9 = Matrix([0, 1, 0, -1, 0, 1, -1, 1, -1])
ey_D2Q9 = Matrix([0, 0, 1, 0, -1, 1, 1, -1, -1])

# Let us choose the following order of moments
# one can denote the variables as f[0], f[1], f[2], f[3]...
# or f_00, f_10, f_01, f_20
# We will use the latter notation.
# observe that f[3]=f_20. It is streamed from direction e[-1,0].

order_of_moments = [
    (0, 0),
    (1, 0),
    (0, 1),
    (2, 0),
    (0, 2),
    (1, 1),
    (2, 1),
    (1, 2),
    (2, 2)]

dzeta_x = Symbol('dzeta_x', real=True)
dzeta_y = Symbol('dzeta_y', real=True)

dzeta2D = Matrix([dzeta_x, dzeta_y])

ux = Symbol('u.x')  # don't set real=True for velocity as it freezes the test suite :/
uy = Symbol('u.y')

u2D = Matrix([ux, uy])

# rho = Symbol(r'\rho', positive=True)
# cs2 = Symbol(r'\sigma', positive=True)

rho = Symbol('rho', positive=True)
cs2 = 1./3.
```

```python
def round_and_simplify(stuff):
    simplified_stuff = simplify(stuff)
    rounded_stuff = simplified_stuff

    for a in preorder_traversal(rounded_stuff):
        if isinstance(a, Float):
            rounded_stuff = rounded_stuff.subs(a, round(a, 10))

    rounded_and_simplified_stuff = simplify(rounded_stuff)
    return rounded_and_simplified_stuff
```

## Task

Fill the body of `get_Maxwellian_DF` function and run the script to calculate (central) moments.

```python
class ContinuousCMTransforms:
    def __init__(self, dzeta, u, rho, cs2):
        """
        :param dzeta: direction (x,y,z)
        :param u: velocity (x,y,z) i.e., mean of the distribution
        :param rho: density (not necessarily m00, for instance in multiphase flows)
        :param cs2: variance of the distribution = (speed of sound)^2,
                    for isothermal LB cs2=1./3;
                    otherwise  cs2 = Symbol('RT', positive=True)

        """
        self.dzeta = dzeta
        self.u = u
        self.rho = rho
        self.sigma2 = cs2

    def get_Maxwellian_DF(self):
        """
        :return: continuous, local Maxwell-Boltzmann distribution
        """
        # fill...
        return df

    def get_m(self, mno):
        fun = self.get_Maxwellian_DF()
        for dzeta_i, mno_i in zip(self.dzeta, mno):
            fun *= pow(dzeta_i, mno_i)

        lim = [(dim, -oo, oo) for dim in self.dzeta]
        result = integrate(fun, *lim)
        return round_and_simplify(result)

    def get_cm(self, mno):
        # fill...
        return round_and_simplify(result)

```

```python
# here the zeroth moment is calculated
ccmt = ContinuousCMTransforms(dzeta2D, u2D, rho=rho, cs2=cs2)
row0 = order_of_moments[0]
moment0 = ccmt.get_cm(row0)

moment0
```

```python
# write a line of code to calculate the whole vector of moments

# m_eq = Matrix(# fill...)
m_eq
```

```python
# cm_eq = Matrix(# fill...)
cm_eq
```

```python
# next, print is as 'C' code

def print_code(order_of_moments, lhs,rhs):
    for moment, expr in zip(order_of_moments, rhs):
        mstr = [str(m) for m in moment]
        mstr = ''.join(mstr)
        print(f"double {lhs}_{mstr} = {ccode(expr)};")

print_code(order_of_moments, "cm_eq", cm_eq)
```

## Moments of non-equlibrium Distribution Function

The discrete distribution function are streamed along the lattice links, which are defined by a set of discrete velocities,$\textbf{e}$.
Using the Euleran basis and a D2Q9 space, the discrete velocities read,

$$
\textbf{e} = [\textbf{e}_x, \textbf{e}_y], \\
\textbf{e}_x = [0,1,0,-1,0,1,-1,-1,1]^\top, \\
\textbf{e}_y = [0,0,1,0,-1,1,1,-1,-1]^\top, \\
$$

The discrete, raw and central moments are introduced based on the work of Geier et al. [^5] as,

$$ k_{mn} = \sum_{\alpha}(e_{\alpha x})^m ( e_{\alpha y})^n \Psi_{\alpha} $$

while the central moments are calculated in a moving reference frame i.e., with respect to the fluid velocity:

$$ \tilde{k}_{mn} = \sum_{\alpha} ( e_{\alpha x} - u_x)^m ( e_{\alpha y} - u_y)^n \Psi_{\alpha} $$

where $ \Psi_{\alpha} $ is the distribution function of interest (for example hydrodynamic or enthalpy).

Notice, that the equations can be expressed by matrix transformations [^1][^2][^3][^4].

$$
\boldsymbol{\Upsilon} = \mathbb{M} \boldsymbol{\Psi} \\
\boldsymbol{\tilde{\Upsilon}} = \mathbb{N} \boldsymbol{\Upsilon} = \underbrace{\mathbb{N} \mathbb{M}}_{\mathbb{T}} \boldsymbol{\Psi}
$$

where $\boldsymbol{\Upsilon}$ and $\boldsymbol{\tilde{\Upsilon}}$ denote the raw and central moments, respectively.
From the computational point of view, it is preferred to perform the transformations in two steps as in above (without explicit usage of the $\mathbb{T}$ matrix).

Rows of the transformation matrices are calculated analogously to $k$ and $\tilde{k}$,
$$
M_{mn} = [ (\textbf{e}_x)^m (\textbf{e}_y)^n ]^\top, \\
T_{mn} = [ (\textbf{e}_x - \mathbb{1} u_x)^m (\textbf{e}_y - \mathbb{1} u_y)^n ]
$$
Then, the matrices are assembled row by row as,

$$
\mathbb{M}
 =
 \left[
 M_{00},
 M_{10},
 M_{01},
 M_{20},
 M_{02},
 M_{11},
 M_{10},
 M_{01},
 M_{22}
 \right]
  \\
\mathbb{T} =
 \left[
 T_{00},
 T_{10},
 T_{01},
 T_{20},
 T_{02},
 T_{11},
 T_{10},
 T_{01},
 T_{22}
 \right]
$$

The $\mathbb{N}$ matrix can be found as $\mathbb{N} = \mathbb{T} \mathbb{M}^{-1} $.

Observe that $ \mathbb{M} $ is a fixed matrix while $ \mathbb{N} $ depends on the fluid velocity, $ \textbf{u} $.

Finally, the set of the central moments can be expressed in vector form as,

$$
\boldsymbol{\tilde{\Upsilon}} =
[\tilde{k}_{00}, \tilde{k}_{10}, \tilde{k}_{01}, \tilde{k}_{20}, \tilde{k}_{02}, \tilde{k}_{11}, \tilde{k}_{21}, \tilde{k}_{12}, \tilde{k}_{22}]^\top.
$$

The physical interpretation of the raw, zeroth and first order moments of the hydrodynamic  DF corresponds to the values of density, $ \rho $ and momentum $  \rho \textbf{u} $.

```python

class MatrixGenerator:
    def __init__(self, ex, ey, order_of_moments):
        self.ex = ex
        self.ey = ey
        self.order_of_moments = order_of_moments

    def __matrix_maker(self, row_maker_fun):
        M = [row_maker_fun(*row) for row in self.order_of_moments]
        return M

    def get_raw_moments_matrix(self):
        """
        :return: transformation matrix from DF to raw moments
        """

        def get_row(m, n):
            row = [pow((self.ex[i]), m) * pow((self.ey[i]), n)  for i in range(0, 9)]
            return row

        m_ = self.__matrix_maker(get_row)
        # M = [get_row(*row) for row in self.order_of_moments] # same as
        return Matrix(m_)

    def get_T_matrix(self):
        """
        :return: transformation matrix from DF to central moments
        """

        def get_row(m, n):
            # fill...
            row = [round_and_simplify(r) for r in row] # simplify the elements in each row
            return row

        m_ = self.__matrix_maker(get_row)
        return Matrix(m_)

```

```python
matrixGenerator = MatrixGenerator(ex_D2Q9, ey_D2Q9, order_of_moments)

Mraw = matrixGenerator.get_raw_moments_matrix()
Mraw
```

```python
Traw = matrixGenerator.get_T_matrix()
Nraw = Traw * Mraw.inv()

Nraw = Matrix([round_and_simplify(Nraw[i,:]) for i in range(9)])
Nraw
```

## Task
We have just generate the matrix of transformation.
Now, let as create the vector of variables which are going to be transformed.
Implement the `get_symbols` function. It shall return a vector (1-D Matrix, i.e. `Matrix([stuff])` ) having the following form $ [f_{00}, f_{10}, f_{01}, f_{20}, f_{02}, etc...] $

```python

def get_symbols(name, directions):
    print_symbols = []
    # fill...
    return Matrix(print_symbols)

fs = get_symbols("f", order_of_moments)
fs
```

```python
m = Mraw * fs
m
```

```python
print("//raw moments from density-probability functions")
print_code(order_of_moments, "m", m)
```

```python
ms = get_symbols("m", order_of_moments)
cm = Nraw * ms
cm
```

```python
print("//central moments from raw moments")
print_code(order_of_moments, "cm", cm)
```

```python
# RELAXATION MATRIX
omega_v = Symbol('omega_nu', positive=True)
omega_b = Symbol('omega_bulk', positive=True)

s_plus_D2Q9 = (omega_b + omega_v) / 2
s_minus_D2Q9 = (omega_b - omega_v) / 2

S_relax_hydro_D2Q9 = diag(1, 1, 1, s_plus_D2Q9, s_plus_D2Q9, omega_v, 1, 1, 1)
S_relax_hydro_D2Q9[3, 4] = s_minus_D2Q9
S_relax_hydro_D2Q9[4, 3] = s_minus_D2Q9
```

```python
cm_after_collision = eye(9) * cm + S_relax_hydro_D2Q9 * (cm_eq - cm)
print("//collision in central moments space")
print_code(order_of_moments, "cm_after_collision", cm_after_collision)
```

## Summary

That's the magic - you have learned how perform symbolic computations and generate code from it.
The back-tranformation from central moments to moments, then from moments to distribution function follow the same way.

References:

[^1]: Linlin Fei, Kai Hong Luo, 'Cascaded lattice Boltzmann method for incompressible thermal flows with heat sources and general thermal boundary conditions' Computers and Fluids (2018).

[^2]: Linlin Fei, Kai Hong Luo, Chuandong Lin, Qing Li, 'Modeling incompressible thermal flows using a central-moments-based lattice Boltzmann method' International Journal of Heat and Mass Transfer (2017).

[^3]: Linlin Fei and Kai Hong Luo, 'Consistent forcing scheme in the cascaded lattice Boltzmann method' Physical Review E 96, 053307 (2017).

[^4]: Linlin Fei, Kai H. Luo and Qing Li, 'Three-dimensional cascaded lattice Boltzmann method: Improved implementation and consistent forcing scheme' Physical Review E 97, 053309 (2018)

[^5]: M. Geier, A. Greiner, J. G. Korvink, 'Cascaded digital lattice Boltzmann automata for high Reynolds number flow' Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 73 (2006).

[^6]: Xiaoyi He, Xiaowen Shan, and Gary D. Doolen, 'Discrete Boltzmann equation model for nonideal gases' in Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics (1998).


Lambda School Data Science

*Unit 1, Sprint 3, Module 1*

---

# Vectors and Matrices

- Student can illustrate why we care about linear algebra in the scope of data science.
- Student can graph vectors, identify their dimensionality, calculate their length (norm), and take the dot product of two vectors.
- Student can identify the dimensionality of matrices, multiply them, identify when matrix multiplication is a legal operation, and transpose a matrix.
- Student can identify special types of square matrices including the identity matrix, as well as find the determinant and inverse of a matrix.
- Student can use NumPy to perform basic Linear Algebra operations with Python.

# [Why Linear Algebra?](#why-linear-algebra)

Student can illustrate why we care about linear algebra in the scope of data science.

## Overview

Data Science, Machine Learning, and Artificial intelligence is all about getting computers to do things for us better, cheaper, and faster than we could do them ourselves.

How do we do that? Computers are good at doing small repetitive tasks (like arithmetic). if we tell them what small repetitive tasks to do in the right order then sometimes all of those combined behaviors will result in something that looks like a human's behavior (or at least the decisions/output look like something a human might decide to do/create).

[Le Comte de Belamy](https://obvious-art.com/le-comte-de-belamy.htm)

The set of instructions that we give to a computer to complete certain tasks is called an **algorithm**. The better that we can organize the set of instructions, the faster that computers can do them. The method that we use to organize and store our set of instructions so that the computer can do them super fast is called a **data structure**. The practice of optimizing the organization of our data structures so that they run really fast and efficiently is called **computer science**. (This is why we will have a unit dedicated solely to computer science in a few months). Data Scientists should care how fast computers can process their sets of instructions (algorithms).

## Follow Along

Here's a simple data structure, in Python it's known as a **list**. It's one of the simplest ways that we can store things (data) and maintain their order. When giving instructions to a computer, it's important that the computer knows in what order to execute them.

```
selfDrivingCarInstructions = ["open door",
          "sit on seat",
          "put key in ignition",
          "turn key to the right until it stops",
          "push brake pedal",
          "change gear to 'Drive'",
          "release brake pedal",
          "push gas pedal",
          '''turn wheel to navigate streets with thousands of small rules and
          exeptions to rules all while avoiding collision with other
          objects/humans/cars, obeying traffic laws, not running out of fuel and
          getting there in a timely manner''',
          "close door"]

# We'll have self-driving cars next week for sure. NBD
```

# Maintaining the order of our sets of ordered instruction-sets
Here's another data structure we can make by putting lists inside of lists, this is called a two-dimensional list. Sometimes it is also known as a two-dimensional array or --if you put some extra methods on it-- a dataframe. As you can see things are starting to get a little bit more complicated.

```
holdMyData = [
    [1,2,3],
    [4,5,6],
    [7,8,9]
]

# Disregard the quality of these bad instructions
```

## Linear Algebra - organize and execute big calculations/operations really fast

So why linear algebra? Because the mathematical principles behinds **vectors** and **matrices** (lists and 2D lists) will help us understand how we can tell computers how to do an insane number of calculations in a very short amount of time.

Remember when we said that computers are really good at doing small and repetitive tasks very quickly?

## I Give You... Matrix Multiplication:

If you mess up any of those multiplications or additions you're up a creek.

## I Give You... Finding the Determinant of a Matrix: (an introductory linear algebra topic)

## 2x2 Matrix

Just use the formula!

## 3x3 Matrix

Just calculate the determinant of 3 different 2x2 matrices and multiply them by 3 other numbers and add it all up.

## 4x4 Matrix

Just calculate 3 diferent 3x3 matrix determinants which will require the calculating of 9 different 2x2 matrix determinants, multiply them all by the right numbers and add them all up. And if you mess up any of those multiplications or additions you're up a creek.

## 5x5 Matrix!

## ...

## ...

Just kidding, any linear algebra professor who assigns the hand calculation of a 5x5 matrix determinant (or larger) is a sadist. This is what computers were invented for! Why risk so much hand calculation in order to do something that computers **never** make a mistake at?

By the way, when was the last time that you worked with a dataframe that was 4 rows x 4 columns or smaller?

Quick, find the determinant of this 42837x42837 dataframe by hand!

### Common Applications of Linear Algebra in Data Science:

### Vectors: Rows, Columns, lists, arrays

### Matrices: tables, spreadsheets, dataframes

### Linear Regression: (You might remember from the intro course)

```
# Linear Regression Example
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Read CSV
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/Ice_Cream_Sales.csv')

# Create Column of 1s
df['Ones'] = np.ones(11)

# Format X and Y Matrices
X = df[['Ones', 'Farenheit']].values
Y = df['Dollars'].values.reshape(-1, 1)

# Calculate Beta Values
beta = np.matmul(np.linalg.inv(np.matmul(np.transpose(X), X)), np.matmul(np.transpose(X), Y))

print(beta)
```

    [[-596.20648399]
     [  24.68849397]]

```
# Assign Beta Values to Variables
beta_0 = beta[0,0]
beta_1 = beta[1,0]

# Plot points with line of best fit
plt.scatter(df['Farenheit'], df['Dollars'])
axes = plt.gca()
x_vals = np.array(axes.get_xlim())
y_vals = beta_0 + beta_1 * x_vals
plt.plot(x_vals, y_vals, '-', color='b')
plt.title('Ice Cream Sales Regression Line')
plt.xlabel('Farenheit')
plt.ylabel('Dollars')
plt.show()
```

### Dimensionality Reduction Techniques: Principle Component Analysis (PCA) and Singular Value Decomposition (SVD)

Take a giant dataset and distill it down to its important parts. (typically as a pre-processing step for creating visualizations or putting into other models.)

### Deep Learning: Convolutional Neural Networks, (Image Recognition)

"Convolving" is the process of passing a filter/kernel (small matrix) over the pixels of an image, multiplying them together, and using the result to create a new matrix. The resulting matrix will be a new image that has been modified by the filter to emphasize certain qualities of an image. This is entirely a linear algebra-based process. A convolutional neural network learns the filters that help it best identify certain aspects of images and thereby classify immages more accurately.

```
!pip install imageio
```

    Requirement already satisfied: imageio in /usr/local/lib/python3.6/dist-packages (2.4.1)
    Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from imageio) (4.3.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from imageio) (1.17.4)
    Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow->imageio) (0.46)

```
# Convolution in action
import imageio
import matplotlib.pyplot as plt
import numpy as np
import scipy.ndimage as nd
from skimage.exposure import rescale_intensity
from skimage import color

img = imageio.imread('https://www.dropbox.com/s/dv3vtiqy439pzag/all_the_things.png?raw=1')
plt.axis('off')
plt.imshow(img);
```

```
# Convert I to grayscale, so it will be MxNx1 instead of MxNx4
grayscale = color.rgb2gray(img)
plt.axis('off')
plt.imshow(grayscale, cmap=plt.cm.gray);
```

```
laplacian = np.array([[0,0,1,0,0],
              [0,0,2,0,0],
              [1,2,-16,2,1],
              [0,0,2,0,0],
              [0,0,1,0,0]])

laplacian_image = nd.convolve(grayscale, laplacian)
plt.axis('off')
plt.imshow(laplacian_image, cmap=plt.cm.gray);
```

```
sobel_x = np.array([
    [-1,0,1],
    [-2,0,2],
    [-1,0,1]
])

sobel_x_image = nd.convolve(grayscale, sobel_x)
plt.axis('off')
plt.imshow(sobel_x_image, cmap=plt.cm.gray);
```

```
sobel_y = np.array([
    [1,2,1],
    [0,0,0],
    [-1,-2,-1]
])

sobel_y_image = nd.convolve(grayscale, sobel_y)
plt.axis('off')
plt.imshow(sobel_y_image, cmap=plt.cm.gray);
```

## Are we going to learn to do Linear Algebra by hand?

Let me quote your seventh grade math teacher:

> "ArE yOu GoInG To CaRrY a CaLcUlAtOr ArOuNd wItH yOu EvErYwHeRe YoU gO???"

Of course you're going to carry a calculator around everywhere, so mostly **NO**, we're not going to do a lot of hand calculating. We're going to try and refrain from calculating things by hand unless it is absolutely necessary in order to understand and implement the concepts.

We're not trying to re-invent the wheel.

We're learning how to  **use** the wheel.

## Challenge

I hope that this teaser helps you be excited about how studying Linear Algebra will help us build the intuition and skills to unlock some of the most exciting and valuable tools and techniques in machine learning. Over the next two modules we are going start building up to the concept of PCA which as you'll see will require a firm understanding of many important linear algebra intuitions.

# [Salars and Vectors](#scalars-and-vectors)

Student can graph vectors, identify their dimensionality, calculate their length (norm), and take the dot product of two vectors.

## Overview

Scalars and vectors are the basic building blocks of linear algebra. Scalars are just numbers stored as a variable, and when vectors or matrices are multiplied by them they will be scaled according to the size of the number. Vectors are a lot like lists of values in Python or like rows/columns in a dataframe many of the principles and intuitions that we learn here can be tied back to our datasets. Understanding vectors is important because matrices can be thought of as a set of vectors just like a dataframe can be thought of as a group of columns or rows.

## Follow Along

### Scalars

A single number. Variables representing scalars are typically written in lower case.

Scalars can be whole numbers or decimals.

\begin{align}
a = 2
\qquad
b = 4.815162342
\end{align}

They can be positive, negative, 0 or any other real number.

\begin{align}
c = -6.022\mathrm{e}{+23}
\qquad
d = \pi
\end{align}

```
import math
import matplotlib.pyplot as plt
import numpy as np

# Start with a simple vector
blue = [.5, .5]

# Then multiply it by a scalar

green = np.multiply(2, blue)

red = np.multiply(math.pi, blue)

orange = np.multiply(-0.5, blue)

# Plot the Scaled Vectors
plt.arrow(0,0, red[0], red[1],head_width=.05, head_length=0.05, color ='red')
plt.arrow(0,0, green[0], green[1],head_width=.05, head_length=0.05, color ='green')
plt.arrow(0,0, blue[0], blue[1],head_width=.05, head_length=0.05, color ='blue')
plt.arrow(0,0, orange[0], orange[1],head_width=.05, head_length=0.05, color ='orange')
plt.xlim(-1,2)
plt.ylim(-1,2)
plt.title("Scaled Vectors")
plt.show()
```

### Vectors:

A vector of dimension *n* is an **ordered** collection of *n* elements, which are called **components** (Note, the components of a vector are **not** referred to as "scalars"). Vector notation variables are commonly written as a bold-faced lowercase letters or italicized non-bold-faced lowercase characters with an arrow (→) above the letters:

Written: $\vec{v}$

Examples:

\begin{align}
   \vec{a} =
   \begin{bmatrix}
           1\\
           2
    \end{bmatrix}
    \qquad
    \vec{b} =
    \begin{bmatrix}
          -1\\
           0\\
           2
    \end{bmatrix}
    \qquad
    \vec{c} =
    \begin{bmatrix}
           4.5
    \end{bmatrix}
    \qquad
    \vec{d} =
    \begin{bmatrix}
           Pl\\
           a\\
           b\\
           \frac{2}{3}
    \end{bmatrix}
\end{align}

The above vectors have dimensions 2, 3, 1, and 4 respectively.

Why do the vectors below only have two components?

```
# Vector Examples
yellow = [.5, .5]
red = [.2, .1]
blue = [.1, .3]

# Coordinate Pairs for where the arrowheads are
# yellow = (.5, .5)
# red = (.2, .1)
# blue = (.1, .3)

plt.arrow(0, 0, .5, .5, head_width=.02, head_length=0.01, color = 'y')
plt.arrow(0, 0, .2, .1, head_width=.02, head_length=0.01, color = 'r')
plt.arrow(0, 0, .1, .3,  head_width=.02, head_length=0.01, color = 'b')
plt.title('Vector Examples')
plt.show()
```

In domains such as physics it is emphasized that vectors have two properties: direction and magnitude. It's rare that we talk about them in that sense in Data Science unless we're specifically in a physics context. We just note that the length of the vector is equal to the number of dimensions of the vector.

What happens if we add a third component to each of our vectors?

```
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

yellow = [.5, .5, .5]
red = [.2, .1, .0]
blue = [.1, .3, .3]

vectors = np.array([[0, 0, 0, .5, .5, .5],
                    [0, 0, 0, .2, .1, .0],
                    [0, 0, 0, .1, .3, .3]])

X, Y, Z, U, V, W = zip(*vectors)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.quiver(X, Y, Z, U, V, W, length=1)
ax.set_xlim([0, 1])
ax.set_ylim([0, 1])
ax.set_zlim([0, 1])
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
plt.show()
```

### Norm of a Vector (Magnitude or length)

The *Norm* or *Magnitude* of a vector is nothing more than the **length** of the vector. Since a vector is just a line (essentially) if you treat it as the hypotenuse of a triangle you could use the pythagorean theorem to find the equation for the norm of a vector. We're essentially just generalizing the equation for the hypotenuse of a triangle that results from the pythagorean theorem to n dimensional space.

We denote the norm of a vector by wrapping it in double pipes (like double absolute value signs)

\begin{align}
||v|| =
\sqrt{v_{1}^2 + v_{2}^2 + \ldots + v_{n}^2}
\\
\vec{a} =
\begin{bmatrix}
3 & 7 & 2 & 4
\end{bmatrix}
\\
||a|| = \sqrt{3^2 + 7^2 + 2^2 + 4^2} \\
||a|| = \sqrt{9 + 49 + 4 + 16} \\
||a|| = \sqrt{78}
\end{align}

The Norm is the square root of the sum of the squared elements of a vector.

Properties of the Norm:

The norm is always positive or zero $||x|| \geq 0$

The norm is only equal to zero if all of the elements of the vector are zero.

The Triangle Inequality: $|| x + y ||\leq ||x|| + ||y||$

### The Equation for the norm of a vector is just Pythagorean Theorem extended to more than two dimensions.

$a^2 + b^2 = c^2$

a = one side of a right triangle

b = the other side

c = the "hypotenuse" of a right triangle (We consider the vector that we're looking at to be the hypotenuse of a triangle

$c = \sqrt{a^2 + b^2}$

### Dot Product

The dot product of two vectors $\vec{a}$ and $\vec{b}$ is a scalar quantity that is equal to the sum of pair-wise products of the components of vectors a and b. An example will make this make much more sense:

\begin{align} \vec{a} \cdot \vec{b} = (a_{1} \times b_{1}) + (a_{2} \times b_{2}) + \ldots + ( a_{n} \times b_{n}) \end{align}

Example:

\begin{align}
\vec{a} =
\begin{bmatrix}
3 & 7 & 2 & 4
\end{bmatrix}
\qquad
\vec{b} =
\begin{bmatrix}
4 & 1 & 12 & 6
\end{bmatrix}
\end{align}

The dot product of two vectors would be:
\begin{align}
a \cdot b = (3)(4) + (7)(1) + (2)(12) + (4)(6) \\
= 12 + 7 + 24 + 24  \\
= 67
\end{align}

The dot product is commutative: $ a \cdot b = b \cdot a$

The dot product is distributive: $a \cdot (b + c) = a \cdot b + a \cdot c$

Two vectors must have the same number of components in order for the dot product to exist. If they lengths differ the dot product is undefined.

## Challenge

Being able to calculate the length of a vector is vital as a vector's length is one of the key attributes that defines it. Likewise taking the dot product of two vectors is a process that you **need** to be comfortable with as it is involved in many more complex linear algebra processes. You will practice both of these concepts as well as graphing vectors on your assignment this afternoon.

# [Matrices](#matrices)

Student can identify the dimensionality of matrices, multiply them, identify when matrix multiplication is a legal operation, and transpose a matrix.

A **matrix** is a rectangular grid of numbers arranged in rows and columns. Variables that represent matrices are typically written as capital letters (boldfaced as well if you want to be super formal).

\begin{align}
A =
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6\\
           7 & 8 & 9
    \end{bmatrix}
    \qquad
    B = \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
 \end{align}

You'll notice that we arrange our dataframes in a similar grid-like pattern meaning that anything we learn about matrices also applies to the dataframes that we work with in Pandas

## Overview

A **matrix** is a rectangular grid of numbers arranged in rows and columns. Variables that represent matrices are typically written as capital letters (boldfaced as well if you want to be super formal).

\begin{align}
A =
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6\\
           7 & 8 & 9
    \end{bmatrix}
    \qquad
    B = \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
 \end{align}

You'll notice that we arrange our dataframes in a similar grid-like pattern meaning that anything we learn about matrices also applies to the dataframes that we work with in Pandas

```
import pandas as pd

df = pd.DataFrame({'a': [1,2,3], 'b': [4,5,6]})

print(df.shape)
df.head()
```

    (3, 2)

      a
      b

      0
      1
      4

      1
      2
      5

      2
      3
      6

## Follow Along

### Dimensionality

The number of rows and columns that a matrix has is called its **dimension**.

When listing the dimension of a matrix we always list rows first and then columns.

The dimension of matrix A is 3x3. (Note: This is read "Three by Three", the 'x' isn't a multiplication sign.)

What is the Dimension of Matrix B?

### Matrix Equality

In order for two Matrices to be equal the following conditions must be true:

1) They must have the same dimensions.

2) Corresponding elements must be equal.

\begin{align}
    \begin{bmatrix}
           1 & 4\\
           2 & 5\\
           3 & 6
    \end{bmatrix}
    \neq
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
\end{align}

### Matrix Multiplication

You can multipy any two matrices where the number of  columns of the first matrix is equal to the number of rows of the second matrix.

The unused dimensions of the factor matrices tell you what the dimensions of the product matrix will be.

There is no commutative property of matrix multiplication (you can't switch the order of the matrices and always get the same result).

Matrix multiplication is best understood in terms of the dot product. Remember:

\begin{align} \vec{a} \cdot \vec{b} = (a_{1} \times b_{1}) + (a_{2} \times b_{2}) + \ldots + ( a_{n} \times b_{n}) \end{align}

To multiply to matrices together, we will take the dot product of each row of the first matrix with each column of the second matrix. The position of the resulting entries will correspond to the row number and column number of the row and column vector that were used to find that scalar. Lets look at an example to make this more clear.

\begin{align}
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6
\end{bmatrix}
\begin{bmatrix}
7 & 8 \\
9 & 10 \\
11 & 12
\end{bmatrix}
=
\begin{bmatrix}
(1)(7)+(2)(9)+(3)(11) & (1)(8)+(2)(10)+(3)(12)\\
(4)(7)+(5)(9)+(6)(11) & (4)(8)+(5)(10)+(6)(12)
\end{bmatrix}
=
\begin{bmatrix}
(7)+(18)+(33) & (8)+(20)+(36)\\
(28)+(45)+(66) & (32)+(50)+(72)
\end{bmatrix}
=
\begin{bmatrix}
58 & 64\\
139 & 154
\end{bmatrix}
\end{align}

## Transpose

A transposed matrix is one whose rows are the columns of the original and whose columns are the rows of the original.

Common notation for the transpose of a matrix is to have a capital $T$ superscript or a tick mark:

\begin{align}
B^{T}
\qquad
B^{\prime}
\end{align}

The first is read "B transpose" the second is sometimes read as "B prime" but can also be read as "B transpose".

The transpose of any matrix can be found easily by fixing the elements on the main diagonal and flipping the placement of all other elements across that diagonal.

\begin{align}
  B =
\begin{bmatrix}
  1 & 2 & 3 \\
  4 & 5 & 6
\end{bmatrix}
  \qquad
  B^{T} =
  \begin{bmatrix}
  1 & 4 \\
  2 & 5 \\
  3 & 6
  \end{bmatrix}
\end{align}

```
df
```

      a
      b

      0
      1
      4

      1
      2
      5

      2
      3
      6

```
# Transposing a dataframe switches its rows and columns
df.T
```

      0
      1
      2

      a
      1
      2
      3

      b
      4
      5
      6

## Challenge

Throughout the week we will be learning how the princples of mathematical matrices apply to our datasets. We'll be focusing mostly on math topics and techniques but as the week goes on we will be making more and more connections back to how we think about and work with our datasets.

# [Square Matrices](#square-matrices)

Student can identify special types of square matrices including the identity matrix, as well as find the determinant and inverse of a matrix.

## Overview

In a traditional linear algebra class after the first few weeks you would deal almost exclusively with square matrices. They have very nice properties that their lopsided sisters and brothers just don't possess.

A square matrix is any matrix that has the same number of rows as columns:

\begin{align}
A =
\begin{bmatrix}
  a_{1,1}
\end{bmatrix}
\qquad
B =
\begin{bmatrix}
b_{1,1} & b_{1,2} \\
b_{2,1} & b_{2,2}
\end{bmatrix}
\qquad
C =
\begin{bmatrix}
c_{1,1} & c_{1,2} & c_{1,3} \\
c_{2,1} & c_{2,2} & c_{2,3} \\
c_{3,1} & c_{3,2} & c_{3,3}
\end{bmatrix}
\end{align}

## Follow Along

### Special Kinds of Square Matrices

**Diagonal:** Values on the main diagonal, zeroes everywhere else.

\begin{align}
A =
\begin{bmatrix}
a_{1,1} & 0 & 0 \\
0 & a_{2,2} & 0 \\
0 & 0 & a_{3,3}
\end{bmatrix}
\end{align}

**Upper Triangular:** Values on and above the main diagonal, zeroes everywhere else.

\begin{align}
B =
\begin{bmatrix}
b_{1,1} & b_{1,2} & b_{1,3} \\
0 & b_{2,2} & b_{2,3} \\
0 & 0 & b_{3,3}
\end{bmatrix}
\end{align}

**Lower Triangular:** Values on and below the main diagonal, zeroes everywhere else.

\begin{align}
C =
\begin{bmatrix}
c_{1,1} & 0 & 0 \\
c_{2,1} & c_{2,2} & 0 \\
c_{3,1} & c_{3,2} & c_{3,3}
\end{bmatrix}
\end{align}

**Identity Matrix:** A diagonal matrix with ones on the main diagonal and zeroes everywhere else. The product of the any square matrix and the identity matrix is the original square matrix $AI == A$. Also, any matrix multiplied by its inverse will give the identity matrix as its product.  $AA^{-1} = I$

\begin{align}
D =
\begin{bmatrix}
  1
\end{bmatrix}
\qquad
E =
\begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}
\qquad
F =
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\end{align}

**Symmetric:** The numbers above the main diagonal are mirrored below/across the main diagonal.

\begin{align}
G =
\begin{bmatrix}
1 & 4 & 5 \\
4 & 2 & 6 \\
5 & 6 & 3
\end{bmatrix}
\end{align}

### Determinant

The determinant is a property that all square matrices possess and is denoted $det(A)$ or using pipes (absolute value symbols) $|A|$

The equation given for finding the determinant of a 2x2 matrix is as follows:

\begin{align}
A = \begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\qquad
|A| = ad-bc
\end{align}

The determinant of larger square matrices recursively by finding the determinats of the smaller matrics that make up the large matrix.

For example:

The above equation is **very** similar to the equation that we use to find the cross-product of a 3x3 matrix.

### Inverse

 There are multiple methods that we could use to find the inverse of a matrix by hand. I would suggest you explore those methods --if this content isn't already overwhelming enough. The inverse is like the reciprocal of the matrix that was used to generate it. Just like $\frac{1}{8}$ is the reciprocal of 8, $A^{-1}$ acts like the reciprocal of $A$.  The equation for finding the determinant of a 2x2 matrix is as follows:

 \begin{align}
A = \begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\qquad
A^{-1} = \frac{1}{ad-bc}\begin{bmatrix}
d & -b\\
-c & a
\end{bmatrix}
\end{align}

 ### What happens if we multiply a matrix by its inverse?

The product of a matrix multiplied by its inverse is the identity matrix of the same dimensions as the original matrix. There is no concept of "matrix division" in linear algebra, but multiplying a matrix by its inverse is very similar since $8\times\frac{1}{8} = 1$.

\begin{align}
A^{-1}A = I
\end{align}

### Not all matrices are invertible

Matrices that are not square are not invertible.

 A matrix is invertible if and only if its determinant is non-zero. You'll notice that the fraction on the left side of the matrix is $\frac{1}{det(A)}$.

 As you know, dividing anything by 0 leads to an undefined quotient. Therefore, if the determinant of a matrix is 0, then the entire inverse becomes undefined.

### What leads to a 0 determinant?

 A square matrix that has a determinant of 0 is known as a "singular" matrix. One thing that can lead to a matrix having a determinant of 0 is if two rows or columns in the matrix are perfectly collinear. Another way of saying this is that the determinant will be zero if the rows or columns of a matrix are not linearly dependent.

One of the most common ways that a matrix can end up having rows that are linearly dependent is if one column a multiple of another column. Lets look at an example:

\begin{align}
C =\begin{bmatrix}
  1 & 5 & 2 \\
  2 & 7 & 4 \\
  3 & 2 & 6
\end{bmatrix}
\end{align}

Look at the columns of the above matrix, column 3 is exactly double column 1. (could be any multiple or fraction) Think about if you had some measure in a dataset of distance in miles, but then you also wanted to convert its units to feet, so you create another column and multiply the mile measure by 5,280 (Thanks Imperial System). But then you forget to drop one of the columns so you end up with two columns that are linearly dependent which causes the determinant of your dataframe to be 0 and will cause certain algorithms to fail. We'll go deeper into this concept next week (this can cause problems with linear regression) so just know that matrices that have columns that are a multiple or fraction of another column will cause the determinant of that matrix to be 0.

## Challenge

Square Matrices have many important properties like the inverse and determinant that we will be using in the future. For more details about the implications of a square matrix having an inverse you can try googling the "Invertible Matrix Theorem" mastering and proving the different partso of this theorem is one of the main focuses of traditional linear algebra education.

# [Doing Linear Algebra with NumPy](#linear-algebra-NumPy)

Student can use NumPy to perform basic Linear Algebra operations with Python.

## Overview

NumPy is at the core of many types of scientific computing and is the main tool that we will use to perform Linear Algebra operations with Python.

We have talked about how vectors are a lot like Python Lists, however we will mostly use NumPy arrays instead of Python lists as the main data structure. Lets demonstrate some of the main differences betwen Python lists and NumPy Arrays.

```
### What will happen when I add the two Python lists together?

```

```
# Lets create two 1-Dimensional NumPy arrays and add them together?
# What do you think the result will be?

# Array Broadcasting
# Doing some arithmetic operation to vectors "element-wise"

```

What we have just demonstrated is one of the primary benefits of NumPy and it is called "Array Broadcasting" This means that arithmetic operations happen "element-wise"

```

```

```

```

```

```

## Follow Along

Lets calculate a dot product. What do we need to do? We need to multiply two vectors "element-wise" and them sum up all of those products

```
# Find the dot product of vectos a and b

# Dot Product

```

We can also use provided linear algebra functions from the [np.linalg](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) documentation in order to perform Linear Algebra processes.

```

```

### 1-Dimensional vs 2-Dimensional NumPy Arrays

One dimensional NumPy arrays can be used to represent vectors but cannot be used to differentiate between row and column vectors.

\begin{align}
\text{column vector} = \begin{bmatrix}1 \\ 2 \\ 3\end{bmatrix}
\end{align}

\begin{align}
\text{row vector} = \begin{bmatrix} 1 & 2 & 3\end{bmatrix}
\end{align}

Notice that one dimensional arrays do not have a second element listed for their shape. so that there is no way to tell if they represent column vectors or row vectors

```

```

```
# Notice how only the number of items in the array is listed

```

```
# And if we swap rows and columns the shape does not change

```

### If I explicitly want a row of column vector then I need to use a 2D NumPy Array

```

```

### Only use NumPy Arrans, DO NOT use the NumPy Matrix class.

NumPy has a matrix class which can be used for representing matrices. However, I don't want you to use that datatype because it will be removed from NumPy in the future. However, I want you to be aware that this other datatype exists so that you don't get confused when you look at stack overflow posts or other tutorials that use the matrix datatype. Keep an eye out for resources that use it and just know that they will soon be outdated. Anything that can be done with the matrix class can also be done with simple NumPY arrays.

```
# Do not work with this datatype
# This will be removed from NumPy in the future

```

```

```

```

```

```
# Matrix as a NumPy Array

# ndarrays

```

```

```

```

```

```

```

## Challenge

In your afternoon assignment you will perform many different linear algebra functions using NumPy. Many of which will be done easily by using these [Helpful NumPy Linear Algebra Functions](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) found in the NumPy documentation.

# Review

Lets good at working with vectors and matrices with NumPy!

Remember that you'll need many functions from [the documentation](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) in order to complete the assignment

Remember to use 2D NumPy arrays to represent row/column vectors as well as matrices and please share any helpful resources that you find in the cohort channel.