```python
# First check the Python version
import sys
if sys.version_info < (3,4):
    print('You are running an older version of Python!\n\n' \
          'You should consider updating to Python 3.4.0 or ' \
          'higher as the libraries built for this course ' \
          'have only been tested in Python 3.4 and higher.\n')
    print('Try installing the Python 3.5 version of anaconda '
          'and then restart `jupyter notebook`:\n' \
          'https://www.continuum.io/downloads\n\n')

# Now get necessary libraries
try:
    import os
    import numpy as np
    import matplotlib.pyplot as plt
    from skimage.transform import resize
except ImportError:
    print('You are missing some packages! ' \
          'We will try installing them before continuing!')
    !pip install "numpy>=1.11.0" "matplotlib>=1.5.1" "scikit-image>=0.11.3" "scikit-learn>=0.17"
    import os
    import numpy as np
    import matplotlib.pyplot as plt
    from skimage.transform import resize
    print('Done!')

# Import Tensorflow
try:
    import tensorflow as tf
except ImportError:
    print("You do not have tensorflow installed!")
    print("Follow the instructions on the following link")
    print("to install tensorflow before continuing:")
    print("")
    print("https://github.com/pkmital/CADL#installation-preliminaries")

# This cell includes the provided libraries from the zip file
try:
    from libs import utils
except ImportError:
    print("Make sure you have started notebook in the same directory" +
          " as the provided zip file which includes the 'libs' folder" +
          " and the file 'utils.py' inside of it.  You will NOT be able"
          " to complete this assignment unless you restart jupyter"
          " notebook inside the directory created by extracting"
          " the zip file or cloning the github repo.")

# We'll tell matplotlib to inline any drawn figures like so:
%matplotlib inline
plt.style.use('ggplot')
```

## Variables

We're first going to import the tensorflow library:

Let's take a look at how we might create a range of numbers.  Using numpy, we could for instance use the linear space function:

```python
x = np.linspace(-3.0, 3.0, 100)

# Immediately, the result is given to us.  An array of 100 numbers equally spaced from -3.0 to 3.0.
print(x)

# We know from numpy arrays that they have a `shape`, in this case a 1-dimensional array of 100 values
print(x.shape)

# and a `dtype`, in this case float64, or 64 bit floating point values.
print(x.dtype)
```

    [-3.         -2.93939394 -2.87878788 -2.81818182 -2.75757576 -2.6969697
     -2.63636364 -2.57575758 -2.51515152 -2.45454545 -2.39393939 -2.33333333
     -2.27272727 -2.21212121 -2.15151515 -2.09090909 -2.03030303 -1.96969697
     -1.90909091 -1.84848485 -1.78787879 -1.72727273 -1.66666667 -1.60606061
     -1.54545455 -1.48484848 -1.42424242 -1.36363636 -1.3030303  -1.24242424
     -1.18181818 -1.12121212 -1.06060606 -1.         -0.93939394 -0.87878788
     -0.81818182 -0.75757576 -0.6969697  -0.63636364 -0.57575758 -0.51515152
     -0.45454545 -0.39393939 -0.33333333 -0.27272727 -0.21212121 -0.15151515
     -0.09090909 -0.03030303  0.03030303  0.09090909  0.15151515  0.21212121
      0.27272727  0.33333333  0.39393939  0.45454545  0.51515152  0.57575758
      0.63636364  0.6969697   0.75757576  0.81818182  0.87878788  0.93939394
      1.          1.06060606  1.12121212  1.18181818  1.24242424  1.3030303
      1.36363636  1.42424242  1.48484848  1.54545455  1.60606061  1.66666667
      1.72727273  1.78787879  1.84848485  1.90909091  1.96969697  2.03030303
      2.09090909  2.15151515  2.21212121  2.27272727  2.33333333  2.39393939
      2.45454545  2.51515152  2.57575758  2.63636364  2.6969697   2.75757576
      2.81818182  2.87878788  2.93939394  3.        ]
    (100,)
    float64

In tensorflow, we could try to do the same thing using their linear space function:

```python
x = tf.linspace(-3.0, 3.0, 100)
print(x)
```

    Tensor("LinSpace_1:0", shape=(100,), dtype=float32)

Instead of a numpy.array, we are returned a tf.Tensor. The name of it is "LinSpace:0". Wherever we see this colon 0, that just means the output of. So the name of this Tensor is saying, the output of LinSpace.
Think of tf.Tensors the same way as you would the numpy.array. It is described by its shape, in this case, only 1 dimension of 100 values. And it has a dtype, in this case, float32. But unlike the numpy.array, there are no values printed here! That's because it actually hasn't computed its values yet. Instead, it just refers to the output of a tf.Operation which has been already been added to Tensorflow's default computational graph. The result of that operation is the tensor that we are returned.

## Graphs

Let's try and inspect the underlying graph.  We can request the "default" graph where all of our operations have been added:

```python
g = tf.get_default_graph()
```

## Operations

And from this graph, we can get a list of all the operations that have been added, and print out their names:

```python
[op.name for op in g.get_operations()]
```

    ['LinSpace/start',
     'LinSpace/stop',
     'LinSpace/num',
     'LinSpace',
     'sub/y',
     'sub',
     'Pow/y',
     'Pow',
     'Pow_1/x',
     'Pow_1/y',
     'Pow_1',
     'mul/x',
     'mul',
     'truediv',
     'Neg',
     'Exp',
     'Sqrt/x',
     'Sqrt',
     'mul_1/x',
     'mul_1',
     'truediv_1/x',
     'truediv_1',
     'mul_2',
     'LinSpace_1/start',
     'LinSpace_1/stop',
     'LinSpace_1/num',
     'LinSpace_1']

So Tensorflow has named each of our operations to generally reflect what they are doing.  There are a few parameters that are all prefixed by LinSpace, and then the last one which is the operation which takes all of the parameters and creates an output for the linspace.

## Tensor

We can request the output of any operation, which is a tensor, by asking the graph for the tensor's name:

```python
g.get_tensor_by_name('LinSpace' + ':0')
```

    <tf.Tensor 'LinSpace:0' shape=(100,) dtype=float32>

What I've done is asked for the `tf.Tensor` that comes from the operation "LinSpace".  So remember, the result of a `tf.Operation` is a `tf.Tensor`.  Remember that was the same name as the tensor `x` we created before.

## Sessions

In order to actually compute anything in tensorflow, we need to create a `tf.Session`.  The session is responsible for evaluating the `tf.Graph`.  Let's see how this works:

```python
import tensorflow as tf
# We're first going to create a session:
sess = tf.Session()

# Now we tell our session to compute anything we've created in the tensorflow graph.
computed_x = sess.run(x)
print(computed_x)

# Alternatively, we could tell the previous Tensor to evaluate itself using this session:
computed_x = x.eval(session=sess)
print(computed_x)

# We can close the session after we're done like so:
sess.close()
```

    [-3.         -2.939394   -2.87878799 -2.81818175 -2.75757575 -2.69696975
     -2.63636351 -2.5757575  -2.5151515  -2.4545455  -2.3939395  -2.33333325
     -2.27272725 -2.21212125 -2.15151501 -2.090909   -2.030303   -1.969697
     -1.90909088 -1.84848475 -1.78787875 -1.72727275 -1.66666663 -1.6060605
     -1.5454545  -1.4848485  -1.42424238 -1.36363626 -1.30303025 -1.24242425
     -1.18181813 -1.12121201 -1.060606   -1.         -0.939394   -0.87878776
     -0.81818175 -0.75757575 -0.69696951 -0.63636351 -0.5757575  -0.5151515
     -0.4545455  -0.39393926 -0.33333325 -0.27272725 -0.21212101 -0.15151501
     -0.090909   -0.030303    0.030303    0.09090924  0.15151525  0.21212125
      0.27272749  0.33333349  0.3939395   0.4545455   0.5151515   0.57575774
      0.63636374  0.69696975  0.75757599  0.81818199  0.87878799  0.939394    1.
      1.060606    1.12121201  1.18181849  1.24242449  1.30303049  1.36363649
      1.4242425   1.4848485   1.5454545   1.60606098  1.66666698  1.72727299
      1.78787899  1.84848499  1.909091    1.969697    2.030303    2.090909
      2.15151548  2.21212149  2.27272749  2.33333349  2.3939395   2.4545455
      2.5151515   2.57575798  2.63636398  2.69696999  2.75757599  2.81818199
      2.87878799  2.939394    3.        ]
    [-3.         -2.939394   -2.87878799 -2.81818175 -2.75757575 -2.69696975
     -2.63636351 -2.5757575  -2.5151515  -2.4545455  -2.3939395  -2.33333325
     -2.27272725 -2.21212125 -2.15151501 -2.090909   -2.030303   -1.969697
     -1.90909088 -1.84848475 -1.78787875 -1.72727275 -1.66666663 -1.6060605
     -1.5454545  -1.4848485  -1.42424238 -1.36363626 -1.30303025 -1.24242425
     -1.18181813 -1.12121201 -1.060606   -1.         -0.939394   -0.87878776
     -0.81818175 -0.75757575 -0.69696951 -0.63636351 -0.5757575  -0.5151515
     -0.4545455  -0.39393926 -0.33333325 -0.27272725 -0.21212101 -0.15151501
     -0.090909   -0.030303    0.030303    0.09090924  0.15151525  0.21212125
      0.27272749  0.33333349  0.3939395   0.4545455   0.5151515   0.57575774
      0.63636374  0.69696975  0.75757599  0.81818199  0.87878799  0.939394    1.
      1.060606    1.12121201  1.18181849  1.24242449  1.30303049  1.36363649
      1.4242425   1.4848485   1.5454545   1.60606098  1.66666698  1.72727299
      1.78787899  1.84848499  1.909091    1.969697    2.030303    2.090909
      2.15151548  2.21212149  2.27272749  2.33333349  2.3939395   2.4545455
      2.5151515   2.57575798  2.63636398  2.69696999  2.75757599  2.81818199
      2.87878799  2.939394    3.        ]

You can also specify which graph you want to use in a given session as an argument to tf.Session()

To simplify things, since we'll be working in iPython's interactive console, we can create an `tf.InteractiveSession`:

```python
sess = tf.InteractiveSession()
x.eval()
```

    array([-3.        , -2.939394  , -2.87878799, -2.81818175, -2.75757575,
           -2.69696975, -2.63636351, -2.5757575 , -2.5151515 , -2.4545455 ,
           -2.3939395 , -2.33333325, -2.27272725, -2.21212125, -2.15151501,
           -2.090909  , -2.030303  , -1.969697  , -1.90909088, -1.84848475,
           -1.78787875, -1.72727275, -1.66666663, -1.6060605 , -1.5454545 ,
           -1.4848485 , -1.42424238, -1.36363626, -1.30303025, -1.24242425,
           -1.18181813, -1.12121201, -1.060606  , -1.        , -0.939394  ,
           -0.87878776, -0.81818175, -0.75757575, -0.69696951, -0.63636351,
           -0.5757575 , -0.5151515 , -0.4545455 , -0.39393926, -0.33333325,
           -0.27272725, -0.21212101, -0.15151501, -0.090909  , -0.030303  ,
            0.030303  ,  0.09090924,  0.15151525,  0.21212125,  0.27272749,
            0.33333349,  0.3939395 ,  0.4545455 ,  0.5151515 ,  0.57575774,
            0.63636374,  0.69696975,  0.75757599,  0.81818199,  0.87878799,
            0.939394  ,  1.        ,  1.060606  ,  1.12121201,  1.18181849,
            1.24242449,  1.30303049,  1.36363649,  1.4242425 ,  1.4848485 ,
            1.5454545 ,  1.60606098,  1.66666698,  1.72727299,  1.78787899,
            1.84848499,  1.909091  ,  1.969697  ,  2.030303  ,  2.090909  ,
            2.15151548,  2.21212149,  2.27272749,  2.33333349,  2.3939395 ,
            2.4545455 ,  2.5151515 ,  2.57575798,  2.63636398,  2.69696999,
            2.75757599,  2.81818199,  2.87878799,  2.939394  ,  3.        ], dtype=float32)

Now we didn't have to explicitly tell the `eval` function about our session.  We'll leave this session open for the rest of the lecture.

## Tensor Shapes

```python
# We can find out the shape of a tensor like so:
print(x.get_shape())

# %% Or in a more friendly format
print(x.get_shape().as_list())
```

    (100,)
    [100]

## Many Operations

Lets try a set of operations now.  We'll try to create a Gaussian curve.  This should resemble a normalized histogram where most of the data is centered around the mean of 0.  It's also sometimes refered to by the bell curve or normal curve.

```python
# The 1 dimensional gaussian takes two parameters, the mean value, and the standard deviation, which is commonly denoted by the name sigma.
mean = 0.0
sigma = 1.0

# Don't worry about trying to learn or remember this formula.  I always have to refer to textbooks or check online for the exact formula.
z = (tf.exp(tf.neg(tf.pow(x - mean, 2.0) /
                   (2.0 * tf.pow(sigma, 2.0)))) *
     (1.0 / (sigma * tf.sqrt(2.0 * 3.1415))))
```

Just like before, amazingly, we haven't actually computed anything.  We *have just added a bunch of operations to Tensorflow's graph.  Whenever we want the value or output of this operation, we'll have to explicitly ask for the part of the graph we're interested in before we can see its result.  Since we've created an interactive session, we should just be able to say the name of the Tensor that we're interested in, and call the `eval` function:

```python
res = z.eval()
plt.plot(res)
# if nothing is drawn, and you are using ipython notebook, uncomment the next two lines:
%matplotlib inline
plt.plot(res)
```

# Deep Nets In TensorFlow

# The Perceptron

The perceptron is a simple model developed by scientist Frank Rosenblatt in the 60s that serves as the basis for modern neural nets.

# Gradient Descent

Whenever we create a neural network, we have to define a set of operations.  These operations try to take us from some input to some output.  For instance, the input might be an image, or frame of a video, or text file, or sound file.  The operations of the network are meant to transform this input data into something meaningful that we want the network to learn about.

Initially, all of the parameters of the network are random.  So whatever is being output will also be random.  But let's say we need it to output something specific about the image.  To teach it to do that, we're going to use something called "Gradient Descent".  Simply, Gradient descent is a way of optimizing a set of parameters.

Let's say we have a few images, and know that given a certain image, when I feed it through a network, its parameters should help the final output of the network be able to spit out the word "orange", or "apple", or some appropriate *label* given the image of that object.  The parameters should somehow accentuate the "orangeness" of my image.  It probably will be able to transform an image in away that it ends up having high intensities for images that have the color orange in them, and probably prefer images that have that color in a fairly round arrangement.

Rather than hand crafting all of the possible ways an orange might be manifested, we're going to learn the best way to optimize its objective: separating oranges and apples.  How can we teach a network to learn something like this?

## Defining Cost

Well we need to define what "best" means.  In order to do so, we need a measure of the "error".  Let's continue with the two options we've been using: orange, or apple.  I can represent these as 0 and 1 instead.

I'm going to get a few images of oranges, and apples, and one by one, feed them into a network that I've randomly initialized.  I'll then filter the image, by just multiplying every value by some random set of values.  And then I'll just add up all the numbers, and then squash the result in a way that means I'll only ever get 0 or 1.  So I put in an image, and I get out a 0 or 1.  Except, the parameters of my network are totally random, and so my network will only ever spit out random 0s or 1s.  How can I get this random network to know when to spit out a 0 for images of oranges, and a 1 for images of apples?

We do that by saying, if the network predicts a 0 for an orange, then the error is 0.  If the network predicts a 1 for an orange, then the error is 1.  And vice-versa for apples.  If it spits out a 1 for an apple, then the error is 0.  If it spits out a 0 for an apple, then the error is 1.  What we've just done is create a function which describes error in terms of our parameters:

Let's write this another way:

\begin{align}
\text{error} = \text{network}(\text{image}) - \text{true_label}
\end{align}

where

\begin{align}
\text{network}(\text{image}) = \text{prediected_label}
\end{align}

More commonly, we'll see these components represented by the following letters:

\begin{align}
E = f(X) - y
\end{align}

Don't worry about trying to remember this equation.  Just see how it is similar to what we've done with the oranges and apples.  `X` is generally the input to the network, which is fed to some network, or a function $f$, which we know should output some label `y`.  Whatever difference there is between what it should output, y, and what it actually outputs, $f(x)$ is what is different, or error, $E$.

## Minimizing Error

Instead of feeding one image at a time, we're going to feed in many.  Let's say 100.  This way, we can see what our network is doing on average.  If our error at the current network parameters is e.g. 50/100, we're correctly guessing about 50 of the 100 images.

Now for the crucial part.  If we move our network's parameters a tiny bit and see what happens to our error, we can actually use that knowledge to find smaller errors.  Let's say the error went up after we moved our network parameters.  Well then we know we should go back the way we came, and try going the other direction entirely.  If our error went down, then we should just keep changing our parameters in the same direction.  The error provides a "training signal" or a measure of the "loss" of our network.  You'll often hear anyone number of these terms to describe the same thing, "Error", "Cost", "Loss", or "Training Signal'.  That's pretty much gradient descent in a nutshell.  Of course we've made a lot of assumptions in assuming our function is continuous and differentiable.  But we're not going to worry about that, and if you don't know what that means, don't worry about it.

## Backpropagation

To summarize, Gradient descent is a simple but very powerful method for finding smaller measures of error by following the negative direction of its gradient.  The gradient is just saying, how does the error change at the current set of parameters?

One thing I didn't mention was how we figure out what the gradient is.  In order to do that, we use something called backpropagation.  When we pass as input something to a network, it's doing what's called forward propagation.  We're sending an input and multiplying it by every weight to an expected output.  Whatever differences that output has with the output we wanted it to have, gets *backpropagated* to every single parameter in our network.  Basically, backprop is a very effective way to find the gradient by simply multiplying many partial derivatives together.  It uses something called the chain rule to find the gradient of the error with respect to every single parameter in a network, and follows this error from the output of the network, all the way back to the input.

While the details won't be necessary for this course, we will come back to it in later sessions as we learn more about how we can use both backprop and forward prop to help us understand the inner workings of deep neural networks.

If you are interested in knowing more details about backprop, I highly recommend both Michael Nielsen's online Deep Learning book:

http://neuralnetworksanddeeplearning.com/

and Yoshua Bengio's online book:

http://www.deeplearningbook.org/

## Local Minima/Optima

```python
xs = np.linspace(-6, 6, 100)
plt.plot(xs, np.maximum(xs, 0), label='relu')
plt.plot(xs, 1 / (1 + np.exp(-xs)), label='sigmoid')
plt.plot(xs, np.tanh(xs), label='tanh')
plt.xlabel('Input')
plt.xlim([-6, 6])
plt.ylabel('Output')
plt.ylim([-1.5, 1.5])
plt.title('Common Activation Functions/Nonlinearities')
plt.legend(loc='lower right')
```

```python

```


# Customize the module before install

Let's suppose we have identified two backends that we like (one for 2D plots, the other for 3D plots). Then, instead of passing in the keyword `backend=SOMETHING` each time we need to create a plot, we can customize the module to make the plotting functions use our backends.

Let's import the necessary tools:

```python
from spb.defaults import cfg, set_defaults
display(cfg)
```

```python
help(set_defaults)
```

We need to change the values in the `cfg` dictionary and then use the `set_defaults` function to apply the new configurations.

Let's say we would like to:
* use Bokeh for 2D plots and Plotly for 3D plots;
* use `"seaborn"` theme in Plotly.

Then:

```python
# we write the name of the plotting library
# available options: bokeh, matplotlib, mayavi, k3d, plotly
cfg["backend_2D"] = "bokeh"
cfg["backend_3D"] = "plotly"
# the following depends on the plotting library
cfg["plotly"]["theme"] = "seaborn"
set_defaults(cfg)
```

We can test our changes right away.

```python
from sympy import *
from spb import *
from spb.backends.plotly import PB
var("u, v, x, y")
plot(sin(x), cos(x), log(x), legend=True)
```

```python
n = 125
r = 2 + sin(7 * u + 5 * v)
expr = (
    r * cos(u) * sin(v),
    r * sin(u) * sin(v),
    r * cos(v)
)
plot3d_parametric_surface((*expr, (u, 0, 2 * pi), (v, 0, pi), "expr"), n=n, backend=PB)
```

Let's now discuss a few customization options:

``` python
# Set the location of the intersection between the horizontal and vertical
# axis of Matplotlib (only works for 2D plots). Possible values:
#       "center", "auto" or None
# If None, use a standard Matplotlib layout with vertical axis on the left,
# horizontal axis on the bottom.
cfg["matplotlib"]["axis_center"] = None

# Turn on the grid on Matplotlib plots
cfg["matplotlib"]["grid"] = True

# Find more Plotly themes at the following page:
# https://plotly.com/python/templates/
cfg["plotly"]["theme"] = "plotly_dark"

# Find more Bokeh themes at the following page:
# https://docs.bokeh.org/en/latest/docs/reference/themes.html
cfg["bokeh"]["theme"] = "dark_minimal"

# K3D-Jupyter colors are represented by an integer number.
# For example, to set a white background:
cfg["k3d"]["bg_color"] = 0xffffff

# we can set the numerical evaluation library for complex plots.
# Available options: "mpmath" or None (the latter uses Numpy/Scipy)
cfg["complex"] = "mpmath"

# set a default (complex) domain coloring option.
cfg["coloring"] = "b"
```

Remember: every time we change a custom option, it is recommended to restart the kernel in order to make changes effective.

```python

```


```python
from sympy import *
```

We consider kernels of the form

$$k(\nu(\boldsymbol{x},\boldsymbol{l}))=k\circ\nu(\boldsymbol{x},\boldsymbol{l}).$$

```python
x, th, l = symbols(r'x, \theta, l')
nu = Function(r'\nu', real=True)(x, th)
dnudx = Function(r'\nu_x')(x, th)
dnudth = Function(r'\nu_\theta')(x, th)
d2nudx2 = Function(r'\nu_{xx}')(x, th)
d2nudth2 = Function(r'\nu_{\theta\theta}')(x, th)
d2nudxdth = Function(r'\nu_{x\theta}')(x, th)
dkdr = Function(r'k_r')(nu)
d2kdr2 = Function(r'k_{rr}')(nu)
k = Function(r'k', real=True)(nu)
```

Derivatives of $k$ w.r.t. $x$ and $l$ are

```python
def subs_explicit(f):
    return (
        f.subs(diff(nu, x), dnudx).subs(diff(nu, th), dnudth).subs(diff(k, nu), dkdr)
        .subs(diff(dnudx, x), d2nudx2).subs(diff(dnudx, th), d2nudxdth).subs(diff(dkdr, nu), d2kdr2)
    )

subs_explicit(diff(k, x))
diff(k, x, x, th).simplify()
```

$\displaystyle \left(\left(\frac{\partial}{\partial x} \nu{\left(x,\theta \right)}\right)^{2} \frac{\partial^{3}}{\partial \nu{\left(x,\theta \right)}^{3}} k{\left(\nu{\left(x,\theta \right)} \right)} + \frac{\partial^{2}}{\partial x^{2}} \nu{\left(x,\theta \right)} \frac{\partial^{2}}{\partial \nu{\left(x,\theta \right)}^{2}} k{\left(\nu{\left(x,\theta \right)} \right)}\right) \frac{\partial}{\partial \theta} \nu{\left(x,\theta \right)} + 2 \frac{\partial}{\partial x} \nu{\left(x,\theta \right)} \frac{\partial^{2}}{\partial \nu{\left(x,\theta \right)}^{2}} k{\left(\nu{\left(x,\theta \right)} \right)} \frac{\partial^{2}}{\partial x\partial \theta} \nu{\left(x,\theta \right)} + \frac{\partial}{\partial \nu{\left(x,\theta \right)}} k{\left(\nu{\left(x,\theta \right)} \right)} \frac{\partial^{3}}{\partial x^{2}\partial \theta} \nu{\left(x,\theta \right)}$

```python
d2kdth2_ov_k_sqexp = (diff(k, th, th)/k).subs(k, exp(-0.5*nu)).simplify().cancel(); d2kdth2_ov_k_sqexp
d2kdth2_ov_k_sqexp_l2 = d2kdth2_ov_k_sqexp.subs(nu, th*x**2).simplify(); d2kdth2_ov_k_sqexp_l2
```

$\displaystyle 0.25 x^{4}$

```python
d2kdx2_ov_k_sqexp = ((diff(k, x, x)/k).subs(k, exp(-nu/2))).simplify().cancel(); d2kdx2_ov_k_sqexp
```

$\displaystyle \frac{\left(\frac{\partial}{\partial x} \nu{\left(x,\theta \right)}\right)^{2}}{4} - \frac{\frac{\partial^{2}}{\partial x^{2}} \nu{\left(x,\theta \right)}}{2}$

```python
d2kdx2_ov_k_sqexp_l2 = d2kdx2_ov_k_sqexp.subs(nu, -2.0*th*x**2).simplify(); d2kdx2_ov_k_sqexp_l2
```

$\displaystyle 4.0 \theta^{2} x^{2} + 2.0 \theta$

```python
d2kdx2_ov_k_sqexp_l2sin = d2kdx2_ov_k_sqexp.subs(nu, sin(x*th)**2).simplify().trigsimp(); d2kdx2_ov_k_sqexp_l2sin
```

$\displaystyle \theta^{2} \left(- \sin^{4}{\left(\theta x \right)} + 3 \sin^{2}{\left(\theta x \right)} - 1\right)$

```python
d3kdx2dth_ov_k_sqexp = ((diff(k, x, x, th)/k).subs(k, exp(-0.5*nu))).simplify().cancel(); d3kdx2dth_ov_k_sqexp
```

$\displaystyle - 0.125 \frac{\partial}{\partial \theta} \nu{\left(x,\theta \right)} \left(\frac{\partial}{\partial x} \nu{\left(x,\theta \right)}\right)^{2} + 0.25 \frac{\partial}{\partial \theta} \nu{\left(x,\theta \right)} \frac{\partial^{2}}{\partial x^{2}} \nu{\left(x,\theta \right)} + 0.5 \frac{\partial}{\partial x} \nu{\left(x,\theta \right)} \frac{\partial^{2}}{\partial x\partial \theta} \nu{\left(x,\theta \right)} - 0.5 \frac{\partial^{3}}{\partial x^{2}\partial \theta} \nu{\left(x,\theta \right)}$

```python
d3kdx2dth_ov_k_sqexp_l2 = d3kdx2dth_ov_k_sqexp.subs(nu, th*x**2).simplify(); d3kdx2dth_ov_k_sqexp_l2
```

$\displaystyle - 0.5 \theta^{2} x^{4} + 2.5 \theta x^{2} - 1.0$

```python
d3kdx2dth_ov_k_sqexp_l2sin = d3kd2xdth.subs(nu, sin(x)**2*th).simplify().trigsimp().cancel(); d3kdx2dth_ov_k_sqexp_l2sin
```

```python
fcode(dkdth_ov_k_sqexp_l2).replace('\\theta', 'th(kd)').replace('x', 'x(kd, :)')
```

```python
template = """
subroutine nu_{name}

end subroutine

"""

kern_sqexp = template.format(name, 'sqexp', expr, 'exp(-sum(')subs(k, exp(-nu)).subs(nu, (x*th)**2))
```

```python

```


# Non Linear Regression Analysis

Estimated time needed: **20** minutes

## Objectives

After completing this lab you will be able to:

*   Differentiate between linear and non-linear regression
*   Use non-linear regression model in Python

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression since linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example in python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Although linear regression can do a great job at modeling some datasets, it cannot be used for all datasets. First recall how linear regression, models a dataset. It models the linear relationship between a dependent variable y and the independent variables x. It has a simple equation, of degree 1, for example y = $2x$ + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

Non-linear regression is a method to model the non-linear relationship between the independent variables $x$ and the dependent variable $y$. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).  For example:

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and so on. For example: $$ y = \log(x)$$

We can have a function that's even more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying the logarithmic map from the input $x$ to the output $y$. It is one of the simplest form of **log()**: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be a polynomial representation of the $x$ values. In general form it would be written as\
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corresponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%202/data/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

**Did you know?** When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)

### Plotting the Dataset

This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerates slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta\_1(X-\beta\_2)}}$$

$\beta\_1$: Controls the curve's steepness,

$\beta\_2$: Slides the curve on the x-axis.

### Building The Model

Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?

we can use **curve_fit** which uses non-linear least squares to fit our sigmoid function, to data. Optimize values for the parameters so that the sum of the squared residuals of sigmoid(xdata, \*popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice

Can you calculate what is the accuracy of our model?

```python
# write your code here

```

Click here for the solution

```python
# split data into train/test
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )

```

Want to learn more?

IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: SPSS Modeler

Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at Watson Studio

### Thank you for completing this lab!

## Author

Saeed Aghabozorgi

### Other Contributors

Joseph Santarcangelo

## Change Log

| Date (YYYY-MM-DD) | Version | Changed By | Change Description                 |
| ----------------- | ------- | ---------- | ---------------------------------- |
| 2020-11-03        | 2.1     | Lakshmi    | Made changes in URL                |
| 2020-08-27        | 2.0     | Lavanya    | Moved lab to course repo in GitLab |
|                   |         |            |                                    |
|                   |         |            |                                    |

##
 © IBM Corporation 2020. All rights reserved.

9.Machine Learning with Python/ML0101EN-Reg-NoneLinearRegression.ipynb


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import pyplot
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix
import math

import keras
from keras import optimizers
from keras.models import Sequential
from keras.layers import Dense

import warnings
warnings.filterwarnings("ignore")
%matplotlib inline
```

    Using TensorFlow backend.

```python
df = pd.read_csv("Churn_Modelling.csv")
```

# Pre-processing

```python
df.head()
```

      RowNumber
      CustomerId
      Surname
      CreditScore
      Geography
      Gender
      Age
      Tenure
      Balance
      NumOfProducts
      HasCrCard
      IsActiveMember
      EstimatedSalary
      Exited

      0
      1
      15634602
      Hargrave
      619
      France
      Female
      42
      2
      0.00
      1
      1
      1
      101348.88
      1

      1
      2
      15647311
      Hill
      608
      Spain
      Female
      41
      1
      83807.86
      1
      0
      1
      112542.58
      0

      2
      3
      15619304
      Onio
      502
      France
      Female
      42
      8
      159660.80
      3
      1
      0
      113931.57
      1

      3
      4
      15701354
      Boni
      699
      France
      Female
      39
      1
      0.00
      2
      0
      0
      93826.63
      0

      4
      5
      15737888
      Mitchell
      850
      Spain
      Female
      43
      2
      125510.82
      1
      1
      1
      79084.10
      0

As it can be seen, the variables "RowNumber", "CustomerId", and "Surname" are not useful for the model.

The values of these variables are unique for each observation.

```python
X = df.iloc[:, 3:13]
y = df.iloc[:, 13]
```

### Creating dummy variables
Machine learning models cannot process words, for this reason, columns that show categories need to be converted into dummy variables.

For example, the variable "Gender" will be transformed into a dummy variable with 0 being a female and 1 being a male. However, variables that

include more than two categories, will be split into more columns.

For instance, the "Geography" category has three levels, namely Germany, Spain, and France. This variable needs to be transformed into two binary columns.

In general, the number of columns/variables that will be created from a categorical variable are c-1, where c is the number of levels.

```python
cat = [1,2]
cat_cols = pd.get_dummies(X.iloc[:, cat], drop_first=True)
```

```python
cat_cols.head()
```

      Geography_Germany
      Geography_Spain
      Gender_Male

      0
      0
      0
      0

      1
      0
      1
      0

      2
      0
      0
      0

      3
      0
      0
      0

      4
      0
      1
      0

```python
X.drop(X.columns[cat], axis=1, inplace=True)
```

```python
X = pd.concat([X,cat_cols], axis=1)
```

```python
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)
```

Standardize the variables for the Neural Network

```python
sc = StandardScaler()
X_train = pd.DataFrame(sc.fit_transform(X_train),columns = X_train.columns)
X_test = pd.DataFrame(sc.transform(X_test),columns = X_test.columns)
```

Now, in an Artificial Neural Network, there is an input layer, some hidden layers and an output layer.

The input layer corresponds to a row/observation that travels through the neural network and outputs a specific value.

So, in the image below,
X1, X2, and X3, are the independent variables for one row. In addition, W1, W1, and W3, are the weights for each variable. Note that this image does not depict the current Neural Network that we are using for this problem, but it rather shows a Neural Network with a dataset of 3 independent variables and a dependent variable.

The neuron in the middle, applies an activation function based on the signals it receives. There are different activation functions, but one of the most common activation functions for the hidden layers is the Rectified Linear Unit (ReLU).
\begin{align}
f (x) = max(x,0)
\end{align}

What the ReLU does, is that it returns x, if the $\sum_{i=1}^{n} W_{i}X_{i}$ is above zero and 0 otherwise, where $x = \sum_{i=1}^{n} W_{i}X_{i}$

```python
# rectified linear function
def rectified(x):
    return max(0.0, x)

# define a series of inputs
series_in = [x for x in range(-10, 11)]
# calculate outputs for our inputs
series_out = [rectified(x) for x in series_in]
# line plot of raw inputs to rectified outputs
pyplot.plot(series_in, series_out)
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.title('ReLU Function')
pyplot.show()
```

The Neural Network above includes only an input layer and an output layer. For a binary classification problem such as this one, where the customer will either churn or not, the sigmoid activation function for the output layer is a more proper function.

\begin{align}
f(x) = \frac{1}{1 + e^{-x}}
\end{align}

The Sigmoid function returns values ranging from 0 to 1, which is useful because in this situation, the dependent variable is either 0 (not churn) or 1 (churn).

Later, by applying a threshold (e.g., 0.5), we can assign values above this threshold as 1 and below as 0.

```python
def sigmoid(x):
    return (1/(1 + math.exp(-x)))

# define a series of inputs
series_in = [x for x in range(-10, 11)]
# calculate outputs for our inputs
series_out = [sigmoid(x) for x in series_in]
# line plot of raw inputs to sigmoid outputs
pyplot.plot(series_in, series_out)
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.title('Sigmoid Function')
pyplot.show()
```

The Neural Network below, includes an input layer with 3 neurons, a hidden layer with 4 neurons, and an output layer.

In this example, the activation function of the hidden layer can be the ReLU function whereas for the output layer, it can be the sigmoid function.

### Neural Network

The ANN is build using the Keras framework, which uses TensorFlow in the backend.

In the code below, the number of nodes in the input layers needs to be defined. This is the "input_dim" argument, which corresponds to the number of independent variables. Next, the "units" argument, defines the number of neurons in the hidden layer. Note that the architecture below shows a Neural Network with two hidden layers. The input layer and the first hidden layer are both defined in one line of code, while the rest of the layers are defined later.

This means that the hidden layers have 6 neurons each with a ReLU activation function as described above while the output layer has 1 neuron and a sigmoid activation function as this is a binary classification problem.

Next, the ANN model is compiled where the optimizer, the loss, and the metric is being defined. The optimizer argument defines the optimizer that will be used in order to update the weights of the model. Amongst those are the Stochastic Gradient Descent, the Adaptive Moment Estimation (Adam), and the Root Mean Squared Propagetion (RMSProp). The most popular optimizers are the Adam and the RMSProp.

For this problem, the binary cross entropy (logarithmic) loss is used.

\begin{align}
\small
Logarithmic Loss = - \frac{1}{n} \; \sum_{i=1}^{n} \bigg(y^{i} \; log\phi \bigg( \sum_{j=1}{m}w_{j}x_{j}^{i} \bigg) + (1-y^{i})log \bigg(1 - \phi \bigg(\sum_{j=1}^{m}w_{j}x_{j}^{i} \bigg) \bigg) \bigg)
\end{align}

where $\phi$ is the sigmoid function, i is the index of the observation and j is the index of the feature.

The metric, is the metric that will be shown, and in this case it is the accuracy.

### Model Architecture

```python
classifier = Sequential()

classifier.add(Dense(units=6, activation='relu',input_dim=11))
classifier.add(Dense(units=6, activation='relu'))
classifier.add(Dense(units=1, activation='sigmoid'))
classifier.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

```

    WARNING:tensorflow:From C:\Users\Hella\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
    Instructions for updating:
    Colocations handled automatically by placer.

The architecture of the current ANN is shown below.

```python
print(classifier.summary())
```

    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    dense_1 (Dense)              (None, 6)                 72
    _________________________________________________________________
    dense_2 (Dense)              (None, 6)                 42
    _________________________________________________________________
    dense_3 (Dense)              (None, 1)                 7
    =================================================================
    Total params: 121
    Trainable params: 121
    Non-trainable params: 0
    _________________________________________________________________
    None

After defining the model architecture, in order to train the model to the training data, the batch size as well as the number of epochs need to be defined.

The batch size essentially shows after how many observations the weights will be updated. For instance, with a batch size of 32, 32 observations will pass through the ANN and after that, the weights will be updated.

Next, the number of epochs shows how many times all the observations will pass through the neural network.

```python
classifier.fit(X_train, y_train, batch_size=32, epochs=50)
```

    WARNING:tensorflow:From C:\Users\Hella\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
    Instructions for updating:
    Use tf.cast instead.
    Epoch 1/50
    8000/8000 [==============================] - 4s 477us/step - loss: 0.5187 - acc: 0.7902
    Epoch 2/50
    8000/8000 [==============================] - 1s 132us/step - loss: 0.4584 - acc: 0.7945
    Epoch 3/50
    8000/8000 [==============================] - 1s 137us/step - loss: 0.4435 - acc: 0.7945
    Epoch 4/50
    8000/8000 [==============================] - 1s 128us/step - loss: 0.4356 - acc: 0.7943
    Epoch 5/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.4285 - acc: 0.8103
    Epoch 6/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.4218 - acc: 0.8197
    Epoch 7/50
    8000/8000 [==============================] - 1s 121us/step - loss: 0.4162 - acc: 0.8216
    Epoch 8/50
    8000/8000 [==============================] - 1s 121us/step - loss: 0.4115 - acc: 0.8233
    Epoch 9/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.4076 - acc: 0.8239
    Epoch 10/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.4039 - acc: 0.8249
    Epoch 11/50
    8000/8000 [==============================] - 1s 125us/step - loss: 0.4004 - acc: 0.8246
    Epoch 12/50
    8000/8000 [==============================] - 1s 133us/step - loss: 0.3975 - acc: 0.8261
    Epoch 13/50
    8000/8000 [==============================] - 1s 131us/step - loss: 0.3945 - acc: 0.8265
    Epoch 14/50
    8000/8000 [==============================] - 1s 131us/step - loss: 0.3922 - acc: 0.8267
    Epoch 15/50
    8000/8000 [==============================] - 1s 133us/step - loss: 0.3897 - acc: 0.8277
    Epoch 16/50
    8000/8000 [==============================] - 1s 133us/step - loss: 0.3870 - acc: 0.8281
    Epoch 17/50
    8000/8000 [==============================] - 1s 130us/step - loss: 0.3847 - acc: 0.8301
    Epoch 18/50
    8000/8000 [==============================] - 1s 121us/step - loss: 0.3821 - acc: 0.8286
    Epoch 19/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3796 - acc: 0.8339
    Epoch 20/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3773 - acc: 0.8410
    Epoch 21/50
    8000/8000 [==============================] - 1s 128us/step - loss: 0.3749 - acc: 0.8415
    Epoch 22/50
    8000/8000 [==============================] - 1s 120us/step - loss: 0.3726 - acc: 0.8427
    Epoch 23/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.3701 - acc: 0.8449
    Epoch 24/50
    8000/8000 [==============================] - 1s 126us/step - loss: 0.3676 - acc: 0.8474
    Epoch 25/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3660 - acc: 0.8471
    Epoch 26/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.3637 - acc: 0.8514
    Epoch 27/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.3615 - acc: 0.8500
    Epoch 28/50
    8000/8000 [==============================] - 1s 121us/step - loss: 0.3604 - acc: 0.8540
    Epoch 29/50
    8000/8000 [==============================] - 1s 124us/step - loss: 0.3580 - acc: 0.8538
    Epoch 30/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.3567 - acc: 0.8526
    Epoch 31/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3550 - acc: 0.8561
    Epoch 32/50
    8000/8000 [==============================] - 1s 134us/step - loss: 0.3533 - acc: 0.8575
    Epoch 33/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.3517 - acc: 0.8578
    Epoch 34/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3506 - acc: 0.8578
    Epoch 35/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3492 - acc: 0.8594
    Epoch 36/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.3487 - acc: 0.8592
    Epoch 37/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.3473 - acc: 0.8606
    Epoch 38/50
    8000/8000 [==============================] - 1s 124us/step - loss: 0.3469 - acc: 0.8587
    Epoch 39/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.3459 - acc: 0.8584
    Epoch 40/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3450 - acc: 0.8589
    Epoch 41/50
    8000/8000 [==============================] - 1s 122us/step - loss: 0.3446 - acc: 0.8604
    Epoch 42/50
    8000/8000 [==============================] - 1s 117us/step - loss: 0.3436 - acc: 0.8594
    Epoch 43/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3432 - acc: 0.8591
    Epoch 44/50
    8000/8000 [==============================] - 1s 119us/step - loss: 0.3429 - acc: 0.8590
    Epoch 45/50
    8000/8000 [==============================] - 1s 122us/step - loss: 0.3421 - acc: 0.8592
    Epoch 46/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3418 - acc: 0.8608
    Epoch 47/50
    8000/8000 [==============================] - 1s 118us/step - loss: 0.3415 - acc: 0.8599
    Epoch 48/50
    8000/8000 [==============================] - 1s 127us/step - loss: 0.3407 - acc: 0.8596
    Epoch 49/50
    8000/8000 [==============================] - 1s 128us/step - loss: 0.3407 - acc: 0.8595
    Epoch 50/50
    8000/8000 [==============================] - 1s 122us/step - loss: 0.3407 - acc: 0.8601

    <keras.callbacks.History at 0x2b8a8328b00>

Furthermore, as explained previously, the model will make predictions on the test set, and a threshold will be set in order to classify the customers that are going to churn and those that are not.

The threshold that is applied here is 0.5.

```python
y_pred = classifier.predict(X_test)
y_pred = (y_pred > 0.5)
```

Finally, with a confusion matrix, we can see how many observations were correctly predicted. More specifically, a confusion matrix gives information about the True Positive rate, True Negative rate, False Positive rate, and False Negative rate. It is also useful in cases when the dataset is severely unbalanced. In these cases, the confusion matrix shows if the model predicts every observation into one class. The accuracy values in these unbalanced datasets can be misleading, as it can be very high.

For this particular problem, the dataset is balanced.

```python
labels = [0, 1]
cm = confusion_matrix(y_test, y_pred, labels)
print(cm)
print('Accuracy: ',(cm[0,0]+cm[1,1])/len(y_test)*100)
```

    [[1528   79]
     [ 206  187]]
    Accuracy:  85.75

For this classification problem regarding customer churn, the model correctly predicted that 1528 customers will not churn and 187 customers will churn. In addition, it falsely predicted that 79 customers will churn and that 206 will not churn. The accuracy of the model is 85.75%. Below, a plot of the confusion matrix is shown.

```python
fig = plt.figure()
ax = fig.add_subplot(111)
cax = ax.matshow(cm)
plt.title('Confusion matrix of the classifier')
fig.colorbar(cax)
ax.set_xticklabels([''] + ['Not Churn', 'Churn'])
ax.set_yticklabels([''] + ['Not Churn', 'Churn'])
plt.xlabel('Predicted')
plt.ylabel('True')
plt.show()
```


```python
import os
import pandas as pd
import pickle
from tqdm.notebook import tqdm
import itertools
from functools import reduce

%matplotlib inline
import matplotlib
from matplotlib import pyplot as plt
import seaborn as sns
from IPython.display import set_matplotlib_formats
set_matplotlib_formats('retina')
sns.set(rc={'figure.figsize':(11.7,8.27)})
sns.set_palette(sns.color_palette())
from plotnine import *
from mizani.palettes import brewer_pal
from palettable import colorbrewer
palette=brewer_pal(type='qual', palette='Set1')

from mcmcmd.samplers import *
from mcmcmd.tests import *

import igraph
import joblib
import grakel

import scipy
```

    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/sklearn/utils/deprecation.py:143: FutureWarning: The sklearn.datasets.lfw module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.datasets. Anything that cannot be imported from sklearn.datasets is now part of the private API.

# Helper functions

```python
def b_nk(n, k, A, B, B_terms):
    if (n, k) not in B:
        if n == k:
            B_terms[(n, k)] = onp.array([1])
            B[(n, k)] = 1
        else:
            s = onp.arange(1, n-k+1)
            B_terms[(n, k)] = ((2**k-1)**s * 2**(k*(n-k-s)) * A[n-k-1, :][s-1]).astype('int')
            B[(n, k)] = int(B_terms[(n, k)].sum())
    return B[(n, k)]

def count_DAG(num_nodes):
    A = onp.identity(num_nodes, dtype='int')
    B = {}
    B_terms = {}

    for n in range(1, num_nodes+1):
        for k in range(1, n+1):
            A[n-1, k-1] = scipy.special.comb(n, k, exact=True) * b_nk(n, k, A, B, B_terms)

    # A[n-1, k-1] is the number of DAGs with n nodes and k outpoints (roots).
    return A

# Sample a DAG uniformly using the enumeration method from Kuipers and Moffa 2013.
def sample_uniform_DAG(num_nodes, rng=None):
    assert num_nodes > 0 and num_nodes <= 10 # overflow
    if rng is None:
        rng = onp.random.default_rng()

    A = count_DAG(num_nodes)

    K = [] # List of k
    n = num_nodes

    r = rng.choice(A[n-1, :].sum())+1

    k = 1
    while r > A[n-1, k-1]:
        r -= A[n-1, k-1]
        k += 1
    K.append(k)

    r = int(onp.ceil(r/scipy.special.comb(n, k, exact=True)))
    m = n-k
    while m > 0:
        s = 1
        t = int((2**k - 1)**s * 2**(k*(m-s)) * A[m-1, s-1])
        while r > t:
            r -= t
            s += 1
            t = (2**k - 1)**s * 2**(k*(m-s)) * A[m-1, s-1]

        r = int(onp.ceil(r/(scipy.special.comb(m, s, exact=True) * t/A[m-1, s-1])))

        n = m
        k = s
        m = n-k
        K.append(k)

    Q = onp.zeros(shape=(num_nodes, num_nodes), dtype='int')
    j = K[-1]
    for i in range(len(K)-1, 0, -1):
        for l in range(j-K[i], j):
            bln_zeroCol = True
            while bln_zeroCol:
                for m in range(j, j+K[i-1]):
                    Q[m, l] = rng.choice(2)
                    if Q[m, l] == 1:
                        bln_zeroCol = False

            for m in range(j+K[i-1], num_nodes):
                Q[m, l] = rng.choice(2)
        j += K[i-1]

    node_labels = rng.permutation(num_nodes)
    Q = Q[node_labels, :][:, node_labels]
    return Q

# Check if DAG is cyclic
def isCyclic(adj_matrix):
    if onp.diag(adj_matrix).sum() != 0:
        return True

    num_nodes = adj_matrix.shape[0]
    nodes = onp.arange(num_nodes)

    done = False
    notLeaf = adj_matrix.sum(1).astype('bool')
    while not done:
        adj_matrix = adj_matrix[notLeaf, :][:, notLeaf]
        notLeaf = adj_matrix.sum(1).astype('bool')
        if notLeaf.sum() == len(notLeaf):
            done = True

    return adj_matrix.shape[0] != 0

# Sample from likelihood
def sample_DAG_data(adj_matrix, N=1, epsilon=1, rng=None):
    if rng is None:
        rng = onp.random.default_rng()

    num_nodes = adj_matrix.shape[0]
    nodes = onp.arange(num_nodes)

    x = onp.zeros(shape=(N, num_nodes))
    node_gen = onp.zeros(num_nodes)
    node_gen_count = 0

    isRoot = ~adj_matrix.sum(0).astype('bool')
    roots = nodes[isRoot]
    if len(roots) == 0:
        raise ValueError('adj_matrix encodes a cyclic graph!')

    children = onp.zeros(num_nodes, dtype='int')
    for r in roots:
        x[:, r] = rng.normal(0, epsilon, size = N)
        node_gen[r] = 1
        node_gen_count += 1
        children += adj_matrix[r, :]

    while node_gen_count < num_nodes:
        for child in nodes[children.astype('bool')]:
            if node_gen[child] == 1:
                raise ValueError('adj_matrix encodes a cyclic graph!')
            parents = nodes[adj_matrix[:, child] == 1]
            if node_gen[parents].sum() == len(parents):
                x[:, child] = rng.normal(x[:, parents].sum(1), epsilon, size = N)
                node_gen[child] = 1
                node_gen_count += 1
                children += adj_matrix[child, :]
                children[child] = 0
    return x

# Calculate log-evidence
def log_likelihood(X, adj_matrix, epsilon):
    num_nodes = adj_matrix.shape[0]
    n, p = X.shape
    mu = (adj_matrix.T.reshape(1, num_nodes, num_nodes) @ X.reshape(n, p, 1)).sum(2)
    return scipy.stats.norm.logpdf(loc=mu, scale=epsilon, x=X).sum()

# Modify an edge
def changeEdge_DAG(adj_matrix, i, j, change_type='toggle'):
    assert change_type in ['toggle', 'reverse']
    proposal = adj_matrix.copy()

    if i == -1 and j == -1:
        return proposal

    if change_type == 'reverse':
        if adj_matrix[i, j] == 1:
            proposal[i, j] = 0
            proposal[j, i] = 1
        elif adj_matrix[i, j] == 0:
            return proposal
#             raise ValueError('Cannot reverse non-existent edge')
        else:
            raise ValueError('adj_matrix is non-binary')
    else:
        if adj_matrix[i, j] == 1:
            proposal[i, j] = 0
        elif adj_matrix[i, j] == 0:
            proposal[i, j] = 1
        else:
            raise ValueError('adj_matrix is non-binary')

    return proposal

# Enumerate all DAGs that can be reached by adding/deleting/reversing edges. Optionally sample one uniformly at random
def neighbors_DAG(adj_matrix, return_sample=False, rng=None):
    if rng is None:
        rng = onp.random.default_rng()

    nodes = onp.arange(adj_matrix.shape[0])
    edges = [(i, j) for i, j in itertools.product(nodes, nodes)]
    edges += [(-1, -1)] # no change

    lst_proposals = []
    for i, j in edges:

        # No change
        if i == -1 and j == -1:
            lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

        # Skip diagonals
        if i == j:
            continue

        # Add DAG reached by toggling edge i,j
        proposal = changeEdge_DAG(adj_matrix, i, j, change_type='toggle')
        if not isCyclic(proposal):
            lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

        # Add DAG reached by reversing edge i,j
        if adj_matrix[i, j] == 1:
            proposal = changeEdge_DAG(adj_matrix, i, j, change_type='reverse')
            if not isCyclic(proposal):
                lst_proposals.append({'i':i, 'j':j, 'change_type':'reverse'})

    k = len(lst_proposals)
    if return_sample == True:
        args = rng.choice(lst_proposals)
        proposal = changeEdge_DAG(adj_matrix, **args)
        return k, proposal
    else:
        return k

# Row-wise
def array_to_strings(z):
    z = z.astype('int').astype('str')
    z = onp.hsplit(z, z.shape[1])
    z = reduce(onp.char.add, z)
    return z

def count_sample_DAG(z):
    z = array_to_strings(z)
    vals, counts = onp.unique(z, return_counts = True)
    return vals, counts

# Generate all possible kernel evaluations for caching
def graph_kernel_cache(num_nodes, kernel_type='random_walk', **kwargs):
    # Sample until all graphs are enumerated... inefficient
    num_DAGs = count_DAG(num_nodes)[-1, :].sum()
    sample_size = 5*num_DAGs
    graph_ids = count_sample_DAG(onp.vstack([sample_uniform_DAG(num_nodes).reshape(1,num_nodes**2) for _ in range(sample_size)]))[0]
    while len(graph_ids) != num_DAGs:
        graph_ids=onp.unique(onp.hstack([graph_ids, count_sample_DAG(onp.vstack([sample_uniform_DAG(num_nodes).reshape(1,num_nodes**2) for _ in range(sample_size)]))[0]]))

    node_labels = dict(zip(onp.arange(num_nodes), onp.arange(num_nodes)))
    graphs = [grakel.Graph(initialization_object=onp.array(list(g)).astype('int').reshape(num_nodes, num_nodes), node_labels=node_labels) for g in graph_ids]

    if kernel_type == 'random_walk':
        K = grakel.RandomWalk(**kwargs)
    elif kernel_type == 'wl':
        K = grakel.WeisfeilerLehman(**kwargs)
    elif kernel_type == 'shortest_path':
        K = grakel.ShortestPath(**kwargs)
    else:
        raise ValueError(f'`kernel_type={kernel_type}` not recognized')
    index_table = dict(zip(graph_ids, onp.arange(num_DAGs))) # lookup table
    kernel_table = K.fit_transform(graphs)
    return kernel_table, index_table

# Calculate graph kernel using adjacency matrices
class graph_kernel(kernel):
    def __init__(self, X, Y, kernel_type=None, cache=True, cache_index_table=None, cache_kernel_table=None, **kwargs):
        assert X.shape[1] == Y.shape[1]
        assert len(X.shape) == 2 and len(X.shape) == len(Y.shape)
        assert int(onp.sqrt(X.shape[1]))**2 == X.shape[1] # adjacency matrix must be square
        if kernel_type is None:
            kernel_type = 'random_walk'

        # Graph ids
        self._ids_X = array_to_strings(X).flatten()
        self._ids_Y = array_to_strings(Y).flatten()

        self._num_nodes = int(onp.sqrt(X.shape[1]))
        self._node_labels = dict(zip(onp.arange(self._num_nodes), onp.arange(self._num_nodes)))
        self._X = [grakel.Graph(initialization_object=X[i,:].reshape(self._num_nodes, self._num_nodes), node_labels=self._node_labels) for i in range(X.shape[0])]
        self._Y = [grakel.Graph(initialization_object=Y[i,:].reshape(self._num_nodes, self._num_nodes), node_labels=self._node_labels) for i in range(Y.shape[0])]
        self._kernel_type = kernel_type
        if self._kernel_type == 'random_walk':
            self._K = grakel.RandomWalk(**kwargs)
        elif self._kernel_type == 'wl':
            self._K = grakel.WeisfeilerLehman(**kwargs)
        elif self._kernel_type == 'shortest_path':
            self._K = grakel.ShortestPath(**kwargs)
        else:
            raise ValueError(f'`kernel_type={self._kernel_type}` not recognized')

        self._cached = False
        if cache is True or (cache_index_table is None and cache_kernel_table is None):
            self.cache(index_table=cache_index_table, kernel_table=cache_kernel_table)

        pass

    @property
    def params(self):
        return self._kernel_type

    def set_params(self, params):
        self._kernel_type = params
        pass

    def learn(self, eval=False):
        if eval==True:
            return self.eval()
        else:
            pass

    def cache(self, index_table=None, kernel_table=None):
        if index_table is not None and kernel_table is not None:
            self._index_table = index_table
            self._kernel_table = kernel_table
        else:
            num_DAGs = count_DAG(self._num_nodes)[-1, :].sum()
            sample_size = 5*num_DAGs
            graph_ids = count_sample_DAG(onp.vstack([sample_uniform_DAG(self._num_nodes).reshape(1,self._num_nodes**2) for _ in range(sample_size)]))[0]
            while len(graph_ids) != num_DAGs:
                graph_ids=onp.unique(onp.hstack([graph_ids, count_sample_DAG(onp.vstack([sample_uniform_DAG(self._num_nodes).reshape(1,self._num_nodes**2) for _ in range(sample_size)]))[0]]))

            graphs = [grakel.Graph(initialization_object=onp.array(list(g)).astype('int').reshape(self._num_nodes, self._num_nodes), node_labels=self._node_labels) for g in graph_ids]

            self._index_table = dict(zip(graph_ids, onp.arange(num_DAGs))) # lookup table
            self._kernel_table = self._K.fit_transform(graphs)

        if self._kernel_type == 'shortest_path':
            self._kernel_table = self._kernel_table.astype('int')

        self._cached = True
        pass

    def eval(self):
        if self._cached == False:
            return self._K.fit(self._X).transform(self._Y)
        else:
            kernel_eval = onp.zeros(shape=(len(self._ids_X), len(self._ids_Y)), dtype=self._kernel_table.dtype)
            for i, id_X in enumerate(self._ids_X):
                for j, id_Y in enumerate(self._ids_Y):
                    kernel_eval[i,j] = self._kernel_table[self._index_table[id_X], self._index_table[id_Y]]
            return kernel_eval

    def f_kernel(self, x, y, **kwargs):
        assert len(x.shape) == len(y.shape) and len(x.shape) == 1
        if self._cached == False:
            x_graph = [grakel.Graph(initialization_object=x.reshape(self._num_nodes, self._num_nodes), node_labels=self._node_labels)]
            y_graph = [grakel.Graph(initialization_object=y.reshape(self._num_nodes, self._num_nodes), node_labels=self._node_labels)]
            return float(self._K.fit(x_graph).transform(y_graph))
        else:
            id_x = array_to_strings(x.astype('int').reshape(1,-1))[0][0]
            id_y = array_to_strings(y.astype('int').reshape(1,-1))[0][0]
            return self._kernel_table[self._index_table[id_x], self._index_table[id_y]]
```

```python
## Samplers
class linear_gaussian_sampler(model_sampler):
    def __init__(self, **kwargs):
        self._N = 1
        self._num_nodes = 3
        self._epsilon = 1
        super().__init__(**kwargs)

        self._D = self._num_nodes

        # Check inputs
        for attr in ['_N', '_num_nodes', '_epsilon']:
            assert hasattr(self, attr)

        assert type(self._N).__name__ == 'int'
        assert type(self._num_nodes).__name__ == 'int'
        assert self._epsilon > 0
#         self._MH = []
        pass

    @property
    def sample_dim(self):
        return self._N*self._num_nodes + self._num_nodes**2

    @property
    def theta_indices(self):
        return onp.arange(self._N*self._num_nodes, self.sample_dim)

    def drawPrior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())
        self._G = sample_uniform_DAG(self._num_nodes, rng)
        return self._G.reshape(1, self._num_nodes**2).flatten()

    def drawLikelihood(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())
        self._X = sample_DAG_data(self._G, self._N, self._epsilon, rng)
        return self._X.reshape(1,self._N*self._num_nodes).flatten()

    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))

#         self._MH.append(MH)

        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    # Input: a full sample, e.g., from the `sample_mc` method
    def test_functions(self, samples):
        assert samples.shape[1] == self._num_nodes*self._N + self._num_nodes**2
        n = samples.shape[0]

        # Well-defined second moments of adj matrix entries
        num_second_moments = int((self._num_nodes**2-self._num_nodes)*(self._num_nodes**2-self._num_nodes+1)/2 - (self._num_nodes**2-self._num_nodes)/2)
        f1_adj = samples[:, (self._num_nodes*self._N):]
        f2_adj = onp.empty([n, num_second_moments])
        counter = 0
        for i in range(self._num_nodes**2):
            for j in range(i+1):
                row_i, col_i = i//self._num_nodes, i % self._num_nodes
                row_j, col_j = j//self._num_nodes, j % self._num_nodes
                if row_i == col_i or row_j == col_j: # exclude diagonal entries
                    continue
                if row_i == col_j and row_j == col_i: # exclude transpose pairs
                    continue
                f2_adj[:, counter] = f1_adj[:, i] * f1_adj[:, j]
                counter += 1

        # First moments of adj_matrix entries, excluding diagonal entries
        f1_adj = f1_adj[:, onp.setdiff1d(onp.arange(self._num_nodes**2), onp.arange(0, self._num_nodes**2, self._num_nodes+1))]
        return onp.hstack([f1_adj, f2_adj])

# Error 1: count all graphs (rather than DAGs) reached by adding/removing/reversing edges
class linear_gaussian_sampler_error_1(linear_gaussian_sampler):
    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = self.neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = self.neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))
#         self._MH.append(MH)
        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    # Error
    def neighbors_DAG(self, adj_matrix, return_sample=False, rng=None):
        if rng is None:
            rng = onp.random.default_rng()

        nodes = onp.arange(adj_matrix.shape[0])
        edges = [(i, j) for i, j in itertools.product(nodes, nodes)]
        edges += [(-1, -1)] # no change

        k = 0 # count neighbors
        lst_proposals = []
        for i, j in edges:

            # No change
            if i == -1 and j == -1:
                k += 1
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            if i == j:
                k += 1
                continue

            # Add DAG reached by toggling edge i,j
            k += 1
            proposal = changeEdge_DAG(adj_matrix, i, j, change_type='toggle')
            if not isCyclic(proposal):
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Add DAG reached by reversing edge i,j
            if adj_matrix[i, j] == 1:
                k += 1
                proposal = changeEdge_DAG(adj_matrix, i, j, change_type='reverse')
                if not isCyclic(proposal):
                    lst_proposals.append({'i':i, 'j':j, 'change_type':'reverse'})

        if return_sample == True:
            args = rng.choice(lst_proposals)
            proposal = changeEdge_DAG(adj_matrix, **args)
            return k, proposal
        else:
            return k

# Error 2: double-count the number of DAGs reached by reversing edges
class linear_gaussian_sampler_error_2(linear_gaussian_sampler):
    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = self.neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = self.neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))
#         self._MH.append(MH)
        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    # Error
    def neighbors_DAG(self, adj_matrix, return_sample=False, rng=None):
        if rng is None:
            rng = onp.random.default_rng()

        nodes = onp.arange(adj_matrix.shape[0])
        edges = [(i, j) for i, j in itertools.product(nodes, nodes)]
        edges += [(-1, -1)] # no change

        lst_proposals = []
        for i, j in edges:

            # No change
            if i == -1 and j == -1:
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Skip diagonals
            if i == j:
                continue

            # Add DAG reached by toggling edge i,j
            proposal = changeEdge_DAG(adj_matrix, i, j, change_type='toggle')
            if not isCyclic(proposal):
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Add DAG reached by reversing edge i,j
            rev_count = 0
            if adj_matrix[i, j] == 1:
                proposal = changeEdge_DAG(adj_matrix, i, j, change_type='reverse')
                if not isCyclic(proposal):
                    lst_proposals.append({'i':i, 'j':j, 'change_type':'reverse'})
                    rev_count += 1

        k = len(lst_proposals) + rev_count
        if return_sample == True:
            args = rng.choice(lst_proposals)
            proposal = changeEdge_DAG(adj_matrix, **args)
            return k, proposal
        else:
            return k

# Error 3: MH ratio is a fraction of what it should be
class linear_gaussian_sampler_error_3(linear_gaussian_sampler):
    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = neighbors_DAG(proposal)
        MH = 0.5 * num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))
#         self._MH.append(MH)
        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()
```

# Setup

We are interested in learning the structure of a DAG given some data. Let each observation on a root node $x_{r}$ be drawn from a normal distribution with standard deviation $\epsilon$, and let each child node be drawn from a normal distribution centered on the sum of its parents, also with standard deviation $\epsilon$. In other words, given graph structure $\mathcal{G}$ and data $\mathbf{X}$
\begin{equation}
    x_{r} \sim \mathcal{N}(0, \epsilon^2)
\end{equation}
\begin{equation}
    x_{j}|\mathbf{pa}(x_{j}) \sim \mathcal{N}(\sum_{z \in \mathbf{pa}(x_{j})} z, \epsilon^2)
\end{equation}

Placing a uniform prior on $\mathcal{G}$, the posterior is proportional to the likelihood
\begin{equation}
    p(\mathcal{G}|\mathbf{X}) \propto p(\mathbf{X}|\mathcal{G}) = \prod_{i=1}^{n} \prod_{j=1}^{p} p(x_{ij}|\mathbf{pa}(x_{ij})) = \prod_{i=1}^{n} \prod_{j=1}^{p}
    \mathcal{N}(\sum_{z_{i} \in \mathbf{pa}(x_{ij})} z_{i}, \epsilon^2)
\end{equation}

The sampling algorithm we will consider is a modified version of the MCMC scheme from Madigan and York 1995 (the algorithm is detailed on p.270 of https://link.springer.com/content/pdf/10.1007/s10994-008-5057-7.pdf ). Given a graph structure $\mathcal{G}_t$, the proposal structure is sampled uniformly from the neighborhood of $\mathcal{G}_t$
\begin{equation}
P(\mathcal{G}' | \mathcal{G}_t) = \frac{1}{|\mathbf{Ne}(\mathcal{G}_t)|}
\end{equation}
where the neighborhood $\mathbf{Ne}(\mathcal{G}_t)$ is defined as the union of $\mathcal{G}_t$ and the set of all DAGs that can be reached by adding, deleting, or reversing an edge.

The Metropolis-Hastings acceptance probability is thus
\begin{equation}
\min{(1,\frac{p(\mathbf{X}|\mathcal{G}')|\mathbf{Ne}(\mathcal{G}_t)|}{p(\mathbf{X}|\mathcal{G}_t)|\mathbf{Ne}(\mathcal{G}')|})}
\end{equation}

The sampler errors considered all affect the Metropolis-Hastings acceptance probability; specifically, they alter the calculation of $|\mathbf{Ne}(\mathcal{G})|$. In the first error, we count all \textit{graphs} that can be reached by modifying a single edge, regardless of whether the modification induces a cycle. In the second error, we double-count the DAGs that can be reached by reversing an edge.

For the MMD tests, we use the random walk kernel implementation in GraKel (https://ysig.github.io/GraKeL/latest/generated/grakel.RandomWalk.html)

# Experiment

Feature engineering: edges, AND/XOR on edge (pairs), likelihood

```python
def exp_label(x):
    if 'sampler_error_1' in x:
        return 'Type II - Cyclic Check'
    elif 'sampler_error_2' in x:
        return 'Type II - Rev Count'
    else:
        return 'Type I'
```

```python
## Samplers
class linear_gaussian_sampler(model_sampler):
    def __init__(self, **kwargs):
        self._N = 1
        self._num_nodes = 3
        self._epsilon = 1
        super().__init__(**kwargs)

        self._D = self._num_nodes

        # Check inputs
        for attr in ['_N', '_num_nodes', '_epsilon']:
            assert hasattr(self, attr)

        assert type(self._N).__name__ == 'int'
        assert type(self._num_nodes).__name__ == 'int'
        assert self._epsilon > 0
#         self._MH = []
        pass

    @property
    def sample_dim(self):
        return self._N*self._num_nodes + self._num_nodes**2

    @property
    def theta_indices(self):
        return onp.arange(self._N*self._num_nodes, self.sample_dim)

    def drawPrior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())
        self._G = sample_uniform_DAG(self._num_nodes, rng)
        return self._G.reshape(1, self._num_nodes**2).flatten()

    def drawLikelihood(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())
        self._X = sample_DAG_data(self._G, self._N, self._epsilon, rng)
        return self._X.reshape(1,self._N*self._num_nodes).flatten()

    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))

#         self._MH.append(MH)

        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    def test_functions(self, samples):
        assert samples.shape[1] == self._num_nodes*self._N + self._num_nodes**2
        n = samples.shape[0]

        # Identity, pair AND and pair XOR features

        # p is the number of off-diagonal entries in adj matrix
        p = self._num_nodes**2-self._num_nodes

        # For any of the p edges, there are p-1 edges other edges to AND with, minus p/2 transpose pairs -> (p-1)*p/2 - p/2
        num_and = int((p-2)*p/2)
        # For any of the p edges, there are p-1 edges other edges to XOR with -> (p-1)*p/2
        num_xor = int((p-1)*p/2)

        id_adj = samples[:, (self._num_nodes*self._N):]
        and_adj = onp.empty([n, num_and])
        xor_adj = onp.empty([n, num_xor])

        counter_and = 0
        counter_xor = 0
        for i in range(self._num_nodes**2):
            for j in range(i+1, self._num_nodes**2):
                row_i, col_i = i//self._num_nodes, i % self._num_nodes
                row_j, col_j = j//self._num_nodes, j % self._num_nodes
                if row_i == col_i or row_j == col_j: # exclude all pairs with diagonal entries
                    continue
                if row_i == col_j and row_j == col_i: # exclude transpose pairs for AND
                    xor_adj[:, counter_xor] = (id_adj[:, i] + id_adj[:, j]) == 1
                    counter_xor += 1
                    continue

                and_adj[:, counter_and] = id_adj[:, i] * id_adj[:, j]
                counter_and += 1

                xor_adj[:, counter_xor] = (id_adj[:, i] + id_adj[:, j]) == 1
                counter_xor += 1

        # First moments of adj_matrix entries, excluding diagonal entries
        id_adj = id_adj[:, onp.setdiff1d(onp.arange(self._num_nodes**2), onp.arange(0, self._num_nodes**2, self._num_nodes+1))]

        # Likelihood
        l=onp.exp(onp.array([log_likelihood(X=samples[i, :(self._num_nodes*self._N)].reshape(self._N, self._num_nodes),
              adj_matrix=samples[i, (self._num_nodes*self._N):].reshape(self._num_nodes, self._num_nodes).astype('int'),
              epsilon=self._epsilon) for i in range(samples.shape[0])]).reshape(n, 1))

        out = onp.hstack([id_adj, and_adj, xor_adj, l])
        assert onp.isnan(out).sum() == 0
        return out

# Error 1: count all graphs (rather than DAGs) reached by adding/removing/reversing edges
class linear_gaussian_sampler_error_1(linear_gaussian_sampler):
    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = self.neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = self.neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))
#         self._MH.append(MH)
        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    # Error
    def neighbors_DAG(self, adj_matrix, return_sample=False, rng=None):
        if rng is None:
            rng = onp.random.default_rng()

        nodes = onp.arange(adj_matrix.shape[0])
        edges = [(i, j) for i, j in itertools.product(nodes, nodes)]
        edges += [(-1, -1)] # no change

        k = 0 # count neighbors
        lst_proposals = []
        for i, j in edges:

            # No change
            if i == -1 and j == -1:
                k += 1
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            if i == j:
                k += 1
                continue

            # Add DAG reached by toggling edge i,j
            k += 1
            proposal = changeEdge_DAG(adj_matrix, i, j, change_type='toggle')
            if not isCyclic(proposal):
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Add DAG reached by reversing edge i,j
            if adj_matrix[i, j] == 1:
                k += 1
                proposal = changeEdge_DAG(adj_matrix, i, j, change_type='reverse')
                if not isCyclic(proposal):
                    lst_proposals.append({'i':i, 'j':j, 'change_type':'reverse'})

        if return_sample == True:
            args = rng.choice(lst_proposals)
            proposal = changeEdge_DAG(adj_matrix, **args)
            return k, proposal
        else:
            return k

# Error 2: double-count the number of DAGs reached by reversing edges
class linear_gaussian_sampler_error_2(linear_gaussian_sampler):
    def drawPosterior(self, rng=None):
        if rng is None:
            rng = onp.random.Generator(onp.random.MT19937())

        num_neighbors, proposal = self.neighbors_DAG(self._G, True, rng)
        num_neighbors_proposal = self.neighbors_DAG(proposal)
        MH = num_neighbors/num_neighbors_proposal * onp.exp(log_likelihood(self._X, proposal, self._epsilon) - log_likelihood(self._X, self._G, self._epsilon))
#         self._MH.append(MH)
        if rng.uniform() <= MH:
            self._G = proposal
        return self._G.reshape(1, self._num_nodes**2).flatten()

    # Error
    def neighbors_DAG(self, adj_matrix, return_sample=False, rng=None):
        if rng is None:
            rng = onp.random.default_rng()

        nodes = onp.arange(adj_matrix.shape[0])
        edges = [(i, j) for i, j in itertools.product(nodes, nodes)]
        edges += [(-1, -1)] # no change

        lst_proposals = []
        for i, j in edges:

            # No change
            if i == -1 and j == -1:
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Skip diagonals
            if i == j:
                continue

            # Add DAG reached by toggling edge i,j
            proposal = changeEdge_DAG(adj_matrix, i, j, change_type='toggle')
            if not isCyclic(proposal):
                lst_proposals.append({'i':i, 'j':j, 'change_type':'toggle'})

            # Add DAG reached by reversing edge i,j
            rev_count = 0
            if adj_matrix[i, j] == 1:
                proposal = changeEdge_DAG(adj_matrix, i, j, change_type='reverse')
                if not isCyclic(proposal):
                    lst_proposals.append({'i':i, 'j':j, 'change_type':'reverse'})
                    rev_count += 1

        k = len(lst_proposals) + rev_count
        if return_sample == True:
            args = rng.choice(lst_proposals)
            proposal = changeEdge_DAG(adj_matrix, **args)
            return k, proposal
        else:
            return k
```

```python
# Random tiebreak
def rank_stat(model, L, test_functions=None, rng=None):
    if test_functions is None:
        test_functions = model.test_functions
    if rng is None:
        rng = onp.random.default_rng()

    M = rng.choice(L)

    chain = onp.zeros(shape=(L, len(model.theta_indices)))
    chain[M, :] = model.drawPrior()

    y = model.drawLikelihood()

    stateDict = model.__dict__.copy()

    # Backward
    for i in range(M-1, -1, -1):
        chain[i, :] = model.drawPosterior()

    # Forward
    model.__dict__ = stateDict.copy()
    for j in range(M+1, L):
        chain[j, :] = model.drawPosterior()

    # Apply test functions
    chain = onp.hstack([onp.repeat(y.reshape(1, model._N*model._D), repeats=L, axis=0), chain])
    chain = test_functions(chain)

    # Random tiebreak for binary data
    e = onp.hstack([onp.ones(shape=(L, model._N*model._D)), rng.uniform(size=(L, chain.shape[1]-model._N*model._D))])
    chain += e

    return rankdata(chain, 'ordinal', axis = 0)[M, :]
```

```python
# Comparison
def run_trial(model, num_samples, bc_burn_in, sc_thinning, alpha, seed):
    num_samples = int(num_samples)
    bc_burn_in = int(bc_burn_in)
    sc_thinning = int(sc_thinning)

    num_samples_bc = int(1/(bc_burn_in+2) * num_samples)
    num_samples_sc = int((sc_thinning)/(sc_thinning+1) * num_samples)

    num_samples_mc_bc = num_samples_bc
    num_samples_mc_sc = num_samples-num_samples_sc
    num_samples_mc = max(num_samples_mc_bc, num_samples_mc_sc)

    model.set_seed(seed)
    samples_mc = model.test_functions(model.sample_mc(num_samples_mc))
    samples_sc = model.test_functions(model.sample_sc(num_samples_sc))[onp.arange(0, num_samples_sc, sc_thinning), :]
    samples_bc = model.test_functions(model.sample_bc(num_samples_bc, bc_burn_in))

    rej_rank = rank_test(model, N=int(num_samples/(bc_burn_in+1)), L=int(bc_burn_in+1))['result'].max()

    rej_geweke = geweke_test(samples_mc[:(num_samples-num_samples_sc), :], samples_sc, l=0.08, alpha=alpha)['result'].max()

    # Normalize for MMD
    X_mmd_wb, Y_mmd_wb = normalizeTwoSamples(samples_mc[:(num_samples-num_samples_sc), :], samples_sc)
    X_mmd, Y_mmd = normalizeTwoSamples(samples_mc[:num_samples_bc, :], samples_bc)

    rej_mmd = mmd_test(X_mmd, Y_mmd, rbf_kernel, alpha=alpha, mmd_type='unbiased')['result']
    rej_mmd_wb = mmd_wb_test(X_mmd_wb, Y_mmd_wb, rbf_kernel, alpha=alpha)['result']

    rej_mmd_linear = mmd_test(X_mmd, Y_mmd, linear_kernel, alpha=alpha, mmd_type='unbiased')['result']
    rej_mmd_wb_linear = mmd_wb_test(X_mmd_wb, Y_mmd_wb, linear_kernel, alpha=alpha)['result']

    # For ChiSq test, use the remaining sample budget to generate more BC samples
    samples_bc_chisq = model.test_functions(model.sample_bc(int(num_samples_mc_bc/bc_burn_in), bc_burn_in))
    samples_bc_chisq = onp.vstack([samples_bc_chisq, samples_bc])
    vals_sc, counts_bc = count_sample_DAG(samples_bc_chisq[:, onp.arange(len(model.theta_indices))])
    total_count = count_DAG(model._num_nodes)[model._num_nodes-1, :].sum()
    counts_bc = onp.hstack([counts_bc, onp.zeros(total_count - len(counts_bc))])
    rej_chisq = scipy.stats.chisquare(counts_bc).pvalue <= alpha

    return onp.array([rej_rank, rej_geweke, rej_mmd_wb, rej_mmd, rej_mmd_wb_linear, rej_mmd_linear, rej_chisq])
```

```python
os.environ['MKL_NUM_THREAD'] = '1'

num_nodes = 3
num_trials = 100
alpha = 0.05

bc_burn_in = 5-1
sc_thinning = 5

lst_experiments = [linear_gaussian_sampler(num_nodes=num_nodes), linear_gaussian_sampler_error_1(num_nodes=num_nodes), linear_gaussian_sampler_error_2(num_nodes=num_nodes)]
lst_sample_budget = [3000,6000,9000,12000]

# lst_res = []

for model in tqdm(lst_experiments):
    for n in tqdm(lst_sample_budget):
        num_samples_bc = int(1/(bc_burn_in+2) * n)
        num_samples_sc = int((sc_thinning)/(sc_thinning+1) * n)
        test_size_bc = num_samples_bc
        test_size_sc = n-num_samples_sc

        results = joblib.Parallel(n_jobs=5)(joblib.delayed(run_trial)(model, n, bc_burn_in, sc_thinning, alpha, seed) for seed in range(num_trials))
        results = onp.vstack(results).mean(0)

       # Convert rejection rate to Type I/II error
        if 'error' in type(model).__name__:
            results = 1-results

        lst_test = ['rank', 'geweke', 'mmd_wb', 'mmd', 'mmd_wb_linear', 'mmd_linear', 'chisq']
        lst_test_size = [test_size_bc, test_size_sc, test_size_sc, test_size_bc, test_size_sc, test_size_bc, test_size_bc]
        lst_res.append(pd.DataFrame({'experiment': type(model).__name__, 'test':lst_test,  'test_size':lst_test_size, 'sample_budget':n, 'result':results}))
```

```python
df_results = pd.concat(lst_res)
# df_results=pd.read_pickle('graph_data')

df_hline = pd.DataFrame({'experiment':['linear_gaussian_sampler'], 'alpha':[alpha]})
mask = ~df_results.test.str.contains('linear|graph')
plot_results=ggplot(df_results.loc[mask], aes(x='test_size', y='result', color='test', group='test')) + \
    geom_point() + geom_line() + geom_hline(df_hline, aes(yintercept='alpha'), linetype='dotted') +  \
    facet_grid('~experiment', labeller=exp_label) +\
    labs(x='Test Sample Size', y='Error Rate', title='', color='Test') + \
    scale_color_discrete(labels=[r'$\chi^2$', 'Geweke', 'MMD-BC', 'MMD-SC', 'Rank'], palette=palette) + theme_bw() +\
    theme(axis_text_x = element_text(size=6.5))
plot_results
```

```python
ggsave(plot_results, 'graph_comparison')
```

    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:727: PlotnineWarning: Saving 6.4 x 4.8 in image.
    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:730: PlotnineWarning: Filename: graph_comparison

```python
df_plot_kernel = df_results.loc[(df_results.test.str.contains('mmd')) & ~(df_results.test.str.contains('graph'))].reset_index()
df_plot_kernel['kernel'] = df_plot_kernel.apply(lambda x: 'linear' if 'linear' in x.test else 'gaussian', axis=1)
df_plot_kernel['test'] = df_plot_kernel.apply(lambda x: x.test.replace('_linear', ''), axis=1)

plot_results=ggplot(df_plot_kernel, aes(x='test_size', y='result', linetype='kernel', color='test')) + \
    geom_point() + geom_line() + geom_hline(df_hline, aes(yintercept='alpha'), linetype='dotted') +  \
    facet_grid('~experiment', labeller=exp_label) + \
    labs(x='Test Sample Size', y='Error Rate', title='', color='Test', linetype='Kernel') + \
    scale_color_discrete(labels=['MMD-BC', 'MMD-SC'], palette=palette) + \
    scale_linetype_discrete(labels=['Gaussian', 'Linear']) + \
    theme_bw() +\
    theme(axis_text_x = element_text(size=6.5))
plot_results
```

```python
ggsave(plot_results, 'graph_kernel')
```

    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:727: PlotnineWarning: Saving 6.4 x 4.8 in image.
    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:730: PlotnineWarning: Filename: graph_kernel

Using the random walk kernel

```python
def run_trial(model, num_samples, bc_burn_in, sc_thinning, kernel_type, kernel_table, index_table, alpha, seed):
    num_samples = int(num_samples)
    bc_burn_in = int(bc_burn_in)
    sc_thinning = int(sc_thinning)

    num_samples_bc = int(1/(bc_burn_in+2) * num_samples)
    num_samples_sc = int((sc_thinning)/(sc_thinning+1) * num_samples)

    num_samples_mc_bc = num_samples_bc
    num_samples_mc_sc = num_samples-num_samples_sc
    num_samples_mc = max(num_samples_mc_bc, num_samples_mc_sc)

    model.set_seed(seed)
    samples_mc = model.sample_mc(num_samples_mc)
    samples_sc = model.sample_sc(num_samples_sc)[onp.arange(0, num_samples_sc, sc_thinning), :]
    samples_bc = model.sample_bc(num_samples_bc, bc_burn_in)

    l_mc = model.test_functions(samples_mc)[:, -1].reshape(-1, 1)
    l_sc = model.test_functions(samples_sc)[:, -1].reshape(-1, 1)
    l_bc = model.test_functions(samples_bc)[:, -1].reshape(-1, 1)

    X_mmd_wb = onp.hstack([samples_mc[:, model.theta_indices], l_mc])[:(num_samples-num_samples_sc), :]
    Y_mmd_wb = onp.hstack([samples_sc[:, model.theta_indices], l_sc])

    X_mmd = onp.hstack([samples_mc[:, model.theta_indices], l_mc])[:num_samples_bc, :]
    Y_mmd = onp.hstack([samples_bc[:, model.theta_indices], l_bc])

    # Not including likelihood
    rej_mmd_wb = mmd_wb_test(X_mmd_wb[:, :-1], Y_mmd_wb[:, :-1], \
                             graph_kernel, alpha=alpha, \
                             cache_index_table = index_table, cache_kernel_table=kernel_table, kernel_type=kernel_type)['result']
    rej_mmd = mmd_test(X_mmd[:, :-1], Y_mmd[:, :-1], \
                       graph_kernel, alpha=alpha, mmd_type='unbiased', \
                       cache_index_table = index_table, cache_kernel_table=kernel_table, kernel_type=kernel_type)['result']

#     rej_mmd_wb_l = mmd_wb_test(X_mmd_wb, Y_mmd_wb, \
#                        sum_kernel, alpha=alpha, mmd_type='unbiased', \
#                        lst_classes = [graph_kernel, rbf_kernel], lst_groups=[onp.arange(len(model.theta_indices)), onp.array([-1])], \
#                        lst_kwargs=[{'cache_index_table':index_table, 'cache_kernel_table':kernel_table, 'kernel_type':kernel_type}, {}])['result']

#     rej_mmd_l = mmd_test(X_mmd, Y_mmd, \
#                        sum_kernel, alpha=alpha, mmd_type='unbiased', \
#                        lst_classes = [graph_kernel, rbf_kernel], lst_groups=[onp.arange(len(model.theta_indices)), onp.array([-1])], \
#                        lst_kwargs=[{'cache_index_table':index_table, 'cache_kernel_table':kernel_table, 'kernel_type':kernel_type}, {}])['result']

#     return onp.array([rej_mmd_wb, rej_mmd_wb_l, rej_mmd, rej_mmd_l])
    return onp.array([rej_mmd_wb, rej_mmd])
```

```python
os.environ['MKL_NUM_THREAD'] = '1'

num_nodes = 3
num_trials = 100
alpha = 0.05

bc_burn_in = 5-1
sc_thinning = 5

kernel_type='random_walk'

# Cache
kernel_table, index_table = graph_kernel_cache(num_nodes, kernel_type)

lst_experiments = [linear_gaussian_sampler(num_nodes=num_nodes), linear_gaussian_sampler_error_1(num_nodes=num_nodes), linear_gaussian_sampler_error_2(num_nodes=num_nodes)]
lst_sample_budget = [3000,6000,9000,12000]

lst_res = []

with joblib.Parallel(n_jobs=5) as para:
    for model in tqdm(lst_experiments):
        for n in tqdm(lst_sample_budget):
            num_samples_bc = int(1/(bc_burn_in+2) * n)
            num_samples_sc = int((sc_thinning)/(sc_thinning+1) * n)
            test_size_bc = num_samples_bc
            test_size_sc = n-num_samples_sc

            results = para(joblib.delayed(run_trial)(model, n, bc_burn_in, sc_thinning, kernel_type, kernel_table, index_table, alpha, seed) for seed in range(num_trials))
            results = onp.vstack(results).mean(0)

           # Convert rejection rate to Type I/II error
            if 'error' in type(model).__name__:
                results = 1-results

            lst_test = ['mmd_wb', 'mmd']
            lst_test_size = [test_size_sc, test_size_bc]
            lst_res.append(pd.DataFrame({'experiment': type(model).__name__, 'test':lst_test,  'test_size':lst_test_size, 'sample_budget':n, 'result':results}))
```

```python
df_graph=pd.concat(lst_res)
df_graph.test = df_graph.test + '_graph'
df_results = pd.read_pickle('graph_data')
df_results=pd.concat([df_results, df_graph])
```

```python
df_plot_kernel = df_results.loc[df_results.test.str.contains('mmd')].reset_index()
df_plot_kernel.loc[df_plot_kernel.test.str.contains('graph'), 'kernel'] = 'graph'
df_plot_kernel.loc[df_plot_kernel.test.str.contains('linear'), 'kernel'] = 'linear'
df_plot_kernel.loc[df_plot_kernel.kernel.isnull(), 'kernel'] = 'gaussian'
df_plot_kernel['test'] = df_plot_kernel.apply(lambda x: x.test.replace('_linear', ''), axis=1)
df_plot_kernel['test'] = df_plot_kernel.apply(lambda x: x.test.replace('_graph', ''), axis=1)
```

```python
df_hline = pd.DataFrame({'experiment':['linear_gaussian_sampler'], 'alpha':[alpha]})
mask = df_results.test.str.contains('graph|chi')
plot_results=ggplot(df_results.loc[mask], aes(x='test_size', y='result', color='test', group='test')) + \
    geom_point() + geom_line() + geom_hline(df_hline, aes(yintercept='alpha'), linetype='dotted') +  \
    facet_grid('~experiment', labeller=exp_label) +\
    labs(x='Test Sample Size', y='Error Rate', title='', color='Test') + \
    scale_color_discrete(labels=[r'$\chi^2$', 'MMD-BC, RW Kernel', 'MMD-SC, RW Kernel'], palette=palette) + theme_bw() +\
    theme(axis_text_x = element_text(size=6.5))
plot_results
```

```python
ggsave(plot_results, 'graph_random_walk')
```

    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:727: PlotnineWarning: Saving 6.4 x 4.8 in image.
    /Users/andrew/miniconda3/envs/msc/lib/python3.8/site-packages/plotnine/ggplot.py:730: PlotnineWarning: Filename: graph_random_walk

Plotting a DAG with igraph

```python
adj_matrix = [[0,1,0,0,0],[0,0,1,1,0],[0,0,0,0,1],[0,0,0,0,1],[0,0,0,0,0]]
g=igraph.Graph.Adjacency(adj_matrix)
g.vs['label'] = list(range(5))
igraph.plot(g)
```

```python

```

```python

```