## Set up the environment

```
!nvidia-smi
```

    Tue Jun 15 01:26:38 2021
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 465.27       Driver Version: 460.32.03    CUDA Version: 11.2     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
    | N/A   46C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+

    +-----------------------------------------------------------------------------+
    | Processes:                                                                  |
    |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
    |        ID   ID                                                   Usage      |
    |=============================================================================|
    |  No running processes found                                                 |
    +-----------------------------------------------------------------------------+

```
#!pip install pymatgen==2020.12.31
!pip install pymatgen==2019.11.11
!pip install --pre graphdot
!pip install gdown
```

    Requirement already satisfied: pymatgen==2019.11.11 in /usr/local/lib/python3.7/dist-packages (2019.11.11)
    Requirement already satisfied: spglib>=1.9.9.44 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (1.16.1)
    Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (0.8.9)
    Requirement already satisfied: ruamel.yaml>=0.15.6 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (0.17.9)
    Requirement already satisfied: scipy>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (1.4.1)
    Requirement already satisfied: monty>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (2021.6.10)
    Requirement already satisfied: palettable>=3.1.1 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (3.3.0)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (2.23.0)
    Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (2.5.1)
    Requirement already satisfied: matplotlib>=1.5 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (3.2.2)
    Requirement already satisfied: numpy>=1.14.3 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (1.19.5)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (1.1.5)
    Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (1.7.1)
    Requirement already satisfied: pydispatcher>=2.0.5 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11) (2.0.5)
    Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.10" in /usr/local/lib/python3.7/dist-packages (from ruamel.yaml>=0.15.6->pymatgen==2019.11.11) (0.2.2)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11) (2021.5.30)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11) (3.0.4)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11) (2.10)
    Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.7/dist-packages (from networkx>=2.2->pymatgen==2019.11.11) (4.4.2)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11) (1.3.1)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11) (2.8.1)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->pymatgen==2019.11.11) (2018.9)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->pymatgen==2019.11.11) (1.2.1)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from cycler>=0.10->matplotlib>=1.5->pymatgen==2019.11.11) (1.15.0)
    Requirement already satisfied: graphdot in /usr/local/lib/python3.7/dist-packages (0.8a17)
    Requirement already satisfied: pandas>=0.24 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.1.5)
    Requirement already satisfied: mendeleev>=0.7 in /usr/local/lib/python3.7/dist-packages (from graphdot) (0.7.0)
    Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.19.5)
    Requirement already satisfied: kahypar>=1.1.4 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.1.6)
    Requirement already satisfied: treelib>=1.6.1 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.6.1)
    Requirement already satisfied: scipy>=1.3.0 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.4.1)
    Requirement already satisfied: pycuda>=2019 in /usr/local/lib/python3.7/dist-packages (from graphdot) (2021.1)
    Requirement already satisfied: networkx>=2.4 in /usr/local/lib/python3.7/dist-packages (from graphdot) (2.5.1)
    Requirement already satisfied: tqdm>=4.55 in /usr/local/lib/python3.7/dist-packages (from graphdot) (4.61.1)
    Requirement already satisfied: pymatgen==2019.11.11 in /usr/local/lib/python3.7/dist-packages (from graphdot) (2019.11.11)
    Requirement already satisfied: numba>=0.51.0 in /usr/local/lib/python3.7/dist-packages (from graphdot) (0.51.2)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from graphdot) (1.7.1)
    Requirement already satisfied: ase>=3.17 in /usr/local/lib/python3.7/dist-packages (from graphdot) (3.21.1)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.24->graphdot) (2.8.1)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.24->graphdot) (2018.9)
    Requirement already satisfied: Pygments<3.0.0,>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from mendeleev>=0.7->graphdot) (2.9.0)
    Requirement already satisfied: SQLAlchemy<2.0.0,>=1.3.23 in /usr/local/lib/python3.7/dist-packages (from mendeleev>=0.7->graphdot) (1.4.18)
    Requirement already satisfied: six<2.0.0,>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from mendeleev>=0.7->graphdot) (1.15.0)
    Requirement already satisfied: colorama<0.5.0,>=0.4.4 in /usr/local/lib/python3.7/dist-packages (from mendeleev>=0.7->graphdot) (0.4.4)
    Requirement already satisfied: pyfiglet<0.9,>=0.8.post1 in /usr/local/lib/python3.7/dist-packages (from mendeleev>=0.7->graphdot) (0.8.post1)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from treelib>=1.6.1->graphdot) (0.16.0)
    Requirement already satisfied: pytools>=2011.2 in /usr/local/lib/python3.7/dist-packages (from pycuda>=2019->graphdot) (2021.2.7)
    Requirement already satisfied: appdirs>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from pycuda>=2019->graphdot) (1.4.4)
    Requirement already satisfied: mako in /usr/local/lib/python3.7/dist-packages (from pycuda>=2019->graphdot) (1.1.4)
    Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.7/dist-packages (from networkx>=2.4->graphdot) (4.4.2)
    Requirement already satisfied: monty>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (2021.6.10)
    Requirement already satisfied: ruamel.yaml>=0.15.6 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (0.17.9)
    Requirement already satisfied: pydispatcher>=2.0.5 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (2.0.5)
    Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (0.8.9)
    Requirement already satisfied: matplotlib>=1.5 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (3.2.2)
    Requirement already satisfied: palettable>=3.1.1 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (3.3.0)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (2.23.0)
    Requirement already satisfied: spglib>=1.9.9.44 in /usr/local/lib/python3.7/dist-packages (from pymatgen==2019.11.11->graphdot) (1.16.1)
    Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba>=0.51.0->graphdot) (0.34.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba>=0.51.0->graphdot) (57.0.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->graphdot) (1.2.1)
    Requirement already satisfied: greenlet!=0.4.17; python_version >= "3" in /usr/local/lib/python3.7/dist-packages (from SQLAlchemy<2.0.0,>=1.3.23->mendeleev>=0.7->graphdot) (1.1.0)
    Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from SQLAlchemy<2.0.0,>=1.3.23->mendeleev>=0.7->graphdot) (4.5.0)
    Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.7/dist-packages (from mako->pycuda>=2019->graphdot) (2.0.1)
    Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.10" in /usr/local/lib/python3.7/dist-packages (from ruamel.yaml>=0.15.6->pymatgen==2019.11.11->graphdot) (0.2.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11->graphdot) (1.3.1)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11->graphdot) (0.10.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=1.5->pymatgen==2019.11.11->graphdot) (2.4.7)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11->graphdot) (1.24.3)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11->graphdot) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11->graphdot) (2021.5.30)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->pymatgen==2019.11.11->graphdot) (2.10)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < "3.8"->SQLAlchemy<2.0.0,>=1.3.23->mendeleev>=0.7->graphdot) (3.4.1)
    Requirement already satisfied: typing-extensions>=3.6.4; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < "3.8"->SQLAlchemy<2.0.0,>=1.3.23->mendeleev>=0.7->graphdot) (3.7.4.3)
    Requirement already satisfied: gdown in /usr/local/lib/python3.7/dist-packages (3.6.4)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from gdown) (4.61.1)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from gdown) (2.23.0)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from gdown) (1.15.0)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (1.24.3)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2021.5.30)

```
%matplotlib inline
import io
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages/')
import os
import urllib
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

import graphdot
from graphdot import Graph
from graphdot.graph.adjacency import AtomicAdjacency
from graphdot.graph.reorder import rcm
from graphdot.kernel.marginalized import MarginalizedGraphKernel # https://graphdot.readthedocs.io/en/latest/apidoc/graphdot.kernel.marginalized.html
from graphdot.kernel.marginalized.starting_probability import Uniform
from graphdot.model.gaussian_process import (
    GaussianProcessRegressor,
    LowRankApproximateGPR
)
from graphdot.kernel.fix import Normalization
import graphdot.microkernel as uX
import ase.io

# for getting all file names into a list under a directory
from os import listdir
# for getting file names that match certain pattern
import glob
import time
```

```
from google.colab import drive
drive.mount('/content/gdrive', force_remount=True)
```

    Mounted at /content/gdrive

```
#cd gdrive/MyDrive/Google\ Colab/Covid-Data
%cd gdrive/MyDrive/Covid-Data/
```

    /content/gdrive/.shortcut-targets-by-id/1wtzMcocuK8kPsz8K0ktjCZPkv567W6M2/Covid-Data

```
!pwd
```

    /content/gdrive/.shortcut-targets-by-id/1wtzMcocuK8kPsz8K0ktjCZPkv567W6M2/Covid-Data

## load the data

```
files  = ['uncharged_NSP15_6W01_A_3_H.Orderable_zinc_db_enaHLL.2col.csv.1.xz']
dataset = pd.read_pickle(files[0]) # length of each csv file is 100000
```

```
target = 'energy'
N_train = 1000
N_test = 2000

np.random.seed(0) # with the same seed, multiple runs can have the same selected index
train_sel = np.random.choice(len(dataset), N_train, replace=False)
test_sel = np.random.choice(np.setxor1d(np.arange(len(dataset)), train_sel), N_test, replace=False)
train = dataset.iloc[train_sel]
test = dataset.iloc[test_sel]
```

```
train.head()
```

      energy
      smiles
      graphs

      3582
      -2.874773
      CN(Cc1nc(no1)c2cccc(c2)Br)C(=O)Nc3cccc(c3)OC
      Graph(nodes={'!i': [ 0, 1, 2, 3, 4, 5, 6, 7, 8...

      60498
      -3.469760
      CCOC(=O)c1ccc(c(c1)C)NC(=S)NCCOc2ccc(cc2)C
      Graph(nodes={'!i': [ 0, 1, 2, 3, 4, 5, 6, 7, 8...

      53227
      -0.487984
      Cc1ccc(cc1S(=O)(=O)Nc2ccccc2OC)NC(=O)c3nc4nc(c...
      Graph(nodes={'!i': [ 0, 1, 2, 3, 4, 5, 6, 7, 8...

      21333
      -1.808283
      COc1ccc(cc1OC)CCNC(=O)COc2ccc(cc2)[N@@](Cc3ccc...
      Graph(nodes={'!i': [ 0, 1, 2, 3, 4, 5, 6, 7, 8...

      3885
      -2.457642
      Cc1cc(ccc1NCc2ccc(c(c2OC)C)OC)Br
      Graph(nodes={'!i': [ 0, 1, 2, 3, 4, 5, 6, 7, 8...

### Baseline mode

```
gpr_base = GaussianProcessRegressor(
    # kernel is the covariance function of the gaussian process (GP)
    kernel=Normalization( # kernel equals to normalization -> normalizes a kernel using the cosine of angle formula, k_normalized(x,y) = k(x,y)/sqrt(k(x,x)*k(y,y))
        # graphdot.kernel.fix.Normalization(kernel), set kernel as marginalized graph kernel, which is used to calculate the similarity between 2 graphs
        # implement the random walk-based graph similarity kernel as Kashima, H., Tsuda, K., & Inokuchi, A. (2003). Marginalized kernels between labeled graphs. ICML
        MarginalizedGraphKernel(
            # node_kernel - A kernelet that computes the similarity between individual nodes
            # uX - graphdot.microkernel - microkernels are positive-semidefinite functions between individual nodes and edges of graphs
            node_kernel=uX.Additive( # addition of kernal matrices: sum of k_a(X_a, Y_a) cross for a in features
                # uX.Constant - a kernel that returns a constant value, always mutlipled with other microkernels as an adjustable weight
                #   c, the first input arg. as 0.5, (0.01, 10) the lower and upper bounds of c that is allowed to vary during hyperpara. optimizartion
                # uX.KroneckerDelta - a kronecker delta returns 1 when two features are equal and return h (the first input arg here, which is 0.5 in this case) otherwise
                #   (0.1, 0.9) the lower and upper bounds that h is allowed to vary during hyperpara. optimization
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)), # the 2nd element of graphdot.graph.Graph.nodes
                atomic_number=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)), # the 3rd element of graphdot.graph.Graph.nodes
                # uX.SquareExponential - Equ. 26 in the paper
                # input arg. length_sacle is a float32, set as 1 in this case, which correspond to approx. 1 of the kernal value.
                # This is used to determins how quicklys should the kernel decay to zero.
                charge=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0), # the 4th element of graphdot.graph.Graph.nodes
                chiral=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)), # the 5th element of graphdot.graph.Graph.nodes
                hcount=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0), # the 6th element of graphdot.graph.Graph.nodes
                hybridization=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)), # the 7th element of graphdot.graph.Graph.nodes
                # uX.Convolution - a convolutional microkernel which averages evaluations of a base microkernel between pairs pf elememts of two variable-length feature sequences
                # uX.KroneckerDelta as the base kernel
                ring_list=uX.Constant(0.5, (0.01, 100.0)) * uX.Convolution(uX.KroneckerDelta(0.5,(0.1, 0.9))) # the 8th element of graphdot.graph.Graph.nodes
            ).normalized,
            # edge_kernel - A kernelet that computes the similarity between individual edge
            edge_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)), # the 3rd element of graphdot.graph.Graph.nodes
                conjugated=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)), # the 4th element of graphdot.graph.Graph.nodes
                order=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)), # the 5th element of graphdot.graph.Graph.nodes
                ring_stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)), # the 6th element of graphdot.graph.Graph.nodes
                stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)) # the 7th element of graphdot.graph.Graph.nodes
            ).normalized,
            p=Uniform(1.0, p_bounds='fixed'), # the strating probability of the random walk on each node
            q=0.05 # the probability for the random walk to stop during each step
        )
    ),
    alpha=1e-4, # value added to the diagonal of the kernel matrix during fitting
    optimizer=True, # default optimizer of L-BFGS-B based on scipy.optimize.minimize
    normalize_y=True, # normalize the y values so taht the means and variance is 0 and 1, repsectively. Will be reversed when predicions are returned
    regularization='+', # alpha (1e-4 in this case) is added to the diagonals of the kernal matrix
)
```

```
gpr_base.fit(train.graphs, train[target], repeat=3, verbose=True)
```

    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  2.8725e+06|  2.4458e+07|  2.8798e+06|     -7290.9|  8.7024e+06|        22|      0.25|
    |  1.9315e+06|  2.0707e+07|  1.9382e+06|     -6742.8|  8.0237e+06|        20|      0.24|
    |  1.2905e+06|  1.5706e+07|  1.2968e+06|     -6252.2|  7.0835e+06|        19|      0.25|
    |  1.0061e+06|  1.2309e+07|  1.0121e+06|     -5971.8|  6.4171e+06|        18|      0.24|
    |  8.2557e+05|  9.7267e+06|  8.3132e+05|     -5755.4|  5.8209e+06|        17|      0.23|
    |  7.1178e+05|  8.0789e+06|  7.1737e+05|     -5588.8|  5.3655e+06|        17|      0.25|
    |  6.4897e+05|  7.1031e+06|  6.5446e+05|     -5488.6|  5.0424e+06|        17|      0.24|
    |  6.2995e+05|   7.043e+06|  6.3539e+05|     -5435.7|  4.9627e+06|        17|      0.24|
    |  6.1001e+05|  6.6424e+06|  6.1542e+05|     -5413.2|  4.8439e+06|        17|      0.24|
    |   6.055e+05|  6.5416e+06|  6.1091e+05|     -5407.7|  4.7988e+06|        16|      0.24|
    |  6.0285e+05|  6.4988e+06|  6.0825e+05|     -5402.9|  4.7885e+06|        16|      0.24|
    |  6.0601e+05|  6.6651e+06|   6.114e+05|     -5395.9|  4.8317e+06|        17|      0.25|
    |  6.0087e+05|  6.5216e+06|  6.0626e+05|     -5395.1|  4.7916e+06|        17|      0.24|
    |  5.9979e+05|  6.4968e+06|  6.0518e+05|     -5393.3|  4.7859e+06|        17|      0.24|
    |  5.9795e+05|  6.4537e+06|  6.0334e+05|     -5390.4|  4.7776e+06|        17|      0.26|
    |  5.9632e+05|   6.417e+06|  6.0171e+05|     -5388.7|  4.7579e+06|        16|      0.23|
    |  5.9535e+05|  6.4032e+06|  6.0074e+05|     -5387.3|  4.7491e+06|        16|      0.25|
    |  5.9515e+05|  6.4246e+06|  6.0054e+05|     -5386.3|  4.7503e+06|        17|      0.24|
    |  5.9519e+05|  6.4222e+06|  6.0058e+05|     -5386.6|  4.7481e+06|        17|      0.25|
    |  5.9528e+05|  6.4269e+06|  6.0067e+05|     -5386.3|  4.7487e+06|        17|      0.25|
    |  5.9518e+05|  6.4251e+06|  6.0056e+05|     -5386.3|   4.747e+06|        17|      0.24|
    |  5.9522e+05|  6.4263e+06|  6.0061e+05|     -5386.3|   4.748e+06|        17|      0.24|
    |  5.9521e+05|  6.4261e+06|   6.006e+05|     -5386.3|  4.7459e+06|        17|      0.24|
    |  5.9521e+05|  6.4263e+06|   6.006e+05|     -5386.3|  4.7474e+06|        17|      0.25|
    |  5.9516e+05|  6.4253e+06|  6.0055e+05|     -5386.2|  4.7489e+06|        17|      0.25|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  1.7413e+06|  2.1782e+07|   1.748e+06|       -6644|  7.5158e+06|        20|      0.24|
    |  1.3451e+06|  1.6773e+07|  1.3514e+06|     -6332.3|  6.9251e+06|        19|      0.25|
    |   1.033e+06|  1.3306e+07|  1.0391e+06|     -6026.9|  6.3155e+06|        19|      0.24|
    |  7.9866e+05|  1.0452e+07|  8.0437e+05|     -5717.2|  5.7392e+06|        19|      0.26|
    |  6.9752e+05|  8.1836e+06|  7.0309e+05|     -5576.4|   5.318e+06|        17|      0.26|
    |  6.4289e+05|   7.195e+06|  6.4837e+05|     -5481.7|  5.0649e+06|        17|      0.23|
    |   6.153e+05|  6.7608e+06|  6.2073e+05|     -5430.1|  4.8862e+06|        17|      0.24|
    |  6.1229e+05|  6.6746e+06|  6.1772e+05|     -5421.4|  4.8115e+06|        16|      0.24|
    |  5.9982e+05|  6.4991e+06|  6.0522e+05|     -5399.3|  4.7705e+06|        16|      0.23|
    |  5.9655e+05|  6.4385e+06|  6.0194e+05|     -5390.8|  4.7638e+06|        17|      0.23|
    |  5.9622e+05|  6.4295e+06|  6.0161e+05|       -5390|  4.7603e+06|        16|      0.25|
    |  5.9531e+05|  6.4019e+06|   6.007e+05|     -5388.4|  4.7574e+06|        16|      0.26|
    |  5.9498e+05|  6.3961e+06|  6.0037e+05|     -5387.3|  4.7535e+06|        16|      0.25|
    |  5.9726e+05|  6.5117e+06|  6.0264e+05|     -5385.9|  4.7837e+06|        17|      0.24|
    |  5.9475e+05|  6.4136e+06|  6.0013e+05|     -5385.8|  4.7581e+06|        17|      0.24|
    |  5.9453e+05|   6.402e+06|  5.9992e+05|       -5386|  4.7582e+06|        16|      0.24|
    |  5.9439e+05|   6.409e+06|  5.9978e+05|     -5385.3|  4.7563e+06|        17|      0.24|
    |  5.9419e+05|  6.4005e+06|  5.9957e+05|     -5385.6|  4.7549e+06|        17|      0.25|
    |  5.9386e+05|  6.3824e+06|  5.9925e+05|     -5386.6|  4.7468e+06|        16|      0.25|
    |  5.9362e+05|  6.3753e+06|  5.9901e+05|     -5386.7|  4.7493e+06|        16|      0.24|
    |  5.9322e+05|  6.3676e+06|  5.9861e+05|     -5387.2|  4.7601e+06|        16|      0.25|
    |  5.9311e+05|   6.373e+06|  5.9849e+05|     -5386.5|  4.7591e+06|        16|      0.26|
    |  5.9318e+05|  6.3877e+06|  5.9857e+05|     -5385.4|  4.7644e+06|        17|      0.25|
    |  5.9334e+05|  6.3815e+06|  5.9873e+05|     -5386.2|  4.7579e+06|        16|      0.23|
    |  5.9312e+05|  6.3728e+06|  5.9851e+05|     -5386.5|  4.7571e+06|        16|      0.25|
    |  5.9326e+05|  6.3759e+06|  5.9864e+05|     -5386.5|  4.7563e+06|        16|      0.25|
    |  5.9319e+05|  6.3747e+06|  5.9858e+05|     -5386.5|  4.7585e+06|        16|      0.24|
    |  5.9323e+05|  6.3755e+06|  5.9862e+05|     -5386.5|  4.7595e+06|        16|      0.25|
    |  5.9327e+05|  6.3764e+06|  5.9866e+05|     -5386.5|  4.7575e+06|        16|      0.24|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  3.4092e+06|  2.5343e+07|  3.4168e+06|     -7528.5|  8.7203e+06|        20|      0.24|
    |  2.2467e+06|  2.1995e+07|  2.2536e+06|     -6926.8|  8.0939e+06|        19|      0.25|
    |  1.4704e+06|  1.6957e+07|  1.4768e+06|     -6399.5|  7.2075e+06|        19|      0.24|
    |  1.1068e+06|  1.3411e+07|  1.1129e+06|     -6074.4|  6.5417e+06|        18|      0.23|
    |  8.6387e+05|  1.0661e+07|  8.6966e+05|     -5795.5|   5.954e+06|        18|      0.25|
    |  7.4225e+05|  8.8175e+06|  7.4788e+05|     -5630.3|  5.5341e+06|        18|      0.24|
    |  6.7235e+05|  7.6658e+06|  6.7787e+05|     -5526.8|  5.2141e+06|        17|      0.25|
    |  6.3387e+05|  7.1558e+06|  6.3932e+05|     -5448.8|  4.9899e+06|        17|      0.24|
    |  6.1335e+05|  6.7496e+06|  6.1877e+05|     -5423.9|  4.8814e+06|        17|      0.24|
    |  6.1076e+05|  6.7056e+06|  6.1618e+05|     -5418.4|   4.863e+06|        17|      0.25|
    |  6.0468e+05|  6.6012e+06|  6.1008e+05|     -5406.1|  4.8233e+06|        17|      0.23|
    |  5.9819e+05|  6.4569e+06|  6.0358e+05|     -5396.3|   4.775e+06|        17|      0.23|
    |   5.968e+05|  6.4322e+06|  6.0219e+05|     -5392.3|  4.7729e+06|        17|      0.24|
    |  5.9554e+05|    6.42e+06|  6.0093e+05|       -5389|  4.7546e+06|        17|      0.23|
    |  5.9516e+05|  6.4092e+06|  6.0054e+05|     -5386.2|  4.7716e+06|        17|      0.23|
    |  5.9485e+05|  6.4051e+06|  6.0024e+05|     -5386.6|  4.7631e+06|        17|      0.24|
    |  5.9451e+05|  6.4007e+06|   5.999e+05|       -5387|  4.7598e+06|        17|      0.25|
    |  5.9416e+05|  6.3958e+06|  5.9954e+05|       -5387|  4.7533e+06|        17|      0.25|
    |  5.9357e+05|  6.3871e+06|  5.9896e+05|     -5386.8|   4.753e+06|        17|      0.25|
    |  5.9331e+05|  6.3992e+06|   5.987e+05|     -5386.3|  4.7538e+06|        17|      0.24|
    |  5.9325e+05|  6.3912e+06|  5.9863e+05|     -5386.2|  4.7553e+06|        17|      0.26|
    |   5.931e+05|  6.3778e+06|  5.9848e+05|       -5386|  4.7619e+06|        16|      0.23|
    |   5.931e+05|  6.3773e+06|  5.9848e+05|     -5385.9|  4.7602e+06|        17|      0.24|
    |  5.9305e+05|  6.3763e+06|  5.9843e+05|       -5386|  4.7597e+06|        17|      0.24|
    |  5.9304e+05|  6.3762e+06|  5.9843e+05|       -5386|  4.7621e+06|        17|      0.24|
    |  5.9307e+05|   6.377e+06|  5.9846e+05|       -5386|  4.7615e+06|        16|      0.23|
    |  5.9304e+05|  6.3761e+06|  5.9843e+05|     -5385.9|  4.7637e+06|        17|      0.26|
    |  5.9299e+05|  6.3766e+06|  5.9838e+05|     -5385.9|  4.7612e+06|        17|      0.24|
    |  5.9306e+05|  6.3801e+06|  5.9844e+05|     -5385.9|  4.7599e+06|        17|      0.24|
    |  5.9295e+05|  6.3753e+06|  5.9834e+05|     -5385.9|  4.7634e+06|        16|      0.25|
    |  5.9296e+05|  6.3752e+06|  5.9834e+05|     -5385.9|  4.7608e+06|        17|      0.23|
    |  5.9309e+05|   6.378e+06|  5.9847e+05|     -5385.9|  4.7631e+06|        16|      0.24|
    Optimization result:
          fun: 593087.1156402231
     hess_inv: <25x25 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 6.37396256e+02,  1.29403894e+02,  1.71592132e+01,  7.87202241e+02,
            1.37346463e+04,  2.83923867e+02,  0.00000000e+00, -9.41929416e+01,
            8.31120362e+01, -9.89697568e+02,  1.85028746e-02,  1.18859915e+02,
            5.20268298e+01, -2.37801534e+02,  1.76013556e+04, -5.72112693e+01,
            1.61763990e+03, -2.77147646e+02,  3.02784539e+03,  2.61391651e+02,
            2.67337779e+02,  5.39630844e+01,  0.00000000e+00,  1.97901890e+01,
            2.39186447e+02])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 32
          nit: 25
       status: 0
      success: True
            x: array([-9.21034037, -4.60517019, -2.30258509,  1.47773763, -2.30258509,
           -4.60517019,  1.38836933, -3.83292358, -2.25417628,  2.30258509,
           -1.84636793, -3.87778892, -2.30258509,  1.7180289 , -2.30258509,
            1.52931263, -2.30258509,  2.15048481, -2.30258509, -0.16374951,
           -2.30258509, -4.06382248, -0.19078879, -0.36875751, -2.30258509])

    <graphdot.model.gaussian_process.gpr.GaussianProcessRegressor at 0x7fa8f0b23690>

```
#gpr_base.kernel.hyperparameters
```

```
mu = gpr_base.predict(train.graphs)
```

```
plt.scatter(train[target], mu)
plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
plt.xlim(-9, 3)
plt.ylim(-9, 3)
plt.show()
```

```
print('Training set')
print('MAE:', np.mean(np.abs(train[target] - mu)))
print('RMSE:', np.std(train[target] - mu))
```

    Training set
    MAE: 0.1026528897885943
    RMSE: 0.1559697478035153

```
mu_test = gpr_base.predict(test.graphs)
```

```
plt.scatter(test[target], mu_test)
plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
plt.xlim(-9, 3)
plt.ylim(-9, 3)
plt.show()
```

```
print('Test set')
print('MAE:', np.mean(np.abs(test[target] - mu_test)))
print('RMSE:', np.std(test[target] - mu_test))
```

    Test set
    MAE: 1.3984523944497356
    RMSE: 1.7902476214464011

### Model 2

```
gpr2 = GaussianProcessRegressor(
    kernel=Normalization(
        MarginalizedGraphKernel(
            node_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                atomic_number=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                charge=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                chiral=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                hcount=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                hybridization=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                ring_list=uX.Constant(0.5, (0.01, 100.0)) * uX.Convolution(uX.KroneckerDelta(0.5,(0.1, 0.9)))
            ).normalized,
            edge_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                conjugated=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                order=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                ring_stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9))
            ).normalized,
            p=Uniform(1.0, p_bounds='fixed'),
            q=0.05
        )
    ),
    alpha=1e-5, #different from gpr in alpha where gpr_base alpha is 1e-4
    optimizer=True,
    normalize_y=True,
    regularization='+',
)
```

```
gpr2.fit(train.graphs, train[target], repeat=3, verbose=True)
```

    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  3.9342e+06|  7.4014e+07|  3.9413e+06|     -7048.5|  4.2676e+07|        20|      0.25|
    |  2.5581e+06|  4.8127e+07|  2.5648e+06|     -6637.5|   3.185e+07|        19|      0.26|
    |  1.9651e+06|  3.6386e+07|  1.9715e+06|     -6391.2|  2.5862e+07|        19|      0.25|
    |  1.4839e+06|  2.6275e+07|  1.4901e+06|     -6132.7|  2.0389e+07|        19|      0.25|
    |  1.1925e+06|  1.9644e+07|  1.1985e+06|     -5931.7|  1.6716e+07|        18|      0.25|
    |  9.9823e+05|  1.5219e+07|   1.004e+06|     -5762.3|  1.4068e+07|        17|      0.25|
    |   8.735e+05|  1.2618e+07|  8.7912e+05|     -5627.6|  1.2412e+07|        17|      0.25|
    |   8.215e+05|  1.1223e+07|  8.2708e+05|     -5576.1|  1.1158e+07|        16|      0.25|
    |  7.7928e+05|  1.0614e+07|  7.8479e+05|     -5517.2|   1.082e+07|        17|      0.25|
    |  7.6384e+05|  1.0397e+07|  7.6934e+05|     -5493.1|  1.0717e+07|        17|      0.25|
    |  7.5683e+05|  1.0246e+07|  7.6232e+05|     -5483.1|  1.0625e+07|        17|      0.26|
    |  7.8307e+05|  1.0957e+07|  7.8857e+05|     -5492.5|  1.1369e+07|        17|      0.25|
    |  7.5592e+05|  1.0264e+07|   7.614e+05|       -5479|  1.0686e+07|        17|      0.26|
    |  7.5278e+05|  1.0169e+07|  7.5826e+05|     -5475.4|  1.0618e+07|        17|      0.26|
    |  7.5039e+05|  1.0088e+07|  7.5587e+05|     -5472.2|  1.0571e+07|        17|      0.25|
    |  7.4793e+05|  1.0005e+07|   7.534e+05|     -5470.1|  1.0495e+07|        16|      0.27|
    |  7.4681e+05|  9.9805e+06|  7.5228e+05|     -5468.4|  1.0454e+07|        16|      0.25|
    |  7.4544e+05|  9.9933e+06|   7.509e+05|     -5465.7|  1.0403e+07|        17|      0.25|
    |  7.4519e+05|  9.9849e+06|  7.5065e+05|     -5466.5|  1.0374e+07|        16|      0.25|
    |  7.4522e+05|  9.9838e+06|  7.5069e+05|     -5466.8|  1.0376e+07|        17|      0.27|
    |  7.4542e+05|   9.991e+06|  7.5089e+05|     -5466.6|  1.0371e+07|        17|      0.25|
    |  7.4539e+05|  9.9914e+06|  7.5086e+05|     -5466.5|  1.0373e+07|        17|      0.27|
    |  7.4543e+05|  9.9918e+06|   7.509e+05|     -5466.5|  1.0368e+07|        16|      0.25|
    |   7.451e+05|  9.9824e+06|  7.5057e+05|     -5466.5|  1.0392e+07|        16|      0.27|
    |  7.4524e+05|  9.9871e+06|   7.507e+05|     -5466.5|   1.037e+07|        17|      0.27|
    |  7.4525e+05|  9.9861e+06|  7.5072e+05|     -5466.5|  1.0386e+07|        17|      0.27|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  2.2754e+06|  4.0939e+07|   2.282e+06|     -6573.6|    2.42e+07|        18|      0.25|
    |  1.7565e+06|  3.1259e+07|  1.7628e+06|     -6332.7|  2.0381e+07|        18|      0.24|
    |  1.2573e+06|   2.224e+07|  1.2633e+06|       -6009|  1.6215e+07|        19|      0.25|
    |    1.05e+06|  1.7608e+07|  1.0558e+06|     -5820.4|  1.4598e+07|        18|      0.25|
    |  8.9771e+05|  1.3272e+07|  9.0338e+05|     -5669.2|  1.2572e+07|        17|      0.27|
    |  8.1417e+05|  1.1552e+07|  8.1973e+05|     -5559.9|  1.1683e+07|        17|      0.25|
    |  7.8461e+05|  1.0854e+07|  7.9014e+05|     -5523.1|  1.1122e+07|        17|      0.26|
    |  7.6515e+05|  1.0363e+07|  7.7064e+05|     -5496.3|  1.0598e+07|        17|      0.26|
    |   7.549e+05|  1.0202e+07|  7.6038e+05|     -5480.5|  1.0455e+07|        17|      0.27|
    |  7.5113e+05|   1.013e+07|   7.566e+05|     -5473.3|  1.0408e+07|        17|      0.25|
    |  7.4842e+05|   1.001e+07|  7.5389e+05|     -5468.3|  1.0405e+07|        16|      0.25|
    |  7.4817e+05|  1.0021e+07|  7.5363e+05|     -5467.8|  1.0392e+07|        16|      0.27|
    |  7.4743e+05|  1.0028e+07|   7.529e+05|     -5467.3|  1.0359e+07|        17|      0.26|
    |  7.4775e+05|   1.007e+07|  7.5321e+05|     -5465.8|   1.042e+07|        17|      0.26|
    |  7.4729e+05|  1.0032e+07|  7.5276e+05|     -5466.8|  1.0374e+07|        17|      0.26|
    |  7.4778e+05|  1.0071e+07|  7.5325e+05|       -5466|  1.0404e+07|        17|      0.26|
    |  7.4737e+05|  1.0035e+07|  7.5284e+05|     -5466.8|  1.0383e+07|        17|      0.26|
    |  7.4755e+05|  1.0041e+07|  7.5301e+05|     -5466.8|  1.0372e+07|        17|      0.25|
    |  7.4755e+05|   1.004e+07|  7.5302e+05|     -5466.8|  1.0377e+07|        17|      0.26|
    |   7.475e+05|  1.0039e+07|  7.5297e+05|     -5466.8|   1.037e+07|        17|      0.25|
    |  7.4752e+05|  1.0039e+07|  7.5298e+05|     -5466.8|  1.0365e+07|        16|      0.26|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  5.5724e+06|  1.0057e+08|  5.5798e+06|     -7408.4|  5.0139e+07|        19|      0.24|
    |  3.3661e+06|  6.0838e+07|   3.373e+06|     -6905.1|  3.6581e+07|        19|      0.24|
    |  2.6218e+06|  4.7142e+07|  2.6285e+06|     -6665.4|  3.0491e+07|        18|      0.24|
    |  1.8232e+06|  3.1903e+07|  1.8296e+06|     -6322.8|  2.3196e+07|        18|      0.27|
    |  1.4038e+06|  2.3742e+07|  1.4099e+06|     -6073.7|  1.9114e+07|        18|      0.25|
    |   1.145e+06|  1.8596e+07|  1.1509e+06|     -5877.2|  1.6369e+07|        18|      0.24|
    |   9.735e+05|  1.4838e+07|  9.7923e+05|     -5727.6|  1.4079e+07|        17|      0.24|
    |  8.7366e+05|  1.2752e+07|  8.7928e+05|     -5625.4|  1.2581e+07|        17|      0.24|
    |  8.0328e+05|  1.1112e+07|  8.0883e+05|     -5549.3|  1.1228e+07|        17|      0.24|
    |  7.8049e+05|  1.0893e+07|  7.8599e+05|     -5501.9|  1.1205e+07|        17|      0.26|
    |  7.7098e+05|  1.0617e+07|  7.7648e+05|     -5498.4|  1.0918e+07|        17|      0.25|
    |  7.6746e+05|  1.0503e+07|  7.7296e+05|       -5497|  1.0782e+07|        17|      0.28|
    |  7.6154e+05|  1.0323e+07|  7.6703e+05|       -5492|  1.0603e+07|        17|      0.25|
    |  7.5364e+05|  1.0183e+07|  7.5912e+05|     -5479.7|  1.0515e+07|        17|      0.25|
    |  7.4872e+05|  1.0065e+07|  7.5419e+05|     -5471.1|   1.047e+07|        17|      0.24|
    |  7.4729e+05|  1.0013e+07|  7.5276e+05|     -5469.1|  1.0422e+07|        17|      0.25|
    |  7.4597e+05|  9.9721e+06|  7.5143e+05|     -5467.8|  1.0406e+07|        16|      0.24|
    |  7.4535e+05|   9.987e+06|  7.5081e+05|     -5465.9|  1.0431e+07|        16|      0.24|
    |  7.4442e+05|  9.9833e+06|  7.4989e+05|     -5465.8|  1.0408e+07|        17|      0.25|
    |  7.4428e+05|  9.9827e+06|  7.4975e+05|     -5466.5|  1.0397e+07|        17|      0.24|
    |  7.4425e+05|  9.9819e+06|  7.4972e+05|     -5466.9|  1.0382e+07|        16|      0.24|
    |  7.4408e+05|   9.975e+06|  7.4955e+05|       -5467|  1.0385e+07|        16|      0.24|
    |  7.4382e+05|  9.9587e+06|  7.4928e+05|       -5467|  1.0402e+07|        17|      0.25|
    |  7.4399e+05|  9.9658e+06|  7.4946e+05|     -5466.8|   1.041e+07|        16|      0.24|
    |   7.442e+05|   9.971e+06|  7.4967e+05|       -5467|  1.0428e+07|        17|      0.26|
    |   7.442e+05|  9.9718e+06|  7.4967e+05|       -5467|  1.0405e+07|        17|      0.26|
    |   7.438e+05|  9.9587e+06|  7.4927e+05|       -5467|  1.0409e+07|        16|      0.26|
    |  7.4399e+05|   9.964e+06|  7.4946e+05|       -5467|  1.0401e+07|        17|      0.26|
    |  7.4412e+05|  9.9681e+06|  7.4959e+05|       -5467|  1.0399e+07|        16|      0.25|
    Optimization result:
          fun: 744121.6320021643
     hess_inv: <25x25 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 9.96227102e+02,  2.00432837e+02,  2.60510180e+01, -5.48958596e+02,
            2.06815585e+04,  4.34825299e+02,  0.00000000e+00, -3.24741237e+02,
            1.95474293e+02,  1.10656164e+03,  0.00000000e+00,  1.22750805e+02,
            6.73673262e+01, -1.00813058e+03,  2.60017795e+04, -2.51396161e+01,
            2.21848468e+03, -5.02235333e+02,  4.39914545e+03,  6.69241667e+01,
            3.26864116e+02,  7.34901570e+01,  0.00000000e+00,  3.90059610e+02,
            3.90397116e+02])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 29
          nit: 23
       status: 0
      success: True
            x: array([-9.21034037, -4.60517019, -2.30258509,  1.45132733, -2.30258509,
           -4.60517019,  1.38836933, -3.41134142, -2.30258509,  2.29909486,
           -6.69419395, -4.08788863, -2.30258509,  1.6789475 , -2.30258509,
            1.54064497, -2.30258509,  2.21314656, -2.30258509, -0.35101412,
           -2.30258509, -4.06327129, -0.19078879, -0.09102722, -2.30258509])

    <graphdot.model.gaussian_process.gpr.GaussianProcessRegressor at 0x7fa8dbae75d0>

```
mu = gpr2.predict(train.graphs)
plt.scatter(train[target], mu)
plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
plt.xlim(-9, 3)
plt.ylim(-9, 3)
plt.show()
```

```
print('Training set')
print('MAE:', np.mean(np.abs(train[target] - mu)))
print('RMSE:', np.std(train[target] - mu))
```

    Training set
    MAE: 0.01420440124959926
    RMSE: 0.021649804308151027

```
mu_test = gpr2.predict(test.graphs)
plt.scatter(test[target], mu_test)
plt.plot(np.linspace(-10, 4), np.linspace(-10, 4), c = 'r')
plt.xlim(-10, 4)
plt.ylim(-10, 4)
plt.show()
```

```
print('Test set')
print('MAE:', np.mean(np.abs(test[target] - mu_test)))
print('RMSE:', np.std(test[target] - mu_test))
```

    Test set
    MAE: 1.517212238428167
    RMSE: 1.943288841802438

### Model 3

```
gpr3 = GaussianProcessRegressor(
    kernel=Normalization(
        MarginalizedGraphKernel(
            node_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                atomic_number=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                charge=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                chiral=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                hcount=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                hybridization=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                ring_list=uX.Constant(0.5, (0.01, 100.0)) * uX.Convolution(uX.KroneckerDelta(0.5,(0.1, 0.9)))
            ).normalized,
            edge_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                conjugated=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                order=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                ring_stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9))
            ).normalized,
            p=Uniform(1.0, p_bounds='fixed'),
            q=0.05
        )
    ),
    alpha=1e-3, #different from gpr in alpha where gpr_base's alpha is 1e-4
    optimizer=True,
    normalize_y=True,
    regularization='+',
)
```

```
gpr3.fit(train.graphs, train[target], repeat=3, verbose=True)
```

    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  5.2754e+05|  1.7773e+06|  5.3362e+05|     -6084.7|  9.0549e+05|        25|      0.28|
    |  4.4224e+05|  1.9515e+06|  4.4805e+05|     -5808.2|  8.7902e+05|        20|      0.27|
    |  3.1405e+05|  1.8185e+06|  3.1934e+05|     -5298.7|  8.0593e+05|        18|      0.24|
    |  2.8469e+05|  1.6176e+06|  2.8985e+05|       -5163|  7.8268e+05|        17|      0.26|
    |  2.6118e+05|  1.4683e+06|  2.6622e+05|     -5042.3|  7.5644e+05|        17|      0.24|
    |  2.5267e+05|  1.3881e+06|  2.5767e+05|     -4997.8|  7.4692e+05|        16|      0.26|
    |   2.496e+05|  1.3914e+06|  2.5458e+05|       -4978|  7.4218e+05|        17|      0.25|
    |  2.4851e+05|  1.3833e+06|  2.5349e+05|     -4971.9|  7.4096e+05|        17|      0.25|
    |  2.4817e+05|  1.3928e+06|  2.5314e+05|     -4967.6|  7.3437e+05|        17|      0.24|
    |  2.4724e+05|  1.3771e+06|  2.5221e+05|     -4964.6|  7.3871e+05|        17|      0.23|
    |  2.4712e+05|  1.3738e+06|  2.5208e+05|     -4964.4|  7.3983e+05|        17|       0.3|
    |  2.4696e+05|  1.3707e+06|  2.5192e+05|     -4963.8|  7.4051e+05|        17|      0.26|
    |  2.4668e+05|  1.3678e+06|  2.5164e+05|     -4962.6|  7.4057e+05|        17|      0.26|
    |  2.4641e+05|  1.3713e+06|  2.5137e+05|       -4960|   7.393e+05|        17|      0.24|
    |  2.4626e+05|  1.3699e+06|  2.5122e+05|     -4959.3|  7.3801e+05|        17|      0.24|
    |  2.4623e+05|  1.3693e+06|  2.5119e+05|     -4959.3|  7.3765e+05|        17|      0.24|
    |  2.4624e+05|  1.3696e+06|   2.512e+05|     -4959.2|  7.3767e+05|        17|      0.25|
    |  2.4624e+05|  1.3693e+06|   2.512e+05|     -4959.3|  7.3763e+05|        17|      0.24|
    |  2.4623e+05|  1.3693e+06|  2.5119e+05|     -4959.3|  7.3758e+05|        17|      0.24|
    |  2.4624e+05|  1.3693e+06|   2.512e+05|     -4959.3|   7.377e+05|        17|      0.26|
    |  2.4624e+05|  1.3693e+06|   2.512e+05|     -4959.3|  7.3766e+05|        17|      0.28|
    |  2.4624e+05|  1.3694e+06|   2.512e+05|     -4959.3|  7.3766e+05|        17|      0.26|
    |  2.4623e+05|  1.3692e+06|  2.5119e+05|     -4959.3|  7.3767e+05|        17|      0.26|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  4.3132e+05|  2.1991e+06|  4.3709e+05|     -5766.2|  8.8044e+05|        20|      0.23|
    |  3.8205e+05|  2.0562e+06|  3.8763e+05|     -5587.8|  8.5782e+05|        19|      0.23|
    |  3.1542e+05|  1.9024e+06|  3.2073e+05|     -5313.3|  8.2092e+05|        19|      0.25|
    |  2.8239e+05|  1.7354e+06|  2.8753e+05|     -5138.7|  7.6999e+05|        19|      0.24|
    |  2.5844e+05|  1.4472e+06|  2.6347e+05|     -5032.7|  7.6497e+05|        17|      0.25|
    |  2.5462e+05|   1.423e+06|  2.5963e+05|     -5010.1|  7.5654e+05|        17|      0.24|
    |  2.4941e+05|  1.3953e+06|  2.5438e+05|     -4977.7|  7.4492e+05|        17|      0.23|
    |  2.4782e+05|  1.3818e+06|  2.5279e+05|     -4968.9|  7.3616e+05|        17|      0.24|
    |  2.4685e+05|  1.3767e+06|  2.5181e+05|     -4963.2|  7.3744e+05|        17|      0.23|
    |  2.4666e+05|  1.3771e+06|  2.5162e+05|     -4960.4|  7.3963e+05|        17|      0.25|
    |  2.4631e+05|  1.3714e+06|  2.5127e+05|     -4959.9|  7.3887e+05|        17|      0.23|
    |  2.4624e+05|  1.3688e+06|   2.512e+05|     -4960.1|  7.3878e+05|        17|      0.23|
    |  2.4618e+05|  1.3657e+06|  2.5114e+05|     -4960.1|  7.3874e+05|        17|      0.23|
    |  2.4615e+05|  1.3645e+06|  2.5111e+05|     -4959.9|  7.3868e+05|        17|      0.25|
    |  2.4608e+05|  1.3633e+06|  2.5103e+05|     -4959.5|  7.3863e+05|        17|      0.23|
    |  2.4597e+05|  1.3625e+06|  2.5093e+05|     -4958.9|  7.3829e+05|        17|      0.23|
    |  2.4593e+05|  1.3645e+06|  2.5089e+05|     -4958.5|   7.376e+05|        17|      0.25|
    |  2.4589e+05|  1.3641e+06|  2.5085e+05|     -4958.3|  7.3836e+05|        17|      0.24|
    |  2.4588e+05|  1.3646e+06|  2.5084e+05|     -4958.2|  7.3802e+05|        17|      0.23|
    |  2.4585e+05|  1.3647e+06|  2.5081e+05|     -4958.2|  7.3765e+05|        17|      0.28|
    |  2.4581e+05|  1.3635e+06|  2.5077e+05|     -4958.3|  7.3747e+05|        17|      0.25|
    |  2.4579e+05|  1.3623e+06|  2.5074e+05|     -4958.3|  7.3782e+05|        17|      0.25|
    |  2.4577e+05|   1.359e+06|  2.5073e+05|     -4958.8|  7.3883e+05|        17|      0.27|
    |  2.4576e+05|  1.3601e+06|  2.5072e+05|     -4958.6|   7.387e+05|        17|      0.24|
    |  2.4576e+05|  1.3607e+06|  2.5072e+05|     -4958.5|  7.3859e+05|        17|      0.23|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |  5.7164e+05|  1.6233e+06|  5.7783e+05|     -6194.5|  8.9548e+05|        21|      0.25|
    |  4.7796e+05|  1.8426e+06|  4.8386e+05|     -5904.1|  8.7295e+05|        20|      0.26|
    |  3.3798e+05|  1.8581e+06|  3.4337e+05|     -5394.5|  8.2045e+05|        18|      0.24|
    |  2.9137e+05|  1.7091e+06|  2.9656e+05|     -5188.7|  7.8714e+05|        19|      0.23|
    |  2.6899e+05|  1.5524e+06|  2.7408e+05|     -5081.7|  7.6873e+05|        18|      0.25|
    |  2.5569e+05|  1.4244e+06|  2.6071e+05|     -5015.7|  7.5632e+05|        17|      0.25|
    |  2.5073e+05|  1.4143e+06|  2.5571e+05|     -4982.2|  7.4341e+05|        18|      0.25|
    |  2.4974e+05|  1.4008e+06|  2.5472e+05|     -4979.3|  7.4547e+05|        17|      0.23|
    |  2.4886e+05|  1.3879e+06|  2.5384e+05|     -4975.9|  7.4569e+05|        17|      0.24|
    |  2.4798e+05|  1.3761e+06|  2.5295e+05|     -4971.5|  7.4426e+05|        17|      0.23|
    |  2.4717e+05|   1.378e+06|  2.5214e+05|     -4965.5|  7.4085e+05|        17|      0.27|
    |  2.4669e+05|  1.3741e+06|  2.5166e+05|     -4962.6|  7.3887e+05|        17|      0.24|
    |   2.464e+05|  1.3657e+06|  2.5136e+05|     -4960.9|   7.412e+05|        17|      0.23|
    |  2.4628e+05|  1.3668e+06|  2.5124e+05|     -4960.5|  7.3916e+05|        17|      0.25|
    |  2.4624e+05|  1.3678e+06|   2.512e+05|     -4960.2|  7.3857e+05|        17|      0.23|
    |  2.4619e+05|  1.3698e+06|  2.5115e+05|     -4959.9|  7.3793e+05|        17|      0.24|
    |  2.4614e+05|  1.3703e+06|   2.511e+05|     -4959.6|  7.3771e+05|        17|      0.24|
    |  2.4595e+05|  1.3683e+06|  2.5091e+05|     -4959.1|  7.3767e+05|        17|      0.24|
    |  2.4602e+05|  1.3688e+06|  2.5098e+05|     -4958.9|  7.3875e+05|        17|      0.24|
    |  2.4592e+05|  1.3683e+06|  2.5088e+05|     -4958.9|  7.3796e+05|        17|      0.23|
    |  2.4589e+05|  1.3655e+06|  2.5085e+05|       -4959|  7.3856e+05|        17|      0.24|
    |  2.4586e+05|  1.3623e+06|  2.5082e+05|     -4959.2|  7.3918e+05|        17|      0.24|
    |  2.4584e+05|  1.3616e+06|   2.508e+05|     -4959.3|  7.3917e+05|        17|      0.24|
    |  2.4583e+05|  1.3615e+06|  2.5079e+05|     -4959.3|  7.3903e+05|        17|      0.23|
    |  2.4579e+05|  1.3614e+06|  2.5075e+05|     -4959.3|  7.3865e+05|        17|      0.24|
    |  2.4578e+05|   1.362e+06|  2.5074e+05|     -4959.2|  7.3874e+05|        17|      0.24|
    |  2.4576e+05|  1.3621e+06|  2.5072e+05|     -4959.1|   7.391e+05|        17|      0.23|
    |  2.4576e+05|  1.3624e+06|  2.5072e+05|     -4959.1|  7.3901e+05|        17|      0.27|
    Optimization result:
          fun: 245757.64498811733
     hess_inv: <25x25 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 1.35950050e+02,  3.77357938e+01,  1.05734705e+01, -2.13075274e+01,
            3.36341811e+03,  6.74591558e+01,  0.00000000e+00,  1.41114101e+00,
            7.33894523e+00, -1.51578382e+02,  0.00000000e+00,  1.62796677e+01,
            5.68562303e+00,  5.22025613e+01,  4.39601078e+03, -3.71667507e+01,
            5.31204650e+02,  1.71062274e+01,  6.87112378e+02,  6.34393286e+01,
            7.78485000e+01,  1.29002790e+01,  0.00000000e+00, -5.61367681e+01,
            4.45773454e+01])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 25
          nit: 24
       status: 0
      success: True
            x: array([-9.21034037, -4.60517019, -1.33774258,  1.50058885, -2.30258509,
           -4.60517019, -0.21165109, -4.60517019, -2.30258509,  2.2686602 ,
           -4.07543091, -4.60517019, -2.30258509,  1.77034144, -2.30258509,
            1.30229701, -2.30258509,  1.57009793, -2.30258509, -0.52372387,
           -2.30258509, -4.60517019, -0.10536052, -1.31858519, -2.30258509])

    <graphdot.model.gaussian_process.gpr.GaussianProcessRegressor at 0x7fb9a0f83150>

```
mu = gpr3.predict(train.graphs)
plt.scatter(train[target], mu)
#plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
#plt.xlim(-9, 3)
#plt.ylim(-9, 3)
plt.show()
```

```
print('Training set')
print('MAE:', np.mean(np.abs(train[target] - mu)))
print('RMSE:', np.std(train[target] - mu))
```

    Training set
    MAE: 0.3711113759832458
    RMSE: 0.5245520571276874

```
mu_test = gpr3.predict(test.graphs)
plt.scatter(test[target], mu_test)
#plt.plot(np.linspace(-10, 4), np.linspace(-10, 4), c = 'r')
#plt.xlim(-10, 4)
#plt.ylim(-10, 4)
plt.show()
```

```
print('Test set')
print('MAE:', np.mean(np.abs(test[target] - mu_test)))
print('RMSE:', np.std(test[target] - mu_test))
```

    Test set
    MAE: 1.1498112446483284
    RMSE: 1.4624191158480997

### Model 4

```
gpr4 = GaussianProcessRegressor(
    kernel=Normalization(
        MarginalizedGraphKernel(
            node_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                atomic_number=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                charge=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                chiral=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                hcount=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                hybridization=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                ring_list=uX.Constant(0.5, (0.01, 100.0)) * uX.Convolution(uX.KroneckerDelta(0.5,(0.1, 0.9)))
            ).normalized,
            edge_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                conjugated=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                order=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                ring_stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9))
            ).normalized,
            p=Uniform(1.0, p_bounds='fixed'),
            q=0.05
        )
    ),
    alpha=1e-2, #different from gpr in alpha where gpr_base's alpha is 1e-4
    optimizer=True,
    normalize_y=True,
    regularization='+',
)
```

```
gpr4.fit(train.graphs, train[target], repeat=3, verbose=True)
```

    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |       66517|       88265|       70813|     -4296.2|       90905|        21|      0.28|
    |       61455|  1.0409e+05|       65648|     -4193.5|       88757|        19|      0.25|
    |       50869|  1.1321e+05|       54789|     -3920.3|       81573|        17|      0.23|
    |       49285|  1.0995e+05|       53162|     -3876.6|       80232|        17|      0.28|
    |       48321|  1.0553e+05|       52167|       -3845|       78704|        16|      0.24|
    |       47912|  1.0664e+05|       51747|     -3834.9|       78334|        17|      0.25|
    |       47815|  1.0611e+05|       51648|     -3832.7|       78473|        17|      0.25|
    |       47734|  1.0602e+05|       51565|     -3830.1|       78399|        17|      0.24|
    |       47721|   1.056e+05|       51549|     -3828.8|       78754|        17|      0.25|
    |       47682|  1.0583e+05|       51510|     -3827.7|       78471|        17|      0.25|
    |       47662|  1.0594e+05|       51489|     -3827.2|       78334|        17|      0.28|
    |       47637|  1.0591e+05|       51463|     -3826.7|       78258|        17|      0.24|
    |       47613|  1.0565e+05|       51439|     -3826.3|       78273|        17|      0.24|
    |       47587|  1.0505e+05|       51413|     -3826.2|       78397|        17|      0.25|
    |       47584|  1.0508e+05|       51410|       -3826|       78364|        17|      0.25|
    |       47579|  1.0506e+05|       51404|     -3825.6|       78277|        17|      0.23|
    |       47574|  1.0486e+05|       51399|     -3825.5|       78232|        17|      0.25|
    |       47570|    1.04e+05|       51397|     -3826.4|       78274|        17|      0.28|
    |       47576|  1.0386e+05|       51402|     -3825.9|       78262|        17|      0.24|
    |       47569|  1.0395e+05|       51395|     -3826.1|       78271|        17|      0.24|
    |       47567|  1.0417e+05|       51393|     -3826.1|       78324|        17|      0.28|
    |       47566|  1.0407e+05|       51392|     -3826.3|       78362|        17|      0.23|
    |       47566|  1.0398e+05|       51392|     -3826.4|       78386|        17|      0.24|
    |       47566|  1.0405e+05|       51393|     -3826.3|       78366|        17|      0.24|
    |       47566|  1.0406e+05|       51392|     -3826.3|       78363|        17|      0.25|
    |       47566|  1.0407e+05|       51392|     -3826.3|       78362|        17|      0.24|
    |       47566|  1.0407e+05|       51392|     -3826.3|       78361|        17|      0.25|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |       73087|       31386|       77530|       -4443|       95203|        16|      0.26|
    |       69027|       48566|       73397|     -4370.2|       93280|        18|      0.25|
    |       56295|  1.0441e+05|       60364|     -4068.9|       85503|        19|      0.26|
    |       49629|  1.0815e+05|       53493|     -3864.5|       76173|        17|      0.26|
    |       48176|  1.0901e+05|       52013|     -3837.5|       77267|        17|      0.26|
    |       49949|   1.003e+05|       53850|       -3901|       82147|        18|      0.26|
    |       47973|   1.074e+05|       51809|     -3836.1|       78349|        18|      0.26|
    |       47868|  1.0647e+05|       51701|     -3833.7|       78588|        17|      0.25|
    |       47825|  1.0603e+05|       51658|     -3832.9|       78699|        17|      0.25|
    |       47758|  1.0569e+05|       51589|     -3831.2|       78709|        17|      0.25|
    |       47712|  1.0601e+05|       51542|     -3829.4|       78587|        17|      0.26|
    |       47692|  1.0615e+05|       51520|     -3828.2|       78413|        17|      0.24|
    |       47690|  1.0616e+05|       51518|     -3828.1|       78396|        17|      0.23|
    |       47681|  1.0614e+05|       51508|     -3827.4|       78346|        17|      0.24|
    |       47681|  1.0645e+05|       51508|     -3827.2|       78253|        17|      0.24|
    |       47674|  1.0629e+05|       51501|     -3827.1|       78299|        17|      0.25|
    |       47654|  1.0622e+05|       51481|     -3826.3|       78288|        17|      0.25|
    |       47645|  1.0603e+05|       51471|     -3826.2|       78342|        17|      0.26|
    |       47644|  1.0605e+05|       51471|     -3826.4|       78352|        17|      0.26|
    |       47644|  1.0604e+05|       51471|     -3826.3|       78347|        17|      0.24|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |       67828|  1.0852e+05|       72138|     -4309.5|       89137|        24|      0.25|
    |       62412|  1.1469e+05|       66616|       -4204|       87873|        21|      0.25|
    |       56410|   1.222e+05|       60478|     -4067.7|       85994|        20|      0.26|
    |       51080|  1.1784e+05|       55004|     -3924.2|       81494|        18|      0.26|
    |       49634|  1.1413e+05|       53522|     -3888.1|       80979|        18|      0.24|
    |       48936|  1.0759e+05|       52800|     -3863.2|       79659|        17|      0.24|
    |       48434|  1.0891e+05|       52284|     -3850.2|       79231|        17|      0.24|
    |       48204|  1.0827e+05|       52049|     -3844.6|       79132|        18|      0.25|
    |       47886|  1.0625e+05|       51723|     -3836.7|       78967|        17|      0.24|
    |       48724|  1.0903e+05|       52577|     -3853.3|       76871|        19|      0.24|
    |       47877|  1.0635e+05|       51713|     -3836.1|       78785|        18|      0.24|
    |       48134|  1.0654e+05|       51976|     -3841.5|       78087|        18|      0.24|
    |       47877|  1.0635e+05|       51712|       -3836|       78744|        17|      0.23|
    |       47789|  1.0549e+05|       51623|     -3834.3|       78780|        17|      0.24|
    |       47692|  1.0404e+05|       51525|     -3832.6|       78739|        17|      0.24|
    |       47665|  1.0386e+05|       51497|     -3831.4|       78613|        17|      0.24|
    |       47626|  1.0394e+05|       51456|     -3829.6|       78486|        17|      0.26|
    |       47604|  1.0463e+05|       51432|     -3827.8|       78343|        17|      0.26|
    |       47590|  1.0446e+05|       51417|     -3827.3|       78398|        17|      0.24|
    |       47588|  1.0442e+05|       51415|     -3827.1|       78415|        17|      0.26|
    |       47586|  1.0443e+05|       51413|     -3826.9|       78414|        17|      0.28|
    |       47582|   1.045e+05|       51409|     -3826.5|       78406|        17|      0.24|
    |       47579|  1.0481e+05|       51405|     -3825.9|       78253|        17|      0.23|
    |       47575|  1.0471e+05|       51401|       -3826|       78318|        17|      0.24|
    |       47573|  1.0459e+05|       51399|     -3826.2|       78354|        17|      0.24|
    |       47571|  1.0449e+05|       51397|     -3826.3|       78369|        17|      0.24|
    |       47568|  1.0432e+05|       51394|     -3826.5|       78382|        17|      0.23|
    |       47566|  1.0421e+05|       51393|     -3826.5|       78376|        17|      0.24|
    |       47565|   1.042e+05|       51391|     -3826.3|       78347|        17|      0.26|
    |       47564|  1.0411e+05|       51390|     -3826.1|       78329|        17|      0.24|
    |       47564|  1.0402e+05|       51390|     -3826.2|       78343|        17|      0.23|
    |       47564|  1.0392e+05|       51390|     -3826.3|       78361|        17|      0.26|
    |       47563|  1.0399e+05|       51389|     -3826.2|       78348|        17|      0.24|
    Optimization result:
          fun: 47563.16826613996
     hess_inv: <25x25 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 1.03860634e+01,  3.99556925e+00,  4.34356887e-01, -4.38765528e+00,
            3.35685830e+02,  5.48613650e+00,  0.00000000e+00,  3.74597978e+00,
            3.71923847e-01, -1.86677689e+01,  1.80494389e-04,  2.34556512e+00,
            6.89757782e-01,  7.46525570e+00,  3.86435984e+02, -4.08724178e+00,
            6.25487975e+01,  3.79069496e+00,  5.00876053e+01, -1.47395577e+00,
            1.52830838e+01,  1.49931223e+00,  0.00000000e+00,  2.53256179e-01,
            2.78310073e-01])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 33
          nit: 29
       status: 0
      success: True
            x: array([-9.21034037, -4.23992363, -2.30258509,  1.70454451, -2.30258509,
           -4.60517019, -0.39405819, -4.60517019, -1.7367717 ,  2.30258509,
           -1.86521329, -4.33629426, -2.08030139,  1.84897251, -2.30258509,
            2.14248562, -2.30258509,  1.93608271, -2.30258509,  0.72987226,
           -2.30258509, -3.7783557 , -1.83514442, -3.16885023, -2.30258509])

    <graphdot.model.gaussian_process.gpr.GaussianProcessRegressor at 0x7fb9896f5fd0>

```
mu = gpr4.predict(train.graphs)
plt.scatter(train[target], mu)
#plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
#plt.xlim(-9, 3)
#plt.ylim(-9, 3)
plt.show()
```

```
print('Training set')
print('MAE:', np.mean(np.abs(train[target] - mu)))
print('RMSE:', np.std(train[target] - mu))
```

    Training set
    MAE: 0.6721642416785486
    RMSE: 0.9001504628434619

```
mu_test = gpr4.predict(test.graphs)
plt.scatter(test[target], mu_test)
#plt.plot(np.linspace(-10, 4), np.linspace(-10, 4), c = 'r')
#plt.xlim(-10, 4)
#plt.ylim(-10, 4)
plt.show()
```

```
print('Test set')
print('MAE:', np.mean(np.abs(test[target] - mu_test)))
print('RMSE:', np.std(test[target] - mu_test))
```

    Test set
    MAE: 0.996800387138675
    RMSE: 1.2687098498475842

### Model 5

```
gpr5 = GaussianProcessRegressor(
    kernel=Normalization(
        MarginalizedGraphKernel(
            node_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                atomic_number=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                charge=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                chiral=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                hcount=uX.Constant(0.5, (0.01, 10.0)) * uX.SquareExponential(1.0),
                hybridization=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                ring_list=uX.Constant(0.5, (0.01, 100.0)) * uX.Convolution(uX.KroneckerDelta(0.5,(0.1, 0.9)))
            ).normalized,
            edge_kernel=uX.Additive(
                aromatic=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                conjugated=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.5,(0.1, 0.9)),
                order=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                ring_stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9)),
                stereo=uX.Constant(0.5, (0.01, 10.0)) * uX.KroneckerDelta(0.8,(0.1, 0.9))
            ).normalized,
            p=Uniform(1.0, p_bounds='fixed'),
            q=0.05
        )
    ),
    alpha=1e-1, #different from gpr in alpha where gpr_base's alpha is 1e-4
    optimizer=True,
    normalize_y=True,
    regularization='+',
)
```

```
gpr5.fit(train.graphs, train[target], repeat=3, verbose=True)
```

    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |      5933.5|      3681.9|        8130|     -2196.5|        9095|        23|      0.27|
    |      5700.6|      4368.7|      7866.6|       -2166|        8897|        21|      0.23|
    |      5244.7|      4143.6|      7321.1|     -2076.4|      8114.4|        17|      0.24|
    |      5161.9|      4261.1|      7222.8|       -2061|      8040.9|        17|      0.23|
    |      5123.7|      4119.8|      7182.2|     -2058.5|      8116.4|        17|      0.25|
    |      5100.9|      4055.4|      7155.4|     -2054.5|      8044.2|        17|      0.25|
    |      5081.3|      3976.5|      7132.4|     -2051.1|      7995.7|        17|      0.24|
    |      5078.6|      3971.4|      7128.4|     -2049.9|      7991.1|        17|      0.24|
    |        5077|      3973.1|      7126.9|     -2049.9|      7990.8|        17|      0.24|
    |      5075.4|      3971.5|      7125.2|     -2049.7|      7988.9|        17|      0.24|
    |        5074|      3966.8|      7123.6|     -2049.5|      7985.4|        17|      0.26|
    |      5071.8|      3952.7|        7121|     -2049.2|      7981.1|        17|      0.25|
    |      5068.8|      3927.3|      7117.9|     -2049.1|      7982.1|        17|      0.26|
    |      5067.9|      3915.3|      7117.1|     -2049.1|      7980.4|        17|      0.25|
    |      5067.8|      3917.4|      7116.9|     -2049.1|      7979.1|        17|      0.25|
    |      5067.8|      3919.2|      7116.8|       -2049|      7978.5|        17|      0.24|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |      6316.2|      4166.3|      8544.1|     -2227.9|        9356|        21|      0.23|
    |      5897.8|      4552.2|      8086.8|       -2189|      9123.3|        20|      0.25|
    |      5402.1|      4470.5|      7523.5|     -2121.3|      8676.4|        18|      0.23|
    |      5263.9|      4484.2|        7355|     -2091.2|      8445.2|        18|      0.26|
    |      5192.2|      4382.6|      7262.9|     -2070.7|      8189.6|        18|      0.25|
    |      5121.6|      4136.4|      7180.3|     -2058.7|      8143.5|        17|      0.24|
    |      5111.8|        4133|      7163.8|       -2052|        7978|        17|      0.23|
    |      5100.7|      4164.8|      7150.7|     -2049.9|      7976.5|        17|      0.24|
    |        5094|        4154|        7143|       -2049|      7983.6|        17|      0.23|
    |      5089.6|        4131|      7138.2|     -2048.5|      7988.5|        17|      0.23|
    |      5086.1|      4087.2|      7133.9|     -2047.8|      7974.4|        17|      0.23|
    |      5082.2|      4085.6|        7130|     -2047.8|      7979.5|        17|      0.26|
    |      5079.1|      4076.2|      7126.9|     -2047.9|      7977.4|        17|      0.23|
    |      5078.3|      4068.3|      7126.3|       -2048|      7976.8|        17|      0.26|
    |      5075.3|      3953.8|        7125|     -2049.7|      7975.3|        17|      0.23|
    |      5073.1|      3984.2|      7122.2|     -2049.1|      7987.6|        17|      0.26|
    |      5072.6|      3973.4|      7121.8|     -2049.2|      7988.8|        17|      0.28|
    |      5071.1|      3934.7|      7120.4|     -2049.3|      7985.3|        17|      0.26|
    |      5069.9|        3928|      7119.1|     -2049.2|      7984.2|        17|      0.25|
    |      5068.5|      3913.3|      7117.8|     -2049.3|      7983.9|        17|      0.24|
    |      5067.8|      3915.3|      7116.6|     -2048.8|      7969.8|        17|      0.25|
    |      5067.5|      3903.7|      7117.2|     -2049.7|      7993.3|        17|      0.25|
    |      5067.3|      3908.8|      7116.5|     -2049.2|      7983.6|        17|      0.24|
    |      5067.1|        3910|      7116.1|       -2049|      7977.3|        17|      0.23|
    |      5067.1|      3910.1|      7116.1|       -2049|      7977.7|        17|      0.26|
    |        logP|       dlogP|     y^T.K.y|     log|K| |     Cond(K)|  GPU time|  CPU time|
    |------------|------------|------------|------------|------------|----------|----------|
    |      5834.4|      4699.1|      8022.2|     -2187.9|      9207.7|        21|      0.23|
    |      5579.2|      4272.2|      7729.4|     -2150.2|      8902.1|        19|      0.24|
    |        5265|      4187.4|      7346.4|     -2081.4|      8221.2|        18|      0.27|
    |      5170.4|      3954.9|      7238.8|     -2068.4|      8110.4|        17|      0.24|
    |      5137.6|      4046.1|      7198.2|     -2060.6|      8053.1|        17|      0.25|
    |      5109.8|      4060.1|      7164.7|     -2054.9|      8036.6|        17|      0.23|
    |      5092.8|      4026.6|      7144.8|       -2052|      8014.6|        17|      0.23|
    |      5086.7|      3944.7|      7136.4|     -2049.7|      7929.4|        17|      0.23|
    |      5078.8|      3926.5|      7130.6|     -2051.8|      8024.8|        17|      0.24|
    |      5077.1|      3942.6|      7128.1|     -2050.9|      8009.4|        17|      0.23|
    |      5075.3|      3953.9|      7125.3|     -2049.9|      7991.6|        17|      0.27|
    |      5072.7|        3957|      7121.7|       -2049|      7973.5|        17|      0.25|
    |      5069.8|      3945.2|      7118.4|     -2048.6|      7968.5|        17|      0.24|
    |      5069.4|        3932|      7118.3|     -2048.9|      7981.7|        17|      0.25|
    |      5068.9|      3930.6|      7117.9|       -2049|        7981|        17|      0.23|
    |      5068.8|      3926.4|      7117.9|     -2049.1|      7983.4|        17|      0.24|
    |      5068.4|      3917.5|      7117.9|     -2049.4|      7987.3|        17|      0.23|
    |      5068.1|      3910.6|      7117.7|     -2049.6|        7989|        17|      0.25|
    |      5067.8|      3913.7|      7117.1|     -2049.3|        7985|        17|      0.23|
    |      5067.6|      3919.1|      7116.2|     -2048.6|      7972.4|        17|      0.23|
    |      5067.4|      3918.6|      7116.2|     -2048.8|      7975.7|        17|      0.23|
    |      5067.4|        3918|      7116.2|     -2048.9|        7976|        17|      0.23|
    Optimization result:
          fun: 5067.119508760024
     hess_inv: <25x25 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 3.90259274e-01,  1.87865706e-01,  2.15153320e-02,  5.95899075e-02,
            1.81887249e+01,  3.14234542e-01,  0.00000000e+00,  2.98486676e-01,
            9.99179948e-03, -1.21282065e+00,  0.00000000e+00,  1.07942514e-01,
            2.29213419e-02,  2.45315195e-01,  1.44913639e+01,  2.38720249e-03,
            6.78953722e-03, -1.08856133e-01,  3.25365990e+00,  4.00300271e-02,
            3.61716207e+00,  3.49621047e-02,  0.00000000e+00,  3.14194196e-02,
            3.39042590e-03])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 25
          nit: 24
       status: 0
      success: True
            x: array([-9.21034037, -4.60517019, -1.9276534 ,  1.65083009, -2.30258509,
           -4.60517019,  0.08473439, -4.60517019, -0.94627051,  2.30258509,
           -4.52691773, -4.60517019, -2.30258509,  1.42510738, -2.30258509,
           -4.00849461, -2.30258509,  2.12160197, -2.30258509,  2.23247174,
           -2.30258509, -4.60517019, -1.5016103 , -4.60517019, -0.71533121])

    <graphdot.model.gaussian_process.gpr.GaussianProcessRegressor at 0x7fb9880bba10>

```
mu = gpr5.predict(train.graphs)
plt.scatter(train[target], mu)
#plt.plot(np.linspace(-9, 3), np.linspace(-9, 3), c = 'r')
#plt.xlim(-9, 3)
#plt.ylim(-9, 3)
plt.show()
```

```
print('Training set')
print('MAE:', np.mean(np.abs(train[target] - mu)))
print('RMSE:', np.std(train[target] - mu))
```

    Training set
    MAE: 0.8642262237055949
    RMSE: 1.1297768774199661

```
mu_test = gpr4.predict(test.graphs)
plt.scatter(test[target], mu_test)
#plt.plot(np.linspace(-10, 4), np.linspace(-10, 4), c = 'r')
#plt.xlim(-10, 4)
#plt.ylim(-10, 4)
plt.show()
```

```
print('Test set')
print('MAE:', np.mean(np.abs(test[target] - mu_test)))
print('RMSE:', np.std(test[target] - mu_test))
```

    Test set
    MAE: 0.9967978325813834
    RMSE: 1.2687065452737312

```

```