#####
 Data Scientist IBM Professional Certification

######
 Course 9: Capstone Project

# Insights on Collision Incidents in Seattle, Washington in 2004-2020
Data Science: Towards Zero Collision Accident!

*Published by **Ihsan Ahmad Zulkarnain**, Indonesia, 6 October 2020*

## Table of Contents
1. [Introduction](#1)
    1.1. [Background](#1.1)
    1.2. [The Questions of Interest](#1.2)
2. [Data Overview](#2)
    2.1. [Metadata](#2.1)
    2.2. [Data Description](#2.2)
3. [Methodology](#3)
    3.1. [Data Preprocessing](#3.1)
    3.2. [Exploratory Data Analysis and Data Visualization](#3.2)
    3.3. [Accident Severity Classification](#3.3)
    3.3.1. [Data Preprocessing for Classification](#3.3.1)
    3.3.2. [Classification Models](#3.3.2)
    3.3.3. [Metrics for Model Evaluation](#3.3.3)
4. [Results and Discussion](#4)
    4.1. [Exploratory Data Analysis](#4.1)
    4.1.1. [Descriptive Statistics](#4.1.1)
    4.1.2. [Grouping Data into Categories](#4.1.2)
    4.1.3. [Waffle Charts of Categorical Features](#4.1.3)
    4.1.4. [Weather and Road Condition Effects on the Number and the Severity of the Accidents](#4.1.4)
    4.1.5. [Time-Series Data](#4.1.5)
    4.1.6. [Analysis of Variance (ANOVA)](#4.1.6)
    4.1.7. [Correlation Analysis](#4.1.7)
    4.1.8. [Choropleth Map Visualization](#4.1.8)
    4.2. [Accident Severity Classification](#4.2)
    4.2.1. [K-Nearest Neighbors (KNN)](#4.2.1)
    4.2.2. [Decision Tree](#4.2.2)
    4.2.3. [Support Vector Machine (SVM)](#4.2.3)
    4.2.4. [Logistic Regression](#4.2.4)
    4.2.5. [Model Evaluation using the Dataset of 2020](#4.2.5)
    4.2.6. [The Significant Features Leading to Accidents with Injury based on the Classification Model](#4.2.6)
5. [Conclusion and Remarks](#5)

## List of Tables
1. [Descriptive statistics of the data](#t1)
2. [The numbers of no injury (1) and with injury (2) accidents](#t2)
3. [The numbers of accidents based on the accident location type](#t3)
4. [The numbers of accidents between 2004-2020 by district](#t4)
5. [The evaluation metrics for each classification model](#t5)

## List of Figures
1. [The waffle charts for seven data features](#f1)
2. [The numbers of accidents by weather/road condition and its severity](#f2)
3. [Time-series data of the monthly (a) and quarterly (b) numbers of accidents](#f3)
4. [The heatmap of the correlation among the accident severity and the features of weather](#f4)
5. [The choropleth map of the numbers of accidents by its district (a: district number from WikiMedia, b: choropleth map)](#f5)
6. [The choropleth map of the numbers of accidents by its neighborhood (a: 2009, b: 2019)](#f6)
7. [The choropleth map of the numbers of accidents per neighborhood area (a: 2009, b: 2019)](#f7)
8. [The Jaccard similarity scores for various values of k in the KNN model](#f8)
9. [The Jaccard similarity scores for various values of maximum depth in the decision tree model](#f9)
10. [The decision tree model with maximum depth of five](#f10)
11. [The confusion matrix for the support vector machine model on the test dataset](#f11)
12. [The confusion matrices for all models on the final test dataset](#f12)
13. [The maps with marker-clusters for cases where pedestrian rights were not granted (a to d zoom level = 11, 13, 15, 17 respectively)](#f13)
14. [The maps with marker-clusters for cases at intersection and under alcohol influence (a to d zoom level = 11, 13, 15, 17 respectively)](#f14)
15. [The maps with marker-clusters for cases on mid-block, speeding, and under alcohol influence (a to d zoom level = 11, 13, 15, 17 respectively)](#f15)

## 1. Introduction [[ToC]](#ToC)

### 1.1. Background [[ToC]](#ToC)

Government is the collective embodiment of its people. It is responsible for the well-being of its citizens. The government of Seattle city is not an exception. In order to serve and protect its citizen, the government plans, executes, and evaluates many public policies which directly and indirectly impact many parts of the life of its citizen. One of the policies is accident management.

The main tasks of accident management are prevention and mitigation. Its goal is to reduce the severity and the number of potential accidents in future. But how? There are many prevention and mitigation strategies available. But which strategies to choose? And how to effectively plan and execute them? The power of data may help answer these questions.

By gaining insight into a problem using data, we can gain valuable knowledge which helps us in planning, executing, and evaluating strategies to solve the problem. This is why collecting, documenting, and securely storing data are really important and can be valuable in long term. This is what the government of Seattle has sought since 2004. The government, through its Seattle Police Department, has been recording 194,673 cases of collision accidents in Seattle. This large number of data can be analysed and mined into valuable knowledge to help the government fulfill its vision towards zero collision accident in future.

### 1.2. The Questions of Interest [[ToC]](#ToC)

In order to gain valuable knowledge from the data, some questions need to be addressed and answered:
1. What insight can be obtained from the data by exploratory data analysis and data visualization?
2. Which classification model is best for predicting the severity status of an accident?

## 2. Data Quick Overview [[ToC]](#ToC)

### 2.1. Metadata [[ToC]](#ToC)

- **Source**: [CognitiveClass](https://s3.us.cloud-object-storage.appdomain.cloud/cf-courses-data/CognitiveClass/DP0701EN/version-2/Data-Collisions.csv) ([Metadata](https://s3.us.cloud-object-storage.appdomain.cloud/cf-courses-data/CognitiveClass/DP0701EN/version-2/Metadata.pdf))
- **Access date**: 25 September 2020
- **Format**: Comma-separated values
- **Size**: **194674 rows** (including the first row as the column headers) representing the observant and **38 columns** representing the features/attributes of the observant

More detailed information about the data can be found in the metadata pdf file in the source.

### 2.2. Data Description [[ToC]](#ToC)

- The data is originally provided by the Seattle Police Department and recorded by the Traffic Records. A copy of the data file is stored and distributed as a learning material by the Cognitive Class organizing IBM Data Science online courses.
- The data preserves records of collision accidents in Seattle, Washington from 2004 to 2020.
- The information about each collision accident is recorded in a tabular format which includes information about the severity of the accident, the location coordinates, the weather and road condition, and some codes to simplify further details about the accident using a predefined code convention explained in the metadata PDF file.
- The 194,673 accident records will be assessed using exploratory data analysis to gain insight into the accident statistics in Seattle, such as the correlation between the severity of the accidents and the other features of the accidents.

## 3. Methodology [[ToC]](#ToC)

### 3.1. Data Preprocessing [[ToC]](#ToC)

Before performing any exploratory data analysis techniques, the data is preprocessed first. The preprocessing steps consist of:
* _data importing_

    The data is imported into one of suitable formats which is, in this case, DataFrame of Pandas.

* _data cleaning_

    The data is cleaned from any invalid or missing values by replacing them with mean or median values and removing the entire row (entity) or column (feature) if there is too many invalid or missing values. The formats of the values for each feature are also checked and corrected to ensure that it is consistent and ready to be processed.

For classification and clustering, additional two steps of the data preprocesing are performed: to choose which features to be analysed and to normalize the values of the chosen features. Further detail about the data preprocessing for classification processes is provided in the [section 3.3.1](#3.3.1).

### 3.2. Exploratory Data Analysis and Data Visualization [[ToC]](#ToC)

The following is the list of the exploratory data analysis performed and discussed on the accident data:
1. descriptive statistics,
2. grouping data into categories:
    2.1. visualized using waffle charts,
    2.2. visualized using multiple-classes bar charts,
    2.3. visualized using time-series line charts,
3. ANOVA, and
4. Correlation

In the descriptive statistics analysis, the data is grouped by its features and the unique values of the grouped data are counted.

Additionally, the accident data is visualized in interactive maps using Folium package. The interactive maps include:
1. a marker-cluster map which shows the numbers of the accidents in 2019 grouped by the relative distance and the map-zooming level, and
2. some choropleth maps which shows the numbers and the density of the accidents in 2009 and 2019 grouped by the district and neighborhood of Seattle.

### 3.3. Accident Severity Classification [[ToC]](#ToC)

#### 3.3.1. Data Preprocessing for Classification [[ToC]](#ToC)

The data is preprocessed with the following steps:
1. *one hot encoding*, to convert any categorical features into binary value sets,
2. *dataset choosing*, to choose the target classes and which years of the data are used to filter the training dataset,
3. *data cleaning*, to manipulate invalid data by removing them (the rows with NaN data) from the dataset,
4. *data balancing*, to balance the numbers of the positive and negative values of the target data,
5. *standard scaling*, to scale the values of the dataset so that each feature has similar range of values, and
6. *training-test set splitting*, to split the dataset into training and test set to enable hyperparametrization of the models.

The target classes of the classification models are the severity code (1: accident with property damage, 2: accident with human injury). There are 37 features included in the models:
* 3 location types:
    * alley, block, intersection
* 6 junction types:
    * at intersection (but not related to intersection), at intersection (intersection related), driveway junction, mid-block (but intersection related), mid-block (not related to intersection), ramp junction
* if caused by inattention
* if under alcohol influence
* 9 weather conditions:
    * blowing sand/dirt, clear, fog/smog/smoke, overcast, partly cloudy, raining, severe crosswind, sleet/hail/freezing rain, snowing
* 7 road conditions:
    * dry, ice, oil, sand/mud/dirt, snow/slush, standing water, wet
* 7 lighting condition:
    * no street lights, street lights off, street lights on, unknown lighting, dawn, daylight, dusk
* if pedestrian rights are not granted
* if speeding
* if hit parked car

For the training process, the dataset from Jan 2004 until Dec 2019 are used. Since the number of positive and negative values of the target classes are imbalance (without injury: 109375 cases, with injury: 54169 cases), we balance them by down-sampling the majority cases into 54169 cases which is equal to the number of the minority cases. The data is then splitted 80:20 as the training and test dataset. For the final evaluation, the models will be tested on the remaining dataset from Jan-May 2020.

#### 3.3.2. Classification Models [[ToC]](#ToC)

There are 4 classification models trained with the data to predict the severity of an accident:
1. k-nearest neighbors (KNN),
2. decision tree,
3. support vector machine (SVM), and
4. logistic regression.

For KNN and decision tree, the hyperparameters _k_ in KNN and _maximum depth_ in decision tree are parameterized to get the best performing models on the test set.

#### 3.3.3. Metrics for Model Evaluation [[ToC]](#ToC)

For KNN and decision tree, the hyperparameters _k_ in KNN and _maximum depth_ in decision tree are parameterized by evaluating the Jaccard similarity indexes to get the best performing models on the test set. For the SVM model, confusion matrix is generated to visualize the precision and recall distribution. In the end, to compare the classification models' performance, the Jaccard similarity index, F1-score, and logistic loss are calculated. The logistic loss is only calculated for the logistic regression model.

Jaccard similarity index is calculated by [(1)](#eq1), whereas F1 score is calculated by harmonic-averaging the precision score and the recall score as described in [(2.a-c)](#eq2). Log loss  is calculated by [(3)](#eq3) but only used for evaluating the logistic regression model.

\begin{equation}\tag{1}
    J(y,\widehat{y})=\frac{|y \cap \widehat{y}|}{|y \cup \widehat{y}|}
\end{equation}

\begin{equation}\tag{2.a}
    Precision = \frac{True Positive}{True Positive + False Positive}
\end{equation}

\begin{equation}\tag{2.b}
    Recall = \frac{True Positive}{True Positive + False Negative}
\end{equation}

\begin{equation}\tag{2.c}
    F_1 = \frac{2 \cdot Precision \cdot Recall}{Precision + Recall}
\end{equation}

\begin{equation}\tag{3}
    LogLoss = -\frac{1}{n}\sum (y \times log(\hat{y}) + (1-y) \times log(1-\hat{y}))
\end{equation}

## 4. Results and Discussion [[ToC]](#ToC)

### 4.1. Exploratory Data Analysis [[ToC]](#ToC)

#### 4.1.1. Descriptive Statistics Analysis [[ToC]](#ToC)

In the [Table 1](#t1), a summary of the descriptive statistics analysis of the data is provided. Only features containing integer or float dtypes are included. The first row represents the counts of the non-NaN data for each features. The rest represent the average, standard deviation, minimum, three quartiles, and maximum, respectively.

 Table 1. descriptive statistics of integer and float dtypes features [[LoT]](#LoT)

| index   |   SEVERITYCODE |              X |              Y |   INTKEY |   PERSONCOUNT |       PEDCOUNT |    PEDCYLCOUNT |      VEHCOUNT |   SDOT_COLCODE_A |   SDOT_COLCODE_B |   SEGLANEKEY |    CROSSWALKKEY |
|:--------|---------------:|---------------:|---------------:|---------:|--------------:|---------------:|---------------:|--------------:|-----------------:|-----------------:|-------------:|----------------:|
| count   |  194673        | 189339         | 189339         |  65070   |  194673       | 194673         | 194673         | 194673        |    194673        |     194673       |   194673     | 194673          |
| mean    |       1.2989   |   -122.331     |     47.6195    |  37558.5 |       2.44443 |      0.0371392 |      0.0283912 |      1.92078  |         1.25328  |          2.84164 |      269.401 |   9782.45       |
| std     |       0.457778 |      0.0299761 |      0.0561566 |  51746   |       1.34593 |      0.19815   |      0.167413  |      0.631047 |         0.763915 |          2.21872 |     3315.78  |  72269.3        |
| min     |       1        |   -122.419     |     47.4956    |  23807   |       0       |      0         |      0         |      0        |         0        |          0       |        0     |      0          |
| 25%     |       1        |   -122.349     |     47.576     |  28667   |       2       |      0         |      0         |      2        |         1        |          1       |        0     |      0          |
| 50%     |       1        |   -122.33      |     47.6154    |  29973   |       2       |      0         |      0         |      2        |         1        |          2       |        0     |      0          |
| 75%     |       2        |   -122.312     |     47.6637    |  33973   |       3       |      0         |      0         |      2        |         1        |          4       |        0     |      0          |
| max     |       2        |   -122.239     |     47.7341    | 757580   |      81       |      6         |      2         |     12        |         7        |          9       |   525241     |      5.2397e+06 |

The output of this descriptive statistics analysis is a good quick overview of the available data. Some useful information that can be taken from this analysis are the average number of involved persons, pedestrians, cyclists, and vehicles.

The average values of the coordinates (the longitude, X and the latitude, Y) don't give enough insights about the data. The coordinates are better to be visualized in a map. Some map visualizations of the data are available in [Section 4.1.5](#4.1.5). There are some data points that don't have coordinate data. In that case, the data points are excluded in the visualization.

To gain more insightful information, the data about the accidents are categorized and visualized using waffle charts in the next section (see [Section 4.1.2](#4.1.2)). Further analysis using ANOVA and Correlation analysis are also provided in [Section 4.1.3](#4.1.3) and [Section 4.1.4](#4.1.4) respectively.

#### 4.1.2. Grouping Data into Categories [[ToC]](#ToC)

For categorical features, useful insights can be extracted by summarizing and visualizing the data based on the counts of its unique values. Some categorical features are summarized in Table 2-3.

 Table 2. The numbers of no injury (1) and with injury (2) accidents [[LoT]](#LoT)

|   index |   SEVERITYCODE |
|--------:|---------------:|
|       1 |         136485 |
|       2 |          58188 |

 Table 3. The numbers of accidents based on the accident location type [[LoT]](#LoT)

| index        |   ADDRTYPE |
|:-------------|-----------:|
| Block        |     126926 |
| Intersection |      65070 |
| nan          |       1926 |
| Alley        |        751 |

#### 4.1.3. Waffle Charts of Categorical Features [[ToC]](#ToC)

The counts of unique values for other features can also be extracted, but the outputs are too exhaustive. A better way to present them is by visualizing them. One of interesting ways to visualize categorical data is using waffle chart. Unlike bar or pie chart, we could visualize the proportions of data categories using symbolic images. This method helps the readers getting the information about the proportions and the categories each of them represents at the same time.

In Figure 1 (a-g), the waffle charts for seven categorical features are presented. Each waffle chart is visualized by 5x20 blocks of symbols. Each block represents about 1% of the total proportions of the feature. Exact numbers inside the parentheses in each legend represent the number of the unique value of each category.

  (a)
  (b)

  (c)
  (d)

  (e)
  (f)

  (g)

Figure 1. The waffle charts for seven data features [[LoF]](#LoF)

Based on Figure 1, we could summarize that most of the cases are dominated by the accidents without any human injuries (up to 60%), on a street block (up to 65%), involved a parked car (up to 25%), on mid-block (up to 46%), during clear weather (up to 57%), on dry road (up to 64%), and during daylight (up to 60%). Unfortunately, some of the unique values in these categorical features can not be compared directly as it is more proper to normalize it into a fair scale, e.g. numbers of accidents on clear weather vs raining can only be compared after dividing the number with the ratio of the frequency of clear weather vs raining in the range of the time. However, this issue is out of the scope of this article and may be covered in future project.

#### 4.1.4. Weather and Road Condition Effects on the Number and the Severity of the Accidents [[ToC]](#ToC)

Although the waffle charts are visually interesting, it is not the best choice to show multiple categorical data at once. For that purpose, we use multiple colored bar charts. [Figure 2 (a-b)](#f2) show the numbers of accidents grouped by weather/road condition and severity at the same time on each chart. In order to also show the low numbers of the minority cases, the charts are also presented in logistic scale as shown in [Figure 2 (c-d)](#f2c).

  (a)
  (b)

  (c)
  (d)

Figure 2. The numbers of accidents by weather/road condition and its severity [[LoF]](#LoF)

It has been shown in the waffle charts that most of the accidents happened during clear weather and on dry road. Based on these bar charts, we get more insights about how often accidents that happened on a certain weather and road condition resulted in human injuries. Despite of the facts mentioned before, the accidents that happened during raining weather or on wet road has a slightly higher probability of resulting in injury (33.2%-33.7%) compared to the accidents that happened during clear weather or on dry road (around 32.2%)

#### 4.1.5. Time-Series Data [[ToC]](#ToC)

It is also interesting to visualize the data into time-series format. In [Figure 3](#f3), the data is grouped into monthly and quarterly numbers of the accidents from January 2004 to May 2020. The accident numbers are also categorized by its severity. The percentage of accidents with injury is also presented and projected to the secondary axis of the charts.

  (a)
  (b)

Figure 3. The time-series data of the monthly (a) and quarterly (b) numbers of accidents [[LoF]](#LoF)

Time-series data is useful to gain insights about the trends of the data over time. Figure 3 shows that there is a slight declining trend of the numbers of the accidents. There is even a significant decrease on the second quarter of 2020 because of the lockdown policy across the globe during Covid-19 pandemic.

#### 4.1.6. Analysis of Variance (ANOVA) [[ToC]](#ToC)

The analysis of variance (ANOVA) of the accident severity is performed on some features of weather: Clear, Raining, and Overcast.  Here are the summary of the ANOVA results (higher F-value is better, lower P-value is better):
* ANOVA of the accident severity between clear, raining, and overcast results: F= 18.236826341147676 , P = 1.2041667644621963e-08
* ANOVA of the accident severity between clear and raining results: F= 25.09876170660936 , P = 5.453197502895751e-07
* ANOVA of the accident severity between clear and overcast results: F= 4.909741750003867 , P = 0.026707249184972547
* ANOVA of the accident severity between raining and overcast results: F= 32.11870221519134 , P = 1.4568807019555952e-08

These results indicate that it is quite likely that the accident severity has a correlation with the weather during the accident. However, for the low variance between the accident severity of accident during clear weather and overcast, it could indicate that these two weather types lead to similar probabilities of a certain accident severity. In this case, it could also be an option to combine the clear and overcast data into one feature, but it is out of the scope of this article.

#### 4.1.7. Correlation Analysis [[ToC]](#ToC)

A correlation analysis between the accident severity and the features of weather. The correlation coefficients between each feature are calculated and compiled as a heat map as shown in [Figure 4](#f4). However, this result of correlation analysis is still raw as there are too many other features that are neglected in the analysis. A more robust correlation analysis of categorical features can be done by building classification models which is a part of [Section 4.2](#4.2).

Figure 4. The heatmap of the correlation among the accident severity and the features of weather [[LoF]](#LoF)

#### 4.1.8. Map Visualization [[ToC]](#ToC)

Since the dataset also records the longitude (X), the latitude (Y), and the date of each accident, it is possible to present the data using map visualization. By using map visualization, we could gain insights which location need more attention so that policy making can plan, execute, and evaluate its policy more effectively.

The numbers of accidents (excluding the cases without XY data) between 2004-2020 are grouped by its district (see [Table 4](#t4))  and visualized using choropleth map (see [Figure 5](#f5)). The geojson data of Seattle districts is taken from [github.com/seattleio/seattle-boundaries-data](https://github.com/seattleio/seattle-boundaries-data/blob/master/data/city-council-districts.geojson).

 Table 4. The numbers of accidents between 2004-2020 by district [[LoT]](#LoT)

| District        |   Number of Cases |
|-----------:|---------------:|
|          2 |          39198 |
|          7 |          37047 |
|          3 |          30195 |
|          6 |          21235 |
|          1 |          20276 |
|          4 |          20274 |
|          5 |          19832 |

  (a)
  (b)

Figure 5. The choropleth map of the numbers of accidents by its district (a: district number from WikiMedia, b: choropleth map) [[LoF]](#LoF)

_Acknowledgement: Figure 6(a) is credited to Dennis Bratland / CC BY-SA (https://creativecommons.org/licenses/by-sa/4.0)_

In order to be more precise, the numbers of cases distribution can also be grouped by its neighborhood as visualized in [Figure 6](#f6). The geojson data of Seattle neighborhoods is taken from [github.com/seattleio/seattle-boundaries-data](https://github.com/seattleio/seattle-boundaries-data/blob/master/data/city-council-districts.geojson).

  (a)
  (b)

Figure 6. The choropleth map of the numbers of accidents by its neighborhood (a: 2009, b: 2019) [[LoF]](#LoF)

Based on the number of cases per year, Belltown has the highest number in 2009 with 539 cases but then reduced to 394 cases in 2019. Industrial district has 457 cases in 2009 but then increased to 500 cases in 2019. However, comparing the numbers of cases between neighborhood directly is unfair as each neighborhood has its own traffic number or traffic density.

Besides normalized by its traffic density (whose data might be not easily obtained), the numbers of cases can also be normalized by its neighborhood area. The area of each neighborhood is provided in the geojson data of Seattle neighborhoods. The numbers of cases per neighborhood area are visualized in [Figure 7](#f7).

  (a)
  (b)

Figure 7. The choropleth map of the numbers of accidents per neighborhood area (a: 2009, b: 2019) [[LoF]](#LoF)

Based on the density of cases per year, Central Business District has the highest density in 2009 and 2019 with 45.3 and 43.7 cases per km2 per year, respectively. It is shown that some neighborhoods have slightly reduced its number or density of cases from 2009 to 2019.

From these map visualizations, there are many findings that may help the Seattle Government make its policy and execute traffic law enforcement in effective way by prioritizing which locations that need more attention.

### 4.2. Accident Severity Classification [[ToC]](#ToC)

In order to help the Seattle government in planning, executing, and evaluating its policy in decreasing the number and the severity of accidents, it's recommended to build classification models for predicting the severity of an accident based on several features. This could help the government make a priority in preventing any severe accidents based on some most frequent features that could lead to the accident.

#### 4.2.1. K-Nearest Neighbors (KNN) [[ToC]](#ToC)

In the K-Nearest Neighbors model, the target classes (accident severity) are predicted by sampling k number of nearest neighbors in the existing data and picking the majority class of the sample as the predicted target class. The value k is the hyperparameter which can be adjusted and tested which value results in higher classification accuracy. [Figure 8](#f8) shows the model accuracies for various value of k. It is shown that k=9 leads to the highest KNN model accuracy with 0.583 Jaccard similarity index on the test set.

Figure 8. The Jaccard similarity scores for various values of k in the KNN model [[LoF]](#LoF)

#### 4.2.2. Decision Tree [[ToC]](#ToC)

In the decision tree model, the target classes (accident severity) are predicted by building a top-down hierarchical tree which split the data based on a threshold to a certain value of a feature that leads to a lower total entropy of the data classification. The value maximum depth is the hyperparameter which can be adjusted and tested which value results in higher classification accuracy. [Figure 9](#f9) shows the model accuracies for various value of maximum depth.

Figure 9. The Jaccard similarity scores for various values of maximum depth in the decision tree model [[LoF]](#LoF)

It is shown that maximum depth=5 leads to the highest decision tree model accuracy with 0.599 Jaccard similarity index on the test set. The hierarchy of the decision tree with maximum depth of five is shown in [Figure 10](#f10). The boxes with blue color represent the data classified as accidents with injury, whereas red color as accidents without injury. The sharper the color of the box is, the lower the entropy of the data is.

Figure 10. The decision tree model with maximum depth of five [[LoF]](#LoF)

#### 4.2.3. Support Vector Machine (SVM) [[ToC]](#ToC)

In the SVM model, the target classes (accident severity) are predicted by creating a linear or non-linear line which separates two regions of different classes in a high-dimensional feature space. The kernelling used in the model is radial basis function (RBF). No hyperparameter of the SVM model is adjusted. The numbers of the true and the predicted positive and negative values are illustrated in the confusion matrix as shown in [Figure 11](#f11). The SVM model has accuracy of 0.598 Jaccard similarity index on the test set.

Figure 11. The confusion matrix for the support vector machine model on the test dataset [[LoF]](#LoF)

#### 4.2.4. Logistic Regression [[ToC]](#ToC)

In the logistic regression model, the target classes (accident severity) are predicted by estimating the probability of a target class using a sigmoid or logistic function. The parameters of the sigmoid function are parameterized to minimize the cost function which represent the loss of classification accuracy. The changes of the parameters in the iterative process are regularized by penalizing large magnitude of changes. The strength of the regularization is represented by the inverse regularization factor, C. The lower value of C, the stronger the regularization is. In this article, we only use one value of inverse regularization factor, C=0.1. This logistic regression model has accuracy of 0.599 Jaccard similarity index on the test set.

#### 4.2.5. Model Evaluation using the Dataset of 2020 [[ToC]](#ToC)

In the final stage of the classification model building process, the models are evaluated against an unseen test dataset. This time, we pick the dataset from Jan-May 2020 as the final test dataset. The dataset consists of 1182 data points with 808 cases of accidents without injury and 374 cases of accidents with injury. The evaluation metrics assessed are F1-score, Jaccard similarity index, and log loss as summarized in [Table 5](#t5).

 Table 5. The evaluation metrics for each classification model [[LoT]](#LoT)

| index               |   F1 score |   Jaccard similarity index |         Log loss |
|:--------------------|-----------:|---------------------------:|-----------------:|
| KNN                 |   0.522397 |                   0.506768 |       nan        |
| Decision Tree       |   0.632704 |                   0.620135 |       nan        |
| SVM                 |   0.63095  |                   0.618443 |       nan        |
| Logistic Regression |   0.621832 |                   0.608291 |         0.651837 |

Based on its F1 score and Jaccard similarity index, the decision tree model performs best compared to the others. In order to visualize the evaluation metrics, the confusion matrices for each model are presented in [Figure 12](#f12). Since the number of postive and negative target values are imbalanced, it seems that all models return low precision scores. In this case, it is recommended to use F1 score as the evaluation metric for classification with imbalanced target dataset.

  (a)
  (b)

  (c)
  (d)

Figure 12. The confusion matrices for all models on the final test dataset [[LoF]](#LoF)

#### 4.2.6. The Significant Features Leading to Accidents with Injury based on the Classification Model [[ToC]](#ToC)

It has been shown that the decision tree model performs best in classifying accidents by its severity based on several features. In this section, we are going to discuss about the significant features which lead to accidents with injury based on the decision tree model.

In [Figure 6], we find that the first branch of the decision tree is the feature of PEDROWNOTGRNT which according to the metadata pdf file refers to the fact if pedestrian rights were not granted. Since the values of the features are standard-scaled, the binary values of 0 or 1 are scaled slightly, e.g. in case of PEDROWNOTGRNT, the values of 0 and 1 are converted into -0.201025 and 4.974507, respectively. Thus, the criteria "PEDROWNOTGRNT ≤ 2.387" is equivalent to "is Pedestrian rights were granted?". Branching to the left means the criteria is true, whereas branching to the right means the criteria is false. A box with sharp blue color on the right side after the first branch means that this feature significantly classify the cases where pedestrian rights were not granted as the cases with injury. This finding could be an encouragement for the Seattle government to enforce the traffic laws on putting pedestrian rights as priority to minimize accidents with injury.

There is no other single significant feature, but rather there are some combinations of features that lead to accident with injuries. Here are the list sorted by its entropy in ascending order (lower entropy means more accurate classification):
* (Entropy: 0.269) Pedestrian rights were not granted: True
* (Entropy: 0.907) Junction: At intersection; Under alcohol influence: True
* (Entropy: 0.951) Junction: Mid-block; Hit parked car: False; Speeding: True; Under alcohol influence: True

Based on this finding, we could add the features of under alcohol influence and of speeding as the other significant features that lead to accidents with injury besides the feature of the pedestrian rights were not granted. The cases with those combinations of features can be visualized in maps with marker-clusters as presented in [Figure 13](#f13), [14](#f14), and [15](#f15). These maps can be used strategically as references for the authority to choose which locations are to be put as the priority in the next policy making or law enforcement by the government.

  (a)
  (b)

  (c)
  (d)

Figure 13. The maps with marker-clusters for cases where pedestrian rights were not granted (a to d zoom level = 11, 13, 15, 17 respectively) [[LoF]](#LoF)

  (a)
  (b)

  (c)
  (d)

Figure 14. The maps with marker-clusters for cases at intersection and under alcohol influence (a to d zoom level = 11, 13, 15, 17 respectively) [[LoF]](#LoF)

  (a)
  (b)

  (c)
  (d)

Figure 15. The maps with marker-clusters for cases on mid-block, speeding, and under alcohol influence (a to d zoom level = 11, 13, 15, 17 respectively) [[LoF]](#LoF)

## 5. Conclusion and Remarks [[ToC]](#ToC)

In this article, some exploratory data analyses have been done and classification models have been trained on a dataset containing 194,673 accident cases in Seattle from Jan 2004 to May 2020. From the exploratory data analyses, some insights have been gained. Some of them are:
* the accidents that happened during raining weather or on wet road has a slightly higher probability of resulting in injury (33.2%-33.7%) compared to the accidents that happened during clear weather or on dry road (around 32.2%);
* there is a significant decrease of numbers of accidents on the second quarter of 2020 because of the lockdown policy across the globe Covid-19 pandemic;
* it is quite likely that the accident severity has a correlation with the weather during the accident;
* Belltown has the highest number in 2009 with 539 cases but then reduced to 394 cases in 2019;
* The industrial district has 457 cases in 2009 but then increased to 500 cases in 2019;
* Central Business District has the highest number of cases per area in 2009 and 2019 with 45.3 and 43.7 cases per km2 per year, respectively.
* Some neighborhoods have slightly reduced its number or density of cases from 2009 to 2019.

From four classification model trained, the decision tree model perform best. Based on the decision tree model, we gain insights about some significant features that lead to accidents with injury. Those are: when the pedestrian rights were not granted, and/or speeding, and/or under alcohol influence.

There are still some assumptions, limitations, and issues that have not been addressed in or are out of the scope of this article. There are also still many alternative options of classification models available. So, any interested readers are encouraged to improve the works presented here and to help the Seattle government fulfill its missions towards zero collision accident in future.


```python
%matplotlib inline
```

Hello everyone! My name is Amit Saha and today I am here to talk to about "Doing Math with Python".

Thank you for coming to my talk - i know you could have chosen the other talk, so it's good to know that my talk's topic interests you.

A bit about me - I am a software engineer at Freelancer.com in Sydney Australia.  I am a fairly regular writer for Linux Voice and other Linux magazines. And last, but not the least, I am the author of the book "Doing Math with Python" (not coincidentally titled the same as this talk - haha) published by No Starch Press in 2015.

There is a link to my blog, GitHub, twitter, etc. so, if you want to learn more about my work or get in touch, those are the means to do so!

Okay, so all that aside - let's start with the talk!

#  My first lab

###  Kirstie Whitaker

####  June 15th 2017

####  Cape Town, SA

## About me

- 2016/17 Mozilla Fellow for Science

- Research Fellow at the Alan Turing Institute for Data Science, London

- Research Associate in the Department of Psychiatry at the University of Cambridge

### Contact

- Twitter: [@kirstie_j](http://twitter.com/kirstie_j)

- Email: [kw401@cam.ac.uk](mailto:kw401@cam.ac.uk)

- GitHub: [KirstieJane](http://github.com/KirstieJane)

Hakim El Hattab

Hello!

So, what am I selling to you today? Not my book (I am, but in a subtle way). I am presenting an idea, a hypothesis or even making a statement - Python can lead to a more enriching learning and teaching experience in the classroom.

Let me explain where I am coming from. When I think back about when I was learning to program and learning all other subjects in standards 7-10. I think it's true today as well. Programming and other subjects such as Math, Science are taught in a disconnected fashion. Programming seems to be all about finding the sum of a series or generating fibonacci numbers. Make no mistake, these exercises are what builds up the programming logic. Some students get really excited about being able to do these, but a lot of them don't.  It's a lot like not everyone gets interested in solving puzzles - i don't, i never took to them.

I think I know of a way we could excite more students! Show them how you can write programs to do your homework, or experiment without having to go the science lab or setup elaborate experimental setups. This is my goal for today - in the following slides and notebooks, I will hypothesise on a way of connecting Python programming and other subjects. That will show that programming is a way to get real work done, not something to learn for the sake of it.

We need some tools to help us on our quest. The Python community has some giant shoulders we can stand upon - Python 3, SymPy and matplotlib.

### This talk - a proposal, a hypothesis, a statement

What? *Python can lead to a more enriching learning and teaching experience in the classroom*

How? *Next slides*

###  Tools (or, Giant shoulders we will stand on)

*Python 3*, *SymPy*, *matplotlib*

*Individual logos are copyright of the respective projects. [Source](http://www.orphancaremovement.org/standing-on-the-shoulder-of-giants/) of the "giant shoulders" image.

Whose calculator looks like this?

Who uses Python as a calculator? Raise of hands please!

I do! Specifically, I use Python 3 because of 1/2=0 messes up my monthly expenditure calculation.

Besides the usual addition and subtraction, we have of course the math module and more recently the statistics module which makes Python a worthy scientific calculator.

But then, there's more! You are not limited to the functions from those libraries, you can write your own custom functions and make them available whenever you start your Python interpreter. How?

Use PYTHONSTARTUP!

### Python - a scientific calculator

Whose calculator looks like this?

```python
>>> (131 + 21.5 + 100.2 + 88.7 + 99.5 + 100.5 + 200.5)/4
185.475
```

*Python 3 is my favorite calculator (not Python 2 because 1/2 = 0)*

Beyond basic operations:

- `fabs()`, `abs()`, `sin()`, `cos()`, `gcd()`, `log()` and more (See [math](https://docs.python.org/3/library/math.html))

- Descriptive statistics (See [statistics](https://docs.python.org/3/library/statistics.html#module-statistics))

### Python - a scientific calculator

- Develop your own functions: unit conversion, finding correlation, .., anything really

- Use PYTHONSTARTUP to extend the battery of readily available mathematical functions

```python
$ PYTHONSTARTUP=~/work/dmwp/pycon-us-2016/startup_math.py idle3 -s
```

### Unit conversion functions

```python

>>> unit_conversion()
1. Kilometers to Miles
2. Miles to Kilometers
3. Kilograms to Pounds
4. Pounds to Kilograms
5. Celsius to Fahrenheit
6. Fahrenheit to Celsius
Which conversion would you like to do? 6
Enter temperature in fahrenheit: 98
Temperature in celsius: 36.66666666666667
>>>

```

### Finding linear correlation

```python
>>>
>>> x = [1, 2, 3, 4]
>>> y = [2, 4, 6.1, 7.9]
>>> find_corr_x_y(x, y)
0.9995411791453812

```

So, that was Python and it's standard libraries. When you bring in third party libraries to the mix, Python becomes a seriously fancy calculator.

Who has heard about SymPy?

You can give it algebraic expressions to a function and a graph will be created for you.

You can give an equation and out comes the solutions for that equation.

We can even solve calculus problems.

### Python - a really fancy calculator

SymPy - a pure Python symbolic math library

*from sympy import awesomeness*  - don't try that :)

```python
# Create graphs from algebraic expressions

from sympy import Symbol, plot
x = Symbol('x')
p = plot(2*x**2 + 2*x + 2)

```

```python
# Solve equations

from sympy import solve, Symbol
x = Symbol('x')
solve(2*x + 1)

```

    [-1/2]

```python
# Limits

from sympy import Symbol, Limit, sin
x = Symbol('x')
Limit(sin(x)/x, x, 0).doit()

```

    1

```python
# Derivative

from sympy import Symbol, Derivative, sin, init_printing
x = Symbol('x')
init_printing()
Derivative(sin(x)**(2*x+1), x).doit()
```

```python
# Indefinite integral

from sympy import Symbol, Integral, sqrt, sin, init_printing
x = Symbol('x')
init_printing()
Integral(sqrt(x)).doit()
```

```python
# Definite integral

from sympy import Symbol, Integral, sqrt
x = Symbol('x')
Integral(sqrt(x), (x, 0, 2)).doit()
```

I will pause for a moment now. In the first two slides, we have seen how Python can be a super awesome calculator. What does that buy us? We have now been able to show that you can make computer programs literally do your homework. Write a program to do your work once and you will never have to make those lengthy calculations yourselves. Can we use Python to do more?

Let's continue.

Can we do more than write smart calculators?

Python can be more than a super powerful calculator. We can use it to enhance the learning experience of other subjects. Next, I have three examples including a demo. First up, a video of a projectile motion. This program uses matplotlib's animation API to create a basic animation of a projectile motion - a fairly common subject introduced in introductory Physics. The program which is linked asks for the angle of projection and speed and then draws the trajectory of the projectile. Just by running the program multiple times, we can see how the trajectory changes. We don't have to go outside and start throwing balls..

Next, we will put Jupyter Notebook's interactive widgets to good effect by drawing a Barnsley Fern. Let's see how the demo goes.

Next, with the help of basemap, we can draw places on a world map like we would draw points on a graph paper.

I know I would be excited if someone was showing me all these cool things when I was learning these things!

### Python - Making other subjects more lively

- matplotlib

- basemap

- Interactive Jupyter Notebooks

#### Bringing Science to life

*Animation of a Projectile motion* [(Python Source)](https://github.com/doingmathwithpython/pycon-us-2016/blob/master/py-files/projectile_animation.py)

```python
from IPython.display import YouTubeVideo
YouTubeVideo("8uWRVh58KdQ")
```

#### Exploring Fractals in Nature

*Interactively drawing a Barnsley Fern* [(Notebook)](https://github.com/doingmathwithpython/pycon-us-2016/blob/master/notebooks/Interactive%20Barnsley%20Fern.ipynb)

#### The world is your graph paper

*Showing places on a digital map* [(Notebook)](https://github.com/doingmathwithpython/pycon-us-2016/blob/master/notebooks/Maps%20using%20Basemap%20-%20demo.ipynb)

Next, I would like to talk about my book "Doing Math with Python". My idea was attractive enough to get it published by No Starch Press which makes me hope that I am probably onto something.

Has anybody read my book? What do you think of it? You have read it and came to my talk? I am feeling better :)

I discuss all of the topics I discuss today in my talk. In addition, I discuss sets, probability and random numbers and descriptive statistics.

It's being translated into several non-English languages.

The reviews/feedback so far has been really positive. I don't have any first hand involvement in teaching, so it's very appreciative of people to share their viewpoints with me.

### Book: Doing Math With Python

Overview

- All of what I have discussed so far

- In addition: Descriptive statistics, Sets and Probability, Random numbers

Published by [No Starch Press](https://www.nostarch.com/doingmathwithpython) in August, 2015.

*Upcoming/In-progress translations*: Simplified Chinese, Japanese, French and Korean.

#### Comments

> Saha does an excellent job providing a clear link between Python and upper-level math concepts, and demonstrates how Python can be transformed into a mathematical stage. This book deserves a spot on every geometry teacher’s bookshelf.

[School Library Journal](http://www.slj.com/2016/05/collection-development/read-watch-alikes/coding-lets-begin/#_)

> Outstanding guide to using Python to do maths. Working back through my undergrad maths using Python.

> Saha does an excellent job providing a clear link between Python and upper-level math concepts, and demonstrates how Python can be transformed into a mathematical stage.

> This book is highly recommended for the high school or college student and anyone who is looking for a more natural way of programming math and scientific functions

> As a teacher I highly recommend this book as a way to work with someone in learning both math and programming

Okay, so that's great. We have successfully used Python to make the learning experience of young learners more fun and immediately applicable. Can we derive more benefit from doing that? Like something for the future? We all love doing things for the future, don't we?

I think yes, i think if we teach young learners the things we have discussed today, it is a great base for someone wanting to go into data science or machine learning.

Statistics and visualising data are two very key factors of data science.

Differential calculus and specifically the gradient descent method is a simple but useful optimization method used in Machine Learning. Let's see a demo of using gradient descent to find the minimum value of a function.

Now, let's apply gradient descent as an optimizer in a Linear Regression problem.

### Great base for the future

*Statistics and Graphing data* -> *Data Science*

*Differential Calculus* -> *Machine learning*

### Application of differentiation

Use gradient descent to find a function's minimum value [(Notebook)](https://github.com/doingmathwithpython/pycon-us-2016/blob/master/notebooks/Gradient%20Descent.ipynb)

### Predict the college admission score based on high school math score

Use gradient descent as the optimizer for single variable linear regression model [(Notebook)](https://github.com/doingmathwithpython/pycon-us-2016/blob/master/notebooks/Simple%20Linear%20Regression.ipynb)

### Advanced libraries

- [scipy](https://scipy.org)

- [numpy](http://www.numpy.org/)

- [scikit-learn](http://scikit-learn.org/stable/)

- [pandas](http://pandas.pydata.org/)

- [Statsmodels](http://statsmodels.sourceforge.net/)

### Dialogue

Questions, Thoughts, comments, discussions?

#### Online

- Twitter: @echorand

- Email: amitsaha.in@gmail.com

### PyCon Special!

*Use PYCONMATH code to get 30% off "Doing Math with Python" from [No Starch Press](https://www.nostarch.com/doingmathwithpython)*

(Valid from  May 26th - June 8th)

Book Signing - May 31st - 2.00 PM - No Starch Press booth

### Acknowledgements

PyCon US Education Summit team for inviting me

Thanks to PyCon US for reduced registration rates

Massive thanks to my employer, Freelancer.com for sponsoring my travel and stay

### Links

- [Upcoming O'Reilly Webcast](http://www.oreilly.com/pub/e/3712)

- [Doing Math with Python](https://nostarch.com/doingmathwithpython)

- [Doing Math with Python Blog](https://doingmathwithpython.github.io)

- [Doing Math with Python on GitHub](https://github.com/doingmathwithpython)


```python
%matplotlib inline
```

Generative Models of Graphs
===========================================

**Author**: `Mufei Li <https://github.com/mufeili>`_,
`Lingfan Yu <https://github.com/ylfdq1118>`_, Zheng Zhang

Warning

The tutorial aims at gaining insights into the paper, with code as a mean
    of explanation. The implementation thus is NOT optimized for running
    efficiency. For recommended implementation, please refer to the `official
    examples <https://github.com/dmlc/dgl/tree/master/examples>`_.

In this tutorial, you learn how to train and generate one graph at
a time. You also explore parallelism within the graph embedding operation, which is an
essential building block. The tutorial ends with a simple optimization that
delivers double the speed by batching across graphs.

Earlier tutorials showed how embedding a graph or
a node enables you to work on tasks such as `semi-supervised classification for nodes
<http://docs.dgl.ai/tutorials/models/1_gcn.html#sphx-glr-tutorials-models-1-gcn-py>`__
or `sentiment analysis
<http://docs.dgl.ai/tutorials/models/3_tree-lstm.html#sphx-glr-tutorials-models-3-tree-lstm-py>`__.
Wouldn't it be interesting to predict the future evolution of the graph and
perform the analysis iteratively?

To address the evolution of the graphs, you generate a variety of graph samples. In other words, you need
**generative models** of graphs. In-addition to learning
node and edge features, you would need to model the distribution of arbitrary graphs.
While general generative models can model the density function explicitly and
implicitly and generate samples at once or sequentially, you only focus
on explicit generative models for sequential generation here. Typical applications
include drug or materials discovery, chemical processes, or proteomics.

Introduction
--------------------
The primitive actions of mutating a graph in Deep Graph Library (DGL) are nothing more than ``add_nodes``
and ``add_edges``. That is, if you were to draw a circle of three nodes,

.. figure:: https://user-images.githubusercontent.com/19576924/48313438-78baf000-e5f7-11e8-931e-cd00ab34fa50.gif
   :alt:

you can write the code as follows.

```python
import dgl

g = dgl.DGLGraph()
g.add_nodes(1)              # Add node 0
g.add_nodes(1)              # Add node 1

# Edges in DGLGraph are directed by default.
# For undirected edges, add edges for both directions.
g.add_edges([1, 0], [0, 1]) # Add edges (1, 0), (0, 1)
g.add_nodes(1)              # Add node 2
g.add_edges([2, 1], [1, 2]) # Add edges (2, 1), (1, 2)
g.add_edges([2, 0], [0, 2]) # Add edges (2, 0), (0, 2)
```

Real-world graphs are much more complex. There are many families of graphs,
with different sizes, topologies, node types, edge types, and the possibility
of multigraphs. Besides, a same graph can be generated in many different
orders. Regardless, the generative process entails a few steps.

- Encode a changing graph.
- Perform actions stochastically.
- If you are training, collect error signals and optimize the model parameters.

When it comes to implementation, another important aspect is speed. How do you
parallelize the computation, given that generating a graph is fundamentally a
sequential process?

Note

To be sure, this is not necessarily a hard constraint. Subgraphs can be
   built in parallel and then get assembled. But we
   will restrict ourselves to the sequential processes for this tutorial.

DGMG: The main flow
--------------------
For this tutorial, you use
`Deep Generative Models of Graphs <https://arxiv.org/abs/1803.03324>`__
) (DGMG) to implement a graph generative model using DGL. Its algorithmic
framework is general but also challenging to parallelize.

Note

While it's possible for DGMG to handle complex graphs with typed nodes,
   typed edges, and multigraphs, here you use a simplified version of it
   for generating graph topologies.

DGMG generates a graph by following a state machine, which is basically a
two-level loop. Generate one node at a time and connect it to a subset of
the existing nodes, one at a time. This is similar to language modeling. The
generative process is an iterative one that emits one word or character or sentence
at a time, conditioned on the sequence generated so far.

At each time step, you either:
     - Add a new node to the graph
     - Select two existing nodes and add an edge between them

.. figure:: https://user-images.githubusercontent.com/19576924/48605003-7f11e900-e9b6-11e8-8880-87362348e154.png
   :alt:

The Python code will look as follows. In fact, this is *exactly* how inference
with DGMG is implemented in DGL.

```python
def forward_inference(self):
    stop = self.add_node_and_update()
    while (not stop) and (self.g.number_of_nodes() < self.v_max + 1):
        num_trials = 0
        to_add_edge = self.add_edge_or_not()
        while to_add_edge and (num_trials < self.g.number_of_nodes() - 1):
            self.choose_dest_and_update()
            num_trials += 1
            to_add_edge = self.add_edge_or_not()
        stop = self.add_node_and_update()

    return self.g
```

Assume you have a pre-trained model for generating cycles of nodes 10-20.
How does it generate a cycle on-the-fly during inference? Use the code below
to create an animation with your own model.

::

    import torch
    import matplotlib.animation as animation
    import matplotlib.pyplot as plt
    import networkx as nx
    from copy import deepcopy

    if __name__ == '__main__':
        # pre-trained model saved with path ./model.pth
        model = torch.load('./model.pth')
        model.eval()
        g = model()

        src_list = g.edges()[1]
        dest_list = g.edges()[0]

        evolution = []

        nx_g = nx.Graph()
        evolution.append(deepcopy(nx_g))

        for i in range(0, len(src_list), 2):
            src = src_list[i].item()
            dest = dest_list[i].item()
            if src not in nx_g.nodes():
                nx_g.add_node(src)
                evolution.append(deepcopy(nx_g))
            if dest not in nx_g.nodes():
                nx_g.add_node(dest)
                evolution.append(deepcopy(nx_g))
            nx_g.add_edges_from([(src, dest), (dest, src)])
            evolution.append(deepcopy(nx_g))

        def animate(i):
            ax.cla()
            g_t = evolution[i]
            nx.draw_circular(g_t, with_labels=True, ax=ax,
                             node_color=['#FEBD69'] * g_t.number_of_nodes())

        fig, ax = plt.subplots()
        ani = animation.FuncAnimation(fig, animate,
                                      frames=len(evolution),
                                      interval=600)

.. figure:: https://user-images.githubusercontent.com/19576924/48928548-2644d200-ef1b-11e8-8591-da93345382ad.gif
   :alt:

DGMG: Optimization objective
------------------------------
Similar to language modeling, DGMG trains the model with *behavior cloning*,
or *teacher forcing*. Assume for each graph there exists a sequence of
*oracle actions* $a_{1},\cdots,a_{T}$ that generates it. What the model
does is to follow these actions, compute the joint probabilities of such
action sequences, and maximize them.

By chain rule, the probability of taking $a_{1},\cdots,a_{T}$ is:

\begin{align}p(a_{1},\cdots, a_{T}) = p(a_{1})p(a_{2}|a_{1})\cdots p(a_{T}|a_{1},\cdots,a_{T-1}).\\\end{align}

The optimization objective is then simply the typical MLE loss:

\begin{align}-\log p(a_{1},\cdots,a_{T})=-\sum_{t=1}^{T}\log p(a_{t}|a_{1},\cdots, a_{t-1}).\\\end{align}

```python
def forward_train(self, actions):
    """
    - actions: list
        - Contains a_1, ..., a_T described above
    - self.prepare_for_train()
        - Initializes self.action_step to be 0, which will get
          incremented by 1 every time it is called.
        - Initializes objects recording log p(a_t|a_1,...a_{t-1})

    Returns
    -------
    - self.get_log_prob(): log p(a_1, ..., a_T)
    """
    self.prepare_for_train()

    stop = self.add_node_and_update(a=actions[self.action_step])
    while not stop:
        to_add_edge = self.add_edge_or_not(a=actions[self.action_step])
        while to_add_edge:
            self.choose_dest_and_update(a=actions[self.action_step])
            to_add_edge = self.add_edge_or_not(a=actions[self.action_step])
        stop = self.add_node_and_update(a=actions[self.action_step])

    return self.get_log_prob()
```

The key difference between ``forward_train`` and ``forward_inference`` is
that the training process takes oracle actions as input and returns log
probabilities for evaluating the loss.

DGMG: The implementation
--------------------------
The ``DGMG`` class
``````````````````````````
Below you can find the skeleton code for the model. You gradually
fill in the details for each function.

```python
import torch.nn as nn

class DGMGSkeleton(nn.Module):
    def __init__(self, v_max):
        """
        Parameters
        ----------
        v_max: int
            Max number of nodes considered
        """
        super(DGMGSkeleton, self).__init__()

        # Graph configuration
        self.v_max = v_max

    def add_node_and_update(self, a=None):
        """Decide if to add a new node.
        If a new node should be added, update the graph."""
        return NotImplementedError

    def add_edge_or_not(self, a=None):
        """Decide if a new edge should be added."""
        return NotImplementedError

    def choose_dest_and_update(self, a=None):
        """Choose destination and connect it to the latest node.
        Add edges for both directions and update the graph."""
        return NotImplementedError

    def forward_train(self, actions):
        """Forward at training time. It records the probability
        of generating a ground truth graph following the actions."""
        return NotImplementedError

    def forward_inference(self):
        """Forward at inference time.
        It generates graphs on the fly."""
        return NotImplementedError

    def forward(self, actions=None):
        # The graph you will work on
        self.g = dgl.DGLGraph()

        # If there are some features for nodes and edges,
        # zero tensors will be set for those of new nodes and edges.
        self.g.set_n_initializer(dgl.frame.zero_initializer)
        self.g.set_e_initializer(dgl.frame.zero_initializer)

        if self.training:
            return self.forward_train(actions=actions)
        else:
            return self.forward_inference()
```

Encoding a dynamic graph
``````````````````````````
All the actions generating a graph are sampled from probability
distributions. In order to do that, you project the structured data,
namely the graph, onto an Euclidean space. The challenge is that such
process, called *embedding*, needs to be repeated as the graphs mutate.

Graph embedding
''''''''''''''''''''''''''
Let $G=(V,E)$ be an arbitrary graph. Each node $v$ has an
embedding vector $\textbf{h}_{v} \in \mathbb{R}^{n}$. Similarly,
the graph has an embedding vector $\textbf{h}_{G} \in \mathbb{R}^{k}$.
Typically, $k > n$ since a graph contains more information than
an individual node.

The graph embedding is a weighted sum of node embeddings under a linear
transformation:

\begin{align}\textbf{h}_{G} =\sum_{v\in V}\text{Sigmoid}(g_m(\textbf{h}_{v}))f_{m}(\textbf{h}_{v}),\\\end{align}

The first term, $\text{Sigmoid}(g_m(\textbf{h}_{v}))$, computes a
gating function and can be thought of as how much the overall graph embedding
attends on each node. The second term $f_{m}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{k}$
maps the node embeddings to the space of graph embeddings.

Implement graph embedding as a ``GraphEmbed`` class.

```python
import torch

class GraphEmbed(nn.Module):
    def __init__(self, node_hidden_size):
        super(GraphEmbed, self).__init__()

        # Setting from the paper
        self.graph_hidden_size = 2 * node_hidden_size

        # Embed graphs
        self.node_gating = nn.Sequential(
            nn.Linear(node_hidden_size, 1),
            nn.Sigmoid()
        )
        self.node_to_graph = nn.Linear(node_hidden_size,
                                       self.graph_hidden_size)

    def forward(self, g):
        if g.number_of_nodes() == 0:
            return torch.zeros(1, self.graph_hidden_size)
        else:
            # Node features are stored as hv in ndata.
            hvs = g.ndata['hv']
            return (self.node_gating(hvs) *
                    self.node_to_graph(hvs)).sum(0, keepdim=True)
```

Update node embeddings via graph propagation
'''''''''''''''''''''''''''''''''''''''''''''

The mechanism of updating node embeddings in DGMG is similar to that for
graph convolutional networks. For a node $v$ in the graph, its
neighbor $u$ sends a message to it with

\begin{align}\textbf{m}_{u\rightarrow v}=\textbf{W}_{m}\text{concat}([\textbf{h}_{v}, \textbf{h}_{u}, \textbf{x}_{u, v}]) + \textbf{b}_{m},\\\end{align}

where $\textbf{x}_{u,v}$ is the embedding of the edge between
$u$ and $v$.

After receiving messages from all its neighbors, $v$ summarizes them
with a node activation vector

\begin{align}\textbf{a}_{v} = \sum_{u: (u, v)\in E}\textbf{m}_{u\rightarrow v}\\\end{align}

and use this information to update its own feature:

\begin{align}\textbf{h}'_{v} = \textbf{GRU}(\textbf{h}_{v}, \textbf{a}_{v}).\\\end{align}

Performing all the operations above once for all nodes synchronously is
called one round of graph propagation. The more rounds of graph propagation
you perform, the longer distance messages travel throughout the graph.

With DGL, you implement graph propagation with ``g.update_all``.
The message notation here can be a bit confusing. Researchers can refer
to $\textbf{m}_{u\rightarrow v}$ as messages, however the message function
below only passes $\text{concat}([\textbf{h}_{u}, \textbf{x}_{u, v}])$.
The operation $\textbf{W}_{m}\text{concat}([\textbf{h}_{v}, \textbf{h}_{u}, \textbf{x}_{u, v}]) + \textbf{b}_{m}$
is then performed across all edges at once for efficiency consideration.

```python
from functools import partial

class GraphProp(nn.Module):
    def __init__(self, num_prop_rounds, node_hidden_size):
        super(GraphProp, self).__init__()

        self.num_prop_rounds = num_prop_rounds

        # Setting from the paper
        self.node_activation_hidden_size = 2 * node_hidden_size

        message_funcs = []
        node_update_funcs = []
        self.reduce_funcs = []

        for t in range(num_prop_rounds):
            # input being [hv, hu, xuv]
            message_funcs.append(nn.Linear(2 * node_hidden_size + 1,
                                           self.node_activation_hidden_size))

            self.reduce_funcs.append(partial(self.dgmg_reduce, round=t))
            node_update_funcs.append(
                nn.GRUCell(self.node_activation_hidden_size,
                           node_hidden_size))

        self.message_funcs = nn.ModuleList(message_funcs)
        self.node_update_funcs = nn.ModuleList(node_update_funcs)

    def dgmg_msg(self, edges):
        """For an edge u->v, return concat([h_u, x_uv])"""
        return {'m': torch.cat([edges.src['hv'],
                                edges.data['he']],
                               dim=1)}

    def dgmg_reduce(self, nodes, round):
        hv_old = nodes.data['hv']
        m = nodes.mailbox['m']
        message = torch.cat([
            hv_old.unsqueeze(1).expand(-1, m.size(1), -1), m], dim=2)
        node_activation = (self.message_funcs[round](message)).sum(1)

        return {'a': node_activation}

    def forward(self, g):
        if g.number_of_edges() > 0:
            for t in range(self.num_prop_rounds):
                g.update_all(message_func=self.dgmg_msg,
                             reduce_func=self.reduce_funcs[t])
                g.ndata['hv'] = self.node_update_funcs[t](
                     g.ndata['a'], g.ndata['hv'])
```

Actions
``````````````````````````
All actions are sampled from distributions parameterized using neural networks
and here they are in turn.

Action 1: Add nodes
''''''''''''''''''''''''''

Given the graph embedding vector $\textbf{h}_{G}$, evaluate

\begin{align}\text{Sigmoid}(\textbf{W}_{\text{add node}}\textbf{h}_{G}+b_{\text{add node}}),\\\end{align}

which is then used to parametrize a Bernoulli distribution for deciding whether
to add a new node.

If a new node is to be added, initialize its feature with

\begin{align}\textbf{W}_{\text{init}}\text{concat}([\textbf{h}_{\text{init}} , \textbf{h}_{G}])+\textbf{b}_{\text{init}},\\\end{align}

where $\textbf{h}_{\text{init}}$ is a learnable embedding module for
untyped nodes.

```python
import torch.nn.functional as F
from torch.distributions import Bernoulli

def bernoulli_action_log_prob(logit, action):
    """Calculate the log p of an action with respect to a Bernoulli
    distribution. Use logit rather than prob for numerical stability."""
    if action == 0:
        return F.logsigmoid(-logit)
    else:
        return F.logsigmoid(logit)

class AddNode(nn.Module):
    def __init__(self, graph_embed_func, node_hidden_size):
        super(AddNode, self).__init__()

        self.graph_op = {'embed': graph_embed_func}

        self.stop = 1
        self.add_node = nn.Linear(graph_embed_func.graph_hidden_size, 1)

        # If to add a node, initialize its hv
        self.node_type_embed = nn.Embedding(1, node_hidden_size)
        self.initialize_hv = nn.Linear(node_hidden_size + \
                                       graph_embed_func.graph_hidden_size,
                                       node_hidden_size)

        self.init_node_activation = torch.zeros(1, 2 * node_hidden_size)

    def _initialize_node_repr(self, g, node_type, graph_embed):
        """Whenver a node is added, initialize its representation."""
        num_nodes = g.number_of_nodes()
        hv_init = self.initialize_hv(
            torch.cat([
                self.node_type_embed(torch.LongTensor([node_type])),
                graph_embed], dim=1))
        g.nodes[num_nodes - 1].data['hv'] = hv_init
        g.nodes[num_nodes - 1].data['a'] = self.init_node_activation

    def prepare_training(self):
        self.log_prob = []

    def forward(self, g, action=None):
        graph_embed = self.graph_op['embed'](g)

        logit = self.add_node(graph_embed)
        prob = torch.sigmoid(logit)

        if not self.training:
            action = Bernoulli(prob).sample().item()
        stop = bool(action == self.stop)

        if not stop:
            g.add_nodes(1)
            self._initialize_node_repr(g, action, graph_embed)

        if self.training:
            sample_log_prob = bernoulli_action_log_prob(logit, action)

            self.log_prob.append(sample_log_prob)

        return stop
```

Action 2: Add edges
''''''''''''''''''''''''''

Given the graph embedding vector $\textbf{h}_{G}$ and the node
embedding vector $\textbf{h}_{v}$ for the latest node $v$,
you evaluate

\begin{align}\text{Sigmoid}(\textbf{W}_{\text{add edge}}\text{concat}([\textbf{h}_{G}, \textbf{h}_{v}])+b_{\text{add edge}}),\\\end{align}

which is then used to parametrize a Bernoulli distribution for deciding
whether to add a new edge starting from $v$.

```python
class AddEdge(nn.Module):
    def __init__(self, graph_embed_func, node_hidden_size):
        super(AddEdge, self).__init__()

        self.graph_op = {'embed': graph_embed_func}
        self.add_edge = nn.Linear(graph_embed_func.graph_hidden_size + \
                                  node_hidden_size, 1)

    def prepare_training(self):
        self.log_prob = []

    def forward(self, g, action=None):
        graph_embed = self.graph_op['embed'](g)
        src_embed = g.nodes[g.number_of_nodes() - 1].data['hv']

        logit = self.add_edge(torch.cat(
            [graph_embed, src_embed], dim=1))
        prob = torch.sigmoid(logit)

        if self.training:
            sample_log_prob = bernoulli_action_log_prob(logit, action)
            self.log_prob.append(sample_log_prob)
        else:
            action = Bernoulli(prob).sample().item()

        to_add_edge = bool(action == 0)
        return to_add_edge
```

Action 3: Choose a destination
'''''''''''''''''''''''''''''''''

When action 2 returns `True`, choose a destination for the
latest node $v$.

For each possible destination $u\in\{0, \cdots, v-1\}$, the
probability of choosing it is given by

\begin{align}\frac{\text{exp}(\textbf{W}_{\text{dest}}\text{concat}([\textbf{h}_{u}, \textbf{h}_{v}])+\textbf{b}_{\text{dest}})}{\sum_{i=0}^{v-1}\text{exp}(\textbf{W}_{\text{dest}}\text{concat}([\textbf{h}_{i}, \textbf{h}_{v}])+\textbf{b}_{\text{dest}})}\\\end{align}

```python
from torch.distributions import Categorical

class ChooseDestAndUpdate(nn.Module):
    def __init__(self, graph_prop_func, node_hidden_size):
        super(ChooseDestAndUpdate, self).__init__()

        self.graph_op = {'prop': graph_prop_func}
        self.choose_dest = nn.Linear(2 * node_hidden_size, 1)

    def _initialize_edge_repr(self, g, src_list, dest_list):
        # For untyped edges, only add 1 to indicate its existence.
        # For multiple edge types, use a one-hot representation
        # or an embedding module.
        edge_repr = torch.ones(len(src_list), 1)
        g.edges[src_list, dest_list].data['he'] = edge_repr

    def prepare_training(self):
        self.log_prob = []

    def forward(self, g, dest):
        src = g.number_of_nodes() - 1
        possible_dests = range(src)

        src_embed_expand = g.nodes[src].data['hv'].expand(src, -1)
        possible_dests_embed = g.nodes[possible_dests].data['hv']

        dests_scores = self.choose_dest(
            torch.cat([possible_dests_embed,
                       src_embed_expand], dim=1)).view(1, -1)
        dests_probs = F.softmax(dests_scores, dim=1)

        if not self.training:
            dest = Categorical(dests_probs).sample().item()

        if not g.has_edge_between(src, dest):
            # For undirected graphs, add edges for both directions
            # so that you can perform graph propagation.
            src_list = [src, dest]
            dest_list = [dest, src]

            g.add_edges(src_list, dest_list)
            self._initialize_edge_repr(g, src_list, dest_list)

            self.graph_op['prop'](g)

        if self.training:
            if dests_probs.nelement() > 1:
                self.log_prob.append(
                    F.log_softmax(dests_scores, dim=1)[:, dest: dest + 1])
```

Putting it together
``````````````````````````

You are now ready to have a complete implementation of the model class.

```python
class DGMG(DGMGSkeleton):
    def __init__(self, v_max, node_hidden_size,
                 num_prop_rounds):
        super(DGMG, self).__init__(v_max)

        # Graph embedding module
        self.graph_embed = GraphEmbed(node_hidden_size)

        # Graph propagation module
        self.graph_prop = GraphProp(num_prop_rounds,
                                    node_hidden_size)

        # Actions
        self.add_node_agent = AddNode(
            self.graph_embed, node_hidden_size)
        self.add_edge_agent = AddEdge(
            self.graph_embed, node_hidden_size)
        self.choose_dest_agent = ChooseDestAndUpdate(
            self.graph_prop, node_hidden_size)

        # Forward functions
        self.forward_train = partial(forward_train, self=self)
        self.forward_inference = partial(forward_inference, self=self)

    @property
    def action_step(self):
        old_step_count = self.step_count
        self.step_count += 1

        return old_step_count

    def prepare_for_train(self):
        self.step_count = 0

        self.add_node_agent.prepare_training()
        self.add_edge_agent.prepare_training()
        self.choose_dest_agent.prepare_training()

    def add_node_and_update(self, a=None):
        """Decide if to add a new node.
        If a new node should be added, update the graph."""

        return self.add_node_agent(self.g, a)

    def add_edge_or_not(self, a=None):
        """Decide if a new edge should be added."""

        return self.add_edge_agent(self.g, a)

    def choose_dest_and_update(self, a=None):
        """Choose destination and connect it to the latest node.
        Add edges for both directions and update the graph."""

        self.choose_dest_agent(self.g, a)

    def get_log_prob(self):
        add_node_log_p = torch.cat(self.add_node_agent.log_prob).sum()
        add_edge_log_p = torch.cat(self.add_edge_agent.log_prob).sum()
        choose_dest_log_p = torch.cat(self.choose_dest_agent.log_prob).sum()
        return add_node_log_p + add_edge_log_p + choose_dest_log_p
```

Below is an animation where a graph is generated on the fly
after every 10 batches of training for the first 400 batches. You
can see how the model improves over time and begins generating cycles.

.. figure:: https://user-images.githubusercontent.com/19576924/48929291-60fe3880-ef22-11e8-832a-fbe56656559a.gif
   :alt:

For generative models, you can evaluate performance by checking the percentage
of valid graphs among the graphs it generates on the fly.

```python
import torch.utils.model_zoo as model_zoo

# Download a pre-trained model state dict for generating cycles with 10-20 nodes.
state_dict = model_zoo.load_url('https://data.dgl.ai/model/dgmg_cycles-5a0c40be.pth')
model = DGMG(v_max=20, node_hidden_size=16, num_prop_rounds=2)
model.load_state_dict(state_dict)
model.eval()

def is_valid(g):
    # Check if g is a cycle having 10-20 nodes.
    def _get_previous(i, v_max):
        if i == 0:
            return v_max
        else:
            return i - 1

    def _get_next(i, v_max):
        if i == v_max:
            return 0
        else:
            return i + 1

    size = g.number_of_nodes()

    if size < 10 or size > 20:
        return False

    for node in range(size):
        neighbors = g.successors(node)

        if len(neighbors) != 2:
            return False

        if _get_previous(node, size - 1) not in neighbors:
            return False

        if _get_next(node, size - 1) not in neighbors:
            return False

    return True

num_valid = 0
for i in range(100):
    g = model()
    num_valid += is_valid(g)

del model
print('Among 100 graphs generated, {}% are valid.'.format(num_valid))
```

For the complete implementation, see the `DGL DGMG example
<https://github.com/dmlc/dgl/tree/master/examples/pytorch/dgmg>`__.


```
pip install pennylane
```

    Requirement already satisfied: pennylane in /usr/local/lib/python3.6/dist-packages (0.10.0)
    Requirement already satisfied: semantic-version==2.6 in /usr/local/lib/python3.6/dist-packages (from pennylane) (2.6.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.18.5)
    Requirement already satisfied: toml in /usr/local/lib/python3.6/dist-packages (from pennylane) (0.10.1)
    Requirement already satisfied: networkx in /usr/local/lib/python3.6/dist-packages (from pennylane) (2.4)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.4.1)
    Requirement already satisfied: appdirs in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.4.4)
    Requirement already satisfied: autograd in /usr/local/lib/python3.6/dist-packages (from pennylane) (1.3)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx->pennylane) (4.4.2)
    Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from autograd->pennylane) (0.16.0)

```
pip install torch
```

    Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.5.1+cu101)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torch) (1.18.5)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch) (0.16.0)

```
pip install tensorflow
```

    Requirement already satisfied: tensorflow in /usr/local/lib/python3.6/dist-packages (2.2.0)
    Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.18.5)
    Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (2.10.0)
    Requirement already satisfied: tensorboard<2.3.0,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (2.2.2)
    Requirement already satisfied: tensorflow-estimator<2.3.0,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (2.2.0)
    Requirement already satisfied: keras-preprocessing>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.1.2)
    Requirement already satisfied: gast==0.3.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.3.3)
    Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (3.10.0)
    Requirement already satisfied: scipy==1.4.1; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.4.1)
    Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.1.0)
    Requirement already satisfied: google-pasta>=0.1.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.2.0)
    Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.9.0)
    Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.12.0)
    Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (3.2.1)
    Requirement already satisfied: astunparse==1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.6.3)
    Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.30.0)
    Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from tensorflow) (0.34.2)
    Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow) (1.12.1)
    Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (3.2.2)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (0.4.1)
    Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (1.0.1)
    Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (1.17.2)
    Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (49.1.0)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (1.7.0)
    Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow) (2.23.0)
    Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<2.3.0,>=2.2.0->tensorflow) (1.7.0)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow) (1.3.0)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow) (0.2.8)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow) (4.1.1)
    Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow) (4.6)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.3.0,>=2.2.0->tensorflow) (2020.6.20)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.3.0,>=2.2.0->tensorflow) (1.24.3)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.3.0,>=2.2.0->tensorflow) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.3.0,>=2.2.0->tensorflow) (3.0.4)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < "3.8"->markdown>=2.6.8->tensorboard<2.3.0,>=2.2.0->tensorflow) (3.1.0)
    Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.3.0,>=2.2.0->tensorflow) (3.1.0)
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard<2.3.0,>=2.2.0->tensorflow) (0.4.8)

```
pip install sklearn
```

    Requirement already satisfied: sklearn in /usr/local/lib/python3.6/dist-packages (0.0)
    Requirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sklearn) (0.22.2.post1)
    Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sklearn) (1.18.5)
    Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sklearn) (1.4.1)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sklearn) (0.16.0)

```
pip install pennylane-qiskit
```

    Requirement already satisfied: pennylane-qiskit in /usr/local/lib/python3.6/dist-packages (0.9.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from pennylane-qiskit) (1.18.5)
    Requirement already satisfied: qiskit>=0.19.1 in /usr/local/lib/python3.6/dist-packages (from pennylane-qiskit) (0.19.6)
    Requirement already satisfied: pennylane>=0.9.0 in /usr/local/lib/python3.6/dist-packages (from pennylane-qiskit) (0.10.0)
    Requirement already satisfied: networkx>=2.2; python_version > "3.5" in /usr/local/lib/python3.6/dist-packages (from pennylane-qiskit) (2.4)
    Requirement already satisfied: qiskit-aqua==0.7.3 in /usr/local/lib/python3.6/dist-packages (from qiskit>=0.19.1->pennylane-qiskit) (0.7.3)
    Requirement already satisfied: qiskit-terra==0.14.2 in /usr/local/lib/python3.6/dist-packages (from qiskit>=0.19.1->pennylane-qiskit) (0.14.2)
    Requirement already satisfied: qiskit-aer==0.5.2 in /usr/local/lib/python3.6/dist-packages (from qiskit>=0.19.1->pennylane-qiskit) (0.5.2)
    Requirement already satisfied: qiskit-ibmq-provider==0.7.2 in /usr/local/lib/python3.6/dist-packages (from qiskit>=0.19.1->pennylane-qiskit) (0.7.2)
    Requirement already satisfied: qiskit-ignis==0.3.3 in /usr/local/lib/python3.6/dist-packages (from qiskit>=0.19.1->pennylane-qiskit) (0.3.3)
    Requirement already satisfied: appdirs in /usr/local/lib/python3.6/dist-packages (from pennylane>=0.9.0->pennylane-qiskit) (1.4.4)
    Requirement already satisfied: toml in /usr/local/lib/python3.6/dist-packages (from pennylane>=0.9.0->pennylane-qiskit) (0.10.1)
    Requirement already satisfied: semantic-version==2.6 in /usr/local/lib/python3.6/dist-packages (from pennylane>=0.9.0->pennylane-qiskit) (2.6.0)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from pennylane>=0.9.0->pennylane-qiskit) (1.4.1)
    Requirement already satisfied: autograd in /usr/local/lib/python3.6/dist-packages (from pennylane>=0.9.0->pennylane-qiskit) (1.3)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx>=2.2; python_version > "3.5"->pennylane-qiskit) (4.4.2)
    Requirement already satisfied: pyscf; sys_platform != "win32" in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.7.3)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (0.22.2.post1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (5.4.8)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.6.1)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (49.1.0)
    Requirement already satisfied: dlx in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.0.4)
    Requirement already satisfied: docplex in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (2.15.194)
    Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (2.10.0)
    Requirement already satisfied: fastdtw in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (0.3.4)
    Requirement already satisfied: quandl in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (3.5.1)
    Requirement already satisfied: marshmallow<4,>=3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (3.7.0)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (1.4.0)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (2.14.4)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (0.3.2)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (3.11)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (2.6.0)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (2.8.1)
    Requirement already satisfied: retworkx>=0.3.2 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (0.3.4)
    Requirement already satisfied: marshmallow-polyfield<6,>=5.7 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit>=0.19.1->pennylane-qiskit) (5.9)
    Requirement already satisfied: cython>=0.27.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit>=0.19.1->pennylane-qiskit) (0.29.21)
    Requirement already satisfied: pybind11>=2.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit>=0.19.1->pennylane-qiskit) (2.5.0)
    Requirement already satisfied: nest-asyncio!=1.1.0,>=1.0.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (1.3.3)
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (2.23.0)
    Requirement already satisfied: websockets<8,>=7 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (7.0)
    Requirement already satisfied: requests-ntlm>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (1.1.0)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (1.24.3)
    Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from autograd->pennylane>=0.9.0->pennylane-qiskit) (0.16.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (0.16.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.3->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.1.0)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from docplex->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.12.0)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (8.4.0)
    Requirement already satisfied: pandas>=0.14 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (1.0.5)
    Requirement already satisfied: inflection>=0.3.1 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (0.5.0)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (2020.6.20)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (2.10)
    Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (2.9.2)
    Requirement already satisfied: ntlm-auth>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (1.5.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.14->quandl->qiskit-aqua==0.7.3->qiskit>=0.19.1->pennylane-qiskit) (2018.9)
    Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (1.14.0)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit>=0.19.1->pennylane-qiskit) (2.20)

```
import pennylane as qml
from pennylane import numpy as np
```

```
dev = qml.device("default.qubit", wires=2)
@qml.qnode(device=dev)
def cos_func(x, w):
  qml.RX(x, wires=0)
  qml.templates.BasicEntanglerLayers(w, wires=range(2))
  return qml.expval(qml.PauliZ(0))

layer = 4
weights = qml.init.basic_entangler_layers_uniform(layer, 2)

xs = np.linspace(-np.pi, 4*np.pi, requires_grad=False)
ys = np.cos(xs)
```

```
opt = qml.AdamOptimizer()
epochs = 10

for epoch in range(epochs):
  for x, y in zip(xs, ys):
    cost = lambda weights:(cos_func(x, weights) - y) ** 2
    weights = opt.step(cost, weights)

ys_trained = [cos_func(x, weights) for x in xs]

```

```
import matplotlib.pyplot as plt

plt.figure()
plt.plot(xs, ys_trained, marker="o", label="Cos(x")
plt.legend()
plt.show()
```

```

```

## Preparing GHZ state

Using the Autograd interface, train a circuit to prepare the 3-qubit W state:

$|W> = {1/sqrt(3)}(001|> + |010> + |100>)

```
qubits = 3

w = np.array([0, 1, 1, 0, 1, 0, 0, 0]) / np.sqrt(3)
w_projector = w[:, np.newaxis] * w
w_decomp = qml.utils.decompose_hamiltonian(w_projector)
H = qml.Hamiltonian(*w_decomp)

def prepare_w(weights, wires):
    qml.templates.StronglyEntanglingLayers(weights, wires=wires)

dev = qml.device("default.qubit", wires=qubits)
qnodes = qml.map(prepare_w, H.ops, dev)
w_overlap = qml.dot(H.coeffs, qnodes)

layers = 4
weights = qml.init.strong_ent_layers_uniform(layers, qubits)

opt = qml.RMSPropOptimizer()

epochs = 50

for i in range(epochs):
    weights = opt.step(lambda weights: -w_overlap(weights), weights)
    if i % 5 == 0:
        print(i, w_overlap(weights))

output_overlap = w_overlap(weights)
output_state = np.round(dev.state, 3)

```

##Quantum-based Optimization

```
dev = qml.device('default.qubit', wires=1)

@qml.qnode(dev)
def rotation(thetas):
    qml.RX(1, wires=0)
    qml.RZ(1, wires=0)

    qml.RX(thetas[0], wires=0)
    qml.RY(thetas[1], wires=0)

    return qml.expval(qml.PauliZ(0))

```

```
opt = qml.RotoselectOptimizer()

```

```
import sklearn.datasets

data = sklearn.datasets.load_iris()
x = data["data"]
y = data["target"]

np.random.seed(1967)
x, y = zip(*np.random.permutation(list(zip(x, y))))

split = 125

x_train = x[:split]
x_test = x[split:]
y_train = y[:split]
y_test = y[split:]

```

```

```