# **CS224W - Colab 4**

In this Colab, we will shift our focus from homogenous graphs to heterogeneous graphs. Heterogeneous graphs extend the traditional homogenous graphs that we have seen before by specifically incorperating different node and edge types. This additional information allows us to extend the graph neural nework models that we have worked with before. Namely, we can apply heterogenous message passing, where different message types now exist between different node, edge type relationships.

At first, we will learn how to transform NetworkX graphs into DeepSNAP representations. Additionally, we will dive deeper into how DeepSNAP stores and represents heterogeneous graphs as PyTorch Tensors.

Then, we will build our own heterogenous graph neural netowrk models using PyTorch Geonetric and DeepSNAP on node property prediction task. To evaluate these models, we will use our model on the heterogeneous ACM dataset.

**Note**: Make sure to **sequentially run all the cells in each section**, so that the intermediate variables / packages will carry over to the next cell

Have fun on Colab 4 :)

# Device
You might need to use GPU for this Colab.

Please click `Runtime` and then `Change runtime type`. Then set the `hardware accelerator` to **GPU**.

# Installation

```python
!pip install -q torch-scatter -f https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html
!pip install -q torch-sparse -f https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html
!pip install -q torch-geometric
!pip install -q git+https://github.com/snap-stanford/deepsnap.git
!pip install -U -q PyDrive
```

         |████████████████████████████████| 2.6MB 32.5MB/s
         |████████████████████████████████| 1.4MB 22.0MB/s
         |████████████████████████████████| 225kB 25.9MB/s
         |████████████████████████████████| 235kB 44.2MB/s
         |████████████████████████████████| 51kB 8.4MB/s
      Building wheel for torch-geometric (setup.py) ... done
      Building wheel for deepsnap (setup.py) ... done

```python
import torch_geometric
torch_geometric.__version__
```

    '1.7.2'

# 1 DeepSNAP Heterogeneous Graph

First, we will explore how to transform NetworkX graphs into the format supported by DeepSNAP.

In order to extend the traditional DeepSNAP representation of graphs to include heterogenous graphs, we include the following graph property features:
* `node_feature`: The feature of each node (`torch.tensor`)
* `edge_feature`: The feautre of each edge (`torch.tensor`)
* `node_label`: The label of each node (`int`)
* `node_type`: The node type of each node (`string`)
* `edge_type`: The edge type of each edge (`string`)

The key new features we add are `node_type` and `edge_type`, which enables us to perform heterogenous message passing.

In this first question we will work with the familiar [karate club graph](https://networkx.github.io/documentation/stable/auto_examples/graph/plot_karate_club.html) seen in assignment one. To start, since each node in the graph belongs to one of two clubs (club "Mr. Hi" or club "Officer"), we will treat the club as the `node_type`. The code below demonstrates how to differentiate the nodes in the NetworkX graph.

```python
from pylab import *
import networkx as nx
from networkx.algorithms.community import greedy_modularity_communities
import matplotlib.pyplot as plt
import copy

G = nx.karate_club_graph()
community_map = {}
for node in G.nodes(data=True):
  if node[1]["club"] == "Mr. Hi":
    community_map[node[0]] = 0
  else:
    community_map[node[0]] = 1
node_color = []
color_map = {0: 0, 1: 1}
node_color = [color_map[community_map[node]] for node in G.nodes()]
pos = nx.spring_layout(G)
plt.figure(figsize=(7, 7))
nx.draw(G, pos=pos, cmap=plt.get_cmap('coolwarm'), node_color=node_color)
show()
```

### Question 1.1: Assigning Node Type and Node Features (Not Specifically Graded)

Using the `community_map` dictionary and graph `G` from above, add node attributes `node_type` and `node_label` to the graph G. Namely, for `node_type` assign nodes in the "Mr. Hi" club to a node type `n0` and nodes in club "Officer" a node type `n1`.

Then for `node_label`, assign nodes in "Mr. Hi" club to a `node_label` `0` and nodes in club "Officer" a `node_label` of `1`.

Lastly, assign every node a feature vector [1, 1, 1, 1, 1].

**Hint**: Look at the NetworkX function `nx.classes.function.set_node_attributes`.

**Note**: This question is not specifically graded but is important for later questions.

```python
import torch

def assign_node_types(G, community_map):
  # TODO: Implement this function that takes in a NetworkX graph
  # G and community map assignment (mapping node ids --> 0/1 labels)
  # and adds 'node_type' as a node_attribute in G.

  ############# Your code here ############
  ## (~2 line of code)
  ## Note
  ## 1. Look up NetworkX `nx.classes.function.set_node_attributes`
  values = {}
  for node in G.nodes(data=True):
      if community_map[node[0]] == 0:
          values[node[0]] = 'n0'
      else:
          values[node[0]] = 'n1'
  nx.classes.function.set_node_attributes(G, values, name='node_type')

  #########################################

def assign_node_labels(G, community_map):
  # TODO: Implement this function that takes in a NetworkX graph
  # G and community map assignment (mapping node ids --> 0/1 labels)
  # and adds 'node_label' as a node_attribute in G.

  ############# Your code here ############
  ## (~2 line of code)
  ## Note
  ## 1. Look up NetworkX `nx.classes.function.set_node_attributes`

  nx.classes.function.set_node_attributes(G, community_map, name='node_label')

  #########################################

def assign_node_features(G):
  # TODO: Implement this function that takes in a NetworkX graph
  # G and adds 'node_feature' as a node_attribute in G. Each node
  # in the graph has the same feature vector [1., 1., 1., 1., 1.]

  ############# Your code here ############
  ## (~2 line of code)
  ## Note
  ## 1. Look up NetworkX `nx.classes.function.set_node_attributes`
  nx.classes.function.set_node_attributes(G, [1., 1., 1., 1., 1.], name='node_feature')

  #########################################

assign_node_types(G, community_map)
assign_node_labels(G, community_map)
assign_node_features(G)

for node in G.nodes(data=True):
    print(node)
```

    (0, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (1, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (2, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (3, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (4, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (5, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (6, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (7, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (8, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (9, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (10, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (11, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (12, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (13, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (14, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (15, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (16, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (17, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (18, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (19, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (20, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (21, {'club': 'Mr. Hi', 'node_type': 'n0', 'node_label': 0, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (22, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (23, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (24, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (25, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (26, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (27, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (28, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (29, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (30, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (31, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (32, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})
    (33, {'club': 'Officer', 'node_type': 'n1', 'node_label': 1, 'node_feature': [1.0, 1.0, 1.0, 1.0, 1.0]})

### Question 1.2: Assigning Edge Types (Not Specifically Graded)

Next, we will assign three different types of `edge_type` to the edges:
* Edges within club "Mr. Hi": `e0`
* Edges within club "Officer": `e1`
* Edges between clubs: `e2`

**Hint**: Use the `community_map` from before and `nx.classes.function.set_edge_attributes`

```python
def assign_edge_types(G, community_map):
  # TODO: Implement this function that takes in a NetworkX graph
  # G and community map assignment (mapping node ids --> 0/1 labels)
  # and adds 'edge_type' as a edge_attribute in G.

  ############# Your code here ############
  ## (~5 line of code)
  ## Note
  ## 1. Create an edge assignment dict following rules above
  values = {}
  for edge in G.edges(data=True):
    if community_map[edge[0]] == 0 and community_map[edge[1]] == 0:
      values[(edge[0], edge[1])] = 'e0'
    elif community_map[edge[0]] == 1 and community_map[edge[1]] == 1:
      values[(edge[0], edge[1])] = 'e1'
    else:
      values[(edge[0], edge[1])] = 'e2'
  nx.classes.function.set_edge_attributes(G, values, name='edge_type')

  #########################################

assign_edge_types(G, community_map)
for edge in G.edges(data=True):
    print(edge)
```

    (0, 1, {'edge_type': 'e0'})
    (0, 2, {'edge_type': 'e0'})
    (0, 3, {'edge_type': 'e0'})
    (0, 4, {'edge_type': 'e0'})
    (0, 5, {'edge_type': 'e0'})
    (0, 6, {'edge_type': 'e0'})
    (0, 7, {'edge_type': 'e0'})
    (0, 8, {'edge_type': 'e0'})
    (0, 10, {'edge_type': 'e0'})
    (0, 11, {'edge_type': 'e0'})
    (0, 12, {'edge_type': 'e0'})
    (0, 13, {'edge_type': 'e0'})
    (0, 17, {'edge_type': 'e0'})
    (0, 19, {'edge_type': 'e0'})
    (0, 21, {'edge_type': 'e0'})
    (0, 31, {'edge_type': 'e2'})
    (1, 2, {'edge_type': 'e0'})
    (1, 3, {'edge_type': 'e0'})
    (1, 7, {'edge_type': 'e0'})
    (1, 13, {'edge_type': 'e0'})
    (1, 17, {'edge_type': 'e0'})
    (1, 19, {'edge_type': 'e0'})
    (1, 21, {'edge_type': 'e0'})
    (1, 30, {'edge_type': 'e2'})
    (2, 3, {'edge_type': 'e0'})
    (2, 7, {'edge_type': 'e0'})
    (2, 8, {'edge_type': 'e0'})
    (2, 9, {'edge_type': 'e2'})
    (2, 13, {'edge_type': 'e0'})
    (2, 27, {'edge_type': 'e2'})
    (2, 28, {'edge_type': 'e2'})
    (2, 32, {'edge_type': 'e2'})
    (3, 7, {'edge_type': 'e0'})
    (3, 12, {'edge_type': 'e0'})
    (3, 13, {'edge_type': 'e0'})
    (4, 6, {'edge_type': 'e0'})
    (4, 10, {'edge_type': 'e0'})
    (5, 6, {'edge_type': 'e0'})
    (5, 10, {'edge_type': 'e0'})
    (5, 16, {'edge_type': 'e0'})
    (6, 16, {'edge_type': 'e0'})
    (8, 30, {'edge_type': 'e2'})
    (8, 32, {'edge_type': 'e2'})
    (8, 33, {'edge_type': 'e2'})
    (9, 33, {'edge_type': 'e1'})
    (13, 33, {'edge_type': 'e2'})
    (14, 32, {'edge_type': 'e1'})
    (14, 33, {'edge_type': 'e1'})
    (15, 32, {'edge_type': 'e1'})
    (15, 33, {'edge_type': 'e1'})
    (18, 32, {'edge_type': 'e1'})
    (18, 33, {'edge_type': 'e1'})
    (19, 33, {'edge_type': 'e2'})
    (20, 32, {'edge_type': 'e1'})
    (20, 33, {'edge_type': 'e1'})
    (22, 32, {'edge_type': 'e1'})
    (22, 33, {'edge_type': 'e1'})
    (23, 25, {'edge_type': 'e1'})
    (23, 27, {'edge_type': 'e1'})
    (23, 29, {'edge_type': 'e1'})
    (23, 32, {'edge_type': 'e1'})
    (23, 33, {'edge_type': 'e1'})
    (24, 25, {'edge_type': 'e1'})
    (24, 27, {'edge_type': 'e1'})
    (24, 31, {'edge_type': 'e1'})
    (25, 31, {'edge_type': 'e1'})
    (26, 29, {'edge_type': 'e1'})
    (26, 33, {'edge_type': 'e1'})
    (27, 33, {'edge_type': 'e1'})
    (28, 31, {'edge_type': 'e1'})
    (28, 33, {'edge_type': 'e1'})
    (29, 32, {'edge_type': 'e1'})
    (29, 33, {'edge_type': 'e1'})
    (30, 32, {'edge_type': 'e1'})
    (30, 33, {'edge_type': 'e1'})
    (31, 32, {'edge_type': 'e1'})
    (31, 33, {'edge_type': 'e1'})
    (32, 33, {'edge_type': 'e1'})

## Heterogeneous Graph Visualization

Now we can visualize the Heterogeneous Graph we have generated.

```python
edge_color = {}
for edge in G.edges():
  n1, n2 = edge
  edge_color[edge] = community_map[n1] if community_map[n1] == community_map[n2] else 2
  if community_map[n1] == community_map[n2] and community_map[n1] == 0:
    edge_color[edge] = 'blue'
  elif community_map[n1] == community_map[n2] and community_map[n1] == 1:
    edge_color[edge] = 'red'
  else:
    edge_color[edge] = 'green'

G_orig = copy.deepcopy(G)
nx.classes.function.set_edge_attributes(G, edge_color, name='color')
colors = nx.get_edge_attributes(G,'color').values()
labels = nx.get_node_attributes(G, 'node_type')
plt.figure(figsize=(8, 8))
nx.draw(G, pos=pos, cmap=plt.get_cmap('coolwarm'), node_color=node_color, edge_color=colors, labels=labels, font_color='white')
show()
```

which include edges within each clubs (2 types) and edges across two clubs (1 type). Different types of nodes and edges are visualized in different colors. The NetworkX object `G` in following code can be transformed into `deepsnap.hetero_graph.HeteroGraph` directly.

## Transforming to DeepSNAP representation

The NetworkX object `G` in following code can be transformed into `deepsnap.hetero_graph.HeteroGraph` directly!

```python
from deepsnap.hetero_graph import HeteroGraph

hete = HeteroGraph(G_orig)
```

## Question 1.3: How many nodes are of each type (10 Points)

Submit your answers on Gradescope.

```python
def get_nodes_per_type(hete):
  # TODO: Implement this function that takes a DeepSNAP dataset object
  # and return the number of nodes per `node_type`.

  num_nodes_n0 = 0
  num_nodes_n1 = 0

  ############# Your code here ############
  ## (~2 line of code)
  ## Note
  ## 1. Colab autocomplete functionality might be useful.

  num_nodes_n0 = hete.num_nodes('n0')
  num_nodes_n1 = hete.num_nodes('n1')
  #########################################

  return num_nodes_n0, num_nodes_n1

num_nodes_n0, num_nodes_n1 = get_nodes_per_type(hete)
print("Node type n0 has {} nodes".format(num_nodes_n0))
print("Node type n1 has {} nodes".format(num_nodes_n1))
```

    Node type n0 has 17 nodes
    Node type n1 has 17 nodes

## Question 1.4: Message Types: How many edges are of each message type (10 Points)

Submit your answers on Gradescope.

When working with heterogenous graphs, as we have discussed before, we now work with heterogenous message types (i.e. different message types for `node_type` and `edge_type` combinations). For example, an edge of type `e0` connecting two nodes in club "Mr. HI" would have a message type of (`n0`, `e0`, `n0`). In this problem we will analyze how many edges in our graph are of each message type.

**Hint**: If you want to learn more about what the different message types are try the call `hete.message_types`

```python
def get_num_message_edges(hete):
  # TODO: Implement this function that takes a DeepSNAP dataset object
  # and return the number of edges for each message type.
  # You should return a list of tuples as
  # (message_type, num_edge)

  message_type_edges = []

  ############# Your code here ############
  ## (~2 line of code)
  ## Note
  ## 1. Colab autocomplete functionality might be useful.
  for message_type in hete.message_types:
      message_type_edges.append([message_type, hete.num_edges(message_type=message_type)])
  #########################################

  return message_type_edges

message_type_edges = get_num_message_edges(hete)
for (message_type, num_edges) in message_type_edges:
  print("Message type {} has {} edges".format(message_type, num_edges))

```

    Message type ('n0', 'e0', 'n0') has 35 edges
    Message type ('n0', 'e2', 'n1') has 11 edges
    Message type ('n1', 'e1', 'n1') has 32 edges

## Question 1.5: Dataset Splitting: How many nodes in each dataset split? (10 Points)

Submit your answers on Gradescope.

DeepSNAP has built in Dataset creation and splitting methods for heterogeneous graphs. Here we will create train, validation, and test datasets for a node prediction task and inspect the resulting subgraphs. Specifically, write a function that computes the number of nodes in each dataset split.

```python
from deepsnap.dataset import GraphDataset

def compute_dataset_split_counts(datasets):
  # TODO: Implement this function that takes a dict of datasets
  # and returns a dict mapping dataset names to the number of labeled
  # nodes used for supervision in that respective dataset.

  data_set_splits = {}

  ############# Your code here ############
  ## (~3 line of code)
  ## Note
  ## 1. DeepSNAP `node_label_index` will be helpful.
  ## 2. Remember to count both node_types
  for split in datasets.keys():
      data_set_splits[split] = len(datasets[split][0].node_label_index['n0']) + len(datasets[split][0].node_label_index['n1'])
  #########################################

  return data_set_splits

dataset = GraphDataset([hete], task='node')
# Splitting the dataset
dataset_train, dataset_val, dataset_test = dataset.split(transductive=True, split_ratio=[0.4, 0.3, 0.3])
datasets = {'train': dataset_train, 'val': dataset_val, 'test': dataset_test}

data_set_splits = compute_dataset_split_counts(datasets)
for dataset_name, num_nodes in data_set_splits.items():
  print("{} dataset has {} nodes".format(dataset_name, num_nodes))
```

    train dataset has 12 nodes
    val dataset has 10 nodes
    test dataset has 12 nodes

## DeepSNAP Dataset Visualization

```python
from deepsnap.dataset import GraphDataset

dataset = GraphDataset([hete], task='node')
# Splitting the dataset
dataset_train, dataset_val, dataset_test = dataset.split(transductive=True, split_ratio=[0.4, 0.3, 0.3])
titles = ['Train', 'Validation', 'Test']

for i, dataset in enumerate([dataset_train, dataset_val, dataset_test]):
  n0 = hete._convert_to_graph_index(dataset[0].node_label_index['n0'], 'n0').tolist()
  n1 = hete._convert_to_graph_index(dataset[0].node_label_index['n1'], 'n1').tolist()

  plt.figure(figsize=(7, 7))
  plt.title(titles[i])
  nx.draw(G_orig, pos=pos, node_color="grey", edge_color=colors, labels=labels, font_color='white')
  nx.draw_networkx_nodes(G_orig.subgraph(n0), pos=pos, node_color="blue")
  nx.draw_networkx_nodes(G_orig.subgraph(n1), pos=pos, node_color="red")
  show()
```

# 2 Heterogeneous Graph Node Property Prediction

In this part we will use PyTorch Geometric and DeepSNAP to implement a GNN model for heterogeneous graph node property prediction (node classification). This part of Colab requires you having good understandings on the heterogeneous graph and how to implement the GNN layers by using PyG.

At first let's take look at the general structure of a heterogeneous layer by an example.

Let's assume we have a graph $G$, which contains two node types $a$ and $b$, and three message types $m_1=(a, r_1, a)$, $m_2=(a, r_2, b)$ and $m_3=(a, r_3, b)$.

Thus, for $G$ a heterogeneous layer will contains three Heterogeneous GNN layers (`HeteroGNNConv` in this Colab) where each `HeteroGNNConv` layer will perform the message passing and aggregation with respect to only one message type. The overview of the heterogeneous layer is shown below:

<br/>
<center>

</center>
<br/>

In this Colab, all the $l^{th}$ Heterogeneous GNN layers will be managed by a ($l^{th}$) Heterogeneous GNN Wrapper layer (the `HeteroGNNWrapperConv`). The $l^{th}$ Heterogeneous GNN Wrapper layer will take in the input node embeddings from $(l-1)^{th}$ layer and aggregate (across message types) the Heterogeneous GNN layers' results. For example, the wrapper layer will aggregate node type $b$'s node embeddings from Heterogeneous GNN layers for $m_2$ and $m_3$. The "simplified" heterogeneous layer structure is shown below:

<br/>
<center>

</center>
<br/>

<font color='red'>We recommend you implement the heterogeneous GNN model in following steps:</font>

1. Implement the `HeteroGNNConv` first.
2. Implement the `mean` aggregation in `HeteroGNNWrapperConv`.
3. Implement the `generate_convs`.
4. Implement the `HeteroGNN` model and the `train` function.
5. Train the model with `mean` aggregation across the message types and make sure your model has reasonable performance.
6. Implement the `attn` aggregation in `HeteroGNNWrapperConv`.
7. Train the model with `attn` aggregation across the message types and make sure your model has reasonable performance.

## Setup

```python
import copy
import torch
import deepsnap
import numpy as np
import torch.nn as nn
import torch.nn.functional as F
import torch_geometric.nn as pyg_nn

from sklearn.metrics import f1_score
from deepsnap.hetero_gnn import forward_op
from deepsnap.hetero_graph import HeteroGraph
from torch_sparse import SparseTensor, matmul
```

## Dataset

You need to login to your Google account and enter the verification code below.

```python
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# Authenticate and create the PyDrive client
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
```

```python
id='1ivlxd6lJMcZ9taS44TMGG72x2V1GeVvk'
downloaded = drive.CreateFile({'id': id})
downloaded.GetContentFile('acm.pkl')
```

## Heterogeneous GNN Layer

Now let's start working on our own implementation of a heterogeneous layer (the `HeteroGNNConv`)! Similar to what we did in Colab 3, we will implement the layer using PyTorch Geometric. In general, our heterogeneous GNN layer draws ideas from the **GraphSAGE** ([Hamilton et al. (2017)](https://arxiv.org/abs/1706.02216)).

At first, let's implement the GNN layer for each message type :

\begin{equation}
m =(s, r, d)
\end{equation}

Each message type is a tuple containing three elements where $s$ refers to the source node type, $r$ refers to the edge (relation) type and $d$ refers to the destination node type. The update rule is very similar to that of GraphSAGE but we need to include the node types and the edge type. The update rule is described as below:

\begin{equation}
h_v^{(l)[m]} = W^{(l)[m]} \cdot \text{CONCAT} \Big( W_d^{(l)[m]} \cdot h_v^{(l-1)}, W_s^{(l)[m]} \cdot AGG(\{h_u^{(l-1)}, \forall u \in N_{m}(v) \})\Big)
\end{equation}

where $[m]$ indicates that the weight matrices or embeddings with respect to message type $m$, $W_s^{(l)[m]}$ computes the messages from neighboring nodes, $W_d^{(l)[m]}$ compute messages from the node itself, and $W^{(l)[m]}$ aggregates messages from both node types. In the equation above, $v$ has the node type $d$, and $u$ has the node type $s$.

For simplicity, we use mean aggregations for $AGG$ where:

\begin{equation}
AGG(\{h_u^{(l-1)}, \forall u \in N_{m}(v) \}) = \frac{1}{|N_{m}(v)|} \sum_{u\in N_{m}(v)} h_u^{(l-1)}
\end{equation}

```python
class HeteroGNNConv(pyg_nn.MessagePassing):
    def __init__(self, in_channels_src, in_channels_dst, out_channels):
        super(HeteroGNNConv, self).__init__(aggr="mean")

        self.in_channels_src = in_channels_src
        self.in_channels_dst = in_channels_dst
        self.out_channels = out_channels

        # To simplify implementation, please initialize both self.lin_dst
        # and self.lin_src out_features to out_channels
        self.lin_dst = None
        self.lin_src = None

        self.lin_update = None

        ############# Your code here #############
        ## (~3 lines of code)

        self.lin_dst = nn.Linear(self.in_channels_src, self.out_channels)
        self.lin_src = nn.Linear(self.in_channels_dst, self.out_channels)
        self.lin_update = nn.Linear(2*self.out_channels, self.out_channels)

        ##########################################

    def forward(
        self,
        node_feature_src,
        node_feature_dst,
        edge_index,
        size=None,
        res_n_id=None,
    ):
        ############# Your code here #############
        ## (~1 line of code)

        return self.propagate(edge_index, node_feature_src=node_feature_src,
                    node_feature_dst=node_feature_dst, size=size, res_n_id=res_n_id)
        ##########################################

    def message_and_aggregate(self, edge_index, node_feature_src):

        ############# Your code here #############
        ## (~1 line of code)
        ## Note:
        ## 1. Different from what we implemented in Colab 3, we use message_and_aggregate
        ## to replace the message and aggregate. The benefit is that we can avoid
        ## materializing x_i and x_j, and make the implementation more efficient.
        ## 2. To implement efficiently, following PyG documentation is helpful:
        ## https://pytorch-geometric.readthedocs.io/en/latest/notes/sparse_tensor.html
        ## 3. Here edge_index is torch_sparse SparseTensor.

        out = matmul(edge_index, node_feature_src, reduce='mean')
        ##########################################

        return out

    def update(self, aggr_out, node_feature_dst, res_n_id):

        ############# Your code here #############
        ## (~4 lines of code)
        dst_out = self.lin_dst(node_feature_dst)
        aggr_out = self.lin_src(aggr_out)
        # print(aggr_out.shape, dst_out.shape)
        aggr_out = torch.cat([dst_out, aggr_out], -1)
        # print(aggr_out.shape, )
        aggr_out = self.lin_update(aggr_out)
        ##########################################

        return aggr_out
```

## Heterogeneous GNN Wrapper Layer

After implementing the GNN layer for each message type, we need to somehow aggregate the the node embedding results (with respect to each message types) together. Here we will implement two types of message type level aggregation.

The first one is simply the mean aggregation:

\begin{equation}
h_v^{(l)} = \frac{1}{M}\sum_{m=1}^{M}h_v^{(l)[m]}
\end{equation}

Here node $v$ has the node type $d$ and $M$ is the total number of message types that the destination node type is $d$.

The other one is the semantic level attention introduced in **HAN** ([Wang et al. (2019)](https://arxiv.org/abs/1903.07293)). Instead of directly averaging on the message type aggregation results, we use attention to learn which message type result can be more important, then aggregate from all the message types. Following are the equations for semantic level attention:

\begin{equation}
e_{m} = \frac{1}{|V_{d}|} \sum_{v \in V_{d}} q_{attn}^T \cdot tanh \Big( W_{attn}^{(l)} \cdot h_v^{(l)[m]} + b \Big)
\end{equation}

where $m$ refers to message type and $d$ refers to the destination node type. Then we can compute the attention and update the $h_v^{(l)}$:

\begin{equation}
\alpha_{m} = \frac{\exp(e_{m})}{\sum_{m=1}^M \exp(e_{m})}
\end{equation}

\begin{equation}
h_v^{(l)} = \sum_{m=1}^{M} \alpha_{m} \cdot h_v^{(l)[m]}
\end{equation}

```python
class HeteroGNNWrapperConv(deepsnap.hetero_gnn.HeteroConv):
    def __init__(self, convs, args, aggr="mean"):
        super(HeteroGNNWrapperConv, self).__init__(convs, None)
        self.aggr = aggr

        # Map the index and message type
        self.mapping = {}

        # A numpy array that stores the final attention probability
        self.alpha = None

        self.attn_proj = None

        if self.aggr == "attn":
            ############# Your code here #############
            ## (~1 line of code)
            ## Note:
            ## 1. Initialize self.attn_proj here.
            ## 2. You should use nn.Sequential for self.attn_proj
            ## 3. nn.Linear and nn.Tanh are useful.
            ## 4. You can create a vector parameter by using:
            ## nn.Linear(some_size, 1, bias=False)
            ## 5. The first linear layer should have out_features as args['attn_size']
            ## 6. You can assume we only have one "head" for the attention.
            ## 7. We recommend you to implement the mean aggregation first. After
            ## the mean aggregation works well in the training, then you can
            ## implement this part.

            self.attn_proj = nn.Sequential(
                nn.Linear(args['hidden_size'], args['attn_size']),
                nn.Tanh(),
                nn.Linear(args['attn_size'], 1, bias=False)
            )
            #########################################

    def reset_parameters(self):
        super(HeteroConvWrapper, self).reset_parameters()
        if self.aggr == "attn":
            for layer in self.attn_proj.children():
                layer.reset_parameters()

    def forward(self, node_features, edge_indices):
        message_type_emb = {}
        for message_key, message_type in edge_indices.items():
            src_type, edge_type, dst_type = message_key
            node_feature_src = node_features[src_type]
            node_feature_dst = node_features[dst_type]
            edge_index = edge_indices[message_key]
            message_type_emb[message_key] = (
                self.convs[message_key](
                    node_feature_src,
                    node_feature_dst,
                    edge_index,
                )
            )
        node_emb = {dst: [] for _, _, dst in message_type_emb.keys()}
        mapping = {}
        for (src, edge_type, dst), item in message_type_emb.items():
            mapping[len(node_emb[dst])] = (src, edge_type, dst)
            node_emb[dst].append(item)
        self.mapping = mapping
        for node_type, embs in node_emb.items():
            if len(embs) == 1:
                node_emb[node_type] = embs[0]
            else:
                node_emb[node_type] = self.aggregate(embs)
        return node_emb

    def aggregate(self, xs):
        # TODO: Implement this function that aggregates all message type results.
        # Here, xs is a list of tensors (embeddings) with respect to message
        # type aggregation results.

        if self.aggr == "mean":

            ############# Your code here #############
            ## (~2 lines of code)
            xs = torch.stack(xs)
            out = torch.mean(xs, dim=0)
            return out
            ##########################################

        elif self.aggr == "attn":

            ############# Your code here #############
            ## (~10 lines of code)
            ## Note:
            ## 1. Store the value of attention alpha (as a numpy array) to self.alpha,
            ## which has the shape (len(xs), ) self.alpha will be not be used
            ## to backpropagate etc. in the model. We will use it to see how much
            ## attention the layer pays on different message types.
            ## 2. torch.softmax and torch.cat are useful.
            ## 3. You might need to reshape the tensors by using the
            ## `view()` function https://pytorch.org/docs/stable/tensor_view.html
            xs = torch.stack(xs, dim=0)
            s = self.attn_proj(xs).squeeze(-1)
            s = torch.mean(s, dim=-1)
            self.alpha = torch.softmax(s, dim=0).detach()
            out = self.alpha.reshape(-1, 1, 1) * xs
            out = torch.sum(out, dim=0)
            return out
            ##########################################
```

## Initialize Heterogeneous GNN Layers

Now let's initialize the Heterogeneous GNN Layers. Different from homogeneous graph case, heterogeneous case can be a little bit complex.

In general, we need to create a dictionary of `HeteroGNNConv` layers where the keys are message types.

* To get all message types, `deepsnap.hetero_graph.HeteroGraph.message_types` is useful.
* If we are initializing the first conv layers, we need to get the feature dimension of each node type. Using `deepsnap.hetero_graph.HeteroGraph.num_node_features(node_type)` will return the node feature dimension of `node_type`. In this function, we will set each `HeteroGNNConv` `out_channels` to be `hidden_size`.
* If we are not initializing the first conv layers, all node types will have the same embedding dimension `hidden_size` and we still set `HeteroGNNConv` `out_channels` to be `hidden_size` for simplicity.

```python
def generate_convs(hetero_graph, conv, hidden_size, first_layer=False):
    # TODO: Implement this function that returns a dictionary of `HeteroGNNConv`
    # layers where the keys are message types. `hetero_graph` is deepsnap `HeteroGraph`
    # object and the `conv` is the `HeteroGNNConv`.

    convs = {}

    ############# Your code here #############
    ## (~9 lines of code)

    all_messages_types = hetero_graph.message_types
    for message_type in all_messages_types:
        if first_layer:
            in_channels_src = hetero_graph.num_node_features(message_type[0])
            in_channels_dst = hetero_graph.num_node_features(message_type[2])
        else:
            in_channels_src = hidden_size
            in_channels_dst = hidden_size
        out_channels = hidden_size
        convs[message_type] = conv(in_channels_src, in_channels_dst, out_channels)
    ##########################################

    return convs
```

## HeteroGNN

Now we will make a simple HeteroGNN model which contains only two `HeteroGNNWrapperConv` layers.

For the forward function in `HeteroGNN`, the model is going to be run as following:

$\text{self.convs1} \rightarrow \text{self.bns1} \rightarrow \text{self.relus1} \rightarrow \text{self.convs2} \rightarrow \text{self.bns2} \rightarrow \text{self.relus2} \rightarrow \text{self.post_mps}$

```python
class HeteroGNN(torch.nn.Module):
    def __init__(self, hetero_graph, args, aggr="mean"):
        super(HeteroGNN, self).__init__()

        self.aggr = aggr
        self.hidden_size = args['hidden_size']

        self.convs1 = None
        self.convs2 = None

        self.bns1 = nn.ModuleDict()
        self.bns2 = nn.ModuleDict()
        self.relus1 = nn.ModuleDict()
        self.relus2 = nn.ModuleDict()
        self.post_mps = nn.ModuleDict()

        ############# Your code here #############
        ## (~10 lines of code)
        ## Note:
        ## 1. For self.convs1 and self.convs2, call generate_convs at first and then
        ## pass the returned dictionary of `HeteroGNNConv` to `HeteroGNNWrapperConv`.
        ## 2. For self.bns, self.relus and self.post_mps, the keys are node_types.
        ## `deepsnap.hetero_graph.HeteroGraph.node_types` will be helpful.
        ## 3. Initialize all batchnorms to torch.nn.BatchNorm1d(hidden_size, eps=1.0).
        ## 4. Initialize all relus to nn.LeakyReLU().
        ## 5. For self.post_mps, each value in the ModuleDict is a linear layer
        ## where the `out_features` is the number of classes for that node type.
        ## `deepsnap.hetero_graph.HeteroGraph.num_node_labels(node_type)` will be
        ## useful.

        self.convs1 = HeteroGNNWrapperConv(
            generate_convs(hetero_graph, HeteroGNNConv, self.hidden_size, first_layer=True),
            args, self.aggr)
        self.convs2 = HeteroGNNWrapperConv(
            generate_convs(hetero_graph, HeteroGNNConv, self.hidden_size, first_layer=False),
            args, self.aggr)

        all_node_types = hetero_graph.node_types
        for node_type in all_node_types:
            self.bns1[node_type] = nn.BatchNorm1d(self.hidden_size, eps=1.0)
            self.bns2[node_type] = nn.BatchNorm1d(self.hidden_size, eps=1.0)
            self.relus1[node_type] = nn.LeakyReLU()
            self.relus2[node_type] = nn.LeakyReLU()
            self.post_mps[node_type] = nn.Linear(self.hidden_size, hetero_graph.num_node_labels(node_type))

        ##########################################

    def forward(self, node_feature, edge_index):
        # TODO: Implement the forward function. Notice that `node_feature` is
        # a dictionary of tensors where keys are node types and values are
        # corresponding feature tensors. The `edge_index` is a dictionary of
        # tensors where keys are message types and values are corresponding
        # edge index tensors (with respect to each message type).

        x = node_feature

        ############# Your code here #############
        ## (~7 lines of code)
        ## Note:
        ## 1. `deepsnap.hetero_gnn.forward_op` can be helpful.
        x = self.convs1(x, edge_index)
        x = forward_op(x, self.bns1)
        x = forward_op(x, self.relus1)
        x = self.convs2(x, edge_index)
        x = forward_op(x, self.bns2)
        x = forward_op(x, self.relus2)
        x = forward_op(x, self.post_mps)
        ##########################################

        return x

    def loss(self, preds, y, indices):

        loss = 0
        loss_func = F.cross_entropy

        ############# Your code here #############
        ## (~3 lines of code)
        ## Note:
        ## 1. For each node type in preds, accumulate computed loss to `loss`
        ## 2. Loss need to be computed with respect to the given index

        for node_type in preds:
            idx = indices[node_type]
            loss += loss_func(preds[node_type][idx], y[node_type][idx])

        ##########################################

        return loss
```

## Training and Testing

Here we provide you with the functions to train and test. You only need to implement one line of code here.

**Please do not modify other parts in `train` and `test` for grading purposes.**

```python
def train(model, optimizer, hetero_graph, train_idx):
    model.train()
    optimizer.zero_grad()
    preds = model(hetero_graph.node_feature, hetero_graph.edge_index)

    loss = None

    ############# Your code here #############
    ## Note:
    ## 1. `deepsnap.hetero_graph.HeteroGraph.node_label` is useful
    ## 2. Compute the loss here

    loss = model.loss(preds, hetero_graph.node_label, train_idx)
    ##########################################

    loss.backward()
    optimizer.step()
    return loss.item()

def test(model, graph, indices, best_model=None, best_val=0):
    model.eval()
    accs = []
    for index in indices:
        preds = model(graph.node_feature, graph.edge_index)
        num_node_types = 0
        micro = 0
        macro = 0
        for node_type in preds:
            idx = index[node_type]
            pred = preds[node_type][idx]
            pred = pred.max(1)[1]
            label_np = graph.node_label[node_type][idx].cpu().numpy()
            pred_np = pred.cpu().numpy()
            micro = f1_score(label_np, pred_np, average='micro')
            macro = f1_score(label_np, pred_np, average='macro')
            num_node_types += 1
        # Averaging f1 score might not make sense, but in our example we only
        # have one node type
        micro /= num_node_types
        macro /= num_node_types
        accs.append((micro, macro))
    if accs[1][0] > best_val:
        best_val = accs[1][0]
        best_model = copy.deepcopy(model)
    return accs, best_model, best_val
```

```python
# Please do not change the following parameters
args = {
    'device': torch.device('cuda' if torch.cuda.is_available() else 'cpu'),
    'hidden_size': 64,
    'epochs': 100,
    'weight_decay': 1e-5,
    'lr': 0.003,
    'attn_size': 32,
}
```

## Dataset and Preprocessing

In the next, we will load the data and create a tensor backend (without a NetworkX graph) `deepsnap.hetero_graph.HeteroGraph` object.

We will use the `ACM(3025)` dataset in our node property prediction task, which is proposed in **HAN** ([Wang et al. (2019)](https://arxiv.org/abs/1903.07293)) and our dataset is extracted from [DGL](https://www.dgl.ai/)'s [ACM.mat](https://data.dgl.ai/dataset/ACM.mat).

The original ACM dataset has three node types and two edge (relation) types. For simplicity, we simplify the heterogeneous graph to one node type and two edge types (shown below). This means that in our heterogeneous graph, we have one node type (paper) and two message types (paper, author, paper) and (paper, subject, paper).

<br/>
<center>

</center>

```python
print("Device: {}".format(args['device']))

# Load the data
data = torch.load("acm.pkl")

# Message types
message_type_1 = ("paper", "author", "paper")
message_type_2 = ("paper", "subject", "paper")

# Dictionary of edge indices
edge_index = {}
edge_index[message_type_1] = data['pap']
edge_index[message_type_2] = data['psp']

# Dictionary of node features
node_feature = {}
node_feature["paper"] = data['feature']

# Dictionary of node labels
node_label = {}
node_label["paper"] = data['label']

# Load the train, validation and test indices
train_idx = {"paper": data['train_idx'].to(args['device'])}
val_idx = {"paper": data['val_idx'].to(args['device'])}
test_idx = {"paper": data['test_idx'].to(args['device'])}

# Construct a deepsnap tensor backend HeteroGraph
hetero_graph = HeteroGraph(
    node_feature=node_feature,
    node_label=node_label,
    edge_index=edge_index,
    directed=True
)

print(f"ACM heterogeneous graph: {hetero_graph.num_nodes()} nodes, {hetero_graph.num_edges()} edges")

# Node feature and node label to device
for key in hetero_graph.node_feature:
    hetero_graph.node_feature[key] = hetero_graph.node_feature[key].to(args['device'])
for key in hetero_graph.node_label:
    hetero_graph.node_label[key] = hetero_graph.node_label[key].to(args['device'])

# Edge_index to sparse tensor and to device
for key in hetero_graph.edge_index:
    edge_index = hetero_graph.edge_index[key]
    adj = SparseTensor(row=edge_index[0], col=edge_index[1], sparse_sizes=(hetero_graph.num_nodes('paper'), hetero_graph.num_nodes('paper')))
    hetero_graph.edge_index[key] = adj.t().to(args['device'])
print(hetero_graph.edge_index[message_type_1])
print(hetero_graph.edge_index[message_type_2])
```

    Device: cuda
    ACM heterogeneous graph: {'paper': 3025} nodes, {('paper', 'author', 'paper'): 26256, ('paper', 'subject', 'paper'): 2207736} edges
    SparseTensor(row=tensor([   0,    0,    0,  ..., 3024, 3024, 3024], device='cuda:0'),
                 col=tensor([   8,   20,   51,  ..., 2948, 2983, 2991], device='cuda:0'),
                 size=(3025, 3025), nnz=26256, density=0.29%)
    SparseTensor(row=tensor([   0,    0,    0,  ..., 3024, 3024, 3024], device='cuda:0'),
                 col=tensor([  75,  434,  534,  ..., 3020, 3021, 3022], device='cuda:0'),
                 size=(3025, 3025), nnz=2207736, density=24.13%)

## Start Training!

Now lets start training!

## Training the Mean Aggregation

```python
best_model = None
best_val = 0

model = HeteroGNN(hetero_graph, args, aggr="mean").to(args['device'])
optimizer = torch.optim.Adam(model.parameters(), lr=args['lr'], weight_decay=args['weight_decay'])

for epoch in range(args['epochs']):
    loss = train(model, optimizer, hetero_graph, train_idx)
    accs, best_model, best_val = test(model, hetero_graph, [train_idx, val_idx, test_idx], best_model, best_val)
    print(
        f"Epoch {epoch + 1}: loss {round(loss, 5)}, "
        f"train micro {round(accs[0][0] * 100, 2)}%, train macro {round(accs[0][1] * 100, 2)}%, "
        f"valid micro {round(accs[1][0] * 100, 2)}%, valid macro {round(accs[1][1] * 100, 2)}%, "
        f"test micro {round(accs[2][0] * 100, 2)}%, test macro {round(accs[2][1] * 100, 2)}%"
    )
best_accs, _, _ = test(best_model, hetero_graph, [train_idx, val_idx, test_idx])
print(
    f"Best model: "
    f"train micro {round(best_accs[0][0] * 100, 2)}%, train macro {round(best_accs[0][1] * 100, 2)}%, "
    f"valid micro {round(best_accs[1][0] * 100, 2)}%, valid macro {round(best_accs[1][1] * 100, 2)}%, "
    f"test micro {round(best_accs[2][0] * 100, 2)}%, test macro {round(best_accs[2][1] * 100, 2)}%"
)
```

    Epoch 1: loss 1.10081, train micro 33.33%, train macro 16.67%, valid micro 33.33%, valid macro 16.67%, test micro 35.81%, test macro 17.58%
    Epoch 2: loss 1.09208, train micro 61.83%, train macro 51.93%, valid micro 56.0%, valid macro 47.06%, test micro 50.92%, test macro 41.48%
    Epoch 3: loss 1.05948, train micro 66.33%, train macro 54.64%, valid micro 66.0%, valid macro 54.58%, test micro 64.75%, test macro 53.44%
    Epoch 4: loss 0.99427, train micro 65.5%, train macro 53.11%, valid micro 65.33%, valid macro 52.81%, test micro 63.44%, test macro 51.44%
    Epoch 5: loss 0.88551, train micro 64.83%, train macro 52.92%, valid micro 64.67%, valid macro 51.99%, test micro 62.16%, test macro 50.08%
    Epoch 6: loss 0.73597, train micro 66.17%, train macro 55.98%, valid micro 65.0%, valid macro 52.77%, test micro 62.21%, test macro 50.22%
    Epoch 7: loss 0.57266, train micro 70.17%, train macro 63.18%, valid micro 66.0%, valid macro 54.65%, test micro 63.06%, test macro 51.2%
    Epoch 8: loss 0.42861, train micro 74.17%, train macro 69.31%, valid micro 67.33%, valid macro 57.29%, test micro 64.14%, test macro 52.72%
    Epoch 9: loss 0.31822, train micro 77.0%, train macro 73.32%, valid micro 70.0%, valid macro 62.4%, test micro 65.13%, test macro 54.4%
    Epoch 10: loss 0.23886, train micro 83.5%, train macro 82.03%, valid micro 76.0%, valid macro 72.38%, test micro 65.98%, test macro 56.0%
    Epoch 11: loss 0.18137, train micro 89.67%, train macro 89.21%, valid micro 80.67%, valid macro 78.63%, test micro 67.91%, test macro 59.48%
    Epoch 12: loss 0.13878, train micro 93.0%, train macro 92.81%, valid micro 86.0%, valid macro 85.25%, test micro 70.64%, test macro 64.39%
    Epoch 13: loss 0.10747, train micro 96.33%, train macro 96.3%, valid micro 89.67%, valid macro 89.34%, test micro 72.99%, test macro 68.36%
    Epoch 14: loss 0.08507, train micro 97.67%, train macro 97.66%, valid micro 91.33%, valid macro 91.12%, test micro 75.39%, test macro 72.01%
    Epoch 15: loss 0.06874, train micro 98.5%, train macro 98.5%, valid micro 92.33%, valid macro 92.15%, test micro 77.93%, test macro 75.49%
    Epoch 16: loss 0.05643, train micro 99.0%, train macro 99.0%, valid micro 92.67%, valid macro 92.5%, test micro 79.62%, test macro 77.77%
    Epoch 17: loss 0.04665, train micro 99.33%, train macro 99.33%, valid micro 94.67%, valid macro 94.58%, test micro 81.36%, test macro 80.03%
    Epoch 18: loss 0.03874, train micro 99.5%, train macro 99.5%, valid micro 96.0%, valid macro 95.95%, test micro 82.87%, test macro 81.88%
    Epoch 19: loss 0.03226, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.98%, test micro 83.53%, test macro 82.7%
    Epoch 20: loss 0.02687, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.32%, test micro 84.24%, test macro 83.54%
    Epoch 21: loss 0.02246, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.66%, test micro 84.89%, test macro 84.3%
    Epoch 22: loss 0.01885, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.66%, test micro 85.18%, test macro 84.65%
    Epoch 23: loss 0.0159, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.32%, test micro 85.22%, test macro 84.75%
    Epoch 24: loss 0.01348, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.66%, test micro 85.13%, test macro 84.66%
    Epoch 25: loss 0.01148, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.66%, test micro 85.27%, test macro 84.83%
    Epoch 26: loss 0.00974, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.66%, test micro 85.55%, test macro 85.13%
    Epoch 27: loss 0.00807, train micro 100.0%, train macro 100.0%, valid micro 98.0%, valid macro 97.99%, test micro 85.84%, test macro 85.43%
    Epoch 28: loss 0.00665, train micro 100.0%, train macro 100.0%, valid micro 98.0%, valid macro 97.99%, test micro 85.93%, test macro 85.54%
    Epoch 29: loss 0.00552, train micro 100.0%, train macro 100.0%, valid micro 97.67%, valid macro 97.67%, test micro 86.07%, test macro 85.69%
    Epoch 30: loss 0.00462, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.33%, test micro 86.07%, test macro 85.7%
    Epoch 31: loss 0.0039, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.33%, test micro 86.16%, test macro 85.81%
    Epoch 32: loss 0.00333, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.33%, test micro 86.02%, test macro 85.68%
    Epoch 33: loss 0.00288, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 86.02%, test macro 85.7%
    Epoch 34: loss 0.00251, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 86.02%, test macro 85.72%
    Epoch 35: loss 0.00221, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 86.02%, test macro 85.72%
    Epoch 36: loss 0.00196, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 85.79%, test macro 85.49%
    Epoch 37: loss 0.00176, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 85.69%, test macro 85.39%
    Epoch 38: loss 0.00159, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 85.79%, test macro 85.49%
    Epoch 39: loss 0.00145, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.66%, test micro 85.98%, test macro 85.7%
    Epoch 40: loss 0.00132, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.66%, test micro 86.16%, test macro 85.91%
    Epoch 41: loss 0.00122, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.66%, test micro 86.12%, test macro 85.87%
    Epoch 42: loss 0.00113, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 86.12%, test macro 85.88%
    Epoch 43: loss 0.00105, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.84%, test macro 85.62%
    Epoch 44: loss 0.00098, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.79%, test macro 85.6%
    Epoch 45: loss 0.00092, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.84%, test macro 85.65%
    Epoch 46: loss 0.00086, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.88%, test macro 85.7%
    Epoch 47: loss 0.00081, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.74%, test macro 85.57%
    Epoch 48: loss 0.00077, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.79%, test macro 85.64%
    Epoch 49: loss 0.00073, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.74%, test macro 85.6%
    Epoch 50: loss 0.0007, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.74%, test macro 85.61%
    Epoch 51: loss 0.00067, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.65%, test macro 85.52%
    Epoch 52: loss 0.00064, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.55%, test macro 85.44%
    Epoch 53: loss 0.00061, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.41%, test macro 85.3%
    Epoch 54: loss 0.00059, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 96.99%, test micro 85.46%, test macro 85.36%
    Epoch 55: loss 0.00057, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.33%, test micro 85.51%, test macro 85.41%
    Epoch 56: loss 0.00055, train micro 100.0%, train macro 100.0%, valid micro 97.33%, valid macro 97.33%, test micro 85.6%, test macro 85.51%
    Epoch 57: loss 0.00053, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 85.55%, test macro 85.47%
    Epoch 58: loss 0.00052, train micro 100.0%, train macro 100.0%, valid micro 97.0%, valid macro 97.0%, test micro 85.46%, test macro 85.38%
    Epoch 59: loss 0.0005, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.27%, test macro 85.19%
    Epoch 60: loss 0.00049, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.18%, test macro 85.1%
    Epoch 61: loss 0.00047, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.18%, test macro 85.1%
    Epoch 62: loss 0.00046, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.18%, test macro 85.1%
    Epoch 63: loss 0.00045, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.22%, test macro 85.16%
    Epoch 64: loss 0.00044, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.27%, test macro 85.21%
    Epoch 65: loss 0.00043, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 85.18%, test macro 85.13%
    Epoch 66: loss 0.00042, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.27%, test macro 85.23%
    Epoch 67: loss 0.00041, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.27%, test macro 85.23%
    Epoch 68: loss 0.00041, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.27%, test macro 85.23%
    Epoch 69: loss 0.0004, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.27%, test macro 85.23%
    Epoch 70: loss 0.00039, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.33%, test micro 85.18%, test macro 85.14%
    Epoch 71: loss 0.00038, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.66%, test micro 85.04%, test macro 85.0%
    Epoch 72: loss 0.00038, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.66%, test micro 85.04%, test macro 85.0%
    Epoch 73: loss 0.00037, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 84.89%, test macro 84.86%
    Epoch 74: loss 0.00037, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 84.85%, test macro 84.82%
    Epoch 75: loss 0.00036, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 84.8%, test macro 84.78%
    Epoch 76: loss 0.00036, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 84.8%, test macro 84.78%
    Epoch 77: loss 0.00035, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 84.8%, test macro 84.78%
    Epoch 78: loss 0.00035, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.77%
    Epoch 79: loss 0.00034, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.77%
    Epoch 80: loss 0.00034, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.77%
    Epoch 81: loss 0.00033, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.77%
    Epoch 82: loss 0.00033, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.78%
    Epoch 83: loss 0.00032, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.78%
    Epoch 84: loss 0.00032, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.78%
    Epoch 85: loss 0.00032, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.78%
    Epoch 86: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.85%, test macro 84.84%
    Epoch 87: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.85%, test macro 84.84%
    Epoch 88: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.85%, test macro 84.84%
    Epoch 89: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.79%
    Epoch 90: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.75%, test macro 84.75%
    Epoch 91: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.75%, test macro 84.75%
    Epoch 92: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.8%
    Epoch 93: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.85%, test macro 84.85%
    Epoch 94: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.8%
    Epoch 95: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.75%, test macro 84.76%
    Epoch 96: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.75%, test macro 84.76%
    Epoch 97: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.8%, test macro 84.81%
    Epoch 98: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.75%, test macro 84.76%
    Epoch 99: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.71%, test macro 84.71%
    Epoch 100: loss 0.00027, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 84.71%, test macro 84.71%
    Best model: train micro 100.0%, train macro 100.0%, valid micro 98.0%, valid macro 97.99%, test micro 85.84%, test macro 85.43%

## Question 2.1: What is the maximum **micro** F1 score you could get for the best_model on test set when using the mean aggregation? (10 points)

## Question 2.2: What is the maximum **macro** F1 score you could get for the best_model on test set when using the mean aggregation? (10 points)

## Training the Attention Aggregation

```python
best_model = None
best_val = 0

output_size = hetero_graph.num_node_labels('paper')
model = HeteroGNN(hetero_graph, args, aggr="attn").to(args['device'])
optimizer = torch.optim.Adam(model.parameters(), lr=args['lr'], weight_decay=args['weight_decay'])

for epoch in range(args['epochs']):
    loss = train(model, optimizer, hetero_graph, train_idx)
    accs, best_model, best_val = test(model, hetero_graph, [train_idx, val_idx, test_idx], best_model, best_val)
    print(
        f"Epoch {epoch + 1}: loss {round(loss, 5)}, "
        f"train micro {round(accs[0][0] * 100, 2)}%, train macro {round(accs[0][1] * 100, 2)}%, "
        f"valid micro {round(accs[1][0] * 100, 2)}%, valid macro {round(accs[1][1] * 100, 2)}%, "
        f"test micro {round(accs[2][0] * 100, 2)}%, test macro {round(accs[2][1] * 100, 2)}%"
    )
best_accs, _, _ = test(best_model, hetero_graph, [train_idx, val_idx, test_idx])
print(
    f"Best model: "
    f"train micro {round(best_accs[0][0] * 100, 2)}%, train macro {round(best_accs[0][1] * 100, 2)}%, "
    f"valid micro {round(best_accs[1][0] * 100, 2)}%, valid macro {round(best_accs[1][1] * 100, 2)}%, "
    f"test micro {round(best_accs[2][0] * 100, 2)}%, test macro {round(best_accs[2][1] * 100, 2)}%"
)
```

    Epoch 1: loss 1.1006, train micro 33.33%, train macro 16.67%, valid micro 33.33%, valid macro 16.67%, test micro 35.81%, test macro 17.58%
    Epoch 2: loss 1.09207, train micro 33.33%, train macro 16.67%, valid micro 33.33%, valid macro 16.67%, test micro 35.81%, test macro 17.58%
    Epoch 3: loss 1.06348, train micro 60.67%, train macro 51.0%, valid micro 55.0%, valid macro 46.16%, test micro 49.6%, test macro 39.96%
    Epoch 4: loss 1.00747, train micro 66.33%, train macro 55.09%, valid micro 66.0%, valid macro 55.07%, test micro 62.31%, test macro 52.08%
    Epoch 5: loss 0.91224, train micro 66.67%, train macro 55.45%, valid micro 66.0%, valid macro 54.82%, test micro 65.04%, test macro 53.97%
    Epoch 6: loss 0.77287, train micro 67.67%, train macro 57.37%, valid micro 66.33%, valid macro 55.06%, test micro 65.18%, test macro 54.02%
    Epoch 7: loss 0.60174, train micro 68.67%, train macro 59.45%, valid micro 66.33%, valid macro 55.06%, test micro 65.41%, test macro 54.25%
    Epoch 8: loss 0.43043, train micro 70.5%, train macro 62.96%, valid micro 67.0%, valid macro 56.52%, test micro 65.41%, test macro 54.31%
    Epoch 9: loss 0.28991, train micro 73.5%, train macro 68.24%, valid micro 67.67%, valid macro 58.37%, test micro 65.46%, test macro 54.47%
    Epoch 10: loss 0.19147, train micro 76.17%, train macro 72.43%, valid micro 68.67%, valid macro 60.45%, test micro 65.88%, test macro 55.39%
    Epoch 11: loss 0.12776, train micro 82.0%, train macro 80.42%, valid micro 74.67%, valid macro 70.84%, test micro 66.4%, test macro 56.69%
    Epoch 12: loss 0.08677, train micro 89.17%, train macro 88.77%, valid micro 80.67%, valid macro 79.15%, test micro 68.05%, test macro 60.0%
    Epoch 13: loss 0.06036, train micro 93.67%, train macro 93.58%, valid micro 85.0%, valid macro 84.36%, test micro 70.54%, test macro 64.64%
    Epoch 14: loss 0.04358, train micro 97.0%, train macro 96.98%, valid micro 89.33%, valid macro 89.15%, test micro 72.89%, test macro 68.53%
    Epoch 15: loss 0.03289, train micro 97.83%, train macro 97.83%, valid micro 90.67%, valid macro 90.52%, test micro 75.44%, test macro 72.5%
    Epoch 16: loss 0.02582, train micro 98.5%, train macro 98.5%, valid micro 91.67%, valid macro 91.6%, test micro 77.41%, test macro 75.31%
    Epoch 17: loss 0.02097, train micro 99.5%, train macro 99.5%, valid micro 92.33%, valid macro 92.29%, test micro 78.82%, test macro 77.2%
    Epoch 18: loss 0.01755, train micro 99.5%, train macro 99.5%, valid micro 94.33%, valid macro 94.33%, test micro 80.24%, test macro 78.99%
    Epoch 19: loss 0.01505, train micro 99.83%, train macro 99.83%, valid micro 95.67%, valid macro 95.67%, test micro 81.36%, test macro 80.38%
    Epoch 20: loss 0.01312, train micro 99.83%, train macro 99.83%, valid micro 96.33%, valid macro 96.34%, test micro 82.4%, test macro 81.63%
    Epoch 21: loss 0.0115, train micro 99.83%, train macro 99.83%, valid micro 96.33%, valid macro 96.34%, test micro 82.96%, test macro 82.3%
    Epoch 22: loss 0.00997, train micro 99.83%, train macro 99.83%, valid micro 96.33%, valid macro 96.34%, test micro 83.29%, test macro 82.71%
    Epoch 23: loss 0.0086, train micro 99.83%, train macro 99.83%, valid micro 96.67%, valid macro 96.68%, test micro 83.44%, test macro 82.9%
    Epoch 24: loss 0.00749, train micro 99.83%, train macro 99.83%, valid micro 96.33%, valid macro 96.34%, test micro 83.67%, test macro 83.16%
    Epoch 25: loss 0.00658, train micro 99.83%, train macro 99.83%, valid micro 96.33%, valid macro 96.34%, test micro 83.72%, test macro 83.23%
    Epoch 26: loss 0.00583, train micro 99.83%, train macro 99.83%, valid micro 96.0%, valid macro 96.01%, test micro 83.86%, test macro 83.39%
    Epoch 27: loss 0.00519, train micro 99.83%, train macro 99.83%, valid micro 96.0%, valid macro 96.01%, test micro 83.91%, test macro 83.48%
    Epoch 28: loss 0.0046, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.95%, test macro 83.51%
    Epoch 29: loss 0.00399, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.91%, test macro 83.48%
    Epoch 30: loss 0.00338, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.81%, test macro 83.39%
    Epoch 31: loss 0.00286, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.67%, test macro 83.27%
    Epoch 32: loss 0.00242, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.62%, test macro 83.22%
    Epoch 33: loss 0.00206, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.58%, test macro 83.18%
    Epoch 34: loss 0.00177, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.48%, test macro 83.08%
    Epoch 35: loss 0.00153, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.48%, test macro 83.09%
    Epoch 36: loss 0.00134, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.01%, test micro 83.53%, test macro 83.13%
    Epoch 37: loss 0.00119, train micro 100.0%, train macro 100.0%, valid micro 95.67%, valid macro 95.67%, test micro 83.58%, test macro 83.18%
    Epoch 38: loss 0.00107, train micro 100.0%, train macro 100.0%, valid micro 95.33%, valid macro 95.34%, test micro 83.53%, test macro 83.14%
    Epoch 39: loss 0.00098, train micro 100.0%, train macro 100.0%, valid micro 95.33%, valid macro 95.34%, test micro 83.53%, test macro 83.14%
    Epoch 40: loss 0.0009, train micro 100.0%, train macro 100.0%, valid micro 95.33%, valid macro 95.34%, test micro 83.25%, test macro 82.88%
    Epoch 41: loss 0.00083, train micro 100.0%, train macro 100.0%, valid micro 95.33%, valid macro 95.34%, test micro 83.39%, test macro 83.04%
    Epoch 42: loss 0.00078, train micro 100.0%, train macro 100.0%, valid micro 95.67%, valid macro 95.67%, test micro 83.53%, test macro 83.19%
    Epoch 43: loss 0.00073, train micro 100.0%, train macro 100.0%, valid micro 95.67%, valid macro 95.67%, test micro 83.39%, test macro 83.05%
    Epoch 44: loss 0.00069, train micro 100.0%, train macro 100.0%, valid micro 96.0%, valid macro 96.0%, test micro 83.39%, test macro 83.05%
    Epoch 45: loss 0.00066, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.29%, test macro 82.96%
    Epoch 46: loss 0.00063, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.44%, test macro 83.13%
    Epoch 47: loss 0.0006, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.39%, test macro 83.08%
    Epoch 48: loss 0.00058, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.34%, test macro 83.03%
    Epoch 49: loss 0.00056, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.25%, test macro 82.95%
    Epoch 50: loss 0.00054, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.44%, test macro 83.16%
    Epoch 51: loss 0.00052, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.39%, test macro 83.12%
    Epoch 52: loss 0.0005, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.44%, test macro 83.18%
    Epoch 53: loss 0.00049, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.53%, test macro 83.27%
    Epoch 54: loss 0.00048, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.39%, test macro 83.13%
    Epoch 55: loss 0.00046, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.44%, test macro 83.18%
    Epoch 56: loss 0.00045, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.44%, test macro 83.19%
    Epoch 57: loss 0.00044, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.58%, test macro 83.34%
    Epoch 58: loss 0.00043, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.58%, test macro 83.35%
    Epoch 59: loss 0.00042, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.62%, test macro 83.41%
    Epoch 60: loss 0.00041, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.58%, test macro 83.37%
    Epoch 61: loss 0.0004, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.48%, test macro 83.29%
    Epoch 62: loss 0.00039, train micro 100.0%, train macro 100.0%, valid micro 96.33%, valid macro 96.34%, test micro 83.34%, test macro 83.16%
    Epoch 63: loss 0.00039, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.39%, test macro 83.21%
    Epoch 64: loss 0.00038, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.44%, test macro 83.25%
    Epoch 65: loss 0.00037, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.29%, test macro 83.12%
    Epoch 66: loss 0.00036, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.2%, test macro 83.03%
    Epoch 67: loss 0.00036, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.2%, test macro 83.03%
    Epoch 68: loss 0.00035, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.11%, test macro 82.93%
    Epoch 69: loss 0.00035, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.11%, test macro 82.93%
    Epoch 70: loss 0.00034, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.06%, test macro 82.89%
    Epoch 71: loss 0.00034, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.01%, test macro 82.84%
    Epoch 72: loss 0.00033, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 83.01%, test macro 82.84%
    Epoch 73: loss 0.00033, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.96%, test macro 82.81%
    Epoch 74: loss 0.00032, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.72%
    Epoch 75: loss 0.00032, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.68%
    Epoch 76: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.78%, test macro 82.63%
    Epoch 77: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.68%
    Epoch 78: loss 0.00031, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.73%
    Epoch 79: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.73%
    Epoch 80: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.68%
    Epoch 81: loss 0.0003, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.73%
    Epoch 82: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.73%
    Epoch 83: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.73%
    Epoch 84: loss 0.00029, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.78%, test macro 82.64%
    Epoch 85: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.69%
    Epoch 86: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.74%
    Epoch 87: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.92%, test macro 82.8%
    Epoch 88: loss 0.00028, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.92%, test macro 82.8%
    Epoch 89: loss 0.00027, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.7%
    Epoch 90: loss 0.00027, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.82%, test macro 82.7%
    Epoch 91: loss 0.00027, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.75%
    Epoch 92: loss 0.00027, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.75%
    Epoch 93: loss 0.00026, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 94: loss 0.00026, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 95: loss 0.00026, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 96: loss 0.00026, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 97: loss 0.00026, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 98: loss 0.00025, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.92%, test macro 82.81%
    Epoch 99: loss 0.00025, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Epoch 100: loss 0.00025, train micro 100.0%, train macro 100.0%, valid micro 96.67%, valid macro 96.67%, test micro 82.87%, test macro 82.76%
    Best model: train micro 99.83%, train macro 99.83%, valid micro 96.67%, valid macro 96.68%, test micro 83.44%, test macro 82.9%

## Question 2.3: What is the maximum **micro** F1 score you could get for the best_model on test set when using the attention aggregation? (4 points)

Submit your answers on Gradescope.

## Question 2.4: What is the maximum **macro** F1 score you could get for the best_model on test set when using the attention aggregation? (4 points)

Submit your answers on Gradescope.

## Attention for each Message Type

Through message type level attention we can learn that which message type is more important to which layer.

Here we will print out and show that each layer pay how much attention on each message type.

```python
if model.convs1.alpha is not None and model.convs2.alpha is not None:
    for idx, message_type in model.convs1.mapping.items():
        print(f"Layer 1 has attention {model.convs1.alpha[idx]} on message type {message_type}")
    for idx, message_type in model.convs2.mapping.items():
        print(f"Layer 2 has attention {model.convs2.alpha[idx]} on message type {message_type}")
```

    Layer 1 has attention 0.47106021642684937 on message type ('paper', 'author', 'paper')
    Layer 1 has attention 0.5289397835731506 on message type ('paper', 'subject', 'paper')
    Layer 2 has attention 0.5076631903648376 on message type ('paper', 'author', 'paper')
    Layer 2 has attention 0.49233683943748474 on message type ('paper', 'subject', 'paper')

# Submission

In order to get credit, you must go submit your answers on Gradescope.

Also, you need to submit the `ipynb` file of Colab 4, by clicking `File` and `Download .ipynb`. Please make sure that your output of each cell is available in your `ipynb` file.


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Toggle cell visibility here.

## Diagonalne matrike: Divergentne modalne oblike

Ta interaktivni primer vključuje prednastavljeno diagonalno matriko z vsaj eno divergetno modalno obliko. Primer omogoča, da prosto spreminjate elemente te matrike in ob tem spremljate ali ima nova matrika divergentene modalne oblike (in je s tem nestabilna) ali ne.

Razišči naslednje možnosti:
* diagonalna matrika z divergentnimi (nestabilnimi) modalnimi oblikami,
* diagonalna matrika s stabilnimi modalnimi oblikami,
* diagonalna matrika s stabilnimi in divergetnimi (nestabilnimi) modalnimi oblikami.

Ali lahko enostavno ugotoviš če ima matrika divergentno modalno obliko, kadar je izražena v diagonalni obliki?

[//]: # "This example shows a diagonal matrix with at least one divergent mode. It is possible to change any element of the matrix and analyze if the new matrix has divergent modes (unstable matrix) or not.

Explore the various possibilities:
* diagonal matrices with divergent (unstable) modes,
* diagonal matrices with stable modes,
* diagonal matrices with both stable and divergent (unstable) modes.

Can you easily tell if a matrix has divergent modes when it is in diagonal form?"

```python
%matplotlib notebook
import control
import numpy
import sympy
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    #def dummychangecallback(self,change):
        #pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(control.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        control.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass
```

```python
A=matrixWidget(4,4)
A.setM(numpy.matrix('1,0,0,0;0,-2,0,0;0,0,-3,0;0,0,0,-4'))

def main_callback(matA,DW):
    (r,c) = numpy.shape(matA)
    sol = numpy.linalg.eig(matA)[0]
    print('Lastne vrednosti matrike so: %s' %str(sol))

    matAs = sympy.Matrix(matA)
    eig = matAs.eigenvals()
    eigvals = list(eig.keys())
    Amul = list(eig.values())

    diag = True
    for i in range(r):
        for j in range(c):
            if i != j:
                if matA[i,j] != 0:
                    diag = False

    if diag:
        for i in range(len(eigvals)):
            if numpy.real(eigvals[i]) > 0:
                print('Ok! Matrika je nestabilna.')
                return
            elif numpy.real(eigvals[i]) == 0:
                if Amul[i] > 1:
                    if len((matAs-eigvals[i]*sympy.eye(4)).nullspace()) < Amul[i]:
                        print('Ok! Matrika je nestabilna.')
                        return
        print('Matrika ni nestabilna.')
    else:
        for i in range(len(eigvals)):
            if numpy.real(eigvals[i]) > 0:
                print('Matrika je nestabilna, a ni diagonalna.')
                return
            elif numpy.real(eigvals[i]) == 0:
                if Amul[i] > 1:
                    if len((matAs-eigvals[i]*sympy.eye(4)).nullspace()) < Amul[i]:
                        print('Matrika je nestabilna, a ni diagonalna.')
                        return
        print('Matrika ni nestabilna in ni diagonalna.')

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

out = widgets.interactive_output(main_callback,{'matA':A,'DW':DW})
display(A,START,out)
```

    matrixWidget(children=(HBox(children=(FloatText(value=1.0, layout=Layout(width='90px')), FloatText(value=0.0, …

    Button(description='Test', icon='check', style=ButtonStyle(), tooltip='Test')

    Output()

```python

```